Merge two large table in StreamSets Control Hub

asked 2018-04-09 10:24:54 -0600

Black gravatar image

I work for an data solution company that help client to built first party data management platform. We need a flexible and powerful etl tool to help us processing big data (terabytes level).

The biggest challenge is that in some scenario, during the etl process, we need to merge two very large table(more than 1 TB) into one. Is that possible for us to process these kind of data in StreamSets Control Hub ? Is is still use hadoop resources to do the calculation ?

edit retag flag offensive close merge delete