Ingest very large database tables over JDBC
I've been playing with StreamSets and love it! Let's say I have 10 billion row table in Oracle DB which is not partitioned and has a single column primary key. Is there a way to load that table using multiple threads? I use sqoop today with 32 mappers which takes about 5 hours. Once initial load is done, I can load data incrementally using only one thread.
Is it possible to use StreamSets for that initial load or sqoop still the best option? I have a few dozens very large tables like that and using single threaded jdbc origin is not really an option..