Ask Your Question
1

Replicate entire database to MapR DB

asked 2019-08-19 10:47:38 -0600

ravi9099 gravatar image

I'm trying to replicate the my sql relational database to Target environment(MapR DB) by using Streamsets pipeline.

Some of the tables will have primary keys and some of them doesn't have primary keys.Pipeline takes a longer time to replicate the data to target environment.How we can improve the performance ? I would appreciate If you provide any suggestions.

edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted
0

answered 2019-08-19 12:35:35 -0600

metadaddy gravatar image

updated 2019-08-19 12:35:44 -0600

You may need to increase the max batch size in sdc.properties- see https://ask.streamsets.com/question/1...

You can also check where the bottleneck is by duplicating the pipeline and replacing the MapR destination with 'Trash'. If it runs quickly, then the problem is on the MapR side.

edit flag offensive delete link more
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2019-08-19 10:47:38 -0600

Seen: 25 times

Last updated: Aug 19