Mysql JDBC Multitable with multiple schemas

asked 2020-10-13 05:10:43 -0600

Firkragg gravatar image

Im running streamsets data collector 1.18 in order to migrate data from a mysql database to redshift. Everything looks good but im having a problem with the JDBC multitable connector

My database configuration is a mysql database with multi databases with the exact same table structure. Now looking at the docs, i should just be able to specify % for the schema and table in the tables tab of the JDBC connector and it should pick up all schemas (which map to databaes in mysql) and all tables which will let me copy the entire server.

Whats actually happening is its just picking up the tables in the database that im forced to have as part of my JDBC string. This means that i would need to manually hard code the database name and run the pipeline for every database on my server which isnt feasible.

I found a Jira ticket that seems to suggest this should be something that its possible to do in streamsets but im not sure if im just approaching this wrong https://jira.streamsets.com/browse/SD...

Is there some sort of special config i need to do to have the JDBC multitable source pick up every database and all tables on my server?

edit retag flag offensive close merge delete