Does StreamSets DC support user defined global parameters on instance level?

asked 2017-11-01 21:58:04 -0500

casel.chen gravatar image

As far as I know StreamSets DC support user defined pipeline level parameters, but how can I define those parameters used cross pipelines? Saying I need setup multiple data pipelines to sync data from multiple MySQL tables to Hive/HBase, each data pipeline handle one table data sync, there are some common parameters I want to share across those pipelines. Is there any other ways to handle better on above issue? I thought of use JDBC MultipleTableQuery origin, connect with StreamSelector, finally each StreamSelector connect with Hive/HBase destination, but there are so many tables to fork.

edit retag flag offensive close merge delete