Ask Your Question

Running two instances of the same pipeline with different parameters

asked 2017-12-07 05:32:18 -0500

anonymous user


updated 2017-12-08 11:02:59 -0500

metadaddy gravatar image

Is there any way i can pass two different parameter values for the same pipeline which run two instance of the same pipeline id.

Scenario :

i have one pipeline which fetch records from one table and push the record to Kafka topic. so I use parameters for SQL and DB connection. i have to run the same pipeline with different SQL and DB connection. just wondering is there any way i can do this.

edit retag flag offensive close merge delete

2 Answers

Sort by ยป oldest newest most voted

answered 2018-08-28 11:28:53 -0500

iamontheinet gravatar image

updated 2018-08-28 11:29:35 -0500

Hi Dean, DPM is (StreamSets) Dataflow Performance Manager. You can read more about it here --

edit flag offensive delete link more

answered 2017-12-08 11:01:57 -0500

metadaddy gravatar image

This is currently not possible in SDC, but with DPM you can have the same pipeline running in multiple jobs with different parameters.

edit flag offensive delete link more


hi, what is DPM?

Dean Han gravatar imageDean Han ( 2018-08-28 05:49:44 -0500 )edit
Login/Signup to Answer

Question Tools


Asked: 2017-12-07 05:32:18 -0500

Seen: 278 times

Last updated: Aug 28 '18