Ask Your Question
1

How to add a JDBC query consumer after another query consumer

asked 2019-12-11 01:08:42 -0500

Shashank Shukla gravatar image

updated 2019-12-11 10:24:00 -0500

metadaddy gravatar image

I am trying to add a query consumer after another query consumer.

  1. I am getting values from one database and storing it in a file using local FS.
  2. I want to get values from another database and store them into another file.
  3. Now I want to end the pipeline.

I have made a pipeline, which reads from one DB and writes to a file, and when JDBC consumer sends a no-more-data event the pipeline stops. I want to insert the step 2 in between the step 1 and 2, such that, after step 1 is completed, step 2 will start, and when step 2 is completed then the pipeline will finish.

edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted
1

answered 2019-12-11 13:46:53 -0500

iamontheinet gravatar image

Hi!

Here's the approach that will work better.

  • Pipeline 1
    • JDBC consumer reads from DB and writes to file
    • Pipeline Finisher stops the pipeline when it receives no-more-data event from JDBC consumer
    • Pipeline Stop Event is configured to execute Shell script that issues curl command to start the second pipeline

image description


image description


Cheers, Dash

edit flag offensive delete link more

Comments

1

Yea, couldn't think of a way that would help in this case though.

iamontheinet gravatar imageiamontheinet ( 2019-12-11 14:43:29 -0500 )edit
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2019-12-11 01:08:42 -0500

Seen: 73 times

Last updated: Dec 11 '19