Call JDBC Query Executor

asked 2020-08-27 12:23:09 -0600

srinath_222 gravatar image

updated 2020-08-27 12:44:55 -0600


I have a requirement to execute few post update/delete statements in my pipeline after data is loaded into Target.

My pipeline source is JDBC Query Consumer and target is Apache Kudu Database. I produced events on JDBC Query Consumer and connected to JDBC Query Executor (included post update/delete statements in this). With JDBC Query Consumer, after the no more data event is generated...this stage of JDBC Query Executor should be executed.

Current behavior: JDBC Query Executor statements are executing before completing the data loading into the target.

Expected behavior: JDBC Query Executor statements should start executing after completing data loading into Target Destination. I tried to produce events with Kudu DB Destination but it doesn't have it.

Can you please suggest if I can use JDBC Query Executor to perform post update/delete scripts or some other way. Below is the screenshot of pipeline:

image description



edit retag flag offensive close merge delete


@iamontheinet...please suggest ?

srinath_222 gravatar imagesrinath_222 ( 2020-08-27 12:24:16 -0600 )edit

Can you update your question with the current behavior (when you run the pipeline with existing setup) and what your desired behavior is? It's unclear right now.

iamontheinet gravatar imageiamontheinet ( 2020-08-27 12:34:21 -0600 )edit

@iamontheinet...updated the question.

srinath_222 gravatar imagesrinath_222 ( 2020-08-27 12:45:17 -0600 )edit

I'd try using Stop Event of the pipeline and then select JDBC Query and enter the details on Stop Event - JDBC Query tab.

iamontheinet gravatar imageiamontheinet ( 2020-08-27 12:56:03 -0600 )edit

My pipeline should be running I specified Query Interval of 1 hour in JDBC Query Consumer. This cycle of data loading to target and executing of post SQL Statements should happen every 1 hour.

srinath_222 gravatar imagesrinath_222 ( 2020-08-27 12:58:36 -0600 )edit