Ask Your Question

Pipeline Finisher not working with JDBC Query Consumer

asked 2017-11-03 09:25:44 -0500

this post is marked as community wiki

This post is a wiki. Anyone with karma >75 is welcome to improve it.

I have a simple pipeline which uses a JDBC query consumer with Offset value. I have a pipeline finisher with a precondition of ${record:eventType() == 'no-more-data'}. If the jdbc query consumer picks up source data then the pipeline finisher works properly and ends the pipeline after the data is processed. However, if the pipeline is started and the jdbc query consumer does not pick up any new data, the pipeline never finishes and continues to send 1 record to the pipeline finisher at each query interval. I changed the 'On Record Error' option to 'Send to Error' within the pipeline finisher to see the error. Now when I run the pipeline and there is no new data being picked up from jdbc query conusmer, it generates the following error in the pipeline finisher stage:

COMMON_0001 - Stage precondition: CONTAINER_0051 - Unsatisfied precondition(s) '${record:eventType() == 'no-more-data'}'

Any ideas?

edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted

answered 2017-11-04 09:55:12 -0500

bob gravatar image

There was an outstanding issue with the Jdbc Query Origin when running in incremental mode -- SDC-7603. This issue causes the problem you're seeing.

As you identified, the current behavior requires at least one record to be processed, then the "no-more-data" event will be trigged when no more records are available. The new behavior will be to trigger "no-more-data" event even if no records were processed by the query.

In non-incremental mode, this was not a problem.

SDC-7603 is resolved and is available in the code line, and will be back ported to the latest builds of

edit flag offensive delete link more


Thanks Bob. Do you know when the fix will be back ported to

jshivers gravatar imagejshivers ( 2017-11-06 08:42:35 -0500 )edit

hi jshivers. The back port is available now in Please be aware that nightly builds are not appropriate for production; you should run this only in a development environment. thanks.

bob gravatar imagebob ( 2017-11-07 15:01:30 -0500 )edit
Login/Signup to Answer

Question Tools

1 follower


Asked: 2017-11-03 09:25:44 -0500

Seen: 805 times

Last updated: Nov 04 '17