Ask Your Question

Revision history [back]

click to hide/show revision 1
initial version

Stream set is for continuous streaming the records from source. I believe you want to use it as a batch mode. In your JDBCQueryConsumer origin if you do not have delta processing logic or requirement then please remove any value or column defined as Offset column. Enable the Produce events check box from general tab. And second event output link from JDBCQueryConsumer you can connect to Pipeline finisher executer stage. And in General tab Pre-condition of Pipeline finisher executer stage define this expression : ${record:eventType() == 'no-more-data'}. What it does is after executing your query output (all source records) it gracefully deactivates the pipeline when it encounters no more data from source.

Stream set is for continuous streaming the records from source. I believe you want to use it as a batch mode. In your JDBCQueryConsumer origin if you do not have delta processing logic or requirement then please remove any value or column defined as Offset column. Enable the Produce events check box from general tab. And second event output link from JDBCQueryConsumer you can connect to Pipeline finisher executer stage. And in General tab Pre-condition of Pipeline finisher executer stage define this expression : ${record:eventType() == 'no-more-data'}. 'no-more-data'}. What it does is after executing your query output (all source records) it gracefully deactivates the pipeline when it encounters no more data from source.