Ask Your Question

Revision history [back]

click to hide/show revision 1
initial version

How to store status in ETL pipeline?

I can't find good example of ETL pipeline.

There are two databases: one with statuses, second data storage with data. ETL steps: 1. Get entries from statuses table with status (state/flag) 'UNPROCESSED' (JDBC Query Consumer) 2. Execute some queries against data storage database to reprocess data (JDBC Query) 3. Mark entry as 'FINISHED''

There is no anything like 'JDBC Query processor'. How should I update entry status as soon as some query had finished processing data in the data storage database?

Sincerely, Sergey.