Ask Your Question

Launch Full Pipeline One Time

asked 2019-12-18 10:03:12 -0600

Fabio Baldaconi Roldao gravatar image

Friends, I need to take a doubt. I created a Pipeline to perform FULL loads once a day, but when starting the pipeline it is in an eternal loop, even after the creation of the full file. I put as an example the "PIPELINE FINISH" but when performing the first 30,000 records he finishes. Note: We must take into consideration that it has 1 million lines. What would be the solution when the 1 million line load finishes it stops?

  • Incremental Mode -> NO
  • Max Batch Size (Records) -> 30000
edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted

answered 2019-12-18 13:11:15 -0600

metadaddy gravatar image

updated 2019-12-18 13:11:47 -0600

Ensure that you set a precondition of ${record:eventType()=='no-more-data'} in the Pipeline Finisher, and set On Record Error to 'Discard'.

image description

This is documented fully in Case Study: Stop the Pipeline

edit flag offensive delete link more
Login/Signup to Answer

Question Tools

1 follower


Asked: 2019-12-18 10:03:12 -0600

Seen: 74 times

Last updated: Dec 18 '19