Ask Your Question
0

Can i use no-more-data event for pipeline finisher & other query activity?

asked 2020-02-05 07:34:34 -0600

Sujata HS gravatar image

updated 2020-02-05 08:23:56 -0600

My main pipeline loads data in database. there is another activity to be done i.e data validation after data load.

I have used no-more-data event for two task 1. to finish the pipeline 2. to query the data base for some validation

Will this ensure both task are guaranteed to be run ? My concern is what if pipeline finisher finishes the pipeline execution before data validation query ?

edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted
1

answered 2020-02-05 11:54:00 -0600

metadaddy gravatar image

The best way to do this would be to enable Produce Events on the query executor, and attach the pipeline finisher there. That way, the pipeline will always finish after the query completes.

edit flag offensive delete link more

Comments

Thank you for your solution, I am planning to use Query executor but again two things i have to use. 1. Email executor 2. Pipeline finisher this will end again using same event for two different task. please let me know you suggestions

Sujata HS gravatar imageSujata HS ( 2020-02-06 00:05:41 -0600 )edit

It looks like the email executor does not generate events, so if you connect both the email executor and pipeline finisher to the query executor's event, the pipeline may be stopped before the email is sent. Try it and see!

metadaddy gravatar imagemetadaddy ( 2020-02-06 12:42:39 -0600 )edit
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2020-02-05 07:34:34 -0600

Seen: 29 times

Last updated: Feb 05