Ask Your Question

How to execute a pipeline once a day?

asked 2019-04-11 13:11:25 -0500

anonymous user


updated 2019-04-11 13:18:49 -0500

metadaddy gravatar image

My origin is an S3 bucket, and I would like to check it once a day and process the data. How can I do that?

edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted

answered 2019-04-11 13:18:13 -0500

metadaddy gravatar image

There are two options. In both cases, you should use a Pipeline Finisher executor to stop the pipeline when processing is complete.

  1. If you using StreamSets Control Hub, you can use its scheduler to start the job.
  2. Otherwise, you can use cron and the Data Collector CLI to start the job.
edit flag offensive delete link more
Login/Signup to Answer

Question Tools

1 follower


Asked: 2019-04-11 13:11:25 -0500

Seen: 505 times

Last updated: Apr 11 '19