Ask Your Question

Best practices on starting/stopping pipelines

asked 2018-09-24 19:17:51 -0600

Ronald M gravatar image

updated 2018-09-25 02:11:48 -0600

Shruthi gravatar image

I'm fairly new to StreamSets Data Collector. I would like to know what are the best practices with running pipelines, specifically the need to stop pipelines and re-starting them later on.

Are there any negatives if pipelines are left running all the time? Or is it better to have some kind of scheduler that could stop pipelines (once they are done) and restart them at a specific time?

edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted

answered 2018-09-24 19:20:51 -0600

metadaddy gravatar image

Data Collector is designed to run pipelines continuously, so, in general, you can leave pipelines running indefinitely. Many of our users run their pipelines for weeks or months, stopping them only to upgrade Data Collector.

edit flag offensive delete link more
Login/Signup to Answer

Question Tools

1 follower


Asked: 2018-09-24 19:17:51 -0600

Seen: 783 times

Last updated: Sep 24 '18