Ask Your Question

How do I recover existing pipelines?

asked 2018-01-23 23:25:49 -0600

this post is marked as community wiki

This post is a wiki. Anyone with karma >75 is welcome to improve it.

Our existing StreamSets instance was not accessible so we restarted it by firing "sudo streamsets dc", however when the instance was up it is not showing existing pipelines (comes up as if it is launched for the first time). Has anyone faced such issue before?

edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted

answered 2018-01-24 11:18:26 -0600

metadaddy gravatar image

Look in the $SDC_DATA directory - by default, this is the data directory wherever you installed StreamSets Data Collector, but it might have been set in or to be somewhere else. See the documentation on configuring the SDC environment for more details. If the files are not there, then look for files named pipeline.json on your disk.

There's no reason SDC would delete pipelines at restart, so my guess is that the environment is somehow different from when you started it before and when you're starting it now. For example, if you previously started SDC as a service and you're not starting it manually, it could be reading the $SDC_DATA value from a different file - rather than

edit flag offensive delete link more
Login/Signup to Answer

Question Tools

1 follower


Asked: 2018-01-23 23:25:49 -0600

Seen: 116 times

Last updated: Jan 24