Ask Your Question

Revision history [back]

click to hide/show revision 1
initial version

"PipelineState.json doesn't exist" issue on v3.9.0

I have been running a Streamsets docker container for a few weeks without issue, and just encountered the following behavior this morning:

Upon logging into my localhost:[port] Streamsets location, I see 0 pipelines, and the following message:

com.streamsets.datacollector.store.PipelineStoreException: CONTAINER_0209 - Pipeline state file '/data/runInfo/WebsiteScraperStatsMain9860b9e3-4212-4b3d-9990-3303733fb29b/0/pipelineState.json' doesn't exist

Luckily I have a backup .json exported of the pipeline in question, so I attempted to import this into a new pipeline, which worked successfully. However, upon navigating back to the main Pipelines list, I see that my newly created pipeline has disappeared, and I see the above error message again. It appears that I am unable to create and save any new pipelines for the time being.

Looking into the Docker container's file system, I do see pipelineState.json and pipelineStateHistory.json files for each of the pipelines I originally had, however none of these pipelines show in Streamsets.

My questions are:

  • What was the root cause of this issue, and what can I do to reduce the chance of this occurring again?
  • What are the correct steps to take to eliminate the "pipelineState.json doesn't exist" error message from returning?
  • Is there any way of recovering the other pipelines that were not backed-up, or a way that I can recover the .json of the pipeline to import?