Ask Your Question

How to kill a pipeline forcefully when its is in starting stage ??

asked 2018-01-19 11:19:49 -0500

Roh gravatar image

updated 2018-01-22 12:22:38 -0500

metadaddy gravatar image

I accidentally started a pipeline, which will take ages to start. it is in starting mode, how can I stop it forcefully ???

edit retag flag offensive close merge delete


What specific stages are involved in this pipeline?

jeff gravatar imagejeff ( 2018-01-19 11:22:51 -0500 )edit

JDBC Consumer and destination trash

Roh gravatar imageRoh ( 2018-01-19 11:57:59 -0500 )edit

What is the query? Is it possible that the query takes a long time to execute, ex: if you test it in an independent SQL client? It should get logged in sdc.log.

jeff gravatar imagejeff ( 2018-01-19 12:00:20 -0500 )edit

Yes, the problem is with the query it will take a lot of time, that's why I wanted to know if there is any way to kill it when it is in the starting stage.

Roh gravatar imageRoh ( 2018-01-19 12:09:28 -0500 )edit

@metadaddy Any idea on how I can kill the pipeline?

Roh gravatar imageRoh ( 2018-01-19 12:26:03 -0500 )edit

2 Answers

Sort by ยป oldest newest most voted

answered 2018-01-19 12:46:37 -0500

jeff gravatar image

Unfortunately, I don't believe there is any way currently to interrupt the thread when the pipeline is in STARTING state. For now, you will need to kill the SDC process. Please open a Jira and we can investigate the feasibility of handling this kind of scenario better.

edit flag offensive delete link more


Sure @jeff

Roh gravatar imageRoh ( 2018-01-22 10:58:41 -0500 )edit

answered 2018-01-25 10:34:55 -0500

For a pipeline which is in DISCONNECTED state can be changed to STOPPED by editing pipelineState.json file under "${SDC}/data/runInfo/<pipeline_id>/0/"

I think you can do the same thing for this scenario too. But you need to restart the SDC. Next time you start SDC, it should not start the pipeline automatically.

This is just a work around.

edit flag offensive delete link more
Login/Signup to Answer

Question Tools

1 follower


Asked: 2018-01-19 11:19:49 -0500

Seen: 149 times

Last updated: Jan 22