Ask Your Question
2

How to kill a pipeline forcefully when its is in starting stage ??

asked 2018-01-19 11:19:49 -0600

Roh gravatar image

updated 2018-01-22 12:22:38 -0600

metadaddy gravatar image

I accidentally started a pipeline, which will take ages to start. it is in starting mode, how can I stop it forcefully ???

edit retag flag offensive close merge delete

Comments

What specific stages are involved in this pipeline?

jeff gravatar imagejeff ( 2018-01-19 11:22:51 -0600 )edit

JDBC Consumer and destination trash

Roh gravatar imageRoh ( 2018-01-19 11:57:59 -0600 )edit

What is the query? Is it possible that the query takes a long time to execute, ex: if you test it in an independent SQL client? It should get logged in sdc.log.

jeff gravatar imagejeff ( 2018-01-19 12:00:20 -0600 )edit

Yes, the problem is with the query it will take a lot of time, that's why I wanted to know if there is any way to kill it when it is in the starting stage.

Roh gravatar imageRoh ( 2018-01-19 12:09:28 -0600 )edit

@metadaddy Any idea on how I can kill the pipeline?

Roh gravatar imageRoh ( 2018-01-19 12:26:03 -0600 )edit

2 Answers

Sort by ยป oldest newest most voted
2

answered 2018-01-19 12:46:37 -0600

jeff gravatar image

Unfortunately, I don't believe there is any way currently to interrupt the thread when the pipeline is in STARTING state. For now, you will need to kill the SDC process. Please open a Jira and we can investigate the feasibility of handling this kind of scenario better.

edit flag offensive delete link more

Comments

Sure @jeff

Roh gravatar imageRoh ( 2018-01-22 10:58:41 -0600 )edit
0

answered 2018-01-25 10:34:55 -0600

For a pipeline which is in DISCONNECTED state can be changed to STOPPED by editing pipelineState.json file under "${SDC}/data/runInfo/<pipeline_id>/0/"

I think you can do the same thing for this scenario too. But you need to restart the SDC. Next time you start SDC, it should not start the pipeline automatically.

This is just a work around.

edit flag offensive delete link more
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2018-01-19 11:19:49 -0600

Seen: 238 times

Last updated: Jan 22