Pipelines got stuck in starting state

asked 2020-03-10 02:45:26 -0500

anonymous user

Anonymous

Hi,

I'm using streamsets to populate the data from DB to ElasticSearch. Here, the pipelines status is showing as starting but not yet started. Even after restarting the service, still we're facing the same issue. And not able to delete the pipeline while the pipeline status is in starting.

PFB the error for the pipeline not starting.

2020-03-04 12:46:38,845 apm_elasticsearch/apmelasticsearch1912ccf7-fb07-4330-953a-6ed3981966b6 ERROR Can't start and get snapshot for pipeline apmelasticsearch1912ccf7-fb07-4330-953a-6ed3981966b6: com.streamsets.datacollector.util.PipelineException: CONTAINER_0606 - Snapshot default for pipeline apmelasticsearch1912ccf7-fb07-4330-953a-6ed3981966b6 already exists StandaloneRunner sirisha.poosarla runner-pool-2-thread-5 com.streamsets.datacollector.util.PipelineException: CONTAINER_0606 - Snapshot default for pipeline apmelasticsearch1912ccf7-fb07-4330-953a-6ed3981966b6 already exists at com.streamsets.datacollector.execution.snapshot.file.FileSnapshotStore.create(FileSnapshotStore.java:69) at com.streamsets.datacollector.execution.snapshot.cache.CacheSnapshotStore.create(CacheSnapshotStore.java:61) at com.streamsets.datacollector.execution.runner.standalone.StandaloneRunner.captureSnapshot(StandaloneRunner.java:512) at com.streamsets.datacollector.execution.runner.standalone.StandaloneRunner.startAndCaptureSnapshot(StandaloneRunner.java:930) at com.streamsets.datacollector.execution.runner.standalone.StandaloneRunner.retryOrStart(StandaloneRunner.java:374) at com.streamsets.datacollector.execution.runner.standalone.StandaloneRunner.onDataCollectorStart(StandaloneRunner.java:354) at com.streamsets.datacollector.execution.runner.common.AsyncRunner.lambda$onDataCollectorStart$0(AsyncRunner.java:110) at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.lambda$call$0(SafeScheduledExecutorService.java:226) at com.streamsets.datacollector.security.GroupsInScope.execute(GroupsInScope.java:33) at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.call(SafeScheduledExecutorService.java:222) at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.lambda$call$0(SafeScheduledExecutorService.java:226) at com.streamsets.datacollector.security.GroupsInScope.execute(GroupsInScope.java:33) at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.call(SafeScheduledExecutorService.java:222) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at com.streamsets.datacollector.metrics.MetricSafeScheduledExecutorService$MetricsTask.run(MetricSafeScheduledExecutorService.java:100) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)

PFB the error for the pipeline we're not able to delete while it is in starting state.

2020-03-10 06:33:49,598 scom_elasticsearch/scomelasticsearch4d3f2830-d6f3-4efe-8e7b-f017abedfa92 ERROR REST API call error: com.streamsets.datacollector.store.PipelineStoreException: CONTAINER_0208 - Pipeline in state 'STARTING' cannot be saved ExceptionToHttpErrorProvider sirisha.poosarla webserver-1013 com.streamsets.datacollector.store.PipelineStoreException: CONTAINER_0208 - Pipeline in state 'STARTING' cannot be saved at com.streamsets.datacollector.store.impl.FilePipelineStoreTask.delete(FilePipelineStoreTask.java:275) at com.streamsets.datacollector.store.impl.CachePipelineStoreTask.delete(CachePipelineStoreTask.java:114) at com.streamsets.datacollector.restapi.PipelineStoreResource.deletePipeline(PipelineStoreResource.java:959) at sun.reflect.GeneratedMethodAccessor345.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:144) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:161) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:160) at ... (more)

edit retag flag offensive close merge delete