Pipelines in Run_error

asked 2019-06-13 08:10:23 -0500

suhas gravatar image

updated 2019-06-14 10:23:56 -0500

metadaddy gravatar image

CONTAINER_0101 - Cannot get state: java.io.IOException: File '/data-sdc/data/runInfo/ireland-lrx-dcp-staging-gender-distribution-enriched/0/pipelineState.json-new' exists, '/data-sdc/data/runInfo/ireland-lrx-dcp-staging-gender-distribution-enriched/0/pipelineState.json' should not exist

pipelineState.json-new' exists pipelineState.json' should not exist

offset.json-new exists offset.json should not exist

This error we are getting very frequently and how we can fix this issue permanently

Please find full log stack

com.streamsets.datacollector.store.PipelineStoreException: CONTAINER_0101 - Cannot get state: java.io.IOException: File '/data-sdc/data/runInfo/IrelandLRxPrescriberBureauRdsPanelweeklybasedreportd5525287-e08c-4469-8929-5f3adbd12689/0/pipelineState.json-tmp' exists, '/data-sdc/data/runInfo/IrelandLRxPrescriberBureauRdsPanelweeklybasedreportd5525287-e08c-4469-8929-5f3adbd12689/0/pipelineState.json-old' should exists
    at com.streamsets.datacollector.execution.store.FilePipelineStateStore.loadState(FilePipelineStateStore.java:158)
    at com.streamsets.datacollector.execution.store.FilePipelineStateStore.getState(FilePipelineStateStore.java:165)
    at com.streamsets.datacollector.execution.store.CachePipelineStateStore$1.load(CachePipelineStateStore.java:66)
    at com.streamsets.datacollector.execution.store.CachePipelineStateStore$1.load(CachePipelineStateStore.java:63)
    at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3527)
    at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2319)
    at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2282)
    at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2197)
    at com.google.common.cache.LocalCache.get(LocalCache.java:3937)
    at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3941)
    at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4824)
    at com.streamsets.datacollector.execution.store.CachePipelineStateStore.getState(CachePipelineStateStore.java:107)
    at com.streamsets.datacollector.execution.manager.standalone.StandaloneAndClusterPipelineManager.getPipelineState(StandaloneAndClusterPipelineManager.java:193)
    at com.streamsets.datacollector.restapi.PipelineStoreResource.getPipelines(PipelineStoreResource.java:431)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81)
    at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:144)
    at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:161)
    at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:160)
    at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:99)
    at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:389)
    at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:347)
    at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:102)
    at org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:326)
    at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271)
    at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267)
    at org.glassfish.jersey.internal.Errors.process(Errors.java:315)
    at org.glassfish.jersey.internal.Errors.process(Errors.java:297)
    at org.glassfish.jersey.internal.Errors.process(Errors.java:267)
    at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317)
    at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305)
    at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154)
    at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:473)
    at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:427)
    at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:388)
    at org ...
(more)
edit retag flag offensive close merge delete

Comments

Can you edit your question and add the full stack trace from sdc.log?

metadaddy gravatar imagemetadaddy ( 2019-06-13 11:31:51 -0500 )edit

Please find the logs

suhas gravatar imagesuhas ( 2019-06-14 09:57:59 -0500 )edit

com.streamsets.datacollector.store.PipelineStoreException: CONTAINER_0101 - Cannot get state: java.io.IOException: File '/data-sdc/data/runInfo/IrelandLRxPrescriberBureauRdsPanelweeklybasedreportd5525287-e08c-4469-8929-5f3adbd12689/0/pipelineState.json-tmp' exists, '/data-sdc/data/runInfo/IrelandLRx

suhas gravatar imagesuhas ( 2019-06-14 09:58:04 -0500 )edit

Hi @suhas - this isn't any more detail than in your question. Please EDIT THE QUESTION (don't leave a comment) and add the FULL STACK TRACE from sdc.log - the long thing with all the Java class names and line numbers.

metadaddy gravatar imagemetadaddy ( 2019-06-14 10:00:34 -0500 )edit

Hi i have added the full stack logs in question

suhas gravatar imagesuhas ( 2019-06-14 10:19:49 -0500 )edit