Ask Your Question
0

SDC RPC to RPC is throwing a mysterious exception after upgrading

asked 2018-01-08 15:38:11 -0500

Roh gravatar image

updated 2018-01-08 15:41:46 -0500

My pipeline with SDC RPC origin and receiving point as SDC RPC between two clusters is failing after upgrading to 3.0 in the origin side stream sets.

Here is the snapshot of the error I'm getting :

image description

Clear error stack :

com.streamsets.pipeline.api.StageException: IPC_DEST_20 - Could not transmit: com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException: Unrecognized field "errorStageLabel" (class com.streamsets.datacollector.restapi.bean.HeaderJson), not marked as ignorable (15 known properties: "values", "errorStackTrace", "raw", "previousTrackingId", "stageCreator", "errorTimestamp", "trackingId", "errorPipelineName", "errorCode", "sourceId", "errorDataCollectorId", "errorStage" [truncated]])? at [Source: com.streamsets.pipeline.lib.io.OverrunReader@78ca4b89; line: 1, column: 2382] (through reference chain: com.streamsets.datacollector.restapi.bean.HeaderJson["errorStageLabel"])
    at com.streamsets.pipeline.stage.destination.sdcipc.SdcIpcTarget.write(SdcIpcTarget.java:182)
    at com.streamsets.pipeline.configurablestage.DTarget.write(DTarget.java:34)
    at com.streamsets.datacollector.runner.StageRuntime$2.call(StageRuntime.java:253)
    at com.streamsets.datacollector.runner.StageRuntime$2.call(StageRuntime.java:237)
    at com.streamsets.datacollector.runner.StageRuntime.execute(StageRuntime.java:195)
    at com.streamsets.datacollector.runner.StageRuntime.execute(StageRuntime.java:264)
    at com.streamsets.datacollector.runner.StagePipe.process(StagePipe.java:226)
    at com.streamsets.datacollector.execution.runner.common.ProductionPipelineRunner.processPipe(ProductionPipelineRunner.java:751)
    at com.streamsets.datacollector.execution.runner.common.ProductionPipelineRunner.lambda$runSourceLessBatch$3(ProductionPipelineRunner.java:779)
    at com.streamsets.datacollector.runner.PipeRunner.executeBatch(PipeRunner.java:132)
    at com.streamsets.datacollector.execution.runner.common.ProductionPipelineRunner.runSourceLessBatch(ProductionPipelineRunner.java:778)
    at com.streamsets.datacollector.execution.runner.common.ProductionPipelineRunner.runPollSource(ProductionPipelineRunner.java:550)
    at com.streamsets.datacollector.execution.runner.common.ProductionPipelineRunner.run(ProductionPipelineRunner.java:375)
    at com.streamsets.datacollector.runner.Pipeline.run(Pipeline.java:504)
    at com.streamsets.datacollector.execution.runner.common.ProductionPipeline.run(ProductionPipeline.java:112)
    at com.streamsets.datacollector.execution.runner.common.ProductionPipelineRunnable.run(ProductionPipelineRunnable.java:74)
    at com.streamsets.datacollector.execution.runner.standalone.StandaloneRunner.start(StandaloneRunner.java:754)
    at com.streamsets.datacollector.execution.runner.common.AsyncRunner.lambda$start$3(AsyncRunner.java:152)
    at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.lambda$call$0(SafeScheduledExecutorService.java:227)
    at com.streamsets.datacollector.security.GroupsInScope.execute(GroupsInScope.java:33)
    at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.call(SafeScheduledExecutorService.java:223)
    at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.lambda$call$0(SafeScheduledExecutorService.java:227)
    at com.streamsets.datacollector.security.GroupsInScope.execute(GroupsInScope.java:33)
    at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.call(SafeScheduledExecutorService.java:223)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
    at com.streamsets.datacollector.metrics.MetricSafeScheduledExecutorService$MetricsTask.run(MetricSafeScheduledExecutorService.java:100)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)

SDC RPC ORIGIN CLUSTER VERSION :

    Version:
StreamSets Data Collector 3.0.0.0 ( built by root on 2017-11-28T00:53Z git: 8f1866fd487b319f78e176c30101d088aef61db9)

Copyright © 2016 StreamSets, Inc. All rights reserved.

SDC RPC AT RECEIVING CLUSTER VERSION :

Version: StreamSets Data Collector 2.2.1.0 ( built by jenkins ...
(more)
edit retag flag offensive close merge delete

Comments

@metadaddy@jeff Tagging you guys to get your attention

Roh gravatar imageRoh ( 2018-01-08 15:43:06 -0500 )edit

1 Answer

Sort by » oldest newest most voted
1

answered 2018-01-08 21:12:58 -0500

jeff gravatar image

It seems that the errorStageLabel was added in between versions 2.2 and 3.0 and the serialization format for the SDC record changed. You should try upgrading the receiver SDC instance before the transmitting side to avoid this.

edit flag offensive delete link more
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2018-01-08 15:38:11 -0500

Seen: 73 times

Last updated: Jan 08