An exception occurred while running the pipeline, java.lang.NullPointerException: record cannot be null

asked 2020-06-09 08:29:58 -0500

Grimur gravatar image

I'm running Oracle CDC->field renamer->JDBC Producer

The Oracle CDC runs on online catalog with specific tables. It is started from scn number I'm trying to figure out what cases this error as their is limited information on this error.

The field renamer takes care of naming fields from oracle to mysql

JDBC Producer is connected to a mysql database.

Here is the full log

java.lang.NullPointerException: record cannot be null at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:226) at com.streamsets.datacollector.runner.StageContext.toError(StageContext.java:418) at com.streamsets.pipeline.stage.common.DefaultErrorRecordHandler.onError(DefaultErrorRecordHandler.java:117) at com.streamsets.pipeline.lib.jdbc.JdbcUtil.write(JdbcUtil.java:1159) at com.streamsets.pipeline.lib.jdbc.JdbcUtil.write(JdbcUtil.java:1078) at com.streamsets.pipeline.stage.destination.jdbc.JdbcTarget.write(JdbcTarget.java:269) at com.streamsets.pipeline.stage.destination.jdbc.JdbcTarget.write(JdbcTarget.java:258) at com.streamsets.pipeline.api.base.configurablestage.DTarget.write(DTarget.java:34) at com.streamsets.datacollector.runner.StageRuntime.lambda$execute$2(StageRuntime.java:303) at com.streamsets.datacollector.runner.StageRuntime.execute(StageRuntime.java:244) at com.streamsets.datacollector.runner.StageRuntime.execute(StageRuntime.java:311) at com.streamsets.datacollector.runner.StagePipe.process(StagePipe.java:220) at com.streamsets.datacollector.execution.runner.common.ProductionPipelineRunner.processPipe(ProductionPipelineRunner.java:854) at com.streamsets.datacollector.execution.runner.common.ProductionPipelineRunner.lambda$executeRunner$3(ProductionPipelineRunner.java:898) at com.streamsets.datacollector.runner.PipeRunner.acceptConsumer(PipeRunner.java:221) at com.streamsets.datacollector.runner.PipeRunner.executeBatch(PipeRunner.java:142) at com.streamsets.datacollector.execution.runner.common.ProductionPipelineRunner.executeRunner(ProductionPipelineRunner.java:897) at com.streamsets.datacollector.execution.runner.common.ProductionPipelineRunner.runSourceLessBatch(ProductionPipelineRunner.java:875) at com.streamsets.datacollector.execution.runner.common.ProductionPipelineRunner.runPollSource(ProductionPipelineRunner.java:599) at com.streamsets.datacollector.execution.runner.common.ProductionPipelineRunner.run(ProductionPipelineRunner.java:390) at com.streamsets.datacollector.runner.Pipeline.run(Pipeline.java:516) at com.streamsets.datacollector.execution.runner.common.ProductionPipeline.run(ProductionPipeline.java:112) at com.streamsets.datacollector.execution.runner.common.ProductionPipelineRunnable.run(ProductionPipelineRunnable.java:75) at com.streamsets.datacollector.execution.runner.standalone.StandaloneRunner.start(StandaloneRunner.java:718) at com.streamsets.datacollector.execution.runner.common.AsyncRunner.lambda$start$3(AsyncRunner.java:151) at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.lambda$call$0(SafeScheduledExecutorService.java:226) at com.streamsets.datacollector.security.GroupsInScope.execute(GroupsInScope.java:33) at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.call(SafeScheduledExecutorService.java:222) at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.lambda$call$0(SafeScheduledExecutorService.java:226) at com.streamsets.datacollector.security.GroupsInScope.execute(GroupsInScope.java:33) at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.call(SafeScheduledExecutorService.java:222) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at com.streamsets.datacollector.metrics.MetricSafeScheduledExecutorService$MetricsTask.run(MetricSafeScheduledExecutorService.java:100) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748 ...

(more)
edit retag flag offensive close merge delete

Comments

1

I had a similar problem in two cases: one when my destination did not have a primary key and one when I was trying to hard code one of the destination columns with a value not coming from the source. I was using 3.16 Oracle CDC Origin and a JDBC PostgreSQL Destination.

KateF gravatar imageKateF ( 2020-06-12 14:22:58 -0500 )edit

Hi KateF, I added primary key to destination table even though source doesn't have one. I ran the replicator for a day and the error showed up again. I am not sure why I have the full log enabled, but it doesn't tell me what record is null. Do you know how one can debug this ?

Grimur gravatar imageGrimur ( 2020-06-15 14:15:30 -0500 )edit
KateF gravatar imageKateF ( 2020-06-18 14:26:25 -0500 )edit