Ask Your Question
0

Temp Empty MySQL table with JDBC consumer issue

asked 2017-11-05 22:13:03 -0500

casel.chen gravatar image

updated 2017-11-07 20:08:35 -0500

I found incremental sync tables from MySQL which currently have empty records (but will have records later) has problem, the DC complains "Pipeline Status: START_ERROR:".

2017-11-08 10:07:22,990 [user:admin] [pipeline:mercury_mysql_hivecopy/mercurymysqlhivecopy9ba71407-6840-4b45-86ab-6c884050cf32] [runner:] [thread:preview-pool-1-thread-1] DEBUG StandaloneAndClusterPipelineManager - Status of previewer with id: 'dab5830f-6163-462a-9241-753a971de2ff' changed to status: 'RUNNING' 2017-11-08 10:07:23,009 [user:admin] [pipeline:mercury_mysql_hivecopy/mercurymysqlhivecopy9ba71407-6840-4b45-86ab-6c884050cf32] [runner:] [thread:preview-pool-1-thread-1] DEBUG PipelineConfigurationValidator - Pipeline 'mercurymysqlhivecopy9ba71407-6840-4b45-86ab-6c884050cf32' validation. valid=false, canPreview=true, issuesCount=1 2017-11-08 10:07:23,023 [user:admin] [pipeline:mercury_mysql_hivecopy/mercurymysqlhivecopy9ba71407-6840-4b45-86ab-6c884050cf32] [runner:] [thread:preview-pool-1-thread-1] INFO Pipeline - Processing lifecycle start event with stage 2017-11-08 10:07:23,023 [user:admin] [pipeline:mercury_mysql_hivecopy/mercurymysqlhivecopy9ba71407-6840-4b45-86ab-6c884050cf32] [runner:] [thread:preview-pool-1-thread-1] INFO JdbcUtil - Registered JDBC drivers: 2017-11-08 10:07:23,023 [user:admin] [pipeline:mercury_mysql_hivecopy/mercurymysqlhivecopy9ba71407-6840-4b45-86ab-6c884050cf32] [runner:] [thread:preview-pool-1-thread-1] INFO JdbcUtil - Driver class com.mysql.fabric.jdbc.FabricMySQLDriver (version 5.1) 2017-11-08 10:07:23,023 [user:admin] [pipeline:mercury_mysql_hivecopy/mercurymysqlhivecopy9ba71407-6840-4b45-86ab-6c884050cf32] [runner:] [thread:preview-pool-1-thread-1] INFO JdbcUtil - Driver class com.mysql.jdbc.Driver (version 5.1) 2017-11-08 10:07:23,023 [user:admin] [pipeline:mercury_mysql_hivecopy/mercurymysqlhivecopy9ba71407-6840-4b45-86ab-6c884050cf32] [runner:] [thread:preview-pool-1-thread-1] INFO HikariDataSource - HikariPool-5 - is starting. 2017-11-08 10:07:23,196 [user:admin] [pipeline:mercury_mysql_hivecopy/mercurymysqlhivecopy9ba71407-6840-4b45-86ab-6c884050cf32] [runner:] [thread:preview-pool-1-thread-1] DEBUG ConnectionManager - Closing connection 2017-11-08 10:07:23,209 [user:admin] [pipeline:mercury_mysql_hivecopy/mercurymysqlhivecopy9ba71407-6840-4b45-86ab-6c884050cf32] [runner:] [thread:preview-pool-1-thread-1] WARN Pipeline - Stage 'JDBCMultitableConsumer_01' initialization error: java.lang.NullPointerException java.lang.NullPointerException at com.streamsets.pipeline.lib.jdbc.JdbcUtil.getMinimumOffsetValues(JdbcUtil.java:309) at com.streamsets.pipeline.lib.jdbc.multithread.TableContextUtil.createTableContext(TableContextUtil.java:198) at com.streamsets.pipeline.lib.jdbc.multithread.TableContextUtil.listTablesForConfig(TableContextUtil.java:372) at com.streamsets.pipeline.stage.origin.jdbc.table.TableJdbcSource.checkConnectionAndBootstrap(TableJdbcSource.java:125) at com.streamsets.pipeline.stage.origin.jdbc.table.TableJdbcSource.init(TableJdbcSource.java:259) at com.streamsets.pipeline.api.base.BaseStage.init(BaseStage.java:48) at com.streamsets.pipeline.configurablestage.DStage.init(DStage.java:36) at com.streamsets.datacollector.runner.StageRuntime.init(StageRuntime.java:159) at com.streamsets.datacollector.runner.StagePipe.init(StagePipe.java:101) at com.streamsets.datacollector.runner.StagePipe.init(StagePipe.java:49) at com.streamsets.datacollector.runner.Pipeline.initPipe(Pipeline.java:382) at com.streamsets.datacollector.runner.Pipeline.init(Pipeline.java:295) at com.streamsets.datacollector.runner.preview.PreviewPipeline.run(PreviewPipeline.java:49) at com.streamsets.datacollector.execution.preview.sync.SyncPreviewer.start(SyncPreviewer.java:206) at com.streamsets.datacollector.execution.preview.async.AsyncPreviewer.lambda$start$0(AsyncPreviewer.java:94) at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.lambda$call$0(SafeScheduledExecutorService.java:249) at com.streamsets.datacollector.security.GroupsInScope.execute(GroupsInScope.java:33) at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.call(SafeScheduledExecutorService.java:245) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2017-11-08 10:07:23,210 [user:admin] [pipeline:mercury_mysql_hivecopy/mercurymysqlhivecopy9ba71407-6840-4b45-86ab-6c884050cf32] [runner:] [thread:preview-pool-1-thread-1] DEBUG StandaloneAndClusterPipelineManager - Status ... (more)

edit retag flag offensive close merge delete

Comments

Can you paste in the full stack trace? You should see some lines after this one, with 'caused by'

metadaddy gravatar imagemetadaddy ( 2017-11-07 14:54:35 -0500 )edit

I added full stack trace there.

casel.chen gravatar imagecasel.chen ( 2017-11-07 20:08:53 -0500 )edit

1 Answer

Sort by ยป oldest newest most voted
0

answered 2017-11-20 10:22:16 -0500

jeff gravatar image

What version of SDC are you running? This sounds a lot like SDC-7278, which is fixed in 2.7.1.1.

edit flag offensive delete link more
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2017-11-05 22:13:03 -0500

Seen: 65 times

Last updated: Nov 20 '17