Ask Your Question
0

Pipelines status is RUNNING_ERROR can not stop

asked 2020-07-04 00:39:12 -0500

gavin gravatar image

StreamSet-Version: StreamSets Data Collector 3.16 my pipeline is read mysqlbinlog and wirte to postgre, when i have some pk error , the pipline is locked at running_error status. and only restart sdc .

logs:

2020-07-04 13:14:24,653 data sync - gzrz-wms/datasync685d1708-869d-4445-9322-6e49d1735e14 ERROR Error while processing batch of records together: SQLState: 23505 JdbcMultiRowRecordWriter *admin 0 ProductionPipelineRunnable-datasync685d1708-869d-4445-9322-6e49d1735e14-data sync - gzrz-wms Error Code: 0 Message: ERROR: duplicate key value violates unique constraint "shipping_pick_detail_pkey" (seg0 172.16.5.97:40000 pid=24080) Detail: Key (pick_id)=(20070400000330) already exists. 2020-07-04 13:14:24,655 data sync - gzrz-wms/datasync685d1708-869d-4445-9322-6e49d1735e14 ERROR SQLState: 25P02 JdbcBaseRecordWriter *admin 0 ProductionPipelineRunnable-datasync685d1708-869d-4445-9322-6e49d1735e14-data sync - gzrz-wms Error Code: 0 Message: ERROR: current transaction is aborted, commands ignored until end of transaction block Cause: org.postgresql.util.PSQLException: ERROR: duplicate key value violates unique constraint "shipping_pick_detail_pkey" (seg0 172.16.5.97:40000 pid=24080) Detail: Key (pick_id)=(20070400000330) already exists.

org.postgresql.util.PSQLException: ERROR: current transaction is aborted, commands ignored until end of transaction block at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2433) at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2178) at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:306) at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:441) at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:365) at org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:155) at org.postgresql.jdbc.PgPreparedStatement.executeUpdate(PgPreparedStatement.java:132) at com.zaxxer.hikari.pool.ProxyPreparedStatement.executeUpdate(ProxyPreparedStatement.java:61) at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeUpdate(HikariProxyPreparedStatement.java) at com.streamsets.pipeline.lib.jdbc.JdbcMultiRowRecordWriter.processBatch(JdbcMultiRowRecordWriter.java:312) at com.streamsets.pipeline.lib.jdbc.JdbcMultiRowRecordWriter.processQueue(JdbcMultiRowRecordWriter.java:294) at com.streamsets.pipeline.lib.jdbc.JdbcMultiRowRecordWriter.writeBatch(JdbcMultiRowRecordWriter.java:169) at com.streamsets.pipeline.lib.jdbc.JdbcUtil.write(JdbcUtil.java:1173) at com.streamsets.pipeline.lib.jdbc.JdbcUtil.write(JdbcUtil.java:1095) at com.streamsets.pipeline.stage.destination.jdbc.JdbcTarget.write(JdbcTarget.java:315) at com.streamsets.pipeline.stage.destination.jdbc.JdbcTarget.write(JdbcTarget.java:304) at com.streamsets.pipeline.api.base.configurablestage.DTarget.write(DTarget.java:34) at com.streamsets.datacollector.runner.StageRuntime.lamb 2020-07-04 13:14:24,657 data sync - gzrz-wms/datasync685d1708-869d-4445-9322-6e49d1735e14 ERROR Failed executing stage 'JDBCProducer_01': com.streamsets.pipeline.api.StageException: JDBC_14 - Error processing batch: [SQLState:25P02][ErrorCode:0] ERROR: current transaction is aborted, commands ignored until end of transaction block PipeRunner *admin 0 ProductionPipelineRunnable-datasync685d1708-869d-4445-9322-6e49d1735e14-data sync - gzrz-wms SQLState: 25P02 Error Code: 0 Message: ERROR: current transaction is aborted, commands ignored until end of transaction block Cause: org.postgresql.util.PSQLException: ERROR: duplicate key value violates unique constraint "shipping_pick_detail_pkey" (seg0 172.16.5.97:40000 pid=24080) Detail: Key (pick_id)=(20070400000330) already exists.

com.streamsets.pipeline.api.StageException: JDBC_14 - Error processing batch: [SQLState:25P02][ErrorCode:0] ERROR: current transaction is aborted, commands ignored until end of transaction block SQLState: 25P02 Error Code: 0 Message: ERROR: current transaction is aborted, commands ignored until end of transaction block Cause: org.postgresql.util.PSQLException: ERROR: duplicate key value violates unique constraint "shipping_pick_detail_pkey" (seg0 172.16.5.97:40000 pid=24080) Detail: Key ... (more)

edit retag flag offensive close merge delete

Comments

Force stop is not work

gavin gravatar imagegavin ( 2020-07-04 00:52:14 -0500 )edit

1 Answer

Sort by » oldest newest most voted
0

answered 2020-07-17 02:22:21 -0500

gavin gravatar image

updated 2020-07-17 02:22:57 -0500

this question is report in new topic

edit flag offensive delete link more
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2020-07-04 00:39:12 -0500

Seen: 231 times

Last updated: Jul 17