Every JDBC pipeline gives error while running second time

asked 2017-10-23 04:57:03 -0500

this post is marked as community wiki

This post is a wiki. Anyone with karma >75 is welcome to improve it.

Every pipeline with JDBC(JDBC multitable, JDBC query consumer) run once successfully. But when i run the same pipeline for the second time it does not. So i have to export and import the pipeline to run it again. Why is it so?...Can anyone suggest any solution

Thanks in advance

edit retag flag offensive close merge delete


What error do you get the second time? Bear in mind that in the default mode of operation (incremental), the source offsets are tracked/stored per pipeline, so you will need to Reset Origin if you want to process the same data again.

jeff gravatar imagejeff ( 2017-10-23 14:05:58 -0500 )edit

i ran a pipeline with JDBC Multitable as origin & JDBC producer as destination with default operation INSERT. It worked successfully.Then i tried to insert few more records using the same pipeline. I got error as follows,

Shruthi gravatar imageShruthi ( 2017-10-24 00:28:55 -0500 )edit

JDBC_14 - Error processing batch. SQLState: 23000 Error Code: 1062 Message: Duplicate entry '60' for key 'PRIMARY' Cause: com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: Duplicate entry '60' for key 'PRIMARY'

Shruthi gravatar imageShruthi ( 2017-10-24 00:29:58 -0500 )edit

Can you describe your use case more? The error is stating that you are trying to insert the same record (same PK value) into the target table.

jeff gravatar imagejeff ( 2017-10-24 11:01:20 -0500 )edit

I am trying to insert new records to the same target table using the same pipeline. Is it not possible?.. I am not inserting the same PK value. But still the error says so.

Shruthi gravatar imageShruthi ( 2017-10-25 00:32:44 -0500 )edit