Ask Your Question

JDBC Producer insert / update

asked 2020-02-11 16:22:54 -0500

srinath_222 gravatar image


I am using JDBC Producer as my destiation in streamsets pipeline. Insert/update can be set using sdc.operation.type attribute in Streamsets. But I am not sure if the record is an insert/update while receiving the data.

DO we have any option to set insert else update in JDBC Producer of Streamsets such that it can update target table based on primary key ??

edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted

answered 2020-02-11 16:40:19 -0500

iamontheinet gravatar image


You can use JDBC Lookup processor to first check if the record exists and then use Expression Evaluator to set sdc.operation.type accordingly. See sample pipeline screenshots.

In this sample pipeline, (new) boolean column exists will be set to true or false depending on whether the record exists based on key column pid. Based on that, sdc.operation.type is set to 1 for insert or 3 for update.

image description

image description

Cheers, Dash

edit flag offensive delete link more



Cheers Dash...It worked

srinath_222 gravatar imagesrinath_222 ( 2020-02-11 17:18:13 -0500 )edit

Awesome! :)

iamontheinet gravatar imageiamontheinet ( 2020-02-11 17:21:19 -0500 )edit
Login/Signup to Answer

Question Tools

1 follower


Asked: 2020-02-11 16:22:54 -0500

Seen: 216 times

Last updated: Feb 11