Ask Your Question
1

Is there a recommended value for JDBC Consumer "Number of Retries on SQL Error?"

asked 2018-08-15 05:18:08 -0500

neiliox gravatar image

We're losing some database events and I wonder if it's because "Number of Retries on SQL Error" is set to the default of 0. Perhaps a transient timeout situation could be causing events to be missed and not retried.

edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted
1

answered 2018-08-15 19:01:52 -0500

metadaddy gravatar image

I don't know of a recommended value, but you should try increasing it from the default of 0. There is, of course, a trade-off in that, if the data source is indeed offline, it will take longer to recognize that.

Note that there are other possible causes of lost data with JDBC. You should carefully examine your SQL query. In particular, if the offset column is not unique, you could be missing data. For example, say your SQL query is:

SELECT * FROM mytable WHERE col1 > ${OFFSET} ORDER BY col1

If the pipeline reads all the data up to a given value of col1, then another row is added with a duplicate value of col1, then that row will not be read on the next query. This can happen with timestamps where they are not unique.

edit flag offensive delete link more

Comments

That's very useful, thank you!

neiliox gravatar imageneiliox ( 2018-08-16 03:43:19 -0500 )edit
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2018-08-15 05:18:08 -0500

Seen: 72 times

Last updated: Aug 15