Ask Your Question

Revision history [back]

click to hide/show revision 1
initial version

Is there a way to not commit offset to kafka when there is an error in a processor?


I want to build a pipeline that reads events from kafka, checks the DB if the event has already been received, and if it doesn't exist, writes to the DB and process the event.

Right now, my problem is that I need to make sure that I process all the events that I receive from kafka, but if by any chance the DB is unnavailable, the record will not be processed, but the kafka offeset will be commited.

Is there any way to make sure that Kafka Consumer does not commit the offset if there was an issue during the processing?

Thank you