Not stopping pipeline on runtime exception

asked 2018-11-30 04:17:19 -0600

niteen22 gravatar image

I have created a pipeline to ingest data into database from csv. While I have made all possible integrity checks, there is always a chance of rumtime exception (eg string value in integer column)

In such cases currently pipeline stops and skip further records and other files as well. What I want is to skip or log that record separately and continue with the processing

I am using JDBC Producer as my destination. Is there any configuration available for this


edit retag flag offensive close merge delete