Ask Your Question
0

Workaround when HTTP Client Request Failed

asked 2017-10-19 05:34:58 -0500

Vivian Y gravatar image

updated 2017-10-19 14:18:09 -0500

metadaddy gravatar image

Aware that this is an issue that when HTTP Client failed , it will cause the whole pipeline to keep on retry. https://issues.streamsets.com/browse/...

Is there any workaround for this? When HTTP Client Return Error, we able to treat the files as error records and send to that particular files to another places instead of letting the files stay at the particular folder forever and let the pipeline keep retry.

My use case will be even there is one records failed, instead of keep on retrying the pipeline, we can move the error files to another folder and raise alert on this error records, and let the rest of the records to continue run.

edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted
0

answered 2017-10-19 13:46:19 -0500

jeff gravatar image

Are you talking about the HTTP Client Processor? That already has support for what you describe. Specifically, if any non success status (outside of 2xx) is seen in the response, it generates an error record. What specific stage are you talking about?

edit flag offensive delete link more

Comments

Which version does this function support? My version is 2.7.1.1 and i does not see this in the the HTTP client processor. Is it in the latest version of release?

Vivian Y gravatar imageVivian Y ( 2017-10-19 22:07:42 -0500 )edit
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2017-10-19 05:34:58 -0500

Seen: 5,228 times

Last updated: Oct 19 '17