HTTP Client Finisher Issue

asked 2020-07-10 08:38:06 -0500

anilkumardvs gravatar image


I am using HTTP Client Origin to read the data through API. The data is being read into Json but once all the data is read, the pipeline is not getting finished. It throws below error and never gets finished.

Error while running: com.streamsets.pipeline.api.StageException: HTTP_08 - When using pagination, the results field must be a list but a MAP was found

I have set the pipeline to run in Batch mode. Ideally the pipeline should get finished by itself in batch mode but it doesn't. Please advise.

I have tried using the pipeline finisher for my destination stage (as the HTTP client Origin doesn't support events) and used the pre condition: ${record:eventType()=='no-more-messages'} (and couple of options like 'finished-file' etc but not working.

I have also tried stream selector to go for trash in case the record error contains the above error text.

I have also tried below jython evaluator approach but still shows same error again and again. Please help

Thanks, Anil

edit retag flag offensive close merge delete