Ask Your Question
0

API call at the end of pipeline execution

asked 2020-01-09 07:52:27 -0600

MuralidharKoti gravatar image

Hi, I have to requirement to trigger refresh of a Power Bi dataset after completion of a StreamSets pipeline. PowerBi provides an API, through which we can trigger the refresh. A typical API calls looks like 'POST https://api.powerbi.com/v1.0/myorg/da.... I see we can call API via HTTP client, but how can I associate a HTTP client to be executed after completion of a pipeline. Please advise what is the best possible way to achieve this requirement in StreamSets.

edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted
0

answered 2020-01-09 12:25:36 -0600

jeff gravatar image

What is your pipeline origin? If it's one of the origins that supports completion events (ex: no-more-data), then you can attach an HTTP client processor to the event lane of the origin, filter for the no-more-data event, and then the stage will run only when the final event is generated. See here for more info.

If somehow the HTTP client processor isn't able to invoke your API, then you can possibly use a script to do it in the shell executor.

edit flag offensive delete link more
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2020-01-09 07:52:27 -0600

Seen: 7 times

Last updated: Jan 09