Store the specific column value in variable coming from REST API in JSON format

asked 2019-01-10 23:27:16 -0600

Syed gravatar image

Hi Team, I have a requirement in which I am extracting data from REST API and data coming in JSON format. Now the requirement is, I have to extract the data in Delta. For example in the URL I am passing start_date, and data extracted from start_date till date, now next day when this pipeline runs then I have to pass the end time of yesterday in the start_date of URL dynamically so that I can get the delta records. Can you please suggest if I can store the end time anywhere in variable in the streamset and when the next batch run it will pick the date and time from that variable and pass that in the start_date of the URL.

Thank You.

Kind Regards, Syed

edit retag flag offensive close merge delete