How to Schedule job quarterly in a incremental way in StreamSets?

asked 2019-05-23 08:00:14 -0500

Jeyakumar gravatar image

I have a folder in that there are 1000 files and i like to fetch these files from that folder to the HDFS folder once in three months. After three months additionally there are other set of 1000 files to placed in that folder,now i want to fetch only the additionally added files.

For Example

June 1 2019 - Schedule the job and the next run will be Sep 1 2019 but only the files between placed between June 2 2019 to Aug 31 2019

Please help me to figure out the solution for this.Thanks in advance.

Thanks Jeyakumar

edit retag flag offensive close merge delete