I have ftp where I want to take the file that is last dropped. I tried First File field with file pattern ABC*.csv.
But I got below exception: org.apache.commons.vfs2.FileSystemException: Could not determine the last modified timestamp of "ftp://*////*/ABC.csv" because it does not exist.
Thanks Rajashree
Hi!
It appears that the Resource URL might not be evaluating correctly. For reference, see below:
In the above config, all files relative to $USER_HOME/datafiles folder will be processed, including new files (with later timestamp) that are uploaded to that folder after pipeline has started.
Hope this helps.
Cheers, Dash
It will process files in earliest to latest timestamp order. Can you update your original question and add screenshot of your config like I did in my answer? Also, are you starting and stopping the pipeline before uploading new files or are you uploading new files while the pipeline is running?
Hi I am not able to upload the screenshot image. The files are already uploaded and then I'm starting the pipeline. But when it go to production the files will be dropped when the pipeline is running, so I would like to know both the scenarios. Thanks in advance
Asked: 2018-12-12 02:47:18 -0500
Seen: 150 times
Last updated: Dec 14 '18
