Ask Your Question
1

use file's mtime in SFTP/FTP Origin event lane

asked 2020-03-24 08:01:23 -0500

nicopadu gravatar image

We're currently using SFTP/FTP Origin.

While reading data we need to include as a new field the file's last modified timestamp. On data lane it is possible by using 'mtime' header value.

The problem is that we also consume "finished-file" event to get the number of rows for audit records. Here we also need the source file's last modified timestamp to match in case we reprocess same filename in a later time. In event lane we only have event's created timestamp, and also we cannot share any info between data & event lanes.

Does anyone has any alternative to achieve such goal?

Thanks!

edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted
0

answered 2020-03-24 12:12:48 -0500

metadaddy gravatar image

At present, you would need to use the Redis destination or a script evaluator to write the mtime with a suitable key to an external key/value store such as Redis or memcache, then retrieve it using the Redis Lookup processor or a second script evaluator.

edit flag offensive delete link more
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2020-03-24 08:01:23 -0500

Seen: 60 times

Last updated: Mar 24