Ask Your Question

adding date as prefix to whole file hdfs target

asked 2018-08-08 20:38:01 -0500

this post is marked as community wiki

This post is a wiki. Anyone with karma >75 is welcome to improve it.

I need to add current date as prefix to hdfs whole file target. Please let me know how I can do this.

edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted

answered 2018-08-08 21:38:58 -0500

Dean Han gravatar image

TIME function can not be used in name prefix, so you can try to use ${runtime:loadResource("date.txt", false)} , and write a script to udpate the date in /streamsets-datacollector-xxxx/resources/date.txt

edit flag offensive delete link more


Thank you! Can you explain in more detail please?

Robot gravatar imageRobot ( 2018-08-08 23:20:33 -0500 )edit

Touch a file named date.txt in your streamsets dc installation path, /streamsets-datacollector-xxxx/resources/, input '2018-08-09' manually, if you can not write a script to update this date automatically, in config of HDFS, set the prefix with ${runtime:loadResource("date.txt", false)}. Then try .

Dean Han gravatar imageDean Han ( 2018-08-09 01:02:59 -0500 )edit
Login/Signup to Answer

Question Tools

1 follower


Asked: 2018-08-08 20:38:01 -0500

Seen: 23 times

Last updated: Aug 08