Ask Your Question

Can we delete the file created the source location using the SFTP/FTP Client Stage in streamsets

asked 2018-05-17 09:40:21 -0600

Janmejay gravatar image

While using SFTP/FTP Client in my pipeline I want to delete the file from the original location once the pipeline is successfully executed.

edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted

answered 2018-05-18 07:45:23 -0600

todd gravatar image

updated 2018-05-18 07:48:23 -0600

One way could be triggering the deletion of the file via "Produce Events" in the SFTP origin. The event could call a shell script executor which could be customized for your environment and delete the file. Skeleton example screenshot

image description

edit flag offensive delete link more


I tried this but the files to be delete are on different server.I tried from the putty of the streamset server to connect to file server, I am able to establish the connection but fails by Shell Executor gives error "stderr: Pseudo-terminal will not be allocated because stdin is not a terminal."

Janmejay gravatar imageJanmejay ( 2018-07-09 02:20:52 -0600 )edit
Login/Signup to Answer

Question Tools

1 follower


Asked: 2018-05-17 09:40:21 -0600

Seen: 1,457 times

Last updated: May 18 '18