post shell executor not working

asked 2018-08-09 20:51:23 -0600

this post is marked as community wiki

This post is a wiki. Anyone with karma >75 is welcome to improve it.

I have a shell script which when placed in pre shell executor is working. When the same script is placed in post completion, it is not working.

Details:

  • source : sftp whole zip file
  • target: hdfs whole zip file
  • pipeline finisher executor to stop pipeline when job completes.
  • shell executor - script to copy hdfs target file to another hdfs location.

Job is completing successfully without error but copy not working 2018-08-10 21:46:11,343 INFO Starting memory collector for HadoopFS_01
2018-08-10 21:46:11,344 INFO Starting memory collector for PipelineFinisherExecutor_01
2018-08-10 21:46:11,346 INFO Starting memory collector for Shell_01
2018-08-10 21:46:25,818 INFO Destroying pipeline with reason=FINISHED

not even executing the shell script. i put some echo stmts for testing. they are not displaying.

edit retag flag offensive close merge delete

Comments

What events are you using - pipeline start/stop or events from the origin/destination? How are you configuring the environment variables for the shell executor? You might be referencing some fields that are present in one context, but not the other.

metadaddy gravatar imagemetadaddy ( 2018-08-09 21:33:55 -0600 )edit

i tried stop event and event from destination. start event is working but not any stop events. it is exactly same script. log info attached above

Robot gravatar imageRobot ( 2018-08-10 23:05:35 -0600 )edit