Shell Executor not executing HDFS and hive commands

asked 2019-05-16 09:43:11 -0500

anonymous user


updated 2019-05-19 12:05:44 -0500

I have shell script which run unix commands , HDFS commands and also hive alter statements .

My streamsets is installed in some 12345 unix box , when ever i am trying to run in unix shell all commands(UNIX,HDFS,HIVE) are executing fine ,

Problem is when i am running shell script through Shell executor only UNIX commands are running , HDFS and Hive commands are not executing .

For my project XYZ group have all access and my user goutham is linked to that xyz group , with my user shell script is running all commands , now Streamsets SDC user is linked to my same project XYZ group which have all access . Still shell script couldn't able to execute HDFS and HIVE commands .

Can anyone help me one this , what are reasons ?

It is possible right to execute HDFS commands from shell executor ?

Thanks Goutham

edit retag flag offensive close merge delete


Does the same command work when you do `sudo -u sdc your-command`?

metadaddy gravatar imagemetadaddy ( 2019-05-16 10:28:12 -0500 )edit

Yes , my command is executing fine with " sudo -u sdc " . Below is my command sudo -u sdc hive -e "ALTER table ws_poc.TRAN_S_TEST SET LOCATION '/user/hive/streamsets'" This is executing completely fine in Shell cmd prompt , but not through Streamsets Shell Executor .

Goutham gravatar imageGoutham ( 2019-05-19 09:55:23 -0500 )edit

Not sure why this would be a problem then. Are you a StreamSets customer? I think you'll have to open a support ticket - they should be able to figure it out.

metadaddy gravatar imagemetadaddy ( 2019-05-19 10:48:13 -0500 )edit

i am Streamsets customer , i will open ticket to them . Could you please tell me we can execute Hive/HDFS commands through Shell Executor ?

Goutham gravatar imageGoutham ( 2019-05-19 11:13:10 -0500 )edit