Ask Your Question

Unable to Find Hadoop FS Origin Stage although having Hadoop FS Destination available

asked 2018-01-18 05:31:17 -0500

Janmejay gravatar image

updated 2018-01-18 21:11:59 -0500

metadaddy gravatar image

I am new to StreamSets. I need to read files from HDFS but I cannot find HDFS Origin Stage in my Stage library. I do have all the necessary libraries as I have used Hadoop FS Destination in other pipelines. For CDH I have libraries with version 5.8 to 5.13.

edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted

answered 2018-01-18 21:16:35 -0500

metadaddy gravatar image

You need to set the pipeline Execution Mode to Cluster Batch. This will allow you to select the Hadoop FS origin.

image description

edit flag offensive delete link more


Hi, even I am unable to find Hadoop FS Origin Stage. I have changed the pipeline execution mode to cluster batch but still cannot see it. Do I need to make any other configuration changes?

mayankpahwa gravatar imagemayankpahwa ( 2018-05-01 08:40:34 -0500 )edit

Thanks @Pat for the answers. I am now able to configure the Hadoop Origin Node

Janmejay gravatar imageJanmejay ( 2018-05-17 09:45:45 -0500 )edit
Login/Signup to Answer

Question Tools

1 follower


Asked: 2018-01-18 05:31:17 -0500

Seen: 215 times

Last updated: Jan 18 '18