Ask Your Question
1

I am new to StreamSets. I need to send files from local to HDFS.

asked 2017-11-29 23:45:33 -0600

Sam Sambath gravatar image

I am new to StreamSets. I need to send files from local to HDFS.

Could you please help me .?

edit retag flag offensive close merge delete

2 Answers

Sort by ยป oldest newest most voted
1

answered 2018-01-30 00:45:08 -0600

updated 2018-01-30 00:45:49 -0600

The given link explain configuring Hadoop along with hive and meta-data: https://github.com/streamsets/tutoria...

If you concerned with only writing to HDFS destination, above answered clearly by Roh.

edit flag offensive delete link more
1

answered 2017-11-30 08:42:11 -0600

Roh gravatar image

You can use the Directory origin for reading the files from local dir, Hadoop FS destination to dump it in the HDFS.

Your pipeline should look something like below image.

image description

Reading from local and writing in the other local directory and in HDFS. You can ignore the Local FS destination if you don't want to dump the files in other local location.

edit flag offensive delete link more
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2017-11-29 23:45:33 -0600

Seen: 335 times

Last updated: Jan 30