Ask Your Question
1

Load data from remote windows server to HDFS

asked 2019-05-23 12:51:28 -0500

KeerthiS gravatar image

There is a requirement to load data from a remote windows server to HDFS. Would like to know what needs to be installed in terms of software packages .

New to Streamsets.

edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted
0

answered 2019-05-29 09:52:12 -0500

metadaddy gravatar image

updated 2019-05-31 19:42:03 -0500

You have a few choices:

edit flag offensive delete link more

Comments

Currently we are able to read file using SFTP / FTP Client with NFS mount to load to HDFS. Would like to know- 1. Can we read a windows file directly (Streamsets Data Collector Edge uses HTTP Client as destination and no whole file option) 2. Any other alternative to SFTP

KeerthiS gravatar imageKeerthiS ( 2019-05-29 10:49:05 -0500 )edit

What is the data format?

metadaddy gravatar imagemetadaddy ( 2019-05-29 11:04:32 -0500 )edit

the file formats could any of the following - gz, zip, bz2 and some other file formats

KeerthiS gravatar imageKeerthiS ( 2019-05-29 14:21:56 -0500 )edit

we are looking we could read a Windows path (\\ServerName\Path) without using NFS or SMB. If there is no possibility, which one is best recommended and what are other options which you would suggest similar to NFS / SMB

KeerthiS gravatar imageKeerthiS ( 2019-05-29 14:23:13 -0500 )edit

For SFTP , what needs to be installed and SFTP client options and best recommended ?

KeerthiS gravatar imageKeerthiS ( 2019-05-29 14:51:47 -0500 )edit
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2019-05-23 12:51:28 -0500

Seen: 105 times

Last updated: May 31