Ingest files from Local directory to hdfs location

asked 2019-01-16 15:31:26 -0600

anonymous user


updated 2019-01-17 10:30:35 -0600

Issue 1:

The files are in sub-directories under parent directory, how to create the same structure in hdfs? For ex: My files reside in /aa/bb/cc/01 should go to hdfs location aa/bb/cc/01 and /aa/bb/cc/02 should go to /aa/bb/cc/02 - Right now all the files in 01 and 02 directory are going to aa/bb/cc location in hdfs.

Issue 2: I have a folder structure as aa/bb/01022019/cc/*. The third level in the folder structure will change (date column). I want to copy all the files with different third level to the hdfs location. The wild card in third level is not working. How to achieve this?

edit retag flag offensive close merge delete