Load Different files to HDFS in Single Pipeline

asked 2018-11-21 08:25:49 -0600

jay1988 gravatar image

We have a requirement to Load 2 different files (Data file & Trailer file) data to 2 HDFS Targets and needs to be Loaded in single Pipeline.The files are a data file and its corresponding trailer file.On the Pipeline execution itself we need to check the counts and ensure the count is matching with data file and have to route it to 2 different targets.Is it ;possible to achieve the same?.

edit retag flag offensive close merge delete