Ask Your Question
1

How can I read Parquet files from HDFS? Using which component?

asked 2020-02-13 06:09:55 -0500

Aakash gravatar image

updated 2020-02-13 10:10:26 -0500

metadaddy gravatar image

I want to read a Parquet format file and tried reading the file using Hadoop FS Standalone origin. In that origin we have the Data Format option but Parquet file format is not listed there.

edit retag flag offensive close merge delete

Comments

Hi @Aakash i trying the same can you please post Hadoop fs origin config. And how did you solved reading data from parquet? Thanks

strem_dev gravatar imagestrem_dev ( 2020-08-19 09:23:15 -0500 )edit

1 Answer

Sort by ยป oldest newest most voted
1

answered 2020-02-13 10:09:32 -0500

metadaddy gravatar image

The Parquet file format does not match StreamSets Data Collector's stream-based architecture. Instead, you should use StreamSets Transformer, which includes support for reading Parquet files.

edit flag offensive delete link more
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2020-02-13 06:09:55 -0500

Seen: 178 times

Last updated: Feb 13