Ask Your Question
1

Does Streamsets support HDP 3.0

asked 2019-04-05 05:09:34 -0500

swapnil2.sonawane gravatar image

Hi,

We are trying to build a pipeline for reading data from JDBC (Source) and Hive Metastore (Destination)

in settings General Tab ==> Stage Library we choose Hive 2.1-HDP 2.6.2 1-1 (as it don't have VERSION Matching to our)

We have below configuration

1) HDP :: 3.0.1

2) Hive : 3.1

3) SDC : 3.8.0

Just have single record in JDBC table.

in preview mode or after running pipeline getting below error :

com.streamsets.pipeline.api.base.OnRecordErrorException: HIVE_17 - Information type missing or invalid in the metadata record: Record[headers='HeaderImpl[select * from shipping WHERE event_id >${offset} ORDER by event_id ;::rowCount:0:1]' data='Field[LIST_MAP:{event_id=Field[INTEGER:1], order_id=Field[INTEGER:123], event_type=Field[STRING:SHIPPED]}]']

at com.streamsets.pipeline.stage.destination.hive.HiveMetastoreTarget.write(HiveMetastoreTarget.java:200)
edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted
1

answered 2019-04-08 11:49:54 -0500

metadaddy gravatar image

Data Collector has a stage library for HDP 3.1, but not specifically 3.0 - see the documentation on additional stage libraries.

edit flag offensive delete link more
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2019-04-05 05:09:34 -0500

Seen: 62 times

Last updated: Apr 08