Ask Your Question

How to sync dynamic Kafka topics into Hive/HBase

asked 2018-04-02 20:55:29 -0500

casel.chen gravatar image

updated 2018-04-03 19:50:25 -0500

metadaddy gravatar image

I want to sync dynamic Kafka topics into Hive/HBase; those Kafka topics are created dynamically by program with some pattern, each topic has its own Avro schema, and want to persist them into HDFS for analysis later. How do I achieve it with SDC?

edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted

answered 2018-04-03 19:50:09 -0500

metadaddy gravatar image

Unfortunately, this is not possible at present, since the Kafka Multitopic Consumer requires a predefined list of topics. We have an issue open for this enhancement - please watch/vote/comment on SDC-8533.

edit flag offensive delete link more
Login/Signup to Answer

Question Tools

1 follower


Asked: 2018-04-02 20:55:29 -0500

Seen: 396 times

Last updated: Apr 03 '18