Kafka consumer kerberos cluster mode HDP 2.5 does not work

asked 2018-01-13 14:03:59 -0600

this post is marked as community wiki

This post is a wiki. Anyone with karma >75 is welcome to improve it.

I'm designing a pipeline consuming data from a Kerberized Kafka cluster installed in an HDP cluster 2.5.3. The designed pipeline works fine in local mode. But if fails when it is deployed in cluster mode. I can't even validate and the error seems to be related to security. The error is:

KAFKA_41 - Could not get partition count for topic 'streamsets5' : com.streamsets.pipeline.api.StageException: KAFKA_41 - Could not get caseition count for topic 'streamsets5' : org.apache.kafka.common.KafkaException: Failed to construct kafka consumer

What am I missing in my configuration ?

Thanks in advance for your help

edit retag flag offensive close merge delete