Ask Your Question

How to you set Kafka producer key to null?

asked 2020-03-27 16:36:56 -0500

meash-nrel gravatar image

Kafka Producer is forcing the message key to be a string. If I omit in the Message Key field in Kafka Producer, it makes the message key an empty string.

Kafka Brokers treat this as a null, and will evenly distribute messages across partitions. All good there on message distribution across partitions. However, other Confluent tooling is NOT happy with a blank string as key. It is expecting a string or a Null. In particular, the Confluent REST API is very upset with empty string, as it is expecting JSON or Avro key if the value is in those formats, as is Confluent Python library. It handles Null keys fine, as it isn't trying to then parse the string.

Is there a way to force in the MessageKey to be a Null value, instead of blank string? Using blank string is causing unintended consequences in other Kafka tools.

edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted

answered 2020-03-28 11:29:37 -0500

J3 gravatar image


I am not sure if this will help because I am using Confluent Cloud, however, if you take a look here. I think adding the Kafka Configuration and leaving the Kafka Message Key blank will work for you.

---J3 Out!

edit flag offensive delete link more


So you think setting key serializer in the raw config to "io.confluent.kafka.serializers.KafkaAvroSerializer" is the key?

meash-nrel gravatar imagemeash-nrel ( 2020-03-30 10:30:55 -0500 )edit

Yes, I think it's worth a try. Adding Kafka Configuration properties, I believe override SDC settings and use the default Kafka functionality. Let me know the outcome. Good luck and be safe!

J3 gravatar imageJ3 ( 2020-03-30 12:47:58 -0500 )edit
Login/Signup to Answer

Question Tools

1 follower


Asked: 2020-03-27 16:36:56 -0500

Seen: 1,470 times

Last updated: Mar 28 '20