Ask Your Question

Does Stream sets-data-collector-3.5, will it supports Apache Kafka 2.3

asked 2020-03-27 02:32:26 -0500

RaviPrakash gravatar image

Hi team,

Currently we are using open source Stream sets data collector 3.5 and Apache Kafka 2.0. Its working fine. Now our Apache Kafka is getting updated to 2.3. We don't see any any option for Apache Kafka 2.3 in Streamset data collector3.5.

Do we need to update our Streamset data collector?

Do we need to upgrade stage libraries ?

edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted

answered 2020-03-28 05:25:51 -0500

updated 2020-03-28 05:27:05 -0500

I can not answer your question regarding if SDC 3.5 will include a newer stage library than Kafka 2.0.

But I can tell you that you can easily continue with the Kafka 2.0 library, even when your Kafka cluster is updated to 2.3. Kafka is both forward and backward compatible, so client version doesn't have to strictly match the server version of Kafka. Of course if you keep using an older version (say 2.0) then you might not be able to use some features included in later versions, especially if they involve changes on the Kafka protocol layer, but this is not the case in the 2.x as far as I know and in the case of using Kafka from SDC the functionality is anyway given by SDC.

And by that you also do not have to upgrade your Streamsets Data Collector.

BR Guido

edit flag offensive delete link more
Login/Signup to Answer

Question Tools

1 follower


Asked: 2020-03-27 02:32:26 -0500

Seen: 31 times

Last updated: Mar 28