Ask Your Question

Revision history [back]

click to hide/show revision 1
initial version

I don't think I've seen a tutorial yet. For now you'll have to refer to the documentation included in the Kafka stages for how to use it with Kafka.

It supports looking up the schema from the registry in the Kafka Consumer based on the embedded Schema ID as well as overriding it manually.

Typical usage would be to set the Key Deserializer and Message Deserializer to "Confluent" on the Kafka tab, and then in the Data Format tab choosing Avro and filling out the Schema Registry URL and using "Schema In Message/Data"

If you are looking to capture drift from SQL origins you can use the Avro Schema Generator Processor and then when sending data to your destination choose the Avro Data Format and configure the Schema Registry URL, set the schema to "In Record Header" and "Register Schema". You'll also need to specify the registry Subject for the schema.

This will register the updated schema with the registry (if a compatible change) or will error if incompatible.

Hope this helps to get you started.