troubleshooting
How can I set the Kafka key without JSON scaffolding when writing to a Kafka Sink?
Neil experimented with the primary_key option in Kafka Sink, which populates the topic with a JSON attribute value. How can the key be set without the JSON scaffolding?
Ne
Neil
Asked on Mar 20, 2024
- For CDC (Change Data Capture) formats like Debezium, the message key is already supported as a hidden column.
- For normal JSON/Avro/Protobuf formats, support for including the message key as a column will be added soon.
- As a workaround for normal JSON messages, you can manually add the key into the message for now.
Mar 20, 2024Edited by