How can I set the Kafka key without JSON scaffolding when writing to a Kafka Sink?

Neil experimented with the primary_key option in Kafka Sink, which populates the topic with a JSON attribute value. How can the key be set without the JSON scaffolding?



Asked on Mar 20, 2024

  • For CDC (Change Data Capture) formats like Debezium, the message key is already supported as a hidden column.
  • For normal JSON/Avro/Protobuf formats, support for including the message key as a column will be added soon.
  • As a workaround for normal JSON messages, you can manually add the key into the message for now.
Mar 20, 2024Edited by