all-things-risingwave
Is it possible to specify Kafka headers for messages when using Kafka as a sink in RW?
Abhishek Amte is looking for a way to specify Kafka headers for messages in a use case where standard headers are sent across every message in Kafka following the CloudEvents spec. Yuhao Su mentions that it is not currently supported by RW and asks for more details on the requirement. Abhishek Amte explains the use case further, and Tao Wu inquires about the downstream system and suggests using the structured-mapping mode.
Ab
Abhishek Amte
Asked on Feb 16, 2024
- Kafka headers for messages are not currently supported by RW when using Kafka as a sink.
- Abhishek Amte's team follows the CloudEvents spec and sends standard headers with every message in Kafka.
- The structured-mapping mode, which encodes everything in Kafka value except the
content-type: application/cloudevents+json, is suggested as an alternative. - Abhishek Amte's team prefers putting CloudEvents headers in Kafka headers and the data in the Kafka message.
Feb 22, 2024Edited by