How to resolve Risingwave Kafka `MessageSizeTooLarge` error when sinking to Redpanda?
I'm working with Risingwave and trying to sink a materialized view (MV) to Redpanda. However, I'm encountering an error Actor 24056 exit unexpectedly: Executor error: Sink error: Kafka error: Message production error: MessageSizeTooLarge (Broker: Message size too large). I've attempted to set message.max.bytes but received another error Client creation error: receive.message.max.bytes must be >= fetch.max.bytes + 512. Adjusting fetch.max.bytes on the producer/sink side doesn't seem to resolve the issue. Additionally, there are problems with consumer group cleanup, as Risingwave doesn't clean up consumer groups, leaving many leftover groups. Here's the behavior I'm observing:
- Create a source from Kafka
- Create a MV from the source (works fine)
- Create a sink back to Kafka
After about 4-5 minutes of sinking, the MessageSizeTooLarge error occurs. I've also tried different configurations for properties.message.max.bytes and properties.receive.message.max.bytes, but the error persists. Is there a way to properly configure Risingwave to avoid this error and successfully sink to Redpanda?
Eddie Wang
Asked on Dec 06, 2023
For the MessageSizeTooLarge error, it seems like a bug that has been fixed in the latest Risingwave image. Make sure to use the latest image where properties.message.max.bytes and properties.receive.message.max.bytes are set correctly. If the problem still occurs, further investigation is needed. As for the consumer group cleanup issue, it's a known inconvenience, and the Risingwave team is discussing ways to reduce the overall consumer group IDs by reusing previous group.id for each actor. To potentially mitigate the batch size issue, try reducing properties.batch.size or properties.batch.num.messages. However, if these settings still result in failure, the bug fix for message.max.bytes should be applied, and you should ensure you're using the latest Risingwave version. Here's the link to the pull request with the fix: https://github.com/risingwavelabs/risingwave/pull/13847. Additionally, check the documentation for Redpanda and other Kafka providers to ensure compatibility with their parameter settings.