I'm using RisingWave (RW) version 1.7.0 with a Cassandra sink, and I'm encountering several errors. Initially, I received an oversized mutation
error, which I attempted to fix by increasing the commitlog_segment_size
to 1GiB. This led to a Batch too large
warning, so I adjusted the batch_size_fail_threshold
. However, I'm now seeing errors related to write query failures at consistency TWO and timeouts. I've tried various configurations, but I'm still facing issues with oversized mutations and timeouts. Is there a workaround or fix for these problems in RW's Cassandra sink?
André Falk
Asked on Mar 05, 2024
The first error you encountered was likely due to the memory limit of Cassandra. After increasing the commitlog_segment_size
to 1GiB, you encountered a Batch too large
error because the max batch limit of RW's Cassandra sink is 16,024 rows. Adjusting the batch_size_fail_threshold
led to further errors due to the previous batch size issue. To resolve this, you should:
commitlog_segment_size
at 1GiB or a smaller value that is still larger than the default.batch_size_fail_threshold
to a value greater than 1MiB.If you continue to encounter oversized mutation errors, you may need to further increase the commitlog_segment_size
. For the timeout errors, it's possible that the Cassandra driver's default timeout of 2 seconds is being exceeded. We are working on a fix for the batch too large
error and will include it in the next release. Additionally, we will consider making these options configurable to accommodate different environments and requirements.