Andrei Nesterov mentioned that frequent writes to Apache Iceberg result in a large number of small files on S3. Tianxiao Shen provided information on the frequency of writes and configuration options for RisingWave integration with Apache Iceberg.
Andrei Nesterov
Asked on Jun 28, 2023
checkpoint_frequency
parameter.checkpoint_frequency
parameter determines how often RisingWave commits to Apache Iceberg.checkpoint_frequency
setting can be adjusted to control the frequency of writes to Apache Iceberg.