all-things-risingwave

What is the mechanism for compacting SST files in Hummock and how is it monitored by the meta service?

JJ is seeking clarification on the compaction mechanism in Hummock and how it is monitored by the meta service. Alex Chi provides an explanation of the compaction process in Hummock.

JJ

JJ

Asked on May 16, 2022

  • Hummock adapts the RocksDB's leveled-compaction strategy for compacting SST files.
  • When the number of SST files at the same level is too high, compaction is triggered to merge them.
  • The meta service monitors the compaction process in Hummock.

Example:

# Compaction process in Hummock
# Using RocksDB's leveled-compaction strategy

# Trigger compaction when the number of SST files is high
if num_sst_files_same_level > threshold:
    merge_sst_files()

# Meta service monitors the compaction process
monitor_compaction_process()
May 16, 2022Edited by