all-things-risingwave
What are the scaling limits to consider with RisingWave compared to Materialize?
James Johnston is comparing RisingWave and Materialize for a streaming pipeline and querying materialized views directly. He is specifically interested in understanding the scaling limits of RisingWave in comparison to Materialize.
Ja
James Johnston (he/him/his)
Asked on Dec 19, 2023
- RisingWave has NO intrinsic limitation in terms of scaling computation and storage.
- The computation engine is designed to be distributed from day one and S3 as the storage backend is virtually infinitely scalable.
- As long as the workload is scalable and can be partitioned onto multiple distributed nodes evenly, RisingWave can scale.
- If the metadata is very large or the operation of manipulating metadata is very frequent,
etcd
could be a bottleneck in terms of scaling. - RisingWave has an MPP-like architecture that allows true scale-out parallelism, while Materialize has an SMP-based architecture that allows only replica-based scalability.
Dec 19, 2023Edited by