all-things-risingwave

Can RisingWave Cloud support multiple clusters sharing the same tables for workload isolation?

I'm evaluating RisingWave Cloud for a streaming pipeline and I'm interested in the ability to isolate workloads by using multiple clusters that share the same data set, similar to Materialize. Materialize allows ingestion, stream processing, and online serving of queries to be handled by separate clusters. I want to know if RisingWave Cloud can have multiple clusters that share the same tables to separate workloads from each other. Is this feature available or on the roadmap, or are there alternatives like job quotas or priorities to prevent rogue jobs from causing problems?

Ja

James Johnston (he/him/his)

Asked on Dec 19, 2023

RisingWave does allow disaggregation of compute resources for different needs, such as pinning compute nodes to do stream processing, serving, etc. However, it currently does not support sharing namespaces or schemas across multiple clusters. A typical setup might involve separate clusters for production, staging, and development, with the ability to disaggregate compute nodes within a single cluster to prevent workloads from interfering with each other. While there is no current feature to disaggregate further within stream processing, it is technically feasible that a future RisingWave product could allow grouping of nodes in the cluster and assigning queries to a group, similar to Materialize's clusters. This is not a firm limitation of RisingWave for the medium to long term future.

Dec 19, 2023Edited by