I'm trying to scale down my RisingWave cluster by reducing the number of replicas and their memory requests. However, after making these changes, I encountered an error stating ExecuteError: Scheduler error: No worker node found for parallel unit id: 44013
. I'm looking for best practices or guidelines on how to properly scale the cluster up or down to minimize disruptions. Is there a specific procedure I should follow?
Bahador Nooraei
Asked on Aug 09, 2023
When scaling down a RisingWave cluster, it's important to ensure that the number of compute nodes is not reduced below the number used when creating materialized views (MVs). If you're creating MVs, you should specify the parallelism manually, with a value that is usually sufficient for your throughput needs (e.g., 4 or 8 for < 100K records/s). This parallelism will remain unchanged unless you use the provided API to adjust it. For Kubernetes deployments, scaling down directly with risectl
is not convenient, and it's better to set the correct replica number in advance. Currently, there is no comprehensive guide for scaling, but the team is working on summarizing relevant issues and will publish guidance soon.