all-things-risingwave
What is the recommended way to handle schema changes in Rising Wave when sinking to a Clickhouse DB?
I would like to know the best practice for managing schema changes in Rising Wave when a RW table is sinking to a Clickhouse DB. Specifically, what happens if the RW table schema changes but the Clickhouse schema hasn't been updated yet, and how should this type of schema change be handled?
Tu
Tuan Vuong
Asked on Sep 15, 2023
- Currently, altering the schema of a table in Rising Wave does not affect downstream streaming jobs like materialized views or sinks. They will still use the original columns.
- To reference new columns in downstream tables, one has to drop and rebuild the downstream jobs. Instantly changing the plan of materialized views or sinks is a work in progress.
- If you want to refresh the schema of a downstream table, you can create a new job, drop the old one, and rename the new one back.
- Materialized views are persisted, so dropping and recreating them will lead to all upstream data being reprocessed.
- The same applies to sinks; if a materialized view is rebuilt, the sink will also take the refreshed columns.
Sep 18, 2023Edited by