all-things-risingwave
How does RisingWave handle out of order data?
I'm wondering how RisingWave handles out of order data and what options are available for processing it.
Ab
Abhay Bothra
Asked on Nov 02, 2022
- RisingWave can handle out-of-order data by retaining all intermediate states and processing it normally if the downstream system supports updates and the user is willing to pay more storage costs.
- If the downstream system cannot handle updates, users can introduce watermarks into SQL queries to make the output stream append-only.
- When out-of-order data arrives after the corresponding watermark, that data is discarded and no updates are output.
- Support for watermarks is in progress, and design details can be found at GitHub RFCs.
Nov 02, 2022Edited by