John is trying to implement watermark and window function in RisingWave while keeping the data for a specific period of time, but encountered issues with the table needing to be 'append only'. He provided SQL code for creating a table with watermark and window function, and faced errors with the delete statement. Renjie Liu suggested using a temporal filter. Eric advised against using watermark for this case and recommended removing the watermark statement and using either delete command or temporal filter. John acknowledged the suggestions and mentioned trying them out.
John
Asked on Nov 01, 2023
Watermark in RisingWave is primarily for query with emit on window close. For scenarios where data needs to be kept for a certain period of time, it is recommended to use other methods such as delete commands or temporal filters.
To address the issue of the table needing to be 'append only' when setting a watermark, consider removing the WATERMARK FOR statement from the table creation SQL.
Instead of using watermark, utilize delete commands or temporal filters to manage data retention within RisingWave.
Temporal filters can be applied to filter out messages based on time criteria. Refer to the RisingWave documentation on SQL pattern temporal filters for more information.
Consider creating a source and then creating a materialized view with a temporal filter to achieve the desired data retention and processing logic.
