all-things-risingwave

Is there a mechanism behind how RisingWave does the initial backload/backfill of data from a JDBC source when creating a materialized view table?

Maciej Kozik is asking about the process of backloading/backfilling data from a JDBC source when creating a materialized view table in RisingWave, specifically in reference to the mechanism behind CREATE MATERIALIZED VIEW table AS SELECT anything FROM source.

Ma

Maciej Kozik

Asked on Jul 03, 2023

  • The snapshot and streaming mechanism in RisingWave is currently the same as the Debezium connector.
  • There are plans to optimize the procedure for a better user experience.
  • While RisingWave does not directly support Debezium's incremental snapshot feature, it is possible to use it with RisingWave by setting up CDC with Kafka and configuring the Debezium server to use the incremental snapshot feature.
  • For ingesting CDC data, RisingWave already supports PubSub connector.
  • Until the backfill RFC is implemented, the recommended approach for backfilling data is to use CDC with Kafka and configure the Debezium server to use the incremental snapshot feature.
Aug 17, 2023Edited by