I'm trying to understand where RisingWave fits in the landscape of data processing tools. Is it meant to replace existing systems like Flink, or is it introducing a new concept altogether? How does it compare to other tools that offer user-defined functions, such as Flink and Spark streaming, given that RisingWave provides a SQL API?
leiysky
Asked on Jun 15, 2022
RisingWave can indeed serve as a data pipeline and is positioned as a powerful ETL tool. It's not a brand new concept, as there are other streaming databases like ksqldb and pipelinedb. However, RisingWave aims to replace big-data-style streaming systems like Flink. The advantage of RisingWave lies in its SQL-centric approach, which is intended to make stream processing accessible to a broader audience, not just advanced users who can code. While competitors also offer SQL interfaces, RisingWave's design and goals are distinct, particularly in its focus on reducing costs rather than solely improving performance. The market for streaming technology is still growing, and there's room for multiple products. RisingWave's initial idea was not to build a streaming data warehouse, but it could naturally evolve into one if users choose to store data within it. The project's vision is to build a better system than Flink and to encourage the adoption of streaming technology.