I have a Kafka topic that receives ~300 JSON events per second (around 1.3 MB/s) and I want to flatten the structure and create a materialized view from it. The topic contains historical data that I would like to include in the query result. Running a simple count takes a long time. What kind of cluster sizing would be necessary to handle this amount of data?
Can Yavuz
Asked on Sep 12, 2023