I'm having trouble with a Risingwave sink not connecting to my Kafka instance. I'm running Risingwave using Docker Compose on macOS Ventura 13.4.1 and Kafka locally (not in Docker). I've set up a materialized view and created a sink with the correct Kafka bootstrap server settings, but there's nothing in the Kafka topic when I change data in the PostgreSQL database. Here's how I created the sink:
CREATE SINK frommvusers FROM mvusers WITH (connector = 'kafka', type = 'debezium', properties.bootstrap.server = 'host.docker.internal:9092', topic = 'risingtest', primary_key = 'id')However, Risingwave seems to be trying to connect to localhost instead of host.docker.internal. The logs show a Connection refused error for localhost:9092. I'm not sure why Risingwave is not using the host.docker.internal:9092 setting. Can anyone help me figure out what's going wrong?
Charlie
Asked on Jul 19, 2023
I found the issue with my Risingwave sink not connecting to Kafka. It turned out to be a problem with Kafka's advertised.listeners configuration. I was using Kafka with the Kraft protocol for the first time, so I needed to adjust my computer's configurations. I couldn't use the host network on macOS, but after fixing the advertised.listeners setting, everything started working. Here's the configuration that worked for me:
advertised.listeners=PLAINTEXT://192.168.1.34:9092And I updated the sink creation command with the same IP for the host:
CREATE SINK sink1 FROM mvusers WITH (connector='kafka', type='debezium', properties.bootstrap.server='192.168.1.34:9092', topic='risingtest', primary_key='id');If anyone else encounters similar problems, make sure to check your Kafka advertised.listeners settings.