How to correctly configure Kafka source with SASL authentication in RisingWave?
I'm trying to set up a Kafka source in RisingWave with SASL authentication, but I'm encountering a GroupAuthorizationFailed
exception. I've confirmed that the credentials are correct. Here's the configuration I'm using:
with (
connector = 'kafka',
topic = 'xxxxxxx',
properties.group.id = 'xxxxxxxxxxxxx',
properties.bootstrap.server = 'alikafka-pre-cn-xxxxxxxxxx-1-vpc.alikafka.aliyuncs.com:9094',
scan.startup.mode = 'latest',
enable.auto.commit = 'true',
properties.security.protocol = 'SASL_PLAINTEXT',
properties.sasl.mechanism = 'SCRAM-SHA-256',
properties.sasl.username = 'xxxxxxxxx',
properties.sasl.password = 'xxxxxxxxxxxxxxx',
properties.sasl.jaas.config = 'org.apache.kafka.common.security.scram.ScramLoginModule required username="xxxxxxx" password="xxxxxxxxxxxxx";',
properties.session.timeout.ms = '240000',
properties.request.timeout.ms = '240000'
) FORMAT PLAIN ENCODE JSON;
Could the issue be related to the configuration format or is there something else I'm missing?
Michael Li
Asked on Jan 03, 2024
It seems like the configuration for SASL authentication might not be set correctly for Alibaba Kafka in RisingWave. Here's a configuration that was suggested to work:
create source s with (
connector = 'kafka',
topic = 'test',
properties.bootstrap.server = 'endpoint',
properties.security.protocol = 'SASL_SSL',
properties.sasl.mechanism = 'PLAIN',
properties.sasl.username = '<user>',
properties.sasl.password = '<password>',
properties.ssl.ca.location = '/Users/wutao/only-4096-ca-cert'
) row format json;
Additionally, it was mentioned that the properties.sasl.jaas.config
is not supported by RisingWave. If you are using a VPC endpoint to access Kafka, you should set the mechanism to 'PLAIN'. Also, ensure that your API key has access to the topic. If the issue persists, consider testing with a direct Rust implementation using rust-rdkafka to verify if the problem is specific to Alibaba Kafka or RisingWave.