How to configure RisingWave to connect to RedPanda with TLS on Kubernetes?
I'm having trouble configuring RisingWave to connect to a RedPanda cluster with TLS enabled on Kubernetes. I can use rpk
to interact with RedPanda using the provided certificates, but when I try to set up RisingWave as either a source or sink, it fails with SSL handshake errors or connection resets. Here's an example of my source configuration:
with (
connector='kafka',
properties.bootstrap.server = 'redpanda-0.redpanda.redpanda.svc.cluster.local:9093,redpanda-1.redpanda.redpanda.svc.cluster.local:9093,redpanda-2.redpanda.redpanda.svc.cluster.local:9093',
topic = 'test-source',
scan.startup.mode='latest',
properties.client.id='data_rw',
properties.security.protocol='SSL',
properties.ssl.ca.location='/external-certs/redpanda/ca.crt',
properties.ssl.certificate.location='/external-certs/redpanda/tls.crt',
properties.ssl.key.location='/external-certs/redpanda/tls.key'
) FORMAT PLAIN ENCODE BYTES;```
The error I receive is related to SSL certificate verification. I've tried placing the certificate files in `/etc/ssl/certs` but then I encounter a timeout with connection reset. Any suggestions on how to resolve this?
Neil
Asked on Dec 13, 2023
It seems like the issue might be related to SSL endpoint identification. A potential solution is to use a RisingWave image that allows configuration of the ssl.endpoint.identification.algorithm
property. You can try pulling the image with the following command:
docker pull ghcr.io/risingwavelabs/risingwave:ssl-endpoint-identification-algorithm
Then, set properties.ssl.endpoint.identification.algorithm = 'none'
in the WITH
options of your source or sink configuration. This change might resolve the SSL handshake issues. Additionally, for the auto topic creation issue, ensure that the properties.allow.auto.create.topics
is set correctly, and if it still doesn't work, creating the topic manually might be necessary.