troubleshooting

How to resolve Risingwave deployment issues with Pulsar on Kubernetes?

I'm having trouble deploying Risingwave with Pulsar on Kubernetes. I'm using Risingwave version 1.7.2 and encountering errors related to gRPC requests and DNS resolution when trying to create a table with Pulsar as the source. Additionally, I'm facing issues with the Risingwave deployment using Helm and ArgoCD, where the risingwave-meta pod keeps restarting with SignatureDoesNotMatch errors related to S3 connectivity, even though I'm not using AWS S3 but have MinIO enabled. Here's the relevant part of my values.yaml for the Risingwave Helm chart:

risingwave:
  tags:
    minio: true
  image:
    tag: v1.7.2
  stateStore:
    minio:
      enabled: true
      # endpoint: "<http://wl-minio.minio.svc.cluster.local:9000>"
      # bucket: "risingwave"
      # authentication:
      #   existingSecretName: "risingwave-minio-creds"
  # Other configurations omitted for brevity

The error logs suggest an issue with S3 signature mismatch. I'm not sure if my Risingwave deployment configuration is causing the issue. Can you help me resolve these deployment problems?

At

Atiqul Islam

Asked on Mar 26, 2024

I was able to resolve the issue with the risingwave-meta pod by updating my Helm chart. I had tags.minio set to true and stateStore.minio.enabled set to true as well, which I initially thought were both associated with the Bitnami MinIO that comes with the Helm chart. However, the latter is for an independent MinIO deployment.

Since we already had MinIO deployed in our Kubernetes cluster, I updated the Helm chart to use the already deployed MinIO instead of the Bitnami MinIO. This seems to have solved the problem. Here's the updated section of the Helm chart:

risingwave:
  tags:
    minio: false
  stateStore:
    minio:
      enabled: true
      endpoint: "<http://wl-minio.minio.svc.cluster.local:9000>"
      bucket: "risingwave"
      authentication:
        existingSecretName: "risingwave-minio-creds"
  # Other configurations omitted for brevity

The risingwave-metastore now seems to be running fine, and I was able to create the Pulsar source successfully.

Mar 26, 2024Edited by