I'm trying to stream Kafka Avro data into RisingWave, but I'm encountering an error related to Avro Union types. The error message is:
ERROR: QueryError: internal error: unsupported type in Avro: Union(UnionSchema { schemas: [Null, String], variant_index: {Null: 0, String: 1} })It seems that the issue might be related to how Null variants are handled. The schema causing the issue looks like this:
{
"name": "environment",
"type": [
"null",
{
"type": "string",
"avro.java.string": "String"
}
],
"default": null
}How can I resolve this error to successfully stream my data?
Cameron Braid
Asked on Feb 18, 2023
This bug has been fixed and the fix will be included in the next release 0.1.17. You can try the latest build or build from the source code before the release is published. Here's how you can access the latest build:
ghcr.io/risingwavelabs/risingwave:latesthttp://risingwave-nightly-pre-built-binary.s3-website.us-east-2.amazonaws.com/If you encounter any issues with the :latest tag, such as pods crashing, make sure to set imagePullPolicy: Always in your Kubernetes deployment to ensure the latest image is pulled. If the compute node crashes due to memory issues, you may need to adjust the memory configurations or increase the resources allocated to the pod.