RisingWave Community - How can I resolve the issue with schema registry in RisingWave when using CREATE SOURCE with Kafka connector?

troubleshooting

How can I resolve the issue with schema registry in RisingWave when using CREATE SOURCE with Kafka connector?

I'm facing an issue where the type in the source created with RisingWave using Kafka connector is VARCHAR[] instead of JSONB[] or Struct[]. The usage of the schema.registry property seems to restrict specifying columns in the 'create source' script. Is there a way to resolve this issue without refactoring the value sent to the topic?

蕭翊

蕭翊安

Asked on Oct 20, 2023

RisingWave relies on rewriting JSON schema to Avro schema for type inference.
The array elements are defined as string in the rewritten Avro definition, causing the issue.
As a temporary workaround, you can manually define the columns to match the expected types.
Refactoring the schema definition to replace $ref with $defs can treat the array as struct<inner_field varchar, other_field varchar>[].
The RisingWave team is working on fixing this issue, so stay tuned for updates.

Oct 20, 2023Edited by