I'm trying to set up an Iceberg sink in RisingWave using the REST catalog, but I'm encountering an error message that says Can't encode empty namespace in url
. Here's the SQL command I used to create the sink:
CREATE SINK s1_sink FROM s1_source
WITH (
connector='iceberg',
s3.region = 'us-west-2',
database.name = 'iceberg_db',
table.name = 'risingwave_test',
catalog.type='rest',
catalog.uri='<http://iceberg-rest.iceberg-rest-catalog.svc.cluster.local:8181>',
type = 'append-only', warehouse.path='<s3://blahblah/blahblah>', s3.access.key ='', s3.secret.key=''
);
I followed the example from the RisingWave documentation, created the source as described, and made an empty Iceberg table outside of RisingWave. However, I'm unsure about some of the properties in the command, such as warehouse.path
, which according to the docs, shouldn't be necessary for REST catalog. Additionally, I'm using AWS Glue as the backend for the REST Catalog and have IRSA set up for the RisingWave ServiceAccount with the correct S3 permissions. Can I use that instead of specifying access_key
and secret_key
?
Neil
Asked on Dec 05, 2023
To resolve the 'Can't encode empty namespace in url' error, you need to include the database name in the table.name
property when creating the Iceberg sink. The correct SQL command should look like this:
CREATE SINK s1_sink FROM s1_source
WITH (
connector='iceberg',
s3.region = 'us-west-2',
database.name = 'iceberg_db',
table.name = 'iceberg_db.risingwave_test',
catalog.type='rest',
catalog.uri='<http://iceberg-rest.iceberg-rest-catalog.svc.cluster.local:8181>',
type = 'append-only', warehouse.path='<s3://your-actual-warehouse-path>', s3.access.key='your-access-key', s3.secret.key='your-secret-key'
);
As for the S3 connection details, currently, RisingWave only supports access_key
and secret_key
. However, if you are using EKS, you can access S3 with an IAM role. Please refer to the RisingWave documentation on AWS EKS IAM for S3 for more details.