gsadmin/rustfs
1
0
Fork 0
mirror of https://github.com/rustfs/rustfs.git synced 2026-08-09 22:59:59 +00:00
Code Issues Packages Projects Releases Wiki Activity

test(table-catalog): add vendor compatibility audit (#4299)

Browse Source 
* test(table-catalog): add vendor compatibility audit

* fix(table-catalog): include OSS data-plane endpoint

---------

Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
This commit is contained in:
Henry Guo
2026-07-06 14:12:19 +08:00
committed by GitHub
parent b09526c0f5
commit be52e35a1f
6 changed files with 393 additions and 13 deletions
Download Patch File Download Diff File Expand all files Collapse all files
scripts/table-catalog/README.md
+28 -6
View File
@@ -150,11 +150,20 @@ the `selected_vendor_profile` object renders the active profile with the
provided endpoint, account, bucket, and warehouse arguments.
Spark vendor config generation only emits S3-compatible data-plane endpoint,
path-style, and static S3 credential properties for profiles that explicitly use
the supplied endpoint as object storage, such as RustFS and MinIO AIStor. AWS S3
Tables, Cloudflare R2 Data Catalog, and OSS Tables reference profiles leave
provider object I/O endpoint and credential selection to the provider-specific
Spark/Iceberg runtime configuration instead of inheriting RustFS local defaults.
path-style, and static S3 credential properties for profiles that require them.
RustFS and MinIO AIStor use the supplied endpoint as object storage. Alibaba
OSS Tables uses the documented `https://oss-{region}.aliyuncs.com` public
S3FileIO endpoint by default. AWS S3 Tables and Cloudflare R2 Data Catalog
reference profiles leave provider object I/O endpoint and credential selection
to the provider-specific Spark/Iceberg runtime configuration instead of
inheriting RustFS local defaults.
The standalone engine helper also prints a vendor compatibility audit. This
records the public reference source, catalog path, warehouse shape, signing
name, auth model, required validation categories, and explicit not-claimed
boundaries for AWS S3 Tables, MinIO AIStor Tables, Cloudflare R2 Data Catalog,
and Alibaba OSS Tables. Use it to plan compatibility work; do not treat it as
live interoperability evidence.
Generate an AWS S3 Tables-style reference profile:
@@ -177,11 +186,24 @@ python3 scripts/table-catalog/pyiceberg_smoke.py \
--print-vendor-profiles
```
Generate an Alibaba OSS Tables-style reference profile:
```bash
python3 scripts/table-catalog/pyiceberg_smoke.py \
--profile oss-tables \
--endpoint https://cn-hangzhou.oss-tables.aliyuncs.com \
--region cn-hangzhou \
--account-id 123456789012 \
--table-bucket analytics \
--print-vendor-profiles
```
The standalone engine helper prints the same compatibility matrix and can also
generate Spark REST catalog input without importing PyIceberg:
```bash
python3 scripts/table-catalog/engine_compatibility.py --print-engine-matrix
python3 scripts/table-catalog/engine_compatibility.py --print-vendor-audit
python3 scripts/table-catalog/engine_compatibility.py --print-spark-config
python3 scripts/table-catalog/engine_compatibility.py \
--profile aws-s3tables \
@@ -352,7 +374,7 @@ expected status, observed status, and metadata location.
| `aws-s3tables` | `https://s3tables.{region}.amazonaws.com/iceberg` | `arn:aws:s3tables:{region}:{account_id}:bucket/{table_bucket}` | `s3tables` | profile generator only; full AWS S3 Tables API parity is not claimed |
| `minio-aistor` | `{endpoint}/_iceberg` | `{warehouse}` | `s3tables` | profile generator plus RustFS alias smoke; full AIStor extension parity is not claimed |
| `cloudflare-r2-data-catalog` | catalog URI returned by R2 | `{warehouse_name}` | `s3` | profile generator only; live RustFS interoperability is not claimed |
| `oss-tables` | provider REST endpoint | `{warehouse}` | `s3` | profile generator only; live RustFS interoperability is not claimed |
| `oss-tables` | provider REST endpoint | `acs:osstables:{region}:{account_id}:bucket/{table_bucket}` | `osstables` | profile generator only; live RustFS interoperability is not claimed |
## Unsupported Inventory