test(table-catalog): verify vended credential data-plane scope (#3429)

This commit is contained in:
Henry Guo
2026-06-14 16:35:54 +08:00
committed by GitHub
parent 5e68cf8a29
commit fc17e75fb2
3 changed files with 471 additions and 26 deletions
+43 -4
View File
@@ -69,6 +69,38 @@ python3 scripts/table-catalog/pyiceberg_smoke.py \
--rest-signing-name s3
```
To verify catalog-vended table credentials, enable server-side credential
vending and use the vended credential profile:
```text
RUSTFS_TABLE_CATALOG_CREDENTIAL_VENDING=enabled
```
```bash
python3 scripts/table-catalog/pyiceberg_smoke.py \
--profile rustfs-vended-credentials \
--endpoint http://127.0.0.1:9000 \
--access-key rustfsadmin \
--secret-key rustfsadmin \
--bucket rustfs-s3table-smoke \
--replace \
--cleanup
```
This profile uses the configured principal to create the bucket, enable the
table bucket, and create the table. After the table exists, it calls the REST
credentials endpoint and reloads the PyIceberg catalog with the returned
table-scoped S3 access key, secret key, and session token before append, reload,
and scan operations.
Before the PyIceberg append, the profile also checks that the returned
credential prefix exactly matches the created table warehouse location, then
runs a direct S3 data-plane scope probe with the returned temporary credentials:
- `PutObject`, `HeadObject`, and `DeleteObject` must work inside the returned
table warehouse prefix.
- `PutObject` to the same bucket outside that prefix must be rejected.
## Machine-Readable Inventories
The script can print the current conformance inventories without importing
@@ -89,7 +121,7 @@ added.
| Client | Current status | Claim |
|---|---|---|
| PyIceberg | Automated smoke target | create namespace, create table, append, reload, scan |
| PyIceberg | Automated smoke target | create namespace, create table, append, reload, scan, optional catalog-vended table credentials with exact-prefix data-plane scope probe |
| Spark Iceberg REST catalog | Manual-ready | create/load/append/reload should be verified against a running RustFS endpoint |
| Trino Iceberg REST catalog | Documented, not automated | no write compatibility claim yet |
| DuckDB Iceberg | Documented, not automated | read-path reference only |
@@ -102,6 +134,7 @@ added.
|---|---|---|---|---|
| `rustfs` | `{endpoint}/iceberg` | `s3` | static S3 credentials | automated smoke target |
| `rustfs-compat` | `{endpoint}/_iceberg` | `s3tables` by default | static S3 credentials | compatibility smoke target |
| `rustfs-vended-credentials` | `{endpoint}/iceberg` | `s3` | catalog-vended table credentials after table creation | automated credential smoke target when server vending is enabled |
| `aws-s3tables` | `https://s3tables.{region}.amazonaws.com/iceberg` | `s3tables` | AWS IAM/session credentials | reference only |
| `minio-aistor` | `{endpoint}/_iceberg` | `s3tables` | policy-scoped S3 credentials | reference only |
| `cloudflare-r2-data-catalog` | catalog URI returned by R2 | `s3` | catalog-vended credentials | reference only |
@@ -112,7 +145,7 @@ added.
Unsupported behavior is documented instead of hidden behind internal errors. The
current unsupported inventory is:
- credential vending: table scope preview and credentials endpoint exist; temporary credentials are available only when explicitly enabled and are not yet covered by automated client profiles
- credential vending: automated after table bootstrap with exact-prefix validation and a data-plane scope probe; full no-long-term-data-credential bootstrap is not claimed
- background maintenance worker: unsupported
- manifest/data reachability cleanup: unsupported
- snapshot expiration and compaction: unsupported
@@ -133,8 +166,14 @@ The endpoint returns an empty `storage-credentials` list unless table catalog
credential vending is explicitly enabled. When enabled, RustFS issues temporary
table-scoped S3 credentials through the credentials endpoint. Those credentials
are constrained to the table warehouse prefix and include a session token and
expiration. The automated smoke profiles still use configured S3 credentials for
object data access until a catalog-vended credential profile is added.
expiration.
The `rustfs-vended-credentials` profile verifies the client handoff from the
catalog principal to the table-scoped temporary credentials. It still uses the
configured principal for setup and REST request signing; the vended credentials
are first checked against the created table warehouse location, then checked
with a direct S3 scope probe, and finally applied to PyIceberg S3 data-plane
access after the table has been created.
Enablement is server-side and fail-closed: