From ceeff522292a98d861927e96c2b2c2d069a29973 Mon Sep 17 00:00:00 2001 From: Zhengchao An Date: Wed, 2 Sep 2026 08:08:37 +0800 Subject: [PATCH] docs(swift): align README feature lists with router and handler wiring (#7033) --- crates/protocols/src/swift/README.md | 152 ++++++++++++++++++++------- 1 file changed, 115 insertions(+), 37 deletions(-) diff --git a/crates/protocols/src/swift/README.md b/crates/protocols/src/swift/README.md index bc56fa8c8..fa16f11cd 100644 --- a/crates/protocols/src/swift/README.md +++ b/crates/protocols/src/swift/README.md @@ -4,23 +4,78 @@ Swift-compatible object storage API implementation for RustFS. ## Features -This implementation provides **Phase 1 Swift API support** (~25% of full Swift API): +The lists below are bounded to what `router.rs` / `handler.rs` dispatch and +what the test suite exercises. A module existing under `src/swift/` does not +by itself mean the feature is reachable over HTTP. -- ✅ Container CRUD operations (create, list, delete, metadata) -- ✅ Object CRUD with streaming downloads (upload, get, head, delete) -- ✅ Keystone token authentication -- ✅ Multi-tenant isolation with secure SHA256-based bucket prefixing -- ✅ Server-side object copy (COPY method) -- ✅ HTTP Range requests for partial downloads (206, 416 responses) -- ✅ Custom metadata support (X-Object-Meta-*, X-Container-Meta-*) +### Wired through the router and handler -**Not yet implemented:** -- ⏳ Account-level operations (statistics, metadata) -- ⏳ Large object support (multi-part uploads >5GB) -- ⏳ Object versioning -- ⏳ Container ACLs and CORS -- ⏳ Temporary URLs (TempURL) -- ⏳ XML/plain-text response formats (JSON only) +- ✅ Account listing (`GET /v1/AUTH_{project}`, JSON) and additive account + metadata updates (`POST`, `X-Account-Meta-*` / `X-Remove-Account-Meta-*`) +- ✅ Container CRUD (create, list, head, update metadata, delete) +- ✅ Object CRUD with streaming downloads, HTTP Range requests (206 / 416), + and server-side copy via the `COPY` method +- ✅ Keystone token authentication and multi-tenant isolation with + SHA256-based bucket prefixing +- ✅ Custom metadata (`X-Object-Meta-*`, `X-Container-Meta-*`); container and + account POSTs are additive, object POSTs replace the set +- ✅ Container ACLs (`X-Container-Read` / `X-Container-Write`, set and remove on + container POST, reported on HEAD). Enforcement is account-level plus + referrer checks; per-user grants are not evaluated because credentials + carry no user id +- ✅ CORS: `OPTIONS` preflight on container and object routes, and response + header injection driven by `X-Container-Meta-Access-Control-*` +- ✅ TempURL (`temp_url_sig` / `temp_url_expires` on object GET, HEAD, PUT; + key stored as account metadata; optional client-IP restriction) +- ✅ FormPost (container POST with `multipart/form-data`, signed with the + account TempURL key) +- ✅ Large objects: Static Large Objects (`?multipart-manifest=put|get|delete`) + and Dynamic Large Objects (`X-Object-Manifest`) +- ✅ Bulk operations: `DELETE /v1/AUTH_{project}?bulk-delete` and + `PUT /v1/AUTH_{project}/{container}?extract-archive=tar|tar.gz|tar.bz2` +- ✅ Object versioning in the Swift `X-Versions-Location` style: the previous + copy is archived on PUT / DELETE and restored on DELETE +- ✅ Symlinks (`X-Symlink-Target` on PUT, resolved on GET / HEAD with loop and + depth checks) +- ✅ Container quotas (`X-Container-Meta-Quota-Bytes` / `-Quota-Count`), + enforced on object PUT +- ✅ Static website serving on object GET when `web-index` / `web-listings` + container metadata is set +- ✅ Object expiration headers: `X-Delete-At` / `X-Delete-After` are validated, + stored, and returned on GET / HEAD + +### Not yet wired, or partially wired + +- ⏳ Account `HEAD` returns `501 Not Implemented`; no account-level usage + statistics are exposed +- ⏳ Automatic deletion of expired objects: `expiration_worker.rs` exists but + the server never starts it, so objects past `X-Delete-At` are not removed +- ⏳ Container sync (`sync.rs`): no `X-Container-Sync-*` header handling and no + background worker; the module is unit-tested only +- ⏳ `X-Copy-From` on object PUT (only the `COPY` method is supported) +- ⏳ `X-History-Location` versioning mode +- ⏳ Static website index / listing pages at the container root (only the + object GET route consults static-web settings) +- ⏳ XML / plain-text listing formats; the `format=` query parameter is + ignored and listings are always JSON + +### Test coverage + +- Unit tests live next to each module (`acl.rs`, `bulk.rs`, `cors.rs`, + `dlo.rs`, `slo.rs`, `tempurl.rs`, `formpost.rs`, `staticweb.rs`, + `symlink.rs`, `quota.rs`, `expiration.rs`, `versioning.rs`, `router.rs`, + `handler.rs`, and others) and run in the CI `swift` feature lane +- `crates/protocols/tests/swift_metadata_persistence.rs` runs account, + container, ACL, TempURL-key, and versioning metadata writes against a real + ECStore and reloads them from disk +- `crates/protocols/tests/swift_versioning_integration.rs`, + `swift_listing_symlink_tests.rs`, `swift_simple_integration.rs`, and + `swift_phase4_integration.rs` cover version naming, listing parameters, + symlink parsing, and module-level helpers without a server +- `rustfs/tests/swift_container_integration_test.rs` and + `swift_object_integration_test.rs` exercise the HTTP surface end to end but + are `#[ignore]` and need a running server (`TEST_RUSTFS_SERVER`); they are + not part of CI ## Enable Feature @@ -42,38 +97,52 @@ cargo build --features full ## Configuration -Swift API uses Keystone for authentication. Configure the following environment variables: +Swift API uses Keystone for authentication. The variables below are read by +`crates/keystone/src/config.rs`; that file is the authoritative list. | Variable | Description | |----------|-------------| -| `RUSTFS_KEYSTONE_URL` | Keystone authentication endpoint URL | -| `RUSTFS_KEYSTONE_ADMIN_TENANT` | Admin tenant/project name | -| `RUSTFS_KEYSTONE_ADMIN_USER` | Admin username | -| `RUSTFS_KEYSTONE_ADMIN_PASSWORD` | Admin password | +| `RUSTFS_KEYSTONE_ENABLE` | Set to `true` to enable Keystone authentication (default `false`; nothing else is read while disabled) | +| `RUSTFS_KEYSTONE_AUTH_URL` | Keystone authentication endpoint URL (required once enabled) | +| `RUSTFS_KEYSTONE_VERSION` | Keystone API version, `v3` or `v2.0` (default `v3`) | +| `RUSTFS_KEYSTONE_ADMIN_USER` | Admin username (optional) | +| `RUSTFS_KEYSTONE_ADMIN_PASSWORD` | Admin password (optional) | +| `RUSTFS_KEYSTONE_ADMIN_PROJECT` | Admin project name (optional) | +| `RUSTFS_KEYSTONE_ADMIN_DOMAIN` | Admin domain name (optional) | +| `RUSTFS_KEYSTONE_VERIFY_SSL` | Verify the Keystone TLS certificate (default `true`) | +| `RUSTFS_KEYSTONE_ENABLE_CACHE` / `RUSTFS_KEYSTONE_CACHE_SIZE` / `RUSTFS_KEYSTONE_CACHE_TTL` | Token cache toggle, entry count, and TTL in seconds (defaults `true`, `10000`, `300`) | +| `RUSTFS_KEYSTONE_TENANT_PREFIX` | Prefix bucket names with the tenant hash (default `true`) | +| `RUSTFS_KEYSTONE_IMPLICIT_TENANTS` | Allow implicit tenant creation (default `true`) | +| `RUSTFS_KEYSTONE_TIMEOUT` | Keystone request timeout in seconds (default `30`) | ## API Endpoints Swift API endpoints follow the pattern: `/v1/AUTH_{project_id}/...` ### Account Operations -- `GET /v1/AUTH_{project}` - List containers -- `HEAD /v1/AUTH_{project}` - Get account metadata (not yet implemented) -- `POST /v1/AUTH_{project}` - Update account metadata (not yet implemented) +- `GET /v1/AUTH_{project}` - List containers (JSON) +- `HEAD /v1/AUTH_{project}` - Get account metadata (returns 501, not yet implemented) +- `POST /v1/AUTH_{project}` - Update account metadata and TempURL key +- `DELETE /v1/AUTH_{project}?bulk-delete` - Bulk delete ### Container Operations -- `PUT /v1/AUTH_{project}/{container}` - Create container -- `GET /v1/AUTH_{project}/{container}` - List objects +- `PUT /v1/AUTH_{project}/{container}` - Create container (`?extract-archive=` for bulk upload) +- `GET /v1/AUTH_{project}/{container}` - List objects (JSON; `limit`, `marker`, `end_marker`, `prefix`, `delimiter`) - `HEAD /v1/AUTH_{project}/{container}` - Get container metadata -- `POST /v1/AUTH_{project}/{container}` - Update container metadata +- `POST /v1/AUTH_{project}/{container}` - Update container metadata, ACLs, versioning location; FormPost when `multipart/form-data` - `DELETE /v1/AUTH_{project}/{container}` - Delete container +- `OPTIONS /v1/AUTH_{project}/{container}` - CORS preflight ### Object Operations -- `PUT /v1/AUTH_{project}/{container}/{object}` - Upload object -- `GET /v1/AUTH_{project}/{container}/{object}` - Download object +- `PUT /v1/AUTH_{project}/{container}/{object}` - Upload object (SLO manifest with `?multipart-manifest=put`, DLO with `X-Object-Manifest`, symlink with `X-Symlink-Target`) +- `GET /v1/AUTH_{project}/{container}/{object}` - Download object (Range, SLO/DLO assembly, symlink resolution, `?multipart-manifest=get`) - `HEAD /v1/AUTH_{project}/{container}/{object}` - Get object metadata - `POST /v1/AUTH_{project}/{container}/{object}` - Update object metadata -- `DELETE /v1/AUTH_{project}/{container}/{object}` - Delete object +- `DELETE /v1/AUTH_{project}/{container}/{object}` - Delete object (`?multipart-manifest=delete` removes SLO segments) - `COPY /v1/AUTH_{project}/{container}/{object}` - Server-side copy +- `OPTIONS /v1/AUTH_{project}/{container}/{object}` - CORS preflight + +Object GET, HEAD, and PUT also accept TempURL query parameters without an auth token. ## Architecture @@ -96,11 +165,18 @@ Handler (fallback) ### Key Components -- **handler.rs** - Main service implementing Tower's Service trait +- **handler.rs** - Main service implementing Tower's Service trait and method dispatch - **router.rs** - URL routing and parsing for Swift paths - **container.rs** - Container operations with tenant isolation - **object.rs** - Object operations including copy and range requests -- **account.rs** - Account validation and tenant access control +- **account.rs** - Account validation, tenant access control, account metadata and TempURL key +- **acl.rs**, **cors.rs** - Container ACL evaluation and CORS config +- **slo.rs**, **dlo.rs** - Static and dynamic large objects +- **tempurl.rs**, **formpost.rs** - Signed URL and form upload validation +- **bulk.rs** - Bulk delete and archive extraction +- **versioning.rs**, **symlink.rs**, **quota.rs**, **staticweb.rs**, **expiration.rs** - Per-feature helpers called from the handler +- **expiration_worker.rs**, **sync.rs** - Background workers that are not started by the server (see above) +- **metadata_update.rs** - Additive account/container metadata merge - **errors.rs** - Swift-specific error types - **types.rs** - Data structures for Swift API responses @@ -121,13 +197,15 @@ This ensures: ## Documentation -See the `docs/` directory for detailed documentation: +There is no separate Swift reference document in the repository. Use these +sources instead: -- `SWIFT_API.md` - Complete API reference -- `TESTING_GUIDE.md` - Manual testing procedures -- `COMPLETION_ANALYSIS.md` - Protocol coverage tracking -- `COPY_IMPLEMENTATION.md` - Server-side copy documentation -- `RANGE_REQUESTS.md` - Range request implementation details +- Module-level `//!` comments in each `crates/protocols/src/swift/*.rs` file + describe the headers and metadata keys that feature reads +- `crates/protocols/tests/swift_*.rs` and `rustfs/tests/swift_*_integration_test.rs` + show the expected request and response shapes +- `docs/testing/ci-gates.md` describes the CI lane that builds and tests with + `--features swift` ## License