mirror of
https://github.com/rustfs/rustfs.git
synced 2026-08-02 11:29:17 +00:00
Zhengchao An
957080bea5
Swift container and account metadata handlers cloned the cached BucketMetadata, set the tagging fields, and called set_bucket_metadata, which only updates the in-memory cache map. Nothing reached .metadata.bin, so every Swift metadata POST was lost on restart and silently overwritten by the next disk-truth reload (a peer LoadBucketMetadata notification or the 15-minute refresh loop) — while the client had already been told 2xx. Route these writes through a new metadata_sys::update_config_with: a read-modify-write that loads the on-disk metadata and persists the result under the same write guard metadata_sys::update uses, so the rewrite merges against disk truth instead of a possibly stale cache and cannot clobber a concurrent update to another config file. Peers are notified afterwards, matching the S3 config handlers. Persisting these writes required hardening the paths that now produce durable state: - Account metadata writes validate account ownership. This metadata holds the account's TempURL signing key, so an unauthenticated write for someone else's account would have become a durable, cluster-wide takeover of that account's pre-signed URLs. Reads stay open because TempURL signature validation runs before credentials exist. - disable_versioning verifies the container exists. Without it the metadata loader's "no metadata on disk" default would be persisted, creating an orphan metadata file and caching a fabricated default as authoritative. - Container and account metadata are size- and count-limited, reusing the Swift limits object metadata already enforces; these tags land in the bucket metadata file that every later config write rewrites whole. - A rewrite refuses to run when the persisted tagging config is unreadable, instead of merging onto an empty set and wiping the container ACL and versioning tags. It reports 409 naming the remedy. - Storage errors are logged in full and reported generically, since they now carry real disk and quorum detail. The tagging arm of BucketMetadata::update_config also clears the parsed config, as the lifecycle arm does: parse_all_configs skips empty XML rather than clearing, so a cleared config kept serving the old tags. Tagging is serialized with the S3 XML serializer the loader can parse back, not quick_xml, whose output was never round-trippable.
OpenStack Swift API for RustFS
Swift-compatible object storage API implementation for RustFS.
Features
This implementation provides Phase 1 Swift API support (~25% of full Swift API):
- ✅ Container CRUD operations (create, list, delete, metadata)
- ✅ Object CRUD with streaming downloads (upload, get, head, delete)
- ✅ Keystone token authentication
- ✅ Multi-tenant isolation with secure SHA256-based bucket prefixing
- ✅ Server-side object copy (COPY method)
- ✅ HTTP Range requests for partial downloads (206, 416 responses)
- ✅ Custom metadata support (X-Object-Meta-, X-Container-Meta-)
Not yet implemented:
- ⏳ Account-level operations (statistics, metadata)
- ⏳ Large object support (multi-part uploads >5GB)
- ⏳ Object versioning
- ⏳ Container ACLs and CORS
- ⏳ Temporary URLs (TempURL)
- ⏳ XML/plain-text response formats (JSON only)
Enable Feature
Swift API is opt-in and must be explicitly enabled.
Build with Swift support:
cargo build --features swift
Or enable all protocol features:
cargo build --features full
Note: Swift is NOT enabled by default to avoid unexpected API surface changes in existing deployments.
Configuration
Swift API uses Keystone for authentication. Configure the following environment variables:
| Variable | Description |
|---|---|
RUSTFS_KEYSTONE_URL |
Keystone authentication endpoint URL |
RUSTFS_KEYSTONE_ADMIN_TENANT |
Admin tenant/project name |
RUSTFS_KEYSTONE_ADMIN_USER |
Admin username |
RUSTFS_KEYSTONE_ADMIN_PASSWORD |
Admin password |
API Endpoints
Swift API endpoints follow the pattern: /v1/AUTH_{project_id}/...
Account Operations
GET /v1/AUTH_{project}- List containersHEAD /v1/AUTH_{project}- Get account metadata (not yet implemented)POST /v1/AUTH_{project}- Update account metadata (not yet implemented)
Container Operations
PUT /v1/AUTH_{project}/{container}- Create containerGET /v1/AUTH_{project}/{container}- List objectsHEAD /v1/AUTH_{project}/{container}- Get container metadataPOST /v1/AUTH_{project}/{container}- Update container metadataDELETE /v1/AUTH_{project}/{container}- Delete container
Object Operations
PUT /v1/AUTH_{project}/{container}/{object}- Upload objectGET /v1/AUTH_{project}/{container}/{object}- Download objectHEAD /v1/AUTH_{project}/{container}/{object}- Get object metadataPOST /v1/AUTH_{project}/{container}/{object}- Update object metadataDELETE /v1/AUTH_{project}/{container}/{object}- Delete objectCOPY /v1/AUTH_{project}/{container}/{object}- Server-side copy
Architecture
The Swift API is implemented as a Tower service layer (SwiftService) that wraps the S3 service:
HTTP Request
│
▼
┌───────────────┐
│ SwiftService │ ← Routes /v1/AUTH_* requests
└───────┬───────┘
│
┌────┴────┐
│ │
▼ ▼
Swift S3 Service
Handler (fallback)
Key Components
- handler.rs - Main service implementing Tower's Service trait
- router.rs - URL routing and parsing for Swift paths
- container.rs - Container operations with tenant isolation
- object.rs - Object operations including copy and range requests
- account.rs - Account validation and tenant access control
- errors.rs - Swift-specific error types
- types.rs - Data structures for Swift API responses
Tenant Isolation
Swift containers are mapped to S3 buckets with a secure hash prefix:
Swift: /v1/AUTH_abc123/mycontainer
↓
S3 Bucket: {sha256(abc123)[0:16]}-mycontainer
This ensures:
- Complete tenant isolation at the storage layer
- No collision between tenants with similar container names
- S3-compatible bucket naming (lowercase alphanumeric + hyphen)
Documentation
See the docs/ directory for detailed documentation:
SWIFT_API.md- Complete API referenceTESTING_GUIDE.md- Manual testing proceduresCOMPLETION_ANALYSIS.md- Protocol coverage trackingCOPY_IMPLEMENTATION.md- Server-side copy documentationRANGE_REQUESTS.md- Range request implementation details
License
Apache License 2.0