mirror of
https://github.com/rustfs/rustfs.git
synced 2026-08-02 11:29:17 +00:00
Zhengchao An
9644064e57
* docs(kms): add the object-side bulk rekey job contract Define the design contract for the bulk DEK re-wrap job before any of it is built: work unit granularity, idempotency model, failure semantics, the exclusion list, the metadata write constraints, and the ownership model. The job re-wraps envelopes only and never rewrites object bodies, and it never destroys a superseded key version: a half-finished run is a fully serviceable state precisely because the old version still decrypts, so folding destruction into the job would turn a resumable action into irreversible loss on partial failure. Records two positions that diverge from the originating request. Pause is not provided; cancel plus cursor restart plus rate control covers what pause is actually asked for, and none of the seven existing long-job frameworks has a pause state. And the KEK version a given envelope was sealed under is not observable today, from object metadata or from the decrypt response, so recognizing the target state requires the re-wrap primitive to record a version witness - stated here as the one interface both sides must agree on. Refs rustfs/backlog#1642 (part of rustfs/backlog#1562) * docs(kms): correct how the wrapping KEK version is recovered The first draft claimed the wrapping KEK version was not observable at all, and built the idempotent-skip and completion-evidence conclusions on that. It is observable for every backend that rotates, so that framing was wrong. The sealed blob under x-rustfs-encryption-key is the base64 of the backend ciphertext, which is DataKeyEnvelope JSON; the read path in sse.rs already discriminates on it via is_data_key_envelope. Vault KV2 records the version in the envelope's master_key_version, and Transit leaves that field None on purpose because its ciphertext self-describes as vault:vN:. Local and Static hardcode None because neither rotates. Only AWS is genuinely opaque. So the skip is achievable, and the honest statement is its cost: a base64 decode plus a JSON parse per scanned work unit, which belongs in the rate budget. AWS becomes a scope-admission refusal instead of a silent every-object rewrite on re-run. Two traps replace the old over-broad claim. A bare None means three different things across backends, so version extraction must be dispatched by backend. And Local omits the version precisely because rotation is rejected there, which couples it to backlog#1565: whichever change gives Local a rotation history must start recording the version in the same change, or Local joins AWS in the unreadable column. The requirement left on the re-wrap primitive shrinks accordingly, from "record a version" to exposing one backend-dispatched accessor and reporting "already at target state" as a distinct outcome. Refs rustfs/backlog#1642 (part of rustfs/backlog#1562)
Architecture Documentation
Durable architecture reference for RustFS: migration guardrails, runtime contracts, boundary rules, and support matrices.
Two rules keep this directory healthy:
- Durable reference only. One-shot implementation plans, task trackers, and PR templates do not belong in the repository — keep them in the issue tracker or your local worktree. When their work closes, delete them rather than archiving them here.
- No copies of other sources of truth. Crate lists come from
Cargo.toml, CI steps from.github/workflows/ci.yml, code structure from the code.scripts/check_doc_paths.shfails the pre-commit gate when a doc here references a file path that no longer exists.
Start here
- overview.md — migration baseline, phase order, core principles
CI-enforced core (required by scripts/check_architecture_migration_rules.sh)
- crate-boundaries.md — dependency direction, PR types, re-export contracts
- runtime-lifecycle.md — startup/shutdown sequencing, readiness guarantees
- readiness-matrix.md — request/dependency behavior, probe semantics
- storage-control-data-plane.md — storage API contracts, control-plane boundaries
- global-state-crate-split-plan.md — remaining global-state owners and split evaluation
- ecstore-module-split-plan.md — ECStore decomposition rules and facade contracts
Contracts & invariants
- erasure-coding.md — normative erasure-coding algorithm and on-disk (
xl.meta) compatibility contract; the frozen invariants for all user-data read/write, encode/decode, quorum, heal, and decode tolerance - placement-repair-invariants.md
- unified-object-generation.md — single per-object generation authority (fencing epoch, transport/encoding/proto/mixed-version contracts)
- runtime-capability-contracts.md
- workload-admission-contracts.md
- background-controller-contract.md
- config-model-boundary-adr.md
- ecstore-layout-boundary.md
- decommission-compatibility.md
- kms-bulk-rekey-contract.md — object-side DEK re-wrap job: work unit, idempotency model, exclusion rules, and the never-destroy-old-key-versions constraint
Support matrices (release-facing, keep current)
- s3-compatibility-matrix.md
- s3-tables-support-matrix.md
- minio-rustfs-router-compatibility.md
- minio-file-format-compat.md
Inventories & baselines (snapshots that feed migration work)
- global-state-inventory.md
- ecstore-api-facade-inventory.md
- ecstore-config-consumer-inventory.md
- obs-ecstore-dependency-inventory.md
- background-services-inventory.md
- admin-route-action-snapshot.md
- compat-cleanup-register.md
Historical plans and trackers (rebalance/decommission phases, migration-progress ledger, and the one-shot migration snapshots that fed it — startup timeline, scheduler baseline, profiling/NUMA capability inventory, KMS development defaults inventory) were retired in 2026-07 once the architecture-review ledger they served closed out (backlog#660/#665). Planning documents are no longer kept in the repository.