From d3f1ff36afe1cedce6a842b0b4122a4440b86a1b Mon Sep 17 00:00:00 2001 From: Zhengchao An Date: Sat, 27 Jun 2026 12:04:59 +0800 Subject: [PATCH] docs: record global state cleanup plan (#3939) --- docs/architecture/crate-boundaries.md | 3 + .../global-state-crate-split-plan.md | 84 +++++++++++++++++++ docs/architecture/migration-progress.md | 70 +++++++++++++--- docs/architecture/overview.md | 3 + scripts/check_architecture_migration_rules.sh | 29 +++++++ 5 files changed, 178 insertions(+), 11 deletions(-) create mode 100644 docs/architecture/global-state-crate-split-plan.md diff --git a/docs/architecture/crate-boundaries.md b/docs/architecture/crate-boundaries.md index fe4f45656..37ef89dd1 100644 --- a/docs/architecture/crate-boundaries.md +++ b/docs/architecture/crate-boundaries.md @@ -50,6 +50,9 @@ their required sections: Required Architecture Documents. - `docs/architecture/readiness-matrix.md`: Request Behavior Matrix, Runtime Dependency Matrix, Probe Semantics. +- `docs/architecture/global-state-crate-split-plan.md`: Remaining Global + Owners, Runtime Source Boundaries, Fallback Removal Plan, Crate Split + Evaluation. ## Pre-Push Expert Review diff --git a/docs/architecture/global-state-crate-split-plan.md b/docs/architecture/global-state-crate-split-plan.md new file mode 100644 index 000000000..84f9bdcd4 --- /dev/null +++ b/docs/architecture/global-state-crate-split-plan.md @@ -0,0 +1,84 @@ +# Global State And Crate Split Plan + +This document records the late global-state cleanup plan after the AppContext +foundation, storage API contracts, ECStore layout, runtime lifecycle, and cluster +control-plane boundaries are stable. + +## Remaining Global Owners + +| Owner | Current role | Migration stance | +|---|---|---| +| `rustfs/src/app/context.rs` | AppContext-first resolver facade with legacy fallback adapters. | Keep resolver entry points stable until every caller has an explicit context or owner-local runtime source. | +| `rustfs/src/app/context/runtime_sources.rs` | Default AppContext fallback adapters for KMS, IAM, object store, endpoints, config, metrics, and notification state. | This is an allowed fallback boundary, not a business logic owner. | +| `rustfs/src/*/runtime_sources.rs` | Root, admin, app, server, startup, and storage owner-local runtime-source boundaries. | Business modules use these boundaries instead of calling global state directly. | +| `rustfs/src/*/storage_api.rs` | Root, admin, app, and storage owner-local storage contract/facade boundaries. | Storage helper and ECStore facade access remains visible at local owner boundaries. | +| `crates/*/storage_api.rs` | External crate-local storage facade boundaries for IAM, scanner, heal, notify, observability, Swift, and S3 Select. | External runtime crates may consume ECStore facade globals only through their local storage API boundary. | +| `crates/ecstore/src/runtime/global.rs` | ECStore bootstrap/runtime state owner. | Keep internal until ECStore has explicit owner handles for all remaining bootstrap state. | +| `crates/ecstore/src/runtime/sources.rs` | ECStore runtime-source adapter over global state. | Preferred ECStore-internal access path while shrinking direct `runtime::global` reads. | + +## Runtime Source Boundaries + +Runtime-source modules are the allowed compatibility layer between migrated +consumers and process-global state. They must keep these properties: + +- context-first lookup when an `AppContext` handle exists; +- explicit fallback to the existing global only where compatibility still + requires it; +- no hidden service construction in business logic; +- no startup, readiness, IAM, KMS, lock, notification, or storage behavior + change in inventory or guardrail PRs. + +## Guarded Boundary List + +The architecture guard snapshots the files currently allowed to reference +`rustfs_ecstore::api::global` directly: + +- `crates/heal/src/heal/storage_api.rs` +- `crates/iam/src/storage_api.rs` +- `crates/notify/src/storage_api.rs` +- `crates/obs/src/metrics/storage_api.rs` +- `crates/protocols/src/swift/storage_api.rs` +- `crates/s3select-api/src/storage_api.rs` +- `crates/scanner/src/storage_api.rs` +- `crates/scanner/tests/storage_api/mod.rs` +- `rustfs/src/storage/storage_api.rs` + +New direct uses must either move behind an existing owner-local boundary or +update this plan and the guard in the same reviewed migration PR. + +## Fallback Removal Plan + +1. Keep AppContext-first plus global fallback while compatibility tests still + cover both paths. +2. Move one remaining consumer group at a time to explicit context or + owner-local runtime-source handles. +3. Remove one fallback family per PR only after scans prove no production caller + depends on it. +4. Keep embedded startup and tests working before deleting any fallback. +5. Do not remove ECStore bootstrap globals until ownership handles exist for + local disks, endpoint pools, lock clients, notification state, tier config, + lifecycle state, and object-store publication. + +## Crate Split Evaluation + +`ecstore-erasure` and `storage-cluster` remain proposal-only until dependency +cycles and hot-path risks are proven safe. + +Required evidence before a split: + +- dependency graph showing no cycle with ECStore, storage API, runtime, or + cluster control-plane owners; +- benchmark or profiling evidence for erasure and bitrot hot paths; +- compatibility plan for public ECStore facade paths and test harnesses; +- rollback plan that preserves quorum, remote disk, lock, and data movement + behavior. + +## Preservation Rules + +- Do not remove AppContext fallback in a broad cleanup PR. +- Do not introduce direct global reads in admin, app, server, storage, scanner, + heal, IAM, notify, observability, Swift, or S3 Select business logic. +- Do not split crates in the same PR that moves runtime state. +- Do not change startup order, readiness, KMS fatal boundaries, IAM recovery, + lock quorum, object placement, reader behavior, or notification/audit + lifecycle while shrinking global state. diff --git a/docs/architecture/migration-progress.md b/docs/architecture/migration-progress.md index cf2912c41..51b870d9e 100644 --- a/docs/architecture/migration-progress.md +++ b/docs/architecture/migration-progress.md @@ -5,22 +5,24 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block ## Current Context - Issue: [`rustfs/backlog#660`](https://github.com/rustfs/backlog/issues/660) -- Branch: `overtrue/arch-docs-phase` +- Branch: `overtrue/arch-global-state-inventory-phase` - Baseline: completed `C-011/C-012/C-013/API-055/API-059/API-079/API-080/API-081/API-082/API-083/API-084/API-085/API-086/API-087/API-088/API-089/API-090/API-091/API-092/API-093/API-094/API-095/API-096/API-097/API-098/API-099/API-100/API-101/API-102/API-103/API-104/API-105/API-106/API-107/API-108/API-109/API-110/API-111/API-112/API-113/API-114/API-115/API-116/API-117/API-118/API-119/API-120/API-121/API-122/API-123/API-124/API-125/API-126/API-127/API-128/API-129/API-130/API-131/API-132/API-133/API-134/API-135/API-136/API-137/API-138/API-139/API-140/API-141/API-142/API-143/API-144/API-145/API-146/API-147/API-148/API-149/API-150/API-151/API-152/API-153/API-154/API-155/API-156/API-157/API-158/API-159/API-160/API-161/API-162/API-163/API-164/API-165/API-166/API-167/API-168/API-169/API-170/API-171/API-172/API-173/API-174/API-175/API-176/API-177/API-178/API-179/API-180/API-181/API-182/API-183/API-184/API-185/API-186/API-187/API-188/API-189/API-190/API-191/API-192/API-193/API-194/API-195/API-196/API-197/API-198/API-199/API-200/API-201/API-202/API-203/API-204/API-205/API-206/API-207/API-208/API-209/API-210/API-211/API-212/API-213/API-214/API-215/API-216/API-217/API-218/API-219/API-220/API-221/API-222/API-223/API-224/API-225/API-226/API-227/API-228/API-229/API-230/API-231/API-232/API-233/API-234/API-235/API-236/API-237/API-238/API-239/API-240/API-241/API-242/API-243/API-244/API-245/API-246/API-247/API-248/API-249/API-250/API-251/API-252/API-253/API-254/CTX-002`. - Current baseline also includes API-255 from PR #3923, API-256 from PR - #3925, CFG-009 from PR #3927, C-007/C-009 from PR #3935, and C-008/C-010 - from PR #3936. -- Current phase PR: DOC-001/DOC-002/DOC-003/DOC-004/DOC-005/TEST-DOC-001 - architecture documentation guard batch. -- Based on: `origin/main` after PR #3936 merged. + #3925, CFG-009 from PR #3927, C-007/C-009 from PR #3935, C-008/C-010 + from PR #3936, and DOC-001/DOC-002/DOC-003/DOC-004/DOC-005/ + TEST-DOC-001 from PR #3938. +- Current phase PR: GLOB-001/GLOB-002/GLOB-003/GLOB-004/GLOB-005/GLOB-006/ + CRATE-001/CRATE-002 global-state inventory and crate-split guard batch. +- Based on: `origin/main` after PR #3938 merged. - PR type for this branch: `ci-gate`. - Runtime behavior changes: none. - Rust code changes: none. -- CI/script changes: require the core architecture docs and readiness matrix - sections in `scripts/check_architecture_migration_rules.sh`. -- Docs changes: add the readiness matrix, link it from the architecture overview, - align runtime/startup/control-plane docs with the gated peer-health readiness - contract, and record required architecture docs in crate boundaries. +- CI/script changes: require the global-state/crate-split plan anchors and lock + direct `rustfs_ecstore::api::global` access to reviewed local `storage_api` + boundary files. +- Docs changes: add the global-state and crate-split plan, link it from the + architecture overview, and include it in the required architecture document + set. ## Phase 0 Tasks @@ -2744,6 +2746,39 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block formatting check, diff hygiene, risk scan, and required three-expert review passed before push. +## Phase 7 Global State And Future Crate Split Tasks + +- [x] `GLOB-001/GLOB-002` Carry forward AppContext split and resolver tests. + - Completed earlier: CTX-001 through CTX-003 split AppContext files and added + resolver plus IAM deferred readiness tests. + - Current slice: document that Phase 7 starts from the existing + AppContext-first plus fallback contract instead of redoing early foundation + work. + - Verification: architecture migration guard and diff hygiene. +- [x] `GLOB-003/GLOB-004/GLOB-005` Record migrated runtime-source boundaries. + - Completed earlier: KMS, bucket metadata, object store, endpoints, tier, + server config, buffer config, admin, app, server, storage, scanner, heal, + IAM, notify, observability, Swift, and S3 Select consumers were routed + through AppContext or owner-local runtime/storage API boundaries. + - Current slice: add + [`global-state-crate-split-plan.md`](global-state-crate-split-plan.md) to + record the remaining owner boundaries and fallback removal sequence. + - Verification: architecture migration guard and diff hygiene. +- [~] `GLOB-006` Shrink `ecstore::global`. + - Current slice: guard the current reviewed files allowed to reference + `rustfs_ecstore::api::global` directly. + - Remaining work: move ECStore bootstrap/runtime globals behind explicit owner + handles where safe; do not remove bootstrap state in this guardrail PR. + - Verification: architecture migration guard and diff hygiene. +- [ ] `GLOB-007` Remove fallbacks. + - Remaining work: remove one fallback family per PR only after scans prove no + production caller depends on it. +- [~] `CRATE-001/CRATE-002` Evaluate future crate splits. + - Current slice: record evidence required before `ecstore-erasure` or + `storage-cluster` split proposals. + - Remaining work: dependency graph, benchmark/profiling evidence, + compatibility plan, and rollback plan before any split code. + ## Phase 8 Background Controller Tasks - [x] `BGC-001` Inventory background services. @@ -6001,6 +6036,9 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block | Expert | Status | Notes | |---|---|---| +| Quality/architecture | pass | The Phase 7 batch documents remaining global owners and locks direct ECStore global facade access to reviewed storage_api boundaries without adding another abstraction layer. | +| Migration preservation | pass | This is docs and guardrail only; AppContext fallback, ECStore bootstrap globals, startup order, readiness, IAM/KMS, lock quorum, and storage behavior are unchanged. | +| Testing/verification | pass | Shell syntax, architecture migration guard, diff hygiene, and full `make pre-pr` passed after rebasing onto current `origin/main`; no Rust source changed. | | Quality/architecture | pass | DOC-001 through DOC-005 add one phase-level architecture documentation batch and keep the guard anchored to required sections instead of duplicating a parallel docs system. | | Migration preservation | pass | The readiness matrix and lifecycle docs only record existing S3/admin/RPC/probe behavior plus the disabled-by-default peer-health readiness gate; no runtime code or behavior changes are included. | | Testing/verification | pass | Shell syntax, architecture migration guard, diff hygiene, and full `make pre-pr` passed after rebasing onto current `origin/main`. | @@ -6352,6 +6390,16 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block Passed before push: +- Issue #660 GLOB-001/GLOB-002/GLOB-003/GLOB-004/GLOB-005/GLOB-006/ + CRATE-001/CRATE-002 current slice: + - Branch freshness check: rebased onto `origin/main` after PR #3938 merged. + - `bash -n scripts/check_architecture_migration_rules.sh`: passed. + - `./scripts/check_architecture_migration_rules.sh`: passed. + - `git diff --check`: passed. + - Rust source risk scan: passed; no Rust source changed. + - Three-expert review: passed. + - Full PR gate: `make pre-pr` passed before PR. + - Issue #660 DOC-001/DOC-002/DOC-003/DOC-004/DOC-005/TEST-DOC-001 current slice: - Branch freshness check: rebased onto `origin/main` after PR #3936 merged. diff --git a/docs/architecture/overview.md b/docs/architecture/overview.md index d696b2c7c..7a7b009f4 100644 --- a/docs/architecture/overview.md +++ b/docs/architecture/overview.md @@ -35,6 +35,9 @@ hot-path behavior must not drift during this migration. future read-only BackgroundController work. - [`crate-boundaries.md`](crate-boundaries.md): PR types, crate direction, compatibility rules, and migration guardrails. +- [`global-state-crate-split-plan.md`](global-state-crate-split-plan.md): late + global-state cleanup, runtime-source boundaries, fallback removal rules, and + crate-split evaluation criteria. - [`ecstore-config-consumer-inventory.md`](ecstore-config-consumer-inventory.md): current `ecstore::config::{Config, KV, KVS}` definitions, consumers, migration risks, and do-not-change contract. diff --git a/scripts/check_architecture_migration_rules.sh b/scripts/check_architecture_migration_rules.sh index ad849a5e7..79058c0b5 100755 --- a/scripts/check_architecture_migration_rules.sh +++ b/scripts/check_architecture_migration_rules.sh @@ -62,6 +62,11 @@ require_source_contains "docs/architecture/readiness-matrix.md" "StorageReady" " require_source_contains "docs/architecture/readiness-matrix.md" "IamReady" "readiness matrix IAM dependency" require_source_contains "docs/architecture/readiness-matrix.md" "FullReady" "readiness matrix full readiness dependency" require_source_contains "docs/architecture/readiness-matrix.md" "RUSTFS_HEALTH_PEER_READY_CHECK_ENABLE" "readiness matrix peer health gate" +require_source_contains "docs/architecture/global-state-crate-split-plan.md" "## Remaining Global Owners" "global state plan owner inventory" +require_source_contains "docs/architecture/global-state-crate-split-plan.md" "## Runtime Source Boundaries" "global state plan runtime source section" +require_source_contains "docs/architecture/global-state-crate-split-plan.md" "## Fallback Removal Plan" "global state plan fallback section" +require_source_contains "docs/architecture/global-state-crate-split-plan.md" "## Crate Split Evaluation" "global state plan crate split section" +require_source_contains "docs/architecture/global-state-crate-split-plan.md" "rustfs_ecstore::api::global" "global state plan facade boundary" TMP_DIR="$(mktemp -d)" trap 'rm -rf "$TMP_DIR"' EXIT @@ -144,6 +149,9 @@ RUSTFS_APP_ECSTORE_SOURCE_HITS_FILE="${TMP_DIR}/rustfs_app_ecstore_source_hits.t RUSTFS_ADMIN_ECSTORE_SOURCE_HITS_FILE="${TMP_DIR}/rustfs_admin_ecstore_source_hits.txt" EXTERNAL_RUNTIME_ECSTORE_COMPAT_BYPASS_HITS_FILE="${TMP_DIR}/external_runtime_ecstore_compat_bypass_hits.txt" EXTERNAL_RUNTIME_STORAGE_API_BYPASS_HITS_FILE="${TMP_DIR}/external_runtime_storage_api_bypass_hits.txt" +GLOBAL_FACADE_BOUNDARY_EXPECTED_FILE="${TMP_DIR}/global_facade_boundary_expected.txt" +GLOBAL_FACADE_BOUNDARY_ACTUAL_FILE="${TMP_DIR}/global_facade_boundary_actual.txt" +GLOBAL_FACADE_BOUNDARY_DIFF_FILE="${TMP_DIR}/global_facade_boundary_diff.txt" EXTERNAL_STORAGE_API_DOMAIN_BYPASS_HITS_FILE="${TMP_DIR}/external_storage_api_domain_bypass_hits.txt" EXTERNAL_STORAGE_API_ROOT_REEXPORT_HITS_FILE="${TMP_DIR}/external_storage_api_root_reexport_hits.txt" RUSTFS_STORAGE_API_ROOT_REEXPORT_HITS_FILE="${TMP_DIR}/rustfs_storage_api_root_reexport_hits.txt" @@ -1517,6 +1525,27 @@ if [[ -s "$EXTERNAL_RUNTIME_ECSTORE_COMPAT_BYPASS_HITS_FILE" ]]; then report_failure "external runtime crates must source ECStore API symbols through their local storage_api boundary: $(paste -sd '; ' "$EXTERNAL_RUNTIME_ECSTORE_COMPAT_BYPASS_HITS_FILE")" fi +cat >"$GLOBAL_FACADE_BOUNDARY_EXPECTED_FILE" <<'EOF' +crates/heal/src/heal/storage_api.rs +crates/iam/src/storage_api.rs +crates/notify/src/storage_api.rs +crates/obs/src/metrics/storage_api.rs +crates/protocols/src/swift/storage_api.rs +crates/s3select-api/src/storage_api.rs +crates/scanner/src/storage_api.rs +crates/scanner/tests/storage_api/mod.rs +rustfs/src/storage/storage_api.rs +EOF + +( + cd "$ROOT_DIR" + rg -l 'rustfs_ecstore::api::global' crates rustfs/src --glob '*.rs' | LC_ALL=C sort +) >"$GLOBAL_FACADE_BOUNDARY_ACTUAL_FILE" + +if ! diff -u "$GLOBAL_FACADE_BOUNDARY_EXPECTED_FILE" "$GLOBAL_FACADE_BOUNDARY_ACTUAL_FILE" >"$GLOBAL_FACADE_BOUNDARY_DIFF_FILE"; then + report_failure "ECStore global facade access must stay inside reviewed local storage_api boundaries: $(tr '\n' '; ' <"$GLOBAL_FACADE_BOUNDARY_DIFF_FILE")" +fi + ( cd "$ROOT_DIR" rg -n --with-filename '^use rustfs_storage_api|rustfs_storage_api::' \