gsadmin/rustfs
1
0
Fork 0
mirror of https://github.com/rustfs/rustfs.git synced 2026-07-26 08:18:18 +00:00
Code Issues Packages Projects Releases Wiki Activity

docs: record global state cleanup plan (#3939)

Browse Source
This commit is contained in:
Zhengchao An
2026-06-27 12:04:59 +08:00
committed by GitHub
parent fe9c0daf0f
commit d3f1ff36af
5 changed files with 178 additions and 11 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/architecture/crate-boundaries.md
+3
View File
@@ -50,6 +50,9 @@ their required sections:
Required Architecture Documents.
- `docs/architecture/readiness-matrix.md`: Request Behavior Matrix, Runtime
Dependency Matrix, Probe Semantics.
- `docs/architecture/global-state-crate-split-plan.md`: Remaining Global
Owners, Runtime Source Boundaries, Fallback Removal Plan, Crate Split
Evaluation.
## Pre-Push Expert Review
docs/architecture/global-state-crate-split-plan.md
+84
View File
@@ -0,0 +1,84 @@
# Global State And Crate Split Plan
This document records the late global-state cleanup plan after the AppContext
foundation, storage API contracts, ECStore layout, runtime lifecycle, and cluster
control-plane boundaries are stable.
## Remaining Global Owners
| Owner | Current role | Migration stance |
|---|---|---|
| `rustfs/src/app/context.rs` | AppContext-first resolver facade with legacy fallback adapters. | Keep resolver entry points stable until every caller has an explicit context or owner-local runtime source. |
| `rustfs/src/app/context/runtime_sources.rs` | Default AppContext fallback adapters for KMS, IAM, object store, endpoints, config, metrics, and notification state. | This is an allowed fallback boundary, not a business logic owner. |
| `rustfs/src/*/runtime_sources.rs` | Root, admin, app, server, startup, and storage owner-local runtime-source boundaries. | Business modules use these boundaries instead of calling global state directly. |
| `rustfs/src/*/storage_api.rs` | Root, admin, app, and storage owner-local storage contract/facade boundaries. | Storage helper and ECStore facade access remains visible at local owner boundaries. |
| `crates/*/storage_api.rs` | External crate-local storage facade boundaries for IAM, scanner, heal, notify, observability, Swift, and S3 Select. | External runtime crates may consume ECStore facade globals only through their local storage API boundary. |
| `crates/ecstore/src/runtime/global.rs` | ECStore bootstrap/runtime state owner. | Keep internal until ECStore has explicit owner handles for all remaining bootstrap state. |
| `crates/ecstore/src/runtime/sources.rs` | ECStore runtime-source adapter over global state. | Preferred ECStore-internal access path while shrinking direct `runtime::global` reads. |
## Runtime Source Boundaries
Runtime-source modules are the allowed compatibility layer between migrated
consumers and process-global state. They must keep these properties:
- context-first lookup when an `AppContext` handle exists;
- explicit fallback to the existing global only where compatibility still
requires it;
- no hidden service construction in business logic;
- no startup, readiness, IAM, KMS, lock, notification, or storage behavior
change in inventory or guardrail PRs.
## Guarded Boundary List
The architecture guard snapshots the files currently allowed to reference
`rustfs_ecstore::api::global` directly:
- `crates/heal/src/heal/storage_api.rs`
- `crates/iam/src/storage_api.rs`
- `crates/notify/src/storage_api.rs`
- `crates/obs/src/metrics/storage_api.rs`
- `crates/protocols/src/swift/storage_api.rs`
- `crates/s3select-api/src/storage_api.rs`
- `crates/scanner/src/storage_api.rs`
- `crates/scanner/tests/storage_api/mod.rs`
- `rustfs/src/storage/storage_api.rs`
New direct uses must either move behind an existing owner-local boundary or
update this plan and the guard in the same reviewed migration PR.
## Fallback Removal Plan
1. Keep AppContext-first plus global fallback while compatibility tests still
cover both paths.
2. Move one remaining consumer group at a time to explicit context or
owner-local runtime-source handles.
3. Remove one fallback family per PR only after scans prove no production caller
depends on it.
4. Keep embedded startup and tests working before deleting any fallback.
5. Do not remove ECStore bootstrap globals until ownership handles exist for
local disks, endpoint pools, lock clients, notification state, tier config,
lifecycle state, and object-store publication.
## Crate Split Evaluation
`ecstore-erasure` and `storage-cluster` remain proposal-only until dependency
cycles and hot-path risks are proven safe.
Required evidence before a split:
- dependency graph showing no cycle with ECStore, storage API, runtime, or
cluster control-plane owners;
- benchmark or profiling evidence for erasure and bitrot hot paths;
- compatibility plan for public ECStore facade paths and test harnesses;
- rollback plan that preserves quorum, remote disk, lock, and data movement
behavior.
## Preservation Rules
- Do not remove AppContext fallback in a broad cleanup PR.
- Do not introduce direct global reads in admin, app, server, storage, scanner,
heal, IAM, notify, observability, Swift, or S3 Select business logic.
- Do not split crates in the same PR that moves runtime state.
- Do not change startup order, readiness, KMS fatal boundaries, IAM recovery,
lock quorum, object placement, reader behavior, or notification/audit
lifecycle while shrinking global state.
docs/architecture/migration-progress.md
+59 -11
View File
@@ -5,22 +5,24 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block
## Current Context
- Issue: [`rustfs/backlog#660`](https://github.com/rustfs/backlog/issues/660)
- Branch: `overtrue/arch-docs-phase`
- Branch: `overtrue/arch-global-state-inventory-phase`
- Baseline: completed `C-011/C-012/C-013/API-055/API-059/API-079/API-080/API-081/API-082/API-083/API-084/API-085/API-086/API-087/API-088/API-089/API-090/API-091/API-092/API-093/API-094/API-095/API-096/API-097/API-098/API-099/API-100/API-101/API-102/API-103/API-104/API-105/API-106/API-107/API-108/API-109/API-110/API-111/API-112/API-113/API-114/API-115/API-116/API-117/API-118/API-119/API-120/API-121/API-122/API-123/API-124/API-125/API-126/API-127/API-128/API-129/API-130/API-131/API-132/API-133/API-134/API-135/API-136/API-137/API-138/API-139/API-140/API-141/API-142/API-143/API-144/API-145/API-146/API-147/API-148/API-149/API-150/API-151/API-152/API-153/API-154/API-155/API-156/API-157/API-158/API-159/API-160/API-161/API-162/API-163/API-164/API-165/API-166/API-167/API-168/API-169/API-170/API-171/API-172/API-173/API-174/API-175/API-176/API-177/API-178/API-179/API-180/API-181/API-182/API-183/API-184/API-185/API-186/API-187/API-188/API-189/API-190/API-191/API-192/API-193/API-194/API-195/API-196/API-197/API-198/API-199/API-200/API-201/API-202/API-203/API-204/API-205/API-206/API-207/API-208/API-209/API-210/API-211/API-212/API-213/API-214/API-215/API-216/API-217/API-218/API-219/API-220/API-221/API-222/API-223/API-224/API-225/API-226/API-227/API-228/API-229/API-230/API-231/API-232/API-233/API-234/API-235/API-236/API-237/API-238/API-239/API-240/API-241/API-242/API-243/API-244/API-245/API-246/API-247/API-248/API-249/API-250/API-251/API-252/API-253/API-254/CTX-002`.
- Current baseline also includes API-255 from PR #3923, API-256 from PR
#3925, CFG-009 from PR #3927, C-007/C-009 from PR #3935, and C-008/C-010
from PR #3936.
- Current phase PR: DOC-001/DOC-002/DOC-003/DOC-004/DOC-005/TEST-DOC-001
architecture documentation guard batch.
- Based on: `origin/main` after PR #3936 merged.
#3925, CFG-009 from PR #3927, C-007/C-009 from PR #3935, C-008/C-010
from PR #3936, and DOC-001/DOC-002/DOC-003/DOC-004/DOC-005/
TEST-DOC-001 from PR #3938.
- Current phase PR: GLOB-001/GLOB-002/GLOB-003/GLOB-004/GLOB-005/GLOB-006/
CRATE-001/CRATE-002 global-state inventory and crate-split guard batch.
- Based on: `origin/main` after PR #3938 merged.
- PR type for this branch: `ci-gate`.
- Runtime behavior changes: none.
- Rust code changes: none.
- CI/script changes: require the core architecture docs and readiness matrix
sections in `scripts/check_architecture_migration_rules.sh`.
- Docs changes: add the readiness matrix, link it from the architecture overview,
align runtime/startup/control-plane docs with the gated peer-health readiness
contract, and record required architecture docs in crate boundaries.
- CI/script changes: require the global-state/crate-split plan anchors and lock
direct `rustfs_ecstore::api::global` access to reviewed local `storage_api`
boundary files.
- Docs changes: add the global-state and crate-split plan, link it from the
architecture overview, and include it in the required architecture document
set.
## Phase 0 Tasks
@@ -2744,6 +2746,39 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block
formatting check, diff hygiene, risk scan, and required three-expert review
passed before push.
## Phase 7 Global State And Future Crate Split Tasks
- [x] `GLOB-001/GLOB-002` Carry forward AppContext split and resolver tests.
- Completed earlier: CTX-001 through CTX-003 split AppContext files and added
resolver plus IAM deferred readiness tests.
- Current slice: document that Phase 7 starts from the existing
AppContext-first plus fallback contract instead of redoing early foundation
work.
- Verification: architecture migration guard and diff hygiene.
- [x] `GLOB-003/GLOB-004/GLOB-005` Record migrated runtime-source boundaries.
- Completed earlier: KMS, bucket metadata, object store, endpoints, tier,
server config, buffer config, admin, app, server, storage, scanner, heal,
IAM, notify, observability, Swift, and S3 Select consumers were routed
through AppContext or owner-local runtime/storage API boundaries.
- Current slice: add
[`global-state-crate-split-plan.md`](global-state-crate-split-plan.md) to
record the remaining owner boundaries and fallback removal sequence.
- Verification: architecture migration guard and diff hygiene.
- [~] `GLOB-006` Shrink `ecstore::global`.
- Current slice: guard the current reviewed files allowed to reference
`rustfs_ecstore::api::global` directly.
- Remaining work: move ECStore bootstrap/runtime globals behind explicit owner
handles where safe; do not remove bootstrap state in this guardrail PR.
- Verification: architecture migration guard and diff hygiene.
- [ ] `GLOB-007` Remove fallbacks.
- Remaining work: remove one fallback family per PR only after scans prove no
production caller depends on it.
- [~] `CRATE-001/CRATE-002` Evaluate future crate splits.
- Current slice: record evidence required before `ecstore-erasure` or
`storage-cluster` split proposals.
- Remaining work: dependency graph, benchmark/profiling evidence,
compatibility plan, and rollback plan before any split code.
## Phase 8 Background Controller Tasks
- [x] `BGC-001` Inventory background services.
@@ -6001,6 +6036,9 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block
| Expert | Status | Notes |
|---|---|---|
| Quality/architecture | pass | The Phase 7 batch documents remaining global owners and locks direct ECStore global facade access to reviewed storage_api boundaries without adding another abstraction layer. |
| Migration preservation | pass | This is docs and guardrail only; AppContext fallback, ECStore bootstrap globals, startup order, readiness, IAM/KMS, lock quorum, and storage behavior are unchanged. |
| Testing/verification | pass | Shell syntax, architecture migration guard, diff hygiene, and full `make pre-pr` passed after rebasing onto current `origin/main`; no Rust source changed. |
| Quality/architecture | pass | DOC-001 through DOC-005 add one phase-level architecture documentation batch and keep the guard anchored to required sections instead of duplicating a parallel docs system. |
| Migration preservation | pass | The readiness matrix and lifecycle docs only record existing S3/admin/RPC/probe behavior plus the disabled-by-default peer-health readiness gate; no runtime code or behavior changes are included. |
| Testing/verification | pass | Shell syntax, architecture migration guard, diff hygiene, and full `make pre-pr` passed after rebasing onto current `origin/main`. |
@@ -6352,6 +6390,16 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block
Passed before push:
- Issue #660 GLOB-001/GLOB-002/GLOB-003/GLOB-004/GLOB-005/GLOB-006/
CRATE-001/CRATE-002 current slice:
- Branch freshness check: rebased onto `origin/main` after PR #3938 merged.
- `bash -n scripts/check_architecture_migration_rules.sh`: passed.
- `./scripts/check_architecture_migration_rules.sh`: passed.
- `git diff --check`: passed.
- Rust source risk scan: passed; no Rust source changed.
- Three-expert review: passed.
- Full PR gate: `make pre-pr` passed before PR.
- Issue #660 DOC-001/DOC-002/DOC-003/DOC-004/DOC-005/TEST-DOC-001 current
slice:
- Branch freshness check: rebased onto `origin/main` after PR #3936 merged.
docs/architecture/overview.md
+3
View File
@@ -35,6 +35,9 @@ hot-path behavior must not drift during this migration.
future read-only BackgroundController work.
- [`crate-boundaries.md`](crate-boundaries.md): PR types, crate direction,
compatibility rules, and migration guardrails.
- [`global-state-crate-split-plan.md`](global-state-crate-split-plan.md): late
global-state cleanup, runtime-source boundaries, fallback removal rules, and
crate-split evaluation criteria.
- [`ecstore-config-consumer-inventory.md`](ecstore-config-consumer-inventory.md):
current `ecstore::config::{Config, KV, KVS}` definitions, consumers,
migration risks, and do-not-change contract.
scripts/check_architecture_migration_rules.sh
+29
View File
@@ -62,6 +62,11 @@ require_source_contains "docs/architecture/readiness-matrix.md" "StorageReady" "
require_source_contains "docs/architecture/readiness-matrix.md" "IamReady" "readiness matrix IAM dependency"
require_source_contains "docs/architecture/readiness-matrix.md" "FullReady" "readiness matrix full readiness dependency"
require_source_contains "docs/architecture/readiness-matrix.md" "RUSTFS_HEALTH_PEER_READY_CHECK_ENABLE" "readiness matrix peer health gate"
require_source_contains "docs/architecture/global-state-crate-split-plan.md" "## Remaining Global Owners" "global state plan owner inventory"
require_source_contains "docs/architecture/global-state-crate-split-plan.md" "## Runtime Source Boundaries" "global state plan runtime source section"
require_source_contains "docs/architecture/global-state-crate-split-plan.md" "## Fallback Removal Plan" "global state plan fallback section"
require_source_contains "docs/architecture/global-state-crate-split-plan.md" "## Crate Split Evaluation" "global state plan crate split section"
require_source_contains "docs/architecture/global-state-crate-split-plan.md" "rustfs_ecstore::api::global" "global state plan facade boundary"
TMP_DIR="$(mktemp -d)"
trap 'rm -rf "$TMP_DIR"' EXIT
@@ -144,6 +149,9 @@ RUSTFS_APP_ECSTORE_SOURCE_HITS_FILE="${TMP_DIR}/rustfs_app_ecstore_source_hits.t
RUSTFS_ADMIN_ECSTORE_SOURCE_HITS_FILE="${TMP_DIR}/rustfs_admin_ecstore_source_hits.txt"
EXTERNAL_RUNTIME_ECSTORE_COMPAT_BYPASS_HITS_FILE="${TMP_DIR}/external_runtime_ecstore_compat_bypass_hits.txt"
EXTERNAL_RUNTIME_STORAGE_API_BYPASS_HITS_FILE="${TMP_DIR}/external_runtime_storage_api_bypass_hits.txt"
GLOBAL_FACADE_BOUNDARY_EXPECTED_FILE="${TMP_DIR}/global_facade_boundary_expected.txt"
GLOBAL_FACADE_BOUNDARY_ACTUAL_FILE="${TMP_DIR}/global_facade_boundary_actual.txt"
GLOBAL_FACADE_BOUNDARY_DIFF_FILE="${TMP_DIR}/global_facade_boundary_diff.txt"
EXTERNAL_STORAGE_API_DOMAIN_BYPASS_HITS_FILE="${TMP_DIR}/external_storage_api_domain_bypass_hits.txt"
EXTERNAL_STORAGE_API_ROOT_REEXPORT_HITS_FILE="${TMP_DIR}/external_storage_api_root_reexport_hits.txt"
RUSTFS_STORAGE_API_ROOT_REEXPORT_HITS_FILE="${TMP_DIR}/rustfs_storage_api_root_reexport_hits.txt"
@@ -1517,6 +1525,27 @@ if [[ -s "$EXTERNAL_RUNTIME_ECSTORE_COMPAT_BYPASS_HITS_FILE" ]]; then
report_failure "external runtime crates must source ECStore API symbols through their local storage_api boundary: $(paste -sd '; ' "$EXTERNAL_RUNTIME_ECSTORE_COMPAT_BYPASS_HITS_FILE")"
fi
cat >"$GLOBAL_FACADE_BOUNDARY_EXPECTED_FILE" <<'EOF'
crates/heal/src/heal/storage_api.rs
crates/iam/src/storage_api.rs
crates/notify/src/storage_api.rs
crates/obs/src/metrics/storage_api.rs
crates/protocols/src/swift/storage_api.rs
crates/s3select-api/src/storage_api.rs
crates/scanner/src/storage_api.rs
crates/scanner/tests/storage_api/mod.rs
rustfs/src/storage/storage_api.rs
EOF
(
cd "$ROOT_DIR"
rg -l 'rustfs_ecstore::api::global' crates rustfs/src --glob '*.rs' | LC_ALL=C sort
) >"$GLOBAL_FACADE_BOUNDARY_ACTUAL_FILE"
if ! diff -u "$GLOBAL_FACADE_BOUNDARY_EXPECTED_FILE" "$GLOBAL_FACADE_BOUNDARY_ACTUAL_FILE" >"$GLOBAL_FACADE_BOUNDARY_DIFF_FILE"; then
report_failure "ECStore global facade access must stay inside reviewed local storage_api boundaries: $(tr '\n' '; ' <"$GLOBAL_FACADE_BOUNDARY_DIFF_FILE")"
fi
(
cd "$ROOT_DIR"
rg -n --with-filename '^use rustfs_storage_api|rustfs_storage_api::' \