gsadmin/rustfs
1
0
Fork 0
mirror of https://github.com/rustfs/rustfs.git synced 2026-08-13 08:36:54 +00:00
Code Issues Packages Projects Releases Wiki Activity

refactor: centralize ecstore runtime owner sources (#3798)

Browse Source
This commit is contained in:
Zhengchao An
2026-06-24 06:34:06 +08:00
committed by GitHub
parent 1735dcde9c
commit 7e60432588
18 changed files with 499 additions and 231 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/architecture/migration-progress.md
+109 -4
View File
@@ -5,9 +5,9 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block
## Current Context
- Issue: [`rustfs/backlog#660`](https://github.com/rustfs/backlog/issues/660)
- Branch: `overtrue/arch-ecstore-data-plane-runtime-sources`
- Baseline: completed `C-011/C-012/C-013/API-055/API-059/API-079/API-080/API-081/API-082/API-083/API-084/API-085/API-086/API-087/API-088/API-089/API-090/API-091/API-092/API-093/API-094/API-095/API-096/API-097/API-098/API-099/API-100/API-101/API-102/API-103/API-104/API-105/API-106/API-107/API-108/API-109/API-110/API-111/API-112/API-113/API-114/API-115/API-116/API-117/API-118/API-119/API-120/API-121/API-122/API-123/API-124/API-125/API-126/API-127/API-128/API-129/API-130/API-131/API-132/API-133/API-134/API-135/API-136/API-137/API-138/API-139/API-140/API-141/API-142/API-143/API-144/API-145/API-146/API-147/API-148/API-149/API-150/API-151/API-152/API-153/API-154/API-155/API-156/API-157/API-158/API-159/API-160/API-161/API-162/API-163/API-164/API-165/API-166/API-167/API-168/API-169/API-170/API-171/API-172/API-173/API-174/API-175/API-176/API-177/API-178/API-179/API-180/API-181/API-182/API-183/API-184/API-185/API-186`.
- Based on: latest `origin/main` after PR #3796 merged API-185.
- Branch: `overtrue/arch-ecstore-replication-runtime-sources`
- Baseline: completed `C-011/C-012/C-013/API-055/API-059/API-079/API-080/API-081/API-082/API-083/API-084/API-085/API-086/API-087/API-088/API-089/API-090/API-091/API-092/API-093/API-094/API-095/API-096/API-097/API-098/API-099/API-100/API-101/API-102/API-103/API-104/API-105/API-106/API-107/API-108/API-109/API-110/API-111/API-112/API-113/API-114/API-115/API-116/API-117/API-118/API-119/API-120/API-121/API-122/API-123/API-124/API-125/API-126/API-127/API-128/API-129/API-130/API-131/API-132/API-133/API-134/API-135/API-136/API-137/API-138/API-139/API-140/API-141/API-142/API-143/API-144/API-145/API-146/API-147/API-148/API-149/API-150/API-151/API-152/API-153/API-154/API-155/API-156/API-157/API-158/API-159/API-160/API-161/API-162/API-163/API-164/API-165/API-166/API-167/API-168/API-169/API-170/API-171/API-172/API-173/API-174/API-175/API-176/API-177/API-178/API-179/API-180/API-181/API-182/API-183/API-184/API-185/API-186/API-187/API-188/API-189`.
- Based on: stacked on API-188 branch while PR #3799 is pending.
- PR type for this branch: `consumer-migration`
- Runtime behavior changes: none.
- Rust code changes: route replication pool, outbound TLS generation, runtime
@@ -18,6 +18,9 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block
runtime source reads, RIO HTTP reader TLS/metrics runtime source reads, and
gRPC/transition network client TLS/metrics runtime source reads, plus ECStore
data-plane KMS/storage-class/deployment-id/lock-manager/erasure metric reads,
plus ECStore observability/status object-store, endpoint, node-name,
boot-time, init-time, root-disk threshold, and cached RPC channel reads,
plus ECStore replication pool, replication stats, and event-host reads,
through AppContext-first or owner-crate resolver boundaries.
- CI/script changes: lock completed owner and test/fuzz boundaries against
bare/glob imports, scattered raw ECStore facade subpaths, and startup
@@ -27,7 +30,7 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block
and storage owner thin bridge regressions, plus app context and notify
event-bridge thin module regressions; accept the reviewed AppContext resolver
reverse dependencies in the layer baseline.
- Docs changes: record the API-136 through API-186 owner facade cleanup.
- Docs changes: record the API-136 through API-189 owner facade cleanup.
## Phase 0 Tasks
@@ -4700,6 +4703,50 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block
source scan, Rust risk scan, branch freshness check, pre-commit quality
gate, and three-expert review.
- [x] `API-187` Centralize ECStore runtime owner source reads.
- Do: expand the ECStore runtime-source boundary across rebalance storage
class and notification reads, bucket monitor cleanup, lifecycle config
lookups, local disk id/path/set-drive maps, peer disk discovery, and store
init runtime handles.
- Acceptance: ECStore data-plane, rebalance, lifecycle, bucket, peer, and
startup paths route those runtime globals through the ECStore-owned runtime
source module instead of importing them directly.
- Must preserve: rebalance parity selection and notifications, lifecycle
config lookup semantics, bucket monitor deletion, local disk id backfill,
endpoint disk lookup fallback, local disk map initialization, deployment id
publication, and tier config initialization.
- Verification: ECStore compile coverage, focused store/pools/set-disk tests,
formatting, migration guard, layer guard, diff hygiene, residual runtime
source scan, Rust risk scan, branch freshness check, pre-commit quality
gate, and three-expert review.
- [x] `API-188` Centralize ECStore observability runtime source reads.
- Do: route ECStore server-info, realtime metrics, data-usage cache,
local-disk root detection, and RPC cached-channel runtime reads through the
ECStore-owned runtime-source module.
- Acceptance: ECStore observability/status paths no longer import those
runtime globals directly outside the owner runtime-source boundary.
- Must preserve: server info endpoint/uptime/deployment-id fields, storage
info and backend summary collection, realtime metrics host fallback,
scanner init-time override, data-usage cache refresh behavior, root-disk
threshold checks, and cached gRPC channel reuse.
- Verification: ECStore compile coverage, formatting, diff hygiene, residual
runtime source scan, Rust risk scan, focused tests, migration/layer guards,
PR-before-push pre-commit quality gate, and three-expert review.
- [x] `API-189` Centralize ECStore replication runtime source reads.
- Do: route replication pool, replication stats, and replication event-host
runtime reads through the ECStore-owned runtime-source module.
- Acceptance: replication pool/resyncer code no longer reads those runtime
globals directly outside the owner runtime-source boundary.
- Must preserve: background replication initialization, async/sync queueing,
delete-task stats updates, proxy request stats, resync status updates, and
emitted replication event host values.
- Verification: ECStore compile coverage, formatting, diff hygiene, residual
replication runtime-source scan, Rust risk scan, focused tests,
migration/layer guards, PR-before-push pre-commit quality gate, and
three-expert review.
## Next PRs
1. `consumer-migration`: continue reducing direct global reads behind AppContext resolver boundaries.
@@ -4814,11 +4861,69 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block
| Quality/architecture | pass | API-186 keeps ECStore data-plane runtime globals behind an ECStore-owned runtime-source module without widening public APIs. |
| Migration preservation | pass | Erasure quorum metric labels, managed-KMS fallback, storage-class decisions, multipart upload id encoding, and lock-manager initialization keep existing behavior. |
| Testing/verification | pass | ECStore compile/focused tests, formatting, residual data-plane runtime source scan, targeted guard checks, and pre-commit passed for API-186. |
| Quality/architecture | pass | API-187 expands the ECStore runtime-source owner boundary across rebalance, lifecycle, local disk maps, peer lookup, and store init handles without adding public APIs. |
| Migration preservation | pass | Rebalance notifications, lifecycle config reads, bucket monitor cleanup, local disk id/path/set-drive lookups, store init map publication, and deployment id publication keep existing semantics. |
| Testing/verification | pass | ECStore compile/focused tests, formatting, migration/layer guards, diff-only Rust risk scan, and pre-commit passed for API-187. |
| Quality/architecture | pass | API-188 keeps ECStore observability/status runtime reads behind the ECStore runtime-source boundary without adding public APIs. |
| Migration preservation | pass | Server info fields, metrics host fallback, data-usage cache refresh, root-disk checks, and cached RPC channel reuse keep existing semantics. |
| Testing/verification | pass | ECStore compile/focused tests, formatting, migration/layer guards, residual scan, diff-only Rust risk scan, and pre-commit passed for API-188. |
| Quality/architecture | pass | API-189 keeps ECStore replication runtime pool/stats/host reads behind the ECStore runtime-source boundary without adding public APIs. |
| Migration preservation | pass | Replication initialization, queueing, delete stats, proxy stats, resync status updates, and emitted event host values keep existing semantics. |
| Testing/verification | pass | ECStore compile/focused test, formatting, migration/layer guards, diff hygiene, residual scan, diff-only Rust risk scan, and pre-commit passed for API-189. |
## Verification Notes
Passed before push:
- Issue #660 API-187 current slice:
- `cargo check -p rustfs-ecstore --tests`: passed.
- `cargo test -p rustfs-ecstore --lib test_find_local_disk_by_ref_backfills_uuid_map -- --test-threads=1`:
passed.
- `cargo test -p rustfs-ecstore --lib should_resume_local_decommission -- --test-threads=1`:
passed.
- `cargo test -p rustfs-ecstore --lib resolve_store_init_stage_result -- --test-threads=1`:
passed.
- `cargo test -p rustfs-ecstore --lib test_find_local_disk -- --test-threads=1`:
passed.
- `cargo fmt --all`: passed.
- `cargo fmt --all --check`: passed.
- `git diff --check`: passed.
- `./scripts/check_architecture_migration_rules.sh`: passed.
- `./scripts/check_layer_dependencies.sh`: passed.
- ECStore runtime source scan: passed for API-187 targets; remaining
`set_disk` tier/local-node/global-map matches are legacy owner boundaries
intentionally left for a later focused slice.
- Rust risk scan: passed; diff adds no new `unwrap`, `expect`, `panic`,
`todo`, `unimplemented`, `unsafe`, production print, boxed public error,
string public error, relaxed ordering, or silent integer cast.
- Branch freshness check: rebased onto latest `origin/main` after PR #3797
merged API-186.
- `make pre-commit`: passed.
- Issue #660 API-188 current slice:
- `cargo check -p rustfs-ecstore --tests`: passed.
- `cargo test -p rustfs-ecstore --lib server_info_includes_global_deployment_id -- --test-threads=1`:
passed.
- `cargo fmt --all`: passed.
- `cargo fmt --all --check`: passed.
- `git diff --check`: passed.
- `./scripts/check_architecture_migration_rules.sh`: passed.
- `./scripts/check_layer_dependencies.sh`: passed.
- ECStore runtime source scan: passed for API-188 targets.
- Rust risk scan: passed.
- `make pre-commit`: passed.
- Issue #660 API-189 current slice:
- `cargo check -p rustfs-ecstore --tests`: passed.
- `cargo test -p rustfs-ecstore --lib replication_queue_admission_combines_target_results -- --test-threads=1`:
passed.
- `cargo fmt --all --check`: passed.
- `git diff --check`: passed.
- `./scripts/check_architecture_migration_rules.sh`: passed.
- `./scripts/check_layer_dependencies.sh`: passed.
- Replication runtime-source scan: passed for API-189 targets.
- Diff-only Rust risk scan: passed.
- `make pre-commit`: passed, including 6552 nextest tests passed and
doctests passed; the existing OPA policy test took 603s.
- Issue #660 API-186 current slice:
- `cargo check -p rustfs-ecstore --tests`: passed.
- `cargo test -p rustfs-ecstore --lib erasure_coding -- --test-threads=1`: