mirror of
https://github.com/rustfs/rustfs.git
synced 2026-08-06 05:17:42 +00:00
refactor: establish ecstore layout foundation (#3645)
This commit is contained in:
安正超
@@ -0,0 +1,53 @@
|
||||
# ECStore Layout Boundary
|
||||
|
||||
This document records the `E-001` and `E-SET-001` foundation slice for the
|
||||
architecture migration.
|
||||
|
||||
## Directory Skeleton
|
||||
|
||||
The ECStore migration uses these internal ownership buckets before any pure
|
||||
file moves:
|
||||
|
||||
- `api`: facade and compatibility re-export ownership.
|
||||
- `core`: store facade, object, bucket, list, multipart, and heal paths.
|
||||
- `layout`: static endpoint, disk, pool, and set layout descriptions.
|
||||
- `disk`: local disk, format, health, and disk error ownership.
|
||||
- `erasure`: erasure coding and bitrot ownership.
|
||||
- `metadata`: bucket metadata, config object-store, and data-usage ownership.
|
||||
- `cluster`: remote disk, peer, lock, membership, and health-control-plane
|
||||
ownership.
|
||||
- `services`: lifecycle, replication, tier, notification, rebalance, and
|
||||
metrics service ownership.
|
||||
|
||||
## Static Set Layout
|
||||
|
||||
Static layout is derived from persisted `FormatV3` data and input endpoint
|
||||
expansion. It may describe:
|
||||
|
||||
- deployment id;
|
||||
- set count and drives per set;
|
||||
- disk UUID positions inside `format.erasure.sets`;
|
||||
- distribution algorithm;
|
||||
- endpoint grouping produced before runtime disk initialization.
|
||||
|
||||
Static layout must not own disk handles, lock clients, reconnect loops, repair
|
||||
state, or shutdown signaling.
|
||||
|
||||
## Runtime Set Orchestration
|
||||
|
||||
Runtime orchestration remains owned by `Sets` and `SetDisks` until a later pure
|
||||
move. It may describe:
|
||||
|
||||
- flat disk index to `(set_index, disk_index)` mapping;
|
||||
- per-set local disk replacement after distributed setup detection;
|
||||
- per-set lock-client host deduplication;
|
||||
- endpoint reconnect monitoring and runtime shutdown signaling;
|
||||
- read/write/heal/list orchestration over initialized disks.
|
||||
|
||||
## Preservation Rules
|
||||
|
||||
- Object-to-set hashing and distribution algorithm selection must not change.
|
||||
- Format `sets` ordering and disk UUID position lookup must not change.
|
||||
- Local disk replacement and lock-client mapping stay runtime-only.
|
||||
- Later file moves must keep old public paths or add explicit compatibility
|
||||
coverage before deleting them.
|
||||
@@ -5,16 +5,16 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block
|
||||
## Current Context
|
||||
|
||||
- Issue: [`rustfs/backlog#660`](https://github.com/rustfs/backlog/issues/660)
|
||||
- Branch: `overtrue/arch-extension-runtime-snapshots`
|
||||
- Baseline: `overtrue/arch-ops-profiler-runtime-contract`
|
||||
(`6d4ff92f398a9decdce53a7203744c7c2a70a4e9`).
|
||||
- Stacked on: local R-032, which is stacked on rustfs/rustfs#3642.
|
||||
- PR type for this branch: `contract`
|
||||
- Branch: `overtrue/arch-ecstore-layout-foundation`
|
||||
- Baseline: `overtrue/arch-extension-runtime-snapshots`
|
||||
(`6ee557f63c4f8eb45334af23a6b3a32e0ee009e9`).
|
||||
- Stacked on: local R-033, which is stacked on local R-032 and rustfs/rustfs#3642.
|
||||
- PR type for this branch: `pure-move`
|
||||
- Runtime behavior changes: none.
|
||||
- Rust code changes: expose builtin diagnostics/profiler runtime capability
|
||||
snapshots through the admin extension catalog response.
|
||||
- Rust code changes: add the ECStore internal layout skeleton and read-only set
|
||||
layout boundary tests.
|
||||
- CI/script changes: none.
|
||||
- Docs changes: record the R-033 extension runtime snapshot slice.
|
||||
- Docs changes: record the ECStore layout boundary and E-001/E-SET-001 slice.
|
||||
|
||||
## Phase 0 Tasks
|
||||
|
||||
@@ -2211,20 +2211,34 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block
|
||||
check, migration/layer guards, formatting, diff hygiene, Rust risk scan,
|
||||
branch freshness check, pre-commit quality gate, and three-expert review.
|
||||
|
||||
- [x] `E-001/E-SET-001` Add ECStore layout skeleton and set-layout boundary.
|
||||
- Do: create the ECStore internal layout ownership buckets and pin static set
|
||||
layout versus runtime `Sets`/`SetDisks` orchestration boundaries before any
|
||||
file moves.
|
||||
- Acceptance: the skeleton documents future ownership buckets, static format
|
||||
set distribution is preserved, and runtime flat disk plus per-set lock-host
|
||||
mapping is described by focused tests.
|
||||
- Must preserve: format distribution, object-to-set hashing owner, local disk
|
||||
replacement, lock client mapping, existing public module paths, and runtime
|
||||
`Sets`/`SetDisks` behavior.
|
||||
- Verification: focused ECStore set layout tests, ECStore/RustFS compile
|
||||
checks, migration/layer guards, formatting, diff hygiene, Rust risk scan,
|
||||
branch freshness check, pre-commit quality gate, and three-expert review.
|
||||
|
||||
## Next PRs
|
||||
|
||||
1. `pure-move`: continue pruning residual embedded startup-only orchestration
|
||||
1. `pure-move`: move ECStore layout files into the new layout bucket with
|
||||
compatibility coverage after E-001/E-SET-001 lands.
|
||||
2. `pure-move`: continue pruning residual embedded startup-only orchestration
|
||||
once the lifecycle helpers are merged.
|
||||
2. `contract`: add the next read-only extension handoff once admin/reporting
|
||||
consumers need additional extension runtime snapshots.
|
||||
|
||||
## Pre-Push Review Log
|
||||
|
||||
| Expert | Status | Notes |
|
||||
|---|---|---|
|
||||
| Quality/architecture | passed | R-033 exposes only read-only diagnostics/profiler capability snapshots through the admin extension catalog response. |
|
||||
| Migration preservation | passed | Catalog auth, plugin instance listing, profiler/diagnostics execution paths, and external plugin flow status semantics remain unchanged. |
|
||||
| Testing/verification | passed | Focused admin catalog and targets runtime checks, lib check, guards, formatting, diff hygiene, Rust risk scan, and full pre-commit passed. |
|
||||
| Quality/architecture | passed | E-001/E-SET-001 adds only ECStore layout skeleton boundaries and read-only set-layout contracts before file moves. |
|
||||
| Migration preservation | passed | Format distribution, local disk replacement, lock client mapping, public module paths, and runtime set behavior remain unchanged. |
|
||||
| Testing/verification | passed | Focused ECStore set-layout checks, compile checks, guards, formatting, diff hygiene, Rust risk scan, and full pre-commit passed. |
|
||||
|
||||
## Verification Notes
|
||||
|
||||
@@ -2274,6 +2288,18 @@ Passed before push:
|
||||
- `make pre-commit`: passed.
|
||||
- Three-expert review: passed.
|
||||
|
||||
- Issue #660 E-001/E-SET-001 current slice:
|
||||
- `cargo test -p rustfs-ecstore test_eset -- --nocapture`: passed.
|
||||
- `cargo check -p rustfs-ecstore -p rustfs -p rustfs-heal`: passed.
|
||||
- `./scripts/check_architecture_migration_rules.sh`: passed.
|
||||
- `./scripts/check_layer_dependencies.sh`: passed.
|
||||
- `cargo fmt --all --check`: passed.
|
||||
- `git diff --check`: passed.
|
||||
- Rust risk scan on changed Rust files: passed; only test-only
|
||||
expectation paths were present.
|
||||
- `make pre-commit`: passed.
|
||||
- Three-expert review: passed.
|
||||
|
||||
- Issue #660 X-012 current slice:
|
||||
- `cargo test -p rustfs-extension-schema`: passed.
|
||||
- `cargo check -p rustfs-extension-schema`: passed.
|
||||
|
||||
Reference in New Issue
Block a user