From f8aa4fa22158011a932adf4ed2e73bb59855ae82 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E5=AE=89=E6=AD=A3=E8=B6=85?= Date: Sun, 7 Jun 2026 14:43:55 +0800 Subject: [PATCH] docs: add architecture migration guardrails (#3253) --- .gitignore | 4 + ARCHITECTURE.md | 2 + docs/architecture/compat-cleanup-register.md | 26 ++++++ docs/architecture/crate-boundaries.md | 73 ++++++++++++++++ docs/architecture/migration-progress.md | 84 +++++++++++++++++++ docs/architecture/overview.md | 64 ++++++++++++++ docs/architecture/runtime-lifecycle.md | 38 +++++++++ .../storage-control-data-plane.md | 54 ++++++++++++ 8 files changed, 345 insertions(+) create mode 100644 docs/architecture/compat-cleanup-register.md create mode 100644 docs/architecture/crate-boundaries.md create mode 100644 docs/architecture/migration-progress.md create mode 100644 docs/architecture/overview.md create mode 100644 docs/architecture/runtime-lifecycle.md create mode 100644 docs/architecture/storage-control-data-plane.md diff --git a/.gitignore b/.gitignore index 18d03f8a6..a94697c58 100644 --- a/.gitignore +++ b/.gitignore @@ -42,6 +42,10 @@ artifacts/ PR_DESCRIPTION.md scripts/s3-tests/selected_tests.txt docs +!docs/ +docs/* +!docs/architecture/ +!docs/architecture/** # nix stuff result* diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 68760811a..33af31a2b 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -6,6 +6,8 @@ > If you want to familiarize yourself with the code base, you are in the right place! > > See also [CONTRIBUTING.md](CONTRIBUTING.md) for development workflow. +> See also [docs/architecture](docs/architecture/overview.md) for active +> architecture migration guardrails. ## Bird's Eye View diff --git a/docs/architecture/compat-cleanup-register.md b/docs/architecture/compat-cleanup-register.md new file mode 100644 index 000000000..582e61fdd --- /dev/null +++ b/docs/architecture/compat-cleanup-register.md @@ -0,0 +1,26 @@ +# Compatibility Cleanup Register + +Use this file to track temporary compatibility code introduced by architecture +migration PRs. Entries are required only for compatibility paths that are planned +for later deletion. + +## Required Source Marker + +```rust +// RUSTFS_COMPAT_TODO(): . Remove after . +``` + +## Open Items + +No compatibility code has been introduced in this branch. + +## Review Checklist + +Before completing a PR that adds wrappers, re-exports, fallbacks, legacy action +mappings, or old endpoint compatibility layers: + +- [ ] The source has a `RUSTFS_COMPAT_TODO()` marker. +- [ ] This register has a matching entry. +- [ ] The entry states why compatibility is needed. +- [ ] The entry states the exact removal condition. +- [ ] The cleanup is not bundled with new migration logic. diff --git a/docs/architecture/crate-boundaries.md b/docs/architecture/crate-boundaries.md new file mode 100644 index 000000000..73a8f8b66 --- /dev/null +++ b/docs/architecture/crate-boundaries.md @@ -0,0 +1,73 @@ +# Crate Boundaries And Migration Guardrails + +These rules apply to architecture-migration PRs linked to +[`rustfs/backlog#660`](https://github.com/rustfs/backlog/issues/660). + +## PR Types + +Every PR must declare exactly one type: + +- `docs-only` +- `test-only` +- `contract` +- `api-extraction` +- `pure-move` +- `consumer-migration` +- `dependency-migration` +- `security-change` +- `behavior-change` +- `ci-gate` + +Do not mix directory movement, security tightening, and behavior changes in one PR. + +## Dependency Direction + +Contract crates must stay below implementation crates. Initial forbidden edges: + +- `storage-api -> ecstore` +- `security-governance -> rustfs` +- `extension-schema -> rustfs` +- `extension-schema -> ecstore` + +Existing layer checks live in `scripts/check_layer_dependencies.sh`. The next +`ci-gate` PR should extend existing guardrails instead of adding a parallel system. + +## Pre-Push Expert Review + +Before pushing any PR branch, record three expert reviews in the task notes: + +| Expert | Required focus | +|---|---| +| Quality/architecture | Structure, naming, dependency direction, PR type, scope, and over-abstraction risk | +| Migration preservation | Startup order, readiness, quorum, reader semantics, AppContext/global fallback, notify/audit lifecycle, IAM/KMS boundaries, and compatibility | +| Testing/verification | Focused tests, regression tests, commands run, missing coverage, and whether tests are forcing business-logic drift | + +Push is allowed only when all three experts return `pass` or +`pass-with-nonblocking-follow-up`. Any `blocker` prevents push until the issue is +fixed and the relevant review is repeated. + +## Temporary Compatibility Code + +Temporary compatibility code that must be removed later must include a searchable +source comment and a cleanup-register entry. + +Use this source-comment format: + +```rust +// RUSTFS_COMPAT_TODO(API-005): keep old ecstore::store_api path during storage-api migration. Remove after all consumers use rustfs-storage-api. +``` + +Rules: + +- Add the marker only to temporary compatibility paths, not permanent APIs. +- Include the task ID in the marker. +- State why the compatibility path exists and when it can be removed. +- Use this for temporary re-exports, wrappers, fallbacks, legacy action mappings, + and old endpoint compatibility layers. +- Delete compatibility layers in their own cleanup PR. + +## Config Model First + +`ecstore::config::{Config, KV, KVS}` should move before extension config adapters +or config-schema work. First inventory consumers, then decide whether existing +`crates/config` is enough or whether a smaller model crate is required. diff --git a/docs/architecture/migration-progress.md b/docs/architecture/migration-progress.md new file mode 100644 index 000000000..765e89fc2 --- /dev/null +++ b/docs/architecture/migration-progress.md @@ -0,0 +1,84 @@ +# Architecture Migration Progress + +Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` blocked. + +## Current Context + +- Issue: [`rustfs/backlog#660`](https://github.com/rustfs/backlog/issues/660) +- Branch: `overtrue/arch-migration-guardrails` +- Baseline: `upstream/main` at `61f0dfbc40f748be313be84d834d8259cf3e19c9` +- PR type for this branch: `docs-only` +- Runtime behavior changes: none +- Rust code changes: none +- Repository metadata changes: `.gitignore` now allows tracking only + `docs/architecture/` under the otherwise ignored `docs` tree. + +## Phase 0 Tasks + +- [x] `G-001` Refresh `main` and record baseline. + - Acceptance: baseline commit, title, and branch are recorded. + - Verification: `git fetch upstream main --prune`; `git rev-parse upstream/main`. +- [x] `G-002` Create migration tracking checklist. + - Acceptance: this file records task state, context, verification, and handoff. +- [x] `G-003` Classify PR types. + - Acceptance: [`crate-boundaries.md`](crate-boundaries.md) lists exactly one + allowed PR type per PR. +- [x] `G-004` Define re-export and wrapper policy. + - Acceptance: temporary compatibility code must use `RUSTFS_COMPAT_TODO`. +- [~] `G-005` Add dependency direction guard. + - Current branch: documentation only. + - Next PR: add a `ci-gate` check for forbidden dependency edges. +- [~] `G-006` Create migration loss-prevention checks. + - Current branch: documentation only. + - Next PR: add checks for public re-export, route matrix, and storage trait + coverage before pure moves. +- [x] `G-009` Enforce pre-push three-expert review. + - Acceptance: [`crate-boundaries.md`](crate-boundaries.md) requires + quality/architecture, migration-preservation, and testing/verification review + before push. +- [~] `TEST-PRTYPE-001` Check PR type enum consistency. + - Current branch: documentation only. + - Next PR: add a mechanical check that all migration docs use the same PR type + vocabulary. + +## Next PRs + +1. `ci-gate`: extend `scripts/check_layer_dependencies.sh` or add a nearby check + for architecture-migration guardrails. +2. `docs-only` or `test-only`: capture startup timeline and admin route-action + snapshot. +3. `docs-only`: inventory `ecstore::config::{Config, KV, KVS}` consumers. + +## Pre-Push Review Log + +| Expert | Status | Notes | +|---|---|---| +| Quality/architecture | pass | Staged diff is scoped to docs, `.gitignore`, and the root architecture index; no over-abstraction or target drift found | +| Migration preservation | pass | No runtime files changed; notify/audit and IAM/KMS no-drift bullets were added after review feedback | +| Testing/verification | pass-with-nonblocking-follow-up | Docs-only verification is sufficient; suggested commands are recorded below | + +## Verification Notes + +Passed: + +- `git diff --cached --check` +- `git diff --cached --name-only -- '*.rs' 'Cargo.toml' 'Cargo.lock' '.github/**' 'scripts/**' 'Makefile' 'Justfile'` +- `git diff --cached --exit-code -- '*.rs' 'Cargo.toml' 'Cargo.lock' '.github/**' 'scripts/**' 'Makefile' 'Justfile'` +- `printf '%s\n' docs/architecture/overview.md docs/foo.md docs/other/file.md | git check-ignore -v --stdin --no-index` +- `git rev-parse upstream/main` +- `git log -1 --format='%H %s' upstream/main` +- `./scripts/check_metrics_migration_refs.sh` + +Known unrelated baseline issue: + +- `./scripts/check_layer_dependencies.sh` currently fails on `upstream/main` + because the script output and `scripts/layer-dependency-baseline.txt` format are + out of sync. Keep that fix in the next `ci-gate` PR. + +## Handoff Notes + +- Keep Phase 0 PRs small. Do not start Config, Storage API, Runtime, or ECStore + movement inside this docs branch. +- Keep CI checks in a separate `ci-gate` PR so the PR type rule remains enforceable. +- Do not add temporary compatibility code without a matching + `RUSTFS_COMPAT_TODO()` marker and cleanup-register entry. diff --git a/docs/architecture/overview.md b/docs/architecture/overview.md new file mode 100644 index 000000000..2264f75a5 --- /dev/null +++ b/docs/architecture/overview.md @@ -0,0 +1,64 @@ +# RustFS Architecture Evolution + +This document set tracks the architecture migration from +[`rustfs/backlog#660`](https://github.com/rustfs/backlog/issues/660). + +## Baseline + +- Baseline branch: `upstream/main` +- Baseline commit: `61f0dfbc40f748be313be84d834d8259cf3e19c9` +- Baseline title: `fix(ecstore): invalidate wiped disk id cache (#3251)` +- First migration PR type: `docs-only` + +## Core Principle + +Cut wrong dependency directions with directories and contracts first, migrate global +state in small steps next, and split crates only after boundaries are stable. Storage +hot-path behavior must not drift during this migration. + +## Architecture Documents + +- [`runtime-lifecycle.md`](runtime-lifecycle.md): runtime, AppContext, + startup/readiness, and shutdown contracts. +- [`storage-control-data-plane.md`](storage-control-data-plane.md): boundaries + between StorageCore, ECStore, ClusterControlPlane, and BackgroundControllers. +- [`crate-boundaries.md`](crate-boundaries.md): PR types, crate direction, + compatibility rules, and migration guardrails. +- [`migration-progress.md`](migration-progress.md): current task state and context + handoff. +- [`compat-cleanup-register.md`](compat-cleanup-register.md): temporary + compatibility code that must be removed later. + +## Phase Order + +```mermaid +flowchart LR + G["Phase 0: Baseline and guardrails"] + CFG["Phase 1a: Config model"] + SEC["Phase 1: Security governance"] + API["Phase 2: Storage API contracts"] + RT["Phase 3: Runtime and lifecycle"] + EC["Phase 4: ECStore internal layout"] + CP["Phase 5: Cluster control plane"] + EXT["Phase 6: Extension plane"] + GS["Phase 7: Global-state reduction"] + CR["Crate split evaluation"] + + G --> CFG + G --> SEC + G --> API + G --> RT + CFG --> EXT + API --> EC + RT --> GS + EC --> CP + EXT --> CR + GS --> CR +``` + +The first implementation sequence is conservative: + +1. Record baseline and migration context. +2. Establish PR and compatibility rules. +3. Add dependency and loss-prevention checks in a separate `ci-gate` PR. +4. Inventory `ecstore::config::{Config, KV, KVS}` before moving any code. diff --git a/docs/architecture/runtime-lifecycle.md b/docs/architecture/runtime-lifecycle.md new file mode 100644 index 000000000..1bc72bda8 --- /dev/null +++ b/docs/architecture/runtime-lifecycle.md @@ -0,0 +1,38 @@ +# Runtime And Lifecycle Contracts + +Runtime and lifecycle work must preserve startup ordering, readiness behavior, and +shutdown semantics. + +## Startup And Readiness + +- HTTP can listen early, but normal requests must remain behind readiness gates. +- `FullReady = storage_ready && iam_ready && lock_quorum_ready`. +- Boot phases must keep the old fatal and non-fatal boundaries. +- AppContext migration keeps context-first lookup with global fallback until the + global path is proven unused. +- Notify and audit lifecycle behavior must not drift during lifecycle movement. +- IAM and KMS startup, deferred recovery, and fatal boundary behavior must not be + changed by pure movement PRs. + +## Service Registry Scope + +`ServiceRegistry` is only for lifecycle and shutdown ordering. It must not become a +general dependency injection container. + +Allowed responsibilities: + +- Register start and stop order. +- Expose read-only status snapshots. +- Coordinate graceful shutdown. + +Disallowed responsibilities: + +- Construct arbitrary dependencies for business logic. +- Hide globals behind a service-locator API. +- Change startup side effects while moving code. + +## AppContext Foundation + +Early AppContext work should split resolver files and add compatibility tests before +boot extraction or consumer migration. This keeps the migration context-first while +preserving the old global fallback path during transition. diff --git a/docs/architecture/storage-control-data-plane.md b/docs/architecture/storage-control-data-plane.md new file mode 100644 index 000000000..cff608c81 --- /dev/null +++ b/docs/architecture/storage-control-data-plane.md @@ -0,0 +1,54 @@ +# Storage, Control Plane, And Background Controllers + +This document defines migration boundaries for the storage hot path and adjacent +control-plane responsibilities. + +## Storage API Contracts + +Storage API contracts must not absorb implementation details from ECStore or the +reader pipeline. + +Out of scope for the contract layer: + +- KMS/SSE implementation. +- Range and compression behavior. +- Erasure coding and bitrot logic. +- Remote disk transport and recovery. + +No-drift behavior: + +- Object-to-set hash remains unchanged. +- Write quorum remains unchanged. +- Reader decryption, etag/checksum, version, and delete-marker behavior remain + unchanged. +- Public compatibility paths remain available through temporary re-exports or + wrappers during pure moves. + +## Cluster Control Plane + +ClusterControlPlane starts as a read-only facade inside `crates/ecstore/src/cluster`. +Do not create a standalone cluster crate until internal dependencies are stable. + +Initial scope: + +- Topology snapshot. +- Membership snapshot. +- Lock registry snapshot. +- Peer health snapshot. +- Pool state snapshot. + +Risk controls: + +- Distributed lock quorum remains per set. +- RemoteDisk suspect/offline/recovery, timeout, and connection eviction semantics + must not be simplified. +- Health impact behavior must be feature-gated if it changes production behavior. + +## Background Controllers + +Scanner, heal, lifecycle, replication, config reload, metrics, and auto-tuning +controllers should move behind explicit controller boundaries after lifecycle +contracts are stable. + +The first controller work should be read-only status and shutdown ordering, not +behavior changes.