gsadmin/rustfs
1
0
Fork 0
mirror of https://github.com/rustfs/rustfs.git synced 2026-07-26 08:18:18 +00:00
Code Issues Packages Projects Releases Wiki Activity

docs: inventory ecstore config consumers (#3259)

Browse Source
This commit is contained in:
安正超
2026-06-07 21:44:55 +08:00
committed by GitHub
parent 241e45d3b0
commit 3c71d5ef1c
3 changed files with 229 additions and 17 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/architecture/ecstore-config-consumer-inventory.md
+195
View File
@@ -0,0 +1,195 @@
# ECStore Config Consumer Inventory
This inventory is the Phase 0 baseline for moving
`rustfs_ecstore::config::{Config, KV, KVS}` safely. It records the current
definitions, persistence helpers, global accessors, and direct consumers before
any contract extraction, global-state migration, or crate split.
Related issue: [`rustfs/backlog#660`](https://github.com/rustfs/backlog/issues/660)
## Scope
In scope:
- `rustfs_ecstore::config::KV`
- `rustfs_ecstore::config::KVS`
- `rustfs_ecstore::config::Config`
- `rustfs_ecstore::config::DEFAULT_KVS`
- `rustfs_ecstore::config::{get_global_server_config, set_global_server_config}`
- `rustfs_ecstore::config::com::{read_config_without_migrate, save_server_config}`
- Consumers that persist, clone, inspect, mutate, or pass these types across
runtime boundaries.
- Adjacent users of `rustfs_ecstore::config::com::{read_config, save_config,
delete_config}` are listed separately because moving `com.rs` would affect
them even when they do not consume `Config`, `KV`, or `KVS`.
Out of scope:
- Unrelated `Config` types from `rustfs::config`, SDKs, TLS, SSH, KMS, OIDC
client libraries, or local module-specific config structs.
- Storage-class-only imports are not treated as `Config`, `KV`, or `KVS`
consumers unless they also use the server-config model.
- Pure route/action snapshot work already covered by
[`admin-route-action-snapshot.md`](admin-route-action-snapshot.md).
## Current Shape
Arrows show current source dependency or call direction: the left node imports
or calls the right node.
```mermaid
flowchart TB
EC["crates/ecstore/src/config"]
Store["crates/ecstore/src/store.rs"]
AppCtx["rustfs/src/app/context.rs"]
Server["rustfs/src/server/{event,audit}.rs"]
Admin["rustfs/src/admin"]
Notify["crates/notify"]
Audit["crates/audit"]
Targets["crates/targets"]
IAM["crates/iam/src/oidc.rs"]
Scanner["crates/scanner/src/{runtime_config,scanner}.rs"]
Store --> EC
AppCtx --> EC
Server --> AppCtx
Admin --> EC
Notify --> EC
Audit --> EC
Targets --> EC
Notify --> Targets
Audit --> Targets
IAM --> EC
Scanner --> EC
```
The config model is currently both a persisted server-config representation and
the runtime carrier for notify, audit, target-plugin, scanner, and OIDC
settings. Any move must preserve that dual role until consumers are migrated
behind narrower contracts.
## Core Model And Global State
| Item | Current owner | Current role | Migration note |
|---|---|---|---|
| `KV` | `crates/ecstore/src/config/mod.rs` | Key/value entry with `hidden_if_empty` metadata and serde compatibility. | Preserve field names, aliases, defaults, and redaction behavior before any model move. |
| `KVS(Vec<KV>)` | `crates/ecstore/src/config/mod.rs` | Ordered key/value set used by server config, target factories, admin rendering, tests, and examples. | Preserve tuple shape and methods: `new`, `get`, `lookup`, `is_empty`, `keys`, `insert`, `extend`. |
| `Config(HashMap<String, HashMap<String, KVS>>)` | `crates/ecstore/src/config/mod.rs` | Server config map by subsystem and target. | Direct `.0` access is widespread; add wrappers only after preserving the current public shape. |
| `DEFAULT_KVS` | `crates/ecstore/src/config/mod.rs` | Registry for defaults across storage class, scanner, notify, audit, and OIDC. | Move defaults only after an explicit registration contract exists. |
| `GLOBAL_SERVER_CONFIG` | `crates/ecstore/src/config/mod.rs` | Process-wide mutable server config snapshot. | Migrate readers behind `AppContext` or a server-config provider before changing storage. |
| `ConfigSys::init` | `crates/ecstore/src/config/mod.rs` | Reads persisted config, looks up derived config, and stores the global snapshot. | Startup order must remain unchanged until the lifecycle contract owns this dependency. |
| `read_config_without_migrate` | `crates/ecstore/src/config/com.rs` | Loads persisted server config through `StorageAPI`. | Persistence stays in `ecstore` until pure model and persistence are separated. |
| `save_server_config` | `crates/ecstore/src/config/com.rs` | Persists the canonical server config object. | Preserve external object shape and config-history behavior. |
| `get_global_server_config` / `set_global_server_config` | `crates/ecstore/src/config/mod.rs` | Clone/read and replace the global server-config snapshot. | Do not remove until all runtime readers have an injected provider path. |
## Consumer Map
### ECStore Ownership, Persistence, And Defaults
| Files | Current usage |
|---|---|
| `crates/ecstore/src/config/mod.rs` | Defines `KV`, `KVS`, `Config`, defaults, global snapshot, initialization, and tests. |
| `crates/ecstore/src/config/com.rs` | Encodes, decodes, reads, writes, creates, and normalizes server config objects through `StorageAPI`. |
| `crates/ecstore/src/config/{notify,audit,oidc,scanner,storageclass}.rs` | Register default `KVS` values and subsystem-specific parsing helpers. |
| `crates/ecstore/src/store.rs` | Exposes store-level server-config accessors that delegate to the global config snapshot. |
### App Context And Server Startup Consumers
| Files | Current usage |
|---|---|
| `rustfs/src/app/context.rs` | Defines `ServerConfigInterface`, keeps an `AppContext` server-config handle, and still falls back to `get_global_server_config`. |
| `rustfs/src/server/event.rs` | Resolves server config through app context/global fallback before starting the notification runtime. |
| `rustfs/src/server/audit.rs` | Resolves server config through app context/global fallback before starting the audit runtime. |
### Admin Control-Plane Readers And Writers
| Files | Current usage |
|---|---|
| `rustfs/src/admin/handlers/config_admin.rs` | Reads active/persisted server config, validates against `DEFAULT_KVS`, mutates `KVS`, saves config history, saves server config, and updates the global snapshot. |
| `rustfs/src/admin/handlers/oidc.rs` | Reads and writes OIDC provider `KVS`, saves server config, and compares persisted config against the global snapshot for restart signaling. |
| `rustfs/src/admin/handlers/audit_runtime_config.rs` | Reads persisted config, applies audit runtime target changes, saves server config, and reloads audit runtime state. |
| `rustfs/src/admin/handlers/notify_runtime_access.rs` | Reads notification runtime config snapshots and passes `KVS` target changes into the notification system. |
| `rustfs/src/admin/handlers/{event,audit}.rs` | Lists and validates notification/audit targets from `Config`; tests build `KV` and `KVS` fixtures. |
| `rustfs/src/admin/handlers/plugins_instances.rs` | Maps target plugin `KVS` to response payloads and applies runtime target edits. |
| `rustfs/src/admin/handlers/target_descriptor.rs` | Converts descriptor payloads into `KVS` for target plugin instances. |
| `rustfs/src/admin/handlers/site_replication.rs` | Reads global server config for LDAP settings and parses LDAP `KVS` fixtures. |
| `rustfs/src/admin/service/config.rs` | Reads persisted server config, validates storage-class `KVS`, derives target state, and updates global config/storage-class state. |
| `rustfs/src/admin/router.rs` | Reads persisted/global server config for admin route behavior; route tests construct `Config`, `KV`, and `KVS`. |
### Adjacent ECStore Config-Object Helper Users
| Files | Current usage |
|---|---|
| `rustfs/src/admin/handlers/kms_dynamic.rs` | Uses generic `read_config` and `save_config` for dynamic KMS config objects. |
| `rustfs/src/admin/handlers/site_replication.rs` | Uses generic `read_config`, `save_config`, and `delete_config` for site-replication state objects. |
| `rustfs/src/admin/service/site_replication.rs` | Uses generic `read_config` and `save_config` for site-replication state normalization. |
| `crates/scanner/src/{scanner,data_usage_define}.rs` | Uses generic `read_config` and `save_config` for scanner metadata and cache persistence paths. |
### Runtime Target, Notify, And Audit Crates
| Files | Current usage |
|---|---|
| `crates/notify/src/{global,integration,services,registry}.rs` | Carries `Config` into notification runtime startup/reload and target creation. |
| `crates/notify/src/config_manager.rs` | Mutates `Config`, reads persisted server config with `read_config_without_migrate`, persists changes with `save_server_config`, and applies per-target `KVS` updates. |
| `crates/notify/src/factory.rs` | Builds notification target arguments from `KVS`. |
| `crates/notify/examples/{full_demo,full_demo_one}.rs` | Constructs `Config`, `KV`, and `KVS` directly for examples. |
| `crates/audit/src/{global,system,registry}.rs` | Carries `Config` into audit runtime startup/reload and target creation. |
| `crates/audit/src/factory.rs` | Builds audit target arguments from `KVS`. |
| `crates/audit/tests/*.rs` | Constructs `Config` and `KVS` directly for runtime and parsing tests. |
| `crates/audit/README.md` | Documents current direct `Config` usage. |
| `crates/targets/src/plugin.rs` | Creates plugin targets from `Config` and merged `KVS`. |
| `crates/targets/src/catalog/builtin.rs` | Declares builtin target descriptors and default `KVS` fields. |
| `crates/targets/src/config/{common,target_args,loader,instance}.rs` | Collects, normalizes, redacts, and materializes target configs from `Config` and `KVS`, including environment overrides. |
### Identity, Scanner, Tests, And Fixtures
| Files | Current usage |
|---|---|
| `crates/iam/src/oidc.rs` | Reads global server config and parses OIDC provider `KVS`. |
| `crates/scanner/src/{runtime_config,scanner}.rs` | Reads the global server-config snapshot and resolves scanner runtime config from `Config` and `KVS`. |
| `rustfs/src/admin` handler/router tests, `crates/audit/tests/*.rs`, and selected in-crate tests in `crates/{targets,scanner}/src` | Build direct tuple-struct fixtures; use them as candidate regression guards during a pure model move. |
## Dependency Risk Classification
| Risk | Why it matters | Guardrail |
|---|---|---|
| `Config` is both persistence model and runtime input | A move can accidentally change persisted JSON/object shape or runtime target behavior. | Separate pure model contract from persistence helpers before moving `com.rs`. |
| Direct `.0` map access is common | Replacing the tuple struct too early would create broad churn and likely behavior drift. | Preserve tuple shape in the first move, then add typed readers in later PRs. |
| `KVS` is the effective target config carrier | Notify, audit, and target factories consume `KVS` after file/env merge. | Keep `KVS` API stable until target descriptor and runtime crates are behind a shared contract. |
| `DEFAULT_KVS` registration is global | Defaults are initialized centrally and used by admin validation/rendering. | Add a registration contract before changing initialization order. |
| Global snapshot readers still exist | Server, admin, IAM, scanner, and site-replication paths can still read global config. | Migrate readers through `AppContext`/provider paths in small steps after the model contract is stable. |
| Persistence helpers depend on `StorageAPI` | Moving them with the pure model would pull storage implementation dependencies upward. | Keep read/write helpers in `ecstore` until a storage-facing persistence contract is explicit. |
## Recommended Migration Order
1. Keep this inventory current while Phase 0 guardrails land.
2. Add a focused contract surface for `KV`, `KVS`, and `Config` without changing
serialization, tuple-struct shape, or method names.
3. Add compile-time or scripted checks for temporary compatibility markers and
config-model re-export coverage.
4. Move only the pure model and defaults registration surface after targeted
regression checks cover unchanged persisted object shape, target `KVS` merge
behavior, and representative admin config rendering paths.
5. Migrate global `Config` readers behind `ServerConfigInterface` or a narrower
provider in small PRs.
6. Move persistence helpers only after `StorageAPI` dependencies can stay below
the model contract.
7. Evaluate crate split only after consumers no longer need old paths except
explicit `RUSTFS_COMPAT_TODO(<task-id>)` compatibility shims.
## Do-Not-Change Contract
The first migration steps must preserve:
- `KV { key, value, hidden_if_empty }` serde behavior and redaction semantics.
- `KVS(Vec<KV>)` tuple shape and public methods.
- `Config(HashMap<String, HashMap<String, KVS>>)` tuple shape and public methods.
- `Config::set_defaults`, `Config::unmarshal`, `Config::marshal`, and
`Config::merge` behavior.
- `read_config_without_migrate` fallback/creation behavior for missing server
config objects.
- `save_server_config` external object shape and config-history compatibility.
- Existing notify, audit, scanner, OIDC, and target-plugin enable/disable
interpretation from `Config`/`KVS` inputs; business-rule changes stay out of
migration PRs.
- AppContext/global fallback behavior until all readers are explicitly migrated.
docs/architecture/migration-progress.md
+30 -16
View File
@@ -5,13 +5,13 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block
## Current Context
- Issue: [`rustfs/backlog#660`](https://github.com/rustfs/backlog/issues/660)
- Branch: `overtrue/arch-admin-route-snapshot`
- Baseline: `upstream/main` at `0f9584c8d9351c757437405ee69a4e64bbcd94b5`
- Branch: `overtrue/arch-ecstore-config-inventory`
- Baseline: `upstream/main` at `241e45d3b040684e5d217d2dd95f84c72f1de316`
- PR type for this branch: `docs-only`
- Runtime behavior changes: none
- Rust code changes: none
- Docs changes: add the admin route/action snapshot baseline for later
admin module movement and route-matrix guard work.
- Docs changes: add the ECStore config consumer inventory for later config-model
contract, pure-move, and global-state migration work.
## Phase 0 Tasks
@@ -44,6 +44,11 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block
- Acceptance: [`crate-boundaries.md`](crate-boundaries.md) requires
quality/architecture, migration-preservation, and testing/verification review
before push.
- [x] `G-010` Inventory `ecstore::config::{Config, KV, KVS}` consumers.
- Acceptance:
[`ecstore-config-consumer-inventory.md`](ecstore-config-consumer-inventory.md)
records the current model definitions, global accessors, persistence helpers,
consumer groups, migration risks, and do-not-change contract.
- [~] `TEST-PRTYPE-001` Check PR type enum consistency.
- Current branch: not in scope.
- Next PR: add a mechanical check that all migration docs use the same PR type
@@ -51,36 +56,45 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block
## Next PRs
1. `docs-only`: inventory `ecstore::config::{Config, KV, KVS}` consumers.
2. `ci-gate`: add focused checks for PR type vocabulary and temporary
1. `ci-gate`: add focused checks for PR type vocabulary and temporary
compatibility marker/register consistency.
3. `test-only`: add a mechanical admin route matrix guard from the current
2. `test-only`: add a mechanical admin route matrix guard from the current
snapshot and `route_registration_test.rs`.
3. `contract`: define the config-model contract surface while preserving the
existing `Config`, `KV`, and `KVS` behavior.
## Pre-Push Review Log
| Expert | Status | Notes |
|---|---|---|
| Quality/architecture | pass | Final review confirmed route/action/public-exception rows are source-backed, including OIDC path-based bypass, console bypass, credential-only metrics/list-remote-targets, notification targets, table-catalog prefix, and site-replication edit |
| Migration preservation | pass | Final review confirmed this branch is docs-only, aligned with Phase 0, and does not touch runtime logic, storage hot paths, global state, compatibility implementation, or crate boundaries |
| Testing/verification | pass | Final review accepted docs-only verification with layer guard, metrics reference guard, diff checks, staged diff coverage, and future route-matrix handoff |
| Quality/architecture | pass | Re-review confirmed dependency/call direction arrows, scanner global readers, adjacent scanner persistence helpers, notify config-manager persistence, narrowed test wording, and `Config model contract` wording are source-backed |
| Migration preservation | pass | Confirmed this branch is docs-only, aligned with `rustfs/backlog#660`, and does not touch runtime logic, storage hot paths, global state implementation, compatibility code, scripts, or crate boundaries |
| Testing/verification | pass | Confirmed docs-only verification is sufficient after wording was narrowed and the final staged diff check covers all docs |
## Verification Notes
Passed:
Passed locally (docs-only):
- `./scripts/check_layer_dependencies.sh`
- `./scripts/check_metrics_migration_refs.sh`
- `git diff --check`
Final pre-push after staging all docs:
- `git diff --cached --check`
- focused source review of `rustfs/src/admin/mod.rs`,
`rustfs/src/admin/router.rs`, `rustfs/src/admin/route_registration_test.rs`,
and `rustfs/src/admin/handlers/*.rs` route/action declarations
- focused source review of `crates/ecstore/src/config/mod.rs`,
`crates/ecstore/src/config/com.rs`, `rustfs/src/app/context.rs`,
`rustfs/src/server/{event,audit}.rs`, `rustfs/src/admin/**/*.rs`,
`crates/{notify,audit,targets,iam,scanner}/**/*.rs`
- three-expert review: quality/architecture, migration preservation, and
testing/verification
## Handoff Notes
- Keep Phase 0 PRs small. Do not start Config, Storage API, Runtime, or ECStore
movement inside this `docs-only` branch.
- Keep Phase 0 PRs small. Do not move Config, Storage API, Runtime, or ECStore
code inside this `docs-only` branch.
- Keep CI checks in a separate `ci-gate` PR so the PR type rule remains enforceable.
- Do not add temporary compatibility code without a matching
`RUSTFS_COMPAT_TODO(<task-id>)` marker and cleanup-register entry.
- The next config-model PR must preserve the current tuple-struct shapes and
persistence behavior before introducing narrower provider contracts.
docs/architecture/overview.md
+4 -1
View File
@@ -24,6 +24,9 @@ hot-path behavior must not drift during this migration.
between StorageCore, ECStore, ClusterControlPlane, and BackgroundControllers.
- [`crate-boundaries.md`](crate-boundaries.md): PR types, crate direction,
compatibility rules, and migration guardrails.
- [`ecstore-config-consumer-inventory.md`](ecstore-config-consumer-inventory.md):
current `ecstore::config::{Config, KV, KVS}` definitions, consumers,
migration risks, and do-not-change contract.
- [`migration-progress.md`](migration-progress.md): current task state and context
handoff.
- [`compat-cleanup-register.md`](compat-cleanup-register.md): temporary
@@ -34,7 +37,7 @@ hot-path behavior must not drift during this migration.
```mermaid
flowchart LR
G["Phase 0: Baseline and guardrails"]
CFG["Phase 1a: Config model"]
CFG["Phase 1a: Config model contract"]
SEC["Phase 1: Security governance"]
API["Phase 2: Storage API contracts"]
RT["Phase 3: Runtime and lifecycle"]