mirror of
https://github.com/rustfs/rustfs.git
synced 2026-08-29 00:17:11 +00:00
a6ea4ac8f3
Mechanical move-only extraction for backlog#1840 PR1+PR4: the site-replication state (load/parse/persist/RMW transaction), repair state machine, peer transport (client cache, DNS resolver, send_peer_* family), retry queue, and the four storage-side hooks move from rustfs/src/admin/handlers/site_replication.rs into the new infra-layer module rustfs/src/site_replication/ ({mod,state,state_lock,identity,transport,retry,repair,hooks}.rs). The admin handler file keeps route registration, all Operation impls, request/response glue, and the in-file test module, and re-exports the moved items so existing paths keep resolving. admin/site_replication_identity.rs and admin/site_replication_state.rs relocate wholesale as identity.rs/state_lock.rs.
Storage access from the moved code goes through a new site_replication consumer module in the root facade (rustfs/src/storage_api.rs), including an s3 shim so the module stays off the direct s3s surface (file count stays at the 215 baseline). The three admin runtime-source wrappers the moved code needs (outbound TLS generation incl. the test atomic, outbound TLS state, runtime port) are reproduced locally; the TLS-generation trio moves out of admin/runtime_sources.rs since site replication was its only consumer. The one non-verbatim rewrite: site_replication_peer_payload inlines encrypt_stream_io in its encrypted branch, which is provably the branch encode_compatible_admin_payload always took for the /minio/admin peer-join wire path.
app/bucket_usecase.rs now imports the three bucket hooks from crate::site_replication, deleting the three app->interface entries from the layer baseline (shrink-only). The peer-client cache test moves with the owner-local SITE_REPLICATION_PEER_CLIENT static into transport.rs (228+1 = 229 tests conserved). New module files are added to the logging-guardrail checked list; the s3_error! line baseline tightens 1620 -> 1619; global-state/config-consumer inventories and ARCHITECTURE.md pointers updated.
Verified: cargo check -p rustfs --all-targets clean; cargo clippy --workspace --all-targets clean; cargo nextest run -p rustfs --lib 3852/3852 passed; make pre-commit green; scripts/check_layer_dependencies.sh green with baseline-only deletions; line-multiset conservation audit over the moved code accounts for every non-verbatim line (visibility bumps, import rewrites, fmt reflow).
Refs rustfs/backlog#1840
13 KiB
13 KiB
ECStore Config Consumer Inventory
This inventory is the Phase 0 baseline for moving
rustfs_ecstore::config::{Config, KV, KVS} safely. It records the current
definitions, persistence helpers, global accessors, and direct consumers before
any contract extraction, global-state migration, or crate split.
Related issue: rustfs/backlog#660
Scope
In scope:
rustfs_ecstore::config::KVrustfs_ecstore::config::KVSrustfs_ecstore::config::Configrustfs_ecstore::config::DEFAULT_KVSrustfs_ecstore::config::{get_global_server_config, set_global_server_config}rustfs_ecstore::config::com::{read_config_without_migrate, save_server_config}- Consumers that persist, clone, inspect, mutate, or pass these types across runtime boundaries.
- Selected adjacent users of
rustfs_ecstore::config::com::{read_config, save_config, delete_config}and related helper variants are listed separately when they appear outside the coreConfig,KV, andKVSconsumer map. This is not a completecom.rsmove inventory; any futurecom.rsmove must first inventory ECStore-internal persistence helper users too.
Out of scope:
- Unrelated
Configtypes fromrustfs::config, SDKs, TLS, SSH, KMS, OIDC client libraries, or local module-specific config structs. - Storage-class-only imports are not treated as
Config,KV, orKVSconsumers unless they also use the server-config model. - Pure route/action snapshot work already covered by
admin-route-action-snapshot.md.
Current Shape
Arrows show current source dependency or call direction: the left node imports or calls the right node.
flowchart TB
EC["crates/ecstore/src/config"]
Store["crates/ecstore/src/store/mod.rs"]
AppCtx["rustfs/src/app/context.rs"]
Server["rustfs/src/server/{event,audit}.rs"]
Admin["rustfs/src/admin"]
Notify["crates/notify"]
Audit["crates/audit"]
Targets["crates/targets"]
IAM["crates/iam/src/oidc.rs"]
Scanner["crates/scanner/src/{runtime_config,scanner}.rs"]
Store --> EC
AppCtx --> EC
Server --> AppCtx
Admin --> EC
Notify --> EC
Audit --> EC
Targets --> EC
Notify --> Targets
Audit --> Targets
IAM --> EC
Scanner --> EC
The config model is currently both a persisted server-config representation and the runtime carrier for notify, audit, target-plugin, scanner, and OIDC settings. Any move must preserve that dual role until consumers are migrated behind narrower contracts.
Core Model And Global State
| Item | Current owner | Current role | Migration note |
|---|---|---|---|
KV |
crates/ecstore/src/config/mod.rs |
Key/value entry with hidden_if_empty metadata and serde compatibility. |
Preserve field names, aliases, defaults, and redaction behavior before any model move. |
KVS(Vec<KV>) |
crates/ecstore/src/config/mod.rs |
Ordered key/value set used by server config, target factories, admin rendering, tests, and examples. | Preserve tuple shape and methods: new, get, lookup, is_empty, keys, insert, extend. |
Config(HashMap<String, HashMap<String, KVS>>) |
crates/ecstore/src/config/mod.rs |
Server config map by subsystem and target. | Direct .0 access is widespread; add wrappers only after preserving the current public shape. |
DEFAULT_KVS |
crates/ecstore/src/config/mod.rs |
Registry for defaults across storage class, scanner, notify, audit, and OIDC. | Move defaults only after an explicit registration contract exists. |
GLOBAL_SERVER_CONFIG |
crates/ecstore/src/config/mod.rs |
Process-wide mutable server config snapshot. | Migrate readers behind AppContext or a server-config provider before changing storage. |
ConfigSys::init |
crates/ecstore/src/config/mod.rs |
Reads persisted config, looks up derived config, and stores the global snapshot. | Startup order must remain unchanged until the lifecycle contract owns this dependency. |
read_config_without_migrate |
crates/ecstore/src/config/com.rs |
Loads persisted server config through ECStore-owned object I/O and storage-admin contracts. | Persistence stays in ecstore until pure model and persistence are separated. |
save_server_config |
crates/ecstore/src/config/com.rs |
Persists the canonical server config object. | Preserve external object shape and config-history behavior. |
get_global_server_config / set_global_server_config |
crates/ecstore/src/config/mod.rs |
Clone/read and replace the global server-config snapshot. | Do not remove until all runtime readers have an injected provider path. |
Consumer Map
ECStore Ownership, Persistence, And Defaults
| Files | Current usage |
|---|---|
crates/ecstore/src/config/mod.rs |
Defines KV, KVS, Config, defaults, global snapshot, initialization, and tests. |
crates/ecstore/src/config/com.rs |
Encodes, decodes, reads, writes, creates, and normalizes server config objects through ECStore-local persistence helpers. |
crates/ecstore/src/config/{notify,audit,oidc,scanner,storageclass}.rs |
Register default KVS values and subsystem-specific parsing helpers. |
crates/ecstore/src/store/mod.rs |
Exposes store-level server-config accessors that delegate to the global config snapshot. |
App Context And Server Startup Consumers
| Files | Current usage |
|---|---|
rustfs/src/app/context.rs |
Defines ServerConfigInterface, keeps an AppContext server-config handle, and still falls back to get_global_server_config. |
rustfs/src/server/event.rs |
Resolves server config through app context/global fallback before starting the notification runtime. |
rustfs/src/server/audit.rs |
Resolves server config through app context/global fallback before starting the audit runtime. |
Admin Control-Plane Readers And Writers
| Files | Current usage |
|---|---|
rustfs/src/admin/handlers/config_admin.rs |
Reads active/persisted server config, validates against DEFAULT_KVS, mutates KVS, saves config history, saves server config, and updates the global snapshot. |
rustfs/src/admin/handlers/oidc.rs |
Reads and writes OIDC provider KVS, saves server config, and compares persisted config against the global snapshot for restart signaling. |
rustfs/src/admin/handlers/audit_runtime_config.rs |
Reads persisted config, applies audit runtime target changes, saves server config, and reloads audit runtime state. |
rustfs/src/admin/handlers/notify_runtime_access.rs |
Reads notification runtime config snapshots and passes KVS target changes into the notification system. |
rustfs/src/admin/handlers/{event,audit}.rs |
Lists and validates notification/audit targets from Config; tests build KV and KVS fixtures. |
rustfs/src/admin/handlers/plugins_instances.rs |
Maps target plugin KVS to response payloads and applies runtime target edits. |
rustfs/src/admin/handlers/target_descriptor.rs |
Converts descriptor payloads into KVS for target plugin instances. |
rustfs/src/admin/handlers/site_replication.rs |
Reads global server config for LDAP settings and parses LDAP KVS fixtures. |
rustfs/src/admin/service/config.rs |
Reads persisted server config, validates storage-class KVS, derives target state, and updates global config/storage-class state. |
rustfs/src/admin/router.rs |
Reads persisted/global server config for admin route behavior; route tests construct Config, KV, and KVS. |
Adjacent ECStore Config-Object Helper Users
| Files | Current usage |
|---|---|
rustfs/src/admin/handlers/kms_dynamic.rs |
Uses generic read_config and save_config for dynamic KMS config objects. |
rustfs/src/site_replication/state.rs |
Uses generic read_config, save_config, and delete_config (via the root storage facade) for site-replication state objects. |
rustfs/src/admin/service/site_replication.rs |
Uses generic read_config and save_config for site-replication state normalization. |
rustfs/src/server/module_switch.rs |
Uses generic read_config and save_config for module-switch config objects. |
crates/iam/src/store/object.rs |
Uses generic read_config_no_lock, read_config_with_metadata, save_config, save_config_with_opts, and delete_config helper variants for IAM object-store persistence paths. |
crates/scanner/src/{scanner,data_usage_define}.rs |
Uses generic read_config and save_config for scanner metadata and cache persistence paths. |
Runtime Target, Notify, And Audit Crates
| Files | Current usage |
|---|---|
crates/notify/src/{global,integration,services,registry}.rs |
Carries Config into notification runtime startup/reload and target creation. |
crates/notify/src/config_manager.rs |
Mutates Config, reads persisted server config with read_config_without_migrate, persists changes with save_server_config, and applies per-target KVS updates. |
crates/notify/src/factory.rs |
Builds notification target arguments from KVS. |
crates/notify/examples/{full_demo,full_demo_one}.rs |
Constructs Config, KV, and KVS directly for examples. |
crates/audit/src/{global,system,registry}.rs |
Carries Config into audit runtime startup/reload and target creation. |
crates/audit/src/factory.rs |
Builds audit target arguments from KVS. |
crates/audit/tests/*.rs |
Constructs Config and KVS directly for runtime and parsing tests. |
crates/audit/README.md |
Documents current direct Config usage. |
crates/targets/src/plugin.rs |
Creates plugin targets from Config and merged KVS. |
crates/targets/src/catalog/builtin.rs |
Declares builtin target descriptors and default KVS fields. |
crates/targets/src/config/{common,target_args,loader,instance}.rs |
Collects, normalizes, redacts, and materializes target configs from Config and KVS, including environment overrides. |
Identity, Scanner, Tests, And Fixtures
| Files | Current usage |
|---|---|
crates/iam/src/oidc.rs |
Reads global server config and parses OIDC provider KVS. |
crates/scanner/src/{runtime_config,scanner}.rs |
Reads the global server-config snapshot and resolves scanner runtime config from Config and KVS. |
rustfs/src/admin handler/router tests, crates/audit/tests/*.rs, and selected in-crate tests in crates/{targets,scanner}/src |
Build direct tuple-struct fixtures; use them as candidate regression guards during a pure model move. |
Dependency Risk Classification
| Risk | Why it matters | Guardrail |
|---|---|---|
Config is both persistence model and runtime input |
A move can accidentally change persisted JSON/object shape or runtime target behavior. | Separate pure model contract from persistence helpers before moving com.rs. |
Direct .0 map access is common |
Replacing the tuple struct too early would create broad churn and likely behavior drift. | Preserve tuple shape in the first move, then add typed readers in later PRs. |
KVS is the effective target config carrier |
Notify, audit, and target factories consume KVS after file/env merge. |
Keep KVS API stable until target descriptor and runtime crates are behind a shared contract. |
DEFAULT_KVS registration is global |
Defaults are initialized centrally and used by admin validation/rendering. | Add a registration contract before changing initialization order. |
| Global snapshot readers still exist | Server, admin, IAM, scanner, and site-replication paths can still read global config. | Migrate readers through AppContext/provider paths in small steps after the model contract is stable. |
| Persistence helpers depend on ECStore storage contracts | Moving them with the pure model would pull storage implementation dependencies upward. | Keep read/write helpers in ecstore until a storage-facing persistence contract is explicit. |
Recommended Migration Order
- Keep this inventory current while Phase 0 guardrails land.
- Add a focused contract surface for
KV,KVS, andConfigwithout changing serialization, tuple-struct shape, or method names. - Add compile-time or scripted checks for temporary compatibility markers and config-model re-export coverage.
- Move only the pure model and defaults registration surface after targeted
regression checks cover unchanged persisted object shape, target
KVSmerge behavior, and representative admin config rendering paths. - Migrate global
Configreaders behindServerConfigInterfaceor a narrower provider in small PRs. - Move persistence helpers only after object-I/O and storage-admin dependencies can stay below the model contract.
- Evaluate crate split only after consumers no longer need old paths except
explicit
RUSTFS_COMPAT_TODO(<task-id>)compatibility shims.
Do-Not-Change Contract
The first migration steps must preserve:
KV { key, value, hidden_if_empty }serde behavior and redaction semantics.KVS(Vec<KV>)tuple shape and public methods.Config(HashMap<String, HashMap<String, KVS>>)tuple shape and public methods.Config::set_defaults,Config::unmarshal,Config::marshal, andConfig::mergebehavior.read_config_without_migratefallback/creation behavior for missing server config objects.save_server_configexternal object shape and config-history compatibility.- Existing notify, audit, scanner, OIDC, and target-plugin enable/disable
interpretation from
Config/KVSinputs; business-rule changes stay out of migration PRs. - AppContext/global fallback behavior until all readers are explicitly migrated.