mirror of
https://github.com/rustfs/rustfs.git
synced 2026-07-28 00:58:59 +00:00
170 lines
7.5 KiB
Markdown
170 lines
7.5 KiB
Markdown
# ECStore Module Split Plan
|
|
|
|
This plan records the remaining ECStore split work after the final audit
|
|
remediation pass. It is intentionally design-only: no crate movement should
|
|
start until each candidate boundary has explicit contracts, compatibility
|
|
coverage, dependency evidence, and rollback steps.
|
|
|
|
## Current Shape
|
|
|
|
| Area | Current owner | Size | Split status |
|
|
|---|---|---:|---|
|
|
| Bucket lifecycle | `crates/ecstore/src/bucket/lifecycle/` | 8,657 lines | Proposal only |
|
|
| Bucket replication | `crates/ecstore/src/bucket/replication/` | 8,730 lines | Proposal only |
|
|
| Set disks | `crates/ecstore/src/set_disk/` | state carrier plus operation modules | Keep in ECStore |
|
|
| Public ECStore facade | `crates/ecstore/src/api/mod.rs` | broad compatibility surface | Shrink only through guarded PRs |
|
|
|
|
The file split inside `set_disk/` is already operation-oriented: read, write,
|
|
list, multipart, lock, heal, and replication code live in separate modules.
|
|
The remaining large surface is the shared `SetDisks` state and cross-cutting
|
|
contracts, not only file layout.
|
|
|
|
## Non-Negotiable Rules
|
|
|
|
- Do not split crates in the same PR that moves runtime state or changes
|
|
startup behavior.
|
|
- Do not change object placement, quorum, reader semantics, lifecycle queues,
|
|
replication queues, notification dispatch, audit events, or scanner repair
|
|
behavior during inventory and contract PRs.
|
|
- Do not expose new direct ECStore internals to outer crates; use the existing
|
|
storage-api and owner-local facade boundaries.
|
|
- Keep `rustfs_ecstore::api` compatibility visible until each consumer path has
|
|
compile coverage and an explicit replacement.
|
|
|
|
## SetDisks Split Direction
|
|
|
|
Do not replace `SetDisks` with several runtime structs in one change. The safe
|
|
path is:
|
|
|
|
1. Keep `SetDisks` as the shared state carrier while operation modules continue
|
|
to own read/write/list/multipart/lock/heal/replication behavior.
|
|
2. Extract pure contracts first: shard source, disk error, bitrot IO, namespace
|
|
lock, metrics labels, and file metadata access.
|
|
3. Move one operation family only after its contracts are covered by focused
|
|
tests and the facade compatibility path is explicit.
|
|
4. Preserve the old `rustfs_ecstore::api::set_disk` surface until downstream
|
|
compatibility tests prove no caller depends on removed names.
|
|
|
|
The first executable SetDisks follow-up should be an inventory or guardrail PR,
|
|
not a runtime split PR.
|
|
|
|
## Lifecycle Candidate
|
|
|
|
`bucket/lifecycle` is not ready for a standalone crate yet.
|
|
|
|
Current coupling:
|
|
|
|
- lifecycle workers and transition state read ECStore runtime sources for
|
|
object-store handles, expiry state, transition state, tier config, deployment
|
|
IDs, and local node names;
|
|
- stale multipart cleanup depends on `SetDisks` internals and bucket metadata;
|
|
- lifecycle expiry schedules bucket replication delete work directly;
|
|
- lifecycle evaluation shares S3 DTOs, object metadata, object lock, replication
|
|
config, scanner metrics, notification/audit side effects, and tier services.
|
|
|
|
Required contracts before crate movement:
|
|
|
|
- `LifecycleObjectStore`: object stat, delete, transition, restore, multipart
|
|
cleanup, and version-aware metadata operations needed by lifecycle workers.
|
|
- `LifecycleMetadataStore`: lifecycle, object-lock, replication, bucket
|
|
versioning, and stale multipart metadata lookups without importing ECStore
|
|
implementation modules.
|
|
- `LifecycleRuntime`: expiry state, transition state, tier config, deployment
|
|
ID, local node name, queue metrics, cancellation, and worker sizing.
|
|
- `LifecycleReplicationSink`: schedule lifecycle-originated replication deletes
|
|
without depending on the replication implementation module.
|
|
- `LifecycleAuditSink`: lifecycle audit and notification emission boundary.
|
|
|
|
First safe PR:
|
|
|
|
- add a lifecycle extraction inventory section or module-level README;
|
|
- list current ECStore/runtime dependencies and the target contract owner for
|
|
each dependency;
|
|
- add no code movement and no behavior changes.
|
|
|
|
The module-level inventory lives in
|
|
`crates/ecstore/src/bucket/lifecycle/README.md`.
|
|
|
|
Focused verification for the first code-bearing lifecycle PR:
|
|
|
|
- `cargo test -p rustfs-ecstore lifecycle --lib`
|
|
- `cargo check -p rustfs-ecstore --tests`
|
|
- `./scripts/check_architecture_migration_rules.sh`
|
|
- `git diff --check`
|
|
|
|
## Replication Candidate
|
|
|
|
`bucket/replication` is not ready for a standalone crate yet.
|
|
|
|
Current coupling:
|
|
|
|
- replication workers depend on `ReplicationStorage`, ECStore object APIs,
|
|
bucket target clients, bucket metadata, file metadata replication state,
|
|
scanner repair classification, runtime replication pool/stat handles, bucket
|
|
monitor, local node names, and notification events;
|
|
- resync and delete replication paths call metadata and bucket target systems
|
|
directly;
|
|
- lifecycle and heal paths schedule replication work through the current ECStore
|
|
module;
|
|
- global replication pool/stat initialization still lives with ECStore runtime
|
|
compatibility state.
|
|
|
|
Required contracts before crate movement:
|
|
|
|
- `ReplicationStorage`: keep the existing trait as the starting point, then
|
|
split object read/write/delete, walk, and metadata update responsibilities
|
|
only when call sites prove a narrower shape.
|
|
- `ReplicationMetadataStore`: replication config, bucket targets, target reset
|
|
headers, MRF/resync state, and status persistence.
|
|
- `ReplicationRuntime`: pool, stats, worker admission, bucket monitor, local
|
|
node identity, cancellation, and queue sizing.
|
|
- `ReplicationEventSink`: notification/audit events for skipped, failed, and
|
|
completed replication operations.
|
|
- `ReplicationLifecycleBridge`: lifecycle-originated delete and version-purge
|
|
scheduling without importing lifecycle internals.
|
|
|
|
First safe PR:
|
|
|
|
- add a replication extraction inventory section or module-level README;
|
|
- list current ECStore/runtime dependencies and the target contract owner for
|
|
each dependency;
|
|
- keep global pool/stat initialization and queue behavior unchanged.
|
|
|
|
The module-level inventory lives in
|
|
`crates/ecstore/src/bucket/replication/README.md`.
|
|
|
|
Focused verification for the first code-bearing replication PR:
|
|
|
|
- `cargo test -p rustfs-ecstore replication --lib`
|
|
- `cargo check -p rustfs-ecstore --tests`
|
|
- `./scripts/check_architecture_migration_rules.sh`
|
|
- `git diff --check`
|
|
|
|
## Facade Shrink Plan
|
|
|
|
The broad `rustfs_ecstore::api` facade remains a compatibility boundary, not a
|
|
new architecture target. The current facade groups and external consumers are
|
|
recorded in
|
|
[`ecstore-api-facade-inventory.md`](ecstore-api-facade-inventory.md).
|
|
Shrinking it must be monotonic:
|
|
|
|
1. Inventory every public facade group and consumer.
|
|
2. Add compile-time coverage before removing or narrowing a facade item.
|
|
3. Move outer consumers to storage-api or owner-local compatibility boundaries.
|
|
4. Remove one facade group per PR only after downstream compatibility tests pass.
|
|
|
|
Do not delete facade groups only because the underlying module moved. Keep the
|
|
facade stable until the replacement path is visible and tested.
|
|
|
|
## Ready-To-Split Checklist
|
|
|
|
A candidate split is ready for code movement only when all items below are true:
|
|
|
|
- dependency graph shows no cycle with ECStore, storage-api, runtime sources, or
|
|
owner-local compatibility modules;
|
|
- contract traits compile without importing ECStore implementation modules;
|
|
- old facade names have compatibility tests or explicit deprecation coverage;
|
|
- focused tests cover the changed owner path before any full gate is attempted;
|
|
- rollback preserves object IO, quorum, lifecycle/replication queues, scanner
|
|
repair, notification/audit events, and metadata compatibility.
|