refactor(ecstore): add per-instance InstanceContext, migrate erasure setup type (#4413)

* refactor(ecstore): add per-instance InstanceContext, migrate erasure setup type

Phase 5 of the global-singleton consolidation (backlog#939): begin moving
runtime identity state out of process globals so multiple ECStore instances
can coexist in one process. Isolation is carried by the object graph
(ECStore -> Sets -> SetDisks holding an Arc<InstanceContext>), not a
task-local, which does not propagate across the many internal tokio::spawn
boundaries in the data/background paths.

This first slice migrates the erasure setup type -- previously three
independent process-global bools -- into a single per-instance
RwLock<SetupType> that derives is_erasure / is_dist_erasure / is_erasure_sd,
removing a triple source of truth that could drift out of sync.

- New runtime::instance module: InstanceContext + process bootstrap context.
- The legacy free-function facade (is_erasure/update_erasure_type/...) keeps
  its signatures and forwards to the current instance's context, falling back
  to the bootstrap context before a store is published.
- ECStore gains a pub(crate) ctx field and setup_is_* accessors; its
  constructors adopt the bootstrap context (never mint a fresh one) so startup
  writes and post-construction reads share one cell -- single-instance
  behavior is byte-for-byte unchanged.

Tests: erasure predicate derivation vs the legacy behavior, object-graph
carrier isolation across two ECStore instances, and bootstrap adoption.

Refs: backlog#939 (Phase 5, Slice 1), backlog#653 (item 8)

* refactor(ecstore): thread InstanceContext down the object graph (Phase 5 Slice 2) (#4415)

* refactor(ecstore): source the namespace lock manager per-instance (#4417)

refactor(ecstore): source the namespace lock manager per-instance (Phase 5 Slice 3)

Phase 5 Slice 3 (backlog#939): give each instance its own lock namespace by
sourcing SetDisks' lock manager from the instance context instead of the
process singleton. This removes the false cross-instance mutual exclusion (and
attendant ABBA risk) that a shared GlobalLockManager would cause once multiple
instances coexist.

- InstanceContext gains a `lock_manager: Arc<GlobalLockManager>`. `new()` mints
  a fresh manager (independent per-instance); `bootstrap_ctx()` aliases the
  process singleton via get_global_lock_manager(), so a single-instance
  deployment keeps exactly one shared namespace.
- SetDisks::new sources `local_lock_manager` from `ctx.lock_manager()` (the ctx
  it already adopts), not `runtime_sources::global_lock_manager()`. Single
  instance: same Arc as before, so behavior is unchanged.
- Remove the now-unused `runtime_sources::global_lock_manager()` wrapper.

Tests: bootstrap lock manager aliases the process singleton; two fresh contexts
own distinct managers; a SetDisks' lock manager is the one from its context and
aliases the global singleton in a single-instance build.

Verification: cargo test -p rustfs-ecstore (10 Phase 5 + set_disk locking
regressions green), cargo clippy -p rustfs-ecstore --all-targets (clean),
make pre-commit (pass).

Refs: backlog#939 (Phase 5, Slice 3). Stacked on #4415 (Slice 2).
This commit is contained in:
Zhengchao An
2026-07-08 15:01:40 +08:00
committed by GitHub
parent cda7688909
commit 91dec123d9
10 changed files with 444 additions and 37 deletions
+40
View File
@@ -1967,9 +1967,49 @@ mod tests {
decommission_cancelers: RwLock::new(Vec::new()),
start_gate: Mutex::new(()),
pool_meta_save_gate: Mutex::new(()),
ctx: crate::runtime::instance::bootstrap_ctx(),
}
}
// Phase 5 Slice 2 (backlog#939): the instance context flows down the whole
// object graph — ECStore, its Sets, and their SetDisks must all carry the
// same `Arc<InstanceContext>` in a single-instance deployment.
#[tokio::test]
async fn instance_context_flows_through_object_graph() {
let store = new_read_lock_test_store().await;
let sets = store.pools.first().expect("test store has one pool");
assert!(
std::sync::Arc::ptr_eq(&store.ctx, sets.instance_ctx()),
"Sets must carry the store's instance context"
);
let set_disks = sets.disk_set.first().expect("pool has one set");
assert!(
std::sync::Arc::ptr_eq(sets.instance_ctx(), set_disks.instance_ctx()),
"SetDisks must carry the Sets' instance context"
);
}
// Phase 5 Slice 3 (backlog#939): a SetDisks sources its lock manager from
// its instance context (not an independent process lookup), and in a
// single-instance build that context aliases the process lock-manager
// singleton — so the lock namespace is unchanged.
#[tokio::test]
async fn set_disks_lock_manager_comes_from_instance_context() {
let store = new_read_lock_test_store().await;
let set_disks = store.pools[0].disk_set.first().expect("pool has one set");
assert!(
std::sync::Arc::ptr_eq(set_disks.local_lock_manager_for_test(), &set_disks.instance_ctx().lock_manager()),
"SetDisks lock manager must be sourced from its instance context"
);
assert!(
std::sync::Arc::ptr_eq(set_disks.local_lock_manager_for_test(), &rustfs_lock::get_global_lock_manager()),
"single-instance lock manager must alias the process singleton"
);
}
#[tokio::test]
async fn acquired_read_lock_marks_metadata_cache_safe_for_set_layer() {
let store = new_read_lock_test_store().await;