refactor(ecstore): migrate the background-services cancel token into InstanceContext (Phase 5 Slice 13) (#4586)

* refactor(ecstore): migrate the background-services cancel token into InstanceContext (Phase 5 Slice 13)

Phase 5 Slice 13 (backlog#939): move the background-services cancellation token
out of the process static into the per-instance InstanceContext, so cancelling
one instance's background workers (scanner/heal/tier/lifecycle) no longer
touches another instance.

- InstanceContext gains `background_cancel_token: OnceLock<CancellationToken>`
  with `init_background_cancel_token` (set-once) and `background_cancel_token()`
  returning an owned clone.
- global.rs `init_/get_/create_/shutdown_` helpers keep their signatures and
  route through the current instance's context; the static is removed. The
  getter now returns an owned `Option<CancellationToken>` instead of a
  `Option<&'static _>`, which is what lets the token live in the context.
- Callers adapt to the owned token: the metadata-refresh loop drops `.cloned()`;
  the lifecycle worker/loops take the owned token (a shared fallback is cloned
  when the token is somehow uninitialized). No Arc cycle is introduced —
  workers hold a token clone, not the instance context.

Single-instance behavior is unchanged: startup creates one token in the
bootstrap context the ECStore adopts, and shutdown cancels that same token.

Tests: the token is set-once and cancelling one instance's token leaves a
distinct instance uninitialized.

Verification: cargo test -p rustfs-ecstore (23 instance-context tests green),
cargo clippy -p rustfs-ecstore --all-targets (clean), make pre-commit (pass).

Refs: backlog#939 (Phase 5, Slice 13)

* test(ecstore): prove multi-instance isolation; document embedded guard retention (Phase 5 Slice 14) (#4588)

Phase 5 (backlog#939) capstone. The prior 13 slices moved every piece of
per-instance runtime state out of process globals into ECStore's
InstanceContext. This slice proves the result and records the remaining work.

- Add `two_instances_isolate_all_migrated_state`: an end-to-end acceptance test
  that constructs two independent InstanceContexts and verifies NONE of the
  migrated state is shared — erasure setup, lock manager, region, deployment id,
  the four service handles (tier/notifier/expiry/transition), the local disk
  registry, the bucket monitor, and the background cancel token. This is the
  object-graph isolation carrier working end to end.
- Document why the embedded single-instance guard (EMBEDDED_SERVER_STARTED) is
  intentionally retained: storage startup still publishes into the process-level
  bootstrap context (write-once region/endpoints/deployment id) and the single
  GLOBAL_OBJECT_API handle, so a second startup would fail-fast on that shared
  state. Lifting the guard requires threading a per-instance context through
  startup — a follow-up beyond migrating the globals. The guard is NOT removed:
  rejecting the second start is safer than the panic it would otherwise become.

Verification: cargo test -p rustfs-ecstore (acceptance test + all instance-context
tests green), cargo clippy -p rustfs-ecstore --all-targets (clean), make
pre-commit (pass).

Refs: backlog#939 (Phase 5, Slice 14). Stacked on Slice 13 (#4586).
This commit is contained in:
Zhengchao An
2026-07-09 06:06:31 +08:00
committed by GitHub
parent c0e2c02e51
commit 359bdc0f1f
7 changed files with 119 additions and 18 deletions
+5 -8
View File
@@ -76,12 +76,9 @@ pub fn get_global_bucket_monitor() -> Option<Arc<Monitor>> {
current_ctx().bucket_monitor()
}
// Startup-owned process globals intentionally fail fast on duplicate writes.
// Startup-owned instance-context state intentionally fails fast on duplicate writes.
// A second write means startup published conflicting runtime scalar state.
/// Global cancellation token for background services (data scanner and auto heal)
static GLOBAL_BACKGROUND_SERVICES_CANCEL_TOKEN: OnceLock<CancellationToken> = OnceLock::new();
/// Get the global rustfs port
///
/// # Returns
@@ -297,7 +294,7 @@ pub fn get_global_region() -> Option<s3s::region::Region> {
/// * `Err(CancellationToken)` if setting fails
///
pub fn init_background_services_cancel_token(cancel_token: CancellationToken) -> Result<(), CancellationToken> {
GLOBAL_BACKGROUND_SERVICES_CANCEL_TOKEN.set(cancel_token)
current_ctx().init_background_cancel_token(cancel_token)
}
/// Get the global background services cancellation token
@@ -305,8 +302,8 @@ pub fn init_background_services_cancel_token(cancel_token: CancellationToken) ->
/// # Returns
/// * `Option<&'static CancellationToken>` - The global cancellation token, if set
///
pub fn get_background_services_cancel_token() -> Option<&'static CancellationToken> {
GLOBAL_BACKGROUND_SERVICES_CANCEL_TOKEN.get()
pub fn get_background_services_cancel_token() -> Option<CancellationToken> {
current_ctx().background_cancel_token()
}
/// Create and initialize the global background services cancellation token
@@ -326,7 +323,7 @@ pub fn create_background_services_cancel_token() -> CancellationToken {
/// # Returns
/// * None
pub fn shutdown_background_services() {
if let Some(cancel_token) = GLOBAL_BACKGROUND_SERVICES_CANCEL_TOKEN.get() {
if let Some(cancel_token) = get_background_services_cancel_token() {
cancel_token.cancel();
}
}