mirror of https://github.com/rustfs/rustfs.git synced 2026-07-26 08:18:18 +00:00

Files

T

Zhengchao An 05833063c7 refactor(concurrency): remove zero-caller facade modules, fix feature build (#4530 )

refactor(concurrency): remove zero-caller facade modules and fix no-default-features build (backlog#1025)

The audit in rustfs/backlog#1010 (consistent with #805) established that most of crates/concurrency was a decorative facade with zero production callers; the real runtime concurrency control lives in rustfs/src/storage/*. This deletes the dead facades and keeps only what the workspace actually consumes.

Deleted (zero callers verified by workspace-wide grep):
- manager.rs: ConcurrencyManager, lifecycle start/stop, misleading 'started' lifecycle logs
- config.rs: ConcurrencyConfig, ConcurrencyFeatures, from_env
- timeout.rs: TimeoutManager, TimeoutGuard, TimeoutManagerPolicy
- lock.rs: LockManager, LockScopeGuard, OptimizedLockGuard
- scheduler.rs: SchedulerManager, SchedulerPolicy, IoStrategy
- deadlock.rs facade: DeadlockManager, RequestTracker
- backpressure.rs facade: BackpressureManager, BackpressurePipe
- the prelude module, unused io-core re-exports, and all feature flags

Kept (real callers in ecstore/heal/rustfs):
- workload.rs admission contract types (unchanged)
- workers.rs Workers pool (unchanged, retained per #4498)
- GetObjectQueueSnapshot (moved from manager.rs to new queue.rs)
- PipeBackpressurePolicy (used by rustfs/src/storage/backpressure.rs)
- DeadlockMonitorPolicy (used by rustfs/src/storage/deadlock_detector.rs)
- OperationProgress re-export (used by rustfs/src/storage/timeout_wrapper.rs)

Removing the feature flags fixes the previously broken cargo check -p rustfs-concurrency --no-default-features (E0432). Docs and the logging guardrail file list are updated to match.

Ref: rustfs/backlog#1025

2026-07-09 01:12:43 +08:00

5.4 KiB

Raw Permalink Blame History

Workload Admission Contracts

This document records the rustfs/backlog#660 PR-05 and PR-07 scheduler preservation and runtime workload-class contract slice.

Preservation Coverage

The rustfs-concurrency tests pin the current reusable admission-facing behavior before later snapshot extraction:

Worker slot over-release remains clamped by the configured worker limit.
GetObjectQueueSnapshot preserves saturated, over-available, and zero-total permit semantics.

The former reusable scheduler and backpressure-pipe facades (and their preservation tests) were removed as zero-caller dead code in backlog#1025; scheduler buffer/priority behavior is now pinned by rustfs-io-core and rustfs/src/storage/concurrency tests.

Workload Class Contract

WorkloadClass defines the required future admission categories:

Foreground read.
Foreground write.
Metadata.
Scanner.
Repair.
Replication.

AdmissionState, WorkloadAdmissionSnapshot, and WorkloadAdmissionRegistrySnapshot define read-only status shapes for later runtime owners. They do not replace the current scheduler, request guard, scanner, heal, replication, or ECStore placement behavior.

Boundary Rules

rustfs-concurrency owns this reusable contract surface.
The contract does not depend on rustfs-ecstore or RustFS binary runtime state.
No scheduler decision logic, queue capacity, Tokio runtime default, scanner admission, heal admission, replication admission, placement, membership, or NUMA behavior changes are part of this slice.

Set-Local Snapshot Extraction

The RustFS storage ConcurrencyManager now implements WorkloadAdmissionSnapshotProvider for local foreground-read admission:

ForegroundRead reports local disk-read permit usage through GetObjectQueueSnapshot.
active is the number of disk-read permits currently in use.
limit is the configured maximum concurrent disk reads.
queued remains None because the current semaphore does not expose waiter counts.
Scanner, repair, replication, foreground write, and metadata entries remain Unknown until their owning runtime components expose read-only status.

This is an observation surface only. Permit acquisition, priority assignment, buffer sizing, storage media detection, request guards, and queue behavior are unchanged.

Heal Repair Snapshot Extraction

The RustFS integration layer now exposes a read-only repair admission snapshot from the heal runtime counters:

Repair reports the current heal active task count.
queued reports the current heal queue length.
limit remains None because the configured heal queue and concurrency limits live behind the async heal manager state.
Other workload classes remain Unknown in this provider until their owning runtime components expose read-only status.

This is an observation surface only. Heal request admission, queue capacity, priority merge/drop policy, task scheduling, retry handling, and repair behavior are unchanged.

Replication Snapshot Extraction

The RustFS integration layer now exposes a read-only replication admission snapshot from the existing replication pool and queue statistics:

Replication reports active regular, large-object, and MRF worker counts.
queued reports the current site replication queue count when queue stats are immediately observable.
limit remains None because replication worker limits remain owned by the async replication pool and resize policy.
If the replication runtime has not initialized, or queue stats are currently locked, the snapshot reports Unknown instead of blocking or guessing.

This is an observation surface only. Replication admission, queue channel capacity, worker resize behavior, MRF handling, target dispatch, and resync behavior are unchanged.

RustFS Runtime Owner Snapshot Extraction

The RustFS integration layer now extends the workload admission registry with additional read-only owner mappings:

ForegroundRead reuses the storage ConcurrencyManager disk-read permit snapshot so the RustFS-level provider exposes the same active and limit counts as the storage-local provider.
Scanner reports the existing scanner active work-unit counter. When the counter is zero, the snapshot remains Unknown because the current counter cannot distinguish an idle scanner from a scanner that has not initialized.
Metadata reports Open once the bucket metadata runtime handle is available, and Unknown before initialization.
ForegroundWrite remains Unknown until a write-specific admission owner exposes a read-only surface.

This is an observation surface only. Disk-read permit acquisition, scanner cycle scheduling, bucket metadata loading, metadata locks, object write paths, and queue behavior are unchanged.

Provider Composition Boundary

WorkloadAdmissionRegistrySnapshot::overlay composes provider-owned registry snapshots without mutating runtime owners:

The storage concurrency provider remains the source of truth for ForegroundRead.
The RustFS runtime owner provider overlays metadata, scanner, repair, replication, and foreground-write status on top of the storage registry.
Matching workload classes are replaced by the later provider snapshot; new classes are appended without reordering existing unrelated entries.

This keeps the later controller/status layer consuming a single read-only registry while preserving the existing storage, scanner, heal, replication, and metadata ownership boundaries.

5.4 KiB Raw Permalink Blame History