mirror of
https://github.com/rustfs/rustfs.git
synced 2026-09-04 11:15:39 +00:00
refactor(concurrency): remove zero-caller facade modules, fix feature build (#4530)
refactor(concurrency): remove zero-caller facade modules and fix no-default-features build (backlog#1025) The audit in rustfs/backlog#1010 (consistent with #805) established that most of crates/concurrency was a decorative facade with zero production callers; the real runtime concurrency control lives in rustfs/src/storage/*. This deletes the dead facades and keeps only what the workspace actually consumes. Deleted (zero callers verified by workspace-wide grep): - manager.rs: ConcurrencyManager, lifecycle start/stop, misleading 'started' lifecycle logs - config.rs: ConcurrencyConfig, ConcurrencyFeatures, from_env - timeout.rs: TimeoutManager, TimeoutGuard, TimeoutManagerPolicy - lock.rs: LockManager, LockScopeGuard, OptimizedLockGuard - scheduler.rs: SchedulerManager, SchedulerPolicy, IoStrategy - deadlock.rs facade: DeadlockManager, RequestTracker - backpressure.rs facade: BackpressureManager, BackpressurePipe - the prelude module, unused io-core re-exports, and all feature flags Kept (real callers in ecstore/heal/rustfs): - workload.rs admission contract types (unchanged) - workers.rs Workers pool (unchanged, retained per #4498) - GetObjectQueueSnapshot (moved from manager.rs to new queue.rs) - PipeBackpressurePolicy (used by rustfs/src/storage/backpressure.rs) - DeadlockMonitorPolicy (used by rustfs/src/storage/deadlock_detector.rs) - OperationProgress re-export (used by rustfs/src/storage/timeout_wrapper.rs) Removing the feature flags fixes the previously broken cargo check -p rustfs-concurrency --no-default-features (E0432). Docs and the logging guardrail file list are updated to match. Ref: rustfs/backlog#1025
This commit is contained in:
Zhengchao An
+21
-148
@@ -12,167 +12,40 @@
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
//! # RustFS Concurrency Management
|
||||
//! # RustFS Concurrency Contracts
|
||||
//!
|
||||
//! This crate provides comprehensive concurrency management for RustFS,
|
||||
//! including timeout control, lock optimization, deadlock detection,
|
||||
//! backpressure management, and I/O scheduling.
|
||||
//! Shared concurrency contract types for RustFS:
|
||||
//!
|
||||
//! ## Features
|
||||
//! - [`workload`]: workload admission snapshot/contract types consumed by
|
||||
//! ecstore, heal, and the server for admission-aware throttling.
|
||||
//! - [`GetObjectQueueSnapshot`]: disk permit queue usage snapshot for
|
||||
//! GetObject orchestration.
|
||||
//! - [`workers`]: a bounded semaphore-backed worker-slot pool.
|
||||
//! - [`PipeBackpressurePolicy`] / [`DeadlockMonitorPolicy`]: policy types
|
||||
//! shared with the runtime implementations.
|
||||
//!
|
||||
//! All features are controlled by feature flags and can be enabled/disabled at compile time:
|
||||
//!
|
||||
//! - **timeout**: Dynamic timeout calculation based on data size and transfer rate
|
||||
//! - **lock**: Early lock release to reduce contention
|
||||
//! - **deadlock**: Request tracking and cycle detection
|
||||
//! - **backpressure**: Buffer-based flow control
|
||||
//! - **scheduler**: Adaptive buffer sizing and priority queuing
|
||||
//!
|
||||
//! ## Architecture
|
||||
//!
|
||||
//! ```text
|
||||
//! rustfs-concurrency (Business Layer)
|
||||
//! ├── timeout (Timeout Control)
|
||||
//! ├── lock (Lock Optimization)
|
||||
//! ├── deadlock (Deadlock Detection)
|
||||
//! ├── backpressure (Backpressure Management)
|
||||
//! └── scheduler (I/O Scheduling)
|
||||
//! │
|
||||
//! ├── rustfs-io-core (Core Algorithms)
|
||||
//! └── rustfs-io-metrics (Metrics Collection)
|
||||
//! ```
|
||||
//!
|
||||
//! ## Usage
|
||||
//!
|
||||
//! ```rust,no_run
|
||||
//! use rustfs_concurrency::{ConcurrencyConfig, ConcurrencyManager};
|
||||
//!
|
||||
//! # #[tokio::main]
|
||||
//! # async fn main() {
|
||||
//! // Create manager with all features enabled
|
||||
//! let config = ConcurrencyConfig::default();
|
||||
//! let manager = ConcurrencyManager::new(config);
|
||||
//!
|
||||
//! // Start services
|
||||
//! manager.start().await;
|
||||
//!
|
||||
//! // Use timeout control (if enabled)
|
||||
//! if manager.is_timeout_enabled() {
|
||||
//! let timeout_manager = manager.timeout();
|
||||
//! let _ = timeout_manager;
|
||||
//! }
|
||||
//!
|
||||
//! // Use lock optimization (if enabled)
|
||||
//! if manager.is_lock_enabled() {
|
||||
//! let lock_manager = manager.lock();
|
||||
//! let _ = lock_manager;
|
||||
//! }
|
||||
//!
|
||||
//! // Stop services
|
||||
//! manager.stop().await;
|
||||
//! # }
|
||||
//! ```
|
||||
//! The actual runtime concurrency control — size-aware timeouts, byte-watermark
|
||||
//! backpressure, request-hang/deadlock detection, and I/O scheduling — is
|
||||
//! implemented in `rustfs/src/storage/*` on top of the `rustfs-io-core`
|
||||
//! primitives. This crate only carries the data and contract types those
|
||||
//! implementations share; it does not run any background tasks itself.
|
||||
|
||||
#![deny(missing_docs)]
|
||||
#![deny(unsafe_code)]
|
||||
#![cfg_attr(docsrs, feature(doc_cfg))]
|
||||
|
||||
// Re-export core types from io-core
|
||||
pub use rustfs_io_core::{
|
||||
// Backpressure types
|
||||
BackpressureConfig as CoreBackpressureConfig,
|
||||
BackpressureMonitor as CoreBackpressureMonitor,
|
||||
BackpressureState,
|
||||
// Re-exported io-core type used by downstream timeout implementations.
|
||||
pub use rustfs_io_core::OperationProgress;
|
||||
|
||||
// Deadlock types
|
||||
DeadlockDetector as CoreDeadlockDetector,
|
||||
IoLoadLevel,
|
||||
IoLoadMetrics,
|
||||
IoPriority,
|
||||
// Scheduler types
|
||||
IoScheduler,
|
||||
IoSchedulingContext,
|
||||
LockInfo,
|
||||
LockOptimizer as CoreLockOptimizer,
|
||||
|
||||
// Lock types
|
||||
LockStats as CoreLockStats,
|
||||
LockType,
|
||||
// Timeout types
|
||||
OperationProgress,
|
||||
TimeoutError,
|
||||
TimeoutStats,
|
||||
WaitGraphEdge,
|
||||
|
||||
calculate_adaptive_timeout,
|
||||
estimate_bytes_per_second,
|
||||
};
|
||||
|
||||
// Module declarations with feature gates
|
||||
#[cfg(feature = "timeout")]
|
||||
mod timeout;
|
||||
|
||||
#[cfg(feature = "lock")]
|
||||
mod lock;
|
||||
|
||||
#[cfg(feature = "deadlock")]
|
||||
mod deadlock;
|
||||
|
||||
#[cfg(feature = "backpressure")]
|
||||
mod backpressure;
|
||||
|
||||
#[cfg(feature = "scheduler")]
|
||||
mod scheduler;
|
||||
|
||||
mod deadlock;
|
||||
mod queue;
|
||||
pub mod workers;
|
||||
pub mod workload;
|
||||
|
||||
// Public module exports with feature gates
|
||||
#[cfg(feature = "timeout")]
|
||||
pub use timeout::{TimeoutGuard, TimeoutManager, TimeoutManagerPolicy};
|
||||
|
||||
#[cfg(feature = "lock")]
|
||||
pub use lock::{LockConfig, LockManager, LockScopeGuard, OptimizedLockGuard};
|
||||
|
||||
#[cfg(feature = "deadlock")]
|
||||
pub use deadlock::{DeadlockManager, DeadlockMonitorPolicy, RequestTracker};
|
||||
|
||||
#[cfg(feature = "backpressure")]
|
||||
pub use backpressure::{BackpressureManager, BackpressurePipe, PipeBackpressurePolicy};
|
||||
|
||||
#[cfg(feature = "scheduler")]
|
||||
pub use scheduler::{IoStrategy, SchedulerManager, SchedulerPolicy};
|
||||
|
||||
// Configuration
|
||||
mod config;
|
||||
pub use config::{ConcurrencyConfig, ConcurrencyFeatures};
|
||||
|
||||
// Manager
|
||||
mod manager;
|
||||
pub use manager::{ConcurrencyManager, GetObjectQueueSnapshot};
|
||||
pub use backpressure::PipeBackpressurePolicy;
|
||||
pub use deadlock::DeadlockMonitorPolicy;
|
||||
pub use queue::GetObjectQueueSnapshot;
|
||||
pub use workload::{
|
||||
AdmissionState, WorkloadAdmissionRegistrySnapshot, WorkloadAdmissionSnapshot, WorkloadAdmissionSnapshotProvider,
|
||||
WorkloadClass,
|
||||
};
|
||||
|
||||
// Prelude for convenient imports
|
||||
pub mod prelude {
|
||||
//! Prelude module for convenient imports
|
||||
|
||||
#[cfg(feature = "timeout")]
|
||||
pub use crate::timeout::{TimeoutGuard, TimeoutManager, TimeoutManagerPolicy};
|
||||
|
||||
#[cfg(feature = "lock")]
|
||||
pub use crate::lock::{LockConfig, LockManager, LockScopeGuard, OptimizedLockGuard};
|
||||
|
||||
#[cfg(feature = "deadlock")]
|
||||
pub use crate::deadlock::{DeadlockManager, DeadlockMonitorPolicy, RequestTracker};
|
||||
|
||||
#[cfg(feature = "backpressure")]
|
||||
pub use crate::backpressure::{BackpressureManager, BackpressurePipe, PipeBackpressurePolicy};
|
||||
|
||||
#[cfg(feature = "scheduler")]
|
||||
pub use crate::scheduler::{IoStrategy, SchedulerManager, SchedulerPolicy};
|
||||
|
||||
pub use crate::{AdmissionState, ConcurrencyConfig, ConcurrencyFeatures, ConcurrencyManager, WorkloadAdmissionSnapshot};
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user