* feat: integrate CancellationToken for unified background services management

- Consolidate data scanner and auto heal cancellation tokens into single unified token
- Move GLOBAL_BACKGROUND_SERVICES_CANCEL_TOKEN to global.rs for centralized management
- Add graceful shutdown support to MRF heal routine with MinIO-compatible logic
- Implement heal_routine_with_cancel method preserving original healing logic
- Update main.rs to use unified background services shutdown mechanism
- Enhance error handling with proper ecstore Result types
- Fix clippy warnings for needless return statements
- Maintain backward compatibility while adding modern cancellation support

This change provides a cleaner architecture for background service lifecycle management
and ensures all healing services can be gracefully shut down through a single token.

Signed-off-by: junxiang Mu <1948535941@qq.com>

* fix: Refact heal and scanner design

Signed-off-by: junxiang Mu <1948535941@qq.com>

* refact: step 2

Signed-off-by: junxiang Mu <1948535941@qq.com>

* feat: refactor scanner module and add data usage statistics

- Move scanner code to scanner/ subdirectory for better organization
- Add data usage statistics collection and persistence
- Implement histogram support for size and version distribution
- Add global cancel token management for scanner operations
- Integrate scanner with ECStore for comprehensive data analysis
- Update error handling and improve test isolation
- Add data usage API endpoints and backend integration

Signed-off-by: junxiang Mu <1948535941@qq.com>

* Chore: fix ref and fix comment

Signed-off-by: junxiang Mu <1948535941@qq.com>

* fix: fix clippy

Signed-off-by: junxiang Mu <1948535941@qq.com>

---------

Signed-off-by: junxiang Mu <1948535941@qq.com>
Co-authored-by: dandan <dandan@dandandeMac-Studio.local>
This commit is contained in:
guojidan
2025-07-10 17:10:44 +08:00
committed by GitHub
parent a3b5445824
commit 2832f0e089
17 changed files with 2885 additions and 116 deletions
+28
View File
@@ -30,6 +30,7 @@ use std::{
time::SystemTime,
};
use tokio::sync::{OnceCell, RwLock};
use tokio_util::sync::CancellationToken;
use uuid::Uuid;
pub const DISK_ASSUME_UNKNOWN_SIZE: u64 = 1 << 30;
@@ -66,6 +67,9 @@ pub static ref GLOBAL_NodeNamesHex: HashMap<String, ()> = HashMap::new();
pub static ref GLOBAL_REGION: OnceLock<String> = OnceLock::new();
}
// Global cancellation token for background services (data scanner and auto heal)
static GLOBAL_BACKGROUND_SERVICES_CANCEL_TOKEN: OnceLock<CancellationToken> = OnceLock::new();
static GLOBAL_ACTIVE_CRED: OnceLock<Credentials> = OnceLock::new();
pub fn init_global_action_cred(ak: Option<String>, sk: Option<String>) {
@@ -192,3 +196,27 @@ pub fn set_global_region(region: String) {
pub fn get_global_region() -> Option<String> {
GLOBAL_REGION.get().cloned()
}
/// Initialize the global background services cancellation token
pub fn init_background_services_cancel_token(cancel_token: CancellationToken) -> Result<(), CancellationToken> {
GLOBAL_BACKGROUND_SERVICES_CANCEL_TOKEN.set(cancel_token)
}
/// Get the global background services cancellation token
pub fn get_background_services_cancel_token() -> Option<&'static CancellationToken> {
GLOBAL_BACKGROUND_SERVICES_CANCEL_TOKEN.get()
}
/// Create and initialize the global background services cancellation token
pub fn create_background_services_cancel_token() -> CancellationToken {
let cancel_token = CancellationToken::new();
init_background_services_cancel_token(cancel_token.clone()).expect("Background services cancel token already initialized");
cancel_token
}
/// Shutdown all background services gracefully
pub fn shutdown_background_services() {
if let Some(cancel_token) = GLOBAL_BACKGROUND_SERVICES_CANCEL_TOKEN.get() {
cancel_token.cancel();
}
}