* feat: integrate CancellationToken for unified background services management

- Consolidate data scanner and auto heal cancellation tokens into single unified token
- Move GLOBAL_BACKGROUND_SERVICES_CANCEL_TOKEN to global.rs for centralized management
- Add graceful shutdown support to MRF heal routine with MinIO-compatible logic
- Implement heal_routine_with_cancel method preserving original healing logic
- Update main.rs to use unified background services shutdown mechanism
- Enhance error handling with proper ecstore Result types
- Fix clippy warnings for needless return statements
- Maintain backward compatibility while adding modern cancellation support

This change provides a cleaner architecture for background service lifecycle management
and ensures all healing services can be gracefully shut down through a single token.

Signed-off-by: junxiang Mu <1948535941@qq.com>

* fix: Refact heal and scanner design

Signed-off-by: junxiang Mu <1948535941@qq.com>

* refact: step 2

Signed-off-by: junxiang Mu <1948535941@qq.com>

* feat: refactor scanner module and add data usage statistics

- Move scanner code to scanner/ subdirectory for better organization
- Add data usage statistics collection and persistence
- Implement histogram support for size and version distribution
- Add global cancel token management for scanner operations
- Integrate scanner with ECStore for comprehensive data analysis
- Update error handling and improve test isolation
- Add data usage API endpoints and backend integration

Signed-off-by: junxiang Mu <1948535941@qq.com>

* Chore: fix ref and fix comment

Signed-off-by: junxiang Mu <1948535941@qq.com>

* fix: fix clippy

Signed-off-by: junxiang Mu <1948535941@qq.com>

---------

Signed-off-by: junxiang Mu <1948535941@qq.com>
Co-authored-by: dandan <dandan@dandandeMac-Studio.local>
This commit is contained in:
guojidan
2025-07-10 17:10:44 +08:00
committed by GitHub
parent a3b5445824
commit 2832f0e089
17 changed files with 2885 additions and 116 deletions
+3
View File
@@ -36,6 +36,7 @@ members = [
"crates/utils", # Utility functions and helpers
"crates/workers", # Worker thread pools and task scheduling
"crates/zip", # ZIP file handling and compression
"crates/ahm",
]
resolver = "2"
@@ -62,6 +63,7 @@ rustfs-filemeta = { path = "crates/filemeta" }
rustfs-rio = { path = "crates/rio" }
[workspace.dependencies]
rustfs-ahm = { path = "crates/ahm", version = "0.0.3" }
rustfs-s3select-api = { path = "crates/s3select-api", version = "0.0.5" }
rustfs-appauth = { path = "crates/appauth", version = "0.0.5" }
rustfs-common = { path = "crates/common", version = "0.0.5" }
@@ -261,6 +263,7 @@ winapi = { version = "0.3.9" }
xxhash-rust = { version = "0.8.15", features = ["xxh64", "xxh3"] }
zip = "2.4.2"
zstd = "0.13.3"
anyhow = "1.0.86"
[profile.wasm-dev]
inherits = "dev"