* feat: integrate CancellationToken for unified background services management

- Consolidate data scanner and auto heal cancellation tokens into single unified token
- Move GLOBAL_BACKGROUND_SERVICES_CANCEL_TOKEN to global.rs for centralized management
- Add graceful shutdown support to MRF heal routine with MinIO-compatible logic
- Implement heal_routine_with_cancel method preserving original healing logic
- Update main.rs to use unified background services shutdown mechanism
- Enhance error handling with proper ecstore Result types
- Fix clippy warnings for needless return statements
- Maintain backward compatibility while adding modern cancellation support

This change provides a cleaner architecture for background service lifecycle management
and ensures all healing services can be gracefully shut down through a single token.

Signed-off-by: junxiang Mu <1948535941@qq.com>

* fix: Refact heal and scanner design

Signed-off-by: junxiang Mu <1948535941@qq.com>

* refact: step 2

Signed-off-by: junxiang Mu <1948535941@qq.com>

* feat: refactor scanner module and add data usage statistics

- Move scanner code to scanner/ subdirectory for better organization
- Add data usage statistics collection and persistence
- Implement histogram support for size and version distribution
- Add global cancel token management for scanner operations
- Integrate scanner with ECStore for comprehensive data analysis
- Update error handling and improve test isolation
- Add data usage API endpoints and backend integration

Signed-off-by: junxiang Mu <1948535941@qq.com>

* Chore: fix ref and fix comment

Signed-off-by: junxiang Mu <1948535941@qq.com>

* fix: fix clippy

Signed-off-by: junxiang Mu <1948535941@qq.com>

---------

Signed-off-by: junxiang Mu <1948535941@qq.com>
Co-authored-by: dandan <dandan@dandandeMac-Studio.local>
This commit is contained in:
guojidan
2025-07-10 17:10:44 +08:00
committed by GitHub
parent a3b5445824
commit 2832f0e089
17 changed files with 2885 additions and 116 deletions
+25
View File
@@ -0,0 +1,25 @@
// Copyright 2024 RustFS Team
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
pub mod data_scanner;
pub mod data_usage;
pub mod histogram;
pub mod metrics;
// Re-export main types for convenience
pub use data_scanner::Scanner;
pub use data_usage::{
load_data_usage_from_backend, store_data_usage_in_backend, BucketTargetUsageInfo, BucketUsageInfo, DataUsageInfo,
};
pub use metrics::ScannerMetrics;