* fix(ecstore): add async-recursion to resolve nightly trait solver regression
The newest nightly compiler switched to the new trait solver, which
currently rejects async recursive functions that were previously accepted.
This causes the following compilation failures:
- `LocalDisk::delete_file()`
- `LocalDisk::scan_dir()`
Add `async-recursion` as a workspace dependency and annotate both functions with `#[async_recursion]` so that the crate compiles cleanly with the latest nightly and will continue to build once the new solver lands in stable.
Signed-off-by: reigadegr <2722688642@qq.com>
* fix: resolve duplicate bound error in scan_dir function
Replaced inline trait bounds with where clause to avoid duplication caused by macro expansion.
Signed-off-by: reigadegr <2722688642@qq.com>
---------
Signed-off-by: reigadegr <2722688642@qq.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
* chore: upgrade actions/checkout from v4 to v5
- Update GitHub Actions checkout action version
- Ensure compatibility with latest workflow features
- Maintain existing checkout behavior and configuration
* upgrade version
* feat(lifecycle): Implement object lifecycle management functionality
Add a lifecycle module to automatically handle object expiration and transition during scanning
Modify the file metadata cache module to be publicly visible to support lifecycle operations
Adjust the scanning interval to a shorter time for testing lifecycle rules
Implement the parsing and execution logic for S3 lifecycle configurations
Add integration tests to verify the lifecycle expiration functionality
Update dependencies to support the new lifecycle features
Signed-off-by: junxiang Mu <1948535941@qq.com>
* fix cargo dependencies
Signed-off-by: junxiang Mu <1948535941@qq.com>
* fix fmt
Signed-off-by: junxiang Mu <1948535941@qq.com>
---------
Signed-off-by: junxiang Mu <1948535941@qq.com>
Co-authored-by: houseme <housemecn@gmail.com>
* fix: add tokio-test
* fix: "called `unwrap` on `v` after checking its variant with `is_some`"
= help: try using `if let` or `match`
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#unnecessary_unwrap
= note: `-D clippy::unnecessary-unwrap` implied by `-D warnings`
= help: to override `-D warnings` add `#[allow(clippy::unnecessary_unwrap)]`
* fmt
* set toolchain 1.88.0
* fmt
* fix: cliip
- Add global metrics system to common crate for cross-module usage
- Integrate global metrics collection into AHM scanner operations
- Update ECStore to use common metrics system instead of local implementation
- Add chrono dependency to AHM crate for timestamp handling
- Re-export IlmAction from common metrics in ECStore lifecycle module
- Update scanner methods to use global metrics for cycle, disk, and volume scans
- Maintain backward compatibility with local metrics collector
- Fix clippy warnings and ensure proper code formatting
This change enables unified metrics collection across the entire RustFS system,
allowing better monitoring and observability of scanner operations.
Signed-off-by: junxiang Mu <1948535941@qq.com>
- Add global unbounded channel in common crate for heal requests
- Implement channel processor in ahm to handle heal commands
- Add Start/Query/Cancel commands support via channel
- Integrate heal manager initialization in main.rs
- Replace direct MRF calls with channel-based heal requests in ecstore
- Support advanced heal options including pool_index and set_index
- Enable admin handlers to send heal requests via channel
- Refactor NamespaceLock to use a unified client vector and quorum mechanism,
removing legacy local/distributed lock split and related code.
- Update LockRequest to split timeout into acquire_timeout and ttl, and add
builder methods for both.
- Adjust all batch lock APIs to accept ttl and use new LockRequest fields.
- Update all affected tests and documentation for the new API.
Signed-off-by: junxiang Mu <1948535941@qq.com>
- Remove unused core/rwlock.rs and manager/ modules (ManagerFactory, LifecycleManager, NamespaceManager)
- Move all lock-related code into crates/lock/src with clear submodules: client, core, utils, etc.
- Ensure only necessary files and APIs are exposed, improve maintainability
- No functional logic change, pure structure and cleanup refactor
Signed-off-by: dandan <dandan@dandandeMac-Studio.local>
This commit introduces two key enhancements: the integration of Grafana Tempo for distributed tracing and the implementation of a dual-stack TCP listener for improved network compatibility.
- **Observability**:
- Adds the `tempo` service to the `docker-compose.yml` observability stack.
- Tempo is configured to collect and store traces, integrating with the existing OpenTelemetry setup.
- A custom `tempo-entrypoint.sh` script is included to manage volume permissions on startup.
- **Networking**:
- Modifies `http.rs` to support dual-stack (IPv4/IPv6) connections on a single socket.
- By setting the `IPV6_V6ONLY` socket option to `false`, the server can now accept both IPv6 and IPv4-mapped IPv6 traffic, enhancing cross-platform support.
This commit streamlines the server configuration by unifying the S3 API and the WebUI (Console) to serve on a single port.
Previously, the console was managed by separate configuration options (`RUSTFS_CONSOLE_ENABLE` and `RUSTFS_CONSOLE_ADDRESS`), requiring a distinct port. This added complexity to deployment and configuration.
With this change:
- The `RUSTFS_CONSOLE_ADDRESS` and `RUSTFS_CONSOLE_FS_ENDPOINT` environment variables are removed.
- The WebUI is now always available and served directly from the main application port defined by `RUSTFS_ADDRESS`.
- This simplifies setup, reduces the number of exposed ports, and makes the application easier to manage and deploy, especially in containerized environments.
Users should update their startup scripts and remove the deprecated `RUSTFS_CONSOLE_*` variables.
* feat: integrate CancellationToken for unified background services management
- Consolidate data scanner and auto heal cancellation tokens into single unified token
- Move GLOBAL_BACKGROUND_SERVICES_CANCEL_TOKEN to global.rs for centralized management
- Add graceful shutdown support to MRF heal routine with MinIO-compatible logic
- Implement heal_routine_with_cancel method preserving original healing logic
- Update main.rs to use unified background services shutdown mechanism
- Enhance error handling with proper ecstore Result types
- Fix clippy warnings for needless return statements
- Maintain backward compatibility while adding modern cancellation support
This change provides a cleaner architecture for background service lifecycle management
and ensures all healing services can be gracefully shut down through a single token.
Signed-off-by: junxiang Mu <1948535941@qq.com>
* fix: Refact heal and scanner design
Signed-off-by: junxiang Mu <1948535941@qq.com>
* refact: step 2
Signed-off-by: junxiang Mu <1948535941@qq.com>
* feat: refactor scanner module and add data usage statistics
- Move scanner code to scanner/ subdirectory for better organization
- Add data usage statistics collection and persistence
- Implement histogram support for size and version distribution
- Add global cancel token management for scanner operations
- Integrate scanner with ECStore for comprehensive data analysis
- Update error handling and improve test isolation
- Add data usage API endpoints and backend integration
Signed-off-by: junxiang Mu <1948535941@qq.com>
* Chore: fix ref and fix comment
Signed-off-by: junxiang Mu <1948535941@qq.com>
* fix: fix clippy
Signed-off-by: junxiang Mu <1948535941@qq.com>
---------
Signed-off-by: junxiang Mu <1948535941@qq.com>
Co-authored-by: dandan <dandan@dandandeMac-Studio.local>
Move the construction of the hybrid service stack, including all middleware and the RPC service, from the main `run` function into the `process_connection` function.
This change ensures that each incoming connection gets its own isolated service instance. This improves modularity by making the connection handling logic more self-contained and simplifies the main server loop.
Key changes:
- The `hybrid_service` and `rpc_service` are now created inside `process_connection`.
- The `run` function's responsibility is reduced to accepting TCP connections and spawning tasks for `process_connection`.
This commit introduces a significant reorganization of the project structure to improve maintainability and clarity.
Key changes include:
- Adjusted the directory layout for a more logical module organization.
- Removed unused crate dependencies, reducing the overall project size and potentially speeding up build times.
- Updated import paths and configuration files to reflect the structural changes.
reigadegr