Remove the pprof performance profiling functionality on the Windows platform, as this platform does not support the relevant features
Move the pprof dependency to the platform-specific configuration for non-Windows systems
Update the performance profiling endpoint handling logic to distinguish between platform support statuses
Add the CLAUDE.md document to explain project build and architecture information
Signed-off-by: RustFS Developer <dandan@rustfs.com>
Co-authored-by: RustFS Developer <dandan@rustfs.com>
* init audit logger module
* add audit webhook default config kvs
* feat: Add comprehensive tests for authentication module (#309)
* feat: add comprehensive tests for authentication module
- Add 33 unit tests covering all public functions in auth.rs
- Test IAMAuth struct creation and secret key validation
- Test check_claims_from_token with various credential types and scenarios
- Test session token extraction from headers and query parameters
- Test condition values generation for different user types
- Test query parameter parsing with edge cases
- Test Credentials helper methods (is_expired, is_temp, is_service_account)
- Ensure tests handle global state dependencies gracefully
- All tests pass successfully with 100% coverage of testable functions
* style: fix code formatting issues
* Add verification script for checking PR branch statuses and tests
Co-authored-by: anzhengchao <anzhengchao@gmail.com>
* fix: resolve clippy uninlined format args warning
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
* feat: add basic tests for core storage module (#313)
* feat: add basic tests for core storage module
- Add 6 unit tests for FS struct and basic functionality
- Test FS creation, Debug and Clone trait implementations
- Test RUSTFS_OWNER constant definition and values
- Test S3 error code creation and handling
- Test compression format detection for common file types
- Include comprehensive documentation about integration test needs
Note: Full S3 API testing requires complex setup with storage backend,
global configuration, and network infrastructure - better suited for
integration tests rather than unit tests.
* style: fix code formatting issues
* fix: resolve clippy warnings in storage tests
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
* feat: add tests for admin handlers module (#314)
* feat: add tests for admin handlers module
- Add 5 new unit tests for admin handler functionality
- Test AccountInfo struct creation, serialization and default values
- Test creation of all admin handler structs (13 handlers)
- Test HealOpts JSON serialization and deserialization
- Test HealOpts URL encoding/decoding with proper field types
- Maintain existing test while adding comprehensive coverage
- Include documentation about integration test requirements
All tests pass successfully with proper error handling for complex dependencies.
* style: fix code formatting issues
* fix: resolve clippy warnings in admin handlers tests
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
* build(deps): bump the dependencies group with 3 updates (#326)
* perf: avoid transmitting parity shards when the object is good (#322)
* upgrade version
* Fix: fix data integrity check
Signed-off-by: junxiang Mu <1948535941@qq.com>
* Fix: Separate Clippy's fix and check commands into two commands.
Signed-off-by: junxiang Mu <1948535941@qq.com>
* fix: miss inline metadata (#345)
* Update dependabot.yml
* fix: Fixed an issue where the list_objects_v2 API did not return dire… (#352)
* fix: Fixed an issue where the list_objects_v2 API did not return directory names when they conflicted with file names in the same bucket (e.g., test/ vs. test.txt, aaa/ vs. aaa.csv) (#335)
* fix: adjusted the order of directory listings
* init
* fix
* fix
* feat: add docker usage for rustfs mcp (#365)
* feat: enhance metadata extraction with object name for MIME type detection
Signed-off-by: junxiang Mu <1948535941@qq.com>
* Feature: lock support auto release
Signed-off-by: junxiang Mu <1948535941@qq.com>
* improve lock
Signed-off-by: junxiang Mu <1948535941@qq.com>
* Fix: fix scanner detect
Signed-off-by: junxiang Mu <1948535941@qq.com>
* Fix: clippy && fmt
Signed-off-by: junxiang Mu <1948535941@qq.com>
* refactor(ecstore): Optimize memory usage for object integrity verification
Change the object integrity verification from reading all data to streaming processing to avoid memory overflow caused by large objects.
Modify the TLS key log check to use environment variables directly instead of configuration constants.
Add memory limits for object data reading in the AHM module.
Signed-off-by: junxiang Mu <1948535941@qq.com>
* Chore: reduce PR template checklist
Signed-off-by: junxiang Mu <1948535941@qq.com>
* Chore: remove comment code (#376)
Signed-off-by: junxiang Mu <1948535941@qq.com>
* chore: upgrade actions/checkout from v4 to v5 (#381)
* chore: upgrade actions/checkout from v4 to v5
- Update GitHub Actions checkout action version
- Ensure compatibility with latest workflow features
- Maintain existing checkout behavior and configuration
* upgrade version
* fix
* add and improve code for notify
* feat: extend rustfs mcp with bucket creation and deletion (#416)
* feat: extend rustfs mcp with bucket creation and deletion
* update file to fix pipeline error
* change variable name to fix pipeline error
* fix(ecstore): add async-recursion to resolve nightly trait solver reg… (#415)
* fix(ecstore): add async-recursion to resolve nightly trait solver regression
The newest nightly compiler switched to the new trait solver, which
currently rejects async recursive functions that were previously accepted.
This causes the following compilation failures:
- `LocalDisk::delete_file()`
- `LocalDisk::scan_dir()`
Add `async-recursion` as a workspace dependency and annotate both functions with `#[async_recursion]` so that the crate compiles cleanly with the latest nightly and will continue to build once the new solver lands in stable.
Signed-off-by: reigadegr <2722688642@qq.com>
* fix: resolve duplicate bound error in scan_dir function
Replaced inline trait bounds with where clause to avoid duplication caused by macro expansion.
Signed-off-by: reigadegr <2722688642@qq.com>
---------
Signed-off-by: reigadegr <2722688642@qq.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
* fix:make bucket exists (#428)
* feat: include user-defined metadata in S3 response (#431)
* fix: simplify Docker entrypoint following efficient user switching pattern (#421)
* fix: simplify Docker entrypoint following efficient user switching pattern
- Remove ALL file permission modifications (no chown at all)
- Use chroot --userspec or gosu to switch user context
- Extremely simple and fast implementation
- Zero filesystem modifications for permissions
Fixes#388
* Update entrypoint.sh
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update entrypoint.sh
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update entrypoint.sh
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* wip
* wip
* wip
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* docs: update doc/docker-data-dir README.md (#432)
* add targets crates
* feat(targets): extract targets module into a standalone crate
- Move all target-related code (MQTT, Webhook, etc.) into a new `targets` crate
- Update imports and dependencies to reference the new crate
- Refactor interfaces to ensure compatibility with the new crate structure
- Adjust Cargo.toml and workspace configuration accordingly
* fix
* fix
---------
Signed-off-by: junxiang Mu <1948535941@qq.com>
Signed-off-by: reigadegr <2722688642@qq.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: zzhpro <56196563+zzhpro@users.noreply.github.com>
Co-authored-by: junxiang Mu <1948535941@qq.com>
Co-authored-by: weisd <im@weisd.in>
Co-authored-by: shiro.lee <69624924+shiroleeee@users.noreply.github.com>
Co-authored-by: majinghe <42570491+majinghe@users.noreply.github.com>
Co-authored-by: guojidan <63799833+guojidan@users.noreply.github.com>
Co-authored-by: reigadegr <103645642+reigadegr@users.noreply.github.com>
Co-authored-by: 0xdx2 <xuedamon2@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
This commit introduces two key enhancements: the integration of Grafana Tempo for distributed tracing and the implementation of a dual-stack TCP listener for improved network compatibility.
- **Observability**:
- Adds the `tempo` service to the `docker-compose.yml` observability stack.
- Tempo is configured to collect and store traces, integrating with the existing OpenTelemetry setup.
- A custom `tempo-entrypoint.sh` script is included to manage volume permissions on startup.
- **Networking**:
- Modifies `http.rs` to support dual-stack (IPv4/IPv6) connections on a single socket.
- By setting the `IPV6_V6ONLY` socket option to `false`, the server can now accept both IPv6 and IPv4-mapped IPv6 traffic, enhancing cross-platform support.
* feat: integrate CancellationToken for unified background services management
- Consolidate data scanner and auto heal cancellation tokens into single unified token
- Move GLOBAL_BACKGROUND_SERVICES_CANCEL_TOKEN to global.rs for centralized management
- Add graceful shutdown support to MRF heal routine with MinIO-compatible logic
- Implement heal_routine_with_cancel method preserving original healing logic
- Update main.rs to use unified background services shutdown mechanism
- Enhance error handling with proper ecstore Result types
- Fix clippy warnings for needless return statements
- Maintain backward compatibility while adding modern cancellation support
This change provides a cleaner architecture for background service lifecycle management
and ensures all healing services can be gracefully shut down through a single token.
Signed-off-by: junxiang Mu <1948535941@qq.com>
* fix: Refact heal and scanner design
Signed-off-by: junxiang Mu <1948535941@qq.com>
* refact: step 2
Signed-off-by: junxiang Mu <1948535941@qq.com>
* feat: refactor scanner module and add data usage statistics
- Move scanner code to scanner/ subdirectory for better organization
- Add data usage statistics collection and persistence
- Implement histogram support for size and version distribution
- Add global cancel token management for scanner operations
- Integrate scanner with ECStore for comprehensive data analysis
- Update error handling and improve test isolation
- Add data usage API endpoints and backend integration
Signed-off-by: junxiang Mu <1948535941@qq.com>
* Chore: fix ref and fix comment
Signed-off-by: junxiang Mu <1948535941@qq.com>
* fix: fix clippy
Signed-off-by: junxiang Mu <1948535941@qq.com>
---------
Signed-off-by: junxiang Mu <1948535941@qq.com>
Co-authored-by: dandan <dandan@dandandeMac-Studio.local>
Move the construction of the hybrid service stack, including all middleware and the RPC service, from the main `run` function into the `process_connection` function.
This change ensures that each incoming connection gets its own isolated service instance. This improves modularity by making the connection handling logic more self-contained and simplifies the main server loop.
Key changes:
- The `hybrid_service` and `rpc_service` are now created inside `process_connection`.
- The `run` function's responsibility is reduced to accepting TCP connections and spawning tasks for `process_connection`.
This commit introduces a significant reorganization of the project structure to improve maintainability and clarity.
Key changes include:
- Adjusted the directory layout for a more logical module organization.
- Removed unused crate dependencies, reducing the overall project size and potentially speeding up build times.
- Updated import paths and configuration files to reflect the structural changes.
* chore: Add copyright and license headers
This commit adds the Apache 2.0 license and a copyright notice to the header of all source files. This ensures that the licensing and copyright information is clearly stated within the codebase.
* cargo fmt
* fix
* fmt
* fix clippy
This change simplifies the crate name to better reflect its core functionality
as the event handling system for RustFS. The renamed package maintains all
existing functionality while improving naming consistency across the project.
- Updated all imports and references to use the new crate name
- Maintained API compatibility with existing implementations
- Updated tests to reflect the name change
* improve code for opentelemetry and add system metrics
* feat: add TraceLayer for HTTP service and improve metrics
- Add TraceLayer to HTTP server for request tracing
- Implement system metrics for process monitoring
- Optimize init_telemetry method for better resource management
- Add graceful shutdown handling for telemetry components
- Fix GracefulShutdown ownership issues with Arc wrapper
* improve code for init_process_observer
* remove tomlfmt.toml
* Translation comment
* improve code for console CompressionLayer params
- Add core event notification interfaces
- Support multiple notification backends:
- Webhook (default)
- Kafka
- MQTT
- HTTP Producer
- Implement configurable event filtering
- Add async event dispatching with backpressure handling
- Provide serialization/deserialization for event payloads
This module enables system events to be published to various endpoints
with consistent delivery guarantees and failure handling.
guojidan