Commit Graph

57 Commits

Author SHA1 Message Date
houseme 4aaeff8e4c add workflow Samply action and modify console address port 9001 2025-05-28 16:02:53 +08:00
overtrue 7069f9e7a2 fix: resolve all doctest failures in rustfs-obs crate 2025-05-28 15:29:09 +08:00
安正超 b2e2c0fecd refactor: remove dead code comment in observability config test 2025-05-28 14:36:38 +08:00
安正超 15befb705f fix: resolve all remaining test failures and Clippy warnings 2025-05-28 11:28:43 +08:00
安正超 a1f4abf6c3 fix: resolve all Clippy warnings across codebase - Fixed field reassignment warnings in ecstore/src/file_meta.rs by using struct initialization instead of default + field assignment - Fixed overly complex boolean expression in ecstore/src/utils/os/mod.rs by removing meaningless assertion - Replaced manual Default implementation with derive in crates/zip/src/lib.rs - Updated io::Error usage to use io::Error::other() instead of deprecated pattern - Removed useless assertions and clone-on-copy warnings - Fixed unwrap usage by replacing with expect() providing meaningful error messages - Fixed useless vec usage by using array repeat instead - All code now passes comprehensive Clippy check with --all-targets --all-features -- -D warnings 2025-05-28 11:00:07 +08:00
overtrue dbe573513e feat: add comprehensive test coverage for config module 2025-05-27 23:11:29 +08:00
overtrue c29841a5e7 feat: add comprehensive test coverage for utils certs module 2025-05-27 22:44:30 +08:00
houseme ca8f399832 format comment 2025-05-27 13:56:19 +08:00
houseme 7abcfe31e8 improve code for otel (#418) 2025-05-26 12:05:57 +08:00
houseme 864071d641 cargo fmt 2025-05-25 17:46:59 +08:00
overtrue 9d90913697 feat: translate Chinese comments to English across codebase 2025-05-25 15:24:34 +08:00
overtrue 0f1e9d0c63 feat: enhance test coverage for IAM utils and OS utils modules 2025-05-25 14:09:40 +08:00
overtrue bc398ccf1b feat: improve test coverage and fix critical crypto bug - Translate all Chinese comments to English in utils/ip.rs and config/constants/app.rs - Add comprehensive test suite for crypto/encdec/id.rs module (14 new tests) - Fix critical bug in Argon2 key generation that was returning all-zero keys - Improve test coverage for IP utilities and configuration constants - Ensure all test cases follow English naming conventions and meaningful descriptions 2025-05-25 13:53:59 +08:00
overtrue df71cea9af refactor: improve test code quality by replacing meaningless names and content 2025-05-25 13:40:54 +08:00
安正超 077fb4ed69 Merge pull request #404 from rustfs/feat/zip-features
feat: complete zip compression module implementation
2025-05-25 13:35:46 +08:00
overtrue 9604efd5cc feat: add comprehensive tests for event-notifier error module 2025-05-25 13:26:25 +08:00
overtrue 5fff195295 feat: complete zip compression module implementation 2025-05-25 13:14:38 +08:00
安正超 1a91371306 Merge pull request #403 from rustfs/feat/zip-tests
feat: add comprehensive test coverage for zip compression module
2025-05-25 13:11:23 +08:00
overtrue c6b3051c67 feat: add comprehensive test coverage for zip compression module 2025-05-25 13:05:11 +08:00
overtrue a6c3b122bd feat: enhance test coverage and fix compilation errors 2025-05-25 12:56:43 +08:00
houseme 136118ed21 fix 2025-05-14 19:04:52 +08:00
houseme 571cedf4ce feat(obs): implement global OpenTelemetry guard management 2025-05-12 13:32:18 +08:00
houseme dd7da015e3 Feature/rustfs config (#396)
* init rustfs config

* improve code for rustfs-config crate

* add

* improve code for comment

* fix: modify rustfs-config crate name

* add default fn

* improve error logger

* fix: modify docker config yaml

* improve code for config

* feat: restrict kafka feature to Linux only

- Add target-specific feature configuration in Cargo.toml for obs and event-notifier crates
- Implement conditional compilation for kafka feature only on Linux systems
- Add appropriate error handling for non-Linux platforms
- Ensure backward compatibility with existing code

* refactor(ci): optimize build workflow for better efficiency

- Integrate GUI build steps into main build-rustfs job
- Add conditional GUI build execution based on tag releases
- Simplify workflow by removing redundant build-rustfs-gui job
- Copy binary directly to embedded-rustfs directory without downloading artifacts
- Update merge job dependency to only rely on build-rustfs
- Improve cross-platform compatibility for Windows binary naming (.exe)
- Streamline artifact uploading and OSS publishing process
- Maintain consistent conditional logic for release operations

* refactor(ci): optimize build workflow for better efficiency

- Integrate GUI build steps into main build-rustfs job
- Add conditional GUI build execution based on tag releases
- Simplify workflow by removing redundant build-rustfs-gui job
- Copy binary directly to embedded-rustfs directory without downloading artifacts
- Update merge job dependency to only rely on build-rustfs
- Improve cross-platform compatibility for Windows binary naming (.exe)
- Streamline artifact uploading and OSS publishing process
- Maintain consistent conditional logic for release operations

* fix(ci): add repo-token to setup-protoc action for authentication

- Add GITHUB_TOKEN parameter to arduino/setup-protoc@v3 action
- Ensure proper authentication for Protoc installation in CI workflow
- Maintain consistent setup across different CI environments

* modify config

* improve readme.md

* remove env config relation

* add allow(dead_code)
2025-05-12 01:17:31 +08:00
weisd 57cbd35de1 Merge branch 'main' into feat/zip 2025-05-09 14:38:14 +08:00
junxiang Mu 5cb040f863 fix zero size object bug
Signed-off-by: junxiang Mu <1948535941@qq.com>
2025-05-08 19:34:58 +08:00
weisd 76fdefeca4 feat: auto-extract support 2025-05-08 17:42:20 +08:00
houseme 29ddf4dbc8 refactor: standardize constant management and fix typos (#387)
* init rustfs config

* init rustfs-utils crate

* improve code for rustfs-config crate

* add

* improve code for comment

* init rustfs config

* improve code for rustfs-config crate

* add

* improve code for comment

* Unified management of configurations and constants

* fix: modify rustfs-config crate name

* add default fn

* improve code for rustfs config

* refactor: standardize constant management and fix typos

- Create centralized constants module for global static constants
- Replace runtime format! expressions with compile-time constants
- Fix DEFAULT_PORT reference issues in configuration arguments
- Use const-str crate for compile-time string concatenation
- Update tokio dependency from 1.42.2 to 1.45.0
- Ensure consistent naming convention for configuration constants

* fix

* Update common/workers/src/workers.rs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-05-07 17:23:22 +08:00
houseme 01d5383ce3 Feature/bucket event notification (#365)
* add tracing instrument

* fix rebalance/decom

* modify Telemetry filter order

* feat: improve address binding and port handling mechanism (#366)

* feat: improve address binding and port handling mechanism

1. Add support for ":port" format to enable dual-stack binding (IPv4/IPv6)
2. Implement automatic port allocation when port 0 is specified
3. Optimize server startup process with unified address resolution
4. Enhance error handling and logging for address resolution
5. Improve graceful shutdown with signal listening
6. Clean up commented code in console.rs

Files:
- ecstore/src/utils/net.rs
- rustfs/src/console.rs
- rustfs/src/main.rs

Branch: feature/server-and-console-port

* improve code for console

* improve code

* improve code for console and net.rs

* Update rustfs/src/main.rs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update rustfs/src/utils/mod.rs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* upgrade config file

* modify

* fix readme

Signed-off-by: junxiang Mu <1948535941@qq.com>

* improve readme.md

* improve code for readme.md
add chinese readme.md

* Implement Storage Service Event Notification System

Added event notification capability to the storage module, enabling the storage service to publish object operation events. Key changes include:

1. Created `event_notifier` module providing core functionality:
   - `create_metadata()` - Creates event metadata objects with default configuration ID
   - `send_event()` - Asynchronously sends event notifications with error handling

2. Integrated the `rustfs_event_notifier` library:
   - Supports object creation, deletion, and access events
   - Provides event metadata building and management
   - Includes proper error propagation

These changes enable the system to trigger notifications when storage operations occur, facilitating auditing, monitoring, and integration with other systems.

* fix

---------

Signed-off-by: junxiang Mu <1948535941@qq.com>
Co-authored-by: weisd <im@weisd.in>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: junxiang Mu <1948535941@qq.com>
2025-04-30 00:31:55 +08:00
houseme e5cd061e18 improve code for request and telemetry 2025-04-29 11:55:59 +08:00
junxiang Mu e346d20228 tmp1
Signed-off-by: junxiang Mu <1948535941@qq.com>
2025-04-29 02:53:42 +00:00
houseme 68a89d59e5 modify 2025-04-28 12:50:56 +08:00
houseme e8f8a0872d modify Telemetry filter order 2025-04-27 09:50:32 +08:00
houseme 9590d99e7c Feature/upgrade obs docker (#364)
* upgrade docker config

* upgrade obs.toml

* modify dockerfile image from alpine to ubuntu
2025-04-26 22:36:38 +08:00
houseme 05f1412323 improve code for opentelemetry 2025-04-26 17:26:54 +08:00
houseme 7dae5f8ab7 improve code 2025-04-25 13:35:03 +08:00
houseme 86353d98d5 feat: add TraceLayer for HTTP service and improve metrics (#361)
* improve code for opentelemetry and add system metrics

* feat: add TraceLayer for HTTP service and improve metrics

- Add TraceLayer to HTTP server for request tracing
- Implement system metrics for process monitoring
- Optimize init_telemetry method for better resource management
- Add graceful shutdown handling for telemetry components
- Fix GracefulShutdown ownership issues with Arc wrapper

* improve code for init_process_observer

* remove tomlfmt.toml

* Translation comment

* improve code for console CompressionLayer params
2025-04-24 19:04:57 +08:00
weisd cffd4fc28d merge fix/pools 2025-04-23 16:48:57 +08:00
weisd fff7e5f827 pool select idx
fixs:#346, #339, #338, #337, #336, #334

test healbucket

test get_available_pool_idx

fix
2025-04-23 15:11:43 +08:00
houseme ea13098beb fix: modify webhook port 2025-04-22 23:06:43 +08:00
houseme 52a1f9b8a7 improve code for global.rs 2025-04-22 22:59:31 +08:00
houseme 6d1a4ab824 improve for logger 2025-04-22 22:41:45 +08:00
houseme c8ab89292e feat: improve webhook server and run script integration
- Enhance webhook example with proper shutdown handling using tokio::select!
- Update run.sh to automatically start webhook server alongside main service
- Add event notification configuration to run.sh using environment variables
- Set proper port bindings to ensure webhook server starts on port 3000
- Improve console output for better debugging experience
- Fix race condition during service startup and shutdown

This change ensures proper integration between the webhook server and
the main rustfs service, providing a seamless development experience
with automatic service discovery and clean termination.
2025-04-22 21:40:20 +08:00
houseme 4f347a92c1 feat: integrate event-notifier system with rustfs
- Rename package from rustfs-event-notifier to event-notifier for consistency
- Add shutdown hooks for event notification system in main process
- Handle graceful termination of notification services on server shutdown
- Implement initialization and configuration loading for event notification
- Fix environment variable configuration to properly parse adapter arrays
- Update example code to demonstrate proper configuration usage

This change ensures proper integration between rustfs and the event
notification system, with clean startup and shutdown sequences to prevent
resource leaks during application lifecycle.
2025-04-22 20:49:39 +08:00
houseme e4453adf82 feat(event-notifier): improve environment variable handling
- Fix deserialization error when parsing config from environment variables
- Add proper array format support for adapters configuration
- Update environment variable examples with correct format
- Improve documentation for configuration loading
- Implement helper functions for environment variable validation

This change fixes the "invalid type: map, expected a sequence" error
by ensuring proper formatting of array-type fields in environment variables.
2025-04-22 20:31:38 +08:00
houseme 15b6a426fb Merge branch 'main' of github.com:rustfs/s3-rustfs into feature/bucket-event-notification
# Conflicts:
#	Cargo.toml
2025-04-22 09:16:48 +08:00
houseme 0ed92b3b6f improve code 2025-04-22 09:14:09 +08:00
weisd 18b50c1752 otel debug filter tower 2025-04-22 09:07:43 +08:00
houseme 6c70c03985 improve code for global 2025-04-21 21:39:57 +08:00
houseme 5af648230a improve code 2025-04-21 19:01:31 +08:00
houseme 3b6397012b feat(event-notifier): improve notification system initialization safety
- Add READY atomic flag to track full initialization status
- Implement initialize_safe and start_safe methods with mutex protection
- Add wait_until_ready function with configurable timeout
- Create initialize_and_start_with_ready_check helper method
- Replace sleep-based waiting with proper readiness checks
- Add safety checks before sending events
- Replace chrono with std::time for time handling
- Update error handling to provide clear initialization status

This change reduces race conditions in multi-threaded environments
and ensures events are only processed when the system is fully ready.
2025-04-21 13:28:01 +08:00