* init event notifer * feat: implement event notification system - Add core event notification interfaces - Support multiple notification backends: - Webhook (default) - Kafka - MQTT - HTTP Producer - Implement configurable event filtering - Add async event dispatching with backpressure handling - Provide serialization/deserialization for event payloads This module enables system events to be published to various endpoints with consistent delivery guarantees and failure handling. * feat(event-notifier): improve notification system initialization safety - Add READY atomic flag to track full initialization status - Implement initialize_safe and start_safe methods with mutex protection - Add wait_until_ready function with configurable timeout - Create initialize_and_start_with_ready_check helper method - Replace sleep-based waiting with proper readiness checks - Add safety checks before sending events - Replace chrono with std::time for time handling - Update error handling to provide clear initialization status This change reduces race conditions in multi-threaded environments and ensures events are only processed when the system is fully ready. * fix sql Signed-off-by: junxiang Mu <1948535941@qq.com> * move protobuf generate into bin crate Signed-off-by: junxiang Mu <1948535941@qq.com> * improve Cargo toml * improve code * improve code for global * improve code * feat(event-notifier): improve environment variable handling - Fix deserialization error when parsing config from environment variables - Add proper array format support for adapters configuration - Update environment variable examples with correct format - Improve documentation for configuration loading - Implement helper functions for environment variable validation This change fixes the "invalid type: map, expected a sequence" error by ensuring proper formatting of array-type fields in environment variables. * feat: integrate event-notifier system with rustfs - Rename package from rustfs-event-notifier to event-notifier for consistency - Add shutdown hooks for event notification system in main process - Handle graceful termination of notification services on server shutdown - Implement initialization and configuration loading for event notification - Fix environment variable configuration to properly parse adapter arrays - Update example code to demonstrate proper configuration usage This change ensures proper integration between rustfs and the event notification system, with clean startup and shutdown sequences to prevent resource leaks during application lifecycle. * feat: improve webhook server and run script integration - Enhance webhook example with proper shutdown handling using tokio::select! - Update run.sh to automatically start webhook server alongside main service - Add event notification configuration to run.sh using environment variables - Set proper port bindings to ensure webhook server starts on port 3000 - Improve console output for better debugging experience - Fix race condition during service startup and shutdown This change ensures proper integration between the webhook server and the main rustfs service, providing a seamless development experience with automatic service discovery and clean termination. * improve for logger * improve code for global.rs * fix: modify webhook port * fix --------- Signed-off-by: junxiang Mu <1948535941@qq.com> Co-authored-by: junxiang Mu <1948535941@qq.com>
How to compile RustFS
| Must package | Version | download link |
|---|---|---|
| Rust | 1.8.5 | https://www.rust-lang.org/tools/install |
| protoc | 30.2 | protoc-30.2-linux-x86_64.zip |
| flatc | 24.0+ | Linux.flatc.binary.g++-13.zip |
Download Links:
https://github.com/google/flatbuffers/releases/download/v25.2.10/Linux.flatc.binary.g++-13.zip
https://github.com/protocolbuffers/protobuf/releases/download/v30.2/protoc-30.2-linux-x86_64.zip
generate protobuf code:
cargo run --bin gproto
Or use Docker:
- uses: arduino/setup-protoc@v3
with:
version: "30.2"
- uses: Nugine/setup-flatc@v1
with:
version: "25.2.10"
How to add Console web
-
wget https://dl.rustfs.com/artifacts/console/rustfs-console-latest.zip -
mkdir in this repos folder
./rustfs/static -
Compile RustFS
Star RustFS
Add Env Information:
export RUSTFS_VOLUMES="./target/volume/test"
export RUSTFS_ADDRESS="0.0.0.0:9000"
export RUSTFS_CONSOLE_ENABLE=true
export RUSTFS_CONSOLE_ADDRESS="0.0.0.0:9001"
# 具体路径修改为配置文件真实路径,obs.example.toml 仅供参考 其中`RUSTFS_OBS_CONFIG` 和下面变量二选一
export RUSTFS_OBS_CONFIG="./deploy/config/obs.example.toml"
# 如下变量需要必须参数都有值才可以,以及会覆盖配置文件`obs.example.toml`中的值
export RUSTFS__OBSERVABILITY__ENDPOINT=http://localhost:4317
export RUSTFS__OBSERVABILITY__USE_STDOUT=true
export RUSTFS__OBSERVABILITY__SAMPLE_RATIO=2.0
export RUSTFS__OBSERVABILITY__METER_INTERVAL=30
export RUSTFS__OBSERVABILITY__SERVICE_NAME=rustfs
export RUSTFS__OBSERVABILITY__SERVICE_VERSION=0.1.0
export RUSTFS__OBSERVABILITY__ENVIRONMENT=develop
export RUSTFS__OBSERVABILITY__LOGGER_LEVEL=info
export RUSTFS__SINKS__FILE__ENABLED=true
export RUSTFS__SINKS__FILE__PATH="./deploy/logs/rustfs.log"
export RUSTFS__SINKS__WEBHOOK__ENABLED=false
export RUSTFS__SINKS__WEBHOOK__ENDPOINT=""
export RUSTFS__SINKS__WEBHOOK__AUTH_TOKEN=""
export RUSTFS__SINKS__KAFKA__ENABLED=false
export RUSTFS__SINKS__KAFKA__BOOTSTRAP_SERVERS=""
export RUSTFS__SINKS__KAFKA__TOPIC=""
export RUSTFS__LOGGER__QUEUE_CAPACITY=10
You need replace your real data folder:
./rustfs /data/rustfs
How to deploy the observability stack
The OpenTelemetry Collector offers a vendor-agnostic implementation on how to receive, process, and export telemetry data. It removes the need to run, operate, and maintain multiple agents/collectors in order to support open-source observability data formats (e.g. Jaeger, Prometheus, etc.) sending to one or more open-source or commercial back-ends.
- Enter the
.docker/observabilitydirectory, - Run the following command:
docker-compose -f docker-compose.yml up -d
-
Access the Grafana dashboard by navigating to
http://localhost:3000in your browser. The default username and password areadminandadmin, respectively. -
Access the Jaeger dashboard by navigating to
http://localhost:16686in your browser. -
Access the Prometheus dashboard by navigating to
http://localhost:9090in your browser.