Commit Graph

1004 Commits

Author SHA1 Message Date
junxiang Mu e346d20228 tmp1
Signed-off-by: junxiang Mu <1948535941@qq.com>
2025-04-29 02:53:42 +00:00
junxiang Mu 66f5bf1bbc tmp1
Signed-off-by: junxiang Mu <1948535941@qq.com>
2025-04-28 21:55:31 +08:00
junxiang Mu c1590a054c tmp
Signed-off-by: junxiang Mu <1948535941@qq.com>
2025-04-28 10:00:28 +00:00
junxiang Mu 569099af9e fix readme
Signed-off-by: junxiang Mu <1948535941@qq.com>
2025-04-28 06:25:32 +00:00
houseme 68a89d59e5 modify 2025-04-28 12:50:56 +08:00
houseme c25c94ec30 upgrade config file 2025-04-28 11:34:52 +08:00
houseme e9d6e2ca95 feat: improve address binding and port handling mechanism (#366)
* feat: improve address binding and port handling mechanism

1. Add support for ":port" format to enable dual-stack binding (IPv4/IPv6)
2. Implement automatic port allocation when port 0 is specified
3. Optimize server startup process with unified address resolution
4. Enhance error handling and logging for address resolution
5. Improve graceful shutdown with signal listening
6. Clean up commented code in console.rs

Files:
- ecstore/src/utils/net.rs
- rustfs/src/console.rs
- rustfs/src/main.rs

Branch: feature/server-and-console-port

* improve code for console

* improve code

* improve code for console and net.rs

* Update rustfs/src/main.rs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update rustfs/src/utils/mod.rs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-04-27 23:44:26 +08:00
houseme e8f8a0872d modify Telemetry filter order 2025-04-27 09:50:32 +08:00
weisd 87bec2e6a6 fix rebalance/decom 2025-04-26 22:55:37 +08:00
houseme 9590d99e7c Feature/upgrade obs docker (#364)
* upgrade docker config

* upgrade obs.toml

* modify dockerfile image from alpine to ubuntu
2025-04-26 22:36:38 +08:00
houseme 05f1412323 improve code for opentelemetry 2025-04-26 17:26:54 +08:00
guojidan ed4a9db9a0 Merge pull request #362 from rustfs/pref
fix
2025-04-25 17:02:03 +08:00
junxiang Mu 8e3c22b595 fix
Signed-off-by: junxiang Mu <1948535941@qq.com>
2025-04-25 08:59:39 +00:00
weisd 9fc4bb919e fix:#355 multi pools select error 2025-04-25 14:52:20 +08:00
houseme 7dae5f8ab7 improve code 2025-04-25 13:35:03 +08:00
houseme 7926ac015a improve code for tracing log 2025-04-24 19:14:46 +08:00
houseme 86353d98d5 feat: add TraceLayer for HTTP service and improve metrics (#361)
* improve code for opentelemetry and add system metrics

* feat: add TraceLayer for HTTP service and improve metrics

- Add TraceLayer to HTTP server for request tracing
- Implement system metrics for process monitoring
- Optimize init_telemetry method for better resource management
- Add graceful shutdown handling for telemetry components
- Fix GracefulShutdown ownership issues with Arc wrapper

* improve code for init_process_observer

* remove tomlfmt.toml

* Translation comment

* improve code for console CompressionLayer params
2025-04-24 19:04:57 +08:00
guojidan a095a607c0 Merge pull request #360 from rustfs/pref
fix
2025-04-24 16:42:32 +08:00
junxiang Mu 4613b36697 fix
Signed-off-by: junxiang Mu <1948535941@qq.com>
2025-04-24 08:41:40 +00:00
guojidan e67c7d977f Merge pull request #359 from rustfs/pref
fix admin info
2025-04-24 16:22:03 +08:00
junxiang Mu e38dd25bf6 fix admin info
Signed-off-by: junxiang Mu <1948535941@qq.com>
2025-04-24 08:21:09 +00:00
guojidan 35dac63c36 Merge pull request #358 from rustfs/pref
Pref
2025-04-24 16:13:44 +08:00
junxiang Mu b2da8148bd support concurrency write
Signed-off-by: junxiang Mu <1948535941@qq.com>
2025-04-24 08:12:57 +00:00
weisd 56575d58f4 fix:#352, #353
fix: verify_file bug
2025-04-24 15:32:12 +08:00
weisd a745b91ded fix:#351 delete object err 2025-04-24 13:22:34 +08:00
junxiang Mu 08d0021cd6 support async calculate etag
Signed-off-by: junxiang Mu <1948535941@qq.com>
2025-04-24 03:04:05 +00:00
weisd 707c773dd9 rm unuse log 2025-04-24 09:22:31 +08:00
guojidan e58eaacb20 Merge pull request #349 from rustfs/fix-data-scan
Fix data scan
2025-04-23 17:30:05 +08:00
junxiang Mu 45a9b6313e fmt
Signed-off-by: junxiang Mu <1948535941@qq.com>
2025-04-23 09:29:37 +00:00
junxiang Mu 380a451bc1 fix capacity
Signed-off-by: junxiang Mu <1948535941@qq.com>
2025-04-23 09:26:22 +00:00
weisd 827818b493 update info version output 2025-04-23 17:03:03 +08:00
weisd cffd4fc28d merge fix/pools 2025-04-23 16:48:57 +08:00
weisd cf2ed47fe8 fix: #331 admin info version,uptime 2025-04-23 16:34:39 +08:00
guojidan ad7ff448d6 Merge pull request #348 from rustfs/fix-data-scan
fix
2025-04-23 16:28:12 +08:00
junxiang Mu 8981dfae9f fix
Signed-off-by: junxiang Mu <1948535941@qq.com>
2025-04-23 07:26:37 +00:00
weisd fff7e5f827 pool select idx
fixs:#346, #339, #338, #337, #336, #334

test healbucket

test get_available_pool_idx

fix
2025-04-23 15:11:43 +08:00
houseme 873a04aed0 Feature/bucket event notification (#347)
* init event notifer

* feat: implement event notification system

- Add core event notification interfaces
- Support multiple notification backends:
  - Webhook (default)
  - Kafka
  - MQTT
  - HTTP Producer
- Implement configurable event filtering
- Add async event dispatching with backpressure handling
- Provide serialization/deserialization for event payloads

This module enables system events to be published to various endpoints
with consistent delivery guarantees and failure handling.

* feat(event-notifier): improve notification system initialization safety

- Add READY atomic flag to track full initialization status
- Implement initialize_safe and start_safe methods with mutex protection
- Add wait_until_ready function with configurable timeout
- Create initialize_and_start_with_ready_check helper method
- Replace sleep-based waiting with proper readiness checks
- Add safety checks before sending events
- Replace chrono with std::time for time handling
- Update error handling to provide clear initialization status

This change reduces race conditions in multi-threaded environments
and ensures events are only processed when the system is fully ready.

* fix sql

Signed-off-by: junxiang Mu <1948535941@qq.com>

* move protobuf generate into bin crate

Signed-off-by: junxiang Mu <1948535941@qq.com>

* improve Cargo toml

* improve code

* improve code for global

* improve code

* feat(event-notifier): improve environment variable handling

- Fix deserialization error when parsing config from environment variables
- Add proper array format support for adapters configuration
- Update environment variable examples with correct format
- Improve documentation for configuration loading
- Implement helper functions for environment variable validation

This change fixes the "invalid type: map, expected a sequence" error
by ensuring proper formatting of array-type fields in environment variables.

* feat: integrate event-notifier system with rustfs

- Rename package from rustfs-event-notifier to event-notifier for consistency
- Add shutdown hooks for event notification system in main process
- Handle graceful termination of notification services on server shutdown
- Implement initialization and configuration loading for event notification
- Fix environment variable configuration to properly parse adapter arrays
- Update example code to demonstrate proper configuration usage

This change ensures proper integration between rustfs and the event
notification system, with clean startup and shutdown sequences to prevent
resource leaks during application lifecycle.

* feat: improve webhook server and run script integration

- Enhance webhook example with proper shutdown handling using tokio::select!
- Update run.sh to automatically start webhook server alongside main service
- Add event notification configuration to run.sh using environment variables
- Set proper port bindings to ensure webhook server starts on port 3000
- Improve console output for better debugging experience
- Fix race condition during service startup and shutdown

This change ensures proper integration between the webhook server and
the main rustfs service, providing a seamless development experience
with automatic service discovery and clean termination.

* improve for logger

* improve code for global.rs

* fix: modify webhook port

* fix

---------

Signed-off-by: junxiang Mu <1948535941@qq.com>
Co-authored-by: junxiang Mu <1948535941@qq.com>
2025-04-22 23:08:03 +08:00
houseme b2e1a7ac33 fix 2025-04-22 23:07:13 +08:00
houseme ea13098beb fix: modify webhook port 2025-04-22 23:06:43 +08:00
houseme 52a1f9b8a7 improve code for global.rs 2025-04-22 22:59:31 +08:00
houseme 6d1a4ab824 improve for logger 2025-04-22 22:41:45 +08:00
houseme c8ab89292e feat: improve webhook server and run script integration
- Enhance webhook example with proper shutdown handling using tokio::select!
- Update run.sh to automatically start webhook server alongside main service
- Add event notification configuration to run.sh using environment variables
- Set proper port bindings to ensure webhook server starts on port 3000
- Improve console output for better debugging experience
- Fix race condition during service startup and shutdown

This change ensures proper integration between the webhook server and
the main rustfs service, providing a seamless development experience
with automatic service discovery and clean termination.
2025-04-22 21:40:20 +08:00
houseme 4f347a92c1 feat: integrate event-notifier system with rustfs
- Rename package from rustfs-event-notifier to event-notifier for consistency
- Add shutdown hooks for event notification system in main process
- Handle graceful termination of notification services on server shutdown
- Implement initialization and configuration loading for event notification
- Fix environment variable configuration to properly parse adapter arrays
- Update example code to demonstrate proper configuration usage

This change ensures proper integration between rustfs and the event
notification system, with clean startup and shutdown sequences to prevent
resource leaks during application lifecycle.
2025-04-22 20:49:39 +08:00
houseme e4453adf82 feat(event-notifier): improve environment variable handling
- Fix deserialization error when parsing config from environment variables
- Add proper array format support for adapters configuration
- Update environment variable examples with correct format
- Improve documentation for configuration loading
- Implement helper functions for environment variable validation

This change fixes the "invalid type: map, expected a sequence" error
by ensuring proper formatting of array-type fields in environment variables.
2025-04-22 20:31:38 +08:00
houseme 15b6a426fb Merge branch 'main' of github.com:rustfs/s3-rustfs into feature/bucket-event-notification
# Conflicts:
#	Cargo.toml
2025-04-22 09:16:48 +08:00
houseme 0ed92b3b6f improve code 2025-04-22 09:14:09 +08:00
weisd 18b50c1752 otel debug filter tower 2025-04-22 09:07:43 +08:00
weisd 59c008b0ee fix pool select idx 2025-04-22 09:07:43 +08:00
DamonXue 27400394e4 Merge pull request #345 from rustfs/dev_objectEncrypt_v1
feat: update launch configuration and clean unuseful docker-compose config
2025-04-21 22:08:02 +08:00
DamonXue 536b2e2ed9 feat: update launch configuration and docker-compose for enhanced observability and logging 2025-04-21 22:05:31 +08:00