Commit Graph

134 Commits

Author SHA1 Message Date
overtrue 5fff195295 feat: complete zip compression module implementation 2025-05-25 13:14:38 +08:00
houseme dd7da015e3 Feature/rustfs config (#396)
* init rustfs config

* improve code for rustfs-config crate

* add

* improve code for comment

* fix: modify rustfs-config crate name

* add default fn

* improve error logger

* fix: modify docker config yaml

* improve code for config

* feat: restrict kafka feature to Linux only

- Add target-specific feature configuration in Cargo.toml for obs and event-notifier crates
- Implement conditional compilation for kafka feature only on Linux systems
- Add appropriate error handling for non-Linux platforms
- Ensure backward compatibility with existing code

* refactor(ci): optimize build workflow for better efficiency

- Integrate GUI build steps into main build-rustfs job
- Add conditional GUI build execution based on tag releases
- Simplify workflow by removing redundant build-rustfs-gui job
- Copy binary directly to embedded-rustfs directory without downloading artifacts
- Update merge job dependency to only rely on build-rustfs
- Improve cross-platform compatibility for Windows binary naming (.exe)
- Streamline artifact uploading and OSS publishing process
- Maintain consistent conditional logic for release operations

* refactor(ci): optimize build workflow for better efficiency

- Integrate GUI build steps into main build-rustfs job
- Add conditional GUI build execution based on tag releases
- Simplify workflow by removing redundant build-rustfs-gui job
- Copy binary directly to embedded-rustfs directory without downloading artifacts
- Update merge job dependency to only rely on build-rustfs
- Improve cross-platform compatibility for Windows binary naming (.exe)
- Streamline artifact uploading and OSS publishing process
- Maintain consistent conditional logic for release operations

* fix(ci): add repo-token to setup-protoc action for authentication

- Add GITHUB_TOKEN parameter to arduino/setup-protoc@v3 action
- Ensure proper authentication for Protoc installation in CI workflow
- Maintain consistent setup across different CI environments

* modify config

* improve readme.md

* remove env config relation

* add allow(dead_code)
2025-05-12 01:17:31 +08:00
loverustfs e6e0821512 Merge pull request #389 from rustfs/feat/zip
feat: auto-extract support
2025-05-09 16:19:21 +08:00
weisd d7cac5c727 Merge branch 'main' into dada/fix-entry 2025-05-09 14:33:30 +08:00
weisd 76fdefeca4 feat: auto-extract support 2025-05-08 17:42:20 +08:00
houseme 29ddf4dbc8 refactor: standardize constant management and fix typos (#387)
* init rustfs config

* init rustfs-utils crate

* improve code for rustfs-config crate

* add

* improve code for comment

* init rustfs config

* improve code for rustfs-config crate

* add

* improve code for comment

* Unified management of configurations and constants

* fix: modify rustfs-config crate name

* add default fn

* improve code for rustfs config

* refactor: standardize constant management and fix typos

- Create centralized constants module for global static constants
- Replace runtime format! expressions with compile-time constants
- Fix DEFAULT_PORT reference issues in configuration arguments
- Use const-str crate for compile-time string concatenation
- Update tokio dependency from 1.42.2 to 1.45.0
- Ensure consistent naming convention for configuration constants

* fix

* Update common/workers/src/workers.rs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-05-07 17:23:22 +08:00
houseme fc47ca9dd2 upgrade version (#380) 2025-05-06 08:54:35 +08:00
weisd 750194e4dc test 2025-04-28 13:17:06 +08:00
houseme e9d6e2ca95 feat: improve address binding and port handling mechanism (#366)
* feat: improve address binding and port handling mechanism

1. Add support for ":port" format to enable dual-stack binding (IPv4/IPv6)
2. Implement automatic port allocation when port 0 is specified
3. Optimize server startup process with unified address resolution
4. Enhance error handling and logging for address resolution
5. Improve graceful shutdown with signal listening
6. Clean up commented code in console.rs

Files:
- ecstore/src/utils/net.rs
- rustfs/src/console.rs
- rustfs/src/main.rs

Branch: feature/server-and-console-port

* improve code for console

* improve code

* improve code for console and net.rs

* Update rustfs/src/main.rs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update rustfs/src/utils/mod.rs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-04-27 23:44:26 +08:00
houseme 86353d98d5 feat: add TraceLayer for HTTP service and improve metrics (#361)
* improve code for opentelemetry and add system metrics

* feat: add TraceLayer for HTTP service and improve metrics

- Add TraceLayer to HTTP server for request tracing
- Implement system metrics for process monitoring
- Optimize init_telemetry method for better resource management
- Add graceful shutdown handling for telemetry components
- Fix GracefulShutdown ownership issues with Arc wrapper

* improve code for init_process_observer

* remove tomlfmt.toml

* Translation comment

* improve code for console CompressionLayer params
2025-04-24 19:04:57 +08:00
weisd cffd4fc28d merge fix/pools 2025-04-23 16:48:57 +08:00
weisd fff7e5f827 pool select idx
fixs:#346, #339, #338, #337, #336, #334

test healbucket

test get_available_pool_idx

fix
2025-04-23 15:11:43 +08:00
houseme e4453adf82 feat(event-notifier): improve environment variable handling
- Fix deserialization error when parsing config from environment variables
- Add proper array format support for adapters configuration
- Update environment variable examples with correct format
- Improve documentation for configuration loading
- Implement helper functions for environment variable validation

This change fixes the "invalid type: map, expected a sequence" error
by ensuring proper formatting of array-type fields in environment variables.
2025-04-22 20:31:38 +08:00
houseme 15b6a426fb Merge branch 'main' of github.com:rustfs/s3-rustfs into feature/bucket-event-notification
# Conflicts:
#	Cargo.toml
2025-04-22 09:16:48 +08:00
weisd 59c008b0ee fix pool select idx 2025-04-22 09:07:43 +08:00
houseme 770f28d205 improve Cargo toml 2025-04-21 15:27:58 +08:00
houseme 3b6397012b feat(event-notifier): improve notification system initialization safety
- Add READY atomic flag to track full initialization status
- Implement initialize_safe and start_safe methods with mutex protection
- Add wait_until_ready function with configurable timeout
- Create initialize_and_start_with_ready_check helper method
- Replace sleep-based waiting with proper readiness checks
- Add safety checks before sending events
- Replace chrono with std::time for time handling
- Update error handling to provide clear initialization status

This change reduces race conditions in multi-threaded environments
and ensures events are only processed when the system is fully ready.
2025-04-21 13:28:01 +08:00
houseme bfc165abe0 feat: implement event notification system
- Add core event notification interfaces
- Support multiple notification backends:
  - Webhook (default)
  - Kafka
  - MQTT
  - HTTP Producer
- Implement configurable event filtering
- Add async event dispatching with backpressure handling
- Provide serialization/deserialization for event payloads

This module enables system events to be published to various endpoints
with consistent delivery guarantees and failure handling.
2025-04-21 00:17:27 +08:00
houseme 21a829e7cf init event notifer 2025-04-19 02:20:50 +08:00
Nugine 800ced24b7 build(deps): upgrade s3s
resolves: #322
2025-04-16 15:16:19 +08:00
Nugine 85c8ea5ba6 build(deps): upgrade s3s 2025-04-15 17:37:08 +08:00
houseme 37109fc618 upgrade crate version 2025-04-11 20:53:51 +08:00
houseme 24d6c555f7 Remove unused crate 2025-04-11 17:38:44 +08:00
houseme 42c8890d4a Merge branch 'main' of github.com:rustfs/s3-rustfs into feature/Systemd.service 2025-04-11 16:48:46 +08:00
houseme ab8b19eb5d improve signal watch 2025-04-11 16:48:07 +08:00
Nugine 1b24fbdb00 fix: upgrade s3s 2025-04-11 16:01:46 +08:00
houseme 952f04149f add notify systemd 2025-04-09 19:11:31 +08:00
houseme c872901269 improve Cargo.toml and modify README.md conosel web static url 2025-04-09 18:36:17 +08:00
houseme a07ca8ae81 improve code for obs 2025-04-09 15:12:31 +08:00
junxiang Mu c900faba81 fix upgrade axum bug
Signed-off-by: junxiang Mu <1948535941@qq.com>
2025-04-09 06:19:36 +00:00
junxiang Mu 99a5866680 degrade rand && object_store
Signed-off-by: junxiang Mu <1948535941@qq.com>
2025-04-09 03:27:03 +00:00
junxiang Mu ebf1a9d9c4 update tonic axum
Signed-off-by: junxiang Mu <1948535941@qq.com>
2025-04-09 02:41:06 +00:00
dependabot[bot] f6b734ca69 Bump the dependencies group across 1 directory with 3 updates
Bumps the dependencies group with 3 updates in the / directory: [rand](https://github.com/rust-random/rand), [tokio](https://github.com/tokio-rs/tokio) and [object_store](https://github.com/apache/arrow-rs).


Updates `rand` from 0.8.5 to 0.9.0
- [Release notes](https://github.com/rust-random/rand/releases)
- [Changelog](https://github.com/rust-random/rand/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-random/rand/compare/0.8.5...0.9.0)

Updates `tokio` from 1.44.1 to 1.44.2
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.44.1...tokio-1.44.2)

Updates `object_store` from 0.11.2 to 0.12.0
- [Release notes](https://github.com/apache/arrow-rs/releases)
- [Changelog](https://github.com/apache/arrow-rs/blob/main/CHANGELOG-old.md)
- [Commits](https://github.com/apache/arrow-rs/compare/object_store_0.11.2...object_store_0.12.0)

---
updated-dependencies:
- dependency-name: rand
  dependency-version: 0.9.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dependencies
- dependency-name: tokio
  dependency-version: 1.44.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dependencies
- dependency-name: object_store
  dependency-version: 0.12.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-04-08 15:33:56 +00:00
houseme 9964457d09 upgrade opentelemetry version from 0.29 to 0.29.1 2025-04-08 23:26:02 +08:00
weisd 885944802a license api 2025-04-08 22:32:21 +08:00
houseme 0b552b1697 run.sh add RUSTFS_OBS_CONFIG = "./config/obs.example.toml" 2025-04-02 17:18:03 +08:00
houseme 0c7748658c Merge main branches 2025-04-02 15:57:11 +08:00
houseme 15efeb572f improve crate and remove log crate 2025-04-02 00:51:59 +08:00
houseme d017409f5b add rustfs tls 2025-04-02 00:48:08 +08:00
houseme 28edca1b63 add rustls 2025-04-01 23:09:47 +08:00
houseme 1994302574 improve tls for console 2025-04-01 22:06:47 +08:00
houseme f2692b78dd TryInto cover 2025-03-31 18:32:22 +08:00
houseme 96cc708b5b upgrade reqwest version from 0.12.12 to 0.12.15 2025-03-31 16:32:22 +08:00
houseme dad94b5c74 Merge branch 'main' of github.com:rustfs/s3-rustfs into feature/observability
# Conflicts:
#	Cargo.lock
#	Cargo.toml
2025-03-31 15:12:30 +08:00
junxiang Mu b950f61b70 rebase to main
Signed-off-by: junxiang Mu <1948535941@qq.com>
2025-03-31 06:32:05 +00:00
junxiang Mu 0b270bf0cc tmp2
Signed-off-by: junxiang Mu <1948535941@qq.com>
2025-03-31 05:44:48 +00:00
junxiang Mu 63ef986bac tmp1
Signed-off-by: junxiang Mu <1948535941@qq.com>
2025-03-31 05:43:01 +00:00
houseme 147df8ab0b Merge branch 'main' of github.com:rustfs/s3-rustfs into feature/observability
# Conflicts:
#	rustfs/src/storage/ecfs.rs
2025-03-26 16:08:41 +08:00
houseme 23ead0ea99 add prometheus 2025-03-26 15:18:23 +08:00
weisd c8e13b8ab5 move policy out of iam 2025-03-26 10:19:47 +08:00