24e3d3a2ce
refactor(ecstore): Optimize memory usage for object integrity verification
...
Change the object integrity verification from reading all data to streaming processing to avoid memory overflow caused by large objects.
Modify the TLS key log check to use environment variables directly instead of configuration constants.
Add memory limits for object data reading in the AHM module.
Signed-off-by: junxiang Mu <1948535941@qq.com >
2025-08-10 21:24:15 +08:00
e0c99bced4
chore: add tls log and removing unused crates ( #359 )
...
* chore: add tls log
* improve code for http
* improve code dependencies for `cargo.toml` and removing unused crates
* modify name
* improve code
* fix
* Update crates/config/src/constants/env.rs
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* improve code
* fix
* add `is_enabled` and `is_disabled`
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
2025-08-07 19:02:09 +08:00
c5df1f92c2
refactor: replace lazy_static with LazyLock and notify crate registry create_targets_from_config ( #311 )
...
* improve code for notify
* improve code for logger and fix typo (#272 )
* Add GNU to build.yml (#275 )
* fix unzip error
* fix url change error
fix url change error
* Simplify user experience and integrate console and endpoint
Simplify user experience and integrate console and endpoint
* Add gnu to build.yml
* upgrade version
* feat: add `cargo clippy --fix --allow-dirty` to pre-commit command (#282 )
Resolves #277
- Add --fix flag to automatically fix clippy warnings
- Add --allow-dirty flag to run on dirty Git trees
- Improves code quality in pre-commit workflow
* fix: the issue where preview fails when the path length exceeds 255 characters (#280 )
* fix
* fix: improve Windows build support and CI/CD workflow (#283 )
- Fix Windows zip command issue by using PowerShell Compress-Archive
- Add Windows support for OSS upload with ossutil
- Replace Chinese comments with English in build.yml
- Fix bash syntax error in package_zip function
- Improve code formatting and consistency
- Update various configuration files for better cross-platform support
Resolves Windows build failures in GitHub Actions.
* fix: update link in README.md leading to a 404 error (#285 )
* add rustfs.spec for rustfs (#103 )
add support on loongarch64
* improve cargo.lock
* build(deps): bump the dependencies group with 5 updates (#289 )
Bumps the dependencies group with 5 updates:
| Package | From | To |
| --- | --- | --- |
| [hyper-util](https://github.com/hyperium/hyper-util ) | `0.1.15` | `0.1.16` |
| [rand](https://github.com/rust-random/rand ) | `0.9.1` | `0.9.2` |
| [serde_json](https://github.com/serde-rs/json ) | `1.0.140` | `1.0.141` |
| [strum](https://github.com/Peternator7/strum ) | `0.27.1` | `0.27.2` |
| [sysinfo](https://github.com/GuillaumeGomez/sysinfo ) | `0.36.0` | `0.36.1` |
Updates `hyper-util` from 0.1.15 to 0.1.16
- [Release notes](https://github.com/hyperium/hyper-util/releases )
- [Changelog](https://github.com/hyperium/hyper-util/blob/master/CHANGELOG.md )
- [Commits](https://github.com/hyperium/hyper-util/compare/v0.1.15...v0.1.16 )
Updates `rand` from 0.9.1 to 0.9.2
- [Release notes](https://github.com/rust-random/rand/releases )
- [Changelog](https://github.com/rust-random/rand/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-random/rand/compare/rand_core-0.9.1...rand_core-0.9.2 )
Updates `serde_json` from 1.0.140 to 1.0.141
- [Release notes](https://github.com/serde-rs/json/releases )
- [Commits](https://github.com/serde-rs/json/compare/v1.0.140...v1.0.141 )
Updates `strum` from 0.27.1 to 0.27.2
- [Release notes](https://github.com/Peternator7/strum/releases )
- [Changelog](https://github.com/Peternator7/strum/blob/master/CHANGELOG.md )
- [Commits](https://github.com/Peternator7/strum/compare/v0.27.1...v0.27.2 )
Updates `sysinfo` from 0.36.0 to 0.36.1
- [Changelog](https://github.com/GuillaumeGomez/sysinfo/blob/master/CHANGELOG.md )
- [Commits](https://github.com/GuillaumeGomez/sysinfo/compare/v0.36.0...v0.36.1 )
---
updated-dependencies:
- dependency-name: hyper-util
dependency-version: 0.1.16
dependency-type: direct:production
update-type: version-update:semver-patch
dependency-group: dependencies
- dependency-name: rand
dependency-version: 0.9.2
dependency-type: direct:production
update-type: version-update:semver-patch
dependency-group: dependencies
- dependency-name: serde_json
dependency-version: 1.0.141
dependency-type: direct:production
update-type: version-update:semver-patch
dependency-group: dependencies
- dependency-name: strum
dependency-version: 0.27.2
dependency-type: direct:production
update-type: version-update:semver-patch
dependency-group: dependencies
- dependency-name: sysinfo
dependency-version: 0.36.1
dependency-type: direct:production
update-type: version-update:semver-patch
dependency-group: dependencies
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* improve code for logger
* improve
* upgrade
* refactor: 优化构建工作流，统一 latest 文件处理和简化制品上传 (#293 )
* Refactor: DatabaseManagerSystem as global
Signed-off-by: junxiang Mu <1948535941@qq.com >
* fix: fmt
Signed-off-by: junxiang Mu <1948535941@qq.com >
* Test: add e2e_test for s3select
Signed-off-by: junxiang Mu <1948535941@qq.com >
* Test: add test script for e2e
Signed-off-by: junxiang Mu <1948535941@qq.com >
* improve code for registry and intergation
* improve code for registry `create_targets_from_config`
* fix
* Feature up/ilm (#305 )
* fix
* fix
* fix
* fix delete-marker expiration. add api_restore.
* fix
* time retry object upload
* lock file
* make fmt
* fix
* restore object
* fix
* fix
* serde-rs-xml -> quick-xml
* fix
* checksum
* fix
* fix
* fix
* fix
* fix
* fix
* fix
* transfer lang to english
* upgrade clap version from 4.5.41 to 4.5.42
* refactor: replace `lazy_static` with `LazyLock`
* add router
* fix: modify comment
* improve code
* fix typos
* fix
* fix: modify name and fmt
* improve code for registry
* fix test
---------
Signed-off-by: dependabot[bot] <support@github.com >
Signed-off-by: junxiang Mu <1948535941@qq.com >
Co-authored-by: loverustfs <155562731+loverustfs@users.noreply.github.com >
Co-authored-by: 安正超 <anzhengchao@gmail.com >
Co-authored-by: shiro.lee <69624924+shiroleeee@users.noreply.github.com >
Co-authored-by: Marco Orlandin <mipnamic@mipnamic.net >
Co-authored-by: zhangwenlong <zhangwenlong@loongson.cn >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: junxiang Mu <1948535941@qq.com >
Co-authored-by: likewu <likewu@126.com >
2025-07-30 19:02:10 +08:00
36acb5bce9
feat(console): Enhance network address handling for WebUI ( #129 )
...
* add crates homepage,description,keywords,categories,documentation
* add readme
* modify version 0.0.3
* cargo fmt
* fix: yaml.docker-compose.security.no-new-privileges.no-new-privileges-docker-compose.yml (#63 )
* Feature up/ilm (#61 )
* fix delete-marker expiration. add api_restore.
* remove target return 204
* log level
* fix: make lint build and clippy happy (#71 )
Signed-off-by: yihong0618 <zouzou0208@gmail.com >
* fix: make ci and local use the same toolchain (#72 )
Signed-off-by: yihong0618 <zouzou0208@gmail.com >
* feat: optimize GitHub Actions workflows with performance improvements (#77 )
* feat: optimize GitHub Actions workflows with performance improvements
- Rename workflows with more descriptive names
- Add unified setup action for consistent environment setup
- Optimize caching strategy with Swatinem/rust-cache@v2
- Implement skip-check mechanism to avoid duplicate builds
- Simplify matrix builds with better include/exclude logic
- Add intelligent build strategy checks
- Optimize Docker multi-arch builds
- Improve artifact naming and retention
- Add performance testing with benchmark support
- Enhance security audit with dependency scanning
- Change Chinese comments to English for better maintainability
Performance improvements:
- CI testing: ~35 min (42% faster)
- Build release: ~60 min (50% faster)
- Docker builds: ~45 min (50% faster)
- Security audit: ~8 min (47% faster)
* fix: correct secrets context usage in GitHub Actions workflow
- Move environment variables to job level to fix secrets access issue
- Fix unrecognized named-value 'secrets' error in if condition
- Ensure OSS upload step can properly check for required secrets
* fix: resolve GitHub API rate limit by adding authentication token
- Add github-token input to setup action to authenticate GitHub API requests
- Pass GITHUB_TOKEN to all setup action usages to avoid rate limiting
- Fix arduino/setup-protoc@v3 API access issues in CI/CD workflows
- Ensure protoc installation can successfully access GitHub releases API
* fix:make bucket err (#85 )
* Rename DEVELOPMENT.md to CONTRIBUTING.md
* Create issue-translator.yml (#89 )
Enable Issues Translator
* fix(dockerfile): correct env variable names for access/secret key and improve compatibility (#90 )
* fix: restore Zig and cargo-zigbuild caching in GitHub Actions setup action (#92 )
* fix: restore Zig and cargo-zigbuild caching in GitHub Actions setup action
Use mlugg/setup-zig and taiki-e/cache-cargo-install-action to speed up cross-compilation tool installation and avoid repeated downloads. All comments and code are in English.
* fix: use correct taiki-e/install-action for cargo-zigbuild
Use taiki-e/install-action@cargo-zigbuild instead of taiki-e/cache-cargo-install-action@v2 to match the original implementation from PR #77 .
* refactor: remove explicit Zig version to use latest stable
* Create CODE_OF_CONDUCT.md
* Create SECURITY.md
* Update issue templates
* Create CLA.md
* docs: update PR template to English version
* fix: improve data scanner random sleep calculation
- Fix random number generation API usage
- Adjust sleep calculation to follow MinIO pattern
- Ensure proper random range for scanner cycles
Signed-off-by: junxiang Mu <1948535941@qq.com >
* fix: soupprt ipv6
* improve log
* add client ip log
* Update rustfs/src/console.rs
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* improve code
* feat: unify package format to zip for all platforms
---------
Signed-off-by: yihong0618 <zouzou0208@gmail.com >
Signed-off-by: junxiang Mu <1948535941@qq.com >
Co-authored-by: kira-offgrid <kira@offgridsec.com >
Co-authored-by: likewu <likewu@126.com >
Co-authored-by: laoliu <lygn128@163.com >
Co-authored-by: yihong <zouzou0208@gmail.com >
Co-authored-by: 安正超 <anzhengchao@gmail.com >
Co-authored-by: weisd <im@weisd.in >
Co-authored-by: Yone <zhiyu@live.cn >
Co-authored-by: loverustfs <155562731+loverustfs@users.noreply.github.com >
Co-authored-by: junxiang Mu <1948535941@qq.com >
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
2025-07-09 14:39:40 +08:00
3dbcaaa221
docs: simplify crates README files and enforce PR-only workflow ( #110 )
...
* docs: simplify all crates README files
- Remove extensive code examples and detailed documentation
- Convert to minimal module introductions with core feature lists
- Direct users to main RustFS repository for comprehensive docs
- Updated 20 crate README files for consistency and brevity
Files updated:
- crates/rio/README.md (415→15 lines)
- crates/s3select-api/README.md (592→15 lines)
- crates/s3select-query/README.md (658→15 lines)
- crates/signer/README.md (407→15 lines)
- crates/utils/README.md (395→15 lines)
- crates/workers/README.md (463→15 lines)
- crates/zip/README.md (408→15 lines)
* docs: restore original headers in crates README files
- Add back RustFS logo image and CI badges
- Restore formatted headers and structured layout
- Keep simplified content with module introductions
- Maintain consistent documentation structure across all crates
All 20 crate README files now have proper headers while keeping
the simplified content that directs users to the main repository.
* rules: enforce PR-only workflow for main branch
- Strengthen rule that ALL changes must go through pull requests
- Explicitly forbid direct commits to main branch under any circumstances
- Add comprehensive PR requirements and enforcement guidelines
- Clarify that PRs are the ONLY way to merge to main branch
- Add requirement for PR approval before merging
- Include enforcement mechanisms for branch protection
2025-07-08 21:10:07 +08:00
fa17f7b1e3
feat: add comprehensive README documentation for all RustFS submodules ( #48 )
2025-07-04 23:02:13 +08:00
5826396cd0
refactor: Restructure project layout and clean up dependencies ( #30 )
...
This commit introduces a significant reorganization of the project structure to improve maintainability and clarity.
Key changes include:
- Adjusted the directory layout for a more logical module organization.
- Removed unused crate dependencies, reducing the overall project size and potentially speeding up build times.
- Updated import paths and configuration files to reflect the structural changes.
2025-07-02 19:33:12 +08:00
2e14b32ccd
chore: Add copyright and license headers ( #23 )
...
* chore: Add copyright and license headers
This commit adds the Apache 2.0 license and a copyright notice to the header of all source files. This ensures that the licensing and copyright information is clearly stated within the codebase.
* cargo fmt
* fix
* fmt
* fix clippy
2025-07-02 15:07:47 +08:00
3c9ed62bbb
improve code
2025-07-01 20:57:58 +08:00
ddf19b3444
fix: fmt
2025-07-01 13:05:33 +08:00
18a3902d2c
feat(admin): enhance notification event handlers ( #530 )
...
* add log
* add logs
* test
* cargo fmt
* add event admin api
* feat: add ImportIam handle
* refact bucket replication
* Update ecstore/src/disk/endpoint.rs
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
---------
Co-authored-by: weisd <weishidavip@163.com >
Co-authored-by: lygn128 <lygn128@163.com >
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
2025-07-01 12:34:24 +08:00
2b1241de96
fix: fmt
2025-07-01 10:17:20 +08:00
68090d1c35
fix
2025-06-30 22:48:09 +08:00
57c8c76e58
add notify
2025-06-30 22:48:09 +08:00
7bd8431006
cargo fmt
2025-06-27 18:53:51 +08:00
749537664f
fix：Apply suggestions from clippy 1.88
2025-06-27 18:16:29 +08:00
bd5d27ffbd
fix
2025-06-27 17:32:30 +08:00
ca30b78a1b
fix：shutdown
2025-06-27 12:37:11 +08:00
ce7cfedfcb
improve code for platform
2025-06-27 11:44:07 +08:00
26f84a9696
Feature/event ( #513 )
...
* improve code for notify
* fix
* cargo fmt
* improve code and create `DEFAULT_DELIMITER`
* fix
* fix
* improve code for notify
* fmt
* Update crates/notify/src/registry.rs
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* Update crates/notify/src/factory.rs
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* fix cllipy
* fix
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
2025-06-27 00:28:28 +08:00
831cb0b6d9
Feature/event: Optimize and Enhance Notify Crate Functionality ( #512 )
...
* improve code for notify
* fix
* cargo fmt
* improve code and create `DEFAULT_DELIMITER`
* fix
* fix
* improve code for notify
* fmt
* Update crates/notify/src/registry.rs
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* Update crates/notify/src/factory.rs
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* fix cllipy
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
2025-06-26 12:24:00 +08:00
caf8d80318
fix ( #504 )
2025-06-23 17:30:40 +08:00
34ec13e532
change objectinfo metadata use HashMap
...
add copyobject compresss support
fix listobject size use actual_size
2025-06-23 16:24:11 +08:00
de3773fbbc
Update crates/notify/src/rules/config.rs
...
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
2025-06-23 14:32:28 +08:00
3817d2d682
Update crates/notify/src/integration.rs
...
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
2025-06-23 14:32:16 +08:00
744bcfc4a9
Merge branch 'main' of github.com:rustfs/s3-rustfs into feature/observability-metrics
...
# Conflicts:
# crates/notify/src/event.rs
# crates/notify/src/global.rs
# crates/notify/src/integration.rs
# crates/notify/src/notifier.rs
2025-06-23 13:33:25 +08:00
7bb7f9e309
improve code notify
2025-06-23 12:47:58 +08:00
5a5712fde0
fix clippy
2025-06-23 10:28:57 +08:00
4559baaeeb
feat: migrate to reed-solomon-simd only implementation
...
- Remove reed-solomon-erasure dependency and all related code
- Simplify ReedSolomonEncoder from enum to struct with SIMD-only implementation
- Eliminate all conditional compilation (#[cfg(feature = ...)])
- Add instance caching with RwLock-based encoder/decoder reuse
- Implement reset mechanism to avoid unnecessary allocations
- Ensure thread safety with proper cache management
- Update documentation and benchmark scripts for SIMD-only approach
- Apply code formatting across all files
Breaking Changes:
- Removes support for reed-solomon-erasure feature flag
- API remains compatible but implementation is now SIMD-only
Performance Impact:
- Improved encoding/decoding performance through SIMD optimization
- Reduced memory allocations via instance caching
- Enhanced thread safety and concurrency support
2025-06-23 10:00:17 +08:00
5155a3d544
fix
2025-06-23 04:15:05 +08:00
928453db62
improve code for notify
2025-06-23 03:34:05 +08:00
c7af6587f5
trace log use local time and get custom user agent
2025-06-22 10:31:32 +08:00
e0f65e5e24
improve code for notify
2025-06-21 10:35:07 +08:00
d3cc36f6e0
fix
2025-06-20 10:51:36 +08:00
c658d88d25
Reconstructing Notify module
2025-06-19 15:40:48 +08:00
e6b019c29d
Merge branch 'main' of github.com:rustfs/s3-rustfs into feature/observability-metrics
...
# Conflicts:
# .github/workflows/build.yml
# .github/workflows/ci.yml
# Cargo.lock
# Cargo.toml
# appauth/src/token.rs
# crates/config/src/config.rs
# crates/event-notifier/examples/simple.rs
# crates/event-notifier/src/global.rs
# crates/event-notifier/src/lib.rs
# crates/event-notifier/src/notifier.rs
# crates/event-notifier/src/store.rs
# crates/filemeta/src/filemeta.rs
# crates/notify/examples/webhook.rs
# crates/utils/Cargo.toml
# ecstore/Cargo.toml
# ecstore/src/cmd/bucket_replication.rs
# ecstore/src/config/com.rs
# ecstore/src/disk/error.rs
# ecstore/src/disk/mod.rs
# ecstore/src/set_disk.rs
# ecstore/src/store_api.rs
# ecstore/src/store_list_objects.rs
# iam/Cargo.toml
# iam/src/manager.rs
# policy/Cargo.toml
# rustfs/src/admin/rpc.rs
# rustfs/src/main.rs
# rustfs/src/storage/mod.rs
2025-06-19 13:16:48 +08:00
09e8bc8f02
fix
2025-06-18 23:46:55 +08:00
9339093638
init event crate
2025-06-10 09:59:49 +08:00
28c71cc351
fix
2025-06-09 12:25:56 +08:00
df9347b34a
init config
2025-06-06 19:05:16 +08:00
2755d4125d
fix
2025-06-06 16:11:40 +08:00
4199bf2ba4
modify crates name
2025-06-06 11:14:36 +08:00
junxiang Mu