3f13d098b4
feat(observability): feature-gated hotpath instrumentation for the data path ( #4394 )
...
Merge the hotpath-rs wall-time instrumentation from the backlog#936 analysis worktree behind an opt-in 'hotpath' cargo feature, keeping the default build at zero overhead and zero dependency.
- hotpath is an optional dependency everywhere (dep:hotpath feature syntax); the default dependency tree contains no hotpath crate at all
- 40+ measurement points across S3 handlers, ECStore/SetDisks object and multipart ops, erasure encode/decode, bitrot, LocalDisk I/O, FileMeta codec, and HashReader
- attribute sites use #[cfg_attr(feature = "hotpath", hotpath::measure)]; async_trait bodies use per-crate hp_guard! macros (ecstore + rustfs bin); rio gates measure_block! behind hp_measure_block!
- feature chain: rustfs -> rustfs-ecstore -> rustfs-rio / rustfs-filemeta, each crate owning its own gate
- hotpath-alloc is intentionally not wired up (hotpath 0.21.x TLS panic on cross-thread guard drop under tokio, see backlog#935); mimalloc stays the unconditional global allocator
- docs/development/hotpath-profiling.md documents building, HOTPATH_* env vars, SIGTERM report flow, and how to reproduce the backlog#936 timing reports
Refs: https://github.com/rustfs/backlog/issues/935 (HP-14, item 2), https://github.com/rustfs/backlog/issues/936
Co-authored-by: heihutu <heihutu@gmail.com >
2026-07-08 07:39:47 +08:00
cc84d9065e
test(property): add filemeta invariants ( #3536 )
...
* test(property): add filemeta invariants for #3523
* docs: relocate issue-3518 property testing plan
* docs: keep issue-3518 property plan local only
2026-06-18 00:36:55 +08:00
c29c8a5a1e
fix(filemeta): harden and optimize metacache path ( #2724 )
...
Co-authored-by: houseme <housemecn@gmail.com >
2026-04-30 07:47:31 +00:00
d2901fd78c
feat(admin): add audit target APIs and harden target source handling ( #2350 )
...
Signed-off-by: houseme <housemecn@gmail.com >
Co-authored-by: 安正超 <anzhengchao@gmail.com >
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com >
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com >
2026-04-04 09:07:22 +08:00
3c28f0a0ba
feat(metrics): migrate system monitoring from rustfs-obs to rustfs-metrics ( #2242 )
...
Co-authored-by: houseme <housemecn@gmail.com >
2026-03-20 18:52:33 +08:00
eb33e82b56
fix: Prevent panic in GetMetrics gRPC handler on invalid input ( #1291 )
...
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com >
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com >
2025-12-29 03:10:23 +08:00
601f3456bc
Fix large file upload freeze with adaptive buffer sizing ( #869 )
...
* Initial plan
* Fix large file upload freeze by increasing StreamReader buffer size
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com >
* Add comprehensive documentation for large file upload freeze fix
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com >
* upgrade s3s version
* Fix compilation error: use BufReader instead of non-existent StreamReader::with_capacity
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com >
* Update documentation with correct BufReader implementation
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com >
* add tokio feature `io-util`
* Implement adaptive buffer sizing based on file size
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com >
* Constants are managed uniformly and fmt code
* fix
* Fix: Trigger self-heal on read when shards missing from rejoined nodes (#871 )
* Initial plan
* Fix: Trigger self-heal when missing shards detected during read
- Added proactive heal detection in get_object_with_fileinfo
- When reading an object, now checks if any shards are missing even if read succeeds
- Sends low-priority heal request to reconstruct missing shards on rejoined nodes
- This fixes the issue where data written during node outage is not healed when node rejoins
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com >
* fix
* Unify CRC implementations to crc-fast (#873 )
* Initial plan
* Replace CRC libraries with unified crc-fast implementation
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com >
* fix
* fix: replace low to Normal
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com >
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com >
Co-authored-by: houseme <housemecn@gmail.com >
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com >
Co-authored-by: houseme <housemecn@gmail.com >
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com >
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com >
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com >
Co-authored-by: houseme <housemecn@gmail.com >
2025-11-17 23:15:20 +08:00
cd1e244c68
Refactor: Introduce content checksums and improve multipart/object metadata handling ( #671 )
...
* feat: adapt to s3s typed etag support
* refactor: move replication struct to rustfs_filemeta, fix filemeta transition bug
* add head_object checksum, filter object metadata output
* fix multipart checksum
* fix multipart checksum
* add content md5,sha256 check
* fix test
* fix cargo
---------
Co-authored-by: overtrue <anzhengchao@gmail.com >
2025-10-20 23:46:13 +08:00
639bf0c233
Revert "feat(append): implement object append operations with state tracking ( #599 )" ( #646 )
...
This reverts commit 4f73760a45
2025-10-12 23:47:51 +08:00
4f73760a45
feat(append): implement object append operations with state tracking ( #599 )
...
* feat(append): implement object append operations with state tracking
Signed-off-by: junxiang Mu <1948535941@qq.com >
* chore: rebase
Signed-off-by: junxiang Mu <1948535941@qq.com >
---------
Signed-off-by: junxiang Mu <1948535941@qq.com >
2025-09-27 20:06:26 -07:00
90f21a9102
refactor: Reimplement bucket replication system with enhanced architecture ( #590 )
...
* feat:refactor replication
* use aws sdk for replication client
* refactor/replication
* merge main
* fix lifecycle test
2025-09-26 14:27:53 +08:00
36acb5bce9
feat(console): Enhance network address handling for WebUI ( #129 )
...
* add crates homepage,description,keywords,categories,documentation
* add readme
* modify version 0.0.3
* cargo fmt
* fix: yaml.docker-compose.security.no-new-privileges.no-new-privileges-docker-compose.yml (#63 )
* Feature up/ilm (#61 )
* fix delete-marker expiration. add api_restore.
* remove target return 204
* log level
* fix: make lint build and clippy happy (#71 )
Signed-off-by: yihong0618 <zouzou0208@gmail.com >
* fix: make ci and local use the same toolchain (#72 )
Signed-off-by: yihong0618 <zouzou0208@gmail.com >
* feat: optimize GitHub Actions workflows with performance improvements (#77 )
* feat: optimize GitHub Actions workflows with performance improvements
- Rename workflows with more descriptive names
- Add unified setup action for consistent environment setup
- Optimize caching strategy with Swatinem/rust-cache@v2
- Implement skip-check mechanism to avoid duplicate builds
- Simplify matrix builds with better include/exclude logic
- Add intelligent build strategy checks
- Optimize Docker multi-arch builds
- Improve artifact naming and retention
- Add performance testing with benchmark support
- Enhance security audit with dependency scanning
- Change Chinese comments to English for better maintainability
Performance improvements:
- CI testing: ~35 min (42% faster)
- Build release: ~60 min (50% faster)
- Docker builds: ~45 min (50% faster)
- Security audit: ~8 min (47% faster)
* fix: correct secrets context usage in GitHub Actions workflow
- Move environment variables to job level to fix secrets access issue
- Fix unrecognized named-value 'secrets' error in if condition
- Ensure OSS upload step can properly check for required secrets
* fix: resolve GitHub API rate limit by adding authentication token
- Add github-token input to setup action to authenticate GitHub API requests
- Pass GITHUB_TOKEN to all setup action usages to avoid rate limiting
- Fix arduino/setup-protoc@v3 API access issues in CI/CD workflows
- Ensure protoc installation can successfully access GitHub releases API
* fix:make bucket err (#85 )
* Rename DEVELOPMENT.md to CONTRIBUTING.md
* Create issue-translator.yml (#89 )
Enable Issues Translator
* fix(dockerfile): correct env variable names for access/secret key and improve compatibility (#90 )
* fix: restore Zig and cargo-zigbuild caching in GitHub Actions setup action (#92 )
* fix: restore Zig and cargo-zigbuild caching in GitHub Actions setup action
Use mlugg/setup-zig and taiki-e/cache-cargo-install-action to speed up cross-compilation tool installation and avoid repeated downloads. All comments and code are in English.
* fix: use correct taiki-e/install-action for cargo-zigbuild
Use taiki-e/install-action@cargo-zigbuild instead of taiki-e/cache-cargo-install-action@v2 to match the original implementation from PR #77 .
* refactor: remove explicit Zig version to use latest stable
* Create CODE_OF_CONDUCT.md
* Create SECURITY.md
* Update issue templates
* Create CLA.md
* docs: update PR template to English version
* fix: improve data scanner random sleep calculation
- Fix random number generation API usage
- Adjust sleep calculation to follow MinIO pattern
- Ensure proper random range for scanner cycles
Signed-off-by: junxiang Mu <1948535941@qq.com >
* fix: soupprt ipv6
* improve log
* add client ip log
* Update rustfs/src/console.rs
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* improve code
* feat: unify package format to zip for all platforms
---------
Signed-off-by: yihong0618 <zouzou0208@gmail.com >
Signed-off-by: junxiang Mu <1948535941@qq.com >
Co-authored-by: kira-offgrid <kira@offgridsec.com >
Co-authored-by: likewu <likewu@126.com >
Co-authored-by: laoliu <lygn128@163.com >
Co-authored-by: yihong <zouzou0208@gmail.com >
Co-authored-by: 安正超 <anzhengchao@gmail.com >
Co-authored-by: weisd <im@weisd.in >
Co-authored-by: Yone <zhiyu@live.cn >
Co-authored-by: loverustfs <155562731+loverustfs@users.noreply.github.com >
Co-authored-by: junxiang Mu <1948535941@qq.com >
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
2025-07-09 14:39:40 +08:00
5826396cd0
refactor: Restructure project layout and clean up dependencies ( #30 )
...
This commit introduces a significant reorganization of the project structure to improve maintainability and clarity.
Key changes include:
- Adjusted the directory layout for a more logical module organization.
- Removed unused crate dependencies, reducing the overall project size and potentially speeding up build times.
- Updated import paths and configuration files to reflect the structural changes.
2025-07-02 19:33:12 +08:00
2e14b32ccd
chore: Add copyright and license headers ( #23 )
...
* chore: Add copyright and license headers
This commit adds the Apache 2.0 license and a copyright notice to the header of all source files. This ensures that the licensing and copyright information is clearly stated within the codebase.
* cargo fmt
* fix
* fmt
* fix clippy
2025-07-02 15:07:47 +08:00
81f0c9763f
Merge branch 'main' of https://github.com/rustfs/s3-rustfs into feature/ilm
...
# Conflicts:
# Cargo.lock
# Cargo.toml
# crates/utils/Cargo.toml
# crates/utils/src/net.rs
# ecstore/Cargo.toml
# ecstore/src/set_disk.rs
# rustfs/src/storage/ecfs.rs
2025-06-23 16:42:18 +08:00
cc71f40a6d
ilm feature add
2025-06-22 23:04:40 +08:00
3a567768c1
refactor(filemeta): ChecksumInfo hash use Bytes
2025-06-15 21:01:38 +08:00
7fe0cc74d2
add rio/filemeta
2025-06-04 14:21:34 +08:00
houseme