43 Commits

Author SHA1 Message Date
cxymds 1e14c05cf0 fix(tiering): support UTF-8 metadata signing (#4969)
Co-authored-by: Zhengchao An <anzhengchao@gmail.com>
2026-07-17 19:03:13 +08:00
houseme 56179210ab chore(deps): simplify dependency features (#4890)
* chore(deps): remove redundant dependency features

Remove manifest feature entries that are implied by other requested features in the same dependency declaration.

Verified that the resolved Cargo feature graph is unchanged after the cleanup.

Co-Authored-By: heihutu <heihutu@gmail.com>

* chore(deps): narrow tokio and reqwest features

Co-Authored-By: heihutu <heihutu@gmail.com>

---------

Co-authored-by: heihutu <heihutu@gmail.com>
2026-07-16 05:20:43 +00:00
houseme f3a7a4b0da chore(deps): localize workspace dependency features (#4888)
Move workspace-level dependency feature lists into the member crates that consume each dependency while keeping required default-features flags at the workspace root.

Also refresh starshard to 2.2.2 via cargo update and cargo upgrade --exclude ratelimit.

Co-authored-by: heihutu <heihutu@gmail.com>
2026-07-16 03:55:27 +00:00
Zhengchao An 7e2e553981 fix(signer): use real timestamp and standard base64 for SigV2 (#4456) 2026-07-08 17:02:01 +08:00
Zhengchao An 31c6859965 chore: converge stale TODOs and apply safe fills (backlog#646) (#4322)
Second TODO-convergence round over the current tree (backlog#646). All
line numbers in the old inventory had gone stale after the set_disk /
diagnostics / cluster refactors, so this re-scans and reduces the marker
count from 144 to 99.

STALE removals (comment describes already-implemented behavior, or dead
commented-out blocks) across ecstore (set_disk ops/core, store,
cluster/rpc, bucket/metadata_sys, services), iam, filemeta, s3select and
rustfs auth/object_usecase. No behavior change.

Safe fills, each verified:
- filemeta: replication_info_equals now also compares
  replication_state_internal (function currently has no callers; adds a
  regression test).
- bitrot: drop the confirmed-unused `_want` parameter from bitrot_verify
  and the now-unused `sum` on LocalDisk::bitrot_verify, removing a
  Bytes::copy_from_slice allocation. Streaming verify uses the file's
  embedded per-shard hash, never the passed sum.
- signer: rename v4_ignored_headers -> V4_IGNORED_HEADERS and drop the
  non_upper_case_globals allow.
- admin/heal: test_decode was #[ignore]d and used serde_urlencoded on a
  JSON body (would panic); rewire to serde_json::from_slice to match the
  production decode path, add assertions, un-ignore.

Verified: cargo fmt; cargo check on touched crates; tests pass
(filemeta, signer, bitrot, heal::test_decode); arch guardrail scripts
pass.
2026-07-06 22:43:32 +08:00
Zhengchao An a37e918936 docs: document SHA-1 HMAC migration proposal (#747) (#4009) 2026-06-28 19:42:18 +08:00
安正超 1dd3839a9f fix(signer): address post-merge review comments (#3216)
* fix: address PR #3150 review comments

- Restore get_host_addr as best-effort wrapper (String return type)
- Replace bare expect("err") with descriptive messages in unsigned trailer
- Simplify aws-chunked header construction
- 20 signer + 96 io-core tests pass

* fix(signer): preserve host fallback

* fix(signer): avoid signer fallback panics

---------

Co-authored-by: houseme <housemecn@gmail.com>
2026-06-04 22:45:50 +00:00
安正超 7b57d5b217 perf(signer): reduce String allocations in V4 signing path (#3197)
* perf(signer): reduce String allocations in V4 signing path

- v4_ignored_headers: HashMap<String, bool> -> HashSet<&'static str>
- Header lookups: k.to_string() -> k.as_str() (zero-copy)
- Query params: Vec<(String, String)> -> Vec<(&str, &str)> (zero-copy)
- Canonical request: single String with push_str instead of 6 intermediate Strings + join
- 20 signer tests pass

* fix(signer): align ignored header set type

---------

Co-authored-by: houseme <housemecn@gmail.com>
2026-06-03 15:40:35 +00:00
安正超 bd571a575f fix(io-core,signer): replace unwrap() with proper error handling (#3150)
* fix(io-core,signer): replace unwrap() with proper error handling

## io-core (issue #653 item 1)
- pool.rs: replace 8 .lock().unwrap() with poisoned recovery
- pool.rs: replace semaphore acquire unwrap with graceful fallback
- deadlock_detector.rs: replace 5 .lock().unwrap() with match/ok

## signer (issue #653 item 1)
- Add SignV2Error enum, try_pre_sign_v2, try_sign_v2
- Replace 14 unwrap() in v2 signing with ? propagation
- Add try_streaming_sign_v4, replace 5 expect("err") with descriptive errors
- get_host_addr returns Result instead of panicking

All backward-compat wrappers preserved. 95 io-core + 18 signer tests pass.

* fix: address signer and pool review comments

* test: update signer v2 string-to-sign test

* fix: address signer and pool review followups

* fix: tighten signer host and pool fallback
2026-05-31 23:40:20 +00:00
安正超 d79720da1d test(signer): cover header fallback helpers (#2711) 2026-04-28 14:09:56 +00:00
houseme a68fe1601f fix(ilm): harden signer failures and guard remote tier delete storms (#2706) 2026-04-27 12:39:02 +00:00
Tunglies 49366ee200 chore(lint): clippy rules redundant_clone (#2554) 2026-04-15 13:54:07 +00:00
heihutu 3c28f0a0ba feat(metrics): migrate system monitoring from rustfs-obs to rustfs-metrics (#2242)
Co-authored-by: houseme <housemecn@gmail.com>
2026-03-20 18:52:33 +08:00
安正超 c47dec8549 fix(signer): avoid panics in v2 signing for missing data (#2121) 2026-03-11 21:58:40 +08:00
houseme 8d7cd4cb1b chore: upgrade dependencies and migrate to aws-lc-rs (#1333) 2026-01-02 00:02:34 +08:00
yihong 834025d9e3 docs: fix some dead link (#1053)
Signed-off-by: yihong0618 <zouzou0208@gmail.com>
2025-12-08 11:23:24 +08:00
weisd 85bc0ce2d5 fix: filemeta version handling and delete operations (#879)
* fix filemeta version

* fix clippy

* fix delete version

* fix clippy/test
2025-11-18 09:24:22 +08:00
likewu 2aca1f77af Fix/ilm (#721)
* fix tip remote tier error
* fix transitioned_object
* fix filemeta
* add GCS R2
* add aliyun tencent huaweicloud azure gcs r2 backend tier
* fix signer
* change azure to s3
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: loverustfs <155562731+loverustfs@users.noreply.github.com>
2025-10-27 20:23:50 +08:00
weisd cd1e244c68 Refactor: Introduce content checksums and improve multipart/object metadata handling (#671)
* feat:  adapt to s3s typed etag support

* refactor: move replication struct to rustfs_filemeta, fix filemeta transition bug

* add head_object checksum, filter object metadata output

* fix multipart checksum

* fix multipart checksum

* add content md5,sha256 check

* fix test

* fix cargo

---------

Co-authored-by: overtrue <anzhengchao@gmail.com>
2025-10-20 23:46:13 +08:00
houseme fd2aab2bd9 fix:revet #443 #446 (#452)
* fix: revet #443 #446

* fix
2025-08-23 17:30:06 +08:00
loverustfs 46bd75c0f8 ahm(scanner): throttle scanning, skip recently-modified objects, and … (#443)
* ahm(scanner): throttle scanning, skip recently-modified objects, and gate missing-object heals to deep mode; adjust conservative defaults

Signed-off-by: loverustfs <hello@rustfs.com>

* ecstore: enable virtual-host AUTO heuristics and URL building; signer: fix SigV2 canonical resource for vhost; add unit tests

* ecstore: AUTO virtual-host style URL selection; signer: SigV2 canonical resource fixes for vhost; tests added.\nahm: fix clippy drop_non_drop; integration tests robust to existing bucket; ignore flaky lifecycle test.\nMakefile: test target falls back to cargo test when nextest missing.\npre-commit: all checks green.

---------

Signed-off-by: loverustfs <hello@rustfs.com>
2025-08-22 16:03:29 +08:00
houseme e0c99bced4 chore: add tls log and removing unused crates (#359)
* chore: add tls log

* improve code for http

* improve code dependencies for `cargo.toml` and removing unused crates

* modify name

* improve code

* fix

* Update crates/config/src/constants/env.rs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* improve code

* fix

* add `is_enabled` and `is_disabled`

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-08-07 19:02:09 +08:00
Nugine 2525b66658 refactor: replace lazy_static with LazyLock (#164)
* refactor: replace `lazy_static` with `LazyLock`

* update cursorrules

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
2025-07-10 23:50:46 +08:00
houseme 36acb5bce9 feat(console): Enhance network address handling for WebUI (#129)
* add crates homepage,description,keywords,categories,documentation

* add readme

* modify version 0.0.3

* cargo fmt

* fix: yaml.docker-compose.security.no-new-privileges.no-new-privileges-docker-compose.yml (#63)

* Feature up/ilm (#61)

* fix delete-marker expiration. add api_restore.

* remove target return 204

* log level

* fix: make lint build and clippy happy (#71)

Signed-off-by: yihong0618 <zouzou0208@gmail.com>

* fix: make ci and local use the same toolchain (#72)

Signed-off-by: yihong0618 <zouzou0208@gmail.com>

* feat: optimize GitHub Actions workflows with performance improvements (#77)

* feat: optimize GitHub Actions workflows with performance improvements

- Rename workflows with more descriptive names
- Add unified setup action for consistent environment setup
- Optimize caching strategy with Swatinem/rust-cache@v2
- Implement skip-check mechanism to avoid duplicate builds
- Simplify matrix builds with better include/exclude logic
- Add intelligent build strategy checks
- Optimize Docker multi-arch builds
- Improve artifact naming and retention
- Add performance testing with benchmark support
- Enhance security audit with dependency scanning
- Change Chinese comments to English for better maintainability

Performance improvements:
- CI testing: ~35 min (42% faster)
- Build release: ~60 min (50% faster)
- Docker builds: ~45 min (50% faster)
- Security audit: ~8 min (47% faster)

* fix: correct secrets context usage in GitHub Actions workflow

- Move environment variables to job level to fix secrets access issue
- Fix unrecognized named-value 'secrets' error in if condition
- Ensure OSS upload step can properly check for required secrets

* fix: resolve GitHub API rate limit by adding authentication token

- Add github-token input to setup action to authenticate GitHub API requests
- Pass GITHUB_TOKEN to all setup action usages to avoid rate limiting
- Fix arduino/setup-protoc@v3 API access issues in CI/CD workflows
- Ensure protoc installation can successfully access GitHub releases API

* fix:make bucket err (#85)

* Rename DEVELOPMENT.md to CONTRIBUTING.md

* Create issue-translator.yml (#89)

Enable Issues Translator

* fix(dockerfile): correct env variable names for access/secret key and improve compatibility (#90)

* fix: restore Zig and cargo-zigbuild caching in GitHub Actions setup action (#92)

* fix: restore Zig and cargo-zigbuild caching in GitHub Actions setup action

Use mlugg/setup-zig and taiki-e/cache-cargo-install-action to speed up cross-compilation tool installation and avoid repeated downloads. All comments and code are in English.

* fix: use correct taiki-e/install-action for cargo-zigbuild

Use taiki-e/install-action@cargo-zigbuild instead of taiki-e/cache-cargo-install-action@v2 to match the original implementation from PR #77.

* refactor: remove explicit Zig version to use latest stable

* Create CODE_OF_CONDUCT.md

* Create SECURITY.md

* Update issue templates

* Create CLA.md

* docs: update PR template to English version

* fix: improve data scanner random sleep calculation

- Fix random number generation API usage
- Adjust sleep calculation to follow MinIO pattern
- Ensure proper random range for scanner cycles

Signed-off-by: junxiang Mu <1948535941@qq.com>

* fix: soupprt ipv6

* improve log

* add client ip log

* Update rustfs/src/console.rs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* improve code

* feat: unify package format to zip for all platforms

---------

Signed-off-by: yihong0618 <zouzou0208@gmail.com>
Signed-off-by: junxiang Mu <1948535941@qq.com>
Co-authored-by: kira-offgrid <kira@offgridsec.com>
Co-authored-by: likewu <likewu@126.com>
Co-authored-by: laoliu <lygn128@163.com>
Co-authored-by: yihong <zouzou0208@gmail.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: weisd <im@weisd.in>
Co-authored-by: Yone <zhiyu@live.cn>
Co-authored-by: loverustfs <155562731+loverustfs@users.noreply.github.com>
Co-authored-by: junxiang Mu <1948535941@qq.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-07-09 14:39:40 +08:00
安正超 3dbcaaa221 docs: simplify crates README files and enforce PR-only workflow (#110)
* docs: simplify all crates README files

- Remove extensive code examples and detailed documentation
- Convert to minimal module introductions with core feature lists
- Direct users to main RustFS repository for comprehensive docs
- Updated 20 crate README files for consistency and brevity

Files updated:
- crates/rio/README.md (415→15 lines)
- crates/s3select-api/README.md (592→15 lines)
- crates/s3select-query/README.md (658→15 lines)
- crates/signer/README.md (407→15 lines)
- crates/utils/README.md (395→15 lines)
- crates/workers/README.md (463→15 lines)
- crates/zip/README.md (408→15 lines)

* docs: restore original headers in crates README files

- Add back RustFS logo image and CI badges
- Restore formatted headers and structured layout
- Keep simplified content with module introductions
- Maintain consistent documentation structure across all crates

All 20 crate README files now have proper headers while keeping
the simplified content that directs users to the main repository.

* rules: enforce PR-only workflow for main branch

- Strengthen rule that ALL changes must go through pull requests
- Explicitly forbid direct commits to main branch under any circumstances
- Add comprehensive PR requirements and enforcement guidelines
- Clarify that PRs are the ONLY way to merge to main branch
- Add requirement for PR approval before merging
- Include enforcement mechanisms for branch protection
2025-07-08 21:10:07 +08:00
yihong 72aead5466 fix: make ci and local use the same toolchain (#72)
Signed-off-by: yihong0618 <zouzou0208@gmail.com>
2025-07-07 10:40:53 +08:00
yihong abd5dff9b5 fix: make lint build and clippy happy (#71)
Signed-off-by: yihong0618 <zouzou0208@gmail.com>
2025-07-07 09:55:53 +08:00
安正超 fa17f7b1e3 feat: add comprehensive README documentation for all RustFS submodules (#48) 2025-07-04 23:02:13 +08:00
likewu 79156d2d82 fix 2025-07-04 21:57:51 +08:00
likewu 26542b741e request::Builder -> request::Request<Body> 2025-07-04 16:59:15 +08:00
houseme 5826396cd0 refactor: Restructure project layout and clean up dependencies (#30)
This commit introduces a significant reorganization of the project structure to improve maintainability and clarity.

Key changes include:
- Adjusted the directory layout for a more logical module organization.
- Removed unused crate dependencies, reducing the overall project size and potentially speeding up build times.
- Updated import paths and configuration files to reflect the structural changes.
2025-07-02 19:33:12 +08:00
houseme 2e14b32ccd chore: Add copyright and license headers (#23)
* chore: Add copyright and license headers

This commit adds the Apache 2.0 license and a copyright notice to the header of all source files. This ensures that the licensing and copyright information is clearly stated within the codebase.

* cargo fmt

* fix

* fmt

* fix clippy
2025-07-02 15:07:47 +08:00
houseme cb19bbef22 fix:Apply suggestions from clippy 1.88 (#20) 2025-07-02 12:14:22 +08:00
weisd 3fd70de4a6 fix: clippy 2025-07-01 09:41:55 +08:00
likewu 37861d9af6 fix: clippy 2025-07-01 09:40:29 +08:00
weisd c63c6187c5 fix: clippy 2025-06-30 22:38:29 +08:00
weisd 24738dc635 fix: resolve clippy warnings in signer module
- Add #[allow(clippy::too_many_arguments)] for functions with 8 parameters
- Replace vec initialization with array where appropriate
- Fix test code warnings by adding #[allow] attributes
- Improve code quality while maintaining functionality
2025-06-30 21:41:34 +08:00
weisd bfeb746c5f style: fix code formatting issues
- Fix spacing and line formatting in signer modules
- Remove unnecessary blank lines
- Ensure consistent code style per project standards
2025-06-30 21:36:26 +08:00
likewu 111f994a9d fix 2025-06-30 12:24:32 +08:00
likewu b69a4ed691 fix 2025-06-28 21:43:26 +08:00
likewu a770b17e0c fix clippy 2025-06-28 14:34:27 +08:00
likewu f38108d20d fix format 2025-06-28 14:34:21 +08:00
likewu 4ed84a6bc4 separate signer.
fix ilm feature.
2025-06-28 14:33:22 +08:00