Commit Graph

580 Commits

Author SHA1 Message Date
Zhengchao An aa089f18f2 refactor: centralize ecstore accessor runtime sources (#3813) 2026-06-24 11:05:30 +08:00
Zhengchao An 1a8ff1ea3e refactor: centralize ecstore setup runtime sources (#3812) 2026-06-24 10:41:33 +08:00
Zhengchao An 6b1b3800d7 refactor: centralize ecstore owner runtime sources (#3811) 2026-06-24 10:00:36 +08:00
Zhengchao An 3f624bf06c refactor: centralize ecstore bucket runtime sources (#3810) 2026-06-24 09:41:11 +08:00
Zhengchao An 5b6c486b13 refactor: centralize ecstore locality runtime sources (#3808) 2026-06-24 08:42:36 +08:00
Zhengchao An 1c04b86494 refactor: centralize ecstore bucket runtime sources (#3806) 2026-06-24 08:00:04 +08:00
cxymds c14a442586 fix(storage): harden rebalance and decommission state (#3730) 2026-06-24 07:59:39 +08:00
Zhengchao An 5046f788be refactor: centralize ecstore lifecycle runtime sources (#3805) 2026-06-24 07:23:13 +08:00
Zhengchao An 7e60432588 refactor: centralize ecstore runtime owner sources (#3798) 2026-06-24 06:34:06 +08:00
Zhengchao An 1735dcde9c refactor: centralize ecstore data-plane runtime sources (#3797) 2026-06-23 23:23:16 +08:00
Zhengchao An e59e1852ec refactor: centralize network client runtime sources (#3796) 2026-06-23 22:35:51 +08:00
houseme 0a00d8d500 perf(server): lighten internode data-plane stack (#3735)
* refactor(server): split internode dispatch scaffold

* test(server): cover internode dispatch prefix split

* refactor(server): name internode stack boundaries

* perf(server): skip internode request logging layer

* perf(server): skip internode trace layer

* perf(server): use lite internode request context

* feat(metrics): track internode rpc duration

* feat(ecstore): add put object stage summary logs

* test(metrics): update internode descriptor expectations

* fix(server): tighten internode path matching

* fix(pr): address review follow-up comments

* style(ecstore): simplify commit tail duration field

* refactor(ecstore): group put stage summary fields

* refactor(ecstore): inline put stage summary log

* fix(s3): return storage class for object attributes

* merge: sync latest main and resolve object attributes conflict

* fmt

* fix(server): remove duplicate rpc imports

* build(deps): bump memmap2 for RUSTSEC-2026-0186

* fix(s3select): align object_store with datafusion

* chore(deps): prune workspace dependencies

* perf(fuzz): optimize CI runtime with build/run split and matrix parallelization

  Separate fuzz harness compilation from execution to eliminate redundant
  builds across targets. Introduce matrix-based parallel execution for
  PR smoke and nightly fuzz jobs.

  Changes:

  - Split CI workflow into `fuzz-build` (compile once) and matrix run jobs
    (`pr-fuzz-smoke`, `nightly-fuzz-corpus`) that execute targets in parallel
  - Add `BUILD_ONLY` mode to run_ci_targets.sh / run_nightly_targets.sh
  - Add run_single_target.sh for matrix jobs (no build phase)
  - Optimize `local_metadata` fuzz target: reduce prefix iterations from
    8-10 (4 functions each) to 5 critical prefixes (parser-only), cutting
    per-iteration cost by ~3-5x
  - Move archive path validation (`validate_extract_relative_path`,
    `normalize_extract_entry_key`) from `rustfs` to `rustfs-utils::path`,
    eliminating `rustfs` binary crate dependency from fuzz harness
  - Remove `rustfs` from fuzz/Cargo.toml (drops significant transitive deps)
  - Add unit tests for archive path validation in rustfs-utils
  - Update fuzz/README.md with new workflow and script documentation

  Expected CI improvement: PR smoke wall-clock from ~120min (frequent
  timeout) to ~40min; nightly from ~180min to ~60min.

* refactor(fuzz): consolidate scripts and fix prefix test alignment

Replace three duplicated shell scripts (run_ci_targets.sh,
run_nightly_targets.sh, run_single_target.sh) with a single
parameterized run.sh that supports BUILD_ONLY, SKIP_BUILD, and
MAX_TOTAL_TIME environment variables.

Fix local_metadata fuzz target prefix testing: replace always-true
'len > 0' guard with lengths aligned to xl.meta binary layout
(4/5/8/12 bytes for magic+version+header fields). Remove redundant
empty-slice test.

Hoist RUSTFLAGS to workflow top-level env to eliminate per-job
duplication. Update README with unified script documentation.

Net: -118 lines, zero functionality loss.

* perf(fuzz): optimize CI runtime with build/run split and matrix parallelization

Restructure fuzz CI workflow to eliminate redundant compilation and
run targets in parallel via matrix strategy.

Workflow changes:
- Split into fuzz-build (compile once) and matrix run jobs
- PR smoke: 3 targets parallel, 60s each, timeout 30min (was 120min)
- Nightly: 3 targets parallel, 300s each, timeout 60min (was 180min)
- Pass compiled harness via actions/artifact between jobs
- Hoist RUSTFLAGS to workflow top-level env

Script consolidation:
- Replace 3 duplicated scripts with single parameterized run.sh
- Supports BUILD_ONLY, SKIP_BUILD, MAX_TOTAL_TIME env vars

Target optimizations:
- Remove rustfs binary crate from fuzz dependencies (was pulling
  979 transitive deps); move archive path validation to rustfs-utils
- Optimize local_metadata: reduce prefix iterations from 8-10x4
  calls to 5 prefixes with parser-only (no decompress), aligned
  with xl.meta binary layout (4/5/8/12 bytes)
- Add unit tests for archive path validation in rustfs-utils
- Update fuzz/README.md with unified script documentation

Expected: PR smoke wall-clock from ~120min (frequent timeout)
to ~40min; nightly from ~180min to ~60min.

* fix(rpc): resolve internode metrics via app context

* fmt

* ci: speed up fuzz smoke artifact restore

---------

Signed-off-by: houseme <housemecn@gmail.com>
2026-06-23 21:36:39 +08:00
Henry Guo 7bdb25ae9d feat(scanner): define replication boundary contract (#3630)
* feat(scanner): define replication boundary contract

* fix(scanner): propagate replication boundary metrics

* fix(scanner): preserve repair metadata on merge

---------

Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-06-23 21:35:36 +08:00
唐小鸭 eff656e086 fix(storage): restore legacy SSE-S3 read compatibility (#3584)
* Update .gitignore

* Fix. fixed SSE-S3 compatibility issues in large-scale testing

* fix

* fix(ecstore): reject whitespace bucket names

* Update replication_extension_test.rs

* style(ecstore): format bucket whitespace test

---------

Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: cxymds <cxymds@gmail.com>
2026-06-23 21:35:17 +08:00
houseme 46775ae019 feat: harden runtime capability snapshots (#3784)
* feat(admin): expose runtime capability snapshots

* feat(runtime): refine workload admission snapshots

* test(ci): align architecture migration checks

* build(deps): bump memmap2 for RUSTSEC-2026-0186

* build(deps): refresh cargo deny lockfile
2026-06-23 18:10:49 +08:00
houseme 583a23bdf2 fix(ecstore): replace panic-driven pool and set stubs (#3753)
* fix(ecstore): replace panic-driven pool and set stubs

* test(runtime): tolerate restricted local bind checks

* fix(ecstore): remove remaining trait stub placeholders

* fix(ecstore): tighten trait stub follow-up semantics

* chore: ignore local worktrees

* chore: update layer dependency baseline for resolve_* context entries

Add 7 accepted infra->app dependency entries introduced by recent
refactoring PRs (#3770, #3771, #3772) that route global state lookups
through app::context::resolve_* functions.

Co-Authored-By: heihutu <heihutu@gmail.com>

---------

Co-authored-by: heihutu <heihutu@gmail.com>
2026-06-23 12:31:17 +08:00
houseme a53ccb2407 fix(ecstore): reject symlink path escapes (#3742)
Co-authored-by: Zhengchao An <anzhengchao@gmail.com>
2026-06-22 19:28:40 +08:00
houseme 2f85ef4455 fix(security): extend outbound egress guard (#3744) 2026-06-22 19:20:31 +08:00
houseme 64aef06c60 fix(replication): harden site-repl versioning sync (#3739) 2026-06-22 19:18:42 +08:00
cxymds 2f25cf606e fix(storage): harden rebalance decommission state (#3515) 2026-06-22 12:03:13 +08:00
安正超 c13f42e9a0 refactor: wrap disk rpc compat trait methods (#3707) 2026-06-22 06:31:24 +08:00
安正超 6b3d96fde3 refactor: prune trait import compat re-exports (#3699) 2026-06-21 23:17:58 +08:00
安正超 a0e7957e92 refactor: prune admin app compat re-exports (#3697) 2026-06-21 20:28:43 +08:00
安正超 48ed331fc8 refactor: prune root runtime compat re-exports (#3694) 2026-06-21 18:42:25 +08:00
安正超 81f11df4dd refactor: add cluster status snapshots (#3682) 2026-06-21 11:25:44 +08:00
安正超 15cdeee775 refactor: add ecstore cluster read models (#3681) 2026-06-21 10:45:36 +08:00
安正超 d26d98104a refactor: prune ecstore root global facades (#3680) 2026-06-21 09:52:19 +08:00
安正超 d4150117c3 refactor: narrow startup and ecstore root facades (#3679)
* refactor: narrow startup owner visibility

* refactor: prune remaining ecstore root facades
2026-06-21 09:42:15 +08:00
安正超 0985225448 refactor: consolidate ecstore public facade (#3678)
* refactor: expand ecstore compatibility facade

* test: enforce ecstore api facade imports

* refactor: hide legacy ecstore layout modules

* refactor: hide ecstore facade root modules
2026-06-21 09:09:11 +08:00
安正超 796f293b80 test(ecstore): cover DNS resolver raw errors (#3674) 2026-06-21 08:17:45 +08:00
安正超 9b3fda9e30 refactor: route ecstore public surfaces through api (#3673) 2026-06-21 05:23:23 +08:00
GatewayJ 3e7e39a59c fix(ecstore): retry transient endpoint DNS failures (#3652)
* fix(ecstore): retry transient endpoint DNS failures

* fix(ecstore): satisfy DNS retry clippy check

* fix(ecstore): harden endpoint DNS retry

* test(ecstore): stabilize DNS retry error test
2026-06-20 23:53:11 +08:00
Hiroaki KAWAI deee13901d fix(logging): suppress scanner leader eviction noise (#3653) 2026-06-20 22:23:42 +08:00
安正超 0cf7e5cf03 refactor: extract embedded startup control helpers (#3661) 2026-06-20 21:57:03 +08:00
安正超 d933d5e938 refactor: move ecstore rebalance flow modules (#3658) 2026-06-20 17:56:11 +08:00
安正超 1595b2bc3f refactor: move ecstore rebalance migration helpers (#3657) 2026-06-20 17:15:30 +08:00
安正超 6d82b6f855 refactor: move ecstore rebalance worker helpers (#3656) 2026-06-20 16:44:48 +08:00
安正超 f5f727e077 refactor: move ecstore rebalance metadata helpers (#3655) 2026-06-20 16:31:58 +08:00
安正超 f5bd31173d refactor: move ecstore pool-space builder (#3651) 2026-06-20 15:38:49 +08:00
安正超 1d9fd3d7ac refactor: move ecstore rebalance support helpers (#3650) 2026-06-20 13:34:23 +08:00
安正超 aad5379e0c refactor: move ecstore pool-space helpers (#3649) 2026-06-20 12:44:16 +08:00
安正超 3035cad082 refactor: move ecstore set heal helpers (#3648) 2026-06-20 12:03:22 +08:00
安正超 1ea7b36ab4 refactor: move ecstore endpoint layout owners (#3647) 2026-06-20 10:28:25 +08:00
安正超 16d667da0a refactor: move ecstore format layout owners (#3646) 2026-06-20 09:58:50 +08:00
安正超 88dfdda4a0 refactor: establish ecstore layout foundation (#3645) 2026-06-20 09:54:36 +08:00
安正超 54ffbedbc8 refactor: remove ecstore operation compat facades (#3608) 2026-06-19 17:01:12 +08:00
houseme 8cf3c0bfbd fix(ecstore): support MinIO DARE fixture compatibility (#3590) 2026-06-19 10:13:39 +08:00
安正超 b1c6578df1 refactor: narrow test harness compatibility surfaces (#3592) 2026-06-19 07:10:52 +08:00
houseme 9b1272529a fix(ecstore): avoid blocking fs in async paths (#3586) 2026-06-19 01:44:17 +08:00
安正超 c28fee0013 refactor: continue storage api contract cleanup (#3580)
* refactor: move delete object contracts to storage api

* refactor: narrow store api compatibility exports

* refactor: route table catalog test through storage compat
2026-06-18 22:42:02 +08:00