gsadmin/rustfs
1
0
Fork 0
mirror of https://github.com/rustfs/rustfs.git synced 2026-09-01 17:58:22 +00:00
Code Issues Packages Projects Releases Wiki Activity
6,172 Commits 48 Branches 130 Tags
ec0a65703a7be514fcceaf49d496b262a457a45f
Commit Graph

3136 Commits

Author SHA1 Message Date
cxymds ec0a65703a fix(s3): align snowball member semantics (#6944) 
* fix(s3): harden Snowball extract error boundaries

* fix(s3): close Snowball extract compatibility gaps

* fix(s3): verify Snowball request body completion

* test(s3): reject forged Snowball streaming signatures

* build(deps): pin Snowball archive parser limits

* fix(s3): preserve Snowball trailer and member errors

* docs(architecture): register Snowball tar fork cleanup

* refactor(s3): route Snowball errors through object boundary

* ci(deps): allow pinned tokio-tar source

* fix: align Snowball archive codec detection

* fix(s3): harden Snowball codec compatibility

* fix(s3): preserve Snowball codec compatibility

* test(zip): align yield wake assertion with Tokio

* fix(rio): preserve legacy large-block reads

* fix(zip): accept blank tar numeric fields

* fix(s3): align Snowball member import semantics

* fix(s3): authorize PAX legal-hold conditions

* refactor(s3): preserve Snowball error boundary

* fix(iam): support legal-hold policy conditions
 2026-08-31 15:18:28 +00:00
houseme 281e40f1cc fix(scanner): fit usage persistence within publication lease (#6967) 
Lower the default scanner cache save timeout so the derived usage persistence budget stays inside the effective distributed publication lease window.

Add focused regressions for the default publication budget and bootstrap-pending observational baselines, and update operator docs with the new default.

Co-authored-by: heihutu <heihutu@gmail.com>
 2026-08-31 22:58:29 +08:00
houseme 7541bb2c5d fix(ecstore): stabilize decommission capacity retries (#6959) 
* fix(heal): retry unavailable recreate targets

* fix(heal): refresh put-file epochs after target restart

* test(e2e): harden heal restart evidence

Co-Authored-By: heihutu <heihutu@gmail.com>

* test(e2e): cancel competing heal before restart

Co-Authored-By: heihutu <heihutu@gmail.com>

* fix(ecstore): complete decommission capacity recovery

* fix(ecstore): stabilize decommission capacity tests

Keep decommission test capacity snapshots deterministic across startup and mutation probes, serialize capacity-ledger entries during retries, and avoid reacquiring a multipart fence already covered by the outer migration fence.

Co-Authored-By: heihutu <heihutu@gmail.com>

* fix(ecstore): satisfy decommission test lint

Co-Authored-By: heihutu <heihutu@gmail.com>

* fix(ecstore): restore free-version decommission owner

Co-Authored-By: heihutu <heihutu@gmail.com>

---------

Co-authored-by: marshawcoco <marshawcoco@gmail.com>
Co-authored-by: heihutu <heihutu@gmail.com>
Co-authored-by: overtrue <anzhengchao@gmail.com>
 2026-08-31 22:52:47 +08:00
唐小鸭 3e3eb4d8d5 fix(replication): let replicated version purges pass the peer WORM gate (#6960) 
A replicated version purge reaches the peer without the governance
bypass header, so a GOVERNANCE-retained version deleted on the source
with x-amz-bypass-governance-retention was rejected by the peer's WORM
deletion gate forever: retryStats ended at a permanent failed count and
the sites stayed diverged (issue #6850).

The source is authoritative for such a purge: the same WORM gate
already ran there, and GOVERNANCE retention with an authorized bypass
is the only lock state it can purge through. The peer's commit-time
deletion gate now treats an authorized replication delete addressed to
an explicit version as carrying that judged bypass, reusing the same
trust judgment as the replication write exemption
(ObjectOptions::replication_request, set only after the handler
authorized ReplicateDeleteAction). COMPLIANCE retention and legal hold
keep blocking replicated purges, and a plain client delete without the
bypass header stays rejected.
 2026-08-31 22:16:38 +08:00
cxymds 655f6ae452 fix(s3): align Snowball codec compatibility (#6943) 
* fix(s3): harden Snowball extract error boundaries

* fix(s3): close Snowball extract compatibility gaps

* fix(s3): verify Snowball request body completion

* test(s3): reject forged Snowball streaming signatures

* build(deps): pin Snowball archive parser limits

* fix(s3): preserve Snowball trailer and member errors

* docs(architecture): register Snowball tar fork cleanup

* refactor(s3): route Snowball errors through object boundary

* ci(deps): allow pinned tokio-tar source

* fix: align Snowball archive codec detection

* fix(s3): harden Snowball codec compatibility

* fix(s3): preserve Snowball codec compatibility

* test(zip): align yield wake assertion with Tokio

* fix(rio): preserve legacy large-block reads

* fix(zip): accept blank tar numeric fields
 2026-08-31 13:09:51 +00:00
Henry Guo 61821a6f3e fix(heal): resume remote rebuilds after target restart (#6941) 
* fix(heal): retry unavailable recreate targets

* fix(heal): refresh put-file epochs after target restart

* test(e2e): harden heal restart evidence

Co-Authored-By: heihutu <heihutu@gmail.com>

* test(e2e): cancel competing heal before restart

Co-Authored-By: heihutu <heihutu@gmail.com>

---------

Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: heihutu <heihutu@gmail.com>
 2026-08-31 19:39:47 +08:00
cxymds ff28b79088 fix(s3): harden Snowball archive extraction (#6942) 
* fix(s3): harden Snowball extract error boundaries

* fix(s3): close Snowball extract compatibility gaps

* fix(s3): verify Snowball request body completion

* test(s3): reject forged Snowball streaming signatures

* build(deps): pin Snowball archive parser limits

* fix(s3): preserve Snowball trailer and member errors

* docs(architecture): register Snowball tar fork cleanup

* refactor(s3): route Snowball errors through object boundary

* ci(deps): allow pinned tokio-tar source

* ci(e2e): refresh Snowball smoke selection
 2026-08-31 11:18:09 +00:00
唐小鸭 35456bcede test(scanner): serialize tests sharing process-global scanner state (#6940) 
Under the cargo test fallback (threads in one process), tests that touch
the process-global scanner cycle recovery status or the global usage-save
metrics raced each other and failed randomly in full-suite runs.

Mark all touchers with #[serial] per docs/testing/README.md:
- 22 tests reading or writing scanner_cycle_recovery_status() via
  load_scanner_cycle_state_for_startup / reset_scanner_cycle_recovery
- 24 tests mutating global_metrics() usage-save counters via
  store_data_usage_in_backend*, which raced the existing serial
  test_deferred_usage_save_keeps_last_real_save_metric

No-op under nextest, which isolates each test in its own process.
 2026-08-31 18:20:25 +08:00
Zhengchao An 9d4ccb7884 fix(ecstore): finalize decommission capacity recovery (#6955) 2026-08-31 18:09:09 +08:00
Zhengchao An 9a22cb85f3 fix(ecstore): complete decommission capacity recovery (#6949) 2026-08-31 16:53:14 +08:00
Zhengchao An 6c67086d0b fix(ecstore): reserve decommission capacity safely (#6917) 2026-08-31 15:20:09 +08:00
houseme bb37841362 chore(capacity): update default refresh tuning (#6938) 
Align object-capacity refresh defaults with the production-oriented environment values and keep docs, script examples, and tests in sync.

Co-authored-by: heihutu <heihutu@gmail.com>
 2026-08-31 14:43:59 +08:00
GatewayJ 59a7194d7f feat(s3select): schedule streaming progress events (#6913) 
* feat(s3select): schedule streaming progress events

* test(s3select): poll permit release until timeout
 2026-08-31 13:36:47 +08:00
GatewayJ 589a954478 feat(s3select): support compressed CSV and JSON input (#6915) 2026-08-31 13:35:59 +08:00
houseme 1d606e1cf6 perf(ecstore): retry degraded GET with late parity (#6933) 
Co-authored-by: heihutu <heihutu@gmail.com>
 2026-08-31 13:32:46 +08:00
houseme d690f5d60d test(ecstore): stabilize tier recovery cursor fixture (#6935) 2026-08-31 12:09:19 +08:00
houseme 3eca80e37d test(ecstore): make heal rename fixture deterministic (#6934) 2026-08-31 12:09:01 +08:00
houseme 45a2ccb734 fix(ecstore): recover late parity after exact quorum (#6927) 
Co-authored-by: heihutu <heihutu@gmail.com>
 2026-08-31 03:26:20 +00:00
houseme c876df53f5 fix(ecstore): fence snapshot stream polls on lock loss (#6930) 
Co-authored-by: heihutu <heihutu@gmail.com>
 2026-08-31 02:26:24 +00:00
Zhengchao An ca46ae9e56 test(ecstore): pin bucket metadata rollback reads (#6928) 2026-08-31 01:48:03 +00:00
Zhengchao An 7df0920c80 test(replication): bind writable paths to DTO fields (#6923) 2026-08-31 00:48:10 +00:00
Zhengchao An c4ac11d22e fix(scanner): persist decommission catch-up debt (#6922) 2026-08-31 08:45:36 +08:00
houseme 602ed2cbcd test(ecstore): add targeted refresh-loss harness (#6924) 
Co-authored-by: heihutu <heihutu@gmail.com>
 2026-08-31 08:45:04 +08:00
houseme 8ecd8f2520 fix(scanner): preserve cache cycle during usage recovery (#6921) 
Co-authored-by: heihutu <heihutu@gmail.com>
 2026-08-31 00:19:54 +00:00
Zhengchao An e6234d3714 test(ecstore): pin default bucket config bytes (#6920) 2026-08-31 00:03:07 +00:00
Zhengchao An 042a0c3014 docs: register persisted XML compatibility cleanup (#6918) 
docs: register persisted XML compatibility
 2026-08-30 23:53:02 +00:00
houseme 87333f7b24 test(e2e): exercise cluster volume fault proxy (#6919) 
Co-authored-by: heihutu <heihutu@gmail.com>
 2026-08-30 23:37:34 +00:00
Zhengchao An 9945c67f7e fix(ecstore): supervise decommission worker recovery (#6908) 2026-08-31 06:18:00 +08:00
houseme fca1514aac fix(scanner): recover legacy empty usage floor (#6914) 
Co-authored-by: heihutu <heihutu@gmail.com>
 2026-08-31 06:17:26 +08:00
houseme 47ad69b691 fix(ecstore): fail closed on unverifiable data quorum (#6903) 
fix(ecstore): require verification source for degraded GET

Fail closed when reconstruction has only an exact decode quorum, because no surplus source remains to validate the rebuilt data. Cover both erasure engines and the data-shards-only rollout gate.

Co-authored-by: heihutu <heihutu@gmail.com>
 2026-08-30 21:07:47 +00:00
houseme 489408c0b0 perf(ecstore): reuse prepared Select metadata (#6911) 
Co-authored-by: heihutu <heihutu@gmail.com>
 2026-08-30 20:41:24 +00:00
houseme 9244eb36ed test(e2e): route cluster volume endpoints through fault proxy (#6909) 
Co-authored-by: heihutu <heihutu@gmail.com>
 2026-08-30 20:17:16 +00:00
houseme 442298d5f7 test(ecstore): prove in-flight prefetch cancellation (#6904) 
Co-authored-by: heihutu <heihutu@gmail.com>
 2026-08-30 19:17:44 +00:00
唐小鸭 ec1cd606d3 fix(replication): surface object-lock denied purges and back off heal retries (#6900) 2026-08-30 18:59:55 +00:00
houseme 16af688a7a fix(rpc): reject unsigned v2 control mutations (#6905) 
Co-authored-by: heihutu <heihutu@gmail.com>
 2026-08-30 18:17:43 +00:00
唐小鸭 37b23a16da fix(replication): verify replica integrity and default to plain signed payloads (#6895) 2026-08-31 01:43:45 +08:00
houseme 006e9b7d28 test(e2e): cover four-node four-drive cluster topology (#6902) 
Co-authored-by: heihutu <heihutu@gmail.com>
 2026-08-30 17:32:36 +00:00
houseme d214c27583 perf(ecstore): consolidate non-inline read planning (#6892) 
Co-authored-by: heihutu <heihutu@gmail.com>
 2026-08-30 17:15:48 +00:00
唐小鸭 1370434f3a fix(scanner): unblock quota usage baseline on never-converged sites (#6896) 2026-08-31 00:20:04 +08:00
唐小鸭 5dde2c188c fix(replication): retry failed multipart aborts on bounded backoff (#6897) 2026-08-31 00:19:49 +08:00
houseme 2f9c75d04f perf(ecstore): reuse prepared metadata across pools (#6889) 
Co-authored-by: heihutu <heihutu@gmail.com>
 2026-08-30 16:15:10 +00:00
唐小鸭 9ee7b1221d fix(admin): replicate user secret-key rotation to peer sites (#6893) 2026-08-30 23:32:07 +08:00
houseme 3d24526704 fix(ecstore): preserve parity reserves for data-only GET (#6888) 
fix(ecstore): hedge data-only GET with parity

Route the opt-in data-shards-only lockstep path through the bounded parity race and preserve deferred parity reserves across canceled hedges.

Co-authored-by: heihutu <heihutu@gmail.com>
 2026-08-30 20:16:32 +08:00
houseme 51532e19fb test(ecstore): cover multipart snapshot overwrite race (#6887) 
test(ecstore): cover multipart GET overwrite snapshot

Co-authored-by: heihutu <heihutu@gmail.com>
 2026-08-30 12:15:02 +00:00
houseme 07212c4e26 perf(ecstore): gate quorum-aware GET early stop (#6885) 
* perf(ecstore): add gated two-phase GET metadata reads

Co-Authored-By: heihutu <heihutu@gmail.com>

* fix(ecstore): require data-shard coverage for read plans

Co-Authored-By: heihutu <heihutu@gmail.com>

* perf(ecstore): avoid inline overhead in read plan rollout

Co-Authored-By: heihutu <heihutu@gmail.com>

* perf(ecstore): accept quorum-complete read candidates

Co-Authored-By: heihutu <heihutu@gmail.com>

---------

Co-authored-by: heihutu <heihutu@gmail.com>
 2026-08-30 09:33:35 +00:00
GatewayJ 4932af080b feat(s3select): expand typed JSON source paths (#6864) 2026-08-30 06:46:36 +00:00
houseme 7345b49cf6 perf(ecstore): gate GET metadata timing when metrics off (#6879) 
Co-authored-by: heihutu <heihutu@gmail.com>
 2026-08-30 05:43:08 +00:00
GatewayJ 96239fc034 feat(s3select): report uncompressed input byte metrics (#6865) 2026-08-30 04:07:20 +00:00
cxymds 0c18012442 fix(admin): version remote target credential capabilities (#6876) 2026-08-30 10:42:10 +08:00
houseme ee39e4fccb fix(scanner): own publication mutations through storage drain (#6867) 
* fix(scanner): own publication mutations through storage drain

Co-Authored-By: heihutu <heihutu@gmail.com>

* fix(storage): remove unused rename data shim

Co-Authored-By: heihutu <heihutu@gmail.com>

---------

Co-authored-by: heihutu <heihutu@gmail.com>
 2026-08-30 02:39:07 +00:00