Commit Graph

5795 Commits

Author SHA1 Message Date
Zhengchao An f52a389652 fix(ecstore): persist unresolved decommission entries (#6415)
* fix(ecstore): persist unresolved decommission entries

* fix(ecstore): type decommission completion result

* fix(ecstore): allow intentional decommission listing signatures under strict clippy

The sftp/swift feature-matrix clippy gates run with -D warnings and
flag the unresolved-entry resolver (large Err payload by design, 8
context parameters) and the decommission listing driver (9 args).
Document why and align with the existing decommission_entry precedent.
2026-08-24 14:05:08 +08:00
Zhengchao An 73cd1b5be2 fix(ecstore): fence rebalance and decommission activation (#6400)
* fix(ecstore): fence rebalance and decommission activation

* fix(ecstore): fence lost activation locks

* fix(ecstore): bind rebalance workers to activation id

* fix(ecstore): close rebalance activation races

* test(ecstore): exercise lost rebalance commit fence

* fix(ecstore): repair rebalance fence test wiring

* test(ecstore): reuse rebalance metadata fixture

* fix(ecstore): satisfy rebalance activation clippy checks

* fix(ecstore): fence stale rebalance workers

* fix(ecstore): commit rebalance activation after persistence

* fix(ecstore): fence rebalance commits and unblock stop

* test(ecstore): exercise real rebalance fences

* fix(rebalance): cancel admin stop before activation wait

* fix(ecstore): fence multipart staging on rebalance lock loss

* fix(ecstore): adopt activations after durable commit

* fix(rebalance): preserve committed activation recovery

* fix(rebalance): make prepared stop terminal-safe

* fix(ecstore): repair rebalance test imports

* fix(ecstore): repair rebalance entry runtime failures

* test(ecstore): fix activation fence synchronization

* test(ecstore): scope rebalance disk trait import

* test(ecstore): observe decommission lock attempt

* fix(ecstore): align activation fence test imports

* fix(ecstore): remove duplicate activation test import

* fix(ecstore): resolve CI clippy failures

* fix: satisfy activation merge lint gates

---------

Co-authored-by: houseme <housemecn@gmail.com>
2026-08-24 14:03:21 +08:00
Zhengchao An 40bf9f0425 fix(ecstore): prefer active pool reads during decommission (#6475) 2026-08-24 09:36:35 +08:00
Zhengchao An eb0384c225 test(app): distinguish usage overlay from quota floor (#6479)
test(app): preserve delete quota floor assertion
2026-08-24 09:33:27 +08:00
Zhengchao An 0e015360cc test(scanner): repair post-fence fixtures (#6478) 2026-08-24 09:33:21 +08:00
Zhengchao An 2bd1df3075 fix(scanner): preserve default usage cache wire format (#6477) 2026-08-24 09:33:16 +08:00
Zhengchao An 9935911e93 fix(ecstore): drop unused decommission bucket runner and refresh e2e linux digest (#6476)
fix(rustfs): drop sftp-dead scanner capability import and unwired heal_bucket trait method

The sftp feature matrix compiles fewer call sites: the plain
sign_ns_scanner_capability re-export had no remaining user, and
StoragePeerS3ClientExt::heal_bucket was superseded by
heal_bucket_with_fence when #6416 wired the fenced path.
2026-08-24 09:33:10 +08:00
Zhengchao An 99fb77b164 fix(ci): restore main checks (#6469)
* fix(ci): restore main checks

* fix(scanner): bootstrap pristine usage state

* fix(scanner): reject empty usage snapshots
2026-08-24 09:33:00 +08:00
Zhengchao An ebff02304d fix(connect): make registration bootstrap retry durable (#6468)
* fix(connect): make registration state durability retry-safe

* fix(connect): reject parent state paths

* fix(connect): harden state directory creation

* fix(connect): bound bootstrap directory syncs

* fix(connect): require durable state parent

* fix(connect): close bootstrap marker race

* test(connect): cover marker sync failure
2026-08-24 09:32:54 +08:00
Zhengchao An 57eaa8228d fix(ecstore): migrate tier free versions during decommission (#6393) 2026-08-24 09:21:53 +08:00
Zhengchao An aa0a374aa4 style: rustfmt decommission import groups on main (#6474)
cargo fmt --check has been failing since #6410 landed: the merged
decommission import groups in core/pools.rs were not canonical
rustfmt output. Apply formatting verbatim, no code changes.
2026-08-24 03:07:49 +08:00
Zhengchao An 1ec1a8d90d fix(ecstore): run metadata decommission before buckets (#6410)
* fix(ecstore): run decommission metadata first

* fix(ecstore): clear clippy warnings

---------

Co-authored-by: houseme <housemecn@gmail.com>
2026-08-24 00:22:46 +08:00
Zhengchao An 28b3ecf547 fix(ci): restore main checks (#6467) 2026-08-24 00:15:58 +08:00
Zhengchao An 33341d5fcf fix(ecstore): unify decommission resume queue (#6403)
* fix(ecstore): unify decommission resume queue

* fix(ecstore): clear swift clippy warnings

* fix(ecstore): serialize decommission recovery

---------

Co-authored-by: houseme <housemecn@gmail.com>
2026-08-23 23:51:33 +08:00
Zhengchao An 2a43e021c9 fix(ecstore): fence bucket heal during decommission (#6416)
* fix(ecstore): fence bucket heal during decommission

* fix(ecstore): preserve unfenced heal compatibility

---------

Co-authored-by: houseme <housemecn@gmail.com>
2026-08-23 23:38:16 +08:00
Zhengchao An 8bf4f20890 test(e2e): activate conditional write regressions (#6454) 2026-08-23 23:34:50 +08:00
Zhengchao An d269b90201 fix(ci): restore main branch checks (#6465)
* fix(protocols): add missing test dependency

* style(ecstore): fix decommission formatting
2026-08-23 23:34:46 +08:00
houseme 83aa9c221b test: add coalescer delay cost report (#6464)
Add a read-only Prometheus report helper for backlog#2007 so the 200us vs 50us coalescer delay experiment can capture RPC, batch distribution, stage latency, and host-cost signals with one fixed evidence format.

Co-authored-by: heihutu <heihutu@gmail.com>
2026-08-23 23:22:38 +08:00
Zhengchao An 7f32c675ac fix(ecstore): recover pool metadata from replicas (#6457) 2026-08-23 23:22:03 +08:00
Zhengchao An 8ccf7151f3 fix(ecstore): persist cancel before signaling (#6423)
* fix(ecstore): persist decommission cancel before signaling

* test(ecstore): align cancel regressions with movement gate

---------

Co-authored-by: houseme <housemecn@gmail.com>
2026-08-23 23:17:14 +08:00
Zhengchao An 8c3aeaebed fix(ecstore): drain multipart uploads before decommission (#6414)
* fix(ecstore): drain multipart uploads before decommission

* fix(ecstore): prioritize active multipart pools

---------

Co-authored-by: houseme <housemecn@gmail.com>
2026-08-23 23:04:35 +08:00
houseme 201c653dcd fix(ci): restore workspace lint compatibility (#6460) 2026-08-23 22:35:43 +08:00
唐小鸭 3ce01dcc73 fix(object-lock): unblock authorized replication writes on locked versions and tolerate cleared lock metadata (#6413) 2026-08-23 22:35:23 +08:00
Zhengchao An 0d15ce1865 ci: install protocol socket oracle (#6411)
Co-authored-by: houseme <housemecn@gmail.com>
2026-08-23 21:33:00 +08:00
Zhengchao An 415427f99d feat(connect): emit low-frequency inventory (#6418)
* feat(connect): emit low-frequency inventory

* fix(connect): retry incomplete inventory samples

* fix(connect): reset inventory sampling backoff

* fix(connect): mark missing drives offline in inventory

* fix(connect): harden inventory snapshots

* fix(connect): validate persisted inventory state

* fix(connect): close inventory lifecycle gaps

* fix(connect): validate inventory topology slots

* fix(connect): validate inventory geometry
2026-08-23 21:32:54 +08:00
Zhengchao An 18cde00fad fix(ecstore): retry decommission entries safely on source changes (#6419)
A single object's SourceChanged during decommission cleanup no longer
cancels the shared worker token and fails the whole pool operation.
Cleanup preflight and source-cleanup outcomes now retry per entry with
bounded attempts and cancellation-aware backoff, applied uniformly to
ordinary versions, delete markers, and tiered copies (removing the
try-once-only branches); every retry re-lists the entry and redoes
version multiset validation before touching the source. Only quorum
loss, unrecoverable system errors, or exceeding a pool-level
SourceChanged exhaustion threshold still fails the decommission, and
exhausted entries never delete their source versions.

Retry attempts, backoff, and deferred/exhausted reasons are logged per
entry for observability. Heavy regression tests spawn on dedicated
32MiB stacks following the existing store-test pattern.

Fixes rustfs/backlog#1913

Co-authored-by: houseme <housemecn@gmail.com>
2026-08-23 21:32:49 +08:00
Zhengchao An 2788ef7229 feat(connect): collect bounded offline diagnostics (#6450)
* feat(connect): collect bounded offline diagnostics

* fix(connect): tighten offline diagnostic boundaries
2026-08-23 21:32:43 +08:00
Zhengchao An 5cedf73d7c fix(ecstore): terminate walk directory streams (#6462)
* fix(ecstore): terminate walk directory streams

* test(e2e): refresh node service selection

* fix(filemeta): initialize empty metacache streams
2026-08-23 21:32:39 +08:00
houseme f694a0000a fix(server): adapt quick-xml name handling (#6458) 2026-08-23 20:37:42 +08:00
Zhengchao An ba8f2e90be feat(connect): add registration bootstrap command (#6452) 2026-08-23 20:24:23 +08:00
Zhengchao An 43a10e3c24 fix(ecstore): tolerate migration identity rewrites (#6447) 2026-08-23 20:24:04 +08:00
Zhengchao An 11a90ce843 test(e2e): add platform-safe selection updates (#6424) 2026-08-23 20:22:46 +08:00
Zhengchao An b52cdd69ae test(e2e): fail incomplete conditional PUT races (#6421) 2026-08-23 20:20:16 +08:00
Zhengchao An 06ef472def fix(ci): make Warp ABBA evidence bounded and complete (#6417) 2026-08-23 20:20:01 +08:00
Zhengchao An d3c0714b3a ci: add s3tests upstream HEAD canary (#6409) 2026-08-23 20:19:13 +08:00
Zhengchao An b5b060a9a1 ci: pin s3tests Python tools (#6407) 2026-08-23 20:18:56 +08:00
Zhengchao An 5bb9ffffcd test(e2e): enforce external client prerequisites (#6402) 2026-08-23 20:18:41 +08:00
Zhengchao An 6f6dd19cc7 ci(coverage): add security ratchet calibration (#6388) 2026-08-23 20:18:04 +08:00
唐小鸭 31933c32f9 fix(replication): apply receiver-side LWW to inbound metadata categories (#6379) 2026-08-23 20:17:50 +08:00
唐小鸭 0e88a27d05 fix(admin): use madmin key names in list-remote-targets response (#6377) 2026-08-23 20:17:34 +08:00
唐小鸭 35e264a9f5 fix(admin): advertise IAM admin capabilities in runtime capabilities (#6336) 2026-08-23 20:17:24 +08:00
Zhengchao An 38d37121ca test(e2e): activate group management regressions (#6405) 2026-08-23 20:17:07 +08:00
houseme 3f3b9fd426 perf(ecstore): attribute batch read version wait stages (#6456) 2026-08-23 19:31:14 +08:00
cxymds a8e4b67d99 feat(metrics): expose deferred usage freshness (#6449) 2026-08-23 19:31:01 +08:00
cxymds b2e60be647 fix(scanner): fence unknown tier accounting (#6396) 2026-08-23 19:28:43 +08:00
cxymds 14e3eb787d fix(heal): correct progress accounting (#6382)
* fix(heal): correct progress accounting

* fix(heal): atomically persist page progress

* fix(heal): preserve terminal progress counters

* fix(heal): make resume handoff crash safe

* fix(heal): preserve resumable bucket checkpoints

* fix(heal): satisfy checkpoint outcome lint

* fix(heal): preserve progress status across nodes

* fix(heal): stabilize progress generations

* style: restore rebalance formatting

* test(heal): cover cross-set baseline generation

---------

Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: overtrue <anzhengchao@gmail.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: heihutu <heihutu@gmail.com>
2026-08-23 17:29:45 +08:00
cxymds 76a863b3ea fix(scanner): unify unknown metadata size accounting (#6394)
* fix(scanner): unify unknown metadata size accounting

* fix(scanner): preserve restore expiry semantics

* fix(ci): resolve ecstore clippy warnings

* fix(scanner): close lifecycle review gaps

---------

Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-08-23 17:28:52 +08:00
cxymds e196a134cc fix(scanner): fence system metadata publication (#6444)
* feat(scanner): fence usage publication during data movement

* fix(scanner): detect movement refresh state changes

* fix(scanner): fence publication during data movement

* fix(scanner): close movement epoch publication races

* fix(scanner): fence movement-sensitive publication paths

* fix(scanner): fence cache and heal recovery paths

* fix(scanner): carry publication epoch through scan cycle

* fix(scanner): recheck remote cache epoch after save

* fix(scanner): recheck local cache epoch before publish

* fix(scanner): fence data usage writers and baseline

* fix(scanner): expose decommission activity to publication fence

* fix(scanner): release publication gate before reads

* fix(scanner): complete publication fence integration

* fix(scanner): avoid empty usage baseline publication

* chore(scanner): gate test-only helpers

* fix: use decommission canceler in reload test

---------

Co-authored-by: houseme <housemecn@gmail.com>
2026-08-23 17:28:21 +08:00
cxymds 9cda615519 fix(scanner): discover sub-quorum heal candidates (#6384)
* fix(scanner): preserve unversioned heal retries

* fix(scanner): bound orphan heal discovery fallback

* fix(filemeta): fence unsafe heal key components

* fix(scanner): preserve exact overflow heal versions

---------

Co-authored-by: houseme <housemecn@gmail.com>
2026-08-23 16:46:36 +08:00
cxymds 9a8ca3a7a9 fix(heal): add bounded resume artifact inspection (#6420)
Co-authored-by: houseme <housemecn@gmail.com>
2026-08-23 16:45:34 +08:00