Commit Graph

2909 Commits

Author SHA1 Message Date
Zhengchao An e091a7e702 fix(ecstore): fence pool metadata replica updates (#6466)
* fix(ecstore): fence pool metadata replica updates

* fix(ecstore): block decommission on unsafe pool metadata

* fix(ecstore): block writes after pool metadata save errors

* fix(ecstore): latch pool metadata writes before await
2026-08-24 14:25:19 +08:00
cxymds eec0e0e056 fix(scanner): fence movement generation publication (#6461)
* feat(scanner): add movement generation fencing

* fix(scanner): prioritize unverified cycle deferral

* feat(ecstore): add scanner publication lease fence

* feat(rpc): add scanner publication lease protocol

* feat(scanner): hold remote leases through usage publish

* test(scanner): cover publication lease fencing

* fix(scanner): fence remote leases across restart and delay

* feat(rpc): fence scanner publication rename writes

* fix(scanner): fence observed cleanup deletes

* fix(proto): qualify lease release test types

* fix(scanner): pin movement notifications

* fix(scanner): clean publication imports

* fix(ecstore): satisfy scanner fence clippy

* refactor(scanner): group wait and publication options

* fix(scanner): satisfy final lint and facade guards

* fix(rpc): resolve facade export conflicts

* fix(ci): remove unused decommission and healing facades

* fix(ci): cfg-gate test-only usage overlay import

* fix(scanner): wake on remote scanner restart
2026-08-24 14:17:35 +08:00
cxymds 1c5c28842a fix(scanner): reject duplicate usage updates (#6445)
* fix(scanner): reject duplicate usage updates

* style: format decommission test imports

* fix(ci): remove unused decommission and healing facades

* fix(ci): cfg-gate test-only usage overlay import

---------

Co-authored-by: houseme <housemecn@gmail.com>
2026-08-24 14:17:11 +08:00
Zhengchao An 16ca65ccab test(e2e): activate S3 Select regressions (#6442)
* test(e2e): activate S3 Select regressions

* test(e2e): bind S3 Select Linux selection
2026-08-24 14:16:48 +08:00
Zhengchao An ecefb5644b fix(e2e): honor custom Cargo target directory (#6434)
* fix(e2e): honor custom Cargo target directory

* test(e2e): bind target-dir selection digests

---------

Co-authored-by: houseme <housemecn@gmail.com>
2026-08-24 14:16:26 +08:00
Zhengchao An f473b6dbf8 test(e2e): activate configured Vault coverage (#6430)
* test(e2e): activate configured Vault roundtrip

* test(e2e): bind Vault selection to Linux listing

---------

Co-authored-by: houseme <housemecn@gmail.com>
2026-08-24 14:14:27 +08:00
Zhengchao An af6545b689 test(e2e): activate data usage regressions (#6429)
* test(e2e): activate data usage regressions

* test(e2e): bind data usage selection to Linux listing

* test(e2e): refresh data usage Darwin selection

---------

Co-authored-by: houseme <housemecn@gmail.com>
2026-08-24 14:14:02 +08:00
Zhengchao An a66b568ba0 test(e2e): require zero KMS concurrency failures (#6428)
* test(e2e): require zero KMS upload failures

* test(e2e): bind KMS selection to Linux listing

---------

Co-authored-by: houseme <housemecn@gmail.com>
2026-08-24 14:13:36 +08:00
Zhengchao An 232190a205 test(e2e): activate policy variable coverage (#6427)
* test(e2e): activate policy variable coverage

* test(e2e): update policy suite membership

* test(e2e): bind policy selection to Linux listing

---------

Co-authored-by: houseme <housemecn@gmail.com>
2026-08-24 14:11:36 +08:00
Zhengchao An f52a389652 fix(ecstore): persist unresolved decommission entries (#6415)
* fix(ecstore): persist unresolved decommission entries

* fix(ecstore): type decommission completion result

* fix(ecstore): allow intentional decommission listing signatures under strict clippy

The sftp/swift feature-matrix clippy gates run with -D warnings and
flag the unresolved-entry resolver (large Err payload by design, 8
context parameters) and the decommission listing driver (9 args).
Document why and align with the existing decommission_entry precedent.
2026-08-24 14:05:08 +08:00
Zhengchao An 73cd1b5be2 fix(ecstore): fence rebalance and decommission activation (#6400)
* fix(ecstore): fence rebalance and decommission activation

* fix(ecstore): fence lost activation locks

* fix(ecstore): bind rebalance workers to activation id

* fix(ecstore): close rebalance activation races

* test(ecstore): exercise lost rebalance commit fence

* fix(ecstore): repair rebalance fence test wiring

* test(ecstore): reuse rebalance metadata fixture

* fix(ecstore): satisfy rebalance activation clippy checks

* fix(ecstore): fence stale rebalance workers

* fix(ecstore): commit rebalance activation after persistence

* fix(ecstore): fence rebalance commits and unblock stop

* test(ecstore): exercise real rebalance fences

* fix(rebalance): cancel admin stop before activation wait

* fix(ecstore): fence multipart staging on rebalance lock loss

* fix(ecstore): adopt activations after durable commit

* fix(rebalance): preserve committed activation recovery

* fix(rebalance): make prepared stop terminal-safe

* fix(ecstore): repair rebalance test imports

* fix(ecstore): repair rebalance entry runtime failures

* test(ecstore): fix activation fence synchronization

* test(ecstore): scope rebalance disk trait import

* test(ecstore): observe decommission lock attempt

* fix(ecstore): align activation fence test imports

* fix(ecstore): remove duplicate activation test import

* fix(ecstore): resolve CI clippy failures

* fix: satisfy activation merge lint gates

---------

Co-authored-by: houseme <housemecn@gmail.com>
2026-08-24 14:03:21 +08:00
Zhengchao An 40bf9f0425 fix(ecstore): prefer active pool reads during decommission (#6475) 2026-08-24 09:36:35 +08:00
Zhengchao An 0e015360cc test(scanner): repair post-fence fixtures (#6478) 2026-08-24 09:33:21 +08:00
Zhengchao An 2bd1df3075 fix(scanner): preserve default usage cache wire format (#6477) 2026-08-24 09:33:16 +08:00
Zhengchao An 99fb77b164 fix(ci): restore main checks (#6469)
* fix(ci): restore main checks

* fix(scanner): bootstrap pristine usage state

* fix(scanner): reject empty usage snapshots
2026-08-24 09:33:00 +08:00
Zhengchao An 57eaa8228d fix(ecstore): migrate tier free versions during decommission (#6393) 2026-08-24 09:21:53 +08:00
Zhengchao An aa0a374aa4 style: rustfmt decommission import groups on main (#6474)
cargo fmt --check has been failing since #6410 landed: the merged
decommission import groups in core/pools.rs were not canonical
rustfmt output. Apply formatting verbatim, no code changes.
2026-08-24 03:07:49 +08:00
Zhengchao An 1ec1a8d90d fix(ecstore): run metadata decommission before buckets (#6410)
* fix(ecstore): run decommission metadata first

* fix(ecstore): clear clippy warnings

---------

Co-authored-by: houseme <housemecn@gmail.com>
2026-08-24 00:22:46 +08:00
Zhengchao An 28b3ecf547 fix(ci): restore main checks (#6467) 2026-08-24 00:15:58 +08:00
Zhengchao An 33341d5fcf fix(ecstore): unify decommission resume queue (#6403)
* fix(ecstore): unify decommission resume queue

* fix(ecstore): clear swift clippy warnings

* fix(ecstore): serialize decommission recovery

---------

Co-authored-by: houseme <housemecn@gmail.com>
2026-08-23 23:51:33 +08:00
Zhengchao An 2a43e021c9 fix(ecstore): fence bucket heal during decommission (#6416)
* fix(ecstore): fence bucket heal during decommission

* fix(ecstore): preserve unfenced heal compatibility

---------

Co-authored-by: houseme <housemecn@gmail.com>
2026-08-23 23:38:16 +08:00
Zhengchao An 8bf4f20890 test(e2e): activate conditional write regressions (#6454) 2026-08-23 23:34:50 +08:00
Zhengchao An d269b90201 fix(ci): restore main branch checks (#6465)
* fix(protocols): add missing test dependency

* style(ecstore): fix decommission formatting
2026-08-23 23:34:46 +08:00
Zhengchao An 7f32c675ac fix(ecstore): recover pool metadata from replicas (#6457) 2026-08-23 23:22:03 +08:00
Zhengchao An 8ccf7151f3 fix(ecstore): persist cancel before signaling (#6423)
* fix(ecstore): persist decommission cancel before signaling

* test(ecstore): align cancel regressions with movement gate

---------

Co-authored-by: houseme <housemecn@gmail.com>
2026-08-23 23:17:14 +08:00
Zhengchao An 8c3aeaebed fix(ecstore): drain multipart uploads before decommission (#6414)
* fix(ecstore): drain multipart uploads before decommission

* fix(ecstore): prioritize active multipart pools

---------

Co-authored-by: houseme <housemecn@gmail.com>
2026-08-23 23:04:35 +08:00
houseme 201c653dcd fix(ci): restore workspace lint compatibility (#6460) 2026-08-23 22:35:43 +08:00
唐小鸭 3ce01dcc73 fix(object-lock): unblock authorized replication writes on locked versions and tolerate cleared lock metadata (#6413) 2026-08-23 22:35:23 +08:00
Zhengchao An 18cde00fad fix(ecstore): retry decommission entries safely on source changes (#6419)
A single object's SourceChanged during decommission cleanup no longer
cancels the shared worker token and fails the whole pool operation.
Cleanup preflight and source-cleanup outcomes now retry per entry with
bounded attempts and cancellation-aware backoff, applied uniformly to
ordinary versions, delete markers, and tiered copies (removing the
try-once-only branches); every retry re-lists the entry and redoes
version multiset validation before touching the source. Only quorum
loss, unrecoverable system errors, or exceeding a pool-level
SourceChanged exhaustion threshold still fails the decommission, and
exhausted entries never delete their source versions.

Retry attempts, backoff, and deferred/exhausted reasons are logged per
entry for observability. Heavy regression tests spawn on dedicated
32MiB stacks following the existing store-test pattern.

Fixes rustfs/backlog#1913

Co-authored-by: houseme <housemecn@gmail.com>
2026-08-23 21:32:49 +08:00
Zhengchao An 5cedf73d7c fix(ecstore): terminate walk directory streams (#6462)
* fix(ecstore): terminate walk directory streams

* test(e2e): refresh node service selection

* fix(filemeta): initialize empty metacache streams
2026-08-23 21:32:39 +08:00
Zhengchao An 43a10e3c24 fix(ecstore): tolerate migration identity rewrites (#6447) 2026-08-23 20:24:04 +08:00
Zhengchao An 11a90ce843 test(e2e): add platform-safe selection updates (#6424) 2026-08-23 20:22:46 +08:00
Zhengchao An b52cdd69ae test(e2e): fail incomplete conditional PUT races (#6421) 2026-08-23 20:20:16 +08:00
Zhengchao An 5bb9ffffcd test(e2e): enforce external client prerequisites (#6402) 2026-08-23 20:18:41 +08:00
唐小鸭 31933c32f9 fix(replication): apply receiver-side LWW to inbound metadata categories (#6379) 2026-08-23 20:17:50 +08:00
唐小鸭 0e88a27d05 fix(admin): use madmin key names in list-remote-targets response (#6377) 2026-08-23 20:17:34 +08:00
Zhengchao An 38d37121ca test(e2e): activate group management regressions (#6405) 2026-08-23 20:17:07 +08:00
houseme 3f3b9fd426 perf(ecstore): attribute batch read version wait stages (#6456) 2026-08-23 19:31:14 +08:00
cxymds a8e4b67d99 feat(metrics): expose deferred usage freshness (#6449) 2026-08-23 19:31:01 +08:00
cxymds b2e60be647 fix(scanner): fence unknown tier accounting (#6396) 2026-08-23 19:28:43 +08:00
cxymds 14e3eb787d fix(heal): correct progress accounting (#6382)
* fix(heal): correct progress accounting

* fix(heal): atomically persist page progress

* fix(heal): preserve terminal progress counters

* fix(heal): make resume handoff crash safe

* fix(heal): preserve resumable bucket checkpoints

* fix(heal): satisfy checkpoint outcome lint

* fix(heal): preserve progress status across nodes

* fix(heal): stabilize progress generations

* style: restore rebalance formatting

* test(heal): cover cross-set baseline generation

---------

Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: overtrue <anzhengchao@gmail.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: heihutu <heihutu@gmail.com>
2026-08-23 17:29:45 +08:00
cxymds 76a863b3ea fix(scanner): unify unknown metadata size accounting (#6394)
* fix(scanner): unify unknown metadata size accounting

* fix(scanner): preserve restore expiry semantics

* fix(ci): resolve ecstore clippy warnings

* fix(scanner): close lifecycle review gaps

---------

Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-08-23 17:28:52 +08:00
cxymds e196a134cc fix(scanner): fence system metadata publication (#6444)
* feat(scanner): fence usage publication during data movement

* fix(scanner): detect movement refresh state changes

* fix(scanner): fence publication during data movement

* fix(scanner): close movement epoch publication races

* fix(scanner): fence movement-sensitive publication paths

* fix(scanner): fence cache and heal recovery paths

* fix(scanner): carry publication epoch through scan cycle

* fix(scanner): recheck remote cache epoch after save

* fix(scanner): recheck local cache epoch before publish

* fix(scanner): fence data usage writers and baseline

* fix(scanner): expose decommission activity to publication fence

* fix(scanner): release publication gate before reads

* fix(scanner): complete publication fence integration

* fix(scanner): avoid empty usage baseline publication

* chore(scanner): gate test-only helpers

* fix: use decommission canceler in reload test

---------

Co-authored-by: houseme <housemecn@gmail.com>
2026-08-23 17:28:21 +08:00
cxymds 9cda615519 fix(scanner): discover sub-quorum heal candidates (#6384)
* fix(scanner): preserve unversioned heal retries

* fix(scanner): bound orphan heal discovery fallback

* fix(filemeta): fence unsafe heal key components

* fix(scanner): preserve exact overflow heal versions

---------

Co-authored-by: houseme <housemecn@gmail.com>
2026-08-23 16:46:36 +08:00
cxymds 9a8ca3a7a9 fix(heal): add bounded resume artifact inspection (#6420)
Co-authored-by: houseme <housemecn@gmail.com>
2026-08-23 16:45:34 +08:00
cxymds 32cc7c8fcf fix(heal): coalesce duplicate MRF intents (#6425)
Co-authored-by: houseme <housemecn@gmail.com>
2026-08-23 16:45:22 +08:00
cxymds 2bb0ab18b2 docs(scanner): baseline scanner heal admission (#6426)
Co-authored-by: houseme <housemecn@gmail.com>
2026-08-23 16:45:07 +08:00
cxymds 8dc2537178 fix(scanner): publish per-set usage freshness (#6432)
* fix(scanner): publish partial usage observations

* fix(ecstore): preserve quota baseline across restart

* style: format usage freshness changes

* fix(scanner): correct observational usage arguments

---------

Co-authored-by: houseme <housemecn@gmail.com>
2026-08-23 16:44:45 +08:00
Zhengchao An 0e70dbd511 fix(app): wait for peer bucket metadata reload (#6381)
Co-authored-by: houseme <housemecn@gmail.com>
2026-08-23 16:43:29 +08:00
Zhengchao An 34bbc1adb3 fix(ecstore): preserve ILM state during pool decommission (#6369)
* fix(ecstore): migrate ILM metadata during decommission

* fix(ecstore): verify ILM metadata before decommission

* fix(ecstore): track ILM recovery across decommission

* fix(ecstore): close ILM receipt recovery gaps

* fix(ecstore): anchor decommission ILM receipts

* fix(ecstore): re-export durable ILM checkpoint

* fix(ecstore): avoid terminal receipt shadowing

* fix(ecstore): harden durable ILM cursor receipts

* fix(ecstore): repair durable ILM receipt recovery

* test(ecstore): cover durable ILM recovery boundaries

* test(ecstore): serialize multi-source ILM recovery

* test(ecstore): compile multi-source ILM recovery

* test(ecstore): isolate durable ILM scenario stack

* fix(ecstore): preserve active ILM source journals

* fix(ecstore): distinguish active ILM target cleanup

* fix(ecstore): restore decommission test imports

* fix(ecstore): drop removed decommission test import

* fix(ecstore): remove duplicate decommission error helper

* fix(ecstore): fence final decommission sweep

* test(ecstore): cover final sweep cancel fence

* fix(ecstore): fence decommission cancellation

* fix(ecstore): remove redundant clone in test

* fix(ecstore): keep manual transition progress compatible

* fix(ecstore): restore decommission worker wrapper

* fix(ecstore): restore decommission compile contracts

* test(ecstore): adapt reload worker canceler
2026-08-23 16:43:11 +08:00