gsadmin/rustfs
1
0
Fork 0
mirror of https://github.com/rustfs/rustfs.git synced 2026-09-06 12:09:12 +00:00
Code Issues Packages Projects Releases Wiki Activity
6,200 Commits 58 Branches 132 Tags
1dcdfe4817327972594bec77afebab03f70226bb
Commit Graph

1670 Commits

Author SHA1 Message Date
cxymds 6e26769265 fix(ecstore): make transitioned cleanup crash-safe (#6978) 
* fix(ecstore): fence transitioned object cleanup

* fix(ecstore): address ILM recovery review findings

* fix(ecstore): complete crash-safe tier cleanup recovery

* test(ecstore): avoid typo false positive

* fix(ecstore): stabilize decommission error buckets

* fix(ecstore): stabilize transition delete validation

* fix(ecstore): resume authorized tier delete dispatch

* fix(ecstore): satisfy feature clippy
 2026-09-01 19:09:22 +08:00
唐小鸭 9e6d34785b test(ecstore): deflake inline fanout gate assertion under load (#6992) 
test(ecstore): assert inline fanout gate on deterministic scheduled metric

non_inline_data_read_early_stop_does_not_add_inline_fanout_on_unequal_layout
compared disk_call_counters::KIND_READ_VERSION totals between the two-phase
read-plan gate being off and on. That counter records inside each spawned
fanout task, so the single-pending inline hedge read races the early-stop
abort_all(): whether the hedge task gets its first poll before cancellation
decides a 4-vs-5 count per read. Under concurrent nextest load the two reads
can disagree (reproduced locally at ~5% when run beside one other test,
matching the CI failure on PR #6961).

Assert on the rustfs_io_get_object_metadata_fanout_scheduled histogram
instead, which records the scheduling decision synchronously in the fanout
loop and is deterministic, using the CapturingRecorder + current-thread
runtime pattern already used by the neighboring tests in this module.
 2026-09-01 18:31:35 +08:00
cxymds 03aecc5c3e fix(heal): avoid pool metadata lock recursion (#6991) 2026-09-01 18:31:21 +08:00
Zhengchao An 23ab078c56 fix(ecstore): reclaim stale object prefixes (#6974) 2026-09-01 10:09:00 +00:00
cxymds a45951260a test(ecstore): stabilize activation race capacity (#6989) 2026-09-01 16:05:04 +08:00
Zhengchao An 14a77f9d79 fix(ecstore): supplement split latest listings (#6977) 2026-09-01 07:08:25 +08:00
Zhengchao An c45a8c35c4 test(ecstore): cover disk metric sequence snapshot (#6872) 2026-09-01 03:43:26 +08:00
houseme 7541bb2c5d fix(ecstore): stabilize decommission capacity retries (#6959) 
* fix(heal): retry unavailable recreate targets

* fix(heal): refresh put-file epochs after target restart

* test(e2e): harden heal restart evidence

Co-Authored-By: heihutu <heihutu@gmail.com>

* test(e2e): cancel competing heal before restart

Co-Authored-By: heihutu <heihutu@gmail.com>

* fix(ecstore): complete decommission capacity recovery

* fix(ecstore): stabilize decommission capacity tests

Keep decommission test capacity snapshots deterministic across startup and mutation probes, serialize capacity-ledger entries during retries, and avoid reacquiring a multipart fence already covered by the outer migration fence.

Co-Authored-By: heihutu <heihutu@gmail.com>

* fix(ecstore): satisfy decommission test lint

Co-Authored-By: heihutu <heihutu@gmail.com>

* fix(ecstore): restore free-version decommission owner

Co-Authored-By: heihutu <heihutu@gmail.com>

---------

Co-authored-by: marshawcoco <marshawcoco@gmail.com>
Co-authored-by: heihutu <heihutu@gmail.com>
Co-authored-by: overtrue <anzhengchao@gmail.com>
 2026-08-31 22:52:47 +08:00
唐小鸭 3e3eb4d8d5 fix(replication): let replicated version purges pass the peer WORM gate (#6960) 
A replicated version purge reaches the peer without the governance
bypass header, so a GOVERNANCE-retained version deleted on the source
with x-amz-bypass-governance-retention was rejected by the peer's WORM
deletion gate forever: retryStats ended at a permanent failed count and
the sites stayed diverged (issue #6850).

The source is authoritative for such a purge: the same WORM gate
already ran there, and GOVERNANCE retention with an authorized bypass
is the only lock state it can purge through. The peer's commit-time
deletion gate now treats an authorized replication delete addressed to
an explicit version as carrying that judged bypass, reusing the same
trust judgment as the replication write exemption
(ObjectOptions::replication_request, set only after the handler
authorized ReplicateDeleteAction). COMPLIANCE retention and legal hold
keep blocking replicated purges, and a plain client delete without the
bypass header stays rejected.
 2026-08-31 22:16:38 +08:00
Henry Guo 61821a6f3e fix(heal): resume remote rebuilds after target restart (#6941) 
* fix(heal): retry unavailable recreate targets

* fix(heal): refresh put-file epochs after target restart

* test(e2e): harden heal restart evidence

Co-Authored-By: heihutu <heihutu@gmail.com>

* test(e2e): cancel competing heal before restart

Co-Authored-By: heihutu <heihutu@gmail.com>

---------

Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: heihutu <heihutu@gmail.com>
 2026-08-31 19:39:47 +08:00
Zhengchao An 9d4ccb7884 fix(ecstore): finalize decommission capacity recovery (#6955) 2026-08-31 18:09:09 +08:00
Zhengchao An 9a22cb85f3 fix(ecstore): complete decommission capacity recovery (#6949) 2026-08-31 16:53:14 +08:00
Zhengchao An 6c67086d0b fix(ecstore): reserve decommission capacity safely (#6917) 2026-08-31 15:20:09 +08:00
houseme 1d606e1cf6 perf(ecstore): retry degraded GET with late parity (#6933) 
Co-authored-by: heihutu <heihutu@gmail.com>
 2026-08-31 13:32:46 +08:00
houseme d690f5d60d test(ecstore): stabilize tier recovery cursor fixture (#6935) 2026-08-31 12:09:19 +08:00
houseme 3eca80e37d test(ecstore): make heal rename fixture deterministic (#6934) 2026-08-31 12:09:01 +08:00
houseme 45a2ccb734 fix(ecstore): recover late parity after exact quorum (#6927) 
Co-authored-by: heihutu <heihutu@gmail.com>
 2026-08-31 03:26:20 +00:00
houseme c876df53f5 fix(ecstore): fence snapshot stream polls on lock loss (#6930) 
Co-authored-by: heihutu <heihutu@gmail.com>
 2026-08-31 02:26:24 +00:00
Zhengchao An ca46ae9e56 test(ecstore): pin bucket metadata rollback reads (#6928) 2026-08-31 01:48:03 +00:00
Zhengchao An c4ac11d22e fix(scanner): persist decommission catch-up debt (#6922) 2026-08-31 08:45:36 +08:00
houseme 602ed2cbcd test(ecstore): add targeted refresh-loss harness (#6924) 
Co-authored-by: heihutu <heihutu@gmail.com>
 2026-08-31 08:45:04 +08:00
Zhengchao An e6234d3714 test(ecstore): pin default bucket config bytes (#6920) 2026-08-31 00:03:07 +00:00
Zhengchao An 042a0c3014 docs: register persisted XML compatibility cleanup (#6918) 
docs: register persisted XML compatibility
 2026-08-30 23:53:02 +00:00
Zhengchao An 9945c67f7e fix(ecstore): supervise decommission worker recovery (#6908) 2026-08-31 06:18:00 +08:00
houseme 47ad69b691 fix(ecstore): fail closed on unverifiable data quorum (#6903) 
fix(ecstore): require verification source for degraded GET

Fail closed when reconstruction has only an exact decode quorum, because no surplus source remains to validate the rebuilt data. Cover both erasure engines and the data-shards-only rollout gate.

Co-authored-by: heihutu <heihutu@gmail.com>
 2026-08-30 21:07:47 +00:00
houseme 489408c0b0 perf(ecstore): reuse prepared Select metadata (#6911) 
Co-authored-by: heihutu <heihutu@gmail.com>
 2026-08-30 20:41:24 +00:00
houseme 442298d5f7 test(ecstore): prove in-flight prefetch cancellation (#6904) 
Co-authored-by: heihutu <heihutu@gmail.com>
 2026-08-30 19:17:44 +00:00
唐小鸭 ec1cd606d3 fix(replication): surface object-lock denied purges and back off heal retries (#6900) 2026-08-30 18:59:55 +00:00
houseme 16af688a7a fix(rpc): reject unsigned v2 control mutations (#6905) 
Co-authored-by: heihutu <heihutu@gmail.com>
 2026-08-30 18:17:43 +00:00
唐小鸭 37b23a16da fix(replication): verify replica integrity and default to plain signed payloads (#6895) 2026-08-31 01:43:45 +08:00
houseme d214c27583 perf(ecstore): consolidate non-inline read planning (#6892) 
Co-authored-by: heihutu <heihutu@gmail.com>
 2026-08-30 17:15:48 +00:00
唐小鸭 1370434f3a fix(scanner): unblock quota usage baseline on never-converged sites (#6896) 2026-08-31 00:20:04 +08:00
唐小鸭 5dde2c188c fix(replication): retry failed multipart aborts on bounded backoff (#6897) 2026-08-31 00:19:49 +08:00
houseme 2f9c75d04f perf(ecstore): reuse prepared metadata across pools (#6889) 
Co-authored-by: heihutu <heihutu@gmail.com>
 2026-08-30 16:15:10 +00:00
houseme 3d24526704 fix(ecstore): preserve parity reserves for data-only GET (#6888) 
fix(ecstore): hedge data-only GET with parity

Route the opt-in data-shards-only lockstep path through the bounded parity race and preserve deferred parity reserves across canceled hedges.

Co-authored-by: heihutu <heihutu@gmail.com>
 2026-08-30 20:16:32 +08:00
houseme 51532e19fb test(ecstore): cover multipart snapshot overwrite race (#6887) 
test(ecstore): cover multipart GET overwrite snapshot

Co-authored-by: heihutu <heihutu@gmail.com>
 2026-08-30 12:15:02 +00:00
houseme 07212c4e26 perf(ecstore): gate quorum-aware GET early stop (#6885) 
* perf(ecstore): add gated two-phase GET metadata reads

Co-Authored-By: heihutu <heihutu@gmail.com>

* fix(ecstore): require data-shard coverage for read plans

Co-Authored-By: heihutu <heihutu@gmail.com>

* perf(ecstore): avoid inline overhead in read plan rollout

Co-Authored-By: heihutu <heihutu@gmail.com>

* perf(ecstore): accept quorum-complete read candidates

Co-Authored-By: heihutu <heihutu@gmail.com>

---------

Co-authored-by: heihutu <heihutu@gmail.com>
 2026-08-30 09:33:35 +00:00
houseme 7345b49cf6 perf(ecstore): gate GET metadata timing when metrics off (#6879) 
Co-authored-by: heihutu <heihutu@gmail.com>
 2026-08-30 05:43:08 +00:00
cxymds 0c18012442 fix(admin): version remote target credential capabilities (#6876) 2026-08-30 10:42:10 +08:00
houseme ee39e4fccb fix(scanner): own publication mutations through storage drain (#6867) 
* fix(scanner): own publication mutations through storage drain

Co-Authored-By: heihutu <heihutu@gmail.com>

* fix(storage): remove unused rename data shim

Co-Authored-By: heihutu <heihutu@gmail.com>

---------

Co-authored-by: heihutu <heihutu@gmail.com>
 2026-08-30 02:39:07 +00:00
houseme 90ab2e24c3 perf(ecstore): reuse local fd metadata snapshots (#6868) 
* perf(ecstore): reuse local fd metadata snapshots

Cache the validated shard length beside each reusable descriptor so read hits avoid a repeated fstat while retaining generation and mutation invalidation semantics.

Co-Authored-By: heihutu <heihutu@gmail.com>

* fix(ecstore): pass cached entry to fd cache

Co-Authored-By: heihutu <heihutu@gmail.com>

---------

Co-authored-by: heihutu <heihutu@gmail.com>
 2026-08-30 09:08:44 +08:00
cxymds 21e5b3dc64 fix(ecstore): require durable decommission ledger format (#6871) 2026-08-30 08:47:09 +08:00
cxymds 1e8c8d4cd5 feat(replication): support temporary target credentials (#6860) 2026-08-30 08:44:34 +08:00
houseme ff3ad30f0c fix(scanner): bound publication proof retries on main (#6870) 
* fix(scanner): retain completed publication candidates

* fix(scanner): export publication activity helper

* test(ecstore): retain activity snapshot across retries

* fix(scanner): rebase publication proof retry onto main

Co-Authored-By: heihutu <heihutu@gmail.com>

* fix(scanner): resolve publication proof retry conflicts

Co-Authored-By: heihutu <heihutu@gmail.com>

---------

Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
Co-authored-by: heihutu <heihutu@gmail.com>
 2026-08-29 22:41:36 +00:00
houseme 47a3f5ef01 perf(ecstore): converge disk metric atomic loads (#6866) 
Use the seqlock version as the publication fence and keep payload reads relaxed while validating the final version. This reduces ordering overhead in disk metric recording and snapshot collection without changing the rolling-window contract.

Co-authored-by: heihutu <heihutu@gmail.com>
 2026-08-29 20:38:59 +00:00
houseme 814ab5bbf3 fix(ecstore): classify system metadata failures (#6862) 
fix(ecstore): classify system metadata volume failures

Preserve retryable quorum errors when system metadata reads or writes encounter missing volumes, and cover the create-bucket data-usage path with regressions.

Co-authored-by: heihutu <heihutu@gmail.com>
 2026-08-29 19:48:50 +00:00
houseme 498205b7ec fix(ecstore): keep 1MiB GET off mid-size reader (#6861) 
Co-authored-by: heihutu <heihutu@gmail.com>
 2026-08-29 19:39:51 +00:00
houseme c235f7c05d fix(scanner): retain usage across transient peer failures (#6859) 
* test(scanner): cover bucket drive guard lifecycle

Co-Authored-By: heihutu <heihutu@gmail.com>

* fix(scanner): recover usage floor from fenced backups

Co-Authored-By: heihutu <heihutu@gmail.com>

* fix(scanner): retry transient activity probes

Retry one failed scanner activity probe after a bounded reconnect when the failure is transport-like or timed out. Keep protocol and response validation failures fail-closed.

Co-Authored-By: heihutu <heihutu@gmail.com>

* fix(scanner): retain post-scan observations

Preserve a complete scanner walk as a non-converged observation when the final activity probe is unavailable. Advance the cycle as partial without acknowledging dirty usage.\n\nCo-Authored-By: heihutu <heihutu@gmail.com>

* fix(scanner): classify publication lease deferrals

Distinguish persistence budget and lease deadline deferrals from unavailable activity baselines, and ensure lease-gate deferrals update usage metrics. Keep the fixed lease gate fail-closed while storage-owned commit scope work remains pending.

Co-Authored-By: heihutu <heihutu@gmail.com>

* fix(scanner): recover usage floor from fenced backups

Co-Authored-By: heihutu <heihutu@gmail.com>

* fix(scanner): preserve publication lease defer reasons

Keep lease expiry and release failures distinct from activity baseline failures so scanner freshness metrics and cycle outcomes identify the publication barrier that blocked progress. Preserve fail-closed behavior.

Co-Authored-By: heihutu <heihutu@gmail.com>

* fix(scanner): reuse recovered usage baseline for publication

Co-Authored-By: heihutu <heihutu@gmail.com>

* fix(scanner): fence legacy usage floor fallback

Co-Authored-By: heihutu <heihutu@gmail.com>

* fix(scanner): use typed activity timeout error

---------

Co-authored-by: heihutu <heihutu@gmail.com>
 2026-08-30 02:54:20 +08:00
cxymds 64cca79fbb feat(admin): expose remote target credential capability state (#6857) 2026-08-30 00:42:14 +08:00
唐小鸭 e009eab4f1 fix(replication): surface failed objects and abort orphaned uploads (#6840) 
fix(replication): surface per-object failures and abort orphaned multipart uploads

Replication could mark an object FAILED with no server-log line naming
the object: the target-offline skip paths logged at debug without the
object key, and several failure branches omitted the key entirely. A
failed multipart transfer also leaked its incomplete upload on the
target, since nothing ever aborted it after CreateMultipartUpload
succeeded.

Log the offline skips at warn with the object key (they report the
object FAILED, matching the per-object put_object failure level), add
the object field to the remaining failure branches, and abort the
target-side multipart upload best-effort on any post-create failure
without masking the original transfer error.

Fixes #6825
 2026-08-29 15:49:59 +08:00