mirror of
https://github.com/rustfs/rustfs.git
synced 2026-10-27 14:27:10 +00:00
c8f14dc3371201cf64b0edd929b2c50f77bf12ff
964 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Chris
|
c8f14dc337 |
test(scanner): attest running benchmark process identity (#8876)
* test(scanner): attest running benchmark process identity * fix(scanner): reject aliased process receipt data roots * fix(scanner): attest scheduled scan configuration |
||
|
Chris
|
334e8d9d3e |
test(scanner): verify retained versions with an independent oracle (#8870)
* test(scanner): add independent version content oracle * test(ecstore): drain guarded delete fixture rename tails * docs(scanner): record SDK seeding requirements |
||
|
Hauser
|
911ea265c5 |
fix(storage): apply and index explicit delete-marker removals (#8854)
* fix(ecstore): purge every explicit delete-marker version Keep explicit marker removals on the canonical version-delete path so same-key batches are not truncated and pending replication does not retain source markers. Preserve marker identity and timestamps in the response. Add real four-disk regressions for mixed version cleanup, duplicate marker IDs, pending replication, and null/UUID marker removal. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * perf(filemeta): index explicit delete-marker batches Reuse the batch index and final compaction for ordinary marker removals. Keep small marker-only histories on the scalar path and preserve fallback for free versions and complex mutations. Add scalar-equivalence, boundary, corruption and deterministic scan-count coverage, a real-disk mixed-history regression, and reproducible marker benchmark cases. Preserve existing benchmark defaults while honoring CLI sampling overrides. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(filemeta): remove redundant marker fixture clone Move the original metadata into its final test fixture after the last borrow, avoiding the redundant clone reported by strict Clippy checks. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(storage): validate explicit marker deletion and recovery Exercise UUID/null and duplicate explicit marker removals over real HTTP with retries disabled, including immediate four-node visibility and exact retained payloads. Cover all-drive PREPARED/APPLIED/DECISION process termination, rollback/commit recovery, and old-ID retries across a second restart without deleting a fresh marker. Use flat sampling and fixed per-case fixture batches in the metadata benchmark. Update profile selection digests for the added regressions. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(heal): synchronize crash fixture startup and flush readiness Give disk setup and the durable flush boundary separate readiness gates, keeping each phase bounded to ten seconds. Preserve the real SIGKILL and restart assertions while reporting early child exits and retaining output. Reap fixture children on assertion failure and cover early-exit diagnostics. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> --------- Co-authored-by: heihutu <heihutu@gmail.com> Co-authored-by: zhi22915 <qiuzgang@gmail.com> |
||
|
Chris
|
8c7b3996cb |
feat(scanner): extract OTLP histogram and counter windows (#8862)
* test(scanner): validate OTLP histogram measurement windows * feat(scanner): extract exact integer counter windows * test(ecstore): drain guarded delete fixture rename tails * test(ilm): retain transition timeout evidence |
||
|
Chris
|
7f1795f275 | test(scanner): add recoverable benchmark service control (#8868) | ||
|
Chris
|
b5f275ca3f |
fix(scanner): collect benchmark metrics from realtime route (#8859)
* fix(scanner): collect benchmark metrics from realtime route * docs(scanner): align benchmark help with realtime route * test(e2e): bound parallel seeding for scanner usage checks |
||
|
Chris
|
b39fcb2e85 |
test(heal): cover rc.5 upgrades and refresh ctutils (#8847)
* test(e2e): use the resumed decommission completion budget * test(heal): exercise checkpoints written by rc.5 * test(heal): route upgrade fixtures through the storage facade * fix(deps): update ctutils to address constant-time advisory * test(lifecycle): gate recovery I/O while advancing test time |
||
|
Chris
|
abe14e2f97 |
feat(scanner): configure startup recovery backoff cap (#8845)
* feat(scanner): configure startup recovery backoff cap * test(scanner): compare recovery retry count without narrowing * docs(scanner): document recovery backoff cap and expose it in Helm List scanner.recovery_backoff_cap in the runtime controls table, drop the fixed ten-minute wording from the follower recovery section, and map config.rustfs.scanner.recovery_backoff_cap_secs to RUSTFS_SCANNER_RECOVERY_BACKOFF_CAP_SECS in the chart. |
||
|
Chris
|
623601d2f9 | fix(test): select the current G09 upgrade case (#8841) | ||
|
Hauser
|
44a73b0bef |
fix(s3): preserve typed upload errors across body readers (#8816)
* fix(s3): report payload SHA256 mismatches accurately Distinguish signed payload SHA256 failures from MD5 and additional checksum failures at the shared API error boundary. Preserve the payload error through archive extraction and use typed s3s errors. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * refactor(s3): classify body failures in one bounded pass Share typed body-error classification across direct and storage I/O. Retain marker precedence while replacing repeated source-chain scans, and map storage-wrapped short HashReader bodies to IncompleteBody. Cover nested priorities, real short reads, and one traversal budget. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(s3): route upload stream errors through the app facade Keep the real stream-error fixtures behind the existing test-only S3 boundary without increasing the direct s3s import footprint. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(s3): preserve typed body failures across archive reads Route TAR entry failures through the shared archive error adapter and preserve every classified body error. Keep generic archive parsing as InvalidArgument without formatting discarded transport diagnostics. Use the existing timeout marker for PUT and Snowball body guards. Add regressions for real stream verification after sniffing and timeout responses, and tighten the reduced S3 macro footprint baseline. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> --------- Co-authored-by: heihutu <heihutu@gmail.com> Co-authored-by: zhi22915 <qiuzgang@gmail.com> |
||
|
Chris
|
d2eda195e6 |
fix(decommission): defer source cleanup quorum failures (#8760)
* fix(decommission): defer source cleanup quorum failures A quorum failure while deleting the source of an already copied entry was fatal and marked the whole pool failed, although the copied source is still in place and an uncommitted delete transaction is settled by delete recovery. Carry the cleanup error typed and resolve a quorum failure as Deferred: the entry is replayed at the end of the round, and a deferral that persists takes the existing capacity-blocked pause path instead of a terminal failure. Other cleanup errors and quorum errors from other stages stay terminal. Add set-level regressions for parent and child keys (`a/b`, `a/b/c`): parent cleanup leaves every child file untouched, and parent cleanup reaches quorum while the child is cleaned up or rewritten concurrently. * test(ecstore): keep source cleanup errors typed in prefix cleanup tests Return SourceCleanupError from the test helper instead of formatting it into Error::other, and shrink the error(format!) ratchet baseline for core/pools.rs, which lost one call site in the cleanup failure change. * fix(decommission): retry cleanup quorum failures and count replays once Retry a source cleanup quorum failure in place, up to three attempts with the existing source-cleanup backoff, before deferring the entry. Carry the versions an entry already counted on its deferral, so the end-of-round replay does not add a fully counted entry to items_decommissioned and bytes_done again. A deferral that persists pauses the pool through the existing deferred drain path. When source cleanup caused it, persist a reason led by "source cleanup blocked" without the carrier error's free-drive wording, and report waitingReason "source_cleanup" instead of "capacity". Add a source_cleanup test fault stage and a routine test that drives a cleanup quorum fault through deferral and replay to completion. * fix(decommission): wait seconds between cleanup quorum retries Give the in-place source cleanup quorum retry a fixed floor of one and then two seconds, with the existing jitter on top, so it can ride out a drive reconnect or health-check blip instead of retrying within a few hundred milliseconds. The wait stays bounded and cancel-safe. Find a source cleanup pause reason through stage wrappers, as the capacity-blocked check does, and add a routine test where cleanup keeps losing quorum: the decommission pauses with a source cleanup reason, the bucket stays queued and the source stays in place. Document the additive waitingReason value source_cleanup and its compatibility with older nodes reading the same pool.bin. * fix(decommission): never retry a lost source cleanup fence The cleanup retry classifier used is_quorum_error(), which also matches NamespaceLockQuorumUnavailable. The delete commit fence reports a lost namespace lock or mutation fence with that variant, so a lost fence was retried in place and only stopped later at the pre-cleanup fence check. A lost fence means this worker no longer owns the key; it must stop without retrying. Retry and defer only drive quorum failures: ErasureReadQuorum, ErasureWriteQuorum, InsufficientReadQuorum and InsufficientWriteQuorum. --------- Co-authored-by: Hauser <housemecn@gmail.com> |
||
|
Chris
|
5313b72baa |
fix(decommission): run the leader-side start to completion (#8723)
* fix(decommission): run the leader-side start to completion A forwarded decommission start awaits `ECStore::decommission` inline in the RPC handler. When the forwarding node's RPC times out, the stream is reset and the handler future is dropped wherever it is. Dropped after `pool.bin` records the pool as active but before the peers reload and the worker is spawned, the start left the pool active with no worker and peers on the old metadata, and a retry was rejected as already active. An admin client that disconnects can do the same. Run the start in its own task and let the caller await only its result. The start then always reaches its end: it fails before persisting anything, or persists the pool, has the peers reload and spawns the local workers, rolling back if the reload fails. A retry still cannot start twice: it waits on the start gate held by the running start and either reports the busy gate or is rejected because the pool is active. `decommission` now takes `self: &Arc<Self>` and spawns workers on that store instead of looking up the global object store, which is the same store in production and makes the path testable; the unused `require_decommission_store` helper is removed. Refs rustfs/backlog#2873 * fix(decommission): detach the leader-side start from its caller |
||
|
Chris
|
ea596b2e2d |
fix(decommission): adopt target intents left by a killed leader (#8710)
* fix(decommission): adopt target intents left by a killed leader
A decommission leader killed after persisting a target capacity intent and
before finalizing it left that intent in the exclusive pending slot of the
still-active reservation. The resumed routine derives the same reservation
identity, but every other mutation on that target was rejected as "another
mutation" until the interrupted object happened to be retried on the same
target, so the resumed decommission exhausted its retries and stalled.
At routine start, before any worker runs, the resumed routine now adopts the
reservation's pending intents: under the target fence (or the legacy pool
metadata fence) it moves each one into a mutation-scoped inflight marker that
keeps its identity and bytes. The interrupted mutation's retry settles the
marker through the existing commit, temporary-release and equivalent-target
paths, and an equivalent committed target now settles a mutation's own marker
when no pending intent remains. Adoption is pinned to the persisted
decommission start time and reservation identity, never runs mid-incarnation,
and a failure pauses the routine instead of marking the decommission failed.
* fix(decommission): keep adopted intents within budget and settle them once
Address the adversarial review of the leftover intent adoption.
- Merge an adopted intent into its mutation's staged marker by summing only
while the target's temporary-copy budget allows, never below the larger of
the two. A multipart completion moves its staged parts into the object, so
summing them could exceed the budget and pause every later routine start;
the adopted marker now exceeds the budget only where the completion's own
finalize would have. Reuse resolve and record_inflight for the move.
- Distinguish a temporary release that proved the exact committed target from
one that could not prove the target identity. When the committed mutation no
longer has a pending intent, release only the bytes the cleanup freed and
keep the adopted completion for the equivalent-target reconciliation.
- Let an equivalent target settle a mutation's own marker while another
mutation holds the pending slot, instead of failing on that unrelated intent.
- Cover real leader kills before and after the target commit at routine level
for both lock models, an injected adoption failure that pauses and then
completes, an adopted multipart completion surviving a background abort, and
a live legacy intent. Routine tests reuse the large-stack test helper.
* fix(decommission): pause instead of failing on target slot contention
The S00 leader-kill run with the null-version fix merged ended the resumed
decommission as failed. A re-run of a versioned key reconciled its delete
marker against an already charged target copy while another worker of the
same incarnation held the target's pending slot between a failed write and
its retry. The reconciliation rejected that unrelated intent, the entry
context wrapper flattened the error to a string the intent-conflict retry
could not recognize, and the bucket runner reported a sibling bucket's
OperationCanceled instead of the capacity block, so the routine marked the
decommission failed.
- An equivalent-target reconciliation, including the temporary checkpoint
variant, no longer depends on another mutation's pending intent: a target
holds one pending intent at a time, so this mutation has none to settle.
- The entry context keeps the typed cause, and the capacity-block and
intent-conflict classifiers also recognize the rendered message that an
error clone keeps.
- Concurrent bucket and listing failures keep the root cause over sibling
cancellations, so a capacity block pauses the routine.
* test(decommission): delete marker copy waits for a busy target gate
The Linux run of the S00 matrix showed the delete marker copy meeting a busy
target capacity gate without any leader kill. The copy already waits on that
gate like an ordinary object copy; pin it with an entry-level test that holds
the gate, checks the entry keeps waiting, and expects the key to migrate once
the gate is released. Share the versioned delete marker fixture with the
foreign pending slot test.
* fix(decommission): keep an adopted completion unless the target is absent
Close the remaining path where an adopted completion could lose its record
before being charged. A multipart cleanup that found a target it could not
prove as the exact owned copy still dropped the whole marker of a mutation
without a pending intent, after which the equivalent-target reconciliation
would treat the commit as already charged.
Any cleanup that did not prove the exact target absent now releases only the
bytes it freed when the mutation has no pending intent, and leaves the rest
for the reconciliation. This makes the three-state release unnecessary, so
the release proof goes back to the existing clear-pending flag.
* chore(ci): shrink the error format ratchet baseline for pools.rs
|
||
|
Chris
|
473cddd7dc |
ci: gate pull requests on Clippy and warm its check artifacts (#8708)
Clippy ran only after merge, so lint regressions merged freely and then failed every main run until fixed (10 of 12 main runs on 2026-10-17/18). Run the workspace Clippy step on every event, warm its check-mode artifacts in cache-warm with the identical invocation so pull requests re-check only what they touch, and pin the unconditional step in the ci_gate contract and its self-test. |
||
|
Chris
|
20d9277ec3 |
ci: return pull-request build jobs to 2 after an IO-bound timeout at 3 (#8698)
The first pull-request run at CARGO_BUILD_JOBS=3 that met a contended node (39193827281) hit the 75-minute nextest timeout before any test started. Its sampler shows cargo spinning for 70 minutes with the pod's io.pressure "full" at 86-88% and cpu.pressure at zero, with node-level IO pressure already at 36% when the step began: three concurrent workspace-scale compile units saturate the pod's shared storage, the mechanism the original #5394 mitigation named. Keep 3 on main pushes and dispatches, where three runs succeeded, and 2 on pull requests, in both the nextest and the serial ILM step; pin the ILM budget guard back to that expression and record the evidence in the step comment. |
||
|
Chris
|
90749fbb06 | ci: run feature lanes nightly and warm caches on a schedule (#8683) | ||
|
Chris
|
441b501189 | fix(ci): align the ILM build-budget guard with CARGO_BUILD_JOBS 3 (#8687) | ||
|
Chris
|
f7b2836d7d |
fix(helm): skip data PVCs when storageclass.dataStorageSize is empty (#8681)
An empty storageclass.dataStorageSize used to be the way to run on user-supplied drives (extraVolumes/extraVolumeMounts, e.g. JBOD hostPath), but the chart now always renders data PVC templates, with an invalid empty storage request. When the value is empty, render no data volumeClaimTemplates and no data volumeMounts, drop the data directory preparation from the init container (and the init container itself when there is nothing left to prepare), and omit volumeClaimTemplates when no PVC template remains. Standalone mode skips the data PVC and data volume unless existingClaim.dataClaim is set. Pools may override the value independently. Default renders are byte-identical. Fixes #8665 |
||
|
Chris
|
4d7cd23f8e |
ci: reuse the dev cache in Connect boundary lanes and raise build jobs (#8682)
Raise CARGO_BUILD_JOBS to 3 on every event in the workspace test lane and record the closed backlog#1601 experiment: the test-profile compile took 28m21s at 2 on a pull request and 16-23 min at 3 across three successful main runs. Merge the two Connect boundary jobs into connect-feature-boundaries. The offline enrollment gate no longer exports a fresh CARGO_TARGET_DIR, so it rebuilds only the rustfs crate on top of the restored ci-dev cache; the E2E root is isolated by the feature gate and the dedicated binary name, not by the target directory. The short-credential release guard is now `cargo rustc -p rustfs --lib --features connect-e2e-short-credentials -- -C debug-assertions=off`, which trips the same cfg(not(debug_assertions)) compile_error against cached dev-profile dependencies instead of a cold release check of the whole graph. Every assertion of both lanes is kept. Update ci_gate.py and the ci-gates.md PR matrix for the merged job. |
||
|
Chris
|
ab7001a714 | docs: unify agent instructions across tools (#8675) | ||
|
0x70
|
3a5742c5fd |
feat(policy): support s3:if-none-match and s3:if-match condition keys (#8615)
* feat(policy): support s3:if-none-match and s3:if-match condition keys Bucket, IAM, and STS session policies that enforce conditional writes with s3:if-none-match or s3:if-match were rejected with "invalid key name". Accept both keys and populate them from the If-None-Match and If-Match request headers. A blank header is left out of the condition values because the write ignores it, so it cannot satisfy a Null check for a write that runs unconditionally. * fix(policy): classify conditional write keys as request-derived s3:if-match and s3:if-none-match are filled from request headers, but is_server_derived() reported them as server-derived because their names lack the x-amz- prefix. Add an explicit arm so header input is never refused for these keys if they join COMMON_KEYS. * fix(s3): reject non-wildcard If-None-Match on writes S3 accepts If-None-Match on PutObject, CopyObject and CompleteMultipartUpload only as "*" and answers other values with 501 NotImplemented. RustFS compared such a value as an ETag, so a write with If-None-Match: "<other etag>" overwrote the key while satisfying a policy that requires s3:if-none-match. Reject ETag values in the three handlers before any I/O. The store-level check now treats any non-blank If-None-Match as create-only, so a caller that skips the handler check cannot weaken the precondition. Drop the CompleteMultipartUpload ETag pre-check that the rejection supersedes. * test: tighten conditional write policy coverage Pin trimming and the non-UTF-8 case in the condition values, and drop the auth test that re-ran policy evaluation already covered by the policy crate. Run the three policy e2e scenarios on one server, assert that ETag If-None-Match cannot overwrite under a Null: false policy and that CompleteMultipartUpload with If-None-Match: * returns 412. Share the AssumeRole response parser through common.rs. * test(ecstore): isolate manual delete recovery from workers Stop fixture background services before seeding recovery journals so the explicit recovery pass owns the decisions its assertions inspect. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(s3): enforce request-derived conditional write guards Validate raw If-None-Match headers as well as DTO fields so POST form uploads cannot skip rejection of unsupported ETag preconditions. Derive conditional policy keys after identity claims and clear colliding claims when the actual request has no usable precondition. Align multipart and pinned Ceph regressions with wildcard-only writes, retaining If-Match, object-content, version and delete-marker assertions. Document that 501 is the implementation's unsupported-value response. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * perf(s3): eliminate conditional write request copies Borrow request headers during policy condition collection and exclude conditional keys at the identity-claim boundary instead of inserting and removing them. Preserve existing aliases and multivalue header behavior. Move the CompleteMultipartUpload input out of its request, compare strong ETags by reference, and remove the ETag If-None-Match branch made unreachable by the write guard. Keep both form/HTTP input checks and storage commit preconditions intact. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(s3): align ignored conditional PUT and COPY cases Use If-None-Match: * for the existing-object 412 regressions and assert 501 for matching and nonmatching ETag values. Cover both header-only callers and parsed DTO-plus-header requests in the serial ILM fixtures. Check object bytes and live version counts after each rejected operation. Keep the existing test names and ignored-test lane selection unchanged. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> --------- Co-authored-by: overtrue <anzhengchao@gmail.com> Co-authored-by: Hauser <housemecn@gmail.com> Co-authored-by: heihutu <heihutu@gmail.com> Co-authored-by: zhi22915 <qiuzgang@gmail.com> |
||
|
Henry Guo
|
ee534fa19b |
test(table-catalog): validate current backing contracts (#8559)
* test(table-catalog): qualify current backing contracts * fix(table-catalog): verify smoke cleanup after object rename * fix(ci): include the baseline writer test in full E2E selection * fix(ci): reserve test capacity for disk-backed lock probes --------- Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com> Co-authored-by: overtrue <anzhengchao@gmail.com> |
||
|
唐小鸭
|
d457378c02 |
feat(scanner): decouple bucket walkers from disk count (#8539)
* fix(scanner): treat replication status writebacks as status-class hits A replication status writeback (persist_replication_state_if_current in Update mode, including the heals the scanner queues during its own walk) marked the bucket dirty on the namespace path: it advanced the dirty-usage generation, woke the scanner, reached the durable dirty journal and entered the activity snapshot. Any replicated bucket therefore made every scanner cycle Superseded, the authoritative usage baseline was never certified and usage freshness stayed stale for as long as replication ran (#8375). The writeback is now status class: record_status_class_mutation records only the process-local producer coverage bit and nothing else. There is no rescan queue for it, because a deployment with an active replication rule already walks every bucket on every cycle (ScannerMaintenanceFeatures::needs_regular_cycle) and that walk refreshes replication_stats. The routing lives in a named, unit-tested function in rustfs/src/storage/storage_api.rs; tier-expiration cleanup removes data and stays on the namespace path. No persisted format, protocol, admin API or configuration changes. This is phase 1 of docs/architecture/scanner-per-bucket-convergence.md, which this change adds together with its index entry. Refs #8375, #8494, rustfs/backlog#2816. * fix(scanner): refresh replication statistics during maintenance Refresh compacted prefixes and completed bucket caches during replication maintenance. Bind set execution reuse to the observed replication statistics so a same-cycle retry cannot restore older counters. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(scanner): avoid expanding the legacy S3 DTO surface Decode the existing replication configuration type from a JSON fixture instead of adding direct S3 DTO references to a new file. Preserve the compacted-prefix and same-cycle assertions and the footprint ratchet. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * perf(scanner): keep raw enumeration bookkeeping linear per entry The page writer inserted into its building page with a push-and-sort on every entry and committed each page through a clone. Insert in sorted position and move the committed page into the index instead. Add a regression test that records 20,000 entries in one prefix and pins the cost to one hash per entry and well under a second, so the quadratic re-normalization removed by #8114 (rustfs#8113) cannot return once pacing builds on this walker. * feat(scanner): decouple bucket walkers from disk count Add the scanner key walkers_per_disk (RUSTFS_SCANNER_WALKERS_PER_DISK, 1 through 16, default 1). Each local coordinator disk of a set spawns that many bucket walkers; remote disks keep one. The walker count is what max_concurrent_disk_scans caps and the disk count stays the floor. The walkers of one disk share its scanner admission through LocalDiskAdmissionShare: the first walker to start a bucket takes the per-disk permit and the last to finish releases it, so a peer's remote request still sees one scan session per physical disk. Without the share the second walker was rejected as busy and retired. The disk_concurrency_budget log line reports walkers_per_disk, walkers and the reason for the effective limit. The default stays 1 because no local disk reports rotational media today; the design keeps a higher flash default as the intent. * docs(scanner): document walkers_per_disk in the Helm chart and configuration manuals RUSTFS_SCANNER_WALKERS_PER_DISK was only described in the scanner runtime controls table. Add it to the Helm chart (values, configmap rendering, README table, template test), the configuration crate's environment reference with operator guidance, and a short section in the operations manual on when to raise it and how to confirm it took effect. --------- Co-authored-by: Hauser <housemecn@gmail.com> Co-authored-by: heihutu <heihutu@gmail.com> Co-authored-by: zhi22915 <qiuzgang@gmail.com> |
||
|
Hauser
|
8c8c74cbc3 |
fix(ecstore): unify bounded delete recovery across upgrades (#8594)
* fix(ecstore): complete delete recovery across upgrades and restarts Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(ecstore): expect typed quorum error after delete rollback Assert the exact InsufficientWriteQuorum bucket and object instead of the removed Io wrapper, retaining the rollback and cleanup ownership checks. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> --------- Co-authored-by: heihutu <heihutu@gmail.com> Co-authored-by: zhi22915 <qiuzgang@gmail.com> |
||
|
Henry Guo
|
22c093e66b | feat(table-catalog): add opt-in periodic maintenance queueing (#8537) | ||
|
Hauser
|
c7848df9a0 |
fix(build): enforce GNU ABI floor and canonical target metadata (#8524)
* fix(build): pin gnu glibc baseline to 2.31 and report compile target in version metadata gnu release binaries were built natively on Ubuntu 24.04, embedding GLIBC_2.38/2.39 symbol references while the RPM/DEB metadata promised glibc >= 2.31, so Rocky/RHEL 9 (glibc 2.34) could not run them (#8062). The two linux-gnu legs now build through cargo-zigbuild pinned to the 2.31 baseline (same-arch; zig acts as the linker), with post-build guards asserting both the embedded version metadata and the highest referenced GLIBC symbol version, keeping the package declarations honest. Version metadata itself now comes from cargo's TARGET cfg instead of shadow-rs' host stamping, so builds whose host differs from their target (cross compilation, musl targets on a glibc-hosted toolchain) report the platform they were compiled for (#8048 follow-up). The cross toolchain (zig 0.14.x, cargo-zigbuild 0.23.4) is pinned in the setup action now that it is load-bearing for releases, the build job gets an explicit matrix-independent name so the connect-* acceptance workflows keep matching it when matrix fields change, and the preview-release workflow contract is updated for the TARGET_ARG build commands. Fixes #8062 Follow-up on #8048 * fix(build): enforce GNU ABI floor and canonical target metadata Pin Zig 0.17.0 and cargo-zigbuild, isolate cross-toolchain caches, and build GNU releases against the declared glibc 2.31 floor. Validate both release ELF binaries using their loader version requirements. Generate target metadata once through shadow-rs and check the actual release version output without adding a parallel metadata facade. Refs: rustfs/backlog#2824 Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * perf(build): reuse GNU release profile across binaries Switching the CLI back to the default strip profile restarted 419 compilation units after the server build. Keep the profiling build profile for both binaries, then strip only the CLI artifact. Retain serial links, server profiling symbols, and post-strip ABI gates. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * perf(build): limit release artifacts to the rustfs server Build only rustfs in the release workflow and remove the CLI build, strip, and packaging paths. Preserve GNU profiling symbols and catalog sidecars, and align strict acceptance archive readers with the new set. Exercise all six build targets, Linux package checks, and all four strict acceptance extractors using their actual workflow scripts. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> --------- Co-authored-by: majinghe <42570491+majinghe@users.noreply.github.com> Co-authored-by: heihutu <heihutu@gmail.com> Co-authored-by: zhi22915 <qiuzgang@gmail.com> |
||
|
Hauser
|
dd90bb7488 |
fix(s3): short-circuit conditional reads at metadata quorum (#8459)
* fix(s3): evaluate conditional reads before opening object bodies Reuse locked metadata snapshots for conditional GETs and unify HEAD preconditions and cache headers. Preserve body-cache hooks and validate encryption metadata and key authorization before returning bodyless errors. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * refactor(s3): unify conditional reads at metadata quorum Apply scoped metadata conditions to the existing reader pipeline instead of maintaining parallel prepared-read branches. Preserve quorum checks, early-stop, coalescing, and cache-hook behavior on condition misses. Track metadata-only termination through selection and reject body access or a changed decision. Keep concrete reader futures off request stacks. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(s3): strengthen conditional read acceptance evidence Assert zero shard, decode, cache-fill, and remote-body calls on matching conditions with live positive controls. Exercise GET and HEAD through s3s and Hyper with conditional-header and precedence matrices. Add reproducible paired application-read benchmarks with isolated metadata controls and strict drift, artifact, and resource gates. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * refactor(s3): remove obsolete conditional comparison helper Keep ETag comparison tests on the canonical read-condition evaluator, remove unused date-trait imports, and name the HTTP case type. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(s3): validate full conditional etag fields Reject adjacent tags and malformed trailing list members before accepting a match. Keep the existing request-scoped metadata read and authorization order, while removing redundant validation wrappers and unused benchmark overrides. Replace simulated phase coverage with real 304/412 cache-fill probes and extend HTTP/1.1 and HTTP/2 ingress regressions. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(ecstore): cache terminal conditional metadata safely Preserve quorum-validated conditional metadata when early termination cancels the remaining fanout. Restrict those entries to terminal GET readers; body and full-layout consumers still obtain their own metadata. Serialize cache publication per key so conditional entries cannot replace complete layouts or renew their TTL. Retain generation fencing and the final selected-metadata predicate check before any body access. Cover real four-disk hedging, nonterminal body and range reads, consumer isolation, generation races, atomic publication and actual cache expiry. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(log-analyzer): align heal writer failure rule with current logs Match the current structured all-targets failure, retain legacy log support, and reject partial or unrelated failures. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> --------- Co-authored-by: heihutu <heihutu@gmail.com> Co-authored-by: zhi22915 <qiuzgang@gmail.com> |
||
|
GatewayJ
|
0018bbf5d7 |
ci: guard OIDC architecture boundaries (#8447)
* ci: guard OIDC architecture boundaries * fix(ci): guard OIDC session ownership * fix(ci): ignore OIDC authorization destructuring * fix(ci): accept typed authorization destructuring --------- Co-authored-by: overtrue <anzhengchao@gmail.com> |
||
|
Hauser
|
4166b9503f |
fix(ecstore): recover interrupted delete transactions (#8468)
* fix(ecstore): recover crash-interrupted delete retries Retain set-quorum cleanup proof and restore metadata before dropping rollback markers. Reclaim eligible exact-key residue during explicit delete retries, with bucket identity, version, retention and namespace fencing guards. Preserve prefix children and snapshot ownership through batched physical cleanup, and cover interrupted transactions and failure boundaries. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(ecstore): route recovery test imports through contracts Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(ecstore): preserve missing-key deletes at quorum * feat(ecstore): add certified set-wide delete recovery Atomically prepare deletion journals and exact targets, publish durable set receipts, and recover through one fenced disk/RPC core used by startup and scoped administrator healing. Preserve returning generations, ordinary missing-key quorum semantics, and legacy rollback wire compatibility. Retain namespace and volume ownership through late physical operations, order related transactions causally, and keep proof until full convergence. Remove the superseded local-only recovery path and add storage, protocol, RPC, durability and isolated Docker regression coverage. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(ecstore): reclaim completed multipart delete sidecars * fix(ecstore): recover multipart sidecars with verified proofs Recognize native multipart sidecars at the shared recovery boundary and restore them without treating metadata or proof files as shard payloads. Validate required proof generations and bind every present live/staged shard copy to its metadata-bound Merkle proof before restoring metadata. Cover protected native MPU rollback, split source layouts, missing or corrupt proofs, and payload corruption with recomputed bitrot checksums. Add real multipart Docker regressions in both integrity modes. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(ecstore): settle superseded committed version deletes Allow a validated committed decision to finish its own data cleanup after a later decision removes current metadata. Prepared copies need an inspected same-transaction receipt matching immutable local intent before promotion, and retain their local journal without importing peer hashes. Reuse the authority matcher for dry-run and settlement. Cover chained explicit-version deletions, partial receipt publication, mismatch rejection and no-resurrection, while retaining the no-decision absence protection. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(ecstore): clarify delete recovery case label Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(ecstore): restore recovery CI guard coverage Keep production error sites visible to the ratchet by placing test modules last. Preserve native encoding errors, group preparation identity and ownership, and reduce RPC enum size without changing the wire payload. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(admin): preserve owned OIDC validation errors Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * refactor(ecstore): unify recovery authority and snapshot work Use one user-bucket legacy settlement boundary and one adoption source predicate for preview and execution. Reuse immutable snapshot digests, consume owned generation metadata, index version membership, and prepare empty witnesses through the canonical journal path. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(ecstore): preserve delete recovery invariants --------- Co-authored-by: heihutu <heihutu@gmail.com> Co-authored-by: zhi22915 <qiuzgang@gmail.com> Co-authored-by: overtrue <anzhengchao@gmail.com> |
||
|
Hauser
|
1266d4c270 |
feat(tls): reload and publish external certificate generations (#8467)
* feat(tls): reload external certificate snapshots coherently Pin projected certificate generations, validate complete TLS material, coordinate server consumers, and invalidate outbound RPC caches coherently. Add externally managed TLS Secret support and rotation guidance. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * feat(tls): publish server certificates through a fenced admin API Add opt-in server-leaf publication with dedicated IAM actions, actual TLS transport checks, immutable generations, durable captured-token CAS and idempotent receipts. Preserve independent trust and client identity sources and distinguish visible publication state from the full serving chain. Validate 64 TLS tests, 41 TLS-runtime tests, the focused IAM policy test, and eight live local API scenarios including least privilege and restart. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(admin): route certificate handler types through S3 facade Use the existing admin S3 facade to keep direct s3s imports within the footprint ratchet. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(admin): include TLS certificate routes in route matrix Account for the registered certificate-state GET and certificate publication POST without changing route behavior or weakening matrix assertions. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(tls): reconcile startup trust before skipping reloads Compare the initialized listener bundle with coherently published outbound material, retain startup trust and client identity when sources disappear, and cover independent startup rotations through real reload passes. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * refactor(tls): reuse pinned generations and prepared listener state Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(tls): preserve startup trust source requirements --------- Co-authored-by: heihutu <heihutu@gmail.com> Co-authored-by: zhi22915 <qiuzgang@gmail.com> Co-authored-by: overtrue <anzhengchao@gmail.com> |
||
|
Chris
|
ba2047b184 | ci: reuse workspace test builds for serial ILM checks (#8465) | ||
|
Chris
|
367f13f3f9 | ci: reduce repeated PR runner work (#8463) | ||
|
Hauser
|
0263d5fcb9 |
fix(storage): preserve heal failures and timeout responses (#8462)
* fix(storage): preserve heal failures and timeout responses Stop repairs with no writable target before opening source readers. Consolidate target failure aggregation, preserve typed causes, and distinguish all-ignored diagnostics from actual successful nil entries. Map typed disk timeouts to retryable S3 responses without changing quorum decisions. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * refactor(heal): unify staging failure cleanup Route data-stage failures through one local result boundary after temporary readers and writers are released. Preserve typed report failures, outer reconstruction errors, partial repair, and the existing commit fences. Exercise actual source loss after second-part writer admission using the existing test pause registry. Assert all staged parts are removed, committed metadata stays unchanged, and restoring the source allows a full-byte retry. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> --------- Co-authored-by: zhi22915 <qiuzgang@gmail.com> Co-authored-by: heihutu <heihutu@gmail.com> Co-authored-by: Chris <anzhengchao@gmail.com> |
||
|
Chris
|
1a450bf228 |
ci(e2e): require fresh distributed JUnit reports (#8384)
* ci(e2e): require fresh distributed JUnit reports * test(e2e): emit JUnit from successful workflow mocks |
||
|
Hauser
|
bf47a919ac |
feat(quota): add sharded durable reservation protocol (#8402)
* feat(quota): add sharded durable reservation protocol Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(quota): preserve typed task error context Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(quota): initialize all durable reservation shards Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(quota): persist bootstrap shard eligibility Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * perf(quota): reuse allocator bootstrap state Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(connect): widen inventory startup wait Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(quota): reserve protocol v3 for unknown fixtures Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(quota): reload allocator after concurrent credit refills --------- Co-authored-by: zhi22915 <qiuzgang@gmail.com> Co-authored-by: overtrue <anzhengchao@gmail.com> |
||
|
Henry Guo
|
3b92708011 |
feat(table-catalog): add durable catalog backups (#8266)
* feat(table-catalog): add durable catalog backups * fix(table-catalog): preserve migration lock ordering * fix(table-catalog): preserve backup recovery fences * fix(table-catalog): order backup publication locks * fix(table-catalog): preserve recovery publication fences * test(table-catalog): use the storage facade for recovery fencing * test(ecstore): repair catalog publication fixtures * fix(table-catalog): address catalog backup clippy failures * test(table-catalog): adapt generation races to publication fencing --------- Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com> Co-authored-by: overtrue <anzhengchao@gmail.com> |
||
|
Chris
|
073b4d838e | test(ecstore): verify multi-generation authority model (#8339) | ||
|
GatewayJ
|
051a54f129 |
fix(table-catalog): verify Iceberg metadata path signatures (#8291)
* fix(table-catalog): verify Iceberg metadata path signatures * fix(deps): allow pinned s3s Git source |
||
|
hector
|
e22414fe68 |
ci: coerce the pool timeout input to a number
timeout-minutes requires a numeric value, but inputs.pool_timeout_minutes
is a string, so '${{ inputs.pool_timeout_minutes || 240 }}' evaluated to
the string '240' and the runner rejected the template: 'An error occurred
when attempting to determine the step timeout ... Unexpected value 240'
(chain run 36505625402 job at 02:25:52, standalone run 36393453114 at
07:46:37). Both pool runs since #8155 merged have therefore executed
without the intended step budget. Wrap the expression in fromJSON and
update the policy test that pinned the broken literal.
|
||
|
Chris
|
e77c76272a | docs(storage): verify object generation authority contract (#8322) | ||
|
Chris
|
c0e63ee543 | docs: trim redundant project skills (#8323) | ||
|
Chris
|
8452910895 |
fix: resolve pagination fixtures and heal recovery CI failures (#8316)
* test(e2e): bound delimiter pagination fixture uploads * fix(heal): classify CAS lock failures before publication * fix(heal): retry unproven absence within the object budget * docs(heal): clarify object retry classification * test(s3): seed the delimiter fixture with bounded concurrent PUTs |
||
|
Chris
|
641c4b3493 |
chore(release): merge 1.0.1 back into main and refresh installation references (#8314)
* fix(ci): include pagination regression in full E2E selection * fix(deps): replace yanked yoke-derive release * fix(scanner): expose pause backlog replica diagnostics (#8258) * fix(scanner): expose pause backlog replica diagnostics Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(connect): stabilize runtime profile lease cancellation Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(connect): tolerate delayed schedule startup in CI Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(e2e): retry quota reads during usage warmup Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(ecstore): avoid meta-bucket incarnation self-deadlock Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(test): use persisted incarnation in heal fixture Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> --------- Co-authored-by: zhi22915 <qiuzgang@gmail.com> * fix(usage): reconcile stale counters after lifecycle expiration (#8108) * fix(usage): reconcile stale counters after lifecycle expiration * fix(usage): account lifecycle expiry during continuous writes * test(usage): run lifecycle usage scenarios on one scanner store * test(usage): use a Windows-representable pre-mutation offset * fix(usage): harden expiry accounting recovery and quota checks Borrow expiry receipt bucket names and avoid allocating a map key on cache hits. Cover cancelled receipts, durable snapshot recovery, and legacy quota admission after scanner confirmation. Use representable timestamp offsets in the quota regression. Refs rustfs/backlog#2689 Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(usage): recover stale persisted counts through the scanner Seed incorrect complete usage for empty and retained-object buckets, then run the real scanner and publication consumer without further object mutations. Verify durable and admin usage over two cycles instead of writing a corrected snapshot in the test. Refs rustfs/backlog#2689 Refs rustfs/backlog#2691 Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(e2e): bound delimiter pagination fixture concurrency The 120-second smoke timeout expired after 1018 of 1200 serial fixture PUTs, before LIST ran. Prepare the same objects with at most eight concurrent requests and await every PUT. Retain the timeout and strengthen exact prefix, KeyCount, empty Contents, and continuation-token assertions with phase diagnostics. Refs rustfs/backlog#2689 Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(upgrade): establish a persisted previous-release baseline Seed the pinned previous-release cluster and restart it once with its data intact before replacing any node. Require every old writer to pass the strict readiness probe and preserve the seed through both mixed phases and the final current cluster. Keep InternalError fail-fast behavior and all existing compatibility deadlines and assertions. Refs rustfs/backlog#2689 Refs rustfs/backlog#2384 Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(ecstore): bound cancellation metadata persistence waits Use the system-bucket incarnation boundary now supplied by main PR #8268. Bound the three cancellation waits that previously hung during pool.bin persistence, retaining their remote-generation, target-cohort, and durable-state assertions. Refs rustfs/backlog#2697 Refs rustfs/backlog#2689 Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> --------- Co-authored-by: Chris <anzhengchao@gmail.com> Co-authored-by: Hauser <housemecn@gmail.com> Co-authored-by: heihutu <heihutu@gmail.com> Co-authored-by: zhi22915 <qiuzgang@gmail.com> * test(e2e): retain startup and shutdown failure diagnostics * fix(test): supply CPU workload for sampler regression * fix(ci): locate security chain scripts in the workspace * fix(usage): recover historical counters with generation fencing (#8273) * fix(usage): recover historical counters during continued writes Use newer converged scanner snapshots to reconcile stale absolute usage baselines while preserving concurrent mutation and expiry receipt fences. Cover durable publication, admin and quota reads, and legacy generations. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * refactor(usage): fence snapshots and move preserved cache entries Apply the cached scanner generation floor before every reconciliation path and retain it even when an older snapshot happens to match core counts. Move preserved usage entries instead of cloning their histogram maps under the cache lock, retaining expiry receipt identity and cancellation fences. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> --------- Co-authored-by: heihutu <heihutu@gmail.com> Co-authored-by: zhi22915 <qiuzgang@gmail.com> * test(scanner): verify checkpoint takeover and repair dispatch (#8275) * test(scanner): cover checkpoint handoff and repair dispatch Drive runtime budget expiry, partial-cycle persistence, leadership claims, and stale checkpoint rejection between real disk-backed fixture scans. Verify that a metadata repair beyond the first bounded prefix is saved in the scanner ledger and dispatched by the MRF consumer. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * ci: isolate scanner fixtures and refresh full e2e membership Reserve nextest capacity for the real-disk scanner publication and MRF admission fixtures. Bind both platform membership checks to the reviewed pagination deadline test added on main. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * refactor(scanner): consolidate checkpoint fixture lifecycle Keep one durable control store across timeout and leadership transitions, and inject generation advancement into the shared checkpoint scenario. Check the actual saved metadata path so late-write rejection also proves that existing checkpoint bytes remain intact. Centralize MRF fixture isolation and reuse nextest process isolation when the startup environment already satisfies the test contract. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> --------- Co-authored-by: heihutu <heihutu@gmail.com> Co-authored-by: zhi22915 <qiuzgang@gmail.com> * fix(obs): distinguish allocator counters from live memory (#8274) * fix(obs): distinguish allocator counters from live memory Preserve count/counter semantics and mark requested-byte attribution unavailable when live statistics or sampling are missing. Document sustained multipart memory diagnosis. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * refactor(obs): parse allocator statistics from one node Resolve each statistic before interpreting its shape, avoiding unsupported-field tree scans and mixing data across wrapper scopes. Preserve unavailable-statistics policy and add precedence regressions. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(ecstore): isolate late parity recovery from metadata hedges Use the existing object-scoped hedge timer barrier in exact-count recovery fixtures. Preserve payload and total-read assertions and verify that the omitted parity disk is read only during late refresh. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> --------- Co-authored-by: heihutu <heihutu@gmail.com> Co-authored-by: zhi22915 <qiuzgang@gmail.com> * fix(usage): combine identical snapshot retention branches * fix(test): await HTTP sender readiness in Top RPC fixture * [release/1.0.1] Gate multipart copy through write admission (#8284) Gate multipart copy through write admission Make UploadPartCopy acquire the shared foreground write admission permit before lifecycle locks or source readers so server-side multipart copy cannot bypass the same backpressure used by UploadPart. Document the shared queue semantics and add focused coverage for saturation, cancellation, lock ordering, and disabled admission. Co-authored-by: zhi22915 <qiuzgang@gmail.com> * Gate multipart copy through write admission (#8283) Make UploadPartCopy acquire the shared foreground write admission permit before lifecycle locks or source readers so server-side multipart copy cannot bypass the same backpressure used by UploadPart. Document the shared queue semantics and add focused coverage for saturation, cancellation, lock ordering, and disabled admission. Co-authored-by: zhi22915 <qiuzgang@gmail.com> * fix: add UploadPart OOM validation guardrails (#8287) * docs(release): validate candidates on release branch * fix(scanner): validate checkpoints against global cycle fence (#8278) ## Related Issues Related to rustfs/backlog#2701. ## Summary of Changes Route scanner checkpoint cycle and leader validation through the global store while retaining the owning set for cache persistence, CAS revisions and publication admission. ## Verification Two independent final-diff source reviews found no issues across correctness, concurrency and durability, test coverage, compatibility, performance and simplicity on head `8b8fe51d092090b053f552ae283960e2e306be33`. Root approval `5373624714` is bound to that exact head. Regression tests cover real two-pool routing, stale fences, post-save rejection and CAS conflicts; their reported local execution belongs to the PR author, not this merge operation. Current required CI remains pending, and this authorized admin squash does not establish CI or runtime acceptance. ## Impact Restores checkpoint progress when global cycle and leader state differ from a set-scoped view. No format, retry, timeout, assertion or scanner-policy changes are introduced by this diff. The three prior main scanner failures remain unproved repaired. ## Additional Notes Full validation must run on the resulting exact main revision. Reverting this patch restores the earlier set-scoped fence lookup and its checkpoint rejection behavior. * fix(ci): restore E2E membership and pagination timeouts (#8281) * ci: locate the auto-testing checkout for lanes that run evidence from a subdirectory (#8279) ## Related Issues Follow-up to #8229. ## Summary of Changes Locate the private auto-testing checkout from the lane root or the workspace root so the nested security checkout can record functional-chain evidence. ## Verification The exact PR head |
||
|
Henry Guo
|
5efa6254eb |
feat(table-catalog): expose durable strong diagnostics (#8232)
* feat(table-catalog): expose durable strong diagnostics * fix(table-catalog): stabilize snapshot diagnostics and metadata scans Reuse one recovery observation for exports and diagnostics in both backing modes. Follow the persisted metadata directory for renamed and registered tables, and bound diagnostics under continuous snapshot changes. Remove redundant recovery work and fix the production dead-code and clone lint failures without relaxing checks. Add deterministic race, retry, and metadata reachability coverage and clarify read-only candidate semantics. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(e2e): bound pagination fixture upload concurrency Keep the 1200-object delimiter boundary fixture while preparing it with at most 16 concurrent PUTs. Serial preparation exhausted the existing 120-second smoke deadline before the listing request was issued. Report fixture preparation time and verify the exact common prefixes, empty contents, KeyCount, and absent continuation token without relaxing the timeout, durability settings, or dataset size. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> --------- Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com> Co-authored-by: Hauser <housemecn@gmail.com> Co-authored-by: heihutu <heihutu@gmail.com> Co-authored-by: zhi22915 <qiuzgang@gmail.com> Co-authored-by: GatewayJ <835269233@qq.com> |
||
|
hector
|
1efa86b8a3 |
ci: locate the auto-testing checkout for lanes that run evidence from a subdirectory (#8279)
## Related Issues
Follow-up to #8229.
## Summary of Changes
Locate the private auto-testing checkout from the lane root or the workspace root so the nested security checkout can record functional-chain evidence.
## Verification
The exact PR head
|
||
|
KarlEmm
|
e573fb087b |
ci(helm): sync README to root of helm repository on release (#8263)
Include helm/rustfs/README.md in the uploaded helm-package artifact so publish-helm-package updates the root README.md in rustfs/helm. The build-helm-package job copies helm/README.md into helm/rustfs/ before running helm package, but upload-artifact previously uploaded only helm/rustfs/*.tgz. Because both files share the helm/rustfs/ parent directory, actions/upload-artifact places both the chart archive and README.md at the artifact root, and download-artifact extracts README.md into the root of rustfs/helm alongside the tarball. |
||
|
Chris
|
9e33d54269 | fix(release): package stable preview tags without updating channels (#8256) | ||
|
Chris
|
a88225d208 |
fix(ci): reduce duplicate work and preserve reliable test failures (#8233)
* fix(ci): reduce duplicate work and preserve reliable test failures * fix(ci): retain protocol evidence and repair stale test fixtures * test(connect): honor parent deadline during API fixture readiness * fix(ci): reserve IO capacity for state writer proofs * test(connect): align RPC fixtures with service capture contracts * test(connect): cover pinned service capture failures |
||
|
hector
|
f54945a4c4 |
fix(ci): repair pool budgets and Connect test failures (#8155)
* ci: give the pool run a real wall-clock budget The pool suite budgets up to 24h each for rebalance and decommission (REBALANCE_TIMEOUT / DECOMMISSION_TIMEOUT in rustfs_pool_expand.sh), but the workflow step capped it at 45 minutes. Four consecutive runs died identically: steps 1-7 all PASS, then the rebalance wait was killed at exactly 45:12 - chain 35680308150, standalone 35702803577, chain 35757773373, chain 36283120750 - with rebalance at completed=1/4 (~21 minutes in), so a full pass has never been observed. Make the budget an input (default 240 minutes: covers the observed rebalance pace plus one decommission pass) and document why. The suite keeps failing the job through its [POOL-STEP] marker adjudication. * fix(ci): align pool job budget and timeout contract tests * fix(connect): preserve legacy heartbeats and isolate I/O tests --------- Signed-off-by: Hauser <housemecn@gmail.com> Co-authored-by: overtrue <anzhengchao@gmail.com> Co-authored-by: Hauser <housemecn@gmail.com> Co-authored-by: RustFS <hello@rustfs.com> |
||
|
Chris
|
afb9ddabb4 |
fix(helm): route Gateway API traffic to the chart service (#8145)
Co-authored-by: Hauser <housemecn@gmail.com> |