mirror of
https://github.com/rustfs/rustfs.git
synced 2026-11-01 08:40:23 +00:00
2db4ed244a58f11cd0e4fe94dea3a3f328191166
625 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 overtrue
|
b4abe64ada |
chore(gateway): merge main into integration
Sync main
|
||
|
Hauser
|
e0444b519d |
feat(ecstore): add sampled batch CPU diagnostics (#9019)
* feat(ecstore): add sampled batch CPU diagnostics Keep timing and request correlation in a separate default-off feature. Preserve the existing positioned-read backend, byte and error contracts. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(ecstore): route batch diagnostics through storage facade Keep the diagnostic implementation private to its owner and expose explicit feature-gated helpers through the existing storage boundary. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(ecstore): retain diagnostic IDs across GET producer tasks Capture request context before spawning the legacy producer. Keep the existing producer, cancellation and read-lock lifetime intact. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(ecstore): avoid unused batch read index Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(obs): isolate local batch diagnostic JSON events Omit span metadata and inferred parent IDs for diagnostic root events. Verify per-shard index order and document existing production metric timing. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(ecstore): accelerate receipt fixture in CI Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(heal): allocate stack for bucket recovery fixture Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(s3): allocate stack for conditional GET fixture Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> --------- Co-authored-by: heihutu <heihutu@gmail.com> Co-authored-by: zhi22915 <qiuzgang@gmail.com> |
||
|
cxymds
|
4a63e53161 |
test(validation): run canonical cases and build pinned Heal tools (#8455)
* test(validation): add pinned auto-testing manual lane * Install pinned ILM client for canonical validation lane * Prepare Python client for pool validation cases * Install validation client for replication cases * test(validation): provide the Python client for Heal cases * test(validation): build the candidate-bound Heal inspector artifact * test(validation): expose tool artifacts through the existing manual lane * test(validation): verify pinned Heal tools before fleet execution * test(validation): verify Heal tools against released shard captures * ci: audit retained validation environments before finalization * test(validation): add optional native Heal regression tooling * test(validation): route native tools through the existing driver * Run validation tool resolution on hosted Linux without acquiring the fleet lock * test(validation): bind C07 to pinned native authority evidence * test(validation): build optional C10 fixture codec on hosted Linux * test(ci): bind C10 execution to the verified shared codec * feat(ci): expose read-only validation fleet inventory * docs(ci): describe supported validation recovery * test(ci): add isolated manager native validation builds * test(ci): expose manager regressions through tools-only dispatch * ci(validation): consume pinned manager native prerequisites * fix(heal): preserve typed bucket errors for bounded retries * fix(heal): preserve typed errors and SSE-C response headers * ci(validation): publish isolated immutable candidate packages * test(validation): enforce private candidate manifest schema * test(validation): verify isolated package publication failures * ci(validation): dispatch isolated packages through the driver * fix(heal): deeply verify read repair payload failures * test(ecstore): verify incarnation fence at repair commit * ci(validation): verify C11 native storage evidence * fix(ci): isolate validation package publication credentials * test(ci): archive scoped F07 directory fsync evidence * fix(ecstore): retain typed quorum errors for missing parts * chore(ecstore): shrink typed-error ratchet baseline * fix(ecstore): retain below-quorum data during no-remove heal * fix(ci): allow recovery of completed cancelled validation runs * fix(heal): retain exhausted admin batch repair receipts * fix(observability): retain typed object error diagnostics * fix(s3): map typed RPC unavailability to service refusal * fix(filemeta): classify supported inline framing damage as corrupt * test(ecstore): exercise heal during committed overwrite cleanup * ci(validation): admit source-bound G01 native evidence before fleet execution * test(ecstore): fence version races and bind cleanup slots * ci(validation): require complete race native runtime preflight * test(ecstore): fence marker and batch deletes during heal * ci(validation): require G04 native and runtime admission * test(ecstore): fence multipart continue complete and abort during heal * test(ecstore): cover default heal owners and copy metadata races * ci(validation): admit source-bound owner and copy race proofs * test(scanner): assert typed corruption after metadata truncation * fix(ci): require complete native admission for multipart races * test(validation): wire I05 queued-successor native prerequisites * fix(scanner): bind object heals to the scanning disk scope * test(scanner): reuse storage boundary for scope regression * style: format RPC digest precedence assertion * fix(heal): query automatic tasks on their owning node * test(admin): reuse error facade in heal query regression * fix(heal): retain cancellation after successful bucket storage I/O * fix(ci): retain validation evidence outside runner temp * fix(heal): retain resolved object identity on terminal failures * test(scanner): adapt scope regression to admission receipts * ci(validation): inspect shared runner without fleet access * ci(validation): prepare Python before creating client environments * ci(validation): isolate each client virtual environment * ci(validation): retain interrupted evidence without fleet recovery * ci(validation): bind recovery to retained runner evidence * ci(validation): wire persistence and lock source plans * fix(validation): align read-only dispatch defaults --------- Co-authored-by: Chris <anzhengchao@gmail.com> |
||
|
overtrue
|
add363f084 |
chore(gateway): merge main into integration
Sync main
|
||
|
Hauser
|
a9d3022f37 |
fix(ecstore): prove multipart part sizes before cleanup (#9037)
* fix(ecstore): prove multipart part sizes before cleanup Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * refactor(ecstore): merge strict multipart observation passes Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test: respect lifecycle architecture boundaries Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test: fix multipart E2E clippy warnings Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> --------- Co-authored-by: heihutu <heihutu@gmail.com> Co-authored-by: zhi22915 <qiuzgang@gmail.com> |
||
|
RustFS
|
7581a70412 |
feat(s3): serve Veeam SOSAPI system and capacity XML (#9007)
* feat(s3): serve Veeam SOSAPI system and capacity XML Authenticated GET and HEAD of the two well-known SOSAPI keys return generated XML, and read preconditions use that generated ETag and Last-Modified. Mutations of those keys are rejected while the feature is on. Co-authored-by: RustFS <hello@rustfs.com> * fix(s3): simplify Veeam quota metadata lookup Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(s3): harden Veeam SOSAPI capacity and reads Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(s3): bind Veeam capacity to bucket identity Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(s3): protect virtual objects and coalesce refreshes Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(e2e): refresh Veeam profile selections Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(e2e): bound pagination fixture writes Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> --------- Co-authored-by: Hauser <housemecn@gmail.com> Co-authored-by: heihutu <heihutu@gmail.com> Co-authored-by: zhi22915 <qiuzgang@gmail.com> |
||
|
Chris An
|
f6ad79dd58 |
feat(plugins): add minimal original-user runner integration (#9017)
* feat(plugins): add minimal original-user runner integration * test(plugins): use the main storage fixture * fix(plugins): reuse runner connections across requests * test(plugins): allocate stack for delegation fixture |
||
|
Chris An
|
7a3bf8cdbd |
fix(ecstore): resolve dangling latest versions in metadata updates (#9009)
* fix(ecstore): select the latest version at read quorum like MinIO GET and HEAD escalated latest-version selection to write quorum when enough metadata answered and skipped a newer version below write quorum on degraded reads. LIST and heal already pick the newer side of a split write at read quorum, so such keys listed but answered 503 until heal. Select with MinIO's getObjectFileInfo rule: the most common mod time that reaches the object's data blocks wins, ties go to the newer time, and one content identity at that time must reach the same quorum. Two identities tied at the selected time fail instead of resolving by hash order, and the selected identity's version-stack shape follows disk order. * fix(ecstore): resolve selection ties by disk order and guard writes Review of the read-quorum selection found three gaps: - Rejecting a same-time half split made explicit-version reads of a partially applied tagging update fail and blocked the retry that repairs it. Resolve ties to the identity on the lowest-index disk, for reads and heal alike, instead of rejecting or using map order. - Delete markers and empty versions can lower the common read quorum below the selected version's data blocks, so HEAD could report a version GET cannot decode. Require that many copies, as MinIO's getLatestFileInfo does. - A selection can now return exactly read-quorum disks, and partial metadata writes are not rolled back. Refuse an update before writing when its target disks cannot reach write quorum. * fix(ecstore): count identity copies in the data-blocks guard The first-candidate shortcut counts byte-identical copies, so copies that differ only in identity-neutral fields such as replication status were missing from the data-blocks check. When the shortcut's count falls short, count every copy of the selected content identity before refusing the version. * fix(ecstore): keep commit-proof selection for system-bucket reads Pool metadata's PREPARE/COMMIT recovery and identity epochs treat an ordinary read of pool.bin as proof that the record committed. Under the read-quorum rule, a COMMIT that reached only half of a 2+2 set was served as committed, and the pool_meta_native_physical_* integration tests failed. Route reads through select_fileinfo_for_read: S3 buckets keep MinIO's read-quorum rule, while is_meta_bucketname reads use the previous algorithm unchanged as select_committed_fileinfo, so a split stays unreadable until native recovery resolves it. * fix(ecstore): serve the latest readable version past dangling versions A failed versioned PUT whose rollback also failed can leave the new version on top of fewer drives than its data blocks. Reading only each drive's top version then left the committed version below it short of read quorum, so GET and HEAD of the latest version answered 503 although that version is intact in every stack. MinIO resolves the stacks in pickLatestQuorumFilesInfo. When every drive answered and an S3 latest read still cannot select a version, walk the newest version headers of each stack through exact-version reads: serve the readable version with the newest selected modification time, skip versions whose exact read proves them dangling (the NoSuchVersion an explicit GET reports) and queue heal for them, and keep the read-quorum error on any other outcome. Candidates are ordered by each version's newest modification time across drives, and at most eight exact reads are spent. Test PUTs now wait for every drive so tests that edit one drive's metadata no longer race the PUT's background tail writes. * fix(ecstore): resolve dangling latest versions in metadata updates A failed versioned PUT whose rollback also failed can leave the new version on fewer drives than its data blocks. GET and HEAD of the latest version already look past such versions, but read-before-write callers without a version id still answered 503: object tagging read each drive's top version with the quorum-error read, and put_object_metadata and the CopyObject source read selected from read_all_xl, whose merged latest version is missing on the other drives, so object_quorum_from_meta failed before selection. Object tagging now reads through a variant that runs the same walk past dangling versions as a latest GET while keeping the quorum error for a dangling key. put_object_metadata and the CopyObject source try the walk when their quorum or selection step fails with read quorum, before the existing dangling cleanup could report a key with a committed version absent. A version-only copy takes the served version's own copies and write quorum. The walk still applies only to S3 buckets, latest reads, and sets where every drive answered, and the update still targets only the served version's disks, so a version below write quorum is refused before writing. * fix(ecstore): refuse version-only copies below write quorum up front Review of the dangling-latest metadata updates found three gaps: - A version-only CopyObject wrote the new version and reverted on a write-quorum miss; refuse it before writing when the served version's disks cannot reach its write quorum, like update_object_meta_with_opts. - Replication status write-back, including the zero-write ValidateOnly mode, now keeps its read-quorum error and never walks past dangling versions, since the walk queues heal for the versions it skips. - The erasure-coding doc overstated that these callers never report a dangling key as absent; put_object_metadata and CopyObject keep their existing dangling cleanup when the walk finds no readable version. |
||
|
Chris An
|
f879946ead | chore: remove redundant review skills (#9022) | ||
|
Chris An
|
d88f524d9f |
fix(ecstore): serve the latest readable version past dangling versions (#9004)
* fix(ecstore): select the latest version at read quorum like MinIO GET and HEAD escalated latest-version selection to write quorum when enough metadata answered and skipped a newer version below write quorum on degraded reads. LIST and heal already pick the newer side of a split write at read quorum, so such keys listed but answered 503 until heal. Select with MinIO's getObjectFileInfo rule: the most common mod time that reaches the object's data blocks wins, ties go to the newer time, and one content identity at that time must reach the same quorum. Two identities tied at the selected time fail instead of resolving by hash order, and the selected identity's version-stack shape follows disk order. * fix(ecstore): resolve selection ties by disk order and guard writes Review of the read-quorum selection found three gaps: - Rejecting a same-time half split made explicit-version reads of a partially applied tagging update fail and blocked the retry that repairs it. Resolve ties to the identity on the lowest-index disk, for reads and heal alike, instead of rejecting or using map order. - Delete markers and empty versions can lower the common read quorum below the selected version's data blocks, so HEAD could report a version GET cannot decode. Require that many copies, as MinIO's getLatestFileInfo does. - A selection can now return exactly read-quorum disks, and partial metadata writes are not rolled back. Refuse an update before writing when its target disks cannot reach write quorum. * fix(ecstore): count identity copies in the data-blocks guard The first-candidate shortcut counts byte-identical copies, so copies that differ only in identity-neutral fields such as replication status were missing from the data-blocks check. When the shortcut's count falls short, count every copy of the selected content identity before refusing the version. * fix(ecstore): keep commit-proof selection for system-bucket reads Pool metadata's PREPARE/COMMIT recovery and identity epochs treat an ordinary read of pool.bin as proof that the record committed. Under the read-quorum rule, a COMMIT that reached only half of a 2+2 set was served as committed, and the pool_meta_native_physical_* integration tests failed. Route reads through select_fileinfo_for_read: S3 buckets keep MinIO's read-quorum rule, while is_meta_bucketname reads use the previous algorithm unchanged as select_committed_fileinfo, so a split stays unreadable until native recovery resolves it. * fix(ecstore): serve the latest readable version past dangling versions A failed versioned PUT whose rollback also failed can leave the new version on top of fewer drives than its data blocks. Reading only each drive's top version then left the committed version below it short of read quorum, so GET and HEAD of the latest version answered 503 although that version is intact in every stack. MinIO resolves the stacks in pickLatestQuorumFilesInfo. When every drive answered and an S3 latest read still cannot select a version, walk the newest version headers of each stack through exact-version reads: serve the readable version with the newest selected modification time, skip versions whose exact read proves them dangling (the NoSuchVersion an explicit GET reports) and queue heal for them, and keep the read-quorum error on any other outcome. Candidates are ordered by each version's newest modification time across drives, and at most eight exact reads are spent. Test PUTs now wait for every drive so tests that edit one drive's metadata no longer race the PUT's background tail writes. * chore(ci): shrink the error(format!) ratchet baseline for rebalance The rebalance control and meta modules now have fewer Error::other(format!) call sites than the baseline records, which fails the shrink-only ratchet in Quick Checks. Record the smaller counts. * fix(ecstore): preserve canonical order in latest-version fallback |
||
|
Chris An
|
f96c94f950 |
fix(listing): treat key markers as literal keys (#8978)
* fix(listing): treat version markers as literal keys * fix(listing): keep start-after and V1 markers literal * docs(listing): state cursor compatibility removal condition * fix(listing): resume protected table pages from raw keys * test(listing): keep protected pagination coverage in app layer |
||
|
Chris An
|
acebee4ea3 |
fix(ecstore): select the latest version at read quorum like MinIO (#8982)
* fix(ecstore): select the latest version at read quorum like MinIO GET and HEAD escalated latest-version selection to write quorum when enough metadata answered and skipped a newer version below write quorum on degraded reads. LIST and heal already pick the newer side of a split write at read quorum, so such keys listed but answered 503 until heal. Select with MinIO's getObjectFileInfo rule: the most common mod time that reaches the object's data blocks wins, ties go to the newer time, and one content identity at that time must reach the same quorum. Two identities tied at the selected time fail instead of resolving by hash order, and the selected identity's version-stack shape follows disk order. * fix(ecstore): resolve selection ties by disk order and guard writes Review of the read-quorum selection found three gaps: - Rejecting a same-time half split made explicit-version reads of a partially applied tagging update fail and blocked the retry that repairs it. Resolve ties to the identity on the lowest-index disk, for reads and heal alike, instead of rejecting or using map order. - Delete markers and empty versions can lower the common read quorum below the selected version's data blocks, so HEAD could report a version GET cannot decode. Require that many copies, as MinIO's getLatestFileInfo does. - A selection can now return exactly read-quorum disks, and partial metadata writes are not rolled back. Refuse an update before writing when its target disks cannot reach write quorum. * fix(ecstore): count identity copies in the data-blocks guard The first-candidate shortcut counts byte-identical copies, so copies that differ only in identity-neutral fields such as replication status were missing from the data-blocks check. When the shortcut's count falls short, count every copy of the selected content identity before refusing the version. * fix(ecstore): keep commit-proof selection for system-bucket reads Pool metadata's PREPARE/COMMIT recovery and identity epochs treat an ordinary read of pool.bin as proof that the record committed. Under the read-quorum rule, a COMMIT that reached only half of a 2+2 set was served as committed, and the pool_meta_native_physical_* integration tests failed. Route reads through select_fileinfo_for_read: S3 buckets keep MinIO's read-quorum rule, while is_meta_bucketname reads use the previous algorithm unchanged as select_committed_fileinfo, so a split stays unreadable until native recovery resolves it. |
||
|
Chris An
|
e452d837dd | docs(scanner): clarify incomplete census diagnostics (#9002) | ||
|
Hauser
|
cf5a9e0ba3 |
docs: define the dirty-journal metadata source contract (#8984)
* docs: define the dirty-journal metadata source contract Specify the single node-owned journal scope: verified bootstrap, physical fencing and tail completion, reader-first replay, exact new responsibility settlement and retained legacy unknowns. Explicitly distinguish the proposed contract from the existing asynchronous hint implementation. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * docs: require receiving peers to revalidate source claims Keep user metadata, decoded records and authenticated transport distinct from storage-verified dirty-journal authority. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> --------- Co-authored-by: heihutu <heihutu@gmail.com> Co-authored-by: zhi22915 <qiuzgang@gmail.com> |
||
|
overtrue
|
67fbafa13e |
chore(gateway): merge main into integration
Source: main |
||
|
Hauser
|
d4d416ed47 |
fix(deps): upgrade russh and replace vulnerable zstd codecs (#8969)
* chore(deps): upgrade russh to 0.65.0 and refresh crates Run two cargo update and cargo upgrade rounds from origin/main. Adapt SFTP public-key callbacks to PublicKeyOffer and VerifiedPublicKey, preserving explicit rejection and password-only authentication. Cover the rejection policy with a pinned-host-key SSH handshake test. Refresh Arrow, Parquet, Brotli, ZIP, and compatible dependencies. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(deps): route legacy zstd consumers to fixed codecs Replace the old registry implementation with a private version-constraint adapter backed only by zstd 0.14.3 and zstd-safe 8.2.0. Preserve legacy features and Avro, Arrow IPC, Parquet, and Kafka formats. Cover abandoned contexts, legacy decode errors, seekable bounds, stored frames, consumer interoperability, and resolved implementation versions. Record the adapter removal condition in the compatibility register. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> --------- Co-authored-by: heihutu <heihutu@gmail.com> Co-authored-by: zhi22915 <qiuzgang@gmail.com> |
||
|
Chris An
|
fca714efe8 |
fix(listing): omit sub-quorum versions from ListObjectVersions (#8964)
Version listings bypassed listing quorum: SetDisks::list_path returned early into list_versions_authoritatively, whose resolver seeds unread slots as DiskNotFound. With any one drive of the set offline, every version held by fewer than ceil(n/2) drives (for example a deleted version kept by drives that were offline during the delete) failed the whole page with ErasureReadQuorum, surfaced as S3 503. Route merged version pages through the listing-quorum branch of SetDisks::list_path, as MinIO's listPath does with requestedVersions = 0, so such versions are omitted instead of failing the page. Keep the set-level floor: fewer than ceil(n/2) online drives still fails the page. Set walks (tier reference proofs, tier-delete journals, pool maintenance) and the recursive prefix-delete scan act on absence, so they keep the authoritative per-version scan and fail closed. |
||
|
Chris An
|
65bd2d575b |
refactor(listing): make cursors plain keys and opaque V2 tokens (#8950)
* refactor(listing): make cursors plain keys and opaque V2 tokens RustFS inherited MinIO's metacache cursor tag and appended `[rustfs_cache:v2,id:<uuid>,src:walker,gen:live]` to every internal continuation marker, although nothing ever consumed the id: four parsers had drifted apart, V1 NextMarker had to be stripped on the way out, and ListObjectVersions leaked the tag onto the wire as NextKeyMarker. ecstore stops assigning `list_id` in `ECStore::list_path` and `SetDisks::list_path_result`, drops `append_list_cache_id_to_marker` and the `cache_id` parameters of the pagination helpers, and removes the unused `ListPathOptions::encode_marker`; internal markers are the plain last key, last common prefix or last raw scanned key. The API layer builds the V2 NextContinuationToken as base64 of the server-side cursor in one place and decodes it in one place; V1 NextMarker is the decoded cursor and the `public_list_marker` tag stripper is gone; NextKeyMarker is the plain key. Base64 of the plain key is what nodes from earlier releases already read, so a rolling upgrade needs no switch. Compatibility, registered as `listing-cursor-legacy-tag`: `parse_marker` and `list_marker_key` still strip a trailing legacy tag from incoming markers so a client that paginates across the upgrade keeps its position. * fix(listing): preserve literal cache-tag cursor boundaries |
||
|
Hauser
|
fea5a584c3 |
fix(storage): fence multipart and metadata recovery (#8919)
* perf(ilm): back off idle tier-delete journal recovery Use bounded idle backoff only after complete empty journal and dispatch scans, while preserving immediate notifications and the retry cadence for observed work. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(ecstore): reconcile interrupted pool metadata publication Bind pool metadata CAS and Undo to exact native predecessors. Recover an interrupted physical publication only from an ordinary committed authority and complete verified identity, current and backup evidence, then require normal V3 CAS and quorum readback. Preserve known-conflict gates, startup and heal lock ordering, targeted admission and the original kill-recovery deadline. Wake proof-ready recovery immediately and bound parallel capacity sampling without removing mutation fences. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * perf(ecstore): reuse proven pool metadata durability Publish native rollback backups directly in their owned UUID directories. Reuse a one-shot data-durability receipt only for an exact hardlink to the pinned, fully synchronized head and root. Preserve new-head and directory synchronization, native predecessor CAS, conditional Undo and uncertainty gates. Keep unknown, copied, raw-parent, cancelled and failed operations cold. Add bounded diagnostics, sync-call regressions and path-isolated test hooks. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(multipart): exercise selective RPC delay and peer loss Add authenticated method-entry barriers for metadata and lifecycle locks under e2e-test-hooks. Verify real dispatch, write quorum, prior part payloads, lock drain and incarnation isolation without changing production budgets. Record cancellation separately from normal exits so late filesystem completion cannot overwrite Drop evidence. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * docs: define multipart delay gates and receipt invariants Document RPC entry evidence, peer-loss and cancellation oracles, remaining incident attribution limits, and immutable head inodes required for durability receipts. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(readiness): expose missing runtime drive slots Report configured slots without a runtime handle separately from unavailable inventoried drives. Preserve quorum and probe status semantics. Wait for every coordinator to regain its complete live inventory before multipart RPC fault windows and single-attempt recovery oracles. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(ecstore): retry committed promotion convergence safely Require peer reload convergence on every worker attempt, including unchanged retries. Recheck durable identity, replicas, namespace fence, generation and cancellation before allowing any data movement. Keep known committed promotions active on a verified propagation pause. Release precise owners, retain queued successors and reuse the existing watchdog. Preserve uncertainty gates and terminal operation behavior. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(ecstore): guard native metadata mutation aliases Reject protected RenamePart source and destination paths before I/O. Classify the protected namespace through the native path resolver, and require canonical conditional RenameData and Undo keys so aliases cannot create separate lease domains. Preserve ordinary multipart and canonical CAS behavior, warm durability receipts and zero-allocation canonical fast paths. Cover real alias, byte identity, receipt and normal mutation behavior. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(e2e): register Darwin multipart RPC delay gates Update distributed membership from the actual Darwin listing. Preserve the full-profile digest and leave Linux unchanged until its own listing is produced, as required by the contributor guide. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(multipart): fence native publication and recovery Publish multipart data through private hardlinks and atomically consume the prepared source head. Bind exact compensation capabilities and native recovery to the physical cohort, source generation and durable publication state. Require fresh terminal confirmation for native replay and preserve indeterminate data when no safe recovery witness exists. Release logical guards only after validated irreversible quorum success while retaining physical mutation leases, accounting and resource ownership through actual callbacks. Preserve local instance proof authority and historical Copy metadata without granting healed donors fresh publication authority. Keep general movement identity strict while preserving directory capacity compatibility in a separate checked comparison. Cover native fault and cancellation boundaries, zero-byte objects, real Copy, heal and migration, exact physical readback and distributed RPC delay. Retain all quorum and durability requirements and document rollout limitations. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(ecstore): fix ratchet scan and Windows fault hooks Move native pool metadata inventory tests after production items so the error-format ratchet sees the existing call site. Preserve the baseline instead of accepting a false count decrease. Make the directory-barrier failure hook available on every test platform and exercise the parent-barrier failure/retry cases in Windows CI. Keep production synchronization behavior and all existing assertions intact. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(protos): remove redundant recovery request clone Move the baseline into its final struct update while retaining the earlier clone needed to construct the first request variant. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(e2e): align RPC delay gates with native abort * fix(storage): validate native source recovery and CI gates Preserve wire layouts while bounding native response sizes and fix the last-use recovery request clone. Avoid destination replay reads only for fully observed initial native sources; keep upgraded and recovered sources conservative. Cover real legacy publication after a cold restart. Align upgrade acceptance with the native mixed-version mutation pause and use exact upload-scoped test completion evidence before advancing time. Keep all original quorum, ownership, byte and deadline assertions. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(e2e): verify native abort under partial admission Delay native cleanup at the exact upload root after all-participant closure. Verify quorum-only admissions retain every source byte on refused Abort and complete the same upload without creating a missing original participant. Keep sub-quorum orphans and their discoverable identities isolated while a fresh all-participant upload proves service recovery. Route shared constants through the existing E2E storage facade. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(e2e): avoid formatting a fixed invalid upload path Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(e2e): align native MPU fault recovery assertions Verify real source participants, preserved orphan bytes and same-upload completion after quorum-only admission. Require full physical inventories so an expected read-quorum error cannot hide unrelated leakage. Preserve ordinary quorum and recovery budgets, and document listing availability through the affected set. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(ecstore): keep MPU tail deadlines on the wall clock Global Tokio time advancement can expire drive-health probes while real filesystem callbacks are pending. Use a short real drive deadline with the existing physical-entry and cleanup barriers, retaining the byte, lock, resource-permit and final-shard assertions. Fixes #8937 Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(storage): align native fixtures with instance ownership Open independent disks for the simulated capacity-loss instance, scope four coordinator journal disks without rebinding source authority, and probe configured namespace keys while retaining physical I/O hooks. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(ecstore): isolate native multipart future construction Construct optional native publication futures behind synchronous heap boundaries so ordinary rename callers do not carry their state machines. Preserve native guards and ownership without adding allocations to ordinary PUT, copy or heal branches. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(heal): assert coordinator journal scope after rejoin Check the exact original four-member journal map after every rejoin and manager stop while preserving all sixteen native data slots. Reconnect expansion fails the test rather than being filtered away. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(heal): bound object repair future construction Construct the normal and FileCorrupt deep-retry state machines behind a synchronous heap boundary. Preserve guards and cancellation while bounding the shared rename and deletion call stacks without increasing test stacks. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(ecstore): qualify fixture disk identity checks Call the same DiskAPI method explicitly so the independently reopened disk identity assertion compiles under every target and feature combination. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(ecstore): isolate native inspection from mutation state Select a dedicated read-only inspection future behind the existing single Box allocation. Preserve validation, source ownership and the canonical inspection body while keeping mutation state out of inline read callers. Reject misrouted operations with typed errors and retain source evidence. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(ecstore): pause abort time after native cleanup Wait for durable source deletion on the wall clock before advancing the capacity lease clock at the post-delete barrier. Paused time can otherwise expire the barrier while native filesystem cleanup is still pending. Retain lease-loss injection, durable inflight-byte accounting and restart reconciliation assertions. Document the test-util feature required to select the native decommission fixtures in package-scoped regressions. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(ecstore): reclaim data from owned multipart undo Retain exact runtime-only Undo ownership through deferred snapshot cleanup. Revalidate every target metadata version and the retirement anchor, keep the delete claim through the physical callback, and remove the anchor after the payload barrier. Preserve generic cleanup refusal when metadata is missing. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(ecstore): preserve unformatted bootstrap routes Keep explicitly unformatted local paths reachable by an in-flight elected formatter while quarantining their service slots and disk-ID membership. Own bounded legacy-read handles so the actual asynchronous creator future remains Send. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(s3): bound CopyObject tracing poll frames Construct the business future behind an outlined single-Box boundary and lazily instrument its handle. Preserve Send, Unpin, span timing and cancellation without embedding concrete business-state construction in the tracing poll. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(mpu): observe recovery after direct legacy bootstrap Seed all sixteen real previous-release sources before attaching the same-port method observers. Require exact per-peer disk-route Unimplemented responses with zero publication calls, retain byte and cleanup assertions, and remove superseded raw transport. Register the two new regressions from actual Linux and macOS listings. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(mpu): prepare the homogeneous previous-release fleet Reuse the existing release baseline before creating the genuine legacy multipart sources. Restart old processes with their normal budgets and require every drive through the reduced-redundancy probes before installing the mixed-version observers and current fault budgets. Keep standard EC:4, single-attempt calls, all mixed-version byte and reclamation assertions, and their original deadlines unchanged. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(mpu): isolate release compatibility from fault profiles Keep normal component configuration for the observed mixed-release rollout, which injects no RPC delay or loss. Retain the short component profile in actual method-fault scenarios and preserve the single-attempt 45-second probe and every typed error, routing, source byte and reclamation assertion. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(mpu): use normal shutdown for planned release replacements Use the existing bounded SIGTERM path during homogeneous release preparation and the observed mixed-version rollout. Preserve every multipart inventory, routing, quorum and byte assertion, and keep actual crash recovery on its original hard-kill path. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * revert: retain the verified release compatibility fixture Restore the published fixture after the local configuration and lifecycle experiments failed to resolve the macOS startup failures. Keep their diagnostic records separate from the delivered implementation. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(copy): bound inherent handler polling stack Build the inherent CopyObject future through a synchronous noinline factory at the existing trait await. This keeps its large state out of the trait wrapper's active polling frame during historical self-copy. Preserve cancellation, tracing and post-write notifications, with one additional allocation restricted to COPY. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(e2e): validate authoritative quota and native pool heals Require actual authoritative quota usage in the existing readiness loop instead of treating HTTP success or unknown usage as a numeric baseline. Keep the original quota assertions and convergence budgets. Allow the owning pool metadata CAS to republish into a fresh valid data directory while retaining exact version, layout, completeness and shard fingerprints. Require every node's real metadata/read/write readiness before accepting recovery. Keep ordinary object manifests and the original 180-second recovery deadline unchanged, with negative oracle regressions. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(e2e): distinguish certified native Abort recovery Require native participants with a persisted full closure certificate to retire their heads and reclaim every surviving source on retry. Keep uncertified legacy authority below identity quorum conservative, while retaining actual partial-delete and committed-object byte checks. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> --------- Co-authored-by: heihutu <heihutu@gmail.com> Co-authored-by: zhi22915 <qiuzgang@gmail.com> Co-authored-by: overtrue <anzhengchao@gmail.com> |
||
|
Hauser
|
c610d11e8f |
fix(storage): guard MPU recovery and startup RPC admission (#8934)
* fix(ecstore): preserve uncertainty after partial multipart abort Keep recovery callers from reporting NoSuchUpload while minority primary metadata remains. Retain all quorum and deletion checks. Exercise actual partial Abort and explicitly retained late RPC packets after caller retirement, staging cleanup and bucket recreation. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(ecstore): prevent retired multipart staging resurrection Require the upload root to exist before preparing recovery publication. Preserve rooted directory walks for nested bases on non-Linux Unix and cover missing-source native work, configured aliases and pinned mixed-version coordinator recovery. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(e2e): use IPv4 peer aliases for pinned mixed recovery Keep direct IPv6 S3 listeners behind IPv4 RPC proxies for the pinned previous release, whose peer health resolver rejects bracketed IPv6 hosts. Preserve signed authorities, ports and strict recovery assertions. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(e2e): separate pinned startup from MPU fault budgets Use the old fleet defaults during bootstrap, then apply the original short fault budgets to the current replacement. Keep direct S3 URLs aligned with IPv6 listeners and preserve all quorum and physical cleanup assertions. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(e2e): seed legacy MPU through transparent transport Retire raw proxy connections before enabling method faults so legacy streaming setup stays byte-transparent and every later scoped mutation must traverse the fault barrier. Verify raw and adapted protocol authority, bodies and trailers. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(e2e): retain direct listeners for legacy MPU peers Advertise same-port localhost peers for the pinned resolver while retaining direct IPv4 S3 listeners. Exact scoped RPC hit counts still reject bypassed injection and all physical authority assertions remain unchanged. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(server): keep startup gRPC on the authenticated router Use the hybrid protocol predicate for startup RPC admission so early control responses retain production authentication and owner validation. Keep REST FullReady gating and pinned replay capability rejection intact. Add deterministic regressions, scoped diagnostics and bounded legacy fixture resources. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(ci): verify MPU recovery across upgrade source versions Run the exact sixteen-drive mixed MPU case for every checksum-verified upgrade source, independently of generic compatibility failures. Require one listed test and retain per-version logs while preserving the historical distributed source pin and every recovery assertion. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(ci): trigger upgrade MPU checks for fixture changes Include distributed fixtures and the RPC fault proxy in the upgrade workflow paths so test-only changes schedule the strict recovery gate. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(storage): verify resend owners across MPU finalization Hold real part-transaction RPCs while another coordinator completes or aborts the upload, with both live and cancelled client response waiters. Require full object bytes, terminal upload state, and all staging roots to remain consistent after the part owner drains. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> --------- Co-authored-by: heihutu <heihutu@gmail.com> Co-authored-by: zhi22915 <qiuzgang@gmail.com> |
||
|
Hauser
|
299609bb83 |
fix(quota): keep reads available without snapshot-wide copies (#8928)
* fix(quota): keep quota reads available during scans Use optional scanner usage for read-only quota statistics across all quota configurations. Preserve strict legacy write admission and durable reservation enforcement while keeping unknown usage distinct from zero. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * perf(usage): avoid snapshot copies for single-bucket reads Project owned snapshots, scalar bucket sizes and refresh-only results through the same TTL-bounded cache loader. Move loaded snapshots into the cache without changing refresh serialization or generation fencing. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> --------- Co-authored-by: heihutu <heihutu@gmail.com> Co-authored-by: zhi22915 <qiuzgang@gmail.com> |
||
|
Hauser
|
017b85a5bf |
fix(ecstore): preserve inactive multipart abort authority (#8920)
Retain per-disk recovery metadata through partial publication and require write-quorum proof for completion replay. Keep completed uploads terminal and strengthen RPC recovery acceptance. Co-authored-by: heihutu <heihutu@gmail.com> Co-authored-by: zhi22915 <qiuzgang@gmail.com> |
||
|
overtrue
|
8cf6187ff4 | chore(gateway): merge main into integration | ||
|
Chris An
|
b23723a51f |
feat(heal): add live node admission feedback (#8897)
* feat(heal): add live node admission feedback * test(heal): use storage facade in admission regression * fix(heal): preserve retryable node admission deferrals |
||
|
Chris
|
a0a0cee9eb |
refactor(auth): derive policy conditions from AuthzFacts (#8442)
* refactor(auth): extract authorization facts * test(e2e): update smoke selection baseline * test(e2e): update full selection baseline |
||
|
Chris
|
da7dc74147 |
fix(quota): retire degraded floors from durable usage reports (#8832)
* fix(quota): retire degraded floors from durable usage reports * docs(quota): clarify retained usage cache refresh behavior * fix(admin): report unknown bucket usage as null in GetBucketQuota GetBucketQuota serialized missing usage as size 0, which contradicted the D3 rule that unknown usage is never reported as zero and, after retiring degraded floors, made a bucket near its limit look empty. Make size optional and stop collapsing a missing or unreadable usage snapshot to 0. |
||
|
Chris
|
db55098aa6 |
fix(scanner): recover dual corruption with retained epoch fencing (#8833)
* fix(scanner): recover dual corruption with retained epoch fencing * fix(scanner): preserve recovery markers and legacy quota floors * test(scanner): expect retained usage recovery marker state * docs(scanner): register dual-corruption recovery evidence objects Add the .bloomcycle.bin.dual-recovery/ evidence family to the persisted objects table and describe its owner, write-once lifecycle and recovery rule in Deletion And Recovery Rules. |
||
|
Chris
|
47e1850d7e |
fix(scanner): preserve remote checkpoint coverage identity (#8827)
* fix(scanner): preserve remote checkpoint coverage identity * test(scanner): verify remote plan identity propagation * test(rpc): disambiguate scanner capability body collection |
||
|
Chris
|
4e88ebcf30 |
fix(heal): preserve typed replacement readiness deferrals (#8823)
* fix(heal): preserve typed replacement readiness deferrals * docs(heal): align retry compatibility markers with guards |
||
|
overtrue
|
62b6f5f3fc |
Merge main into gateway/integration
The scheduled sync has stopped on conflicts since 2026-10-19. Resolved by keeping both sides: main's restructured delete-rollback path in disk/local.rs with the integration-only native-publication record, main's per-instance region with the rustfs-s3-types Region, main's new lifecycle, rebalance and admin code ported to the persisted lifecycle types and rustfs-s3-types errors, both sides' new scanner tests, and the union of the access_consumer imports. The e2e smoke, full (darwin) and repl-nightly selections and the test-count baseline are regenerated from the merged tree. |
||
|
Chris
|
8af4070d19 |
fix(decommission): move the free version of an expired tier version (#8777)
* fix(decommission): move the free version of an expired tier version Decommission only queued the lifecycle expiry of a transitioned version, so a source cleanup that ran first deleted the version itself and appended its tier free version after the entry's last pass. The post-check then failed the decommission until an operator re-ran it. Apply the expiry of a transitioned version synchronously, as rebalance does. Its free version is on the source before the cleanup preflight, which reports the entry changed, and the retried entry moves the record to a target in the same pass. An expiry that cannot be applied leaves the version live, so it moves like any other version. Refs rustfs/backlog#2873 * fix(decommission): move a delete-all expired tier version live Applying a delete-all expiry inline made its prefix delete wait for the bucket lifecycle write lock behind the entry's own read lock until the lock timeout, blocking every write in the bucket meanwhile. Move such a transitioned version live and leave the delete-all to lifecycle on the target. Cover a genuinely unapplied expiry and a free-version worker that consumes the source record before the entry moves it. Refs rustfs/backlog#2873 |
||
|
Hauser
|
c1de42af2c |
fix(heal): type legacy MRF decisions and defer diagnostics (#8778)
* fix(heal): explain unverified legacy heal outcomes Preserve storage diagnostics in canonical unknown outcomes and explain legacy payload identity limits through the shared local/peer status path. Borrow expected receipt identities while preserving all proof checks. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(heal): keep diagnostics from pausing merged MRF repairs Require the actual MRF executor before publishing legacy pause events. An ordinary queued owner can retain a merged durable MRF notice without having performed the specialized deep check. Preserve its diagnostic without granting it authority to park the repair responsibility. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(heal): type MRF outcomes and defer status diagnostics Carry storage-owned legacy observations in process and return a typed terminal result only after the qualified MRF Deep check. Publish pause events from that result and exact identities instead of source or text. Keep object names out of retry classification and compute only the status diagnostic selected by the canonical execution state. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> --------- Co-authored-by: heihutu <heihutu@gmail.com> Co-authored-by: zhi22915 <qiuzgang@gmail.com> |
||
|
Chris
|
dcb22bd4d5 |
refactor(ecstore): move bucket configs onto gateway persisted types (#8709)
* refactor(ecstore): move bucket configs onto gateway persisted types Finish the ECStore batch of T1.4 (rustfs/backlog#2745). The remaining eleven bucket configuration families (Website, CORS, Tagging, Versioning, ObjectLock, Encryption, Notification, Logging, Accelerate, RequestPayment, PublicAccessBlock) now read and write their stored XML through the rustfs-gateway-types persistence and cors_tagging codecs instead of the s3s codec, so the legacy s3s bucket-config codec and its s3gate-metadata-xml compat marker are removed. - ecstore: BucketConfigXml is implemented by the gateway codec for every family; metadata, metadata_sys, versioning, object lock, replication and lifecycle consumers take the persisted types. Region moves to rustfs_s3_types::Region, and the ETag conversion to s3s moves to the rustfs edge with its tests. Only warm_backend.rs and warm_backend_s3.rs still reference s3s, held for T1.5 (rustfs/backlog#2746). - scanner: object lock cache, item actions and data usage use the persisted types. - rustfs: the legacy S3 edge converts each family between s3s DTOs and the persisted types in storage/storage_api.rs by writing one codec's XML and reading it with the other; Tagging converts field by field. - Bump the rustfs-gateway-types pin to 21283e87. The 39 persisted-byte goldens are unchanged and pass, so a document the old codec wrote reads back and re-encodes to the same bytes. s3s footprint: files 181 -> 146, ecstore_files 28 -> 2, s3_error_lines unchanged. * test(app): prepare the PutBucketEncryption config from the request The uninitialized-store test passed an empty persisted configuration, which the use case now validates before it reaches the store check, so it answered MalformedXML instead of InternalError. Convert the request's configuration the way the S3 edge does. * style(app): format the PutBucketEncryption test |
||
|
Chris
|
d2eda195e6 |
fix(decommission): defer source cleanup quorum failures (#8760)
* fix(decommission): defer source cleanup quorum failures A quorum failure while deleting the source of an already copied entry was fatal and marked the whole pool failed, although the copied source is still in place and an uncommitted delete transaction is settled by delete recovery. Carry the cleanup error typed and resolve a quorum failure as Deferred: the entry is replayed at the end of the round, and a deferral that persists takes the existing capacity-blocked pause path instead of a terminal failure. Other cleanup errors and quorum errors from other stages stay terminal. Add set-level regressions for parent and child keys (`a/b`, `a/b/c`): parent cleanup leaves every child file untouched, and parent cleanup reaches quorum while the child is cleaned up or rewritten concurrently. * test(ecstore): keep source cleanup errors typed in prefix cleanup tests Return SourceCleanupError from the test helper instead of formatting it into Error::other, and shrink the error(format!) ratchet baseline for core/pools.rs, which lost one call site in the cleanup failure change. * fix(decommission): retry cleanup quorum failures and count replays once Retry a source cleanup quorum failure in place, up to three attempts with the existing source-cleanup backoff, before deferring the entry. Carry the versions an entry already counted on its deferral, so the end-of-round replay does not add a fully counted entry to items_decommissioned and bytes_done again. A deferral that persists pauses the pool through the existing deferred drain path. When source cleanup caused it, persist a reason led by "source cleanup blocked" without the carrier error's free-drive wording, and report waitingReason "source_cleanup" instead of "capacity". Add a source_cleanup test fault stage and a routine test that drives a cleanup quorum fault through deferral and replay to completion. * fix(decommission): wait seconds between cleanup quorum retries Give the in-place source cleanup quorum retry a fixed floor of one and then two seconds, with the existing jitter on top, so it can ride out a drive reconnect or health-check blip instead of retrying within a few hundred milliseconds. The wait stays bounded and cancel-safe. Find a source cleanup pause reason through stage wrappers, as the capacity-blocked check does, and add a routine test where cleanup keeps losing quorum: the decommission pauses with a source cleanup reason, the bucket stays queued and the source stays in place. Document the additive waitingReason value source_cleanup and its compatibility with older nodes reading the same pool.bin. * fix(decommission): never retry a lost source cleanup fence The cleanup retry classifier used is_quorum_error(), which also matches NamespaceLockQuorumUnavailable. The delete commit fence reports a lost namespace lock or mutation fence with that variant, so a lost fence was retried in place and only stopped later at the pre-cleanup fence check. A lost fence means this worker no longer owns the key; it must stop without retrying. Retry and defer only drive quorum failures: ErasureReadQuorum, ErasureWriteQuorum, InsufficientReadQuorum and InsufficientWriteQuorum. --------- Co-authored-by: Hauser <housemecn@gmail.com> |
||
|
Chris
|
77a36d92a3 |
fix(decommission): let a failed start roll back its own activation (#8752)
* fix(decommission): let a failed start roll back its own activation A fresh decommission start whose peer reload failed could not roll back: the pool metadata stale-write guard from #6604 refuses any save that clears a persisted active or queued decommission, so the pool stayed active with no worker until the watchdog or a leader restart picked it up. The rollback save now carries the activation that start committed. The guard exempts a pool only while its persisted record still equals that activation field for field, so a worker checkpoint or a later start keeps the rejection. A canceled record is not an active decommission and is cleared by the rollback as well. Every other save path passes no activation and is guarded as before. A peer may already have reloaded the activation and started a worker before the rollback. Saves are now fenced by decommission generation: a write carrying a start_time is rejected when the persisted record belongs to a later start, or has no decommission state and was updated after that start_time, so such a worker cannot make the pool active again. Its queued promotion and progress checkpoint also adopt the persisted record and stop the worker. Refs rustfs/backlog#2873 * fix(decommission): fence terminal writes of a withdrawn decommission Terminal records carry no start_time, so the generation fence did not cover them: after a start rollback, a peer worker of the rolled back generation that failed (for example on a bucket completion) could still persist a failed record over the restored pool. The merge now rejects a failed, canceled or complete record over a persisted record without decommission state; every attempt persists decommission state first, so no legitimate path writes a terminal record there. Peers also drop a withdrawn decommission sooner. A pool metadata reload replaces a running decommission with a newer record that withdrew it, even under an active worker; bucket completion adopts such a record like promotion and progress checkpoints do; and a withdrawn progress checkpoint stops the worker instead of logging a failed save and moving more entries. Tests cover the peer failure mark, a rollback retry that sees a worker checkpoint land between attempts, and the reload withdrawal. A heal fixture that wrote complete records over a fresh pool.bin now persists the active attempt first. Refs rustfs/backlog#2873 * fix(decommission): fence peer writes after a retry start rollback A retry start replaces a failed or canceled record that keeps its unresolved ledger, so its rollback restores that terminal record rather than clearing the pool. The withdrawal checks only recognized a record without decommission state, and the generation fence ignored a persisted record without a start_time. A peer worker of the rolled back retry could therefore checkpoint the pool back to active. The generation fence now compares the persisted last_update whenever the persisted record has no start_time of its own (cleared, terminal or queued). The progress checkpoint also checks the generation against the record read under the pool metadata lock, as bucket completion already does, and the save-path withdrawal check treats a terminal record newer than the running view as a withdrawal, so the peer adopts it and stops. A reload still never lets a newer terminal record replace an active worker's view, so after a retry's rollback a peer worker stops at its next bucket completion or progress checkpoint rather than on reload. Tests cover the retry rollback against a peer checkpoint and direct write, and a peer bucket completion after a fresh start's rollback. Refs rustfs/backlog#2873 * fix(decommission): keep a retry rollback's restored record intact After a retry start rolls back, the restored record is failed, so a peer worker's own failure mark passed every merge guard and replaced it. A failure mark now saves through save_current_pool_meta_checked, which runs the withdrawal check against the record read under the pool metadata lock and adopts that record instead of writing. The withdrawal check compared the restored record with the worker's in-memory last_update, which a rejected save had already raised, so the peer never adopted the rollback and the watchdog kept resuming it. It now compares against the running generation (start_time), and a rejected unresolved-entry save undoes its in-memory change. save_current_pool_meta_for_decommission_start is also compiled for the test-util feature again; a test-util helper still calls it, which broke the workspace clippy and Connect builds. Refs rustfs/backlog#2873 * fix(decommission): keep an unresolved entry when its own save fails The rollback of a rejected unresolved-entry save also dropped the entry on transient failures such as a quorum or lock-fence error, so the following failure mark persisted a failed record without it and an operator could clear a decommission that skipped an unreadable object. Keep the entry and undo only the last_update bump, which is all a withdrawn worker needs to still see the withdrawal. Refs rustfs/backlog#2873 |
||
|
唐小鸭
|
444eb5ddba |
fix(scanner): admit dirty scope across the bucket-listing window and pace full-walk fallbacks (#8690)
Under sustained writes nearly every cycle fell back to a full walk: scope admission required the global dirty generation to stay unchanged across the bucket listing, and after a Complete cycle the dirty-usage wake ran the next full walk immediately, back to back. - Track dirty-map removals with a process-local removal generation and admit a scoped walk when no entry was removed or replayed during the listing (`covers_listed_pending`). Writes to listed buckets in the window join the start snapshot; unlisted buckets dirtied after the cutoff stay pending and are never cleared by compare-and-remove certification. The strict `covers_all_pending` still gates segment-reuse evidence. - Classify scope fallbacks (removal during listing, generation exhausted, dirty-bucket overflow, no listed dirty bucket, baseline unavailable, peer scope unavailable), log `scoped_scan_fallback`, and report them in usage_freshness as additive serde(default) fields. - Pace the cycle after a Complete fallback while dirty buckets remain on the capped retry schedule clamped to the cycle interval, with the dirty wake disabled; the paced timer wake still requests a scope. - Update the per-bucket convergence design doc. No persisted format or internode protocol change. Co-authored-by: Chris <anzhengchao@gmail.com> |
||
|
GatewayJ
|
4dba635d48 | fix(table-catalog): authorize warehouse location claims (#8736) | ||
|
Chris
|
7cc4a9dbd0 |
fix(ecstore): leave no source free version for a moved tier version (#8732)
* fix(ecstore): leave no source free version for a moved tier version Data-movement source cleanup deletes each source version separately and never set skip_tier_free_version, so every transitioned version that decommission or rebalance had already copied to a target pool left a new tier free version on the source. Decommission then failed its post-check for any pool with tiered objects; rebalance left the record behind. The record claims the remote object the target copy still reads and stays blocked behind that live reference. Decommission and rebalance now pass the identities of transitioned versions whose copy committed on a target, and the cleanup skips the free version for exactly those. A transitioned version that did not move, such as one lifecycle expired during the move, still gets its free version because that record is its only remote-delete owner. Refs rustfs/backlog#2873, rustfs/backlog#2872 * fix(ecstore): keep the free version of a superseded moved source Data movement treats a source null version as complete when a newer unversioned object already sits on the target, without copying it. That outcome returned the same Ok as a committed copy, so a restored transitioned null version superseded this way was recorded as transferred and lost its source free version, leaving its remote object with no owner. The copy paths now return DataMovementCopy::Committed or ::Superseded, and decommission and rebalance record only committed copies of transitioned versions. A superseded source keeps its free version. Tests cover the superseded restored null version for rebalance and decommission, a restored and a null transitioned version moving by rebalance, an entry with one lifecycle-expired and one moved version, the per-version skip with two versions of one key, and a directory object. The tiered stale-target routine test from #8724 now expects the decommission to complete. The ILM ownership contract and the decommission notes describe the moved, superseded, expired and legacy cases. Refs rustfs/backlog#2873, rustfs/backlog#2872 * test(ecstore): build the decommission ILM fixtures without s3s The superseded and expiry-mix decommission tests named s3s request and lifecycle types directly, which added pools_test.rs to the ecstore files that reference s3s and failed the footprint ratchet. Set the restore request through ObjectOptions, and store the lifecycle rule as bucket metadata XML and load it with get_expiry_configs, as the decommission routine does. * test(ecstore): pin restored decommission copies and multipart supersession Decommission recorded a committed data copy of a restored transitioned version without any test failing if the record were dropped; add a routine-level decommission of a restored version that checks the post- check passes, the source keeps nothing, and the target keeps the full remote tuple. Add the multipart variant of the superseded restored null version (rebalance), and a unit test that separates committed, superseded and mismatched targets in the overwrite resume adjudication. |
||
|
Chris
|
5313b72baa |
fix(decommission): run the leader-side start to completion (#8723)
* fix(decommission): run the leader-side start to completion A forwarded decommission start awaits `ECStore::decommission` inline in the RPC handler. When the forwarding node's RPC times out, the stream is reset and the handler future is dropped wherever it is. Dropped after `pool.bin` records the pool as active but before the peers reload and the worker is spawned, the start left the pool active with no worker and peers on the old metadata, and a retry was rejected as already active. An admin client that disconnects can do the same. Run the start in its own task and let the caller await only its result. The start then always reaches its end: it fails before persisting anything, or persists the pool, has the peers reload and spawns the local workers, rolling back if the reload fails. A retry still cannot start twice: it waits on the start gate held by the running start and either reports the busy gate or is rejected because the pool is active. `decommission` now takes `self: &Arc<Self>` and spawns workers on that store instead of looking up the global object store, which is the same store in production and makes the path testable; the unused `require_decommission_store` helper is removed. Refs rustfs/backlog#2873 * fix(decommission): detach the leader-side start from its caller |
||
|
overtrue
|
a55f2f671b |
fix(decommission): keep status reads and starts off the movement gate
The data-movement operation gate is a write-preferring RwLock. A scanner publication lease stores a read guard on the leasing node, and the coordinator's lease-scoped rename, delete and validation on that node queued a second read. A movement writer queued in between (a decommission start, a status refresh, a peer reload) then waited for the lease while the coordinator's second read waited for the writer, so both stalled until the 60 s lease TTL. Admin status reads also took the movement writer, so every status call waited for the slowest in-flight copy or cleanup and stalled new movement reads while it queued. - Lease-scoped target admission shares the read admission the lease already holds instead of queueing a new read. - Pool and rebalance status reads publish under the gates only when they are free; otherwise they merge the persisted metadata into a private copy and publish nothing. - A decommission start bounds its start, activation and movement gate waits with one 30 s budget and fails with a retryable lock timeout before persisting anything; the admin handler maps it to 503 with Retry-After. Refs rustfs/backlog#2873 |
||
|
Chris
|
6f582b0548 |
fix(heal): decide object absence from storage evidence, not source (#8697)
* test(heal): add failing regressions for a source-independent absent verdict Add the regression tests for backlog#2900 that fail on the current code, where the same object-level not-found is treated as deleted on the plain heal path but as recreate_failed on the recreate path, and where the erasure-set healer, the task predicates and the error helpers disagree on wrapped not-found and transient inputs. Also add the per-object disposition cache module with its unit tests, the manager admission tests for it, the compat register entry for the display boundary, and the admission-invariant notes. The module and the manager tests are not declared yet, so nothing is wired into the build. * fix(heal): decide object absence from storage evidence, not request source Replace the five heal error classifiers (erasure-set healer outcome mapping, task-level not-found and lock predicates, error.rs recovery helpers and the storage existence-check matcher) with one typed classify_heal_error. An object-level not-found from the locked storage read is Absent for every source and every recreate_missing value; the recreate path, the usage-observation and scanner object-dir whitelists are gone, and display-text matching survives only in the marked classify_error_display boundary. Add a bounded, expiring per-object disposition memory consulted before queue admission so a best-effort producer cannot requeue an absent or terminally failed object on every tick; durable MRF, admin, forced, high-priority and dry-run requests bypass it. The read-repair producer no longer forces recreate_missing. * fix(heal): keep rendered grace windows out of task retries A grace window rendered into a task message belongs to an attempt that already finished; its owner schedules the next one. Only typed grace errors replay the task, as before the classifier merge. Update the ecstore read-repair black-box test: the producer no longer forces recreate_missing. * fix(heal): classify ambiguous legacy delete residue as permanent explicitly * fix(heal): scope disposition memory and bound absence parsing |
||
|
Chris
|
f063a8e862 |
fix(readiness): hold startup readiness until peers re-admit the node (#8704)
A restarted node published FullReady from its own view while survivors still counted its drives unavailable (stale DNS for the old pod IP, then the Offline->Returning->Online hysteresis). A rolling update could then stop the next pod and cost a set its read quorum (#8669). Add a read-only PeerDriveAdmission internode RPC that reports how a peer sees another host's drives, and gate the startup FullReady transition until a strict majority of each erasure set's peers count this node's drives online. The gate shares the startup budget, releases after 15s of unreachable peers, skips immediately when older peers cannot answer, and can be disabled with RUSTFS_STARTUP_PEER_READMISSION_ENABLE=false. |
||
|
Chris
|
9a1f3d2c4c |
fix(quota): keep ledger-admitted writes independent of scanner usage (#8699)
Hard quota admission is owned by the durable reservation ledger (decision D3, rustfs/backlog#2872), but the request-path quota check still read scanner usage on every write and mapped `UsageUnavailable` to a retryable 503 even for buckets the ledger admits. For a durable quota the checker now treats scanner usage as reporting input: an unavailable lookup yields `current_usage = None` and `allowed = true`, counted and logged at debug level, and admin quota stats report the usage as null instead of zero. Buckets without a reservation protocol keep the fail-closed snapshot path. Quotas saved before the reservation protocols are re-saved under protocol v2 with the same limit, type and created_at. The write goes through a new guarded save that keeps the admin save's lifecycle/transaction guards, incarnation check and fleet capability proof and additionally refuses when the quota revision moved since the cached read, so a concurrent quota edit wins. It is triggered lazily from the quota read path (background, one attempt per bucket with a retry interval, never blocking the read) and in bulk by `POST /rustfs/admin/v3/quota-migration`, which reports every bucket. The ADR's Consequences section inventories the remaining scanner-usage "authority" consumers with their meaning after D3, retirement condition and compatibility window; the operations guide documents the migration and the report-only usage semantics. Refs rustfs/backlog#2904, rustfs/backlog#2872, rustfs/rustfs#8678, rustfs/rustfs#5716, rustfs/rustfs#7350 Co-authored-by: Hauser <housemecn@gmail.com> |
||
|
Chris
|
7c75f319bc |
fix(scanner): stop treating an ordinary PUT as data movement in cycle termination (#8696)
* fix(scanner): stop treating an ordinary PUT as data movement in cycle termination A continuously written deployment never reached a `Complete` scanner cycle (#8494, #8305, #7108). Three places in the termination path vetoed the cycle for an ordinary PUT; each now lets the write shade its bucket, the path per-bucket certification (#8542) built, and keeps the data-movement fences. 1. Usage publication is fenced by data movement only. `scanner_data_usage_publication_blocked` no longer includes `namespace_commits_pending()`, so the post-walk probe, the local defer decision, the post-lease recheck and the usage-store route probes stop superseding or deferring a cycle for a pending rename fanout. The scanner publication leases (acquire, validate, target guard) drop the pending-commit rejection and the namespace-generation staleness check and keep the movement generation, the operation gate, generation exhaustion and the purpose/target restriction. A test-util probe lets fixtures wait for tails directly. 2. The plan identity is split into a coverage identity and a reuse identity. `scanner_activity_coverage_digest` (peer set, protocol, topology, movement, and a remote peer's generations) is what a bucket checkpoint is started and resumed under, through `ScannerBucketScanPlan::checkpoint_coverage_digest`, `DataUsageCacheReuseOptions::checkpoint_plan_digest` and `prepare_bucket_checkpoint_with_coverage`. The reuse identity (`scanner_activity_structural_digest`) keeps `instance_id` and `maintenance_generation`, so a restart or a maintenance change still invalidates the baseline and the set caches; a completed sweep is stored under it. `publication_blocked` leaves every digest and stays a guardrail veto. A remote worker still resumes under the single digest on the wire. 3. A bucket walk that stops early no longer makes the cycle `Incomplete` when the bucket has a certified value. The set scan stands in with the bucket's flattened value from its previous complete set cache (`carry_partial_bucket_values`), the cycle reports it pending (`ScannerBucketScanStatus::PartialCarried`), keeps it in the dirty map, and the next scoped cycle walks it because the baseline's pending list joins the dirty buckets. A bucket with no certified value keeps the cycle `Incomplete`; failed buckets are unchanged. No persisted file, generation, receipt or environment variable is added. Refs #8494, #8305, #7108, rustfs/backlog#2903. * chore(ecstore): gate the namespace commit probe to test builds The scanner publication barrier no longer consults InstanceContext::namespace_commits_pending, so it has no production caller left; keep it for tests and the test-util facade only. * test(rustfs): observe namespace commit ownership directly in node_service tests The scanner usage publication barrier is movement-only now, so it no longer reports a pending rename. The node_service tests that assert a cancelled handler keeps its physical owner, or that wait for rename tails to drain, read the namespace commit state through a test-only ECStore probe instead. |
||
|
Hauser
|
05b483b31a |
fix(storage): preserve scanner checkpoints and bound MRF journal I/O (#8692)
* fix(scanner): preserve checkpoints on identity read failures Distinguish confirmed missing incarnation sidecars from lookup errors before cache preparation. Preserve durable checkpoint progress when identity reads are unavailable and report post-scan uncertainty separately from replacement. Reuse the transaction-fenced reader for optional and required lookups while retaining existing missing, malformed, system-bucket and lock-loss behavior. Verify preservation and recovery through a real coordinator with the identity sidecar write-locked, including unchanged cache revisions and dirty work. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * refactor(ecstore): share required incarnation conversion Keep the missing-incarnation error contract in one conversion used by both required lookup entry points. This removes a duplicate formatted error site and satisfies the existing error-format ratchet without changing its baseline. Pin the full missing-identity error text in the real SetDisks regression. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(heal): bound cross-disk MRF checkpoint I/O Run independent journal disks with bounded concurrency while retaining each replica's payload, companion and manifest ordering and the mirror phases. Share encoded bodies, deduplicate local roots and collect every result without changing receipt deadlines or persistence failure semantics. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> --------- Co-authored-by: heihutu <heihutu@gmail.com> Co-authored-by: zhi22915 <qiuzgang@gmail.com> |
||
|
Chris
|
f3cef98afc |
fix(scanner): anchor the bitrot cadence and budget per-disk I/O (#8701)
* test(scanner): pin Normal mode when no bitrot start is recorded Regression for rustfs/backlog#2905: three consecutive cycles that find no recorded bitrot start currently run Deep through the selection window, and a Normal cycle never records the cadence start. * fix(scanner): anchor the bitrot cadence and budget per-disk I/O Decide the scan mode from a recorded Deep window start only. A cycle that finds no recorded start runs Normal and anchors the bitrot cadence in process memory instead of opening the selection window; the Deep cycle that follows the elapsed cadence records the window start exactly as before. A missing record used to be read as a window that started at cycle 0, which ran Deep for deep_window_cycles cycles whenever the start could not be read back (rustfs/rustfs#8015). The recorded start is also remembered across a skipped or failed save, and /v3/scanner/status reports the cadence state under a new bitrot object. Add a per-local-disk background I/O budget (operations and bytes per second, defaults by detected media, auto|off|<rate>) that DynamicSleeper charges before directory listings and object metadata reads as a hard ceiling layered on the foreground-priority disk admission gate, with runtime config keys, per-disk status, cycle metrics, pacing pressure, usage_freshness, Prometheus series, Helm values, admin config help, and operator docs. Refs rustfs/backlog#2905, rustfs/rustfs#8015, rustfs/rustfs#8377. * test(scanner): expect Normal retries on erasure sets without a bitrot start The pending-PUT retry fixture asserted that a 4-drive set revisits every bucket because the cycle ran Deep. Without a recorded Deep window the cycle now runs Normal on every layout and reuses the cold inventory. |
||
|
Chris
|
2534b6175b |
fix(scanner): startup gates back off, rebuild and run degraded (#8695)
* fix(scanner): give startup gates automatic exits instead of a pause Every fail-closed scanner gate now has a bounded automatic exit without deleting quarantine evidence or relaxing the leader, CAS, movement or usage-floor fences (rustfs/backlog#2901). - Recovery retries back off instead of pausing: after the short budget the observation reads `backoff`, stays retryable, names the next attempt and is capped at ten minutes; one success resets the count and a 24 h window restarts the budget (rustfs/rustfs#8297, #8670). The usage-floor failure no longer carries the retry count across attempts. - A corrupt `.bloomcycle.bin` is rebuilt on the iteration after its quarantine marker is durable: the fenced leadership claim CAS-replaces the quarantined revision under a higher epoch and the marker stays as evidence. `future_schema` and `cleanup-pending` remain blocked. - An unclaimable pause ledger runs the leader degraded: cycles run untracked behind the data-movement fence, `persistence_state` reports `degraded`, and the claim is retried at most every five minutes (rustfs/rustfs#8251). - The usage-state full-rebuild reset starts from the empty baseline when every usage slot is corrupt and no cycle state exists (rustfs/rustfs#8670, #6951). - A cycle-state save reconciles non-conflict failures against durable state and retries under the same epoch; an exhausted save reloads durable state in place. Only a newer owner or an unreadable primary ends the iteration, so peers see a changed fence only when ownership changed (rustfs/rustfs#8141). * test(scanner): align usage-state reset tests with the empty-baseline rebuild The full-rebuild reset no longer refuses corrupt or identity-less usage slots when no cycle state exists (rustfs/rustfs#8670). The two tests that encoded that refusal now check the new contract: the reset starts from the empty baseline with a bootstrap-pending marker, never an authoritative zero floor, and a valid backup still supplies the recovery floor. --------- Co-authored-by: cxymds <cxymds@gmail.com> |
||
|
Chris
|
90749fbb06 | ci: run feature lanes nightly and warm caches on a schedule (#8683) | ||
|
Chris
|
22740876b3 |
docs(architecture): record scanner usage authority and PUT-tail decisions (#8678)
Supersede the 2026-09-03 decision: hard quota admission is owned by the durable reservation ledger, and scanner data usage is best-effort reporting input. Record that a pending ordinary PUT commit is a write-barrier event rather than a usage-publication veto, and split the scanner plan identity into a coverage identity and a reuse identity. Refs rustfs/backlog#2872, rustfs/backlog#2903, rustfs/backlog#2904. |
||
|
Hauser
|
6681dd12e3 |
fix(scanner): expose mixed sweep checkpoint transitions (#8657)
* fix(scanner): expose mixed sweep checkpoint transitions Correlate checkpoint admission, incomplete sweeps, and successful terminal Partial persistence by bucket/source and cycle/leader epoch. Share the traversal-position predicate and local/remote save diagnostic without changing certification, acknowledgement, or persistence contracts. Extend real-disk cache regressions across cycles and leader changes and verify that failed terminal saves retain an older periodic frontier until a successful terminal save permits fresh certification. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(scanner): report the aggregate incomplete sweep gate Expose the existing unresolved_objects decision in the incomplete-sweep event. Nested cache-entry failures can keep a sweep partial while both the root counter and the capped diagnostic failure map are empty. Extend the real-disk regression with diagnostics disabled and distinguish this case from mixed-only coverage without adding another cache walk. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> --------- Co-authored-by: heihutu <heihutu@gmail.com> Co-authored-by: zhi22915 <qiuzgang@gmail.com> |