ci(parity): run dual-stack differential against pinned release (#9315 )

* feat(server): add the RUSTFS_S3_STACK selector and dual-stack e2e lane (#8411) * refactor(server): group the s3s-compat tower layers under legacy_compat (#8410) Move the ten tower layers that patch s3s behaviour out of rustfs/src/server/layer.rs into rustfs/src/server/legacy_compat/, one file per layer with its tests. Each file header names the s3s behaviour it patches, the gateway mechanism that replaces it (Replaced by:), and the task that removes it (Removed by: T2.12, rustfs/backlog#2771). The layers keep their code, their order in external_service_stack!, and their behaviour; no layer is deleted. Add server::http::tests::stack_census, which renders the ordered service chain of the real external_service_stack! expansion and compares it with a committed constant, so a reorder, insertion or removal is a visible diff. Route the two s3s error types the moved layers use through a storage_api::server::legacy_compat shim so the split adds no s3s-importing file, and lower the s3s footprint baselines to the measured values (files 209 to 208, s3_error! lines 1586 to 1585). Keep the server runtime-source bypass guard covering the moved code. Refs rustfs/backlog#2787, rustfs/backlog#2734 * feat(s3-types): add the RustFS-owned S3 error type and s3_error! macro (#8406) `rustfs-s3-types` now provides `S3ErrorCode` (generated by build.rs from src/codes.txt, the grep-derived call surface, plus `Custom`), `S3Error`, `S3Result`, `StdError` and the `s3_error` macro with the same call syntax as s3s, so the T1.2 codemod is a `use` swap. Statuses are hand-written and pinned to the s3s behaviour RustFS ships today by an oracle test that uses s3s as a dev-dependency only. The `From` conversions for the legacy edge live behind the off-by-default `compat-s3s` feature (the orphan rule forbids them in rustfs/src/error.rs); `rustfs` enables it. Refs rustfs/backlog#2742, rustfs/backlog#2734 * ci: add the per-crate test-count ratchet (#8404) Quick Checks now fails any PR that lowers the number of test attributes in a workspace crate or adds #[ignore], measured per crate against the exact rows of .config/test-count-baseline.txt (rustfs/backlog#2734 section 1 rule 3). The counter widens the spec's literal #[(tokio::)?test] to the tokio attributes that carry arguments and both spellings of #[test_case(...)], and anchors at line start so commented-out attributes do not count. The initial baseline covers 52 crates, 21077 tests and 164 ignored. Refs rustfs/backlog#2738 Refs rustfs/backlog#2734 * test(ecstore): pin the persisted bucket-config XML bytes for 13 families (#8408) Before any DTO or codec is replaced, pin the exact bytes that deserialize followed by serialize produces for every persisted bucket-configuration XML family in its three source shapes (AWS documentation, MinIO-written, RustFS-written), and the bytes RustFS writes today for a typed value. Byte equality only: a structural comparison would let a codec replacement change the persisted encoding unnoticed. The generator script refuses to overwrite existing goldens without --force, refuses on dirty codec paths, cannot report success from its write mode, and records the generating commit in the fixtures README. Refs rustfs/backlog#2744, rustfs/backlog#2734 * ci: raise the test-count baseline to the integration tree (#8426) * ci: make the s3s footprint baseline exact and lower-only (#8403) * ci: make the s3s footprint baseline exact and lower-only The s3s footprint guard compared three counters against ceilings held in the script itself, so a ceiling could sit above the tree and a drift was fixable by raising it (rustfs/rustfs#5964 raised the s3_error! ceiling to go green). The three values now live in .config/s3s-footprint-baseline.txt as files, s3_error_lines and ecstore_files, the script reads only that file, and a measurement that differs from its baseline in either direction fails: growth prints "s3s footprint grew: <measured> > <baseline>", shrinkage prints "lower the baseline to <measured> in this PR". The counting method is unchanged; the baseline is the current measurement of gateway/integration (209 / 1585 / 36). The header records the end state: once all three reach 0, T4.2 switches the script to failing on any occurrence. Refs rustfs/backlog#2739, rustfs/backlog#2734 * ci: add the s3s edge allowlist mode to the footprint guard (#8405) `scripts/check_s3s_footprint.sh --mode allowlist` fails on every Rust file that references s3s (`s3s::` or `s3s_sigv4`) and is not matched by an entry of the new `.config/s3s-edge-allowlist.txt`; `--dry-run` lists those files and exits 0. The allowlist carries the initial edge from the spec, and the script rejects an entry that names or matches anything under crates/ other than crates/e2e_test. The baseline mode and the argument-less CI invocation are unchanged. Refs rustfs/backlog#2741, rustfs/backlog#2734 * test(ecstore): reach the bucket-config codec through the test facade (#8428) * refactor(s3-types): own the HTTP Body and drop s3s from three crates (#8431) Add rustfs_s3_types::Body, a BoxBody<Bytes, StdError> wrapper with empty() and From<Bytes | Vec<u8> | String | &'static str>, and the legacy From conversions to and from s3s::Body behind compat-s3s. The signer is now generic over the request body: it only reads and writes the request head, so it needs no body type and its callers keep theirs. s3-client sends its requests with the owned Body, madmin signs a bodiless head, and targets drops its two S3Request/S3Response header wrappers in favour of extract_params_header. rustfs-signer, rustfs-madmin and rustfs-targets no longer depend on s3s; the s3s footprint baseline drops from 209 to 202 files. Refs rustfs/backlog#2747, rustfs/backlog#2734 * refactor(ecstore): move S3ErrorCode to rustfs-s3-types by codemod (#8432) * test(ecstore): pin the InvalidRange responses of new_getobjectreader Both error branches of new_getobjectreader build an ErrorResponse with S3ErrorCode::InvalidRange, and no test observed either code. Pin the code, its wire name and the location fields of each branch before the code type moves to rustfs-s3-types (rustfs/backlog#2743), so a changed code on either branch fails a test. The ecstore test-count row rises by the two tests. * chore(scripts): add the s3s-to-rustfs-s3-types error codemod Task T1.2 of rustfs/backlog#2734 (rustfs/backlog#2743) moves S3Error, S3ErrorCode, S3Result and s3_error! from s3s to rustfs-s3-types by script, one crate closure per PR. The codemod rewrites use trees (splitting mixed s3s imports) and qualified code paths, never comments or literals, adds the rustfs-s3-types dependency, drops s3s only from a crate that no longer names it, refuses every file of the s3s edge allowlist, takes --hold for a file whose type is fixed outside the closure, and prints a per-crate census. A second run changes nothing. Its self-test runs under make script-tests. * refactor(ecstore): take S3ErrorCode from rustfs-s3-types Output of scripts/codemods/s3_error_to_s3_types.py over crates/ecstore, crates/lifecycle, crates/scanner and crates/replication (rustfs/backlog#2743 batches 1 and 2). The ErrorResponse code field, the two InvalidRange constructions in new_getobjectreader and an unused tier import now use rustfs_s3_types::S3ErrorCode; the variant names and wire names are unchanged. lifecycle, scanner and replication reference none of the four error names. warm_backend.rs and warm_backend_s3.rs are held: they compare against rustfs_s3_client::ErrorResponse.code, which stays the s3s type until s3-client moves. * ci(footprint): count only s3_error! lines still on s3s's macro s3_error_lines counted every line holding the text `s3_error!`, which also matches the RustFS-owned `rustfs_s3_types::s3_error!`. A codemod that moves call sites onto rustfs-s3-types therefore could not lower it (rustfs/backlog#2743, decision 3; rustfs/rustfs#8432, gotcha 2). The counter now applies a textual, per-file and per-line rule that errs on the side of counting (full rule in the script header): - occurrences spelled `rustfs_s3_types::s3_error!` never count; - a line holding `s3s::s3_error!` always counts; - any other `s3_error!` line counts unless its file is cleared, that is, it imports the macro from rustfs_s3_types by name on a line starting with the use item (or defines it with `macro_rules! s3_error`) and does not also bring s3s's macro in (`use s3s::s3_error`, a `use s3s::{..}` tree naming it or a crate-root `*`, `use s3s::*`, or `#[macro_use] extern crate s3s`); - fail closed: a file matching neither form (`use super::*`, a crate-local re-export, a renamed or glob rustfs_s3_types import, a commented-out import, a comment-only mention) counts. Census of the 1585 lines on 96fc06776: 1478 in 64 files that import the macro from s3s; 107 in 15 files matching neither form (105 through `use super::*`, 1 qualified `s3s::s3_error!`, 1 comment line); none in files that import it from rustfs_s3_types or define it. Every line stays counted, so the baseline remains s3_error_lines=1585 (files=202 and ecstore_files=36 unchanged). Rewriting rustfs/src/auth.rs's import to rustfs_s3_types lowers the new counter to 1567; the old one stayed 1585. `--self-test` (wired into `make script-tests`) copies the script into 22 fixture trees: 6 positive cases and 16 negative ones (lines that must not count, forms that must not clear a file, and a missing baseline, missing key or failing classification search that must fail). Each tree has a control line that must count, and each run gets s3s text on stdin. Seven cases were red against the old counter. 19 mutants each turn their case red: old definition, flipped import test, dropped fail-closed branch, dropped e2e_test exclusion, dropped s3s veto, dropped qualified s3s rule, dropped rustfs-qualified strip, unanchored rustfs import, renamed import accepted, nested glob taken as root glob, single-line s3s trees only, dropped s3s glob forms, dropped macro_use form, dropped definition clearing, ignored classification rg errors, error-line search without its path, classification searches without their path, missing baseline skipped, missing key skipped. * test(e2e): compare two RUSTFS_S3_STACK servers over every S3 operation (#8435) Add dual_stack_differential to e2e_test. It starts two rustfs processes from the same binary, stack A on RUSTFS_S3_STACK=legacy and stack B on RUSTFS_E2E_STACK_B (default legacy until rustfs/backlog#2734 T2.5), sends both the same aws-sdk-s3 script covering the 73 methods of impl S3 for FS (a normal request and a 4xx negative each), and compares status, error code, an allowlist of response headers and a few typed body fields. Error message text is never compared. Each run writes <target>/dual-stack/<run>.json with compared_ops and the (op, field, a, b) differences; any difference fails the test. differential_detects_an_injected_difference starts B with RUSTFS_REGION=eu-west-1 and requires GetBucketLocation to be reported. The comparator is pure and unit-tested without a server. Wire the module into the e2e-smoke filter and update the e2e-smoke and e2e-full selection digests and the e2e_test test-count row. Refs rustfs/backlog#2740 * feat(app): add the RustFS-owned RequestEnvelope and its s3s constructor (#8437) * feat(app): add the RustFS-owned RequestEnvelope and its s3s constructor * ci: classify the s3s edge adapters in storage/s3_compat as interface * feat(s3-types): add Body::from_stream and move Swift off s3s::Body (#8438) rustfs/backlog#2747 batch 3. `Body::from_stream` wraps a chunk stream of unknown length over `StreamBody`/`BoxBody`: one data frame per chunk, the first error ends the body, the size hint stays unknown, and dropping the body drops the stream. Parity tests pin it to the s3s `StreamingBlob::wrap` it replaces, at the `http_body` level and byte for byte through hyper. The Swift handlers in rustfs-protocols build their responses as the owned `Body`. `SwiftService` converts them into the wrapped S3 service's response body type through `From<Body>`, so rustfs keeps serving `s3s::Body` at the legacy edge unchanged. Request bodies stay Send-only (`UnsyncBoxBody`); the two storage writers and the DLO/SLO segment streams get `Sync` from `sync_wrapper::SyncStream`, which is what s3s's `http_body_unsync` mutex provided before. The DTO and error uses of s3s in protocols are untouched (T1.5 and T1.2). * refactor(protocols): move error types to rustfs-s3-types by codemod (#8439) T1.2 batch 3 (rustfs/backlog#2743): s3select-api, s3select-query and the error items of protocols move off s3s by scripts/codemods, run with --hold crates/protocols/src/sftp/errors.rs. That file stays on s3s: it downcasts the backend error, whose runtime type is still s3s::S3Error. - rustfs/src/protocols/client.rs: the two webdav session methods now return rustfs_s3_types::S3Result and convert the s3s error with From. - codemod: a child module that calls s3_error! through `use super::*` gets its own `use rustfs_s3_types::s3_error;` once its parent imports that macro by name; otherwise it is listed as GLOB. - s3-types: a table test pins code, status and the s3s error the legacy edge renders for every variant a moved call site names. - s3_error_lines 1585 -> 1584 (a lifecycle comment reworded). * test(connect): restore one-second drive request budgets Apply the exact test-only repair already landed on main in 5972a09cc3. The filesystem probe and production CLI success test request one second and keep their existing one-second duration and parameter checks. All outcome, I/O, error, cleanup and resource-limit assertions remain. Scope: rustfs/tests/connect_perf_drive.rs only. Patch-id: cb0a1dfc8f68a67727525adea7458e2cee7d09f8. Validation environment: LANG=en_US.UTF-8, CARGO_BUILD_JOBS=1, CARGO_INCREMENTAL=0, exclusive APFS-cloned CARGO_TARGET_DIR. Quick Checks: every command from .github/actions/quick-checks/action.yml, with captured output and exit status below. $ shellcheck --version && actionlint ShellCheck - shell script analysis tool version: 0.11.0 license: GNU General Public License, version 3 website: https://www.shellcheck.net exit_code=0 $ cargo fmt --all --check exit_code=0 $ ./scripts/check_unsafe_code_allowances.sh exit_code=0 $ ./scripts/check_layer_dependencies.sh Layer dependency guard passed (no new reverse dependencies/cycles). exit_code=0 $ ./scripts/check_architecture_migration_rules.sh Architecture migration rules passed. exit_code=0 $ ./scripts/check_logging_guardrails.sh ✅ Logging guardrails check passed exit_code=0 $ ./scripts/check_error_other_format_ratchet.sh error(format!) ratchet passed (total call sites: 582). exit_code=0 $ ./scripts/check_no_tokio_io_uring.sh exit_code=0 $ ./scripts/check_extension_schema_boundaries.sh Extension schema boundaries passed. exit_code=0 $ ./scripts/check_body_cache_whitelist.sh Body-cache whitelist guard passed. exit_code=0 $ ./scripts/check_s3s_footprint.sh s3s footprint OK: files importing s3s is 195 (baseline files=195) s3s footprint OK: s3s s3_error! lines is 1584 (baseline s3_error_lines=1584) s3s footprint OK: ecstore files referencing s3s is 36 (baseline ecstore_files=36) ✅ s3s footprint baseline check passed exit_code=0 $ ./scripts/check_test_count_ratchet.sh test-count ratchet OK: 52 crates, 21222 tests, 164 ignored match .config/test-count-baseline.txt exit_code=0 $ ./scripts/check_fips_wording.sh FIPS wording guard passed (README.md and CHANGELOG.md contain no forbidden claims). KMS wording guard passed (crates/kms claims no Vault KV2 Transit wrapping). exit_code=0 $ ./scripts/check_embedded_secrets.sh Embedded secret guard passed (no private key material or provider credential literal in the tree). exit_code=0 $ make script-tests Running script tests... entrypoint credential policy tests passed internode grpc A/B bench env tests passed object batch benchmark enhanced tests passed PASS package service script contract hotpath warp A/B gate tests passed hotpath warp ABBA tests passed Scanner validation artifacts written to /var/folders/0w/s4wxsn2n4w9b6f3cd4xrz53w0000gn/T/tmp.1pFe6Kl1Yu/out Scanner validation artifacts written to /var/folders/0w/s4wxsn2n4w9b6f3cd4xrz53w0000gn/T/tmp.1pFe6Kl1Yu/out-missing-pid ............................. ---------------------------------------------------------------------- Ran 29 tests in 181.286s OK {"decision":"verified","pending_gates":["G01","G03","G04","G05","G06","G07","G08","G09","G10","G11","G12","G13","G14","P1","P2","P3","P4","R-D","R-E","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"G02"} {"decision":"verified","pending_gates":["G01","G02","G03","G04","G05","G06","G07","G08","G09","G10","G11","G12","G13","G14","P1","P2","P3","P4","R-D","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"R-E"} {"decision":"verified","pending_gates":["G01","G03","G04","G05","G06","G07","G08","G09","G10","G11","G12","G13","G14","P1","P2","P3","P4","R-D","R-E","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"G02"} {"decision":"verified","pending_gates":["G01","G02","G03","G04","G05","G06","G07","G08","G09","G10","G11","G12","G13","G14","P1","P2","P3","P4","R-D","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"R-E"} {"decision":"verified","pending_gates":["G02","G03","G04","G05","G06","G07","G08","G09","G10","G11","G12","G13","G14","P1","P2","P3","P4","R-D","R-E","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"G01"} {"decision":"verified","pending_gates":["G01","G02","G04","G05","G06","G07","G08","G09","G10","G11","G12","G13","G14","P1","P2","P3","P4","R-D","R-E","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"G03"} {"decision":"verified","pending_gates":["G01","G02","G03","G04","G05","G06","G07","G08","G09","G10","G11","G12","G13","G14","P1","P2","P3","P4","R-D","R-E"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"R-L"} {"decision":"verified","pending_gates":["G01","G02","G03","G04","G05","G06","G07","G08","G09","G10","G11","G12","G13","P1","P2","P3","P4","R-D","R-E","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"G14"} {"decision":"verified","pending_gates":["G01","G02","G03","G04","G05","G06","G07","G08","G09","G10","G11","G12","G13","P1","P2","P3","P4","R-D","R-E","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"G14"} {"decision":"verified","pending_gates":["G01","G02","G03","G04","G05","G06","G07","G08","G09","G11","G12","G13","G14","P1","P2","P3","P4","R-D","R-E","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"G10"} {"decision":"verified","pending_gates":["G01","G02","G03","G04","G05","G06","G07","G08","G09","G10","G11","G12","G13","G14","P2","P3","P4","R-D","R-E","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"P1"} {"decision":"verified","pending_gates":["G01","G02","G03","G04","G05","G06","G07","G08","G09","G10","G11","G12","G13","G14","P1","P3","P4","R-D","R-E","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"P2"} {"decision":"verified","pending_gates":["G01","G02","G03","G04","G05","G06","G07","G08","G09","G10","G11","G12","G13","G14","P1","P2","P4","R-D","R-E","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"P3"} {"decision":"verified","pending_gates":["G01","G02","G03","G04","G06","G07","G08","G09","G10","G11","G12","G13","G14","P1","P2","P3","P4","R-D","R-E","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"G05"} {"decision":"verified","pending_gates":["G01","G02","G03","G04","G05","G07","G08","G09","G10","G11","G12","G13","G14","P1","P2","P3","P4","R-D","R-E","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"G06"} {"decision":"verified","pending_gates":["G01","G02","G03","G04","G05","G06","G07","G08","G09","G10","G11","G12","G13","G14","P1","P2","P3","P4","R-E","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"R-D"} Status-and-outcome release descriptor verified: /private/var/folders/0w/s4wxsn2n4w9b6f3cd4xrz53w0000gn/T/tmp8rwn1pe0/descriptor/release-bundle-status-outcome.json {"decision":"verified","pending_gates":["G01","G02","G03","G04","G06","G07","G08","G09","G10","G11","G12","G13","G14","P1","P2","P3","P4","R-D","R-E","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"G05"} {"decision":"verified","pending_gates":["G01","G02","G03","G04","G05","G07","G08","G09","G10","G11","G12","G13","G14","P1","P2","P3","P4","R-D","R-E","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"G06"} {"decision":"verified","pending_gates":["G01","G02","G03","G04","G05","G06","G07","G08","G09","G10","G11","G12","G13","G14","P1","P2","P3","P4","R-E","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"R-D"} {"decision":"verified","pending_gates":["G01","G02","G03","G04","G05","G06","G08","G09","G10","G11","G12","G13","G14","P1","P2","P3","P4","R-D","R-E","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"G07"} {"decision":"verified","pending_gates":["G01","G02","G03","G04","G05","G06","G07","G09","G10","G11","G12","G13","G14","P1","P2","P3","P4","R-D","R-E","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"G08"} {"decision":"verified","pending_gates":["G01","G02","G03","G04","G05","G06","G07","G08","G09","G10","G11","G12","G13","G14","P1","P2","P3","R-D","R-E","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"P4"} exact 1MiB handoff ABBA tests passed pinned paired ABBA bench tests passed ==> /var/folders/0w/s4wxsn2n4w9b6f3cd4xrz53w0000gn/T/tmp.mYmuI01hNq/bin/path_containment (-max_total_time=7, -seed=123456789, -artifact_prefix=/var/folders/0w/s4wxsn2n4w9b6f3cd4xrz53w0000gn/T/tmp.mYmuI01hNq/explicit-artifacts/path_containment/, corpus=/private/tmp/gw-orchestrator-2734/ci-drive/fuzz/corpus/path_containment) -seed=123456789 target=path_containment seed=123456789 max_total_time=7 git_revision=4d15b816236630b2c42f321ffc9ed3654edc94e3 git_dirty=true runner_mode=prebuilt ==> /var/folders/0w/s4wxsn2n4w9b6f3cd4xrz53w0000gn/T/tmp.mYmuI01hNq/bin/path_containment (-max_total_time=1, -seed=2322617706, -artifact_prefix=/var/folders/0w/s4wxsn2n4w9b6f3cd4xrz53w0000gn/T/tmp.mYmuI01hNq/auto-artifacts/path_containment/, corpus=/private/tmp/gw-orchestrator-2734/ci-drive/fuzz/corpus/path_containment) -seed=2322617706 ==> cargo +nightly fuzz run path_containment (-max_total_time=9, -seed=987654321, -artifact_prefix=/var/folders/0w/s4wxsn2n4w9b6f3cd4xrz53w0000gn/T/tmp.mYmuI01hNq/cargo-artifacts/path_containment/) +nightly fuzz run path_containment -seed=987654321 seed=987654321 runner_mode=cargo-fuzz fuzz runner seed manifest tests passed ✅ scripts/python_bin.sh resolver checks passed self-test ok: clean fixture with every exemption present passes self-test ok: all 11 pattern families fire self-test ok: moving a pinned source is reported Embedded secret guard self-test passed. self-test ok: p1-s3s-import: 'use s3s::s3_error;' counts every line, the rustfs_s3_types module's too (s3_error_lines=3) self-test ok: p2-s3s-mixed-tree: a multi-line s3s use tree naming s3_error counts (s3_error_lines=3) self-test ok: p3-s3s-glob: a crate-root glob of s3s, bare or inside a tree, counts (s3_error_lines=5) self-test ok: p4-macro-use: '#[macro_use] extern crate s3s' counts (s3_error_lines=3) self-test ok: p5-qualified-s3s: a qualified s3s::s3_error! line counts in a cleared file (s3_error_lines=2) self-test ok: p6-exact-pass: a baseline equal to the tree passes (exit 0) self-test ok: n1-rustfs-import: 'use rustfs_s3_types::s3_error;' alone counts nothing (s3_error_lines=1) self-test ok: n2-rustfs-mixed-tree: a rustfs_s3_types tree plus an s3s tree without the macro counts nothing (s3_error_lines=1) self-test ok: n3-defines: the file defining the macro counts nothing (s3_error_lines=1) self-test ok: n4-qualified-rustfs: a qualified rustfs_s3_types::s3_error! line counts nothing (s3_error_lines=1) self-test ok: n5-renamed-call: a call through a renamed macro is not text-matched (out of scope) (s3_error_lines=1) self-test ok: n6-comment-string: a comment or string mentioning s3_error! in a cleared file counts nothing (s3_error_lines=1) self-test ok: n7-e2e-excluded: crates/e2e_test is excluded (s3_error_lines=1) self-test ok: n8-super-glob: an unrecognised import form ('use super::*;') counts (s3_error_lines=3) self-test ok: n9-facade: a crate-local re-export counts (s3_error_lines=2) self-test ok: n10-renamed-import: a renamed rustfs_s3_types import does not clear s3_error! lines (s3_error_lines=2) self-test ok: n11-commented-import: a commented-out rustfs_s3_types import does not clear (s3_error_lines=2) self-test ok: n12-rustfs-glob: a glob import of rustfs_s3_types does not clear (s3_error_lines=2) self-test ok: n13-comment-only: a comment mentioning s3_error! in a file with no import counts (s3_error_lines=2) self-test ok: f1-baseline-missing: a missing baseline file fails (exit 1) self-test ok: f2-key-missing: a baseline without s3_error_lines fails (exit 1) self-test ok: f3-rg-fails: a failing classification search fails the guard (exit 1) ✅ s3s footprint self-test passed test_core_gate_rejects_missing_ignored_filtered_and_corrupt_inputs (__main__.SelfTests.test_core_gate_rejects_missing_ignored_filtered_and_corrupt_inputs) ... ok test_e2e_requires_registration (__main__.SelfTests.test_e2e_requires_registration) ... ok test_fuzz_runtime_uses_matrix_target (__main__.SelfTests.test_fuzz_runtime_uses_matrix_target) ... ok test_ilm_lane_keeps_the_measured_cargo_build_budget (__main__.SelfTests.test_ilm_lane_keeps_the_measured_cargo_build_budget) ... ok test_profile_listing_binds_platform_digest (__main__.SelfTests.test_profile_listing_binds_platform_digest) ... ok test_profile_listing_enforces_selection (__main__.SelfTests.test_profile_listing_enforces_selection) ... ok test_quick_checks_commands_propagate_failure (__main__.SelfTests.test_quick_checks_commands_propagate_failure) ... ok test_quick_checks_rejects_caller_and_execution_bypasses (__main__.SelfTests.test_quick_checks_rejects_caller_and_execution_bypasses) ... ok test_runner_readiness_fails_closed (__main__.SelfTests.test_runner_readiness_fails_closed) ... ok test_s3_runner_rejects_unbounded_failure_locals (__main__.SelfTests.test_s3_runner_rejects_unbounded_failure_locals) ... ok test_scanner_heal_begin_requires_embedded_source_provenance (__main__.SelfTests.test_scanner_heal_begin_requires_embedded_source_provenance) ... ok test_scanner_heal_case_does_not_approve_pending_release (__main__.SelfTests.test_scanner_heal_case_does_not_approve_pending_release) ... ok test_scanner_heal_case_lane_is_required_for_runner_profile (__main__.SelfTests.test_scanner_heal_case_lane_is_required_for_runner_profile) ... ok test_scanner_heal_case_only_schema_cannot_approve_release (__main__.SelfTests.test_scanner_heal_case_only_schema_cannot_approve_release) ... ok test_scanner_heal_crash_case_rejects_restart_oracle (__main__.SelfTests.test_scanner_heal_crash_case_rejects_restart_oracle) ... ok test_scanner_heal_ec84_case_rejects_wrong_erasure_geometry (__main__.SelfTests.test_scanner_heal_ec84_case_rejects_wrong_erasure_geometry) ... ok test_scanner_heal_finish_collects_oracles_from_registry (__main__.SelfTests.test_scanner_heal_finish_collects_oracles_from_registry) ... ok test_scanner_heal_multipool_case_accepts_deferred_optional_outage_write (__main__.SelfTests.test_scanner_heal_multipool_case_accepts_deferred_optional_outage_write) ... ok test_scanner_heal_pending_gate_cannot_map_to_implemented_lane (__main__.SelfTests.test_scanner_heal_pending_gate_cannot_map_to_implemented_lane) ... ok test_scanner_heal_receipts_reject_reuse_and_missing_builds (__main__.SelfTests.test_scanner_heal_receipts_reject_reuse_and_missing_builds) ... ok test_scanner_heal_rejects_boolean_fractional_and_out_of_geometry_integers (__main__.SelfTests.test_scanner_heal_rejects_boolean_fractional_and_out_of_geometry_integers) ... ok test_scanner_heal_rejects_broken_execution_and_artifacts (__main__.SelfTests.test_scanner_heal_rejects_broken_execution_and_artifacts) ... ok test_scanner_heal_rejects_copied_junit_and_wrong_suite_build (__main__.SelfTests.test_scanner_heal_rejects_copied_junit_and_wrong_suite_build) ... ok test_scanner_heal_release_bundle_accepts_complete_measured_evidence (__main__.SelfTests.test_scanner_heal_release_bundle_accepts_complete_measured_evidence) ... ok test_scanner_heal_release_bundle_assembler_copies_measured_artifacts (__main__.SelfTests.test_scanner_heal_release_bundle_assembler_copies_measured_artifacts) ... ok test_scanner_heal_release_bundle_assembler_merges_measured_descriptors (__main__.SelfTests.test_scanner_heal_release_bundle_assembler_merges_measured_descriptors) ... ok test_scanner_heal_release_bundle_assembler_rejects_duplicate_descriptor_gate (__main__.SelfTests.test_scanner_heal_release_bundle_assembler_rejects_duplicate_descriptor_gate) ... ok test_scanner_heal_release_bundle_assembler_rejects_fixture_descriptor (__main__.SelfTests.test_scanner_heal_release_bundle_assembler_rejects_fixture_descriptor) ... ok test_scanner_heal_release_bundle_assembler_rejects_hash_mismatch (__main__.SelfTests.test_scanner_heal_release_bundle_assembler_rejects_hash_mismatch) ... ok test_scanner_heal_release_bundle_assembler_rejects_mixed_descriptor_revisions (__main__.SelfTests.test_scanner_heal_release_bundle_assembler_rejects_mixed_descriptor_revisions) ... ok test_scanner_heal_release_bundle_enforces_topology_duration_profile_and_versions (__main__.SelfTests.test_scanner_heal_release_bundle_enforces_topology_duration_profile_and_versions) ... ok test_scanner_heal_release_bundle_fixture_writer_is_not_release_evidence (__main__.SelfTests.test_scanner_heal_release_bundle_fixture_writer_is_not_release_evidence) ... ok test_scanner_heal_release_bundle_gate_accepts_single_measured_gate (__main__.SelfTests.test_scanner_heal_release_bundle_gate_accepts_single_measured_gate) ... ok test_scanner_heal_release_bundle_gate_rejects_missing_field (__main__.SelfTests.test_scanner_heal_release_bundle_gate_rejects_missing_field) ... ok test_scanner_heal_release_bundle_gate_rejects_unknown_gate (__main__.SelfTests.test_scanner_heal_release_bundle_gate_rejects_unknown_gate) ... ok test_scanner_heal_release_bundle_rejects_synthetic_or_missing_fields (__main__.SelfTests.test_scanner_heal_release_bundle_rejects_synthetic_or_missing_fields) ... ok test_scanner_heal_release_bundle_requires_domain_evidence (__main__.SelfTests.test_scanner_heal_release_bundle_requires_domain_evidence) ... ok test_scanner_heal_release_bundle_requires_field_provenance (__main__.SelfTests.test_scanner_heal_release_bundle_requires_field_provenance) ... ok test_scanner_heal_release_bundle_requires_json_artifact_provenance (__main__.SelfTests.test_scanner_heal_release_bundle_requires_json_artifact_provenance) ... ok test_scanner_heal_release_bundle_requires_mixed_version_field_roles (__main__.SelfTests.test_scanner_heal_release_bundle_requires_mixed_version_field_roles) ... ok test_scanner_heal_release_bundle_requires_same_gate_measurement_window (__main__.SelfTests.test_scanner_heal_release_bundle_requires_same_gate_measurement_window) ... ok test_scanner_heal_release_consumes_multiple_registry_oracles (__main__.SelfTests.test_scanner_heal_release_consumes_multiple_registry_oracles) ... ok test_scanner_heal_release_matrix_lanes_remain_pending_without_real_evidence (__main__.SelfTests.test_scanner_heal_release_matrix_lanes_remain_pending_without_real_evidence) ... ok test_scanner_heal_release_requirements_cannot_be_cleared (__main__.SelfTests.test_scanner_heal_release_requirements_cannot_be_cleared) ... ok test_scanner_heal_release_status_rejects_focused_case_run (__main__.SelfTests.test_scanner_heal_release_status_rejects_focused_case_run) ... ok test_scanner_heal_release_status_rejects_synthetic_case (__main__.SelfTests.test_scanner_heal_release_status_rejects_synthetic_case) ... ok test_scanner_heal_required_evidence_fields_cannot_be_removed (__main__.SelfTests.test_scanner_heal_required_evidence_fields_cannot_be_removed) ... ok test_scheduled_alerts_require_completion_watchdog (__main__.SelfTests.test_scheduled_alerts_require_completion_watchdog) ... ok test_update_profile_selection_changes_only_requested_platform (__main__.SelfTests.test_update_profile_selection_changes_only_requested_platform) ... ok test_validate_includes_quick_checks (__main__.SelfTests.test_validate_includes_quick_checks) ... ok test_vault_tests_require_cross_process_serialization (__main__.SelfTests.test_vault_tests_require_cross_process_serialization) ... ok test_workflow_readiness_requires_dependency_probe (__main__.SelfTests.test_workflow_readiness_requires_dependency_probe) ... ok ---------------------------------------------------------------------- Ran 52 tests in 14.301s OK e2e-full selection OK: 1 tests, sha256=29e30e6ed2ed38d9e5b0a7cdd9536abf5c719f7021a02b578ae1381563a3c273 ................................... ---------------------------------------------------------------------- Ran 35 tests in 1.161s OK ................... ---------------------------------------------------------------------- Ran 19 tests in 17.397s OK .......... ---------------------------------------------------------------------- Ran 10 tests in 0.183s OK test_actual_selector_bootstrap_uses_base_policy_and_fails_closed (__main__.SelfTests.test_actual_selector_bootstrap_uses_base_policy_and_fails_closed) ... ok test_documentation_paths_do_not_hide_build_or_fixture_changes (__main__.SelfTests.test_documentation_paths_do_not_hide_build_or_fixture_changes) ... ok test_e2e_scope_keeps_production_and_shared_configuration_full (__main__.SelfTests.test_e2e_scope_keeps_production_and_shared_configuration_full) ... ok test_e2e_shortcut_rejects_new_direct_renamed_and_target_dependencies (__main__.SelfTests.test_e2e_shortcut_rejects_new_direct_renamed_and_target_dependencies) ... ok test_e2e_shortcut_traverses_implicit_path_dependencies (__main__.SelfTests.test_e2e_shortcut_traverses_implicit_path_dependencies) ... ok test_event_contract_requires_complete_candidate_and_optional_lanes (__main__.SelfTests.test_event_contract_requires_complete_candidate_and_optional_lanes) ... ok test_every_wrong_result_missing_job_or_selection_fails_closed (__main__.SelfTests.test_every_wrong_result_missing_job_or_selection_fails_closed) ... ok test_full_e2e_gate_preserves_workflow_branch_and_event_scope (__main__.SelfTests.test_full_e2e_gate_preserves_workflow_branch_and_event_scope) ... ok test_git_range_includes_deleted_source_and_rename_origins (__main__.SelfTests.test_git_range_includes_deleted_source_and_rename_origins) ... ok test_job_ids_and_display_names_cannot_hide_validation (__main__.SelfTests.test_job_ids_and_display_names_cannot_hide_validation) ... ok test_report_only_stack_lane_mirrors_e2e_smoke_outside_the_aggregate (__main__.SelfTests.test_report_only_stack_lane_mirrors_e2e_smoke_outside_the_aggregate) ... ok test_repository_wiring_and_missing_dependency_regression (__main__.SelfTests.test_repository_wiring_and_missing_dependency_regression) ... ok test_verify_command_preserves_failures (__main__.SelfTests.test_verify_command_preserves_failures) ... ok ---------------------------------------------------------------------- Ran 13 tests in 3.280s OK security coverage self-test passed test_api_failures_preserve_other_evidence_and_never_enter_grace (__main__.SelfTests.test_api_failures_preserve_other_evidence_and_never_enter_grace) ... ok test_attempt_outcome_does_not_replace_recent_success (__main__.SelfTests.test_attempt_outcome_does_not_replace_recent_success) ... ok test_cancelled_attempt_cannot_refresh_expired_success (__main__.SelfTests.test_cancelled_attempt_cannot_refresh_expired_success) ... ok test_cli_requires_the_repository_default_branch (__main__.SelfTests.test_cli_requires_the_repository_default_branch) ... ok test_config_rejects_duplicate_and_invalid_entries (__main__.SelfTests.test_config_rejects_duplicate_and_invalid_entries) ... ok test_freshness_boundaries (__main__.SelfTests.test_freshness_boundaries) ... ok test_grace_requires_two_successful_queries_with_no_history (__main__.SelfTests.test_grace_requires_two_successful_queries_with_no_history) ... ok test_invalid_api_evidence_fails_closed (__main__.SelfTests.test_invalid_api_evidence_fails_closed) ... ok test_report_retains_every_workflow (__main__.SelfTests.test_report_retains_every_workflow) ... ok test_requests_filter_schedule_default_branch_and_success_on_server (__main__.SelfTests.test_requests_filter_schedule_default_branch_and_success_on_server) ... ok ---------------------------------------------------------------------- Ran 10 tests in 0.051s OK .............................. ---------------------------------------------------------------------- Ran 30 tests in 22.354s OK .......... ---------------------------------------------------------------------- Ran 10 tests in 2.969s OK ................................. ---------------------------------------------------------------------- Ran 33 tests in 0.160s OK Online functional runner labels: pf-testing ....... ---------------------------------------------------------------------- Ran 7 tests in 0.002s OK ....... ---------------------------------------------------------------------- Ran 7 tests in 0.001s OK .... ---------------------------------------------------------------------- Ran 4 tests in 0.128s OK ... ---------------------------------------------------------------------- Ran 3 tests in 0.547s OK PASS: cold stampede script self-test exit_code=0 $ python3 ./scripts/test_object_generation_protocol_model.py test_a_recovery_barrier_cannot_choose_an_empty_value (__main__.ObjectGenerationProtocolModel.test_a_recovery_barrier_cannot_choose_an_empty_value) ... ok test_all_bounded_two_proposer_interleavings_choose_at_most_one_value (__main__.ObjectGenerationProtocolModel.test_all_bounded_two_proposer_interleavings_choose_at_most_one_value) ... ok test_an_unavailable_decision_quorum_cannot_choose_a_candidate (__main__.ObjectGenerationProtocolModel.test_an_unavailable_decision_quorum_cannot_choose_a_candidate) ... ok test_counter_allocation_requires_persistence_and_is_consumed_once (__main__.ObjectGenerationProtocolModel.test_counter_allocation_requires_persistence_and_is_consumed_once) ... ok test_counter_restart_burns_persisted_allocations_before_or_after_issue (__main__.ObjectGenerationProtocolModel.test_counter_restart_burns_persisted_allocations_before_or_after_issue) ... ok test_crash_after_promise_before_reply_does_not_restore_an_older_ballot (__main__.ObjectGenerationProtocolModel.test_crash_after_promise_before_reply_does_not_restore_an_older_ballot) ... ok test_duplicate_or_unconfigured_voters_cannot_form_a_quorum (__main__.ObjectGenerationProtocolModel.test_duplicate_or_unconfigured_voters_cannot_form_a_quorum) ... ok test_empty_or_unobserved_minority_slot_can_retry_without_allocating_another_revision (__main__.ObjectGenerationProtocolModel.test_empty_or_unobserved_minority_slot_can_retry_without_allocating_another_revision) ... ok test_equal_ballot_cannot_accept_different_bytes_and_lower_ballot_is_rejected (__main__.ObjectGenerationProtocolModel.test_equal_ballot_cannot_accept_different_bytes_and_lower_ballot_is_rejected) ... ok test_forgetting_tombstone_authority_allows_an_aba_create (__main__.ObjectGenerationProtocolModel.test_forgetting_tombstone_authority_allows_an_aba_create) ... ok test_issuing_before_persistence_reuses_a_ballot_after_restart (__main__.ObjectGenerationProtocolModel.test_issuing_before_persistence_reuses_a_ballot_after_restart) ... ok test_lost_ack_and_restart_preserve_the_chosen_operation (__main__.ObjectGenerationProtocolModel.test_lost_ack_and_restart_preserve_the_chosen_operation) ... ok test_minority_accepted_work_is_adopted_and_finished (__main__.ObjectGenerationProtocolModel.test_minority_accepted_work_is_adopted_and_finished) ... ok test_omitting_highest_accepted_adoption_has_a_double_choice_counterexample (__main__.ObjectGenerationProtocolModel.test_omitting_highest_accepted_adoption_has_a_double_choice_counterexample) ... ok test_quorum_choice_does_not_require_an_uncontacted_disk_to_learn_it (__main__.ObjectGenerationProtocolModel.test_quorum_choice_does_not_require_an_uncontacted_disk_to_learn_it) ... ok test_recovery_failure_or_stale_expected_head_does_not_allocate_a_successor (__main__.ObjectGenerationProtocolModel.test_recovery_failure_or_stale_expected_head_does_not_allocate_a_successor) ... ok test_replacing_voters_with_empty_state_can_choose_a_conflicting_value (__main__.ObjectGenerationProtocolModel.test_replacing_voters_with_empty_state_can_choose_a_conflicting_value) ... ok test_skipping_predecessor_recovery_chooses_a_broken_revision_chain (__main__.ObjectGenerationProtocolModel.test_skipping_predecessor_recovery_chooses_a_broken_revision_chain) ... ok test_stale_publication_rollback_and_gc_cannot_change_the_winner (__main__.ObjectGenerationProtocolModel.test_stale_publication_rollback_and_gc_cannot_change_the_winner) ... ok test_superseded_operation_identity_cannot_be_reused_for_different_input (__main__.ObjectGenerationProtocolModel.test_superseded_operation_identity_cannot_be_reused_for_different_input) ... ok test_superseded_operation_retry_returns_its_original_successor (__main__.ObjectGenerationProtocolModel.test_superseded_operation_retry_returns_its_original_successor) ... ok test_tombstone_and_null_marker_payloads_do_not_reset_authority_revision (__main__.ObjectGenerationProtocolModel.test_tombstone_and_null_marker_payloads_do_not_reset_authority_revision) ... ok test_tombstone_rejects_a_delayed_never_present_create (__main__.ObjectGenerationProtocolModel.test_tombstone_rejects_a_delayed_never_present_create) ... ok test_unlearned_predecessor_is_recovered_before_the_second_slot (__main__.ObjectGenerationProtocolModel.test_unlearned_predecessor_is_recovered_before_the_second_slot) ... ok ---------------------------------------------------------------------- Ran 24 tests in 0.958s OK model: states=51023, transitions=167520, terminal=732, chosen_candidates=2 negative control: double-choice trace=((0, 'prepare', 0), (0, 'prepare', 1), (0, 'prepare', 2), (0, 'accept', 0), (0, 'accept', 1), (0, 'accept', 2), (1, 'prepare', 0), (1, 'prepare', 1), (1, 'prepare', 2), (1, 'accept', 0), (1, 'accept', 1), (1, 'accept', 2)) multi-slot round schedules: 640 exit_code=0 $ python3 ./scripts/check_test_wiring.py OK: e2e modules, runner selection, fuzz matrices, profiles, scheduled alerts, and Quick Checks are wired exit_code=0 $ ./scripts/check_no_planning_docs.sh exit_code=0 $ ./scripts/check_uring_lane_lib_only.sh OK: no 'uring_'-matching test functions under crates/ecstore/tests; --lib is safe exit_code=0 Initial complete gate $ cargo nextest run --profile ci -p rustfs --test connect_perf_drive --no-fail-fast Nextest run ID e07fc5b9-c6c8-4cdd-ae61-54839bf24437 with nextest profile: ci Starting 10 tests across 1 binary PASS [ 0.016s] rustfs::connect_perf_drive cancellation_and_single_collector_leave_no_scratch PASS [ 0.011s] rustfs::connect_perf_drive cleanup_failure_overrides_cancellation_and_remains_attributed PASS [ 0.009s] rustfs::connect_perf_drive consent_and_resource_budgets_fail_before_scratch_io PASS [ 0.009s] rustfs::connect_perf_drive invalid_roots_and_existing_scratch_fail_without_clobbering PASS [ 16.950s] rustfs::connect_perf_drive production_cli_measures_and_signs_exact_binary_provenance PASS [ 4.025s] rustfs::connect_perf_drive production_cli_rejects_missing_l1_consent_and_limits_before_artifacts PASS [ 0.008s] rustfs::connect_perf_drive read_only_root_and_full_drive_errors_are_classified PASS [ 0.015s] rustfs::connect_perf_drive real_local_write_read_is_measured_and_scratch_is_removed PASS [ 0.007s] rustfs::connect_perf_drive runtime_timeout_and_full_drive_use_schema_valid_failed_reason PASS [ 0.035s] rustfs::connect_perf_drive successful_result_has_signed_bounded_private_offline_export ──────────── Summary [ 21.087s] 10 tests run: 10 passed, 0 skipped exit_code=0 Restored complete gate $ cargo nextest run --profile ci -p rustfs --test connect_perf_drive --no-fail-fast Nextest run ID 233b7307-5361-4eb7-94d5-b384f17d2d6a with nextest profile: ci Starting 10 tests across 1 binary PASS [ 0.009s] rustfs::connect_perf_drive cancellation_and_single_collector_leave_no_scratch PASS [ 0.008s] rustfs::connect_perf_drive cleanup_failure_overrides_cancellation_and_remains_attributed PASS [ 0.008s] rustfs::connect_perf_drive consent_and_resource_budgets_fail_before_scratch_io PASS [ 0.008s] rustfs::connect_perf_drive invalid_roots_and_existing_scratch_fail_without_clobbering PASS [ 19.518s] rustfs::connect_perf_drive production_cli_measures_and_signs_exact_binary_provenance PASS [ 4.146s] rustfs::connect_perf_drive production_cli_rejects_missing_l1_consent_and_limits_before_artifacts PASS [ 0.009s] rustfs::connect_perf_drive read_only_root_and_full_drive_errors_are_classified PASS [ 0.018s] rustfs::connect_perf_drive real_local_write_read_is_measured_and_scratch_is_removed PASS [ 0.009s] rustfs::connect_perf_drive runtime_timeout_and_full_drive_use_schema_valid_failed_reason PASS [ 0.054s] rustfs::connect_perf_drive successful_result_has_signed_bounded_private_offline_export ──────────── Summary [ 23.787s] 10 tests run: 10 passed, 0 skipped exit_code=0 Mutation M1: success_measurement reports max(actual elapsed, requested budget + 1), forcing 1001 ms for the one-second request. The retained duration assertion at rustfs/tests/connect_perf_drive.rs:105 fails. $ cargo nextest run --profile ci -p rustfs --test connect_perf_drive --no-fail-fast -E 'test(real_local_write_read_is_measured_and_scratch_is_removed)' info: experimental features enabled: setup-scripts Compiling rustfs v1.0.1 (/private/tmp/gw-orchestrator-2734/ci-drive/rustfs) warning: linker stderr: ld: __eh_frame section too large (max 16MB) to encode dwarf unwind offsets in compact unwind table, performance of exception handling might be affected | = note: `#[warn(linker_messages)]` on by default warning: `rustfs` (bin "rustfs") generated 1 warning warning: `rustfs` (bin "rustfs-cli") generated 1 warning (1 duplicate) Finished `test` profile [unoptimized + debuginfo] target(s) in 7m 06s ──────────── Nextest run ID 9f618777-7bf6-4cbc-8803-dd7f50f449ed with nextest profile: ci Starting 1 test across 1 binary (9 tests skipped) FAIL [ 0.016s] rustfs::connect_perf_drive real_local_write_read_is_measured_and_scratch_is_removed stdout ─── running 1 test test real_local_write_read_is_measured_and_scratch_is_removed ... FAILED failures: failures: real_local_write_read_is_measured_and_scratch_is_removed test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 9 filtered out; finished in 0.01s stderr ─── thread 'real_local_write_read_is_measured_and_scratch_is_removed' (106986846) panicked at rustfs/tests/connect_perf_drive.rs:105:5: assertion failed: (1..=1_000).contains(&data.duration_millis) note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace ──────────── Summary [ 0.017s] 1 test run: 0 passed, 1 failed, 9 skipped FAIL [ 0.016s] rustfs::connect_perf_drive real_local_write_read_is_measured_and_scratch_is_removed error: test run failed exit_code=100 Mutation M2: target_parameters reports requested duration + 1. The in-process assertion at rustfs/tests/connect_perf_drive.rs:111 and CLI JSON assertion at rustfs/tests/connect_perf_drive.rs:527 both reject 1001 against 1000. $ cargo nextest run --profile ci -p rustfs --test connect_perf_drive --no-fail-fast -E 'test(real_local_write_read_is_measured_and_scratch_is_removed) | test(production_cli_measures_and_signs_exact_binary_provenance)' info: experimental features enabled: setup-scripts Compiling rustfs v1.0.1 (/private/tmp/gw-orchestrator-2734/ci-drive/rustfs) warning: linker stderr: ld: __eh_frame section too large (max 16MB) to encode dwarf unwind offsets in compact unwind table, performance of exception handling might be affected | = note: `#[warn(linker_messages)]` on by default warning: `rustfs` (bin "rustfs") generated 1 warning warning: `rustfs` (bin "rustfs-cli") generated 1 warning (1 duplicate) Finished `test` profile [unoptimized + debuginfo] target(s) in 7m 20s ──────────── Nextest run ID b73d46ab-c910-4e1e-8063-7fc7e332ec15 with nextest profile: ci Starting 2 tests across 1 binary (8 tests skipped) FAIL [ 13.968s] rustfs::connect_perf_drive production_cli_measures_and_signs_exact_binary_provenance stdout ─── running 1 test test production_cli_measures_and_signs_exact_binary_provenance ... FAILED failures: failures: production_cli_measures_and_signs_exact_binary_provenance test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 9 filtered out; finished in 13.96s stderr ─── thread 'production_cli_measures_and_signs_exact_binary_provenance' (107015462) panicked at rustfs/tests/connect_perf_drive.rs:527:5: assertion `left == right` failed left: Number(1001) right: 1000 note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace FAIL [ 0.015s] rustfs::connect_perf_drive real_local_write_read_is_measured_and_scratch_is_removed stdout ─── running 1 test test real_local_write_read_is_measured_and_scratch_is_removed ... FAILED failures: failures: real_local_write_read_is_measured_and_scratch_is_removed test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 9 filtered out; finished in 0.01s stderr ─── thread 'real_local_write_read_is_measured_and_scratch_is_removed' (107016667) panicked at rustfs/tests/connect_perf_drive.rs:111:5: assertion `left == right` failed left: 1001 right: 1000 note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace ──────────── Summary [ 13.985s] 2 tests run: 0 passed, 2 failed, 8 skipped FAIL [ 13.968s] rustfs::connect_perf_drive production_cli_measures_and_signs_exact_binary_provenance FAIL [ 0.015s] rustfs::connect_perf_drive real_local_write_read_is_measured_and_scratch_is_removed error: test run failed exit_code=100 Transient production mutations were restored byte-for-byte before the complete green rerun. Original/restored SHA256: 8d389e6d8836c041aab8c6a93e56addd65ba3f70b33264e1c74f639d0572a2da. $ git diff --check exit_code=0 $ git diff --cached --check exit_code=0 Independent orchestrator acceptance Tested code tree: e67c72fb01 Tested tree: 16fbbd2ea4 shellcheck --version && actionlint -> exit 0 cargo fmt --all --check -> exit 0 ./scripts/check_unsafe_code_allowances.sh -> exit 0 ./scripts/check_layer_dependencies.sh -> exit 0 ./scripts/check_architecture_migration_rules.sh -> exit 0 ./scripts/check_logging_guardrails.sh -> exit 0 ./scripts/check_error_other_format_ratchet.sh -> exit 0 ./scripts/check_no_tokio_io_uring.sh -> exit 0 ./scripts/check_extension_schema_boundaries.sh -> exit 0 ./scripts/check_body_cache_whitelist.sh -> exit 0 ./scripts/check_s3s_footprint.sh -> exit 0 ./scripts/check_test_count_ratchet.sh -> exit 0 ./scripts/check_fips_wording.sh -> exit 0 ./scripts/check_embedded_secrets.sh -> exit 0 make script-tests -> exit 0 python3 ./scripts/test_object_generation_protocol_model.py -> exit 0 python3 ./scripts/check_test_wiring.py -> exit 0 ./scripts/check_no_planning_docs.sh -> exit 0 ./scripts/check_uring_lane_lib_only.sh -> exit 0 $ LANG=en_US.UTF-8 CARGO_BUILD_JOBS=1 CARGO_INCREMENTAL=0 cargo nextest run --profile ci -p rustfs --test connect_perf_drive --no-fail-fast info: experimental features enabled: setup-scripts warning: linker stderr: ld: __eh_frame section too large (max 16MB) to encode dwarf unwind offsets in compact unwind table, performance of exception handling might be affected | = note: `#[warn(linker_messages)]` on by default warning: `rustfs` (bin "rustfs-cli") generated 1 warning warning: `rustfs` (bin "rustfs") generated 1 warning (1 duplicate) Finished `test` profile [unoptimized + debuginfo] target(s) in 0.69s ──────────── Nextest run ID cd1edf4c-f1c0-456d-ad1a-76cc00ce05c9 with nextest profile: ci Starting 10 tests across 1 binary PASS [ 0.012s] rustfs::connect_perf_drive cancellation_and_single_collector_leave_no_scratch PASS [ 0.010s] rustfs::connect_perf_drive cleanup_failure_overrides_cancellation_and_remains_attributed PASS [ 0.010s] rustfs::connect_perf_drive consent_and_resource_budgets_fail_before_scratch_io PASS [ 0.009s] rustfs::connect_perf_drive invalid_roots_and_existing_scratch_fail_without_clobbering PASS [ 20.328s] rustfs::connect_perf_drive production_cli_measures_and_signs_exact_binary_provenance PASS [ 5.527s] rustfs::connect_perf_drive production_cli_rejects_missing_l1_consent_and_limits_before_artifacts PASS [ 0.013s] rustfs::connect_perf_drive read_only_root_and_full_drive_errors_are_classified PASS [ 0.019s] rustfs::connect_perf_drive real_local_write_read_is_measured_and_scratch_is_removed PASS [ 0.011s] rustfs::connect_perf_drive runtime_timeout_and_full_drive_use_schema_valid_failed_reason PASS [ 0.049s] rustfs::connect_perf_drive successful_result_has_signed_bounded_private_offline_export ──────────── Summary [ 25.991s] 10 tests run: 10 passed, 0 skipped exit_code=0 All 54 tracked Cargo.toml files parse without duplicate keys. git diff --check passed. The production source remains unchanged; the two deliberate implementation mutations above were restored before this complete green run. * fix(test): compare legacy body wire parity as raw bytes Lossy UTF-8 comparison concealed a changed binary response byte. Compare the already captured Vec<u8> values directly. Validation: - Binary mutant: migrated payload 0x84 vs legacy 0x83; old comparator passed 1/1, raw comparator failed 0/1 (exit 100); restored green. - Non-Swift body-drain mutant: untouched-body delegation test failed with empty vs payload (exit 100); restored handler without a diff. - cargo nextest run --profile ci -p rustfs-s3-types -p rustfs-protocols --features rustfs-protocols/swift --no-fail-fast: 543 tests run, 543 passed, 0 skipped after both mutants restored. - cargo fmt --all --check and git diff --check passed. Orchestrator acceptance (LANG=en_US.UTF-8): cargo nextest run --profile ci -p rustfs-s3-types -p rustfs-protocols --features rustfs-protocols/swift --no-fail-fast Summary [2.445s] 543 tests run: 543 passed, 0 skipped. All shared quick checks on the final code tree: shellcheck --version && actionlint -> exit 0 cargo fmt --all --check -> exit 0 ./scripts/check_unsafe_code_allowances.sh -> exit 0 ./scripts/check_layer_dependencies.sh -> exit 0 ./scripts/check_architecture_migration_rules.sh -> exit 0 ./scripts/check_logging_guardrails.sh -> exit 0 ./scripts/check_error_other_format_ratchet.sh -> exit 0 ./scripts/check_no_tokio_io_uring.sh -> exit 0 ./scripts/check_extension_schema_boundaries.sh -> exit 0 ./scripts/check_body_cache_whitelist.sh -> exit 0 ./scripts/check_s3s_footprint.sh -> exit 0 ./scripts/check_test_count_ratchet.sh -> exit 0 ./scripts/check_fips_wording.sh -> exit 0 ./scripts/check_embedded_secrets.sh -> exit 0 make script-tests -> exit 0 python3 ./scripts/test_object_generation_protocol_model.py -> exit 0 python3 ./scripts/check_test_wiring.py -> exit 0 ./scripts/check_no_planning_docs.sh -> exit 0 ./scripts/check_uring_lane_lib_only.sh -> exit 0 Final independent acceptance after the drive-test CI repair Parent: a99d8f4f89 Tested code tree: 93cbd96bf0 Tested tree: 075ae5161c2b62a96812bc1e3c904a7663b4f2f8 shellcheck --version && actionlint -> exit 0 cargo fmt --all --check -> exit 0 ./scripts/check_unsafe_code_allowances.sh -> exit 0 ./scripts/check_layer_dependencies.sh -> exit 0 ./scripts/check_architecture_migration_rules.sh -> exit 0 ./scripts/check_logging_guardrails.sh -> exit 0 ./scripts/check_error_other_format_ratchet.sh -> exit 0 ./scripts/check_no_tokio_io_uring.sh -> exit 0 ./scripts/check_extension_schema_boundaries.sh -> exit 0 ./scripts/check_body_cache_whitelist.sh -> exit 0 ./scripts/check_s3s_footprint.sh -> exit 0 ./scripts/check_test_count_ratchet.sh -> exit 0 ./scripts/check_fips_wording.sh -> exit 0 ./scripts/check_embedded_secrets.sh -> exit 0 make script-tests -> exit 0 python3 ./scripts/test_object_generation_protocol_model.py -> exit 0 python3 ./scripts/check_test_wiring.py -> exit 0 ./scripts/check_no_planning_docs.sh -> exit 0 ./scripts/check_uring_lane_lib_only.sh -> exit 0 $ LANG=en_US.UTF-8 CARGO_BUILD_JOBS=1 CARGO_INCREMENTAL=0 cargo nextest run --profile ci -p rustfs-s3-types -p rustfs-protocols --features rustfs-protocols/swift --no-fail-fast Finished `test` profile [unoptimized + debuginfo] target(s) in 10m 08s Nextest run ID 5c7ff102-ec6d-479a-86e0-810859b28bbf with nextest profile: ci Starting 543 tests across 8 binaries Summary [ 6.729s] 543 tests run: 543 passed, 0 skipped exit_code=0 The only source change is the raw-byte wire comparator. Cargo manifests and the ratchet baselines remain unchanged from the independently validated parent. The invalid-byte and consumed-request mutations remain recorded above; their implementation changes were restored. * fix: repair integration checks and IPC test readiness Wait for the RPC/locks test sockets to pass the existing owner and mode checks inside the original 5s readiness window and 10ms poll interval. Use the existing admin S3 carrier for the new recovery and quota error calls. Regenerate the test-count baseline for main's 118 existing tests across seven crates. Pair the private pagination helper's raw-scan inputs and remove the redundant Copy clone from the retirement event. Keep the original assertions, responses, authorization, locks and storage operations. The pagination body and raw-scan expressions at all five calls are byte-identical; reinserting only the Copy clone restores the entire base heal file. All ignored baseline values and s3s counts stay unchanged. Validation base: c47432e353 Validation tree: f081084bc5 Patch SHA-256: 2dcf0028691bfb2772edfb38e262829b04f4151832dcce823c9dd9f7434b52de Trace production-prefix SHA-256: fd994f218a21429d62866f4efb8eee1ee57b58f4a51865aa35d05a54ef1ccb45 All commands used LANG=en_US.UTF-8, CARGO_BUILD_JOBS=1, CARGO_INCREMENTAL=0 and CARGO_NET_OFFLINE=true in one exclusive target. Every gate retained the same tree and six source hashes before and after. 54 Cargo.toml files parse without duplicate keys. cargo clippy -p rustfs --all-targets -- -D warnings Finished `dev` profile [unoptimized + debuginfo] target(s) in 12m 50s Exit: 0. Wall time: 770.855s. The original 19 quick-check steps: shellcheck --version && actionlint -> exit 0 cargo fmt --all --check -> exit 0 ./scripts/check_unsafe_code_allowances.sh -> exit 0 ./scripts/check_layer_dependencies.sh -> exit 0 ./scripts/check_architecture_migration_rules.sh -> exit 0 ./scripts/check_logging_guardrails.sh -> exit 0 ./scripts/check_error_other_format_ratchet.sh -> exit 0 ./scripts/check_no_tokio_io_uring.sh -> exit 0 ./scripts/check_extension_schema_boundaries.sh -> exit 0 ./scripts/check_body_cache_whitelist.sh -> exit 0 ./scripts/check_s3s_footprint.sh -> exit 0 ./scripts/check_test_count_ratchet.sh -> exit 0 ./scripts/check_fips_wording.sh -> exit 0 ./scripts/check_embedded_secrets.sh -> exit 0 make script-tests -> exit 0 python3 ./scripts/test_object_generation_protocol_model.py -> exit 0 python3 ./scripts/check_test_wiring.py -> exit 0 ./scripts/check_no_planning_docs.sh -> exit 0 ./scripts/check_uring_lane_lib_only.sh -> exit 0 Wall time: 286.337s. cargo nextest run --profile ci -p rustfs --no-fail-fast Summary [ 217.017s] 5382 tests run: 5382 passed (4 leaky), 40 skipped Exit: 0. Wall time: 1324.414s. cargo nextest run --profile ci -p rustfs-s3-types -p rustfs-protocols --features rustfs-protocols/swift --no-fail-fast Summary [ 2.189s] 543 tests run: 543 passed, 0 skipped Exit: 0. Wall time: 123.192s. cargo nextest run --profile ci -p rustfs-ecstore list_objects_paginate --no-fail-fast Summary [ 0.170s] 8 tests run: 8 passed, 5921 skipped Exit: 0. Wall time: 261.324s. cargo nextest run --profile ci -p rustfs-heal durable_mrf_anchor_retires_only_after_exact_bucket_generation_is_deleted --no-fail-fast Summary [ 0.805s] 1 test run: 1 passed, 835 skipped Exit: 0. Wall time: 199.801s. Baseline regeneration: ./scripts/check_test_count_ratchet.sh --update wrote .config/test-count-baseline.txt: 52 crates, 21340 tests, 164 ignored Test-only increases: ecstore48, heal3, iam14, io-metrics2, obs3, policy14, rustfs34. Existing ignored values and all other rows are byte-identical. s3s footprint OK: files importing s3s is 195 (baseline files=195) s3s footprint OK: s3s s3_error! lines is 1584 (baseline s3_error_lines=1584) s3s footprint OK: ecstore files referencing s3s is 36 (baseline ecstore_files=36) Historical runtime red controls on f622 (same retained two assertions): cargo nextest run --profile ci -p rustfs --lib -E 'test(=connect::diagnostics::trace_runtime::tests::local_top_rpc_reads_service_http_completions_and_signs_offline) | test(=connect::diagnostics::trace_runtime::tests::local_top_locks_reads_service_manager_and_signs_offline)' --no-fail-fast Original path readiness with a temporary 500ms bind/seal gap: Summary [ 0.863s] 2 tests run: 0 passed, 2 failed, 5113 skipped Exit: 100. Sealed readiness in that same gap: Summary [ 5.621s] 2 tests run: 2 passed, 5113 skipped Exit: 0. Compiled M1: replace the helper predicate with path.exists(): Summary [ 0.870s] 2 tests run: 0 passed, 2 failed, 5113 skipped Exit: 100. Restore the sealed predicate byte-for-byte: Summary [ 5.640s] 2 tests run: 2 passed, 5113 skipped Exit: 0. Both reds observe mode755 with owner_matches=true and fail at the original RPC/locks unwraps with StateSecurity. M1 failure locations: trace_runtime.rs 2547:29 and 2713:29. No new assertions were introduced. cargo nextest run --profile ci -p rustfs --lib -E 'test(connect::diagnostics::trace_runtime::tests::)' --no-fail-fast Summary [ 20.748s] 28 tests run: 28 passed (1 leaky), 5087 skipped Exit: 0. This security sweep is historical; the full final unit gate above covers the actual c474 candidate. The first disk-stopped compile ran no tests and is excluded from red credit. All temporary seal instrumentation is removed from the final source. Final unit cleanup markers: 4 leaky, 40 original skips (exit0). The ld __eh_frame linker warning also appears verbatim in the original f622 run and the earlier c474 run; there are no new source warnings. * fix: remove redundant pending closure in disk test Pass the capture-free pending future factory as a function item in the existing slow-operation test. This fixes the redundant_closure error at disk_store.rs:2914 that stopped strict Clippy before the feature tests could run. The single-line test edit preserves the factory's result, warning and timeout behavior. Reversing the replacement reproduces the parent file byte for byte. All 220 assertion lines and the production prefix are unchanged; no tests, assertions, guards, profiles or baselines were added or relaxed. Original Linux CI evidence (run 37888552268 at 53245be9f): - RIO job113685483992: strict Clippy exit101, redundant closure at disk_store.rs:2914; feature tests skipped. - Workspace Test and Lint job113685484058: same strict Clippy exit101; nextest and doc tests skipped. These are lint failures, not runtime mutation kills or known flakes. Native macOS arm64 validation on tree7a5bf47e7533b29479bc85207416102f1337c566, LANG=en_US.UTF-8, one build job, CARGO_INCREMENTAL=0 and offline dependencies: shellcheck --version && actionlint -> exit 0 cargo fmt --all --check -> exit 0 ./scripts/check_unsafe_code_allowances.sh -> exit 0 ./scripts/check_layer_dependencies.sh -> exit 0 ./scripts/check_architecture_migration_rules.sh -> exit 0 ./scripts/check_logging_guardrails.sh -> exit 0 ./scripts/check_error_other_format_ratchet.sh -> exit 0 ./scripts/check_no_tokio_io_uring.sh -> exit 0 ./scripts/check_extension_schema_boundaries.sh -> exit 0 ./scripts/check_body_cache_whitelist.sh -> exit 0 ./scripts/check_s3s_footprint.sh -> exit 0 ./scripts/check_test_count_ratchet.sh -> exit 0 ./scripts/check_fips_wording.sh -> exit 0 ./scripts/check_embedded_secrets.sh -> exit 0 make script-tests -> exit 0 python3 ./scripts/test_object_generation_protocol_model.py -> exit 0 python3 ./scripts/check_test_wiring.py -> exit 0 ./scripts/check_no_planning_docs.sh -> exit 0 ./scripts/check_uring_lane_lib_only.sh -> exit 0 All 19 checks: exit0, 312.161s. $ cargo clippy -p rustfs -p rustfs-ecstore --all-targets --features rio-v2 -- -D warnings Finished `dev` profile [unoptimized + debuginfo] target(s) in 20m 11s exit0, 1213.210s total. $ cargo nextest run --profile ci -p rustfs-ecstore --features rio-v2 slow_disk_operation_warns_once_while_pending_and_preserves_result --no-fail-fast Finished `test` profile [unoptimized + debuginfo] target(s) in 4m 34s PASS [0.026s] rustfs-ecstore disk::disk_store::tests::slow_disk_operation_warns_once_while_pending_and_preserves_result Summary [ 0.170s] 1 test run: 1 passed, 5943 skipped exit0, 281.920s total; 5943 existing filter skips. warning: linker stderr: ld: __eh_frame section too large (max 16MB) to encode dwarf unwind offsets in compact unwind table, performance of exception handling might be affected This linker warning is retained; flags, profiles and allowances are unchanged. 54 tracked Cargo.toml files parsed without duplicate keys. git diff --check passed. No new assertions were introduced, so no new runtime mutation is claimed. Native compilation and the existing helper test do not substitute for actual Linux RIO execution; the new integration CI run still needs to recover before the RPC merge. Preservation and output hashes: parent file SHA256: 8e59ee745e3aaaa22848b943d3b10f6353b515956282cde1f3af2c394200c9ac final file SHA256: b69ba3237061b161448ae8b62b615885d5770b98a5b4a2479bcfbd70f1bd1f17 unchanged production prefix SHA256: 7903a6e288252c420231242d7b58a25fa2a7140ea3dd430cbe7f0dde11369dcb RIO Linux failure log SHA256:421addb9868fd88e4109a922311e54f91f0dfd70862f8ba7fcaba193482ed90b Workspace Linux failure log SHA256:e0aea8d0f014c022252a9534968e083c98cfc1fd760da445ecd04ec3f522e26a 19 output SHA256:35a54f28d707956ed1edb29d492c4dfe8a9fe753d103b111b5a2f1b5c41c2b98 RIO Clippy output SHA256:d6046e3868f2b3992fe11cdd04e9e98eb50a87d1234d9a70dbfeb154ff61638d slow-operation output SHA256:605b7b258d2fd09c6ef2ab2ffc90d433d0509480f1e45125d20c823759e83608 Refs rustfs/backlog#2734. * test(ecstore): isolate tier-free recovery fixture startup The fixture constructor can take the production recovery lock and wait for 60 seconds before this test installs its queue and seeds metadata. Disable its background recovery only while creating this fixture, then restore the environment before calling the existing production recovery entrypoint. Only the existing test setup changes. Its production entrypoint, default lock parameters, original assertions, 30-second timeout, cancellation and cleanup remain intact. No tests or assertions were added. Production and store/init.rs are byte-identical to the parent. Independent acceptance ran on merge tree 8ec358cb42ba9ebb3bcfeafcef048a3f0366355b; the runner header reports parent HEAD, while before/after index and source hashes confirm this candidate. Worker task branch d8055766b8 has the same tree. Every command used LANG=en_US.UTF-8. Task compilation also used CARGO_BUILD_JOBS=1, CARGO_INCREMENTAL=0, CARGO_NET_OFFLINE=true, an independent target, and empty RUSTC_WRAPPER/RUSTC_WORKSPACE_WRAPPER to keep compilers in its owned process group without changing repository configuration. Command: shellcheck --version && actionlint Exit 0; observed output excerpt: version: 0.11.0 Log SHA-256: fc9eec16b4191d058a295813ca9afbe9d2b37b59975da860170726be162a3924 Command: cargo fmt --all --check Exit 0; observed output excerpt: (no stdout or stderr) Log SHA-256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 Command: ./scripts/check_unsafe_code_allowances.sh Exit 0; observed output excerpt: (no stdout or stderr) Log SHA-256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 Command: ./scripts/check_layer_dependencies.sh Exit 0; observed output excerpt: Layer dependency guard passed (no new reverse dependencies/cycles). Log SHA-256: 69c6cb1896c0586b28d1167380d3260d99d8017a4265c8df5a2348f289e93a12 Command: ./scripts/check_architecture_migration_rules.sh Exit 0; observed output excerpt: Architecture migration rules passed. Log SHA-256: 28790e70d070607ed4de0704f292a65030be92ec224ef106d4b72fd8296f896e Command: ./scripts/check_logging_guardrails.sh Exit 0; observed output excerpt: ✅ Logging guardrails check passed Log SHA-256: 7adf41abff8ed8f7617decf86498888ddde78937099dd9ce47f6d359dfc2d317 Command: ./scripts/check_error_other_format_ratchet.sh Exit 0; observed output excerpt: error(format!) ratchet passed (total call sites: 579). Log SHA-256: 66172aef18f150ba0605af122e26ffcf91f6dd86f43e38b05e2e982ce8d783ee Command: ./scripts/check_no_tokio_io_uring.sh Exit 0; observed output excerpt: (no stdout or stderr) Log SHA-256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 Command: ./scripts/check_extension_schema_boundaries.sh Exit 0; observed output excerpt: Extension schema boundaries passed. Log SHA-256: f13d0acd0e2ce5d9815241b12b4dab7d01cc11b9b65bcf6dfb7812c878532a51 Command: ./scripts/check_body_cache_whitelist.sh Exit 0; observed output excerpt: Body-cache whitelist guard passed. Log SHA-256: 6b828847b0383950ee6eef9cd846d0c322c945e47efd4c7d1ad881178810d6bb Command: ./scripts/check_s3s_footprint.sh Exit 0; observed output excerpt: s3s footprint OK: files importing s3s is 195 (baseline files=195) s3s footprint OK: s3s s3_error! lines is 1584 (baseline s3_error_lines=1584) s3s footprint OK: ecstore files referencing s3s is 36 (baseline ecstore_files=36) ✅ s3s footprint baseline check passed Log SHA-256: 47f53d04ef645618563fe1f22abc64667f19078f1974feedcca46e4f2c016844 Command: ./scripts/check_test_count_ratchet.sh Exit 0; observed output excerpt: test-count ratchet OK: 52 crates, 21340 tests, 164 ignored match .config/test-count-baseline.txt Log SHA-256: 618a5e330e933de0c6ff484546df401457024a88bb5986d11d588794e33dab60 Command: ./scripts/check_fips_wording.sh Exit 0; observed output excerpt: FIPS wording guard passed (README.md and CHANGELOG.md contain no forbidden claims). KMS wording guard passed (crates/kms claims no Vault KV2 Transit wrapping). Log SHA-256: 99221cc9979c480ec4f46f50d76c52b18c72fe499f12e15e4f4a22a1b8bfb140 Command: ./scripts/check_embedded_secrets.sh Exit 0; observed output excerpt: Embedded secret guard passed (no private key material or provider credential literal in the tree). Log SHA-256: 437a3ceba32094d2901889965ad191e411aad32cdb18bf0b8659e818011c406b Command: make script-tests Exit 0; observed output excerpt: test_cli_requires_the_repository_default_branch (__main__.SelfTests.test_cli_requires_the_repository_default_branch) ... ok test_config_rejects_duplicate_and_invalid_entries (__main__.SelfTests.test_config_rejects_duplicate_and_invalid_entries) ... ok test_freshness_boundaries (__main__.SelfTests.test_freshness_boundaries) ... ok test_grace_requires_two_successful_queries_with_no_history (__main__.SelfTests.test_grace_requires_two_successful_queries_with_no_history) ... ok test_invalid_api_evidence_fails_closed (__main__.SelfTests.test_invalid_api_evidence_fails_closed) ... ok test_report_retains_every_workflow (__main__.SelfTests.test_report_retains_every_workflow) ... ok test_requests_filter_schedule_default_branch_and_success_on_server (__main__.SelfTests.test_requests_filter_schedule_default_branch_and_success_on_server) ... ok Ran 10 tests in 0.042s OK Ran 30 tests in 20.627s OK Ran 10 tests in 2.268s OK Ran 33 tests in 0.127s OK Ran 7 tests in 0.002s OK Ran 7 tests in 0.001s OK Ran 4 tests in 0.108s OK Ran 3 tests in 0.564s OK PASS: cold stampede script self-test Log SHA-256: 75fda2d069a373c63667b908baaf99e6591aab9099a739b0c4c22ceb6da8d68c Command: python3 ./scripts/test_object_generation_protocol_model.py Exit 0; observed output excerpt: test_an_unavailable_decision_quorum_cannot_choose_a_candidate (__main__.ObjectGenerationProtocolModel.test_an_unavailable_decision_quorum_cannot_choose_a_candidate) ... ok test_counter_allocation_requires_persistence_and_is_consumed_once (__main__.ObjectGenerationProtocolModel.test_counter_allocation_requires_persistence_and_is_consumed_once) ... ok test_counter_restart_burns_persisted_allocations_before_or_after_issue (__main__.ObjectGenerationProtocolModel.test_counter_restart_burns_persisted_allocations_before_or_after_issue) ... ok test_crash_after_promise_before_reply_does_not_restore_an_older_ballot (__main__.ObjectGenerationProtocolModel.test_crash_after_promise_before_reply_does_not_restore_an_older_ballot) ... ok test_duplicate_or_unconfigured_voters_cannot_form_a_quorum (__main__.ObjectGenerationProtocolModel.test_duplicate_or_unconfigured_voters_cannot_form_a_quorum) ... ok test_empty_or_unobserved_minority_slot_can_retry_without_allocating_another_revision (__main__.ObjectGenerationProtocolModel.test_empty_or_unobserved_minority_slot_can_retry_without_allocating_another_revision) ... ok test_equal_ballot_cannot_accept_different_bytes_and_lower_ballot_is_rejected (__main__.ObjectGenerationProtocolModel.test_equal_ballot_cannot_accept_different_bytes_and_lower_ballot_is_rejected) ... ok test_forgetting_tombstone_authority_allows_an_aba_create (__main__.ObjectGenerationProtocolModel.test_forgetting_tombstone_authority_allows_an_aba_create) ... ok test_issuing_before_persistence_reuses_a_ballot_after_restart (__main__.ObjectGenerationProtocolModel.test_issuing_before_persistence_reuses_a_ballot_after_restart) ... ok test_lost_ack_and_restart_preserve_the_chosen_operation (__main__.ObjectGenerationProtocolModel.test_lost_ack_and_restart_preserve_the_chosen_operation) ... ok test_minority_accepted_work_is_adopted_and_finished (__main__.ObjectGenerationProtocolModel.test_minority_accepted_work_is_adopted_and_finished) ... ok test_omitting_highest_accepted_adoption_has_a_double_choice_counterexample (__main__.ObjectGenerationProtocolModel.test_omitting_highest_accepted_adoption_has_a_double_choice_counterexample) ... ok test_quorum_choice_does_not_require_an_uncontacted_disk_to_learn_it (__main__.ObjectGenerationProtocolModel.test_quorum_choice_does_not_require_an_uncontacted_disk_to_learn_it) ... ok test_recovery_failure_or_stale_expected_head_does_not_allocate_a_successor (__main__.ObjectGenerationProtocolModel.test_recovery_failure_or_stale_expected_head_does_not_allocate_a_successor) ... ok test_replacing_voters_with_empty_state_can_choose_a_conflicting_value (__main__.ObjectGenerationProtocolModel.test_replacing_voters_with_empty_state_can_choose_a_conflicting_value) ... ok test_skipping_predecessor_recovery_chooses_a_broken_revision_chain (__main__.ObjectGenerationProtocolModel.test_skipping_predecessor_recovery_chooses_a_broken_revision_chain) ... ok test_stale_publication_rollback_and_gc_cannot_change_the_winner (__main__.ObjectGenerationProtocolModel.test_stale_publication_rollback_and_gc_cannot_change_the_winner) ... ok test_superseded_operation_identity_cannot_be_reused_for_different_input (__main__.ObjectGenerationProtocolModel.test_superseded_operation_identity_cannot_be_reused_for_different_input) ... ok test_superseded_operation_retry_returns_its_original_successor (__main__.ObjectGenerationProtocolModel.test_superseded_operation_retry_returns_its_original_successor) ... ok test_tombstone_and_null_marker_payloads_do_not_reset_authority_revision (__main__.ObjectGenerationProtocolModel.test_tombstone_and_null_marker_payloads_do_not_reset_authority_revision) ... ok test_tombstone_rejects_a_delayed_never_present_create (__main__.ObjectGenerationProtocolModel.test_tombstone_rejects_a_delayed_never_present_create) ... ok test_unlearned_predecessor_is_recovered_before_the_second_slot (__main__.ObjectGenerationProtocolModel.test_unlearned_predecessor_is_recovered_before_the_second_slot) ... ok Ran 24 tests in 0.929s OK Log SHA-256: 0b4be926efb0609d497ae58930340153f01957b606206cbbdb3e0d47eb29d3a2 Command: python3 ./scripts/check_test_wiring.py Exit 0; observed output excerpt: OK: e2e modules, runner selection, fuzz matrices, profiles, scheduled alerts, and Quick Checks are wired Log SHA-256: 21f676f458451525535ab0ef73d6a9c0d9774e16d76a4d19e396ad34936dd39c Command: ./scripts/check_no_planning_docs.sh Exit 0; observed output excerpt: (no stdout or stderr) Log SHA-256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 Command: ./scripts/check_uring_lane_lib_only.sh Exit 0; observed output excerpt: OK: no 'uring_'-matching test functions under crates/ecstore/tests; --lib is safe Log SHA-256: 43f3f851e27ec2ab9e34d6b05fefdede38af992699c473904bfe6f53a46d52e3 Command: cargo nextest run --profile ci -p rustfs-ecstore --lib --features rio-v2 -E 'test(=bucket::lifecycle::bucket_lifecycle_ops::tests::tier_free_version_recovery_production_entrypoint_enqueues_seeded_item)' --no-fail-fast Exit 0; wall 234.399s. Observed output: Finished `test` profile [unoptimized + debuginfo] target(s) in 3m 49s Starting 1 test across 1 binary (5871 tests skipped) SETUP PASS [ 0.006s] ecstore-base-stack: sh -c 'echo RUST_MIN_STACK=4194304 >> "$NEXTEST_ENV"' PASS [ 0.440s] rustfs-ecstore bucket::lifecycle::bucket_lifecycle_ops::tests::tier_free_version_recovery_production_entrypoint_enqueues_seeded_item Summary [ 0.586s] 1 test run: 1 passed, 5871 skipped Log SHA-256: 34e973f86e8dc2102c1a0e483e0ac802ce7df3d1b40a603932f62c0805c4de27 Command: cargo nextest run --profile ci -p rustfs-ecstore --lib --features rio-v2 -E 'test(=bucket::lifecycle::bucket_lifecycle_ops::tests::tier_free_version_recovery_eight_coordinators_hold_one_leader_and_take_over) | test(=store::bucket::tests::creation_commit_backfill_runs_under_the_config_write_fence) | test(=store::bucket::tests::orphaned_bucket_recovery_requires_every_volume_absent_and_is_retryable)' --no-fail-fast Exit 0; wall 4.010s. Observed output: Finished `test` profile [unoptimized + debuginfo] target(s) in 0.43s Starting 3 tests across 1 binary (5869 tests skipped) SETUP PASS [ 0.005s] ecstore-base-stack: sh -c 'echo RUST_MIN_STACK=4194304 >> "$NEXTEST_ENV"' PASS [ 0.068s] rustfs-ecstore bucket::lifecycle::bucket_lifecycle_ops::tests::tier_free_version_recovery_eight_coordinators_hold_one_leader_and_take_over PASS [ 0.637s] rustfs-ecstore store::bucket::tests::creation_commit_backfill_runs_under_the_config_write_fence PASS [ 0.846s] rustfs-ecstore store::bucket::tests::orphaned_bucket_recovery_requires_every_volume_absent_and_is_retryable Summary [ 0.987s] 3 tests run: 3 passed, 5869 skipped Log SHA-256: b5da2a927ff18ca9aa217762f7186a6d4371bd9e8feedd7b77940e10b64dffac Command: cargo clippy -p rustfs -p rustfs-ecstore --all-targets --features rio-v2 -- -D warnings Exit 0; wall 1084.508s. Observed output: Finished `dev` profile [unoptimized + debuginfo] target(s) in 18m 03s Log SHA-256: 28f98be7970faaa71cd118da1da4e6e2e3d23b9dda7e6449eaa7a37e741260b5 Compiled mutation command (unchanged for each red and restored run): cargo nextest run --profile ci -p rustfs-ecstore --lib --features rio-v2 -E 'test(=bucket::lifecycle::bucket_lifecycle_ops::tests::tier_free_version_recovery_production_entrypoint_enqueues_seeded_item)' --no-fail-fast M1: reverse only this test-local constructor flag from false to true. The mutant compiled, then failed the original recovered-item queue assertion after 30.481s. Restoring the entire observed candidate passed in 0.460s. Red log 55bf3b1535529ac0bcfb8071f9b56f9f40631007a85c315367e7e93786d65293; restored log 204d5fbe4574a7917f5e740a795d864fe28a7e7fb5e33d27b0a9cf68e2f17f52. M2: replace only the actual awaited enqueue with false, consuming its original inputs. The mutant compiled, then failed the same original queue assertion after 30.516s. Restoring the entire observed candidate passed in 0.457s. Red log 89186330413215e231e2b06bf013cc31097ea8f4a106d91421a36d6c8986a7fd; restored log e848b42d2302401afa5dbcd9e37bf76d2e56a96ab6de5a488513b997f53c5e9b. All temporary observation and mutation code was removed before final acceptance. Historical baseline and diagnostic failures remain recorded; they are not mutation credit. The separate Object Lock intent-retry diagnostic still fails, and these native macOS checks do not establish Linux CI recovery. * refactor(rpc): serve internode rpc bodies with the owned Body (#8441) * test(rpc): pin the wire bytes of every internode rpc response body rustfs/backlog#2747 batch 4, step 1 of 2, on the s3s bodies the rpc paths serve today. Each response body builder in storage/rpc/http_service.rs is served through hyper's HTTP/1.1 server and the raw bytes the peer reads are compared with goldens captured from this commit: status, content-length or chunked framing, chunk boundaries, the terminating chunk, HEAD, and the cut connection a failure after data produces. The capability responses carry a per-process proof, so their length header and payload are checked against the body they encode instead. The read_file_stream response builder moves into read_file_stream_response so the test reaches the same code the handler runs; behaviour is unchanged. * refactor(rpc): serve internode rpc bodies with the owned Body rustfs/backlog#2747 batch 4, step 2 of 2. The internode rpc HTTP paths build their responses with rustfs_s3_types::Body: Body::from_stream replaces Body::from(StreamingBlob::wrap(..)) at the six stream sites, and the in-memory and empty bodies keep their constructors. InternodeRpcService converts its responses into the wrapped service's response body through From<Body>, so the server keeps composing it with the s3s service as before. The wire-byte goldens from the previous commit pass unchanged; storage/rpc no longer names s3s. * fix(rpc): simplify wire test response builder type Name the boxed response builder used by the capability wire test so all-target Clippy can check it without a type-complexity warning. Validation (LANG=en_US.UTF-8, CARGO_INCREMENTAL=0): - cargo clippy -p rustfs --all-targets -- -D warnings: exit 0. - cargo nextest run --profile ci -p rustfs --lib -E 'test(/storage::rpc::/)': 279 tests run, 279 passed, 4842 skipped. - All 19 quick-check commands, including script-tests, both Python scripts and cargo fmt --all --check: exit 0. - Footprint baseline regenerated: files=194, s3_error_lines=1584, ecstore_files=36; test-count baseline regenerated: 52 crates, 21228 tests, 164 ignored. - All 54 Cargo.toml files parse without duplicate keys. No assertion was added or changed by this lint fix. R1-R8 are retained in the existing PR verification record. * fix(ecstore): resume fenced Object Lock creation intents * fix(ecstore): bound creation retry futures on the stack * fix(ecstore): require native proof for creation retries * fix(ecstore): bound bucket creation stack usage (#8466) * fix(ecstore): retain creation fences through rollback Keep lifecycle, metadata-transaction, and namespace guards around rollback and metadata cleanup. Preserve the original publication error when a guard is lost. Build the timestamp retry fixture through the real interrupted producer so it checks a valid pending creation. Validated source tree: 4e8e207999. Two independent source reviews found no blocking regression. Validation (LANG=en_US.UTF-8): - bash -c 'shellcheck --version && actionlint' — exit 0 - cargo fmt --all --check — exit 0 - ./scripts/check_unsafe_code_allowances.sh — exit 0 - ./scripts/check_layer_dependencies.sh — exit 0 - ./scripts/check_architecture_migration_rules.sh — exit 0 - ./scripts/check_logging_guardrails.sh — exit 0 - ./scripts/check_error_other_format_ratchet.sh — exit 0 - ./scripts/check_no_tokio_io_uring.sh — exit 0 - ./scripts/check_extension_schema_boundaries.sh — exit 0 - ./scripts/check_body_cache_whitelist.sh — exit 0 - ./scripts/check_s3s_footprint.sh — exit 0 - ./scripts/check_test_count_ratchet.sh — exit 0 - ./scripts/check_fips_wording.sh — exit 0 - ./scripts/check_embedded_secrets.sh — exit 0 - make script-tests — exit 0 - python3 ./scripts/test_object_generation_protocol_model.py — exit 0 - python3 ./scripts/check_test_wiring.py — exit 0 - ./scripts/check_no_planning_docs.sh — exit 0 - ./scripts/check_uring_lane_lib_only.sh — exit 0 - cargo nextest run --profile ci -p rustfs-ecstore --lib --locked --no-fail-fast — 5894 passed, 0 skipped - cargo check -p rustfs-ecstore --all-targets --locked — exit 0 - cargo nextest run --profile ci -p rustfs --lib --locked --no-fail-fast -E 'test(=admin::handlers::table_catalog::tests::cold_strong_registration_completes_while_other_bucket_backup_waits) | test(=admin::handlers::table_catalog::tests::commit_backend_borrowed_put_preserves_owner_authorization_and_precondition) | test(=admin::handlers::table_catalog::tests::disable_table_bucket_preserves_objects_and_can_be_reenabled) | test(=admin::handlers::table_catalog::tests::load_table_handler_negotiates_vended_credentials_without_breaking_metadata_only_callers) | test(=admin::handlers::table_catalog::tests::namespace_property_handler_updates_object_backed_catalog_and_maps_errors) | test(=app::object::get::tests::execute_get_object_resumes_from_relocated_pool_without_splicing_body) | test(=table_catalog::store::tests::catalog_writer_loss_prevents_recovery_storage_commit)' — 7 passed - cargo nextest run --profile ci -p rustfs --lib --locked --no-fail-fast -E 'test(unsupported_remote_disk_)' — 2 passed Additional fault-injection evidence remains incomplete. Timestamp M1 reached its original assertion; its restore run is being collected. M2–M5 and the three planned rollback faults have not run. An automated risk check interrupted the timestamp proof task; no assertions or validation settings were weakened. * fix(ecstore): box nested bucket rollback state Keep the guarded rollback future behind a heap allocation so bucket creation does not store its nested state inline. The rollback body, all three fence references and pending-proof revocation order are unchanged. The same three original Linux tests overflowed on both e611 CI attempts. Local format, ecstore test compilation and all 19 quick-check steps pass; the original Linux feature lanes still need to validate this candidate. * test(ecstore): isolate rollback guard loss fixtures Run the three rollback metadata-cleanup cases in separate test processes so reused /proc/self/fd aliases from earlier cases cannot invalidate their creation witnesses. Keep the original fixture, guards, assertions and io_uring test selection, and verify that each child runs the intended case successfully. Update the generated test-count baseline for the added macro arm. Validation: cargo check -p rustfs-ecstore --tests; cargo fmt --all --check; all 19 quick-check commands, including make script-tests and the two Python checks. Linux io_uring acceptance and fault controls remain pending. * refactor(replication): move the replication config onto gateway types (#8443) * wip: T1.4 replication family * refactor(replication): move the replication config onto gateway types The replication configuration family leaves s3s: rustfs-replication no longer depends on s3s, and ECStore bucket metadata, the scanner and the rustfs app/admin layers hold rustfs_gateway_types::persistence::PersistedReplicationConfiguration. The persisted XML goes through the gateway persistence codec behind a new BucketConfigXml trait (the 39 byte goldens pass with unchanged fixtures); the legacy s3s edge converts once in storage/ecfs.rs through storage/s3_compat/replication.rs; the scanner data usage cache keeps its MessagePack bytes through a serde mirror pinned by bytes recorded from s3s 5761ddfe. rustfs-gateway-types 0.32.0 is pinned once (rev 772d3385). Refs rustfs/backlog#2745. * test: use S3 serializer for list response tests * fix(ci): restore integration test compilation Gate the diagnostic test import with its feature, initialize the conditional MetadataWriteStore fields explicitly, and decode the scanner replication fixture through its existing cache boundary. Preserve the original fixture and assertions. Validation was run on tree 544acb80c0 at 6003d4b8. LANG=en_US.UTF-8; no source changes follow validation. Quick checks: 19/19 passed. - shellcheck --version && actionlint: exit 0 - cargo fmt --all --check: exit 0 - ./scripts/check_unsafe_code_allowances.sh: exit 0 - ./scripts/check_layer_dependencies.sh: exit 0 - ./scripts/check_architecture_migration_rules.sh: exit 0 - ./scripts/check_logging_guardrails.sh: exit 0 - ./scripts/check_error_other_format_ratchet.sh: exit 0 - ./scripts/check_no_tokio_io_uring.sh: exit 0 - ./scripts/check_extension_schema_boundaries.sh: exit 0 - ./scripts/check_body_cache_whitelist.sh: exit 0 - ./scripts/check_s3s_footprint.sh: exit 0 - ./scripts/check_test_count_ratchet.sh: exit 0 - ./scripts/check_fips_wording.sh: exit 0 - ./scripts/check_embedded_secrets.sh: exit 0 - make script-tests: exit 0 - python3 ./scripts/test_object_generation_protocol_model.py: exit 0 - python3 ./scripts/check_test_wiring.py: exit 0 - ./scripts/check_no_planning_docs.sh: exit 0 - ./scripts/check_uring_lane_lib_only.sh: exit 0 Darwin validation: - cargo clippy --locked -p rustfs-ecstore -p rustfs-scanner --all-targets -- -D warnings: exit 0 - cargo nextest run --locked -p rustfs-ecstore -p rustfs-scanner --lib -E 'test(config::com::tests::save_config_preserves_retryable_system_volume_errors) | test(scanner_io::tests::replication_maintenance_refreshes_compacted_stats_across_cycles_and_retries)' --test-threads 1 --no-fail-fast: exit 0, 2/2 passed - cargo clippy --locked -p rustfs-ecstore -p rustfs-scanner --all-targets --features rustfs-ecstore/diagnostic-read-version-caller,rustfs-scanner/diagnostic-read-version-caller -- -D warnings: exit 101 - cargo nextest run --locked -p rustfs-ecstore -p rustfs-scanner --lib --features rustfs-ecstore/diagnostic-read-version-caller,rustfs-scanner/diagnostic-read-version-caller -E 'test(config::com::tests::save_config_preserves_retryable_system_volume_errors) | test(scanner_io::tests::replication_maintenance_refreshes_compacted_stats_across_cycles_and_retries) | test(cluster::rpc::runtime_sources::tests::) | test(config::com::tests::diagnostic_config_write_retains_origin_and_api_for_results_and_cancellation)' --test-threads 1 --no-fail-fast: exit 0, 14/14 passed Diagnostic strict Clippy remains red on four unchanged production lints: runtime_sources.rs if_same_then_else and three redundant_locals reports in set_disk/core/metadata_read.rs and set_disk/read.rs. Their source ranges and feature wiring match the base; the base was not compiled separately. Linux aarch64 validation (Rust/Clippy 1.99.0, UID 501, jobs=1): - cargo clippy --all-targets -- -D warnings: exit 0; Finished dev profile in 68m 12s - cargo clippy -p rustfs -p rustfs-ecstore --all-targets --features rio-v2 -- -D warnings: exit 0; Finished dev profile in 49m 45s Source HEAD/tree and clean status matched before and after both commands. No compiler or stack overrides were used. protoc 35.1 was installed from its verified ARM64 release; flatc was absent and not invoked by the workspace build scripts. The local environment differs from hosted CI. * fix(ecstore): recover delete batches with missing versions Treat an explicit non-null version as a no-op when it is absent from both recovery sources. Keep rejecting missing null versions, delete-marker mismatches and conflicting local generations. Add returning-disk and partial-receipt regressions, extend journal boundary checks, and regenerate the ecstore test-count baseline. Production faults exercise the 24 added assertion sites and 7 semantic checks: 33 faults failed at the expected sites and passed after full restoration. Validation (LANG=en_US.UTF-8, one build job): cargo nextest run --locked -p rustfs-ecstore --lib -E 'test(delete_recovery) | test(quorum_delete_cleanup_coordinator_binds_single_and_batch) | test(crash_delete_retry_batch_reclaims_inline_and_reserved_external)' --test-threads 1 --no-fail-fast Summary [ 34.580s] 69 tests run: 69 passed, 6058 skipped; exit 0 cargo clippy --locked -p rustfs-ecstore --all-targets -- -D warnings Finished `dev` profile [unoptimized + debuginfo] target(s) in 5m 37s; exit 0 All 19 final Quick Checks completed with exit 0: - shellcheck --version && actionlint - cargo fmt --all --check - ./scripts/check_unsafe_code_allowances.sh - ./scripts/check_layer_dependencies.sh - ./scripts/check_architecture_migration_rules.sh - ./scripts/check_logging_guardrails.sh - ./scripts/check_error_other_format_ratchet.sh - ./scripts/check_no_tokio_io_uring.sh - ./scripts/check_extension_schema_boundaries.sh - ./scripts/check_body_cache_whitelist.sh - ./scripts/check_s3s_footprint.sh - ./scripts/check_test_count_ratchet.sh - ./scripts/check_fips_wording.sh - ./scripts/check_embedded_secrets.sh - make script-tests - python3 ./scripts/test_object_generation_protocol_model.py - python3 ./scripts/check_test_wiring.py - ./scripts/check_no_planning_docs.sh - ./scripts/check_uring_lane_lib_only.sh test-count ratchet OK: 52 crates, 22009 tests, 165 ignored match .config/test-count-baseline.txt PASS: cold stampede script self-test model: states=51023, transitions=167520, terminal=732, chosen_candidates=2 multi-slot round schedules: 640 OK: e2e modules, runner selection, fuzz matrices, profiles, scheduled alerts, and Quick Checks are wired * fix(ecstore): retry tier commit intent lock contention Retry transient namespace lock timeouts within the existing three-attempt CAS budget. Reload and verify the original mutation identity after contention, and preserve the final typed error when attempts are exhausted. Cover save/read contention, exhausted retries, replacement identities and CAS reconciliation without changing the existing timeout or lock policy. Validation: - cargo nextest run --locked -p rustfs-ecstore --features test-util --lib -E 'test(tier_mutation)' --test-threads 1 --no-fail-fast: 52 passed. - cargo nextest run --locked -p rustfs-ecstore --features test-util --lib -E 'test(intent_advance_) | test(=services::tier::tier::tests::caller_cancellation_after_durable_prepare_does_not_hide_the_mutation)' --test-threads 1 --no-fail-fast: 5 passed. - cargo clippy --locked -p rustfs-ecstore --all-targets -- -D warnings: passed. - cargo clippy --locked -p rustfs-ecstore --all-targets --features test-util -- -D warnings: passed. - 27 mutation pairs failed at the intended assertions and passed after restoring the production code. - bash scripts/check_test_count_ratchet.sh --update, then bash scripts/check_test_count_ratchet.sh: 52 crates, 22013 tests, 165 ignored; ecstore increases from 6581 to 6585. * refactor(lifecycle): replace s3s domain types with Gateway DTOs Keep lifecycle rules, restore requests, Object Lock, and scanner cached tags in domain types. Convert at the existing HTTP boundary while preserving legacy XML, scanner cache bytes, date ordering, and tombstones. Linux test validation before the final import-path-only adjustment: cargo nextest run -p rustfs-ecstore bucket_config_goldens 39 passed; 6211 skipped by selector. cargo nextest run --lib -p rustfs -E 'test(/test_zero_rule_lifecycle_tombstone_recognition/) | test(/test_local_lifecycle_staleness_axis_selection/) | test(/test_lifecycle_staleness_preserves_legacy_submillisecond_axis/) | test(/test_merge_incoming_lifecycle/) | test(/test_lifecycle_expiry_statement_matrix/) | test(/test_lifecycle_expiry_subset_xml/) | test(/put_bucket_lifecycle_validation_errors_keep_their_s3_code/) | test(/execute_put_bucket_lifecycle_configuration_rejects_missing_configuration/) | test(/execute_restore_object_rejects_missing_restore_request/) | test(/execute_restore_object_rejects_select_type/) | test(/execute_restore_object_maps_failures_to_typed_s3_errors/) | test(/execute_restore_object_returns_internal_error_when_store_uninitialized/) | test(/restore_request_conversion_preserves_legacy_output_values/) | test(/restore_request_conversion_preserves_presence_and_raw_enums/) | test(/restore_request_conversion_distinguishes_absent_and_empty_containers/)' 25 passed; 5318 skipped by selector. cargo nextest run --profile ci -p rustfs-ecstore -p rustfs-lifecycle -p rustfs-replication -p rustfs-scanner 7872 passed; 21 skipped under the existing CI profile. Previously verified mutation controls: 39 production faults, 41 RED/GREEN pairs. cargo clippy --locked -p rustfs -p rustfs-ecstore -p rustfs-lifecycle -p rustfs-scanner --all-targets -- -D warnings Passed in 40m 45s before the import-path adjustment, then in 1m 59s on the final source with warnings denied. The final adjustment imports the existing helper through the object domain facade; fmt and the architecture check also passed. Part of rustfs/backlog#2745. The remaining ecstore configuration families follow separately. * refactor(admin): move error types to rustfs-s3-types (#8440) * refactor(protocols): move error types to rustfs-s3-types by codemod T1.2 batch 3 (rustfs/backlog#2743): s3select-api, s3select-query and the error items of protocols move off s3s by scripts/codemods, run with --hold crates/protocols/src/sftp/errors.rs. That file stays on s3s: it downcasts the backend error, whose runtime type is still s3s::S3Error. - rustfs/src/protocols/client.rs: the two webdav session methods now return rustfs_s3_types::S3Result and convert the s3s error with From. - codemod: a child module that calls s3_error! through `use super::*` gets its own `use rustfs_s3_types::s3_error;` once its parent imports that macro by name; otherwise it is listed as GLOB. - s3-types: a table test pins code, status and the s3s error the legacy edge renders for every variant a moved call site names. - s3_error_lines 1585 -> 1584 (a lifecycle comment reworded). * refactor(admin): move error types to rustfs-s3-types (WIP, no build) T1.2 batch 4 (rustfs/backlog#2743), work in progress: rustfs check still fails with 16 errors (listed in the PR body). - codemod: --facade PATH treats a crate-local re-export of s3s's error items like s3s for the four names (9 new self-tests, 9 mutants red). - codemod output for rustfs/src/admin and rustfs/src/site_replication (--facade crate::storage_api::site_replication::s3); 6 glob children get an explicit rustfs_s3_types::s3_error import. - admin/router.rs: the Operation trait returns rustfs_s3_types::S3Result. - error.rs: From<ApiError> builds the rustfs-s3-types error; the s3s conversion is derived from it. * fix(admin): convert error types at the s3s boundary Batch 4 of T1.2 left the admin and site_replication modules returning rustfs_s3_types::S3Error while their callers and callees on the storage and auth side still speak s3s::S3Error. Convert at each edge with the existing From impls in rustfs-s3-types (compat_s3s.rs) instead of changing any code, message or status: - admin/auth.rs, admin/handlers/object_zip_download.rs, admin/handlers/table_catalog/mod.rs: map the s3s result returned by check_key_valid / authorize_request / authorize_internal_object_request into the admin error type with .map_err(Into::into). - storage/ecfs.rs: the s3s-facing side maps the admin site_replication gate results the other way. - admin/handlers/pools.rs: ApiError.code stays s3s::S3ErrorCode, so the rustfs code is converted with .into() when the struct is built, in production and in the two tests that build or compare it. - admin/handlers/heal.rs, admin/runtime_sources.rs: the router returns s3s errors (S3Route impl); the expected rustfs code is converted with .into() before the comparison. - admin/handlers/table_catalog/mod.rs: iceberg_rest_error takes &'static str so the Cow<'static, str> custom code borrows it; every caller already passes an ICEBERG_ERROR_* constant or a &'static str. - admin/handlers/quota.rs: drop a now same-type .into() after s3_error!. - storage_api.rs, site_replication/mod.rs: drop re-exports and an import that no longer have users (clippy -D warnings). cargo check -p rustfs --all-targets and cargo clippy -p rustfs --all-targets -- -D warnings are clean; the s3s footprint baseline is unchanged at files=172, s3_error_lines=303, ecstore_files=28. Refs rustfs/backlog#2743, rustfs/backlog#2734. * refactor(ecstore): move bucket configs onto gateway persisted types (#8709) * refactor(ecstore): move bucket configs onto gateway persisted types Finish the ECStore batch of T1.4 (rustfs/backlog#2745). The remaining eleven bucket configuration families (Website, CORS, Tagging, Versioning, ObjectLock, Encryption, Notification, Logging, Accelerate, RequestPayment, PublicAccessBlock) now read and write their stored XML through the rustfs-gateway-types persistence and cors_tagging codecs instead of the s3s codec, so the legacy s3s bucket-config codec and its s3gate-metadata-xml compat marker are removed. - ecstore: BucketConfigXml is implemented by the gateway codec for every family; metadata, metadata_sys, versioning, object lock, replication and lifecycle consumers take the persisted types. Region moves to rustfs_s3_types::Region, and the ETag conversion to s3s moves to the rustfs edge with its tests. Only warm_backend.rs and warm_backend_s3.rs still reference s3s, held for T1.5 (rustfs/backlog#2746). - scanner: object lock cache, item actions and data usage use the persisted types. - rustfs: the legacy S3 edge converts each family between s3s DTOs and the persisted types in storage/storage_api.rs by writing one codec's XML and reading it with the other; Tagging converts field by field. - Bump the rustfs-gateway-types pin to 21283e87. The 39 persisted-byte goldens are unchanged and pass, so a document the old codec wrote reads back and re-encodes to the same bytes. s3s footprint: files 181 -> 146, ecstore_files 28 -> 2, s3_error_lines unchanged. * test(app): prepare the PutBucketEncryption config from the request The uninitialized-store test passed an empty persisted configuration, which the use case now validates before it reaches the store check, so it answered MalformedXML instead of InternalError. Convert the request's configuration the way the S3 edge does. * style(app): format the PutBucketEncryption test * refactor(s3-client): move remote error codes and DTO enums off s3s (#8783) Part of T1.5 (rustfs/backlog#2746), batch 1. * refactor(s3select): run queries on gateway DTO request shapes (#8794) * refactor(s3select): run queries on an engine request built from gateway DTO shapes Part of T1.5 (rustfs/backlog#2746), batch 2. * fix(s3select): keep the select request conversion in the app layer The layer guard forbids app -> storage; the conversion moves from storage/s3_compat/select.rs to app/select_request.rs unchanged. * fix(heal): retry transient metadata conflicts (#8812) * fix(heal): retry transient metadata conflicts * chore: update heal test count baseline * test: reserve nextest capacity for progress probes (#8818) * test(e2e): use a non-default region for the differential negative control Since main's region-aware CreateBucket and GetBucketLocation (#8739, #8743), us-east-1 answers an empty LocationConstraint and 200 for an owned re-create, so the control no longer reported its own region. Also refresh the e2e-full linux selection from the CI listing (682 tests). * refactor(protocols): move StorageBackend to gateway types (#8825) * refactor(protocols): move storage backend to gateway types * test: update e2e test count baseline * fix: preserve copy source keys and unknown lengths * test: update protocols test count baseline * fix(protocols): preserve typed SFTP backend errors * test(protocols): use gateway list input in policy tests * fix(protocols): clear clippy failures after T1.5 merge (#8831) fix(protocols): clear clippy failures after merge * fix(ecstore): finalize restore metadata after local copy-back (#8836) * fix(ecstore): finalize restore metadata after local copy-back * test(ecstore): ratchet added restore finalization cases * fix(ecstore): keep restore copyback compatible with older peers * fix(ecstore): gate restore copyback on fleet compatibility * fix(ecstore): retain transition identity during restore copyback * fix(ecstore): route committed restore copyback locally * fix restore copyback cleanup * fix(ecstore): honor restore probe test fallback * refactor(storage): migrate option errors to domain types (#8839) * refactor(s3): migrate private multipart validation errors (#8844) * refactor(auth): derive policy conditions from AuthzFacts (#8442) * refactor(auth): extract authorization facts * test(e2e): update smoke selection baseline * test(e2e): update full selection baseline * refactor(admin): migrate health and prefix usage to owned requests (#8887) * refactor(admin): add owned health router boundary * refactor(admin): migrate prefix usage to owned request * refactor(admin): complete the owned admin request boundary (#8949) * refactor(admin): complete the owned admin request boundary Every type implementing the owned admin `Operation` now serves the legacy router through one blanket impl that converts the s3s request and response at that single edge, so handlers register with `AdminOperation(&Handler {})` and need no adapter statics; `LegacyAdminAdapter` is removed. `AdminRequest` gains the accessors handlers read today (`remote_addr`, `client_info`, `request_context`, `req_info`, `req_info_mut`, `install_req_info`, `envelope_mut`, `take_body`), the request helpers gain envelope forms (`authenticate_envelope_credentials`, `object_store_from_envelope`, the table-catalog `*_from_envelope` helpers, `KmsAdminAudit::from_envelope`, `AccountAuditContext::from_envelope`, `CallerIdentity::resolve_envelope`), `read_compatible_admin_body` accepts either body type, and the owned `Body` gains `store_all_limited` and `Default`. `admin::operation::test_support::TestRequest` builds owned requests for handler tests. Unused accessors carry a per-method dead-code expectation that each migrating handler batch removes. No handler behavior changes; the remaining migrations follow in batches. * chore(admin): add the T1.12 admin handler codemod The script rewrites an admin handler file from the s3s request and response types to the owned boundary (imports, signatures, request field and helper calls, body access, response construction, and test request literals) and leaves a TODO comment at every site it cannot rewrite mechanically. It is idempotent and is removed once T1.12 completes. * refactor(admin): migrate replication and KMS handlers to owned requests (#8951) Move the T1.12 batch 2 admin handlers (site_replication, kms_keys, event, replication, diagnostics, on_demand_migration, audit, account, kms_rekey, kms_key_lifecycle, object_data_cache, tls_debug, integrity) from the s3s request/response types to the owned AdminRequest/AdminResponse boundary. Handler logic, authorization order, error codes and response bytes are unchanged; the request/response plumbing now goes through the envelope accessors and helpers introduced by the owned boundary. The shared response helpers (utils::json_response, utils::empty_response, handlers::admin_json_response, target_descriptor::build_json_response, profile::authorize_profile_request) and the site replication repair executor still answer with the s3s types, so the migrated files carry owned-boundary forms of those helpers and convert the repair response at its call edge. The streaming console-log body and the two devnull drains move from the s3s byte stream to the owned Body stream/frame API. Drop the dead_code expectations for the owned accessors and envelope helpers this batch starts using, and lower the s3s footprint baseline to the measured 99 files. Part of rustfs/backlog#2788 (rustfs/backlog#2734 T1.12). * refactor(admin): add owned forms of the shared admin response helpers Handler batches were copying `json_response`, `empty_response`, `admin_json_response`, and `build_json_response` into each migrated file because the shared helpers still return the s3s response. Provide owned forms next to each legacy helper (`owned_json_response`, `owned_empty_response`, `owned_admin_json_response`, `owned_build_json_response`, `authorize_profile_envelope`) with parity tests against the legacy shape, derive `Debug` for `AdminResponse` so tests can use `expect_err`, and fix three codemod rules: strip owned string conversions in test credentials, annotate non-literal URI parses, and leave a rebound request parameter immutable. * refactor(admin): use the shared owned response helpers in batch 2 (#8954) Replace the per-file owned-boundary copies of json_response, empty_response, admin_json_response, build_json_response and authorize_profile_request in the batch 2 admin handlers with the shared owned_* helpers and authorize_profile_envelope that the boundary branch now provides, and restore expect_err in the handler tests now that AdminResponse implements Debug; the `.err().expect(..)` stand-ins were failing the base branch's clippy step (clippy::err_expect under -D warnings). The site replication repair call edge keeps its conversion because the repair executor still answers with the s3s response type. Drop the dead_code expectations on the shared owned helpers this batch now uses in non-test code. Part of rustfs/backlog#2788 (rustfs/backlog#2734 T1.12). * fix(admin): preserve legacy body limit error responses * refactor(admin): move policy, ILM and batch 3 handlers to owned requests (#8956) Batch 3 of T1.12 (rustfs/backlog#2788, parent rustfs/backlog#2734) moves 14 admin handler files to AdminRequest/AdminResponse and the owned Operation trait: ilm_transition, policies, idp_compat, config_admin, plugins_instances, profile_admin, group, heal, object_zip_download, kms_key_metadata, durability, inspect_archive, sts and gateway_key_inventory. Handler logic, authorization order, error codes and response bytes are unchanged; test request literals use test_support::TestRequest. The handlers use the shared owned helpers (owned_json_response, owned_admin_json_response, owned_build_json_response, authorize_profile_envelope). The dead_code expectations on the AdminRequest accessors and RequestEnvelope::uri_mut they now exercise are removed. The legacy target_descriptor::build_json_response keeps only its equivalence test caller and carries the matching expectation until it retires. object_zip_download builds the s3s request the S3 gate still reads from the owned request facts; heal authenticates through authenticate_envelope_credentials; the trace, inspect-archive and zip bodies use Body::from_stream. extract_plugin_instance_filters takes the URI so extensions.rs keeps calling it, and the two idp_compat delegations stay on the legacy trait until their delegates migrate. The s3s footprint baseline drops from 99 to 90 files. * test(admin): anchor the replication diff body contract on the call The source-contract test looked for `let body = read_compatible_admin_body(` on one line; rustfmt wrapped that binding after the handler moved to the owned request boundary, so the marker was no longer found. Anchor on the call itself so formatting cannot change what the contract checks. * refactor(admin): migrate table catalog and batch 1 to owned requests (#8955) * fix(admin): preserve catalog body limit response bytes * refactor(admin): migrate identity and cluster handlers to owned requests (#8966) * refactor(admin): migrate identity and cluster handlers to owned requests Move the T1.12 batch 4 admin handlers (user, service_account, oidc, tier, system, pools, quota, kms_dynamic, kms_backup, rebalance, module_switch, bucket_meta, cluster_snapshot, realtime, bucket_recovery, profile) from the s3s request/response types to the owned AdminRequest/AdminResponse boundary. All 90 impl Operation blocks now take AdminRequest and return AdminResponse; handler logic, authorization order, error codes and response bytes are unchanged. The handlers use the shared owned helpers (owned_json_response, owned_admin_json_response, authorize_profile_envelope, the envelope-form runtime sources and audit constructors) instead of local copies. profile.rs drops the s3s forms of authorize_profile_request and profile_not_implemented_response now that profile_admin.rs is on the envelope gate too; profile_not_implemented_response answers with an AdminResponse. The realtime metrics stream moves from the s3s ByteStream adapter to Body::from_stream, and the rebalance idle-404 document still renders through s3s::S3Error::to_http_response with its body converted at the edge. The idp_compat delegators ImportIamV2 and AddServiceAccountLdap move to the owned trait as well: with their targets on both traits, the method-call delegation became ambiguous between the owned impl and the router blanket impl, so they now take the owned request directly. KmsAdminAudit::from_request loses its last production caller with this batch and is deleted; its comparison test now asserts the envelope facts directly. Lower the s3s footprint baseline to the measured 68 files. Part of rustfs/backlog#2788 (rustfs/backlog#2734 T1.12). * refactor(admin): remove the legacy admin helpers batch 4 superseded Batch 4 moved the last production callers of the s3s-typed admin helpers to their owned forms, which left `admin_json_response`, `json_response`, `authenticate_request`, `authorize_admin_request`, `object_store_from_req` and `object_store_from_extensions` unused, along with the `Body`, `S3Request` and `S3Response` re-exports of the admin `s3` facade. Remove them. The owned forms keep their documentation; the behavior tests of the removed `json_response` now exercise `owned_json_response`, and the compat-prefix test asserts the owned response directly. Two tests existed only to compare a removed legacy helper with its owned twin (`owned_json_response_matches_the_legacy_shape`) or to cover the removed legacy gate (`authorize_admin_request_without_credentials_is_rejected`); they go with the helpers, and the owned equivalents stay. The rustfs test-count row drops from 5957 to 5955 and the s3s footprint from 68 to 66 files. * refactor(admin): drop the legacy target descriptor JSON response helper (#8968) `target_descriptor::build_json_response` returned the s3s response and had no production caller left once the event, audit and plugin instance handlers moved to `owned_build_json_response`; it survived only for the legacy/owned equivalence test. Remove it, give the owned helper its own documentation, and make the test assert the owned response directly: status, `application/json`, the echoed `x-request-id` when present and only then, and the body bytes. The file no longer imports s3s, so the footprint baseline drops from 66 to 65 files. * test(ecstore): pin native history on cancellation under the cancellable queue main moved the local write path onto the disk blocking queue, which skips an operation once its waiter is dropped, so a borrowed write cancelled while it is queued no longer creates its file. The integration-only native publication test still expected the queued open to run. Assert the new execution model instead (the open never runs) while keeping its safety claims: the Started record is taken before queueing and the cancelled publication never regrants native history. * chore(gateway): raise test-count rows for tests merged from main * test(s3select): use the owned JSON input types in merged planner tests * fix(ecstore): scope the merged direct-writer test hook to test builds * fix(ecstore): create parents before admitted truncate opens Align the heal tests with removal-disabled preservation and local-only outcome fields, and refresh the e2e-full selection for the synced tests. * chore(gateway): update test-count baseline after main sync Main #9241 removed the node pool-metadata readiness cache with 8 tests and added 6, so the rustfs row drops from 6440 to 6438. The ecstore and s3select rows rise for tests added by #9237, #9239 and #9241. * feat(scripts): add junit_diff.py for dual-stack JUnit comparison (#9275) * test(e2e): allow a pinned release binary as differential reference side RUSTFS_E2E_REFERENCE_BINARY makes side A of the dual-stack differential a released rustfs binary, started without RUSTFS_S3_STACK. A missing, non-file or non-executable path fails the run instead of falling back to the same-binary mode. The report records which side is the reference and each side's --version output. * ci(gateway): make the test-count ratchet a floor, not exact (#9278) ci(gateway): make the test-count ratchet a floor instead of an exact baseline Tests below a row still fail; tests above it, a member without a row and ignores below the row now pass with a note. Ignores above a row still fail. Adds contract tests wired into make script-tests. * chore(gateway): record ignored microbenchmarks merged from main (#9290) chore(gateway): record the ignored microbenchmarks merged from main main added three throughput microbenchmarks that are ignored by design (#9217): one each in rustfs-checksums, rustfs-rio and rustfs-utils. The test-count ratchet treats a higher ignored count as a failure until the row is raised, so raise those three rows to 1 and tighten the test rows to the merged tree with --update. * test(e2e): update smoke selection for reference-binary unit tests * ci(s3-tests): add reference parity lane against pinned 1.0.5 (#9288) * ci(parity): run the dual-stack differential against the pinned reference Adds a differential job to reference-parity.yml: side A is the pinned release, side B the candidate artifact built for the s3-tests comparison. The job prints reference_side, version_a and version_b from the report, fails when side A is not the reference or both versions are equal, and lists every difference in the job summary. * ci(parity): print the first version line in the differential summary * chore: restore main's Cargo and baseline files after merging main --------- Co-authored-by: rustfs-cla-assistant[bot] <273121464+rustfs-cla-assistant[bot]@users.noreply.github.com>
ci(e2e): exclude reference unit tests from e2e-full selection (#9342 )
2026-11-11 20:54:09 +00:00 · 2026-11-11 15:15:12 -05:00 · 2026-11-11 14:13:10 -05:00 · 2026-11-11 14:12:54 -05:00 · 2026-11-11 14:07:20 -05:00 · 2026-11-11 12:16:15 -05:00
652 changed files with 68823 additions and 21768 deletions
@@ -1,43 +1,38 @@
 ---
 name: rustfs-release-publish
-description: Publish a RustFS version through Console, release-branch CI, preview acceptance, human confirmation, and final publication. Use only for an explicit release request.
+description: Publish a RustFS version through frozen Linux candidate acceptance, desktop completion, manifest approval, byte-preserving publication and post-release follow-up. Use only for an explicit release request.
 ---
 # RustFS Release

-Require the exact final SemVer target before version edits or publication. If unspecified, inspect current releases/tags and ask with concrete candidates; do not guess the channel or bump. A named bump type resolves to an exact version that still needs confirmation. Report the confirmed target verbatim; resolve any later mismatch before proceeding. Independent read-only preflight may continue.
+Require the exact final SemVer target before version edits or publication. If unspecified, inspect releases/tags and ask with concrete candidates; do not guess the channel or bump. Reuse the user's confirmed target and authorized delivery scope. Independent read-only preflight may continue.

 ## Invariants

- Cargo stores `<target>`, never `-preview.N`. Before publication, installation defaults retain the previous available deliverable. Final and preview binaries embed their own tag names, so version/asset names differ despite identical source.
- Use annotated tags without `v`. Preview names are exactly `<target>-preview.<digits>`; choose the next unused iteration after fetching tags. Preview Releases are prereleases, never Latest; publish only versioned assets/exact Docker tags, never latest assets, R2, Helm, moving Docker channels, or installation/banner updates.
- `release/<target>` owns the candidate. Retain its history, backport only reviewed fixes, and never merge main wholesale or force-push it. Pin `PREVIEW_HASH` to its exact commit with complete passing CI. Preview and final tags both point there; never tag a later main/branch HEAD or insert another bump.
- Phase failures block dependents. Keep preview Releases until final publication and notes generation finish; CI removes Releases while retaining tags. Never move an existing preview or final tag.
+- Cargo and every candidate executable contain the final product version. `<target>-preview.<run_number>` is an immutable candidate identifier, not a public tag or displayed binary version.
+- Freeze the exact source, reviewed workflow recipe, toolchain, Console, CLI, dependency lock, previous stable version and validation plan. Newer unrelated main or Console releases do not change those inputs.
+- Retain `release/<target>` history when a release branch is used; backport only reviewed fixes. Pin `CANDIDATE_SHA` to its exact CI-validated commit. Never force-push a release branch or move a public tag.
+- Build Linux first. Complete field acceptance before macOS/Windows and packaging. Full certification and final acceptance are required before publication; a partial Linux PASS grants no release authority.
+- Promote the accepted bytes. Final publication never recompiles binaries or rebuilds packages/images. Infrastructure retries reuse successful artifacts; changed source or inputs require a new candidate and affected acceptance.
+- Phase failures block dependents. No latest/default-channel update before versioned delivery. Use the [pipeline runbook](../../../docs/operations/release-pipeline.md) for executable entry points, locks and recovery.

-## Read by Phase
-
-Read only the current phase's procedure; retain completed evidence across turns. Follow root verification, Git/PR, and authorization rules.
+## Read by phase

 | Phase | Procedure |
 |---|---|
-| 0 | Clean tree, fetch main/tags, verify GitHub access, then complete the mandatory [Console release gate](references/console-gate.md). A green Console build alone is insufficient. |
-| 1–2 | [Prepare/reuse the release candidate and run exact-SHA branch CI](references/release-candidate.md). Source version changes use `rustfs-release-version-bump` with authorized delivery scope. |
-| 3 | [Verify the preview build, assets, channels, and notes](references/preview-release.md). |
-| 4–5 | [Run the downloaded binary, Console CRUD, and latest-rc matrix](references/preview-acceptance.md). Every check must pass. |
-| Confirmation | Complete the gate below and end the turn. |
-| 6 | After fresh confirmation, [publish and verify the final tag](references/final-release.md) at `PREVIEW_HASH`. |
-| 7 | [Reconcile Security Advisories, milestones, installation references, and the website; reconcile main and remove the completed release branch](references/post-release-updates.md) only after complete publication. |
+| 0 | Clean tree, fetch main/tags, verify access, then complete the mandatory [Console gate](references/console-gate.md). |
+| 1–2 | [Prepare the exact source and run branch CI](references/release-candidate.md). Version edits use `rustfs-release-version-bump`. |
+| 3 | [Build and verify the Linux candidate](references/preview-release.md). |
+| 4–5 | [Run artifact, field, Console and rc acceptance](references/preview-acceptance.md), then complete desktop/package/image/chart validation. |
+| Confirmation | Report the complete manifest and obtain the confirmation below. |
+| 6 | [Publish the accepted manifest](references/final-release.md). |
+| 7 | [Reconcile advisories, milestones, installation references, website and release branch](references/post-release-updates.md). |

-## Human Confirmation
+## Human confirmation

-After Phases 3–5 pass, recheck that the release branch still equals `PREVIEW_HASH` and that the current Console asset tag/digest match the preview's recorded build inputs. Report the target, branch, preview tag/hash, branch CI run, preview Release URL, accepted Console asset identity/result, and rc matrix. Explicitly ask whether to publish the final tag and end the turn without creating/pushing it.
+After all acceptance passes, report the target, source SHA, source CI run, Linux and completion run URLs, final manifest SHA-256, accepted Console/CLI identities, compatibility/performance results and backlog report. Explicitly ask whether to publish this exact manifest and end the turn without dispatching publication.

-Only a new affirmative user reply to that exact report authorizes Phase 6. The original release request, earlier confirmation, silence, or automated follow-up does not count. Confirmation binds the target, preview tag, hash, and accepted build inputs; a changed value or repeated/failed acceptance cycle invalidates it. Revalidate and obtain fresh confirmation before continuing.
+Only a new affirmative reply to that exact report authorizes Phase 6. An earlier general release request, silence or automatic continuation does not count. Reuse confirmation for retries of that same manifest; do not ask again merely because a destination failed. A changed source, input, manifest or acceptance result invalidates it and requires renewed acceptance/confirmation.

-## Recovery
+## Recovery and reporting

- Before the final tag exists, land fixes on main, backport to the candidate, verify target versions, and restart branch CI/acceptance. Use a new preview iteration once a preview was tagged; main may advance independently.
- If only an external build input changed, record it, create a new preview iteration even when the source SHA is unchanged, and repeat the affected build and artifact acceptance before fresh confirmation. Reuse source checks whose inputs remain valid.
- Once the final tag exists, retry publication jobs for that exact tag. A source fix requires a newly confirmed target, not retagging or a newer preview. Phase 7 retries recheck artifact availability and superseding releases.
- If abandoned after the bump merged, report the unpublished version on main and record whether to revert or let the next release overwrite it. Deleting unpublished candidate branches or preview tags requires an explicit decision. Completed release branches follow Phase 7 cleanup; keep release tags.
-
-Report completed phase results and incomplete work with evidence: Console previous/latest tags, release need, hash and run/Release URLs; target, branch CI/SHA, previews and hash, confirmation state; preview/final assets, latest-channel state, Console/rc results and cleanup; advisory reconciliation results and pending disclosure decisions; both milestone states, installation PR checks/merge state, and website text/link/PR/deployment state. Keep artifact publication separate from follow-up delivery; do not call an open PR a live update. Report deviations and their authorization.
+Resume failed infrastructure jobs without rebuilding successful units. Fix product defects on main, backport when needed, and start a new candidate. After a final tag exists, retry the same bytes or use a new patch version; never move the tag. Do not conflate implemented, CI-passed, field-accepted, approved, published and post-release-complete states. Report incomplete work and deviations with evidence; a tag or green build alone is not a completed release.
@@ -4,7 +4,7 @@ Read during Phase 0, before changing RustFS version files or tags. Follow the pa

 ### Console release gate

-Complete this gate before changing any RustFS version file or creating any RustFS tag. RustFS `build.yml` downloads the asset returned by `repos/rustfs/console/releases/latest`, so a successful Console build alone is insufficient.
+Complete this gate before changing any RustFS version file or creating any RustFS tag. Candidate creation resolves `repos/rustfs/console/releases/latest` once and freezes the exact asset for every build, so a successful Console build alone is insufficient. After freezing, later Console releases do not silently change or invalidate the accepted inputs.

 1. Read the latest published Console tag and compare it with Console `main`:

@@ -1,22 +1,13 @@
-# Final Release Publication
+# Final release publication

-Read only after fresh user confirmation of the preview acceptance report. The [parent confirmation gate](../SKILL.md#human-confirmation) applies.
+Read only after fresh confirmation of the complete candidate report. The [parent confirmation gate](../SKILL.md#human-confirmation) applies.

-## Phase 6 — Publish the final tag on the validated commit
+Dispatch `release-publish.yml` on main with the successful completion run ID and approved `release-manifest.json` SHA-256. Do not create the tag manually. The workflow revalidates source, bytes and trusted certification, then creates the annotated final tag at `CANDIDATE_SHA`. A recipe change fails closed and requires a new candidate; unrelated main changes do not require a rebuild.

-No second source version bump. Before creating the final tag, recheck the remote `release/<target>` HEAD equals `PREVIEW_HASH` and the current Console release tag/asset digest still match the accepted preview inputs. If either changed after confirmation, the approval no longer applies; validate the changed inputs with a new preview and obtain fresh confirmation. The final tag goes on the exact commit the preview validated:
+Require both versioned publication and channel publication jobs to succeed. Verify the final tag's commit and manifest identity; all six platform ZIPs and compatibility aliases; DEB/RPM, checksums, SBOM, provenance and evidence bundle; both multi-architecture image variants in Docker Hub, GHCR and Quay; and the accepted Helm chart in the chart repository. Download a host-compatible final artifact and confirm its digest is exactly the accepted candidate digest and its version/source output agrees.

-```bash
-git fetch origin --tags
-git rev-parse "<preview-tag>^{commit}"          # must equal PREVIEW_HASH — abort if not
-git tag -a "<target>" -m "Release <target>" "$PREVIEW_HASH"
-git push origin "<target>"
-```
+For stable releases verify GitHub Latest, R2 download aliases and `latest.json` unless a newer stable release already owns those channels. Prereleases must remain GitHub prereleases and leave stable pointers unchanged. Verify generated release notes compare against the frozen previous stable release, excluding internal previews.

- CI rebuilds from the same source with a different tag name, so the binary self-reports `<target>`. Other inputs can change: `build.yml` reads Console `releases/latest` during each build. Compare the actual Console tag, URL, and verified digest in every final build job with the accepted preview record, and inspect available build provenance for other relevant changes. A pre-tag check cannot prevent a later input change. On a mismatch or missing evidence, repeat affected acceptance against the final artifacts and report the limitation; failed acceptance leaves release validation incomplete even if publication already succeeded. Never move the final tag to conceal a failure.
- Verify the final tag's complete publication path: all matrix and release jobs green; `gh release view "<target>"` shows the full versioned and `-latest` asset set plus checksums, SBOM, and provenance; Docker and Helm workflows succeed; `latest.json` points to `<target>`. A stable target must have `isPrerelease=false` and `isLatest=true`. An alpha/beta/rc target must have `isPrerelease=true`; GitHub does not permit prereleases to be Latest, but the project `latest.json` still advances to the final non-preview target.
- Verify the final Release body contains `## What's Changed` and a Full Changelog link. When `PREVIOUS_DELIVERABLE` exists, the link MUST be `https://github.com/rustfs/rustfs/compare/<PREVIOUS_DELIVERABLE>...<target>` and the baseline MUST equal the preview Release baseline; for example, both `1.0.0-beta.12-preview.1` and `1.0.0-beta.12` compare from `1.0.0-beta.11`.
- Verify the preview cleanup: `cleanup-preview-releases` must succeed and `gh release view "<preview-tag>"` must then report `release not found` for every preview iteration of this target. Each surviving preview tag must still resolve to its own recorded commit; the accepted preview resolves to `PREVIEW_HASH`. If the job failed, delete the leftover Releases manually with `gh release delete "<preview-tag>" --yes` and report it.
- Verify a host-compatible final artifact's checksum and `./rustfs --version`; it must report `<target>` and the expected source SHA. This identity check does not replace affected acceptance when build inputs changed.
+On infrastructure failure retry the same workflow/seal; matching destinations are reused. Never clobber an asset, force a tag, rebuild the image or silently accept a changed digest. A source fix after the final tag requires a new patch version. Cross-registry writes are not atomic: report the unfinished destination rather than claiming publication complete.

-Never pass `--cleanup-tag` when removing preview Releases. Remove them manually only if `cleanup-preview-releases` failed, after final publication and notes generation. Record deleted Releases and surviving tags.
+New candidates create no preview Releases to clean up. For a legacy preview release, preserve its tag and remove its Release only after final delivery and notes are verified. Do not pass `--cleanup-tag`. Complete the [post-release follow-up](post-release-updates.md) separately.
@@ -24,7 +24,7 @@ Reconcile advisories after every completed final release, including Console-only

 1. Re-read the published deliverable state before edits or a retry. If a newer release has superseded this target, do not downgrade installation references or the website announcement. Report the superseding release and leave those defaults intact.
 2. Invoke `rustfs-release-version-bump` with stage `post-release`, the exact target, and the authorized delivery scope. Prepare the follow-up on current main; do not edit or move either validated release tag. Cargo versions may already be preparing the next target and must remain untouched.
-3. Verify the exact Docker image manifest, Release download links, RPM source archive, and Helm repository entry before publishing the installation PR. The chart workflow overrides chart/app versions from the final tag, pins the source checkout to that release, and waits for its default Docker image before packaging.
+3. Verify the exact Docker image manifest, Release download links, RPM source archive, and Helm repository entry before publishing the installation PR. The chart is packaged and tested before sealing; publication uploads those same bytes only after versioned Docker images are available.
 4. Report the installation PR and its merge state separately from the completed release. Reuse existing merge authorization and repository checks; do not infer new merge authority from this reference.

 ## rustfs.com top banner
@@ -54,4 +54,4 @@ Use the existing announcement configuration from [website PR #112](https://githu
 1. Compare the exact published tag with current main. Audit every release-side commit since their merge base, including inherited backports. An ahead count is a history difference, not proof of missing changes. Use ancestry and patch equivalence, then inspect current code and regression evidence for squashed, adapted, superseded, or reverted changes. Record each commit as present, superseded, release-only, or missing; a positive `git cherry` result alone does not prove a missing fix.
 2. Bring missing shared changes to main through a reviewed PR and the required checks before closing the follow-up. Preserve main's later changes; decide version bumps and pinned Console configuration separately. If a history merge is needed, use a merge commit and verify the release commit is an ancestor of main; squash does not establish that ancestry. Do not merge main into the published release branch or move its tags. Report unresolved changes instead of claiming reconciliation is complete.
 3. After publication and authorized follow-ups are complete, delete the remote `release/<target>` branch. Immediately before deletion, verify its HEAD still equals the published tag's commit and no open PR targets or originates from it. Preserve a branch with newer work or open PRs and report the blocker. Use a deletion with an expected-old-SHA check to avoid removing a concurrently advanced branch.
-4. Verify the remote branch is absent and the final and preview tags still resolve to their recorded commits. Remove a local release branch only if it is task-owned, unchecked out, and has no unpreserved work. Record cleanup separately from artifact publication; an explicitly authorized deletion before reconciliation does not make unresolved main changes complete.
+4. Verify the remote branch is absent and the final tag and any legacy preview tags still resolve to their recorded commits. Remove a local release branch only if it is task-owned, unchecked out, and has no unpreserved work. Record cleanup separately from artifact publication; an explicitly authorized deletion before reconciliation does not make unresolved main changes complete.
@@ -1,10 +1,10 @@
-# Preview Artifact Acceptance
+# Candidate Artifact Acceptance

-Read after Phase 3 succeeds. Complete every check below before the parent skill's manual confirmation gate. These checks cover the downloaded artifact, embedded Console, and latest rc client.
+Read after Phase 3 succeeds. Complete every check below before the parent skill's manual confirmation gate. These checks cover the downloaded artifact, embedded Console, and the rc release frozen in the candidate inputs.

 ## Phase 4 — Run the artifact locally, verify the console

-Use a unique task-owned scratch directory. Select the versioned archive matching the host OS, architecture, and runtime, download its checksum file, and verify the archive before extraction. Do not substitute a different build when a matching artifact is unavailable; report that acceptance gap. The shell example below assumes the verified executable is at `RUSTFS_BINARY` (`rustfs.exe` on Windows), `SCRATCH` is the task directory, and `S3_PORT` and `CONSOLE_PORT` are distinct available ports. Use equivalent native commands when the shell differs.
+Use a unique task-owned scratch directory. Select the versioned archive matching the acceptance host, and verify it against the sealed manifest before extraction. Before desktop completion, use the Linux acceptance environment rather than rebuilding for the local desktop. Do not substitute a different build when a matching artifact is unavailable; report that acceptance gap. The shell example below assumes the verified executable is at `RUSTFS_BINARY` (`rustfs.exe` on Windows), `SCRATCH` is the task directory, and `S3_PORT` and `CONSOLE_PORT` are distinct available ports. Use equivalent native commands when the shell differs.

 Bind both listeners to loopback. Choose task-only credentials, set loopback proxy exclusions for the server/client processes, and record the server PID. A port probe does not reserve a port; retry startup with fresh ports on a bind conflict rather than connecting to an existing service.

@@ -24,16 +24,16 @@ PREVIEW_SERVER_PID=$!

 Checks (all must pass):

- `"$RUSTFS_BINARY" --version` reports the preview tag name and the short SHA of `PREVIEW_HASH`. Reporting `<target>` without the `-preview.N` suffix means the build did not embed the tag — treat as FAIL and investigate before proceeding.
+- `"$RUSTFS_BINARY" --version` reports the exact final `<target>` and full `CANDIDATE_SHA`. A preview suffix, another version or another source is a failure.
 - Wait with a bounded readiness deadline while checking the recorded server process remains alive; `curl --noproxy '*' -fsS "$S3_ENDPOINT/health/ready"` must return ready from this process's listener.
 - Startup log shows the embedded console being served (this was the regression that `fix(release): require embedded console assets` guards).
 - Open `CONSOLE_ENDPOINT` in the browser and log in with the task credentials. The dashboard must render without JS console errors; create a bucket, upload a file, download it back (byte-identical), and delete the object and bucket. Keep the server running for Phase 5.

-## Phase 5 — Validate with the latest rc client
+## Phase 5 — Validate with the pinned rc client

 `rc` is the RustFS CLI client from <https://github.com/rustfs/cli>.

- Resolve the latest release with `gh api repos/rustfs/cli/releases/latest --jq .tag_name`; verify the local client's version or download and verify a host-compatible client into the task directory. Set `RC_BIN` to that executable and record its version/digest. Avoid changing the user's installed client for this check.
+- Read the CLI release and asset digests from `inputs.json`; verify the local client's version or download a host-compatible asset from that exact release into the task directory. If the host is a desktop, resolve its asset within the same pinned CLI release and record the digest; do not independently follow `latest`. Set `RC_BIN` to that executable and record its version/digest. Avoid changing the user's installed client for this check.
 - Inspect the client's current help for an isolated configuration option and use it when supported. Otherwise choose a unique `PREVIEW_ALIAS` and confirm it is absent before creating it; never overwrite an existing alias. Use task-owned bucket/user names and input/output files. The names below are safe only inside the newly started isolated server.
 - Point the client at `S3_ENDPOINT` with loopback proxy exclusions and run the command matrix, recording PASS/FAIL per command:

@@ -60,3 +60,9 @@ cmp <local-file> "$SCRATCH/out"
 ```

 - Any FAIL blocks acceptance. On success or failure, stop only the recorded task process and remove only task-created aliases/data after retaining the needed failure evidence. Do not erase an existing client configuration or unrelated server data.
+
+## Field acceptance and desktop completion
+
+Before accepting the Linux manifest, record Scanner/Heal, multi-pool, memory, major features, high-concurrency 503, performance and previous-version integrity/upgrade results in backlog. Preserve the release-specific agreed limits and baseline; a PASS label without artifact identities or test results does not satisfy acceptance.
+
+Dispatch `release-complete.yml` only after that report passes, using the exact Linux manifest digest and backlog URL. Require native macOS/Windows checks, package extraction, image startup/scans, chart tests and complete certification to pass. Complete any desktop Console/client checks applicable to the release against these same archives. Record the completion run and final manifest SHA-256 before entering the parent confirmation gate. Partial Linux acceptance does not authorize final publication.
@@ -1,25 +1,9 @@
-# Preview Release Verification
+# Linux candidate verification

-Read for Phase 3 after the candidate passed exact-SHA CI. The [parent release invariants](../SKILL.md) apply.
+Read for Phase 3 after the source passed exact-SHA CI. The [parent invariants](../SKILL.md) apply; commands and recovery live in the [pipeline runbook](../../../../docs/operations/release-pipeline.md).

-## Preview channels
+Dispatch `release-candidate.yml` on main with the confirmed final version and `CANDIDATE_SHA`. Record the run ID, frozen `inputs.json`, every Linux archive digest and `linux-manifest.json` SHA-256. Require the four Linux builds, native startup/S3 checks, previous-release upgrade matrix and Linux seal to succeed. Missing, skipped, cancelled or failed required work is incomplete acceptance.

- Use `<target>-preview.N` for every target, e.g. `1.0.0-beta.10-preview.3` or `1.1.0-preview.1`.
- The canonical suffix is exactly `-preview.<digits>`. `build.yml` recognizes it before alpha/beta/rc classification and routes it to the preview-only path; any other tag containing `-preview` fails closed instead of being treated as a release.
- A preview Release MUST be published with `isPrerelease=true` and `isLatest=false`. Any `*-latest` preview asset or preview-triggered `latest.json`, R2, Helm, or moving Docker channel tag is a pipeline failure. Docker images may use the exact preview version tag, including its existing variant suffixes.
- Preview Releases are cleaned up by the `cleanup-preview-releases` job after `publish-release` succeeds for the deliverable tag. It deletes every Release whose tag is exactly `<target>-preview.<digits>` and never passes `--cleanup-tag`, so the tags survive.
+Verify the frozen Console identity equals the one accepted in Phase 0. Every matrix job consumes that same digest and exact toolchain. The binary must report the final product version and source commit; its candidate ID is separate. No public tag, Release, registry tag, R2 release path, Helm repository or latest pointer should change during this stage.

-## Phase 3 — CI and preview Release verification
-
- Find and watch the tag build: `gh run list --workflow build.yml --branch "<preview-tag>" --limit 1` then `gh run watch <run-id>`. Every build matrix target must succeed (linux x86_64/aarch64 × musl/gnu, macos-aarch64, windows-x86_64).
- Confirm the preview tag resolves to the Phase 2 CI-validated `PREVIEW_HASH`. If the release branch advances during acceptance, repeat Phases 2–5 with a new preview tag.
- Confirm the Release publication jobs (`create-release`, `upload-release-assets`, and `publish-release`) succeed while `update-latest-version` is skipped.
- Record the inputs actually consumed by each build job: source SHA and the embedded Console release tag, asset URL, and verified SHA256 from the download logs. The workflow reads Console `releases/latest` separately for each build; confirm the matrix used the Console asset accepted in Phase 0. A missing identity or differing asset blocks acceptance until resolved. Retain available toolchain and build provenance with the artifact checksums; the same source SHA alone does not establish equivalent builds.
- Verify `gh release view "<preview-tag>" --json isPrerelease,assets,url`: `isPrerelease` must be `true`, and the Release must contain all 6 versioned platform zips, checksums, SBOM, and provenance with no `-latest` assets. Confirm `gh api repos/{owner}/{repo}/releases/latest --jq .tag_name` does not return `<preview-tag>`.
- Record `PREVIOUS_DELIVERABLE`, selected from published Releases by `publishedAt` after excluding the current tag and every `-preview.N` tag. Verify `gh release view "<preview-tag>" --json body --jq .body` contains `## What's Changed` and, when `PREVIOUS_DELIVERABLE` exists, `**Full Changelog**: https://github.com/rustfs/rustfs/compare/<PREVIOUS_DELIVERABLE>...<preview-tag>`. For a repository with no previous deliverable, verify a Full Changelog link exists and record the GitHub baseline fallback.
- Confirm Helm is skipped. If preview Docker images are published, confirm they use only exact preview tags (and variant suffixes), with no `latest`, `alpha`, `beta`, or `rc` channel updates. Preview validation covers the built RustFS binaries, embedded console, and rc compatibility; it does not authorize advancing any installation default.
-
-## Release notes
-
- When a previous deliverable exists, GitHub Release notes for the preview and final tags MUST use it as their shared comparison baseline: the most recently published non-preview Release before the target. Internal `-preview.N` Releases are explicitly excluded from that selection, even when they point at the same commit as the final tag — cleanup runs after the notes are generated, so the preview Release is still present and would otherwise be picked as the baseline. If no previous deliverable exists, omit `previous_tag_name` and record that GitHub's default baseline fallback was used.
- Generated Release notes carry a workflow-management marker so retries can repair them. Before manually curating a generated body, remove that marker; unmarked non-placeholder notes are preserved by later workflow runs.
+A green Linux run is ready for field acceptance, not publication. The partial certification must have `publish_allowed: false`. Retain the exact Linux manifest for the next stage; a new source/build-input candidate must not inherit the old approval. A newer unrelated main commit does not invalidate this evidence.
@@ -16,22 +16,13 @@ Read for Phases 1–2. The [parent release invariants](../SKILL.md) and Console
 - Only when neither the release branch, an unfinished preview tag, nor current main already has `<target>`: if main is at a later version, select and verify a historical commit containing `<target>` rather than downgrading main; block if none exists. Otherwise invoke the `rustfs-release-version-bump` skill with stage `prepare`, the final `<target>` (NOT a preview version), and full GitHub flow (commit/push/PR).
 - If a bump PR was needed, get it merged into main and record its merge commit. For a new release branch when main already has `<target>`, identify the main commit that introduced those versions. Phase 2 chooses a release candidate containing that commit; an existing release branch keeps its own history and a newer main HEAD is not required.

-Do not wait for the moving latest main HEAD to become green. The release-branch CI in Phase 2 validates the chosen commit before a preview tag is created.
+Do not wait for the moving latest main HEAD to become green. The release-branch CI in Phase 2 validates the chosen commit before a candidate build is dispatched.

-## Phase 2 — Pin and validate the release candidate, then publish the preview tag
+## Phase 2 — Pin and validate the release source

- Use `release/<target>`. On the first attempt, create it from a selected main commit containing the version bump and intended changes; it need not be the latest main HEAD. On a restart, retain the existing branch and backport only reviewed fixes from main. Verify the branch's Cargo versions still equal `<target>`. If adopting an existing preview, run this branch CI before continuing its acceptance; reuse the tag only when it points to the validated SHA.
- The current `ci.yml` runs automatically on main pushes, not release-branch pushes. Dispatch it explicitly with `gh workflow run ci.yml --ref "release/<target>"`. Locate that dispatch run, verify its `head_sha` equals the release branch's remote HEAD, and require the full expected CI job set to pass, including `Quick Checks`, `Test and Lint`, and `End-to-End Tests (full merge gate)`. Treat a missing or skipped required lane as incomplete. A successful PR check or CI run for a different SHA does not qualify. If the branch advances or a run is cancelled, validate the new SHA again. Do not tag a candidate with failing or incomplete CI.
+- Use `release/<target>`. On the first attempt, create it from a selected main commit containing the version bump and intended changes; it need not be the latest main HEAD. On a restart, retain the existing branch and backport only reviewed fixes from main. Verify the branch's Cargo versions still equal `<target>`. A legacy preview can anchor source selection, but its tag and old artifacts do not constitute a candidate in the new protocol; freeze and verify a new candidate after the release recipe is aligned.
+- The current `ci.yml` runs automatically on main pushes, not release-branch pushes. Dispatch it explicitly with `gh workflow run ci.yml --ref "release/<target>"`. Locate that dispatch run, verify its `head_sha` equals the release branch's remote HEAD, and require the full expected CI job set to pass, including `Quick Checks`, `Test and Lint`, and `End-to-End Tests (full merge gate)`. Treat a missing or skipped required lane as incomplete. A successful PR check or CI run for a different SHA does not qualify. If the branch advances or a run is cancelled, validate the new SHA again. Do not dispatch a candidate with failing or incomplete CI.
 - Prefer protecting `release/*` against force pushes and deletion and requiring reviewed backport PRs. Check live rulesets rather than assuming the main ruleset covers release branches; report a missing release-branch rule as a gap.
- Set `PREVIEW_HASH` to the CI-validated release-branch commit and recheck the remote branch HEAD immediately before tagging. Report the branch, commit, and CI run URL; do not derive `PREVIEW_HASH` from a later `origin/main` fetch. When adopting an existing preview tag at this SHA, skip tag creation and continue Phase 3.
-
-```bash
-git tag -a "<preview-tag>" -m "Release <preview-tag>" "$PREVIEW_HASH"
-git push origin "<preview-tag>"
-```
-
-Pushing the tag triggers `.github/workflows/build.yml` ("Build and Release"); `docker.yml` chains off it via `workflow_run`.
-
-The preview run builds versioned artifacts and publishes them in a GitHub prerelease. Docker may publish exact preview image tags. Latest-channel, R2, and Helm publication must be skipped; installation references and the website announcement stay unchanged.
-
-On a restart after a backport, re-run release-branch CI before setting the new `PREVIEW_HASH`. Use the next unused preview iteration when the validated commit differs from an existing preview tag; never move an existing tag.
+- Set `CANDIDATE_SHA` to the CI-validated release-branch commit. Record the branch, exact commit and CI run URL. The new candidate protocol starts from a commit whose release recipe matches the reviewed main dispatcher; backport the recipe before freezing an older release branch if needed.
+- Dispatch the Linux candidate as described in [candidate verification](preview-release.md). Do not create a preview or final tag here. The final executable already contains `<target>`; publication later promotes those same bytes.
+- After a source or build-input change, run the affected source checks and create a new candidate. Reuse the frozen candidate after an infrastructure failure. Do not restart acceptance for unrelated main advances.
@@ -4,18 +4,18 @@ description: Prepare an exact RustFS Cargo version or align installation referen
 ---
 # Release Version Files

-Require an exact target; ask if missing or ambiguous. Reject targets containing `-preview`: those are tag-only. Infer `post-release` only from an explicit request or parent workflow; otherwise use `prepare`. Delivery defaults to local edit/verify; commit, push, PR, and merge follow existing authorization and [Git rules](../../references/pull-requests.md).
+Require an exact target; ask if missing or ambiguous. Reject targets containing `-preview`: those identify candidates, not product versions or new public tags. Infer `post-release` only from an explicit request or parent workflow; otherwise use `prepare`. Delivery defaults to local edit/verify; commit, push, PR, and merge follow existing authorization and [Git rules](../../references/pull-requests.md).

 | Stage | Files and constraints |
 |---|---|
 | `prepare` | Only `Cargo.toml` and `Cargo.lock`: workspace package, internal dependency, and workspace member versions become `<target>`. External dependencies stay unchanged. |
 | `post-release` | Only `README.md`, `README_ZH.md`, `flake.nix`, `helm/rustfs/Chart.yaml`, and `rustfs.spec`. Cargo may already target the next release; leave it untouched. |

-Before installation edits, verify the exact non-preview Release, downloadable assets/source archive, pullable `rustfs/rustfs:<target>` image manifest, and published Helm chart. Block on missing artifacts. Do not downgrade defaults if a newer deliverable superseded the target. Preview preparation retains the previous published installation references; follow-up commits never replace either validated tag.
+Before installation edits, verify the exact non-preview Release, downloadable assets/source archive, pullable `rustfs/rustfs:<target>` image manifest, and published Helm chart. Block on missing artifacts. Do not downgrade defaults if a newer deliverable superseded the target. Candidate preparation retains the previous published installation references; follow-up commits never replace validated tags or sealed artifacts.

 Packaging policy:

- Docker image tags have no `v` prefix. Derive chart/app versions with `scripts/helm_chart_version.sh <target>` (`beta.N -> 0.N.0`; other versions unchanged). Helm CI derives versions from the final tag, so no early chart bump is required.
+- Docker image tags have no `v` prefix. Derive chart/app versions with `scripts/helm_chart_version.sh <target>`; both equal the final product SemVer, including prerelease suffixes. Helm CI uses the frozen candidate version before sealing, so no early installation chart bump is required. Previously published chart versions remain unchanged.
 - RPM `Version` is numeric; `Release` is the prerelease suffix or `1` for stable. Expanded `Source0` and the unpack directory must resolve to the exact target including its suffix. Add the changelog entry using current git identity, date, and target; never copy an example identity/date.
 - Changing these mappings requires explicit confirmation.

@@ -1,2 +1,2 @@
-sha256-darwin=9901a4950099b94d3d2618524d540f1ed4bf6023044903886f98b6c4bf5f30fb
-sha256-linux=09fd5f48f5556c9fc576357222dccd7ebb0b90b790798f1834db2ec5d67b4952
+sha256-darwin=0374e6b9b061dd5d622686301386232a4aa1cb22eedc18af70d2a7411bc66f8d
+sha256-linux=f2cee240f7a744ae991971820f48a6c9c28d19f525e8b2345fcb76eeaaf1e382
@@ -1 +1 @@
-sha256=e4ee3898b840a8374cb02c64342cc15f8d7ecd1becaf4b4a2a4657290f9e7ff1
+sha256=4eac269f9a59b2f9be4e7a87ea6aa38454f5ebe10ede35c9b8d4218137d2cd35
@@ -1 +1 @@
-sha256=5bdd4c2eadbc41625ae3184fff34062e000d328108436259b2b7c4fe2531ecaf
+sha256=21e9a1561ac20382503c38353bbb9b830658534a4f277c254a42d0afc6ebed8b
@@ -70,6 +70,16 @@ s3s-footprint-check: ## Check the s3s dependency footprint ratchet stays frozen
 	@echo "📦 Checking s3s footprint ratchet..."
 	./scripts/check_s3s_footprint.sh

+.PHONY: test-count-ratchet-check
+test-count-ratchet-check: ## Check no workspace crate fell below its test floor or rose above its #[ignore] ceiling in the committed baseline
+	@echo "🧪 Checking per-crate test-count ratchet..."
+	./scripts/check_test_count_ratchet.sh
+
+.PHONY: test-seams-check
+test-seams-check: ## Check tests outside an operation's owner call its use-case only through rustfs/src/app/test_seams
+	@echo "🧪 Checking cross-owner test seams..."
+	$(RUSTFS_PYTHON_BIN) ./scripts/check_test_seams.py
+
 .PHONY: fips-wording-check
 fips-wording-check: ## Check docs and crates/kms do not over-claim crypto capabilities
 	@echo "📣 Checking cryptographic capability wording guard..."
@@ -20,11 +20,11 @@ planning-docs-check: ## Check that no planning-type documents are committed
 	./scripts/check_no_planning_docs.sh

 .PHONY: pre-commit
-pre-commit: fmt-check unsafe-code-check architecture-migration-check logging-guardrails-check error-other-ratchet-check tokio-io-uring-check extension-schema-check body-cache-whitelist-check s3s-footprint-check fips-wording-check embedded-secrets-check test-wiring-check doc-paths-check planning-docs-check quick-check ## Run fast pre-commit checks without clippy/full tests
+pre-commit: fmt-check unsafe-code-check architecture-migration-check logging-guardrails-check error-other-ratchet-check tokio-io-uring-check extension-schema-check body-cache-whitelist-check s3s-footprint-check test-count-ratchet-check test-seams-check fips-wording-check embedded-secrets-check test-wiring-check doc-paths-check planning-docs-check quick-check ## Run fast pre-commit checks without clippy/full tests
 	@echo "✅ All pre-commit checks passed!"

 .PHONY: pre-pr
-pre-pr: fmt-check unsafe-code-check architecture-migration-check logging-guardrails-check error-other-ratchet-check tokio-io-uring-check extension-schema-check body-cache-whitelist-check s3s-footprint-check fips-wording-check embedded-secrets-check test-wiring-check doc-paths-check planning-docs-check log-analyzer-rules-check offline-enrollment-e2e-check clippy-check test ## Run full pre-PR checks with clippy and tests
+pre-pr: fmt-check unsafe-code-check architecture-migration-check logging-guardrails-check error-other-ratchet-check tokio-io-uring-check extension-schema-check body-cache-whitelist-check s3s-footprint-check test-count-ratchet-check test-seams-check fips-wording-check embedded-secrets-check test-wiring-check doc-paths-check planning-docs-check log-analyzer-rules-check offline-enrollment-e2e-check clippy-check test ## Run full pre-PR checks with clippy and tests
 	@echo "✅ All pre-PR checks passed!"

 .PHONY: dev-check
@@ -22,6 +22,14 @@ TEST_THREADS ?= 1
 # differ from CI and .config/nextest.toml test-groups will NOT apply.

 .PHONY: script-tests
+.PHONY: release-package-tests
+release-package-tests: ## Run release package integration on Linux with fpm, dpkg and rpm installed
+	$(RUSTFS_PYTHON_BIN) ./scripts/test_release_packages.py
+
+.PHONY: release-image-tests
+release-image-tests: ## Run OCI integration with Docker, Go and skopeo; set RELEASE_IMAGE_VARIANT and RELEASE_IMAGE_OUTPUT
+	$(RUSTFS_PYTHON_BIN) ./scripts/test_release_images.py
+
 script-tests: ## Run shell script tests
 	@echo "Running script tests..."
 	trap 'status=$$?; printf "ERROR: script-tests failed (exit %s): %s\n" "$$status" "$$BASH_COMMAND" >&2; exit "$$status"' ERR
@@ -52,11 +60,18 @@ script-tests: ## Run shell script tests
 	./scripts/test_python_bin.sh
 	$(RUSTFS_PYTHON_BIN) ./scripts/test_check_doc_paths.py
 	./scripts/check_embedded_secrets.sh --self-test
+	./scripts/check_s3s_footprint.sh --self-test
+	./scripts/test_check_test_count_ratchet.sh
 	$(RUSTFS_PYTHON_BIN) ./scripts/check_test_wiring.py --self-test
+	$(RUSTFS_PYTHON_BIN) ./scripts/codemods/s3_error_to_s3_types.py --self-test
+	$(RUSTFS_PYTHON_BIN) ./scripts/codemods/split_lifecycle_transition_api_test.py --self-test
+	$(RUSTFS_PYTHON_BIN) ./scripts/check_test_seams.py --self-test
 	$(RUSTFS_PYTHON_BIN) ./scripts/test_e2e_binary.py
 	$(RUSTFS_PYTHON_BIN) ./scripts/test_download_release_asset.py
+	$(RUSTFS_PYTHON_BIN) ./scripts/test_reference_binary.py
 	$(RUSTFS_PYTHON_BIN) ./scripts/test_inspect_btf_prototypes.py
 	$(RUSTFS_PYTHON_BIN) ./scripts/test_release_build_workflow.py
+	$(RUSTFS_PYTHON_BIN) -m unittest discover -s scripts/release -p "test_*.py"
 	$(RUSTFS_PYTHON_BIN) ./scripts/test_release_glibc.py
 	$(RUSTFS_PYTHON_BIN) ./scripts/test_upgrade_workflow.py
 	$(RUSTFS_PYTHON_BIN) ./scripts/test_migration_gate_evidence.py
@@ -77,8 +92,10 @@ script-tests: ## Run shell script tests
 	$(RUSTFS_PYTHON_BIN) ./scripts/test_inspect_validation_runner.py
 	$(RUSTFS_PYTHON_BIN) ./scripts/test_retain_validation_evidence.py
 	$(RUSTFS_PYTHON_BIN) ./scripts/test_ci_timing_report.py
+	$(RUSTFS_PYTHON_BIN) ./scripts/test_junit_diff.py
 	$(RUSTFS_PYTHON_BIN) ./scripts/s3-tests/test_report_compat.py
 	$(RUSTFS_PYTHON_BIN) ./scripts/s3-tests/test_runner_tools.py
+	$(RUSTFS_PYTHON_BIN) ./scripts/s3-tests/test_reference_parity.py
 	bash -n ./scripts/validate_object_data_cache_cold_stampede.sh
 	$(RUSTFS_PYTHON_BIN) ./scripts/check_object_data_cache_follower_samples.py --self-test
 	./scripts/validate_object_data_cache_cold_stampede.sh --self-test
@@ -356,6 +356,12 @@ threads-required = "num-test-threads"
 filter = 'package(rustfs-ecstore) & test(=store::init::tests::dispatch_manifest_rollback_bounded_concurrency_reaches_tail_behind_slow_member)'
 threads-required = "num-test-threads"

+# These short-deadline real-disk progress probes need the test run's capacity
+# for scanner and object-lock handoffs; keep their assertions and deadlines.
+[[profile.default.overrides]]
+filter = '(package(rustfs-scanner) & (test(=scanner::tests::cycle_persist_failure::later_cycle_post_commit_failure_reloads_durable_state) | test(=scanner::tests::running_main_loop_catches_up_pause_cleared_after_startup_observe))) | (package(rustfs-ecstore) & test(=store::init::tests::restored_transitioned_delete_causally_enqueues_free_version))'
+threads-required = "num-test-threads"
+
 # ---------------------------------------------------------------------------
 # ci profile — the strict CI gate (ci.yml `cargo nextest run --profile ci`)
 # ---------------------------------------------------------------------------
@@ -424,6 +430,12 @@ threads-required = "num-test-threads"
 filter = 'package(rustfs-scanner) & (test(/^scanner::backlog::tests::native_/) | test(=scanner_folder::tests::mrf_ownership::mrf_ownership_manager_completion_preserves_scanner_pending) | test(/^scanner_io::tests::service_cohort::/))'
 threads-required = "num-test-threads"

+# Keep short-deadline real-disk progress probes from competing with unrelated
+# tests for scanner and object-lock capacity. Assertions and deadlines stay on.
+[[profile.ci.overrides]]
+filter = '(package(rustfs-scanner) & (test(=scanner::tests::cycle_persist_failure::later_cycle_post_commit_failure_reloads_durable_state) | test(=scanner::tests::running_main_loop_catches_up_pause_cleared_after_startup_observe))) | (package(rustfs-ecstore) & test(=store::init::tests::restored_transitioned_delete_causally_enqueues_free_version))'
+threads-required = "num-test-threads"
+
 # Emit this expensive fixture's stage timings in CI and use tmpfs when safe.
 [[profile.ci.overrides]]
 filter = 'package(rustfs-ecstore) & test(=store::init::tests::decommission_durable_ilm_receipt_pagination_fails_closed_on_second_page)'
@@ -698,6 +710,14 @@ test-group = 'ecstore-serial-flaky'
 # the PR-lane e2e coverage for scanner usage snapshots consumed by quota and
 # admin surfaces. It uses the same single-node, random-port, isolated-temp-dir
 # fixture as the existing smoke modules.
+#
+# Dual-stack differential (rustfs/backlog#2740): `dual_stack_differential::`
+# starts two single-node servers from the same binary, one per
+# RUSTFS_S3_STACK value, on random ports with isolated temp dirs, and runs one
+# SDK script covering every operation of `impl S3 for FS` against both. Its
+# comparator unit tests need no server. It meets the admission criteria
+# unchanged; the module clause admits a new differential test without another
+# filter edit.
 [profile.e2e-smoke]
 default-filter = """
    package(e2e_test) & (
@@ -706,6 +726,7 @@ default-filter = """
        | test(/^reliant::lifecycle::/)
        | test(/^conditional_delete_objects_test::/)
        | test(/^reliant::tiering::/)
+        | test(/^dual_stack_differential::/)
        | test(/^upgrade_compatibility_test::upgrade_write_readiness_tests::/)
        | test(/^on_demand_migration::(get_basic_test::(get_miss_pulls_inline_and_serves_locally_afterwards|head_miss_answers_from_the_source_without_persisting)|interaction_test::test_odm_admin_config_is_redacted_and_status_counts_match_the_source)$/)
    )
@@ -925,6 +946,10 @@ path = "junit.xml"
 # parallel-safe — the same property e2e-smoke relies on. The exceptions are the
 # 4-disk reliability / degraded-read fault-injection tests, multi-node heal
 # clusters, and the fixed-port Vault tests, all serialized below.
+# `dual_stack_differential::reference::` holds hermetic unit tests of how the
+# differential picks side A (shell-script stand-ins, no server, no reference
+# binary). They already run on every pull request in e2e-smoke, so e2e-full
+# excludes them to keep its selection digest unchanged.
 [profile.e2e-full]
 default-filter = """
    package(e2e_test)
@@ -935,6 +960,7 @@ default-filter = """
    & !test(/^replication_target_matrix_test::/)
    & !test(/^on_demand_migration::(concurrency_test|fault_test|interop_test|real_source_test)::/)
    & !test(/^upgrade_compatibility_test::upgrade_write_readiness_tests::/)
+    & !test(/^dual_stack_differential::reference::/)
 """
 fail-fast = false

@@ -0,0 +1,20 @@
+# Pinned RustFS release used as the "reference" target of parity lanes
+# (.github/workflows/reference-parity.yml). This is the single place that names
+# the reference release; .github/actions/reference-binary resolves, downloads and
+# verifies it, and fails on a missing pin, a missing asset or a digest mismatch.
+#
+# Bumping the reference is a reviewed event: every per-test difference between
+# the candidate and the reference is measured against this exact build.
+#
+# Each sha256 is the value published in the release's SHA256SUMS asset, which
+# matches the digest GitHub reports for the same asset. The action checks both.
+
+schema = 1
+repository = "rustfs/rustfs"
+version = "1.0.5"
+tag = "1.0.5"
+
+[platforms.linux-x86_64-gnu]
+asset = "rustfs-linux-x86_64-gnu-v1.0.5.zip"
+sha256 = "5addae2f74756bf80104d0975e6347cda843bc2daffdfcd1f0cb80118b95cbd0"
+executable = "rustfs"
@@ -0,0 +1,24 @@
+# Targets of the s3-tests reference parity lane (.github/workflows/reference-parity.yml).
+#
+# Exactly one target uses `binary = "reference"` (the release pinned in
+# .config/reference-binary.toml). Every other target uses `binary = "candidate"`
+# (the binary built from the branch under test) and is compared per node id
+# against the reference. All targets run the same harness, test list and server
+# configuration; `env` adds only the target's own RUSTFS_* settings.
+#
+# Adding a target is one more [[targets]] table, for example:
+#
+#   [[targets]]
+#   name = "candidate-gateway"
+#   binary = "candidate"
+#   env = { RUSTFS_S3_STACK = "gateway" }
+
+schema = 1
+
+[[targets]]
+name = "reference"
+binary = "reference"
+
+[[targets]]
+name = "candidate-legacy"
+binary = "candidate"
@@ -0,0 +1,12 @@
+# Tolerated per-test differences of the s3-tests reference parity lane.
+#
+# One entry per line, four fields separated by " | ":
+#
+#   <target> | <pytest node id> | <tracking issue URL> | <reason>
+#
+# <target> is a candidate name from .config/reference-parity-targets.toml, the
+# node id is exactly as the lane prints it (s3tests/functional/test_s3.py::...),
+# and the issue URL is https://github.com/rustfs/<repo>/issues/<number>.
+#
+# Same semantics as scripts/junit_diff.py --allow: an entry whose difference no
+# longer occurs is stale and fails the lane, so remove it when it is fixed.
@@ -1,44 +1,93 @@
 {
  "schema_version": 1,
  "plan_id": "rustfs-release-validation",
-  "description": "Versioned certification plan for every public RustFS release exit (rustfs/backlog#2662, R01). preview, prerelease and stable all require this explicit plan. Scripts/release/release_gate.py fail-closes publication on any missing, duplicate, stale, substituted or non-PASS required item.",
-  "release_types": ["preview", "prerelease", "stable"],
+  "description": "Certification for immutable candidates. All shipped platforms require native identity and startup; stable and prerelease require the pinned upgrade matrix. Linux-only acceptance never authorizes publication.",
+  "release_types": [
+    "preview",
+    "prerelease",
+    "stable"
+  ],
  "targets": [
-    {"os": "linux", "arch": "x86_64", "abi": "musl"},
-    {"os": "linux", "arch": "aarch64", "abi": "musl"},
-    {"os": "linux", "arch": "x86_64", "abi": "gnu"},
-    {"os": "linux", "arch": "aarch64", "abi": "gnu"},
-    {"os": "macos", "arch": "aarch64", "abi": "none"},
-    {"os": "windows", "arch": "x86_64", "abi": "msvc"}
+    {
+      "os": "linux",
+      "arch": "x86_64",
+      "abi": "musl"
+    },
+    {
+      "os": "linux",
+      "arch": "aarch64",
+      "abi": "musl"
+    },
+    {
+      "os": "linux",
+      "arch": "x86_64",
+      "abi": "gnu"
+    },
+    {
+      "os": "linux",
+      "arch": "aarch64",
+      "abi": "gnu"
+    },
+    {
+      "os": "macos",
+      "arch": "aarch64",
+      "abi": "none"
+    },
+    {
+      "os": "windows",
+      "arch": "x86_64",
+      "abi": "msvc"
+    }
  ],
  "targets_basis": "Platform matrix of .github/workflows/build.yml (prepare-platform-matrix). gnu and macOS targets build with the pyroscope feature; musl and Windows targets build with no extra features.",
-  "features": ["pyroscope"],
-  "trusted_workflows": [
-    {"repository": "rustfs/s3chaos", "workflow_id": "release-gate.yml"}
+  "features": [
+    "pyroscope"
  ],
-  "trusted_workflows_basis": "The shared evidence envelope producer lives in rustfs/s3chaos (G01, fault/release_gate.rs) and its plan/live split workflow is .github/workflows/release-gate.yml there. Only results from these identities may enter a release certification; PR-uploaded JSON never gains release authority. R02 must extend this list when additional verifier workflows land, never widen it to untrusted sources.",
+  "trusted_workflows": [
+    {
+      "repository": "rustfs/rustfs",
+      "workflow_id": "release-candidate.yml"
+    },
+    {
+      "repository": "rustfs/rustfs",
+      "workflow_id": "release-complete.yml"
+    }
+  ],
+  "trusted_workflows_basis": "Receipts are produced by the reviewed main workflows and fetched by exact run ID with GitHub artifact digest verification. Candidate inputs freeze the harness commit separately from product source.",
  "lanes": [
    {
      "lane": "identity",
-      "harness_repository": "rustfs/s3chaos",
-      "harness_sha": "9a48f55c1d6d58ce63a8fb1ea7317e024f32f11d",
-      "harness_basis": "Fixed G01 evidence-envelope producer revision (rustfs/s3chaos PR #102 head recorded in rustfs/backlog#2658). R02 re-pins this when the live verifier ships.",
+      "harness_repository": "rustfs/rustfs",
+      "harness_sha": "candidate.workflow_sha",
+      "harness_basis": "The exact workflow commit is frozen in candidate inputs; no mutable branch is accepted as a harness identity.",
      "timeout_seconds": 1800,
-      "environment_capabilities": ["process-exec"],
-      "cases": ["artifact-identity", "startup"],
+      "environment_capabilities": [
+        "process-exec"
+      ],
+      "cases": [
+        "artifact-identity",
+        "startup"
+      ],
      "applies_to": {
-        "release_types": ["preview", "prerelease", "stable"],
+        "release_types": [
+          "preview",
+          "prerelease",
+          "stable"
+        ],
        "platforms": "all"
      },
      "applies_basis": "Artifact identity and startup checks are mandatory for every release type, preview included, on every shipped target (rustfs/backlog#2662 frozen design item 5)."
    },
    {
      "lane": "upgrade",
-      "harness_repository": "rustfs/s3chaos",
-      "harness_sha": "9a48f55c1d6d58ce63a8fb1ea7317e024f32f11d",
-      "harness_basis": "Same fixed envelope producer as the identity lane; the workflow SHA and the product build SHA are bound separately and need not be equal.",
+      "harness_repository": "rustfs/rustfs",
+      "harness_sha": "candidate.workflow_sha",
+      "harness_basis": "The exact workflow commit is frozen in candidate inputs; no mutable branch is accepted as a harness identity.",
      "timeout_seconds": 5400,
-      "environment_capabilities": ["process-exec", "previous-release-artifact"],
+      "environment_capabilities": [
+        "process-exec",
+        "previous-release-artifact"
+      ],
      "cases": [
        "direct-upgrade",
        "mixed-version-upgrade",
@@ -49,8 +98,17 @@
      ],
      "cases_basis": "Case IDs of the pinned previous-release matrix in .github/workflows/e2e-upgrade.yml (source.version/asset/sha256 and the case list mapped there).",
      "applies_to": {
-        "release_types": ["prerelease", "stable"],
-        "platforms": [{"os": "linux", "arch": "x86_64", "abi": "gnu"}]
+        "release_types": [
+          "prerelease",
+          "stable"
+        ],
+        "platforms": [
+          {
+            "os": "linux",
+            "arch": "x86_64",
+            "abi": "gnu"
+          }
+        ]
      },
      "applies_basis": "preview releases use the pre-declared smaller set (identity lane only). The upgrade lane is only exercised on linux-x86_64-gnu, the platform of the pinned source assets."
    }
@@ -0,0 +1,37 @@
+# Files that may still reference s3s (`s3s::` or `s3s_sigv4`) when Phase 1 of
+# the gateway migration ends (rustfs/backlog#2734 task T0.6, rustfs/backlog#2741).
+# Consumed only by `scripts/check_s3s_footprint.sh --mode allowlist`.
+#
+# One entry per line: a repository-relative file path or a git `:(glob)`
+# pathspec. `**` matches across directories, `*` stays within one path
+# component, and a directory entry matches everything under it. Blank lines
+# and lines starting with `#` are ignored; an entry may not contain whitespace,
+# so a note belongs on its own comment line.
+#
+# Every Rust file listed by `git grep -l '\bs3s\(::\|_sigv4\)' -- '*.rs'` that
+# no entry matches fails the allowlist mode. Task T1.9 switches CI to that
+# mode; until then `--dry-run` prints the files still outside this list, which
+# is the Phase 1 tracking list.
+#
+# Never add crates/** other than crates/e2e_test: the engine crates leave s3s
+# entirely, and the script rejects such an entry. Only the s3s-facing edge of
+# the rustfs binary and the e2e oracle keep it.
+
+# rustfs binary: the s3s-facing edge that survives Phase 1
+rustfs/src/storage/ecfs.rs
+rustfs/src/storage/s3_compat/**
+rustfs/src/storage/access.rs
+rustfs/src/auth.rs
+rustfs/src/error.rs
+rustfs/src/storage_api.rs
+rustfs/src/app/storage_api.rs
+rustfs/src/admin/router.rs
+rustfs/src/app/metadata_route.rs
+rustfs/src/server/http.rs
+rustfs/src/server/layer.rs
+rustfs/src/server/hybrid.rs
+rustfs/src/protocols/client.rs
+
+# e2e oracle: the test target that verifies S3 behaviour against s3s
+crates/e2e_test/src/fake_s3_target/**
+crates/e2e_test/src/**
@@ -0,0 +1,17 @@
+# Exact baseline for the direct s3s footprint, consumed only by
+# scripts/check_s3s_footprint.sh (rustfs/backlog#1677; rustfs/backlog#2734 T0.4).
+#
+# Each value is the exact measurement of the tree, not a ceiling. CI fails when a
+# measured counter differs from its line here in either direction:
+#   - a larger measurement means new code widened the s3s surface: route it
+#     through the gateway abstractions instead; never raise a value here;
+#   - a smaller measurement means the PR shrank the surface: lower the matching
+#     value to the measured number in the same PR.
+#
+# files           Rust files referencing s3s paths, repo-wide minus crates/e2e_test
+# s3_error_lines  lines holding s3_error!, same scope, minus those the script
+#                 header's rule clears (rustfs_s3_types's macro); fails closed
+# ecstore_files   Rust files referencing s3s paths under crates/ecstore/src
+files=65
+s3_error_lines=217
+ecstore_files=0
@@ -0,0 +1,67 @@
+# Per-crate test-count baseline consumed by scripts/check_test_count_ratchet.sh
+# (rustfs/backlog#2734 section 1 rule 3; T0.3 in rustfs/backlog#2738).
+#
+# One row per workspace member: <package>, tests, ignored, tab-separated.
+# tests counts #[test], #[tokio::test], #[tokio::test(...)] and #[test_case(...)]
+# attribute lines under the package directory; ignored counts #[ignore] and
+# #[ignore = "..."]. The tests column is a floor and the ignored column is a
+# ceiling: CI fails when a crate's tests fall below its row or its ignored count
+# rises above it. Extra tests and fewer ignores pass; --update tightens the rows.
+# Lowering a tests row or raising an ignored row is a reviewed statement that
+# tests were intentionally removed or ignored; justify it in the PR description.
+# Regenerate with:
+#   scripts/check_test_count_ratchet.sh --update
+e2e_test	1026	20
+rustfs	6449	64
+rustfs-audit	70	0
+rustfs-checksums	28	1
+rustfs-common	36	0
+rustfs-concurrency	87	0
+rustfs-config	43	0
+rustfs-connect-symbol-catalog	6	0
+rustfs-credentials	24	0
+rustfs-crypto	67	0
+rustfs-data-usage	58	0
+rustfs-ecstore	7858	2
+rustfs-extension-schema	23	0
+rustfs-filemeta	361	0
+rustfs-gateway-adapter	74	0
+rustfs-heal	1128	3
+rustfs-heal-contracts	8	0
+rustfs-iam	415	0
+rustfs-io-core	52	0
+rustfs-io-metrics	157	0
+rustfs-keystone	31	0
+rustfs-kms	716	25
+rustfs-license	2	0
+rustfs-lifecycle	156	0
+rustfs-lock	182	0
+rustfs-log-analyzer	91	0
+rustfs-madmin	210	0
+rustfs-notify	140	0
+rustfs-object-capacity	92	0
+rustfs-object-data-cache	141	0
+rustfs-obs	387	0
+rustfs-policy	758	0
+rustfs-protocols	770	1
+rustfs-protos	75	0
+rustfs-replication	209	0
+rustfs-rio	225	1
+rustfs-rio-v2	34	6
+rustfs-s3-client	71	0
+rustfs-s3-ops	8	0
+rustfs-s3-types	105	0
+rustfs-s3select-api	232	0
+rustfs-s3select-query	235	2
+rustfs-scanner	1392	20
+rustfs-scanner-metrics	90	0
+rustfs-security-governance	20	0
+rustfs-signer	34	0
+rustfs-storage-api	34	0
+rustfs-targets	612	26
+rustfs-test-utils	1	0
+rustfs-tls-runtime	44	0
+rustfs-trusted-proxies	72	0
+rustfs-utils	253	1
+rustfs-zip	30	0
+zstd	11	0
@@ -97,6 +97,14 @@ runs:
      shell: bash
      run: ./scripts/check_s3s_footprint.sh

+    - name: Check per-crate test-count ratchet
+      shell: bash
+      run: ./scripts/check_test_count_ratchet.sh
+
+    - name: Check cross-owner tests use test seams
+      shell: bash
+      run: python3 ./scripts/check_test_seams.py
+
    - name: Check cryptographic capability wording
      shell: bash
      run: ./scripts/check_fips_wording.sh
@@ -0,0 +1,96 @@
+# Copyright 2026 RustFS Team
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+name: "Reference RustFS binary"
+description: "Download the pinned reference RustFS release, verify its sha256 and expose the executable path. A missing pin, missing asset or digest mismatch fails the step."
+
+inputs:
+  pin-file:
+    description: "Pin file naming the reference release (repository-relative)"
+    required: false
+    default: ".config/reference-binary.toml"
+  platform:
+    description: "Platform table in the pin file"
+    required: false
+    default: "linux-x86_64-gnu"
+  destination:
+    description: "Directory that receives the executable; defaults to $RUNNER_TEMP/reference-binary"
+    required: false
+    default: ""
+
+outputs:
+  path:
+    description: "Absolute path of the verified executable"
+    value: ${{ steps.unpack.outputs.path }}
+  version:
+    description: "Pinned reference version"
+    value: ${{ steps.pin.outputs.version }}
+  sha256:
+    description: "Pinned and verified archive sha256"
+    value: ${{ steps.pin.outputs.sha256 }}
+
+runs:
+  using: "composite"
+  steps:
+    - name: Resolve reference pin
+      id: pin
+      shell: bash
+      env:
+        PIN_TOOL: ${{ github.action_path }}/reference_binary.py
+        PIN_FILE: ${{ inputs.pin-file }}
+        PIN_PLATFORM: ${{ inputs.platform }}
+      run: |
+        set -euo pipefail
+        resolved="$(python3 "$PIN_TOOL" resolve --pin "$PIN_FILE" --platform "$PIN_PLATFORM")"
+        printf '%s\n' "$resolved"
+        printf '%s\n' "$resolved" >> "$GITHUB_OUTPUT"
+
+    - name: Download and verify reference archive
+      id: download
+      shell: bash
+      env:
+        GH_TOKEN: ${{ github.token }}
+        ASSET_DOWNLOADER: ${{ github.action_path }}/../setup/download_release_asset.py
+        REF_REPOSITORY: ${{ steps.pin.outputs.repository }}
+        REF_TAG: ${{ steps.pin.outputs.tag }}
+        REF_ASSET: ${{ steps.pin.outputs.asset }}
+        REF_SHA256: ${{ steps.pin.outputs.sha256 }}
+      run: |
+        set -euo pipefail
+        archive="$RUNNER_TEMP/reference-archive/$REF_ASSET"
+        # Fails unless the release lists exactly one asset of this name whose
+        # published digest equals the pin and whose downloaded bytes match it.
+        python3 "$ASSET_DOWNLOADER" --repository "$REF_REPOSITORY" --tag "$REF_TAG" \
+          --asset "$REF_ASSET" --output "$archive" --sha256 "$REF_SHA256"
+        echo "archive=$archive" >> "$GITHUB_OUTPUT"
+
+    - name: Unpack reference executable
+      id: unpack
+      shell: bash
+      env:
+        PIN_TOOL: ${{ github.action_path }}/reference_binary.py
+        REF_ARCHIVE: ${{ steps.download.outputs.archive }}
+        REF_SHA256: ${{ steps.pin.outputs.sha256 }}
+        REF_EXECUTABLE: ${{ steps.pin.outputs.executable }}
+        REF_DESTINATION: ${{ inputs.destination }}
+      run: |
+        set -euo pipefail
+        destination="${REF_DESTINATION:-$RUNNER_TEMP/reference-binary}"
+        unpacked="$(python3 "$PIN_TOOL" unpack --archive "$REF_ARCHIVE" --sha256 "$REF_SHA256" \
+          --executable "$REF_EXECUTABLE" --destination "$destination")"
+        path="${unpacked#path=}"
+        test -x "$path"
+        echo "Reference executable: $path"
+        "$path" --version
+        echo "path=$path" >> "$GITHUB_OUTPUT"
@@ -0,0 +1,144 @@
+#!/usr/bin/env python3
+# Copyright 2026 RustFS Team
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+"""Resolve and unpack the pinned reference RustFS release binary.
+
+`resolve` reads the pin file and prints the selected platform's identity as
+`key=value` lines (GitHub step-output format). `unpack` re-verifies the
+downloaded archive digest against the pin and extracts exactly one executable.
+Every missing or malformed input is a failure; nothing is skipped.
+"""
+
+import argparse
+import hashlib
+from pathlib import Path
+import re
+import sys
+import tomllib
+import zipfile
+
+REPOSITORY = re.compile(r"[A-Za-z0-9][A-Za-z0-9_.-]*/[A-Za-z0-9][A-Za-z0-9_.-]*")
+TAG = re.compile(r"[A-Za-z0-9][A-Za-z0-9_.+-]*")
+PLATFORM = re.compile(r"[a-z0-9][a-z0-9_-]*")
+ASSET = re.compile(r"[A-Za-z0-9][A-Za-z0-9_.+-]*\.zip")
+SHA256 = re.compile(r"[0-9a-f]{64}")
+EXECUTABLE = re.compile(r"[A-Za-z0-9][A-Za-z0-9_.-]*")
+MAX_EXECUTABLE_BYTES = 1024 * 1024 * 1024
+
+
+class PinError(Exception):
+    """A pin, archive or argument that cannot be trusted."""
+
+
+def field(table, key, pattern, where):
+    value = table.get(key)
+    if not isinstance(value, str) or not pattern.fullmatch(value):
+        raise PinError(f"{where}: missing or invalid '{key}'")
+    return value
+
+
+def resolve(pin_path, platform):
+    if not PLATFORM.fullmatch(platform or ""):
+        raise PinError("invalid platform name")
+    try:
+        data = tomllib.loads(Path(pin_path).read_text(encoding="utf-8"))
+    except FileNotFoundError as exc:
+        raise PinError(f"{pin_path}: reference pin file is missing") from exc
+    except (OSError, UnicodeDecodeError, tomllib.TOMLDecodeError) as exc:
+        raise PinError(f"{pin_path}: cannot read reference pin: {exc}") from exc
+    if data.get("schema") != 1 or type(data.get("schema")) is not int:
+        raise PinError(f"{pin_path}: unsupported pin schema")
+    identity = {
+        "repository": field(data, "repository", REPOSITORY, pin_path),
+        "version": field(data, "version", TAG, pin_path),
+        "tag": field(data, "tag", TAG, pin_path),
+    }
+    platforms = data.get("platforms")
+    if not isinstance(platforms, dict) or not platforms:
+        raise PinError(f"{pin_path}: no pinned platforms")
+    entry = platforms.get(platform)
+    if not isinstance(entry, dict):
+        raise PinError(f"{pin_path}: no pin for platform '{platform}'")
+    where = f"{pin_path} [platforms.{platform}]"
+    identity["asset"] = field(entry, "asset", ASSET, where)
+    identity["sha256"] = field(entry, "sha256", SHA256, where)
+    identity["executable"] = field(entry, "executable", EXECUTABLE, where)
+    if identity["version"] not in identity["asset"]:
+        raise PinError(f"{where}: asset name does not carry version {identity['version']}")
+    unknown = set(entry) - {"asset", "sha256", "executable"}
+    if unknown:
+        raise PinError(f"{where}: unknown keys {sorted(unknown)}")
+    return identity
+
+
+def file_sha256(path):
+    with open(path, "rb") as stream:
+        return hashlib.file_digest(stream, "sha256").hexdigest()
+
+
+def unpack(archive, sha256, executable, destination):
+    if not SHA256.fullmatch(sha256 or "") or not EXECUTABLE.fullmatch(executable or ""):
+        raise PinError("invalid expected digest or executable name")
+    try:
+        actual = file_sha256(archive)
+    except OSError as exc:
+        raise PinError(f"{archive}: reference archive is missing or unreadable") from exc
+    if actual != sha256:
+        raise PinError(f"{archive}: sha256 {actual} does not match the pinned digest")
+    try:
+        with zipfile.ZipFile(archive) as bundle:
+            matches = [info for info in bundle.infolist() if info.filename == executable]
+            if len(matches) != 1:
+                raise PinError(f"{archive}: must contain exactly one top-level '{executable}'")
+            info = matches[0]
+            if info.is_dir() or not 0 < info.file_size <= MAX_EXECUTABLE_BYTES:
+                raise PinError(f"{archive}: '{executable}' is not a plausible executable")
+            destination = Path(destination)
+            destination.mkdir(parents=True, exist_ok=True)
+            target = destination / executable
+            with bundle.open(info) as source, open(target, "wb") as sink:
+                while chunk := source.read(1024 * 1024):
+                    sink.write(chunk)
+    except zipfile.BadZipFile as exc:
+        raise PinError(f"{archive}: not a valid zip archive") from exc
+    target.chmod(0o755)
+    return target
+
+
+def main(argv=None):
+    parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
+    sub = parser.add_subparsers(dest="mode", required=True)
+    res = sub.add_parser("resolve")
+    res.add_argument("--pin", required=True)
+    res.add_argument("--platform", required=True)
+    unp = sub.add_parser("unpack")
+    unp.add_argument("--archive", required=True)
+    unp.add_argument("--sha256", required=True)
+    unp.add_argument("--executable", required=True)
+    unp.add_argument("--destination", required=True)
+    args = parser.parse_args(argv)
+    try:
+        if args.mode == "resolve":
+            for key, value in resolve(args.pin, args.platform).items():
+                print(f"{key}={value}")
+        else:
+            print(f"path={unpack(args.archive, args.sha256, args.executable, args.destination)}")
+    except PinError as exc:
+        print(f"reference binary: {exc}", file=sys.stderr)
+        return 1
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())
@@ -350,10 +350,10 @@
      "roles": [
        "periodic",
        "manual",
-        "release"
+        "reusable"
      ],
      "events": [
-        "push",
+        "workflow_call",
        "schedule",
        "workflow_dispatch"
      ],
@@ -367,7 +367,7 @@
      "monitor": "workflow",
      "max_age_hours": 36,
      "result_owner": "build.yml",
-      "disposition": "retained independent entry",
+      "disposition": "Builds artifacts only; public release writes belong to release-publish.yml.",
      "failure_owner": "workflow",
      "origin": "baseline",
      "alert_contract": "scheduled-validation"
@@ -745,14 +745,13 @@
    },
    {
      "workflow": ".github/workflows/docker.yml",
-      "name": "Docker Images",
+      "name": "Build candidate images",
      "roles": [
-        "manual",
-        "release"
+        "release",
+        "reusable"
      ],
      "events": [
-        "workflow_run",
-        "workflow_dispatch"
+        "workflow_call"
      ],
      "cadence": "event",
      "timezone": "UTC",
@@ -761,7 +760,7 @@
      "policy": "active",
      "monitor": "none",
      "result_owner": "docker.yml",
-      "disposition": "retained independent entry",
+      "disposition": "Builds artifacts only; public release writes belong to release-publish.yml.",
      "monitor_reason": "Event-driven or operator-requested work has no periodic freshness obligation.",
      "origin": "baseline"
    },
@@ -870,6 +869,33 @@
      "failure_owner": "watchdog",
      "origin": "baseline"
    },
+    {
+      "workflow": ".github/workflows/reference-parity.yml",
+      "name": "Reference Parity",
+      "roles": [
+        "pr",
+        "periodic",
+        "manual"
+      ],
+      "events": [
+        "pull_request",
+        "schedule",
+        "workflow_dispatch"
+      ],
+      "cadence": "nightly",
+      "timezone": "UTC",
+      "schedule": [
+        "37 1 * * *"
+      ],
+      "owner": "@overtrue",
+      "policy": "active",
+      "monitor": "workflow",
+      "max_age_hours": 36,
+      "result_owner": "reference-parity.yml",
+      "disposition": "s3-tests per-node-id parity of branch candidates against the pinned reference release",
+      "failure_owner": "watchdog",
+      "origin": "added"
+    },
    {
      "workflow": ".github/workflows/functional-chain-health.yml",
      "name": "Functional chain health",
@@ -954,14 +980,13 @@
    },
    {
      "workflow": ".github/workflows/helm-package.yml",
-      "name": "Publish helm chart to artifacthub",
+      "name": "Build candidate Helm chart",
      "roles": [
-        "manual",
-        "release"
+        "release",
+        "reusable"
      ],
      "events": [
-        "workflow_run",
-        "workflow_dispatch"
+        "workflow_call"
      ],
      "cadence": "event",
      "timezone": "UTC",
@@ -970,7 +995,7 @@
      "policy": "active",
      "monitor": "none",
      "result_owner": "helm-package.yml",
-      "disposition": "retained independent entry",
+      "disposition": "Builds artifacts only; public release writes belong to release-publish.yml.",
      "monitor_reason": "Event-driven or operator-requested work has no periodic freshness obligation.",
      "origin": "baseline"
    },
@@ -996,6 +1021,26 @@
      "policy_reason": "Preserve operator-disabled state; enabling requires a separate policy decision.",
      "origin": "baseline"
    },
+    {
+      "workflow": ".github/workflows/manual-main-packages.yml",
+      "name": "Manual main packages",
+      "roles": [
+        "manual"
+      ],
+      "events": [
+        "workflow_dispatch"
+      ],
+      "cadence": "event",
+      "timezone": "UTC",
+      "schedule": [],
+      "owner": "@overtrue",
+      "policy": "active",
+      "monitor": "none",
+      "result_owner": "manual-main-packages.yml",
+      "disposition": "Packages the origin/main tip as workflow artifacts only. No GitHub Release or download-channel writes.",
+      "monitor_reason": "Operator-requested packaging has no periodic freshness obligation.",
+      "origin": "manual-main-packages"
+    },
    {
      "workflow": ".github/workflows/minio-interop.yml",
      "name": "minio-interop",
@@ -1182,14 +1227,13 @@
    },
    {
      "workflow": ".github/workflows/package.yml",
-      "name": "Package DEB/RPM",
+      "name": "Build candidate packages",
      "roles": [
-        "manual",
-        "release"
+        "release",
+        "reusable"
      ],
      "events": [
-        "workflow_run",
-        "workflow_dispatch"
+        "workflow_call"
      ],
      "cadence": "event",
      "timezone": "UTC",
@@ -1198,7 +1242,7 @@
      "policy": "active",
      "monitor": "none",
      "result_owner": "package.yml",
-      "disposition": "retained independent entry",
+      "disposition": "Builds artifacts only; public release writes belong to release-publish.yml.",
      "monitor_reason": "Event-driven or operator-requested work has no periodic freshness obligation.",
      "origin": "baseline"
    },
@@ -1727,6 +1771,69 @@
      "disposition": "retained independent entry",
      "monitor_reason": "Event-driven or operator-requested work has no periodic freshness obligation.",
      "origin": "baseline"
+    },
+    {
+      "workflow": ".github/workflows/release-candidate.yml",
+      "name": "Release candidate (Linux)",
+      "roles": [
+        "manual",
+        "release"
+      ],
+      "events": [
+        "workflow_dispatch"
+      ],
+      "cadence": "event",
+      "timezone": "UTC",
+      "schedule": [],
+      "owner": "@overtrue",
+      "policy": "active",
+      "monitor": "none",
+      "result_owner": "release-candidate.yml",
+      "disposition": "Explicit immutable candidate stage",
+      "monitor_reason": "Operator-requested release stage has no periodic freshness obligation.",
+      "origin": "release-candidate-migration"
+    },
+    {
+      "workflow": ".github/workflows/release-complete.yml",
+      "name": "Release candidate (complete)",
+      "roles": [
+        "manual",
+        "release"
+      ],
+      "events": [
+        "workflow_dispatch"
+      ],
+      "cadence": "event",
+      "timezone": "UTC",
+      "schedule": [],
+      "owner": "@overtrue",
+      "policy": "active",
+      "monitor": "none",
+      "result_owner": "release-complete.yml",
+      "disposition": "Explicit immutable candidate stage",
+      "monitor_reason": "Operator-requested release stage has no periodic freshness obligation.",
+      "origin": "release-candidate-migration"
+    },
+    {
+      "workflow": ".github/workflows/release-publish.yml",
+      "name": "Publish accepted release",
+      "roles": [
+        "manual",
+        "release"
+      ],
+      "events": [
+        "workflow_dispatch"
+      ],
+      "cadence": "event",
+      "timezone": "UTC",
+      "schedule": [],
+      "owner": "@overtrue",
+      "policy": "active",
+      "monitor": "none",
+      "result_owner": "release-publish.yml",
+      "disposition": "Explicit immutable candidate stage",
+      "monitor_reason": "Operator-requested release stage has no periodic freshness obligation.",
+      "origin": "release-candidate-migration"
    }
  ]
 }
@@ -5,7 +5,7 @@

 ## Billing facts

-This repository is public. For a public repository, GitHub's standard hosted runners (`ubuntu-latest`, `windows-latest`, `macos-latest`) cost nothing and have no minute cap, whatever plan the organisation is on. GitHub bills only two things: larger hosted runners, which have custom labels and are not used here, and standard runners in private repositories.
+This repository is public. For a public repository, GitHub's standard hosted runners (`ubuntu-latest`, `windows-latest`, `macos-latest`) cost nothing and have no minute cap, whatever plan the organisation is on. GitHub bills only two things: larger hosted runners, including the `macos-latest-xlarge` release lane used here, and standard runners in private repositories.

 Every other label in this repository is capacity RustFS pays for. ARC pods and lab machines accrue cost while the machine exists. A Ubicloud label is metered for the minutes a job holds its VM:

@@ -13,6 +13,7 @@ Every other label in this repository is capacity RustFS pays for. ARC pods and l
 |---|---|---|---|
 | `ubuntu-latest` | nobody | GitHub-hosted, Ubuntu 24.04, 4 vCPU, 16 GB RAM, about 14 GB of free disk. Ships a C toolchain, `docker`, `gh`, `curl`, `wget`, `jq`, Python. Can `mount` tmpfs and loop devices; io_uring works | the default for every job |
 | `windows-latest`, `macos-latest` | nobody | GitHub-hosted standard runners | Windows and macOS targets |
+| `macos-latest-xlarge` | RustFS | GitHub-hosted larger Apple Silicon runner | macOS release builds, deferred until Linux acceptance for candidates |
 | `sm-standard-2` | RustFS | ARC pod, Ubuntu 24.04, minimal image: no `gh`, no C toolchain; Docker only on the `dind-` label | only a job that a hosted runner cannot run (see the placement rule) |
 | `sm-standard-4` | RustFS | ARC pod, Ubuntu 24.04, 14 CPUs and 28 GB (the label is not the core count), large disk | workspace-scale Cargo builds |
 | `ubicloud-standard-16-arm` | RustFS | Ubicloud VM, alias of `ubicloud-standard-16-arm-ubuntu-2404`: Ubuntu 24.04 arm64, 16 vCPU, 48 GB, 300 GB disk. Reached through the `build.yml` matrix, so `actionlint.yaml` does not list it | native aarch64 release builds (`linux-aarch64-musl`, `linux-aarch64-gnu`) |
@@ -32,7 +33,7 @@ A job runs on `ubuntu-latest` unless it has one of these reasons, and the reason

 Jobs that do only the following have no reason to be self-hosted: shell or Python scripts, `cargo fmt`, `cargo deny`, typos, actionlint, API calls (`gh`, dependency review, issue and stale bots, CLA checks), downloading an artifact and running a prebuilt binary, or a `cargo build -p rustfs` with its own cache key. `e2e-distributed.yml` and `e2e-upgrade.yml` build the server and the `e2e_test` harness on `ubuntu-latest` inside their budgets, so "it compiles something" alone is not a reason.

-The functional chain's `prepare` and `complete-chain` jobs, the performance runner availability check, and runner hygiene use `ubuntu-latest`: they inspect APIs or artifacts, not the lab hosts. The actual functional and performance suites remain on their lab runners. Functional-chain health collection uses `ubuntu-latest` so a lab outage cannot also stop its independent monitor. Docker image packaging also uses `ubuntu-latest` for both musl and glibc variants; it downloads release binaries rather than compiling the workspace, and still sets up QEMU and Buildx explicitly.
+The functional chain's `prepare` and `complete-chain` jobs, the performance runner availability check, and runner hygiene use `ubuntu-latest`: they inspect APIs or artifacts, not the lab hosts. The actual functional and performance suites remain on their lab runners. Functional-chain health collection uses `ubuntu-latest` so a lab outage cannot also stop its independent monitor. Docker image packaging also uses `ubuntu-latest` for both musl and glibc variants; it wraps verified candidate binaries rather than compiling the workspace, and still sets up QEMU and Buildx explicitly.

 Moving these jobs off AKS reduces capacity demand, not necessarily the Azure bill immediately. VM costs fall only when nodes scale down or smaller capacity replaces them.

@@ -52,7 +53,7 @@ A hosted runner is also the only choice for a few things the pods cannot do: `mo
 These are the patterns that have cost real money here without validating anything more:

 - An `if: always()` summary or alert job on a self-hosted label inside a `workflow_run`-triggered workflow. The upstream workflow completes many times a day, the real jobs skip, and the summary still starts a pod each time. Gate it on the upstream job having run, and keep it on `ubuntu-latest`.
- Per-push workflows on `main`. A merge triggers the full `ci.yml` matrix, `cache-warm.yml` and the `build.yml` development build; with tens of merges a day that is as much self-hosted time as all pull requests together. Anything that does not verify the merged tree itself belongs on a schedule or behind a `paths` filter.
+- Per-push workflows on `main`. A merge triggers the applicable `ci.yml` matrix and cache warming; `build.yml` development builds are scheduled or manual; with tens of merges a day that is as much self-hosted time as all pull requests together. Anything that does not verify the merged tree itself belongs on a schedule or behind a `paths` filter.
 - `apt-get install` of tools the hosted image already ships. It is only needed on the minimal pod images.
 - A throttled `CARGO_BUILD_JOBS` on a 14-CPU pod. The limit exists for link-storm reasons documented in `ci.yml`; measure before raising it, and measure before lowering it further.
 - Tiny jobs on ARC pods. Each one pays roughly a minute of pod start-up before its first step; a compile-free job that takes ten seconds costs more to schedule than to run.
@@ -66,7 +67,7 @@ gh api "repos/rustfs/rustfs/actions/runs/<run_id>/jobs?per_page=100" \
  --jq '.jobs[] | "\(.conclusion)\t\(.labels | join(","))\t\(.started_at)\t\(.completed_at)\t\(.name)"'
 ```

-Only rows whose label starts with `sm-`, `dind-`, `ubicloud-`, `smoke-` or `pf-` cost anything. At the October 2026 review, a full pull-request run of `ci.yml` spent about 150 self-hosted minutes, a third of it in jobs that never compiled; treat that as the scale of what a placement mistake costs, not as a current figure.
+Rows using `macos-latest-xlarge` or a label starting with `sm-`, `dind-`, `ubicloud-`, `smoke-` or `pf-` use paid capacity. At the October 2026 review, a full pull-request run of `ci.yml` spent about 150 self-hosted minutes, a third of it in jobs that never compiled; treat that as the scale of what a placement mistake costs, not as a current figure.

 ## Checklist for a new or moved job

@@ -76,3 +77,9 @@ Only rows whose label starts with `sm-`, `dind-`, `ubicloud-`, `smoke-` or `pf-`
 - A self-hosted `runs-on` carries a comment naming its reason from the placement rule.
 - Every `./.github/actions/setup` call states `cache-save-if`, and a key read from one pool is not written from the other.
 - Moving a job does not change its check name, so the `main` ruleset and `docs/testing/ci-gates.md` stay as they are; renaming it does, and then that document's change checklist applies.
+
+## Release stages
+
+The [candidate release pipeline](../../docs/operations/release-pipeline.md) builds Linux first and defers paid desktop capacity until acceptance. Daily development builds also select Linux; use manual platform selection for desktop development artifacts. All six platforms remain required before publication. The pipeline records and promotes the exact tested bytes. Builder workflows have no public-release credentials. Keep periodic development builds, PR verification and field-test chains separate when their contracts differ; do not re-enable disabled workflows as part of a source-file migration.
+
+Larger runners remain billed for public repositories; see [GitHub runner billing](https://docs.github.com/en/actions/concepts/runners/larger-runners#billing). Actual spending comes from organization billing, not runner labels or an estimated build duration.
@@ -23,9 +23,16 @@ on:
      - 'deny.toml'
      - '.github/actions/**'
      - '.github/workflows/**'
-      - 'scripts/release/create_or_update_release.sh'
-      - 'scripts/release/package_service_scripts.sh'
-      - 'scripts/release/package_versions.sh'
+      - 'scripts/release/**'
+      - 'scripts/security/generate_release_supply_chain_assets.py'
+      - '.config/release-validation-plan.json'
+      - '.github/scheduled-validations.json'
+      - 'scripts/test_helm_release_workflow.py'
+      - 'scripts/test_release_packages.py'
+      - 'scripts/test_release_images.py'
+      - 'Dockerfile*'
+      - 'entrypoint.sh'
+      - 'bootstrap/**'
      - 'scripts/test_package_service_scripts.sh'
      - 'scripts/test_package_versions.sh'
      - 'scripts/test_docker_workflow.py'
@@ -41,9 +48,16 @@ on:
      - 'deny.toml'
      - '.github/actions/**'
      - '.github/workflows/**'
-      - 'scripts/release/create_or_update_release.sh'
-      - 'scripts/release/package_service_scripts.sh'
-      - 'scripts/release/package_versions.sh'
+      - 'scripts/release/**'
+      - 'scripts/security/generate_release_supply_chain_assets.py'
+      - '.config/release-validation-plan.json'
+      - '.github/scheduled-validations.json'
+      - 'scripts/test_helm_release_workflow.py'
+      - 'scripts/test_release_packages.py'
+      - 'scripts/test_release_images.py'
+      - 'Dockerfile*'
+      - 'entrypoint.sh'
+      - 'bootstrap/**'
      - 'scripts/test_package_service_scripts.sh'
      - 'scripts/test_package_versions.sh'
      - 'scripts/test_docker_workflow.py'
@@ -132,12 +146,38 @@ jobs:
    if: github.repository == 'rustfs/rustfs' && (github.event_name != 'pull_request' || github.event.action != 'closed')
    runs-on: ubuntu-latest
    timeout-minutes: 5
+    permissions:
+      contents: read
+      actions: read
+    outputs:
+      release_changed: ${{ steps.scope.outputs.release_changed }}
    steps:
      - name: Checkout repository
        uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
        with:
+          fetch-depth: 0
          persist-credentials: false

+      - name: Scope native packaging checks to release changes
+        id: scope
+        env:
+          BASE_SHA: ${{ github.event.pull_request.base.sha || github.event.before }}
+        run: |
+          python3 - <<'PY'
+          import os, re, subprocess
+          event, base = os.environ['GITHUB_EVENT_NAME'], os.environ.get('BASE_SHA', '')
+          changed = event == 'workflow_dispatch'
+          if event in ('push', 'pull_request'):
+              if not re.fullmatch(r'[0-9a-f]{40}', base) or base == '0' * 40:
+                  changed = True
+              else:
+                  paths = subprocess.check_output(['git', 'diff', '--name-only', base, os.environ['GITHUB_SHA']], text=True).splitlines()
+                  pattern = r'^(scripts/release/|scripts/test_release_|Dockerfile|entrypoint.sh$|bootstrap/|\.dockerignore$|\.config/release-validation-plan.json$|\.github/workflows/(audit|build|docker|package|helm-package|release-[^.]+)\.yml$)'
+                  changed = any(re.match(pattern, path) for path in paths)
+          with open(os.environ['GITHUB_OUTPUT'], 'a') as output:
+              output.write(f'release_changed={str(changed).lower()}\n')
+          PY
+
      - name: Report unpinned GitHub Actions
        run: ./scripts/security/check_workflow_pins.sh --enforce

@@ -150,8 +190,35 @@ jobs:
      - name: Check checkouts clear their credentials
        run: ./scripts/security/check_persist_credentials.sh

-      - name: Check preview release workflow policy
-        run: ./scripts/security/check_preview_release_workflow.sh
+      - name: Check release candidate and promotion contracts
+        run: python3 -m unittest discover -s scripts/release -p 'test_*.py'
+
+      - name: Prepare artifact transport rehearsal
+        run: |
+          mkdir -p "$RUNNER_TEMP/release-transport"
+          printf '{"purpose":"transport contract, not release evidence"}\n' > "$RUNNER_TEMP/release-transport/transport.json"
+
+      - uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+        with:
+          name: release-transport-contract-${{ github.run_attempt }}
+          path: ${{ runner.temp }}/release-transport/transport.json
+          if-no-files-found: error
+          retention-days: 1
+
+      - name: Verify actual GitHub artifact metadata and downloaded bytes
+        env:
+          GH_TOKEN: ${{ github.token }}
+        run: |
+          python3 - <<'PY'
+          import os, sys
+          from pathlib import Path
+          sys.path.insert(0, 'scripts/release')
+          import candidate as c
+          run = c.github(f"repos/{c.REPOSITORY}/actions/runs/{os.environ['GITHUB_RUN_ID']}")
+          destination = Path(os.environ['RUNNER_TEMP']) / 'release-transport-downloaded'
+          c.download_artifact(run, f"release-transport-contract-{os.environ['GITHUB_RUN_ATTEMPT']}", destination)
+          c.require(c.read_json(destination / 'transport.json') == {'purpose': 'transport contract, not release evidence'}, 'artifact round trip changed')
+          PY

      - name: Check tier evidence workflow isolation
        run: ./scripts/security/check_tier_artifact_workflow.sh
@@ -162,6 +229,80 @@ jobs:
      - name: Check package service lifecycle contract
        run: ./scripts/test_package_service_scripts.sh

+  release-package-contract:
+    name: Release package contract
+    needs: workflow-pin-report
+    if: needs.workflow-pin-report.outputs.release_changed == 'true'
+    runs-on: ubuntu-latest
+    timeout-minutes: 15
+    steps:
+      - uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
+        with:
+          persist-credentials: false
+      - name: Install real package managers
+        run: |
+          sudo apt-get update
+          sudo apt-get install -y ruby ruby-dev build-essential rpm fakeroot cpio
+          sudo gem install fpm --version 1.17.0 --no-document
+      - name: Exercise accepted and substituted payloads through both recipes
+        run: python3 scripts/test_release_packages.py
+
+  release-image-contract:
+    name: Release image contract (${{ matrix.variant }})
+    needs: workflow-pin-report
+    if: needs.workflow-pin-report.outputs.release_changed == 'true'
+    runs-on: ubuntu-latest
+    timeout-minutes: 35
+    strategy:
+      fail-fast: false
+      matrix:
+        variant: [musl, gnu]
+    env:
+      RELEASE_IMAGE_VARIANT: ${{ matrix.variant }}
+      NO_PROXY: 127.0.0.1,localhost,::1
+    steps:
+      - uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
+        with:
+          persist-credentials: false
+      - name: Install OCI tools
+        run: |
+          echo "RELEASE_IMAGE_OUTPUT=$RUNNER_TEMP/image-contract" >> "$GITHUB_ENV"
+          echo 'no_proxy=127.0.0.1,localhost,::1' >> "$GITHUB_ENV"
+          sudo apt-get update
+          sudo apt-get install -y skopeo
+          go version
+      - uses: docker/setup-qemu-action@c7c53464625b32c7a7e944ae62b3e17d2b600130 # v3
+      - uses: docker/setup-buildx-action@8d2750c68a42422c14e847fe6c8ac0403b4cbd6f # v3
+      - name: Exercise real OCI packaging, startup, registry retry and conflicts
+        run: python3 scripts/test_release_images.py
+      - name: Exercise amd64 OCI archive scanning
+        uses: aquasecurity/trivy-action@ed142fd0673e97e23eac54620cfb913e5ce36c25
+        env:
+          TRIVY_PLATFORM: linux/amd64
+        with:
+          input: ${{ runner.temp }}/image-contract-scan/image-${{ matrix.variant }}-amd64.tar
+          format: sarif
+          output: ${{ runner.temp }}/image-contract/scan-${{ matrix.variant }}-amd64.sarif
+          exit-code: "0"
+      - name: Exercise arm64 OCI archive scanning
+        uses: aquasecurity/trivy-action@ed142fd0673e97e23eac54620cfb913e5ce36c25
+        env:
+          TRIVY_PLATFORM: linux/arm64
+        with:
+          input: ${{ runner.temp }}/image-contract-scan/image-${{ matrix.variant }}-arm64.tar
+          format: sarif
+          output: ${{ runner.temp }}/image-contract/scan-${{ matrix.variant }}-arm64.sarif
+          exit-code: "0"
+      - name: Verify scan receipts before recording the archive
+        run: |
+          python3 - <<'PY'
+          import os, sys
+          from pathlib import Path
+          sys.path.insert(0, 'scripts/release')
+          import deliver, test_candidate
+          deliver.image_record(test_candidate.inputs(), os.environ['RELEASE_IMAGE_VARIANT'], Path(os.environ['RELEASE_IMAGE_OUTPUT']))
+          PY
+
  dependency-review:
    name: Dependency Review
    runs-on: ubuntu-latest
@@ -16,7 +16,9 @@ name: Continuous Integration

 on:
  push:
-    branches: [ main ]
+    # gateway/integration receives the daily merge of main as a direct push
+    # (rustfs/backlog#2734), so its pushes must prove the result like main's.
+    branches: [ main, gateway/integration ]
    paths-ignore:
      - "**.md"
      - "docs/**"
@@ -965,6 +967,8 @@ jobs:
    # Retain the producer image and fixture paths until the runtime pilot passes.
    runs-on: sm-standard-4
    timeout-minutes: 30
+    env:
+      RUSTFS_S3_STACK: legacy
    steps:
      - name: Checkout repository
        uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
@@ -1085,18 +1089,122 @@ jobs:
          path: ${{ runner.temp }}/rustfs-e2e-*/rustfs.log
          retention-days: 3

+  # The same e2e-smoke suite against the same debug binary with the gateway
+  # S3 stack selected (rustfs/backlog#2734). Report-only: it is outside the
+  # Test and Lint aggregate and becomes required at T2.10. The test-binary
+  # download, identity, membership, run and completeness steps must stay
+  # identical to e2e-tests; scripts/ci_gate.py enforces that.
+  e2e-tests-gateway:
+    name: e2e-smoke (gateway, report-only until T2.10)
+    needs: [ build-rustfs-debug-binary ]
+    runs-on: sm-standard-2
+    timeout-minutes: 30
+    continue-on-error: true
+    env:
+      RUSTFS_S3_STACK: gateway
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
+        with:
+          persist-credentials: false
+
+      - name: Install verification runtime
+        uses: ./.github/actions/test-runtime
+
+      - name: Download E2E test binaries
+        uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
+        with:
+          name: e2e-test-binaries
+          path: ${{ runner.temp }}
+
+      - name: Verify E2E archive identity
+        run: python3 scripts/ci_test_artifact.py verify --configuration e2e --archive "$RUNNER_TEMP/e2e-tests.tar.zst"
+
+      - name: Set up Python
+        uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
+        with:
+          python-version: "3.12"
+
+      - name: Install awscurl
+        run: |
+          python3 -m pip install --user --upgrade pip "awscurl==0.44"
+          echo "AWSCURL_PATH=$HOME/.local/bin/awscurl" >> "$GITHUB_ENV"
+
+      - name: Verify awscurl
+        run: test -x "$AWSCURL_PATH"
+
+      # Download after the cache restore so the freshly built binary from the
+      # build job always wins over anything restored into target/debug.
+      - name: Download debug binary
+        uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
+        with:
+          name: rustfs-debug-binary
+          path: target/debug
+
+      - name: Make binary executable
+        run: chmod +x ./target/debug/rustfs
+
+      # Build the e2e test graph once. The archive is reused by the smoke
+      # selection guard, security exact-count check, and run below, avoiding a
+      # second compile of the same e2e_test target on cold runners (backlog#1645).
+      - name: Verify e2e smoke test membership
+        env:
+          NEXTEST_ARCHIVE: ${{ runner.temp }}/e2e-tests.tar.zst
+          NEXTEST_LISTING: ${{ runner.temp }}/rustfs-e2e-smoke-list.json
+        run: |
+          cargo nextest list --profile e2e-smoke --archive-file "${NEXTEST_ARCHIVE}" --workspace-remap "$GITHUB_WORKSPACE" --extract-to "$GITHUB_WORKSPACE" --extract-overwrite --message-format json > "${NEXTEST_LISTING}"
+          python3 ./scripts/check_test_wiring.py --check-profile e2e-smoke "${NEXTEST_LISTING}"
+          ./scripts/check_security_smoke_count.sh check "${NEXTEST_LISTING}"
+
+      # PR smoke subset of the in-repo e2e suite (backlog#1149 ci-4). The
+      # profile.e2e-smoke default-filter in .config/nextest.toml is the single
+      # wiring mechanism for e2e tests in CI — extend that filter instead of
+      # adding new e2e jobs here. Each test spawns its own rustfs server on a
+      # random port and reuses the downloaded debug binary above.
+      - name: Run e2e smoke suite
+        env:
+          NEXTEST_ARCHIVE: ${{ runner.temp }}/e2e-tests.tar.zst
+          RUSTFS_E2E_LOG_DIR: ${{ runner.temp }}/rustfs-e2e-smoke-logs
+          SAMPLER_INTERVAL_SECS: "15"
+        run: |
+          ./scripts/ci/resource_sampler.sh start e2e-smoke
+          trap './scripts/ci/resource_sampler.sh stop' EXIT
+          python3 scripts/e2e_binary.py run --features e2e-test-hooks -- cargo nextest run --profile e2e-smoke --cargo-metadata target/nextest/cargo-metadata.json --binaries-metadata target/nextest/binaries-metadata.json --workspace-remap "$GITHUB_WORKSPACE" --target-dir-remap "$GITHUB_WORKSPACE/target" \
+            --status-level all --final-status-level all --failure-output final
+
+      - name: Verify e2e smoke execution completeness
+        run: python3 scripts/ci_test_artifact.py evidence "$RUNNER_TEMP/rustfs-e2e-smoke-list.json" target/nextest/e2e-smoke/junit.xml
+
+      - name: Upload e2e smoke diagnostics
+        if: always()
+        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+        with:
+          name: e2e-smoke-gateway-diagnostics-${{ github.run_number }}-${{ github.run_attempt }}
+          path: |
+            ${{ runner.temp }}/rustfs-e2e-smoke-logs/
+            ${{ runner.temp }}/rustfs-e2e-smoke-list.json
+          if-no-files-found: warn
+
+      - name: Upload e2e smoke JUnit report
+        if: always()
+        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+        with:
+          name: e2e-smoke-gateway-junit-${{ github.run_number }}-${{ github.run_attempt }}
+          path: target/nextest/e2e-smoke/junit.xml
+          if-no-files-found: warn
+
  e2e-full:
    name: End-to-End Tests (full merge gate)
    # Merge gate only (backlog#1149 ci-5): the never-automated user-visible
    # suites — KMS, object_lock, multipart_auth, quota, checksum, encryption,
    # security-boundary, ... — via the e2e-full nextest profile. Too heavy for
-    # every PR, so it is gated to main pushes, the merge queue, and manual
-    # dispatch. protocols / the 7 cluster suites / replication / #[ignore] are
-    # owned by other lanes (see .config/nextest.toml profile.e2e-full).
+    # every PR, so it is gated to main and gateway/integration pushes, the merge
+    # queue, and manual dispatch. protocols / the 7 cluster suites / replication
+    # / #[ignore] are owned by other lanes (see .config/nextest.toml profile.e2e-full).
    if: >-
      github.event_name == 'workflow_dispatch' ||
      github.event_name == 'merge_group' ||
-      (github.event_name == 'push' && github.ref == 'refs/heads/main')
+      (github.event_name == 'push' && (github.ref == 'refs/heads/main' || github.ref == 'refs/heads/gateway/integration'))
    needs: [ build-rustfs-debug-binary ]
    # Retain the producer image and fixture paths until the runtime pilot passes.
    runs-on: sm-standard-4
@@ -1,779 +1,99 @@
-# Copyright 2024 RustFS Team
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-# Docker Images Workflow
-#
-# This workflow builds Docker images using pre-built binaries from the build workflow.
-#
-# Trigger Types:
-# 1. workflow_run: Automatically triggered when "Build and Release" workflow completes
-# 2. workflow_dispatch: Manual trigger for standalone Docker builds
-#
-# Key Features:
-# - Only triggers when Linux builds (x86_64 + aarch64) are successful
-# - Independent of macOS/Windows build status
-# - Uses workflow_run event for precise control
-# - Only builds Docker images for releases and prereleases (development builds are skipped)
-
-name: Docker Images
-
-# Permissions needed for workflow_run event and Docker registry access
+# Copyright 2026 RustFS Team
+# SPDX-License-Identifier: Apache-2.0
+name: Build candidate images
+on:
+  workflow_call:
+    inputs:
+      candidate_manifest:
+        type: string
+        required: true
 permissions:
  contents: read
-  packages: write
-
-on:
-  # Automatically triggered when build workflow completes
-  workflow_run:
-    workflows: [ "Build and Release" ]
-    types: [ completed ]
-  # Manual trigger with same parameters for consistency
-  workflow_dispatch:
-    inputs:
-      push_images:
-        description: "Push images to registries"
-        required: false
-        default: true
-        type: boolean
-      version:
-        description: "Version to build (latest, v1.0.0, v1.0.0-alpha1, or v1.0.1-preview.1)"
-        required: false
-        default: "latest"
-        type: string
-      force_rebuild:
-        description: "Force rebuild even if binary exists (useful for testing)"
-        required: false
-        default: false
-        type: boolean
-
-env:
-  CONCLUSION: ${{ github.event.workflow_run.conclusion }}
-  HEAD_BRANCH: ${{ github.event.workflow_run.head_branch }}
-  HEAD_SHA: ${{ github.event.workflow_run.head_sha }}
-  TRIGGERING_EVENT: ${{ github.event.workflow_run.event }}
-  DOCKERHUB_USERNAME: rustfs
-  CARGO_TERM_COLOR: always
-  REGISTRY_DOCKERHUB: rustfs/rustfs
-  REGISTRY_GHCR: ghcr.io/${{ github.repository }}
-  REGISTRY_QUAY: quay.io/rustfs/rustfs
-  DOCKER_PLATFORMS: linux/amd64,linux/arm64
-
+  actions: read
 jobs:
-  # Check if we should build Docker images.
-  # Short-circuit at job level so per-merge development builds (workflow_run
-  # from a push to main) skip the whole run without spawning a checkout job:
-  # images are only published for tag builds — build.yml push triggers cover
-  # only main and tags, so a non-main head_branch is a tag name — and for
-  # manual dispatch.
-  build-check:
-    name: Docker Build Check
-    if: >-
-      github.event_name == 'workflow_dispatch' ||
-      (github.event.workflow_run.conclusion == 'success' &&
-       github.event.workflow_run.event == 'push' &&
-       github.event.workflow_run.head_branch != 'main')
-    runs-on: ubuntu-latest
-    timeout-minutes: 30
-    outputs:
-      should_build: ${{ steps.check.outputs.should_build }}
-      should_push: ${{ steps.check.outputs.should_push }}
-      build_type: ${{ steps.check.outputs.build_type }}
-      version: ${{ steps.check.outputs.version }}
-      short_sha: ${{ steps.check.outputs.short_sha }}
-      is_prerelease: ${{ steps.check.outputs.is_prerelease }}
-      create_latest: ${{ steps.check.outputs.create_latest }}
-      source_ref: ${{ steps.check.outputs.source_ref }}
-    steps:
-      - name: Checkout repository
-        uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
-        with:
-          persist-credentials: false
-          # For workflow_run events, checkout the specific commit that triggered the workflow
-          ref: ${{ github.event.workflow_run.head_sha || github.sha }}
-
-      - name: Check build conditions
-        id: check
-        env:
-          # dispatch inputs via env, not `${{ }}` interpolation: they are
-          # free-form strings and would otherwise be evaluated by bash.
-          INPUT_VERSION: ${{ github.event.inputs.version }}
-          INPUT_PUSH_IMAGES: ${{ github.event.inputs.push_images }}
-          INPUT_FORCE_REBUILD: ${{ github.event.inputs.force_rebuild }}
-        run: |
-          should_build=false
-          should_push=false
-          build_type="none"
-          version=""
-          short_sha=""
-          is_prerelease=false
-          create_latest=false
-          source_ref="$GITHUB_SHA"
-
-          # Pre-GA policy: until the first stable (vX.Y.Z) tag exists, every
-          # prerelease (alpha/beta/rc) also moves `latest`, so users pulling
-          # `latest` get the newest test build. Once a stable tag is published
-          # this returns false and `latest` follows stable releases only.
-          prerelease_moves_latest() {
-            local stable_tags
-            stable_tags=$(git ls-remote --tags --refs origin 2>/dev/null \
-              | awk '{print $2}' \
-              | grep -E '^refs/tags/v?[0-9]+\.[0-9]+\.[0-9]+$' || true)
-            if [[ -z "$stable_tags" ]]; then
-              return 0
-            fi
-            echo "ℹ️  Stable release tag(s) already exist; prereleases no longer update latest"
-            return 1
-          }
-
-          if [[ "${{ github.event_name }}" == "workflow_run" ]]; then
-            # Triggered by build workflow completion
-            echo "🔗 Triggered by build workflow completion"
-
-            # Check if the triggering workflow was successful
-            # If the workflow succeeded, it means ALL builds (including Linux x86_64 and aarch64) succeeded
-            if [[ "$CONCLUSION" == "success" ]]; then
-              echo "✅ Build workflow succeeded, all builds including Linux are successful"
-              should_build=true
-              should_push=true
-            else
-              echo "❌ Build workflow failed (conclusion: $CONCLUSION), skipping Docker build"
-              should_build=false
-            fi
-
-            # Extract version info from commit message or use commit SHA
-            # Use Git to generate consistent short SHA (ensures uniqueness like build.yml)
-            short_sha=$(git rev-parse --short "$HEAD_SHA")
-            source_ref="$HEAD_SHA"
-
-            # Determine build type based on triggering workflow event and ref
-            triggering_event="$TRIGGERING_EVENT"
-            head_branch="$HEAD_BRANCH"
-
-            echo "🔍 Analyzing triggering workflow:"
-            echo "   📋 Event: $triggering_event"
-            echo "   🌿 Head branch: $head_branch"
-            echo "   📎 Head SHA: $HEAD_SHA"
-
-            # Check if this was triggered by a tag push
-            if [[ "$triggering_event" == "push" ]]; then
-              # For tag pushes, head_branch will be like "refs/tags/v1.0.0" or just "v1.0.0"
-              if [[ "$head_branch" == refs/tags/* ]]; then
-                # Extract tag name from refs/tags/TAG_NAME
-                tag_name="${head_branch#refs/tags/}"
-                version="$tag_name"
-              elif [[ "$head_branch" =~ ^v?[0-9]+\.[0-9]+\.[0-9]+ ]]; then
-                # Direct tag name like "v1.0.0" or "1.0.0-alpha.1"
-                version="$head_branch"
-              elif [[ "$head_branch" == "main" ]]; then
-                # Regular branch push to main
-                build_type="development"
-                version="dev-${short_sha}"
-                should_build=false
-                echo "⏭️  Skipping Docker build for development version (main branch push)"
-              else
-                # Other branch push
-                build_type="development"
-                version="dev-${short_sha}"
-                should_build=false
-                echo "⏭️  Skipping Docker build for development version (branch: $head_branch)"
-              fi
-
-              # If we extracted a version (tag), determine release type
-              if [[ -n "$version" ]] && [[ "$version" != "dev-${short_sha}" ]]; then
-                # Remove 'v' prefix if present for consistent version format
-                if [[ "$version" == v* ]]; then
-                  version="${version#v}"
-                fi
-
-                if [[ "$version" =~ -preview\.[0-9]+$ ]]; then
-                  build_type="preview"
-                  is_prerelease=true
-                  echo "🔍 Building Docker image for preview: $version (version tags only)"
-                elif [[ "$version" == *"-preview"* ]]; then
-                  echo "❌ Invalid preview tag: $version (expected suffix: -preview.<number>)" >&2
-                  exit 1
-                elif [[ "$version" == *"alpha"* ]] || [[ "$version" == *"beta"* ]] || [[ "$version" == *"rc"* ]]; then
-                  build_type="prerelease"
-                  is_prerelease=true
-                  # Pre-GA policy: prereleases update latest until the first stable tag exists.
-                  if prerelease_moves_latest; then
-                    create_latest=true
-                    echo "🧪 Building Docker image for prerelease: $version (creating latest tag)"
-                  else
-                    echo "🧪 Building Docker image for prerelease: $version"
-                  fi
-                else
-                  build_type="release"
-                  create_latest=true
-                  echo "🚀 Building Docker image for release: $version"
-                fi
-              fi
-            else
-              # Non-push events
-              build_type="development"
-              version="dev-${short_sha}"
-              should_build=false
-              echo "⏭️  Skipping Docker build for development version (event: $triggering_event)"
-            fi
-
-            echo "🔄 Build triggered by workflow_run:"
-            echo "   📋 Conclusion: $CONCLUSION"
-            echo "   🌿 Branch: $HEAD_BRANCH"
-            echo "   📎 SHA: $HEAD_SHA"
-            echo "   🎯 Event: $TRIGGERING_EVENT"
-
-          elif [[ "${{ github.event_name }}" == "workflow_dispatch" ]]; then
-            # Manual trigger
-            input_version="$INPUT_VERSION"
-            version="${input_version}"
-            should_push="$INPUT_PUSH_IMAGES"
-            should_build=true
-
-            # Get short SHA
-            short_sha=$(git rev-parse --short HEAD)
-
-            echo "🎯 Manual Docker build triggered:"
-            echo "   📋 Requested version: $input_version"
-            echo "   🔧 Force rebuild: $INPUT_FORCE_REBUILD"
-            echo "   🚀 Push images: $should_push"
-
-            case "$input_version" in
-              "latest")
-                build_type="release"
-                create_latest=true
-                echo "🚀 Building with latest stable release version"
-                ;;
-              *-preview*)
-                if [[ ! "$input_version" =~ ^v?[0-9]+\.[0-9]+\.[0-9]+(-[0-9A-Za-z.-]+)?-preview\.[0-9]+$ ]]; then
-                  echo "❌ Invalid preview version: $input_version" >&2
-                  exit 1
-                fi
-                build_type="preview"
-                is_prerelease=true
-                version="${input_version#v}"
-                echo "🔍 Building with preview version: $version (version tags only)"
-                ;;
-              # Prerelease versions (must match first, more specific)
-              v*alpha*|v*beta*|v*rc*|*alpha*|*beta*|*rc*)
-                build_type="prerelease"
-                is_prerelease=true
-                # Pre-GA policy: prereleases update latest until the first stable tag exists.
-                if prerelease_moves_latest; then
-                  create_latest=true
-                  echo "🧪 Building with prerelease version: $input_version (creating latest tag)"
-                else
-                  echo "🧪 Building with prerelease version: $input_version"
-                fi
-                ;;
-              # Release versions (match after prereleases, more general)
-              v[0-9]*|[0-9]*.*.*)
-                build_type="release"
-                create_latest=true
-                echo "📦 Building with specific release version: $input_version"
-                ;;
-              *)
-                # Invalid version for Docker build
-                should_build=false
-                echo "❌ Invalid version for Docker build: $input_version"
-                echo "⚠️  Only release versions (latest, v1.0.0, 1.0.0) and prereleases (v1.0.0-alpha1, 1.0.0-beta2) are supported"
-                ;;
-            esac
-
-            if [[ "$should_build" == true && "$input_version" != "latest" ]]; then
-              tag_ref="refs/tags/$input_version"
-              if ! git ls-remote --exit-code origin "$tag_ref" >/dev/null 2>&1; then
-                if [[ "$input_version" == v* ]]; then
-                  tag_ref="refs/tags/${input_version#v}"
-                else
-                  tag_ref="refs/tags/v$input_version"
-                fi
-              fi
-
-              if ! git ls-remote --exit-code origin "$tag_ref" >/dev/null 2>&1; then
-                echo "❌ Release tag not found for Docker build: $input_version"
-                exit 1
-              fi
-              source_ref="$tag_ref"
-            fi
-          fi
-
-          {
-            echo "should_build=$should_build"
-            echo "should_push=$should_push"
-            echo "build_type=$build_type"
-            echo "version=$version"
-            echo "short_sha=$short_sha"
-            echo "is_prerelease=$is_prerelease"
-            echo "create_latest=$create_latest"
-            echo "source_ref=$source_ref"
-          } >> "$GITHUB_OUTPUT"
-
-          echo "🐳 Docker Build Summary:"
-          echo "  - Should build: $should_build"
-          echo "  - Should push: $should_push"
-          echo "  - Build type: $build_type"
-          echo "  - Version: $version"
-          echo "  - Short SHA: $short_sha"
-          echo "  - Is prerelease: $is_prerelease"
-          echo "  - Create latest: $create_latest"
-          echo "  - Source ref: $source_ref"
-
-  # Consume the trusted release certification for the exact version this
-  # workflow wraps and verify the certified input digests. A manual dispatch
-  # without that certification fails here and pushes no official tag; only
-  # results from the trusted verification sources carry release authority.
-  release-certification:
-    name: Release Certification
-    needs: build-check
-    if: needs.build-check.outputs.should_build == 'true'
-    runs-on: ubuntu-latest
-    timeout-minutes: 30
-    permissions:
-      contents: read
-      actions: read
-    steps:
-      - name: Checkout repository
-        uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
-        with:
-          persist-credentials: false
-          ref: ${{ needs.build-check.outputs.source_ref }}
-
-      - name: Download and verify release certification
-        env:
-          GH_TOKEN: ${{ secrets.RELEASE_VALIDATION_TOKEN || github.token }}
-          TRIGGERING_RUN_ID: ${{ github.event.workflow_run.id }}
-          VERSION: ${{ needs.build-check.outputs.version }}
-        shell: bash
-        run: |
-          set -euo pipefail
-          mkdir -p certification
-          run_id="$TRIGGERING_RUN_ID"
-          if [[ -z "$run_id" ]]; then
-            run_id="$(gh api --method GET \
-              "repos/${GITHUB_REPOSITORY}/actions/workflows/build.yml/runs" \
-              -f branch="$VERSION" -f status=success -F per_page=1 \
-              --jq '.workflow_runs[0].id // empty' || true)"
-          fi
-          if [[ -z "$run_id" ]]; then
-            echo "No certified Build and Release run found for ${VERSION}; failing closed" >&2
-            exit 1
-          fi
-          if ! gh run download "$run_id" -n release-certification -D certification; then
-            echo "Build run ${run_id} published no release-certification artifact; failing closed" >&2
-            exit 1
-          fi
-          python3 scripts/release/release_gate.py verify-promotion \
-            --plan .config/release-validation-plan.json \
-            --candidate certification/candidate.json \
-            --summary certification/release-gate-output \
-            --artifact-sha256 "$(jq -r '.deliverables[0].sha256' certification/candidate.json)" \
-            --artifact-name "$(jq -r '.deliverables[0].name' certification/candidate.json)"
-
-          # Input digest binding: every zip this image wraps must be the
-          # certified byte-for-byte content, never a mutable latest pointer.
-          while IFS=$'\t' read -r name location expected; do
-            echo "Verifying certified input ${name}"
-            actual="$(curl -fsSL "$location" | sha256sum | awk '{print $1}')"
-            if [[ "$actual" != "$expected" ]]; then
-              echo "Certified input ${name} does not match its recorded digest" >&2
-              exit 1
-            fi
-          done < <(jq -r '.deliverables[] | select(.name | contains("linux")) | [.name, .location, .sha256] | @tsv' certification/candidate.json)
-
-  # Build multi-arch Docker images
-  # Strategy: Build images using pre-built binaries from dl.rustfs.com
-  # Supports both release and dev channel binaries based on build context
-  # Only runs when should_build is true (which includes workflow success check)
  build-docker:
-    name: Build Docker Images
-    needs: [ build-check, release-certification ]
-    if: needs.build-check.outputs.should_build == 'true'
    runs-on: ubuntu-latest
-    timeout-minutes: 60
+    timeout-minutes: 90
    strategy:
      fail-fast: false
      matrix:
        include:
          - variant: musl
            file: Dockerfile
-            suffix: ""
-          - variant: glibc
+          - variant: gnu
            file: Dockerfile.glibc
-            suffix: "-glibc"
-
+    env:
+      CANDIDATE_MANIFEST: ${{ inputs.candidate_manifest }}
+      VARIANT: ${{ matrix.variant }}
    steps:
-      - name: Checkout repository
-        uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
+      - uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
        with:
+          ref: ${{ fromJson(inputs.candidate_manifest).source_sha }}
          persist-credentials: false
-          ref: ${{ needs.build-check.outputs.source_ref }}
-
-      - name: Login to Docker Hub
-        uses: docker/login-action@c94ce9fb468520275223c153574b00df6fe4bcc9 # v3
-        with:
-          username: ${{ env.DOCKERHUB_USERNAME }}
-          password: ${{ secrets.DOCKERHUB_TOKEN }}
-
-      - name: Login to GitHub Container Registry
-        uses: docker/login-action@c94ce9fb468520275223c153574b00df6fe4bcc9 # v3
-        with:
-          registry: ghcr.io
-          username: ${{ secrets.GHCR_USERNAME }}
-          password: ${{ secrets.GHCR_PASSWORD }}
-
-      - name: Login to Quay.io
-        uses: docker/login-action@c94ce9fb468520275223c153574b00df6fe4bcc9 # v3
-        with:
-          registry: quay.io
-          username: ${{ secrets.QUAY_USERNAME }}
-          password: ${{ secrets.QUAY_PASSWORD }}
-
-      - name: Set up QEMU
-        uses: docker/setup-qemu-action@c7c53464625b32c7a7e944ae62b3e17d2b600130 # v3
-
-      - name: Set up Docker Buildx
-        uses: docker/setup-buildx-action@8d2750c68a42422c14e847fe6c8ac0403b4cbd6f # v3
-
-      - name: Extract metadata and generate tags
-        id: meta
+      - name: Reuse completed image
+        id: reuse
+        env:
+          GH_TOKEN: ${{ github.token }}
+        run: python3 scripts/release/deliver.py reuse --kind image --target "$VARIANT"
+      - name: Install OCI tools
+        if: steps.reuse.outputs.reused != 'true'
        run: |
-          BUILD_TYPE="${{ needs.build-check.outputs.build_type }}"
-          VERSION="${{ needs.build-check.outputs.version }}"
-          CREATE_LATEST="${{ needs.build-check.outputs.create_latest }}"
-          VARIANT_SUFFIX="${{ matrix.suffix }}"
-
-          # Convert version format for Dockerfile compatibility. The former
-          # DOCKER_CHANNEL was "release" down every branch and was passed as a
-          # build-arg no Dockerfile declares, so it is gone.
-          case "$VERSION" in
-            "latest")
-              DOCKER_RELEASE="latest"
-              ;;
-            v*)
-              # For versioned releases (v1.0.0), remove 'v' prefix for Dockerfile
-              DOCKER_RELEASE="${VERSION#v}"
-              ;;
-            *)
-              # For other versions, pass as-is
-              DOCKER_RELEASE="${VERSION}"
-              ;;
-          esac
-
-          echo "docker_release=$DOCKER_RELEASE" >> "$GITHUB_OUTPUT"
-
-          echo "🐳 Docker build parameters:"
-          echo "  - Original version: $VERSION"
-          echo "  - Docker RELEASE: $DOCKER_RELEASE"
-
-          # Generate tags based on build type
-          # Only support release and prerelease builds (no development builds)
-          TAG_BASE="${VERSION}${VARIANT_SUFFIX}"
-          TAGS="${{ env.REGISTRY_DOCKERHUB }}:$TAG_BASE,${{ env.REGISTRY_GHCR }}:$TAG_BASE,${{ env.REGISTRY_QUAY }}:$TAG_BASE"
-
-          # Add latest when requested (stable releases, and prereleases before GA)
-          if [[ "$CREATE_LATEST" == "true" ]]; then
-            TAGS="$TAGS,${{ env.REGISTRY_DOCKERHUB }}:latest${VARIANT_SUFFIX},${{ env.REGISTRY_GHCR }}:latest${VARIANT_SUFFIX},${{ env.REGISTRY_QUAY }}:latest${VARIANT_SUFFIX}"
-          fi
-
-          # Always add the channel tag for prereleases, independent of latest
-          if [[ "$BUILD_TYPE" == "prerelease" ]]; then
-            # Prerelease channel tags (alpha, beta, rc)
-            if [[ "$VERSION" == *"alpha"* ]]; then
-              CHANNEL="alpha"
-            elif [[ "$VERSION" == *"beta"* ]]; then
-              CHANNEL="beta"
-            elif [[ "$VERSION" == *"rc"* ]]; then
-              CHANNEL="rc"
-            fi
-
-            if [[ -n "$CHANNEL" ]]; then
-              TAGS="$TAGS,${{ env.REGISTRY_DOCKERHUB }}:${CHANNEL}${VARIANT_SUFFIX},${{ env.REGISTRY_GHCR }}:${CHANNEL}${VARIANT_SUFFIX},${{ env.REGISTRY_QUAY }}:${CHANNEL}${VARIANT_SUFFIX}"
-            fi
-          fi
-
-          # Output tags
-          echo "tags=$TAGS" >> "$GITHUB_OUTPUT"
-
-          # Staging identity for this run's promotion: official tags are only
-          # created after the staged image verifies against the certification,
-          # and they point at this same digest (no rebuild-after-verify).
-          STAGING_TAG="staging-${GITHUB_RUN_ID}-${GITHUB_RUN_ATTEMPT}${VARIANT_SUFFIX}"
-          STAGING_TAGS="${{ env.REGISTRY_DOCKERHUB }}:${STAGING_TAG},${{ env.REGISTRY_GHCR }}:${STAGING_TAG},${{ env.REGISTRY_QUAY }}:${STAGING_TAG}"
-          echo "staging_tags=$STAGING_TAGS" >> "$GITHUB_OUTPUT"
-          echo "staging_tag=$STAGING_TAG" >> "$GITHUB_OUTPUT"
-
-          # Generate labels
-          LABELS="org.opencontainers.image.title=RustFS"
-          LABELS="$LABELS,org.opencontainers.image.description=RustFS distributed object storage system"
-          LABELS="$LABELS,org.opencontainers.image.version=$VERSION"
-          SOURCE_REVISION="$(git rev-parse HEAD)"
-          LABELS="$LABELS,org.opencontainers.image.revision=$SOURCE_REVISION"
-          LABELS="$LABELS,org.opencontainers.image.source=${{ github.server_url }}/${{ github.repository }}"
-          LABELS="$LABELS,org.opencontainers.image.created=$(date -u +'%Y-%m-%dT%H:%M:%SZ')"
-          LABELS="$LABELS,org.opencontainers.image.build-type=$BUILD_TYPE"
-
-          echo "labels=$LABELS" >> "$GITHUB_OUTPUT"
-
-          echo "🐳 Generated Docker tags:"
-          echo "$TAGS" | tr ',' '\n' | sed 's/^/  - /'
-          echo "📋 Build type: $BUILD_TYPE"
-          echo "🔖 Version: $VERSION"
-
-      - name: Build and push Docker image
+          sudo apt-get update
+          sudo apt-get install -y skopeo
+      - uses: docker/setup-qemu-action@c7c53464625b32c7a7e944ae62b3e17d2b600130 # v3
+        if: steps.reuse.outputs.reused != 'true'
+      - uses: docker/setup-buildx-action@8d2750c68a42422c14e847fe6c8ac0403b4cbd6f # v3
+        if: steps.reuse.outputs.reused != 'true'
+      - name: Supply accepted binaries and pinned CLI
+        if: steps.reuse.outputs.reused != 'true'
+        env:
+          GH_TOKEN: ${{ github.token }}
+        run: python3 scripts/release/deliver.py image-prepare --target "$VARIANT"
+      - name: Build OCI archive without registry writes
+        if: steps.reuse.outputs.reused != 'true'
        uses: docker/build-push-action@10e90e3645eae34f1e60eeb005ba3a3d33f178e8 # v6
        with:
          context: .
          file: ${{ matrix.file }}
-          platforms: ${{ env.DOCKER_PLATFORMS }}
-          push: ${{ needs.build-check.outputs.should_push == 'true' }}
-          # Push to staging tags only: official tags are created by the
-          # certification-gated promotion step below from this same digest.
-          tags: ${{ steps.meta.outputs.staging_tags }}
-          labels: ${{ steps.meta.outputs.labels }}
-          # No layer cache. This build compiles nothing — it downloads a
-          # release zip and runs apk/apt — so the cache could only save the
-          # minute or two those take, while creating a correctness problem: with
-          # RELEASE=latest the binary URL is resolved by curl *inside* a RUN
-          # layer, and the layer key does not include what that resolved to. A
-          # rebuild at the same RELEASE value (dispatch with version=latest, or
-          # a re-run of the same version) would hit the old layer and ship the
-          # previous release's binary. mode=max also consumed the same 10GB
-          # Actions cache quota the Rust lanes are fighting over.
-          #
-          # Only RELEASE is passed: it is the sole build-arg the Dockerfiles
-          # declare besides TARGETARCH. BUILDTIME, VERSION, BUILD_TYPE, REVISION
-          # and CHANNEL were never read by any stage (and BUILDTIME's $(date ...)
-          # was a literal here, not a shell substitution). BUILD_DATE and VCS_REF
-          # are declared by the Dockerfiles but deliberately left unset —
-          # supplying them would change the published image labels.
+          platforms: linux/amd64,linux/arm64
+          push: false
+          provenance: false
+          outputs: type=oci,dest=delivery/image-${{ matrix.variant }}.tar
          build-args: |
-            RELEASE=${{ steps.meta.outputs.docker_release }}
-          provenance: true
-          sbom: true
-          # Add retry mechanism by splitting the build process
-          no-cache: false
-          pull: true
-
-      # A later-built wrapper: the staged image must complete this platform's
-      # own binary identity verification before any official tag is written.
-      # It may not inherit an unverified package's conclusion.
-      - name: Verify staged image against certification
-        if: needs.build-check.outputs.should_push == 'true'
-        env:
-          GH_TOKEN: ${{ secrets.RELEASE_VALIDATION_TOKEN || github.token }}
-          TRIGGERING_RUN_ID: ${{ github.event.workflow_run.id }}
-          VERSION: ${{ needs.build-check.outputs.version }}
-        shell: bash
-        run: |
-          set -euo pipefail
-          mkdir -p certification
-          run_id="$TRIGGERING_RUN_ID"
-          if [[ -z "$run_id" ]]; then
-            run_id="$(gh api --method GET \
-              "repos/${GITHUB_REPOSITORY}/actions/workflows/build.yml/runs" \
-              -f branch="$VERSION" -f status=success -F per_page=1 \
-              --jq '.workflow_runs[0].id // empty' || true)"
-          fi
-          gh run download "$run_id" -n release-certification -D certification
-
-          STAGING_REF="${{ env.REGISTRY_DOCKERHUB }}:staging-${GITHUB_RUN_ID}-${GITHUB_RUN_ATTEMPT}${{ matrix.suffix }}"
-          case "${{ matrix.variant }}" in
-            musl) libc="musl" ;;
-            glibc) libc="gnu" ;;
-          esac
-          for platform in linux/amd64 linux/arm64; do
-            case "$platform" in
-              linux/amd64) arch_sub="x86_64" ;;
-              linux/arm64) arch_sub="aarch64" ;;
-            esac
-            docker pull --platform "$platform" "$STAGING_REF"
-            container="$(docker create --platform "$platform" "$STAGING_REF")"
-            docker cp "$container:/usr/bin/rustfs" "$RUNNER_TEMP/staged-rustfs"
-            docker rm "$container" >/dev/null
-            actual="$(sha256sum "$RUNNER_TEMP/staged-rustfs" | awk '{print $1}')"
-            expected="$(jq -r --arg needle "linux-${arch_sub}-${libc}" \
-              '[.deliverables[] | select(.name | contains($needle)) | .binary_sha256][0] // empty' \
-              certification/candidate.json)"
-            if [[ -z "$expected" || "$actual" != "$expected" ]]; then
-              echo "Staged image binary for ${platform} does not match the certified binary digest" >&2
-              exit 1
-            fi
-            python3 scripts/release/release_gate.py verify-promotion \
-              --plan .config/release-validation-plan.json \
-              --candidate certification/candidate.json \
-              --summary certification/release-gate-output \
-              --artifact "$RUNNER_TEMP/staged-rustfs" \
-              --artifact-role binary
-          done
-
-      - name: Promote staged image to official tags
-        if: needs.build-check.outputs.should_push == 'true'
-        shell: bash
-        run: |
-          set -euo pipefail
-          STAGING_REF="${{ env.REGISTRY_DOCKERHUB }}:staging-${GITHUB_RUN_ID}-${GITHUB_RUN_ATTEMPT}${{ matrix.suffix }}"
-          IFS=',' read -r -a official_tags <<< "${{ steps.meta.outputs.tags }}"
-          for tag in "${official_tags[@]}"; do
-            docker buildx imagetools create -t "$tag" "$STAGING_REF"
-            echo "Promoted $tag -> $STAGING_REF"
-          done
-
-      # Downstream consumers (Helm) verify this record instead of inheriting
-      # an unverified wrapper's conclusion.
-      - name: Record image promotion
-        if: needs.build-check.outputs.should_push == 'true'
-        shell: bash
-        env:
-          GH_TOKEN: ${{ secrets.RELEASE_VALIDATION_TOKEN || github.token }}
-          TRIGGERING_RUN_ID: ${{ github.event.workflow_run.id }}
-          VERSION: ${{ needs.build-check.outputs.version }}
-        run: |
-          set -euo pipefail
-          mkdir -p certification
-          run_id="$TRIGGERING_RUN_ID"
-          if [[ -z "$run_id" ]]; then
-            run_id="$(gh api --method GET \
-              "repos/${GITHUB_REPOSITORY}/actions/workflows/build.yml/runs" \
-              -f branch="$VERSION" -f status=success -F per_page=1 \
-              --jq '.workflow_runs[0].id // empty' || true)"
-          fi
-          gh run download "$run_id" -n release-certification -D certification
-          STAGING_REF="${{ env.REGISTRY_DOCKERHUB }}:staging-${GITHUB_RUN_ID}-${GITHUB_RUN_ATTEMPT}${{ matrix.suffix }}"
-          digest="$(docker buildx imagetools inspect "$STAGING_REF" --format '{{json .Manifest}}' | jq -r '.digest')"
-          candidate_sha="$(python3 -c "import json,hashlib,sys; sys.path.insert(0,'scripts/release'); import release_gate; print(release_gate.canonical_sha256(json.load(open('certification/candidate.json'))))")"
-          jq -n \
-            --arg version "$VERSION" \
-            --arg digest "$digest" \
-            --arg staging_ref "$STAGING_REF" \
-            --arg candidate_sha256 "$candidate_sha" \
-            --arg tags "${{ steps.meta.outputs.tags }}" \
-            --slurpfile candidate certification/candidate.json \
-            '{schema_version: 1, version: $version, image_digest: $digest, staging_ref: $staging_ref,
-              promoted_tags: ($tags | split(",")), candidate_sha256: $candidate_sha256,
-              binary_sha256s: ($candidate[0].deliverables | map(.binary_sha256) | unique)}' \
-            > image-promotion-record.json
-
-      - name: Upload image promotion record
-        if: needs.build-check.outputs.should_push == 'true'
-        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
-        with:
-          name: image-promotion-record${{ matrix.suffix }}
-          path: image-promotion-record.json
-          if-no-files-found: error
-          retention-days: 30
-
-  # Note: Manifest creation is no longer needed as we only build one variant
-  # Multi-arch manifests are automatically created by docker/build-push-action
-
-  # Report-only container image vulnerability scan
-  scan-docker-image:
-    name: Scan Docker Images
-    needs: [ build-check, build-docker ]
-    if: needs.build-check.outputs.should_build == 'true' && needs.build-check.outputs.should_push == 'true'
-    runs-on: ubuntu-latest
-    timeout-minutes: 30
-    permissions:
-      contents: read
-      security-events: write
-    strategy:
-      fail-fast: false
-      matrix:
-        include:
-          - variant: musl
-            suffix: ""
-          - variant: glibc
-            suffix: "-glibc"
-    steps:
-      - name: Login to GitHub Container Registry
-        # docker/login-action v3
-        uses: docker/login-action@c94ce9fb468520275223c153574b00df6fe4bcc9
-        with:
-          registry: ghcr.io
-          username: ${{ secrets.GHCR_USERNAME }}
-          password: ${{ secrets.GHCR_PASSWORD }}
-
-      - name: Scan image with Trivy
-        # aquasecurity/trivy-action v0.36.0
+            BINARY_SOURCE=supplied
+            RELEASE=${{ fromJson(inputs.candidate_manifest).version }}
+            VCS_REF=${{ fromJson(inputs.candidate_manifest).source_sha }}
+      - name: Verify wrapped binaries and startup on both architectures
+        if: steps.reuse.outputs.reused != 'true'
+        run: python3 scripts/release/deliver.py image-smoke --target "$VARIANT"
+      - name: Scan amd64 image before sealing
+        if: steps.reuse.outputs.reused != 'true'
        uses: aquasecurity/trivy-action@ed142fd0673e97e23eac54620cfb913e5ce36c25
+        env:
+          TRIVY_PLATFORM: linux/amd64
        with:
-          image-ref: ${{ env.REGISTRY_GHCR }}:${{ needs.build-check.outputs.version }}${{ matrix.suffix }}
+          input: delivery-scan/image-${{ matrix.variant }}-amd64.tar
          format: sarif
-          output: trivy-${{ matrix.variant }}.sarif
-          ignore-unfixed: true
-          vuln-type: os,library
-          severity: CRITICAL,HIGH
+          output: delivery/scan-${{ matrix.variant }}-amd64.sarif
+          exit-code: "0" # Keep the repository's report-only vulnerability policy.
+      - name: Scan arm64 image before sealing
+        if: steps.reuse.outputs.reused != 'true'
+        uses: aquasecurity/trivy-action@ed142fd0673e97e23eac54620cfb913e5ce36c25
+        env:
+          TRIVY_PLATFORM: linux/arm64
+        with:
+          input: delivery-scan/image-${{ matrix.variant }}-arm64.tar
+          format: sarif
+          output: delivery/scan-${{ matrix.variant }}-arm64.sarif
          exit-code: "0"
-
-      # Surface findings in the Security tab; an artifact alone is write-only
-      # reporting nobody reviews.
-      - name: Upload scan results to code scanning
-        uses: github/codeql-action/upload-sarif@99df26d4f13ea111d4ec1a7dddef6063f76b97e9 # v4.37.0
+      - name: Record verified OCI identity
+        if: steps.reuse.outputs.reused != 'true'
+        run: python3 scripts/release/deliver.py image-record --target "$VARIANT"
+      - uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+        if: steps.reuse.outputs.reused != 'true'
        with:
-          sarif_file: trivy-${{ matrix.variant }}.sarif
-          category: container-image-${{ matrix.variant }}
-
-      - name: Upload container scan report
-        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
-        with:
-          name: container-image-scan-${{ matrix.variant }}
-          path: trivy-${{ matrix.variant }}.sarif
+          name: image-${{ matrix.variant }}
+          path: delivery/*
          if-no-files-found: error
          retention-days: 30
-
-  # Docker build summary
-  docker-summary:
-    name: Docker Build Summary
-    needs: [ build-check, build-docker ]
-    if: always() && needs.build-check.outputs.should_build == 'true'
-    runs-on: ubuntu-latest
-    timeout-minutes: 10
-    steps:
-      - name: Docker build completion summary
-        run: |
-          BUILD_TYPE="${{ needs.build-check.outputs.build_type }}"
-          VERSION="${{ needs.build-check.outputs.version }}"
-          CREATE_LATEST="${{ needs.build-check.outputs.create_latest }}"
-
-          echo "🐳 Docker build completed successfully!"
-          echo "📦 Build type: $BUILD_TYPE"
-          echo "🔢 Version: $VERSION"
-          echo "🚀 Strategy: Images using pre-built binaries (release channel only)"
-          echo ""
-
-          case "$BUILD_TYPE" in
-            "preview")
-              echo "🔍 Preview Docker image has been built with ${VERSION} tags"
-              echo "⏭️  Preview images do not update latest or prerelease channels"
-              ;;
-            "release")
-              echo "🚀 Release Docker image has been built with ${VERSION} tags"
-              echo "✅ This image is ready for production use"
-              if [[ "$CREATE_LATEST" == "true" ]]; then
-                echo "🏷️  Latest tag has been created for stable release"
-              fi
-              ;;
-            "prerelease")
-              echo "🧪 Prerelease Docker image has been built with ${VERSION} tags"
-              echo "⚠️  This is a prerelease image - use with caution"
-              # Prereleases move latest until the first stable tag exists (pre-GA policy).
-              if [[ "$CREATE_LATEST" == "true" ]]; then
-                echo "🏷️  Latest tag has been created for prerelease: $VERSION"
-              else
-                echo "🚫 Latest tag NOT created for prerelease"
-              fi
-              ;;
-            *)
-              echo "❌ Unexpected build type: $BUILD_TYPE"
-              ;;
-          esac
+          compression-level: 0
@@ -1,229 +1,26 @@
-# Copyright 2024 RustFS Team
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-name: Publish helm chart to artifacthub
-
+# Copyright 2026 RustFS Team
+# SPDX-License-Identifier: Apache-2.0
+name: Build candidate Helm chart
 on:
-  workflow_run:
-    workflows: [ "Build and Release" ]
-    types: [ completed ]
-  workflow_dispatch:
+  workflow_call:
    inputs:
-      version:
-        description: "Release version to publish, e.g. 1.0.0-beta.1 or v1.0.0-beta.1"
-        required: true
-        default: "1.0.0-beta.1"
+      candidate_manifest:
        type: string
-
+        required: true
 permissions:
  contents: read
-
+  actions: read
 jobs:
-  # Consume the same trusted release certification as every other publication
-  # exit and verify the image promotion record of the Docker workflow. The
-  # chart wraps no binary of its own, so it may only reference an image whose
-  # platform verification already passed; it must not inherit an unverified
-  # package's conclusion. A run without that evidence fails closed here and
-  # never reaches the chart repository.
-  release-certification:
+  package:
    runs-on: ubuntu-latest
-    timeout-minutes: 30
-    permissions:
-      contents: read
-      actions: read
-    outputs:
-      version: ${{ steps.version.outputs.raw_tag }}
+    timeout-minutes: 15
    steps:
-      - name: Checkout repository
-        uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
+      - uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
        with:
+          ref: ${{ fromJson(inputs.candidate_manifest).source_sha }}
          persist-credentials: false
-
-      - name: Normalize release version
-        id: version
-        env:
-          RAW_INPUT: ${{ github.event.inputs.version }}
-          RAW_BRANCH: ${{ github.event.workflow_run.head_branch }}
-        run: |
-          set -euo pipefail
-          if [ "${{ github.event_name }}" = "workflow_dispatch" ]; then
-            RAW="$RAW_INPUT"
-          else
-            RAW="$RAW_BRANCH"
-          fi
-          case "$RAW" in
-            refs/tags/*) RAW_TAG="${RAW#refs/tags/}" ;;
-            *) RAW_TAG="$RAW" ;;
-          esac
-          echo "raw_tag=$RAW_TAG" >> "$GITHUB_OUTPUT"
-
-      - name: Download and verify release certification and image record
-        env:
-          GH_TOKEN: ${{ secrets.RELEASE_VALIDATION_TOKEN || github.token }}
-          TRIGGERING_RUN_ID: ${{ github.event.workflow_run.id }}
-          RAW_TAG: ${{ steps.version.outputs.raw_tag }}
-        shell: bash
-        run: |
-          set -euo pipefail
-          mkdir -p certification image-record
-
-          build_run="$TRIGGERING_RUN_ID"
-          if [[ -z "$build_run" ]]; then
-            build_run="$(gh api --method GET \
-              "repos/${GITHUB_REPOSITORY}/actions/workflows/build.yml/runs" \
-              -f branch="$RAW_TAG" -f status=success -F per_page=1 \
-              --jq '.workflow_runs[0].id // empty' || true)"
-          fi
-          if [[ -z "$build_run" ]]; then
-            echo "No certified Build and Release run found for ${RAW_TAG}; failing closed" >&2
-            exit 1
-          fi
-          if ! gh run download "$build_run" -n release-certification -D certification; then
-            echo "Build run ${build_run} published no release-certification artifact; failing closed" >&2
-            exit 1
-          fi
-          python3 scripts/release/release_gate.py verify-promotion \
-            --plan .config/release-validation-plan.json \
-            --candidate certification/candidate.json \
-            --summary certification/release-gate-output \
-            --artifact-sha256 "$(jq -r '.deliverables[0].sha256' certification/candidate.json)" \
-            --artifact-name "$(jq -r '.deliverables[0].name' certification/candidate.json)"
-
-          docker_run="$(gh api --method GET \
-            "repos/${GITHUB_REPOSITORY}/actions/workflows/docker.yml/runs" \
-            -f branch="$RAW_TAG" -f status=success -F per_page=1 \
-            --jq '.workflow_runs[0].id // empty' || true)"
-          if [[ -z "$docker_run" ]] || ! gh run download "$docker_run" -n image-promotion-record -D image-record; then
-            echo "No image promotion record for ${RAW_TAG}; a chart may not wrap an unverified image" >&2
-            exit 1
-          fi
-          candidate_sha="$(python3 -c "import json,hashlib,sys; sys.path.insert(0,'scripts/release'); import release_gate; print(release_gate.canonical_sha256(json.load(open('certification/candidate.json'))))")"
-          record_sha="$(jq -r '.candidate_sha256 // empty' image-record/image-promotion-record.json)"
-          if [[ "$record_sha" != "$candidate_sha" ]]; then
-            echo "Image promotion record does not belong to this certification" >&2
-            exit 1
-          fi
-          overlap="$(jq -r --slurpfile candidate certification/candidate.json \
-            '[.binary_sha256s[]] - ($candidate[0].deliverables | map(.binary_sha256)) | length' \
-            image-record/image-promotion-record.json)"
-          if [[ "$overlap" -ne 0 ]]; then
-            echo "Image promotion record carries binary digests outside the certification" >&2
-            exit 1
-          fi
-
-  build-helm-package:
-    runs-on: ubuntu-latest
-    timeout-minutes: 30
-    needs: [ release-certification ]
-    if: |
-      (github.event_name == 'workflow_dispatch' && !contains(github.event.inputs.version, '-preview')) ||
-      (
-        github.event.workflow_run.conclusion == 'success' &&
-        github.event.workflow_run.event == 'push' &&
-        contains(github.event.workflow_run.head_branch, '.') &&
-        !contains(github.event.workflow_run.head_branch, '-preview')
-      )
-
-    outputs:
-      raw_tag: ${{ steps.version.outputs.raw_tag }}
-      app_version: ${{ steps.version.outputs.app_version }}
-      chart_version: ${{ steps.version.outputs.chart_version }}
-
-    steps:
-      - name: Checkout release source
-        uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
-        with:
-          ref: ${{ github.event.workflow_run.head_sha || (startsWith(inputs.version, 'refs/tags/') && inputs.version || format('refs/tags/{0}', inputs.version)) }}
-          persist-credentials: false
-
-      # Both inputs reach the shell through env rather than `${{ }}`
-      # interpolation. A git ref name may contain `$(...)` — anything without a
-      # space is a legal tag — and interpolation pastes it into the script
-      # verbatim, where bash would run it. Reading "$RAW_INPUT" instead makes it
-      # data.
-      - name: Normalize release version
-        id: version
-        env:
-          RAW_INPUT: ${{ github.event.inputs.version }}
-          RAW_BRANCH: ${{ github.event.workflow_run.head_branch }}
-        run: |
-          set -eux
-
-          if [ "${{ github.event_name }}" = "workflow_dispatch" ]; then
-            RAW="$RAW_INPUT"
-          else
-            RAW="$RAW_BRANCH"
-          fi
-
-          case "$RAW" in
-            refs/tags/*)
-              RAW_TAG="${RAW#refs/tags/}"
-              ;;
-            *)
-              RAW_TAG="$RAW"
-              ;;
-          esac
-
-          ./scripts/helm_chart_version.sh "$RAW_TAG"
-
-      # Docker and Helm share the binary-build trigger. Do not publish a chart
-      # until the image selected by its default appVersion is pullable.
-      - name: Wait for release image
-        timeout-minutes: 20
-        env:
-          APP_VERSION: ${{ steps.version.outputs.app_version }}
-        run: |
-          set -euo pipefail
-          if [[ ! "$APP_VERSION" =~ ^[0-9]+\.[0-9]+\.[0-9]+(-[0-9A-Za-z.-]+)?$ ]] ||
-             [[ "$APP_VERSION" == *-preview* ]]; then
-            echo "Invalid deliverable image version: $APP_VERSION" >&2
-            exit 1
-          fi
-
-          for attempt in {1..60}; do
-            if token=$(curl --fail --silent --show-error --connect-timeout 10 --max-time 20 \
-              'https://auth.docker.io/token?service=registry.docker.io&scope=repository:rustfs/rustfs:pull' \
-              | jq -er '.token // empty') &&
-              curl --fail --silent --show-error --head --connect-timeout 10 --max-time 20 \
-                --header "Authorization: Bearer $token" \
-                --header 'Accept: application/vnd.oci.image.index.v1+json, application/vnd.docker.distribution.manifest.list.v2+json' \
-                "https://registry-1.docker.io/v2/rustfs/rustfs/manifests/$APP_VERSION" > /dev/null; then
-              echo "Release image is available: rustfs/rustfs:$APP_VERSION"
-              exit 0
-            fi
-            echo "Waiting for rustfs/rustfs:$APP_VERSION ($attempt/60)"
-            if (( attempt < 60 )); then
-              sleep 15
-            fi
-          done
-
-          echo "Release image is unavailable; refusing to publish Helm chart" >&2
-          exit 1
-
-      - name: Replace chart version and app version
-        env:
-          CHART_VERSION: ${{ steps.version.outputs.chart_version }}
-          APP_VERSION: ${{ steps.version.outputs.app_version }}
-        run: |
-          set -eux
-          sed -i -E "s/^version:.*/version: \"${CHART_VERSION}\"/" helm/rustfs/Chart.yaml
-          sed -i -E "s/^appVersion:.*/appVersion: \"${APP_VERSION}\"/" helm/rustfs/Chart.yaml
-
-      - name: Set up Helm
-        uses: azure/setup-helm@b9e51907a09c216f16ebe8536097933489208112 # v4.3.0
-
-      - name: Install yq
+      - uses: azure/setup-helm@b9e51907a09c216f16ebe8536097933489208112 # v4.3.0
+      - name: Install pinned yq
        run: |
          set -euo pipefail
          yq_version="4.54.1"
@@ -238,66 +35,43 @@ jobs:
          echo "$tools_dir" >> "$GITHUB_PATH"
          "$tools_dir/yq" --version

-      - name: Test Helm Chart Templates
-        run: ./scripts/test_helm_templates.sh
-
-      - name: Package Helm Chart
-        run: |
-          set -eux
-          cp helm/README.md helm/rustfs/
-          helm package ./helm/rustfs \
-            --destination helm/rustfs/ \
-            --version "${{ steps.version.outputs.chart_version }}"
-
-      - name: Upload helm package as artifact
-        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
-        with:
-          name: helm-package
-          path: |
-            helm/rustfs/*.tgz
-            helm/rustfs/README.md
-          retention-days: 1
-
-  publish-helm-package:
-    runs-on: ubuntu-latest
-    timeout-minutes: 30
-    needs: [ build-helm-package ]
-    if: needs.build-helm-package.result == 'success'
-
-    steps:
-      - name: Checkout helm package repo
-        uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
-        with:
-          # persist-credentials-exempt: this checkout's token IS the push credential —
-          # the job git-pushes to rustfs/helm below. Clearing it breaks chart publishing.
-          repository: rustfs/helm
-          token: ${{ secrets.RUSTFS_HELM_PACKAGE }}
-
-      - name: Download helm package
-        uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
-        with:
-          name: helm-package
-          path: ./
-
-      - name: Set up helm
-        uses: azure/setup-helm@b9e51907a09c216f16ebe8536097933489208112 # v4.3.0
-
-      - name: Generate index
-        run: helm repo index . --url https://charts.rustfs.com
-
-      # app_version is derived from the triggering tag name, and this job holds
-      # the cross-repository push token with rustfs/helm already checked out —
-      # the worst place in the repo to paste an attacker-influenced string into
-      # a shell line. Passed through env so bash treats it as data.
-      - name: Push helm package and index file
+      - name: Test and package the exact application version
+        id: chart
        env:
-          GIT_USERNAME: ${{ secrets.USERNAME }}
-          GIT_EMAIL: ${{ secrets.EMAIL_ADDRESS }}
-          APP_VERSION: ${{ needs.build-helm-package.outputs.app_version }}
+          GH_TOKEN: ${{ github.token }}
+          CANDIDATE_MANIFEST: ${{ inputs.candidate_manifest }}
        run: |
-          set -eux
-          git config --global user.name "${GIT_USERNAME}"
-          git config --global user.email "${GIT_EMAIL}"
-          git add .
-          git commit -m "Update rustfs helm package with ${APP_VERSION}." || echo "No changes to commit"
-          git push origin main
+          python3 - <<'PYCHART'
+          import sys, subprocess, shutil
+          from pathlib import Path
+          sys.path.insert(0, 'scripts/release')
+          import candidate as c, pipeline
+          inputs = pipeline.manifest_env()
+          current = pipeline.current_run('release-complete.yml')
+          if any(a['name'] == 'helm-package' for a in c.artifact_list(current['id'])):
+              c.download_artifact(current, 'helm-package', 'chart')
+              record = c.read_json('chart/chart.json')
+              c.require(record['inputs_sha256'] == c.digest(inputs), 'chart belongs to another candidate')
+              c.emit_output('reused', 'true')
+          else:
+              chart = Path('helm/rustfs/Chart.yaml')
+              import re
+              value = chart.read_text()
+              for key in ('version', 'appVersion'):
+                  value, count = re.subn(rf'(?m)^{key}:.*$', f'{key}: "{inputs["version"]}"', value)
+                  c.require(count == 1, 'ambiguous chart metadata')
+              chart.write_text(value)
+              subprocess.run(['bash', 'scripts/test_helm_templates.sh'], check=True)
+              shutil.copyfile('helm/README.md', 'helm/rustfs/README.md')
+              subprocess.run(['helm', 'package', 'helm/rustfs', '--destination', 'chart'], check=True)
+              path = Path('chart') / f"rustfs-{inputs['version']}.tgz"
+              c.write_json('chart/chart.json', {'version': inputs['version'], 'inputs_sha256': c.digest(inputs),
+                                               'name': path.name, 'sha256': c.file_digest(path), 'size': path.stat().st_size})
+          PYCHART
+      - uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+        if: steps.chart.outputs.reused != 'true'
+        with:
+          name: helm-package
+          path: chart/*
+          if-no-files-found: error
+          retention-days: 30
@@ -0,0 +1,415 @@
+# Copyright 2026 RustFS Team
+# SPDX-License-Identifier: Apache-2.0
+
+# Manual packaging of the current origin/main tip. Workflow artifacts only.
+# Dispatch uses this file from the selected branch. The build fetches
+# refs/heads/main and checks out that commit, not the dispatch ref or a tag.
+
+name: Manual main packages
+
+on:
+  workflow_dispatch:
+    inputs:
+      architecture:
+        description: "Linux targets: x86 (x86_64 musl+gnu), arm (aarch64 musl+gnu), or all"
+        required: true
+        type: choice
+        options:
+          - x86
+          - arm
+          - all
+        default: all
+      version:
+        description: "Binary and package version: MAJOR.MINOR.PATCH, or MAJOR.MINOR.PATCH-(alpha|beta|rc).N. No leading v."
+        required: true
+        type: string
+
+permissions:
+  contents: read
+
+env:
+  CARGO_TERM_COLOR: always
+  RUST_BACKTRACE: 1
+  CARGO_INCREMENTAL: 0
+
+jobs:
+  prepare:
+    name: Resolve main tip and version
+    if: github.repository == 'rustfs/rustfs'
+    runs-on: ubuntu-latest
+    timeout-minutes: 15
+    outputs:
+      source_sha: ${{ steps.pin.outputs.source_sha }}
+      version: ${{ steps.version.outputs.version }}
+      toolchain: ${{ steps.inputs.outputs.toolchain }}
+      console: ${{ steps.inputs.outputs.console }}
+      matrix: ${{ steps.matrix.outputs.matrix }}
+    steps:
+      - name: Checkout main
+        uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
+        with:
+          ref: main
+          fetch-depth: 0
+          persist-credentials: false
+
+      - name: Pin the tip of origin/main
+        id: pin
+        shell: bash
+        run: |
+          set -euo pipefail
+          GIT_TERMINAL_PROMPT=0 git fetch --no-tags origin refs/heads/main
+          sha=$(git rev-parse FETCH_HEAD)
+          git checkout --detach "$sha"
+          test "$(git rev-parse HEAD)" = "$sha"
+          printf 'source_sha=%s\n' "$sha" >> "$GITHUB_OUTPUT"
+          printf 'Pinned origin/main %s\n' "$sha"
+
+      - name: Validate package version
+        id: version
+        shell: bash
+        env:
+          PACKAGE_VERSION: ${{ inputs.version }}
+        run: |
+          set -euo pipefail
+          bash scripts/release/package_manual_artifacts.sh validate \
+            --version "$PACKAGE_VERSION" \
+            --arch x86_64 \
+            --libc gnu | tee "$RUNNER_TEMP/package-version.txt"
+          printf 'version=%s\n' "$PACKAGE_VERSION" >> "$GITHUB_OUTPUT"
+
+      - name: Install stable Rust
+        uses: dtolnay/rust-toolchain@29eef336d9b2848a0b548edc03f92a220660cdb8 # stable
+        with:
+          toolchain: stable
+
+      - name: Resolve console assets and toolchain
+        id: inputs
+        shell: bash
+        env:
+          GH_TOKEN: ${{ github.token }}
+        run: |
+          set -euo pipefail
+          toolchain=$(rustc --version | awk '{print $2}')
+          [[ "$toolchain" =~ ^[0-9]+\.[0-9]+\.[0-9]+$ ]]
+          gh api repos/rustfs/console/releases/latest > console-release.json
+          TOOLCHAIN="$toolchain" python3 - <<'PY'
+          import json, os, sys
+          sys.path.insert(0, "scripts/release")
+          import candidate
+          console = candidate.console_asset(candidate.read_json("console-release.json"))
+          candidate.emit_output("console", json.dumps(console, separators=(",", ":")))
+          candidate.emit_output("toolchain", os.environ["TOOLCHAIN"])
+          print(f"toolchain={os.environ['TOOLCHAIN']}")
+          print(f"console={console['tag']} {console['name']}")
+          PY
+
+      - name: Select Linux targets
+        id: matrix
+        shell: bash
+        env:
+          ARCHITECTURE: ${{ inputs.architecture }}
+        run: |
+          set -euo pipefail
+          matrix=$(bash scripts/release/package_manual_artifacts.sh matrix "$ARCHITECTURE")
+          count=$(python3 -c 'import json,sys; print(len(json.loads(sys.argv[1])["include"]))' "$matrix")
+          [[ "$count" -gt 0 ]]
+          printf 'matrix=%s\n' "$matrix" >> "$GITHUB_OUTPUT"
+          printf 'Selected %s target(s) for %s\n' "$count" "$ARCHITECTURE"
+
+  package:
+    name: Package RustFS (${{ matrix.target_id }})
+    needs: prepare
+    if: github.repository == 'rustfs/rustfs'
+    # x86_64 uses sm-standard-4, the official release compile pool: the workspace
+    # release build needs that disk (placement rule: disk) and reads build.yml's
+    # cache without writing it. aarch64 uses ubicloud-standard-16-arm so the
+    # packaged binary and console smoke execute natively (placement rule: hardware).
+    runs-on: ${{ matrix.os }}
+    timeout-minutes: 210
+    env:
+      FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"
+      RUSTUP_TOOLCHAIN: ${{ needs.prepare.outputs.toolchain }}
+      RUSTFLAGS: "${{ matrix.rustflags }}"
+    strategy:
+      fail-fast: false
+      matrix: ${{ fromJson(needs.prepare.outputs.matrix) }}
+    steps:
+      - name: Checkout the pinned origin/main commit
+        uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
+        with:
+          ref: ${{ needs.prepare.outputs.source_sha }}
+          fetch-depth: 0
+          persist-credentials: false
+
+      - name: Confirm the checkout is that main commit
+        shell: bash
+        env:
+          SOURCE_SHA: ${{ needs.prepare.outputs.source_sha }}
+        run: |
+          set -euo pipefail
+          test "$(git rev-parse HEAD)" = "$SOURCE_SHA"
+          GIT_TERMINAL_PROMPT=0 git fetch --no-tags origin refs/heads/main
+          git merge-base --is-ancestor "$SOURCE_SHA" FETCH_HEAD
+
+      - name: Setup Rust environment
+        uses: ./.github/actions/setup
+        with:
+          rust-version: ${{ needs.prepare.outputs.toolchain }}
+          target: ${{ matrix.target }}
+          cache-shared-key: build-${{ matrix.target }}${{ matrix.glibc && format('-glibc-{0}', matrix.glibc) || '' }}${{ matrix.os == 'ubicloud-standard-16-arm' && '-ubicloud-arm' || '' }}
+          # build.yml owns this key. This job only restores it.
+          cache-save-if: 'false'
+          install-cross-tools: ${{ matrix.cross }}
+          install-build-packaging-tools: 'true'
+          install-test-tools: 'false'
+
+      - name: Download static console assets
+        shell: bash
+        env:
+          GH_TOKEN: ${{ github.token }}
+          CONSOLE_IDENTITY: ${{ needs.prepare.outputs.console }}
+        run: |
+          set -euo pipefail
+          python3 - <<'PYCONSOLE'
+          import json, os, pathlib, subprocess, zipfile
+          value = json.loads(os.environ["CONSOLE_IDENTITY"])
+          subprocess.run(["python3", ".github/actions/setup/download_release_asset.py",
+              "--repository", "rustfs/console", "--tag", value["tag"], "--asset", value["name"],
+              "--sha256", value["sha256"], "--output", "console.zip"], check=True)
+          destination = pathlib.Path("rustfs/static")
+          destination.mkdir(parents=True, exist_ok=True)
+          with zipfile.ZipFile("console.zip") as archive:
+              for member in archive.infolist():
+                  path = pathlib.PurePosixPath(member.filename)
+                  if path.is_absolute() or ".." in path.parts or "\\" in member.filename:
+                      raise SystemExit("unsafe Console archive member")
+              archive.extractall(destination)
+          if not (destination / "index.html").is_file():
+              raise SystemExit("Console archive has no index.html")
+          PYCONSOLE
+          rm -f console.zip
+
+      - name: Build RustFS
+        shell: bash
+        env:
+          RUSTFS_BUILD_VERSION: ${{ needs.prepare.outputs.version }}
+          BUILD_TARGET: ${{ matrix.target }}
+          BUILD_OS: ${{ matrix.build_os }}
+          BUILD_FEATURES: ${{ matrix.features }}
+          BUILD_GLIBC: ${{ matrix.glibc }}
+          BUILD_CROSS: ${{ matrix.cross }}
+        run: |
+          set -euo pipefail
+          touch rustfs/build.rs
+
+          feature_args=()
+          if [[ -n "$BUILD_FEATURES" ]]; then
+            feature_args=(--features "$BUILD_FEATURES")
+          fi
+
+          # gnu targets pin glibc 2.31 through cargo-zigbuild (<target>.<glibc>).
+          target_arg="$BUILD_TARGET"
+          if [[ -n "$BUILD_GLIBC" ]]; then
+            target_arg="${target_arg}.${BUILD_GLIBC}"
+          fi
+
+          profile_args=()
+          if [[ "$BUILD_TARGET" == *-linux-gnu && "$BUILD_FEATURES" == pyroscope ]]; then
+            profile_args=(--config 'profile.release.strip="none"')
+          fi
+
+          case "$BUILD_CROSS" in
+            true)
+              cargo zigbuild --release --locked --target "$target_arg" "${feature_args[@]}" "${profile_args[@]}" -p rustfs --bin rustfs
+              ;;
+            false)
+              cargo build --release --locked --target "$target_arg" "${feature_args[@]}" "${profile_args[@]}" -p rustfs --bin rustfs
+              ;;
+            *)
+              printf 'unexpected cross flag: %q\n' "$BUILD_CROSS" >&2
+              exit 1
+              ;;
+          esac
+
+          bin="target/${BUILD_TARGET}/release/rustfs"
+          version_output=$("$bin" --version)
+          first=${version_output%%$'\n'*}
+          if [[ "$first" != "rustfs ${RUSTFS_BUILD_VERSION}" ]]; then
+            printf 'binary version line is %q, expected %q\n' "$first" "rustfs ${RUSTFS_BUILD_VERSION}" >&2
+            printf '%s\n' "$version_output" >&2
+            exit 1
+          fi
+          if ! grep -Fxq "build target : ${BUILD_TARGET}" <<< "$version_output" || \
+             ! grep -Fxq "build os     : ${BUILD_OS}" <<< "$version_output" || \
+             ! grep -Fxq "rust channel : stable-${BUILD_TARGET}" <<< "$version_output"; then
+            printf 'release binary reports an unexpected compile target\n' >&2
+            printf '%s\n' "$version_output" >&2
+            exit 1
+          fi
+          if [[ -n "$BUILD_GLIBC" ]]; then
+            python3 scripts/release/check_glibc.py "$BUILD_GLIBC" "$bin"
+          fi
+
+      - name: Generate the CPU symbol catalog
+        if: endsWith(matrix.target, '-linux-gnu') && matrix.features == 'pyroscope'
+        shell: bash
+        env:
+          BUILD_TARGET: ${{ matrix.target }}
+        run: |
+          set -euo pipefail
+          cargo metadata --locked --format-version 1 --filter-platform "$BUILD_TARGET" \
+            --features rustfs/pyroscope > "$RUNNER_TEMP/cpu-catalog-metadata.json"
+          features=$(jq -ce '
+            (.packages[] | select(.name == "rustfs") | .id) as $id |
+            .resolve.nodes[] | select(.id == $id) | .features | map(select(. != "default")) | sort
+          ' "$RUNNER_TEMP/cpu-catalog-metadata.json")
+          release_dir="target/${BUILD_TARGET}/release"
+          rm -f "$release_dir/rustfs.cpu-symbol-catalog.json" "$release_dir/rustfs.cpu-symbol-catalog.sha256"
+          cargo run --locked -p rustfs-connect-symbol-catalog -- \
+            "$release_dir/rustfs" "$(git rev-parse HEAD)" "$BUILD_TARGET" "$features" \
+            "$release_dir/rustfs.cpu-symbol-catalog.json" > "$release_dir/rustfs.cpu-symbol-catalog.sha256"
+
+      - name: Install DEB and RPM tools
+        shell: bash
+        run: |
+          set -euo pipefail
+          sudo apt-get update
+          sudo apt-get install -y ruby ruby-dev build-essential rpm fakeroot cpio
+          sudo gem install fpm --version 1.17.0 --no-document
+
+      - name: Package zip, deb, and rpm
+        shell: bash
+        env:
+          PACKAGE_VERSION: ${{ needs.prepare.outputs.version }}
+          ARCH: ${{ matrix.arch }}
+          LIBC: ${{ matrix.libc }}
+          BUILD_TARGET: ${{ matrix.target }}
+          BUILD_FEATURES: ${{ matrix.features }}
+          SOURCE_SHA: ${{ needs.prepare.outputs.source_sha }}
+        run: |
+          set -euo pipefail
+          binary="target/${BUILD_TARGET}/release/rustfs"
+          extra=()
+          if [[ "$BUILD_TARGET" == *-linux-gnu && "$BUILD_FEATURES" == pyroscope ]]; then
+            extra=(
+              --zip-extra "target/${BUILD_TARGET}/release/rustfs.cpu-symbol-catalog.json"
+              --zip-extra "target/${BUILD_TARGET}/release/rustfs.cpu-symbol-catalog.sha256"
+            )
+          fi
+          bash scripts/release/package_manual_artifacts.sh package \
+            --version "$PACKAGE_VERSION" \
+            --arch "$ARCH" \
+            --libc "$LIBC" \
+            --binary "$binary" \
+            --output dist \
+            "${extra[@]}"
+          printf 'Packaged %s from origin/main %s\n' "$PACKAGE_VERSION" "$SOURCE_SHA"
+
+      - name: Verify the packaged binary
+        shell: bash
+        env:
+          PACKAGE_VERSION: ${{ needs.prepare.outputs.version }}
+          ARCH: ${{ matrix.arch }}
+          LIBC: ${{ matrix.libc }}
+          BUILD_TARGET: ${{ matrix.target }}
+          BUILD_OS: ${{ matrix.build_os }}
+          BUILD_FEATURES: ${{ matrix.features }}
+        run: |
+          set -euo pipefail
+          stem="rustfs-linux-${ARCH}-${LIBC}-v${PACKAGE_VERSION}"
+          package_file="dist/${stem}.zip"
+          [[ -f "$package_file" && -f "dist/${stem}.deb" && -f "dist/${stem}.rpm" ]]
+          [[ "$(find dist -maxdepth 1 -type f | wc -l)" -eq 3 ]]
+          archive_members=$(unzip -Z1 "$package_file" | sort)
+          if [[ "$BUILD_TARGET" == *-linux-gnu && "$BUILD_FEATURES" == pyroscope ]]; then
+            expected_members=$(printf '%s\n' rustfs rustfs.cpu-symbol-catalog.json rustfs.cpu-symbol-catalog.sha256 | sort)
+          else
+            expected_members=$(printf '%s\n' rustfs)
+          fi
+          if [[ "$archive_members" != "$expected_members" ]]; then
+            printf 'Linux package does not contain the expected files\n' >&2
+            printf '%s\n' "$archive_members" >&2
+            exit 1
+          fi
+          binary_dir=$(mktemp -d)
+          trap 'rm -rf "$binary_dir"' EXIT
+          unzip -q "$package_file" rustfs -d "$binary_dir"
+          version_output=$("$binary_dir/rustfs" --version)
+          first=${version_output%%$'\n'*}
+          if [[ "$first" != "rustfs ${PACKAGE_VERSION}" ]]; then
+            printf 'packaged binary version line is %q, expected %q\n' "$first" "rustfs ${PACKAGE_VERSION}" >&2
+            exit 1
+          fi
+          if ! grep -Fxq "build target : ${BUILD_TARGET}" <<< "$version_output" || \
+             ! grep -Fxq "build os     : ${BUILD_OS}" <<< "$version_output"; then
+            printf 'packaged binary reports an unexpected compile target\n' >&2
+            exit 1
+          fi
+
+      - name: Verify packaged console
+        if: matrix.target == 'x86_64-unknown-linux-gnu' || matrix.target == 'aarch64-unknown-linux-gnu'
+        shell: bash
+        env:
+          PACKAGE_VERSION: ${{ needs.prepare.outputs.version }}
+          ARCH: ${{ matrix.arch }}
+          LIBC: ${{ matrix.libc }}
+        run: |
+          set -euo pipefail
+          package_file="dist/rustfs-linux-${ARCH}-${LIBC}-v${PACKAGE_VERSION}.zip"
+
+          find_free_port() {
+            python3 - <<'PY'
+          import socket
+
+          with socket.socket() as sock:
+              sock.bind(("127.0.0.1", 0))
+              print(sock.getsockname()[1])
+          PY
+          }
+
+          package_dir=$(mktemp -d)
+          data_dir=$(mktemp -d)
+          log_file="${RUNNER_TEMP}/rustfs-console-smoke.log"
+          server_pid=""
+          trap '
+            if [[ -n "${server_pid:-}" ]]; then
+              kill "$server_pid" 2>/dev/null || true
+              wait "$server_pid" 2>/dev/null || true
+            fi
+            rm -rf "$package_dir" "$data_dir"
+          ' EXIT
+
+          unzip -q "$package_file" -d "$package_dir"
+          api_port=$(find_free_port)
+          console_port=$(find_free_port)
+          console_url="http://127.0.0.1:${console_port}/rustfs/console/"
+
+          "$package_dir/rustfs" server \
+            --address "127.0.0.1:${api_port}" \
+            --console-enable \
+            --console-address "127.0.0.1:${console_port}" \
+            --access-key console-smoke \
+            --secret-key console-smoke-secret \
+            "$data_dir" >"$log_file" 2>&1 &
+          server_pid=$!
+
+          for _ in {1..40}; do
+            response=$(curl --silent --output /dev/null --write-out '%{http_code} %{content_type}' "$console_url" || true)
+            if [[ "$response" == 200\ text/html* ]]; then
+              exit 0
+            fi
+            sleep 0.25
+          done
+
+          printf 'Console endpoint did not return 200 text/html: %s\n' "$response" >&2
+          cat "$log_file"
+          exit 1
+
+      - name: Upload packages
+        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+        with:
+          name: packages-${{ matrix.target_id }}-v${{ needs.prepare.outputs.version }}
+          path: dist/*
+          if-no-files-found: error
+          compression-level: 0
+          retention-days: 30
@@ -112,9 +112,11 @@ jobs:
    # GitHub-hosted: reliably provides a running Docker daemon + buildx +
    # python3. See the header note (ci-2) for why the self-hosted sm-standard-4
    # label was abandoned (no docker.sock on the pod its only run landed on).
+    # sm-standard-2 has no docker.sock either (weekly runs failed at "Enable
+    # buildx" from 2026-09-27 to 2026-11-08), so keep this job on ubuntu-latest.
    # TODO(ci-8): scheduled-failure alerting (auto-open issue on a red weekly
    # run) is added by the ci-8 composite action; do not implement it here.
-    runs-on: sm-standard-2
+    runs-on: ubuntu-latest
    timeout-minutes: 120
    steps:
      - uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
@@ -47,8 +47,8 @@ permissions:
  contents: read

 concurrency:
-  group: oidc-keycloak-live-${{ github.ref }}
-  cancel-in-progress: true
+  group: oidc-keycloak-live-${{ github.event_name }}-${{ github.ref }}
+  cancel-in-progress: ${{ github.event_name != 'schedule' }}

 jobs:
  oidc-keycloak-live:
@@ -1,840 +1,53 @@
-# Copyright 2024 RustFS Team
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-# Package Workflow - Build DEB/RPM packages
-#
-# This workflow builds DEB and RPM packages from pre-built Linux binaries
-# and uploads them to Cloudflare R2 and the GitHub release.
-#
-# Trigger:
-# - workflow_run: automatically package after "Build and Release" completes
-#   for a release tag (the mac/windows/linux binaries are already uploaded
-#   to the GitHub release before packaging starts)
-# - workflow_dispatch: manual fallback with a release tag and/or exact build run ID
-#
-# Flow:
-# 1. Resolve and validate the selected Build workflow run and source identity
-# 2. Download Linux binaries (x86_64-gnu, aarch64-gnu, x86_64-musl,
-#    aarch64-musl) from build artifacts
-# 3. Build DEB packages for amd64 and arm64, gnu and musl variants
-# 4. Build RPM packages for x86_64 and aarch64, gnu and musl variants
-# 5. Upload all packages to Cloudflare R2 and the GitHub release
-
-name: Package DEB/RPM
-
-permissions:
-  # contents: write is required to upload packages to the GitHub release
-  contents: write
-  actions: read
-
+# Copyright 2026 RustFS Team
+# SPDX-License-Identifier: Apache-2.0
+name: Build candidate packages
 on:
-  # Follows the same pattern as docker.yml: run after the release build
-  # workflow completes, so packaging is triggered only by release tags
-  # (e.g. 1.0.0-rc.2, 1.0.0-rc.3), never by development builds.
-  workflow_run:
-    workflows: [ "Build and Release" ]
-    types: [ completed ]
-  workflow_dispatch:
+  workflow_call:
    inputs:
-      tag:
-        description: "Release tag to package (e.g. 1.0.0-beta.12). Leave empty for latest main build."
-        required: false
+      candidate_manifest:
        type: string
-      build_run_id:
-        description: "Build workflow run ID (when combined with tag, both must identify the same release commit)"
-        required: false
-        type: string
-
-concurrency:
-  group: ${{ github.workflow }}-${{ github.event.workflow_run.head_branch || github.event.inputs.tag || github.run_id }}
-  cancel-in-progress: true
-
-env:
-  HEAD_BRANCH: ${{ github.event.workflow_run.head_branch }}
-  WORKFLOW_RUN_ID: ${{ github.event.workflow_run.id }}
-
+        required: true
+permissions:
+  contents: read
+  actions: read
 jobs:
-  # Resolve which build run to use and extract version info
-  resolve:
-    name: Resolve Build
-    # Auto-trigger only from successful tag builds of "Build and Release".
-    # Tag pushes arrive as event == push with head_branch != main (a
-    # non-main push head_branch is the release tag name). Manual dispatch
-    # stays available as a fallback for backfills and re-runs.
-    if: >-
-      github.event_name == 'workflow_dispatch' ||
-      (github.event.workflow_run.conclusion == 'success' &&
-       github.event.workflow_run.event == 'push' &&
-       github.event.workflow_run.head_branch != 'main')
-    runs-on: ubuntu-latest # gh CLI required; the sm-standard-2 fleet does not ship it
-    timeout-minutes: 10
-    outputs:
-      version: ${{ steps.resolve.outputs.version }}
-      build_type: ${{ steps.resolve.outputs.build_type }}
-      build_run_id: ${{ steps.resolve.outputs.build_run_id }}
-      build_run_number: ${{ steps.resolve.outputs.build_run_number }}
-      head_sha: ${{ steps.resolve.outputs.head_sha }}
-      dev_sequence: ${{ steps.resolve.outputs.dev_sequence }}
-      tag: ${{ steps.resolve.outputs.tag }}
-    steps:
-      - name: Resolve build run
-        id: resolve
-        shell: bash
-        env:
-          GH_TOKEN: ${{ github.token }}
-          EVENT_NAME: ${{ github.event_name }}
-          REPOSITORY: ${{ github.repository }}
-          INPUT_TAG: ${{ github.event.inputs.tag }}
-          INPUT_RUN_ID: ${{ github.event.inputs.build_run_id }}
-        run: |
-          set -euo pipefail
-
-          fail() {
-            echo "❌ $1" >&2
-            exit 1
-          }
-
-          TAG=""
-          BUILD_RUN_ID=""
-          case "$EVENT_NAME" in
-            workflow_run)
-              TAG="$HEAD_BRANCH"
-              BUILD_RUN_ID="$WORKFLOW_RUN_ID"
-              ;;
-            workflow_dispatch)
-              TAG="$INPUT_TAG"
-              BUILD_RUN_ID="$INPUT_RUN_ID"
-              ;;
-            *) fail "unsupported event: $EVENT_NAME" ;;
-          esac
-
-          # Validate and classify tags before using them in API paths or logs.
-          semver_core='(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)'
-          prerelease_id='(alpha|beta|rc)\.(0|[1-9][0-9]*)'
-          if [[ -n "$TAG" ]]; then
-            if [[ "$TAG" =~ ^${semver_core}(-${prerelease_id})?-preview\.(0|[1-9][0-9]*)$ ]]; then
-              BUILD_TYPE=preview
-            elif [[ "$TAG" =~ ^${semver_core}-${prerelease_id}$ ]]; then
-              BUILD_TYPE=prerelease
-            elif [[ "$TAG" =~ ^${semver_core}$ ]]; then
-              BUILD_TYPE=release
-            else
-              fail "tag is not a supported strict package version"
-            fi
-          else
-            BUILD_TYPE=development
-          fi
-
-          if [[ -n "$BUILD_RUN_ID" ]]; then
-            [[ "$BUILD_RUN_ID" =~ ^[1-9][0-9]*$ ]] || fail "build run ID must be a positive decimal integer"
-            echo "Using selected build run: $BUILD_RUN_ID"
-          elif [[ -n "$TAG" ]]; then
-            echo "Looking for build run for tag: $TAG"
-            BUILD_RUN_ID=$(gh api --method GET \
-              "repos/${REPOSITORY}/actions/workflows/build.yml/runs" \
-              -f branch="$TAG" -f status=success -F per_page=1 \
-              --jq '.workflow_runs[0].id // empty' 2>/dev/null || true)
-
-            if [[ -z "$BUILD_RUN_ID" ]]; then
-              BUILD_RUN_ID=$(gh api --method GET \
-                "repos/${REPOSITORY}/actions/workflows/build.yml/runs" \
-                -f event=push -f status=success -F per_page=100 2>/dev/null |
-                jq -r --arg tag "$TAG" \
-                  '[.workflow_runs[] | select(.head_branch == $tag)][0].id // empty' || true)
-            fi
-            [[ "$BUILD_RUN_ID" =~ ^[1-9][0-9]*$ ]] || fail "no successful build run found for tag"
-            echo "Found build run: $BUILD_RUN_ID"
-          else
-            echo "No tag specified, looking for latest main build"
-            BUILD_RUN_ID=$(gh api --method GET \
-              "repos/${REPOSITORY}/actions/workflows/build.yml/runs" \
-              -f branch=main -f status=success -F per_page=1 \
-              --jq '.workflow_runs[0].id // empty' 2>/dev/null || true)
-            [[ "$BUILD_RUN_ID" =~ ^[1-9][0-9]*$ ]] || fail "no successful main build found"
-            echo "Latest main build: $BUILD_RUN_ID"
-          fi
-
-          # Fetch once and use the same immutable run metadata for identity,
-          # ordering, workflow provenance, and release-channel validation.
-          RUN_JSON=$(gh api "repos/${REPOSITORY}/actions/runs/${BUILD_RUN_ID}") ||
-            fail "cannot read selected build run"
-          RUN_ID=$(jq -r '.id // empty' <<<"$RUN_JSON")
-          RUN_NUMBER=$(jq -r '.run_number // empty' <<<"$RUN_JSON")
-          RUN_STATUS=$(jq -r '.status // empty' <<<"$RUN_JSON")
-          RUN_CONCLUSION=$(jq -r '.conclusion // empty' <<<"$RUN_JSON")
-          RUN_PATH=$(jq -r '.path // empty' <<<"$RUN_JSON")
-          HEAD_SHA=$(jq -r '.head_sha // empty' <<<"$RUN_JSON")
-          RUN_HEAD_BRANCH=$(jq -r '.head_branch // empty' <<<"$RUN_JSON")
-
-          [[ "$RUN_ID" == "$BUILD_RUN_ID" ]] || fail "run metadata ID mismatch"
-          [[ "$RUN_NUMBER" =~ ^[1-9][0-9]*$ ]] || fail "build run number must be a positive decimal integer"
-          [[ "$RUN_STATUS" == completed && "$RUN_CONCLUSION" == success ]] || fail "selected build run is not successful"
-          [[ "$RUN_PATH" == .github/workflows/build.yml ]] || fail "selected run is not Build and Release"
-          [[ "$HEAD_SHA" =~ ^[0-9a-f]{40}$ ]] || fail "selected build run has an invalid head SHA"
-          [[ "$RUN_HEAD_BRANCH" != *$'\n'* && -n "$RUN_HEAD_BRANCH" ]] || fail "selected build run has an invalid head branch"
-
-          if [[ -n "$TAG" ]]; then
-            [[ "$RUN_HEAD_BRANCH" == "$TAG" ]] || fail "tag and build run head branch do not match"
-
-            TAG_REF_JSON=$(gh api "repos/${REPOSITORY}/git/ref/tags/${TAG}") ||
-              fail "cannot resolve release tag ref"
-            TAG_OBJECT_TYPE=$(jq -r '.object.type // empty' <<<"$TAG_REF_JSON")
-            TAG_OBJECT_SHA=$(jq -r '.object.sha // empty' <<<"$TAG_REF_JSON")
-            depth=0
-            while [[ "$TAG_OBJECT_TYPE" == tag && $depth -lt 5 ]]; do
-              TAG_OBJECT_JSON=$(gh api "repos/${REPOSITORY}/git/tags/${TAG_OBJECT_SHA}") ||
-                fail "cannot peel annotated release tag"
-              TAG_OBJECT_TYPE=$(jq -r '.object.type // empty' <<<"$TAG_OBJECT_JSON")
-              TAG_OBJECT_SHA=$(jq -r '.object.sha // empty' <<<"$TAG_OBJECT_JSON")
-              depth=$((depth + 1))
-            done
-            [[ "$TAG_OBJECT_TYPE" == commit && "$TAG_OBJECT_SHA" =~ ^[0-9a-f]{40}$ ]] ||
-              fail "release tag does not resolve to a commit"
-            [[ "$TAG_OBJECT_SHA" == "$HEAD_SHA" ]] || fail "release tag commit and build run head SHA do not match"
-            VERSION="$TAG"
-            DEV_SEQUENCE=""
-          else
-            VERSION="dev-${HEAD_SHA}"
-            DEV_SEQUENCE="$RUN_NUMBER"
-          fi
-
-          {
-            echo "version=$VERSION"
-            echo "build_type=$BUILD_TYPE"
-            echo "build_run_id=$BUILD_RUN_ID"
-            echo "build_run_number=$RUN_NUMBER"
-            echo "head_sha=$HEAD_SHA"
-            echo "dev_sequence=$DEV_SEQUENCE"
-            echo "tag=${TAG}"
-          } >> "$GITHUB_OUTPUT"
-
-          echo "📊 Resolved:"
-          echo "  Version: $VERSION"
-          echo "  Build type: $BUILD_TYPE"
-          echo "  Build run ID: $BUILD_RUN_ID"
-          echo "  Build run number: $RUN_NUMBER"
-
-  # Consume the trusted release certification of the resolved build run and
-  # verify the certified input digests before any package is wrapped. A run
-  # without that certification fails closed here; packages built later must
-  # complete their own verification and cannot inherit an unverified
-  # conclusion.
-  release-certification:
-    name: Release Certification
-    needs: resolve
-    if: needs.resolve.result == 'success'
-    runs-on: ubuntu-latest # gh CLI required; the sm-standard-2 fleet does not ship it
-    timeout-minutes: 30
-    permissions:
-      contents: read
-      actions: read
-    steps:
-      - name: Checkout repository
-        uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
-        with:
-          persist-credentials: false
-
-      - name: Download and verify release certification
-        env:
-          GH_TOKEN: ${{ secrets.RELEASE_VALIDATION_TOKEN || github.token }}
-          BUILD_RUN_ID: ${{ needs.resolve.outputs.build_run_id }}
-          BUILD_TYPE: ${{ needs.resolve.outputs.build_type }}
-        shell: bash
-        run: |
-          set -euo pipefail
-          mkdir -p certification
-          if [[ "$BUILD_TYPE" == "development" ]]; then
-            # Development artifacts are explicitly uncertified and never touch
-            # official or latest channels; only the staging/dev paths remain.
-            jq -n '{schema_version: 1, certified: false, reason: "development artifacts are explicitly uncertified"}' \
-              > certification/development-uncertified.json
-            exit 0
-          fi
-          if ! gh run download "$BUILD_RUN_ID" -n release-certification -D certification; then
-            echo "Build run ${BUILD_RUN_ID} published no release-certification artifact; failing closed" >&2
-            exit 1
-          fi
-          python3 scripts/release/release_gate.py verify-promotion \
-            --plan .config/release-validation-plan.json \
-            --candidate certification/candidate.json \
-            --summary certification/release-gate-output \
-            --artifact-sha256 "$(jq -r '.deliverables[0].sha256' certification/candidate.json)" \
-            --artifact-name "$(jq -r '.deliverables[0].name' certification/candidate.json)"
-
-  # Build DEB and RPM packages for each architecture and libc variant
  package:
-    name: Package (${{ matrix.arch }}-${{ matrix.libc }})
-    needs: [ resolve, release-certification ]
-    runs-on: ubuntu-latest # gh CLI required; the sm-standard-2 fleet does not ship it
-    timeout-minutes: 30
+    runs-on: ubuntu-latest
+    timeout-minutes: 45
    strategy:
      fail-fast: false
-      max-parallel: 1
      matrix:
-        include:
-          - arch: x86_64
-            libc: gnu
-            deb_arch: amd64
-            rpm_arch: x86_64
-            artifact_name: "rustfs-linux-x86_64-gnu"
-            # gnu binaries dynamically link glibc; musl binaries are static
-            # and run without a libc dependency.
-            deb_depends: "libc6 (>= 2.31)"
-            rpm_depends: "glibc >= 2.31"
-          - arch: aarch64
-            libc: gnu
-            deb_arch: arm64
-            rpm_arch: aarch64
-            artifact_name: "rustfs-linux-aarch64-gnu"
-            deb_depends: "libc6 (>= 2.31)"
-            rpm_depends: "glibc >= 2.31"
-          - arch: x86_64
-            libc: musl
-            deb_arch: amd64
-            rpm_arch: x86_64
-            artifact_name: "rustfs-linux-x86_64-musl"
-            deb_depends: ""
-            rpm_depends: ""
-          - arch: aarch64
-            libc: musl
-            deb_arch: arm64
-            rpm_arch: aarch64
-            artifact_name: "rustfs-linux-aarch64-musl"
-            deb_depends: ""
-            rpm_depends: ""
+        target: [linux-x86_64-musl, linux-aarch64-musl, linux-x86_64-gnu, linux-aarch64-gnu]
    steps:
-      - name: Checkout repository
-        uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
+      - uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
        with:
+          ref: ${{ fromJson(inputs.candidate_manifest).source_sha }}
          persist-credentials: false
-
-      - name: Normalize package metadata
-        id: versions
-        shell: bash
-        env:
-          BUILD_TYPE: ${{ needs.resolve.outputs.build_type }}
-          SOURCE_VERSION: ${{ needs.resolve.outputs.version }}
-          DEV_SEQUENCE: ${{ needs.resolve.outputs.dev_sequence }}
-          ARCH: ${{ matrix.rpm_arch }}
-          LIBC: ${{ matrix.libc }}
-        run: |
-          set -euo pipefail
-
-          normalized=$(./scripts/release/package_versions.sh \
-            "$BUILD_TYPE" "$SOURCE_VERSION" "$DEV_SEQUENCE" "$ARCH" "$LIBC")
-          printf '%s\n' "$normalized" >> "$GITHUB_OUTPUT"
-
-      - name: Download binary artifact from build run
-        uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
-        with:
-          pattern: ${{ matrix.artifact_name }}*
-          path: ./binary-artifact
-          run-id: ${{ needs.resolve.outputs.build_run_id }}
-          github-token: ${{ github.token }}
-          merge-multiple: true
-
-      - name: Extract binary
-        id: binary
-        shell: bash
-        run: |
-          set -euo pipefail
-
-          ZIP_FILE=$(find ./binary-artifact -name "*.zip" -type f | head -1)
-          if [[ -z "$ZIP_FILE" ]]; then
-            echo "❌ No binary artifact found"
-            find ./binary-artifact -mindepth 1 -maxdepth 1 -print 2>/dev/null || true
-            exit 1
-          fi
-
-          echo "Found artifact: $ZIP_FILE"
-
-          mkdir -p ./bin
-          unzip -o "$ZIP_FILE" -d ./bin
-
-          if [[ ! -f ./bin/rustfs ]]; then
-            echo "❌ rustfs binary not found in archive"
-            exit 1
-          fi
-
-          chmod +x ./bin/rustfs
-          stat --printf='%n %s bytes\n' ./bin/rustfs
-          echo "✅ Binary extracted"
-
-      - name: Build DEB package
-        id: deb
-        shell: bash
-        env:
-          DEB_VERSION: ${{ steps.versions.outputs.deb_version }}
-          DEB_ARCH: ${{ matrix.deb_arch }}
-          DEB_FILE: ${{ steps.versions.outputs.deb_file }}
-          DEB_DEPENDS: ${{ matrix.deb_depends }}
-          LIBC: ${{ matrix.libc }}
-        run: |
-          set -euo pipefail
-
-          PKG_DIR="${DEB_FILE%.deb}"
-
-          echo "Building DEB: ${DEB_FILE}"
-
-          mkdir -p "${PKG_DIR}/DEBIAN"
-          mkdir -p "${PKG_DIR}/usr/bin"
-          mkdir -p "${PKG_DIR}/etc/default"
-          mkdir -p "${PKG_DIR}/lib/systemd/system"
-          mkdir -p "${PKG_DIR}/usr/share/doc/rustfs"
-
-          cp ./bin/rustfs "${PKG_DIR}/usr/bin/"
-          chmod 755 "${PKG_DIR}/usr/bin/rustfs"
-
-          cp deploy/build/rustfs.service "${PKG_DIR}/lib/systemd/system/"
-
-          cat > "${PKG_DIR}/etc/default/rustfs" << 'ENVEOF'
-          # RustFS Environment Configuration
-          # See https://docs.rustfs.com for more information
-          # RUSTFS_ACCESS_KEY=<changeme>
-          # RUSTFS_SECRET_KEY=<changeme>
-          # RUSTFS_VOLUMES=""
-          # RUSTFS_ADDRESS=":9000"
-          # RUSTFS_CONSOLE_ADDRESS=":9001"
-          # RUSTFS_CONSOLE_ENABLE=true
-          # RUSTFS_OBS_LOGGER_LEVEL=error
-          # RUSTFS_OBS_LOG_DIRECTORY="/var/log/rustfs/"
-          ENVEOF
-
-          cat > "${PKG_DIR}/DEBIAN/control" << EOF
-          Package: rustfs
-          Version: ${DEB_VERSION}
-          Section: utils
-          Priority: optional
-          Architecture: ${DEB_ARCH}
-          EOF
-
-          # gnu binaries dynamically link glibc and must declare it; musl
-          # binaries are statically linked and need no libc dependency.
-          if [[ -n "$DEB_DEPENDS" ]]; then
-            echo "Depends: ${DEB_DEPENDS}" >> "${PKG_DIR}/DEBIAN/control"
-          fi
-
-          cat >> "${PKG_DIR}/DEBIAN/control" << EOF
-          Maintainer: RustFS Team <support@rustfs.com>
-          Description: High-performance distributed object storage (${LIBC} build)
-           RustFS is a high-performance distributed object storage software
-           built using Rust. It is compatible with MinIO and S3 API.
-          Homepage: https://rustfs.com
-          EOF
-
-          # Declare /etc/default/rustfs as a conffile so dpkg preserves user
-          # modifications on upgrade instead of silently overwriting them.
-          cat > "${PKG_DIR}/DEBIAN/conffiles" << 'CONFFILES'
-          /etc/default/rustfs
-          CONFFILES
-
-          ./scripts/release/package_service_scripts.sh after-install > "${PKG_DIR}/DEBIAN/postinst"
-          chmod 755 "${PKG_DIR}/DEBIAN/postinst"
-
-          ./scripts/release/package_service_scripts.sh before-remove > "${PKG_DIR}/DEBIAN/prerm"
-          chmod 755 "${PKG_DIR}/DEBIAN/prerm"
-
-          ./scripts/release/package_service_scripts.sh after-remove > "${PKG_DIR}/DEBIAN/postrm"
-          chmod 755 "${PKG_DIR}/DEBIAN/postrm"
-
-          cp LICENSE "${PKG_DIR}/usr/share/doc/rustfs/"
-          cp README.md "${PKG_DIR}/usr/share/doc/rustfs/"
-
-          fakeroot dpkg-deb --build "${PKG_DIR}" "$DEB_FILE"
-
-          [[ $(dpkg-deb -f "$DEB_FILE" Package) == rustfs ]]
-          [[ $(dpkg-deb -f "$DEB_FILE" Version) == "$DEB_VERSION" ]]
-          [[ $(dpkg-deb -f "$DEB_FILE" Architecture) == "$DEB_ARCH" ]]
-          dpkg-deb --fsys-tarfile "$DEB_FILE" | tar -tf - | grep -Fx './usr/bin/rustfs' >/dev/null
-          stat --printf='%n %s bytes\n' "$DEB_FILE"
-          echo "deb_file=$DEB_FILE" >> "$GITHUB_OUTPUT"
-          echo "✅ DEB built: $DEB_FILE"
-
-      - name: Build RPM package
-        id: rpm
-        shell: bash
-        env:
-          RPM_VERSION: ${{ steps.versions.outputs.rpm_version }}
-          RPM_RELEASE: ${{ steps.versions.outputs.rpm_release }}
-          RPM_ARCH: ${{ matrix.rpm_arch }}
-          RPM_FILE: ${{ steps.versions.outputs.rpm_file }}
-          RPM_DEPENDS: ${{ matrix.rpm_depends }}
-          LIBC: ${{ matrix.libc }}
-        run: |
-          set -euo pipefail
-
-          echo "Building RPM for ${RPM_ARCH} (${LIBC})"
-
-          sudo apt-get update && sudo apt-get install -y ruby ruby-dev build-essential rpm
-          sudo gem install fpm
-          ./scripts/test_package_versions.sh --require-package-managers
-
-          # gnu binaries dynamically link glibc and must declare it; musl
-          # binaries are statically linked and need no libc dependency.
-          depends_args=()
-          if [[ -n "$RPM_DEPENDS" ]]; then
-            depends_args+=(--depends "$RPM_DEPENDS")
-          fi
-
-          # Create config file for fpm (DEB build creates it in its package dir structure,
-          # but fpm needs the file to exist before packaging)
-          mkdir -p ./tmp-pkg/etc/default
-          cat > ./tmp-pkg/etc/default/rustfs << 'ENVEOF'
-          # RustFS Environment Configuration
-          # See https://docs.rustfs.com for more information
-          # RUSTFS_ACCESS_KEY=<changeme>
-          # RUSTFS_SECRET_KEY=<changeme>
-          # RUSTFS_VOLUMES=""
-          # RUSTFS_ADDRESS=":9000"
-          # RUSTFS_CONSOLE_ADDRESS=":9001"
-          # RUSTFS_CONSOLE_ENABLE=true
-          # RUSTFS_OBS_LOGGER_LEVEL=error
-          # RUSTFS_OBS_LOG_DIRECTORY="/var/log/rustfs/"
-          ENVEOF
-
-          fpm -s dir -t rpm \
-            --name rustfs \
-            --version "$RPM_VERSION" \
-            --iteration "$RPM_RELEASE" \
-            --architecture "$RPM_ARCH" \
-            --package "$RPM_FILE" \
-            "${depends_args[@]}" \
-            --description "High-performance distributed object storage (${LIBC} build)" \
-            --maintainer "RustFS Team <support@rustfs.com>" \
-            --url "https://rustfs.com" \
-            --license "Apache-2.0" \
-            --before-install <(./scripts/release/package_service_scripts.sh rpm-before-install) \
-            --before-upgrade <(./scripts/release/package_service_scripts.sh rpm-before-upgrade) \
-            --after-install <(./scripts/release/package_service_scripts.sh after-install) \
-            --after-upgrade <(./scripts/release/package_service_scripts.sh after-install) \
-            --before-remove <(./scripts/release/package_service_scripts.sh before-remove) \
-            --after-remove <(./scripts/release/package_service_scripts.sh after-remove) \
-            --rpm-posttrans <(./scripts/release/package_service_scripts.sh rpm-posttrans) \
-            --config-files /etc/default/rustfs \
-            ./bin/rustfs=/usr/bin/rustfs \
-            ./tmp-pkg/etc/default/rustfs=/etc/default/rustfs \
-            deploy/build/rustfs.service=/lib/systemd/system/rustfs.service \
-            LICENSE=/usr/share/doc/rustfs/LICENSE \
-            README.md=/usr/share/doc/rustfs/README.md
-
-          if [[ ! -f "$RPM_FILE" ]]; then
-            echo "❌ RPM build failed"
-            exit 1
-          fi
-
-          RPM_METADATA=$(rpm -qp --qf '%{NAME}\n%{VERSION}\n%{RELEASE}\n%{ARCH}\n' "$RPM_FILE")
-          EXPECTED_METADATA=$(printf 'rustfs\n%s\n%s\n%s' "$RPM_VERSION" "$RPM_RELEASE" "$RPM_ARCH")
-          [[ "$RPM_METADATA" == "$EXPECTED_METADATA" ]]
-          rpm -qpl "$RPM_FILE" | grep -Fx '/usr/bin/rustfs' >/dev/null
-          stat --printf='%n %s bytes\n' "$RPM_FILE"
-          echo "rpm_file=$RPM_FILE" >> "$GITHUB_OUTPUT"
-          echo "✅ RPM built: $RPM_FILE"
-
-      - name: Download certification evidence
-        if: needs.resolve.outputs.build_type != 'development'
-        uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
-        with:
-          name: release-certification
-          path: ./certification
-
-      # Later-built wrapper verification: the binary inside each package must
-      # match the certified binary digest for this platform before anything is
-      # uploaded, and the package itself stays on staging until promotion.
-      - name: Verify wrapped binary against certification
-        if: needs.resolve.outputs.build_type != 'development'
-        shell: bash
-        env:
-          DEB_FILE: ${{ steps.deb.outputs.deb_file }}
-          RPM_FILE: ${{ steps.rpm.outputs.rpm_file }}
-          ARTIFACT_NAME: ${{ matrix.artifact_name }}
-        run: |
-          set -euo pipefail
-          expected="$(jq -r --arg needle "$ARTIFACT_NAME" \
-            '[.deliverables[] | select(.name | contains($needle)) | .binary_sha256][0] // empty' \
-            certification/candidate.json)"
-          if [[ -z "$expected" ]]; then
-            echo "No certified binary digest recorded for ${ARTIFACT_NAME}" >&2
-            exit 1
-          fi
-          if ! command -v rpm2cpio >/dev/null 2>&1; then
-            sudo apt-get update && sudo apt-get install -y rpm2cpio
-          fi
-          if ! command -v cpio >/dev/null 2>&1; then
-            sudo apt-get update && sudo apt-get install -y cpio
-          fi
-          for package_file in "$DEB_FILE" "$RPM_FILE"; do
-            case "$package_file" in
-              *.deb)
-                dpkg-deb --fsys-tarfile "$package_file" | tar -xOf - ./usr/bin/rustfs > "$RUNNER_TEMP/wrapped-rustfs"
-                ;;
-              *.rpm)
-                rpm2cpio "$package_file" | cpio -i --to-stdout ./usr/bin/rustfs > "$RUNNER_TEMP/wrapped-rustfs"
-                ;;
-            esac
-            python3 scripts/release/release_gate.py verify-promotion \
-              --plan .config/release-validation-plan.json \
-              --candidate certification/candidate.json \
-              --summary certification/release-gate-output \
-              --artifact "$RUNNER_TEMP/wrapped-rustfs" \
-              --artifact-role binary
-          done
-
-      - name: Upload packages to artifacts
-        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
-        with:
-          name: packages-${{ matrix.arch }}-${{ matrix.libc }}
-          path: |
-            *.deb
-            *.rpm
-          retention-days: 30
-
-      - name: Upload packages to Cloudflare R2
-        if: needs.resolve.outputs.build_type != 'preview' && env.R2_ACCESS_KEY_ID != ''
-        env:
-          R2_ACCESS_KEY_ID: ${{ secrets.R2_ACCESS_KEY_ID }}
-          R2_SECRET_ACCESS_KEY: ${{ secrets.R2_SECRET_ACCESS_KEY }}
-          R2_ENDPOINT: ${{ secrets.R2_ENDPOINT }}
-          R2_BUCKET: ${{ secrets.R2_BUCKET }}
-          AWS_EC2_METADATA_DISABLED: true
-          BUILD_TYPE: ${{ needs.resolve.outputs.build_type }}
-          DEB_FILE: ${{ steps.deb.outputs.deb_file }}
-          RPM_FILE: ${{ steps.rpm.outputs.rpm_file }}
-        shell: bash
-        run: |
-          set -euo pipefail
-
-          if [[ -z "$R2_ACCESS_KEY_ID" || -z "$R2_SECRET_ACCESS_KEY" || -z "$R2_ENDPOINT" || -z "$R2_BUCKET" ]]; then
-            echo "⚠️ R2 credentials missing, skipping upload"
-            exit 0
-          fi
-
-          if ! command -v aws >/dev/null 2>&1; then
-            sudo apt-get update && sudo apt-get install -y awscli
-          fi
-
-          export AWS_ACCESS_KEY_ID="$R2_ACCESS_KEY_ID"
-          export AWS_SECRET_ACCESS_KEY="$R2_SECRET_ACCESS_KEY"
-          export AWS_DEFAULT_REGION="auto"
-
-          # Build-phase writes stay on staging for release types: the official
-          # packages/release and packages/latest paths are written only by the
-          # certification-gated promote-packages job, from these same staged
-          # objects. Development packages keep the uncertified dev channel.
-          if [[ "$BUILD_TYPE" == "development" ]]; then
-            R2_PREFIX="artifacts/rustfs/packages/dev"
-          else
-            R2_PREFIX="artifacts/rustfs/packages/staging/${GITHUB_RUN_ID}/${GITHUB_RUN_ATTEMPT}"
-          fi
-          R2_PATH="s3://${R2_BUCKET}/${R2_PREFIX}/"
-
-          echo "📤 Uploading to $R2_PATH"
-
-          for f in "$DEB_FILE" "$RPM_FILE"; do
-            if [[ -n "$f" && -f "$f" ]]; then
-              echo "Uploading: $f"
-              aws s3 cp "$f" "$R2_PATH" --endpoint-url "$R2_ENDPOINT" --only-show-errors
-            fi
-          done
-
-          echo "✅ Upload complete"
-
-      - name: Upload packages to GitHub Release
-        if: needs.resolve.outputs.tag != ''
+      - name: Reuse completed package set
+        id: reuse
        env:
          GH_TOKEN: ${{ github.token }}
-          TAG: ${{ needs.resolve.outputs.tag }}
-          DEB_FILE: ${{ steps.deb.outputs.deb_file }}
-          RPM_FILE: ${{ steps.rpm.outputs.rpm_file }}
-        shell: bash
+          CANDIDATE_MANIFEST: ${{ inputs.candidate_manifest }}
+          TARGET: ${{ matrix.target }}
+        run: python3 scripts/release/deliver.py reuse --kind packages --target "$TARGET"
+      - name: Install packaging tools
+        if: steps.reuse.outputs.reused != 'true'
        run: |
-          set -euo pipefail
-
-          # Upload the packages, then refresh the release checksums so the new
-          # assets are covered, matching the binary release flow.
-          for f in "$DEB_FILE" "$RPM_FILE"; do
-            if [[ -n "$f" && -f "$f" ]]; then
-              echo "📤 Uploading $(basename "$f") to GitHub release ${TAG}..."
-              gh release upload "$TAG" "$f" --clobber
-            fi
-          done
-
-          CHECKSUM_DIR="$(mktemp -d)"
-          gh release download "$TAG" -p 'SHA256SUMS' -p 'SHA512SUMS' \
-            -D "$CHECKSUM_DIR" --clobber
-
-          for spec in "SHA256SUMS:sha256sum" "SHA512SUMS:sha512sum"; do
-            asset="${spec%%:*}"
-            checksum_cmd="${spec##*:}"
-            checksum_file="${CHECKSUM_DIR}/${asset}"
-
-            for f in "$DEB_FILE" "$RPM_FILE"; do
-              if [[ -n "$f" && -f "$f" ]]; then
-                base="$(basename "$f")"
-                # GitHub stores release asset names with '~' normalized to '.'.
-                # Current package file names embed the raw tag and contain no
-                # '~', but older releases may still ship such assets, so the
-                # checksum entries must reference the name as stored on the
-                # release.
-                github_base="${base//\~/.}"
-                # Remove any stale entry (both naming variants), then append
-                grep -Fv -- "$base" "$checksum_file" > "${checksum_file}.tmp" || true
-                grep -Fv -- "$github_base" "${checksum_file}.tmp" > "${checksum_file}.tmp2" || true
-                mv "${checksum_file}.tmp2" "$checksum_file"
-                digest=$("$checksum_cmd" -- "$f" | awk '{print $1}')
-                printf '%s  %s\n' "$digest" "$github_base" >> "$checksum_file"
-              fi
-            done
-
-            echo "📤 Updating ${asset} for release ${TAG}..."
-            gh release upload "$TAG" "$checksum_file" --clobber
-          done
-
-          echo "✅ GitHub release assets updated"
-
-  # The only writer of the official packages/release and packages/latest R2
-  # paths. Promotion re-checks each package's wrapped binary against the
-  # passed certification list and copies the staged objects byte-for-byte:
-  # no rebuild-after-verify, no mutable latest resolution downstream.
-  promote-packages:
-    name: Promote Packages
-    needs: [ resolve, package, release-certification ]
-    # Target-channel lock for the official package paths.
-    concurrency:
-      group: release-channel-r2-packages
-      cancel-in-progress: false
-    if: needs.resolve.outputs.tag != '' && (needs.resolve.outputs.build_type == 'release' || needs.resolve.outputs.build_type == 'prerelease')
-    runs-on: ubuntu-latest # gh CLI required; the sm-standard-2 fleet does not ship it
-    timeout-minutes: 30
-    permissions:
-      contents: read
-      actions: read
-    env:
-      R2_ACCESS_KEY_ID: ${{ secrets.R2_ACCESS_KEY_ID }}
-      R2_SECRET_ACCESS_KEY: ${{ secrets.R2_SECRET_ACCESS_KEY }}
-      R2_ENDPOINT: ${{ secrets.R2_ENDPOINT }}
-      R2_BUCKET: ${{ secrets.R2_BUCKET }}
-      AWS_EC2_METADATA_DISABLED: true
-    steps:
-      - name: Checkout repository
-        uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
-        with:
-          persist-credentials: false
-
-      - name: Download packages and certification
-        uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
-        with:
-          path: ./packages
-          pattern: packages-*
-          merge-multiple: true
-
-      - name: Download certification evidence
-        uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
-        with:
-          name: release-certification
-          path: ./certification
-
-      - name: Verify and promote staged packages
-        shell: bash
-        run: |
-          set -euo pipefail
-
-          if [[ -z "$R2_ACCESS_KEY_ID" || -z "$R2_SECRET_ACCESS_KEY" || -z "$R2_ENDPOINT" || -z "$R2_BUCKET" ]]; then
-            echo "❌ R2 credentials missing; refusing to promote to the official channel" >&2
-            exit 1
-          fi
-
-          STAGING_PREFIX="s3://${R2_BUCKET}/artifacts/rustfs/packages/staging/${GITHUB_RUN_ID}/${GITHUB_RUN_ATTEMPT}/"
-          RELEASE_PATH="s3://${R2_BUCKET}/artifacts/rustfs/packages/release/"
-          LATEST_PATH="s3://${R2_BUCKET}/artifacts/rustfs/packages/latest/"
-
-          if ! command -v aws >/dev/null 2>&1; then
-            sudo apt-get update && sudo apt-get install -y awscli
-          fi
-          export AWS_ACCESS_KEY_ID="$R2_ACCESS_KEY_ID"
-          export AWS_SECRET_ACCESS_KEY="$R2_SECRET_ACCESS_KEY"
-          export AWS_DEFAULT_REGION="auto"
-
-          if ! command -v rpm2cpio >/dev/null 2>&1; then
-            sudo apt-get install -y rpm2cpio
-          fi
-          if ! command -v cpio >/dev/null 2>&1; then
-            sudo apt-get install -y cpio
-          fi
-
-          promoted=0
-          for package_file in packages/*.deb packages/*.rpm; do
-            [[ -f "$package_file" ]] || continue
-            name="$(basename "$package_file")"
-            case "$package_file" in
-              *.deb)
-                dpkg-deb --fsys-tarfile "$package_file" | tar -xOf - ./usr/bin/rustfs > "$RUNNER_TEMP/wrapped-rustfs"
-                ;;
-              *.rpm)
-                rpm2cpio "$package_file" | cpio -i --to-stdout ./usr/bin/rustfs > "$RUNNER_TEMP/wrapped-rustfs"
-                ;;
-            esac
-            python3 scripts/release/release_gate.py verify-promotion \
-              --plan .config/release-validation-plan.json \
-              --candidate certification/candidate.json \
-              --summary certification/release-gate-output \
-              --artifact "$RUNNER_TEMP/wrapped-rustfs" \
-              --artifact-role binary
-
-            staged="$(mktemp)"
-            aws s3 cp "${STAGING_PREFIX}${name}" "$staged" --endpoint-url "$R2_ENDPOINT" --only-show-errors
-            staged_sha="$(sha256sum "$staged" | awk '{print $1}')"
-            built_sha="$(sha256sum "$package_file" | awk '{print $1}')"
-            if [[ "$staged_sha" != "$built_sha" ]]; then
-              echo "❌ Staged package ${name} does not match the verified bytes" >&2
-              exit 1
-            fi
-            aws s3 cp "$staged" "${RELEASE_PATH}${name}" --endpoint-url "$R2_ENDPOINT" --only-show-errors
-            aws s3 cp "$staged" "${LATEST_PATH}${name}" --endpoint-url "$R2_ENDPOINT" --only-show-errors
-            rm -f "$staged"
-            promoted=$((promoted + 1))
-            echo "✅ Promoted ${name} (${built_sha})"
-          done
-          if [[ "$promoted" -eq 0 ]]; then
-            echo "No staged packages found to promote" >&2
-            exit 1
-          fi
-
-  # Summary
-  summary:
-    name: Summary
-    needs: [ resolve, package ]
-    # Every completed Build and Release run triggers this workflow; when
-    # resolve declined it (not a release tag), there is nothing to summarise.
-    if: always() && needs.resolve.result != 'skipped'
-    runs-on: ubuntu-latest
-    timeout-minutes: 5
-    steps:
-      - name: Print summary
-        shell: bash
+          sudo apt-get update
+          sudo apt-get install -y ruby ruby-dev build-essential rpm fakeroot cpio
+          sudo gem install fpm --version 1.17.0 --no-document
+      - name: Package accepted executable and verify contents
+        if: steps.reuse.outputs.reused != 'true'
        env:
-          SUMMARY_VERSION: ${{ needs.resolve.outputs.version }}
-          SUMMARY_BUILD_TYPE: ${{ needs.resolve.outputs.build_type }}
-          SUMMARY_BUILD_RUN_ID: ${{ needs.resolve.outputs.build_run_id }}
-          SUMMARY_PACKAGE_STATUS: ${{ needs.package.result }}
-        run: |
-          {
-            echo "## 📦 Package Summary"
-            echo ""
-            echo "| Item | Value |"
-            echo "|------|-------|"
-            echo "| Version | \`${SUMMARY_VERSION}\` |"
-            echo "| Build Type | ${SUMMARY_BUILD_TYPE} |"
-            echo "| Build Run | #${SUMMARY_BUILD_RUN_ID} |"
-            echo "| Package Status | ${SUMMARY_PACKAGE_STATUS} |"
-          } >> "$GITHUB_STEP_SUMMARY"
+          GH_TOKEN: ${{ github.token }}
+          CANDIDATE_MANIFEST: ${{ inputs.candidate_manifest }}
+          TARGET: ${{ matrix.target }}
+        run: python3 scripts/release/deliver.py package --target "$TARGET" --output delivery
+      - uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+        if: steps.reuse.outputs.reused != 'true'
+        with:
+          name: packages-${{ matrix.target }}
+          path: delivery/*
+          if-no-files-found: error
+          compression-level: 0
+          retention-days: 30
@@ -0,0 +1,444 @@
+# Copyright 2026 RustFS Team
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# s3-tests reference parity lane (rustfs/backlog#2773).
+#
+# Runs the ceph/s3-tests implemented list (scripts/s3-tests/implemented_tests.txt)
+# against every target in .config/reference-parity-targets.toml and fails on
+# any per-node-id difference between a candidate and the reference:
+#
+#   - reference: the release pinned in .config/reference-binary.toml, downloaded
+#     and sha256-verified by .github/actions/reference-binary;
+#   - candidate targets: the debug binary built from this checkout exactly as
+#     ci.yml's build-rustfs-debug-binary job builds it for S3 Implemented Tests.
+#
+# Every target runs the same harness (scripts/s3-tests/run.sh from this
+# checkout), test list and server configuration on its own hosted runner. A
+# target's failures get one rerun against a fresh server; a case that passes it
+# is flaky, not a difference. scripts/s3-tests/reference_parity.py judges the
+# result and honours .config/reference-parity-tolerated.txt, whose entries must
+# each match a real difference.
+#
+# The `differential` job adds the 73-operation dual-stack differential
+# (crates/e2e_test/src/dual_stack_differential, rustfs/backlog#2740) against the
+# same pinned reference: side A is the released binary (RUSTFS_E2E_REFERENCE_BINARY,
+# started without RUSTFS_S3_STACK), side B is the candidate built above on its
+# legacy stack. It reuses the candidate build artifact, never builds the candidate
+# twice, and prints reference_side, version_a and version_b from the report the test
+# wrote; it fails when side A was not the reference or both sides report the same
+# version. Every (op, field, a, b) difference goes to the job summary. Report-only
+# until the first result has been reviewed.
+#
+# Report-only: not part of the Test and Lint aggregate. Scheduled failures are
+# owned by the scheduled-validation watchdog (.github/scheduled-validations.json).
+
+name: Reference Parity
+
+on:
+  pull_request:
+    paths:
+      - ".github/workflows/reference-parity.yml"
+      - ".github/actions/reference-binary/**"
+      - ".config/reference-binary.toml"
+      - ".config/reference-parity-targets.toml"
+      - ".config/reference-parity-tolerated.txt"
+      - "scripts/s3-tests/reference_parity.py"
+      - "scripts/junit_diff.py"
+      - "crates/e2e_test/src/dual_stack_differential/**"
+  schedule:
+    - cron: "37 1 * * *"
+  workflow_dispatch:
+
+# Matches ci.yml: these variables are hashed into the rust-cache key, so the
+# candidate build reads the same ci-dev cache as build-rustfs-debug-binary.
+env:
+  CARGO_TERM_COLOR: always
+  RUST_BACKTRACE: 1
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: true
+
+permissions:
+  contents: read
+
+defaults:
+  run:
+    shell: bash
+
+jobs:
+  plan:
+    name: Plan parity targets
+    runs-on: ubuntu-latest
+    timeout-minutes: 5
+    outputs:
+      targets: ${{ steps.targets.outputs.targets }}
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
+        with:
+          persist-credentials: false
+
+      - name: Validate target list
+        id: targets
+        run: |
+          set -euo pipefail
+          targets="$(python3 scripts/s3-tests/reference_parity.py targets --config .config/reference-parity-targets.toml)"
+          echo "targets=${targets}" >> "$GITHUB_OUTPUT"
+          echo "Parity targets: ${targets}"
+
+  build-candidate:
+    name: Build candidate binary
+    # Critical-path CPU and cache ownership: the same debug build as ci.yml's
+    # build-rustfs-debug-binary, reading the ci-dev cache written on sm-standard-4.
+    runs-on: sm-standard-4
+    timeout-minutes: 60
+    env:
+      FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
+        with:
+          persist-credentials: false
+
+      - name: Setup Rust environment
+        uses: ./.github/actions/setup
+        with:
+          rust-version: stable
+          cache-shared-key: ci-dev
+          cache-save-if: 'false'
+          install-build-packaging-tools: 'false'
+
+      - name: Build debug binary
+        run: python3 scripts/e2e_binary.py build --bins --features e2e-test-hooks
+
+      - name: Upload candidate binary
+        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+        with:
+          name: reference-parity-candidate-binary
+          overwrite: true
+          path: |
+            target/debug/rustfs
+            target/debug/rustfs.e2e.json
+          if-no-files-found: error
+          retention-days: 1
+
+  s3tests:
+    name: s3-tests (${{ matrix.target.name }})
+    needs: [ plan, build-candidate ]
+    runs-on: ubuntu-latest
+    timeout-minutes: 75
+    strategy:
+      fail-fast: false
+      matrix:
+        target: ${{ fromJSON(needs.plan.outputs.targets) }}
+    env:
+      TARGET_NAME: ${{ matrix.target.name }}
+      TARGET_KIND: ${{ matrix.target.binary }}
+      # Above this many first-run failures the target is systemically broken;
+      # a rerun would only repeat it, so the first run is judged as is.
+      RERUN_CAP: "50"
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
+        with:
+          persist-credentials: false
+
+      - name: Fetch pinned reference binary
+        id: reference
+        if: matrix.target.binary == 'reference'
+        uses: ./.github/actions/reference-binary
+
+      - name: Download candidate binary
+        if: matrix.target.binary == 'candidate'
+        uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
+        with:
+          name: reference-parity-candidate-binary
+          path: target/debug
+
+      - name: Select target binary
+        env:
+          REFERENCE_PATH: ${{ steps.reference.outputs.path }}
+        run: |
+          set -euo pipefail
+          case "$TARGET_KIND" in
+            reference) binary="$REFERENCE_PATH" ;;
+            candidate) chmod +x target/debug/rustfs; binary="$PWD/target/debug/rustfs" ;;
+            *) echo "unknown target binary kind: $TARGET_KIND" >&2; exit 1 ;;
+          esac
+          test -x "$binary"
+          echo "TARGET_BINARY=$binary" >> "$GITHUB_ENV"
+          mkdir -p "parity/$TARGET_NAME"
+          {
+            echo "target=$TARGET_NAME"
+            echo "binary=$TARGET_KIND"
+            echo "sha256=$(sha256sum "$binary" | cut -d' ' -f1)"
+            echo "version=$("$binary" --version | tr '\n' ' ')"
+          } | tee "parity/$TARGET_NAME/target.txt"
+
+      - name: Apply target environment
+        env:
+          TARGET_JSON: ${{ toJSON(matrix.target) }}
+        run: |
+          set -euo pipefail
+          python3 scripts/s3-tests/reference_parity.py export-env --target-json "$TARGET_JSON" \
+            | tee -a "$GITHUB_ENV" "parity/$TARGET_NAME/target-env.txt"
+
+      - name: Run implemented s3-tests
+        run: |
+          set -uo pipefail
+          report="parity/$TARGET_NAME"
+          harness="artifacts/s3tests-single"
+
+          # Same invocation as ci.yml's S3 Implemented Tests, with a fresh data
+          # root and port per run. $1 optionally replaces the test list.
+          run_suite() {
+            local list="$1" label="$2" root port
+            root="${RUNNER_TEMP}/rustfs-s3tests-${label}"
+            rm -rf "$harness" "$root"
+            mkdir -p "$root"
+            port="$(python3 -c 'import socket; s=socket.socket(); s.bind(("127.0.0.1", 0)); print(s.getsockname()[1]); s.close()')"
+            env DEPLOY_MODE=binary \
+              RUSTFS_BINARY="$TARGET_BINARY" \
+              TEST_MODE=single \
+              MAXFAIL=0 \
+              XDIST=4 \
+              S3_PORT="$port" \
+              DATA_ROOT="$root" \
+              S3TESTS_CONF="$harness/s3tests.conf" \
+              ${list:+IMPLEMENTED_TESTS_FILE="$list"} \
+              ./scripts/s3-tests/run.sh
+          }
+
+          # A run is usable only if it produced a JUnit report; test failures
+          # are data for the comparison, a missing report is a lane failure.
+          keep() {
+            local prefix="$1"
+            for file in pytest.log compat-report.md; do
+              if [[ -f "$harness/$file" ]]; then cp "$harness/$file" "$report/${prefix}${file}"; fi
+            done
+            if [[ -f "$harness/rustfs-single/rustfs.log" ]]; then cp "$harness/rustfs-single/rustfs.log" "$report/${prefix}rustfs.log"; fi
+            test -s "$harness/junit.xml" || { echo "::error::$TARGET_NAME produced no JUnit report (${prefix:-first} run)"; return 1; }
+            cp "$harness/junit.xml" "$report/${prefix}junit.xml"
+          }
+
+          run_suite "" first
+          echo "first_run_exit=$?" >> "$report/target.txt"
+          keep "" || exit 1
+          cp "$harness/selected-nodeids.txt" "$report/selected-nodeids.txt" || exit 1
+
+          failures="$(python3 scripts/s3-tests/reference_parity.py failed --report "$report")" || exit 1
+          count="$(printf '%s' "$failures" | grep -c . || true)"
+          echo "first_run_failures=$count" >> "$report/target.txt"
+          if (( count == 0 )); then
+            exit 0
+          fi
+          if (( count > RERUN_CAP )); then
+            echo "::warning::$TARGET_NAME has $count failures (cap $RERUN_CAP); judging the first run without a rerun"
+            echo "rerun=skipped-over-cap" >> "$report/target.txt"
+            exit 0
+          fi
+          printf '%s\n' "$failures" > "$RUNNER_TEMP/rerun-tests.txt"
+          run_suite "$RUNNER_TEMP/rerun-tests.txt" rerun
+          echo "rerun_exit=$?" >> "$report/target.txt"
+          keep rerun- || exit 1
+
+      - name: Upload target report
+        if: always()
+        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+        with:
+          name: reference-parity-report-${{ matrix.target.name }}
+          overwrite: true
+          path: parity/${{ matrix.target.name }}/
+          if-no-files-found: error
+          retention-days: 14
+
+  differential:
+    name: Dual-stack differential (reference vs candidate)
+    needs: [ build-candidate ]
+    # Compiles the e2e_test harness against the ci-dev cache like the other
+    # sm-standard-4 jobs; the candidate server itself is the downloaded artifact.
+    runs-on: sm-standard-4
+    timeout-minutes: 90
+    env:
+      FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"
+      NO_PROXY: 127.0.0.1,localhost,::1
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
+        with:
+          persist-credentials: false
+
+      - name: Setup Rust environment
+        uses: ./.github/actions/setup
+        with:
+          rust-version: stable
+          cache-shared-key: ci-dev
+          cache-save-if: 'false'
+          install-build-packaging-tools: 'false'
+
+      - name: Fetch pinned reference binary
+        id: reference
+        uses: ./.github/actions/reference-binary
+
+      - name: Download candidate binary
+        uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
+        with:
+          name: reference-parity-candidate-binary
+          path: target/debug
+
+      - name: Run dual-stack differential
+        id: differential
+        continue-on-error: true
+        env:
+          REFERENCE_PATH: ${{ steps.reference.outputs.path }}
+          RUSTFS_E2E_LOG_DIR: ${{ runner.temp }}/rustfs-differential-logs
+        run: |
+          set -euo pipefail
+          export no_proxy="$NO_PROXY"
+          chmod +x target/debug/rustfs
+          test -x "$REFERENCE_PATH"
+          mkdir -p "$RUSTFS_E2E_LOG_DIR"
+          # Side A: the released binary. Side B: the candidate on its legacy stack.
+          export RUSTFS_E2E_REFERENCE_BINARY="$REFERENCE_PATH"
+          test_name='dual_stack_differential::tests::both_stacks_answer_every_operation_identically'
+          python3 scripts/e2e_binary.py run --features e2e-test-hooks -- \
+            cargo test --locked -p e2e_test --lib "$test_name" -- --exact --nocapture \
+            2>&1 | tee "$RUNNER_TEMP/differential-test.log"
+
+      - name: Summarize report and verify the reference side
+        if: always()
+        run: |
+          set -euo pipefail
+          python3 - <<'PY'
+          import glob, json, os, sys
+
+          summary = os.environ["GITHUB_STEP_SUMMARY"]
+          reports = sorted(glob.glob("target/dual-stack/self-comparison-*.json"))
+          lines = ["## Dual-stack differential: reference vs candidate", ""]
+          failures = []
+          if len(reports) != 1:
+              failures.append(f"expected exactly one self-comparison report, found {len(reports)}: {reports}")
+          else:
+              report = json.load(open(reports[0]))
+              side = report.get("reference_side")
+              version_a = report.get("version_a")
+              version_b = report.get("version_b")
+              lines += [
+                  f"- report: `{reports[0]}`",
+                  f"- reference_side: `{side}`",
+                  f"- version_a: `{(version_a or '').splitlines()[0] if version_a else version_a}`",
+                  f"- version_b: `{(version_b or '').splitlines()[0] if version_b else version_b}`",
+                  f"- stack_a: `{report.get('stack_a')}`, stack_b: `{report.get('stack_b')}`",
+                  f"- compared_ops: {report.get('compared_ops')}, compared_cases: {report.get('compared_cases')}",
+                  "",
+              ]
+              print(f"reference_side={side}\nversion_a={version_a}\nversion_b={version_b}")
+              lines += ["```", "version_a:", version_a or "", "", "version_b:", version_b or "", "```", ""]
+              if side != "a":
+                  failures.append(f"reference_side is {side!r}, expected 'a'")
+              if not version_a or not version_b:
+                  failures.append("version_a or version_b is missing")
+              elif version_a == version_b:
+                  failures.append("version_a equals version_b: the reference was replaced by the candidate")
+              diffs = report.get("differences") or []
+              lines.append(f"### Differences: {len(diffs)}")
+              by_op = {}
+              for d in diffs:
+                  by_op.setdefault(d["op"], []).append(d)
+              for op in sorted(by_op):
+                  lines += ["", f"#### {op}", "", "| case | field | a (reference) | b (candidate) |", "| --- | --- | --- | --- |"]
+                  for d in by_op[op]:
+                      cell = lambda v: "`" + str(v).replace("|", "\\|").replace("`", "'") + "`"
+                      lines.append(f"| {d['case']} | {d['field']} | {cell(d['a'])} | {cell(d['b'])} |")
+          if failures:
+              lines += ["", "### Observation check failed", ""] + [f"- {f}" for f in failures]
+          open(summary, "a").write("\n".join(lines) + "\n")
+          print("\n".join(lines))
+          for f in failures:
+              print(f"::error::{f}")
+          sys.exit(1 if failures else 0)
+          PY
+
+      - name: Upload differential reports
+        if: always()
+        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+        with:
+          name: reference-differential-report
+          overwrite: true
+          path: |
+            target/dual-stack/*.json
+            ${{ runner.temp }}/differential-test.log
+          if-no-files-found: warn
+          retention-days: 14
+
+      - name: Fail when the differential found differences
+        if: steps.differential.outcome == 'failure'
+        run: |
+          echo "::error::the dual-stack differential failed; see the job summary for every (op, field, a, b)"
+          exit 1
+
+  compare:
+    name: Compare targets
+    needs: [ plan, s3tests ]
+    if: always() && needs.plan.result == 'success'
+    runs-on: ubuntu-latest
+    timeout-minutes: 10
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
+        with:
+          persist-credentials: false
+
+      - name: Download target reports
+        uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
+        with:
+          pattern: reference-parity-report-*
+          path: parity-download
+
+      - name: Compare per node id
+        env:
+          TARGETS: ${{ needs.plan.outputs.targets }}
+          S3TESTS_RESULT: ${{ needs.s3tests.result }}
+        run: |
+          set -uo pipefail
+          mkdir -p parity
+          for dir in parity-download/reference-parity-report-*; do
+            if [[ -d "$dir" ]]; then mv "$dir" "parity/${dir#parity-download/reference-parity-report-}"; fi
+          done
+          python3 scripts/s3-tests/reference_parity.py compare \
+            --targets-json "$TARGETS" \
+            --reports parity \
+            --tolerated .config/reference-parity-tolerated.txt \
+            --markdown "$GITHUB_STEP_SUMMARY" \
+            --json parity/diff.json > parity/diff.md
+          status=$?
+          cat parity/diff.md
+          if [[ "$S3TESTS_RESULT" != "success" ]]; then
+            echo "::error::s3-tests target jobs finished with '$S3TESTS_RESULT'"
+            status=1
+          fi
+          exit "$status"
+
+      - name: Upload parity diff
+        if: always()
+        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+        with:
+          name: reference-parity-diff
+          overwrite: true
+          path: |
+            parity/diff.md
+            parity/diff.json
+          if-no-files-found: warn
+          retention-days: 14
@@ -0,0 +1,133 @@
+# Copyright 2026 RustFS Team
+# SPDX-License-Identifier: Apache-2.0
+name: Release candidate (Linux)
+on:
+  workflow_dispatch:
+    inputs:
+      version:
+        description: Final product version (no preview suffix)
+        type: string
+        required: true
+      source_sha:
+        description: Full reviewed source commit
+        type: string
+        required: true
+permissions:
+  contents: read
+  actions: read
+concurrency:
+  group: release-candidate-${{ inputs.version }}
+  cancel-in-progress: false
+jobs:
+  freeze:
+    if: github.repository == 'rustfs/rustfs' && github.ref == 'refs/heads/main'
+    runs-on: ubuntu-latest
+    timeout-minutes: 15
+    outputs:
+      manifest: ${{ steps.freeze.outputs.manifest }}
+    steps:
+      - uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
+        with:
+          fetch-depth: 0
+          persist-credentials: false
+      - name: Freeze or recover original inputs
+        id: freeze
+        env:
+          GH_TOKEN: ${{ github.token }}
+          VERSION: ${{ inputs.version }}
+          SOURCE_SHA: ${{ inputs.source_sha }}
+        run: python3 scripts/release/pipeline.py prepare --version "$VERSION" --source "$SOURCE_SHA" --directory frozen
+      - uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+        if: steps.freeze.outputs.reused != 'true'
+        with:
+          name: release-inputs
+          path: frozen/inputs.json
+          if-no-files-found: error
+          retention-days: 30
+  linux:
+    needs: freeze
+    uses: ./.github/workflows/build.yml
+    with:
+      candidate_manifest: ${{ needs.freeze.outputs.manifest }}
+      platforms: linux
+  upgrade:
+    needs: [freeze, linux]
+    runs-on: ubuntu-latest
+    timeout-minutes: 150
+    env:
+      RUSTUP_TOOLCHAIN: ${{ fromJson(needs.freeze.outputs.manifest).toolchain }}
+    steps:
+      - uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
+        with:
+          ref: ${{ fromJson(needs.freeze.outputs.manifest).workflow_sha }}
+          persist-credentials: false
+      - uses: ./.github/actions/setup
+        with:
+          rust-version: ${{ fromJson(needs.freeze.outputs.manifest).toolchain }}
+          cache-shared-key: release-upgrade-harness-hosted
+          cache-save-if: ${{ github.ref == 'refs/heads/main' }}
+          install-build-packaging-tools: "false"
+          install-test-tools: "false"
+      - name: Verify archived server against previous releases
+        env:
+          GH_TOKEN: ${{ github.token }}
+          CANDIDATE_MANIFEST: ${{ needs.freeze.outputs.manifest }}
+        run: |
+          python3 - <<'PYVERIFY'
+          import sys
+          from pathlib import Path
+          sys.path.insert(0, 'scripts/release')
+          import candidate as c, pipeline, verify
+          inputs = pipeline.manifest_env()
+          run = pipeline.current_run('release-candidate.yml')
+          if any(a['name'] == 'release-upgrade' for a in c.artifact_list(run['id'])):
+              c.download_artifact(run, 'release-upgrade', 'receipts')
+              pipeline.verify_evidence(inputs, Path('receipts'), {run['id']: run})
+              c.emit_output('reused', 'true')
+          else:
+              target = 'linux-x86_64-gnu'
+              c.download_artifact(run, f"candidate-{inputs['candidate_id']}-{target}", 'binary')
+              verify.upgrade(inputs, Path('binary') / f"rustfs-{target}-v{inputs['version']}.zip", Path('receipts'))
+          PYVERIFY
+        id: verify
+      - uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+        if: steps.verify.outputs.reused != 'true'
+        with:
+          name: release-upgrade
+          path: receipts/*
+          if-no-files-found: error
+          retention-days: 30
+      - uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+        if: failure()
+        with:
+          name: release-upgrade-failure-${{ github.run_attempt }}
+          path: receipts/*
+          retention-days: 7
+  seal-linux:
+    needs: [freeze, linux, upgrade]
+    runs-on: ubuntu-latest
+    timeout-minutes: 30
+    steps:
+      - uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
+        with:
+          persist-credentials: false
+      - name: Collect trusted receipts and seal Linux bytes
+        id: seal
+        env:
+          GH_TOKEN: ${{ github.token }}
+        run: |
+          python3 scripts/release/pipeline.py linux-seal --directory sealed
+          {
+            echo '## Linux acceptance'
+            echo 'Use this run ID and manifest SHA-256 when starting Release candidate (complete).'
+            sha256sum sealed/linux-manifest.json
+            echo 'Record cluster, performance, integrity and upgrade acceptance in backlog before continuing.'
+          } >> "$GITHUB_STEP_SUMMARY"
+      - uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+        if: steps.seal.outputs.reused != 'true'
+        with:
+          name: release-linux
+          path: sealed/*
+          if-no-files-found: error
+          compression-level: 0
+          retention-days: 30
@@ -0,0 +1,97 @@
+# Copyright 2026 RustFS Team
+# SPDX-License-Identifier: Apache-2.0
+name: Release candidate (complete)
+on:
+  workflow_dispatch:
+    inputs:
+      candidate_run_id:
+        description: Successful Linux candidate run ID
+        type: string
+        required: true
+      linux_manifest_sha256:
+        description: Exact accepted Linux manifest SHA-256
+        type: string
+        required: true
+      acceptance_report:
+        description: Backlog report accepting cluster, performance, integrity and upgrade results
+        type: string
+        required: true
+permissions:
+  contents: read
+  actions: read
+concurrency:
+  group: release-complete-${{ inputs.candidate_run_id }}
+  cancel-in-progress: false
+jobs:
+  acceptance:
+    if: github.repository == 'rustfs/rustfs' && github.ref == 'refs/heads/main'
+    runs-on: ubuntu-latest
+    timeout-minutes: 30
+    environment: release-linux-acceptance
+    outputs:
+      manifest: ${{ steps.accept.outputs.manifest }}
+    steps:
+      - uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
+        with:
+          fetch-depth: 0
+          persist-credentials: false
+      - name: Bind acceptance to the exact Linux manifest
+        id: accept
+        env:
+          GH_TOKEN: ${{ github.token }}
+          CANDIDATE_RUN: ${{ inputs.candidate_run_id }}
+          MANIFEST_SHA: ${{ inputs.linux_manifest_sha256 }}
+          REPORT: ${{ inputs.acceptance_report }}
+        run: python3 scripts/release/pipeline.py complete-prepare --run-id "$CANDIDATE_RUN" --digest "$MANIFEST_SHA" --report "$REPORT" --directory linux
+  desktop:
+    needs: acceptance
+    uses: ./.github/workflows/build.yml
+    with:
+      candidate_manifest: ${{ needs.acceptance.outputs.manifest }}
+      platforms: desktop
+  packages:
+    needs: acceptance
+    uses: ./.github/workflows/package.yml
+    with:
+      candidate_manifest: ${{ needs.acceptance.outputs.manifest }}
+  images:
+    needs: acceptance
+    uses: ./.github/workflows/docker.yml
+    with:
+      candidate_manifest: ${{ needs.acceptance.outputs.manifest }}
+  chart:
+    needs: acceptance
+    uses: ./.github/workflows/helm-package.yml
+    with:
+      candidate_manifest: ${{ needs.acceptance.outputs.manifest }}
+  seal:
+    needs: [acceptance, desktop, packages, images, chart]
+    runs-on: ubuntu-latest
+    timeout-minutes: 45
+    steps:
+      - uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
+        with:
+          fetch-depth: 0
+          persist-credentials: false
+      - name: Verify every deliverable and seal the candidate
+        id: seal
+        env:
+          GH_TOKEN: ${{ github.token }}
+          CANDIDATE_RUN: ${{ inputs.candidate_run_id }}
+          MANIFEST_SHA: ${{ inputs.linux_manifest_sha256 }}
+          REPORT: ${{ inputs.acceptance_report }}
+        run: |
+          python3 scripts/release/pipeline.py complete-seal --run-id "$CANDIDATE_RUN" --digest "$MANIFEST_SHA" --report "$REPORT" --directory sealed
+          {
+            echo '## Final publication approval'
+            echo 'Review this manifest, then explicitly dispatch Publish accepted release with its SHA-256.'
+            sha256sum sealed/release-manifest.json
+          } >> "$GITHUB_STEP_SUMMARY"
+      - uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+        if: steps.seal.outputs.reused != 'true'
+        with:
+          name: release-seal
+          path: sealed/*
+          if-no-files-found: error
+          compression-level: 0
+          retention-days: 30
@@ -0,0 +1,193 @@
+# Copyright 2026 RustFS Team
+# SPDX-License-Identifier: Apache-2.0
+name: Publish accepted release
+on:
+  workflow_dispatch:
+    inputs:
+      complete_run_id:
+        description: Successful completed candidate run ID
+        type: string
+        required: true
+      manifest_sha256:
+        description: Exact approved final manifest SHA-256 (authorizes publication)
+        type: string
+        required: true
+permissions:
+  contents: read
+  actions: read
+concurrency:
+  group: release-publish-${{ inputs.complete_run_id }}
+  cancel-in-progress: false
+jobs:
+  verify:
+    if: github.repository == 'rustfs/rustfs' && github.ref == 'refs/heads/main'
+    runs-on: ubuntu-latest
+    timeout-minutes: 45
+    outputs:
+      version: ${{ steps.verify.outputs.version }}
+    steps:
+      - uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
+        with:
+          fetch-depth: 0
+          persist-credentials: false
+      - name: Recheck approved bytes and trusted certification
+        id: verify
+        env:
+          GH_TOKEN: ${{ github.token }}
+          COMPLETE_RUN: ${{ inputs.complete_run_id }}
+          MANIFEST_SHA: ${{ inputs.manifest_sha256 }}
+        run: python3 scripts/release/publish.py verify --run-id "$COMPLETE_RUN" --digest "$MANIFEST_SHA" --directory sealed
+  version:
+    needs: verify
+    runs-on: ubuntu-latest
+    timeout-minutes: 90
+    concurrency:
+      group: release-version-${{ needs.verify.outputs.version }}
+      cancel-in-progress: false
+    environment: release-publication
+    permissions:
+      contents: write
+      actions: read
+      packages: write
+      security-events: write
+    env:
+      GH_TOKEN: ${{ github.token }}
+      COMPLETE_RUN: ${{ inputs.complete_run_id }}
+      MANIFEST_SHA: ${{ inputs.manifest_sha256 }}
+      AWS_ACCESS_KEY_ID: ${{ secrets.R2_ACCESS_KEY_ID }}
+      AWS_SECRET_ACCESS_KEY: ${{ secrets.R2_SECRET_ACCESS_KEY }}
+      AWS_DEFAULT_REGION: auto
+      AWS_REQUEST_CHECKSUM_CALCULATION: when_required
+      AWS_RESPONSE_CHECKSUM_VALIDATION: when_required
+      AWS_EC2_METADATA_DISABLED: "true"
+      R2_ENDPOINT: ${{ secrets.R2_ENDPOINT }}
+      R2_BUCKET: ${{ secrets.R2_BUCKET }}
+    steps:
+      - uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
+        with:
+          fetch-depth: 0
+          persist-credentials: false
+      - name: Install promotion tools
+        run: |
+          sudo apt-get update
+          sudo apt-get install -y skopeo
+          aws --version
+      - uses: docker/login-action@c94ce9fb468520275223c153574b00df6fe4bcc9 # v3
+        with:
+          username: rustfs
+          password: ${{ secrets.DOCKERHUB_TOKEN }}
+      - uses: docker/login-action@c94ce9fb468520275223c153574b00df6fe4bcc9 # v3
+        with:
+          registry: ghcr.io
+          username: ${{ secrets.GHCR_USERNAME }}
+          password: ${{ secrets.GHCR_PASSWORD }}
+      - uses: docker/login-action@c94ce9fb468520275223c153574b00df6fe4bcc9 # v3
+        with:
+          registry: quay.io
+          username: ${{ secrets.QUAY_USERNAME }}
+          password: ${{ secrets.QUAY_PASSWORD }}
+      - name: Share registry credentials with the OCI copier
+        run: |
+          install -m 600 "$HOME/.docker/config.json" "$RUNNER_TEMP/registry-auth.json"
+          echo "REGISTRY_AUTH_FILE=$RUNNER_TEMP/registry-auth.json" >> "$GITHUB_ENV"
+      - name: Promote verified version
+        run: python3 scripts/release/publish.py version --run-id "$COMPLETE_RUN" --digest "$MANIFEST_SHA" --directory sealed
+      - name: Publish accepted musl amd64 scan
+        uses: github/codeql-action/upload-sarif@99df26d4f13ea111d4ec1a7dddef6063f76b97e9 # v4
+        with:
+          sarif_file: sealed/scan-musl-amd64.sarif
+          category: container-image-musl-amd64
+      - name: Publish accepted musl arm64 scan
+        uses: github/codeql-action/upload-sarif@99df26d4f13ea111d4ec1a7dddef6063f76b97e9 # v4
+        with:
+          sarif_file: sealed/scan-musl-arm64.sarif
+          category: container-image-musl-arm64
+      - name: Publish accepted GNU amd64 scan
+        uses: github/codeql-action/upload-sarif@99df26d4f13ea111d4ec1a7dddef6063f76b97e9 # v4
+        with:
+          sarif_file: sealed/scan-gnu-amd64.sarif
+          category: container-image-gnu-amd64
+      - name: Publish accepted GNU arm64 scan
+        uses: github/codeql-action/upload-sarif@99df26d4f13ea111d4ec1a7dddef6063f76b97e9 # v4
+        with:
+          sarif_file: sealed/scan-gnu-arm64.sarif
+          category: container-image-gnu-arm64
+      - uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+        with:
+          name: release-version-receipt-${{ github.run_attempt }}
+          path: publication.json
+          if-no-files-found: error
+          retention-days: 90
+  channels:
+    needs: [verify, version]
+    runs-on: ubuntu-latest
+    timeout-minutes: 90
+    concurrency:
+      group: rustfs-release-channels
+      cancel-in-progress: false
+    permissions:
+      contents: write
+      actions: read
+      packages: write
+    env:
+      GH_TOKEN: ${{ github.token }}
+      COMPLETE_RUN: ${{ inputs.complete_run_id }}
+      MANIFEST_SHA: ${{ inputs.manifest_sha256 }}
+      AWS_ACCESS_KEY_ID: ${{ secrets.R2_ACCESS_KEY_ID }}
+      AWS_SECRET_ACCESS_KEY: ${{ secrets.R2_SECRET_ACCESS_KEY }}
+      AWS_DEFAULT_REGION: auto
+      AWS_REQUEST_CHECKSUM_CALCULATION: when_required
+      AWS_RESPONSE_CHECKSUM_VALIDATION: when_required
+      AWS_EC2_METADATA_DISABLED: "true"
+      R2_ENDPOINT: ${{ secrets.R2_ENDPOINT }}
+      R2_BUCKET: ${{ secrets.R2_BUCKET }}
+    steps:
+      - uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
+        with:
+          fetch-depth: 0
+          persist-credentials: false
+      - name: Install promotion tools
+        run: |
+          sudo apt-get update
+          sudo apt-get install -y skopeo
+          aws --version
+      - uses: docker/login-action@c94ce9fb468520275223c153574b00df6fe4bcc9 # v3
+        with:
+          username: rustfs
+          password: ${{ secrets.DOCKERHUB_TOKEN }}
+      - uses: docker/login-action@c94ce9fb468520275223c153574b00df6fe4bcc9 # v3
+        with:
+          registry: ghcr.io
+          username: ${{ secrets.GHCR_USERNAME }}
+          password: ${{ secrets.GHCR_PASSWORD }}
+      - uses: docker/login-action@c94ce9fb468520275223c153574b00df6fe4bcc9 # v3
+        with:
+          registry: quay.io
+          username: ${{ secrets.QUAY_USERNAME }}
+          password: ${{ secrets.QUAY_PASSWORD }}
+      - name: Share registry credentials with the OCI copier
+        run: |
+          install -m 600 "$HOME/.docker/config.json" "$RUNNER_TEMP/registry-auth.json"
+          echo "REGISTRY_AUTH_FILE=$RUNNER_TEMP/registry-auth.json" >> "$GITHUB_ENV"
+      - uses: azure/setup-helm@b9e51907a09c216f16ebe8536097933489208112 # v4.3.0
+      - name: Install pinned OSS client
+        run: |
+          curl --fail --location --retry 3 --output "$RUNNER_TEMP/ossutil.zip" https://gosspublic.alicdn.com/ossutil/v2/2.1.1/ossutil-2.1.1-linux-amd64.zip
+          printf '%s  %s\n' 60f3a808cbbdfd4b5269b8f967c6a66f564c9dacff52d93a5d21a588976ab5a2 "$RUNNER_TEMP/ossutil.zip" | sha256sum --check -
+          unzip -q "$RUNNER_TEMP/ossutil.zip" -d "$RUNNER_TEMP/ossutil-extract"
+          install -m 755 "$RUNNER_TEMP/ossutil-extract/ossutil-2.1.1-linux-amd64/ossutil" "$RUNNER_TEMP/ossutil"
+          echo "OSSUTIL_BIN=$RUNNER_TEMP/ossutil" >> "$GITHUB_ENV"
+      - name: Promote verified channels
+        env:
+          HELM_TOKEN: ${{ secrets.RUSTFS_HELM_PACKAGE }}
+          OSS_ACCESS_KEY_ID: ${{ secrets.ALICLOUDOSS_KEY_ID }}
+          OSS_ACCESS_KEY_SECRET: ${{ secrets.ALICLOUDOSS_KEY_SECRET }}
+          OSS_REGION: cn-beijing
+          OSS_ENDPOINT: https://oss-cn-beijing.aliyuncs.com
+        run: python3 scripts/release/publish.py channels --run-id "$COMPLETE_RUN" --digest "$MANIFEST_SHA" --directory sealed
+      - uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+        with:
+          name: release-channels-receipt-${{ github.run_attempt }}
+          path: publication.json
+          if-no-files-found: error
+          retention-days: 90
@@ -26,7 +26,13 @@ permissions:
 jobs:
  alert-on-incomplete-run:
    name: Alert on incomplete scheduled run
-    if: github.repository == 'rustfs/rustfs'
+    # The registry still owns responsibility and cadence selection. Successful
+    # and PR runs can never produce an incomplete scheduled-run alert.
+    if: >-
+      github.repository == 'rustfs/rustfs' &&
+      github.event.workflow_run.conclusion != 'success' &&
+      (github.event.workflow_run.event == 'schedule' ||
+       github.event.workflow_run.event == 'workflow_run')
    # Stay on GitHub-hosted runners: this monitor must keep running when the
    # self-hosted sm-standard-* pool is down, which is exactly when the
    # scheduled validations it watches stall.
@@ -155,7 +155,7 @@ default build (lifecycle:
   - ⚠️ PARTIALLY VIOLATED: serving-side `s3s` references remain in ecstore
     (bucket metadata/replication/lifecycle DTOs and error mapping). The
     count is ratcheted shrink-only by `scripts/check_s3s_footprint.sh`
-     (`S3S_ECSTORE_FILES_BASELINE`; the `object_lock` module was converted to
+     (the `ecstore_files` line of `.config/s3s-footprint-baseline.txt`; the `object_lock` module was converted to
     storage-level types as the first ratchet step). Target state: the
     baseline reaches zero and ecstore's `Cargo.toml` drops `s3s`.

@@ -7,6 +7,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

 ## [Unreleased]

+### Notifications and audit
+
+- Notify and audit targets that keep the default `queue_dir` now work on native Windows: the defaults `/opt/rustfs/events` and `/opt/rustfs/audit` resolve per node, to `%ProgramData%\RustFS\events` and `%ProgramData%\RustFS\audit` on Windows, instead of failing validation as non-absolute paths. The new `RUSTFS_TARGET_STORE_DIR` (Helm `config.rustfs.target_store_dir`) relocates both default queues on any platform. Linux and macOS behavior is unchanged when it is unset, persisted configuration is unchanged, and pending entries are not migrated when the root changes. See `docs/operations/target-queue-storage.md`.
+
 ### Deployment

 - Existing volumes created by older root-running container images must be writable by the non-root runtime identity before upgrading. The default container UID/GID is `10001:10001`; stop the affected instances, verify host mount paths, and for ordinary bind mounts run `sudo chown -R 10001:10001 /actual/data /actual/logs` on each RustFS data/log mount. Account for ACLs, NFS, Kubernetes, and rootless UID mappings. Image-build ownership changes do not migrate mounted data. See the Docker Quick Start in [README.md](README.md).
@@ -33,6 +37,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - **Presigned URLs honour only signed headers** (GHSA-g8w9-qw9q-fghr): a SigV4 presigned request that carries an `x-amz-*` request header not listed in `X-Amz-SignedHeaders` is now rejected with `403 AccessDenied` ("There were headers present in the request which were not signed"), matching AWS S3. Previously the holder of a presigned `PutObject` URL could add unsigned `x-amz-tagging`, `x-amz-storage-class`, `x-amz-website-redirect-location`, ACL, metadata, Object Lock or SSE headers and have them applied. Presigners that intend a property must set it before signing so the SDK lists the header in `SignedHeaders`; `x-amz-cf-id` (CloudFront) remains tolerated unsigned. Header-signed SigV4 and SigV2 requests are unchanged.

 ### Fixed
+- **Immediate KMS key deletion no longer reports success without destroying the key** (behavior change): with `RUSTFS_KMS_ALLOW_IMMEDIATE_DELETION=true`, a `DELETE /rustfs/admin/v3/kms/keys/delete` request with `force_immediate` against an `Enabled` or `Disabled` key on the Vault KV2 or Vault Transit backend returned success but only moved the key to `PendingDeletion`; the key material stayed in Vault and objects encrypted under it remained readable, while the Local backend destroyed the key in that one request. Key destruction is now the same two explicit steps on every backend: schedule the deletion, then either let the waiting window elapse or send `force_immediate` to destroy the pending key early. `force_immediate` against a key that is not `PendingDeletion` is refused with `400` and changes nothing, so a success response always means the key is gone and no single request can destroy a key in use. Automation that destroyed Local keys with one `force_immediate` request must schedule the deletion first. On Vault Transit, a read under a key that no longer exists in Vault now fails with `KMS.NotFoundException` instead of `503 ServiceUnavailable`. No persisted format changes. See `docs/operations/kms-key-destruction.md`.
 - **`mc admin info` and `mc admin replicate resync` work again**: since the cluster snapshot view (#3803), `GET /v3/info` and `GET /v3/storageinfo` returned the madmin `InfoMessage` / `StorageInfo` nested under an `info` key next to `admin_discovery` (and, for `/info`, `bitrotSelftest`). madmin decodes those bodies directly, so `mc admin info` printed empty values and `mc admin replicate resync start/status` rejected every alias with "alias provided is not part of cluster replication" because the local `deploymentID` read as empty. Requests on the MinIO-compatible `/minio/admin/v3` prefix now receive the bare madmin body again; `/rustfs/admin/v3`, which rc and the console use, keeps the envelope unchanged.
 - **Replication status writebacks no longer invalidate scanner cycles**: every replication status writeback (`persist_replication_state_if_current` in `Update` mode, including the heals the scanner queues during its own walk) marked the bucket dirty on the namespace path, advanced the dirty-usage generation and woke the scanner, so a site-replication resync or any steadily replicated bucket ended every cycle `superseded`, the authoritative usage baseline was never certified, and the console reported usage freshness as stale for as long as replication ran (#8375). Those writebacks are now status class: they do not touch the dirty-usage generation, the activity snapshot, the dirty-usage wake or the durable dirty journal, and only the process-local producer coverage is recorded. They need no rescan queue, because a deployment with an active replication rule already walks every bucket on every cycle and that walk refreshes `replication_stats`. Tier-expiration cleanup, which removes data, stays on the namespace path. No persisted format, protocol, admin API or configuration changes. Design: `docs/architecture/scanner-per-bucket-convergence.md`.
 - **A bucket written during a scanner walk no longer supersedes the whole cycle**: the post-walk cluster activity proof required the entire activity snapshot and the dirty-usage generation to be unchanged, so a single-node deployment with one bucket written every few seconds never reached a `Complete` cycle, `usage_snapshot_converged` never became true and the authoritative usage baseline was never certified (#8494). Dirty usage is now certified per bucket: a structurally complete walk that the guardrail did not veto is `Complete`, buckets whose dirty generation still matches the start snapshot are cleared, and buckets written during the walk stay dirty and are published as the pending set, so `usage_snapshot_converged` means "no bucket pending" and the next cycle is scoped to the pending buckets. Such a cycle writes the authoritative usage object; the observed object is now selected by an explicit observational mark that vetoed cycles carry. The guardrail itself vetoes only on the fields that change which namespace the walk described (peer set, process instance, protocol version, topology, maintenance generation, data movement, movement generation, publication blocked) and reports the field and host in the scanner log and in the new `usage_freshness` fields `guardrail_veto_total`, `last_guardrail_veto_unix_secs` and `last_guardrail_veto_reason`. In a distributed deployment a remote peer whose dirty usage moved during the walk still supersedes the cycle until scoped acknowledgements can name its buckets. `.usage.v2.json` gains the additive, default-when-absent fields `usage_snapshot_pending_buckets`, `usage_snapshot_pending_bucket_count` and `usage_snapshot_observational`; older readers ignore them and treat the snapshot as an authoritative but unconverged publication. Quota is unchanged: an uncertified bucket keeps its last usage value as the floor and live writes do not advance it. No protocol or configuration changes. Design: `docs/architecture/scanner-per-bucket-convergence.md`.
@@ -26,6 +26,7 @@ members = [
    "crates/ecstore", # Erasure coding storage implementation
    "crates/e2e_test", # End-to-end test suite
    "crates/filemeta", # File metadata management
+    "crates/gateway-adapter", # RustFS implementations of the gateway extension traits
    "crates/heal", # Erasure set and object healing
    "crates/heal-contracts", # Heal request/response channel contracts
    "crates/iam", # Identity and Access Management
@@ -48,7 +49,7 @@ members = [
    "crates/replication", # Replication contracts and wire formats
    "crates/concurrency", # Concurrency management for RustFS - timeout, locking, backpressure, and I/O scheduling
    "crates/s3-client", # S3 client for engine-side consumption of remote S3 endpoints (tiering, transition targets)
-    "crates/s3-types", # S3 event type definitions
+    "crates/s3-types", # S3 event and error type definitions
    "crates/s3-ops", # S3 operation definitions and mapping
    "crates/s3select-api", # S3 Select API interface
    "crates/s3select-query", # S3 Select query engine
@@ -75,7 +76,7 @@ edition = "2024"
 license = "Apache-2.0"
 repository = "https://github.com/rustfs/rustfs"
 rust-version = "1.99.0"
-version = "1.0.2"
+version = "1.0.5"
 homepage = "https://rustfs.com"
 description = "RustFS is a high-performance distributed object storage software built using Rust, one of the most popular languages worldwide. "
 keywords = ["RustFS", "Minio", "object-storage", "filesystem", "s3"]
@@ -92,56 +93,70 @@ redundant_clone = "warn"

 [workspace.dependencies]
 # RustFS Internal Crates
-rustfs = { path = "./rustfs", version = "1.0.2" }
-rustfs-heal = { path = "crates/heal", version = "1.0.2" }
-rustfs-heal-contracts = { path = "crates/heal-contracts", version = "1.0.2" }
-rustfs-scanner-metrics = { path = "crates/scanner-metrics", version = "1.0.2" }
-rustfs-audit = { path = "crates/audit", version = "1.0.2" }
-rustfs-checksums = { path = "crates/checksums", version = "1.0.2" }
-rustfs-common = { path = "crates/common", version = "1.0.2" }
-rustfs-data-usage = { path = "crates/data-usage", version = "1.0.2" }
-rustfs-config = { path = "./crates/config", version = "1.0.2" }
-rustfs-concurrency = { path = "./crates/concurrency", version = "1.0.2" }
-rustfs-credentials = { path = "crates/credentials", version = "1.0.2" }
-rustfs-crypto = { path = "crates/crypto", version = "1.0.2" }
-rustfs-ecstore = { path = "crates/ecstore", version = "1.0.2" }
-rustfs-filemeta = { path = "crates/filemeta", version = "1.0.2" }
-rustfs-iam = { path = "crates/iam", version = "1.0.2" }
-rustfs-keystone = { path = "crates/keystone", version = "1.0.2" }
-rustfs-license = { path = "crates/license", version = "1.0.2" }
-rustfs-lifecycle = { path = "crates/lifecycle", version = "1.0.2" }
-rustfs-kms = { path = "crates/kms", version = "1.0.2" }
-rustfs-lock = { path = "crates/lock", version = "1.0.2" }
-rustfs-madmin = { path = "crates/madmin", version = "1.0.2" }
-rustfs-notify = { path = "crates/notify", version = "1.0.2" }
-rustfs-io-metrics = { path = "crates/io-metrics", version = "1.0.2" }
-rustfs-io-core = { path = "crates/io-core", version = "1.0.2" }
-rustfs-object-capacity = { path = "crates/object-capacity", version = "1.0.2" }
-rustfs-object-data-cache = { path = "crates/object-data-cache", version = "1.0.2", default-features = false }
-rustfs-log-analyzer = { path = "crates/log-analyzer", version = "1.0.2" }
-rustfs-obs = { path = "crates/obs", version = "1.0.2" }
-rustfs-policy = { path = "crates/policy", version = "1.0.2" }
-rustfs-protos = { path = "crates/protos", version = "1.0.2" }
-rustfs-protocols = { path = "crates/protocols", version = "1.0.2" }
-rustfs-replication = { path = "crates/replication", version = "1.0.2" }
-rustfs-rio = { path = "crates/rio", version = "1.0.2" }
-rustfs-rio-v2 = { path = "crates/rio-v2", version = "1.0.2" }
-rustfs-s3-client = { path = "crates/s3-client", version = "1.0.2" }
-rustfs-s3-types = { path = "crates/s3-types", version = "1.0.2" }
-rustfs-s3-ops = { path = "crates/s3-ops", version = "1.0.2" }
-rustfs-s3select-api = { path = "crates/s3select-api", version = "1.0.2" }
-rustfs-s3select-query = { path = "crates/s3select-query", version = "1.0.2" }
-rustfs-scanner = { path = "crates/scanner", version = "1.0.2" }
-rustfs-security-governance = { path = "crates/security-governance", version = "1.0.2" }
-rustfs-extension-schema = { path = "crates/extension-schema", version = "1.0.2" }
-rustfs-signer = { path = "crates/signer", version = "1.0.2" }
-rustfs-storage-api = { path = "crates/storage-api", version = "1.0.2" }
-rustfs-trusted-proxies = { path = "crates/trusted-proxies", version = "1.0.2" }
-rustfs-targets = { path = "crates/targets", version = "1.0.2" }
-rustfs-test-utils = { path = "crates/test-utils", version = "1.0.2" }
-rustfs-tls-runtime = { path = "crates/tls-runtime", version = "1.0.2" }
-rustfs-utils = { path = "crates/utils", version = "1.0.2" }
-rustfs-zip = { path = "./crates/zip", version = "1.0.2" }
+rustfs = { path = "./rustfs", version = "1.0.5" }
+rustfs-gateway-adapter = { path = "crates/gateway-adapter", version = "1.0.5" }
+rustfs-heal = { path = "crates/heal", version = "1.0.5" }
+rustfs-heal-contracts = { path = "crates/heal-contracts", version = "1.0.5" }
+rustfs-scanner-metrics = { path = "crates/scanner-metrics", version = "1.0.5" }
+rustfs-audit = { path = "crates/audit", version = "1.0.5" }
+rustfs-checksums = { path = "crates/checksums", version = "1.0.5" }
+rustfs-common = { path = "crates/common", version = "1.0.5" }
+rustfs-data-usage = { path = "crates/data-usage", version = "1.0.5" }
+rustfs-config = { path = "./crates/config", version = "1.0.5" }
+rustfs-concurrency = { path = "./crates/concurrency", version = "1.0.5" }
+rustfs-credentials = { path = "crates/credentials", version = "1.0.5" }
+rustfs-crypto = { path = "crates/crypto", version = "1.0.5" }
+rustfs-ecstore = { path = "crates/ecstore", version = "1.0.5" }
+rustfs-filemeta = { path = "crates/filemeta", version = "1.0.5" }
+rustfs-iam = { path = "crates/iam", version = "1.0.5" }
+rustfs-keystone = { path = "crates/keystone", version = "1.0.5" }
+rustfs-license = { path = "crates/license", version = "1.0.5" }
+rustfs-lifecycle = { path = "crates/lifecycle", version = "1.0.5" }
+rustfs-kms = { path = "crates/kms", version = "1.0.5" }
+rustfs-lock = { path = "crates/lock", version = "1.0.5" }
+rustfs-madmin = { path = "crates/madmin", version = "1.0.5" }
+rustfs-notify = { path = "crates/notify", version = "1.0.5" }
+rustfs-io-metrics = { path = "crates/io-metrics", version = "1.0.5" }
+rustfs-io-core = { path = "crates/io-core", version = "1.0.5" }
+rustfs-object-capacity = { path = "crates/object-capacity", version = "1.0.5" }
+rustfs-object-data-cache = { path = "crates/object-data-cache", version = "1.0.5", default-features = false }
+rustfs-log-analyzer = { path = "crates/log-analyzer", version = "1.0.5" }
+rustfs-obs = { path = "crates/obs", version = "1.0.5" }
+rustfs-policy = { path = "crates/policy", version = "1.0.5" }
+rustfs-protos = { path = "crates/protos", version = "1.0.5" }
+rustfs-protocols = { path = "crates/protocols", version = "1.0.5" }
+rustfs-replication = { path = "crates/replication", version = "1.0.5" }
+rustfs-rio = { path = "crates/rio", version = "1.0.5" }
+rustfs-rio-v2 = { path = "crates/rio-v2", version = "1.0.5" }
+rustfs-s3-client = { path = "crates/s3-client", version = "1.0.5" }
+rustfs-s3-types = { path = "crates/s3-types", version = "1.0.5" }
+rustfs-s3-ops = { path = "crates/s3-ops", version = "1.0.5" }
+rustfs-s3select-api = { path = "crates/s3select-api", version = "1.0.5" }
+rustfs-s3select-query = { path = "crates/s3select-query", version = "1.0.5" }
+rustfs-scanner = { path = "crates/scanner", version = "1.0.5" }
+rustfs-security-governance = { path = "crates/security-governance", version = "1.0.5" }
+rustfs-extension-schema = { path = "crates/extension-schema", version = "1.0.5" }
+rustfs-signer = { path = "crates/signer", version = "1.0.5" }
+rustfs-storage-api = { path = "crates/storage-api", version = "1.0.5" }
+rustfs-trusted-proxies = { path = "crates/trusted-proxies", version = "1.0.5" }
+rustfs-targets = { path = "crates/targets", version = "1.0.5" }
+rustfs-test-utils = { path = "crates/test-utils", version = "1.0.5" }
+rustfs-tls-runtime = { path = "crates/tls-runtime", version = "1.0.5" }
+rustfs-utils = { path = "crates/utils", version = "1.0.5" }
+rustfs-zip = { path = "./crates/zip", version = "1.0.5" }
+
+# The gateway's S3 types: the DTOs and persisted bucket-configuration codecs the
+# engine crates move onto from s3s (rustfs/backlog#2745). The one pin of the
+# gateway in this workspace; no crate may enable a `compat-s3s*` feature except
+# the s3s-facing edge of the rustfs binary.
+rustfs-gateway-types = { git = "https://github.com/rustfs/gateway.git", rev = "7915d06950f88611d883f6ab3d95efc90c65dec0", version = "0.40.0", default-features = false }
+rustfs-gateway-stream = { git = "https://github.com/rustfs/gateway.git", rev = "7915d06950f88611d883f6ab3d95efc90c65dec0", version = "0.2.0", default-features = false }
+# The gateway framework itself, consumed for its extension traits by the ring-2
+# adapter. `default-features = false` keeps the `server` feature, and with it the
+# server crate, out of the build.
+rustfs-gateway = { git = "https://github.com/rustfs/gateway.git", rev = "7915d06950f88611d883f6ab3d95efc90c65dec0", version = "0.62.0", default-features = false }
+# Test-only: builds the `EffectiveHost` a host-resolution test queries with.
+rustfs-gateway-http = { git = "https://github.com/rustfs/gateway.git", rev = "7915d06950f88611d883f6ab3d95efc90c65dec0", default-features = false }

 # Async Runtime and Networking
 async-channel = "2.5.0"
@@ -149,7 +164,7 @@ async_zip = { default-features = false, version = "0.0.19" }
 mysql_async = { default-features = false, version = "0.37.1" }
 async-compression = { version = "0.4.52" }
 async-trait = "0.1.92"
-async-nats = { version = "0.51.0", default-features = false }
+async-nats = { version = "0.52.0", default-features = false }
 axum = "0.8.9"
 futures = "0.3.34"
 futures-core = "0.3.34"
@@ -249,8 +264,8 @@ anyhow = "1.0.104"
 arc-swap = "1.9.3"
 rustfs-tokio-tar = { version = "0.7.1" }
 # Candidate Snowball parser versions exercised by rustfs-zip compatibility fixtures.
-tar-codec = "0.0.15"
-tar-framing = "0.0.15"
+tar-codec = "0.0.16"
+tar-framing = "0.0.16"
 atoi = "3.1.1"
 atomic_enum = "0.3.0"
 aws-config = { version = "1.13.0" }
@@ -329,6 +344,10 @@ rust-embed = { version = "8.13.0" }
 rustc-hash = { version = "2.1.3" }
 s3s = { git = "https://github.com/s3s-project/s3s", rev = "5761ddfe4c509fcac87e3b99fd5bd9e176b442c0", features = ["minio"] }
 s3s-sigv4 = { git = "https://github.com/s3s-project/s3s", rev = "5761ddfe4c509fcac87e3b99fd5bd9e176b442c0" }
+# The same s3s pin under the name the legacy edge of the gateway migration uses
+# (crates/s3-types, feature `compat-s3s`). Keep `rev` identical to `s3s` above.
+# DELETE BY T4.2 (rustfs/backlog#2784).
+legacy-s3s = { package = "s3s", git = "https://github.com/s3s-project/s3s", rev = "5761ddfe4c509fcac87e3b99fd5bd9e176b442c0", features = ["minio"] }
 serial_test = "4.0.1"
 shadow-rs = { default-features = false, version = "2.0.0" }
 siphasher = "1.0.4"
@@ -337,6 +356,7 @@ compact_str = "0.10.0"
 snap = "1.1.2"
 starshard = { version = "2.4.0" }
 strum = { version = "0.28.0" }
+sync_wrapper = "1.0.2"
 sysinfo = "0.40.1"
 temp-env = "0.3.6"
 tempfile = "3.27.0"
@@ -389,7 +409,7 @@ dav-server = "0.11.0"
 # Performance Analysis and Memory Profiling
 rustfs-mimalloc = { version = "0.7.3" }
 # Preserve Unicode focus filters until rustfs/backlog#2302 is resolved.
-hotpath = { version = "0.30.3", default-features = false }
+hotpath = { version = "0.31.1", default-features = false }
 # Snapshot testing for output format regression detection
 insta = { version = "1.49" }

@@ -12,7 +12,9 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.

-FROM alpine:3.24.2 AS build
+ARG BINARY_SOURCE=download
+
+FROM alpine:3.24.2 AS download

 ARG TARGETARCH
 ARG RELEASE=latest
@@ -74,6 +76,14 @@ COPY scripts/download_rc.sh /usr/local/bin/download_rc.sh
 RUN sh /usr/local/bin/download_rc.sh "$TARGETARCH" musl "$RC_RELEASE" /build/rc


+# Candidate packaging supplies the accepted executables from verified archives.
+# The default download stage remains available for local image builds.
+FROM scratch AS supplied
+ARG TARGETARCH
+COPY .release-inputs/${TARGETARCH}/ /build/
+
+FROM ${BINARY_SOURCE} AS build
+
 FROM alpine:3.24.2

 ARG RELEASE=latest
@@ -103,7 +113,6 @@ RUN apk upgrade --no-cache && \
    tzdata \
    && test "$(TZ=Asia/Kolkata date +%z)" = "+0530"

-COPY --from=build /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/
 COPY --from=build /build/rustfs /usr/bin/rustfs
 COPY --from=build /build/rc /usr/bin/rc
 COPY entrypoint.sh /etc/rustfs/entrypoint.sh
@@ -12,7 +12,9 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.

-FROM ubuntu:26.04 AS build
+ARG BINARY_SOURCE=download
+
+FROM ubuntu:26.04 AS download

 ARG TARGETARCH
 ARG RELEASE=latest
@@ -80,6 +82,14 @@ RUN set -eux; \
 COPY scripts/download_rc.sh /usr/local/bin/download_rc.sh
 RUN sh /usr/local/bin/download_rc.sh "$TARGETARCH" gnu "$RC_RELEASE" /build/rc

+# Candidate packaging supplies the accepted executables from verified archives.
+# The default download stage remains available for local image builds.
+FROM scratch AS supplied
+ARG TARGETARCH
+COPY .release-inputs/${TARGETARCH}/ /build/
+
+FROM ${BINARY_SOURCE} AS build
+
 FROM ubuntu:26.04

 ARG RELEASE=latest
@@ -24,9 +24,10 @@
 #
 # Supports cross-compilation for amd64 and arm64 via TARGETPLATFORM.

-ARG TARGETPLATFORM
-ARG BUILDPLATFORM
-ARG TARGETARCH
+# Do not declare TARGETPLATFORM, BUILDPLATFORM or TARGETARCH here: a global
+# declaration without a value shadows BuildKit's automatic values, so the
+# stage-level re-declarations below see empty strings. They are only
+# declared inside the stage that uses them.
 ARG RUSTFS_BUILD_FEATURES=""
 ARG RC_RELEASE=latest

@@ -169,7 +170,24 @@ RUN --mount=type=cache,target=/usr/local/cargo/registry \
    ;; \
    esac

-RUN sh /usr/src/rustfs/scripts/download_rc.sh "$TARGETARCH" musl "$RC_RELEASE" /usr/local/bin/rc
+# TARGETARCH is only provided by BuildKit. Fall back to TARGETPLATFORM (for
+# example set by docker-compose) and then to the build host, like the cargo
+# step above, so every path that builds this file passes a real architecture.
+RUN set -eu; \
+    rc_arch="${TARGETARCH:-}"; \
+    if [ -z "${rc_arch}" ]; then \
+    case "${TARGETPLATFORM:-}" in \
+    linux/amd64) rc_arch=amd64 ;; \
+    linux/arm64) rc_arch=arm64 ;; \
+    *) \
+    case "$(uname -m)" in \
+    x86_64) rc_arch=amd64 ;; \
+    aarch64|arm64) rc_arch=arm64 ;; \
+    *) echo "TARGETARCH is empty and TARGETPLATFORM='${TARGETPLATFORM:-}' is not linux/amd64 or linux/arm64; cannot choose an rc architecture on $(uname -m)" >&2; exit 1 ;; \
+    esac ;; \
+    esac; \
+    fi; \
+    sh /usr/src/rustfs/scripts/download_rc.sh "${rc_arch}" musl "${RC_RELEASE}" /usr/local/bin/rc

 # -----------------------------
 # Development stage (keeps toolchain)
@@ -145,7 +145,7 @@ chown -R 10001:10001 data logs
 docker run -d -p 9000:9000 -p 9001:9001 -v $(pwd)/data:/data -v $(pwd)/logs:/logs rustfs/rustfs:latest

 # Using specific version
-docker run -d -p 9000:9000 -p 9001:9001 -v $(pwd)/data:/data -v $(pwd)/logs:/logs rustfs/rustfs:1.0.4
+docker run -d -p 9000:9000 -p 9001:9001 -v $(pwd)/data:/data -v $(pwd)/logs:/logs rustfs/rustfs:1.0.5
 ```

 If you use [podman](https://github.com/containers/podman) instead of docker, you can install the RustFS with the below command
@@ -199,7 +199,7 @@ docker run -d --name rustfs -p 9000:9000 \
 Notes:
 - `RUSTFS_NOTIFY_ENABLE=true` enables the global notify module switch.
 - For ARN `arn:rustfs:sqs::primary:webhook`, use instance-scoped env vars with `_PRIMARY`.
- If queue dir is omitted, the official image maps `/opt/rustfs/events` into the persistent `/data` volume so it is writable by the runtime user and pending events survive container recreation. Override `queue_dir` when using another deployment layout.
+- If queue dir is omitted, the official image maps `/opt/rustfs/events` into the persistent `/data` volume so it is writable by the runtime user and pending events survive container recreation. Override `queue_dir` when using another deployment layout, or set `RUSTFS_TARGET_STORE_DIR` to move every default queue; native Windows defaults to `%ProgramData%\RustFS`. See [Target queue storage](docs/operations/target-queue-storage.md).
 - `RUSTFS_NOTIFY_WEBHOOK_SKIP_TLS_VERIFY_PRIMARY` defaults to `false`; enabling it skips webhook TLS certificate verification, allows MITM attacks, and emits a startup warning. Prefer `RUSTFS_NOTIFY_WEBHOOK_CLIENT_CA_PRIMARY` for private CAs.
 - Since `1.0.0-beta.11`, webhook endpoints on private or container networks
  (`Docker Compose service names`, `host.docker.internal`, RFC 1918 addresses) are
@@ -224,7 +224,7 @@ For developers who want to build RustFS Docker images from source with multi-arc
 ./docker-buildx.sh --push

 # Build specific version
-./docker-buildx.sh --release 1.0.4 --push
+./docker-buildx.sh --release 1.0.5 --push

 # Build for custom registry
 ./docker-buildx.sh --registry your-registry.com --namespace yourname --push
@@ -242,7 +242,7 @@ You can also use Make targets for convenience:
 ```bash
 make docker-buildx                    # Build locally
 make docker-buildx-push               # Build and push
-make docker-buildx-version VERSION=1.0.4  # Build specific version
+make docker-buildx-version VERSION=1.0.5  # Build specific version
 make help-docker                      # Show all Docker-related commands
 ```

@@ -138,7 +138,7 @@ chown -R 10001:10001 data logs
 docker run -d -p 9000:9000 -p 9001:9001 -v $(pwd)/data:/data -v $(pwd)/logs:/logs rustfs/rustfs:latest

 # 使用指定版本运行
-docker run -d -p 9000:9000 -p 9001:9001 -v $(pwd)/data:/data -v $(pwd)/logs:/logs rustfs/rustfs:1.0.4
+docker run -d -p 9000:9000 -p 9001:9001 -v $(pwd)/data:/data -v $(pwd)/logs:/logs rustfs/rustfs:1.0.5
 ```

 如果您通过绑定挂载启用 TLS 证书目录，也请用同样方式准备该目录：
@@ -176,7 +176,7 @@ docker compose -f docker-compose-simple.yml up -d
 ./docker-buildx.sh --push

 # 构建指定版本
-./docker-buildx.sh --release 1.0.4 --push
+./docker-buildx.sh --release 1.0.5 --push

 # 构建并推送到自定义仓库
 ./docker-buildx.sh --registry your-registry.com --namespace yourname --push
@@ -193,7 +193,7 @@ docker compose -f docker-compose-simple.yml up -d
 ```bash
 make docker-buildx                    # 本地构建
 make docker-buildx-push               # 构建并推送
-make docker-buildx-version VERSION=1.0.4  # 构建指定版本
+make docker-buildx-version VERSION=1.0.5  # 构建指定版本
 make help-docker                      # 显示所有 Docker 相关命令
 ```

@@ -902,4 +902,104 @@ mod tests {
        assert_eq!(raw.len(), 8);
        assert_eq!(&raw[..], reference.digest().to_be_bytes().as_slice());
    }
+
+    #[test]
+    fn checksums_streaming_chunks_and_header_encoding_match_oneshot_across_unaligned_boundaries() {
+        use crate::http::HttpChecksum;
+
+        let all = [
+            ChecksumAlgorithm::Crc32,
+            ChecksumAlgorithm::Crc32c,
+            ChecksumAlgorithm::Crc64Nvme,
+            ChecksumAlgorithm::Sha1,
+            ChecksumAlgorithm::Sha256,
+            ChecksumAlgorithm::Sha512,
+            ChecksumAlgorithm::Xxhash3,
+            ChecksumAlgorithm::Xxhash64,
+            ChecksumAlgorithm::Xxhash128,
+        ];
+
+        for &payload_len in &[0usize, 1, 31, 33, 1024, 4096 + 37, 64 * 1024 + 19] {
+            let payload: Vec<u8> = (0..payload_len).map(|i| ((i * 19 + 5) % 253) as u8).collect();
+            for algorithm in all {
+                let mut one_shot = algorithm.into_impl();
+                one_shot.update(&payload);
+                let expected_raw = one_shot.finalize();
+                assert_eq!(expected_raw.len(), algorithm.raw_len());
+
+                let mut one_shot_hdr = algorithm.into_impl();
+                one_shot_hdr.update(&payload);
+                let expected_headers = one_shot_hdr.headers();
+
+                for &chunk_size in &[1usize, 31, 1021, 4096] {
+                    if payload_len > 4096 && chunk_size == 1 {
+                        continue;
+                    }
+                    let mut streaming: Box<dyn HttpChecksum> = algorithm.into_impl();
+                    for chunk in payload.chunks(chunk_size) {
+                        streaming.update(chunk);
+                    }
+                    let actual_headers = streaming.headers();
+                    assert_eq!(
+                        actual_headers.get(algorithm.http_header_name()),
+                        expected_headers.get(algorithm.http_header_name()),
+                        "{algorithm:?} streaming mismatch at payload_len={payload_len}, chunk_size={chunk_size}"
+                    );
+                }
+            }
+        }
+    }
+
+    #[test]
+    #[ignore = "utility throughput microbenchmark; run via cargo test -p rustfs-checksums --lib --release -- --ignored --nocapture"]
+    fn microbench_checksums_streaming_chunks_and_header_encoding() {
+        use crate::http::HttpChecksum;
+        use std::hint::black_box;
+        use std::time::Instant;
+
+        let all = [
+            ChecksumAlgorithm::Crc32,
+            ChecksumAlgorithm::Crc32c,
+            ChecksumAlgorithm::Crc64Nvme,
+            ChecksumAlgorithm::Sha1,
+            ChecksumAlgorithm::Sha256,
+            ChecksumAlgorithm::Sha512,
+            ChecksumAlgorithm::Xxhash3,
+            ChecksumAlgorithm::Xxhash64,
+            ChecksumAlgorithm::Xxhash128,
+        ];
+
+        let payload: Vec<u8> = (0..(256 * 1024)).map(|i| ((i * 19 + 5) % 253) as u8).collect();
+        for &chunk_size in &[4 * 1024usize, 64 * 1024, 256 * 1024] {
+            for algorithm in all {
+                let mut one_shot = algorithm.into_impl();
+                one_shot.update(&payload);
+                let expected_headers = one_shot.headers();
+
+                let iters = 8usize;
+                let t0 = Instant::now();
+                let mut last_headers = http::HeaderMap::new();
+                for _ in 0..iters {
+                    let mut streaming: Box<dyn HttpChecksum> = algorithm.into_impl();
+                    for chunk in payload.chunks(chunk_size) {
+                        streaming.update(black_box(chunk));
+                    }
+                    last_headers = streaming.headers();
+                    black_box(&last_headers);
+                }
+                let elapsed = t0.elapsed();
+                assert_eq!(
+                    last_headers.get(algorithm.http_header_name()),
+                    expected_headers.get(algorithm.http_header_name()),
+                    "{algorithm:?} streaming chunk_size={chunk_size} header mismatch"
+                );
+                let total_mib = (payload.len() * iters) as f64 / (1024.0 * 1024.0);
+                let mibs = total_mib / elapsed.as_secs_f64().max(1e-9);
+                println!(
+                    "[microbench][checksums] algo={algorithm:?} chunk_size={chunk_size}B throughput={mibs:.1} MiB/s (size={}B)",
+                    algorithm.raw_len()
+                );
+            }
+        }
+    }
 }
@@ -276,6 +276,13 @@ Drive timeout profile preset:
  - values must be positive; `0` or an invalid value falls back to the default, and values above `60` are clamped to `60`.
  - the setting is read by the aggregating node only; it does not change the internode RPC wire contract. Any retry shares one round deadline rather than receiving a fresh timeout.

+## Target queue directory
+
+- `RUSTFS_TARGET_STORE_DIR`
+  - absolute root for notify and audit target queues whose `queue_dir` is the default `/opt/rustfs/events` or `/opt/rustfs/audit`; they open `<root>/events` and `<root>/audit` on this node.
+  - unset keeps those paths on Linux and macOS; native Windows uses `%ProgramData%\RustFS\events` and `%ProgramData%\RustFS\audit`.
+  - explicit `queue_dir` values are never relocated, and the persisted target configuration is unchanged. Changing the root does not migrate pending entries. See `docs/operations/target-queue-storage.md`.
+
 ## Startup filesystem boundary policy

 - `RUSTFS_UNSUPPORTED_FS_POLICY` controls startup behavior when RustFS detects local endpoint filesystems that are outside the supported production boundary.
@@ -144,6 +144,12 @@ pub const POSTGRES_QUEUE_LIMIT: &str = "queue_limit";
 /// Applies to both notify and audit target queue stores.
 pub const ENV_TARGET_STORE_COMPRESS: &str = "RUSTFS_TARGET_STORE_COMPRESS";

+/// Environment variable that relocates the root of the default notify and audit queue directories.
+///
+/// It applies only to targets whose `queue_dir` is the logical default (`EVENT_DEFAULT_DIR` or
+/// `AUDIT_DEFAULT_DIR`); their queues move to `<root>/events` and `<root>/audit` on this node.
+pub const ENV_TARGET_STORE_DIR: &str = "RUSTFS_TARGET_STORE_DIR";
+
 /// Queue-store compression is enabled by default to reduce disk footprint.
 pub const DEFAULT_TARGET_STORE_COMPRESS: bool = true;

@@ -91,7 +91,12 @@ pub const ENV_OBS_LOG_MATCH_MODE: &str = "RUSTFS_OBS_LOG_MATCH_MODE";

 /// Default values for log cleanup
 pub const DEFAULT_OBS_LOG_MAX_TOTAL_SIZE_BYTES: u64 = 2 * 1024 * 1024 * 1024; // 2 GiB
-pub const DEFAULT_OBS_LOG_MAX_SINGLE_FILE_SIZE_BYTES: u64 = 0; // No single file limit
+pub const DEFAULT_OBS_LOG_MAX_SINGLE_FILE_SIZE_BYTES: u64 = 0; // No per-file trim by the cleaner
+/// Size at which the active log file is rolled when
+/// `RUSTFS_OBS_LOG_MAX_SINGLE_FILE_SIZE_BYTES` is unset. Without a size roll
+/// the active file is bounded only by the rotation period, and the cleaner
+/// never touches the active file. An explicit `0` keeps size rolling off.
+pub const DEFAULT_OBS_LOG_ACTIVE_FILE_ROLL_SIZE_BYTES: u64 = 256 * 1024 * 1024; // 256 MiB
 pub const DEFAULT_OBS_LOG_COMPRESS_OLD_FILES: bool = true;
 pub const DEFAULT_OBS_LOG_GZIP_COMPRESSION_LEVEL: u32 = 6;
 pub const DEFAULT_OBS_LOG_COMPRESSION_ALGORITHM_GZIP: &str = "gzip";
@@ -29,6 +29,7 @@ Registered in [`src/lib.rs`](src/lib.rs). Grouped by concern:
 | **distributed 4×4** | [`src/distributed/`](src/distributed) | Nightly `e2e-distributed` lane: S3, object lock/WORM, versioning, bucket/site replication, quota, expand/decommission/rebalance, concurrency, chaos, 4-node upgrade of historical data and IAM AK/SK. Map: [`docs/testing/distributed-e2e.md`](../../docs/testing/distributed-e2e.md) |
 | **chaos / reliability** | [`src/chaos.rs`](src/chaos.rs), `reliability_disk_fault_test`, `heal_erasure_disk_rebuild_test`, `server_startup_failfast_test` | Disk offline/replace/corrupt, EC rebuild, heal, fail-fast startup |
 | **upgrade compatibility** | `upgrade_compatibility_test` | Pinned previous-release writes followed by current-build reads on the same data directory |
+| **dual-stack differential** | `dual_stack_differential` | Two single-node servers answer one SDK script for all 73 S3 operations and every differing `(op, field, a, b)` is reported. Side A is the binary under test on `RUSTFS_S3_STACK=legacy`; side B runs `RUSTFS_E2E_STACK_B`. Set `RUSTFS_E2E_REFERENCE_BINARY` to a released `rustfs` binary to make side A that binary instead (started without `RUSTFS_S3_STACK`, which older releases do not know). A missing, non-file or non-executable path fails the run; it never skips or falls back to the same-binary mode. The report in `<target>/dual-stack/<run>.json` records `reference_side` and each side's `--version` output |

 The external-tool `storage_metric_ownership_test` validates the OTLP/Collector/Prometheus path, including a rolling upgrade and node failures. See the [storage metrics guide](../../docs/operations/storage-metrics.md) for its required binaries and focused command.

@@ -17,7 +17,8 @@
 //! when PublicAccessBlock configuration is missing or explicitly set.

 use crate::common::{RustFSTestEnvironment, init_logging, local_http_client};
-use aws_sdk_s3::types::PublicAccessBlockConfiguration;
+use crate::sts_query_compat_test::{OPA_AUTH_TOKEN, OpaMock};
+use aws_sdk_s3::types::{PublicAccessBlockConfiguration, Tag, Tagging};
 use tracing::info;

 async fn setup_public_bucket(
@@ -69,6 +70,110 @@ async fn anonymous_get_object(
    local_http_client().get(&url).send().await
 }

+#[tokio::test]
+async fn test_plain_http_matches_secure_transport_false_bucket_policy() -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
+    init_logging();
+
+    let mut env = RustFSTestEnvironment::new().await?;
+    env.start_rustfs_server(vec![]).await?;
+    assert!(env.url.starts_with("http://"), "fixture must exercise plaintext HTTP: {}", env.url);
+
+    let bucket = "secure-transport-policy";
+    let key = "plain-http.txt";
+    let admin = env.create_s3_client();
+    admin.create_bucket().bucket(bucket).send().await?;
+    admin
+        .put_object()
+        .bucket(bucket)
+        .key(key)
+        .body(aws_sdk_s3::primitives::ByteStream::from_static(b"plain http"))
+        .send()
+        .await?;
+    admin
+        .put_bucket_policy()
+        .bucket(bucket)
+        .policy(
+            serde_json::json!({
+                "Version": "2012-10-17",
+                "Statement": [{
+                    "Effect": "Allow",
+                    "Principal": "*",
+                    "Action": "s3:GetObject",
+                    "Resource": format!("arn:aws:s3:::{bucket}/*"),
+                    "Condition": {"Bool": {"aws:SecureTransport": "false"}}
+                }]
+            })
+            .to_string(),
+        )
+        .send()
+        .await?;
+
+    let response = anonymous_get_object(&env, bucket, key).await?;
+    assert_eq!(response.status().as_u16(), 200);
+    assert_eq!(response.bytes().await?.as_ref(), b"plain http");
+    Ok(())
+}
+
+/// Root credentials have no attached IAM policy, so these requests depend on
+/// the owner flag reaching policy evaluation unchanged.
+#[tokio::test]
+async fn test_root_requests_keep_policy_owner_flag() -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
+    init_logging();
+
+    let opa = OpaMock::start().await?;
+    let mut env = RustFSTestEnvironment::new().await?;
+    env.start_rustfs_server_with_env(
+        vec![],
+        &[
+            ("RUSTFS_POLICY_PLUGIN_URL", opa.url.as_str()),
+            ("RUSTFS_POLICY_PLUGIN_AUTH_TOKEN", OPA_AUTH_TOKEN),
+        ],
+    )
+    .await?;
+
+    let bucket = "root-owner-flag";
+    let key = "owner.txt";
+    let client = env.create_s3_client();
+    client.create_bucket().bucket(bucket).send().await?;
+    client
+        .put_object()
+        .bucket(bucket)
+        .key(key)
+        .body(aws_sdk_s3::primitives::ByteStream::from_static(b"owner access"))
+        .send()
+        .await?;
+
+    // Duplicate tag keys are ambiguous, so PutBucketTagging rejects them
+    // before they can reach stored metadata or the OPA tag-loading path.
+    let duplicate = client
+        .put_bucket_tagging()
+        .bucket(bucket)
+        .tagging(
+            Tagging::builder()
+                .tag_set(Tag::builder().key("department").value("finance").build()?)
+                .tag_set(Tag::builder().key("department").value("engineering").build()?)
+                .build()?,
+        )
+        .send()
+        .await
+        .expect_err("duplicate bucket tag keys must be rejected");
+    assert_eq!(duplicate.into_service_error().meta().code(), Some("InvalidTag"));
+    client
+        .put_bucket_tagging()
+        .bucket(bucket)
+        .tagging(
+            Tagging::builder()
+                .tag_set(Tag::builder().key("department").value("finance").build()?)
+                .build()?,
+        )
+        .send()
+        .await?;
+    let object = client.get_object().bucket(bucket).key(key).send().await?;
+
+    assert_eq!(object.body.collect().await?.into_bytes().as_ref(), b"owner access");
+    Ok(())
+}
+
 /// Issue #2036: Anonymous GetObject should succeed when bucket policy allows it
 /// and no PublicAccessBlock configuration exists (ConfigNotFound).
 #[tokio::test]
@@ -774,6 +774,18 @@ impl RustFSTestEnvironment {
            .await
    }

+    /// Like [`Self::start_rustfs_server_from_binary`], but leaves other running RustFS processes
+    /// alone, so that two servers of different binaries can be alive at once.
+    pub async fn start_rustfs_server_from_binary_without_cleanup(
+        &mut self,
+        binary_path: &Path,
+        extra_args: Vec<&str>,
+        extra_env: &[(&str, &str)],
+    ) -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
+        self.start_rustfs_server_inner_with_binary(binary_path, extra_args, extra_env, false)
+            .await
+    }
+
    /// Start RustFS server with basic configuration
    pub async fn start_rustfs_server(&mut self, extra_args: Vec<&str>) -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
        self.start_rustfs_server_inner(extra_args, &[], true).await
@@ -1285,7 +1285,9 @@ async fn list_parts_disk_loss_after_metadata_read(
    let error = result.expect_err("one missing directory and three unavailable participants cannot prove an empty upload");
    let response = error.raw_response().expect("ListParts must return an S3 service response");
    assert_eq!(response.status().as_u16(), 503);
-    assert_eq!(error.as_service_error().and_then(|err| err.meta().code()), Some("SlowDownRead"));
+    // Since #9149, SlowDownRead is reserved for GET/HEAD object reads; a
+    // ListParts read-quorum failure is reported as ServiceUnavailable.
+    assert_eq!(error.as_service_error().and_then(|err| err.meta().code()), Some("ServiceUnavailable"));
    assert!(response.headers().get("x-rustfs-error-cause").is_none());
    assert!(response.headers().get("retry-after").is_none());
    assert!(
@@ -493,7 +493,12 @@ async fn compare_degraded_legacy_list_parts(dist: &mut DistCluster, seed: &Upgra
    assert!(legacy.parts().is_empty());
    let current = current.expect_err("current coordinator must not infer empty from missing directories and offline disks");
    assert_eq!(current.raw_response().expect("current S3 response").status().as_u16(), 503);
-    assert_eq!(current.as_service_error().and_then(|error| error.meta().code()), Some("SlowDownRead"));
+    // Since #9149, SlowDownRead is reserved for GET/HEAD object reads; a
+    // ListParts read-quorum failure is reported as ServiceUnavailable.
+    assert_eq!(
+        current.as_service_error().and_then(|error| error.meta().code()),
+        Some("ServiceUnavailable")
+    );
    dist.cluster.start_node_from_binary(3, previous).await?;
    wait_for_upgrade_write_readiness(&clients, &seed.history_bucket, "mixed-list-restored", CREDENTIAL_TIMEOUT).await?;
    for (path, before) in original_files {
@@ -0,0 +1,524 @@
+// Copyright 2024 RustFS Team
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+//! What one stack answered, and how two answers are compared.
+//!
+//! An [`Observation`] keeps only what a client acts on: the HTTP status, the S3 error code, the
+//! allowlisted response headers and a few typed fields decoded from the response. The error
+//! message is kept for diagnostics and never compared: message wording is judged by the wire
+//! corpus, not here. The comparator is pure so it can be tested without a server.
+
+use serde::Serialize;
+use std::collections::{BTreeMap, BTreeSet};
+use std::path::{Path, PathBuf};
+
+/// Response headers compared exactly (after [`normalized_header_value`]).
+const HEADER_ALLOWLIST: [&str; 7] = [
+    "content-type",
+    "x-amz-version-id",
+    "etag",
+    "x-amz-delete-marker",
+    "content-range",
+    "accept-ranges",
+    "x-amz-request-charged",
+];
+
+/// Every response header with this prefix is compared as well.
+const HEADER_ALLOWLIST_PREFIX: &str = "x-amz-server-side-encryption";
+
+/// Placeholder for a field one side did not send.
+const ABSENT: &str = "<absent>";
+
+/// The lowercase header name if the comparator looks at this header.
+pub(crate) fn compared_header(name: &str) -> Option<String> {
+    let name = name.to_ascii_lowercase();
+    (HEADER_ALLOWLIST.contains(&name.as_str()) || name.starts_with(HEADER_ALLOWLIST_PREFIX)).then_some(name)
+}
+
+/// A version ID is minted per process, so its value never matches across two servers; only
+/// whether it is the literal `null` (an unversioned write) carries meaning.
+fn normalized_header_value(name: &str, value: &str) -> String {
+    if name == "x-amz-version-id" && value != "null" {
+        "<version-id>".to_owned()
+    } else {
+        value.to_owned()
+    }
+}
+
+/// One request's outcome as a client sees it.
+#[derive(Clone, Debug, Serialize)]
+pub(crate) struct Observation {
+    /// The S3 operation, named as in the `s3s::S3` trait (`GetObject`).
+    pub op: &'static str,
+    /// Which request of the script this was (`normal`, `negative`, or a setup label).
+    pub case: &'static str,
+    /// HTTP status; 0 when no response arrived.
+    pub status: u16,
+    /// The S3 error code, or the SDK's error kind when there was no service error.
+    pub error_code: Option<String>,
+    /// Allowlisted headers only, lowercase names, normalized values.
+    pub headers: BTreeMap<String, String>,
+    /// Typed values decoded from the response body, never free text.
+    pub fields: BTreeMap<String, String>,
+    /// Diagnostic only; never compared.
+    #[serde(skip)]
+    pub message: Option<String>,
+}
+
+impl Observation {
+    /// Builds an observation, keeping only the allowlisted headers.
+    pub(crate) fn new<'h>(
+        op: &'static str,
+        case: &'static str,
+        status: u16,
+        error_code: Option<String>,
+        headers: impl IntoIterator<Item = (&'h str, &'h str)>,
+    ) -> Self {
+        let headers = headers
+            .into_iter()
+            .filter_map(|(name, value)| compared_header(name).map(|name| (name, value)))
+            .map(|(name, value)| {
+                let value = normalized_header_value(&name, value);
+                (name, value)
+            })
+            .collect();
+        Self {
+            op,
+            case,
+            status,
+            error_code,
+            headers,
+            fields: BTreeMap::new(),
+            message: None,
+        }
+    }
+
+    pub(crate) fn with_field(mut self, name: &str, value: impl Into<String>) -> Self {
+        self.fields.insert(name.to_owned(), value.into());
+        self
+    }
+
+    pub(crate) fn is_success(&self) -> bool {
+        (200..300).contains(&self.status)
+    }
+}
+
+/// One field on which the two stacks answered differently.
+#[derive(Clone, Debug, PartialEq, Eq, Serialize)]
+pub(crate) struct Difference {
+    pub op: String,
+    pub case: String,
+    pub field: String,
+    pub a: String,
+    pub b: String,
+}
+
+/// Every difference between two transcripts of the same script, matched by `(op, case)`.
+pub(crate) fn compare(a: &[Observation], b: &[Observation]) -> Vec<Difference> {
+    let mut differences = Vec::new();
+    let (index_a, duplicates_a) = index(a);
+    let (index_b, duplicates_b) = index(b);
+    for (op, case) in duplicates_a.union(&duplicates_b) {
+        let count = |side: &[Observation]| {
+            side.iter()
+                .filter(|observation| (observation.op, observation.case) == (*op, *case))
+                .count()
+                .to_string()
+        };
+        differences.push(difference(op, case, "duplicate_case", count(a), count(b)));
+    }
+    let keys: BTreeSet<_> = index_a.keys().chain(index_b.keys()).copied().collect();
+    for key @ (op, case) in keys {
+        match (index_a.get(&key), index_b.get(&key)) {
+            (Some(a), Some(b)) => compare_one(a, b, &mut differences),
+            (a, b) => {
+                let presence = |side: Option<&&Observation>| if side.is_some() { "present" } else { ABSENT }.to_owned();
+                differences.push(difference(op, case, "observation", presence(a), presence(b)));
+            }
+        }
+    }
+    differences
+}
+
+type Key = (&'static str, &'static str);
+
+fn index(side: &[Observation]) -> (BTreeMap<Key, &Observation>, BTreeSet<Key>) {
+    let mut index = BTreeMap::new();
+    let mut duplicates = BTreeSet::new();
+    for observation in side {
+        let key = (observation.op, observation.case);
+        if index.insert(key, observation).is_some() {
+            duplicates.insert(key);
+        }
+    }
+    (index, duplicates)
+}
+
+fn difference(op: &str, case: &str, field: &str, a: String, b: String) -> Difference {
+    Difference {
+        op: op.to_owned(),
+        case: case.to_owned(),
+        field: field.to_owned(),
+        a,
+        b,
+    }
+}
+
+fn compare_one(a: &Observation, b: &Observation, differences: &mut Vec<Difference>) {
+    let mut field = |name: String, value_a: Option<&str>, value_b: Option<&str>| {
+        if value_a != value_b {
+            let show = |value: Option<&str>| value.unwrap_or(ABSENT).to_owned();
+            differences.push(difference(a.op, a.case, &name, show(value_a), show(value_b)));
+        }
+    };
+    field(
+        "status".to_owned(),
+        Some(a.status.to_string().as_str()),
+        Some(b.status.to_string().as_str()),
+    );
+    field("error_code".to_owned(), a.error_code.as_deref(), b.error_code.as_deref());
+    let headers: BTreeSet<_> = a.headers.keys().chain(b.headers.keys()).collect();
+    for name in headers {
+        field(
+            format!("header.{name}"),
+            a.headers.get(name).map(String::as_str),
+            b.headers.get(name).map(String::as_str),
+        );
+    }
+    let fields: BTreeSet<_> = a.fields.keys().chain(b.fields.keys()).collect();
+    for name in fields {
+        field(
+            format!("body.{name}"),
+            a.fields.get(name).map(String::as_str),
+            b.fields.get(name).map(String::as_str),
+        );
+    }
+}
+
+/// Operations observed on both sides.
+pub(crate) fn compared_ops(a: &[Observation], b: &[Observation]) -> BTreeSet<&'static str> {
+    let ops_b: BTreeSet<_> = b.iter().map(|observation| observation.op).collect();
+    a.iter()
+        .map(|observation| observation.op)
+        .filter(|op| ops_b.contains(op))
+        .collect()
+}
+
+/// `(op, case)` pairs observed on both sides.
+pub(crate) fn compared_cases(a: &[Observation], b: &[Observation]) -> usize {
+    let cases_b: BTreeSet<_> = b.iter().map(|observation| (observation.op, observation.case)).collect();
+    a.iter()
+        .map(|observation| (observation.op, observation.case))
+        .collect::<BTreeSet<_>>()
+        .intersection(&cases_b)
+        .count()
+}
+
+/// The JSON report written for every run.
+#[derive(Debug, Serialize)]
+pub(crate) struct Report {
+    pub schema: u32,
+    pub run: String,
+    pub stack_a: String,
+    pub stack_b: String,
+    /// Which side is the pinned release binary: `"a"`, or `None` when both sides are the binary
+    /// under test.
+    pub reference_side: Option<&'static str>,
+    /// `--version` output of the binary each side actually ran.
+    pub version_a: String,
+    pub version_b: String,
+    pub region_a: Option<String>,
+    pub region_b: Option<String>,
+    pub compared_ops: usize,
+    pub compared_cases: usize,
+    pub differences: Vec<Difference>,
+}
+
+impl Report {
+    /// Writes the report as `<directory>/<run>.json` and returns its path.
+    pub(crate) fn write_to(&self, directory: &Path) -> std::io::Result<PathBuf> {
+        std::fs::create_dir_all(directory)?;
+        let path = directory.join(format!("{}.json", self.run));
+        let json = serde_json::to_vec_pretty(self).map_err(std::io::Error::other)?;
+        std::fs::write(&path, json)?;
+        Ok(path)
+    }
+}
+
+/// One line per difference, for a failing assertion.
+pub(crate) fn render(differences: &[Difference]) -> String {
+    differences
+        .iter()
+        .map(|d| format!("({}, {}, {}, a={:?}, b={:?})", d.op, d.case, d.field, d.a, d.b))
+        .collect::<Vec<_>>()
+        .join("\n")
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    fn ok(op: &'static str) -> Observation {
+        Observation::new(op, "normal", 200, None, [("content-type", "application/xml")])
+    }
+
+    fn fields(differences: &[Difference]) -> Vec<&str> {
+        differences.iter().map(|d| d.field.as_str()).collect()
+    }
+
+    #[test]
+    fn identical_transcripts_have_no_difference() {
+        let a = vec![
+            ok("HeadBucket"),
+            ok("GetBucketLocation").with_field("location_constraint", "us-east-1"),
+        ];
+        let b = a.clone();
+        assert_eq!(compare(&a, &b), Vec::new());
+        assert_eq!(compared_ops(&a, &b).len(), 2);
+    }
+
+    #[test]
+    fn a_status_difference_is_reported() {
+        let a = vec![ok("PutBucketAcl")];
+        let b = vec![Observation::new(
+            "PutBucketAcl",
+            "normal",
+            403,
+            None,
+            [("content-type", "application/xml")],
+        )];
+        assert_eq!(
+            compare(&a, &b),
+            vec![Difference {
+                op: "PutBucketAcl".into(),
+                case: "normal".into(),
+                field: "status".into(),
+                a: "200".into(),
+                b: "403".into(),
+            }]
+        );
+    }
+
+    #[test]
+    fn an_error_code_difference_is_reported() {
+        let a = vec![Observation::new("GetObject", "negative", 404, Some("NoSuchKey".into()), [])];
+        let b = vec![Observation::new(
+            "GetObject",
+            "negative",
+            404,
+            Some("NoSuchBucket".into()),
+            [],
+        )];
+        let differences = compare(&a, &b);
+        assert_eq!(fields(&differences), ["error_code"]);
+        assert_eq!((differences[0].a.as_str(), differences[0].b.as_str()), ("NoSuchKey", "NoSuchBucket"));
+    }
+
+    #[test]
+    fn an_error_code_on_one_side_only_is_reported() {
+        let a = vec![Observation::new("HeadBucket", "negative", 404, Some("NotFound".into()), [])];
+        let b = vec![Observation::new("HeadBucket", "negative", 404, None, [])];
+        let differences = compare(&a, &b);
+        assert_eq!(fields(&differences), ["error_code"]);
+        assert_eq!(differences[0].b, ABSENT);
+    }
+
+    #[test]
+    fn an_allowlisted_header_difference_is_reported() {
+        let a = vec![Observation::new("HeadObject", "normal", 200, None, [("ETag", "\"aa\"")])];
+        let b = vec![Observation::new("HeadObject", "normal", 200, None, [("etag", "\"bb\"")])];
+        assert_eq!(fields(&compare(&a, &b)), ["header.etag"]);
+    }
+
+    #[test]
+    fn a_header_sent_by_one_side_only_is_reported() {
+        let a = vec![Observation::new(
+            "DeleteObject",
+            "normal",
+            204,
+            None,
+            [("x-amz-delete-marker", "true")],
+        )];
+        let b = vec![Observation::new("DeleteObject", "normal", 204, None, [])];
+        let differences = compare(&a, &b);
+        assert_eq!(fields(&differences), ["header.x-amz-delete-marker"]);
+        assert_eq!((differences[0].a.as_str(), differences[0].b.as_str()), ("true", ABSENT));
+    }
+
+    #[test]
+    fn every_server_side_encryption_header_is_compared() {
+        let header = "x-amz-server-side-encryption-customer-key-md5";
+        let a = vec![Observation::new("PutObject", "sse-c", 200, None, [(header, "one")])];
+        let b = vec![Observation::new("PutObject", "sse-c", 200, None, [(header, "two")])];
+        assert_eq!(fields(&compare(&a, &b)), [format!("header.{header}")]);
+    }
+
+    #[test]
+    fn headers_outside_the_allowlist_are_not_compared() {
+        let a = vec![Observation::new(
+            "ListBuckets",
+            "normal",
+            200,
+            None,
+            [
+                ("date", "Mon"),
+                ("x-amz-request-id", "1"),
+                ("server", "a"),
+                ("content-length", "10"),
+            ],
+        )];
+        let b = vec![Observation::new(
+            "ListBuckets",
+            "normal",
+            200,
+            None,
+            [
+                ("date", "Tue"),
+                ("x-amz-request-id", "2"),
+                ("server", "b"),
+                ("content-length", "11"),
+            ],
+        )];
+        assert_eq!(compare(&a, &b), Vec::new());
+        assert!(a[0].headers.is_empty(), "non-allowlisted headers were kept: {:?}", a[0].headers);
+    }
+
+    #[test]
+    fn distinct_version_ids_are_not_a_difference() {
+        let a = vec![Observation::new(
+            "PutObject",
+            "normal",
+            200,
+            None,
+            [("x-amz-version-id", "0b1c")],
+        )];
+        let b = vec![Observation::new(
+            "PutObject",
+            "normal",
+            200,
+            None,
+            [("x-amz-version-id", "9f8e")],
+        )];
+        assert_eq!(compare(&a, &b), Vec::new());
+    }
+
+    #[test]
+    fn a_null_version_id_against_a_real_one_is_reported() {
+        let a = vec![Observation::new(
+            "PutObject",
+            "normal",
+            200,
+            None,
+            [("x-amz-version-id", "null")],
+        )];
+        let b = vec![Observation::new(
+            "PutObject",
+            "normal",
+            200,
+            None,
+            [("x-amz-version-id", "9f8e")],
+        )];
+        assert_eq!(fields(&compare(&a, &b)), ["header.x-amz-version-id"]);
+    }
+
+    #[test]
+    fn a_body_field_difference_is_reported() {
+        let a = vec![ok("GetBucketLocation").with_field("location_constraint", "us-east-1")];
+        let b = vec![ok("GetBucketLocation").with_field("location_constraint", "eu-west-1")];
+        let differences = compare(&a, &b);
+        assert_eq!(fields(&differences), ["body.location_constraint"]);
+        assert_eq!((differences[0].a.as_str(), differences[0].b.as_str()), ("us-east-1", "eu-west-1"));
+    }
+
+    #[test]
+    fn a_body_field_on_one_side_only_is_reported() {
+        let a = vec![ok("ListObjectsV2").with_field("keys", "[]")];
+        let b = vec![ok("ListObjectsV2")];
+        assert_eq!(fields(&compare(&a, &b)), ["body.keys"]);
+    }
+
+    #[test]
+    fn error_messages_are_never_compared() {
+        let mut a = Observation::new("GetObject", "negative", 404, Some("NoSuchKey".into()), []);
+        let mut b = a.clone();
+        a.message = Some("The specified key does not exist.".into());
+        b.message = Some("Object not found".into());
+        assert_eq!(compare(&[a], &[b]), Vec::new());
+    }
+
+    #[test]
+    fn a_case_observed_on_one_side_only_is_reported() {
+        let a = vec![ok("HeadBucket"), ok("ListBuckets")];
+        let b = vec![ok("HeadBucket")];
+        let differences = compare(&a, &b);
+        assert_eq!(fields(&differences), ["observation"]);
+        assert_eq!(differences[0].op, "ListBuckets");
+        assert_eq!(compared_ops(&a, &b).into_iter().collect::<Vec<_>>(), ["HeadBucket"]);
+    }
+
+    #[test]
+    fn a_case_recorded_twice_is_reported() {
+        let a = vec![ok("HeadBucket"), ok("HeadBucket")];
+        let b = vec![ok("HeadBucket")];
+        assert_eq!(fields(&compare(&a, &b)), ["duplicate_case"]);
+    }
+
+    #[test]
+    fn compared_ops_counts_operations_not_cases() {
+        let negative = Observation::new("HeadBucket", "negative", 404, Some("NotFound".into()), []);
+        let a = vec![ok("HeadBucket"), negative, ok("ListBuckets")];
+        let b = a.clone();
+        assert_eq!(compared_ops(&a, &b).len(), 2);
+    }
+
+    #[test]
+    fn the_report_lists_each_difference_as_op_field_a_b() {
+        let directory = std::env::temp_dir().join(format!("dual-stack-report-{}", uuid::Uuid::new_v4()));
+        let report = Report {
+            schema: 2,
+            run: "unit".into(),
+            stack_a: "legacy".into(),
+            stack_b: "legacy".into(),
+            reference_side: Some("a"),
+            version_a: "rustfs 1.0.4".into(),
+            version_b: "rustfs 1.1.0".into(),
+            region_a: None,
+            region_b: Some("eu-west-1".into()),
+            compared_ops: 1,
+            compared_cases: 1,
+            differences: compare(
+                &[ok("GetBucketLocation").with_field("location_constraint", "us-east-1")],
+                &[ok("GetBucketLocation").with_field("location_constraint", "eu-west-1")],
+            ),
+        };
+        let path = report.write_to(&directory).expect("write the report");
+        let json: serde_json::Value = serde_json::from_slice(&std::fs::read(&path).expect("read the report")).expect("parse");
+        let _ = std::fs::remove_dir_all(&directory);
+        assert_eq!(path.file_name().and_then(|name| name.to_str()), Some("unit.json"));
+        assert_eq!(json["compared_ops"], 1);
+        assert_eq!(json["reference_side"], "a");
+        assert_eq!(json["version_a"], "rustfs 1.0.4");
+        assert_eq!(json["version_b"], "rustfs 1.1.0");
+        assert_eq!(
+            json["differences"],
+            serde_json::json!([{
+                "op": "GetBucketLocation",
+                "case": "normal",
+                "field": "body.location_constraint",
+                "a": "us-east-1",
+                "b": "eu-west-1",
+            }])
+        );
+    }
+}
@@ -0,0 +1,257 @@
+// Copyright 2024 RustFS Team
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+//! Dual-stack differential: one `rustfs` binary, two processes, one SDK script (rustfs/backlog#2740).
+//!
+//! Stack A runs `RUSTFS_S3_STACK=legacy` of the binary under test, unless `RUSTFS_E2E_REFERENCE_BINARY`
+//! names a released binary: then A is that binary, started without `RUSTFS_S3_STACK` (older
+//! releases do not know it), and a bad path fails the run instead of falling back (see
+//! `reference`). The report names which side is the reference and the `--version` each side ran. Stack B runs the value of `RUSTFS_E2E_STACK_B`
+//! (default `legacy` until the gateway stack is assembled), and `RUSTFS_E2E_STACK_B_REGION`, when
+//! set, starts B in another region so that a difference can be injected by hand. Both servers
+//! answer the script in `scenarios`, which covers every operation of `impl S3 for FS`; `compare`
+//! reports each `(op, field, a, b)` on which the answers differ, and every run writes its report
+//! to `<target>/dual-stack/<run>.json`. Two processes, not two embedded servers: bucket metadata
+//! is process-global, so two stacks in one process would read each other's buckets.
+
+mod compare;
+mod reference;
+mod scenarios;
+
+use crate::common::{RustFSTestEnvironment, init_logging, rustfs_binary_path, workspace_root};
+use compare::{Difference, Observation, Report, compare, compared_cases, compared_ops, render};
+use reference::{SideA, binary_version};
+use std::path::{Path, PathBuf};
+use std::time::{SystemTime, UNIX_EPOCH};
+
+type BoxError = Box<dyn std::error::Error + Send + Sync>;
+
+const STACK_A: &str = "legacy";
+/// Label of side A in the report when it is the released reference binary.
+const STACK_A_REFERENCE: &str = "reference";
+const ENV_STACK_B: &str = "RUSTFS_E2E_STACK_B";
+const ENV_STACK_B_REGION: &str = "RUSTFS_E2E_STACK_B_REGION";
+/// Becomes `gateway` once rustfs/backlog#2734 T2.5 assembles the gateway stack.
+const DEFAULT_STACK_B: &str = "legacy";
+/// The region the negative control gives stack A explicitly, so an inherited `RUSTFS_REGION`
+/// cannot erase the injected difference. Not `us-east-1`: that region answers GetBucketLocation
+/// with an empty LocationConstraint and re-creating an owned bucket with 200, so it would not
+/// report its own name and would add CreateBucket differences to the control.
+const CONTROL_REGION_A: &str = "us-west-2";
+const CONTROL_REGION_B: &str = "eu-west-1";
+/// `normal` cases the legacy stack must answer with a 2xx. The comparison alone cannot tell a
+/// script that exercises real behavior from one in which both stacks fail the same way. Five of
+/// the 73 fail on legacy by design of this script: PutBucketReplication (no registered remote
+/// target), GetBucketReplication (so no configuration), PutBucketNotificationConfiguration (no
+/// notification target configured), RestoreObject (the object is not archived) and
+/// GetObjectTorrent (legacy answers `NoSuchKey`).
+const MIN_NORMAL_SUCCESSES: usize = 68;
+
+struct PairConfig {
+    side_a: SideA,
+    stack_b: String,
+    region_a: Option<String>,
+    region_b: Option<String>,
+}
+
+impl PairConfig {
+    fn from_env() -> Result<Self, BoxError> {
+        Ok(Self {
+            side_a: SideA::from_env()?,
+            stack_b: std::env::var(ENV_STACK_B).unwrap_or_else(|_| DEFAULT_STACK_B.to_owned()),
+            region_a: None,
+            region_b: std::env::var(ENV_STACK_B_REGION).ok(),
+        })
+    }
+
+    fn stack_a_label(&self) -> &'static str {
+        match self.side_a {
+            SideA::SameBinary => STACK_A,
+            SideA::Reference(_) => STACK_A_REFERENCE,
+        }
+    }
+}
+
+/// Starts one server of the binary under test on `stack`, overriding its region only when asked.
+async fn start_stack(stack: &str, region: Option<&str>) -> Result<RustFSTestEnvironment, BoxError> {
+    let mut env = RustFSTestEnvironment::new().await?;
+    let mut vars = vec![("RUSTFS_S3_STACK", stack)];
+    if let Some(region) = region {
+        vars.push(("RUSTFS_REGION", region));
+    }
+    env.start_rustfs_server_without_cleanup_with_env(&vars).await?;
+    Ok(env)
+}
+
+/// Starts one server of a released binary. `RUSTFS_S3_STACK` is not passed: older releases do not
+/// know it and have exactly one stack.
+async fn start_reference(binary: &Path, region: Option<&str>) -> Result<RustFSTestEnvironment, BoxError> {
+    let mut env = RustFSTestEnvironment::new().await?;
+    let mut vars = Vec::new();
+    if let Some(region) = region {
+        vars.push(("RUSTFS_REGION", region));
+    }
+    env.start_rustfs_server_from_binary_without_cleanup(binary, vec![], &vars)
+        .await?;
+    Ok(env)
+}
+
+/// Starts stack A and stack B side by side, each with its own port and data directory.
+async fn spawn_pair(config: &PairConfig) -> Result<(RustFSTestEnvironment, RustFSTestEnvironment), BoxError> {
+    let side_a = async {
+        match &config.side_a {
+            SideA::SameBinary => start_stack(STACK_A, config.region_a.as_deref()).await,
+            SideA::Reference(binary) => start_reference(binary, config.region_a.as_deref()).await,
+        }
+    };
+    tokio::try_join!(side_a, start_stack(&config.stack_b, config.region_b.as_deref()))
+}
+
+fn report_directory() -> PathBuf {
+    let target = std::env::var_os("CARGO_TARGET_DIR").map_or_else(|| PathBuf::from("target"), PathBuf::from);
+    let target = if target.is_absolute() {
+        target
+    } else {
+        workspace_root().join(target)
+    };
+    target.join("dual-stack")
+}
+
+struct Run {
+    report: Report,
+    path: PathBuf,
+    a: Vec<Observation>,
+}
+
+/// Runs the script against both stacks concurrently and writes the report.
+async fn run_differential(label: &str, config: &PairConfig) -> Run {
+    let binary_under_test = rustfs_binary_path();
+    let (version_a, version_b) = match &config.side_a {
+        SideA::SameBinary => {
+            let version = binary_version(&binary_under_test)
+                .await
+                .expect("capture the binary under test version");
+            (version.clone(), version)
+        }
+        SideA::Reference(binary) => (
+            binary_version(binary).await.expect("capture the reference binary version"),
+            binary_version(&binary_under_test)
+                .await
+                .expect("capture the binary under test version"),
+        ),
+    };
+    let (server_a, server_b) = spawn_pair(config).await.expect("start stack A and stack B");
+    let (a, b) = tokio::join!(Box::pin(scenarios::run(&server_a)), Box::pin(scenarios::run(&server_b)));
+    let millis = SystemTime::now()
+        .duration_since(UNIX_EPOCH)
+        .map_or(0, |elapsed| elapsed.as_millis());
+    let report = Report {
+        schema: 2,
+        run: format!("{label}-{millis}-{}", std::process::id()),
+        stack_a: config.stack_a_label().to_owned(),
+        stack_b: config.stack_b.clone(),
+        reference_side: matches!(config.side_a, SideA::Reference(_)).then_some("a"),
+        version_a,
+        version_b,
+        region_a: config.region_a.clone(),
+        region_b: config.region_b.clone(),
+        compared_ops: compared_ops(&a, &b).len(),
+        compared_cases: compared_cases(&a, &b),
+        differences: compare(&a, &b),
+    };
+    let path = report.write_to(&report_directory()).expect("write the dual-stack report");
+    Run { report, path, a }
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    #[tokio::test]
+    async fn both_stacks_answer_every_operation_identically() {
+        init_logging();
+        let config = PairConfig::from_env().expect("RUSTFS_E2E_REFERENCE_BINARY must be valid when set");
+        let run = run_differential("self-comparison", &config).await;
+        let report = &run.report;
+
+        let mut observed: Vec<_> = run.a.iter().map(|observation| observation.op).collect();
+        observed.sort_unstable();
+        observed.dedup();
+        assert_eq!(
+            observed,
+            scenarios::OPERATIONS,
+            "the script must drive exactly the operations of impl S3 for FS"
+        );
+        assert_eq!(report.compared_ops, scenarios::OPERATIONS.len(), "report: {}", run.path.display());
+        assert!(
+            report.differences.is_empty(),
+            "stack {} ({}) and stack {} ({}) differ on {} of {} cases (report: {}):\n{}",
+            report.stack_a,
+            report.version_a,
+            report.stack_b,
+            report.version_b,
+            report.differences.len(),
+            report.compared_cases,
+            run.path.display(),
+            render(&report.differences)
+        );
+
+        let not_4xx: Vec<_> = run
+            .a
+            .iter()
+            .filter(|observation| observation.case == "negative" && !(400..500).contains(&observation.status))
+            .collect();
+        assert!(
+            not_4xx.is_empty(),
+            "negative cases must fail with a 4xx on side A ({}): {not_4xx:#?}",
+            report.version_a
+        );
+        let successes = run
+            .a
+            .iter()
+            .filter(|observation| observation.case == "normal" && observation.is_success())
+            .count();
+        assert!(
+            successes >= MIN_NORMAL_SUCCESSES,
+            "only {successes} normal cases succeeded on side A ({}): {:#?}",
+            report.version_a,
+            run.a
+        );
+    }
+
+    #[tokio::test]
+    async fn differential_detects_an_injected_difference() {
+        init_logging();
+        let config = PairConfig {
+            side_a: SideA::SameBinary,
+            stack_b: STACK_A.to_owned(),
+            region_a: Some(CONTROL_REGION_A.to_owned()),
+            region_b: Some(CONTROL_REGION_B.to_owned()),
+        };
+        let run = run_differential("injected-region", &config).await;
+        let expected = Difference {
+            op: "GetBucketLocation".to_owned(),
+            case: "normal".to_owned(),
+            field: "body.location_constraint".to_owned(),
+            a: CONTROL_REGION_A.to_owned(),
+            b: CONTROL_REGION_B.to_owned(),
+        };
+        assert!(
+            run.report.differences.contains(&expected),
+            "B started in {CONTROL_REGION_B} but GetBucketLocation was not reported (report: {}):\n{}",
+            run.path.display(),
+            render(&run.report.differences)
+        );
+    }
+}
@@ -0,0 +1,163 @@
+// Copyright 2024 RustFS Team
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+//! Which binary side A of the differential runs, and what it reports about itself.
+//!
+//! `RUSTFS_E2E_REFERENCE_BINARY` names a released `rustfs` binary to use as the reference. Unset,
+//! side A is the binary under test on `RUSTFS_S3_STACK=legacy`. Set, it must resolve to an
+//! executable file or the run fails: a reference that was asked for and silently replaced by the
+//! binary under test would report an agreement with itself as agreement with the release.
+
+use std::ffi::OsString;
+use std::path::{Path, PathBuf};
+
+type BoxError = Box<dyn std::error::Error + Send + Sync>;
+
+pub(super) const ENV_REFERENCE_BINARY: &str = "RUSTFS_E2E_REFERENCE_BINARY";
+
+#[derive(Clone, Debug, PartialEq, Eq)]
+pub(super) enum SideA {
+    /// Both sides are the binary under test; A runs `RUSTFS_S3_STACK=legacy`.
+    SameBinary,
+    /// A is this released binary, started without `RUSTFS_S3_STACK`.
+    Reference(PathBuf),
+}
+
+impl SideA {
+    pub(super) fn from_env() -> Result<Self, BoxError> {
+        Self::resolve(std::env::var_os(ENV_REFERENCE_BINARY))
+    }
+
+    /// `None` is the only value that selects same-binary mode; every other value must validate.
+    pub(super) fn resolve(value: Option<OsString>) -> Result<Self, BoxError> {
+        let Some(value) = value else {
+            return Ok(Self::SameBinary);
+        };
+        let path = PathBuf::from(value);
+        validate_executable(&path)?;
+        Ok(Self::Reference(path))
+    }
+}
+
+fn validate_executable(path: &Path) -> Result<(), BoxError> {
+    if path.as_os_str().is_empty() {
+        return Err(format!("{ENV_REFERENCE_BINARY} is set but empty").into());
+    }
+    let metadata = std::fs::metadata(path)
+        .map_err(|error| format!("{ENV_REFERENCE_BINARY} points to {}, which cannot be read: {error}", path.display()))?;
+    if !metadata.is_file() {
+        return Err(format!("{ENV_REFERENCE_BINARY} points to {}, which is not a file", path.display()).into());
+    }
+    if !is_executable(&metadata) {
+        return Err(format!("{ENV_REFERENCE_BINARY} points to {}, which is not executable", path.display()).into());
+    }
+    Ok(())
+}
+
+#[cfg(unix)]
+fn is_executable(metadata: &std::fs::Metadata) -> bool {
+    use std::os::unix::fs::PermissionsExt;
+    metadata.permissions().mode() & 0o111 != 0
+}
+
+#[cfg(not(unix))]
+fn is_executable(_metadata: &std::fs::Metadata) -> bool {
+    true
+}
+
+/// What `<binary> --version` printed, trimmed. Fails when the binary does not run, exits
+/// non-zero, or prints nothing, because an empty version would report nothing as observed.
+pub(super) async fn binary_version(binary: &Path) -> Result<String, BoxError> {
+    let output = tokio::process::Command::new(binary).arg("--version").output().await?;
+    if !output.status.success() {
+        return Err(format!("{} --version failed with {}", binary.display(), output.status).into());
+    }
+    let version = String::from_utf8_lossy(&output.stdout).trim().to_owned();
+    if version.is_empty() {
+        return Err(format!("{} --version printed nothing", binary.display()).into());
+    }
+    Ok(version)
+}
+
+#[cfg(all(test, unix))]
+mod tests {
+    use super::*;
+    use std::os::unix::fs::PermissionsExt;
+
+    fn script(dir: &Path, name: &str, body: &str, mode: u32) -> PathBuf {
+        let path = dir.join(name);
+        std::fs::write(&path, body).expect("write the stand-in");
+        std::fs::set_permissions(&path, std::fs::Permissions::from_mode(mode)).expect("chmod");
+        path
+    }
+
+    #[test]
+    fn missing_path_is_an_error_not_a_fallback() {
+        let dir = tempfile::tempdir().expect("tempdir");
+        let error = SideA::resolve(Some(dir.path().join("absent").into_os_string())).expect_err("must fail closed");
+        assert!(error.to_string().contains("cannot be read"), "{error}");
+    }
+
+    #[test]
+    fn non_executable_file_is_an_error() {
+        let dir = tempfile::tempdir().expect("tempdir");
+        let path = script(dir.path(), "rustfs", "#!/bin/sh\necho v\n", 0o644);
+        let error = SideA::resolve(Some(path.into_os_string())).expect_err("must fail closed");
+        assert!(error.to_string().contains("not executable"), "{error}");
+    }
+
+    #[test]
+    fn directory_is_an_error() {
+        let dir = tempfile::tempdir().expect("tempdir");
+        let error = SideA::resolve(Some(dir.path().as_os_str().to_owned())).expect_err("must fail closed");
+        assert!(error.to_string().contains("not a file"), "{error}");
+    }
+
+    #[test]
+    fn empty_value_is_an_error() {
+        let error = SideA::resolve(Some(OsString::new())).expect_err("must fail closed");
+        assert!(error.to_string().contains("empty"), "{error}");
+    }
+
+    #[test]
+    fn unset_selects_same_binary_mode() {
+        assert_eq!(SideA::resolve(None).expect("unset is valid"), SideA::SameBinary);
+    }
+
+    #[test]
+    fn executable_file_selects_reference_mode() {
+        let dir = tempfile::tempdir().expect("tempdir");
+        let path = script(dir.path(), "rustfs", "#!/bin/sh\necho v\n", 0o755);
+        assert_eq!(
+            SideA::resolve(Some(path.clone().into_os_string())).expect("valid"),
+            SideA::Reference(path)
+        );
+    }
+
+    #[tokio::test]
+    async fn version_output_is_captured_from_the_binary() {
+        let dir = tempfile::tempdir().expect("tempdir");
+        let path = script(dir.path(), "rustfs", "#!/bin/sh\necho 'rustfs 1.0.4 (abc123)'\n", 0o755);
+        assert_eq!(binary_version(&path).await.expect("version"), "rustfs 1.0.4 (abc123)");
+    }
+
+    #[tokio::test]
+    async fn failing_or_silent_version_is_an_error() {
+        let dir = tempfile::tempdir().expect("tempdir");
+        let failing = script(dir.path(), "failing", "#!/bin/sh\nexit 3\n", 0o755);
+        assert!(binary_version(&failing).await.is_err());
+        let silent = script(dir.path(), "silent", "#!/bin/sh\nexit 0\n", 0o755);
+        assert!(binary_version(&silent).await.is_err());
+    }
+}
@@ -586,7 +586,18 @@ async fn test_reads_under_a_deleted_kms_key_report_key_not_found() -> Result<(),
        .await?;
    assert_eq!(put.ssekms_key_id(), Some(doomed_key_id.as_str()));

-    info!("🗑️ deleting {doomed_key_id} immediately");
+    // Destruction is two requests: schedule the deletion, then destroy the
+    // pending key without waiting out its window.
+    info!("🗑️ scheduling {doomed_key_id} for deletion, then destroying it");
+    kms_admin_request(
+        &base.url,
+        http::Method::DELETE,
+        "/rustfs/admin/v3/kms/keys/delete",
+        Some(&serde_json::json!({ "key_id": doomed_key_id }).to_string()),
+        &base.access_key,
+        &base.secret_key,
+    )
+    .await?;
    kms_admin_request(
        &base.url,
        http::Method::DELETE,
@@ -21,12 +21,15 @@

 #[cfg(test)]
 mod tests {
-    use crate::common::{RustFSTestEnvironment, init_logging};
+    use crate::common::{RustFSTestEnvironment, admin_create_user, init_logging};
    use aws_sdk_s3::Client;
    use aws_sdk_s3::primitives::ByteStream;
+    use aws_sdk_s3::types::{BucketVersioningStatus, Delete, ObjectIdentifier, VersioningConfiguration};
    use std::error::Error;
    use tracing::info;

+    type TestResult = Result<(), Box<dyn Error + Send + Sync>>;
+
    async fn create_bucket(client: &Client, bucket: &str) -> Result<(), Box<dyn Error + Send + Sync>> {
        client.create_bucket().bucket(bucket).send().await?;
        Ok(())
@@ -90,7 +93,7 @@ mod tests {
        Ok(())
    }

-    /// Duplicate and repeated slashes after a leading slash collapse MinIO-style.
+    /// Duplicate and repeated slashes after a leading slash collapse MinIO-style across single and bulk operations.
    #[tokio::test]
    async fn test_put_object_with_duplicate_slashes_normalized() -> Result<(), Box<dyn Error + Send + Sync>> {
        init_logging();
@@ -144,8 +147,258 @@ mod tests {
            "GET after DELETE with raw key must return HTTP 404, got {error:?}"
        );

+        assert_bulk_delete_normalizes_keys(&client).await?;
+        assert_bulk_delete_authorizes_normalized_keys(&env, &client).await?;
+        assert_bulk_delete_preserves_version_identity(&client).await?;
+
        env.stop_server();
        info!("Test completed successfully");
        Ok(())
    }
+
+    async fn assert_bulk_delete_normalizes_keys(client: &Client) -> TestResult {
+        let bucket = "test-bulk-leading-slash";
+        create_bucket(client, bucket).await?;
+        let cases = [
+            ("/foo/bar", "foo/bar"),
+            ("//keyname", "keyname"),
+            ("/dir///sub//file", "dir/sub/file"),
+            ("/folder///", "folder/"),
+            ("/literal%2F+名", "literal%2F+名"),
+            ("plain", "plain"),
+        ];
+        let invalid = ["///", "/dir/../keep", "dir//keep"];
+        client
+            .put_object()
+            .bucket(bucket)
+            .key("keep")
+            .body(ByteStream::from_static(b"keep"))
+            .send()
+            .await?;
+        client
+            .put_object()
+            .bucket(bucket)
+            .key("dir/keep")
+            .body(ByteStream::from_static(b"keep"))
+            .send()
+            .await?;
+
+        for quiet in [false, true] {
+            for (raw, _) in cases {
+                client
+                    .put_object()
+                    .bucket(bucket)
+                    .key(raw)
+                    .body(ByteStream::from_static(b""))
+                    .send()
+                    .await?;
+            }
+            let listing = client.list_objects_v2().bucket(bucket).send().await?;
+            let stored: Vec<_> = listing.contents().iter().filter_map(|object| object.key()).collect();
+            let mut expected: Vec<_> = cases
+                .iter()
+                .map(|(_, normalized)| *normalized)
+                .chain(["dir/keep", "keep"])
+                .collect();
+            expected.sort_unstable();
+            assert_eq!(stored, expected, "PUT must store the exact normalized targets before bulk deletion");
+            let objects = cases
+                .iter()
+                .map(|(raw, _)| *raw)
+                .chain(invalid)
+                .map(|key| ObjectIdentifier::builder().key(key).build())
+                .collect::<Result<Vec<_>, _>>()?;
+            let delete = Delete::builder().set_objects(Some(objects)).quiet(quiet).build()?;
+            // A second identical batch also pins idempotent deletion of absent normalized keys.
+            for _ in 0..2 {
+                let output = client.delete_objects().bucket(bucket).delete(delete.clone()).send().await?;
+                let errors: Vec<_> = output.errors().iter().map(|error| (error.key(), error.code())).collect();
+                assert_eq!(
+                    errors,
+                    invalid.map(|key| (Some(key), Some("InvalidArgument"))),
+                    "unexpected per-key errors: {output:?}"
+                );
+                let deleted: Vec<_> = output.deleted().iter().filter_map(|object| object.key()).collect();
+                let expected = if quiet {
+                    vec![]
+                } else {
+                    cases.iter().map(|(raw, _)| *raw).collect()
+                };
+                assert_eq!(deleted, expected, "responses must retain request keys, including directory slashes");
+            }
+            let listing = client.list_objects_v2().bucket(bucket).send().await?;
+            let remaining: Vec<_> = listing.contents().iter().filter_map(|object| object.key()).collect();
+            assert_eq!(
+                remaining,
+                ["dir/keep", "keep"],
+                "bulk delete must remove normalized objects without resolving dot segments"
+            );
+        }
+        Ok(())
+    }
+
+    async fn assert_bulk_delete_authorizes_normalized_keys(env: &RustFSTestEnvironment, admin: &Client) -> TestResult {
+        let bucket = "test-bulk-slash-policy";
+        let user = "bulk-slash-user";
+        admin_create_user(env, user, "bulkSlashTestSecret123!").await?;
+        create_bucket(admin, bucket).await?;
+        for key in ["allowed/item", "protected/item"] {
+            admin
+                .put_object()
+                .bucket(bucket)
+                .key(key)
+                .body(ByteStream::from_static(b"keep"))
+                .send()
+                .await?;
+        }
+        let policy = serde_json::json!({
+            "Version": "2012-10-17",
+            "Statement": [
+                {"Effect": "Allow", "Principal": {"AWS": [user]}, "Action": ["s3:DeleteObject"],
+                 "Resource": [format!("arn:aws:s3:::{bucket}/*")]},
+                {"Effect": "Deny", "Principal": {"AWS": [user]}, "Action": ["s3:DeleteObject"],
+                 "Resource": [format!("arn:aws:s3:::{bucket}/protected/*")]}
+            ]
+        });
+        admin
+            .put_bucket_policy()
+            .bucket(bucket)
+            .policy(policy.to_string())
+            .send()
+            .await?;
+        let client = env.create_s3_client_with_credentials(user, "bulkSlashTestSecret123!");
+        let output = client
+            .delete_objects()
+            .bucket(bucket)
+            .delete(
+                Delete::builder()
+                    .objects(ObjectIdentifier::builder().key("/allowed///item").build()?)
+                    .objects(ObjectIdentifier::builder().key("/protected///item").build()?)
+                    .build()?,
+            )
+            .send()
+            .await?;
+        assert_eq!(output.deleted().len(), 1, "allowed normalized key must be deleted: {output:?}");
+        assert_eq!(output.deleted()[0].key(), Some("/allowed///item"));
+        assert_eq!(output.errors().len(), 1, "normalized resource denial must apply: {output:?}");
+        assert_eq!(output.errors()[0].key(), Some("/protected///item"));
+        assert_eq!(output.errors()[0].code(), Some("AccessDenied"));
+        let listing = admin.list_objects_v2().bucket(bucket).send().await?;
+        let remaining: Vec<_> = listing.contents().iter().filter_map(|object| object.key()).collect();
+        assert_eq!(remaining, ["protected/item"], "only the authorized object may be removed");
+        let protected = admin.get_object().bucket(bucket).key("protected/item").send().await?;
+        assert_eq!(protected.body.collect().await?.into_bytes().as_ref(), b"keep");
+        Ok(())
+    }
+
+    async fn assert_bulk_delete_preserves_version_identity(client: &Client) -> TestResult {
+        let bucket = "test-bulk-slash-versions";
+        let raw = "/history///file";
+        let normalized = "history/file";
+        create_bucket(client, bucket).await?;
+        client
+            .put_bucket_versioning()
+            .bucket(bucket)
+            .versioning_configuration(
+                VersioningConfiguration::builder()
+                    .status(BucketVersioningStatus::Enabled)
+                    .build(),
+            )
+            .send()
+            .await?;
+        let first = client
+            .put_object()
+            .bucket(bucket)
+            .key(raw)
+            .body(ByteStream::from_static(b"old"))
+            .send()
+            .await?;
+        let first_id = first.version_id().ok_or("first PUT omitted version ID")?;
+        let latest = client
+            .put_object()
+            .bucket(bucket)
+            .key(raw)
+            .body(ByteStream::from_static(b"latest"))
+            .send()
+            .await?;
+        let latest_id = latest.version_id().ok_or("latest PUT omitted version ID")?;
+        let invalid = client
+            .delete_objects()
+            .bucket(bucket)
+            .delete(
+                Delete::builder()
+                    .objects(ObjectIdentifier::builder().key("///").build()?)
+                    .build()?,
+            )
+            .send()
+            .await?;
+        assert!(invalid.deleted().is_empty(), "slash-only keys must not delete a versioned object");
+        assert_eq!(
+            invalid.errors().len(),
+            1,
+            "slash-only keys must remain invalid in versioned buckets: {invalid:?}"
+        );
+        assert_eq!(invalid.errors()[0].key(), Some("///"));
+        assert_eq!(invalid.errors()[0].code(), Some("InvalidArgument"));
+        let deleted = client
+            .delete_objects()
+            .bucket(bucket)
+            .delete(
+                Delete::builder()
+                    .objects(ObjectIdentifier::builder().key(raw).version_id(first_id).build()?)
+                    .build()?,
+            )
+            .send()
+            .await?;
+        assert!(deleted.errors().is_empty(), "historical delete failed: {deleted:?}");
+        assert_eq!(deleted.deleted().len(), 1);
+        assert_eq!(deleted.deleted()[0].key(), Some(raw));
+        assert_eq!(deleted.deleted()[0].version_id(), Some(first_id));
+        let listing = client.list_object_versions().bucket(bucket).send().await?;
+        assert_eq!(listing.versions().len(), 1, "only the latest data version should remain: {listing:?}");
+        assert_eq!(listing.versions()[0].version_id(), Some(latest_id));
+
+        let deleted = client
+            .delete_objects()
+            .bucket(bucket)
+            .delete(
+                Delete::builder()
+                    .objects(ObjectIdentifier::builder().key(raw).build()?)
+                    .build()?,
+            )
+            .send()
+            .await?;
+        assert!(deleted.errors().is_empty(), "delete marker creation failed: {deleted:?}");
+        assert_eq!(deleted.deleted().len(), 1);
+        assert_eq!(deleted.deleted()[0].key(), Some(raw));
+        assert_eq!(deleted.deleted()[0].delete_marker(), Some(true));
+        let marker = deleted.deleted()[0]
+            .delete_marker_version_id()
+            .ok_or("missing delete marker ID")?;
+        let listing = client.list_object_versions().bucket(bucket).send().await?;
+        assert_eq!(listing.delete_markers().len(), 1);
+        assert_eq!(listing.delete_markers()[0].key(), Some(normalized));
+        assert_eq!(listing.delete_markers()[0].version_id(), Some(marker));
+
+        let deleted = client
+            .delete_objects()
+            .bucket(bucket)
+            .delete(
+                Delete::builder()
+                    .objects(ObjectIdentifier::builder().key(raw).version_id(marker).build()?)
+                    .build()?,
+            )
+            .send()
+            .await?;
+        assert!(deleted.errors().is_empty(), "explicit marker removal failed: {deleted:?}");
+        assert_eq!(deleted.deleted().len(), 1);
+        assert_eq!(deleted.deleted()[0].key(), Some(raw));
+        assert_eq!(deleted.deleted()[0].version_id(), Some(marker));
+        let listing = client.list_object_versions().bucket(bucket).send().await?;
+        assert!(listing.delete_markers().is_empty());
+        let restored = client.get_object().bucket(bucket).key(normalized).send().await?;
+        assert_eq!(restored.version_id(), Some(latest_id));
+        assert_eq!(restored.body.collect().await?.into_bytes().as_ref(), b"latest");
+        Ok(())
+    }
 }
@@ -31,6 +31,12 @@ pub mod chaos;
 #[cfg(test)]
 pub mod fake_s3_target;

+// Dual-stack differential (rustfs/backlog#2740): one binary, two processes on
+// different RUSTFS_S3_STACK values, the same SDK script for every S3 operation,
+// compared on status, error code, allowlisted headers and typed body fields.
+#[cfg(test)]
+mod dual_stack_differential;
+
 // On-demand migration (backlog#2147): shared two-server environment, admin
 // wrappers, and the harness self-test (backlog#2151). Behavior scenarios are
 // added by later ODM tasks.
@@ -32,7 +32,9 @@
 //! Advisory: <https://github.com/rustfs/rustfs/security/advisories/GHSA-3p3x-734c-h5vx>

 use crate::common::rustfs_binary_path_with_features;
-use crate::common::{AdminTransport, admin_add_canned_policy_via, admin_attach_user_policy_via, admin_create_user_via};
+use crate::common::{
+    AdminTransport, admin_add_canned_policy_via, admin_attach_user_policy_via, admin_create_user_via, signed_s3_request,
+};
 use crate::common::{FAST_DATA_USAGE_SCANNER_ENV, admin_request};
 use crate::protocols::test_env::{DEFAULT_ACCESS_KEY, DEFAULT_SECRET_KEY, ProtocolTestEnvironment};
 use anyhow::Result;
@@ -774,6 +776,81 @@ async fn test_webdav_core_operations_direct() -> Result<()> {
    test_webdav_core_operations().await
 }

+#[tokio::test]
+async fn test_webdav_content_type_directory_marker() -> Result<()> {
+    let env = ProtocolTestEnvironment::new().map_err(|e| anyhow::anyhow!("{}", e))?;
+    let admin_base_url = format!("http://{}", S3_TEST_ADDRESS);
+    let binary_path = rustfs_binary_path_with_features(Some("webdav"));
+    let mut server_process = Command::new(&binary_path)
+        .arg("--address")
+        .arg(S3_TEST_ADDRESS)
+        .env("RUSTFS_CONSOLE_ENABLE", "false")
+        .env("RUSTFS_WEBDAV_ENABLE", "true")
+        .env("RUSTFS_WEBDAV_ADDRESS", WEBDAV_ADDRESS)
+        .env("RUSTFS_WEBDAV_TLS_ENABLED", "false")
+        .arg(&env.temp_dir)
+        .spawn()?;
+
+    let result = async {
+        ProtocolTestEnvironment::wait_for_port_ready(WEBDAV_PORT, 30)
+            .await
+            .map_err(|e| anyhow::anyhow!("{}", e))?;
+
+        let client = create_client();
+        let bucket_name = "webdav-content-type-bucket";
+        let directory_name = "content-type-directory";
+        let webdav_base_url = format!("http://{}", WEBDAV_ADDRESS);
+        let mkcol_bucket = client
+            .request(
+                reqwest::Method::from_bytes(b"MKCOL").expect("MKCOL is a valid HTTP method"),
+                format!("{}/{}", webdav_base_url, bucket_name),
+            )
+            .header("Authorization", basic_auth_header())
+            .send()
+            .await?;
+        anyhow::ensure!(
+            mkcol_bucket.status().is_success() || mkcol_bucket.status().as_u16() == 201,
+            "MKCOL bucket should succeed, got: {}",
+            mkcol_bucket.status()
+        );
+
+        let mkcol_directory = client
+            .request(
+                reqwest::Method::from_bytes(b"MKCOL").expect("MKCOL is a valid HTTP method"),
+                format!("{}/{}/{}", webdav_base_url, bucket_name, directory_name),
+            )
+            .header("Authorization", basic_auth_header())
+            .send()
+            .await?;
+        anyhow::ensure!(
+            mkcol_directory.status().is_success() || mkcol_directory.status().as_u16() == 201,
+            "MKCOL directory should succeed, got: {}",
+            mkcol_directory.status()
+        );
+
+        let head_url = format!("{}/{}/{}/", admin_base_url, bucket_name, directory_name);
+        let head = signed_s3_request(http::Method::HEAD, &head_url, None, None, DEFAULT_ACCESS_KEY, DEFAULT_SECRET_KEY)
+            .await
+            .map_err(|error| anyhow::anyhow!("{error}"))?;
+        anyhow::ensure!(head.status().is_success(), "HEAD directory marker should succeed, got: {}", head.status());
+        let content_type = head
+            .headers()
+            .get("content-type")
+            .and_then(|value| value.to_str().ok())
+            .unwrap_or_default();
+        anyhow::ensure!(
+            content_type == "application/x-directory",
+            "WebDAV directory marker must preserve its content type, got: {content_type}"
+        );
+        Ok(())
+    }
+    .await;
+
+    let _ = server_process.kill().await;
+    let _ = server_process.wait().await;
+    result
+}
+
 /// Extract the numeric value of the first occurrence of a DAV property in a
 /// PROPFIND multistatus body. Namespace-prefix agnostic; a self-closing
 /// (unreported) property yields None.
@@ -141,6 +141,7 @@ rustfs-lifecycle.workspace = true
 rustfs-s3-client = { workspace = true }
 rustfs-s3-types = { workspace = true }
 rustfs-data-usage.workspace = true
+rustfs-gateway-types.workspace = true
 rustfs-object-capacity.workspace = true
 rustfs-object-data-cache = { workspace = true, features = ["runtime-memory"] }
 arc-swap.workspace = true
@@ -159,7 +160,6 @@ serde = { workspace = true, features = ["derive"] }
 time = { workspace = true, features = ["parsing", "formatting", "macros", "serde"] }
 bytesize.workspace = true
 serde_json = { workspace = true, features = ["raw_value"] }
-s3s = { workspace = true, features = ["minio"] }
 http.workspace = true
 opentelemetry.workspace = true
 http-body = { workspace = true }
@@ -253,6 +253,7 @@ windows-sys = { workspace = true, features = [
 windows-sys = { workspace = true, features = ["Win32_System_Ioctl"] }

 [dev-dependencies]
+s3s = { workspace = true, features = ["minio"] }
 aws-smithy-async.workspace = true
 rustfs-s3-ops.workspace = true
 tokio = { workspace = true, features = ["rt-multi-thread", "macros", "test-util", "fs"] }
@@ -66,6 +66,8 @@ pub mod bucket {
        }

        pub mod bucket_lifecycle_ops {
+            pub use rustfs_lifecycle::{RestoreOutputLocation, RestoreRequest, RestoreS3Location, RestoreTag, RestoreTagging};
+
            pub use crate::bucket::lifecycle::bucket_lifecycle_ops::{
                ExpiryState, LifecycleOps, ManualTransitionCancelCheck, ManualTransitionProgressSink,
                ManualTransitionQueueSnapshot, ManualTransitionRunExecution, ManualTransitionRunOptions,
@@ -137,7 +139,7 @@ pub mod bucket {
            pub use crate::bucket::lifecycle::lifecycle::{
                Event, ExpirationOptions, IlmAction, LIFECYCLE_MALFORMED_XML_ERROR_KIND, Lifecycle, LifecycleCalculate,
                ObjectOpts, RuleValidate, TRANSITION_COMPLETE, TRANSITION_PENDING, TransitionOptions, expected_expiry_time,
-                object_opts_from_object_info,
+                format_lifecycle_date, object_opts_from_object_info, parse_lifecycle_date,
            };
        }

@@ -240,7 +242,7 @@ pub mod bucket {
    }

    pub mod object_lock {
-        pub use crate::bucket::object_lock::{ObjectLockApi, ObjectLockStatusExt};
+        pub use crate::bucket::object_lock::ObjectLockApi;

        pub mod types {
            pub use crate::bucket::object_lock::types::{
@@ -335,9 +337,9 @@ pub mod bucket {

    pub mod utils {
        pub use crate::bucket::utils::{
-            check_bucket_and_object_names, check_list_objs_args, check_object_name_for_length_and_slash,
-            check_valid_bucket_name_strict, deserialize, has_bad_path_component, has_xml_invalid_control_characters,
-            is_meta_bucketname, is_valid_object_prefix, serialize,
+            BucketConfigXml, BucketConfigXmlError, check_bucket_and_object_names, check_list_objs_args,
+            check_object_name_for_length_and_slash, check_valid_bucket_name_strict, deserialize, has_bad_path_component,
+            has_xml_invalid_control_characters, is_meta_bucketname, is_valid_object_prefix, serialize,
        };
    }

@@ -364,7 +366,7 @@ pub mod capacity {
 }

 pub mod object_api_utils {
-    pub use crate::object_api::object_api_utils::{ObjReaderFn, PutObjReader, get_raw_etag, new_getobjectreader, to_s3s_etag};
+    pub use crate::object_api::object_api_utils::{ObjReaderFn, PutObjReader, get_raw_etag, new_getobjectreader};
 }

 pub mod cluster {
@@ -88,12 +88,14 @@ use rustfs_filemeta::{
    FileInfo, FileInfoOpts, NULL_VERSION_ID, RestoreStatus, RestoreStatusOps, TRANSITION_COMPLETE, get_file_info,
    is_restored_object_on_disk,
 };
+use rustfs_gateway_types::persistence::PersistedObjectLockConfiguration as ObjectLockConfiguration;
+use rustfs_gateway_types::persistence::{PersistedLifecycleConfiguration, PersistedLifecycleFilter};
+use rustfs_lifecycle::{RestoreRequest, RestoreS3Location};
 use rustfs_lock::{NamespaceLockGuard, error::LockError};
 use rustfs_scanner_metrics::metrics::{
    IlmAction, Metrics, ScannerLifecycleExpiryStateUpdate, ScannerLifecycleTransitionStateUpdate, global_metrics,
 };
 use rustfs_utils::{get_env_i64, get_env_usize, path::encode_dir_object, string::parse_bool};
-use s3s::dto::{BucketLifecycleConfiguration, ExpirationStatus, ObjectLockConfiguration, RestoreRequest, RestoreRequestType};
 use sha2::{Digest, Sha256};
 use std::any::Any;
 use std::collections::{BTreeMap, HashMap, HashSet};
@@ -300,7 +302,7 @@ impl LifecycleSys {
        Arc::new(Self)
    }

-    pub async fn get(&self, bucket: &str) -> Option<BucketLifecycleConfiguration> {
+    pub async fn get(&self, bucket: &str) -> Option<PersistedLifecycleConfiguration> {
        match metadata_boundary::get_lifecycle_config(bucket).await {
            Ok((lc, _)) => Some(lc),
            Err(Error::ConfigNotFound) => None,
@@ -3557,7 +3559,7 @@ async fn stale_upload_lifecycle_due(
        Err(_) => return None,
    };

-    let has_size_filter = |filter: &s3s::dto::LifecycleRuleFilter| {
+    let has_size_filter = |filter: &PersistedLifecycleFilter| {
        filter.object_size_greater_than.is_some()
            || filter.object_size_less_than.is_some()
            || filter
@@ -3566,7 +3568,7 @@ async fn stale_upload_lifecycle_due(
                .is_some_and(|and| and.object_size_greater_than.is_some() || and.object_size_less_than.is_some())
    };
    let needs_size = lifecycle.rules.iter().any(|rule| {
-        rule.status.as_str() != ExpirationStatus::DISABLED
+        rule.status.as_str() != "Disabled"
            && rule.abort_incomplete_multipart_upload.is_some()
            && rule.filter.as_ref().is_some_and(has_size_filter)
    });
@@ -4000,7 +4002,7 @@ pub fn init_background_stale_multipart_upload_cleanup(api: Arc<ECStore>) {
    });
 }

-pub async fn validate_transition_tier(lc: &BucketLifecycleConfiguration) -> Result<(), std::io::Error> {
+pub async fn validate_transition_tier(lc: &PersistedLifecycleConfiguration) -> Result<(), std::io::Error> {
    for rule in &lc.rules {
        if let Some(transitions) = &rule.transitions {
            for transition in transitions {
@@ -4117,8 +4119,9 @@ pub(crate) async fn enqueue_immediate_expiry(api: Arc<ECStore>, oi: &ObjectInfo,
        .map(lifecycle::object_opts_from_object_info)
        .collect::<Vec<ObjectOpts>>();
    let lock_config = configs.object_lock;
+    let evaluator_lock_config = lock_config.clone();
    let events = match Evaluator::new(lifecycle)
-        .with_lock_retention(lock_config.clone())
+        .with_lock_retention(evaluator_lock_config)
        .eval(&object_opts)
        .await
    {
@@ -4598,19 +4601,26 @@ pub async fn enqueue_transition_for_existing_objects_scoped(
    }
 }

-fn lifecycle_rule_has_date_expiration(lc: &BucketLifecycleConfiguration, rule_id: &str) -> bool {
+fn lifecycle_rule_has_date_expiration(lc: &PersistedLifecycleConfiguration, rule_id: &str) -> bool {
    lc.rules.iter().any(|rule| {
-        rule.status == ExpirationStatus::from_static(ExpirationStatus::ENABLED)
+        rule.status == "Enabled"
            && rule.id.as_deref() == Some(rule_id)
            && rule.expiration.as_ref().is_some_and(|expiration| expiration.date.is_some())
    })
 }

-fn should_defer_date_expiry_for_recent_config_update(lc: &BucketLifecycleConfiguration, now: OffsetDateTime) -> bool {
-    lc.expiry_updated_at.as_ref().is_some_and(|updated_at| {
-        let updated_at = OffsetDateTime::from(updated_at.clone());
-        now.unix_timestamp().saturating_sub(updated_at.unix_timestamp()) < DATE_EXPIRY_EXISTING_OBJECTS_GRACE_SECS
-    })
+fn should_defer_date_expiry_for_recent_config_update(lc: &PersistedLifecycleConfiguration, now: OffsetDateTime) -> bool {
+    lc.expiry_updated_at
+        .as_ref()
+        .is_some_and(|updated_at| match rustfs_lifecycle::parse_lifecycle_date(updated_at) {
+            Ok(updated_at) => {
+                now.unix_timestamp().saturating_sub(updated_at.unix_timestamp()) < DATE_EXPIRY_EXISTING_OBJECTS_GRACE_SECS
+            }
+            Err(error) => {
+                warn!(error = %error, "Deferring date expiry with an unreadable update timestamp");
+                true
+            }
+        })
 }

 async fn apply_existing_object_expiry(
@@ -4630,7 +4640,7 @@ async fn apply_existing_object_expiry(
 struct ExistingObjectExpiryContext<'a> {
    api: Arc<ECStore>,
    bucket: &'a str,
-    lc: Arc<BucketLifecycleConfiguration>,
+    lc: Arc<PersistedLifecycleConfiguration>,
    lock_config: Option<Arc<ObjectLockConfiguration>>,
    bucket_incarnation_id: Uuid,
    src: &'a LcEventSrc,
@@ -4650,8 +4660,9 @@ async fn enqueue_expiry_for_existing_object_group(
        .iter()
        .map(lifecycle::object_opts_from_object_info)
        .collect::<Vec<ObjectOpts>>();
+    let evaluator_lock_config = context.lock_config.clone();
    let events = match Evaluator::new(context.lc.clone())
-        .with_lock_retention(context.lock_config.clone())
+        .with_lock_retention(evaluator_lock_config)
        .eval(&object_opts)
        .await
    {
@@ -4845,7 +4856,7 @@ fn manual_transition_version_marker(oi: &ObjectInfo) -> String {
 async fn enqueue_transition_with_lifecycle_report(
    api: Option<Arc<ECStore>>,
    oi: &ObjectInfo,
-    lc: &BucketLifecycleConfiguration,
+    lc: &PersistedLifecycleConfiguration,
    src: &LcEventSrc,
    options: &ManualTransitionRunOptions,
    report: &mut ManualTransitionRunReport,
@@ -4911,7 +4922,7 @@ async fn enqueue_transition_with_lifecycle_report(
    false
 }

-async fn enqueue_transition_with_lifecycle(oi: &ObjectInfo, lc: &BucketLifecycleConfiguration, src: &LcEventSrc) -> bool {
+async fn enqueue_transition_with_lifecycle(oi: &ObjectInfo, lc: &PersistedLifecycleConfiguration, src: &LcEventSrc) -> bool {
    let event = lc.eval(&oi.to_lifecycle_opts()).await;
    match event.action {
        IlmAction::TransitionAction | IlmAction::TransitionVersionAction => {
@@ -5425,12 +5436,8 @@ pub async fn post_restore_opts(version_id: &str, bucket: &str, object: &str) ->
    })
 }

-fn select_restore_s3_location(rreq: &RestoreRequest) -> Result<Option<&s3s::dto::S3Location>, std::io::Error> {
-    if rreq
-        .type_
-        .as_ref()
-        .is_none_or(|type_| type_.as_str() != RestoreRequestType::SELECT)
-    {
+fn select_restore_s3_location(rreq: &RestoreRequest) -> Result<Option<&RestoreS3Location>, std::io::Error> {
+    if rreq.type_.as_ref().is_none_or(|type_| type_.as_str() != "SELECT") {
        return Ok(None);
    }
    let output_location = rreq
@@ -5472,21 +5479,20 @@ pub async fn put_restore_opts(
    // the OutputLocation metadata instead of a write to `OutputLocation.S3`
    // (backlog#1341). RestoreObject rejects SELECT at the API boundary; this
    // is the fail-closed backstop for any other caller.
-    if rreq
-        .type_
-        .as_ref()
-        .is_some_and(|type_| type_.as_str() == RestoreRequestType::SELECT)
-    {
+    if rreq.type_.as_ref().is_some_and(|type_| type_.as_str() == "SELECT") {
        return Err(std::io::Error::other("SELECT restore requests are not supported"));
    }
    for (k, v) in oi.user_defined.iter() {
        meta.insert(k.to_string(), v.clone());
    }
+    // The fleet capability gate guarantees physical copy-back support. Keep
+    // the completed transition identity for orphan recovery and remote cleanup.
    rustfs_utils::http::metadata_compat::remove_str(&mut meta, rustfs_utils::http::metadata_compat::SUFFIX_RESTORE_OPERATION_ID);
-    rustfs_utils::http::metadata_compat::remove_str(&mut meta, rustfs_utils::http::metadata_compat::SUFFIX_RESTORE_WORKER_LOCK);
    if !oi.user_tags.is_empty() {
        meta.insert(AMZ_OBJECT_TAGGING.to_string(), (*oi.user_tags).clone());
    }
+    // Finalization publishes completion and its expiry under the object lock.
+    // The local copy-back commit must not make unfinished work eligible for expiry.
    if let Some(expires) = oi.expires {
        meta.insert(
            "expires".to_string(),
@@ -5495,12 +5501,11 @@ pub async fn put_restore_opts(
                .map_err(std::io::Error::other)?,
        );
    }
-    let restore_expiry = lifecycle::expected_expiry_time(OffsetDateTime::now_utc(), rreq.days.unwrap_or(1));
    meta.insert(
        metadata_keys::RESTORE.to_string(),
        RestoreStatus {
-            is_restore_in_progress: Some(false),
-            restore_expiry_date: Some(restore_expiry),
+            is_restore_in_progress: Some(true),
+            restore_expiry_date: None,
        }
        .to_string(),
    );
@@ -5514,6 +5519,7 @@ pub async fn put_restore_opts(
        // computed MD5 is not the object's public plaintext ETag.
        preserve_etag: oi.etag.clone(),
        shard_integrity_write_mode: Some(oi.shard_integrity_write_mode()),
+        restore_copyback: true,
        ..Default::default()
    })
 }
@@ -5532,6 +5538,18 @@ impl LifecycleOps for ObjectInfo {
        if self.transitioned_object.status != lifecycle::TRANSITION_COMPLETE {
            return false;
        }
+        if rustfs_utils::http::metadata_compat::get_consistent_str(
+            &self.user_defined,
+            rustfs_utils::http::metadata_compat::SUFFIX_RESTORE_COPYBACK,
+        ) == Some(rustfs_utils::http::metadata_compat::RESTORE_COPYBACK_PROTOCOL_V1)
+            && self
+                .user_defined
+                .get(metadata_keys::RESTORE)
+                .and_then(|value| rustfs_filemeta::parse_restore_obj_status(value).ok())
+                .is_some_and(|status| status.is_restore_in_progress == Some(true))
+        {
+            return false;
+        }
        !is_restored_object_on_disk(&self.user_defined)
    }
 }
@@ -5543,25 +5561,25 @@ pub trait RestoreRequestOps {
 impl RestoreRequestOps for RestoreRequest {
    fn validate(&self, _api: Arc<ECStore>) -> Result<(), std::io::Error> {
        // SELECT type requires select_parameters, and vice versa
-        if self.type_.as_ref().is_none_or(|t| t.as_str() != RestoreRequestType::SELECT) && self.select_parameters.is_some() {
+        if self.type_.as_ref().is_none_or(|t| t.as_str() != "SELECT") && self.select_parameters.is_some() {
            return Err(std::io::Error::other("Select parameters can only be specified with SELECT request type"));
        }
        if let Some(type_) = &self.type_
-            && type_.as_str() == RestoreRequestType::SELECT
+            && type_.as_str() == "SELECT"
            && self.select_parameters.is_none()
        {
            return Err(std::io::Error::other("SELECT restore request requires select parameters to be specified"));
        }

        // OutputLocation is only valid for SELECT requests
-        if self.type_.as_ref().is_none_or(|t| t.as_str() != RestoreRequestType::SELECT) && self.output_location.is_some() {
+        if self.type_.as_ref().is_none_or(|t| t.as_str() != "SELECT") && self.output_location.is_some() {
            return Err(std::io::Error::other("OutputLocation can only be specified with SELECT request type"));
        }
        select_restore_s3_location(self)?;

        // Days must not be specified with SELECT requests
        if let Some(type_) = &self.type_
-            && type_.as_str() == RestoreRequestType::SELECT
+            && type_.as_str() == "SELECT"
            && self.days.is_some_and(|d| d > 0)
        {
            return Err(std::io::Error::other("Days cannot be specified with SELECT restore request"));
@@ -5603,7 +5621,7 @@ pub fn lifecycle_version_delete_target(oi: &ObjectInfo) -> Option<ObjectToDelete
 }

 pub async fn eval_action_from_lifecycle(
-    lc: &BucketLifecycleConfiguration,
+    lc: &PersistedLifecycleConfiguration,
    lock_config: Option<&ObjectLockConfiguration>,
    oi: &ObjectInfo,
 ) -> lifecycle::Event {
@@ -6308,14 +6326,19 @@ mod tests {
    use rustfs_data_usage::TierStats;
    use rustfs_filemeta::metadata_keys;
    use rustfs_filemeta::{FileInfo, FileMeta};
+    use rustfs_gateway_types::dto::MetadataEntry;
+    use rustfs_gateway_types::persistence::{
+        PersistedDefaultRetention as DefaultRetention, PersistedObjectLockConfiguration as ObjectLockConfiguration,
+        PersistedObjectLockRule as ObjectLockRule,
+    };
+    use rustfs_gateway_types::persistence::{
+        PersistedLifecycleConfiguration, PersistedLifecycleExpiration, PersistedLifecycleRule,
+        PersistedNoncurrentVersionExpiration, PersistedTransition,
+    };
+    use rustfs_lifecycle::{RestoreOutputLocation as OutputLocation, RestoreRequest, RestoreS3Location as S3Location};
    #[cfg(feature = "test-util")]
    use rustfs_s3_client::transition_api::ReaderImpl;
    use rustfs_scanner_metrics::metrics::{IlmAction, global_metrics};
-    use s3s::dto::{
-        BucketLifecycleConfiguration, DefaultRetention, ExpirationStatus, LifecycleExpiration, LifecycleRule, MetadataEntry,
-        NoncurrentVersionExpiration, ObjectLockConfiguration, ObjectLockEnabled, ObjectLockRetentionMode, ObjectLockRule,
-        OutputLocation, RestoreRequest, RestoreRequestType, S3Location, Timestamp, Transition, TransitionStorageClass,
-    };
    use serial_test::serial;
    use sha2::{Digest, Sha256};
    use std::collections::HashMap;
@@ -9341,7 +9364,7 @@ mod tests {
            output_location,
            select_parameters: None,
            tier: None,
-            type_: Some(RestoreRequestType::from_static(RestoreRequestType::SELECT)),
+            type_: Some("SELECT".to_owned()),
        }
    }

@@ -9658,12 +9681,12 @@ mod tests {

    #[test]
    fn lifecycle_rule_has_date_expiration_detects_enabled_date_rule() {
-        let lc = BucketLifecycleConfiguration {
+        let lc = PersistedLifecycleConfiguration {
            expiry_updated_at: None,
-            rules: vec![LifecycleRule {
-                status: ExpirationStatus::from_static(ExpirationStatus::ENABLED),
-                expiration: Some(LifecycleExpiration {
-                    date: Some(Timestamp::from(OffsetDateTime::now_utc())),
+            rules: vec![PersistedLifecycleRule {
+                status: "Enabled".to_owned(),
+                expiration: Some(PersistedLifecycleExpiration {
+                    date: Some(lifecycle::format_lifecycle_date(OffsetDateTime::now_utc()).unwrap()),
                    ..Default::default()
                }),
                id: Some("rule-date".to_string()),
@@ -10180,14 +10203,15 @@ mod tests {
    #[test]
    fn should_defer_date_expiry_for_recent_config_update_respects_grace_window() {
        let now = OffsetDateTime::now_utc();
-        let recent = BucketLifecycleConfiguration {
-            expiry_updated_at: Some(Timestamp::from(now - time::Duration::seconds(1))),
+        let recent = PersistedLifecycleConfiguration {
+            expiry_updated_at: Some(lifecycle::format_lifecycle_date(now - time::Duration::seconds(1)).unwrap()),
            rules: Vec::new(),
        };
-        let stale = BucketLifecycleConfiguration {
-            expiry_updated_at: Some(Timestamp::from(
-                now - time::Duration::seconds(DATE_EXPIRY_EXISTING_OBJECTS_GRACE_SECS + 1),
-            )),
+        let stale = PersistedLifecycleConfiguration {
+            expiry_updated_at: Some(
+                lifecycle::format_lifecycle_date(now - time::Duration::seconds(DATE_EXPIRY_EXISTING_OBJECTS_GRACE_SECS + 1))
+                    .unwrap(),
+            ),
            rules: Vec::new(),
        };

@@ -10315,12 +10339,12 @@ mod tests {
        assert_eq!(deleted.version_id, None);
    }

-    fn expired_delete_marker_lifecycle() -> BucketLifecycleConfiguration {
-        BucketLifecycleConfiguration {
+    fn expired_delete_marker_lifecycle() -> PersistedLifecycleConfiguration {
+        PersistedLifecycleConfiguration {
            expiry_updated_at: None,
-            rules: vec![LifecycleRule {
-                status: ExpirationStatus::from_static(ExpirationStatus::ENABLED),
-                expiration: Some(LifecycleExpiration {
+            rules: vec![PersistedLifecycleRule {
+                status: "Enabled".to_owned(),
+                expiration: Some(PersistedLifecycleExpiration {
                    expired_object_delete_marker: Some(true),
                    ..Default::default()
                }),
@@ -10336,12 +10360,12 @@ mod tests {
        }
    }

-    fn latest_expiration_lifecycle() -> BucketLifecycleConfiguration {
-        BucketLifecycleConfiguration {
+    fn latest_expiration_lifecycle() -> PersistedLifecycleConfiguration {
+        PersistedLifecycleConfiguration {
            expiry_updated_at: None,
-            rules: vec![LifecycleRule {
-                status: ExpirationStatus::from_static(ExpirationStatus::ENABLED),
-                expiration: Some(LifecycleExpiration {
+            rules: vec![PersistedLifecycleRule {
+                status: "Enabled".to_owned(),
+                expiration: Some(PersistedLifecycleExpiration {
                    days: Some(1),
                    ..Default::default()
                }),
@@ -10357,12 +10381,12 @@ mod tests {
        }
    }

-    fn current_and_noncurrent_expiration_lifecycle() -> BucketLifecycleConfiguration {
-        BucketLifecycleConfiguration {
+    fn current_and_noncurrent_expiration_lifecycle() -> PersistedLifecycleConfiguration {
+        PersistedLifecycleConfiguration {
            expiry_updated_at: None,
-            rules: vec![LifecycleRule {
-                status: ExpirationStatus::from_static(ExpirationStatus::ENABLED),
-                expiration: Some(LifecycleExpiration {
+            rules: vec![PersistedLifecycleRule {
+                status: "Enabled".to_owned(),
+                expiration: Some(PersistedLifecycleExpiration {
                    days: Some(30),
                    ..Default::default()
                }),
@@ -10370,7 +10394,7 @@ mod tests {
                del_marker_expiration: None,
                filter: None,
                id: Some("expire-current-and-noncurrent".to_string()),
-                noncurrent_version_expiration: Some(NoncurrentVersionExpiration {
+                noncurrent_version_expiration: Some(PersistedNoncurrentVersionExpiration {
                    noncurrent_days: Some(1),
                    newer_noncurrent_versions: None,
                }),
@@ -10381,12 +10405,12 @@ mod tests {
        }
    }

-    fn all_versions_expiration_lifecycle() -> BucketLifecycleConfiguration {
-        BucketLifecycleConfiguration {
+    fn all_versions_expiration_lifecycle() -> PersistedLifecycleConfiguration {
+        PersistedLifecycleConfiguration {
            expiry_updated_at: None,
-            rules: vec![LifecycleRule {
-                status: ExpirationStatus::from_static(ExpirationStatus::ENABLED),
-                expiration: Some(LifecycleExpiration {
+            rules: vec![PersistedLifecycleRule {
+                status: "Enabled".to_owned(),
+                expiration: Some(PersistedLifecycleExpiration {
                    days: Some(1),
                    expired_object_all_versions: Some(true),
                    ..Default::default()
@@ -10405,29 +10429,29 @@ mod tests {

    fn lock_enabled_without_default_retention() -> ObjectLockConfiguration {
        ObjectLockConfiguration {
-            object_lock_enabled: Some(ObjectLockEnabled::from_static(ObjectLockEnabled::ENABLED)),
+            object_lock_enabled: Some("Enabled".to_owned()),
            rule: None,
        }
    }

    fn lock_enabled_with_default_retention() -> ObjectLockConfiguration {
        ObjectLockConfiguration {
-            object_lock_enabled: Some(ObjectLockEnabled::from_static(ObjectLockEnabled::ENABLED)),
+            object_lock_enabled: Some("Enabled".to_owned()),
            rule: Some(ObjectLockRule {
                default_retention: Some(DefaultRetention {
                    days: Some(30),
-                    mode: Some(ObjectLockRetentionMode::from_static(ObjectLockRetentionMode::COMPLIANCE)),
+                    mode: Some("COMPLIANCE".to_owned()),
                    years: None,
                }),
            }),
        }
    }

-    fn latest_transition_lifecycle() -> BucketLifecycleConfiguration {
-        BucketLifecycleConfiguration {
+    fn latest_transition_lifecycle() -> PersistedLifecycleConfiguration {
+        PersistedLifecycleConfiguration {
            expiry_updated_at: None,
-            rules: vec![LifecycleRule {
-                status: ExpirationStatus::from_static(ExpirationStatus::ENABLED),
+            rules: vec![PersistedLifecycleRule {
+                status: "Enabled".to_owned(),
                expiration: None,
                abort_incomplete_multipart_upload: None,
                del_marker_expiration: None,
@@ -10436,10 +10460,10 @@ mod tests {
                noncurrent_version_expiration: None,
                noncurrent_version_transitions: None,
                prefix: None,
-                transitions: Some(vec![Transition {
+                transitions: Some(vec![PersistedTransition {
                    days: Some(1),
                    date: None,
-                    storage_class: Some(TransitionStorageClass::from_static("WARM")),
+                    storage_class: Some("WARM".to_owned()),
                }]),
            }],
        }
@@ -12976,8 +13000,9 @@ mod tests {

        for (name, object, lock_config, expected) in cases {
            let object_opts = lifecycle::object_opts_from_object_info(&object);
+            let evaluator_lock_config = lock_config.clone().map(Arc::new);
            let batch_event = Evaluator::new(Arc::new(lifecycle.clone()))
-                .with_lock_retention(lock_config.clone().map(Arc::new))
+                .with_lock_retention(evaluator_lock_config)
                .eval(&[object_opts])
                .await
                .expect("batch lifecycle evaluation should succeed")
@@ -14234,10 +14259,7 @@ mod tests {
        let object = current_object_with_metadata(
            ReplicationStatusType::Completed,
            HashMap::from([
-                (
-                    metadata_keys::OBJECT_LOCK_MODE.to_string(),
-                    s3s::dto::ObjectLockRetentionMode::COMPLIANCE.to_string(),
-                ),
+                (metadata_keys::OBJECT_LOCK_MODE.to_string(), "COMPLIANCE".to_string()),
                (metadata_keys::OBJECT_LOCK_RETAIN_UNTIL_DATE.to_string(), retain_until),
            ]),
        );
@@ -14249,7 +14271,7 @@ mod tests {

    #[tokio::test]
    async fn restored_copy_expiry_is_not_blocked_by_retention() {
-        let lifecycle = BucketLifecycleConfiguration {
+        let lifecycle = PersistedLifecycleConfiguration {
            expiry_updated_at: None,
            rules: Vec::new(),
        };
@@ -14259,10 +14281,7 @@ mod tests {
        let mut object = current_object_with_metadata(
            ReplicationStatusType::Completed,
            HashMap::from([
-                (
-                    metadata_keys::OBJECT_LOCK_MODE.to_string(),
-                    ObjectLockRetentionMode::COMPLIANCE.to_string(),
-                ),
+                (metadata_keys::OBJECT_LOCK_MODE.to_string(), "COMPLIANCE".to_string()),
                (metadata_keys::OBJECT_LOCK_RETAIN_UNTIL_DATE.to_string(), retain_until),
            ]),
        );
@@ -14479,7 +14498,7 @@ mod tests {
                "removed" => {
                    metadata_sys::delete(&bucket, BUCKET_LIFECYCLE_CONFIG).await.unwrap();
                }
-                "disabled" => changed.rules[0].status = ExpirationStatus::from_static(ExpirationStatus::DISABLED),
+                "disabled" => changed.rules[0].status = "Disabled".to_owned(),
                "longer-days" => {
                    changed.rules[0]
                        .noncurrent_version_expiration
@@ -14595,7 +14614,7 @@ mod tests {
        );
    }

-    async fn install_test_expiry_policy(ecstore: &Arc<ECStore>, bucket: &str) -> Arc<BucketLifecycleConfiguration> {
+    async fn install_test_expiry_policy(ecstore: &Arc<ECStore>, bucket: &str) -> Arc<PersistedLifecycleConfiguration> {
        let xml = br#"<LifecycleConfiguration><Rule><ID>expire</ID><Status>Enabled</Status><Filter><Prefix></Prefix></Filter><Expiration><Days>0</Days></Expiration><NoncurrentVersionExpiration><NoncurrentDays>0</NoncurrentDays></NoncurrentVersionExpiration></Rule></LifecycleConfiguration>"#;
        metadata_sys::update_in(&ecstore.ctx, bucket, BUCKET_LIFECYCLE_CONFIG, xml.to_vec())
            .await
@@ -15288,7 +15307,11 @@ mod tests {
    #[tokio::test]
    #[serial]
    async fn tier_free_version_recovery_production_entrypoint_enqueues_seeded_item() {
-        let (disk_paths, ecstore) = setup_test_env().await;
+        let (disk_paths, ecstore) =
+            temp_env::async_with_vars([(super::ENV_TIER_FREE_VERSION_RECOVERY_ENABLED, Some("false"))], async {
+                setup_test_env().await
+            })
+            .await;
        let tier = format!("RECOVERY{}", Uuid::new_v4().simple()).to_uppercase();
        crate::services::tier::test_util::register_mock_tier(&ecstore.tier_config_mgr(), &tier).await;
        let runtime_state = install_unconsumed_runtime_expiry_worker(&ecstore, 1).await;
@@ -17,7 +17,7 @@ use crate::object_api::ObjectInfo;
 pub use rustfs_lifecycle::{
    Event, ExpirationOptions, IlmAction, LIFECYCLE_MALFORMED_XML_ERROR_KIND, Lifecycle, LifecycleCalculate, ObjectOpts,
    RuleValidate, TRANSITION_COMPLETE, TRANSITION_PENDING, TransitionOptions, abort_incomplete_multipart_upload_due,
-    expected_expiry_time, expiration_action_has_valid_target,
+    expected_expiry_time, expiration_action_has_valid_target, format_lifecycle_date, parse_lifecycle_date,
 };

 pub fn object_opts_from_object_info(oi: &ObjectInfo) -> ObjectOpts {
@@ -14,7 +14,8 @@

 use std::sync::Arc;

-use s3s::dto::{BucketLifecycleConfiguration, ObjectLockConfiguration};
+use rustfs_gateway_types::persistence::PersistedLifecycleConfiguration;
+use rustfs_gateway_types::persistence::PersistedObjectLockConfiguration as ObjectLockConfiguration;
 use time::OffsetDateTime;
 use uuid::Uuid;

@@ -24,7 +25,7 @@ use crate::error::{Error, Result};

 #[derive(Debug)]
 pub(crate) struct LifecycleExpiryConfigs {
-    pub(crate) lifecycle: Option<Arc<BucketLifecycleConfiguration>>,
+    pub(crate) lifecycle: Option<Arc<PersistedLifecycleConfiguration>>,
    pub(crate) object_lock: Option<Arc<ObjectLockConfiguration>>,
    pub(crate) bucket_incarnation_id: Uuid,
    pub(crate) table_bucket_enabled: bool,
@@ -100,7 +101,7 @@ pub(crate) async fn get_expiry_configs_for_options(
    })
 }

-pub(crate) async fn get_lifecycle_config(bucket: &str) -> Result<(BucketLifecycleConfiguration, OffsetDateTime)> {
+pub(crate) async fn get_lifecycle_config(bucket: &str) -> Result<(PersistedLifecycleConfiguration, OffsetDateTime)> {
    metadata_sys::get_lifecycle_config(bucket).await
 }

@@ -110,15 +111,15 @@ mod tests {
    use crate::bucket::metadata::BucketMetadata;
    use crate::bucket::metadata_sys::{self, test_support::isolated_store_over_temp_disks};
    use crate::storage_api_contracts::bucket::MakeBucketOptions;
-    use s3s::dto::{ExpirationStatus, LifecycleExpiration, LifecycleRule};
+    use rustfs_gateway_types::persistence::{PersistedLifecycleExpiration, PersistedLifecycleRule};
    use serial_test::serial;

-    fn lifecycle_config() -> BucketLifecycleConfiguration {
-        BucketLifecycleConfiguration {
+    fn lifecycle_config() -> PersistedLifecycleConfiguration {
+        PersistedLifecycleConfiguration {
            expiry_updated_at: None,
-            rules: vec![LifecycleRule {
-                status: ExpirationStatus::from_static(ExpirationStatus::ENABLED),
-                expiration: Some(LifecycleExpiration {
+            rules: vec![PersistedLifecycleRule {
+                status: "Enabled".to_owned(),
+                expiration: Some(PersistedLifecycleExpiration {
                    days: Some(1),
                    ..Default::default()
                }),
@@ -22,7 +22,7 @@ pub(crate) fn is_object_locked_by_metadata(user_defined: &HashMap<String, String
 }

 pub(crate) fn check_object_lock_for_deletion_with_config(
-    config: Option<&s3s::dto::ObjectLockConfiguration>,
+    config: Option<&rustfs_gateway_types::persistence::PersistedObjectLockConfiguration>,
    obj_info: &ObjectInfo,
    bypass_governance: bool,
 ) -> crate::error::Result<Option<ObjectLockBlockReason>> {
@@ -82,12 +82,13 @@ mod tests {
    use crate::bucket::replication::{DeleteReplicationConfigSnapshot, ReplicationObjectBridge};
    use crate::object_api::{ObjectInfo, ObjectOptions};
    use crate::storage_api_contracts::object::ObjectToDelete;
-    use rustfs_scanner_metrics::metrics::IlmAction;
-    use s3s::dto::{
-        BucketVersioningStatus, DeleteMarkerReplication, DeleteMarkerReplicationStatus, DeleteReplication,
-        DeleteReplicationStatus, Destination, ReplicationConfiguration, ReplicationRule, ReplicationRuleStatus,
-        VersioningConfiguration,
+    use rustfs_gateway_types::dto::Status;
+    use rustfs_gateway_types::persistence::PersistedVersioningConfiguration as VersioningConfiguration;
+    use rustfs_gateway_types::persistence::{
+        PersistedOptionalReplicationStatus, PersistedReplicationConfiguration, PersistedReplicationDestination,
+        PersistedReplicationRule, PersistedReplicationStatus,
    };
+    use rustfs_scanner_metrics::metrics::IlmAction;
    use uuid::Uuid;

    use super::*;
@@ -160,27 +161,27 @@ mod tests {
            for purge_enabled in [false, true] {
                let snapshot = DeleteReplicationConfigSnapshot::from_configs_for_test(
                    VersioningConfiguration {
-                        status: Some(BucketVersioningStatus::from_static(BucketVersioningStatus::ENABLED)),
+                        status: Some("Enabled".to_owned()),
                        ..Default::default()
                    },
-                    Some(ReplicationConfiguration {
+                    Some(PersistedReplicationConfiguration {
                        role: String::new(),
-                        rules: vec![ReplicationRule {
-                            delete_marker_replication: Some(DeleteMarkerReplication {
+                        rules: vec![PersistedReplicationRule {
+                            delete_marker_replication: Some(PersistedOptionalReplicationStatus {
                                status: Some(if marker_enabled {
-                                    DeleteMarkerReplicationStatus::from_static(DeleteMarkerReplicationStatus::ENABLED)
+                                    Status::ENABLED.to_string()
                                } else {
-                                    DeleteMarkerReplicationStatus::from_static(DeleteMarkerReplicationStatus::DISABLED)
+                                    Status::DISABLED.to_string()
                                }),
                            }),
-                            delete_replication: Some(DeleteReplication {
+                            delete_replication: Some(PersistedReplicationStatus {
                                status: if purge_enabled {
-                                    DeleteReplicationStatus::from_static(DeleteReplicationStatus::ENABLED)
+                                    Status::ENABLED.to_string()
                                } else {
-                                    DeleteReplicationStatus::from_static(DeleteReplicationStatus::DISABLED)
+                                    Status::DISABLED.to_string()
                                },
                            }),
-                            destination: Destination {
+                            destination: PersistedReplicationDestination {
                                bucket: "arn:rustfs:replication:target".to_string(),
                                ..Default::default()
                            },
@@ -190,7 +191,7 @@ mod tests {
                            prefix: Some(String::new()),
                            priority: Some(1),
                            source_selection_criteria: None,
-                            status: ReplicationRuleStatus::from_static(ReplicationRuleStatus::ENABLED),
+                            status: Status::ENABLED.to_string(),
                        }],
                    }),
                );
@@ -14,7 +14,7 @@

 use std::sync::Arc;

-use s3s::dto::BucketLifecycleConfiguration;
+use rustfs_gateway_types::persistence::PersistedLifecycleConfiguration;
 use tokio::sync::RwLock;
 use tokio_util::sync::CancellationToken;

@@ -52,7 +52,7 @@ pub(crate) fn deployment_id() -> Option<String> {
    sources::deployment_id()
 }

-pub(crate) async fn bucket_lifecycle_config(bucket: &str) -> Option<BucketLifecycleConfiguration> {
+pub(crate) async fn bucket_lifecycle_config(bucket: &str) -> Option<PersistedLifecycleConfiguration> {
    sources::bucket_lifecycle_config(bucket).await
 }

@@ -13,7 +13,7 @@
 // limitations under the License.

 use super::msgp_decode::{read_msgp_ext8_time, skip_msgp_value, write_msgp_time};
-use super::object_lock::{ObjectLockApi, ObjectLockStatusExt};
+use super::object_lock::ObjectLockApi;
 use super::versioning::VersioningApi;
 use super::{quota::BucketQuota, target::BucketTargets};
 use crate::bucket::replication::invalid_replication_config_status_field;
@@ -25,12 +25,17 @@ use crate::object_api::WriteCompletion;
 use crate::runtime::sources as runtime_sources;
 use crate::store::ECStore;
 use byteorder::{BigEndian, ByteOrder, LittleEndian};
-use rustfs_policy::policy::BucketPolicy;
-use s3s::dto::{
-    AccelerateConfiguration, BucketLifecycleConfiguration, BucketLoggingStatus, BucketVersioningStatus, CORSConfiguration,
-    NotificationConfiguration, ObjectLockConfiguration, PublicAccessBlockConfiguration, ReplicationConfiguration,
-    RequestPaymentConfiguration, ServerSideEncryptionConfiguration, Tagging, VersioningConfiguration, WebsiteConfiguration,
+use rustfs_gateway_types::cors_tagging::{PersistedCorsConfiguration as CORSConfiguration, PersistedTagging as Tagging};
+use rustfs_gateway_types::persistence::{
+    PersistedAccelerateConfiguration as AccelerateConfiguration,
+    PersistedBucketEncryptionConfiguration as ServerSideEncryptionConfiguration,
+    PersistedBucketLoggingStatus as BucketLoggingStatus, PersistedLifecycleConfiguration,
+    PersistedNotificationConfiguration as NotificationConfiguration, PersistedObjectLockConfiguration as ObjectLockConfiguration,
+    PersistedPublicAccessBlockConfiguration as PublicAccessBlockConfiguration, PersistedReplicationConfiguration,
+    PersistedRequestPaymentConfiguration as RequestPaymentConfiguration,
+    PersistedVersioningConfiguration as VersioningConfiguration, PersistedWebsiteConfiguration as WebsiteConfiguration,
 };
+use rustfs_policy::policy::BucketPolicy;
 use serde::Serializer;
 use sha2::{Digest, Sha256};
 use std::collections::HashMap;
@@ -40,23 +45,11 @@ use time::{Date, OffsetDateTime, PrimitiveDateTime, Time as CivilTime, UtcOffset
 use tracing::error;
 use uuid::Uuid;

-// The serving-layer DTO impls for the storage-level Object Lock traits live
-// here because this module owns the persisted `ObjectLockConfiguration`
-// during the s3s ratchet migration (rustfs/backlog#1842).
+// The persisted Object Lock configuration answers the storage-level trait
+// here because this module owns the persisted `ObjectLockConfiguration`.
 impl ObjectLockApi for ObjectLockConfiguration {
    fn enabled(&self) -> bool {
-        self.object_lock_enabled
-            .as_ref()
-            .is_some_and(|v| v.as_str() == s3s::dto::ObjectLockEnabled::ENABLED)
-    }
-}
-
-impl ObjectLockStatusExt for s3s::dto::ObjectLockLegalHoldStatus {
-    fn valid(&self) -> bool {
-        matches!(
-            self.as_str(),
-            s3s::dto::ObjectLockLegalHoldStatus::ON | s3s::dto::ObjectLockLegalHoldStatus::OFF
-        )
+        self.object_lock_enabled.as_deref() == Some("Enabled")
    }
 }

@@ -257,12 +250,14 @@ pub const BUCKET_NOTIFICATION_CONFIG: &str = "notification.xml";
 pub const BUCKET_LIFECYCLE_CONFIG: &str = "lifecycle.xml";
 pub const BUCKET_SSECONFIG: &str = "bucket-encryption.xml";
 /// Validate tag entries supplied to the S3 API.
-pub fn validate_tag_set_entries(tag_set: &[s3s::dto::Tag]) -> std::result::Result<(), &'static str> {
+pub fn validate_tag_set_entries(
+    tag_set: &[rustfs_gateway_types::cors_tagging::PersistedTag],
+) -> std::result::Result<(), &'static str> {
    validate_tag_entries(tag_set, validate_s3_tag_entry)
 }

 fn validate_tag_entries(
-    tag_set: &[s3s::dto::Tag],
+    tag_set: &[rustfs_gateway_types::cors_tagging::PersistedTag],
    mut validate_entry: impl FnMut(&str, &str) -> std::result::Result<(), &'static str>,
 ) -> std::result::Result<(), &'static str> {
    let mut keys = std::collections::HashSet::with_capacity(tag_set.len());
@@ -323,7 +318,9 @@ pub fn is_swift_bucket_metadata_tag_key(key: &str) -> bool {
        || key.starts_with(SWIFT_CONTAINER_META_TAG_PREFIX)
 }

-fn validate_persisted_bucket_tags(tags: &[s3s::dto::Tag]) -> std::result::Result<(), &'static str> {
+fn validate_persisted_bucket_tags(
+    tags: &[rustfs_gateway_types::cors_tagging::PersistedTag],
+) -> std::result::Result<(), &'static str> {
    const SWIFT_ACL_MAX_BYTES: usize = 8192;
    const SWIFT_META_NAME_MAX_BYTES: usize = 128;
    const MAX_PERSISTED_TAGS: usize = MAX_BUCKET_TAGS + 2 * MAX_SWIFT_METADATA_COUNT + 4;
@@ -561,6 +558,8 @@ pub struct BucketMetadata {
    pub bucket_incarnation_id: Uuid,
    pub(crate) bucket_incarnation_sidecar: bool,
    pub(crate) bucket_creation_committed: bool,
+    // A legacy default is not proof of an interrupted creation.
+    pub(crate) bucket_creation_commit_record_present: bool,
    pub policy_config_json: Vec<u8>,
    pub notification_config_xml: Vec<u8>,
    pub lifecycle_config_xml: Vec<u8>,
@@ -609,13 +608,13 @@ pub struct BucketMetadata {

    pub policy_config: Option<BucketPolicy>,
    pub notification_config: Option<NotificationConfiguration>,
-    pub lifecycle_config: Option<BucketLifecycleConfiguration>,
+    pub lifecycle_config: Option<PersistedLifecycleConfiguration>,
    pub object_lock_config: Option<ObjectLockConfiguration>,
    pub versioning_config: Option<VersioningConfiguration>,
    pub sse_config: Option<ServerSideEncryptionConfiguration>,
    pub tagging_config: Option<Tagging>,
    pub quota_config: Option<BucketQuota>,
-    pub replication_config: Option<ReplicationConfiguration>,
+    pub replication_config: Option<PersistedReplicationConfiguration>,
    pub bucket_target_config: Option<BucketTargets>,
    pub bucket_target_config_meta: Option<HashMap<String, String>>,
    pub cors_config: Option<CORSConfiguration>,
@@ -636,6 +635,7 @@ impl Default for BucketMetadata {
            bucket_incarnation_id: Uuid::nil(),
            bucket_incarnation_sidecar: false,
            bucket_creation_committed: false,
+            bucket_creation_commit_record_present: false,
            policy_config_json: Default::default(),
            notification_config_xml: Default::default(),
            lifecycle_config_xml: Default::default(),
@@ -874,7 +874,10 @@ impl BucketMetadata {
                "TableBucketConfigUpdatedAt" => self.table_bucket_config_updated_at = read_msgp_time_value(rd)?,
                "DurabilityConfigUpdatedAt" => self.durability_config_updated_at = read_msgp_time_value(rd)?,
                "OnDemandMigrationConfigUpdatedAt" => self.on_demand_migration_config_updated_at = read_msgp_time_value(rd)?,
-                "BucketCreationCommitted" => self.bucket_creation_committed = read_msgp_bool(rd)?,
+                "BucketCreationCommitted" => {
+                    self.bucket_creation_committed = read_msgp_bool(rd)?;
+                    self.bucket_creation_commit_record_present = true;
+                }
                other => {
                    tracing::debug!(field = %other, "BucketMetadata decode_from: skipping unknown field");
                    skip_msgp_value(rd)?;
@@ -887,8 +890,10 @@ impl BucketMetadata {

    /// Encode to msgp bytes. Field order follows MinIO BucketMetadata for compatibility.
    pub fn encode_to<W: Write>(&self, wr: &mut W) -> Result<()> {
-        // Map size: MinIO fields (25) + RustFS extensions (22)
-        let map_len: u32 = 47;
+        // Preserve the absence of a legacy commit record across config rewrites.
+        let commit_record_present = self.bucket_creation_commit_record_present || self.bucket_creation_committed;
+        // Map size: MinIO fields (25) + RustFS extensions (21) + optional commit record.
+        let map_len: u32 = 46 + u32::from(commit_record_present);
        rmp::encode::write_map_len(wr, map_len)?;

        // MinIO field order (same as Go struct)
@@ -969,8 +974,10 @@ impl BucketMetadata {
        write_msgp_time(wr, self.durability_config_updated_at)?;
        rmp::encode::write_str(wr, "OnDemandMigrationConfigUpdatedAt")?;
        write_msgp_time(wr, self.on_demand_migration_config_updated_at)?;
-        rmp::encode::write_str(wr, "BucketCreationCommitted")?;
-        rmp::encode::write_bool(wr, self.bucket_creation_committed)?;
+        if commit_record_present {
+            rmp::encode::write_str(wr, "BucketCreationCommitted")?;
+            rmp::encode::write_bool(wr, self.bucket_creation_committed)?;
+        }

        Ok(())
    }
@@ -1147,9 +1154,11 @@ impl BucketMetadata {
                    None
                } else {
                    let config = deserialize::<VersioningConfiguration>(&data)?;
-                    if config.status.as_ref().is_some_and(|status| {
-                        !matches!(status.as_str(), BucketVersioningStatus::ENABLED | BucketVersioningStatus::SUSPENDED)
-                    }) {
+                    if config
+                        .status
+                        .as_ref()
+                        .is_some_and(|status| !matches!(status.as_str(), "Enabled" | "Suspended"))
+                    {
                        return Err(Error::other("bucket versioning configuration has an invalid status"));
                    }
                    Some(config)
@@ -1162,7 +1171,7 @@ impl BucketMetadata {
                let config = if data.is_empty() {
                    None
                } else {
-                    let config = deserialize::<ReplicationConfiguration>(&data)?;
+                    let config = deserialize::<PersistedReplicationConfiguration>(&data)?;
                    if let Some(field) = invalid_replication_config_status_field(&config) {
                        return Err(Error::other(format!("replication field {field} has an invalid status")));
                    }
@@ -1356,8 +1365,8 @@ impl BucketMetadata {
        }
        self.lifecycle_config = None;
        if !self.lifecycle_config_xml.is_empty()
-            && let Err(e) =
-                deserialize::<BucketLifecycleConfiguration>(&self.lifecycle_config_xml).map(|c| self.lifecycle_config = Some(c))
+            && let Err(e) = deserialize::<PersistedLifecycleConfiguration>(&self.lifecycle_config_xml)
+                .map(|c| self.lifecycle_config = Some(c))
        {
            tracing::warn!(
                event = "bucket_metadata_parse_failed",
@@ -1443,8 +1452,8 @@ impl BucketMetadata {
        }
        self.replication_config = None;
        if !self.replication_config_xml.is_empty()
-            && let Err(e) =
-                deserialize::<ReplicationConfiguration>(&self.replication_config_xml).map(|c| self.replication_config = Some(c))
+            && let Err(e) = deserialize::<PersistedReplicationConfiguration>(&self.replication_config_xml)
+                .map(|c| self.replication_config = Some(c))
        {
            tracing::warn!(
                event = "bucket_metadata_parse_failed",
@@ -1824,6 +1833,13 @@ mod test {
        let mut bm = BucketMetadata::unmarshal(&body[4..]).expect("unmarshal recovered blob");
        assert_eq!(bm.name, "interop");
        assert!(!bm.bucket_creation_committed, "legacy metadata must default to uncommitted");
+        assert!(!bm.bucket_creation_commit_record_present, "legacy metadata has no creation-commit record");
+        let rewritten = BucketMetadata::unmarshal(&bm.marshal_msg().expect("rewrite legacy metadata"))
+            .expect("read rewritten legacy metadata");
+        assert!(
+            !rewritten.bucket_creation_commit_record_present,
+            "rewriting legacy metadata must not fabricate a creation intent"
+        );
        bm.parse_all_configs().expect("parse recovered configs");
        assert!(bm.lifecycle_config.is_some());
    }
@@ -1841,6 +1857,7 @@ mod test {

        assert_eq!(bm.name, new.name);
        assert!(new.bucket_creation_committed);
+        assert!(new.bucket_creation_commit_record_present);
        assert!(!bm.bucket_incarnation_id.is_nil());
        assert_eq!(bm.bucket_incarnation_id, new.bucket_incarnation_id);
    }
@@ -1912,7 +1929,7 @@ mod test {

    #[test]
    fn persisted_bucket_tags_keep_separate_s3_and_swift_limits() {
-        use s3s::dto::Tag;
+        use rustfs_gateway_types::cors_tagging::PersistedTag as Tag;

        let tag = |key: String, value: &str| Tag {
            key: Some(key),
@@ -2004,17 +2021,15 @@ mod test {

    #[test]
    fn object_locking_requires_lock_metadata_not_plain_versioning() {
-        use s3s::dto::ObjectLockEnabled;
-
        let mut bm = BucketMetadata::new("test-bucket");
        bm.versioning_config = Some(VersioningConfiguration {
-            status: Some(s3s::dto::BucketVersioningStatus::from_static("Enabled")),
+            status: Some("Enabled".to_owned()),
            ..Default::default()
        });
        assert!(!bm.object_locking());

        bm.object_lock_config = Some(ObjectLockConfiguration {
-            object_lock_enabled: Some(ObjectLockEnabled::from_static(ObjectLockEnabled::ENABLED)),
+            object_lock_enabled: Some("Enabled".to_owned()),
            ..Default::default()
        });
        assert!(bm.object_locking());
@@ -31,14 +31,18 @@ use crate::storage_api_contracts::heal::HealOperations as _;
 use crate::storage_api_contracts::namespace::NamespaceLocking as _;
 use crate::store::{ECStore, await_bucket_namespace_operation};
 use futures::future::join_all;
+use rustfs_gateway_types::cors_tagging::{PersistedCorsConfiguration as CORSConfiguration, PersistedTagging as Tagging};
+use rustfs_gateway_types::persistence::{
+    PersistedAccelerateConfiguration as AccelerateConfiguration,
+    PersistedBucketEncryptionConfiguration as ServerSideEncryptionConfiguration,
+    PersistedBucketLoggingStatus as BucketLoggingStatus, PersistedLifecycleConfiguration,
+    PersistedNotificationConfiguration as NotificationConfiguration, PersistedObjectLockConfiguration as ObjectLockConfiguration,
+    PersistedPublicAccessBlockConfiguration as PublicAccessBlockConfiguration, PersistedReplicationConfiguration,
+    PersistedRequestPaymentConfiguration as RequestPaymentConfiguration,
+    PersistedVersioningConfiguration as VersioningConfiguration, PersistedWebsiteConfiguration as WebsiteConfiguration,
+};
 use rustfs_heal_contracts::heal_channel::HealOpts;
 use rustfs_policy::policy::BucketPolicy;
-use s3s::dto::ReplicationConfiguration;
-use s3s::dto::{
-    AccelerateConfiguration, BucketLifecycleConfiguration, BucketLoggingStatus, CORSConfiguration, NotificationConfiguration,
-    ObjectLockConfiguration, ObjectLockEnabled, ObjectLockRetentionMode, PublicAccessBlockConfiguration,
-    RequestPaymentConfiguration, ServerSideEncryptionConfiguration, Tagging, VersioningConfiguration, WebsiteConfiguration,
-};
 use std::collections::HashSet;
 use std::time::Duration;
 use std::{
@@ -169,7 +173,7 @@ pub(crate) fn object_lock_config_state_from_authoritative_metadata(bm: &BucketMe
    if bm.lock_enabled {
        return Ok(ObjectLockConfigState::Configured {
            config: ObjectLockConfiguration {
-                object_lock_enabled: Some(ObjectLockEnabled::from_static(ObjectLockEnabled::ENABLED)),
+                object_lock_enabled: Some("Enabled".to_owned()),
                rule: None,
            },
            updated_at: bm.object_lock_config_updated_at,
@@ -213,17 +217,10 @@ pub(crate) fn configured_object_lock_state_for_tests(
 ) -> ObjectLockConfigState {
    ObjectLockConfigState::Configured {
        config: ObjectLockConfiguration {
-            object_lock_enabled: Some(ObjectLockEnabled::from_static(ObjectLockEnabled::ENABLED)),
-            rule: Some(s3s::dto::ObjectLockRule {
-                default_retention: Some(s3s::dto::DefaultRetention {
-                    mode: Some(s3s::dto::ObjectLockRetentionMode::from_static(match mode {
-                        crate::bucket::object_lock::types::RetentionMode::Governance => {
-                            s3s::dto::ObjectLockRetentionMode::GOVERNANCE
-                        }
-                        crate::bucket::object_lock::types::RetentionMode::Compliance => {
-                            s3s::dto::ObjectLockRetentionMode::COMPLIANCE
-                        }
-                    })),
+            object_lock_enabled: Some("Enabled".to_owned()),
+            rule: Some(rustfs_gateway_types::persistence::PersistedObjectLockRule {
+                default_retention: Some(rustfs_gateway_types::persistence::PersistedDefaultRetention {
+                    mode: Some(mode.as_str().to_owned()),
                    days: Some(days),
                    years: None,
                }),
@@ -234,7 +231,7 @@ pub(crate) fn configured_object_lock_state_for_tests(
 }

 fn validate_authoritative_object_lock_config(config: &ObjectLockConfiguration) -> Result<()> {
-    if config.object_lock_enabled.as_ref().map(ObjectLockEnabled::as_str) != Some(ObjectLockEnabled::ENABLED) {
+    if config.object_lock_enabled.as_deref() != Some("Enabled") {
        return Err(Error::other("persisted bucket Object Lock enabled state is invalid"));
    }

@@ -247,7 +244,7 @@ fn validate_authoritative_object_lock_config(config: &ObjectLockConfiguration) -
    if !retention
        .mode
        .as_ref()
-        .is_some_and(|mode| matches!(mode.as_str(), ObjectLockRetentionMode::COMPLIANCE | ObjectLockRetentionMode::GOVERNANCE))
+        .is_some_and(|mode| matches!(mode.as_str(), "COMPLIANCE" | "GOVERNANCE"))
    {
        return Err(Error::other("persisted bucket Object Lock retention mode is invalid"));
    }
@@ -1641,7 +1638,7 @@ pub async fn get_public_access_block_config(bucket: &str) -> Result<(PublicAcces
    bucket_meta_sys.get_public_access_block_config(bucket).await
 }

-pub async fn get_lifecycle_config(bucket: &str) -> Result<(BucketLifecycleConfiguration, OffsetDateTime)> {
+pub async fn get_lifecycle_config(bucket: &str) -> Result<(PersistedLifecycleConfiguration, OffsetDateTime)> {
    let bucket_meta_sys_lock = get_bucket_metadata_sys()?;
    let bucket_meta_sys = bucket_meta_sys_lock.read().await;

@@ -1917,7 +1914,7 @@ pub(crate) async fn get_cached_quota_config_and_incarnation_in(
    bucket_meta_sys.cached_quota_config_and_incarnation(bucket).await
 }

-pub async fn get_replication_config(bucket: &str) -> Result<(ReplicationConfiguration, OffsetDateTime)> {
+pub async fn get_replication_config(bucket: &str) -> Result<(PersistedReplicationConfiguration, OffsetDateTime)> {
    let bucket_meta_sys_lock = get_bucket_metadata_sys()?;
    let bucket_meta_sys = bucket_meta_sys_lock.read().await;

@@ -1927,7 +1924,7 @@ pub async fn get_replication_config(bucket: &str) -> Result<(ReplicationConfigur
 pub(crate) async fn get_replication_config_in(
    ctx: &crate::runtime::instance::InstanceContext,
    bucket: &str,
-) -> Result<(ReplicationConfiguration, OffsetDateTime)> {
+) -> Result<(PersistedReplicationConfiguration, OffsetDateTime)> {
    let bucket_meta_sys_lock = bucket_metadata_sys_of(ctx)?;
    let bucket_meta_sys = bucket_meta_sys_lock.read().await;

@@ -2065,9 +2062,27 @@ pub(crate) fn current_local_meta_mutation_gen() -> u64 {

 type LocalAuthorityCache = Arc<StdMutex<HashMap<String, (u64, Arc<BucketMetadata>)>>>;

+#[derive(Debug, Clone)]
+pub(crate) struct PendingBucketCreationProof {
+    pub(crate) generation: Uuid,
+    pub(crate) intent: Vec<u8>,
+    pub(crate) versioning_requested: bool,
+    pub(crate) witnesses: Vec<crate::disk::os::NativeBucketCreationWitness>,
+}
+
+impl PendingBucketCreationProof {
+    pub(crate) fn matches_metadata(&self, metadata: &BucketMetadata) -> Result<bool> {
+        Ok(!metadata.bucket_creation_committed
+            && metadata.bucket_incarnation_sidecar
+            && metadata.bucket_incarnation_id == self.generation
+            && metadata.marshal_msg()? == self.intent)
+    }
+}
+
 #[derive(Debug, Clone)]
 pub struct BucketMetadataSys {
    metadata_map: Arc<RwLock<HashMap<String, Arc<BucketMetadata>>>>,
+    pending_bucket_creations: Arc<RwLock<HashMap<String, PendingBucketCreationProof>>>,
    local_authority_cache: LocalAuthorityCache,
    /// Serializes metadata-map commits and their derived cache updates for one
    /// bucket. Namespace locks, when present, are acquired before this lock.
@@ -2112,6 +2127,7 @@ impl BucketMetadataSys {
    pub fn new(api: Arc<ECStore>) -> Self {
        Self {
            metadata_map: Arc::new(RwLock::new(HashMap::new())),
+            pending_bucket_creations: Arc::new(RwLock::new(HashMap::new())),
            local_authority_cache: Arc::new(StdMutex::new(HashMap::new())),
            metadata_publish_locks: Arc::new(MetadataPublishLockRegistry {
                locks: StdMutex::new(HashMap::new()),
@@ -2143,6 +2159,81 @@ impl BucketMetadataSys {
            .expect("bucket metadata object store should still be live")
    }

+    pub(crate) async fn pending_bucket_creation(&self, bucket: &str) -> Option<PendingBucketCreationProof> {
+        self.pending_bucket_creations.read().await.get(bucket).cloned()
+    }
+
+    pub(crate) async fn grant_pending_bucket_creation(&self, bucket: &str, proof: PendingBucketCreationProof) {
+        self.pending_bucket_creations.write().await.insert(bucket.to_owned(), proof);
+    }
+
+    pub(crate) async fn revoke_pending_bucket_creation(&self, bucket: &str) {
+        self.pending_bucket_creations.write().await.remove(bucket);
+    }
+
+    pub(crate) async fn consume_pending_bucket_creation(
+        &self,
+        bucket: &str,
+        metadata: &BucketMetadata,
+        witnesses: &[crate::disk::os::NativeBucketCreationWitness],
+    ) -> Result<bool> {
+        let mut pending = self.pending_bucket_creations.write().await;
+        let Some(proof) = pending.get(bucket) else {
+            return Ok(false);
+        };
+        if !proof.matches_metadata(metadata)?
+            || proof.witnesses.len() != witnesses.len()
+            || !proof
+                .witnesses
+                .iter()
+                .zip(witnesses)
+                .all(|(old, current)| old.matches(current))
+            || !crate::disk::os::consume_native_creation_witnesses(witnesses)
+        {
+            return Ok(false);
+        }
+        pending.remove(bucket);
+        Ok(true)
+    }
+
+    #[cfg(test)]
+    pub(crate) async fn refresh_bucket_for_test(&self, bucket: &str) -> Result<()> {
+        let expected = self.metadata_map.read().await.get(bucket).cloned();
+        let lock = self.object_store().new_ns_lock(bucket, bucket).await?;
+        let guard = lock.get_read_lock(crate::set_disk::get_lock_acquire_timeout()).await?;
+        self.load_bucket_under_namespace(bucket, MetadataLoadMode::Refresh, expected.as_ref(), &guard)
+            .await
+    }
+
+    #[cfg(test)]
+    pub(crate) async fn publish_persisted_metadata_for_test(&self, bucket: &str) -> Result<()> {
+        let expected = self.metadata_map.read().await.get(bucket).cloned();
+        let lock = self.object_store().new_ns_lock(bucket, bucket).await?;
+        let guard = lock.get_read_lock(crate::set_disk::get_lock_acquire_timeout()).await?;
+        let (metadata, persisted) = await_bucket_namespace_operation(
+            Some(&guard),
+            bucket,
+            "test cached metadata load",
+            load_bucket_metadata_parse_with_presence(self.object_store(), bucket, true),
+        )
+        .await?;
+        self.publish_if_unchanged(bucket, expected.as_ref(), metadata, persisted, &guard)
+            .await
+    }
+
+    #[cfg(test)]
+    pub(crate) async fn migrate_bucket_creation_commit_for_test(&self, bucket: &str) -> Result<()> {
+        self.migrate_bucket_creation_commit(bucket).await.map(|_| ())
+    }
+
+    #[cfg(test)]
+    pub(crate) async fn legacy_migration_reports_missing_bucket_for_test(&self, bucket: &str) -> Result<bool> {
+        Ok(matches!(
+            self.migrate_legacy_metadata(bucket).await?,
+            BucketMetadataAuthority::MissingBucket
+        ))
+    }
+
    fn object_store_if_live(&self) -> Option<Arc<ECStore>> {
        self.api.upgrade()
    }
@@ -2342,6 +2433,7 @@ impl BucketMetadataSys {
        expected: Option<&Arc<BucketMetadata>>,
        namespace_guard: &rustfs_lock::NamespaceLockGuard,
    ) -> Result<()> {
+        self.revoke_pending_bucket_creation(bucket).await;
        if !await_bucket_namespace_operation(
            Some(namespace_guard),
            bucket,
@@ -2425,6 +2517,7 @@ impl BucketMetadataSys {
        persisted: bool,
        namespace_guard: &rustfs_lock::NamespaceLockGuard,
    ) -> Result<()> {
+        self.revoke_pending_bucket_creation(bucket).await;
        if !persisted {
            let _publish_guard = self
                .lock_metadata_publish(bucket, namespace_guard, "refreshed bucket metadata absence publish")
@@ -2486,6 +2579,7 @@ impl BucketMetadataSys {
    }

    pub async fn set(&self, bucket: String, bm: Arc<BucketMetadata>) {
+        self.revoke_pending_bucket_creation(&bucket).await;
        bump_local_meta_mutation_gen();
        if !is_meta_bucketname(&bucket) {
            let publish_lock = self.metadata_publish_lock(&bucket);
@@ -2509,6 +2603,7 @@ impl BucketMetadataSys {
        if is_meta_bucketname(bucket) {
            return false;
        }
+        self.revoke_pending_bucket_creation(bucket).await;
        let publish_lock = self.metadata_publish_lock(bucket);
        let _publish_guard = publish_lock.lock().await;
        let mut map = self.metadata_map.write().await;
@@ -2526,6 +2621,7 @@ impl BucketMetadataSys {
    }

    async fn _reset(&mut self) {
+        self.pending_bucket_creations.write().await.clear();
        bump_local_meta_mutation_gen();
        let mut map = self.metadata_map.write().await;
        map.clear();
@@ -2665,6 +2761,7 @@ impl BucketMetadataSys {
    /// server's metadata never leaks into the ambient (first) instance.
    pub(crate) async fn persist_and_set(&self, bm: BucketMetadata) -> Result<()> {
        let mut bm = bm;
+        self.revoke_pending_bucket_creation(&bm.name).await;
        bm.save_with_store(self.object_store()).await?;

        self.set(bm.name.clone(), Arc::new(bm)).await;
@@ -2673,6 +2770,7 @@ impl BucketMetadataSys {
    }

    async fn persist_new_and_set(&self, mut bm: BucketMetadata) -> Result<()> {
+        self.revoke_pending_bucket_creation(&bm.name).await;
        bm.bucket_creation_committed = true;
        bm.save_with_store_committed(self.object_store()).await?;
        save_bucket_incarnation(self.object_store(), &bm.name, bm.bucket_incarnation_id).await?;
@@ -2682,7 +2780,9 @@ impl BucketMetadataSys {
    }

    async fn persist_new_bucket_metadata_intent(&self, mut bm: BucketMetadata) -> Result<()> {
+        self.revoke_pending_bucket_creation(&bm.name).await;
        bm.bucket_creation_committed = false;
+        bm.bucket_creation_commit_record_present = true;
        bm.save_with_store(self.object_store()).await?;
        save_bucket_incarnation(self.object_store(), &bm.name, bm.bucket_incarnation_id).await?;
        bm.bucket_incarnation_sidecar = true;
@@ -2692,6 +2792,7 @@ impl BucketMetadataSys {
    }

    async fn commit_bucket_metadata(&self, mut bm: BucketMetadata) -> Result<()> {
+        self.revoke_pending_bucket_creation(&bm.name).await;
        bm.parse_all_configs()?;
        bm.bucket_incarnation_sidecar = true;
        // The caller holds the full bucket creation fence (lifecycle, metadata
@@ -2740,6 +2841,7 @@ impl BucketMetadataSys {
        bucket: &str,
        namespace_guard: &rustfs_lock::NamespaceLockGuard,
    ) -> Result<()> {
+        self.revoke_pending_bucket_creation(bucket).await;
        let expected = self.metadata_map.read().await.get(bucket).cloned();
        if !self
            .bucket_exists(bucket, namespace_guard, "peer bucket metadata existence check")
@@ -2796,6 +2898,7 @@ impl BucketMetadataSys {
            #[cfg(test)]
            self.lazy_disk_loads.fetch_add(1, std::sync::atomic::Ordering::Relaxed);

+            self.revoke_pending_bucket_creation(bucket).await;
            let lock = self.object_store().new_ns_lock(bucket, bucket).await?;
            let guard = lock.get_read_lock(crate::set_disk::get_lock_acquire_timeout()).await?;
            #[cfg(test)]
@@ -3121,6 +3224,7 @@ impl BucketMetadataSys {
    /// revalidated at write quorum, so a stale snapshot can neither revert an
    /// acknowledged configuration update nor outlive a delete/recreate.
    async fn migrate_bucket_creation_commit(&self, bucket: &str) -> Result<BucketMetadataAuthority> {
+        self.revoke_pending_bucket_creation(bucket).await;
        let transaction_guard = acquire_metadata_transaction_guard(
            self.object_store(),
            bucket,
@@ -3198,6 +3302,7 @@ impl BucketMetadataSys {
        &self,
        bucket: &str,
    ) -> Result<BucketMetadataAuthority> {
+        self.revoke_pending_bucket_creation(bucket).await;
        #[cfg(test)]
        if self.object_lock_disk_read_errors.write().await.remove(bucket) {
            return Err(Error::other(format!("injected Object Lock metadata disk read failure: {bucket}")));
@@ -3420,7 +3525,7 @@ impl BucketMetadataSys {
        }
    }

-    pub async fn get_lifecycle_config(&self, bucket: &str) -> Result<(BucketLifecycleConfiguration, OffsetDateTime)> {
+    pub async fn get_lifecycle_config(&self, bucket: &str) -> Result<(PersistedLifecycleConfiguration, OffsetDateTime)> {
        let (bm, _) = self.get_config(bucket).await?;

        match ConfigState::of(&bm.lifecycle_config_xml, &bm.lifecycle_config).require(bucket, BUCKET_LIFECYCLE_CONFIG)? {
@@ -3543,7 +3648,7 @@ impl BucketMetadataSys {
        }))
    }

-    pub async fn get_replication_config(&self, bucket: &str) -> Result<(ReplicationConfiguration, OffsetDateTime)> {
+    pub async fn get_replication_config(&self, bucket: &str) -> Result<(PersistedReplicationConfiguration, OffsetDateTime)> {
        let (bm, _) = self.get_config(bucket).await?;

        if !bm.replication_config_xml.is_empty() && bm.replication_config.is_none() {
@@ -3645,6 +3750,7 @@ mod tests {
    };
    use crate::bucket::target::{BucketTarget, BucketTargetType, Credentials};
    use crate::config::com::read_config;
+    use crate::disk::{DiskAPI as _, RUSTFS_META_TMP_BUCKET};
    use crate::storage_api_contracts::bucket::{BucketOperations as _, DeleteBucketOptions, MakeBucketOptions};
    use byteorder::{ByteOrder as _, LittleEndian};
    use serial_test::serial;
@@ -3699,6 +3805,180 @@ mod tests {
        ),
    ];

+    #[tokio::test]
+    #[serial(storage_class_env)]
+    async fn persistence_entry_failures_revoke_pending_creation_before_cache_publish() {
+        let (dirs, store) = isolated_store_over_temp_disks().await;
+        init_bucket_metadata_sys(store.clone(), Vec::new()).await;
+        let sys = bucket_metadata_sys_of(&store.ctx).expect("instance metadata system");
+        let sys = sys.read().await.clone();
+        let options = MakeBucketOptions {
+            lock_enabled: true,
+            ..Default::default()
+        };
+        for entry in ["save", "persist", "persist-new", "commit"] {
+            let bucket = format!("revoke-{entry}-{}", Uuid::new_v4());
+            store.fail_next_bucket_creation_after_intent_for_test(&bucket);
+            store
+                .make_bucket(&bucket, &options)
+                .await
+                .expect_err("stop the real producer before physical publication");
+            sys.pending_bucket_creation(&bucket)
+                .await
+                .expect("the producer must leave a real live creation permission");
+            let (intent, persisted) = get_config_from_disk_with_presence_in(&store.ctx, &bucket)
+                .await
+                .expect("read the original producer intent");
+            assert!(persisted, "{entry}: the complete original intent must be persisted");
+            let original_intent = intent.marshal_msg().expect("encode the original intent");
+
+            // Keep the creation fence order used by the real producer.
+            let lifecycle_guard = store
+                .acquire_bucket_lifecycle_write_lock(&bucket)
+                .await
+                .expect("hold the bucket lifecycle fence");
+            let transaction_guard = acquire_bucket_metadata_transaction_lock_in(&store.ctx, &bucket)
+                .await
+                .expect("hold the metadata transaction fence");
+            let namespace_lock = store
+                .new_ns_lock(&bucket, &bucket)
+                .await
+                .expect("create the exact bucket namespace lock");
+            let namespace_guard = namespace_lock
+                .get_write_lock(crate::set_disk::get_lock_acquire_timeout())
+                .await
+                .expect("hold the exact bucket namespace fence");
+
+            // A file at each native staging root causes actual IO to fail before
+            // metadata can reach the cache-publishing set call. Preserve the
+            // original staging directories so no fixture data is discarded.
+            for dir in &dirs {
+                let staging = dir.path().join(RUSTFS_META_TMP_BUCKET);
+                let saved = dir.path().join(".rustfs.sys/revocation-saved-tmp");
+                std::fs::rename(&staging, &saved).expect("preserve the native staging directory");
+                std::fs::write(&staging, b"block metadata staging with a non-directory")
+                    .expect("install an actual non-directory IO obstruction");
+            }
+            let result = match entry {
+                "save" => sys.save(intent).await,
+                "persist" => sys.persist_and_set(intent).await,
+                "persist-new" => sys.persist_new_and_set(intent).await,
+                "commit" => sys.commit_bucket_metadata(intent).await,
+                _ => unreachable!(),
+            };
+            for dir in &dirs {
+                let staging = dir.path().join(RUSTFS_META_TMP_BUCKET);
+                let saved = dir.path().join(".rustfs.sys/revocation-saved-tmp");
+                std::fs::remove_file(&staging).expect("remove only the fixture's staging obstruction");
+                std::fs::rename(&saved, &staging).expect("restore the complete original staging directory");
+            }
+            drop((namespace_guard, transaction_guard, lifecycle_guard));
+            result.expect_err("native persistence must fail before its downstream cache set");
+            assert!(
+                sys.pending_bucket_creation(&bucket).await.is_none(),
+                "{entry}: the entry revoker must retire live permission even when persistence fails"
+            );
+            let (after_failure, present) = get_config_from_disk_with_presence_in(&store.ctx, &bucket)
+                .await
+                .expect("read the intent after the failed persistence entry");
+            assert!(present, "{entry}: a failed persistence entry must retain the original metadata");
+            assert_eq!(
+                after_failure
+                    .marshal_msg()
+                    .expect("encode the original intent after persistence failure"),
+                original_intent,
+                "{entry}: native IO failure must not change the original generation or complete intent"
+            );
+            let error = store
+                .make_bucket(&bucket, &options)
+                .await
+                .expect_err("a failed ordinary persistence entry cannot preserve creation permission");
+            assert!(error.to_string().contains("administrator recovery"), "{entry}: {error}");
+            let (after_retry, _) = get_config_from_disk_with_presence_in(&store.ctx, &bucket)
+                .await
+                .expect("read the original intent after the rejected retry");
+            assert_eq!(
+                after_retry.marshal_msg().expect("encode the intent after the rejected retry"),
+                original_intent,
+                "{entry}: the rejected retry must preserve the complete original intent"
+            );
+            for disk in store.pools[0].disk_set[0].disks.read().await.iter() {
+                assert!(
+                    matches!(
+                        disk.as_ref().expect("native disk").stat_volume(&bucket).await,
+                        Err(crate::disk::error::DiskError::VolumeNotFound)
+                    ),
+                    "{entry}: failed persistence and the rejected retry must not publish a physical volume"
+                );
+            }
+        }
+    }
+
+    #[tokio::test]
+    #[serial(storage_class_env)]
+    async fn metadata_reset_revokes_real_pending_creation_without_changing_persisted_intent() {
+        let (_dirs, store) = isolated_store_over_temp_disks().await;
+        init_bucket_metadata_sys(store.clone(), Vec::new()).await;
+        let bucket = format!("revoke-reset-{}", Uuid::new_v4());
+        let options = MakeBucketOptions {
+            lock_enabled: true,
+            ..Default::default()
+        };
+        store.fail_next_bucket_creation_after_intent_for_test(&bucket);
+        store
+            .make_bucket(&bucket, &options)
+            .await
+            .expect_err("stop the real producer before physical publication");
+        let sys = bucket_metadata_sys_of(&store.ctx).expect("instance metadata system");
+        let mut sys = sys.read().await.clone();
+        sys.pending_bucket_creation(&bucket)
+            .await
+            .expect("the real producer must leave live creation permission before reset");
+        let (intent, persisted) = get_config_from_disk_with_presence_in(&store.ctx, &bucket)
+            .await
+            .expect("read the original intent before cache reset");
+        assert!(persisted, "cache reset starts from a complete real persisted intent");
+        let original_intent = intent.marshal_msg().expect("encode the original reset intent");
+        sys._reset().await;
+        assert!(
+            sys.pending_bucket_creation(&bucket).await.is_none(),
+            "reset must revoke the producer-issued permission shared with the instance metadata system"
+        );
+        let (after_reset, present) = get_config_from_disk_with_presence_in(&store.ctx, &bucket)
+            .await
+            .expect("read the original intent after cache reset");
+        assert!(present, "cache reset must retain the persisted intent");
+        assert_eq!(
+            after_reset.marshal_msg().expect("encode the intent after cache reset"),
+            original_intent,
+            "cache reset must leave the original generation and complete persisted intent unchanged"
+        );
+        let error = store
+            .make_bucket(&bucket, &options)
+            .await
+            .expect_err("reset cannot regrant the producer's old creation permission");
+        assert!(error.to_string().contains("administrator recovery"), "{error}");
+        let (after_retry, _) = get_config_from_disk_with_presence_in(&store.ctx, &bucket)
+            .await
+            .expect("read the intent after the reset-induced retry rejection");
+        assert_eq!(
+            after_retry
+                .marshal_msg()
+                .expect("encode the intent after the rejected reset retry"),
+            original_intent,
+            "the reset-induced rejection must preserve the original complete intent"
+        );
+        for disk in store.pools[0].disk_set[0].disks.read().await.iter() {
+            assert!(
+                matches!(
+                    disk.as_ref().expect("native disk").stat_volume(&bucket).await,
+                    Err(crate::disk::error::DiskError::VolumeNotFound)
+                ),
+                "cache reset and the rejected retry must not publish a physical volume"
+            );
+        }
+    }
+
    #[tokio::test]
    async fn g_d3_003_new_writer_replication_loads_without_fail_closed_state() {
        let (dirs, store) = isolated_store_over_temp_disks().await;
@@ -3887,7 +4167,7 @@ mod tests {
        let mut versioned = BucketMetadata::new("authoritative-versioned");
        versioned.versioning_config_xml = b"<VersioningConfiguration><Status>Enabled</Status></VersioningConfiguration>".to_vec();
        versioned.versioning_config = Some(VersioningConfiguration {
-            status: Some(s3s::dto::BucketVersioningStatus::from_static(s3s::dto::BucketVersioningStatus::ENABLED)),
+            status: Some("Enabled".to_owned()),
            ..Default::default()
        });
        sys.set("authoritative-versioned".to_string(), Arc::new(versioned)).await;
@@ -4010,25 +4290,24 @@ mod tests {
        else {
            panic!("legacy lock-enabled metadata must remain configured");
        };
-        assert_eq!(
-            config.object_lock_enabled.as_ref().map(|value| value.as_str()),
-            Some(ObjectLockEnabled::ENABLED)
-        );
+        assert_eq!(config.object_lock_enabled.as_deref(), Some("Enabled"));
        assert!(config.rule.is_none());
    }

    #[test]
    fn authoritative_object_lock_state_rejects_invalid_retention_periods() {
-        use s3s::dto::{DefaultRetention, ObjectLockRule};
+        use rustfs_gateway_types::persistence::{
+            PersistedDefaultRetention as DefaultRetention, PersistedObjectLockRule as ObjectLockRule,
+        };

        for (days, years) in [(Some(0), None), (Some(-1), None), (Some(1), Some(1)), (None, None)] {
            let mut metadata = BucketMetadata::new("invalid-object-lock");
            metadata.object_lock_config = Some(ObjectLockConfiguration {
-                object_lock_enabled: Some(ObjectLockEnabled::from_static(ObjectLockEnabled::ENABLED)),
+                object_lock_enabled: Some("Enabled".to_owned()),
                rule: Some(ObjectLockRule {
                    default_retention: Some(DefaultRetention {
                        days,
-                        mode: Some(ObjectLockRetentionMode::from_static(ObjectLockRetentionMode::COMPLIANCE)),
+                        mode: Some("COMPLIANCE".to_owned()),
                        years,
                    }),
                }),
@@ -4043,9 +4322,11 @@ mod tests {

    #[test]
    fn authoritative_object_lock_state_rejects_invalid_enabled_mode_and_rule() {
-        use s3s::dto::{DefaultRetention, ObjectLockRule};
+        use rustfs_gateway_types::persistence::{
+            PersistedDefaultRetention as DefaultRetention, PersistedObjectLockRule as ObjectLockRule,
+        };

-        for enabled in [None, Some(ObjectLockEnabled::from("Disabled".to_string()))] {
+        for enabled in [None, Some("Disabled".to_string())] {
            let mut metadata = BucketMetadata::new("invalid-object-lock-enabled");
            metadata.lock_enabled = true;
            metadata.object_lock_config = Some(ObjectLockConfiguration {
@@ -4057,11 +4338,11 @@ mod tests {

        let mut invalid_mode = BucketMetadata::new("invalid-object-lock-mode");
        invalid_mode.object_lock_config = Some(ObjectLockConfiguration {
-            object_lock_enabled: Some(ObjectLockEnabled::from_static(ObjectLockEnabled::ENABLED)),
+            object_lock_enabled: Some("Enabled".to_owned()),
            rule: Some(ObjectLockRule {
                default_retention: Some(DefaultRetention {
                    days: Some(1),
-                    mode: Some(ObjectLockRetentionMode::from("invalid".to_string())),
+                    mode: Some("invalid".to_string()),
                    years: None,
                }),
            }),
@@ -4070,7 +4351,7 @@ mod tests {

        let mut missing_retention = BucketMetadata::new("missing-object-lock-retention");
        missing_retention.object_lock_config = Some(ObjectLockConfiguration {
-            object_lock_enabled: Some(ObjectLockEnabled::from_static(ObjectLockEnabled::ENABLED)),
+            object_lock_enabled: Some("Enabled".to_owned()),
            rule: Some(ObjectLockRule { default_retention: None }),
        });
        assert!(object_lock_config_state_from_authoritative_metadata(&missing_retention).is_err());
@@ -4080,7 +4361,7 @@ mod tests {
    fn object_lock_ever_enabled_marker_survives_config_removal() {
        let mut metadata = BucketMetadata::new("lock-config-removed");
        let config = ObjectLockConfiguration {
-            object_lock_enabled: Some(ObjectLockEnabled::from_static(ObjectLockEnabled::ENABLED)),
+            object_lock_enabled: Some("Enabled".to_owned()),
            rule: None,
        };
        metadata
@@ -4098,10 +4379,7 @@ mod tests {
        else {
            panic!("an ever-enabled bucket must remain Object Lock configured");
        };
-        assert_eq!(
-            config.object_lock_enabled.as_ref().map(ObjectLockEnabled::as_str),
-            Some(ObjectLockEnabled::ENABLED)
-        );
+        assert_eq!(config.object_lock_enabled.as_deref(), Some("Enabled"));
    }

    #[tokio::test]
@@ -4117,7 +4395,7 @@ mod tests {

        let node = Arc::new(RwLock::new(BucketMetadataSys::new(ecstore.clone())));
        let config = ObjectLockConfiguration {
-            object_lock_enabled: Some(ObjectLockEnabled::from_static(ObjectLockEnabled::ENABLED)),
+            object_lock_enabled: Some("Enabled".to_owned()),
            rule: None,
        };
        update_with_sys(
@@ -5231,7 +5509,7 @@ mod tests {
    #[tokio::test]
    async fn update_config_with_persists_tagging_rewrite_across_disk_reload() {
        use crate::bucket::metadata::BUCKET_TAGGING_CONFIG;
-        use s3s::dto::Tag;
+        use rustfs_gateway_types::cors_tagging::PersistedTag as Tag;

        let (dirs, ecstore) = isolated_store_over_temp_disks().await;

@@ -5395,7 +5673,7 @@ mod tests {
    #[tokio::test]
    async fn concurrent_update_config_with_calls_do_not_lose_writes() {
        use crate::bucket::metadata::BUCKET_TAGGING_CONFIG;
-        use s3s::dto::Tag;
+        use rustfs_gateway_types::cors_tagging::PersistedTag as Tag;

        let (dirs, ecstore) = isolated_store_over_temp_disks().await;
        let sys = Arc::new(RwLock::new(BucketMetadataSys::new(ecstore)));
@@ -5495,7 +5773,7 @@ mod tests {
        let mut persisted = BucketMetadata::new("reload-bucket");
        persisted.policy_config_json = b"persisted-marker".to_vec();
        let lock_config = ObjectLockConfiguration {
-            object_lock_enabled: Some(ObjectLockEnabled::from_static(ObjectLockEnabled::ENABLED)),
+            object_lock_enabled: Some("Enabled".to_owned()),
            rule: None,
        };
        persisted
@@ -5829,7 +6107,7 @@ mod tests {
    #[serial]
    async fn bucket_delete_waits_for_config_mutation_fence() {
        use crate::bucket::metadata::BUCKET_TAGGING_CONFIG;
-        use s3s::dto::Tag;
+        use rustfs_gateway_types::cors_tagging::PersistedTag as Tag;

        let (_dirs, store) = isolated_store_over_temp_disks().await;
        init_bucket_metadata_sys(store.clone(), Vec::new()).await;
@@ -17,7 +17,7 @@
 use crate::bucket::metadata::BUCKET_METADATA_FILE;
 use crate::bucket::replication::ReplicationMigrationBridge;
 use crate::disk::{BUCKET_META_PREFIX, MIGRATING_META_BUCKET, RUSTFS_META_BUCKET};
-use crate::error::{Error, Result, is_err_strict_not_found, is_err_strict_volume_not_found};
+use crate::error::{Error, Result, is_err_strict_not_found, is_err_strict_volume_not_found, to_object_err};
 use crate::object_api::{GetObjectReader, ObjectInfo, ObjectOptions, PutObjReader};
 use crate::storage_api_contracts::{
    bucket::{BucketInfo, BucketOperations, BucketOptions},
@@ -334,6 +334,14 @@ async fn migration_target_exists<S: EcstoreObjectOperations>(store: &S, path: &s
    }
 }

+/// Names the target record in a failed migration write, as the existence
+/// probe's error already does. The commit path returns bare disk errors, and on
+/// Windows a blocked record is first seen here: reading through a regular-file
+/// ancestor reports the record missing instead of refusing access to it.
+fn migration_write_error(err: Error, path: &str) -> Error {
+    to_object_err(err, vec![RUSTFS_META_BUCKET, path])
+}
+
 async fn migrate_one_if_missing<S>(
    store: Arc<S>,
    opts: &ObjectOptions,
@@ -369,7 +377,10 @@ where
        .unwrap_or(data);

    let mut put_data = PutObjReader::from_vec(data);
-    store.put_object(RUSTFS_META_BUCKET, path, &mut put_data, opts).await?;
+    store
+        .put_object(RUSTFS_META_BUCKET, path, &mut put_data, opts)
+        .await
+        .map_err(|err| migration_write_error(err, path))?;
    info!("Migrated {label}");
    Ok(())
 }
@@ -481,7 +492,10 @@ where
                Err(_) => return Err(MigrationMetadataError::Incompatible(path.to_owned()).into()),
            };
            let mut put_data = PutObjReader::from_vec(data);
-            store.put_object(RUSTFS_META_BUCKET, path, &mut put_data, &opts).await?;
+            store
+                .put_object(RUSTFS_META_BUCKET, path, &mut put_data, &opts)
+                .await
+                .map_err(|err| migration_write_error(err, path))?;
            info!("Migrated IAM config: {path}");
            total_migrated += 1;
        }
@@ -531,6 +545,27 @@ mod tests {
        ));
    }

+    #[test]
+    fn migration_write_errors_name_the_target_record_without_fragmenting_groups() {
+        use super::{Error, migration_write_error};
+
+        let first = migration_write_error(Error::FileAccessDenied, "buckets/first/.metadata.bin");
+        let second = migration_write_error(Error::FileAccessDenied, "config/iam/format.json");
+        assert!(matches!(first, Error::PrefixAccessDenied(..)), "unexpected error: {first:?}");
+        assert!(
+            first.to_string().contains(".rustfs.sys/buckets/first/.metadata.bin"),
+            "the startup error must name the blocked record: {first}"
+        );
+        assert_eq!(first, second, "record paths must not fragment error grouping");
+
+        let unrelated = migration_write_error(Error::other("disk offline"), "buckets/first/.metadata.bin");
+        assert_eq!(
+            unrelated,
+            Error::other("disk offline"),
+            "errors without a record form pass through unchanged"
+        );
+    }
+
    #[test]
    fn migration_errors_group_by_cause_and_retain_typed_record_context() {
        use super::{Error, MigrationMetadataError};
@@ -17,15 +17,8 @@ pub mod objectlock_sys;
 pub mod types;

 /// Whether a bucket Object Lock configuration has locking enabled. The
-/// serving-layer `ObjectLockConfiguration` DTO implements this in
-/// the bucket-metadata module, which owns the persisted configuration type
-/// during the s3s ratchet migration (rustfs/backlog#1842).
+/// persisted `ObjectLockConfiguration` implements this in the bucket-metadata
+/// module, which owns the persisted configuration type.
 pub trait ObjectLockApi {
    fn enabled(&self) -> bool;
 }
-
-/// Whether a legal-hold status value is one of the two valid wire values.
-/// Implemented for the serving-layer DTO in the bucket-metadata module.
-pub trait ObjectLockStatusExt {
-    fn valid(&self) -> bool;
-}
@@ -12,6 +12,7 @@
 // See the License for the specific language governing permissions and
 // limitations under the License.

+pub use rustfs_replication::config_cache_serde;
 pub use rustfs_replication::{
    ObjectOpts, OperatorRuleContract, REMOTE_TARGET_CAPABILITY_CONTRACT_VERSION, REMOTE_TARGET_READ_ONLY_HISTORICAL_FIELDS,
    REMOTE_TARGET_UNSUPPORTED_FIELDS, REMOTE_TARGET_WRITABLE_FIELDS, REPLICATION_CAPABILITY_CONTRACT_VERSION,
@@ -12,7 +12,7 @@
 // See the License for the specific language governing permissions and
 // limitations under the License.

-use s3s::dto::ReplicationConfiguration;
+use rustfs_gateway_types::persistence::PersistedReplicationConfiguration;

 use super::replication_config_boundary::ReplicationConfigurationExt as _;
 use super::replication_filemeta_boundary::{
@@ -42,7 +42,7 @@ impl ReplicationLifecycleBridge {
        dead_code,
        reason = "declared boundary surface for the ECStore replication split plan; no caller in this port (backlog#1823)"
    )]
-    pub(crate) fn new_config(config: ReplicationConfiguration) -> ReplicationLifecycleConfig {
+    pub(crate) fn new_config(config: PersistedReplicationConfiguration) -> ReplicationLifecycleConfig {
        ReplicationConfig::new(Some(config), None)
    }

@@ -118,16 +118,17 @@ impl ReplicationLifecycleBridge {

 #[cfg(test)]
 mod tests {
-    use s3s::dto::{Destination, ReplicationRule, ReplicationRuleStatus};
+    use rustfs_gateway_types::dto::Status;
+    use rustfs_gateway_types::persistence::{PersistedReplicationDestination, PersistedReplicationRule};

    use super::super::replication_filemeta_boundary::{ReplicateTargetDecision, VersionPurgeStatusType};
    use super::*;

-    fn replication_rule() -> ReplicationRule {
-        ReplicationRule {
+    fn replication_rule() -> PersistedReplicationRule {
+        PersistedReplicationRule {
            delete_marker_replication: None,
            delete_replication: None,
-            destination: Destination {
+            destination: PersistedReplicationDestination {
                bucket: "arn:aws:s3:::target-bucket".to_string(),
                ..Default::default()
            },
@@ -137,13 +138,13 @@ mod tests {
            prefix: Some(String::new()),
            priority: Some(1),
            source_selection_criteria: None,
-            status: ReplicationRuleStatus::from_static(ReplicationRuleStatus::ENABLED),
+            status: Status::ENABLED.to_string(),
        }
    }

    #[test]
    fn has_pending_version_purge_preserves_replication_active_rule_behavior() {
-        let config = ReplicationLifecycleBridge::new_config(ReplicationConfiguration {
+        let config = ReplicationLifecycleBridge::new_config(PersistedReplicationConfiguration {
            role: String::new(),
            rules: vec![replication_rule()],
        });
@@ -17,8 +17,8 @@ use std::sync::Arc;
 use crate::bucket::{metadata::BucketMetadata, metadata_sys};
 use crate::disk::{BUCKET_META_PREFIX, RUSTFS_META_BUCKET};
 use crate::runtime::instance::InstanceContext;
+use rustfs_gateway_types::persistence::PersistedReplicationConfiguration;
 use rustfs_utils::path::path_join_buf;
-use s3s::dto::ReplicationConfiguration;
 use time::OffsetDateTime;

 use super::replication_error_boundary::{Error, Result};
@@ -37,11 +37,11 @@ impl ReplicationMetadataStore {
    pub(crate) const FORCE_DELETE_REPLICATION_FILE: &'static str = "config/replication/force-delete.bin";
    pub(crate) const FORCE_DELETE_REPLICATION_TRANSACTION_LOCK: &'static str = "config/replication/force-delete.bin.transaction";

-    pub(crate) async fn replication_config(bucket: &str) -> Result<(ReplicationConfiguration, OffsetDateTime)> {
+    pub(crate) async fn replication_config(bucket: &str) -> Result<(PersistedReplicationConfiguration, OffsetDateTime)> {
        metadata_sys::get_replication_config(bucket).await
    }

-    pub(crate) async fn optional_replication_config(bucket: &str) -> Result<Option<ReplicationConfiguration>> {
+    pub(crate) async fn optional_replication_config(bucket: &str) -> Result<Option<PersistedReplicationConfiguration>> {
        let config = match Self::replication_config(bucket).await {
            Ok((config, _)) => Some(config),
            Err(err) => {
@@ -16,7 +16,9 @@ use std::{collections::HashMap, fmt, sync::Arc};

 use super::replication_filemeta_boundary::metadata_keys;
 use crate::bucket::metadata::BucketMetadata;
-use s3s::dto::{BucketVersioningStatus, ReplicationConfiguration, ReplicationRuleStatus, VersioningConfiguration};
+use rustfs_gateway_types::dto::Status;
+use rustfs_gateway_types::persistence::PersistedReplicationConfiguration;
+use rustfs_gateway_types::persistence::PersistedVersioningConfiguration as VersioningConfiguration;
 use serde::{Deserialize, Serialize};
 use time::OffsetDateTime;
 use tracing::error;
@@ -39,7 +41,7 @@ use super::replication_target_boundary::{BucketTargets, ReplicationTargetStore};
 use super::replication_versioning_boundary::ReplicationVersioningStore;
 use super::runtime_boundary as runtime_sources;

-pub(crate) async fn get_replication_config(bucket: &str) -> Result<Option<ReplicationConfiguration>> {
+pub(crate) async fn get_replication_config(bucket: &str) -> Result<Option<PersistedReplicationConfiguration>> {
    let config = ReplicationMetadataStore::optional_replication_config(bucket).await?;
    validate_delete_replication_config(&VersioningConfiguration::default(), config.as_ref())?;
    Ok(config)
@@ -64,7 +66,7 @@ impl DeleteReplicationConfigSnapshot {
    #[cfg(test)]
    pub(crate) fn from_configs_for_test(
        versioning: VersioningConfiguration,
-        replication: Option<ReplicationConfiguration>,
+        replication: Option<PersistedReplicationConfiguration>,
    ) -> Self {
        let metadata = replication.map(|config| {
            let mut metadata = BucketMetadata::new("test-bucket");
@@ -78,7 +80,7 @@ impl DeleteReplicationConfigSnapshot {
        &self.versioning
    }

-    pub fn replication_config(&self) -> Option<&ReplicationConfiguration> {
+    pub fn replication_config(&self) -> Option<&PersistedReplicationConfiguration> {
        self.metadata
            .as_ref()
            .and_then(|metadata| metadata.replication_config.as_ref())
@@ -101,7 +103,7 @@ impl DeleteReplicationConfigSnapshot {
    pub(crate) fn active_delete_marker_rules_require_tags(&self, object: &str) -> bool {
        self.replication_config().is_some_and(|config| {
            config.rules.iter().any(|rule| {
-                if rule.status == ReplicationRuleStatus::from_static(ReplicationRuleStatus::DISABLED) {
+                if rule.status == Status::DISABLED.as_str() {
                    return false;
                }
                if !object.starts_with(rule.prefix()) {
@@ -122,12 +124,12 @@ impl DeleteReplicationConfigSnapshot {

 fn validate_delete_replication_config(
    versioning: &VersioningConfiguration,
-    config: Option<&ReplicationConfiguration>,
+    config: Option<&PersistedReplicationConfiguration>,
 ) -> Result<()> {
    if versioning
        .status
        .as_ref()
-        .is_some_and(|status| !matches!(status.as_str(), BucketVersioningStatus::ENABLED | BucketVersioningStatus::SUSPENDED))
+        .is_some_and(|status| !matches!(status.as_str(), "Enabled" | "Suspended"))
    {
        return Err(super::replication_error_boundary::Error::other(
            "bucket versioning configuration has an invalid status",
@@ -144,7 +146,7 @@ fn validate_delete_replication_config(
        let role = config.role.trim();
        let mut role_destination = None;
        for rule in &config.rules {
-            if rule.status.as_str() == ReplicationRuleStatus::ENABLED {
+            if rule.status.as_str() == Status::ENABLED.as_str() {
                let destination = rule.destination.bucket.trim();
                if role.is_empty() && destination.is_empty() {
                    return Err(super::replication_error_boundary::Error::other(
@@ -169,7 +171,7 @@ fn validate_delete_replication_config(
    Ok(())
 }

-fn replication_config_from_metadata(metadata: &BucketMetadata) -> Result<Option<&ReplicationConfiguration>> {
+fn replication_config_from_metadata(metadata: &BucketMetadata) -> Result<Option<&PersistedReplicationConfiguration>> {
    if !metadata.replication_config_xml.is_empty() && metadata.replication_config.is_none() {
        return Err(super::replication_error_boundary::Error::other(
            "persisted bucket replication configuration is invalid",
@@ -250,12 +252,15 @@ pub(crate) async fn load_delete_replication_config_in(

 #[derive(Debug, Clone, Default, Serialize, Deserialize)]
 pub struct ReplicationConfig {
-    pub config: Option<ReplicationConfiguration>,
+    /// Carried in the scanner's persisted data usage cache, so its serde form
+    /// is the one the s3s DTO wrote there (see `config_cache_serde` in the replication crate).
+    #[serde(with = "super::replication_config_boundary::config_cache_serde")]
+    pub config: Option<PersistedReplicationConfiguration>,
    pub remotes: Option<BucketTargets>,
 }

 impl ReplicationConfig {
-    pub fn new(config: Option<ReplicationConfiguration>, remotes: Option<BucketTargets>) -> Self {
+    pub fn new(config: Option<PersistedReplicationConfiguration>, remotes: Option<BucketTargets>) -> Self {
        Self { config, remotes }
    }

@@ -468,7 +473,7 @@ fn check_replicate_delete_with_config(
    oi: &ObjectInfo,
    del_opts: &ObjectOptions,
    source_error: bool,
-    config: Option<&ReplicationConfiguration>,
+    config: Option<&PersistedReplicationConfiguration>,
    existing_delete_marker: bool,
    trust_persisted_replica_status: bool,
 ) -> ReplicateDecision {
@@ -587,7 +592,7 @@ pub(crate) async fn must_replicate(bucket: &str, object: &str, mopts: MustReplic
 }

 fn metadata_target_should_replicate(
-    cfg: &ReplicationConfiguration,
+    cfg: &PersistedReplicationConfiguration,
    opts: &ObjectOpts,
    mopts: &MustReplicateOptions,
    arn: &str,
@@ -597,20 +602,21 @@ fn metadata_target_should_replicate(

 #[cfg(test)]
 mod tests {
-    use s3s::dto::{
-        DeleteMarkerReplication, DeleteMarkerReplicationStatus, DeleteReplication, DeleteReplicationStatus, Destination,
-        ReplicaModifications, ReplicationRule, ReplicationRuleFilter, ReplicationRuleStatus, SourceSelectionCriteria, Tag,
+    use rustfs_gateway_types::dto::Status;
+    use rustfs_gateway_types::persistence::{
+        PersistedOptionalReplicationStatus, PersistedReplicationDestination, PersistedReplicationFilter,
+        PersistedReplicationRule, PersistedReplicationStatus, PersistedReplicationTag, PersistedSourceSelectionCriteria,
    };

    use super::super::replication_filemeta_boundary::VersionPurgeStatusType;
    use super::super::replication_target_boundary::BucketTarget;
    use super::*;

-    fn replication_rule() -> ReplicationRule {
-        ReplicationRule {
+    fn replication_rule() -> PersistedReplicationRule {
+        PersistedReplicationRule {
            delete_marker_replication: None,
            delete_replication: None,
-            destination: Destination {
+            destination: PersistedReplicationDestination {
                bucket: "arn:aws:s3:::target-bucket".to_string(),
                ..Default::default()
            },
@@ -620,7 +626,7 @@ mod tests {
            prefix: Some(String::new()),
            priority: Some(1),
            source_selection_criteria: None,
-            status: ReplicationRuleStatus::from_static(ReplicationRuleStatus::ENABLED),
+            status: Status::ENABLED.to_string(),
        }
    }

@@ -629,11 +635,11 @@ mod tests {
        let arn = "arn:rustfs:replication:us-east-1:target:bucket";
        let mut rule = replication_rule();
        rule.destination.bucket = arn.to_string();
-        rule.filter = Some(ReplicationRuleFilter {
+        rule.filter = Some(PersistedReplicationFilter {
            prefix: Some("admitted/".to_string()),
            ..Default::default()
        });
-        let cfg = ReplicationConfiguration {
+        let cfg = PersistedReplicationConfiguration {
            role: String::new(),
            rules: vec![rule],
        };
@@ -671,7 +677,7 @@ mod tests {
        assert!(!empty.replicate(&ObjectOpts::default()));

        let config = ReplicationConfig::new(
-            Some(ReplicationConfiguration {
+            Some(PersistedReplicationConfiguration {
                role: String::new(),
                rules: vec![replication_rule()],
            }),
@@ -725,7 +731,7 @@ mod tests {
    fn delete_snapshot_rejects_enabled_rules_without_a_destination() {
        let mut rule = replication_rule();
        rule.destination.bucket.clear();
-        let config = ReplicationConfiguration {
+        let config = PersistedReplicationConfiguration {
            role: String::new(),
            rules: vec![rule],
        };
@@ -741,7 +747,7 @@ mod tests {
        let first = replication_rule();
        let mut second = replication_rule();
        second.destination.bucket = "arn:aws:s3:::other-target".to_string();
-        let config = ReplicationConfiguration {
+        let config = PersistedReplicationConfiguration {
            role: "arn:aws:s3:::role-target".to_string(),
            rules: vec![first, second],
        };
@@ -752,47 +758,47 @@ mod tests {
    #[test]
    fn delete_snapshot_rejects_unknown_status_values() {
        let invalid_versioning = VersioningConfiguration {
-            status: Some("Enabld".to_string().into()),
+            status: Some("Enabld".to_string()),
            ..Default::default()
        };
        assert!(validate_delete_replication_config(&invalid_versioning, None).is_err());

        let mut invalid_rule = replication_rule();
-        invalid_rule.status = "Enabld".to_string().into();
-        let config = ReplicationConfiguration {
+        invalid_rule.status = "Enabld".to_string();
+        let config = PersistedReplicationConfiguration {
            role: String::new(),
            rules: vec![invalid_rule],
        };
        assert!(validate_delete_replication_config(&VersioningConfiguration::default(), Some(&config)).is_err());

        let mut invalid_delete = replication_rule();
-        invalid_delete.delete_replication = Some(DeleteReplication {
-            status: "Enabld".to_string().into(),
+        invalid_delete.delete_replication = Some(PersistedReplicationStatus {
+            status: "Enabld".to_string(),
        });
-        let config = ReplicationConfiguration {
+        let config = PersistedReplicationConfiguration {
            role: String::new(),
            rules: vec![invalid_delete],
        };
        assert!(validate_delete_replication_config(&VersioningConfiguration::default(), Some(&config)).is_err());

        let mut invalid_delete_marker = replication_rule();
-        invalid_delete_marker.delete_marker_replication = Some(DeleteMarkerReplication {
-            status: Some("Enabld".to_string().into()),
+        invalid_delete_marker.delete_marker_replication = Some(PersistedOptionalReplicationStatus {
+            status: Some("Enabld".to_string()),
        });
-        let config = ReplicationConfiguration {
+        let config = PersistedReplicationConfiguration {
            role: String::new(),
            rules: vec![invalid_delete_marker],
        };
        assert!(validate_delete_replication_config(&VersioningConfiguration::default(), Some(&config)).is_err());

        let mut invalid_replica_modifications = replication_rule();
-        invalid_replica_modifications.source_selection_criteria = Some(SourceSelectionCriteria {
-            replica_modifications: Some(ReplicaModifications {
-                status: "Enabld".to_string().into(),
+        invalid_replica_modifications.source_selection_criteria = Some(PersistedSourceSelectionCriteria {
+            replica_modifications: Some(PersistedReplicationStatus {
+                status: "Enabld".to_string(),
            }),
            sse_kms_encrypted_objects: None,
        });
-        let config = ReplicationConfiguration {
+        let config = PersistedReplicationConfiguration {
            role: String::new(),
            rules: vec![invalid_replica_modifications],
        };
@@ -804,11 +810,11 @@ mod tests {
        let mut metadata = BucketMetadata::new("bucket");
        metadata.versioning_config_xml = b"configured".to_vec();
        metadata.versioning_config = Some(VersioningConfiguration {
-            status: Some(BucketVersioningStatus::from_static(BucketVersioningStatus::ENABLED)),
+            status: Some("Enabled".to_owned()),
            ..Default::default()
        });
        metadata.replication_config_xml = b"configured".to_vec();
-        metadata.replication_config = Some(ReplicationConfiguration {
+        metadata.replication_config = Some(PersistedReplicationConfiguration {
            role: String::new(),
            rules: vec![replication_rule()],
        });
@@ -848,7 +854,7 @@ mod tests {
        );

        let mut inconsistent = BucketMetadata::new("bucket");
-        inconsistent.replication_config = Some(ReplicationConfiguration {
+        inconsistent.replication_config = Some(PersistedReplicationConfiguration {
            role: String::new(),
            rules: vec![replication_rule()],
        });
@@ -863,11 +869,11 @@ mod tests {
        let arn = "arn:rustfs:replication:us-east-1:target:bucket";
        let mut rule = replication_rule();
        rule.destination.bucket = arn.to_string();
-        rule.delete_marker_replication = Some(DeleteMarkerReplication {
-            status: Some(DeleteMarkerReplicationStatus::from_static(DeleteMarkerReplicationStatus::ENABLED)),
+        rule.delete_marker_replication = Some(PersistedOptionalReplicationStatus {
+            status: Some(Status::ENABLED.to_string()),
        });
        let mut metadata = BucketMetadata::new("bucket");
-        metadata.replication_config = Some(ReplicationConfiguration {
+        metadata.replication_config = Some(PersistedReplicationConfiguration {
            role: String::new(),
            rules: vec![rule],
        });
@@ -898,27 +904,27 @@ mod tests {
    fn delete_marker_source_read_is_required_only_for_tag_filtered_rules() {
        let mut prefix_rule = replication_rule();
        prefix_rule.prefix = Some("logs/".to_string());
-        prefix_rule.delete_marker_replication = Some(DeleteMarkerReplication {
-            status: Some(DeleteMarkerReplicationStatus::from_static(DeleteMarkerReplicationStatus::ENABLED)),
+        prefix_rule.delete_marker_replication = Some(PersistedOptionalReplicationStatus {
+            status: Some(Status::ENABLED.to_string()),
        });

        let prefix_snapshot = DeleteReplicationConfigSnapshot::from_configs_for_test(
            VersioningConfiguration {
-                status: Some(BucketVersioningStatus::from_static(BucketVersioningStatus::ENABLED)),
+                status: Some("Enabled".to_owned()),
                ..Default::default()
            },
-            Some(ReplicationConfiguration {
+            Some(PersistedReplicationConfiguration {
                role: String::new(),
                rules: vec![prefix_rule],
            }),
        );

        let mut tag_rule = replication_rule();
-        tag_rule.delete_marker_replication = Some(DeleteMarkerReplication {
-            status: Some(DeleteMarkerReplicationStatus::from_static(DeleteMarkerReplicationStatus::ENABLED)),
+        tag_rule.delete_marker_replication = Some(PersistedOptionalReplicationStatus {
+            status: Some(Status::ENABLED.to_string()),
        });
-        tag_rule.filter = Some(ReplicationRuleFilter {
-            tag: Some(Tag {
+        tag_rule.filter = Some(PersistedReplicationFilter {
+            tag: Some(PersistedReplicationTag {
                key: Some("class".to_string()),
                value: Some("audit".to_string()),
            }),
@@ -926,10 +932,10 @@ mod tests {
        });
        let tag_snapshot = DeleteReplicationConfigSnapshot::from_configs_for_test(
            VersioningConfiguration {
-                status: Some(BucketVersioningStatus::from_static(BucketVersioningStatus::ENABLED)),
+                status: Some("Enabled".to_owned()),
                ..Default::default()
            },
-            Some(ReplicationConfiguration {
+            Some(PersistedReplicationConfiguration {
                role: String::new(),
                rules: vec![tag_rule],
            }),
@@ -944,14 +950,14 @@ mod tests {
        let arn = "arn:rustfs:replication:us-east-1:target:bucket";
        let mut rule = replication_rule();
        rule.destination.bucket = arn.to_string();
-        rule.delete_replication = Some(DeleteReplication {
-            status: DeleteReplicationStatus::from_static(DeleteReplicationStatus::ENABLED),
+        rule.delete_replication = Some(PersistedReplicationStatus {
+            status: Status::ENABLED.to_string(),
        });
-        rule.delete_marker_replication = Some(DeleteMarkerReplication {
-            status: Some(DeleteMarkerReplicationStatus::from_static(DeleteMarkerReplicationStatus::DISABLED)),
+        rule.delete_marker_replication = Some(PersistedOptionalReplicationStatus {
+            status: Some(Status::DISABLED.to_string()),
        });
        let config = ReplicationConfig::new(
-            Some(ReplicationConfiguration {
+            Some(PersistedReplicationConfiguration {
                role: String::new(),
                rules: vec![rule],
            }),
@@ -985,22 +991,22 @@ mod tests {
    #[test]
    fn live_delete_does_not_trust_persisted_replica_status() {
        let mut rule = replication_rule();
-        rule.delete_replication = Some(DeleteReplication {
-            status: DeleteReplicationStatus::from_static(DeleteReplicationStatus::ENABLED),
+        rule.delete_replication = Some(PersistedReplicationStatus {
+            status: Status::ENABLED.to_string(),
        });
-        rule.source_selection_criteria = Some(SourceSelectionCriteria {
-            replica_modifications: Some(ReplicaModifications {
-                status: s3s::dto::ReplicaModificationsStatus::from_static(s3s::dto::ReplicaModificationsStatus::DISABLED),
+        rule.source_selection_criteria = Some(PersistedSourceSelectionCriteria {
+            replica_modifications: Some(PersistedReplicationStatus {
+                status: Status::DISABLED.to_string(),
            }),
            sse_kms_encrypted_objects: None,
        });
-        let replication = ReplicationConfiguration {
+        let replication = PersistedReplicationConfiguration {
            role: String::new(),
            rules: vec![rule],
        };
        let snapshot = DeleteReplicationConfigSnapshot::from_configs_for_test(
            VersioningConfiguration {
-                status: Some(BucketVersioningStatus::from_static(BucketVersioningStatus::ENABLED)),
+                status: Some("Enabled".to_owned()),
                ..Default::default()
            },
            Some(replication.clone()),
@@ -1036,14 +1042,14 @@ mod tests {
        let arn = "arn:rustfs:replication:us-east-1:target:bucket";
        let mut rule = replication_rule();
        rule.destination.bucket = arn.to_string();
-        rule.delete_replication = Some(DeleteReplication {
-            status: DeleteReplicationStatus::from_static(DeleteReplicationStatus::ENABLED),
+        rule.delete_replication = Some(PersistedReplicationStatus {
+            status: Status::ENABLED.to_string(),
        });
-        rule.delete_marker_replication = Some(DeleteMarkerReplication {
-            status: Some(DeleteMarkerReplicationStatus::from_static(DeleteMarkerReplicationStatus::DISABLED)),
+        rule.delete_marker_replication = Some(PersistedOptionalReplicationStatus {
+            status: Some(Status::DISABLED.to_string()),
        });
        let config = ReplicationConfig::new(
-            Some(ReplicationConfiguration {
+            Some(PersistedReplicationConfiguration {
                role: String::new(),
                rules: vec![rule],
            }),
@@ -5074,9 +5074,10 @@ mod tests {
    #[tokio::test]
    async fn heal_queue_marks_missing_versioning_state_as_missed() {
        use super::super::replication_target_boundary::BucketTargets;
-        use s3s::dto::{
-            DeleteReplication, DeleteReplicationStatus, Destination, ReplicationConfiguration, ReplicationRule,
-            ReplicationRuleStatus,
+        use rustfs_gateway_types::dto::Status;
+        use rustfs_gateway_types::persistence::{
+            PersistedReplicationConfiguration, PersistedReplicationDestination, PersistedReplicationRule,
+            PersistedReplicationStatus,
        };

        let arn = "arn:rustfs:replication:us-east-1:target:bucket";
@@ -5091,14 +5092,14 @@ mod tests {
                ..Default::default()
            },
            ReplicationConfig::new(
-                Some(ReplicationConfiguration {
+                Some(PersistedReplicationConfiguration {
                    role: String::new(),
-                    rules: vec![ReplicationRule {
+                    rules: vec![PersistedReplicationRule {
                        delete_marker_replication: None,
-                        delete_replication: Some(DeleteReplication {
-                            status: DeleteReplicationStatus::from_static(DeleteReplicationStatus::ENABLED),
+                        delete_replication: Some(PersistedReplicationStatus {
+                            status: Status::ENABLED.to_string(),
                        }),
-                        destination: Destination {
+                        destination: PersistedReplicationDestination {
                            bucket: arn.to_string(),
                            ..Default::default()
                        },
@@ -5108,7 +5109,7 @@ mod tests {
                        prefix: Some(String::new()),
                        priority: Some(1),
                        source_selection_criteria: None,
-                        status: ReplicationRuleStatus::from_static(ReplicationRuleStatus::ENABLED),
+                        status: Status::ENABLED.to_string(),
                    }],
                }),
                Some(BucketTargets::default()),
@@ -85,6 +85,8 @@ use http_body_util::StreamBody;
 use metrics::counter;
 #[cfg(test)]
 use rmp_serde;
+#[cfg(test)]
+use rustfs_gateway_types::persistence::PersistedReplicationConfiguration;
 use rustfs_s3_types::EventName;
 use rustfs_utils::http::{
    AMZ_OBJECT_LOCK_LEGAL_HOLD, AMZ_OBJECT_LOCK_MODE, AMZ_OBJECT_LOCK_RETAIN_UNTIL_DATE, AMZ_TAGGING_DIRECTIVE,
@@ -92,8 +94,6 @@ use rustfs_utils::http::{
    insert_str, replication_target_versions,
 };
 use rustfs_utils::{DEFAULT_SIP_HASH_KEY, get_env_usize, sip_hash};
-#[cfg(test)]
-use s3s::dto::ReplicationConfiguration;
 use std::collections::{HashMap, HashSet};
 use std::fmt::Display;
 use std::pin::Pin;
@@ -6579,9 +6579,10 @@ mod tests {

    use super::super::replication_target_boundary::{BucketTarget, BucketTargets};
    use super::*;
-    use s3s::dto::{
-        BucketVersioningStatus, DeleteReplication, DeleteReplicationStatus, Destination, ExcludedPrefix, ReplicationRule,
-        ReplicationRuleStatus, VersioningConfiguration,
+    use rustfs_gateway_types::dto::Status;
+    use rustfs_gateway_types::persistence::PersistedVersioningConfiguration as VersioningConfiguration;
+    use rustfs_gateway_types::persistence::{
+        PersistedReplicationDestination, PersistedReplicationRule, PersistedReplicationStatus,
    };
    use std::collections::HashMap;
    use time::OffsetDateTime;
@@ -8693,7 +8694,7 @@ mod tests {
            ..Default::default()
        };
        let rcfg = ReplicationConfig::new(
-            Some(ReplicationConfiguration {
+            Some(PersistedReplicationConfiguration {
                role: role.to_string(),
                rules: vec![],
            }),
@@ -8719,7 +8720,7 @@ mod tests {
        ReplicationVersioningStore::install_prefix_state_test_config(
            &bucket,
            VersioningConfiguration {
-                status: Some(BucketVersioningStatus::from_static(BucketVersioningStatus::ENABLED)),
+                status: Some("Enabled".to_owned()),
                ..Default::default()
            },
        );
@@ -8752,22 +8753,20 @@ mod tests {
        ReplicationVersioningStore::install_prefix_state_test_config(
            &bucket,
            VersioningConfiguration {
-                status: Some(BucketVersioningStatus::from_static(BucketVersioningStatus::ENABLED)),
-                excluded_prefixes: Some(vec![ExcludedPrefix {
-                    prefix: Some("archive/".to_string()),
-                }]),
+                status: Some("Enabled".to_owned()),
+                excluded_prefixes: Some(vec![Some("archive/".to_string())]),
                ..Default::default()
            },
        );
        let rcfg = ReplicationConfig::new(
-            Some(ReplicationConfiguration {
+            Some(PersistedReplicationConfiguration {
                role: String::new(),
-                rules: vec![ReplicationRule {
+                rules: vec![PersistedReplicationRule {
                    delete_marker_replication: None,
-                    delete_replication: Some(DeleteReplication {
-                        status: DeleteReplicationStatus::from_static(DeleteReplicationStatus::ENABLED),
+                    delete_replication: Some(PersistedReplicationStatus {
+                        status: Status::ENABLED.to_string(),
                    }),
-                    destination: Destination {
+                    destination: PersistedReplicationDestination {
                        bucket: arn.to_string(),
                        ..Default::default()
                    },
@@ -8777,7 +8776,7 @@ mod tests {
                    prefix: Some(String::new()),
                    priority: Some(1),
                    source_selection_criteria: None,
-                    status: ReplicationRuleStatus::from_static(ReplicationRuleStatus::ENABLED),
+                    status: Status::ENABLED.to_string(),
                }],
            }),
            Some(BucketTargets {
@@ -8813,14 +8812,14 @@ mod tests {
        let admitted_arn = "arn:rustfs:replication:us-east-1:target:admitted";
        let current_role = "arn:rustfs:replication:us-east-1:target:current";
        let rcfg = ReplicationConfig::new(
-            Some(ReplicationConfiguration {
+            Some(PersistedReplicationConfiguration {
                role: current_role.to_string(),
-                rules: vec![ReplicationRule {
+                rules: vec![PersistedReplicationRule {
                    delete_marker_replication: None,
-                    delete_replication: Some(DeleteReplication {
-                        status: DeleteReplicationStatus::from_static(DeleteReplicationStatus::DISABLED),
+                    delete_replication: Some(PersistedReplicationStatus {
+                        status: Status::DISABLED.to_string(),
                    }),
-                    destination: Destination {
+                    destination: PersistedReplicationDestination {
                        bucket: current_role.to_string(),
                        ..Default::default()
                    },
@@ -8830,7 +8829,7 @@ mod tests {
                    prefix: Some(String::new()),
                    priority: Some(1),
                    source_selection_criteria: None,
-                    status: ReplicationRuleStatus::from_static(ReplicationRuleStatus::ENABLED),
+                    status: Status::ENABLED.to_string(),
                }],
            }),
            Some(BucketTargets {
@@ -8878,12 +8877,12 @@ mod tests {
    #[tokio::test]
    async fn heal_rejects_semantically_invalid_replication_config() {
        let rcfg = ReplicationConfig::new(
-            Some(ReplicationConfiguration {
+            Some(PersistedReplicationConfiguration {
                role: String::new(),
-                rules: vec![ReplicationRule {
+                rules: vec![PersistedReplicationRule {
                    delete_marker_replication: None,
                    delete_replication: None,
-                    destination: Destination {
+                    destination: PersistedReplicationDestination {
                        bucket: "arn:rustfs:replication:us-east-1:target:bucket".to_string(),
                        ..Default::default()
                    },
@@ -8893,7 +8892,7 @@ mod tests {
                    prefix: Some(String::new()),
                    priority: Some(1),
                    source_selection_criteria: None,
-                    status: ReplicationRuleStatus::from_static("Enabld"),
+                    status: "Enabld".to_owned(),
                }],
            }),
            Some(BucketTargets::default()),
@@ -12,7 +12,7 @@
 // See the License for the specific language governing permissions and
 // limitations under the License.

-use s3s::dto::ReplicationConfiguration;
+use rustfs_gateway_types::persistence::PersistedReplicationConfiguration;

 use super::replication_config_boundary::{ObjectOpts, ReplicationConfigurationExt};
 use super::replication_filemeta_boundary::ReplicationType;
@@ -20,7 +20,7 @@ use super::replication_filemeta_boundary::ReplicationType;
 pub(crate) struct ReplicationTargetConfigBridge;

 impl ReplicationTargetConfigBridge {
-    pub(crate) fn target_is_used_by_rules(config: &ReplicationConfiguration, arn: &str) -> bool {
+    pub(crate) fn target_is_used_by_rules(config: &PersistedReplicationConfiguration, arn: &str) -> bool {
        config
            .filter_target_arns(&ObjectOpts {
                op_type: ReplicationType::All,
@@ -37,7 +37,7 @@ mod tests {

    #[test]
    fn target_config_bridge_matches_role_target() {
-        let config = ReplicationConfiguration {
+        let config = PersistedReplicationConfiguration {
            role: " arn:target ".to_string(),
            rules: Vec::new(),
        };
@@ -15,7 +15,7 @@
 use super::replication_error_boundary::Result;
 use crate::bucket::{versioning::VersioningApi as _, versioning_sys::BucketVersioningSys};
 #[cfg(test)]
-use s3s::dto::VersioningConfiguration;
+use rustfs_gateway_types::persistence::PersistedVersioningConfiguration as VersioningConfiguration;
 #[cfg(test)]
 use std::{collections::HashMap, sync::LazyLock, sync::Mutex};

@@ -12,7 +12,7 @@
 // See the License for the specific language governing permissions and
 // limitations under the License.

-use s3s::dto::Tag;
+use rustfs_gateway_types::cors_tagging::PersistedTag as Tag;
 use std::cmp::Ordering;
 use std::collections::HashMap;
 use url::form_urlencoded;
@@ -15,9 +15,23 @@
 use crate::disk::{MIGRATING_META_BUCKET, RUSTFS_META_BUCKET};
 use crate::error::{Error, Result, StorageError};
 use regex::Regex;
+use rustfs_gateway_types::cors_tagging::{
+    CorsTaggingCodecError, PersistedCorsConfiguration, PersistedTagging, parse_cors, parse_tagging, serialize_cors,
+    serialize_tagging,
+};
+use rustfs_gateway_types::persistence::{
+    PersistedAccelerateConfiguration, PersistedBucketEncryptionConfiguration, PersistedBucketLoggingStatus,
+    PersistedLifecycleConfiguration, PersistedNotificationConfiguration, PersistedObjectLockConfiguration,
+    PersistedPublicAccessBlockConfiguration, PersistedReplicationConfiguration, PersistedRequestPaymentConfiguration,
+    PersistedVersioningConfiguration, PersistedWebsiteConfiguration, PersistenceCodecError, parse_accelerate,
+    parse_bucket_encryption, parse_bucket_logging, parse_lifecycle_legacy_dates, parse_notification, parse_object_lock,
+    parse_public_access_block, parse_replication, parse_request_payment, parse_versioning, parse_website, serialize_accelerate,
+    serialize_bucket_encryption, serialize_bucket_logging, serialize_lifecycle_legacy_dates, serialize_notification,
+    serialize_object_lock, serialize_public_access_block, serialize_replication, serialize_request_payment, serialize_versioning,
+    serialize_website,
+};
 use rustfs_utils::WindowsFileNameError;
 use rustfs_utils::path::SLASH_SEPARATOR;
-use s3s::xml;
 use tracing::instrument;

 pub fn is_meta_bucketname(name: &str) -> bool {
@@ -74,37 +88,173 @@ pub fn check_valid_bucket_name_strict(bucket_name: &str) -> Result<()> {
    check_bucket_name_common(bucket_name, true)
 }

-// RUSTFS_COMPAT_TODO(s3gate-metadata-xml): the s3s codec reads persisted XML during migration. Remove after every supported writer uses the gateway codec and every retained metadata object and backup archive is verified or rewritten.
-pub fn deserialize<T>(input: &[u8]) -> xml::DeResult<T>
-where
-    T: for<'xml> xml::Deserialize<'xml>,
-{
-    let mut d = xml::Deserializer::new(input);
-    let ans = T::deserialize(&mut d)?;
-    d.expect_eof()?;
-    Ok(ans)
+/// A bucket configuration persisted as XML in the bucket metadata, together
+/// with the codec that reads and writes its stored bytes.
+///
+/// Every family reads and writes its stored bytes through the gateway
+/// persistence codecs (`rustfs_gateway_types::persistence` and
+/// `rustfs_gateway_types::cors_tagging`, rustfs/backlog#2745). The codecs keep
+/// the persisted bytes the `bucket_config_goldens` pin, so a document the old
+/// s3s codec wrote reads back and re-encodes unchanged.
+pub trait BucketConfigXml: Sized {
+    /// Decodes the stored bytes of one configuration document.
+    fn decode_config_xml(input: &[u8]) -> std::result::Result<Self, BucketConfigXmlError>;
+
+    /// Encodes the value as RustFS stores it.
+    fn encode_config_xml(&self) -> std::result::Result<Vec<u8>, BucketConfigXmlError>;
 }

-#[allow(
-    dead_code,
-    reason = "xml serialize helper with no caller in this port; the live sibling is deserialize (backlog#1823)"
-)]
-pub fn serialize_content<T: xml::SerializeContent>(val: &T) -> xml::SerResult<String> {
-    let mut buf = Vec::with_capacity(256);
-    {
-        let mut ser = xml::Serializer::new(&mut buf);
-        val.serialize_content(&mut ser)?;
-    }
-    Ok(String::from_utf8(buf).unwrap())
+/// Why a persisted bucket configuration could not be decoded or encoded.
+#[derive(Debug, thiserror::Error)]
+pub enum BucketConfigXmlError {
+    /// A gateway persistence codec refused the bytes or the value.
+    #[error(transparent)]
+    Persisted(#[from] PersistenceCodecError),
+    /// The gateway CORS or Tagging codec refused the bytes or the value.
+    #[error(transparent)]
+    CorsTagging(#[from] CorsTaggingCodecError),
 }

-pub fn serialize<T: xml::Serialize>(val: &T) -> xml::SerResult<Vec<u8>> {
-    let mut buf = Vec::with_capacity(256);
-    {
-        let mut ser = xml::Serializer::new(&mut buf);
-        val.serialize(&mut ser)?;
+/// Decodes one persisted bucket configuration document.
+pub fn deserialize<T: BucketConfigXml>(input: &[u8]) -> std::result::Result<T, BucketConfigXmlError> {
+    T::decode_config_xml(input)
+}
+
+/// Encodes one bucket configuration as RustFS persists it.
+pub fn serialize<T: BucketConfigXml>(val: &T) -> std::result::Result<Vec<u8>, BucketConfigXmlError> {
+    val.encode_config_xml()
+}
+
+impl BucketConfigXml for PersistedReplicationConfiguration {
+    fn decode_config_xml(input: &[u8]) -> std::result::Result<Self, BucketConfigXmlError> {
+        Ok(parse_replication(input)?)
+    }
+
+    fn encode_config_xml(&self) -> std::result::Result<Vec<u8>, BucketConfigXmlError> {
+        Ok(serialize_replication(self)?)
+    }
+}
+
+impl BucketConfigXml for PersistedLifecycleConfiguration {
+    fn decode_config_xml(input: &[u8]) -> std::result::Result<Self, BucketConfigXmlError> {
+        Ok(parse_lifecycle_legacy_dates(input)?)
+    }
+
+    fn encode_config_xml(&self) -> std::result::Result<Vec<u8>, BucketConfigXmlError> {
+        Ok(serialize_lifecycle_legacy_dates(self)?)
+    }
+}
+
+// The gateway writers of these families are infallible: a value that exists is
+// always encodable, so only the readers can refuse bytes.
+impl BucketConfigXml for PersistedAccelerateConfiguration {
+    fn decode_config_xml(input: &[u8]) -> std::result::Result<Self, BucketConfigXmlError> {
+        Ok(parse_accelerate(input)?)
+    }
+
+    fn encode_config_xml(&self) -> std::result::Result<Vec<u8>, BucketConfigXmlError> {
+        Ok(serialize_accelerate(self))
+    }
+}
+
+impl BucketConfigXml for PersistedBucketLoggingStatus {
+    fn decode_config_xml(input: &[u8]) -> std::result::Result<Self, BucketConfigXmlError> {
+        Ok(parse_bucket_logging(input)?)
+    }
+
+    fn encode_config_xml(&self) -> std::result::Result<Vec<u8>, BucketConfigXmlError> {
+        Ok(serialize_bucket_logging(self))
+    }
+}
+
+impl BucketConfigXml for PersistedCorsConfiguration {
+    fn decode_config_xml(input: &[u8]) -> std::result::Result<Self, BucketConfigXmlError> {
+        Ok(parse_cors(input)?)
+    }
+
+    fn encode_config_xml(&self) -> std::result::Result<Vec<u8>, BucketConfigXmlError> {
+        Ok(serialize_cors(self))
+    }
+}
+
+impl BucketConfigXml for PersistedNotificationConfiguration {
+    fn decode_config_xml(input: &[u8]) -> std::result::Result<Self, BucketConfigXmlError> {
+        Ok(parse_notification(input)?)
+    }
+
+    fn encode_config_xml(&self) -> std::result::Result<Vec<u8>, BucketConfigXmlError> {
+        Ok(serialize_notification(self))
+    }
+}
+
+impl BucketConfigXml for PersistedObjectLockConfiguration {
+    fn decode_config_xml(input: &[u8]) -> std::result::Result<Self, BucketConfigXmlError> {
+        Ok(parse_object_lock(input)?)
+    }
+
+    fn encode_config_xml(&self) -> std::result::Result<Vec<u8>, BucketConfigXmlError> {
+        Ok(serialize_object_lock(self))
+    }
+}
+
+impl BucketConfigXml for PersistedPublicAccessBlockConfiguration {
+    fn decode_config_xml(input: &[u8]) -> std::result::Result<Self, BucketConfigXmlError> {
+        Ok(parse_public_access_block(input)?)
+    }
+
+    fn encode_config_xml(&self) -> std::result::Result<Vec<u8>, BucketConfigXmlError> {
+        Ok(serialize_public_access_block(self))
+    }
+}
+
+impl BucketConfigXml for PersistedRequestPaymentConfiguration {
+    fn decode_config_xml(input: &[u8]) -> std::result::Result<Self, BucketConfigXmlError> {
+        Ok(parse_request_payment(input)?)
+    }
+
+    fn encode_config_xml(&self) -> std::result::Result<Vec<u8>, BucketConfigXmlError> {
+        Ok(serialize_request_payment(self))
+    }
+}
+
+impl BucketConfigXml for PersistedBucketEncryptionConfiguration {
+    fn decode_config_xml(input: &[u8]) -> std::result::Result<Self, BucketConfigXmlError> {
+        Ok(parse_bucket_encryption(input)?)
+    }
+
+    fn encode_config_xml(&self) -> std::result::Result<Vec<u8>, BucketConfigXmlError> {
+        Ok(serialize_bucket_encryption(self))
+    }
+}
+
+impl BucketConfigXml for PersistedTagging {
+    fn decode_config_xml(input: &[u8]) -> std::result::Result<Self, BucketConfigXmlError> {
+        Ok(parse_tagging(input)?)
+    }
+
+    fn encode_config_xml(&self) -> std::result::Result<Vec<u8>, BucketConfigXmlError> {
+        Ok(serialize_tagging(self))
+    }
+}
+
+impl BucketConfigXml for PersistedVersioningConfiguration {
+    fn decode_config_xml(input: &[u8]) -> std::result::Result<Self, BucketConfigXmlError> {
+        Ok(parse_versioning(input)?)
+    }
+
+    fn encode_config_xml(&self) -> std::result::Result<Vec<u8>, BucketConfigXmlError> {
+        Ok(serialize_versioning(self))
+    }
+}
+
+impl BucketConfigXml for PersistedWebsiteConfiguration {
+    fn decode_config_xml(input: &[u8]) -> std::result::Result<Self, BucketConfigXmlError> {
+        Ok(parse_website(input)?)
+    }
+
+    fn encode_config_xml(&self) -> std::result::Result<Vec<u8>, BucketConfigXmlError> {
+        Ok(serialize_website(self))
    }
-    Ok(buf)
 }

 pub fn has_bad_path_component(path: &str) -> bool {
@@ -446,6 +596,40 @@ mod tests {
    use super::*;
    use proptest::prelude::*;

+    /// Every bucket-configuration family reads and writes through its own
+    /// gateway codec (rustfs/backlog#2745): the family's golden document
+    /// decodes and re-encodes to the same bytes, and a document of another
+    /// family is refused instead of decoding into an empty configuration.
+    #[test]
+    fn every_bucket_config_family_uses_its_own_gateway_codec() {
+        fn assert_family<T: BucketConfigXml>(family: &str, own: &[u8], foreign: &[u8]) {
+            let value = deserialize::<T>(own).unwrap_or_else(|err| panic!("{family}: own document must decode: {err}"));
+            let encoded = serialize(&value).unwrap_or_else(|err| panic!("{family}: value must encode: {err}"));
+            assert_eq!(encoded, own, "{family}: re-encoding must keep the persisted bytes");
+            assert!(deserialize::<T>(foreign).is_err(), "{family}: a foreign root element must be refused");
+        }
+
+        macro_rules! golden {
+            ($family:literal) => {
+                include_bytes!(concat!("../../tests/fixtures/bucket-config-goldens/", $family, "/minio.out.xml"))
+            };
+        }
+        let tagging = golden!("tagging");
+        let versioning = golden!("versioning");
+
+        assert_family::<PersistedAccelerateConfiguration>("accelerate", golden!("accelerate"), tagging);
+        assert_family::<PersistedBucketLoggingStatus>("logging", golden!("logging"), tagging);
+        assert_family::<PersistedCorsConfiguration>("cors", golden!("cors"), tagging);
+        assert_family::<PersistedNotificationConfiguration>("notification", golden!("notification"), tagging);
+        assert_family::<PersistedObjectLockConfiguration>("object_lock", golden!("object_lock"), tagging);
+        assert_family::<PersistedPublicAccessBlockConfiguration>("public_access_block", golden!("public_access_block"), tagging);
+        assert_family::<PersistedRequestPaymentConfiguration>("request_payment", golden!("request_payment"), tagging);
+        assert_family::<PersistedBucketEncryptionConfiguration>("encryption", golden!("encryption"), tagging);
+        assert_family::<PersistedTagging>("tagging", tagging, versioning);
+        assert_family::<PersistedVersioningConfiguration>("versioning", versioning, tagging);
+        assert_family::<PersistedWebsiteConfiguration>("website", golden!("website"), tagging);
+    }
+
    /// rustfs#5785: keys whose path segments exceed the on-disk NAME_MAX
    /// budget must be rejected up front as ObjectNameInvalid (4xx), not leak
    /// ENAMETOOLONG as InternalError 500 from the disk layer.
@@ -12,8 +12,8 @@
 // See the License for the specific language governing permissions and
 // limitations under the License.

+use rustfs_gateway_types::persistence::PersistedVersioningConfiguration as VersioningConfiguration;
 use rustfs_utils::string::match_simple;
-use s3s::dto::{BucketVersioningStatus, VersioningConfiguration};

 pub trait VersioningApi {
    fn enabled(&self) -> bool;
@@ -28,10 +28,10 @@ pub trait VersioningApi {

 impl VersioningApi for VersioningConfiguration {
    fn enabled(&self) -> bool {
-        self.status == Some(BucketVersioningStatus::from_static(BucketVersioningStatus::ENABLED))
+        self.status.as_deref() == Some("Enabled")
    }
    fn prefix_enabled(&self, prefix: &str) -> bool {
-        if self.status != Some(BucketVersioningStatus::from_static(BucketVersioningStatus::ENABLED)) {
+        if self.status.as_deref() != Some("Enabled") {
            return false;
        }

@@ -44,12 +44,10 @@ impl VersioningApi for VersioningConfiguration {
        }

        if let Some(ref excluded_prefixes) = self.excluded_prefixes {
-            for p in excluded_prefixes.iter() {
-                if let Some(ref sprefix) = p.prefix {
-                    let pattern = format!("{sprefix}*");
-                    if match_simple(&pattern, prefix) {
-                        return false;
-                    }
+            for sprefix in excluded_prefixes.iter().flatten() {
+                let pattern = format!("{sprefix}*");
+                if match_simple(&pattern, prefix) {
+                    return false;
                }
            }
        }
@@ -58,11 +56,11 @@ impl VersioningApi for VersioningConfiguration {
    }

    fn prefix_suspended(&self, prefix: &str) -> bool {
-        if self.status == Some(BucketVersioningStatus::from_static(BucketVersioningStatus::SUSPENDED)) {
+        if self.status.as_deref() == Some("Suspended") {
            return true;
        }

-        if self.status == Some(BucketVersioningStatus::from_static(BucketVersioningStatus::ENABLED)) {
+        if self.status.as_deref() == Some("Enabled") {
            if prefix.is_empty() {
                return false;
            }
@@ -75,12 +73,10 @@ impl VersioningApi for VersioningConfiguration {
            }

            if let Some(ref excluded_prefixes) = self.excluded_prefixes {
-                for p in excluded_prefixes.iter() {
-                    if let Some(ref sprefix) = p.prefix {
-                        let pattern = format!("{sprefix}*");
-                        if match_simple(&pattern, prefix) {
-                            return true;
-                        }
+                for sprefix in excluded_prefixes.iter().flatten() {
+                    let pattern = format!("{sprefix}*");
+                    if match_simple(&pattern, prefix) {
+                        return true;
                    }
                }
            }
@@ -92,14 +88,12 @@ impl VersioningApi for VersioningConfiguration {
        self.prefix_enabled(prefix) || self.prefix_suspended(prefix)
    }
    fn suspended(&self) -> bool {
-        self.status == Some(BucketVersioningStatus::from_static(BucketVersioningStatus::SUSPENDED))
+        self.status.as_deref() == Some("Suspended")
    }
 }

 #[cfg(test)]
 mod tests {
-    use s3s::dto::{BucketVersioningStatus, ExcludedPrefix};
-
    use super::*;

    struct LegacyVersioning;
@@ -134,10 +128,8 @@ mod tests {
    #[test]
    fn delete_state_treats_excluded_prefixes_as_unversioned() {
        let config = VersioningConfiguration {
-            status: Some(BucketVersioningStatus::from_static(BucketVersioningStatus::ENABLED)),
-            excluded_prefixes: Some(vec![ExcludedPrefix {
-                prefix: Some("archive/".to_string()),
-            }]),
+            status: Some("Enabled".to_owned()),
+            excluded_prefixes: Some(vec![Some("archive/".to_string())]),
            ..Default::default()
        };

@@ -146,7 +138,7 @@ mod tests {
        assert_eq!(config.delete_state("live/object"), (true, false));

        let suspended = VersioningConfiguration {
-            status: Some(BucketVersioningStatus::from_static(BucketVersioningStatus::SUSPENDED)),
+            status: Some("Suspended".to_owned()),
            ..Default::default()
        };
        assert_eq!(suspended.delete_state("archive/object"), (false, true));
@@ -17,7 +17,7 @@ use super::metadata::unreadable_config_refusal;
 use super::{metadata_sys::get_bucket_metadata_sys, versioning::VersioningApi};
 use crate::disk::RUSTFS_META_BUCKET;
 use crate::error::Result;
-use s3s::dto::VersioningConfiguration;
+use rustfs_gateway_types::persistence::PersistedVersioningConfiguration as VersioningConfiguration;
 use tracing::{error, warn};

 pub struct BucketVersioningSys {}
@@ -163,7 +163,7 @@ mod tests {

    fn enabled() -> VersioningConfiguration {
        VersioningConfiguration {
-            status: Some(s3s::dto::BucketVersioningStatus::from_static(s3s::dto::BucketVersioningStatus::ENABLED)),
+            status: Some("Enabled".to_owned()),
            ..Default::default()
        }
    }
@@ -41,8 +41,12 @@ use std::{
    future::Future,
    io::ErrorKind,
    pin::Pin,
-    sync::{LazyLock, Mutex},
+    sync::{
+        Arc, LazyLock, Mutex,
+        atomic::{AtomicU8, Ordering},
+    },
    task::{Context, Poll},
+    time::Duration,
 };
 use tonic::{service::interceptor::InterceptedService, transport::Channel};
 use tower::Service;
@@ -70,7 +74,7 @@ pub(crate) async fn scanner_control_time_out_client(
    let channel = get_channel_for_class(addr, ChannelClass::Control)
        .await
        .map_err(|err| crate::error::Error::other(err.to_string()))?;
-    let channel = ReplayScopeChannel::new(channel, interceptor.replay_scope_audience());
+    let channel = ReplayScopeChannel::new(BatchLockDeadlineChannel::new(channel), interceptor.replay_scope_audience());
    let limit = rustfs_protos::scoped_dirty_usage::SCOPED_DIRTY_USAGE_MAX_REQUEST_BYTES as usize;
    Ok(ScannerControlServiceClient::with_interceptor(channel, interceptor)
        .max_decoding_message_size(limit)
@@ -84,7 +88,7 @@ pub async fn heal_control_time_out_client(
    let interceptor = interceptor.with_rpc_audience(addr)?;
    let channel = get_channel_for_class(addr, ChannelClass::Control).await?;
    let max_message_size = rustfs_protos::HEAL_CONTROL_RPC_MAX_MESSAGE_SIZE;
-    let channel = ReplayScopeChannel::new(channel, interceptor.replay_scope_audience());
+    let channel = ReplayScopeChannel::new(BatchLockDeadlineChannel::new(channel), interceptor.replay_scope_audience());
    Ok(HealControlServiceClient::with_interceptor(channel, interceptor)
        .max_decoding_message_size(max_message_size)
        .max_encoding_message_size(max_message_size))
@@ -97,7 +101,7 @@ pub async fn tier_mutation_control_time_out_client(
    let interceptor = interceptor.with_rpc_audience(addr)?;
    let channel = get_channel_for_class(addr, ChannelClass::Control).await?;
    let max_message_size = rustfs_protos::TIER_MUTATION_RPC_MAX_MESSAGE_SIZE;
-    let channel = ReplayScopeChannel::new(channel, interceptor.replay_scope_audience());
+    let channel = ReplayScopeChannel::new(BatchLockDeadlineChannel::new(channel), interceptor.replay_scope_audience());
    Ok(TierMutationControlServiceClient::with_interceptor(channel, interceptor)
        .max_decoding_message_size(max_message_size)
        .max_encoding_message_size(max_message_size))
@@ -118,7 +122,7 @@ pub async fn node_service_time_out_client_for_class(
    let channel = get_channel_for_class(addr, class).await?;

    let max_message_size = rustfs_protos::internode_rpc_max_message_size();
-    let channel = ReplayScopeChannel::new(channel, interceptor.replay_scope_audience());
+    let channel = ReplayScopeChannel::new(BatchLockDeadlineChannel::new(channel), interceptor.replay_scope_audience());
    Ok(NodeServiceClient::with_interceptor(channel, interceptor)
        .max_decoding_message_size(max_message_size)
        .max_encoding_message_size(max_message_size))
@@ -133,7 +137,7 @@ pub(crate) async fn node_service_delete_recovery_client(
    let channel = rustfs_protos::create_channel_with_timeout(addr, timeout)
        .await
        .map_err(|error| DiskError::RemoteClientUnavailable(error.to_string()))?;
-    let channel = ReplayScopeChannel::new(channel, interceptor.replay_scope_audience());
+    let channel = ReplayScopeChannel::new(BatchLockDeadlineChannel::new(channel), interceptor.replay_scope_audience());
    let max_message_size = rustfs_protos::internode_rpc_max_message_size();
    Ok(NodeServiceClient::with_interceptor(channel, interceptor)
        .max_decoding_message_size(max_message_size)
@@ -249,7 +253,95 @@ pub struct ReplayScopeChannel<S> {
 }

 /// The channel type used by internode clients after v2 authentication and replay-scope handling.
-pub type AuthenticatedChannel = ReplayScopeChannel<Channel>;
+pub type AuthenticatedChannel = ReplayScopeChannel<BatchLockDeadlineChannel<Channel>>;
+
+/// Gate lock-batch payloads after the transport's internal readiness queues.
+/// Other RPC bodies pass through without an additional wrapper.
+#[derive(Clone, Debug)]
+pub struct BatchLockDeadlineChannel<S> {
+    inner: S,
+}
+
+impl<S> BatchLockDeadlineChannel<S> {
+    fn new(inner: S) -> Self {
+        Self { inner }
+    }
+}
+
+impl<S> Service<HttpRequest<tonic::body::Body>> for BatchLockDeadlineChannel<S>
+where
+    S: Service<HttpRequest<tonic::body::Body>>,
+{
+    type Response = S::Response;
+    type Error = S::Error;
+    type Future = S::Future;
+
+    fn poll_ready(&mut self, cx: &mut Context<'_>) -> Poll<Result<(), Self::Error>> {
+        self.inner.poll_ready(cx)
+    }
+
+    fn call(&mut self, mut request: HttpRequest<tonic::body::Body>) -> Self::Future {
+        if let Some(deadline) = request.extensions().get::<BatchLockRpcDeadline>().cloned() {
+            let body = std::mem::replace(request.body_mut(), tonic::body::Body::empty());
+            *request.body_mut() = tonic::body::Body::new(BatchLockDeadlineBody {
+                inner: body,
+                deadline,
+                dispatched: false,
+                failed: false,
+            });
+        }
+        self.inner.call(request)
+    }
+}
+
+struct BatchLockDeadlineBody {
+    inner: tonic::body::Body,
+    deadline: BatchLockRpcDeadline,
+    dispatched: bool,
+    failed: bool,
+}
+
+impl http_body::Body for BatchLockDeadlineBody {
+    type Data = bytes::Bytes;
+    type Error = tonic::Status;
+
+    fn poll_frame(
+        mut self: Pin<&mut Self>,
+        cx: &mut Context<'_>,
+    ) -> Poll<Option<Result<http_body::Frame<Self::Data>, Self::Error>>> {
+        if self.failed {
+            return Poll::Ready(None);
+        }
+        if !self.dispatched
+            && let Err(error) = self.deadline.remaining_before_dispatch()
+        {
+            self.failed = true;
+            return Poll::Ready(Some(Err(error)));
+        }
+        let frame = std::task::ready!(Pin::new(&mut self.inner).poll_frame(cx));
+        if let Some(Ok(frame)) = &frame
+            && frame.is_data()
+            && !self.dispatched
+        {
+            // Codec work can consume the deadline in this poll. Arbitrate only
+            // after encoding, immediately before any payload leaves the body.
+            if let Err(error) = self.deadline.mark_dispatched() {
+                self.failed = true;
+                return Poll::Ready(Some(Err(error)));
+            }
+            self.dispatched = true;
+        }
+        Poll::Ready(frame)
+    }
+
+    fn is_end_stream(&self) -> bool {
+        self.failed || self.inner.is_end_stream()
+    }
+
+    fn size_hint(&self) -> http_body::SizeHint {
+        self.inner.size_hint()
+    }
+}

 #[derive(Clone, Copy, Debug, Eq, PartialEq)]
 enum PeerReplayCapability {
@@ -426,7 +518,9 @@ where
                    )
                }) {
                    Ok(headers) => request.headers_mut().extend(headers),
-                    Err(error) => debug!(error = %error, "could not attach replay-scoped RPC signature"),
+                    Err(error) => {
+                        debug!(error = %error, "could not attach replay-scoped RPC signature")
+                    }
                }
            }
        }
@@ -488,6 +582,82 @@ where
    }
 }

+/// A batch may leave local readiness queues only while its original budget is live.
+/// Cancellation and submission arbitrate once so a cancelled caller cannot send later.
+#[derive(Clone, Debug)]
+pub(crate) struct BatchLockRpcDeadline {
+    deadline: tokio::time::Instant,
+    max_acquire_wait: Duration,
+    dispatch: Arc<AtomicU8>,
+}
+
+impl BatchLockRpcDeadline {
+    const PENDING: u8 = 0;
+    const DISPATCHED: u8 = 1;
+    const CANCELLED: u8 = 2;
+
+    pub(crate) fn new(deadline: tokio::time::Instant, max_acquire_wait: Duration) -> Self {
+        Self {
+            deadline,
+            max_acquire_wait,
+            dispatch: Arc::new(AtomicU8::new(Self::PENDING)),
+        }
+    }
+
+    /// True proves that this batch cannot send payload or enter the server lock handler.
+    /// Transport headers may already have left an internal channel queue.
+    pub(crate) fn cancel_before_dispatch(&self) -> bool {
+        match self
+            .dispatch
+            .compare_exchange(Self::PENDING, Self::CANCELLED, Ordering::AcqRel, Ordering::Acquire)
+        {
+            Ok(_) => true,
+            Err(state) => state == Self::CANCELLED,
+        }
+    }
+
+    pub(crate) fn was_dispatched(&self) -> bool {
+        self.dispatch.load(Ordering::Acquire) == Self::DISPATCHED
+    }
+
+    fn remaining_before_dispatch(&self) -> Result<Duration, tonic::Status> {
+        let remaining = self.deadline.saturating_duration_since(tokio::time::Instant::now());
+        if remaining <= self.max_acquire_wait || self.dispatch.load(Ordering::Acquire) == Self::CANCELLED {
+            return Err(tonic::Status::deadline_exceeded("Lock batch budget exhausted before payload dispatch"));
+        }
+        Ok(remaining)
+    }
+
+    fn update_timeout(&self, request: &mut tonic::Request<()>) -> Result<(), tonic::Status> {
+        request.set_timeout(self.remaining_before_dispatch()?);
+        Ok(())
+    }
+
+    fn mark_dispatched(&self) -> Result<(), tonic::Status> {
+        self.remaining_before_dispatch()?;
+        self.dispatch
+            .compare_exchange(Self::PENDING, Self::DISPATCHED, Ordering::AcqRel, Ordering::Acquire)
+            .map(|_| ())
+            .map_err(|_| tonic::Status::deadline_exceeded("Lock batch cancelled before dispatch"))
+    }
+
+    #[cfg(test)]
+    pub(super) async fn dispatch_payload_for_test(&self) -> Result<(), tonic::Status> {
+        use http_body_util::BodyExt as _;
+        let mut body = BatchLockDeadlineBody {
+            inner: tonic::body::Body::new(http_body_util::Full::new(bytes::Bytes::from_static(b"payload"))),
+            deadline: self.clone(),
+            dispatched: false,
+            failed: false,
+        };
+        match body.frame().await {
+            Some(Ok(frame)) if frame.is_data() => Ok(()),
+            Some(Err(error)) => Err(error),
+            _ => Err(tonic::Status::internal("test lock payload did not produce a data frame")),
+        }
+    }
+}
+
 pub struct TonicSignatureInterceptor {
    audience: Option<String>,
    body_digest_strict: bool,
@@ -582,11 +752,21 @@ impl TonicInterceptor {
 }

 impl tonic::service::Interceptor for TonicInterceptor {
-    fn call(&mut self, req: tonic::Request<()>) -> Result<tonic::Request<()>, tonic::Status> {
-        match self {
+    fn call(&mut self, mut req: tonic::Request<()>) -> Result<tonic::Request<()>, tonic::Status> {
+        let batch_deadline = req.extensions().get::<BatchLockRpcDeadline>().cloned();
+        if let Some(deadline) = &batch_deadline {
+            deadline.update_timeout(&mut req)?;
+        }
+        let mut req = match self {
            TonicInterceptor::Signature(interceptor) => interceptor.call(req),
            TonicInterceptor::NoOp(interceptor) => interceptor.call(req),
+        }?;
+        if let Some(deadline) = batch_deadline {
+            // Signing consumes budget, but Channel readiness only reserves a
+            // queue slot. The payload body arbitrates actual dispatch later.
+            deadline.update_timeout(&mut req)?;
        }
+        Ok(req)
    }
 }

@@ -782,6 +962,136 @@ mod tests {
        test_request_for("Ping")
    }

+    #[test]
+    fn batch_dispatch_and_cancellation_have_one_winner() {
+        for _ in 0..100 {
+            let deadline = BatchLockRpcDeadline::new(tokio::time::Instant::now() + Duration::from_secs(1), Duration::ZERO);
+            let other = deadline.clone();
+            let barrier = Arc::new(std::sync::Barrier::new(2));
+            let other_barrier = barrier.clone();
+            let submit = std::thread::spawn(move || {
+                other_barrier.wait();
+                other.mark_dispatched().is_ok()
+            });
+            barrier.wait();
+            let cancelled = deadline.cancel_before_dispatch();
+            let submitted = submit.join().expect("submission thread");
+            assert_ne!(cancelled, submitted);
+            assert_eq!(deadline.was_dispatched(), submitted);
+            assert!(deadline.mark_dispatched().is_err());
+        }
+    }
+
+    #[test]
+    fn batch_interceptor_rejects_exhausted_wait_reserve_before_submission() {
+        let deadline =
+            BatchLockRpcDeadline::new(tokio::time::Instant::now() + Duration::from_millis(20), Duration::from_millis(30));
+        let mut request = tonic::Request::new(());
+        request.extensions_mut().insert(deadline.clone());
+        let status = TonicInterceptor::NoOp(NoOpInterceptor)
+            .call(request)
+            .expect_err("insufficient wait reserve");
+        assert_eq!(status.code(), tonic::Code::DeadlineExceeded);
+        assert!(!deadline.was_dispatched());
+        assert!(deadline.cancel_before_dispatch());
+    }
+
+    #[test]
+    fn batch_interceptor_rejects_cancelled_ready_request_and_accepts_live_budget() {
+        let deadline = BatchLockRpcDeadline::new(tokio::time::Instant::now() + Duration::from_secs(2), Duration::from_millis(1));
+        let mut request = tonic::Request::new(());
+        request.extensions_mut().insert(deadline.clone());
+        assert!(deadline.cancel_before_dispatch());
+        assert!(TonicInterceptor::NoOp(NoOpInterceptor).call(request).is_err());
+        assert!(!deadline.was_dispatched());
+
+        let deadline = BatchLockRpcDeadline::new(tokio::time::Instant::now() + Duration::from_secs(2), Duration::from_millis(1));
+        let mut request = tonic::Request::new(());
+        request.extensions_mut().insert(deadline.clone());
+        let request = TonicInterceptor::NoOp(NoOpInterceptor).call(request).expect("live budget");
+        assert!(!deadline.was_dispatched(), "interceptor completion does not bypass the transport queue");
+        assert!(request.metadata().contains_key("grpc-timeout"));
+        assert!(deadline.cancel_before_dispatch());
+    }
+
+    #[tokio::test(start_paused = true)]
+    async fn batch_body_rejects_payload_after_channel_queue_deadline() {
+        let deadline = BatchLockRpcDeadline::new(tokio::time::Instant::now() + Duration::from_millis(20), Duration::ZERO);
+        tokio::time::advance(Duration::from_millis(21)).await;
+        let error = deadline
+            .dispatch_payload_for_test()
+            .await
+            .expect_err("queued payload expired");
+        assert_eq!(error.code(), tonic::Code::DeadlineExceeded);
+        assert!(!deadline.was_dispatched());
+        assert!(deadline.cancel_before_dispatch());
+    }
+
+    #[tokio::test(start_paused = true)]
+    async fn batch_body_rejects_payload_when_encoding_crosses_deadline_in_one_poll() {
+        use http_body_util::BodyExt as _;
+        struct SlowEncodingBody {
+            deadline: tokio::time::Instant,
+        }
+        impl http_body::Body for SlowEncodingBody {
+            type Data = bytes::Bytes;
+            type Error = tonic::Status;
+
+            fn poll_frame(
+                self: Pin<&mut Self>,
+                _cx: &mut Context<'_>,
+            ) -> Poll<Option<Result<http_body::Frame<Self::Data>, Self::Error>>> {
+                // Model codec work that begins before the deadline but blocks
+                // this single poll until the original budget has expired.
+                tokio::time::resume();
+                std::thread::sleep(
+                    self.deadline.saturating_duration_since(tokio::time::Instant::now()) + Duration::from_millis(1),
+                );
+                Poll::Ready(Some(Ok(http_body::Frame::data(bytes::Bytes::from_static(b"encoded payload")))))
+            }
+        }
+        let expires = tokio::time::Instant::now() + Duration::from_millis(20);
+        let deadline = BatchLockRpcDeadline::new(expires, Duration::ZERO);
+        let mut body = BatchLockDeadlineBody {
+            inner: tonic::body::Body::new(SlowEncodingBody { deadline: expires }),
+            deadline: deadline.clone(),
+            dispatched: false,
+            failed: false,
+        };
+        let error = body
+            .frame()
+            .await
+            .expect("first frame result")
+            .expect_err("late encoded payload");
+        assert_eq!(error.code(), tonic::Code::DeadlineExceeded);
+        assert!(!deadline.was_dispatched());
+        assert!(deadline.cancel_before_dispatch());
+        assert!(body.frame().await.is_none(), "failed body must not expose a later frame");
+    }
+
+    #[tokio::test(start_paused = true)]
+    async fn batch_body_keeps_cleanup_responsibility_after_first_payload() {
+        use http_body_util::BodyExt as _;
+        let deadline = BatchLockRpcDeadline::new(tokio::time::Instant::now() + Duration::from_millis(20), Duration::ZERO);
+        let frames = futures::stream::iter([
+            Ok::<_, tonic::Status>(http_body::Frame::data(bytes::Bytes::from_static(b"first"))),
+            Ok(http_body::Frame::data(bytes::Bytes::from_static(b"last"))),
+        ]);
+        let mut body = BatchLockDeadlineBody {
+            inner: tonic::body::Body::new(http_body_util::StreamBody::new(frames)),
+            deadline: deadline.clone(),
+            dispatched: false,
+            failed: false,
+        };
+        assert!(body.frame().await.expect("first frame").expect("live payload").is_data());
+        assert!(deadline.was_dispatched());
+        assert!(!deadline.cancel_before_dispatch());
+        tokio::time::advance(Duration::from_millis(21)).await;
+        assert!(body.frame().await.expect("last frame").expect("late drain").is_data());
+        assert!(body.frame().await.is_none());
+        assert!(!deadline.cancel_before_dispatch(), "a sent payload requires late cleanup ownership");
+    }
+
    fn test_request_for(method: &'static str) -> tonic::Request<()> {
        let mut request = tonic::Request::new(());
        request
@@ -1965,6 +1965,15 @@ impl PeerRestClient {
        Ok((self.topology_member.clone(), epoch))
    }

+    pub async fn probe_restore_copyback_capability(&self, topology_fingerprint: String) -> Result<(String, Uuid)> {
+        let probe = rustfs_protos::restore_copyback_capability_probe(Uuid::new_v4().as_bytes());
+        let result = self
+            .heal_control(rustfs_protos::HEAL_CONTROL_PROTOCOL_VERSION, topology_fingerprint, probe)
+            .await?;
+        let epoch = decode_remote_version_state_capability(&self.topology_member, &result)?;
+        Ok((self.topology_member.clone(), epoch))
+    }
+
    pub async fn probe_cross_pool_fence(&self, topology_fingerprint: String) -> Result<(String, u32, Uuid)> {
        let mut probe = rustfs_protos::CROSS_POOL_FENCE_CAPABILITY_PROBE_PREFIX.to_vec();
        probe.extend_from_slice(Uuid::new_v4().as_bytes());
@@ -169,6 +169,72 @@ fn decode_delete_recovery_batch_errors(codes: Vec<Option<u32>>, expected_len: us
        .collect()
 }

+fn log_delete_recovery_batch_result(
+    endpoint: &Endpoint,
+    expected_len: usize,
+    started: Instant,
+    result: &Result<Vec<Option<DiskError>>>,
+) {
+    let elapsed_ms = started.elapsed().as_millis();
+    let mut disk_error_counts = std::collections::BTreeMap::<u32, usize>::new();
+    let (failure_scope, rpc_status) = match result {
+        Ok(errors) => {
+            for error in errors.iter().flatten() {
+                *disk_error_counts.entry(error.to_u32()).or_default() += 1;
+            }
+            ("object_group", None)
+        }
+        Err(error) => {
+            disk_error_counts.insert(error.to_u32(), expected_len);
+            let rpc_status = match error {
+                DiskError::Io(source) => crate::cluster::rpc::client::embedded_tonic_status(source).map(tonic::Status::code),
+                _ => None,
+            };
+            ("rpc_or_reply", rpc_status)
+        }
+    };
+    let failed_group_count = disk_error_counts.values().sum::<usize>();
+    let result = if result.is_err() || failed_group_count > 0 {
+        "failed"
+    } else {
+        "ok"
+    };
+    if result == "failed" || elapsed_ms >= 1_000 {
+        warn!(
+            event = EVENT_REMOTE_DISK_RPC,
+            component = LOG_COMPONENT_ECSTORE,
+            subsystem = LOG_SUBSYSTEM_REMOTE_DISK,
+            result,
+            operation = "apply_versions",
+            failure_scope,
+            pool_index = endpoint.pool_idx,
+            set_index = endpoint.set_idx,
+            disk_index = endpoint.disk_idx,
+            attempted_group_count = expected_len,
+            failed_group_count,
+            disk_error_counts = ?disk_error_counts,
+            rpc_status = ?rpc_status,
+            elapsed_ms,
+            "Recoverable batch delete RPC completed"
+        );
+    } else {
+        debug!(
+            event = EVENT_REMOTE_DISK_RPC,
+            component = LOG_COMPONENT_ECSTORE,
+            subsystem = LOG_SUBSYSTEM_REMOTE_DISK,
+            result,
+            operation = "apply_versions",
+            pool_index = endpoint.pool_idx,
+            set_index = endpoint.set_idx,
+            disk_index = endpoint.disk_idx,
+            attempted_group_count = expected_len,
+            failed_group_count,
+            elapsed_ms,
+            "Recoverable batch delete RPC completed"
+        );
+    }
+}
+
 fn validate_delete_recovery_inspection(
    volume: &str,
    object: &str,
@@ -931,8 +997,16 @@ pub struct RemoteDisk {
    #[cfg(test)]
    recovery_monitor_teardown_hook: Arc<tokio::sync::Mutex<Option<Arc<RecoveryMonitorTeardownHook>>>>,
    data_transport: Arc<dyn InternodeDataTransport>,
+    /// Bound the per-operation failure warnings of this peer. An unreachable
+    /// peer fails every in-flight shard operation; the counters keep the exact
+    /// numbers per operation. Per handle, so one peer cannot hide another's.
+    timeout_logs: rustfs_utils::LogThrottle,
+    network_error_logs: rustfs_utils::LogThrottle,
 }

+/// Shortest gap between two failure warnings of one kind from one remote disk.
+const REMOTE_DISK_RPC_FAILURE_LOG_INTERVAL_MS: u64 = 5_000;
+
 struct RecoveryMonitorLease {
    active: Arc<AtomicBool>,
 }
@@ -1364,6 +1438,8 @@ impl RemoteDisk {
            #[cfg(test)]
            recovery_monitor_teardown_hook: Arc::new(tokio::sync::Mutex::new(None)),
            data_transport,
+            timeout_logs: rustfs_utils::LogThrottle::new(REMOTE_DISK_RPC_FAILURE_LOG_INTERVAL_MS),
+            network_error_logs: rustfs_utils::LogThrottle::new(REMOTE_DISK_RPC_FAILURE_LOG_INTERVAL_MS),
        };
        record_drive_runtime_state(ep, RuntimeDriveHealthState::Online);

@@ -2090,17 +2166,20 @@ impl RemoteDisk {
            "op" => op.to_string()
        )
        .increment(1);
-        warn!(
-            event = EVENT_REMOTE_DISK_RPC,
-            component = LOG_COMPONENT_ECSTORE,
-            subsystem = LOG_SUBSYSTEM_REMOTE_DISK,
-            endpoint = %self.endpoint,
-            addr = %self.addr,
-            op,
-            timeout_ms = timeout_duration.as_millis(),
-            state = "timeout",
-            "Remote disk operation timed out"
-        );
+        if let Some(suppressed) = self.timeout_logs.claim() {
+            warn!(
+                event = EVENT_REMOTE_DISK_RPC,
+                component = LOG_COMPONENT_ECSTORE,
+                subsystem = LOG_SUBSYSTEM_REMOTE_DISK,
+                endpoint = %self.endpoint,
+                addr = %self.addr,
+                op,
+                timeout_ms = timeout_duration.as_millis(),
+                suppressed,
+                state = "timeout",
+                "Remote disk operation timed out"
+            );
+        }
    }

    async fn handle_network_like_error<T>(
@@ -2120,17 +2199,20 @@ impl RemoteDisk {
                "op" => op.to_string()
            )
            .increment(1);
-            warn!(
-                event = EVENT_REMOTE_DISK_RPC,
-                component = LOG_COMPONENT_ECSTORE,
-                subsystem = LOG_SUBSYSTEM_REMOTE_DISK,
-                endpoint = %self.endpoint,
-                addr = %self.addr,
-                op,
-                timeout_ms = timeout_duration.as_millis(),
-                state = "network_like_error",
-                "Remote disk operation returned network-like error"
-            );
+            if let Some(suppressed) = self.network_error_logs.claim() {
+                warn!(
+                    event = EVENT_REMOTE_DISK_RPC,
+                    component = LOG_COMPONENT_ECSTORE,
+                    subsystem = LOG_SUBSYSTEM_REMOTE_DISK,
+                    endpoint = %self.endpoint,
+                    addr = %self.addr,
+                    op,
+                    timeout_ms = timeout_duration.as_millis(),
+                    suppressed,
+                    state = "network_like_error",
+                    "Remote disk operation returned network-like error"
+                );
+            }
            if failure_health_action == FailureHealthAction::MarkFailure {
                self.mark_faulty_and_evict("operation_network_error").await;
            }
@@ -4373,6 +4455,7 @@ impl DiskAPI for RemoteDisk {
        opts: DeleteOptions,
    ) -> Vec<Option<DiskError>> {
        let expected_len = versions.len();
+        let started = Instant::now();
        let result = match self
            .recover_delete_transaction(volume, DeleteRecoveryRpcOperation::ApplyVersions { versions, opts })
            .await
@@ -4381,6 +4464,7 @@ impl DiskAPI for RemoteDisk {
            Ok(_) => Err(Error::other("unexpected recoverable batch delete reply")),
            Err(error) => Err(error),
        };
+        log_delete_recovery_batch_result(&self.endpoint, expected_len, started, &result);
        match result {
            Ok(errors) => errors,
            Err(error) => vec![Some(error); expected_len],
@@ -5673,6 +5757,62 @@ mod tests {
        assert!(decode_delete_recovery_batch_errors(vec![Some(u32::MAX)], 1).is_err());
    }

+    #[test]
+    fn recoverable_batch_delete_diagnostics_preserve_context_and_redact_error_details() {
+        let logs = CapturedLogs::default();
+        let subscriber = Registry::default().with(
+            tracing_subscriber::fmt::layer()
+                .with_writer(logs.clone())
+                .with_ansi(false)
+                .without_time()
+                .json()
+                .flatten_event(true)
+                .with_span_list(true),
+        );
+        let _guard = tracing::subscriber::set_default(subscriber);
+        let _callsite_pin = crate::test_tracing::pin_callsite_interest_for_test();
+        let endpoint = Endpoint {
+            url: url::Url::parse("http://127.0.0.1:9000/private-disk-path").expect("valid diagnostic endpoint"),
+            is_local: false,
+            pool_idx: 1,
+            set_idx: 2,
+            disk_idx: 3,
+        };
+        let rpc_failure = Err(DiskError::from(Status::deadline_exceeded("private-error-detail")));
+        let group_failures = Ok(vec![None, Some(DiskError::FileNotFound), Some(DiskError::FileNotFound)]);
+        tracing::info_span!("request-span", request_id = "req-batch-delete-diagnostics").in_scope(|| {
+            log_delete_recovery_batch_result(&endpoint, 1000, Instant::now(), &rpc_failure);
+            log_delete_recovery_batch_result(&endpoint, 3, Instant::now(), &group_failures);
+        });
+        let entries = logs.lines();
+        assert_eq!(entries.len(), 2, "one summary per disk batch, never per object");
+        assert_eq!(entries[0]["failure_scope"], "rpc_or_reply");
+        assert_eq!(entries[0]["failed_group_count"], 1000);
+        assert_eq!(entries[0]["rpc_status"], "Some(DeadlineExceeded)");
+        assert_eq!(entries[1]["failure_scope"], "object_group");
+        assert_eq!(entries[1]["failed_group_count"], 2);
+        for entry in entries {
+            assert_eq!(entry["pool_index"], 1);
+            assert_eq!(entry["set_index"], 2);
+            assert_eq!(entry["disk_index"], 3);
+            assert!(
+                entry["spans"]
+                    .as_array()
+                    .expect("request spans recorded")
+                    .iter()
+                    .any(|span| { span["request_id"] == "req-batch-delete-diagnostics" })
+            );
+            let text = entry.to_string();
+            assert!(!text.contains("private-error-detail"));
+            assert!(!text.contains("private-disk-path"));
+        }
+        assert!(rpc_failure.is_err(), "diagnostics borrow and preserve the original failure");
+        assert_eq!(
+            group_failures,
+            Ok(vec![None, Some(DiskError::FileNotFound), Some(DiskError::FileNotFound)])
+        );
+    }
+
    #[test]
    fn delete_recovery_inspection_rejects_wrong_transaction_object_and_protocol() {
        let transaction = Uuid::new_v4();
@@ -11688,6 +11828,66 @@ mod tests {
        );
    }

+    /// A peer that fails every operation logs one warning per interval, each
+    /// operation still gets its own error, and another peer's first warning is
+    /// not held back by the noisy one.
+    #[tokio::test]
+    async fn rpc_failure_warnings_are_bounded_per_remote_disk() {
+        async fn remote_disk(port: u16) -> RemoteDisk {
+            let url = url::Url::parse(&format!("http://127.0.0.1:{port}/data")).expect("endpoint url");
+            let endpoint = Endpoint {
+                url,
+                is_local: false,
+                pool_idx: 0,
+                set_idx: 0,
+                disk_idx: 0,
+            };
+            RemoteDisk::new(
+                &endpoint,
+                &DiskOption {
+                    cleanup: false,
+                    health_check: false,
+                },
+                Arc::new(TcpHttpInternodeDataTransport),
+            )
+            .await
+            .expect("remote disk handle")
+        }
+        let noisy = remote_disk(59981).await;
+        let quiet = remote_disk(59982).await;
+
+        for _ in 0..50 {
+            let error = noisy
+                .execute_with_timeout_for_op_and_health_action(
+                    "read_all",
+                    || async {
+                        Err::<(), Error>(DiskError::Io(std::io::Error::new(
+                            std::io::ErrorKind::ConnectionRefused,
+                            "connection refused",
+                        )))
+                    },
+                    Duration::from_secs(1),
+                    FailureHealthAction::IgnoreFailure,
+                )
+                .await
+                .expect_err("every failing operation still fails");
+            assert!(matches!(error, DiskError::Io(io) if io.kind() == std::io::ErrorKind::ConnectionRefused));
+            noisy.record_timeout("read_all", Duration::from_secs(1));
+        }
+
+        assert!(
+            noisy.network_error_logs.claim().is_none(),
+            "the burst used this peer's network-error window"
+        );
+        assert!(noisy.timeout_logs.claim().is_none(), "the burst used this peer's timeout window");
+        assert_eq!(
+            quiet.network_error_logs.claim(),
+            Some(0),
+            "another peer's first warning is not suppressed"
+        );
+        assert_eq!(quiet.timeout_logs.claim(), Some(0));
+    }
+
    #[tokio::test]
    async fn test_execute_with_timeout_keeps_remote_disk_online_for_business_error() {
        let addr = "http://127.0.0.1:59994".to_string();
@@ -84,12 +84,13 @@ use rand::RngExt as _;
 use rmp_serde::Deserializer;
 use rmp_serde::Serializer;
 use rustfs_filemeta::{FileInfo, FileInfoVersions, MetaCacheEntries, MetaCacheEntry, MetadataResolutionParams};
+use rustfs_gateway_types::persistence::PersistedObjectLockConfiguration as ObjectLockConfiguration;
+use rustfs_gateway_types::persistence::{PersistedLifecycleConfiguration, PersistedReplicationConfiguration};
 use rustfs_heal_contracts::heal_channel::HealOpts;
 use rustfs_utils::crypto::{hex_sha256, is_sha256_checksum};
 use rustfs_utils::path::{
    decode_dir_object, encode_dir_object, path_join, path_to_bucket_object, path_to_bucket_object_with_base_path,
 };
-use s3s::dto::{BucketLifecycleConfiguration, ObjectLockConfiguration, ReplicationConfiguration};
 use serde::{Deserialize, Serialize};
 use sha2::{Digest, Sha256};
 use std::collections::{HashMap, HashSet};
@@ -13115,7 +13116,7 @@ pub(crate) async fn should_skip_lifecycle_for_data_movement(
    store: Arc<ECStore>,
    bucket: &str,
    version: &rustfs_filemeta::FileInfo,
-    lifecycle_config: Option<&BucketLifecycleConfiguration>,
+    lifecycle_config: Option<&PersistedLifecycleConfiguration>,
    object_lock_config: Option<&ObjectLockConfiguration>,
    apply_actions: bool,
    event_source: &LcEventSrc,
@@ -17046,9 +17047,9 @@ impl ECStore {
        generation: OffsetDateTime,
        bucket: String,
        set: Arc<SetDisks>,
-        lifecycle_config: Option<BucketLifecycleConfiguration>,
+        lifecycle_config: Option<PersistedLifecycleConfiguration>,
        object_lock_config: Option<ObjectLockConfiguration>,
-        replication_config: Option<(ReplicationConfiguration, OffsetDateTime)>,
+        replication_config: Option<(PersistedReplicationConfiguration, OffsetDateTime)>,
        expected_bucket_incarnation_id: Option<uuid::Uuid>,
        source_changed_exhaustions: Arc<AtomicUsize>,
        entry_budget: Arc<Semaphore>,
@@ -17220,9 +17221,9 @@ impl ECStore {
        mut deferred_entry: DeferredDecommissionEntry,
        bucket: String,
        set: Arc<SetDisks>,
-        lifecycle_config: Option<BucketLifecycleConfiguration>,
+        lifecycle_config: Option<PersistedLifecycleConfiguration>,
        object_lock_config: Option<ObjectLockConfiguration>,
-        replication_config: Option<(ReplicationConfiguration, OffsetDateTime)>,
+        replication_config: Option<(PersistedReplicationConfiguration, OffsetDateTime)>,
        expected_bucket_incarnation_id: Option<uuid::Uuid>,
        source_changed_exhaustions: Arc<AtomicUsize>,
        entry_budget: Arc<Semaphore>,
@@ -17329,9 +17330,9 @@ impl ECStore {
        generation: OffsetDateTime,
        set: Arc<SetDisks>,
        bi: DecomBucketInfo,
-        lifecycle_config: Option<BucketLifecycleConfiguration>,
+        lifecycle_config: Option<PersistedLifecycleConfiguration>,
        object_lock_config: Option<ObjectLockConfiguration>,
-        replication_config: Option<(ReplicationConfiguration, OffsetDateTime)>,
+        replication_config: Option<(PersistedReplicationConfiguration, OffsetDateTime)>,
        expected_bucket_incarnation_id: Option<uuid::Uuid>,
        source_changed_exhaustions: Arc<AtomicUsize>,
        entry_budget: Arc<Semaphore>,
@@ -17700,6 +17701,16 @@ impl ECStore {
            .map(|_| ())
    }

+    #[cfg(feature = "test-util")]
+    pub async fn complete_scanner_pause_backlog_retirement_for_test(&self, source_pool_index: usize) -> Result<()> {
+        // Native backlog fixtures use bounded capacity inputs rather than the
+        // host filesystem's unrelated usage. Keep the real ILM completion gate.
+        let run_token = self.durable_ilm_receipt_run_token(source_pool_index).await?;
+        self.persist_decommission_durable_ilm_manifest_for_run(source_pool_index, &run_token)
+            .await?;
+        self.complete_decommission(source_pool_index).await
+    }
+
    #[cfg(feature = "test-util")]
    pub async fn retire_scanner_pause_backlog_for_test(
        self: &Arc<Self>,
@@ -18006,9 +18017,9 @@ impl ECStore {
        entry: MetaCacheEntry,
        bucket: String,
        set: Arc<SetDisks>,
-        lifecycle_config: Option<BucketLifecycleConfiguration>,
+        lifecycle_config: Option<PersistedLifecycleConfiguration>,
        object_lock_config: Option<ObjectLockConfiguration>,
-        replication_config: Option<(ReplicationConfiguration, OffsetDateTime)>,
+        replication_config: Option<(PersistedReplicationConfiguration, OffsetDateTime)>,
        expected_bucket_incarnation_id: Option<uuid::Uuid>,
        source_changed_exhaustions: Arc<AtomicUsize>,
        attempts: &mut usize,
@@ -18192,9 +18203,9 @@ impl ECStore {
        entry: MetaCacheEntry,
        bucket: String,
        set: Arc<SetDisks>,
-        lifecycle_config: Option<BucketLifecycleConfiguration>,
+        lifecycle_config: Option<PersistedLifecycleConfiguration>,
        object_lock_config: Option<ObjectLockConfiguration>,
-        replication_config: Option<(ReplicationConfiguration, OffsetDateTime)>,
+        replication_config: Option<(PersistedReplicationConfiguration, OffsetDateTime)>,
        expected_bucket_incarnation_id: Option<uuid::Uuid>,
        entry_attempt: usize,
        scanner_backlog_handoff_attempt: usize,
@@ -19480,7 +19491,7 @@ impl ECStore {
        entry: MetaCacheEntry,
        bucket: String,
        set: Arc<SetDisks>,
-        lifecycle_config: Option<BucketLifecycleConfiguration>,
+        lifecycle_config: Option<PersistedLifecycleConfiguration>,
    ) -> Result<()> {
        let expected_bucket_incarnation_id = if is_meta_bucketname(&bucket) {
            None
@@ -65,6 +65,38 @@ use uuid::Uuid;

 const LIST_MULTIPART_SETS_CONCURRENCY: usize = 4;

+#[cfg(test)]
+type TestSetLockers = HashMap<usize, Vec<Arc<dyn rustfs_lock::LockClient>>>;
+
+#[cfg(test)]
+static TEST_SET_LOCKERS: std::sync::LazyLock<std::sync::Mutex<TestSetLockers>> =
+    std::sync::LazyLock::new(|| std::sync::Mutex::new(HashMap::new()));
+
+/// Keep a fixture's real locker domain scoped to its owning instance.
+#[cfg(test)]
+pub(crate) struct ScopedSetLockers(Arc<InstanceContext>);
+
+#[cfg(test)]
+impl ScopedSetLockers {
+    pub(crate) fn install(ctx: Arc<InstanceContext>, lockers: Vec<Arc<dyn rustfs_lock::LockClient>>) -> Self {
+        let key = Arc::as_ptr(&ctx) as usize;
+        let mut domains = TEST_SET_LOCKERS.lock().expect("test set lockers mutex");
+        assert!(!domains.contains_key(&key), "the instance already owns a test locker domain");
+        domains.insert(key, lockers);
+        Self(ctx)
+    }
+}
+
+#[cfg(test)]
+impl Drop for ScopedSetLockers {
+    fn drop(&mut self) {
+        TEST_SET_LOCKERS
+            .lock()
+            .expect("test set lockers mutex")
+            .remove(&(Arc::as_ptr(&self.0) as usize));
+    }
+}
+
 fn is_idempotent_delete_prefix_error(err: &Error) -> bool {
    is_err_object_not_found(err) || is_err_strict_volume_not_found(err)
 }
@@ -266,6 +298,14 @@ impl Sets {
                }
            }

+            #[cfg(test)]
+            let pool_lockers = TEST_SET_LOCKERS
+                .lock()
+                .expect("test set lockers mutex")
+                .get(&(Arc::as_ptr(&instance_ctx) as usize))
+                .cloned()
+                .unwrap_or_else(|| pool_lockers.clone());
+
            let set_disks = SetDisks::new_with_instance_ctx(
                runtime_sources::local_node_name().await,
                Arc::new(RwLock::new(set_drive)),
@@ -85,6 +85,7 @@ impl ScannerPauseBacklogRetirementPhase {
 pub struct ScannerPauseBacklogRetirementReplica {
    pub pool_index: usize,
    pub set_index: usize,
+    pub set_id: Option<uuid::Uuid>,
    /// False for retiring members retained only to verify the old commit proof.
    pub writable: bool,
    pub data: Option<Vec<u8>>,
@@ -140,6 +141,7 @@ async fn read_replica((set, writable): (Arc<SetDisks>, bool)) -> Result<ScannerP
    let mut replica = ScannerPauseBacklogRetirementReplica {
        pool_index: set.pool_index,
        set_index: set.set_index,
+        set_id: Some(set.persistent_set_id()?),
        writable,
        data: None,
    };
@@ -222,6 +224,7 @@ pub(crate) fn plan_scanner_pause_backlog_retirement(
        .map(|read| ScannerPauseBacklogRetirementReplica {
            pool_index: read.replica.pool_index,
            set_index: read.replica.set_index,
+            set_id: read.replica.set_id,
            writable: read.replica.writable,
            data: read.replica.data.clone(),
        })
@@ -685,12 +685,8 @@ async fn clear_bucket_usage_memory(bucket: &str, guard: Option<&rustfs_lock::Nam
 }

 pub async fn remove_bucket_usage_from_backend(store: Arc<ECStore>, bucket: &str) -> Result<(), Error> {
-    remove_bucket_usage_for_namespace_change(store.as_ref(), bucket).await
-}
-
-pub(crate) async fn remove_bucket_usage_for_namespace_change(store: &ECStore, bucket: &str) -> Result<(), Error> {
    prepare_bucket_usage_for_namespace_change(bucket, None).await?;
-    remove_bucket_usage_from_backend_with_guard_fenced(store, bucket, None).await
+    remove_bucket_usage_with_owned_namespace(store, bucket, None).await
 }

 pub(crate) async fn prepare_bucket_usage_for_namespace_change(
@@ -730,11 +726,37 @@ where
 pub(crate) async fn remove_bucket_usage_from_backend_with_guard_fenced(
    store: &ECStore,
    bucket: &str,
-    guard: Option<&rustfs_lock::NamespaceLockGuard>,
+    guard: Option<&Arc<rustfs_lock::NamespaceLockGuard>>,
 ) -> Result<(), Error> {
-    let result = remove_bucket_usage_from_backend_with_store_and_guard_and_publication(store, bucket, guard, Some(store)).await;
-    invalidate_bucket_usage_snapshot_caches(guard, bucket).await?;
-    result
+    let owned_store = crate::bucket::metadata_sys::object_store_in(&store.ctx).await?;
+    if !std::ptr::eq(store, owned_store.as_ref()) {
+        return Err(Error::other("usage cleanup requires its own metadata store"));
+    }
+    remove_bucket_usage_with_owned_namespace(owned_store, bucket, guard.cloned()).await
+}
+
+async fn remove_bucket_usage_with_owned_namespace(
+    store: Arc<ECStore>,
+    bucket: &str,
+    guard: Option<Arc<rustfs_lock::NamespaceLockGuard>>,
+) -> Result<(), Error> {
+    let bucket = bucket.to_owned();
+    let ctx = Arc::clone(&store.ctx);
+    // Retain the bucket generation and movement/object guards through every
+    // physical commit even when the namespace request stops waiting.
+    ctx.run_detached_mutation(async move {
+        let result = remove_bucket_usage_from_backend_with_store_and_guard_and_publication(
+            store.as_ref(),
+            &bucket,
+            guard.as_deref(),
+            Some(store.as_ref()),
+        )
+        .await;
+        invalidate_bucket_usage_snapshot_caches(guard.as_deref(), &bucket).await?;
+        result
+    })
+    .await
+    .map_err(Error::from)?
 }

 async fn invalidate_bucket_usage_snapshot_caches(
@@ -849,9 +871,9 @@ where
    let primary_seed_epoch = match publication_store {
        Some(publication_store) => Some(
            publication_store
-                .scanner_data_usage_publication_epoch()
+                .data_usage_namespace_cleanup_epoch()
                .await
-                .ok_or_else(|| Error::other("data usage publication is blocked by data movement"))?,
+                .ok_or_else(|| Error::other("data usage namespace cleanup epoch is unavailable"))?,
        ),
        None => None,
    };
@@ -871,9 +893,9 @@ where
    let backup_seed_epoch = match publication_store {
        Some(publication_store) => Some(
            publication_store
-                .scanner_data_usage_publication_epoch()
+                .data_usage_namespace_cleanup_epoch()
                .await
-                .ok_or_else(|| Error::other("data usage publication is blocked by data movement"))?,
+                .ok_or_else(|| Error::other("data usage namespace cleanup epoch is unavailable"))?,
        ),
        None => None,
    };
@@ -985,9 +1007,9 @@ where
        let read_epoch = match publication {
            Some((publication_store, expected_publication_epoch)) => {
                let epoch = publication_store
-                    .scanner_data_usage_publication_epoch()
+                    .data_usage_namespace_cleanup_epoch()
                    .await
-                    .ok_or_else(|| Error::other("data usage publication is blocked by data movement"))?;
+                    .ok_or_else(|| Error::other("data usage namespace cleanup epoch is unavailable"))?;
                if expected_publication_epoch.is_some_and(|expected| expected != epoch) {
                    return Err(Error::other("data usage publication epoch changed before snapshot read"));
                }
@@ -1022,9 +1044,8 @@ where
        ensure_bucket_namespace_guard(guard, bucket, "data usage snapshot commit")?;
        let publication_guard = match publication {
            Some((publication_store, expected_publication_epoch)) => {
-                let Some((guard, publication_epoch)) = publication_store.scanner_data_usage_publication_admission_guard().await
-                else {
-                    return Err(Error::other("data usage publication is blocked by data movement"));
+                let Some((guard, publication_epoch)) = publication_store.data_usage_namespace_cleanup_guard().await else {
+                    return Err(Error::other("data usage namespace cleanup epoch is unavailable"));
                };
                if expected_publication_epoch
                    .or(read_epoch)
@@ -1036,18 +1057,27 @@ where
            }
            None => None,
        };
-        let save_result = store
-            .put_object(
-                RUSTFS_META_BUCKET,
-                object,
-                &mut put_data,
-                &ObjectOptions {
-                    max_parity: true,
-                    http_preconditions: Some(http_preconditions),
-                    ..Default::default()
-                },
-            )
-            .await;
+        let save_result = match publication {
+            Some((publication_store, _)) => {
+                publication_store
+                    .put_usage_snapshot_for_namespace_cleanup(object, revision.as_deref(), &mut put_data, guard)
+                    .await
+            }
+            None => {
+                store
+                    .put_object(
+                        RUSTFS_META_BUCKET,
+                        object,
+                        &mut put_data,
+                        &ObjectOptions {
+                            max_parity: true,
+                            http_preconditions: Some(http_preconditions),
+                            ..Default::default()
+                        },
+                    )
+                    .await
+            }
+        };
        drop(publication_guard);
        match save_result.map_err(|err| crate::config::com::map_system_metadata_write_error(err, object)) {
            Ok(_) => return Ok(()),
@@ -3053,6 +3053,15 @@ impl DiskAPI for LocalDiskWrapper {
        })
    }

+    async fn bucket_creation_witness(&self, volume: &str) -> Result<Option<super::os::NativeBucketCreationWitness>> {
+        self.track_disk_health_with_op(
+            "bucket_creation_witness",
+            || async { self.disk.bucket_creation_witness(volume).await },
+            Duration::ZERO,
+        )
+        .await
+    }
+
    async fn make_volume(&self, volume: &str) -> Result<()> {
        // Scoped heal must drain directory creation before releasing its lifecycle owner.
        let timeout = if crate::store::bucket_heal_scope(volume).is_some() {
--- a/Show More
+++ b/Show More