test(rpc): expose msgpack fleet confirmation

Co-Authored-By: heihutu <heihutu@gmail.com>

Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

.agents/references/adversarial-validation.md

@@ -1,33 +1,13 @@
 # Adversarial Review Shape
 
-Use when root `AGENTS.md` triggers adversarial validation or for a substantial PR
-review. Paths below are repository-relative. The root finding standard and
-completion rule apply; selecting a lens does not require finding a defect.
+Use when root `AGENTS.md` triggers adversarial validation or for a substantial PR review. Paths below are repository-relative. The root finding standard and completion rule apply; selecting a lens does not require finding a defect.
 
 Risk and review shape:
 
-- **Exempt:** documentation, comments, formatting, or typos with no runtime,
-  build, test, or agent-execution effect.
-- **Mechanical:** renames, moves, test/tooling-only changes, and agent-rule
-  changes. Run correctness and simplicity lenses.
-- **Standard:** localized behavior changes. Run one integrated final-diff pass
-  covering correctness, simplicity, and test coverage; add only domain lenses
-  matched by the diff.
-- **High risk / substantial PR review:** high risk includes locking,
-  erasure/quorum/heal, replication, multipart, RPC, lifecycle/tiering,
-  persistence/fsync, IAM/KMS/auth, cryptography, on-disk/on-wire formats, and
-  S3-visible semantics. Cover all applicable lenses using exactly two
-  independent reviewers when delegation is explicitly authorized. Split the
-  lenses between them. Otherwise perform two fresh sequential passes.
-- **Outbound client defaults:** what `TargetClient`, `PutObjectOptions`, or
-  the remote SDK configuration sends to every replication or migration target
-  is high risk for every target class even when the change fixes one. Follow
-  the SOP in `docs/postmortems/2026-09-03-replication-checksum-default-regression.md`:
-  run the outbound target matrix, document each new env knob in the same PR,
-  and list verified and unverified target classes in the PR Impact section.
+- **Exempt:** documentation, comments, formatting, or typos with no runtime, build, test, or agent-execution effect.
+- **Mechanical:** changes shown to preserve behavior, such as a symbol rename with unchanged callers or a file move with equivalent wiring. Run correctness and simplicity lenses. Classify tests, tooling, CI, and agent instructions by the behavior and authority they change, not their file type; deleting regression protection or changing failure propagation, publishing, or destructive actions can be high risk.
+- **Standard:** localized behavior changes. Run one integrated final-diff pass covering correctness, simplicity, and test coverage; add only domain lenses matched by the diff.
+- **High risk / substantial PR review:** high risk includes locking, erasure/quorum/heal, replication, multipart, RPC, lifecycle/tiering, persistence/fsync, IAM/KMS/auth, cryptography, on-disk/on-wire formats, and S3-visible semantics. Cover the applicable lenses from two perspectives: first trace the intended contract and implementation, then independently challenge its assumptions, failure paths, and evidence. Use separate reviewers when delegation is authorized and useful, or fresh sequential passes otherwise. Choose staffing and split lenses to fit the risk; a fixed reviewer count or duplicated review is not the goal.
+- **Outbound client defaults:** what `TargetClient`, `PutObjectOptions`, or the remote SDK configuration sends to every replication or migration target is high risk for every target class even when the change fixes one. Follow the SOP in `docs/postmortems/2026-09-03-replication-checksum-default-regression.md`: run the outbound target matrix, document each new env knob in the same PR, and list verified and unverified target classes in the PR Impact section.
 
-Available domain lenses are security, concurrency/durability, compatibility,
-and performance. Select `.agents/skills/adversarial-validation/SKILL.md` for an
-explicit adversarial request, a high-risk change, or a substantial PR review;
-then read only its matching role references. A routine standard pass does not
-load the playbook unless the reviewer needs a RustFS-specific probe.
+Available domain lenses are security, concurrency/durability, compatibility, and performance; select only those matched by the change.

.agents/references/implementation.md

@@ -1,72 +1,79 @@
 # Implementation Rules
 
-Applies when changing code or running artifact-heavy work. Paths below
-are repository-relative. Read only the relevant sections during read-only review.
+These implementation, style, and quality rules apply to all contributions, regardless of author or editor. Paths below are repository-relative. Read only the relevant sections during read-only review; select verification from the root `AGENTS.md`.
+
+## Shared Style Contract
+
+- Use the toolchain declared in `rust-toolchain.toml`, respect the minimum supported Rust version in `Cargo.toml`, and let `rustfmt.toml` and the configured Rust/Clippy lints define mechanical style. Do not override repository settings with personal formatter or lint preferences.
+- Follow the owning module's organization, imports, and domain terminology where repository rules leave a choice. Existing violations do not justify new ones; avoid unrelated cleanup in a focused change.
+- Rules using "must" or "do not" are requirements. Rules using "prefer" are defaults; depart when correctness, ownership, performance, or readability benefits justify it. Explain consequential tradeoffs, not every idiomatic choice. Where several approaches meet the contract equally well, the implementer chooses.
+- Follow the declared toolchain locally and the affected CI lane's toolchain for reproduction. The repository currently tracks `stable`; for a formatter/lint mismatch, compare `rustc -Vv`, `cargo fmt --version`, and `cargo clippy --version` with the failing run before changing code. Toolchain upgrades must keep local and CI settings aligned and pass their affected gates; do not silently raise `rust-version` or change lint policy to make a check pass.
 
 ## Worktree and Disk Hygiene
 
-- Start implementation from the latest `origin/main` and confirm the requested
-  change is not already present.
-- An existing clean, isolated task worktree is sufficient. Create another
-  worktree only when the current checkout is shared, dirty with unrelated work,
-  or belongs to another task.
+- For a new task, start from the latest `origin/main` unless a different base is requested, and confirm the requested change is not already present. When continuing existing work or a PR, preserve its branch, base, and task-owned changes; do not restart it from `main`.
+- An existing clean, isolated task worktree is sufficient. Create another worktree only when the current checkout is shared, dirty with unrelated work, or belongs to another task.
 - Never commit from a shared checkout.
-- Use a task-specific branch named `<type>/<topic>`, such as `fix/...`,
-  `feat/...`, `test/...`, or `docs/...`, unless the user specifies a name.
-- Do not include agent, tool, contributor, account, or organization names in
-  branch names.
-- Push to the user-requested remote or the repository's configured push remote.
-  Do not hard-code or infer a remote from an account name.
-- Check free space before artifact-heavy builds, tests, coverage, or downloads.
-  Re-check before a broad gate when space is tight.
-- Remove only task-owned temporary/build artifacts. Never delete another task's
-  worktree or uncommitted data.
-- At handoff, mention disk or cleanup details only when they affected execution
-  or artifacts/worktrees remain intentionally.
+- Use a task-specific branch named `<type>/<topic>`, such as `fix/...`, `feat/...`, `test/...`, or `docs/...`, unless the user specifies a name.
+- Do not include agent, tool, contributor, account, or organization names in branch names.
+- Push to the user-requested remote or the repository's configured push remote. Do not hard-code or infer a remote from an account name.
+- Check free space before artifact-heavy builds, tests, coverage, or downloads. Re-check before a broad gate when space is tight.
+- Before handoff, remove the Cargo target directories and temporary worktrees the task created unless they stay intentionally or hold unmerged work. Identify build output by Cargo's markers (`.rustc_info.json`, `CACHEDIR.TAG`), not by directory name: `target` is also a source directory here.
+- Remove only task-owned temporary/build artifacts. Never delete another task's worktree or uncommitted data.
+- At handoff, mention disk or cleanup details only when they affected execution or artifacts/worktrees remain intentionally.
+- For Docker verification, ensure the tested image contains the intended binary. `buildx build` without `--load` leaves the image in the build cache; use the Makefile's loaded image and rebuild without cache only when stale artifacts are evidenced.
 
 ## Change Style
 
-- Preserve existing control flow unless changing it is required for correctness.
-- Prefer a direct local edit over new files, wrappers, managers, or speculative
-  abstractions.
-- Add a helper only when it removes current duplication, names a real domain
-  boundary, or isolates a non-trivial invariant.
-- Remove an in-scope path superseded by the change. If compatibility requires it,
-  adapt at the boundary to one canonical core and use the repository's
-  `RUSTFS_COMPAT_TODO` policy.
-- Comments explain non-obvious invariants or reasons. Do not narrate code or
-  record change history.
+- Preserve unrelated control flow. Restructure touched code only when needed for the requested behavior or an explicitly requested refactor; style preferences alone do not justify changing execution order or cleanup behavior.
+- Prefer a direct local edit over new files, wrappers, managers, or speculative abstractions.
+- Add a helper only when it removes current duplication, names a real domain boundary, or isolates a non-trivial invariant. Keep helpers near the code they support when the module's organization permits it.
+- Remove an in-scope path superseded by the change. If compatibility requires it, adapt at the boundary to one canonical core and use the repository's `RUSTFS_COMPAT_TODO` policy.
+- Comments explain non-obvious invariants or reasons. Do not narrate code or record change history.
 - Mention unrelated problems when useful; do not fix them in a narrow task.
 
+## Rust Style
+
+These repository-owned rules incorporate selected ideas from the [OpenCode style guide](https://github.com/anomalyco/opencode/blob/dev/AGENTS.md#style-guide), adapted to RustFS. That link is background, not an additional rule source. Apply these defaults to new or changed code; correctness and domain invariants take priority.
+
+- Default to immutable `let` bindings; use `mut` for actual state changes. Prefer `if` or `match` expressions when branches compute a value instead of initializing a placeholder and reassigning it.
+- Use type inference for obvious local values. Add annotations when inference needs help or the numeric, collection, or error type matters to understanding the code. Keep explicit types in function signatures and public contracts.
+- Keep the main path readable with guard clauses, `let ... else`, and `?` when immediate propagation is correct. Preserve explicit handling for recovery, cleanup, aggregation, and quorum decisions; `if`/`else` expressions and exhaustive `match` remain idiomatic.
+- Prefer readable iterator chains for straightforward mapping and filtering. Keep loops when they make state changes, early exits, or async operations clearer. Do not add intermediate collections or clones just to use iterators.
+- Inline trivial single-use expressions only when a binding adds no meaning. Keep names that explain domain concepts, aid debugging, or make ownership and guard lifetimes clear. Do not change lock release or drop timing for style.
+- Keep pure parsing, validation, and option-building helpers synchronous unless an async interface contract requires otherwise. This does not make blocking I/O or CPU-heavy work safe to run on async executor threads; offload it when needed to keep those threads responsive.
+
 ## Reuse and Boundary Rules
 
-- Before adding helpers, constants, fixtures, or wrappers, search the touched
-  crate, the domain-owning crate, `crates/utils`, `crates/common`, and relevant
-  direct dependencies.
-- Reuse requires matching semantics: normalization, error types, deadlines,
-  durability, and compatibility must fit the call site. A narrowly named local
-  helper is better than forced reuse with different semantics.
-- Validate untrusted input at its trust boundary, then trust the validated type.
-  Values crossing disk, RPC, persistence, or version boundaries remain
-  untrusted at every consumer.
-- Re-check boundary values immediately before destructive actions such as
-  delete, overwrite, or quorum decisions.
-- Every new branch needs a concrete triggering input/state. For decoded or peer
-  data, corruption and mixed-version input are valid triggers.
-- Required values must return a typed error when absent or corrupt; do not use a
-  default that converts corruption into a plausible result.
-- Attach error context once where it is actionable. Do not erase typed errors
-  below aggregation or quorum layers.
+- Before adding helpers, constants, fixtures, or wrappers, search the touched code and domain owner; check `crates/utils`, `crates/common`, and direct dependencies when they could own the behavior. Stop once the relevant reuse boundary is clear.
+- Reuse requires matching semantics: normalization, error types, deadlines, durability, and compatibility must fit the call site. A narrowly named local helper is better than forced reuse with different semantics.
+- Validate untrusted input at the boundary that owns it, then pass validated types inward without repeating checks whose guarantees still hold. Deserializing into a Rust type alone does not establish domain validity. Values newly read from disk, RPC, persistence, or another version must be validated at that consuming boundary.
+- For destructive actions such as delete or overwrite and for quorum decisions, check the relevant identity, generation, and authority under the operation's existing synchronization or precondition contract. A separate unguarded recheck does not prevent a check/use race.
+- Every new control-flow branch needs a concrete triggering input/state. For decoded or peer data, corruption and mixed-version input are valid triggers.
+- Required values must return a typed error when absent or corrupt; do not use a default that converts corruption into a plausible result.
+- Attach error context once where it is actionable. Do not erase typed errors below aggregation or quorum layers.
 
 ## Rust Boundaries
 
-- Production `unwrap`/`expect` must follow a type guarantee or checked invariant; explain only non-obvious guarantees.
+- Production `unwrap`/`expect` must follow a type guarantee or checked invariant; explain only non-obvious guarantees. Stricter scoped prohibitions, including `crates/AGENTS.md`, still apply. Invalid client, disk, or peer data must return errors rather than trigger panics.
 - Public APIs do not return `Result<_, String>`. Library APIs use domain errors unless intentional erasure is part of the boundary contract; expose a stored inner error through `Error::source()`.
 - Numeric conversions must not silently truncate. Use fallible conversions for untrusted values; validate finiteness, sign, and range before float-to-integer conversion. Clamp/saturate only when required by the domain.
 - Do not add crate-root `#![allow(dead_code)]`.
 
 ## Naming
 
-Use Rust API naming: `SCREAMING_SNAKE_CASE` constants/statics, `snake_case`
-functions/variables, and `PascalCase` types. Do not rename unrelated existing
-violations.
+Use Rust API naming: `SCREAMING_SNAKE_CASE` constants/statics, `snake_case` functions/variables/modules, and `PascalCase` types/traits. Name domain concepts and include units where a value would otherwise be ambiguous; do not optimize for short names. Do not rename public, serialized, or persisted fields merely to match a local style preference, or rename unrelated existing violations.
+
+## API and Operational Contracts
+
+- For new or changed public APIs, document the caller-visible behavior and relevant `# Errors`, `# Panics`, and `# Safety` conditions. Include a useful example or link to one when usage is non-obvious; avoid boilerplate sections that say nothing. Follow the [Rust API documentation guidelines](https://rust-lang.github.io/api-guidelines/documentation.html) within the scope of the change.
+- For async or storage APIs, state cancellation, retry/idempotency, partial-completion, ownership, and durability guarantees when callers rely on them. Make failure cleanup and commit points explicit in code and tests; do not imply atomicity that the implementation does not provide.
+- For protocol, persisted-format, configuration, dependency, or operational changes, check affected consumers and supported versions. Update relevant docs/examples and describe compatibility, upgrade/rollback limits, and new configuration defaults when they change. New dependencies must solve a current need and fit the workspace's license, toolchain, and feature policy.
+- For hot-path or resource-management changes, check allocation/copying, blocking, unbounded work, and lock duration where affected. Support performance claims with an appropriate measurement or clearly label them as expectations; a benchmark is not required for unrelated changes.
+
+## Behavioral Tests
+
+- For behavior changes, add or update the narrowest test that calls the real implementation and asserts an observable result. Existing tests can provide coverage when they directly exercise the changed behavior; do not duplicate production logic as the test oracle.
+- Cover the changed contract's relevant failure and boundary cases, not only success. A bug regression should fail without the fix; do not broaden assertions or rely only on a successful call when the result or side effect is what matters.
+- Follow `docs/testing/README.md` for test layers, runners, isolation, synchronization, and flaky tests. Use controlled clocks or explicit synchronization for time/concurrency tests instead of fixed sleeps, and keep waits bounded.
+- Report the checks actually run and their outcomes, including skipped or blocked checks and the behavior they leave unverified. Do not describe focused local checks as full CI coverage.

.agents/references/pull-requests.md

@@ -1,51 +1,31 @@
 # Git and Pull Request Rules
 
-Applies to commits, pushes, PRs, and issue/discussion actions. Paths below are
-repository-relative. User authorization and root `AGENTS.md` still govern scope.
+Applies to commits, pushes, PRs, and issue/discussion actions. Paths below are repository-relative. User authorization and root `AGENTS.md` still govern scope.
 
 ## Final PR Preflight
 
 Before creating or updating a PR, reuse completed review and verification:
 
-- Verify the actual base (normally `origin/main`) and the complete task diff,
-  including file names and whitespace. Exclude secrets, logs, generated
-  artifacts, and unrelated edits. Retain an existing PR's base unless requested.
-- Confirm the final diff passed the root verification tier; fix task-owned
-  failures and run missing scoped checks. Report unresolved required checks or
-  authority without expanding the task. Do not start another general review.
-- Keep the English Conventional Commit title at most 72 characters. Use the
-  template headings, actual checks, material risks, and rollback notes.
-- Immediately before writing to GitHub, confirm the head and task diff are
-  unchanged. Rerun only checks invalidated by edits or relevant state changes.
+- Verify the actual base (normally `origin/main`) and the complete task diff, including file names and whitespace. Exclude secrets, logs, generated artifacts, and unrelated edits. Retain an existing PR's base unless requested.
+- Confirm the final diff passed the root verification tier; fix task-owned failures and run missing scoped checks. Report unresolved required checks or authority without expanding the task. Do not start another general review.
+- Keep the English Conventional Commit title at most 72 characters. Use the template headings, actual checks, material risks, and rollback notes.
+- Immediately before writing to GitHub, confirm the head and task diff are unchanged. Rerun only checks invalidated by edits or relevant state changes.
 
 ## Pull Request Lifecycle
 
-- Creating or updating a PR includes one immediate snapshot of checks,
-  mergeability, reviews, and unresolved threads.
-- Unless the user explicitly requests monitoring, a release workflow requires
-  it, or an automation already owns it, hand off after the PR is open with the
-  current state and next event to watch. Do not delay ordinary handoff with
-  fixed quiet-period sleeps.
-- For requested monitoring, use event-driven or bounded waits. Report only state
-  changes, actionable failures, or a meaningful prolonged delay.
-- Investigate failures/comments before changing code. Fix task-attributable
-  issues, rerun affected verification, push, reply or resolve the thread, then
-  resume the requested monitor.
-- Never merge without required reviewer approval or explicit authority.
-- After an observed merge, verify the commit reached the base, then clean the
-  task worktree/branch when safe. Preserve unmerged work for closed PRs unless
-  deletion was explicitly authorized.
+- Creating or updating a PR includes one immediate snapshot of checks, mergeability, reviews, and unresolved threads.
+- Unless the user explicitly requests monitoring, a release workflow requires it, or an automation already owns it, hand off after the PR is open with the current state and next event to watch. Do not delay ordinary handoff with fixed quiet-period sleeps.
+- For requested monitoring, use event-driven or bounded waits. Report only state changes, actionable failures, or a meaningful prolonged delay.
+- Investigate failures/comments before changing code. Monitoring alone authorizes observation and diagnosis; fixes, pushes, replies, and thread resolution require their own existing task authority. When authorized, fix task-attributable issues, rerun affected verification, and resume the requested monitor. Otherwise report the concrete failure and next action without expanding scope.
+- Merge only with existing user authorization and satisfied applicable reviewer, check, and branch-protection requirements. Reviewer approval does not grant merge authority; user authorization does not bypass repository protections.
+- After an observed merge, verify the commit reached the base, then clean the task worktree/branch when safe. Preserve unmerged work for closed PRs unless deletion was explicitly authorized.
 
 ## Git and PR Baseline
 
 - Follow Conventional Commits; keep the subject at most 72 characters.
 - Source comments, commits, PR titles, and PR bodies are in English.
-- Keep every heading from `.github/pull_request_template.md`; use `N/A` where
-  needed and include commands actually run.
+- Keep every heading from `.github/pull_request_template.md`; use `N/A` where needed and include commands actually run.
 - Use `--body-file` for multiline `gh pr create`/`gh pr edit` content.
-- PR/issue/discussion content must not contain the literal sequence `\n` or
-  hard-wrapped prose paragraphs.
-- Do not include local absolute paths or tool-specific labels/prefixes in GitHub
-  content.
-- Resolve review threads after the underlying issue is fixed. If declining a
-  suggestion, reply with a short evidence-based reason.
+- PR/issue/discussion content must not contain the literal sequence `\n` or hard-wrapped prose paragraphs.
+- Do not include local absolute paths or tool-specific labels/prefixes in GitHub content.
+- Resolve review threads after the underlying issue is fixed. If declining a suggestion, reply with a short evidence-based reason.

.agents/skills/adversarial-validation/SKILL.md

@@ -1,48 +0,0 @@
----
-name: adversarial-validation
-description: Review RustFS diffs or designs for explicit adversarial requests, high-risk changes under the repository review policy, or substantial PR reviews. Skip ordinary questions, diagnosis, planning, status, routine low-risk implementation, and prose with no execution effect.
----
-
-# RustFS Adversarial Validation
-
-Use the [repository risk tiers and review shape](../../references/adversarial-validation.md). This skill
-routes a review to RustFS-specific probes without loading unrelated domains.
-
-## Select Lenses
-
-Read only the references required by the diff:
-
-| Lens | When to read |
-|---|---|
-| [Correctness](references/correctness.md) | Every non-exempt adversarial review |
-| [Simplicity](references/simplicity.md) | Mechanical/standard changes and production growth |
-| [Test coverage](references/test-coverage.md) | Behavior or test changes |
-| [Security](references/security.md) | Authn/authz, IAM, RPC trust, paths, secrets, parsing, browser, encryption |
-| [Concurrency/durability](references/concurrency-durability.md) | Async shared state, locks, storage commit, cancellation, persisted queues |
-| [Compatibility](references/compatibility.md) | S3 surface, MinIO interop, metadata, wire/disk formats, mixed versions |
-| [Performance](references/performance.md) | Request/object hot paths, allocation, blocking work, fsync, fan-out |
-
-Do not read all references as a precaution. A path name alone is insufficient;
-the changed behavior must touch the lens's domain.
-
-For a dedicated security audit or advisory analysis, use
-`security-advisory-lessons` instead of loading it automatically during every
-adversarial review.
-
-## Review Protocol
-
-1. Freeze the exact final diff/head (or the design under review) and list the
-   selected lenses.
-2. Run the review shape required by the repository risk tier.
-3. For each selected lens, either report a concrete finding or a null verdict
-   naming the attacks performed.
-4. Apply root `AGENTS.md`'s finding standard. Test each candidate against callers,
-   existing coverage, and invariants before accepting it; an adversarial role
-   does not have to produce a defect.
-5. Fix or rebut supported findings with code-path, test, or invariant evidence.
-6. After a non-trivial edit, rerun only lenses affected by that edit against the
-   new exact diff.
-
-Do not turn a null verdict into a long checklist. Record concise evidence that
-the relevant failure classes were attacked, then stop under the root completion
-rule. Keep the required per-lens verdicts for high-risk PRs.

.agents/skills/adversarial-validation/references/compatibility.md

@@ -1,38 +0,0 @@
-# Compatibility Lens
-
-- Internal metadata uses `metadata_compat` helpers for dual RustFS/MinIO keys,
-  including mixed casing and removal of both twins.
-- Binary UUID metadata treats absent, empty, and nil as no value. Unversioned
-  remote tiers receive no `versionId`; versioned purge requests retain the real
-  version ID.
-- `xl.meta` changes preserve supported header/meta versions, recompute
-  signatures, decode legacy fixtures, and remain readable by old RustFS/MinIO.
-- Foreign/corrupt metadata validates parallel array lengths and missing fields;
-  it returns a decode error rather than indexing, panicking, or fabricating data.
-- Do not “correct” byte-for-byte MinIO ports without legacy fixture evidence.
-  Bitrot framing, shard math, distribution, and inline prefixes are contracts.
-- Client-visible metadata/events strip both internal prefixes
-  case-insensitively.
-- Proto fields are appended, never reused/renumbered; FlatBuffers tables extend
-  compatibly and absent new fields fail closed where authorization/quorum is
-  involved.
-- Replay real client request shapes and exact pagination boundaries for S3
-  handler changes.
-- Bucket metadata/IAM/config parsing remains compatible with pinned real MinIO
-  fixtures and encrypted migration data.
-- Compatibility shims use `RUSTFS_COMPAT_TODO(<task-id>)`, have a removal
-  condition, and default toward reading old data safely.
-
-## Outbound targets
-
-- A change to what the replication or migration client sends by default
-  (checksum policy, payload framing, headers, version-id addressing) is judged
-  against every target class, not the one it fixes. Name each target-side rule
-  the current default satisfies — checksum required with Object Lock
-  parameters, `aws-chunked` decoding, version-id adoption, ETag equals content
-  MD5 — and show which cell of
-  `crates/e2e_test/src/replication_target_matrix_test.rs` covers each.
-- A test that asserts the fix ("no trailer header") is not evidence; the
-  matrix cell that asserts the target accepted and stored the object is.
-- Every new environment escape hatch appears in
-  `docs/operations/replication-outbound-transport.md` in the same diff.

.agents/skills/adversarial-validation/references/concurrency-durability.md

@@ -1,27 +0,0 @@
-# Concurrency and Durability Lens
-
-- For every changed lock, enumerate overlapping lock sets and construct the
-  ABBA interleaving. Multiple-lock order must be documented and consistent.
-- Mark guard lifetimes and every `.await`, disk,
-  and RPC call; estimate contention/timeouts. Tokio read/write guards across
-  `.await` need bounded hold time; long-lived reads can wedge writers (#4195).
-  Keep `std::sync::Mutex` holds brief and never across `.await`.
-- Use direct `fetch_*` for unconditional atomic read-modify-write and
-  `compare_exchange` for conditional updates.
-- Object commits remain fenced if the distributed lock is lost after shard
-  writes and before metadata rename.
-- For write/rename changes, trace `write tmp -> sync tmp -> rename -> sync parent
-  -> sync required ancestors`; simulate a crash after each step and honor the
-  configured durability gate.
-- Multi-disk fan-out counts every result. Quorum-minus-one cannot become success;
-  heal remains best-effort per target where that is the established contract.
-- At every new cancellable await between mutation and cleanup/commit, drop the
-  future and inspect leftover files, counters, permits, and replay state.
-- Multipart operations on the same upload ID are serialized where required;
-  abort/complete/list races cannot delete parts before durable commit.
-- Post-commit cleanup is best-effort, retry-safe, and cannot fail an already
-  committed write or delete the last surviving copy.
-- Persisted read-modify-write uses serialization/CAS. Queue replay is crash-safe
-  and duplicate delivery has an idempotency contract.
-- Streaming reconstruction failures after partial output surface as errors, not
-  successful EOF.

.agents/skills/adversarial-validation/references/correctness.md

@@ -1,31 +0,0 @@
-# Correctness Lens
-
-Attack the changed behavior, not every subsystem in the repository.
-
-- Trace new error paths to the caller. Inject the ignored/wildcard variants and
-  verify they cannot become success, not-found, or a plausible default.
-- Exercise zero/empty/missing, maximum, and exact-boundary inputs for every
-  changed count, size, index, page limit, or optional value.
-- For aggregation/quorum changes, test exactly quorum and quorum-minus-one with
-  mixed disk errors and nil/placeholder entries.
-- For listing/pagination, test `n == max`, `n == max + 1`, delimiter folding,
-  continuation markers, and object/prefix name collisions.
-- For EC/read/streaming changes, inject failure after partial output and verify
-  the client receives an error rather than a clean truncated body. Assert exact
-  bytes and length.
-- For multipart/object commits, fail before/after rename and cleanup; committed
-  data must remain readable and pre-commit cleanup must not destroy parts.
-- Bound recursion over untrusted or persisted input; handle corrupted/cyclic
-  tree and cache traversals safely.
-- For version/index ordering, test `len - 1`, `len`, equal timestamps, missing
-  versions, and deterministic tie-breaking.
-- For directory-object behavior, trace `__XLDIR__` at the store layer; branches
-  below the layer that sees trailing slashes are dead.
-- For binary UUID metadata, absent, empty, and nil all mean no value. Never send
-  nil/empty `versionId` to an unversioned tier.
-- For agent rules/skill routers, test a trigger matrix covering ordinary
-  inquiry, low-risk implementation, explicit review, high-risk code, PR
-  creation, release, and post-PR monitoring. Each case must select only the
-  intended workflow and retain required safety/authorization boundaries.
-
-Null verdicts name only the probes relevant to the diff.

.agents/skills/adversarial-validation/references/performance.md

@@ -1,20 +0,0 @@
-# Performance Lens
-
-- For added clones/allocations on request/object/block paths, quantify copied
-  data and frequency. Recommend borrowing, move, `Bytes`/`Arc`, `Cow`, or
-  capacity reservation only for a concrete repeated cost.
-- Route every new sync/flush through the durability-mode and bucket override
-  gates; mode `none` must not pay the new fsync.
-- Keep blocking filesystem/CPU work off async runtime threads, but do not split
-  one small operation into many `spawn_blocking` round trips.
-- Measure lock hold time across I/O and compare acquisition order for ABBA.
-- Keep cleanup, extra stat/rename, and diagnostics out of the PUT commit critical
-  section when they need not be there.
-- Detect per-item serial I/O/RPC in batch APIs and accidental quadratic scans;
-  use a gate or bounded concurrency when the concrete fan-out warrants it.
-- Count buffer growth and byte copies in EC/bitrot paths; preserve pool gauge
-  balance and avoid repeated metadata decode/fetch per object.
-- Repetitive success logs stay at `trace`; metrics/instrumentation on hot paths
-  require an existing gate.
-- Claims of no impact on PUT/GET/commit/erasure paths need relevant benchmark or
-  A/B evidence, especially for 4 KiB objects.

.agents/skills/adversarial-validation/references/security.md

@@ -1,31 +0,0 @@
-# Security Lens
-
-Use `security-advisory-lessons` only for a dedicated advisory/security audit.
-For an ordinary matched diff, attack these boundaries:
-
-- Admin routes: route registration, whitelist, handler authn, and the exact
-  `AdminAction` must agree. Read-only diagnostics still require admin authz.
-- IAM/service accounts: treat parent, claims, keys, groups, status, and policy
-  names as attacker-controlled; prove ownership/root authority before writes.
-- Protocol frontends: every changed/sibling command authorizes the matching S3
-  action before reaching storage.
-- Secrets/signatures: use constant-time comparison, normalize public failures,
-  keep RPC/root/STS keys independent, and fail closed when secrets are absent.
-- RPC: bind signatures to the exact method/path and timestamp; reject replay,
-  stale, malformed, truncated, and invalid-enum payloads without panic.
-- Paths/object/archive entries: reject traversal, absolute/platform escapes,
-  and normalization differences between authz and storage.
-- Copy/multipart/presigned POST: enforce source, destination, version-aware
-  actions, copy-source conditions, and every signed policy condition.
-- Logging/errors: never expose credentials, tokens, expected signatures, raw
-  secret-bearing input, or merged configs—including via `Debug` and parse errors.
-- Untrusted serde: reject unknown fields where compatible and validate
-  security-critical defaults/ranges before numeric conversion.
-- SSE/browser/CORS/trusted proxy: inspect stored ciphertext and wrapper order;
-  isolate user content; never reflect credentialed arbitrary origins or trust
-  forwarded identity from direct clients.
-- Object Lock: unreadable/fabricated/unparsable metadata fails closed across
-  foreground, lifecycle, scanner, and force-delete paths.
-
-Security findings distinguish unauthenticated compromise from a
-low-privileged authenticated bypass.

.agents/skills/adversarial-validation/references/simplicity.md

@@ -1,22 +0,0 @@
-# Simplicity Lens
-
-- Compare the production diff with the smallest equivalent local edit. Fewer
-  lines alone are not evidence; the replacement must preserve correctness,
-  compatibility, readability, and real boundaries.
-- Search the touched crate, domain owner, `crates/utils`, `crates/common`, and
-  relevant dependencies for each new helper, constant, wrapper, or fixture.
-- Reject forced reuse when normalization, error, deadline, or durability
-  semantics differ.
-- Require a concrete trigger for every new defensive branch. Keep boundary
-  checks for disk/RPC/version data and checks immediately before destructive
-  actions.
-- Flag one-caller helpers only when they merely forward or split a short linear
-  flow without adding domain naming, invariant isolation, or useful context.
-- Ensure a replacement removes the superseded in-scope path or keeps one
-  canonical core behind a documented compatibility adapter.
-- Remove narration/change-history comments; preserve concise safety, lock,
-  durability, and compatibility invariants.
-- Treat tests, fixtures, generated code, and documentation separately from
-  production growth. Do not optimize away meaningful regression coverage.
-
-A finding must include a concrete smaller design, not a style preference.

.agents/skills/adversarial-validation/references/test-coverage.md

@@ -1,26 +0,0 @@
-# Test-Coverage Lens
-
-- For every behavior claim, name the focused test/check that fails if the
-  changed hunk is reverted. If none is practical, require the reason and
-  residual risk.
-- Confirm tests exercise the real production path and distinguish the intended
-  behavior from the named regression. A success, `is_err()`, or no-panic check
-  can be sufficient when that is the actual contract; require exact values,
-  bytes, state, or error variants when those distinctions matter to the change.
-- For new flags/modes, verify each branch and ask which test fails if the branch
-  is inverted.
-- For new error propagation, inject the failure and assert the caller observes
-  it; mentally replacing `?`/`return Err` with success must break a test.
-- Streaming GET tests assert the complete body and length under degraded reads.
-- Disk/wire-format tests use pinned foreign/legacy fixtures; same-code
-  round-trips are insufficient for compatibility.
-- Concurrency tests use readiness polling, isolate global state, and avoid fixed
-  sleeps or unrealistically short timeouts. Use nextest groups when process-level
-  serialization is required.
-- Internal metadata tests assert both RustFS and MinIO keys, not only read-back
-  through a helper that prefers one key.
-- Avoid duplicate tests of the same production path and poison-value class.
-  Boundary companions are distinct coverage: `n == max` vs `max + 1`, and
-  absent vs empty vs nil UUID.
-- A focused test proves only the targets/features it builds. Add compilation or
-  Clippy only for uncovered changed targets.

.agents/skills/plugin-contract-guard/SKILL.md

@@ -16,68 +16,28 @@ The "plugin system" spans four surfaces that must stay consistent:
 
 ## Hard invariants (verify before merging)
 
-1. **Secrets have one source of truth.** Secret config keys are declared only
-   in the plugin manifest (`TargetPluginManifest.secret_fields`,
-   `crates/targets/src/manifest.rs`) and flow to admin via
-   `AdminTargetSpec.secret_fields`. Never add a hand-maintained per-service
-   secret table in a handler; if redaction misses a field, fix the manifest.
+1. **Secrets have one source of truth.** Secret config keys are declared only in the plugin manifest (`TargetPluginManifest.secret_fields`, `crates/targets/src/manifest.rs`) and flow to admin via `AdminTargetSpec.secret_fields`. Never add a hand-maintained per-service secret table in a handler; if redaction misses a field, fix the manifest.
 
-2. **Redaction must round-trip.** Instance GET responses replace secret values
-   with `***redacted***` (`REDACTED_SECRET_VALUE` in `plugins_instances.rs`).
-   Instance PUT restores the stored secret when it receives that placeholder
-   back (`restore_redacted_secret_values`). Any new read or write path for
-   target config must keep both halves: redact on the way out, restore the
-   placeholder on the way in. The placeholder literal must never be persisted.
+2. **Redaction must round-trip.** Instance GET responses replace secret values with `***redacted***` (`REDACTED_SECRET_VALUE` in `plugins_instances.rs`). Instance PUT restores the stored secret when it receives that placeholder back (`restore_redacted_secret_values`). Any new read or write path for target config must keep both halves: redact on the way out, restore the placeholder on the way in. The placeholder literal must never be persisted.
 
-3. **Fixtures never reach production responses.**
-   `example_external_webhook_plugin()` (`crates/targets/src/catalog/mod.rs`)
-   is a test/demo fixture for control-plane planning tests. Production
-   catalog/extension handlers must not include it; regression tests
-   (`plugin_catalog_never_exposes_example_or_external_fixtures`,
-   `extension_catalog_never_exposes_example_or_external_fixtures`) enforce it.
+3. **Fixtures never reach production responses.** `example_external_webhook_plugin()` (`crates/targets/src/catalog/mod.rs`) is a test/demo fixture for control-plane planning tests. Production catalog/extension handlers must not include it; regression tests (`plugin_catalog_never_exposes_example_or_external_fixtures`, `extension_catalog_never_exposes_example_or_external_fixtures`) enforce it.
 
-4. **External plugin flow is planning-only and deny-by-default.**
-   `plan_external_target_plugin_action` returns decisions, it executes
-   nothing. `TargetPluginExternalFlowGate::default()` is fully closed and
-   `TargetPluginInstallPolicy::default().allowed_download_hosts` is empty —
-   keep it that way; tests opt in via explicit policies. Install validation
-   requires https, an allowlisted host, a full 64-hex-char sha256 digest,
-   signature and provenance URIs, and an artifact matching the host
-   `target_triple`.
+4. **External plugin flow is planning-only and deny-by-default.** `plan_external_target_plugin_action` returns decisions, it executes nothing. `TargetPluginExternalFlowGate::default()` is fully closed and `TargetPluginInstallPolicy::default().allowed_download_hosts` is empty — keep it that way; tests opt in via explicit policies. Install validation requires https, an allowlisted host, a full 64-hex-char sha256 digest, signature and provenance URIs, and an artifact matching the host `target_triple`.
 
-5. **Custom target types must not collide.** Unknown target types get an
-   interned unique `custom:<type>` plugin id (`custom_plugin_id` in
-   `manifest.rs`). Custom plugins with secrets must register via
-   `TargetPluginDescriptor::with_manifest` and declare `secret_fields`;
-   `::new` derives a manifest with no secrets.
+5. **Custom target types must not collide.** Unknown target types get an interned unique `custom:<type>` plugin id (`custom_plugin_id` in `manifest.rs`). Custom plugins with secrets must register via `TargetPluginDescriptor::with_manifest` and declare `secret_fields`; `::new` derives a manifest with no secrets.
 
 ## Changing the admin JSON contract
 
-- Shapes are locked twice in `plugin_contract.rs` tests: insta snapshots
-  (`rustfs/src/admin/snapshots/`) plus literal `json!` assertions. Update
-  both deliberately; a shape change is a console-facing API change.
-- Field naming is `snake_case`, except discovery blocks
-  (`runtimeCapabilities`, `clusterSnapshot`, `extensionsCatalog`) which are
-  camelCase **by cross-endpoint convention** (same shape in `system.rs`,
-  `console.rs`, `pools.rs`). Do not "fix" that inconsistency locally.
-- Contract types deliberately duplicate `rustfs_targets` types
-  (anti-corruption layer). Add a `From` impl; do not serialize internal
-  types directly.
+- Shapes are locked twice in `plugin_contract.rs` tests: insta snapshots (`rustfs/src/admin/snapshots/`) plus literal `json!` assertions. Update both deliberately; a shape change is a console-facing API change.
+- Field naming is `snake_case`, except discovery blocks (`runtimeCapabilities`, `clusterSnapshot`, `extensionsCatalog`) which are camelCase **by cross-endpoint convention** (same shape in `system.rs`, `console.rs`, `pools.rs`). Do not "fix" that inconsistency locally.
+- Contract types deliberately duplicate `rustfs_targets` types (anti-corruption layer). Add a `From` impl; do not serialize internal types directly.
 
 ## Handler conventions
 
-- Every new admin plugin/extension route needs authorization at the top of
-  `call` and an `include_str!` guard test asserting it (repo-wide pattern —
-  see `plugin_instance_handlers_require_admin_authorization_contract`).
-- Reads use `GetBucketTargetAction` (instances) or `ServerInfoAdminAction`
-  (catalogs); writes use `SetBucketTargetAction`.
-- Refresh persisted module switches once per request
-  (`refresh_persisted_module_switches`), then evaluate the sync
-  `module_disabled_block_reason` per domain — do not re-read the store per
-  domain or per instance.
+- Every new admin plugin/extension route needs authorization at the top of `call` and an `include_str!` guard test asserting it (repo-wide pattern — see `plugin_instance_handlers_require_admin_authorization_contract`). The source guard checks wiring, not runtime authorization; use focused behavior evidence for changed allow/deny semantics.
+- Reads use `GetBucketTargetAction` (instances) or `ServerInfoAdminAction` (catalogs); writes use `SetBucketTargetAction`.
+- Refresh persisted module switches once per request (`refresh_persisted_module_switches`), then evaluate the sync `module_disabled_block_reason` per domain — do not re-read the store per domain or per instance.
 
 ## Generic bounds
 
-Event-payload generics use the `PluginEvent` blanket trait
-(`crates/targets/src/plugin.rs`). Do not respell
-`Send + Sync + 'static + Clone + Serialize + DeserializeOwned`.
+Event-payload generics use the `PluginEvent` blanket trait (`crates/targets/src/plugin.rs`). Do not respell `Send + Sync + 'static + Clone + Serialize + DeserializeOwned`.

.agents/skills/pr-review/SKILL.md

@@ -13,8 +13,8 @@ description: Review a GitHub PR using its exact base/head and repository risk po
    ```
 
    Record both SHAs; refresh if either moved during fetching. Follow relevant [implementation boundaries](../../references/implementation.md).
-2. Apply the [risk policy](../../references/adversarial-validation.md). Ordinary reviews use the root finding standard directly. For explicit adversarial, substantial, or high-risk reviews, use [the domain probes](../adversarial-validation/SKILL.md). File count does not authorize delegation; use exactly two independent reviewers only when authorized, otherwise two fresh sequential passes for that risk tier. Reviewers do not spawn further agents.
+2. Apply the [risk policy](../../references/adversarial-validation.md), including its two perspectives for high-risk work. Ordinary reviews use the root finding standard directly. File count does not determine risk or authorize delegation; keep any delegated review within the authorized scope.
 3. Check `gh pr checks <N> --repo <owner/repo>`. Investigate failures when relevant to a finding or a requested CI/readiness diagnosis. Distinguish pending, unavailable, pre-existing, flaky, and PR-caused states using evidence; a code review does not authorize unrelated repairs.
-4. Report the reviewed SHAs, risk tier, supported findings or `No findings`, material verification limits, check state, and verdict. P0/P1 findings block approval. Severity: P0 = demonstrated data loss, security breach, remote crash, or deadlock; P1 = correctness, compatibility, or material hot-path regression; P2 = a concrete maintainability defect; P3 = optional style, only when requested.
-5. When posting is authorized, follow [Git and PR rules](../../references/pull-requests.md), recheck the remote head, and review any delta before submitting. Use `gh pr review` with `--approve`, `--request-changes`, or `--comment` and `--body-file`; use [the API example](references/posting.md) for authorized inline comments. Reuse existing authorization.
+4. Report the reviewed SHAs, risk tier, supported findings or `No findings`, material verification limits, check state, and verdict. Set severity from evidenced impact, triggering conditions, reach, and urgency: P0 is a critical issue requiring immediate action with broad or unavoidable impact; P1 is a high-impact defect requiring prompt correction; P2 is an ordinary concrete correctness, compatibility, performance, or maintainability defect; P3 is an optional improvement only when requested. A remote trigger, crash, or deadlock does not by itself establish P0. P0/P1 findings block approval; evaluate other findings and verification gaps against the actual acceptance requirements rather than approving automatically below that threshold.
+5. When posting is authorized, follow [Git and PR rules](../../references/pull-requests.md), recheck the remote base and head, and review any changed diff or integration assumptions before submitting. Reuse unaffected evidence. Use `gh pr review` with `--approve`, `--request-changes`, or `--comment` and `--body-file`; use [the API example](references/posting.md) for authorized inline comments. Reuse existing authorization.
 6. Follow the repository PR lifecycle and explicit monitoring scope. Fetch new heads before comparing to the reviewed SHA; never rely on an unfetched `origin/pull/<N>/head`. Push fixes or resolve threads only when authorized; check `maintainerCanModify` before a fork push.

.agents/skills/pr-review/references/posting.md

@@ -4,19 +4,20 @@ Read only when an inline review is authorized. Recheck the PR head before postin
 
 For inline comments on specific lines, use the GitHub API:
 ```bash
-cat > /tmp/pr_review.json <<'EOF'
+REVIEW_PAYLOAD=$(mktemp "${TMPDIR:-/tmp}/pr-review.XXXXXX")
+cat > "$REVIEW_PAYLOAD" <<'EOF'
 {
   "commit_id": "<reviewed-head-sha>",
   "body": "review body",
   "event": "REQUEST_CHANGES",
   "comments": [
     {
-      "path": "crates/foo/src/bar.rs",
+      "path": "crates/<crate>/src/<file>.rs",
       "line": 42,
       "body": "finding description"
     }
   ]
 }
 EOF
-gh api --method POST /repos/{owner}/{repo}/pulls/<N>/reviews --input /tmp/pr_review.json
+gh api --method POST /repos/{owner}/{repo}/pulls/<N>/reviews --input "$REVIEW_PAYLOAD" && rm -f "$REVIEW_PAYLOAD"
 ```

.agents/skills/rustfs-release-publish/SKILL.md

@@ -1,42 +1,38 @@
 ---
 name: rustfs-release-publish
-description: Publish a RustFS version through Console, release-branch CI, preview acceptance, human confirmation, and final publication. Use only for an explicit release request.
+description: Publish a RustFS version through frozen Linux candidate acceptance, desktop completion, manifest approval, byte-preserving publication and post-release follow-up. Use only for an explicit release request.
 ---
 # RustFS Release
 
-Require the exact final SemVer target before version edits or publication. If unspecified, inspect current releases/tags and ask with concrete candidates; do not guess the channel or bump. A named bump type resolves to an exact version that still needs confirmation. Report the confirmed target verbatim; resolve any later mismatch before proceeding. Independent read-only preflight may continue.
+Require the exact final SemVer target before version edits or publication. If unspecified, inspect releases/tags and ask with concrete candidates; do not guess the channel or bump. Reuse the user's confirmed target and authorized delivery scope. Independent read-only preflight may continue.
 
 ## Invariants
 
-- Cargo stores `<target>`, never `-preview.N`. Before publication, installation defaults retain the previous available deliverable. Final and preview binaries embed their own tag names, so version/asset names differ despite identical source.
-- Use annotated tags without `v`. Preview names are exactly `<target>-preview.<digits>`; choose the next unused iteration after fetching tags. Preview Releases are prereleases, never Latest; publish only versioned assets/exact Docker tags, never latest assets, R2, Helm, moving Docker channels, or installation/banner updates.
-- `release/<target>` owns the candidate. Retain its history, backport only reviewed fixes, and never merge main wholesale or force-push it. Pin `PREVIEW_HASH` to its exact commit with complete passing CI. Preview and final tags both point there; never tag a later main/branch HEAD or insert another bump.
-- Phase failures block dependents. Keep preview Releases until final publication and notes generation finish; CI removes Releases while retaining tags. Never move an existing preview or final tag.
+- Cargo and every candidate executable contain the final product version. `<target>-preview.<run_number>` is an immutable candidate identifier, not a public tag or displayed binary version.
+- Freeze the exact source, reviewed workflow recipe, toolchain, Console, CLI, dependency lock, previous stable version and validation plan. Newer unrelated main or Console releases do not change those inputs.
+- Retain `release/<target>` history when a release branch is used; backport only reviewed fixes. Pin `CANDIDATE_SHA` to its exact CI-validated commit. Never force-push a release branch or move a public tag.
+- Build Linux first. Complete field acceptance before macOS/Windows and packaging. Full certification and final acceptance are required before publication; a partial Linux PASS grants no release authority.
+- Promote the accepted bytes. Final publication never recompiles binaries or rebuilds packages/images. Infrastructure retries reuse successful artifacts; changed source or inputs require a new candidate and affected acceptance.
+- Phase failures block dependents. No latest/default-channel update before versioned delivery. Use the [pipeline runbook](../../../docs/operations/release-pipeline.md) for executable entry points, locks and recovery.
 
-## Read by Phase
-
-Read only the current phase's procedure; retain completed evidence across turns. Follow root verification, Git/PR, and authorization rules.
+## Read by phase
 
 | Phase | Procedure |
 |---|---|
-| 0 | Clean tree, fetch main/tags, verify GitHub access, then complete the mandatory [Console release gate](references/console-gate.md). A green Console build alone is insufficient. |
-| 1–2 | [Prepare/reuse the release candidate and run exact-SHA branch CI](references/release-candidate.md). Source version changes use `rustfs-release-version-bump` with authorized delivery scope. |
-| 3 | [Verify the preview build, assets, channels, and notes](references/preview-release.md). |
-| 4–5 | [Run the downloaded binary, Console CRUD, and latest-rc matrix](references/preview-acceptance.md). Every check must pass. |
-| Confirmation | Complete the gate below and end the turn. |
-| 6 | After fresh confirmation, [publish and verify the final tag](references/final-release.md) at `PREVIEW_HASH`. |
-| 7 | [Reconcile Security Advisories, milestones, installation references, and the website](references/post-release-updates.md) only after complete publication. |
+| 0 | Clean tree, fetch main/tags, verify access, then complete the mandatory [Console gate](references/console-gate.md). |
+| 1–2 | [Prepare the exact source and run branch CI](references/release-candidate.md). Version edits use `rustfs-release-version-bump`. |
+| 3 | [Build and verify the Linux candidate](references/preview-release.md). |
+| 4–5 | [Run artifact, field, Console and rc acceptance](references/preview-acceptance.md), then complete desktop/package/image/chart validation. |
+| Confirmation | Report the complete manifest and obtain the confirmation below. |
+| 6 | [Publish the accepted manifest](references/final-release.md). |
+| 7 | [Reconcile advisories, milestones, installation references, website and release branch](references/post-release-updates.md). |
 
-## Human Confirmation
+## Human confirmation
 
-After Phases 3–5 pass, recheck that the release branch still equals `PREVIEW_HASH`. Report the target, branch, preview tag/hash, branch CI run, preview Release URL, Console result, and rc matrix. Explicitly ask whether to publish the final tag and end the turn without creating/pushing it.
+After all acceptance passes, report the target, source SHA, source CI run, Linux and completion run URLs, final manifest SHA-256, accepted Console/CLI identities, compatibility/performance results and backlog report. Explicitly ask whether to publish this exact manifest and end the turn without dispatching publication.
 
-Only a new affirmative user reply to that exact report authorizes Phase 6. The original release request, earlier confirmation, silence, or automated follow-up does not count. Confirmation binds the target, preview tag, and hash; a changed value or repeated/failed acceptance cycle invalidates it. Revalidate and obtain fresh confirmation before continuing.
+Only a new affirmative reply to that exact report authorizes Phase 6. An earlier general release request, silence or automatic continuation does not count. Reuse confirmation for retries of that same manifest; do not ask again merely because a destination failed. A changed source, input, manifest or acceptance result invalidates it and requires renewed acceptance/confirmation.
 
-## Recovery
+## Recovery and reporting
 
-- Before the final tag exists, land fixes on main, backport to the candidate, verify target versions, and restart branch CI/acceptance. Use a new preview iteration once a preview was tagged; main may advance independently.
-- Once the final tag exists, retry publication jobs for that exact tag. A source fix requires a newly confirmed target, not retagging or a newer preview. Phase 7 retries recheck artifact availability and superseding releases.
-- If abandoned after the bump merged, report the unpublished version on main and record whether to revert or let the next release overwrite it. Deleting candidate branches or preview tags requires an explicit decision.
-
-Report completed phase results and incomplete work with evidence: Console previous/latest tags, release need, hash and run/Release URLs; target, branch CI/SHA, previews and hash, confirmation state; preview/final assets, latest-channel state, Console/rc results and cleanup; advisory reconciliation results and pending disclosure decisions; both milestone states, installation PR checks/merge state, and website text/link/PR/deployment state. Keep artifact publication separate from follow-up delivery; do not call an open PR a live update. Report deviations and their authorization.
+Resume failed infrastructure jobs without rebuilding successful units. Fix product defects on main, backport when needed, and start a new candidate. After a final tag exists, retry the same bytes or use a new patch version; never move the tag. Do not conflate implemented, CI-passed, field-accepted, approved, published and post-release-complete states. Report incomplete work and deviations with evidence; a tag or green build alone is not a completed release.

.agents/skills/rustfs-release-publish/references/console-gate.md

@@ -4,7 +4,7 @@ Read during Phase 0, before changing RustFS version files or tags. Follow the pa
 
 ### Console release gate
 
-Complete this gate before changing any RustFS version file or creating any RustFS tag. RustFS `build.yml` downloads the asset returned by `repos/rustfs/console/releases/latest`, so a successful Console build alone is insufficient.
+Complete this gate before changing any RustFS version file or creating any RustFS tag. Candidate creation resolves `repos/rustfs/console/releases/latest` once and freezes the exact asset for every build, so a successful Console build alone is insufficient. After freezing, later Console releases do not silently change or invalidate the accepted inputs.
 
 1. Read the latest published Console tag and compare it with Console `main`:
 
@@ -37,7 +37,7 @@ git -C "$CONSOLE_SCRATCH/console" tag -a "<console-tag>" -m "Release <console-ta
 git -C "$CONSOLE_SCRATCH/console" push origin "<console-tag>"
 ```
 
-Console tags include the `v` prefix. Pushing the tag triggers `.github/workflows/release.yml` (`🚀 Release`). Remove `CONSOLE_SCRATCH` after the gate completes.
+Console tags include the `v` prefix. Pushing the tag triggers the Console repository's [release workflow](https://github.com/rustfs/console/blob/main/.github/workflows/release.yml) (`🚀 Release`). Remove the task-owned `CONSOLE_SCRATCH` after the gate completes.
 
 3. Find the exact tag run and wait for completion:
 
@@ -55,4 +55,4 @@ test "$(gh api "repos/${CONSOLE_REPO}/releases/tags/<console-tag>" \
   --jq '[.assets[] | select(.name == "rustfs-console-<console-tag>.zip" and .state == "uploaded" and .size > 0 and (.digest | startswith("sha256:")))] | length')" -eq 1
 ```
 
-Treat a missing/mismatched asset, digest, latest tag, or failed/cancelled workflow as BLOCKED. Do not start Phase 1 until the Console gate passes. Record `CONSOLE_TAG`, `CONSOLE_HASH`, Console run URL, and Release URL for the final report.
+Treat a missing/mismatched asset, digest, latest tag, or failed/cancelled workflow as BLOCKED. Do not start Phase 1 until the Console gate passes. Record `CONSOLE_TAG`, `CONSOLE_HASH`, the asset URL/SHA256, Console run URL, and Release URL for comparison with the inputs actually consumed by the RustFS builds and the final report.

.agents/skills/rustfs-release-publish/references/final-release.md

@@ -1,22 +1,13 @@
-# Final Release Publication
+# Final release publication
 
-Read only after fresh user confirmation of the preview acceptance report. The [parent confirmation gate](../SKILL.md#human-confirmation) applies.
+Read only after fresh confirmation of the complete candidate report. The [parent confirmation gate](../SKILL.md#human-confirmation) applies.
 
-## Phase 6 — Publish the final tag on the validated commit
+Dispatch `release-publish.yml` on main with the successful completion run ID and approved `release-manifest.json` SHA-256. Do not create the tag manually. The workflow revalidates source, bytes and trusted certification, then creates the annotated final tag at `CANDIDATE_SHA`. A recipe change fails closed and requires a new candidate; unrelated main changes do not require a rebuild.
 
-No second source version bump. Before creating the final tag, recheck the remote `release/<target>` HEAD equals `PREVIEW_HASH`. If it moved after confirmation, the approval no longer applies; validate and obtain confirmation for a new preview. The final tag goes on the exact commit the preview validated:
+Require both versioned publication and channel publication jobs to succeed. Verify the final tag's commit and manifest identity; all six platform ZIPs and compatibility aliases; DEB/RPM, checksums, SBOM, provenance and evidence bundle; both multi-architecture image variants in Docker Hub, GHCR and Quay; and the accepted Helm chart in the chart repository. Download a host-compatible final artifact and confirm its digest is exactly the accepted candidate digest and its version/source output agrees.
 
-```bash
-git fetch origin --tags
-git rev-parse "<preview-tag>^{commit}"          # must equal PREVIEW_HASH — abort if not
-git tag -a "<target>" -m "Release <target>" "$PREVIEW_HASH"
-git push origin "<target>"
-```
+For stable releases verify GitHub Latest, R2 download aliases and `latest.json` unless a newer stable release already owns those channels. Prereleases must remain GitHub prereleases and leave stable pointers unchanged. Verify generated release notes compare against the frozen previous stable release, excluding internal previews.
 
-- CI rebuilds from the same source; the only changed input is the tag name, so the binary now self-reports `<target>`.
-- Verify the final tag's complete publication path: all matrix and release jobs green; `gh release view "<target>"` shows the full versioned and `-latest` asset set plus checksums, SBOM, and provenance; Docker and Helm workflows succeed; `latest.json` points to `<target>`. A stable target must have `isPrerelease=false` and `isLatest=true`. An alpha/beta/rc target must have `isPrerelease=true`; GitHub does not permit prereleases to be Latest, but the project `latest.json` still advances to the final non-preview target.
-- Verify the final Release body contains `## What's Changed` and a Full Changelog link. When `PREVIOUS_DELIVERABLE` exists, the link MUST be `https://github.com/rustfs/rustfs/compare/<PREVIOUS_DELIVERABLE>...<target>` and the baseline MUST equal the preview Release baseline; for example, both `1.0.0-beta.12-preview.1` and `1.0.0-beta.12` compare from `1.0.0-beta.11`.
-- Verify the preview cleanup: `cleanup-preview-releases` must succeed, `gh release view "<preview-tag>"` must then report `release not found` for every preview iteration of this target, and `git rev-parse "<preview-tag>^{commit}"` must still resolve to `PREVIEW_HASH` (the tag is kept). If the job failed, delete the leftover Releases manually with `gh release delete "<preview-tag>" --yes` and report it.
-- Optionally spot-check `./rustfs --version` from a final-tag artifact — it must report `<target>`.
+On infrastructure failure retry the same workflow/seal; matching destinations are reused. Never clobber an asset, force a tag, rebuild the image or silently accept a changed digest. A source fix after the final tag requires a new patch version. Cross-registry writes are not atomic: report the unfinished destination rather than claiming publication complete.
 
-Never pass `--cleanup-tag` when removing preview Releases. Remove them manually only if `cleanup-preview-releases` failed, after final publication and notes generation. Record deleted Releases and surviving tags.
+New candidates create no preview Releases to clean up. For a legacy preview release, preserve its tag and remove its Release only after final delivery and notes are verified. Do not pass `--cleanup-tag`. Complete the [post-release follow-up](post-release-updates.md) separately.

.agents/skills/rustfs-release-publish/references/post-release-updates.md

@@ -24,7 +24,7 @@ Reconcile advisories after every completed final release, including Console-only
 
 1. Re-read the published deliverable state before edits or a retry. If a newer release has superseded this target, do not downgrade installation references or the website announcement. Report the superseding release and leave those defaults intact.
 2. Invoke `rustfs-release-version-bump` with stage `post-release`, the exact target, and the authorized delivery scope. Prepare the follow-up on current main; do not edit or move either validated release tag. Cargo versions may already be preparing the next target and must remain untouched.
-3. Verify the exact Docker image manifest, Release download links, RPM source archive, and Helm repository entry before publishing the installation PR. The chart workflow overrides chart/app versions from the final tag, pins the source checkout to that release, and waits for its default Docker image before packaging.
+3. Verify the exact Docker image manifest, Release download links, RPM source archive, and Helm repository entry before publishing the installation PR. The chart is packaged and tested before sealing; publication uploads those same bytes only after versioned Docker images are available.
 4. Report the installation PR and its merge state separately from the completed release. Reuse existing merge authorization and repository checks; do not infer new merge authority from this reference.
 
 ## rustfs.com top banner
@@ -48,3 +48,10 @@ Use the existing announcement configuration from [website PR #112](https://githu
 3. Use the site's current package manager and required checks. For this config-only change, type-check and lint the changed file, build the site, and inspect the generated homepage for the version, visible link text, and destination. Confirm `/download` actually offers the announced version before using that link.
 4. Commit, push, and create or update the website PR when included in the authorized release scope. Include the published RustFS Release URL and checks run. Keep merge and production-deployment authority separate; reuse permission already given, but this reference grants none by itself.
 5. After an authorized merge/deployment, verify the live `https://rustfs.com/` banner and its destination. If only the PR is ready, report that state and URL; do not report the banner as deployed. A website failure does not invalidate or move the published release tag: retry this phase after resolving the failure.
+
+## Reconcile main and remove the completed release branch
+
+1. Compare the exact published tag with current main. Audit every release-side commit since their merge base, including inherited backports. An ahead count is a history difference, not proof of missing changes. Use ancestry and patch equivalence, then inspect current code and regression evidence for squashed, adapted, superseded, or reverted changes. Record each commit as present, superseded, release-only, or missing; a positive `git cherry` result alone does not prove a missing fix.
+2. Bring missing shared changes to main through a reviewed PR and the required checks before closing the follow-up. Preserve main's later changes; decide version bumps and pinned Console configuration separately. If a history merge is needed, use a merge commit and verify the release commit is an ancestor of main; squash does not establish that ancestry. Do not merge main into the published release branch or move its tags. Report unresolved changes instead of claiming reconciliation is complete.
+3. After publication and authorized follow-ups are complete, delete the remote `release/<target>` branch. Immediately before deletion, verify its HEAD still equals the published tag's commit and no open PR targets or originates from it. Preserve a branch with newer work or open PRs and report the blocker. Use a deletion with an expected-old-SHA check to avoid removing a concurrently advanced branch.
+4. Verify the remote branch is absent and the final tag and any legacy preview tags still resolve to their recorded commits. Remove a local release branch only if it is task-owned, unchecked out, and has no unpreserved work. Record cleanup separately from artifact publication; an explicitly authorized deletion before reconciliation does not make unresolved main changes complete.

.agents/skills/rustfs-release-publish/references/preview-acceptance.md

@@ -1,52 +1,68 @@
-# Preview Artifact Acceptance
+# Candidate Artifact Acceptance
 
-Read after Phase 3 succeeds. Complete every check below before the parent skill's manual confirmation gate. These checks cover the downloaded artifact, embedded Console, and latest rc client.
+Read after Phase 3 succeeds. Complete every check below before the parent skill's manual confirmation gate. These checks cover the downloaded artifact, embedded Console, and the rc release frozen in the candidate inputs.
 
 ## Phase 4 — Run the artifact locally, verify the console
 
-Work inside the session scratchpad directory; never leave stray data dirs.
+Use a unique task-owned scratch directory. Select the versioned archive matching the acceptance host, and verify it against the sealed manifest before extraction. Before desktop completion, use the Linux acceptance environment rather than rebuilding for the local desktop. Do not substitute a different build when a matching artifact is unavailable; report that acceptance gap. The shell example below assumes the verified executable is at `RUSTFS_BINARY` (`rustfs.exe` on Windows), `SCRATCH` is the task directory, and `S3_PORT` and `CONSOLE_PORT` are distinct available ports. Use equivalent native commands when the shell differs.
+
+Bind both listeners to loopback. Choose task-only credentials, set loopback proxy exclusions for the server/client processes, and record the server PID. A port probe does not reserve a port; retry startup with fresh ports on a bind conflict rather than connecting to an existing service.
 
 ```bash
-gh release download "<preview-tag>" -p "rustfs-macos-aarch64-v<preview-tag>.zip" -D "$SCRATCH"
-cd "$SCRATCH" && unzip -o rustfs-*.zip
-./rustfs --version        # must report the PREVIEW TAG (build::TAG), not the Cargo.toml version, plus expected short SHA
-mkdir -p data
-RUSTFS_ACCESS_KEY=rustfsadmin RUSTFS_SECRET_KEY=rustfsadmin ./rustfs ./data
+"$RUSTFS_BINARY" --version
+mkdir -p "$SCRATCH/data"
+S3_ENDPOINT="http://127.0.0.1:${S3_PORT}"
+CONSOLE_ENDPOINT="http://127.0.0.1:${CONSOLE_PORT}"
+RUSTFS_ACCESS_KEY="$PREVIEW_ACCESS_KEY" RUSTFS_SECRET_KEY="$PREVIEW_SECRET_KEY" \
+  "$RUSTFS_BINARY" server \
+  --address "127.0.0.1:${S3_PORT}" \
+  --console-enable \
+  --console-address "127.0.0.1:${CONSOLE_PORT}" \
+  "$SCRATCH/data" >"$SCRATCH/server.log" 2>&1 &
+PREVIEW_SERVER_PID=$!
 ```
 
-Defaults: S3 endpoint `:9000`, embedded console `:9001`.
-
 Checks (all must pass):
 
-- `./rustfs --version` reports the preview tag name and the short SHA of `PREVIEW_HASH`. Reporting `<target>` without the `-preview.N` suffix means the build did not embed the tag — treat as FAIL and investigate before proceeding.
-- `curl -fsS http://localhost:9000/health/ready` returns ready.
+- `"$RUSTFS_BINARY" --version` reports the exact final `<target>` and full `CANDIDATE_SHA`. A preview suffix, another version or another source is a failure.
+- Wait with a bounded readiness deadline while checking the recorded server process remains alive; `curl --noproxy '*' -fsS "$S3_ENDPOINT/health/ready"` must return ready from this process's listener.
 - Startup log shows the embedded console being served (this was the regression that `fix(release): require embedded console assets` guards).
-- Open `http://localhost:9001` in the browser: login with `rustfsadmin`/`rustfsadmin`; dashboard renders without JS console errors; create a bucket, upload a file, download it back (byte-identical), delete the object and bucket. Keep the server running for Phase 5.
+- Open `CONSOLE_ENDPOINT` in the browser and log in with the task credentials. The dashboard must render without JS console errors; create a bucket, upload a file, download it back (byte-identical), and delete the object and bucket. Keep the server running for Phase 5.
 
-## Phase 5 — Validate with the latest rc client
+## Phase 5 — Validate with the pinned rc client
 
 `rc` is the RustFS CLI client from <https://github.com/rustfs/cli>.
 
-- Ensure the latest release is installed: compare `rc --version` against `gh api repos/rustfs/cli/releases/latest --jq .tag_name`; update via `brew upgrade rustfs/tap/rc` (or download the release binary).
-- Point it at the preview server and run the command matrix, recording PASS/FAIL per command:
+- Read the CLI release and asset digests from `inputs.json`; verify the local client's version or download a host-compatible asset from that exact release into the task directory. If the host is a desktop, resolve its asset within the same pinned CLI release and record the digest; do not independently follow `latest`. Set `RC_BIN` to that executable and record its version/digest. Avoid changing the user's installed client for this check.
+- Inspect the client's current help for an isolated configuration option and use it when supported. Otherwise choose a unique `PREVIEW_ALIAS` and confirm it is absent before creating it; never overwrite an existing alias. Use task-owned bucket/user names and input/output files. The names below are safe only inside the newly started isolated server.
+- Point the client at `S3_ENDPOINT` with loopback proxy exclusions and run the command matrix, recording PASS/FAIL per command:
 
 ```bash
-rc alias set preview http://localhost:9000 rustfsadmin rustfsadmin
-rc ls preview/
-rc mb preview/rel-check
-rc cp <local-file> preview/rel-check/
-rc stat preview/rel-check/<file>
-rc cat preview/rel-check/<file>          # matches source
-rc cp preview/rel-check/<file> ./out && cmp <local-file> ./out
-rc cp -r <local-dir>/ preview/rel-check/dir/
-rc find preview/rel-check --name "*"
-rc share download preview/rel-check/<file> --expire 1h   # presigned URL fetchable via curl
-rc rm preview/rel-check/<file> && rc rm -r --force preview/rel-check/dir
-rc rb preview/rel-check
-rc admin user list preview/
-rc admin user add preview/ relcheckuser relchecksecret12
-rc admin user remove preview/ relcheckuser
-rc alias remove preview
+"$RC_BIN" alias set "$PREVIEW_ALIAS" "$S3_ENDPOINT" "$PREVIEW_ACCESS_KEY" "$PREVIEW_SECRET_KEY"
+"$RC_BIN" ls "$PREVIEW_ALIAS/"
+"$RC_BIN" mb "$PREVIEW_ALIAS/rel-check"
+"$RC_BIN" cp <local-file> "$PREVIEW_ALIAS/rel-check/"
+"$RC_BIN" stat "$PREVIEW_ALIAS/rel-check/<file>"
+"$RC_BIN" cat "$PREVIEW_ALIAS/rel-check/<file>"          # matches source
+"$RC_BIN" cp "$PREVIEW_ALIAS/rel-check/<file>" "$SCRATCH/out"
+cmp <local-file> "$SCRATCH/out"
+"$RC_BIN" cp -r <local-dir>/ "$PREVIEW_ALIAS/rel-check/dir/"
+"$RC_BIN" find "$PREVIEW_ALIAS/rel-check" --name "*"
+"$RC_BIN" share download "$PREVIEW_ALIAS/rel-check/<file>" --expire 1h
+# Fetch the presigned URL with curl --noproxy '*' and compare its body.
+"$RC_BIN" rm "$PREVIEW_ALIAS/rel-check/<file>"
+"$RC_BIN" rm -r --force "$PREVIEW_ALIAS/rel-check/dir"
+"$RC_BIN" rb "$PREVIEW_ALIAS/rel-check"
+"$RC_BIN" admin user list "$PREVIEW_ALIAS/"
+"$RC_BIN" admin user add "$PREVIEW_ALIAS/" relcheckuser relchecksecret12
+"$RC_BIN" admin user remove "$PREVIEW_ALIAS/" relcheckuser
+"$RC_BIN" alias remove "$PREVIEW_ALIAS"
 ```
 
-- Any FAIL blocks the release. Afterwards stop the server and delete the scratch data directory.
+- Any FAIL blocks acceptance. On success or failure, stop only the recorded task process and remove only task-created aliases/data after retaining the needed failure evidence. Do not erase an existing client configuration or unrelated server data.
+
+## Field acceptance and desktop completion
+
+Before accepting the Linux manifest, record Scanner/Heal, multi-pool, memory, major features, high-concurrency 503, performance and previous-version integrity/upgrade results in backlog. Preserve the release-specific agreed limits and baseline; a PASS label without artifact identities or test results does not satisfy acceptance.
+
+Dispatch `release-complete.yml` only after that report passes, using the exact Linux manifest digest and backlog URL. Require native macOS/Windows checks, package extraction, image startup/scans, chart tests and complete certification to pass. Complete any desktop Console/client checks applicable to the release against these same archives. Record the completion run and final manifest SHA-256 before entering the parent confirmation gate. Partial Linux acceptance does not authorize final publication.

.agents/skills/rustfs-release-publish/references/preview-release.md

@@ -1,24 +1,9 @@
-# Preview Release Verification
+# Linux candidate verification
 
-Read for Phase 3 after the candidate passed exact-SHA CI. The [parent release invariants](../SKILL.md) apply.
+Read for Phase 3 after the source passed exact-SHA CI. The [parent invariants](../SKILL.md) apply; commands and recovery live in the [pipeline runbook](../../../../docs/operations/release-pipeline.md).
 
-## Preview channels
+Dispatch `release-candidate.yml` on main with the confirmed final version and `CANDIDATE_SHA`. Record the run ID, frozen `inputs.json`, every Linux archive digest and `linux-manifest.json` SHA-256. Require the four Linux builds, native startup/S3 checks, previous-release upgrade matrix and Linux seal to succeed. Missing, skipped, cancelled or failed required work is incomplete acceptance.
 
-- Use `<target>-preview.N` for every target, e.g. `1.0.0-beta.10-preview.3` or `1.1.0-preview.1`.
-- The canonical suffix is exactly `-preview.<digits>`. `build.yml` recognizes it before alpha/beta/rc classification and routes it to the preview-only path; any other tag containing `-preview` fails closed instead of being treated as a release.
-- A preview Release MUST be published with `isPrerelease=true` and `isLatest=false`. Any `*-latest` preview asset or preview-triggered `latest.json`, R2, Helm, or moving Docker channel tag is a pipeline failure. Docker images may use the exact preview version tag, including its existing variant suffixes.
-- Preview Releases are cleaned up by the `cleanup-preview-releases` job after `publish-release` succeeds for the deliverable tag. It deletes every Release whose tag is exactly `<target>-preview.<digits>` and never passes `--cleanup-tag`, so the tags survive.
+Verify the frozen Console identity equals the one accepted in Phase 0. Every matrix job consumes that same digest and exact toolchain. The binary must report the final product version and source commit; its candidate ID is separate. No public tag, Release, registry tag, R2 release path, Helm repository or latest pointer should change during this stage.
 
-## Phase 3 — CI and preview Release verification
-
-- Find and watch the tag build: `gh run list --workflow build.yml --branch "<preview-tag>" --limit 1` then `gh run watch <run-id>`. Every build matrix target must succeed (linux x86_64/aarch64 × musl/gnu, macos-aarch64, windows-x86_64).
-- Confirm the preview tag resolves to the Phase 2 CI-validated `PREVIEW_HASH`. If the release branch advances during acceptance, repeat Phases 2–5 with a new preview tag.
-- Confirm the Release publication jobs (`create-release`, `upload-release-assets`, and `publish-release`) succeed while `update-latest-version` is skipped.
-- Verify `gh release view "<preview-tag>" --json isPrerelease,assets,url`: `isPrerelease` must be `true`, and the Release must contain all 6 versioned platform zips, checksums, SBOM, and provenance with no `-latest` assets. Confirm `gh api repos/{owner}/{repo}/releases/latest --jq .tag_name` does not return `<preview-tag>`.
-- Record `PREVIOUS_DELIVERABLE`, selected from published Releases by `publishedAt` after excluding the current tag and every `-preview.N` tag. Verify `gh release view "<preview-tag>" --json body --jq .body` contains `## What's Changed` and, when `PREVIOUS_DELIVERABLE` exists, `**Full Changelog**: https://github.com/rustfs/rustfs/compare/<PREVIOUS_DELIVERABLE>...<preview-tag>`. For a repository with no previous deliverable, verify a Full Changelog link exists and record the GitHub baseline fallback.
-- Confirm Helm is skipped. If preview Docker images are published, confirm they use only exact preview tags (and variant suffixes), with no `latest`, `alpha`, `beta`, or `rc` channel updates. Preview validation covers the built RustFS binaries, embedded console, and rc compatibility; it does not authorize advancing any installation default.
-
-## Release notes
-
-- When a previous deliverable exists, GitHub Release notes for the preview and final tags MUST use it as their shared comparison baseline: the most recently published non-preview Release before the target. Internal `-preview.N` Releases are explicitly excluded from that selection, even when they point at the same commit as the final tag — cleanup runs after the notes are generated, so the preview Release is still present and would otherwise be picked as the baseline. If no previous deliverable exists, omit `previous_tag_name` and record that GitHub's default baseline fallback was used.
-- Generated Release notes carry a workflow-management marker so retries can repair them. Before manually curating a generated body, remove that marker; unmarked non-placeholder notes are preserved by later workflow runs.
+A green Linux run is ready for field acceptance, not publication. The partial certification must have `publish_allowed: false`. Retain the exact Linux manifest for the next stage; a new source/build-input candidate must not inherit the old approval. A newer unrelated main commit does not invalidate this evidence.

.agents/skills/rustfs-release-publish/references/release-candidate.md

@@ -16,22 +16,13 @@ Read for Phases 1–2. The [parent release invariants](../SKILL.md) and Console
 - Only when neither the release branch, an unfinished preview tag, nor current main already has `<target>`: if main is at a later version, select and verify a historical commit containing `<target>` rather than downgrading main; block if none exists. Otherwise invoke the `rustfs-release-version-bump` skill with stage `prepare`, the final `<target>` (NOT a preview version), and full GitHub flow (commit/push/PR).
 - If a bump PR was needed, get it merged into main and record its merge commit. For a new release branch when main already has `<target>`, identify the main commit that introduced those versions. Phase 2 chooses a release candidate containing that commit; an existing release branch keeps its own history and a newer main HEAD is not required.
 
-Do not wait for the moving latest main HEAD to become green. The release-branch CI in Phase 2 validates the chosen commit before a preview tag is created.
+Do not wait for the moving latest main HEAD to become green. The release-branch CI in Phase 2 validates the chosen commit before a candidate build is dispatched.
 
-## Phase 2 — Pin and validate the release candidate, then publish the preview tag
+## Phase 2 — Pin and validate the release source
 
-- Use `release/<target>`. On the first attempt, create it from a selected main commit containing the version bump and intended changes; it need not be the latest main HEAD. On a restart, retain the existing branch and backport only reviewed fixes from main. Verify the branch's Cargo versions still equal `<target>`. If adopting an existing preview, run this branch CI before continuing its acceptance; reuse the tag only when it points to the validated SHA.
-- The current `ci.yml` runs automatically on main pushes, not release-branch pushes. Dispatch it explicitly with `gh workflow run ci.yml --ref "release/<target>"`. Locate that dispatch run, verify its `head_sha` equals the release branch's remote HEAD, and require the full expected CI job set to pass, including `Quick Checks`, `Test and Lint`, and `End-to-End Tests (full merge gate)`. Treat a missing or skipped required lane as incomplete. A successful PR check or CI run for a different SHA does not qualify. If the branch advances or a run is cancelled, validate the new SHA again. Do not tag a candidate with failing or incomplete CI.
+- Use `release/<target>`. On the first attempt, create it from a selected main commit containing the version bump and intended changes; it need not be the latest main HEAD. On a restart, retain the existing branch and backport only reviewed fixes from main. Verify the branch's Cargo versions still equal `<target>`. A legacy preview can anchor source selection, but its tag and old artifacts do not constitute a candidate in the new protocol; freeze and verify a new candidate after the release recipe is aligned.
+- The current `ci.yml` runs automatically on main pushes, not release-branch pushes. Dispatch it explicitly with `gh workflow run ci.yml --ref "release/<target>"`. Locate that dispatch run, verify its `head_sha` equals the release branch's remote HEAD, and require the full expected CI job set to pass, including `Quick Checks`, `Test and Lint`, and `End-to-End Tests (full merge gate)`. Treat a missing or skipped required lane as incomplete. A successful PR check or CI run for a different SHA does not qualify. If the branch advances or a run is cancelled, validate the new SHA again. Do not dispatch a candidate with failing or incomplete CI.
 - Prefer protecting `release/*` against force pushes and deletion and requiring reviewed backport PRs. Check live rulesets rather than assuming the main ruleset covers release branches; report a missing release-branch rule as a gap.
-- Set `PREVIEW_HASH` to the CI-validated release-branch commit and recheck the remote branch HEAD immediately before tagging. Report the branch, commit, and CI run URL; do not derive `PREVIEW_HASH` from a later `origin/main` fetch. When adopting an existing preview tag at this SHA, skip tag creation and continue Phase 3.
-
-```bash
-git tag -a "<preview-tag>" -m "Release <preview-tag>" "$PREVIEW_HASH"
-git push origin "<preview-tag>"
-```
-
-Pushing the tag triggers `.github/workflows/build.yml` ("Build and Release"); `docker.yml` chains off it via `workflow_run`.
-
-The preview run builds versioned artifacts and publishes them in a GitHub prerelease. Docker may publish exact preview image tags. Latest-channel, R2, and Helm publication must be skipped; installation references and the website announcement stay unchanged.
-
-On a restart after a backport, re-run release-branch CI before setting the new `PREVIEW_HASH`. Use the next unused preview iteration when the validated commit differs from an existing preview tag; never move an existing tag.
+- Set `CANDIDATE_SHA` to the CI-validated release-branch commit. Record the branch, exact commit and CI run URL. The new candidate protocol starts from a commit whose release recipe matches the reviewed main dispatcher; backport the recipe before freezing an older release branch if needed.
+- Dispatch the Linux candidate as described in [candidate verification](preview-release.md). Do not create a preview or final tag here. The final executable already contains `<target>`; publication later promotes those same bytes.
+- After a source or build-input change, run the affected source checks and create a new candidate. Reuse the frozen candidate after an infrastructure failure. Do not restart acceptance for unrelated main advances.

.agents/skills/rustfs-release-version-bump/SKILL.md

@@ -4,18 +4,18 @@ description: Prepare an exact RustFS Cargo version or align installation referen
 ---
 # Release Version Files
 
-Require an exact target; ask if missing or ambiguous. Reject targets containing `-preview`: those are tag-only. Infer `post-release` only from an explicit request or parent workflow; otherwise use `prepare`. Delivery defaults to local edit/verify; commit, push, PR, and merge follow existing authorization and [Git rules](../../references/pull-requests.md).
+Require an exact target; ask if missing or ambiguous. Reject targets containing `-preview`: those identify candidates, not product versions or new public tags. Infer `post-release` only from an explicit request or parent workflow; otherwise use `prepare`. Delivery defaults to local edit/verify; commit, push, PR, and merge follow existing authorization and [Git rules](../../references/pull-requests.md).
 
 | Stage | Files and constraints |
 |---|---|
 | `prepare` | Only `Cargo.toml` and `Cargo.lock`: workspace package, internal dependency, and workspace member versions become `<target>`. External dependencies stay unchanged. |
 | `post-release` | Only `README.md`, `README_ZH.md`, `flake.nix`, `helm/rustfs/Chart.yaml`, and `rustfs.spec`. Cargo may already target the next release; leave it untouched. |
 
-Before installation edits, verify the exact non-preview Release, downloadable assets/source archive, pullable `rustfs/rustfs:<target>` image manifest, and published Helm chart. Block on missing artifacts. Do not downgrade defaults if a newer deliverable superseded the target. Preview preparation retains the previous published installation references; follow-up commits never replace either validated tag.
+Before installation edits, verify the exact non-preview Release, downloadable assets/source archive, pullable `rustfs/rustfs:<target>` image manifest, and published Helm chart. Block on missing artifacts. Do not downgrade defaults if a newer deliverable superseded the target. Candidate preparation retains the previous published installation references; follow-up commits never replace validated tags or sealed artifacts.
 
 Packaging policy:
 
-- Docker image tags have no `v` prefix. Derive chart/app versions with `scripts/helm_chart_version.sh <target>` (`beta.N -> 0.N.0`; other versions unchanged). Helm CI derives versions from the final tag, so no early chart bump is required.
+- Docker image tags have no `v` prefix. Derive chart/app versions with `scripts/helm_chart_version.sh <target>`; both equal the final product SemVer, including prerelease suffixes. Helm CI uses the frozen candidate version before sealing, so no early installation chart bump is required. Previously published chart versions remain unchanged.
 - RPM `Version` is numeric; `Release` is the prerelease suffix or `1` for stable. Expanded `Source0` and the unpack directory must resolve to the exact target including its suffix. Add the changelog entry using current git identity, date, and target; never copy an example identity/date.
 - Changing these mappings requires explicit confirmation.
 

.agents/skills/security-advisory-lessons/SKILL.md

@@ -1,56 +0,0 @@
----
-name: security-advisory-lessons
-description: Perform a dedicated RustFS security/advisory review for authn/authz, IAM, RPC trust, paths, secrets, browser isolation, encryption, Object Lock, or other security boundaries. Use only when the user requests a security/advisory review or an adversarial review explicitly escalates to the full advisory map; do not auto-load solely because code touches a sensitive path.
----
-
-# RustFS Security Advisory Lessons
-
-Use this skill as the deep security lens. For a normal adversarial review with a
-matched security surface, the concise security reference under
-`adversarial-validation` is sufficient.
-
-## Workflow
-
-1. Freeze the exact diff/head and identify the changed trust boundaries.
-2. Inspect the headings in [advisory-patterns.md](references/advisory-patterns.md),
-   then read the matching sections. Read the full map only for a broad security
-   audit. Useful headings are
-   auth/admin, IAM/STS/OIDC, policy/plugins, S3/copy/multipart, protocols, paths,
-   secrets/logging/RPC, browser/CORS/proxy, SSE, Object Lock, and serde.
-3. Trace unauthenticated, low-privilege, wrong-action/owner/bucket, malformed,
-   and default-config cases. Security decisions must fail closed.
-4. Require a focused negative regression test for the bypass/exploit form, not
-   only the intended success path. State residual risk when a test is impractical.
-5. Report proven vulnerabilities separately from defense-in-depth hardening.
-
-When advisory currentness matters, fetch the live inventory instead of treating
-the reference as a status mirror:
-
-```bash
-gh api repos/rustfs/rustfs/security-advisories --paginate \
-  --jq '.[] | {ghsa_id,state,severity,summary,updated_at}'
-```
-
-Fetch an individual advisory only when the live summary indicates a new or
-changed lesson.
-
-## Finding Standard
-
-Every candidate names the lower-trust principal, the boundary it crosses, the
-affected resource, and the observable result. A best-practice gap or an impact
-confined to the caller's own principal is hardening, not a candidate.
-
-Report each candidate under exactly one verdict:
-
-- `confirmed`: a complete `file:line` source trace and a bounded observed
-  result. Only confirmed findings carry severity; each also states attacker
-  prerequisites, concrete input/path, impact, smallest safe fix, and a
-  regression check.
-- `needs_validation`: the trace is blocked on one exact unresolved fact, such
-  as deployment or infrastructure behavior. State that fact; assign no severity
-  and do not substitute an assumption.
-- `rejected`: a disproved candidate, with the caller, invariant, or test that
-  disproves it.
-
-Do not exaggerate unauthenticated impact when the actual issue requires
-authenticated low privilege.

.agents/skills/security-advisory-lessons/references/advisory-patterns.md

@@ -1,158 +0,0 @@
-# RustFS Advisory Pattern Map
-
-This file is a lesson map, not an advisory inventory mirror. It keeps durable security patterns distilled from RustFS GitHub Security Advisories.
-
-When current advisory state, severity, URLs, or full text matters, fetch it live:
-
-```bash
-gh api repos/rustfs/rustfs/security-advisories --paginate \
-  --jq '.[] | {ghsa_id,state,severity,summary,updated_at}'
-gh api repos/rustfs/rustfs/security-advisories/<GHSA_ID>
-```
-
-Update this file only when an advisory adds or changes a reusable lesson, affected surface, validation pattern, or regression-test expectation. Do not update it for state-only, URL-only, count-only, or timestamp-only changes.
-
-## Pattern Index
-
-### Admin authorization and route exposure
-
-- `GHSA-pfcq-4gjr-6gjm`: notification target endpoints accepted authenticated users but skipped admin authorization. Lesson: distinguish authn from authz; admin target CRUD must call the operation-specific admin authorization path.
-- `GHSA-mm2q-qcmx-gw4w`: `ListServiceAccount` used `UpdateServiceAccountAdminAction`, while update lacked target ownership checks. Lesson: exact action constants and ownership checks are both required; information disclosure can chain into secret rotation and takeover.
-- `GHSA-vcwh-pff9-64cc`: `ImportIam` checked `ExportIAMAction` for an import/write operation. Lesson: every admin handler must authorize the action it actually performs.
-- `GHSA-jqmc-mg33-v45g` and `GHSA-8784-9m7f-c6p6`: `/profile/cpu` and `/profile/memory` were whitelisted from auth and allowed expensive diagnostics plus path disclosure. Lesson: profiling/debug endpoints need admin auth, opt-in, rate limits, and non-sensitive responses.
-- `GHSA-f5cv-v44x-2xgf`: `/rustfs/admin/v3/metrics` accepted any authenticated IAM user and skipped admin authorization. Lesson: read-only metrics and diagnostic admin endpoints still require an operation-specific admin action check.
-- `GHSA-796f-j7xp-hwf4`: `/rustfs/admin/v3/list-remote-targets` checked only that credentials existed and leaked replication target credentials. Lesson: replication target reads are privileged admin operations, and stored remote credentials require strict authz plus response redaction review.
-- `GHSA-xp32-gxq2-3v52`: console license metadata endpoint was public and exposed subject and expiration fields. Lesson: management metadata endpoints should require admin auth or return only coarse public status.
-
-### IAM import, service accounts, and privilege boundaries
-
-- `GHSA-566f-q62r-wcr8`: `ImportIam` accepted attacker-controlled service account `parent`, `claims`, `accessKey`, and `secretKey`, enabling persistent backdoor accounts under root. Lesson: imported IAM payloads are untrusted data and must be validated against privilege boundaries.
-- `GHSA-3495-h8r9-gfqg`: `ExportIAM` wrote regular-user and service-account secret keys into exported ZIP data. Lesson: IAM export is a credential-disclosure boundary; redact, seal, or strictly justify every exported secret before treating export permission as safe.
-- `GHSA-5354-r3w2-34m8`: `AddServiceAccount` checked `CreateServiceAccountAdminAction` but trusted caller-supplied `target_user`, allowing service accounts under the root parent. Lesson: service-account create paths must validate parent ownership or root/admin authority, not only the create action.
-- `GHSA-xgr5-qc6w-vcg9`: `deny_only=true` skipped allow checks and let restricted service accounts mint unrestricted children. Lesson: deny-only logic must never become implicit allow for privilege creation.
-- `GHSA-mm2q-qcmx-gw4w`: leaked service account access keys plus update-without-ownership formed an escalation chain. Lesson: service-account identifiers are security-sensitive because update APIs consume them.
-
-### STS, OIDC, and federation flows
-
-- `GHSA-5qfg-mf7r-jp3w` and `GHSA-3473-5353-xhwh`: `AssumeRoleWithWebIdentity` was reachable through unauthenticated `POST /` routing and could issue temporary credentials from crafted web identity input. Lesson: every STS route needs explicit SigV4 or trusted identity-provider validation before role assumption, and unauthenticated exemptions must be narrowed to the exact action with uniform failure responses.
-- `GHSA-jxrr-r6pv-h958`: unsigned JWT issuer data was decoded before verification to select an OIDC provider, and distinguishable failures could expose provider configuration. Lesson: web-identity routing may be unauthenticated, but pre-verification claims are untrusted routing hints; bound and rate-limit the request, normalize public errors, and verify signature, issuer, audience, and expiration before issuing credentials.
-- `GHSA-ccrv-v8v9-ch9q`, `GHSA-48rf-7j3q-3hfv`, and `GHSA-xvfh-7c9g-hpw2`: service-account-controlled material could self-sign JWT session tokens with forged policy claims, and missing `exp` was accepted for service-account tokens. Lesson: session tokens must be signed by a trusted issuer/key path, enforce required claims and expiration, and reject self-signed or principal-controlled tokens.
-- `GHSA-9pjf-w3c2-m32r`, `GHSA-4x2q-cpx9-9h26`, and `GHSA-xvpm-p3f7-34c3`: public OIDC authorize/callback flows trusted request `Host` or forwarded scheme when building credential-bearing redirects. Lesson: OIDC redirects must use configured allowlisted origins and trusted-proxy handling; never derive the post-login credential destination from direct client headers.
-- `GHSA-m479-9x88-94w6`, `GHSA-frwq-mfqx-83p8`, `GHSA-q9q8-rf9r-fg9f`, and `GHSA-j5c2-hhf7-6gf5`: OIDC validation accepted attacker-controlled discovery URLs because hostname checks rejected only literal forbidden IPs, allowing DNS rebinding SSRF. Lesson: outbound federation URL validation must resolve and classify hostnames at the connection boundary and reject loopback, private, link-local, and rebound addresses.
-
-### IAM policy conditions and external policy plugins
-
-- `GHSA-6r96-hmgc-726c`: request headers collided with lowercase server-derived condition keys such as `userid`, `groups`, `versionid`, and JWT/LDAP claims. Lesson: never let caller-controlled headers append to or replace server-derived policy context; reserve trusted condition keys and keep intentional request-header keys separate.
-- `GHSA-v9cp-qfw9-9pfp`: quantified negated string conditions applied negation after aggregation, transposing `ForAllValues` and `ForAnyValue` semantics. Lesson: push negation into the per-value predicate for quantified operators and test partially overlapping multi-value sets.
-- `GHSA-5w8r-p896-6vq2`: OPA policy mode skipped `ExistingObjectTag/*` loading, so tagged objects looked untagged to external policies. Lesson: external authorization plugins need the same object-tag and request context as built-in policy evaluation before they decide.
-- `GHSA-9wgh-5r68-49p7`: substituted policy-variable values could recreate `${...}` and loop inside a single resolution pass, never reaching the outer iteration limit. Lesson: bound substitutions within each pass and test cycles formed across otherwise literal resolver values; confirm whether attacker-controlled claims can reach those values before assigning remote impact.
-
-### S3 object actions, copy, multipart, and upload policy validation
-
-- `GHSA-g8w9-qw9q-fghr` and `GHSA-xm99-m3gq-83g8`: a valid presigned `PutObject` accepted extra `x-amz-*` headers omitted from `SignedHeaders`; `x-amz-copy-source` could turn that upload capability into a cross-bucket read performed as the signer. Lesson: a presigned URL is a bounded capability; inspect all received security-sensitive headers and reject unsigned ones before selecting the S3 operation or reaching storage.
-- `GHSA-3ppv-fx5m-m749`: explicit `versionId` reads and copy sources authorized `s3:GetObject` instead of `s3:GetObjectVersion`. Lesson: version-specific object access must select version-specific actions for direct reads, `CopyObject`, and `UploadPartCopy`, with tests proving the backend is not reached on denial.
-- `GHSA-cg33-pxfp-qcj3` and `GHSA-65wq-gjw8-xr9w`: `HeadObject` and object-tagging paths authorized current-object actions while operating on an explicit historical `versionId`. Lesson: select the corresponding version action for metadata and tag reads and writes too; audit every operation that accepts a version ID.
-- `GHSA-gjph-xcgw-4p7w`: `PutObject` and browser POST accepted a caller-supplied `versionId` and replaced an unlocked historical version with only `s3:PutObject`. Lesson: only replication-authorized writes may choose an existing version ID; ordinary uploads must not rewrite version history.
-- `GHSA-x298-9x87-fvjq`: anonymous `ListObjectVersions` fell back to `ListBucket` and returned before public-access-block gates. Lesson: compatibility fallbacks must converge on the same post-authorization checks as direct grants, especially `RestrictPublicBuckets` and anonymous data-plane denies.
-- `GHSA-mx42-j6wv-px98`: `UploadPartCopy` missed source authorization and allowed cross-bucket object exfiltration. Lesson: multipart copy must enforce the same source and destination contract as `CopyObject`.
-- `GHSA-wfxj-ph3v-7mjf`: `UploadPartCopy` checked source and destination independently but missed destination copy-source policy constraints. Lesson: source read and destination write checks are not sufficient when policy constrains allowed copy sources.
-- `GHSA-w5fh-f8xh-5x3p`: presigned POST accepted uploads without enforcing signed policy conditions. Lesson: parse and enforce all POST policy constraints server-side, including size, key prefix, and content type.
-
-### Protocol frontends and IAM parity
-
-- `GHSA-3g29-xff2-92vp`: FTP `RETR` and `SIZE`/`MDTM` read paths authenticated the user but skipped IAM before calling storage. Lesson: non-HTTP protocol frontends must enforce the same per-operation authorization as the S3 API before backend access.
-- `GHSA-g3vq-vv42-f647`: FTPS `MKD` called `create_bucket` without checking `s3:CreateBucket`. Lesson: protocol command handlers need action-specific checks even when sibling handlers already authorize correctly.
-- `GHSA-3p3x-734c-h5vx`: FTPS and WebDAV compared secret keys with early-return string equality, while FTPS also returned distinguishable invalid-user and invalid-password failures. Lesson: password-style protocol auth needs constant-time secret comparison, indistinguishable failures where practical, and rate limiting.
-- `GHSA-rhg6-jrf2-29c4`: WebDAV, FTP/FTPS, and SFTP used identity-policy checks with empty conditions and bypassed S3 bucket-policy explicit denies and condition-key denies. Lesson: alternate listeners must evaluate the same effective policy and trustworthy request context before storage access.
-
-### Filesystem paths and object key traversal
-
-- `GHSA-pq29-69jg-9mxc`: RPC `read_file_stream` joined untrusted paths under a volume directory without canonical boundary checks. Lesson: `PathBuf::join` plus length checks are not path security.
-- `GHSA-8r6f-hmq2-28rg`: object keys containing traversal sequences bypassed bucket/object authorization when mapped to filesystem paths. Lesson: reject traversal at object-key parsing and verify final storage paths remain under the expected bucket/key root.
-- `GHSA-f4vq-9ffr-m8m3`: Snowball auto-extract accepted archive entries such as `../victim-bucket/object`, authorized the raw attacker-bucket path, then storage path cleaning crossed bucket boundaries. Lesson: archive entries become object keys and need traversal rejection plus consistent authz/storage normalization before writes.
-- `GHSA-g7fq-6xjm-mxf4`: versioned `DeleteObjects` accepted dot-segment keys, authorized the literal prefix, then wrote delete markers under a resolved sibling prefix. Lesson: validate each batch key before authorization and storage normalization, using the same object-name rules as single delete.
-
-### Secrets, defaults, and cryptographic misuse
-
-- `GHSA-j59h-h7q5-q348`, `GHSA-3wm5-wpm5-hmfm`, `GHSA-6wc8-xm48-qhmx`, `GHSA-9gf3-jx4p-4xxf`, `GHSA-63xc-c3w3-m2cf`, and `GHSA-ch63-6q4v-hwp5`: RustFS shipped known default root credentials that could authenticate to S3, admin APIs, IAM, KMS, console, and token-signing surfaces. Lesson: root credentials must be operator-provided or generated per install; known defaults and warnings are not acceptable for network-reachable deployments.
-- `GHSA-h956-rh7x-ppgj`: gRPC used the hard-coded token `rustfs rpc` on both client and server. Lesson: source-visible shared tokens are authentication bypasses.
-- `GHSA-r5qv-rc46-hv8q`: internode RPC HMAC secret fell back to the public default `rustfsadmin`. Lesson: RPC/internode auth must fail closed instead of silently using public defaults.
-- `GHSA-75fx-qg6f-8rm7` and `GHSA-68cw-96m3-h2cf`: internode RPC secrets were derivable from known root credentials, making raw storage RPC signatures forgeable when explicit RPC secrets were unset. Lesson: RPC auth keys must be independent random secrets, never derived from S3 root credentials, and raw storage RPC should not share the public S3 listener without an internode-only boundary.
-- `GHSA-m77q-r63m-pj89`: STS JWTs used the root secret key as the shared token signing key, allowing token forgery when the root secret was known. Lesson: STS signing keys need key separation, rotation, and key IDs; do not reuse root credentials for JWT/HMAC signing.
-- `GHSA-923g-jp7v-f97f`: license verification embedded a production RSA private key and used private-key decryption as authenticity. Lesson: ship verifying/public keys only and use real signature verification.
-
-### Sensitive logging and debug output
-
-- `GHSA-r54g-49rx-98cr`: STS credentials were logged at info level. Lesson: generated credentials must never be logged in plaintext.
-- `GHSA-8cm2-h255-v749`: debug logs leaked session tokens, secret keys, JWT claims, and raw STS response bodies. Lesson: redaction must cover custom `Debug` implementations and dependency response-body logging.
-- `GHSA-333v-68xh-8mmq`: invalid RPC signature logging included the shared HMAC secret and expected signature. Lesson: error paths often leak secrets; never log raw secrets or derived authenticators.
-
-### RPC input validation and panic safety
-
-- `GHSA-gw2x-q739-qhcr`: malformed gRPC `GetMetrics` payloads reached `unwrap()` on deserialization and caused remote DoS. Lesson: every network/RPC deserialization failure returns an error, not a panic.
-- `GHSA-h956-rh7x-ppgj` and `GHSA-r5qv-rc46-hv8q`: weak RPC auth increased reachability of otherwise internal handlers. Lesson: panic bugs become more severe when internode auth is weak or defaulted.
-- `GHSA-c667-rgrv-99vj`: NodeService authentication signed the service prefix instead of the concrete generated method path, so valid metadata for one RPC could be replayed to another method during the timestamp window. Lesson: RPC HMAC payloads must bind exact gRPC method path, HTTP method surrogate, timestamp, and secret.
-
-### Browser, CORS, and console isolation
-
-- `GHSA-v9fg-3cr2-277j`: object preview rendered attacker-controlled HTML in a same-origin iframe, exposing console credentials stored in `localStorage`. Lesson: user content must be origin-isolated from the console and protected with `nosniff`, CSP, and strict content-type handling.
-- `GHSA-7gcx-wg4x-q9x6`: an incomplete preview fix reintroduced extension-based PDF detection and bypassed the sandboxed fallback for attacker-controlled content. Lesson: browser-surface fixes need regression tests for alternate viewers and file-type branches, and preview trust must come from validated content type plus sandboxing rather than object names.
-- `GHSA-x5xv-223c-8vm7`: default CORS reflected arbitrary origins with credentials. Lesson: never combine reflected origins with `Access-Control-Allow-Credentials: true`; default should be fail-closed.
-
-### Trusted proxy and source IP conditions
-
-- `GHSA-fc6g-2gcp-2qrq`: `aws:SourceIp` trusted client-supplied `X-Forwarded-For` or `X-Real-IP`. Lesson: forwarded IP headers are valid only behind configured trusted proxies; direct clients use socket peer IP.
-
-### SSE and on-disk storage invariants
-
-- `GHSA-xrrf-67jm-3c2r`: SSE metadata reported encryption while reader composition bypassed `EncryptReader` and stored plaintext. Lesson: test actual bytes on disk and wrapper order, not only API metadata.
-- `GHSA-wqmc-vjgv-jrpw`: SSE-C persisted and listed the plaintext MD5 as the ETag, allowing list-only users to confirm or brute-force low-entropy content and correlate equal plaintext across keys. Lesson: encrypted-object ETags must not expose deterministic plaintext fingerprints; protect them with per-object key material and keep listing, conditional, copy, and multipart semantics consistent.
-
-### Object Lock and retention invariants
-
-- `GHSA-j548-9grx-fh4f`: Object Lock enforcement treated unreadable, fabricated, or unparsable bucket metadata as absent configuration and allowed retained objects to be deleted or expired. Lesson: retention must fail closed unless Object Lock absence is authoritative, and every delete, lifecycle, scanner, force-delete, and default-retention path needs the same state distinction.
-- `GHSA-28w4-rp2p-636w` and `GHSA-c7hx-vr97-45rg`: an authorized `DeleteBucket` with `x-rustfs-force-delete: true` removed active COMPLIANCE-retained versions even though ordinary version deletion was denied. Lesson: bucket force-delete must enforce retention across every version, including against root/owner callers.
-- `GHSA-q23w-fp7p-472v`: `PutObjectRetention` policy conditions read `x-amz-object-lock-*` headers while the handler applied mode and date from the XML body. Lesson: authorize the parsed retention that will be stored, or reject mismatched header/body values; compare retention dates as instants so offsets cannot change the applied bound.
-
-### Serde deserialization and input validation
-
-- Reject unknown fields in untrusted S3 API XML/JSON, lifecycle, policy, and replication input where compatibility permits. Check the current type and supported payload fixtures; do not infer the repository's current coverage from an older audit. Where extra fields are part of the compatibility contract, validate security-critical values explicitly.
-- `#[serde(default)]` on security-critical fields silently accepts missing values as zero/empty. Lesson: when a field has security implications (retention days, permissions, limits), validate the deserialized value explicitly rather than relying on defaults.
-- Integer fields deserialized from user input and cast with `as` (e.g., `i32 as u32`) can wrap negative values to large positives. Lesson: validate ranges before casting; use `try_into()` with a typed error, or clamp only when the domain explicitly requires saturation.
-- XML config typos (e.g., `"NoncurentDays"` instead of `"NoncurrentDays"`) are silently accepted when `deny_unknown_fields` is absent. Lesson: strict deserialization prevents silent misconfiguration that could cause data loss or unexpected retention behavior.
-
-## Useful Search Seeds
-
-Use these targeted searches when a diff touches security-sensitive code:
-
-```bash
-rg -n "validate_admin_request|check_permissions|AdminAction::|deny_only|is_allowed" rustfs crates
-rg -n "authorize_operation|FtpsDriver|SftpDriver|RETR|MKD|SIZE|MDTM|CreateBucket|GetObject|HeadObject" crates/protocols rustfs
-rg -n "UploadPartCopy|upload_part_copy|CompleteMultipart|PostObject|presign|SignedHeaders|x-amz-copy-source|collect_signed_headers|content-length-range|starts-with" rustfs crates
-rg -n "ListBucketVersions|GetObjectVersion|versionId|VersionId|ExistingObjectTag|ForAllValues|ForAnyValue|POLICY_PLUGIN|opa" rustfs crates
-rg -n "normalize_extract_entry_key|Snowball|auto-extract|PathBuf::join|canonicalize|\\.\\.|x-forwarded-for|x-real-ip|SourceIp" rustfs crates
-rg -n "DEFAULT_SECRET|DEFAULT_ACCESS|TEST_PRIVATE_KEY|rustfs rpc|RUSTFS_RPC_SECRET" rustfs crates
-rg -n "TONIC_RPC_PREFIX|verify_rpc_signature|check_auth|NodeServiceServer|x-rustfs-signature" rustfs crates
-rg -n "debug!|trace!|info!|error!|\\?resp|\\?merged_config|session_token|secret_key" rustfs crates
-rg -n "HashReader|EncryptReader|try_resolve_etag|ETag|SSE|server-side encryption|Access-Control-Allow-Credentials|Origin" rustfs crates
-rg -n "ObjectLock|object_lock|retention|COMPLIANCE|GOVERNANCE|delete_prefix|lifecycle|scanner" rustfs crates
-rg -n "deny_unknown_fields|serde.default|as u32|as usize|as i32" rustfs crates
-```
-
-## Minimum Regression Test Expectations
-
-- Authz fixes: include unauthenticated, valid low-privilege, wrong-action, correct-action, owner, non-owner, and root/admin cases as applicable.
-- Protocol frontend authz fixes: include denied `RETR`, `SIZE`/`MDTM`, `MKD`, bucket probe, and sibling allowed-operation cases; test bucket-policy explicit denies and condition-key denies on each enabled listener against an S3 control, and assert denied paths do not reach storage.
-- IAM fixes: include import/update/list service-account cases with attacker-controlled parent, claims, access key, secret key, and policy.
-- Copy/upload fixes: include cross-bucket, cross-user, source-denied, destination-denied, copy-source-condition, and multipart completion cases.
-- Presigned upload fixes: include a valid presign with extra unsigned tagging, redirect, storage-class, and cross-bucket copy-source headers; require rejection before source or destination storage access, and verify explicitly signed equivalents still work.
-- Version-action fixes: include historical UUID, explicit current version, `null`, range, partNumber, presigned, STS/session, service-account, anonymous bucket-policy, copy source, multipart-copy source, `HeadObject`, and tag read/write cases.
-- Version-write fixes: a `PutObject`-only principal must not replace an existing version through `PutObject` or browser POST `?versionId=`; verify replication-authorized writes and versioned, suspended, and unversioned buckets separately.
-- Policy-condition fixes: include reserved-key header collisions, missing keys, partially overlapping multi-value sets, cyclic variable substitutions, plugin mode, and built-in policy mode.
-- Path fixes: include encoded traversal, absolute path, nested traversal, archive entries with `..`, valid object keys that resemble traversal text but should be rejected, and canonical bucket/prefix boundary checks; test per-key `DeleteObjects` errors and prefix-scoped denials on versioned buckets.
-- Logging fixes: assert redacted output for structs and response bodies that may contain credentials.
-- IAM export fixes: assert exported archives omit plaintext user and service-account secrets unless the format deliberately encrypts or seals them.
-- RPC auth fixes: include captured metadata replay across two concrete methods, stale timestamps, wrong path, wrong method surrogate, wrong secret, and valid same-method calls.
-- Browser/CORS fixes: assert no credentials on reflected/default origins, correct behavior for explicit allowlists, and no same-origin script execution for previewed object content.
-- SSE fixes: inspect stored bytes and verify API metadata, read-back behavior, and on-disk ciphertext together; for ETags, upload equal plaintext under different keys and assert list-only callers cannot derive or correlate the plaintext digest, including multipart objects.
-- Object Lock fixes: include unreadable metadata, fabricated metadata defaults, unparsable config, confirmed absent config, COMPLIANCE/GOVERNANCE retention, lifecycle expiry, scanner sweeps, and force-delete paths; an authorized force-delete must preserve active COMPLIANCE versions. For `PutObjectRetention`, test conflicting header/body values and offset dates against policy bounds and read-back retention.

.config/ci-plan.json

@@ -0,0 +1,16 @@
+{
+  "schema_version": 1,
+  "rollout": "enforced",
+  "core_packages": ["rustfs", "rustfs-ecstore", "rustfs-filemeta", "rustfs-iam", "rustfs-policy"],
+  "connect_packages": [
+    "rustfs-credentials", "rustfs-config", "rustfs-common", "rustfs-crypto",
+    "rustfs-iam", "rustfs-policy", "rustfs-kms", "rustfs-license", "rustfs-security-governance",
+    "rustfs-utils", "rustfs-tls-runtime", "rustfs-trusted-proxies", "rustfs-filemeta",
+    "rustfs-lock", "rustfs-obs", "rustfs-signer", "rustfs-io-metrics", "rustfs-s3-ops"
+  ],
+  "full_paths": [
+    "Cargo.toml", "Cargo.lock", "*/Cargo.toml", "rust-toolchain*", "rustfmt.toml",
+    "clippy.toml", ".cargo/*", ".config/*", ".github/*", "Makefile",
+    "scripts/*", "build.rs", "*.proto"
+  ]
+}

.config/conformance-rulings.toml

@@ -0,0 +1,10 @@
+# Rulings for the gateway conformance lane (.github/workflows/gateway-acceptance.yml).
+#
+# One row per case that fails on the GATEWAY stack for a reviewed reason: `case`; `ledger`, the
+# rustfs/backlog#2684 entry that accepts the difference (R-0xx or K-n); a one-line `reason`; and
+# `failure`, a prefix of the failure the ruling accepts, as the runner's report.json renders it
+# (`[expect/status] at /expect/status: exchange #1, attempt 1: expected status 400, observed 403`).
+# scripts/check_conformance_rulings.py fails the lane on a failed case without a row, on a ruled
+# case whose failures do not start with its `failure`, and on a row whose case no longer fails or
+# no longer exists. The legacy stack is a reference run and takes no rulings: the `rustfs` profile
+# describes the gateway edge, not the legacy one.

.config/e2e-distributed-selection.txt

@@ -1,2 +1,2 @@
-sha256-linux=74a1d6975e2d6dfe01363585cb9bfa3232d79556a154d4f599653fe2ed1ce696
-sha256-darwin=74a1d6975e2d6dfe01363585cb9bfa3232d79556a154d4f599653fe2ed1ce696
+sha256-linux=348fa5eed9ae58962c06b33283ff4f15122cff97bce9b4eb1e3ffcb227264b0b
+sha256-darwin=348fa5eed9ae58962c06b33283ff4f15122cff97bce9b4eb1e3ffcb227264b0b

.config/e2e-full-selection.txt

@@ -1,2 +1,2 @@
-sha256-darwin=b53fe6d46cdffedfad6a83da0dfffa9aec10cd7c8ace53469f129587bc9f4d50
-sha256-linux=6be90f5c3886e9852ac27d51f2d47e6d2fa8bb424e0d03ee931268dcd0020db8
+sha256-darwin=8fef326a3fddd8a548e2cbcc6ea8159f831113595fb003eeebbd9aeb4ac23e48
+sha256-linux=8b4381578377704ac8b1b5156128b83f036ec4aedc9d863fff25df15112420fc

.config/e2e-nightly-selection.txt

@@ -1,2 +1,2 @@
-sha256-darwin=347a0d4e5e547d5388703062ce66a66167a845b6e57ec591ccb4c732a567c68d
-sha256-linux=4f9efa47194279185bb68249e82c788fa147abe944b6c764050e3d47f31e7926
+sha256-darwin=e84e04208f19dc0596b10d4d1b799ce43d6cc431415902f9c5458aa8232bc2d8
+sha256-linux=1f9c7add72c44d1c12ef4ac050bb7917431be657c0c90b29f70c3df56f8bbae3

.config/e2e-protocols-selection.txt

@@ -1,2 +1,2 @@
-sha256-darwin=a2cb67d91eb0d093928371d6e84a84b2dd41b2ba975a28094e81879bd65838ad
-sha256-linux=82db0561cf9b111f11782d647d0859c9d2a44c5c68481b5e39c48505828bd392
+sha256-darwin=dcc076f8a36320d2fe36b4d2f26e2d54c3b9bc2ff7e14c6e90241f4fd8cf4a33
+sha256-linux=229ebc0cbc9b35863600ae60c6fbb548516efe2a2f41f4b5db503b3283c2f64f

.config/e2e-repl-nightly-selection.txt

@@ -1 +1 @@
-sha256=e4ee3898b840a8374cb02c64342cc15f8d7ecd1becaf4b4a2a4657290f9e7ff1
+sha256=4eac269f9a59b2f9be4e7a87ea6aa38454f5ebe10ede35c9b8d4218137d2cd35

.config/e2e-smoke-selection.txt

@@ -1 +1 @@
-sha256=002ec5d973f82288e50a096a2c8b3b38749f62a7a3e169775566874d8ee98b4a
+sha256=061d0a697967d2dc8f472ff885886e069b1cf4aeaa5f230e177b7236017c23b6

.config/ecstore-required-tests.json

@@ -29,7 +29,7 @@
     {
       "invariant": "list-uncommitted-version",
       "suite": "rustfs-filemeta",
-      "name": "metacache::tests::resolve_with_write_quorum_slack_keeps_partial_latest_hidden_during_merge"
+      "name": "metacache::tests::resolve_merge_keeps_the_newest_version_at_listing_quorum"
     },
     {
       "invariant": "minio-object-fixture",
@@ -229,7 +229,7 @@
     {
       "invariant": "list-objects-shared-corpus-orphan-cleanup-requires-authoritative-evidence",
       "suite": "rustfs-ecstore",
-      "name": "store::list_objects::differential_tests::list_objects_shared_corpus_orphan_cleanup_requires_authoritative_evidence"
+      "name": "store::list_objects::differential_tests::list_objects_shared_corpus_orphan_listing_preserves_inputs"
     },
     {
       "invariant": "list-objects-shared-corpus-consumer-drop-joins-real-disk-producer",
@@ -241,11 +241,6 @@
       "suite": "rustfs-ecstore",
       "name": "store::list_objects::differential_tests::list_objects_shared_corpus_cancelled_quorum_walk_releases_senders_and_disk_lock"
     },
-    {
-      "invariant": "list-objects-shared-corpus-enabled-index-contracts",
-      "suite": "rustfs-ecstore",
-      "name": "store::list_objects::differential_tests::list_objects_shared_corpus_enabled_index_contracts"
-    },
     {
       "invariant": "borrowed-context-boundary",
       "suite": "rustfs-ecstore",

.config/make/lint-fmt.mak

@@ -70,6 +70,26 @@ s3s-footprint-check: ## Check the s3s dependency footprint ratchet stays frozen
 	@echo "📦 Checking s3s footprint ratchet..."
 	./scripts/check_s3s_footprint.sh
 
+.PHONY: release-feature-boundary-check
+release-feature-boundary-check: ## Check no release-reachable feature enables a debug-only one
+	@echo "🚧 Checking release feature boundaries..."
+	$(RUSTFS_PYTHON_BIN) ./scripts/check_connect_release_boundary.py --features-only
+
+.PHONY: test-count-ratchet-check
+test-count-ratchet-check: ## Check no workspace crate fell below its test floor or rose above its #[ignore] ceiling in the committed baseline
+	@echo "🧪 Checking per-crate test-count ratchet..."
+	./scripts/check_test_count_ratchet.sh
+
+.PHONY: test-seams-check
+test-seams-check: ## Check tests outside an operation's owner call its use-case only through rustfs/src/app/test_seams
+	@echo "🧪 Checking cross-owner test seams..."
+	$(RUSTFS_PYTHON_BIN) ./scripts/check_test_seams.py
+
+.PHONY: test-retirements-check
+test-retirements-check: ## Check every test retired by a migration slice names a replacement that exists
+	@echo "🧪 Checking test retirement tables..."
+	$(RUSTFS_PYTHON_BIN) ./scripts/check_test_retirements.py
+
 .PHONY: fips-wording-check
 fips-wording-check: ## Check docs and crates/kms do not over-claim crypto capabilities
 	@echo "📣 Checking cryptographic capability wording guard..."
@@ -90,6 +110,7 @@ test-wiring-check: ## Check tests stay registered and selected by their intended
 	@echo "🧪 Checking test wiring..."
 	$(RUSTFS_PYTHON_BIN) ./scripts/check_test_wiring.py
 	$(RUSTFS_PYTHON_BIN) ./scripts/ci_gate.py --check-workflow
+	$(RUSTFS_PYTHON_BIN) ./scripts/validation_registry.py check
 
 .PHONY: log-analyzer-rules-check
 log-analyzer-rules-check: core-deps ## Check log-analyzer rule anchors still exist verbatim in source

.config/make/pre-commit.mak

@@ -20,11 +20,11 @@ planning-docs-check: ## Check that no planning-type documents are committed
 	./scripts/check_no_planning_docs.sh
 
 .PHONY: pre-commit
-pre-commit: fmt-check unsafe-code-check architecture-migration-check logging-guardrails-check error-other-ratchet-check tokio-io-uring-check extension-schema-check body-cache-whitelist-check s3s-footprint-check fips-wording-check embedded-secrets-check test-wiring-check doc-paths-check planning-docs-check quick-check ## Run fast pre-commit checks without clippy/full tests
+pre-commit: fmt-check unsafe-code-check architecture-migration-check logging-guardrails-check error-other-ratchet-check tokio-io-uring-check extension-schema-check body-cache-whitelist-check s3s-footprint-check release-feature-boundary-check test-count-ratchet-check test-seams-check test-retirements-check fips-wording-check embedded-secrets-check test-wiring-check doc-paths-check planning-docs-check quick-check ## Run fast pre-commit checks without clippy/full tests
 	@echo "✅ All pre-commit checks passed!"
 
 .PHONY: pre-pr
-pre-pr: fmt-check unsafe-code-check architecture-migration-check logging-guardrails-check error-other-ratchet-check tokio-io-uring-check extension-schema-check body-cache-whitelist-check s3s-footprint-check fips-wording-check embedded-secrets-check test-wiring-check doc-paths-check planning-docs-check log-analyzer-rules-check offline-enrollment-e2e-check clippy-check test ## Run full pre-PR checks with clippy and tests
+pre-pr: fmt-check unsafe-code-check architecture-migration-check logging-guardrails-check error-other-ratchet-check tokio-io-uring-check extension-schema-check body-cache-whitelist-check s3s-footprint-check release-feature-boundary-check test-count-ratchet-check test-seams-check test-retirements-check fips-wording-check embedded-secrets-check test-wiring-check doc-paths-check planning-docs-check log-analyzer-rules-check offline-enrollment-e2e-check clippy-check test ## Run full pre-PR checks with clippy and tests
 	@echo "✅ All pre-PR checks passed!"
 
 .PHONY: dev-check

.config/make/tests.mak

@@ -22,13 +22,23 @@ TEST_THREADS ?= 1
 # differ from CI and .config/nextest.toml test-groups will NOT apply.
 
 .PHONY: script-tests
+.PHONY: release-package-tests
+release-package-tests: ## Run release package integration on Linux with fpm, dpkg and rpm installed
+	$(RUSTFS_PYTHON_BIN) ./scripts/test_release_packages.py
+
+.PHONY: release-image-tests
+release-image-tests: ## Run OCI integration with Docker, Go and skopeo; set RELEASE_IMAGE_VARIANT and RELEASE_IMAGE_OUTPUT
+	$(RUSTFS_PYTHON_BIN) ./scripts/test_release_images.py
+
 script-tests: ## Run shell script tests
 	@echo "Running script tests..."
 	trap 'status=$$?; printf "ERROR: script-tests failed (exit %s): %s\n" "$$status" "$$BASH_COMMAND" >&2; exit "$$status"' ERR
 	./scripts/test_build_rustfs_options.sh
 	./scripts/test_docker_runtime_timezone.sh
 	./scripts/test_entrypoint_credentials.sh
+	./scripts/test_entrypoint_directories.sh
 	./scripts/test_internode_grpc_ab_bench.sh
+	$(RUSTFS_PYTHON_BIN) ./scripts/test_internode_grpc_window.py
 	./scripts/test_object_batch_bench_enhanced.sh
 	./scripts/test_package_service_scripts.sh
 	./scripts/test_hotpath_warp_ab_gate.sh
@@ -49,23 +59,54 @@ script-tests: ## Run shell script tests
 	./scripts/test_manual_transition_runbooks.sh
 	./scripts/test_fuzz_runner.sh
 	./scripts/test_python_bin.sh
+	$(RUSTFS_PYTHON_BIN) ./scripts/test_check_doc_paths.py
 	./scripts/check_embedded_secrets.sh --self-test
+	./scripts/check_s3s_footprint.sh --self-test
+	./scripts/test_check_test_count_ratchet.sh
+	./scripts/test_guard_source_set.sh
+	./scripts/test_split_guard_rules.sh
 	$(RUSTFS_PYTHON_BIN) ./scripts/check_test_wiring.py --self-test
+	$(RUSTFS_PYTHON_BIN) ./scripts/codemods/s3_error_to_s3_types.py --self-test
+	$(RUSTFS_PYTHON_BIN) ./scripts/codemods/split_bucket_usecase.py --self-test
+	$(RUSTFS_PYTHON_BIN) ./scripts/codemods/split_ecfs_legacy_edge.py --self-test
+	$(RUSTFS_PYTHON_BIN) ./scripts/codemods/split_lifecycle_transition_api_test.py --self-test
+	$(RUSTFS_PYTHON_BIN) ./scripts/check_test_seams.py --self-test
+	$(RUSTFS_PYTHON_BIN) ./scripts/check_test_retirements.py --self-test
 	$(RUSTFS_PYTHON_BIN) ./scripts/test_e2e_binary.py
+	$(RUSTFS_PYTHON_BIN) ./scripts/test_download_release_asset.py
+	$(RUSTFS_PYTHON_BIN) ./scripts/test_reference_binary.py
+	$(RUSTFS_PYTHON_BIN) ./scripts/test_inspect_btf_prototypes.py
 	$(RUSTFS_PYTHON_BIN) ./scripts/test_release_build_workflow.py
+	$(RUSTFS_PYTHON_BIN) -m unittest discover -s scripts/release -p "test_*.py"
 	$(RUSTFS_PYTHON_BIN) ./scripts/test_release_glibc.py
 	$(RUSTFS_PYTHON_BIN) ./scripts/test_upgrade_workflow.py
 	$(RUSTFS_PYTHON_BIN) ./scripts/test_migration_gate_evidence.py
 	$(RUSTFS_PYTHON_BIN) ./scripts/ci_gate.py --self-test
+	$(RUSTFS_PYTHON_BIN) ./scripts/check_connect_release_boundary.py --self-test
+	$(RUSTFS_PYTHON_BIN) ./scripts/test_ci_test_artifact.py
+	$(RUSTFS_PYTHON_BIN) ./scripts/test_ci_merge_group_proof.py
+	$(RUSTFS_PYTHON_BIN) ./scripts/test_ci_merge_queue.py
+	$(RUSTFS_PYTHON_BIN) -m unittest discover -s scripts/ci -p 'test_*.py'
 	$(RUSTFS_PYTHON_BIN) ./scripts/check_security_coverage.py --self-test
 	$(RUSTFS_PYTHON_BIN) ./scripts/check_scheduled_validation_freshness.py --self-test
 	$(RUSTFS_PYTHON_BIN) ./scripts/test_security_workflow.py
 	$(RUSTFS_PYTHON_BIN) ./scripts/test_nightly_candidate.py
 	$(RUSTFS_PYTHON_BIN) ./scripts/test_functional_chain.py
 	$(RUSTFS_PYTHON_BIN) ./scripts/test_functional_chain_health.py
+	$(RUSTFS_PYTHON_BIN) ./scripts/test_repro_rejoin_readiness_docker.py
+	$(RUSTFS_PYTHON_BIN) ./scripts/test_validation_registry.py
+	$(RUSTFS_PYTHON_BIN) ./scripts/test_inspect_validation_runner.py
+	$(RUSTFS_PYTHON_BIN) ./scripts/test_retain_validation_evidence.py
 	$(RUSTFS_PYTHON_BIN) ./scripts/test_ci_timing_report.py
+	$(RUSTFS_PYTHON_BIN) ./scripts/test_junit_diff.py
+	$(RUSTFS_PYTHON_BIN) ./scripts/test_dual_stack.py
+	$(RUSTFS_PYTHON_BIN) ./scripts/test_mint_stack_diff.py
+	$(RUSTFS_PYTHON_BIN) ./scripts/check_conformance_rulings.py --self-test
+	$(RUSTFS_PYTHON_BIN) ./scripts/compat_matrix_report.py --self-test
 	$(RUSTFS_PYTHON_BIN) ./scripts/s3-tests/test_report_compat.py
 	$(RUSTFS_PYTHON_BIN) ./scripts/s3-tests/test_runner_tools.py
+	$(RUSTFS_PYTHON_BIN) ./scripts/s3-tests/test_api_coverage.py
+	$(RUSTFS_PYTHON_BIN) ./scripts/s3-tests/test_reference_parity.py
 	bash -n ./scripts/validate_object_data_cache_cold_stampede.sh
 	$(RUSTFS_PYTHON_BIN) ./scripts/check_object_data_cache_follower_samples.py --self-test
 	./scripts/validate_object_data_cache_cold_stampede.sh --self-test

.config/nextest.toml

@@ -57,6 +57,11 @@ command = ['sh', '-c', 'echo RUST_MIN_STACK=4194304 >> "$NEXTEST_ENV"']
 [scripts.setup.ecstore-large-stack]
 command = ['sh', '-c', 'echo RUST_MIN_STACK=33554432 >> "$NEXTEST_ENV"']
 
+# The receipt corruption oracle does not simulate a crash. Prefer tmpfs only
+# when its full fixture fits; retain the ordinary temporary directory otherwise.
+[scripts.setup.ecstore-receipt-tmpdir]
+command = ['sh', 'scripts/ci/receipt-fixture-tmpdir.sh']
+
 # The serial ILM selection builds the same deep storage futures in both the
 # lifecycle transition module and scanner integration binary. Different tests
 # in each have overflowed first across otherwise unrelated CI runs.
@@ -68,6 +73,17 @@ command = ['sh', '-c', 'echo RUST_MIN_STACK=33554432 >> "$NEXTEST_ENV"']
 # boundaries. Reserve the run's capacity so unrelated fixtures cannot starve
 # their existing deadlines and typed-failure assertions (rustfs/backlog#2706).
 # Keep each test's internal concurrency and zero-retry policy unchanged.
+[[profile.default.overrides]]
+filter = """
+package(rustfs-ecstore) & (
+    test(/^set_disk::ops::multipart::tests::complete_multipart_(quorum_success_releases_logical_guards_before_physical_tail|waits_for_tail_before_releasing_guards_and_marking_capacity)$/)
+    | test(=store::heal::tests::full_format_heal_lost_after_last_save_does_not_publish_or_renew_and_latches)
+    | test(/^store::init::tests::(decommission_durable_ilm_receipt_pagination_fails_closed_on_second_page|dispatch_manifest_shutdown_(drains_admitted_delete_batch_without_tail_admission|stops_aborting_validation_tail_reads|stops_authorized_tail_reads_before_receipt_or_cas))$/)
+    | (binary(metadata_read_lock_reuse_test) & test(=replica_multipart_completes_while_metadata_writer_waits_for_its_snapshot))
+)
+"""
+threads-required = "num-test-threads"
+
 [[profile.default.overrides]]
 filter = 'package(rustfs) & test(/^connect::diagnostics::schedule::tests::/)'
 threads-required = "num-test-threads"
@@ -80,7 +96,12 @@ threads-required = "num-test-threads"
 # acquisition budget. Reserve capacity from sibling test processes while
 # preserving the probes' internal concurrency, deadlines, and assertions.
 [[profile.default.overrides]]
-filter = '(package(rustfs-ecstore) & test(=core::pools_test::decommission_lock_order_tests::active_decommission_migration_does_not_invert_capacity_and_delete_object_tag_locks)) | (package(rustfs-heal) & test(=heal::mrf_queue::partial_write::tests::durable_retry_submission_error_keeps_checkpoint_and_uses_backoff))'
+filter = '(package(rustfs-ecstore) & (test(/^core::pools_test::decommission_lock_order_tests::(active_decommission_migration_does_not_invert_capacity_and_delete_object(_tag)?_locks|decommission_moves_a_transitioned_version_expired_by_a_delete_all_rule_without_stalling)$/) | test(=services::integrity::tests::audit_migration_and_ack_recovery_preserve_source_and_reject_conflicts))) | (package(rustfs-heal) & test(=heal::mrf_queue::partial_write::tests::durable_retry_submission_error_keeps_checkpoint_and_uses_backoff))'
+threads-required = "num-test-threads"
+
+# These eight-disk namespace-owner probes overlap durable cleanup with a lock contender. Reserve capacity so sibling disk fixtures cannot consume their lock and completion budgets.
+[[profile.default.overrides]]
+filter = 'package(rustfs-ecstore) & test(/^store::object::tests::(canceled_create_retains_usage_cleanup_namespace_owner|batch_delete_excludes_recovery_on_nonzero_pool|delete_excludes_recovery_with_(disjoint|identical|overlapping)_pool_lock_clients)$/)'
 threads-required = "num-test-threads"
 
 # These eight-disk decommission fixtures exercise concurrent retries,
@@ -94,7 +115,7 @@ threads-required = "num-test-threads"
 # Reserve capacity so sibling disk-heavy tests cannot consume its 2s
 # registration budget; retain the queued backup and deadlock assertions.
 [[profile.default.overrides]]
-filter = 'package(rustfs) & test(=admin::handlers::table_catalog::tests::cold_strong_registration_completes_while_other_bucket_backup_waits)'
+filter = 'package(rustfs) & test(=admin::handlers::table_catalog::tests::cold_object_registration_completes_while_other_bucket_backup_waits)'
 threads-required = "num-test-threads"
 
 [[profile.default.overrides]]
@@ -117,11 +138,11 @@ setup = 'ecstore-large-stack'
 # path on a real store; measured with RUST_MIN_STACK on 2026-10-15 they need
 # 2016-2040 KiB of the 2 MiB libtest default and overflowed on Linux CI.
 [[profile.default.scripts]]
-filter = '(package(rustfs) & test(/^(admin::handlers::replication::target_repair_tests::[a-z_]+|admin::handlers::site_replication::tests::durable_iam_state_apply_iam_item_rejects_a_stale_recreate_after_a_replicated_delete|admin::service::federated_identity::tests::(binding_keeps_explicit_iam_mapping_strict|markerless_legacy_oidc_session_crosses_auth_and_iam_deletion_boundaries)|storage::scanner_dirty_journal::tests::(clear_callback_keeps_unknown_coverage_until_all_pending_work_is_confirmed|owned_journals_keep_peer_records_after_interleaved_writes_and_clear))$/)) | (package(rustfs-heal) & ((binary(heal_b5_versioned_regression_test) & test(/^serial_tests::(test_heal_unversioned_bucket_e2e|test_historical_null_recursive_heal_restores_physical_versions|test_recursive_deep_heal_reports_stale_delete_marker_repair)$/)) | (binary(usage_observation_read_repair_test) & test(=usage_observation_read_repair_survives_snapshot_cleanup)))) | (package(rustfs-test-utils) & test(=data_usage_snapshot_tests::data_usage_snapshot_recovery_wires_primary_backup_and_negative_cache))'
+filter = '(package(rustfs) & test(/^(admin::handlers::replication::target_repair_tests::[a-z_]+|admin::handlers::plugin_attempt::v2_tests::delegation_v2_persists_and_vends_with_real_parent_intersection|admin::handlers::site_replication::tests::durable_iam_state_apply_iam_item_rejects_a_stale_recreate_after_a_replicated_delete|admin::service::federated_identity::tests::(binding_keeps_explicit_iam_mapping_strict|markerless_legacy_oidc_session_crosses_auth_and_iam_deletion_boundaries)|storage::scanner_dirty_journal::tests::(clear_callback_keeps_unknown_coverage_until_all_pending_work_is_confirmed|owned_journals_keep_peer_records_after_interleaved_writes_and_clear))$/)) | (package(rustfs-heal) & ((binary(heal_b5_versioned_regression_test) & test(/^serial_tests::(test_heal_unversioned_bucket_e2e|test_historical_null_recursive_heal_restores_physical_versions|test_recursive_deep_heal_reports_stale_delete_marker_repair)$/)) | (binary(usage_observation_read_repair_test) & test(=usage_observation_read_repair_survives_snapshot_cleanup)))) | (package(rustfs-test-utils) & test(=data_usage_snapshot_tests::data_usage_snapshot_recovery_wires_primary_backup_and_negative_cache))'
 setup = 'ecstore-base-stack'
 
 [[profile.default.scripts]]
-filter = 'package(rustfs-ecstore) | package(rustfs-s3select-api) | package(rustfs-scanner) | (package(rustfs) & test(/^(app::multipart_usecase::tests::concurrent_completions_share_durable_bucket_quota_reservations|app::object::delete::tests::compressed_delete_requests_update_observed_usage_without_releasing_quota_floor|app::object::internal_put::tests::internal_multipart_roundtrip_completes_and_abort_leaves_nothing|app::object::restore::tests::execute_restore_object_maps_failures_to_typed_s3_errors|storage::access::tests::(delete_object_access_captures_authorized_bucket_incarnation|copy_operations_reject_recreated_source_bucket_after_authorization|request_slot_keeps_bucket_policy_bound_to_its_store))$/))'
+filter = 'package(rustfs-ecstore) | package(rustfs-s3select-api) | package(rustfs-scanner) | (package(rustfs) & test(/^(app::multipart_usecase::complete::tests::concurrent_completions_share_durable_bucket_quota_reservations|app::object::delete::tests::compressed_delete_requests_update_observed_usage_without_releasing_quota_floor|app::object::internal_put::tests::internal_multipart_roundtrip_completes_and_abort_leaves_nothing|app::object::restore::tests::execute_restore_object_maps_failures_to_typed_s3_errors|storage::access::tests::(delete_object_access_captures_authorized_bucket_incarnation|copy_operations_reject_recreated_source_bucket_after_authorization|request_slot_keeps_bucket_policy_bound_to_its_store))$/))'
 setup = 'ecstore-base-stack'
 
 # These handler/store fixtures and the relocated-pool GET compose deep storage
@@ -134,10 +155,16 @@ setup = 'ecstore-base-stack'
 filter = 'binary(lifecycle_integration_test) | (package(rustfs) & test(/^app::lifecycle_transition_api_test::/))'
 setup = 'lifecycle-large-stack'
 
-# The pool metadata placement regression composes the full erasure-heal
-# startup future and overflows the default Linux test-thread stack.
+# This conditional GET fixture composes real-store futures that overflowed
+# the default Linux test-thread stack.
 [[profile.default.scripts]]
-filter = 'package(rustfs-heal) & test(=replacement_pool_metadata_follows_real_two_set_placement)'
+filter = 'package(rustfs) & test(=app::object::get::tests::conditional_get_304_uses_metadata_without_decoding_damaged_data)'
+setup = 'ecstore-large-stack'
+
+# These heal fixtures compose deep real-store recovery futures that can
+# overflow the default Linux test-thread stack.
+[[profile.default.scripts]]
+filter = 'package(rustfs-heal) & (test(=replacement_pool_metadata_follows_real_two_set_placement) | (binary(heal_integration_test) & test(=bucket_removal_recovers_delete_proofs_before_removing_empty_volumes)))'
 setup = 'ecstore-large-stack'
 
 # These real-disk scanner fixtures assert quiet-workload publication and
@@ -212,7 +239,7 @@ test-group = 'embedded-test-ports'
 # processes cannot exhaust those budgets. Keep deadlines and assertions intact.
 # The short HTTP fixtures are tracked in rustfs/backlog#2714.
 [[profile.default.overrides]]
-filter = 'package(rustfs) & (binary(/^connect_registration$/) | binary(connect_inventory) | binary(connect_perf_drive) | (binary(connect_perf_client) & test(=configured_proxy_and_direct_endpoint_failures_are_distinct)) | (binary(connect_perf_object) & test(/^(real_rustfs_endpoint_and_production_cli_support_bounded_get_and_put|cleanup_uses_a_budget_independent_of_the_measurement_window|oversized_response_is_rejected_without_buffering_it)$/)))'
+filter = 'package(rustfs) & (binary(/^connect_registration$/) | binary(connect_inventory) | binary(connect_perf_drive) | (binary(connect_perf_client) & test(=configured_proxy_and_direct_endpoint_failures_are_distinct)) | (binary(connect_perf_object) & test(/^(real_rustfs_endpoint_and_production_cli_support_bounded_get_and_put|cleanup_uses_a_budget_independent_of_the_measurement_window|cleanup_accepts_missing_object_and_still_deletes_the_bucket|cleanup_diagnostics_identify_each_disconnected_transport_stage|cleanup_diagnostics_identify_each_failed_http_stage_without_private_context|cleanup_diagnostics_identify_timeouts_without_resetting_the_shared_deadline|oversized_response_is_rejected_without_buffering_it)$/)))'
 threads-required = "num-test-threads"
 
 # This cancellation probe hashes the serving executable before acquiring its
@@ -334,6 +361,12 @@ threads-required = "num-test-threads"
 filter = 'package(rustfs-ecstore) & test(=store::init::tests::dispatch_manifest_rollback_bounded_concurrency_reaches_tail_behind_slow_member)'
 threads-required = "num-test-threads"
 
+# These short-deadline real-disk progress probes need the test run's capacity
+# for scanner and object-lock handoffs; keep their assertions and deadlines.
+[[profile.default.overrides]]
+filter = '(package(rustfs-scanner) & (test(=scanner::tests::cycle_persist_failure::later_cycle_post_commit_failure_reloads_durable_state) | test(=scanner::tests::running_main_loop_catches_up_pause_cleared_after_startup_observe))) | (package(rustfs-ecstore) & test(=store::init::tests::restored_transitioned_delete_causally_enqueues_free_version))'
+threads-required = "num-test-threads"
+
 # ---------------------------------------------------------------------------
 # ci profile — the strict CI gate (ci.yml `cargo nextest run --profile ci`)
 # ---------------------------------------------------------------------------
@@ -351,6 +384,17 @@ path = "junit.xml"
 
 # Match the local durable-progress reservations. All members, deadlines,
 # assertions, and internal concurrency remain enabled without retries.
+[[profile.ci.overrides]]
+filter = """
+package(rustfs-ecstore) & (
+    test(/^set_disk::ops::multipart::tests::complete_multipart_(quorum_success_releases_logical_guards_before_physical_tail|waits_for_tail_before_releasing_guards_and_marking_capacity)$/)
+    | test(=store::heal::tests::full_format_heal_lost_after_last_save_does_not_publish_or_renew_and_latches)
+    | test(/^store::init::tests::(decommission_durable_ilm_receipt_pagination_fails_closed_on_second_page|dispatch_manifest_shutdown_(drains_admitted_delete_batch_without_tail_admission|stops_aborting_validation_tail_reads|stops_authorized_tail_reads_before_receipt_or_cas))$/)
+    | (binary(metadata_read_lock_reuse_test) & test(=replica_multipart_completes_while_metadata_writer_waits_for_its_snapshot))
+)
+"""
+threads-required = "num-test-threads"
+
 [[profile.ci.overrides]]
 filter = 'package(rustfs) & test(/^connect::diagnostics::schedule::tests::/)'
 threads-required = "num-test-threads"
@@ -363,7 +407,12 @@ threads-required = "num-test-threads"
 # acquisition budget. Reserve capacity from sibling test processes while
 # preserving the probes' internal concurrency, deadlines, and assertions.
 [[profile.ci.overrides]]
-filter = '(package(rustfs-ecstore) & test(=core::pools_test::decommission_lock_order_tests::active_decommission_migration_does_not_invert_capacity_and_delete_object_tag_locks)) | (package(rustfs-heal) & test(=heal::mrf_queue::partial_write::tests::durable_retry_submission_error_keeps_checkpoint_and_uses_backoff))'
+filter = '(package(rustfs-ecstore) & (test(/^core::pools_test::decommission_lock_order_tests::(active_decommission_migration_does_not_invert_capacity_and_delete_object(_tag)?_locks|decommission_moves_a_transitioned_version_expired_by_a_delete_all_rule_without_stalling)$/) | test(=services::integrity::tests::audit_migration_and_ack_recovery_preserve_source_and_reject_conflicts))) | (package(rustfs-heal) & test(=heal::mrf_queue::partial_write::tests::durable_retry_submission_error_keeps_checkpoint_and_uses_backoff))'
+threads-required = "num-test-threads"
+
+# These eight-disk namespace-owner probes overlap durable cleanup with a lock contender. Reserve capacity so sibling disk fixtures cannot consume their lock and completion budgets.
+[[profile.ci.overrides]]
+filter = 'package(rustfs-ecstore) & test(/^store::object::tests::(canceled_create_retains_usage_cleanup_namespace_owner|batch_delete_excludes_recovery_on_nonzero_pool|delete_excludes_recovery_with_(disjoint|identical|overlapping)_pool_lock_clients)$/)'
 threads-required = "num-test-threads"
 
 # These eight-disk decommission fixtures exercise concurrent retries,
@@ -377,7 +426,7 @@ threads-required = "num-test-threads"
 # Reserve capacity so sibling disk-heavy tests cannot consume its 2s
 # registration budget; retain the queued backup and deadlock assertions.
 [[profile.ci.overrides]]
-filter = 'package(rustfs) & test(=admin::handlers::table_catalog::tests::cold_strong_registration_completes_while_other_bucket_backup_waits)'
+filter = 'package(rustfs) & test(=admin::handlers::table_catalog::tests::cold_object_registration_completes_while_other_bucket_backup_waits)'
 threads-required = "num-test-threads"
 
 [[profile.ci.overrides]]
@@ -391,17 +440,32 @@ threads-required = "num-test-threads"
 filter = 'package(rustfs-scanner) & (test(/^scanner::backlog::tests::native_/) | test(=scanner_folder::tests::mrf_ownership::mrf_ownership_manager_completion_preserves_scanner_pending) | test(/^scanner_io::tests::service_cohort::/))'
 threads-required = "num-test-threads"
 
+# Keep short-deadline real-disk progress probes from competing with unrelated
+# tests for scanner and object-lock capacity. Assertions and deadlines stay on.
+[[profile.ci.overrides]]
+filter = '(package(rustfs-scanner) & (test(=scanner::tests::cycle_persist_failure::later_cycle_post_commit_failure_reloads_durable_state) | test(=scanner::tests::running_main_loop_catches_up_pause_cleared_after_startup_observe))) | (package(rustfs-ecstore) & test(=store::init::tests::restored_transitioned_delete_causally_enqueues_free_version))'
+threads-required = "num-test-threads"
+
+# Emit this expensive fixture's stage timings in CI and use tmpfs when safe.
+[[profile.ci.overrides]]
+filter = 'package(rustfs-ecstore) & test(=store::init::tests::decommission_durable_ilm_receipt_pagination_fails_closed_on_second_page)'
+success-output = 'immediate'
+
+[[profile.ci.scripts]]
+filter = 'package(rustfs-ecstore) & test(=store::init::tests::decommission_durable_ilm_receipt_pagination_fails_closed_on_second_page)'
+setup = 'ecstore-receipt-tmpdir'
+
 [[profile.ci.scripts]]
 filter = 'package(rustfs-ecstore) & test(/^(bucket::lifecycle::bucket_lifecycle_ops::tests::manual_transition_worker_result_recovery_marks_unknown_for_corrupt_marker|services::rebalance::entry::tests::real_rebalance_run_fence_loss_blocks_multipart_publication|store::init::tests::(batch_transitioned_delete_uses_free_version_per_item|decommission_entry_(allows_free_version_consumed_before_source_lock|rejects_subquorum_free_version_conflict_and_retains_source|skips_cleanup_only_marker_when_free_version_is_present)|dispatched_tier_delete_recovery_(checks_later_pool_then_commits_after_source_removal|finds_directory_source_on_encoded_set|retains_journal_on_source_metadata_error)|force_tier_remove_blocks_on_physical_free_version_hidden_by_other_pool|legacy_unknown_transition_delete_falls_back_for_single_batch_and_blocks_prefix|multi_pool_(recursive_prefix_rejects_legacy_or_hidden_merge_loser_before_delete|same_remote_tuple_(batch|single)_delete_waits_for_all_sources|same_tuple_recursive_prefix_uses_one_journal_owner|transitioned_delete_persists_one_free_version_per_remote_tuple)|recursive_prefix_partial_(pool|set)_failure_keeps_prepared_cleanup_owners|restored_transitioned_delete_uses_free_version_as_cleanup_owner|stable_transitioned_recursive_prefix_delete_uses_journal_owners|suspended_null_transition_delete_uses_free_version_as_sole_owner|tier_mutation_peer_handler_applies_prepare_commit_and_abort_idempotently|transition_response_loss_persists_unknown_outcome_for_provider_recovery|transition_transaction_recovery_(drops_record_after_confirmed_local_commit|keeps_cleanup_pending_local_commit)|transitioned_delete_(free_version_replays_after_store_restart|local_quorum_failure_rolls_back_without_cleanup_owner|uses_free_version_as_cleanup_owner)|versioned_delete_marker_keeps_transitioned_source_and_remote_object|versioned_explicit_transition_delete_preserves_other_version_then_allows_bucket_delete))$/)'
 setup = 'ecstore-large-stack'
 
 # These cross-crate real-storage fixtures need the same bounded stack as ECStore tests.
 [[profile.ci.scripts]]
-filter = '(package(rustfs) & test(/^(admin::handlers::replication::target_repair_tests::[a-z_]+|admin::handlers::site_replication::tests::durable_iam_state_apply_iam_item_rejects_a_stale_recreate_after_a_replicated_delete|admin::service::federated_identity::tests::(binding_keeps_explicit_iam_mapping_strict|markerless_legacy_oidc_session_crosses_auth_and_iam_deletion_boundaries)|storage::scanner_dirty_journal::tests::(clear_callback_keeps_unknown_coverage_until_all_pending_work_is_confirmed|owned_journals_keep_peer_records_after_interleaved_writes_and_clear))$/)) | (package(rustfs-heal) & ((binary(heal_b5_versioned_regression_test) & test(/^serial_tests::(test_heal_unversioned_bucket_e2e|test_historical_null_recursive_heal_restores_physical_versions|test_recursive_deep_heal_reports_stale_delete_marker_repair)$/)) | (binary(usage_observation_read_repair_test) & test(=usage_observation_read_repair_survives_snapshot_cleanup)))) | (package(rustfs-test-utils) & test(=data_usage_snapshot_tests::data_usage_snapshot_recovery_wires_primary_backup_and_negative_cache))'
+filter = '(package(rustfs) & test(/^(admin::handlers::replication::target_repair_tests::[a-z_]+|admin::handlers::plugin_attempt::v2_tests::delegation_v2_persists_and_vends_with_real_parent_intersection|admin::handlers::site_replication::tests::durable_iam_state_apply_iam_item_rejects_a_stale_recreate_after_a_replicated_delete|admin::service::federated_identity::tests::(binding_keeps_explicit_iam_mapping_strict|markerless_legacy_oidc_session_crosses_auth_and_iam_deletion_boundaries)|storage::scanner_dirty_journal::tests::(clear_callback_keeps_unknown_coverage_until_all_pending_work_is_confirmed|owned_journals_keep_peer_records_after_interleaved_writes_and_clear))$/)) | (package(rustfs-heal) & ((binary(heal_b5_versioned_regression_test) & test(/^serial_tests::(test_heal_unversioned_bucket_e2e|test_historical_null_recursive_heal_restores_physical_versions|test_recursive_deep_heal_reports_stale_delete_marker_repair)$/)) | (binary(usage_observation_read_repair_test) & test(=usage_observation_read_repair_survives_snapshot_cleanup)))) | (package(rustfs-test-utils) & test(=data_usage_snapshot_tests::data_usage_snapshot_recovery_wires_primary_backup_and_negative_cache))'
 setup = 'ecstore-base-stack'
 
 [[profile.ci.scripts]]
-filter = 'package(rustfs-ecstore) | package(rustfs-s3select-api) | package(rustfs-scanner) | (package(rustfs) & test(/^(app::multipart_usecase::tests::concurrent_completions_share_durable_bucket_quota_reservations|app::object::delete::tests::compressed_delete_requests_update_observed_usage_without_releasing_quota_floor|app::object::internal_put::tests::internal_multipart_roundtrip_completes_and_abort_leaves_nothing|app::object::restore::tests::execute_restore_object_maps_failures_to_typed_s3_errors|storage::access::tests::(delete_object_access_captures_authorized_bucket_incarnation|copy_operations_reject_recreated_source_bucket_after_authorization|request_slot_keeps_bucket_policy_bound_to_its_store))$/))'
+filter = 'package(rustfs-ecstore) | package(rustfs-s3select-api) | package(rustfs-scanner) | (package(rustfs) & test(/^(app::multipart_usecase::complete::tests::concurrent_completions_share_durable_bucket_quota_reservations|app::object::delete::tests::compressed_delete_requests_update_observed_usage_without_releasing_quota_floor|app::object::internal_put::tests::internal_multipart_roundtrip_completes_and_abort_leaves_nothing|app::object::restore::tests::execute_restore_object_maps_failures_to_typed_s3_errors|storage::access::tests::(delete_object_access_captures_authorized_bucket_incarnation|copy_operations_reject_recreated_source_bucket_after_authorization|request_slot_keeps_bucket_policy_bound_to_its_store))$/))'
 setup = 'ecstore-base-stack'
 
 [[profile.ci.scripts]]
@@ -412,10 +476,16 @@ setup = 'ecstore-base-stack'
 filter = 'binary(lifecycle_integration_test) | (package(rustfs) & test(/^app::lifecycle_transition_api_test::/))'
 setup = 'lifecycle-large-stack'
 
-# The pool metadata placement regression composes the full erasure-heal
-# startup future and overflows the default Linux test-thread stack.
+# This conditional GET fixture composes real-store futures that overflowed
+# the default Linux test-thread stack.
 [[profile.ci.scripts]]
-filter = 'package(rustfs-heal) & test(=replacement_pool_metadata_follows_real_two_set_placement)'
+filter = 'package(rustfs) & test(=app::object::get::tests::conditional_get_304_uses_metadata_without_decoding_damaged_data)'
+setup = 'ecstore-large-stack'
+
+# These heal fixtures compose deep real-store recovery futures that can
+# overflow the default Linux test-thread stack.
+[[profile.ci.scripts]]
+filter = 'package(rustfs-heal) & (test(=replacement_pool_metadata_follows_real_two_set_placement) | (binary(heal_integration_test) & test(=bucket_removal_recovers_delete_proofs_before_removing_empty_volumes)))'
 setup = 'ecstore-large-stack'
 
 # These real-disk scanner fixtures assert quiet-workload publication and
@@ -500,7 +570,7 @@ filter = 'package(rustfs) & (binary(/^embedded.*_test$/) | binary(admin_diagnost
 test-group = 'embedded-test-ports'
 
 [[profile.ci.overrides]]
-filter = 'package(rustfs) & (binary(/^connect_registration$/) | binary(connect_inventory) | binary(connect_perf_drive) | (binary(connect_perf_client) & test(=configured_proxy_and_direct_endpoint_failures_are_distinct)) | (binary(connect_perf_object) & test(/^(real_rustfs_endpoint_and_production_cli_support_bounded_get_and_put|cleanup_uses_a_budget_independent_of_the_measurement_window|oversized_response_is_rejected_without_buffering_it)$/)))'
+filter = 'package(rustfs) & (binary(/^connect_registration$/) | binary(connect_inventory) | binary(connect_perf_drive) | (binary(connect_perf_client) & test(=configured_proxy_and_direct_endpoint_failures_are_distinct)) | (binary(connect_perf_object) & test(/^(real_rustfs_endpoint_and_production_cli_support_bounded_get_and_put|cleanup_uses_a_budget_independent_of_the_measurement_window|cleanup_accepts_missing_object_and_still_deletes_the_bucket|cleanup_diagnostics_identify_each_disconnected_transport_stage|cleanup_diagnostics_identify_each_failed_http_stage_without_private_context|cleanup_diagnostics_identify_timeouts_without_resetting_the_shared_deadline|oversized_response_is_rejected_without_buffering_it)$/)))'
 threads-required = "num-test-threads"
 
 # Preserve the cancellation probe's startup budget and assertions in CI too.
@@ -650,14 +720,23 @@ test-group = 'ecstore-serial-flaky'
 # the PR-lane e2e coverage for scanner usage snapshots consumed by quota and
 # admin surfaces. It uses the same single-node, random-port, isolated-temp-dir
 # fixture as the existing smoke modules.
+#
+# Dual-stack differential (rustfs/backlog#2740): `dual_stack_differential::`
+# starts two single-node servers from the same binary, one per
+# RUSTFS_S3_STACK value, on random ports with isolated temp dirs, and runs one
+# SDK script covering every operation of `impl S3 for FS` against both. Its
+# comparator unit tests need no server. It meets the admission criteria
+# unchanged; the module clause admits a new differential test without another
+# filter edit.
 [profile.e2e-smoke]
 default-filter = """
     package(e2e_test) & (
-        test(/^(delete_marker_migration_semantics|version_id_regression|list_objects_v2_pagination|list_object_versions_regression|list_objects_duplicates|list_buckets_double_slash|list_buckets_auth|list_buckets_iam_filter|leading_slash_key|special_chars|create_bucket_region|delete_objects_versioning|head_object_consistency|head_object_range|copy_object_metadata|copy_object_tagging|copy_source_invalid_date|content_encoding|compression|multipart_storage_class|storage_class_capability|ssec_copy|anonymous_access|bucket_policy_check|presigned_negative|negative_sigv4|admin_auth|notification_webhook|tls_hot_reload|console_smoke|admin_iam_crud|admin_pools|sts_query_compat|data_usage)_test::|^fake_s3_target::/)
+        test(/^(delete_marker_migration_semantics|version_id_regression|list_objects_v2_pagination|list_object_versions_regression|list_objects_duplicates|list_buckets_double_slash|list_buckets_auth|list_buckets_iam_filter|leading_slash_key|special_chars|create_bucket_region|delete_objects_versioning|head_object_consistency|head_object_range|copy_object_metadata|copy_object_tagging|copy_source_invalid_date|content_encoding|compression|multipart_storage_class|multipart_upload_id|storage_class_capability|ssec_copy|anonymous_access|bucket_policy_check|presigned_negative|negative_sigv4|admin_auth|notification_webhook|tls_hot_reload|console_smoke|admin_iam_crud|admin_pools|sts_query_compat|data_usage)_test::|^fake_s3_target::/)
         | test(/^replication_extension_test::(test_replication_check_succeeds_with_remote_target|test_replication_check_rejects_target_without_object_lock|test_set_remote_target_rejects_unversioned_source_bucket|test_replication_check_rejects_unversioned_source_bucket|test_replication_check_rejects_missing_replication_config|test_replication_check_rejects_invalid_bucket|test_set_remote_target_rejects_same_bucket_on_same_deployment|test_set_remote_target_rejects_unversioned_target_bucket|test_set_remote_target_update_requires_arn|test_set_remote_target_update_rejects_missing_target|test_set_remote_target_rejects_invalid_target_url|test_set_remote_target_rejects_self_signed_https_target_without_skip_tls_verify|test_set_remote_target_rejects_private_ca_https_target_without_ca_cert_pem|test_list_remote_targets_rejects_empty_bucket|test_list_remote_targets_rejects_invalid_bucket|test_remove_remote_target_rejects_missing_target|test_remove_remote_target_rejects_missing_arn|test_remove_remote_target_rejects_invalid_bucket|test_remove_remote_target_rejects_target_used_by_replication|test_delete_bucket_replication_removes_remote_target)$/)
         | test(/^reliant::lifecycle::/)
         | test(/^conditional_delete_objects_test::/)
         | test(/^reliant::tiering::/)
+        | test(/^dual_stack_differential::/)
         | test(/^upgrade_compatibility_test::upgrade_write_readiness_tests::/)
         | test(/^on_demand_migration::(get_basic_test::(get_miss_pulls_inline_and_serves_locally_afterwards|head_miss_answers_from_the_source_without_persisting)|interaction_test::test_odm_admin_config_is_redacted_and_status_counts_match_the_source)$/)
     )
@@ -779,7 +858,7 @@ slow-timeout = { period = "120s", terminate-after = 12, grace-period = "10s" }
 # ---------------------------------------------------------------------------
 # e2e-distributed profile — 4-node 4-disk Actions suite
 # ---------------------------------------------------------------------------
-# Storage-sensitive PR / nightly / dispatch lane owned by
+# Nightly and manual-dispatch lane owned by
 # .github/workflows/e2e-distributed.yml.
 # Each case starts four rustfs processes (and for site replication, two
 # clusters). Upgrade cases also require RUSTFS_UPGRADE_SOURCE_BINARY.
@@ -877,6 +956,10 @@ path = "junit.xml"
 # parallel-safe — the same property e2e-smoke relies on. The exceptions are the
 # 4-disk reliability / degraded-read fault-injection tests, multi-node heal
 # clusters, and the fixed-port Vault tests, all serialized below.
+# `dual_stack_differential::reference::` holds hermetic unit tests of how the
+# differential picks side A (shell-script stand-ins, no server, no reference
+# binary). They already run on every pull request in e2e-smoke, so e2e-full
+# excludes them to keep its selection digest unchanged.
 [profile.e2e-full]
 default-filter = """
     package(e2e_test)
@@ -887,6 +970,7 @@ default-filter = """
     & !test(/^replication_target_matrix_test::/)
     & !test(/^on_demand_migration::(concurrency_test|fault_test|interop_test|real_source_test)::/)
     & !test(/^upgrade_compatibility_test::upgrade_write_readiness_tests::/)
+    & !test(/^dual_stack_differential::reference::/)
 """
 fail-fast = false
 
@@ -912,6 +996,12 @@ test-group = 'e2e-inline-boundaries'
 filter = 'package(e2e_test) & test(/^heal_erasure_disk_rebuild_test::/)'
 test-group = 'e2e-cluster-nightly'
 
+# The multi-pool crash fixture seeds three nodes and 36 drives. Reserve this
+# nextest run's workers so other multi-node tests do not starve its deadline.
+[[profile.e2e-full.overrides]]
+filter = 'package(e2e_test) & test(=heal_erasure_disk_rebuild_test::tests::test_cluster_root_heal_recovers_ec84_shards_across_multi_pool_after_background_target_crash)'
+threads-required = "num-test-threads"
+
 [[profile.e2e-full.overrides]]
 filter = 'package(e2e_test) & (test(/^kms::kms_vault_test::/) | test(/^kms::kms_rekey_sweep_test::/) | test(/^kms::configured_roundtrip_test::test_configured_vault_kms_admin_and_versioned_cleanup$/))'
 test-group = 'e2e-vault'
@@ -921,3 +1011,23 @@ test-group = 'e2e-vault'
 [[profile.e2e-full.overrides]]
 filter = 'package(e2e_test) & (test(/^list_objects_v2_pagination_test::tests::(test_list_objects_v2_delimiter_small_page_traverses_all|test_list_objects_v2_max_keys_above_limit_returns_token|test_list_objects_v2_maxkeys_above_limit_with_delimiter)$/) | test(=reliant::tiering::test_hermetic_transition_restore_failure_expiry_and_retry))'
 threads-required = "num-test-threads"
+
+# The mc fixture uploads hundreds of objects concurrently before measuring listing.
+# Reserve the runner so other server fixtures do not starve its disk operations.
+[[profile.e2e-full.overrides]]
+filter = 'package(e2e_test) & test(=mc_mirror_small_bucket_test::test_mc_mirror_small_bucket_completes_without_list_timeout)'
+threads-required = "num-test-threads"
+
+# Fixed, small contract for every executable server change. These existing
+# scenarios exercise write/read, anonymous denial and retained bytes on restart.
+[profile.core-contract]
+default-filter = '''
+    package(e2e_test) & test(/^(anonymous_access_test::test_anonymous_access_denied_when_restrict_public_buckets_enabled|head_object_consistency_test::head_object_consistency_after_write_and_multipart_and_presigned_head|storage_class_capability_test::tests::historical_label_only_metadata_is_standard_without_hiding_a_real_transition_tier)$/)
+'''
+retries = 0
+fail-fast = false
+test-threads = 1
+slow-timeout = { period = "60s", terminate-after = 3 }
+
+[profile.core-contract.junit]
+path = "junit.xml"

.config/reference-binary.toml

@@ -0,0 +1,20 @@
+# Pinned RustFS release used as the "reference" target of parity lanes
+# (.github/workflows/reference-parity.yml). This is the single place that names
+# the reference release; .github/actions/reference-binary resolves, downloads and
+# verifies it, and fails on a missing pin, a missing asset or a digest mismatch.
+#
+# Bumping the reference is a reviewed event: every per-test difference between
+# the candidate and the reference is measured against this exact build.
+#
+# Each sha256 is the value published in the release's SHA256SUMS asset, which
+# matches the digest GitHub reports for the same asset. The action checks both.
+
+schema = 1
+repository = "rustfs/rustfs"
+version = "1.0.5"
+tag = "1.0.5"
+
+[platforms.linux-x86_64-gnu]
+asset = "rustfs-linux-x86_64-gnu-v1.0.5.zip"
+sha256 = "5addae2f74756bf80104d0975e6347cda843bc2daffdfcd1f0cb80118b95cbd0"
+executable = "rustfs"

.config/reference-parity-targets.toml

@@ -0,0 +1,24 @@
+# Targets of the s3-tests reference parity lane (.github/workflows/reference-parity.yml).
+#
+# Exactly one target uses `binary = "reference"` (the release pinned in
+# .config/reference-binary.toml). Every other target uses `binary = "candidate"`
+# (the binary built from the branch under test) and is compared per node id
+# against the reference. All targets run the same harness, test list and server
+# configuration; `env` adds only the target's own RUSTFS_* settings.
+#
+# Adding a target is one more [[targets]] table, for example:
+#
+#   [[targets]]
+#   name = "candidate-gateway"
+#   binary = "candidate"
+#   env = { RUSTFS_S3_STACK = "gateway" }
+
+schema = 1
+
+[[targets]]
+name = "reference"
+binary = "reference"
+
+[[targets]]
+name = "candidate-legacy"
+binary = "candidate"

.config/reference-parity-tolerated.txt

@@ -0,0 +1,12 @@
+# Tolerated per-test differences of the s3-tests reference parity lane.
+#
+# One entry per line, four fields separated by " | ":
+#
+#   <target> | <pytest node id> | <tracking issue URL> | <reason>
+#
+# <target> is a candidate name from .config/reference-parity-targets.toml, the
+# node id is exactly as the lane prints it (s3tests/functional/test_s3.py::...),
+# and the issue URL is https://github.com/rustfs/<repo>/issues/<number>.
+#
+# Same semantics as scripts/junit_diff.py --allow: an entry whose difference no
+# longer occurs is stale and fails the lane, so remove it when it is fixed.

.config/release-validation-plan.json

@@ -0,0 +1,135 @@
+{
+  "schema_version": 1,
+  "plan_id": "rustfs-release-validation",
+  "description": "Certification for immutable candidates. All shipped platforms require native identity and startup; stable and prerelease require the pinned upgrade matrix. Linux-only acceptance never authorizes publication.",
+  "release_types": [
+    "preview",
+    "prerelease",
+    "stable"
+  ],
+  "targets": [
+    {
+      "os": "linux",
+      "arch": "x86_64",
+      "abi": "musl"
+    },
+    {
+      "os": "linux",
+      "arch": "aarch64",
+      "abi": "musl"
+    },
+    {
+      "os": "linux",
+      "arch": "x86_64",
+      "abi": "gnu"
+    },
+    {
+      "os": "linux",
+      "arch": "aarch64",
+      "abi": "gnu"
+    },
+    {
+      "os": "macos",
+      "arch": "aarch64",
+      "abi": "none"
+    },
+    {
+      "os": "windows",
+      "arch": "x86_64",
+      "abi": "msvc"
+    }
+  ],
+  "targets_basis": "Platform matrix of .github/workflows/build.yml (prepare-platform-matrix). gnu and macOS targets build with the pyroscope feature; musl and Windows targets build with no extra features.",
+  "features": [
+    "pyroscope"
+  ],
+  "trusted_workflows": [
+    {
+      "repository": "rustfs/rustfs",
+      "workflow_id": "release-candidate.yml"
+    },
+    {
+      "repository": "rustfs/rustfs",
+      "workflow_id": "release-complete.yml"
+    }
+  ],
+  "trusted_workflows_basis": "Receipts are produced by the reviewed main workflows and fetched by exact run ID with GitHub artifact digest verification. Candidate inputs freeze the harness commit separately from product source.",
+  "lanes": [
+    {
+      "lane": "identity",
+      "harness_repository": "rustfs/rustfs",
+      "harness_sha": "candidate.workflow_sha",
+      "harness_basis": "The exact workflow commit is frozen in candidate inputs; no mutable branch is accepted as a harness identity.",
+      "timeout_seconds": 1800,
+      "environment_capabilities": [
+        "process-exec"
+      ],
+      "cases": [
+        "artifact-identity",
+        "startup"
+      ],
+      "applies_to": {
+        "release_types": [
+          "preview",
+          "prerelease",
+          "stable"
+        ],
+        "platforms": "all"
+      },
+      "applies_basis": "Artifact identity and startup checks are mandatory for every release type, preview included, on every shipped target (rustfs/backlog#2662 frozen design item 5)."
+    },
+    {
+      "lane": "upgrade",
+      "harness_repository": "rustfs/rustfs",
+      "harness_sha": "candidate.workflow_sha",
+      "harness_basis": "The exact workflow commit is frozen in candidate inputs; no mutable branch is accepted as a harness identity.",
+      "timeout_seconds": 5400,
+      "environment_capabilities": [
+        "process-exec",
+        "previous-release-artifact"
+      ],
+      "cases": [
+        "direct-upgrade",
+        "mixed-version-upgrade",
+        "bucket-config-upgrade",
+        "bucket-config-rollback",
+        "multipart-layout-upgrade",
+        "multipart-layout-baseline"
+      ],
+      "cases_basis": "Case IDs of the pinned previous-release matrix in .github/workflows/e2e-upgrade.yml (source.version/asset/sha256 and the case list mapped there).",
+      "applies_to": {
+        "release_types": [
+          "prerelease",
+          "stable"
+        ],
+        "platforms": [
+          {
+            "os": "linux",
+            "arch": "x86_64",
+            "abi": "gnu"
+          }
+        ]
+      },
+      "applies_basis": "preview releases use the pre-declared smaller set (identity lane only). The upgrade lane is only exercised on linux-x86_64-gnu, the platform of the pinned source assets."
+    }
+  ],
+  "upgrade_sources": {
+    "basis": "Exactly the two previous-release sources pinned in .github/workflows/e2e-upgrade.yml are exercised today (source.version / source.asset / source.sha256). No other N-1/N-2 compatibility is claimed, and these digests are not inferred from tags.",
+    "sources": [
+      {
+        "version": "1.0.1",
+        "role": "N-1",
+        "artifact_name": "rustfs-linux-x86_64-gnu-v1.0.1.zip",
+        "artifact_sha256": "9742e183cb0dfb0d48fa909797af572fe47b7f827569036baa4156e0d3958ee5"
+      },
+      {
+        "version": "1.0.0-rc.5",
+        "role": "N-2",
+        "artifact_name": "rustfs-linux-x86_64-gnu-v1.0.0-rc.5.zip",
+        "artifact_sha256": "3ee8df71e8edcfada533be452c4135868f697bc515460ae97b027313eade7a3d"
+      }
+    ],
+    "rollback_supported": false,
+    "rollback_basis": "Only the bucket-config-rollback case (rollback_to_previous_release_reads_current_bucket_metadata) is exercised. Full data-plane rollback is not certified and is never a default claim."
+  }
+}

.config/s3s-edge-allowlist.txt

@@ -0,0 +1,37 @@
+# Files that may still reference s3s (`s3s::` or `s3s_sigv4`) when Phase 1 of
+# the gateway migration ends (rustfs/backlog#2734 task T0.6, rustfs/backlog#2741).
+# Consumed only by `scripts/check_s3s_footprint.sh --mode allowlist`.
+#
+# One entry per line: a repository-relative file path or a git `:(glob)`
+# pathspec. `**` matches across directories, `*` stays within one path
+# component, and a directory entry matches everything under it. Blank lines
+# and lines starting with `#` are ignored; an entry may not contain whitespace,
+# so a note belongs on its own comment line.
+#
+# Every Rust file listed by `git grep -l '\bs3s\(::\|_sigv4\)' -- '*.rs'` that
+# no entry matches fails the allowlist mode. Task T1.9 switches CI to that
+# mode; until then `--dry-run` prints the files still outside this list, which
+# is the Phase 1 tracking list.
+#
+# Never add crates/** other than crates/e2e_test: the engine crates leave s3s
+# entirely, and the script rejects such an entry. Only the s3s-facing edge of
+# the rustfs binary and the e2e oracle keep it.
+
+# rustfs binary: the s3s-facing edge that survives Phase 1
+rustfs/src/storage/ecfs.rs
+rustfs/src/storage/s3_compat/**
+rustfs/src/storage/access.rs
+rustfs/src/auth.rs
+rustfs/src/error.rs
+rustfs/src/storage_api.rs
+rustfs/src/app/storage_api.rs
+rustfs/src/admin/router.rs
+rustfs/src/app/metadata_route.rs
+rustfs/src/server/http.rs
+rustfs/src/server/layer.rs
+rustfs/src/server/hybrid.rs
+rustfs/src/protocols/client.rs
+
+# e2e oracle: the test target that verifies S3 behaviour against s3s
+crates/e2e_test/src/fake_s3_target/**
+crates/e2e_test/src/**

.config/s3s-footprint-baseline.txt

@@ -0,0 +1,18 @@
+# Baseline for the direct s3s footprint, consumed only by
+# scripts/check_s3s_footprint.sh (rustfs/backlog#1677; rustfs/backlog#2734 T0.4;
+# comparison rules on rustfs/backlog#2765).
+#
+# Each line is a ceiling everywhere: a counter above it fails, because new code
+# widened the s3s surface; route it through the gateway abstractions instead and
+# never raise a value here. Pull requests and merge groups are additionally held
+# to their base revision's measurement, so a slice that shrinks the surface need
+# not edit this file. A counter below its line passes and the check prints the
+# lowered line; the migration coordinator lowers the lines in batches.
+#
+# files           Rust files referencing s3s paths, repo-wide minus crates/e2e_test
+# s3_error_lines  lines holding s3_error!, same scope, minus those the script
+#                 header's rule clears (rustfs_s3_types's macro); fails closed
+# ecstore_files   Rust files referencing s3s paths under crates/ecstore/src
+files=63
+s3_error_lines=209
+ecstore_files=0

.config/test-count-baseline.txt

@@ -0,0 +1,67 @@
+# Per-crate test-count baseline consumed by scripts/check_test_count_ratchet.sh
+# (rustfs/backlog#2734 section 1 rule 3; T0.3 in rustfs/backlog#2738).
+#
+# One row per workspace member: <package>, tests, ignored, tab-separated.
+# tests counts #[test], #[tokio::test], #[tokio::test(...)] and #[test_case(...)]
+# attribute lines under the package directory; ignored counts #[ignore] and
+# #[ignore = "..."]. The tests column is a floor and the ignored column is a
+# ceiling: CI fails when a crate's tests fall below its row or its ignored count
+# rises above it. Extra tests and fewer ignores pass; --update tightens the rows.
+# Lowering a tests row or raising an ignored row is a reviewed statement that
+# tests were intentionally removed or ignored; justify it in the PR description.
+# Regenerate with:
+#   scripts/check_test_count_ratchet.sh --update
+e2e_test	1026	20
+rustfs	6449	64
+rustfs-audit	70	0
+rustfs-checksums	28	1
+rustfs-common	36	0
+rustfs-concurrency	87	0
+rustfs-config	43	0
+rustfs-connect-symbol-catalog	6	0
+rustfs-credentials	24	0
+rustfs-crypto	67	0
+rustfs-data-usage	58	0
+rustfs-ecstore	7858	2
+rustfs-extension-schema	23	0
+rustfs-filemeta	361	0
+rustfs-gateway-adapter	74	0
+rustfs-heal	1128	3
+rustfs-heal-contracts	8	0
+rustfs-iam	415	0
+rustfs-io-core	52	0
+rustfs-io-metrics	157	0
+rustfs-keystone	31	0
+rustfs-kms	716	25
+rustfs-license	2	0
+rustfs-lifecycle	156	0
+rustfs-lock	182	0
+rustfs-log-analyzer	91	0
+rustfs-madmin	210	0
+rustfs-notify	140	0
+rustfs-object-capacity	92	0
+rustfs-object-data-cache	141	0
+rustfs-obs	387	0
+rustfs-policy	758	0
+rustfs-protocols	770	1
+rustfs-protos	75	0
+rustfs-replication	209	0
+rustfs-rio	225	1
+rustfs-rio-v2	34	6
+rustfs-s3-client	71	0
+rustfs-s3-ops	8	0
+rustfs-s3-types	105	0
+rustfs-s3select-api	232	0
+rustfs-s3select-query	235	2
+rustfs-scanner	1392	20
+rustfs-scanner-metrics	90	0
+rustfs-security-governance	20	0
+rustfs-signer	34	0
+rustfs-storage-api	34	0
+rustfs-targets	612	26
+rustfs-test-utils	1	0
+rustfs-tls-runtime	44	0
+rustfs-trusted-proxies	72	0
+rustfs-utils	253	1
+rustfs-zip	30	0
+zstd	11	0

.config/test-retirements/bucket_access.txt

@@ -0,0 +1,3 @@
+# Tests the bucket_access slice retires (rustfs/backlog#2765); format and checks: scripts/check_test_retirements.py.
+# <crate> | <module path>::<test fn> | <kind: test, differential or e2e> | <replacement> | <reason>
+# The retired test must still exist when its entry is added; the change that deletes it deletes the entry.

.config/test-retirements/bucket_configs.txt

@@ -0,0 +1,3 @@
+# Tests the bucket_configs slice retires (rustfs/backlog#2765); format and checks: scripts/check_test_retirements.py.
+# <crate> | <module path>::<test fn> | <kind: test, differential or e2e> | <replacement> | <reason>
+# The retired test must still exist when its entry is added; the change that deletes it deletes the entry.

.config/test-retirements/bucket_core.txt

@@ -0,0 +1,3 @@
+# Tests the bucket_core slice retires (rustfs/backlog#2765); format and checks: scripts/check_test_retirements.py.
+# <crate> | <module path>::<test fn> | <kind: test, differential or e2e> | <replacement> | <reason>
+# The retired test must still exist when its entry is added; the change that deletes it deletes the entry.

.config/test-retirements/bucket_replication_lifecycle.txt

@@ -0,0 +1,3 @@
+# Tests the bucket_replication_lifecycle slice retires (rustfs/backlog#2765); format and checks: scripts/check_test_retirements.py.
+# <crate> | <module path>::<test fn> | <kind: test, differential or e2e> | <replacement> | <reason>
+# The retired test must still exist when its entry is added; the change that deletes it deletes the entry.

.config/test-retirements/cutover.txt

@@ -0,0 +1,3 @@
+# Tests the cutover slice retires (rustfs/backlog#2765); format and checks: scripts/check_test_retirements.py.
+# <crate> | <module path>::<test fn> | <kind: test, differential or e2e> | <replacement> | <reason>
+# The retired test must still exist when its entry is added; the change that deletes it deletes the entry.

.config/test-retirements/listing_multipart.txt

@@ -0,0 +1,3 @@
+# Tests the listing_multipart slice retires (rustfs/backlog#2765); format and checks: scripts/check_test_retirements.py.
+# <crate> | <module path>::<test fn> | <kind: test, differential or e2e> | <replacement> | <reason>
+# The retired test must still exist when its entry is added; the change that deletes it deletes the entry.

.config/test-retirements/object_commit.txt

@@ -0,0 +1,3 @@
+# Tests the object_commit slice retires (rustfs/backlog#2765); format and checks: scripts/check_test_retirements.py.
+# <crate> | <module path>::<test fn> | <kind: test, differential or e2e> | <replacement> | <reason>
+# The retired test must still exist when its entry is added; the change that deletes it deletes the entry.

.config/test-retirements/object_meta.txt

@@ -0,0 +1,3 @@
+# Tests the object_meta slice retires (rustfs/backlog#2765); format and checks: scripts/check_test_retirements.py.
+# <crate> | <module path>::<test fn> | <kind: test, differential or e2e> | <replacement> | <reason>
+# The retired test must still exist when its entry is added; the change that deletes it deletes the entry.

.config/test-retirements/object_misc.txt

@@ -0,0 +1,3 @@
+# Tests the object_misc slice retires (rustfs/backlog#2765); format and checks: scripts/check_test_retirements.py.
+# <crate> | <module path>::<test fn> | <kind: test, differential or e2e> | <replacement> | <reason>
+# The retired test must still exist when its entry is added; the change that deletes it deletes the entry.

.config/test-retirements/object_read.txt

@@ -0,0 +1,3 @@
+# Tests the object_read slice retires (rustfs/backlog#2765); format and checks: scripts/check_test_retirements.py.
+# <crate> | <module path>::<test fn> | <kind: test, differential or e2e> | <replacement> | <reason>
+# The retired test must still exist when its entry is added; the change that deletes it deletes the entry.

.config/test-retirements/object_write.txt

@@ -0,0 +1,3 @@
+# Tests the object_write slice retires (rustfs/backlog#2765); format and checks: scripts/check_test_retirements.py.
+# <crate> | <module path>::<test fn> | <kind: test, differential or e2e> | <replacement> | <reason>
+# The retired test must still exist when its entry is added; the change that deletes it deletes the entry.

.config/test-retirements/t211.txt

@@ -0,0 +1,3 @@
+# Tests the t211 slice retires (rustfs/backlog#2765); format and checks: scripts/check_test_retirements.py.
+# <crate> | <module path>::<test fn> | <kind: test, differential or e2e> | <replacement> | <reason>
+# The retired test must still exist when its entry is added; the change that deletes it deletes the entry.

.config/test-retirements/t212.txt

@@ -0,0 +1,3 @@
+# Tests the t212 slice retires (rustfs/backlog#2765); format and checks: scripts/check_test_retirements.py.
+# <crate> | <module path>::<test fn> | <kind: test, differential or e2e> | <replacement> | <reason>
+# The retired test must still exist when its entry is added; the change that deletes it deletes the entry.

.docker/README.md

@@ -22,9 +22,9 @@ The following files in the project root are essential for Docker operations:
 | File | Description | Usage |
 | :--- | :--- | :--- |
 | **`docker-buildx.sh`** | **Multi-Arch Build Script**<br>Automates building and pushing Docker images for `amd64` and `arm64`. Supports release and dev channels. | `./docker-buildx.sh --push` |
-| **`Dockerfile`** | **Production Image (Alpine)**<br>Lightweight image using musl libc. Downloads pre-built binaries from GitHub Releases. | `docker build -t rustfs:latest .` |
-| **`Dockerfile.glibc`** | **Production Image (Ubuntu)**<br>Standard image using glibc. Useful if you need specific dynamic libraries. | `docker build -f Dockerfile.glibc .` |
-| **`Dockerfile.source`** | **Development Image**<br>Builds RustFS from source code. Includes build tools. Ideal for local development and CI. | `docker build -f Dockerfile.source .` |
+| **`Dockerfile`** | **Production Image (Alpine)**<br>Lightweight image using musl libc. Downloads verified RustFS and `rc` binaries from GitHub Releases and bundles `/etc/rustfs/bootstrap`. | `docker build -t rustfs:latest .` |
+| **`Dockerfile.glibc`** | **Production Image (Ubuntu)**<br>Standard image using glibc. Includes the bundled bootstrap configuration. | `docker build -f Dockerfile.glibc .` |
+| **`Dockerfile.source`** | **Development Image**<br>Builds RustFS from source code and bundles `rc` and the default bootstrap configuration. | `docker build -f Dockerfile.source .` |
 
 ### Docker Compose Configurations
 
@@ -110,6 +110,42 @@ docker compose up -d
 
 ---
 
+## Docker Compose environment values
+
+Use map syntax when quoting environment values in YAML:
+
+```yaml
+services:
+  rustfs:
+    image: rustfs/rustfs:latest
+    ports:
+      - "9000:9000"
+      - "9001:9001"
+    volumes:
+      - ./data:/data
+      - ./logs:/var/log/rustfs
+    environment:
+      RUSTFS_ADDRESS: ":9000"
+      RUSTFS_CONSOLE_ADDRESS: ":9001"
+      RUSTFS_CONSOLE_ENABLE: "true"
+      RUSTFS_OBS_LOGGER_LEVEL: error
+      RUSTFS_OBS_LOG_DIRECTORY: "/var/log/rustfs/"
+      RUSTFS_ACCESS_KEY: "${RUSTFS_ACCESS_KEY:?Set RUSTFS_ACCESS_KEY}"
+      RUSTFS_SECRET_KEY: "${RUSTFS_SECRET_KEY:?Set RUSTFS_SECRET_KEY}"
+```
+
+Create the host directories and grant the container user write access before starting this example; official images run as UID/GID `10001:10001`. Set both credential variables to your own values, then run `docker compose config` to inspect the resolved configuration locally before `docker compose up -d`. The resolved output includes credentials, so do not share it publicly.
+
+With list syntax, use `- RUSTFS_OBS_LOG_DIRECTORY=/var/log/rustfs/` or quote the whole item as `- "RUSTFS_OBS_LOG_DIRECTORY=/var/log/rustfs/"`. Writing `- RUSTFS_OBS_LOG_DIRECTORY="/var/log/rustfs/"` passes the inner quotes literally, making this a relative path beginning with `"`, which can fail with `mkdir: cannot create directory '"': Permission denied`. This also applies to addresses and credentials: fix every affected list item, rather than only the log directory. The entrypoint preserves configured values, including literal quotes in valid paths and secrets.
+
+---
+
+## Local storage pool expansion
+
+To append a second four-drive pool, set `RUSTFS_VOLUMES: "/data/rustfs0/{0...3} /data/rustfs1/{0...3}"` and keep the original pool's drives, order, and erasure set width. The container prepares each drive directory and passes one argument per pool to RustFS.
+
+If an older image already formatted both ranges together as a single eight-drive pool, list the eight individual paths in their original order to keep that layout when upgrading.
+
 ## 🔧 Common Operations
 
 ### Cleaning Up

.docker/compose/docker-compose.cluster.local-build.yml

@@ -34,6 +34,9 @@ services:
       - RUSTFS_OBJECT_LOCK_ACQUIRE_TIMEOUT=${RUSTFS_OBJECT_LOCK_ACQUIRE_TIMEOUT:-5}
       - RUSTFS_LOCK_ACQUIRE_TIMEOUT=${RUSTFS_LOCK_ACQUIRE_TIMEOUT:-5}
       - RUSTFS_UNSAFE_BYPASS_DISK_CHECK=${RUSTFS_UNSAFE_BYPASS_DISK_CHECK:-true}
+      - RUSTFS_DURABILITY_MODE=${RUSTFS_DURABILITY_MODE:-strict}
+      - RUSTFS_EXPERIMENTAL_FILE_FDATASYNC_GROUP_COMMIT_ENABLE=${RUSTFS_EXPERIMENTAL_FILE_FDATASYNC_GROUP_COMMIT_ENABLE:-false}
+      - RUSTFS_EXPERIMENTAL_FILE_FDATASYNC_GROUP_COMMIT_WAIT_MICROS=${RUSTFS_EXPERIMENTAL_FILE_FDATASYNC_GROUP_COMMIT_WAIT_MICROS:-0}
       # Internode gRPC optimization knobs (grpc-optimization P0-P3). Defaults match the binary
       # defaults, so leaving these unset is a no-op; the A/B driver exports them to toggle a stage.
       - RUSTFS_INTERNODE_RPC_TCP_NODELAY=${RUSTFS_INTERNODE_RPC_TCP_NODELAY:-true}
@@ -43,6 +46,8 @@ services:
       - RUSTFS_INTERNODE_CHANNEL_ISOLATION=${RUSTFS_INTERNODE_CHANNEL_ISOLATION:-false}
       - RUSTFS_INTERNODE_BULK_CHANNELS=${RUSTFS_INTERNODE_BULK_CHANNELS:-2}
       - RUSTFS_INTERNODE_RPC_MSGPACK_ONLY=${RUSTFS_INTERNODE_RPC_MSGPACK_ONLY:-false}
+      - RUSTFS_INTERNODE_RPC_MSGPACK_ONLY_FLEET_CONFIRMED=${RUSTFS_INTERNODE_RPC_MSGPACK_ONLY_FLEET_CONFIRMED:-false}
+      - RUSTFS_OBS_PUT_STAGE_METRICS_ENABLED=${RUSTFS_OBS_PUT_STAGE_METRICS_ENABLED:-false}
       - RUSTFS_INTERNODE_PREWARM=${RUSTFS_INTERNODE_PREWARM:-false}
       - RUSTFS_INTERNODE_OFFLINE_BYPASS=${RUSTFS_INTERNODE_OFFLINE_BYPASS:-false}
       - RUSTFS_INTERNODE_OFFLINE_REPROBE_SECS=${RUSTFS_INTERNODE_OFFLINE_REPROBE_SECS:-5}
@@ -81,6 +86,9 @@ services:
       - RUSTFS_OBJECT_LOCK_ACQUIRE_TIMEOUT=${RUSTFS_OBJECT_LOCK_ACQUIRE_TIMEOUT:-5}
       - RUSTFS_LOCK_ACQUIRE_TIMEOUT=${RUSTFS_LOCK_ACQUIRE_TIMEOUT:-5}
       - RUSTFS_UNSAFE_BYPASS_DISK_CHECK=${RUSTFS_UNSAFE_BYPASS_DISK_CHECK:-true}
+      - RUSTFS_DURABILITY_MODE=${RUSTFS_DURABILITY_MODE:-strict}
+      - RUSTFS_EXPERIMENTAL_FILE_FDATASYNC_GROUP_COMMIT_ENABLE=${RUSTFS_EXPERIMENTAL_FILE_FDATASYNC_GROUP_COMMIT_ENABLE:-false}
+      - RUSTFS_EXPERIMENTAL_FILE_FDATASYNC_GROUP_COMMIT_WAIT_MICROS=${RUSTFS_EXPERIMENTAL_FILE_FDATASYNC_GROUP_COMMIT_WAIT_MICROS:-0}
       # Internode gRPC optimization knobs (grpc-optimization P0-P3). Defaults match the binary
       # defaults, so leaving these unset is a no-op; the A/B driver exports them to toggle a stage.
       - RUSTFS_INTERNODE_RPC_TCP_NODELAY=${RUSTFS_INTERNODE_RPC_TCP_NODELAY:-true}
@@ -90,6 +98,8 @@ services:
       - RUSTFS_INTERNODE_CHANNEL_ISOLATION=${RUSTFS_INTERNODE_CHANNEL_ISOLATION:-false}
       - RUSTFS_INTERNODE_BULK_CHANNELS=${RUSTFS_INTERNODE_BULK_CHANNELS:-2}
       - RUSTFS_INTERNODE_RPC_MSGPACK_ONLY=${RUSTFS_INTERNODE_RPC_MSGPACK_ONLY:-false}
+      - RUSTFS_INTERNODE_RPC_MSGPACK_ONLY_FLEET_CONFIRMED=${RUSTFS_INTERNODE_RPC_MSGPACK_ONLY_FLEET_CONFIRMED:-false}
+      - RUSTFS_OBS_PUT_STAGE_METRICS_ENABLED=${RUSTFS_OBS_PUT_STAGE_METRICS_ENABLED:-false}
       - RUSTFS_INTERNODE_PREWARM=${RUSTFS_INTERNODE_PREWARM:-false}
       - RUSTFS_INTERNODE_OFFLINE_BYPASS=${RUSTFS_INTERNODE_OFFLINE_BYPASS:-false}
       - RUSTFS_INTERNODE_OFFLINE_REPROBE_SECS=${RUSTFS_INTERNODE_OFFLINE_REPROBE_SECS:-5}
@@ -128,6 +138,9 @@ services:
       - RUSTFS_OBJECT_LOCK_ACQUIRE_TIMEOUT=${RUSTFS_OBJECT_LOCK_ACQUIRE_TIMEOUT:-5}
       - RUSTFS_LOCK_ACQUIRE_TIMEOUT=${RUSTFS_LOCK_ACQUIRE_TIMEOUT:-5}
       - RUSTFS_UNSAFE_BYPASS_DISK_CHECK=${RUSTFS_UNSAFE_BYPASS_DISK_CHECK:-true}
+      - RUSTFS_DURABILITY_MODE=${RUSTFS_DURABILITY_MODE:-strict}
+      - RUSTFS_EXPERIMENTAL_FILE_FDATASYNC_GROUP_COMMIT_ENABLE=${RUSTFS_EXPERIMENTAL_FILE_FDATASYNC_GROUP_COMMIT_ENABLE:-false}
+      - RUSTFS_EXPERIMENTAL_FILE_FDATASYNC_GROUP_COMMIT_WAIT_MICROS=${RUSTFS_EXPERIMENTAL_FILE_FDATASYNC_GROUP_COMMIT_WAIT_MICROS:-0}
       # Internode gRPC optimization knobs (grpc-optimization P0-P3). Defaults match the binary
       # defaults, so leaving these unset is a no-op; the A/B driver exports them to toggle a stage.
       - RUSTFS_INTERNODE_RPC_TCP_NODELAY=${RUSTFS_INTERNODE_RPC_TCP_NODELAY:-true}
@@ -137,6 +150,8 @@ services:
       - RUSTFS_INTERNODE_CHANNEL_ISOLATION=${RUSTFS_INTERNODE_CHANNEL_ISOLATION:-false}
       - RUSTFS_INTERNODE_BULK_CHANNELS=${RUSTFS_INTERNODE_BULK_CHANNELS:-2}
       - RUSTFS_INTERNODE_RPC_MSGPACK_ONLY=${RUSTFS_INTERNODE_RPC_MSGPACK_ONLY:-false}
+      - RUSTFS_INTERNODE_RPC_MSGPACK_ONLY_FLEET_CONFIRMED=${RUSTFS_INTERNODE_RPC_MSGPACK_ONLY_FLEET_CONFIRMED:-false}
+      - RUSTFS_OBS_PUT_STAGE_METRICS_ENABLED=${RUSTFS_OBS_PUT_STAGE_METRICS_ENABLED:-false}
       - RUSTFS_INTERNODE_PREWARM=${RUSTFS_INTERNODE_PREWARM:-false}
       - RUSTFS_INTERNODE_OFFLINE_BYPASS=${RUSTFS_INTERNODE_OFFLINE_BYPASS:-false}
       - RUSTFS_INTERNODE_OFFLINE_REPROBE_SECS=${RUSTFS_INTERNODE_OFFLINE_REPROBE_SECS:-5}
@@ -175,6 +190,9 @@ services:
       - RUSTFS_OBJECT_LOCK_ACQUIRE_TIMEOUT=${RUSTFS_OBJECT_LOCK_ACQUIRE_TIMEOUT:-5}
       - RUSTFS_LOCK_ACQUIRE_TIMEOUT=${RUSTFS_LOCK_ACQUIRE_TIMEOUT:-5}
       - RUSTFS_UNSAFE_BYPASS_DISK_CHECK=${RUSTFS_UNSAFE_BYPASS_DISK_CHECK:-true}
+      - RUSTFS_DURABILITY_MODE=${RUSTFS_DURABILITY_MODE:-strict}
+      - RUSTFS_EXPERIMENTAL_FILE_FDATASYNC_GROUP_COMMIT_ENABLE=${RUSTFS_EXPERIMENTAL_FILE_FDATASYNC_GROUP_COMMIT_ENABLE:-false}
+      - RUSTFS_EXPERIMENTAL_FILE_FDATASYNC_GROUP_COMMIT_WAIT_MICROS=${RUSTFS_EXPERIMENTAL_FILE_FDATASYNC_GROUP_COMMIT_WAIT_MICROS:-0}
       # Internode gRPC optimization knobs (grpc-optimization P0-P3). Defaults match the binary
       # defaults, so leaving these unset is a no-op; the A/B driver exports them to toggle a stage.
       - RUSTFS_INTERNODE_RPC_TCP_NODELAY=${RUSTFS_INTERNODE_RPC_TCP_NODELAY:-true}
@@ -184,6 +202,8 @@ services:
       - RUSTFS_INTERNODE_CHANNEL_ISOLATION=${RUSTFS_INTERNODE_CHANNEL_ISOLATION:-false}
       - RUSTFS_INTERNODE_BULK_CHANNELS=${RUSTFS_INTERNODE_BULK_CHANNELS:-2}
       - RUSTFS_INTERNODE_RPC_MSGPACK_ONLY=${RUSTFS_INTERNODE_RPC_MSGPACK_ONLY:-false}
+      - RUSTFS_INTERNODE_RPC_MSGPACK_ONLY_FLEET_CONFIRMED=${RUSTFS_INTERNODE_RPC_MSGPACK_ONLY_FLEET_CONFIRMED:-false}
+      - RUSTFS_OBS_PUT_STAGE_METRICS_ENABLED=${RUSTFS_OBS_PUT_STAGE_METRICS_ENABLED:-false}
       - RUSTFS_INTERNODE_PREWARM=${RUSTFS_INTERNODE_PREWARM:-false}
       - RUSTFS_INTERNODE_OFFLINE_BYPASS=${RUSTFS_INTERNODE_OFFLINE_BYPASS:-false}
       - RUSTFS_INTERNODE_OFFLINE_REPROBE_SECS=${RUSTFS_INTERNODE_OFFLINE_REPROBE_SECS:-5}

.docker/observability/otel-collector-config.yaml

@@ -28,6 +28,15 @@ processors:
     check_interval: 1s
     limit_mib: 1024
     spike_limit_mib: 256
+  transform/metrics:
+    error_mode: ignore
+    metric_statements:
+      # RustFS process metrics already expose process_pid as a metric label.
+      # Remove only the duplicate datapoint label and keep the shared resource
+      # process.pid available to sibling metrics and lock histograms.
+      - context: datapoint
+        statements:
+          - delete_key(attributes, "process_pid") where IsMatch(metric.name, "^rustfs_system_process_")
   transform/logs:
     error_mode: ignore
     log_statements:
@@ -112,7 +121,7 @@ service:
       exporters: [ otlp/tempo, otlp/jaeger ]
     metrics:
       receivers: [ otlp ]
-      processors: [ batch ]
+      processors: [ transform/metrics, memory_limiter, batch ]
       exporters: [ prometheus ]
     logs:
       receivers: [ otlp ]

.docker/rejoin-readiness.Dockerfile

@@ -0,0 +1,40 @@
+# syntax=docker/dockerfile:1.6
+# Copyright 2026 RustFS Team
+# Licensed under the Apache License, Version 2.0.
+
+# A compatible native runtime with Python 3.11+, getent and stat is required by
+# the isolated fixture's staging/provenance checks. This is a test image.
+ARG RUNTIME_BASE=python:3.13-slim-trixie
+FROM ${RUNTIME_BASE}
+
+ARG SOURCE_REVISION
+ARG BINARY_SHA256
+ARG BUILD_PROFILE=dev
+COPY rustfs /usr/bin/rustfs
+RUN python3 - "$SOURCE_REVISION" "$BINARY_SHA256" "$BUILD_PROFILE" <<'PY'
+import hashlib
+import os
+import re
+import shutil
+import sys
+
+source, digest, profile = sys.argv[1:]
+if not re.fullmatch(r"[0-9a-f]{40}", source):
+    raise SystemExit("invalid source revision")
+if not re.fullmatch(r"[0-9a-f]{64}", digest):
+    raise SystemExit("invalid binary digest")
+if profile not in ("dev", "release"):
+    raise SystemExit("invalid build profile")
+with open("/usr/bin/rustfs", "rb") as binary:
+    if hashlib.file_digest(binary, "sha256").hexdigest() != digest:
+        raise SystemExit("binary digest mismatch")
+for tool in ("getent", "stat"):
+    if shutil.which(tool) is None:
+        raise SystemExit(f"missing runtime tool: {tool}")
+os.chmod("/usr/bin/rustfs", 0o755)
+PY
+LABEL io.rustfs.repro.source_revision="${SOURCE_REVISION}" \
+      io.rustfs.repro.binary_sha256="${BINARY_SHA256}" \
+      io.rustfs.repro.build_profile="${BUILD_PROFILE}"
+USER 10001:10001
+ENTRYPOINT ["/usr/bin/rustfs"]

.gemini/settings.json

@@ -0,0 +1,5 @@
+{
+  "context": {
+    "fileName": "AGENTS.md"
+  }
+}

.github/AGENTS.md

@@ -4,34 +4,15 @@ Applies to `.github/`.
 
 ## Pull Requests
 
-PR conventions (English title/body, template usage, `--body-file` with the
-heredoc pattern, review-thread etiquette) live in the root `AGENTS.md` under
-"Git and PR Baseline" — that section is the single normative home; do not
-duplicate its rules here.
+PR conventions (English title/body, template usage, `--body-file` with the heredoc pattern, review-thread etiquette) live in [Git and PR rules](../.agents/references/pull-requests.md); do not duplicate them here.
 
 ## Workflow Changes
 
-- Any change touching `.github/workflows/` must pass `actionlint` (run from
-  the repo root) with zero findings before commit/PR. Install via
-  `brew install actionlint`, or the download script in the actionlint repo
-  (rhysd/actionlint); `make pre-pr` does not cover workflow files, so this is
-  the required check for them.
-- Custom self-hosted runner labels (`sm-standard-*`, `dind-*`) are declared
-  in `.github/actionlint.yaml`. When introducing a new `runs-on:` label,
-  declare it there in the same change. Never use the config or `-ignore` to
-  silence real findings — fix the workflow (root `AGENTS.md`: make checks
-  pass by fixing the cause, not by weakening the gate).
-- If `shellcheck` is installed, actionlint also lints `run:` scripts; treat
-  those findings as part of the gate.
-- Runner placement and billing rules live in `.github/workflows/README.md`.
-  A new job defaults to `ubuntu-latest` (free for this public repository);
-  a self-hosted `runs-on` needs one of the reasons listed there, stated in a
-  comment next to it, and a `rust-cache` key is never shared between the
-  hosted and self-hosted images.
+- Changes to workflow YAML or `.github/actionlint.yaml` must pass `actionlint` (run from the repo root) with zero findings before commit/PR. Install via `brew install actionlint`, or the download script in the actionlint repo (rhysd/actionlint); `make pre-pr` does not cover workflow files, so this is the required check for them.
+- Custom self-hosted runner labels (`sm-standard-*`, `dind-*`) are declared in `.github/actionlint.yaml`. When introducing a new `runs-on:` label, declare it there in the same change. Never use the config or `-ignore` to silence real findings — fix the workflow (root `AGENTS.md`: make checks pass by fixing the cause, not by weakening the gate).
+- If `shellcheck` is installed, actionlint also lints `run:` scripts; treat those findings as part of the gate.
+- Runner placement and billing rules live in `.github/workflows/README.md`. A new job defaults to `ubuntu-latest` (free for this public repository); a self-hosted `runs-on` needs one of the reasons listed there, stated in a comment next to it, and a `rust-cache` key is never shared between the hosted and self-hosted images.
 
 ## CI Alignment
 
-When changing CI-sensitive behavior, keep local validation aligned with
-`.github/workflows/ci.yml`. Read the workflow file directly for the current
-gate steps — do not rely on (or add) a copied command list here; copies go
-stale. `make pre-pr` is the local equivalent of the main gate.
+When changing CI-sensitive behavior, keep local validation aligned with `.github/workflows/ci.yml`. Read the workflow file directly for the current gate steps — do not rely on (or add) a copied command list here; copies go stale. `make pre-pr` covers the shared local gates, not the full CI feature, platform, or end-to-end matrix. Select checks using the root verification tiers.

.github/actions/dual-stack-leg/action.yml

@@ -0,0 +1,54 @@
+# Copyright 2026 RustFS Team
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+name: Both-stack leg
+description: Prove which S3 stack one e2e leg's servers selected and upload the leg for the summary (call with if always)
+
+inputs:
+  profile:
+    description: The nextest profile the leg ran
+    required: true
+  stack:
+    description: The stack the leg requested, legacy or gateway
+    required: true
+  logs:
+    description: Directory or file holding the servers' output
+    required: true
+
+runs:
+  using: composite
+  steps:
+    - name: Prove which stack the servers selected
+      shell: bash
+      env:
+        PROFILE: ${{ inputs.profile }}
+        STACK: ${{ inputs.stack }}
+        LOGS: ${{ inputs.logs }}
+      run: |
+        # Outside the workspace: e2e_binary.py refuses a run whose source tree changed.
+        mkdir -p "$RUNNER_TEMP/leg"
+        junit="target/nextest/${PROFILE}/junit.xml"
+        if [ -s "$junit" ]; then cp "$junit" "$RUNNER_TEMP/leg/junit.xml"; fi
+        # A gateway leg stopped by its preflight has no output; the preflight's stack.json stays.
+        if [ -e "$LOGS" ]; then python3 scripts/dual_stack.py stack --expect "$STACK" --out "$RUNNER_TEMP/leg/stack.json" "$LOGS"; fi
+
+    - name: Upload the leg
+      if: always()
+      uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+      with:
+        name: dual-stack-leg-${{ inputs.profile }}-${{ inputs.stack }}
+        overwrite: true
+        path: ${{ runner.temp }}/leg
+        if-no-files-found: warn
+        retention-days: 7

.github/actions/dual-stack-summary/action.yml

@@ -0,0 +1,56 @@
+# Copyright 2026 RustFS Team
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+name: Both-stack summary
+description: Diff the legacy and gateway legs of this run (scripts/junit_diff.py) and upload one dated summary
+
+inputs:
+  profiles:
+    description: Comma-separated nextest profiles whose two legs this run uploaded
+    required: true
+
+runs:
+  using: composite
+  steps:
+    - name: Download the legs
+      uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
+      with:
+        pattern: dual-stack-leg-*
+        path: legs
+
+    - name: Diff every profile and write the summary
+      id: summary
+      shell: bash
+      env:
+        PROFILES: ${{ inputs.profiles }}
+      run: |
+        # The shell runs with -e; the exit status is reported after the summary is written.
+        set +e -u -o pipefail
+        date="$(date -u +%F)"
+        echo "date=${date}" >> "$GITHUB_OUTPUT"
+        mkdir -p legs
+        python3 scripts/dual_stack.py summary --profiles "$PROFILES" --legs legs --date "$date" --out-dir dual-stack
+        status=$?
+        if [ -f dual-stack/summary.md ]; then cat dual-stack/summary.md >> "$GITHUB_STEP_SUMMARY"; fi
+        exit "$status"
+
+    - name: Upload the dated summary
+      if: always() && steps.summary.outputs.date != ''
+      uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+      with:
+        name: dual-stack-summary-${{ steps.summary.outputs.date }}
+        overwrite: true
+        path: dual-stack
+        if-no-files-found: error
+        retention-days: 30

.github/actions/quick-checks/action.yml

@@ -13,7 +13,7 @@
 # limitations under the License.
 
 name: Quick Checks
-description: Run the shared compile-free RustFS quality checks.
+description: Run the shared RustFS quality checks.
 
 runs:
   using: composite
@@ -24,19 +24,30 @@ runs:
         tool: |
           ripgrep@15.2.0
           shellcheck@0.11.0
+          cargo-nextest@0.9.138
 
     - name: Install actionlint
       shell: bash
+      env:
+        GH_TOKEN: ${{ github.token }}
+        ASSET_DOWNLOADER: ${{ github.action_path }}/../setup/download_release_asset.py
       run: |
         actionlint_dir="$(mktemp -d "${RUNNER_TEMP}/actionlint.XXXXXX")"
-        curl --fail --location --silent --show-error \
-          --output "$actionlint_dir/actionlint.tar.gz" \
-          https://github.com/rhysd/actionlint/releases/download/v1.7.12/actionlint_1.7.12_linux_amd64.tar.gz
+        python3 "$ASSET_DOWNLOADER" --repository rhysd/actionlint --tag v1.7.12 --asset actionlint_1.7.12_linux_amd64.tar.gz --output "$actionlint_dir/actionlint.tar.gz" --sha256 8aca8db96f1b94770f1b0d72b6dddcb1ebb8123cb3712530b08cc387b349a3d8
         echo "8aca8db96f1b94770f1b0d72b6dddcb1ebb8123cb3712530b08cc387b349a3d8  $actionlint_dir/actionlint.tar.gz" | sha256sum --check --status
         tar -xzf "$actionlint_dir/actionlint.tar.gz" -C "$actionlint_dir" actionlint
         rm "$actionlint_dir/actionlint.tar.gz"
         echo "$actionlint_dir" >> "$GITHUB_PATH"
 
+    - name: Install Python
+      uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
+      with:
+        python-version: "3.12"
+
+    - name: Install workflow parser
+      shell: bash
+      run: python3 -m pip install --disable-pip-version-check PyYAML==6.0.3
+
     - name: Install Rust toolchain
       uses: dtolnay/rust-toolchain@29eef336d9b2848a0b548edc03f92a220660cdb8 # stable
       with:
@@ -84,7 +95,33 @@ runs:
 
     - name: Check s3s footprint ratchet
       shell: bash
-      run: ./scripts/check_s3s_footprint.sh
+      env:
+        # Pull requests and merge groups compare with their base revision; a
+        # push or scheduled run holds the tree to the baseline file exactly.
+        FOOTPRINT_BASE: ${{ github.event.pull_request.base.sha || github.event.merge_group.base_sha }}
+      run: |
+        if [[ -z "$FOOTPRINT_BASE" ]]; then
+          ./scripts/check_s3s_footprint.sh
+          exit
+        fi
+        git cat-file -e "$FOOTPRINT_BASE^{commit}" 2>/dev/null || git fetch --no-tags --depth=1 origin "$FOOTPRINT_BASE"
+        ./scripts/check_s3s_footprint.sh --base "$FOOTPRINT_BASE"
+
+    - name: Check release feature boundaries
+      shell: bash
+      run: python3 scripts/check_connect_release_boundary.py --features-only
+
+    - name: Check per-crate test-count ratchet
+      shell: bash
+      run: ./scripts/check_test_count_ratchet.sh
+
+    - name: Check cross-owner tests use test seams
+      shell: bash
+      run: python3 ./scripts/check_test_seams.py
+
+    - name: Check retired tests name existing replacements
+      shell: bash
+      run: python3 ./scripts/check_test_retirements.py
 
     - name: Check cryptographic capability wording
       shell: bash
@@ -104,7 +141,10 @@ runs:
 
     - name: Check test wiring
       shell: bash
-      run: python3 ./scripts/check_test_wiring.py
+      run: |
+        python3 ./scripts/check_test_wiring.py
+        python3 ./scripts/ci_gate.py --check-workflow
+        python3 ./scripts/validation_registry.py check
 
     - name: Check no planning docs committed
       shell: bash

.github/actions/reference-binary/action.yml

@@ -0,0 +1,96 @@
+# Copyright 2026 RustFS Team
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+name: "Reference RustFS binary"
+description: "Download the pinned reference RustFS release, verify its sha256 and expose the executable path. A missing pin, missing asset or digest mismatch fails the step."
+
+inputs:
+  pin-file:
+    description: "Pin file naming the reference release (repository-relative)"
+    required: false
+    default: ".config/reference-binary.toml"
+  platform:
+    description: "Platform table in the pin file"
+    required: false
+    default: "linux-x86_64-gnu"
+  destination:
+    description: "Directory that receives the executable; defaults to $RUNNER_TEMP/reference-binary"
+    required: false
+    default: ""
+
+outputs:
+  path:
+    description: "Absolute path of the verified executable"
+    value: ${{ steps.unpack.outputs.path }}
+  version:
+    description: "Pinned reference version"
+    value: ${{ steps.pin.outputs.version }}
+  sha256:
+    description: "Pinned and verified archive sha256"
+    value: ${{ steps.pin.outputs.sha256 }}
+
+runs:
+  using: "composite"
+  steps:
+    - name: Resolve reference pin
+      id: pin
+      shell: bash
+      env:
+        PIN_TOOL: ${{ github.action_path }}/reference_binary.py
+        PIN_FILE: ${{ inputs.pin-file }}
+        PIN_PLATFORM: ${{ inputs.platform }}
+      run: |
+        set -euo pipefail
+        resolved="$(python3 "$PIN_TOOL" resolve --pin "$PIN_FILE" --platform "$PIN_PLATFORM")"
+        printf '%s\n' "$resolved"
+        printf '%s\n' "$resolved" >> "$GITHUB_OUTPUT"
+
+    - name: Download and verify reference archive
+      id: download
+      shell: bash
+      env:
+        GH_TOKEN: ${{ github.token }}
+        ASSET_DOWNLOADER: ${{ github.action_path }}/../setup/download_release_asset.py
+        REF_REPOSITORY: ${{ steps.pin.outputs.repository }}
+        REF_TAG: ${{ steps.pin.outputs.tag }}
+        REF_ASSET: ${{ steps.pin.outputs.asset }}
+        REF_SHA256: ${{ steps.pin.outputs.sha256 }}
+      run: |
+        set -euo pipefail
+        archive="$RUNNER_TEMP/reference-archive/$REF_ASSET"
+        # Fails unless the release lists exactly one asset of this name whose
+        # published digest equals the pin and whose downloaded bytes match it.
+        python3 "$ASSET_DOWNLOADER" --repository "$REF_REPOSITORY" --tag "$REF_TAG" \
+          --asset "$REF_ASSET" --output "$archive" --sha256 "$REF_SHA256"
+        echo "archive=$archive" >> "$GITHUB_OUTPUT"
+
+    - name: Unpack reference executable
+      id: unpack
+      shell: bash
+      env:
+        PIN_TOOL: ${{ github.action_path }}/reference_binary.py
+        REF_ARCHIVE: ${{ steps.download.outputs.archive }}
+        REF_SHA256: ${{ steps.pin.outputs.sha256 }}
+        REF_EXECUTABLE: ${{ steps.pin.outputs.executable }}
+        REF_DESTINATION: ${{ inputs.destination }}
+      run: |
+        set -euo pipefail
+        destination="${REF_DESTINATION:-$RUNNER_TEMP/reference-binary}"
+        unpacked="$(python3 "$PIN_TOOL" unpack --archive "$REF_ARCHIVE" --sha256 "$REF_SHA256" \
+          --executable "$REF_EXECUTABLE" --destination "$destination")"
+        path="${unpacked#path=}"
+        test -x "$path"
+        echo "Reference executable: $path"
+        "$path" --version
+        echo "path=$path" >> "$GITHUB_OUTPUT"

.github/actions/reference-binary/reference_binary.py

@@ -0,0 +1,144 @@
+#!/usr/bin/env python3
+# Copyright 2026 RustFS Team
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+"""Resolve and unpack the pinned reference RustFS release binary.
+
+`resolve` reads the pin file and prints the selected platform's identity as
+`key=value` lines (GitHub step-output format). `unpack` re-verifies the
+downloaded archive digest against the pin and extracts exactly one executable.
+Every missing or malformed input is a failure; nothing is skipped.
+"""
+
+import argparse
+import hashlib
+from pathlib import Path
+import re
+import sys
+import tomllib
+import zipfile
+
+REPOSITORY = re.compile(r"[A-Za-z0-9][A-Za-z0-9_.-]*/[A-Za-z0-9][A-Za-z0-9_.-]*")
+TAG = re.compile(r"[A-Za-z0-9][A-Za-z0-9_.+-]*")
+PLATFORM = re.compile(r"[a-z0-9][a-z0-9_-]*")
+ASSET = re.compile(r"[A-Za-z0-9][A-Za-z0-9_.+-]*\.zip")
+SHA256 = re.compile(r"[0-9a-f]{64}")
+EXECUTABLE = re.compile(r"[A-Za-z0-9][A-Za-z0-9_.-]*")
+MAX_EXECUTABLE_BYTES = 1024 * 1024 * 1024
+
+
+class PinError(Exception):
+    """A pin, archive or argument that cannot be trusted."""
+
+
+def field(table, key, pattern, where):
+    value = table.get(key)
+    if not isinstance(value, str) or not pattern.fullmatch(value):
+        raise PinError(f"{where}: missing or invalid '{key}'")
+    return value
+
+
+def resolve(pin_path, platform):
+    if not PLATFORM.fullmatch(platform or ""):
+        raise PinError("invalid platform name")
+    try:
+        data = tomllib.loads(Path(pin_path).read_text(encoding="utf-8"))
+    except FileNotFoundError as exc:
+        raise PinError(f"{pin_path}: reference pin file is missing") from exc
+    except (OSError, UnicodeDecodeError, tomllib.TOMLDecodeError) as exc:
+        raise PinError(f"{pin_path}: cannot read reference pin: {exc}") from exc
+    if data.get("schema") != 1 or type(data.get("schema")) is not int:
+        raise PinError(f"{pin_path}: unsupported pin schema")
+    identity = {
+        "repository": field(data, "repository", REPOSITORY, pin_path),
+        "version": field(data, "version", TAG, pin_path),
+        "tag": field(data, "tag", TAG, pin_path),
+    }
+    platforms = data.get("platforms")
+    if not isinstance(platforms, dict) or not platforms:
+        raise PinError(f"{pin_path}: no pinned platforms")
+    entry = platforms.get(platform)
+    if not isinstance(entry, dict):
+        raise PinError(f"{pin_path}: no pin for platform '{platform}'")
+    where = f"{pin_path} [platforms.{platform}]"
+    identity["asset"] = field(entry, "asset", ASSET, where)
+    identity["sha256"] = field(entry, "sha256", SHA256, where)
+    identity["executable"] = field(entry, "executable", EXECUTABLE, where)
+    if identity["version"] not in identity["asset"]:
+        raise PinError(f"{where}: asset name does not carry version {identity['version']}")
+    unknown = set(entry) - {"asset", "sha256", "executable"}
+    if unknown:
+        raise PinError(f"{where}: unknown keys {sorted(unknown)}")
+    return identity
+
+
+def file_sha256(path):
+    with open(path, "rb") as stream:
+        return hashlib.file_digest(stream, "sha256").hexdigest()
+
+
+def unpack(archive, sha256, executable, destination):
+    if not SHA256.fullmatch(sha256 or "") or not EXECUTABLE.fullmatch(executable or ""):
+        raise PinError("invalid expected digest or executable name")
+    try:
+        actual = file_sha256(archive)
+    except OSError as exc:
+        raise PinError(f"{archive}: reference archive is missing or unreadable") from exc
+    if actual != sha256:
+        raise PinError(f"{archive}: sha256 {actual} does not match the pinned digest")
+    try:
+        with zipfile.ZipFile(archive) as bundle:
+            matches = [info for info in bundle.infolist() if info.filename == executable]
+            if len(matches) != 1:
+                raise PinError(f"{archive}: must contain exactly one top-level '{executable}'")
+            info = matches[0]
+            if info.is_dir() or not 0 < info.file_size <= MAX_EXECUTABLE_BYTES:
+                raise PinError(f"{archive}: '{executable}' is not a plausible executable")
+            destination = Path(destination)
+            destination.mkdir(parents=True, exist_ok=True)
+            target = destination / executable
+            with bundle.open(info) as source, open(target, "wb") as sink:
+                while chunk := source.read(1024 * 1024):
+                    sink.write(chunk)
+    except zipfile.BadZipFile as exc:
+        raise PinError(f"{archive}: not a valid zip archive") from exc
+    target.chmod(0o755)
+    return target
+
+
+def main(argv=None):
+    parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
+    sub = parser.add_subparsers(dest="mode", required=True)
+    res = sub.add_parser("resolve")
+    res.add_argument("--pin", required=True)
+    res.add_argument("--platform", required=True)
+    unp = sub.add_parser("unpack")
+    unp.add_argument("--archive", required=True)
+    unp.add_argument("--sha256", required=True)
+    unp.add_argument("--executable", required=True)
+    unp.add_argument("--destination", required=True)
+    args = parser.parse_args(argv)
+    try:
+        if args.mode == "resolve":
+            for key, value in resolve(args.pin, args.platform).items():
+                print(f"{key}={value}")
+        else:
+            print(f"path={unpack(args.archive, args.sha256, args.executable, args.destination)}")
+    except PinError as exc:
+        print(f"reference binary: {exc}", file=sys.stderr)
+        return 1
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

.github/actions/runner-pilot/action.yml

@@ -0,0 +1,107 @@
+name: Runner capability and build pilot
+description: Inspect an existing runner and measure isolated fixed-source cold/warm builds
+inputs:
+  workload:
+    description: probe, server or workspace-tests
+    required: true
+  source-sha:
+    description: Exact merged main commit for builds
+    default: ""
+  toolchain:
+    description: Exact Rust version for builds
+    default: ""
+  scratch-root:
+    description: Existing absolute scratch root
+    default: ""
+  build-jobs:
+    description: Cargo concurrency
+    default: "2"
+  local-ssd:
+    description: Require operator storage attestation
+    default: "false"
+runs:
+  using: composite
+  steps:
+    - name: Validate pilot input and inspect existing capabilities
+      shell: bash
+      env:
+        PILOT_WORKLOAD: ${{ inputs.workload }}
+        PILOT_SHA: ${{ inputs.source-sha }}
+        PILOT_TOOLCHAIN: ${{ inputs.toolchain }}
+        PILOT_SCRATCH: ${{ inputs.scratch-root }}
+        PILOT_JOBS: ${{ inputs.build-jobs }}
+        PILOT_SSD: ${{ inputs.local-ssd }}
+      run: |
+        [[ "$PILOT_WORKLOAD" =~ ^(probe|server|workspace-tests)$ ]]
+        [[ "$PILOT_JOBS" =~ ^([1-9]|1[0-6])$ ]]
+        if [[ "$PILOT_WORKLOAD" != probe ]]; then
+          [[ "$PILOT_SHA" =~ ^[a-f0-9]{40}$ ]]
+          [[ "$PILOT_TOOLCHAIN" =~ ^[0-9]+\.[0-9]+\.[0-9]+$ ]]
+        fi
+        scratch="${PILOT_SCRATCH:-$RUNNER_TEMP}"
+        [[ "$scratch" == /* && -d "$scratch" ]]
+        minimum_disk=1
+        minimum_memory=0
+        if [[ "$PILOT_WORKLOAD" == server ]]; then minimum_disk=24; minimum_memory=8; fi
+        if [[ "$PILOT_WORKLOAD" == workspace-tests ]]; then minimum_disk=90; minimum_memory=16; fi
+        extra=()
+        if [[ "$PILOT_SSD" == true ]]; then extra+=(--local-ssd-attestation /etc/rustfs-runner/storage.json); fi
+        python3 scripts/ci/runner_pilot.py --root "$scratch" \
+          --output "$RUNNER_TEMP/runner-pilot-evidence" \
+          --min-free-gib "$minimum_disk" --min-memory-gib "$minimum_memory" --architecture x86_64 "${extra[@]}"
+    - name: Checkout the trusted baseline history
+      if: inputs.workload != 'probe'
+      uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
+      with:
+        ref: main
+        path: .runner-pilot-source
+        fetch-depth: 0
+        persist-credentials: false
+    - name: Bind the build to a merged main commit
+      if: inputs.workload != 'probe'
+      shell: bash
+      env:
+        PILOT_SHA: ${{ inputs.source-sha }}
+      run: |
+        git -C .runner-pilot-source merge-base --is-ancestor "$PILOT_SHA" HEAD
+        git -C .runner-pilot-source checkout --detach "$PILOT_SHA"
+    - name: Install build dependencies
+      if: inputs.workload != 'probe'
+      shell: bash
+      run: |
+        sudo apt-get update
+        sudo apt-get install -y build-essential pkg-config libssl-dev cmake clang time
+    - name: Install the fixed protobuf compiler
+      if: inputs.workload != 'probe'
+      uses: rustfs/setup-protoc@a3705324d8f9bf5b6c3573fb6cf8ae421db55dd6 # v3.0.1
+      with:
+        version: "35.1"
+        repo-token: ${{ github.token }}
+    - name: Install the fixed flatbuffer compiler
+      if: inputs.workload != 'probe'
+      uses: Nugine/setup-flatc@698800de72a96bfb22cf60431dc21a2ff9a7e07b # v1
+      with:
+        version: "25.12.19"
+    - name: Install the requested exact Rust toolchain
+      if: inputs.workload != 'probe'
+      uses: dtolnay/rust-toolchain@29eef336d9b2848a0b548edc03f92a220660cdb8 # stable
+      with:
+        toolchain: ${{ inputs.toolchain }}
+    - name: Measure cold and warm builds without shared target caches
+      if: inputs.workload != 'probe'
+      shell: bash
+      env:
+        PILOT_WORKLOAD: ${{ inputs.workload }}
+        PILOT_SHA: ${{ inputs.source-sha }}
+        PILOT_TOOLCHAIN: ${{ inputs.toolchain }}
+        PILOT_SCRATCH: ${{ inputs.scratch-root }}
+        PILOT_JOBS: ${{ inputs.build-jobs }}
+        PILOT_SSD: ${{ inputs.local-ssd }}
+      run: |
+        extra=()
+        if [[ "$PILOT_SSD" == true ]]; then extra+=(--local-ssd-attestation /etc/rustfs-runner/storage.json); fi
+        python3 scripts/ci/runner_pilot.py --root "${PILOT_SCRATCH:-$RUNNER_TEMP}" \
+          --output "$RUNNER_TEMP/runner-pilot-evidence" \
+          --source .runner-pilot-source --source-sha "$PILOT_SHA" \
+          --toolchain "$PILOT_TOOLCHAIN" --benchmark "$PILOT_WORKLOAD" \
+          --jobs "$PILOT_JOBS" --timeout-seconds 3600 "${extra[@]}"

.github/actions/schedule-failure-issue/README.md

@@ -74,7 +74,6 @@ Notes:
 - `.github/workflows/e2e-s3tests.yml` (weekly full s3-tests sweep)
 - `.github/workflows/mint.yml` (weekly multi-SDK mint run)
 - `.github/workflows/fuzz.yml` (nightly fuzz corpus jobs)
-- `.github/workflows/performance-ab.yml` (nightly warp A/B gate)
 - **Pending:** the ci-7 e2e nightly-full workflow does not exist yet
   (backlog#1149 ci-7). When it lands, wire it up with the snippet above.
 

.github/actions/setup/action.yml

@@ -63,13 +63,15 @@ inputs:
     required: false
     default: "true"
 
+outputs:
+  cache-hit:
+    description: "'true' when the Rust cache was restored from an exact key match"
+    value: ${{ steps.rust-cache.outputs.cache-hit }}
+
 runs:
   using: "composite"
   steps:
-    # protobuf-compiler is deliberately absent: the setup-protoc step below
-    # installs 35.1 into the tool cache and prepends it to PATH, so the apt
-    # build (older, and never version-matched) was shadowed on every run and
-    # simply never used.
+    # Use the pinned code generators rather than Ubuntu's older packages.
     - name: Install system dependencies (Ubuntu)
       if: runner.os == 'Linux'
       shell: bash
@@ -79,7 +81,9 @@ runs:
           build-essential \
           pkg-config \
           libssl-dev \
-          ripgrep
+          ripgrep \
+          python3 \
+          gh
 
     # musl-gcc is needed by the native musl release leg, and zip/unzip by the
     # release packaging steps. No CI test lane touches any of them.
@@ -88,13 +92,32 @@ runs:
       shell: bash
       run: sudo apt-get install -y musl-tools zip unzip
 
+    - name: Install pinned code generators through GitHub API (Linux x64)
+      if: runner.os == 'Linux' && runner.arch == 'X64'
+      shell: bash
+      env:
+        GH_TOKEN: ${{ github.token }}
+        ASSET_DOWNLOADER: ${{ github.action_path }}/download_release_asset.py
+      run: |
+        python3 "$ASSET_DOWNLOADER" --repository protocolbuffers/protobuf --tag v35.1 --asset protoc-35.1-linux-x86_64.zip --output "$RUNNER_TEMP/protoc.zip"
+        python3 "$ASSET_DOWNLOADER" --repository google/flatbuffers --tag v25.12.19 --asset 'Linux.flatc.binary.g++-13.zip' --output "$RUNNER_TEMP/flatc.zip"
+        python3 -m zipfile -e "$RUNNER_TEMP/protoc.zip" "$RUNNER_TEMP/protoc"
+        python3 -m zipfile -e "$RUNNER_TEMP/flatc.zip" "$RUNNER_TEMP/flatc"
+        chmod +x "$RUNNER_TEMP/protoc/bin/protoc" "$RUNNER_TEMP/flatc/flatc"
+        "$RUNNER_TEMP/protoc/bin/protoc" --version
+        "$RUNNER_TEMP/flatc/flatc" --version
+        echo "$RUNNER_TEMP/protoc/bin" >> "$GITHUB_PATH"
+        echo "$RUNNER_TEMP/flatc" >> "$GITHUB_PATH"
+
     - name: Install protoc
+      if: runner.os != 'Linux' || runner.arch != 'X64'
       uses: rustfs/setup-protoc@a3705324d8f9bf5b6c3573fb6cf8ae421db55dd6 # v3.0.1
       with:
         version: "35.1"
         repo-token: ${{ github.token }}
 
     - name: Install flatc
+      if: runner.os != 'Linux' || runner.arch != 'X64'
       uses: Nugine/setup-flatc@698800de72a96bfb22cf60431dc21a2ff9a7e07b # v1
       with:
         version: "25.12.19"
@@ -125,6 +148,7 @@ runs:
         tool: cargo-nextest@0.9.138
 
     - name: Setup Rust cache
+      id: rust-cache
       uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2
       with:
         # false is rust-cache's own default. With true, cleanup.ts returns

.github/actions/setup/download_release_asset.py

@@ -0,0 +1,84 @@
+#!/usr/bin/env python3
+"""Download a named GitHub release asset through its API and verify its digest."""
+
+import argparse
+import hashlib
+import json
+from pathlib import Path
+import re
+import subprocess
+import sys
+import tempfile
+from urllib.parse import quote
+
+
+def download(repository, tag, name, output, expected_sha256=None):
+    if not re.fullmatch(r"[A-Za-z0-9][A-Za-z0-9_.-]*/[A-Za-z0-9][A-Za-z0-9_.-]*", repository):
+        raise ValueError("invalid release repository")
+    if not tag or not name or "/" in name or "\\" in name:
+        raise ValueError("invalid release tag or asset name")
+    if expected_sha256 is not None and not re.fullmatch(r"[0-9a-f]{64}", expected_sha256):
+        raise ValueError("invalid pinned asset digest")
+    result = subprocess.run(
+        ["gh", "api", "--hostname", "github.com", f"repos/{repository}/releases/tags/{quote(tag, safe='')}"],
+        check=True, capture_output=True, text=True, timeout=60,
+    )
+    release = json.loads(result.stdout)
+    if not isinstance(release, dict) or release.get("tag_name") != tag:
+        raise ValueError("release metadata does not match the requested tag")
+    assets = release.get("assets")
+    if not isinstance(assets, list) or any(not isinstance(asset, dict) for asset in assets):
+        raise ValueError("invalid release asset inventory")
+    matches = [asset for asset in assets if asset.get("name") == name]
+    if len(matches) != 1:
+        raise ValueError("release must contain exactly one named asset")
+    asset = matches[0]
+    asset_id, size, digest = asset.get("id"), asset.get("size"), asset.get("digest")
+    if type(asset_id) is not int or asset_id <= 0 or type(size) is not int or not 0 < size <= 512 * 1024 * 1024:
+        raise ValueError("invalid release asset identity or size")
+    if not isinstance(digest, str) or not re.fullmatch(r"sha256:[0-9a-f]{64}", digest):
+        raise ValueError("release asset has no valid SHA-256 digest")
+    sha256 = digest.removeprefix("sha256:")
+    if expected_sha256 is not None and expected_sha256 != sha256:
+        raise ValueError("release digest differs from the pinned digest")
+    output = Path(output)
+    output.parent.mkdir(parents=True, exist_ok=True)
+    temporary = None
+    try:
+        with tempfile.NamedTemporaryFile(dir=output.parent, prefix=".release-asset-", delete=False) as stream:
+            temporary = Path(stream.name)
+            # gh authenticates to the API; its HTTP client strips credentials on
+            # cross-host redirects. No browser URL or token is supplied in argv.
+            subprocess.run(
+                ["gh", "api", "--hostname", "github.com", "--header", "Accept: application/octet-stream", f"repos/{repository}/releases/assets/{asset_id}"],
+                check=True, stdout=stream, timeout=300,
+            )
+        with temporary.open("rb") as stream:
+            actual = hashlib.file_digest(stream, "sha256").hexdigest()
+        if temporary.stat().st_size != size or actual != sha256:
+            raise ValueError("downloaded release asset failed size or SHA-256 verification")
+        temporary.replace(output)
+        print(f"Verified release asset: {repository} {tag} {name} sha256={actual}")
+    finally:
+        if temporary is not None:
+            temporary.unlink(missing_ok=True)
+
+
+def main():
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument("--repository", required=True)
+    parser.add_argument("--tag", required=True)
+    parser.add_argument("--asset", required=True)
+    parser.add_argument("--output", required=True, type=Path)
+    parser.add_argument("--sha256")
+    args = parser.parse_args()
+    try:
+        download(args.repository, args.tag, args.asset, args.output, args.sha256)
+    except (ValueError, KeyError, TypeError, json.JSONDecodeError, subprocess.SubprocessError, OSError) as error:
+        print(f"Release asset download failed: {error}", file=sys.stderr)
+        return 1
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

.github/actions/test-runtime/action.yml

@@ -0,0 +1,16 @@
+name: Verification runtime
+description: Install the runtime for prebuilt test archives without a target cache
+runs:
+  using: composite
+  steps:
+    - name: Install runtime tools
+      shell: bash
+      run: |
+        sudo apt-get update
+        sudo apt-get install -y libssl-dev ripgrep
+    - name: Install Rust toolchain
+      uses: dtolnay/rust-toolchain@29eef336d9b2848a0b548edc03f92a220660cdb8 # stable
+    - name: Install cargo-nextest
+      uses: taiki-e/install-action@96c7780c1d8a2b8723e12031def873a434d39d8d # nextest
+      with:
+        tool: cargo-nextest@0.9.138

.github/dependabot.yml

@@ -32,6 +32,8 @@ updates:
       - "overtrue"
       - "majinghe"
     ignore:
+      - dependency-name: "pulsar"
+        versions: [ "6.9.x" ]
       - dependency-name: "object_store"
         versions: [ "0.13.x" ]
       - dependency-name: "libunftp"

.github/pull_request_template.md

@@ -24,8 +24,7 @@ List relevant checks not run and the remaining risk. Use the validation tier in
 
 ## Impact
 <!--
-Describe user-facing, compatibility, API, deployment, configuration, or
-documentation impact. Use N/A when there is no expected impact.
+Describe user-facing, compatibility, API, deployment, configuration, or documentation impact. Use N/A when there is no expected impact.
 -->
 
 ## Additional Notes

.github/workflows/README.md

@@ -5,18 +5,19 @@
 
 ## Billing facts
 
-This repository is public. For a public repository, GitHub's standard hosted runners (`ubuntu-latest`, `windows-latest`, `macos-latest`) cost nothing and have no minute cap, whatever plan the organisation is on. GitHub bills only two things: larger hosted runners, which have custom labels and are not used here, and standard runners in private repositories.
+This repository is public. For a public repository, GitHub's standard hosted runners (`ubuntu-latest`, `windows-latest`, `macos-latest`) cost nothing and have no minute cap, whatever plan the organisation is on. GitHub bills only two things: larger hosted runners, including the `macos-latest-xlarge` release lane used here, and standard runners in private repositories.
 
-Every other label in this repository is capacity RustFS pays for by the hour whether or not a job is running well:
+Every other label in this repository is capacity RustFS pays for. ARC pods and lab machines accrue cost while the machine exists. A Ubicloud label is metered for the minutes a job holds its VM:
 
 | Label | Paid by | What it is | Use it for |
 |---|---|---|---|
 | `ubuntu-latest` | nobody | GitHub-hosted, Ubuntu 24.04, 4 vCPU, 16 GB RAM, about 14 GB of free disk. Ships a C toolchain, `docker`, `gh`, `curl`, `wget`, `jq`, Python. Can `mount` tmpfs and loop devices; io_uring works | the default for every job |
 | `windows-latest`, `macos-latest` | nobody | GitHub-hosted standard runners | Windows and macOS targets |
+| `macos-latest-xlarge` | RustFS | GitHub-hosted larger Apple Silicon runner | macOS release builds, deferred until Linux acceptance for candidates |
 | `sm-standard-2` | RustFS | ARC pod, Ubuntu 24.04, minimal image: no `gh`, no C toolchain; Docker only on the `dind-` label | only a job that a hosted runner cannot run (see the placement rule) |
 | `sm-standard-4` | RustFS | ARC pod, Ubuntu 24.04, 14 CPUs and 28 GB (the label is not the core count), large disk | workspace-scale Cargo builds |
-| `sm-standard-4-arm` | RustFS | ARC pod, aarch64; reached through the `build.yml` matrix, so `actionlint.yaml` does not list it | native aarch64 release builds |
-| `dind-sm-standard-2` | RustFS | ARC pod with Docker-in-Docker | image builds that need `buildx` and a registry push |
+| `ubicloud-standard-16-arm` | RustFS | Ubicloud VM, alias of `ubicloud-standard-16-arm-ubuntu-2404`: Ubuntu 24.04 arm64, 16 vCPU, 48 GB, 300 GB disk. Reached through the `build.yml` matrix, so `actionlint.yaml` does not list it | native aarch64 release builds (`linux-aarch64-musl`, `linux-aarch64-gnu`) |
+| `dind-sm-standard-2` | RustFS | ARC pod with Docker-in-Docker | Docker workloads with a measured resource or environment requirement beyond the hosted runner |
 | `smoke-testing`, `pf-testing` | RustFS | long-lived lab machines | the packaged functional chain and performance suites |
 
 The self-hosted pods and the hosted image are both Ubuntu 24.04, so a binary built on one runs on the other; `ci.yml` builds the debug binary on `sm-standard-4` and runs it on both pools.
@@ -27,11 +28,15 @@ A job runs on `ubuntu-latest` unless it has one of these reasons, and the reason
 
 1. **Disk.** It restores or produces the workspace `target/`: `cargo build` or `cargo test` with `--all-targets`, `cargo nextest run --all`, or any reader of the `ci-dev` / `ci-feat-*` caches. The workspace test build leaves about 64 GB in `target/`; a hosted runner has about 14 GB.
 2. **Critical-path CPU.** It is a build that several downstream lanes wait on and would blow its timeout at 4 vCPU (`build-rustfs-debug-binary` gates the e2e and s3-tests lanes).
-3. **Hardware.** Native aarch64, Docker-in-Docker, or the lab machines.
+3. **Hardware or environment.** Native aarch64 or access to the lab machines. Docker alone is not a reason: hosted Ubuntu ships Docker and supports the QEMU and Buildx setup actions.
 4. **Cache ownership.** It reads a `rust-cache` key that a self-hosted job writes (see the cache rule below).
 
 Jobs that do only the following have no reason to be self-hosted: shell or Python scripts, `cargo fmt`, `cargo deny`, typos, actionlint, API calls (`gh`, dependency review, issue and stale bots, CLA checks), downloading an artifact and running a prebuilt binary, or a `cargo build -p rustfs` with its own cache key. `e2e-distributed.yml` and `e2e-upgrade.yml` build the server and the `e2e_test` harness on `ubuntu-latest` inside their budgets, so "it compiles something" alone is not a reason.
 
+The functional chain's `prepare` and `complete-chain` jobs, the performance runner availability check, and runner hygiene use `ubuntu-latest`: they inspect APIs or artifacts, not the lab hosts. The actual functional and performance suites remain on their lab runners. Functional-chain health collection uses `ubuntu-latest` so a lab outage cannot also stop its independent monitor. Docker image packaging also uses `ubuntu-latest` for both musl and glibc variants; it wraps verified candidate binaries rather than compiling the workspace, and still sets up QEMU and Buildx explicitly.
+
+Moving these jobs off AKS reduces capacity demand, not necessarily the Azure bill immediately. VM costs fall only when nodes scale down or smaller capacity replaces them.
+
 A hosted runner is also the only choice for a few things the pods cannot do: `mount -o loop` and `mount -t tmpfs` fail inside the ARC pods, and a container runtime cannot guarantee that `io_uring_setup` is allowed. `uring-integration` and `e2e-distributed` are on `ubuntu-latest` for that reason, not for cost.
 
 ## Cache rule
@@ -39,8 +44,9 @@ A hosted runner is also the only choice for a few things the pods cannot do: `mo
 `Swatinem/rust-cache` builds its key from the OS, architecture, toolchain and lockfile. It does not include the runner label or image, so a cache written on an `sm-standard-4` pod and one written on `ubuntu-latest` would overwrite each other under the same `shared-key`.
 
 - Every `shared-key` has exactly one writer. `cache-warm.yml` owns `ci-dev`, `ci-feat-rio`, `ci-feat-proto` and `ci-uring`; every other `./.github/actions/setup` call states `cache-save-if` explicitly, and `scripts/security/check_cache_save_if.sh` fails the build when one does not.
+- The `ci-dev` key ends in a digest of `warm-ci-dev`'s build commands. Changing what that job warms therefore needs a new key, or readers keep restoring the old artifacts: `scripts/ci_gate.py --check-workflow` prints the key the commands require and fails until the writer and every reader in `ci.yml` and `reference-parity.yml` name it.
 - All readers of a key run on the same image as its writer. Keys that live on hosted runners say so in their name (`ci-uring` is warmed on `ubuntu-latest`; `ci-e2e-distributed-hosted`, `e2e-upgrade-server-hosted`, `e2e-upgrade-tests-hosted`).
-- Moving a job between pools therefore means renaming its key. The new key is empty until a run from `main` (a scheduled run, or a manual dispatch of that workflow from `main`) saves it; until then that lane does a cold build.
+- Moving a job between pools therefore means renaming its key. The new key is empty until a run from `main` (a scheduled run, or a manual dispatch of that workflow from `main`) saves it; until then that lane does a cold build. The `build.yml` aarch64 release legs append `-ubicloud-arm` for this reason. `cache-warm.yml` does not write those keys.
 - `CARGO_BUILD_JOBS` and every other `CARGO*`, `CC*`, `CFLAGS*`, `CXX*`, `CMAKE*` and `RUST*` variable is hashed into the key. `ci.yml` sets `CARGO_BUILD_JOBS` at step level for that reason; promoting it to job level rotates the key for every lane that shares it.
 
 ## Where self-hosted minutes leak
@@ -48,7 +54,7 @@ A hosted runner is also the only choice for a few things the pods cannot do: `mo
 These are the patterns that have cost real money here without validating anything more:
 
 - An `if: always()` summary or alert job on a self-hosted label inside a `workflow_run`-triggered workflow. The upstream workflow completes many times a day, the real jobs skip, and the summary still starts a pod each time. Gate it on the upstream job having run, and keep it on `ubuntu-latest`.
-- Per-push workflows on `main`. A merge triggers the full `ci.yml` matrix, `cache-warm.yml` and the `build.yml` development build; with tens of merges a day that is as much self-hosted time as all pull requests together. Anything that does not verify the merged tree itself belongs on a schedule or behind a `paths` filter.
+- Per-push workflows on `main`. A merge triggers cache warming and a `ci.yml` run, which validates the merged tree in full unless a merge-queue run already validated that tree; `build.yml` development builds are scheduled or manual; with tens of merges a day that is as much self-hosted time as all pull requests together. Anything that does not verify the merged tree itself belongs on a schedule or behind a `paths` filter.
 - `apt-get install` of tools the hosted image already ships. It is only needed on the minimal pod images.
 - A throttled `CARGO_BUILD_JOBS` on a 14-CPU pod. The limit exists for link-storm reasons documented in `ci.yml`; measure before raising it, and measure before lowering it further.
 - Tiny jobs on ARC pods. Each one pays roughly a minute of pod start-up before its first step; a compile-free job that takes ten seconds costs more to schedule than to run.
@@ -62,7 +68,7 @@ gh api "repos/rustfs/rustfs/actions/runs/<run_id>/jobs?per_page=100" \
   --jq '.jobs[] | "\(.conclusion)\t\(.labels | join(","))\t\(.started_at)\t\(.completed_at)\t\(.name)"'
 ```
 
-Only rows whose label starts with `sm-`, `dind-`, `smoke-` or `pf-` cost anything. At the October 2026 review, a full pull-request run of `ci.yml` spent about 150 self-hosted minutes, a third of it in jobs that never compiled; treat that as the scale of what a placement mistake costs, not as a current figure.
+Rows using `macos-latest-xlarge` or a label starting with `sm-`, `dind-`, `ubicloud-`, `smoke-` or `pf-` use paid capacity. At the October 2026 review, a full pull-request run of `ci.yml` spent about 150 self-hosted minutes, a third of it in jobs that never compiled; treat that as the scale of what a placement mistake costs, not as a current figure.
 
 ## Checklist for a new or moved job
 
@@ -72,3 +78,9 @@ Only rows whose label starts with `sm-`, `dind-`, `smoke-` or `pf-` cost anythin
 - A self-hosted `runs-on` carries a comment naming its reason from the placement rule.
 - Every `./.github/actions/setup` call states `cache-save-if`, and a key read from one pool is not written from the other.
 - Moving a job does not change its check name, so the `main` ruleset and `docs/testing/ci-gates.md` stay as they are; renaming it does, and then that document's change checklist applies.
+
+## Release stages
+
+The [candidate release pipeline](../../docs/operations/release-pipeline.md) builds Linux first and defers paid desktop capacity until acceptance. The daily `build.yml` schedule selects Linux; the Sunday 16:17 UTC schedule selects macOS and Windows. Manual dispatch can still select any platform list. All six platforms remain required before publication. The pipeline records and promotes the exact tested bytes. Builder workflows have no public-release credentials. Keep periodic development builds, PR verification and field-test chains separate when their contracts differ; do not re-enable disabled workflows as part of a source-file migration.
+
+Larger runners remain billed for public repositories; see [GitHub runner billing](https://docs.github.com/en/actions/concepts/runners/larger-runners#billing). Actual spending comes from organization billing, not runner labels or an estimated build duration.

.github/workflows/audit.yml

@@ -23,13 +23,19 @@ on:
       - 'deny.toml'
       - '.github/actions/**'
       - '.github/workflows/**'
-      - 'scripts/release/create_or_update_release.sh'
-      - 'scripts/release/package_service_scripts.sh'
-      - 'scripts/release/package_versions.sh'
+      - 'scripts/release/**'
+      - 'scripts/security/generate_release_supply_chain_assets.py'
+      - '.config/release-validation-plan.json'
+      - '.github/scheduled-validations.json'
+      - 'scripts/test_helm_release_workflow.py'
+      - 'scripts/test_release_packages.py'
+      - 'scripts/test_release_images.py'
+      - 'Dockerfile*'
+      - 'entrypoint.sh'
+      - 'bootstrap/**'
       - 'scripts/test_package_service_scripts.sh'
       - 'scripts/test_package_versions.sh'
       - 'scripts/test_docker_workflow.py'
-      - 'scripts/security/check_performance_ab_workflow.sh'
       - 'scripts/security/check_preview_release_workflow.sh'
       - 'scripts/security/check_tier_artifact_workflow.sh'
       - 'scripts/security/check_workflow_pins.sh'
@@ -42,13 +48,19 @@ on:
       - 'deny.toml'
       - '.github/actions/**'
       - '.github/workflows/**'
-      - 'scripts/release/create_or_update_release.sh'
-      - 'scripts/release/package_service_scripts.sh'
-      - 'scripts/release/package_versions.sh'
+      - 'scripts/release/**'
+      - 'scripts/security/generate_release_supply_chain_assets.py'
+      - '.config/release-validation-plan.json'
+      - '.github/scheduled-validations.json'
+      - 'scripts/test_helm_release_workflow.py'
+      - 'scripts/test_release_packages.py'
+      - 'scripts/test_release_images.py'
+      - 'Dockerfile*'
+      - 'entrypoint.sh'
+      - 'bootstrap/**'
       - 'scripts/test_package_service_scripts.sh'
       - 'scripts/test_package_versions.sh'
       - 'scripts/test_docker_workflow.py'
-      - 'scripts/security/check_performance_ab_workflow.sh'
       - 'scripts/security/check_preview_release_workflow.sh'
       - 'scripts/security/check_tier_artifact_workflow.sh'
       - 'scripts/security/check_workflow_pins.sh'
@@ -134,12 +146,38 @@ jobs:
     if: github.repository == 'rustfs/rustfs' && (github.event_name != 'pull_request' || github.event.action != 'closed')
     runs-on: ubuntu-latest
     timeout-minutes: 5
+    permissions:
+      contents: read
+      actions: read
+    outputs:
+      release_changed: ${{ steps.scope.outputs.release_changed }}
     steps:
       - name: Checkout repository
         uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
         with:
+          fetch-depth: 0
           persist-credentials: false
 
+      - name: Scope native packaging checks to release changes
+        id: scope
+        env:
+          BASE_SHA: ${{ github.event.pull_request.base.sha || github.event.before }}
+        run: |
+          python3 - <<'PY'
+          import os, re, subprocess
+          event, base = os.environ['GITHUB_EVENT_NAME'], os.environ.get('BASE_SHA', '')
+          changed = event == 'workflow_dispatch'
+          if event in ('push', 'pull_request'):
+              if not re.fullmatch(r'[0-9a-f]{40}', base) or base == '0' * 40:
+                  changed = True
+              else:
+                  paths = subprocess.check_output(['git', 'diff', '--name-only', base, os.environ['GITHUB_SHA']], text=True).splitlines()
+                  pattern = r'^(scripts/release/|scripts/test_release_|Dockerfile|entrypoint.sh$|bootstrap/|\.dockerignore$|\.config/release-validation-plan.json$|\.github/workflows/(audit|build|docker|package|helm-package|release-[^.]+)\.yml$)'
+                  changed = any(re.match(pattern, path) for path in paths)
+          with open(os.environ['GITHUB_OUTPUT'], 'a') as output:
+              output.write(f'release_changed={str(changed).lower()}\n')
+          PY
+
       - name: Report unpinned GitHub Actions
         run: ./scripts/security/check_workflow_pins.sh --enforce
 
@@ -152,11 +190,35 @@ jobs:
       - name: Check checkouts clear their credentials
         run: ./scripts/security/check_persist_credentials.sh
 
-      - name: Check preview release workflow policy
-        run: ./scripts/security/check_preview_release_workflow.sh
+      - name: Check release candidate and promotion contracts
+        run: python3 -m unittest discover -s scripts/release -p 'test_*.py'
 
-      - name: Check performance A/B workflow trust boundary
-        run: ./scripts/security/check_performance_ab_workflow.sh
+      - name: Prepare artifact transport rehearsal
+        run: |
+          mkdir -p "$RUNNER_TEMP/release-transport"
+          printf '{"purpose":"transport contract, not release evidence"}\n' > "$RUNNER_TEMP/release-transport/transport.json"
+
+      - uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+        with:
+          name: release-transport-contract-${{ github.run_attempt }}
+          path: ${{ runner.temp }}/release-transport/transport.json
+          if-no-files-found: error
+          retention-days: 1
+
+      - name: Verify actual GitHub artifact metadata and downloaded bytes
+        env:
+          GH_TOKEN: ${{ github.token }}
+        run: |
+          python3 - <<'PY'
+          import os, sys
+          from pathlib import Path
+          sys.path.insert(0, 'scripts/release')
+          import candidate as c
+          run = c.github(f"repos/{c.REPOSITORY}/actions/runs/{os.environ['GITHUB_RUN_ID']}")
+          destination = Path(os.environ['RUNNER_TEMP']) / 'release-transport-downloaded'
+          c.download_artifact(run, f"release-transport-contract-{os.environ['GITHUB_RUN_ATTEMPT']}", destination)
+          c.require(c.read_json(destination / 'transport.json') == {'purpose': 'transport contract, not release evidence'}, 'artifact round trip changed')
+          PY
 
       - name: Check tier evidence workflow isolation
         run: ./scripts/security/check_tier_artifact_workflow.sh
@@ -167,6 +229,80 @@ jobs:
       - name: Check package service lifecycle contract
         run: ./scripts/test_package_service_scripts.sh
 
+  release-package-contract:
+    name: Release package contract
+    needs: workflow-pin-report
+    if: needs.workflow-pin-report.outputs.release_changed == 'true'
+    runs-on: ubuntu-latest
+    timeout-minutes: 15
+    steps:
+      - uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
+        with:
+          persist-credentials: false
+      - name: Install real package managers
+        run: |
+          sudo apt-get update
+          sudo apt-get install -y ruby ruby-dev build-essential rpm fakeroot cpio
+          sudo gem install fpm --version 1.17.0 --no-document
+      - name: Exercise accepted and substituted payloads through both recipes
+        run: python3 scripts/test_release_packages.py
+
+  release-image-contract:
+    name: Release image contract (${{ matrix.variant }})
+    needs: workflow-pin-report
+    if: needs.workflow-pin-report.outputs.release_changed == 'true'
+    runs-on: ubuntu-latest
+    timeout-minutes: 35
+    strategy:
+      fail-fast: false
+      matrix:
+        variant: [musl, gnu]
+    env:
+      RELEASE_IMAGE_VARIANT: ${{ matrix.variant }}
+      NO_PROXY: 127.0.0.1,localhost,::1
+    steps:
+      - uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
+        with:
+          persist-credentials: false
+      - name: Install OCI tools
+        run: |
+          echo "RELEASE_IMAGE_OUTPUT=$RUNNER_TEMP/image-contract" >> "$GITHUB_ENV"
+          echo 'no_proxy=127.0.0.1,localhost,::1' >> "$GITHUB_ENV"
+          sudo apt-get update
+          sudo apt-get install -y skopeo
+          go version
+      - uses: docker/setup-qemu-action@c7c53464625b32c7a7e944ae62b3e17d2b600130 # v3
+      - uses: docker/setup-buildx-action@8d2750c68a42422c14e847fe6c8ac0403b4cbd6f # v3
+      - name: Exercise real OCI packaging, startup, registry retry and conflicts
+        run: python3 scripts/test_release_images.py
+      - name: Exercise amd64 OCI archive scanning
+        uses: aquasecurity/trivy-action@ed142fd0673e97e23eac54620cfb913e5ce36c25
+        env:
+          TRIVY_PLATFORM: linux/amd64
+        with:
+          input: ${{ runner.temp }}/image-contract-scan/image-${{ matrix.variant }}-amd64.tar
+          format: sarif
+          output: ${{ runner.temp }}/image-contract/scan-${{ matrix.variant }}-amd64.sarif
+          exit-code: "0"
+      - name: Exercise arm64 OCI archive scanning
+        uses: aquasecurity/trivy-action@ed142fd0673e97e23eac54620cfb913e5ce36c25
+        env:
+          TRIVY_PLATFORM: linux/arm64
+        with:
+          input: ${{ runner.temp }}/image-contract-scan/image-${{ matrix.variant }}-arm64.tar
+          format: sarif
+          output: ${{ runner.temp }}/image-contract/scan-${{ matrix.variant }}-arm64.sarif
+          exit-code: "0"
+      - name: Verify scan receipts before recording the archive
+        run: |
+          python3 - <<'PY'
+          import os, sys
+          from pathlib import Path
+          sys.path.insert(0, 'scripts/release')
+          import deliver, test_candidate
+          deliver.image_record(test_candidate.inputs(), os.environ['RELEASE_IMAGE_VARIANT'], Path(os.environ['RELEASE_IMAGE_OUTPUT']))
+          PY
+
   dependency-review:
     name: Dependency Review
     runs-on: ubuntu-latest

.github/workflows/btf-prototype-reference.yml

@@ -0,0 +1,75 @@
+name: BTF prototype reference
+
+on:
+  pull_request:
+    paths:
+      - "scripts/inspect_btf_prototypes.py"
+      - "scripts/test_inspect_btf_prototypes.py"
+      - "scripts/fixtures/btf_prototypes.c"
+      - ".github/workflows/btf-prototype-reference.yml"
+  workflow_dispatch:
+
+permissions:
+  contents: read
+
+concurrency:
+  group: btf-prototype-reference-${{ github.ref }}
+  cancel-in-progress: true
+
+jobs:
+  reference:
+    runs-on: ubuntu-latest
+    timeout-minutes: 10
+    steps:
+      - uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
+        with:
+          persist-credentials: false
+      - uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
+        with:
+          python-version: "3.12"
+      - name: Check prototype and CLI contracts
+        run: python3 -B scripts/test_inspect_btf_prototypes.py
+      - name: Install only missing native reference tools
+        run: |
+          select_bpftool() {
+            local candidate
+            local candidates=()
+            if command -v bpftool >/dev/null; then
+              candidates+=("$(command -v bpftool)")
+            fi
+            for candidate in /usr/lib/linux-tools-*/bpftool /usr/lib/linux-tools/*/bpftool; do
+              if [[ -x "$candidate" ]]; then
+                candidates+=("$candidate")
+              fi
+            done
+            while IFS= read -r candidate; do
+              if [[ -x "$candidate" ]] && file -Lb "$candidate" | grep -q 'ELF' \
+                && "$candidate" version >/dev/null 2>&1; then
+                printf '%s\n' "$candidate"
+                return 0
+              fi
+            done < <(printf '%s\n' "${candidates[@]}" | sort -Vr)
+            return 1
+          }
+          packages=()
+          command -v clang >/dev/null || packages+=(clang)
+          command -v llvm-objcopy >/dev/null || packages+=(llvm)
+          command -v gcc >/dev/null || packages+=(gcc)
+          select_bpftool >/dev/null || packages+=(linux-tools-generic)
+          if (( ${#packages[@]} )); then
+            sudo apt-get update
+            sudo apt-get install --no-install-recommends -y "${packages[@]}"
+          fi
+          # File parsing needs a working binary, not a host-kernel wrapper.
+          selected_bpftool="$(select_bpftool)"
+          "$selected_bpftool" version
+          printf 'BPFTOOL=%s\n' "$selected_bpftool" >> "$GITHUB_ENV"
+      - name: Compare declarations and layouts with independent references
+        run: |
+          clang -target bpf -O2 -g -c scripts/fixtures/btf_prototypes.c -o "$RUNNER_TEMP/prototypes.o"
+          llvm-objcopy --dump-section ".BTF=$RUNNER_TEMP/prototypes.btf" "$RUNNER_TEMP/prototypes.o"
+          gcc -std=c11 -Wall -Wextra -Werror -DRUSTFS_BTF_LAYOUT_REFERENCE \
+            scripts/fixtures/btf_prototypes.c -o "$RUNNER_TEMP/layout-reference"
+          "$RUNNER_TEMP/layout-reference" > "$RUNNER_TEMP/layout-reference.json"
+          python3 -B scripts/test_inspect_btf_prototypes.py --native-reference "$RUNNER_TEMP/prototypes.btf" \
+            --layout-reference "$RUNNER_TEMP/layout-reference.json"

.github/workflows/build.yml

@@ -12,51 +12,34 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-# Build and Release Workflow
-#
-# This workflow builds RustFS binaries and automatically triggers Docker image builds.
-#
-# Flow:
-# 1. Build binaries for multiple platforms
-# 2. Upload binaries to OSS storage
-# 3. Trigger docker.yml to build and push images using the uploaded binaries
-#
-# Platform scope:
-# - Development builds come from the nightly schedule and manual dispatch and
-#   cover the full platform matrix; they feed the dev download channel
-#   (artifacts/rustfs/dev on R2, the *-main-latest packages). Tags build the
-#   release matrix. Main pushes do not build: at the usual merge rate most
-#   per-merge builds were cancelled by the next merge before publishing
-#   anything, and the dev channel does not need per-merge granularity.
-#
-# Manual Parameters:
-# - publish: Publish binaries/releases for manual runs (default: false)
-# - build_docker: Advisory only; manual runs do not trigger Docker publishing
-# - platforms: Comma-separated platform IDs or 'all' (default: all)
+# Scheduled development artifacts and reusable immutable-candidate builds.
+# The daily cron builds Linux. Sunday 16:17 UTC builds macOS and Windows.
+# Public release writes live exclusively in release-publish.yml. Candidate callers
+# select Linux or desktop and pin release inputs before any matrix starts.
 
 name: Build and Release
 
 on:
-  push:
-    tags: [ "*.*.*" ]
+  workflow_call:
+    inputs:
+      candidate_manifest:
+        description: Frozen candidate inputs; candidates never publish from this workflow
+        required: true
+        type: string
+      platforms:
+        description: linux, desktop, all, or an explicit platform list
+        required: true
+        type: string
   schedule:
-    - cron: "13 1 * * *" # Nightly 01:13 UTC
+    - cron: "13 1 * * *" # Daily Linux, 01:13 UTC
+    - cron: "17 16 * * 0" # Weekly macOS and Windows, Sunday 16:17 UTC
   workflow_dispatch:
     inputs:
       publish:
-        description: "Publish to download channels and GitHub Releases (false keeps only workflow artifacts)"
+        description: "Publish the development download channel (never a release)"
         required: false
         default: false
         type: boolean
-      build_docker:
-        # Advisory only. docker.yml triggers on workflow_run and its job-level
-        # condition requires the triggering event to be a tag push, so a manual
-        # dispatch of this workflow never produces images regardless of this
-        # value. Publishing remains controlled by the explicit publish input.
-        description: "Build and push Docker images after binary build (ignored: dispatch runs never reach docker.yml)"
-        required: false
-        default: true
-        type: boolean
       platforms:
         description: "Comma-separated targets or 'all' (e.g. linux-x86_64-musl,macos-aarch64)"
         required: false
@@ -65,10 +48,11 @@ on:
 
 permissions:
   contents: read
+  actions: read
 
 concurrency:
   group: ${{ github.workflow }}-${{ github.event_name == 'push' && github.ref || github.run_id }}
-  cancel-in-progress: ${{ github.event_name == 'push' }}
+  cancel-in-progress: false
 
 env:
   CARGO_TERM_COLOR: always
@@ -84,19 +68,39 @@ jobs:
     timeout-minutes: 30
     outputs:
       should_build: ${{ steps.check.outputs.should_build }}
-      should_publish: ${{ github.event_name != 'workflow_dispatch' || inputs.publish == true }}
+      should_publish: ${{ inputs.candidate_manifest == '' && github.ref == 'refs/heads/main' && (github.event_name == 'schedule' || inputs.publish == true) }}
       build_type: ${{ steps.check.outputs.build_type }}
       version: ${{ steps.check.outputs.version }}
       short_sha: ${{ steps.check.outputs.short_sha }}
       is_prerelease: ${{ steps.check.outputs.is_prerelease }}
+      candidate_id: ${{ steps.candidate.outputs.candidate_id }}
     steps:
       - name: Checkout repository
         uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
         with:
           persist-credentials: false
+          ref: ${{ inputs.candidate_manifest != '' && fromJson(inputs.candidate_manifest).source_sha || github.sha }}
+
+      - name: Validate frozen candidate
+        if: inputs.candidate_manifest != ''
+        id: candidate
+        env:
+          CANDIDATE_MANIFEST: ${{ inputs.candidate_manifest }}
+        run: |
+          python3 - <<'PYINPUT'
+          import json, os, sys
+          sys.path.insert(0, 'scripts/release')
+          import candidate
+          value = candidate.validate_inputs(json.loads(os.environ['CANDIDATE_MANIFEST']))
+          candidate.write_json('candidate-inputs.json', value)
+          candidate.emit_output('candidate_id', value['candidate_id'])
+          PYINPUT
+          python3 scripts/release/candidate.py validate-inputs --inputs candidate-inputs.json --check-checkout
 
       - name: Determine build strategy
         id: check
+        env:
+          CANDIDATE_MANIFEST: ${{ inputs.candidate_manifest }}
         run: |
           should_build=false
           build_type="none"
@@ -107,8 +111,14 @@ jobs:
           # Get short SHA for all builds
           short_sha=$(git rev-parse --short HEAD)
 
+          # Candidates contain the final product version but have no release
+          # authority. Their only publication is immutable workflow artifacts.
+          if [[ -n "$CANDIDATE_MANIFEST" ]]; then
+            should_build=true
+            build_type="candidate"
+            version=$(python3 -c 'import json; print(json.load(open("candidate-inputs.json"))["version"])')
           # Determine build type based on trigger
-          if [[ "${{ startsWith(github.ref, 'refs/tags/') }}" == "true" ]]; then
+          elif [[ "${{ startsWith(github.ref, 'refs/tags/') }}" == "true" ]]; then
             # Tag push - preview, release, or prerelease
             should_build=true
             tag_name="${GITHUB_REF#refs/tags/}"
@@ -155,6 +165,46 @@ jobs:
           echo "  - Short SHA: $short_sha"
           echo "  - Is prerelease: $is_prerelease"
 
+  prepare-build-inputs:
+    needs: build-check
+    if: needs.build-check.outputs.should_build == 'true'
+    runs-on: ubuntu-latest
+    timeout-minutes: 15
+    outputs:
+      console: ${{ steps.inputs.outputs.console }}
+      toolchain: ${{ steps.inputs.outputs.toolchain }}
+    steps:
+      - uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
+        with:
+          persist-credentials: false
+          ref: ${{ inputs.candidate_manifest != '' && fromJson(inputs.candidate_manifest).source_sha || github.sha }}
+      - name: Resolve one Console and toolchain identity
+        id: inputs
+        env:
+          GH_TOKEN: ${{ github.token }}
+          CANDIDATE_MANIFEST: ${{ inputs.candidate_manifest }}
+        shell: bash
+        run: |
+          set -euo pipefail
+          if [[ -z "$CANDIDATE_MANIFEST" ]]; then
+            gh api repos/rustfs/console/releases/latest > console-release.json
+            rustup update stable --no-self-update
+          fi
+          python3 - <<'PYINPUT'
+          import json, os, subprocess, sys
+          sys.path.insert(0, 'scripts/release')
+          import candidate
+          raw = os.environ['CANDIDATE_MANIFEST']
+          if raw:
+              frozen = candidate.validate_inputs(json.loads(raw))
+              console, toolchain = frozen['console'], frozen['toolchain']
+          else:
+              console = candidate.console_asset(candidate.read_json('console-release.json'))
+              toolchain = subprocess.check_output(['rustc', '+stable', '--version'], text=True).split()[1]
+          candidate.emit_output('console', json.dumps(console, separators=(',', ':')))
+          candidate.emit_output('toolchain', toolchain)
+          PYINPUT
+
   # Build RustFS binaries
   prepare-platform-matrix:
     name: Prepare Platform Matrix
@@ -171,26 +221,55 @@ jobs:
         env:
           # via env, not interpolation: a dispatch input is free-form text and
           # would otherwise be pasted into the script for bash to evaluate.
-          RAW_PLATFORMS: ${{ github.event_name == 'workflow_dispatch' && github.event.inputs.platforms || 'all' }}
+          # A direct schedule has no platforms input. Choose its subset from
+          # github.event.schedule, which is fixed when that cron fires, so a
+          # queued job cannot switch subsets because the runner clock moved.
+          # Callers that pass platforms keep that list, including a scheduled
+          # caller: inputs.platforms is set even though event_name stays schedule.
+          RAW_PLATFORMS: ${{ inputs.platforms || 'linux' }}
+          HAS_PLATFORM_INPUT: ${{ inputs.platforms && 'true' || 'false' }}
+          EVENT_NAME: ${{ github.event_name }}
+          EVENT_SCHEDULE: ${{ github.event.schedule }}
         run: |
           set -euo pipefail
 
-          selected="$RAW_PLATFORMS"
-          selected="$(echo "${selected}" | tr -d '[:space:]')"
-          if [[ -z "${selected}" ]]; then
-            selected="all"
+          if [[ "${EVENT_NAME}" == "schedule" && "${HAS_PLATFORM_INPUT}" != "true" ]]; then
+            if [[ "${EVENT_SCHEDULE}" == "13 1 * * *" ]]; then
+              selected="linux"
+            elif [[ "${EVENT_SCHEDULE}" == "17 16 * * 0" ]]; then
+              selected="desktop"
+            else
+              echo "Unrecognized schedule cron: ${EVENT_SCHEDULE}" >&2
+              exit 1
+            fi
+          else
+            selected="${RAW_PLATFORMS}"
+            selected="$(echo "${selected}" | tr -d '[:space:]')"
+            if [[ -z "${selected}" ]]; then
+              selected="all"
+            fi
           fi
 
-
+          # linux-aarch64-* need a real aarch64 host (placement rule: hardware)
+          # so the packaged binary and console smoke execute the artifact.
+          # ubicloud-standard-16-arm is that host: Ubuntu 24.04 arm64, alias of
+          # ubicloud-standard-16-arm-ubuntu-2404 (16 vCPU, 48 GB). linux-x86_64-* stay on the
+          # ARC sm-standard-4 pool. GNU rows keep cross=true so cargo-zigbuild
+          # pins the glibc 2.31 floor on the same architecture; musl stays a
+          # native cargo build.
           all='{"include":[
             {"target_id":"linux-x86_64-musl","os":"sm-standard-4","target":"x86_64-unknown-linux-musl","cross":false,"platform":"linux","rustflags":"","features":"","glibc":""},
-            {"target_id":"linux-aarch64-musl","os":"sm-standard-4-arm","target":"aarch64-unknown-linux-musl","cross":false,"platform":"linux","rustflags":"","features":"","glibc":""},
+            {"target_id":"linux-aarch64-musl","os":"ubicloud-standard-16-arm","target":"aarch64-unknown-linux-musl","cross":false,"platform":"linux","rustflags":"","features":"","glibc":""},
             {"target_id":"linux-x86_64-gnu","os":"sm-standard-4","target":"x86_64-unknown-linux-gnu","cross":true,"platform":"linux","rustflags":"","features":"pyroscope","glibc":"2.31"},
-            {"target_id":"linux-aarch64-gnu","os":"sm-standard-4-arm","target":"aarch64-unknown-linux-gnu","cross":true,"platform":"linux","rustflags":"","features":"pyroscope","glibc":"2.31"},
-            {"target_id":"macos-aarch64","os":"macos-latest","target":"aarch64-apple-darwin","cross":false,"platform":"macos","rustflags":"","features":"pyroscope","glibc":""},
+            {"target_id":"linux-aarch64-gnu","os":"ubicloud-standard-16-arm","target":"aarch64-unknown-linux-gnu","cross":true,"platform":"linux","rustflags":"","features":"pyroscope","glibc":"2.31"},
+            {"target_id":"macos-aarch64","os":"macos-latest-xlarge","target":"aarch64-apple-darwin","cross":false,"platform":"macos","rustflags":"","features":"pyroscope","glibc":""},
             {"target_id":"windows-x86_64","os":"windows-latest","target":"x86_64-pc-windows-msvc","cross":false,"platform":"windows","rustflags":"","features":"","glibc":""}
           ]}'
 
+          if [[ "$selected" == "linux" || "$selected" == "desktop" ]]; then
+            selected=$(jq -r --arg group "$selected" '.include[] | select(if $group == "linux" then .platform == "linux" else .platform != "linux" end) | .target_id' <<<"$all" | paste -sd ',' -)
+          fi
+
           if [[ "${selected}" == "all" ]]; then
             matrix="$(jq -c . <<<"${all}")"
           else
@@ -218,12 +297,13 @@ jobs:
   build-rustfs:
     # Acceptance workflows select artifacts by this stable platform name.
     name: Build RustFS (${{ matrix.target_id }})
-    needs: [ build-check, prepare-platform-matrix ]
+    needs: [ build-check, prepare-platform-matrix, prepare-build-inputs ]
     if: needs.build-check.outputs.should_build == 'true' && needs.prepare-platform-matrix.result == 'success'
     runs-on: ${{ matrix.os }}
     timeout-minutes: 180
     env:
       FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"
+      RUSTUP_TOOLCHAIN: ${{ needs.prepare-build-inputs.outputs.toolchain }}
       # Release binaries ship without dial9 telemetry and therefore do not need
       # --cfg tokio_unstable. Telemetry builds opt in explicitly (see
       # `make build-profiling`); crates/obs/build.rs fails the compile if the
@@ -238,13 +318,39 @@ jobs:
         with:
           persist-credentials: false
           fetch-depth: 0
+          ref: ${{ inputs.candidate_manifest != '' && fromJson(inputs.candidate_manifest).source_sha || github.sha }}
+
+      - name: Ensure Python for artifact verification
+        if: runner.os == 'Linux'
+        shell: bash
+        run: |
+          if ! command -v python3 >/dev/null 2>&1; then
+            sudo apt-get update
+            sudo apt-get install -y python3
+          fi
+
+      - name: Reuse completed candidate platform
+        id: reuse
+        if: needs.build-check.outputs.build_type == 'candidate'
+        env:
+          GH_TOKEN: ${{ github.token }}
+          CANDIDATE_MANIFEST: ${{ inputs.candidate_manifest }}
+          TARGET: ${{ matrix.target_id }}
+        shell: bash
+        run: |
+          python3 scripts/release/pipeline.py reuse-platform --target "$TARGET"
 
       - name: Setup Rust environment
+        if: steps.reuse.outputs.reused != 'true'
         uses: ./.github/actions/setup
         with:
-          rust-version: stable
+          rust-version: ${{ needs.prepare-build-inputs.outputs.toolchain }}
           target: ${{ matrix.target }}
-          cache-shared-key: build-${{ matrix.target }}${{ matrix.glibc && format('-glibc-{0}', matrix.glibc) || '' }}
+          # rust-cache keys include OS and architecture but not the runner image.
+          # The Ubicloud aarch64 legs therefore use their own suffix so they
+          # cannot restore a cache written on a different arm64 image.
+          # x86_64, macOS, and Windows keys stay unsuffixed.
+          cache-shared-key: build-${{ matrix.target }}${{ matrix.glibc && format('-glibc-{0}', matrix.glibc) || '' }}${{ matrix.os == 'ubicloud-standard-16-arm' && '-ubicloud-arm' || '' }}
           # main only. A cache saved on refs/tags/X is scoped to that tag: no
           # other tag, no main run and no PR can restore it, so every release
           # cycle wrote up to 12 entries of 1-2GB (preview tag plus final tag,
@@ -258,128 +364,35 @@ jobs:
           install-test-tools: 'false'
 
       - name: Download static console assets
+        if: steps.reuse.outputs.reused != 'true'
         shell: bash
         env:
-          GITHUB_TOKEN: ${{ github.token }}
+          GH_TOKEN: ${{ github.token }}
+          CONSOLE_IDENTITY: ${{ needs.prepare-build-inputs.outputs.console }}
         run: |
-          mkdir -p ./rustfs/static
-
-          verify_sha256() {
-            local expected="$1"
-            local file="$2"
-            local actual=""
-            local actual_line=""
-
-            if command -v sha256sum >/dev/null 2>&1; then
-              actual_line=$(sha256sum -- "$file") || return 1
-            elif command -v shasum >/dev/null 2>&1; then
-              actual_line=$(shasum -a 256 -- "$file") || return 1
-            else
-              echo "No SHA-256 verification tool found" >&2
-              return 1
-            fi
-            actual="${actual_line%%[[:space:]]*}"
-
-            if [ "$actual" = "$expected" ]; then
-              echo "SHA256 verified OK: $actual"
-              return 0
-            else
-              echo "SHA256 mismatch: expected=$expected actual=$actual" >&2
-              return 1
-            fi
-          }
-
-          download_console_assets() {
-            local curl_bin="curl"
-            local python_bin="python3"
-            local console_api="https://api.github.com/repos/rustfs/console/releases/latest"
-            local console_json="console-release.json"
-            local console_url
-            local console_sha256
-            local curl_auth_args=()
-            console_tag=""
-
-            if [[ "${{ matrix.platform }}" == "windows" ]]; then
-              curl_bin="curl.exe"
-            else
-              chmod +w ./rustfs/static/LICENSE || true
-            fi
-
-            if ! command -v "$python_bin" >/dev/null 2>&1; then
-              python_bin="python"
-            fi
-            if ! command -v "$python_bin" >/dev/null 2>&1; then
-              echo "No Python interpreter found for release metadata parsing" >&2
-              return 1
-            fi
-
-            if [[ -n "${GITHUB_TOKEN:-}" ]]; then
-              curl_auth_args=(-H "Authorization: Bearer ${GITHUB_TOKEN}" -H "X-GitHub-Api-Version: 2022-11-28")
-            fi
-
-            "$curl_bin" "${curl_auth_args[@]}" --fail -L "$console_api" \
-              -o "$console_json" --retry 3 --retry-delay 5 --max-time 300 || return 1
-
-            read -r console_tag console_url console_sha256 < <("$python_bin" - "$console_json" <<'PY'
-          import json
-          import re
-          import sys
-
-          with open(sys.argv[1], encoding="utf-8") as handle:
-              release = json.load(handle)
-
-          tag = release.get("tag_name", "") or "unknown"
-
-          for asset in release.get("assets", []):
-              name = asset.get("name", "")
-              digest = asset.get("digest", "")
-              url = asset.get("browser_download_url", "")
-              if name.startswith("rustfs-console-") and name.endswith(".zip") and digest.startswith("sha256:") and url:
-                  sha256 = digest.split(":", 1)[1]
-                  if not re.fullmatch(r"[0-9a-fA-F]{64}", sha256):
-                      raise SystemExit(f"console zip asset has invalid sha256 digest: {sha256}")
-                  sys.stdout.buffer.write(f"{tag} {url} {sha256}\n".encode("utf-8"))
-                  break
-          else:
-              raise SystemExit("no console zip asset with sha256 digest found")
-          PY
-            ) || return 1
-
-            if [[ -z "$console_url" || -z "$console_sha256" ]]; then
-              echo "Console release metadata is missing URL or SHA-256 digest" >&2
-              return 1
-            fi
-
-            echo "Console release: ${console_tag}"
-            echo "Downloading console asset: ${console_url}"
-            "$curl_bin" --fail -L "$console_url" -o console.zip --retry 3 --retry-delay 5 --max-time 300 || return 1
-            verify_sha256 "$console_sha256" console.zip || return 2
-            unzip -o console.zip -d ./rustfs/static || return 2
-          }
-
-          if download_console_assets; then
-            rm -f console.zip console-release.json
-          else
-            status=$?
-            rm -f console.zip console-release.json
-            if [[ "$status" -eq 2 ]]; then
-              echo "Console asset integrity verification failed" >&2
-            fi
-            echo "Failed to download verified console assets" >&2
-            exit 1
-          fi
-
-          if [[ ! -s ./rustfs/static/index.html ]]; then
-            echo "Console asset archive is missing static/index.html" >&2
-            exit 1
-          fi
-
-          asset_count=$(find ./rustfs/static -type f | wc -l | tr -d '[:space:]')
-          echo "Console assets ready: version=${console_tag:-unknown}, ${asset_count} files extracted to ./rustfs/static"
+          python3 - <<'PYCONSOLE'
+          import json, os, pathlib, subprocess, zipfile
+          value = json.loads(os.environ['CONSOLE_IDENTITY'])
+          subprocess.run(['python3', '.github/actions/setup/download_release_asset.py',
+              '--repository', 'rustfs/console', '--tag', value['tag'], '--asset', value['name'],
+              '--sha256', value['sha256'], '--output', 'console.zip'], check=True)
+          destination = pathlib.Path('rustfs/static')
+          destination.mkdir(parents=True, exist_ok=True)
+          with zipfile.ZipFile('console.zip') as archive:
+              for member in archive.infolist():
+                  path = pathlib.PurePosixPath(member.filename)
+                  if path.is_absolute() or '..' in path.parts or '\\' in member.filename:
+                      raise SystemExit('unsafe Console archive member')
+              archive.extractall(destination)
+          if not (destination / 'index.html').is_file():
+              raise SystemExit('Console archive has no index.html')
+          PYCONSOLE
 
       - name: Build RustFS
+        if: steps.reuse.outputs.reused != 'true'
         shell: bash
         env:
+          RUSTFS_BUILD_VERSION: ${{ needs.build-check.outputs.build_type == 'candidate' && needs.build-check.outputs.version || '' }}
           BUILD_TARGET: ${{ matrix.target }}
           BUILD_OS: ${{ matrix.platform }}-${{ startsWith(matrix.target, 'aarch64-') && 'aarch64' || 'x86_64' }}
         run: |
@@ -430,7 +443,7 @@ jobs:
           fi
 
       - name: Verify macOS binary has no dynamic liblzma dependency
-        if: matrix.platform == 'macos'
+        if: steps.reuse.outputs.reused != 'true' && (matrix.platform == 'macos')
         shell: bash
         run: |
           binary="target/${{ matrix.target }}/release/rustfs"
@@ -441,7 +454,7 @@ jobs:
           "$binary" --version
 
       - name: Generate the CPU symbol catalog
-        if: endsWith(matrix.target, '-linux-gnu') && matrix.features == 'pyroscope'
+        if: steps.reuse.outputs.reused != 'true' && (endsWith(matrix.target, '-linux-gnu') && matrix.features == 'pyroscope')
         shell: bash
         run: |
           cargo metadata --locked --format-version 1 --filter-platform '${{ matrix.target }}' \
@@ -458,6 +471,7 @@ jobs:
             "$release_dir/rustfs.cpu-symbol-catalog.json" > "$release_dir/rustfs.cpu-symbol-catalog.sha256"
 
       - name: Create release package
+        if: steps.reuse.outputs.reused != 'true'
         id: package
         shell: bash
         run: |
@@ -658,7 +672,7 @@ jobs:
           echo "📊 Version: ${VERSION}"
 
       - name: Verify packaged Linux binary
-        if: matrix.platform == 'linux'
+        if: steps.reuse.outputs.reused != 'true' && (matrix.platform == 'linux')
         shell: bash
         run: |
           set -euo pipefail
@@ -687,7 +701,7 @@ jobs:
           "$binary_dir/rustfs" --version
 
       - name: Verify packaged console
-        if: matrix.target == 'x86_64-unknown-linux-gnu' || matrix.target == 'aarch64-unknown-linux-gnu'
+        if: steps.reuse.outputs.reused != 'true' && (matrix.target == 'x86_64-unknown-linux-gnu' || matrix.target == 'aarch64-unknown-linux-gnu')
         shell: bash
         run: |
           set -euo pipefail
@@ -740,15 +754,49 @@ jobs:
           cat "$log_file"
           exit 1
 
+      - name: Verify native candidate and retain receipts
+        if: needs.build-check.outputs.build_type == 'candidate' && steps.reuse.outputs.reused != 'true'
+        env:
+          CANDIDATE_MANIFEST: ${{ inputs.candidate_manifest }}
+          TARGET: ${{ matrix.target_id }}
+          ARCHIVE: ${{ steps.package.outputs.package_file }}
+        shell: bash
+        run: |
+          python3 -c 'import os; from pathlib import Path; Path("inputs.json").write_text(os.environ["CANDIDATE_MANIFEST"])'
+          python3 scripts/release/verify.py native --inputs inputs.json --archive "$ARCHIVE" --target "$TARGET" --output .
+
+      - name: Record candidate artifact identity
+        if: steps.reuse.outputs.reused != 'true' && (needs.build-check.outputs.build_type == 'candidate')
+        env:
+          CANDIDATE_MANIFEST: ${{ inputs.candidate_manifest }}
+          TARGET_ID: ${{ matrix.target_id }}
+          PACKAGE_FILE: ${{ steps.package.outputs.package_file }}
+        shell: bash
+        run: |
+          python3 - <<'PYRECORD'
+          import json, os, sys
+          sys.path.insert(0, 'scripts/release')
+          import candidate
+          value = candidate.validate_inputs(json.loads(os.environ['CANDIDATE_MANIFEST']))
+          record = candidate.archive_record(os.environ['PACKAGE_FILE'], os.environ['TARGET_ID'], value)
+          candidate.write_json('record-' + os.environ['TARGET_ID'] + '.json', record)
+          PYRECORD
+
       - name: Upload to GitHub artifacts
+        if: steps.reuse.outputs.reused != 'true'
         uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
         with:
-          name: ${{ steps.package.outputs.package_name }}
-          path: "rustfs-*.zip"
-          retention-days: ${{ startsWith(github.ref, 'refs/tags/') && 30 || 7 }}
+          name: ${{ needs.build-check.outputs.build_type == 'candidate' && format('candidate-{0}-{1}', needs.build-check.outputs.candidate_id, matrix.target_id) || steps.package.outputs.package_name }}
+          path: |
+            rustfs-*.zip
+            record-${{ matrix.target_id }}.json
+            identity-${{ matrix.target_id }}*
+            startup-${{ matrix.target_id }}.log
+          if-no-files-found: error
+          retention-days: ${{ (startsWith(github.ref, 'refs/tags/') || needs.build-check.outputs.build_type == 'candidate') && 30 || 7 }}
 
       - name: Upload to Cloudflare R2
-        if: needs.build-check.outputs.should_publish == 'true' && env.R2_ACCESS_KEY_ID != '' && (needs.build-check.outputs.build_type == 'release' || needs.build-check.outputs.build_type == 'prerelease' || needs.build-check.outputs.build_type == 'development')
+        if: steps.reuse.outputs.reused != 'true' && (needs.build-check.outputs.should_publish == 'true' && env.R2_ACCESS_KEY_ID != '' && (needs.build-check.outputs.build_type == 'release' || needs.build-check.outputs.build_type == 'prerelease' || needs.build-check.outputs.build_type == 'development'))
         env:
           R2_ACCESS_KEY_ID: ${{ secrets.R2_ACCESS_KEY_ID }}
           R2_SECRET_ACCESS_KEY: ${{ secrets.R2_SECRET_ACCESS_KEY }}
@@ -764,9 +812,9 @@ jobs:
             exit 0
           fi
 
-          # The self-hosted Linux runners do not ship the aws CLI (GitHub-hosted
-          # images did). Install the official CLI bundle on demand so R2 uploads
-          # survive a fresh runner without relying on a distro package.
+          # ARC pods (the x86_64 legs) do not ship the aws CLI. Ubicloud arm64
+          # images follow the GitHub runner image and often do. Install the
+          # official CLI bundle when it is missing so R2 uploads work on either.
           if ! command -v aws >/dev/null 2>&1; then
             echo "aws CLI not found on runner; installing..."
             if command -v apt-get >/dev/null 2>&1; then
@@ -804,13 +852,18 @@ jobs:
           export AWS_SECRET_ACCESS_KEY="$R2_SECRET_ACCESS_KEY"
           export AWS_DEFAULT_REGION="auto"
 
-          # Determine upload path based on build type
+          # Determine upload path based on build type. Release and prerelease
+          # builds only stage their bytes here: the official
+          # artifacts/rustfs/release/ path is written solely by the
+          # certification-gated promote-release-artifacts job, from these same
+          # staged objects. Development builds keep the uncertified dev channel
+          # and never touch official or latest channels.
           if [[ "$BUILD_TYPE" == "development" ]]; then
             R2_PATH="s3://${R2_BUCKET}/artifacts/rustfs/dev/"
             echo "📤 Uploading development build to R2 dev directory"
           else
-            R2_PATH="s3://${R2_BUCKET}/artifacts/rustfs/release/"
-            echo "📤 Uploading release build to R2 release directory"
+            R2_PATH="s3://${R2_BUCKET}/artifacts/rustfs/staging/${GITHUB_RUN_ID}/${GITHUB_RUN_ATTEMPT}/"
+            echo "📤 Staging release build for certification-gated promotion"
           fi
 
           # Upload all rustfs zip files to R2 using S3-compatible API
@@ -825,6 +878,7 @@ jobs:
 
           echo "✅ Upload completed successfully"
 
+
   # Build summary
   build-summary:
     name: Build Summary
@@ -859,309 +913,9 @@ jobs:
             echo "This run does not trigger Docker publishing."
           fi
 
-  # Create GitHub Release for every valid release tag, including previews
-  create-release:
-    name: Create GitHub Release
-    needs: [ build-check, build-rustfs ]
-    if: needs.build-check.outputs.should_publish == 'true' && startsWith(github.ref, 'refs/tags/') && (needs.build-check.outputs.build_type == 'preview' || needs.build-check.outputs.build_type == 'release' || needs.build-check.outputs.build_type == 'prerelease')
-    runs-on: ubuntu-latest # gh CLI required; the sm-standard-2 fleet does not ship it
-    timeout-minutes: 30
-    permissions:
-      contents: write
-    outputs:
-      release_id: ${{ steps.create.outputs.release_id }}
-      release_url: ${{ steps.create.outputs.release_url }}
-    steps:
-      - name: Checkout repository
-        uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
-        with:
-          persist-credentials: false
-          fetch-depth: 0
-
-      - name: Create GitHub Release
-        id: create
-        env:
-          GH_TOKEN: ${{ github.token }}
-        shell: bash
-        run: |
-          TAG="${{ needs.build-check.outputs.version }}"
-          VERSION="${{ needs.build-check.outputs.version }}"
-          IS_PRERELEASE="${{ needs.build-check.outputs.is_prerelease }}"
-          BUILD_TYPE="${{ needs.build-check.outputs.build_type }}"
-          TARGET_COMMITISH=$(git rev-parse --verify "refs/tags/${TAG}^{commit}")
-
-          # Determine release type for title
-          if [[ "$BUILD_TYPE" == "preview" ]]; then
-            RELEASE_TYPE="preview"
-          elif [[ "$BUILD_TYPE" == "prerelease" ]]; then
-            if [[ "$TAG" == *"alpha"* ]]; then
-              RELEASE_TYPE="alpha"
-            elif [[ "$TAG" == *"beta"* ]]; then
-              RELEASE_TYPE="beta"
-            elif [[ "$TAG" == *"rc"* ]]; then
-              RELEASE_TYPE="rc"
-            else
-              RELEASE_TYPE="prerelease"
-            fi
-          else
-            RELEASE_TYPE="release"
-          fi
-
-          # Create release title
-          if [[ "$IS_PRERELEASE" == "true" ]]; then
-            TITLE="RustFS $VERSION (${RELEASE_TYPE})"
-          else
-            TITLE="RustFS $VERSION"
-          fi
-
-          ./scripts/release/create_or_update_release.sh \
-            "$TAG" \
-            "$TARGET_COMMITISH" \
-            "$TITLE" \
-            "$IS_PRERELEASE"
-
-  # Prepare and upload release assets
-  upload-release-assets:
-    name: Upload Release Assets
-    needs: [ build-check, build-rustfs, create-release ]
-    if: needs.build-check.outputs.should_publish == 'true' && startsWith(github.ref, 'refs/tags/') && (needs.build-check.outputs.build_type == 'preview' || needs.build-check.outputs.build_type == 'release' || needs.build-check.outputs.build_type == 'prerelease')
-    runs-on: ubuntu-latest # gh CLI required; the sm-standard-2 fleet does not ship it
-    timeout-minutes: 30
-    permissions:
-      contents: write
-      actions: read
-    steps:
-      - name: Checkout repository
-        uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
-        with:
-          persist-credentials: false
-
-      - name: Download all build artifacts
-        uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
-        with:
-          path: ./artifacts
-          pattern: rustfs-*
-          merge-multiple: true
-
-      - name: Prepare release assets
-        id: prepare
-        shell: bash
-        run: |
-          VERSION="${{ needs.build-check.outputs.version }}"
-          TAG="${{ needs.build-check.outputs.version }}"
-
-          mkdir -p ./release-assets
-
-          # Copy and verify artifacts (including latest files created during build)
-          ASSETS_COUNT=0
-          for file in ./artifacts/*.zip; do
-            if [[ -f "$file" ]]; then
-              cp "$file" ./release-assets/
-              ASSETS_COUNT=$((ASSETS_COUNT + 1))
-            fi
-          done
-
-          if [[ $ASSETS_COUNT -eq 0 ]]; then
-            echo "❌ No artifacts found!"
-            exit 1
-          fi
-
-          cd ./release-assets
-
-          # Generate checksums for all files (including latest versions)
-          if compgen -G "*.zip" >/dev/null; then
-            sha256sum -- *.zip > SHA256SUMS
-            sha512sum -- *.zip > SHA512SUMS
-          fi
-
-          cd ..
-          python3 scripts/security/generate_release_supply_chain_assets.py \
-            --asset-dir ./release-assets \
-            --version "$VERSION" \
-            --tag "$TAG" \
-            --build-type "${{ needs.build-check.outputs.build_type }}" \
-            --repository "${{ github.repository }}" \
-            --ref "${{ github.ref }}" \
-            --sha "${{ github.sha }}" \
-            --run-id "${{ github.run_id }}" \
-            --run-attempt "${{ github.run_attempt }}"
-          cd ./release-assets
-
-          echo "📦 Prepared assets:"
-          ls -la
-
-          echo "🔢 Total asset count: $ASSETS_COUNT"
-
-      - name: Upload to GitHub Release
-        env:
-          GH_TOKEN: ${{ github.token }}
-        shell: bash
-        run: |
-          TAG="${{ needs.build-check.outputs.version }}"
-
-          cd ./release-assets
-
-          # Upload all files
-          for file in *; do
-            if [[ -f "$file" ]]; then
-              echo "📤 Uploading $file..."
-              gh release upload "$TAG" "$file" --clobber
-            fi
-          done
-
-          echo "✅ All assets uploaded successfully"
-
-  # Update latest.json for every release tag (stable and prerelease): the
-  # project currently ships prerelease tags only, so gating this to stable
-  # left the version pointer permanently stale. release_type records whether
-  # the pointed-to version is a prerelease.
-  update-latest-version:
-    name: Update Latest Version
-    needs: [ build-check, publish-release ]
-    if: needs.build-check.outputs.should_publish == 'true' && startsWith(github.ref, 'refs/tags/') && (needs.build-check.outputs.build_type == 'release' || needs.build-check.outputs.build_type == 'prerelease')
-    runs-on: ubuntu-latest
-    timeout-minutes: 30
-    steps:
-      - name: Update latest.json
-        env:
-          OSS_ACCESS_KEY_ID: ${{ secrets.ALICLOUDOSS_KEY_ID }}
-          OSS_ACCESS_KEY_SECRET: ${{ secrets.ALICLOUDOSS_KEY_SECRET }}
-          OSS_REGION: cn-beijing
-          OSS_ENDPOINT: https://oss-cn-beijing.aliyuncs.com
-        shell: bash
-        run: |
-          if [[ -z "$OSS_ACCESS_KEY_ID" ]]; then
-            echo "⚠️ OSS credentials not available, skipping latest.json update"
-            exit 0
-          fi
-
-          VERSION="${{ needs.build-check.outputs.version }}"
-          TAG="${{ needs.build-check.outputs.version }}"
-
-          if [[ "${{ needs.build-check.outputs.build_type }}" == "prerelease" ]]; then
-            RELEASE_TYPE="prerelease"
-          else
-            RELEASE_TYPE="stable"
-          fi
-
-          # Install ossutil
-          OSSUTIL_VERSION="2.1.1"
-          OSSUTIL_ZIP="ossutil-${OSSUTIL_VERSION}-linux-amd64.zip"
-          OSSUTIL_DIR="ossutil-${OSSUTIL_VERSION}-linux-amd64"
-          OSSUTIL_SHA256="60f3a808cbbdfd4b5269b8f967c6a66f564c9dacff52d93a5d21a588976ab5a2"
-
-          curl --fail -L -o "$OSSUTIL_ZIP" "https://gosspublic.alicdn.com/ossutil/v2/${OSSUTIL_VERSION}/${OSSUTIL_ZIP}"
-          printf '%s  %s\n' "$OSSUTIL_SHA256" "$OSSUTIL_ZIP" | sha256sum -c -
-          unzip "$OSSUTIL_ZIP"
-          OSSUTIL_BIN="${RUNNER_TEMP}/ossutil"
-          mv "${OSSUTIL_DIR}/ossutil" "$OSSUTIL_BIN"
-          rm -rf "$OSSUTIL_DIR" "$OSSUTIL_ZIP"
-          chmod +x "$OSSUTIL_BIN"
-
-          # Create latest.json
-          cat > latest.json << EOF
-          {
-            "version": "${VERSION}",
-            "tag": "${TAG}",
-            "release_date": "$(date -u +%Y-%m-%dT%H:%M:%SZ)",
-            "release_type": "${RELEASE_TYPE}",
-            "download_url": "https://github.com/${{ github.repository }}/releases/tag/${TAG}"
-          }
-          EOF
-
-          # Upload to OSS
-          "$OSSUTIL_BIN" cp latest.json oss://rustfs-version/latest.json --force
-
-          echo "✅ Updated latest.json for ${RELEASE_TYPE} release $VERSION"
-
-  # Publish release (remove draft status)
-  publish-release:
-    name: Publish Release
-    needs: [ build-check, create-release, upload-release-assets ]
-    if: needs.build-check.outputs.should_publish == 'true' && startsWith(github.ref, 'refs/tags/') && (needs.build-check.outputs.build_type == 'preview' || needs.build-check.outputs.build_type == 'release' || needs.build-check.outputs.build_type == 'prerelease')
-    runs-on: ubuntu-latest # gh CLI required; the sm-standard-2 fleet does not ship it
-    timeout-minutes: 30
-    permissions:
-      contents: write
-    steps:
-      - name: Publish release
-        env:
-          GH_TOKEN: ${{ github.token }}
-        shell: bash
-        run: |
-          TAG="${{ needs.build-check.outputs.version }}"
-          BUILD_TYPE="${{ needs.build-check.outputs.build_type }}"
-          RELEASE_ID="${{ needs.create-release.outputs.release_id }}"
-
-          # Publish the release and correct its channel state on retries.
-          # Only a stable final release may become GitHub Latest.
-          if [[ "$BUILD_TYPE" == "release" ]]; then
-            gh api --method PATCH "repos/${GITHUB_REPOSITORY}/releases/${RELEASE_ID}" \
-              -F draft=false \
-              -F prerelease=false \
-              -f make_latest=true >/dev/null
-          else
-            gh api --method PATCH "repos/${GITHUB_REPOSITORY}/releases/${RELEASE_ID}" \
-              -F draft=false \
-              -F prerelease=true \
-              -f make_latest=false >/dev/null
-          fi
-
-          echo "🎉 Released $TAG successfully!"
-          echo "📄 Release URL: ${{ needs.create-release.outputs.release_url }}"
-
-  # Remove the internal preview releases once the deliverable release is live.
-  # Only the Releases are deleted; the -preview.N tags stay so the validated
-  # commit remains traceable.
-  cleanup-preview-releases:
-    name: Cleanup Preview Releases
-    needs: [ build-check, publish-release ]
-    if: needs.build-check.outputs.should_publish == 'true' && startsWith(github.ref, 'refs/tags/') && (needs.build-check.outputs.build_type == 'release' || needs.build-check.outputs.build_type == 'prerelease')
-    runs-on: ubuntu-latest # gh CLI required; the sm-standard-2 fleet does not ship it
-    timeout-minutes: 30
-    permissions:
-      contents: write
-    steps:
-      - name: Delete preview releases for this target
-        env:
-          GH_TOKEN: ${{ github.token }}
-        shell: bash
-        run: |
-          set -euo pipefail
-
-          TAG="${{ needs.build-check.outputs.version }}"
-          RELEASES_JSON="${RUNNER_TEMP}/releases.json"
-
-          # Fetch before filtering: a failed listing must abort here instead of
-          # looking like "nothing to clean up".
-          gh api --paginate "repos/${GITHUB_REPOSITORY}/releases?per_page=100" > "$RELEASES_JSON"
-
-          # Match only <target>-preview.<digits>. String operations, not a
-          # regex over the tag, so dots in the version cannot widen the match.
-          DELETED=0
-          while IFS= read -r preview_tag; do
-            [[ -n "$preview_tag" ]] || continue
-            echo "🧹 Deleting preview release $preview_tag (tag kept)"
-            gh release delete "$preview_tag" --repo "${GITHUB_REPOSITORY}" --yes
-            DELETED=$((DELETED + 1))
-          done < <(
-            jq -r --arg tag "$TAG" '
-              .[]
-              | select(.tag_name | startswith($tag + "-preview."))
-              | select(.tag_name | ltrimstr($tag + "-preview.") | test("^[0-9]+$"))
-              | .tag_name
-            ' "$RELEASES_JSON"
-          )
-
-          if [[ "$DELETED" -eq 0 ]]; then
-            echo "ℹ️  No preview releases to clean up for $TAG"
-          else
-            echo "✅ Removed $DELETED preview release(s) for $TAG"
-          fi
-
   alert-on-failure:
     name: Alert on scheduled failure
-    needs: [build-check, prepare-platform-matrix, build-rustfs, build-summary]
+    needs: [build-check, prepare-build-inputs, prepare-platform-matrix, build-rustfs, build-summary]
     if: >-
       always() && github.event_name == 'schedule' &&
       (contains(needs.*.result, 'failure') || contains(needs.*.result, 'cancelled'))

.github/workflows/cache-warm.yml

@@ -33,22 +33,22 @@
 # cache-save-if to anything but 'false' for it; every lane in ci.yml reads.
 # scripts/security/check_cache_save_if.sh keeps the declarations explicit.
 #
-# The builds are supersets of what the reading lanes compile, because a reader
-# restores only what the writer saved. Feature resolution matters here: a lane
-# built with e2e-test-hooks resolves dependency features differently, which
-# changes -Cmetadata, so the plain build does not cover it. See
-# rustfs/backlog#1600.
+# A reader restores only what the writer saved, and Cargo reuses a dependency
+# only when it resolves the same features for it. Every package selection and
+# feature set resolves differently and changes -Cmetadata, so each job builds
+# its readers' own invocations, not a superset (rustfs/backlog#1600).
+#
+# Rust-cache never replaces an entry restored from an exact key match: the key
+# hashes the lockfile, toolchain and compiler environment, not the commit. A
+# job whose restore was an exact hit therefore skips its builds; they could
+# only produce what is already stored.
 
 name: Cache Warm
 
 on:
   schedule:
-    # Every six hours; the 22:41 UTC run lands before ci.yml's nightly 00:11
-    # UTC matrix. A reader restores the newest entry under its key and
-    # rebuilds whatever its own diff changed, so a cache up to six hours old
-    # costs a PR the handful of crates merged since. Warming after every merge
-    # (fourteen completed runs a day, about two sm-standard-4 hours each)
-    # spent far more than that to avoid it.
+    # Writes a key that no push created, such as one for a new stable
+    # toolchain; with every key present the jobs stop after the restore.
     - cron: "41 4,10,16,22 * * *"
   push:
     branches: [ main ]
@@ -96,8 +96,8 @@ env:
   RUST_BACKTRACE: 1
 
 jobs:
-  # Readers: test-and-lint (workspace and ILM), build-rustfs-debug-binary,
-  # e2e-tests, e2e-full.
+  # Readers: workspace-clippy, workspace-lint-and-build, build-rustfs-debug-binary,
+  # connect-feature-boundaries (nightly) and reference-parity.yml.
   warm-ci-dev:
     name: Warm ci-dev
     runs-on: sm-standard-4
@@ -111,10 +111,13 @@ jobs:
           persist-credentials: false
 
       - name: Setup Rust environment
+        id: setup
         uses: ./.github/actions/setup
         with:
           rust-version: stable
-          cache-shared-key: ci-dev
+          # Digest of the build steps below; scripts/ci_gate.py --check-workflow
+          # names the new key when they change, and every reader must follow.
+          cache-shared-key: ci-dev-ea7eec74fe14
           cache-save-if: 'true'
           install-build-packaging-tools: 'false'
 
@@ -130,7 +133,7 @@ jobs:
       # 50% and 25% respectively; if linking dominates instead, the answer is
       # mold/lld plus split-debuginfo, which is exactly the part sccache cannot
       # touch. Off by default — this doubles the ci-dev build.
-      - name: Build ci-dev superset (with --timings)
+      - name: Build the workspace with --timings
         if: inputs.emit_timings
         env:
           CARGO_BUILD_JOBS: "2"
@@ -145,18 +148,18 @@ jobs:
           retention-days: 30
           if-no-files-found: error
 
-      # --all-targets covers the test binaries nextest builds, including
-      # e2e_test, which test-and-lint's own run excludes. The second build adds
-      # the e2e-test-hooks feature resolution that build-rustfs-debug-binary uses
-      # and that no lint lane enables.
-      - name: Build ci-dev superset
+      # The workspace and E2E test archives of workspace-lint-and-build and
+      # build-rustfs-debug-binary, then that job's hooks server.
+      - name: Build ci-dev reader selections
+        if: steps.setup.outputs.cache-hit != 'true'
         env:
-          # Same limit ci.yml puts on its nextest step: this builds the same
-          # ~100 workspace test binaries, and three concurrent links saturate the
-          # self-hosted runner's overlay I/O and can wedge Cargo (#5394).
+          # Same limit ci.yml puts on its archive steps: three concurrent test
+          # binary links saturate the self-hosted runner's overlay I/O and can
+          # wedge Cargo (#5394).
           CARGO_BUILD_JOBS: "2"
         run: |
-          cargo build --workspace --all-targets
+          python3 scripts/ci_test_artifact.py cargo warm --configuration workspace
+          python3 scripts/ci_test_artifact.py cargo warm --configuration e2e
           cargo build -p rustfs --bins --features e2e-test-hooks
 
       # ci.yml runs Clippy on every pull request. Clippy's check-mode artifacts
@@ -165,8 +168,9 @@ jobs:
       # touches. A lint failure on main must not stop the warm-up: the cache is
       # still saved (cache-on-failure), and the failure belongs to ci.yml.
       - name: Warm Clippy check artifacts
+        if: steps.setup.outputs.cache-hit != 'true'
         run: |
-          if ! cargo clippy --all-targets -- -D warnings; then
+          if ! python3 scripts/ci_test_artifact.py cargo lint --configuration workspace; then
             echo "::warning::cargo clippy reports findings on this commit; check artifacts are still cached"
           fi
 
@@ -213,6 +217,7 @@ jobs:
           persist-credentials: false
 
       - name: Setup Rust environment
+        id: setup
         uses: ./.github/actions/setup
         with:
           rust-version: stable
@@ -221,6 +226,7 @@ jobs:
           install-build-packaging-tools: 'false'
 
       - name: Build ci-feat-rio superset
+        if: steps.setup.outputs.cache-hit != 'true'
         env:
           # --all-targets links the same test binaries as the reader lane.
           CARGO_BUILD_JOBS: "2"
@@ -244,6 +250,7 @@ jobs:
           persist-credentials: false
 
       - name: Setup Rust environment
+        id: setup
         uses: ./.github/actions/setup
         with:
           rust-version: stable
@@ -252,6 +259,7 @@ jobs:
           install-build-packaging-tools: 'false'
 
       - name: Build ci-feat-proto superset
+        if: steps.setup.outputs.cache-hit != 'true'
         env:
           # Avoid an unbounded burst of concurrent test-binary links (#5394).
           CARGO_BUILD_JOBS: "2"
@@ -274,6 +282,7 @@ jobs:
           persist-credentials: false
 
       - name: Setup Rust environment
+        id: setup
         uses: ./.github/actions/setup
         with:
           rust-version: stable
@@ -282,7 +291,9 @@ jobs:
           install-build-packaging-tools: 'false'
 
       - name: Install build dependencies
+        if: steps.setup.outputs.cache-hit != 'true'
         run: sudo apt-get update && sudo apt-get install -y protobuf-compiler
 
       - name: Build ci-uring superset
+        if: steps.setup.outputs.cache-hit != 'true'
         run: cargo build -p rustfs-ecstore --all-targets

.github/workflows/ci-resource-accounting.yml

@@ -0,0 +1,60 @@
+name: CI Resource Accounting
+
+on:
+  workflow_dispatch:
+    inputs:
+      since:
+        description: "Inclusive run creation timestamp, for example 2026-10-01T00:00:00Z"
+        required: true
+        type: string
+      until:
+        description: "Inclusive end timestamp; keep the window below 1000 runs"
+        required: true
+        type: string
+
+permissions:
+  contents: read
+  actions: read
+  pull-requests: read
+
+concurrency:
+  group: ci-resource-accounting
+  cancel-in-progress: false
+
+jobs:
+  collect:
+    if: github.repository == 'rustfs/rustfs'
+    runs-on: ubuntu-latest
+    timeout-minutes: 30
+    steps:
+      - uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
+        with:
+          persist-credentials: false
+      - name: Collect all workflow attempts in the window
+        env:
+          GH_TOKEN: ${{ github.token }}
+          ACCOUNTING_SINCE: ${{ inputs.since }}
+          ACCOUNTING_UNTIL: ${{ inputs.until }}
+          ACCOUNTING_REPOSITORY: ${{ github.repository }}
+        run: |
+          python3 scripts/ci/actions_cost_report.py \
+            --repository "$ACCOUNTING_REPOSITORY" \
+            --since "$ACCOUNTING_SINCE" --until "$ACCOUNTING_UNTIL" \
+            --output "$RUNNER_TEMP/ci-accounting.json" \
+            --summary "$RUNNER_TEMP/ci-accounting.md"
+      - name: Show available accounting summary
+        if: always()
+        run: |
+          if [[ -f "$RUNNER_TEMP/ci-accounting.md" ]]; then
+            cat "$RUNNER_TEMP/ci-accounting.md" >> "$GITHUB_STEP_SUMMARY"
+          fi
+      - name: Retain recomputable accounting evidence
+        if: always()
+        uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
+        with:
+          name: ci-accounting-${{ github.run_id }}-${{ github.run_attempt }}
+          path: |
+            ${{ runner.temp }}/ci-accounting.json
+            ${{ runner.temp }}/ci-accounting.md
+          if-no-files-found: error
+          retention-days: 30

.github/workflows/ci-runner-pilot.yml

@@ -0,0 +1,92 @@
+name: CI Runner Pilot
+
+on:
+  workflow_dispatch:
+    inputs:
+      pool:
+        description: "Existing runner pool; no infrastructure is created"
+        required: true
+        type: choice
+        options: [hosted, arc]
+        default: hosted
+      workload:
+        description: "Probe only, or perform cold/warm fixed-source builds"
+        required: true
+        type: choice
+        options: [probe, server, workspace-tests]
+        default: probe
+      source_sha:
+        description: "Exact 40-character merged main SHA; required for builds"
+        type: string
+      toolchain:
+        description: "Exact Rust version such as 1.92.0; required for builds"
+        type: string
+      scratch_root:
+        description: "Existing absolute scratch directory; empty uses runner.temp"
+        type: string
+      build_jobs:
+        description: "Fixed Cargo concurrency for both comparisons (1-16)"
+        type: string
+        default: "2"
+      local_ssd:
+        description: "Require operator /etc/rustfs-runner/storage.json attestation"
+        type: boolean
+        default: false
+
+permissions:
+  contents: read
+
+concurrency:
+  group: ci-runner-pilot
+  cancel-in-progress: false
+
+jobs:
+  hosted:
+    if: github.repository == 'rustfs/rustfs' && github.ref == 'refs/heads/main' && inputs.pool == 'hosted'
+    runs-on: ubuntu-latest
+    timeout-minutes: 150
+    steps:
+      - uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
+        with:
+          persist-credentials: false
+      - uses: ./.github/actions/runner-pilot
+        with:
+          workload: ${{ inputs.workload }}
+          source-sha: ${{ inputs.source_sha }}
+          toolchain: ${{ inputs.toolchain }}
+          scratch-root: ${{ inputs.scratch_root }}
+          build-jobs: ${{ inputs.build_jobs }}
+          local-ssd: ${{ inputs.local_ssd }}
+      - name: Retain hosted pilot evidence
+        if: always()
+        uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
+        with:
+          name: runner-pilot-hosted-${{ github.run_id }}-${{ github.run_attempt }}
+          path: ${{ runner.temp }}/runner-pilot-evidence
+          if-no-files-found: error
+          retention-days: 30
+  arc:
+    if: github.repository == 'rustfs/rustfs' && github.ref == 'refs/heads/main' && inputs.pool == 'arc'
+    # Disk and hardware: measure the existing workspace build pool's storage.
+    runs-on: sm-standard-4
+    timeout-minutes: 150
+    steps:
+      - uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
+        with:
+          persist-credentials: false
+      - uses: ./.github/actions/runner-pilot
+        with:
+          workload: ${{ inputs.workload }}
+          source-sha: ${{ inputs.source_sha }}
+          toolchain: ${{ inputs.toolchain }}
+          scratch-root: ${{ inputs.scratch_root }}
+          build-jobs: ${{ inputs.build_jobs }}
+          local-ssd: ${{ inputs.local_ssd }}
+      - name: Retain ARC pilot evidence
+        if: always()
+        uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
+        with:
+          name: runner-pilot-arc-${{ github.run_id }}-${{ github.run_attempt }}
+          path: ${{ runner.temp }}/runner-pilot-evidence
+          if-no-files-found: error
+          retention-days: 30

.github/workflows/cla.yml

@@ -95,7 +95,7 @@ jobs:
 
       - name: Run CLA Bot
         if: github.event_name != 'merge_group'
-        uses: overtrue/cla-bot@5e5929846a60cfb42b16d0d38e135ef67d94fc88 # v0.0.9
+        uses: overtrue/cla-bot@4bd1387c19691168c42fcf5edfa17b15819b2ab3 # v0.1.1
         with:
           github-token: ${{ github.token }}
           registry-token: ${{ steps.registry-token.outputs.token }}

.github/workflows/coverage.yml

@@ -45,7 +45,7 @@ on:
     # 07:00 UTC Sunday — staggered clear of the other Sunday crons: ci (00:00),
     # build (01:00), e2e-s3tests (02:00), audit (03:00), nix-flake-update
     # (05:00), mint (06:00), and the daily fuzz (02:00), minio-interop (03:17),
-    # e2e-replication-nightly (04:00) and performance-ab (06:00) lanes.
+    # and e2e-replication-nightly (04:00) lanes.
     - cron: "43 7 * * 0"
 
 concurrency:

.github/workflows/docker.yml

@@ -1,599 +1,99 @@
-# Copyright 2024 RustFS Team
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-# Docker Images Workflow
-#
-# This workflow builds Docker images using pre-built binaries from the build workflow.
-#
-# Trigger Types:
-# 1. workflow_run: Automatically triggered when "Build and Release" workflow completes
-# 2. workflow_dispatch: Manual trigger for standalone Docker builds
-#
-# Key Features:
-# - Only triggers when Linux builds (x86_64 + aarch64) are successful
-# - Independent of macOS/Windows build status
-# - Uses workflow_run event for precise control
-# - Only builds Docker images for releases and prereleases (development builds are skipped)
-
-name: Docker Images
-
-# Permissions needed for workflow_run event and Docker registry access
+# Copyright 2026 RustFS Team
+# SPDX-License-Identifier: Apache-2.0
+name: Build candidate images
+on:
+  workflow_call:
+    inputs:
+      candidate_manifest:
+        type: string
+        required: true
 permissions:
   contents: read
-  packages: write
-
-on:
-  # Automatically triggered when build workflow completes
-  workflow_run:
-    workflows: [ "Build and Release" ]
-    types: [ completed ]
-  # Manual trigger with same parameters for consistency
-  workflow_dispatch:
-    inputs:
-      push_images:
-        description: "Push images to registries"
-        required: false
-        default: true
-        type: boolean
-      version:
-        description: "Version to build (latest, v1.0.0, v1.0.0-alpha1, or v1.0.1-preview.1)"
-        required: false
-        default: "latest"
-        type: string
-      force_rebuild:
-        description: "Force rebuild even if binary exists (useful for testing)"
-        required: false
-        default: false
-        type: boolean
-
-env:
-  CONCLUSION: ${{ github.event.workflow_run.conclusion }}
-  HEAD_BRANCH: ${{ github.event.workflow_run.head_branch }}
-  HEAD_SHA: ${{ github.event.workflow_run.head_sha }}
-  TRIGGERING_EVENT: ${{ github.event.workflow_run.event }}
-  DOCKERHUB_USERNAME: rustfs
-  CARGO_TERM_COLOR: always
-  REGISTRY_DOCKERHUB: rustfs/rustfs
-  REGISTRY_GHCR: ghcr.io/${{ github.repository }}
-  REGISTRY_QUAY: quay.io/rustfs/rustfs
-  DOCKER_PLATFORMS: linux/amd64,linux/arm64
-
+  actions: read
 jobs:
-  # Check if we should build Docker images.
-  # Short-circuit at job level so per-merge development builds (workflow_run
-  # from a push to main) skip the whole run without spawning a checkout job:
-  # images are only published for tag builds — build.yml push triggers cover
-  # only main and tags, so a non-main head_branch is a tag name — and for
-  # manual dispatch.
-  build-check:
-    name: Docker Build Check
-    if: >-
-      github.event_name == 'workflow_dispatch' ||
-      (github.event.workflow_run.conclusion == 'success' &&
-       github.event.workflow_run.event == 'push' &&
-       github.event.workflow_run.head_branch != 'main')
-    runs-on: ubuntu-latest
-    timeout-minutes: 30
-    outputs:
-      should_build: ${{ steps.check.outputs.should_build }}
-      should_push: ${{ steps.check.outputs.should_push }}
-      build_type: ${{ steps.check.outputs.build_type }}
-      version: ${{ steps.check.outputs.version }}
-      short_sha: ${{ steps.check.outputs.short_sha }}
-      is_prerelease: ${{ steps.check.outputs.is_prerelease }}
-      create_latest: ${{ steps.check.outputs.create_latest }}
-      source_ref: ${{ steps.check.outputs.source_ref }}
-    steps:
-      - name: Checkout repository
-        uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
-        with:
-          persist-credentials: false
-          # For workflow_run events, checkout the specific commit that triggered the workflow
-          ref: ${{ github.event.workflow_run.head_sha || github.sha }}
-
-      - name: Check build conditions
-        id: check
-        env:
-          # dispatch inputs via env, not `${{ }}` interpolation: they are
-          # free-form strings and would otherwise be evaluated by bash.
-          INPUT_VERSION: ${{ github.event.inputs.version }}
-          INPUT_PUSH_IMAGES: ${{ github.event.inputs.push_images }}
-          INPUT_FORCE_REBUILD: ${{ github.event.inputs.force_rebuild }}
-        run: |
-          should_build=false
-          should_push=false
-          build_type="none"
-          version=""
-          short_sha=""
-          is_prerelease=false
-          create_latest=false
-          source_ref="$GITHUB_SHA"
-
-          # Pre-GA policy: until the first stable (vX.Y.Z) tag exists, every
-          # prerelease (alpha/beta/rc) also moves `latest`, so users pulling
-          # `latest` get the newest test build. Once a stable tag is published
-          # this returns false and `latest` follows stable releases only.
-          prerelease_moves_latest() {
-            local stable_tags
-            stable_tags=$(git ls-remote --tags --refs origin 2>/dev/null \
-              | awk '{print $2}' \
-              | grep -E '^refs/tags/v?[0-9]+\.[0-9]+\.[0-9]+$' || true)
-            if [[ -z "$stable_tags" ]]; then
-              return 0
-            fi
-            echo "ℹ️  Stable release tag(s) already exist; prereleases no longer update latest"
-            return 1
-          }
-
-          if [[ "${{ github.event_name }}" == "workflow_run" ]]; then
-            # Triggered by build workflow completion
-            echo "🔗 Triggered by build workflow completion"
-
-            # Check if the triggering workflow was successful
-            # If the workflow succeeded, it means ALL builds (including Linux x86_64 and aarch64) succeeded
-            if [[ "$CONCLUSION" == "success" ]]; then
-              echo "✅ Build workflow succeeded, all builds including Linux are successful"
-              should_build=true
-              should_push=true
-            else
-              echo "❌ Build workflow failed (conclusion: $CONCLUSION), skipping Docker build"
-              should_build=false
-            fi
-
-            # Extract version info from commit message or use commit SHA
-            # Use Git to generate consistent short SHA (ensures uniqueness like build.yml)
-            short_sha=$(git rev-parse --short "$HEAD_SHA")
-            source_ref="$HEAD_SHA"
-
-            # Determine build type based on triggering workflow event and ref
-            triggering_event="$TRIGGERING_EVENT"
-            head_branch="$HEAD_BRANCH"
-
-            echo "🔍 Analyzing triggering workflow:"
-            echo "   📋 Event: $triggering_event"
-            echo "   🌿 Head branch: $head_branch"
-            echo "   📎 Head SHA: $HEAD_SHA"
-
-            # Check if this was triggered by a tag push
-            if [[ "$triggering_event" == "push" ]]; then
-              # For tag pushes, head_branch will be like "refs/tags/v1.0.0" or just "v1.0.0"
-              if [[ "$head_branch" == refs/tags/* ]]; then
-                # Extract tag name from refs/tags/TAG_NAME
-                tag_name="${head_branch#refs/tags/}"
-                version="$tag_name"
-              elif [[ "$head_branch" =~ ^v?[0-9]+\.[0-9]+\.[0-9]+ ]]; then
-                # Direct tag name like "v1.0.0" or "1.0.0-alpha.1"
-                version="$head_branch"
-              elif [[ "$head_branch" == "main" ]]; then
-                # Regular branch push to main
-                build_type="development"
-                version="dev-${short_sha}"
-                should_build=false
-                echo "⏭️  Skipping Docker build for development version (main branch push)"
-              else
-                # Other branch push
-                build_type="development"
-                version="dev-${short_sha}"
-                should_build=false
-                echo "⏭️  Skipping Docker build for development version (branch: $head_branch)"
-              fi
-
-              # If we extracted a version (tag), determine release type
-              if [[ -n "$version" ]] && [[ "$version" != "dev-${short_sha}" ]]; then
-                # Remove 'v' prefix if present for consistent version format
-                if [[ "$version" == v* ]]; then
-                  version="${version#v}"
-                fi
-
-                if [[ "$version" =~ -preview\.[0-9]+$ ]]; then
-                  build_type="preview"
-                  is_prerelease=true
-                  echo "🔍 Building Docker image for preview: $version (version tags only)"
-                elif [[ "$version" == *"-preview"* ]]; then
-                  echo "❌ Invalid preview tag: $version (expected suffix: -preview.<number>)" >&2
-                  exit 1
-                elif [[ "$version" == *"alpha"* ]] || [[ "$version" == *"beta"* ]] || [[ "$version" == *"rc"* ]]; then
-                  build_type="prerelease"
-                  is_prerelease=true
-                  # Pre-GA policy: prereleases update latest until the first stable tag exists.
-                  if prerelease_moves_latest; then
-                    create_latest=true
-                    echo "🧪 Building Docker image for prerelease: $version (creating latest tag)"
-                  else
-                    echo "🧪 Building Docker image for prerelease: $version"
-                  fi
-                else
-                  build_type="release"
-                  create_latest=true
-                  echo "🚀 Building Docker image for release: $version"
-                fi
-              fi
-            else
-              # Non-push events
-              build_type="development"
-              version="dev-${short_sha}"
-              should_build=false
-              echo "⏭️  Skipping Docker build for development version (event: $triggering_event)"
-            fi
-
-            echo "🔄 Build triggered by workflow_run:"
-            echo "   📋 Conclusion: $CONCLUSION"
-            echo "   🌿 Branch: $HEAD_BRANCH"
-            echo "   📎 SHA: $HEAD_SHA"
-            echo "   🎯 Event: $TRIGGERING_EVENT"
-
-          elif [[ "${{ github.event_name }}" == "workflow_dispatch" ]]; then
-            # Manual trigger
-            input_version="$INPUT_VERSION"
-            version="${input_version}"
-            should_push="$INPUT_PUSH_IMAGES"
-            should_build=true
-
-            # Get short SHA
-            short_sha=$(git rev-parse --short HEAD)
-
-            echo "🎯 Manual Docker build triggered:"
-            echo "   📋 Requested version: $input_version"
-            echo "   🔧 Force rebuild: $INPUT_FORCE_REBUILD"
-            echo "   🚀 Push images: $should_push"
-
-            case "$input_version" in
-              "latest")
-                build_type="release"
-                create_latest=true
-                echo "🚀 Building with latest stable release version"
-                ;;
-              *-preview*)
-                if [[ ! "$input_version" =~ ^v?[0-9]+\.[0-9]+\.[0-9]+(-[0-9A-Za-z.-]+)?-preview\.[0-9]+$ ]]; then
-                  echo "❌ Invalid preview version: $input_version" >&2
-                  exit 1
-                fi
-                build_type="preview"
-                is_prerelease=true
-                version="${input_version#v}"
-                echo "🔍 Building with preview version: $version (version tags only)"
-                ;;
-              # Prerelease versions (must match first, more specific)
-              v*alpha*|v*beta*|v*rc*|*alpha*|*beta*|*rc*)
-                build_type="prerelease"
-                is_prerelease=true
-                # Pre-GA policy: prereleases update latest until the first stable tag exists.
-                if prerelease_moves_latest; then
-                  create_latest=true
-                  echo "🧪 Building with prerelease version: $input_version (creating latest tag)"
-                else
-                  echo "🧪 Building with prerelease version: $input_version"
-                fi
-                ;;
-              # Release versions (match after prereleases, more general)
-              v[0-9]*|[0-9]*.*.*)
-                build_type="release"
-                create_latest=true
-                echo "📦 Building with specific release version: $input_version"
-                ;;
-              *)
-                # Invalid version for Docker build
-                should_build=false
-                echo "❌ Invalid version for Docker build: $input_version"
-                echo "⚠️  Only release versions (latest, v1.0.0, 1.0.0) and prereleases (v1.0.0-alpha1, 1.0.0-beta2) are supported"
-                ;;
-            esac
-
-            if [[ "$should_build" == true && "$input_version" != "latest" ]]; then
-              tag_ref="refs/tags/$input_version"
-              if ! git ls-remote --exit-code origin "$tag_ref" >/dev/null 2>&1; then
-                if [[ "$input_version" == v* ]]; then
-                  tag_ref="refs/tags/${input_version#v}"
-                else
-                  tag_ref="refs/tags/v$input_version"
-                fi
-              fi
-
-              if ! git ls-remote --exit-code origin "$tag_ref" >/dev/null 2>&1; then
-                echo "❌ Release tag not found for Docker build: $input_version"
-                exit 1
-              fi
-              source_ref="$tag_ref"
-            fi
-          fi
-
-          {
-            echo "should_build=$should_build"
-            echo "should_push=$should_push"
-            echo "build_type=$build_type"
-            echo "version=$version"
-            echo "short_sha=$short_sha"
-            echo "is_prerelease=$is_prerelease"
-            echo "create_latest=$create_latest"
-            echo "source_ref=$source_ref"
-          } >> "$GITHUB_OUTPUT"
-
-          echo "🐳 Docker Build Summary:"
-          echo "  - Should build: $should_build"
-          echo "  - Should push: $should_push"
-          echo "  - Build type: $build_type"
-          echo "  - Version: $version"
-          echo "  - Short SHA: $short_sha"
-          echo "  - Is prerelease: $is_prerelease"
-          echo "  - Create latest: $create_latest"
-          echo "  - Source ref: $source_ref"
-
-  # Build multi-arch Docker images
-  # Strategy: Build images using pre-built binaries from dl.rustfs.com
-  # Supports both release and dev channel binaries based on build context
-  # Only runs when should_build is true (which includes workflow success check)
   build-docker:
-    name: Build Docker Images
-    needs: build-check
-    if: needs.build-check.outputs.should_build == 'true'
-    runs-on: dind-sm-standard-2
-    timeout-minutes: 60
+    runs-on: ubuntu-latest
+    timeout-minutes: 90
     strategy:
       fail-fast: false
       matrix:
         include:
           - variant: musl
             file: Dockerfile
-            suffix: ""
-          - variant: glibc
+          - variant: gnu
             file: Dockerfile.glibc
-            suffix: "-glibc"
-
+    env:
+      CANDIDATE_MANIFEST: ${{ inputs.candidate_manifest }}
+      VARIANT: ${{ matrix.variant }}
     steps:
-      - name: Checkout repository
-        uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
+      - uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
         with:
+          ref: ${{ fromJson(inputs.candidate_manifest).source_sha }}
           persist-credentials: false
-          ref: ${{ needs.build-check.outputs.source_ref }}
-
-      - name: Login to Docker Hub
-        uses: docker/login-action@c94ce9fb468520275223c153574b00df6fe4bcc9 # v3
-        with:
-          username: ${{ env.DOCKERHUB_USERNAME }}
-          password: ${{ secrets.DOCKERHUB_TOKEN }}
-
-      - name: Login to GitHub Container Registry
-        uses: docker/login-action@c94ce9fb468520275223c153574b00df6fe4bcc9 # v3
-        with:
-          registry: ghcr.io
-          username: ${{ secrets.GHCR_USERNAME }}
-          password: ${{ secrets.GHCR_PASSWORD }}
-
-      - name: Login to Quay.io
-        uses: docker/login-action@c94ce9fb468520275223c153574b00df6fe4bcc9 # v3
-        with:
-          registry: quay.io
-          username: ${{ secrets.QUAY_USERNAME }}
-          password: ${{ secrets.QUAY_PASSWORD }}
-
-      - name: Set up QEMU
-        uses: docker/setup-qemu-action@c7c53464625b32c7a7e944ae62b3e17d2b600130 # v3
-
-      - name: Set up Docker Buildx
-        uses: docker/setup-buildx-action@8d2750c68a42422c14e847fe6c8ac0403b4cbd6f # v3
-
-      - name: Extract metadata and generate tags
-        id: meta
+      - name: Reuse completed image
+        id: reuse
+        env:
+          GH_TOKEN: ${{ github.token }}
+        run: python3 scripts/release/deliver.py reuse --kind image --target "$VARIANT"
+      - name: Install OCI tools
+        if: steps.reuse.outputs.reused != 'true'
         run: |
-          BUILD_TYPE="${{ needs.build-check.outputs.build_type }}"
-          VERSION="${{ needs.build-check.outputs.version }}"
-          CREATE_LATEST="${{ needs.build-check.outputs.create_latest }}"
-          VARIANT_SUFFIX="${{ matrix.suffix }}"
-
-          # Convert version format for Dockerfile compatibility. The former
-          # DOCKER_CHANNEL was "release" down every branch and was passed as a
-          # build-arg no Dockerfile declares, so it is gone.
-          case "$VERSION" in
-            "latest")
-              DOCKER_RELEASE="latest"
-              ;;
-            v*)
-              # For versioned releases (v1.0.0), remove 'v' prefix for Dockerfile
-              DOCKER_RELEASE="${VERSION#v}"
-              ;;
-            *)
-              # For other versions, pass as-is
-              DOCKER_RELEASE="${VERSION}"
-              ;;
-          esac
-
-          echo "docker_release=$DOCKER_RELEASE" >> "$GITHUB_OUTPUT"
-
-          echo "🐳 Docker build parameters:"
-          echo "  - Original version: $VERSION"
-          echo "  - Docker RELEASE: $DOCKER_RELEASE"
-
-          # Generate tags based on build type
-          # Only support release and prerelease builds (no development builds)
-          TAG_BASE="${VERSION}${VARIANT_SUFFIX}"
-          TAGS="${{ env.REGISTRY_DOCKERHUB }}:$TAG_BASE,${{ env.REGISTRY_GHCR }}:$TAG_BASE,${{ env.REGISTRY_QUAY }}:$TAG_BASE"
-
-          # Add latest when requested (stable releases, and prereleases before GA)
-          if [[ "$CREATE_LATEST" == "true" ]]; then
-            TAGS="$TAGS,${{ env.REGISTRY_DOCKERHUB }}:latest${VARIANT_SUFFIX},${{ env.REGISTRY_GHCR }}:latest${VARIANT_SUFFIX},${{ env.REGISTRY_QUAY }}:latest${VARIANT_SUFFIX}"
-          fi
-
-          # Always add the channel tag for prereleases, independent of latest
-          if [[ "$BUILD_TYPE" == "prerelease" ]]; then
-            # Prerelease channel tags (alpha, beta, rc)
-            if [[ "$VERSION" == *"alpha"* ]]; then
-              CHANNEL="alpha"
-            elif [[ "$VERSION" == *"beta"* ]]; then
-              CHANNEL="beta"
-            elif [[ "$VERSION" == *"rc"* ]]; then
-              CHANNEL="rc"
-            fi
-
-            if [[ -n "$CHANNEL" ]]; then
-              TAGS="$TAGS,${{ env.REGISTRY_DOCKERHUB }}:${CHANNEL}${VARIANT_SUFFIX},${{ env.REGISTRY_GHCR }}:${CHANNEL}${VARIANT_SUFFIX},${{ env.REGISTRY_QUAY }}:${CHANNEL}${VARIANT_SUFFIX}"
-            fi
-          fi
-
-          # Output tags
-          echo "tags=$TAGS" >> "$GITHUB_OUTPUT"
-
-          # Generate labels
-          LABELS="org.opencontainers.image.title=RustFS"
-          LABELS="$LABELS,org.opencontainers.image.description=RustFS distributed object storage system"
-          LABELS="$LABELS,org.opencontainers.image.version=$VERSION"
-          SOURCE_REVISION="$(git rev-parse HEAD)"
-          LABELS="$LABELS,org.opencontainers.image.revision=$SOURCE_REVISION"
-          LABELS="$LABELS,org.opencontainers.image.source=${{ github.server_url }}/${{ github.repository }}"
-          LABELS="$LABELS,org.opencontainers.image.created=$(date -u +'%Y-%m-%dT%H:%M:%SZ')"
-          LABELS="$LABELS,org.opencontainers.image.build-type=$BUILD_TYPE"
-
-          echo "labels=$LABELS" >> "$GITHUB_OUTPUT"
-
-          echo "🐳 Generated Docker tags:"
-          echo "$TAGS" | tr ',' '\n' | sed 's/^/  - /'
-          echo "📋 Build type: $BUILD_TYPE"
-          echo "🔖 Version: $VERSION"
-
-      - name: Build and push Docker image
+          sudo apt-get update
+          sudo apt-get install -y skopeo
+      - uses: docker/setup-qemu-action@c7c53464625b32c7a7e944ae62b3e17d2b600130 # v3
+        if: steps.reuse.outputs.reused != 'true'
+      - uses: docker/setup-buildx-action@8d2750c68a42422c14e847fe6c8ac0403b4cbd6f # v3
+        if: steps.reuse.outputs.reused != 'true'
+      - name: Supply accepted binaries and pinned CLI
+        if: steps.reuse.outputs.reused != 'true'
+        env:
+          GH_TOKEN: ${{ github.token }}
+        run: python3 scripts/release/deliver.py image-prepare --target "$VARIANT"
+      - name: Build OCI archive without registry writes
+        if: steps.reuse.outputs.reused != 'true'
         uses: docker/build-push-action@10e90e3645eae34f1e60eeb005ba3a3d33f178e8 # v6
         with:
           context: .
           file: ${{ matrix.file }}
-          platforms: ${{ env.DOCKER_PLATFORMS }}
-          push: ${{ needs.build-check.outputs.should_push == 'true' }}
-          tags: ${{ steps.meta.outputs.tags }}
-          labels: ${{ steps.meta.outputs.labels }}
-          # No layer cache. This build compiles nothing — it downloads a
-          # release zip and runs apk/apt — so the cache could only save the
-          # minute or two those take, while creating a correctness problem: with
-          # RELEASE=latest the binary URL is resolved by curl *inside* a RUN
-          # layer, and the layer key does not include what that resolved to. A
-          # rebuild at the same RELEASE value (dispatch with version=latest, or
-          # a re-run of the same version) would hit the old layer and ship the
-          # previous release's binary. mode=max also consumed the same 10GB
-          # Actions cache quota the Rust lanes are fighting over.
-          #
-          # Only RELEASE is passed: it is the sole build-arg the Dockerfiles
-          # declare besides TARGETARCH. BUILDTIME, VERSION, BUILD_TYPE, REVISION
-          # and CHANNEL were never read by any stage (and BUILDTIME's $(date ...)
-          # was a literal here, not a shell substitution). BUILD_DATE and VCS_REF
-          # are declared by the Dockerfiles but deliberately left unset —
-          # supplying them would change the published image labels.
+          platforms: linux/amd64,linux/arm64
+          push: false
+          provenance: false
+          outputs: type=oci,dest=delivery/image-${{ matrix.variant }}.tar
           build-args: |
-            RELEASE=${{ steps.meta.outputs.docker_release }}
-          provenance: true
-          sbom: true
-          # Add retry mechanism by splitting the build process
-          no-cache: false
-          pull: true
-
-  # Note: Manifest creation is no longer needed as we only build one variant
-  # Multi-arch manifests are automatically created by docker/build-push-action
-
-  # Report-only container image vulnerability scan
-  scan-docker-image:
-    name: Scan Docker Images
-    needs: [ build-check, build-docker ]
-    if: needs.build-check.outputs.should_build == 'true' && needs.build-check.outputs.should_push == 'true'
-    runs-on: ubuntu-latest
-    timeout-minutes: 30
-    permissions:
-      contents: read
-      security-events: write
-    strategy:
-      fail-fast: false
-      matrix:
-        include:
-          - variant: musl
-            suffix: ""
-          - variant: glibc
-            suffix: "-glibc"
-    steps:
-      - name: Login to GitHub Container Registry
-        # docker/login-action v3
-        uses: docker/login-action@c94ce9fb468520275223c153574b00df6fe4bcc9
-        with:
-          registry: ghcr.io
-          username: ${{ secrets.GHCR_USERNAME }}
-          password: ${{ secrets.GHCR_PASSWORD }}
-
-      - name: Scan image with Trivy
-        # aquasecurity/trivy-action v0.36.0
+            BINARY_SOURCE=supplied
+            RELEASE=${{ fromJson(inputs.candidate_manifest).version }}
+            VCS_REF=${{ fromJson(inputs.candidate_manifest).source_sha }}
+      - name: Verify wrapped binaries and startup on both architectures
+        if: steps.reuse.outputs.reused != 'true'
+        run: python3 scripts/release/deliver.py image-smoke --target "$VARIANT"
+      - name: Scan amd64 image before sealing
+        if: steps.reuse.outputs.reused != 'true'
         uses: aquasecurity/trivy-action@ed142fd0673e97e23eac54620cfb913e5ce36c25
+        env:
+          TRIVY_PLATFORM: linux/amd64
         with:
-          image-ref: ${{ env.REGISTRY_GHCR }}:${{ needs.build-check.outputs.version }}${{ matrix.suffix }}
+          input: delivery-scan/image-${{ matrix.variant }}-amd64.tar
           format: sarif
-          output: trivy-${{ matrix.variant }}.sarif
-          ignore-unfixed: true
-          vuln-type: os,library
-          severity: CRITICAL,HIGH
+          output: delivery/scan-${{ matrix.variant }}-amd64.sarif
+          exit-code: "0" # Keep the repository's report-only vulnerability policy.
+      - name: Scan arm64 image before sealing
+        if: steps.reuse.outputs.reused != 'true'
+        uses: aquasecurity/trivy-action@ed142fd0673e97e23eac54620cfb913e5ce36c25
+        env:
+          TRIVY_PLATFORM: linux/arm64
+        with:
+          input: delivery-scan/image-${{ matrix.variant }}-arm64.tar
+          format: sarif
+          output: delivery/scan-${{ matrix.variant }}-arm64.sarif
           exit-code: "0"
-
-      # Surface findings in the Security tab; an artifact alone is write-only
-      # reporting nobody reviews.
-      - name: Upload scan results to code scanning
-        uses: github/codeql-action/upload-sarif@99df26d4f13ea111d4ec1a7dddef6063f76b97e9 # v4.37.0
+      - name: Record verified OCI identity
+        if: steps.reuse.outputs.reused != 'true'
+        run: python3 scripts/release/deliver.py image-record --target "$VARIANT"
+      - uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+        if: steps.reuse.outputs.reused != 'true'
         with:
-          sarif_file: trivy-${{ matrix.variant }}.sarif
-          category: container-image-${{ matrix.variant }}
-
-      - name: Upload container scan report
-        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
-        with:
-          name: container-image-scan-${{ matrix.variant }}
-          path: trivy-${{ matrix.variant }}.sarif
+          name: image-${{ matrix.variant }}
+          path: delivery/*
           if-no-files-found: error
           retention-days: 30
-
-  # Docker build summary
-  docker-summary:
-    name: Docker Build Summary
-    needs: [ build-check, build-docker ]
-    if: always() && needs.build-check.outputs.should_build == 'true'
-    runs-on: ubuntu-latest
-    timeout-minutes: 10
-    steps:
-      - name: Docker build completion summary
-        run: |
-          BUILD_TYPE="${{ needs.build-check.outputs.build_type }}"
-          VERSION="${{ needs.build-check.outputs.version }}"
-          CREATE_LATEST="${{ needs.build-check.outputs.create_latest }}"
-
-          echo "🐳 Docker build completed successfully!"
-          echo "📦 Build type: $BUILD_TYPE"
-          echo "🔢 Version: $VERSION"
-          echo "🚀 Strategy: Images using pre-built binaries (release channel only)"
-          echo ""
-
-          case "$BUILD_TYPE" in
-            "preview")
-              echo "🔍 Preview Docker image has been built with ${VERSION} tags"
-              echo "⏭️  Preview images do not update latest or prerelease channels"
-              ;;
-            "release")
-              echo "🚀 Release Docker image has been built with ${VERSION} tags"
-              echo "✅ This image is ready for production use"
-              if [[ "$CREATE_LATEST" == "true" ]]; then
-                echo "🏷️  Latest tag has been created for stable release"
-              fi
-              ;;
-            "prerelease")
-              echo "🧪 Prerelease Docker image has been built with ${VERSION} tags"
-              echo "⚠️  This is a prerelease image - use with caution"
-              # Prereleases move latest until the first stable tag exists (pre-GA policy).
-              if [[ "$CREATE_LATEST" == "true" ]]; then
-                echo "🏷️  Latest tag has been created for prerelease: $VERSION"
-              else
-                echo "🚫 Latest tag NOT created for prerelease"
-              fi
-              ;;
-            *)
-              echo "❌ Unexpected build type: $BUILD_TYPE"
-              ;;
-          esac
+          compression-level: 0

.github/workflows/dual-stack-nightly.yml

@@ -0,0 +1,242 @@
+# Copyright 2026 RustFS Team
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# Nightly both-stack diff of the two e2e profiles only ci.yml runs, e2e-smoke
+# and e2e-full (rustfs/backlog#2772, T3.1). The other profiles carry a `stack`
+# dimension in their own workflows, where the existing lane is the legacy leg.
+#
+# One build serves every leg. One preflight starts that binary with
+# RUSTFS_S3_STACK=gateway (scripts/dual_stack.py preflight); without a proven
+# gateway start the gateway legs do not run and the summary says `skipped:
+# gateway stack refuses startup: <why>`. Which stack a leg ran is read from the
+# servers' own start-up line. The gateway legs are report-only until T2.10.
+#
+# Left out of both stacks: `dual_stack_differential::` (servers with an explicit
+# stack), `fake_s3_target::`, the upgrade write-readiness cases and one deadline
+# case (no server), and `distributed_startup_regression_test::` (needs ci.yml's
+# hooks-binary manifest). Both e2e-full stacks use ci.yml's pinned mc and
+# Vault binaries and network fault-injection tools.
+
+name: dual-stack-nightly
+
+on:
+  workflow_dispatch:
+  schedule:
+    # 03:13 UTC nightly, clear of e2e-nightly (04:29), the ODM lane (05:23) and
+    # e2e-distributed (05:53).
+    - cron: "13 3 * * *"
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: ${{ github.event_name != 'schedule' }}
+
+permissions:
+  contents: read
+
+defaults:
+  run:
+    shell: bash
+
+jobs:
+  build:
+    name: Build once and probe the gateway stack
+    if: github.repository == 'rustfs/rustfs'
+    # The runner and cache of ci.yml's build-rustfs-debug-binary.
+    runs-on: sm-standard-4
+    timeout-minutes: 60
+    outputs:
+      stacks: ${{ steps.preflight.outputs.stacks }}
+    env:
+      FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
+        with:
+          persist-credentials: false
+
+      - name: Setup Rust environment
+        uses: ./.github/actions/setup
+        with:
+          rust-version: stable
+          cache-shared-key: ci-dev
+          cache-save-if: 'false'
+          install-build-packaging-tools: 'false'
+
+      - name: Build the rustfs binary
+        run: python3 scripts/e2e_binary.py build --bins --features e2e-test-hooks
+
+      - name: Upload the binary
+        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+        with:
+          name: dual-stack-binary
+          path: |
+            target/debug/rustfs
+            target/debug/rustfs.e2e.json
+          if-no-files-found: error
+          retention-days: 1
+
+      # Without a proven gateway start there is no gateway leg to run.
+      - name: Gateway preflight
+        id: preflight
+        run: |
+          if python3 scripts/dual_stack.py preflight --binary target/debug/rustfs --out "$RUNNER_TEMP/preflight/stack.json"; then
+            echo 'stacks=["legacy","gateway"]' >> "$GITHUB_OUTPUT"
+          else
+            echo 'stacks=["legacy"]' >> "$GITHUB_OUTPUT"
+          fi
+
+      # The evidence stands in for a gateway leg that is skipped; a leg that runs overwrites it.
+      - name: Upload the preflight evidence for e2e-smoke
+        if: always()
+        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+        with:
+          name: dual-stack-leg-e2e-smoke-gateway
+          path: ${{ runner.temp }}/preflight
+          if-no-files-found: warn
+          retention-days: 7
+
+      - name: Upload the preflight evidence for e2e-full
+        if: always()
+        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+        with:
+          name: dual-stack-leg-e2e-full-gateway
+          path: ${{ runner.temp }}/preflight
+          if-no-files-found: warn
+          retention-days: 7
+
+  leg:
+    name: ${{ matrix.profile }} (${{ matrix.stack }}${{ matrix.stack == 'gateway' && ', report-only until T2.10' || '' }})
+    needs: [ build ]
+    runs-on: sm-standard-4
+    timeout-minutes: 120
+    # Report-only until T2.10: the gateway stack cannot serve the suites yet.
+    continue-on-error: ${{ matrix.stack == 'gateway' }}
+    strategy:
+      fail-fast: false
+      matrix:
+        profile: [ e2e-smoke, e2e-full ]
+        stack: ${{ fromJSON(needs.build.outputs.stacks) }}
+    env:
+      FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
+        with:
+          persist-credentials: false
+
+      - name: Setup Rust environment
+        uses: ./.github/actions/setup
+        with:
+          rust-version: stable
+          cache-shared-key: ci-dev
+          cache-save-if: 'false'
+          install-build-packaging-tools: 'false'
+
+      - name: Download the binary
+        uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
+        with:
+          name: dual-stack-binary
+          path: target/debug
+
+      - name: Set up Python
+        uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
+        with:
+          python-version: "3.12"
+
+      - name: Install awscurl
+        run: |
+          python3 -m pip install --user --upgrade pip "awscurl==0.44"
+          echo "AWSCURL_PATH=$HOME/.local/bin/awscurl" >> "$GITHUB_ENV"
+          test -x "$HOME/.local/bin/awscurl"
+
+      - name: Install network fault-injection tools
+        if: matrix.profile == 'e2e-full'
+        run: |
+          sudo apt-get install -y iptables
+          sudo -n iptables --version
+          # The endpoint-blackhole heal scenario needs CAP_NET_ADMIN. Containerised
+          # runners can run iptables but not touch the rule set; the test then logs
+          # a skip instead of failing, so surface that here where it is visible.
+          if ! sudo -n iptables -w 5 -S OUTPUT >/dev/null 2>&1; then
+            echo "::warning::iptables cannot read the OUTPUT chain on this runner (no CAP_NET_ADMIN); the endpoint-blackhole heal scenario will be skipped"
+          fi
+
+      - name: Install mc
+        if: matrix.profile == 'e2e-full'
+        env:
+          GH_TOKEN: ${{ github.token }}
+          MC_VERSION: RELEASE.2025-08-13T08-35-41Z
+          MC_SHA256: 01f866e9c5f9b87c2b09116fa5d7c06695b106242d829a8bb32990c00312e891
+        run: |
+          if ! command -v gh >/dev/null 2>&1; then
+            sudo apt-get update
+            sudo apt-get install -y gh
+          fi
+          MC_BINARY="mc.linux-amd64.${MC_VERSION}"
+          python3 .github/actions/setup/download_release_asset.py --repository minio/mc --tag "$MC_VERSION" --asset "$MC_BINARY" --output "$RUNNER_TEMP/mc" --sha256 "$MC_SHA256"
+          echo "${MC_SHA256}  $RUNNER_TEMP/mc" | sha256sum --check --status
+          chmod +x "$RUNNER_TEMP/mc"
+          echo "$RUNNER_TEMP" >> "$GITHUB_PATH"
+
+      - name: Verify mc
+        if: matrix.profile == 'e2e-full'
+        run: mc --version
+
+      - name: Install Vault
+        if: matrix.profile == 'e2e-full'
+        run: |
+          VAULT_VERSION="1.17.6"
+          VAULT_ARCHIVE="vault_${VAULT_VERSION}_linux_amd64.zip"
+          curl -fsSLo "$RUNNER_TEMP/$VAULT_ARCHIVE" "https://releases.hashicorp.com/vault/${VAULT_VERSION}/${VAULT_ARCHIVE}"
+          echo "0cddc1fbbb88583b5ba5b845f9f8fae47c6fb39a6d48cd543c6ba6fd3ac1a669  $RUNNER_TEMP/$VAULT_ARCHIVE" | sha256sum --check --status
+          unzip -q "$RUNNER_TEMP/$VAULT_ARCHIVE" -d "$RUNNER_TEMP/vault-bin"
+          echo "RUSTFS_TEST_VAULT_BIN=$RUNNER_TEMP/vault-bin/vault" >> "$GITHUB_ENV"
+
+      - name: Verify Vault
+        if: matrix.profile == 'e2e-full'
+        run: |
+          "$RUSTFS_TEST_VAULT_BIN" version
+
+      - name: Run the profile on the ${{ matrix.stack }} stack
+        env:
+          PROFILE: ${{ matrix.profile }}
+          RUSTFS_S3_STACK: ${{ matrix.stack }}
+          RUSTFS_E2E_LOG_DIR: ${{ runner.temp }}/server-logs
+        run: |
+          chmod +x target/debug/rustfs
+          filter='not test(/^(dual_stack_differential|fake_s3_target|distributed_startup_regression_test)::/)'
+          filter="$filter & not test(/^upgrade_compatibility_test::upgrade_write_readiness_tests::/)"
+          filter="$filter & not test(=list_objects_v2_pagination_test::tests::test_list_objects_v2_deadline_rejects_unresponsive_peer)"
+          python3 scripts/e2e_binary.py run --features e2e-test-hooks -- cargo nextest run --profile "$PROFILE" -p e2e_test -E "$filter"
+
+      - name: Prove the stack and upload the leg
+        if: always()
+        uses: ./.github/actions/dual-stack-leg
+        with: { profile: "${{ matrix.profile }}", stack: "${{ matrix.stack }}", logs: "${{ runner.temp }}/server-logs" }
+
+  summary:
+    name: Both-stack diff summary
+    needs: [ build, leg ]
+    if: github.repository == 'rustfs/rustfs' && always()
+    runs-on: ubuntu-latest
+    timeout-minutes: 15
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
+        with:
+          persist-credentials: false
+
+      - uses: ./.github/actions/dual-stack-summary
+        with:
+          profiles: e2e-smoke,e2e-full

.github/workflows/e2e-distributed.yml

@@ -17,8 +17,8 @@
 # Each selected test starts a real localhost cluster via
 # `RustFSTestClusterEnvironment` (4 processes; 4 drives per node unless the
 # case is a two-site 4-node 1-drive pair or a 4-node upgrade). Membership is
-# `[profile.e2e-distributed]` in `.config/nextest.toml`. Storage-sensitive PRs,
-# nightly runs, and manual dispatches all execute the same fail-closed suite.
+# `[profile.e2e-distributed]` in `.config/nextest.toml`. The nightly schedule
+# and manual dispatches execute the same fail-closed suite.
 # Upgrade cases download the same pinned previous release as e2e-upgrade.yml.
 #
 # Isolated pool filesystems: expand/decommission/rebalance cases require
@@ -33,32 +33,6 @@
 name: e2e-distributed
 
 on:
-  pull_request:
-    paths:
-      - "Cargo.lock"
-      - "Cargo.toml"
-      - ".config/nextest.toml"
-      - ".config/e2e-distributed-selection.txt"
-      - ".github/workflows/e2e-distributed.yml"
-      - "crates/audit/**"
-      - "crates/common/**"
-      - "crates/config/**"
-      - "crates/e2e_test/**"
-      - "crates/ecstore/**"
-      - "crates/filemeta/**"
-      - "crates/heal/**"
-      - "crates/iam/**"
-      - "crates/lock/**"
-      - "crates/madmin/**"
-      - "crates/notify/**"
-      - "crates/replication/**"
-      - "crates/s3-client/**"
-      - "crates/s3-ops/**"
-      - "crates/s3-types/**"
-      - "crates/scanner/**"
-      - "crates/storage-api/**"
-      - "crates/utils/**"
-      - "rustfs/**"
   workflow_dispatch:
     inputs:
       filter:
@@ -79,11 +53,14 @@ concurrency:
 jobs:
   distributed:
     if: github.repository == 'rustfs/rustfs'
-    name: Distributed 4-node 4-disk e2e
+    name: Distributed 4-node 4-disk e2e (${{ matrix.stack }}${{ matrix.stack == 'gateway' && ', report-only until T2.10' || '' }})
     # GitHub-hosted VM: loop and tmpfs mounts work here. sm-standard-4 is an
     # ARC pod and rejects both (`mount -o loop` ENOENT, tmpfs "read-only").
     runs-on: ubuntu-latest
     timeout-minutes: 180
+    # The gateway leg is report-only until T2.10.
+    continue-on-error: ${{ matrix.stack == 'gateway' }}
+    strategy: { fail-fast: false, matrix: { stack: [legacy, gateway] } }
     env:
       FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"
       NO_PROXY: 127.0.0.1,localhost
@@ -155,6 +132,11 @@ jobs:
         run: |
           python3 scripts/e2e_binary.py build --bins --features e2e-test-hooks
 
+      # A gateway binary that refuses to start skips the suite; the summary says why.
+      - name: Gateway preflight
+        if: matrix.stack == 'gateway'
+        run: python3 scripts/dual_stack.py preflight --binary target/debug/rustfs --out "$RUNNER_TEMP/leg/stack.json"
+
       - name: Verify distributed e2e membership
         env:
           NEXTEST_LISTING: ${{ runner.temp }}/rustfs-e2e-distributed-list.json
@@ -166,6 +148,7 @@ jobs:
         env:
           RUSTFS_E2E_LOG_DIR: ${{ runner.temp }}/rustfs-e2e-distributed-logs
           FILTER: ${{ inputs.filter }}
+          RUSTFS_S3_STACK: ${{ matrix.stack }}
         run: |
           set -euo pipefail
           rm -f target/nextest/e2e-distributed/junit.xml
@@ -176,11 +159,19 @@ jobs:
           fi
           test -s target/nextest/e2e-distributed/junit.xml
 
+      - name: Prove the stack and upload the leg
+        if: always()
+        uses: ./.github/actions/dual-stack-leg
+        with:
+          profile: e2e-distributed
+          stack: ${{ matrix.stack }}
+          logs: ${{ runner.temp }}/rustfs-e2e-distributed-logs
+
       - name: Upload distributed e2e diagnostics
         if: always()
         uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
         with:
-          name: e2e-distributed-${{ github.run_number }}
+          name: e2e-distributed-${{ github.run_number }}-${{ matrix.stack }}
           path: |
             target/nextest/e2e-distributed/junit.xml
             ${{ runner.temp }}/rustfs-e2e-distributed-list.json
@@ -200,6 +191,20 @@ jobs:
             fi
           done
 
+  dual-stack-summary:
+    name: Both-stack diff summary
+    needs: [distributed]
+    if: github.repository == 'rustfs/rustfs' && always()
+    runs-on: ubuntu-latest
+    timeout-minutes: 15
+    steps:
+      - uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
+        with:
+          persist-credentials: false
+      - uses: ./.github/actions/dual-stack-summary
+        with:
+          profiles: e2e-distributed
+
   alert-on-failure:
     name: Alert on scheduled failure
     needs: [distributed]

.github/workflows/e2e-replication-nightly.yml

@@ -38,8 +38,8 @@ name: e2e-nightly
 on:
   workflow_dispatch:
   schedule:
-    # 04:00 UTC nightly — staggered clear of fuzz/e2e-s3tests (02:00),
-    # stale (01:30) and performance-ab (06:00).
+    # 04:00 UTC nightly — staggered clear of fuzz/e2e-s3tests (02:00)
+    # and stale (01:30).
     - cron: "29 4 * * *"
 
 # Only alert-on-failure needs more than read access; it declares its own
@@ -54,14 +54,18 @@ concurrency:
 jobs:
   repl-nightly:
     if: github.repository == 'rustfs/rustfs'
-    name: Replication e2e (nightly)
+    name: Replication e2e (nightly, ${{ matrix.stack }}${{ matrix.stack == 'gateway' && ', report-only until T2.10' || '' }})
     # dual-node tests spawn two full rustfs servers each; the extra headroom
     # over the per-PR sm-standard-2 e2e job keeps the parallel process fan-out
     # from starving the runner. Mirrors the ILM serial lane's runner class.
     runs-on: sm-standard-4
     timeout-minutes: 45
+    # The gateway leg is report-only until T2.10.
+    continue-on-error: ${{ matrix.stack == 'gateway' }}
+    strategy: { fail-fast: false, matrix: { stack: [legacy, gateway] } }
     env:
       FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"
+      RUSTFS_BUILD_FEATURES: ${{ matrix.stack == 'gateway' && 'gateway-stack' || '' }}
     steps:
       - name: Checkout repository
         uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
@@ -93,7 +97,12 @@ jobs:
       # Build once and carry its source/binary identity into the test invocation.
       - name: Build rustfs binary
         run: |
-          python3 scripts/e2e_binary.py build --bins
+          python3 scripts/e2e_binary.py build --bins --features "$RUSTFS_BUILD_FEATURES"
+
+      # A gateway binary that refuses to start skips the suite; the summary says why.
+      - name: Gateway preflight
+        if: matrix.stack == 'gateway'
+        run: python3 scripts/dual_stack.py preflight --binary target/debug/rustfs --out "$RUNNER_TEMP/leg/stack.json"
 
       - name: Verify replication e2e membership
         env:
@@ -105,13 +114,22 @@ jobs:
       - name: Run replication e2e nightly suite
         env:
           RUSTFS_E2E_LOG_DIR: ${{ runner.temp }}/rustfs-e2e-repl-nightly-logs
-        run: python3 scripts/e2e_binary.py run -- cargo nextest run --profile e2e-repl-nightly -p e2e_test
+          RUSTFS_S3_STACK: ${{ matrix.stack }}
+        run: python3 scripts/e2e_binary.py run --features "$RUSTFS_BUILD_FEATURES" -- cargo nextest run --profile e2e-repl-nightly -p e2e_test
+
+      - name: Prove the stack and upload the leg
+        if: always()
+        uses: ./.github/actions/dual-stack-leg
+        with:
+          profile: e2e-repl-nightly
+          stack: ${{ matrix.stack }}
+          logs: ${{ runner.temp }}/rustfs-e2e-repl-nightly-logs
 
       - name: Upload nextest junit report
         if: always()
         uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
         with:
-          name: e2e-replication-nightly-junit-${{ github.run_number }}
+          name: e2e-replication-nightly-junit-${{ github.run_number }}-${{ matrix.stack }}
           path: |
             target/nextest/e2e-repl-nightly/junit.xml
             ${{ runner.temp }}/rustfs-e2e-repl-nightly-list.json
@@ -121,9 +139,12 @@ jobs:
 
   cluster-nightly:
     if: github.repository == 'rustfs/rustfs'
-    name: Cluster fault e2e (nightly)
+    name: Cluster fault e2e (nightly, ${{ matrix.stack }}${{ matrix.stack == 'gateway' && ', report-only until T2.10' || '' }})
     runs-on: sm-standard-4
     timeout-minutes: 90
+    # The gateway leg is report-only until T2.10.
+    continue-on-error: ${{ matrix.stack == 'gateway' }}
+    strategy: { fail-fast: false, matrix: { stack: [legacy, gateway] } }
     env:
       FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"
     steps:
@@ -144,6 +165,11 @@ jobs:
         run: |
           python3 scripts/e2e_binary.py build --bins --features e2e-test-hooks
 
+      # A gateway binary that refuses to start skips the suite; the summary says why.
+      - name: Gateway preflight
+        if: matrix.stack == 'gateway'
+        run: python3 scripts/dual_stack.py preflight --binary target/debug/rustfs --out "$RUNNER_TEMP/leg/stack.json"
+
       - name: Verify cluster fault e2e membership
         env:
           NEXTEST_LISTING: ${{ runner.temp }}/rustfs-e2e-nightly-list.json
@@ -155,13 +181,22 @@ jobs:
         env:
           CARGO_BIN_EXE_rustfs: ${{ github.workspace }}/target/debug/rustfs
           RUSTFS_E2E_LOG_DIR: ${{ runner.temp }}/rustfs-e2e-nightly-logs
+          RUSTFS_S3_STACK: ${{ matrix.stack }}
         run: python3 scripts/e2e_binary.py run --features e2e-test-hooks -- cargo nextest run --profile e2e-nightly -p e2e_test
 
+      - name: Prove the stack and upload the leg
+        if: always()
+        uses: ./.github/actions/dual-stack-leg
+        with:
+          profile: e2e-nightly
+          stack: ${{ matrix.stack }}
+          logs: ${{ runner.temp }}/rustfs-e2e-nightly-logs
+
       - name: Upload cluster fault diagnostics
         if: always()
         uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
         with:
-          name: e2e-cluster-nightly-${{ github.run_number }}
+          name: e2e-cluster-nightly-${{ github.run_number }}-${{ matrix.stack }}
           path: |
             target/nextest/e2e-nightly/junit.xml
             ${{ runner.temp }}/rustfs-e2e-nightly-list.json
@@ -171,12 +206,15 @@ jobs:
 
   protocols-nightly:
     if: github.repository == 'rustfs/rustfs'
-    name: Protocol e2e (nightly)
+    name: Protocol e2e (nightly, ${{ matrix.stack }}${{ matrix.stack == 'gateway' && ', report-only until T2.10' || '' }})
     runs-on: sm-standard-4
     timeout-minutes: 90
+    # The gateway leg is report-only until T2.10.
+    continue-on-error: ${{ matrix.stack == 'gateway' }}
+    strategy: { fail-fast: false, matrix: { stack: [legacy, gateway] } }
     env:
       FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"
-      RUSTFS_BUILD_FEATURES: ftps,webdav,sftp
+      RUSTFS_BUILD_FEATURES: ${{ matrix.stack == 'gateway' && 'ftps,webdav,sftp,gateway-stack' || 'ftps,webdav,sftp' }}
     steps:
       - name: Checkout repository
         uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
@@ -200,6 +238,11 @@ jobs:
       - name: Build protocol server
         run: python3 scripts/e2e_binary.py build --features "$RUSTFS_BUILD_FEATURES"
 
+      # A gateway binary that refuses to start skips the suite; the summary says why.
+      - name: Gateway preflight
+        if: matrix.stack == 'gateway'
+        run: python3 scripts/dual_stack.py preflight --binary target/debug/rustfs --out "$RUNNER_TEMP/leg/stack.json"
+
       # The suite owns fixed protocol ports and serializes its internal cases.
       - name: Verify protocol e2e membership
         env:
@@ -211,14 +254,28 @@ jobs:
       - name: Run protocol e2e nightly suite
         env:
           RUSTFS_E2E_LOG_DIR: ${{ runner.temp }}/rustfs-protocol-e2e-logs
-        run: >-
-          python3 scripts/e2e_binary.py run --features "$RUSTFS_BUILD_FEATURES" -- cargo nextest run -j 1 --profile e2e-protocols -p e2e_test --no-capture
+          RUSTFS_S3_STACK: ${{ matrix.stack }}
+          # These servers set no log level of their own; the stack proof reads their start-up line.
+          RUST_LOG: rustfs=info
+        # The servers inherit nextest's stdout (--no-capture): keep it where the stack proof reads it.
+        run: |
+          set -o pipefail
+          mkdir -p "$RUSTFS_E2E_LOG_DIR"
+          python3 scripts/e2e_binary.py run --features "$RUSTFS_BUILD_FEATURES" -- cargo nextest run -j 1 --profile e2e-protocols -p e2e_test --no-capture 2>&1 | tee "$RUSTFS_E2E_LOG_DIR/nextest.log"
+
+      - name: Prove the stack and upload the leg
+        if: always()
+        uses: ./.github/actions/dual-stack-leg
+        with:
+          profile: e2e-protocols
+          stack: ${{ matrix.stack }}
+          logs: ${{ runner.temp }}/rustfs-protocol-e2e-logs
 
       - name: Upload protocol diagnostics
         if: always()
         uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
         with:
-          name: e2e-protocol-nightly-${{ github.run_number }}
+          name: e2e-protocol-nightly-${{ github.run_number }}-${{ matrix.stack }}
           path: |
             target/nextest/e2e-protocols/junit.xml
             ${{ runner.temp }}/rustfs-e2e-protocols-list.json
@@ -226,6 +283,20 @@ jobs:
           retention-days: 7
           if-no-files-found: warn
 
+  dual-stack-summary:
+    name: Both-stack diff summary
+    needs: [repl-nightly, cluster-nightly, protocols-nightly]
+    if: github.repository == 'rustfs/rustfs' && always()
+    runs-on: ubuntu-latest
+    timeout-minutes: 15
+    steps:
+      - uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
+        with:
+          persist-credentials: false
+      - uses: ./.github/actions/dual-stack-summary
+        with:
+          profiles: e2e-repl-nightly,e2e-nightly,e2e-protocols
+
   alert-on-failure:
     name: Alert on scheduled failure
     needs: [repl-nightly, cluster-nightly, protocols-nightly]

.github/workflows/e2e-upgrade.yml

@@ -12,28 +12,19 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
+# Single-node upgrade and rollback lane.
+#
+# The weekly Monday schedule, version tags, and workflow_dispatch run this
+# suite. Distributed upgrade cases stay on the nightly e2e-distributed lane.
+
 name: Upgrade Compatibility
 
 on:
-  pull_request:
-    paths:
-      - ".github/workflows/e2e-upgrade.yml"
-      - "crates/e2e_test/src/common.rs"
-      - "crates/e2e_test/src/fake_s3_target/**"
-      - "crates/e2e_test/src/lib.rs"
-      - "crates/e2e_test/src/replication_extension_test.rs"
-      - "crates/e2e_test/src/upgrade_compatibility_test.rs"
-      - "crates/ecstore/**"
-      - "crates/filemeta/**"
-      - "crates/heal/**"
-      - "crates/kms/**"
-      - "crates/storage-api/**"
-      - "rustfs/**"
-      - "Cargo.lock"
   push:
     tags:
       - "[0-9]*.[0-9]*.[0-9]*"
   schedule:
+    # 03:17 UTC every Monday.
     - cron: "17 3 * * 1"
   workflow_dispatch:
 
@@ -42,7 +33,7 @@ permissions:
 
 concurrency:
   group: ${{ github.workflow }}-${{ github.ref }}
-  cancel-in-progress: ${{ github.event_name == 'pull_request' }}
+  cancel-in-progress: false
 
 env:
   CARGO_TERM_COLOR: always
@@ -128,6 +119,7 @@ jobs:
         run: chmod +x target/debug/rustfs
 
       - name: Verify and unpack pinned previous release
+        id: verify_source
         env:
           SOURCE_DIR: ${{ runner.temp }}/rustfs-upgrade-source
         run: |
@@ -185,6 +177,28 @@ jobs:
           done
           exit "$status"
 
+      - name: Run mixed-version MPU recovery
+        if: ${{ !cancelled() && steps.verify_source.outcome == 'success' }}
+        timeout-minutes: 15
+        env:
+          NO_PROXY: 127.0.0.1,localhost,::1
+          HTTP_PROXY: ""
+          HTTPS_PROXY: ""
+          ALL_PROXY: ""
+          RUSTFS_E2E_LOG_FILTER: "warn,rustfs_ecstore::cluster::rpc::client=debug"
+        run: |
+          set -euo pipefail
+          export no_proxy="$NO_PROXY"
+          export RUSTFS_E2E_LOG_DIR="$RUNNER_TEMP/rustfs-upgrade-logs/mpu-mixed-recovery"
+          mkdir -p "$RUSTFS_E2E_LOG_DIR"
+          mpu_test='distributed::multipart_fence_acceptance_test::mixed_version_failed_complete_requires_current_coordinator_for_abort'
+          python3 scripts/e2e_binary.py run -- cargo test --locked -p e2e_test --lib \
+            "$mpu_test" -- --list --exact | tee "$RUSTFS_E2E_LOG_DIR/list.log"
+          test "$(grep -Fxc "${mpu_test}: test" "$RUSTFS_E2E_LOG_DIR/list.log")" -eq 1
+          python3 scripts/e2e_binary.py run -- cargo test --locked -p e2e_test --lib \
+            "$mpu_test" -- --exact --nocapture --test-threads=1 \
+            2>&1 | tee "$RUSTFS_E2E_LOG_DIR/test.log"
+
       - name: Upload scanner/heal G09 evidence
         if: always()
         uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1

.github/workflows/functional-chain-health.yml

@@ -12,44 +12,13 @@ concurrency:
 jobs:
   collect:
     if: github.repository == 'rustfs/rustfs'
-    runs-on: smoke-testing
+    # Health remains available when either laboratory fleet is offline.
+    runs-on: ubuntu-latest
     timeout-minutes: 15
     steps:
       - uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
         with:
           persist-credentials: false
-      - name: Ensure GitHub CLI
-        shell: bash
-        run: |
-          if command -v gh >/dev/null 2>&1; then
-            gh --version
-            exit 0
-          fi
-
-          gh_version="2.101.0"
-          case "$(uname -m)" in
-            x86_64) gh_arch="amd64" ;;
-            aarch64|arm64) gh_arch="arm64" ;;
-            *)
-              echo "Unsupported Linux architecture for GitHub CLI: $(uname -m)" >&2
-              exit 1
-              ;;
-          esac
-
-          gh_install_root="${RUNNER_TEMP:-$PWD}/gh-cli"
-          rm -rf "$gh_install_root"
-          mkdir -p "$gh_install_root/bin"
-          archive="$gh_install_root/gh.tar.gz"
-          curl --fail --location --retry 3 \
-            "https://github.com/cli/cli/releases/download/v${gh_version}/gh_${gh_version}_linux_${gh_arch}.tar.gz" \
-            --output "$archive"
-          tar -xzf "$archive" -C "$gh_install_root"
-          install -m 0755 \
-            "$gh_install_root/gh_${gh_version}_linux_${gh_arch}/bin/gh" \
-            "$gh_install_root/bin/gh"
-          echo "$gh_install_root/bin" >> "$GITHUB_PATH"
-          export PATH="$gh_install_root/bin:$PATH"
-          gh --version
       - name: Collect and publish verified chain health
         env:
           GH_TOKEN: ${{ secrets.PF_TESTING_GH_TOKEN }}

.github/workflows/gateway-acceptance.yml

@@ -0,0 +1,395 @@
+# Copyright 2026 RustFS Team
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# Gateway black-box acceptance lanes (rustfs/backlog#2775 T3.4, #2776 T3.5).
+#
+# Runs two suites of the rustfs/gateway repository against the candidate binary,
+# once per S3 stack (RUSTFS_S3_STACK=legacy and =gateway), one server per stack:
+#
+#   - the conformance corpus, `--profile rustfs`. That profile describes the
+#     gateway edge, so scripts/check_conformance_rulings.py judges the gateway
+#     stack against .config/conformance-rulings.toml (red: an unruled failure, a
+#     ruled case failing differently, a stale ruling, a skip on the server's own
+#     answer) and publishes the legacy run as a reference that is never judged:
+#     it only has to serve, stay up and complete;
+#   - the client matrix, ci/compat/run_matrix.sh --external, judged per stack by
+#     scripts/compat_matrix_report.py (no `fail`, no `sut-unregistered`, every
+#     client ran; a client that could not be installed is reported as skipped).
+#     The legacy matrix is the baseline: every cell that passes on legacy must
+#     pass on gateway (`compare`).
+#
+# The suites are checked out at the rustfs/gateway rev Cargo.toml pins, so the
+# suite and the framework under test never drift apart. The verdict of each
+# stack is computed from what the run left on disk (startup record, the stack
+# the server's own log names, runner exit code, liveness after the corpus,
+# report), never from the exit status of an earlier step. Needs
+# scripts/dual_stack.py and the `s3_stack` startup field of rustfs/rustfs#9367.
+#
+# The gateway stack is report-only until T2.10: its job may fail without failing
+# the workflow, and its summary says what it observed (did not start, or how many
+# cases ran, passed, failed and were answered 501). The stack comparison is
+# report-only for the same reason; its conformance half is information only.
+#
+# Not a pull-request gate. Scheduled failures are owned by the scheduled-validation
+# watchdog (.github/scheduled-validations.json).
+
+name: Gateway Acceptance
+
+on:
+  schedule:
+    - cron: "29 3 * * *"
+  workflow_dispatch:
+
+# Matches ci.yml: these variables are hashed into the rust-cache key, so the
+# candidate build reads the same ci-dev cache as build-rustfs-debug-binary.
+env:
+  CARGO_TERM_COLOR: always
+  RUST_BACKTRACE: 1
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: true
+
+permissions:
+  contents: read
+
+defaults:
+  run:
+    shell: bash
+
+jobs:
+  plan:
+    name: Read the gateway pin
+    runs-on: ubuntu-latest
+    timeout-minutes: 5
+    outputs:
+      gateway-rev: ${{ steps.pin.outputs.rev }}
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
+        with:
+          persist-credentials: false
+
+      - name: Read the rustfs/gateway rev from Cargo.toml
+        id: pin
+        run: |
+          set -euo pipefail
+          python3 - <<'PY' >> "$GITHUB_OUTPUT"
+          import re, sys, tomllib
+          dependencies = tomllib.load(open("Cargo.toml", "rb"))["workspace"]["dependencies"]
+          revs = {spec.get("rev") for spec in dependencies.values()
+                  if isinstance(spec, dict) and spec.get("git") == "https://github.com/rustfs/gateway.git"}
+          if len(revs) != 1 or not re.fullmatch(r"[0-9a-f]{40}", str(next(iter(revs)))):
+              sys.exit(f"Cargo.toml must pin rustfs/gateway at exactly one full rev, found {sorted(map(str, revs))}")
+          print(f"rev={revs.pop()}")
+          PY
+          cat "$GITHUB_OUTPUT"
+
+  build-candidate:
+    name: Build candidate binary
+    # The same debug build as ci.yml's build-rustfs-debug-binary, reading the
+    # ci-dev cache written on sm-standard-4.
+    runs-on: sm-standard-4
+    timeout-minutes: 60
+    env:
+      FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
+        with:
+          persist-credentials: false
+
+      - name: Setup Rust environment
+        uses: ./.github/actions/setup
+        with:
+          rust-version: stable
+          cache-shared-key: ci-dev
+          cache-save-if: 'false'
+          install-build-packaging-tools: 'false'
+
+      - name: Build debug binary
+        run: python3 scripts/e2e_binary.py build --bins --features e2e-test-hooks
+
+      - name: Upload candidate binary
+        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+        with:
+          name: gateway-acceptance-candidate-binary
+          overwrite: true
+          path: |
+            target/debug/rustfs
+            target/debug/rustfs.e2e.json
+          if-no-files-found: error
+          retention-days: 1
+
+  build-gateway-tools:
+    name: Build gateway conformance runner and compat-sut
+    needs: [ plan ]
+    # The gateway workspace builds with its own rust-toolchain.toml, which rustup
+    # installs on first use; nothing here names a toolchain.
+    runs-on: ubuntu-latest
+    timeout-minutes: 60
+    steps:
+      - name: Checkout rustfs/gateway at the pinned rev
+        uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
+        with:
+          repository: rustfs/gateway
+          ref: ${{ needs.plan.outputs.gateway-rev }}
+          path: gateway
+          persist-credentials: false
+
+      - name: Build
+        working-directory: gateway
+        run: |
+          cargo build --locked --release -p rustfs-gateway-conformance -p rustfs-gateway-compat-sut \
+            --bin rustfs-gateway-conformance --bin compat-sut
+
+      - name: Upload tools
+        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+        with:
+          name: gateway-acceptance-tools
+          overwrite: true
+          path: |
+            gateway/target/release/rustfs-gateway-conformance
+            gateway/target/release/compat-sut
+          if-no-files-found: error
+          retention-days: 1
+
+  acceptance:
+    name: ${{ matrix.stack == 'gateway' && 'Acceptance (gateway, report-only until T2.10)' || 'Acceptance (legacy)' }}
+    needs: [ plan, build-candidate, build-gateway-tools ]
+    # GitHub-hosted like the s3-tests targets of reference-parity.yml: the job
+    # only runs prebuilt binaries and the pinned clients, none of which needs Docker.
+    runs-on: ubuntu-latest
+    # About 3x the 15 minutes the legacy job took in run 43325101234 (client install about
+    # 8, matrix about 3, corpus about 3), so a hung server costs 45 runner minutes, not more.
+    timeout-minutes: 45
+    continue-on-error: ${{ matrix.stack == 'gateway' }}
+    strategy:
+      fail-fast: false
+      matrix:
+        stack: [ legacy, gateway ]
+    env:
+      STACK: ${{ matrix.stack }}
+      ENDPOINT: http://127.0.0.1:9000
+      # The conformance suite signs with these fixed fixture credentials (the AWS
+      # documentation example pair), so the candidate is started with them and the
+      # client matrix is handed the same pair.
+      ACCESS_KEY: AKIAIOSFODNN7EXAMPLE
+      SECRET_KEY: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
+        with:
+          persist-credentials: false
+
+      - name: Checkout rustfs/gateway at the pinned rev
+        uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
+        with:
+          repository: rustfs/gateway
+          ref: ${{ needs.plan.outputs.gateway-rev }}
+          path: gateway
+          persist-credentials: false
+
+      - name: Download candidate binary
+        uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
+        with:
+          name: gateway-acceptance-candidate-binary
+          path: target/debug
+
+      - name: Download gateway tools
+        uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
+        with:
+          name: gateway-acceptance-tools
+          path: tools
+
+      # Records `ready`, or what the process said when it exited, in lane/startup.txt. `ready`
+      # also needs the server's own `server_starting` log event to name the requested stack
+      # (its `s3_stack` field comes from rustfs/rustfs#9367; scripts/dual_stack.py reads it).
+      - name: Start candidate (${{ matrix.stack }} stack)
+        run: |
+          set -uo pipefail
+          chmod +x target/debug/rustfs tools/*
+          mkdir -p lane "$RUNNER_TEMP"/data/rustfs{0..3}
+          target/debug/rustfs --version | tee lane/candidate-version.txt
+          # The four directories share the runner's one disk; the bypass is the CI case the
+          # physical-disk guard sanctions (as in mint.yml).
+          RUSTFS_S3_STACK="$STACK" RUSTFS_UNSAFE_BYPASS_DISK_CHECK=true RUST_LOG=info target/debug/rustfs --address 127.0.0.1:9000 \
+            --access-key "$ACCESS_KEY" --secret-key "$SECRET_KEY" "$RUNNER_TEMP"/data/rustfs{0..3} \
+            > lane/rustfs.log 2>&1 &
+          pid=$!
+          echo "$pid" > lane/rustfs.pid
+          # Liveness first: a ready answer counts only while this process is still running.
+          for _ in $(seq 120); do
+            if ! kill -0 "$pid" 2>/dev/null; then
+              status=0
+              wait "$pid" || status=$?
+              echo "did not start: exit ${status}: $(grep -v '^[[:space:]]*$' lane/rustfs.log | tail -n 1)" > lane/startup.txt
+              cat lane/startup.txt
+              exit 1
+            fi
+            if curl -sf "$ENDPOINT/health/ready" >/dev/null; then
+              if python3 scripts/dual_stack.py stack --expect "$STACK" --out lane/stack.json lane/rustfs.log > /dev/null; then
+                echo ready > lane/startup.txt
+                exit 0
+              fi
+              echo "did not start: stack not observed" > lane/startup.txt
+              cat lane/startup.txt
+              exit 1
+            fi
+            sleep 1
+          done
+          echo "did not start: /health/ready did not answer within 120 seconds" > lane/startup.txt
+          exit 1
+
+      # The exit code is data for the judge step, not this step's result.
+      - name: Run gateway conformance (rustfs profile)
+        run: |
+          set +e
+          tools/rustfs-gateway-conformance run --root gateway/conformance --profile rustfs \
+            --endpoint "$ENDPOINT" --allow-external-fixtures \
+            --json lane/report.json > lane/conformance.log 2>&1
+          echo "$?" > lane/runner-exit.txt
+          if kill -0 "$(cat lane/rustfs.pid)" 2>/dev/null; then echo alive; else echo exited; fi > lane/alive-after-conformance.txt
+          tail -n 20 lane/conformance.log
+
+      - name: Set up Python
+        uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
+        with:
+          python-version: "3.12"
+
+      # Toolchains for the matrix's SDK driver programs, as the gateway's own
+      # client-matrix.yml sets them up; the client versions come from
+      # gateway/compat/versions.toml.
+      - uses: actions/setup-go@41dfa10bad2bb2ae585af6ee5bb4d7d973ad74ed # v5.1.0
+        with:
+          go-version: "1.26"
+          cache: false
+      - uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
+        with:
+          node-version: "22"
+      - uses: actions/setup-dotnet@67a3573c9a986a3f9c594539f4ab511d57bb3ce9 # v4.3.1
+        with:
+          dotnet-version: "8.0.x"
+      - uses: actions/setup-java@cf277c60eb25467037889841efdb72551f06f6c3 # v4.9.1
+        with:
+          distribution: temurin
+          java-version: "21"
+      - name: Install Maven
+        env:
+          MAVEN_VERSION: "3.9.16"
+          MAVEN_SHA512: "831a8591fe20c8243b1dbe7d71e3244f31d1665b0804b2e825e38cbbe5ce0cafb8338851f90780735568773e0a6cd07bbec107cda0b896b008b861075358b6f6"
+        run: |
+          set -euo pipefail
+          archive="$RUNNER_TEMP/apache-maven.tar.gz"
+          curl -sSfL -o "$archive" "https://archive.apache.org/dist/maven/maven-3/${MAVEN_VERSION}/binaries/apache-maven-${MAVEN_VERSION}-bin.tar.gz"
+          echo "${MAVEN_SHA512}  ${archive}" | sha512sum -c -
+          tar -xzf "$archive" -C "$RUNNER_TEMP"
+          echo "$RUNNER_TEMP/apache-maven-${MAVEN_VERSION}/bin" >> "$GITHUB_PATH"
+
+      # One client at a time, so a client that cannot be installed is reported
+      # as skipped with its reason instead of stopping the others.
+      - name: Install pinned clients
+        working-directory: gateway
+        run: |
+          set -uo pipefail
+          python3 -m pip install --disable-pip-version-check "pyyaml==6.0.2"
+          installed=()
+          for client in $(python3 -c 'import tomllib; print(" ".join(sorted(tomllib.load(open("compat/versions.toml", "rb"))["clients"])))'); do
+            if bash ci/compat/install_clients.sh --clients "$client" > "../lane/install-${client}.log" 2>&1; then
+              installed+=("$client")
+            else
+              echo "${client}: skipped: $(tail -n 1 "../lane/install-${client}.log")" >> ../lane/skipped-clients.txt
+            fi
+          done
+          echo "COMPAT_CLIENTS=$(IFS=,; echo "${installed[*]}")" >> "$GITHUB_ENV"
+          cat ../lane/skipped-clients.txt 2>/dev/null || echo "every pinned client installed"
+
+      # The exit code is data for the judge step, not this step's result.
+      - name: Run client matrix (external)
+        working-directory: gateway
+        env:
+          GATEWAY_COMPAT_SUT_BIN: ${{ github.workspace }}/tools/compat-sut
+          GATEWAY_COMPAT_RUN_DIR: ${{ github.workspace }}/lane/compat
+          GATEWAY_COMPAT_MATRIX_OUT: ${{ github.workspace }}/lane/compat/matrix.json
+        run: |
+          set +e
+          export GATEWAY_COMPAT_ACCESS_KEY="$ACCESS_KEY" GATEWAY_COMPAT_SECRET_KEY="$SECRET_KEY"
+          PATH="$(go env GOPATH)/bin:$PATH"
+          if [[ -f ../lane/skipped-clients.txt ]]; then
+            bash ci/compat/run_matrix.sh --external "$ENDPOINT" --product rustfs --clients "$COMPAT_CLIENTS"
+          else
+            bash ci/compat/run_matrix.sh --external "$ENDPOINT" --product rustfs
+          fi
+          echo "$?" > ../lane/matrix-exit.txt
+
+      - name: Judge and summarize
+        if: always()
+        run: |
+          set -uo pipefail
+          status=0
+          python3 scripts/check_conformance_rulings.py --stack "$STACK" --run-dir lane | tee -a "$GITHUB_STEP_SUMMARY" || status=1
+          python3 scripts/compat_matrix_report.py check --stack "$STACK" --run-dir lane | tee -a "$GITHUB_STEP_SUMMARY" || status=1
+          exit "$status"
+
+      - name: Upload lane results
+        if: always()
+        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+        with:
+          name: gateway-acceptance-lane-${{ matrix.stack }}
+          overwrite: true
+          path: |
+            lane/*.txt
+            lane/*.log
+            lane/report.json
+            lane/stack.json
+            lane/compat/matrix.json
+            lane/compat/probe.jsonl
+            lane/compat/sut.log
+            lane/compat/results/**
+          if-no-files-found: error
+          retention-days: 14
+
+  compare:
+    name: Compare stacks (report-only until T2.10)
+    needs: [ acceptance ]
+    if: always()
+    runs-on: ubuntu-latest
+    timeout-minutes: 10
+    continue-on-error: true
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
+        with:
+          persist-credentials: false
+
+      - name: Download lane results
+        uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
+        with:
+          pattern: gateway-acceptance-lane-*
+          path: lanes
+
+      - name: Diff legacy and gateway
+        run: |
+          set -uo pipefail
+          status=0
+          {
+            python3 scripts/check_conformance_rulings.py \
+              --diff lanes/gateway-acceptance-lane-legacy lanes/gateway-acceptance-lane-gateway || status=1
+            echo
+            python3 scripts/compat_matrix_report.py diff \
+              --legacy lanes/gateway-acceptance-lane-legacy --gateway lanes/gateway-acceptance-lane-gateway || status=1
+          } > diff.md
+          cat diff.md
+          cat diff.md >> "$GITHUB_STEP_SUMMARY"
+          exit "$status"

.github/workflows/helm-package.yml

@@ -1,136 +1,26 @@
-# Copyright 2024 RustFS Team
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-name: Publish helm chart to artifacthub
-
+# Copyright 2026 RustFS Team
+# SPDX-License-Identifier: Apache-2.0
+name: Build candidate Helm chart
 on:
-  workflow_run:
-    workflows: [ "Build and Release" ]
-    types: [ completed ]
-  workflow_dispatch:
+  workflow_call:
     inputs:
-      version:
-        description: "Release version to publish, e.g. 1.0.0-beta.1 or v1.0.0-beta.1"
-        required: true
-        default: "1.0.0-beta.1"
+      candidate_manifest:
         type: string
-
+        required: true
 permissions:
   contents: read
-
+  actions: read
 jobs:
-  build-helm-package:
+  package:
     runs-on: ubuntu-latest
-    timeout-minutes: 30
-    if: |
-      (github.event_name == 'workflow_dispatch' && !contains(github.event.inputs.version, '-preview')) ||
-      (
-        github.event.workflow_run.conclusion == 'success' &&
-        github.event.workflow_run.event == 'push' &&
-        contains(github.event.workflow_run.head_branch, '.') &&
-        !contains(github.event.workflow_run.head_branch, '-preview')
-      )
-
-    outputs:
-      raw_tag: ${{ steps.version.outputs.raw_tag }}
-      app_version: ${{ steps.version.outputs.app_version }}
-      chart_version: ${{ steps.version.outputs.chart_version }}
-
+    timeout-minutes: 15
     steps:
-      - name: Checkout release source
-        uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
+      - uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
         with:
-          ref: ${{ github.event.workflow_run.head_sha || (startsWith(inputs.version, 'refs/tags/') && inputs.version || format('refs/tags/{0}', inputs.version)) }}
+          ref: ${{ fromJson(inputs.candidate_manifest).source_sha }}
           persist-credentials: false
-
-      # Both inputs reach the shell through env rather than `${{ }}`
-      # interpolation. A git ref name may contain `$(...)` — anything without a
-      # space is a legal tag — and interpolation pastes it into the script
-      # verbatim, where bash would run it. Reading "$RAW_INPUT" instead makes it
-      # data.
-      - name: Normalize release version
-        id: version
-        env:
-          RAW_INPUT: ${{ github.event.inputs.version }}
-          RAW_BRANCH: ${{ github.event.workflow_run.head_branch }}
-        run: |
-          set -eux
-
-          if [ "${{ github.event_name }}" = "workflow_dispatch" ]; then
-            RAW="$RAW_INPUT"
-          else
-            RAW="$RAW_BRANCH"
-          fi
-
-          case "$RAW" in
-            refs/tags/*)
-              RAW_TAG="${RAW#refs/tags/}"
-              ;;
-            *)
-              RAW_TAG="$RAW"
-              ;;
-          esac
-
-          ./scripts/helm_chart_version.sh "$RAW_TAG"
-
-      # Docker and Helm share the binary-build trigger. Do not publish a chart
-      # until the image selected by its default appVersion is pullable.
-      - name: Wait for release image
-        timeout-minutes: 20
-        env:
-          APP_VERSION: ${{ steps.version.outputs.app_version }}
-        run: |
-          set -euo pipefail
-          if [[ ! "$APP_VERSION" =~ ^[0-9]+\.[0-9]+\.[0-9]+(-[0-9A-Za-z.-]+)?$ ]] ||
-             [[ "$APP_VERSION" == *-preview* ]]; then
-            echo "Invalid deliverable image version: $APP_VERSION" >&2
-            exit 1
-          fi
-
-          for attempt in {1..60}; do
-            if token=$(curl --fail --silent --show-error --connect-timeout 10 --max-time 20 \
-              'https://auth.docker.io/token?service=registry.docker.io&scope=repository:rustfs/rustfs:pull' \
-              | jq -er '.token // empty') &&
-              curl --fail --silent --show-error --head --connect-timeout 10 --max-time 20 \
-                --header "Authorization: Bearer $token" \
-                --header 'Accept: application/vnd.oci.image.index.v1+json, application/vnd.docker.distribution.manifest.list.v2+json' \
-                "https://registry-1.docker.io/v2/rustfs/rustfs/manifests/$APP_VERSION" > /dev/null; then
-              echo "Release image is available: rustfs/rustfs:$APP_VERSION"
-              exit 0
-            fi
-            echo "Waiting for rustfs/rustfs:$APP_VERSION ($attempt/60)"
-            if (( attempt < 60 )); then
-              sleep 15
-            fi
-          done
-
-          echo "Release image is unavailable; refusing to publish Helm chart" >&2
-          exit 1
-
-      - name: Replace chart version and app version
-        env:
-          CHART_VERSION: ${{ steps.version.outputs.chart_version }}
-          APP_VERSION: ${{ steps.version.outputs.app_version }}
-        run: |
-          set -eux
-          sed -i -E "s/^version:.*/version: \"${CHART_VERSION}\"/" helm/rustfs/Chart.yaml
-          sed -i -E "s/^appVersion:.*/appVersion: \"${APP_VERSION}\"/" helm/rustfs/Chart.yaml
-
-      - name: Set up Helm
-        uses: azure/setup-helm@b9e51907a09c216f16ebe8536097933489208112 # v4.3.0
-
-      - name: Install yq
+      - uses: azure/setup-helm@b9e51907a09c216f16ebe8536097933489208112 # v4.3.0
+      - name: Install pinned yq
         run: |
           set -euo pipefail
           yq_version="4.54.1"
@@ -145,66 +35,43 @@ jobs:
           echo "$tools_dir" >> "$GITHUB_PATH"
           "$tools_dir/yq" --version
 
-      - name: Test Helm Chart Templates
-        run: ./scripts/test_helm_templates.sh
-
-      - name: Package Helm Chart
-        run: |
-          set -eux
-          cp helm/README.md helm/rustfs/
-          helm package ./helm/rustfs \
-            --destination helm/rustfs/ \
-            --version "${{ steps.version.outputs.chart_version }}"
-
-      - name: Upload helm package as artifact
-        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
-        with:
-          name: helm-package
-          path: |
-            helm/rustfs/*.tgz
-            helm/rustfs/README.md
-          retention-days: 1
-
-  publish-helm-package:
-    runs-on: ubuntu-latest
-    timeout-minutes: 30
-    needs: [ build-helm-package ]
-    if: needs.build-helm-package.result == 'success'
-
-    steps:
-      - name: Checkout helm package repo
-        uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
-        with:
-          # persist-credentials-exempt: this checkout's token IS the push credential —
-          # the job git-pushes to rustfs/helm below. Clearing it breaks chart publishing.
-          repository: rustfs/helm
-          token: ${{ secrets.RUSTFS_HELM_PACKAGE }}
-
-      - name: Download helm package
-        uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
-        with:
-          name: helm-package
-          path: ./
-
-      - name: Set up helm
-        uses: azure/setup-helm@b9e51907a09c216f16ebe8536097933489208112 # v4.3.0
-
-      - name: Generate index
-        run: helm repo index . --url https://charts.rustfs.com
-
-      # app_version is derived from the triggering tag name, and this job holds
-      # the cross-repository push token with rustfs/helm already checked out —
-      # the worst place in the repo to paste an attacker-influenced string into
-      # a shell line. Passed through env so bash treats it as data.
-      - name: Push helm package and index file
+      - name: Test and package the exact application version
+        id: chart
         env:
-          GIT_USERNAME: ${{ secrets.USERNAME }}
-          GIT_EMAIL: ${{ secrets.EMAIL_ADDRESS }}
-          APP_VERSION: ${{ needs.build-helm-package.outputs.app_version }}
+          GH_TOKEN: ${{ github.token }}
+          CANDIDATE_MANIFEST: ${{ inputs.candidate_manifest }}
         run: |
-          set -eux
-          git config --global user.name "${GIT_USERNAME}"
-          git config --global user.email "${GIT_EMAIL}"
-          git add .
-          git commit -m "Update rustfs helm package with ${APP_VERSION}." || echo "No changes to commit"
-          git push origin main
+          python3 - <<'PYCHART'
+          import sys, subprocess, shutil
+          from pathlib import Path
+          sys.path.insert(0, 'scripts/release')
+          import candidate as c, pipeline
+          inputs = pipeline.manifest_env()
+          current = pipeline.current_run('release-complete.yml')
+          if any(a['name'] == 'helm-package' for a in c.artifact_list(current['id'])):
+              c.download_artifact(current, 'helm-package', 'chart')
+              record = c.read_json('chart/chart.json')
+              c.require(record['inputs_sha256'] == c.digest(inputs), 'chart belongs to another candidate')
+              c.emit_output('reused', 'true')
+          else:
+              chart = Path('helm/rustfs/Chart.yaml')
+              import re
+              value = chart.read_text()
+              for key in ('version', 'appVersion'):
+                  value, count = re.subn(rf'(?m)^{key}:.*$', f'{key}: "{inputs["version"]}"', value)
+                  c.require(count == 1, 'ambiguous chart metadata')
+              chart.write_text(value)
+              subprocess.run(['bash', 'scripts/test_helm_templates.sh'], check=True)
+              shutil.copyfile('helm/README.md', 'helm/rustfs/README.md')
+              subprocess.run(['helm', 'package', 'helm/rustfs', '--destination', 'chart'], check=True)
+              path = Path('chart') / f"rustfs-{inputs['version']}.tgz"
+              c.write_json('chart/chart.json', {'version': inputs['version'], 'inputs_sha256': c.digest(inputs),
+                                               'name': path.name, 'sha256': c.file_digest(path), 'size': path.stat().st_size})
+          PYCHART
+      - uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+        if: steps.chart.outputs.reused != 'true'
+        with:
+          name: helm-package
+          path: chart/*
+          if-no-files-found: error
+          retention-days: 30

.github/workflows/manual-main-packages.yml

@@ -0,0 +1,463 @@
+# Copyright 2026 RustFS Team
+# SPDX-License-Identifier: Apache-2.0
+
+# Manual Linux zip/deb/rpm packaging. Workflow artifacts only.
+# Dispatch uses this file from the selected branch. An empty ref packages
+# the tip of refs/heads/main. A non-empty ref must already exist; the
+# resolver fetches that tag, branch, or commit so checkout is not limited
+# to the default branch. Nothing is published.
+
+name: Manual Linux packages
+
+on:
+  workflow_dispatch:
+    inputs:
+      architecture:
+        description: "Linux targets: x86 (x86_64 musl+gnu), arm (aarch64 musl+gnu), or all"
+        required: true
+        type: choice
+        options:
+          - x86
+          - arm
+          - all
+        default: all
+      ref:
+        description: "Git tag to package. Leave empty to package the tip of origin/main. A branch name or full commit SHA is also accepted."
+        required: false
+        type: string
+        default: ""
+      version:
+        description: "Package version: MAJOR.MINOR.PATCH, or MAJOR.MINOR.PATCH-(alpha|beta|rc).N. No leading v. Required for origin/main. For a tag, leave empty to derive it from the tag; an explicit value is used as entered."
+        required: false
+        type: string
+        default: ""
+
+permissions:
+  contents: read
+
+env:
+  CARGO_TERM_COLOR: always
+  RUST_BACKTRACE: 1
+  CARGO_INCREMENTAL: 0
+
+jobs:
+  prepare:
+    name: Resolve source and version
+    if: github.repository == 'rustfs/rustfs'
+    runs-on: ubuntu-latest
+    timeout-minutes: 15
+    outputs:
+      source_sha: ${{ steps.source.outputs.source_sha }}
+      source_kind: ${{ steps.source.outputs.source_kind }}
+      source_name: ${{ steps.source.outputs.source_name }}
+      version: ${{ steps.source.outputs.version }}
+      packaging: ${{ steps.source.outputs.packaging }}
+      tool_sha: ${{ steps.source.outputs.tool_sha }}
+      confirm_from: ${{ steps.source.outputs.confirm_from }}
+      toolchain: ${{ steps.inputs.outputs.toolchain }}
+      console: ${{ steps.inputs.outputs.console }}
+      matrix: ${{ steps.matrix.outputs.matrix }}
+    steps:
+      # Tooling checkout. An empty ref then detaches this tree to the tip of
+      # origin/main. A tag or other ref is fetched and pinned without
+      # replacing this tree, because console resolution needs candidate.py.
+      - name: Checkout repository
+        uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
+        with:
+          ref: main
+          fetch-depth: 0
+          persist-credentials: false
+
+      - name: Resolve source and version
+        id: source
+        shell: bash
+        env:
+          SOURCE_REF: ${{ inputs.ref }}
+          PACKAGE_VERSION: ${{ inputs.version }}
+        run: |
+          set -euo pipefail
+          bash scripts/release/package_manual_artifacts.sh resolve-source \
+            --source "$SOURCE_REF" \
+            --version "$PACKAGE_VERSION" | tee -a "$GITHUB_OUTPUT"
+
+      - name: Install stable Rust
+        uses: dtolnay/rust-toolchain@29eef336d9b2848a0b548edc03f92a220660cdb8 # stable
+        with:
+          toolchain: stable
+
+      - name: Resolve console assets and toolchain
+        id: inputs
+        shell: bash
+        env:
+          GH_TOKEN: ${{ github.token }}
+        run: |
+          set -euo pipefail
+          toolchain=$(rustc --version | awk '{print $2}')
+          [[ "$toolchain" =~ ^[0-9]+\.[0-9]+\.[0-9]+$ ]]
+          gh api repos/rustfs/console/releases/latest > console-release.json
+          TOOLCHAIN="$toolchain" python3 - <<'PY'
+          import json, os, sys
+          sys.path.insert(0, "scripts/release")
+          import candidate
+          console = candidate.console_asset(candidate.read_json("console-release.json"))
+          candidate.emit_output("console", json.dumps(console, separators=(",", ":")))
+          candidate.emit_output("toolchain", os.environ["TOOLCHAIN"])
+          print(f"toolchain={os.environ['TOOLCHAIN']}")
+          print(f"console={console['tag']} {console['name']}")
+          PY
+
+      - name: Select Linux targets
+        id: matrix
+        shell: bash
+        env:
+          ARCHITECTURE: ${{ inputs.architecture }}
+        run: |
+          set -euo pipefail
+          matrix=$(bash scripts/release/package_manual_artifacts.sh matrix "$ARCHITECTURE")
+          count=$(python3 -c 'import json,sys; print(len(json.loads(sys.argv[1])["include"]))' "$matrix")
+          [[ "$count" -gt 0 ]]
+          printf 'matrix=%s\n' "$matrix" >> "$GITHUB_OUTPUT"
+          printf 'Selected %s target(s) for %s\n' "$count" "$ARCHITECTURE"
+
+  package:
+    name: Package RustFS (${{ matrix.target_id }})
+    needs: prepare
+    if: github.repository == 'rustfs/rustfs'
+    # x86_64 uses sm-standard-4, the official release compile pool: the workspace
+    # release build needs that disk (placement rule: disk) and reads build.yml's
+    # cache without writing it. aarch64 uses ubicloud-standard-16-arm so the
+    # packaged binary and console smoke execute natively (placement rule: hardware).
+    runs-on: ${{ matrix.os }}
+    timeout-minutes: 210
+    env:
+      FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"
+      RUSTUP_TOOLCHAIN: ${{ needs.prepare.outputs.toolchain }}
+      RUSTFLAGS: "${{ matrix.rustflags }}"
+    strategy:
+      fail-fast: false
+      matrix: ${{ fromJson(needs.prepare.outputs.matrix) }}
+    steps:
+      - name: Checkout the pinned source commit
+        uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
+        with:
+          ref: ${{ needs.prepare.outputs.source_sha }}
+          fetch-depth: 0
+          persist-credentials: false
+
+      # Tags cut before the manual packaging script still compile from their
+      # own tree. The official zip/deb/rpm recipe, and confirm-source when the
+      # tagged script does not implement it, come from the prepare checkout.
+      - name: Checkout packaging recipe
+        if: ${{ needs.prepare.outputs.packaging == 'tool' || needs.prepare.outputs.confirm_from == 'tool' }}
+        uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
+        with:
+          ref: ${{ needs.prepare.outputs.tool_sha }}
+          path: packaging-tools
+          fetch-depth: 1
+          persist-credentials: false
+
+      - name: Confirm the pinned source
+        shell: bash
+        env:
+          SOURCE_KIND: ${{ needs.prepare.outputs.source_kind }}
+          SOURCE_NAME: ${{ needs.prepare.outputs.source_name }}
+          SOURCE_SHA: ${{ needs.prepare.outputs.source_sha }}
+        run: |
+          set -euo pipefail
+          helper=scripts/release/package_manual_artifacts.sh
+          if [[ ! -f "$helper" ]] || ! grep -q 'confirm-source)' "$helper"; then
+            helper=packaging-tools/scripts/release/package_manual_artifacts.sh
+          fi
+          if [[ ! -f "$helper" ]] || ! grep -q 'confirm-source)' "$helper"; then
+            printf 'no packaging script implements confirm-source\n' >&2
+            exit 1
+          fi
+          bash "$helper" confirm-source \
+            --kind "$SOURCE_KIND" \
+            --name "$SOURCE_NAME" \
+            --sha "$SOURCE_SHA"
+
+      - name: Setup Rust environment
+        uses: ./.github/actions/setup
+        with:
+          rust-version: ${{ needs.prepare.outputs.toolchain }}
+          target: ${{ matrix.target }}
+          cache-shared-key: build-${{ matrix.target }}${{ matrix.glibc && format('-glibc-{0}', matrix.glibc) || '' }}${{ matrix.os == 'ubicloud-standard-16-arm' && '-ubicloud-arm' || '' }}
+          # build.yml owns this key. This job only restores it.
+          cache-save-if: 'false'
+          install-cross-tools: ${{ matrix.cross }}
+          install-build-packaging-tools: 'true'
+          install-test-tools: 'false'
+
+      - name: Download static console assets
+        shell: bash
+        env:
+          GH_TOKEN: ${{ github.token }}
+          CONSOLE_IDENTITY: ${{ needs.prepare.outputs.console }}
+        run: |
+          set -euo pipefail
+          python3 - <<'PYCONSOLE'
+          import json, os, pathlib, subprocess, zipfile
+          value = json.loads(os.environ["CONSOLE_IDENTITY"])
+          subprocess.run(["python3", ".github/actions/setup/download_release_asset.py",
+              "--repository", "rustfs/console", "--tag", value["tag"], "--asset", value["name"],
+              "--sha256", value["sha256"], "--output", "console.zip"], check=True)
+          destination = pathlib.Path("rustfs/static")
+          destination.mkdir(parents=True, exist_ok=True)
+          with zipfile.ZipFile("console.zip") as archive:
+              for member in archive.infolist():
+                  path = pathlib.PurePosixPath(member.filename)
+                  if path.is_absolute() or ".." in path.parts or "\\" in member.filename:
+                      raise SystemExit("unsafe Console archive member")
+              archive.extractall(destination)
+          if not (destination / "index.html").is_file():
+              raise SystemExit("Console archive has no index.html")
+          PYCONSOLE
+          rm -f console.zip
+
+      - name: Build RustFS
+        shell: bash
+        env:
+          RUSTFS_BUILD_VERSION: ${{ needs.prepare.outputs.version }}
+          BUILD_TARGET: ${{ matrix.target }}
+          BUILD_OS: ${{ matrix.build_os }}
+          BUILD_FEATURES: ${{ matrix.features }}
+          BUILD_GLIBC: ${{ matrix.glibc }}
+          BUILD_CROSS: ${{ matrix.cross }}
+        run: |
+          set -euo pipefail
+          touch rustfs/build.rs
+
+          feature_args=()
+          if [[ -n "$BUILD_FEATURES" ]]; then
+            feature_args=(--features "$BUILD_FEATURES")
+          fi
+
+          # gnu targets pin glibc 2.31 through cargo-zigbuild (<target>.<glibc>).
+          target_arg="$BUILD_TARGET"
+          if [[ -n "$BUILD_GLIBC" ]]; then
+            target_arg="${target_arg}.${BUILD_GLIBC}"
+          fi
+
+          profile_args=()
+          if [[ "$BUILD_TARGET" == *-linux-gnu && "$BUILD_FEATURES" == pyroscope ]]; then
+            profile_args=(--config 'profile.release.strip="none"')
+          fi
+
+          case "$BUILD_CROSS" in
+            true)
+              cargo zigbuild --release --locked --target "$target_arg" "${feature_args[@]}" "${profile_args[@]}" -p rustfs --bin rustfs
+              ;;
+            false)
+              cargo build --release --locked --target "$target_arg" "${feature_args[@]}" "${profile_args[@]}" -p rustfs --bin rustfs
+              ;;
+            *)
+              printf 'unexpected cross flag: %q\n' "$BUILD_CROSS" >&2
+              exit 1
+              ;;
+          esac
+
+          bin="target/${BUILD_TARGET}/release/rustfs"
+          version_output=$("$bin" --version)
+          first=${version_output%%$'\n'*}
+          if [[ "$first" != "rustfs ${RUSTFS_BUILD_VERSION}" ]]; then
+            printf 'binary version line is %q, expected %q\n' "$first" "rustfs ${RUSTFS_BUILD_VERSION}" >&2
+            printf '%s\n' "$version_output" >&2
+            exit 1
+          fi
+          if ! grep -Fxq "build target : ${BUILD_TARGET}" <<< "$version_output" || \
+             ! grep -Fxq "build os     : ${BUILD_OS}" <<< "$version_output" || \
+             ! grep -Fxq "rust channel : stable-${BUILD_TARGET}" <<< "$version_output"; then
+            printf 'release binary reports an unexpected compile target\n' >&2
+            printf '%s\n' "$version_output" >&2
+            exit 1
+          fi
+          if [[ -n "$BUILD_GLIBC" ]]; then
+            glibc_check=scripts/release/check_glibc.py
+            if [[ ! -f "$glibc_check" ]]; then
+              glibc_check=packaging-tools/scripts/release/check_glibc.py
+            fi
+            if [[ ! -f "$glibc_check" ]]; then
+              printf 'glibc check script not found\n' >&2
+              exit 1
+            fi
+            python3 "$glibc_check" "$BUILD_GLIBC" "$bin"
+          fi
+
+      - name: Generate the CPU symbol catalog
+        if: endsWith(matrix.target, '-linux-gnu') && matrix.features == 'pyroscope'
+        shell: bash
+        env:
+          BUILD_TARGET: ${{ matrix.target }}
+        run: |
+          set -euo pipefail
+          cargo metadata --locked --format-version 1 --filter-platform "$BUILD_TARGET" \
+            --features rustfs/pyroscope > "$RUNNER_TEMP/cpu-catalog-metadata.json"
+          features=$(jq -ce '
+            (.packages[] | select(.name == "rustfs") | .id) as $id |
+            .resolve.nodes[] | select(.id == $id) | .features | map(select(. != "default")) | sort
+          ' "$RUNNER_TEMP/cpu-catalog-metadata.json")
+          release_dir="target/${BUILD_TARGET}/release"
+          rm -f "$release_dir/rustfs.cpu-symbol-catalog.json" "$release_dir/rustfs.cpu-symbol-catalog.sha256"
+          cargo run --locked -p rustfs-connect-symbol-catalog -- \
+            "$release_dir/rustfs" "$(git rev-parse HEAD)" "$BUILD_TARGET" "$features" \
+            "$release_dir/rustfs.cpu-symbol-catalog.json" > "$release_dir/rustfs.cpu-symbol-catalog.sha256"
+
+      - name: Install DEB and RPM tools
+        shell: bash
+        run: |
+          set -euo pipefail
+          sudo apt-get update
+          sudo apt-get install -y ruby ruby-dev build-essential rpm fakeroot cpio
+          sudo gem install fpm --version 1.17.0 --no-document
+
+      - name: Package zip, deb, and rpm
+        shell: bash
+        env:
+          PACKAGE_VERSION: ${{ needs.prepare.outputs.version }}
+          ARCH: ${{ matrix.arch }}
+          LIBC: ${{ matrix.libc }}
+          BUILD_TARGET: ${{ matrix.target }}
+          BUILD_FEATURES: ${{ matrix.features }}
+          SOURCE_KIND: ${{ needs.prepare.outputs.source_kind }}
+          SOURCE_SHA: ${{ needs.prepare.outputs.source_sha }}
+          PACKAGING: ${{ needs.prepare.outputs.packaging }}
+        run: |
+          set -euo pipefail
+          case "$PACKAGING" in
+            source) script=scripts/release/package_manual_artifacts.sh ;;
+            tool) script=packaging-tools/scripts/release/package_manual_artifacts.sh ;;
+            *)
+              printf 'unexpected packaging recipe: %q\n' "$PACKAGING" >&2
+              exit 1
+              ;;
+          esac
+          if [[ ! -f "$script" ]]; then
+            printf 'packaging script not found: %s\n' "$script" >&2
+            exit 1
+          fi
+          binary="target/${BUILD_TARGET}/release/rustfs"
+          extra=()
+          if [[ "$BUILD_TARGET" == *-linux-gnu && "$BUILD_FEATURES" == pyroscope ]]; then
+            extra=(
+              --zip-extra "target/${BUILD_TARGET}/release/rustfs.cpu-symbol-catalog.json"
+              --zip-extra "target/${BUILD_TARGET}/release/rustfs.cpu-symbol-catalog.sha256"
+            )
+          fi
+          bash "$script" package \
+            --version "$PACKAGE_VERSION" \
+            --arch "$ARCH" \
+            --libc "$LIBC" \
+            --binary "$binary" \
+            --output dist \
+            "${extra[@]}"
+          printf 'Packaged %s from %s %s\n' "$PACKAGE_VERSION" "$SOURCE_KIND" "$SOURCE_SHA"
+
+      - name: Verify the packaged binary
+        shell: bash
+        env:
+          PACKAGE_VERSION: ${{ needs.prepare.outputs.version }}
+          ARCH: ${{ matrix.arch }}
+          LIBC: ${{ matrix.libc }}
+          BUILD_TARGET: ${{ matrix.target }}
+          BUILD_OS: ${{ matrix.build_os }}
+          BUILD_FEATURES: ${{ matrix.features }}
+        run: |
+          set -euo pipefail
+          stem="rustfs-linux-${ARCH}-${LIBC}-v${PACKAGE_VERSION}"
+          package_file="dist/${stem}.zip"
+          [[ -f "$package_file" && -f "dist/${stem}.deb" && -f "dist/${stem}.rpm" ]]
+          [[ "$(find dist -maxdepth 1 -type f | wc -l)" -eq 3 ]]
+          archive_members=$(unzip -Z1 "$package_file" | sort)
+          if [[ "$BUILD_TARGET" == *-linux-gnu && "$BUILD_FEATURES" == pyroscope ]]; then
+            expected_members=$(printf '%s\n' rustfs rustfs.cpu-symbol-catalog.json rustfs.cpu-symbol-catalog.sha256 | sort)
+          else
+            expected_members=$(printf '%s\n' rustfs)
+          fi
+          if [[ "$archive_members" != "$expected_members" ]]; then
+            printf 'Linux package does not contain the expected files\n' >&2
+            printf '%s\n' "$archive_members" >&2
+            exit 1
+          fi
+          binary_dir=$(mktemp -d)
+          trap 'rm -rf "$binary_dir"' EXIT
+          unzip -q "$package_file" rustfs -d "$binary_dir"
+          version_output=$("$binary_dir/rustfs" --version)
+          first=${version_output%%$'\n'*}
+          if [[ "$first" != "rustfs ${PACKAGE_VERSION}" ]]; then
+            printf 'packaged binary version line is %q, expected %q\n' "$first" "rustfs ${PACKAGE_VERSION}" >&2
+            exit 1
+          fi
+          if ! grep -Fxq "build target : ${BUILD_TARGET}" <<< "$version_output" || \
+             ! grep -Fxq "build os     : ${BUILD_OS}" <<< "$version_output"; then
+            printf 'packaged binary reports an unexpected compile target\n' >&2
+            exit 1
+          fi
+
+      - name: Verify packaged console
+        if: matrix.target == 'x86_64-unknown-linux-gnu' || matrix.target == 'aarch64-unknown-linux-gnu'
+        shell: bash
+        env:
+          PACKAGE_VERSION: ${{ needs.prepare.outputs.version }}
+          ARCH: ${{ matrix.arch }}
+          LIBC: ${{ matrix.libc }}
+        run: |
+          set -euo pipefail
+          package_file="dist/rustfs-linux-${ARCH}-${LIBC}-v${PACKAGE_VERSION}.zip"
+
+          find_free_port() {
+            python3 - <<'PY'
+          import socket
+
+          with socket.socket() as sock:
+              sock.bind(("127.0.0.1", 0))
+              print(sock.getsockname()[1])
+          PY
+          }
+
+          package_dir=$(mktemp -d)
+          data_dir=$(mktemp -d)
+          log_file="${RUNNER_TEMP}/rustfs-console-smoke.log"
+          server_pid=""
+          trap '
+            if [[ -n "${server_pid:-}" ]]; then
+              kill "$server_pid" 2>/dev/null || true
+              wait "$server_pid" 2>/dev/null || true
+            fi
+            rm -rf "$package_dir" "$data_dir"
+          ' EXIT
+
+          unzip -q "$package_file" -d "$package_dir"
+          api_port=$(find_free_port)
+          console_port=$(find_free_port)
+          console_url="http://127.0.0.1:${console_port}/rustfs/console/"
+
+          "$package_dir/rustfs" server \
+            --address "127.0.0.1:${api_port}" \
+            --console-enable \
+            --console-address "127.0.0.1:${console_port}" \
+            --access-key console-smoke \
+            --secret-key console-smoke-secret \
+            "$data_dir" >"$log_file" 2>&1 &
+          server_pid=$!
+
+          for _ in {1..40}; do
+            response=$(curl --silent --output /dev/null --write-out '%{http_code} %{content_type}' "$console_url" || true)
+            if [[ "$response" == 200\ text/html* ]]; then
+              exit 0
+            fi
+            sleep 0.25
+          done
+
+          printf 'Console endpoint did not return 200 text/html: %s\n' "$response" >&2
+          cat "$log_file"
+          exit 1
+
+      - name: Upload packages
+        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+        with:
+          name: packages-${{ matrix.target_id }}-v${{ needs.prepare.outputs.version }}
+          path: dist/*
+          if-no-files-found: error
+          compression-level: 0
+          retention-days: 30

.github/workflows/mint.yml

@@ -44,6 +44,13 @@
 # well inside the 120-minute budget and buys deterministic infra. The
 # docker-capable self-hosted `dind-sm-standard-2` label was the alternative but
 # has fewer cores and reintroduces fleet-state risk for no reliability gain.
+#
+# Stack dimension (rustfs/backlog#2774, T3.3): one image, run with RUSTFS_S3_STACK=legacy and
+# =gateway. `compare` (scripts/mint_stack_diff.py) holds the gate that every case legacy passes
+# passes on gateway with no new failure. The gateway leg and `compare` are report-only until
+# T2.10. A gateway server that refuses to start stops its leg at "Wait for RustFS ready", before
+# mint runs; the summary says `skipped: gateway stack refuses startup: <why>`. The stack is read
+# from the server's own start-up line (scripts/dual_stack.py stack).
 
 name: mint
 
@@ -108,13 +115,9 @@ defaults:
     shell: bash
 
 jobs:
-  mint:
-    # GitHub-hosted: reliably provides a running Docker daemon + buildx +
-    # python3. See the header note (ci-2) for why the self-hosted sm-standard-4
-    # label was abandoned (no docker.sock on the pod its only run landed on).
-    # TODO(ci-8): scheduled-failure alerting (auto-open issue on a red weekly
-    # run) is added by the ci-8 composite action; do not implement it here.
-    runs-on: sm-standard-2
+  image:
+    name: Build the RustFS image once
+    runs-on: ubuntu-latest
     timeout-minutes: 120
     steps:
       - uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
@@ -132,6 +135,48 @@ jobs:
             --cache-to "type=gha,mode=max,scope=${BUILDX_CACHE_SCOPE}" \
             -t rustfs-ci \
             -f Dockerfile.source .
+          docker save rustfs-ci --output "${RUNNER_TEMP}/rustfs-ci.tar"
+
+      - name: Upload the image
+        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+        with:
+          name: mint-image
+          path: ${{ runner.temp }}/rustfs-ci.tar
+          retention-days: 1
+
+  mint:
+    name: mint (${{ matrix.stack }}${{ matrix.stack == 'gateway' && ', report-only until T2.10' || '' }})
+    needs: [image]
+    # Report-only until T2.10: the gateway stack cannot serve the suites yet.
+    continue-on-error: ${{ matrix.stack == 'gateway' }}
+    strategy:
+      fail-fast: false
+      matrix:
+        stack: [legacy, gateway]
+    env:
+      STACK: ${{ matrix.stack }}
+    # GitHub-hosted: reliably provides a running Docker daemon + buildx +
+    # python3. See the header note (ci-2) for why the self-hosted sm-standard-4
+    # label was abandoned (no docker.sock on the pod its only run landed on).
+    # sm-standard-2 has no docker.sock either (weekly runs failed at "Enable
+    # buildx" from 2026-09-27 to 2026-11-08), so keep this job on ubuntu-latest.
+    # TODO(ci-8): scheduled-failure alerting (auto-open issue on a red weekly
+    # run) is added by the ci-8 composite action; do not implement it here.
+    runs-on: ubuntu-latest
+    timeout-minutes: 120
+    steps:
+      - uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
+        with:
+          persist-credentials: false
+
+      - name: Download the image
+        uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
+        with:
+          name: mint-image
+          path: ${{ runner.temp }}
+
+      - name: Load the image
+        run: docker load --input "${RUNNER_TEMP}/rustfs-ci.tar"
 
       - name: Start RustFS
         run: |
@@ -149,6 +194,8 @@ jobs:
             -e RUSTFS_SECRET_KEY="${S3_SECRET_KEY}" \
             -e RUSTFS_VOLUMES="/data/rustfs{0...3}" \
             -e RUSTFS_UNSAFE_BYPASS_DISK_CHECK=true \
+            -e RUSTFS_S3_STACK="${STACK}" \
+            -e RUST_LOG=rustfs=info \
             -v /tmp/rustfs-mint:/data \
             rustfs-ci
 
@@ -244,16 +291,50 @@ jobs:
           mkdir -p artifacts/rustfs-mint
           docker logs rustfs-mint > artifacts/rustfs-mint/rustfs.log 2>&1 || true
 
+      - name: Prove which stack the server selected
+        if: always()
+        run: python3 scripts/dual_stack.py stack --expect "${STACK}" --out artifacts/stack.json artifacts/rustfs-mint
+
       - name: Upload artifacts
         if: always() && env.ACT != 'true'
         uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
         with:
-          name: mint
+          name: mint-${{ matrix.stack }}
           path: artifacts/**
 
+  compare:
+    name: Compare stacks (report-only until T2.10)
+    needs: [mint]
+    if: always()
+    continue-on-error: true
+    runs-on: ubuntu-latest
+    timeout-minutes: 10
+    steps:
+      - uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
+        with:
+          persist-credentials: false
+
+      - name: Download both legs
+        uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
+        with:
+          pattern: mint-{legacy,gateway}
+          path: legs
+
+      # A leg that never uploaded leaves no directory: create both so that the
+      # comparison reports "no result" for it instead of failing to find it.
+      - name: Compare the stacks
+        run: |
+          # The shell runs with -e; the exit status is reported after the summary is written.
+          set +e -u -o pipefail
+          mkdir -p legs/mint-legacy legs/mint-gateway
+          python3 scripts/mint_stack_diff.py legs/mint-legacy legs/mint-gateway | tee report.txt
+          status="${PIPESTATUS[0]}"
+          { echo '## Mint: gateway versus legacy'; echo; echo '```'; cat report.txt; echo '```'; } >> "$GITHUB_STEP_SUMMARY"
+          exit "$status"
+
   alert-on-failure:
     name: Alert on scheduled failure
-    needs: [mint]
+    needs: [image, mint]
     # `always()` is required: without it this job is skipped when a needed
     # job fails. Alerts only for scheduled runs (backlog#1149 ci-8) — manual
     # dispatch failures are already watched by a human.

.github/workflows/nightly-gnu.yml

@@ -87,18 +87,206 @@ jobs:
           install-build-packaging-tools: 'false'
           install-test-tools: 'false'
 
+      - name: Download static console assets
+        shell: bash
+        env:
+          GITHUB_TOKEN: ${{ github.token }}
+        run: |
+          # Packaging tools are off on this lane, so setup does not install unzip.
+          if ! command -v unzip >/dev/null 2>&1; then
+            sudo apt-get update -qq
+            sudo apt-get install -y -qq unzip
+          fi
+          if ! command -v unzip >/dev/null 2>&1; then
+            echo "unzip is required to extract console assets" >&2
+            exit 1
+          fi
+
+          mkdir -p ./rustfs/static
+
+          verify_sha256() {
+            local expected="$1"
+            local file="$2"
+            local actual=""
+            local actual_line=""
+
+            if command -v sha256sum >/dev/null 2>&1; then
+              actual_line=$(sha256sum -- "$file") || return 1
+            elif command -v shasum >/dev/null 2>&1; then
+              actual_line=$(shasum -a 256 -- "$file") || return 1
+            else
+              echo "No SHA-256 verification tool found" >&2
+              return 1
+            fi
+            actual="${actual_line%%[[:space:]]*}"
+
+            if [ "$actual" = "$expected" ]; then
+              echo "SHA256 verified OK: $actual"
+              return 0
+            else
+              echo "SHA256 mismatch: expected=$expected actual=$actual" >&2
+              return 1
+            fi
+          }
+
+          download_console_assets() {
+            local curl_bin="curl"
+            local python_bin="python3"
+            local console_api="https://api.github.com/repos/rustfs/console/releases/latest"
+            local console_json="console-release.json"
+            local console_url
+            local console_sha256
+            local curl_auth_args=()
+            console_tag=""
+
+            chmod +w ./rustfs/static/LICENSE || true
+
+            if ! command -v "$python_bin" >/dev/null 2>&1; then
+              python_bin="python"
+            fi
+            if ! command -v "$python_bin" >/dev/null 2>&1; then
+              echo "No Python interpreter found for release metadata parsing" >&2
+              return 1
+            fi
+
+            if [[ -n "${GITHUB_TOKEN:-}" ]]; then
+              curl_auth_args=(-H "Authorization: Bearer ${GITHUB_TOKEN}" -H "X-GitHub-Api-Version: 2022-11-28")
+            fi
+
+            "$curl_bin" "${curl_auth_args[@]}" --fail -L "$console_api" \
+              -o "$console_json" --retry 3 --retry-delay 5 --max-time 300 || return 1
+
+            read -r console_tag console_url console_sha256 < <("$python_bin" - "$console_json" <<'PY'
+          import json
+          import re
+          import sys
+
+          with open(sys.argv[1], encoding="utf-8") as handle:
+              release = json.load(handle)
+
+          tag = release.get("tag_name", "") or "unknown"
+
+          for asset in release.get("assets", []):
+              name = asset.get("name", "")
+              digest = asset.get("digest", "")
+              url = asset.get("browser_download_url", "")
+              if name.startswith("rustfs-console-") and name.endswith(".zip") and digest.startswith("sha256:") and url:
+                  sha256 = digest.split(":", 1)[1]
+                  if not re.fullmatch(r"[0-9a-fA-F]{64}", sha256):
+                      raise SystemExit(f"console zip asset has invalid sha256 digest: {sha256}")
+                  sys.stdout.buffer.write(f"{tag} {url} {sha256}\n".encode("utf-8"))
+                  break
+          else:
+              raise SystemExit("no console zip asset with sha256 digest found")
+          PY
+            ) || return 1
+
+            if [[ -z "$console_url" || -z "$console_sha256" ]]; then
+              echo "Console release metadata is missing URL or SHA-256 digest" >&2
+              return 1
+            fi
+
+            echo "Console release: ${console_tag}"
+            echo "Downloading console asset: ${console_url}"
+            "$curl_bin" --fail -L "$console_url" -o console.zip --retry 3 --retry-delay 5 --max-time 300 || return 1
+            verify_sha256 "$console_sha256" console.zip || return 2
+            unzip -o console.zip -d ./rustfs/static || return 2
+          }
+
+          if download_console_assets; then
+            rm -f console.zip console-release.json
+          else
+            status=$?
+            rm -f console.zip console-release.json
+            if [[ "$status" -eq 2 ]]; then
+              echo "Console asset integrity verification failed" >&2
+            fi
+            echo "Failed to download verified console assets" >&2
+            exit 1
+          fi
+
+          if [[ ! -s ./rustfs/static/index.html ]]; then
+            echo "Console asset archive is missing static/index.html" >&2
+            exit 1
+          fi
+
+          asset_count=$(find ./rustfs/static -type f | wc -l | tr -d '[:space:]')
+          echo "Console assets ready: version=${console_tag:-unknown}, ${asset_count} files extracted to ./rustfs/static"
+
       - name: Build RustFS
         # Pin the glibc baseline (`<target>.<glibc>` for cargo-zigbuild) so
         # nightlies run on older distros such as RHEL/Rocky 9 (glibc 2.34); #8062.
-        run: cargo zigbuild --release --locked --target x86_64-unknown-linux-gnu.2.31 -p rustfs --bins
+        shell: bash
+        run: |
+          # Force rebuild by touching build.rs so a restored cache cannot
+          # reuse a rustfs artifact compiled before these console assets existed.
+          touch rustfs/build.rs
+          cargo zigbuild --release --locked --target x86_64-unknown-linux-gnu.2.31 -p rustfs --bin rustfs
 
       - name: Verify glibc symbol baseline
         run: |
           python3 scripts/release/check_glibc.py 2.31 \
-            target/x86_64-unknown-linux-gnu/release/rustfs \
-            target/x86_64-unknown-linux-gnu/release/rustfs-cli
+            target/x86_64-unknown-linux-gnu/release/rustfs
           target/x86_64-unknown-linux-gnu/release/rustfs --version | grep -Fx 'build target : x86_64-unknown-linux-gnu'
 
+      - name: Verify console
+        shell: bash
+        run: |
+          set -euo pipefail
+
+          find_free_port() {
+            python3 - <<'PY'
+          import socket
+
+          with socket.socket() as sock:
+              sock.bind(("127.0.0.1", 0))
+              print(sock.getsockname()[1])
+          PY
+          }
+
+          binary="target/x86_64-unknown-linux-gnu/release/rustfs"
+          data_dir="$(mktemp -d)"
+          log_file="${RUNNER_TEMP}/rustfs-console-smoke.log"
+          server_pid=""
+          trap '
+            if [[ -n "${server_pid:-}" ]]; then
+              kill "$server_pid" 2>/dev/null || true
+              wait "$server_pid" 2>/dev/null || true
+            fi
+            rm -rf "$data_dir"
+          ' EXIT
+
+          if [[ ! -x "$binary" ]]; then
+            echo "rustfs binary not found: ${binary}" >&2
+            exit 1
+          fi
+
+          api_port="$(find_free_port)"
+          console_port="$(find_free_port)"
+          console_url="http://127.0.0.1:${console_port}/rustfs/console/"
+
+          "$binary" server \
+            --address "127.0.0.1:${api_port}" \
+            --console-enable \
+            --console-address "127.0.0.1:${console_port}" \
+            --access-key console-smoke \
+            --secret-key console-smoke-secret \
+            "$data_dir" >"$log_file" 2>&1 &
+          server_pid=$!
+
+          response=""
+          for _ in {1..40}; do
+            response="$(curl --silent --output /dev/null --write-out '%{http_code} %{content_type}' "$console_url" || true)"
+            if [[ "$response" == 200\ text/html* ]]; then
+              exit 0
+            fi
+            sleep 0.25
+          done
+
+          echo "Console endpoint did not return 200 text/html: $response" >&2
+          cat "$log_file"
+          exit 1
+
       - name: Build DEB package
         id: deb
         shell: bash
@@ -346,12 +534,12 @@ jobs:
           path: ${{ steps.publish.outputs.candidate_file }}
           if-no-files-found: error
 
-      # Publish the deb/rpm pair to the auto-testing repo's `assets` branch so
-      # engineers can download and install the nightly directly. The branch is
-      # a single-commit orphan rewritten on every build, which keeps the repo
-      # small while the latest files stay reachable at stable raw URLs.
       # Publish the deb/rpm pair as assets of the rolling `nightly` release on
       # rustfs/auto-testing (see scripts/release/publish_nightly_assets.sh).
+      # The dispatch below runs only after this upload and the immutable R2
+      # candidate both succeed. It does not replace rustfs-functional-chain.yml,
+      # which still starts from this workflow's workflow_run completion.
+      # kms-vault-lane and kms-vault-ha-failover stay parallel with this job.
       - name: Publish packages to auto-testing release assets
         env:
           ASSETS_TOKEN: ${{ secrets.PF_TESTING_GH_TOKEN }}
@@ -360,6 +548,90 @@ jobs:
           DEB_DATE: ${{ steps.deb.outputs.deb_date }}
           BUILD_REF: ${{ env.NIGHTLY_BUILD_REF }}
         run: bash scripts/release/publish_nightly_assets.sh
+
+      # Cross-repo repository_dispatch cannot use GITHUB_TOKEN. The secret is a
+      # PAT or GitHub App token for rustfs/auto-testing: classic `repo` scope,
+      # or fine-grained Contents write (the permission GitHub checks for
+      # POST /repos/{owner}/{repo}/dispatches). A dispatch failure fails this
+      # step, and therefore this job, so the scheduled failure alert fires.
+      # GitHub allows at most 10 top-level client_payload properties, so the
+      # schema 2 candidate object is nested under `candidate` and the release
+      # fields sit beside it.
+      - name: Dispatch nightly candidate to auto-testing
+        if: ${{ steps.publish.outputs.candidate_file != '' }}
+        env:
+          DISPATCH_TOKEN: ${{ secrets.AUTO_TESTING_DISPATCH_TOKEN }}
+          CANDIDATE_FILE: ${{ steps.publish.outputs.candidate_file }}
+          DEB_DATE: ${{ steps.deb.outputs.deb_date }}
+          SOURCE_REPO: ${{ github.repository }}
+        shell: bash
+        run: |
+          set -euo pipefail
+          : "${DISPATCH_TOKEN:?AUTO_TESTING_DISPATCH_TOKEN is required to dispatch rustfs/auto-testing}"
+          : "${CANDIDATE_FILE:?nightly candidate manifest is required}"
+          : "${DEB_DATE:?DEB_DATE is required}"
+          [[ "${DEB_DATE}" =~ ^[0-9]{4}-[0-9]{2}-[0-9]{2}$ ]]
+          [[ "${SOURCE_REPO}" == "rustfs/rustfs" ]]
+
+          payload="${RUNNER_TEMP}/nightly-candidate-dispatch-${GITHUB_RUN_ID}-${GITHUB_RUN_ATTEMPT}.json"
+          jq -n --slurpfile candidate "${CANDIDATE_FILE}" \
+            --arg event_type "nightly-candidate-ready" \
+            --arg source_repo "${SOURCE_REPO}" \
+            --arg latest_alias_url "https://dl.rustfs.com/artifacts/rustfs/packages/nightly/rustfs-nightly-latest.deb" \
+            --arg auto_testing_release "https://github.com/rustfs/auto-testing/releases/tag/nightly" \
+            --arg channel "nightly" \
+            --arg deb_date "${DEB_DATE}" \
+            --arg nightly_workflow ".github/workflows/nightly-gnu.yml" \
+            '{
+              event_type: $event_type,
+              source_repo: $source_repo,
+              latest_alias_url: $latest_alias_url,
+              auto_testing_release: $auto_testing_release,
+              channel: $channel,
+              deb_date: $deb_date,
+              nightly_workflow: $nightly_workflow,
+              candidate: $candidate[0]
+            }' > "${payload}"
+          if ! jq -e '
+            .event_type == "nightly-candidate-ready"
+            and .source_repo == "rustfs/rustfs"
+            and .channel == "nightly"
+            and .deb_date == env.DEB_DATE
+            and .nightly_workflow == ".github/workflows/nightly-gnu.yml"
+            and .latest_alias_url == "https://dl.rustfs.com/artifacts/rustfs/packages/nightly/rustfs-nightly-latest.deb"
+            and .auto_testing_release == "https://github.com/rustfs/auto-testing/releases/tag/nightly"
+            and .candidate.schema == 2
+            and ((.candidate | keys | sort) == ["build_run_attempt","build_run_id","package_sha256","package_url","schema","source_ref","source_sha","workflow_sha"])
+            and ((keys | sort) == ["auto_testing_release","candidate","channel","deb_date","event_type","latest_alias_url","nightly_workflow","source_repo"])
+          ' "${payload}" >/dev/null; then
+            echo "nightly dispatch payload does not match the nightly-candidate-ready contract" >&2
+            exit 1
+          fi
+
+          body="${RUNNER_TEMP}/nightly-candidate-dispatch-body-${GITHUB_RUN_ID}-${GITHUB_RUN_ATTEMPT}.json"
+          jq -n --slurpfile client_payload "${payload}" \
+            '{event_type: "nightly-candidate-ready", client_payload: $client_payload[0]}' > "${body}"
+          response="${RUNNER_TEMP}/nightly-candidate-dispatch-response-${GITHUB_RUN_ID}-${GITHUB_RUN_ATTEMPT}.txt"
+          status="$(curl -sS --retry 3 --connect-timeout 15 --max-time 60 \
+            -o "${response}" -w '%{http_code}' \
+            -X POST \
+            -H "Authorization: Bearer ${DISPATCH_TOKEN}" \
+            -H "Accept: application/vnd.github+json" \
+            -H "X-GitHub-Api-Version: 2022-11-28" \
+            -H "Content-Type: application/json" \
+            --data-binary @"${body}" \
+            https://api.github.com/repos/rustfs/auto-testing/dispatches)" || {
+            echo "repository_dispatch request to rustfs/auto-testing failed before an HTTP status" >&2
+            exit 1
+          }
+          if [[ "${status}" != "204" ]]; then
+            echo "repository_dispatch to rustfs/auto-testing failed: HTTP ${status}" >&2
+            if [[ -s "${response}" ]]; then
+              cat "${response}" >&2
+            fi
+            exit 1
+          fi
+          echo "dispatched nightly-candidate-ready to rustfs/auto-testing (deb_date=${DEB_DATE}, run=${GITHUB_RUN_ID}, attempt=${GITHUB_RUN_ATTEMPT})"
   # Live-Vault lane for the rustfs-kms suite (rustfs/backlog#1774).
   #
   # RUSTFS_KMS_VAULT_TOKEN is the single switch that adds the Vault KV2 and

.github/workflows/oidc-keycloak.yml

@@ -47,8 +47,8 @@ permissions:
   contents: read
 
 concurrency:
-  group: oidc-keycloak-live-${{ github.ref }}
-  cancel-in-progress: true
+  group: oidc-keycloak-live-${{ github.event_name }}-${{ github.ref }}
+  cancel-in-progress: ${{ github.event_name != 'schedule' }}
 
 jobs:
   oidc-keycloak-live:

.github/workflows/on-demand-migration-interop.yml

@@ -68,11 +68,14 @@ env:
 
 jobs:
   minio-source:
-    name: MinIO source (read-through, list-through, backfill)
+    name: MinIO source (read-through, list-through, backfill, ${{ matrix.stack }}${{ matrix.stack == 'gateway' && ', report-only until T2.10' || '' }})
     # Keep this scheduled interoperability lane scoped to the upstream repo.
     if: github.repository == 'rustfs/rustfs'
     runs-on: ubuntu-latest
     timeout-minutes: 90
+    # The gateway leg is report-only until T2.10.
+    continue-on-error: ${{ matrix.stack == 'gateway' }}
+    strategy: { fail-fast: false, matrix: { stack: [legacy, gateway] } }
     env:
       NO_PROXY: 127.0.0.1,localhost
       # Fixed credentials of the container this job starts and throws away;
@@ -137,6 +140,11 @@ jobs:
       - name: Build the RustFS binary under test
         run: python3 scripts/e2e_binary.py build --bins
 
+      # A gateway binary that refuses to start skips the suite; the summary says why.
+      - name: Gateway preflight
+        if: matrix.stack == 'gateway'
+        run: python3 scripts/dual_stack.py preflight --binary target/debug/rustfs --out "$RUNNER_TEMP/leg/stack.json"
+
       # The lane selects tests by module, so a rename would quietly shrink it.
       # The committed digest in .config/e2e-odm-interop-selection.txt fails
       # closed on that.
@@ -146,8 +154,19 @@ jobs:
           python3 ./scripts/check_test_wiring.py --check-profile e2e-odm-interop "${NEXTEST_LISTING}"
 
       - name: Run the interop cases against MinIO
+        env:
+          RUSTFS_S3_STACK: ${{ matrix.stack }}
+          RUSTFS_E2E_LOG_DIR: ${{ runner.temp }}/rustfs-e2e-logs
         run: python3 scripts/e2e_binary.py run -- cargo nextest run --profile e2e-odm-interop -p e2e_test --no-tests=fail
 
+      - name: Prove the stack and upload the leg
+        if: always()
+        uses: ./.github/actions/dual-stack-leg
+        with:
+          profile: e2e-odm-interop
+          stack: ${{ matrix.stack }}
+          logs: ${{ runner.temp }}/rustfs-e2e-logs
+
       - name: Build the MinIO interop report
         if: always()
         uses: ./.github/actions/odm-interop-report
@@ -171,7 +190,7 @@ jobs:
         if: always()
         uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
         with:
-          name: odm-interop-minio-${{ github.run_number }}-${{ github.run_attempt }}
+          name: odm-interop-minio-${{ github.run_number }}-${{ github.run_attempt }}-${{ matrix.stack }}
           path: |
             artifacts/odm-interop/minio
             target/nextest/e2e-odm-interop/junit.xml
@@ -297,6 +316,20 @@ jobs:
             target/nextest/e2e-odm-interop/junit.xml
           retention-days: 14
 
+  dual-stack-summary:
+    name: Both-stack diff summary
+    needs: [minio-source]
+    if: github.repository == 'rustfs/rustfs' && always()
+    runs-on: ubuntu-latest
+    timeout-minutes: 15
+    steps:
+      - uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
+        with:
+          persist-credentials: false
+      - uses: ./.github/actions/dual-stack-summary
+        with:
+          profiles: e2e-odm-interop
+
   alert-on-failure:
     name: Alert on scheduled failure
     needs: [minio-source, cloud-source]

.github/workflows/package.yml

@@ -1,644 +1,53 @@
-# Copyright 2024 RustFS Team
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-# Package Workflow - Build DEB/RPM packages
-#
-# This workflow builds DEB and RPM packages from pre-built Linux binaries
-# and uploads them to Cloudflare R2 and the GitHub release.
-#
-# Trigger:
-# - workflow_run: automatically package after "Build and Release" completes
-#   for a release tag (the mac/windows/linux binaries are already uploaded
-#   to the GitHub release before packaging starts)
-# - workflow_dispatch: manual fallback with a release tag and/or exact build run ID
-#
-# Flow:
-# 1. Resolve and validate the selected Build workflow run and source identity
-# 2. Download Linux binaries (x86_64-gnu, aarch64-gnu, x86_64-musl,
-#    aarch64-musl) from build artifacts
-# 3. Build DEB packages for amd64 and arm64, gnu and musl variants
-# 4. Build RPM packages for x86_64 and aarch64, gnu and musl variants
-# 5. Upload all packages to Cloudflare R2 and the GitHub release
-
-name: Package DEB/RPM
-
-permissions:
-  # contents: write is required to upload packages to the GitHub release
-  contents: write
-  actions: read
-
+# Copyright 2026 RustFS Team
+# SPDX-License-Identifier: Apache-2.0
+name: Build candidate packages
 on:
-  # Follows the same pattern as docker.yml: run after the release build
-  # workflow completes, so packaging is triggered only by release tags
-  # (e.g. 1.0.0-rc.2, 1.0.0-rc.3), never by development builds.
-  workflow_run:
-    workflows: [ "Build and Release" ]
-    types: [ completed ]
-  workflow_dispatch:
+  workflow_call:
     inputs:
-      tag:
-        description: "Release tag to package (e.g. 1.0.0-beta.12). Leave empty for latest main build."
-        required: false
+      candidate_manifest:
         type: string
-      build_run_id:
-        description: "Build workflow run ID (when combined with tag, both must identify the same release commit)"
-        required: false
-        type: string
-
-concurrency:
-  group: ${{ github.workflow }}-${{ github.event.workflow_run.head_branch || github.event.inputs.tag || github.run_id }}
-  cancel-in-progress: true
-
-env:
-  HEAD_BRANCH: ${{ github.event.workflow_run.head_branch }}
-  WORKFLOW_RUN_ID: ${{ github.event.workflow_run.id }}
-
+        required: true
+permissions:
+  contents: read
+  actions: read
 jobs:
-  # Resolve which build run to use and extract version info
-  resolve:
-    name: Resolve Build
-    # Auto-trigger only from successful tag builds of "Build and Release".
-    # Tag pushes arrive as event == push with head_branch != main (a
-    # non-main push head_branch is the release tag name). Manual dispatch
-    # stays available as a fallback for backfills and re-runs.
-    if: >-
-      github.event_name == 'workflow_dispatch' ||
-      (github.event.workflow_run.conclusion == 'success' &&
-       github.event.workflow_run.event == 'push' &&
-       github.event.workflow_run.head_branch != 'main')
-    runs-on: ubuntu-latest # gh CLI required; the sm-standard-2 fleet does not ship it
-    timeout-minutes: 10
-    outputs:
-      version: ${{ steps.resolve.outputs.version }}
-      build_type: ${{ steps.resolve.outputs.build_type }}
-      build_run_id: ${{ steps.resolve.outputs.build_run_id }}
-      build_run_number: ${{ steps.resolve.outputs.build_run_number }}
-      head_sha: ${{ steps.resolve.outputs.head_sha }}
-      dev_sequence: ${{ steps.resolve.outputs.dev_sequence }}
-      tag: ${{ steps.resolve.outputs.tag }}
-    steps:
-      - name: Resolve build run
-        id: resolve
-        shell: bash
-        env:
-          GH_TOKEN: ${{ github.token }}
-          EVENT_NAME: ${{ github.event_name }}
-          REPOSITORY: ${{ github.repository }}
-          INPUT_TAG: ${{ github.event.inputs.tag }}
-          INPUT_RUN_ID: ${{ github.event.inputs.build_run_id }}
-        run: |
-          set -euo pipefail
-
-          fail() {
-            echo "❌ $1" >&2
-            exit 1
-          }
-
-          TAG=""
-          BUILD_RUN_ID=""
-          case "$EVENT_NAME" in
-            workflow_run)
-              TAG="$HEAD_BRANCH"
-              BUILD_RUN_ID="$WORKFLOW_RUN_ID"
-              ;;
-            workflow_dispatch)
-              TAG="$INPUT_TAG"
-              BUILD_RUN_ID="$INPUT_RUN_ID"
-              ;;
-            *) fail "unsupported event: $EVENT_NAME" ;;
-          esac
-
-          # Validate and classify tags before using them in API paths or logs.
-          semver_core='(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)'
-          prerelease_id='(alpha|beta|rc)\.(0|[1-9][0-9]*)'
-          if [[ -n "$TAG" ]]; then
-            if [[ "$TAG" =~ ^${semver_core}(-${prerelease_id})?-preview\.(0|[1-9][0-9]*)$ ]]; then
-              BUILD_TYPE=preview
-            elif [[ "$TAG" =~ ^${semver_core}-${prerelease_id}$ ]]; then
-              BUILD_TYPE=prerelease
-            elif [[ "$TAG" =~ ^${semver_core}$ ]]; then
-              BUILD_TYPE=release
-            else
-              fail "tag is not a supported strict package version"
-            fi
-          else
-            BUILD_TYPE=development
-          fi
-
-          if [[ -n "$BUILD_RUN_ID" ]]; then
-            [[ "$BUILD_RUN_ID" =~ ^[1-9][0-9]*$ ]] || fail "build run ID must be a positive decimal integer"
-            echo "Using selected build run: $BUILD_RUN_ID"
-          elif [[ -n "$TAG" ]]; then
-            echo "Looking for build run for tag: $TAG"
-            BUILD_RUN_ID=$(gh api --method GET \
-              "repos/${REPOSITORY}/actions/workflows/build.yml/runs" \
-              -f branch="$TAG" -f status=success -F per_page=1 \
-              --jq '.workflow_runs[0].id // empty' 2>/dev/null || true)
-
-            if [[ -z "$BUILD_RUN_ID" ]]; then
-              BUILD_RUN_ID=$(gh api --method GET \
-                "repos/${REPOSITORY}/actions/workflows/build.yml/runs" \
-                -f event=push -f status=success -F per_page=100 2>/dev/null |
-                jq -r --arg tag "$TAG" \
-                  '[.workflow_runs[] | select(.head_branch == $tag)][0].id // empty' || true)
-            fi
-            [[ "$BUILD_RUN_ID" =~ ^[1-9][0-9]*$ ]] || fail "no successful build run found for tag"
-            echo "Found build run: $BUILD_RUN_ID"
-          else
-            echo "No tag specified, looking for latest main build"
-            BUILD_RUN_ID=$(gh api --method GET \
-              "repos/${REPOSITORY}/actions/workflows/build.yml/runs" \
-              -f branch=main -f status=success -F per_page=1 \
-              --jq '.workflow_runs[0].id // empty' 2>/dev/null || true)
-            [[ "$BUILD_RUN_ID" =~ ^[1-9][0-9]*$ ]] || fail "no successful main build found"
-            echo "Latest main build: $BUILD_RUN_ID"
-          fi
-
-          # Fetch once and use the same immutable run metadata for identity,
-          # ordering, workflow provenance, and release-channel validation.
-          RUN_JSON=$(gh api "repos/${REPOSITORY}/actions/runs/${BUILD_RUN_ID}") ||
-            fail "cannot read selected build run"
-          RUN_ID=$(jq -r '.id // empty' <<<"$RUN_JSON")
-          RUN_NUMBER=$(jq -r '.run_number // empty' <<<"$RUN_JSON")
-          RUN_STATUS=$(jq -r '.status // empty' <<<"$RUN_JSON")
-          RUN_CONCLUSION=$(jq -r '.conclusion // empty' <<<"$RUN_JSON")
-          RUN_PATH=$(jq -r '.path // empty' <<<"$RUN_JSON")
-          HEAD_SHA=$(jq -r '.head_sha // empty' <<<"$RUN_JSON")
-          RUN_HEAD_BRANCH=$(jq -r '.head_branch // empty' <<<"$RUN_JSON")
-
-          [[ "$RUN_ID" == "$BUILD_RUN_ID" ]] || fail "run metadata ID mismatch"
-          [[ "$RUN_NUMBER" =~ ^[1-9][0-9]*$ ]] || fail "build run number must be a positive decimal integer"
-          [[ "$RUN_STATUS" == completed && "$RUN_CONCLUSION" == success ]] || fail "selected build run is not successful"
-          [[ "$RUN_PATH" == .github/workflows/build.yml ]] || fail "selected run is not Build and Release"
-          [[ "$HEAD_SHA" =~ ^[0-9a-f]{40}$ ]] || fail "selected build run has an invalid head SHA"
-          [[ "$RUN_HEAD_BRANCH" != *$'\n'* && -n "$RUN_HEAD_BRANCH" ]] || fail "selected build run has an invalid head branch"
-
-          if [[ -n "$TAG" ]]; then
-            [[ "$RUN_HEAD_BRANCH" == "$TAG" ]] || fail "tag and build run head branch do not match"
-
-            TAG_REF_JSON=$(gh api "repos/${REPOSITORY}/git/ref/tags/${TAG}") ||
-              fail "cannot resolve release tag ref"
-            TAG_OBJECT_TYPE=$(jq -r '.object.type // empty' <<<"$TAG_REF_JSON")
-            TAG_OBJECT_SHA=$(jq -r '.object.sha // empty' <<<"$TAG_REF_JSON")
-            depth=0
-            while [[ "$TAG_OBJECT_TYPE" == tag && $depth -lt 5 ]]; do
-              TAG_OBJECT_JSON=$(gh api "repos/${REPOSITORY}/git/tags/${TAG_OBJECT_SHA}") ||
-                fail "cannot peel annotated release tag"
-              TAG_OBJECT_TYPE=$(jq -r '.object.type // empty' <<<"$TAG_OBJECT_JSON")
-              TAG_OBJECT_SHA=$(jq -r '.object.sha // empty' <<<"$TAG_OBJECT_JSON")
-              depth=$((depth + 1))
-            done
-            [[ "$TAG_OBJECT_TYPE" == commit && "$TAG_OBJECT_SHA" =~ ^[0-9a-f]{40}$ ]] ||
-              fail "release tag does not resolve to a commit"
-            [[ "$TAG_OBJECT_SHA" == "$HEAD_SHA" ]] || fail "release tag commit and build run head SHA do not match"
-            VERSION="$TAG"
-            DEV_SEQUENCE=""
-          else
-            VERSION="dev-${HEAD_SHA}"
-            DEV_SEQUENCE="$RUN_NUMBER"
-          fi
-
-          {
-            echo "version=$VERSION"
-            echo "build_type=$BUILD_TYPE"
-            echo "build_run_id=$BUILD_RUN_ID"
-            echo "build_run_number=$RUN_NUMBER"
-            echo "head_sha=$HEAD_SHA"
-            echo "dev_sequence=$DEV_SEQUENCE"
-            echo "tag=${TAG}"
-          } >> "$GITHUB_OUTPUT"
-
-          echo "📊 Resolved:"
-          echo "  Version: $VERSION"
-          echo "  Build type: $BUILD_TYPE"
-          echo "  Build run ID: $BUILD_RUN_ID"
-          echo "  Build run number: $RUN_NUMBER"
-
-  # Build DEB and RPM packages for each architecture and libc variant
   package:
-    name: Package (${{ matrix.arch }}-${{ matrix.libc }})
-    needs: resolve
-    runs-on: ubuntu-latest # gh CLI required; the sm-standard-2 fleet does not ship it
-    timeout-minutes: 30
+    runs-on: ubuntu-latest
+    timeout-minutes: 45
     strategy:
       fail-fast: false
-      max-parallel: 1
       matrix:
-        include:
-          - arch: x86_64
-            libc: gnu
-            deb_arch: amd64
-            rpm_arch: x86_64
-            artifact_name: "rustfs-linux-x86_64-gnu"
-            # gnu binaries dynamically link glibc; musl binaries are static
-            # and run without a libc dependency.
-            deb_depends: "libc6 (>= 2.31)"
-            rpm_depends: "glibc >= 2.31"
-          - arch: aarch64
-            libc: gnu
-            deb_arch: arm64
-            rpm_arch: aarch64
-            artifact_name: "rustfs-linux-aarch64-gnu"
-            deb_depends: "libc6 (>= 2.31)"
-            rpm_depends: "glibc >= 2.31"
-          - arch: x86_64
-            libc: musl
-            deb_arch: amd64
-            rpm_arch: x86_64
-            artifact_name: "rustfs-linux-x86_64-musl"
-            deb_depends: ""
-            rpm_depends: ""
-          - arch: aarch64
-            libc: musl
-            deb_arch: arm64
-            rpm_arch: aarch64
-            artifact_name: "rustfs-linux-aarch64-musl"
-            deb_depends: ""
-            rpm_depends: ""
+        target: [linux-x86_64-musl, linux-aarch64-musl, linux-x86_64-gnu, linux-aarch64-gnu]
     steps:
-      - name: Checkout repository
-        uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
+      - uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
         with:
+          ref: ${{ fromJson(inputs.candidate_manifest).source_sha }}
           persist-credentials: false
-
-      - name: Normalize package metadata
-        id: versions
-        shell: bash
-        env:
-          BUILD_TYPE: ${{ needs.resolve.outputs.build_type }}
-          SOURCE_VERSION: ${{ needs.resolve.outputs.version }}
-          DEV_SEQUENCE: ${{ needs.resolve.outputs.dev_sequence }}
-          ARCH: ${{ matrix.rpm_arch }}
-          LIBC: ${{ matrix.libc }}
-        run: |
-          set -euo pipefail
-
-          normalized=$(./scripts/release/package_versions.sh \
-            "$BUILD_TYPE" "$SOURCE_VERSION" "$DEV_SEQUENCE" "$ARCH" "$LIBC")
-          printf '%s\n' "$normalized" >> "$GITHUB_OUTPUT"
-
-      - name: Download binary artifact from build run
-        uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
-        with:
-          pattern: ${{ matrix.artifact_name }}*
-          path: ./binary-artifact
-          run-id: ${{ needs.resolve.outputs.build_run_id }}
-          github-token: ${{ github.token }}
-          merge-multiple: true
-
-      - name: Extract binary
-        id: binary
-        shell: bash
-        run: |
-          set -euo pipefail
-
-          ZIP_FILE=$(find ./binary-artifact -name "*.zip" -type f | head -1)
-          if [[ -z "$ZIP_FILE" ]]; then
-            echo "❌ No binary artifact found"
-            find ./binary-artifact -mindepth 1 -maxdepth 1 -print 2>/dev/null || true
-            exit 1
-          fi
-
-          echo "Found artifact: $ZIP_FILE"
-
-          mkdir -p ./bin
-          unzip -o "$ZIP_FILE" -d ./bin
-
-          if [[ ! -f ./bin/rustfs ]]; then
-            echo "❌ rustfs binary not found in archive"
-            exit 1
-          fi
-
-          chmod +x ./bin/rustfs
-          stat --printf='%n %s bytes\n' ./bin/rustfs
-          echo "✅ Binary extracted"
-
-      - name: Build DEB package
-        id: deb
-        shell: bash
-        env:
-          DEB_VERSION: ${{ steps.versions.outputs.deb_version }}
-          DEB_ARCH: ${{ matrix.deb_arch }}
-          DEB_FILE: ${{ steps.versions.outputs.deb_file }}
-          DEB_DEPENDS: ${{ matrix.deb_depends }}
-          LIBC: ${{ matrix.libc }}
-        run: |
-          set -euo pipefail
-
-          PKG_DIR="${DEB_FILE%.deb}"
-
-          echo "Building DEB: ${DEB_FILE}"
-
-          mkdir -p "${PKG_DIR}/DEBIAN"
-          mkdir -p "${PKG_DIR}/usr/bin"
-          mkdir -p "${PKG_DIR}/etc/default"
-          mkdir -p "${PKG_DIR}/lib/systemd/system"
-          mkdir -p "${PKG_DIR}/usr/share/doc/rustfs"
-
-          cp ./bin/rustfs "${PKG_DIR}/usr/bin/"
-          chmod 755 "${PKG_DIR}/usr/bin/rustfs"
-
-          cp deploy/build/rustfs.service "${PKG_DIR}/lib/systemd/system/"
-
-          cat > "${PKG_DIR}/etc/default/rustfs" << 'ENVEOF'
-          # RustFS Environment Configuration
-          # See https://docs.rustfs.com for more information
-          # RUSTFS_ACCESS_KEY=<changeme>
-          # RUSTFS_SECRET_KEY=<changeme>
-          # RUSTFS_VOLUMES=""
-          # RUSTFS_ADDRESS=":9000"
-          # RUSTFS_CONSOLE_ADDRESS=":9001"
-          # RUSTFS_CONSOLE_ENABLE=true
-          # RUSTFS_OBS_LOGGER_LEVEL=error
-          # RUSTFS_OBS_LOG_DIRECTORY="/var/log/rustfs/"
-          ENVEOF
-
-          cat > "${PKG_DIR}/DEBIAN/control" << EOF
-          Package: rustfs
-          Version: ${DEB_VERSION}
-          Section: utils
-          Priority: optional
-          Architecture: ${DEB_ARCH}
-          EOF
-
-          # gnu binaries dynamically link glibc and must declare it; musl
-          # binaries are statically linked and need no libc dependency.
-          if [[ -n "$DEB_DEPENDS" ]]; then
-            echo "Depends: ${DEB_DEPENDS}" >> "${PKG_DIR}/DEBIAN/control"
-          fi
-
-          cat >> "${PKG_DIR}/DEBIAN/control" << EOF
-          Maintainer: RustFS Team <support@rustfs.com>
-          Description: High-performance distributed object storage (${LIBC} build)
-           RustFS is a high-performance distributed object storage software
-           built using Rust. It is compatible with MinIO and S3 API.
-          Homepage: https://rustfs.com
-          EOF
-
-          # Declare /etc/default/rustfs as a conffile so dpkg preserves user
-          # modifications on upgrade instead of silently overwriting them.
-          cat > "${PKG_DIR}/DEBIAN/conffiles" << 'CONFFILES'
-          /etc/default/rustfs
-          CONFFILES
-
-          ./scripts/release/package_service_scripts.sh after-install > "${PKG_DIR}/DEBIAN/postinst"
-          chmod 755 "${PKG_DIR}/DEBIAN/postinst"
-
-          ./scripts/release/package_service_scripts.sh before-remove > "${PKG_DIR}/DEBIAN/prerm"
-          chmod 755 "${PKG_DIR}/DEBIAN/prerm"
-
-          ./scripts/release/package_service_scripts.sh after-remove > "${PKG_DIR}/DEBIAN/postrm"
-          chmod 755 "${PKG_DIR}/DEBIAN/postrm"
-
-          cp LICENSE "${PKG_DIR}/usr/share/doc/rustfs/"
-          cp README.md "${PKG_DIR}/usr/share/doc/rustfs/"
-
-          fakeroot dpkg-deb --build "${PKG_DIR}" "$DEB_FILE"
-
-          [[ $(dpkg-deb -f "$DEB_FILE" Package) == rustfs ]]
-          [[ $(dpkg-deb -f "$DEB_FILE" Version) == "$DEB_VERSION" ]]
-          [[ $(dpkg-deb -f "$DEB_FILE" Architecture) == "$DEB_ARCH" ]]
-          dpkg-deb --fsys-tarfile "$DEB_FILE" | tar -tf - | grep -Fx './usr/bin/rustfs' >/dev/null
-          stat --printf='%n %s bytes\n' "$DEB_FILE"
-          echo "deb_file=$DEB_FILE" >> "$GITHUB_OUTPUT"
-          echo "✅ DEB built: $DEB_FILE"
-
-      - name: Build RPM package
-        id: rpm
-        shell: bash
-        env:
-          RPM_VERSION: ${{ steps.versions.outputs.rpm_version }}
-          RPM_RELEASE: ${{ steps.versions.outputs.rpm_release }}
-          RPM_ARCH: ${{ matrix.rpm_arch }}
-          RPM_FILE: ${{ steps.versions.outputs.rpm_file }}
-          RPM_DEPENDS: ${{ matrix.rpm_depends }}
-          LIBC: ${{ matrix.libc }}
-        run: |
-          set -euo pipefail
-
-          echo "Building RPM for ${RPM_ARCH} (${LIBC})"
-
-          sudo apt-get update && sudo apt-get install -y ruby ruby-dev build-essential rpm
-          sudo gem install fpm
-          ./scripts/test_package_versions.sh --require-package-managers
-
-          # gnu binaries dynamically link glibc and must declare it; musl
-          # binaries are statically linked and need no libc dependency.
-          depends_args=()
-          if [[ -n "$RPM_DEPENDS" ]]; then
-            depends_args+=(--depends "$RPM_DEPENDS")
-          fi
-
-          # Create config file for fpm (DEB build creates it in its package dir structure,
-          # but fpm needs the file to exist before packaging)
-          mkdir -p ./tmp-pkg/etc/default
-          cat > ./tmp-pkg/etc/default/rustfs << 'ENVEOF'
-          # RustFS Environment Configuration
-          # See https://docs.rustfs.com for more information
-          # RUSTFS_ACCESS_KEY=<changeme>
-          # RUSTFS_SECRET_KEY=<changeme>
-          # RUSTFS_VOLUMES=""
-          # RUSTFS_ADDRESS=":9000"
-          # RUSTFS_CONSOLE_ADDRESS=":9001"
-          # RUSTFS_CONSOLE_ENABLE=true
-          # RUSTFS_OBS_LOGGER_LEVEL=error
-          # RUSTFS_OBS_LOG_DIRECTORY="/var/log/rustfs/"
-          ENVEOF
-
-          fpm -s dir -t rpm \
-            --name rustfs \
-            --version "$RPM_VERSION" \
-            --iteration "$RPM_RELEASE" \
-            --architecture "$RPM_ARCH" \
-            --package "$RPM_FILE" \
-            "${depends_args[@]}" \
-            --description "High-performance distributed object storage (${LIBC} build)" \
-            --maintainer "RustFS Team <support@rustfs.com>" \
-            --url "https://rustfs.com" \
-            --license "Apache-2.0" \
-            --before-install <(./scripts/release/package_service_scripts.sh rpm-before-install) \
-            --before-upgrade <(./scripts/release/package_service_scripts.sh rpm-before-upgrade) \
-            --after-install <(./scripts/release/package_service_scripts.sh after-install) \
-            --after-upgrade <(./scripts/release/package_service_scripts.sh after-install) \
-            --before-remove <(./scripts/release/package_service_scripts.sh before-remove) \
-            --after-remove <(./scripts/release/package_service_scripts.sh after-remove) \
-            --rpm-posttrans <(./scripts/release/package_service_scripts.sh rpm-posttrans) \
-            --config-files /etc/default/rustfs \
-            ./bin/rustfs=/usr/bin/rustfs \
-            ./tmp-pkg/etc/default/rustfs=/etc/default/rustfs \
-            deploy/build/rustfs.service=/lib/systemd/system/rustfs.service \
-            LICENSE=/usr/share/doc/rustfs/LICENSE \
-            README.md=/usr/share/doc/rustfs/README.md
-
-          if [[ ! -f "$RPM_FILE" ]]; then
-            echo "❌ RPM build failed"
-            exit 1
-          fi
-
-          RPM_METADATA=$(rpm -qp --qf '%{NAME}\n%{VERSION}\n%{RELEASE}\n%{ARCH}\n' "$RPM_FILE")
-          EXPECTED_METADATA=$(printf 'rustfs\n%s\n%s\n%s' "$RPM_VERSION" "$RPM_RELEASE" "$RPM_ARCH")
-          [[ "$RPM_METADATA" == "$EXPECTED_METADATA" ]]
-          rpm -qpl "$RPM_FILE" | grep -Fx '/usr/bin/rustfs' >/dev/null
-          stat --printf='%n %s bytes\n' "$RPM_FILE"
-          echo "rpm_file=$RPM_FILE" >> "$GITHUB_OUTPUT"
-          echo "✅ RPM built: $RPM_FILE"
-
-      - name: Upload packages to artifacts
-        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
-        with:
-          name: packages-${{ matrix.arch }}-${{ matrix.libc }}
-          path: |
-            *.deb
-            *.rpm
-          retention-days: 30
-
-      - name: Upload packages to Cloudflare R2
-        if: needs.resolve.outputs.build_type != 'preview' && env.R2_ACCESS_KEY_ID != ''
-        env:
-          R2_ACCESS_KEY_ID: ${{ secrets.R2_ACCESS_KEY_ID }}
-          R2_SECRET_ACCESS_KEY: ${{ secrets.R2_SECRET_ACCESS_KEY }}
-          R2_ENDPOINT: ${{ secrets.R2_ENDPOINT }}
-          R2_BUCKET: ${{ secrets.R2_BUCKET }}
-          AWS_EC2_METADATA_DISABLED: true
-          BUILD_TYPE: ${{ needs.resolve.outputs.build_type }}
-          DEB_FILE: ${{ steps.deb.outputs.deb_file }}
-          RPM_FILE: ${{ steps.rpm.outputs.rpm_file }}
-        shell: bash
-        run: |
-          set -euo pipefail
-
-          if [[ -z "$R2_ACCESS_KEY_ID" || -z "$R2_SECRET_ACCESS_KEY" || -z "$R2_ENDPOINT" || -z "$R2_BUCKET" ]]; then
-            echo "⚠️ R2 credentials missing, skipping upload"
-            exit 0
-          fi
-
-          if ! command -v aws >/dev/null 2>&1; then
-            sudo apt-get update && sudo apt-get install -y awscli
-          fi
-
-          export AWS_ACCESS_KEY_ID="$R2_ACCESS_KEY_ID"
-          export AWS_SECRET_ACCESS_KEY="$R2_SECRET_ACCESS_KEY"
-          export AWS_DEFAULT_REGION="auto"
-
-          if [[ "$BUILD_TYPE" == "development" ]]; then
-            R2_PREFIX="artifacts/rustfs/packages/dev"
-          else
-            R2_PREFIX="artifacts/rustfs/packages/release"
-          fi
-          R2_PATH="s3://${R2_BUCKET}/${R2_PREFIX}/"
-
-          echo "📤 Uploading to $R2_PATH"
-
-          for f in "$DEB_FILE" "$RPM_FILE"; do
-            if [[ -n "$f" && -f "$f" ]]; then
-              echo "Uploading: $f"
-              aws s3 cp "$f" "$R2_PATH" --endpoint-url "$R2_ENDPOINT" --only-show-errors
-            fi
-          done
-
-          echo "✅ Upload complete"
-
-          # Also upload as latest for release/prerelease
-          if [[ "$BUILD_TYPE" == "release" || "$BUILD_TYPE" == "prerelease" ]]; then
-            LATEST_PATH="s3://${R2_BUCKET}/artifacts/rustfs/packages/latest/"
-            for f in "$DEB_FILE" "$RPM_FILE"; do
-              if [[ -n "$f" && -f "$f" ]]; then
-                echo "Uploading latest: $(basename "$f")"
-                aws s3 cp "$f" "$LATEST_PATH" --endpoint-url "$R2_ENDPOINT" --only-show-errors
-              fi
-            done
-            echo "✅ Latest packages updated"
-          fi
-
-      - name: Upload packages to GitHub Release
-        if: needs.resolve.outputs.tag != ''
+      - name: Reuse completed package set
+        id: reuse
         env:
           GH_TOKEN: ${{ github.token }}
-          TAG: ${{ needs.resolve.outputs.tag }}
-          DEB_FILE: ${{ steps.deb.outputs.deb_file }}
-          RPM_FILE: ${{ steps.rpm.outputs.rpm_file }}
-        shell: bash
+          CANDIDATE_MANIFEST: ${{ inputs.candidate_manifest }}
+          TARGET: ${{ matrix.target }}
+        run: python3 scripts/release/deliver.py reuse --kind packages --target "$TARGET"
+      - name: Install packaging tools
+        if: steps.reuse.outputs.reused != 'true'
         run: |
-          set -euo pipefail
-
-          # Upload the packages, then refresh the release checksums so the new
-          # assets are covered, matching the binary release flow.
-          for f in "$DEB_FILE" "$RPM_FILE"; do
-            if [[ -n "$f" && -f "$f" ]]; then
-              echo "📤 Uploading $(basename "$f") to GitHub release ${TAG}..."
-              gh release upload "$TAG" "$f" --clobber
-            fi
-          done
-
-          CHECKSUM_DIR="$(mktemp -d)"
-          gh release download "$TAG" -p 'SHA256SUMS' -p 'SHA512SUMS' \
-            -D "$CHECKSUM_DIR" --clobber
-
-          for spec in "SHA256SUMS:sha256sum" "SHA512SUMS:sha512sum"; do
-            asset="${spec%%:*}"
-            checksum_cmd="${spec##*:}"
-            checksum_file="${CHECKSUM_DIR}/${asset}"
-
-            for f in "$DEB_FILE" "$RPM_FILE"; do
-              if [[ -n "$f" && -f "$f" ]]; then
-                base="$(basename "$f")"
-                # GitHub stores release asset names with '~' normalized to '.'.
-                # Current package file names embed the raw tag and contain no
-                # '~', but older releases may still ship such assets, so the
-                # checksum entries must reference the name as stored on the
-                # release.
-                github_base="${base//\~/.}"
-                # Remove any stale entry (both naming variants), then append
-                grep -Fv -- "$base" "$checksum_file" > "${checksum_file}.tmp" || true
-                grep -Fv -- "$github_base" "${checksum_file}.tmp" > "${checksum_file}.tmp2" || true
-                mv "${checksum_file}.tmp2" "$checksum_file"
-                digest=$("$checksum_cmd" -- "$f" | awk '{print $1}')
-                printf '%s  %s\n' "$digest" "$github_base" >> "$checksum_file"
-              fi
-            done
-
-            echo "📤 Updating ${asset} for release ${TAG}..."
-            gh release upload "$TAG" "$checksum_file" --clobber
-          done
-
-          echo "✅ GitHub release assets updated"
-
-  # Summary
-  summary:
-    name: Summary
-    needs: [ resolve, package ]
-    # Every completed Build and Release run triggers this workflow; when
-    # resolve declined it (not a release tag), there is nothing to summarise.
-    if: always() && needs.resolve.result != 'skipped'
-    runs-on: ubuntu-latest
-    timeout-minutes: 5
-    steps:
-      - name: Print summary
-        shell: bash
+          sudo apt-get update
+          sudo apt-get install -y ruby ruby-dev build-essential rpm fakeroot cpio
+          sudo gem install fpm --version 1.17.0 --no-document
+      - name: Package accepted executable and verify contents
+        if: steps.reuse.outputs.reused != 'true'
         env:
-          SUMMARY_VERSION: ${{ needs.resolve.outputs.version }}
-          SUMMARY_BUILD_TYPE: ${{ needs.resolve.outputs.build_type }}
-          SUMMARY_BUILD_RUN_ID: ${{ needs.resolve.outputs.build_run_id }}
-          SUMMARY_PACKAGE_STATUS: ${{ needs.package.result }}
-        run: |
-          {
-            echo "## 📦 Package Summary"
-            echo ""
-            echo "| Item | Value |"
-            echo "|------|-------|"
-            echo "| Version | \`${SUMMARY_VERSION}\` |"
-            echo "| Build Type | ${SUMMARY_BUILD_TYPE} |"
-            echo "| Build Run | #${SUMMARY_BUILD_RUN_ID} |"
-            echo "| Package Status | ${SUMMARY_PACKAGE_STATUS} |"
-          } >> "$GITHUB_STEP_SUMMARY"
+          GH_TOKEN: ${{ github.token }}
+          CANDIDATE_MANIFEST: ${{ inputs.candidate_manifest }}
+          TARGET: ${{ matrix.target }}
+        run: python3 scripts/release/deliver.py package --target "$TARGET" --output delivery
+      - uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+        if: steps.reuse.outputs.reused != 'true'
+        with:
+          name: packages-${{ matrix.target }}
+          path: delivery/*
+          if-no-files-found: error
+          compression-level: 0
+          retention-days: 30