* feat(server): add the RUSTFS_S3_STACK selector and dual-stack e2e lane (#8411)
* refactor(server): group the s3s-compat tower layers under legacy_compat (#8410)
Move the ten tower layers that patch s3s behaviour out of
rustfs/src/server/layer.rs into rustfs/src/server/legacy_compat/, one file
per layer with its tests. Each file header names the s3s behaviour it
patches, the gateway mechanism that replaces it (Replaced by:), and the task
that removes it (Removed by: T2.12, rustfs/backlog#2771). The layers keep
their code, their order in external_service_stack!, and their behaviour; no
layer is deleted.
Add server::http::tests::stack_census, which renders the ordered service
chain of the real external_service_stack! expansion and compares it with a
committed constant, so a reorder, insertion or removal is a visible diff.
Route the two s3s error types the moved layers use through a
storage_api::server::legacy_compat shim so the split adds no s3s-importing
file, and lower the s3s footprint baselines to the measured values (files 209
to 208, s3_error! lines 1586 to 1585). Keep the server runtime-source bypass
guard covering the moved code.
Refs rustfs/backlog#2787, rustfs/backlog#2734
* feat(s3-types): add the RustFS-owned S3 error type and s3_error! macro (#8406)
`rustfs-s3-types` now provides `S3ErrorCode` (generated by build.rs from
src/codes.txt, the grep-derived call surface, plus `Custom`), `S3Error`,
`S3Result`, `StdError` and the `s3_error` macro with the same call syntax
as s3s, so the T1.2 codemod is a `use` swap. Statuses are hand-written and
pinned to the s3s behaviour RustFS ships today by an oracle test that uses
s3s as a dev-dependency only. The `From` conversions for the legacy edge
live behind the off-by-default `compat-s3s` feature (the orphan rule
forbids them in rustfs/src/error.rs); `rustfs` enables it.
Refs rustfs/backlog#2742, rustfs/backlog#2734
* ci: add the per-crate test-count ratchet (#8404)
Quick Checks now fails any PR that lowers the number of test attributes
in a workspace crate or adds #[ignore], measured per crate against the
exact rows of .config/test-count-baseline.txt (rustfs/backlog#2734
section 1 rule 3). The counter widens the spec's literal
#[(tokio::)?test] to the tokio attributes that carry arguments and both
spellings of #[test_case(...)], and anchors at line start so
commented-out attributes do not count. The initial baseline covers 52
crates, 21077 tests and 164 ignored.
Refs rustfs/backlog#2738
Refs rustfs/backlog#2734
* test(ecstore): pin the persisted bucket-config XML bytes for 13 families (#8408)
Before any DTO or codec is replaced, pin the exact bytes that
deserialize followed by serialize produces for every persisted
bucket-configuration XML family in its three source shapes (AWS
documentation, MinIO-written, RustFS-written), and the bytes RustFS
writes today for a typed value. Byte equality only: a structural
comparison would let a codec replacement change the persisted encoding
unnoticed.
The generator script refuses to overwrite existing goldens without
--force, refuses on dirty codec paths, cannot report success from its
write mode, and records the generating commit in the fixtures README.
Refs rustfs/backlog#2744, rustfs/backlog#2734
* ci: raise the test-count baseline to the integration tree (#8426)
* ci: make the s3s footprint baseline exact and lower-only (#8403)
* ci: make the s3s footprint baseline exact and lower-only
The s3s footprint guard compared three counters against ceilings held in
the script itself, so a ceiling could sit above the tree and a drift was
fixable by raising it (rustfs/rustfs#5964 raised the s3_error! ceiling
to go green). The three values now live in
.config/s3s-footprint-baseline.txt as files, s3_error_lines and
ecstore_files, the script reads only that file, and a measurement that
differs from its baseline in either direction fails: growth prints
"s3s footprint grew: <measured> > <baseline>", shrinkage prints
"lower the baseline to <measured> in this PR". The counting method is
unchanged; the baseline is the current measurement of
gateway/integration (209 / 1585 / 36). The header records the end state:
once all three reach 0, T4.2 switches the script to failing on any
occurrence.
Refs rustfs/backlog#2739, rustfs/backlog#2734
* ci: add the s3s edge allowlist mode to the footprint guard (#8405)
`scripts/check_s3s_footprint.sh --mode allowlist` fails on every Rust
file that references s3s (`s3s::` or `s3s_sigv4`) and is not matched by
an entry of the new `.config/s3s-edge-allowlist.txt`; `--dry-run` lists
those files and exits 0. The allowlist carries the initial edge from the
spec, and the script rejects an entry that names or matches anything
under crates/ other than crates/e2e_test. The baseline mode and the
argument-less CI invocation are unchanged.
Refs rustfs/backlog#2741, rustfs/backlog#2734
* test(ecstore): reach the bucket-config codec through the test facade (#8428)
* refactor(s3-types): own the HTTP Body and drop s3s from three crates (#8431)
Add rustfs_s3_types::Body, a BoxBody<Bytes, StdError> wrapper with
empty() and From<Bytes | Vec<u8> | String | &'static str>, and the
legacy From conversions to and from s3s::Body behind compat-s3s.
The signer is now generic over the request body: it only reads and
writes the request head, so it needs no body type and its callers keep
theirs. s3-client sends its requests with the owned Body, madmin signs a
bodiless head, and targets drops its two S3Request/S3Response header
wrappers in favour of extract_params_header.
rustfs-signer, rustfs-madmin and rustfs-targets no longer depend on
s3s; the s3s footprint baseline drops from 209 to 202 files.
Refs rustfs/backlog#2747, rustfs/backlog#2734
* refactor(ecstore): move S3ErrorCode to rustfs-s3-types by codemod (#8432)
* test(ecstore): pin the InvalidRange responses of new_getobjectreader
Both error branches of new_getobjectreader build an ErrorResponse with S3ErrorCode::InvalidRange, and no test observed either code. Pin the code, its wire name and the location fields of each branch before the code type moves to rustfs-s3-types (rustfs/backlog#2743), so a changed code on either branch fails a test. The ecstore test-count row rises by the two tests.
* chore(scripts): add the s3s-to-rustfs-s3-types error codemod
Task T1.2 of rustfs/backlog#2734 (rustfs/backlog#2743) moves S3Error, S3ErrorCode, S3Result and s3_error! from s3s to rustfs-s3-types by script, one crate closure per PR. The codemod rewrites use trees (splitting mixed s3s imports) and qualified code paths, never comments or literals, adds the rustfs-s3-types dependency, drops s3s only from a crate that no longer names it, refuses every file of the s3s edge allowlist, takes --hold for a file whose type is fixed outside the closure, and prints a per-crate census. A second run changes nothing. Its self-test runs under make script-tests.
* refactor(ecstore): take S3ErrorCode from rustfs-s3-types
Output of scripts/codemods/s3_error_to_s3_types.py over crates/ecstore, crates/lifecycle, crates/scanner and crates/replication (rustfs/backlog#2743 batches 1 and 2). The ErrorResponse code field, the two InvalidRange constructions in new_getobjectreader and an unused tier import now use rustfs_s3_types::S3ErrorCode; the variant names and wire names are unchanged. lifecycle, scanner and replication reference none of the four error names. warm_backend.rs and warm_backend_s3.rs are held: they compare against rustfs_s3_client::ErrorResponse.code, which stays the s3s type until s3-client moves.
* ci(footprint): count only s3_error! lines still on s3s's macro
s3_error_lines counted every line holding the text `s3_error!`, which
also matches the RustFS-owned `rustfs_s3_types::s3_error!`. A codemod
that moves call sites onto rustfs-s3-types therefore could not lower it
(rustfs/backlog#2743, decision 3; rustfs/rustfs#8432, gotcha 2).
The counter now applies a textual, per-file and per-line rule that errs
on the side of counting (full rule in the script header):
- occurrences spelled `rustfs_s3_types::s3_error!` never count;
- a line holding `s3s::s3_error!` always counts;
- any other `s3_error!` line counts unless its file is cleared, that is,
it imports the macro from rustfs_s3_types by name on a line starting
with the use item (or defines it with `macro_rules! s3_error`) and does
not also bring s3s's macro in (`use s3s::s3_error`, a `use s3s::{..}`
tree naming it or a crate-root `*`, `use s3s::*`, or
`#[macro_use] extern crate s3s`);
- fail closed: a file matching neither form (`use super::*`, a
crate-local re-export, a renamed or glob rustfs_s3_types import, a
commented-out import, a comment-only mention) counts.
Census of the 1585 lines on 96fc06776: 1478 in 64 files that import the
macro from s3s; 107 in 15 files matching neither form (105 through
`use super::*`, 1 qualified `s3s::s3_error!`, 1 comment line); none in
files that import it from rustfs_s3_types or define it. Every line stays
counted, so the baseline remains s3_error_lines=1585 (files=202 and
ecstore_files=36 unchanged). Rewriting rustfs/src/auth.rs's import to
rustfs_s3_types lowers the new counter to 1567; the old one stayed 1585.
`--self-test` (wired into `make script-tests`) copies the script into
22 fixture trees: 6 positive cases and 16 negative ones (lines that must
not count, forms that must not clear a file, and a missing baseline,
missing key or failing classification search that must fail). Each tree
has a control line that must count, and each run gets s3s text on stdin.
Seven cases were red against the old counter. 19 mutants each turn their
case red: old definition, flipped import test, dropped fail-closed
branch, dropped e2e_test exclusion, dropped s3s veto, dropped qualified
s3s rule, dropped rustfs-qualified strip, unanchored rustfs import,
renamed import accepted, nested glob taken as root glob, single-line
s3s trees only, dropped s3s glob forms, dropped macro_use form, dropped
definition clearing, ignored classification rg errors, error-line search
without its path, classification searches without their path, missing
baseline skipped, missing key skipped.
* test(e2e): compare two RUSTFS_S3_STACK servers over every S3 operation (#8435)
Add dual_stack_differential to e2e_test. It starts two rustfs processes
from the same binary, stack A on RUSTFS_S3_STACK=legacy and stack B on
RUSTFS_E2E_STACK_B (default legacy until rustfs/backlog#2734 T2.5),
sends both the same aws-sdk-s3 script covering the 73 methods of
impl S3 for FS (a normal request and a 4xx negative each), and compares
status, error code, an allowlist of response headers and a few typed
body fields. Error message text is never compared. Each run writes
<target>/dual-stack/<run>.json with compared_ops and the (op, field,
a, b) differences; any difference fails the test.
differential_detects_an_injected_difference starts B with
RUSTFS_REGION=eu-west-1 and requires GetBucketLocation to be reported.
The comparator is pure and unit-tested without a server.
Wire the module into the e2e-smoke filter and update the e2e-smoke and
e2e-full selection digests and the e2e_test test-count row.
Refs rustfs/backlog#2740
* feat(app): add the RustFS-owned RequestEnvelope and its s3s constructor (#8437)
* feat(app): add the RustFS-owned RequestEnvelope and its s3s constructor
* ci: classify the s3s edge adapters in storage/s3_compat as interface
* feat(s3-types): add Body::from_stream and move Swift off s3s::Body (#8438)
rustfs/backlog#2747 batch 3. `Body::from_stream` wraps a chunk stream of
unknown length over `StreamBody`/`BoxBody`: one data frame per chunk, the
first error ends the body, the size hint stays unknown, and dropping the
body drops the stream. Parity tests pin it to the s3s `StreamingBlob::wrap`
it replaces, at the `http_body` level and byte for byte through hyper.
The Swift handlers in rustfs-protocols build their responses as the owned
`Body`. `SwiftService` converts them into the wrapped S3 service's response
body type through `From<Body>`, so rustfs keeps serving `s3s::Body` at the
legacy edge unchanged. Request bodies stay Send-only (`UnsyncBoxBody`); the
two storage writers and the DLO/SLO segment streams get `Sync` from
`sync_wrapper::SyncStream`, which is what s3s's `http_body_unsync` mutex
provided before. The DTO and error uses of s3s in protocols are untouched
(T1.5 and T1.2).
* refactor(protocols): move error types to rustfs-s3-types by codemod (#8439)
T1.2 batch 3 (rustfs/backlog#2743): s3select-api, s3select-query and
the error items of protocols move off s3s by scripts/codemods, run with
--hold crates/protocols/src/sftp/errors.rs. That file stays on s3s: it
downcasts the backend error, whose runtime type is still s3s::S3Error.
- rustfs/src/protocols/client.rs: the two webdav session methods now
return rustfs_s3_types::S3Result and convert the s3s error with From.
- codemod: a child module that calls s3_error! through `use super::*`
gets its own `use rustfs_s3_types::s3_error;` once its parent imports
that macro by name; otherwise it is listed as GLOB.
- s3-types: a table test pins code, status and the s3s error the legacy
edge renders for every variant a moved call site names.
- s3_error_lines 1585 -> 1584 (a lifecycle comment reworded).
* test(connect): restore one-second drive request budgets
Apply the exact test-only repair already landed on main in 5972a09cc3. The filesystem probe and production CLI success test request one second and keep their existing one-second duration and parameter checks. All outcome, I/O, error, cleanup and resource-limit assertions remain.
Scope: rustfs/tests/connect_perf_drive.rs only. Patch-id: cb0a1dfc8f68a67727525adea7458e2cee7d09f8.
Validation environment: LANG=en_US.UTF-8, CARGO_BUILD_JOBS=1, CARGO_INCREMENTAL=0, exclusive APFS-cloned CARGO_TARGET_DIR.
Quick Checks: every command from .github/actions/quick-checks/action.yml, with captured output and exit status below.
$ shellcheck --version && actionlint
ShellCheck - shell script analysis tool
version: 0.11.0
license: GNU General Public License, version 3
website: https://www.shellcheck.net
exit_code=0
$ cargo fmt --all --check
exit_code=0
$ ./scripts/check_unsafe_code_allowances.sh
exit_code=0
$ ./scripts/check_layer_dependencies.sh
Layer dependency guard passed (no new reverse dependencies/cycles).
exit_code=0
$ ./scripts/check_architecture_migration_rules.sh
Architecture migration rules passed.
exit_code=0
$ ./scripts/check_logging_guardrails.sh
✅ Logging guardrails check passed
exit_code=0
$ ./scripts/check_error_other_format_ratchet.sh
error(format!) ratchet passed (total call sites: 582).
exit_code=0
$ ./scripts/check_no_tokio_io_uring.sh
exit_code=0
$ ./scripts/check_extension_schema_boundaries.sh
Extension schema boundaries passed.
exit_code=0
$ ./scripts/check_body_cache_whitelist.sh
Body-cache whitelist guard passed.
exit_code=0
$ ./scripts/check_s3s_footprint.sh
s3s footprint OK: files importing s3s is 195 (baseline files=195)
s3s footprint OK: s3s s3_error! lines is 1584 (baseline s3_error_lines=1584)
s3s footprint OK: ecstore files referencing s3s is 36 (baseline ecstore_files=36)
✅ s3s footprint baseline check passed
exit_code=0
$ ./scripts/check_test_count_ratchet.sh
test-count ratchet OK: 52 crates, 21222 tests, 164 ignored match .config/test-count-baseline.txt
exit_code=0
$ ./scripts/check_fips_wording.sh
FIPS wording guard passed (README.md and CHANGELOG.md contain no forbidden claims).
KMS wording guard passed (crates/kms claims no Vault KV2 Transit wrapping).
exit_code=0
$ ./scripts/check_embedded_secrets.sh
Embedded secret guard passed (no private key material or provider credential literal in the tree).
exit_code=0
$ make script-tests
Running script tests...
entrypoint credential policy tests passed
internode grpc A/B bench env tests passed
object batch benchmark enhanced tests passed
PASS package service script contract
hotpath warp A/B gate tests passed
hotpath warp ABBA tests passed
Scanner validation artifacts written to /var/folders/0w/s4wxsn2n4w9b6f3cd4xrz53w0000gn/T/tmp.1pFe6Kl1Yu/out
Scanner validation artifacts written to /var/folders/0w/s4wxsn2n4w9b6f3cd4xrz53w0000gn/T/tmp.1pFe6Kl1Yu/out-missing-pid
.............................
----------------------------------------------------------------------
Ran 29 tests in 181.286s
OK
{"decision":"verified","pending_gates":["G01","G03","G04","G05","G06","G07","G08","G09","G10","G11","G12","G13","G14","P1","P2","P3","P4","R-D","R-E","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"G02"}
{"decision":"verified","pending_gates":["G01","G02","G03","G04","G05","G06","G07","G08","G09","G10","G11","G12","G13","G14","P1","P2","P3","P4","R-D","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"R-E"}
{"decision":"verified","pending_gates":["G01","G03","G04","G05","G06","G07","G08","G09","G10","G11","G12","G13","G14","P1","P2","P3","P4","R-D","R-E","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"G02"}
{"decision":"verified","pending_gates":["G01","G02","G03","G04","G05","G06","G07","G08","G09","G10","G11","G12","G13","G14","P1","P2","P3","P4","R-D","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"R-E"}
{"decision":"verified","pending_gates":["G02","G03","G04","G05","G06","G07","G08","G09","G10","G11","G12","G13","G14","P1","P2","P3","P4","R-D","R-E","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"G01"}
{"decision":"verified","pending_gates":["G01","G02","G04","G05","G06","G07","G08","G09","G10","G11","G12","G13","G14","P1","P2","P3","P4","R-D","R-E","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"G03"}
{"decision":"verified","pending_gates":["G01","G02","G03","G04","G05","G06","G07","G08","G09","G10","G11","G12","G13","G14","P1","P2","P3","P4","R-D","R-E"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"R-L"}
{"decision":"verified","pending_gates":["G01","G02","G03","G04","G05","G06","G07","G08","G09","G10","G11","G12","G13","P1","P2","P3","P4","R-D","R-E","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"G14"}
{"decision":"verified","pending_gates":["G01","G02","G03","G04","G05","G06","G07","G08","G09","G10","G11","G12","G13","P1","P2","P3","P4","R-D","R-E","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"G14"}
{"decision":"verified","pending_gates":["G01","G02","G03","G04","G05","G06","G07","G08","G09","G11","G12","G13","G14","P1","P2","P3","P4","R-D","R-E","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"G10"}
{"decision":"verified","pending_gates":["G01","G02","G03","G04","G05","G06","G07","G08","G09","G10","G11","G12","G13","G14","P2","P3","P4","R-D","R-E","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"P1"}
{"decision":"verified","pending_gates":["G01","G02","G03","G04","G05","G06","G07","G08","G09","G10","G11","G12","G13","G14","P1","P3","P4","R-D","R-E","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"P2"}
{"decision":"verified","pending_gates":["G01","G02","G03","G04","G05","G06","G07","G08","G09","G10","G11","G12","G13","G14","P1","P2","P4","R-D","R-E","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"P3"}
{"decision":"verified","pending_gates":["G01","G02","G03","G04","G06","G07","G08","G09","G10","G11","G12","G13","G14","P1","P2","P3","P4","R-D","R-E","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"G05"}
{"decision":"verified","pending_gates":["G01","G02","G03","G04","G05","G07","G08","G09","G10","G11","G12","G13","G14","P1","P2","P3","P4","R-D","R-E","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"G06"}
{"decision":"verified","pending_gates":["G01","G02","G03","G04","G05","G06","G07","G08","G09","G10","G11","G12","G13","G14","P1","P2","P3","P4","R-E","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"R-D"}
Status-and-outcome release descriptor verified: /private/var/folders/0w/s4wxsn2n4w9b6f3cd4xrz53w0000gn/T/tmp8rwn1pe0/descriptor/release-bundle-status-outcome.json
{"decision":"verified","pending_gates":["G01","G02","G03","G04","G06","G07","G08","G09","G10","G11","G12","G13","G14","P1","P2","P3","P4","R-D","R-E","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"G05"}
{"decision":"verified","pending_gates":["G01","G02","G03","G04","G05","G07","G08","G09","G10","G11","G12","G13","G14","P1","P2","P3","P4","R-D","R-E","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"G06"}
{"decision":"verified","pending_gates":["G01","G02","G03","G04","G05","G06","G07","G08","G09","G10","G11","G12","G13","G14","P1","P2","P3","P4","R-E","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"R-D"}
{"decision":"verified","pending_gates":["G01","G02","G03","G04","G05","G06","G08","G09","G10","G11","G12","G13","G14","P1","P2","P3","P4","R-D","R-E","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"G07"}
{"decision":"verified","pending_gates":["G01","G02","G03","G04","G05","G06","G07","G09","G10","G11","G12","G13","G14","P1","P2","P3","P4","R-D","R-E","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"G08"}
{"decision":"verified","pending_gates":["G01","G02","G03","G04","G05","G06","G07","G08","G09","G10","G11","G12","G13","G14","P1","P2","P3","R-D","R-E","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"P4"}
exact 1MiB handoff ABBA tests passed
pinned paired ABBA bench tests passed
==> /var/folders/0w/s4wxsn2n4w9b6f3cd4xrz53w0000gn/T/tmp.mYmuI01hNq/bin/path_containment (-max_total_time=7, -seed=123456789, -artifact_prefix=/var/folders/0w/s4wxsn2n4w9b6f3cd4xrz53w0000gn/T/tmp.mYmuI01hNq/explicit-artifacts/path_containment/, corpus=/private/tmp/gw-orchestrator-2734/ci-drive/fuzz/corpus/path_containment)
-seed=123456789
target=path_containment
seed=123456789
max_total_time=7
git_revision=4d15b816236630b2c42f321ffc9ed3654edc94e3
git_dirty=true
runner_mode=prebuilt
==> /var/folders/0w/s4wxsn2n4w9b6f3cd4xrz53w0000gn/T/tmp.mYmuI01hNq/bin/path_containment (-max_total_time=1, -seed=2322617706, -artifact_prefix=/var/folders/0w/s4wxsn2n4w9b6f3cd4xrz53w0000gn/T/tmp.mYmuI01hNq/auto-artifacts/path_containment/, corpus=/private/tmp/gw-orchestrator-2734/ci-drive/fuzz/corpus/path_containment)
-seed=2322617706
==> cargo +nightly fuzz run path_containment (-max_total_time=9, -seed=987654321, -artifact_prefix=/var/folders/0w/s4wxsn2n4w9b6f3cd4xrz53w0000gn/T/tmp.mYmuI01hNq/cargo-artifacts/path_containment/)
+nightly
fuzz
run
path_containment
-seed=987654321
seed=987654321
runner_mode=cargo-fuzz
fuzz runner seed manifest tests passed
✅ scripts/python_bin.sh resolver checks passed
self-test ok: clean fixture with every exemption present passes
self-test ok: all 11 pattern families fire
self-test ok: moving a pinned source is reported
Embedded secret guard self-test passed.
self-test ok: p1-s3s-import: 'use s3s::s3_error;' counts every line, the rustfs_s3_types module's too (s3_error_lines=3)
self-test ok: p2-s3s-mixed-tree: a multi-line s3s use tree naming s3_error counts (s3_error_lines=3)
self-test ok: p3-s3s-glob: a crate-root glob of s3s, bare or inside a tree, counts (s3_error_lines=5)
self-test ok: p4-macro-use: '#[macro_use] extern crate s3s' counts (s3_error_lines=3)
self-test ok: p5-qualified-s3s: a qualified s3s::s3_error! line counts in a cleared file (s3_error_lines=2)
self-test ok: p6-exact-pass: a baseline equal to the tree passes (exit 0)
self-test ok: n1-rustfs-import: 'use rustfs_s3_types::s3_error;' alone counts nothing (s3_error_lines=1)
self-test ok: n2-rustfs-mixed-tree: a rustfs_s3_types tree plus an s3s tree without the macro counts nothing (s3_error_lines=1)
self-test ok: n3-defines: the file defining the macro counts nothing (s3_error_lines=1)
self-test ok: n4-qualified-rustfs: a qualified rustfs_s3_types::s3_error! line counts nothing (s3_error_lines=1)
self-test ok: n5-renamed-call: a call through a renamed macro is not text-matched (out of scope) (s3_error_lines=1)
self-test ok: n6-comment-string: a comment or string mentioning s3_error! in a cleared file counts nothing (s3_error_lines=1)
self-test ok: n7-e2e-excluded: crates/e2e_test is excluded (s3_error_lines=1)
self-test ok: n8-super-glob: an unrecognised import form ('use super::*;') counts (s3_error_lines=3)
self-test ok: n9-facade: a crate-local re-export counts (s3_error_lines=2)
self-test ok: n10-renamed-import: a renamed rustfs_s3_types import does not clear s3_error! lines (s3_error_lines=2)
self-test ok: n11-commented-import: a commented-out rustfs_s3_types import does not clear (s3_error_lines=2)
self-test ok: n12-rustfs-glob: a glob import of rustfs_s3_types does not clear (s3_error_lines=2)
self-test ok: n13-comment-only: a comment mentioning s3_error! in a file with no import counts (s3_error_lines=2)
self-test ok: f1-baseline-missing: a missing baseline file fails (exit 1)
self-test ok: f2-key-missing: a baseline without s3_error_lines fails (exit 1)
self-test ok: f3-rg-fails: a failing classification search fails the guard (exit 1)
✅ s3s footprint self-test passed
test_core_gate_rejects_missing_ignored_filtered_and_corrupt_inputs (__main__.SelfTests.test_core_gate_rejects_missing_ignored_filtered_and_corrupt_inputs) ... ok
test_e2e_requires_registration (__main__.SelfTests.test_e2e_requires_registration) ... ok
test_fuzz_runtime_uses_matrix_target (__main__.SelfTests.test_fuzz_runtime_uses_matrix_target) ... ok
test_ilm_lane_keeps_the_measured_cargo_build_budget (__main__.SelfTests.test_ilm_lane_keeps_the_measured_cargo_build_budget) ... ok
test_profile_listing_binds_platform_digest (__main__.SelfTests.test_profile_listing_binds_platform_digest) ... ok
test_profile_listing_enforces_selection (__main__.SelfTests.test_profile_listing_enforces_selection) ... ok
test_quick_checks_commands_propagate_failure (__main__.SelfTests.test_quick_checks_commands_propagate_failure) ... ok
test_quick_checks_rejects_caller_and_execution_bypasses (__main__.SelfTests.test_quick_checks_rejects_caller_and_execution_bypasses) ... ok
test_runner_readiness_fails_closed (__main__.SelfTests.test_runner_readiness_fails_closed) ... ok
test_s3_runner_rejects_unbounded_failure_locals (__main__.SelfTests.test_s3_runner_rejects_unbounded_failure_locals) ... ok
test_scanner_heal_begin_requires_embedded_source_provenance (__main__.SelfTests.test_scanner_heal_begin_requires_embedded_source_provenance) ... ok
test_scanner_heal_case_does_not_approve_pending_release (__main__.SelfTests.test_scanner_heal_case_does_not_approve_pending_release) ... ok
test_scanner_heal_case_lane_is_required_for_runner_profile (__main__.SelfTests.test_scanner_heal_case_lane_is_required_for_runner_profile) ... ok
test_scanner_heal_case_only_schema_cannot_approve_release (__main__.SelfTests.test_scanner_heal_case_only_schema_cannot_approve_release) ... ok
test_scanner_heal_crash_case_rejects_restart_oracle (__main__.SelfTests.test_scanner_heal_crash_case_rejects_restart_oracle) ... ok
test_scanner_heal_ec84_case_rejects_wrong_erasure_geometry (__main__.SelfTests.test_scanner_heal_ec84_case_rejects_wrong_erasure_geometry) ... ok
test_scanner_heal_finish_collects_oracles_from_registry (__main__.SelfTests.test_scanner_heal_finish_collects_oracles_from_registry) ... ok
test_scanner_heal_multipool_case_accepts_deferred_optional_outage_write (__main__.SelfTests.test_scanner_heal_multipool_case_accepts_deferred_optional_outage_write) ... ok
test_scanner_heal_pending_gate_cannot_map_to_implemented_lane (__main__.SelfTests.test_scanner_heal_pending_gate_cannot_map_to_implemented_lane) ... ok
test_scanner_heal_receipts_reject_reuse_and_missing_builds (__main__.SelfTests.test_scanner_heal_receipts_reject_reuse_and_missing_builds) ... ok
test_scanner_heal_rejects_boolean_fractional_and_out_of_geometry_integers (__main__.SelfTests.test_scanner_heal_rejects_boolean_fractional_and_out_of_geometry_integers) ... ok
test_scanner_heal_rejects_broken_execution_and_artifacts (__main__.SelfTests.test_scanner_heal_rejects_broken_execution_and_artifacts) ... ok
test_scanner_heal_rejects_copied_junit_and_wrong_suite_build (__main__.SelfTests.test_scanner_heal_rejects_copied_junit_and_wrong_suite_build) ... ok
test_scanner_heal_release_bundle_accepts_complete_measured_evidence (__main__.SelfTests.test_scanner_heal_release_bundle_accepts_complete_measured_evidence) ... ok
test_scanner_heal_release_bundle_assembler_copies_measured_artifacts (__main__.SelfTests.test_scanner_heal_release_bundle_assembler_copies_measured_artifacts) ... ok
test_scanner_heal_release_bundle_assembler_merges_measured_descriptors (__main__.SelfTests.test_scanner_heal_release_bundle_assembler_merges_measured_descriptors) ... ok
test_scanner_heal_release_bundle_assembler_rejects_duplicate_descriptor_gate (__main__.SelfTests.test_scanner_heal_release_bundle_assembler_rejects_duplicate_descriptor_gate) ... ok
test_scanner_heal_release_bundle_assembler_rejects_fixture_descriptor (__main__.SelfTests.test_scanner_heal_release_bundle_assembler_rejects_fixture_descriptor) ... ok
test_scanner_heal_release_bundle_assembler_rejects_hash_mismatch (__main__.SelfTests.test_scanner_heal_release_bundle_assembler_rejects_hash_mismatch) ... ok
test_scanner_heal_release_bundle_assembler_rejects_mixed_descriptor_revisions (__main__.SelfTests.test_scanner_heal_release_bundle_assembler_rejects_mixed_descriptor_revisions) ... ok
test_scanner_heal_release_bundle_enforces_topology_duration_profile_and_versions (__main__.SelfTests.test_scanner_heal_release_bundle_enforces_topology_duration_profile_and_versions) ... ok
test_scanner_heal_release_bundle_fixture_writer_is_not_release_evidence (__main__.SelfTests.test_scanner_heal_release_bundle_fixture_writer_is_not_release_evidence) ... ok
test_scanner_heal_release_bundle_gate_accepts_single_measured_gate (__main__.SelfTests.test_scanner_heal_release_bundle_gate_accepts_single_measured_gate) ... ok
test_scanner_heal_release_bundle_gate_rejects_missing_field (__main__.SelfTests.test_scanner_heal_release_bundle_gate_rejects_missing_field) ... ok
test_scanner_heal_release_bundle_gate_rejects_unknown_gate (__main__.SelfTests.test_scanner_heal_release_bundle_gate_rejects_unknown_gate) ... ok
test_scanner_heal_release_bundle_rejects_synthetic_or_missing_fields (__main__.SelfTests.test_scanner_heal_release_bundle_rejects_synthetic_or_missing_fields) ... ok
test_scanner_heal_release_bundle_requires_domain_evidence (__main__.SelfTests.test_scanner_heal_release_bundle_requires_domain_evidence) ... ok
test_scanner_heal_release_bundle_requires_field_provenance (__main__.SelfTests.test_scanner_heal_release_bundle_requires_field_provenance) ... ok
test_scanner_heal_release_bundle_requires_json_artifact_provenance (__main__.SelfTests.test_scanner_heal_release_bundle_requires_json_artifact_provenance) ... ok
test_scanner_heal_release_bundle_requires_mixed_version_field_roles (__main__.SelfTests.test_scanner_heal_release_bundle_requires_mixed_version_field_roles) ... ok
test_scanner_heal_release_bundle_requires_same_gate_measurement_window (__main__.SelfTests.test_scanner_heal_release_bundle_requires_same_gate_measurement_window) ... ok
test_scanner_heal_release_consumes_multiple_registry_oracles (__main__.SelfTests.test_scanner_heal_release_consumes_multiple_registry_oracles) ... ok
test_scanner_heal_release_matrix_lanes_remain_pending_without_real_evidence (__main__.SelfTests.test_scanner_heal_release_matrix_lanes_remain_pending_without_real_evidence) ... ok
test_scanner_heal_release_requirements_cannot_be_cleared (__main__.SelfTests.test_scanner_heal_release_requirements_cannot_be_cleared) ... ok
test_scanner_heal_release_status_rejects_focused_case_run (__main__.SelfTests.test_scanner_heal_release_status_rejects_focused_case_run) ... ok
test_scanner_heal_release_status_rejects_synthetic_case (__main__.SelfTests.test_scanner_heal_release_status_rejects_synthetic_case) ... ok
test_scanner_heal_required_evidence_fields_cannot_be_removed (__main__.SelfTests.test_scanner_heal_required_evidence_fields_cannot_be_removed) ... ok
test_scheduled_alerts_require_completion_watchdog (__main__.SelfTests.test_scheduled_alerts_require_completion_watchdog) ... ok
test_update_profile_selection_changes_only_requested_platform (__main__.SelfTests.test_update_profile_selection_changes_only_requested_platform) ... ok
test_validate_includes_quick_checks (__main__.SelfTests.test_validate_includes_quick_checks) ... ok
test_vault_tests_require_cross_process_serialization (__main__.SelfTests.test_vault_tests_require_cross_process_serialization) ... ok
test_workflow_readiness_requires_dependency_probe (__main__.SelfTests.test_workflow_readiness_requires_dependency_probe) ... ok
----------------------------------------------------------------------
Ran 52 tests in 14.301s
OK
e2e-full selection OK: 1 tests, sha256=29e30e6ed2ed38d9e5b0a7cdd9536abf5c719f7021a02b578ae1381563a3c273
...................................
----------------------------------------------------------------------
Ran 35 tests in 1.161s
OK
...................
----------------------------------------------------------------------
Ran 19 tests in 17.397s
OK
..........
----------------------------------------------------------------------
Ran 10 tests in 0.183s
OK
test_actual_selector_bootstrap_uses_base_policy_and_fails_closed (__main__.SelfTests.test_actual_selector_bootstrap_uses_base_policy_and_fails_closed) ... ok
test_documentation_paths_do_not_hide_build_or_fixture_changes (__main__.SelfTests.test_documentation_paths_do_not_hide_build_or_fixture_changes) ... ok
test_e2e_scope_keeps_production_and_shared_configuration_full (__main__.SelfTests.test_e2e_scope_keeps_production_and_shared_configuration_full) ... ok
test_e2e_shortcut_rejects_new_direct_renamed_and_target_dependencies (__main__.SelfTests.test_e2e_shortcut_rejects_new_direct_renamed_and_target_dependencies) ... ok
test_e2e_shortcut_traverses_implicit_path_dependencies (__main__.SelfTests.test_e2e_shortcut_traverses_implicit_path_dependencies) ... ok
test_event_contract_requires_complete_candidate_and_optional_lanes (__main__.SelfTests.test_event_contract_requires_complete_candidate_and_optional_lanes) ... ok
test_every_wrong_result_missing_job_or_selection_fails_closed (__main__.SelfTests.test_every_wrong_result_missing_job_or_selection_fails_closed) ... ok
test_full_e2e_gate_preserves_workflow_branch_and_event_scope (__main__.SelfTests.test_full_e2e_gate_preserves_workflow_branch_and_event_scope) ... ok
test_git_range_includes_deleted_source_and_rename_origins (__main__.SelfTests.test_git_range_includes_deleted_source_and_rename_origins) ... ok
test_job_ids_and_display_names_cannot_hide_validation (__main__.SelfTests.test_job_ids_and_display_names_cannot_hide_validation) ... ok
test_report_only_stack_lane_mirrors_e2e_smoke_outside_the_aggregate (__main__.SelfTests.test_report_only_stack_lane_mirrors_e2e_smoke_outside_the_aggregate) ... ok
test_repository_wiring_and_missing_dependency_regression (__main__.SelfTests.test_repository_wiring_and_missing_dependency_regression) ... ok
test_verify_command_preserves_failures (__main__.SelfTests.test_verify_command_preserves_failures) ... ok
----------------------------------------------------------------------
Ran 13 tests in 3.280s
OK
security coverage self-test passed
test_api_failures_preserve_other_evidence_and_never_enter_grace (__main__.SelfTests.test_api_failures_preserve_other_evidence_and_never_enter_grace) ... ok
test_attempt_outcome_does_not_replace_recent_success (__main__.SelfTests.test_attempt_outcome_does_not_replace_recent_success) ... ok
test_cancelled_attempt_cannot_refresh_expired_success (__main__.SelfTests.test_cancelled_attempt_cannot_refresh_expired_success) ... ok
test_cli_requires_the_repository_default_branch (__main__.SelfTests.test_cli_requires_the_repository_default_branch) ... ok
test_config_rejects_duplicate_and_invalid_entries (__main__.SelfTests.test_config_rejects_duplicate_and_invalid_entries) ... ok
test_freshness_boundaries (__main__.SelfTests.test_freshness_boundaries) ... ok
test_grace_requires_two_successful_queries_with_no_history (__main__.SelfTests.test_grace_requires_two_successful_queries_with_no_history) ... ok
test_invalid_api_evidence_fails_closed (__main__.SelfTests.test_invalid_api_evidence_fails_closed) ... ok
test_report_retains_every_workflow (__main__.SelfTests.test_report_retains_every_workflow) ... ok
test_requests_filter_schedule_default_branch_and_success_on_server (__main__.SelfTests.test_requests_filter_schedule_default_branch_and_success_on_server) ... ok
----------------------------------------------------------------------
Ran 10 tests in 0.051s
OK
..............................
----------------------------------------------------------------------
Ran 30 tests in 22.354s
OK
..........
----------------------------------------------------------------------
Ran 10 tests in 2.969s
OK
.................................
----------------------------------------------------------------------
Ran 33 tests in 0.160s
OK
Online functional runner labels: pf-testing
.......
----------------------------------------------------------------------
Ran 7 tests in 0.002s
OK
.......
----------------------------------------------------------------------
Ran 7 tests in 0.001s
OK
....
----------------------------------------------------------------------
Ran 4 tests in 0.128s
OK
...
----------------------------------------------------------------------
Ran 3 tests in 0.547s
OK
PASS: cold stampede script self-test
exit_code=0
$ python3 ./scripts/test_object_generation_protocol_model.py
test_a_recovery_barrier_cannot_choose_an_empty_value (__main__.ObjectGenerationProtocolModel.test_a_recovery_barrier_cannot_choose_an_empty_value) ... ok
test_all_bounded_two_proposer_interleavings_choose_at_most_one_value (__main__.ObjectGenerationProtocolModel.test_all_bounded_two_proposer_interleavings_choose_at_most_one_value) ... ok
test_an_unavailable_decision_quorum_cannot_choose_a_candidate (__main__.ObjectGenerationProtocolModel.test_an_unavailable_decision_quorum_cannot_choose_a_candidate) ... ok
test_counter_allocation_requires_persistence_and_is_consumed_once (__main__.ObjectGenerationProtocolModel.test_counter_allocation_requires_persistence_and_is_consumed_once) ... ok
test_counter_restart_burns_persisted_allocations_before_or_after_issue (__main__.ObjectGenerationProtocolModel.test_counter_restart_burns_persisted_allocations_before_or_after_issue) ... ok
test_crash_after_promise_before_reply_does_not_restore_an_older_ballot (__main__.ObjectGenerationProtocolModel.test_crash_after_promise_before_reply_does_not_restore_an_older_ballot) ... ok
test_duplicate_or_unconfigured_voters_cannot_form_a_quorum (__main__.ObjectGenerationProtocolModel.test_duplicate_or_unconfigured_voters_cannot_form_a_quorum) ... ok
test_empty_or_unobserved_minority_slot_can_retry_without_allocating_another_revision (__main__.ObjectGenerationProtocolModel.test_empty_or_unobserved_minority_slot_can_retry_without_allocating_another_revision) ... ok
test_equal_ballot_cannot_accept_different_bytes_and_lower_ballot_is_rejected (__main__.ObjectGenerationProtocolModel.test_equal_ballot_cannot_accept_different_bytes_and_lower_ballot_is_rejected) ... ok
test_forgetting_tombstone_authority_allows_an_aba_create (__main__.ObjectGenerationProtocolModel.test_forgetting_tombstone_authority_allows_an_aba_create) ... ok
test_issuing_before_persistence_reuses_a_ballot_after_restart (__main__.ObjectGenerationProtocolModel.test_issuing_before_persistence_reuses_a_ballot_after_restart) ... ok
test_lost_ack_and_restart_preserve_the_chosen_operation (__main__.ObjectGenerationProtocolModel.test_lost_ack_and_restart_preserve_the_chosen_operation) ... ok
test_minority_accepted_work_is_adopted_and_finished (__main__.ObjectGenerationProtocolModel.test_minority_accepted_work_is_adopted_and_finished) ... ok
test_omitting_highest_accepted_adoption_has_a_double_choice_counterexample (__main__.ObjectGenerationProtocolModel.test_omitting_highest_accepted_adoption_has_a_double_choice_counterexample) ... ok
test_quorum_choice_does_not_require_an_uncontacted_disk_to_learn_it (__main__.ObjectGenerationProtocolModel.test_quorum_choice_does_not_require_an_uncontacted_disk_to_learn_it) ... ok
test_recovery_failure_or_stale_expected_head_does_not_allocate_a_successor (__main__.ObjectGenerationProtocolModel.test_recovery_failure_or_stale_expected_head_does_not_allocate_a_successor) ... ok
test_replacing_voters_with_empty_state_can_choose_a_conflicting_value (__main__.ObjectGenerationProtocolModel.test_replacing_voters_with_empty_state_can_choose_a_conflicting_value) ... ok
test_skipping_predecessor_recovery_chooses_a_broken_revision_chain (__main__.ObjectGenerationProtocolModel.test_skipping_predecessor_recovery_chooses_a_broken_revision_chain) ... ok
test_stale_publication_rollback_and_gc_cannot_change_the_winner (__main__.ObjectGenerationProtocolModel.test_stale_publication_rollback_and_gc_cannot_change_the_winner) ... ok
test_superseded_operation_identity_cannot_be_reused_for_different_input (__main__.ObjectGenerationProtocolModel.test_superseded_operation_identity_cannot_be_reused_for_different_input) ... ok
test_superseded_operation_retry_returns_its_original_successor (__main__.ObjectGenerationProtocolModel.test_superseded_operation_retry_returns_its_original_successor) ... ok
test_tombstone_and_null_marker_payloads_do_not_reset_authority_revision (__main__.ObjectGenerationProtocolModel.test_tombstone_and_null_marker_payloads_do_not_reset_authority_revision) ... ok
test_tombstone_rejects_a_delayed_never_present_create (__main__.ObjectGenerationProtocolModel.test_tombstone_rejects_a_delayed_never_present_create) ... ok
test_unlearned_predecessor_is_recovered_before_the_second_slot (__main__.ObjectGenerationProtocolModel.test_unlearned_predecessor_is_recovered_before_the_second_slot) ... ok
----------------------------------------------------------------------
Ran 24 tests in 0.958s
OK
model: states=51023, transitions=167520, terminal=732, chosen_candidates=2
negative control: double-choice trace=((0, 'prepare', 0), (0, 'prepare', 1), (0, 'prepare', 2), (0, 'accept', 0), (0, 'accept', 1), (0, 'accept', 2), (1, 'prepare', 0), (1, 'prepare', 1), (1, 'prepare', 2), (1, 'accept', 0), (1, 'accept', 1), (1, 'accept', 2))
multi-slot round schedules: 640
exit_code=0
$ python3 ./scripts/check_test_wiring.py
OK: e2e modules, runner selection, fuzz matrices, profiles, scheduled alerts, and Quick Checks are wired
exit_code=0
$ ./scripts/check_no_planning_docs.sh
exit_code=0
$ ./scripts/check_uring_lane_lib_only.sh
OK: no 'uring_'-matching test functions under crates/ecstore/tests; --lib is safe
exit_code=0
Initial complete gate
$ cargo nextest run --profile ci -p rustfs --test connect_perf_drive --no-fail-fast
Nextest run ID e07fc5b9-c6c8-4cdd-ae61-54839bf24437 with nextest profile: ci
Starting 10 tests across 1 binary
PASS [ 0.016s] rustfs::connect_perf_drive cancellation_and_single_collector_leave_no_scratch
PASS [ 0.011s] rustfs::connect_perf_drive cleanup_failure_overrides_cancellation_and_remains_attributed
PASS [ 0.009s] rustfs::connect_perf_drive consent_and_resource_budgets_fail_before_scratch_io
PASS [ 0.009s] rustfs::connect_perf_drive invalid_roots_and_existing_scratch_fail_without_clobbering
PASS [ 16.950s] rustfs::connect_perf_drive production_cli_measures_and_signs_exact_binary_provenance
PASS [ 4.025s] rustfs::connect_perf_drive production_cli_rejects_missing_l1_consent_and_limits_before_artifacts
PASS [ 0.008s] rustfs::connect_perf_drive read_only_root_and_full_drive_errors_are_classified
PASS [ 0.015s] rustfs::connect_perf_drive real_local_write_read_is_measured_and_scratch_is_removed
PASS [ 0.007s] rustfs::connect_perf_drive runtime_timeout_and_full_drive_use_schema_valid_failed_reason
PASS [ 0.035s] rustfs::connect_perf_drive successful_result_has_signed_bounded_private_offline_export
────────────
Summary [ 21.087s] 10 tests run: 10 passed, 0 skipped
exit_code=0
Restored complete gate
$ cargo nextest run --profile ci -p rustfs --test connect_perf_drive --no-fail-fast
Nextest run ID 233b7307-5361-4eb7-94d5-b384f17d2d6a with nextest profile: ci
Starting 10 tests across 1 binary
PASS [ 0.009s] rustfs::connect_perf_drive cancellation_and_single_collector_leave_no_scratch
PASS [ 0.008s] rustfs::connect_perf_drive cleanup_failure_overrides_cancellation_and_remains_attributed
PASS [ 0.008s] rustfs::connect_perf_drive consent_and_resource_budgets_fail_before_scratch_io
PASS [ 0.008s] rustfs::connect_perf_drive invalid_roots_and_existing_scratch_fail_without_clobbering
PASS [ 19.518s] rustfs::connect_perf_drive production_cli_measures_and_signs_exact_binary_provenance
PASS [ 4.146s] rustfs::connect_perf_drive production_cli_rejects_missing_l1_consent_and_limits_before_artifacts
PASS [ 0.009s] rustfs::connect_perf_drive read_only_root_and_full_drive_errors_are_classified
PASS [ 0.018s] rustfs::connect_perf_drive real_local_write_read_is_measured_and_scratch_is_removed
PASS [ 0.009s] rustfs::connect_perf_drive runtime_timeout_and_full_drive_use_schema_valid_failed_reason
PASS [ 0.054s] rustfs::connect_perf_drive successful_result_has_signed_bounded_private_offline_export
────────────
Summary [ 23.787s] 10 tests run: 10 passed, 0 skipped
exit_code=0
Mutation M1: success_measurement reports max(actual elapsed, requested budget + 1), forcing 1001 ms for the one-second request. The retained duration assertion at rustfs/tests/connect_perf_drive.rs:105 fails.
$ cargo nextest run --profile ci -p rustfs --test connect_perf_drive --no-fail-fast -E 'test(real_local_write_read_is_measured_and_scratch_is_removed)'
info: experimental features enabled: setup-scripts
Compiling rustfs v1.0.1 (/private/tmp/gw-orchestrator-2734/ci-drive/rustfs)
warning: linker stderr: ld: __eh_frame section too large (max 16MB) to encode dwarf unwind offsets in compact unwind table, performance of exception handling might be affected
|
= note: `#[warn(linker_messages)]` on by default
warning: `rustfs` (bin "rustfs") generated 1 warning
warning: `rustfs` (bin "rustfs-cli") generated 1 warning (1 duplicate)
Finished `test` profile [unoptimized + debuginfo] target(s) in 7m 06s
────────────
Nextest run ID 9f618777-7bf6-4cbc-8803-dd7f50f449ed with nextest profile: ci
Starting 1 test across 1 binary (9 tests skipped)
FAIL [ 0.016s] rustfs::connect_perf_drive real_local_write_read_is_measured_and_scratch_is_removed
stdout ───
running 1 test
test real_local_write_read_is_measured_and_scratch_is_removed ... FAILED
failures:
failures:
real_local_write_read_is_measured_and_scratch_is_removed
test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 9 filtered out; finished in 0.01s
stderr ───
thread 'real_local_write_read_is_measured_and_scratch_is_removed' (106986846) panicked at rustfs/tests/connect_perf_drive.rs:105:5:
assertion failed: (1..=1_000).contains(&data.duration_millis)
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
────────────
Summary [ 0.017s] 1 test run: 0 passed, 1 failed, 9 skipped
FAIL [ 0.016s] rustfs::connect_perf_drive real_local_write_read_is_measured_and_scratch_is_removed
error: test run failed
exit_code=100
Mutation M2: target_parameters reports requested duration + 1. The in-process assertion at rustfs/tests/connect_perf_drive.rs:111 and CLI JSON assertion at rustfs/tests/connect_perf_drive.rs:527 both reject 1001 against 1000.
$ cargo nextest run --profile ci -p rustfs --test connect_perf_drive --no-fail-fast -E 'test(real_local_write_read_is_measured_and_scratch_is_removed) | test(production_cli_measures_and_signs_exact_binary_provenance)'
info: experimental features enabled: setup-scripts
Compiling rustfs v1.0.1 (/private/tmp/gw-orchestrator-2734/ci-drive/rustfs)
warning: linker stderr: ld: __eh_frame section too large (max 16MB) to encode dwarf unwind offsets in compact unwind table, performance of exception handling might be affected
|
= note: `#[warn(linker_messages)]` on by default
warning: `rustfs` (bin "rustfs") generated 1 warning
warning: `rustfs` (bin "rustfs-cli") generated 1 warning (1 duplicate)
Finished `test` profile [unoptimized + debuginfo] target(s) in 7m 20s
────────────
Nextest run ID b73d46ab-c910-4e1e-8063-7fc7e332ec15 with nextest profile: ci
Starting 2 tests across 1 binary (8 tests skipped)
FAIL [ 13.968s] rustfs::connect_perf_drive production_cli_measures_and_signs_exact_binary_provenance
stdout ───
running 1 test
test production_cli_measures_and_signs_exact_binary_provenance ... FAILED
failures:
failures:
production_cli_measures_and_signs_exact_binary_provenance
test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 9 filtered out; finished in 13.96s
stderr ───
thread 'production_cli_measures_and_signs_exact_binary_provenance' (107015462) panicked at rustfs/tests/connect_perf_drive.rs:527:5:
assertion `left == right` failed
left: Number(1001)
right: 1000
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
FAIL [ 0.015s] rustfs::connect_perf_drive real_local_write_read_is_measured_and_scratch_is_removed
stdout ───
running 1 test
test real_local_write_read_is_measured_and_scratch_is_removed ... FAILED
failures:
failures:
real_local_write_read_is_measured_and_scratch_is_removed
test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 9 filtered out; finished in 0.01s
stderr ───
thread 'real_local_write_read_is_measured_and_scratch_is_removed' (107016667) panicked at rustfs/tests/connect_perf_drive.rs:111:5:
assertion `left == right` failed
left: 1001
right: 1000
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
────────────
Summary [ 13.985s] 2 tests run: 0 passed, 2 failed, 8 skipped
FAIL [ 13.968s] rustfs::connect_perf_drive production_cli_measures_and_signs_exact_binary_provenance
FAIL [ 0.015s] rustfs::connect_perf_drive real_local_write_read_is_measured_and_scratch_is_removed
error: test run failed
exit_code=100
Transient production mutations were restored byte-for-byte before the complete green rerun. Original/restored SHA256: 8d389e6d8836c041aab8c6a93e56addd65ba3f70b33264e1c74f639d0572a2da.
$ git diff --check
exit_code=0
$ git diff --cached --check
exit_code=0
Independent orchestrator acceptance
Tested code tree: e67c72fb01
Tested tree: 16fbbd2ea4
shellcheck --version && actionlint -> exit 0
cargo fmt --all --check -> exit 0
./scripts/check_unsafe_code_allowances.sh -> exit 0
./scripts/check_layer_dependencies.sh -> exit 0
./scripts/check_architecture_migration_rules.sh -> exit 0
./scripts/check_logging_guardrails.sh -> exit 0
./scripts/check_error_other_format_ratchet.sh -> exit 0
./scripts/check_no_tokio_io_uring.sh -> exit 0
./scripts/check_extension_schema_boundaries.sh -> exit 0
./scripts/check_body_cache_whitelist.sh -> exit 0
./scripts/check_s3s_footprint.sh -> exit 0
./scripts/check_test_count_ratchet.sh -> exit 0
./scripts/check_fips_wording.sh -> exit 0
./scripts/check_embedded_secrets.sh -> exit 0
make script-tests -> exit 0
python3 ./scripts/test_object_generation_protocol_model.py -> exit 0
python3 ./scripts/check_test_wiring.py -> exit 0
./scripts/check_no_planning_docs.sh -> exit 0
./scripts/check_uring_lane_lib_only.sh -> exit 0
$ LANG=en_US.UTF-8 CARGO_BUILD_JOBS=1 CARGO_INCREMENTAL=0 cargo nextest run --profile ci -p rustfs --test connect_perf_drive --no-fail-fast
info: experimental features enabled: setup-scripts
warning: linker stderr: ld: __eh_frame section too large (max 16MB) to encode dwarf unwind offsets in compact unwind table, performance of exception handling might be affected
|
= note: `#[warn(linker_messages)]` on by default
warning: `rustfs` (bin "rustfs-cli") generated 1 warning
warning: `rustfs` (bin "rustfs") generated 1 warning (1 duplicate)
Finished `test` profile [unoptimized + debuginfo] target(s) in 0.69s
────────────
Nextest run ID cd1edf4c-f1c0-456d-ad1a-76cc00ce05c9 with nextest profile: ci
Starting 10 tests across 1 binary
PASS [ 0.012s] rustfs::connect_perf_drive cancellation_and_single_collector_leave_no_scratch
PASS [ 0.010s] rustfs::connect_perf_drive cleanup_failure_overrides_cancellation_and_remains_attributed
PASS [ 0.010s] rustfs::connect_perf_drive consent_and_resource_budgets_fail_before_scratch_io
PASS [ 0.009s] rustfs::connect_perf_drive invalid_roots_and_existing_scratch_fail_without_clobbering
PASS [ 20.328s] rustfs::connect_perf_drive production_cli_measures_and_signs_exact_binary_provenance
PASS [ 5.527s] rustfs::connect_perf_drive production_cli_rejects_missing_l1_consent_and_limits_before_artifacts
PASS [ 0.013s] rustfs::connect_perf_drive read_only_root_and_full_drive_errors_are_classified
PASS [ 0.019s] rustfs::connect_perf_drive real_local_write_read_is_measured_and_scratch_is_removed
PASS [ 0.011s] rustfs::connect_perf_drive runtime_timeout_and_full_drive_use_schema_valid_failed_reason
PASS [ 0.049s] rustfs::connect_perf_drive successful_result_has_signed_bounded_private_offline_export
────────────
Summary [ 25.991s] 10 tests run: 10 passed, 0 skipped
exit_code=0
All 54 tracked Cargo.toml files parse without duplicate keys. git diff --check passed. The production source remains unchanged; the two deliberate implementation mutations above were restored before this complete green run.
* fix(test): compare legacy body wire parity as raw bytes
Lossy UTF-8 comparison concealed a changed binary response byte.
Compare the already captured Vec<u8> values directly.
Validation:
- Binary mutant: migrated payload 0x84 vs legacy 0x83; old comparator
passed 1/1, raw comparator failed 0/1 (exit 100); restored green.
- Non-Swift body-drain mutant: untouched-body delegation test failed
with empty vs payload (exit 100); restored handler without a diff.
- cargo nextest run --profile ci -p rustfs-s3-types -p rustfs-protocols
--features rustfs-protocols/swift --no-fail-fast:
543 tests run, 543 passed, 0 skipped after both mutants restored.
- cargo fmt --all --check and git diff --check passed.
Orchestrator acceptance (LANG=en_US.UTF-8):
cargo nextest run --profile ci -p rustfs-s3-types -p rustfs-protocols --features rustfs-protocols/swift --no-fail-fast
Summary [2.445s] 543 tests run: 543 passed, 0 skipped.
All shared quick checks on the final code tree:
shellcheck --version && actionlint -> exit 0
cargo fmt --all --check -> exit 0
./scripts/check_unsafe_code_allowances.sh -> exit 0
./scripts/check_layer_dependencies.sh -> exit 0
./scripts/check_architecture_migration_rules.sh -> exit 0
./scripts/check_logging_guardrails.sh -> exit 0
./scripts/check_error_other_format_ratchet.sh -> exit 0
./scripts/check_no_tokio_io_uring.sh -> exit 0
./scripts/check_extension_schema_boundaries.sh -> exit 0
./scripts/check_body_cache_whitelist.sh -> exit 0
./scripts/check_s3s_footprint.sh -> exit 0
./scripts/check_test_count_ratchet.sh -> exit 0
./scripts/check_fips_wording.sh -> exit 0
./scripts/check_embedded_secrets.sh -> exit 0
make script-tests -> exit 0
python3 ./scripts/test_object_generation_protocol_model.py -> exit 0
python3 ./scripts/check_test_wiring.py -> exit 0
./scripts/check_no_planning_docs.sh -> exit 0
./scripts/check_uring_lane_lib_only.sh -> exit 0
Final independent acceptance after the drive-test CI repair
Parent: a99d8f4f89
Tested code tree: 93cbd96bf0
Tested tree: 075ae5161c2b62a96812bc1e3c904a7663b4f2f8
shellcheck --version && actionlint -> exit 0
cargo fmt --all --check -> exit 0
./scripts/check_unsafe_code_allowances.sh -> exit 0
./scripts/check_layer_dependencies.sh -> exit 0
./scripts/check_architecture_migration_rules.sh -> exit 0
./scripts/check_logging_guardrails.sh -> exit 0
./scripts/check_error_other_format_ratchet.sh -> exit 0
./scripts/check_no_tokio_io_uring.sh -> exit 0
./scripts/check_extension_schema_boundaries.sh -> exit 0
./scripts/check_body_cache_whitelist.sh -> exit 0
./scripts/check_s3s_footprint.sh -> exit 0
./scripts/check_test_count_ratchet.sh -> exit 0
./scripts/check_fips_wording.sh -> exit 0
./scripts/check_embedded_secrets.sh -> exit 0
make script-tests -> exit 0
python3 ./scripts/test_object_generation_protocol_model.py -> exit 0
python3 ./scripts/check_test_wiring.py -> exit 0
./scripts/check_no_planning_docs.sh -> exit 0
./scripts/check_uring_lane_lib_only.sh -> exit 0
$ LANG=en_US.UTF-8 CARGO_BUILD_JOBS=1 CARGO_INCREMENTAL=0 cargo nextest run --profile ci -p rustfs-s3-types -p rustfs-protocols --features rustfs-protocols/swift --no-fail-fast
Finished `test` profile [unoptimized + debuginfo] target(s) in 10m 08s
Nextest run ID 5c7ff102-ec6d-479a-86e0-810859b28bbf with nextest profile: ci
Starting 543 tests across 8 binaries
Summary [ 6.729s] 543 tests run: 543 passed, 0 skipped
exit_code=0
The only source change is the raw-byte wire comparator. Cargo manifests and the ratchet baselines remain unchanged from the independently validated parent. The invalid-byte and consumed-request mutations remain recorded above; their implementation changes were restored.
* fix: repair integration checks and IPC test readiness
Wait for the RPC/locks test sockets to pass the existing owner and mode
checks inside the original 5s readiness window and 10ms poll interval.
Use the existing admin S3 carrier for the new recovery and quota error
calls. Regenerate the test-count baseline for main's 118 existing tests
across seven crates. Pair the private pagination helper's raw-scan inputs
and remove the redundant Copy clone from the retirement event.
Keep the original assertions, responses, authorization, locks and storage
operations. The pagination body and raw-scan expressions at all five calls
are byte-identical; reinserting only the Copy clone restores the entire
base heal file. All ignored baseline values and s3s counts stay unchanged.
Validation base: c47432e353
Validation tree: f081084bc5
Patch SHA-256: 2dcf0028691bfb2772edfb38e262829b04f4151832dcce823c9dd9f7434b52de
Trace production-prefix SHA-256: fd994f218a21429d62866f4efb8eee1ee57b58f4a51865aa35d05a54ef1ccb45
All commands used LANG=en_US.UTF-8, CARGO_BUILD_JOBS=1,
CARGO_INCREMENTAL=0 and CARGO_NET_OFFLINE=true in one exclusive target.
Every gate retained the same tree and six source hashes before and after.
54 Cargo.toml files parse without duplicate keys.
cargo clippy -p rustfs --all-targets -- -D warnings
Finished `dev` profile [unoptimized + debuginfo] target(s) in 12m 50s
Exit: 0. Wall time: 770.855s.
The original 19 quick-check steps:
shellcheck --version && actionlint -> exit 0
cargo fmt --all --check -> exit 0
./scripts/check_unsafe_code_allowances.sh -> exit 0
./scripts/check_layer_dependencies.sh -> exit 0
./scripts/check_architecture_migration_rules.sh -> exit 0
./scripts/check_logging_guardrails.sh -> exit 0
./scripts/check_error_other_format_ratchet.sh -> exit 0
./scripts/check_no_tokio_io_uring.sh -> exit 0
./scripts/check_extension_schema_boundaries.sh -> exit 0
./scripts/check_body_cache_whitelist.sh -> exit 0
./scripts/check_s3s_footprint.sh -> exit 0
./scripts/check_test_count_ratchet.sh -> exit 0
./scripts/check_fips_wording.sh -> exit 0
./scripts/check_embedded_secrets.sh -> exit 0
make script-tests -> exit 0
python3 ./scripts/test_object_generation_protocol_model.py -> exit 0
python3 ./scripts/check_test_wiring.py -> exit 0
./scripts/check_no_planning_docs.sh -> exit 0
./scripts/check_uring_lane_lib_only.sh -> exit 0
Wall time: 286.337s.
cargo nextest run --profile ci -p rustfs --no-fail-fast
Summary [ 217.017s] 5382 tests run: 5382 passed (4 leaky), 40 skipped
Exit: 0. Wall time: 1324.414s.
cargo nextest run --profile ci -p rustfs-s3-types -p rustfs-protocols --features rustfs-protocols/swift --no-fail-fast
Summary [ 2.189s] 543 tests run: 543 passed, 0 skipped
Exit: 0. Wall time: 123.192s.
cargo nextest run --profile ci -p rustfs-ecstore list_objects_paginate --no-fail-fast
Summary [ 0.170s] 8 tests run: 8 passed, 5921 skipped
Exit: 0. Wall time: 261.324s.
cargo nextest run --profile ci -p rustfs-heal durable_mrf_anchor_retires_only_after_exact_bucket_generation_is_deleted --no-fail-fast
Summary [ 0.805s] 1 test run: 1 passed, 835 skipped
Exit: 0. Wall time: 199.801s.
Baseline regeneration:
./scripts/check_test_count_ratchet.sh --update
wrote .config/test-count-baseline.txt: 52 crates, 21340 tests, 164 ignored
Test-only increases: ecstore48, heal3, iam14, io-metrics2, obs3, policy14,
rustfs34. Existing ignored values and all other rows are byte-identical.
s3s footprint OK: files importing s3s is 195 (baseline files=195)
s3s footprint OK: s3s s3_error! lines is 1584 (baseline s3_error_lines=1584)
s3s footprint OK: ecstore files referencing s3s is 36 (baseline ecstore_files=36)
Historical runtime red controls on f622 (same retained two assertions):
cargo nextest run --profile ci -p rustfs --lib -E 'test(=connect::diagnostics::trace_runtime::tests::local_top_rpc_reads_service_http_completions_and_signs_offline) | test(=connect::diagnostics::trace_runtime::tests::local_top_locks_reads_service_manager_and_signs_offline)' --no-fail-fast
Original path readiness with a temporary 500ms bind/seal gap:
Summary [ 0.863s] 2 tests run: 0 passed, 2 failed, 5113 skipped
Exit: 100.
Sealed readiness in that same gap:
Summary [ 5.621s] 2 tests run: 2 passed, 5113 skipped
Exit: 0.
Compiled M1: replace the helper predicate with path.exists():
Summary [ 0.870s] 2 tests run: 0 passed, 2 failed, 5113 skipped
Exit: 100.
Restore the sealed predicate byte-for-byte:
Summary [ 5.640s] 2 tests run: 2 passed, 5113 skipped
Exit: 0.
Both reds observe mode755 with owner_matches=true and fail at the original
RPC/locks unwraps with StateSecurity. M1 failure locations: trace_runtime.rs
2547:29 and 2713:29. No new assertions were introduced.
cargo nextest run --profile ci -p rustfs --lib -E 'test(connect::diagnostics::trace_runtime::tests::)' --no-fail-fast
Summary [ 20.748s] 28 tests run: 28 passed (1 leaky), 5087 skipped
Exit: 0. This security sweep is historical; the full final unit gate above
covers the actual c474 candidate.
The first disk-stopped compile ran no tests and is excluded from red credit.
All temporary seal instrumentation is removed from the final source.
Final unit cleanup markers: 4 leaky, 40 original skips (exit0). The ld
__eh_frame linker warning also appears verbatim in the original f622 run
and the earlier c474 run; there are no new source warnings.
* fix: remove redundant pending closure in disk test
Pass the capture-free pending future factory as a function item in the existing slow-operation test. This fixes the redundant_closure error at disk_store.rs:2914 that stopped strict Clippy before the feature tests could run.
The single-line test edit preserves the factory's result, warning and timeout behavior. Reversing the replacement reproduces the parent file byte for byte. All 220 assertion lines and the production prefix are unchanged; no tests, assertions, guards, profiles or baselines were added or relaxed.
Original Linux CI evidence (run 37888552268 at 53245be9f):
- RIO job113685483992: strict Clippy exit101, redundant closure at disk_store.rs:2914; feature tests skipped.
- Workspace Test and Lint job113685484058: same strict Clippy exit101; nextest and doc tests skipped.
These are lint failures, not runtime mutation kills or known flakes.
Native macOS arm64 validation on tree7a5bf47e7533b29479bc85207416102f1337c566, LANG=en_US.UTF-8, one build job, CARGO_INCREMENTAL=0 and offline dependencies:
shellcheck --version && actionlint -> exit 0
cargo fmt --all --check -> exit 0
./scripts/check_unsafe_code_allowances.sh -> exit 0
./scripts/check_layer_dependencies.sh -> exit 0
./scripts/check_architecture_migration_rules.sh -> exit 0
./scripts/check_logging_guardrails.sh -> exit 0
./scripts/check_error_other_format_ratchet.sh -> exit 0
./scripts/check_no_tokio_io_uring.sh -> exit 0
./scripts/check_extension_schema_boundaries.sh -> exit 0
./scripts/check_body_cache_whitelist.sh -> exit 0
./scripts/check_s3s_footprint.sh -> exit 0
./scripts/check_test_count_ratchet.sh -> exit 0
./scripts/check_fips_wording.sh -> exit 0
./scripts/check_embedded_secrets.sh -> exit 0
make script-tests -> exit 0
python3 ./scripts/test_object_generation_protocol_model.py -> exit 0
python3 ./scripts/check_test_wiring.py -> exit 0
./scripts/check_no_planning_docs.sh -> exit 0
./scripts/check_uring_lane_lib_only.sh -> exit 0
All 19 checks: exit0, 312.161s.
$ cargo clippy -p rustfs -p rustfs-ecstore --all-targets --features rio-v2 -- -D warnings
Finished `dev` profile [unoptimized + debuginfo] target(s) in 20m 11s
exit0, 1213.210s total.
$ cargo nextest run --profile ci -p rustfs-ecstore --features rio-v2 slow_disk_operation_warns_once_while_pending_and_preserves_result --no-fail-fast
Finished `test` profile [unoptimized + debuginfo] target(s) in 4m 34s
PASS [0.026s] rustfs-ecstore disk::disk_store::tests::slow_disk_operation_warns_once_while_pending_and_preserves_result
Summary [ 0.170s] 1 test run: 1 passed, 5943 skipped
exit0, 281.920s total; 5943 existing filter skips.
warning: linker stderr: ld: __eh_frame section too large (max 16MB) to encode dwarf unwind offsets in compact unwind table, performance of exception handling might be affected
This linker warning is retained; flags, profiles and allowances are unchanged.
54 tracked Cargo.toml files parsed without duplicate keys. git diff --check passed. No new assertions were introduced, so no new runtime mutation is claimed. Native compilation and the existing helper test do not substitute for actual Linux RIO execution; the new integration CI run still needs to recover before the RPC merge.
Preservation and output hashes:
parent file SHA256: 8e59ee745e3aaaa22848b943d3b10f6353b515956282cde1f3af2c394200c9ac
final file SHA256: b69ba3237061b161448ae8b62b615885d5770b98a5b4a2479bcfbd70f1bd1f17
unchanged production prefix SHA256: 7903a6e288252c420231242d7b58a25fa2a7140ea3dd430cbe7f0dde11369dcb
RIO Linux failure log SHA256:421addb9868fd88e4109a922311e54f91f0dfd70862f8ba7fcaba193482ed90b
Workspace Linux failure log SHA256:e0aea8d0f014c022252a9534968e083c98cfc1fd760da445ecd04ec3f522e26a
19 output SHA256:35a54f28d707956ed1edb29d492c4dfe8a9fe753d103b111b5a2f1b5c41c2b98
RIO Clippy output SHA256:d6046e3868f2b3992fe11cdd04e9e98eb50a87d1234d9a70dbfeb154ff61638d
slow-operation output SHA256:605b7b258d2fd09c6ef2ab2ffc90d433d0509480f1e45125d20c823759e83608
Refs rustfs/backlog#2734.
* test(ecstore): isolate tier-free recovery fixture startup
The fixture constructor can take the production recovery lock and wait for 60 seconds before this test installs its queue and seeds metadata. Disable its background recovery only while creating this fixture, then restore the environment before calling the existing production recovery entrypoint.
Only the existing test setup changes. Its production entrypoint, default lock parameters, original assertions, 30-second timeout, cancellation and cleanup remain intact. No tests or assertions were added. Production and store/init.rs are byte-identical to the parent.
Independent acceptance ran on merge tree 8ec358cb42ba9ebb3bcfeafcef048a3f0366355b; the runner header reports parent HEAD, while before/after index and source hashes confirm this candidate. Worker task branch d8055766b8 has the same tree.
Every command used LANG=en_US.UTF-8. Task compilation also used CARGO_BUILD_JOBS=1, CARGO_INCREMENTAL=0, CARGO_NET_OFFLINE=true, an independent target, and empty RUSTC_WRAPPER/RUSTC_WORKSPACE_WRAPPER to keep compilers in its owned process group without changing repository configuration.
Command:
shellcheck --version && actionlint
Exit 0; observed output excerpt:
version: 0.11.0
Log SHA-256: fc9eec16b4191d058a295813ca9afbe9d2b37b59975da860170726be162a3924
Command:
cargo fmt --all --check
Exit 0; observed output excerpt:
(no stdout or stderr)
Log SHA-256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
Command:
./scripts/check_unsafe_code_allowances.sh
Exit 0; observed output excerpt:
(no stdout or stderr)
Log SHA-256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
Command:
./scripts/check_layer_dependencies.sh
Exit 0; observed output excerpt:
Layer dependency guard passed (no new reverse dependencies/cycles).
Log SHA-256: 69c6cb1896c0586b28d1167380d3260d99d8017a4265c8df5a2348f289e93a12
Command:
./scripts/check_architecture_migration_rules.sh
Exit 0; observed output excerpt:
Architecture migration rules passed.
Log SHA-256: 28790e70d070607ed4de0704f292a65030be92ec224ef106d4b72fd8296f896e
Command:
./scripts/check_logging_guardrails.sh
Exit 0; observed output excerpt:
✅ Logging guardrails check passed
Log SHA-256: 7adf41abff8ed8f7617decf86498888ddde78937099dd9ce47f6d359dfc2d317
Command:
./scripts/check_error_other_format_ratchet.sh
Exit 0; observed output excerpt:
error(format!) ratchet passed (total call sites: 579).
Log SHA-256: 66172aef18f150ba0605af122e26ffcf91f6dd86f43e38b05e2e982ce8d783ee
Command:
./scripts/check_no_tokio_io_uring.sh
Exit 0; observed output excerpt:
(no stdout or stderr)
Log SHA-256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
Command:
./scripts/check_extension_schema_boundaries.sh
Exit 0; observed output excerpt:
Extension schema boundaries passed.
Log SHA-256: f13d0acd0e2ce5d9815241b12b4dab7d01cc11b9b65bcf6dfb7812c878532a51
Command:
./scripts/check_body_cache_whitelist.sh
Exit 0; observed output excerpt:
Body-cache whitelist guard passed.
Log SHA-256: 6b828847b0383950ee6eef9cd846d0c322c945e47efd4c7d1ad881178810d6bb
Command:
./scripts/check_s3s_footprint.sh
Exit 0; observed output excerpt:
s3s footprint OK: files importing s3s is 195 (baseline files=195)
s3s footprint OK: s3s s3_error! lines is 1584 (baseline s3_error_lines=1584)
s3s footprint OK: ecstore files referencing s3s is 36 (baseline ecstore_files=36)
✅ s3s footprint baseline check passed
Log SHA-256: 47f53d04ef645618563fe1f22abc64667f19078f1974feedcca46e4f2c016844
Command:
./scripts/check_test_count_ratchet.sh
Exit 0; observed output excerpt:
test-count ratchet OK: 52 crates, 21340 tests, 164 ignored match .config/test-count-baseline.txt
Log SHA-256: 618a5e330e933de0c6ff484546df401457024a88bb5986d11d588794e33dab60
Command:
./scripts/check_fips_wording.sh
Exit 0; observed output excerpt:
FIPS wording guard passed (README.md and CHANGELOG.md contain no forbidden claims).
KMS wording guard passed (crates/kms claims no Vault KV2 Transit wrapping).
Log SHA-256: 99221cc9979c480ec4f46f50d76c52b18c72fe499f12e15e4f4a22a1b8bfb140
Command:
./scripts/check_embedded_secrets.sh
Exit 0; observed output excerpt:
Embedded secret guard passed (no private key material or provider credential literal in the tree).
Log SHA-256: 437a3ceba32094d2901889965ad191e411aad32cdb18bf0b8659e818011c406b
Command:
make script-tests
Exit 0; observed output excerpt:
test_cli_requires_the_repository_default_branch (__main__.SelfTests.test_cli_requires_the_repository_default_branch) ... ok
test_config_rejects_duplicate_and_invalid_entries (__main__.SelfTests.test_config_rejects_duplicate_and_invalid_entries) ... ok
test_freshness_boundaries (__main__.SelfTests.test_freshness_boundaries) ... ok
test_grace_requires_two_successful_queries_with_no_history (__main__.SelfTests.test_grace_requires_two_successful_queries_with_no_history) ... ok
test_invalid_api_evidence_fails_closed (__main__.SelfTests.test_invalid_api_evidence_fails_closed) ... ok
test_report_retains_every_workflow (__main__.SelfTests.test_report_retains_every_workflow) ... ok
test_requests_filter_schedule_default_branch_and_success_on_server (__main__.SelfTests.test_requests_filter_schedule_default_branch_and_success_on_server) ... ok
Ran 10 tests in 0.042s
OK
Ran 30 tests in 20.627s
OK
Ran 10 tests in 2.268s
OK
Ran 33 tests in 0.127s
OK
Ran 7 tests in 0.002s
OK
Ran 7 tests in 0.001s
OK
Ran 4 tests in 0.108s
OK
Ran 3 tests in 0.564s
OK
PASS: cold stampede script self-test
Log SHA-256: 75fda2d069a373c63667b908baaf99e6591aab9099a739b0c4c22ceb6da8d68c
Command:
python3 ./scripts/test_object_generation_protocol_model.py
Exit 0; observed output excerpt:
test_an_unavailable_decision_quorum_cannot_choose_a_candidate (__main__.ObjectGenerationProtocolModel.test_an_unavailable_decision_quorum_cannot_choose_a_candidate) ... ok
test_counter_allocation_requires_persistence_and_is_consumed_once (__main__.ObjectGenerationProtocolModel.test_counter_allocation_requires_persistence_and_is_consumed_once) ... ok
test_counter_restart_burns_persisted_allocations_before_or_after_issue (__main__.ObjectGenerationProtocolModel.test_counter_restart_burns_persisted_allocations_before_or_after_issue) ... ok
test_crash_after_promise_before_reply_does_not_restore_an_older_ballot (__main__.ObjectGenerationProtocolModel.test_crash_after_promise_before_reply_does_not_restore_an_older_ballot) ... ok
test_duplicate_or_unconfigured_voters_cannot_form_a_quorum (__main__.ObjectGenerationProtocolModel.test_duplicate_or_unconfigured_voters_cannot_form_a_quorum) ... ok
test_empty_or_unobserved_minority_slot_can_retry_without_allocating_another_revision (__main__.ObjectGenerationProtocolModel.test_empty_or_unobserved_minority_slot_can_retry_without_allocating_another_revision) ... ok
test_equal_ballot_cannot_accept_different_bytes_and_lower_ballot_is_rejected (__main__.ObjectGenerationProtocolModel.test_equal_ballot_cannot_accept_different_bytes_and_lower_ballot_is_rejected) ... ok
test_forgetting_tombstone_authority_allows_an_aba_create (__main__.ObjectGenerationProtocolModel.test_forgetting_tombstone_authority_allows_an_aba_create) ... ok
test_issuing_before_persistence_reuses_a_ballot_after_restart (__main__.ObjectGenerationProtocolModel.test_issuing_before_persistence_reuses_a_ballot_after_restart) ... ok
test_lost_ack_and_restart_preserve_the_chosen_operation (__main__.ObjectGenerationProtocolModel.test_lost_ack_and_restart_preserve_the_chosen_operation) ... ok
test_minority_accepted_work_is_adopted_and_finished (__main__.ObjectGenerationProtocolModel.test_minority_accepted_work_is_adopted_and_finished) ... ok
test_omitting_highest_accepted_adoption_has_a_double_choice_counterexample (__main__.ObjectGenerationProtocolModel.test_omitting_highest_accepted_adoption_has_a_double_choice_counterexample) ... ok
test_quorum_choice_does_not_require_an_uncontacted_disk_to_learn_it (__main__.ObjectGenerationProtocolModel.test_quorum_choice_does_not_require_an_uncontacted_disk_to_learn_it) ... ok
test_recovery_failure_or_stale_expected_head_does_not_allocate_a_successor (__main__.ObjectGenerationProtocolModel.test_recovery_failure_or_stale_expected_head_does_not_allocate_a_successor) ... ok
test_replacing_voters_with_empty_state_can_choose_a_conflicting_value (__main__.ObjectGenerationProtocolModel.test_replacing_voters_with_empty_state_can_choose_a_conflicting_value) ... ok
test_skipping_predecessor_recovery_chooses_a_broken_revision_chain (__main__.ObjectGenerationProtocolModel.test_skipping_predecessor_recovery_chooses_a_broken_revision_chain) ... ok
test_stale_publication_rollback_and_gc_cannot_change_the_winner (__main__.ObjectGenerationProtocolModel.test_stale_publication_rollback_and_gc_cannot_change_the_winner) ... ok
test_superseded_operation_identity_cannot_be_reused_for_different_input (__main__.ObjectGenerationProtocolModel.test_superseded_operation_identity_cannot_be_reused_for_different_input) ... ok
test_superseded_operation_retry_returns_its_original_successor (__main__.ObjectGenerationProtocolModel.test_superseded_operation_retry_returns_its_original_successor) ... ok
test_tombstone_and_null_marker_payloads_do_not_reset_authority_revision (__main__.ObjectGenerationProtocolModel.test_tombstone_and_null_marker_payloads_do_not_reset_authority_revision) ... ok
test_tombstone_rejects_a_delayed_never_present_create (__main__.ObjectGenerationProtocolModel.test_tombstone_rejects_a_delayed_never_present_create) ... ok
test_unlearned_predecessor_is_recovered_before_the_second_slot (__main__.ObjectGenerationProtocolModel.test_unlearned_predecessor_is_recovered_before_the_second_slot) ... ok
Ran 24 tests in 0.929s
OK
Log SHA-256: 0b4be926efb0609d497ae58930340153f01957b606206cbbdb3e0d47eb29d3a2
Command:
python3 ./scripts/check_test_wiring.py
Exit 0; observed output excerpt:
OK: e2e modules, runner selection, fuzz matrices, profiles, scheduled alerts, and Quick Checks are wired
Log SHA-256: 21f676f458451525535ab0ef73d6a9c0d9774e16d76a4d19e396ad34936dd39c
Command:
./scripts/check_no_planning_docs.sh
Exit 0; observed output excerpt:
(no stdout or stderr)
Log SHA-256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
Command:
./scripts/check_uring_lane_lib_only.sh
Exit 0; observed output excerpt:
OK: no 'uring_'-matching test functions under crates/ecstore/tests; --lib is safe
Log SHA-256: 43f3f851e27ec2ab9e34d6b05fefdede38af992699c473904bfe6f53a46d52e3
Command:
cargo nextest run --profile ci -p rustfs-ecstore --lib --features rio-v2 -E 'test(=bucket::lifecycle::bucket_lifecycle_ops::tests::tier_free_version_recovery_production_entrypoint_enqueues_seeded_item)' --no-fail-fast
Exit 0; wall 234.399s.
Observed output:
Finished `test` profile [unoptimized + debuginfo] target(s) in 3m 49s
Starting 1 test across 1 binary (5871 tests skipped)
SETUP PASS [ 0.006s] ecstore-base-stack: sh -c 'echo RUST_MIN_STACK=4194304 >> "$NEXTEST_ENV"'
PASS [ 0.440s] rustfs-ecstore bucket::lifecycle::bucket_lifecycle_ops::tests::tier_free_version_recovery_production_entrypoint_enqueues_seeded_item
Summary [ 0.586s] 1 test run: 1 passed, 5871 skipped
Log SHA-256: 34e973f86e8dc2102c1a0e483e0ac802ce7df3d1b40a603932f62c0805c4de27
Command:
cargo nextest run --profile ci -p rustfs-ecstore --lib --features rio-v2 -E 'test(=bucket::lifecycle::bucket_lifecycle_ops::tests::tier_free_version_recovery_eight_coordinators_hold_one_leader_and_take_over) | test(=store::bucket::tests::creation_commit_backfill_runs_under_the_config_write_fence) | test(=store::bucket::tests::orphaned_bucket_recovery_requires_every_volume_absent_and_is_retryable)' --no-fail-fast
Exit 0; wall 4.010s.
Observed output:
Finished `test` profile [unoptimized + debuginfo] target(s) in 0.43s
Starting 3 tests across 1 binary (5869 tests skipped)
SETUP PASS [ 0.005s] ecstore-base-stack: sh -c 'echo RUST_MIN_STACK=4194304 >> "$NEXTEST_ENV"'
PASS [ 0.068s] rustfs-ecstore bucket::lifecycle::bucket_lifecycle_ops::tests::tier_free_version_recovery_eight_coordinators_hold_one_leader_and_take_over
PASS [ 0.637s] rustfs-ecstore store::bucket::tests::creation_commit_backfill_runs_under_the_config_write_fence
PASS [ 0.846s] rustfs-ecstore store::bucket::tests::orphaned_bucket_recovery_requires_every_volume_absent_and_is_retryable
Summary [ 0.987s] 3 tests run: 3 passed, 5869 skipped
Log SHA-256: b5da2a927ff18ca9aa217762f7186a6d4371bd9e8feedd7b77940e10b64dffac
Command:
cargo clippy -p rustfs -p rustfs-ecstore --all-targets --features rio-v2 -- -D warnings
Exit 0; wall 1084.508s.
Observed output:
Finished `dev` profile [unoptimized + debuginfo] target(s) in 18m 03s
Log SHA-256: 28f98be7970faaa71cd118da1da4e6e2e3d23b9dda7e6449eaa7a37e741260b5
Compiled mutation command (unchanged for each red and restored run):
cargo nextest run --profile ci -p rustfs-ecstore --lib --features rio-v2 -E 'test(=bucket::lifecycle::bucket_lifecycle_ops::tests::tier_free_version_recovery_production_entrypoint_enqueues_seeded_item)' --no-fail-fast
M1: reverse only this test-local constructor flag from false to true. The mutant compiled, then failed the original recovered-item queue assertion after 30.481s. Restoring the entire observed candidate passed in 0.460s. Red log 55bf3b1535529ac0bcfb8071f9b56f9f40631007a85c315367e7e93786d65293; restored log 204d5fbe4574a7917f5e740a795d864fe28a7e7fb5e33d27b0a9cf68e2f17f52.
M2: replace only the actual awaited enqueue with false, consuming its original inputs. The mutant compiled, then failed the same original queue assertion after 30.516s. Restoring the entire observed candidate passed in 0.457s. Red log 89186330413215e231e2b06bf013cc31097ea8f4a106d91421a36d6c8986a7fd; restored log e848b42d2302401afa5dbcd9e37bf76d2e56a96ab6de5a488513b997f53c5e9b.
All temporary observation and mutation code was removed before final acceptance. Historical baseline and diagnostic failures remain recorded; they are not mutation credit. The separate Object Lock intent-retry diagnostic still fails, and these native macOS checks do not establish Linux CI recovery.
* refactor(rpc): serve internode rpc bodies with the owned Body (#8441)
* test(rpc): pin the wire bytes of every internode rpc response body
rustfs/backlog#2747 batch 4, step 1 of 2, on the s3s bodies the rpc paths
serve today. Each response body builder in storage/rpc/http_service.rs is
served through hyper's HTTP/1.1 server and the raw bytes the peer reads are
compared with goldens captured from this commit: status, content-length or
chunked framing, chunk boundaries, the terminating chunk, HEAD, and the cut
connection a failure after data produces. The capability responses carry a
per-process proof, so their length header and payload are checked against
the body they encode instead.
The read_file_stream response builder moves into read_file_stream_response
so the test reaches the same code the handler runs; behaviour is unchanged.
* refactor(rpc): serve internode rpc bodies with the owned Body
rustfs/backlog#2747 batch 4, step 2 of 2. The internode rpc HTTP paths
build their responses with rustfs_s3_types::Body: Body::from_stream
replaces Body::from(StreamingBlob::wrap(..)) at the six stream sites, and
the in-memory and empty bodies keep their constructors. InternodeRpcService
converts its responses into the wrapped service's response body through
From<Body>, so the server keeps composing it with the s3s service as
before. The wire-byte goldens from the previous commit pass unchanged;
storage/rpc no longer names s3s.
* fix(rpc): simplify wire test response builder type
Name the boxed response builder used by the capability wire test so
all-target Clippy can check it without a type-complexity warning.
Validation (LANG=en_US.UTF-8, CARGO_INCREMENTAL=0):
- cargo clippy -p rustfs --all-targets -- -D warnings: exit 0.
- cargo nextest run --profile ci -p rustfs --lib -E 'test(/storage::rpc::/)': 279 tests run, 279 passed, 4842 skipped.
- All 19 quick-check commands, including script-tests, both Python scripts and cargo fmt --all --check: exit 0.
- Footprint baseline regenerated: files=194, s3_error_lines=1584, ecstore_files=36; test-count baseline regenerated: 52 crates, 21228 tests, 164 ignored.
- All 54 Cargo.toml files parse without duplicate keys.
No assertion was added or changed by this lint fix. R1-R8 are retained
in the existing PR verification record.
* fix(ecstore): resume fenced Object Lock creation intents
* fix(ecstore): bound creation retry futures on the stack
* fix(ecstore): require native proof for creation retries
* fix(ecstore): bound bucket creation stack usage (#8466)
* fix(ecstore): retain creation fences through rollback
Keep lifecycle, metadata-transaction, and namespace guards around rollback and metadata cleanup. Preserve the original publication error when a guard is lost. Build the timestamp retry fixture through the real interrupted producer so it checks a valid pending creation.
Validated source tree: 4e8e207999.
Two independent source reviews found no blocking regression.
Validation (LANG=en_US.UTF-8):
- bash -c 'shellcheck --version && actionlint' — exit 0
- cargo fmt --all --check — exit 0
- ./scripts/check_unsafe_code_allowances.sh — exit 0
- ./scripts/check_layer_dependencies.sh — exit 0
- ./scripts/check_architecture_migration_rules.sh — exit 0
- ./scripts/check_logging_guardrails.sh — exit 0
- ./scripts/check_error_other_format_ratchet.sh — exit 0
- ./scripts/check_no_tokio_io_uring.sh — exit 0
- ./scripts/check_extension_schema_boundaries.sh — exit 0
- ./scripts/check_body_cache_whitelist.sh — exit 0
- ./scripts/check_s3s_footprint.sh — exit 0
- ./scripts/check_test_count_ratchet.sh — exit 0
- ./scripts/check_fips_wording.sh — exit 0
- ./scripts/check_embedded_secrets.sh — exit 0
- make script-tests — exit 0
- python3 ./scripts/test_object_generation_protocol_model.py — exit 0
- python3 ./scripts/check_test_wiring.py — exit 0
- ./scripts/check_no_planning_docs.sh — exit 0
- ./scripts/check_uring_lane_lib_only.sh — exit 0
- cargo nextest run --profile ci -p rustfs-ecstore --lib --locked --no-fail-fast — 5894 passed, 0 skipped
- cargo check -p rustfs-ecstore --all-targets --locked — exit 0
- cargo nextest run --profile ci -p rustfs --lib --locked --no-fail-fast -E 'test(=admin::handlers::table_catalog::tests::cold_strong_registration_completes_while_other_bucket_backup_waits) | test(=admin::handlers::table_catalog::tests::commit_backend_borrowed_put_preserves_owner_authorization_and_precondition) | test(=admin::handlers::table_catalog::tests::disable_table_bucket_preserves_objects_and_can_be_reenabled) | test(=admin::handlers::table_catalog::tests::load_table_handler_negotiates_vended_credentials_without_breaking_metadata_only_callers) | test(=admin::handlers::table_catalog::tests::namespace_property_handler_updates_object_backed_catalog_and_maps_errors) | test(=app::object::get::tests::execute_get_object_resumes_from_relocated_pool_without_splicing_body) | test(=table_catalog::store::tests::catalog_writer_loss_prevents_recovery_storage_commit)' — 7 passed
- cargo nextest run --profile ci -p rustfs --lib --locked --no-fail-fast -E 'test(unsupported_remote_disk_)' — 2 passed
Additional fault-injection evidence remains incomplete. Timestamp M1 reached its original assertion; its restore run is being collected. M2–M5 and the three planned rollback faults have not run. An automated risk check interrupted the timestamp proof task; no assertions or validation settings were weakened.
* fix(ecstore): box nested bucket rollback state
Keep the guarded rollback future behind a heap allocation so bucket creation does not store its nested state inline. The rollback body, all three fence references and pending-proof revocation order are unchanged.
The same three original Linux tests overflowed on both e611 CI attempts. Local format, ecstore test compilation and all 19 quick-check steps pass; the original Linux feature lanes still need to validate this candidate.
* test(ecstore): isolate rollback guard loss fixtures
Run the three rollback metadata-cleanup cases in separate test processes so reused /proc/self/fd aliases from earlier cases cannot invalidate their creation witnesses. Keep the original fixture, guards, assertions and io_uring test selection, and verify that each child runs the intended case successfully.
Update the generated test-count baseline for the added macro arm.
Validation: cargo check -p rustfs-ecstore --tests; cargo fmt --all --check; all 19 quick-check commands, including make script-tests and the two Python checks. Linux io_uring acceptance and fault controls remain pending.
* refactor(replication): move the replication config onto gateway types (#8443)
* wip: T1.4 replication family
* refactor(replication): move the replication config onto gateway types
The replication configuration family leaves s3s: rustfs-replication no longer depends on s3s, and ECStore bucket metadata, the scanner and the rustfs app/admin layers hold rustfs_gateway_types::persistence::PersistedReplicationConfiguration. The persisted XML goes through the gateway persistence codec behind a new BucketConfigXml trait (the 39 byte goldens pass with unchanged fixtures); the legacy s3s edge converts once in storage/ecfs.rs through storage/s3_compat/replication.rs; the scanner data usage cache keeps its MessagePack bytes through a serde mirror pinned by bytes recorded from s3s 5761ddfe. rustfs-gateway-types 0.32.0 is pinned once (rev 772d3385). Refs rustfs/backlog#2745.
* test: use S3 serializer for list response tests
* fix(ci): restore integration test compilation
Gate the diagnostic test import with its feature, initialize the conditional
MetadataWriteStore fields explicitly, and decode the scanner replication fixture
through its existing cache boundary. Preserve the original fixture and assertions.
Validation was run on tree 544acb80c0 at 6003d4b8.
LANG=en_US.UTF-8; no source changes follow validation.
Quick checks: 19/19 passed.
- shellcheck --version && actionlint: exit 0
- cargo fmt --all --check: exit 0
- ./scripts/check_unsafe_code_allowances.sh: exit 0
- ./scripts/check_layer_dependencies.sh: exit 0
- ./scripts/check_architecture_migration_rules.sh: exit 0
- ./scripts/check_logging_guardrails.sh: exit 0
- ./scripts/check_error_other_format_ratchet.sh: exit 0
- ./scripts/check_no_tokio_io_uring.sh: exit 0
- ./scripts/check_extension_schema_boundaries.sh: exit 0
- ./scripts/check_body_cache_whitelist.sh: exit 0
- ./scripts/check_s3s_footprint.sh: exit 0
- ./scripts/check_test_count_ratchet.sh: exit 0
- ./scripts/check_fips_wording.sh: exit 0
- ./scripts/check_embedded_secrets.sh: exit 0
- make script-tests: exit 0
- python3 ./scripts/test_object_generation_protocol_model.py: exit 0
- python3 ./scripts/check_test_wiring.py: exit 0
- ./scripts/check_no_planning_docs.sh: exit 0
- ./scripts/check_uring_lane_lib_only.sh: exit 0
Darwin validation:
- cargo clippy --locked -p rustfs-ecstore -p rustfs-scanner --all-targets -- -D warnings: exit 0
- cargo nextest run --locked -p rustfs-ecstore -p rustfs-scanner --lib -E 'test(config::com::tests::save_config_preserves_retryable_system_volume_errors) | test(scanner_io::tests::replication_maintenance_refreshes_compacted_stats_across_cycles_and_retries)' --test-threads 1 --no-fail-fast: exit 0, 2/2 passed
- cargo clippy --locked -p rustfs-ecstore -p rustfs-scanner --all-targets --features rustfs-ecstore/diagnostic-read-version-caller,rustfs-scanner/diagnostic-read-version-caller -- -D warnings: exit 101
- cargo nextest run --locked -p rustfs-ecstore -p rustfs-scanner --lib --features rustfs-ecstore/diagnostic-read-version-caller,rustfs-scanner/diagnostic-read-version-caller -E 'test(config::com::tests::save_config_preserves_retryable_system_volume_errors) | test(scanner_io::tests::replication_maintenance_refreshes_compacted_stats_across_cycles_and_retries) | test(cluster::rpc::runtime_sources::tests::) | test(config::com::tests::diagnostic_config_write_retains_origin_and_api_for_results_and_cancellation)' --test-threads 1 --no-fail-fast: exit 0, 14/14 passed
Diagnostic strict Clippy remains red on four unchanged production lints:
runtime_sources.rs if_same_then_else and three redundant_locals reports in
set_disk/core/metadata_read.rs and set_disk/read.rs. Their source ranges and
feature wiring match the base; the base was not compiled separately.
Linux aarch64 validation (Rust/Clippy 1.99.0, UID 501, jobs=1):
- cargo clippy --all-targets -- -D warnings: exit 0; Finished dev profile in 68m 12s
- cargo clippy -p rustfs -p rustfs-ecstore --all-targets --features rio-v2 -- -D warnings: exit 0; Finished dev profile in 49m 45s
Source HEAD/tree and clean status matched before and after both commands.
No compiler or stack overrides were used. protoc 35.1 was installed from its
verified ARM64 release; flatc was absent and not invoked by the workspace build scripts.
The local environment differs from hosted CI.
* fix(ecstore): recover delete batches with missing versions
Treat an explicit non-null version as a no-op when it is absent from both recovery sources. Keep rejecting missing null versions, delete-marker mismatches and conflicting local generations.
Add returning-disk and partial-receipt regressions, extend journal boundary checks, and regenerate the ecstore test-count baseline. Production faults exercise the 24 added assertion sites and 7 semantic checks: 33 faults failed at the expected sites and passed after full restoration.
Validation (LANG=en_US.UTF-8, one build job):
cargo nextest run --locked -p rustfs-ecstore --lib -E 'test(delete_recovery) | test(quorum_delete_cleanup_coordinator_binds_single_and_batch) | test(crash_delete_retry_batch_reclaims_inline_and_reserved_external)' --test-threads 1 --no-fail-fast
Summary [ 34.580s] 69 tests run: 69 passed, 6058 skipped; exit 0
cargo clippy --locked -p rustfs-ecstore --all-targets -- -D warnings
Finished `dev` profile [unoptimized + debuginfo] target(s) in 5m 37s; exit 0
All 19 final Quick Checks completed with exit 0:
- shellcheck --version && actionlint
- cargo fmt --all --check
- ./scripts/check_unsafe_code_allowances.sh
- ./scripts/check_layer_dependencies.sh
- ./scripts/check_architecture_migration_rules.sh
- ./scripts/check_logging_guardrails.sh
- ./scripts/check_error_other_format_ratchet.sh
- ./scripts/check_no_tokio_io_uring.sh
- ./scripts/check_extension_schema_boundaries.sh
- ./scripts/check_body_cache_whitelist.sh
- ./scripts/check_s3s_footprint.sh
- ./scripts/check_test_count_ratchet.sh
- ./scripts/check_fips_wording.sh
- ./scripts/check_embedded_secrets.sh
- make script-tests
- python3 ./scripts/test_object_generation_protocol_model.py
- python3 ./scripts/check_test_wiring.py
- ./scripts/check_no_planning_docs.sh
- ./scripts/check_uring_lane_lib_only.sh
test-count ratchet OK: 52 crates, 22009 tests, 165 ignored match .config/test-count-baseline.txt
PASS: cold stampede script self-test
model: states=51023, transitions=167520, terminal=732, chosen_candidates=2
multi-slot round schedules: 640
OK: e2e modules, runner selection, fuzz matrices, profiles, scheduled alerts, and Quick Checks are wired
* fix(ecstore): retry tier commit intent lock contention
Retry transient namespace lock timeouts within the existing three-attempt
CAS budget. Reload and verify the original mutation identity after
contention, and preserve the final typed error when attempts are exhausted.
Cover save/read contention, exhausted retries, replacement identities and
CAS reconciliation without changing the existing timeout or lock policy.
Validation:
- cargo nextest run --locked -p rustfs-ecstore --features test-util --lib -E 'test(tier_mutation)' --test-threads 1 --no-fail-fast: 52 passed.
- cargo nextest run --locked -p rustfs-ecstore --features test-util --lib -E 'test(intent_advance_) | test(=services::tier::tier::tests::caller_cancellation_after_durable_prepare_does_not_hide_the_mutation)' --test-threads 1 --no-fail-fast: 5 passed.
- cargo clippy --locked -p rustfs-ecstore --all-targets -- -D warnings: passed.
- cargo clippy --locked -p rustfs-ecstore --all-targets --features test-util -- -D warnings: passed.
- 27 mutation pairs failed at the intended assertions and passed after restoring the production code.
- bash scripts/check_test_count_ratchet.sh --update, then bash scripts/check_test_count_ratchet.sh: 52 crates, 22013 tests, 165 ignored; ecstore increases from 6581 to 6585.
* refactor(lifecycle): replace s3s domain types with Gateway DTOs
Keep lifecycle rules, restore requests, Object Lock, and scanner cached tags in domain types. Convert at the existing HTTP boundary while preserving legacy XML, scanner cache bytes, date ordering, and tombstones.
Linux test validation before the final import-path-only adjustment:
cargo nextest run -p rustfs-ecstore bucket_config_goldens
39 passed; 6211 skipped by selector.
cargo nextest run --lib -p rustfs -E 'test(/test_zero_rule_lifecycle_tombstone_recognition/) | test(/test_local_lifecycle_staleness_axis_selection/) | test(/test_lifecycle_staleness_preserves_legacy_submillisecond_axis/) | test(/test_merge_incoming_lifecycle/) | test(/test_lifecycle_expiry_statement_matrix/) | test(/test_lifecycle_expiry_subset_xml/) | test(/put_bucket_lifecycle_validation_errors_keep_their_s3_code/) | test(/execute_put_bucket_lifecycle_configuration_rejects_missing_configuration/) | test(/execute_restore_object_rejects_missing_restore_request/) | test(/execute_restore_object_rejects_select_type/) | test(/execute_restore_object_maps_failures_to_typed_s3_errors/) | test(/execute_restore_object_returns_internal_error_when_store_uninitialized/) | test(/restore_request_conversion_preserves_legacy_output_values/) | test(/restore_request_conversion_preserves_presence_and_raw_enums/) | test(/restore_request_conversion_distinguishes_absent_and_empty_containers/)'
25 passed; 5318 skipped by selector.
cargo nextest run --profile ci -p rustfs-ecstore -p rustfs-lifecycle -p rustfs-replication -p rustfs-scanner
7872 passed; 21 skipped under the existing CI profile.
Previously verified mutation controls: 39 production faults, 41 RED/GREEN pairs.
cargo clippy --locked -p rustfs -p rustfs-ecstore -p rustfs-lifecycle -p rustfs-scanner --all-targets -- -D warnings
Passed in 40m 45s before the import-path adjustment, then in 1m 59s on
the final source with warnings denied. The final adjustment imports the
existing helper through the object domain facade; fmt and the architecture
check also passed.
Part of rustfs/backlog#2745. The remaining ecstore configuration families follow separately.
* refactor(admin): move error types to rustfs-s3-types (#8440)
* refactor(protocols): move error types to rustfs-s3-types by codemod
T1.2 batch 3 (rustfs/backlog#2743): s3select-api, s3select-query and
the error items of protocols move off s3s by scripts/codemods, run with
--hold crates/protocols/src/sftp/errors.rs. That file stays on s3s: it
downcasts the backend error, whose runtime type is still s3s::S3Error.
- rustfs/src/protocols/client.rs: the two webdav session methods now
return rustfs_s3_types::S3Result and convert the s3s error with From.
- codemod: a child module that calls s3_error! through `use super::*`
gets its own `use rustfs_s3_types::s3_error;` once its parent imports
that macro by name; otherwise it is listed as GLOB.
- s3-types: a table test pins code, status and the s3s error the legacy
edge renders for every variant a moved call site names.
- s3_error_lines 1585 -> 1584 (a lifecycle comment reworded).
* refactor(admin): move error types to rustfs-s3-types (WIP, no build)
T1.2 batch 4 (rustfs/backlog#2743), work in progress: rustfs check
still fails with 16 errors (listed in the PR body).
- codemod: --facade PATH treats a crate-local re-export of s3s's error
items like s3s for the four names (9 new self-tests, 9 mutants red).
- codemod output for rustfs/src/admin and rustfs/src/site_replication
(--facade crate::storage_api::site_replication::s3); 6 glob children
get an explicit rustfs_s3_types::s3_error import.
- admin/router.rs: the Operation trait returns rustfs_s3_types::S3Result.
- error.rs: From<ApiError> builds the rustfs-s3-types error; the s3s
conversion is derived from it.
* fix(admin): convert error types at the s3s boundary
Batch 4 of T1.2 left the admin and site_replication modules returning
rustfs_s3_types::S3Error while their callers and callees on the storage
and auth side still speak s3s::S3Error. Convert at each edge with the
existing From impls in rustfs-s3-types (compat_s3s.rs) instead of
changing any code, message or status:
- admin/auth.rs, admin/handlers/object_zip_download.rs,
admin/handlers/table_catalog/mod.rs: map the s3s result returned by
check_key_valid / authorize_request / authorize_internal_object_request
into the admin error type with .map_err(Into::into).
- storage/ecfs.rs: the s3s-facing side maps the admin site_replication
gate results the other way.
- admin/handlers/pools.rs: ApiError.code stays s3s::S3ErrorCode, so the
rustfs code is converted with .into() when the struct is built, in
production and in the two tests that build or compare it.
- admin/handlers/heal.rs, admin/runtime_sources.rs: the router returns
s3s errors (S3Route impl); the expected rustfs code is converted with
.into() before the comparison.
- admin/handlers/table_catalog/mod.rs: iceberg_rest_error takes
&'static str so the Cow<'static, str> custom code borrows it; every
caller already passes an ICEBERG_ERROR_* constant or a &'static str.
- admin/handlers/quota.rs: drop a now same-type .into() after s3_error!.
- storage_api.rs, site_replication/mod.rs: drop re-exports and an import
that no longer have users (clippy -D warnings).
cargo check -p rustfs --all-targets and cargo clippy -p rustfs
--all-targets -- -D warnings are clean; the s3s footprint baseline is
unchanged at files=172, s3_error_lines=303, ecstore_files=28.
Refs rustfs/backlog#2743, rustfs/backlog#2734.
* refactor(ecstore): move bucket configs onto gateway persisted types (#8709)
* refactor(ecstore): move bucket configs onto gateway persisted types
Finish the ECStore batch of T1.4 (rustfs/backlog#2745). The remaining
eleven bucket configuration families (Website, CORS, Tagging, Versioning,
ObjectLock, Encryption, Notification, Logging, Accelerate, RequestPayment,
PublicAccessBlock) now read and write their stored XML through the
rustfs-gateway-types persistence and cors_tagging codecs instead of the
s3s codec, so the legacy s3s bucket-config codec and its
s3gate-metadata-xml compat marker are removed.
- ecstore: BucketConfigXml is implemented by the gateway codec for every
family; metadata, metadata_sys, versioning, object lock, replication and
lifecycle consumers take the persisted types. Region moves to
rustfs_s3_types::Region, and the ETag conversion to s3s moves to the
rustfs edge with its tests. Only warm_backend.rs and warm_backend_s3.rs
still reference s3s, held for T1.5 (rustfs/backlog#2746).
- scanner: object lock cache, item actions and data usage use the
persisted types.
- rustfs: the legacy S3 edge converts each family between s3s DTOs and the
persisted types in storage/storage_api.rs by writing one codec's XML and
reading it with the other; Tagging converts field by field.
- Bump the rustfs-gateway-types pin to 21283e87.
The 39 persisted-byte goldens are unchanged and pass, so a document the
old codec wrote reads back and re-encodes to the same bytes. s3s footprint:
files 181 -> 146, ecstore_files 28 -> 2, s3_error_lines unchanged.
* test(app): prepare the PutBucketEncryption config from the request
The uninitialized-store test passed an empty persisted configuration, which the use case now validates before it reaches the store check, so it answered MalformedXML instead of InternalError. Convert the request's configuration the way the S3 edge does.
* style(app): format the PutBucketEncryption test
* refactor(s3-client): move remote error codes and DTO enums off s3s (#8783)
Part of T1.5 (rustfs/backlog#2746), batch 1.
* refactor(s3select): run queries on gateway DTO request shapes (#8794)
* refactor(s3select): run queries on an engine request built from gateway DTO shapes
Part of T1.5 (rustfs/backlog#2746), batch 2.
* fix(s3select): keep the select request conversion in the app layer
The layer guard forbids app -> storage; the conversion moves from storage/s3_compat/select.rs to app/select_request.rs unchanged.
* fix(heal): retry transient metadata conflicts (#8812)
* fix(heal): retry transient metadata conflicts
* chore: update heal test count baseline
* test: reserve nextest capacity for progress probes (#8818)
* test(e2e): use a non-default region for the differential negative control
Since main's region-aware CreateBucket and GetBucketLocation (#8739, #8743), us-east-1 answers an empty LocationConstraint and 200 for an owned re-create, so the control no longer reported its own region. Also refresh the e2e-full linux selection from the CI listing (682 tests).
* refactor(protocols): move StorageBackend to gateway types (#8825)
* refactor(protocols): move storage backend to gateway types
* test: update e2e test count baseline
* fix: preserve copy source keys and unknown lengths
* test: update protocols test count baseline
* fix(protocols): preserve typed SFTP backend errors
* test(protocols): use gateway list input in policy tests
* fix(protocols): clear clippy failures after T1.5 merge (#8831)
fix(protocols): clear clippy failures after merge
* fix(ecstore): finalize restore metadata after local copy-back (#8836)
* fix(ecstore): finalize restore metadata after local copy-back
* test(ecstore): ratchet added restore finalization cases
* fix(ecstore): keep restore copyback compatible with older peers
* fix(ecstore): gate restore copyback on fleet compatibility
* fix(ecstore): retain transition identity during restore copyback
* fix(ecstore): route committed restore copyback locally
* fix restore copyback cleanup
* fix(ecstore): honor restore probe test fallback
* refactor(storage): migrate option errors to domain types (#8839)
* refactor(s3): migrate private multipart validation errors (#8844)
* refactor(auth): derive policy conditions from AuthzFacts (#8442)
* refactor(auth): extract authorization facts
* test(e2e): update smoke selection baseline
* test(e2e): update full selection baseline
* refactor(admin): migrate health and prefix usage to owned requests (#8887)
* refactor(admin): add owned health router boundary
* refactor(admin): migrate prefix usage to owned request
* refactor(admin): complete the owned admin request boundary (#8949)
* refactor(admin): complete the owned admin request boundary
Every type implementing the owned admin `Operation` now serves the legacy
router through one blanket impl that converts the s3s request and response
at that single edge, so handlers register with `AdminOperation(&Handler {})`
and need no adapter statics; `LegacyAdminAdapter` is removed.
`AdminRequest` gains the accessors handlers read today (`remote_addr`,
`client_info`, `request_context`, `req_info`, `req_info_mut`,
`install_req_info`, `envelope_mut`, `take_body`), the request helpers gain
envelope forms (`authenticate_envelope_credentials`,
`object_store_from_envelope`, the table-catalog `*_from_envelope` helpers,
`KmsAdminAudit::from_envelope`, `AccountAuditContext::from_envelope`,
`CallerIdentity::resolve_envelope`), `read_compatible_admin_body` accepts
either body type, and the owned `Body` gains `store_all_limited` and
`Default`. `admin::operation::test_support::TestRequest` builds owned
requests for handler tests. Unused accessors carry a per-method dead-code
expectation that each migrating handler batch removes.
No handler behavior changes; the remaining migrations follow in batches.
* chore(admin): add the T1.12 admin handler codemod
The script rewrites an admin handler file from the s3s request and response
types to the owned boundary (imports, signatures, request field and helper
calls, body access, response construction, and test request literals) and
leaves a TODO comment at every site it cannot rewrite mechanically. It is
idempotent and is removed once T1.12 completes.
* refactor(admin): migrate replication and KMS handlers to owned requests (#8951)
Move the T1.12 batch 2 admin handlers (site_replication, kms_keys, event,
replication, diagnostics, on_demand_migration, audit, account, kms_rekey,
kms_key_lifecycle, object_data_cache, tls_debug, integrity) from the s3s
request/response types to the owned AdminRequest/AdminResponse boundary.
Handler logic, authorization order, error codes and response bytes are
unchanged; the request/response plumbing now goes through the envelope
accessors and helpers introduced by the owned boundary.
The shared response helpers (utils::json_response, utils::empty_response,
handlers::admin_json_response, target_descriptor::build_json_response,
profile::authorize_profile_request) and the site replication repair
executor still answer with the s3s types, so the migrated files carry
owned-boundary forms of those helpers and convert the repair response at
its call edge. The streaming console-log body and the two devnull drains
move from the s3s byte stream to the owned Body stream/frame API.
Drop the dead_code expectations for the owned accessors and envelope
helpers this batch starts using, and lower the s3s footprint baseline to
the measured 99 files.
Part of rustfs/backlog#2788 (rustfs/backlog#2734 T1.12).
* refactor(admin): add owned forms of the shared admin response helpers
Handler batches were copying `json_response`, `empty_response`,
`admin_json_response`, and `build_json_response` into each migrated file
because the shared helpers still return the s3s response. Provide owned
forms next to each legacy helper (`owned_json_response`,
`owned_empty_response`, `owned_admin_json_response`,
`owned_build_json_response`, `authorize_profile_envelope`) with parity
tests against the legacy shape, derive `Debug` for `AdminResponse` so
tests can use `expect_err`, and fix three codemod rules: strip owned
string conversions in test credentials, annotate non-literal URI parses,
and leave a rebound request parameter immutable.
* refactor(admin): use the shared owned response helpers in batch 2 (#8954)
Replace the per-file owned-boundary copies of json_response,
empty_response, admin_json_response, build_json_response and
authorize_profile_request in the batch 2 admin handlers with the shared
owned_* helpers and authorize_profile_envelope that the boundary branch
now provides, and restore expect_err in the handler tests now that
AdminResponse implements Debug; the `.err().expect(..)` stand-ins were
failing the base branch's clippy step (clippy::err_expect under
-D warnings). The site replication repair call edge
keeps its conversion because the repair executor still answers with the
s3s response type.
Drop the dead_code expectations on the shared owned helpers this batch
now uses in non-test code.
Part of rustfs/backlog#2788 (rustfs/backlog#2734 T1.12).
* fix(admin): preserve legacy body limit error responses
* refactor(admin): move policy, ILM and batch 3 handlers to owned requests (#8956)
Batch 3 of T1.12 (rustfs/backlog#2788, parent rustfs/backlog#2734) moves
14 admin handler files to AdminRequest/AdminResponse and the owned
Operation trait: ilm_transition, policies, idp_compat, config_admin,
plugins_instances, profile_admin, group, heal, object_zip_download,
kms_key_metadata, durability, inspect_archive, sts and
gateway_key_inventory. Handler logic, authorization order, error codes
and response bytes are unchanged; test request literals use
test_support::TestRequest.
The handlers use the shared owned helpers (owned_json_response,
owned_admin_json_response, owned_build_json_response,
authorize_profile_envelope). The dead_code expectations on the
AdminRequest accessors and RequestEnvelope::uri_mut they now exercise
are removed. The legacy target_descriptor::build_json_response keeps
only its equivalence test caller and carries the matching expectation
until it retires.
object_zip_download builds the s3s request the S3 gate still reads from
the owned request facts; heal authenticates through
authenticate_envelope_credentials; the trace, inspect-archive and zip
bodies use Body::from_stream. extract_plugin_instance_filters takes the
URI so extensions.rs keeps calling it, and the two idp_compat
delegations stay on the legacy trait until their delegates migrate.
The s3s footprint baseline drops from 99 to 90 files.
* test(admin): anchor the replication diff body contract on the call
The source-contract test looked for `let body = read_compatible_admin_body(`
on one line; rustfmt wrapped that binding after the handler moved to the
owned request boundary, so the marker was no longer found. Anchor on the
call itself so formatting cannot change what the contract checks.
* refactor(admin): migrate table catalog and batch 1 to owned requests (#8955)
* fix(admin): preserve catalog body limit response bytes
* refactor(admin): migrate identity and cluster handlers to owned requests (#8966)
* refactor(admin): migrate identity and cluster handlers to owned requests
Move the T1.12 batch 4 admin handlers (user, service_account, oidc, tier,
system, pools, quota, kms_dynamic, kms_backup, rebalance, module_switch,
bucket_meta, cluster_snapshot, realtime, bucket_recovery, profile) from the
s3s request/response types to the owned AdminRequest/AdminResponse
boundary. All 90 impl Operation blocks now take AdminRequest and return
AdminResponse; handler logic, authorization order, error codes and response
bytes are unchanged. The handlers use the shared owned helpers
(owned_json_response, owned_admin_json_response, authorize_profile_envelope,
the envelope-form runtime sources and audit constructors) instead of local
copies.
profile.rs drops the s3s forms of authorize_profile_request and
profile_not_implemented_response now that profile_admin.rs is on the
envelope gate too; profile_not_implemented_response answers with an
AdminResponse. The realtime metrics stream moves from the s3s ByteStream
adapter to Body::from_stream, and the rebalance idle-404 document still
renders through s3s::S3Error::to_http_response with its body converted at
the edge.
The idp_compat delegators ImportIamV2 and AddServiceAccountLdap move to the
owned trait as well: with their targets on both traits, the method-call
delegation became ambiguous between the owned impl and the router blanket
impl, so they now take the owned request directly.
KmsAdminAudit::from_request loses its last production caller with this batch
and is deleted; its comparison test now asserts the envelope facts directly.
Lower the s3s footprint baseline to the measured 68 files.
Part of rustfs/backlog#2788 (rustfs/backlog#2734 T1.12).
* refactor(admin): remove the legacy admin helpers batch 4 superseded
Batch 4 moved the last production callers of the s3s-typed admin helpers
to their owned forms, which left `admin_json_response`, `json_response`,
`authenticate_request`, `authorize_admin_request`, `object_store_from_req`
and `object_store_from_extensions` unused, along with the `Body`,
`S3Request` and `S3Response` re-exports of the admin `s3` facade. Remove
them. The owned forms keep their documentation; the behavior tests of the
removed `json_response` now exercise `owned_json_response`, and the
compat-prefix test asserts the owned response directly.
Two tests existed only to compare a removed legacy helper with its owned
twin (`owned_json_response_matches_the_legacy_shape`) or to cover the
removed legacy gate (`authorize_admin_request_without_credentials_is_rejected`);
they go with the helpers, and the owned equivalents stay. The rustfs
test-count row drops from 5957 to 5955 and the s3s footprint from 68 to 66
files.
* refactor(admin): drop the legacy target descriptor JSON response helper (#8968)
`target_descriptor::build_json_response` returned the s3s response and
had no production caller left once the event, audit and plugin instance
handlers moved to `owned_build_json_response`; it survived only for the
legacy/owned equivalence test. Remove it, give the owned helper its own
documentation, and make the test assert the owned response directly:
status, `application/json`, the echoed `x-request-id` when present and
only then, and the body bytes. The file no longer imports s3s, so the
footprint baseline drops from 66 to 65 files.
* test(ecstore): pin native history on cancellation under the cancellable queue
main moved the local write path onto the disk blocking queue, which skips an
operation once its waiter is dropped, so a borrowed write cancelled while it is
queued no longer creates its file. The integration-only native publication
test still expected the queued open to run. Assert the new execution model
instead (the open never runs) while keeping its safety claims: the Started
record is taken before queueing and the cancelled publication never regrants
native history.
* chore(gateway): raise test-count rows for tests merged from main
* test(s3select): use the owned JSON input types in merged planner tests
* fix(ecstore): scope the merged direct-writer test hook to test builds
* fix(ecstore): create parents before admitted truncate opens
Align the heal tests with removal-disabled preservation and local-only outcome
fields, and refresh the e2e-full selection for the synced tests.
* chore(gateway): update test-count baseline after main sync
Main #9241 removed the node pool-metadata readiness cache with 8 tests
and added 6, so the rustfs row drops from 6440 to 6438. The ecstore and
s3select rows rise for tests added by #9237, #9239 and #9241.
* feat(scripts): add junit_diff.py for dual-stack JUnit comparison (#9275)
* ci(gateway): make the test-count ratchet a floor, not exact (#9278)
ci(gateway): make the test-count ratchet a floor instead of an exact baseline
Tests below a row still fail; tests above it, a member without a row and ignores below the row now pass with a note. Ignores above a row still fail. Adds contract tests wired into make script-tests.
* chore(gateway): record ignored microbenchmarks merged from main (#9290)
chore(gateway): record the ignored microbenchmarks merged from main
main added three throughput microbenchmarks that are ignored by design
(#9217): one each in rustfs-checksums, rustfs-rio and rustfs-utils. The
test-count ratchet treats a higher ignored count as a failure until the
row is raised, so raise those three rows to 1 and tighten the test rows
to the merged tree with --update.
* ci(s3-tests): add reference parity lane against pinned 1.0.5 (#9288)
* test(app): add per-operation test seams for oracle use-case calls
* chore(codemods): add lifecycle transition test split codemod
* test(app): split lifecycle transition tests by operation owner
* test: call oracle use-cases through test seams in cross-owner tests
* ci: guard cross-owner tests to call use-cases through test seams
---------
Co-authored-by: rustfs-cla-assistant[bot] <273121464+rustfs-cla-assistant[bot]@users.noreply.github.com>
* fix(ci): run the mint lane on ubuntu-latest, which has a Docker daemon
* fix(docker): let BuildKit populate TARGETARCH for the rc download
* fix(docker): derive the rc architecture when TARGETARCH is not set
* test(e2e): expect ServiceUnavailable for degraded ListParts after #9149
PR #9149 made ServiceUnavailable the default API code for storage
read-quorum failures and limited SlowDownRead to GET/HEAD object reads.
ListParts maps its storage error through ApiError::from, so a degraded
ListParts now answers 503 ServiceUnavailable. The two distributed
acceptance cases still expected the previous SlowDownRead code and have
failed every nightly e2e-distributed run since 2026-11-05.
The 503 status and the fail-closed (non-empty-list) assertions are
unchanged; only the error code tracks the intended contract.
* style(e2e): apply rustfmt to the ListParts error-code assertion
* feat(server): add the RUSTFS_S3_STACK selector and dual-stack e2e lane (#8411)
* refactor(server): group the s3s-compat tower layers under legacy_compat (#8410)
Move the ten tower layers that patch s3s behaviour out of
rustfs/src/server/layer.rs into rustfs/src/server/legacy_compat/, one file
per layer with its tests. Each file header names the s3s behaviour it
patches, the gateway mechanism that replaces it (Replaced by:), and the task
that removes it (Removed by: T2.12, rustfs/backlog#2771). The layers keep
their code, their order in external_service_stack!, and their behaviour; no
layer is deleted.
Add server::http::tests::stack_census, which renders the ordered service
chain of the real external_service_stack! expansion and compares it with a
committed constant, so a reorder, insertion or removal is a visible diff.
Route the two s3s error types the moved layers use through a
storage_api::server::legacy_compat shim so the split adds no s3s-importing
file, and lower the s3s footprint baselines to the measured values (files 209
to 208, s3_error! lines 1586 to 1585). Keep the server runtime-source bypass
guard covering the moved code.
Refs rustfs/backlog#2787, rustfs/backlog#2734
* feat(s3-types): add the RustFS-owned S3 error type and s3_error! macro (#8406)
`rustfs-s3-types` now provides `S3ErrorCode` (generated by build.rs from
src/codes.txt, the grep-derived call surface, plus `Custom`), `S3Error`,
`S3Result`, `StdError` and the `s3_error` macro with the same call syntax
as s3s, so the T1.2 codemod is a `use` swap. Statuses are hand-written and
pinned to the s3s behaviour RustFS ships today by an oracle test that uses
s3s as a dev-dependency only. The `From` conversions for the legacy edge
live behind the off-by-default `compat-s3s` feature (the orphan rule
forbids them in rustfs/src/error.rs); `rustfs` enables it.
Refs rustfs/backlog#2742, rustfs/backlog#2734
* ci: add the per-crate test-count ratchet (#8404)
Quick Checks now fails any PR that lowers the number of test attributes
in a workspace crate or adds #[ignore], measured per crate against the
exact rows of .config/test-count-baseline.txt (rustfs/backlog#2734
section 1 rule 3). The counter widens the spec's literal
#[(tokio::)?test] to the tokio attributes that carry arguments and both
spellings of #[test_case(...)], and anchors at line start so
commented-out attributes do not count. The initial baseline covers 52
crates, 21077 tests and 164 ignored.
Refs rustfs/backlog#2738
Refs rustfs/backlog#2734
* test(ecstore): pin the persisted bucket-config XML bytes for 13 families (#8408)
Before any DTO or codec is replaced, pin the exact bytes that
deserialize followed by serialize produces for every persisted
bucket-configuration XML family in its three source shapes (AWS
documentation, MinIO-written, RustFS-written), and the bytes RustFS
writes today for a typed value. Byte equality only: a structural
comparison would let a codec replacement change the persisted encoding
unnoticed.
The generator script refuses to overwrite existing goldens without
--force, refuses on dirty codec paths, cannot report success from its
write mode, and records the generating commit in the fixtures README.
Refs rustfs/backlog#2744, rustfs/backlog#2734
* ci: raise the test-count baseline to the integration tree (#8426)
* ci: make the s3s footprint baseline exact and lower-only (#8403)
* ci: make the s3s footprint baseline exact and lower-only
The s3s footprint guard compared three counters against ceilings held in
the script itself, so a ceiling could sit above the tree and a drift was
fixable by raising it (rustfs/rustfs#5964 raised the s3_error! ceiling
to go green). The three values now live in
.config/s3s-footprint-baseline.txt as files, s3_error_lines and
ecstore_files, the script reads only that file, and a measurement that
differs from its baseline in either direction fails: growth prints
"s3s footprint grew: <measured> > <baseline>", shrinkage prints
"lower the baseline to <measured> in this PR". The counting method is
unchanged; the baseline is the current measurement of
gateway/integration (209 / 1585 / 36). The header records the end state:
once all three reach 0, T4.2 switches the script to failing on any
occurrence.
Refs rustfs/backlog#2739, rustfs/backlog#2734
* ci: add the s3s edge allowlist mode to the footprint guard (#8405)
`scripts/check_s3s_footprint.sh --mode allowlist` fails on every Rust
file that references s3s (`s3s::` or `s3s_sigv4`) and is not matched by
an entry of the new `.config/s3s-edge-allowlist.txt`; `--dry-run` lists
those files and exits 0. The allowlist carries the initial edge from the
spec, and the script rejects an entry that names or matches anything
under crates/ other than crates/e2e_test. The baseline mode and the
argument-less CI invocation are unchanged.
Refs rustfs/backlog#2741, rustfs/backlog#2734
* test(ecstore): reach the bucket-config codec through the test facade (#8428)
* refactor(s3-types): own the HTTP Body and drop s3s from three crates (#8431)
Add rustfs_s3_types::Body, a BoxBody<Bytes, StdError> wrapper with
empty() and From<Bytes | Vec<u8> | String | &'static str>, and the
legacy From conversions to and from s3s::Body behind compat-s3s.
The signer is now generic over the request body: it only reads and
writes the request head, so it needs no body type and its callers keep
theirs. s3-client sends its requests with the owned Body, madmin signs a
bodiless head, and targets drops its two S3Request/S3Response header
wrappers in favour of extract_params_header.
rustfs-signer, rustfs-madmin and rustfs-targets no longer depend on
s3s; the s3s footprint baseline drops from 209 to 202 files.
Refs rustfs/backlog#2747, rustfs/backlog#2734
* refactor(ecstore): move S3ErrorCode to rustfs-s3-types by codemod (#8432)
* test(ecstore): pin the InvalidRange responses of new_getobjectreader
Both error branches of new_getobjectreader build an ErrorResponse with S3ErrorCode::InvalidRange, and no test observed either code. Pin the code, its wire name and the location fields of each branch before the code type moves to rustfs-s3-types (rustfs/backlog#2743), so a changed code on either branch fails a test. The ecstore test-count row rises by the two tests.
* chore(scripts): add the s3s-to-rustfs-s3-types error codemod
Task T1.2 of rustfs/backlog#2734 (rustfs/backlog#2743) moves S3Error, S3ErrorCode, S3Result and s3_error! from s3s to rustfs-s3-types by script, one crate closure per PR. The codemod rewrites use trees (splitting mixed s3s imports) and qualified code paths, never comments or literals, adds the rustfs-s3-types dependency, drops s3s only from a crate that no longer names it, refuses every file of the s3s edge allowlist, takes --hold for a file whose type is fixed outside the closure, and prints a per-crate census. A second run changes nothing. Its self-test runs under make script-tests.
* refactor(ecstore): take S3ErrorCode from rustfs-s3-types
Output of scripts/codemods/s3_error_to_s3_types.py over crates/ecstore, crates/lifecycle, crates/scanner and crates/replication (rustfs/backlog#2743 batches 1 and 2). The ErrorResponse code field, the two InvalidRange constructions in new_getobjectreader and an unused tier import now use rustfs_s3_types::S3ErrorCode; the variant names and wire names are unchanged. lifecycle, scanner and replication reference none of the four error names. warm_backend.rs and warm_backend_s3.rs are held: they compare against rustfs_s3_client::ErrorResponse.code, which stays the s3s type until s3-client moves.
* ci(footprint): count only s3_error! lines still on s3s's macro
s3_error_lines counted every line holding the text `s3_error!`, which
also matches the RustFS-owned `rustfs_s3_types::s3_error!`. A codemod
that moves call sites onto rustfs-s3-types therefore could not lower it
(rustfs/backlog#2743, decision 3; rustfs/rustfs#8432, gotcha 2).
The counter now applies a textual, per-file and per-line rule that errs
on the side of counting (full rule in the script header):
- occurrences spelled `rustfs_s3_types::s3_error!` never count;
- a line holding `s3s::s3_error!` always counts;
- any other `s3_error!` line counts unless its file is cleared, that is,
it imports the macro from rustfs_s3_types by name on a line starting
with the use item (or defines it with `macro_rules! s3_error`) and does
not also bring s3s's macro in (`use s3s::s3_error`, a `use s3s::{..}`
tree naming it or a crate-root `*`, `use s3s::*`, or
`#[macro_use] extern crate s3s`);
- fail closed: a file matching neither form (`use super::*`, a
crate-local re-export, a renamed or glob rustfs_s3_types import, a
commented-out import, a comment-only mention) counts.
Census of the 1585 lines on 96fc06776: 1478 in 64 files that import the
macro from s3s; 107 in 15 files matching neither form (105 through
`use super::*`, 1 qualified `s3s::s3_error!`, 1 comment line); none in
files that import it from rustfs_s3_types or define it. Every line stays
counted, so the baseline remains s3_error_lines=1585 (files=202 and
ecstore_files=36 unchanged). Rewriting rustfs/src/auth.rs's import to
rustfs_s3_types lowers the new counter to 1567; the old one stayed 1585.
`--self-test` (wired into `make script-tests`) copies the script into
22 fixture trees: 6 positive cases and 16 negative ones (lines that must
not count, forms that must not clear a file, and a missing baseline,
missing key or failing classification search that must fail). Each tree
has a control line that must count, and each run gets s3s text on stdin.
Seven cases were red against the old counter. 19 mutants each turn their
case red: old definition, flipped import test, dropped fail-closed
branch, dropped e2e_test exclusion, dropped s3s veto, dropped qualified
s3s rule, dropped rustfs-qualified strip, unanchored rustfs import,
renamed import accepted, nested glob taken as root glob, single-line
s3s trees only, dropped s3s glob forms, dropped macro_use form, dropped
definition clearing, ignored classification rg errors, error-line search
without its path, classification searches without their path, missing
baseline skipped, missing key skipped.
* test(e2e): compare two RUSTFS_S3_STACK servers over every S3 operation (#8435)
Add dual_stack_differential to e2e_test. It starts two rustfs processes
from the same binary, stack A on RUSTFS_S3_STACK=legacy and stack B on
RUSTFS_E2E_STACK_B (default legacy until rustfs/backlog#2734 T2.5),
sends both the same aws-sdk-s3 script covering the 73 methods of
impl S3 for FS (a normal request and a 4xx negative each), and compares
status, error code, an allowlist of response headers and a few typed
body fields. Error message text is never compared. Each run writes
<target>/dual-stack/<run>.json with compared_ops and the (op, field,
a, b) differences; any difference fails the test.
differential_detects_an_injected_difference starts B with
RUSTFS_REGION=eu-west-1 and requires GetBucketLocation to be reported.
The comparator is pure and unit-tested without a server.
Wire the module into the e2e-smoke filter and update the e2e-smoke and
e2e-full selection digests and the e2e_test test-count row.
Refs rustfs/backlog#2740
* feat(app): add the RustFS-owned RequestEnvelope and its s3s constructor (#8437)
* feat(app): add the RustFS-owned RequestEnvelope and its s3s constructor
* ci: classify the s3s edge adapters in storage/s3_compat as interface
* feat(s3-types): add Body::from_stream and move Swift off s3s::Body (#8438)
rustfs/backlog#2747 batch 3. `Body::from_stream` wraps a chunk stream of
unknown length over `StreamBody`/`BoxBody`: one data frame per chunk, the
first error ends the body, the size hint stays unknown, and dropping the
body drops the stream. Parity tests pin it to the s3s `StreamingBlob::wrap`
it replaces, at the `http_body` level and byte for byte through hyper.
The Swift handlers in rustfs-protocols build their responses as the owned
`Body`. `SwiftService` converts them into the wrapped S3 service's response
body type through `From<Body>`, so rustfs keeps serving `s3s::Body` at the
legacy edge unchanged. Request bodies stay Send-only (`UnsyncBoxBody`); the
two storage writers and the DLO/SLO segment streams get `Sync` from
`sync_wrapper::SyncStream`, which is what s3s's `http_body_unsync` mutex
provided before. The DTO and error uses of s3s in protocols are untouched
(T1.5 and T1.2).
* refactor(protocols): move error types to rustfs-s3-types by codemod (#8439)
T1.2 batch 3 (rustfs/backlog#2743): s3select-api, s3select-query and
the error items of protocols move off s3s by scripts/codemods, run with
--hold crates/protocols/src/sftp/errors.rs. That file stays on s3s: it
downcasts the backend error, whose runtime type is still s3s::S3Error.
- rustfs/src/protocols/client.rs: the two webdav session methods now
return rustfs_s3_types::S3Result and convert the s3s error with From.
- codemod: a child module that calls s3_error! through `use super::*`
gets its own `use rustfs_s3_types::s3_error;` once its parent imports
that macro by name; otherwise it is listed as GLOB.
- s3-types: a table test pins code, status and the s3s error the legacy
edge renders for every variant a moved call site names.
- s3_error_lines 1585 -> 1584 (a lifecycle comment reworded).
* test(connect): restore one-second drive request budgets
Apply the exact test-only repair already landed on main in 5972a09cc3. The filesystem probe and production CLI success test request one second and keep their existing one-second duration and parameter checks. All outcome, I/O, error, cleanup and resource-limit assertions remain.
Scope: rustfs/tests/connect_perf_drive.rs only. Patch-id: cb0a1dfc8f68a67727525adea7458e2cee7d09f8.
Validation environment: LANG=en_US.UTF-8, CARGO_BUILD_JOBS=1, CARGO_INCREMENTAL=0, exclusive APFS-cloned CARGO_TARGET_DIR.
Quick Checks: every command from .github/actions/quick-checks/action.yml, with captured output and exit status below.
$ shellcheck --version && actionlint
ShellCheck - shell script analysis tool
version: 0.11.0
license: GNU General Public License, version 3
website: https://www.shellcheck.net
exit_code=0
$ cargo fmt --all --check
exit_code=0
$ ./scripts/check_unsafe_code_allowances.sh
exit_code=0
$ ./scripts/check_layer_dependencies.sh
Layer dependency guard passed (no new reverse dependencies/cycles).
exit_code=0
$ ./scripts/check_architecture_migration_rules.sh
Architecture migration rules passed.
exit_code=0
$ ./scripts/check_logging_guardrails.sh
✅ Logging guardrails check passed
exit_code=0
$ ./scripts/check_error_other_format_ratchet.sh
error(format!) ratchet passed (total call sites: 582).
exit_code=0
$ ./scripts/check_no_tokio_io_uring.sh
exit_code=0
$ ./scripts/check_extension_schema_boundaries.sh
Extension schema boundaries passed.
exit_code=0
$ ./scripts/check_body_cache_whitelist.sh
Body-cache whitelist guard passed.
exit_code=0
$ ./scripts/check_s3s_footprint.sh
s3s footprint OK: files importing s3s is 195 (baseline files=195)
s3s footprint OK: s3s s3_error! lines is 1584 (baseline s3_error_lines=1584)
s3s footprint OK: ecstore files referencing s3s is 36 (baseline ecstore_files=36)
✅ s3s footprint baseline check passed
exit_code=0
$ ./scripts/check_test_count_ratchet.sh
test-count ratchet OK: 52 crates, 21222 tests, 164 ignored match .config/test-count-baseline.txt
exit_code=0
$ ./scripts/check_fips_wording.sh
FIPS wording guard passed (README.md and CHANGELOG.md contain no forbidden claims).
KMS wording guard passed (crates/kms claims no Vault KV2 Transit wrapping).
exit_code=0
$ ./scripts/check_embedded_secrets.sh
Embedded secret guard passed (no private key material or provider credential literal in the tree).
exit_code=0
$ make script-tests
Running script tests...
entrypoint credential policy tests passed
internode grpc A/B bench env tests passed
object batch benchmark enhanced tests passed
PASS package service script contract
hotpath warp A/B gate tests passed
hotpath warp ABBA tests passed
Scanner validation artifacts written to /var/folders/0w/s4wxsn2n4w9b6f3cd4xrz53w0000gn/T/tmp.1pFe6Kl1Yu/out
Scanner validation artifacts written to /var/folders/0w/s4wxsn2n4w9b6f3cd4xrz53w0000gn/T/tmp.1pFe6Kl1Yu/out-missing-pid
.............................
----------------------------------------------------------------------
Ran 29 tests in 181.286s
OK
{"decision":"verified","pending_gates":["G01","G03","G04","G05","G06","G07","G08","G09","G10","G11","G12","G13","G14","P1","P2","P3","P4","R-D","R-E","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"G02"}
{"decision":"verified","pending_gates":["G01","G02","G03","G04","G05","G06","G07","G08","G09","G10","G11","G12","G13","G14","P1","P2","P3","P4","R-D","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"R-E"}
{"decision":"verified","pending_gates":["G01","G03","G04","G05","G06","G07","G08","G09","G10","G11","G12","G13","G14","P1","P2","P3","P4","R-D","R-E","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"G02"}
{"decision":"verified","pending_gates":["G01","G02","G03","G04","G05","G06","G07","G08","G09","G10","G11","G12","G13","G14","P1","P2","P3","P4","R-D","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"R-E"}
{"decision":"verified","pending_gates":["G02","G03","G04","G05","G06","G07","G08","G09","G10","G11","G12","G13","G14","P1","P2","P3","P4","R-D","R-E","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"G01"}
{"decision":"verified","pending_gates":["G01","G02","G04","G05","G06","G07","G08","G09","G10","G11","G12","G13","G14","P1","P2","P3","P4","R-D","R-E","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"G03"}
{"decision":"verified","pending_gates":["G01","G02","G03","G04","G05","G06","G07","G08","G09","G10","G11","G12","G13","G14","P1","P2","P3","P4","R-D","R-E"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"R-L"}
{"decision":"verified","pending_gates":["G01","G02","G03","G04","G05","G06","G07","G08","G09","G10","G11","G12","G13","P1","P2","P3","P4","R-D","R-E","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"G14"}
{"decision":"verified","pending_gates":["G01","G02","G03","G04","G05","G06","G07","G08","G09","G10","G11","G12","G13","P1","P2","P3","P4","R-D","R-E","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"G14"}
{"decision":"verified","pending_gates":["G01","G02","G03","G04","G05","G06","G07","G08","G09","G11","G12","G13","G14","P1","P2","P3","P4","R-D","R-E","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"G10"}
{"decision":"verified","pending_gates":["G01","G02","G03","G04","G05","G06","G07","G08","G09","G10","G11","G12","G13","G14","P2","P3","P4","R-D","R-E","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"P1"}
{"decision":"verified","pending_gates":["G01","G02","G03","G04","G05","G06","G07","G08","G09","G10","G11","G12","G13","G14","P1","P3","P4","R-D","R-E","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"P2"}
{"decision":"verified","pending_gates":["G01","G02","G03","G04","G05","G06","G07","G08","G09","G10","G11","G12","G13","G14","P1","P2","P4","R-D","R-E","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"P3"}
{"decision":"verified","pending_gates":["G01","G02","G03","G04","G06","G07","G08","G09","G10","G11","G12","G13","G14","P1","P2","P3","P4","R-D","R-E","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"G05"}
{"decision":"verified","pending_gates":["G01","G02","G03","G04","G05","G07","G08","G09","G10","G11","G12","G13","G14","P1","P2","P3","P4","R-D","R-E","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"G06"}
{"decision":"verified","pending_gates":["G01","G02","G03","G04","G05","G06","G07","G08","G09","G10","G11","G12","G13","G14","P1","P2","P3","P4","R-E","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"R-D"}
Status-and-outcome release descriptor verified: /private/var/folders/0w/s4wxsn2n4w9b6f3cd4xrz53w0000gn/T/tmp8rwn1pe0/descriptor/release-bundle-status-outcome.json
{"decision":"verified","pending_gates":["G01","G02","G03","G04","G06","G07","G08","G09","G10","G11","G12","G13","G14","P1","P2","P3","P4","R-D","R-E","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"G05"}
{"decision":"verified","pending_gates":["G01","G02","G03","G04","G05","G07","G08","G09","G10","G11","G12","G13","G14","P1","P2","P3","P4","R-D","R-E","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"G06"}
{"decision":"verified","pending_gates":["G01","G02","G03","G04","G05","G06","G07","G08","G09","G10","G11","G12","G13","G14","P1","P2","P3","P4","R-E","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"R-D"}
{"decision":"verified","pending_gates":["G01","G02","G03","G04","G05","G06","G08","G09","G10","G11","G12","G13","G14","P1","P2","P3","P4","R-D","R-E","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"G07"}
{"decision":"verified","pending_gates":["G01","G02","G03","G04","G05","G06","G07","G09","G10","G11","G12","G13","G14","P1","P2","P3","P4","R-D","R-E","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"G08"}
{"decision":"verified","pending_gates":["G01","G02","G03","G04","G05","G06","G07","G08","G09","G10","G11","G12","G13","G14","P1","P2","P3","R-D","R-E","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"P4"}
exact 1MiB handoff ABBA tests passed
pinned paired ABBA bench tests passed
==> /var/folders/0w/s4wxsn2n4w9b6f3cd4xrz53w0000gn/T/tmp.mYmuI01hNq/bin/path_containment (-max_total_time=7, -seed=123456789, -artifact_prefix=/var/folders/0w/s4wxsn2n4w9b6f3cd4xrz53w0000gn/T/tmp.mYmuI01hNq/explicit-artifacts/path_containment/, corpus=/private/tmp/gw-orchestrator-2734/ci-drive/fuzz/corpus/path_containment)
-seed=123456789
target=path_containment
seed=123456789
max_total_time=7
git_revision=4d15b816236630b2c42f321ffc9ed3654edc94e3
git_dirty=true
runner_mode=prebuilt
==> /var/folders/0w/s4wxsn2n4w9b6f3cd4xrz53w0000gn/T/tmp.mYmuI01hNq/bin/path_containment (-max_total_time=1, -seed=2322617706, -artifact_prefix=/var/folders/0w/s4wxsn2n4w9b6f3cd4xrz53w0000gn/T/tmp.mYmuI01hNq/auto-artifacts/path_containment/, corpus=/private/tmp/gw-orchestrator-2734/ci-drive/fuzz/corpus/path_containment)
-seed=2322617706
==> cargo +nightly fuzz run path_containment (-max_total_time=9, -seed=987654321, -artifact_prefix=/var/folders/0w/s4wxsn2n4w9b6f3cd4xrz53w0000gn/T/tmp.mYmuI01hNq/cargo-artifacts/path_containment/)
+nightly
fuzz
run
path_containment
-seed=987654321
seed=987654321
runner_mode=cargo-fuzz
fuzz runner seed manifest tests passed
✅ scripts/python_bin.sh resolver checks passed
self-test ok: clean fixture with every exemption present passes
self-test ok: all 11 pattern families fire
self-test ok: moving a pinned source is reported
Embedded secret guard self-test passed.
self-test ok: p1-s3s-import: 'use s3s::s3_error;' counts every line, the rustfs_s3_types module's too (s3_error_lines=3)
self-test ok: p2-s3s-mixed-tree: a multi-line s3s use tree naming s3_error counts (s3_error_lines=3)
self-test ok: p3-s3s-glob: a crate-root glob of s3s, bare or inside a tree, counts (s3_error_lines=5)
self-test ok: p4-macro-use: '#[macro_use] extern crate s3s' counts (s3_error_lines=3)
self-test ok: p5-qualified-s3s: a qualified s3s::s3_error! line counts in a cleared file (s3_error_lines=2)
self-test ok: p6-exact-pass: a baseline equal to the tree passes (exit 0)
self-test ok: n1-rustfs-import: 'use rustfs_s3_types::s3_error;' alone counts nothing (s3_error_lines=1)
self-test ok: n2-rustfs-mixed-tree: a rustfs_s3_types tree plus an s3s tree without the macro counts nothing (s3_error_lines=1)
self-test ok: n3-defines: the file defining the macro counts nothing (s3_error_lines=1)
self-test ok: n4-qualified-rustfs: a qualified rustfs_s3_types::s3_error! line counts nothing (s3_error_lines=1)
self-test ok: n5-renamed-call: a call through a renamed macro is not text-matched (out of scope) (s3_error_lines=1)
self-test ok: n6-comment-string: a comment or string mentioning s3_error! in a cleared file counts nothing (s3_error_lines=1)
self-test ok: n7-e2e-excluded: crates/e2e_test is excluded (s3_error_lines=1)
self-test ok: n8-super-glob: an unrecognised import form ('use super::*;') counts (s3_error_lines=3)
self-test ok: n9-facade: a crate-local re-export counts (s3_error_lines=2)
self-test ok: n10-renamed-import: a renamed rustfs_s3_types import does not clear s3_error! lines (s3_error_lines=2)
self-test ok: n11-commented-import: a commented-out rustfs_s3_types import does not clear (s3_error_lines=2)
self-test ok: n12-rustfs-glob: a glob import of rustfs_s3_types does not clear (s3_error_lines=2)
self-test ok: n13-comment-only: a comment mentioning s3_error! in a file with no import counts (s3_error_lines=2)
self-test ok: f1-baseline-missing: a missing baseline file fails (exit 1)
self-test ok: f2-key-missing: a baseline without s3_error_lines fails (exit 1)
self-test ok: f3-rg-fails: a failing classification search fails the guard (exit 1)
✅ s3s footprint self-test passed
test_core_gate_rejects_missing_ignored_filtered_and_corrupt_inputs (__main__.SelfTests.test_core_gate_rejects_missing_ignored_filtered_and_corrupt_inputs) ... ok
test_e2e_requires_registration (__main__.SelfTests.test_e2e_requires_registration) ... ok
test_fuzz_runtime_uses_matrix_target (__main__.SelfTests.test_fuzz_runtime_uses_matrix_target) ... ok
test_ilm_lane_keeps_the_measured_cargo_build_budget (__main__.SelfTests.test_ilm_lane_keeps_the_measured_cargo_build_budget) ... ok
test_profile_listing_binds_platform_digest (__main__.SelfTests.test_profile_listing_binds_platform_digest) ... ok
test_profile_listing_enforces_selection (__main__.SelfTests.test_profile_listing_enforces_selection) ... ok
test_quick_checks_commands_propagate_failure (__main__.SelfTests.test_quick_checks_commands_propagate_failure) ... ok
test_quick_checks_rejects_caller_and_execution_bypasses (__main__.SelfTests.test_quick_checks_rejects_caller_and_execution_bypasses) ... ok
test_runner_readiness_fails_closed (__main__.SelfTests.test_runner_readiness_fails_closed) ... ok
test_s3_runner_rejects_unbounded_failure_locals (__main__.SelfTests.test_s3_runner_rejects_unbounded_failure_locals) ... ok
test_scanner_heal_begin_requires_embedded_source_provenance (__main__.SelfTests.test_scanner_heal_begin_requires_embedded_source_provenance) ... ok
test_scanner_heal_case_does_not_approve_pending_release (__main__.SelfTests.test_scanner_heal_case_does_not_approve_pending_release) ... ok
test_scanner_heal_case_lane_is_required_for_runner_profile (__main__.SelfTests.test_scanner_heal_case_lane_is_required_for_runner_profile) ... ok
test_scanner_heal_case_only_schema_cannot_approve_release (__main__.SelfTests.test_scanner_heal_case_only_schema_cannot_approve_release) ... ok
test_scanner_heal_crash_case_rejects_restart_oracle (__main__.SelfTests.test_scanner_heal_crash_case_rejects_restart_oracle) ... ok
test_scanner_heal_ec84_case_rejects_wrong_erasure_geometry (__main__.SelfTests.test_scanner_heal_ec84_case_rejects_wrong_erasure_geometry) ... ok
test_scanner_heal_finish_collects_oracles_from_registry (__main__.SelfTests.test_scanner_heal_finish_collects_oracles_from_registry) ... ok
test_scanner_heal_multipool_case_accepts_deferred_optional_outage_write (__main__.SelfTests.test_scanner_heal_multipool_case_accepts_deferred_optional_outage_write) ... ok
test_scanner_heal_pending_gate_cannot_map_to_implemented_lane (__main__.SelfTests.test_scanner_heal_pending_gate_cannot_map_to_implemented_lane) ... ok
test_scanner_heal_receipts_reject_reuse_and_missing_builds (__main__.SelfTests.test_scanner_heal_receipts_reject_reuse_and_missing_builds) ... ok
test_scanner_heal_rejects_boolean_fractional_and_out_of_geometry_integers (__main__.SelfTests.test_scanner_heal_rejects_boolean_fractional_and_out_of_geometry_integers) ... ok
test_scanner_heal_rejects_broken_execution_and_artifacts (__main__.SelfTests.test_scanner_heal_rejects_broken_execution_and_artifacts) ... ok
test_scanner_heal_rejects_copied_junit_and_wrong_suite_build (__main__.SelfTests.test_scanner_heal_rejects_copied_junit_and_wrong_suite_build) ... ok
test_scanner_heal_release_bundle_accepts_complete_measured_evidence (__main__.SelfTests.test_scanner_heal_release_bundle_accepts_complete_measured_evidence) ... ok
test_scanner_heal_release_bundle_assembler_copies_measured_artifacts (__main__.SelfTests.test_scanner_heal_release_bundle_assembler_copies_measured_artifacts) ... ok
test_scanner_heal_release_bundle_assembler_merges_measured_descriptors (__main__.SelfTests.test_scanner_heal_release_bundle_assembler_merges_measured_descriptors) ... ok
test_scanner_heal_release_bundle_assembler_rejects_duplicate_descriptor_gate (__main__.SelfTests.test_scanner_heal_release_bundle_assembler_rejects_duplicate_descriptor_gate) ... ok
test_scanner_heal_release_bundle_assembler_rejects_fixture_descriptor (__main__.SelfTests.test_scanner_heal_release_bundle_assembler_rejects_fixture_descriptor) ... ok
test_scanner_heal_release_bundle_assembler_rejects_hash_mismatch (__main__.SelfTests.test_scanner_heal_release_bundle_assembler_rejects_hash_mismatch) ... ok
test_scanner_heal_release_bundle_assembler_rejects_mixed_descriptor_revisions (__main__.SelfTests.test_scanner_heal_release_bundle_assembler_rejects_mixed_descriptor_revisions) ... ok
test_scanner_heal_release_bundle_enforces_topology_duration_profile_and_versions (__main__.SelfTests.test_scanner_heal_release_bundle_enforces_topology_duration_profile_and_versions) ... ok
test_scanner_heal_release_bundle_fixture_writer_is_not_release_evidence (__main__.SelfTests.test_scanner_heal_release_bundle_fixture_writer_is_not_release_evidence) ... ok
test_scanner_heal_release_bundle_gate_accepts_single_measured_gate (__main__.SelfTests.test_scanner_heal_release_bundle_gate_accepts_single_measured_gate) ... ok
test_scanner_heal_release_bundle_gate_rejects_missing_field (__main__.SelfTests.test_scanner_heal_release_bundle_gate_rejects_missing_field) ... ok
test_scanner_heal_release_bundle_gate_rejects_unknown_gate (__main__.SelfTests.test_scanner_heal_release_bundle_gate_rejects_unknown_gate) ... ok
test_scanner_heal_release_bundle_rejects_synthetic_or_missing_fields (__main__.SelfTests.test_scanner_heal_release_bundle_rejects_synthetic_or_missing_fields) ... ok
test_scanner_heal_release_bundle_requires_domain_evidence (__main__.SelfTests.test_scanner_heal_release_bundle_requires_domain_evidence) ... ok
test_scanner_heal_release_bundle_requires_field_provenance (__main__.SelfTests.test_scanner_heal_release_bundle_requires_field_provenance) ... ok
test_scanner_heal_release_bundle_requires_json_artifact_provenance (__main__.SelfTests.test_scanner_heal_release_bundle_requires_json_artifact_provenance) ... ok
test_scanner_heal_release_bundle_requires_mixed_version_field_roles (__main__.SelfTests.test_scanner_heal_release_bundle_requires_mixed_version_field_roles) ... ok
test_scanner_heal_release_bundle_requires_same_gate_measurement_window (__main__.SelfTests.test_scanner_heal_release_bundle_requires_same_gate_measurement_window) ... ok
test_scanner_heal_release_consumes_multiple_registry_oracles (__main__.SelfTests.test_scanner_heal_release_consumes_multiple_registry_oracles) ... ok
test_scanner_heal_release_matrix_lanes_remain_pending_without_real_evidence (__main__.SelfTests.test_scanner_heal_release_matrix_lanes_remain_pending_without_real_evidence) ... ok
test_scanner_heal_release_requirements_cannot_be_cleared (__main__.SelfTests.test_scanner_heal_release_requirements_cannot_be_cleared) ... ok
test_scanner_heal_release_status_rejects_focused_case_run (__main__.SelfTests.test_scanner_heal_release_status_rejects_focused_case_run) ... ok
test_scanner_heal_release_status_rejects_synthetic_case (__main__.SelfTests.test_scanner_heal_release_status_rejects_synthetic_case) ... ok
test_scanner_heal_required_evidence_fields_cannot_be_removed (__main__.SelfTests.test_scanner_heal_required_evidence_fields_cannot_be_removed) ... ok
test_scheduled_alerts_require_completion_watchdog (__main__.SelfTests.test_scheduled_alerts_require_completion_watchdog) ... ok
test_update_profile_selection_changes_only_requested_platform (__main__.SelfTests.test_update_profile_selection_changes_only_requested_platform) ... ok
test_validate_includes_quick_checks (__main__.SelfTests.test_validate_includes_quick_checks) ... ok
test_vault_tests_require_cross_process_serialization (__main__.SelfTests.test_vault_tests_require_cross_process_serialization) ... ok
test_workflow_readiness_requires_dependency_probe (__main__.SelfTests.test_workflow_readiness_requires_dependency_probe) ... ok
----------------------------------------------------------------------
Ran 52 tests in 14.301s
OK
e2e-full selection OK: 1 tests, sha256=29e30e6ed2ed38d9e5b0a7cdd9536abf5c719f7021a02b578ae1381563a3c273
...................................
----------------------------------------------------------------------
Ran 35 tests in 1.161s
OK
...................
----------------------------------------------------------------------
Ran 19 tests in 17.397s
OK
..........
----------------------------------------------------------------------
Ran 10 tests in 0.183s
OK
test_actual_selector_bootstrap_uses_base_policy_and_fails_closed (__main__.SelfTests.test_actual_selector_bootstrap_uses_base_policy_and_fails_closed) ... ok
test_documentation_paths_do_not_hide_build_or_fixture_changes (__main__.SelfTests.test_documentation_paths_do_not_hide_build_or_fixture_changes) ... ok
test_e2e_scope_keeps_production_and_shared_configuration_full (__main__.SelfTests.test_e2e_scope_keeps_production_and_shared_configuration_full) ... ok
test_e2e_shortcut_rejects_new_direct_renamed_and_target_dependencies (__main__.SelfTests.test_e2e_shortcut_rejects_new_direct_renamed_and_target_dependencies) ... ok
test_e2e_shortcut_traverses_implicit_path_dependencies (__main__.SelfTests.test_e2e_shortcut_traverses_implicit_path_dependencies) ... ok
test_event_contract_requires_complete_candidate_and_optional_lanes (__main__.SelfTests.test_event_contract_requires_complete_candidate_and_optional_lanes) ... ok
test_every_wrong_result_missing_job_or_selection_fails_closed (__main__.SelfTests.test_every_wrong_result_missing_job_or_selection_fails_closed) ... ok
test_full_e2e_gate_preserves_workflow_branch_and_event_scope (__main__.SelfTests.test_full_e2e_gate_preserves_workflow_branch_and_event_scope) ... ok
test_git_range_includes_deleted_source_and_rename_origins (__main__.SelfTests.test_git_range_includes_deleted_source_and_rename_origins) ... ok
test_job_ids_and_display_names_cannot_hide_validation (__main__.SelfTests.test_job_ids_and_display_names_cannot_hide_validation) ... ok
test_report_only_stack_lane_mirrors_e2e_smoke_outside_the_aggregate (__main__.SelfTests.test_report_only_stack_lane_mirrors_e2e_smoke_outside_the_aggregate) ... ok
test_repository_wiring_and_missing_dependency_regression (__main__.SelfTests.test_repository_wiring_and_missing_dependency_regression) ... ok
test_verify_command_preserves_failures (__main__.SelfTests.test_verify_command_preserves_failures) ... ok
----------------------------------------------------------------------
Ran 13 tests in 3.280s
OK
security coverage self-test passed
test_api_failures_preserve_other_evidence_and_never_enter_grace (__main__.SelfTests.test_api_failures_preserve_other_evidence_and_never_enter_grace) ... ok
test_attempt_outcome_does_not_replace_recent_success (__main__.SelfTests.test_attempt_outcome_does_not_replace_recent_success) ... ok
test_cancelled_attempt_cannot_refresh_expired_success (__main__.SelfTests.test_cancelled_attempt_cannot_refresh_expired_success) ... ok
test_cli_requires_the_repository_default_branch (__main__.SelfTests.test_cli_requires_the_repository_default_branch) ... ok
test_config_rejects_duplicate_and_invalid_entries (__main__.SelfTests.test_config_rejects_duplicate_and_invalid_entries) ... ok
test_freshness_boundaries (__main__.SelfTests.test_freshness_boundaries) ... ok
test_grace_requires_two_successful_queries_with_no_history (__main__.SelfTests.test_grace_requires_two_successful_queries_with_no_history) ... ok
test_invalid_api_evidence_fails_closed (__main__.SelfTests.test_invalid_api_evidence_fails_closed) ... ok
test_report_retains_every_workflow (__main__.SelfTests.test_report_retains_every_workflow) ... ok
test_requests_filter_schedule_default_branch_and_success_on_server (__main__.SelfTests.test_requests_filter_schedule_default_branch_and_success_on_server) ... ok
----------------------------------------------------------------------
Ran 10 tests in 0.051s
OK
..............................
----------------------------------------------------------------------
Ran 30 tests in 22.354s
OK
..........
----------------------------------------------------------------------
Ran 10 tests in 2.969s
OK
.................................
----------------------------------------------------------------------
Ran 33 tests in 0.160s
OK
Online functional runner labels: pf-testing
.......
----------------------------------------------------------------------
Ran 7 tests in 0.002s
OK
.......
----------------------------------------------------------------------
Ran 7 tests in 0.001s
OK
....
----------------------------------------------------------------------
Ran 4 tests in 0.128s
OK
...
----------------------------------------------------------------------
Ran 3 tests in 0.547s
OK
PASS: cold stampede script self-test
exit_code=0
$ python3 ./scripts/test_object_generation_protocol_model.py
test_a_recovery_barrier_cannot_choose_an_empty_value (__main__.ObjectGenerationProtocolModel.test_a_recovery_barrier_cannot_choose_an_empty_value) ... ok
test_all_bounded_two_proposer_interleavings_choose_at_most_one_value (__main__.ObjectGenerationProtocolModel.test_all_bounded_two_proposer_interleavings_choose_at_most_one_value) ... ok
test_an_unavailable_decision_quorum_cannot_choose_a_candidate (__main__.ObjectGenerationProtocolModel.test_an_unavailable_decision_quorum_cannot_choose_a_candidate) ... ok
test_counter_allocation_requires_persistence_and_is_consumed_once (__main__.ObjectGenerationProtocolModel.test_counter_allocation_requires_persistence_and_is_consumed_once) ... ok
test_counter_restart_burns_persisted_allocations_before_or_after_issue (__main__.ObjectGenerationProtocolModel.test_counter_restart_burns_persisted_allocations_before_or_after_issue) ... ok
test_crash_after_promise_before_reply_does_not_restore_an_older_ballot (__main__.ObjectGenerationProtocolModel.test_crash_after_promise_before_reply_does_not_restore_an_older_ballot) ... ok
test_duplicate_or_unconfigured_voters_cannot_form_a_quorum (__main__.ObjectGenerationProtocolModel.test_duplicate_or_unconfigured_voters_cannot_form_a_quorum) ... ok
test_empty_or_unobserved_minority_slot_can_retry_without_allocating_another_revision (__main__.ObjectGenerationProtocolModel.test_empty_or_unobserved_minority_slot_can_retry_without_allocating_another_revision) ... ok
test_equal_ballot_cannot_accept_different_bytes_and_lower_ballot_is_rejected (__main__.ObjectGenerationProtocolModel.test_equal_ballot_cannot_accept_different_bytes_and_lower_ballot_is_rejected) ... ok
test_forgetting_tombstone_authority_allows_an_aba_create (__main__.ObjectGenerationProtocolModel.test_forgetting_tombstone_authority_allows_an_aba_create) ... ok
test_issuing_before_persistence_reuses_a_ballot_after_restart (__main__.ObjectGenerationProtocolModel.test_issuing_before_persistence_reuses_a_ballot_after_restart) ... ok
test_lost_ack_and_restart_preserve_the_chosen_operation (__main__.ObjectGenerationProtocolModel.test_lost_ack_and_restart_preserve_the_chosen_operation) ... ok
test_minority_accepted_work_is_adopted_and_finished (__main__.ObjectGenerationProtocolModel.test_minority_accepted_work_is_adopted_and_finished) ... ok
test_omitting_highest_accepted_adoption_has_a_double_choice_counterexample (__main__.ObjectGenerationProtocolModel.test_omitting_highest_accepted_adoption_has_a_double_choice_counterexample) ... ok
test_quorum_choice_does_not_require_an_uncontacted_disk_to_learn_it (__main__.ObjectGenerationProtocolModel.test_quorum_choice_does_not_require_an_uncontacted_disk_to_learn_it) ... ok
test_recovery_failure_or_stale_expected_head_does_not_allocate_a_successor (__main__.ObjectGenerationProtocolModel.test_recovery_failure_or_stale_expected_head_does_not_allocate_a_successor) ... ok
test_replacing_voters_with_empty_state_can_choose_a_conflicting_value (__main__.ObjectGenerationProtocolModel.test_replacing_voters_with_empty_state_can_choose_a_conflicting_value) ... ok
test_skipping_predecessor_recovery_chooses_a_broken_revision_chain (__main__.ObjectGenerationProtocolModel.test_skipping_predecessor_recovery_chooses_a_broken_revision_chain) ... ok
test_stale_publication_rollback_and_gc_cannot_change_the_winner (__main__.ObjectGenerationProtocolModel.test_stale_publication_rollback_and_gc_cannot_change_the_winner) ... ok
test_superseded_operation_identity_cannot_be_reused_for_different_input (__main__.ObjectGenerationProtocolModel.test_superseded_operation_identity_cannot_be_reused_for_different_input) ... ok
test_superseded_operation_retry_returns_its_original_successor (__main__.ObjectGenerationProtocolModel.test_superseded_operation_retry_returns_its_original_successor) ... ok
test_tombstone_and_null_marker_payloads_do_not_reset_authority_revision (__main__.ObjectGenerationProtocolModel.test_tombstone_and_null_marker_payloads_do_not_reset_authority_revision) ... ok
test_tombstone_rejects_a_delayed_never_present_create (__main__.ObjectGenerationProtocolModel.test_tombstone_rejects_a_delayed_never_present_create) ... ok
test_unlearned_predecessor_is_recovered_before_the_second_slot (__main__.ObjectGenerationProtocolModel.test_unlearned_predecessor_is_recovered_before_the_second_slot) ... ok
----------------------------------------------------------------------
Ran 24 tests in 0.958s
OK
model: states=51023, transitions=167520, terminal=732, chosen_candidates=2
negative control: double-choice trace=((0, 'prepare', 0), (0, 'prepare', 1), (0, 'prepare', 2), (0, 'accept', 0), (0, 'accept', 1), (0, 'accept', 2), (1, 'prepare', 0), (1, 'prepare', 1), (1, 'prepare', 2), (1, 'accept', 0), (1, 'accept', 1), (1, 'accept', 2))
multi-slot round schedules: 640
exit_code=0
$ python3 ./scripts/check_test_wiring.py
OK: e2e modules, runner selection, fuzz matrices, profiles, scheduled alerts, and Quick Checks are wired
exit_code=0
$ ./scripts/check_no_planning_docs.sh
exit_code=0
$ ./scripts/check_uring_lane_lib_only.sh
OK: no 'uring_'-matching test functions under crates/ecstore/tests; --lib is safe
exit_code=0
Initial complete gate
$ cargo nextest run --profile ci -p rustfs --test connect_perf_drive --no-fail-fast
Nextest run ID e07fc5b9-c6c8-4cdd-ae61-54839bf24437 with nextest profile: ci
Starting 10 tests across 1 binary
PASS [ 0.016s] rustfs::connect_perf_drive cancellation_and_single_collector_leave_no_scratch
PASS [ 0.011s] rustfs::connect_perf_drive cleanup_failure_overrides_cancellation_and_remains_attributed
PASS [ 0.009s] rustfs::connect_perf_drive consent_and_resource_budgets_fail_before_scratch_io
PASS [ 0.009s] rustfs::connect_perf_drive invalid_roots_and_existing_scratch_fail_without_clobbering
PASS [ 16.950s] rustfs::connect_perf_drive production_cli_measures_and_signs_exact_binary_provenance
PASS [ 4.025s] rustfs::connect_perf_drive production_cli_rejects_missing_l1_consent_and_limits_before_artifacts
PASS [ 0.008s] rustfs::connect_perf_drive read_only_root_and_full_drive_errors_are_classified
PASS [ 0.015s] rustfs::connect_perf_drive real_local_write_read_is_measured_and_scratch_is_removed
PASS [ 0.007s] rustfs::connect_perf_drive runtime_timeout_and_full_drive_use_schema_valid_failed_reason
PASS [ 0.035s] rustfs::connect_perf_drive successful_result_has_signed_bounded_private_offline_export
────────────
Summary [ 21.087s] 10 tests run: 10 passed, 0 skipped
exit_code=0
Restored complete gate
$ cargo nextest run --profile ci -p rustfs --test connect_perf_drive --no-fail-fast
Nextest run ID 233b7307-5361-4eb7-94d5-b384f17d2d6a with nextest profile: ci
Starting 10 tests across 1 binary
PASS [ 0.009s] rustfs::connect_perf_drive cancellation_and_single_collector_leave_no_scratch
PASS [ 0.008s] rustfs::connect_perf_drive cleanup_failure_overrides_cancellation_and_remains_attributed
PASS [ 0.008s] rustfs::connect_perf_drive consent_and_resource_budgets_fail_before_scratch_io
PASS [ 0.008s] rustfs::connect_perf_drive invalid_roots_and_existing_scratch_fail_without_clobbering
PASS [ 19.518s] rustfs::connect_perf_drive production_cli_measures_and_signs_exact_binary_provenance
PASS [ 4.146s] rustfs::connect_perf_drive production_cli_rejects_missing_l1_consent_and_limits_before_artifacts
PASS [ 0.009s] rustfs::connect_perf_drive read_only_root_and_full_drive_errors_are_classified
PASS [ 0.018s] rustfs::connect_perf_drive real_local_write_read_is_measured_and_scratch_is_removed
PASS [ 0.009s] rustfs::connect_perf_drive runtime_timeout_and_full_drive_use_schema_valid_failed_reason
PASS [ 0.054s] rustfs::connect_perf_drive successful_result_has_signed_bounded_private_offline_export
────────────
Summary [ 23.787s] 10 tests run: 10 passed, 0 skipped
exit_code=0
Mutation M1: success_measurement reports max(actual elapsed, requested budget + 1), forcing 1001 ms for the one-second request. The retained duration assertion at rustfs/tests/connect_perf_drive.rs:105 fails.
$ cargo nextest run --profile ci -p rustfs --test connect_perf_drive --no-fail-fast -E 'test(real_local_write_read_is_measured_and_scratch_is_removed)'
info: experimental features enabled: setup-scripts
Compiling rustfs v1.0.1 (/private/tmp/gw-orchestrator-2734/ci-drive/rustfs)
warning: linker stderr: ld: __eh_frame section too large (max 16MB) to encode dwarf unwind offsets in compact unwind table, performance of exception handling might be affected
|
= note: `#[warn(linker_messages)]` on by default
warning: `rustfs` (bin "rustfs") generated 1 warning
warning: `rustfs` (bin "rustfs-cli") generated 1 warning (1 duplicate)
Finished `test` profile [unoptimized + debuginfo] target(s) in 7m 06s
────────────
Nextest run ID 9f618777-7bf6-4cbc-8803-dd7f50f449ed with nextest profile: ci
Starting 1 test across 1 binary (9 tests skipped)
FAIL [ 0.016s] rustfs::connect_perf_drive real_local_write_read_is_measured_and_scratch_is_removed
stdout ───
running 1 test
test real_local_write_read_is_measured_and_scratch_is_removed ... FAILED
failures:
failures:
real_local_write_read_is_measured_and_scratch_is_removed
test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 9 filtered out; finished in 0.01s
stderr ───
thread 'real_local_write_read_is_measured_and_scratch_is_removed' (106986846) panicked at rustfs/tests/connect_perf_drive.rs:105:5:
assertion failed: (1..=1_000).contains(&data.duration_millis)
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
────────────
Summary [ 0.017s] 1 test run: 0 passed, 1 failed, 9 skipped
FAIL [ 0.016s] rustfs::connect_perf_drive real_local_write_read_is_measured_and_scratch_is_removed
error: test run failed
exit_code=100
Mutation M2: target_parameters reports requested duration + 1. The in-process assertion at rustfs/tests/connect_perf_drive.rs:111 and CLI JSON assertion at rustfs/tests/connect_perf_drive.rs:527 both reject 1001 against 1000.
$ cargo nextest run --profile ci -p rustfs --test connect_perf_drive --no-fail-fast -E 'test(real_local_write_read_is_measured_and_scratch_is_removed) | test(production_cli_measures_and_signs_exact_binary_provenance)'
info: experimental features enabled: setup-scripts
Compiling rustfs v1.0.1 (/private/tmp/gw-orchestrator-2734/ci-drive/rustfs)
warning: linker stderr: ld: __eh_frame section too large (max 16MB) to encode dwarf unwind offsets in compact unwind table, performance of exception handling might be affected
|
= note: `#[warn(linker_messages)]` on by default
warning: `rustfs` (bin "rustfs") generated 1 warning
warning: `rustfs` (bin "rustfs-cli") generated 1 warning (1 duplicate)
Finished `test` profile [unoptimized + debuginfo] target(s) in 7m 20s
────────────
Nextest run ID b73d46ab-c910-4e1e-8063-7fc7e332ec15 with nextest profile: ci
Starting 2 tests across 1 binary (8 tests skipped)
FAIL [ 13.968s] rustfs::connect_perf_drive production_cli_measures_and_signs_exact_binary_provenance
stdout ───
running 1 test
test production_cli_measures_and_signs_exact_binary_provenance ... FAILED
failures:
failures:
production_cli_measures_and_signs_exact_binary_provenance
test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 9 filtered out; finished in 13.96s
stderr ───
thread 'production_cli_measures_and_signs_exact_binary_provenance' (107015462) panicked at rustfs/tests/connect_perf_drive.rs:527:5:
assertion `left == right` failed
left: Number(1001)
right: 1000
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
FAIL [ 0.015s] rustfs::connect_perf_drive real_local_write_read_is_measured_and_scratch_is_removed
stdout ───
running 1 test
test real_local_write_read_is_measured_and_scratch_is_removed ... FAILED
failures:
failures:
real_local_write_read_is_measured_and_scratch_is_removed
test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 9 filtered out; finished in 0.01s
stderr ───
thread 'real_local_write_read_is_measured_and_scratch_is_removed' (107016667) panicked at rustfs/tests/connect_perf_drive.rs:111:5:
assertion `left == right` failed
left: 1001
right: 1000
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
────────────
Summary [ 13.985s] 2 tests run: 0 passed, 2 failed, 8 skipped
FAIL [ 13.968s] rustfs::connect_perf_drive production_cli_measures_and_signs_exact_binary_provenance
FAIL [ 0.015s] rustfs::connect_perf_drive real_local_write_read_is_measured_and_scratch_is_removed
error: test run failed
exit_code=100
Transient production mutations were restored byte-for-byte before the complete green rerun. Original/restored SHA256: 8d389e6d8836c041aab8c6a93e56addd65ba3f70b33264e1c74f639d0572a2da.
$ git diff --check
exit_code=0
$ git diff --cached --check
exit_code=0
Independent orchestrator acceptance
Tested code tree: e67c72fb01
Tested tree: 16fbbd2ea4
shellcheck --version && actionlint -> exit 0
cargo fmt --all --check -> exit 0
./scripts/check_unsafe_code_allowances.sh -> exit 0
./scripts/check_layer_dependencies.sh -> exit 0
./scripts/check_architecture_migration_rules.sh -> exit 0
./scripts/check_logging_guardrails.sh -> exit 0
./scripts/check_error_other_format_ratchet.sh -> exit 0
./scripts/check_no_tokio_io_uring.sh -> exit 0
./scripts/check_extension_schema_boundaries.sh -> exit 0
./scripts/check_body_cache_whitelist.sh -> exit 0
./scripts/check_s3s_footprint.sh -> exit 0
./scripts/check_test_count_ratchet.sh -> exit 0
./scripts/check_fips_wording.sh -> exit 0
./scripts/check_embedded_secrets.sh -> exit 0
make script-tests -> exit 0
python3 ./scripts/test_object_generation_protocol_model.py -> exit 0
python3 ./scripts/check_test_wiring.py -> exit 0
./scripts/check_no_planning_docs.sh -> exit 0
./scripts/check_uring_lane_lib_only.sh -> exit 0
$ LANG=en_US.UTF-8 CARGO_BUILD_JOBS=1 CARGO_INCREMENTAL=0 cargo nextest run --profile ci -p rustfs --test connect_perf_drive --no-fail-fast
info: experimental features enabled: setup-scripts
warning: linker stderr: ld: __eh_frame section too large (max 16MB) to encode dwarf unwind offsets in compact unwind table, performance of exception handling might be affected
|
= note: `#[warn(linker_messages)]` on by default
warning: `rustfs` (bin "rustfs-cli") generated 1 warning
warning: `rustfs` (bin "rustfs") generated 1 warning (1 duplicate)
Finished `test` profile [unoptimized + debuginfo] target(s) in 0.69s
────────────
Nextest run ID cd1edf4c-f1c0-456d-ad1a-76cc00ce05c9 with nextest profile: ci
Starting 10 tests across 1 binary
PASS [ 0.012s] rustfs::connect_perf_drive cancellation_and_single_collector_leave_no_scratch
PASS [ 0.010s] rustfs::connect_perf_drive cleanup_failure_overrides_cancellation_and_remains_attributed
PASS [ 0.010s] rustfs::connect_perf_drive consent_and_resource_budgets_fail_before_scratch_io
PASS [ 0.009s] rustfs::connect_perf_drive invalid_roots_and_existing_scratch_fail_without_clobbering
PASS [ 20.328s] rustfs::connect_perf_drive production_cli_measures_and_signs_exact_binary_provenance
PASS [ 5.527s] rustfs::connect_perf_drive production_cli_rejects_missing_l1_consent_and_limits_before_artifacts
PASS [ 0.013s] rustfs::connect_perf_drive read_only_root_and_full_drive_errors_are_classified
PASS [ 0.019s] rustfs::connect_perf_drive real_local_write_read_is_measured_and_scratch_is_removed
PASS [ 0.011s] rustfs::connect_perf_drive runtime_timeout_and_full_drive_use_schema_valid_failed_reason
PASS [ 0.049s] rustfs::connect_perf_drive successful_result_has_signed_bounded_private_offline_export
────────────
Summary [ 25.991s] 10 tests run: 10 passed, 0 skipped
exit_code=0
All 54 tracked Cargo.toml files parse without duplicate keys. git diff --check passed. The production source remains unchanged; the two deliberate implementation mutations above were restored before this complete green run.
* fix(test): compare legacy body wire parity as raw bytes
Lossy UTF-8 comparison concealed a changed binary response byte.
Compare the already captured Vec<u8> values directly.
Validation:
- Binary mutant: migrated payload 0x84 vs legacy 0x83; old comparator
passed 1/1, raw comparator failed 0/1 (exit 100); restored green.
- Non-Swift body-drain mutant: untouched-body delegation test failed
with empty vs payload (exit 100); restored handler without a diff.
- cargo nextest run --profile ci -p rustfs-s3-types -p rustfs-protocols
--features rustfs-protocols/swift --no-fail-fast:
543 tests run, 543 passed, 0 skipped after both mutants restored.
- cargo fmt --all --check and git diff --check passed.
Orchestrator acceptance (LANG=en_US.UTF-8):
cargo nextest run --profile ci -p rustfs-s3-types -p rustfs-protocols --features rustfs-protocols/swift --no-fail-fast
Summary [2.445s] 543 tests run: 543 passed, 0 skipped.
All shared quick checks on the final code tree:
shellcheck --version && actionlint -> exit 0
cargo fmt --all --check -> exit 0
./scripts/check_unsafe_code_allowances.sh -> exit 0
./scripts/check_layer_dependencies.sh -> exit 0
./scripts/check_architecture_migration_rules.sh -> exit 0
./scripts/check_logging_guardrails.sh -> exit 0
./scripts/check_error_other_format_ratchet.sh -> exit 0
./scripts/check_no_tokio_io_uring.sh -> exit 0
./scripts/check_extension_schema_boundaries.sh -> exit 0
./scripts/check_body_cache_whitelist.sh -> exit 0
./scripts/check_s3s_footprint.sh -> exit 0
./scripts/check_test_count_ratchet.sh -> exit 0
./scripts/check_fips_wording.sh -> exit 0
./scripts/check_embedded_secrets.sh -> exit 0
make script-tests -> exit 0
python3 ./scripts/test_object_generation_protocol_model.py -> exit 0
python3 ./scripts/check_test_wiring.py -> exit 0
./scripts/check_no_planning_docs.sh -> exit 0
./scripts/check_uring_lane_lib_only.sh -> exit 0
Final independent acceptance after the drive-test CI repair
Parent: a99d8f4f89
Tested code tree: 93cbd96bf0
Tested tree: 075ae5161c2b62a96812bc1e3c904a7663b4f2f8
shellcheck --version && actionlint -> exit 0
cargo fmt --all --check -> exit 0
./scripts/check_unsafe_code_allowances.sh -> exit 0
./scripts/check_layer_dependencies.sh -> exit 0
./scripts/check_architecture_migration_rules.sh -> exit 0
./scripts/check_logging_guardrails.sh -> exit 0
./scripts/check_error_other_format_ratchet.sh -> exit 0
./scripts/check_no_tokio_io_uring.sh -> exit 0
./scripts/check_extension_schema_boundaries.sh -> exit 0
./scripts/check_body_cache_whitelist.sh -> exit 0
./scripts/check_s3s_footprint.sh -> exit 0
./scripts/check_test_count_ratchet.sh -> exit 0
./scripts/check_fips_wording.sh -> exit 0
./scripts/check_embedded_secrets.sh -> exit 0
make script-tests -> exit 0
python3 ./scripts/test_object_generation_protocol_model.py -> exit 0
python3 ./scripts/check_test_wiring.py -> exit 0
./scripts/check_no_planning_docs.sh -> exit 0
./scripts/check_uring_lane_lib_only.sh -> exit 0
$ LANG=en_US.UTF-8 CARGO_BUILD_JOBS=1 CARGO_INCREMENTAL=0 cargo nextest run --profile ci -p rustfs-s3-types -p rustfs-protocols --features rustfs-protocols/swift --no-fail-fast
Finished `test` profile [unoptimized + debuginfo] target(s) in 10m 08s
Nextest run ID 5c7ff102-ec6d-479a-86e0-810859b28bbf with nextest profile: ci
Starting 543 tests across 8 binaries
Summary [ 6.729s] 543 tests run: 543 passed, 0 skipped
exit_code=0
The only source change is the raw-byte wire comparator. Cargo manifests and the ratchet baselines remain unchanged from the independently validated parent. The invalid-byte and consumed-request mutations remain recorded above; their implementation changes were restored.
* fix: repair integration checks and IPC test readiness
Wait for the RPC/locks test sockets to pass the existing owner and mode
checks inside the original 5s readiness window and 10ms poll interval.
Use the existing admin S3 carrier for the new recovery and quota error
calls. Regenerate the test-count baseline for main's 118 existing tests
across seven crates. Pair the private pagination helper's raw-scan inputs
and remove the redundant Copy clone from the retirement event.
Keep the original assertions, responses, authorization, locks and storage
operations. The pagination body and raw-scan expressions at all five calls
are byte-identical; reinserting only the Copy clone restores the entire
base heal file. All ignored baseline values and s3s counts stay unchanged.
Validation base: c47432e353
Validation tree: f081084bc5
Patch SHA-256: 2dcf0028691bfb2772edfb38e262829b04f4151832dcce823c9dd9f7434b52de
Trace production-prefix SHA-256: fd994f218a21429d62866f4efb8eee1ee57b58f4a51865aa35d05a54ef1ccb45
All commands used LANG=en_US.UTF-8, CARGO_BUILD_JOBS=1,
CARGO_INCREMENTAL=0 and CARGO_NET_OFFLINE=true in one exclusive target.
Every gate retained the same tree and six source hashes before and after.
54 Cargo.toml files parse without duplicate keys.
cargo clippy -p rustfs --all-targets -- -D warnings
Finished `dev` profile [unoptimized + debuginfo] target(s) in 12m 50s
Exit: 0. Wall time: 770.855s.
The original 19 quick-check steps:
shellcheck --version && actionlint -> exit 0
cargo fmt --all --check -> exit 0
./scripts/check_unsafe_code_allowances.sh -> exit 0
./scripts/check_layer_dependencies.sh -> exit 0
./scripts/check_architecture_migration_rules.sh -> exit 0
./scripts/check_logging_guardrails.sh -> exit 0
./scripts/check_error_other_format_ratchet.sh -> exit 0
./scripts/check_no_tokio_io_uring.sh -> exit 0
./scripts/check_extension_schema_boundaries.sh -> exit 0
./scripts/check_body_cache_whitelist.sh -> exit 0
./scripts/check_s3s_footprint.sh -> exit 0
./scripts/check_test_count_ratchet.sh -> exit 0
./scripts/check_fips_wording.sh -> exit 0
./scripts/check_embedded_secrets.sh -> exit 0
make script-tests -> exit 0
python3 ./scripts/test_object_generation_protocol_model.py -> exit 0
python3 ./scripts/check_test_wiring.py -> exit 0
./scripts/check_no_planning_docs.sh -> exit 0
./scripts/check_uring_lane_lib_only.sh -> exit 0
Wall time: 286.337s.
cargo nextest run --profile ci -p rustfs --no-fail-fast
Summary [ 217.017s] 5382 tests run: 5382 passed (4 leaky), 40 skipped
Exit: 0. Wall time: 1324.414s.
cargo nextest run --profile ci -p rustfs-s3-types -p rustfs-protocols --features rustfs-protocols/swift --no-fail-fast
Summary [ 2.189s] 543 tests run: 543 passed, 0 skipped
Exit: 0. Wall time: 123.192s.
cargo nextest run --profile ci -p rustfs-ecstore list_objects_paginate --no-fail-fast
Summary [ 0.170s] 8 tests run: 8 passed, 5921 skipped
Exit: 0. Wall time: 261.324s.
cargo nextest run --profile ci -p rustfs-heal durable_mrf_anchor_retires_only_after_exact_bucket_generation_is_deleted --no-fail-fast
Summary [ 0.805s] 1 test run: 1 passed, 835 skipped
Exit: 0. Wall time: 199.801s.
Baseline regeneration:
./scripts/check_test_count_ratchet.sh --update
wrote .config/test-count-baseline.txt: 52 crates, 21340 tests, 164 ignored
Test-only increases: ecstore48, heal3, iam14, io-metrics2, obs3, policy14,
rustfs34. Existing ignored values and all other rows are byte-identical.
s3s footprint OK: files importing s3s is 195 (baseline files=195)
s3s footprint OK: s3s s3_error! lines is 1584 (baseline s3_error_lines=1584)
s3s footprint OK: ecstore files referencing s3s is 36 (baseline ecstore_files=36)
Historical runtime red controls on f622 (same retained two assertions):
cargo nextest run --profile ci -p rustfs --lib -E 'test(=connect::diagnostics::trace_runtime::tests::local_top_rpc_reads_service_http_completions_and_signs_offline) | test(=connect::diagnostics::trace_runtime::tests::local_top_locks_reads_service_manager_and_signs_offline)' --no-fail-fast
Original path readiness with a temporary 500ms bind/seal gap:
Summary [ 0.863s] 2 tests run: 0 passed, 2 failed, 5113 skipped
Exit: 100.
Sealed readiness in that same gap:
Summary [ 5.621s] 2 tests run: 2 passed, 5113 skipped
Exit: 0.
Compiled M1: replace the helper predicate with path.exists():
Summary [ 0.870s] 2 tests run: 0 passed, 2 failed, 5113 skipped
Exit: 100.
Restore the sealed predicate byte-for-byte:
Summary [ 5.640s] 2 tests run: 2 passed, 5113 skipped
Exit: 0.
Both reds observe mode755 with owner_matches=true and fail at the original
RPC/locks unwraps with StateSecurity. M1 failure locations: trace_runtime.rs
2547:29 and 2713:29. No new assertions were introduced.
cargo nextest run --profile ci -p rustfs --lib -E 'test(connect::diagnostics::trace_runtime::tests::)' --no-fail-fast
Summary [ 20.748s] 28 tests run: 28 passed (1 leaky), 5087 skipped
Exit: 0. This security sweep is historical; the full final unit gate above
covers the actual c474 candidate.
The first disk-stopped compile ran no tests and is excluded from red credit.
All temporary seal instrumentation is removed from the final source.
Final unit cleanup markers: 4 leaky, 40 original skips (exit0). The ld
__eh_frame linker warning also appears verbatim in the original f622 run
and the earlier c474 run; there are no new source warnings.
* fix: remove redundant pending closure in disk test
Pass the capture-free pending future factory as a function item in the existing slow-operation test. This fixes the redundant_closure error at disk_store.rs:2914 that stopped strict Clippy before the feature tests could run.
The single-line test edit preserves the factory's result, warning and timeout behavior. Reversing the replacement reproduces the parent file byte for byte. All 220 assertion lines and the production prefix are unchanged; no tests, assertions, guards, profiles or baselines were added or relaxed.
Original Linux CI evidence (run 37888552268 at 53245be9f):
- RIO job113685483992: strict Clippy exit101, redundant closure at disk_store.rs:2914; feature tests skipped.
- Workspace Test and Lint job113685484058: same strict Clippy exit101; nextest and doc tests skipped.
These are lint failures, not runtime mutation kills or known flakes.
Native macOS arm64 validation on tree7a5bf47e7533b29479bc85207416102f1337c566, LANG=en_US.UTF-8, one build job, CARGO_INCREMENTAL=0 and offline dependencies:
shellcheck --version && actionlint -> exit 0
cargo fmt --all --check -> exit 0
./scripts/check_unsafe_code_allowances.sh -> exit 0
./scripts/check_layer_dependencies.sh -> exit 0
./scripts/check_architecture_migration_rules.sh -> exit 0
./scripts/check_logging_guardrails.sh -> exit 0
./scripts/check_error_other_format_ratchet.sh -> exit 0
./scripts/check_no_tokio_io_uring.sh -> exit 0
./scripts/check_extension_schema_boundaries.sh -> exit 0
./scripts/check_body_cache_whitelist.sh -> exit 0
./scripts/check_s3s_footprint.sh -> exit 0
./scripts/check_test_count_ratchet.sh -> exit 0
./scripts/check_fips_wording.sh -> exit 0
./scripts/check_embedded_secrets.sh -> exit 0
make script-tests -> exit 0
python3 ./scripts/test_object_generation_protocol_model.py -> exit 0
python3 ./scripts/check_test_wiring.py -> exit 0
./scripts/check_no_planning_docs.sh -> exit 0
./scripts/check_uring_lane_lib_only.sh -> exit 0
All 19 checks: exit0, 312.161s.
$ cargo clippy -p rustfs -p rustfs-ecstore --all-targets --features rio-v2 -- -D warnings
Finished `dev` profile [unoptimized + debuginfo] target(s) in 20m 11s
exit0, 1213.210s total.
$ cargo nextest run --profile ci -p rustfs-ecstore --features rio-v2 slow_disk_operation_warns_once_while_pending_and_preserves_result --no-fail-fast
Finished `test` profile [unoptimized + debuginfo] target(s) in 4m 34s
PASS [0.026s] rustfs-ecstore disk::disk_store::tests::slow_disk_operation_warns_once_while_pending_and_preserves_result
Summary [ 0.170s] 1 test run: 1 passed, 5943 skipped
exit0, 281.920s total; 5943 existing filter skips.
warning: linker stderr: ld: __eh_frame section too large (max 16MB) to encode dwarf unwind offsets in compact unwind table, performance of exception handling might be affected
This linker warning is retained; flags, profiles and allowances are unchanged.
54 tracked Cargo.toml files parsed without duplicate keys. git diff --check passed. No new assertions were introduced, so no new runtime mutation is claimed. Native compilation and the existing helper test do not substitute for actual Linux RIO execution; the new integration CI run still needs to recover before the RPC merge.
Preservation and output hashes:
parent file SHA256: 8e59ee745e3aaaa22848b943d3b10f6353b515956282cde1f3af2c394200c9ac
final file SHA256: b69ba3237061b161448ae8b62b615885d5770b98a5b4a2479bcfbd70f1bd1f17
unchanged production prefix SHA256: 7903a6e288252c420231242d7b58a25fa2a7140ea3dd430cbe7f0dde11369dcb
RIO Linux failure log SHA256:421addb9868fd88e4109a922311e54f91f0dfd70862f8ba7fcaba193482ed90b
Workspace Linux failure log SHA256:e0aea8d0f014c022252a9534968e083c98cfc1fd760da445ecd04ec3f522e26a
19 output SHA256:35a54f28d707956ed1edb29d492c4dfe8a9fe753d103b111b5a2f1b5c41c2b98
RIO Clippy output SHA256:d6046e3868f2b3992fe11cdd04e9e98eb50a87d1234d9a70dbfeb154ff61638d
slow-operation output SHA256:605b7b258d2fd09c6ef2ab2ffc90d433d0509480f1e45125d20c823759e83608
Refs rustfs/backlog#2734.
* test(ecstore): isolate tier-free recovery fixture startup
The fixture constructor can take the production recovery lock and wait for 60 seconds before this test installs its queue and seeds metadata. Disable its background recovery only while creating this fixture, then restore the environment before calling the existing production recovery entrypoint.
Only the existing test setup changes. Its production entrypoint, default lock parameters, original assertions, 30-second timeout, cancellation and cleanup remain intact. No tests or assertions were added. Production and store/init.rs are byte-identical to the parent.
Independent acceptance ran on merge tree 8ec358cb42ba9ebb3bcfeafcef048a3f0366355b; the runner header reports parent HEAD, while before/after index and source hashes confirm this candidate. Worker task branch d8055766b8 has the same tree.
Every command used LANG=en_US.UTF-8. Task compilation also used CARGO_BUILD_JOBS=1, CARGO_INCREMENTAL=0, CARGO_NET_OFFLINE=true, an independent target, and empty RUSTC_WRAPPER/RUSTC_WORKSPACE_WRAPPER to keep compilers in its owned process group without changing repository configuration.
Command:
shellcheck --version && actionlint
Exit 0; observed output excerpt:
version: 0.11.0
Log SHA-256: fc9eec16b4191d058a295813ca9afbe9d2b37b59975da860170726be162a3924
Command:
cargo fmt --all --check
Exit 0; observed output excerpt:
(no stdout or stderr)
Log SHA-256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
Command:
./scripts/check_unsafe_code_allowances.sh
Exit 0; observed output excerpt:
(no stdout or stderr)
Log SHA-256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
Command:
./scripts/check_layer_dependencies.sh
Exit 0; observed output excerpt:
Layer dependency guard passed (no new reverse dependencies/cycles).
Log SHA-256: 69c6cb1896c0586b28d1167380d3260d99d8017a4265c8df5a2348f289e93a12
Command:
./scripts/check_architecture_migration_rules.sh
Exit 0; observed output excerpt:
Architecture migration rules passed.
Log SHA-256: 28790e70d070607ed4de0704f292a65030be92ec224ef106d4b72fd8296f896e
Command:
./scripts/check_logging_guardrails.sh
Exit 0; observed output excerpt:
✅ Logging guardrails check passed
Log SHA-256: 7adf41abff8ed8f7617decf86498888ddde78937099dd9ce47f6d359dfc2d317
Command:
./scripts/check_error_other_format_ratchet.sh
Exit 0; observed output excerpt:
error(format!) ratchet passed (total call sites: 579).
Log SHA-256: 66172aef18f150ba0605af122e26ffcf91f6dd86f43e38b05e2e982ce8d783ee
Command:
./scripts/check_no_tokio_io_uring.sh
Exit 0; observed output excerpt:
(no stdout or stderr)
Log SHA-256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
Command:
./scripts/check_extension_schema_boundaries.sh
Exit 0; observed output excerpt:
Extension schema boundaries passed.
Log SHA-256: f13d0acd0e2ce5d9815241b12b4dab7d01cc11b9b65bcf6dfb7812c878532a51
Command:
./scripts/check_body_cache_whitelist.sh
Exit 0; observed output excerpt:
Body-cache whitelist guard passed.
Log SHA-256: 6b828847b0383950ee6eef9cd846d0c322c945e47efd4c7d1ad881178810d6bb
Command:
./scripts/check_s3s_footprint.sh
Exit 0; observed output excerpt:
s3s footprint OK: files importing s3s is 195 (baseline files=195)
s3s footprint OK: s3s s3_error! lines is 1584 (baseline s3_error_lines=1584)
s3s footprint OK: ecstore files referencing s3s is 36 (baseline ecstore_files=36)
✅ s3s footprint baseline check passed
Log SHA-256: 47f53d04ef645618563fe1f22abc64667f19078f1974feedcca46e4f2c016844
Command:
./scripts/check_test_count_ratchet.sh
Exit 0; observed output excerpt:
test-count ratchet OK: 52 crates, 21340 tests, 164 ignored match .config/test-count-baseline.txt
Log SHA-256: 618a5e330e933de0c6ff484546df401457024a88bb5986d11d588794e33dab60
Command:
./scripts/check_fips_wording.sh
Exit 0; observed output excerpt:
FIPS wording guard passed (README.md and CHANGELOG.md contain no forbidden claims).
KMS wording guard passed (crates/kms claims no Vault KV2 Transit wrapping).
Log SHA-256: 99221cc9979c480ec4f46f50d76c52b18c72fe499f12e15e4f4a22a1b8bfb140
Command:
./scripts/check_embedded_secrets.sh
Exit 0; observed output excerpt:
Embedded secret guard passed (no private key material or provider credential literal in the tree).
Log SHA-256: 437a3ceba32094d2901889965ad191e411aad32cdb18bf0b8659e818011c406b
Command:
make script-tests
Exit 0; observed output excerpt:
test_cli_requires_the_repository_default_branch (__main__.SelfTests.test_cli_requires_the_repository_default_branch) ... ok
test_config_rejects_duplicate_and_invalid_entries (__main__.SelfTests.test_config_rejects_duplicate_and_invalid_entries) ... ok
test_freshness_boundaries (__main__.SelfTests.test_freshness_boundaries) ... ok
test_grace_requires_two_successful_queries_with_no_history (__main__.SelfTests.test_grace_requires_two_successful_queries_with_no_history) ... ok
test_invalid_api_evidence_fails_closed (__main__.SelfTests.test_invalid_api_evidence_fails_closed) ... ok
test_report_retains_every_workflow (__main__.SelfTests.test_report_retains_every_workflow) ... ok
test_requests_filter_schedule_default_branch_and_success_on_server (__main__.SelfTests.test_requests_filter_schedule_default_branch_and_success_on_server) ... ok
Ran 10 tests in 0.042s
OK
Ran 30 tests in 20.627s
OK
Ran 10 tests in 2.268s
OK
Ran 33 tests in 0.127s
OK
Ran 7 tests in 0.002s
OK
Ran 7 tests in 0.001s
OK
Ran 4 tests in 0.108s
OK
Ran 3 tests in 0.564s
OK
PASS: cold stampede script self-test
Log SHA-256: 75fda2d069a373c63667b908baaf99e6591aab9099a739b0c4c22ceb6da8d68c
Command:
python3 ./scripts/test_object_generation_protocol_model.py
Exit 0; observed output excerpt:
test_an_unavailable_decision_quorum_cannot_choose_a_candidate (__main__.ObjectGenerationProtocolModel.test_an_unavailable_decision_quorum_cannot_choose_a_candidate) ... ok
test_counter_allocation_requires_persistence_and_is_consumed_once (__main__.ObjectGenerationProtocolModel.test_counter_allocation_requires_persistence_and_is_consumed_once) ... ok
test_counter_restart_burns_persisted_allocations_before_or_after_issue (__main__.ObjectGenerationProtocolModel.test_counter_restart_burns_persisted_allocations_before_or_after_issue) ... ok
test_crash_after_promise_before_reply_does_not_restore_an_older_ballot (__main__.ObjectGenerationProtocolModel.test_crash_after_promise_before_reply_does_not_restore_an_older_ballot) ... ok
test_duplicate_or_unconfigured_voters_cannot_form_a_quorum (__main__.ObjectGenerationProtocolModel.test_duplicate_or_unconfigured_voters_cannot_form_a_quorum) ... ok
test_empty_or_unobserved_minority_slot_can_retry_without_allocating_another_revision (__main__.ObjectGenerationProtocolModel.test_empty_or_unobserved_minority_slot_can_retry_without_allocating_another_revision) ... ok
test_equal_ballot_cannot_accept_different_bytes_and_lower_ballot_is_rejected (__main__.ObjectGenerationProtocolModel.test_equal_ballot_cannot_accept_different_bytes_and_lower_ballot_is_rejected) ... ok
test_forgetting_tombstone_authority_allows_an_aba_create (__main__.ObjectGenerationProtocolModel.test_forgetting_tombstone_authority_allows_an_aba_create) ... ok
test_issuing_before_persistence_reuses_a_ballot_after_restart (__main__.ObjectGenerationProtocolModel.test_issuing_before_persistence_reuses_a_ballot_after_restart) ... ok
test_lost_ack_and_restart_preserve_the_chosen_operation (__main__.ObjectGenerationProtocolModel.test_lost_ack_and_restart_preserve_the_chosen_operation) ... ok
test_minority_accepted_work_is_adopted_and_finished (__main__.ObjectGenerationProtocolModel.test_minority_accepted_work_is_adopted_and_finished) ... ok
test_omitting_highest_accepted_adoption_has_a_double_choice_counterexample (__main__.ObjectGenerationProtocolModel.test_omitting_highest_accepted_adoption_has_a_double_choice_counterexample) ... ok
test_quorum_choice_does_not_require_an_uncontacted_disk_to_learn_it (__main__.ObjectGenerationProtocolModel.test_quorum_choice_does_not_require_an_uncontacted_disk_to_learn_it) ... ok
test_recovery_failure_or_stale_expected_head_does_not_allocate_a_successor (__main__.ObjectGenerationProtocolModel.test_recovery_failure_or_stale_expected_head_does_not_allocate_a_successor) ... ok
test_replacing_voters_with_empty_state_can_choose_a_conflicting_value (__main__.ObjectGenerationProtocolModel.test_replacing_voters_with_empty_state_can_choose_a_conflicting_value) ... ok
test_skipping_predecessor_recovery_chooses_a_broken_revision_chain (__main__.ObjectGenerationProtocolModel.test_skipping_predecessor_recovery_chooses_a_broken_revision_chain) ... ok
test_stale_publication_rollback_and_gc_cannot_change_the_winner (__main__.ObjectGenerationProtocolModel.test_stale_publication_rollback_and_gc_cannot_change_the_winner) ... ok
test_superseded_operation_identity_cannot_be_reused_for_different_input (__main__.ObjectGenerationProtocolModel.test_superseded_operation_identity_cannot_be_reused_for_different_input) ... ok
test_superseded_operation_retry_returns_its_original_successor (__main__.ObjectGenerationProtocolModel.test_superseded_operation_retry_returns_its_original_successor) ... ok
test_tombstone_and_null_marker_payloads_do_not_reset_authority_revision (__main__.ObjectGenerationProtocolModel.test_tombstone_and_null_marker_payloads_do_not_reset_authority_revision) ... ok
test_tombstone_rejects_a_delayed_never_present_create (__main__.ObjectGenerationProtocolModel.test_tombstone_rejects_a_delayed_never_present_create) ... ok
test_unlearned_predecessor_is_recovered_before_the_second_slot (__main__.ObjectGenerationProtocolModel.test_unlearned_predecessor_is_recovered_before_the_second_slot) ... ok
Ran 24 tests in 0.929s
OK
Log SHA-256: 0b4be926efb0609d497ae58930340153f01957b606206cbbdb3e0d47eb29d3a2
Command:
python3 ./scripts/check_test_wiring.py
Exit 0; observed output excerpt:
OK: e2e modules, runner selection, fuzz matrices, profiles, scheduled alerts, and Quick Checks are wired
Log SHA-256: 21f676f458451525535ab0ef73d6a9c0d9774e16d76a4d19e396ad34936dd39c
Command:
./scripts/check_no_planning_docs.sh
Exit 0; observed output excerpt:
(no stdout or stderr)
Log SHA-256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
Command:
./scripts/check_uring_lane_lib_only.sh
Exit 0; observed output excerpt:
OK: no 'uring_'-matching test functions under crates/ecstore/tests; --lib is safe
Log SHA-256: 43f3f851e27ec2ab9e34d6b05fefdede38af992699c473904bfe6f53a46d52e3
Command:
cargo nextest run --profile ci -p rustfs-ecstore --lib --features rio-v2 -E 'test(=bucket::lifecycle::bucket_lifecycle_ops::tests::tier_free_version_recovery_production_entrypoint_enqueues_seeded_item)' --no-fail-fast
Exit 0; wall 234.399s.
Observed output:
Finished `test` profile [unoptimized + debuginfo] target(s) in 3m 49s
Starting 1 test across 1 binary (5871 tests skipped)
SETUP PASS [ 0.006s] ecstore-base-stack: sh -c 'echo RUST_MIN_STACK=4194304 >> "$NEXTEST_ENV"'
PASS [ 0.440s] rustfs-ecstore bucket::lifecycle::bucket_lifecycle_ops::tests::tier_free_version_recovery_production_entrypoint_enqueues_seeded_item
Summary [ 0.586s] 1 test run: 1 passed, 5871 skipped
Log SHA-256: 34e973f86e8dc2102c1a0e483e0ac802ce7df3d1b40a603932f62c0805c4de27
Command:
cargo nextest run --profile ci -p rustfs-ecstore --lib --features rio-v2 -E 'test(=bucket::lifecycle::bucket_lifecycle_ops::tests::tier_free_version_recovery_eight_coordinators_hold_one_leader_and_take_over) | test(=store::bucket::tests::creation_commit_backfill_runs_under_the_config_write_fence) | test(=store::bucket::tests::orphaned_bucket_recovery_requires_every_volume_absent_and_is_retryable)' --no-fail-fast
Exit 0; wall 4.010s.
Observed output:
Finished `test` profile [unoptimized + debuginfo] target(s) in 0.43s
Starting 3 tests across 1 binary (5869 tests skipped)
SETUP PASS [ 0.005s] ecstore-base-stack: sh -c 'echo RUST_MIN_STACK=4194304 >> "$NEXTEST_ENV"'
PASS [ 0.068s] rustfs-ecstore bucket::lifecycle::bucket_lifecycle_ops::tests::tier_free_version_recovery_eight_coordinators_hold_one_leader_and_take_over
PASS [ 0.637s] rustfs-ecstore store::bucket::tests::creation_commit_backfill_runs_under_the_config_write_fence
PASS [ 0.846s] rustfs-ecstore store::bucket::tests::orphaned_bucket_recovery_requires_every_volume_absent_and_is_retryable
Summary [ 0.987s] 3 tests run: 3 passed, 5869 skipped
Log SHA-256: b5da2a927ff18ca9aa217762f7186a6d4371bd9e8feedd7b77940e10b64dffac
Command:
cargo clippy -p rustfs -p rustfs-ecstore --all-targets --features rio-v2 -- -D warnings
Exit 0; wall 1084.508s.
Observed output:
Finished `dev` profile [unoptimized + debuginfo] target(s) in 18m 03s
Log SHA-256: 28f98be7970faaa71cd118da1da4e6e2e3d23b9dda7e6449eaa7a37e741260b5
Compiled mutation command (unchanged for each red and restored run):
cargo nextest run --profile ci -p rustfs-ecstore --lib --features rio-v2 -E 'test(=bucket::lifecycle::bucket_lifecycle_ops::tests::tier_free_version_recovery_production_entrypoint_enqueues_seeded_item)' --no-fail-fast
M1: reverse only this test-local constructor flag from false to true. The mutant compiled, then failed the original recovered-item queue assertion after 30.481s. Restoring the entire observed candidate passed in 0.460s. Red log 55bf3b1535529ac0bcfb8071f9b56f9f40631007a85c315367e7e93786d65293; restored log 204d5fbe4574a7917f5e740a795d864fe28a7e7fb5e33d27b0a9cf68e2f17f52.
M2: replace only the actual awaited enqueue with false, consuming its original inputs. The mutant compiled, then failed the same original queue assertion after 30.516s. Restoring the entire observed candidate passed in 0.457s. Red log 89186330413215e231e2b06bf013cc31097ea8f4a106d91421a36d6c8986a7fd; restored log e848b42d2302401afa5dbcd9e37bf76d2e56a96ab6de5a488513b997f53c5e9b.
All temporary observation and mutation code was removed before final acceptance. Historical baseline and diagnostic failures remain recorded; they are not mutation credit. The separate Object Lock intent-retry diagnostic still fails, and these native macOS checks do not establish Linux CI recovery.
* refactor(rpc): serve internode rpc bodies with the owned Body (#8441)
* test(rpc): pin the wire bytes of every internode rpc response body
rustfs/backlog#2747 batch 4, step 1 of 2, on the s3s bodies the rpc paths
serve today. Each response body builder in storage/rpc/http_service.rs is
served through hyper's HTTP/1.1 server and the raw bytes the peer reads are
compared with goldens captured from this commit: status, content-length or
chunked framing, chunk boundaries, the terminating chunk, HEAD, and the cut
connection a failure after data produces. The capability responses carry a
per-process proof, so their length header and payload are checked against
the body they encode instead.
The read_file_stream response builder moves into read_file_stream_response
so the test reaches the same code the handler runs; behaviour is unchanged.
* refactor(rpc): serve internode rpc bodies with the owned Body
rustfs/backlog#2747 batch 4, step 2 of 2. The internode rpc HTTP paths
build their responses with rustfs_s3_types::Body: Body::from_stream
replaces Body::from(StreamingBlob::wrap(..)) at the six stream sites, and
the in-memory and empty bodies keep their constructors. InternodeRpcService
converts its responses into the wrapped service's response body through
From<Body>, so the server keeps composing it with the s3s service as
before. The wire-byte goldens from the previous commit pass unchanged;
storage/rpc no longer names s3s.
* fix(rpc): simplify wire test response builder type
Name the boxed response builder used by the capability wire test so
all-target Clippy can check it without a type-complexity warning.
Validation (LANG=en_US.UTF-8, CARGO_INCREMENTAL=0):
- cargo clippy -p rustfs --all-targets -- -D warnings: exit 0.
- cargo nextest run --profile ci -p rustfs --lib -E 'test(/storage::rpc::/)': 279 tests run, 279 passed, 4842 skipped.
- All 19 quick-check commands, including script-tests, both Python scripts and cargo fmt --all --check: exit 0.
- Footprint baseline regenerated: files=194, s3_error_lines=1584, ecstore_files=36; test-count baseline regenerated: 52 crates, 21228 tests, 164 ignored.
- All 54 Cargo.toml files parse without duplicate keys.
No assertion was added or changed by this lint fix. R1-R8 are retained
in the existing PR verification record.
* fix(ecstore): resume fenced Object Lock creation intents
* fix(ecstore): bound creation retry futures on the stack
* fix(ecstore): require native proof for creation retries
* fix(ecstore): bound bucket creation stack usage (#8466)
* fix(ecstore): retain creation fences through rollback
Keep lifecycle, metadata-transaction, and namespace guards around rollback and metadata cleanup. Preserve the original publication error when a guard is lost. Build the timestamp retry fixture through the real interrupted producer so it checks a valid pending creation.
Validated source tree: 4e8e207999.
Two independent source reviews found no blocking regression.
Validation (LANG=en_US.UTF-8):
- bash -c 'shellcheck --version && actionlint' — exit 0
- cargo fmt --all --check — exit 0
- ./scripts/check_unsafe_code_allowances.sh — exit 0
- ./scripts/check_layer_dependencies.sh — exit 0
- ./scripts/check_architecture_migration_rules.sh — exit 0
- ./scripts/check_logging_guardrails.sh — exit 0
- ./scripts/check_error_other_format_ratchet.sh — exit 0
- ./scripts/check_no_tokio_io_uring.sh — exit 0
- ./scripts/check_extension_schema_boundaries.sh — exit 0
- ./scripts/check_body_cache_whitelist.sh — exit 0
- ./scripts/check_s3s_footprint.sh — exit 0
- ./scripts/check_test_count_ratchet.sh — exit 0
- ./scripts/check_fips_wording.sh — exit 0
- ./scripts/check_embedded_secrets.sh — exit 0
- make script-tests — exit 0
- python3 ./scripts/test_object_generation_protocol_model.py — exit 0
- python3 ./scripts/check_test_wiring.py — exit 0
- ./scripts/check_no_planning_docs.sh — exit 0
- ./scripts/check_uring_lane_lib_only.sh — exit 0
- cargo nextest run --profile ci -p rustfs-ecstore --lib --locked --no-fail-fast — 5894 passed, 0 skipped
- cargo check -p rustfs-ecstore --all-targets --locked — exit 0
- cargo nextest run --profile ci -p rustfs --lib --locked --no-fail-fast -E 'test(=admin::handlers::table_catalog::tests::cold_strong_registration_completes_while_other_bucket_backup_waits) | test(=admin::handlers::table_catalog::tests::commit_backend_borrowed_put_preserves_owner_authorization_and_precondition) | test(=admin::handlers::table_catalog::tests::disable_table_bucket_preserves_objects_and_can_be_reenabled) | test(=admin::handlers::table_catalog::tests::load_table_handler_negotiates_vended_credentials_without_breaking_metadata_only_callers) | test(=admin::handlers::table_catalog::tests::namespace_property_handler_updates_object_backed_catalog_and_maps_errors) | test(=app::object::get::tests::execute_get_object_resumes_from_relocated_pool_without_splicing_body) | test(=table_catalog::store::tests::catalog_writer_loss_prevents_recovery_storage_commit)' — 7 passed
- cargo nextest run --profile ci -p rustfs --lib --locked --no-fail-fast -E 'test(unsupported_remote_disk_)' — 2 passed
Additional fault-injection evidence remains incomplete. Timestamp M1 reached its original assertion; its restore run is being collected. M2–M5 and the three planned rollback faults have not run. An automated risk check interrupted the timestamp proof task; no assertions or validation settings were weakened.
* fix(ecstore): box nested bucket rollback state
Keep the guarded rollback future behind a heap allocation so bucket creation does not store its nested state inline. The rollback body, all three fence references and pending-proof revocation order are unchanged.
The same three original Linux tests overflowed on both e611 CI attempts. Local format, ecstore test compilation and all 19 quick-check steps pass; the original Linux feature lanes still need to validate this candidate.
* test(ecstore): isolate rollback guard loss fixtures
Run the three rollback metadata-cleanup cases in separate test processes so reused /proc/self/fd aliases from earlier cases cannot invalidate their creation witnesses. Keep the original fixture, guards, assertions and io_uring test selection, and verify that each child runs the intended case successfully.
Update the generated test-count baseline for the added macro arm.
Validation: cargo check -p rustfs-ecstore --tests; cargo fmt --all --check; all 19 quick-check commands, including make script-tests and the two Python checks. Linux io_uring acceptance and fault controls remain pending.
* refactor(replication): move the replication config onto gateway types (#8443)
* wip: T1.4 replication family
* refactor(replication): move the replication config onto gateway types
The replication configuration family leaves s3s: rustfs-replication no longer depends on s3s, and ECStore bucket metadata, the scanner and the rustfs app/admin layers hold rustfs_gateway_types::persistence::PersistedReplicationConfiguration. The persisted XML goes through the gateway persistence codec behind a new BucketConfigXml trait (the 39 byte goldens pass with unchanged fixtures); the legacy s3s edge converts once in storage/ecfs.rs through storage/s3_compat/replication.rs; the scanner data usage cache keeps its MessagePack bytes through a serde mirror pinned by bytes recorded from s3s 5761ddfe. rustfs-gateway-types 0.32.0 is pinned once (rev 772d3385). Refs rustfs/backlog#2745.
* test: use S3 serializer for list response tests
* fix(ci): restore integration test compilation
Gate the diagnostic test import with its feature, initialize the conditional
MetadataWriteStore fields explicitly, and decode the scanner replication fixture
through its existing cache boundary. Preserve the original fixture and assertions.
Validation was run on tree 544acb80c0 at 6003d4b8.
LANG=en_US.UTF-8; no source changes follow validation.
Quick checks: 19/19 passed.
- shellcheck --version && actionlint: exit 0
- cargo fmt --all --check: exit 0
- ./scripts/check_unsafe_code_allowances.sh: exit 0
- ./scripts/check_layer_dependencies.sh: exit 0
- ./scripts/check_architecture_migration_rules.sh: exit 0
- ./scripts/check_logging_guardrails.sh: exit 0
- ./scripts/check_error_other_format_ratchet.sh: exit 0
- ./scripts/check_no_tokio_io_uring.sh: exit 0
- ./scripts/check_extension_schema_boundaries.sh: exit 0
- ./scripts/check_body_cache_whitelist.sh: exit 0
- ./scripts/check_s3s_footprint.sh: exit 0
- ./scripts/check_test_count_ratchet.sh: exit 0
- ./scripts/check_fips_wording.sh: exit 0
- ./scripts/check_embedded_secrets.sh: exit 0
- make script-tests: exit 0
- python3 ./scripts/test_object_generation_protocol_model.py: exit 0
- python3 ./scripts/check_test_wiring.py: exit 0
- ./scripts/check_no_planning_docs.sh: exit 0
- ./scripts/check_uring_lane_lib_only.sh: exit 0
Darwin validation:
- cargo clippy --locked -p rustfs-ecstore -p rustfs-scanner --all-targets -- -D warnings: exit 0
- cargo nextest run --locked -p rustfs-ecstore -p rustfs-scanner --lib -E 'test(config::com::tests::save_config_preserves_retryable_system_volume_errors) | test(scanner_io::tests::replication_maintenance_refreshes_compacted_stats_across_cycles_and_retries)' --test-threads 1 --no-fail-fast: exit 0, 2/2 passed
- cargo clippy --locked -p rustfs-ecstore -p rustfs-scanner --all-targets --features rustfs-ecstore/diagnostic-read-version-caller,rustfs-scanner/diagnostic-read-version-caller -- -D warnings: exit 101
- cargo nextest run --locked -p rustfs-ecstore -p rustfs-scanner --lib --features rustfs-ecstore/diagnostic-read-version-caller,rustfs-scanner/diagnostic-read-version-caller -E 'test(config::com::tests::save_config_preserves_retryable_system_volume_errors) | test(scanner_io::tests::replication_maintenance_refreshes_compacted_stats_across_cycles_and_retries) | test(cluster::rpc::runtime_sources::tests::) | test(config::com::tests::diagnostic_config_write_retains_origin_and_api_for_results_and_cancellation)' --test-threads 1 --no-fail-fast: exit 0, 14/14 passed
Diagnostic strict Clippy remains red on four unchanged production lints:
runtime_sources.rs if_same_then_else and three redundant_locals reports in
set_disk/core/metadata_read.rs and set_disk/read.rs. Their source ranges and
feature wiring match the base; the base was not compiled separately.
Linux aarch64 validation (Rust/Clippy 1.99.0, UID 501, jobs=1):
- cargo clippy --all-targets -- -D warnings: exit 0; Finished dev profile in 68m 12s
- cargo clippy -p rustfs -p rustfs-ecstore --all-targets --features rio-v2 -- -D warnings: exit 0; Finished dev profile in 49m 45s
Source HEAD/tree and clean status matched before and after both commands.
No compiler or stack overrides were used. protoc 35.1 was installed from its
verified ARM64 release; flatc was absent and not invoked by the workspace build scripts.
The local environment differs from hosted CI.
* fix(ecstore): recover delete batches with missing versions
Treat an explicit non-null version as a no-op when it is absent from both recovery sources. Keep rejecting missing null versions, delete-marker mismatches and conflicting local generations.
Add returning-disk and partial-receipt regressions, extend journal boundary checks, and regenerate the ecstore test-count baseline. Production faults exercise the 24 added assertion sites and 7 semantic checks: 33 faults failed at the expected sites and passed after full restoration.
Validation (LANG=en_US.UTF-8, one build job):
cargo nextest run --locked -p rustfs-ecstore --lib -E 'test(delete_recovery) | test(quorum_delete_cleanup_coordinator_binds_single_and_batch) | test(crash_delete_retry_batch_reclaims_inline_and_reserved_external)' --test-threads 1 --no-fail-fast
Summary [ 34.580s] 69 tests run: 69 passed, 6058 skipped; exit 0
cargo clippy --locked -p rustfs-ecstore --all-targets -- -D warnings
Finished `dev` profile [unoptimized + debuginfo] target(s) in 5m 37s; exit 0
All 19 final Quick Checks completed with exit 0:
- shellcheck --version && actionlint
- cargo fmt --all --check
- ./scripts/check_unsafe_code_allowances.sh
- ./scripts/check_layer_dependencies.sh
- ./scripts/check_architecture_migration_rules.sh
- ./scripts/check_logging_guardrails.sh
- ./scripts/check_error_other_format_ratchet.sh
- ./scripts/check_no_tokio_io_uring.sh
- ./scripts/check_extension_schema_boundaries.sh
- ./scripts/check_body_cache_whitelist.sh
- ./scripts/check_s3s_footprint.sh
- ./scripts/check_test_count_ratchet.sh
- ./scripts/check_fips_wording.sh
- ./scripts/check_embedded_secrets.sh
- make script-tests
- python3 ./scripts/test_object_generation_protocol_model.py
- python3 ./scripts/check_test_wiring.py
- ./scripts/check_no_planning_docs.sh
- ./scripts/check_uring_lane_lib_only.sh
test-count ratchet OK: 52 crates, 22009 tests, 165 ignored match .config/test-count-baseline.txt
PASS: cold stampede script self-test
model: states=51023, transitions=167520, terminal=732, chosen_candidates=2
multi-slot round schedules: 640
OK: e2e modules, runner selection, fuzz matrices, profiles, scheduled alerts, and Quick Checks are wired
* fix(ecstore): retry tier commit intent lock contention
Retry transient namespace lock timeouts within the existing three-attempt
CAS budget. Reload and verify the original mutation identity after
contention, and preserve the final typed error when attempts are exhausted.
Cover save/read contention, exhausted retries, replacement identities and
CAS reconciliation without changing the existing timeout or lock policy.
Validation:
- cargo nextest run --locked -p rustfs-ecstore --features test-util --lib -E 'test(tier_mutation)' --test-threads 1 --no-fail-fast: 52 passed.
- cargo nextest run --locked -p rustfs-ecstore --features test-util --lib -E 'test(intent_advance_) | test(=services::tier::tier::tests::caller_cancellation_after_durable_prepare_does_not_hide_the_mutation)' --test-threads 1 --no-fail-fast: 5 passed.
- cargo clippy --locked -p rustfs-ecstore --all-targets -- -D warnings: passed.
- cargo clippy --locked -p rustfs-ecstore --all-targets --features test-util -- -D warnings: passed.
- 27 mutation pairs failed at the intended assertions and passed after restoring the production code.
- bash scripts/check_test_count_ratchet.sh --update, then bash scripts/check_test_count_ratchet.sh: 52 crates, 22013 tests, 165 ignored; ecstore increases from 6581 to 6585.
* refactor(lifecycle): replace s3s domain types with Gateway DTOs
Keep lifecycle rules, restore requests, Object Lock, and scanner cached tags in domain types. Convert at the existing HTTP boundary while preserving legacy XML, scanner cache bytes, date ordering, and tombstones.
Linux test validation before the final import-path-only adjustment:
cargo nextest run -p rustfs-ecstore bucket_config_goldens
39 passed; 6211 skipped by selector.
cargo nextest run --lib -p rustfs -E 'test(/test_zero_rule_lifecycle_tombstone_recognition/) | test(/test_local_lifecycle_staleness_axis_selection/) | test(/test_lifecycle_staleness_preserves_legacy_submillisecond_axis/) | test(/test_merge_incoming_lifecycle/) | test(/test_lifecycle_expiry_statement_matrix/) | test(/test_lifecycle_expiry_subset_xml/) | test(/put_bucket_lifecycle_validation_errors_keep_their_s3_code/) | test(/execute_put_bucket_lifecycle_configuration_rejects_missing_configuration/) | test(/execute_restore_object_rejects_missing_restore_request/) | test(/execute_restore_object_rejects_select_type/) | test(/execute_restore_object_maps_failures_to_typed_s3_errors/) | test(/execute_restore_object_returns_internal_error_when_store_uninitialized/) | test(/restore_request_conversion_preserves_legacy_output_values/) | test(/restore_request_conversion_preserves_presence_and_raw_enums/) | test(/restore_request_conversion_distinguishes_absent_and_empty_containers/)'
25 passed; 5318 skipped by selector.
cargo nextest run --profile ci -p rustfs-ecstore -p rustfs-lifecycle -p rustfs-replication -p rustfs-scanner
7872 passed; 21 skipped under the existing CI profile.
Previously verified mutation controls: 39 production faults, 41 RED/GREEN pairs.
cargo clippy --locked -p rustfs -p rustfs-ecstore -p rustfs-lifecycle -p rustfs-scanner --all-targets -- -D warnings
Passed in 40m 45s before the import-path adjustment, then in 1m 59s on
the final source with warnings denied. The final adjustment imports the
existing helper through the object domain facade; fmt and the architecture
check also passed.
Part of rustfs/backlog#2745. The remaining ecstore configuration families follow separately.
* refactor(admin): move error types to rustfs-s3-types (#8440)
* refactor(protocols): move error types to rustfs-s3-types by codemod
T1.2 batch 3 (rustfs/backlog#2743): s3select-api, s3select-query and
the error items of protocols move off s3s by scripts/codemods, run with
--hold crates/protocols/src/sftp/errors.rs. That file stays on s3s: it
downcasts the backend error, whose runtime type is still s3s::S3Error.
- rustfs/src/protocols/client.rs: the two webdav session methods now
return rustfs_s3_types::S3Result and convert the s3s error with From.
- codemod: a child module that calls s3_error! through `use super::*`
gets its own `use rustfs_s3_types::s3_error;` once its parent imports
that macro by name; otherwise it is listed as GLOB.
- s3-types: a table test pins code, status and the s3s error the legacy
edge renders for every variant a moved call site names.
- s3_error_lines 1585 -> 1584 (a lifecycle comment reworded).
* refactor(admin): move error types to rustfs-s3-types (WIP, no build)
T1.2 batch 4 (rustfs/backlog#2743), work in progress: rustfs check
still fails with 16 errors (listed in the PR body).
- codemod: --facade PATH treats a crate-local re-export of s3s's error
items like s3s for the four names (9 new self-tests, 9 mutants red).
- codemod output for rustfs/src/admin and rustfs/src/site_replication
(--facade crate::storage_api::site_replication::s3); 6 glob children
get an explicit rustfs_s3_types::s3_error import.
- admin/router.rs: the Operation trait returns rustfs_s3_types::S3Result.
- error.rs: From<ApiError> builds the rustfs-s3-types error; the s3s
conversion is derived from it.
* fix(admin): convert error types at the s3s boundary
Batch 4 of T1.2 left the admin and site_replication modules returning
rustfs_s3_types::S3Error while their callers and callees on the storage
and auth side still speak s3s::S3Error. Convert at each edge with the
existing From impls in rustfs-s3-types (compat_s3s.rs) instead of
changing any code, message or status:
- admin/auth.rs, admin/handlers/object_zip_download.rs,
admin/handlers/table_catalog/mod.rs: map the s3s result returned by
check_key_valid / authorize_request / authorize_internal_object_request
into the admin error type with .map_err(Into::into).
- storage/ecfs.rs: the s3s-facing side maps the admin site_replication
gate results the other way.
- admin/handlers/pools.rs: ApiError.code stays s3s::S3ErrorCode, so the
rustfs code is converted with .into() when the struct is built, in
production and in the two tests that build or compare it.
- admin/handlers/heal.rs, admin/runtime_sources.rs: the router returns
s3s errors (S3Route impl); the expected rustfs code is converted with
.into() before the comparison.
- admin/handlers/table_catalog/mod.rs: iceberg_rest_error takes
&'static str so the Cow<'static, str> custom code borrows it; every
caller already passes an ICEBERG_ERROR_* constant or a &'static str.
- admin/handlers/quota.rs: drop a now same-type .into() after s3_error!.
- storage_api.rs, site_replication/mod.rs: drop re-exports and an import
that no longer have users (clippy -D warnings).
cargo check -p rustfs --all-targets and cargo clippy -p rustfs
--all-targets -- -D warnings are clean; the s3s footprint baseline is
unchanged at files=172, s3_error_lines=303, ecstore_files=28.
Refs rustfs/backlog#2743, rustfs/backlog#2734.
* refactor(ecstore): move bucket configs onto gateway persisted types (#8709)
* refactor(ecstore): move bucket configs onto gateway persisted types
Finish the ECStore batch of T1.4 (rustfs/backlog#2745). The remaining
eleven bucket configuration families (Website, CORS, Tagging, Versioning,
ObjectLock, Encryption, Notification, Logging, Accelerate, RequestPayment,
PublicAccessBlock) now read and write their stored XML through the
rustfs-gateway-types persistence and cors_tagging codecs instead of the
s3s codec, so the legacy s3s bucket-config codec and its
s3gate-metadata-xml compat marker are removed.
- ecstore: BucketConfigXml is implemented by the gateway codec for every
family; metadata, metadata_sys, versioning, object lock, replication and
lifecycle consumers take the persisted types. Region moves to
rustfs_s3_types::Region, and the ETag conversion to s3s moves to the
rustfs edge with its tests. Only warm_backend.rs and warm_backend_s3.rs
still reference s3s, held for T1.5 (rustfs/backlog#2746).
- scanner: object lock cache, item actions and data usage use the
persisted types.
- rustfs: the legacy S3 edge converts each family between s3s DTOs and the
persisted types in storage/storage_api.rs by writing one codec's XML and
reading it with the other; Tagging converts field by field.
- Bump the rustfs-gateway-types pin to 21283e87.
The 39 persisted-byte goldens are unchanged and pass, so a document the
old codec wrote reads back and re-encodes to the same bytes. s3s footprint:
files 181 -> 146, ecstore_files 28 -> 2, s3_error_lines unchanged.
* test(app): prepare the PutBucketEncryption config from the request
The uninitialized-store test passed an empty persisted configuration, which the use case now validates before it reaches the store check, so it answered MalformedXML instead of InternalError. Convert the request's configuration the way the S3 edge does.
* style(app): format the PutBucketEncryption test
* refactor(s3-client): move remote error codes and DTO enums off s3s (#8783)
Part of T1.5 (rustfs/backlog#2746), batch 1.
* refactor(s3select): run queries on gateway DTO request shapes (#8794)
* refactor(s3select): run queries on an engine request built from gateway DTO shapes
Part of T1.5 (rustfs/backlog#2746), batch 2.
* fix(s3select): keep the select request conversion in the app layer
The layer guard forbids app -> storage; the conversion moves from storage/s3_compat/select.rs to app/select_request.rs unchanged.
* fix(heal): retry transient metadata conflicts (#8812)
* fix(heal): retry transient metadata conflicts
* chore: update heal test count baseline
* test: reserve nextest capacity for progress probes (#8818)
* test(e2e): use a non-default region for the differential negative control
Since main's region-aware CreateBucket and GetBucketLocation (#8739, #8743), us-east-1 answers an empty LocationConstraint and 200 for an owned re-create, so the control no longer reported its own region. Also refresh the e2e-full linux selection from the CI listing (682 tests).
* refactor(protocols): move StorageBackend to gateway types (#8825)
* refactor(protocols): move storage backend to gateway types
* test: update e2e test count baseline
* fix: preserve copy source keys and unknown lengths
* test: update protocols test count baseline
* fix(protocols): preserve typed SFTP backend errors
* test(protocols): use gateway list input in policy tests
* fix(protocols): clear clippy failures after T1.5 merge (#8831)
fix(protocols): clear clippy failures after merge
* fix(ecstore): finalize restore metadata after local copy-back (#8836)
* fix(ecstore): finalize restore metadata after local copy-back
* test(ecstore): ratchet added restore finalization cases
* fix(ecstore): keep restore copyback compatible with older peers
* fix(ecstore): gate restore copyback on fleet compatibility
* fix(ecstore): retain transition identity during restore copyback
* fix(ecstore): route committed restore copyback locally
* fix restore copyback cleanup
* fix(ecstore): honor restore probe test fallback
* refactor(storage): migrate option errors to domain types (#8839)
* refactor(s3): migrate private multipart validation errors (#8844)
* refactor(auth): derive policy conditions from AuthzFacts (#8442)
* refactor(auth): extract authorization facts
* test(e2e): update smoke selection baseline
* test(e2e): update full selection baseline
* refactor(admin): migrate health and prefix usage to owned requests (#8887)
* refactor(admin): add owned health router boundary
* refactor(admin): migrate prefix usage to owned request
* refactor(admin): complete the owned admin request boundary (#8949)
* refactor(admin): complete the owned admin request boundary
Every type implementing the owned admin `Operation` now serves the legacy
router through one blanket impl that converts the s3s request and response
at that single edge, so handlers register with `AdminOperation(&Handler {})`
and need no adapter statics; `LegacyAdminAdapter` is removed.
`AdminRequest` gains the accessors handlers read today (`remote_addr`,
`client_info`, `request_context`, `req_info`, `req_info_mut`,
`install_req_info`, `envelope_mut`, `take_body`), the request helpers gain
envelope forms (`authenticate_envelope_credentials`,
`object_store_from_envelope`, the table-catalog `*_from_envelope` helpers,
`KmsAdminAudit::from_envelope`, `AccountAuditContext::from_envelope`,
`CallerIdentity::resolve_envelope`), `read_compatible_admin_body` accepts
either body type, and the owned `Body` gains `store_all_limited` and
`Default`. `admin::operation::test_support::TestRequest` builds owned
requests for handler tests. Unused accessors carry a per-method dead-code
expectation that each migrating handler batch removes.
No handler behavior changes; the remaining migrations follow in batches.
* chore(admin): add the T1.12 admin handler codemod
The script rewrites an admin handler file from the s3s request and response
types to the owned boundary (imports, signatures, request field and helper
calls, body access, response construction, and test request literals) and
leaves a TODO comment at every site it cannot rewrite mechanically. It is
idempotent and is removed once T1.12 completes.
* refactor(admin): migrate replication and KMS handlers to owned requests (#8951)
Move the T1.12 batch 2 admin handlers (site_replication, kms_keys, event,
replication, diagnostics, on_demand_migration, audit, account, kms_rekey,
kms_key_lifecycle, object_data_cache, tls_debug, integrity) from the s3s
request/response types to the owned AdminRequest/AdminResponse boundary.
Handler logic, authorization order, error codes and response bytes are
unchanged; the request/response plumbing now goes through the envelope
accessors and helpers introduced by the owned boundary.
The shared response helpers (utils::json_response, utils::empty_response,
handlers::admin_json_response, target_descriptor::build_json_response,
profile::authorize_profile_request) and the site replication repair
executor still answer with the s3s types, so the migrated files carry
owned-boundary forms of those helpers and convert the repair response at
its call edge. The streaming console-log body and the two devnull drains
move from the s3s byte stream to the owned Body stream/frame API.
Drop the dead_code expectations for the owned accessors and envelope
helpers this batch starts using, and lower the s3s footprint baseline to
the measured 99 files.
Part of rustfs/backlog#2788 (rustfs/backlog#2734 T1.12).
* refactor(admin): add owned forms of the shared admin response helpers
Handler batches were copying `json_response`, `empty_response`,
`admin_json_response`, and `build_json_response` into each migrated file
because the shared helpers still return the s3s response. Provide owned
forms next to each legacy helper (`owned_json_response`,
`owned_empty_response`, `owned_admin_json_response`,
`owned_build_json_response`, `authorize_profile_envelope`) with parity
tests against the legacy shape, derive `Debug` for `AdminResponse` so
tests can use `expect_err`, and fix three codemod rules: strip owned
string conversions in test credentials, annotate non-literal URI parses,
and leave a rebound request parameter immutable.
* refactor(admin): use the shared owned response helpers in batch 2 (#8954)
Replace the per-file owned-boundary copies of json_response,
empty_response, admin_json_response, build_json_response and
authorize_profile_request in the batch 2 admin handlers with the shared
owned_* helpers and authorize_profile_envelope that the boundary branch
now provides, and restore expect_err in the handler tests now that
AdminResponse implements Debug; the `.err().expect(..)` stand-ins were
failing the base branch's clippy step (clippy::err_expect under
-D warnings). The site replication repair call edge
keeps its conversion because the repair executor still answers with the
s3s response type.
Drop the dead_code expectations on the shared owned helpers this batch
now uses in non-test code.
Part of rustfs/backlog#2788 (rustfs/backlog#2734 T1.12).
* fix(admin): preserve legacy body limit error responses
* refactor(admin): move policy, ILM and batch 3 handlers to owned requests (#8956)
Batch 3 of T1.12 (rustfs/backlog#2788, parent rustfs/backlog#2734) moves
14 admin handler files to AdminRequest/AdminResponse and the owned
Operation trait: ilm_transition, policies, idp_compat, config_admin,
plugins_instances, profile_admin, group, heal, object_zip_download,
kms_key_metadata, durability, inspect_archive, sts and
gateway_key_inventory. Handler logic, authorization order, error codes
and response bytes are unchanged; test request literals use
test_support::TestRequest.
The handlers use the shared owned helpers (owned_json_response,
owned_admin_json_response, owned_build_json_response,
authorize_profile_envelope). The dead_code expectations on the
AdminRequest accessors and RequestEnvelope::uri_mut they now exercise
are removed. The legacy target_descriptor::build_json_response keeps
only its equivalence test caller and carries the matching expectation
until it retires.
object_zip_download builds the s3s request the S3 gate still reads from
the owned request facts; heal authenticates through
authenticate_envelope_credentials; the trace, inspect-archive and zip
bodies use Body::from_stream. extract_plugin_instance_filters takes the
URI so extensions.rs keeps calling it, and the two idp_compat
delegations stay on the legacy trait until their delegates migrate.
The s3s footprint baseline drops from 99 to 90 files.
* test(admin): anchor the replication diff body contract on the call
The source-contract test looked for `let body = read_compatible_admin_body(`
on one line; rustfmt wrapped that binding after the handler moved to the
owned request boundary, so the marker was no longer found. Anchor on the
call itself so formatting cannot change what the contract checks.
* refactor(admin): migrate table catalog and batch 1 to owned requests (#8955)
* fix(admin): preserve catalog body limit response bytes
* refactor(admin): migrate identity and cluster handlers to owned requests (#8966)
* refactor(admin): migrate identity and cluster handlers to owned requests
Move the T1.12 batch 4 admin handlers (user, service_account, oidc, tier,
system, pools, quota, kms_dynamic, kms_backup, rebalance, module_switch,
bucket_meta, cluster_snapshot, realtime, bucket_recovery, profile) from the
s3s request/response types to the owned AdminRequest/AdminResponse
boundary. All 90 impl Operation blocks now take AdminRequest and return
AdminResponse; handler logic, authorization order, error codes and response
bytes are unchanged. The handlers use the shared owned helpers
(owned_json_response, owned_admin_json_response, authorize_profile_envelope,
the envelope-form runtime sources and audit constructors) instead of local
copies.
profile.rs drops the s3s forms of authorize_profile_request and
profile_not_implemented_response now that profile_admin.rs is on the
envelope gate too; profile_not_implemented_response answers with an
AdminResponse. The realtime metrics stream moves from the s3s ByteStream
adapter to Body::from_stream, and the rebalance idle-404 document still
renders through s3s::S3Error::to_http_response with its body converted at
the edge.
The idp_compat delegators ImportIamV2 and AddServiceAccountLdap move to the
owned trait as well: with their targets on both traits, the method-call
delegation became ambiguous between the owned impl and the router blanket
impl, so they now take the owned request directly.
KmsAdminAudit::from_request loses its last production caller with this batch
and is deleted; its comparison test now asserts the envelope facts directly.
Lower the s3s footprint baseline to the measured 68 files.
Part of rustfs/backlog#2788 (rustfs/backlog#2734 T1.12).
* refactor(admin): remove the legacy admin helpers batch 4 superseded
Batch 4 moved the last production callers of the s3s-typed admin helpers
to their owned forms, which left `admin_json_response`, `json_response`,
`authenticate_request`, `authorize_admin_request`, `object_store_from_req`
and `object_store_from_extensions` unused, along with the `Body`,
`S3Request` and `S3Response` re-exports of the admin `s3` facade. Remove
them. The owned forms keep their documentation; the behavior tests of the
removed `json_response` now exercise `owned_json_response`, and the
compat-prefix test asserts the owned response directly.
Two tests existed only to compare a removed legacy helper with its owned
twin (`owned_json_response_matches_the_legacy_shape`) or to cover the
removed legacy gate (`authorize_admin_request_without_credentials_is_rejected`);
they go with the helpers, and the owned equivalents stay. The rustfs
test-count row drops from 5957 to 5955 and the s3s footprint from 68 to 66
files.
* refactor(admin): drop the legacy target descriptor JSON response helper (#8968)
`target_descriptor::build_json_response` returned the s3s response and
had no production caller left once the event, audit and plugin instance
handlers moved to `owned_build_json_response`; it survived only for the
legacy/owned equivalence test. Remove it, give the owned helper its own
documentation, and make the test assert the owned response directly:
status, `application/json`, the echoed `x-request-id` when present and
only then, and the body bytes. The file no longer imports s3s, so the
footprint baseline drops from 66 to 65 files.
* test(ecstore): pin native history on cancellation under the cancellable queue
main moved the local write path onto the disk blocking queue, which skips an
operation once its waiter is dropped, so a borrowed write cancelled while it is
queued no longer creates its file. The integration-only native publication
test still expected the queued open to run. Assert the new execution model
instead (the open never runs) while keeping its safety claims: the Started
record is taken before queueing and the cancelled publication never regrants
native history.
* chore(gateway): raise test-count rows for tests merged from main
* test(s3select): use the owned JSON input types in merged planner tests
* fix(ecstore): scope the merged direct-writer test hook to test builds
* fix(ecstore): create parents before admitted truncate opens
Align the heal tests with removal-disabled preservation and local-only outcome
fields, and refresh the e2e-full selection for the synced tests.
* chore(gateway): update test-count baseline after main sync
Main #9241 removed the node pool-metadata readiness cache with 8 tests
and added 6, so the rustfs row drops from 6440 to 6438. The ecstore and
s3select rows rise for tests added by #9237, #9239 and #9241.
* feat(gateway-adapter): add IAM-backed CredentialProvider
Add the ring-2 crate rustfs-gateway-adapter with IamCredentials, the
gateway CredentialProvider answered by RustFS IAM (rustfs/backlog#2760,
credentials half; PolicySource is deferred).
- Pin rustfs-gateway at the existing gateway rev with
default-features = false so the server crate stays out of the build.
- IamKeyLookup is a narrow IAM trait implemented for IamSys so the
adapter is testable without an in-memory IAM backend.
- Root credentials are injected at construction and resolve before IAM
is ready.
- Mapping: keystone: prefix -> NotFound (deliberate change from the
legacy empty secret); IAM not ready or not initialized -> Unavailable;
unknown key or NoSuch{User,ServiceAccount,TempAccount,Account} ->
NotFound; other IAM errors -> Backend; invalid or "off" principals
resolve disabled; STS credentials carry a SessionBinding, and an STS
credential without an expiry is a Backend fault rather than a
never-ending session.
- No authorization in lookup; no Debug on secret-holding types; IAM
error text is dropped after classification.
* feat(scripts): add junit_diff.py for dual-stack JUnit comparison (#9275)
* feat(gateway-adapter): add IAM-backed PolicySource
Add IamPolicySource, the gateway PolicySource answered by RustFS IAM
(rustfs/backlog#2760, policy half). One snapshot per request carries a
RustfsPrincipal: principal kind, account, effective username, parent
user, credential groups, live group memberships, token claims and
is_owner.
- Anonymous callers get an empty snapshot without asking IAM.
- IAM not ready, not initialized or failing is PolicyError::unavailable,
never an empty snapshot.
- Unknown, disabled and session-kind-mismatched principals are refused
with constant labels; unreadable tokens are malformed.
- Claims are read from the stored token (root secret for STS, the
account's own secret for service accounts), and is_owner mirrors the
legacy has_root_access rule.
- RustfsPrincipal holds no secret or session token and has no Debug.
* ci(gateway): make the test-count ratchet a floor, not exact (#9278)
ci(gateway): make the test-count ratchet a floor instead of an exact baseline
Tests below a row still fail; tests above it, a member without a row and ignores below the row now pass with a note. Ignores above a row still fail. Adds contract tests wired into make script-tests.
* feat(gateway-adapter): add host, bucket-owner and request-id adapters
* fix(gateway-adapter): avoid ecstore path in doc comment
* fix(gateway-adapter): keep the s3s path pattern out of docs
* chore(gateway): record ignored microbenchmarks merged from main (#9290)
chore(gateway): record the ignored microbenchmarks merged from main
main added three throughput microbenchmarks that are ignored by design
(#9217): one each in rustfs-checksums, rustfs-rio and rustfs-utils. The
test-count ratchet treats a higher ignored count as a failure until the
row is raised, so raise those three rows to 1 and tighten the test rows
to the merged tree with --update.
* test(gateway-adapter): make the test carrier Clone for Extensions::insert
* fix(gateway-adapter): share live root and refuse short secrets
Address the security review of the T2.1 adapters (rustfs/backlog#2760).
- Add RootCredentials (validated: root secret and STS signing key at
least MIN_SECRET_KEY_LEN bytes), the RootSource trait and the settable
RootCell. IamAdapters::new is now the only constructor of
IamCredentials and IamPolicySource, so both read one live root source
per request; no root source answers unavailable.
- A looked-up identity whose secret is shorter than 8 bytes is NotFound
in the credential provider and DISABLED_PRINCIPAL in the policy
source, whatever the lookup reports as valid.
- STS claims are read with the STS signing key from the root source
instead of assuming it equals the root secret.
- Cover the off-status-alone branch, the OIDC owner exclusion, service
accounts of root, the name live groups are read under, and another
session's token.
* ci(s3-tests): add reference parity lane against pinned 1.0.5 (#9288)
* fix(gateway-adapter): effective secret length and unsplittable pair
Address the confirmation review of the T2.1 adapters
(rustfs/backlog#2760).
- One rule for every secret: its length after trailing NUL bytes are
removed must be at least MIN_SECRET_KEY_LEN. HMAC zero-pads its key,
so trailing NULs add nothing; interior NULs count. Applied to the root
secret, the STS signing key and every looked-up credential.
- IamAdapters keeps both halves private and hands them to a builder
together through the consuming install(builder, make_authenticator);
compile-fail doctests show a mixed pair cannot be assembled.
- Any Fn() -> Option<Arc<RootCredentials>> is a RootSource, so the
binary reads its live root accessor directly; RootCell is now a
test-only helper.
* test(gateway-adapter): use bucket names the rules accept in host tests
* refactor(gateway-adapter): never refuse on request id; owner is the reported constant
* docs(gateway-adapter): state what install guarantees about the provider handle
---------
Co-authored-by: rustfs-cla-assistant[bot] <273121464+rustfs-cla-assistant[bot]@users.noreply.github.com>
Lands gateway/integration on main: the guard rails and the domain
decoupling for replacing s3s with rustfs/gateway (rustfs/backlog#2734,
Phase 0 and Phase 1). The S3 edge is still s3s and no wire behaviour
change is intended.
- Guard rails: lower-only s3s footprint baseline and edge allow-list,
per-crate test-count floor, 39 byte-level goldens for the persisted
bucket-config XML families, a two-server differential harness over
all 73 S3 operations, a JUnit differ, and a reference-parity lane
against the sha256-pinned 1.0.5 binary.
- Owned types: rustfs-s3-types (S3Error, S3ErrorCode, Body),
RequestEnvelope, AuthzFacts.
- Engine crates off s3s: ecstore, lifecycle, replication, scanner,
protocols, s3select, s3-client, rpc.
- Admin: error types moved; 300 of 314 Operation impls take the owned
AdminRequest.
- Server: the s3s-compat tower layers grouped under
server/legacy_compat; RUSTFS_S3_STACK=legacy|gateway selector, where
gateway refuses to start until the gateway edge exists.
Fixes made on the branch that change ecstore and heal behaviour are
listed in the pull request description. The per-commit history stays
on gateway/integration.
Make the super:: import rules depth-aware for split files, cover ecfs_test.rs moving to legacy_edge/tests, read production text only in the Rust count and contains guards, bound method slices by their own closing line, keep the split arrays safe on bash 3.2, and add an end-to-end split probe for the architecture, layer and logging guards. Refs rustfs/backlog#2765.
Dispatch-only workflow that compiles the current main commit with a
caller-supplied version and uploads zip, deb, and rpm artifacts.
Co-authored-by: hector <majinghe@users.noreply.github.com>
* feat(server): add the RUSTFS_S3_STACK selector and dual-stack e2e lane (#8411)
* refactor(server): group the s3s-compat tower layers under legacy_compat (#8410)
Move the ten tower layers that patch s3s behaviour out of
rustfs/src/server/layer.rs into rustfs/src/server/legacy_compat/, one file
per layer with its tests. Each file header names the s3s behaviour it
patches, the gateway mechanism that replaces it (Replaced by:), and the task
that removes it (Removed by: T2.12, rustfs/backlog#2771). The layers keep
their code, their order in external_service_stack!, and their behaviour; no
layer is deleted.
Add server::http::tests::stack_census, which renders the ordered service
chain of the real external_service_stack! expansion and compares it with a
committed constant, so a reorder, insertion or removal is a visible diff.
Route the two s3s error types the moved layers use through a
storage_api::server::legacy_compat shim so the split adds no s3s-importing
file, and lower the s3s footprint baselines to the measured values (files 209
to 208, s3_error! lines 1586 to 1585). Keep the server runtime-source bypass
guard covering the moved code.
Refs rustfs/backlog#2787, rustfs/backlog#2734
* feat(s3-types): add the RustFS-owned S3 error type and s3_error! macro (#8406)
`rustfs-s3-types` now provides `S3ErrorCode` (generated by build.rs from
src/codes.txt, the grep-derived call surface, plus `Custom`), `S3Error`,
`S3Result`, `StdError` and the `s3_error` macro with the same call syntax
as s3s, so the T1.2 codemod is a `use` swap. Statuses are hand-written and
pinned to the s3s behaviour RustFS ships today by an oracle test that uses
s3s as a dev-dependency only. The `From` conversions for the legacy edge
live behind the off-by-default `compat-s3s` feature (the orphan rule
forbids them in rustfs/src/error.rs); `rustfs` enables it.
Refs rustfs/backlog#2742, rustfs/backlog#2734
* ci: add the per-crate test-count ratchet (#8404)
Quick Checks now fails any PR that lowers the number of test attributes
in a workspace crate or adds #[ignore], measured per crate against the
exact rows of .config/test-count-baseline.txt (rustfs/backlog#2734
section 1 rule 3). The counter widens the spec's literal
#[(tokio::)?test] to the tokio attributes that carry arguments and both
spellings of #[test_case(...)], and anchors at line start so
commented-out attributes do not count. The initial baseline covers 52
crates, 21077 tests and 164 ignored.
Refs rustfs/backlog#2738
Refs rustfs/backlog#2734
* test(ecstore): pin the persisted bucket-config XML bytes for 13 families (#8408)
Before any DTO or codec is replaced, pin the exact bytes that
deserialize followed by serialize produces for every persisted
bucket-configuration XML family in its three source shapes (AWS
documentation, MinIO-written, RustFS-written), and the bytes RustFS
writes today for a typed value. Byte equality only: a structural
comparison would let a codec replacement change the persisted encoding
unnoticed.
The generator script refuses to overwrite existing goldens without
--force, refuses on dirty codec paths, cannot report success from its
write mode, and records the generating commit in the fixtures README.
Refs rustfs/backlog#2744, rustfs/backlog#2734
* ci: raise the test-count baseline to the integration tree (#8426)
* ci: make the s3s footprint baseline exact and lower-only (#8403)
* ci: make the s3s footprint baseline exact and lower-only
The s3s footprint guard compared three counters against ceilings held in
the script itself, so a ceiling could sit above the tree and a drift was
fixable by raising it (rustfs/rustfs#5964 raised the s3_error! ceiling
to go green). The three values now live in
.config/s3s-footprint-baseline.txt as files, s3_error_lines and
ecstore_files, the script reads only that file, and a measurement that
differs from its baseline in either direction fails: growth prints
"s3s footprint grew: <measured> > <baseline>", shrinkage prints
"lower the baseline to <measured> in this PR". The counting method is
unchanged; the baseline is the current measurement of
gateway/integration (209 / 1585 / 36). The header records the end state:
once all three reach 0, T4.2 switches the script to failing on any
occurrence.
Refs rustfs/backlog#2739, rustfs/backlog#2734
* ci: add the s3s edge allowlist mode to the footprint guard (#8405)
`scripts/check_s3s_footprint.sh --mode allowlist` fails on every Rust
file that references s3s (`s3s::` or `s3s_sigv4`) and is not matched by
an entry of the new `.config/s3s-edge-allowlist.txt`; `--dry-run` lists
those files and exits 0. The allowlist carries the initial edge from the
spec, and the script rejects an entry that names or matches anything
under crates/ other than crates/e2e_test. The baseline mode and the
argument-less CI invocation are unchanged.
Refs rustfs/backlog#2741, rustfs/backlog#2734
* test(ecstore): reach the bucket-config codec through the test facade (#8428)
* refactor(s3-types): own the HTTP Body and drop s3s from three crates (#8431)
Add rustfs_s3_types::Body, a BoxBody<Bytes, StdError> wrapper with
empty() and From<Bytes | Vec<u8> | String | &'static str>, and the
legacy From conversions to and from s3s::Body behind compat-s3s.
The signer is now generic over the request body: it only reads and
writes the request head, so it needs no body type and its callers keep
theirs. s3-client sends its requests with the owned Body, madmin signs a
bodiless head, and targets drops its two S3Request/S3Response header
wrappers in favour of extract_params_header.
rustfs-signer, rustfs-madmin and rustfs-targets no longer depend on
s3s; the s3s footprint baseline drops from 209 to 202 files.
Refs rustfs/backlog#2747, rustfs/backlog#2734
* refactor(ecstore): move S3ErrorCode to rustfs-s3-types by codemod (#8432)
* test(ecstore): pin the InvalidRange responses of new_getobjectreader
Both error branches of new_getobjectreader build an ErrorResponse with S3ErrorCode::InvalidRange, and no test observed either code. Pin the code, its wire name and the location fields of each branch before the code type moves to rustfs-s3-types (rustfs/backlog#2743), so a changed code on either branch fails a test. The ecstore test-count row rises by the two tests.
* chore(scripts): add the s3s-to-rustfs-s3-types error codemod
Task T1.2 of rustfs/backlog#2734 (rustfs/backlog#2743) moves S3Error, S3ErrorCode, S3Result and s3_error! from s3s to rustfs-s3-types by script, one crate closure per PR. The codemod rewrites use trees (splitting mixed s3s imports) and qualified code paths, never comments or literals, adds the rustfs-s3-types dependency, drops s3s only from a crate that no longer names it, refuses every file of the s3s edge allowlist, takes --hold for a file whose type is fixed outside the closure, and prints a per-crate census. A second run changes nothing. Its self-test runs under make script-tests.
* refactor(ecstore): take S3ErrorCode from rustfs-s3-types
Output of scripts/codemods/s3_error_to_s3_types.py over crates/ecstore, crates/lifecycle, crates/scanner and crates/replication (rustfs/backlog#2743 batches 1 and 2). The ErrorResponse code field, the two InvalidRange constructions in new_getobjectreader and an unused tier import now use rustfs_s3_types::S3ErrorCode; the variant names and wire names are unchanged. lifecycle, scanner and replication reference none of the four error names. warm_backend.rs and warm_backend_s3.rs are held: they compare against rustfs_s3_client::ErrorResponse.code, which stays the s3s type until s3-client moves.
* ci(footprint): count only s3_error! lines still on s3s's macro
s3_error_lines counted every line holding the text `s3_error!`, which
also matches the RustFS-owned `rustfs_s3_types::s3_error!`. A codemod
that moves call sites onto rustfs-s3-types therefore could not lower it
(rustfs/backlog#2743, decision 3; rustfs/rustfs#8432, gotcha 2).
The counter now applies a textual, per-file and per-line rule that errs
on the side of counting (full rule in the script header):
- occurrences spelled `rustfs_s3_types::s3_error!` never count;
- a line holding `s3s::s3_error!` always counts;
- any other `s3_error!` line counts unless its file is cleared, that is,
it imports the macro from rustfs_s3_types by name on a line starting
with the use item (or defines it with `macro_rules! s3_error`) and does
not also bring s3s's macro in (`use s3s::s3_error`, a `use s3s::{..}`
tree naming it or a crate-root `*`, `use s3s::*`, or
`#[macro_use] extern crate s3s`);
- fail closed: a file matching neither form (`use super::*`, a
crate-local re-export, a renamed or glob rustfs_s3_types import, a
commented-out import, a comment-only mention) counts.
Census of the 1585 lines on 96fc06776: 1478 in 64 files that import the
macro from s3s; 107 in 15 files matching neither form (105 through
`use super::*`, 1 qualified `s3s::s3_error!`, 1 comment line); none in
files that import it from rustfs_s3_types or define it. Every line stays
counted, so the baseline remains s3_error_lines=1585 (files=202 and
ecstore_files=36 unchanged). Rewriting rustfs/src/auth.rs's import to
rustfs_s3_types lowers the new counter to 1567; the old one stayed 1585.
`--self-test` (wired into `make script-tests`) copies the script into
22 fixture trees: 6 positive cases and 16 negative ones (lines that must
not count, forms that must not clear a file, and a missing baseline,
missing key or failing classification search that must fail). Each tree
has a control line that must count, and each run gets s3s text on stdin.
Seven cases were red against the old counter. 19 mutants each turn their
case red: old definition, flipped import test, dropped fail-closed
branch, dropped e2e_test exclusion, dropped s3s veto, dropped qualified
s3s rule, dropped rustfs-qualified strip, unanchored rustfs import,
renamed import accepted, nested glob taken as root glob, single-line
s3s trees only, dropped s3s glob forms, dropped macro_use form, dropped
definition clearing, ignored classification rg errors, error-line search
without its path, classification searches without their path, missing
baseline skipped, missing key skipped.
* test(e2e): compare two RUSTFS_S3_STACK servers over every S3 operation (#8435)
Add dual_stack_differential to e2e_test. It starts two rustfs processes
from the same binary, stack A on RUSTFS_S3_STACK=legacy and stack B on
RUSTFS_E2E_STACK_B (default legacy until rustfs/backlog#2734 T2.5),
sends both the same aws-sdk-s3 script covering the 73 methods of
impl S3 for FS (a normal request and a 4xx negative each), and compares
status, error code, an allowlist of response headers and a few typed
body fields. Error message text is never compared. Each run writes
<target>/dual-stack/<run>.json with compared_ops and the (op, field,
a, b) differences; any difference fails the test.
differential_detects_an_injected_difference starts B with
RUSTFS_REGION=eu-west-1 and requires GetBucketLocation to be reported.
The comparator is pure and unit-tested without a server.
Wire the module into the e2e-smoke filter and update the e2e-smoke and
e2e-full selection digests and the e2e_test test-count row.
Refs rustfs/backlog#2740
* feat(app): add the RustFS-owned RequestEnvelope and its s3s constructor (#8437)
* feat(app): add the RustFS-owned RequestEnvelope and its s3s constructor
* ci: classify the s3s edge adapters in storage/s3_compat as interface
* feat(s3-types): add Body::from_stream and move Swift off s3s::Body (#8438)
rustfs/backlog#2747 batch 3. `Body::from_stream` wraps a chunk stream of
unknown length over `StreamBody`/`BoxBody`: one data frame per chunk, the
first error ends the body, the size hint stays unknown, and dropping the
body drops the stream. Parity tests pin it to the s3s `StreamingBlob::wrap`
it replaces, at the `http_body` level and byte for byte through hyper.
The Swift handlers in rustfs-protocols build their responses as the owned
`Body`. `SwiftService` converts them into the wrapped S3 service's response
body type through `From<Body>`, so rustfs keeps serving `s3s::Body` at the
legacy edge unchanged. Request bodies stay Send-only (`UnsyncBoxBody`); the
two storage writers and the DLO/SLO segment streams get `Sync` from
`sync_wrapper::SyncStream`, which is what s3s's `http_body_unsync` mutex
provided before. The DTO and error uses of s3s in protocols are untouched
(T1.5 and T1.2).
* refactor(protocols): move error types to rustfs-s3-types by codemod (#8439)
T1.2 batch 3 (rustfs/backlog#2743): s3select-api, s3select-query and
the error items of protocols move off s3s by scripts/codemods, run with
--hold crates/protocols/src/sftp/errors.rs. That file stays on s3s: it
downcasts the backend error, whose runtime type is still s3s::S3Error.
- rustfs/src/protocols/client.rs: the two webdav session methods now
return rustfs_s3_types::S3Result and convert the s3s error with From.
- codemod: a child module that calls s3_error! through `use super::*`
gets its own `use rustfs_s3_types::s3_error;` once its parent imports
that macro by name; otherwise it is listed as GLOB.
- s3-types: a table test pins code, status and the s3s error the legacy
edge renders for every variant a moved call site names.
- s3_error_lines 1585 -> 1584 (a lifecycle comment reworded).
* test(connect): restore one-second drive request budgets
Apply the exact test-only repair already landed on main in 5972a09cc3. The filesystem probe and production CLI success test request one second and keep their existing one-second duration and parameter checks. All outcome, I/O, error, cleanup and resource-limit assertions remain.
Scope: rustfs/tests/connect_perf_drive.rs only. Patch-id: cb0a1dfc8f68a67727525adea7458e2cee7d09f8.
Validation environment: LANG=en_US.UTF-8, CARGO_BUILD_JOBS=1, CARGO_INCREMENTAL=0, exclusive APFS-cloned CARGO_TARGET_DIR.
Quick Checks: every command from .github/actions/quick-checks/action.yml, with captured output and exit status below.
$ shellcheck --version && actionlint
ShellCheck - shell script analysis tool
version: 0.11.0
license: GNU General Public License, version 3
website: https://www.shellcheck.net
exit_code=0
$ cargo fmt --all --check
exit_code=0
$ ./scripts/check_unsafe_code_allowances.sh
exit_code=0
$ ./scripts/check_layer_dependencies.sh
Layer dependency guard passed (no new reverse dependencies/cycles).
exit_code=0
$ ./scripts/check_architecture_migration_rules.sh
Architecture migration rules passed.
exit_code=0
$ ./scripts/check_logging_guardrails.sh
✅ Logging guardrails check passed
exit_code=0
$ ./scripts/check_error_other_format_ratchet.sh
error(format!) ratchet passed (total call sites: 582).
exit_code=0
$ ./scripts/check_no_tokio_io_uring.sh
exit_code=0
$ ./scripts/check_extension_schema_boundaries.sh
Extension schema boundaries passed.
exit_code=0
$ ./scripts/check_body_cache_whitelist.sh
Body-cache whitelist guard passed.
exit_code=0
$ ./scripts/check_s3s_footprint.sh
s3s footprint OK: files importing s3s is 195 (baseline files=195)
s3s footprint OK: s3s s3_error! lines is 1584 (baseline s3_error_lines=1584)
s3s footprint OK: ecstore files referencing s3s is 36 (baseline ecstore_files=36)
✅ s3s footprint baseline check passed
exit_code=0
$ ./scripts/check_test_count_ratchet.sh
test-count ratchet OK: 52 crates, 21222 tests, 164 ignored match .config/test-count-baseline.txt
exit_code=0
$ ./scripts/check_fips_wording.sh
FIPS wording guard passed (README.md and CHANGELOG.md contain no forbidden claims).
KMS wording guard passed (crates/kms claims no Vault KV2 Transit wrapping).
exit_code=0
$ ./scripts/check_embedded_secrets.sh
Embedded secret guard passed (no private key material or provider credential literal in the tree).
exit_code=0
$ make script-tests
Running script tests...
entrypoint credential policy tests passed
internode grpc A/B bench env tests passed
object batch benchmark enhanced tests passed
PASS package service script contract
hotpath warp A/B gate tests passed
hotpath warp ABBA tests passed
Scanner validation artifacts written to /var/folders/0w/s4wxsn2n4w9b6f3cd4xrz53w0000gn/T/tmp.1pFe6Kl1Yu/out
Scanner validation artifacts written to /var/folders/0w/s4wxsn2n4w9b6f3cd4xrz53w0000gn/T/tmp.1pFe6Kl1Yu/out-missing-pid
.............................
----------------------------------------------------------------------
Ran 29 tests in 181.286s
OK
{"decision":"verified","pending_gates":["G01","G03","G04","G05","G06","G07","G08","G09","G10","G11","G12","G13","G14","P1","P2","P3","P4","R-D","R-E","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"G02"}
{"decision":"verified","pending_gates":["G01","G02","G03","G04","G05","G06","G07","G08","G09","G10","G11","G12","G13","G14","P1","P2","P3","P4","R-D","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"R-E"}
{"decision":"verified","pending_gates":["G01","G03","G04","G05","G06","G07","G08","G09","G10","G11","G12","G13","G14","P1","P2","P3","P4","R-D","R-E","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"G02"}
{"decision":"verified","pending_gates":["G01","G02","G03","G04","G05","G06","G07","G08","G09","G10","G11","G12","G13","G14","P1","P2","P3","P4","R-D","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"R-E"}
{"decision":"verified","pending_gates":["G02","G03","G04","G05","G06","G07","G08","G09","G10","G11","G12","G13","G14","P1","P2","P3","P4","R-D","R-E","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"G01"}
{"decision":"verified","pending_gates":["G01","G02","G04","G05","G06","G07","G08","G09","G10","G11","G12","G13","G14","P1","P2","P3","P4","R-D","R-E","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"G03"}
{"decision":"verified","pending_gates":["G01","G02","G03","G04","G05","G06","G07","G08","G09","G10","G11","G12","G13","G14","P1","P2","P3","P4","R-D","R-E"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"R-L"}
{"decision":"verified","pending_gates":["G01","G02","G03","G04","G05","G06","G07","G08","G09","G10","G11","G12","G13","P1","P2","P3","P4","R-D","R-E","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"G14"}
{"decision":"verified","pending_gates":["G01","G02","G03","G04","G05","G06","G07","G08","G09","G10","G11","G12","G13","P1","P2","P3","P4","R-D","R-E","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"G14"}
{"decision":"verified","pending_gates":["G01","G02","G03","G04","G05","G06","G07","G08","G09","G11","G12","G13","G14","P1","P2","P3","P4","R-D","R-E","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"G10"}
{"decision":"verified","pending_gates":["G01","G02","G03","G04","G05","G06","G07","G08","G09","G10","G11","G12","G13","G14","P2","P3","P4","R-D","R-E","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"P1"}
{"decision":"verified","pending_gates":["G01","G02","G03","G04","G05","G06","G07","G08","G09","G10","G11","G12","G13","G14","P1","P3","P4","R-D","R-E","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"P2"}
{"decision":"verified","pending_gates":["G01","G02","G03","G04","G05","G06","G07","G08","G09","G10","G11","G12","G13","G14","P1","P2","P4","R-D","R-E","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"P3"}
{"decision":"verified","pending_gates":["G01","G02","G03","G04","G06","G07","G08","G09","G10","G11","G12","G13","G14","P1","P2","P3","P4","R-D","R-E","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"G05"}
{"decision":"verified","pending_gates":["G01","G02","G03","G04","G05","G07","G08","G09","G10","G11","G12","G13","G14","P1","P2","P3","P4","R-D","R-E","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"G06"}
{"decision":"verified","pending_gates":["G01","G02","G03","G04","G05","G06","G07","G08","G09","G10","G11","G12","G13","G14","P1","P2","P3","P4","R-E","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"R-D"}
Status-and-outcome release descriptor verified: /private/var/folders/0w/s4wxsn2n4w9b6f3cd4xrz53w0000gn/T/tmp8rwn1pe0/descriptor/release-bundle-status-outcome.json
{"decision":"verified","pending_gates":["G01","G02","G03","G04","G06","G07","G08","G09","G10","G11","G12","G13","G14","P1","P2","P3","P4","R-D","R-E","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"G05"}
{"decision":"verified","pending_gates":["G01","G02","G03","G04","G05","G07","G08","G09","G10","G11","G12","G13","G14","P1","P2","P3","P4","R-D","R-E","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"G06"}
{"decision":"verified","pending_gates":["G01","G02","G03","G04","G05","G06","G07","G08","G09","G10","G11","G12","G13","G14","P1","P2","P3","P4","R-E","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"R-D"}
{"decision":"verified","pending_gates":["G01","G02","G03","G04","G05","G06","G08","G09","G10","G11","G12","G13","G14","P1","P2","P3","P4","R-D","R-E","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"G07"}
{"decision":"verified","pending_gates":["G01","G02","G03","G04","G05","G06","G07","G09","G10","G11","G12","G13","G14","P1","P2","P3","P4","R-D","R-E","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"G08"}
{"decision":"verified","pending_gates":["G01","G02","G03","G04","G05","G06","G07","G08","G09","G10","G11","G12","G13","G14","P1","P2","P3","R-D","R-E","R-L"],"pending_lanes":["authority-coverage","checkpoint-and-crash","ec8-4-multiset","maintenance-producers","mixed-version-rollback","mrf-responsibility","scheduler-pressure","status-and-outcome"],"rejected_errors":[],"rejected_gate":null,"release_approved":false,"release_schema_capable":true,"schema":1,"verified_gate":"P4"}
exact 1MiB handoff ABBA tests passed
pinned paired ABBA bench tests passed
==> /var/folders/0w/s4wxsn2n4w9b6f3cd4xrz53w0000gn/T/tmp.mYmuI01hNq/bin/path_containment (-max_total_time=7, -seed=123456789, -artifact_prefix=/var/folders/0w/s4wxsn2n4w9b6f3cd4xrz53w0000gn/T/tmp.mYmuI01hNq/explicit-artifacts/path_containment/, corpus=/private/tmp/gw-orchestrator-2734/ci-drive/fuzz/corpus/path_containment)
-seed=123456789
target=path_containment
seed=123456789
max_total_time=7
git_revision=4d15b816236630b2c42f321ffc9ed3654edc94e3
git_dirty=true
runner_mode=prebuilt
==> /var/folders/0w/s4wxsn2n4w9b6f3cd4xrz53w0000gn/T/tmp.mYmuI01hNq/bin/path_containment (-max_total_time=1, -seed=2322617706, -artifact_prefix=/var/folders/0w/s4wxsn2n4w9b6f3cd4xrz53w0000gn/T/tmp.mYmuI01hNq/auto-artifacts/path_containment/, corpus=/private/tmp/gw-orchestrator-2734/ci-drive/fuzz/corpus/path_containment)
-seed=2322617706
==> cargo +nightly fuzz run path_containment (-max_total_time=9, -seed=987654321, -artifact_prefix=/var/folders/0w/s4wxsn2n4w9b6f3cd4xrz53w0000gn/T/tmp.mYmuI01hNq/cargo-artifacts/path_containment/)
+nightly
fuzz
run
path_containment
-seed=987654321
seed=987654321
runner_mode=cargo-fuzz
fuzz runner seed manifest tests passed
✅ scripts/python_bin.sh resolver checks passed
self-test ok: clean fixture with every exemption present passes
self-test ok: all 11 pattern families fire
self-test ok: moving a pinned source is reported
Embedded secret guard self-test passed.
self-test ok: p1-s3s-import: 'use s3s::s3_error;' counts every line, the rustfs_s3_types module's too (s3_error_lines=3)
self-test ok: p2-s3s-mixed-tree: a multi-line s3s use tree naming s3_error counts (s3_error_lines=3)
self-test ok: p3-s3s-glob: a crate-root glob of s3s, bare or inside a tree, counts (s3_error_lines=5)
self-test ok: p4-macro-use: '#[macro_use] extern crate s3s' counts (s3_error_lines=3)
self-test ok: p5-qualified-s3s: a qualified s3s::s3_error! line counts in a cleared file (s3_error_lines=2)
self-test ok: p6-exact-pass: a baseline equal to the tree passes (exit 0)
self-test ok: n1-rustfs-import: 'use rustfs_s3_types::s3_error;' alone counts nothing (s3_error_lines=1)
self-test ok: n2-rustfs-mixed-tree: a rustfs_s3_types tree plus an s3s tree without the macro counts nothing (s3_error_lines=1)
self-test ok: n3-defines: the file defining the macro counts nothing (s3_error_lines=1)
self-test ok: n4-qualified-rustfs: a qualified rustfs_s3_types::s3_error! line counts nothing (s3_error_lines=1)
self-test ok: n5-renamed-call: a call through a renamed macro is not text-matched (out of scope) (s3_error_lines=1)
self-test ok: n6-comment-string: a comment or string mentioning s3_error! in a cleared file counts nothing (s3_error_lines=1)
self-test ok: n7-e2e-excluded: crates/e2e_test is excluded (s3_error_lines=1)
self-test ok: n8-super-glob: an unrecognised import form ('use super::*;') counts (s3_error_lines=3)
self-test ok: n9-facade: a crate-local re-export counts (s3_error_lines=2)
self-test ok: n10-renamed-import: a renamed rustfs_s3_types import does not clear s3_error! lines (s3_error_lines=2)
self-test ok: n11-commented-import: a commented-out rustfs_s3_types import does not clear (s3_error_lines=2)
self-test ok: n12-rustfs-glob: a glob import of rustfs_s3_types does not clear (s3_error_lines=2)
self-test ok: n13-comment-only: a comment mentioning s3_error! in a file with no import counts (s3_error_lines=2)
self-test ok: f1-baseline-missing: a missing baseline file fails (exit 1)
self-test ok: f2-key-missing: a baseline without s3_error_lines fails (exit 1)
self-test ok: f3-rg-fails: a failing classification search fails the guard (exit 1)
✅ s3s footprint self-test passed
test_core_gate_rejects_missing_ignored_filtered_and_corrupt_inputs (__main__.SelfTests.test_core_gate_rejects_missing_ignored_filtered_and_corrupt_inputs) ... ok
test_e2e_requires_registration (__main__.SelfTests.test_e2e_requires_registration) ... ok
test_fuzz_runtime_uses_matrix_target (__main__.SelfTests.test_fuzz_runtime_uses_matrix_target) ... ok
test_ilm_lane_keeps_the_measured_cargo_build_budget (__main__.SelfTests.test_ilm_lane_keeps_the_measured_cargo_build_budget) ... ok
test_profile_listing_binds_platform_digest (__main__.SelfTests.test_profile_listing_binds_platform_digest) ... ok
test_profile_listing_enforces_selection (__main__.SelfTests.test_profile_listing_enforces_selection) ... ok
test_quick_checks_commands_propagate_failure (__main__.SelfTests.test_quick_checks_commands_propagate_failure) ... ok
test_quick_checks_rejects_caller_and_execution_bypasses (__main__.SelfTests.test_quick_checks_rejects_caller_and_execution_bypasses) ... ok
test_runner_readiness_fails_closed (__main__.SelfTests.test_runner_readiness_fails_closed) ... ok
test_s3_runner_rejects_unbounded_failure_locals (__main__.SelfTests.test_s3_runner_rejects_unbounded_failure_locals) ... ok
test_scanner_heal_begin_requires_embedded_source_provenance (__main__.SelfTests.test_scanner_heal_begin_requires_embedded_source_provenance) ... ok
test_scanner_heal_case_does_not_approve_pending_release (__main__.SelfTests.test_scanner_heal_case_does_not_approve_pending_release) ... ok
test_scanner_heal_case_lane_is_required_for_runner_profile (__main__.SelfTests.test_scanner_heal_case_lane_is_required_for_runner_profile) ... ok
test_scanner_heal_case_only_schema_cannot_approve_release (__main__.SelfTests.test_scanner_heal_case_only_schema_cannot_approve_release) ... ok
test_scanner_heal_crash_case_rejects_restart_oracle (__main__.SelfTests.test_scanner_heal_crash_case_rejects_restart_oracle) ... ok
test_scanner_heal_ec84_case_rejects_wrong_erasure_geometry (__main__.SelfTests.test_scanner_heal_ec84_case_rejects_wrong_erasure_geometry) ... ok
test_scanner_heal_finish_collects_oracles_from_registry (__main__.SelfTests.test_scanner_heal_finish_collects_oracles_from_registry) ... ok
test_scanner_heal_multipool_case_accepts_deferred_optional_outage_write (__main__.SelfTests.test_scanner_heal_multipool_case_accepts_deferred_optional_outage_write) ... ok
test_scanner_heal_pending_gate_cannot_map_to_implemented_lane (__main__.SelfTests.test_scanner_heal_pending_gate_cannot_map_to_implemented_lane) ... ok
test_scanner_heal_receipts_reject_reuse_and_missing_builds (__main__.SelfTests.test_scanner_heal_receipts_reject_reuse_and_missing_builds) ... ok
test_scanner_heal_rejects_boolean_fractional_and_out_of_geometry_integers (__main__.SelfTests.test_scanner_heal_rejects_boolean_fractional_and_out_of_geometry_integers) ... ok
test_scanner_heal_rejects_broken_execution_and_artifacts (__main__.SelfTests.test_scanner_heal_rejects_broken_execution_and_artifacts) ... ok
test_scanner_heal_rejects_copied_junit_and_wrong_suite_build (__main__.SelfTests.test_scanner_heal_rejects_copied_junit_and_wrong_suite_build) ... ok
test_scanner_heal_release_bundle_accepts_complete_measured_evidence (__main__.SelfTests.test_scanner_heal_release_bundle_accepts_complete_measured_evidence) ... ok
test_scanner_heal_release_bundle_assembler_copies_measured_artifacts (__main__.SelfTests.test_scanner_heal_release_bundle_assembler_copies_measured_artifacts) ... ok
test_scanner_heal_release_bundle_assembler_merges_measured_descriptors (__main__.SelfTests.test_scanner_heal_release_bundle_assembler_merges_measured_descriptors) ... ok
test_scanner_heal_release_bundle_assembler_rejects_duplicate_descriptor_gate (__main__.SelfTests.test_scanner_heal_release_bundle_assembler_rejects_duplicate_descriptor_gate) ... ok
test_scanner_heal_release_bundle_assembler_rejects_fixture_descriptor (__main__.SelfTests.test_scanner_heal_release_bundle_assembler_rejects_fixture_descriptor) ... ok
test_scanner_heal_release_bundle_assembler_rejects_hash_mismatch (__main__.SelfTests.test_scanner_heal_release_bundle_assembler_rejects_hash_mismatch) ... ok
test_scanner_heal_release_bundle_assembler_rejects_mixed_descriptor_revisions (__main__.SelfTests.test_scanner_heal_release_bundle_assembler_rejects_mixed_descriptor_revisions) ... ok
test_scanner_heal_release_bundle_enforces_topology_duration_profile_and_versions (__main__.SelfTests.test_scanner_heal_release_bundle_enforces_topology_duration_profile_and_versions) ... ok
test_scanner_heal_release_bundle_fixture_writer_is_not_release_evidence (__main__.SelfTests.test_scanner_heal_release_bundle_fixture_writer_is_not_release_evidence) ... ok
test_scanner_heal_release_bundle_gate_accepts_single_measured_gate (__main__.SelfTests.test_scanner_heal_release_bundle_gate_accepts_single_measured_gate) ... ok
test_scanner_heal_release_bundle_gate_rejects_missing_field (__main__.SelfTests.test_scanner_heal_release_bundle_gate_rejects_missing_field) ... ok
test_scanner_heal_release_bundle_gate_rejects_unknown_gate (__main__.SelfTests.test_scanner_heal_release_bundle_gate_rejects_unknown_gate) ... ok
test_scanner_heal_release_bundle_rejects_synthetic_or_missing_fields (__main__.SelfTests.test_scanner_heal_release_bundle_rejects_synthetic_or_missing_fields) ... ok
test_scanner_heal_release_bundle_requires_domain_evidence (__main__.SelfTests.test_scanner_heal_release_bundle_requires_domain_evidence) ... ok
test_scanner_heal_release_bundle_requires_field_provenance (__main__.SelfTests.test_scanner_heal_release_bundle_requires_field_provenance) ... ok
test_scanner_heal_release_bundle_requires_json_artifact_provenance (__main__.SelfTests.test_scanner_heal_release_bundle_requires_json_artifact_provenance) ... ok
test_scanner_heal_release_bundle_requires_mixed_version_field_roles (__main__.SelfTests.test_scanner_heal_release_bundle_requires_mixed_version_field_roles) ... ok
test_scanner_heal_release_bundle_requires_same_gate_measurement_window (__main__.SelfTests.test_scanner_heal_release_bundle_requires_same_gate_measurement_window) ... ok
test_scanner_heal_release_consumes_multiple_registry_oracles (__main__.SelfTests.test_scanner_heal_release_consumes_multiple_registry_oracles) ... ok
test_scanner_heal_release_matrix_lanes_remain_pending_without_real_evidence (__main__.SelfTests.test_scanner_heal_release_matrix_lanes_remain_pending_without_real_evidence) ... ok
test_scanner_heal_release_requirements_cannot_be_cleared (__main__.SelfTests.test_scanner_heal_release_requirements_cannot_be_cleared) ... ok
test_scanner_heal_release_status_rejects_focused_case_run (__main__.SelfTests.test_scanner_heal_release_status_rejects_focused_case_run) ... ok
test_scanner_heal_release_status_rejects_synthetic_case (__main__.SelfTests.test_scanner_heal_release_status_rejects_synthetic_case) ... ok
test_scanner_heal_required_evidence_fields_cannot_be_removed (__main__.SelfTests.test_scanner_heal_required_evidence_fields_cannot_be_removed) ... ok
test_scheduled_alerts_require_completion_watchdog (__main__.SelfTests.test_scheduled_alerts_require_completion_watchdog) ... ok
test_update_profile_selection_changes_only_requested_platform (__main__.SelfTests.test_update_profile_selection_changes_only_requested_platform) ... ok
test_validate_includes_quick_checks (__main__.SelfTests.test_validate_includes_quick_checks) ... ok
test_vault_tests_require_cross_process_serialization (__main__.SelfTests.test_vault_tests_require_cross_process_serialization) ... ok
test_workflow_readiness_requires_dependency_probe (__main__.SelfTests.test_workflow_readiness_requires_dependency_probe) ... ok
----------------------------------------------------------------------
Ran 52 tests in 14.301s
OK
e2e-full selection OK: 1 tests, sha256=29e30e6ed2ed38d9e5b0a7cdd9536abf5c719f7021a02b578ae1381563a3c273
...................................
----------------------------------------------------------------------
Ran 35 tests in 1.161s
OK
...................
----------------------------------------------------------------------
Ran 19 tests in 17.397s
OK
..........
----------------------------------------------------------------------
Ran 10 tests in 0.183s
OK
test_actual_selector_bootstrap_uses_base_policy_and_fails_closed (__main__.SelfTests.test_actual_selector_bootstrap_uses_base_policy_and_fails_closed) ... ok
test_documentation_paths_do_not_hide_build_or_fixture_changes (__main__.SelfTests.test_documentation_paths_do_not_hide_build_or_fixture_changes) ... ok
test_e2e_scope_keeps_production_and_shared_configuration_full (__main__.SelfTests.test_e2e_scope_keeps_production_and_shared_configuration_full) ... ok
test_e2e_shortcut_rejects_new_direct_renamed_and_target_dependencies (__main__.SelfTests.test_e2e_shortcut_rejects_new_direct_renamed_and_target_dependencies) ... ok
test_e2e_shortcut_traverses_implicit_path_dependencies (__main__.SelfTests.test_e2e_shortcut_traverses_implicit_path_dependencies) ... ok
test_event_contract_requires_complete_candidate_and_optional_lanes (__main__.SelfTests.test_event_contract_requires_complete_candidate_and_optional_lanes) ... ok
test_every_wrong_result_missing_job_or_selection_fails_closed (__main__.SelfTests.test_every_wrong_result_missing_job_or_selection_fails_closed) ... ok
test_full_e2e_gate_preserves_workflow_branch_and_event_scope (__main__.SelfTests.test_full_e2e_gate_preserves_workflow_branch_and_event_scope) ... ok
test_git_range_includes_deleted_source_and_rename_origins (__main__.SelfTests.test_git_range_includes_deleted_source_and_rename_origins) ... ok
test_job_ids_and_display_names_cannot_hide_validation (__main__.SelfTests.test_job_ids_and_display_names_cannot_hide_validation) ... ok
test_report_only_stack_lane_mirrors_e2e_smoke_outside_the_aggregate (__main__.SelfTests.test_report_only_stack_lane_mirrors_e2e_smoke_outside_the_aggregate) ... ok
test_repository_wiring_and_missing_dependency_regression (__main__.SelfTests.test_repository_wiring_and_missing_dependency_regression) ... ok
test_verify_command_preserves_failures (__main__.SelfTests.test_verify_command_preserves_failures) ... ok
----------------------------------------------------------------------
Ran 13 tests in 3.280s
OK
security coverage self-test passed
test_api_failures_preserve_other_evidence_and_never_enter_grace (__main__.SelfTests.test_api_failures_preserve_other_evidence_and_never_enter_grace) ... ok
test_attempt_outcome_does_not_replace_recent_success (__main__.SelfTests.test_attempt_outcome_does_not_replace_recent_success) ... ok
test_cancelled_attempt_cannot_refresh_expired_success (__main__.SelfTests.test_cancelled_attempt_cannot_refresh_expired_success) ... ok
test_cli_requires_the_repository_default_branch (__main__.SelfTests.test_cli_requires_the_repository_default_branch) ... ok
test_config_rejects_duplicate_and_invalid_entries (__main__.SelfTests.test_config_rejects_duplicate_and_invalid_entries) ... ok
test_freshness_boundaries (__main__.SelfTests.test_freshness_boundaries) ... ok
test_grace_requires_two_successful_queries_with_no_history (__main__.SelfTests.test_grace_requires_two_successful_queries_with_no_history) ... ok
test_invalid_api_evidence_fails_closed (__main__.SelfTests.test_invalid_api_evidence_fails_closed) ... ok
test_report_retains_every_workflow (__main__.SelfTests.test_report_retains_every_workflow) ... ok
test_requests_filter_schedule_default_branch_and_success_on_server (__main__.SelfTests.test_requests_filter_schedule_default_branch_and_success_on_server) ... ok
----------------------------------------------------------------------
Ran 10 tests in 0.051s
OK
..............................
----------------------------------------------------------------------
Ran 30 tests in 22.354s
OK
..........
----------------------------------------------------------------------
Ran 10 tests in 2.969s
OK
.................................
----------------------------------------------------------------------
Ran 33 tests in 0.160s
OK
Online functional runner labels: pf-testing
.......
----------------------------------------------------------------------
Ran 7 tests in 0.002s
OK
.......
----------------------------------------------------------------------
Ran 7 tests in 0.001s
OK
....
----------------------------------------------------------------------
Ran 4 tests in 0.128s
OK
...
----------------------------------------------------------------------
Ran 3 tests in 0.547s
OK
PASS: cold stampede script self-test
exit_code=0
$ python3 ./scripts/test_object_generation_protocol_model.py
test_a_recovery_barrier_cannot_choose_an_empty_value (__main__.ObjectGenerationProtocolModel.test_a_recovery_barrier_cannot_choose_an_empty_value) ... ok
test_all_bounded_two_proposer_interleavings_choose_at_most_one_value (__main__.ObjectGenerationProtocolModel.test_all_bounded_two_proposer_interleavings_choose_at_most_one_value) ... ok
test_an_unavailable_decision_quorum_cannot_choose_a_candidate (__main__.ObjectGenerationProtocolModel.test_an_unavailable_decision_quorum_cannot_choose_a_candidate) ... ok
test_counter_allocation_requires_persistence_and_is_consumed_once (__main__.ObjectGenerationProtocolModel.test_counter_allocation_requires_persistence_and_is_consumed_once) ... ok
test_counter_restart_burns_persisted_allocations_before_or_after_issue (__main__.ObjectGenerationProtocolModel.test_counter_restart_burns_persisted_allocations_before_or_after_issue) ... ok
test_crash_after_promise_before_reply_does_not_restore_an_older_ballot (__main__.ObjectGenerationProtocolModel.test_crash_after_promise_before_reply_does_not_restore_an_older_ballot) ... ok
test_duplicate_or_unconfigured_voters_cannot_form_a_quorum (__main__.ObjectGenerationProtocolModel.test_duplicate_or_unconfigured_voters_cannot_form_a_quorum) ... ok
test_empty_or_unobserved_minority_slot_can_retry_without_allocating_another_revision (__main__.ObjectGenerationProtocolModel.test_empty_or_unobserved_minority_slot_can_retry_without_allocating_another_revision) ... ok
test_equal_ballot_cannot_accept_different_bytes_and_lower_ballot_is_rejected (__main__.ObjectGenerationProtocolModel.test_equal_ballot_cannot_accept_different_bytes_and_lower_ballot_is_rejected) ... ok
test_forgetting_tombstone_authority_allows_an_aba_create (__main__.ObjectGenerationProtocolModel.test_forgetting_tombstone_authority_allows_an_aba_create) ... ok
test_issuing_before_persistence_reuses_a_ballot_after_restart (__main__.ObjectGenerationProtocolModel.test_issuing_before_persistence_reuses_a_ballot_after_restart) ... ok
test_lost_ack_and_restart_preserve_the_chosen_operation (__main__.ObjectGenerationProtocolModel.test_lost_ack_and_restart_preserve_the_chosen_operation) ... ok
test_minority_accepted_work_is_adopted_and_finished (__main__.ObjectGenerationProtocolModel.test_minority_accepted_work_is_adopted_and_finished) ... ok
test_omitting_highest_accepted_adoption_has_a_double_choice_counterexample (__main__.ObjectGenerationProtocolModel.test_omitting_highest_accepted_adoption_has_a_double_choice_counterexample) ... ok
test_quorum_choice_does_not_require_an_uncontacted_disk_to_learn_it (__main__.ObjectGenerationProtocolModel.test_quorum_choice_does_not_require_an_uncontacted_disk_to_learn_it) ... ok
test_recovery_failure_or_stale_expected_head_does_not_allocate_a_successor (__main__.ObjectGenerationProtocolModel.test_recovery_failure_or_stale_expected_head_does_not_allocate_a_successor) ... ok
test_replacing_voters_with_empty_state_can_choose_a_conflicting_value (__main__.ObjectGenerationProtocolModel.test_replacing_voters_with_empty_state_can_choose_a_conflicting_value) ... ok
test_skipping_predecessor_recovery_chooses_a_broken_revision_chain (__main__.ObjectGenerationProtocolModel.test_skipping_predecessor_recovery_chooses_a_broken_revision_chain) ... ok
test_stale_publication_rollback_and_gc_cannot_change_the_winner (__main__.ObjectGenerationProtocolModel.test_stale_publication_rollback_and_gc_cannot_change_the_winner) ... ok
test_superseded_operation_identity_cannot_be_reused_for_different_input (__main__.ObjectGenerationProtocolModel.test_superseded_operation_identity_cannot_be_reused_for_different_input) ... ok
test_superseded_operation_retry_returns_its_original_successor (__main__.ObjectGenerationProtocolModel.test_superseded_operation_retry_returns_its_original_successor) ... ok
test_tombstone_and_null_marker_payloads_do_not_reset_authority_revision (__main__.ObjectGenerationProtocolModel.test_tombstone_and_null_marker_payloads_do_not_reset_authority_revision) ... ok
test_tombstone_rejects_a_delayed_never_present_create (__main__.ObjectGenerationProtocolModel.test_tombstone_rejects_a_delayed_never_present_create) ... ok
test_unlearned_predecessor_is_recovered_before_the_second_slot (__main__.ObjectGenerationProtocolModel.test_unlearned_predecessor_is_recovered_before_the_second_slot) ... ok
----------------------------------------------------------------------
Ran 24 tests in 0.958s
OK
model: states=51023, transitions=167520, terminal=732, chosen_candidates=2
negative control: double-choice trace=((0, 'prepare', 0), (0, 'prepare', 1), (0, 'prepare', 2), (0, 'accept', 0), (0, 'accept', 1), (0, 'accept', 2), (1, 'prepare', 0), (1, 'prepare', 1), (1, 'prepare', 2), (1, 'accept', 0), (1, 'accept', 1), (1, 'accept', 2))
multi-slot round schedules: 640
exit_code=0
$ python3 ./scripts/check_test_wiring.py
OK: e2e modules, runner selection, fuzz matrices, profiles, scheduled alerts, and Quick Checks are wired
exit_code=0
$ ./scripts/check_no_planning_docs.sh
exit_code=0
$ ./scripts/check_uring_lane_lib_only.sh
OK: no 'uring_'-matching test functions under crates/ecstore/tests; --lib is safe
exit_code=0
Initial complete gate
$ cargo nextest run --profile ci -p rustfs --test connect_perf_drive --no-fail-fast
Nextest run ID e07fc5b9-c6c8-4cdd-ae61-54839bf24437 with nextest profile: ci
Starting 10 tests across 1 binary
PASS [ 0.016s] rustfs::connect_perf_drive cancellation_and_single_collector_leave_no_scratch
PASS [ 0.011s] rustfs::connect_perf_drive cleanup_failure_overrides_cancellation_and_remains_attributed
PASS [ 0.009s] rustfs::connect_perf_drive consent_and_resource_budgets_fail_before_scratch_io
PASS [ 0.009s] rustfs::connect_perf_drive invalid_roots_and_existing_scratch_fail_without_clobbering
PASS [ 16.950s] rustfs::connect_perf_drive production_cli_measures_and_signs_exact_binary_provenance
PASS [ 4.025s] rustfs::connect_perf_drive production_cli_rejects_missing_l1_consent_and_limits_before_artifacts
PASS [ 0.008s] rustfs::connect_perf_drive read_only_root_and_full_drive_errors_are_classified
PASS [ 0.015s] rustfs::connect_perf_drive real_local_write_read_is_measured_and_scratch_is_removed
PASS [ 0.007s] rustfs::connect_perf_drive runtime_timeout_and_full_drive_use_schema_valid_failed_reason
PASS [ 0.035s] rustfs::connect_perf_drive successful_result_has_signed_bounded_private_offline_export
────────────
Summary [ 21.087s] 10 tests run: 10 passed, 0 skipped
exit_code=0
Restored complete gate
$ cargo nextest run --profile ci -p rustfs --test connect_perf_drive --no-fail-fast
Nextest run ID 233b7307-5361-4eb7-94d5-b384f17d2d6a with nextest profile: ci
Starting 10 tests across 1 binary
PASS [ 0.009s] rustfs::connect_perf_drive cancellation_and_single_collector_leave_no_scratch
PASS [ 0.008s] rustfs::connect_perf_drive cleanup_failure_overrides_cancellation_and_remains_attributed
PASS [ 0.008s] rustfs::connect_perf_drive consent_and_resource_budgets_fail_before_scratch_io
PASS [ 0.008s] rustfs::connect_perf_drive invalid_roots_and_existing_scratch_fail_without_clobbering
PASS [ 19.518s] rustfs::connect_perf_drive production_cli_measures_and_signs_exact_binary_provenance
PASS [ 4.146s] rustfs::connect_perf_drive production_cli_rejects_missing_l1_consent_and_limits_before_artifacts
PASS [ 0.009s] rustfs::connect_perf_drive read_only_root_and_full_drive_errors_are_classified
PASS [ 0.018s] rustfs::connect_perf_drive real_local_write_read_is_measured_and_scratch_is_removed
PASS [ 0.009s] rustfs::connect_perf_drive runtime_timeout_and_full_drive_use_schema_valid_failed_reason
PASS [ 0.054s] rustfs::connect_perf_drive successful_result_has_signed_bounded_private_offline_export
────────────
Summary [ 23.787s] 10 tests run: 10 passed, 0 skipped
exit_code=0
Mutation M1: success_measurement reports max(actual elapsed, requested budget + 1), forcing 1001 ms for the one-second request. The retained duration assertion at rustfs/tests/connect_perf_drive.rs:105 fails.
$ cargo nextest run --profile ci -p rustfs --test connect_perf_drive --no-fail-fast -E 'test(real_local_write_read_is_measured_and_scratch_is_removed)'
info: experimental features enabled: setup-scripts
Compiling rustfs v1.0.1 (/private/tmp/gw-orchestrator-2734/ci-drive/rustfs)
warning: linker stderr: ld: __eh_frame section too large (max 16MB) to encode dwarf unwind offsets in compact unwind table, performance of exception handling might be affected
|
= note: `#[warn(linker_messages)]` on by default
warning: `rustfs` (bin "rustfs") generated 1 warning
warning: `rustfs` (bin "rustfs-cli") generated 1 warning (1 duplicate)
Finished `test` profile [unoptimized + debuginfo] target(s) in 7m 06s
────────────
Nextest run ID 9f618777-7bf6-4cbc-8803-dd7f50f449ed with nextest profile: ci
Starting 1 test across 1 binary (9 tests skipped)
FAIL [ 0.016s] rustfs::connect_perf_drive real_local_write_read_is_measured_and_scratch_is_removed
stdout ───
running 1 test
test real_local_write_read_is_measured_and_scratch_is_removed ... FAILED
failures:
failures:
real_local_write_read_is_measured_and_scratch_is_removed
test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 9 filtered out; finished in 0.01s
stderr ───
thread 'real_local_write_read_is_measured_and_scratch_is_removed' (106986846) panicked at rustfs/tests/connect_perf_drive.rs:105:5:
assertion failed: (1..=1_000).contains(&data.duration_millis)
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
────────────
Summary [ 0.017s] 1 test run: 0 passed, 1 failed, 9 skipped
FAIL [ 0.016s] rustfs::connect_perf_drive real_local_write_read_is_measured_and_scratch_is_removed
error: test run failed
exit_code=100
Mutation M2: target_parameters reports requested duration + 1. The in-process assertion at rustfs/tests/connect_perf_drive.rs:111 and CLI JSON assertion at rustfs/tests/connect_perf_drive.rs:527 both reject 1001 against 1000.
$ cargo nextest run --profile ci -p rustfs --test connect_perf_drive --no-fail-fast -E 'test(real_local_write_read_is_measured_and_scratch_is_removed) | test(production_cli_measures_and_signs_exact_binary_provenance)'
info: experimental features enabled: setup-scripts
Compiling rustfs v1.0.1 (/private/tmp/gw-orchestrator-2734/ci-drive/rustfs)
warning: linker stderr: ld: __eh_frame section too large (max 16MB) to encode dwarf unwind offsets in compact unwind table, performance of exception handling might be affected
|
= note: `#[warn(linker_messages)]` on by default
warning: `rustfs` (bin "rustfs") generated 1 warning
warning: `rustfs` (bin "rustfs-cli") generated 1 warning (1 duplicate)
Finished `test` profile [unoptimized + debuginfo] target(s) in 7m 20s
────────────
Nextest run ID b73d46ab-c910-4e1e-8063-7fc7e332ec15 with nextest profile: ci
Starting 2 tests across 1 binary (8 tests skipped)
FAIL [ 13.968s] rustfs::connect_perf_drive production_cli_measures_and_signs_exact_binary_provenance
stdout ───
running 1 test
test production_cli_measures_and_signs_exact_binary_provenance ... FAILED
failures:
failures:
production_cli_measures_and_signs_exact_binary_provenance
test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 9 filtered out; finished in 13.96s
stderr ───
thread 'production_cli_measures_and_signs_exact_binary_provenance' (107015462) panicked at rustfs/tests/connect_perf_drive.rs:527:5:
assertion `left == right` failed
left: Number(1001)
right: 1000
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
FAIL [ 0.015s] rustfs::connect_perf_drive real_local_write_read_is_measured_and_scratch_is_removed
stdout ───
running 1 test
test real_local_write_read_is_measured_and_scratch_is_removed ... FAILED
failures:
failures:
real_local_write_read_is_measured_and_scratch_is_removed
test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 9 filtered out; finished in 0.01s
stderr ───
thread 'real_local_write_read_is_measured_and_scratch_is_removed' (107016667) panicked at rustfs/tests/connect_perf_drive.rs:111:5:
assertion `left == right` failed
left: 1001
right: 1000
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
────────────
Summary [ 13.985s] 2 tests run: 0 passed, 2 failed, 8 skipped
FAIL [ 13.968s] rustfs::connect_perf_drive production_cli_measures_and_signs_exact_binary_provenance
FAIL [ 0.015s] rustfs::connect_perf_drive real_local_write_read_is_measured_and_scratch_is_removed
error: test run failed
exit_code=100
Transient production mutations were restored byte-for-byte before the complete green rerun. Original/restored SHA256: 8d389e6d8836c041aab8c6a93e56addd65ba3f70b33264e1c74f639d0572a2da.
$ git diff --check
exit_code=0
$ git diff --cached --check
exit_code=0
Independent orchestrator acceptance
Tested code tree: e67c72fb01
Tested tree: 16fbbd2ea4
shellcheck --version && actionlint -> exit 0
cargo fmt --all --check -> exit 0
./scripts/check_unsafe_code_allowances.sh -> exit 0
./scripts/check_layer_dependencies.sh -> exit 0
./scripts/check_architecture_migration_rules.sh -> exit 0
./scripts/check_logging_guardrails.sh -> exit 0
./scripts/check_error_other_format_ratchet.sh -> exit 0
./scripts/check_no_tokio_io_uring.sh -> exit 0
./scripts/check_extension_schema_boundaries.sh -> exit 0
./scripts/check_body_cache_whitelist.sh -> exit 0
./scripts/check_s3s_footprint.sh -> exit 0
./scripts/check_test_count_ratchet.sh -> exit 0
./scripts/check_fips_wording.sh -> exit 0
./scripts/check_embedded_secrets.sh -> exit 0
make script-tests -> exit 0
python3 ./scripts/test_object_generation_protocol_model.py -> exit 0
python3 ./scripts/check_test_wiring.py -> exit 0
./scripts/check_no_planning_docs.sh -> exit 0
./scripts/check_uring_lane_lib_only.sh -> exit 0
$ LANG=en_US.UTF-8 CARGO_BUILD_JOBS=1 CARGO_INCREMENTAL=0 cargo nextest run --profile ci -p rustfs --test connect_perf_drive --no-fail-fast
info: experimental features enabled: setup-scripts
warning: linker stderr: ld: __eh_frame section too large (max 16MB) to encode dwarf unwind offsets in compact unwind table, performance of exception handling might be affected
|
= note: `#[warn(linker_messages)]` on by default
warning: `rustfs` (bin "rustfs-cli") generated 1 warning
warning: `rustfs` (bin "rustfs") generated 1 warning (1 duplicate)
Finished `test` profile [unoptimized + debuginfo] target(s) in 0.69s
────────────
Nextest run ID cd1edf4c-f1c0-456d-ad1a-76cc00ce05c9 with nextest profile: ci
Starting 10 tests across 1 binary
PASS [ 0.012s] rustfs::connect_perf_drive cancellation_and_single_collector_leave_no_scratch
PASS [ 0.010s] rustfs::connect_perf_drive cleanup_failure_overrides_cancellation_and_remains_attributed
PASS [ 0.010s] rustfs::connect_perf_drive consent_and_resource_budgets_fail_before_scratch_io
PASS [ 0.009s] rustfs::connect_perf_drive invalid_roots_and_existing_scratch_fail_without_clobbering
PASS [ 20.328s] rustfs::connect_perf_drive production_cli_measures_and_signs_exact_binary_provenance
PASS [ 5.527s] rustfs::connect_perf_drive production_cli_rejects_missing_l1_consent_and_limits_before_artifacts
PASS [ 0.013s] rustfs::connect_perf_drive read_only_root_and_full_drive_errors_are_classified
PASS [ 0.019s] rustfs::connect_perf_drive real_local_write_read_is_measured_and_scratch_is_removed
PASS [ 0.011s] rustfs::connect_perf_drive runtime_timeout_and_full_drive_use_schema_valid_failed_reason
PASS [ 0.049s] rustfs::connect_perf_drive successful_result_has_signed_bounded_private_offline_export
────────────
Summary [ 25.991s] 10 tests run: 10 passed, 0 skipped
exit_code=0
All 54 tracked Cargo.toml files parse without duplicate keys. git diff --check passed. The production source remains unchanged; the two deliberate implementation mutations above were restored before this complete green run.
* fix(test): compare legacy body wire parity as raw bytes
Lossy UTF-8 comparison concealed a changed binary response byte.
Compare the already captured Vec<u8> values directly.
Validation:
- Binary mutant: migrated payload 0x84 vs legacy 0x83; old comparator
passed 1/1, raw comparator failed 0/1 (exit 100); restored green.
- Non-Swift body-drain mutant: untouched-body delegation test failed
with empty vs payload (exit 100); restored handler without a diff.
- cargo nextest run --profile ci -p rustfs-s3-types -p rustfs-protocols
--features rustfs-protocols/swift --no-fail-fast:
543 tests run, 543 passed, 0 skipped after both mutants restored.
- cargo fmt --all --check and git diff --check passed.
Orchestrator acceptance (LANG=en_US.UTF-8):
cargo nextest run --profile ci -p rustfs-s3-types -p rustfs-protocols --features rustfs-protocols/swift --no-fail-fast
Summary [2.445s] 543 tests run: 543 passed, 0 skipped.
All shared quick checks on the final code tree:
shellcheck --version && actionlint -> exit 0
cargo fmt --all --check -> exit 0
./scripts/check_unsafe_code_allowances.sh -> exit 0
./scripts/check_layer_dependencies.sh -> exit 0
./scripts/check_architecture_migration_rules.sh -> exit 0
./scripts/check_logging_guardrails.sh -> exit 0
./scripts/check_error_other_format_ratchet.sh -> exit 0
./scripts/check_no_tokio_io_uring.sh -> exit 0
./scripts/check_extension_schema_boundaries.sh -> exit 0
./scripts/check_body_cache_whitelist.sh -> exit 0
./scripts/check_s3s_footprint.sh -> exit 0
./scripts/check_test_count_ratchet.sh -> exit 0
./scripts/check_fips_wording.sh -> exit 0
./scripts/check_embedded_secrets.sh -> exit 0
make script-tests -> exit 0
python3 ./scripts/test_object_generation_protocol_model.py -> exit 0
python3 ./scripts/check_test_wiring.py -> exit 0
./scripts/check_no_planning_docs.sh -> exit 0
./scripts/check_uring_lane_lib_only.sh -> exit 0
Final independent acceptance after the drive-test CI repair
Parent: a99d8f4f89
Tested code tree: 93cbd96bf0
Tested tree: 075ae5161c2b62a96812bc1e3c904a7663b4f2f8
shellcheck --version && actionlint -> exit 0
cargo fmt --all --check -> exit 0
./scripts/check_unsafe_code_allowances.sh -> exit 0
./scripts/check_layer_dependencies.sh -> exit 0
./scripts/check_architecture_migration_rules.sh -> exit 0
./scripts/check_logging_guardrails.sh -> exit 0
./scripts/check_error_other_format_ratchet.sh -> exit 0
./scripts/check_no_tokio_io_uring.sh -> exit 0
./scripts/check_extension_schema_boundaries.sh -> exit 0
./scripts/check_body_cache_whitelist.sh -> exit 0
./scripts/check_s3s_footprint.sh -> exit 0
./scripts/check_test_count_ratchet.sh -> exit 0
./scripts/check_fips_wording.sh -> exit 0
./scripts/check_embedded_secrets.sh -> exit 0
make script-tests -> exit 0
python3 ./scripts/test_object_generation_protocol_model.py -> exit 0
python3 ./scripts/check_test_wiring.py -> exit 0
./scripts/check_no_planning_docs.sh -> exit 0
./scripts/check_uring_lane_lib_only.sh -> exit 0
$ LANG=en_US.UTF-8 CARGO_BUILD_JOBS=1 CARGO_INCREMENTAL=0 cargo nextest run --profile ci -p rustfs-s3-types -p rustfs-protocols --features rustfs-protocols/swift --no-fail-fast
Finished `test` profile [unoptimized + debuginfo] target(s) in 10m 08s
Nextest run ID 5c7ff102-ec6d-479a-86e0-810859b28bbf with nextest profile: ci
Starting 543 tests across 8 binaries
Summary [ 6.729s] 543 tests run: 543 passed, 0 skipped
exit_code=0
The only source change is the raw-byte wire comparator. Cargo manifests and the ratchet baselines remain unchanged from the independently validated parent. The invalid-byte and consumed-request mutations remain recorded above; their implementation changes were restored.
* fix: repair integration checks and IPC test readiness
Wait for the RPC/locks test sockets to pass the existing owner and mode
checks inside the original 5s readiness window and 10ms poll interval.
Use the existing admin S3 carrier for the new recovery and quota error
calls. Regenerate the test-count baseline for main's 118 existing tests
across seven crates. Pair the private pagination helper's raw-scan inputs
and remove the redundant Copy clone from the retirement event.
Keep the original assertions, responses, authorization, locks and storage
operations. The pagination body and raw-scan expressions at all five calls
are byte-identical; reinserting only the Copy clone restores the entire
base heal file. All ignored baseline values and s3s counts stay unchanged.
Validation base: c47432e353
Validation tree: f081084bc5
Patch SHA-256: 2dcf0028691bfb2772edfb38e262829b04f4151832dcce823c9dd9f7434b52de
Trace production-prefix SHA-256: fd994f218a21429d62866f4efb8eee1ee57b58f4a51865aa35d05a54ef1ccb45
All commands used LANG=en_US.UTF-8, CARGO_BUILD_JOBS=1,
CARGO_INCREMENTAL=0 and CARGO_NET_OFFLINE=true in one exclusive target.
Every gate retained the same tree and six source hashes before and after.
54 Cargo.toml files parse without duplicate keys.
cargo clippy -p rustfs --all-targets -- -D warnings
Finished `dev` profile [unoptimized + debuginfo] target(s) in 12m 50s
Exit: 0. Wall time: 770.855s.
The original 19 quick-check steps:
shellcheck --version && actionlint -> exit 0
cargo fmt --all --check -> exit 0
./scripts/check_unsafe_code_allowances.sh -> exit 0
./scripts/check_layer_dependencies.sh -> exit 0
./scripts/check_architecture_migration_rules.sh -> exit 0
./scripts/check_logging_guardrails.sh -> exit 0
./scripts/check_error_other_format_ratchet.sh -> exit 0
./scripts/check_no_tokio_io_uring.sh -> exit 0
./scripts/check_extension_schema_boundaries.sh -> exit 0
./scripts/check_body_cache_whitelist.sh -> exit 0
./scripts/check_s3s_footprint.sh -> exit 0
./scripts/check_test_count_ratchet.sh -> exit 0
./scripts/check_fips_wording.sh -> exit 0
./scripts/check_embedded_secrets.sh -> exit 0
make script-tests -> exit 0
python3 ./scripts/test_object_generation_protocol_model.py -> exit 0
python3 ./scripts/check_test_wiring.py -> exit 0
./scripts/check_no_planning_docs.sh -> exit 0
./scripts/check_uring_lane_lib_only.sh -> exit 0
Wall time: 286.337s.
cargo nextest run --profile ci -p rustfs --no-fail-fast
Summary [ 217.017s] 5382 tests run: 5382 passed (4 leaky), 40 skipped
Exit: 0. Wall time: 1324.414s.
cargo nextest run --profile ci -p rustfs-s3-types -p rustfs-protocols --features rustfs-protocols/swift --no-fail-fast
Summary [ 2.189s] 543 tests run: 543 passed, 0 skipped
Exit: 0. Wall time: 123.192s.
cargo nextest run --profile ci -p rustfs-ecstore list_objects_paginate --no-fail-fast
Summary [ 0.170s] 8 tests run: 8 passed, 5921 skipped
Exit: 0. Wall time: 261.324s.
cargo nextest run --profile ci -p rustfs-heal durable_mrf_anchor_retires_only_after_exact_bucket_generation_is_deleted --no-fail-fast
Summary [ 0.805s] 1 test run: 1 passed, 835 skipped
Exit: 0. Wall time: 199.801s.
Baseline regeneration:
./scripts/check_test_count_ratchet.sh --update
wrote .config/test-count-baseline.txt: 52 crates, 21340 tests, 164 ignored
Test-only increases: ecstore48, heal3, iam14, io-metrics2, obs3, policy14,
rustfs34. Existing ignored values and all other rows are byte-identical.
s3s footprint OK: files importing s3s is 195 (baseline files=195)
s3s footprint OK: s3s s3_error! lines is 1584 (baseline s3_error_lines=1584)
s3s footprint OK: ecstore files referencing s3s is 36 (baseline ecstore_files=36)
Historical runtime red controls on f622 (same retained two assertions):
cargo nextest run --profile ci -p rustfs --lib -E 'test(=connect::diagnostics::trace_runtime::tests::local_top_rpc_reads_service_http_completions_and_signs_offline) | test(=connect::diagnostics::trace_runtime::tests::local_top_locks_reads_service_manager_and_signs_offline)' --no-fail-fast
Original path readiness with a temporary 500ms bind/seal gap:
Summary [ 0.863s] 2 tests run: 0 passed, 2 failed, 5113 skipped
Exit: 100.
Sealed readiness in that same gap:
Summary [ 5.621s] 2 tests run: 2 passed, 5113 skipped
Exit: 0.
Compiled M1: replace the helper predicate with path.exists():
Summary [ 0.870s] 2 tests run: 0 passed, 2 failed, 5113 skipped
Exit: 100.
Restore the sealed predicate byte-for-byte:
Summary [ 5.640s] 2 tests run: 2 passed, 5113 skipped
Exit: 0.
Both reds observe mode755 with owner_matches=true and fail at the original
RPC/locks unwraps with StateSecurity. M1 failure locations: trace_runtime.rs
2547:29 and 2713:29. No new assertions were introduced.
cargo nextest run --profile ci -p rustfs --lib -E 'test(connect::diagnostics::trace_runtime::tests::)' --no-fail-fast
Summary [ 20.748s] 28 tests run: 28 passed (1 leaky), 5087 skipped
Exit: 0. This security sweep is historical; the full final unit gate above
covers the actual c474 candidate.
The first disk-stopped compile ran no tests and is excluded from red credit.
All temporary seal instrumentation is removed from the final source.
Final unit cleanup markers: 4 leaky, 40 original skips (exit0). The ld
__eh_frame linker warning also appears verbatim in the original f622 run
and the earlier c474 run; there are no new source warnings.
* fix: remove redundant pending closure in disk test
Pass the capture-free pending future factory as a function item in the existing slow-operation test. This fixes the redundant_closure error at disk_store.rs:2914 that stopped strict Clippy before the feature tests could run.
The single-line test edit preserves the factory's result, warning and timeout behavior. Reversing the replacement reproduces the parent file byte for byte. All 220 assertion lines and the production prefix are unchanged; no tests, assertions, guards, profiles or baselines were added or relaxed.
Original Linux CI evidence (run 37888552268 at 53245be9f):
- RIO job113685483992: strict Clippy exit101, redundant closure at disk_store.rs:2914; feature tests skipped.
- Workspace Test and Lint job113685484058: same strict Clippy exit101; nextest and doc tests skipped.
These are lint failures, not runtime mutation kills or known flakes.
Native macOS arm64 validation on tree7a5bf47e7533b29479bc85207416102f1337c566, LANG=en_US.UTF-8, one build job, CARGO_INCREMENTAL=0 and offline dependencies:
shellcheck --version && actionlint -> exit 0
cargo fmt --all --check -> exit 0
./scripts/check_unsafe_code_allowances.sh -> exit 0
./scripts/check_layer_dependencies.sh -> exit 0
./scripts/check_architecture_migration_rules.sh -> exit 0
./scripts/check_logging_guardrails.sh -> exit 0
./scripts/check_error_other_format_ratchet.sh -> exit 0
./scripts/check_no_tokio_io_uring.sh -> exit 0
./scripts/check_extension_schema_boundaries.sh -> exit 0
./scripts/check_body_cache_whitelist.sh -> exit 0
./scripts/check_s3s_footprint.sh -> exit 0
./scripts/check_test_count_ratchet.sh -> exit 0
./scripts/check_fips_wording.sh -> exit 0
./scripts/check_embedded_secrets.sh -> exit 0
make script-tests -> exit 0
python3 ./scripts/test_object_generation_protocol_model.py -> exit 0
python3 ./scripts/check_test_wiring.py -> exit 0
./scripts/check_no_planning_docs.sh -> exit 0
./scripts/check_uring_lane_lib_only.sh -> exit 0
All 19 checks: exit0, 312.161s.
$ cargo clippy -p rustfs -p rustfs-ecstore --all-targets --features rio-v2 -- -D warnings
Finished `dev` profile [unoptimized + debuginfo] target(s) in 20m 11s
exit0, 1213.210s total.
$ cargo nextest run --profile ci -p rustfs-ecstore --features rio-v2 slow_disk_operation_warns_once_while_pending_and_preserves_result --no-fail-fast
Finished `test` profile [unoptimized + debuginfo] target(s) in 4m 34s
PASS [0.026s] rustfs-ecstore disk::disk_store::tests::slow_disk_operation_warns_once_while_pending_and_preserves_result
Summary [ 0.170s] 1 test run: 1 passed, 5943 skipped
exit0, 281.920s total; 5943 existing filter skips.
warning: linker stderr: ld: __eh_frame section too large (max 16MB) to encode dwarf unwind offsets in compact unwind table, performance of exception handling might be affected
This linker warning is retained; flags, profiles and allowances are unchanged.
54 tracked Cargo.toml files parsed without duplicate keys. git diff --check passed. No new assertions were introduced, so no new runtime mutation is claimed. Native compilation and the existing helper test do not substitute for actual Linux RIO execution; the new integration CI run still needs to recover before the RPC merge.
Preservation and output hashes:
parent file SHA256: 8e59ee745e3aaaa22848b943d3b10f6353b515956282cde1f3af2c394200c9ac
final file SHA256: b69ba3237061b161448ae8b62b615885d5770b98a5b4a2479bcfbd70f1bd1f17
unchanged production prefix SHA256: 7903a6e288252c420231242d7b58a25fa2a7140ea3dd430cbe7f0dde11369dcb
RIO Linux failure log SHA256:421addb9868fd88e4109a922311e54f91f0dfd70862f8ba7fcaba193482ed90b
Workspace Linux failure log SHA256:e0aea8d0f014c022252a9534968e083c98cfc1fd760da445ecd04ec3f522e26a
19 output SHA256:35a54f28d707956ed1edb29d492c4dfe8a9fe753d103b111b5a2f1b5c41c2b98
RIO Clippy output SHA256:d6046e3868f2b3992fe11cdd04e9e98eb50a87d1234d9a70dbfeb154ff61638d
slow-operation output SHA256:605b7b258d2fd09c6ef2ab2ffc90d433d0509480f1e45125d20c823759e83608
Refs rustfs/backlog#2734.
* test(ecstore): isolate tier-free recovery fixture startup
The fixture constructor can take the production recovery lock and wait for 60 seconds before this test installs its queue and seeds metadata. Disable its background recovery only while creating this fixture, then restore the environment before calling the existing production recovery entrypoint.
Only the existing test setup changes. Its production entrypoint, default lock parameters, original assertions, 30-second timeout, cancellation and cleanup remain intact. No tests or assertions were added. Production and store/init.rs are byte-identical to the parent.
Independent acceptance ran on merge tree 8ec358cb42ba9ebb3bcfeafcef048a3f0366355b; the runner header reports parent HEAD, while before/after index and source hashes confirm this candidate. Worker task branch d8055766b8 has the same tree.
Every command used LANG=en_US.UTF-8. Task compilation also used CARGO_BUILD_JOBS=1, CARGO_INCREMENTAL=0, CARGO_NET_OFFLINE=true, an independent target, and empty RUSTC_WRAPPER/RUSTC_WORKSPACE_WRAPPER to keep compilers in its owned process group without changing repository configuration.
Command:
shellcheck --version && actionlint
Exit 0; observed output excerpt:
version: 0.11.0
Log SHA-256: fc9eec16b4191d058a295813ca9afbe9d2b37b59975da860170726be162a3924
Command:
cargo fmt --all --check
Exit 0; observed output excerpt:
(no stdout or stderr)
Log SHA-256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
Command:
./scripts/check_unsafe_code_allowances.sh
Exit 0; observed output excerpt:
(no stdout or stderr)
Log SHA-256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
Command:
./scripts/check_layer_dependencies.sh
Exit 0; observed output excerpt:
Layer dependency guard passed (no new reverse dependencies/cycles).
Log SHA-256: 69c6cb1896c0586b28d1167380d3260d99d8017a4265c8df5a2348f289e93a12
Command:
./scripts/check_architecture_migration_rules.sh
Exit 0; observed output excerpt:
Architecture migration rules passed.
Log SHA-256: 28790e70d070607ed4de0704f292a65030be92ec224ef106d4b72fd8296f896e
Command:
./scripts/check_logging_guardrails.sh
Exit 0; observed output excerpt:
✅ Logging guardrails check passed
Log SHA-256: 7adf41abff8ed8f7617decf86498888ddde78937099dd9ce47f6d359dfc2d317
Command:
./scripts/check_error_other_format_ratchet.sh
Exit 0; observed output excerpt:
error(format!) ratchet passed (total call sites: 579).
Log SHA-256: 66172aef18f150ba0605af122e26ffcf91f6dd86f43e38b05e2e982ce8d783ee
Command:
./scripts/check_no_tokio_io_uring.sh
Exit 0; observed output excerpt:
(no stdout or stderr)
Log SHA-256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
Command:
./scripts/check_extension_schema_boundaries.sh
Exit 0; observed output excerpt:
Extension schema boundaries passed.
Log SHA-256: f13d0acd0e2ce5d9815241b12b4dab7d01cc11b9b65bcf6dfb7812c878532a51
Command:
./scripts/check_body_cache_whitelist.sh
Exit 0; observed output excerpt:
Body-cache whitelist guard passed.
Log SHA-256: 6b828847b0383950ee6eef9cd846d0c322c945e47efd4c7d1ad881178810d6bb
Command:
./scripts/check_s3s_footprint.sh
Exit 0; observed output excerpt:
s3s footprint OK: files importing s3s is 195 (baseline files=195)
s3s footprint OK: s3s s3_error! lines is 1584 (baseline s3_error_lines=1584)
s3s footprint OK: ecstore files referencing s3s is 36 (baseline ecstore_files=36)
✅ s3s footprint baseline check passed
Log SHA-256: 47f53d04ef645618563fe1f22abc64667f19078f1974feedcca46e4f2c016844
Command:
./scripts/check_test_count_ratchet.sh
Exit 0; observed output excerpt:
test-count ratchet OK: 52 crates, 21340 tests, 164 ignored match .config/test-count-baseline.txt
Log SHA-256: 618a5e330e933de0c6ff484546df401457024a88bb5986d11d588794e33dab60
Command:
./scripts/check_fips_wording.sh
Exit 0; observed output excerpt:
FIPS wording guard passed (README.md and CHANGELOG.md contain no forbidden claims).
KMS wording guard passed (crates/kms claims no Vault KV2 Transit wrapping).
Log SHA-256: 99221cc9979c480ec4f46f50d76c52b18c72fe499f12e15e4f4a22a1b8bfb140
Command:
./scripts/check_embedded_secrets.sh
Exit 0; observed output excerpt:
Embedded secret guard passed (no private key material or provider credential literal in the tree).
Log SHA-256: 437a3ceba32094d2901889965ad191e411aad32cdb18bf0b8659e818011c406b
Command:
make script-tests
Exit 0; observed output excerpt:
test_cli_requires_the_repository_default_branch (__main__.SelfTests.test_cli_requires_the_repository_default_branch) ... ok
test_config_rejects_duplicate_and_invalid_entries (__main__.SelfTests.test_config_rejects_duplicate_and_invalid_entries) ... ok
test_freshness_boundaries (__main__.SelfTests.test_freshness_boundaries) ... ok
test_grace_requires_two_successful_queries_with_no_history (__main__.SelfTests.test_grace_requires_two_successful_queries_with_no_history) ... ok
test_invalid_api_evidence_fails_closed (__main__.SelfTests.test_invalid_api_evidence_fails_closed) ... ok
test_report_retains_every_workflow (__main__.SelfTests.test_report_retains_every_workflow) ... ok
test_requests_filter_schedule_default_branch_and_success_on_server (__main__.SelfTests.test_requests_filter_schedule_default_branch_and_success_on_server) ... ok
Ran 10 tests in 0.042s
OK
Ran 30 tests in 20.627s
OK
Ran 10 tests in 2.268s
OK
Ran 33 tests in 0.127s
OK
Ran 7 tests in 0.002s
OK
Ran 7 tests in 0.001s
OK
Ran 4 tests in 0.108s
OK
Ran 3 tests in 0.564s
OK
PASS: cold stampede script self-test
Log SHA-256: 75fda2d069a373c63667b908baaf99e6591aab9099a739b0c4c22ceb6da8d68c
Command:
python3 ./scripts/test_object_generation_protocol_model.py
Exit 0; observed output excerpt:
test_an_unavailable_decision_quorum_cannot_choose_a_candidate (__main__.ObjectGenerationProtocolModel.test_an_unavailable_decision_quorum_cannot_choose_a_candidate) ... ok
test_counter_allocation_requires_persistence_and_is_consumed_once (__main__.ObjectGenerationProtocolModel.test_counter_allocation_requires_persistence_and_is_consumed_once) ... ok
test_counter_restart_burns_persisted_allocations_before_or_after_issue (__main__.ObjectGenerationProtocolModel.test_counter_restart_burns_persisted_allocations_before_or_after_issue) ... ok
test_crash_after_promise_before_reply_does_not_restore_an_older_ballot (__main__.ObjectGenerationProtocolModel.test_crash_after_promise_before_reply_does_not_restore_an_older_ballot) ... ok
test_duplicate_or_unconfigured_voters_cannot_form_a_quorum (__main__.ObjectGenerationProtocolModel.test_duplicate_or_unconfigured_voters_cannot_form_a_quorum) ... ok
test_empty_or_unobserved_minority_slot_can_retry_without_allocating_another_revision (__main__.ObjectGenerationProtocolModel.test_empty_or_unobserved_minority_slot_can_retry_without_allocating_another_revision) ... ok
test_equal_ballot_cannot_accept_different_bytes_and_lower_ballot_is_rejected (__main__.ObjectGenerationProtocolModel.test_equal_ballot_cannot_accept_different_bytes_and_lower_ballot_is_rejected) ... ok
test_forgetting_tombstone_authority_allows_an_aba_create (__main__.ObjectGenerationProtocolModel.test_forgetting_tombstone_authority_allows_an_aba_create) ... ok
test_issuing_before_persistence_reuses_a_ballot_after_restart (__main__.ObjectGenerationProtocolModel.test_issuing_before_persistence_reuses_a_ballot_after_restart) ... ok
test_lost_ack_and_restart_preserve_the_chosen_operation (__main__.ObjectGenerationProtocolModel.test_lost_ack_and_restart_preserve_the_chosen_operation) ... ok
test_minority_accepted_work_is_adopted_and_finished (__main__.ObjectGenerationProtocolModel.test_minority_accepted_work_is_adopted_and_finished) ... ok
test_omitting_highest_accepted_adoption_has_a_double_choice_counterexample (__main__.ObjectGenerationProtocolModel.test_omitting_highest_accepted_adoption_has_a_double_choice_counterexample) ... ok
test_quorum_choice_does_not_require_an_uncontacted_disk_to_learn_it (__main__.ObjectGenerationProtocolModel.test_quorum_choice_does_not_require_an_uncontacted_disk_to_learn_it) ... ok
test_recovery_failure_or_stale_expected_head_does_not_allocate_a_successor (__main__.ObjectGenerationProtocolModel.test_recovery_failure_or_stale_expected_head_does_not_allocate_a_successor) ... ok
test_replacing_voters_with_empty_state_can_choose_a_conflicting_value (__main__.ObjectGenerationProtocolModel.test_replacing_voters_with_empty_state_can_choose_a_conflicting_value) ... ok
test_skipping_predecessor_recovery_chooses_a_broken_revision_chain (__main__.ObjectGenerationProtocolModel.test_skipping_predecessor_recovery_chooses_a_broken_revision_chain) ... ok
test_stale_publication_rollback_and_gc_cannot_change_the_winner (__main__.ObjectGenerationProtocolModel.test_stale_publication_rollback_and_gc_cannot_change_the_winner) ... ok
test_superseded_operation_identity_cannot_be_reused_for_different_input (__main__.ObjectGenerationProtocolModel.test_superseded_operation_identity_cannot_be_reused_for_different_input) ... ok
test_superseded_operation_retry_returns_its_original_successor (__main__.ObjectGenerationProtocolModel.test_superseded_operation_retry_returns_its_original_successor) ... ok
test_tombstone_and_null_marker_payloads_do_not_reset_authority_revision (__main__.ObjectGenerationProtocolModel.test_tombstone_and_null_marker_payloads_do_not_reset_authority_revision) ... ok
test_tombstone_rejects_a_delayed_never_present_create (__main__.ObjectGenerationProtocolModel.test_tombstone_rejects_a_delayed_never_present_create) ... ok
test_unlearned_predecessor_is_recovered_before_the_second_slot (__main__.ObjectGenerationProtocolModel.test_unlearned_predecessor_is_recovered_before_the_second_slot) ... ok
Ran 24 tests in 0.929s
OK
Log SHA-256: 0b4be926efb0609d497ae58930340153f01957b606206cbbdb3e0d47eb29d3a2
Command:
python3 ./scripts/check_test_wiring.py
Exit 0; observed output excerpt:
OK: e2e modules, runner selection, fuzz matrices, profiles, scheduled alerts, and Quick Checks are wired
Log SHA-256: 21f676f458451525535ab0ef73d6a9c0d9774e16d76a4d19e396ad34936dd39c
Command:
./scripts/check_no_planning_docs.sh
Exit 0; observed output excerpt:
(no stdout or stderr)
Log SHA-256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
Command:
./scripts/check_uring_lane_lib_only.sh
Exit 0; observed output excerpt:
OK: no 'uring_'-matching test functions under crates/ecstore/tests; --lib is safe
Log SHA-256: 43f3f851e27ec2ab9e34d6b05fefdede38af992699c473904bfe6f53a46d52e3
Command:
cargo nextest run --profile ci -p rustfs-ecstore --lib --features rio-v2 -E 'test(=bucket::lifecycle::bucket_lifecycle_ops::tests::tier_free_version_recovery_production_entrypoint_enqueues_seeded_item)' --no-fail-fast
Exit 0; wall 234.399s.
Observed output:
Finished `test` profile [unoptimized + debuginfo] target(s) in 3m 49s
Starting 1 test across 1 binary (5871 tests skipped)
SETUP PASS [ 0.006s] ecstore-base-stack: sh -c 'echo RUST_MIN_STACK=4194304 >> "$NEXTEST_ENV"'
PASS [ 0.440s] rustfs-ecstore bucket::lifecycle::bucket_lifecycle_ops::tests::tier_free_version_recovery_production_entrypoint_enqueues_seeded_item
Summary [ 0.586s] 1 test run: 1 passed, 5871 skipped
Log SHA-256: 34e973f86e8dc2102c1a0e483e0ac802ce7df3d1b40a603932f62c0805c4de27
Command:
cargo nextest run --profile ci -p rustfs-ecstore --lib --features rio-v2 -E 'test(=bucket::lifecycle::bucket_lifecycle_ops::tests::tier_free_version_recovery_eight_coordinators_hold_one_leader_and_take_over) | test(=store::bucket::tests::creation_commit_backfill_runs_under_the_config_write_fence) | test(=store::bucket::tests::orphaned_bucket_recovery_requires_every_volume_absent_and_is_retryable)' --no-fail-fast
Exit 0; wall 4.010s.
Observed output:
Finished `test` profile [unoptimized + debuginfo] target(s) in 0.43s
Starting 3 tests across 1 binary (5869 tests skipped)
SETUP PASS [ 0.005s] ecstore-base-stack: sh -c 'echo RUST_MIN_STACK=4194304 >> "$NEXTEST_ENV"'
PASS [ 0.068s] rustfs-ecstore bucket::lifecycle::bucket_lifecycle_ops::tests::tier_free_version_recovery_eight_coordinators_hold_one_leader_and_take_over
PASS [ 0.637s] rustfs-ecstore store::bucket::tests::creation_commit_backfill_runs_under_the_config_write_fence
PASS [ 0.846s] rustfs-ecstore store::bucket::tests::orphaned_bucket_recovery_requires_every_volume_absent_and_is_retryable
Summary [ 0.987s] 3 tests run: 3 passed, 5869 skipped
Log SHA-256: b5da2a927ff18ca9aa217762f7186a6d4371bd9e8feedd7b77940e10b64dffac
Command:
cargo clippy -p rustfs -p rustfs-ecstore --all-targets --features rio-v2 -- -D warnings
Exit 0; wall 1084.508s.
Observed output:
Finished `dev` profile [unoptimized + debuginfo] target(s) in 18m 03s
Log SHA-256: 28f98be7970faaa71cd118da1da4e6e2e3d23b9dda7e6449eaa7a37e741260b5
Compiled mutation command (unchanged for each red and restored run):
cargo nextest run --profile ci -p rustfs-ecstore --lib --features rio-v2 -E 'test(=bucket::lifecycle::bucket_lifecycle_ops::tests::tier_free_version_recovery_production_entrypoint_enqueues_seeded_item)' --no-fail-fast
M1: reverse only this test-local constructor flag from false to true. The mutant compiled, then failed the original recovered-item queue assertion after 30.481s. Restoring the entire observed candidate passed in 0.460s. Red log 55bf3b1535529ac0bcfb8071f9b56f9f40631007a85c315367e7e93786d65293; restored log 204d5fbe4574a7917f5e740a795d864fe28a7e7fb5e33d27b0a9cf68e2f17f52.
M2: replace only the actual awaited enqueue with false, consuming its original inputs. The mutant compiled, then failed the same original queue assertion after 30.516s. Restoring the entire observed candidate passed in 0.457s. Red log 89186330413215e231e2b06bf013cc31097ea8f4a106d91421a36d6c8986a7fd; restored log e848b42d2302401afa5dbcd9e37bf76d2e56a96ab6de5a488513b997f53c5e9b.
All temporary observation and mutation code was removed before final acceptance. Historical baseline and diagnostic failures remain recorded; they are not mutation credit. The separate Object Lock intent-retry diagnostic still fails, and these native macOS checks do not establish Linux CI recovery.
* refactor(rpc): serve internode rpc bodies with the owned Body (#8441)
* test(rpc): pin the wire bytes of every internode rpc response body
rustfs/backlog#2747 batch 4, step 1 of 2, on the s3s bodies the rpc paths
serve today. Each response body builder in storage/rpc/http_service.rs is
served through hyper's HTTP/1.1 server and the raw bytes the peer reads are
compared with goldens captured from this commit: status, content-length or
chunked framing, chunk boundaries, the terminating chunk, HEAD, and the cut
connection a failure after data produces. The capability responses carry a
per-process proof, so their length header and payload are checked against
the body they encode instead.
The read_file_stream response builder moves into read_file_stream_response
so the test reaches the same code the handler runs; behaviour is unchanged.
* refactor(rpc): serve internode rpc bodies with the owned Body
rustfs/backlog#2747 batch 4, step 2 of 2. The internode rpc HTTP paths
build their responses with rustfs_s3_types::Body: Body::from_stream
replaces Body::from(StreamingBlob::wrap(..)) at the six stream sites, and
the in-memory and empty bodies keep their constructors. InternodeRpcService
converts its responses into the wrapped service's response body through
From<Body>, so the server keeps composing it with the s3s service as
before. The wire-byte goldens from the previous commit pass unchanged;
storage/rpc no longer names s3s.
* fix(rpc): simplify wire test response builder type
Name the boxed response builder used by the capability wire test so
all-target Clippy can check it without a type-complexity warning.
Validation (LANG=en_US.UTF-8, CARGO_INCREMENTAL=0):
- cargo clippy -p rustfs --all-targets -- -D warnings: exit 0.
- cargo nextest run --profile ci -p rustfs --lib -E 'test(/storage::rpc::/)': 279 tests run, 279 passed, 4842 skipped.
- All 19 quick-check commands, including script-tests, both Python scripts and cargo fmt --all --check: exit 0.
- Footprint baseline regenerated: files=194, s3_error_lines=1584, ecstore_files=36; test-count baseline regenerated: 52 crates, 21228 tests, 164 ignored.
- All 54 Cargo.toml files parse without duplicate keys.
No assertion was added or changed by this lint fix. R1-R8 are retained
in the existing PR verification record.
* fix(ecstore): resume fenced Object Lock creation intents
* fix(ecstore): bound creation retry futures on the stack
* fix(ecstore): require native proof for creation retries
* fix(ecstore): bound bucket creation stack usage (#8466)
* fix(ecstore): retain creation fences through rollback
Keep lifecycle, metadata-transaction, and namespace guards around rollback and metadata cleanup. Preserve the original publication error when a guard is lost. Build the timestamp retry fixture through the real interrupted producer so it checks a valid pending creation.
Validated source tree: 4e8e207999.
Two independent source reviews found no blocking regression.
Validation (LANG=en_US.UTF-8):
- bash -c 'shellcheck --version && actionlint' — exit 0
- cargo fmt --all --check — exit 0
- ./scripts/check_unsafe_code_allowances.sh — exit 0
- ./scripts/check_layer_dependencies.sh — exit 0
- ./scripts/check_architecture_migration_rules.sh — exit 0
- ./scripts/check_logging_guardrails.sh — exit 0
- ./scripts/check_error_other_format_ratchet.sh — exit 0
- ./scripts/check_no_tokio_io_uring.sh — exit 0
- ./scripts/check_extension_schema_boundaries.sh — exit 0
- ./scripts/check_body_cache_whitelist.sh — exit 0
- ./scripts/check_s3s_footprint.sh — exit 0
- ./scripts/check_test_count_ratchet.sh — exit 0
- ./scripts/check_fips_wording.sh — exit 0
- ./scripts/check_embedded_secrets.sh — exit 0
- make script-tests — exit 0
- python3 ./scripts/test_object_generation_protocol_model.py — exit 0
- python3 ./scripts/check_test_wiring.py — exit 0
- ./scripts/check_no_planning_docs.sh — exit 0
- ./scripts/check_uring_lane_lib_only.sh — exit 0
- cargo nextest run --profile ci -p rustfs-ecstore --lib --locked --no-fail-fast — 5894 passed, 0 skipped
- cargo check -p rustfs-ecstore --all-targets --locked — exit 0
- cargo nextest run --profile ci -p rustfs --lib --locked --no-fail-fast -E 'test(=admin::handlers::table_catalog::tests::cold_strong_registration_completes_while_other_bucket_backup_waits) | test(=admin::handlers::table_catalog::tests::commit_backend_borrowed_put_preserves_owner_authorization_and_precondition) | test(=admin::handlers::table_catalog::tests::disable_table_bucket_preserves_objects_and_can_be_reenabled) | test(=admin::handlers::table_catalog::tests::load_table_handler_negotiates_vended_credentials_without_breaking_metadata_only_callers) | test(=admin::handlers::table_catalog::tests::namespace_property_handler_updates_object_backed_catalog_and_maps_errors) | test(=app::object::get::tests::execute_get_object_resumes_from_relocated_pool_without_splicing_body) | test(=table_catalog::store::tests::catalog_writer_loss_prevents_recovery_storage_commit)' — 7 passed
- cargo nextest run --profile ci -p rustfs --lib --locked --no-fail-fast -E 'test(unsupported_remote_disk_)' — 2 passed
Additional fault-injection evidence remains incomplete. Timestamp M1 reached its original assertion; its restore run is being collected. M2–M5 and the three planned rollback faults have not run. An automated risk check interrupted the timestamp proof task; no assertions or validation settings were weakened.
* fix(ecstore): box nested bucket rollback state
Keep the guarded rollback future behind a heap allocation so bucket creation does not store its nested state inline. The rollback body, all three fence references and pending-proof revocation order are unchanged.
The same three original Linux tests overflowed on both e611 CI attempts. Local format, ecstore test compilation and all 19 quick-check steps pass; the original Linux feature lanes still need to validate this candidate.
* test(ecstore): isolate rollback guard loss fixtures
Run the three rollback metadata-cleanup cases in separate test processes so reused /proc/self/fd aliases from earlier cases cannot invalidate their creation witnesses. Keep the original fixture, guards, assertions and io_uring test selection, and verify that each child runs the intended case successfully.
Update the generated test-count baseline for the added macro arm.
Validation: cargo check -p rustfs-ecstore --tests; cargo fmt --all --check; all 19 quick-check commands, including make script-tests and the two Python checks. Linux io_uring acceptance and fault controls remain pending.
* refactor(replication): move the replication config onto gateway types (#8443)
* wip: T1.4 replication family
* refactor(replication): move the replication config onto gateway types
The replication configuration family leaves s3s: rustfs-replication no longer depends on s3s, and ECStore bucket metadata, the scanner and the rustfs app/admin layers hold rustfs_gateway_types::persistence::PersistedReplicationConfiguration. The persisted XML goes through the gateway persistence codec behind a new BucketConfigXml trait (the 39 byte goldens pass with unchanged fixtures); the legacy s3s edge converts once in storage/ecfs.rs through storage/s3_compat/replication.rs; the scanner data usage cache keeps its MessagePack bytes through a serde mirror pinned by bytes recorded from s3s 5761ddfe. rustfs-gateway-types 0.32.0 is pinned once (rev 772d3385). Refs rustfs/backlog#2745.
* test: use S3 serializer for list response tests
* fix(ci): restore integration test compilation
Gate the diagnostic test import with its feature, initialize the conditional
MetadataWriteStore fields explicitly, and decode the scanner replication fixture
through its existing cache boundary. Preserve the original fixture and assertions.
Validation was run on tree 544acb80c0 at 6003d4b8.
LANG=en_US.UTF-8; no source changes follow validation.
Quick checks: 19/19 passed.
- shellcheck --version && actionlint: exit 0
- cargo fmt --all --check: exit 0
- ./scripts/check_unsafe_code_allowances.sh: exit 0
- ./scripts/check_layer_dependencies.sh: exit 0
- ./scripts/check_architecture_migration_rules.sh: exit 0
- ./scripts/check_logging_guardrails.sh: exit 0
- ./scripts/check_error_other_format_ratchet.sh: exit 0
- ./scripts/check_no_tokio_io_uring.sh: exit 0
- ./scripts/check_extension_schema_boundaries.sh: exit 0
- ./scripts/check_body_cache_whitelist.sh: exit 0
- ./scripts/check_s3s_footprint.sh: exit 0
- ./scripts/check_test_count_ratchet.sh: exit 0
- ./scripts/check_fips_wording.sh: exit 0
- ./scripts/check_embedded_secrets.sh: exit 0
- make script-tests: exit 0
- python3 ./scripts/test_object_generation_protocol_model.py: exit 0
- python3 ./scripts/check_test_wiring.py: exit 0
- ./scripts/check_no_planning_docs.sh: exit 0
- ./scripts/check_uring_lane_lib_only.sh: exit 0
Darwin validation:
- cargo clippy --locked -p rustfs-ecstore -p rustfs-scanner --all-targets -- -D warnings: exit 0
- cargo nextest run --locked -p rustfs-ecstore -p rustfs-scanner --lib -E 'test(config::com::tests::save_config_preserves_retryable_system_volume_errors) | test(scanner_io::tests::replication_maintenance_refreshes_compacted_stats_across_cycles_and_retries)' --test-threads 1 --no-fail-fast: exit 0, 2/2 passed
- cargo clippy --locked -p rustfs-ecstore -p rustfs-scanner --all-targets --features rustfs-ecstore/diagnostic-read-version-caller,rustfs-scanner/diagnostic-read-version-caller -- -D warnings: exit 101
- cargo nextest run --locked -p rustfs-ecstore -p rustfs-scanner --lib --features rustfs-ecstore/diagnostic-read-version-caller,rustfs-scanner/diagnostic-read-version-caller -E 'test(config::com::tests::save_config_preserves_retryable_system_volume_errors) | test(scanner_io::tests::replication_maintenance_refreshes_compacted_stats_across_cycles_and_retries) | test(cluster::rpc::runtime_sources::tests::) | test(config::com::tests::diagnostic_config_write_retains_origin_and_api_for_results_and_cancellation)' --test-threads 1 --no-fail-fast: exit 0, 14/14 passed
Diagnostic strict Clippy remains red on four unchanged production lints:
runtime_sources.rs if_same_then_else and three redundant_locals reports in
set_disk/core/metadata_read.rs and set_disk/read.rs. Their source ranges and
feature wiring match the base; the base was not compiled separately.
Linux aarch64 validation (Rust/Clippy 1.99.0, UID 501, jobs=1):
- cargo clippy --all-targets -- -D warnings: exit 0; Finished dev profile in 68m 12s
- cargo clippy -p rustfs -p rustfs-ecstore --all-targets --features rio-v2 -- -D warnings: exit 0; Finished dev profile in 49m 45s
Source HEAD/tree and clean status matched before and after both commands.
No compiler or stack overrides were used. protoc 35.1 was installed from its
verified ARM64 release; flatc was absent and not invoked by the workspace build scripts.
The local environment differs from hosted CI.
* fix(ecstore): recover delete batches with missing versions
Treat an explicit non-null version as a no-op when it is absent from both recovery sources. Keep rejecting missing null versions, delete-marker mismatches and conflicting local generations.
Add returning-disk and partial-receipt regressions, extend journal boundary checks, and regenerate the ecstore test-count baseline. Production faults exercise the 24 added assertion sites and 7 semantic checks: 33 faults failed at the expected sites and passed after full restoration.
Validation (LANG=en_US.UTF-8, one build job):
cargo nextest run --locked -p rustfs-ecstore --lib -E 'test(delete_recovery) | test(quorum_delete_cleanup_coordinator_binds_single_and_batch) | test(crash_delete_retry_batch_reclaims_inline_and_reserved_external)' --test-threads 1 --no-fail-fast
Summary [ 34.580s] 69 tests run: 69 passed, 6058 skipped; exit 0
cargo clippy --locked -p rustfs-ecstore --all-targets -- -D warnings
Finished `dev` profile [unoptimized + debuginfo] target(s) in 5m 37s; exit 0
All 19 final Quick Checks completed with exit 0:
- shellcheck --version && actionlint
- cargo fmt --all --check
- ./scripts/check_unsafe_code_allowances.sh
- ./scripts/check_layer_dependencies.sh
- ./scripts/check_architecture_migration_rules.sh
- ./scripts/check_logging_guardrails.sh
- ./scripts/check_error_other_format_ratchet.sh
- ./scripts/check_no_tokio_io_uring.sh
- ./scripts/check_extension_schema_boundaries.sh
- ./scripts/check_body_cache_whitelist.sh
- ./scripts/check_s3s_footprint.sh
- ./scripts/check_test_count_ratchet.sh
- ./scripts/check_fips_wording.sh
- ./scripts/check_embedded_secrets.sh
- make script-tests
- python3 ./scripts/test_object_generation_protocol_model.py
- python3 ./scripts/check_test_wiring.py
- ./scripts/check_no_planning_docs.sh
- ./scripts/check_uring_lane_lib_only.sh
test-count ratchet OK: 52 crates, 22009 tests, 165 ignored match .config/test-count-baseline.txt
PASS: cold stampede script self-test
model: states=51023, transitions=167520, terminal=732, chosen_candidates=2
multi-slot round schedules: 640
OK: e2e modules, runner selection, fuzz matrices, profiles, scheduled alerts, and Quick Checks are wired
* fix(ecstore): retry tier commit intent lock contention
Retry transient namespace lock timeouts within the existing three-attempt
CAS budget. Reload and verify the original mutation identity after
contention, and preserve the final typed error when attempts are exhausted.
Cover save/read contention, exhausted retries, replacement identities and
CAS reconciliation without changing the existing timeout or lock policy.
Validation:
- cargo nextest run --locked -p rustfs-ecstore --features test-util --lib -E 'test(tier_mutation)' --test-threads 1 --no-fail-fast: 52 passed.
- cargo nextest run --locked -p rustfs-ecstore --features test-util --lib -E 'test(intent_advance_) | test(=services::tier::tier::tests::caller_cancellation_after_durable_prepare_does_not_hide_the_mutation)' --test-threads 1 --no-fail-fast: 5 passed.
- cargo clippy --locked -p rustfs-ecstore --all-targets -- -D warnings: passed.
- cargo clippy --locked -p rustfs-ecstore --all-targets --features test-util -- -D warnings: passed.
- 27 mutation pairs failed at the intended assertions and passed after restoring the production code.
- bash scripts/check_test_count_ratchet.sh --update, then bash scripts/check_test_count_ratchet.sh: 52 crates, 22013 tests, 165 ignored; ecstore increases from 6581 to 6585.
* refactor(lifecycle): replace s3s domain types with Gateway DTOs
Keep lifecycle rules, restore requests, Object Lock, and scanner cached tags in domain types. Convert at the existing HTTP boundary while preserving legacy XML, scanner cache bytes, date ordering, and tombstones.
Linux test validation before the final import-path-only adjustment:
cargo nextest run -p rustfs-ecstore bucket_config_goldens
39 passed; 6211 skipped by selector.
cargo nextest run --lib -p rustfs -E 'test(/test_zero_rule_lifecycle_tombstone_recognition/) | test(/test_local_lifecycle_staleness_axis_selection/) | test(/test_lifecycle_staleness_preserves_legacy_submillisecond_axis/) | test(/test_merge_incoming_lifecycle/) | test(/test_lifecycle_expiry_statement_matrix/) | test(/test_lifecycle_expiry_subset_xml/) | test(/put_bucket_lifecycle_validation_errors_keep_their_s3_code/) | test(/execute_put_bucket_lifecycle_configuration_rejects_missing_configuration/) | test(/execute_restore_object_rejects_missing_restore_request/) | test(/execute_restore_object_rejects_select_type/) | test(/execute_restore_object_maps_failures_to_typed_s3_errors/) | test(/execute_restore_object_returns_internal_error_when_store_uninitialized/) | test(/restore_request_conversion_preserves_legacy_output_values/) | test(/restore_request_conversion_preserves_presence_and_raw_enums/) | test(/restore_request_conversion_distinguishes_absent_and_empty_containers/)'
25 passed; 5318 skipped by selector.
cargo nextest run --profile ci -p rustfs-ecstore -p rustfs-lifecycle -p rustfs-replication -p rustfs-scanner
7872 passed; 21 skipped under the existing CI profile.
Previously verified mutation controls: 39 production faults, 41 RED/GREEN pairs.
cargo clippy --locked -p rustfs -p rustfs-ecstore -p rustfs-lifecycle -p rustfs-scanner --all-targets -- -D warnings
Passed in 40m 45s before the import-path adjustment, then in 1m 59s on
the final source with warnings denied. The final adjustment imports the
existing helper through the object domain facade; fmt and the architecture
check also passed.
Part of rustfs/backlog#2745. The remaining ecstore configuration families follow separately.
* refactor(admin): move error types to rustfs-s3-types (#8440)
* refactor(protocols): move error types to rustfs-s3-types by codemod
T1.2 batch 3 (rustfs/backlog#2743): s3select-api, s3select-query and
the error items of protocols move off s3s by scripts/codemods, run with
--hold crates/protocols/src/sftp/errors.rs. That file stays on s3s: it
downcasts the backend error, whose runtime type is still s3s::S3Error.
- rustfs/src/protocols/client.rs: the two webdav session methods now
return rustfs_s3_types::S3Result and convert the s3s error with From.
- codemod: a child module that calls s3_error! through `use super::*`
gets its own `use rustfs_s3_types::s3_error;` once its parent imports
that macro by name; otherwise it is listed as GLOB.
- s3-types: a table test pins code, status and the s3s error the legacy
edge renders for every variant a moved call site names.
- s3_error_lines 1585 -> 1584 (a lifecycle comment reworded).
* refactor(admin): move error types to rustfs-s3-types (WIP, no build)
T1.2 batch 4 (rustfs/backlog#2743), work in progress: rustfs check
still fails with 16 errors (listed in the PR body).
- codemod: --facade PATH treats a crate-local re-export of s3s's error
items like s3s for the four names (9 new self-tests, 9 mutants red).
- codemod output for rustfs/src/admin and rustfs/src/site_replication
(--facade crate::storage_api::site_replication::s3); 6 glob children
get an explicit rustfs_s3_types::s3_error import.
- admin/router.rs: the Operation trait returns rustfs_s3_types::S3Result.
- error.rs: From<ApiError> builds the rustfs-s3-types error; the s3s
conversion is derived from it.
* fix(admin): convert error types at the s3s boundary
Batch 4 of T1.2 left the admin and site_replication modules returning
rustfs_s3_types::S3Error while their callers and callees on the storage
and auth side still speak s3s::S3Error. Convert at each edge with the
existing From impls in rustfs-s3-types (compat_s3s.rs) instead of
changing any code, message or status:
- admin/auth.rs, admin/handlers/object_zip_download.rs,
admin/handlers/table_catalog/mod.rs: map the s3s result returned by
check_key_valid / authorize_request / authorize_internal_object_request
into the admin error type with .map_err(Into::into).
- storage/ecfs.rs: the s3s-facing side maps the admin site_replication
gate results the other way.
- admin/handlers/pools.rs: ApiError.code stays s3s::S3ErrorCode, so the
rustfs code is converted with .into() when the struct is built, in
production and in the two tests that build or compare it.
- admin/handlers/heal.rs, admin/runtime_sources.rs: the router returns
s3s errors (S3Route impl); the expected rustfs code is converted with
.into() before the comparison.
- admin/handlers/table_catalog/mod.rs: iceberg_rest_error takes
&'static str so the Cow<'static, str> custom code borrows it; every
caller already passes an ICEBERG_ERROR_* constant or a &'static str.
- admin/handlers/quota.rs: drop a now same-type .into() after s3_error!.
- storage_api.rs, site_replication/mod.rs: drop re-exports and an import
that no longer have users (clippy -D warnings).
cargo check -p rustfs --all-targets and cargo clippy -p rustfs
--all-targets -- -D warnings are clean; the s3s footprint baseline is
unchanged at files=172, s3_error_lines=303, ecstore_files=28.
Refs rustfs/backlog#2743, rustfs/backlog#2734.
* refactor(ecstore): move bucket configs onto gateway persisted types (#8709)
* refactor(ecstore): move bucket configs onto gateway persisted types
Finish the ECStore batch of T1.4 (rustfs/backlog#2745). The remaining
eleven bucket configuration families (Website, CORS, Tagging, Versioning,
ObjectLock, Encryption, Notification, Logging, Accelerate, RequestPayment,
PublicAccessBlock) now read and write their stored XML through the
rustfs-gateway-types persistence and cors_tagging codecs instead of the
s3s codec, so the legacy s3s bucket-config codec and its
s3gate-metadata-xml compat marker are removed.
- ecstore: BucketConfigXml is implemented by the gateway codec for every
family; metadata, metadata_sys, versioning, object lock, replication and
lifecycle consumers take the persisted types. Region moves to
rustfs_s3_types::Region, and the ETag conversion to s3s moves to the
rustfs edge with its tests. Only warm_backend.rs and warm_backend_s3.rs
still reference s3s, held for T1.5 (rustfs/backlog#2746).
- scanner: object lock cache, item actions and data usage use the
persisted types.
- rustfs: the legacy S3 edge converts each family between s3s DTOs and the
persisted types in storage/storage_api.rs by writing one codec's XML and
reading it with the other; Tagging converts field by field.
- Bump the rustfs-gateway-types pin to 21283e87.
The 39 persisted-byte goldens are unchanged and pass, so a document the
old codec wrote reads back and re-encodes to the same bytes. s3s footprint:
files 181 -> 146, ecstore_files 28 -> 2, s3_error_lines unchanged.
* test(app): prepare the PutBucketEncryption config from the request
The uninitialized-store test passed an empty persisted configuration, which the use case now validates before it reaches the store check, so it answered MalformedXML instead of InternalError. Convert the request's configuration the way the S3 edge does.
* style(app): format the PutBucketEncryption test
* refactor(s3-client): move remote error codes and DTO enums off s3s (#8783)
Part of T1.5 (rustfs/backlog#2746), batch 1.
* refactor(s3select): run queries on gateway DTO request shapes (#8794)
* refactor(s3select): run queries on an engine request built from gateway DTO shapes
Part of T1.5 (rustfs/backlog#2746), batch 2.
* fix(s3select): keep the select request conversion in the app layer
The layer guard forbids app -> storage; the conversion moves from storage/s3_compat/select.rs to app/select_request.rs unchanged.
* fix(heal): retry transient metadata conflicts (#8812)
* fix(heal): retry transient metadata conflicts
* chore: update heal test count baseline
* test: reserve nextest capacity for progress probes (#8818)
* test(e2e): use a non-default region for the differential negative control
Since main's region-aware CreateBucket and GetBucketLocation (#8739, #8743), us-east-1 answers an empty LocationConstraint and 200 for an owned re-create, so the control no longer reported its own region. Also refresh the e2e-full linux selection from the CI listing (682 tests).
* refactor(protocols): move StorageBackend to gateway types (#8825)
* refactor(protocols): move storage backend to gateway types
* test: update e2e test count baseline
* fix: preserve copy source keys and unknown lengths
* test: update protocols test count baseline
* fix(protocols): preserve typed SFTP backend errors
* test(protocols): use gateway list input in policy tests
* fix(protocols): clear clippy failures after T1.5 merge (#8831)
fix(protocols): clear clippy failures after merge
* fix(ecstore): finalize restore metadata after local copy-back (#8836)
* fix(ecstore): finalize restore metadata after local copy-back
* test(ecstore): ratchet added restore finalization cases
* fix(ecstore): keep restore copyback compatible with older peers
* fix(ecstore): gate restore copyback on fleet compatibility
* fix(ecstore): retain transition identity during restore copyback
* fix(ecstore): route committed restore copyback locally
* fix restore copyback cleanup
* fix(ecstore): honor restore probe test fallback
* refactor(storage): migrate option errors to domain types (#8839)
* refactor(s3): migrate private multipart validation errors (#8844)
* refactor(auth): derive policy conditions from AuthzFacts (#8442)
* refactor(auth): extract authorization facts
* test(e2e): update smoke selection baseline
* test(e2e): update full selection baseline
* refactor(admin): migrate health and prefix usage to owned requests (#8887)
* refactor(admin): add owned health router boundary
* refactor(admin): migrate prefix usage to owned request
* refactor(admin): complete the owned admin request boundary (#8949)
* refactor(admin): complete the owned admin request boundary
Every type implementing the owned admin `Operation` now serves the legacy
router through one blanket impl that converts the s3s request and response
at that single edge, so handlers register with `AdminOperation(&Handler {})`
and need no adapter statics; `LegacyAdminAdapter` is removed.
`AdminRequest` gains the accessors handlers read today (`remote_addr`,
`client_info`, `request_context`, `req_info`, `req_info_mut`,
`install_req_info`, `envelope_mut`, `take_body`), the request helpers gain
envelope forms (`authenticate_envelope_credentials`,
`object_store_from_envelope`, the table-catalog `*_from_envelope` helpers,
`KmsAdminAudit::from_envelope`, `AccountAuditContext::from_envelope`,
`CallerIdentity::resolve_envelope`), `read_compatible_admin_body` accepts
either body type, and the owned `Body` gains `store_all_limited` and
`Default`. `admin::operation::test_support::TestRequest` builds owned
requests for handler tests. Unused accessors carry a per-method dead-code
expectation that each migrating handler batch removes.
No handler behavior changes; the remaining migrations follow in batches.
* chore(admin): add the T1.12 admin handler codemod
The script rewrites an admin handler file from the s3s request and response
types to the owned boundary (imports, signatures, request field and helper
calls, body access, response construction, and test request literals) and
leaves a TODO comment at every site it cannot rewrite mechanically. It is
idempotent and is removed once T1.12 completes.
* refactor(admin): migrate replication and KMS handlers to owned requests (#8951)
Move the T1.12 batch 2 admin handlers (site_replication, kms_keys, event,
replication, diagnostics, on_demand_migration, audit, account, kms_rekey,
kms_key_lifecycle, object_data_cache, tls_debug, integrity) from the s3s
request/response types to the owned AdminRequest/AdminResponse boundary.
Handler logic, authorization order, error codes and response bytes are
unchanged; the request/response plumbing now goes through the envelope
accessors and helpers introduced by the owned boundary.
The shared response helpers (utils::json_response, utils::empty_response,
handlers::admin_json_response, target_descriptor::build_json_response,
profile::authorize_profile_request) and the site replication repair
executor still answer with the s3s types, so the migrated files carry
owned-boundary forms of those helpers and convert the repair response at
its call edge. The streaming console-log body and the two devnull drains
move from the s3s byte stream to the owned Body stream/frame API.
Drop the dead_code expectations for the owned accessors and envelope
helpers this batch starts using, and lower the s3s footprint baseline to
the measured 99 files.
Part of rustfs/backlog#2788 (rustfs/backlog#2734 T1.12).
* refactor(admin): add owned forms of the shared admin response helpers
Handler batches were copying `json_response`, `empty_response`,
`admin_json_response`, and `build_json_response` into each migrated file
because the shared helpers still return the s3s response. Provide owned
forms next to each legacy helper (`owned_json_response`,
`owned_empty_response`, `owned_admin_json_response`,
`owned_build_json_response`, `authorize_profile_envelope`) with parity
tests against the legacy shape, derive `Debug` for `AdminResponse` so
tests can use `expect_err`, and fix three codemod rules: strip owned
string conversions in test credentials, annotate non-literal URI parses,
and leave a rebound request parameter immutable.
* refactor(admin): use the shared owned response helpers in batch 2 (#8954)
Replace the per-file owned-boundary copies of json_response,
empty_response, admin_json_response, build_json_response and
authorize_profile_request in the batch 2 admin handlers with the shared
owned_* helpers and authorize_profile_envelope that the boundary branch
now provides, and restore expect_err in the handler tests now that
AdminResponse implements Debug; the `.err().expect(..)` stand-ins were
failing the base branch's clippy step (clippy::err_expect under
-D warnings). The site replication repair call edge
keeps its conversion because the repair executor still answers with the
s3s response type.
Drop the dead_code expectations on the shared owned helpers this batch
now uses in non-test code.
Part of rustfs/backlog#2788 (rustfs/backlog#2734 T1.12).
* fix(admin): preserve legacy body limit error responses
* refactor(admin): move policy, ILM and batch 3 handlers to owned requests (#8956)
Batch 3 of T1.12 (rustfs/backlog#2788, parent rustfs/backlog#2734) moves
14 admin handler files to AdminRequest/AdminResponse and the owned
Operation trait: ilm_transition, policies, idp_compat, config_admin,
plugins_instances, profile_admin, group, heal, object_zip_download,
kms_key_metadata, durability, inspect_archive, sts and
gateway_key_inventory. Handler logic, authorization order, error codes
and response bytes are unchanged; test request literals use
test_support::TestRequest.
The handlers use the shared owned helpers (owned_json_response,
owned_admin_json_response, owned_build_json_response,
authorize_profile_envelope). The dead_code expectations on the
AdminRequest accessors and RequestEnvelope::uri_mut they now exercise
are removed. The legacy target_descriptor::build_json_response keeps
only its equivalence test caller and carries the matching expectation
until it retires.
object_zip_download builds the s3s request the S3 gate still reads from
the owned request facts; heal authenticates through
authenticate_envelope_credentials; the trace, inspect-archive and zip
bodies use Body::from_stream. extract_plugin_instance_filters takes the
URI so extensions.rs keeps calling it, and the two idp_compat
delegations stay on the legacy trait until their delegates migrate.
The s3s footprint baseline drops from 99 to 90 files.
* test(admin): anchor the replication diff body contract on the call
The source-contract test looked for `let body = read_compatible_admin_body(`
on one line; rustfmt wrapped that binding after the handler moved to the
owned request boundary, so the marker was no longer found. Anchor on the
call itself so formatting cannot change what the contract checks.
* refactor(admin): migrate table catalog and batch 1 to owned requests (#8955)
* fix(admin): preserve catalog body limit response bytes
* refactor(admin): migrate identity and cluster handlers to owned requests (#8966)
* refactor(admin): migrate identity and cluster handlers to owned requests
Move the T1.12 batch 4 admin handlers (user, service_account, oidc, tier,
system, pools, quota, kms_dynamic, kms_backup, rebalance, module_switch,
bucket_meta, cluster_snapshot, realtime, bucket_recovery, profile) from the
s3s request/response types to the owned AdminRequest/AdminResponse
boundary. All 90 impl Operation blocks now take AdminRequest and return
AdminResponse; handler logic, authorization order, error codes and response
bytes are unchanged. The handlers use the shared owned helpers
(owned_json_response, owned_admin_json_response, authorize_profile_envelope,
the envelope-form runtime sources and audit constructors) instead of local
copies.
profile.rs drops the s3s forms of authorize_profile_request and
profile_not_implemented_response now that profile_admin.rs is on the
envelope gate too; profile_not_implemented_response answers with an
AdminResponse. The realtime metrics stream moves from the s3s ByteStream
adapter to Body::from_stream, and the rebalance idle-404 document still
renders through s3s::S3Error::to_http_response with its body converted at
the edge.
The idp_compat delegators ImportIamV2 and AddServiceAccountLdap move to the
owned trait as well: with their targets on both traits, the method-call
delegation became ambiguous between the owned impl and the router blanket
impl, so they now take the owned request directly.
KmsAdminAudit::from_request loses its last production caller with this batch
and is deleted; its comparison test now asserts the envelope facts directly.
Lower the s3s footprint baseline to the measured 68 files.
Part of rustfs/backlog#2788 (rustfs/backlog#2734 T1.12).
* refactor(admin): remove the legacy admin helpers batch 4 superseded
Batch 4 moved the last production callers of the s3s-typed admin helpers
to their owned forms, which left `admin_json_response`, `json_response`,
`authenticate_request`, `authorize_admin_request`, `object_store_from_req`
and `object_store_from_extensions` unused, along with the `Body`,
`S3Request` and `S3Response` re-exports of the admin `s3` facade. Remove
them. The owned forms keep their documentation; the behavior tests of the
removed `json_response` now exercise `owned_json_response`, and the
compat-prefix test asserts the owned response directly.
Two tests existed only to compare a removed legacy helper with its owned
twin (`owned_json_response_matches_the_legacy_shape`) or to cover the
removed legacy gate (`authorize_admin_request_without_credentials_is_rejected`);
they go with the helpers, and the owned equivalents stay. The rustfs
test-count row drops from 5957 to 5955 and the s3s footprint from 68 to 66
files.
* refactor(admin): drop the legacy target descriptor JSON response helper (#8968)
`target_descriptor::build_json_response` returned the s3s response and
had no production caller left once the event, audit and plugin instance
handlers moved to `owned_build_json_response`; it survived only for the
legacy/owned equivalence test. Remove it, give the owned helper its own
documentation, and make the test assert the owned response directly:
status, `application/json`, the echoed `x-request-id` when present and
only then, and the body bytes. The file no longer imports s3s, so the
footprint baseline drops from 66 to 65 files.
* test(ecstore): pin native history on cancellation under the cancellable queue
main moved the local write path onto the disk blocking queue, which skips an
operation once its waiter is dropped, so a borrowed write cancelled while it is
queued no longer creates its file. The integration-only native publication
test still expected the queued open to run. Assert the new execution model
instead (the open never runs) while keeping its safety claims: the Started
record is taken before queueing and the cancelled publication never regrants
native history.
* chore(gateway): raise test-count rows for tests merged from main
* test(s3select): use the owned JSON input types in merged planner tests
* fix(ecstore): scope the merged direct-writer test hook to test builds
* fix(ecstore): create parents before admitted truncate opens
Align the heal tests with removal-disabled preservation and local-only outcome
fields, and refresh the e2e-full selection for the synced tests.
* chore(gateway): update test-count baseline after main sync
Main #9241 removed the node pool-metadata readiness cache with 8 tests
and added 6, so the rustfs row drops from 6440 to 6438. The ecstore and
s3select rows rise for tests added by #9237, #9239 and #9241.
* feat(gateway-adapter): add IAM-backed CredentialProvider
Add the ring-2 crate rustfs-gateway-adapter with IamCredentials, the
gateway CredentialProvider answered by RustFS IAM (rustfs/backlog#2760,
credentials half; PolicySource is deferred).
- Pin rustfs-gateway at the existing gateway rev with
default-features = false so the server crate stays out of the build.
- IamKeyLookup is a narrow IAM trait implemented for IamSys so the
adapter is testable without an in-memory IAM backend.
- Root credentials are injected at construction and resolve before IAM
is ready.
- Mapping: keystone: prefix -> NotFound (deliberate change from the
legacy empty secret); IAM not ready or not initialized -> Unavailable;
unknown key or NoSuch{User,ServiceAccount,TempAccount,Account} ->
NotFound; other IAM errors -> Backend; invalid or "off" principals
resolve disabled; STS credentials carry a SessionBinding, and an STS
credential without an expiry is a Backend fault rather than a
never-ending session.
- No authorization in lookup; no Debug on secret-holding types; IAM
error text is dropped after classification.
* feat(scripts): add junit_diff.py for dual-stack JUnit comparison (#9275)
* feat(gateway-adapter): add IAM-backed PolicySource
Add IamPolicySource, the gateway PolicySource answered by RustFS IAM
(rustfs/backlog#2760, policy half). One snapshot per request carries a
RustfsPrincipal: principal kind, account, effective username, parent
user, credential groups, live group memberships, token claims and
is_owner.
- Anonymous callers get an empty snapshot without asking IAM.
- IAM not ready, not initialized or failing is PolicyError::unavailable,
never an empty snapshot.
- Unknown, disabled and session-kind-mismatched principals are refused
with constant labels; unreadable tokens are malformed.
- Claims are read from the stored token (root secret for STS, the
account's own secret for service accounts), and is_owner mirrors the
legacy has_root_access rule.
- RustfsPrincipal holds no secret or session token and has no Debug.
* ci(gateway): make the test-count ratchet a floor, not exact (#9278)
ci(gateway): make the test-count ratchet a floor instead of an exact baseline
Tests below a row still fail; tests above it, a member without a row and ignores below the row now pass with a note. Ignores above a row still fail. Adds contract tests wired into make script-tests.
* chore(gateway): record ignored microbenchmarks merged from main (#9290)
chore(gateway): record the ignored microbenchmarks merged from main
main added three throughput microbenchmarks that are ignored by design
(#9217): one each in rustfs-checksums, rustfs-rio and rustfs-utils. The
test-count ratchet treats a higher ignored count as a failure until the
row is raised, so raise those three rows to 1 and tighten the test rows
to the merged tree with --update.
* fix(gateway-adapter): share live root and refuse short secrets
Address the security review of the T2.1 adapters (rustfs/backlog#2760).
- Add RootCredentials (validated: root secret and STS signing key at
least MIN_SECRET_KEY_LEN bytes), the RootSource trait and the settable
RootCell. IamAdapters::new is now the only constructor of
IamCredentials and IamPolicySource, so both read one live root source
per request; no root source answers unavailable.
- A looked-up identity whose secret is shorter than 8 bytes is NotFound
in the credential provider and DISABLED_PRINCIPAL in the policy
source, whatever the lookup reports as valid.
- STS claims are read with the STS signing key from the root source
instead of assuming it equals the root secret.
- Cover the off-status-alone branch, the OIDC owner exclusion, service
accounts of root, the name live groups are read under, and another
session's token.
* ci(s3-tests): add reference parity lane against pinned 1.0.5 (#9288)
* fix(gateway-adapter): effective secret length and unsplittable pair
Address the confirmation review of the T2.1 adapters
(rustfs/backlog#2760).
- One rule for every secret: its length after trailing NUL bytes are
removed must be at least MIN_SECRET_KEY_LEN. HMAC zero-pads its key,
so trailing NULs add nothing; interior NULs count. Applied to the root
secret, the STS signing key and every looked-up credential.
- IamAdapters keeps both halves private and hands them to a builder
together through the consuming install(builder, make_authenticator);
compile-fail doctests show a mixed pair cannot be assembled.
- Any Fn() -> Option<Arc<RootCredentials>> is a RootSource, so the
binary reads its live root accessor directly; RootCell is now a
test-only helper.
* docs(gateway-adapter): state what install guarantees about the provider handle
---------
Co-authored-by: rustfs-cla-assistant[bot] <273121464+rustfs-cla-assistant[bot]@users.noreply.github.com>
A multipart index marker is recorded before its staging root is published, and a listing prunes markers whose root is gone. The window in between was covered by a five-minute age check, which cannot prove that the publisher's filesystem work has ended: a publisher stalled past it lost its marker, and the upload it then published stayed out of ListMultipartUploads on that drive until restart.
The publisher now holds an in-memory publication pin from before it records the marker until the filesystem work that publishes the root has returned. The pin travels with that work (the blocking closure or the lease owner chain), so it also covers work that outlives a cancelled caller. Pruning reads the pin first and the staging root second, under the marker lease. The age check and the per-write marker touch are removed.
chore(gateway): record the ignored microbenchmarks merged from main
main added three throughput microbenchmarks that are ignored by design
(#9217): one each in rustfs-checksums, rustfs-rio and rustfs-utils. The
test-count ratchet treats a higher ignored count as a failure until the
row is raised, so raise those three rows to 1 and tighten the test rows
to the merged tree with --update.
Text guards that read one file with include_str! or by path broke, or silently stopped seeing the code they guard, when that code moved to another file. They now read the file they read today, which stays required, plus the sibling directory a mechanical split will populate (storage/legacy_edge, app/bucket_usecase, app/multipart_usecase, app/lifecycle_transition_api_test). Counts are summed over the set and split markers must occur exactly once across it. Refs rustfs/backlog#2765.
ci(gateway): make the test-count ratchet a floor instead of an exact baseline
Tests below a row still fail; tests above it, a member without a row and ignores below the row now pass with a note. Ignores above a row still fail. Adds contract tests wired into make script-tests.
Main #9241 removed the node pool-metadata readiness cache with 8 tests
and added 6, so the rustfs row drops from 6440 to 6438. The ecstore and
s3select rows rise for tests added by #9237, #9239 and #9241.
Port main's changes into integration's relocated code: the XML compatibility
body bound and presigned-guard tests into legacy_compat, the policy-condition
fixes (#9150, #9159, #9163) into auth facts and the owned admin request API,
and record native publications on main's new fused delete-recovery paths.
Keep native publication recording in main's native_io helpers and resource
writers, and adapt merged tests to the owned Body, Bytes write input, and
fallible find_disk.
Record native publications on the paths main's commit-fusion work opened
directly: the lazy truncate writer and the fused delete-recovery activation
and receipt renames. Keep extract_params_header in the shared audit builder.
Sync main 84e24e767 into gateway/integration. Ports main-side additions to
the integration boundaries: the tier deletion diagnostics admin handlers move
to the owned admin request/response types, and the new lifecycle rule checks
in ecstore use the persisted lifecycle types instead of s3s DTOs. Keeps the
gateway git source in deny.toml while dropping the smithy-rs pin main retired,
and refreshes the test-count baseline for the merged tree.
Sync main 9601a7c4b into gateway/integration. Ports main-side additions to the
owned admin boundary (plugin delegation/attempt handlers, notification target
edit tests), drops a new ecstore s3s::dto reference in favour of the persisted
lifecycle filter, and refreshes the e2e selection digests and test-count
baseline for the merged tree.
main moved the local write path onto the disk blocking queue, which skips an
operation once its waiter is dropped, so a borrowed write cancelled while it is
queued no longer creates its file. The integration-only native publication
test still expected the queued open to run. Assert the new execution model
instead (the open never runs) while keeping its safety claims: the Started
record is taken before queueing and the cancelled publication never regrants
native history.
Source: main 4e48c390e7
Target: gateway/integration 1000860cb8
Conflicts resolved, keeping both sides:
- crates/ecstore/src/disk/{fs,local}.rs: main's batched rollback writes,
unified blocking write path and relocated open_file, with integration's
native publication records restored at each publication point and the
publication_history field kept beside main's multipart proof authority.
- crates/ecstore/src/set_disk/ops/multipart.rs: union of the imports.
- rustfs/src/storage/rpc/node_service.rs: main's MRF lifecycle reader
capability branch; the heal capability keeps integration's restore
copy-back probe.
- rustfs/src/admin/handlers/{audit,event,plugins_instances}.rs: main's
target_read inventory snapshot on integration's owned request boundary;
the new event.rs inventory test now builds owned requests.
- scripts/check_s3s_footprint.sh: integration's baseline-file version.
- .config/e2e-{full,smoke}-selection.txt: digests recomputed over the
union of both heads' CI listings (full 700 darwin / 708 linux, smoke 232).
Also migrated main's new rebalance delete-all expiry test to the persisted
lifecycle types integration uses in ecstore, and raised the test-count
baseline to the merged counts (every decrease comes from main).
`target_descriptor::build_json_response` returned the s3s response and
had no production caller left once the event, audit and plugin instance
handlers moved to `owned_build_json_response`; it survived only for the
legacy/owned equivalence test. Remove it, give the owned helper its own
documentation, and make the test assert the owned response directly:
status, `application/json`, the echoed `x-request-id` when present and
only then, and the body bytes. The file no longer imports s3s, so the
footprint baseline drops from 66 to 65 files.
* refactor(admin): migrate identity and cluster handlers to owned requests
Move the T1.12 batch 4 admin handlers (user, service_account, oidc, tier,
system, pools, quota, kms_dynamic, kms_backup, rebalance, module_switch,
bucket_meta, cluster_snapshot, realtime, bucket_recovery, profile) from the
s3s request/response types to the owned AdminRequest/AdminResponse
boundary. All 90 impl Operation blocks now take AdminRequest and return
AdminResponse; handler logic, authorization order, error codes and response
bytes are unchanged. The handlers use the shared owned helpers
(owned_json_response, owned_admin_json_response, authorize_profile_envelope,
the envelope-form runtime sources and audit constructors) instead of local
copies.
profile.rs drops the s3s forms of authorize_profile_request and
profile_not_implemented_response now that profile_admin.rs is on the
envelope gate too; profile_not_implemented_response answers with an
AdminResponse. The realtime metrics stream moves from the s3s ByteStream
adapter to Body::from_stream, and the rebalance idle-404 document still
renders through s3s::S3Error::to_http_response with its body converted at
the edge.
The idp_compat delegators ImportIamV2 and AddServiceAccountLdap move to the
owned trait as well: with their targets on both traits, the method-call
delegation became ambiguous between the owned impl and the router blanket
impl, so they now take the owned request directly.
KmsAdminAudit::from_request loses its last production caller with this batch
and is deleted; its comparison test now asserts the envelope facts directly.
Lower the s3s footprint baseline to the measured 68 files.
Part of rustfs/backlog#2788 (rustfs/backlog#2734 T1.12).
* refactor(admin): remove the legacy admin helpers batch 4 superseded
Batch 4 moved the last production callers of the s3s-typed admin helpers
to their owned forms, which left `admin_json_response`, `json_response`,
`authenticate_request`, `authorize_admin_request`, `object_store_from_req`
and `object_store_from_extensions` unused, along with the `Body`,
`S3Request` and `S3Response` re-exports of the admin `s3` facade. Remove
them. The owned forms keep their documentation; the behavior tests of the
removed `json_response` now exercise `owned_json_response`, and the
compat-prefix test asserts the owned response directly.
Two tests existed only to compare a removed legacy helper with its owned
twin (`owned_json_response_matches_the_legacy_shape`) or to cover the
removed legacy gate (`authorize_admin_request_without_credentials_is_rejected`);
they go with the helpers, and the owned equivalents stay. The rustfs
test-count row drops from 5957 to 5955 and the s3s footprint from 68 to 66
files.
* refactor(admin): complete the owned admin request boundary
Every type implementing the owned admin `Operation` now serves the legacy
router through one blanket impl that converts the s3s request and response
at that single edge, so handlers register with `AdminOperation(&Handler {})`
and need no adapter statics; `LegacyAdminAdapter` is removed.
`AdminRequest` gains the accessors handlers read today (`remote_addr`,
`client_info`, `request_context`, `req_info`, `req_info_mut`,
`install_req_info`, `envelope_mut`, `take_body`), the request helpers gain
envelope forms (`authenticate_envelope_credentials`,
`object_store_from_envelope`, the table-catalog `*_from_envelope` helpers,
`KmsAdminAudit::from_envelope`, `AccountAuditContext::from_envelope`,
`CallerIdentity::resolve_envelope`), `read_compatible_admin_body` accepts
either body type, and the owned `Body` gains `store_all_limited` and
`Default`. `admin::operation::test_support::TestRequest` builds owned
requests for handler tests. Unused accessors carry a per-method dead-code
expectation that each migrating handler batch removes.
No handler behavior changes; the remaining migrations follow in batches.
* chore(admin): add the T1.12 admin handler codemod
The script rewrites an admin handler file from the s3s request and response
types to the owned boundary (imports, signatures, request field and helper
calls, body access, response construction, and test request literals) and
leaves a TODO comment at every site it cannot rewrite mechanically. It is
idempotent and is removed once T1.12 completes.
* refactor(admin): migrate replication and KMS handlers to owned requests (#8951)
Move the T1.12 batch 2 admin handlers (site_replication, kms_keys, event,
replication, diagnostics, on_demand_migration, audit, account, kms_rekey,
kms_key_lifecycle, object_data_cache, tls_debug, integrity) from the s3s
request/response types to the owned AdminRequest/AdminResponse boundary.
Handler logic, authorization order, error codes and response bytes are
unchanged; the request/response plumbing now goes through the envelope
accessors and helpers introduced by the owned boundary.
The shared response helpers (utils::json_response, utils::empty_response,
handlers::admin_json_response, target_descriptor::build_json_response,
profile::authorize_profile_request) and the site replication repair
executor still answer with the s3s types, so the migrated files carry
owned-boundary forms of those helpers and convert the repair response at
its call edge. The streaming console-log body and the two devnull drains
move from the s3s byte stream to the owned Body stream/frame API.
Drop the dead_code expectations for the owned accessors and envelope
helpers this batch starts using, and lower the s3s footprint baseline to
the measured 99 files.
Part of rustfs/backlog#2788 (rustfs/backlog#2734 T1.12).
* refactor(admin): add owned forms of the shared admin response helpers
Handler batches were copying `json_response`, `empty_response`,
`admin_json_response`, and `build_json_response` into each migrated file
because the shared helpers still return the s3s response. Provide owned
forms next to each legacy helper (`owned_json_response`,
`owned_empty_response`, `owned_admin_json_response`,
`owned_build_json_response`, `authorize_profile_envelope`) with parity
tests against the legacy shape, derive `Debug` for `AdminResponse` so
tests can use `expect_err`, and fix three codemod rules: strip owned
string conversions in test credentials, annotate non-literal URI parses,
and leave a rebound request parameter immutable.
* refactor(admin): use the shared owned response helpers in batch 2 (#8954)
Replace the per-file owned-boundary copies of json_response,
empty_response, admin_json_response, build_json_response and
authorize_profile_request in the batch 2 admin handlers with the shared
owned_* helpers and authorize_profile_envelope that the boundary branch
now provides, and restore expect_err in the handler tests now that
AdminResponse implements Debug; the `.err().expect(..)` stand-ins were
failing the base branch's clippy step (clippy::err_expect under
-D warnings). The site replication repair call edge
keeps its conversion because the repair executor still answers with the
s3s response type.
Drop the dead_code expectations on the shared owned helpers this batch
now uses in non-test code.
Part of rustfs/backlog#2788 (rustfs/backlog#2734 T1.12).
* fix(admin): preserve legacy body limit error responses
* refactor(admin): move policy, ILM and batch 3 handlers to owned requests (#8956)
Batch 3 of T1.12 (rustfs/backlog#2788, parent rustfs/backlog#2734) moves
14 admin handler files to AdminRequest/AdminResponse and the owned
Operation trait: ilm_transition, policies, idp_compat, config_admin,
plugins_instances, profile_admin, group, heal, object_zip_download,
kms_key_metadata, durability, inspect_archive, sts and
gateway_key_inventory. Handler logic, authorization order, error codes
and response bytes are unchanged; test request literals use
test_support::TestRequest.
The handlers use the shared owned helpers (owned_json_response,
owned_admin_json_response, owned_build_json_response,
authorize_profile_envelope). The dead_code expectations on the
AdminRequest accessors and RequestEnvelope::uri_mut they now exercise
are removed. The legacy target_descriptor::build_json_response keeps
only its equivalence test caller and carries the matching expectation
until it retires.
object_zip_download builds the s3s request the S3 gate still reads from
the owned request facts; heal authenticates through
authenticate_envelope_credentials; the trace, inspect-archive and zip
bodies use Body::from_stream. extract_plugin_instance_filters takes the
URI so extensions.rs keeps calling it, and the two idp_compat
delegations stay on the legacy trait until their delegates migrate.
The s3s footprint baseline drops from 99 to 90 files.
* test(admin): anchor the replication diff body contract on the call
The source-contract test looked for `let body = read_compatible_admin_body(`
on one line; rustfmt wrapped that binding after the handler moved to the
owned request boundary, so the marker was no longer found. Anchor on the
call itself so formatting cannot change what the contract checks.
* refactor(admin): migrate table catalog and batch 1 to owned requests (#8955)
* fix(admin): preserve catalog body limit response bytes
Source: main 0f24511535
Target: gateway/integration 8cf6187ff4
Resolve the e2e-full selection digest conflict by recomputing both
platform digests over the merged nextest listing (685 darwin / 693
linux): main adds two rpc_fault_proxy unit tests, integration keeps its
dual-stack differential and anonymous-access additions. Raise the rustfs
test-count baseline row to the merged count (5932).
Brings in #8825, #8831, #8836 and #8839. Conflicts were import lists and test blocks (both sides kept), plus the new restore_copyback request flag next to main's data-movement field. Main replaced completed_multipart_object_part with prepare_complete_multipart_parts, so the stale import is dropped.
Main's xl.meta fast-path scanner (inspect_v2_object_slice) predates #8836 and classified a transitioned version with a pending v1 restore copyback as having no physical data dir, which made find_unshared_data_dir_for_version drop the copyback data dir. The scanner now reads the copyback marker with get_consistent_bytes semantics (both aliases, ASCII case-insensitive, empty or conflicting values count as absent), and uses_data_dir keeps main's allocation-free transition check plus #8836's copyback exclusion. find_unshared_data_dir_for_version_keeps_pending_copyback_data_dir failed on the plain merge and passes now.
Picks up main through 44a73b0be. Conflicts: rustfs/src/error.rs keeps the rustfs-s3-types ApiError conversion and drops the helper main's #8816 replaced; scripts/check_s3s_footprint.sh keeps the integration script, whose baseline lives in .config (main only retuned a constant the integration script does not have). Main's new tests use the rustfs-s3-types error code in admin replication and reach s3s::BodySizeLimitExceeded through the app's test-only s3 facade, so the s3s file count stays at 118; s3_error_lines tightens to 302. The test-count baseline is regenerated; the e2e selections are unchanged.
Brings in #8812 and #8818 from gateway/integration. #8812 predates main's heal error classifier (#8778); its two retryable variants (TooManyOpenFiles, OutdatedXLMeta) move into the classifier's transient arms for storage and disk errors, and its test is kept.
Since main's region-aware CreateBucket and GetBucketLocation (#8739, #8743), us-east-1 answers an empty LocationConstraint and 200 for an owned re-create, so the control no longer reported its own region. Also refresh the e2e-full linux selection from the CI listing (682 tests).
Picks up the 20 main commits since 62b6f5f3f. The one conflict is the tier free-version recovery test setup: it keeps the integration-only fixture isolation (background recovery disabled while the fixture starts) and main's mock tier registration (#8804 skips scans with no tier). The e2e selections are unchanged by these commits; the test-count baseline is regenerated.
The scheduled sync has stopped on conflicts since 2026-10-19. Resolved by keeping both sides: main's restructured delete-rollback path in disk/local.rs with the integration-only native-publication record, main's per-instance region with the rustfs-s3-types Region, main's new lifecycle, rebalance and admin code ported to the persisted lifecycle types and rustfs-s3-types errors, both sides' new scanner tests, and the union of the access_consumer imports. The e2e smoke, full (darwin) and repl-nightly selections and the test-count baseline are regenerated from the merged tree.
* refactor(s3select): run queries on an engine request built from gateway DTO shapes
Part of T1.5 (rustfs/backlog#2746), batch 2.
* fix(s3select): keep the select request conversion in the app layer
The layer guard forbids app -> storage; the conversion moves from storage/s3_compat/select.rs to app/select_request.rs unchanged.
* refactor(ecstore): move bucket configs onto gateway persisted types
Finish the ECStore batch of T1.4 (rustfs/backlog#2745). The remaining
eleven bucket configuration families (Website, CORS, Tagging, Versioning,
ObjectLock, Encryption, Notification, Logging, Accelerate, RequestPayment,
PublicAccessBlock) now read and write their stored XML through the
rustfs-gateway-types persistence and cors_tagging codecs instead of the
s3s codec, so the legacy s3s bucket-config codec and its
s3gate-metadata-xml compat marker are removed.
- ecstore: BucketConfigXml is implemented by the gateway codec for every
family; metadata, metadata_sys, versioning, object lock, replication and
lifecycle consumers take the persisted types. Region moves to
rustfs_s3_types::Region, and the ETag conversion to s3s moves to the
rustfs edge with its tests. Only warm_backend.rs and warm_backend_s3.rs
still reference s3s, held for T1.5 (rustfs/backlog#2746).
- scanner: object lock cache, item actions and data usage use the
persisted types.
- rustfs: the legacy S3 edge converts each family between s3s DTOs and the
persisted types in storage/storage_api.rs by writing one codec's XML and
reading it with the other; Tagging converts field by field.
- Bump the rustfs-gateway-types pin to 21283e87.
The 39 persisted-byte goldens are unchanged and pass, so a document the
old codec wrote reads back and re-encodes to the same bytes. s3s footprint:
files 181 -> 146, ecstore_files 28 -> 2, s3_error_lines unchanged.
* test(app): prepare the PutBucketEncryption config from the request
The uninitialized-store test passed an empty persisted configuration, which the use case now validates before it reaches the store check, so it answered MalformedXML instead of InternalError. Convert the request's configuration the way the S3 edge does.
* style(app): format the PutBucketEncryption test
* refactor(protocols): move error types to rustfs-s3-types by codemod
T1.2 batch 3 (rustfs/backlog#2743): s3select-api, s3select-query and
the error items of protocols move off s3s by scripts/codemods, run with
--hold crates/protocols/src/sftp/errors.rs. That file stays on s3s: it
downcasts the backend error, whose runtime type is still s3s::S3Error.
- rustfs/src/protocols/client.rs: the two webdav session methods now
return rustfs_s3_types::S3Result and convert the s3s error with From.
- codemod: a child module that calls s3_error! through `use super::*`
gets its own `use rustfs_s3_types::s3_error;` once its parent imports
that macro by name; otherwise it is listed as GLOB.
- s3-types: a table test pins code, status and the s3s error the legacy
edge renders for every variant a moved call site names.
- s3_error_lines 1585 -> 1584 (a lifecycle comment reworded).
* refactor(admin): move error types to rustfs-s3-types (WIP, no build)
T1.2 batch 4 (rustfs/backlog#2743), work in progress: rustfs check
still fails with 16 errors (listed in the PR body).
- codemod: --facade PATH treats a crate-local re-export of s3s's error
items like s3s for the four names (9 new self-tests, 9 mutants red).
- codemod output for rustfs/src/admin and rustfs/src/site_replication
(--facade crate::storage_api::site_replication::s3); 6 glob children
get an explicit rustfs_s3_types::s3_error import.
- admin/router.rs: the Operation trait returns rustfs_s3_types::S3Result.
- error.rs: From<ApiError> builds the rustfs-s3-types error; the s3s
conversion is derived from it.
* fix(admin): convert error types at the s3s boundary
Batch 4 of T1.2 left the admin and site_replication modules returning
rustfs_s3_types::S3Error while their callers and callees on the storage
and auth side still speak s3s::S3Error. Convert at each edge with the
existing From impls in rustfs-s3-types (compat_s3s.rs) instead of
changing any code, message or status:
- admin/auth.rs, admin/handlers/object_zip_download.rs,
admin/handlers/table_catalog/mod.rs: map the s3s result returned by
check_key_valid / authorize_request / authorize_internal_object_request
into the admin error type with .map_err(Into::into).
- storage/ecfs.rs: the s3s-facing side maps the admin site_replication
gate results the other way.
- admin/handlers/pools.rs: ApiError.code stays s3s::S3ErrorCode, so the
rustfs code is converted with .into() when the struct is built, in
production and in the two tests that build or compare it.
- admin/handlers/heal.rs, admin/runtime_sources.rs: the router returns
s3s errors (S3Route impl); the expected rustfs code is converted with
.into() before the comparison.
- admin/handlers/table_catalog/mod.rs: iceberg_rest_error takes
&'static str so the Cow<'static, str> custom code borrows it; every
caller already passes an ICEBERG_ERROR_* constant or a &'static str.
- admin/handlers/quota.rs: drop a now same-type .into() after s3_error!.
- storage_api.rs, site_replication/mod.rs: drop re-exports and an import
that no longer have users (clippy -D warnings).
cargo check -p rustfs --all-targets and cargo clippy -p rustfs
--all-targets -- -D warnings are clean; the s3s footprint baseline is
unchanged at files=172, s3_error_lines=303, ecstore_files=28.
Refs rustfs/backlog#2743, rustfs/backlog#2734.
Keep lifecycle rules, restore requests, Object Lock, and scanner cached tags in domain types. Convert at the existing HTTP boundary while preserving legacy XML, scanner cache bytes, date ordering, and tombstones.
Linux test validation before the final import-path-only adjustment:
cargo nextest run -p rustfs-ecstore bucket_config_goldens
39 passed; 6211 skipped by selector.
cargo nextest run --lib -p rustfs -E 'test(/test_zero_rule_lifecycle_tombstone_recognition/) | test(/test_local_lifecycle_staleness_axis_selection/) | test(/test_lifecycle_staleness_preserves_legacy_submillisecond_axis/) | test(/test_merge_incoming_lifecycle/) | test(/test_lifecycle_expiry_statement_matrix/) | test(/test_lifecycle_expiry_subset_xml/) | test(/put_bucket_lifecycle_validation_errors_keep_their_s3_code/) | test(/execute_put_bucket_lifecycle_configuration_rejects_missing_configuration/) | test(/execute_restore_object_rejects_missing_restore_request/) | test(/execute_restore_object_rejects_select_type/) | test(/execute_restore_object_maps_failures_to_typed_s3_errors/) | test(/execute_restore_object_returns_internal_error_when_store_uninitialized/) | test(/restore_request_conversion_preserves_legacy_output_values/) | test(/restore_request_conversion_preserves_presence_and_raw_enums/) | test(/restore_request_conversion_distinguishes_absent_and_empty_containers/)'
25 passed; 5318 skipped by selector.
cargo nextest run --profile ci -p rustfs-ecstore -p rustfs-lifecycle -p rustfs-replication -p rustfs-scanner
7872 passed; 21 skipped under the existing CI profile.
Previously verified mutation controls: 39 production faults, 41 RED/GREEN pairs.
cargo clippy --locked -p rustfs -p rustfs-ecstore -p rustfs-lifecycle -p rustfs-scanner --all-targets -- -D warnings
Passed in 40m 45s before the import-path adjustment, then in 1m 59s on
the final source with warnings denied. The final adjustment imports the
existing helper through the object domain facade; fmt and the architecture
check also passed.
Part of rustfs/backlog#2745. The remaining ecstore configuration families follow separately.
Keep the migrated compatibility adapters in their active modules while bringing in upstream fixes. Preserve native publication history on the new create, rename and borrowed-open paths.
Validation:
- `cargo nextest run --locked -p rustfs-ecstore --lib -E 'test(disk::os::native_creation_history_tests::) | test(disk::local::test::borrowed_open_cancellation_retains_started_history_before_execution) | test(disk::local::test::owned_direct_child_open_attempt_consumes_native_history)' --no-fail-fast`: 9/9 passed after restoring all 12 mutation cases at cea78f2590645eb0d303cd7cf56f4dd0d9c00e87; these source blobs are unchanged in this merge.
- `cargo nextest run --locked -p rustfs-s3select-query --lib -E 'test(dispatcher::json_document_schema::tests::)' --no-fail-fast`: 5/5 passed at 5421ba682d1329cc7ebeaac7c6a863da2cea1996, with the same tree.
- `cargo fmt --all --check`: passed.
- `./scripts/check_test_count_ratchet.sh --update`: 52 crates, 22007 tests, 165 ignored.
- Profile baselines use seven actual Darwin listings and seven actual Linux listings.
- Earlier conflict snapshot f5e0cdf0c58aed1ebe53b9243cf52f94832ab049: `cargo nextest run --locked -p rustfs-ecstore -p rustfs --lib -E 'test(disk::os::) | test(bucket::metadata_sys::) | test(server::legacy_compat::) | test(server::layer::tests::request_logging_watchdog) | test(server::http::tests::conditional_object_http::) | test(storage::helper::tests::operation_helper)' --no-fail-fast` passed 218/218 before the borrowed-open fix. This is historical evidence.
* wip: T1.4 replication family
* refactor(replication): move the replication config onto gateway types
The replication configuration family leaves s3s: rustfs-replication no longer depends on s3s, and ECStore bucket metadata, the scanner and the rustfs app/admin layers hold rustfs_gateway_types::persistence::PersistedReplicationConfiguration. The persisted XML goes through the gateway persistence codec behind a new BucketConfigXml trait (the 39 byte goldens pass with unchanged fixtures); the legacy s3s edge converts once in storage/ecfs.rs through storage/s3_compat/replication.rs; the scanner data usage cache keeps its MessagePack bytes through a serde mirror pinned by bytes recorded from s3s 5761ddfe. rustfs-gateway-types 0.32.0 is pinned once (rev 772d3385). Refs rustfs/backlog#2745.
* test: use S3 serializer for list response tests
Merge frozen main acd26afd8b and include
compatibility fixes and measured Darwin/Linux membership corrections
across six files.
Validation on candidate b4c3ab560a:
- All 20 local gates passed.
- Native smoke: 218 passed. Native full: 661 passed.
- Full passed on the same-head failed-only rerun; the initial failure
remains recorded.
- Gateway remains report-only: 38 passed, 180 failed, under the existing
exemption.
CI: https://github.com/rustfs/rustfs/actions/runs/38285788334
Run the three rollback metadata-cleanup cases in separate test processes so reused /proc/self/fd aliases from earlier cases cannot invalidate their creation witnesses. Keep the original fixture, guards, assertions and io_uring test selection, and verify that each child runs the intended case successfully.
Update the generated test-count baseline for the added macro arm.
Validation: cargo check -p rustfs-ecstore --tests; cargo fmt --all --check; all 19 quick-check commands, including make script-tests and the two Python checks. Linux io_uring acceptance and fault controls remain pending.
Keep the guarded rollback future behind a heap allocation so bucket creation does not store its nested state inline. The rollback body, all three fence references and pending-proof revocation order are unchanged.
The same three original Linux tests overflowed on both e611 CI attempts. Local format, ecstore test compilation and all 19 quick-check steps pass; the original Linux feature lanes still need to validate this candidate.
* test(rpc): pin the wire bytes of every internode rpc response body
rustfs/backlog#2747 batch 4, step 1 of 2, on the s3s bodies the rpc paths
serve today. Each response body builder in storage/rpc/http_service.rs is
served through hyper's HTTP/1.1 server and the raw bytes the peer reads are
compared with goldens captured from this commit: status, content-length or
chunked framing, chunk boundaries, the terminating chunk, HEAD, and the cut
connection a failure after data produces. The capability responses carry a
per-process proof, so their length header and payload are checked against
the body they encode instead.
The read_file_stream response builder moves into read_file_stream_response
so the test reaches the same code the handler runs; behaviour is unchanged.
* refactor(rpc): serve internode rpc bodies with the owned Body
rustfs/backlog#2747 batch 4, step 2 of 2. The internode rpc HTTP paths
build their responses with rustfs_s3_types::Body: Body::from_stream
replaces Body::from(StreamingBlob::wrap(..)) at the six stream sites, and
the in-memory and empty bodies keep their constructors. InternodeRpcService
converts its responses into the wrapped service's response body through
From<Body>, so the server keeps composing it with the s3s service as
before. The wire-byte goldens from the previous commit pass unchanged;
storage/rpc no longer names s3s.
* fix(rpc): simplify wire test response builder type
Name the boxed response builder used by the capability wire test so
all-target Clippy can check it without a type-complexity warning.
Validation (LANG=en_US.UTF-8, CARGO_INCREMENTAL=0):
- cargo clippy -p rustfs --all-targets -- -D warnings: exit 0.
- cargo nextest run --profile ci -p rustfs --lib -E 'test(/storage::rpc::/)': 279 tests run, 279 passed, 4842 skipped.
- All 19 quick-check commands, including script-tests, both Python scripts and cargo fmt --all --check: exit 0.
- Footprint baseline regenerated: files=194, s3_error_lines=1584, ecstore_files=36; test-count baseline regenerated: 52 crates, 21228 tests, 164 ignored.
- All 54 Cargo.toml files parse without duplicate keys.
No assertion was added or changed by this lint fix. R1-R8 are retained
in the existing PR verification record.
* fix(ecstore): resume fenced Object Lock creation intents
* fix(ecstore): bound creation retry futures on the stack
* fix(ecstore): require native proof for creation retries
The fixture constructor can take the production recovery lock and wait for 60 seconds before this test installs its queue and seeds metadata. Disable its background recovery only while creating this fixture, then restore the environment before calling the existing production recovery entrypoint.
Only the existing test setup changes. Its production entrypoint, default lock parameters, original assertions, 30-second timeout, cancellation and cleanup remain intact. No tests or assertions were added. Production and store/init.rs are byte-identical to the parent.
Independent acceptance ran on merge tree 8ec358cb42ba9ebb3bcfeafcef048a3f0366355b; the runner header reports parent HEAD, while before/after index and source hashes confirm this candidate. Worker task branch d8055766b8 has the same tree.
Every command used LANG=en_US.UTF-8. Task compilation also used CARGO_BUILD_JOBS=1, CARGO_INCREMENTAL=0, CARGO_NET_OFFLINE=true, an independent target, and empty RUSTC_WRAPPER/RUSTC_WORKSPACE_WRAPPER to keep compilers in its owned process group without changing repository configuration.
Command:
shellcheck --version && actionlint
Exit 0; observed output excerpt:
version: 0.11.0
Log SHA-256: fc9eec16b4191d058a295813ca9afbe9d2b37b59975da860170726be162a3924
Command:
cargo fmt --all --check
Exit 0; observed output excerpt:
(no stdout or stderr)
Log SHA-256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
Command:
./scripts/check_unsafe_code_allowances.sh
Exit 0; observed output excerpt:
(no stdout or stderr)
Log SHA-256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
Command:
./scripts/check_layer_dependencies.sh
Exit 0; observed output excerpt:
Layer dependency guard passed (no new reverse dependencies/cycles).
Log SHA-256: 69c6cb1896c0586b28d1167380d3260d99d8017a4265c8df5a2348f289e93a12
Command:
./scripts/check_architecture_migration_rules.sh
Exit 0; observed output excerpt:
Architecture migration rules passed.
Log SHA-256: 28790e70d070607ed4de0704f292a65030be92ec224ef106d4b72fd8296f896e
Command:
./scripts/check_logging_guardrails.sh
Exit 0; observed output excerpt:
✅ Logging guardrails check passed
Log SHA-256: 7adf41abff8ed8f7617decf86498888ddde78937099dd9ce47f6d359dfc2d317
Command:
./scripts/check_error_other_format_ratchet.sh
Exit 0; observed output excerpt:
error(format!) ratchet passed (total call sites: 579).
Log SHA-256: 66172aef18f150ba0605af122e26ffcf91f6dd86f43e38b05e2e982ce8d783ee
Command:
./scripts/check_no_tokio_io_uring.sh
Exit 0; observed output excerpt:
(no stdout or stderr)
Log SHA-256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
Command:
./scripts/check_extension_schema_boundaries.sh
Exit 0; observed output excerpt:
Extension schema boundaries passed.
Log SHA-256: f13d0acd0e2ce5d9815241b12b4dab7d01cc11b9b65bcf6dfb7812c878532a51
Command:
./scripts/check_body_cache_whitelist.sh
Exit 0; observed output excerpt:
Body-cache whitelist guard passed.
Log SHA-256: 6b828847b0383950ee6eef9cd846d0c322c945e47efd4c7d1ad881178810d6bb
Command:
./scripts/check_s3s_footprint.sh
Exit 0; observed output excerpt:
s3s footprint OK: files importing s3s is 195 (baseline files=195)
s3s footprint OK: s3s s3_error! lines is 1584 (baseline s3_error_lines=1584)
s3s footprint OK: ecstore files referencing s3s is 36 (baseline ecstore_files=36)
✅ s3s footprint baseline check passed
Log SHA-256: 47f53d04ef645618563fe1f22abc64667f19078f1974feedcca46e4f2c016844
Command:
./scripts/check_test_count_ratchet.sh
Exit 0; observed output excerpt:
test-count ratchet OK: 52 crates, 21340 tests, 164 ignored match .config/test-count-baseline.txt
Log SHA-256: 618a5e330e933de0c6ff484546df401457024a88bb5986d11d588794e33dab60
Command:
./scripts/check_fips_wording.sh
Exit 0; observed output excerpt:
FIPS wording guard passed (README.md and CHANGELOG.md contain no forbidden claims).
KMS wording guard passed (crates/kms claims no Vault KV2 Transit wrapping).
Log SHA-256: 99221cc9979c480ec4f46f50d76c52b18c72fe499f12e15e4f4a22a1b8bfb140
Command:
./scripts/check_embedded_secrets.sh
Exit 0; observed output excerpt:
Embedded secret guard passed (no private key material or provider credential literal in the tree).
Log SHA-256: 437a3ceba32094d2901889965ad191e411aad32cdb18bf0b8659e818011c406b
Command:
make script-tests
Exit 0; observed output excerpt:
test_cli_requires_the_repository_default_branch (__main__.SelfTests.test_cli_requires_the_repository_default_branch) ... ok
test_config_rejects_duplicate_and_invalid_entries (__main__.SelfTests.test_config_rejects_duplicate_and_invalid_entries) ... ok
test_freshness_boundaries (__main__.SelfTests.test_freshness_boundaries) ... ok
test_grace_requires_two_successful_queries_with_no_history (__main__.SelfTests.test_grace_requires_two_successful_queries_with_no_history) ... ok
test_invalid_api_evidence_fails_closed (__main__.SelfTests.test_invalid_api_evidence_fails_closed) ... ok
test_report_retains_every_workflow (__main__.SelfTests.test_report_retains_every_workflow) ... ok
test_requests_filter_schedule_default_branch_and_success_on_server (__main__.SelfTests.test_requests_filter_schedule_default_branch_and_success_on_server) ... ok
Ran 10 tests in 0.042s
OK
Ran 30 tests in 20.627s
OK
Ran 10 tests in 2.268s
OK
Ran 33 tests in 0.127s
OK
Ran 7 tests in 0.002s
OK
Ran 7 tests in 0.001s
OK
Ran 4 tests in 0.108s
OK
Ran 3 tests in 0.564s
OK
PASS: cold stampede script self-test
Log SHA-256: 75fda2d069a373c63667b908baaf99e6591aab9099a739b0c4c22ceb6da8d68c
Command:
python3 ./scripts/test_object_generation_protocol_model.py
Exit 0; observed output excerpt:
test_an_unavailable_decision_quorum_cannot_choose_a_candidate (__main__.ObjectGenerationProtocolModel.test_an_unavailable_decision_quorum_cannot_choose_a_candidate) ... ok
test_counter_allocation_requires_persistence_and_is_consumed_once (__main__.ObjectGenerationProtocolModel.test_counter_allocation_requires_persistence_and_is_consumed_once) ... ok
test_counter_restart_burns_persisted_allocations_before_or_after_issue (__main__.ObjectGenerationProtocolModel.test_counter_restart_burns_persisted_allocations_before_or_after_issue) ... ok
test_crash_after_promise_before_reply_does_not_restore_an_older_ballot (__main__.ObjectGenerationProtocolModel.test_crash_after_promise_before_reply_does_not_restore_an_older_ballot) ... ok
test_duplicate_or_unconfigured_voters_cannot_form_a_quorum (__main__.ObjectGenerationProtocolModel.test_duplicate_or_unconfigured_voters_cannot_form_a_quorum) ... ok
test_empty_or_unobserved_minority_slot_can_retry_without_allocating_another_revision (__main__.ObjectGenerationProtocolModel.test_empty_or_unobserved_minority_slot_can_retry_without_allocating_another_revision) ... ok
test_equal_ballot_cannot_accept_different_bytes_and_lower_ballot_is_rejected (__main__.ObjectGenerationProtocolModel.test_equal_ballot_cannot_accept_different_bytes_and_lower_ballot_is_rejected) ... ok
test_forgetting_tombstone_authority_allows_an_aba_create (__main__.ObjectGenerationProtocolModel.test_forgetting_tombstone_authority_allows_an_aba_create) ... ok
test_issuing_before_persistence_reuses_a_ballot_after_restart (__main__.ObjectGenerationProtocolModel.test_issuing_before_persistence_reuses_a_ballot_after_restart) ... ok
test_lost_ack_and_restart_preserve_the_chosen_operation (__main__.ObjectGenerationProtocolModel.test_lost_ack_and_restart_preserve_the_chosen_operation) ... ok
test_minority_accepted_work_is_adopted_and_finished (__main__.ObjectGenerationProtocolModel.test_minority_accepted_work_is_adopted_and_finished) ... ok
test_omitting_highest_accepted_adoption_has_a_double_choice_counterexample (__main__.ObjectGenerationProtocolModel.test_omitting_highest_accepted_adoption_has_a_double_choice_counterexample) ... ok
test_quorum_choice_does_not_require_an_uncontacted_disk_to_learn_it (__main__.ObjectGenerationProtocolModel.test_quorum_choice_does_not_require_an_uncontacted_disk_to_learn_it) ... ok
test_recovery_failure_or_stale_expected_head_does_not_allocate_a_successor (__main__.ObjectGenerationProtocolModel.test_recovery_failure_or_stale_expected_head_does_not_allocate_a_successor) ... ok
test_replacing_voters_with_empty_state_can_choose_a_conflicting_value (__main__.ObjectGenerationProtocolModel.test_replacing_voters_with_empty_state_can_choose_a_conflicting_value) ... ok
test_skipping_predecessor_recovery_chooses_a_broken_revision_chain (__main__.ObjectGenerationProtocolModel.test_skipping_predecessor_recovery_chooses_a_broken_revision_chain) ... ok
test_stale_publication_rollback_and_gc_cannot_change_the_winner (__main__.ObjectGenerationProtocolModel.test_stale_publication_rollback_and_gc_cannot_change_the_winner) ... ok
test_superseded_operation_identity_cannot_be_reused_for_different_input (__main__.ObjectGenerationProtocolModel.test_superseded_operation_identity_cannot_be_reused_for_different_input) ... ok
test_superseded_operation_retry_returns_its_original_successor (__main__.ObjectGenerationProtocolModel.test_superseded_operation_retry_returns_its_original_successor) ... ok
test_tombstone_and_null_marker_payloads_do_not_reset_authority_revision (__main__.ObjectGenerationProtocolModel.test_tombstone_and_null_marker_payloads_do_not_reset_authority_revision) ... ok
test_tombstone_rejects_a_delayed_never_present_create (__main__.ObjectGenerationProtocolModel.test_tombstone_rejects_a_delayed_never_present_create) ... ok
test_unlearned_predecessor_is_recovered_before_the_second_slot (__main__.ObjectGenerationProtocolModel.test_unlearned_predecessor_is_recovered_before_the_second_slot) ... ok
Ran 24 tests in 0.929s
OK
Log SHA-256: 0b4be926efb0609d497ae58930340153f01957b606206cbbdb3e0d47eb29d3a2
Command:
python3 ./scripts/check_test_wiring.py
Exit 0; observed output excerpt:
OK: e2e modules, runner selection, fuzz matrices, profiles, scheduled alerts, and Quick Checks are wired
Log SHA-256: 21f676f458451525535ab0ef73d6a9c0d9774e16d76a4d19e396ad34936dd39c
Command:
./scripts/check_no_planning_docs.sh
Exit 0; observed output excerpt:
(no stdout or stderr)
Log SHA-256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
Command:
./scripts/check_uring_lane_lib_only.sh
Exit 0; observed output excerpt:
OK: no 'uring_'-matching test functions under crates/ecstore/tests; --lib is safe
Log SHA-256: 43f3f851e27ec2ab9e34d6b05fefdede38af992699c473904bfe6f53a46d52e3
Command:
cargo nextest run --profile ci -p rustfs-ecstore --lib --features rio-v2 -E 'test(=bucket::lifecycle::bucket_lifecycle_ops::tests::tier_free_version_recovery_production_entrypoint_enqueues_seeded_item)' --no-fail-fast
Exit 0; wall 234.399s.
Observed output:
Finished `test` profile [unoptimized + debuginfo] target(s) in 3m 49s
Starting 1 test across 1 binary (5871 tests skipped)
SETUP PASS [ 0.006s] ecstore-base-stack: sh -c 'echo RUST_MIN_STACK=4194304 >> "$NEXTEST_ENV"'
PASS [ 0.440s] rustfs-ecstore bucket::lifecycle::bucket_lifecycle_ops::tests::tier_free_version_recovery_production_entrypoint_enqueues_seeded_item
Summary [ 0.586s] 1 test run: 1 passed, 5871 skipped
Log SHA-256: 34e973f86e8dc2102c1a0e483e0ac802ce7df3d1b40a603932f62c0805c4de27
Command:
cargo nextest run --profile ci -p rustfs-ecstore --lib --features rio-v2 -E 'test(=bucket::lifecycle::bucket_lifecycle_ops::tests::tier_free_version_recovery_eight_coordinators_hold_one_leader_and_take_over) | test(=store::bucket::tests::creation_commit_backfill_runs_under_the_config_write_fence) | test(=store::bucket::tests::orphaned_bucket_recovery_requires_every_volume_absent_and_is_retryable)' --no-fail-fast
Exit 0; wall 4.010s.
Observed output:
Finished `test` profile [unoptimized + debuginfo] target(s) in 0.43s
Starting 3 tests across 1 binary (5869 tests skipped)
SETUP PASS [ 0.005s] ecstore-base-stack: sh -c 'echo RUST_MIN_STACK=4194304 >> "$NEXTEST_ENV"'
PASS [ 0.068s] rustfs-ecstore bucket::lifecycle::bucket_lifecycle_ops::tests::tier_free_version_recovery_eight_coordinators_hold_one_leader_and_take_over
PASS [ 0.637s] rustfs-ecstore store::bucket::tests::creation_commit_backfill_runs_under_the_config_write_fence
PASS [ 0.846s] rustfs-ecstore store::bucket::tests::orphaned_bucket_recovery_requires_every_volume_absent_and_is_retryable
Summary [ 0.987s] 3 tests run: 3 passed, 5869 skipped
Log SHA-256: b5da2a927ff18ca9aa217762f7186a6d4371bd9e8feedd7b77940e10b64dffac
Command:
cargo clippy -p rustfs -p rustfs-ecstore --all-targets --features rio-v2 -- -D warnings
Exit 0; wall 1084.508s.
Observed output:
Finished `dev` profile [unoptimized + debuginfo] target(s) in 18m 03s
Log SHA-256: 28f98be7970faaa71cd118da1da4e6e2e3d23b9dda7e6449eaa7a37e741260b5
Compiled mutation command (unchanged for each red and restored run):
cargo nextest run --profile ci -p rustfs-ecstore --lib --features rio-v2 -E 'test(=bucket::lifecycle::bucket_lifecycle_ops::tests::tier_free_version_recovery_production_entrypoint_enqueues_seeded_item)' --no-fail-fast
M1: reverse only this test-local constructor flag from false to true. The mutant compiled, then failed the original recovered-item queue assertion after 30.481s. Restoring the entire observed candidate passed in 0.460s. Red log 55bf3b1535529ac0bcfb8071f9b56f9f40631007a85c315367e7e93786d65293; restored log 204d5fbe4574a7917f5e740a795d864fe28a7e7fb5e33d27b0a9cf68e2f17f52.
M2: replace only the actual awaited enqueue with false, consuming its original inputs. The mutant compiled, then failed the same original queue assertion after 30.516s. Restoring the entire observed candidate passed in 0.457s. Red log 89186330413215e231e2b06bf013cc31097ea8f4a106d91421a36d6c8986a7fd; restored log e848b42d2302401afa5dbcd9e37bf76d2e56a96ab6de5a488513b997f53c5e9b.
All temporary observation and mutation code was removed before final acceptance. Historical baseline and diagnostic failures remain recorded; they are not mutation credit. The separate Object Lock intent-retry diagnostic still fails, and these native macOS checks do not establish Linux CI recovery.
Pass the capture-free pending future factory as a function item in the existing slow-operation test. This fixes the redundant_closure error at disk_store.rs:2914 that stopped strict Clippy before the feature tests could run.
The single-line test edit preserves the factory's result, warning and timeout behavior. Reversing the replacement reproduces the parent file byte for byte. All 220 assertion lines and the production prefix are unchanged; no tests, assertions, guards, profiles or baselines were added or relaxed.
Original Linux CI evidence (run 37888552268 at 53245be9f):
- RIO job113685483992: strict Clippy exit101, redundant closure at disk_store.rs:2914; feature tests skipped.
- Workspace Test and Lint job113685484058: same strict Clippy exit101; nextest and doc tests skipped.
These are lint failures, not runtime mutation kills or known flakes.
Native macOS arm64 validation on tree7a5bf47e7533b29479bc85207416102f1337c566, LANG=en_US.UTF-8, one build job, CARGO_INCREMENTAL=0 and offline dependencies:
shellcheck --version && actionlint -> exit 0
cargo fmt --all --check -> exit 0
./scripts/check_unsafe_code_allowances.sh -> exit 0
./scripts/check_layer_dependencies.sh -> exit 0
./scripts/check_architecture_migration_rules.sh -> exit 0
./scripts/check_logging_guardrails.sh -> exit 0
./scripts/check_error_other_format_ratchet.sh -> exit 0
./scripts/check_no_tokio_io_uring.sh -> exit 0
./scripts/check_extension_schema_boundaries.sh -> exit 0
./scripts/check_body_cache_whitelist.sh -> exit 0
./scripts/check_s3s_footprint.sh -> exit 0
./scripts/check_test_count_ratchet.sh -> exit 0
./scripts/check_fips_wording.sh -> exit 0
./scripts/check_embedded_secrets.sh -> exit 0
make script-tests -> exit 0
python3 ./scripts/test_object_generation_protocol_model.py -> exit 0
python3 ./scripts/check_test_wiring.py -> exit 0
./scripts/check_no_planning_docs.sh -> exit 0
./scripts/check_uring_lane_lib_only.sh -> exit 0
All 19 checks: exit0, 312.161s.
$ cargo clippy -p rustfs -p rustfs-ecstore --all-targets --features rio-v2 -- -D warnings
Finished `dev` profile [unoptimized + debuginfo] target(s) in 20m 11s
exit0, 1213.210s total.
$ cargo nextest run --profile ci -p rustfs-ecstore --features rio-v2 slow_disk_operation_warns_once_while_pending_and_preserves_result --no-fail-fast
Finished `test` profile [unoptimized + debuginfo] target(s) in 4m 34s
PASS [0.026s] rustfs-ecstore disk::disk_store::tests::slow_disk_operation_warns_once_while_pending_and_preserves_result
Summary [ 0.170s] 1 test run: 1 passed, 5943 skipped
exit0, 281.920s total; 5943 existing filter skips.
warning: linker stderr: ld: __eh_frame section too large (max 16MB) to encode dwarf unwind offsets in compact unwind table, performance of exception handling might be affected
This linker warning is retained; flags, profiles and allowances are unchanged.
54 tracked Cargo.toml files parsed without duplicate keys. git diff --check passed. No new assertions were introduced, so no new runtime mutation is claimed. Native compilation and the existing helper test do not substitute for actual Linux RIO execution; the new integration CI run still needs to recover before the RPC merge.
Preservation and output hashes:
parent file SHA256: 8e59ee745e3aaaa22848b943d3b10f6353b515956282cde1f3af2c394200c9ac
final file SHA256: b69ba3237061b161448ae8b62b615885d5770b98a5b4a2479bcfbd70f1bd1f17
unchanged production prefix SHA256: 7903a6e288252c420231242d7b58a25fa2a7140ea3dd430cbe7f0dde11369dcb
RIO Linux failure log SHA256:421addb9868fd88e4109a922311e54f91f0dfd70862f8ba7fcaba193482ed90b
Workspace Linux failure log SHA256:e0aea8d0f014c022252a9534968e083c98cfc1fd760da445ecd04ec3f522e26a
19 output SHA256:35a54f28d707956ed1edb29d492c4dfe8a9fe753d103b111b5a2f1b5c41c2b98
RIO Clippy output SHA256:d6046e3868f2b3992fe11cdd04e9e98eb50a87d1234d9a70dbfeb154ff61638d
slow-operation output SHA256:605b7b258d2fd09c6ef2ab2ffc90d433d0509480f1e45125d20c823759e83608
Refs rustfs/backlog#2734.
Wait for the RPC/locks test sockets to pass the existing owner and mode
checks inside the original 5s readiness window and 10ms poll interval.
Use the existing admin S3 carrier for the new recovery and quota error
calls. Regenerate the test-count baseline for main's 118 existing tests
across seven crates. Pair the private pagination helper's raw-scan inputs
and remove the redundant Copy clone from the retirement event.
Keep the original assertions, responses, authorization, locks and storage
operations. The pagination body and raw-scan expressions at all five calls
are byte-identical; reinserting only the Copy clone restores the entire
base heal file. All ignored baseline values and s3s counts stay unchanged.
Validation base: c47432e353
Validation tree: f081084bc5
Patch SHA-256: 2dcf0028691bfb2772edfb38e262829b04f4151832dcce823c9dd9f7434b52de
Trace production-prefix SHA-256: fd994f218a21429d62866f4efb8eee1ee57b58f4a51865aa35d05a54ef1ccb45
All commands used LANG=en_US.UTF-8, CARGO_BUILD_JOBS=1,
CARGO_INCREMENTAL=0 and CARGO_NET_OFFLINE=true in one exclusive target.
Every gate retained the same tree and six source hashes before and after.
54 Cargo.toml files parse without duplicate keys.
cargo clippy -p rustfs --all-targets -- -D warnings
Finished `dev` profile [unoptimized + debuginfo] target(s) in 12m 50s
Exit: 0. Wall time: 770.855s.
The original 19 quick-check steps:
shellcheck --version && actionlint -> exit 0
cargo fmt --all --check -> exit 0
./scripts/check_unsafe_code_allowances.sh -> exit 0
./scripts/check_layer_dependencies.sh -> exit 0
./scripts/check_architecture_migration_rules.sh -> exit 0
./scripts/check_logging_guardrails.sh -> exit 0
./scripts/check_error_other_format_ratchet.sh -> exit 0
./scripts/check_no_tokio_io_uring.sh -> exit 0
./scripts/check_extension_schema_boundaries.sh -> exit 0
./scripts/check_body_cache_whitelist.sh -> exit 0
./scripts/check_s3s_footprint.sh -> exit 0
./scripts/check_test_count_ratchet.sh -> exit 0
./scripts/check_fips_wording.sh -> exit 0
./scripts/check_embedded_secrets.sh -> exit 0
make script-tests -> exit 0
python3 ./scripts/test_object_generation_protocol_model.py -> exit 0
python3 ./scripts/check_test_wiring.py -> exit 0
./scripts/check_no_planning_docs.sh -> exit 0
./scripts/check_uring_lane_lib_only.sh -> exit 0
Wall time: 286.337s.
cargo nextest run --profile ci -p rustfs --no-fail-fast
Summary [ 217.017s] 5382 tests run: 5382 passed (4 leaky), 40 skipped
Exit: 0. Wall time: 1324.414s.
cargo nextest run --profile ci -p rustfs-s3-types -p rustfs-protocols --features rustfs-protocols/swift --no-fail-fast
Summary [ 2.189s] 543 tests run: 543 passed, 0 skipped
Exit: 0. Wall time: 123.192s.
cargo nextest run --profile ci -p rustfs-ecstore list_objects_paginate --no-fail-fast
Summary [ 0.170s] 8 tests run: 8 passed, 5921 skipped
Exit: 0. Wall time: 261.324s.
cargo nextest run --profile ci -p rustfs-heal durable_mrf_anchor_retires_only_after_exact_bucket_generation_is_deleted --no-fail-fast
Summary [ 0.805s] 1 test run: 1 passed, 835 skipped
Exit: 0. Wall time: 199.801s.
Baseline regeneration:
./scripts/check_test_count_ratchet.sh --update
wrote .config/test-count-baseline.txt: 52 crates, 21340 tests, 164 ignored
Test-only increases: ecstore48, heal3, iam14, io-metrics2, obs3, policy14,
rustfs34. Existing ignored values and all other rows are byte-identical.
s3s footprint OK: files importing s3s is 195 (baseline files=195)
s3s footprint OK: s3s s3_error! lines is 1584 (baseline s3_error_lines=1584)
s3s footprint OK: ecstore files referencing s3s is 36 (baseline ecstore_files=36)
Historical runtime red controls on f622 (same retained two assertions):
cargo nextest run --profile ci -p rustfs --lib -E 'test(=connect::diagnostics::trace_runtime::tests::local_top_rpc_reads_service_http_completions_and_signs_offline) | test(=connect::diagnostics::trace_runtime::tests::local_top_locks_reads_service_manager_and_signs_offline)' --no-fail-fast
Original path readiness with a temporary 500ms bind/seal gap:
Summary [ 0.863s] 2 tests run: 0 passed, 2 failed, 5113 skipped
Exit: 100.
Sealed readiness in that same gap:
Summary [ 5.621s] 2 tests run: 2 passed, 5113 skipped
Exit: 0.
Compiled M1: replace the helper predicate with path.exists():
Summary [ 0.870s] 2 tests run: 0 passed, 2 failed, 5113 skipped
Exit: 100.
Restore the sealed predicate byte-for-byte:
Summary [ 5.640s] 2 tests run: 2 passed, 5113 skipped
Exit: 0.
Both reds observe mode755 with owner_matches=true and fail at the original
RPC/locks unwraps with StateSecurity. M1 failure locations: trace_runtime.rs
2547:29 and 2713:29. No new assertions were introduced.
cargo nextest run --profile ci -p rustfs --lib -E 'test(connect::diagnostics::trace_runtime::tests::)' --no-fail-fast
Summary [ 20.748s] 28 tests run: 28 passed (1 leaky), 5087 skipped
Exit: 0. This security sweep is historical; the full final unit gate above
covers the actual c474 candidate.
The first disk-stopped compile ran no tests and is excluded from red credit.
All temporary seal instrumentation is removed from the final source.
Final unit cleanup markers: 4 leaky, 40 original skips (exit0). The ld
__eh_frame linker warning also appears verbatim in the original f622 run
and the earlier c474 run; there are no new source warnings.
T1.2 batch 3 (rustfs/backlog#2743): s3select-api, s3select-query and
the error items of protocols move off s3s by scripts/codemods, run with
--hold crates/protocols/src/sftp/errors.rs. That file stays on s3s: it
downcasts the backend error, whose runtime type is still s3s::S3Error.
- rustfs/src/protocols/client.rs: the two webdav session methods now
return rustfs_s3_types::S3Result and convert the s3s error with From.
- codemod: a child module that calls s3_error! through `use super::*`
gets its own `use rustfs_s3_types::s3_error;` once its parent imports
that macro by name; otherwise it is listed as GLOB.
- s3-types: a table test pins code, status and the s3s error the legacy
edge renders for every variant a moved call site names.
- s3_error_lines 1585 -> 1584 (a lifecycle comment reworded).
rustfs/backlog#2747 batch 3. `Body::from_stream` wraps a chunk stream of
unknown length over `StreamBody`/`BoxBody`: one data frame per chunk, the
first error ends the body, the size hint stays unknown, and dropping the
body drops the stream. Parity tests pin it to the s3s `StreamingBlob::wrap`
it replaces, at the `http_body` level and byte for byte through hyper.
The Swift handlers in rustfs-protocols build their responses as the owned
`Body`. `SwiftService` converts them into the wrapped S3 service's response
body type through `From<Body>`, so rustfs keeps serving `s3s::Body` at the
legacy edge unchanged. Request bodies stay Send-only (`UnsyncBoxBody`); the
two storage writers and the DLO/SLO segment streams get `Sync` from
`sync_wrapper::SyncStream`, which is what s3s's `http_body_unsync` mutex
provided before. The DTO and error uses of s3s in protocols are untouched
(T1.5 and T1.2).
Add dual_stack_differential to e2e_test. It starts two rustfs processes
from the same binary, stack A on RUSTFS_S3_STACK=legacy and stack B on
RUSTFS_E2E_STACK_B (default legacy until rustfs/backlog#2734 T2.5),
sends both the same aws-sdk-s3 script covering the 73 methods of
impl S3 for FS (a normal request and a 4xx negative each), and compares
status, error code, an allowlist of response headers and a few typed
body fields. Error message text is never compared. Each run writes
<target>/dual-stack/<run>.json with compared_ops and the (op, field,
a, b) differences; any difference fails the test.
differential_detects_an_injected_difference starts B with
RUSTFS_REGION=eu-west-1 and requires GetBucketLocation to be reported.
The comparator is pure and unit-tested without a server.
Wire the module into the e2e-smoke filter and update the e2e-smoke and
e2e-full selection digests and the e2e_test test-count row.
Refs rustfs/backlog#2740
s3_error_lines counted every line holding the text `s3_error!`, which
also matches the RustFS-owned `rustfs_s3_types::s3_error!`. A codemod
that moves call sites onto rustfs-s3-types therefore could not lower it
(rustfs/backlog#2743, decision 3; rustfs/rustfs#8432, gotcha 2).
The counter now applies a textual, per-file and per-line rule that errs
on the side of counting (full rule in the script header):
- occurrences spelled `rustfs_s3_types::s3_error!` never count;
- a line holding `s3s::s3_error!` always counts;
- any other `s3_error!` line counts unless its file is cleared, that is,
it imports the macro from rustfs_s3_types by name on a line starting
with the use item (or defines it with `macro_rules! s3_error`) and does
not also bring s3s's macro in (`use s3s::s3_error`, a `use s3s::{..}`
tree naming it or a crate-root `*`, `use s3s::*`, or
`#[macro_use] extern crate s3s`);
- fail closed: a file matching neither form (`use super::*`, a
crate-local re-export, a renamed or glob rustfs_s3_types import, a
commented-out import, a comment-only mention) counts.
Census of the 1585 lines on 96fc06776: 1478 in 64 files that import the
macro from s3s; 107 in 15 files matching neither form (105 through
`use super::*`, 1 qualified `s3s::s3_error!`, 1 comment line); none in
files that import it from rustfs_s3_types or define it. Every line stays
counted, so the baseline remains s3_error_lines=1585 (files=202 and
ecstore_files=36 unchanged). Rewriting rustfs/src/auth.rs's import to
rustfs_s3_types lowers the new counter to 1567; the old one stayed 1585.
`--self-test` (wired into `make script-tests`) copies the script into
22 fixture trees: 6 positive cases and 16 negative ones (lines that must
not count, forms that must not clear a file, and a missing baseline,
missing key or failing classification search that must fail). Each tree
has a control line that must count, and each run gets s3s text on stdin.
Seven cases were red against the old counter. 19 mutants each turn their
case red: old definition, flipped import test, dropped fail-closed
branch, dropped e2e_test exclusion, dropped s3s veto, dropped qualified
s3s rule, dropped rustfs-qualified strip, unanchored rustfs import,
renamed import accepted, nested glob taken as root glob, single-line
s3s trees only, dropped s3s glob forms, dropped macro_use form, dropped
definition clearing, ignored classification rg errors, error-line search
without its path, classification searches without their path, missing
baseline skipped, missing key skipped.
* test(ecstore): pin the InvalidRange responses of new_getobjectreader
Both error branches of new_getobjectreader build an ErrorResponse with S3ErrorCode::InvalidRange, and no test observed either code. Pin the code, its wire name and the location fields of each branch before the code type moves to rustfs-s3-types (rustfs/backlog#2743), so a changed code on either branch fails a test. The ecstore test-count row rises by the two tests.
* chore(scripts): add the s3s-to-rustfs-s3-types error codemod
Task T1.2 of rustfs/backlog#2734 (rustfs/backlog#2743) moves S3Error, S3ErrorCode, S3Result and s3_error! from s3s to rustfs-s3-types by script, one crate closure per PR. The codemod rewrites use trees (splitting mixed s3s imports) and qualified code paths, never comments or literals, adds the rustfs-s3-types dependency, drops s3s only from a crate that no longer names it, refuses every file of the s3s edge allowlist, takes --hold for a file whose type is fixed outside the closure, and prints a per-crate census. A second run changes nothing. Its self-test runs under make script-tests.
* refactor(ecstore): take S3ErrorCode from rustfs-s3-types
Output of scripts/codemods/s3_error_to_s3_types.py over crates/ecstore, crates/lifecycle, crates/scanner and crates/replication (rustfs/backlog#2743 batches 1 and 2). The ErrorResponse code field, the two InvalidRange constructions in new_getobjectreader and an unused tier import now use rustfs_s3_types::S3ErrorCode; the variant names and wire names are unchanged. lifecycle, scanner and replication reference none of the four error names. warm_backend.rs and warm_backend_s3.rs are held: they compare against rustfs_s3_client::ErrorResponse.code, which stays the s3s type until s3-client moves.
Add rustfs_s3_types::Body, a BoxBody<Bytes, StdError> wrapper with
empty() and From<Bytes | Vec<u8> | String | &'static str>, and the
legacy From conversions to and from s3s::Body behind compat-s3s.
The signer is now generic over the request body: it only reads and
writes the request head, so it needs no body type and its callers keep
theirs. s3-client sends its requests with the owned Body, madmin signs a
bodiless head, and targets drops its two S3Request/S3Response header
wrappers in favour of extract_params_header.
rustfs-signer, rustfs-madmin and rustfs-targets no longer depend on
s3s; the s3s footprint baseline drops from 209 to 202 files.
Refs rustfs/backlog#2747, rustfs/backlog#2734
* ci: make the s3s footprint baseline exact and lower-only
The s3s footprint guard compared three counters against ceilings held in
the script itself, so a ceiling could sit above the tree and a drift was
fixable by raising it (rustfs/rustfs#5964 raised the s3_error! ceiling
to go green). The three values now live in
.config/s3s-footprint-baseline.txt as files, s3_error_lines and
ecstore_files, the script reads only that file, and a measurement that
differs from its baseline in either direction fails: growth prints
"s3s footprint grew: <measured> > <baseline>", shrinkage prints
"lower the baseline to <measured> in this PR". The counting method is
unchanged; the baseline is the current measurement of
gateway/integration (209 / 1585 / 36). The header records the end state:
once all three reach 0, T4.2 switches the script to failing on any
occurrence.
Refs rustfs/backlog#2739, rustfs/backlog#2734
* ci: add the s3s edge allowlist mode to the footprint guard (#8405)
`scripts/check_s3s_footprint.sh --mode allowlist` fails on every Rust
file that references s3s (`s3s::` or `s3s_sigv4`) and is not matched by
an entry of the new `.config/s3s-edge-allowlist.txt`; `--dry-run` lists
those files and exits 0. The allowlist carries the initial edge from the
spec, and the script rejects an entry that names or matches anything
under crates/ other than crates/e2e_test. The baseline mode and the
argument-less CI invocation are unchanged.
Refs rustfs/backlog#2741, rustfs/backlog#2734
Before any DTO or codec is replaced, pin the exact bytes that
deserialize followed by serialize produces for every persisted
bucket-configuration XML family in its three source shapes (AWS
documentation, MinIO-written, RustFS-written), and the bytes RustFS
writes today for a typed value. Byte equality only: a structural
comparison would let a codec replacement change the persisted encoding
unnoticed.
The generator script refuses to overwrite existing goldens without
--force, refuses on dirty codec paths, cannot report success from its
write mode, and records the generating commit in the fixtures README.
Refs rustfs/backlog#2744, rustfs/backlog#2734
Quick Checks now fails any PR that lowers the number of test attributes
in a workspace crate or adds #[ignore], measured per crate against the
exact rows of .config/test-count-baseline.txt (rustfs/backlog#2734
section 1 rule 3). The counter widens the spec's literal
#[(tokio::)?test] to the tokio attributes that carry arguments and both
spellings of #[test_case(...)], and anchors at line start so
commented-out attributes do not count. The initial baseline covers 52
crates, 21077 tests and 164 ignored.
Refs rustfs/backlog#2738
Refs rustfs/backlog#2734
`rustfs-s3-types` now provides `S3ErrorCode` (generated by build.rs from
src/codes.txt, the grep-derived call surface, plus `Custom`), `S3Error`,
`S3Result`, `StdError` and the `s3_error` macro with the same call syntax
as s3s, so the T1.2 codemod is a `use` swap. Statuses are hand-written and
pinned to the s3s behaviour RustFS ships today by an oracle test that uses
s3s as a dev-dependency only. The `From` conversions for the legacy edge
live behind the off-by-default `compat-s3s` feature (the orphan rule
forbids them in rustfs/src/error.rs); `rustfs` enables it.
Refs rustfs/backlog#2742, rustfs/backlog#2734
Move the ten tower layers that patch s3s behaviour out of
rustfs/src/server/layer.rs into rustfs/src/server/legacy_compat/, one file
per layer with its tests. Each file header names the s3s behaviour it
patches, the gateway mechanism that replaces it (Replaced by:), and the task
that removes it (Removed by: T2.12, rustfs/backlog#2771). The layers keep
their code, their order in external_service_stack!, and their behaviour; no
layer is deleted.
Add server::http::tests::stack_census, which renders the ordered service
chain of the real external_service_stack! expansion and compares it with a
committed constant, so a reorder, insertion or removal is a visible diff.
Route the two s3s error types the moved layers use through a
storage_api::server::legacy_compat shim so the split adds no s3s-importing
file, and lower the s3s footprint baselines to the measured values (files 209
to 208, s3_error! lines 1586 to 1585). Keep the server runtime-source bypass
guard covering the moved code.
Refs rustfs/backlog#2787, rustfs/backlog#2734
@@ -20,11 +20,11 @@ planning-docs-check: ## Check that no planning-type documents are committed
./scripts/check_no_planning_docs.sh
.PHONY:pre-commit
pre-commit:fmt-checkunsafe-code-checkarchitecture-migration-checklogging-guardrails-checkerror-other-ratchet-checktokio-io-uring-checkextension-schema-checkbody-cache-whitelist-checks3s-footprint-checkfips-wording-checkembedded-secrets-checktest-wiring-checkdoc-paths-checkplanning-docs-checkquick-check## Run fast pre-commit checks without clippy/full tests
pre-commit:fmt-checkunsafe-code-checkarchitecture-migration-checklogging-guardrails-checkerror-other-ratchet-checktokio-io-uring-checkextension-schema-checkbody-cache-whitelist-checks3s-footprint-checktest-count-ratchet-checktest-seams-checkfips-wording-checkembedded-secrets-checktest-wiring-checkdoc-paths-checkplanning-docs-checkquick-check## Run fast pre-commit checks without clippy/full tests
@echo "✅ All pre-commit checks passed!"
.PHONY:pre-pr
pre-pr:fmt-checkunsafe-code-checkarchitecture-migration-checklogging-guardrails-checkerror-other-ratchet-checktokio-io-uring-checkextension-schema-checkbody-cache-whitelist-checks3s-footprint-checkfips-wording-checkembedded-secrets-checktest-wiring-checkdoc-paths-checkplanning-docs-checklog-analyzer-rules-checkoffline-enrollment-e2e-checkclippy-checktest## Run full pre-PR checks with clippy and tests
pre-pr:fmt-checkunsafe-code-checkarchitecture-migration-checklogging-guardrails-checkerror-other-ratchet-checktokio-io-uring-checkextension-schema-checkbody-cache-whitelist-checks3s-footprint-checktest-count-ratchet-checktest-seams-checkfips-wording-checkembedded-secrets-checktest-wiring-checkdoc-paths-checkplanning-docs-checklog-analyzer-rules-checkoffline-enrollment-e2e-checkclippy-checktest## Run full pre-PR checks with clippy and tests
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
name:"Reference RustFS binary"
description:"Download the pinned reference RustFS release, verify its sha256 and expose the executable path. A missing pin, missing asset or digest mismatch fails the step."
inputs:
pin-file:
description:"Pin file naming the reference release (repository-relative)"
required:false
default:".config/reference-binary.toml"
platform:
description:"Platform table in the pin file"
required:false
default:"linux-x86_64-gnu"
destination:
description:"Directory that receives the executable; defaults to $RUNNER_TEMP/reference-binary"
required:false
default:""
outputs:
path:
description:"Absolute path of the verified executable"
# Supports cross-compilation for amd64 and arm64 via TARGETPLATFORM.
ARG TARGETPLATFORM
ARG BUILDPLATFORM
ARG TARGETARCH
# Do not declare TARGETPLATFORM, BUILDPLATFORM or TARGETARCH here: a global
# declaration without a value shadows BuildKit's automatic values, so the
# stage-level re-declarations below see empty strings. They are only
# declared inside the stage that uses them.
ARGRUSTFS_BUILD_FEATURES=""
ARGRC_RELEASE=latest
@@ -169,7 +170,24 @@ RUN --mount=type=cache,target=/usr/local/cargo/registry \
;;\
esac
RUN sh /usr/src/rustfs/scripts/download_rc.sh "$TARGETARCH" musl "$RC_RELEASE" /usr/local/bin/rc
# TARGETARCH is only provided by BuildKit. Fall back to TARGETPLATFORM (for
# example set by docker-compose) and then to the build host, like the cargo
# step above, so every path that builds this file passes a real architecture.
RUNset -eu;\
rc_arch="${TARGETARCH:-}";\
if[ -z "${rc_arch}"];then\
case"${TARGETPLATFORM:-}" in \
linux/amd64)rc_arch=amd64 ;;\
linux/arm64)rc_arch=arm64 ;;\
*)\
case"$(uname -m)" in \
x86_64)rc_arch=amd64 ;;\
aarch64|arm64)rc_arch=arm64 ;;\
*)echo"TARGETARCH is empty and TARGETPLATFORM='${TARGETPLATFORM:-}' is not linux/amd64 or linux/arm64; cannot choose an rc architecture on $(uname -m)" >&2;exit1;;\
esac;;\
esac;\
fi;\
sh /usr/src/rustfs/scripts/download_rc.sh "${rc_arch}" musl "${RC_RELEASE}" /usr/local/bin/rc
// RUSTFS_COMPAT_TODO(s3gate-metadata-xml): the s3s codec reads persisted XML during migration. Remove after every supported writer uses the gateway codec and every retained metadata object and backup archive is verified or rewritten.
"ordinary overwrite must remove the remote object or retain its cleanup owner at write quorum: during_copyback={during_copyback}, owners={cleanup_owners:?}, remote_objects={remote_objects}"
# Persisted bucket-configuration XML goldens (rustfs/backlog#2744)
Byte-level goldens for the 13 XML families that `BucketMetadata` persists as
`*_config_xml`. They pin what `rustfs_ecstore::api::bucket::utils::deserialize`
followed by `serialize` emits today, so that the DTO/codec replacement (T1.4)
and the persisted-bytes comparison (T3.7) can be proven byte-identical: an
older binary must keep reading what a newer one writes, and the reverse.
Generated at commit: `6b1554003ebf8f2037ffb7da9c9b906527e758da` (2026-10-07T17:40:56Z)
Checked by `crates/ecstore/tests/bucket_config_goldens.rs`
(`cargo nextest run -p rustfs-ecstore bucket_config_goldens`, 39 tests).
Generated once by `scripts/gen_bucket_config_goldens.sh`, which records the
commit above and refuses to run again unless `--force` is passed. Do not
regenerate after a DTO or codec replacement: at that point a diff here is the
finding, not noise.
## Layout
`<family>/<shape>.in.xml` is an input; `<family>/<shape>.out.xml` is the exact
bytes of `serialize(deserialize(in))` on the generating commit. The test
asserts `bytes == out` (never structural equality), that `out` is a fixed point
of the same round trip, and for the `rustfs` shape that `serialize(typed
value)` still equals `rustfs.in.xml`.
| Shape | Source |
| --- | --- |
| `aws` | Written for this corpus from the public S3 XML schema in the shape of the AWS documentation examples: XML declaration, `xmlns` on the root, indented. No AWS prose is copied. |
| `minio` | The bytes a MinIO server persists. Seven families are carved verbatim out of `../minio/bucket_metadata.blob.hex` (MinIO `RELEASE.2025-07-23T15-54-02Z`); the rest are hand-shaped as compact Go `encoding/xml` output with the S3 namespace on the root, because MinIO does not persist those families. |
| `rustfs` | The bytes RustFS writes today through `serialize` for a typed value (`rustfs_values` in the test). This shape carries the MinIO-only members RustFS accepts (`DelMarkerExpiration`, `ExcludeFolders`, `ExcludedPrefixes`, `DeleteReplication`). |
| Family | Root element | `minio.in.xml` provenance |
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.