mirror of
https://github.com/rustfs/rustfs.git
synced 2026-10-04 04:21:35 +00:00
Chris
641c4b3493
* fix(ci): include pagination regression in full E2E selection * fix(deps): replace yanked yoke-derive release * fix(scanner): expose pause backlog replica diagnostics (#8258) * fix(scanner): expose pause backlog replica diagnostics Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(connect): stabilize runtime profile lease cancellation Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(connect): tolerate delayed schedule startup in CI Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(e2e): retry quota reads during usage warmup Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(ecstore): avoid meta-bucket incarnation self-deadlock Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(test): use persisted incarnation in heal fixture Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> --------- Co-authored-by: zhi22915 <qiuzgang@gmail.com> * fix(usage): reconcile stale counters after lifecycle expiration (#8108) * fix(usage): reconcile stale counters after lifecycle expiration * fix(usage): account lifecycle expiry during continuous writes * test(usage): run lifecycle usage scenarios on one scanner store * test(usage): use a Windows-representable pre-mutation offset * fix(usage): harden expiry accounting recovery and quota checks Borrow expiry receipt bucket names and avoid allocating a map key on cache hits. Cover cancelled receipts, durable snapshot recovery, and legacy quota admission after scanner confirmation. Use representable timestamp offsets in the quota regression. Refs rustfs/backlog#2689 Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(usage): recover stale persisted counts through the scanner Seed incorrect complete usage for empty and retained-object buckets, then run the real scanner and publication consumer without further object mutations. Verify durable and admin usage over two cycles instead of writing a corrected snapshot in the test. Refs rustfs/backlog#2689 Refs rustfs/backlog#2691 Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(e2e): bound delimiter pagination fixture concurrency The 120-second smoke timeout expired after 1018 of 1200 serial fixture PUTs, before LIST ran. Prepare the same objects with at most eight concurrent requests and await every PUT. Retain the timeout and strengthen exact prefix, KeyCount, empty Contents, and continuation-token assertions with phase diagnostics. Refs rustfs/backlog#2689 Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(upgrade): establish a persisted previous-release baseline Seed the pinned previous-release cluster and restart it once with its data intact before replacing any node. Require every old writer to pass the strict readiness probe and preserve the seed through both mixed phases and the final current cluster. Keep InternalError fail-fast behavior and all existing compatibility deadlines and assertions. Refs rustfs/backlog#2689 Refs rustfs/backlog#2384 Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(ecstore): bound cancellation metadata persistence waits Use the system-bucket incarnation boundary now supplied by main PR #8268. Bound the three cancellation waits that previously hung during pool.bin persistence, retaining their remote-generation, target-cohort, and durable-state assertions. Refs rustfs/backlog#2697 Refs rustfs/backlog#2689 Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> --------- Co-authored-by: Chris <anzhengchao@gmail.com> Co-authored-by: Hauser <housemecn@gmail.com> Co-authored-by: heihutu <heihutu@gmail.com> Co-authored-by: zhi22915 <qiuzgang@gmail.com> * test(e2e): retain startup and shutdown failure diagnostics * fix(test): supply CPU workload for sampler regression * fix(ci): locate security chain scripts in the workspace * fix(usage): recover historical counters with generation fencing (#8273) * fix(usage): recover historical counters during continued writes Use newer converged scanner snapshots to reconcile stale absolute usage baselines while preserving concurrent mutation and expiry receipt fences. Cover durable publication, admin and quota reads, and legacy generations. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * refactor(usage): fence snapshots and move preserved cache entries Apply the cached scanner generation floor before every reconciliation path and retain it even when an older snapshot happens to match core counts. Move preserved usage entries instead of cloning their histogram maps under the cache lock, retaining expiry receipt identity and cancellation fences. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> --------- Co-authored-by: heihutu <heihutu@gmail.com> Co-authored-by: zhi22915 <qiuzgang@gmail.com> * test(scanner): verify checkpoint takeover and repair dispatch (#8275) * test(scanner): cover checkpoint handoff and repair dispatch Drive runtime budget expiry, partial-cycle persistence, leadership claims, and stale checkpoint rejection between real disk-backed fixture scans. Verify that a metadata repair beyond the first bounded prefix is saved in the scanner ledger and dispatched by the MRF consumer. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * ci: isolate scanner fixtures and refresh full e2e membership Reserve nextest capacity for the real-disk scanner publication and MRF admission fixtures. Bind both platform membership checks to the reviewed pagination deadline test added on main. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * refactor(scanner): consolidate checkpoint fixture lifecycle Keep one durable control store across timeout and leadership transitions, and inject generation advancement into the shared checkpoint scenario. Check the actual saved metadata path so late-write rejection also proves that existing checkpoint bytes remain intact. Centralize MRF fixture isolation and reuse nextest process isolation when the startup environment already satisfies the test contract. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> --------- Co-authored-by: heihutu <heihutu@gmail.com> Co-authored-by: zhi22915 <qiuzgang@gmail.com> * fix(obs): distinguish allocator counters from live memory (#8274) * fix(obs): distinguish allocator counters from live memory Preserve count/counter semantics and mark requested-byte attribution unavailable when live statistics or sampling are missing. Document sustained multipart memory diagnosis. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * refactor(obs): parse allocator statistics from one node Resolve each statistic before interpreting its shape, avoiding unsupported-field tree scans and mixing data across wrapper scopes. Preserve unavailable-statistics policy and add precedence regressions. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * test(ecstore): isolate late parity recovery from metadata hedges Use the existing object-scoped hedge timer barrier in exact-count recovery fixtures. Preserve payload and total-read assertions and verify that the omitted parity disk is read only during late refresh. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> --------- Co-authored-by: heihutu <heihutu@gmail.com> Co-authored-by: zhi22915 <qiuzgang@gmail.com> * fix(usage): combine identical snapshot retention branches * fix(test): await HTTP sender readiness in Top RPC fixture * [release/1.0.1] Gate multipart copy through write admission (#8284) Gate multipart copy through write admission Make UploadPartCopy acquire the shared foreground write admission permit before lifecycle locks or source readers so server-side multipart copy cannot bypass the same backpressure used by UploadPart. Document the shared queue semantics and add focused coverage for saturation, cancellation, lock ordering, and disabled admission. Co-authored-by: zhi22915 <qiuzgang@gmail.com> * Gate multipart copy through write admission (#8283) Make UploadPartCopy acquire the shared foreground write admission permit before lifecycle locks or source readers so server-side multipart copy cannot bypass the same backpressure used by UploadPart. Document the shared queue semantics and add focused coverage for saturation, cancellation, lock ordering, and disabled admission. Co-authored-by: zhi22915 <qiuzgang@gmail.com> * fix: add UploadPart OOM validation guardrails (#8287) * docs(release): validate candidates on release branch * fix(scanner): validate checkpoints against global cycle fence (#8278) ## Related Issues Related to rustfs/backlog#2701. ## Summary of Changes Route scanner checkpoint cycle and leader validation through the global store while retaining the owning set for cache persistence, CAS revisions and publication admission. ## Verification Two independent final-diff source reviews found no issues across correctness, concurrency and durability, test coverage, compatibility, performance and simplicity on head `8b8fe51d092090b053f552ae283960e2e306be33`. Root approval `5373624714` is bound to that exact head. Regression tests cover real two-pool routing, stale fences, post-save rejection and CAS conflicts; their reported local execution belongs to the PR author, not this merge operation. Current required CI remains pending, and this authorized admin squash does not establish CI or runtime acceptance. ## Impact Restores checkpoint progress when global cycle and leader state differ from a set-scoped view. No format, retry, timeout, assertion or scanner-policy changes are introduced by this diff. The three prior main scanner failures remain unproved repaired. ## Additional Notes Full validation must run on the resulting exact main revision. Reverting this patch restores the earlier set-scoped fence lookup and its checkpoint rejection behavior. * fix(ci): restore E2E membership and pagination timeouts (#8281) * ci: locate the auto-testing checkout for lanes that run evidence from a subdirectory (#8279) ## Related Issues Follow-up to #8229. ## Summary of Changes Locate the private auto-testing checkout from the lane root or the workspace root so the nested security checkout can record functional-chain evidence. ## Verification The exact PR headb66129ab9fpassed one mechanical correctness and simplicity review, nine real-Git layout and provenance checks, and sixteen existing evidence/envelope tests. The baseline sibling layout failed with git exit 128; the corrected layout succeeded while revision mismatches and missing checkouts stayed rejected. Current PR checks are completed with successful or skipped conclusions, including the aggregate. ## Impact Both lane and private-script revision checks remain intact. No time limits, assertions, production behavior, or evidence validation requirements change. The synthetic layout checks do not execute the actual scheduled security suite; integrated main CI and release acceptance remain separate gates. ## Additional Notes Approved review 5374559393 is bound to the exact head above. Reverting the single-file change restores the previous checkout lookup. * fix: add UploadPart OOM validation guardrails Add a Docker validation harness for backlog#2704 so the ordinary UploadPart low-concurrency memory workload can be reproduced with comparable case metadata, process/cgroup sampling, TLS and metrics toggles, cache-env controls, and write reclaim/direct-write experiments. Warn when operators set the unrecognized RUSTFS_OBJECT_CACHE_* variables that appeared in the reporter compose file. The variables are reported but remain ignored, so startup does not silently change object data cache behavior. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> --------- Co-authored-by: overtrue <anzhengchao@gmail.com> Co-authored-by: AL <allan.bednarowski@gmail.com> Co-authored-by: hector <42570491+majinghe@users.noreply.github.com> Co-authored-by: heihutu <heihutu@gmail.com> Co-authored-by: zhi22915 <qiuzgang@gmail.com> * fix(scanner): pass fence store to checkpoint fixture * fix(s3): queue bucket operations and restore strict Clippy checks (#8290) * fix(s3): queue concurrent bucket creation and deletion Keep eight active bucket transactions and bound admission waiting to 128 requests and 30 seconds. Preserve detached transaction ownership and return Retry-After with overload responses. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * fix(ecstore): restore strict Clippy compatibility on Rust 1.99 Use try_update without changing atomic ordering or overflow behavior. Keep recursive storage futures boxed once at each frame and remove the redundant async-recursion macro, including its non-recursive SQL planner use. Remove needless closure borrows and orphaned dependency entries. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> --------- Co-authored-by: heihutu <heihutu@gmail.com> Co-authored-by: zhi22915 <qiuzgang@gmail.com> * chore: refresh dependencies and atomic update APIs Update workspace dependencies and the lockfile. Replace deprecated atomic fetch_update aliases with try_update while preserving closures and memory ordering. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * improve * fix(scanner): diagnose and verify pause backlog recovery (#8293) * fix(scanner): diagnose and verify pause backlog recovery Expose the retained replica snapshot and claimed membership in abnormal admin status responses. Keep diagnostics off metrics updates and verify single-pool recovery and conflicting-proof preservation across 24 sets. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> * refactor(scanner): move replica snapshots into diagnostics Consume the terminal admin read snapshot in a single state match and move membership, revision, and error buffers into the response. Verify buffer handoff and the unchanged JSON contract without altering ledger authority. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> --------- Co-authored-by: heihutu <heihutu@gmail.com> Co-authored-by: zhi22915 <qiuzgang@gmail.com> * fix(heal): wait for held legacy responsibility in replay test * ci: remove Docker Hub description sync * fix: emit NextPartNumberMarker only when ListParts is truncated ListPartsInfo.next_part_number_marker was a non-optional usize that defaulted to 0 and was only assigned when the response was truncated. The S3 serializer then emitted it unconditionally as Some(0), causing AWS SDK paginators to loop infinitely on part_number_marker=0 instead of terminating. Change the field to Option<usize> (None by default) and set it only inside the is_truncated branch. The S3 output layer now uses .and_then() so NextPartNumberMarker is absent when IsTruncated=false, matching AWS S3 behavior. Fixes #8208 (cherry picked from commit44de803a38) * fix(s3): honor sparse ListParts markers and verify termination Resume part listings at the first part above the numeric marker, even when that marker is absent. Use binary search over the sorted part numbers and retain the existing exact-tail empty-slice path. Add storage, XML, and real AWS SDK paginator regressions for empty and terminal pages, sparse markers, and multipart completion integrity. Co-Authored-By: heihutu <heihutu@gmail.com> Co-Authored-By: zhi22915 <qiuzgang@gmail.com> (cherry picked from commit673031eea1) * fix(storage): publish delete rollback backups atomically Stage rollback metadata outside the rollback directory and publish it only after the full write succeeds. A short write must not leave a backup that quorum rollback can rename over acknowledged version history. Add an isolated real short-write regression and register the backported ListParts SDK test in the smoke and Linux full inventories. * test(e2e): register paginator regression in Darwin inventory * fix(release): install yq before Helm template checks * chore(release): align installation references for 1.0.1 --------- Co-authored-by: Hauser <housemecn@gmail.com> Co-authored-by: zhi22915 <qiuzgang@gmail.com> Co-authored-by: Peder Bergan <pederbe@users.noreply.github.com> Co-authored-by: heihutu <heihutu@gmail.com> Co-authored-by: AL <allan.bednarowski@gmail.com> Co-authored-by: hector <42570491+majinghe@users.noreply.github.com> Co-authored-by: Chapman <touch65536@gmail.com>
scripts/ index
Authoritative inventory of everything under scripts/ (backlog#1153
infra-13). One row per top-level entry; subdirectories get one row each and
keep their own READMEs. The test-layer map that ties the major runners
together lives in docs/testing/README.md.
Statuses
- ci-gate — wired into CI, release, or image-build pipelines. Do not move, rename, or change flags without updating the wiring listed in the last column.
- dev-tool — run by humans: local dev loops, runbooks, validation harnesses. Kept working, not wired into CI.
- archived — one-shot scripts whose investigation/issue is finished, moved
to
scripts/archive/. Unmaintained reference material: never wire into CI, and expect bit-rot. To resurrect one, move it back and give it an index row here.
Adding a script? Add an index row in the same PR. Issue-scoped scripts
(run_issueNNN_*, validate_issue_NNN_*) are expected to be archived when
their issue closes.
Repository & CI gates
| Entry | Status | Purpose | Wiring / docs |
|---|---|---|---|
check_architecture_migration_rules.sh |
ci-gate | Architecture-boundary anti-regression guard | ci.yml Quick Checks; make pre-commit |
check_body_cache_whitelist.sh |
ci-gate | Keeps the app-layer body-cache eligibility gate fail-closed | ci.yml Quick Checks |
check_doc_paths.sh |
ci-gate | Fails when instruction/architecture docs reference repo paths that no longer exist | make pre-commit / pre-pr |
check_embedded_secrets.sh |
ci-gate | Repo-wide scan blocking committed private key material and provider credential literals | ci.yml Quick Checks; make pre-commit / pre-pr |
check_extension_schema_boundaries.sh |
ci-gate | Extension-schema crate boundary guard | ci.yml Quick Checks; make pre-commit |
check_layer_dependencies.sh |
ci-gate | Crate-layering DAG guard (reads layer-dependency-baseline.txt) |
ci.yml Quick Checks |
check_logging_guardrails.sh |
ci-gate | Blocks legacy logging patterns from returning | make pre-commit / pre-pr |
check_migration_gate_count.sh |
ci-gate | Migration-critical test gate with committed count floor (.config/migration-gate-floor.txt) |
ci.yml Test and Lint; docs/testing/README.md |
check_no_planning_docs.sh |
ci-gate | Blocks committed planning-type documents | ci.yml Quick Checks; make pre-commit |
check_no_tokio_io_uring.sh |
ci-gate | Keeps tokio's io-uring backend disabled | ci.yml Quick Checks |
check_s3s_footprint.sh |
ci-gate | Lower-only ratchet freezing the direct s3s surface ahead of the s3gate migration | ci.yml Quick Checks; make pre-commit |
check_unsafe_code_allowances.sh |
ci-gate | Unsafe-code allowance ledger guard | ci.yml Quick Checks |
layer-dependency-baseline.txt |
ci-gate (data) | Committed baseline consumed by check_layer_dependencies.sh |
arch-checks skill |
static.sh |
ci-gate | Static-build helper executed inside image builds | Dockerfile.source, Dockerfile.decommission-local |
helm_chart_version.sh |
ci-gate | Keeps the Helm chart version in sync with the release | helm-package.yml |
test_helm_templates.sh |
ci-gate | Helm template rendering test | helm-package.yml |
Test & e2e runners
| Entry | Status | Purpose | Wiring / docs |
|---|---|---|---|
diagnose_scanner_enumeration_restart.py |
dev-tool | Strict fixed raw-entry-budget scanner-worker restart diagnostic | Checkpoint fixture |
prepare_replacement_migration.py |
dev-tool | Prepares digest-bound schema 5/6 replacement maintenance approvals | Replacement recovery |
test_prepare_replacement_migration.py |
dev-tool | Verifies maintenance approval scope, publication, and stopped-writer assertion | Python unittest; same runbook |
test_diagnose_scanner_enumeration_restart.py |
dev-tool | Driver report validation and positive convergence oracle tests | Python unittest; same guide |
e2e-run.sh |
ci-gate | Boots a rustfs server and runs the s3s-e2e black-box conformance tool against it |
ci.yml e2e-tests jobs; docs/testing/README.md |
run_ecstore_validation_suite.sh |
dev-tool | ecstore black-box validation suite (quick/full/destructive/fuzz profiles) |
docs/testing/README.md, docs/testing/ecstore-validation-suite-design.md |
run_e2e_tests.sh |
dev-tool | Local e2e_test crate runner (starts a server, applies filters, cleans up) |
crates/e2e_test/README.md |
run.sh |
dev-tool | Local rustfs startup wrapper | make e2e-server; Justfile |
run.ps1 |
dev-tool | Windows counterpart of run.sh |
— |
probe.sh |
dev-tool | Probe-style e2e run | make probe-e2e |
run_scanner_validation_harness.sh |
dev-tool | Scanner validation harness | docs/operations/scanner-benchmark-runbook.md |
check_test_wiring.py |
dev-tool | Validates test wiring and assembles one or more measured Scanner/Heal release descriptors into a single release evidence bundle | .config/scanner-heal-required-tests.json; check_test_wiring.py --self-test |
run_scanner_heal_evidence_case.sh |
dev-tool | Runs one Scanner/Heal release-evidence registry case and checks the produced receipt/oracle | .config/scanner-heal-required-tests.json; check_test_wiring.py --check-scanner-heal |
run_scanner_heal_authority_evidence.py |
dev-tool | Assembles measured Scanner/Heal G01 root/quota authority release descriptors from operator-collected authority artifacts | .config/scanner-heal-required-tests.json; test_scanner_heal_authority_evidence.sh |
run_scanner_heal_checkpoint_crash_evidence.py |
dev-tool | Assembles measured Scanner/Heal G02/R-E checkpoint and restart release descriptors from scanner restart diagnostic reports | diagnose_scanner_enumeration_restart.py; test_scanner_heal_checkpoint_crash_evidence.sh |
run_scanner_heal_g14_multiset_evidence.py |
dev-tool | Assembles measured Scanner/Heal G14 same-window EC8+4 multi-set/multi-pool release descriptors from e2e case directories or an operator-collected proof | .config/scanner-heal-required-tests.json; test_scanner_heal_g14_multiset_evidence.sh |
run_scanner_heal_g09_upgrade_evidence.sh |
dev-tool | Runs the G09 mixed-version and rollback upgrade E2E lanes against a pinned previous release and verifies the raw evidence artifacts | docs/testing/ci-gates.md; .github/workflows/e2e-upgrade.yml; test_scanner_heal_g09_upgrade_evidence.sh |
run_scanner_heal_linux_evidence_plan.py |
dev-tool | Writes the unified Scanner/Heal Linux release-evidence execution manifest and can run lightweight preflight checks without producing measured evidence | docs/testing/ci-gates.md; test_scanner_heal_linux_evidence_plan.sh |
run_scanner_heal_scoped_ack_evidence.py |
dev-tool | Assembles measured Scanner/Heal G03 scoped ACK publication and mixed-peer fallback release descriptors | .config/scanner-heal-required-tests.json; test_scanner_heal_scoped_ack_evidence.sh |
run_scanner_heal_legacy_rollback_evidence.py |
dev-tool | Assembles measured Scanner/Heal R-L legacy source-conflict, migration-gap, and source-retirement release descriptors | .config/scanner-heal-required-tests.json; test_scanner_heal_legacy_rollback_evidence.sh |
run_scanner_heal_mrf_evidence.py |
dev-tool | Assembles measured Scanner/Heal G07/G08/P4 MRF release descriptors from W13 raw artifacts | .config/scanner-heal-required-tests.json; test_scanner_heal_w13_mrf_evidence.sh |
run_scanner_heal_scheduler_pressure_evidence.py |
dev-tool | Assembles measured Scanner/Heal G10/P1/P3 scheduler-pressure release descriptors from a completed measured ABBA run, recovery-window proof, and profile artifacts | docs/operations/scanner-benchmark-runbook.md; test_scanner_heal_scheduler_pressure_evidence.sh |
run_scanner_heal_status_outcome_probe.py |
dev-tool | Normalizes live Scanner/Heal status/outcome observations into the measured G05/G06/R-D raw artifacts consumed by the descriptor producer | .config/scanner-heal-required-tests.json; test_scanner_heal_status_outcome_evidence.sh |
run_scanner_heal_status_outcome_evidence.py |
dev-tool | Assembles measured Scanner/Heal G05/G06/R-D status-and-outcome release descriptors from same-run status, compatibility, and disposition artifacts | .config/scanner-heal-required-tests.json; test_scanner_heal_status_outcome_evidence.sh |
run_scanner_heal_maintenance_evidence.py |
dev-tool | Assembles measured Scanner/Heal G11/G13 maintenance-producer release descriptors from operator-collected proof JSON | .config/scanner-heal-required-tests.json; test_scanner_heal_maintenance_evidence.sh |
run_scanner_heal_w13_mrf_evidence.sh |
dev-tool | Runs the W13 durable MRF replay lanes and writes G07/G08/P4 bundle-ready evidence descriptors | docs/testing/ci-gates.md; test_scanner_heal_w13_mrf_evidence.sh |
run_scanner_heal_w16_recovery_evidence.sh |
dev-tool | Runs the W16 recovery-intent and quota authority lanes and writes G04/G12 bundle-ready evidence descriptors | docs/testing/ci-gates.md; test_scanner_heal_w16_recovery_evidence.sh |
test_scanner_validation_harness.sh |
dev-tool | Self-test for the scanner validation harness | — |
test_scanner_heal_checkpoint_crash_evidence.sh |
dev-tool | Shell self-test for the Scanner/Heal checkpoint/crash evidence assembler | — |
test_scanner_heal_authority_evidence.sh |
dev-tool | Shell self-test for the Scanner/Heal G01 authority evidence assembler | — |
test_scanner_heal_scoped_ack_evidence.sh |
dev-tool | Shell self-test for the Scanner/Heal G03 scoped ACK evidence assembler | — |
test_scanner_heal_legacy_rollback_evidence.sh |
dev-tool | Shell self-test for the Scanner/Heal R-L legacy rollback evidence assembler | — |
test_scanner_heal_g14_multiset_evidence.sh |
dev-tool | Shell self-test for the Scanner/Heal G14 multi-set/multi-pool evidence assembler | — |
test_scanner_heal_scheduler_pressure_evidence.sh |
dev-tool | Shell self-test for the Scanner/Heal scheduler-pressure evidence assembler | — |
test_scanner_heal_status_outcome_evidence.sh |
dev-tool | Shell self-test for the Scanner/Heal status-and-outcome evidence assembler | — |
test_scanner_heal_maintenance_evidence.sh |
dev-tool | Shell self-test for the Scanner/Heal G11/G13 maintenance evidence assembler | — |
test_scanner_heal_g09_upgrade_evidence.sh |
dev-tool | Shell self-test for the Scanner/Heal G09 upgrade evidence runner | — |
test_scanner_heal_w16_recovery_evidence.sh |
dev-tool | Shell self-test for the Scanner/Heal W16 recovery evidence runner | — |
test_scanner_heal_w13_mrf_evidence.sh |
dev-tool | Shell self-test for the Scanner/Heal W13 MRF evidence runner | — |
scanner_abba.py |
dev-tool | Scanner/heal ABBA orchestration and evidence gates via run_scanner_validation_harness.sh --abba |
docs/operations/scanner-benchmark-runbook.md |
test_scanner_abba.py |
dev-tool | Synthetic ABBA adapter and failure-path tests | test_scanner_validation_harness.sh |
test_build_rustfs_options.sh |
dev-tool | Shell test for rustfs build-option wiring | make test (script-tests) |
test_entrypoint_credentials.sh |
dev-tool | Container entrypoint credential-handling test | make test (script-tests) |
test_helm_chart_version.sh |
dev-tool | Test for helm_chart_version.sh |
— |
test_package_service_scripts.sh |
ci-gate | Verifies DEB/RPM install, upgrade, removal, and service restart scriptlets | audit.yml workflow-pin-report |
test_package_versions.sh |
ci-gate | Exact-output and fail-closed tests for DEB/RPM package version normalization | audit.yml workflow-pin-report; package.yml RPM build |
windows-sftp-listener-smoke.sh |
dev-tool | Confirms rustfs.exe --features sftp binds an SFTP listener on Windows |
— |
Benchmark & performance harnesses
| Entry | Status | Purpose | Wiring / docs |
|---|---|---|---|
run_hotpath_warp_ab.sh |
dev-tool | Linux warp A/B rig for the hotpath series (quick local A/B) | docs/operations/hotpath-warp-ab-runbook.md |
run_hotpath_warp_abba.sh |
ci-gate | Formal ABBA warp runner (baseline/candidate interleaved, --rounds >= 3); the runner performance-ab.yml executes |
performance-ab.yml (scheduled); docs/operations/hotpath-warp-ab-runbook.md |
hotpath_warp_ab_gate.sh |
dev-tool | Relative-budget gate evaluated over the warp A/B results | used by run_hotpath_warp_ab.sh; hotpath runbook |
run_internode_grpc_ab_bench.sh |
dev-tool | One-click A/B driver for the internode gRPC optimization stages | docs/operations/internode-grpc-benchmark-runbook.md |
run_internode_transport_baseline.sh |
dev-tool | Internode transport baseline runner | internode runbook; crates/io-metrics/README.md |
run_four_node_cluster_failover_bench.sh |
dev-tool | Four-node cluster failover benchmark | docker/compose/README.md; internode runbook |
run_object_batch_bench.sh |
dev-tool | Batch object benchmark runner (warp/s3bench) | internode + scanner runbooks |
run_object_batch_bench_enhanced.sh |
dev-tool | Enhanced batch benchmark runner; hub used by the smoke rigs | hotpath runbook |
run_pinned_paired_abba_bench.sh |
dev-tool | Pinned RustFS/MinIO paired ABBA benchmark orchestrator for backlog#1432 | test_pinned_paired_abba_bench.sh |
run_get_codec_streaming_smoke.sh |
dev-tool | Local GET benchmark harness for the codec streaming read path | docs/testing/ecstore-validation-suite-design.md |
run_get_1mib_abba_stage_metrics.sh |
dev-tool | Exact-1MiB isolated-host GET ABBA/stage-metrics harness for backlog#1434 | test_get_1mib_abba_stage_metrics.sh |
issue_2007_coalescer_prometheus_report.py |
dev-tool | Read-only Prometheus report for GET metadata coalescer delay cost validation; usage in the module docstring | test_issue_2007_coalescer_prometheus_report.sh |
prometheus_metrics_1649_smoke.py |
dev-tool | Read-only Prometheus instant-query smoke check for the backlog#1649 metric dimensions, required labels, and retired series; usage in the module docstring | --self-test |
run_gt1g_get_http_matrix.sh |
dev-tool | >1 GiB GET HTTP matrix | docs/testing/ecstore-validation-suite-design.md |
run_gt1g_multipart_put_matrix.sh |
dev-tool | >1 GiB multipart PUT matrix | docs/testing/ecstore-validation-suite-design.md |
sample_remote_rustfs_rss.sh |
dev-tool | Remote RustFS PID CPU/RSS TSV sampler for hotpath profiling runs | test_sample_remote_rustfs_rss.sh; backlog#1647 |
summarize_samply_profile_symbols.py |
dev-tool | Offline samply profile.json.gz + .syms.json function-level hotpath summarizer |
test_summarize_samply_profile_symbols.py; backlog#1647 |
run_scanner_benchmarks.sh |
dev-tool (disposition pending) | Scanner performance benchmark runner. Contains a hardcoded stale path; disposition owned by backlog perf-10 — do not fix, move, or delete it here | — |
Local development & operations
| Entry | Status | Purpose | Wiring / docs |
|---|---|---|---|
dev_clear.sh |
dev-tool | Local dev cleanup. scripts/dev_*.sh is a CI paths-filter glob — keep the naming |
ci.yml/build.yml paths filters |
dev_deploy.sh |
dev-tool | Copy a built binary to dev servers | make deploy (.config/make/deploy.mak); Justfile |
dev_rustfs.sh |
dev-tool | Local dev run loop | — |
dev_rustfs.env |
dev-tool (data) | Env presets for the dev scripts | — |
restart_local_single_node_multidisk_rustfs.sh |
dev-tool | Restart a local single-node multi-disk instance | — |
inspect_dashboard.sh |
dev-tool | Sanity-checks the Grafana dashboard JSON | .docker/observability |
notify.sh |
dev-tool | Starts a local webhook receiver for notify-target development | — |
manual_transition_debug.sh |
dev-tool | Log/metrics helper for manual transition troubleshooting | — |
manual_transition_journal_audit.sh |
dev-tool | Journal + metrics + log audit for manual transition jobs | — |
manual_transition_mixed_rollout_matrix.sh |
dev-tool | Matrix generator for mixed-version rollout phases | — |
manual_transition_mixed_rollout_runbook.sh |
dev-tool | Reusable mixed-version rollout runbook generator (external run) | — |
manual_transition_mixed_version_docker_harness.sh |
dev-tool | Dedicated #1508 Docker harness for old/new manual-transition rollout evidence with strict/baseline/blocked result classification | test_manual_transition_runbooks.sh |
monitor_manual_transition_ci.sh |
dev-tool | CI workflow/status watcher for manual transition follow-up monitoring | — |
manual_transition_soak_matrix.sh |
dev-tool | Matrix generator for nightly stress windows | — |
manual_transition_nightly_stress_runbook.sh |
dev-tool | Nightly stress entrypoint with failure snapshot templates | — |
install-flatc.sh |
dev-tool | Local flatc installer (macOS) | — |
install-protoc.sh |
dev-tool | Local protoc installer (macOS/Linux) | — |
cargo_publish_workspace.sh |
dev-tool | Generates the workspace crate publish order, checks existing registry versions, and optionally runs ordered cargo publish dry-runs or publication |
docs/operations/cargo-publish-workspace.md |
makefile-header.sh |
dev-tool | Generates the ## —— section —— header lines used in .config/make/*.mak |
— |
tls_gen.md |
dev-tool (doc) | Notes on generating local TLS certificates | — |
Subdirectories
| Entry | Status | Purpose | Wiring / docs |
|---|---|---|---|
fuzz/ |
ci-gate | Unified cargo-fuzz runner and helpers for the fuzz/ sub-workspace |
fuzz.yml; fuzz/README.md |
release/ |
ci-gate | Release creation and DEB/RPM version-normalization helpers | build.yml; package.yml |
s3-tests/ |
ci-gate | ceph/s3-tests compatibility harness (allow-lists, patches, report tooling) | ci.yml; e2e-s3tests.yml; scripts/s3-tests/README.md |
security/ |
ci-gate | Workflow-pin enforcement and release supply-chain asset generation | audit.yml; build.yml |
table-catalog/ |
dev-tool | S3-Tables / pyiceberg validation suite | docs/architecture/s3-tables-support-matrix.md |
test/ |
dev-tool | Manual operational validation runbooks (decommission, tier lifecycle), paired .sh + .md |
— |
archive/ |
archived | Retired one-shot scripts (see below) | — |
Archived (scripts/archive/)
Moved 2026-07 (backlog#1153 infra-13) after a whole-tree reference census: each entry had zero references from CI, Makefiles, docs, or code — or was referenced only by other scripts in this same archived set. Reasons:
| Entry | Was |
|---|---|
validate_issue_785_list_objects.sh |
One-shot issue validation (list-objects series) |
validate_issue_786_list_objects.sh |
One-shot issue validation (list-objects series) |
validate_issue_787_list_quorum.sh |
One-shot issue validation (list-quorum) |
validate_issue_841_list_objects_observability.sh |
One-shot issue validation (list observability) |
validate_issue_1365_docker.sh |
One-shot issue validation (docker repro) |
validate_issue_2723_site_replication.sh |
One-shot issue validation (site replication) |
validate_issue_3031_docker.sh |
One-shot issue validation (docker repro) |
run_issue712_deeper_zero_copy_put_with_capture.sh |
One-shot perf capture for backlog#712 |
run_issue797_local_4node_16disk_ab.sh |
One-shot 4-node/16-disk A/B for backlog#797 |
run_issue_2573_acceptance.sh |
One-shot acceptance run for issue #2573 |
run_issue_2941_perf_capture.sh |
One-shot perf capture for issue #2941 |
run_put_large_stage_breakdown.sh |
backlog#706 large-PUT stage breakdown (family) |
run_put_large_stage_breakdown_with_capture.sh |
backlog#706 one-shot wrapper (family) |
run_put_large_tuning_matrix.sh |
backlog#706 tuning matrix (family) |
collect_put_large_stage_breakdown_artifacts.sh |
backlog#706 artifact collector (family) |
analyze_put_service_metrics_deltas.py |
backlog#706 metrics-delta analyzer (family) |
README-stress-test.md |
GET-optimization one-shot suite doc |
stress-test-get-optimization.sh |
GET-optimization one-shot stress test |
quick-validate-get-optimization.sh |
GET-optimization one-shot validation |
benchmark-sf-optimization.sh |
GET-optimization one-shot benchmark |
prepare_gt1g_get_test_objects.sh |
>1 GiB GET investigation one-shot fixture prep |
run_gt1g_multipart_put_server_path_focus.sh |
>1 GiB PUT investigation one-shot focus run |
run_get_metrics_gate_smoke.sh |
One-shot GET metrics-gate smoke |
run_listobjects_verified_bench.sh |
One-shot verified list-objects bench |
run_object_batch_bench_abc.sh |
One-shot capacity/object profile A/B/C controller |
run_object_data_cache_bench.sh |
One-shot GET bench for the object-data-cache rollout gate |
setup-test-binaries.sh |
One-shot Docker-build test binary fixture |
test.sh |
Ancient manual mc bucket smoke scratchpad |
test_policy.json |
Orphaned IAM policy fixture (hardcoded test bucket) |