gsadmin/rustfs
1
0
Fork 0
mirror of https://github.com/rustfs/rustfs.git synced 2026-09-09 13:46:05 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
8e987ce0a6e3b3bec0aa2b071f8d14b2881b2a08
rustfs/scripts/README.md
T
houseme 8e987ce0a6 test(scanner): add G09 upgrade evidence runner (#7522) 
* fix(replication): close the GA blocker set from backlog#2366 (#7503)

* fix(replication): close GA blockers from backlog#2366

Implements the P1 set from the pre-GA replication audit:

- Replication rule tag filters now require every And.Tag to match, replacing
  the s3s OR semantics with a local AND matcher that fails closed on a
  malformed tag.
- A replicated group membership change no longer writes the group status, so
  a membership update carrying the default Enabled status cannot silently
  re-enable a disabled group on the peer.
- A successful IAM import schedules one collapsed full-IAM snapshot per remote
  peer instead of leaving the imported entities local-only.
- A pending endpoint refresh is redriven by the heavyweight reconcile tick,
  carries its own ilm-expiry override, and no longer blocks a remove that
  drops every unacknowledged peer.
- Site metrics expose local replication failure totals and rolling windows;
  node-level counters no longer report a constructed zero.
- set/remove-remote-target notify peer metadata caches before returning, so a
  follow-up put-bucket-replication on another node sees the target.
- Adds the site-replication operations runbook, a docs index, a replication
  support boundary section, and the Replication changelog section.

* fix(site-replication): resume only a locally driven endpoint refresh

The peer-side edit handler journals a pending endpoint refresh with an empty
`remote_peers` map and commits it inside the same request through
`apply_internal_peer_edit`. The reconcile tick could not tell that journal
from the coordinator's own: with no required peers it reads as complete on
sight, so the tick committed it with `edit_state` - losing the local-name
sync - and cleared it under the request that owned it, whose commit then
reported the refresh as changed and denied the coordinator the peer
acknowledgement it was waiting for.

Resume now runs only for a journal that carries the fan-out topology. A
receiver's journal stays for the coordinator to redrive with the same
refresh id, which is the path that already recovers it.

* fix(site-replication): keep an explicit disabled group status on a snapshot

Skipping the group-status write whenever an item carries members stopped a
membership change from re-enabling a disabled group, but it also silenced the
full-IAM snapshot, which always sends members together with the sender's real
status. A peer that did not have the group yet created it through
`GroupInfo::new` - enabled - so a bootstrap, a repair, or the snapshot an IAM
import now schedules handed every member of a frozen group live access there.

The madmin wire maps an unset `groupStatus` to Enabled, so only Enabled can be
a default. Disabled is always explicit and is applied again.

* fix(site-replication): schedule the import snapshot without recording a failure

`import-iam` reused the failure-recording path to queue its full-IAM
snapshot. That raises `retry_count` on every call, so three imports - the
normal shape of a bulk migration done one archive at a time - escalated a
healthy peer to `retryStats.failed` with the scheduling note shown as
`lastError`, which is exactly the signal the runbook tells operators to
repair. A full retry queue also turned a completed import into a 503.

Scheduling now only ensures the collapsed entry exists, and a failure to
schedule is logged instead of failing the request: the entities are already
imported and the reconcile pass still closes the gap.

* fix(admin): stop reporting replication failures as retries

`retries` is the minio-go counter for redeliveries, and mc prints it as such.
Filling it with the failure count claimed a redelivery that never happens: a
failed object is not retried by an event today, it waits for the scanner heal
pass. `errors` keeps the failure counters; `retries` stays zero until there is
a real redelivery to count, and the runbook now says so.

* perf(site-replication): aggregate failure windows without cloning bucket stats

`site_metrics_snapshot` went through `get_all`, which clones every bucket's
stats, and then scanned each target's sample deque twice. That deque is
bounded only by the one-hour window, so an unreachable target under load -
the case an operator polls this endpoint for - made every
`mc admin replicate status` copy the whole backlog and hold the read lock
against the failure path while doing it.

It now folds under the read lock and takes both windows in one walk. The
`max` against the serialized `last_minute` / `last_hour` snapshots is dropped:
those are stamped onto per-bucket clones elsewhere and are always zero in this
node-local cache.

* fix(site-replication): reject a conflicting ilm-expiry override on a re-run

The commit now reads the ilm-expiry override back out of the pending refresh
journal, so a second edit that asks for a different value had it dropped while
the request still reported success. Re-running without the flag keeps pinning
the recorded value - that is the documented way to redrive a stuck refresh -
but an explicit different value is now rejected instead of ignored.

* fix(admin): do not fail a remote-target write on a peer reload error

set/remove-remote-target propagated the peer metadata reload error, so a
target that was already persisted and live on this node reported a 5xx to the
client whenever one peer could not be reached. Every S3 bucket-config write
path treats that reload as best effort and only warns; these two admin
handlers now do the same, and the reason is logged with the bucket and action.

* fix(site-replication): undo every bucket a cut-short refresh rewrote

When a remove accepted on another node clears the refresh journal mid-pass,
only the bucket holding the lock at that moment had its restored target
undone. The buckets rewritten earlier in the same pass kept a target pointing
at the removed peer whenever the remove's own cleanup had already walked past
them. The undo now covers every bucket this pass rewrote, attempting all of
them so one failure does not strand the rest.

* fix(site-replication): keep replay running while an endpoint refresh is pending

A pending endpoint refresh took the whole heavyweight pass with it, so a peer
that never came back froze IAM and bucket replay to every healthy peer too -
the stall this journal's resume path was meant to end. The refresh arm now
drains the retry queue before returning; it replays per-peer deliveries
against the endpoints currently committed in state, so it is unaffected by the
edit in flight. Bucket wiring reconciliation still waits, because it rewrites
the very targets the refresh is changing, and the runbook now says so.

* test(e2e): cover the AND semantics of a two-tag replication filter

The acceptance matrix only had a single-tag rule, which matches under both AND
and OR semantics and therefore proved nothing about the filter this fix
changed. It now also carries a two-tag `And` rule - the shape
`mc replicate add --tags "k1=v1&k2=v2"` writes - and asserts that an object
with one of the two tags is not admitted while an object with both is.

No new test function, so the nightly selection digest is unchanged.

* refactor(site-replication): fold the refresh state-change error into one constructor

The endpoint-refresh work added three `s3_error!` invocation lines, which the
s3s footprint ratchet is meant to prevent. Five copies of the same
concurrent-change error now share one constructor, so the surface nets one
line smaller than main; the baseline is retightened to match.

* fix(site-replication): report a peer whose IAM snapshot waits for a repair

An escalated snapshot entry records a deletion a snapshot cannot replay, so
only a repair settles it and the marker must survive. Scheduling an import
snapshot therefore leaves that peer's entry alone - and now says so, instead
of returning success while nothing was scheduled for it.

* docs(operations): state the group-status and escalation convergence limits

Two boundaries the fixes in this branch make load-bearing: a membership change
never carries an enable, so a group disabled on one site only has to be
re-enabled there explicitly; and a peer holding an escalated IAM entry does
not receive a scheduled snapshot, including the one a bulk import schedules,
until a repair settles it.

* fix(ci): bind performance runs to selected inputs (#7512)

* test(scanner): add G09 upgrade evidence runner

Add a reusable Linux x86_64 runner for the Scanner/Heal G09 mixed-version and rollback upgrade evidence lanes.

The helper reads the pinned previous-release asset metadata from the upgrade workflow, verifies the downloaded binary, builds the current head, runs both ignored E2E tests, and fails unless the expected G09 JSON artifacts exist.

Co-Authored-By: heihutu <heihutu@gmail.com>

Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

---------

Co-authored-by: 唐小鸭 <tangtang1251@qq.com>
Co-authored-by: Zhengchao An <anzhengchao@gmail.com>
Co-authored-by: zhi22915 <qiuzgang@gmail.com>
2026-09-08 23:38:54 +08:00

165 lines
15 KiB
Markdown
Raw Blame History

# scripts/ index
Authoritative inventory of everything under `scripts/` (backlog#1153
infra-13). One row per top-level entry; subdirectories get one row each and
keep their own READMEs. The test-layer map that ties the major runners
together lives in [`docs/testing/README.md`](../docs/testing/README.md).
**Statuses**
- **ci-gate** — wired into CI, release, or image-build pipelines. Do not move,
rename, or change flags without updating the wiring listed in the last
column.
- **dev-tool** — run by humans: local dev loops, runbooks, validation
harnesses. Kept working, not wired into CI.
- **archived** — one-shot scripts whose investigation/issue is finished, moved
to [`scripts/archive/`](archive/). Unmaintained reference material: never
wire into CI, and expect bit-rot. To resurrect one, move it back and give it
an index row here.
**Adding a script?** Add an index row in the same PR. Issue-scoped scripts
(`run_issueNNN_*`, `validate_issue_NNN_*`) are expected to be archived when
their issue closes.
## Repository & CI gates
| Entry | Status | Purpose | Wiring / docs |
|---|---|---|---|
| `check_architecture_migration_rules.sh` | ci-gate | Architecture-boundary anti-regression guard | ci.yml Quick Checks; `make pre-commit` |
| `check_body_cache_whitelist.sh` | ci-gate | Keeps the app-layer body-cache eligibility gate fail-closed | ci.yml Quick Checks |
| `check_doc_paths.sh` | ci-gate | Fails when instruction/architecture docs reference repo paths that no longer exist | `make pre-commit` / `pre-pr` |
| `check_embedded_secrets.sh` | ci-gate | Repo-wide scan blocking committed private key material and provider credential literals | ci.yml Quick Checks; `make pre-commit` / `pre-pr` |
| `check_extension_schema_boundaries.sh` | ci-gate | Extension-schema crate boundary guard | ci.yml Quick Checks; `make pre-commit` |
| `check_layer_dependencies.sh` | ci-gate | Crate-layering DAG guard (reads `layer-dependency-baseline.txt`) | ci.yml Quick Checks |
| `check_logging_guardrails.sh` | ci-gate | Blocks legacy logging patterns from returning | `make pre-commit` / `pre-pr` |
| `check_migration_gate_count.sh` | ci-gate | Migration-critical test gate with committed count floor (`.config/migration-gate-floor.txt`) | ci.yml Test and Lint; `docs/testing/README.md` |
| `check_no_planning_docs.sh` | ci-gate | Blocks committed planning-type documents | ci.yml Quick Checks; `make pre-commit` |
| `check_no_tokio_io_uring.sh` | ci-gate | Keeps tokio's io-uring backend disabled | ci.yml Quick Checks |
| `check_s3s_footprint.sh` | ci-gate | Lower-only ratchet freezing the direct s3s surface ahead of the s3gate migration | ci.yml Quick Checks; `make pre-commit` |
| `check_unsafe_code_allowances.sh` | ci-gate | Unsafe-code allowance ledger guard | ci.yml Quick Checks |
| `layer-dependency-baseline.txt` | ci-gate (data) | Committed baseline consumed by `check_layer_dependencies.sh` | arch-checks skill |
| `static.sh` | ci-gate | Static-build helper executed inside image builds | `Dockerfile.source`, `Dockerfile.decommission-local` |
| `helm_chart_version.sh` | ci-gate | Keeps the Helm chart version in sync with the release | helm-package.yml |
| `test_helm_templates.sh` | ci-gate | Helm template rendering test | helm-package.yml |
## Test & e2e runners
| Entry | Status | Purpose | Wiring / docs |
|---|---|---|---|
| `diagnose_scanner_enumeration_restart.py` | dev-tool | Strict fixed raw-entry-budget scanner-worker restart diagnostic | [Checkpoint fixture](../docs/testing/scanner-checkpoint-fixture.md) |
| `test_diagnose_scanner_enumeration_restart.py` | dev-tool | Driver report validation and positive convergence oracle tests | Python unittest; same guide |
| `e2e-run.sh` | ci-gate | Boots a rustfs server and runs the `s3s-e2e` black-box conformance tool against it | ci.yml `e2e-tests` jobs; `docs/testing/README.md` |
| `run_ecstore_validation_suite.sh` | dev-tool | ecstore black-box validation suite (`quick`/`full`/`destructive`/`fuzz` profiles) | `docs/testing/README.md`, `docs/testing/ecstore-validation-suite-design.md` |
| `run_e2e_tests.sh` | dev-tool | Local `e2e_test` crate runner (starts a server, applies filters, cleans up) | `crates/e2e_test/README.md` |
| `run_scanner_heal_g09_upgrade_evidence.sh` | dev-tool | Runs the Scanner/Heal G09 mixed-version and rollback upgrade evidence lanes against the pinned previous Linux release binary | e2e-upgrade workflow constants; backlog#2269 |
| `run.sh` | dev-tool | Local rustfs startup wrapper | `make e2e-server`; Justfile |
| `run.ps1` | dev-tool | Windows counterpart of `run.sh` | — |
| `probe.sh` | dev-tool | Probe-style e2e run | `make probe-e2e` |
| `run_scanner_validation_harness.sh` | dev-tool | Scanner validation harness | `docs/operations/scanner-benchmark-runbook.md` |
| `run_scanner_heal_evidence_case.sh` | dev-tool | Runs one Scanner/Heal release-evidence registry case and checks the produced receipt/oracle | `.config/scanner-heal-required-tests.json`; `check_test_wiring.py --check-scanner-heal` |
| `test_scanner_validation_harness.sh` | dev-tool | Self-test for the scanner validation harness | — |
| `scanner_abba.py` | dev-tool | Scanner/heal ABBA orchestration and evidence gates via `run_scanner_validation_harness.sh --abba` | `docs/operations/scanner-benchmark-runbook.md` |
| `test_scanner_abba.py` | dev-tool | Synthetic ABBA adapter and failure-path tests | `test_scanner_validation_harness.sh` |
| `test_build_rustfs_options.sh` | dev-tool | Shell test for rustfs build-option wiring | `make test` (script-tests) |
| `test_entrypoint_credentials.sh` | dev-tool | Container entrypoint credential-handling test | `make test` (script-tests) |
| `test_helm_chart_version.sh` | dev-tool | Test for `helm_chart_version.sh` | — |
| `test_package_versions.sh` | ci-gate | Exact-output and fail-closed tests for DEB/RPM package version normalization | audit.yml `workflow-pin-report`; package.yml RPM build |
| `windows-sftp-listener-smoke.sh` | dev-tool | Confirms `rustfs.exe --features sftp` binds an SFTP listener on Windows | — |
## Benchmark & performance harnesses
| Entry | Status | Purpose | Wiring / docs |
|---|---|---|---|
| `run_hotpath_warp_ab.sh` | dev-tool | Linux warp A/B rig for the hotpath series (quick local A/B) | `docs/operations/hotpath-warp-ab-runbook.md` |
| `run_hotpath_warp_abba.sh` | ci-gate | Formal ABBA warp runner (baseline/candidate interleaved, `--rounds >= 3`); the runner performance-ab.yml executes | performance-ab.yml (scheduled); `docs/operations/hotpath-warp-ab-runbook.md` |
| `hotpath_warp_ab_gate.sh` | dev-tool | Relative-budget gate evaluated over the warp A/B results | used by `run_hotpath_warp_ab.sh`; hotpath runbook |
| `run_internode_grpc_ab_bench.sh` | dev-tool | One-click A/B driver for the internode gRPC optimization stages | `docs/operations/internode-grpc-benchmark-runbook.md` |
| `run_internode_transport_baseline.sh` | dev-tool | Internode transport baseline runner | internode runbook; `crates/io-metrics/README.md` |
| `run_four_node_cluster_failover_bench.sh` | dev-tool | Four-node cluster failover benchmark | `docker/compose/README.md`; internode runbook |
| `run_object_batch_bench.sh` | dev-tool | Batch object benchmark runner (warp/s3bench) | internode + scanner runbooks |
| `run_object_batch_bench_enhanced.sh` | dev-tool | Enhanced batch benchmark runner; hub used by the smoke rigs | hotpath runbook |
| `run_pinned_paired_abba_bench.sh` | dev-tool | Pinned RustFS/MinIO paired ABBA benchmark orchestrator for backlog#1432 | `test_pinned_paired_abba_bench.sh` |
| `run_get_codec_streaming_smoke.sh` | dev-tool | Local GET benchmark harness for the codec streaming read path | `docs/testing/ecstore-validation-suite-design.md` |
| `run_get_1mib_abba_stage_metrics.sh` | dev-tool | Exact-1MiB isolated-host GET ABBA/stage-metrics harness for backlog#1434 | `test_get_1mib_abba_stage_metrics.sh` |
| `issue_2007_coalescer_prometheus_report.py` | dev-tool | Read-only Prometheus report for GET metadata coalescer delay cost validation; usage in the module docstring | `test_issue_2007_coalescer_prometheus_report.sh` |
| `prometheus_metrics_1649_smoke.py` | dev-tool | Read-only Prometheus instant-query smoke check for the backlog#1649 metric dimensions, required labels, and retired series; usage in the module docstring | `--self-test` |
| `run_gt1g_get_http_matrix.sh` | dev-tool | >1 GiB GET HTTP matrix | `docs/testing/ecstore-validation-suite-design.md` |
| `run_gt1g_multipart_put_matrix.sh` | dev-tool | >1 GiB multipart PUT matrix | `docs/testing/ecstore-validation-suite-design.md` |
| `sample_remote_rustfs_rss.sh` | dev-tool | Remote RustFS PID CPU/RSS TSV sampler for hotpath profiling runs | `test_sample_remote_rustfs_rss.sh`; backlog#1647 |
| `summarize_samply_profile_symbols.py` | dev-tool | Offline samply `profile.json.gz` + `.syms.json` function-level hotpath summarizer | `test_summarize_samply_profile_symbols.py`; backlog#1647 |
| `run_scanner_benchmarks.sh` | dev-tool (disposition pending) | Scanner performance benchmark runner. Contains a hardcoded stale path; **disposition owned by backlog perf-10 — do not fix, move, or delete it here** | — |
## Local development & operations
| Entry | Status | Purpose | Wiring / docs |
|---|---|---|---|
| `dev_clear.sh` | dev-tool | Local dev cleanup. `scripts/dev_*.sh` is a CI paths-filter glob — keep the naming | ci.yml/build.yml paths filters |
| `dev_deploy.sh` | dev-tool | Copy a built binary to dev servers | `make deploy` (`.config/make/deploy.mak`); Justfile |
| `dev_rustfs.sh` | dev-tool | Local dev run loop | — |
| `dev_rustfs.env` | dev-tool (data) | Env presets for the dev scripts | — |
| `restart_local_single_node_multidisk_rustfs.sh` | dev-tool | Restart a local single-node multi-disk instance | — |
| `inspect_dashboard.sh` | dev-tool | Sanity-checks the Grafana dashboard JSON | `.docker/observability` |
| `notify.sh` | dev-tool | Starts a local webhook receiver for notify-target development | — |
| `manual_transition_debug.sh` | dev-tool | Log/metrics helper for manual transition troubleshooting | — |
| `manual_transition_journal_audit.sh` | dev-tool | Journal + metrics + log audit for manual transition jobs | — |
| `manual_transition_mixed_rollout_matrix.sh` | dev-tool | Matrix generator for mixed-version rollout phases | — |
| `manual_transition_mixed_rollout_runbook.sh` | dev-tool | Reusable mixed-version rollout runbook generator (external run) | — |
| `manual_transition_mixed_version_docker_harness.sh` | dev-tool | Dedicated #1508 Docker harness for old/new manual-transition rollout evidence with strict/baseline/blocked result classification | `test_manual_transition_runbooks.sh` |
| `monitor_manual_transition_ci.sh` | dev-tool | CI workflow/status watcher for manual transition follow-up monitoring | — |
| `manual_transition_soak_matrix.sh` | dev-tool | Matrix generator for nightly stress windows | — |
| `manual_transition_nightly_stress_runbook.sh` | dev-tool | Nightly stress entrypoint with failure snapshot templates | — |
| `install-flatc.sh` | dev-tool | Local flatc installer (macOS) | — |
| `install-protoc.sh` | dev-tool | Local protoc installer (macOS/Linux) | — |
| `makefile-header.sh` | dev-tool | Generates the `## —— section ——` header lines used in `.config/make/*.mak` | — |
| `tls_gen.md` | dev-tool (doc) | Notes on generating local TLS certificates | — |
## Subdirectories
| Entry | Status | Purpose | Wiring / docs |
|---|---|---|---|
| `fuzz/` | ci-gate | Unified cargo-fuzz runner and helpers for the `fuzz/` sub-workspace | fuzz.yml; `fuzz/README.md` |
| `release/` | ci-gate | Release creation and DEB/RPM version-normalization helpers | build.yml; package.yml |
| `s3-tests/` | ci-gate | ceph/s3-tests compatibility harness (allow-lists, patches, report tooling) | ci.yml; e2e-s3tests.yml; `scripts/s3-tests/README.md` |
| `security/` | ci-gate | Workflow-pin enforcement and release supply-chain asset generation | audit.yml; build.yml |
| `table-catalog/` | dev-tool | S3-Tables / pyiceberg validation suite | `docs/architecture/s3-tables-support-matrix.md` |
| `test/` | dev-tool | Manual operational validation runbooks (decommission, tier lifecycle), paired `.sh` + `.md` | — |
| `archive/` | archived | Retired one-shot scripts (see below) | — |
## Archived (scripts/archive/)
Moved 2026-07 (backlog#1153 infra-13) after a whole-tree reference census:
each entry had **zero references** from CI, Makefiles, docs, or code — or was
referenced only by other scripts in this same archived set. Reasons:
| Entry | Was |
|---|---|
| `validate_issue_785_list_objects.sh` | One-shot issue validation (list-objects series) |
| `validate_issue_786_list_objects.sh` | One-shot issue validation (list-objects series) |
| `validate_issue_787_list_quorum.sh` | One-shot issue validation (list-quorum) |
| `validate_issue_841_list_objects_observability.sh` | One-shot issue validation (list observability) |
| `validate_issue_1365_docker.sh` | One-shot issue validation (docker repro) |
| `validate_issue_2723_site_replication.sh` | One-shot issue validation (site replication) |
| `validate_issue_3031_docker.sh` | One-shot issue validation (docker repro) |
| `run_issue712_deeper_zero_copy_put_with_capture.sh` | One-shot perf capture for backlog#712 |
| `run_issue797_local_4node_16disk_ab.sh` | One-shot 4-node/16-disk A/B for backlog#797 |
| `run_issue_2573_acceptance.sh` | One-shot acceptance run for issue #2573 |
| `run_issue_2941_perf_capture.sh` | One-shot perf capture for issue #2941 |
| `run_put_large_stage_breakdown.sh` | backlog#706 large-PUT stage breakdown (family) |
| `run_put_large_stage_breakdown_with_capture.sh` | backlog#706 one-shot wrapper (family) |
| `run_put_large_tuning_matrix.sh` | backlog#706 tuning matrix (family) |
| `collect_put_large_stage_breakdown_artifacts.sh` | backlog#706 artifact collector (family) |
| `analyze_put_service_metrics_deltas.py` | backlog#706 metrics-delta analyzer (family) |
| `README-stress-test.md` | GET-optimization one-shot suite doc |
| `stress-test-get-optimization.sh` | GET-optimization one-shot stress test |
| `quick-validate-get-optimization.sh` | GET-optimization one-shot validation |
| `benchmark-sf-optimization.sh` | GET-optimization one-shot benchmark |
| `prepare_gt1g_get_test_objects.sh` | >1 GiB GET investigation one-shot fixture prep |
| `run_gt1g_multipart_put_server_path_focus.sh` | >1 GiB PUT investigation one-shot focus run |
| `run_get_metrics_gate_smoke.sh` | One-shot GET metrics-gate smoke |
| `run_listobjects_verified_bench.sh` | One-shot verified list-objects bench |
| `run_object_batch_bench_abc.sh` | One-shot capacity/object profile A/B/C controller |
| `run_object_data_cache_bench.sh` | One-shot GET bench for the object-data-cache rollout gate |
| `setup-test-binaries.sh` | One-shot Docker-build test binary fixture |
| `test.sh` | Ancient manual `mc` bucket smoke scratchpad |
| `test_policy.json` | Orphaned IAM policy fixture (hardcoded test bucket) |
Reference in New Issue View Git Blame Copy Permalink