mirror of
https://github.com/rustfs/rustfs.git
synced 2026-10-04 20:43:04 +00:00
hector
ee6de7d786
* ci(package): build gnu and musl DEB/RPM variants with distinct file names
The Build and Release workflow produces four Linux binaries
(x86_64-gnu, aarch64-gnu, x86_64-musl, aarch64-musl), but packaging
only consumed the two gnu artifacts. Add matrix entries for the two
musl artifacts so every release ships all four DEB/RPM variants.
The libc variant is now part of the package file names, which would
otherwise collide between gnu and musl builds of the same version:
- deb: rustfs_<version>_<libc>_<arch>.deb
- rpm: rustfs-<libc>-<version>-<release>.<arch>.rpm
The dpkg Package and rpm Name stay plain "rustfs", so gnu and musl
remain mutually exclusive upgrades of one package rather than
co-installable packages fighting over /usr/bin/rustfs.
Dependency declarations now follow the linkage: gnu binaries
dynamically link glibc and keep Depends: libc6 (>= 2.31) /
glibc >= 2.31; musl binaries are statically linked and declare no
libc dependency. The libc variant is also visible in the package
description.
scripts/release/package_versions.sh gains a LIBC argument and its
contract tests cover both variants plus the invalid-libc cases.
* ci(package): align deb/rpm file names with the zip artifact naming
Rename the package file names so every release asset of one build
shares the same stem as its binary artifact, differing only by
extension:
- before: rustfs_<deb_version>_<libc>_<deb_arch>.deb
rustfs-<libc>-<rpm_version>-<rpm_release>.<rpm_arch>.rpm
- after: rustfs-linux-<arch>-<libc>-v<version>.deb / .rpm
e.g. rustfs-linux-x86_64-gnu-v1.0.0.zip,
rustfs-linux-x86_64-gnu-v1.0.0.deb,
rustfs-linux-x86_64-gnu-v1.0.0.rpm.
Non-development builds embed the raw release tag (with 'v'), like the
zips; development builds embed dev-<full sha>. The dpkg/rpm versions
(including the '~' prerelease ordering) are unchanged - they live in
the package metadata, and a side effect is that release asset names no
longer contain '~' (which GitHub normalizes to '.').
package_versions.sh now takes the target arch (x86_64|aarch64) instead
of the deb/rpm arch pair; the deb Architecture (amd64/arm64) in the
control metadata still comes from the workflow matrix. The two test
workflows that assemble deb download URLs from a release tag
(rustfs-table-test, rustfs-upgrade-test) are updated to the new name,
which also removes their '~'-to-'.' asset name workaround.
scripts/ index
Authoritative inventory of everything under scripts/ (backlog#1153
infra-13). One row per top-level entry; subdirectories get one row each and
keep their own READMEs. The test-layer map that ties the major runners
together lives in docs/testing/README.md.
Statuses
- ci-gate — wired into CI, release, or image-build pipelines. Do not move, rename, or change flags without updating the wiring listed in the last column.
- dev-tool — run by humans: local dev loops, runbooks, validation harnesses. Kept working, not wired into CI.
- archived — one-shot scripts whose investigation/issue is finished, moved
to
scripts/archive/. Unmaintained reference material: never wire into CI, and expect bit-rot. To resurrect one, move it back and give it an index row here.
Adding a script? Add an index row in the same PR. Issue-scoped scripts
(run_issueNNN_*, validate_issue_NNN_*) are expected to be archived when
their issue closes.
Repository & CI gates
| Entry | Status | Purpose | Wiring / docs |
|---|---|---|---|
check_architecture_migration_rules.sh |
ci-gate | Architecture-boundary anti-regression guard | ci.yml Quick Checks; make pre-commit |
check_body_cache_whitelist.sh |
ci-gate | Keeps the app-layer body-cache eligibility gate fail-closed | ci.yml Quick Checks |
check_doc_paths.sh |
ci-gate | Fails when instruction/architecture docs reference repo paths that no longer exist | make pre-commit / pre-pr |
check_embedded_secrets.sh |
ci-gate | Repo-wide scan blocking committed private key material and provider credential literals | ci.yml Quick Checks; make pre-commit / pre-pr |
check_extension_schema_boundaries.sh |
ci-gate | Extension-schema crate boundary guard | ci.yml Quick Checks; make pre-commit |
check_layer_dependencies.sh |
ci-gate | Crate-layering DAG guard (reads layer-dependency-baseline.txt) |
ci.yml Quick Checks |
check_logging_guardrails.sh |
ci-gate | Blocks legacy logging patterns from returning | make pre-commit / pre-pr |
check_migration_gate_count.sh |
ci-gate | Migration-critical test gate with committed count floor (.config/migration-gate-floor.txt) |
ci.yml Test and Lint; docs/testing/README.md |
check_no_planning_docs.sh |
ci-gate | Blocks committed planning-type documents | ci.yml Quick Checks; make pre-commit |
check_no_tokio_io_uring.sh |
ci-gate | Keeps tokio's io-uring backend disabled | ci.yml Quick Checks |
check_s3s_footprint.sh |
ci-gate | Lower-only ratchet freezing the direct s3s surface ahead of the s3gate migration | ci.yml Quick Checks; make pre-commit |
check_unsafe_code_allowances.sh |
ci-gate | Unsafe-code allowance ledger guard | ci.yml Quick Checks |
layer-dependency-baseline.txt |
ci-gate (data) | Committed baseline consumed by check_layer_dependencies.sh |
arch-checks skill |
static.sh |
ci-gate | Static-build helper executed inside image builds | Dockerfile.source, Dockerfile.decommission-local |
helm_chart_version.sh |
ci-gate | Keeps the Helm chart version in sync with the release | helm-package.yml |
test_helm_templates.sh |
ci-gate | Helm template rendering test | helm-package.yml |
Test & e2e runners
| Entry | Status | Purpose | Wiring / docs |
|---|---|---|---|
diagnose_scanner_enumeration_restart.py |
dev-tool | Strict fixed raw-entry-budget scanner-worker restart diagnostic | Checkpoint fixture |
prepare_replacement_migration.py |
dev-tool | Prepares digest-bound schema 5/6 replacement maintenance approvals | Replacement recovery |
test_prepare_replacement_migration.py |
dev-tool | Verifies maintenance approval scope, publication, and stopped-writer assertion | Python unittest; same runbook |
test_diagnose_scanner_enumeration_restart.py |
dev-tool | Driver report validation and positive convergence oracle tests | Python unittest; same guide |
e2e-run.sh |
ci-gate | Boots a rustfs server and runs the s3s-e2e black-box conformance tool against it |
ci.yml e2e-tests jobs; docs/testing/README.md |
run_ecstore_validation_suite.sh |
dev-tool | ecstore black-box validation suite (quick/full/destructive/fuzz profiles) |
docs/testing/README.md, docs/testing/ecstore-validation-suite-design.md |
run_e2e_tests.sh |
dev-tool | Local e2e_test crate runner (starts a server, applies filters, cleans up) |
crates/e2e_test/README.md |
run.sh |
dev-tool | Local rustfs startup wrapper | make e2e-server; Justfile |
run.ps1 |
dev-tool | Windows counterpart of run.sh |
— |
probe.sh |
dev-tool | Probe-style e2e run | make probe-e2e |
run_scanner_validation_harness.sh |
dev-tool | Scanner validation harness | docs/operations/scanner-benchmark-runbook.md |
check_test_wiring.py |
dev-tool | Validates test wiring and assembles one or more measured Scanner/Heal release descriptors into a single release evidence bundle | .config/scanner-heal-required-tests.json; check_test_wiring.py --self-test |
run_scanner_heal_evidence_case.sh |
dev-tool | Runs one Scanner/Heal release-evidence registry case and checks the produced receipt/oracle | .config/scanner-heal-required-tests.json; check_test_wiring.py --check-scanner-heal |
run_scanner_heal_authority_evidence.py |
dev-tool | Assembles measured Scanner/Heal G01 root/quota authority release descriptors from operator-collected authority artifacts | .config/scanner-heal-required-tests.json; test_scanner_heal_authority_evidence.sh |
run_scanner_heal_checkpoint_crash_evidence.py |
dev-tool | Assembles measured Scanner/Heal G02/R-E checkpoint and restart release descriptors from scanner restart diagnostic reports | diagnose_scanner_enumeration_restart.py; test_scanner_heal_checkpoint_crash_evidence.sh |
run_scanner_heal_g14_multiset_evidence.py |
dev-tool | Assembles measured Scanner/Heal G14 same-window EC8+4 multi-set/multi-pool release descriptors from e2e case directories or an operator-collected proof | .config/scanner-heal-required-tests.json; test_scanner_heal_g14_multiset_evidence.sh |
run_scanner_heal_g09_upgrade_evidence.sh |
dev-tool | Runs the G09 mixed-version and rollback upgrade E2E lanes against a pinned previous release and verifies the raw evidence artifacts | docs/testing/ci-gates.md; .github/workflows/e2e-upgrade.yml; test_scanner_heal_g09_upgrade_evidence.sh |
run_scanner_heal_linux_evidence_plan.py |
dev-tool | Writes the unified Scanner/Heal Linux release-evidence execution manifest and can run lightweight preflight checks without producing measured evidence | docs/testing/ci-gates.md; test_scanner_heal_linux_evidence_plan.sh |
run_scanner_heal_scoped_ack_evidence.py |
dev-tool | Assembles measured Scanner/Heal G03 scoped ACK publication and mixed-peer fallback release descriptors | .config/scanner-heal-required-tests.json; test_scanner_heal_scoped_ack_evidence.sh |
run_scanner_heal_legacy_rollback_evidence.py |
dev-tool | Assembles measured Scanner/Heal R-L legacy source-conflict, migration-gap, and source-retirement release descriptors | .config/scanner-heal-required-tests.json; test_scanner_heal_legacy_rollback_evidence.sh |
run_scanner_heal_mrf_evidence.py |
dev-tool | Assembles measured Scanner/Heal G07/G08/P4 MRF release descriptors from W13 raw artifacts | .config/scanner-heal-required-tests.json; test_scanner_heal_w13_mrf_evidence.sh |
run_scanner_heal_scheduler_pressure_evidence.py |
dev-tool | Assembles measured Scanner/Heal G10/P1/P3 scheduler-pressure release descriptors from a completed measured ABBA run, recovery-window proof, and profile artifacts | docs/operations/scanner-benchmark-runbook.md; test_scanner_heal_scheduler_pressure_evidence.sh |
run_scanner_heal_status_outcome_probe.py |
dev-tool | Normalizes live Scanner/Heal status/outcome observations into the measured G05/G06/R-D raw artifacts consumed by the descriptor producer | .config/scanner-heal-required-tests.json; test_scanner_heal_status_outcome_evidence.sh |
run_scanner_heal_status_outcome_evidence.py |
dev-tool | Assembles measured Scanner/Heal G05/G06/R-D status-and-outcome release descriptors from same-run status, compatibility, and disposition artifacts | .config/scanner-heal-required-tests.json; test_scanner_heal_status_outcome_evidence.sh |
run_scanner_heal_maintenance_evidence.py |
dev-tool | Assembles measured Scanner/Heal G11/G13 maintenance-producer release descriptors from operator-collected proof JSON | .config/scanner-heal-required-tests.json; test_scanner_heal_maintenance_evidence.sh |
run_scanner_heal_w13_mrf_evidence.sh |
dev-tool | Runs the W13 durable MRF replay lanes and writes G07/G08/P4 bundle-ready evidence descriptors | docs/testing/ci-gates.md; test_scanner_heal_w13_mrf_evidence.sh |
run_scanner_heal_w16_recovery_evidence.sh |
dev-tool | Runs the W16 recovery-intent and quota authority lanes and writes G04/G12 bundle-ready evidence descriptors | docs/testing/ci-gates.md; test_scanner_heal_w16_recovery_evidence.sh |
test_scanner_validation_harness.sh |
dev-tool | Self-test for the scanner validation harness | — |
test_scanner_heal_checkpoint_crash_evidence.sh |
dev-tool | Shell self-test for the Scanner/Heal checkpoint/crash evidence assembler | — |
test_scanner_heal_authority_evidence.sh |
dev-tool | Shell self-test for the Scanner/Heal G01 authority evidence assembler | — |
test_scanner_heal_scoped_ack_evidence.sh |
dev-tool | Shell self-test for the Scanner/Heal G03 scoped ACK evidence assembler | — |
test_scanner_heal_legacy_rollback_evidence.sh |
dev-tool | Shell self-test for the Scanner/Heal R-L legacy rollback evidence assembler | — |
test_scanner_heal_g14_multiset_evidence.sh |
dev-tool | Shell self-test for the Scanner/Heal G14 multi-set/multi-pool evidence assembler | — |
test_scanner_heal_scheduler_pressure_evidence.sh |
dev-tool | Shell self-test for the Scanner/Heal scheduler-pressure evidence assembler | — |
test_scanner_heal_status_outcome_evidence.sh |
dev-tool | Shell self-test for the Scanner/Heal status-and-outcome evidence assembler | — |
test_scanner_heal_maintenance_evidence.sh |
dev-tool | Shell self-test for the Scanner/Heal G11/G13 maintenance evidence assembler | — |
test_scanner_heal_g09_upgrade_evidence.sh |
dev-tool | Shell self-test for the Scanner/Heal G09 upgrade evidence runner | — |
test_scanner_heal_w16_recovery_evidence.sh |
dev-tool | Shell self-test for the Scanner/Heal W16 recovery evidence runner | — |
test_scanner_heal_w13_mrf_evidence.sh |
dev-tool | Shell self-test for the Scanner/Heal W13 MRF evidence runner | — |
scanner_abba.py |
dev-tool | Scanner/heal ABBA orchestration and evidence gates via run_scanner_validation_harness.sh --abba |
docs/operations/scanner-benchmark-runbook.md |
test_scanner_abba.py |
dev-tool | Synthetic ABBA adapter and failure-path tests | test_scanner_validation_harness.sh |
test_build_rustfs_options.sh |
dev-tool | Shell test for rustfs build-option wiring | make test (script-tests) |
test_entrypoint_credentials.sh |
dev-tool | Container entrypoint credential-handling test | make test (script-tests) |
test_helm_chart_version.sh |
dev-tool | Test for helm_chart_version.sh |
— |
test_package_service_scripts.sh |
ci-gate | Verifies DEB/RPM install, upgrade, removal, and service restart scriptlets | audit.yml workflow-pin-report |
test_package_versions.sh |
ci-gate | Exact-output and fail-closed tests for DEB/RPM package version normalization | audit.yml workflow-pin-report; package.yml RPM build |
windows-sftp-listener-smoke.sh |
dev-tool | Confirms rustfs.exe --features sftp binds an SFTP listener on Windows |
— |
Benchmark & performance harnesses
| Entry | Status | Purpose | Wiring / docs |
|---|---|---|---|
run_hotpath_warp_ab.sh |
dev-tool | Linux warp A/B rig for the hotpath series (quick local A/B) | docs/operations/hotpath-warp-ab-runbook.md |
run_hotpath_warp_abba.sh |
ci-gate | Formal ABBA warp runner (baseline/candidate interleaved, --rounds >= 3); the runner performance-ab.yml executes |
performance-ab.yml (scheduled); docs/operations/hotpath-warp-ab-runbook.md |
hotpath_warp_ab_gate.sh |
dev-tool | Relative-budget gate evaluated over the warp A/B results | used by run_hotpath_warp_ab.sh; hotpath runbook |
run_internode_grpc_ab_bench.sh |
dev-tool | One-click A/B driver for the internode gRPC optimization stages | docs/operations/internode-grpc-benchmark-runbook.md |
run_internode_transport_baseline.sh |
dev-tool | Internode transport baseline runner | internode runbook; crates/io-metrics/README.md |
run_four_node_cluster_failover_bench.sh |
dev-tool | Four-node cluster failover benchmark | docker/compose/README.md; internode runbook |
run_object_batch_bench.sh |
dev-tool | Batch object benchmark runner (warp/s3bench) | internode + scanner runbooks |
run_object_batch_bench_enhanced.sh |
dev-tool | Enhanced batch benchmark runner; hub used by the smoke rigs | hotpath runbook |
run_pinned_paired_abba_bench.sh |
dev-tool | Pinned RustFS/MinIO paired ABBA benchmark orchestrator for backlog#1432 | test_pinned_paired_abba_bench.sh |
run_get_codec_streaming_smoke.sh |
dev-tool | Local GET benchmark harness for the codec streaming read path | docs/testing/ecstore-validation-suite-design.md |
run_get_1mib_abba_stage_metrics.sh |
dev-tool | Exact-1MiB isolated-host GET ABBA/stage-metrics harness for backlog#1434 | test_get_1mib_abba_stage_metrics.sh |
issue_2007_coalescer_prometheus_report.py |
dev-tool | Read-only Prometheus report for GET metadata coalescer delay cost validation; usage in the module docstring | test_issue_2007_coalescer_prometheus_report.sh |
prometheus_metrics_1649_smoke.py |
dev-tool | Read-only Prometheus instant-query smoke check for the backlog#1649 metric dimensions, required labels, and retired series; usage in the module docstring | --self-test |
run_gt1g_get_http_matrix.sh |
dev-tool | >1 GiB GET HTTP matrix | docs/testing/ecstore-validation-suite-design.md |
run_gt1g_multipart_put_matrix.sh |
dev-tool | >1 GiB multipart PUT matrix | docs/testing/ecstore-validation-suite-design.md |
sample_remote_rustfs_rss.sh |
dev-tool | Remote RustFS PID CPU/RSS TSV sampler for hotpath profiling runs | test_sample_remote_rustfs_rss.sh; backlog#1647 |
summarize_samply_profile_symbols.py |
dev-tool | Offline samply profile.json.gz + .syms.json function-level hotpath summarizer |
test_summarize_samply_profile_symbols.py; backlog#1647 |
run_scanner_benchmarks.sh |
dev-tool (disposition pending) | Scanner performance benchmark runner. Contains a hardcoded stale path; disposition owned by backlog perf-10 — do not fix, move, or delete it here | — |
Local development & operations
| Entry | Status | Purpose | Wiring / docs |
|---|---|---|---|
dev_clear.sh |
dev-tool | Local dev cleanup. scripts/dev_*.sh is a CI paths-filter glob — keep the naming |
ci.yml/build.yml paths filters |
dev_deploy.sh |
dev-tool | Copy a built binary to dev servers | make deploy (.config/make/deploy.mak); Justfile |
dev_rustfs.sh |
dev-tool | Local dev run loop | — |
dev_rustfs.env |
dev-tool (data) | Env presets for the dev scripts | — |
restart_local_single_node_multidisk_rustfs.sh |
dev-tool | Restart a local single-node multi-disk instance | — |
inspect_dashboard.sh |
dev-tool | Sanity-checks the Grafana dashboard JSON | .docker/observability |
notify.sh |
dev-tool | Starts a local webhook receiver for notify-target development | — |
manual_transition_debug.sh |
dev-tool | Log/metrics helper for manual transition troubleshooting | — |
manual_transition_journal_audit.sh |
dev-tool | Journal + metrics + log audit for manual transition jobs | — |
manual_transition_mixed_rollout_matrix.sh |
dev-tool | Matrix generator for mixed-version rollout phases | — |
manual_transition_mixed_rollout_runbook.sh |
dev-tool | Reusable mixed-version rollout runbook generator (external run) | — |
manual_transition_mixed_version_docker_harness.sh |
dev-tool | Dedicated #1508 Docker harness for old/new manual-transition rollout evidence with strict/baseline/blocked result classification | test_manual_transition_runbooks.sh |
monitor_manual_transition_ci.sh |
dev-tool | CI workflow/status watcher for manual transition follow-up monitoring | — |
manual_transition_soak_matrix.sh |
dev-tool | Matrix generator for nightly stress windows | — |
manual_transition_nightly_stress_runbook.sh |
dev-tool | Nightly stress entrypoint with failure snapshot templates | — |
install-flatc.sh |
dev-tool | Local flatc installer (macOS) | — |
install-protoc.sh |
dev-tool | Local protoc installer (macOS/Linux) | — |
cargo_publish_workspace.sh |
dev-tool | Generates the workspace crate publish order, checks existing registry versions, and optionally runs ordered cargo publish dry-runs or publication |
docs/operations/cargo-publish-workspace.md |
makefile-header.sh |
dev-tool | Generates the ## —— section —— header lines used in .config/make/*.mak |
— |
tls_gen.md |
dev-tool (doc) | Notes on generating local TLS certificates | — |
Subdirectories
| Entry | Status | Purpose | Wiring / docs |
|---|---|---|---|
fuzz/ |
ci-gate | Unified cargo-fuzz runner and helpers for the fuzz/ sub-workspace |
fuzz.yml; fuzz/README.md |
release/ |
ci-gate | Release creation and DEB/RPM version-normalization helpers | build.yml; package.yml |
s3-tests/ |
ci-gate | ceph/s3-tests compatibility harness (allow-lists, patches, report tooling) | ci.yml; e2e-s3tests.yml; scripts/s3-tests/README.md |
security/ |
ci-gate | Workflow-pin enforcement and release supply-chain asset generation | audit.yml; build.yml |
table-catalog/ |
dev-tool | S3-Tables / pyiceberg validation suite | docs/architecture/s3-tables-support-matrix.md |
test/ |
dev-tool | Manual operational validation runbooks (decommission, tier lifecycle), paired .sh + .md |
— |
archive/ |
archived | Retired one-shot scripts (see below) | — |
Archived (scripts/archive/)
Moved 2026-07 (backlog#1153 infra-13) after a whole-tree reference census: each entry had zero references from CI, Makefiles, docs, or code — or was referenced only by other scripts in this same archived set. Reasons:
| Entry | Was |
|---|---|
validate_issue_785_list_objects.sh |
One-shot issue validation (list-objects series) |
validate_issue_786_list_objects.sh |
One-shot issue validation (list-objects series) |
validate_issue_787_list_quorum.sh |
One-shot issue validation (list-quorum) |
validate_issue_841_list_objects_observability.sh |
One-shot issue validation (list observability) |
validate_issue_1365_docker.sh |
One-shot issue validation (docker repro) |
validate_issue_2723_site_replication.sh |
One-shot issue validation (site replication) |
validate_issue_3031_docker.sh |
One-shot issue validation (docker repro) |
run_issue712_deeper_zero_copy_put_with_capture.sh |
One-shot perf capture for backlog#712 |
run_issue797_local_4node_16disk_ab.sh |
One-shot 4-node/16-disk A/B for backlog#797 |
run_issue_2573_acceptance.sh |
One-shot acceptance run for issue #2573 |
run_issue_2941_perf_capture.sh |
One-shot perf capture for issue #2941 |
run_put_large_stage_breakdown.sh |
backlog#706 large-PUT stage breakdown (family) |
run_put_large_stage_breakdown_with_capture.sh |
backlog#706 one-shot wrapper (family) |
run_put_large_tuning_matrix.sh |
backlog#706 tuning matrix (family) |
collect_put_large_stage_breakdown_artifacts.sh |
backlog#706 artifact collector (family) |
analyze_put_service_metrics_deltas.py |
backlog#706 metrics-delta analyzer (family) |
README-stress-test.md |
GET-optimization one-shot suite doc |
stress-test-get-optimization.sh |
GET-optimization one-shot stress test |
quick-validate-get-optimization.sh |
GET-optimization one-shot validation |
benchmark-sf-optimization.sh |
GET-optimization one-shot benchmark |
prepare_gt1g_get_test_objects.sh |
>1 GiB GET investigation one-shot fixture prep |
run_gt1g_multipart_put_server_path_focus.sh |
>1 GiB PUT investigation one-shot focus run |
run_get_metrics_gate_smoke.sh |
One-shot GET metrics-gate smoke |
run_listobjects_verified_bench.sh |
One-shot verified list-objects bench |
run_object_batch_bench_abc.sh |
One-shot capacity/object profile A/B/C controller |
run_object_data_cache_bench.sh |
One-shot GET bench for the object-data-cache rollout gate |
setup-test-binaries.sh |
One-shot Docker-build test binary fixture |
test.sh |
Ancient manual mc bucket smoke scratchpad |
test_policy.json |
Orphaned IAM policy fixture (hardcoded test bucket) |