ci(perf): cache the main baseline binary to cut the nightly double build (#4816)

Every push to main now builds the release binary once and stores it in the
actions cache as rustfs-baseline-<sha> (build-baseline-cache job). The A/B job
restores it for origin/main and passes --baseline-bin; on the nightly, where
the candidate commit equals the baseline, one cached binary serves both phases
with --skip-build and the run does zero source builds. A cache miss falls back
to the source double-build via --baseline-ref origin/main.

This removes the ~32min-per-side double build that pushed the 24-cell nightly
past its 120min ceiling (2026-07-11..07-14 all cancelled on timeout).

Also:
- alert-on-failure now fires on cancelled/timed-out, not just failure, so a
  timed-out nightly is no longer silent (the composite action already reports
  cancelled jobs); removed the stale perf-2 TODO now that the alert job exists.
- run_hotpath_warp_ab.sh appends a Provenance section to gate.md (baseline and
  candidate SHAs + binary source, runner, warp version, matrix params) via a
  repeatable --provenance-note; the gate exit code is preserved.

Refs rustfs/backlog#1152 (perf-3).
This commit is contained in:
Zhengchao An
2026-07-15 09:33:56 +08:00
committed by GitHub
parent 0e7b8ea16b
commit d73c8a783a
3 changed files with 193 additions and 31 deletions
+19 -3
View File
@@ -37,15 +37,31 @@ Six workloads × the drive-sync on/off matrix × baseline/candidate = 24 cells:
Sizes are passed to the load driver via `--sizes` (one size per cell); the
driver's `DEFAULT_SIZES` covers 1KiB..10MiB, so any of those can be added by
editing `WORKLOADS` in `scripts/run_hotpath_warp_ab.sh`. A 1KiB cell is left
out for now to keep the nightly matrix under the 90-minute job budget (the
baseline+candidate double build dominates until perf-3 caches it); re-enable it
once that lands.
out for now to keep the nightly matrix comfortably under budget; re-enable it
(one line in `WORKLOADS`) once perf-6 recalibrates the warp params.
CI runs a **short** warp matrix (`--duration`/`--rounds`/`--cooldown` tuned in
`.github/workflows/performance-ab.yml`) so all 24 cells fit the budget without
dropping cells. These params are deliberately noisy-but-fast for the Phase-0
"keep the pipeline alive" goal; perf-6 recalibrates them.
## Baseline binary cache (CI)
The nightly no longer builds both binaries from source. Every push to `main`
runs a `build-baseline-cache` job that builds the release binary once and stores
it in the actions cache under `rustfs-baseline-<sha>` (perf-3). The A/B job
restores the binary for `origin/main` by that key and passes it as
`--baseline-bin`; on the nightly, where the candidate commit equals the baseline
commit, the same cached binary serves both phases (`--skip-build`) and the run
does zero source builds — the common path finishes well under 50 minutes. A
cache miss (binary evicted, or not built for that SHA yet) transparently falls
back to the source double-build via `--baseline-ref origin/main`.
Each `gate.md` ends with a **Provenance** section recording the baseline and
candidate commit SHAs and whether each binary came from the cache or a source
build, plus the runner, warp version, and matrix params. `perf-5`/`perf-12`
reuse this contract for their archived baselines.
## Local mode (quick / CI smoke)
Builds both binaries and runs a throwaway single-node server on local disks.