Keep paired PR benchmarks inside their time budget

Since 15:00 UTC on 5 October every paired benchmark run that was not
superseded has hit its 25-minute step timeout. Each step runs ten base
and ten candidate samples, so per-sample growth is multiplied by twenty.

BenchmarkRegistryProjectionSnapshot (#2518) rebuilt a registry behind
StopTimer on every cold iteration. Go sizes b.N from timed work only, so
the empty-registry cases ran hundreds of thousands of untimed ingests
and took 16 s of a 100 ms benchtime. With #2486's new owner-index and
metadata benchmarks, internal/unifiedresources went from 8 s to 38 s per
sample. Cold now invalidates the registry's derived state instead. The
refresh never reads its own output, so this repeats a fresh ingest's
first read with the same allocations, and the family drops from 22 s to
about 5 s per sample.

Paired runs also measured every package even when both trees compiled
identical code, as on frontend-only changes. They now sample only
packages whose in-module dependency closure the change reaches. Module
files, dirty or non-Git trees and an unreadable graph still select
everything.

A module or shared-model change still samples nearly every package,
about 24 minutes with warm-up. The step therefore gets a measured 30
minutes, and the script stops at a 28-minute projected budget with
per-package costs instead of meeting a bare timeout.
This commit is contained in:
rcourtman
2026-10-05 22:39:28 +01:00
parent 1dcf9418aa
commit b027bb0e8e
7 changed files with 335 additions and 23 deletions
+7 -1
View File
@@ -546,12 +546,18 @@ jobs:
PULSE_BENCH_SAMPLE_COUNT: "10"
run: bash scripts/run-ci-benchmarks.sh
# Only packages whose dependency closure the pull request changes are
# sampled. A module or shared-model change still samples nearly every
# package: about 24 minutes, warm-up included, as of October 2026. The
# script's budget stops the run with per-package costs before the step
# timeout can kill it without them.
- name: Paired base and candidate benchmarks (pull requests)
if: github.event_name == 'pull_request'
timeout-minutes: 25
timeout-minutes: 30
env:
PULSE_BENCH_BASELINE_DIR: ${{ github.workspace }}/benchmark-base
PULSE_BENCH_SAMPLE_COUNT: "10"
PULSE_BENCH_BUDGET_SECONDS: "1680"
run: bash scripts/run-ci-benchmarks.sh
- name: Install benchstat