mirror of
https://github.com/rcourtman/Pulse.git
synced 2026-10-23 20:25:56 +00:00
Keep paired PR benchmarks inside their time budget
Since 15:00 UTC on 5 October every paired benchmark run that was not superseded has hit its 25-minute step timeout. Each step runs ten base and ten candidate samples, so per-sample growth is multiplied by twenty. BenchmarkRegistryProjectionSnapshot (#2518) rebuilt a registry behind StopTimer on every cold iteration. Go sizes b.N from timed work only, so the empty-registry cases ran hundreds of thousands of untimed ingests and took 16 s of a 100 ms benchtime. With #2486's new owner-index and metadata benchmarks, internal/unifiedresources went from 8 s to 38 s per sample. Cold now invalidates the registry's derived state instead. The refresh never reads its own output, so this repeats a fresh ingest's first read with the same allocations, and the family drops from 22 s to about 5 s per sample. Paired runs also measured every package even when both trees compiled identical code, as on frontend-only changes. They now sample only packages whose in-module dependency closure the change reaches. Module files, dirty or non-Git trees and an unreadable graph still select everything. A module or shared-model change still samples nearly every package, about 24 minutes with warm-up. The step therefore gets a measured 30 minutes, and the script stops at a 28-minute projected budget with per-package costs instead of meeting a bare timeout.
This commit is contained in:
@@ -546,12 +546,18 @@ jobs:
|
||||
PULSE_BENCH_SAMPLE_COUNT: "10"
|
||||
run: bash scripts/run-ci-benchmarks.sh
|
||||
|
||||
# Only packages whose dependency closure the pull request changes are
|
||||
# sampled. A module or shared-model change still samples nearly every
|
||||
# package: about 24 minutes, warm-up included, as of October 2026. The
|
||||
# script's budget stops the run with per-package costs before the step
|
||||
# timeout can kill it without them.
|
||||
- name: Paired base and candidate benchmarks (pull requests)
|
||||
if: github.event_name == 'pull_request'
|
||||
timeout-minutes: 25
|
||||
timeout-minutes: 30
|
||||
env:
|
||||
PULSE_BENCH_BASELINE_DIR: ${{ github.workspace }}/benchmark-base
|
||||
PULSE_BENCH_SAMPLE_COUNT: "10"
|
||||
PULSE_BENCH_BUDGET_SECONDS: "1680"
|
||||
run: bash scripts/run-ci-benchmarks.sh
|
||||
|
||||
- name: Install benchstat
|
||||
|
||||
Reference in New Issue
Block a user