Replace September timing weights with median observations from all five API shards of main runs 37554828003 and 37548993373. Keep the five checks, source-derived exhaustive coverage, contiguous test order, race execution and thresholds unchanged. Two new report tests have only the latest observation.
Contract-Neutral: Timing-only calibration; no product, test-selection, required-check or verification-contract delta.
Change-source: pulse-maintainer
The weights from #2324 came from one local run and left the api shards at
4.5, 8.8, 17.1 and 12.5 minutes on CI. Most of the 17.1 was not test time.
That shard ran `-run . -skip <1231 names>`, and the test binary caches only
its last compiled pattern, so alternating the two recompiled the 62 KB skip
regex for every test and subtest (about 50 ms each under -race locally, 30s
against 1.3s for 300 trivial tests). Two runs put that shard at 887s and
875s against roughly 450s of tests. The skip path now passes -skip alone.
Every api shard now runs go test -json through
.github/scripts/record-internal-api-test-seconds.py. It prints what plain go
test would (package lines and the output of failing or unfinished tests),
exits non-zero on any failure behind pipefail, and writes each top-level
test's seconds to an internal-api-test-seconds-N artifact that is uploaded
even when the shard fails. .github/scripts/refresh-internal-api-test-seconds.py
rebuilds the weights file from those artifacts (gh run download, median
across runs, DEFAULT_WEIGHT set to the mean of the unlisted tests, which the
selector now reads from the file).
Until those artifacts exist, the weights are a local -race -json run with
each region of go test's order scaled to what CI reported for it.
Equal-count quarters averaged 107, 437, 100 and 787s over four runs, and the
weighted cut gave 131s for the first 1187 tests, 350s for the next 23 and
608s for the last 21 over two. The last quarter and the last 21 tests
disagree on how the tail splits, so the tail takes the midpoint of an exact
fit and a pooled factor.
With these weights four shards still predict a slowest shard near 10
minutes, and a 20% slower runner on the tail would take it past the 11
minutes of Frontend and rest-1. Five shards predict about 131, 325, 280-340,
330-355 and 330-365s of tests, or 4.5 to 8.5 minutes a job with about 2.3
minutes of setup, so API_SHARD_COUNT goes to 5. Every test still runs
exactly once in contiguous go test order, and all five slices passed locally
from their new starting points. The required Backend tests (api) verdict
keeps its name.
The four Backend tests (api-N) shards took equal-count quarters of go
test's list, which left them at 4.0, 10.6, 4.0 and 16.8 minutes on the
first PR run, and at 1.9, 8.0, 1.5 and 10.3 minutes of go test time on
the following main push (run 36560070496). Almost all of it sits in a
few tests. One contract mock chart test runs about six minutes on the
runner, and the 28 integration server tests at the end of the list take
about twenty seconds each, so the fourth quarter held nearly half the
package.
Shards are still contiguous runs of go test's own order, since some
tests depend on package state from their neighbours, but the cut points
now come from estimated seconds.
.github/scripts/select-internal-api-shard.sh weighs each test from
.github/scripts/internal-api-test-seconds.txt (the 199 tests at 0.5s or
more, seeded from a local -race -json run scaled to that main run's
shard totals, with a 0.05s default for anything unlisted), binary
searches the smallest per-shard budget that fits the list into four
slices, and fills up to it. The weights only move cut points. Every test
is assigned exactly one shard in a single pass, so a stale file can
unbalance shards but never drop or repeat a test, and no shard is ever
empty.
With today's list the slices hold 1187, 23, 2556 and 21 tests, estimated
at 1.9, 6.9, 6.5 and 6.7 minutes of tests, or about 9.5 minutes per job
with the roughly 2.6 minutes of setup and race compile. The 2556-test
slice would need a 124 KB -run pattern, so a shard passes -skip of the
other shards' tests whenever that is shorter, which runs the same tests
in the same order.
pulse-triage[bot]