mirror of
https://github.com/rcourtman/Pulse.git
synced 2026-10-23 20:25:56 +00:00
Balance internal/api race shards by measured run time
The four Backend tests (api-N) shards took equal-count quarters of go test's list, which left them at 4.0, 10.6, 4.0 and 16.8 minutes on the first PR run, and at 1.9, 8.0, 1.5 and 10.3 minutes of go test time on the following main push (run 36560070496). Almost all of it sits in a few tests. One contract mock chart test runs about six minutes on the runner, and the 28 integration server tests at the end of the list take about twenty seconds each, so the fourth quarter held nearly half the package. Shards are still contiguous runs of go test's own order, since some tests depend on package state from their neighbours, but the cut points now come from estimated seconds. .github/scripts/select-internal-api-shard.sh weighs each test from .github/scripts/internal-api-test-seconds.txt (the 199 tests at 0.5s or more, seeded from a local -race -json run scaled to that main run's shard totals, with a 0.05s default for anything unlisted), binary searches the smallest per-shard budget that fits the list into four slices, and fills up to it. The weights only move cut points. Every test is assigned exactly one shard in a single pass, so a stale file can unbalance shards but never drop or repeat a test, and no shard is ever empty. With today's list the slices hold 1187, 23, 2556 and 21 tests, estimated at 1.9, 6.9, 6.5 and 6.7 minutes of tests, or about 9.5 minutes per job with the roughly 2.6 minutes of setup and race compile. The 2556-test slice would need a 124 KB -run pattern, so a shard passes -skip of the other shards' tests whenever that is shorter, which runs the same tests in the same order.
This commit is contained in:
@@ -323,9 +323,9 @@ jobs:
|
||||
# internal/api alone took ~27 of the ~31 minutes of the old single
|
||||
# `Backend tests (api)` job under -race, which made it the critical path of
|
||||
# every pull request. Its top-level tests are split across these shards
|
||||
# as contiguous runs of the package's own test order, and the
|
||||
# `Backend tests (api)` job below keeps the required check name by passing
|
||||
# only when every shard passed.
|
||||
# as contiguous runs of the package's own test order, cut by estimated run
|
||||
# time, and the `Backend tests (api)` job below keeps the required check
|
||||
# name by passing only when every shard passed.
|
||||
backend-api:
|
||||
name: Backend tests (api-${{ matrix.index }})
|
||||
needs: changes
|
||||
@@ -379,9 +379,15 @@ jobs:
|
||||
# tests still lean on package state left by the tests just before
|
||||
# them (for example admin bypass or session store globals), and an
|
||||
# interleaved split by name broke dozens of them, while contiguous
|
||||
# slices keep those neighbours together. Listing with -race builds
|
||||
# the same test binary the run then reuses from cache. Benchmarks
|
||||
# are left out because plain go test never runs them.
|
||||
# slices keep those neighbours together. The slices are cut by
|
||||
# estimated run time from .github/scripts/internal-api-test-seconds.txt
|
||||
# rather than by count, because a handful of tests take most of the
|
||||
# time (one contract test runs about six minutes and the integration
|
||||
# server tail about twenty seconds per test). Unlisted tests get a
|
||||
# small default weight, and the weights only move the cut points, so
|
||||
# coverage never depends on them. Listing with -race builds the same
|
||||
# test binary the run then reuses from cache. Benchmarks are left out
|
||||
# because plain go test never runs them.
|
||||
run: |
|
||||
set -euo pipefail
|
||||
listing=$(go test -race -list . ./internal/api)
|
||||
@@ -391,21 +397,33 @@ jobs:
|
||||
exit 1
|
||||
fi
|
||||
total=$(printf '%s\n' "$tests" | wc -l | tr -d ' ')
|
||||
selected=$(printf '%s\n' "$tests" | awk -v n="$API_SHARD_COUNT" -v i="$API_SHARD_INDEX" -v total="$total" \
|
||||
'NR > int(total * i / n) && NR <= int(total * (i + 1) / n)')
|
||||
selected=$(printf '%s\n' "$tests" | bash .github/scripts/select-internal-api-shard.sh \
|
||||
.github/scripts/internal-api-test-seconds.txt "$API_SHARD_COUNT" "$API_SHARD_INDEX")
|
||||
if [ -z "$selected" ]; then
|
||||
echo "::error::internal/api shard ${API_SHARD_INDEX} of ${API_SHARD_COUNT} resolved to an empty test list"
|
||||
exit 1
|
||||
fi
|
||||
others=$(printf '%s\n' "$tests" | grep -vxF -f <(printf '%s\n' "$selected") || true)
|
||||
# Name whichever side is shorter. A shard holding most of the tests
|
||||
# is passed as -skip of every other shard's tests, which runs the
|
||||
# same set in the same order.
|
||||
pattern="^($(printf '%s\n' "$selected" | paste -sd '|' -))\$"
|
||||
filter=(-run "$pattern")
|
||||
if [ -n "$others" ]; then
|
||||
skip="^($(printf '%s\n' "$others" | paste -sd '|' -))\$"
|
||||
if [ "${#skip}" -lt "${#pattern}" ]; then
|
||||
pattern=$skip
|
||||
filter=(-run . -skip "$pattern")
|
||||
fi
|
||||
fi
|
||||
# Linux rejects a single argument over 128 KiB. Fail with a clear
|
||||
# instruction instead of an opaque exec error.
|
||||
if [ "${#pattern}" -gt 120000 ]; then
|
||||
echo "::error::internal/api shard pattern is ${#pattern} bytes; add a shard index and raise API_SHARD_COUNT"
|
||||
exit 1
|
||||
fi
|
||||
echo "internal/api shard ${API_SHARD_INDEX} of ${API_SHARD_COUNT} runs $(printf '%s\n' "$selected" | wc -l | tr -d ' ') of ${total} tests"
|
||||
go test -race -timeout 50m -run "$pattern" ./internal/api
|
||||
echo "internal/api shard ${API_SHARD_INDEX} of ${API_SHARD_COUNT} runs $(printf '%s\n' "$selected" | wc -l | tr -d ' ') of ${total} tests (${selected%%$'\n'*} .. ${selected##*$'\n'})"
|
||||
go test -race -timeout 50m "${filter[@]}" ./internal/api
|
||||
|
||||
# Required check name. Branch protection matches `Backend tests (api)`
|
||||
# exactly, so this job must keep that name and must fail unless every
|
||||
|
||||
Reference in New Issue
Block a user