fix(ci): scope e2e telemetry per cell/job and define suite topology eligibility once (#573)

* feat(ci): extend per-cell telemetry scoping to every e2e workflow

ci.cell only reached e2e-nightly and e2e-suites, so a run of e2e, e2e-ddil
or gpu-e2e still landed in the sink as one undifferentiated bucket. Stamp it
on the remaining workflows: single/multi on e2e's two provisioning jobs, ddil
and gpu on their single-job workflows.

The best-effort numeric job id lookup moves out of e2e-nightly into a shared
composite action, so the four other provisioning jobs get ci.job_id without
copying the gh/jq incantation. The lookup now matches on env.JOB_NAME rather
than interpolating the name into the jq filter, which broke on names carrying
quotes or non-ASCII separators, and no-ops when telemetry is disabled.

gpu-e2e gets ci.cell but deliberately no job id lookup: nothing in that job
bakes the attrs into a daemon, so the API call would be paid for an env var
no exporter reads.

* fix(ci): define e2e suite topology eligibility once

Three workflows carried three different answers to which suites a topology
can run. e2e-suites and e2e-nightly ran lb on their single-node leg, where
every internet-facing subtest skips for want of a peer, so a fraction of the
suite reported as a green lb run. e2e-suites' multi leg ran multinode alone,
dropping the lb and cert coverage that only a multi-node env can complete —
so full lb coverage never ran there at all.

.github/scripts/suite-sets.sh is now the single definition, sourced by all
three. e2e.yml's two allow-list regexes derive from it, e2e-suites derives
both matrices from it (multi_provision gains the suite-matrix step
single_provision already had), and e2e-nightly's deliberately narrower
per-cell lists are declared there too, guarded by an assert that fails the
cell if a subset ever names a suite its topology cannot fully exercise.

The nightly stays narrow on purpose: its cells have a 35-minute budget and
exist to prove each install/network/host-OS permutation boots and serves.
Suite breadth is the PR gate's job and already runs on every push. That is
now stated in one place rather than implied by a third divergent list.
This commit is contained in:
Josh Siviour
2026-07-22 17:18:55 +10:00
committed by GitHub
parent cae76ba1b6
commit 68abc0bfb3
7 changed files with 260 additions and 49 deletions
+58
View File
@@ -0,0 +1,58 @@
# shellcheck shell=bash
# shellcheck disable=SC2034 # every variable here is consumed by the sourcing workflow
# Source from a workflow `run:` block:
# . "${GITHUB_WORKSPACE}/spinifex/.github/scripts/suite-sets.sh"
#
# Which e2e suites each topology can run. Sourced by every workflow that drives
# suites (e2e.yml, e2e-suites.yml, e2e-nightly.yml) so the lists cannot drift
# apart — they previously disagreed three ways.
#
# Eligibility is a property of the suite, not of the workflow: a suite belongs
# to a topology when all of its assertions are reachable there. lb is
# multi-only for exactly that reason — on a single node every internet-facing
# subtest skips for want of a peer, so a single-node lb run reports green
# having exercised a fraction of the suite.
# Suites runnable against a one-node environment.
#
# partialblock gates merges because it earned it, not because it is new: on one
# node, same binary and workload, it fails against the pre-fix engine (1758 of
# 8192 halves reverted a generation) and passes against the fixed one (0 of
# 8192). A suite that has never been shown to go red for the reason it claims
# does not belong here.
E2E_SUITES_SINGLE="single iam cert eks ecs storagegrowth partialblock"
# Suites runnable against a multi-node environment.
E2E_SUITES_MULTI="multinode lb cert"
# grep -xE alternation form, for narrowing a requested set to the eligible one.
E2E_SUITES_SINGLE_RE="$(printf '%s' "${E2E_SUITES_SINGLE}" | tr ' ' '|')"
E2E_SUITES_MULTI_RE="$(printf '%s' "${E2E_SUITES_MULTI}" | tr ' ' '|')"
# e2e-nightly runs a narrower set than the two sets above, deliberately: its
# ~19 cells each have a 35-minute budget and exist to prove that every
# install / network / host-OS permutation boots and serves. Suite breadth is
# the PR gate's job, and it already runs on every push. These are subsets, and
# e2e_suites_assert_subset below enforces that.
E2E_SUITES_NIGHTLY_SINGLE="single cert"
E2E_SUITES_NIGHTLY_MULTI="multinode cert lb"
# Fail loudly when a narrowed list names a suite its topology cannot run. This
# is the guard that stops the lists drifting again: a suite added to a nightly
# subset without also being declared eligible above stops the job here rather
# than silently skipping half its assertions in CI.
# e2e_suites_assert_subset <label> <subset> <eligible-set>
e2e_suites_assert_subset() {
local label="$1" subset="$2" eligible="$3" suite found eligible_suite
for suite in ${subset}; do
found=0
for eligible_suite in ${eligible}; do
if [ "${suite}" = "${eligible_suite}" ]; then found=1; break; fi
done
if [ "${found}" -ne 1 ]; then
echo "::error::${label} names suite '${suite}', which is not eligible for that topology (eligible: ${eligible}). Fix the list in .github/scripts/suite-sets.sh." >&2
return 1
fi
done
return 0
}