ci: render JUnit into job summary + collapsible groups in noisy steps

Add a reusable composite action (.github/actions/junit-report) that parses
a glob of JUnit XML files, writes a totals table + collapsible per-failure
details into $GITHUB_STEP_SUMMARY, and emits one ::error annotation per
failed/errored test case so failures surface in the run header instead of
being buried in step logs.

Apply the action to both e2e-go.yml and e2e-ddil.yml (e2e-ddil gotestsum
now also emits --junitfile so the action has input).

Same workflows also wrap noisy phases (tofu apply/destroy, scrub-env,
tar-over-ssh, per-suite test runs, cluster-log collection) in
::group::/::endgroup:: markers so the Actions UI collapses the chatter
by default and the human can expand only the failing phase. Tofu output
still tees to $TOFU_LOG so the artifact bundle is unchanged.

Future workflows (multinode, baremetal) can adopt the same pattern via
the README in .github/actions/junit-report.
This commit is contained in:
joshsiv-mulga
2026-05-18 16:44:17 +10:00
parent e1c164d663
commit 1a494fed58
6 changed files with 385 additions and 45 deletions
+1
View File
@@ -0,0 +1 @@
__pycache__/
+46
View File
@@ -0,0 +1,46 @@
# junit-report
Composite action that renders a glob of JUnit XML files into the GitHub
Actions job summary and emits one `::error` annotation per failed test case.
## Usage
```yaml
- name: Publish test summary
if: always()
uses: ./spinifex/.github/actions/junit-report # or your own checkout path
with:
junit-glob: ${{ env.ARTIFACT_DIR }}/junit-*.xml
title: "Go E2E results"
# fail-on-empty: "true" # opt-in: fail the step if the glob matches nothing
```
Outputs:
- `failures` — total `<failure>` + `<error>` counts across all matched files.
- `total` — total `<testcase>` count across all matched files.
The action expects standard JUnit XML (e.g. emitted by
`gotestsum --junitfile`, `go-junit-report`, or pytest's `--junitxml`). Both
single `<testsuite>` and wrapping `<testsuites>` roots are accepted.
## Companion convention: `::group::` for noisy steps
Long-running shell phases (tofu apply, package installs, tar-over-ssh) should
wrap their output in `::group::name` / `::endgroup::` markers so the Actions
UI collapses them by default. Pattern used in the e2e-go / e2e-ddil workflows:
```bash
run() {
local label="$1"; shift
echo "::group::${label}"
"$@" 2>&1 | tee -a "$TOFU_LOG"
local rc=${PIPESTATUS[0]}
echo "::endgroup::"
return "$rc"
}
run "tofu apply" tofu apply -var-file=... -auto-approve -input=false
```
Combined: groups make the live UI readable; this action makes the run header
land on the failures without scrolling the log.
+41
View File
@@ -0,0 +1,41 @@
name: JUnit Report
description: >-
Render JUnit XML files into the GitHub Actions job summary and emit
inline ::error annotations for each failed test case. Designed to be
called once per job after the test step (with `if: always()`) so summary
+ annotations appear on both passing and failing runs.
inputs:
junit-glob:
description: Shell glob matching JUnit XML files to render.
required: true
title:
description: Heading rendered at the top of the job summary section.
required: false
default: "Test results"
fail-on-empty:
description: >-
When "true", the action fails if the glob matches no files. Useful
for catching silently-skipped test phases.
required: false
default: "false"
outputs:
failures:
description: Total failed + errored test cases across all matched files.
value: ${{ steps.render.outputs.failures }}
total:
description: Total test cases across all matched files.
value: ${{ steps.render.outputs.total }}
runs:
using: composite
steps:
- id: render
shell: bash
env:
JUNIT_GLOB: ${{ inputs.junit-glob }}
JUNIT_TITLE: ${{ inputs.title }}
JUNIT_FAIL_ON_EMPTY: ${{ inputs.fail-on-empty }}
run: |
python3 "${{ github.action_path }}/render.py"
+197
View File
@@ -0,0 +1,197 @@
#!/usr/bin/env python3
"""Render JUnit XML files into GitHub Actions job summary + annotations.
Inputs (env):
JUNIT_GLOB Glob matching JUnit XML files.
JUNIT_TITLE Heading text for the summary section.
JUNIT_FAIL_ON_EMPTY "true" to exit non-zero when glob matches nothing.
GitHub-Actions outputs (via $GITHUB_OUTPUT):
failures Sum of <failures> + <errors> across all suites.
total Sum of <tests> across all suites.
Side effects:
Appends a markdown section to $GITHUB_STEP_SUMMARY with a per-file
totals table and a collapsible details block per failed test.
Emits one `::error` workflow command per failed/errored test case so
they surface as annotations in the run UI.
"""
from __future__ import annotations
import glob
import os
import sys
import xml.etree.ElementTree as ET
from dataclasses import dataclass
# Workflow commands carry their payload on one line; literal newlines must
# be percent-escaped per GitHub's workflow-command spec.
def _esc(s: str) -> str:
return (
s.replace("%", "%25")
.replace("\r", "%0D")
.replace("\n", "%0A")
)
@dataclass
class Case:
suite: str
classname: str
name: str
time: float
status: str # "pass" | "fail" | "error" | "skip"
message: str
detail: str
@dataclass
class FileTotals:
path: str
tests: int
failures: int
errors: int
skipped: int
time: float
cases: list[Case]
def _parse_file(path: str) -> FileTotals:
root = ET.parse(path).getroot()
# JUnit XML has two shapes: <testsuite> as root, or <testsuites> wrapping
# multiple <testsuite> children. Iterating with .iter() handles both.
suites = list(root.iter("testsuite"))
tests = sum(int(s.get("tests", 0) or 0) for s in suites)
failures = sum(int(s.get("failures", 0) or 0) for s in suites)
errors = sum(int(s.get("errors", 0) or 0) for s in suites)
skipped = sum(int(s.get("skipped", 0) or 0) for s in suites)
time = sum(float(s.get("time", 0) or 0) for s in suites)
cases: list[Case] = []
for s in suites:
suite_name = s.get("name", "")
for tc in s.findall("testcase"):
status = "pass"
message = ""
detail = ""
for tag in ("failure", "error"):
node = tc.find(tag)
if node is not None:
status = "fail" if tag == "failure" else "error"
message = (node.get("message") or "").strip()
detail = (node.text or "").strip()
break
if tc.find("skipped") is not None and status == "pass":
status = "skip"
cases.append(
Case(
suite=suite_name,
classname=tc.get("classname", ""),
name=tc.get("name", ""),
time=float(tc.get("time", 0) or 0),
status=status,
message=message,
detail=detail,
)
)
return FileTotals(
path=path,
tests=tests,
failures=failures,
errors=errors,
skipped=skipped,
time=time,
cases=cases,
)
def _file_label(path: str) -> str:
base = os.path.basename(path)
if base.endswith(".xml"):
base = base[:-4]
if base.startswith("junit-"):
base = base[len("junit-"):]
return base or path
def main() -> int:
pattern = os.environ.get("JUNIT_GLOB", "").strip()
title = os.environ.get("JUNIT_TITLE", "Test results").strip() or "Test results"
fail_on_empty = os.environ.get("JUNIT_FAIL_ON_EMPTY", "").strip().lower() == "true"
if not pattern:
print("::error::junit-report: junit-glob input is empty", file=sys.stderr)
return 1
paths = sorted(glob.glob(pattern))
summary_path = os.environ.get("GITHUB_STEP_SUMMARY", "/dev/null")
output_path = os.environ.get("GITHUB_OUTPUT", "/dev/null")
with open(summary_path, "a", encoding="utf-8") as summary, \
open(output_path, "a", encoding="utf-8") as outputs:
summary.write(f"## {title}\n\n")
if not paths:
summary.write(f"_No JUnit files matched `{pattern}`._\n\n")
outputs.write("failures=0\n")
outputs.write("total=0\n")
if fail_on_empty:
print(f"::error::junit-report: no files matched {pattern}", file=sys.stderr)
return 1
return 0
totals_by_file = [_parse_file(p) for p in paths]
grand_total = sum(f.tests for f in totals_by_file)
grand_fail = sum(f.failures + f.errors for f in totals_by_file)
grand_skip = sum(f.skipped for f in totals_by_file)
grand_time = sum(f.time for f in totals_by_file)
verdict = ":white_check_mark: pass" if grand_fail == 0 else f":x: {grand_fail} failed"
summary.write(
f"**{verdict}** &middot; {grand_total} tests &middot; "
f"{grand_skip} skipped &middot; {grand_time:.1f}s total\n\n"
)
summary.write("| Suite | Tests | Pass | Fail | Err | Skip | Time |\n")
summary.write("|---|---:|---:|---:|---:|---:|---:|\n")
for f in totals_by_file:
passed = f.tests - f.failures - f.errors - f.skipped
icon = ":white_check_mark:" if (f.failures + f.errors) == 0 else ":x:"
summary.write(
f"| {icon} `{_file_label(f.path)}` | {f.tests} | {passed} | "
f"{f.failures} | {f.errors} | {f.skipped} | {f.time:.1f}s |\n"
)
summary.write("\n")
failed_cases = [
c for f in totals_by_file for c in f.cases if c.status in ("fail", "error")
]
if failed_cases:
summary.write("### Failures\n\n")
for c in failed_cases:
header = f"{c.classname}.{c.name}" if c.classname else c.name
summary.write(f"<details><summary><code>{header}</code></summary>\n\n")
if c.message:
summary.write(f"**{c.message}**\n\n")
if c.detail:
summary.write("```\n" + c.detail + "\n```\n\n")
summary.write("</details>\n\n")
# Workflow annotation — surfaces in run header + Files-changed view.
# classname for Go tests is the package path; map to a file when we can.
file_hint = c.classname.replace(".", "/")
msg = c.message or f"{c.status}: {header}"
print(f"::error file={file_hint},title={header}::{_esc(msg)}")
outputs.write(f"failures={grand_fail}\n")
outputs.write(f"total={grand_total}\n")
return 0
if __name__ == "__main__":
sys.exit(main())
+49 -11
View File
@@ -87,12 +87,24 @@ jobs:
- name: Provision VMs
working-directory: mulga/scripts/tofu-cluster
run: |
set -euo pipefail
export TF_VAR_git_branch="${GIT_BRANCH}"
tofu init -input=false >> "$TOFU_LOG" 2>&1
tofu workspace select -or-create "${SPINIFEX_CI_ENV}" >> "$TOFU_LOG" 2>&1
tofu destroy -var-file=envs/${SPINIFEX_CI_ENV}.tfvars -auto-approve -input=false >> "$TOFU_LOG" 2>&1 || true
./scrub-env.sh "${SPINIFEX_CI_ENV}" >> "$TOFU_LOG" 2>&1 || true
tofu apply -var-file=envs/${SPINIFEX_CI_ENV}.tfvars -auto-approve -input=false >> "$TOFU_LOG" 2>&1
# Each phase wrapped in ::group:: so the Actions UI collapses the
# noisy tofu output by default but the human can expand the
# failing phase without downloading $TOFU_LOG.
run() {
local label="$1"; shift
echo "::group::${label}"
"$@" 2>&1 | tee -a "$TOFU_LOG"
local rc=${PIPESTATUS[0]}
echo "::endgroup::"
return "$rc"
}
run "tofu init" tofu init -input=false
run "tofu workspace select" tofu workspace select -or-create "${SPINIFEX_CI_ENV}"
run "tofu destroy (pre-clean)" tofu destroy -var-file=envs/${SPINIFEX_CI_ENV}.tfvars -auto-approve -input=false || true
run "scrub-env (pre-clean)" ./scrub-env.sh "${SPINIFEX_CI_ENV}" || true
run "tofu apply" tofu apply -var-file=envs/${SPINIFEX_CI_ENV}.tfvars -auto-approve -input=false
- name: Bootstrap via Binary Install
working-directory: mulga/scripts/tofu-cluster
@@ -147,8 +159,11 @@ jobs:
mkdir -p "${RUNNER_TEMP}/ddil-results"
cd "${GITHUB_WORKSPACE}/spinifex"
# gotestsum: human-readable PASS/FAIL/SKIP lines in Actions stdout
# while still emitting the machine-readable jsonfile artifact.
# gotestsum: human-readable PASS/FAIL/SKIP lines in Actions stdout,
# plus machine-readable jsonfile + JUnit XML artifacts. The JUnit
# file feeds the junit-report composite action below so failures
# become summary table rows + workflow annotations.
echo "::group::gotestsum scenarios"
DDIL_NODES="${NODE_IPS}" \
DDIL_SSH_USER="${SSH_USER}" \
DDIL_SSH_KEY="${SSH_KEY}" \
@@ -158,8 +173,19 @@ jobs:
AWS_ACCESS_KEY_ID="${AWS_ACCESS_KEY_ID}" \
AWS_SECRET_ACCESS_KEY="${AWS_SECRET_ACCESS_KEY}" \
gotestsum --jsonfile "${RUNNER_TEMP}/ddil-results/results.json" \
--junitfile "${RUNNER_TEMP}/ddil-results/junit-ddil.xml" \
--format testname \
-- -tags=e2e -timeout 45m ./tests/e2e/ddil/scenarios/...
rc=$?
echo "::endgroup::"
exit $rc
- name: Publish test summary
if: always()
uses: ./spinifex/.github/actions/junit-report
with:
junit-glob: ${{ runner.temp }}/ddil-results/junit-*.xml
title: "DDIL scenario results"
- name: Collect Cluster Logs
if: failure()
@@ -179,11 +205,14 @@ jobs:
mapfile -t NODE_IPS < <(echo "$INVENTORY" | jq -r '.nodes[].wan_ip')
for ip in "${NODE_IPS[@]}"; do
echo "=== collecting ${ip} ==="
echo "::group::collect ${ip}"
SSH="ssh -i ${SSH_KEY} -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null ${SSH_USER}@${ip}"
${SSH} 'sudo journalctl -u "spinifex-*" --no-pager -n 5000' > "${LOG_DIR}/${ip}-spinifex.log" 2>&1
${SSH} 'ps auxw | grep -E "qemu-system|spinifex" | grep -v grep' > "${LOG_DIR}/${ip}-procs.txt" 2>&1
${SSH} 'ls -la /var/lib/spinifex 2>/dev/null; ls -la /var/log/spinifex 2>/dev/null' > "${LOG_DIR}/${ip}-state.txt" 2>&1
echo " spinifex.log: $(wc -l < "${LOG_DIR}/${ip}-spinifex.log") lines"
echo " procs.txt: $(wc -l < "${LOG_DIR}/${ip}-procs.txt") lines"
echo "::endgroup::"
done
- name: Upload DDIL Results
@@ -198,6 +227,15 @@ jobs:
if: always()
working-directory: mulga/scripts/tofu-cluster
run: |
tofu workspace select -or-create "${SPINIFEX_CI_ENV}" >> "$TOFU_LOG" 2>&1
tofu destroy -var-file=envs/${SPINIFEX_CI_ENV}.tfvars -auto-approve -input=false >> "$TOFU_LOG" 2>&1
./scrub-env.sh "${SPINIFEX_CI_ENV}" >> "$TOFU_LOG" 2>&1 || true
set -euo pipefail
run() {
local label="$1"; shift
echo "::group::${label}"
"$@" 2>&1 | tee -a "$TOFU_LOG"
local rc=${PIPESTATUS[0]}
echo "::endgroup::"
return "$rc"
}
run "tofu workspace select" tofu workspace select -or-create "${SPINIFEX_CI_ENV}"
run "tofu destroy" tofu destroy -var-file=envs/${SPINIFEX_CI_ENV}.tfvars -auto-approve -input=false
run "scrub-env" ./scrub-env.sh "${SPINIFEX_CI_ENV}" || true
+51 -34
View File
@@ -84,12 +84,24 @@ jobs:
- name: Provision VM
working-directory: mulga/scripts/tofu-cluster
run: |
set -euo pipefail
export TF_VAR_git_branch="${GIT_BRANCH}"
tofu init -input=false >> "$TOFU_LOG" 2>&1
tofu workspace select -or-create "${SPINIFEX_CI_ENV}" >> "$TOFU_LOG" 2>&1
tofu destroy -var-file=envs/${SPINIFEX_CI_ENV}.tfvars -auto-approve -input=false >> "$TOFU_LOG" 2>&1 || true
./scrub-env.sh "${SPINIFEX_CI_ENV}" >> "$TOFU_LOG" 2>&1 || true
tofu apply -var-file=envs/${SPINIFEX_CI_ENV}.tfvars -auto-approve -input=false >> "$TOFU_LOG" 2>&1
# Each phase wrapped in ::group:: so the Actions UI collapses
# the noisy tofu output by default but the human can expand the
# failing phase without downloading $TOFU_LOG.
run() {
local label="$1"; shift
echo "::group::${label}"
"$@" 2>&1 | tee -a "$TOFU_LOG"
local rc=${PIPESTATUS[0]}
echo "::endgroup::"
return "$rc"
}
run "tofu init" tofu init -input=false
run "tofu workspace select" tofu workspace select -or-create "${SPINIFEX_CI_ENV}"
run "tofu destroy (pre-clean)" tofu destroy -var-file=envs/${SPINIFEX_CI_ENV}.tfvars -auto-approve -input=false || true
run "scrub-env (pre-clean)" ./scrub-env.sh "${SPINIFEX_CI_ENV}" || true
run "tofu apply" tofu apply -var-file=envs/${SPINIFEX_CI_ENV}.tfvars -auto-approve -input=false
- name: Bootstrap via Binary Install
working-directory: mulga/scripts/tofu-cluster
@@ -114,11 +126,14 @@ jobs:
- name: Build E2E test binaries on runner
working-directory: spinifex/tests/e2e
run: |
set -euo pipefail
# SUITES is in make-target form (e.g. "e2e-cert e2e-lb"); strip the
# "e2e-" prefix to get package directory names for `go test -c`.
SUITES_BUILD=$(echo "${SUITES}" | sed 's/e2e-//g')
echo "::group::go test -c (build ${SUITES_BUILD// /, })"
make e2e-build SUITES_BUILD="${SUITES_BUILD}"
ls -lh _bin/
echo "::endgroup::"
- name: Install go-junit-report on runner
run: |
@@ -129,6 +144,8 @@ jobs:
- name: Ship test binaries to VM
run: |
set -euo pipefail
echo "::group::tar-over-ssh: prep remote dir + stream binaries"
ssh -o StrictHostKeyChecking=no -i "${{ steps.vm.outputs.ssh_key }}" \
tf-user@${{ steps.vm.outputs.wan_ip }} \
"rm -rf \$HOME/spinifex-e2e-bin && mkdir -p \$HOME/spinifex-e2e-bin"
@@ -136,6 +153,7 @@ jobs:
| ssh -o StrictHostKeyChecking=no -i "${{ steps.vm.outputs.ssh_key }}" \
tf-user@${{ steps.vm.outputs.wan_ip }} \
"tar -xf - -C \$HOME/spinifex-e2e-bin"
echo "::endgroup::"
- name: Run Go E2E suites
id: e2e
@@ -144,6 +162,8 @@ jobs:
# SUITES are make-target names ("e2e-cert" → cert.test). Each
# pre-built binary is self-contained (testdata //go:embed'd) so the
# VM needs no go toolchain, no source tree, no gotestsum.
# Each suite is wrapped in its own ::group:: so the Actions UI
# gives the human one collapsible block per suite.
SUITES_RUN=$(echo "${SUITES}" | sed 's/e2e-//g')
ssh -o StrictHostKeyChecking=no -i "${{ steps.vm.outputs.ssh_key }}" \
tf-user@${{ steps.vm.outputs.wan_ip }} \
@@ -151,13 +171,15 @@ jobs:
cd \$HOME/spinifex-e2e-bin && \
FAIL=0; \
for SUITE in ${SUITES_RUN}; do \
echo \"===> \$SUITE\"; \
echo \"::group::suite \$SUITE\"; \
SPINIFEX_E2E=1 SPINIFEX_MODE=single \
ARTIFACT_DIR='${ARTIFACT_DIR}' \
./\$SUITE.test -test.v -test.timeout=30m \
2>&1 | tee '${ARTIFACT_DIR}/test-'\$SUITE'.log'; \
rc=\${PIPESTATUS[0]}; \
echo \"::endgroup::\"; \
if [ \$rc -ne 0 ]; then \
echo \"::error::suite \$SUITE failed (exit \$rc)\"; \
echo \"\$SUITE: FAILED\" >> '${ARTIFACT_DIR}/suite-status.txt'; \
FAIL=1; \
fi; \
@@ -171,17 +193,21 @@ jobs:
if: always()
run: |
mkdir -p "${ARTIFACT_DIR}"
# tar-over-ssh (VM has no rsync). `|| true` so a missing dir on
# an early failure doesn't mask the original error.
echo "::group::tar-over-ssh: pull artifact dir"
# `|| true` so a missing dir on an early failure doesn't mask the
# original error.
ssh -o StrictHostKeyChecking=no -i "${{ steps.vm.outputs.ssh_key }}" \
tf-user@${{ steps.vm.outputs.wan_ip }} \
"tar -C '${ARTIFACT_DIR}' -cf - . 2>/dev/null || true" \
| tar -C "${ARTIFACT_DIR}" -xf - || true
# Pull daemon journal + spinifex config snapshot for the same run window
echo "::endgroup::"
echo "::group::journalctl snapshot (spinifex-*)"
ssh -o StrictHostKeyChecking=no -i "${{ steps.vm.outputs.ssh_key }}" \
tf-user@${{ steps.vm.outputs.wan_ip }} \
'sudo journalctl -u "spinifex-*" --no-pager --since "60 min ago"' \
> "${ARTIFACT_DIR}/spinifex-journal.log" || true
echo "wrote ${ARTIFACT_DIR}/spinifex-journal.log ($(wc -l < "${ARTIFACT_DIR}/spinifex-journal.log" 2>/dev/null || echo 0) lines)"
echo "::endgroup::"
- name: Convert test logs to JUnit XML
if: always()
@@ -199,28 +225,10 @@ jobs:
- name: Publish test summary
if: always()
run: |
{
echo "# Go E2E results"
echo
for J in "${ARTIFACT_DIR}"/junit-*.xml; do
[ -f "$J" ] || continue
echo "## $(basename "$J" .xml)"
python3 -c "
import sys, xml.etree.ElementTree as ET
t = ET.parse('$J').getroot()
tests = sum(int(s.get('tests', 0)) for s in t.iter('testsuite'))
fails = sum(int(s.get('failures', 0)) for s in t.iter('testsuite'))
errs = sum(int(s.get('errors', 0)) for s in t.iter('testsuite'))
skips = sum(int(s.get('skipped', 0)) for s in t.iter('testsuite'))
print(f'- total: {tests}, failed: {fails}, errors: {errs}, skipped: {skips}')
for tc in t.iter('testcase'):
for f in tc.findall('failure'):
print(f' - FAIL: {tc.get(\"classname\")}.{tc.get(\"name\")}')
print(' ::error file=' + tc.get('classname','').replace('.', '/') + '::' + (f.get('message') or 'failed'))
"
done
} >> "$GITHUB_STEP_SUMMARY"
uses: ./spinifex/.github/actions/junit-report
with:
junit-glob: ${{ env.ARTIFACT_DIR }}/junit-*.xml
title: "Go E2E results"
- uses: actions/upload-artifact@v4
if: always()
@@ -234,9 +242,18 @@ jobs:
if: always()
working-directory: mulga/scripts/tofu-cluster
run: |
tofu workspace select -or-create "${SPINIFEX_CI_ENV}" >> "$TOFU_LOG" 2>&1
tofu destroy -var-file=envs/${SPINIFEX_CI_ENV}.tfvars -auto-approve -input=false >> "$TOFU_LOG" 2>&1
./scrub-env.sh "${SPINIFEX_CI_ENV}" >> "$TOFU_LOG" 2>&1 || true
set -euo pipefail
run() {
local label="$1"; shift
echo "::group::${label}"
"$@" 2>&1 | tee -a "$TOFU_LOG"
local rc=${PIPESTATUS[0]}
echo "::endgroup::"
return "$rc"
}
run "tofu workspace select" tofu workspace select -or-create "${SPINIFEX_CI_ENV}"
run "tofu destroy" tofu destroy -var-file=envs/${SPINIFEX_CI_ENV}.tfvars -auto-approve -input=false
run "scrub-env" ./scrub-env.sh "${SPINIFEX_CI_ENV}" || true
- name: Fail job if any suite failed
if: always()