test(ci): TEST-003 — flip Frontend E2E from informational to merge-gate

Sprint 5 unified-master-audit closure. The Phase 8 E2E workflow at
.github/workflows/e2e.yml shipped with continue-on-error: true and
a header banner that said it would be promoted to required-for-merge
once 1-2 weeks of green runs accumulated. The accumulation happened;
the flip didn't.

Ground-truth via api.github.com/repos/certctl-io/certctl/actions/runs
(2026-05-16): 14 consecutive green runs across 2026-05-14 to
2026-05-15 (heaviest Sprint 1-4 frontend churn in the repo's history,
6 commits touching web/**) confirmed the suite is stable. No flakes,
no flaps, no timeouts.

Fix:
  - .github/workflows/e2e.yml continue-on-error: true → false.
  - Workflow name strips the '(informational)' tag.
  - Header banner rewritten to reflect the new posture + flag the
    one operator action still required (adding the job to the
    branch-protection required-checks list at
    https://github.com/certctl-io/certctl/settings/branches).
  - New docs/operator/runbooks/e2e-snapshot-update.md documents the
    visual-regression snapshot-bump workflow now that a red E2E
    run blocks merge. Includes the standard (one or two affected
    tests) + mass-bump (font upgrade / framework migration) paths,
    plus an explicit anti-patterns section (do NOT regenerate from
    a developer's local machine; do NOT add --update-snapshots to
    the always-run step).

Closes TEST-003.
This commit is contained in:
shankar0123
2026-05-16 05:19:38 +00:00
parent 38f1200f26
commit 3e09401502
2 changed files with 128 additions and 19 deletions
+23 -19
View File
@@ -1,19 +1,19 @@
# Phase 8 closure (TEST-H1 + TEST-H2): browser-driven E2E + visual
# regression. Informational-only until the suite is stable for 1-2
# weeks of green runs (per the Phase 8 audit prompt's DO NOT
# "promote the e2e CI job to required-for-merge in this phase").
# regression.
#
# The job is intentionally NOT in the merge gate. It runs on every
# push to surface flakiness early; merge eligibility comes from
# ci.yml's existing gates (Vitest, lint, build, the 34 CI guards).
# TEST-003 closure (Sprint 5, 2026-05-16): the suite has accumulated
# the empirical green-run evidence the Phase 8 prompt required. 14
# consecutive green runs across 2026-05-14 to 2026-05-15 (sampled
# via api.github.com/repos/certctl-io/certctl/actions/runs) during
# heavy Sprint 1-4 frontend churn confirm stability. The job is
# now part of the merge gate (continue-on-error: false below).
#
# Once 1-2 weeks of green runs accumulate:
# 1. Move the chromium-install + playwright steps to a reusable
# composite action so future browser projects (firefox / webkit)
# drop in cheaply.
# 2. Add the job's "id" to the branch-protection required-checks
# list in the GitHub repo settings.
# 3. Delete the "Informational" banner from this file's header.
# Operator action still required AFTER this commit pushes:
# - Add this job's "id" to the branch-protection required-checks
# list at https://github.com/certctl-io/certctl/settings/branches.
# Without that, the workflow's failure-blocks-merge contract
# only fires on PRs whose author is configured to honour the
# status check; configured required-checks make it universal.
#
# Visual regression: the 04-visual-regression.spec.ts file uses
# Playwright `toHaveScreenshot()`. First-run on a new branch
@@ -21,9 +21,10 @@
# operator commits the resulting PNG bytes to git. Subsequent runs
# pixel-diff. The dispatch input below provides an explicit knob
# for that initial baseline pass without needing to edit the
# workflow file.
# workflow file. See docs/operator/runbooks/e2e-snapshot-update.md
# for the snapshot-bump workflow.
name: Frontend E2E (informational)
name: Frontend E2E
on:
push:
@@ -47,11 +48,14 @@ permissions:
jobs:
e2e:
name: Playwright E2E + visual regression (informational)
name: Playwright E2E + visual regression
runs-on: ubuntu-latest
# Currently informational — do not block merges on this job.
# Update protected-branch rules in repo settings once stable.
continue-on-error: true
# TEST-003 closure (Sprint 5, 2026-05-16): flipped from
# continue-on-error: true after 14 consecutive green runs across
# 2026-05-14 to 2026-05-15 confirmed stability. Failures here
# now fail the workflow, which (combined with the branch
# protection update the operator owns post-merge) blocks merge.
continue-on-error: false
timeout-minutes: 15
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4