mirror of
https://github.com/shankar0123/certctl.git
synced 2026-06-07 15:11:29 +00:00
shankar0123
76e9380389
The "Frontend E2E (informational)" workflow has been red on every push since Phase 8 (commita9e229b) shipped TEST-H1+H2. The workflow's own header acknowledges this is non-blocking: "The job is intentionally NOT in the merge gate. It runs on every push to surface flakiness early; merge eligibility comes from ci.yml's existing gates (Vitest, lint, build, the 34 CI guards)." But the red badge on every commit is noise. Two ground-truthed root causes (NOT regressions from any recent commit): (1) NO BACKEND IN CI. playwright.config.ts:48-53 only spins up `npm run dev` (Vite frontend). The Vite dev-server proxy forwards /api/v1/* and /health to a backend that doesn't exist in the CI environment → ECONNREFUSED flood throughout the run log. 6 specs need backend data to drive AuthGate bootstrap / lazy palette mount / settings reload: - 01-login-redirect (3 tests): all 3 depend on AuthGate deciding to redirect to /login, which requires /api/v1/auth/info to resolve - 02-dashboard-shell (2 of 4): the palette tests need the Dashboard page to hydrate past loading state → React.lazy palette chunk only mounts after backend data lands - 03-settings-timestamp-pref (1 of 3): the reload+persist test calls page.reload() which re-runs AuthProvider's 4-endpoint bootstrap (2) NO VISUAL-REGRESSION BASELINES COMMITTED. 04-visual- regression.spec.ts uses Playwright `toHaveScreenshot()` against PNG baselines that don't exist (`find web/src/__tests__/e2e -name '*.png'` returns 0). First-run = "snapshot doesn't exist, writing actual" = expected fail. The e2e.yml workflow exposes an `update_snapshots` dispatch input for the controlled first-run pass, but on default push runs that flag is false → tests fail. Operator choice (2026-05-14): "skip backend-dependent specs" over spinning up backend in CI (1-2 days of CI engineering, premature per the e2e.yml comment's "do not promote to required-for-merge in this phase" guidance) or dropping the e2e job from push triggers entirely (loses early-flakiness signal). ═══════════════════════════ CHANGES ═══════════════════════════════ web/src/__tests__/e2e/01-login-redirect.spec.ts: describe-level test.skip(NEEDS_BACKEND, '...') guard. All 3 tests in this file depend on AuthGate. web/src/__tests__/e2e/02-dashboard-shell.spec.ts: Per-test test.skip(NEEDS_BACKEND, '...') on the 2 palette tests (47, 59). Sidebar IA test (31) and breadcrumb test (70) stay ungated — both passed in CI today because they don't depend on Dashboard data resolving. web/src/__tests__/e2e/03-settings-timestamp-pref.spec.ts: Per-test test.skip(NEEDS_BACKEND, '...') on the reload+persist test (39). Card-render (28) and invalid-IANA-fallback (54) tests stay ungated — both passed. web/src/__tests__/e2e/04-visual-regression.spec.ts: describe-level skip guard. All 5 tests need both backend AND committed baselines; neither exists in CI today. The workflow_ dispatch update_snapshots input is the controlled-update path when both prereqs land. Skip condition is `!process.env.CERTCTL_E2E_BACKEND_URL && !!process.env.CI`: • In CI without a backend → skip • Locally where operator runs `make demo` + `npm run e2e` → no CI env var, so skip evaluates false → all tests run • In CI WITH a backend set via CERTCTL_E2E_BACKEND_URL env → tests run; this is the path the e2e.yml's "next steps" will use when backend-in-CI infra lands ═══════════════════════════ AUDIT FRAMING ════════════════════════ This is honest signal, not test deletion: • 11 tests don't run in CI today; they're SKIPPED with a clear operator-facing reason and an env-var unlock path. • The 5 tests that DO run in CI today (sidebar IA, breadcrumb, timestamp card render, invalid-IANA fallback, smoke "login renders brand") continue to run and protect the no-backend- needed surface. • The "1-2 weeks of green runs" promotion criterion in e2e.yml's header is now achievable for the no-backend subset. ═══════════════════════════ VERIFICATION ═══════════════════════════ • npx tsc --noEmit — exit 0 • Visual diff of skip-guard patterns across 4 files — consistent NEEDS_BACKEND const + test.skip(...) + operator-facing reason • Falsifiable proof: the next push's e2e workflow run should show 5 passing + 11 skipped + 0 failed; exit 0; informational job goes from RED to GREEN. Ground-truth: origin/master tip7268d12(FE-M6 just pushed) verified via GitHub API BEFORE commit.