mirror of
https://github.com/PerpetualSoftware/pad.git
synced 2026-10-26 06:19:49 +00:00
xarmian
1923c5a04d
* ci: split E2E into two Playwright shards behind one E2E (Playwright) verdict (IDEA-3189) IDEA-3189's tripwire fired: the median of the last 10 green E2E jobs is 16.5m (13.7-19.3m) against its 14m threshold, and #1811's E2E was cancelled at the 20m cap with no failing test. The suite now runs as --shard=1/2 and 2/2, each job keeping the 20m cap and the TASK-3415 binary guards. A project cut was measured and rejected: desktop-chromium carries 1222s of 1359s. The check named "E2E (Playwright)" is now an aggregate that needs both shards, runs if: always(), and passes only when every shard succeeded. Report and flaky-evidence artifacts carry the shard number. Claude-Session: https://claude.ai/code/session_01Q9WssCjFuHjSfLABoW4tmS * ci: shard desktop E2E in halves and run mobile whole (IDEA-3189) A bare --shard=N/2 split exactly on the project boundary: Playwright cuts the ordered test list by count, the projects run in sequence and have the same test count, so the first run's shard 2/2 was mobile-chromium alone (63 passed, 553 skipped, 1.9m) and shard 1/2 all of desktop. The matrix is now desktop 1/2, desktop 2/2 (308 tests each by --list) and mobile whole. Claude-Session: https://claude.ai/code/session_01Q9WssCjFuHjSfLABoW4tmS
757 lines
37 KiB
YAML
757 lines
37 KiB
YAML
name: CI
|
|
|
|
on:
|
|
push:
|
|
branches: [main]
|
|
pull_request:
|
|
branches: [main]
|
|
|
|
concurrency:
|
|
group: ci-${{ github.ref }}
|
|
cancel-in-progress: true
|
|
|
|
permissions:
|
|
contents: read
|
|
|
|
# All third-party Actions are pinned to a 40-char commit SHA with a trailing
|
|
# '# vX.Y.Z' comment so a compromised maintainer or moved tag cannot silently
|
|
# execute attacker code in CI. Bump the SHA + comment together when updating.
|
|
|
|
jobs:
|
|
go:
|
|
# Backstop only. A hung TEST is caught by -timeout=45m on the steps
|
|
# below, which fails fast AND prints the goroutine dump that tells
|
|
# you which test hung; this cap exists because a job with no
|
|
# timeout-minutes inherits GitHub's 6-HOUR default, so a runaway
|
|
# that isn't a single test (a wedged service container, a stuck
|
|
# download) would otherwise sit there. Set well above the two 45m test
|
|
# steps so that in practice the per-binary timeout is what fires — not a
|
|
# guarantee, since this cap covers setup and every step, not just those
|
|
# two (TASK-2545).
|
|
timeout-minutes: 100
|
|
name: Go
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
|
|
|
- uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e # v7.0.0
|
|
with:
|
|
go-version: "1.26"
|
|
|
|
- name: Allow Go to fetch the toolchain go.mod pins
|
|
# actions/setup-go unconditionally exports GOTOOLCHAIN=local, which
|
|
# forbids Go from fetching the toolchain go.mod requires. go.mod pins
|
|
# `go 1.26.5`, but setup-go's "1.26" resolves to the newest patch in
|
|
# its manifest (1.26.4 here), so every `go` command fails with
|
|
# "go.mod requires go >= 1.26.5 (running go 1.26.4)". Written AFTER
|
|
# setup-go so it wins the $GITHUB_ENV last-write; `auto` lets Go pull
|
|
# what go.mod asks for on demand. Added after #896 raised the go.mod
|
|
# floor.
|
|
#
|
|
# Load-bearing for the govulncheck step below as of BUG-2565: go.mod
|
|
# also carries `toolchain go1.26.6`, which is what actually gets
|
|
# stamped into the scanned binary and clears 8 reachable stdlib
|
|
# advisories. `auto` is what honours that line — under `local` the
|
|
# build silently falls back to whatever patch setup-go installed and
|
|
# the gate goes red again.
|
|
run: echo "GOTOOLCHAIN=auto" >> "$GITHUB_ENV"
|
|
|
|
- name: Create web build placeholder for embed
|
|
run: mkdir -p web/build && echo "placeholder" > web/build/.gitkeep
|
|
|
|
- uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
|
|
with:
|
|
node-version: "24"
|
|
cache: "npm"
|
|
cache-dependency-path: web/package-lock.json
|
|
|
|
# The materializer bundle (TASK-2198) is embedded like web/build, but it
|
|
# is built for real rather than stubbed: internal/materialize's tests run
|
|
# it (TestCorpusParity replays the corpus through it in goja), so a
|
|
# placeholder would fail them. Only the bundle, not the whole web build.
|
|
- name: Build the materializer bundle for embed
|
|
working-directory: web
|
|
run: npm ci && node scripts/build-materializer.mjs
|
|
|
|
- name: Run go vet
|
|
run: go vet ./...
|
|
|
|
# The Nix vendorHash bump parser (TASK-2954). Its own gate lives HERE, in
|
|
# the Go job, and not in nix.yml: over there the script only does anything
|
|
# when a build has ALREADY failed on a hash mismatch, so an ordinary PR
|
|
# would exercise none of its logic and a change that broke it would land
|
|
# green and surface on the next dependency bump. The load-bearing case in
|
|
# the suite is negative — an npm fetchurl FOD mismatch must NOT be written
|
|
# into vendorHash.
|
|
- name: Test the Nix vendorHash bump parser
|
|
run: nix/bump-vendor-hash_test.sh
|
|
|
|
# The heal job that pushes the parser's output to main (BUG-2974). Gated
|
|
# here for the same reason and one more: in nix.yml that job runs only
|
|
# after a real merge has moved the module set, so its defect shipped green
|
|
# for a day and was found by reading run logs. The suite drives the script
|
|
# against a real git remote, and its load-bearing case is the one that
|
|
# shipped — a heal owed by an EARLIER push, on a push that touched no
|
|
# go.mod/go.sum.
|
|
- name: Test the Nix vendorHash heal job
|
|
run: nix/heal-vendor-hash_test.sh
|
|
|
|
- name: Run golangci-lint
|
|
# only-new-issues: false means CI fails on ANY linter finding,
|
|
# not just findings on PR-changed lines. The IDEA-732 cleanup
|
|
# (PRs #247/#249/#251/#252) cleared the existing findings under
|
|
# the configured linter set in .golangci.yml — staticcheck SA*,
|
|
# govet, ineffassign, gofmt, and the standalone `unused` linter
|
|
# (which reports U1000). Flipping the gate now prevents
|
|
# regression drift going forward.
|
|
# v2 of golangci-lint is required because v1 is capped at older
|
|
# Go releases that we no longer support.
|
|
uses: golangci/golangci-lint-action@ba0d7d2ec06a0ea1cb5fa41b2e4a3ab91d21278a # v9.3.0
|
|
with:
|
|
version: v2.11.4
|
|
args: --timeout=5m
|
|
only-new-issues: false
|
|
# Skip the pre-lint `golangci-lint config verify`, which fetches its
|
|
# JSON schema over the network from golangci-lint.run and fails the
|
|
# required Go gate when that host is slow/unreachable — no lint
|
|
# finding, no code involvement (BUG-2568). A malformed .golangci.yml
|
|
# is still caught by the lint run itself, just with a worse message.
|
|
verify: false
|
|
# Cap the blast radius of a stale-cache event to ~24h. The action's
|
|
# cache stores the resolved issue list from a prior pass; if any
|
|
# one pass writes degenerate results (analyzer upgrade, plugin
|
|
# reset, sub-package version drift), every downstream restore
|
|
# replays that list verbatim until the cache key rotates. PR #635
|
|
# hit exactly this — 30+ SA5011/SA4023 false positives against
|
|
# unchanged code while local (cold-cache) runs reported 0 issues.
|
|
# Default is 7 days; 1 day still keeps most runs cache-hot while
|
|
# guaranteeing daily refresh. See BUG-1624.
|
|
cache-invalidation-interval: "1"
|
|
|
|
- name: Run govulncheck
|
|
# Fails the build on any known vulnerability in a package we
|
|
# actually reach via the call graph. Net-positive: catches CVEs
|
|
# in indirect deps early, without the noise of hitting every
|
|
# stale entry in our dependency tree.
|
|
#
|
|
# Runs in BINARY mode against a freshly-built pad binary rather than
|
|
# source mode (`govulncheck ./...`). Source mode builds an SSA call-
|
|
# graph over the whole dependency tree and can balloon to multiple GB
|
|
# of RAM (BUG-2084); binary mode reads the binary's symbol table — a
|
|
# fraction of the memory, still call-graph-precise, and detects stdlib
|
|
# vulns from the Go version stamped in the binary. The "Create web
|
|
# build placeholder for embed" step above satisfies the //go:embed so
|
|
# the scan build succeeds. Mirrors the Makefile `vuln` target — keep
|
|
# the two in sync.
|
|
#
|
|
# Pinned to a specific govulncheck release. Track upstream in
|
|
# Pad's workspace; bump intentionally so an upstream behavior
|
|
# change can't break unrelated PRs. Update via:
|
|
# go install golang.org/x/vuln/cmd/govulncheck@<new-tag>
|
|
run: |
|
|
go install golang.org/x/vuln/cmd/govulncheck@v1.2.0
|
|
go build -o pad-vulnscan ./cmd/pad
|
|
"$(go env GOPATH)/bin/govulncheck" -mode binary pad-vulnscan
|
|
|
|
- name: Run tests
|
|
# -timeout is EXPLICIT here on purpose. `go test` without it uses a
|
|
# 10m per-test-binary default that nobody chose. The two race steps
|
|
# in this file were raised to 45m when the suite outgrew 30m
|
|
# (BUG-1913) while their non-race siblings silently kept the
|
|
# default — which is how the PostgreSQL leg below panicked at 10m on
|
|
# the v0.13.0 release commit (TASK-2545, which also found the ~40
|
|
# minutes it cost the cut and the goroutine dump showing nothing
|
|
# actually hung). Matching the race
|
|
# legs keeps ONE number in this file. It is a hang-catcher, not a
|
|
# performance budget: the signal for "the suite got slow" is job
|
|
# wall-clock, not this value.
|
|
run: go test -timeout=45m ./...
|
|
|
|
- name: Run tests with race detector
|
|
# Runs on both push-to-main AND pull_request. Previously gated to
|
|
# main only because GitHub Actions minutes were billed on private
|
|
# repos; the repo is public now, so PR minutes are free and we'd
|
|
# rather catch race regressions on the contributing branch than
|
|
# after merge. See BUG-1371 (also dropped test-only bcrypt cost via
|
|
# TestMain, which brought this step back under the then-30m budget —
|
|
# it did NOT raise the budget; BUG-1913 later did, 30m→45m, when the
|
|
# suite outgrew it again).
|
|
#
|
|
# Default 10m is tight: the full server-package suite under -race
|
|
# measures ~13m locally on a developer laptop after BUG-851 (the
|
|
# ipRateLimiter goroutine drain). The PLAN-866 attachment work
|
|
# (image decode/encode/resize across thumbnail + transform tests)
|
|
# pushes total race-step runtime past 20m on the GitHub-hosted
|
|
# runner. BUG-1913: the suite organically grew past the old 30m
|
|
# budget (734 server-package tests, ~30.3m under -race even on a
|
|
# fast local machine; no single test exceeds 14s — aggregate
|
|
# weight, not a hang), turning most main runs red. 45m restores
|
|
# headroom; genuine deadlocks still hit this and produce the
|
|
# goroutine-dump panic, just up to 15m later. The real fix
|
|
# (cheaper suite: shared fixtures / sharding) is tracked on
|
|
# BUG-1913.
|
|
run: go test -race -timeout=45m ./...
|
|
|
|
- name: Build binary
|
|
run: go build -o pad ./cmd/pad
|
|
|
|
- name: Verify binary runs
|
|
run: ./pad --help
|
|
|
|
native-smoke:
|
|
name: Smoke (${{ matrix.os }})
|
|
runs-on: ${{ matrix.os }}
|
|
# 10 for the smoke itself, plus the materializer bundle build and the
|
|
# supervisor tests (TASK-2198 U3), which spawn real workers and compile
|
|
# the package's test binary.
|
|
timeout-minutes: 35
|
|
defaults:
|
|
run:
|
|
shell: pwsh
|
|
strategy:
|
|
fail-fast: false
|
|
matrix:
|
|
include:
|
|
- os: macos-latest
|
|
binary: pad
|
|
path: ./pad
|
|
- os: windows-latest
|
|
binary: pad.exe
|
|
path: .\pad.exe
|
|
steps:
|
|
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
|
|
|
- uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e # v7.0.0
|
|
# No GOTOOLCHAIN=auto step here, unlike every other Go job in this
|
|
# file, and the reason is worth stating because the asymmetry looks
|
|
# like an oversight: `go-version-file` makes setup-go PARSE go.mod
|
|
# itself, and v7's parser prefers the `toolchain` directive over the
|
|
# `go` directive, so it installs go1.26.6 (BUG-2565) directly. The
|
|
# GOTOOLCHAIN=local it then exports is satisfied by that install —
|
|
# nothing left to fetch. The other jobs pass `go-version: "1.26"`,
|
|
# which never reads go.mod at all, so they genuinely need `auto` to
|
|
# pick the toolchain line up at build time.
|
|
#
|
|
# The one way this silently regresses: setup-go's parser falls back
|
|
# to the `go` directive when GOTOOLCHAIN is ALREADY `local` in the
|
|
# environment at parse time (installer.ts parseGoVersionFile). So if
|
|
# a job- or workflow-level `env: GOTOOLCHAIN: local` is ever added
|
|
# above this step, these smoke binaries drop to the 1.26.5 floor
|
|
# while everything else ships 1.26.6 — no red, just coverage that
|
|
# quietly stops matching the artifact.
|
|
with:
|
|
go-version-file: go.mod
|
|
|
|
- name: Create web build placeholder
|
|
run: |
|
|
New-Item -ItemType Directory -Force web/build | Out-Null
|
|
Set-Content web/build/index.html '<!doctype html>'
|
|
|
|
- uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
|
|
with:
|
|
node-version: "24"
|
|
cache: "npm"
|
|
cache-dependency-path: web/package-lock.json
|
|
|
|
# The materializer bundle (TASK-2198) is built for real here, as in the
|
|
# go job, and not stubbed: the supervisor tests at the end of this job
|
|
# spawn real workers that load it, and internal/materialize.New refuses
|
|
# a placeholder by name. Only the bundle, not the whole web build (the
|
|
# go job's shape): web/build stays the placeholder above.
|
|
- name: Build the materializer bundle for embed
|
|
working-directory: web
|
|
run: npm ci && node scripts/build-materializer.mjs
|
|
|
|
- name: Build binary
|
|
run: go build -o "${{ matrix.binary }}" ./cmd/pad
|
|
|
|
- name: Verify help output
|
|
run: '& "${{ matrix.path }}" --help'
|
|
|
|
- name: Run CLI smoke test
|
|
run: |
|
|
$ErrorActionPreference = 'Stop'
|
|
$PSNativeCommandUseErrorActionPreference = $true
|
|
$binary = '${{ matrix.path }}'
|
|
$env:PAD_DATA_DIR = Join-Path $env:RUNNER_TEMP 'pad-smoke'
|
|
$env:PAD_BYPASS_SETUP_TOKEN = 'true'
|
|
$stdout = Join-Path $env:RUNNER_TEMP 'pad-server.out.log'
|
|
$stderr = Join-Path $env:RUNNER_TEMP 'pad-server.err.log'
|
|
$server = Start-Process -FilePath $binary -ArgumentList 'server','start','--port','17777' -PassThru -RedirectStandardOutput $stdout -RedirectStandardError $stderr
|
|
|
|
try {
|
|
$ready = $false
|
|
for ($attempt = 0; $attempt -lt 60; $attempt++) {
|
|
try {
|
|
Invoke-WebRequest -UseBasicParsing http://127.0.0.1:17777/api/v1/health | Out-Null
|
|
$ready = $true
|
|
break
|
|
} catch {
|
|
Start-Sleep -Seconds 1
|
|
}
|
|
}
|
|
if (-not $ready) {
|
|
Get-Content $stdout, $stderr -ErrorAction SilentlyContinue
|
|
throw 'Pad server did not become ready'
|
|
}
|
|
|
|
& $binary auth configure --mode local --port 17777
|
|
& $binary auth setup --email smoke@example.com --name Smoke --password 'SmokePass123!'
|
|
& $binary workspace create Smoke --slug smoke --template startup
|
|
|
|
$item = (& $binary --workspace smoke --format json item create task 'Native smoke item') | ConvertFrom-Json
|
|
$shown = (& $binary --workspace smoke --format json item show $item.ref) | ConvertFrom-Json
|
|
if ($shown.title -ne 'Native smoke item') {
|
|
throw "Unexpected item title: $($shown.title)"
|
|
}
|
|
# Stop through the CLI, not Stop-Process: `pad server stop` is
|
|
# the only place the Windows PID-file ownership check runs
|
|
# (BUG-2969 — flock covers Linux and macOS, GetProcessTimes covers
|
|
# this platform), and a smoke that killed the process directly
|
|
# would leave that implementation unexercised anywhere. If it
|
|
# fails, the smoke fails; the finally below is the backstop that
|
|
# keeps a wedged server from holding the runner.
|
|
& $binary server stop
|
|
if (-not $server.HasExited) {
|
|
$server.WaitForExit(10000) | Out-Null
|
|
}
|
|
if (-not $server.HasExited) {
|
|
throw 'pad server stop did not stop the server'
|
|
}
|
|
} finally {
|
|
Stop-Process -Id $server.Id -ErrorAction SilentlyContinue
|
|
}
|
|
|
|
# A headless disarm must hold for as long as its session lives, on both
|
|
# platforms, through the REAL owner probe (no Go test runs in this job).
|
|
# Windows cannot probe a pid, so the verdict is unknown: before BUG-2771
|
|
# that read as dead, and the next read reaped the file and let auto_arm
|
|
# forget the disarm. On unix the file named the exiting `pad session
|
|
# disarm` instead of the session, so it was dead on arrival (BUG-3227).
|
|
# This step's shell stands in for the session, named the way a
|
|
# non-Claude harness names it: PAD_SESSION_PID.
|
|
- name: Headless disarm holds for the session (BUG-2771, BUG-3227)
|
|
run: |
|
|
$ErrorActionPreference = 'Stop'
|
|
$env:PAD_SESSION_PID = "$PID"
|
|
$binary = (Resolve-Path '${{ matrix.path }}').Path
|
|
$env:PAD_DATA_DIR = Join-Path $env:RUNNER_TEMP 'pad-arm-smoke'
|
|
Remove-Item Env:CLAUDE_CODE_MESSAGING_SOCKET -ErrorAction SilentlyContinue
|
|
$repo = Join-Path $env:RUNNER_TEMP 'arm-repo'
|
|
New-Item -ItemType Directory -Force $repo | Out-Null
|
|
Set-Content (Join-Path $repo '.pad.toml') "workspace = `"demo`"`n[push]`nauto_arm = true`n"
|
|
Push-Location $repo
|
|
try {
|
|
& $binary session should-arm
|
|
if ($LASTEXITCODE -ne 0) { throw 'premise: an auto_arm repo with no override must arm' }
|
|
|
|
$d = & $binary session disarm --format json | ConvertFrom-Json
|
|
if (-not (Test-Path $d.path)) { throw 'disarm wrote no arm-state file' }
|
|
foreach ($i in 1, 2) {
|
|
& $binary session should-arm
|
|
if ($LASTEXITCODE -eq 0) { throw "read ${i}: the disarm was forgotten and the session arms via auto_arm" }
|
|
if (-not (Test-Path $d.path)) { throw "read ${i} reaped the disarm file" }
|
|
}
|
|
|
|
$r = & $binary session reset --format json | ConvertFrom-Json
|
|
if (-not $r.removed) { throw 'pad session reset removed nothing' }
|
|
& $binary session should-arm
|
|
if ($LASTEXITCODE -ne 0) { throw 'after reset, auto_arm must decide again (armed)' }
|
|
} finally {
|
|
Pop-Location
|
|
}
|
|
$global:LASTEXITCODE = 0
|
|
|
|
# The memory-cap tests below allocate 2 GiB against a 1600 MiB cap, and
|
|
# the adversarial one drives a real worker to a 2 GiB cap. Logged so a
|
|
# failure there can be read against what the runner actually had.
|
|
- name: Runner memory
|
|
run: |
|
|
if ($IsMacOS) { sysctl hw.memsize hw.ncpu } else { Get-CimInstance Win32_ComputerSystem | Select-Object TotalPhysicalMemory, NumberOfLogicalProcessors | Format-List }
|
|
|
|
# The materializer supervisor (TASK-2198 U3) on each OS's REAL memory
|
|
# cap, which only these runners have: macOS's parent-side RSS watchdog
|
|
# (proc_info) and Windows' Job Object with a suspended start. The Go job
|
|
# covers Linux's RLIMIT_AS. PAD_MATERIALIZE_TEST_NO_SKIP turns the cap
|
|
# tests' skips into failures, so a cap that stopped running cannot pass
|
|
# as "skipped".
|
|
- name: Materializer supervisor and memory cap (TASK-2198 U3)
|
|
env:
|
|
PAD_MATERIALIZE_TEST_NO_SKIP: "1"
|
|
run: go test ./internal/materialize/ -run '^(TestSupervisor.*|TestParseMemLimit|TestConfigFromEnv|TestParsePSRSS|TestPSRSSLive|TestParseStatVsize|TestStderrOOMMarkers|TestDeepNestFrameShallow)$' -count=1 -timeout 20m -v
|
|
|
|
go-postgres:
|
|
# Backstop only. A hung TEST is caught by -timeout=45m on the steps
|
|
# below, which fails fast AND prints the goroutine dump that tells
|
|
# you which test hung; this cap exists because a job with no
|
|
# timeout-minutes inherits GitHub's 6-HOUR default, so a runaway
|
|
# that isn't a single test (a wedged service container, a stuck
|
|
# download) would otherwise sit there. Set well above the two 45m test
|
|
# steps so that in practice the per-binary timeout is what fires — not a
|
|
# guarantee, since this cap covers setup and every step, not just those
|
|
# two (TASK-2545).
|
|
timeout-minutes: 100
|
|
name: Go (PostgreSQL)
|
|
runs-on: ubuntu-latest
|
|
services:
|
|
postgres:
|
|
image: postgres:17-alpine
|
|
env:
|
|
POSTGRES_USER: pad
|
|
POSTGRES_PASSWORD: pad
|
|
POSTGRES_DB: pad
|
|
ports:
|
|
- 5432:5432
|
|
options: >-
|
|
--health-cmd "pg_isready -U pad"
|
|
--health-interval 5s
|
|
--health-timeout 3s
|
|
--health-retries 10
|
|
steps:
|
|
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
|
|
|
- uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e # v7.0.0
|
|
with:
|
|
go-version: "1.26"
|
|
|
|
- name: Allow Go to fetch the toolchain go.mod pins
|
|
# See the identical step in the `go` job — setup-go pins
|
|
# GOTOOLCHAIN=local, blocking the toolchain fetch go.mod requires
|
|
# (`go 1.26.5` floor, `toolchain go1.26.6` per BUG-2565).
|
|
run: echo "GOTOOLCHAIN=auto" >> "$GITHUB_ENV"
|
|
|
|
- name: Create web build placeholder for embed
|
|
run: mkdir -p web/build && echo "placeholder" > web/build/.gitkeep
|
|
|
|
- uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
|
|
with:
|
|
node-version: "24"
|
|
cache: "npm"
|
|
cache-dependency-path: web/package-lock.json
|
|
|
|
# The materializer bundle (TASK-2198) is embedded like web/build, but it
|
|
# is built for real rather than stubbed: internal/materialize's tests run
|
|
# it (TestCorpusParity replays the corpus through it in goja), so a
|
|
# placeholder would fail them. Only the bundle, not the whole web build.
|
|
- name: Build the materializer bundle for embed
|
|
working-directory: web
|
|
run: npm ci && node scripts/build-materializer.mjs
|
|
|
|
# pad's backup and restore shell out to pg_dump and psql, and pg_dump
|
|
# refuses to dump a server newer than itself. ubuntu-latest ships the
|
|
# 16 clients against this job's 17 server, so the native round trip
|
|
# (TestPostgresBackupRestoreConnection) could never run here (TASK-2962).
|
|
# The PGDG signing key is checked against its published fingerprint
|
|
# before the repository is trusted, in the spirit of the action pins
|
|
# above.
|
|
- name: Install PostgreSQL 17 clients
|
|
run: |
|
|
set -euo pipefail
|
|
key=/usr/share/keyrings/pgdg.asc
|
|
curl -fsSL -o /tmp/pgdg.asc https://www.postgresql.org/media/keys/ACCC4CF8.asc
|
|
# signed-by trusts EVERY key in the file, so the file must hold
|
|
# exactly one primary key and it must be PGDG's.
|
|
keys=$(gpg --show-keys --with-colons /tmp/pgdg.asc)
|
|
pubs=$(printf '%s\n' "$keys" | awk -F: '$1 == "pub"' | wc -l)
|
|
fpr=$(printf '%s\n' "$keys" | awk -F: '$1 == "fpr" { print $10; exit }')
|
|
if [ "$pubs" != "1" ] || [ "$fpr" != "B97B0AFCAA1A47F044F244A07FCC7D46ACCC4CF8" ]; then
|
|
echo "PGDG key check failed: $pubs primary key(s), first fingerprint $fpr" >&2
|
|
exit 1
|
|
fi
|
|
sudo install -m 0644 /tmp/pgdg.asc "$key"
|
|
echo "deb [signed-by=$key] https://apt.postgresql.org/pub/repos/apt $(lsb_release -cs)-pgdg main" \
|
|
| sudo tee /etc/apt/sources.list.d/pgdg.list > /dev/null
|
|
sudo apt-get update -o Dir::Etc::sourcelist=/etc/apt/sources.list.d/pgdg.list -o Dir::Etc::sourceparts=- -o APT::Get::List-Cleanup=0
|
|
sudo apt-get install -y --no-install-recommends postgresql-client-17
|
|
echo "/usr/lib/postgresql/17/bin" >> "$GITHUB_PATH"
|
|
|
|
- name: Check the PostgreSQL clients match the server
|
|
run: |
|
|
set -euo pipefail
|
|
for tool in pg_dump psql; do
|
|
"$tool" --version
|
|
"$tool" --version | grep -Eq ' 17\.' || { echo "$tool is not version 17" >&2; exit 1; }
|
|
done
|
|
|
|
# Runs the native backup/restore round trip ALONE and verbosely, and
|
|
# fails unless it PASSED: in the quiet suite below a skip and a pass
|
|
# look the same, which is how this test went unrun in CI. It creates
|
|
# and drops its own uuid-named databases, so it needs the superuser the
|
|
# service starts with, not the suite's shared database.
|
|
- name: Native backup/restore round trip against PostgreSQL
|
|
env:
|
|
PAD_TEST_POSTGRES_TOOLS_URL: "postgres://pad:pad@localhost:5432/pad?sslmode=disable"
|
|
run: |
|
|
set -euo pipefail
|
|
go test ./cmd/pad/ -run '^TestPostgresBackupRestoreConnection$' -count=1 -v | tee /tmp/roundtrip.log
|
|
grep -q '^--- PASS: TestPostgresBackupRestoreConnection ' /tmp/roundtrip.log || { echo "round trip did not run to a PASS" >&2; exit 1; }
|
|
|
|
- name: Run tests against PostgreSQL
|
|
env:
|
|
PAD_TEST_POSTGRES_URL: "postgres://pad:pad@localhost:5432/pad?sslmode=disable"
|
|
# Explicit -timeout: see the SQLite "Run tests" step. This is the
|
|
# binary that actually blew the 10m default — internal/store creates a
|
|
# fresh database AND replays the full migration chain per test
|
|
# (~0.43s each locally), so the package's runtime grows linearly with
|
|
# the test count. Measured on a dev box at 212d59e7: internal/store
|
|
# 280s, whole PG suite 4m43s wall. CI's own runtime is UNMEASURED
|
|
# here; all that follows from the panic is that its store binary
|
|
# exceeded 10m, i.e. >2.14x this box on that package. (An earlier
|
|
# version of this comment said "~2x slower, which put store over
|
|
# 10m" — 280s x 2 is 9m20s, so that explained nothing.)
|
|
run: go test -timeout=45m ./... -count=1
|
|
|
|
- name: Run tests with race detector against PostgreSQL
|
|
env:
|
|
PAD_TEST_POSTGRES_URL: "postgres://pad:pad@localhost:5432/pad?sslmode=disable"
|
|
# Runs on both push-to-main AND pull_request — see SQLite race-step
|
|
# comment for the public-repo / BUG-1371 reasoning.
|
|
#
|
|
# Headroom over the default 10m. PostgreSQL adds latency on
|
|
# every CREATE/DROP, and the PLAN-866 attachment work pushed the
|
|
# cumulative wall over 20m. The bootstrap-user bcrypt cost that
|
|
# blew past 30m on main (BUG-1371) is now handled by TestMain
|
|
# dropping the cost to bcrypt.MinCost for test binaries.
|
|
# BUG-1913: raised 30m → 45m alongside the SQLite step — the
|
|
# suite's aggregate runtime crossed the old budget (this job
|
|
# variant failed main at f235a04 with the same timeout panic).
|
|
run: go test -race -timeout=45m ./... -count=1
|
|
|
|
web:
|
|
name: Web
|
|
runs-on: ubuntu-latest
|
|
defaults:
|
|
run:
|
|
working-directory: web
|
|
steps:
|
|
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
|
|
|
- uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
|
|
with:
|
|
node-version: "24"
|
|
cache: "npm"
|
|
cache-dependency-path: web/package-lock.json
|
|
|
|
- name: Install dependencies
|
|
run: npm ci
|
|
|
|
- name: Check coordinated Tiptap pins
|
|
# The Y.Doc/ProseMirror schema is shared across @tiptap/core,
|
|
# @tiptap/extension-collaboration, @tiptap/y-tiptap and @tiptap/pm.
|
|
# A stray `npm update` that slides one of them can silently change
|
|
# the persisted Y.Doc shape, producing divergent collab ops the
|
|
# relay can't reconcile (see CLAUDE.md "Tiptap multi-package
|
|
# coordinated bumps" and BUG-2009). This guard fails if any of them
|
|
# loses its exact pin in package.json or resolves to more than one
|
|
# version in the lockfile.
|
|
run: npm run check:tiptap-pins
|
|
|
|
- name: Build
|
|
run: npm run build
|
|
|
|
- name: Type check (svelte-check)
|
|
run: npm run check
|
|
|
|
- name: Run web unit tests (vitest)
|
|
run: npm run test
|
|
|
|
- name: Audit npm dependencies (production, high+)
|
|
# Fail the build on any HIGH or CRITICAL advisory in production deps.
|
|
# Dev-only advisories are treated as informational — they don't ship
|
|
# and fixing them can require waiting on upstream maintainers.
|
|
#
|
|
# LAST, and through scripts/ci-audit.mjs, not bare `npm audit`
|
|
# (BUG-2881). Bare `npm audit` exits non-zero the same way for "an
|
|
# advisory exists" and "the advisory service was unreachable", and this
|
|
# job's shell is `bash -e` — so a registry timeout here used to SKIP
|
|
# Build, Type check and vitest and leave a red row that read like the
|
|
# web tests caught something, when nothing had run at all (2026-09-04,
|
|
# main and #1246, two transports, one bad window). The script decides
|
|
# from the JSON report: real advisories fail the step and are named; an
|
|
# unreachable service retries, then fails under a DISTINCT title saying
|
|
# the gate did not run — closed, not open, because a gate that passes
|
|
# when it cannot run is not a gate. Running last means the frontend's
|
|
# own verdict always exists, whichever way the audit goes.
|
|
run: npm run audit:ci
|
|
|
|
e2e:
|
|
name: E2E ${{ matrix.label }}
|
|
runs-on: ubuntu-latest
|
|
# IDEA-3189's tripwire fired on 2026-10-05: the median of the last 10
|
|
# green E2E jobs was 16.5m (13.7-19.3m) against its 14m threshold, and a
|
|
# green-on-rerun job (#1811) was cancelled at the 20m cap with no failing
|
|
# test. So the suite is split across jobs, each building for itself.
|
|
#
|
|
# The cut is per PROJECT, then desktop in halves. A bare `--shard=N/2`
|
|
# does not balance: Playwright cuts the ordered test list by COUNT and the
|
|
# projects run in sequence, so with both projects the same size each half
|
|
# was exactly one project (measured on this unit's first run: shard 2/2
|
|
# ran mobile-chromium alone, 63 passed and 553 skipped, in 1.9m). And the
|
|
# projects are lopsided: desktop-chromium carried 1222s of the 1359s of
|
|
# test time (572 of 635 passing tests). Desktop is therefore sharded in
|
|
# two and mobile runs whole. The "E2E (Playwright)" job below is the one
|
|
# check that says whether E2E passed.
|
|
strategy:
|
|
fail-fast: false
|
|
matrix:
|
|
include:
|
|
- label: desktop 1/2
|
|
project: desktop-chromium
|
|
shard: 1/2
|
|
artifact: desktop-1
|
|
- label: desktop 2/2
|
|
project: desktop-chromium
|
|
shard: 2/2
|
|
artifact: desktop-2
|
|
- label: mobile
|
|
project: mobile-chromium
|
|
shard: 1/1
|
|
artifact: mobile
|
|
# Build the binary + UI once and reuse across Playwright projects.
|
|
# The cap is a runaway sanity check, not a perf budget. It covers
|
|
# build-web + build-binary + install-playwright + the suite. A single
|
|
# flaky-test retry (~+2m) past the cap CANCELS the job mid-suite while
|
|
# every test passed: a required gate going red with zero code
|
|
# involvement (BUG-2645 mechanism b). History:
|
|
# - 10 -> 15 (#1158): a green job took 7m54s. Evidence pair: an 8m10s
|
|
# clean rerun vs a 10m18s cancel at the same SHA.
|
|
# - 15 -> 20 (BUG-3188): measured 2026-09-24, green E2E median 10.9m
|
|
# (9.3-11.7m over 5 runs) as the suite grew ~190 -> ~280 tests. The
|
|
# growth is new specs; tests present in both runs were not slower.
|
|
# Raising the cap treats the symptom. The tripwire is IDEA-3189: shard
|
|
# or trim when the median of the last 10 green E2E jobs passes 14m (the
|
|
# measurement recipe is on that item). Still kills a genuine hang.
|
|
timeout-minutes: 20
|
|
steps:
|
|
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
|
|
|
- uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e # v7.0.0
|
|
with:
|
|
go-version: "1.26"
|
|
|
|
- name: Allow Go to fetch the toolchain go.mod pins
|
|
# See the identical step in the `go` job — setup-go pins
|
|
# GOTOOLCHAIN=local, blocking the toolchain fetch go.mod requires
|
|
# (`go 1.26.5` floor, `toolchain go1.26.6` per BUG-2565; the
|
|
# "Build pad binary" step below fails without this).
|
|
run: echo "GOTOOLCHAIN=auto" >> "$GITHUB_ENV"
|
|
|
|
- uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
|
|
with:
|
|
node-version: "24"
|
|
cache: "npm"
|
|
cache-dependency-path: web/package-lock.json
|
|
|
|
- name: Install web dependencies
|
|
working-directory: web
|
|
run: npm ci
|
|
|
|
- name: Build web UI
|
|
working-directory: web
|
|
run: npm run build
|
|
|
|
- name: Build pad binary
|
|
# Web build output is embedded via //go:embed; it must exist before
|
|
# the Go build. The CI `go` job above builds against a placeholder,
|
|
# which is fine for tests — for e2e we need the real embedded UI.
|
|
run: go build -o pad ./cmd/pad
|
|
|
|
- name: Build the e2etest pad binary
|
|
# TASK-3415: the real-install leg starts its own pad from this
|
|
# build, which carries the PAD_E2E_APP_CA door (cmd/pad/e2e_app_ca.go)
|
|
# so it can trust the test app's CA. Only this build has it.
|
|
run: go build -tags e2etest -o pad-e2etest ./cmd/pad
|
|
|
|
- name: The shipped build has no e2e CA door
|
|
# Lead ruling on TASK-3415: no environment variable may widen what a
|
|
# default build trusts. The e2etest build is the positive control:
|
|
# if it lost the string, the first check would pass vacuously.
|
|
run: |
|
|
grep -aq PAD_E2E_APP_CA pad-e2etest || { echo "control: the e2etest build has no PAD_E2E_APP_CA"; exit 1; }
|
|
if grep -aq PAD_E2E_APP_CA pad; then echo "the default build contains PAD_E2E_APP_CA"; exit 1; fi
|
|
|
|
# Playwright browser binaries are downloaded from cdn.playwright.dev,
|
|
# which occasionally hangs (PR #635 hit two consecutive 10-minute
|
|
# timeouts during a CDN slow patch — see BUG-1625). Cache them per
|
|
# @playwright/test version so warm-cache runs skip the ~150 MB
|
|
# Chromium download entirely; only the apt system libraries
|
|
# (libatk, libnss, libcups, …) need a fresh install, which is
|
|
# ~10s on a healthy runner. Cache key is the resolved version from
|
|
# package-lock.json so a Playwright bump auto-invalidates.
|
|
- name: Get Playwright version
|
|
id: playwright-version
|
|
working-directory: web
|
|
run: echo "version=$(node -p "require('@playwright/test/package.json').version")" >> $GITHUB_OUTPUT
|
|
|
|
- name: Cache Playwright browsers
|
|
id: playwright-cache
|
|
uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
|
|
with:
|
|
path: ~/.cache/ms-playwright
|
|
key: playwright-${{ runner.os }}-${{ steps.playwright-version.outputs.version }}-chromium
|
|
|
|
- name: Install Playwright browsers (cache miss)
|
|
if: steps.playwright-cache.outputs.cache-hit != 'true'
|
|
working-directory: web
|
|
# --with-deps pulls in the Ubuntu libraries Playwright needs
|
|
# (libatk, libnss, libcups, …). Scoped to chromium to cut download
|
|
# time — the suite's mobile project uses Pixel 7, which defaults to
|
|
# Chromium, so we don't need WebKit.
|
|
run: npx playwright install --with-deps chromium
|
|
|
|
- name: Install Playwright system deps (cache hit)
|
|
if: steps.playwright-cache.outputs.cache-hit == 'true'
|
|
working-directory: web
|
|
# When the browser binary cache hits, we still need the apt-level
|
|
# system libraries on the fresh runner — `install-deps` does just
|
|
# that without re-downloading the browser binary itself.
|
|
run: npx playwright install-deps chromium
|
|
|
|
- name: Run Playwright
|
|
working-directory: web
|
|
env:
|
|
CI: "1"
|
|
PAD_E2E_BINARY: ${{ github.workspace }}/pad-e2etest
|
|
run: npx playwright test --project=${{ matrix.project }} --shard=${{ matrix.shard }}
|
|
|
|
- name: Upload Playwright report on failure
|
|
if: failure()
|
|
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
|
|
with:
|
|
name: playwright-report-${{ matrix.artifact }}
|
|
path: web/playwright-report/
|
|
retention-days: 14
|
|
|
|
# A test that fails and then passes on retry leaves the job green, so the
|
|
# upload above never runs and the flake ships no screenshot, trace or
|
|
# video, though retain-on-failure kept them in test-results (BUG-3278).
|
|
# Upload them on a green run only when a failed attempt left a trace; a
|
|
# run with no flakes uploads nothing.
|
|
- name: Upload flaky-test evidence
|
|
if: success() && hashFiles('web/test-results/**/trace.zip') != ''
|
|
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
|
|
with:
|
|
name: playwright-flaky-evidence-${{ matrix.artifact }}
|
|
path: web/test-results/
|
|
retention-days: 7
|
|
|
|
# The single E2E verdict. Every reader of CI (merge rules, waiters, people)
|
|
# keys on the check named "E2E (Playwright)", and the repo has no branch
|
|
# protection to list the shard jobs by name, so this job keeps that name and
|
|
# is green only when EVERY matrix job succeeded. `if: always()` is load-bearing:
|
|
# without it a failed or cancelled matrix job SKIPS this job, and a skipped check
|
|
# reads as neither red nor missing.
|
|
e2e-result:
|
|
name: E2E (Playwright)
|
|
needs: e2e
|
|
if: always()
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 2
|
|
steps:
|
|
- name: Every E2E job succeeded
|
|
env:
|
|
SHARDS: ${{ needs.e2e.result }}
|
|
run: |
|
|
echo "e2e matrix result: $SHARDS"
|
|
test "$SHARDS" = "success"
|