mirror of
https://github.com/PerpetualSoftware/pad.git
synced 2026-09-10 15:05:40 +00:00
xarmian
f9195c5b09
* ci: make the go test timeout explicit everywhere (TASK-2545) The v0.13.0 release pre-flight died on `panic: test timed out after 10m0s` in internal/store, on a commit whose Go tree was identical to a green run an hour earlier. Nothing hung — the package's runtime simply crossed a budget nobody had chosen. `go test` without -timeout uses a 10m per-test-binary default. This repo raised the two RACE steps to 45m twice as the suite grew (BUG-1371 30m, BUG-1913 30m→45m), each time with a careful comment — and each time left their non-race siblings on the silent default. Three steps were still running on it, including the release gate: ci.yml "Run tests" (SQLite) ci.yml "Run tests against PostgreSQL" (the one that panicked) release.yml "Run tests" (the release gate itself) All three now carry -timeout=45m, matching the race legs so the file has one number, with comments saying it is a hang-catcher rather than a performance budget and that job wall-clock is the signal for "the suite got slow". Measured at212d59e7on a dev box, both drivers, before and after: PostgreSQL whole suite 4m43s wall; internal/store 280s; server 103s SQLite whole suite 1m52s wall; internal/server 107s; store 64s CI runners are roughly 2x slower, which is what put store's PG binary over 10m. 45m is ~4.5x current CI headroom. This raises the ceiling; it does not change the slope. internal/store on PG costs ~0.43s per test in database setup alone (CREATE DATABASE plus a full migration replay, where the SQLite harness copies a pre-migrated template — IDEA-1914), so every test added costs PG CI ~0.43s forever and that package is 99% of the job's critical path. Measured and filed as IDEA-2550 rather than fixed here: it changes shared test infrastructure that gates every merge and deserves its own review. Verified by running the exact post-change commands on both drivers: PG green in 4m42s, SQLite green in 1m52s, 25 packages each. Claude-Session: https://claude.ai/code/session_01QGbUKZBAZoWdEgiTNWsXag * ci: time the Makefile's go test targets too (TASK-2545) The previous commit said the timeout was explicit "everywhere" and it wasn't — `make test`, `make test-pg`, and `make check` were all still on the 10m default. That matters twice over: it's the same trap the commit is about, and `make test-pg` is the local mirror of the CI leg that actually panicked, so a developer reproducing the failure would have hit a different budget than the one they were debugging. Found by sweeping every `go test` in the repo rather than only the workflows — which is what the commit message's own claim required and I hadn't done when I wrote it. Verified: `make test` green, 25 packages. Claude-Session: https://claude.ai/code/session_01QGbUKZBAZoWdEgiTNWsXag * ci: time the nix checkPhase, cap the Go jobs, correct two claims (TASK-2545) Codex review. No P1s; the two P2s were both right and one of them catches me stating an explanation I had not checked. COVERAGE. `nix/package.nix`'s checkPhase runs `go test ./...` on the default too, and .github/workflows/nix.yml exercises it — a fourth site after the three workflow steps and the three Makefile targets. Now timed. Every `go test` invocation in the repo carries an explicit -timeout; the sweep is `grep -rn "go test"` over workflows, Makefile and nix, not just the workflows I happened to be looking at. JOB CAPS. Codex objected that 45m lets a hung binary burn a release-gating job. Fair, and the real hole was worse: `go` and `go-postgres` had NO `timeout-minutes`, so they inherit GitHub's 6-HOUR default. Both now capped at 100m — deliberately above the two 45m test steps so the per-binary timeout always fires first, because that is the one that prints the goroutine dump naming the hung test. The cap only catches a runaway that isn't a single test (wedged service container, stuck download). CORRECTIONS to 496f521f's message: - It said the race steps were raised "twice (BUG-1371 30m, BUG-1913 30m→45m)". BUG-1371 kept 30m and fixed the bcrypt cost that had blown past it; BUG-1913 made the only 30m→45m change. One raise, not two. - It said CI runners are "roughly 2x slower, which is what put store's PG binary over 10m". That does not survive its own arithmetic: 280s local x 2 is 9m20s, under the budget. What is actually known is that the CI binary exceeded 10m and the local one takes 280s, so CI is >2.14x slower on that binary — a lower bound derived from the failure, not an explanation of it. I have not measured CI's runtime and should not have written a factor as if I had. - "Nothing hung" and "cost the cut ~40 minutes" are TASK-2545's findings from the goroutine dump and the release timeline, not mine. Attributed rather than restated as my own observation. The 0.43s per-test setup figure and both driver runtimes are mine, taken on this box at212d59e7and reproducible with the commands in IDEA-2550. Claude-Session: https://claude.ai/code/session_01QGbUKZBAZoWdEgiTNWsXag * ci: put the corrections in the file, not only in a commit message (TASK-2545) Codex's re-review came back with no P1s or P2s and four nits, all the same shape: the claims I retracted in 4623cae9's COMMIT MESSAGE were still sitting in the workflow comments. That's the half that matters — nobody reads a commit message while editing a CI file, and a correction that lives only in git log is a correction almost nobody receives. Fixed in place: - The raise history: BUG-1913 raised 30m→45m once. BUG-1371 kept 30m and dropped the test-only bcrypt cost that had blown past it. My comment said "raised twice (BUG-1371, BUG-1913)". - The pre-existing race-step comment claiming BUG-1371 kept the step "well under the 30m budget" — contradicted by BUG-1913 having to raise it later. Reworded to say what each change actually did. Not my text, but it is wrong in the file I am editing and the next reader inherits it either way. - The "~2x slower, which put store over 10m" line, which its own arithmetic refutes (280s x 2 = 9m20s). Now states the lower bound the failure actually supports — CI's store binary exceeded 10m, so >2.14x this box — and names the retracted claim so a reader who saw the old version knows it was withdrawn rather than lost. - "so it never fires before they do" on the job caps, which a job-level timeout cannot promise: it covers setup and every step, not just the two 45m ones. Now says "in practice", not a guarantee. Attribution of TASK-2545's own findings (the ~40 minutes, the goroutine dump showing nothing hung) moved into the comment too. Claude-Session: https://claude.ai/code/session_01QGbUKZBAZoWdEgiTNWsXag
174 lines
7.3 KiB
Makefile
174 lines
7.3 KiB
Makefile
.PHONY: build test test-pg test-pg-down dev clean web dev-web serve restart lint install check vuln web-check web-test
|
|
|
|
BINARY=pad
|
|
BUILD_DIR=./cmd/pad
|
|
HOST?=127.0.0.1
|
|
INSTALL_DIR?=$(HOME)/.local/bin
|
|
|
|
VERSION ?= dev
|
|
COMMIT := $(shell git rev-parse --short HEAD 2>/dev/null)
|
|
BUILD_TIME := $(shell date -u +%Y-%m-%dT%H:%M:%SZ)
|
|
LDFLAGS := -X main.version=$(VERSION) -X main.commit=$(COMMIT) -X main.buildTime=$(BUILD_TIME)
|
|
|
|
# Pin to the same golangci-lint and govulncheck versions CI runs (see
|
|
# .github/workflows/ci.yml). Bump these and CI together when upgrading.
|
|
GOLANGCI_LINT_VERSION ?= v2.11.4
|
|
GOLANGCI_LINT := $(shell go env GOPATH)/bin/golangci-lint
|
|
GOVULNCHECK_VERSION ?= v1.2.0
|
|
GOVULNCHECK := $(shell go env GOPATH)/bin/govulncheck
|
|
|
|
build: web
|
|
go build -ldflags "$(LDFLAGS)" -o $(BINARY) $(BUILD_DIR)
|
|
|
|
build-go:
|
|
go build -ldflags "$(LDFLAGS)" -o $(BINARY) $(BUILD_DIR)
|
|
|
|
install: build
|
|
@# Stop running server, install binary, clear stale pid.
|
|
@# CAUTION: `pkill -x pad` is SYSTEM-WIDE (matches the binary name on the
|
|
@# whole host). If another user or project on the same machine is running
|
|
@# a `pad` process, it will get signaled too. Designed for single-developer
|
|
@# local setups; don't run `make install` on a shared host.
|
|
@#
|
|
@# SIGTERM (not SIGKILL) so the server's graceful-shutdown path runs:
|
|
@# it closes the event bus, which terminates SSE handler goroutines so
|
|
@# the http.Server can write the final 0-chunk before closing each
|
|
@# stream. SIGKILL drops every open SSE connection mid-write, leaving
|
|
@# every browser tab with `ERR_INCOMPLETE_CHUNKED_ENCODING` and a noisy
|
|
@# reconnect storm. Falls back to SIGKILL after 5s for stuck processes.
|
|
@# BUG-1531 / SSE follow-up.
|
|
-pkill -TERM -x $(BINARY) 2>/dev/null; \
|
|
for i in 1 2 3 4 5; do \
|
|
pgrep -x $(BINARY) >/dev/null 2>&1 || break; \
|
|
sleep 1; \
|
|
done; \
|
|
pkill -KILL -x $(BINARY) 2>/dev/null || true
|
|
@mkdir -p $(INSTALL_DIR)
|
|
cp -f $(BINARY) $(INSTALL_DIR)/$(BINARY)
|
|
rm -f ~/.pad/pad.pid
|
|
@echo "Installed $(BINARY) to $(INSTALL_DIR)/$(BINARY)"
|
|
@# Trigger server auto-start by running a command
|
|
@$(INSTALL_DIR)/$(BINARY) auth whoami 2>/dev/null || true
|
|
@echo "Server restarted."
|
|
|
|
# -timeout matches CI (see .github/workflows/ci.yml). Without it `go test`
|
|
# uses a 10m per-test-binary default nobody chose — the shape that killed
|
|
# the v0.13.0 release pre-flight (TASK-2545).
|
|
test:
|
|
go test -timeout=45m ./... -v
|
|
|
|
# Run tests against PostgreSQL (starts a container automatically).
|
|
# Uses port 5445 to avoid conflicts with any local PostgreSQL.
|
|
test-pg:
|
|
docker compose -f docker-compose.test.yml up -d --wait
|
|
PAD_TEST_POSTGRES_URL="postgres://pad:pad@localhost:5445/pad?sslmode=disable" go test -timeout=45m ./... -v -count=1; \
|
|
EXIT_CODE=$$?; \
|
|
docker compose -f docker-compose.test.yml down -v; \
|
|
exit $$EXIT_CODE
|
|
|
|
test-pg-down:
|
|
docker compose -f docker-compose.test.yml down -v
|
|
|
|
dev: build-go
|
|
./$(BINARY) server start --host $(HOST)
|
|
|
|
serve: build
|
|
-./$(BINARY) server stop 2>/dev/null
|
|
@sleep 1
|
|
./$(BINARY) server start --host $(HOST)
|
|
|
|
restart: build-go
|
|
-./$(BINARY) server stop 2>/dev/null
|
|
@sleep 1
|
|
./$(BINARY) server start --host $(HOST)
|
|
|
|
web:
|
|
cd web && npm ci && npm run build
|
|
|
|
dev-web:
|
|
cd web && npm run dev
|
|
|
|
clean:
|
|
rm -f $(BINARY)
|
|
rm -rf web/build
|
|
go clean ./...
|
|
|
|
# Run the same golangci-lint suite CI runs (.golangci.yml: govet,
|
|
# ineffassign, staticcheck SA*, unused, plus the gofmt formatter with
|
|
# simplify: true). The lint suite already includes go vet via the govet
|
|
# linter, so we don't double-run it here.
|
|
#
|
|
# Version enforcement: the recipe checks the installed binary against
|
|
# GOLANGCI_LINT_VERSION and reinstalls on mismatch. A file-target
|
|
# dependency wouldn't enforce the pin — make only runs the install rule
|
|
# when the binary is missing, so an outdated local binary would be
|
|
# silently reused and disagree with CI (Codex review on PR #322).
|
|
lint:
|
|
@bin="$(GOLANGCI_LINT)"; pin="$(GOLANGCI_LINT_VERSION)"; want="$${pin#v}"; \
|
|
have=$$( $$bin version 2>/dev/null | sed -n 's/.*version \([0-9.]*\) built.*/\1/p' ); \
|
|
if [ "$$have" != "$$want" ]; then \
|
|
echo "Installing golangci-lint $$pin (had: $${have:-none})..."; \
|
|
go install github.com/golangci/golangci-lint/v2/cmd/golangci-lint@$$pin; \
|
|
fi
|
|
$(GOLANGCI_LINT) run --timeout=5m ./...
|
|
|
|
# Run govulncheck in BINARY mode against a freshly-built pad binary, NOT
|
|
# source mode (`govulncheck ./...`). Source mode builds an SSA call-graph
|
|
# over the entire dependency tree (BigQuery / OTel / gRPC / Google Cloud),
|
|
# which balloons to multiple GB of RAM and can lock up a memory-constrained
|
|
# host (BUG-2084). Binary mode reads the compiled binary's symbol table
|
|
# instead: a fraction of the memory, still call-graph-precise (it walks the
|
|
# binary's symbol graph), and it detects stdlib vulns from the Go version
|
|
# stamped in the binary. Because `pad` is a single binary containing the
|
|
# whole codebase (server + CLI), scanning it covers everything; not-called
|
|
# module vulns are naturally suppressed since their symbols aren't linked in.
|
|
# Mirrors the "Run govulncheck" step in CI's Go job — keep the two in sync.
|
|
#
|
|
# The build needs web/build to exist for the //go:embed directive. Locally
|
|
# `make web` / `make install` provides the real assets; the guard below
|
|
# drops a placeholder when it's absent (e.g. a fresh clone) so a standalone
|
|
# `make vuln` never fails on the embed. `go install foo@vX.Y.Z` is idempotent
|
|
# and rebuilds quickly when the pinned version is already cached.
|
|
#
|
|
# The scan binary is written to the repo root (real disk), NOT /tmp: some
|
|
# hosts mount /tmp as a small tmpfs (RAM-backed), where a large embedded
|
|
# binary can hit "no space left" and, worse, consume the very RAM we're
|
|
# trying not to exhaust (BUG-2084). It's removed on completion and
|
|
# .gitignore'd so an interrupted run can't leave a tracked artifact.
|
|
vuln:
|
|
go install golang.org/x/vuln/cmd/govulncheck@$(GOVULNCHECK_VERSION)
|
|
@[ -n "$$(ls -A web/build 2>/dev/null)" ] || { mkdir -p web/build && echo placeholder > web/build/.gitkeep; }
|
|
go build -o pad-vulnscan ./cmd/pad
|
|
$(GOVULNCHECK) -mode binary pad-vulnscan; status=$$?; rm -f pad-vulnscan; exit $$status
|
|
|
|
# Web pre-flight that mirrors CI's Web job beyond the build step:
|
|
# `npm audit` (production dependencies, high severity+) and svelte-check
|
|
# type checking. Depends on `web` so npm ci + build are already done.
|
|
# Separate target so a contributor iterating on the UI can run just the
|
|
# extra checks via `make web-check`.
|
|
web-check: web
|
|
cd web && npm audit --audit-level=high --omit=dev && npm run check
|
|
|
|
# Run the web unit-test suite (vitest, non-watch). Mirrors the "Run web unit
|
|
# tests" step in CI's Web job. Kept separate from web-check so a contributor
|
|
# can run just the vitest suite via `make web-test`; `check` invokes both.
|
|
web-test:
|
|
cd web && npm run test
|
|
|
|
# Pre-flight target that mirrors CI's Go and Web jobs. Run this before
|
|
# pushing — if it passes, the corresponding CI checks should pass too.
|
|
#
|
|
# Covers: golangci-lint suite (lint), Go test suite, govulncheck, npm ci,
|
|
# npm audit, web build, svelte-check, vitest unit tests. The race-detector +
|
|
# Postgres jobs only run on push to main (per .github/workflows/ci.yml) and
|
|
# are not included here; run `make test-pg` separately if you want them
|
|
# locally.
|
|
#
|
|
# `make install` stays lightweight (build + restart only) so the inner
|
|
# dev loop is fast; opt into `check` when you're ready to push.
|
|
check: lint
|
|
go test -timeout=45m ./...
|
|
$(MAKE) vuln
|
|
$(MAKE) web-check
|
|
$(MAKE) web-test
|