mirror of
https://github.com/PerpetualSoftware/pad.git
synced 2026-10-25 14:00:40 +00:00
xarmian
fa2755e183
* feat(materialize): headless op-log materializer bundle, goja runner and worker (TASK-2198 U2)
Builds the editor's own JS (Yjs -> y-tiptap -> Tiptap schema -> tiptap-markdown
-> the tab's flush pipeline) into one IIFE, embeds it, and runs it in goja. Not
wired into the server yet (U3 supervisor, U4 write path).
- web/src/lib/collab/materializer/: bundle entry (extension list mirrors
Editor.svelte's collab branch via the U1 shared modules), op-log replay
mirroring wsProvider's MESSAGE_SYNC arm, linkedom DOM shim and polyfills
from the spike, and a vendored patch of prosemirror-markdown
MarkdownSerializerState.atBlank (quadratic under goja; equivalent).
- web/scripts/build-materializer.mjs (esbuild) -> web/build-materializer/,
run by `npm run build`; embedded as pad.MaterializerJS.
- internal/materialize: Runner (Go-native atob/btoa/TextEncoder/TextDecoder/
getRandomValues, schema-version check, ctx interrupt with a clean VM after),
and RunWorker's length-prefixed JSON protocol behind the hidden
`pad __materialize-worker` command.
- Parity: vitest drives two real Editor.svelte mounts and compares
materialize() with the tab's flush on 50 seeded cases; the same cases are
committed as testdata/corpus.json and replayed in goja (TestCorpusParity).
- CI Go jobs, release tests, Docker and Nix produce the bundle; linkedom and
esbuild are new devDependencies.
Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC
* test(materialize): pin the atBlank patch's equivalence under V8 too (TASK-2198 U2, codex r1)
Codex r1 claimed upstream's /(^|\n)$/ is true for a trailing CR in JS. It is
not (checked in V8: false for "\r" and "a\u2028", as the patch returns),
but the only equivalence test ran under goja and the patch header cited a
V8-side test that did not exist. This adds it, with the line-terminator edge
cases, and it fails when the patch checks CR instead of LF.
Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC
* build(nix): vendorHash for goja (TASK-2198 U2)
Taken from the Nix job's recompute on c7fa1931 (specified sha256-iEqv…,
got sha256-P1bQ…), so main does not depend on the heal job after merge.
Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC
354 lines
18 KiB
Makefile
354 lines
18 KiB
Makefile
.PHONY: build test test-nix-hash test-pg test-pg-down test-pg-project dev clean web dev-web serve restart lint install check vuln web-check web-test web-audit
|
|
|
|
BINARY=pad
|
|
BUILD_DIR=./cmd/pad
|
|
HOST?=127.0.0.1
|
|
INSTALL_DIR?=$(HOME)/.local/bin
|
|
|
|
VERSION ?= dev
|
|
COMMIT := $(shell git rev-parse --short HEAD 2>/dev/null)
|
|
BUILD_TIME := $(shell date -u +%Y-%m-%dT%H:%M:%SZ)
|
|
LDFLAGS := -X main.version=$(VERSION) -X main.commit=$(COMMIT) -X main.buildTime=$(BUILD_TIME)
|
|
|
|
# Pin to the same golangci-lint and govulncheck versions CI runs (see
|
|
# .github/workflows/ci.yml). Bump these and CI together when upgrading.
|
|
GOLANGCI_LINT_VERSION ?= v2.11.4
|
|
GOLANGCI_LINT := $(shell go env GOPATH)/bin/golangci-lint
|
|
GOVULNCHECK_VERSION ?= v1.2.0
|
|
GOVULNCHECK := $(shell go env GOPATH)/bin/govulncheck
|
|
|
|
build: web
|
|
go build -ldflags "$(LDFLAGS)" -o $(BINARY) $(BUILD_DIR)
|
|
|
|
build-go:
|
|
go build -ldflags "$(LDFLAGS)" -o $(BINARY) $(BUILD_DIR)
|
|
|
|
install: build
|
|
@# Delegated to scripts/install-refresh.sh (BUG-2897, TASK-2787). The
|
|
@# logic lives in a script rather than in this recipe for one reason
|
|
@# above readability: a script can be TESTED. install_refresh_test.go
|
|
@# drives it against a stub `pad` on PATH, including the branch where
|
|
@# the probe fails. Recipe-inline logic can only be exercised by running
|
|
@# `make install`, which stops the developer's real server — a test
|
|
@# nobody runs twice, which is how this target accumulated three
|
|
@# unverified claims.
|
|
@#
|
|
@# What the script guarantees, and what this target could not:
|
|
@# - the binary it installs carries THIS invocation's commit, so a
|
|
@# sibling's build sitting at ./pad cannot be installed silently
|
|
@# (TASK-2787);
|
|
@# - the server comes back with the argv of the process it killed,
|
|
@# rather than whatever an auto-start defaults to (BUG-2897), and in
|
|
@# that process's working directory rather than the caller's; a server
|
|
@# whose directory is gone when checked, just before the stop, is
|
|
@# refused with nothing stopped (BUG-3196);
|
|
@# - nothing is printed about a restart until the server answers on
|
|
@# BOTH 127.0.0.1 and the configured host.
|
|
@#
|
|
@# - it replaces one server, stopped by pid: the one listening on the
|
|
@# configured port, or the only one running, started with an
|
|
@# explicit --port, when nothing listens there. Other `pad server
|
|
@# start` processes (a sibling's e2e server on its own port) are
|
|
@# left running, and an ambiguous port is refused (BUG-3194).
|
|
@#
|
|
@# CAUTION: this target's `build` step runs `npm ci`, which deletes a
|
|
@# node_modules that sibling worktrees symlink to. When another
|
|
@# session's worktree is live, build by hand (`npx vite build`, then
|
|
@# `make build-go`) and run scripts/install-refresh.sh directly, per
|
|
@# CONVE-2687.
|
|
@bash scripts/install-refresh.sh $(BINARY) $(INSTALL_DIR)/$(BINARY) $(COMMIT)
|
|
|
|
# -timeout matches CI (see .github/workflows/ci.yml). Without it `go test`
|
|
# uses a 10m per-test-binary default nobody chose — the shape that killed
|
|
# the v0.13.0 release pre-flight (TASK-2545).
|
|
test:
|
|
go test -timeout=45m ./... -v
|
|
|
|
# The vendorHash bump parser (TASK-2954) and the heal job that pushes its output
|
|
# to main (BUG-2974). Pure bash + git, no toolchain, ~3s — their own target
|
|
# rather than part of `test`, which means `go test`. CI runs these same scripts
|
|
# in the Go job so a change to either is gated rather than trusted.
|
|
test-nix-hash:
|
|
@nix/bump-vendor-hash_test.sh
|
|
@nix/heal-vendor-hash_test.sh
|
|
|
|
# Run tests against PostgreSQL (starts a container automatically).
|
|
#
|
|
# THE HOST PORT IS EPHEMERAL (TASK-2708). It was hardcoded to 5445, which let
|
|
# exactly one worktree run this target at a time; with concurrent worktrees the
|
|
# normal operating mode that produced three incidents in an afternoon, the worst
|
|
# of which forged a green-looking gate leg — `go test` exited 2 having run NO
|
|
# TESTS because the container was unreachable, and "exit 2 with zero FAIL lines"
|
|
# reads a lot like a pass to a quick glance.
|
|
#
|
|
# So: Docker assigns the port, we read it back, and we REFUSE TO RUN rather than
|
|
# let an unreachable database look like a test result. Never put a fixed host
|
|
# port back in docker-compose.test.yml or hardcode one here.
|
|
#
|
|
# AN INTERRUPT TEARS THE STACK DOWN TOO. Ctrl-C during `go test` kills this
|
|
# recipe's shell, and without the trap above `down -v` never runs — orphaning
|
|
# exactly the stack this task was filed about. Set before `up`, so an interrupt
|
|
# during startup is covered as well.
|
|
#
|
|
# WHAT THIS DELIBERATELY DOES NOT DO: report a database that dies AFTER a
|
|
# passing run. The post-run banner is gated on the tests having failed, and a
|
|
# reviewer asked for that gate to be dropped. It stays, because the premise
|
|
# does not hold — storetest.NewPostgres skips only when PAD_TEST_POSTGRES_URL
|
|
# is EMPTY, and a database that is gone produces t.Fatalf, not a skip. So under
|
|
# this target a dead database always fails the tests that touch it, and exit 0
|
|
# means every Postgres-backed test completed against a live one. Failing the
|
|
# leg on a container that stopped after the suite finished would turn honest
|
|
# greens red.
|
|
#
|
|
# THE READINESS PROBE TESTS THE HOST-PUBLISHED PORT, which is the path the
|
|
# tests take — not the container's own socket. Two ways in, because neither is
|
|
# universal: the host's pg_isready when it exists, otherwise a throwaway
|
|
# container reaching back through host.docker.internal (native on Docker
|
|
# Desktop, and `--add-host=...:host-gateway` makes it resolve on Linux too).
|
|
# An earlier version used `--network host`, which is Linux-only and would have
|
|
# made a healthy database read as unreachable on Desktop; the version after
|
|
# that fell back to `compose exec`, which answers "is the server alive inside
|
|
# the container" and would let a broken port mapping through the guard. This
|
|
# box has no host pg_isready, so the fallback is the branch that actually runs
|
|
# here — it is not a rarely-exercised path.
|
|
#
|
|
# THE BANNER IS THE DISCRIMINATOR, NOT THE EXIT CODE. make collapses every
|
|
# failed recipe to exit 2, so "the database was unreachable" and "tests failed"
|
|
# are indistinguishable by status — measured, not assumed. Do not key automation
|
|
# off the exit code expecting to tell them apart; grep the output for
|
|
# NO TESTS EXECUTED (nothing ran) or THE DATABASE DIED (it ran against a
|
|
# database that went away).
|
|
#
|
|
# Recovering an orphan (a stack whose worktree was removed before teardown):
|
|
# `docker ps --filter name=padtest-` lists them, and the container name carries
|
|
# the project. Reap one from anywhere with
|
|
# docker compose -p <project> down -v
|
|
# From inside the worktree itself, `make test-pg-project` prints the name and
|
|
# `make test-pg-down` does it for you.
|
|
# TEST_PG_PKGS narrows the run. Defaults to everything, which is what a gate
|
|
# wants; a narrower value is for checking this target's own plumbing (e.g. the
|
|
# concurrency acceptance) without two full-suite runs. A gate leg reported from
|
|
# a narrowed run is not a gate leg.
|
|
TEST_PG_PKGS ?= ./...
|
|
|
|
# An EXPLICIT project name, unique per absolute path (codex round 1, P2).
|
|
# Compose otherwise defaults it to the directory BASENAME, so two checkouts
|
|
# that happen to share a basename — /a/docapp and /b/docapp — share a stack,
|
|
# and one `down -v` tears down the other's database mid-run. That is the
|
|
# cross-worktree teardown this task exists to make impossible, reachable
|
|
# through a second door. The basename is kept in the name so an orphan is
|
|
# still identifiable by eye; the checksum of the full path is what makes it
|
|
# unique. Lowercased and punctuation-stripped because compose rejects
|
|
# anything else.
|
|
# $$PWD and pwd rather than $(CURDIR): make interpolates CURDIR into the
|
|
# command TEXT, so a checkout path containing a quote would break the quoting
|
|
# and run whatever followed it. The shell reads its own working directory
|
|
# instead, so no path text is ever parsed as command text. cksum is 32-bit and
|
|
# that is deliberate — it is portable to every platform this repo builds on,
|
|
# unlike sha1sum/shasum, and the basename is in the name too, so the checksum
|
|
# only has to separate same-named siblings rather than be cryptographic.
|
|
# `tr -d '\n'` BEFORE the -c translation, not after: `tr -c` treats the
|
|
# trailing newline basename emits as an invalid character and turns it into a
|
|
# dash, which produced `padtest-docapp-2708--1800854141`. Compose accepted it,
|
|
# so nothing failed — the doubled dash in the printed name is what showed it.
|
|
TEST_PG_PROJECT := padtest-$(shell basename "$$PWD" | tr -d '\n' | tr 'A-Z' 'a-z' | tr -c 'a-z0-9_-' '-')-$(shell pwd | cksum | cut -d' ' -f1)
|
|
COMPOSE_TEST := docker compose -p $(TEST_PG_PROJECT) -f docker-compose.test.yml
|
|
|
|
test-pg:
|
|
@trap 'echo ""; if $(COMPOSE_TEST) down -v >/dev/null 2>&1; then echo "test-pg: INTERRUPTED - stack $(TEST_PG_PROJECT) torn down."; else echo "test-pg: INTERRUPTED and TEARDOWN FAILED - reap it with: docker compose -p $(TEST_PG_PROJECT) down -v"; fi; exit 130' INT TERM; \
|
|
port=""; \
|
|
if ! $(COMPOSE_TEST) up -d --wait; then \
|
|
echo ""; \
|
|
echo "################################################################"; \
|
|
echo "# NO TESTS EXECUTED - the database container never came up #"; \
|
|
echo "# This is NOT a test result. The Postgres leg did not run. #"; \
|
|
echo "################################################################"; \
|
|
$(COMPOSE_TEST) down -v >/dev/null 2>&1 || echo "# ...and TEARDOWN ALSO FAILED: docker compose -p $(TEST_PG_PROJECT) down -v"; \
|
|
exit 1; \
|
|
fi; \
|
|
port=$$($(COMPOSE_TEST) port postgres 5432 2>/dev/null | sed 's/.*://'); \
|
|
if [ -z "$$port" ]; then \
|
|
echo ""; \
|
|
echo "################################################################"; \
|
|
echo "# NO TESTS EXECUTED - could not read the container's host port #"; \
|
|
echo "# This is NOT a test result. The Postgres leg did not run. #"; \
|
|
echo "################################################################"; \
|
|
$(COMPOSE_TEST) down -v >/dev/null 2>&1 || echo "# ...and TEARDOWN ALSO FAILED: docker compose -p $(TEST_PG_PROJECT) down -v"; \
|
|
exit 1; \
|
|
fi; \
|
|
url="postgres://pad:pad@127.0.0.1:$$port/pad?sslmode=disable"; \
|
|
if command -v pg_isready >/dev/null 2>&1; then \
|
|
pg_isready -h 127.0.0.1 -p "$$port" -U pad -q; ready=$$?; \
|
|
else \
|
|
docker run --rm --add-host=host.docker.internal:host-gateway postgres:17-alpine \
|
|
pg_isready -h host.docker.internal -p "$$port" -U pad -q; ready=$$?; \
|
|
fi; \
|
|
if [ $$ready -ne 0 ]; then \
|
|
echo ""; \
|
|
echo "################################################################"; \
|
|
echo "# NO TESTS EXECUTED - Postgres on port $$port is not ready #"; \
|
|
echo "# This is NOT a test result. The Postgres leg did not run. #"; \
|
|
echo "################################################################"; \
|
|
$(COMPOSE_TEST) down -v >/dev/null 2>&1 || echo "# ...and TEARDOWN ALSO FAILED: docker compose -p $(TEST_PG_PROJECT) down -v"; \
|
|
exit 1; \
|
|
fi; \
|
|
echo "test-pg: Postgres on 127.0.0.1:$$port (project $(TEST_PG_PROJECT))"; \
|
|
PAD_TEST_POSTGRES_URL="$$url" go test -timeout=45m $(TEST_PG_PKGS) -v -count=1; \
|
|
EXIT_CODE=$$?; \
|
|
if command -v pg_isready >/dev/null 2>&1; then \
|
|
pg_isready -h 127.0.0.1 -p "$$port" -U pad -q; still_up=$$?; \
|
|
else \
|
|
docker run --rm --add-host=host.docker.internal:host-gateway postgres:17-alpine \
|
|
pg_isready -h host.docker.internal -p "$$port" -U pad -q; still_up=$$?; \
|
|
fi; \
|
|
if [ $$EXIT_CODE -ne 0 ] && [ $$still_up -ne 0 ]; then \
|
|
echo ""; \
|
|
echo "################################################################"; \
|
|
echo "# THE DATABASE DIED DURING THE RUN. #"; \
|
|
echo "# Treat the failures above as INFRASTRUCTURE, not as evidence #"; \
|
|
echo "# about the code, and re-run before drawing any conclusion. #"; \
|
|
echo "################################################################"; \
|
|
fi; \
|
|
if ! $(COMPOSE_TEST) down -v; then \
|
|
echo ""; \
|
|
echo "################################################################"; \
|
|
echo "# TEARDOWN FAILED - the stack is still running. Reap it with: #"; \
|
|
echo "# docker compose -p $(TEST_PG_PROJECT) down -v"; \
|
|
echo "# The test status below is honest; this leak is a separate fact. #"; \
|
|
echo "################################################################"; \
|
|
fi; \
|
|
exit $$EXIT_CODE
|
|
|
|
# Tears down THIS directory's stack only: the project name is keyed to this
|
|
# directory's ABSOLUTE path, so it cannot reach a sibling's container even if
|
|
# the two directories share a basename.
|
|
test-pg-down:
|
|
$(COMPOSE_TEST) down -v
|
|
|
|
# Prints this directory's compose project name, so an orphan can be reaped
|
|
# from anywhere without re-deriving it by hand.
|
|
test-pg-project:
|
|
@echo $(TEST_PG_PROJECT)
|
|
|
|
dev: build-go
|
|
./$(BINARY) server start --host $(HOST)
|
|
|
|
serve: build
|
|
-./$(BINARY) server stop 2>/dev/null
|
|
@sleep 1
|
|
./$(BINARY) server start --host $(HOST)
|
|
|
|
restart: build-go
|
|
-./$(BINARY) server stop 2>/dev/null
|
|
@sleep 1
|
|
./$(BINARY) server start --host $(HOST)
|
|
|
|
web:
|
|
cd web && npm ci && npm run build
|
|
|
|
dev-web:
|
|
cd web && npm run dev
|
|
|
|
clean:
|
|
rm -f $(BINARY)
|
|
rm -rf web/build
|
|
go clean ./...
|
|
|
|
# Run the same golangci-lint suite CI runs (.golangci.yml: govet,
|
|
# ineffassign, staticcheck SA*, unused, plus the gofmt formatter with
|
|
# simplify: true). The lint suite already includes go vet via the govet
|
|
# linter, so we don't double-run it here.
|
|
#
|
|
# Version enforcement: the recipe checks the installed binary against
|
|
# GOLANGCI_LINT_VERSION and reinstalls on mismatch. A file-target
|
|
# dependency wouldn't enforce the pin — make only runs the install rule
|
|
# when the binary is missing, so an outdated local binary would be
|
|
# silently reused and disagree with CI (Codex review on PR #322).
|
|
lint:
|
|
@bin="$(GOLANGCI_LINT)"; pin="$(GOLANGCI_LINT_VERSION)"; want="$${pin#v}"; \
|
|
have=$$( $$bin version 2>/dev/null | sed -n 's/.*version \([0-9.]*\) built.*/\1/p' ); \
|
|
if [ "$$have" != "$$want" ]; then \
|
|
echo "Installing golangci-lint $$pin (had: $${have:-none})..."; \
|
|
go install github.com/golangci/golangci-lint/v2/cmd/golangci-lint@$$pin; \
|
|
fi
|
|
$(GOLANGCI_LINT) run --timeout=5m ./...
|
|
|
|
# Run govulncheck in BINARY mode against a freshly-built pad binary, NOT
|
|
# source mode (`govulncheck ./...`). Source mode builds an SSA call-graph
|
|
# over the entire dependency tree (BigQuery / OTel / gRPC / Google Cloud),
|
|
# which balloons to multiple GB of RAM and can lock up a memory-constrained
|
|
# host (BUG-2084). Binary mode reads the compiled binary's symbol table
|
|
# instead: a fraction of the memory, still call-graph-precise (it walks the
|
|
# binary's symbol graph), and it detects stdlib vulns from the Go version
|
|
# stamped in the binary. Because `pad` is a single binary containing the
|
|
# whole codebase (server + CLI), scanning it covers everything; not-called
|
|
# module vulns are naturally suppressed since their symbols aren't linked in.
|
|
# Mirrors the "Run govulncheck" step in CI's Go job — keep the two in sync.
|
|
#
|
|
# The build needs web/build (and web/build-materializer/materializer.js,
|
|
# TASK-2198) to exist for the //go:embed directives. Locally
|
|
# `make web` / `make install` provides the real assets; the guard below
|
|
# drops a placeholder when it's absent (e.g. a fresh clone) so a standalone
|
|
# `make vuln` never fails on the embed. `go install foo@vX.Y.Z` is idempotent
|
|
# and rebuilds quickly when the pinned version is already cached.
|
|
#
|
|
# The scan binary is written to the repo root (real disk), NOT /tmp: some
|
|
# hosts mount /tmp as a small tmpfs (RAM-backed), where a large embedded
|
|
# binary can hit "no space left" and, worse, consume the very RAM we're
|
|
# trying not to exhaust (BUG-2084). It's removed on completion and
|
|
# .gitignore'd so an interrupted run can't leave a tracked artifact.
|
|
vuln:
|
|
go install golang.org/x/vuln/cmd/govulncheck@$(GOVULNCHECK_VERSION)
|
|
@[ -n "$$(ls -A web/build 2>/dev/null)" ] || { mkdir -p web/build && echo placeholder > web/build/.gitkeep; }
|
|
@[ -f web/build-materializer/materializer.js ] || { mkdir -p web/build-materializer && echo '// placeholder' > web/build-materializer/materializer.js; }
|
|
go build -o pad-vulnscan ./cmd/pad
|
|
$(GOVULNCHECK) -mode binary pad-vulnscan; status=$$?; rm -f pad-vulnscan; exit $$status
|
|
|
|
# Web pre-flight that mirrors CI's Web job beyond the build step: svelte-check
|
|
# type checking. Depends on `web` so npm ci + build are already done.
|
|
# Separate target so a contributor iterating on the UI can run just the
|
|
# extra checks via `make web-check`.
|
|
web-check: web
|
|
cd web && npm run check
|
|
|
|
# Run the web unit-test suite (vitest, non-watch). Mirrors the "Run web unit
|
|
# tests" step in CI's Web job. Kept separate from web-check so a contributor
|
|
# can run just the vitest suite via `make web-test`; `check` invokes both.
|
|
web-test:
|
|
cd web && npm run test
|
|
|
|
# `npm audit` (production dependencies, high severity+), through the same
|
|
# gate CI uses — web/scripts/ci-audit.mjs — and, like CI, LAST (BUG-2881).
|
|
# Bare `npm audit` exits non-zero the same way for "an advisory exists" and
|
|
# "the advisory service was unreachable", and with `&&` in front of the other
|
|
# checks a registry blip used to stop svelte-check and vitest from running at
|
|
# all. The script tells the two apart, retries the second, and fails closed
|
|
# under its own title; running it after the correctness checks means their
|
|
# verdict exists whichever way it goes.
|
|
#
|
|
# NO `web` prerequisite (codex round 4 on #1247): `npm audit` reads the
|
|
# lockfile and needs neither node_modules nor a build, and `web` is .PHONY,
|
|
# so depending on it made `check` run `npm ci` twice and made this the one
|
|
# new target to reach `npm ci` — the command CLAUDE.md forbids in a worktree
|
|
# with a symlinked node_modules. Standalone, it is safe to run anywhere.
|
|
web-audit:
|
|
cd web && npm run audit:ci
|
|
|
|
# Pre-flight target that mirrors CI's Go and Web jobs. Run this before
|
|
# pushing — if it passes, the corresponding CI checks should pass too.
|
|
#
|
|
# Covers: golangci-lint suite (lint), Go test suite, govulncheck, npm ci,
|
|
# web build, svelte-check, vitest unit tests, npm audit (last, see web-audit). The race-detector +
|
|
# Postgres jobs only run on push to main (per .github/workflows/ci.yml) and
|
|
# are not included here; run `make test-pg` separately if you want them
|
|
# locally.
|
|
#
|
|
# `make install` stays lightweight (build + restart only) so the inner
|
|
# dev loop is fast; opt into `check` when you're ready to push.
|
|
check: lint
|
|
go test -timeout=45m ./...
|
|
$(MAKE) vuln
|
|
$(MAKE) web-check
|
|
$(MAKE) web-test
|
|
$(MAKE) web-audit
|