Commit Graph

16 Commits

Author SHA1 Message Date
xarmian a1716d8170 ci(web): decide the npm audit gate from the report, not the exit code, and run it last (BUG-2881) (#1247)
* ci(web): decide the npm audit gate from the report, not the exit code, and run it last (BUG-2881)

`npm audit` exits non-zero identically for "a HIGH/CRITICAL advisory
exists" and "the advisory service was unreachable". The Web job ran it
before Build / Type check / vitest under `bash -e`, so a registry
timeout (main, 03:50Z) and a 503 (#1246, 04:33Z) on 2026-09-04 each
produced a red row with every frontend verification step SKIPPED — a
lane that read like a failure and had asked nothing.

scripts/ci-audit.mjs runs the audit in --json mode and decides from the
report: metadata.vulnerabilities present → fail iff high+critical > 0,
naming the advisories; an error envelope or unparseable output → a
GitHub warning annotation saying the gate did not run, exit 0. The step
moves to the end of the job so the frontend's own verdict always exists
whatever the audit does.

Verified locally against five report shapes (transport timeout envelope,
E503 envelope, one high advisory, clean, garbage) and two live runs (the
real registry: clean; a dead registry: warning, exit 0). `--input <file>`
is the seam those checks use.

Fixes BUG-2881

* ci(web): the audit gate fails closed — retry an unreachable advisory service, then fail under its own title

Codex round 1 on #1247: the first draft warned and exited 0 when the
advisory service could not be asked, which made the only supply-chain
gate pass exactly when it had not run. A gate that passes when it cannot
run is not a gate.

Now: up to three attempts with backoff (registry blips are usually
seconds long), then `::error title=npm audit did not run` and exit 1.
The title is distinct from `::error title=npm audit` (a real advisory)
so the checks tab tells the two apart without opening the log; re-running
is the remedy for the first and never for the second. Because the step
runs last, Build / Type check / vitest have already produced their result
either way — the original blindness is gone regardless of which way this
step fails.

Verified against the same five saved shapes (transport and E503 envelopes
and garbage now exit 1 under the did-not-run title; a high advisory exits
1 under the advisory title; clean exits 0) and two live runs (real
registry: clean; dead registry: three attempts logged, exit 1).

Refs BUG-2881

* ci(web): the audit gate refuses counts it cannot read, and refuses bad tuning without crashing

Codex round 2 on #1247. (1) metadata.vulnerabilities was checked for
presence, not for shape: Number("x") + Number(null) > 0 is false, so a
malformed count read as a clean audit — a second fail-open, one layer
deeper than round 1's. high/critical must now be non-negative integers
or the report is unreadable, which is the fail-closed path. (2) The two
env knobs are operator-set, but CI_AUDIT_ATTEMPTS=NaN left the retry loop
unexecuted and threw a TypeError, and CI_AUDIT_BACKOFF_MS=Infinity parked
Atomics.wait forever; both now fall back to the default with a line
saying so.

Refs BUG-2881

* build: the local preflight runs the same audit gate CI does, and runs it last

Codex round 3 on #1247 (blast radius): `make web-check` still chained
bare `npm audit && npm run check`, so a registry blip stopped svelte-check
locally exactly as it had in CI, and CONTRIBUTING documented the bare
command as the way to reproduce the gate. New `web-audit` target runs
`npm run audit:ci`; `check` runs it after web-check and web-test, mirroring
the Web job's order. CONTRIBUTING and docs/architecture.md say so.

Refs BUG-2881

* build: web-audit stands alone — no `web` prerequisite, so `check` runs npm ci once and no new target reaches it

Codex round 4 on #1247: `web-audit: web` made `check` run `npm ci` twice
(`web` is .PHONY) and added a target CLAUDE.md's worktree rule did not
list as reaching `npm ci`. `npm audit` reads the lockfile and needs
neither node_modules nor a build — verified by running it with
node_modules removed — so the prerequisite goes; CLAUDE.md's safe list
gains `web-audit`.

Refs BUG-2881
2026-09-04 10:45:21 -04:00
xarmian f9195c5b09 ci: make the go test timeout explicit everywhere (TASK-2545) (#1089)
* ci: make the go test timeout explicit everywhere (TASK-2545)

The v0.13.0 release pre-flight died on `panic: test timed out after
10m0s` in internal/store, on a commit whose Go tree was identical to a
green run an hour earlier. Nothing hung — the package's runtime simply
crossed a budget nobody had chosen.

`go test` without -timeout uses a 10m per-test-binary default. This repo
raised the two RACE steps to 45m twice as the suite grew (BUG-1371 30m,
BUG-1913 30m→45m), each time with a careful comment — and each time left
their non-race siblings on the silent default. Three steps were still
running on it, including the release gate:

  ci.yml       "Run tests"                    (SQLite)
  ci.yml       "Run tests against PostgreSQL" (the one that panicked)
  release.yml  "Run tests"                    (the release gate itself)

All three now carry -timeout=45m, matching the race legs so the file has
one number, with comments saying it is a hang-catcher rather than a
performance budget and that job wall-clock is the signal for "the suite
got slow".

Measured at 212d59e7 on a dev box, both drivers, before and after:

  PostgreSQL  whole suite 4m43s wall; internal/store 280s; server 103s
  SQLite      whole suite 1m52s wall; internal/server 107s; store 64s

CI runners are roughly 2x slower, which is what put store's PG binary
over 10m. 45m is ~4.5x current CI headroom.

This raises the ceiling; it does not change the slope. internal/store on
PG costs ~0.43s per test in database setup alone (CREATE DATABASE plus a
full migration replay, where the SQLite harness copies a pre-migrated
template — IDEA-1914), so every test added costs PG CI ~0.43s forever
and that package is 99% of the job's critical path. Measured and filed
as IDEA-2550 rather than fixed here: it changes shared test
infrastructure that gates every merge and deserves its own review.

Verified by running the exact post-change commands on both drivers: PG
green in 4m42s, SQLite green in 1m52s, 25 packages each.

Claude-Session: https://claude.ai/code/session_01QGbUKZBAZoWdEgiTNWsXag

* ci: time the Makefile's go test targets too (TASK-2545)

The previous commit said the timeout was explicit "everywhere" and it
wasn't — `make test`, `make test-pg`, and `make check` were all still on
the 10m default. That matters twice over: it's the same trap the commit
is about, and `make test-pg` is the local mirror of the CI leg that
actually panicked, so a developer reproducing the failure would have hit
a different budget than the one they were debugging.

Found by sweeping every `go test` in the repo rather than only the
workflows — which is what the commit message's own claim required and I
hadn't done when I wrote it.

Verified: `make test` green, 25 packages.

Claude-Session: https://claude.ai/code/session_01QGbUKZBAZoWdEgiTNWsXag

* ci: time the nix checkPhase, cap the Go jobs, correct two claims (TASK-2545)

Codex review. No P1s; the two P2s were both right and one of them
catches me stating an explanation I had not checked.

COVERAGE. `nix/package.nix`'s checkPhase runs `go test ./...` on the
default too, and .github/workflows/nix.yml exercises it — a fourth site
after the three workflow steps and the three Makefile targets. Now
timed. Every `go test` invocation in the repo carries an explicit
-timeout; the sweep is `grep -rn "go test"` over workflows, Makefile and
nix, not just the workflows I happened to be looking at.

JOB CAPS. Codex objected that 45m lets a hung binary burn a
release-gating job. Fair, and the real hole was worse: `go` and
`go-postgres` had NO `timeout-minutes`, so they inherit GitHub's 6-HOUR
default. Both now capped at 100m — deliberately above the two 45m test
steps so the per-binary timeout always fires first, because that is the
one that prints the goroutine dump naming the hung test. The cap only
catches a runaway that isn't a single test (wedged service container,
stuck download).

CORRECTIONS to 496f521f's message:

- It said the race steps were raised "twice (BUG-1371 30m, BUG-1913
  30m→45m)". BUG-1371 kept 30m and fixed the bcrypt cost that had blown
  past it; BUG-1913 made the only 30m→45m change. One raise, not two.
- It said CI runners are "roughly 2x slower, which is what put store's
  PG binary over 10m". That does not survive its own arithmetic: 280s
  local x 2 is 9m20s, under the budget. What is actually known is that
  the CI binary exceeded 10m and the local one takes 280s, so CI is
  >2.14x slower on that binary — a lower bound derived from the failure,
  not an explanation of it. I have not measured CI's runtime and should
  not have written a factor as if I had.
- "Nothing hung" and "cost the cut ~40 minutes" are TASK-2545's findings
  from the goroutine dump and the release timeline, not mine. Attributed
  rather than restated as my own observation.

The 0.43s per-test setup figure and both driver runtimes are mine, taken
on this box at 212d59e7 and reproducible with the commands in IDEA-2550.

Claude-Session: https://claude.ai/code/session_01QGbUKZBAZoWdEgiTNWsXag

* ci: put the corrections in the file, not only in a commit message (TASK-2545)

Codex's re-review came back with no P1s or P2s and four nits, all the
same shape: the claims I retracted in 4623cae9's COMMIT MESSAGE were
still sitting in the workflow comments. That's the half that matters —
nobody reads a commit message while editing a CI file, and a correction
that lives only in git log is a correction almost nobody receives.

Fixed in place:

- The raise history: BUG-1913 raised 30m→45m once. BUG-1371 kept 30m and
  dropped the test-only bcrypt cost that had blown past it. My comment
  said "raised twice (BUG-1371, BUG-1913)".
- The pre-existing race-step comment claiming BUG-1371 kept the step
  "well under the 30m budget" — contradicted by BUG-1913 having to raise
  it later. Reworded to say what each change actually did. Not my text,
  but it is wrong in the file I am editing and the next reader inherits
  it either way.
- The "~2x slower, which put store over 10m" line, which its own
  arithmetic refutes (280s x 2 = 9m20s). Now states the lower bound the
  failure actually supports — CI's store binary exceeded 10m, so >2.14x
  this box — and names the retracted claim so a reader who saw the old
  version knows it was withdrawn rather than lost.
- "so it never fires before they do" on the job caps, which a job-level
  timeout cannot promise: it covers setup and every step, not just the
  two 45m ones. Now says "in practice", not a guarantee.

Attribution of TASK-2545's own findings (the ~40 minutes, the goroutine
dump showing nothing hung) moved into the comment too.

Claude-Session: https://claude.ai/code/session_01QGbUKZBAZoWdEgiTNWsXag
2026-08-13 17:15:30 -04:00
xarmian d6c6dfe682 build(deps): govulncheck binary mode + Go 1.26.5 / x-crypto / gRPC security bumps (#896)
BUG-2084. Two parts.

RAM fix: `make vuln` and CI's Go job now run govulncheck in BINARY mode
(`-mode binary` against a freshly-built pad binary) instead of source mode
(`govulncheck ./...`). Source mode builds an SSA call-graph over the whole
dependency tree (BigQuery/OTel/gRPC/Cloud) and balloons to multiple GB of
RAM, which was locking up a memory-constrained host. Binary mode reads the
binary's symbol table — ~99 MB peak here — while staying call-graph-precise
and still detecting stdlib vulns from the Go version stamped in the binary.
The scan binary is written to the repo root (real disk, gitignored), never
/tmp, since some hosts mount /tmp as a small RAM-backed tmpfs where a large
embedded binary can hit "no space left" and consume the RAM we're sparing.

Vuln fix (govulncheck binary mode: 0 vulnerabilities after):
- go 1.26.4 -> 1.26.5: clears the only CALLED vuln GO-2026-5856 (crypto/tls)
  plus not-called os GO-2026-4970.
- golang.org/x/crypto v0.51.0 -> v0.52.0: clears 13 not-called advisories.
- google.golang.org/grpc v1.59.0 -> v1.79.3: clears GO-2026-4762 (gRPC
  authorization bypass). pad runs no gRPC server, but grpc.Server.Serve ships
  transitively (OTel/ory/grpc-gateway) so binary mode flags the symbol.
  Contained 12-line go.mod bump (genproto/protobuf/oauth2 family), no cascade.

Remaining not-called advisories deferred to a follow-up dependency sweep:
GO-2026-4985 (otel otlptracehttp) and GO-2026-5932 (x/crypto, Fixed in: N/A).
2026-07-10 12:02:20 -04:00
xarmian 88c5771aed ci(web): run vitest unit tests in CI + Makefile web-test target (#835)
The 128-test vitest suite (7 files, incl. the WebMCP dispatch/descriptor
tests backing PLAN-1888) ran nowhere in CI. Add a "Run web unit tests"
step to the Web job after the build/check steps, a `web-test` Makefile
target wired into the `check` chain, and a CLAUDE.md Testing note.

Fixes TASK-1999.

Claude-Session: https://claude.ai/code/session_01BoPkYhKqMiWPYmxQigeWsA
2026-07-07 15:12:19 -04:00
xarmian bfce069f28 fix(web,build): search palette hang on numeric query + graceful SSE shutdown (BUG-1531) (#588)
* fix(web,build): search palette hang on numeric query + graceful SSE shutdown (BUG-1531)

CommandPalette's reactive `$effect` subscribed to every workspace's
`localSearch.epoch` + `localIndex.bootstrapStateFor`. Bare-digit queries
short-circuit to `exactItemNumberLookup` (synchronous, very fast) and
stacked re-fires of the effect inside one microtask tick whenever an SSE
delta arrived — Svelte tripped `effect_update_depth_exceeded` and the
palette froze. Treat bare-digit queries the same as `body:` queries
(skip the subscription reads) and wrap `doSearch()` in `untrack()` so
its internal reactive reads can't smuggle hidden dependencies into the
effect.

The SSE churn that fanned the loop was rooted in `make install` using
`killall -9` — SIGKILL drops every open SSE stream mid-chunk so every
browser tab logs `ERR_INCOMPLETE_CHUNKED_ENCODING` and reconnects.
Switch to SIGTERM + 5s wait + SIGKILL fallback so the server's existing
graceful-shutdown path (cmd/pad/main.go:811-857) actually runs and the
http.Server writes a final 0-chunk on each open stream.

Follow-up tidy-ups (unchecked write errors in writeSSEEvent, link the
30s keepalive to the 120s IdleTimeout in code) tracked in BUG-1532.

* fix(web): track workspace slug in palette $effect per Codex review (round 1)

After wrapping doSearch() in untrack(), the workspaceStore.current?.slug
read that doSearch performs at line 209 no longer registered as a
tracked dep of the search effect. The non-body / non-bare-digit branch
still reads the slug via localIndex.bootstrapStateFor(...), so workspace
switches re-fire the effect for that branch — but body: and bare-digit
queries skip that block entirely. Without an explicit slug subscription
they wouldn't re-dispatch on workspace switch; an in-flight server
response would land stale, get discarded by isSameDispatch(), and
loading could stick true.

Hoist `void workspaceStore.current?.slug` into the unconditional void
block so all four query shapes re-fire on workspace switch.

Refs BUG-1531.

* chore: gofmt handlers_claim_code_test.go

Drive-by formatting fix to unblock CI on this PR. The file landed
slightly unaligned in #586 (TASK-1525) — gofmt straightens the struct
tag column on claimCodeResponse.
2026-05-18 15:25:27 -04:00
xarmian 21e8ca6a20 chore(make): add make check mirroring CI lint + test + web build (IDEA-921) (#322)
* chore(make): add `make check` mirroring CI lint + test + web build (IDEA-921)

Closes the local-vs-CI gap that let PR #321 ship a trivial gofmt
violation past every step of CONVE-190's pre-flight (`go build &&
go test && cd web && npm run build`).

- `make lint` now runs the same golangci-lint v2.11.4 suite CI runs
  (govet, ineffassign, staticcheck SA*, unused, plus the gofmt
  formatter with simplify: true). The bootstrap rule auto-installs
  the pinned binary into $(go env GOPATH)/bin on first run, so
  contributors don't need a separate setup step.

- `make check` is a new umbrella target that runs lint, the Go test
  suite, and the web build — the exact set of jobs CI's "Go" and
  "Web" jobs run. Run it before pushing.

- `make install` is unchanged (build + restart) so the inner dev loop
  stays fast. `check` is the opt-in pre-push gate.

CONVE-190 updated separately via the Pad CLI to point contributors at
`make check` instead of the old three-command list.

Verified locally: `make check` passes on a clean tree (after a one-
time `golangci-lint cache clean` to clear stale entries from a prior
run — that's a known golangci-lint quirk, not a workflow bug).

* fix(make): enforce lint version pin + cover full CI surface (round 1)

Codex review on PR #322 round 1 surfaced two real gaps in the
make check / make lint plumbing.

1. Makefile:83 — `lint` did not actually enforce GOLANGCI_LINT_VERSION.
   The previous file-target dependency only fired the install rule
   when the binary was missing, so an older or newer locally-installed
   golangci-lint was silently reused, defeating the pin. The recipe
   now compares the installed version against the pin and reinstalls
   on mismatch.

2. Makefile:97 — `check` claimed to mirror CI's Go and Web jobs but
   omitted Web's `npm run check` (svelte-check type checking) and the
   Go job's `govulncheck` step. CI could fail on either while local
   `make check` passed. Added new `vuln` (pinned to GOVULNCHECK_VERSION
   = v1.2.0, matches CI) and `web-check` targets, both wired into
   `make check`.

`make check` now runs: lint + go test + govulncheck + npm ci + npm
audit + npm run build + svelte-check — exactly mirroring the gates
the CI Go and Web jobs use to fail a PR. The race-detector and
PostgreSQL jobs only run on push to main and are intentionally not
part of `make check` (run `make test-pg` separately if needed).

Verified locally: `make check` exits 0; `make lint` correctly no-ops
when the pinned version is already installed.
2026-04-30 16:32:40 -04:00
xarmian 062eef41b2 docs: architecture guide + full .env.example + gitattributes + Makefile note (TASK-687) (#222)
Grouped nice-to-haves called out in the pre-launch audit.

1. docs/architecture.md — new contributor-focused architecture doc.
   CLAUDE.md covers the same ground but is agent-oriented; this is the
   human companion. Covers backend layout, request flow, frontend /
   data model / CLI↔daemon model / agent integration / testing.

2. .env.example — extended to document every PAD_* variable in
   docs/deployment.md (core, database, real-time events, security,
   email). Existing Postgres/Redis + encryption secrets kept at the
   top; new variables grouped by concern with inline comments and
   safe defaults commented out.

3. .gitattributes — normalize LF line endings repo-wide, mark binary
   assets, and flag web/build + web/.svelte-kit as generated so they
   don't pollute GitHub linguist stats or PR diffs.

4. Makefile — CAUTION comment on `make install` noting that the
   `killall -9 pad` step is system-wide; anyone else's pad daemon on
   the same machine gets killed too. Designed for single-developer
   local setups; not for shared hosts.

Parent: PLAN-644.
2026-04-22 20:59:15 -04:00
xarmian b027046605 fix: address review findings for PR #91 (iteration 1)
Ensure make test-pg cleans up Docker containers even when tests fail
by capturing the exit code and running cleanup unconditionally. Remove
dead CSS rules from root page after welcome template simplification.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-04-13 01:32:12 +00:00
xarmian b2b4feecb9 feat: console navigation, PostgreSQL CI, and operational improvements
- Route root (/) to /console for centralized workspace management
- Update TopBar user dropdown with console nav links (workspaces, settings, billing, admin)
- Move account settings (profile, password, tokens) from workspace settings to /console/settings
- Enhance admin page with email configuration UI and CSRF-protected writes
- Add PostgreSQL CI job to GitHub Actions with race detector on main
- Add `make test-pg` for local PostgreSQL testing via docker-compose
- Expand health/ready endpoint with DB connection pool stats
- Increase item number retry limit for high-concurrency environments
- Add concurrent store benchmarks and FTS search quality tests
- Add AGENTS.md for multi-agent development guidance
2026-04-13 01:29:15 +00:00
xarmian f5649b912e refactor(cli): group first-release commands for TASK-127 (#45) 2026-04-02 15:28:16 -04:00
xarmian eb4e41ad0b build(embed): stop rewriting embed.go for TASK-112 (#32) 2026-04-01 15:00:18 -04:00
xarmian 056beba12b fix: run npm ci before web build to ensure dependencies are installed
Prevents "vite: not found" errors on fresh clones or after node_modules
is removed, by installing dependencies before building.
2026-03-31 16:49:34 -04:00
xarmian 0972347cf0 Fix svelte warnings and add build version info (#26)
* Fix all 17 svelte-check warnings across 7 components

- Add tabindex to toolbar role elements (BoardView, Editor)
- Replace nested buttons with div[role=button] in ListView group headers
- Add role="none" to click-to-close backdrop overlays (Editor, slug page)
- Fix label→span for non-input field labels (CreateWorkspaceModal)
- Add keyboard handlers to interactive divs (CreateWorkspaceModal drop zone, slug page modal)
- Remove unused .slash-backdrop CSS (Editor) and input[type=text] selector (CreateWorkspaceModal)
- Fix state_referenced_locally in RawMarkdownEditor ($state init)
- Add svelte-ignore for conditional tabindex false positive (ToastContainer)

* feat: add build version, commit hash, and timestamp to CLI and web UI

Inject version info via ldflags during build (Makefile, GoReleaser,
Dockerfile). Expose version/commit/build_time in the health API
endpoint and display it in the sidebar footer. Dev builds show
"dev (abc1234 ...)", releases show "v1.2.3 (abc1234 ...)".
2026-03-30 11:54:45 -04:00
xarmian a6befde8bf feat: email-based password reset flow (#24)
* feat: email-based password reset flow (IDEA-81)

Add full password reset flow: forgot password request, time-limited
reset tokens (1hr, single-use, SHA-256 hashed), new password form,
and automatic session creation after reset.

* fix: use all: prefix in go:embed to include _-prefixed files

Go's embed package excludes files starting with _ or . when recursing
directories. SvelteKit/Vite occasionally generates chunk filenames with
_ prefixes (e.g. _VLZtjCJ.js), causing them to be silently dropped
from the embedded filesystem and served as HTML by the SPA fallback.

The all: prefix includes everything regardless of filename prefix.
Fixed in both embed.go and the Makefile which regenerates it.

* fix: address PR review — atomic token consumption, error handling, log reset URL

- Replace ValidatePasswordReset + MarkPasswordResetUsed with atomic
  ConsumePasswordReset using UPDATE ... WHERE ... RETURNING to prevent
  race conditions where two concurrent requests consume the same token
- Handle DeleteUserSessions errors (log instead of silently ignoring)
- Log the full reset URL when email is not configured so the admin
  CLI fallback is actually usable
2026-03-29 19:31:16 -04:00
xarmian f7ac533ec1 Fix critical security issues found in pre-public audit
- Fix SQL injection in sort parameter: validate field names against
  alphanumeric regex before interpolating into json_extract queries
- Restrict CORS to localhost origins only (remove http://* wildcard)
- Change Makefile HOST default from 0.0.0.0 to 127.0.0.1
- Add Apache-2.0 license field to web/package.json
2026-03-28 04:20:59 +00:00
xarmian 81579847c6 Initial release
Pad — project management for developers and AI agents.
Single Go binary with embedded SvelteKit web UI, SQLite storage,
CLI, and Claude Code /pad skill integration.

https://getpad.dev
2026-03-26 01:52:36 +00:00