Commit Graph

15 Commits

Author SHA1 Message Date
xarmian f9195c5b09 ci: make the go test timeout explicit everywhere (TASK-2545) (#1089)
* ci: make the go test timeout explicit everywhere (TASK-2545)

The v0.13.0 release pre-flight died on `panic: test timed out after
10m0s` in internal/store, on a commit whose Go tree was identical to a
green run an hour earlier. Nothing hung — the package's runtime simply
crossed a budget nobody had chosen.

`go test` without -timeout uses a 10m per-test-binary default. This repo
raised the two RACE steps to 45m twice as the suite grew (BUG-1371 30m,
BUG-1913 30m→45m), each time with a careful comment — and each time left
their non-race siblings on the silent default. Three steps were still
running on it, including the release gate:

  ci.yml       "Run tests"                    (SQLite)
  ci.yml       "Run tests against PostgreSQL" (the one that panicked)
  release.yml  "Run tests"                    (the release gate itself)

All three now carry -timeout=45m, matching the race legs so the file has
one number, with comments saying it is a hang-catcher rather than a
performance budget and that job wall-clock is the signal for "the suite
got slow".

Measured at 212d59e7 on a dev box, both drivers, before and after:

  PostgreSQL  whole suite 4m43s wall; internal/store 280s; server 103s
  SQLite      whole suite 1m52s wall; internal/server 107s; store 64s

CI runners are roughly 2x slower, which is what put store's PG binary
over 10m. 45m is ~4.5x current CI headroom.

This raises the ceiling; it does not change the slope. internal/store on
PG costs ~0.43s per test in database setup alone (CREATE DATABASE plus a
full migration replay, where the SQLite harness copies a pre-migrated
template — IDEA-1914), so every test added costs PG CI ~0.43s forever
and that package is 99% of the job's critical path. Measured and filed
as IDEA-2550 rather than fixed here: it changes shared test
infrastructure that gates every merge and deserves its own review.

Verified by running the exact post-change commands on both drivers: PG
green in 4m42s, SQLite green in 1m52s, 25 packages each.

Claude-Session: https://claude.ai/code/session_01QGbUKZBAZoWdEgiTNWsXag

* ci: time the Makefile's go test targets too (TASK-2545)

The previous commit said the timeout was explicit "everywhere" and it
wasn't — `make test`, `make test-pg`, and `make check` were all still on
the 10m default. That matters twice over: it's the same trap the commit
is about, and `make test-pg` is the local mirror of the CI leg that
actually panicked, so a developer reproducing the failure would have hit
a different budget than the one they were debugging.

Found by sweeping every `go test` in the repo rather than only the
workflows — which is what the commit message's own claim required and I
hadn't done when I wrote it.

Verified: `make test` green, 25 packages.

Claude-Session: https://claude.ai/code/session_01QGbUKZBAZoWdEgiTNWsXag

* ci: time the nix checkPhase, cap the Go jobs, correct two claims (TASK-2545)

Codex review. No P1s; the two P2s were both right and one of them
catches me stating an explanation I had not checked.

COVERAGE. `nix/package.nix`'s checkPhase runs `go test ./...` on the
default too, and .github/workflows/nix.yml exercises it — a fourth site
after the three workflow steps and the three Makefile targets. Now
timed. Every `go test` invocation in the repo carries an explicit
-timeout; the sweep is `grep -rn "go test"` over workflows, Makefile and
nix, not just the workflows I happened to be looking at.

JOB CAPS. Codex objected that 45m lets a hung binary burn a
release-gating job. Fair, and the real hole was worse: `go` and
`go-postgres` had NO `timeout-minutes`, so they inherit GitHub's 6-HOUR
default. Both now capped at 100m — deliberately above the two 45m test
steps so the per-binary timeout always fires first, because that is the
one that prints the goroutine dump naming the hung test. The cap only
catches a runaway that isn't a single test (wedged service container,
stuck download).

CORRECTIONS to 496f521f's message:

- It said the race steps were raised "twice (BUG-1371 30m, BUG-1913
  30m→45m)". BUG-1371 kept 30m and fixed the bcrypt cost that had blown
  past it; BUG-1913 made the only 30m→45m change. One raise, not two.
- It said CI runners are "roughly 2x slower, which is what put store's
  PG binary over 10m". That does not survive its own arithmetic: 280s
  local x 2 is 9m20s, under the budget. What is actually known is that
  the CI binary exceeded 10m and the local one takes 280s, so CI is
  >2.14x slower on that binary — a lower bound derived from the failure,
  not an explanation of it. I have not measured CI's runtime and should
  not have written a factor as if I had.
- "Nothing hung" and "cost the cut ~40 minutes" are TASK-2545's findings
  from the goroutine dump and the release timeline, not mine. Attributed
  rather than restated as my own observation.

The 0.43s per-test setup figure and both driver runtimes are mine, taken
on this box at 212d59e7 and reproducible with the commands in IDEA-2550.

Claude-Session: https://claude.ai/code/session_01QGbUKZBAZoWdEgiTNWsXag

* ci: put the corrections in the file, not only in a commit message (TASK-2545)

Codex's re-review came back with no P1s or P2s and four nits, all the
same shape: the claims I retracted in 4623cae9's COMMIT MESSAGE were
still sitting in the workflow comments. That's the half that matters —
nobody reads a commit message while editing a CI file, and a correction
that lives only in git log is a correction almost nobody receives.

Fixed in place:

- The raise history: BUG-1913 raised 30m→45m once. BUG-1371 kept 30m and
  dropped the test-only bcrypt cost that had blown past it. My comment
  said "raised twice (BUG-1371, BUG-1913)".
- The pre-existing race-step comment claiming BUG-1371 kept the step
  "well under the 30m budget" — contradicted by BUG-1913 having to raise
  it later. Reworded to say what each change actually did. Not my text,
  but it is wrong in the file I am editing and the next reader inherits
  it either way.
- The "~2x slower, which put store over 10m" line, which its own
  arithmetic refutes (280s x 2 = 9m20s). Now states the lower bound the
  failure actually supports — CI's store binary exceeded 10m, so >2.14x
  this box — and names the retracted claim so a reader who saw the old
  version knows it was withdrawn rather than lost.
- "so it never fires before they do" on the job caps, which a job-level
  timeout cannot promise: it covers setup and every step, not just the
  two 45m ones. Now says "in practice", not a guarantee.

Attribution of TASK-2545's own findings (the ~40 minutes, the goroutine
dump showing nothing hung) moved into the comment too.

Claude-Session: https://claude.ai/code/session_01QGbUKZBAZoWdEgiTNWsXag
2026-08-13 17:15:30 -04:00
xarmian d6c6dfe682 build(deps): govulncheck binary mode + Go 1.26.5 / x-crypto / gRPC security bumps (#896)
BUG-2084. Two parts.

RAM fix: `make vuln` and CI's Go job now run govulncheck in BINARY mode
(`-mode binary` against a freshly-built pad binary) instead of source mode
(`govulncheck ./...`). Source mode builds an SSA call-graph over the whole
dependency tree (BigQuery/OTel/gRPC/Cloud) and balloons to multiple GB of
RAM, which was locking up a memory-constrained host. Binary mode reads the
binary's symbol table — ~99 MB peak here — while staying call-graph-precise
and still detecting stdlib vulns from the Go version stamped in the binary.
The scan binary is written to the repo root (real disk, gitignored), never
/tmp, since some hosts mount /tmp as a small RAM-backed tmpfs where a large
embedded binary can hit "no space left" and consume the RAM we're sparing.

Vuln fix (govulncheck binary mode: 0 vulnerabilities after):
- go 1.26.4 -> 1.26.5: clears the only CALLED vuln GO-2026-5856 (crypto/tls)
  plus not-called os GO-2026-4970.
- golang.org/x/crypto v0.51.0 -> v0.52.0: clears 13 not-called advisories.
- google.golang.org/grpc v1.59.0 -> v1.79.3: clears GO-2026-4762 (gRPC
  authorization bypass). pad runs no gRPC server, but grpc.Server.Serve ships
  transitively (OTel/ory/grpc-gateway) so binary mode flags the symbol.
  Contained 12-line go.mod bump (genproto/protobuf/oauth2 family), no cascade.

Remaining not-called advisories deferred to a follow-up dependency sweep:
GO-2026-4985 (otel otlptracehttp) and GO-2026-5932 (x/crypto, Fixed in: N/A).
2026-07-10 12:02:20 -04:00
xarmian 88c5771aed ci(web): run vitest unit tests in CI + Makefile web-test target (#835)
The 128-test vitest suite (7 files, incl. the WebMCP dispatch/descriptor
tests backing PLAN-1888) ran nowhere in CI. Add a "Run web unit tests"
step to the Web job after the build/check steps, a `web-test` Makefile
target wired into the `check` chain, and a CLAUDE.md Testing note.

Fixes TASK-1999.

Claude-Session: https://claude.ai/code/session_01BoPkYhKqMiWPYmxQigeWsA
2026-07-07 15:12:19 -04:00
xarmian bfce069f28 fix(web,build): search palette hang on numeric query + graceful SSE shutdown (BUG-1531) (#588)
* fix(web,build): search palette hang on numeric query + graceful SSE shutdown (BUG-1531)

CommandPalette's reactive `$effect` subscribed to every workspace's
`localSearch.epoch` + `localIndex.bootstrapStateFor`. Bare-digit queries
short-circuit to `exactItemNumberLookup` (synchronous, very fast) and
stacked re-fires of the effect inside one microtask tick whenever an SSE
delta arrived — Svelte tripped `effect_update_depth_exceeded` and the
palette froze. Treat bare-digit queries the same as `body:` queries
(skip the subscription reads) and wrap `doSearch()` in `untrack()` so
its internal reactive reads can't smuggle hidden dependencies into the
effect.

The SSE churn that fanned the loop was rooted in `make install` using
`killall -9` — SIGKILL drops every open SSE stream mid-chunk so every
browser tab logs `ERR_INCOMPLETE_CHUNKED_ENCODING` and reconnects.
Switch to SIGTERM + 5s wait + SIGKILL fallback so the server's existing
graceful-shutdown path (cmd/pad/main.go:811-857) actually runs and the
http.Server writes a final 0-chunk on each open stream.

Follow-up tidy-ups (unchecked write errors in writeSSEEvent, link the
30s keepalive to the 120s IdleTimeout in code) tracked in BUG-1532.

* fix(web): track workspace slug in palette $effect per Codex review (round 1)

After wrapping doSearch() in untrack(), the workspaceStore.current?.slug
read that doSearch performs at line 209 no longer registered as a
tracked dep of the search effect. The non-body / non-bare-digit branch
still reads the slug via localIndex.bootstrapStateFor(...), so workspace
switches re-fire the effect for that branch — but body: and bare-digit
queries skip that block entirely. Without an explicit slug subscription
they wouldn't re-dispatch on workspace switch; an in-flight server
response would land stale, get discarded by isSameDispatch(), and
loading could stick true.

Hoist `void workspaceStore.current?.slug` into the unconditional void
block so all four query shapes re-fire on workspace switch.

Refs BUG-1531.

* chore: gofmt handlers_claim_code_test.go

Drive-by formatting fix to unblock CI on this PR. The file landed
slightly unaligned in #586 (TASK-1525) — gofmt straightens the struct
tag column on claimCodeResponse.
2026-05-18 15:25:27 -04:00
xarmian 21e8ca6a20 chore(make): add make check mirroring CI lint + test + web build (IDEA-921) (#322)
* chore(make): add `make check` mirroring CI lint + test + web build (IDEA-921)

Closes the local-vs-CI gap that let PR #321 ship a trivial gofmt
violation past every step of CONVE-190's pre-flight (`go build &&
go test && cd web && npm run build`).

- `make lint` now runs the same golangci-lint v2.11.4 suite CI runs
  (govet, ineffassign, staticcheck SA*, unused, plus the gofmt
  formatter with simplify: true). The bootstrap rule auto-installs
  the pinned binary into $(go env GOPATH)/bin on first run, so
  contributors don't need a separate setup step.

- `make check` is a new umbrella target that runs lint, the Go test
  suite, and the web build — the exact set of jobs CI's "Go" and
  "Web" jobs run. Run it before pushing.

- `make install` is unchanged (build + restart) so the inner dev loop
  stays fast. `check` is the opt-in pre-push gate.

CONVE-190 updated separately via the Pad CLI to point contributors at
`make check` instead of the old three-command list.

Verified locally: `make check` passes on a clean tree (after a one-
time `golangci-lint cache clean` to clear stale entries from a prior
run — that's a known golangci-lint quirk, not a workflow bug).

* fix(make): enforce lint version pin + cover full CI surface (round 1)

Codex review on PR #322 round 1 surfaced two real gaps in the
make check / make lint plumbing.

1. Makefile:83 — `lint` did not actually enforce GOLANGCI_LINT_VERSION.
   The previous file-target dependency only fired the install rule
   when the binary was missing, so an older or newer locally-installed
   golangci-lint was silently reused, defeating the pin. The recipe
   now compares the installed version against the pin and reinstalls
   on mismatch.

2. Makefile:97 — `check` claimed to mirror CI's Go and Web jobs but
   omitted Web's `npm run check` (svelte-check type checking) and the
   Go job's `govulncheck` step. CI could fail on either while local
   `make check` passed. Added new `vuln` (pinned to GOVULNCHECK_VERSION
   = v1.2.0, matches CI) and `web-check` targets, both wired into
   `make check`.

`make check` now runs: lint + go test + govulncheck + npm ci + npm
audit + npm run build + svelte-check — exactly mirroring the gates
the CI Go and Web jobs use to fail a PR. The race-detector and
PostgreSQL jobs only run on push to main and are intentionally not
part of `make check` (run `make test-pg` separately if needed).

Verified locally: `make check` exits 0; `make lint` correctly no-ops
when the pinned version is already installed.
2026-04-30 16:32:40 -04:00
xarmian 062eef41b2 docs: architecture guide + full .env.example + gitattributes + Makefile note (TASK-687) (#222)
Grouped nice-to-haves called out in the pre-launch audit.

1. docs/architecture.md — new contributor-focused architecture doc.
   CLAUDE.md covers the same ground but is agent-oriented; this is the
   human companion. Covers backend layout, request flow, frontend /
   data model / CLI↔daemon model / agent integration / testing.

2. .env.example — extended to document every PAD_* variable in
   docs/deployment.md (core, database, real-time events, security,
   email). Existing Postgres/Redis + encryption secrets kept at the
   top; new variables grouped by concern with inline comments and
   safe defaults commented out.

3. .gitattributes — normalize LF line endings repo-wide, mark binary
   assets, and flag web/build + web/.svelte-kit as generated so they
   don't pollute GitHub linguist stats or PR diffs.

4. Makefile — CAUTION comment on `make install` noting that the
   `killall -9 pad` step is system-wide; anyone else's pad daemon on
   the same machine gets killed too. Designed for single-developer
   local setups; not for shared hosts.

Parent: PLAN-644.
2026-04-22 20:59:15 -04:00
xarmian b027046605 fix: address review findings for PR #91 (iteration 1)
Ensure make test-pg cleans up Docker containers even when tests fail
by capturing the exit code and running cleanup unconditionally. Remove
dead CSS rules from root page after welcome template simplification.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-04-13 01:32:12 +00:00
xarmian b2b4feecb9 feat: console navigation, PostgreSQL CI, and operational improvements
- Route root (/) to /console for centralized workspace management
- Update TopBar user dropdown with console nav links (workspaces, settings, billing, admin)
- Move account settings (profile, password, tokens) from workspace settings to /console/settings
- Enhance admin page with email configuration UI and CSRF-protected writes
- Add PostgreSQL CI job to GitHub Actions with race detector on main
- Add `make test-pg` for local PostgreSQL testing via docker-compose
- Expand health/ready endpoint with DB connection pool stats
- Increase item number retry limit for high-concurrency environments
- Add concurrent store benchmarks and FTS search quality tests
- Add AGENTS.md for multi-agent development guidance
2026-04-13 01:29:15 +00:00
xarmian f5649b912e refactor(cli): group first-release commands for TASK-127 (#45) 2026-04-02 15:28:16 -04:00
xarmian eb4e41ad0b build(embed): stop rewriting embed.go for TASK-112 (#32) 2026-04-01 15:00:18 -04:00
xarmian 056beba12b fix: run npm ci before web build to ensure dependencies are installed
Prevents "vite: not found" errors on fresh clones or after node_modules
is removed, by installing dependencies before building.
2026-03-31 16:49:34 -04:00
xarmian 0972347cf0 Fix svelte warnings and add build version info (#26)
* Fix all 17 svelte-check warnings across 7 components

- Add tabindex to toolbar role elements (BoardView, Editor)
- Replace nested buttons with div[role=button] in ListView group headers
- Add role="none" to click-to-close backdrop overlays (Editor, slug page)
- Fix label→span for non-input field labels (CreateWorkspaceModal)
- Add keyboard handlers to interactive divs (CreateWorkspaceModal drop zone, slug page modal)
- Remove unused .slash-backdrop CSS (Editor) and input[type=text] selector (CreateWorkspaceModal)
- Fix state_referenced_locally in RawMarkdownEditor ($state init)
- Add svelte-ignore for conditional tabindex false positive (ToastContainer)

* feat: add build version, commit hash, and timestamp to CLI and web UI

Inject version info via ldflags during build (Makefile, GoReleaser,
Dockerfile). Expose version/commit/build_time in the health API
endpoint and display it in the sidebar footer. Dev builds show
"dev (abc1234 ...)", releases show "v1.2.3 (abc1234 ...)".
2026-03-30 11:54:45 -04:00
xarmian a6befde8bf feat: email-based password reset flow (#24)
* feat: email-based password reset flow (IDEA-81)

Add full password reset flow: forgot password request, time-limited
reset tokens (1hr, single-use, SHA-256 hashed), new password form,
and automatic session creation after reset.

* fix: use all: prefix in go:embed to include _-prefixed files

Go's embed package excludes files starting with _ or . when recursing
directories. SvelteKit/Vite occasionally generates chunk filenames with
_ prefixes (e.g. _VLZtjCJ.js), causing them to be silently dropped
from the embedded filesystem and served as HTML by the SPA fallback.

The all: prefix includes everything regardless of filename prefix.
Fixed in both embed.go and the Makefile which regenerates it.

* fix: address PR review — atomic token consumption, error handling, log reset URL

- Replace ValidatePasswordReset + MarkPasswordResetUsed with atomic
  ConsumePasswordReset using UPDATE ... WHERE ... RETURNING to prevent
  race conditions where two concurrent requests consume the same token
- Handle DeleteUserSessions errors (log instead of silently ignoring)
- Log the full reset URL when email is not configured so the admin
  CLI fallback is actually usable
2026-03-29 19:31:16 -04:00
xarmian f7ac533ec1 Fix critical security issues found in pre-public audit
- Fix SQL injection in sort parameter: validate field names against
  alphanumeric regex before interpolating into json_extract queries
- Restrict CORS to localhost origins only (remove http://* wildcard)
- Change Makefile HOST default from 0.0.0.0 to 127.0.0.1
- Add Apache-2.0 license field to web/package.json
2026-03-28 04:20:59 +00:00
xarmian 81579847c6 Initial release
Pad — project management for developers and AI agents.
Single Go binary with embedded SvelteKit web UI, SQLite storage,
CLI, and Claude Code /pad skill integration.

https://getpad.dev
2026-03-26 01:52:36 +00:00