365 Commits

Author	SHA1	Message	Date
xarmian	cb99879d53	feat(materialize): recover unsaved editor sessions into items.content (TASK-2198 U4) (#1683) ... * feat(materialize): recover unsaved editor sessions into items.content in the background (TASK-2198 U4) Wires the U3 supervisor in. A collab room reclaimed at the end of its grace TTL with content-bearing rows above the flush watermark (T3), or a dormant pending item found by a one-minute sweep (T1), is queued; one worker goroutine materializes the item's whole content-bearing op-log with the whole-workspace link index and lands it through store.MaterializeFlush, under the collab item lock with no room open. The write is a flush: cursor == MAX(op-log id), rows pending, no set-aside rows, checked in the write's own transaction; watermark to the cursor only; seq bumped; one version row attributed to system/recovery; the op-log is never pruned. PAD_MATERIALIZE=off disables it entirely. A per-item in-memory failure budget (3 failures, doubling backoff, reset by a new op-log row) keeps untrusted input from being retried in a loop. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * test(store): recovery watermark is the cursor, not MAX at UPDATE time (TASK-2198 U4) Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * test(store): a second recovery still writes its own version row (TASK-2198 U4) Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * docs(mcp): op-log recovery bounds the stale-body window; tool surface v0.60 (TASK-2198 U4) content_state applied_pending_flush now clears on its own after the item's room closes, unless recovery fails or PAD_MATERIALIZE=off. Every sentence that called the window unbounded, or said nothing server-side moves op-log content into items.content, is corrected (CLAUDE.md, instructions.md, README, docs/deployment.md, docs/backup.md, code comments). BEHAVIOR bump to 0.60; goldens regenerated. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * fix(materialize): count ErrNoMemoryCap silently against the per-item budget (TASK-2198 U4) U3 now refuses every job with ErrNoMemoryCap when no memory cap can be established, and warns once itself. The recovery worker counts it like any failure but logs it at debug, not per item. Its log lines go through a logger seam so the test can tell. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * style: gofmt (TASK-2198 U4) Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * feat(materialize): one log line per budget exhaustion; count set-aside skips on the sweep line (TASK-2198 U4) An item that uses up its failure budget logs exactly one WARN naming the item, the failure count and the last error kind (DEBUG when the kind is no_memory_cap, which the supervisor already warned about once), and logs again only after op-log growth resets the budget and it is exhausted anew. The schema-version skip goes through the same line. Items skipped for set-aside rows are counted and reported on a new single sweep summary line, written only when it has news; never per item. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * fix(materialize): keyset-page the recovery sweep so no candidate starves (TASK-2198 U4, codex r1) The sweep read the oldest 200 candidates from the start every tick, and the room check and the failure budget filtered them afterwards. Once 200 sat in backoff or exhausted, every sweep returned the same 200 and the 201st was never queued: filtering above a bound is starvation. Each tick now reads one page strictly after an in-memory keyset cursor (newest op-log created_at, item id) and wraps to the start after a short page, so every candidate is offered within ceil(N/limit)+1 ticks, whatever sorts ahead of it. Keyset over NOT IN: no parameter-count limit, and it also covers the room check, which an exclusion list would not. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * test(materialize): the sweep cursor wraps; docs for the exhaustion line, the set-aside count and paging (TASK-2198 U4) Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * docs(deployment): reflow the recovery budget paragraph (TASK-2198 U4) Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * test(materialize): recovery strips the follows-title marker as a tab's save does (TASK-2198 U4, BUG-3315) Since #1688 a tab loads a link that follows its target's title with the follows-title marker in the link's title attribute, so the marker lives in Y.Docs that recovery replays. Three fixture cases (a follows-title link; the same link after a rename; a URL edited to an external address) are generated from documents that really hold the marker, and the existing 8 are unchanged. A new leg recovers each through the production path (whole-workspace index) and through the runner with a restricted and an empty index, and asserts no 'pad-follows-title' reaches the body. It goes red in all 7 ways when the bundle's marker strips are disabled. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * test(store): a recovery version row carries #1691's history data (TASK-2198 U4) Rebased onto PLAN-2348 U2 (#1691), which adds user_id, line counts and a create marker to version rows. Recovery writes its row through the same UpdateItem insert, so it gets them automatically: no user_id (a system row), not a create, and line counts equal to diff.LineCounts of the stale and recovered bodies. The leg pins that. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC	2026-09-30 11:03:07 -04:00
xarmian	d78ff9bdc5	feat(materialize): worker supervisor with per-OS memory cap (TASK-2198 U3) (#1682) ... * feat(materialize): parent-side worker supervisor with per-OS memory cap (TASK-2198 U3) The Supervisor runs materializer jobs in ONE long-lived child process (`pad __materialize-worker`), started lazily on the first job and fed jobs serially. A job past PAD_MATERIALIZE_TIMEOUT (default 2s) kills the child (goja's Interrupt, sent as timeout_ms a tenth earlier, is only the soft layer); a dead child is respawned on the next job after an exponential backoff (1s doubling, capped at 5 min, reset by a child that lived a minute). Every stop is logged with its reason (deadline, memory, exit, protocol, start, canceled, closed); every start logs the effective timeout, memory limit and cap mechanism. The child's stderr goes to the log, pid-tagged and bounded, never near the protocol. A job whose child died is never retried here. Memory cap (PAD_MATERIALIZE_MEM_LIMIT, default 2GiB, bytes or KiB/MiB/GiB, clamped to 1536MiB-16GiB): - Linux: RLIMIT_AS set on the child by prlimit before its first byte of input; the child only runs its own startup uncapped. - Windows: Job Object with JOB_OBJECT_LIMIT_PROCESS_MEMORY; the child is created suspended, assigned, then resumed: no uncapped window. - macOS: parent-side RSS watchdog every 100ms (proc_info syscall, ps fallback). - Other: deadline kill only. Not wired into the server yet (U4). Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * ci(materialize): run the supervisor on the real macOS and Windows caps (TASK-2198 U3) native-smoke now builds the real materializer bundle (as the go job does) and runs the supervisor, memory-cap and deadline tests on macos-latest (proc_info RSS watchdog) and windows-latest (suspended start + Job Object). PAD_MATERIALIZE_TEST_NO_SKIP turns the cap tests' skips into failures there, so a cap that stops running cannot pass as skipped. Tests: a real Windows processGone (OpenProcess + GetExitCodeProcess), RSS samplers for darwin (the production proc_info path) and Windows (K32GetProcessMemoryInfo), and an allocation helper that writes noise into every byte so macOS memory compression cannot hide it from the watchdog. The worker-loop tests share one loaded Runner through serveWorker (RunWorker's loop, split out); TestWorkerRoundTrip still goes through RunWorker end to end. go test -race -count=3 on the package: 714s -> 413s. docs: the materializer section says the worker and its knobs are inert until U4 wires it in. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * fix(materialize): a worker never outlives its supervising process (TASK-2198 U3, codex r1) If the server died mid-job (SIGKILL, OOM kill, crash) its worker ran on with no deadline kill and, on macOS, no memory watchdog. Two layers: - Linux: Pdeathsig = SIGKILL. The fork runs on a fresh goroutine (startUnlocked), never locked to its thread, so it cannot land on a thread the runtime later retires (golang/go#27505); Go's forkExec re-checks getppid if the parent is already gone. - Every Unix (the only layer on macOS): the worker command calls ExitWhenOrphaned before the bundle loads, polling its parent pid every 200ms against PAD_MATERIALIZE_PARENT_PID and exiting (status 70) when it changes. It writes nothing. - Windows was already covered by the Job Object's KILL_ON_JOB_CLOSE. TestSupervisorOrphanedWorkerExits kills a helper process that supervises a worker hung in a job and requires the worker gone within 1s: 0s on Linux, 105ms with the ppid watch alone; with both layers removed the worker outlives its parent by 5s. It matches native-smoke's TestSupervisor.* selection, so it runs on macOS and Windows too. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * fix(materialize): relative memory cap, refuse when uncappable, GOMAXPROCS=2; nix-safe ps test (TASK-2198 U3) The absolute Linux RLIMIT_AS failed on CI: a 64 MiB allocation died under a 1600 MiB cap, because the runtime there reserved more address space than on the machine the floor was measured on. The cap is now RELATIVE on every OS: once the worker has loaded and before it is sent any job, its usage is read as a baseline and the cap is set to baseline + PAD_MATERIALIZE_MEM_LIMIT — address space on Linux (/proc/<pid>/stat vsize, prlimit, read back), committed memory on Windows (the Job Object's per-process limit, set after the suspended start and load), resident memory on macOS (the watchdog's threshold). Every start logs baseline, limit and effective cap. Lead ruling, REFUSE: if the baseline probe fails or is implausible (0, or above MaxPlausibleBaseline = 8 GiB, ~4.8x the largest measured 1704 MiB), onBaselineProbeFailure kills the worker without sending it the job and fails the job with ErrNoMemoryCap; one WARN until a probe succeeds again. An OS with no cap mechanism never starts a worker and refuses every job with ErrNoMemoryCap, one WARN naming runtime.GOOS. The worker runs with GOMAXPROCS=2: measured loaded-worker baseline 1357 MiB vs 1597 MiB at this host's default (8) and 1704 MiB at 64. MinMemLimit drops from 1536 MiB (an artefact of the absolute cap) to 256 MiB: the corpus grows the worker by at most +10 MiB of address space and +18 MiB resident, and a 4,000-deep nested document by +129..194 MiB. A cgo worker's "pthread_create failed" at the cap is now classified as a memory death. Nix: TestParsePSRSS parses fixed strings only; the live `ps` leg is TestPSRSSLive and skips when there is no `ps` (the Nix sandbox has none; ps is only macOS's fallback), unless PAD_MATERIALIZE_TEST_NO_SKIP is set. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * fix(materialize): Windows kills on the job's memory notification; steady thread set before the baseline (TASK-2198 U3) Windows (CI on c1866957): the Job Object refused the 1 GiB commit (VirtualAlloc errno=1455), but the worker did not die and the job ran to its deadline. The cap no longer depends on the runtime choosing to crash: the job is associated with an I/O completion port, and JOB_OBJECT_MSG_PROCESS_MEMORY_LIMIT for the worker's pid kills it at once (ErrMemoryLimit). errno=1455 on stderr is a second signal. On every OS a cap mechanism that sees the limit hit calls onLimit, and the kill is reported as "reached its <mechanism> cap". Linux (CI's cgo test binary at GOMAXPROCS=64): threads created after the baseline ate the growth budget. The worker now brings its thread set to steady state before readiness (warmThreads: an allocating goroutine per P, then a GC). Measured on a cgo build: threads added after the baseline are 0 at GOMAXPROCS=2 with or without it; at 64, +3 threads and +272 MiB of address space during the corpus without it, +0 and +0 MiB with it. 256 MiB stays the floor (largest growth at GOMAXPROCS=2: +193 MiB, a 4,000-deep nested document; the corpus adds +0 MiB). The cap test's second leg no longer uses GOMAXPROCS=64 (which also changed thread creation) to stand in for a bigger host: the helper inflates its own baseline before readiness, with never-touched address space on Linux, committed memory on Windows and resident memory on macOS. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * feat(materialize): the worker stops after an idle timeout; the next job restarts it (TASK-2198 U3) Dave's ruling on the worker process: GO, with an idle exit. A loaded worker holds ~200 MB resident, which an install that rarely materializes should not pay for all day. PAD_MATERIALIZE_IDLE_TIMEOUT: default 5m, clamped to 30s-24h, 0 = never. After each job the supervisor (re)arms a timer. When it fires, the stop runs only while holding the job slot, so it never takes a worker with a job in flight. It re-checks the idle time, forgets the worker WITHOUT recording a death (backoff is untouched), and closes the worker's stdin so its read loop ends with status 0, with a kill after a grace if it does not go. The stop is logged at INFO, reason=idle. A job that arrives meanwhile waits for the slot, finds no worker, and starts a new one with no backoff wait. Legs: the idle exit after the timeout (INFO, status 0, no WARN); the next job respawns and succeeds with no wait; an idle stop leaves the backoff streak alone (crash, idle, crash waits 2s, not 4s); a job arriving during the stop runs on a new worker; a timer that fires mid-job does not stop that worker; NeverIdle arms no timer; env parsing and clamps. Mutations, each red on its own: recording a death (3 legs), logging as a kill (1), not holding the slot (1), never arming (3). Docs: the knob beside the other two, and `idle` in the stop reasons. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * fix(materialize): Close does not return while an idle-stopping worker still runs (TASK-2198 U3, codex r1) An idle stop forgets the worker before it has exited, so a Close in that window found no worker and returned with it still alive. The supervisor now tracks the worker an idle stop is ending, and Close kills and reaps it. A leg pins it, and goes red without the fix. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * fix(materialize): an idle stop that Close ended says so (TASK-2198 U3, codex r2) The idle stop stored its kill reason unconditionally, so it overwrote the 'closed' of a Close that had already killed the worker, and its stop line said idle. It now claims the reason only if none is set, and the line reports reason=closed. The Close-during-idle-stop leg asserts it. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC	2026-09-30 09:52:49 -04:00
xarmian	a61f50421f	fix: a move refused for a value only the caller can supply gets a way forward on web, CLI and MCP (BUG-3200) (#1690) ... * fix(web): a pane-menu move refused for a value only the user can supply opens the copy dialog's picker, preset to that collection (BUG-3200) * fix(cli): item move refused as missing_required_fields prints what each field needs, from the copy preflight (BUG-3200) * fix(mcp): a remote move refused for a destination value names the field parameter that supplies it (BUG-3200) * fix(web): fence and classify the copy dialog's preset continuation; the handoff is behind stillOnSource (BUG-3200, full vitest) * test(web): the handed-off move keeps the item's id and ref and sends no copy (BUG-3200, lead check)	2026-09-30 09:34:38 -04:00
xarmian	fa2755e183	feat(materialize): headless op-log materializer bundle, goja runner and worker (TASK-2198 U2) (#1678) ... * feat(materialize): headless op-log materializer bundle, goja runner and worker (TASK-2198 U2) Builds the editor's own JS (Yjs -> y-tiptap -> Tiptap schema -> tiptap-markdown -> the tab's flush pipeline) into one IIFE, embeds it, and runs it in goja. Not wired into the server yet (U3 supervisor, U4 write path). - web/src/lib/collab/materializer/: bundle entry (extension list mirrors Editor.svelte's collab branch via the U1 shared modules), op-log replay mirroring wsProvider's MESSAGE_SYNC arm, linkedom DOM shim and polyfills from the spike, and a vendored patch of prosemirror-markdown MarkdownSerializerState.atBlank (quadratic under goja; equivalent). - web/scripts/build-materializer.mjs (esbuild) -> web/build-materializer/, run by `npm run build`; embedded as pad.MaterializerJS. - internal/materialize: Runner (Go-native atob/btoa/TextEncoder/TextDecoder/ getRandomValues, schema-version check, ctx interrupt with a clean VM after), and RunWorker's length-prefixed JSON protocol behind the hidden `pad __materialize-worker` command. - Parity: vitest drives two real Editor.svelte mounts and compares materialize() with the tab's flush on 50 seeded cases; the same cases are committed as testdata/corpus.json and replayed in goja (TestCorpusParity). - CI Go jobs, release tests, Docker and Nix produce the bundle; linkedom and esbuild are new devDependencies. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * test(materialize): pin the atBlank patch's equivalence under V8 too (TASK-2198 U2, codex r1) Codex r1 claimed upstream's /(^|\n)$/ is true for a trailing CR in JS. It is not (checked in V8: false for "\r" and "a\u2028", as the patch returns), but the only equivalence test ran under goja and the patch header cited a V8-side test that did not exist. This adds it, with the line-terminator edge cases, and it fails when the patch checks CR instead of LF. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * build(nix): vendorHash for goja (TASK-2198 U2) Taken from the Nix job's recompute on c7fa1931 (specified sha256-iEqv…, got sha256-P1bQ…), so main does not depend on the heal job after merge. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC	2026-09-30 02:50:10 -04:00
xarmian	a2e7768035	fix(comments): a deleted comment with replies leaves a tombstone (BUG-3252) (#1675) ... * fix(comments): a deleted comment with replies leaves a tombstone (BUG-3252) Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * test(comments): backdate so a tombstone moving updated_at is seen (BUG-3252) Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * chore(mcp,store): v0.59 instructions heading, NUL baseline for comments.deleted_at (BUG-3252) Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * fix(comments): reactions wait for a tombstoning delete; unreact on a tombstone answers 409 (BUG-3252, codex r1) Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * fix(comments): unreact racing a tombstoning delete answers 409, not 404 (BUG-3252, codex r2) Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC	2026-09-30 00:41:20 -04:00
xarmian	a90f75fde2	fix(server): sign protected share-page images on every install (BUG-3305) (#1673) ... * fix(server): sign protected share-page images on every install (BUG-3305) A password or require-auth share link on a self-hosted instance showed a placeholder for every embedded image. The key that signs those image refs was set only in the cloud block, and only with PAD_MCP_PUBLIC_URL set, so it was nil on every self-host, and the mint side omitted the refs. The key is now set right after server.New on every install, from the deployment key every running install has (EnsureEncryptionKey generates one on SQLite and refuses to start Postgres without one). The MAC stays domain-separated from the key's other uses. The cloud-only site is gone. Deliberate behaviour change: after upgrade, self-hosted protected share pages render their images. So does a cloud instance without PAD_MCP_PUBLIC_URL, which had the same gap. Comments that gave the old reason for a missing key ("self-host without an encryption key") are corrected; the no-key branch stays as the defensive floor, with its tests. Evidence: a live boot of the built binary as a plain self-host. A password share resolves with a signed image ref; the signed asset answers 200 and the unsigned one 404. The same run on main's binary has no ref. Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9 * docs(server): say how the share-asset MAC stays apart from the key's other uses (BUG-3305 codex r1) Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9	2026-09-29 20:35:28 -04:00
xarmian	b510d67696	fix(oauth): sweep expired token, code and PKCE rows (BUG-3301) (#1671) ... * fix(oauth): sweep expired token, code and PKCE rows (BUG-3301) Nothing deleted OAuth rows by age. Each refresh leaves the chain's previous access and refresh rows inactive and inserts a fresh pair, measured at +1 access and +1 refresh row per refresh (48 rows a day for one active client at the 1 h access lifetime), forever, on Pad Cloud and on any https self-host that turns MCP on. The token reaper now also runs SweepExpiredOAuthRows every hour. A row goes once its requested_at (its own issuance, measured) is older than its token's lifespan plus 24 h: - refresh and authorization codes: the refresh lifespan. A replayed rotated token or a reused code keeps revoking the family for as long as the token could have been valid, because fosite checks "inactive" before "expired". - access: the access lifespan. Nothing reads an expired access row. - PKCE: the code lifespan. Only abandoned authorizations leave one. The lifespans come from the running OAuth server (defaults when OAuth is not built). No row a live chain or a concurrent exchange could use is ever below a cutoff. Deletes run 500 rows per statement with a 20 ms pause between batches. Measured on SQLite with 50,000 rows and a concurrent token write every 5 ms: with no pause, one write waited 1.55 s; with the pause, the worst write took 52 ms. The per-tick cap is 200 batches per table. Connected Apps and the MCP resume count now drop a chain whose refresh token has been expired for the grace period, so their "upper bound" comments say how far. The deployment guide's "never swept" gap is gone. Tests: store cutoffs, zero cutoffs, and batches with the cap, on SQLite and Postgres. A server end-to-end test through /oauth/token: the live chain survives and still refreshes; an aged-out chain leaves Connected Apps and the resume count; a swept token is refused without revoking the family; a retained rotated token still revokes it. Mutants: inverted age comparison (store tests fail), refresh cutoff at now (e2e fails: the live chain's refresh answers invalid_grant). Claude-Session: https://claude.ai/code/session_01NidgPePY7UvHcEJCMkD2Qz * fix(oauth): code retention adds the code lifespan; Capped means "may remain" (BUG-3301, codex r1) - A code's requested_at is when it was authorized, but the first refresh token it yields starts at the exchange, up to a code lifespan later. The code cutoff now adds the code lifespan to the refresh lifespan, so reuse detection lasts as long as that refresh token could be valid. The comment no longer claims "the chain's lifetime": a chain renews forever. - Capped says the cap was used, so rows MAY remain; the last full batch may have been the end. Claude-Session: https://claude.ai/code/session_01NidgPePY7UvHcEJCMkD2Qz * fix(oauth): a non-expiring lifespan skips its tables (BUG-3301, codex r2) fosite reads a negative lifespan as "never expires"; the cutoff math turned -1 into "now minus the grace", which would delete a live non-expiring token after a day. A table whose retention depends on a negative lifespan now gets a zero cutoff, which the store skips (codes depend on both the code and refresh lifespans). Claude-Session: https://claude.ai/code/session_01NidgPePY7UvHcEJCMkD2Qz	2026-09-29 19:29:17 -04:00
xarmian	22bfe8cc89	feat(mcp): off-cloud Host allowlist on the non-API MCP paths (TASK-2319, PLAN-2310 U3) (#1665) ... * feat(mcp): off-cloud Host allowlist on the non-API MCP paths (TASK-2319, PLAN-2310 U3) Off cloud, /mcp, /oauth/* and /.well-known/oauth-* answer 421 Misdirected Request (JSON) when Host does not name the configured origin, MCP URL or auth-server URL. Matching is case-insensitive with the port defaulted from the configured scheme; a loopback origin admits every loopback spelling on its port; an empty Host is refused. It runs after the availability gate (MCP off still answers 404 whatever the Host) and before auth, audit and the OAuth group's rate limiter. The /api/v1 routes are not wrapped: they keep the API perimeter, and the in-process dispatcher sends them an empty Host. Cloud is unchanged. mcp-go's DisableLocalhostProtection stays, with its comment rewritten. The e2e harness sets PAD_URL to its base URL. Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9 * feat(mcp): honour X-Forwarded-Host from trusted proxies; 421 names the fix (TASK-2319 lead review) Behind a proxy that rewrites Host to the backend's name (Azure Application Gateway and App Service, some ingresses) every MCP request would get a 421. The allowlist now checks the first X-Forwarded-Host entry when the TCP peer is in PAD_TRUSTED_PROXIES, the same trust test cliAuthScheme and TrustedProxyRealIP use; from any other peer the header is ignored, so DNS-rebinding protection is unchanged. The 421 names the received host and its header, the configured origin, and the proxy fix, and the first refusal per host is logged at WARN behind one overall limiter (5, then 1 a minute; at most 1024 hosts remembered). Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9	2026-09-29 15:25:44 -04:00
xarmian	ae7f77d9d8	feat(mcp): construct MCP off cloud, mount always, gate per request (TASK-2317, PLAN-2310 U2) (#1663) ... * feat(mcp): construct MCP off cloud, mount always, gate per request (TASK-2317, PLAN-2310 U2) The remote MCP capability (transport, in-process dispatcher and resource fetcher, audit writer, session tracker) is now constructed on every install by wireMCP, and the OAuth server, claim secret and connections backfill whenever the resolved auth-server URL is https. Nothing is served until Server.mcpAvailable() / oauthAvailable() say so, per request: every DR-5 route is mounted always and answers the JSON 404 while unavailable, before auth, audit or rate limiting. The two audit-history routes are unchanged. - Discovery takes its URLs from configuration only. The four "https://"+r.Host fallbacks and rewriteMcpSubdomain are gone; with nothing configured the documents answer 404, and the /mcp 401 is a bare Bearer realm="pad" unless OAuth is available. - resource_metadata follows RFC 9728 section 3.1, so a resource with a path (a self-host's <origin>/mcp) points at a route that exists. Byte-identical for a path-less resource (cloud). - An explicitly set PAD_MCP_PUBLIC_URL / PAD_AUTH_SERVER_URL keeps its historical spelling (TrimRight "/"), so the OAuth audience of every existing deployment is byte-identical and issued tokens keep validating. - Protected share-link image signing gets its own key field, set exactly where the claim secret used to be (cloud with PAD_MCP_PUBLIC_URL), so building OAuth on an https self-host changes nothing on share pages (lead ruling B; BUG-3305 is the deliberate fix). - The session and setup payloads keep mcp_public_url cloud-only until the connect modal gains its PAT path (DR-8, unit 6). - The claim routes answer 404 instead of 412 claim_disabled where OAuth is not available; the auth-server metadata answers 404 instead of 503. Tests: the DR-5 route table across the eight acceptance states and the upgrade test (no mcp_enabled row, https origin: every gated route 404, the audit routes unchanged, a valid dispatcher claim refused; the control leg reaches every route and the same claim succeeds), both over the production wiring; an https self-host OAuth flow for audience <origin>/mcp; a PAT call over http that writes an audit row. Release gate (PLAN-2310): must not ship without units 3, 4 and 7. Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9 * docs(mcp): the availability gates run inside the API perimeter on /api/v1 routes (TASK-2317 codex r1) Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9	2026-09-29 14:13:03 -04:00
xarmian	eb167bea00	feat(mcp): MCP addressing from config; mcp_enabled setting with an env lock (TASK-3302, PLAN-2310 U1) (#1662) ... * feat(mcp): resolve MCP addressing from config; mcp_enabled setting with an env lock (TASK-3302) PLAN-2310 unit 1 (DR-2, DR-3, the data behind DR-7). No route changes: nothing is gated or exposed by this unit, and U2 wires the predicate into the routes. Addressing (internal/config/mcp.go, ResolveMCPEndpoints): - The origin is cfg.URL (PAD_URL, toml url, --url), else PUBLIC_URL, never the Host:Port fallback and never a request's Host. - The MCP URL, which is also the OAuth audience, is PAD_MCP_PUBLIC_URL, else <origin>/mcp. The auth-server URL is PAD_AUTH_SERVER_URL, else the origin. - A value must be an absolute http(s) URL without user info, query or fragment; an origin also has no path. An unusable value never fails startup (PAD_URL also serves the CLI). It is logged, leaves its field empty, and keeps MCP unavailable. An unusable override does not fall back to the origin. - HTTPS() is the auth-server URL's scheme: https means OAuth plus PATs, http means PATs only. The setting (internal/server/handlers_admin_mcp.go): - platform_settings.mcp_enabled, default off, read per request with no cache (the webmcp_enabled precedent). A read error counts as off. - PAD_MCP_ENABLED (true/false/1/0/yes/no/on/off) forces it and locks the toggle. An unparseable value forces nothing and is logged. Cloud is always on. - mcpAvailable() is DR-1's predicate: cloud, or the setting on and the addressing usable. - GET/PUT /api/v1/admin/mcp, admin-only. GET reports enabled, source (setting|environment|cloud), locked and the readiness: origin, MCP URL, auth-server URL, https, auth methods, problems, state on|off|blocked with the reason, and the resume counts. PUT answers 409 set_by_environment under the env lock (the decision-provider shape), 403 managed_by_operator on cloud, and 400 without {enabled}. It is audit-logged as settings_changed. - The resume counts (internal/store/mcp_resume.go) count live OAuth chains the way the Connected Apps page does (request_id with an active access or refresh row), and PATs the way /mcp accepts them (has a user_id; expiry parsed as a time, as ValidateToken does). Tests: - config resolution table (11 cases); PAD_MCP_ENABLED parsing and Load. - Store counts: a rotated chain counted once, a revoked chain excluded, a refresh-only chain counted; the legacy and expired PATs excluded. - Router tests: default off (the upgrade case); https on; http PAT-only; blocked without or with an unusable origin; the env lock with a stored value underneath; cloud; admin-only and body validation; a setting read error is off, with a control leg. - Mutants, each BUILD OK and killed by the test aimed at it: env override ignored; availability ignoring addressing; PUT skipping the env lock; origin path allowed; PUT allowed on cloud; read error counted as on (error branch only); invalid override falling back; expired PATs counted; refresh-only chains missed. Docs for the variables and the endpoint land with PLAN-2310 U7. Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9 * fix(mcp): canonical MCP URLs; PAT count mirrors /mcp's user check; one setting read per response (TASK-3302) Codex round 1 on 7b000405 (gpt-6-luna): - P2, URL parsing (fixed). The origin's no-path rule read the decoded path, so https://host/%2F passed. An empty port (https://host:) was kept, HTTPS:// was refused, and host case and default ports were preserved. parseMCPURL now judges the path on the escaped path, refuses an empty port, and lower-cases scheme and host. It drops a default port (443 for https, 80 for http), brackets IPv6 hosts, and trims a trailing slash. The result is one canonical spelling, which matters because the MCP URL is the OAuth audience, compared byte for byte. A new test pins that cloud's PAD_MCP_PUBLIC_URL (with and without a trailing slash) resolves to exactly the audience cmd_server.go has always built, TrimRight(value, "/"). If U2 switches the audience to the resolved URL, no issued Cloud token changes audience. - P2, PAT count vs /mcp (fixed). /mcp also refuses a PAT whose user row is gone. The count now joins users. api_tokens.user_id has a foreign key (migration 068), so such a row cannot exist today; the join is defensive. A fixture for it was tried: the FK refused the insert, and its t.Skipf would have skipped the whole test. So it was removed rather than left skipping the assertions next to it. - P2, orphan OAuth chain counted but not shown (REJECTED with evidence). ListUserOAuthConnections does not fail on a chain without an oauth_connections row. It takes the documented HasConnection == false fallback and lists it (connected_apps.go:190-199); only a DB error fails. A new test seeds an orphan chain and asserts the count and the page both see it. - Minor (fixed). The readiness response read the setting twice. mcpAvailableWith(on) takes the value already read, so the reported state and the predicate cannot disagree across a concurrent toggle. - Test gap (fixed). The PUT's audit event is now asserted. Mutants, each BUILD OK and killed by the test aimed at it: default port kept; host not lower-cased; path judged on the decoded path; empty port accepted; PUT not audited. Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9 * fix(mcp): refuse a zoned IPv6 origin instead of re-spelling it (TASK-3302) Codex round 2 on a655c56e (P2): canonicalising a zoned IPv6 host (http://[fe80::1%25eth0]) dropped the %25 escape and lower-cased the zone, emitting a malformed URL. A zoned link-local address names an interface on this machine, and no remote client can reach it, so parseMCPURL now refuses it with a reason. Test added; the mutant that disables the check is killed by it. Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9 * docs(mcp): the resume count is an upper bound until BUG-3301 sweeps expired grants (TASK-3302) Codex round 3 (P2): CountLiveOAuthConnections counts active rows, and active is cleared on revocation, not expiry, so expired-but-unswept grants are counted. That is the same definition the Connected Apps page uses, and it lists the same chains. An expiry filter here alone would make the panel and the page disagree. Disposition: keep them consistent, state the count as an upper bound in both comments, and add the inflation to BUG-3301's scope, since its sweep fixes both at once. Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9	2026-09-29 12:46:03 -04:00
xarmian	237333681b	feat(attachments): attach binds an unattached upload to an item, so its share renders it (TASK-2247) (#1652) ... The July audit's share-page broken-image finding was fixed by BUG-2389 and TASK-2637: the share byte endpoint serves variants of attachments the shared item OWNS. What was left: an image uploaded with `pad attachment upload -` (item_id NULL) and then embedded is never owned, so its share shows the placeholder. A census of this workspace found 4 such refs out of 13, all from that path. Lead ruling (option C): fix it explicitly, with no implicit binding on content writes. - POST /workspaces/{ws}/attachments/{id}/attach {item}. It binds a live, unattached original in this workspace, and its live variants, to an item. Every UPDATE requires item_id IS NULL, so an attached row is never moved, and the bind is exclusive with the never-attached GC claim. The item is pinned like CreateAttachmentForLiveItem. - Authorization: the caller can see and edit the item, and uploaded the attachment or is a workspace owner. An attached row answers 409 to a caller who can see its owner, and the shared 404 to anyone else. Restricted members and guests cannot read unattached rows, so they get the 404. That closes the case where a guest binds an id they know to an item they can edit and publishes it through its share. - The attachment_attach capability. `pad attachment attach <id> <item-ref>` refuses against a server that does not advertise it, because an older build's bare 404 would read as "attachment not found". - The `upload -` help text, the attachment group help, both skill copies and CLAUDE.md say an unattached image does not render on share pages. - No MCP surface: pad_attachment has no upload. Tests: 12 server/store tests, including an end-to-end share leg (404 before, variant bytes after), and 9/9 mutants killed. CLI capability gate: 2 tests, mutant killed. e2e: the share renders the image after attach; with attach made a no-op it goes red at "attached: no placeholder". Claude-Session: https://claude.ai/code/session_01NidgPePY7UvHcEJCMkD2Qz	2026-09-28 19:25:55 -04:00
xarmian	c29af5731c	test(mcp): pin the initialize and tools/list wire bytes on both transports (TASK-2306 unit 1) (#1610) ... Golden files for the initialize and tools/list results, from the production bindings: the real `pad mcp serve` subcommand for stdio, and registerRemoteMCP behind NewRemoteTransport for remote /mcp. The remote registration is extracted from the cloud branch of serverStartCmd into registerRemoteMCP, a mechanical move, so the test drives the binding rather than assembling its own server. It turns 'behavior-neutral' into a checked claim for the alias/funnel work that follows and for any SDK bump. Re-injecting BUG-2302 (destructiveHint true on read-only tools) fails both transports on the exact line. serverInfo.version is the only normalised field. Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9	2026-09-27 10:03:47 -04:00
xarmian	9b7e0832ac	fix(comments): deleting a comment that has replies answers 409 comment_has_replies instead of 500 (BUG-3252) (#1608) ... * fix(comments): deleting a comment that has replies answers 409 comment_has_replies instead of 500 (BUG-3252) comments.parent_id references comments(id) with no ON DELETE, so deleting a parent with replies failed the foreign key inside the store. Every door said "internal error", the web delete button included, and nothing was deleted. Per the lead's ruling (reversible with Dave's veto), the store now refuses by name. The DELETE carries a NOT EXISTS on replies, so the refusal is part of the statement itself, and a zero-row result is told apart by counting. Nothing is deleted. Every door reports it: - Both server routes answer 409 comment_has_replies with details {comment_id, reply_count}. - The CLI root writes a structured marker, so stdio MCP reports the code and hint rather than server_error. - Remote MCP passes it through the 409 allow-list with the same hint. - The web timeline shows the server's message in place of "An internal error occurred". MCP 0.58 (behavior bump, on the v0.45 grounds). Cascade versus tombstone stays an open product decision; this refusal is the floor under either. Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9 * fix(comments): a delete whose replies vanish between statements retries instead of reporting not-found (BUG-3252, codex r2) Under Postgres READ COMMITTED the conditional DELETE and the reply count take separate snapshots, so a reply the DELETE saw could be gone by the count, and a comment that still existed was reported missing. The zero-row branch now rechecks that the comment exists and, if it does, runs the conditional delete again, at most three times. Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9 * test(comments): pin the replies refusal on Postgres (BUG-3252) The server and MCP tests run on SQLite, so the conditional DELETE had never executed on Postgres. This test runs there (make test-pg). Against the bare DELETE mutant it fails with comments_parent_id_fkey, which is the original defect. Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9	2026-09-27 09:11:48 -04:00
xarmian	0f1069e9df	feat(comments): agents can edit and delete comments from the CLI and every MCP transport (TASK-2695) (#1607) ... Web users could edit and delete comments since PLAN-1662. Agents, who write most comments, could only append corrections. - Server: new item-scoped routes, PATCH/DELETE /workspaces/{ws}/items/{ref}/comments/{id}. They answer 404 unless the comment is on {ref}, then run the existing handlers unchanged (edit is author-only, delete needs item-edit). GET /server/capabilities advertises them as item_scoped_comment_writes. - CLI: `pad item comment-edit <ref> <comment-id> [message|--stdin]` and `pad item comment-delete <ref> <comment-id>`. Both refuse to send to a server without the capability, whose bare 404 would otherwise read as "comment not found". `pad item comments` now prints each comment's id, plus "edited" once its body has changed. - models.Comment serialises a derived `edited` bool, true when updated_at > created_at (the web timeline's rule). - MCP 0.57: pad_item edit-comment / delete-comment and a comment_id param, on stdio, remote /mcp and WebMCP. All three use the item-scoped routes, so the check that the comment is on ref happens on the server for every transport. Remote packages a delete's 204 as the CLI's JSON. Found along the way, and out of scope: deleting a comment that has replies answers 500, because parent_id has no ON DELETE. Filed as BUG-3252. Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9	2026-09-27 07:59:11 -04:00
xarmian	2d51308d39	fix(collab): a schema-version rebuild sets unflushed edits aside instead of deleting them (BUG-3244 A) (#1598) ... * wip(collab): the schema-version rebuild sets unflushed edits aside and the item stays marked (BUG-3244 A) Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9 * wip(server): set-aside rows refuse every content write and restore unless overwrite_pending_edits, which discards and reports them (BUG-3244 A) Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9 * wip: set-aside read/discard endpoints and pad item set-aside; census and migrated-table entries (BUG-3244 A) Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9 * wip(store): set-aside edits travel in the workspace bundle; the migrate gate asks the op-log only (BUG-3244 A) Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9 * wip: set-aside advice branches at every door that names a remedy; mark-only doors share one helper (BUG-3244 A, ruling 1) Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9 * test(cli): set-aside notices name the right remedy (BUG-3244 A) Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9 * docs: the rebuild sets unflushed edits aside; freeze reason, API docs and op-log prose follow (BUG-3244 A) Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9 * test(web): identity-gate rows re-reviewed for the set-aside reason threading (BUG-3244 A) The edits in each covered unit are a pure pendingEditsReason(e) argument or string branch, or a reason passed to the dialog; no await, write or fence changed. Three anonymous-unit matchers follow the new call text, and the keepalive failure row lists the pure classifier in may. Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9 * test(server): a restore override counts the set-aside rows it discarded (BUG-3244 A) Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9 * fix(store): the rebuild sets aside the rows its DELETE returned, so an append between choosing and clearing is not lost (BUG-3244 A, codex r1) On Postgres each statement takes its own snapshot and the setup lock does not stop a connected peer appending, so a row committed between the old INSERT ... SELECT copy and the DELETE was deleted without being set aside. The kept rows are now chosen from DELETE ... RETURNING. A test hook in the gap drives a Postgres leg; a mutant that chooses before the gap loses the row. Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9 * test(web): playbook save row re-reviewed after the BUG-3230 U3 rebase (BUG-3244 A) Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9 * fix(store): the set-aside move's first statement is the DELETE, so SQLite never upgrades a stale read snapshot (BUG-3244 A, codex r2) Reading the watermark first took a read snapshot on SQLite; a concurrent append would then make the DELETE fail with SQLITE_BUSY_SNAPSHOT instead of waiting. The watermark is now read after the DELETE, where a later value is only more accurate. Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9 * feat(mcp): ToolSurfaceVersion 0.56, content_state gains superseded_set_aside (BUG-3244 A) The value reaches pad_item get / full list, pad_playbook run and the item resource's line above the body, and its remedy differs from applied_pending_flush (opening the item will not restore the edits). An additive bump with a version.go entry; README, CLAUDE.md and the MCP instructions follow. Read/discard of set-aside rows stays off the catalog. Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9 * test(server): after a bumped rebuild the next open is granted the seed once and the item stays superseded (BUG-3244 A x BUG-3240) Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9	2026-09-27 05:40:55 -04:00
xarmian	b459892d8b	fix(search): the collection filter resolves exact-first per workspace in scope, so a collection named task is no longer shadowed by tasks (BUG-2659, BUG-2839) (#1576) ... * fix(search): the collection filter resolves exact-first per workspace in scope, so a collection named `task` is no longer shadowed by `tasks` (BUG-2659, BUG-2839) /search matched its collection filter as a literal slug, so shorthand found nothing and both clients normalised it first, which rewrote `task` to `tasks` even where a `task` collection existed. The handler now resolves the filter with resolveItemCollectionSlug in every workspace in scope and filters the store on the resulting collection IDs; one slug cannot express that, since the same input can resolve differently per workspace. The capability endpoint advertises search_collection_resolution, and the CLI sends the name as typed only to a server that does. The in-process remote MCP mapper stops normalising outright. Also corrects cascadeTitleRename's doc comment, which named links.ReplaceTitle for a cascade that rewrites through links.RewriteBracketsAt (BUG-2839). * chore(mcp): tool surface 0.55 — pad_item.search's collection resolves exact-first per workspace (BUG-2659) Lead-ruled BEHAVIOR bump on the v0.36/v0.40 grounds: in a workspace holding both task and tasks, the same call now answers from task.	2026-09-26 12:20:58 -04:00
xarmian	9059862431	fix(session): a headless disarm holds for as long as its session lives: an unverifiable owner fails closed and is kept, and the file names the session, not the exiting command (BUG-2771, BUG-3227) (#1574) ... * ci(smoke): a headless disarm on Windows must survive later reads (BUG-2771) Added before the fix, so the first CI run shows it red against the pre-fix binary. * fix(session): an arm-state file whose session cannot be verified is kept and fails closed, so a disarm is never forgotten (BUG-2771) The unknown owner verdict (every headless file on Windows; on unix an unexaminable /proc entry or socket) was treated as dead: the next read reaped the file and auto_arm re-armed over an explicit disarm. It now resolves to LocalArmUnverifiable, which is not armed and not reaped. `pad session arm` reports the real state, `status` explains it, and a new `pad session reset` removes the kept file so auto_arm decides again. * ci(smoke): the headless-disarm step covers unix too, owned by the step's shell via PAD_SESSION_PID (BUG-3227) Pushed on top of the BUG-2771 fix but before BUG-3227's, so this run shows the unix path red pre-fix while Windows turns green. * fix(session): a headless arm or disarm is owned by the session process, not the exiting command, and a write no session can own is refused (BUG-3227) A headless arm-state file recorded os.Getpid(), the pid of the short-lived `pad session arm`/`disarm` itself, so on unix it was dead on the next read and reaped: an explicit disarm never held. It now records CaptureSessionOwner() (PAD_SESSION_PID, then CLAUDE_PID), as the session registry does. With no session process to name, the owner is the command, and arm/disarm refuse rather than report a consent change that did not happen. * docs(session): arm and disarm --help say how a headless session is identified, and that with no owner they exit non-zero (BUG-3227)	2026-09-26 11:55:16 -04:00
xarmian	e422d6b183	fix(cli,web): the CLI activity surfaces apply the change-pill display rule through a Go twin sharing one fixture with the web, and the admin audit log expands a row to its full record (BUG-2789) (#1570)	2026-09-26 09:47:55 -04:00
xarmian	547527d7dc	fix(server): Content-Disposition carries a non-ASCII or unquotable filename in filename* with an ASCII fallback, so NBSP and zero-width names no longer reach clients as Go escape text (BUG-3190) (#1569) ... * fix(server): Content-Disposition carries a non-ASCII or unquotable filename in filename* with an ASCII fallback, so NBSP and zero-width names no longer reach clients as Go escape text (BUG-3190) * test(server): the Content-Disposition population carries characters the extended form must encode, and the doc comment survives gofmt (BUG-3190)	2026-09-26 09:47:51 -04:00
xarmian	bd21b97c70	refactor(store): delete the NUL SUSPECT path, now that the predicate sees a NUL behind a repeated key (BUG-3220) (#1561) ... The suspect class held pre-filter matches the NUL predicate cleared, and migrate-to-pg cast each one on the destination, because one shape in it (a NUL shadowed by a literal duplicate key) was invisible to every layer. Since BUG-2812 the token walk reports that shape as an ordinary violation and the ordinary repair fixes it, so the only suspects left were doubled-backslash literals, which PostgreSQL accepts. ScanNUL judges stored bytes, so the same holds for any earlier release's rows (the PostgreSQL 12/17 x UTF8/SQL_ASCII/LATIN1 matrix is on the trail). Deleted: NULSuspect and the Suspects bucket, the destination oracle (CheckJSONBAcceptable and its classification), RepairSuspectValue and its buckets, the preflight's destination cast (preflightNULForMigration no longer takes a destination), the suspect heading in scan-nul/repair-nul, the exported RepairJSONEscapes whose only other caller was the suspect repair, and the docs paragraph. MigratedTables and nulKeyPredicate move out of the deleted file.	2026-09-26 01:09:03 -04:00
xarmian	c31fb06c86	fix(store): the migrate-to-pg preflight refuses a stored value that is not valid UTF-8, and scan-nul / repair-nul report and repair it (BUG-3222) (#1559) ... * fix(store): the migrate-to-pg preflight refuses a stored value that is not valid UTF-8, and scan-nul / repair-nul report and repair it (BUG-3222) PostgreSQL refuses a text or jsonb value holding invalid UTF-8 (SQLSTATE 22021 under a UTF8 database); SQLite stores it silently. A legacy row of that class failed `pad db migrate-to-pg` partway through the copy, leaving the destination half-populated, and the NUL preflight never looked for it. The destination cast on NUL suspects caught a sliver of the class by coincidence (a value that also mentioned the escape), which BUG-3220 is about to delete. The NUL census now reads every protected column in full as a BLOB and reports invalid UTF-8 as a violation kind (merged with a NUL finding on the same row), so scan-nul lists it, repair-nul replaces each invalid sequence with U+FFFD, and the preflight refuses on it through the path it already had (migrated tables only; nothing written first). Measured on a copy of a 306 MB live database: scan-nul 0.51 s -> 1.52-1.62 s, peak RSS +10 MB. * fix(store): a value with both defects on an incomplete-key row is counted once (merged by rowid), and the JSON and SQLSTATE wording is exact (BUG-3222 codex r1) * test(store): no protected table may declare a column that shadows SQLite's rowid, which the NUL scan merges on (BUG-3222 codex r2)	2026-09-26 00:15:21 -04:00
xarmian	7376023ff2	fix(server): the request-body gate reads tokens with jsontext, so a NUL behind a repeated key or beside 1e999 is refused, and a repeated request member is refused on every body (BUG-2812) (#1550) ... * fix(server): the request-body gate reads tokens with jsontext, so a NUL behind a repeated key or beside 1e999 is refused, and a repeated request member is refused on every body (BUG-2812) The body gate and the store's JSON-column guard decoded into map[string]any and walked the tree. The tree is a lossy model: it keeps one value per repeated key, so a NUL in a shadowed copy passed (and the typed decode could merge it into storage), and a number no float64 holds made the decode fail, so the gate answered "no NUL" for a body the typed decode then accepted. Walking that tree also cost about 2x on any body that merely mentioned an escape. - internal/jsonscan wraps github.com/go-json-experiment/json/jsontext (pinned v0.0.0-20260820222146-c27c302e5fc3, BSD-3, The Go Authors). It is the module's only importer; its doc comment names the exit to stdlib encoding/json/jsontext. A differential test pins its acceptance to encoding/json v1's, which found that jsontext refuses a lone surrogate v1 accepts; that case falls back to v1's unquote. - scanRequestBody reads every body once, recording a NUL and a repeated request member separately and reading to the end, so malformed input still reports nothing. Repeats count in the request's own structure, and not in caller data under JSON-encoded keys (BUG-2896 collapses those). Names are case-folded where the destination is a struct: decodeJSONBytes reflects its destination's TOP-LEVEL type, so a struct folds its members, a slice of structs each element's, and a map (admin settings, admin user limits) none. Below the top level only an exact repeat counts. decodeJSONBytes and both bundle paths refuse either. - textguard's document walk and artifactIsBindableText read tokens too, so every layer refuses the shadowed NUL together: the one textguard.KnownGaps case moves into Corpus. - The store's suspect path stays, redundant but correct, by ruling; its deletion is its own unit. Its instruments now pin what is true: no refused value is a suspect, and the remaining suspects are the doubled-backslash class the destination accepts. - The map-model walk survives only as the differential's oracle. * fix(server): update vendorHash and the Postgres-only suspect instruments for the token scan (BUG-2812) The Postgres-backed legs of the suspect path fired when the shared predicate learned to see a NUL behind a repeated key: that value is now a violation everywhere, and the store's own guard refuses it before the destination cast. The preflight legs now assert that, the unmigrated- table fixture uses a doubled-backslash literal (still a suspect), and PostgreSQL's native refusal stays measured by TestNativePostgresAgreesWithTheCorpus. vendorHash is the value this PR's Nix job recomputed.	2026-09-25 21:48:30 -04:00
xarmian	25d76af0ba	fix(fields): field writes keep every number's literal, so an integer above 2^53 is no longer rounded by a write that never named it (BUG-3202) (#1545) ... * fix(fields): field writes keep every number's literal, so an integer above 2^53 is no longer rounded by a write that never named it (BUG-3202) Every door that writes an item's fields decoded the blob into map[string]any, turning each JSON number into a float64, so the re-encode rounded integers above 2^53 (9007199254740993 stored as ...992). The stored blob was decoded that way too, so a fields_patch to `status` rewrote every large integer in the row. - models.DecodeFieldsJSON / DecodeJSONKeepingNumbers decode with UseNumber. They accept and refuse exactly what json.Unmarshal does (trailing data, out-of-float64-range numbers); only the representation changes. - Used at every fields decode that feeds a write: create, full-fields update, the fields object on the wire, fields_patch (request and store merge), the carried-blank-relation normalisation, the Append/Set metadata helpers, move, bulk move, cross-workspace copy and its preflight, and the move/copy field_overrides (models.FieldValues). - items.validateFieldType accepts a json.Number that parses as a finite number. items.coerceValue and the CLI's parseFieldFlag keep a numeric string that is a JSON number literal as that literal. - Read-only decoders are unchanged (display, hydration, guards). Claude-Session: https://claude.ai/code/session_01A3rXV7GDLAhBe9mL4V6z2m * fix(fields): compare field values by value, so an equal number in another spelling is no change and two integers that share a float64 are (BUG-3202) Field writes now keep a number's literal, so one value can be stored as different text (1e3 vs 1000). Three places compared field values: - item.updated's change detector compared the fields blob's TEXT whenever it was not masking a string done key: an equal-value rewrite, or a key-reorder-only rewrite, emitted item.updated. - the activity `changes` diff and the BUG-3163 carry check compared float64s: equal spellings were fine, but two different integers above 2^53 compared equal, so such a change went unrecorded and a changed reserved value passed as carried. models.CanonicalJSONNumbers gives every number one exact canonical spelling. All three decode keeping literals and compare canonical forms. Claude-Session: https://claude.ai/code/session_01A3rXV7GDLAhBe9mL4V6z2m	2026-09-25 14:22:09 -04:00
xarmian	8d81f3a75c	fix(timeline): structured entries get a persisted id instead of a positional one that renumbers when a sibling moves (BUG-2788) (#1540) ... * fix(timeline): structured entries get a persisted id instead of a positional one that renumbers when a sibling moves (BUG-2788) An implementation note or decision with no usable id, or with an id an earlier entry already claims, fell to the timeline's "<prefix>-idx-<i>" fallback. That id is the cursor's tie-breaker and the client's list key, and it changed whenever an earlier entry was inserted or removed, or when the first of two duplicates was deleted, so an entry could be skipped or shown twice across a page boundary. models.StructuredEntryIDKey is now the one rule for what a stored id means in the timeline, and models.EnsureStructuredEntryIDs persists a minted id on exactly the entries that rule leaves without one (first holder of a duplicate keeps it). It runs in both append helpers, in workspace import, and in a startup backfill that is silent by ruling (no seq, updated_at, activity or event); the Q1 census that licenses the silence is in its comment. A repaired entry's timeline id changes once, from the positional form to its persisted id, and never again. No MCP surface change. Claude-Session: https://claude.ai/code/session_01A3rXV7GDLAhBe9mL4V6z2m * fix(timeline): the entry-id repair touches only what the timeline reads, keeps values byte-stable, and its backfill compares the fields it read (BUG-2788 review round 1) - Skip a kind whose array the timeline's typed extraction rejects (a non-object element or non-string id hides the whole kind there), so the repaired population is exactly the positional one. - Leave a blob that is not exactly one JSON value untouched, instead of dropping its trailing bytes; re-encode without HTML escaping. - The backfill's UPDATE compares the fields text it read as well as seq, so a write that changed fields without moving seq is never overwritten. - Import's comment no longer claims byte-verbatim storage for a repaired blob. Claude-Session: https://claude.ai/code/session_01A3rXV7GDLAhBe9mL4V6z2m * fix(timeline): the entry-id repair leaves a blob the timeline cannot parse at all (BUG-2788 review round 2) The timeline parses the whole fields blob with a plain Unmarshal first, which fails on a number outside float64's range and then shows no entries; the repair decoded with UseNumber and would still have minted ids there. It now requires that same first-stage parse. Claude-Session: https://claude.ai/code/session_01A3rXV7GDLAhBe9mL4V6z2m	2026-09-25 07:10:26 -04:00
xarmian	70f7c0c15c	fix(watchevents): a resume from a cursor inside an abandoned id space is refused on every arm, instead of being served the new space's ids or told it is caught up (BUG-2743, BUG-2728) (#1518) ... * fix(watchevents): a resume from a cursor inside an abandoned id space is refused on every arm, instead of being served the new space's ids or told it is caught up (BUG-2743, BUG-2728) Every arm that detects a watch-stream id-space restart (the epoch arm and both counter-backward arms) now records the old space's high-water mark, and replaySince refuses any cursor at or below it. The boundary is never cleared: climbing past the old peak does not make a cursor below it any less ambiguous. Claude-Session: https://claude.ai/code/session_01A3rXV7GDLAhBe9mL4V6z2m * docs(BUG-2743): the counter-reset arm's prose states the boundary's cost and which id the resume check compares (codex round 1) Claude-Session: https://claude.ai/code/session_01A3rXV7GDLAhBe9mL4V6z2m * fix(watch): pad watch --stream prints a PAD (resync) line on sync_required, so the agent reading it learns it missed notifications instead of carrying on (BUG-2743 follow-through) Clearing the cursor was the process's whole response; the consumer of its stdout could not see a cursor and was never told. The line goes through the same consent gate as a notification, and repeats with no notification between them are folded into one. Claude-Session: https://claude.ai/code/session_01A3rXV7GDLAhBe9mL4V6z2m * docs(plugin): tell the agent what a PAD (resync) monitor line means and what to do with it (BUG-2743 follow-through) Claude-Session: https://claude.ai/code/session_01A3rXV7GDLAhBe9mL4V6z2m * docs(BUG-2743): the resync line says the stream MAY have missed notifications, since sync_required means the span is unvouched, not certainly lossy (codex round 3) Claude-Session: https://claude.ai/code/session_01A3rXV7GDLAhBe9mL4V6z2m	2026-09-24 23:40:26 -04:00
xarmian	a231d0b65a	fix(BUG-2367): a move or copy that would reopen, close or un-abandon an item must name the status instead of inheriting whatever survived migration (#1516) ... Before this, every migrate door carried whichever done-field value survived migration, or injected the destination default. A done task moved into ideas silently reopened as `new`, and an abandoned item could land counted as shipped. Now every door (single move, bulk move, cross-workspace copy and its preflight) compares the item's CloseState (open / done / abandoned, derived from IsTerminalItem + IsAbandonedItem) before and after. A change is refused unless the caller names the destination done field. Only the state is compared, so two different values that both mean done carry without asking. A destination with no done select field is exempt. - move: 400 state_change_requires_value, details {field, options, from, to} - bulk move: the same refusal in failed[]; `move` + `status` counts as supplied - copy: 400 validation_error with the same sentence, after the unique check, in the preflight's order - preflight: a needs_value row with reason `state_change`, which the web dialog's existing picker renders - CLI: `hint: pad item move REF COLL --field status=<a|b|c>` ToolSurfaceVersion 0.52 (behaviour bump). Missing needs_value routes on the pane-menu move, CLI move and MCP are BUG-3200. Claude-Session: https://claude.ai/code/session_01RYiqbqXmxQMoKHsGm8B77N	2026-09-24 15:53:06 -04:00
xarmian	fd95b79694	fix(BUG-2367): moves and copies never carry into a computed field, and a carried unique-field collision is dropped and named instead of 500ing or storing a duplicate (#1514) ... MigrateFields matched fields by key and type only. On every migrate door (single move, bulk move, cross-workspace copy and its preflight): - a value was carried into a destination field marked `computed`. It is now dropped, with preflight reason `target_computed`. - a value colliding on a destination `unique_scope` field answered 500 on single move (invocation_slug, which has an index), leaked the SQL error through bulk move, and stored a duplicate on an unindexed unique field. Provenance decides now, as TASK-2878 did for relation referents: a CARRIED collision is dropped and reported as `not_unique`, and an override or schema default that collides is refused with 409 conflict. A required field emptied by the drop is refused as required. One store check (UniqueFieldConflictsQ / DropCarriedUniqueCollisionsQ) serves all four doors, including the tx-bound copy. The report names the value, and names the item holding it only when the caller may see that item. The activity row always uses the holder-free sentence. Reporting, all additive and omitempty: warnings.not_unique on the move response, the copy response and bulk updated[] rows; `detail` on the preflight's dropped row; CLI copy dry run and result, plus ` dropped:` lines on `item move` stdout; web copy dialog, move and copy toasts, and the timeline move card. ToolSurfaceVersion 0.51 (behaviour bump). The relation leg of the original report was already closed by TASK-2878. Item 4 (terminal/abandoned state change) is PR 2. Out of scope and filed separately: BUG-3200 (missing needs_value routes) and BUG-3202 (float64 precision; the migrateCopyFields comment now points there). Claude-Session: https://claude.ai/code/session_01RYiqbqXmxQMoKHsGm8B77N	2026-09-24 13:12:32 -04:00
xarmian	03ff8fc43a	fix(BUG-3186): the CLI turns an attachment name into a path Windows can store, on Windows only; stored names stay as given (#1498) ... A stored attachment name keeps < > : " | ? * (legal and common on Linux and macOS: "Meeting: notes.pdf"). On Windows those are refused, and a colon is worse: "a.svg:x.txt" names an NTFS alternate data stream, so the bytes land in a stream on an empty "a.svg". - attachments.WindowsPathComponent, beside WindowsDeviceName: reserved characters and C0 controls become "_", trailing dots and spaces are trimmed, and a device stem gets the "_" prefix. It is for the boundary where a name becomes a path, never for the stored name. - The CLI's safeLocalFilename applies it only when it RUNS on Windows (safeLocalFilenameFor(runtime.GOOS, ...), so it is testable on any host). Each saver applies its own filesystem's rules, as browsers do. The accepted edge is written down there: a Linux CLI writing onto an NTFS mount keeps the colon. Not changed, by lead ruling on the trail: the Content-Disposition name (it stays faithful; the server cannot know the client filesystem) and the bundle tar path. Import recomputes that path from the manifest and skips unmatched blobs silently, so changing it would make older servers drop bytes. A cross-version contract comment on bundleAttachmentPath says so. Claude-Session: https://claude.ai/code/session_01Vf8uZmYdsrQqJtbAmjUFSX	2026-09-24 05:25:53 -04:00
xarmian	c50f0585ad	fix(collab): WebSocket dials get their own per-user bucket and a per-user socket cap, and the editor jitters its reconnects (BUG-1308) (#1499) ... * fix(collab): WebSocket dials get their own per-user bucket and a per-user socket cap; the web provider jitters its reconnects (BUG-1308) A collab dial drew on the general per-user API bucket (burst 60), so 100 dials from one user got 60 through, and a user's own dials and REST calls refused each other. Dials now draw on a CollabDial bucket (5/s, burst 50), and how many sockets a principal may hold is bounded separately by PAD_COLLAB_MAX_PER_USER (default 50), refused with 429 collab_limit_exceeded and Retry-After. Both defaults are sized from the reconnect-storm measurement on BUG-1308's trail (checkpoint 2). A browser cannot read a refused handshake's status, so the provider's backoff is what spaces a refused herd out: each step is now jittered to between half and all of itself. Claude-Session: https://claude.ai/code/session_01RYiqbqXmxQMoKHsGm8B77N * docs(collab): PAD_COLLAB_MAX_PER_USER and the collab dial bucket, in the deployment docs, samples and load-test usage (BUG-1308) Claude-Session: https://claude.ai/code/session_01RYiqbqXmxQMoKHsGm8B77N * fix(collab): only a WebSocket upgrade is a dial; per-user cap pinned within one workspace; dial-bucket keying stated correctly (BUG-1308, codex round 1) Claude-Session: https://claude.ai/code/session_01RYiqbqXmxQMoKHsGm8B77N * docs(collab): a PAD_COLLAB_MAX_PER_USER below 1 means unlimited, as the SSE knobs do (BUG-1308, codex round 2) Claude-Session: https://claude.ai/code/session_01RYiqbqXmxQMoKHsGm8B77N * docs(collab): the struct field agrees, 0 or less is unlimited (BUG-1308, codex round 3) Claude-Session: https://claude.ai/code/session_01RYiqbqXmxQMoKHsGm8B77N	2026-09-24 04:51:57 -04:00
xarmian	80ec51c18c	fix(BUG-3185): an item export's file name whose stem is a Windows device gets a "_" prefix, on the server and in the CLI (#1495) ... `pad item export` saves to "<slug>.pad.md", so an item slugged nul, con, aux, prn, com1-9 or lpt1-9 named a Windows device, and the CLI on Windows wrote the artifact nowhere. - attachments.WindowsDeviceName is exported (a rename, no behaviour change): the one device rule from BUG-2822 (#1493), where the stem ends at the first "." or ":". Its doc says not to write a second one. - artifactExportFilename prefixes "_" to a device name, so the Content-Disposition both clients save under is safe. - The CLI applies the same rule to the path it will write (defaultExportPath): it is the process writing the file, an older server still suggests the device name, and the "<ref>.pad.md" fallback never passed through the server. Population (every Content-Disposition and client default name): the bundle export ("<slug>-export.tar.gz", whose stem is never bare), the account export (a fixed name) and the edit temp/recovery files (the "pad-edit-" prefix) are safe; attachments are #1493. Import has no mirror: nothing it creates is named from content. Claude-Session: https://claude.ai/code/session_01Vf8uZmYdsrQqJtbAmjUFSX	2026-09-24 03:44:11 -04:00
xarmian	1db30daa7f	fix(project): standup and changelog rows show the value of the item's own done field, not a blank read from a literal status (BUG-2640) (#1477) ... Both copies of the completed-work listing (the server's listTerminalItemsSince and the CLI's listCompletedWorkSince) already resolve each collection's done field per collection when querying. The rows rendering those items read the literal "status", so an item from a collection whose done field is something else (board_group_by: stage) was included with a blank Status. The resolved field now travels beside the items as a collection-id map, and the four row builders (server and CLI, standup and changelog) read it through completedWorkValue, falling back to status. The JSON key stays "status"; only the value it reads moves. Dashboard and item-list views that read the literal status to decide active, in-progress and stalled ask a different question, and are left alone. Claude-Session: https://claude.ai/code/session_01RYiqbqXmxQMoKHsGm8B77N	2026-09-23 19:37:58 -04:00
xarmian	9b8f72d202	fix(insights): a default report leaves out system collections (Conventions, Playbooks), so they stop counting in Insights (BUG-2410, ToolSurfaceVersion 0.49) (#1471) ... * fix(BUG-2410): a default report leaves out system collections; Insights' picker stops offering them; ToolSurfaceVersion 0.49 * fix(BUG-2410): a saved layout naming a system collection no longer filters the report by a selection the picker cannot show (codex r1) * fix(BUG-2410): with no collection list the report goes unfiltered, and a failed load never erases the saved layout (codex r2) * fix(BUG-2410): a collections response for a workspace the user left is ignored (codex r3)	2026-09-23 18:21:38 -04:00
xarmian	a3ebf69932	fix(project): a guest's item grants count as completed work only under their own collection's terminal values, and each item is listed once (BUG-2639) (#1474) ... listTerminalItemsSince passed the caller's full item-grant list into every per-(field, value) completed-work query. ListItems ORs ItemIDs past the CollectionIDs restriction, which is right for visibility, but here CollectionIDs also means 'the collections for which this value is completed work'. So a granted item was counted under another collection's terminal value, and one matching two groups' keys was listed twice. Each granted item's collection is now resolved once, and each group receives only the granted items that live in one of its collections (the group's full list, so item-grant-only collections still contribute). Both copies (the server's and the CLI's listCompletedWorkSince) now also dedup by id. That is not only defensive: the list endpoint splits a filter value containing a comma into an IN list, so a collection whose completed-work values include both 'x' and 'x,y' matched one item twice, for members too. Claude-Session: https://claude.ai/code/session_01RYiqbqXmxQMoKHsGm8B77N	2026-09-23 17:34:27 -04:00
xarmian	6cc2c7eb95	fix(items): create refuses reserved metadata keys; convention metadata goes through a typed create member; a full fields update may only carry them (BUG-3163, ToolSurfaceVersion 0.48) (#1470) ... Item CREATE was the last door that stored a hand-written reserved metadata key (implementation_notes, decision_log, github_pr, convention). A `--field` or MCP `field` value landed as a string no extractor reads. For notes and decisions, that also made `note` / `decide` refuse on the item until the row was repaired. Create's `fields` now refuses all four keys with 400 validation_error. The check runs on the caller's map. Library activation, the only system writer that used this door, moves to a new typed `convention` create member, which is validated (an empty object is refused), normalized and lowered into the blob. Every convention writer switches to it: the CLI and remote MCP library activate, the web library activate, the web Conventions page create, and the e2e seed helper. The CLI checks the STORED `convention` key to confirm the member landed. It cannot use item.Convention for that, because that value is derived from the sibling keys and is non-nil even when an older server has dropped the member. A FULL `fields` update may CARRY stored reserved values unchanged. A differing value, or an OMITTED stored key (which the whole-blob replace would delete), is refused with 400. The comparison uses decoded JSON, never bytes, and runs in the update transaction against the row re-read under the write lock (composed into the precheck, like the BUG-3133 guard). A note appended concurrently therefore cannot be erased by a stale blob. A stored blob that is not a JSON object has no readable reserved metadata, so a full write can still repair it. Artifact import shares createItemChecked but needs no gate, because artifact.Decode fills Fields from a fixed frontmatter key list. Existing test fixtures that put reserved keys in through the two closed doors now seed at the store level. The stdio refusal texts are pinned to the classifier's validation_failed table. Claude-Session: https://claude.ai/code/session_01RYiqbqXmxQMoKHsGm8B77N	2026-09-23 16:23:06 -04:00
xarmian	bee399bc94	fix(items): github_pr is written only through a typed, validated update member; the field-setter door refuses it (BUG-2696, ToolSurfaceVersion 0.47) (#1469) ... A `--field github_pr=...` / MCP `field` setter stored the PR as a string on every transport, so no link rendered, and `=null` stored the string "null". BUG-2627 had exempted github_pr from the reserved-key refusal as remote MCP's only writer; that writer never worked. ItemUpdate gains a typed `github_pr` PR object (number > 0 and an absolute http(s) url required) plus `clear_github_pr`, lowered by the handler into the field patch after the caller's own fields_patch is refused every reserved key. `pad github link` / `unlink` / `project reconcile` use it and verify from the response that the change landed (an older server ignores the member and answers 200). Not exposed on the MCP catalog; a remote PR-link action is deferred to the catalog-trim decision. The point is validity, not sender identity. Create (still accepts hand-written reserved keys) is BUG-3163. Claude-Session: https://claude.ai/code/session_01RYiqbqXmxQMoKHsGm8B77N	2026-09-23 14:35:41 -04:00
xarmian	c95168ff0d	fix(mcp): bulk-update refuses an undeclared status/priority per item on stdio and remote too (BUG-3156, ToolSurfaceVersion 0.46) (#1463) ... * fix(mcp): bulk-update refuses an undeclared status/priority per item on stdio and remote too (BUG-3156, ToolSurfaceVersion 0.46) Over WebMCP, pad_item bulk-update has refused (since 0.44) a status or priority change on an item whose collection declares no such field; stdio and remote went through the single-item PATCH, which accepted it with a warning and stored an orphan field. The item update door gains a general, additive strict opt-in, `refuse_undeclared_fields`: with it, the write is refused with 400 validation_error for exactly the keys warnings.undeclared_fields would have named, and nothing is written. Both bulk-update transports set it. A write without it is unchanged; a server that predates it ignores it, which is the old accept-and-warn. Claude-Session: https://claude.ai/code/session_01RYiqbqXmxQMoKHsGm8B77N * test(server): a refused write also leaves the title and body alone (BUG-3156, codex round 1 on #1463) Claude-Session: https://claude.ai/code/session_01RYiqbqXmxQMoKHsGm8B77N	2026-09-23 10:48:28 -04:00
xarmian	919c2975c2	fix(items): note and decide append server-side under the write lock, so they no longer revert a concurrent field write (BUG-3056) (#1458) ... * wip(BUG-3056): server-side note/decide append, capability-gated CLI * test(BUG-3056): server append tests * test(BUG-3056): CLI capability gate tests * test(BUG-3056): store append tests on both backends; CLAUDE.md API note * fix(BUG-3056): keep the remote /mcp door's created_by label (codex r1) * docs(BUG-3056): CLAUDE.md names the remote door's author label	2026-09-23 07:46:45 -04:00
xarmian	033615ac94	fix(metrics): bound the MCP tool label and the HTTP method label, which callers could fill with invented values (BUG-2817) (#1456) ... * fix(metrics): bound the MCP tool label and the HTTP method label, which callers could fill with invented values (BUG-2817) Two metric labels took caller text verbatim, and each new value minted a new Prometheus series. - pad_mcp_tool_calls_total and pad_mcp_tool_call_duration_seconds (tool): the value is params.name or the JSON-RPC method, recorded even for rejected and denied calls. It is now bounded by mcp.Server.IsKnownCallName, which asks the LIVE server for registered tools and adds the protocol methods. Anything else is recorded as "unknown". The predicate is injected (SetMCPMetricsCallNames) because internal/mcp imports internal/server. Unwired, every name is "unknown". The audit row keeps the full value. - pad_http_requests_total, _request_duration_seconds and _response_size_bytes (method): raw r.Method, recorded by middleware that runs AHEAD of auth, and Go accepts any token as a method, so an unauthenticated caller could mint series. Measured: 50 invented methods gave 50 series. A method outside the standard set is now "OTHER". The checkpoint-1 population on BUG-2817's trail enumerates all 14 labelled metrics and their 22 label slots; the other slots take constants or bounded values. Claude-Session: https://claude.ai/code/session_01Vf8uZmYdsrQqJtbAmjUFSX * fix(mcp): every mcp-go method constant is a known metrics call name (BUG-2817, codex round 2) The protocol set was a hand-picked subset, so a legitimate method such as notifications/tools/list_changed would have been counted as "unknown". It is now every method constant mcp-go v1.1.0 defines (34). Each is a library constant, so the label stays bounded. Claude-Session: https://claude.ai/code/session_01Vf8uZmYdsrQqJtbAmjUFSX * test(mcp): pin all 34 protocol methods as literals, so a dropped entry fails (BUG-2817, codex round 3) Claude-Session: https://claude.ai/code/session_01Vf8uZmYdsrQqJtbAmjUFSX * fix(server): the MCP metrics bound is a required SetMCPTransport argument, and the method test covers all nine standard methods (BUG-2817, codex round 4) A separate setter could be left uncalled with every test still green, so the predicate is now a parameter of the call that mounts the transport. nil fails closed. The HTTP test now keeps each of the nine standard methods as its own value, so mapping any one of them to OTHER fails it. Claude-Session: https://claude.ai/code/session_01Vf8uZmYdsrQqJtbAmjUFSX * test(server): the bounded-label test passes its predicate through SetMCPTransport, so a broken assignment there fails it (BUG-2817, codex round 5) Claude-Session: https://claude.ai/code/session_01Vf8uZmYdsrQqJtbAmjUFSX	2026-09-23 06:01:08 -04:00
xarmian	969955c810	fix(mcp): a 413 is too_large on both transports, not server_error (BUG-2829) (#1455) ... * fix(mcp): a 413 is too_large on both transports, not server_error (BUG-2829) Claude-Session: https://claude.ai/code/session_01RYiqbqXmxQMoKHsGm8B77N * docs(mcp): tool surface 0.45 in instructions, README and CLAUDE.md (BUG-2829) Claude-Session: https://claude.ai/code/session_01RYiqbqXmxQMoKHsGm8B77N	2026-09-23 05:17:16 -04:00
xarmian	a92af3eebc	fix(attachments): drop Bidi_Control characters from attachment names and rewrite legacy rows (BUG-3153) (#1452) ... * fix(attachments): drop Bidi_Control characters from attachment names and rewrite legacy rows (BUG-3153) A name such as "x<U+202E>gvs.txt" is a .txt that every bidi-aware renderer shows as "xtxt.svg". About forty web read sites, the CLI list and the server render the stored name, so the fix is at the source rather than per renderer. DroppedFilenameRune now also drops the Unicode Bidi_Control property: U+202A to U+202E, U+2066 to U+2069, and the marks U+200E, U+200F and U+061C. It is chosen by property, so ZWJ (Join_Control) and emoji sequences are untouched. The rule is shared, so both ingest doors store the name clean and the download header drops the characters for any row. One of these characters inside an extension also hid that extension from the blocklist, as a control byte did: an SVG uploaded as "x.s<U+202E>vg" was accepted, and is now refused as ".svg". Legacy rows are rewritten once, at startup, by BackfillBidiAttachmentFilenames, and only rows containing a Bidi_Control character are touched. The new name is the SERVED name (ServedFilename), not a plain strip, so a legacy "x.s<U+202E>vg" becomes "x.bin" rather than storing the blocked "x.svg". The pass is idempotent with no marker, since a rewritten row no longer matches. Each UPDATE is conditional on the old name. Every rewrite is logged with its old and new names quoted to ASCII, so the rewrite is reversible, and the count is logged on every boot, 0 included. Claude-Session: https://claude.ai/code/session_01Vf8uZmYdsrQqJtbAmjUFSX * fix(store): the bidi filename backfill states its wider rewrite and is tested per rune on both dialects (BUG-3153, codex round 1) - The rewrite is the served name, so a selected row also loses whatever else NormalizeFilename removes from a legacy name. That is now said in the comment and pinned by a case, and rows without a Bidi_Control are still never touched. - "Reversible by the log" now says: for as long as the log is retained, with no durable record in the database. - A row that gains such a name later (derived from a parent a failed pass left) is picked up by the next boot, since selection is by content. - New test: one row per member of unicode.Bidi_Control, each selected by the SQL and rewritten. It enumerates the property itself rather than the list under test, so a rune missing from the LIKE list fails it. Claude-Session: https://claude.ai/code/session_01Vf8uZmYdsrQqJtbAmjUFSX * fix(store): a rows-affected error stops the bidi backfill instead of passing as a concurrent rename (BUG-3153, codex round 2) The UPDATE may have landed, and without a count the row could not be logged as rewritten, which the reversibility rests on. The comment also names the readers it means, instead of an unmeasured count. Claude-Session: https://claude.ai/code/session_01Vf8uZmYdsrQqJtbAmjUFSX * docs(attachments): NormalizeFilename covers the caller-supplied doors, and derived rows inherit (BUG-3153, codex round 3) "Every door that writes attachments.filename calls it" was not true of thumbnails, transforms or copies, which derive their name from another row. Claude-Session: https://claude.ai/code/session_01Vf8uZmYdsrQqJtbAmjUFSX * test(e2e): the hostile-name parity spec expects the stored name, with Bidi_Control dropped (BUG-3153) The spec uploads "start <U+202E>reversed<U+202C> مرحبا.png" and asserted that the tile and dialog carry it exactly as uploaded. Ingest now drops the override. The spec still uploads the hostile name, and now asserts that the STORED name (RTL letters kept, controls gone) is addressable and names its dialog. Its fixture now uses \u escapes instead of literal controls. CI caught this on #1452; the local sweep had not searched web/e2e. Against this tree's binary the spec passes, and against a binary without the fix it fails, so the spec discriminates. Claude-Session: https://claude.ai/code/session_01Vf8uZmYdsrQqJtbAmjUFSX	2026-09-23 04:35:58 -04:00
xarmian	88a5af1a74	fix(mcp): stdio passes "-"-leading free text as text, and reports argv refusals and 429s with the right code (BUG-3142, BUG-3147) (#1444) ... * fix(mcp): stdio sends positionals behind "--", so free text starting with "-" is no longer parsed as a flag (BUG-3142) BuildCLIArgs emitted positionals first with no terminator, so cobra read `item search "-ship it"` as an unknown shorthand flag and refused the call before the command ran, on all 12 free-text positionals the stdio transport carries. It now emits every flag (root flags, --workspace, --format included) and then `--` and the positionals; the remote door ignores this vector. Pinned against the real cobra tree through each command's own flag parser, including a positional that is literally "--workspace". Claude-Session: https://claude.ai/code/session_01RYiqbqXmxQMoKHsGm8B77N * fix(mcp): stdio reports an argv refusal as validation_failed, with its message, instead of a retryable server_error (BUG-3142) Two defects on one path. The root command's flag-error hook prints cobra's usage block BEFORE the "Error:" line, and stripCobraUsageBlock truncated at "Usage:", so a flag-parse refusal reached the classifier as an empty string. And no stderr pattern matched cobra's or pflag's argv refusals at all. The strip now stops at a following "Error:" line, and one anchored pattern covers the parse-time family enumerated from cobra v1.10.2 and pflag v1.0.10. Pinned end to end: the test binary re-executes as the real CLI under ExecDispatcher, so the real stderr ordering is what gets classified. Claude-Session: https://claude.ai/code/session_01RYiqbqXmxQMoKHsGm8B77N * fix(mcp): a 429 reaches a stdio caller as rate_limited with the server's wait, and the tool surface is 0.42 (BUG-3147, BUG-3142) The CLI printed only the limiter's message, which no stderr pattern matched, so every 429 reached a stdio MCP caller as a retryable server_error while the remote transport said rate_limited. The CLI now writes the structured marker line at its root for a 429 on any command, carrying Retry-After as details.retry_after_seconds, and stdio lifts it. ToolSurfaceVersion 0.42 covers this and BUG-3142's classification change in one entry: both change the code a stdio caller receives, and nothing else. Claude-Session: https://claude.ai/code/session_01RYiqbqXmxQMoKHsGm8B77N * test(cmd/pad): the stdio argv test restores the root flag globals it parses into (BUG-3142) Claude-Session: https://claude.ai/code/session_01RYiqbqXmxQMoKHsGm8B77N	2026-09-22 22:49:36 -04:00
xarmian	6fdd8816f3	feat(playbook): match free text to an active playbook via the decision provider (TASK-3120) (#1429) ... * feat(server): POST /playbooks/match — typed-decision Choice over active playbooks (TASK-3120) PLAN-3114 unit 5. A synchronous, read-only endpoint: given free text, asks the workspace's typed-decision provider which active (non-draft, non-deprecated) playbook it best matches, with a reserved "none" option so unrelated text gets an honest answer instead of a forced pick — and, as a side effect, keeps a workspace with exactly one active playbook askable (decision.Question.Validate needs >=2 Choice options). - decision.Runner gains a nil-safe Provider() accessor: this endpoint needs a plain synchronous Ask, not the owed-jobs pipeline Evaluate drives. - No provider configured -> 404 decision_provider_unavailable (callers fall back to slug/trigger routing). A provider error or an out-of-set answer the provider was never offered -> 502 decision_provider_error, distinct from the 404 case so a caller can tell "try again" from "not configured". Zero active playbooks -> 200 with a stated reason, no provider call. Over the provider's 255-option ceiling (including "none") -> refused rather than silently dropping playbooks and answering from an incomplete set. - listPlaybookItems is factored out of handleListPlaybooks so list and match share one visibility-scoped item source and can't drift on which playbooks a caller can see. - 30s timeout on the synchronous provider call, tighter than the provider's own worst-case retry budget (~330s) since an HTTP caller blocks on this one directly, unlike the async decision tick. - content_state_population_test.go's BUG-3033 sweep gets a new ruling: the PlaybookSummary call here feeds the provider prompt only, never the response, so there's no reader-facing text to mark. Claude-Session: https://claude.ai/code/session_01ByMjAhvdWgbAWWeGE9jwWd * feat(cli): pad playbook match "<text>" (TASK-3120) CLI + client-method half of PLAN-3114 unit 5 — POST /playbooks/match. Table output prints the matched ref/title (or "none"), confidence, model, and a probabilities table sorted descending; --format json passes the response through unmodified. Read-only, side-effect-free. Claude-Session: https://claude.ai/code/session_01ByMjAhvdWgbAWWeGE9jwWd * docs(skill): route to playbook match as rung 3, offer-only (TASK-3120) PLAN-3114 unit 5's skill half. Exact invocation-slug match and trigger match stay rungs 1-2; match is consulted only when the user is plainly asking for a procedure/workflow and both of those missed — never on ordinary conversation, since every call spends provider money. Any match returned is offered, never auto-run, at any confidence: TASK-3137's replay of this same provider's attention question set found no validated threshold, so nothing here invents one for this question either. A 404 (no provider configured), a "none" choice, or any other error falls back silently to today's slug/trigger-only behavior. Claude-Session: https://claude.ai/code/session_01ByMjAhvdWgbAWWeGE9jwWd * chore(mcp): de-indent padItemToolDescription's continuation lines (TASK-3120) Whitespace-only — git diff -w on this file is empty. The description's continuation lines carried 18 leading spaces purely for visual alignment under each action's " name — " bullet in the Go source; that padding is emitted verbatim into the tools/list JSON, where it conveys nothing an LLM reading the tool description gets any value from. De-indented by exactly 14 spaces on every line that had >= 18 (all 116 of them, uniformly, since the file carries only three indentation levels: 0, 2, 18 — no deeper nesting to preserve a relative offset for). Continuations now land at 4 (bullet's 2 + 2), so a reader can still tell which action a line belongs to; a 1-space collapse would have inverted that (continuation LESS indented than its own bullet). tools/list: 55290 -> 53666 bytes (-1624). initialize: unchanged at 24530 (it carries no tool descriptions). Frees headroom against the 55296 budget for the pad_playbook.match action landing in the next commit — budget constants themselves are untouched, per standing instruction. Claude-Session: https://claude.ai/code/session_01ByMjAhvdWgbAWWeGE9jwWd * feat(mcp): pad_playbook.match action, ToolSurfaceVersion 0.39 -> 0.41 (TASK-3120) PLAN-3114 unit 5's MCP half, landing on top of the whitespace-only catalog_item.go trim (previous commit) so this stays within the tools/list budget with room to spare. - catalog_playbook.go: new `text` param, new `match` action (passThrough — the CLI's positional arg is named "text", matching this tool's param, so no custom shaping is needed) plus a description bullet in padPlaybookToolDescription. - tool_surface.go: `match` marked read-only (nothing stored, no job enqueued). - dispatch_http_routes.go: `playbook match` routeSpec (POST, flat {text} body) so the remote /mcp transport can dispatch it in-process. - version.go: ToolSurfaceVersion 0.39 -> 0.41. 0.40 is reserved for PR #1427 (the 0.32/0.33 precedent — a gap is cheaper than a collision). 0.41's own entry: pure ADDITIVE bump, same disposition as 0.13/0.28/0.11 — nothing existing moved. - instructions.md, README.md, CLAUDE.md: version headings + the pad_playbook bullet/table-row/changelog-sentence updated per the drift tests (TestInstructionsMDCoversEveryCatalogAction, TestReadmeTableCoversEveryCatalogAction, the two version-match tests). instructions.md's addition is intentionally terse ("`match` finds a playbook from text.") — it's embedded in the `initialize` handshake payload, which has its own 24576-byte budget and had only 46 bytes of headroom; a fuller explanation lives in the tools/list param/action docs instead, which have budget to spare after the trim. - catalog_readonly_test.go, dispatch_http_parity_test.go: new `pad_playbook.match` entries in three hand-built fixture maps (expected cmdPath ×2, a hand-rolled cmdhelp stub, and the shared parity-drive fixture input) so the existing bijection/parity/dispatch guards actually exercise the new action instead of skipping it. Bytes measured (go test ./internal/mcp -run Budget -v): tools/list: 55290 (baseline) -> 53666 (after the trim commit) -> 54805 (after this commit) — 491 bytes of headroom left against the 55296 budget. initialize: 24530 (baseline, unchanged by the trim) -> 24574 (after this commit) — 2 bytes of headroom against the 24576 budget. Flagged in the implementer report: this is now as tight as tools/list was before the trim, and nothing in this unit addresses it. Claude-Session: https://claude.ai/code/session_01ByMjAhvdWgbAWWeGE9jwWd * fix(server,decision): don't leak provider errors, classify oversized requests, fix none's wording (TASK-3120) Lead's shallow-pass findings on cd7a41ed, applied after the MCP commit. F1 (disclosure): the provider-error path answered with err.Error() verbatim. typesafe.go's client surfaces the provider's raw 4xx body in that string (APIError.Error()), which is the INSTANCE's own account state with typesafe.ai — key validity, quota, billing wording — not anything about the caller's request. This endpoint requires only viewer access, so any workspace member could read it. Now logged server-side in full (slog, workspace + err) and answered with a fixed message plus a coarse details.reason ("timeout" for a context deadline, "upstream" otherwise) via a new writeDecisionProviderError helper. Test constructs a fake error containing a secret-looking string and asserts it never reaches the response body. F2 (misclassification): the request budget is 32000 tokens (~80000 chars). At a couple hundred active playbooks with realistic summaries, the Choice question's options alone can exceed it before any state is even added — or leave no room for the input text. Both were plain fmt.Errorf with no sentinel, so they fell into the generic "provider failed" 502 path, telling a caller "try again" for a request that will never fit no matter how many times it's retried. decision/typesafe.go gains ErrRequestTooLarge (wrapped with %w onto both pre-send refusals; existing message text unchanged, just appended-to) alongside the existing ErrMaxTokensExceeded (the provider's own after-the-fact refusal of a request fitState let through) — both now map to a distinct 400 match_request_too_large. New decision-package unit test builds a realistic 254-option Choice (title + near-cap summary + trigger + slug per option, mirroring what playbook-match could plausibly send) and confirms fitState refuses it before the fake HTTP handler is ever called; server-side table test covers both sentinels through the fake provider. F3 (nit): the "none" option's description read "...listed above", but Options is an unordered map — nothing puts them "above" it. Reworded to "...any of the other options' procedures"; the pinned exact-prompt test updated to match. Also (lead's separate MCP-round instruction): decision.go's maxChoiceOptions is now exported as MaxChoiceOptions, and the server's over-limit refusal reads that constant instead of a copied `255` literal, so the two ceilings cannot drift apart. go test ./internal/decision/... ./internal/mcp/... ./internal/server/... ./cmd/pad/... ./internal/cli/... — all green. golangci-lint run ./... — 0 issues. Full `go test ./...` run separately; final line in the report. Claude-Session: https://claude.ai/code/session_01ByMjAhvdWgbAWWeGE9jwWd * chore(mcp): cut pad_playbook.match's tools/list text to the approved +292 variant (TASK-3120) The lead's shape-(a) approval was for the measured "+292 documented" prototype, not the fuller wording ae38353c actually shipped (+1139 bytes) — the trim commit exists to pay for the approved action text, not to fund arbitrary new prose. Cut the `text` param to one sentence (the 404-fallback advice lives once, in the action bullet, not repeated in the param) and the `match` action bullet down to four lines at consistent bullet+2 continuation indentation, matching the rest of padPlaybookToolDescription's style (the previous wording had drifted to 20-space continuation on its last two lines). tools/list: 53666 (post-trim baseline) -> 54023 (+357, close to the ~300 target — wording differs from the original prototype's exact text, so an exact match isn't expected). initialize: unchanged at 24574 (this file's bullet is tools/list-only; instructions.md is a separate file, untouched here). Claude-Session: https://claude.ai/code/session_01ByMjAhvdWgbAWWeGE9jwWd * fix(server): don't overclaim writeDecisionProviderError's timeout reason (TASK-3120) The doc comment said "timeout" covered a cancelled caller request too, but context.Canceled is a distinct error from context.DeadlineExceeded and errors.Is(err, context.DeadlineExceeded) does not match it — that case actually falls through to "upstream", contradicting what the comment claimed. Narrowed the comment to deadline-only (the preferred fix per review): a cancelled caller's own request also never reads this response body, so the distinction was moot as well as wrong. Code was already correct; only the comment overclaimed. Claude-Session: https://claude.ai/code/session_01ByMjAhvdWgbAWWeGE9jwWd * fix(cli,skill): pad playbook match needs -- before dash-led text (TASK-3120) Codex round 2 finding on f31d5d64. Text starting with "-" (a dash-led sentence, or literal text like "-ship it") is parsed as a flag by cobra's default scanner, on both the CLI and stdio MCP (which shells out to this same command). Pre-existing and class-wide — `pad item search "-x"` fails the same way — so the general stdio fix across every command is filed as BUG-3142 rather than done here; BuildCLIArgs is untouched. This command's own help now shows the workaround: an Example section with `pad playbook match -- "-ship it"`. skills/pad/SKILL.md's rung-3 routing entry calls it the same way, with a one-line reason, and notes the MCP form is unaffected (a structured `text` param, not a CLI token). New CLI test asserts BOTH directions: `playbook match "-ship it"` (no --) fails at cobra's flag parser and never reaches the server; `playbook match -- "-ship it"` succeeds and the server receives the text with its leading dash intact. Ran both legs — see the test. Claude-Session: https://claude.ai/code/session_01ByMjAhvdWgbAWWeGE9jwWd * fix(skill): local stdio MCP is not exempt from the leading-dash flag parse (TASK-3120) Codex round 4 finding. The rung-3 routing entry claimed the MCP form of match was "unaffected" by BUG-3142, but that's only true for remote /mcp, which keeps `text` a structured param. Local stdio MCP shells out to `pad playbook match`, turning `text` into a CLI positional via BuildCLIArgs (dispatch.go:435) — the same flag parser, the same bug. Corrected: remote /mcp is safe as-is; local stdio MCP still hits it and should prefer the CLI's `--` form when the text starts with `-`. Claude-Session: https://claude.ai/code/session_01ByMjAhvdWgbAWWeGE9jwWd * fix(cmdhelp): the example validator's flag scan must stop at "--" (TASK-3120) CI red on #1429 (a2a5a853), Go/Go-PG/Nix all failing the same way: TestRealPadTree_ExampleDriftValidator flagged the new `pad playbook match -- "-ship it"` example's own dash-led TEXT ARGUMENT as an unknown flag `--ship it`. Cobra stops flag parsing at a lone "--" — that's the whole point of the token — but validateExample's scan treated "--" as just another flag-shaped token: TrimLeft reduced it to an empty name, which fell into the "skip, keep scanning" branch instead of stopping the scan. Everything after "--" kept getting flag-checked, so the terminated example's own positional text was misread as a flag. Fixed by checking `t == "--"` BEFORE the dash-stripping and breaking the scan there, per review. A lone "-" (the stdin convention, e.g. `pad item import -`) is a different token — it still reaches the dash-stripping, still reduces to an empty name, and still just `continue`s past — so it does not terminate the scan. Three new unit tests in internal/cmdhelp cover all three legs: (a) a dash-led positional after "--" produces no finding, (b) an unknown flag BEFORE "--" is still caught — the break cannot hide drift ahead of the terminator, and (c) a lone "-" does not stop the scan (an unknown flag after it is still caught). Manually re-ran (a) with the break removed to confirm it goes red without the fix, then restored the real fix — ran, not just described: FAIL "a dash-led positional after -- should not be reported as an unknown flag" without the break, PASS with it. The playbook match example itself is unchanged. Claude-Session: https://claude.ai/code/session_01ByMjAhvdWgbAWWeGE9jwWd	2026-09-22 11:28:57 -04:00
xarmian	442dcbdc6a	feat(decision): instance-admin settings for the decision provider (TASK-3121) (#1426) ... console/admin/settings gains a Decision provider section with a provider select, an enable toggle, a write-only API key stored encrypted, a model pin, and the privacy line. It is backed by GET/PUT /api/v1/admin/decision-provider. - The admin setting is a third source to decision.ResolveNamed: config file < admin setting < environment, per field. Resolve is now implemented on ResolveNamed, so the page's "set by environment" comes from the same pass that builds the provider. - Changing the setting rebuilds the provider and swaps the runner in place. The tick and the write doors' enqueue resolver both follow the swap, so no restart is needed. A swap cancels any pass in flight, so a disabled provider is not called again; the resolver is installed before the runner is published; and no tick can start after Stop(). - The key is encrypted through the PAD_ENCRYPTION_KEY path (store.SetSecretPlatformSetting). It is never returned by any read, masked or otherwise, and it is not writable through /admin/settings. The store refuses a secret write without an encryption key (`pad server` never reaches that state), and it refuses to READ a secret row that is not ciphertext. - A PUT to a field the environment sets is refused with 409 set_by_environment. On Pad Cloud the environment is the only source and PUT answers 403 managed_by_operator. - `pad server info` still resolves only the file and the environment on the CLI host. It now says so, in an additive `sources` JSON member and in its human line. Claude-Session: https://claude.ai/code/session_01ByMjAhvdWgbAWWeGE9jwWd	2026-09-22 07:34:33 -04:00
xarmian	5fe478b7d0	fix: refuse a token-guarded content write over unflushed tab edits (BUG-3133) (#1423) ... expected_seq guards the row, and a browser tab's typing is not in the row until the tab flushes. So a content PATCH carrying a current token replaced those edits: through the applier's setContent with a tab open, and with no tab open the direct path's PruneItemOpLogTx deleted them, and they existed nowhere else. A content write that carries expected_seq or expected_updated_at is now refused with 409 content_pending_flush while content-bearing op-log rows sit above the flush watermark. The check is composed into the existing precheck, so the direct, applier and collab-disabled paths all inherit it, and runs in the write's own tx after the token check (a stale token still answers update_conflict). It has its own code because re-reading returns the same row and seq. overwrite_pending_edits lifts it on the API, the CLI (--overwrite-pending-edits; item edit --force sets it on the save) and MCP on both transports, with a shared hint that does not say re-read and retry. A tokenless write is unchanged, except that a direct write now reports warnings.pruned_pending_edits when its prune deleted such rows (the CLI prints it to stderr). MCP ToolSurfaceVersion 0.39: a behavior bump with one additive param. Claude-Session: https://claude.ai/code/session_01X4uYKs5ZPF28eCWfm8VPvE	2026-09-22 00:42:02 -04:00
xarmian	1681be74f5	feat(decision): content-aware attention on the dashboard and item page (TASK-3118) (#1421) ... * feat(decision): attention question set on the dashboard (TASK-3118) Registers the `attention` set (needs_human_decision and blocked verbatim from the day-73 eval, plus waiting_on_external) through a production registry. A new QuestionSet.Eligible gate, checked by the runner at evaluation time, keeps terminal items and system collections from being sent. The dashboard reads the latest answers to the current questions in one batch query: needs_human >= 0.7 becomes a "needs_human" attention entry, and blocked >= 0.7 joins the blocked section only where the dependency graph is silent. A failing provider marks attention.decisions degraded. Claude-Session: https://claude.ai/code/session_01ByMjAhvdWgbAWWeGE9jwWd * feat(web): attention chips on the item page, dashboard icons (TASK-3118) DecisionChips fetches the item's decisions itself (the item object is also fed by projections that do not carry them) and shows only CURRENT attention answers, emphasised at the dashboard's 0.7 threshold. It renders nothing without a provider. A request token keeps a switched-away item's late response from painting. It is listed in the BUG-3095 child population, and stubbed in the ItemDetail identity harness like its self-fetching siblings. Claude-Session: https://claude.ai/code/session_01ByMjAhvdWgbAWWeGE9jwWd * test(server): threshold edge and below-threshold blocked legs (TASK-3118) Claude-Session: https://claude.ai/code/session_01ByMjAhvdWgbAWWeGE9jwWd * docs: attention set in CLAUDE.md; registry comment no longer says empty (TASK-3118) Claude-Session: https://claude.ai/code/session_01ByMjAhvdWgbAWWeGE9jwWd * fix(decision): dashboard attention applies the item page's currency rule (TASK-3118) Codex round 1: the dashboard read the latest answer without checking it against the item's present state, so an item edited or commented since its last evaluation kept a stale "needs a human" entry, while the item page, which hides non-current answers, showed nothing. WorkspaceFlags now returns only CURRENT answers at or above the threshold. It does one state read per flagged item, so the cost is bounded by the flagged items rather than the workspace. Also adds a leg for an item made terminal by a collection schema edit, which changes no item state; mutant M11 survived without that leg. Claude-Session: https://claude.ai/code/session_01ByMjAhvdWgbAWWeGE9jwWd * fix(decision): an answer is current only while its set still applies (TASK-3118) Codex round 2: a collection schema edit can make an item terminal without changing its state hash. /decisions then kept calling its attention answers current, and the item page kept showing chips the dashboard had dropped. Evaluate, Decisions and WorkspaceFlags now share one appliesNow rule, and the dashboard's separate done-skip is gone. Declined from the same round: a TOCTOU between the dashboard's state read and its response. An edit landing after the read is reflected on the next load, which holds for every read in the handler. Claude-Session: https://claude.ai/code/session_01ByMjAhvdWgbAWWeGE9jwWd * fix(store): renumber the decisions workspace index to 092/069 after #1419 took 091/068 (TASK-3118) Claude-Session: https://claude.ai/code/session_01ByMjAhvdWgbAWWeGE9jwWd	2026-09-21 23:55:54 -04:00
xarmian	3e099423f8	fix(store): content_state counts only op-log rows that can change the document (BUG-3124 unit A) (#1419) ... * fix(store): content_state counts only op-log rows that can change the document (BUG-3124 unit A) Claude-Session: https://claude.ai/code/session_01X4uYKs5ZPF28eCWfm8VPvE * test(store): BUG-3124 — the backfill's twin must be an EARLIER row Claude-Session: https://claude.ai/code/session_01X4uYKs5ZPF28eCWfm8VPvE * docs: BUG-3124 prose sweep — content_state counts content-bearing rows only Claude-Session: https://claude.ai/code/session_01X4uYKs5ZPF28eCWfm8VPvE * fix(store): BUG-3124 — renumber migrations to 091/068 after #1417 took 090/067 Claude-Session: https://claude.ai/code/session_01X4uYKs5ZPF28eCWfm8VPvE	2026-09-21 22:02:48 -04:00
xarmian	1144fe8060	fix(cli): item edit refuses a stale body, guards its save with expected_seq, and keeps a recovery file on failure (BUG-3035) (#1418) ... * fix(cli): item edit refuses a stale seed, guards the save with expected_seq, and never loses the edit (BUG-3035) Claude-Session: https://claude.ai/code/session_01X4uYKs5ZPF28eCWfm8VPvE * docs(cli): BUG-3035 prose sweep — edit refuses by default, the warning is the forced path's Claude-Session: https://claude.ai/code/session_01X4uYKs5ZPF28eCWfm8VPvE * fix(cli): BUG-3035 codex round 1 — help states the unflushed-typing gap (BUG-3133); a nonzero editor exit aborts and says so Claude-Session: https://claude.ai/code/session_01X4uYKs5ZPF28eCWfm8VPvE	2026-09-21 21:03:05 -04:00
xarmian	304673faa1	feat(decision): typed-decision storage and async runner (TASK-3117) (#1417) ... * feat(store): item_decisions + decision_jobs schema (TASK-3117) Answers keyed on (item_id, question_set, question_key, state_hash) with item_seq kept for the audit; a private per-(item, question_set) job queue with generation-based coalescing and lease reclaim, in place of the public event outbox. NUL census: three columns excluded with reasons, fifteen id/timestamp/enum columns baselined. Claude-Session: https://claude.ai/code/session_01ByMjAhvdWgbAWWeGE9jwWd * feat(decision): job queue store layer + Runner with state-hash idempotency (TASK-3117) Item create, item update and comment create enqueue a coalesced job per applicable question set in their own transaction; nothing waits on the network. The Runner builds one serialised state (title, collection, fields, bounded body, last RecentTrailWindow comments), hashes those exact bytes, and sends the same bytes as json.RawMessage. Claude-Session: https://claude.ai/code/session_01ByMjAhvdWgbAWWeGE9jwWd * test(decision): queue, runner and read pins; move enqueues (TASK-3117) Move is a door (ruling 3, keyed on the target collection's sets). The three system fan-out rewrites are named as non-doors at their sites. Tests pin: each door enqueues and coalesces; nil resolver writes no row; lease reclaim; mid-evaluation re-enqueue survives completion; backoff; latest-per-key read; hashed bytes == wire bytes; a comment re-evaluates at an unchanged seq; reorder / parent link / role assignment move seq but do not re-evaluate; current flips false after a comment. Claude-Session: https://claude.ai/code/session_01ByMjAhvdWgbAWWeGE9jwWd * feat(server): decision tick, read API, decisions on item GET (TASK-3117) Boot builds the provider from one shared resolver (also used by `pad server info`); a nil provider means no resolver, no job rows, no tick. GET /items/{ref}/decisions returns the latest answer per question with current=true only when computed from the item's present state; the single-item GET and the agent view carry `decisions` when non-empty. A shutdown releases in-flight claims without counting a failed attempt. Claude-Session: https://claude.ai/code/session_01ByMjAhvdWgbAWWeGE9jwWd * test(decision): cancelled pass releases; agent view carries decisions (TASK-3117) Claude-Session: https://claude.ai/code/session_01ByMjAhvdWgbAWWeGE9jwWd * feat(store): comment edit/delete enqueue; mid-call cancel test (TASK-3117) Comment edit (when the body changed) and comment delete change the hashed trail, so they are doors under ruling 3's rule. Adds the mid-call shutdown test that mutant M20 showed was missing, and the decisions endpoint to CLAUDE.md. Claude-Session: https://claude.ai/code/session_01ByMjAhvdWgbAWWeGE9jwWd * fix(decision): never send an item in a soft-deleted workspace or collection (TASK-3117) GetItem filters only the item's own soft delete, so a job owed for an item whose workspace or collection was soft-deleted would have sent that content to the third-party provider during the restore window. The runner now checks three-way liveness (the reminder tick's rule) before building state, and drops such a job. Claude-Session: https://claude.ai/code/session_01ByMjAhvdWgbAWWeGE9jwWd * fix(decision): drop is generation-matched; inapplicable sets are not asked (TASK-3117) Codex round 2. P1: DropDecisionJob deleted the row whatever its generation, so a write that landed while a run was hitting its retry limit (or a gone item) was lost; it now deletes only the claimed generation and otherwise releases, as CompleteDecisionJob does. P2: a job owed before a move still named the old collection's scoped set; Evaluate re-checks applicability against the item's current collection and the job is dropped. Claude-Session: https://claude.ai/code/session_01ByMjAhvdWgbAWWeGE9jwWd * fix(store): idempotency counts only the asked question keys (TASK-3117) Codex round 3. The completeness check counted every stored key at the state, so a RENAMED key kept the total equal and the set read complete; the new key was never asked at an unchanged state. The comment claiming over-counting was only possible in the safe direction was wrong and is replaced. Claude-Session: https://claude.ai/code/session_01ByMjAhvdWgbAWWeGE9jwWd * fix(decision): claim one job at a time; restore enqueues; recheck before ask (TASK-3117) Codex round 4. Claiming a whole pass up front let the last job wait out its lease behind earlier provider calls (worst case ~330s per job) and be re-claimed elsewhere; jobs are now claimed one at a time and the lease comment carries the arithmetic. Restore is a door: a job dropped while the item was deleted left it stale with nothing scheduled. Liveness is re-checked as the last statement before the send, with the residual window stated. Collection rename / field migration not enqueuing is the ruled fan-out gap and stays. Claude-Session: https://claude.ai/code/session_01ByMjAhvdWgbAWWeGE9jwWd * fix(decision): hold jobs of a soft-deleted workspace for its restore (TASK-3117) Codex round 5: a job processed while its workspace was soft-deleted was dropped, and a workspace restore (a fan-out, so it enqueues nothing) never brought the work back. Such jobs are now held: the claim scan skips them and they resume on restore; a purge cascades them away. Collections have no restore path and still drop. Liveness is one guard, the last statement before the send; the earlier copy in State was redundant (mutant M26 survived its removal) and is removed. Claude-Session: https://claude.ai/code/session_01ByMjAhvdWgbAWWeGE9jwWd * fix(decision): answers are keyed on question+model; outages never drop work (TASK-3117) Codex round 6. An answer is now current only for the question definition and model that produced it (question_hash, per key): rewording a question or re-pinning the model at an unchanged item state re-asks. Jobs are dropped only after repeated PERMANENT refusals (a 4xx other than 429); transient failures back off to an hourly cap and are never dropped, so a provider outage loses no owed evaluation. Adds the RunOnce-level test for a workspace deleted mid-job (mutant M35 survived without it). The empty production registry is the contract ("ships the rail with an empty registry"), not a defect. Claude-Session: https://claude.ai/code/session_01ByMjAhvdWgbAWWeGE9jwWd * docs(store): name workspace import as a non-door for decisions (TASK-3117) Codex round 7. Import is the one item-creation path that bypasses createItemTxWithID; it stays off the decision queue for the same fan-out reason SPEC-3 keeps it off events/1. Named at the site and in CLAUDE.md, and raised to the lead as an open policy question. Claude-Session: https://claude.ai/code/session_01ByMjAhvdWgbAWWeGE9jwWd	2026-09-21 20:46:41 -04:00
xarmian	b7e0d6ec05	feat(cli): item update reports the body outcome on its success line; show flags a stale body (TASK-3132) (#1416) ... * feat(cli): item update's success line carries the body outcome; show's table output flags a stale body (TASK-3132) Claude-Session: https://claude.ai/code/session_01X4uYKs5ZPF28eCWfm8VPvE * fix(cli): TASK-3132 round 1 — "may" not "will" for an unguaranteed flush; byte, empty-stale and json/agent legs Claude-Session: https://claude.ai/code/session_01X4uYKs5ZPF28eCWfm8VPvE * fix(cli): TASK-3132 — the body outcome is the LAST stdout line, so it survives | tail -1 past the field summary Claude-Session: https://claude.ai/code/session_01X4uYKs5ZPF28eCWfm8VPvE	2026-09-21 19:53:23 -04:00
xarmian	98213b0334	fix(cli): refuse a blank --stdin body on item create/update; add --clear-content (BUG-3100) (#1415) ... Claude-Session: https://claude.ai/code/session_01X4uYKs5ZPF28eCWfm8VPvE	2026-09-21 18:52:28 -04:00