513 Commits

Author	SHA1	Message	Date
xarmian	dc70ff3d7f	fix(server): write first, apply second on the collab applier path (TASK-2989 / BUG-2840 half A) (#1318) ... * test(server): measure BUG-2840 half A's premise before designing a fix Half A's plan makes step one an experiment, not a design: the claim that a refused PATCH still lands its content was a reading of the snapshot branch rather than an observation, and the shape of the fix depends on which half actually bites. Measured, on the applier path with a live room: a PATCH carrying content and a stale expected_updated_at answers 409, leaves items.content untouched, and adds an op-log row that outlives the request. The caller's refusal is true of the row and false of the collaborative document. The first version of this test was CIRCULAR and reported the premise confirmed. It drove a ?source=collab-snapshot PATCH carrying the refused string, which proves only that a snapshot write writes what it is given. The server cannot close that loop at all: collab here is a dumb relay that persists opaque Yjs updates and never parses them, so nothing server-side derives markdown from a room's document — in production that markdown comes from a live tab's Y.Doc. What IS observable server-side is durable collab state created by a request that was refused, which is what this now measures. Two details that make the harness faithful rather than convenient: - The fake applier emits a binary op as well as the ack. A real applier is a browser tab that applies the markdown and broadcasts the resulting update; acking alone would leave no durable trace, so the experiment would have been measuring a peer that does not exist. - Readiness is detected by the observable difference between the two paths — a succeeding probe PATCH that leaves items.content untouched proves the applier answered — because no exported accessor for electable connections exists and the manager's state is not reachable from this package. The test asserts today's behaviour, defect included, so the fix has a baseline to move. It skips with an explicit "premise NOT established" message if the harness ever stops reproducing the applier writing durable state, rather than passing vacuously. Refs: BUG-2840 * feat(server): write first, apply second on the collab applier path (TASK-2989 / BUG-2840 half A) PLAN-2975 decisions 2-4. A refused PATCH no longer changes the item. The applier path used to push content into the live Y.Doc before the row write, so any of the four typed refusals answered 4xx while the collaborative document had already moved and the next collab-snapshot flush carried the refused content into items.content. The reorder is possible because TASK-2987's HasElectableApplier answers which path the request is on without taking it. routeContentUpdate owns the re-decision deliberately: the predecessor retried ErrRoomActiveDuringPrune inside applyContentViaCollab and re-called ApplyExternalContent, which could succeed through a freshly joined applier and return nil, after which the row write still ran last and reproduced the defect. Re-deciding before anything is written makes that impossible rather than unlikely. Two typed 409s join the structured family. content_not_applied answers the hybrid the reorder creates - row write committed, content not in the document - naming the landed fields and the new updated_at so a content-only retry does not trip OCC. room_settling answers the standoff where PruneAndApply blocks on any writer while election also demands unfrozen and replay-done: the predecessor gave up after three tries and wrote past the live peer, losing the write on its next flush. applier_ambiguous is untouched; its outcome is unknown and a claim either way would be false. The measurement harness is inverted rather than deleted: it asserted the defect and would have become a SKIP, which reads as a pass. Claude-Session: https://claude.ai/code/session_01GqaEDuCtRiSJfa7eppWecn * refactor(server): retire the route-flipping helper chain the reorder replaced (TASK-2989) applyContentViaCollab, applyContentViaCollabOnce, directWriteFn, applyContentMaxRetries and isDeterministicWriteFailure are dead once the router owns the decision, and golangci-lint said so. Removing them is the point rather than tidying: that chain retried ErrRoomActiveDuringPrune internally and re-called ApplyExternalContent, which could succeed through a freshly joined applier and let the row write run last after all. Two things ported rather than dropped. isDeterministicWriteFailure's closed-set warning moves onto writeTypedItemRefusal, which inherits the job of recognising every typed permanent refusal. Its regression test is ported too, unchanged in property: a refusal the handler does not recognise is treated as recoverable and the request re-derives it by another route, which BUG-2804 measured as a rename cascade run twice. CONVE-23 sweep: my own comment on HasElectableApplier, merged four hours ago, said the fallback could write content past live peers. This unit made that false. It now states what the sentence was true of and what replaced it, rather than being quietly deleted. The structural guard needed teaching, not weakening: it counts the handler's refusal blocks and failed closed when one moved into a shared function. It now scans both files and says why three is still three. Claude-Session: https://claude.ai/code/session_01GqaEDuCtRiSJfa7eppWecn * test(server): pin the settle budget itself, which every other test bypassed (TASK-2989) Found by mutation: applierSettleBudget = 0 survived the whole suite. The decision tests pass their own budget, so the constant had no coverage at all — and a zero budget makes the retryable refusal the normal answer for any room with a writer still anchoring. The floor is the measurement the constant was sized from rather than a number: 47ms, just above the 46.41ms worst anchoring time measured for this deployment. Claude-Session: https://claude.ai/code/session_01GqaEDuCtRiSJfa7eppWecn * test(server): bound the standoff subtest so a broken deadline fails instead of hanging (TASK-2989) The only exit from the standoff branch is the deadline, so the mutant that makes it unreachable spins and the failure arrives as a package timeout with no --- FAIL line — which a mutation harness reads as 'the package broke' rather than as a detection. Measured: that is exactly what M5 produced. Same shape as the waiter rule: a failure mode indistinguishable from the waiting mode is not a signal. Claude-Session: https://claude.ai/code/session_01GqaEDuCtRiSJfa7eppWecn * fix(server): content_not_applied must not assert a timeout did not land (TASK-2989) Codex round 1, and the sharpest finding in it. ApplyExternalContent returns ErrAllAppliersTimedOut only AFTER an applier_request has gone out on the wire, so the elected peer may have applied the markdown and persisted its ops while the ack was lost or merely late. Answering content_landed:false there states as fact something the server cannot know — the same overclaim the ruling avoided by leaving applier_ambiguous alone, arriving one door over. The discriminator already existed upstream and needed no new machinery: electAndApply returns ErrNoApplierAvailable when anyWriteSucceeded is false (nothing reached a peer) and ErrAllAppliersTimedOut when something did. The envelope now carries content_outcome, and content_landed is ABSENT rather than false when the outcome is unknown, because a caller that reads false may act on a premise nothing supports. Three smaller round-1 items. The settle budget's comment now says it bounds how long the route keeps ASKING, not how long the request takes — the deadline is only consulted between attempts and PruneAndApply can block on the per-item lock. A comment on fullWriteHandled still named applyContentViaCollab, which this unit deleted; my own sweep missed it. The ported classifier test now inspects the recorder rather than only the boolean, since a mutant could return true while writing the wrong status. Verified and NOT changed: nil-ing content on the row write does not newly suppress version bracketing. main already set input.Content = nil on the applier path before its row write, so that behaviour is identical before and after the reorder. Claude-Session: https://claude.ai/code/session_01GqaEDuCtRiSJfa7eppWecn * fix(collab,server): the not-applied claim was still false on two post-wire paths (TASK-2989) Codex round 2, and it refuted the reasoning I gave in round 1's fix rather than just finding another case. I said the discriminator already existed upstream because electAndApply tracks anyWriteSucceeded. It does — PER ELECTION — and two paths escape it: - a restore storm returns ErrNoApplierAvailable after up to applierMaxRestartsAfterRestore elections, each of which may have put an applier_request on the wire, with the per-election flag discarded at every restart; - a registerPendingAck failure on a retry attempt returns a raw error after an earlier attempt had already sent one. Both would have answered content_landed:false about content that may have landed. Same shape as the finding they follow: a reason that was sufficient-sounding and one file short of true. Fixed at the source where the source can know it — ApplyExternalContent now carries sentAny across restarts, so ErrNoApplierAvailable means what its callers read it to mean — and by construction everywhere else: classifyApplyOutcome is a whitelist, so only the two sentinels that mean nothing reached a peer may make the claim and every other error, including ones nobody has written yet, degrades to unknown. Cancellation: the re-decision wait is the only new blocking wait this branch adds, and it now ends when the caller goes away. The rest of the path was context-blind on main and stays that way; threading a context into the store and the applier round-trip is a different change. Not fixed here, deliberately: the ambiguous-commit double-write. Codex confirmed against main that it has the identical shape there, so it is pre-existing and gets filed rather than folded into this unit. Claude-Session: https://claude.ai/code/session_01GqaEDuCtRiSJfa7eppWecn * docs(collab): sweep the prose my own round-2 fix falsified (TASK-2989) Codex round 3, one P3, and it is CONVE-23 arriving for the third time in this unit. Carrying sentAny across the restart loop changed which sentinel a restore storm returns, and left two comments describing the old behaviour: the cap's doc still said exhaustion falls back with ErrNoApplierAvailable, and the sentinel's own doc still said every attempt timed out. Both now say what the sentinel MEANS rather than how it usually arises — bytes reached a peer and the outcome is unknown — because that is the half two callers depend on: the op-log prune stays suppressed, and the PATCH handler reports the content outcome as unknown rather than not-applied. Claude-Session: https://claude.ai/code/session_01GqaEDuCtRiSJfa7eppWecn * fix(server): restore the UNIQUE-constraint 409 the applier path used to inherit (TASK-2989) Codex round 5, and a regression rather than a gap. The ordinary error block maps a UNIQUE-constraint race — two updates that both pass checkUniqueFields and then both hit the partial unique index on invocation_slug — to a 409. Before the reorder the applier path's row write ran through that block and inherited the mapping. Routing it through a helper built from 'the four typed refusals' dropped the arm and turned a benign race into a 500 on that route alone. The irony is the lesson, and it belongs on the record: writeTypedItemRefusal exists BECAUSE this handler's refusal set has been under-counted three times, and I under-counted it again while building the thing meant to stop that — by taking the population from the errors that have a Go type rather than from the block that actually answers them. The new arm's first version panicked on a nil error, since it dereferences where the typed arms use errors.As. The existing nil control leg caught it immediately, which is the entire reason that leg is there. The structural guard now DERIVES its file set — every non-test file in the package that calls UpdateItemWithParentLink — instead of listing two names, so a future block in a third file cannot sit unmapped while the test passes. Claude-Session: https://claude.ai/code/session_01GqaEDuCtRiSJfa7eppWecn * test(server): the guard now requires the fifth arm, scoped to the block's own function (TASK-2989) Codex round 6. Two gaps in the guard as it stood: it verified only the four typed arms, so removing the UNIQUE-constraint mapping from either ordinary block still passed; and its file set matched on the store call text, so a file reaching the store through a wrapper would not be scanned at all. The file set is now the UNION of files calling UpdateItemWithParentLink and files calling any of the arms — a refusal block lives where the arms are called, whatever it calls the store through. The fifth-arm check is scoped to the ENCLOSING FUNCTION, and that is the part worth reading. The first version asked whether a UNIQUE literal appeared between one block's start and the next block's start in token.Pos. Those windows span whole files, so the gap between the last block of one file and the first block of the next swallowed every literal in between — two in handlers_items.go belonging to the create and restore paths, one in handlers_items_bulk.go. All three mutation controls survived it. It asserted nothing, and it passed, which is the only reason I looked. Committed BEFORE the controls run this time. The previous round's controls used git checkout -- internal/ against uncommitted guard work and deleted it; the tree read clean afterwards, which is the ambiguity — clean means the mutation was reverted OR the mutation and my work both were. Claude-Session: https://claude.ai/code/session_01GqaEDuCtRiSJfa7eppWecn * test(server): scope the fifth-arm check to the block statement, not the function (TASK-2989) Per-function was the second wrong containment and the controls said so: handleUpdateItem holds TWO refusal blocks with a UNIQUE arm each, so neutralising either hid behind the other and survived. Only the writeTypedItemRefusal control was detected — the check covered one of the three blocks it claimed to cover. Innermost enclosing BlockStmt is the containment that matches what the sentence means by 'the block's own arm'. Claude-Session: https://claude.ai/code/session_01GqaEDuCtRiSJfa7eppWecn * test(server): the fifth-arm check reads if-conditions, not any literal in the block (TASK-2989) Codex round 7. Scanning the whole BlockStmt for a matching string literal let an unrelated nested closure — or a message string quoting the phrase — satisfy the guard after the real mapping had been deleted. That is the guard passing for a reason unrelated to what it asserts, which is the failure this whole check exists to prevent one level down. It now reads only IfStmt conditions, which is the shape the arm actually has. Claude-Session: https://claude.ai/code/session_01GqaEDuCtRiSJfa7eppWecn	2026-09-10 08:54:41 -04:00
xarmian	793fad959c	docs(mcp): the reason given for the protocol restriction was false — replace it with the true one (TASK-2977) (#1312) ... TASK-2977 step 1 restricted the remote transport to the handshake era and gave this reason: pad_set_workspace pins a session default workspace that the stateless era has nowhere to keep, so pad "is not known to be able to serve" that era. It is plausible and it is false for THIS transport, and I wrote it reasoning from the tool's purpose rather than from its remote behaviour. cmd/pad builds the cloud dispatcher with a SHARED workspace state whose ResolveDefault() returns "" by construction — BUG-1865, the cross-user workspace bleed — so the pin is recorded and never consulted here. Resolution on /mcp is the explicit workspace argument, else a default derived per request from the caller's own OAuth identity and token allow-list. Every input comes from the request. This transport has been stateless with respect to workspace resolution since that bug was fixed, and the fix for a cross-user bug turns out to be most of the work a stateless era would need. THE TRUE REASON IS BETTER AND WAS ONE FILE AWAY. The mcp-active-sessions gauge is keyed on the Mcp-Session-Id header, and the generate-only session-id manager at this transport's call site exists so that header is always minted and the gauge stays observable (PR #400 round 1). SEP-2567 REMOVES session IDs in 2026-07-28 — a server serving that revision never mints or echoes one — so in that era nothing pad mints is available to key on. Ruled day 62: that is an accepted cost, not a blocker, and it is recorded where the key is CHOSEN rather than only where the era is refused — middleware_mcp_session.go now carries the obligation on whoever opens that era to re-key the gauge first, and says why the era's arrival is exactly the moment a silently-flat gauge gets read as "no MCP traffic" instead of "no measurement". The superseded reason is kept in the comment as superseded, four lines of it, because the false reason is the PLAUSIBLE one: the next person to reason about the stateless era from pad_set_workspace's name will reach for it, and the comment now meets them with the shared-state mechanism instead. TWO CODEX ROUNDS, TWO FINDINGS, both about this change's own prose and both verified in the code before accepting: - The server-package comment named mcp.ServedProtocolVersions as a symbol. It is not reachable from there — internal/mcp imports internal/server and not the reverse, which is also why the transport reaches the router as a plain http.Handler. Named by path now, with the direction stated. - "Modern-era traffic would be invisible to the gauge" was OVERSTATED, which is this unit's own defect class arriving inside the fix for it. trackMCPSession resolves the id from the response header and FALLS BACK to the request header, so a modern-era client that volunteers an Mcp-Session-Id is still tracked. The accurate claim, now in both files: the gauge stops depending on anything pad mints and starts depending on whether clients keep sending a header the spec removed — under-counting by a margin nobody controls, rather than a flat zero. Comment-only; no behaviour changes. The restriction, its four tests and the derived version set are untouched. Claude-Session: https://claude.ai/code/session_01GqaEDuCtRiSJfa7eppWecn	2026-09-09 20:35:44 -04:00
xarmian	734b53f33c	fix(attachments): a HEIC embed is decided by what can be served and painted, not by an image/ prefix (BUG-2964) ... A pure-Go build derives no HEIC thumbnail, and the byte endpoint SILENTLY serves the original when the requested variant is missing — so an editor that chose <img> on a MIME PREFIX handed the browser HEIC bytes, which Chrome and Firefox render as the broken-image icon. The server now says what it did: X-Pad-Attachment-Variant names what the BYTES are (a fallback still reports `original`), and X-Pad-Attachment-Derived names which variants EXIST, answered only on the no-variant path so the hot image path pays nothing. `none` is a sentinel rather than an empty value, because ABSENCE has to keep meaning "server predates this fix". The rule, in both renderers: embed as <img> iff THE VARIANT THIS RENDER WILL REQUEST exists, OR the browser paints the original. Not a prefix (the bug); not availability alone (the same build derives no AVIF thumbnail, and browsers decode AVIF). The second disjunct is a new fourth predicate beside display.ts's three, NOT canOpenInViewer — that one excludes image/svg+xml for active-content reasons, and an SVG inside an <img> runs no script, so reusing it would have flipped every existing SVG embed to a chip. Verified on a REAL HEIF against a pure-Go build, with a PNG positive control on the same instance: HEIF reads `none` and answers ?variant=thumb-md with 200 + Content-Type image/heif — the defect itself; PNG reads `thumb-sm,thumb-md` and answers with the actual thumbnail. Three counterfactuals and a negative control on the new TS/Go lock-step test, which replaces a lock-step that had been asserted in a comment only. Codex CLEAN after 3 rounds; rounds 1 and 2 each found a real defect in the fix (a per-variant fact collapsed to a boolean, and an async fact cached as durable). The share-link 404 is deliberately unchanged — that path serves variants only because the variant pipeline is the privacy boundary, and serving an original to an anonymous viewer would trade a broken image for an EXIF/GPS leak. The limitation is documented per surface beside the capabilities endpoint. Claude-Session: https://claude.ai/code/session_01Xk9M5UVPdc84xL5E1mZkm8	2026-09-08 19:40:37 -04:00
xarmian	5ec17a7c92	fix(cli): pad server stop stops the server that is running, or says it is (BUG-2965) (#1298) ... fix(cli): `pad server stop` stops the server that is running, or says it is (BUG-2965) `StopServer` read the PID file and, on any read error, answered "server not running (no PID file)" — without asking whether anything was listening. The file was written in exactly one place, EnsureServer's auto-start branch, so a server started any other way held the port with no file to find: a service unit, a human running `pad server start`, the seats' refresh recipe relaunching with the killed process's argv. A stop command that says "not running" about a running process leaves the caller believing they stopped something, and the next thing they do rests on that belief. Two halves, per the item's property and corollary: - A missing PID file now asks the port. Only an unhealthy address earns "not running"; a healthy one earns a message naming the address, the missing file, and what to do instead. Deliberately NOT "find the listener and kill it" — resolving a pid from a port is platform-specific, and the process holding it may not be ours. A stop that kills by port can kill a stranger. - `pad server start` claims the PID file itself, so the file exists for every start path rather than only the auto-started one. The second half took four codex rounds to get right, and each round found the previous shape reintroducing the defect it was fixing: 1. Write-then-defer-remove let a duplicate start overwrite a running server's entry and then delete it on the way out, leaving a healthy server unaddressable. 2. Refusing to replace a live pid fixed that and opened its mirror: the start that LOST the port could still own the file, so the winner was unaddressable. The fix is ordering, not arbitration — BIND FIRST, then claim, so the file always names the process that owns the address. internal/server grows Listen and Serve for that; ListenAndServe is now the two together. 3. With the bind first, EnsureServer's parent-side write became the stale mechanism (it records a child that may never bind) and the live-pid refusal became actively wrong (no live process can be serving an address we just bound). Both removed, along with processIsAlive, whose only remaining callers were its own tests. 4. Cleanup is a read-then-remove, so running it AFTER the listener closes let a successor bind and claim between the two steps and lose its file to us. It now runs before the listener closes, while nothing else can legitimately own the file. The cost is a drain-window where a healthy server has no PID file and `stop` says so — a true message in place of a silent wrong one. Verified live against the built binary, in a throwaway HOME, in both shapes: start writes the file naming the serving process; a second start against the held port fails at bind and leaves the first server's file intact; stop then stops it and removes the file; a further stop reports "not running". The first live run also caught a flaw in my own method — `stop` reads the config's port, so the probe answered about 127.0.0.1:7777 (this box's dev server) until it was re-run with PAD_PORT set. Re-checked after the restructure. Mutants: the health check removed, the health branch still answering "not running", an empty PID file, a cleanup that does not remove, and a cleanup that removes a successor's file are each killed by a named test. The call site itself is wiring a unit test cannot vouch for (CONVE-19) — that is what the live runs cover, and the Listen/Serve split is pinned in internal/server. make lint, make test green. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR	2026-09-08 18:26:32 -04:00
xarmian	a2bab75c73	fix(server): the op-log prune rides inside the write it justifies (BUG-2840 half B) (#1295) ... fix(server): the op-log prune rides inside the write it justifies (BUG-2840 half B) On the no-room / no-applier path a content PATCH pruned the item's Yjs op-log and then wrote items.content. The prune ran FIRST, in its own statement, on this justification: "any prior collab state is strictly older than the items.content the caller is about to write". That premise holds when the write LANDS. Four typed refusals can come out of that write — the open-children guard, the optimistic-concurrency conflict, the rename-cascade byte refusal and the item-title refusal — and on every one of them the caller wrote nothing, so the pruned ops were superseded by nothing. They were simply gone. It is not hypothetical on this branch. It fires on ErrNoApplierAvailable, i.e. a room inside its 60s grace TTL with zero connections: exactly the state where the op-log holds a closed tab's edits that never reached items.content. Those edits exist nowhere else, and a request that wrote nothing destroyed them. The prune now runs INSIDE the write's own transaction, composed onto the precheck hook that UpdateItemWithParentLink already runs there, so a refusal rolls it back. directWriteFn takes the prune as a hook rather than performing it, which keeps the choice of transaction with the caller that owns the write. This is the shape version-restore already uses: PruneItemOpLogTx's own comment says a split prune/commit "leaves a divergent state on any failure" in EITHER order, and closes that split by running the wipe in the update's transaction. This path was the remaining split, and it also closes the opposite window a plain reorder would have left — a crash between a successful write and a later prune, leaving stale ops to be replayed over fresh content. One deliberate behaviour change: a prune failure now rolls the content write back, where before it was logged and the write proceeded. That leniency assumed the prune was optional cleanup; it is not. A write that commits with a stale op-log is the "resurrect stale content on the next flush" hazard the prune exists to prevent, and it is the same trade the restore path made. Also removes internal/server's distantFuture, whose only use this was; the collab package keeps its own copy for its own prune. Grepped the repo before deleting rather than inferring deadness from the edit in front of me. Verified by a negative control — restoring the prune-first ordering fails the refusal test — and by three mutants: never invoking the hook and passing nil for it are both killed by the success test. A fourth, swapping the guard and the prune inside the hook, SURVIVES, and the comment now says so: both orders are equivalent while they share a transaction, so that ordering is a preference and not a rule anything enforces. make lint, make test and make test-pg green; PG run because this moves a DELETE into a transaction. Codex CLEAN. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR	2026-09-08 16:46:28 -04:00
xarmian	696b477b12	test(server): the mobile shells' unversioned contracts fail CI when they move (TASK-2053) (#1294) ... test(server): the mobile shells' unversioned contracts fail CI when they move (TASK-2053) The native shells talk to this server over string contracts nothing protects: route paths, cookie names, JSON keys, a header shape. None sits behind the MCP tool-surface version or any other gate, and the min-server-version warning the app shows covers the opposite direction — it protects a NEW app against an OLD server and says nothing when the server moves forward under a shipped build. A shipped build cannot be patched on our schedule, so a renamed key is a silent break for every installed copy until its owner updates. One file, one table, six contracts, each naming the shell surface that breaks: - Both session cookie spellings. __Host-pad_session is the one that matters — every deployment a phone talks to is TLS — and it appeared in NO Go test assertion, only in a config-test comment. The insecure spelling is in nineteen test files as a helper building a request, which is a mention and not a claim about the name. - The six auth routes, asserted by WALKING the route table rather than by firing requests: a request-based check answers "did something handle this", which a catch-all or a redirect satisfies while the route itself is gone. - The /auth/session keys the app branches on before showing any UI, version included. A rename there reads on a phone as a blank screen or a login form on an instance that needs setup, not as an error. - The full two-step 2FA sign-in. requires_2fa, challenge_token and recovery_code were live request/response keys appearing in ZERO test files; the test drives login → login-verify with a real recovery code, so a rename on either side fails here. - Content-Disposition carrying a filename on BOTH dispositions (BUG-2910). The existing download tests assert the inline;/attachment; prefix and say nothing about the parameter after it, so the filename could be dropped from either branch with the suite green. Scope is stated in the file rather than left for a reader to re-derive. /auth/apple/native is a pad-cloud route; the app-scheme redirect allowlist does not exist in this repo (greps for the scheme forms and for app_scheme/appScheme return nothing in Go, TypeScript or Svelte); and the OAuth error codes this repo emits belong to Dynamic Client Registration for MCP clients, not to mobile sign-in — pinning them here would look like mobile coverage while protecting a different client entirely. Verified by an eight-mutant matrix: renaming either cookie name, the login-verify route, requires_2fa, challenge_token or recovery_code, dropping version from the session payload, or dropping the filename from Content-Disposition each fails a named test. Every mutant is a change someone could plausibly make in an afternoon, and every one of them was silent before. Two of my own assumptions were caught by running it rather than by reading: chi's router is built lazily, so walking it on a server that has served nothing panicked; and the "downloaded document" case used a .txt, which is on the read path's inline-safe allowlist and came back inline — the case would have tested the same branch twice under a name claiming otherwise. It uses a real zip now. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR	2026-09-08 15:49:05 -04:00
xarmian	89a9fb0241	fix(server): a refused collection prefix reaches the caller as a 400 naming the rule (BUG-2951) (#1293) ... fix(server): a refused collection prefix reaches the caller as a 400 naming the rule (BUG-2951) BUG-2943 made the store refuse a prefix outside the grammar, with a message naming the rule and an example. Three of the four doors onto that refusal threw the message away: they mapped conflict shapes and sent everything else to writeInternalError, so `pad collection update docs --prefix "ab1"` answered "An internal error occurred". The refusal kept its data-protection value and lost its entire teaching value — a 500 with no text reads as an outage, so the honest user response is to retry or report one. An MCP agent was told something worse than nothing. internal/mcp classifies a stdio failure by matching CLI stderr prose; the generic message matches none of the validation patterns, so a permanently-invalid prefix arrived as the RETRYABLE server_error code and the correct agent response was to retry a call that can never succeed. Nothing in internal/mcp changes here: the store's own message now reaches stderr and the existing `invalid` pattern recognises it. Both directions are pinned by tests, including the negative control that the old generic string still classifies as server_error — which is right for a real internal failure, and is why the refusal had to stop wearing that message. The population, read door by door rather than grepped: - CREATE and UPDATE lost the message entirely (500, no text). - DELETE kept it via strings.Contains on the store's error text — the right status by the wrong mechanism: a reworded refusal became a 500 silently. - Workspace IMPORT kept the text under a 500, while its sibling bundle-import door already answered 400 for the same class. store.ValidationError carries the caller-facing Reason, with AsValidationError for the doors, following the InvalidDocumentTitleError precedent in the same package. Constructing it is the per-site DECISION that a message is safe to show; the alternative — returning err.Error() from the generic path — makes that decision by default for every error any layer may later add. Doors render Reason, never Error(), because Error() carries the sentinel prefix and whatever a call path wrapped around it; a test helper asserts no response leaks that prefix, after a mutant swapping Reason for Error() survived every message assertion (Reason is a substring of Error(), so a contains-check cannot see it). Two sites are deliberately NOT converted, both read and left: - The template-seeding trait validation (collections.go) checks FIRST-PARTY template code, not caller input. A 500 is the honest answer there. - The two expected_updated_at refusals are converted for uniformity but are unreachable through HTTP — both doors validate the token at the boundary. They are defence in depth, not live paths. WIRE CHANGE: POST /workspaces/import now answers 400 for a caller-input refusal where it answered 500. The code string (import_failed) and the message are unchanged, and its sibling bundle-import door has always answered 400 for this class, so this aligns two doors onto one refusal. Ruled by the lead rather than decided here. "Cannot delete a default collection" stays 400. Codex round 1 caught the consumer this change created: the tar.gz import door renders a bundle failure through its own envelope and its fallback wraps err.Error(), so the very edit that made the JSON door actionable moved the sentinel prefix into the bundle door's message. It now detects the type and renders Reason in its own "Bundle pad-export.json is not importable" envelope, with a test and a mutant. A producer change is not finished until its consumers have been read; round 2 was CLEAN. Verified by negative control (each door arm removed in turn) and a ten-mutant matrix in which every arm removal, every store site reverted to fmt.Errorf, and both Reason→Error() swaps are killed by a named test. One matrix attribution was wrong on first run — a store test appeared as a casualty of a server-side mutant — and re-running it in isolation showed the mutant does not affect it; the runner had attributed every FAIL line in a two-package run to the mutation. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR	2026-09-08 14:51:21 -04:00
xarmian	6a5eb3dee0	fix(attachments): HEIC, HEIF and AVIF are recognised from their bytes, not refused (BUG-2961) (#1291) ... fix(attachments): HEIC, HEIF and AVIF are recognised from their bytes, not refused (BUG-2961) image/heic, image/heif and image/avif have been on the upload allowlist all along and every real file of those types was rejected 415 mime_not_allowed: http.DetectContentType implements the WHATWG mimesniff table, which has no signature for ISO base media file format still images, so they sniffed as application/octet-stream and ValidateUpload's first rule refused them before the extension cross-check was ever consulted. HEIC is the iPhone camera default, so that was every photo shared straight from an Apple device, through the web UI today and the mobile share sheet being built. The fix stays inside rule 1 rather than around it. sniffISOBMFFImage reads the ftyp box and returns the allowlisted MIME, so the sniffed type is still what the allowlist is consulted with; it returns "" for everything else, so it can only add detections. The alternative was to trust the filename for these three, the way ValidateUpload already trusts it for zip-based Office documents, and that was refused in triage because a renamed .heic would then pass on a name alone. Both tests layers pin that: an executable named .heic is still refused, and real HEIC bytes named .jpg are still stored as image/heic. Four decisions, each argued at its site: - Every brand in the box, not just the major one. The measured Apple HEIC carries major heic with compatible mif1 MiPr miaf MiHB heic; the measured Apple AVIF carries major avif with compatible MiPr avif miaf mif1; libheif writes major mif1. A check on bytes 8..12 alone is right for one encoder and wrong for the next. A specific brand wins wherever it appears; mif1 alone means image/heif. - Sequence brands (hevc/hevx/hevm/hevs, avis, msf1) are deliberately absent: they name image/*-sequence types, none of which is on the allowlist, so recognising them would produce a sniff ValidateUpload then refuses with a stranger message. - A container carrying an mp4 brand stays a video and is left to the stdlib. - A size-1 (64-bit largesize) ftyp box is declined, because the brands shift eight bytes and the fixed offsets would be reading the size field. Fixtures are real encoder output with provenance recorded in testdata/README.md (sips on macOS 26 for the Apple pair, libheif 1.20.2 here for the other two); hand-built byte vectors appear only for shapes no available encoder produces. Verified by negative control — with the pre-check disabled all four new acceptance tests fail — and by a nine-mutant matrix in which the major-only scan, the dropped mp4 yield, generic-over-specific preference, the minor version read as a brand, the missing box bound, the removed largesize bail, a dropped brand table entry and an unconsulted sniffer are each killed by a named test. The one survivor is equivalent: an 8..11-byte buffer yields no whole brand either way. Codex found two more refusals this owed, both now guarded and mutation-checked: a declared box size of 2..15 is impossible for an ftyp box, and a buffer that stops inside the 16-byte header is a truncated file — in either case the bytes at offset 8 are not brands of a box that exists, and classifying from them let a caller-controlled payload be stored and served as an image. Also corrects processor.go's claim that "display always works (browsers handle WebP / AVIF / HEIC natively)", which the client's own viewer table contradicts and which this change would have made load-bearing (CONVE-23). What it means for rendering on builds that cannot derive a thumbnail is filed as BUG-2964; the wider class of allowlist entries the sniffer cannot produce is BUG-2963. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR	2026-09-08 13:30:42 -04:00
xarmian	367aae8e18	fix: one prefix grammar, and the ref parser widens to it (BUG-2943) (#1286) ... * fix(collections): a DERIVED prefix is A-Z only (BUG-2943) DerivePrefix took the first BYTE of each word, so a collection named 'TEMP Rook A 2870' got the prefix 'TRA2'. parseItemRef resolves a PREFIX-NUMBER ref only when every prefix character is A-Z and otherwise falls through to a slug lookup, so every item in that collection printed an issue ID the CLI then refused: 'pad item show TRA2-2942' answered 'item not found' while the slug resolved fine. Two functions, each locally reasonable, disagreeing about what a prefix may contain — and the generator was the permissive one, so the failure surfaced at read time on an identifier the product itself minted and printed. The first-BYTE bug had a second half: a word starting with a multi-byte rune contributed a UTF-8 lead byte, so a collection named in most non-Latin scripts produced a prefix that is not even valid text. Non-letters are SKIPPED rather than mapped — there is no honest A-Z substitute for '2' or 'Omega', and inventing one puts a character in the ID that is in nobody's collection name. A name with no ASCII letters yields the empty string, which store.CreateCollection already turns into its ITEM fallback. SCOPE, stated because the first draft of this message overstated it (codex round 1 [P2]): DERIVED prefixes are safe now; the INVARIANT IS NOT ENFORCED. Three other doors store a prefix verbatim and unvalidated — CreateCollection with an explicit input.Prefix, UpdateCollection, and workspace import — so the same unresolvable-ID defect is still reachable through the API, the --prefix flag and a restore. Named on the trail with their call sites, held for a ruling rather than swept into this commit, because the import door wants a different answer from the other two: refusing a restore is not obviously right. The parity test lives in internal/store, where parseItemRef is: it asserts the generator against the RESOLVER rather than against a restatement of the resolver's rule, which is how these two drifted apart in the first place. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR * fix(store): one prefix grammar at all four doors, and the parser widens to it (BUG-2943) The ruled shape, which dissolves the import dilemma rather than choosing a side of it: collections.IsValidPrefix is the single definition — an uppercase letter followed by uppercase letters or digits — and parseItemRef now asks it instead of carrying its own stricter A-Z rule. Because the PARSER widened, a workspace already carrying a prefix like AB1 resolves every item by its printed ID the moment this ships. No migration, no rewrite of an identifier a user's other records may reference. The four doors: - derive: unchanged from the previous commit, still letters-only, still within the grammar; - create with an explicit prefix: REFUSED if outside the grammar, with a message naming the rule. The caller typed it, so a refusal is actionable; - update: same, and it matters more here — update is the door someone reaches for to FIX a bad prefix, so it must not accept another one; - import: the most permissive door that can still be honest. Anything the parser resolves is accepted (which now includes digits); only a prefix NO surface could resolve is refused, naming the collection and saying the export can be edited. Carrying that verbatim would restore a workspace whose items print IDs the CLI answers 'not found' to, which is this item's defect rather than a compatibility owed. An ABSENT prefix on import is not an unresolvable one. Old exports and every fixture in the suite carry "", and the first version of this check refused them — turning a fix for unresolvable IDs into one that cannot restore an old bundle at all (caught by three server tests). It now takes the same derive-then-ITEM fallback CreateCollection applies, which also upgrades it: an empty prefix is itself unresolvable, since the ref would begin with a dash. A prefix accepted only because the parser widened is logged at WARN, so an operator can see an id-space that would have been rejected before rather than inferring it from a resolve failure that no longer happens. Tests for each door follow in the next commit. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR * test(store): one test per prefix door, plus the parser round trip (BUG-2943) Each door asserted separately: 'they all call the same helper' is a claim about the code, not about behaviour, and the bug was two definitions disagreeing. - create with an explicit prefix: AB1 accepted and resolves; ab1, 1AB, 'A B', A-B, A!, a non-Latin letter and a bare digit refused, with the rule named; - update: AB1 accepted, a bad replacement refused AND the stored prefix unchanged after the refusal — update is the door someone uses to FIX a bad prefix, so it must not swap one unresolvable id-space for the next; - import: a digit-bearing prefix restores unrewritten and resolves; one no surface can resolve is refused naming the collection and the export; an ABSENT prefix takes the create-path fallback and comes back resolvable; - the parser: every prefix the doors accept round-trips, and 1AB / 9 / 'A B' / A! / a trailing dash / a bare prefix stay refused. One correction: my first version of the parser test asserted that 'ab1-42' is refused. It is not, and the code is right — parseItemRef upper-cases before splitting, which is what makes "pad item show task-5" work. Case-insensitivity is now PINNED rather than mis-asserted, because a later reader working from the grammar comment alone would otherwise 'fix' it. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR (This message was rewritten once: the sentence above lost its example because the original was written with backticks inside a double-quoted shell string, which the shell EXECUTED and replaced with the command's empty output. The span was blank in the commit as first written.) * fix: the widened grammar reaches its consumers too (BUG-2943) Codex round 2. Widening parseItemRef without widening what CONSUMES a ref would have left the same two-definitions bug this unit is about, introduced by its own fix: - cmd/pad/cmd_github.go matched [A-Z]+-\d+, so 'pad github link' on a branch carrying a digit-bearing ref silently found nothing; - web localSearch's palette Enter fast-path could not recognise one either. Both now match collections.IsValidPrefix. Tests strengthened, both on codex's reading: - the import fallback pinned the VALUE, not just resolvability — asserting 'non-empty and parseable' passes an implementation that stamps ITEM on every absent prefix, giving every collection in a restored workspace the same id-space. Two legs now: an ordinary name derives TASK, a letterless name falls through to ITEM, which is what makes it 'derive, THEN ITEM'; - the WARN the ruling asked for had no test, so it was a line nobody would notice was gone. Now asserted, with a control that an ordinary prefix does NOT warn — a log everything trips is a log an operator learns to skip. Three comments still described the parser's A-Z rule as current, including one in the file that changed it. STILL OPEN, on the trail for a ruling: web paneTarget.ts keeps the narrow grammar on PURPOSE — its comment argues a digit-permitting shape would misclassify a slug like 'roadmap2-5' as a ref — and that argument cited the server rule this unit just widened. Whether the guard follows or stays is a question about the widening's blast radius, not a line to change quietly. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR * fix(web,store): the pane guard follows the server, and the precedence is written down (BUG-2943) Both lead-ruled after codex round 2 surfaced them. paneTarget's REF_SHAPE kept a LETTERS-ONLY grammar deliberately, citing the server's A-Z loop as its warrant. The server dropped that rule, so the guard was holding a grammar nothing else holds — which does not avoid a wrong answer, it produces a different one. It now matches IsValidPrefix. The cost is real and is stated in the test rather than buried: an HREF whose last segment is ref-shaped under the wider grammar is compared by NUMBER with the prefix discarded, so a genuine slug like 'roadmap2-5' now counts as the same pane target as TASK-5. The existing test pinned the opposite and is REPLACED, naming what changed and why. The prefix is dropped because a moved item keeps a stale one (the server's own number-only fallback) and PaneGuardItem carries no prefix to compare; tightening that means widening that type and its callers, which is a separate change and is on the trail. The SLUG-channel leg is kept as its own test: provenance, not grammar, is what protects it — a target naming an item by slug is judged only as a slug. ResolveItem's ref-before-slug precedence is now documented on the function and pinned in both directions: 'ab1-42' resolves as a SLUG when no AB1-42 exists, and a live ref wins when it does (case-insensitively). The widening made more strings ref-shaped, so 'is my slug still findable' needed an answer that does not depend on reading the resolver. Web unit tests run here via a node_modules SYMLINK to the main checkout, which CLAUDE.md permits; npm ci was not run and must not be. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR * fix(web): the self-pane guard was the fourth copy of the ref grammar (BUG-2943) Codex round 3 [P1]. The route-level refNumber() still matched [A-Za-z]+, so a master whose ref is R2-1 parsed as null, its item_number fell back to 0, and the same-item guard stopped recognising ?item=R2-1 as the master — mounting a second provider for the item already on screen. That is the FOURTH consumer found carrying its own copy of this grammar (github branch extraction, the search palette, the pane target guard, and now this). Four independent copies is the argument for the shared definition rather than for four careful edits, and it is why the widening had to be swept rather than applied where it was noticed. Also from round 3: comments saying these client regexes 'match collections.IsValidPrefix' were imprecise — the validator accepts uppercase only, while the client patterns accept either case on purpose, because a user types a ref however they like and the server upper-cases before splitting. They mirror the ref GRAMMAR, and now say so. Web gates run here through a node_modules symlink to the main checkout (permitted; npm ci is not): vitest 2195 passed, svelte-check 0 errors. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR * fix(collections,e2e): the fifth and sixth copies, and the docs that taught the old rule (BUG-2943) Codex round 4, after I claimed the sweep was complete twice. - Two SEEDED PLAYBOOK BODIES carry their own ref grammar and instruct agents with it: playbook_library_plan.go and templates_sdd_spec.go both said a ref 'matches ^[A-Z]+-\d+$'. An agent following those literally would refuse to treat AB1-42 as a ref — a grammar copy that lives in PROSE and is executed by a reader rather than a regexp engine, which is why two sweeps of the code missed it. - Three e2e comments taught the defect as a rule: one of them carries the empirical confirmation ('GET /items/BS1-10 404'd while the slug worked'), which is precisely this bug. They now say the by-ref 404 is fixed and that the explicit prefix those suites pass buys DETERMINISM rather than dodging it. Counting honestly: six live copies of one grammar, found in four rounds of review, two of which I opened by asserting there were no more. The shared definition is the fix; every one of these was a place that had quietly made its own. Gates: go test ./... 0, make lint 0 issues, vitest 2195 passed, svelte-check 0 errors (web run through a node_modules symlink to the main checkout — permitted; npm ci is not). Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR	2026-09-07 17:32:36 -04:00
xarmian	ee0d945863	fix(cli,mcp): one --field key=value entry means one thing at every door (BUG-2870) (#1283) ... * feat(items): one shared parse for a --field key=value entry (BUG-2870) Six sites parsed that entry independently — item create, list, update, move and copy in cmd/pad, plus ingestFieldKVP on the remote /mcp door — in four spellings, and they disagreed about what it meant. The CLI sites used both halves verbatim, so `--field " effort=l"` stored an undeclared field named " effort" and left the declared `effort` untouched; the remote door trimmed both halves and wrote `effort`. Same call, two stored keys, decided by which transport the caller was on. This is the helper only; the call sites move over in the commits that follow. Two rules, deliberately asymmetric, per the day-60 ruling: - a KEY whose trimmed form differs from what was written is REFUSED at every door, rather than silently retargeted to a different field; - a VALUE is carried VERBATIM at every door, because trimming reinterprets a caller's bytes and on a text field the space is content. A padded value against a typed field is refused one layer down by validation, naming the field — measured, not assumed. ErrFieldEntryMalformed is returned rather than handled because the six sites deliberately disagree about a malformed entry (four skip it, copy hard-errors) and unifying that is a separate decision. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR * fix(cli,mcp): all six --field parse sites go through the one helper (BUG-2870) item create, list, update, move and copy in cmd/pad, plus ingestFieldKVP on the remote /mcp door, now call items.SplitFieldEntry instead of each rolling its own split. A padded key is refused at every door; a value reaches every door verbatim. Two sites keep something specific to them, both documented in place: - `item list` is a READ filter, and it takes the same key rule deliberately: a padded key there filters on a field nobody declared and returns empty, which is indistinguishable from "no rows match". - `item move` gets KEY normalisation only. Its values stay strings because the server types a declared field on that path too, so a clean `--field n=3` already stores the number 3 — measured before the change. Each site keeps its historical disposition toward a MALFORMED entry (four skip silently, copy hard-errors), which is why the helper classifies that case rather than deciding it. NOT YET EVIDENCE: ./internal/mcp, ./cmd/pad and ./internal/items all pass, and that green does not show the divergence closed — the three BUG-2850 pinned tests exercise the catalog conflict pass, which never reaches ingestFieldKVP. The door-level test and the re-grounding of that pass are the next commits. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR * test(cli,mcp): pin the door-parity claim at both doors (BUG-2870) Nothing in the suite asserted what the remote door STORED for a padded entry — the three BUG-2850 tests that cite its trimming all exercise the catalog conflict pass, which never reaches ingestFieldKVP. So the previous commit's green was not evidence for the thing it changed. Three files now hold the claim: internal/items pins the rule, internal/mcp pins the remote door, cmd/pad pins the CLI door, and each cites the other two. Padded key refused at both; padded value carried verbatim at both; a refusal aborts the call rather than dropping one entry, and on the CLI it happens before any request reaches the server. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR * fix(mcp): re-ground the conflict pass on the new door behaviour (BUG-2870) The pass's rules were derived from ingestFieldKVP trimming, so changing the door without changing the layer built on it would have been the same one-door lapse a level up. - parseFieldArray splits through items.SplitFieldEntry: a padded key is REFUSED before dispatch on both transports, and values are indexed RAW, because raw is now what both doors write. - Both comparison sites compare raw for the same reason. The round-19 "COMPARED TRIMMED" rule is superseded and its comment says so. - detectFieldConflicts PROPAGATES the parse refusal instead of returning nil. It swallowed it as "the caller owns this error surface", which was true when the only possible error was a shape error — reshapeItemFields returns early with no `fields` object, so on the no-`fields` path (this bug's path) nobody owned it and a padded entry turned back into a success. - A padded entry is refused in the pass rather than skipped. Skipping dropped it from conflict detection entirely, turning four existing refusals into successes. The last two were caught by the BUG-2850 tests, not by reasoning: the first shape of this commit passed a full package build and turned four guards off. Seven tests still fail. They assert the OLD door behaviour and are the specification being changed; each gets read on its own next, and is either kept because the behaviour survives or replaced by a test stating the new behaviour that cites the old name. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR * test(mcp): restate the seven BUG-2850 pins on the new rule (BUG-2870) Each was read on its own and either kept or replaced; every replacement names the test it replaces and why the old assertion was right at the time, so the deletion is traceable rather than a green that appeared. - padded value is not a conflict → IS a disagreement now that no door trims (" done" and "done" are two values), with an equal-values control leg. - padded entries still caught (hierarchy) → refused EARLIER, by the padded-key rule, before the alias pass observes both keys. The alias guard keeps its three unpadded cases, which is what stops this being a hole. - PaddedEqualDuplicateIsCanonicalized → IsRefused, plus a canonical control that still emits --field exactly once. - MixedCanonicalAndPaddedDuplicatesCollapse → Refused. The round-8 finding survives: one canonical entry still does not make its padded sibling harmless, it is refused rather than swallowed. - PaddedEntryAloneIsUntouched → IsRefused. That test pinned a DEFERRAL, in its own words "BUG-2870's business, not this PR's". This is that business. - "fields carries the key — canonicalized, so accepted" → still refused, since nothing canonicalizes now; the per-key question it defended is still tested by the two legs beside it, and a canonical control was added. - ReEmittedValueKeepsItsWhitespace → the re-emission path is gone, so it becomes a refusal test that also asserts the ADVISED form is accepted with its value untouched. The property it defended is pinned at both doors. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR * test(server): pin that a move override is typed server-side (BUG-2870) The fact the ruling turned on, and the easiest one in this unit to lose: it is invisible from cmd/pad, where moveCmd plainly sends a string. - a declared number field given the STRING "3" through field_overrides ends up as the NUMBER 3, which is why move needs the shared KEY parse and no client-side typing; - a padded " 3" is REFUSED with a 400 and the item does not move, which is the answer the remote door will now give too instead of trimming and succeeding. t.Parallel per CONVE-2086 — both build their own server through testServer, so each has its own database, limiter and bus. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR * chore(mcp): bump tool surface to 0.30 and sync the docs the guards enforce (BUG-2870) Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR * refactor(mcp): remove the canonicalization the door change made unreachable (BUG-2870) Two mechanisms existed to make a padded entry reach both doors as the same write: the nonCanonical conflict guard (round 16) and the re-emission path that rewrote a padded entry to canonical form (rounds 7/8). Both are dead now — items.SplitFieldEntry refuses a padded key, so every entry that parses satisfies `entry == key + "=" + value` BY CONSTRUCTION. Removing each changed no test. That is consistent with "dead" and with "untested" alike, so the construction argument above is what settles it — recorded in the comments that replace them, along with what the removed guard was defending and where that premise is enforced now. Rewriting a caller's key was also the behaviour this bug is about, applied by us rather than by a door: canonicalization silently changed the key the caller wrote. Refusing says so instead. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR * docs(mcp): put the trimming narrations in the past tense (BUG-2870, CONVE-23) Six comments described the old door behaviour in the present tense ("HTTP trims and writes effort"), which reads as a claim about the code as it stands. The rounds they narrate still explain why the surrounding rules exist, so they are re-tensed rather than deleted. Two references were checked and left alone because they are still true: ingestFieldKVP does still store every field value as a STRING (coerce.go's BUG-2850 note, and the github_pr hint in dispatch_http.go). This change stopped it TRIMMING, not stringifying. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR * docs: sync CLAUDE.md to tool surface 0.30 (BUG-2870) The drift guards cover instructions.md and README.md but not this file, and its own 0.27 entry records the consequence: 'This entry was missing from CLAUDE.md — the 0.27 unit swept instructions.md and README.md and not this file.' The unit that makes a version line stale is the unit that owes it. Both markers updated, and the entry states the two behaviour changes in the terms they were ruled: /mcp refuses what it silently accepted, and the swallowed parseFieldArray refusal that was landing four refusals as successes on the no-fields path. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR * fix(mcp): finish the removal, and correct a claim I made twice (BUG-2870) Codex round 1: no P1/P2, two nits, both real. 1. The re-emission removal was incomplete. `reEmitFields` and the branch that appended its entries survived with nothing populating the map, and two comments still described canonical re-emission as something this code does. Unreachable, but my own commit message had said the path was removed, so the code contradicted the claim. Removed, and the round-16/17 paragraphs that decided WHEN to canonicalize go with it — they answered a question that no longer arises. 2. "The only behaviour change is /mcp refusing what it silently accepted" is WRONG, and it was in version.go, README.md and CLAUDE.md. Every door refuses a padded key now; they were merely accepting it differently — /mcp trimmed it and wrote the declared field, the CLI stored a ghost field beside it. What is /mcp-only is the VALUE half. Corrected in all three, with the correction itself recorded in the version.go entry so the next reader sees the claim was checked rather than a sentence that quietly changed shape. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR * docs(mcp): rename the predicate to the question it asks (BUG-2870) Codex round 2: no P1/P2, three nits, all naming and prose. - `canonicalized` is renamed `coveredByFieldsObject`. Nothing canonicalizes anything any more, and the only thing that predicate ever asked was whether the `fields` object carries THIS key — it kept the old name only because the guard it used to feed had been removed a commit earlier. - parseFieldKVP's doc said invalid entries are skipped silently. True of a MALFORMED entry, false of a padded key, which now aborts the call. - Three test comments still described re-emission as live, and version.go described this door's trimming in the present tense. Nothing in these two rounds was a defect in the change itself; both rounds found prose describing a version of the code that stopped existing partway through the unit, which is the failure mode a re-grounding pass invites. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR * docs(mcp): last of the prose that outlived the code (BUG-2870) Codex round 3: no P1/P2, prose only. - the predicate's own comment still asked 'will anything canonicalize THIS key'; it asks whether the fields object carries the key, and always did; - two test comments described re-emission and trimmed comparison as current. Both tests are kept — what they pin is narrower now and still worth pinning — with the change in what they mean written down. Deliberately NOT changed: the comments and replacement-test names that cite the OLD test names. Codex reads them as stale terminology; they are the traceability the restatement commit was asked for, so a reader can find what each replacement replaced. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR * docs(items): the coercion note names what the door does now (BUG-2870) Codex round 4. The paragraph described ingestFieldKVP as doing `dst[key] = val` unconditionally. Its CLAIM — every value arrives at the server as a string — is still true and is the reason this file exists; the description of the line is not, since that door now parses through items.SplitFieldEntry. Restated so the still-true part is not carried by a sentence a reader can falsify. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR	2026-09-07 15:33:05 -04:00
xarmian	bb8ec04ef1	fix(server,store): both workspace mint doors enforce their preconditions from one place (BUG-2809) (#1268) ... handleCreateWorkspace and handleImportWorkspace mint the same thing through the same store.CreateWorkspace, and enforced preconditions in two places. Two had already diverged and been fixed one at a time, each found by a reviewer rather than by the door that lacked it: the OAuth consent grant (IDEA-2756) and the user-scoped plan limit (BUG-2793). A third was live. The shared place is internal/server/workspace_mint.go, split by WHEN a precondition can run, and the split is load-bearing rather than tidy: beginWorkspaceMint — everything that does not need the body (consent, plan limit, and the owner/source attributions). Runs before the body read, so a refused caller never uploads a bundle and a refusal cannot be probed by body shape; on the import route it sits above the Content-Type dispatch, so one line covers both body shapes. validateWorkspaceMintPayload — the payload-shaped rules. Returns an error rather than writing one, because the JSON doors answer 400 bad_request and the bundle door answers 400 bad_bundle through importStatusError. The rule is shared; the envelope stays each door's. Callers: handleCreateWorkspace, handleImportWorkspace, and importBundle. The mint context reaches the bundle path as an ARGUMENT rather than on the Server, because it is per-request state and the two things it carries are exactly what two concurrent requests would differ on. THE LIVE DEFECT. Import accepted an empty workspace name. Measured before the fix: it created a workspace with name="" and slug="", and a second such import landed on slug "-2" -- the first had taken the empty slug, globally, and a slug is a routing key. Both import doors now refuse it, checking the EFFECTIVE name (the ?name= override when given, the bundle's own otherwise) because that is what becomes the slug. A control leg covers the override, or the rule would be indistinguishable from "reject any bundle whose payload name is empty" and would break rename-on-import. SETTINGS: the item's premise was wrong and this corrects it rather than fixing it. Malformed settings never reached the store unnormalized -- createWorkspaceQ calls NormalizeWorkspaceSettings itself and refuses. What diverged was the STATUS: create answers 400, import answered 500 import_failed because handleImportWorkspace maps every store error that way. Validating in the shared payload step makes both 400. Context stays create-only: an export carries none, so applying it on import would be inventing input. SOURCE: imported workspaces got no attribution at all (BUG-1557). store.ImportWorkspace now takes a source parameter, derived by the caller from the request's auth shape exactly as create derives it -- a parameter rather than an export field, because a bundle says what the workspace WAS and where this copy is minted from is a fact about this request. The operator path (pad db migrate-to-pg) passes "": it is a copy, not a creation surface, and inventing "cli" would relabel every migrated workspace's origin. Userless callers (the inventory's fourth item) are deliberately unchanged. beginWorkspaceMint preserves the userID != "" guard exactly as both doors had it rather than changing behaviour under cover of a refactor; the measurement and the ruling are on BUG-2914. Five mutants, each verified to COMPILE first and each detected by its own leg: either import door skipping the payload check, the create door skipping it, checking the payload name instead of the effective name, and passing "" for source. Two of them initially did not compile, and go test answers a build failure with FAIL <pkg> [build failed], which in a filtered run reads exactly like detection -- a false DETECTED, the mirror of the false SURVIVED. Re-run with the orphaned variable kept alive. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR	2026-09-06 23:36:58 -04:00
xarmian	cdc5b301e2	fix(server): minting or rotating an API token requires an interactive session (BUG-2890) (#1267) ... A PAT that could reach POST /auth/tokens minted further tokens with independent names and expiries. Those survive the revocation of the token that created them, and nothing in the token list records which token minted which -- so revoking a leaked credential did not end the access it had been used to establish. Ruled on the item's trail day 57: create and rotate require session auth and refuse a PAT-authenticated call with 403 session_required; list and revoke stay PAT-reachable. Population is THREE doors, not the two the filing named. Enumerating every route that reaches store.CreateAPIToken / RotateAPIToken turns up the workspace-scoped mint at POST /workspaces/{ws}/tokens, which had only requireMinRole("owner") in front of it -- and a user-owned PAT held by an owner satisfies that. Measured, not read: with the fixture's owner also written into workspace_members, that door returned 201 with a live token on the unfixed tree. All three doors are gated; list and revoke on both the user and workspace routes are deliberately untouched, because neither extends access and revocation is the compromised-credential response. The gate is on the CREDENTIAL, not on the door: isAPITokenAuth is false for a session cookie AND for a padsess_ CLI bearer, a distinction that predates this fix (see ctxValidatedSessionBearer's note in middleware_auth.go) because a CLI session IS an interactive session. At the workspace door the check runs before requireMinRole, so a PAT-borne caller cannot learn its own membership status from the difference between the two 403s. Five mutants run, each detected by its own leg: dropping the gate at each of the three doors fails that door's test; gating on Authorization != "" instead of the credential kind fails the CLI-session leg AND NOTHING ELSE; over-applying the gate to list and revoke fails both PAT-still-works controls. The fourth is why the CLI leg exists -- a header-shaped gate passes every PAT assertion and breaks every logged-in CLI. Also documents the refusal on the CLAUDE.md route line: the routes were listed without saying anything about credential kind, so an agent holding a PAT would have met an unexplained 403. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR	2026-09-06 22:02:05 -04:00
xarmian	7659ad3cd3	feat(server,web,cli): say when a relation's copy target is unusable, instead of offering a picker that cannot answer (IDEA-2899) (#1262) ... * feat(server): the copy preflight says when a relation's target is not usable (IDEA-2899) TASK-2869 made a `needs_value` relation row collectable as soon as it names a target collection. Naming one is not having one: the slug can name a collection that has been DELETED, or one this caller cannot READ. The dialog then mounts a picker that can return nothing and, because the row is not blocked, Confirm stays disabled carrying only the generic required-field message — the user is told a value is missing and never told that no value is reachable. `collection_unavailable` on the needs_value row is the server saying so. THE CLIENT CANNOT COMPUTE THIS, which is why it belongs here. The dialog's destination collection list is filtered through `canEditCollection`, because it drives the copy-INTO picker; a relation TARGET needs only READ access, so a perfectly usable target routinely does not appear in that list. Testing against it would refuse rows the user could have filled in — over-blocking, which is the worse failure and invisible to whoever hits it. `visibleCollectionIDs` is the read-scoped view, and its NAV-LENIENT shape is right here rather than merely tolerable: it includes a collection reachable only through an item-level grant, and the question is "could a picker here return anything at all". One granted item is a picker with one row. DELETED and UNREADABLE are deliberately not distinguished. Same consequence, no client branch would differ — and separating them would tell a caller who cannot read a collection that it nonetheless exists. `omitempty` on a BOOL drops `false`, so the field is phrased NEGATIVELY. Present-and-true means the server checked and the target is unusable; ABSENT means available, or a server that does not report. A client must block only on an explicit true, so absence stays "no information" rather than becoming a value — the rule `access_epoch` follows on the item doors, and the one whose violation cost two review rounds on IDEA-2898 this morning. Costs nothing on the common path: a destination schema declaring no relation field runs no query at all. Claude-Session: https://claude.ai/code/session_01Xk9M5UVPdc84xL5E1mZkm8 * test(server): pin the type gate on collection_unavailable (IDEA-2899) Found by a surviving mutant rather than by inspection: dropping the `def.Type == "relation"` gate left every other test in the file green. Nothing stops a schema declaring `collection` on a field of another type — the validator does not police keys it has no use for — and such a field would then pick up a flag whose meaning is defined only for relations. The dialog would block a perfectly collectable `select` because some relation elsewhere in the same schema points at a collection that happens to be gone. The fixture is the discriminating one: ONE deleted collection, TWO required rows that name it, and only one of them means anything by it. Six mutants on this half, all killed: flag never set, flag always set, deleted target not flagged, unreadable target not flagged, type gate dropped, and the nil-visible-set case (an admin's "no filtering" read as "nothing visible", which would flag every target for the callers who can see everything). Claude-Session: https://claude.ai/code/session_01Xk9M5UVPdc84xL5E1mZkm8 * feat(web): block a relation whose target is unavailable, and stop advising a command that cannot work (IDEA-2899) The client half. `isCollectable` now refuses a relation row the server has flagged, so the row lands in `blockedFields`, Confirm is disabled with a reason, and no picker mounts that could only come back empty. `collection_unavailable !== true` is STRICT on purpose. The field is absent when the target is fine and absent from a server that predates it, so absence must read as "no information". (Over the domain the type admits — `boolean | undefined` — the truthiness spelling is EQUIVALENT and a mutant swapping it in survives; that is recorded in the source rather than papered over with an off-contract fixture. The strict form is kept because it states the contract where the next edit will read it, and the inverse spelling would block every row against an older server.) THE PART THAT IS NOT WIRING: the existing blocked-field notice said the field "is a required <type> field. This dialog can't collect a value for that type safely" and then printed `pad item copy … --field key=value`. Both halves are FALSE here. The type is perfectly collectable; the TARGET is gone. And the CLI runs as the same user against the same referent validation, so the command it prints is refused for exactly the reason the user is already stuck — advice that sends someone to do work that cannot succeed is worse than no advice. So the message branches on `uncollectableReason`, names the collection and the destination workspace, and the CLI line is now gated on `cliFillableField` — the first blocked row the CLI can ACTUALLY fill. `blockedFields[0]` was correct while every blocked row was type-shaped; with an unavailable relation sorted first it named the one field `--field` cannot set either. Eleven unit tests on `copyNeedsValue`, plus a source pin on the dialog whose own measured limit is in its docblock. Client mutants: 7 real, 6 killed, 1 recorded as equivalent with the domain argument that makes it equivalent. Claude-Session: https://claude.ai/code/session_01Xk9M5UVPdc84xL5E1mZkm8 * feat(cli): the copy preview marks an unavailable relation target and stops suggesting it (IDEA-2899) Caught by `TestItemCopyMirrorsMatchServerShapes`, not by me. The CLI keeps a mirror of the preflight response, and adding a field server-side without mirroring it fails that test by design — a mirror that silently lags is a mirror that lies. Working exactly as intended, and the reason this half exists at all. Mirroring the field turned out to be the smaller part. The CLI already prints `target collection: people` for a relation row, and it builds an `Add: --field owner_ref=<value>` suggestion from every unsupplied row. Both are wrong when the target is unavailable: the first sends a user looking for a ref in a collection they cannot read, and the second hands them a command the referent validation refuses for exactly the reason they are already stuck. So the target line is marked NOT AVAILABLE, and the row is excluded from the suggestion with a sentence saying why — modelled on the empty-key branch, which was written for the identical reason (a `--field =<value>` nobody can run) and is three lines away. That the same defect had to be fixed in two places is the shape worth naming: the dialog and the CLI independently built "here is how to supply it" from "here is a field needing a value", and neither had a notion of a field that CANNOT be supplied. The empty-key case was the first instance and was fixed locally; this is the second. Five mutants on this half, all killed: suppression removed, suppression applied to everything, the unavailable label dropped, the explanation dropped, and the mirror field ignored. The available-target control leg is a separate test so the omitempty contract is exercised on this surface too. Claude-Session: https://claude.ai/code/session_01Xk9M5UVPdc84xL5E1mZkm8 * fix(cli): route all three "how to supply it" sites through one predicate (IDEA-2899) Review found the fix applied at one door and not its siblings — my own recurring shape, arriving again. THREE places tell a CLI user how to resolve an unsatisfied field: the detailed `renderItemCopyNeedsValue`, the `--dry-run` summary, and the error the command returns. The first commit fixed the render. The other two went on printing `--field key=value` at someone for whom no value exists — and the ERROR is the line a script or a hurried reader actually sees, so it was the worst of the three to leave. `itemCopyUnfillable` is now the single definition all three consult. Not because three call sites are tidier than one, but because three sites independently answering "how do I supply this" is exactly how they diverged in the first place. The dry-run summary branches three ways rather than two, because the MIXED case is the one a boolean gets wrong: some fields can be supplied and some cannot, and collapsing that either suppresses advice the user needs or offers advice they cannot use. The error hint is suppressed only when NO field can be supplied — with one fillable field left, `--field key=value` is still true. Also pins the BOUNDARY the same review probed: a target collection that is live and readable but EMPTY is deliberately not flagged. The symptom looks identical — an empty picker — but the cases differ where it matters. An unavailable target is unfixable from inside the dialog, so blocking costs the user nothing they had; an empty collection is resolved by creating the item and retrying, and blocking would refuse a copy they were about to complete. It would also cost a live-visible-item count per relation target on a dry run the UI calls on every keystroke. The weaker case — an empty picker that says nothing about WHY — is filed as IDEA-2905 and belongs to the picker. Ten mutants across this round, all killed, including both directions on the error hint and both directions on the dry-run branch. Claude-Session: https://claude.ai/code/session_01Xk9M5UVPdc84xL5E1mZkm8 * fix: unfillable means EITHER reason, and a select never names a relation target (IDEA-2899) Review round 2, two findings, both real and both about a rule stated in one place and enforced in another. **"Unfillable" answered for one of two reasons.** An EMPTY KEY cannot be supplied either — `--field =value` is rejected by this command's own parser, and the detailed render has explained that since Codex round 6. Only that render knew: the --dry-run summary and the returned error went on advising `--field` for those rows, because the predicate I extracted last commit covered the relation reason alone. A predicate named "unfillable" that answers for half its name is a worse trap than no predicate — right at the site that defined it, wrong everywhere it was reused, which is precisely what extracting it was meant to prevent. Two functions now: `itemCopyUnfillable` (either reason — advice), and `itemCopyUnavailableTarget` (the relation half — the render's own sentence, since the two explanations are not interchangeable to a reader). `itemCopyUnavailableTarget` deliberately does NOT also exclude empty keys, though my first version did. A row can carry both faults, and a mutant removing that exclusion survived every test — correctly, because all it changes is printing two sentences that are both TRUE about such a row. The guard was tidiness dressed as a rule; a condition nothing can distinguish is one the next reader has to re-derive. **`Collection` was emitted for non-relation fields**, while its own doc said it is empty for every other type. That was a claim about the schemas people write, not a property of the code: a `select` carrying `"collection": "people"` is storable — field validation has no use for the key and does not police it — and the value was copied straight through, so the CLI printed "target collection: people" beneath a select. A relation fact asserted about a field that has none. `relationTargetSlug` makes the documented contract true at the only place that can make it true; my own type-gate test had created exactly that shape and asserted only the FLAG, not the slug. Three mutants on these fixes, all killed. Claude-Session: https://claude.ai/code/session_01Xk9M5UVPdc84xL5E1mZkm8 * fix(cli): the explanation now names the reason that actually applies (IDEA-2899) Review round 3, and the sharpest miss of this unit — my own, one commit old. Broadening what a predicate ACTS on silently broadened what a sentence SAYS. Once `itemCopyUnfillable` counted empty keys as well as unavailable relation targets, a set of empty-key rows selected the all-unfillable branch and was explained as "the relation target is not available to you" — a false statement about rows that contain no relation at all. Same in the returned error, which is the line a script sees. The tell was there to be read: a sentence that was TRUE while the predicate was narrower is a sentence to re-read the moment it widens. I broadened the predicate deliberately, wrote a commit message about how a half-answering predicate is a trap, and left the sentence describing the half. `itemCopyUnfillableWhy` names the reasons actually present — relation targets, empty keys, or both — and the two one-sentence sites consult it. The detailed render is unchanged: it explains each reason where the row is printed, which is why it uses the narrower count. Four mutants, all killed, including the two that matter: the explanation always saying "relation" (the defect) and never saying it (the same defect pointing the other way). The test carries a mixed-reason leg, because a sentence that picks one of two true reasons is the failure a single-reason fixture cannot see. Also corrected: three comments claiming `itemCopyUnfillable` is relation-only or that the detailed render consults it. Both stopped being true last commit. Claude-Session: https://claude.ai/code/session_01Xk9M5UVPdc84xL5E1mZkm8 * fix: one row can carry both faults, and four docs said this was simpler than it is (IDEA-2899) Review round 4. Four findings, no P1s, and the first is the one worth the round. **A `continue` between the two counts.** `itemCopyUnfillableWhy` counted a row as an unavailable relation target and then skipped the empty-key check, so ONE row carrying both faults reported only the first. My mixed-case test used TWO rows with one fault each — a different input, and the only one it exercised. Two rows with one fault each and one row with two are not the same fixture, and I built the weaker one while writing a commit message about fixtures that cannot discriminate. **The dialog could still print `--field =value`.** `cliFillableField` excluded unavailable relation targets and not empty keys, so a required `json` field the destination reported with no key was type-shaped, blocked, and still offered a command the CLI's own parser rejects. The CLI has refused those since Codex round 6; the web side had never learned it. Same defect, other surface — which is the third time this unit has fixed one door and not its sibling. **Cardinality.** "no --field can supply it" for several fields, and "reported them with an empty key" for one. Both sites now agree with their counts, and the empty-key phrase is neutral on number so it reads correctly after either. **Four documents claimed every needs_value row is resolvable with an override** — the CLI renderer's docblock, the server's `NeedsValue` field, the CLI mirror type, and the dialog's collectability comment. That was true when each was written and this unit falsified all four; a reader following any of them would conclude the CLI had simply forgotten to print a flag. Two mutants on the fixes, both killed: the `continue` restored, and the dialog's empty-key exclusion removed. Claude-Session: https://claude.ai/code/session_01Xk9M5UVPdc84xL5E1mZkm8 * refactor(cli): one tally, because the rounds said the branching was the problem (IDEA-2899) Four review rounds returned 2, 2, 2 and 4 findings. The counts looked like slow convergence; the DISTRIBUTION was the finding. Every defect after round 1 lived in this one layer — how the CLI and the dialog say "here is how to supply it" — while the server half that computes availability stayed clean throughout. The layer had accreted exactly the way IDEA-2898's cold path did: a count, then a second count for the other reason, then a phrase function, then a `continue` between two counters that made a dual-fault row report half of itself. Round 4 fixed something round 3 introduced to fix something round 2 introduced. That is not a run of bad luck, it is a shape. So this round removes branches instead of adding a seventh guard. `itemCopyTally` walks the rows once and returns what every caller needs; `AllUnfillable()` is the condition both one-sentence sites test, and `Why()` is the phrase both interpolate. Three helpers become one type. There is no second definition of "unfillable" to drift from the first, and no sentence describing a subset of what a predicate counts, because the sentence and the count come from the same walk. `Unfillable` is deliberately NOT `UnavailableTarget + EmptyKey`: one row can carry both, and double-counting makes `Unfillable == Total` false for a set that is entirely unfillable — the comparison every caller makes. A mutant does the addition and dies. Five mutants, all killed. The last needed a new test rather than a new fixture: `AllUnfillable`'s `Total > 0` guard is unreachable from both current callers, so a mutant removing it survived every command-level test. Keeping an unreachable guard and calling it defence is how a promise becomes a lie, so the tally is now unit-tested directly — an empty set is not "entirely unfillable", and a future caller outside the `len() > 0` gate would otherwise be told silently that nothing can be supplied. Claude-Session: https://claude.ai/code/session_01Xk9M5UVPdc84xL5E1mZkm8	2026-09-06 15:48:22 -04:00
xarmian	00d650a861	feat(server,web): detect a revocation that writes no item, and evict the cache it left stale (IDEA-2898) (#1261) ... * feat(server): fingerprint the caller's visible set on the item doors (IDEA-2898) The delta stream can only express changes to ROWS. A revocation that writes no item — the ordinary shape of revocation — therefore produces no signal at all, and a client's warm local index goes on serving rows for a collection the caller can no longer see. `ItemPicker` lists those titles. `computeAccessEpoch` hashes the caller's EFFECTIVE visible set (collection ids + item grants, canonicalised by sorting, separated by a byte no id can contain) and `/items-index`, `/items-changes` and the 60s SSE tick all carry it. The value is opaque, derived only from the caller's own access, and costs no extra query — both lists are already resolved before the response is written. ONE DEFINITION, deliberately. The delta door computes its epoch from the LIVE grant set rather than from its own include-deleted query set: a caller holding a grant on a soft-deleted item would otherwise get a different epoch from each door, forever, and the client would resync to its page cap on every poll. `handlers_items_access_epoch_test.go` pins the two doors against each other on exactly that fixture — a grant on a soft-deleted item is the one input that discriminates, and an earlier version of this test agreed for the wrong reason because its fixture had no grants at all. The unrestricted caller gets a SENTINEL ("all") rather than a hash of the empty set, because the empty set is a real and opposite state: a restricted member with zero visible collections. Hashing both the same would make the widest and narrowest access indistinguishable. Claude-Session: https://claude.ai/code/session_01Xk9M5UVPdc84xL5E1mZkm8 * feat(web): compare the access fingerprint and evict the cache when it changes (IDEA-2898) The client half of the signal. `WorkspaceState.accessEpoch` is the baseline the cache was built under; it is persisted beside the cursor because the revocation this closes can land while the tab is CLOSED, and a cache that forgot its scope on reload could not detect that at all. `ensureAccessScope` is the comparison, and it is ONE function with two call sites — the page's `/items-changes` poll and bootstrap's own reconcile loop. The first draft inlined it in the loop and had already diverged: the inline copy skipped the null-baseline case, which is precisely the offline-revocation cache the change exists for. Three properties the tests pin, each of which was wrong at some point: - ABSENCE IS NOT A VALUE. A server that does not send the field is a mid-deploy older build, not a changed scope; and a snapshot carrying no epoch must not erase a baseline we already know, which sent the reconcile loop resyncing to its 50-page cap. - THE RESYNC MUST TERMINATE. When the snapshot carries no epoch the baseline would be unchanged and the next poll would ask again forever, so the epoch we were TOLD goes in as the fallback — and again after the await, for the case where the resync was JOINED rather than started and the fallback was never seen. - RAM AND DISK AGREE. `persistReplace` runs inside the resync, so the baseline is set before it, not patched after: a durable meta row a version behind the in-memory one makes the next warm boot resync for a scope that never changed. SCOPE, stated plainly because it is a real limitation and not a rounding error: this closes the single-tab case and the offline case. A write from a tab that has not yet learned the new scope can still reinsert a row into the durable cache while the cache advertises the current epoch, and nothing re-fires until the next access change. That is F2, and PLAN-2903 owns it together with the cross-tab cache-coherence work. Strictly better than the pre-change state, where no revocation without a row change was detected at all. `LOCAL_INDEX_SCHEMA_VERSION` 3 -> 4: a cache written before this has no baseline, and adopting the incoming epoch for it would be exactly the silent adopt the change exists to prevent. Two fixtures that hard-coded 3 now read the constant, so the next bump does not turn them into stale-cache fixtures by accident. Claude-Session: https://claude.ai/code/session_01Xk9M5UVPdc84xL5E1mZkm8 * test(web): close the four seams a mutation run on the reduced tip found open (IDEA-2898) The gates that ran on the full branch describe a different tree. A fresh 25-mutant run scoped to what this branch actually ships found four seams with no coverage, three of them the same shape: the baseline being ADOPTED is untested everywhere the adoption is silent. That shape is why they survived a suite written for the eviction. Dropping an adopt line does not stop a revocation being detected — the cache simply has no baseline, and a null baseline over a populated cache resyncs, which LOOKS like the change working. The discriminating case is the QUIET one: an unchanged scope must cost nothing. All three new tests assert the absence of a resync. - warm hydrate adopting the PERSISTED epoch (the offline-revocation case; needs the mocked persistence module, since jsdom has no IndexedDB and the warm branch is otherwise unreachable) - the cold snapshot's epoch becoming the first-ever baseline - `applyDelta` handing `persistDelta` the baseline its rows were applied under, so the durable meta row is not stamped null after every delta The fourth is the collection route's `ensureAccessScope` call, whose source pin moved to PLAN-2903 with the pairing-guard argument it also covered. Half of what it pinned still ships here, and the mutation run proved that half uncovered — so the pin comes back NARROWED to the one surviving call site. Its measured limit is in its docblock rather than assumed: deleting the call kills it; short-circuiting the call (`if (false && await ...)`) SURVIVES, because the text it matches is still on the line. A source pin cannot see reachability. That survivor is reported, not hidden. Instrument corrections made before any of this counted, both caught by the runner's own controls rather than by inspection: - `go vet` was in the Go build gate. It flags unreachable code, so the positive control — an early `return` — scored BUILD-FAIL while compiling perfectly. A vet failure is not a build failure. - the web runs passed `--reporter=basic`, which this vitest does not have. Every web mutant failed to START and the classifier read that as a verdict. Fixed, and the classifier now requires the FULL baseline population (50 tests across 7 files) to have run before it will call anything killed — a mutant that stops a file loading also prints a failing summary. Claude-Session: https://claude.ai/code/session_01Xk9M5UVPdc84xL5E1mZkm8 * fix(web): an empty RAM state is not an empty cache, and a joined resync owes a durable epoch (IDEA-2898) Two real defects from the review round on the reduced tip. Both are the SILENT ADOPT class the change exists to prevent, arriving in the two places the change itself created. **A cache that has not answered yet is not an empty cache.** `bootstrap` awaits `hydrate` before merging anything, but the SSE-driven `deltaSync` runs on its own subscription rather than behind that await — so it can reach `ensureAccessScope` with RAM empty and IDB holding rows from a scope nobody has checked. The "nothing to evict, adopt silently" branch then stamps the new epoch onto the durable cache through the delta that follows, and the stale row hydrates under an epoch that agrees with the server FOREVER. Not merely wrong once: permanently inert, which is worse than the defect this change closes. `cacheRead` makes the distinction the code was eliding. Declining to adopt costs nothing and fails in the safe direction — the delta persists a null epoch, and a null baseline over a populated cache resyncs on the next reconcile. **A joined resync updates RAM and leaves the disk behind.** Resyncs are deduplicated per workspace. When `ensureAccessScope` JOINS one, that resync already ran its own `persistReplace` under its own baseline, so assigning the told epoch afterwards leaves the meta row recording the old one. The session converges and every RELOAD hydrates the stale baseline and pays a full resync for a scope that has not changed since. The code's own comment claimed the epoch "lands in RAM and IDB together", which was true of the started path and false of the joined one three lines below it. `persistAccessEpoch` repairs just the epoch on an existing cache, on BOTH branches — the first version of the fix had it on one, and no test in the file could tell them apart until a mutant did. Two tests changed because a PROPERTY changed, said out loud rather than quietly rewritten: "adopts silently when there is no baseline and nothing cached" held for a workspace that had never been bootstrapped, and no longer does. It is now "…and the cache is known empty", and the unbootstrapped case is its own test asserting the opposite. The `seedUnder` helper acquires its baseline through a cold bootstrap, which is how a real session gets one anyway. Mutation matrix re-run whole on this tip: 31 real mutants, 30 killed. The survivor is the page pin's short-circuit case, documented in its docblock. Six of the mutants target these two fixes; one of them (`persistAccessEpoch` minting a meta row) had to be rewritten after it survived for the wrong reason — the naive version put a keyless row that IDB rejects, so the error path compensated for the defect and the test never had to. Claude-Session: https://claude.ai/code/session_01Xk9M5UVPdc84xL5E1mZkm8 * fix(web): a failed cache read is not an empty cache, and the epoch patch is a compare-and-set (IDEA-2898) Round 2 on the reduced tip, and both findings are the previous round's fixes being not quite finished. **`hydrate` returns the same empty payload for a FAILURE as for an empty cache** — deliberately, since a best-effort cache should not take the app down. `cacheRead` was set unconditionally after the await, so a transient IDB failure read as "there is nothing stored" and re-opened the exact silent adopt round 1 closed: the durable cache may hold rows from a scope nobody checked, and adopting stamps the new epoch onto them through the next delta. `HydrateResult.durableRead` carries the difference the payload cannot. True when the read succeeded, true when IndexedDB is unsupported (nothing durable can contradict anything later), true when the read found an incompatible cache and wiped it — false only when a database that might hold rows could not be opened or read. Failing that way costs a resync and hides nothing. **`persistAccessEpoch` was a blind read-modify-write on a row other writers own.** IDB serializes transactions, but a `persistDelta` or `persistReplace` carrying a NEWER epoch can commit between the resync this caller joined and the patch — and the overwrite would then stamp the older epoch onto rows fetched under the newer one, so the next comparison reports a change that never happened and pays a full resync for it. Now a compare-and-set against the epoch the caller believes it is repairing. Both hydrate failure paths are covered, and they needed different fixtures: a database at a HIGHER format version (open fails outright) and one whose `items` store is missing (the open succeeds, the read throws). They are one statement written twice, and a mutant on either is invisible to a test of the other — which is how the second one was found, by a mutant surviving a test written for the first. The mutation matrix is 36 real mutants on this tip, 35 killed; the survivor remains the page pin's short-circuit case, documented in that file. One mutant is worth naming because it survived for the WRONG reason twice: the naive removal of `persistAccessEpoch`'s "no meta row" guard cannot be detected, because without it the code dereferences `undefined` and the catch swallows the throw, so the outcome is identical. The faithful version — a guard replaced by code that actually mints a valid row — is killed. A mutant has to be the defect, not a crash that happens to look like it. Also corrected: `accessEpoch`'s doc claimed null exists only before the first response of a session. Two things falsify that now — a server that sends no epoch, and the pre-hydration guard. Claude-Session: https://claude.ai/code/session_01Xk9M5UVPdc84xL5E1mZkm8 * docs,test(web): the epoch patch's expected value is asserted, and null is not unreachable (IDEA-2898) Round 3, two P3s and no behavioural findings. The `HydrateResult.accessEpoch` doc said the version bump to 4 made a null epoch unreachable for caches this build writes. It does not: a server that predates `access_epoch` omits it, and `persistDelta`/`persistReplace` record that absence honestly rather than inventing a value — a case the tests already cover. What the bump actually rules out is a PRE-IDEA-2898 cache being READ as though it had a baseline. Comment corrected to say the thing that is true. The two joined-resync tests asserted the epoch `persistAccessEpoch` is given but not the value it expects to be REPLACING. The patch is a compare-and-set, so a wrong `expectedPrevious` makes it a silent no-op that still looks right in RAM, and the disagreement surfaces only as a resync on the next reload — invisible to a single-session test. Both call sites now have their expected value asserted, and two mutants (each call site given the wrong expectation) are killed by them. Claude-Session: https://claude.ai/code/session_01Xk9M5UVPdc84xL5E1mZkm8	2026-09-06 13:36:22 -04:00
xarmian	07b2e439f2	TASK-2869 (U2b): the preflight names a relation's target collection, and the copy dialog scopes its picker to the destination (#1258) ... * feat: the preflight's needs-value row names its relation target, and the copy dialog scopes its picker to the destination (TASK-2869) U2b, per the day-55 ruling. Both blockers (U1 referent validation, U2 the FieldEditor branch + picker) are in. THE DEFECT. `ItemCopyPreflightNeedsValue` carried `type: "relation"` and no target. FieldEditor gates its relation branch on `wsSlug` AND `field.collection`, so a required relation in the destination reached the copy dialog as a field it knew was a relation with no idea what to point at, and rendered as FREE TEXT. Before U1 the copy stored whatever was typed. SERVER. `collection` is added to the needs-value row, populated from the DESTINATION schema's `def.Collection` — the only place it is known, since the row is built from that schema. Additive and `omitempty`: a client that does not read it is unaffected, and a row for a non-relation field is byte-identical to before. Not a wire-version question, for the same reason `models.ItemWriteWarnings` was not. CLIENT. `toFieldDef` carries the collection through, and the FieldEditor call passes `wsSlug={destWs}` — the DESTINATION, never the source. A relation resolves at the destination, so the picker must list items the copy can actually point at; that is same-workspace resolution AT the destination, not the cross-workspace case PLAN-2857 rules out. `relation` becomes collectable ONLY IF THE ROW NAMES ITS TARGET. Without a collection, FieldEditor's gate renders the non-editable state, so offering the row would produce a control that cannot be filled and a Confirm that cannot be satisfied. Such a row now lands in the blocked list and the user is told which field and why — the same disposition `multi_select` gets, for the same reason: a control that silently cannot do its job is worse than an honest refusal. TWO THINGS THIS UNIT TAUGHT ME THAT ARE NOT IN THE RULING. 1. THE WEB MUTANT SURVIVED, AND THAT IS WHY `isCollectable` MOVED. My first version left the predicate inline in `CopyItemDialog.svelte`. Making `relation` unconditionally collectable — the exact defect the negative leg of the proving test is about — passed EVERY suite in the repo. That is IDEA-2894's lesson arriving one unit later in the same file, so the predicate now lives in `$lib/items/copyNeedsValue` with tests. Two mutants die there: relation-always-collectable, and `multi_select` slipped into the collectable set. The Go half was pinned from the start (drop `Collection: def.Collection` -> FAIL naming the empty value and the expected slug). Only the client half was unpinned, and only because of where the code lived. 2. A U2-ERA TEST ASSERTED THE ABSENCE THIS UNIT CLOSES, and asserted it CORRECTLY. `fieldEditorRelationCallers.test.ts` required that the dialog pass no `wsSlug` and build its FieldDef from a shape with no `collection` — which was the behaviour, and withholding `wsSlug` was what kept an unscoped picker out. It also named this task by ref and told its successor to revisit the gate WITH the change rather than let it drift. Inverted here: the block now asserts the destination slug is passed, that the collection reaches the FieldDef, and — the half that is easy to lose — that the dialog still DELEGATES the collectability decision, so a future inlined predicate would pass the unit tests and fail this. Worth keeping: a test that pins a temporary absence should name what would make it wrong. This one did, and that is the only reason its inversion was a five-minute job instead of an argument about whether it was load-bearing. Gates: `internal/server` ok 158.335s, `go vet` and `gofmt` clean, `npm run check` 0 errors (6 pre-existing warnings), `make web-test` 127 files / 2123 tests. Postgres and CI are owed on this tip. Claude-Session: https://claude.ai/code/session_01Xk9M5UVPdc84xL5E1mZkm8 * fix(cli): mirror the needs-value collection, and name the relation target in the CLI (TASK-2869) TWO CONSUMERS I DID NOT SWEEP. The previous commit added `collection` to the preflight's needs-value row and updated the server struct and the TypeScript type — "both sides", as its own message put it. There are THREE sides. `internal/cli` keeps a mirror of the preflight shape and `TestItemCopyMirrorsMatchServerShapes` requires it to match the server field for field. It failed in the Postgres gate, in a package the change did not touch. That is the third time this session a producer change broke a consumer I had not enumerated, and the shape is always the same: I name the surfaces I edited and call that the population. The instrument that would have caught it is not "run more tests" but "grep for the type's name before claiming the sweep is done" — `ItemCopyPreflightNeedsValue` appears in exactly three files and I looked at two. Mirrored, with a comment saying why it exists: a mirror that silently lags is a mirror that lies, and the CLI renders these rows. AND THE CLI NOW NAMES THE TARGET COLLECTION, which is the point of the unit on the surface that has no picker at all. A row reading owner_ref (Owner, relation) required — required, with no value… tells a user a value is needed and nothing about what kind of value exists. The dialog answers that with a scoped picker; the CLI had no answer. It now prints the relation analogue of the `options:` line a select already gets: owner_ref (Owner, relation) required — … target collection: people Test asserts the line appears for the relation row, appears EXACTLY ONCE with a select row rendered alongside — so it cannot pass by printing unconditionally — and that the select's own `options:` line still renders, so this did not displace it. Gates: `internal/cli` and `cmd/pad` green, build and gofmt clean. The full Postgres run and CI are owed on this tip; the earlier PG run is the one that caught the mirror and is superseded. Claude-Session: https://claude.ai/code/session_01Xk9M5UVPdc84xL5E1mZkm8 * fix(web): the relation picker searches the preflight's canonical destination slug (TASK-2869) Codex review, finding 3 of 3, and the only one of the three that belongs in this unit. `wsSlug={destWs}` handed `FieldEditor` a value that is NOT always a workspace slug. An item can be opened through a workspace-UUID URL, the route parameter is passed straight through as `sourceWsSlug`, and a same-workspace copy then puts that UUID in `destWs`. `/search` resolves a workspace by SLUG only, so a picker handed a UUID searches nothing and returns no results — a control that looks usable, is not, and says nothing about why. Now `pickerWsSlug`, which is the preflight response's own `destination.workspace_slug`. The preflight IS the canonicalising round-trip: the server resolved whatever it was given and answered with the real slug. Falls back to `destWs` only before the first preflight returns, at which point no needs-value row is rendered anyway. The caller test asserts the prop AND the derivation, because asserting only the prop would pass against a `pickerWsSlug` that was just `destWs` renamed. THE OTHER TWO FINDINGS ARE REAL AND ARE FILED, NOT FIXED HERE. IDEA-2898 — `ItemPicker` serves warm local-index results without re-authorising them, so a collection whose access was revoked can still be listed. The cold `/search` path is visibility-filtered and correct; the warm path is not. This is PRE-EXISTING and applies to every caller of the picker, `ItemDetail` included — last touched by TASK-2877, not by this unit. U2b widened the exposure by adding a caller; it did not create the defect, and rewriting the picker's cache-authorisation model inside a feature branch would be an unrelated change riding along. The fix needs a decision about where the client learns its access set from, which no current signal provides. IDEA-2899 — a relation row can name a target collection that is DELETED or UNREADABLE, so `isCollectable` says yes on a non-empty string and the user meets a picker with nothing in it. I could not fix this correctly here, and the reason is measured rather than assumed: the obvious test is to check the target against `destCollections`, and that list is filtered by `canEditCollection` — it is what the user may copy INTO. A relation TARGET needs only READ access, so a perfectly usable target routinely is not in it. Using it would OVER-BLOCK, refusing rows the user could have filled, which is a worse failure than the one being fixed and invisible to whoever hits it. The right shape is probably the server reporting the target's availability on the row it already builds — an additive field on the same row this unit just changed, worth doing deliberately rather than bolted on at the end of a branch. Gates: `npm run check` 0 errors (6 pre-existing warnings), `make web-test` 127 files / 2123 tests. Postgres is running on this tip; CI is owed. Claude-Session: https://claude.ai/code/session_01Xk9M5UVPdc84xL5E1mZkm8	2026-09-05 21:37:44 -04:00
xarmian	47d2b15e10	feat(store,server): make collection trait uniqueness a database invariant — de-dup pass + partial unique indexes (TASK-2710) (#1257) ... * feat(store): make one-collection-per-trait a database invariant (TASK-2710) Partial unique indexes on both drivers over the artifact_kind and invocation_field declarations, with the de-duplication pass that has to precede them. The de-dup is Go, not SQL, and runs BEFORE migrate(). The ruling requires every resolution to be REPORTED, because it silently changes which collection owns a kernel behaviour, and a SQL migration cannot log — Postgres RAISE NOTICE goes nowhere here and SQLite has no equivalent. Splitting decide-in-SQL from report-in-Go would give one rule two spellings to keep in step, which is the defect class IDEA-2883 closed. It runs before migrate() because the CREATE statements fail on exactly the databases needing repair. The rule, after three proposals each retired by a measurement: most user-written items wins; ties break on lowest (created_at, id), reported as ARBITRARY rather than as age, because created_at is second-resolution and newID() is a random uuid v4, so same-second rows carry no age at all. The loser keeps every item and loses only the declaration. Routing MAY change on affected deployments; there is no current behaviour to preserve, since with two declarations live the winner was measured flipping between runs on Postgres. SeedCollectionsFromTemplate now skips a definition whose artifact kind is already declared. Renaming re-slugs, so a workspace whose conventions became house-rules looked slug-empty while its kind was still claimed; seeding used to mint the duplicate and would now fail the whole seed instead. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR * test(server): keep the shadowing tests under the new trait invariant (TASK-2710) TestResolvePlaybookIgnoresInvisibleCollections and TestCollectionIDForKindIgnoresInvisibleCollections build two collections declaring one trait, which the partial unique indexes now forbid. They are not obsolete and I did not weaken them. They guard the round-2 shadowing fix: when two collections declare one kind and the first-sorting one is invisible to the caller, resolution must return the visible one instead of failing. TASK-2710 makes that state unrepresentable going forward and repairs it at startup on databases holding it, but the resolver is what stands between a legacy database and a wrong answer in the window before that repair, and on any deployment where an operator dropped the index. So the fixture now constructs the state the way it exists in the wild — with the constraint suspended — and the assertions are untouched. Same reasoning as IDEA-2883's disagreeing-reminder fixture. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR * test(store): SuspendTraitUniquenessForTesting beside its neighbour, with a restore (TASK-2710) Lead ruling: shape it like SetBcryptCostForTesting — same file, ForTesting suffix, returns a restore the caller defers, so a suspended constraint cannot outlive the test that suspended it. The restore recreates the indexes from the SHIPPED migration text rather than a hand-copied approximation, which would drift and then attest to an index the product does not have. Its failure is information, not noise: recreating a unique index while a duplicate is live is exactly what the migration would hit. The de-dup tests therefore assert the restore SUCCEEDS, which is the migration's precondition checked rather than assumed; the server shadowing tests leave the duplicate live for their whole duration and ignore the error explicitly rather than by omission. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR * fix(store): de-dup uses the index's own extraction, strips every declaration once, counts NULL source as user-written (TASK-2710) Three P1s from codex round 1, each verified before accepting. A collection can lose BOTH declarations — the playbooks definition declares artifact_kind and invocation_field — and resolving them in two passes, each re-parsing the row's ORIGINAL traits, made the second write restore what the first stripped. The duplicate survived and the migration would still have failed on it, surfacing as a broken upgrade rather than a test. Now one write per collection accumulating every strip, with a regression test. Detection asked Go what a declaration is while the indexes ask json_extract / ->>, so the two could disagree: a row the Go parser rejects still carries a value the index sees, and the de-dup would leave a pair the CREATE then refuses. It now asks the database the same question the index asks, which is the same one-rule-one-spelling reasoning that put the report in Go. source IS NULL now counts as user-written. The column is nullable and legacy rows predate it; 'source <> template' alone is NULL for those, which SQL treats as not-true, so a workspace whose only user content is old would have had it ignored when picking the winner. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR * fix(server,store): seed skips either held declaration; unique violations map to 409 on both drivers (TASK-2710) Two P2s from codex round 1. The seeder checked only artifact_kind, but the playbooks definition also declares invocation_field and TASK-2710 adds an index for each — so a workspace whose invocation-routing collection had been renamed would still have failed its seed. It now skips when EITHER declaration is held, and says which. Collection create recognised only SQLite's "UNIQUE constraint" text, so the identical race answered 409 on SQLite and 500 on Postgres; the update path recognised neither. Both now use one named isUniqueViolation covering both drivers, matching what every item handler already did. The conflict message also distinguishes the indexes: "a collection with this name already exists" is actively misleading for a trait conflict, where the name is fine and the declaration is taken — a user told to rename would rename forever. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR * chore(store): drop the unused order slice from the de-dup pass (TASK-2710) Leftover scaffolding from the one-write-per-collection rewrite; staticcheck caught it (SA4010). Mine to catch earlier — I ran lint at the tip BEFORE that rewrite and not after it, so the gate found what a re-run would have. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR * fix(store): de-duplicate a conflicting archive on import; guard json_extract against malformed traits (TASK-2710) Both from codex round 2, both real. Import warned about a duplicate declaration and inserted both, which was right while nothing forbade the pair. With the unique indexes the second INSERT is refused, the whole transaction rolls back, and the workspace minted beforehand survives as a husk (BUG-2892) — so an archive carrying a duplicate would become unimportable, and those archives are exactly the ones this release repairs. This is the task's item 4, which I had not done. The first declaring collection in bundle order keeps it and later ones are stripped and reported; the rule cannot use user-item counts here because items are inserted after collections, so bundle order IS the terminator and the log says so. SQLite's json_extract RAISES on malformed JSON rather than returning NULL, so the unguarded expressions in the index predicates and the de-dup scan would have failed STARTUP on any database holding one bad blob. json_valid now guards both, matching what every other reader does with malformed traits — treat the row as declaring nothing. Postgres needs no equivalent: traits is JSONB, so the column type makes malformed content unrepresentable at rest. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR * test(store): assert the malformed-traits asymmetry per driver (TASK-2710) My own Postgres gate caught this: the test planted a malformed traits blob and Postgres refused it — invalid input syntax for type json — because traits is JSONB there. That refusal IS the reason migration 064 carries no json_valid guard while 087 does, and it was prose in the migration until the gate turned it into an observation. The test now asserts it per driver: on Postgres the plant must be REFUSED, on SQLite it must succeed and the guard must keep both the de-dup pass and index creation working. It therefore also catches someone 'fixing' the asymmetry later — adding a guard Postgres does not need, or dropping the one SQLite does. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR * fix(store,server): de-duplicate on the bytes being written; stop claiming a name conflict for item-index violations (TASK-2710) Round 3, two findings. P1, a regression I introduced: the import de-duplication pre-computed its strips from the traits the BUNDLE carries, which is not what gets written — coercion, validation-discard and canonical inference all run afterwards. A pre-traits archive carrying conventions with an empty blob has its declaration INFERRED from the slug (BUG-2702), so a bundle pairing that with an explicit declarer showed the pre-pass one declaration and the database two, and the index aborted the whole import. The check now runs immediately before the INSERT, on the final bytes, which turns the question from 'what did the file say' into 'what am I about to write'. Reproduced first, then fixed. P2: a collection UPDATE can migrate item field values, and an item-level unique index can fail there — invocation_slug is the live example. My catch-all reported that as 'a collection with this name already exists', sending the caller to rename something that is not the problem. The name message now requires the error to name the collections table; otherwise it says what it knows. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR * test(store): fail when an archived collection takes the live one's declaration (TASK-2710) The test for the rebase onto BUG-2884, written before the fix and failing against the naive resolution: live collections declaring convention = [], want exactly [conventions] BUG-2884 made the bundle carry soft-deleted collections. This branch moved import's duplicate-declaration check out of a pre-pass and into the insert loop, so it operates on the bytes actually being written (round 3's P1) — but `dropDuplicateImportDeclarations` has no notion of liveness, which the pre-pass had gained on main. An archived collection travelling ahead of the live one that replaced it therefore CLAIMS the kind, and the live collection is stripped of it. Every resolver filters deleted_at IS NULL, so the workspace imports with no live convention routing at all. The second assertion is the other direction: the archived collection must KEEP its declaration. Nothing routes to it, both partial unique indexes exclude it, and stripping it would edit data the operator archived rather than deleted. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR * fix(store): an archived collection neither takes nor loses a trait declaration on import (TASK-2710) The rebase fix for the test in the previous commit. dropDuplicateImportDeclarations now returns an archived collection's traits untouched. A soft-deleted row sits outside both partial unique indexes (each carries `AND deleted_at IS NULL`) and outside every trait resolver, so it can neither create the conflict this function prevents nor be harmed by holding a stale declaration. Letting it take a claim was the real damage: the live collection later in the bundle lost the declaration and the workspace imported with no routing for that kind at all. BUG-2884's pre-pass had grown the same condition; this branch replaced that pre-pass with an in-loop check on the final bytes (round 3's P1) and the condition did not come with it. Keeping both is what the rebase owes. TestImportRoutingIgnoresSoftDeletedCollections builds its fixture in a new order — declare, archive, then seed — because the unique index refuses two LIVE collections declaring one kind. The order is not a workaround: it is the production path that mints this state (delete the conventions collection, seed again), every step legal under the invariant, and it needs no test-only suspension of the constraint. Its assertions are unchanged. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR * docs(server): checkTraitConflicts no longer claims the index it now has (TASK-2710) CONVE-23 sweep. The doc comment describing that gate was written when the invariant did not exist and this branch falsified three of its sentences: "workspace IMPORT bypasses it entirely by design" (import de-duplicates on the way in now), "the database-level version is deliberately NOT added in phase 0" (migration 087 adds it), and the closing paragraph handing duplicates back to the resolvers' order-dependent behaviour. Rewritten to say what the division of labour actually is — the pre-check survives for the MESSAGE, because a unique violation is a 409 about a name unless something tells the handler otherwise and "rename your collection" is useless when the name is fine and the declaration is taken; the index is what holds. It also states the two things the invariant genuinely does not cover: import (which de-duplicates rather than refusing a restore) and archived collections (outside both indexes and every resolver, so a soft-deleted row may hold a declaration a live one also holds). Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR * fix(store): take the pre-migration snapshot before the trait repair writes (TASK-2710) Codex round 5, P2, verified. dedupeTraitDeclarations ran from the two constructors, ahead of migrate() — and snapshotBeforeMigrate() runs INSIDE migrate(). The repair changes data: it strips a declaration, moving which collection owns a kernel behavior. So the altered ownership was already committed when the snapshot was copied, and `<db>.pre-<version>` — the operator's rollback for a bad upgrade — contained it. Restoring after a failed migration handed back the old schema with the repair applied and unrecorded: the one thing the rollback could not undo was the only thing that had silently changed routing. The call moves into migrate(), immediately after the snapshot and before the migration loop, which satisfies both constraints at once — 087 / 064 still cannot run against a database holding duplicates, and the snapshot now precedes the write. The Postgres path takes the same position for symmetry; there is no snapshot there to sit after, so the ordering argument is one-sided on that dialect and the comment says so. TestTheSnapshotIsTakenBeforeTheRepairWrites pins it end to end: plant a duplicate, un-apply 087 so a migration is genuinely pending, reopen, then read the snapshot with the RAW driver — New() would migrate and repair the snapshot too, destroying the thing being measured — and assert it still holds both declarations while the live database holds one. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR	2026-09-05 20:43:43 -04:00
xarmian	47cc106ab8	IDEA-2893 + IDEA-2894: record the accepted carry disclosure, and make the drop-reason mapper testable (#1254) ... * refactor(web): extract the copy dialog's drop-reason mapper so it can be tested (IDEA-2894) The mapping from a server drop reason to the sentence a user reads lived inline in `CopyItemDialog.svelte`, unexported, with no test file for the component at all. Two separate review rounds found defects in it and NEITHER FIX WAS PINNED BY ANYTHING: - round 12: the UI asserted NON-EXISTENCE from `not_found`, which the server also emits for a target the caller merely cannot see. Telling those apart is the existence oracle the collapse exists to prevent. - round 18: `referent_not_portable` read "it points at something in the source workspace", claiming both existence and location for a reason emitted WITHOUT resolving the target — and which `github_pr` reaches too, where the referent is in no workspace at all. A third defect of the same shape would have been found the same way, by a reviewer happening to read it, or not at all. Moved to `$lib/items/copyDropReasons` with the reason vocabulary as an explicit exported list, which makes two tests possible that could not be written before: 1. EVERY reason the server can emit has a sentence. This is round 12's other finding as a test: BUG-2674 added `referent_not_portable` server-side, nothing here learned it, and it rendered through the fallback as a raw enum string in front of a user. A reason that maps to itself IS that defect. Mutant: delete `referent_not_portable`'s message -> FAIL. 2. THE TWO HAZARDOUS REASONS STAY NEUTRAL. `not_found` and `referent_not_portable` must not claim a target exists, does not exist, or say where it is. Asserted over those two rather than all ten: `wrong_collection` legitimately says the target is outside the field's collection, and it may, because the server only emits it to a caller who can SEE the target. Mutant: restore round 18's wording -> FAIL, naming the sentence and why. The unknown-reason fallback returns the raw string, and a third test pins that deliberately: a reason this build has never heard of means the server is ahead of the client, and showing the enum is more honest than inventing a sentence or hiding the row. It is also what keeps test 1 from being vacuous. WHAT THIS DOES NOT FIX, stated because the list is the thing a future reader will trust: the reason vocabulary is DUPLICATED from Go (five constants in `handlers_items_copy_preflight.go`, five in `internal/store/relation_referents.go`) rather than generated, so it can still go stale in the one direction that matters — a reason added to Go and not added here. The test cannot see that. What it can see is a reason listed here without a sentence, and the list is now the single place to update. Generating it from the Go constants would close the gap properly and is a bigger change than this one. Gates: `npm run check` 0 errors (6 pre-existing warnings in unrelated files), `make web-test` 126 files / 2118 tests passed. Frontend-only; no Go touched. Claude-Session: https://claude.ai/code/session_01Xk9M5UVPdc84xL5E1mZkm8 * docs(store): record why a carried relation's survival is observable and accepted (IDEA-2893) Comment only; no behaviour change. A carried relation value naming a live item in a collection the mover cannot see resolves and survives a same-workspace move, while one naming nothing is dropped — so a mover can tell those apart, and on a stored REF they also learn the target's canonical id. The lead's ruling is ACCEPT AND DOCUMENT, and this is the documentation, placed at the branch that produces the behaviour rather than in a doc nobody reading that code will open. Four measurements decided it, and the comment carries the two that matter so a future reviewer reaches the reasoning instead of re-deriving it: - NOT ENUMERABLE. A caller cannot choose what to test: create, update and fields_patch all refuse a caller-supplied ref naming an item they cannot see, with the COLLAPSED `not_found` wording, so no door turns a chosen value into a carried one. It can only confirm a value already sitting in an item the caller can read and did not put there. - THEY ALREADY HAVE THE VALUE. An ordinary GET returns the raw stored relation value verbatim; reads apply no redaction. The increment is "it currently resolves" plus the ref-to-id mapping — not the target's existence in any general sense, not its title, and backlinks do not widen it either. The comment also records why each candidate fix costs more than the increment, because "we considered it" is worth nothing to a successor without the reasons: redacting the response closes nothing (the id is in the blob and comes back from a plain GET); not canonicalising removes only the id half and makes a relation value stop meaning one thing everywhere; dropping by the MOVER's visibility silently destroys a valid relation because of who moved the item; and canonicalising only for movers who can see the target would make the STORED BYTES depend on who performed the move. The last paragraph is the one I most want read. The ONE change that would close this is carrying unresolvable values verbatim instead of dropping them — which is precisely the drop-and-report rule three lines below, the rule #1246 exists to enforce. So the comment doubles as a warning: changing that rule changes this, in the other direction. A design decision that is load-bearing for a second decision should say so at the place where someone would change it. I also WITHDREW my own earlier proposal rather than leaving it standing: "evaluate the carry against the ITEM's access rather than the mover's" was confused, because an item has no access identity and resolving without a requester IS the status quo. A plausible-sounding option left in a trail is a successor's wasted afternoon. Gates: `gofmt` clean, `go build ./internal/store/`, `go vet ./internal/store/`, `go test ./internal/store -run 'TestRelation|TestMigrateRelation'` ok. Scoped to the package because this worktree has no `web/build` for the root embed, and the change is a comment in one file. Claude-Session: https://claude.ai/code/session_01Xk9M5UVPdc84xL5E1mZkm8 * fix(test): repoint the drop-reason parity gate at the extracted module (IDEA-2894) `f2c4a722` moved the drop-reason mapper out of `CopyItemDialog.svelte` and broke `TestCopyPreflightDropReasonsAreRenderedByTheDialog`, which reads the component for that function. CI caught it; my local gates did not, because this worktree had no `web/build` for the root embed so I had scoped the Go run to `internal/store` — and I stated that boundary in the commit message while it was hiding a real failure. Naming a gate's scope is not the same as the scope being adequate. THE GATE FAILED THE RIGHT WAY, and that is worth recording. It does not search the file for `case 'not_found':` and shrug when the file changes; it looks for the declaration by name and calls `t.Fatalf` if it is gone, saying "this gate is reading for a function that moved or was renamed, so its green means nothing until it is repointed". A parity gate that cannot tell "no such reason" from "no such function" is worse than none, because the second reads as the first passing. Repointed at `web/src/lib/items/copyDropReasons.ts` and STRENGTHENED, because the extraction split the thing it was checking in two. It now requires each server reason to appear in BOTH: - `COPY_DROP_REASONS`, the exported list; - the `MESSAGES` map. They fail differently, and the first is the one that matters. The module's own completeness test ITERATES that list, so a reason missing from the list is invisible to that test as well — this gate is the only place it shows. A test driven by a list cannot notice something absent from the list. I ALSO HAVE A CORRECTION TO MAKE, to my own prose in `f2c4a722`. That commit message and the PR body say the Go-to-TypeScript direction "can still go stale in the one direction that matters — a reason added to Go and not added here. The test cannot see that." That is FALSE, and I wrote it without checking: this parity gate has enumerated the Go vocabulary and required a renderer for every entry since BUG-2674, which is precisely that direction. My TS test cannot see it; the repo already had a test that could, and I asserted its absence rather than looking. Same failure as the fixture I designed around a hazard yesterday instead of asking whether the product had it — a claim about what is NOT covered owes a grep exactly as much as a claim about what is. The PR body is corrected in the same push. Mutants, all three arms, each restored after: - remove a reason from the LIST -> FAIL, naming the list and why the module's own test cannot see it; - remove its entry from the MESSAGES map -> FAIL, naming the fallthrough; - rename the `MESSAGES` declaration -> FATAL with the repoint message, so the fail-safe itself is exercised rather than assumed. Gates: `go test ./internal/server` ok **279.535s** — the full package this time, with `web/build` populated so the root embed resolves. `gofmt` clean. Claude-Session: https://claude.ai/code/session_01Xk9M5UVPdc84xL5E1mZkm8 * docs(store): point the carry comment at the trail rather than at a person (IDEA-2893) Comment wording only. The attribution now reads `(IDEA-2893, lead ruling day 58; the measurements it rests on are on that idea's trail, which is where to check this reasoning rather than take it)`. The lead asked for `confirmed by Dave in chat` and I declined to write it: Dave had said nothing to me about this disposition, so the only evidence was a relay through a channel BUG-2542 proved cannot carry provenance, and the artifact is a permanent comment asserting what a specific person decided. The lead withdrew the line and agreed the hold was right. Dave then ruled the general case — no code comment needs to name him — which is the wording above and is better than either version, because a ref is CHECKABLE and a name is not. A reader who doubts this comment can open the idea and read the measurement; a reader who meets a name can only take it or leave it. Recorded team-side as CONVE-32 so the successor does not relearn it: code comments cite the trail, never a person by name. Its scope is source comments only — commit messages, PR bodies and trail comments are where naming who decided something is often the entire content, and those artifacts sit beside their own evidence. Two things the convention says out loud rather than gloss: - The rule reached me as a RELAY, and I acted on it because it only ever REMOVES a claim about a person. Acting on a relay to stop asserting something is safe in a way that acting on a relay to start asserting it is not — which is the same distinction that made the hold correct an hour earlier. Read as general licence to act on relayed instruction it would be a misreading; the direction is the whole point. - About nineteen comments already in `internal/` name a person. They are NOT rewritten. Churning merged history to apply a new rule retroactively costs more than it returns and a sibling rebasing onto it pays the bill. Fix one only while editing that comment for another reason. Gates: `gofmt` clean, `go build ./internal/store/`, and the parity gate green (`TestCopyPreflightDropReasonsAreRenderedByTheDialog` ok) since this touches the same file the previous commit repointed it away from. Claude-Session: https://claude.ai/code/session_01Xk9M5UVPdc84xL5E1mZkm8	2026-09-05 19:01:57 -04:00
xarmian	14cb97593f	WIP feat(server,store): referent validation for relation values — 6 of 8 doors (TASK-2878) (#1246) ... * feat(store): referent resolution for relation values (TASK-2878) PLAN-2857 U1, first slice: the rule itself, with no door wired to it yet. `ResolveRelationReferents` canonicalises every `relation` value in a field map to the target item's ID and reports the ones that cannot be resolved — same workspace, and the collection the field DECLARES. WHERE IT LIVES was forced, not chosen. `internal/items` is DB-free by construction and keeps the shape check only. `internal/server` cannot own it either: six of the eight coercion doors live there, but the eighth is `store.migrateFieldsForCopy`, and `store` does not import `server`. Putting it here is what lets the cross-workspace copy door and the preflight door reach the SAME function instead of two implementations of one rule — those two already carry a comment saying they sit in different packages and that is how they drift unnoticed. VISIBILITY IS NOT HERE, deliberately. "Can this requester see that item" is request-scoped and needs the user, role and auth mode; the server layer adds it via `checkItemVisible`, which already exists as the context-free predicate for exactly this reason. NO SLUG FALLBACK, which is a deliberate divergence from `ResolveItem` (UUID, then ref, then slug). Found by a test failing rather than by reading: "red" resolved, because it is the slug of the live Red colour. A relation field's contract is that it stores an item ID; a slug is neither an ID nor stable, so the same stored value could point elsewhere tomorrow. Worse, "red" is exactly the free-text value the pre-U2 editor wrote into these fields, so accepting it makes the corruption this unit exists to stop indistinguishable from a legitimate write. The client refuses the same match for the same reason (TASK-2868). Exact-TITLE resolution is U6. Issues are reported in SCHEMA order, not map order, because the copy preflight is one of the callers and is specified to be safe to call repeatedly and return identical results. Unresolvable values are left EXACTLY as supplied: the caller quotes them back, and a half-canonicalised map would make a drop report lie about what the source held. Verified rather than asserted: both lookups exclude soft-deleted rows (`ResolveItem` by contrast with `ResolveItemIncludeDeleted`; `GetItem` via `getItemScanQ`, which appends `AND i.deleted_at IS NULL`). That is what keeps "target was deleted" distinguishable from "never resolved" — the read half U2 shipped. * feat(server): refuse unresolvable relation values at the four write doors (TASK-2878) PLAN-2857 U1, second slice: the doors that take CALLER-SUPPLIED field values now refuse a relation value that does not name a live item in the declared target collection — create, update (full fields), update (fields_patch), and bulk update. The server half adds the one thing the store resolver deliberately does not: visibility. It folds into the SAME `not_found` reason rather than getting its own, because "that item exists but you may not see it" is an existence oracle, and this codebase has a standing rule against handing one out. Ordering at every door is after the shape check and after coercion, so one bad value produces one error rather than two describing it differently, and so the value is in its final form when it is resolved. `fields_patch` examines only the keys the patch carries — the resolver skips absent keys — so an unresolvable value already stored on an item is not re-litigated by an update that does not touch it. That mirrors the undeclared-key rule immediately above it, and it is what stops this turning every edit of a legacy item into a failure. Refusals use the ORDINARY `validation_error` shape with no new details key. The MCP stdio transport classifies errors by matching CLI stderr prose, so a structured field it cannot see would help nobody there, and a new error shape is a contract change for every client. Existing suites unchanged: internal/server ok (224.5s), internal/store ok (258.0s), internal/items ok. Nothing in the tree was writing a bogus relation value through these doors, which is what made this slice safe to land before the per-door pins. * feat(store): one migrate decision for all four carrying doors (TASK-2878) PLAN-2857 U1, third slice, on the lead's refined ruling: PROVENANCE decides, not which door you came through. * SUPPLIED (an explicit `--field` override on a move or copy) is a write like any other, so an unresolvable value REFUSES. * CARRIED (everything the source item already held) was asserted by nobody. `internal/items` has accepted any string for a relation all along, so most stored values are legacy — refusing them would make those items unmovable and uncopyable. Dropped and REPORTED instead. And carried values are not all alike, which is the refinement that keeps this from being one rule wearing four coats: * WITHIN a workspace (move, bulk move) the targets are still here, so a valid relation SURVIVES the move and only an unresolvable one is dropped, through the `dropped_fields` channel BUG-2674 established. * ACROSS workspaces (copy, and its preflight) every carried relation is dropped WITHOUT a lookup: the value names a source-workspace row and v1 excludes cross-workspace targets, so no amount of resolving in the destination changes what it means. Reported as `referent_not_portable` — the same reason `github_pr` uses, because it is the same fact about the same kind of value. `MigrateRelationReferents` is one function because the four doors sharing it is the point, not tidiness: the preflight lives in `internal/server` and the copy in `internal/store`, and the code already carries a comment saying those two sit in different packages and that is how they drift unnoticed. A preflight that says "carried" while the copy drops is one request answered two ways. Tests drive both provenances against both modes, because the same bad value must be a drop when carried and a refusal when supplied — a suite that only drove carried values would pass against a build that never refuses anything. * feat(server): the two same-workspace migrate doors resolve and report (TASK-2878) PLAN-2857 U1: `handleMoveItem` and `bulkMoveCollection` now take their relation decision from `store.MigrateRelationReferents` — the same function the two copy doors will call, which is the point of it existing. Within a workspace the targets are still present, so a correctly-related item KEEPS its relation across a move; only an unresolvable value is dropped, and it joins the `dropped_fields` report BUG-2674 established rather than failing the move. Refusing carried values here would make every legacy item permanently unmovable, and `internal/items` has accepted any string for a relation all along, so "legacy" is most of them. The bulk path carries no per-field overrides — only `status` — so every relation value reaching it is CARRIED and nothing there can refuse. It passes nil for `supplied` to say so, and keeps the refusal branch: it is unreachable today and stops being a silent no-op the day that path grows overrides. internal/server ok (270.8s), internal/store ok. KNOWN GAP, recorded rather than half-built: the two CROSS-workspace doors are not wired yet, and the reason is a real constraint rather than running out of road. `migrateCopyFields` is called from `copyItemAcrossWorkspacesTx` with a transaction already open (`s.db.Begin()` at the top of that function), so resolving a SUPPLIED override there would issue POOL reads while holding a tx — the deadlock shape this repo keeps a deterministic test for. The carried half needs no lookup at all and is safe; the supplied half needs `GetCollectionBySlugQ` / `GetItemByRefQ` so the resolver can run on the tx's connection, which is exactly the `...Q` convention the store already uses (`GetItemQ`, `getCollectionInWorkspaceTx`, `uniqueSlugQ`). Adding those two is the remaining work, and it is what makes one function genuinely serve all four doors. * refactor(store): thread a Queryer through referent resolution (TASK-2878) Preparation for the two cross-workspace copy doors, landed on its own because it is independently correct and the doors are not. `migrateCopyFields` runs inside `copyItemAcrossWorkspacesTx`, which opens a transaction as its second statement. A resolver reading from the POOL there would issue pool reads while holding a tx — the deadlock this repo keeps a deterministic test for. So `ResolveRelationReferentsQ` and `MigrateRelationReferentsQ` take the executor, following the store's own convention (`GetItemQ`, `uniqueSlugQ`, `getCollectionInWorkspaceTx`); the pool-backed names stay as one-line shims for the six wired doors. Two small read helpers come with it. `collectionIDBySlugQ` returns the ID only — the referent check compares `item.CollectionID`, and the full model would pull in per-collection counts nothing here uses. `itemByRefQ` keeps `GetItemByRef`'s fallback to a bare item-number lookup, because a relation written as COLO-3 must keep resolving after its target collection is renamed, which is exactly what BUG-2873 made possible. internal/store ok (341.7s), vet and gofmt clean. WHY THE COPY DOORS ARE NOT IN THIS COMMIT. They were written and building, and I reverted them. Team CONVE-29 and the lead's condition both say the copy pair lands WITH its pin — one case driving BOTH doors, asserting identical drop-and-report for a carried relation and refusal for a supplied override — and I measured 58.9% context against a 65% ceiling, which is not enough for that pin plus the 270s server suite plus the commit. Landing the behaviour change unpinned would have been worse than landing nothing: the preflight and the store copy are the pair the code already warns will drift unnoticed, so they are the last place to accept an untested agreement. The design is complete and on the trail: derive the carry mode from the existing `items.MigrateScope` rather than a second flag, pass `tx` on the store side and the pool on the preflight side, refusals through the copy's existing validation-error channel, drops appended to `migrated.Dropped`. * feat(copy): the two cross-workspace doors resolve referents, with their pin (TASK-2878) PLAN-2857 U1, doors seven and eight. `migrateCopyFields` and `handleCopyItemPreflight` now take their relation decision from `store.MigrateRelationReferents` — the same function the four write doors and the two move doors already call, which is the entire reason it exists. The defect this closes: MigrateFields matches on key and TYPE, so a same-named `relation` field carried a SOURCE-workspace item id across the boundary and the preflight reported it as a clean carry. What landed in workspace B was a value naming a row in workspace A — unrenderable, and indistinguishable on read from a legitimate reference. Provenance decides, as at the move doors. A CARRIED value on a cross-workspace copy is dropped without a lookup (no id from A can mean anything in B) and reported through the `dropped_fields` channel BUG-2674 established; a SUPPLIED override is an ordinary write and an unresolvable one is refused, 400 validation_error on both doors, rendered by the same `store.RelationIssuesMessage` so one refusal cannot acquire two phrasings. `internal/server`'s `relationIssuesMessage` now delegates to it: the eighth door refuses from inside `store`, so the sentence had to be reachable there. Two things are threaded rather than re-derived, and both are load-bearing: - The TRANSACTION, not the pool. `migrateCopyFields` becomes a method taking a Queryer, and `copyItemAcrossWorkspacesTx` passes its `tx`. That function has held a transaction since its second statement, so a pool read from inside it can wait for a free connection while every pooled connection is blocked on this transaction's locks — the starvation shape BUG-2409 fixed for the attachment planner and this repo keeps a deterministic test for. This is what the day-70 handoff named as the reason these two doors were not wired with the other six. - The MODE comes from the `scope` MigrateFields was already given, not from a second boundary test. Two independent answers to "is this crossing a workspace" is how one request gets migrated one way and validated the other, and this path also serves a copy whose target IS the source workspace, where relations resolve and survive exactly as on a move. The destination workspace id is the resolution scope: a supplied override is a write into B and must name something that exists there. THE PIN, and why it is not a per-door table. These two doors sit in different PACKAGES and the code at both sites says so is how they drift unnoticed. A table with a row per door can be fully green while the two disagree about one request, which is the defect rather than a gap in coverage of it. So every case sends ONE body to BOTH endpoints: - carried relation — must drop on both, and the preflight must say referent_not_portable rather than the generic no_target_field, which is false here (the destination DOES declare the key, so that answer sends the reader to fix a schema that is fine); - supplied + unresolvable — both refuse, same status, same code, both name the offending field and value, and nothing is written; - supplied + resolvable — the positive control, supplied as a REF so resolution is visible in the result. Without it the first two legs are equally consistent with "relations always fail". Negative controls run, all three mutants BUILD-CHECKED first (a non-compiling mutant produces no `--- FAIL` lines and reads as survived): both doors unwired = DETECTED; preflight unwired alone = DETECTED; store unwired alone = DETECTED. Each single-door mutant failing is the pin's whole claim — neither door can be wired without the other. CONVE-23 sweep: the preflight's LIMITATION comment said this gap belonged in MigrateFields "for both callers at once". That is now false in its prescription as well as its premise — `internal/items` is DB-free by construction and cannot ask whether a string names a live item — so the comment records where the fix actually went and what of it remains open (`computed`, `terminal_options`, `unique_scope`). Gates: internal/server ok 170.0s · internal/store ok 296.1s · internal/items ok · go vet clean · gofmt clean · make lint 0 issues. * test(server): the per-door x per-provenance table, and the defect it found (TASK-2878) PLAN-2857 U1. `internal/store` already tests the resolver exhaustively, but those tests call it DIRECTLY: they vouch for the component and say nothing about whether any door is bound to it. A door that never calls the resolver passes every one of them. This table is the binding claim — one leg per (door, provenance) pair, driven through the handler a client reaches. PROVENANCE IS THE SECOND AXIS BECAUSE THE ANSWER DEPENDS ON IT, not for symmetry. A SUPPLIED value is the caller's assertion and an unresolvable one is refused; a CARRIED value was asserted by nobody, and refusing it makes legacy items un-updatable, un-movable and un-copyable — the failure this unit would otherwise CAUSE while fixing another. Which provenance a door sees is a property OF THE DOOR, and getting it wrong is invisible until a legacy item meets it. THE TABLE FOUND ONE, ON ITS FIRST RUN. `bulkFieldUpdate` merges the item's STORED fields blob with the caller's `changes` before validating, and the resolver was pointed at the MERGED map — so a bulk status move or set-priority re-litigated every stored relation value and REFUSED the item. An item carrying a legacy relation value had its status and priority frozen by a field the operation never mentioned. Fixed the way the fields_patch door already handles it: resolve only the keys the operation CHANGES, read out of the coerced map so the value is final, written back so a supplied ref is still canonicalised. Verified by prediction before the run and by the leg failing against the unfixed code. THE DISPATCH MUTANT, which is what makes the table's coverage a measurement rather than a hope. Wire ONLY the two `extractParentLink` doors (update fields, update fields_patch) and neuter the other six by swapping their field-map argument for an empty one — types unchanged, so the mutant compiles and its verdict means something: build OK · failing legs: create (both), move (both), bulk move, bulk update supplied, copy, preflight. Update and update-patch pass. Exactly the six unwired doors, and only those. Then each door alone, eight runs: every one detected by its own legs and no other door's. That is the claim the table exists to make — each leg reaches its OWN door rather than being satisfied by a neighbour's check. ONE DOOR NEEDS A WEAKER INSTRUMENT, AND THE TEST SAYS SO. No bulk op puts a relation key into `changes` — `op` is a closed list and the only field values any of them set are `status` and `priority` — so that door's SUPPLIED branch is unreachable from outside. The first mutant run proved it: unwiring door 4 alone left every black-box leg passing. It gets a direct-call leg, labelled as vouching for the FUNCTION and not for a binding that does not exist yet, and kept for the same reason `bulkMoveCollection`'s refusal branch is kept: the day the bulk path grows per-field overrides, the branch must already refuse rather than be a silent no-op nobody notices is missing. Every refusing leg has a resolvable counterpart. Without them the table is equally consistent with a build that refuses every relation value. Gates: internal/server ok 156.1s · go vet clean · gofmt clean · make lint 0 issues. * feat(mcp): ToolSurfaceVersion 0.29, and the drop-reason renderer it exposed (TASK-2878) PLAN-2857 U1. The bump, its documentation sweep, and the consumer this change turned from a rare wart into a routine one. THE BUMP, at 0.29 rather than 0.28. Rebasing onto main found b437cc58 (IDEA-2641, reminders) had ALSO taken 0.28 — a SEMANTIC collision, not a textual one: two different contracts under one number, and a client pinning "0.28" would have had no way to know which it got. Renumbered to 0.29, and main's 0.28 entry kept intact ahead of it. Also swept while resolving: main's CLAUDE.md carried v0.28 straight after v0.26, because the v0.27 bump (BUG-2850) never reached that file. Both places in CLAUDE.md now read v0.26 -> v0.27 -> v0.28 -> v0.29, and the README changelog line gains the v0.28 entry it never got. A BEHAVIOR bump on the 0.27 / 0.26 / 0.16 / 0.10 / 0.9 grounds — not on 0.28's, which was purely additive — no tool name, action enum or parameter shape changed, and `pad_item` now refuses calls it used to accept. A `relation` value must name a live item in the collection the field declares; a caller writing a resolvable value sees no difference, and one writing an unresolvable value was storing something no surface could render, so the break is the fix. No escape hatch, deliberately: unlike 0.10's `allow_draft` there is no legitimate call this refuses, and the case with a real claim to leniency — a CARRIED value — is already exempt by provenance rather than by a flag. Full entry in internal/mcp/version.go. instructions.md and README.md follow the constant because `internal/mcp`'s own drift gates require it; CLAUDE.md does not, which is exactly why it drifted. CLAUDE.md's `pad item move` / `pad item copy` reference also gains the relation semantics, which is the part a reader of that file is most likely to need and the part that just changed. THE CONSUMER, which is the interesting half. `dropped[].reason` is a wire enum with its renderer on the other side of a language boundary, and nothing made the two meet. BUG-2674 added `referent_not_portable` server-side; the TypeScript union and CopyItemDialog's `dropReason` switch never learned it, so it fell through `default: return reason` and the dialog showed a user the raw string. It went unnoticed because only `github_pr` produced it — and this change makes it the reason for EVERY carried relation on a cross-workspace copy, plus three more (`not_found`, `wrong_collection`, `target_missing`). A latent defect going live because my emission made the form routine. So the reasons are ENUMERATED rather than left as literals: `store.RelationIssueReasons()` owns the four the store decides, `preflightDropReasons()` composes them with the five this package originates, and the emission sites now use the constants. The new gate requires every entry to have a union member and a switch case. The gate is SCOPED to `dropReason`'s own body rather than searching the component, because another switch matching `case 'not_found':` for an unrelated purpose would satisfy it while the dialog still rendered the raw enum — a guard passing on the wrong evidence. If that function is renamed the test FAILS rather than silently passing on a body it can no longer find. Negative controls, all DETECTED: remove the dialog's `referent_not_portable` case; remove `wrong_collection` from the TS union; rename `dropReason` (which checks the scoping guard's own premise). What the gate does not claim: that a case EXISTS, not that its sentence is good — no test judges that. It is also blind to a renderer handling a reason the server never sends; that direction is a dead branch, the other is the defect. Gates: internal/server ok 162.8s · internal/mcp ok 13.6s · internal/store ok 192.2s · `npm run check` 0 errors (6 pre-existing warnings, unrelated files) · go vet clean · gofmt clean · make lint 0 issues. * fix: four codex round-1 findings — origin, visibility, bulk reporting (TASK-2878) Codex round 1 named four; three were real P1s and I verified each in the code before accepting it. Codex could not run anything ("Go could not create its build cache because the workspace is read-only"), so every finding here is a static read that I confirmed and pinned. ## A THIRD ORIGIN, not two `items.MigrateFields` injects the DESTINATION schema's defaults for keys the source item has nothing for. My classifier split on `supplied` versus everything-else, so a destination default was filed as CARRIED — and on a cross-workspace copy every carried relation drops without a lookup. The destination's own default was discarded and reported `referent_not_portable`, which is flatly false about a value the destination chose. There are three origins: SUPPLIED (refuse on failure), CARRIED from the source item (cross-workspace: drop as not-portable), and DESTINATION DEFAULT (resolve against the destination in BOTH modes; drop with the resolver's own reason on failure, because nobody in this request typed it). Telling the last two apart needs the source field map, which all four migrate doors have as `currentFields`, so it is now a parameter. Empty values are skipped at every origin. An empty relation is a cleared field, not a referent, and reporting it as dropped tells a user they lost something they never had. WHAT THE FIRST VERSION OF THIS TEST PROVED: nothing. The mutant that reverts the classifier SURVIVED it. `ValidateFields` re-injects the default after my resolver deleted the key, so the value comes back either way and `StillDropped` filters the false report out — the end state is identical unless the default is a REF. A UUID default is already its own canonical form, so "resolved" and "dropped then re-injected raw" produce the same bytes. With `PEOP-1` as the default the mutant is DETECTED, because only a resolved default lands as the id. ## SUPPLIED OVERRIDES AT THE MIGRATE DOORS SKIPPED THE VISIBILITY CHECK The four write doors go through `s.resolveRelationReferents`, which adds `checkItemVisible` on top of the store resolver. The migrate doors called `store.MigrateRelationReferents` directly — it is a store function and cannot answer a request-scoped question — so their SUPPLIED half, which this unit's own rule calls an ordinary write, resolved against the database alone. A caller able to edit both collections could point a relation at an item they cannot see. The ROLE is the part worth getting right. For `move` it is `workspaceRole(r)`. For copy and preflight it is the caller's role in the DESTINATION, and `CrossWorkspaceAccess.Role` is exactly that — its own doc says never to substitute `workspaceRole(r)`. So `resolveRelationReferents` now takes the role explicitly (`resolveRelationReferentsAs`), and the new `refuseInvisibleRelationOverrides` runs at all three doors with the right one. At the copy it runs in the HANDLER, before the store call: a pre-write refusal must not open a transaction to roll it back, and the preflight runs the identical check — DR-6's "the preview IS the copy" only holds if both doors refuse the same request. ## BULK COLLECTION MOVE DISCARDED FIELDS SILENTLY `bulkMoveCollection` has populated `result.Dropped` since MigrateFields existed and NOTHING read it — the only reference in the file was my own append. BUG-2674 fixed the single-item door and left this one, so a bulk move discarded values with no record anywhere. Pre-existing, and routing relation drops into the same dead list is what made it mine to fix. Reported on the activity row, same key, same joined-string shape and the same BUG-2628 reason as `handleMoveItem`, filtered against the final map so the report is true when written. Threaded as an out-parameter, deliberately: only this branch produces drops, the caller needs them for ONE activity row per item, and a third return value would put `nil` in fourteen unrelated returns. ## THE P2, AND WHAT IT IS NOT PINNED BY `resolveRelationReferents` did `if item == nil { continue }` after the visibility read — "treat a race as someone else's 404". It now refuses with the same `not_found` the resolver would have given moments later. This whole unit exists to keep a dangling referent out of the blob, and a target that vanished mid-request is the one case where waving it through would have been deliberate. NO TEST. Reproducing it means deleting a row between two reads inside one request, and a test that faked that would pin the fake. Stated here rather than left to look covered. ## Counterfactuals Every fix has a mutant that its own test detects, each build-checked first: classifier reverted -> DETECTED (destination default); empty-skip removed -> DETECTED; visibility helper neutered -> DETECTED at both the move door and the copy/preflight pair; bulk-move report removed -> DETECTED. Gates: internal/server ok 224.6s · internal/store ok 261.5s · internal/items ok · internal/mcp ok 15.1s · go vet clean · gofmt clean · make lint 0 issues. * fix: two codex round-2 findings — late defaults, UUID disclosure (TASK-2878) Round 2 confirmed the three round-1 fixes and found two more. Both real, both verified in the code before acceptance; codex still could not run anything ("the read-only filesystem prevented Go cache/temp creation"). ## THE DESTINATION-DEFAULT FIX ONLY COVERED HALF THE WAYS IN The migrate doors resolve BEFORE they validate, and that order is load-bearing: the required-field check has to see a value referent resolution dropped, or a dropped value in a REQUIRED relation field would store the item with the field absent instead of refusing. But `ValidateFields` INJECTS schema defaults, so a default can land after the resolver has finished. Two ways in, both now closed: - a NULL OVERRIDE deletes the key and the default fills the hole, arriving uncanonicalised — a `PEOP-1` default reached the row as the literal string; - a default the resolver DELETED as unresolvable is put straight back by validation, and `StillDropped` then suppresses the warning about it. Dropped, reported, restored, and reported as not-dropped. `ResolveLateRelationDefaults` is a narrow second pass over exactly the keys validation added. Reordering the two would have traded this defect for the required-field one; a second pass costs one lookup in the rare case a relation field declares a default and nothing otherwise. A default is asserted by nobody, so an unresolvable one is DROPPED and reported, never refused — the same disposition the main pass gives RelationOriginDestinationDefault, which is what this is: the same origin, arriving late. THE SNAPSHOT IS TAKEN AFTER THE MAIN PASS, NOT BEFORE, and getting that wrong is what the second sub-case caught. Snapshotting before the pass treats a key the pass DELETED as already examined, so the late pass skips the very value validation just put back — the arrangement that hid it. My first version did exactly this and the second sub-case failed on it. ## A REFUSAL FOR AN INVISIBLE ITEM DISCLOSED ITS UUID The store resolver rewrites a supplied ref into its target's id before the visibility check runs, so a message built from the resolved value handed back the canonical UUID of an item the requester may not see — confirming both its existence and its identity. That is the existence oracle the `not_found` collapse exists to prevent, reopened by the message. Every issue this function raises now quotes what the CALLER sent. ## Prior findings, per codex Override visibility: fixed. Bulk move dropped list: fixed. Nil-target race: no dereference. Destination defaults: this commit. ## Counterfactuals Neuter the late-default pass -> TestCopyEndpoint_LateInjectedRelationDefault DETECTED on both sub-cases. Restore the canonical UUID in the message -> TestCopyEndpoint_InvisibleRelationOverrideIsRefused DETECTED. Both mutants build-checked first. Gates: internal/server ok 346.3s · internal/store ok 351.3s · internal/items ok · internal/mcp ok 22.9s · go vet clean · gofmt clean · make lint 0 issues. Postgres green on the parent commit (store 534.5s, server 268.5s, private container port 5481); re-running on this tree. * fix: three codex round-3 findings — oracle, bulk defaults, required (TASK-2878) Round 3 confirmed round 2's fixes and found three more. Codex still could not execute anything, so all three were static reads I verified myself. ## `wrong_collection` DISCLOSED EXISTENCE The store emits `wrong_collection` when a value names a LIVE item outside the field's declared target, and the server's visibility layer skipped any key that already carried an issue. So the message — "is not an item in collection X" — told a caller the value EXISTS, distinguishably from the `not_found` a nonexistent value gets. An existence oracle for anyone who cannot see that item, and the exact shape the `not_found` collapse exists to prevent. Collapsed to `not_found` when the requester cannot see the target, and KEPT otherwise: "you linked a task where a person belongs" is the useful half of this reason, and blanket-collapsing would have passed the security leg while destroying it. Both legs are in the test for that reason. Needed one new store export, `ResolveRelationTarget`: deciding what to disclose about a live item requires the item, and the resolver is the only thing that maps a value to one under the no-slug rule. ## BULK STATUS / PRIORITY BYPASSED RELATION DEFAULTS `bulkFieldUpdate` resolves only the keys `changes` names — correctly, since re-litigating stored values would freeze legacy items — but `ValidateFields` runs BEFORE it and INJECTS schema defaults. A defaulted relation was persisted raw: never canonicalised, never checked against its collection. The same late-arrival the migrate doors hit, reached by a different route, and the narrow pass built for them covers it unchanged. `dropped_fields` now rides every bulk op's activity row, not just the move branch: a status or priority change can discard a relation default too, and a drop nobody records is the defect BUG-2674 closed. ## A REQUIRED RELATION COULD END UP ABSENT AND REPORTED VALID The late pass deletes a key AFTER validation passed, so nothing re-checked required-ness: a required relation whose default did not resolve left the item with the field absent, and the preflight reported `valid: true`. Re-running validation is not the fix — it would re-inject the same broken default. There is no valid value for that field, so the doors refuse: `missing_required_fields` on move and bulk move, a FieldValidationError on the copy, and on the preflight a `needs_value` row with `valid: false`. That split is the one this pair has everywhere else — the preview says what is wrong, the copy refuses. ## Counterfactuals Remove the oracle collapse -> the visibility leg DETECTED. Remove the bulk-update late pass -> DETECTED. Remove the required-relation refusal at the copy door -> DETECTED. All build-checked first. Gates: internal/server ok 352.1s · internal/store ok 358.7s · internal/items ok · internal/mcp ok 21.0s · go vet clean · gofmt clean · make lint 0 issues. Postgres green on the parent commit (store 596.7s, server 294.7s, private container port 5481); re-running on this tree. * fix: two codex round-4 findings — oracle race, null-source provenance (TASK-2878) Round 4 confirmed round 3 and dropped from P1 to P2, which is what convergence looks like. Both still real. ## THE ORACLE COLLAPSE HAD A HOLE, AND MY OWN COMMENT ASSERTED IT DID NOT Round 3's fix collapses `wrong_collection` to `not_found` when the requester cannot see the target. It needs the item to ask that, and when the second lookup came back nil — deleted between the resolver's read and this one — it did `continue` under a comment reading "vanished since; already the safe answer". That comment was wrong. Nothing had rewritten the reason, so the issue still said `wrong_collection`, whose message tells the caller the value named something a moment ago. The disclosure round 3 closed, reachable by a race. Collapsed now. Worth stating plainly because it is the same defect twice: I wrote a comment asserting a state ("already safe") that the code did not establish, and I wrote it in the fix for exactly that disclosure. ## A NULL SOURCE VALUE IS NOT A CARRIED ONE The preflight marked any key PRESENT in the source's field map as `from: "migrated"`. A source key holding `null` is present and carries nothing: MigrateFields keeps the key, the relation pass skips a nil value, and validation then treats it as missing and fills the DESTINATION's default in its place. So the response reported the source as the origin of a value the destination chose. `from` is what a dialog uses to say "this came across" versus "this is the destination's default", so naming the wrong one is the preview lying about the thing it exists to report. Presence with a VALUE is the test now. Not relation-specific — any nulled source field with a destination default had it — so the guard is on the shared origin loop rather than beside the relation pass. ## Counterfactuals Remove the null guard from the origin loop -> TestCopyEndpoint_NullSourceRelationIsNotReportedAsMigrated DETECTED (build-checked first). The oracle-race collapse has NO test, for the reason the vanished-target refusal has none: reproducing it means deleting a row between two lookups inside one request, and a test that faked the interleaving would pin the fake. Both are listed in the PR body under "Not tested, and why". Gates: internal/server ok 343.8s · internal/store ok 349.9s · internal/mcp ok 22.1s · go vet clean · gofmt clean · make lint 0 issues. Postgres green on the parent commit (store 553.9s, server 297.1s, private container port 5481); re-running on this tree. * fix: codex round-5 finding — the other direction of the origin guard (TASK-2878) One finding, P2, and it is the tail of round 4's fix. Round 4 stopped labelling a null-source key `migrated`, because validation treats null as missing and fills the DESTINATION's default in its place — so `migrated` named the source for a value the destination chose. With NO destination default there is nothing to fill it: the null is what carries, and `default` names a value the schema never declared. One guard, wrong in both directions, and the reason is the same in both: "was the key present in the source" is not "where did the final value come from". The label now follows the value: - source value, non-nil -> migrated - source null, destination default -> default - source null, NO default -> migrated `from` is what a dialog uses to say "this came across" versus "this is the destination's default", so either error is the preview misreporting the thing it exists to report. ## Counterfactual Remove the round-5 branch so a null source always reads as `default` -> TestCopyEndpoint_NullSourceWithoutDefaultIsNotReportedAsDefault DETECTED, build-checked first. The mutant also confirms the key really is in `carried` in that scenario — the test returns without asserting if it is not, so without the mutant its green would have been consistent with a vacuous pass. Gates: internal/server ok 288.8s · internal/store ok 302.4s · internal/mcp ok 13.7s · go vet clean · gofmt clean · make lint 0 issues. Postgres green on the parent commit (store 626.4s, server 311.9s, private container port 5481); re-running on this tree. * fix: three codex round-6 findings, one premise corrected (TASK-2878) Round 6 confirmed round 5 and found three. All three are real; one arrived with an account of its own cause that the test refuted, and the corrected route is narrower than the report. ## A NON-STRING RELATION DEFAULT, AND WHERE IT ACTUALLY GETS IN The finding: injected defaults are never type-checked, so `42` or `[]` can persist in a relation field. True, but not by the route described. `MigrateFields` injects destination defaults ITSELF, so in the ordinary case the key is present when `ValidateFieldsDetailed` runs and its type IS checked — a numeric default lands in needs_value with "must be a string", which is correct behaviour. My first test asserted the wrong thing and FAILED against the fixed build, which is how I found this out. The unchecked route is narrower: a NULL OVERRIDE deletes the key after MigrateFields filled it, so validation injects the default itself — and its own injection branch `continue`s PAST the type check. That is the one way a non-string reaches a relation field unchallenged. The late-default pass owns values that arrive from defaults, so it reports this one: `invalid_shape`, a new reason, because every existing reason describes a lookup that never happened. Retargeting the test then exposed a second defect IN MY OWN FIX: the late pass's `if len(late) == 0 { return nil, nil }` discarded the non-string drops it had just recorded. The value vanished from all three buckets. Green on the first route, silent on the second. THE PARITY GATE FROM AN EARLIER COMMIT CAUGHT THE NEW REASON: adding `invalid_shape` failed TestCopyPreflightDropReasonsAreRenderedByTheDialog until the TypeScript union and CopyItemDialog learned it. That gate exists because `referent_not_portable` shipped unrendered in BUG-2674, and it just did its job on its author. ## A WHITESPACE-ONLY VALUE IS "NO REFERENCE", NOT A BAD ONE The store resolver trims and ignores `" "`. The server wrapper checked the UNTRIMMED string, so the value fell through to the visibility loop — and since round 1's vanished-target arm turns a missing lookup into a refusal, `" "` came back as not_found instead of an empty field. A defect my own round-1 fix introduced: before it, that path did `continue`. ## A MALFORMED CARRIED VALUE IS NOT "NOT PORTABLE" The cross-workspace branch dropped every carried value without looking, including non-strings, and labelled them `referent_not_portable` — a false account of why the value is going. It is not a reference at all. Left in place now for ValidateFields to reject on shape, which is what the SAME-workspace branch already did with it: the two modes disagreeing about one malformed value was the defect. ## Counterfactuals Skip non-string defaults again -> TestCopyEndpoint_NonStringRelationDefault DETECTED. Restore the untrimmed skip -> TestRelationDoors_WhitespaceOnly DETECTED. Both build-checked, and the first form of the second mutant did NOT build (unused import) — reported as such rather than scored, since a non-compiling mutant produces no failures and reads as survived. Gates: internal/server ok 324.8s · internal/store ok 344.5s · internal/mcp ok 16.9s · go vet clean · gofmt clean · make lint 0 issues · npm run check 0 errors. Postgres green on the parent commit (store 596.1s, server 299.4s); re-running on this tree. * fix: codex round-7 finding — the write doors had the same default hole (TASK-2878) One finding, and it is round 6's defect in the two places round 6 did not reach. `ValidateFields` assigns a schema default and `continue`s PAST its own type check, and the resolver skips a non-string — so `default: 42` on a relation field reached the blob unchallenged at CREATE and at a full `fields` UPDATE. Only the default escapes; a value the caller supplies is type-checked and refused like any other. Closed with the same pass the migrate doors use, so there is one implementation of "a default that arrived after the resolver" rather than two. ## DROPPED, NOT REFUSED, AND REPORTED Nobody in the request typed it. Refusing would make every write into that collection fail on a schema defect its author has to fix elsewhere — the same reasoning that makes a CARRIED value a drop rather than a refusal. A REQUIRED relation is the exception and still refuses, because dropping there would store the item with a required field absent. Silence was not an option either, so item write responses gain `warnings.dropped_fields`, additive and omitempty exactly like 0.27's `undeclared_fields`, naming the discarded keys. `internal/mcp/version.go`'s 0.29 entry and CLAUDE.md's API section both carry it — the response shape is documented in two places and a change to it owes both. ## Counterfactual Remove the late pass from createItemChecked -> TestRelationDoors_NonStringDefaultIsDroppedAndReportedOnWrite DETECTED, build-checked first. Gates: internal/server ok 470.1s · internal/store ok 463.5s · internal/mcp ok 29.6s (the four drift gates included) · internal/models ok · go vet clean · gofmt clean · make lint 0 issues. Postgres green on the parent commit (store 608.1s, server 343.1s); re-running on this tree. * fix: codex round-8 finding — the import door forwarded half the warnings (TASK-2878) One finding, P2. `createItemChecked` records dropped relation defaults alongside undeclared keys; the artifact-import handler enumerated `UndeclaredFields` and nothing else, so an import that DISCARDED a value said nothing about it. The shape is worth naming: a forwarding loop written per-member of a warnings struct is a gap that widens every time that struct grows. `DroppedFields` was added one commit earlier and this loop kept reporting only what it already knew about. An import discarding a value is the same class of news as one storing an unrecognized key, and worse — that value is gone. ## Counterfactual Forward only undeclared fields again -> TestImportArtifactReportsDropped RelationDefault DETECTED, build-checked first. ## A GATE FAILURE THAT WAS NOT A DEFECT, read from the elapsed time `internal/store` reported FAIL at 600.017s while the Postgres suite was running concurrently at load average ~10. That is the default `go test` 10-minute timeout, not a test failure: no `--- FAIL` line anywhere in the output, and the same package under `-timeout=45m` on Postgres passed in 824.9s on the same tree. Re-run alone with an explicit timeout: ok 332.1s. Recorded rather than quietly re-run, because "600.017s" is the whole diagnosis and a second glance at the exit code would not have produced it. Gates on this tree: internal/store ok 332.1s · internal/server ok 338.3s · internal/mcp ok 26.5s · internal/models ok · go vet clean · gofmt clean · make lint 0 issues. Postgres green on the parent commit: store 824.9s, server 703.4s (private container port 5481; both slow for the contention reason above). * refactor: classify by survival, not key presence — and the mutant survives (TASK-2878) Codex round 9, one P1: the relation classifier calls a value CARRIED because its key exists in the source's field map. `MigrateFields` matches on key AND type, so an incompatible source value is DROPPED and the default loop then refills the same key from the DESTINATION schema — the key is still in the source map while the value in hand came from the destination. A cross-workspace copy would discard that default as non-portable. `CarriedSourceValues` narrows the source map to the values that actually survived, using MigrateFields' own Dropped list, and all four migrate doors plus the preflight's ORIGIN loop now use it. Both were answering "did this value come across?" from a map that answers "did the source declare this key?". ## THE MUTANT SURVIVES THIS TEST, AND THAT IS THE FINDING Restoring the misclassification leaves the test green. I chased that rather than accepting it, and the reason is that THE DEFECT HAS NO OBSERVABLE EFFECT. Two independent downstream repairs erase it: - the misclassified default is dropped, `ValidateFields` re-injects it, and the late-default pass resolves it — landing the same bytes; - the drop deletes the key's `origin` entry, and the carried loop's own fallback for a missing origin is `"default"` — the same label. Value and label agree on both builds. My first attempt to discriminate added an origin assertion, which is the sharper instrument and STILL survives, because of the second repair. So this is a robustness change, not a bug fix: it stops the classifier depending on two rescues to produce a right answer, and makes "carried" mean what the word says. The test says so in its own comment and claims nothing about the classification — it pins the OUTCOME, which is worth holding however it is reached. Recording it this way rather than shipping a green labelled "regression test", because a test that passes on the broken build is exactly the fixture CONVE-30 warns about, and the honest label is the whole value. Gates: internal/server ok 226.0s · internal/store ok 247.9s · internal/mcp ok 13.8s · go vet clean · gofmt clean · make lint 0 issues. Postgres owed on this tip. * fix: codex round-10 finding — the write doors refused schema defaults (TASK-2878) One finding, P2, and it is the asymmetry this unit's own rule forbids. Validation injects a schema default BEFORE the resolver runs, and the resolver treats the whole field map as caller input. So an optional relation whose schema default names nothing turned every create and full-fields update into a 400 — on a defect the caller neither caused nor can fix from that call. The migrate doors have always dropped such a value and said so; the write doors refused it. `IssuesForCallerInput` keeps only the issues raised against keys that were present BEFORE validation. Anything validation injected falls through to the late-default pass, which drops it and reports it in `warnings.dropped_fields` — a REQUIRED one still refuses, because dropping there would store the item with a required field absent. This is the third place the same rule has had to be stated: a value asserted by nobody is dropped and reported, never refused. Carried values (round 1), late-injected defaults (round 2), and now defaults at the write doors. ## Counterfactuals, both directions Neuter the filter so defaults refuse again -> the new test DETECTED. Make the filter drop everything so the door refuses nothing -> four tests DETECTED, including the caller-supplied-bad-value legs. The second is the one that matters: without it this fix could have quietly disabled the write doors' refusal entirely and still looked green. Gates: internal/server ok 330.5s · internal/store ok 339.1s · internal/mcp ok 20.5s · go vet clean · gofmt clean · make lint 0 issues. * fix: two codex round-11 findings — default visibility, bulk status (TASK-2878) ## A SCHEMA DEFAULT NAMING A HIDDEN ITEM HANDED BACK ITS ID Round 10 filtered the refusal set down to caller input, which correctly stopped a dangling default refusing the write — and also removed the VISIBILITY issues raised against those same keys. `ResolveLateRelationDefaults` then re-resolved them in the store, which has no visibility layer by construction, and the write's response carries the item's fields. So a caller received the canonical id of an item they cannot see. A fix from the previous round opening a hole in the round before it. The two changes are individually right and the seam between them is where it went. DROPPED, not refused, because the origin has not changed: the schema author chose the value, and the caller can neither fix it nor be blamed for it. What they must not get is the id. Reported through the same `not_found` every other visibility failure collapses to, so "names something you may not see" and "names nothing" stay indistinguishable. Six doors wired. The seventh — the cross-workspace copy inside `store` — is NOT, and cannot be from here: it has no request, and handing the store a request-scoped visibility callback is the structural change IDEA-2886 files. Stated rather than left as an apparent oversight. ## `req.Status` ON A BULK MOVE IS CALLER INPUT That path merges exactly one field and passed `supplied=nil` on the grounds that it carries no per-field overrides — true of every field except that one. A destination schema may declare `status` as a relation, and then a value the caller typed was classified as CARRIED: silently dropped instead of refused, and never checked for visibility. The refusal branch there had been written as "unreachable today, kept so it stops being a silent no-op the day this path grows overrides". It was already reachable when that sentence was written. **A branch nobody can reach is a branch nobody checks** — the comment asserting unreachability is what stopped anyone testing the classification that made it false. ## Counterfactuals Neuter the late-default visibility helper -> DETECTED. Pass `nil` for supplied on the bulk move again -> DETECTED, but only AFTER I wrote the test: the first run of that mutant SURVIVED, because I had no case exercising a destination whose `status` is a relation. The survivor is what said the test was missing. Both build-checked. Every fix carries a control leg — the visibility one asserts that the OWNER's identical create still lands the default, without which it would pass against a build that dropped every default. Gates: internal/server ok 403.8s · internal/store ok 406.8s · internal/mcp ok 24.7s · go vet clean · gofmt clean · make lint 0 issues. Postgres green on the parent commit (store 524.8s, server 302.9s). * refactor: one write-door relation path, and neutral wording for two reasons (TASK-2878) ## The web round, which had never had one The TypeScript union and CopyItemDialog changed across three commits with only the Go-side parity gate looking at them. First review round on those files found two, both in the sentences a user actually reads: `not_found` said "the item it refers to no longer exists". That reason is ALSO what the server collapses a hidden target to — telling the two apart is the existence oracle the collapse exists to prevent — so asserting non-existence is wrong for half the cases and is a claim the response cannot support. Now "could not be found". `target_missing` said "the field declares no collection to link to". It is emitted for two causes: no target declared, AND a declared target that is not a collection in this workspace. The wording named only the first and misdiagnosed the second as a schema that says nothing when it says something broken. Now "has no valid collection to link to", and the store's own message for the same reason was over-claiming identically and is fixed with it. Neither is a code defect. Both are the write-up half of the same rule this unit keeps enforcing: say what is true, not what is convenient. ## The consolidation, per the lead's CONVE-139 ruling Create and full-`fields` update each spelled out the SAME four steps — resolve the whole map, keep only the caller's issues, resolve the defaults validation injected after the main pass, drop a default whose target the caller cannot see. Round 10 was the third time one of those steps had to be stated separately, which is what showed the shape. Extracted into `resolveRelationsForWrite`, called twice, with the rule written once above it. NOT extended to the migrate doors, and stated in the code rather than left to look like an omission: they reach the same rule through `store.MigrateRelationReferents`, which cannot call this — the visibility layer is request-scoped by construction and `store` cannot import `server`. Unifying the families needs a caller-supplied visibility predicate on the store API. That is IDEA-2886, filed, with the shape it would take. Gates: internal/server ok 359.4s · internal/store ok 365.5s · go vet clean · gofmt clean · make lint 0 issues · npm run check 0 errors. Postgres green on the parent commit: store 543.9s, server 298.4s. * fix: an overflowing issue ref resolved to a different item (TASK-2878) First per-package round on `internal/store`, per the lead's split. One P1, one P2 accepted with reasoning. ## OVERFLOW IS A WRONG ANSWER, NOT A BIG ONE `parseItemRef` accumulates the numeric half into a machine int with no bound: `num = num*10 + digit`, unchecked. So `COLO-18446744073709551617` wraps to 1 and names the item numbered 1. A caller-supplied ref canonicalising to a DIFFERENT item is precisely the corruption relation referent validation exists to stop — reached through the PARSER rather than through the lookup, which is why every door-level instrument in this unit was blind to it. Bounded at 1<<31, far above any real item number and below the wrap point, so an out-of-range value is rejected rather than silently reinterpreted. `parseItemRef` is shared, so this fixes every caller, not only the relation resolver. It is outside this unit's eight doors and I fixed it anyway: my resolver hands untrusted input straight to it, and the defect is a wrong resolution, which is the unit's subject. ## The P2, accepted rather than fixed, with the reasoning The resolver caches the target COLLECTION id and does not re-check it, so on Postgres a concurrent `DeleteCollection` can archive that collection between the lookup and the insert, leaving a relation into an archived collection. Not fixed, for the reasons the copy path already records about its own attachment-staleness window: on SQLite there is no window at all (BEGIN IMMEDIATE holds the write lock, so the archive blocks until commit); on Postgres closing it would mean putting the workspace advisory lock on every collection writer; and the outcome of losing the race is benign and self-describing — the read half renders a reference into an archived collection honestly rather than as a live link. Same shape, same disposition, recorded rather than left silent. ## Counterfactual Remove the bound -> TestResolveRelationReferents_OverflowingRefDoesNotResolve DETECTED, build-checked. The test carries a control leg asserting the honest ref for the same item still resolves, so a failure is about the overflow and not about the fixture. Gates: internal/store ok 273.1s · internal/server ok 272.7s · go vet clean · gofmt clean · make lint 0 issues. Claude-Session: https://claude.ai/code/session_01Xk9M5UVPdc84xL5E1mZkm8 * fix: three codex round-13 findings — the visibility helper checked the wrong set (TASK-2878) First per-package round on `internal/server`. Three findings; two are defects in the two rounds immediately before, which is what a per-package read buys that a whole-diff read had stopped buying. ## THE VISIBILITY HELPER SKIPPED EXACTLY WHAT IT EXISTS TO CHECK `dropInvisibleRelationDefaults` skipped keys "present before the late pass". At the WRITE doors that set is approximately the caller's own values, which is right. At the MIGRATE doors it also contains every default `MigrateFields` injected — so the values the helper was written for were the ones it skipped, and only the write doors were ever covered. The predicate is now "not a destination default": the caller's own values and the values carried from the source, excluded for two different reasons. A caller's value is checked on its own door. A CARRIED value must never be dropped for visibility at all, or an item referencing something the mover cannot see becomes unmovable — the failure mode this unit has refused since round 1. Getting that predicate wrong was easy because both sets are "keys already in the map"; they are the same shape and mean different things. ## A SIBLING LIST ADDED BESIDE A CHECK IS NOT COVERED BY IT The required-relation check was written for `lateDropped`. The visibility drops were added beside it one round later, and nothing extended the check — so a REQUIRED relation whose default the caller cannot see was deleted after validation had passed, leaving the field absent and the write reported valid. Same shape as the round-3 finding it repeats: deleting a key after validation means nothing re-checks required-ness, whichever list recorded the deletion. ## LOOKUP ERRORS WERE ECHOED VERBATIM `relErr.Error()` went into the response body at create and at three bulk sites. Those errors can carry SQL and driver detail. Replaced with a fixed sentence; the real error still reaches the log. ## Counterfactuals Skip destination defaults again -> DETECTED at the create door AND the move door. Remove the required check on the visibility drops -> DETECTED, but only after I wrote the test: the first run SURVIVED, and the survivor is what said the case was missing. Both build-checked. Every visibility test carries a control leg asserting the OWNER's identical request still lands the default, without which they would pass against a build that dropped every default. Gates: internal/server ok 305.6s · internal/store ok 317.2s · internal/mcp ok 15.8s · go vet clean · gofmt clean · make lint 0 issues. Postgres green on the parent commit: store 506.6s, server 269.8s. Claude-Session: https://claude.ai/code/session_01Xk9M5UVPdc84xL5E1mZkm8 * fix: two codex round-14 findings, two recorded unfixed (TASK-2878) Second `internal/server` round. Four findings: two fixed here, two recorded with reasons rather than half-done at the end of a run. ## A NIL IS NOT A VALUE — third time in this unit `notDefaultKeys` counted every key present in the supplied or carried map, nil included. `ValidateFields` treats a present-but-nil key as ABSENT and injects the destination default in its place, so a nil counted the key OUT of exactly the visibility check that injected default needs. Rounds 4 and 5 were the same distinction in the origin label; this is the same mistake in a set-membership test. The shape worth carrying: **"the key is in the map" and "the map has a value for the key" are different predicates, and the second is almost always the one meant.** ## A VANISHED TARGET LEFT A DANGLING CANONICAL ID The visibility helper did `continue` when its lookup came back nil — leaving the resolved id in the map and skipping the required-field handling below. Now dropped like any other default that does not stand. That is the THIRD time a `continue` on a nil lookup has been the defect (rounds 1 and 4 were the other two), each in a different function, each written by me while fixing something else. ## Recorded, not fixed **The mutating copy still applies no visibility check to destination defaults.** The handler checks explicit overrides only; the defaults are injected inside `store` during the copy transaction, and the handler cannot know which without duplicating `MigrateFields`. Closing it needs the caller-supplied visibility predicate on the store API — IDEA-2886, filed, with the shape it would take. Stated here so the gap is inherited as a known one rather than found again. **Two bulk error sites still echo `err.Error()` verbatim** (item resolution and the visibility gate in `handleBulkItems`). Both PRE-DATE this unit and cover every bulk op, not relations — changing them changes messages for all of them and belongs with whoever owns that door. ## Counterfactuals Count nils as carried again -> DETECTED. Both fixes build-checked; the vanished-target arm has no test for the reason the other two race arms have none, and joins them in the PR body's "Not tested, and why". Gates: internal/server ok 434.4s · internal/store ok 438.4s · go vet clean · gofmt clean · make lint 0 issues. Postgres green on the parent commit: store 713.6s, server 375.8s. Claude-Session: https://claude.ai/code/session_01Xk9M5UVPdc84xL5E1mZkm8 * feat: artifact import CARRIES junk relation values (TASK-2878, Dave's ruling) Dave, day 57, on the write posture: "all of that sounds like exactly what we want. we should ensure import is allowed as I would want to explicitly allow import of the junk to avoid breaking import." On this branch artifact import routed through `createItemChecked` and was therefore REFUSING any artifact whose relation value did not resolve — which is every artifact written before referent validation existed. That is a live break introduced by this unit, not a pre-existing one. ## IMPORT IS A CARRY DOOR An artifact was written elsewhere, possibly years ago; the person importing it did not choose its field values and cannot fix them from the import call. So `relationPosture` splits the two callers of `createItemChecked`: create refuses, import carries. NOT the migrate doors' carry, and the difference is the point. Those DROP what they cannot resolve, because the value has no home in the destination. An import's value has a home — the artifact IS the record — so it is stored verbatim and REPORTED, through a new `warnings.unresolved_relations`. Kept is not the same as unreported; an import that quietly changed what it imported would be the worse failure. Additive and omitempty, like `undeclared_fields` and `dropped_fields` before it. Distinct from both: `dropped_fields` means the value is gone, and `undeclared_fields` is about the KEY rather than what it points at. The door count in the PR body becomes 4 write / 6 migrate-or-carry. ## The fixture was unreachable, and the file already said why My first version declared an invented relation key and asserted a carry that `artifact.Decode` had already dropped: the artifact FORMAT decides which keys reach the field map at all (FieldKeysForKind), so a key the format does not know never gets near this door. It has to be a CANONICAL artifact key that the DESTINATION declares as a relation — `role`, here. The warning about this shape is in the same test file, from BUG-2850's round 10, and I walked into it anyway. Recorded in the test so the next person gets it from the fixture rather than from the failure. ## Not pinned, and where to pick it up The OTHER import door — `store.ImportWorkspace` — writes raw rows and does not go through `createItemChecked`, so it already carries and this change cannot have altered it. It is unpinned only because I am at the context bar. Model it on TestImportWorkspace_CoercesTitles in internal/store/items_title_validation_test.go, which builds the export blob this needs. ## Counterfactual Flip artifact import back to `relationsRefuse` -> TestImportArtifactCarriesUnresolvableRelationValue DETECTED, build-checked. Gates: internal/server ok 290.0s · internal/store ok 305.2s · internal/models ok · go vet clean · gofmt clean · make lint 0 issues. Claude-Session: https://claude.ai/code/session_01Xk9M5UVPdc84xL5E1mZkm8 * test(store): pin the workspace-import carry door for unresolvable relation values (TASK-2878) Checkpoint 12 left this open: Dave's import ruling was implemented for the ARTIFACT import door (relationPosture splits create-refuses from import-carries) but store.ImportWorkspace — the other import door — was unpinned. It writes raw rows and never goes through createItemChecked, so referent validation cannot have reached it. That is an argument the behaviour is unchanged, not evidence it is correct or that it will stay: the carry is a ruling, and an unpinned ruling is one refactor away from turning every archive written before referent validation existed into a hard import failure. The question the test answers, stated before the result (CONVE-30): does ImportWorkspace store a relation value whose referent is not in the bundle VERBATIM — neither refusing the import nor dropping the value — while still remapping the relation values it CAN resolve? The resolvable leg is a CONTROL, not decoration. "Carried verbatim" and "never processed" produce byte-identical output for the junk rows, so without a leg whose expected output DIFFERS from its input, a build that deleted the second-pass remap outright would pass every carry assertion (CONVE-30 instance 5, day 71: when a test asserts a TRANSFORMATION happened, pick an input whose output differs from it). Mutation matrix — three mutants, each BUILD-CHECKED (go build ./internal/store clean) and each killed by a named `--- FAIL:` line, export.go restored from a backup file and re-verified byte-identical afterwards: M1 import REFUSES an unresolvable relation value -> FAIL relation_referents_test.go:513, the must-SUCCEED assertion M2 import DROPS unresolvable relation values from the field map -> FAIL both carry subtests at :567; the control leg still PASSED, which is what shows the two legs discriminate different faults M3 second-pass remapFieldIDs removed -> FAIL relation_referents_test.go:550, the control leg only Fixture note carried in the test: the exported ids deliberately share no common prefix, because remapFieldIDs rewrites by strings.ReplaceAll over every id in the bundle — an unresolvable value CONTAINING a resolvable one ("old-color-1" inside "old-color-10") would be partially rewritten and the carry assertion would fail for a reason unrelated to the rule under test. Gates on this tip: internal/store ok 327.9s (SQLite), go vet clean, gofmt clean. Postgres not yet run on this tip — stated as a boundary, not implied currency; it is the next step on the successor list. Claude-Session: https://claude.ai/code/session_01Xk9M5UVPdc84xL5E1mZkm8 * fix: three codex round-15 findings — two real, one unobservable (TASK-2878) Round 15 (internal/server) returned four findings. One re-reported the known IDEA-2886 gap, already recorded unfixed. The other three are handled here, and two of the three were only settled by trying to kill them. 1. IMPORT SKIPPED THE DEFAULT CLEANUP (real, fixed, pinned). Step 2 of resolveRelationsForWrite returned early whenever the caller's values had issues — correct while every caller of that path REFUSED, since steps 3 and 4 only prepare a field map nobody stores. Checkpoint 12's carry posture added a caller that does not refuse, and the early return stayed: an artifact holding one junk relation skipped the late-default pass AND the default-visibility drop, stored whatever the destination schema injected, and reported none of it. The early return is now posture-aware; the carry report is threaded through the remaining steps rather than lost. This is the fourth time in this unit that one of my own fixes opened the next round's defect, and the first where the interval was a day. Mutant: restore the unconditional early return -> FAIL, warnings list `role` only and `owner_ref` is stored raw as 42. 2. LATE DEFAULTS BYPASSED THE VISIBILITY COLLAPSE (real, fixed, pinned). store.ResolveLateRelationDefaults cannot know who is asking, so every issue it returns carries the raw reason, and every door renders those into a caller-visible message. `wrong_collection` is the one reason that names a LIVE item, so a schema default pointing at an item the caller cannot see announced that it EXISTS — the oracle round 3 closed on the main pass, reopened through the door round 10 added. Collapse hoisted into collapseInvisibleRelationIssues and applied at ALL FIVE late-default sites, not the one the reviewer named (CONVE-18). Mutant: remove the collapse at the move door -> FAIL, the 400 reads `"LV-2" is not an item in collection "people"` to a caller who cannot see LV-2. Control leg included: a caller who CAN see the target keeps the specific reason, so collapsing everything to not_found fails too. 3. STALE CARRIED-SOURCE SET (kept as robustness, NOT a bug fix, documented). Reviewer named the preflight. Grepping the class found three doors with the shape — and the mutants say only one of them could ever have been wrong, and even that one is unobservable: - move and bulk move fold their relation drops into result.Dropped and then recompute CarriedSourceValues INLINE at the visibility call, so they read the already-extended list. Restoring the stale form leaves their tests green because there is nothing there to break. Both edits REVERTED. - the preflight does hand its visibility call a variable captured before that append, but restoring it leaves every test green too: a default whose target the caller cannot see is already collapsed to not_found by the MAIN pass and dropped before this check runs. Probed on both builds — the owner's preflight discloses the default's id, a restricted editor's reports owner_ref dropped as not_found either way. Kept because carriedSource should mean what its name says at every use rather than being correct at three uses and stale at the fourth because two later passes repair it. The disposition is in the helper's doc comment. A regression test was WRITTEN for this and then DELETED: it passed against the unfixed build, and a test that cannot fail is worse than none — it reads as a guard while guarding nothing. CONVE-18 asks for the population of a class; it does not license assuming every member is defective. Gates: build, go vet, gofmt clean on this tip. The full SQLite suite (server/store/items/mcp) was STILL RUNNING when this was committed and its result is NOT claimed here. Postgres has NOT been re-run since 21dfb115. Both are the successor's first two steps. Claude-Session: https://claude.ai/code/session_01Xk9M5UVPdc84xL5E1mZkm8 * revert: drop the preflight carried-source robustness change (TASK-2878) Lead ruling, day-57 02:4xZ: drop round-15 finding 3 from a119cf39 in one commit. It shipped as a documented robustness change whose mutants survived at every door, with no regression test, because the test written for it passed against the unfixed build and was deleted. The ruling's reasoning, which I agree with: a change with no observable behaviour and no test is not robustness, it is prose in code. The instrument answered "not a defect", so there is nothing to fix, and CONVE-30 says the answer is only as wide as the question I asked — I asked whether the stale set was reachable, got "no" at three doors and "no, unobservably" at the fourth, and then shipped a fix anyway on a naming argument. Reverted, exactly and only: - handlers_items_copy_preflight.go: the visibility call takes `carriedSource` again, the pre-relation-drop snapshot. - internal/server/relation_referents.go: `carriedAfterRelationDrops` and its 50-line doc comment removed. - internal/store/relation_referents.go: `RelationIssueKeys` removed — it had exactly one caller, the helper above, and is dead without it. Findings 1 and 2 from a119cf39 are UNTOUCHED and still pinned. Verified by diffing against the parent rather than by reading this diff: against 21dfb115 the preflight now shows only the `collapseInvisibleRelationIssues` call (finding 2), and internal/store/relation_referents.go is byte-identical to the parent. The finding stays recorded — here and on TASK-2878's trail — as REPORTED, NOT A DEFECT, CLASS-CHECKED, so round 16 and later stop re-reporting it. The class grep and both probe legs are in a119cf39's message, which remains the record of what was measured. Gates on this tip: build, `go vet` (server+store), `gofmt -l internal/` all clean. `go test ./internal/server -run 'TestRelationDoors|TestImportArtifact'` ok 10.479s — the pins for findings 1 and 2, run because this commit touches the file one of them lives in. The full SQLite suite and Postgres are owed on this tip and are NOT claimed here. Claude-Session: https://claude.ai/code/session_01Xk9M5UVPdc84xL5E1mZkm8 * fix: four codex round-16 findings, all four real and pinned (TASK-2878) Round 16 (internal/server) returned five findings. Four are real, confirmed by a test that FAILS against the build without its own fix and passes with it; the fifth is a true observation whose proposed remedy closes nothing, and is recorded rather than fixed. Every test ships with an owner control leg, so none of them passes against a build that simply refuses or drops everything. 1. BULK MOVE NEVER RAN THE SUPPLIED-HALF VISIBILITY CHECK (real, fixed). Round 11 established that `req.Status` on a bulk collection move is CALLER INPUT and wired `suppliedByCaller` so the store classifier would REFUSE an unresolvable value rather than drop it. It did not carry across the other half of the supplied contract: the store resolver cannot answer a request-scoped question, which is why the single move door, the copy and the preflight all call refuseInvisibleRelationOverrides first. Bulk move was the only one of the three MigrateRelationReferents call sites that never did. So a caller who cannot see the relation's target collection could name a live item in it and have the value stored, receiving its canonical id back. Mutant: remove the refuseInvisibleRelationOverrides block -> FAIL, "a caller who cannot see People pointed a relation at one of its items: 21c93e0b-8340-423c-a528-4eb78340bbf5". 2. THE PREFLIGHT'S CARRIED DROPS NEVER PASSED THE VISIBILITY COLLAPSE (real, fixed). Round 15 hoisted the collapse so ResolveLateRelationDefaults' issues stop naming live items to callers who cannot see them. MigrateRelationReferents returns its issues down a SECOND path — `relationDropReason`, rendered straight into `fields.dropped[].reason` — and that path was never collapsed. `wrong_collection` is the reason that names a LIVE item, so a hidden-but-live target reported differently from a value naming nothing. Class: this is the ONLY site where a MigrateRelationReferents drop reason reaches a caller. Move and bulk move report dropped KEYS and no reasons, so the class is one site, not five, and the grep says so rather than the assumption. Mutant: remove the collapse -> FAIL, "a LIVE hidden item reports 'wrong_collection' and a nonexistent value reports 'not_found'". 3. THE STALE CARRIED SET IS REAL AFTER ALL, AND ROUND 15's CLEARANCE WAS MY INSTRUMENT'S FAULT (real, fixed, and it reverses ace2d3f5). Round 15 reported this; every mutant survived; it shipped as a documented robustness change; the lead ruled it out as unobservable and I reverted it in ace2d3f5. The ruling required a concrete request under which the two sets differ before it could be re-reported. Round 16 supplied one, and the test fails against the unfixed build: "carried":[{"key":"owner_ref","type":"relation", "value":"a8fdfa7a-3487-4acb-9ad4-53b67cd96a19","from":"default"}] with valid:true and no dropped row — a live item in a collection the caller cannot see. WHY EVERY MUTANT SURVIVED, which is the part worth keeping: my probe used a source value that RESOLVED. The relation pass never drops a resolvable value, so the pre-drop and post-drop carried sets were identical BY CONSTRUCTION and the probe could not have failed. That is the same defect as the fixture I recorded yesterday — two outcomes with identical bytes — with the instrument being a probe rather than a test, which is why the CONVE-30 habit did not catch it. The discriminating shape is a non-nil DANGLING source value plus a required destination default naming a hidden item. The fix is NOT the one that was dropped. That was 71 lines across three files with a new exported store helper. This is one expression at the call site — store.CarriedSourceValues(currentFields, migrated.Dropped) — because the loop above already extended migrated.Dropped with the relation pass's drops. No new export, no helper, and it makes the preflight agree with move and bulk move BY CONSTRUCTION rather than by argument, which is what the lead's ruling was right to object to in the first version. 4. A NULL OVERRIDE DID NOT CLEAR A STALE NON-NIL CARRIED KEY (real, fixed). `notDefaultKeys` learned in round 14 that a nil value is not a value, and skipped nils PER MAP. A caller who nulls a key whose STORED value is non-nil still had it counted — out of `carried`, on the strength of a value the request had just discarded — so the destination default injected in its place was exempted from the visibility check. The relation pass does not rescue this: an override of nil leaves nothing to resolve, so the key is never dropped and never leaves the carried set. The existing null-source test cannot reach it; that fixture has no stored value to go stale. Fixed at the shared helper, so move, bulk move and preflight get it at once. Mutant: drop the explicit-nil-in-supplied clause -> FAIL, the move stores and returns 550d947c-5970-4d61-9db2-d8e22c8a25a1 to a caller who cannot see People. 5. CANONICALISING A CARRIED REF DISCLOSES A HIDDEN TARGET'S UUID (true observation, NOT fixed, and the reason is a measurement rather than a judgement). A carried ref naming a live item in a hidden collection canonicalises to its UUID and survives; a dangling one drops. The caller can therefore tell the two apart, and the reviewer asked for the response to redact the UUID. Redacting the response closes nothing. The canonical UUID is written into the stored blob of an item the caller CAN read, so the same bytes come back from the ordinary read door: probed as the same restricted editor, GET of the moved item returned 200 with the UUID present. A response-only fix would move the disclosure one request to the left. The remedies that would actually close it are not this unit's to make: stop canonicalising carried values, or drop carried values by visibility — and the second is precisely the data destruction this unit's carry rule exists to avoid, since it would silently delete a valid relation because the MOVER cannot see its target. Recorded as an open design question alongside IDEA-2886. Gates on this tip: build, go vet, gofmt clean; go test ./internal/server -run 'TestRelationDoors|TestCopyEndpoint|TestCopyPreflight|TestImportArtifact' ok 44.309s. The full SQLite suite and Postgres are owed on this tip and are NOT claimed here. Claude-Session: https://claude.ai/code/session_01Xk9M5UVPdc84xL5E1mZkm8 * fix: two codex round-17 findings in internal/store; two recorded, not defects (TASK-2878) Round 17 (internal/store) returned four findings. Two are real and pinned with killed, build-checked mutants. Two are recorded with the measurement that says they are not defects, so round 18 stops re-raising them. 1. A MALFORMED DESTINATION DEFAULT REFUSED OR DROPPED DEPENDING ON AN UNRELATED REQUEST DETAIL (real, fixed, pinned). MigrateRelationReferents skipped non-string values on the general rule that shape is ValidateFields's to reject, so one defect makes one error. That rule is right for a SUPPLIED or CARRIED value and wrong for a DESTINATION DEFAULT, because the two disagree about the outcome: ValidateFields REFUSES the request, and a default is not the caller's assertion, so this unit's posture for it is drop-and-report. The observable is worse than the inconsistency. A default MigrateFields injects is in the map before validation and was REFUSED — 400, the whole copy blocked, over an OPTIONAL field. The identical default that ValidateFields injects, which is what a `{"owner_ref": null}` override causes, reached the late pass and was DROPPED, and the copy completed. Same malformed schema, opposite answers, chosen by a request detail with nothing to do with it. Non-string defaults are now dropped as `invalid_shape` in the early pass, exactly as ResolveLateRelationDefaultsQ already dropped them, so the two paths agree BY CONSTRUCTION. Supplied and carried values are untouched: the comment's reasoning holds there and only there. The existing non-string-default test sends the null override and so only ever exercised the forgiving path. The new test runs BOTH legs. Mutant: remove the drop -> FAIL on the `no override` leg only, `explicit null override` still passing — which is the same fact stated twice: the old test could not have caught this. 2. THE ID REMAP CORRUPTED AN UNRESOLVABLE CARRIED VALUE (real, fixed, pinned). remapFieldIDs rewrote relation values with strings.ReplaceAll over the raw JSON for every id in the bundle, so a value that merely CONTAINED another id was partially rewritten: with ids `old-color-1` and a relation value `"old-color-10"`, the import stored `<new-id>0` — a string that references nothing and existed on neither side. Bundle ids are whatever the exporting instance had and an import accepts a caller-supplied file, so this is not confined to well-formed UUIDs. The same substitution also rewrote ids appearing inside ordinary text values, which was never the intent. It matters here because of the carry posture: an unresolvable relation value is deliberately imported VERBATIM rather than dropped, and verbatim is the whole promise. Now a JSON walk matching WHOLE values, recursing into arrays so a multi-valued relation is covered. An unparseable blob is returned untouched rather than guessed at. WORTH RECORDING ABOUT HOW THIS WAS MISSED: my own fixture in TestImportWorkspace_CarriesUnresolvableRelationValues carries a comment explaining that its ids "deliberately share no common prefix" because otherwise ReplaceAll "would partially rewrite" the value. I identified the mechanism exactly, engineered the fixture AROUND it, and never asked whether the product had the defect the fixture was dodging. A hazard worth designing around is a hazard worth filing. Mutant: restore the ReplaceAll form -> FAIL, the dangling value comes back rewritten. Control leg included: an EXACT-match value must still remap to the imported item's new id, so the test fails against a build that stopped remapping. 3. A CARRIED DROP IS NOT REPORTED WHEN A DESTINATION DEFAULT REFILLS THE KEY (recorded, not changed). Measured: `fields.dropped` and `warnings.dropped_fields` are both empty, and the preflight's carried row says `"from":"default"` — so the preflight DOES tell the reader the value came from the destination rather than the source, while the copy's dropped_fields does not. This is round 3's deliberate decision (reporting a key as dropped while it is populated made three surfaces give two answers), the two doors agree, and changing it is a response-CONTRACT change — a new bucket distinguishing "replaced by a default" from "dropped" — not a bug fix. Left for a ruling rather than taken unilaterally. 4. AN UNRESOLVABLE STRING DEFAULT IS RESOLVED TWICE (recorded, not a defect). The claim is true and has no observable: probed, the preflight reports exactly one `dropped` row and the copy exactly one `dropped_fields` entry, not two. The cost is one redundant lookup for a default that is already broken. The READ COMMITTED divergence the reviewer raises is the same target-vanishes-mid-request race already recorded as deliberately untested, and it resolves the same way — the value is dropped either way. Gates on this tip: build, go vet, gofmt clean; the store and server relation suites green. The full SQLite suite and Postgres were green on 8f3f7f57 and are OWED AGAIN on this tip; they are not claimed here. Claude-Session: https://claude.ai/code/session_01Xk9M5UVPdc84xL5E1mZkm8 * fix: the remap must preserve bytes; round-18 web findings (TASK-2878) Three changes: one is a regression I introduced two commits ago and the full suite caught, two are codex round 18 (web + internal/models). 1. remapFieldIDs PRESERVES EVERY BYTE IT DOES NOT DELIBERATELY CHANGE. 3e7a6751 fixed the substring-corruption defect by unmarshalling the fields blob, walking it, and re-marshalling. That fixed the defect and broke TestImportRepairsANULInsideAFieldsBlob: re-encoding rewrites the WHOLE blob, so a stored `�` ESCAPE came back as the literal replacement CHARACTER. stored fields = "{\"note\":\"x<U+FFFD>y\"}", want "{\"note\":\"x\\ufffdy\"}" The rule I broke is one my own harness memory already carries in another costume: an escape sequence and the character it denotes are not the same artifact, and a round-trip through a decoder silently converts one into the other. A function whose job is to substitute ids must not be the thing that re-encodes everything else. Now a textual substitution of the QUOTED JSON token — `"c-1"` does not occur inside `"c-10"`, which is the whole-value property the fix needed, and it still reaches ids inside arrays for a multi-valued relation. Untouched bytes are untouched. WORTH KEEPING: my four targeted tests all passed on the broken build. Only the full SQLite suite failed, in a test named for NUL handling that has nothing to do with relations, in a package I was not editing. This is the argument for running the whole gate rather than the tests I judged relevant — the fix and the test that catches it were three directories apart. Mutant: replace the quoted token with the raw id -> FAIL, the dangling value is rewritten. Both the prefix-collision test and the NUL test are green on this tip. 2. THE `referent_not_portable` MESSAGE ASSERTED SOMETHING THE RESPONSE DOES NOT SAY (codex round 18). The dialog rendered "it points at something in the source workspace". That reason is emitted for EVERY carried cross-workspace relation WITHOUT resolving the target, so the value may name a live item, a deleted one, one the caller cannot see, or nothing at all — and `github_pr` reaches the same reason, where the referent is not in any workspace. The sentence claimed both existence and location. Now "this reference cannot be carried to the destination". Same class as round 12's `not_found` overclaim, and the neutral-wording comment explaining why THAT one is careful sits four lines below the one that was not. A rule written next to its own exception is easy to read as already applied everywhere it belongs. SHIPS WITHOUT A TEST, named here rather than left looking covered: the reason-to-message mapper is inline in the .svelte component and not exported, and there is no test file for the dialog at all. Extracting it is a refactor this unit should not take mid-flight — filed as an idea instead, because two separate review rounds have now found defects in this one unexported function and nothing can regression-test either fix. 3. THE WEB `Item` TYPE NOW MIRRORS `models.ItemWriteWarnings` (codex round 18). The Go item has carried `warnings` on create/update responses since BUG-2850, and TASK-2878 added `dropped_fields` and `unresolved_relations` to it. Nothing on the TypeScript side mirrored any of it, so typed frontend code could not read a warning the server was already sending. Additive and fully optional, matching the Go struct's `omitempty` on every member. Gates on this tip: build, go vet, gofmt, `npm run check` 0 errors (6 pre-existing warnings in unrelated files), `make web-test` 124 files / 2108 tests passed. The full SQLite suite and Postgres are owed on this tip and are NOT claimed here — the SQLite run on 3e7a6751 is the one that failed above. Claude-Session: https://claude.ai/code/session_01Xk9M5UVPdc84xL5E1mZkm8 * fix: bulk move refused a required field the same request supplied (TASK-2878) Codex round 19 reviewed whether rounds 16-18's fixes broke anything. They did not — it states explicitly that the collapse, the carried-set recomputation, the notDefaultKeys change and the non-string-default drop do not narrow owner behaviour or alter non-nil / never-supplied override behaviour. That negative result is the round's main product, given that three earlier fixes in this unit each opened the next round's defect. It raised three findings. One is fixed here; two are recorded for a ruling. FIXED — BULK MOVE CHECKED A REQUIRED-FIELD ERROR COMPUTED BEFORE THE OVERRIDE EXISTED. `MigrateFields` records `required field "status" has no value` when a source `status` holding a select value cannot migrate into a destination `status` declared as a required RELATION. The caller supplies a perfectly good referent in the same request, `result.Fields["status"]` is set from it — and the check that follows reads `result.Errors`, which was computed before any of that. So the move was refused for a field the request had just filled, and the item stayed put. This is the defect PLAN-2357 DR-12 fixed at the SINGLE move door. The comment there says it in as many words: "an override that SATISFIED a required destination field still 400'd". Nobody swept the fix to the bulk door, and it became reachable when round 11 established that `req.Status` is caller input. Third instance in this unit of a rule fixed at one door and not at its siblings, which is the CONVE-18 shape at the level of DOORS rather than call sites. Filtering the error list rather than adopting the single door's validate-the-merged-map shape, deliberately: `ValidateFields` below already covers the merged map, and switching this check to it would change the error code for a genuinely-missing required field from `missing_required_fields` to `validation_error` — a compatibility break to fix a defect that does not need one. The filter matches MigrateFields' exact rendering per supplied key, not a substring, so a key whose name contains another key's name cannot collide. Mutant: restore the unfiltered `result.Errors` -> FAIL, the move is refused for the supplied field. Control leg included: with NOTHING supplied, the move must STILL be refused AND still carry `missing_required_fields`, so the test fails against a build that dropped the check rather than narrowing it. RECORDED, NOT FIXED — the copy and its preflight disagree about error PRECEDENCE in two cases. Both doors REFUSE in both cases and both refusals are non-disclosing; what differs is which error wins. a. Overrides `{"owner_ref":"<invisible live item>", "ghost":"x"}`: the preflight returns 400 `malformed_override` for the undeclared key, the copy returns 400 `validation_error` for the invisible relation. b. Override `{"owner_ref":42}`: the preflight returns 400 `invalid_override`, the copy returns 400 `validation_error` from store field validation. Both are real — the pair is specified to give one answer to one body — and neither is a security or data defect. Closing them means choosing a precedence and applying it at two doors, which is a response-contract decision and a reordering of error handling in the MUTATING path. In a unit where three fixes have each opened the next round's defect, that is not a change to make unilaterally at the end of a session. Left for a ruling, with the two concrete bodies above so whoever takes it does not have to re-derive them. Gates: build, go vet, gofmt clean on this tip. SQLite and Postgres were BOTH green on f601a2a1 (SQLite server 372.7s / store 376.0s / items / mcp 24.7s, EXIT=0; PG store 525.8s / server 267.8s, EXIT=0) and are owed again on this tip; they are NOT claimed here. Claude-Session: https://claude.ai/code/session_01Xk9M5UVPdc84xL5E1mZkm8 * fix: a legacy item with a non-string relation value was unmovable (TASK-2878) Codex round 20 was an ENUMERATION round rather than another confirmation round: every door and its three origin classes, every path a relation reason can reach a caller, every place a value is deleted from the field map, every pair of doors specified to agree, and every guard added by this diff with what breaks if it is removed. Nineteen rounds had each found something, and CONVE-24 is the reason the twentieth asked for the population instead. It returned five candidates. FOUR are this unit's deliberate decisions read against a literal contract, and all four are already pinned by tests that assert the current behaviour on purpose: - `fields_patch` and bulk field update leave an UNTOUCHED carried relation value alone. That is the carry rule itself. Resolving keys the operation does not mention would refuse an item's status change because of a legacy value in a field nobody touched. - Required destination defaults REFUSE rather than drop. A default is never refused for being invalid; a REQUIRED field left with no value is refused because the item cannot be stored valid. Round 3 established this. - Workspace archive import carries unresolved values with no relation validation, per Dave's import ruling. The fifth is real and is fixed here. A NON-STRING CARRIED VALUE MADE A LEGACY ITEM PERMANENTLY UNMOVABLE. `internal/items` has accepted anything in a relation field for as long as the type has existed, and this unit's carry rule exists precisely so those items stay usable — the move door's own comment says refusing carried values "would make every legacy item permanently unmovable". A carried value that is not a string was the case where that happened anyway. Both carried branches deliberately left it for `ValidateFields`, and `ValidateFields` REFUSES: single move: 400 invalid_fields, `field "owner_ref" must be a string (item ID)` bulk move: per-item validation_error, same sentence On every attempt, with no way for the owner to fix it through these doors at all — the value cannot be corrected by a move and the move cannot proceed past it. Round 6 established that the two migration MODES must not disagree about a malformed value, and they did not: they agreed on the wrong outcome. The drop is hoisted ABOVE the mode switch, so they keep agreeing by construction and what they agree on is drop-and-report. `invalid_shape`, not `not_portable` or `not_found`, because the value is not a reference and every reason describing a LOOKUP is a false account of why it is going — the same reason and disposition the destination-default branch uses and that `ResolveLateRelationDefaultsQ` has always used. All three now agree. The cross-workspace branch's non-string check is kept as an unreachable type assertion rather than deleted, so that branch cannot silently start treating a non-string as a ref if the hoisted loop is ever narrowed. Mutant: remove the hoisted drop -> FAIL on both the `single move` and `bulk move` legs. Control leg included: a carried value that RESOLVES must still survive the move, so the test fails against a build that dropped every carried relation rather than only the malformed ones. Round 20's direct answers, recorded so the next round starts from them: the only remaining ways to store an unresolvable value, to learn an invisible item exists, or to have a value discarded without a report are the already-recorded ones — IDEA-2886, IDEA-2893, the archive-import ruling, the carry rule's own untouched-key posture, and the two precedence divergences awaiting a ruling. It found NO additional uncollapsed `wrong_collection` path and NO additional validation-induced silent deletion. Gates: build, go vet, gofmt clean; the full relation suites green (`internal/server` 43.041s, `internal/store` 0.936s). SQLite and Postgres were both green on 7573d042 and are owed again on this tip; NOT claimed here. Claude-Session: https://claude.ai/code/session_01Xk9M5UVPdc84xL5E1mZkm8 * fix: the eighth door's visibility pass, override precedence, refilled carried drops (TASK-2878) The lead's day-58 ruling, all three items, plus the regression my own first attempt at item 2 introduced. 1. THE EIGHTH DOOR GETS THE VISIBILITY PASS (night-11 finding, ruling shape (b)). `dropInvisibleRelationDefaults` reached five call sites, all in `internal/server`. The SIXTH caller of the late-default resolver is `store.migrateCopyFields`, inside the cross-workspace copy's transaction, where a *Server method cannot go — so the copy STORED a destination default's canonical id for a caller who cannot see the collection it points into, and handed it back in the 201, while the PREFLIGHT reported the same key dropped for the identical request. One request, two answers. Round 15's sweep said "applied at all five late-default sites, per CONVE-18" and was right about the sites it could see. THE CLASS IS ONE WIDER THAN THE PACKAGE. That is this unit's recurring lesson — a rule applied at one door and not its siblings — one level up, at the package boundary rather than the call site, and it is why a sweep should name the VANTAGE POINT it enumerated from. Shape (b) as ruled: `checkItemVisibleQ` — which already exists parameterised over its executor for exactly this (BUG-2409) — is bound into a `store.RelationVisibilityFunc` closure by the server and threaded into the copy request, so the copy runs the rule on its own transaction while holding both workspaces' advisory locks. The body moved to `store.DropInvisibleRelationDefaultsQ` and the server's method now delegates to it, so both doors run ONE implementation rather than two kept in step by hand. `notDefaultKeys` moved to `store.NotDefaultKeys` for the same reason. A nil callback means no check, for internal callers with no requester whose visibility could be evaluated. Pinned with the night reviewer's three legs. Leg 3 is what makes it a test: the OWNER's identical copy must still resolve and store the default, so a build that dropped every default cannot pass. Mutant: remove the pass -> FAIL. 2. OVERRIDE PRECEDENCE: STRUCTURAL BEFORE SEMANTIC, ONE CLASSIFIER (ruling). The preflight's order is the contract. `structuralOverrideError` decides both structural problems — an undeclared key (`malformed_override`) and a wrong-shaped value (`invalid_override`) — and BOTH doors call it before any semantic check. The copy previously reached `refuseInvisibleRelationOverrides` first, because that check lived in the handler while the structural ones lived inside the store call. Nobody chose that order; it was an emergent property of where each check happened to live, which is why the fix is one function rather than two orderings maintained by hand. Both of round 19's bodies are pinned AT BOTH DOORS in one table test, on one body per case: `{"owner_ref":"<invisible>","ghost":"x"}` -> malformed_override, `{"owner_ref":42}` -> invalid_override, with a leg asserting nothing was written. Asserting them per-door is what let them drift, since each door's own test passed. Mutant: restore the semantic-first order at the copy -> FAIL on both cases. MY FIRST VERSION OF THIS BROKE COERCION, and the full suite caught it, not my targeted tests. The classifier validated the raw overrides, so `{"cost":"42"}` against a number field — coercible, and pinned as acceptable at both doors by TestCopyAndPreflightCoerceIdentically — was refused `invalid_override`. It now coerces the probe first, as both doors do. Hoisting a check above the coercion that makes it pass is the same class of error as the ordering this function exists to fix, committed while fixing it. 3. A CARRIED RELATION DROP REFILLED BY A DESTINATION DEFAULT IS REPORTED (ruling). Round 3's `StillDropped` filter suppresses a key the final map has a value for, because reporting a populated key as dropped made three surfaces give two answers. Right for a key `MigrateFields` dropped and a default refilled — the caller never had anything else there. Wrong for a CARRIED RELATION: the source's value was genuinely discarded and what sits in the key is a different value the destination chose, so the row said nothing was lost when something was. Scoped to relation drops deliberately; the type/schema-mismatch case is round 3's, unchanged. The preflight already discloses this in its carried row's `"from":"default"`, so the copy adds no new information — it makes the copy say it too, which is the agreement. The test asserts the drop IS reported AND that the destination default still lands. Gates on this tip: build, go vet, gofmt clean; `internal/server` ok 283.247s and `internal/store` ok 301.461s (SQLite). Postgres, the full four-package SQLite gate and CI are owed on this tip and are NOT claimed here. Claude-Session: https://claude.ai/code/session_01Xk9M5UVPdc84xL5E1mZkm8	2026-09-05 10:07:44 -04:00
xarmian	b437cc582d	feat: item reminders — the fire-at-an-instant primitive, and one overdue rule for all four surfaces (IDEA-2641, closes #1010) (#1244) ... * feat(store): item reminders — the fire-at-an-instant primitive (IDEA-2641) Adds the storage, the scheduler tick, and the canonical event for one-shot item reminders (GitHub #1010). Nothing in Pad acted at a target time before this: a due_date makes an item show up as overdue once somebody asks the dashboard, so "revisit TASK-X on the 1st" had to live in an external cron. A TABLE, NOT A SCHEMA-FIELD ANNOTATION. The design sketch proposed marking schema date fields with a `reminds: true` key on models.FieldDef; recon overturned it. Such a key does not survive an ordinary collection edit, two independent ways: the web editor destructures each field into an EditableField and rebuilds a fresh definition key-by-key on save, so unknown keys are dropped (`pattern` and `unique_scope` survive only because two lines were hand-added for them), and models.CollectionSchema has fixed fields with no catch-all, so any Go unmarshal+marshal round-trip strips unknown properties — the hazard retargetRelationFieldsTx mutates raw JSON to avoid. Both failures are silent and both disarm a whole collection's reminders at once. It is the same defect class that moved traits out of the schema column in TASK-2657. The table also gives the lifecycle a home. A reminder is armed, then fired, then acknowledged, and a re-arm returns it to armed — per-reminder state a field definition has nowhere to keep. remind_at is an RFC3339 UTC instant, deliberately not a `date` schema value: those admit both YYYY-MM-DD and full RFC3339 and are compared against the SERVER'S LOCAL calendar day. A fire-at time cannot carry that ambiguity. The remaining timezone question for due_date is filed separately. Firing is one transaction per reminder carrying BOTH the fired_at write and the outbox insert. That pairing is the point: a fired_at committed without its event is a reminder that silently notifies nobody and can never be retried, because the row has left the armed set; an event without fired_at fires every tick forever. The UPDATE's own `fired_at IS NULL` predicate is the arbiter, so two instances ticking at once produce exactly one winner. item.reminder_due is admitted to the closed events/1 set as v1.2, with a new PayloadReminder family and no SSE name. The subject is the REMINDER, not the item: two reminders can be armed on one item, so an item-subject event could not say which fired, and the reminder id is what an acknowledgement addresses. A new payload family rather than reusing the item snapshot for the same reason — a snapshot would validate and still not answer the only question the event exists to answer. No SSE name in v1 because the poll surface is the contract; adding one later is additive, removing one is not. Ack is explicit and nothing else acks. An item reaching a terminal status deliberately does NOT ack: that would make every status write a reminder mutation, and it would silently consume a reminder set to fire after the work was done. * feat(server): reminder surfaces, and one shared overdue rule for all four Second half of IDEA-2641: the HTTP surface, the scheduler tick's wiring, and the fix for the finding that justified the unit — `ready` / `next` did no date handling at all. OVERDUE NOW HAS ONE IMPLEMENTATION. It used to live inline in the dashboard's attention loop, which meant `pad project stale` inherited it (it filters that very list) and the recommendation surface never saw it. So a deadline reached the two surfaces that REPORT on work and never the one an agent PULLS from. overdue.go is now the only place that decides, and all four call it. Two behaviour changes fall out, both deliberate: - An overdue item bypasses the orphan branch's high/critical priority gate. That gate was where a deadline quietly stopped: a low-priority item three weeks late was reported by `stale` and never suggested by `next`. - Overdue sorts above in-progress. The list is capped at three, so a rank below in-progress would not merely order the deadline lower — on any workspace with three things in flight it would keep an overdue item off the surface entirely, which is indistinguishable from not shipping this. The server-local-today comparison is UNCHANGED and known to be wrong for multi-timezone deployments; it is filed as its own item with the cloud case stated. Changing what "overdue" means on every existing instance inside a change about where the rule LIVES is the kind of behaviour change nobody reviews. Fired reminders reach `next` / `ready` two ways, from one filtered list: PendingReminders is the addressable form (it carries the id an ack needs), and a prepended suggestion is the rendered form. They are prepended AFTER the cap rather than entered as ranking candidates — a reminder is not a task competing on priority, and whether it appeared should not depend on how busy the workspace is. Terminal-item reminders are FILTERED from the surface, never acked. Acking on terminal status would couple every status write to reminder state and would consume a reminder armed to fire after the work was done. The row stays exactly as the user left it; the distinction is observable, and asserted. Three guard tests caught this change and each was answered rather than silenced: - The request-body reader guard was right: the handlers now go through decodeJSON, inheriting the NUL refusal and the size cap. - The canonical-events guard was right: item.reminder_due is admitted to the duplicated contract table as SPEC-3 v1.7, with the reminder subject kind and the new payload family. SPEC-3's own text owes the same amendment. - The NUL census asked for a decision on eight new columns. None carries caller text: ids and FKs are server-generated, four are the server clock, and remind_at is now re-parsed and re-formatted in the STORE as well as at the edge — so the stored value is always machine-produced from a parsed time and no caller bytes reach the column. The doc comment that used to say "the caller normalizes" protected nothing. Regenerating the baseline also found that GEN_NUL_BASELINE=1, which the test's own instructions name, was never implemented — the flag did nothing, so the documented path was hand-editing the file. Implemented, so the next reader gets the mechanism the instructions promise. * test(reminders): the lifecycle, the four surfaces, and 22 killed mutants Every test here was designed against a specific mutation and the mutation was RUN. A green suite proves nothing about a suite nobody tried to break, and three of the mutants I first wrote were not experiments at all. Store (10 mutants, all killed): candidate predicate <= flipped to >=; the event emission lifted out of the fire transaction; the fire UPDATE's `fired_at IS NULL` arbiter removed; the RowsAffected check ignored; re-arm clearing fired_at but not acked_at; ack losing `fired_at IS NOT NULL`; the poll surface losing `acked_at IS NULL`; normalizeRemindAt no longer refusing; it dropping .UTC(); GetReminder losing its workspace scope. Surfaces (12, all killed): the priority gate no longer bypassing on overdue; the sort no longer ranking overdue first; attention leaving the shared helper; the reason losing its OVERDUE prefix; the comparison flipped to >; terminal items no longer skipped; terminal reminders no longer filtered; the filter ACKING instead of hiding; reminders appended instead of prepended; the tick running on a far-future clock; ack answering 200 for an unfired reminder; parseRemindAt accepting a bare date. THREE MUTANTS DID NOT COUNT ON THE FIRST PASS and were rewritten. Two failed to compile (`if false` orphaned a variable; deleting a parse orphaned an import) and one had an anchor matching two call sites. A non-compiling mutant emits zero FAIL lines and reads exactly like a surviving one — it invents a hole that is not there — so the harness reports BUILD-FAIL and ANCHOR-BAD as outcomes distinct from SURVIVED. It also restores files from an in-memory copy rather than `git checkout`, which would delete uncommitted work in the tree. ONE MUTANT GENUINELY SURVIVED and the test was at fault, not the mutant: appending rather than prepending reminder suggestions was undetectable because the fixture had a single item, so the reminder sat at index 0 either way. The fixture now fills the three-item cap with in-progress work, where an appended reminder lands fourth and vanishes. Faithful mutant, weak test — checked in that order. The same lesson shapes the four-surface fixture: it is a LOW-priority open orphan, because that is the case the old code handled worst. A high-priority task would have made the ready/next leg pass against the unfixed tree, which is a green that measures nothing. Negative controls throughout: a future deadline is not overdue and does not reach the gate bypass; a tick with nothing due fires nothing; a completed item is neither overdue nor suggested. Without them a helper that reported every date, or a tick that fired everything, would satisfy every positive leg. The lead's pin is asserted in both directions: a fired reminder on a done item is ABSENT from the surface and PRESENT and still unacknowledged in the table. Asserting only the absence would pass against an implementation that consumed the row, which is the behaviour the pin exists to forbid. * feat(mcp): pad_item.remind + ack-reminder, ToolSurfaceVersion 0.28 An agent that can RECEIVE a reminder but not set one has half the primitive. The poll surface is pad_project.next / ready, both long exposed, so reminders already reached agents — what was missing is the other half: deferring a piece of work is exactly the moment an agent knows when it wants to be asked again, and it had no way to say so. Two additive actions, two optional params. Nothing existing moved, so a v0.27 consumer enumerating neither is unaffected — the v0.13 / v0.11 / v0.8 disposition, which likewise wired existing CLI verbs onto the catalog. remind_at REFUSES a bare date rather than reading it as midnight. Worth stating because the `date` schema type accepts YYYY-MM-DD and a caller will reasonably try it here: a bare date names a 24-hour span, and choosing an hour inside it would fire at a time nobody picked. Re-arm and disarm stay CLI-only. Both address a reminder by an id the agent would have to list first, and no listing action exists on this surface — a door with no handle. Adding them later is additive. Five guards had to be taught, and each was answered on its merits rather than excluded: the HTTP parity test (route mappers added, so the actions work on the remote transport rather than being advertised and unrouted), the read-only catalog's cmdhelp fixture and expected cmdPath map, the field- conflict classifier (remind_at / reminder_id are NOT field writers — a reminder is a row in its own table addressed by its own id, so listing them as classified sources would have pointed detectFieldConflicts at something that is not a field source), and the instructions.md / README action tables. That machinery is why the version bump is safe to make now, and it earned its keep on this change: every one of the five failed on the first build after the catalog entry landed. CONVE-23 sweep for prose this falsifies: - SPEC-3 (DOC-2653) amended to v1.7 in the room, recording item.reminder_due with its new subject kind and payload family — the first canonical event with no user mutation behind it, since a scheduler tick produces it. - CLAUDE.md gains the reminder routes, the CLI verbs, and the v0.28 entry. It was also stale at 0.26 with NO v0.27 entry at all: the 0.27 unit swept instructions.md and README.md and missed this file. Both added. - skills/pad/SKILL.md gains the verbs and a routing entry, including the two things an agent will get wrong — the time is an instant, so ask for a time of day rather than picking one, and finishing the item does not acknowledge the reminder. * fix(reminders): codex round 1 — four findings, all real, all with a pin Round 1 found four defects and refuted none of them. Each fix carries a test that fails against the code as it was, and each of those was mutation-checked. **P1 — pending reminders bypassed item-level visibility.** Every other dashboard section reads `allItems`, which the store already scoped to the caller's collections AND their granted item ids. The pending-reminder list is a direct workspace-wide query and inherited none of that, so a guest holding a grant on ONE item could read the refs and titles of every other item in the collection through its reminders — an item-level leak wearing a notification's clothes. Now filtered with the same `isItemVisibleToGuest` call the sibling sections use. The test's two items share a COLLECTION on purpose: a collection-level filter was already applied, so separate collections would have made it pass against the unfixed code. **P1 — soft-deleted items could starve the queue permanently.** Candidate selection ignored `deleted_at`, and `fireOneReminder` rolls back when it finds the item gone — which leaves the reminder ARMED and therefore a candidate again on the next pass. Candidates are ordered oldest-first and bounded by a limit, so enough archived reminders fill every batch and no live reminder ever fires. Silent, too: the tick reports zero fired and looks idle. Excluded in the candidate query rather than skipped downstream, so those rows never occupy a slot; the reminders themselves are kept, so restoring an item restores its reminder with it — asserted, because a fix that reaped them would pass the starvation test alone. **P2 — the pass stopped at the first failing reminder.** The per-reminder transaction exists precisely so one unfireable row cannot hold back the rest, and `return fired, err` made that comment false — with candidates oldest-first, one persistently broken old reminder blocks every newer one forever. Now continues and joins the errors, so a pass that fired seven and failed three reports both halves rather than reading as clean. The loop is split behind an injected seam because a real mid-transaction failure is not reachable from outside: the database refuses the corrupt rows that would cause one (verified — invalid JSON in items.fields is rejected by the schema). **P2 — suggestions dropped the reminder id.** The docs tell an agent to acknowledge what it sees in next/ready, and the payload carried no handle: a stateless poller could read the reminder and had no way to retire it, so it would be shown the same item forever. `DashboardSuggestion` now carries `reminder_id` (omitempty), `pad project next` prints the exact ack command, and the test acks with the id the surface handed out rather than merely checking the field is populated — a wrong-but-present id satisfies equality with itself. Four mutants, four killed; one was rewritten first because its anchor matched two call sites and was therefore not an experiment. * fix(reminders): codex round 2 — four findings, all real **`--rearm` was unusable.** `ExactArgs(1)` forced an item ref that the rearm branch then ignored, so the flag could not be reached without supplying a ref that was silently discarded. Now `MaximumNArgs(1)`, with each mode checked explicitly: a ref is required to arm, and a ref supplied ALONGSIDE `--rearm` is refused rather than ignored — it names an item the reminder may not even belong to, and quietly dropping it is how a user learns nothing about the reminder they just moved. **`unremind --format json` emitted plain text**, breaking the parseable-output contract every sibling command honours. **The MCP `ref` param did not list `remind`.** Agents read that flat description to decide what to send, so an action missing from it is an invalid call waiting to happen. It now also says what `ack-reminder` takes instead, and why: a reminder is addressed by its own id because an item can carry several. **Fractional seconds fired early.** `time.Parse` accepts `09:00:00.900Z` and `Format(RFC3339)` drops the fraction, so it was stored as `09:00:00Z` and fired 900ms BEFORE the moment the caller named — silently, having rewritten their value on the way in. Seconds are genuinely the stored resolution (the column is compared as a string against a whole-second clock, and the tick runs every 30s), so the only question was which way to resolve it, and truncation resolved it the wrong way. `NormalizeInstant` now rounds UP: at most a second of lateness, in exchange for a guarantee that can be stated — a reminder never fires before the instant it was set for. Late is a reminder; early is a wrong answer. Whole seconds round-trip exactly, which is asserted, because an implementation that added a second unconditionally would otherwise pass. Three mutants for this round, three killed (round-up→truncate, round-up→unconditional-add, MaximumNArgs→ExactArgs). Thirty across the unit. Two fixes carry no dedicated test and it is worth being explicit rather than implying coverage: the `--format json` branch on `unremind` is a one-line output change with no server-free way to drive it, and the MCP `ref` description is prose the drift tests do not read — they assert an action is DOCUMENTED, not that a param's sentence lists it. * docs(reminders): the ack id is on the surface an agent polls, not only on the arm response CONVE-23 follow-through on the round-1 fix. Both agent-facing docs told a caller to acknowledge a reminder with the id "returned when you armed it" — true, and useless to the caller that matters: a poller reading next/ready never armed anything. The suggestion now carries reminder_id and `pad project next` prints the exact ack command, so the docs say that instead. The prose was written before the fix existed, which is exactly the case CONVE-23 is about: a change that makes an instruction stale without touching the file the instruction lives in. * test(reminders): bind the tick LOOP to the work, not just the pass (CONVE-19) Every other test in this file calls runReminderTick directly. That vouches for the component and says nothing about whether anything ever calls it — a tick that is never started is indistinguishable, from those tests, from one that is. It is the convention's exact case, and the failure I recorded on my own identity doc three times in one unit: I test the component and not the binding. Driven through the injectable tick channel so the assertion pins a SPECIFIC pass instead of racing a 30-second ticker, and polled to a bounded deadline so a loop that never runs FAILS rather than hanging the suite. Mutant: drop `s.runReminderTick()` from the select and this goes red while every direct-call test stays green. Killed. The idempotence leg exists because a second Start spawning a second loop would leave one running after Stop, making the BUG-842 drain invariant false for this sweeper specifically — the one property a copied lifecycle is most likely to get right by accident and least likely to be checked. The cmd/pad call site (cmd_server.go, alongside StartTokenReaper) stays verified by inspection: a source-scanning guard for it would be an instrument asserting facts about source, which is code with an adversary and not worth it for one line that sits in the middle of five identical neighbours. * fix(reminders): codex round 3 — a deferred reminder fired anyway, and the poll surface was unbounded **A re-arm mid-pass did not stop the fire.** The candidate scan selects an id; before the UPDATE runs, a `--rearm` can move that reminder into the future. Re-arm clears `fired_at`, so a predicate checking only `fired_at IS NULL` still matched — the pass fired a reminder the user had just deferred and emitted its event. The re-arm cannot undo that: it can clear the mark, but the event is already on the outbox and at-least-once means a consumer has seen it. The fire UPDATE now revalidates `remind_at <= nowTS` against the SAME nowTS the candidate scan used. Same-value deliberately: the arbiter and the scan must agree about when this pass is, or a reminder could pass one and fail the other for no reason but clock drift inside a single pass. **The poll surface was unbounded.** Every fired-and-unacknowledged reminder was loaded and turned into a suggestion prepended to a list that is otherwise capped at three, so a workspace with five hundred unacknowledged reminders returned five hundred suggestions — in the dashboard response, the hottest read in the product, growing until somebody acknowledged them. Two bounds, because they are two different guarantees: the query takes a window (default 50, oldest-fired first, so it holds what has waited longest), and the prepended suggestions are capped at 5 so `suggested_next` stays a recommendation rather than a second inbox. The full set stays addressable in `pending_reminders`. Truncation is REPORTED as a boolean, not a count. A count would have to be post-visibility-filter to be true for the caller reading it, and the store cannot compute that — the filter runs per item, above. "There are more than you can see here" is the strongest claim the data supports, so it is the one made. Four mutants; two killed outright, two survived and were run down under CONVE-28: - **Uncapped suggestions survived because the fixture had ONE reminder** — capped and uncapped are the same list at n=1. That is the SECOND time a single-item fixture hid a count-or-order property in this file. Fixture now arms eight; it also asserts all eight remain in `pending_reminders`, so the cap is pinned to the recommendation and not to the data. - **Removing the SQL LIMIT survived, correctly, and the test comment now says so.** The Go slice cap bounds the PAYLOAD; the SQL LIMIT bounds the DATABASE'S work. Only the first is observable at this level — with the LIMIT gone the response is still bounded, while the query silently goes back to materialising every pending row before discarding most of them. That is a memory and I/O property with no assertion available here, so it is stated as a coverage boundary rather than papered over with a green that would not have measured it. * docs(reminders): the fire predicate arbitrates against two actors, not one CONVE-23 inside the file the round-3 fix touched. The comment described the UPDATE as an arbiter for concurrent TICKS, which is what it was written for and is why I did not re-read it when asked whether a user edit could race the pass. It now says what it actually defends against, and names the general shape: an arbiter is only an arbiter with respect to the writers it can see. * fix(reminders): codex round 4 — the round-3 bound recreated the round-1 starvation Round 3 bounded the poll surface. Round 4 caught what that bound did: the query took the first N rows and the dashboard then discarded the ones it could not show — hidden items, unauthorised items, completed items — so N such rows hide a visible reminder behind them indefinitely, with no continuation to reach it. That is the SAME defect I had removed from the fire path one round earlier, reintroduced in the read path within the hour. The general form is worth stating because I clearly did not hold it: **a bounded window is only safe when the discarding happens BEFORE the bound.** Filtering above a limit is a starvation every time, and it does not matter what the filter is for. Two halves, because the two filters are not the same kind of thing: **Visibility is now scoped IN SQL**, using the same collection-id / item-id sets every other dashboard section gets through `allItems` — the same three-way shape as ItemListParams, where holding both collection grants and item grants is an OR. Invisible rows no longer occupy the window at all, which is strictly better than filtering them out afterwards and is what the sibling sections have always done. **Terminality is paged**, because SQL cannot evaluate it — a collection's schema defines which statuses are terminal. The collector refills from the next page when a page comes back short, bounded by a max scan so a workspace full of completed items cannot turn a dashboard read into a table scan. The bound is 10x the window: the common shape fills on the first page, and the pathological shape terminates in a fixed number of indexed reads. Stopping at the scan bound reports truncation, which is honest — there may be more, and we did not look. The empty-scope case is a THIRD state that reads like the second: nil CollectionIDs means unrestricted, a non-nil EMPTY slice means this caller sees no collections. Without an explicit guard they collapse, because the switch matches none of its cases at length zero and adds no clause at all — so "nothing visible" would return the whole workspace. Three mutants, one survived: the empty-scope guard, because no dashboard-level test produces that state (callers that would are refused earlier by workspace access). Faithful mutant, missing test — it now has a direct one, with a sanity leg so a build returning nothing cannot pass it by accident. A guard for a state nothing exercises is exactly the one that rots. * fix(reminders): codex round 5 — the MCP action I shipped did not work over stdio **P1: local stdio MCP `remind` was unusable.** cmdhelp derives positionals by regex from a command's `Use` string, and `<instant>` inside `remind <ref> --remind-at <instant>` matched — it became a second REQUIRED positional, so dispatch failed with `missing required argument "instant"`. The action was advertised on a transport where it could not run. **The MCP catalog's own tests did not catch it, and the reason is the finding.** That suite builds its cmdhelp document BY HAND: I wrote `Args: mkArgs("ref")` in it, so the fixture agreed with what I meant rather than with what the CLI says. Five parity and drift tests passed against a document I authored to match my own intention — the "a test that agrees with whatever the table says is not a test of the table" shape, which the canonical-events test warns about in its own comment two packages away. The new test reads the REAL command tree via cmdhelp.Build, which is the only thing in this repo that can disagree with me about what the CLI declares. **P2: `pad project ready` withheld the ack handle** that `next` prints. Showing a fired reminder on the surface an agent polls while withholding the id it needs to retire it means the same entry comes back on every poll, forever. **P2: suggestions asserted a collection they did not have.** The orphan branch admits ANY collection — its own comment claimed it gated on tasks "mirroring the active-plan branch", and that comment was simply false — while the output hardcoded `Collection: "tasks"` and the reason said "Open task". Pre-existing for high-priority items since BUG-1082; my overdue bypass widened it to any overdue item, which is how it surfaced. Fixed by carrying the item's REAL collection rather than by narrowing the branch: narrowing would silently drop the non-task items this has surfaced for a year, and the defect is the mislabelling, not the inclusion. The false comment is replaced with what the code actually does. The first version of that test used an overdue IDEA and SKIPPED — ideas use `new`, and the branch requires `open` or an active status, so it never became a candidate. A test that cannot fire is a failed reconstruction, not a pass; the fixture is now a bug-like collection whose vocabulary contains `open`, which is the population the defect can actually reach. Three mutants, three killed. Forty-one across the unit. * fix(reminders): codex round 6 — reminders fired from soft-deleted workspaces **P1, and the only defect in this unit whose consequence leaves the process.** Workspace soft-delete deliberately keeps items for the 30-day restore window, so the candidate query's filter on the ITEM's deleted_at found nothing wrong — and the tick kept firing, emitting outbound webhook events for a workspace whose owner had deleted it, possibly while deleting their account. Both queries now join workspaces and require `w.deleted_at IS NULL`. Nothing is destroyed: a restored workspace resumes firing, which the test asserts, because "stops firing" and "is destroyed" are very different answers to someone who restores a workspace and only one of them is right. That test first failed for the WRONG REASON and the fixture was at fault: it counted every outbox row in the workspace, and item creation writes its own, so the assertion was satisfiable by the fixture itself and discriminated nothing. Scoped to the reminder event type. **`Use: "remind <ref>"` declared a requirement the command contradicts.** cmdhelp derives the machine-readable arg spec from that string, and `--rearm` takes no ref — so the published contract said "required" for something optional. The requirement is CONDITIONAL, which cmdhelp cannot express, so the honest declaration is `[ref]` plus the explicit check that names both call shapes. The round-5 test grew a `required` column, which is what makes this observable at all: asserting only the arg NAMES would have passed. **The pad_item tool description omitted both new actions.** The params were declared and the actions dispatched, but the prose an agent reads to decide what a tool can do did not mention them — discoverable only by someone who already knew to look. It now describes both, including the two things an agent gets wrong: remind_at is an instant, and nothing but an explicit ack retires a fired reminder. Three mutants, three killed. Forty-four across the unit. * fix(reminders): codex round 7 — one predicate for the scan and the arbiter Third instance of one class, so this fixes the SHAPE rather than the instance. The class: the candidate scan filters on something the fire transaction does not revalidate, so a change committed between them fires a reminder that no longer qualifies. Round 3 was a re-armed instant. Round 1's soft-deleted item was the same thing caught from the other side. Round 7 is a workspace deleted between the scan and the fire — the round-6 fix added the condition to the SCAN only, and the arbiter went on not knowing about it. Fixing those one at a time is what let the third happen. `reminderFireable` is now a single string that both sites reference: the scan asks it and the fire UPDATE re-asks it, so they cannot disagree, and a fourth condition is one edit in one place rather than two edits someone has to remember are paired. Written as a correlated EXISTS on item_reminders.item_id rather than a JOIN precisely so the identical text is valid in both a SELECT and an UPDATE, and the scan drops its table alias so the two uses are the same characters. What deliberately stays outside it: `fired_at IS NULL` and `remind_at <= ?` live on the reminder row itself, are already spelled identically at both sites, and folding them in would need a parameter order the shared form cannot express. Said in the comment so the omission reads as a decision. Both directions are now tested at the arbiter — a workspace deleted mid-pass and an item deleted mid-pass — because the item case previously relied on the item load coming back nil, and someone simplifying the EXISTS down to the workspace check alone would otherwise still see green. Three mutants, three killed: the arbiter dropping the shared predicate, and the predicate dropping each of its two halves. Forty-seven across the unit. * fix(reminders): codex round 8 — workspace export silently dropped every reminder WorkspaceExport is a hand-maintained field list, so a new table joins it only if someone remembers. Reminders did not: a backup/restore, or a SQLite→Postgres migration via `pad db migrate-to-pg`, dropped every pending reminder with nothing in the destination to show anything had gone. The line that list has always drawn is item-scoped workspace CONTENT (comments, links, versions — exported) versus per-user state (stars, watches — not). A reminder has no user column and hangs off an item, which puts it on the exported side. Stating the rule rather than just adding the field, because the next person adding a table needs to know which side they are on. LIFECYCLE MARKS ARE CARRIED, not reset. A fired-and-unacknowledged reminder is still owed to whoever armed it, so it arrives pending; an armed one whose instant has passed fires once on the destination's first tick, which is what would have happened had the workspace never moved. Re-arming everything on import would invent a schedule the user did not set. NULL rather than empty string for the unset marks — the lifecycle is defined by NULL-ness, and "" would make a never-fired reminder read as fired at "". TestMigratedTablesCoversTheExport caught the second half, which I would have missed: `pad db migrate-to-pg`'s NUL preflight decides what to REFUSE on from MigratedTables, so a table the migration copies and the preflight does not know about is a gap in exactly the guard that exists to prevent one. Added there too, with the reason it can never actually fire — every column is machine-produced, so it is listed for coverage rather than expectation — and the "six tables" prose it falsified is now seven. Two mutants, two killed: export dropping the block, and import discarding the marks. Forty-nine across the unit. * test(reminders): state the fire-path invariant and pin it from the invariant The lead's read on why rounds 4 and 7 were the same class: the fire path had no stated invariant, so each fix defended an instance. This states it, and derives the pin from the paragraph rather than from the bug history. THE INVARIANT: the candidate scan is a hint and may be assumed to prove nothing. Every condition that made a row a candidate is re-asserted inside the transaction that marks it fired, in the same statement that does the marking, so checking and writing are one atomic act. Worded as "the scan proves nothing" rather than as a list on purpose — a list invites the next person to add a condition to the scan and stop, which is exactly what happened four times here. TestFirePathInvariant is the pin: one table, one row per scan-side condition, each invalidating that condition in the window between the scan and the fire and asserting the same three things — nothing fires, no event leaves, the reminder is not consumed. The earlier per-defect tests are folded in as rows; they said the same thing one instance at a time, which is how four of these shipped. Adding a fifth condition to the scan without a row here should feel like an omission. It carries a positive control, because four cases that all assert nothing happens would pass against a build that never fires at all. The matrix immediately falsified a claim in the paragraph I had just written. I wrote that the item load inside the transaction is "for the payload, not for the check"; removing the item half of reminderFireable alone changes no observable behaviour, because the load then returns nil and the deferred rollback undoes the write. Item liveness is defended TWICE and a single-mutant experiment cannot say which guard is carrying it — removing both is what kills the test. Both are kept, the predicate is named as primary (the row never matches, so no write happens at all), and the asymmetry is stated: workspace liveness has no second line, which is why dropping ITS half does fail the pin. Six mutants: five singles plus the pair. Five killed alone; the item single survives by design and is documented as such rather than left as an unexplained green. Fifty-five across the unit. * fix(reminders): codex round 9 — one legacy row could hide every reminder **P1: items.item_number is NULLABLE and I scanned it into an int.** Migration 006 added the column to existing rows, so a pre-numbering item still carries NULL — and scanning NULL into an int fails the Scan, which fails the QUERY, which degrades the whole pending-reminder section. One old row, and the feature is dark for everyone in that workspace. ListWatchesForUser, which this query was modelled on, uses sql.NullInt64 for exactly this column. I copied its shape and dropped the part that handles the column's actual nullability — the same way of being wrong as the round-5 cmdhelp fixture: borrowing a form without borrowing what it knows. The legacy row now carries no ref rather than a fabricated "PREFIX-0", which would name a different item. **P1: export shipped reminders that import could only discard.** The items section filters on deleted_at IS NULL, so a soft-deleted item is not in the bundle and its reminder can never be reunited with it. My comment claimed the item_links rationale — round-trip the raw graph so a restore reunites them — which is true for links and false here, because links keep soft-deleted endpoints in the bundle and items do not. A link is a row ABOUT two items; a reminder whose item is absent is a dangling schedule. **P2: import wrote remind_at raw.** Import is a writer, and a bundle is not necessarily one this server produced — hand-edited, or from another instance. A local offset or a bare date would land in the one column every comparison downstream treats as a UTC instant, firing early, late, or never. It now normalizes like every other door. An unparseable value is SKIPPED with a warning rather than failing the restore, matching the lenient import-side precedent already in this file, and the raw value's LENGTH is logged rather than its content. Three mutants, three killed; two needed rewriting because the single-line form did not compile — reverting the nullable scan also requires reverting the render, and dropping the normalization orphans a variable. PROCESS FAULT, recorded because it makes this round's findings weaker than they look: I edited the tree while this review was reading it — committed the invariant work and ran five mutation experiments, which write and restore source, over the same files. A review binds to the tree it read and I moved it underneath. Every finding above was re-verified against the current tree before being acted on, and the next round runs with no concurrent edits. * fix(reminders): codex round 10 — one orphaned item aborted a whole restore An ORPHANED item — one whose collection is missing from the bundle — still gets an itemMap entry. It has to: the entry is written before the skip because parent resolution inside the same loop reads the map for items it has not reached yet. So `itemMap[x] != ""` is satisfied by an id that names no row, and inserting a foreign key to it fails (SQLite enforces FKs here via the DSN's `_pragma=foreign_keys(on)`; Postgres always does). The pre-existing mapping is the sharp edge. The aggravating half was mine: this loop treated a failed reminder insert as FATAL, where item_links and item_versions both skip, so one orphaned item carrying a reminder rolled back an entire 900-item workspace restore. A reminder is the least critical thing in a bundle and it had the strictest failure handling in the file. Both halves fixed: the loop gates on items that actually landed, and a failed insert warns and skips like its siblings. TWO GUARDS THAT ONLY DIE TOGETHER, and this is measured rather than assumed. Reverting either alone leaves the test green — with the map gate restored the skip survives the FK failure, and with the fatal return restored the gate means the insert never fails. Removing both is what fails it. They are kept as a pair because they defend the same failure at different depths (prevent the bad write / survive a bad write arriving some other way), and the pair is recorded in the code so a future reader does not delete one as dead after watching its mutant survive. Second time this shape appeared today; the first was item liveness on the fire path. The bundle in the test is hand-built, because ExportWorkspace cannot produce an orphan — which is the reason it needed a test. That shape only arrives from a hand-edited or foreign bundle, and surviving those is what import is for. Three mutants: two singles that survive by design, plus the pair that kills. Sixty-one across the unit. * fix(reminders): codex round 11 — four contract slips, one of them another unit's **suggested_next returned up to eight entries against a cap of three.** Round 3 prepended reminders PAST the list's own cap, reasoning they should not compete for slots. Every consumer — the web dashboard, `pad project next`, `pad project ready` — is written for three. Worse, it silently falsified a decision recorded elsewhere: BootstrapDashboard deliberately has no suggested_next_overflow_count BECAUSE this list is capped at three upstream, and its comment names raising that cap as the moment to add one. My change made another unit's reasoning wrong in a file I never opened. The combined list is now trimmed back to three, reminders still leading — a reminder can push a task suggestion out, which is the right way round, and the full set stays addressable in pending_reminders. My first version of that trim used `limit`, which is REASSIGNED above to len(candidates) — so on a workspace whose only entries are reminders it would have truncated to zero, killing precisely the case the surface exists for. Caught by reading the surrounding lines before running anything; it has its own test now. **pending_reminders was uncapped in the bootstrap projection.** BootstrapDashboard embeds *DashboardResponse, so every new field joins the boot payload automatically — here, a window of up to 50, which is the budget PLAN-1410 spent a unit trimming. Capped at 5 with an overflow count, under its own constant rather than borrowing bootstrapAttentionCap: they answer different questions and a future change to one must not silently move the other. **Truncation was reported from the wrong question.** The collector used the store's `more` flag, which answers "is there another PAGE", not "did I read all of THIS one" — so a window filling part way through the final page reported that the caller had seen everything while unread rows sat behind the fill point. The paging bounds are now injectable so the case is testable at all: building it with a window of 50 needs ~75 rows in a specific pattern, with a window of 3 it is four. **Import accepted acked-without-fired**, which is not one of the lifecycle's three states. Such a row fires, is excluded from the pending surface because it is already acked, and can never be acknowledged because AckReminder requires acked_at IS NULL — an event emitted into permanent invisibility. The acknowledgement is dropped and the schedule kept, since an ack of something that never fired means nothing. Five mutants, five killed (one rewritten — removing the flag orphans a variable). Sixty-six across the unit. * fix(reminders): codex round 12 — a read is not a hold; scope the arm; ack from the ack Four P2s from round 12 (two independent runs, both landing on the same line of the fire path), each closed at the layer where it lives: - fireOneReminder pins the item and workspace rows FOR NO KEY UPDATE on Postgres before the arbiter UPDATE. reminderFireable re-asserted liveness at the predicate's instant and nothing held it to the commit instant; under READ COMMITTED an archival could commit in between and the event left the process about a deleted resource. Same idiom and same lock strength as CreateAttachmentForLiveItem; SQLite is excluded by its BEGIN IMMEDIATE, not skipped for convenience. Two PG-only pins verify "blocked" in pg_stat_activity, not by elapsed time; the pin-removed mutant fails both. - CreateReminder asserts "live item of THIS workspace" in the INSERT's own SELECT and returns ErrReminderItemGone otherwise. The table had an FK and no same-workspace constraint; a mismatched pair fed another workspace's title to this one's dashboard and webhooks. Handler maps it to 404. - AckReminder matches every fired row (COALESCE keeps the first ack, updated_at moves only when acked_at does), so a no-match means exactly "not fired at the instant of the ack". The handler no longer decides 409-vs-200 from the row it read before the UPDATE. - The invariant paragraph gains its missing sentence: "at that instant" means the commit instant, and the pin is what makes the predicate's instant and the commit instant the same one. Round-12 caveat carried: both runs were static reads (sandbox blocked Go's build cache), so "four" is a floor, not a measurement. Refs IDEA-2641 * fix(reminders): codex round 13 — a reminder's workspace must agree with its item's, at every read Every reader scoped by r.workspace_id and then joined the item without asserting the two agree. No door writes a disagreeing row today (CreateReminder derives the pair from the item; import maps within the workspace), and the table has nothing that forbids one — so a hand-edited bundle, a future move door, or a direct write would carry one workspace's item into another's dashboard, export, and webhooks. The identity goes into reminderFireable (scan + arbiter), the Postgres row pin, ListPendingReminders and the export query. One test writes the row raw — the only way one can exist — and asserts it is inert at each site; the predicate-removed mutant scans and fires it. Refs IDEA-2641 * fix(reminders): codex round 14 — the by-id and by-item reads assert the same identity as every other read GetReminder scoped by the row's own workspace_id and ListRemindersForItem by item_id alone, so a row whose two columns disagree — the class rounds 12 and 13 closed at the scan, the arbiter, the pin, the pending surface and the export — was still readable through the two reads that reach a single row. reminderOwned is that identity on its own, without the liveness half those two reads must not have (a fired reminder on an archived item is history worth showing). The write paths reach a row only through GetReminder, so scoping it scopes them; a row no door can write needs no door to delete it. ListRemindersForItem now takes the workspace its caller already resolved the item in. The raw-row test asserts both reads refuse the row from both sides; the reminderOwned-removed mutant surfaces it through GetReminder. Refs IDEA-2641 * fix(reminders): codex round 16 — an archived item's reminders are readable, and its verbs say "archived" The doors resolved the item live. Listing an archived item's reminders answered 409 from a GET, and ack/re-arm/delete answered a bare 404 for a reminder that exists on an item that exists — while the store, since round 14, deliberately keeps that history readable. The API already has a posture for archived items: GET reads them, mutations answer 409 "archived … restore it before editing" (writeItemResolveError). The list now follows handleGetItem; the lifecycle verbs load the item include-deleted, run the visibility check first, and then answer the same 409 every other item mutation does. One test walks archive → list 200 / ack 409 / arm 409 → restore → ack 200 on the same rows. Refs IDEA-2641 * fix(reminders): codex round 17 — one suggestion per item, the archived 409 by slug, and the door courtesy named Three findings on the server pass. (1) An item that was both a fired reminder and an ordinary candidate appeared in suggested_next twice; the ordinary entry is dropped, the reminder entry (which carries the ack id) stays, and two reminders on one item remain two entries. (2) Round 16's 409 for an archived item's reminder was written by re-resolving item.Ref, which is derived and empty for a legacy item with no item_number — so the class most likely to be legacy fell through to a bare 404. The slug is handed over instead. (3) The archived check in resolveReminderForWrite is check-then-write, and an archive landing in between lets the verb through: accepted and documented — it is the posture of every item mutation here (UpdateItem's UPDATE has no liveness clause), the outcome is benign, and putting liveness in AckReminder's WHERE would re-create the no-match ambiguity round 12 removed. Refs IDEA-2641	2026-09-04 11:36:11 -04:00
xarmian	c2bb1bad34	fix(mcp,server): close three codex round-11 findings, one of them my own bad refutation (BUG-2850) ... Two P1 and one P2. The P2 is the important one, because I had already dismissed it in round 10 and was wrong. [P2 — CORRECTION] Artifact imports DO drop undeclared-field warnings, and the case is reachable. Round 10 refuted this on the grounds that artifact.Decode populates Fields only from FieldKeysForKind, so no undeclared key could arrive. That check was real, and it was the WRONG SIDE of the comparison: UndeclaredFieldKeys compares the field map against the DESTINATION COLLECTION'S SCHEMA, not against the artifact format's key list. The destination schema is editable, so a canonical artifact key can be undeclared THERE while being perfectly legal in the artifact. Verified before reinstating, not argued: narrow the conventions collection's schema to declare only `status`, import an ordinary convention carrying trigger/scope/priority — the blob stores all three and UndeclaredFieldKeys names all three. The merge is back, and the comment now records the correction rather than the refutation, so the next reader inherits the right reason. What I got wrong is worth naming exactly: I verified a true fact and then drew a conclusion one step wider than it supported, because I never asked what the OTHER operand of the comparison could be. "No key outside the artifact's list arrives" does not imply "no undeclared key arrives" unless the destination declares every key on that list — an assumption I never stated and never checked. The test I wrote at the time could not pass, and I read that as confirming the refutation instead of as the setup being wrong. [P1] An empty hierarchy value inside `fields` was a silent no-op. `fields:{"parent":""}` promotes onto the top-level `parent`, where both doors treat empty as NOT PROVIDED — so the call reported success and detached nothing. Refused now, pointing at clear_parent and the raw `field:["parent="]` form. Not silently promoted to a clear: that decides what this door MEANS, and v0.19 already made clear_parent canonical so the empty string would not have to carry it. [P1] The v0.16 compat ID params were not conflict-checked against `fields`. Never schema-declared by design, they are invisible to padItemPromotedFieldKeys and took the generic path, where the check only consults the `field` array. `assigned_user_id:"A"` with `fields:{"assigned_user_id":"B"}` made the doors disagree outright — the remote mapper reads A, stdio emits only `--field assigned_user_id=B`, because the top-level form has no CLI flag behind it. One call, two different people assigned. Conflicting values refuse; equal ones collapse to one form. Also corrected, per CONVE-23: the round-10 test asserting that `fields:{"parent":""}` conflicts with the plan alias now refuses for a DIFFERENT reason and its stated rationale had become false. The case moved to the new test with the right reason, and the field-array clear — which really is an effective directive — stays where it was, with a control leg proving the new refusal did not swallow it. Mutation matrix, each mutant from a file backup: revert the empty-hierarchy refusal -> only EmptyHierarchyValueInFieldsRefused fails (both subtests) revert the compat-ID check -> only CompatIDConflictRefused fails (both subtests) revert the import warning merge -> only the narrowed-schema import test fails gofmt clean · go vet clean · go test ./... green (29 packages)	2026-09-03 17:12:20 +00:00
xarmian	0a71ad3aa9	fix(mcp): an empty parent param is not a hierarchy directive (BUG-2850) ... Codex round 10: three P2, no P1. One fixed, one already filed, one refuted and reverted. [FIXED] An empty top-level `parent` was counted as an alias directive, so `parent: ""` with `fields:{"plan":"X"}` refused a perfectly good call. Every declared string param on this tool treats "" as NOT PROVIDED — it is why promotedParamValue does, and why `assign: ""` is deliberately inert — so a client that fills declared optional params with their zero value rather than omitting them got a refusal for asking one hierarchy question. My own round-9 snapshot carried this forward from the out[]-based check it replaced. Deliberately NOT applied to the other empty forms: `field:["parent="]` and `fields:{"parent":""}` are the documented CLEAR signal (BUG-2013 / BUG-2078), so they are semantically effective and still conflict. One is a param left blank, the other is an instruction that happens to look like one. Both directions are pinned, and the mutation matrix drives both: remove the empty-param exclusion -> only EmptyParentParamIsNotAnAliasConflict fails extend it to the field-array clear -> only EmptyClearFormsStillConflict fails [ALREADY FILED] The transport-dependent whitespace finding is BUG-2870, ruled out of this PR's scope. Round 10 did add something the filing missed and BUG-2870 now records it: the divergence covers VALUES too (`--field "cost= 3"` stores the number 3 remotely and the string " 3" over stdio), which is worse than the key half because both doors report success and only the stored type differs. [REFUTED, REVERTED] "Artifact imports discard createItemChecked's undeclared-field warnings." True as a code reading — this handler builds its own response shape and ignores item.Warnings — but the condition is unreachable. artifact.Decode populates Fields exclusively from FieldKeysForKind via a closed switch over a typed frontmatter struct, so a key outside that per-kind list never enters the map. Verified both ways before reverting, because the import door takes raw bytes and the hand-written case is the one that mattered: Encode drops extra keys, and Decode of a hand-written artifact carrying extra frontmatter keys drops them too. I had written the merge and a test for it before checking; the test could not pass through the public door, which is what exposed the finding rather than my fix. Reverted to a comment recording the mechanism, so the next reader — or the next round — does not re-find it. A branch nothing can enter is not defence in depth, it is a claim that something is handled when it never happens. gofmt clean · go vet clean · go test ./... green (29 packages)	2026-09-03 16:53:14 +00:00
xarmian	7f25283a41	test(server): pin the last three coercion call sites (BUG-2850) ... Five of the eight `CoerceFields` sites had a test that goes red if that site alone is dropped. Move, bulk move and bulk update did not, so the PR's "typed on every door" claim rested on reading the code — CONVE-19 and the shape rounds 2-5 of this unit kept finding. Why the move pins are faithful rather than green-for-free: migrateValue already permits text->number (migrate.go:190), but it returns `value`, the ORIGINAL, not a parsed float. So a text field holding "42" reaches a number-typed destination as the STRING "42", and only CoerceFields at the move site turns it into a number before validation. Assertions are on the STORED NATIVE TYPE, re-read from the item rather than taken from the mutation's own response, so a handler that answered 200 and stored the string is still red. Bulk update merges request STRINGS (status, priority), so it is observable only where the schema declares one of those keys as a non-string type; a collection declaring `priority` as a number is unusual but legal and is the honest way to reach that site. Bulk ops answer 200 with per-item failures in the envelope, so the pins read the envelope too — a status-code-only assertion would pass on a dropped coercion. Per-site mutation matrix, run this turn against these tests, each mutant applied and reverted from a file backup (never `git checkout`, which would have taken the uncommitted tests with it): drop coercion at handlers_items.go:2314 -> only TestItemFieldsCoercedOnMove fails drop coercion at handlers_items_bulk.go:683 -> only TestItemFieldsCoercedOnBulkMove fails drop coercion at handlers_items_bulk.go:499 -> only TestItemFieldsCoercedOnBulkFieldUpdate fails Each mutant is the defect at the site the test targets — not a call-site patch next to a still-correct function (CONVE-28) — and each kills exactly one test, which is the per-site discrimination the PR claims. Files restored and verified identical after the matrix; suite green.	2026-09-03 12:49:33 +00:00
xarmian	2cf9f0035a	fix(mcp,server,cli): three codex round-2 findings (BUG-2850) ... 1. [P1] The structured-value refusal was in the wrong place and killed the fix. It went into BuildCLIArgs, which env.Dispatch runs for BOTH transports before handing off to whichever Dispatcher is configured — so it blocked the remote /mcp door too, and the native-field handling that is the whole point of this change was never reached. Moved into ExecDispatcher, which IS the stdio door. My own test could not see this: it called mapItemCreate directly, so it vouched for the mapper and not for the path that reaches it — CONVE-19's exact shape, in a unit where I had already written binding tests for the other half. The tests are now split along the two claims the first version conflated: nested values REACH the dispatcher (the remote door is unblocked), and refuseStructuredFieldsOverCLI refuses them at the CLI door naming the transport. 2. [P2] The CLI warning sat after the `--format json` early return, so the caller most likely to have sent a mistyped key — one piping stdout into a parser — was the one caller who never saw it. Moved above the return, and out of the `ref != ""` branch it was also trapped in. Still stderr. 3. [P2] A nil value in fields_patch DELETES the key (store/items.go), so reporting it as an undeclared field told the caller a field was stored that the same request removed. Filtered at the patch site, not inside UndeclaredFieldKeys, because nil means "store JSON null" on the full-fields path where reporting it is correct. Gates: gofmt clean, go vet clean, go test ./... 29 packages ok. Claude-Session: https://claude.ai/code/session_011Q4b1iHtJtSyMs7BA2ySxo	2026-09-03 01:07:45 +00:00
xarmian	dc3fc2d50e	feat(server,cli): name undeclared field keys on the write response (BUG-2850) ... Undeclared keys are ACCEPTED — the census found 168 live values under 14 such keys, and refusing them would break read-modify-write on items nobody edited wrongly. But once stored, a typo and a deliberate extra field are indistinguishable, so the write now says which keys it did not recognize. - models.Item gains `Warnings *ItemWriteWarnings` with `undeclared_fields`, omitempty and additive. NEW API SURFACE: item write responses carried no warnings element before. Wrapping the response as {item, warnings} was the alternative and would have broken every existing parser; a clean write is byte-identical to before. - items.UndeclaredFieldKeys consults models.IsReservedItemField rather than re-listing the reserved set — that set exists so callers ask, and its doc comment records what re-listing cost last time. So a write carrying implementation_notes or github_pr reports nothing. - fields_patch reports only the PATCHED keys. A stray key already on the item is not something this write introduced, and naming it on every touch would train the reader to ignore the field. - The CLI prints one line to STDERR. Never stdout: `--format json` output is piped into scripts, and a warning there would corrupt the JSON they parse. - CLAUDE.md documents the element as new surface. Controls: never attaching the warnings fails the pin; reverting the HTTP mapper's native overlay fails the remote-door type test; dropping the reserved-key exclusion fails its own test. Two coverage gaps the controls FOUND rather than confirmed, both now closed: the remote door's native overlay was covered by no MCP test at all (a revert left the package green), and the reserved-key exclusion had no test either. Both were written after the control survived, which is the only reason they exist. Gates: gofmt clean, go vet clean, go test ./... 29 packages ok. Claude-Session: https://claude.ai/code/session_011Q4b1iHtJtSyMs7BA2ySxo	2026-09-03 00:54:56 +00:00
xarmian	b451fb50de	test(server): bind the coercion to its call sites, and enforce copy/preflight agreement (BUG-2850) ... CONVE-19: wiring is a claim. The previous commit threaded CoerceFields through eight validate sites; a test at the items package vouches for the function, not for any of those bindings. - Three HTTP-door tests (create, update, fields_patch) assert the stored NATIVE TYPE, not that the request returned 201 — a test that only checked the status passes on an implementation that stores the string, which is the shape the reporter described. - A text field holding "42" must stay a string in every one of them. Fixing this bug by coercing anything that parses would retype real data. - An un-coercible value must still be REFUSED with the validator's existing message, so coercion is not quietly widening what the server accepts. - TestCopyAndPreflightCoerceIdentically makes the cross-package invariant real. The preflight validates in internal/server and the copy in internal/store; both files carry a comment saying they must match, and a comment protects nobody. The assertion is agreement FIRST — whatever they do, they must do the same thing — and only then that both accept and the copy stores a number. Controls, each run against the mutated tree: - CoerceFields reduced to the identity function (the unfixed build) fails the two door tests and the items typing test. - Dropping the call at CREATE alone fails only the create test; dropping it at FIELDS_PATCH alone fails only that one. The bindings are individually covered, not covered in aggregate. - Coercing in the copy but not the preflight FAILS, and so does the reverse. Both drift directions are caught. BOUNDARY, stated rather than implied: four of the eight sites — move, bulk update, bulk move, and the migrated-schema paths they share — are wired identically but have no test that fails if that specific wiring is dropped. They are covered by the existing suites for their own behaviour, not for coercion. A follow-up should extend the door tests to them. Gates: gofmt clean, go vet clean, go test ./... 29 packages ok. Claude-Session: https://claude.ai/code/session_011Q4b1iHtJtSyMs7BA2ySxo	2026-09-02 23:38:26 +00:00
xarmian	ae793e6fa6	fix(server,store,items): coerce field values to their declared types server-side (BUG-2850) ... The write doors disagreed about what `key=value` means. The CLI has coerced by schema type since BUG-1125, and local stdio MCP inherits that by shelling out to the binary — but the remote /mcp transport builds its field map in ingestFieldKVP with `dst[key] = val`, so every value arrives as a string. validateFieldType then correctly refuses a string for a declared number or json field, and the net effect was that an MCP agent on that transport could not write those fields AT ALL: every attempt a 400, not a mis-typed value. Measured before writing anything (repro table on BUG-2850's trail): CLI and stdio MCP store 42 and an array; the HTTP door 400s on both; an UNDECLARED key is stored as a string on every door. items.CoerceFields(fields, schema) converts strings to the declared type — number via ParseFloat (NaN/±Inf refused, because json.Marshal cannot encode them and the ignored downstream error would silently drop the whole payload), json/multi_select via Unmarshal, checkbox via ParseBool — and is applied immediately before every Validate* call. Three deliberate non-behaviours, each with a test: - A value that will not parse is left as the string for the validator, so the existing "must be a number" error still fires. Coercion invents no error path, and cannot turn a currently-PASSING write into a failure. - Non-string values pass through untouched; an int stays an int. - Text-typed fields holding "42" stay strings. Coercing anything that parses would retype real data while fixing the bug. Not folded into ValidateFields, though that would be the single call site: a function named Validate that mutates its input is a trap, and two callers re-marshal the map they pass. THE POPULATION IS 8 CALL SITES, and finding them took two sweeps. The first was scoped to internal/server and found 7; the copy path validates in internal/store (items_cross_workspace_copy.go), which only a repo-wide sweep sees. The preflight and the store-side copy now carry cross-references to each other: the preflight exists to PREDICT the copy, they live in different packages, and that is exactly how they would drift unnoticed. The undeclared-key half of BUG-2850 is untouched and marked as a decision point in CoerceFields — refuse/warn/keep is with Dave. A test pins today's keep behaviour so the ruling lands as a deliberate change. The CLI's parseFieldFlag deliberately STAYS: it is why two of four doors are correct today, and removing it alongside its replacement would put all four at risk of one mistake. Retiring it is a follow-up. Claude-Session: https://claude.ai/code/session_011Q4b1iHtJtSyMs7BA2ySxo	2026-09-02 23:30:26 +00:00
xarmian	178b6b5010	fix(server,store): two more from codex rounds 3 and 4 (BUG-2810) ... **The import repair could silently change what gets imported.** It decodes into map[string]any, where a repeated object member keeps only the LAST value. The TYPED decode that runs next does not agree: encoding/json unmarshals members in order into the same struct field, so two `"workspace"` objects MERGE there and collapse here. A body with duplicate members would therefore import differently with --repair-nul than without, which is outside what a flag by that name may do. It now DECLINES such a body: returns it untouched, lets the gate judge it exactly as it would without the flag, and says why in the refusal — "the payload repeats the member X, and repairing it would change which value is imported". Detection is a token walk, because a decode is what loses the information: by the time there is a map the duplicate is gone. The detector's own test carries the false positive that matters — the same member name in SIBLING objects is not a duplicate, and a single shared set of names would decline every real export, since items all carry `id`, `title`, `slug`. Rewriting such a body faithfully wants a token-preserving pass, which is BUG-2812's token-walk and not a rider on this. A real export cannot contain duplicate members (json.Marshal does not emit them), so declining costs nothing an operator meets by accident. The tally now owns the repair — decodeJSONRepairingNUL takes it and calls Apply — so the count and the declined reason come back through one object instead of a return value a caller has to remember to record. That is the same mistake this branch already made once, when the JSON path dropped the count and the header reported 0 for an import that had rewritten a value. **A row the repair could not address was reported as a failure.** A NUL in a key column the list does not protect, on a row whose violation is elsewhere, makes the address unbindable: Layer A inspects every bound parameter, including a WHERE clause's, so the lookup is refused before SQLite is asked to find the row. It landed in Failed carrying "invalid text parameter: parameter 2" — the same information phrased as a fault in the repair rather than a property of the row. Now detected up front and reported as a skip with the reason, alongside the two skips that already existed. **One finding NOT fixed, deliberately, and recorded instead.** Round 3 raised that the scan misses a NUL in a value shadowed by a LITERAL duplicate key, so such a row passes the migrate-to-pg preflight and then fails during the copy — the exact failure the preflight replaces, surviving for one shape. That is textguard.KnownGaps: a blind spot every layer shares on purpose, which DOC-2823 forbids closing in one layer alone, because layers disagreeing about one value is the defect this cluster is made of. So it is named in ScanNUL's doc comment, in the preflight's, and in docs/backup.md for the operator, and TestScanNULInheritsTheRecordedKnownGaps pins the miss and FAILS when it stops being one — the notification that BUG-2812 has landed and those three prose sites need updating. The consequence is recorded on BUG-2812's trail. Round 2's single finding was refuted rather than fixed: it predicted TestRepairFlagReachesTheNestedAndObliqueForms would fail, on a mechanism that describes the raw-byte scanner this branch had already replaced. The test passes; the outer decode resolves the oblique spelling before the walk sees it.	2026-09-02 16:24:25 +00:00
xarmian	49bd342e4c	fix(store,server,cli): three defects from codex round 1 (BUG-2810) ... **The import flag could not repair the column it exists for.** `--repair-nul` scanned the RAW body for a live escape, which is right for a value the gate reads at the top level and wrong for the one that actually matters. An item's `fields` blob travels through an export as a STRING: a NUL escape in the stored blob marshals into the body with a DOUBLED backslash, which a raw scan must leave alone because at that layer it is literal text — while the gate refuses it anyway, since it decodes the body and re-parses that string as the document it is. So the repair now walks the DECODED body with the same classing bodyDecodesNUL uses, one verb changed: where the gate asks textguard whether a value decodes to a NUL, this asks textguard to repair it. Two walks of one shape in one package is a real risk, and the mitigation is that they are measured against the same corpus in both directions rather than reviewed for similarity — TestBodyRepairMirrorsTheGateOverTheCorpus drives every case through the body shape and asserts refused-becomes-accepted and accepted-stays-byte-identical. Two consequences worth stating. The walk also reaches the OBLIQUE spelling — the backslash written as its own escape, so the six characters never appear in the raw bytes at all — which the scanner could not, so the test that pinned that limit is replaced by one asserting the capability. And re-encoding is now possible, so it is bounded: UseNumber, so an integer wider than float64 is not silently re-emitted in scientific notation; SetEscapeHTML(false); and a body with nothing to repair is returned byte-identical rather than round-tripped. The mutation that removes UseNumber turns 9007199254740993 into ...992, and a test says so. The header is now X-Pad-Repaired-NUL-Values, because at the decoded layer an escape is not a thing that exists any more and one nested document may have carried several. **The scan could not run on the databases it exists for.** Several protected tables carry a NULLABLE workspace_id — activities, api_tokens, mcp_audit_log — and the scan selected it into a plain *string, which fails with "converting NULL to string is unsupported" and takes the scan, the repair and the migrate-to-pg preflight down with it. Every column is now scanned as sql.NullString: SQLite also permits NULL in a declared PRIMARY KEY that is neither INTEGER PRIMARY KEY nor NOT NULL, which no other engine does, and a NULL key cannot address a row for an UPDATE — such rows are reported and skipped with the reason rather than handed a WHERE that matches nothing. Verified against the unfixed code: the scan returned `scan activities.actor row: sql: Scan error ... converting NULL to string`. It needed a VIOLATING row in such a table, which is why every fixture that planted its rows in `items` missed it. **--force by accident.** The repair skipped the running-server check whenever --from was given — and the most natural --from an operator types is the path `pad db scan-nul` just printed, which IS the live database. The check is now on the resolved path (Abs + EvalSymlinks, so a symlinked data directory or a relative path still matches), and a --from naming an unrelated backup stays unguarded, which is correct: nothing is writing it. The ordering moved with it. `store.New` runs pending migrations, so the refusal now happens BEFORE the database is opened; opening first and refusing second made the guard arrive after the thing it guards against.	2026-09-02 15:15:11 +00:00
xarmian	63da2f4f5f	feat(store,server,cli): count and repair the legacy NUL population (BUG-2810) ... Layers A and B stop the value being written. Neither makes a row that already carries one go away, and BUG-2810's filing is what that costs: an affected workspace exports with a 200 and re-imports with a 400, so a self-hoster restoring their own backup is blocked with no path forward in the product, and `pad db migrate-to-pg` fails partway through the copy against PostgreSQL's jsonb parser rather than up front. This is DOC-2823's S3, on Dave's day-54 rulings: U+FFFD as the replacement, repair standalone only with a migrate-to-pg preflight that refuses and prints the command, `--repair-nul` on import shipping default-strict. ONE REPAIR, beside the one predicate. textguard.Repair lives next to ParameterRefused because four layers that agree about what is REFUSED and disagree about what a repair PRODUCES is this bug family arriving one step later. Its contract is a property over the same corpus, in both directions: every refused value becomes one all four layers accept, and every accepted value comes back IDENTICAL. The second half is the load-bearing one — a repair that tidies values nobody complained about rewrites `{"a":"x\\u0000y"}`, six literal characters after a doubled backslash, and corrupts it. The JSON arm is a string-literal SCANNER, not decode-walk-remarshal, which is what the recon write-up proposed before it was written. Re-marshalling changes four things nobody asked to change — object key order, insignificant whitespace, integers wider than float64, HTML-ish characters — and silently drops one of a document's LITERAL duplicate keys, which is a gap BUG-2812 owns and the last thing a repair should do. Scanning copies every byte it does not deliberately rewrite, so an untouched document is byte-identical without that having to be argued. A substring replace is not equivalent and the test that proves it took a mutation to find: a doubled-backslash literal ALONE never reaches the scanner, so the discriminating fixture is one document carrying a live escape AND a literal. THE COUNT IS COMPUTED IN GO. Measured on the read path in this worktree: a row planted with `bad<NUL>name` reads back into a Go string with all 8 bytes and the NUL intact, while `length(name)` in the same database answers 3. TASK-2824 found that C-truncation and concluded no DB-side REPAIR could be trusted; the same measurement on the read path says no DB-side COUNT can be either. SQL narrows — `instr(col, char(0))`, plus the escape prefix on JSON-classed columns, which is textguard's own pre-filter — and never decides. The decision stays ParameterRefused with isJSON from the shared 86-column list, i.e. Layer B's classing. Row addressing is read from the live schema rather than a hand-kept map: 39 tables carry protected columns, one (item_wiki_links) declares no primary key and is addressed by rowid, two have composite keys, and five have a single key that is not `id`. The repair checks RowsAffected because an address that stopped selecting its row would otherwise commit an UPDATE that touched nothing and report it as repaired — the one failure an operator cannot see in the output. `email_optouts(email)` is both a protected column and its own primary key. Repairing it changes the row's identity and can collide with an existing row, which in that table means somebody starts receiving mail again. It is reported and skipped, with the reason. The import flag is NOT an exemption from the gate. `--repair-nul` buys the body one repair attempt and then runs the same `bodyDecodesNUL` on the repaired bytes, which still decides — a decode path that skipped the check is the door BUG-2803 spent thirty rounds closing, on the endpoint carrying the largest attacker-controlled body in the product. Only the ESCAPE form is repaired: a raw NUL byte makes the document invalid JSON, and widening what parses is not this flag's job. Both doors are covered, JSON and tar.gz, because giving them different answers is how one of them keeps being forgotten. Postgres is settled with evidence rather than sent up as a ruling: it cannot hold either defect (22021, 22P05) and the four-way differential test already pins that, so the scan reports not-applicable WITH the reason rather than returning a zero a reader could mistake for a clean database. Spellings settled here, per the dispatch: `pad db scan-nul` and `pad db repair-nul` as siblings rather than `repair --nul`, matching `migrate-to-pg`'s hyphenated compound — a repair verb that errors when given no flag is a worse shape, and there is no second repair to share it with. scan-nul IS the dry run, so repair-nul grows no --dry-run. It refuses while the server is running unless --force, on the `pad db restore` precedent: the report is a claim about a database, and one somebody else is concurrently writing makes it a claim about a moment that has passed. docs/backup.md's section on this is rewritten. It still said the rule lives in the binary and not the database, which S2 made false, and it pointed at this item for a preflight and a repair that now exist. Its import examples also showed `pad workspace import < file`, which has never worked — the file is an argument. Closes BUG-2810.	2026-09-02 14:53:31 +00:00
xarmian	f54a0e41d4	docs(store,server): four comments and one log line that had stopped being true (BUG-2827) ... Codex round 6. No logic finding; it confirmed the refusal ordering, the split-budget claim and the first-tick scan as sound. Five statements in the branch's own prose were untrue of the code as it stands: - MaxOutboxPayloadBytes' comment still argued from 64 MiB ("two orders below both ceilings") after the constant became 128 MiB, which is 4x under the lowest ceiling, not two orders. - maxOutboxClaimBytes' comment counted a scan-into-string-then-copy transient that round 1 removed; the scan lands straight in []byte. - maxOutboxClaimRows' comment used the item BODY mean (~2.4 KB) as the payload mean; the measured payload mean is ~3.5 KB, so 5,000 rows is ~17 MiB, not ~12. - emitBulkItemEventTx's early-out said the numbers the caller sees still come from writeOutboxTx; when the early-out fires they come from the projection, and the error says so in Measured. - The drain's oversized-row log said "not claimed". OversizedPendingOutbox filters only dispatched_at, so during a rolling upgrade a binary older than the ceiling may be holding a claim on the row it names. The line now states what the query establishes: this instance will not claim it. The doc says why claimed_at is deliberately not a filter.	2026-09-02 12:33:08 +00:00
xarmian	0836808ca5	fix(store): drop a past-the-hop-bound event before judging its size, and correct three comments (BUG-2827) ... Codex round 5. No production defect found; one ordering edge and three comments that had stopped being true. THE ORDERING. writeOutboxTx has two refusals that disagree about the mutation. The hop bound drops the event and lets the mutation stand, because only the cascade it would extend is illegitimate. The size cap fails the mutation, because there the mutation and the event are the same fact. An event that trips BOTH was judged for size first, so it failed a mutation over a row that was never going to be written. The hop drop now comes first. Unreachable today - nothing propagates a hop - which is exactly why the ordering is worth pinning before something does. TestAnOversizedEventPastTheHopBoundIsDroppedNotRefused fails with the two checks swapped back (run before the crash that interrupted this round, and again on this tree). THREE COMMENTS. The drain-limit constant said whole batches are claimed past it; the byte budget and row cap can now split one. The claim candidates' doc said every sibling; it is as many as the budget still allows. OversizedPendingOutbox's doc named the write cap while its query uses the claim ceiling, and said it ran every tick when the caller throttles it to once per five minutes. Gates: gofmt clean, golangci-lint 0 issues, full SQLite suite exit 0, full Postgres suite exit 0.	2026-09-02 12:06:02 +00:00
xarmian	57caa7f92e	docs,test(store): correct five comments and strengthen the shrink fixture (BUG-2827) ... Codex round 4. Its one P1 does not hold, but the test it named as weak genuinely was, and five comments in this branch had drifted from the code they describe. THE P1, CHECKED RATHER THAN ARGUED. The claim measures size at candidate selection and never rechecks it, so a payload that GREW during a concurrent scrub could be claimed over the ceiling. The proposed growth path was Go's HTML escaping: json.Marshal writes < as its six-character unicode escape, where the source had one byte. Measured against Postgres 16: a one-key object whose value holds the four characters x<y>z&w, written with those characters LITERAL -> 16 bytes the same object written with < > and & as their six-character JSON unicode escapes instead -> 16 bytes Postgres parses the escapes and stores the characters, so the escaped and literal forms are the same size and the round trip cannot grow the row. On SQLite the payload is stored exactly as Go wrote it, so re-marshalling is idempotent. The rejection from round 3 stands, now on a measurement instead of an assertion about key removal. But the test defending it was weak, and codex was right about that: its fixture was one repeated ASCII letter, which cannot tell any of these encoder paths apart. It now carries <, >, & and non-ASCII, so it exercises the divergence rather than asserting past it. Mutation note worth keeping: a whitespace-padding mutant is caught on SQLite and NOT on Postgres, because jsonb discards insignificant whitespace — the mutant does not actually grow the stored row there. The faithful mutant adds a key, and that one dies on both. FIVE COMMENTS THAT SAID SOMETHING UNTRUE, all introduced by this branch: - OversizedOutboxPayloadError was documented as the write cap's error; three sites raise it, against two different limits. - "The two things Measured can name" listed three. - measuredStoredRow said "as the database stored it", but OctetLength measures what the driver hands back, which on Postgres is the ::text rendering rather than storage. - OversizedPendingOutbox described its threshold as the write cap while querying the claim ceiling. - ClaimPendingOutboxEvents still said batches are claimed whole, which the byte budget and row cap deliberately interrupt. Tests also now assert Measured at all three refusal sites — without it the field could be blank everywhere and every existing assertion still passes — and the claimability property test's refusal branch checks the row actually rolled back, so "refused" cannot be satisfied by a write that committed anyway. Gates: gofmt clean, golangci-lint 0 issues, full SQLite suite exit 0, full Postgres suite exit 0. Claude-Session: https://claude.ai/code/session_01XLtX4dbjBpApbAv3SuBcTm	2026-09-02 12:05:57 +00:00
xarmian	213142c4b5	fix(store,server): honest refusal figures and a throttled oversized scan (BUG-2827) ... Codex round 3. Two of three findings acted on, one rejected with an invariant test in place of the change it asked for. REFUSAL FIGURES SAID SOMETHING FALSE. The store refuses on two different measurements against two different limits — the member content before marshalling, against the write cap, and the row exactly as stored, against the claim ceiling — and both reported the number as "a %d-byte payload". For the first that is untrue: projectedBulkPayloadBytes is explicitly a lower bound, so the message named a size the payload did not have. A caller seeing two different numbers for one mutation had no way to reconcile them. OversizedOutboxPayloadError now carries what it measured, and both the error and the 413 say so. THE DIAGNOSTIC WAS THE MOST EXPENSIVE THING THE DRAIN DID, and it was most expensive when it found nothing. OversizedPendingOutbox has a non-sargable size predicate and no index to help it, so an empty result means evaluating octet_length over every pending row — on Postgres, detoasting and serializing each JSONB payload — and it ran every 5s tick. Now throttled to once every 5 minutes. Latency is the cheap thing to spend here: the rows it reports are permanently unclaimable and sit until the 7-day retention takes them, so a five-minute alarm delay changes no decision anyone makes about them. The first tick after a restart still scans, so an existing oversized row is reported promptly. REJECTED: that the claim needs to revalidate size, because a concurrent scrub could grow a payload between candidate selection and the claim UPDATE. It cannot. scrubOutboxRowTx is the only UPDATE of payload in the tree and it removes keys and re-marshals compactly, so a rewrite is strictly smaller — and shrinking is harmless, since a row judged claimable stays claimable. That is load-bearing for the claim needing no revalidation, so it is now stated at the function and pinned by TestScrubOnlyEverShrinksAPayload rather than left as an assumption for the next person adding a payload rewrite to break silently. The throttle test found its own gap on the way in. Written first against the helper, it stayed green when the call site was mutated to `if true` — a helper nothing calls is still correct in isolation. TestOutboxDrainTickConsultsTheThrottle covers the call site through the stamp the tick leaves behind. Also caught by the gate rather than by review: the field carrying the throttle clock landed on outboxDrainSettings as well as outboxDrainConfig, because the edit matched a line both structs have. Tests passed with both; lint named the dead one. Gates: gofmt clean, golangci-lint 0 issues, full SQLite suite exit 0, full Postgres suite exit 0. Mutants: growing the written payload kills TestScrubOnlyEverShrinksAPayload, removing the throttle call site kills TestOutboxDrainTickConsultsTheThrottle. One mutant discarded as unfaithful — corrupting the payload BEFORE the compare-and-swap is neutralised by the retry, which re-reads and redoes the work correctly, so it tests the retry rather than the invariant. Claude-Session: https://claude.ai/code/session_01XLtX4dbjBpApbAv3SuBcTm	2026-09-02 01:41:47 +00:00
xarmian	5b12d5eeeb	fix(store,server): bound the outbox's unbounded reads and writes (BUG-2827) ... item.bulk_updated marshals every cascaded member body into ONE event_outbox row, and nothing bounded either the row or the drain's reading of it. The v1 doc called the size deliberately unbounded and named a follow-up condition; measurement met it. MEASURED FIRST, against an 8,434-item instance: widest wiki-title cascade 23 members / 175.6 KiB of bodies widest wiki-ref cascade 50 members / 365.5 KiB worst option rename 4,429 members / 7.34 MiB (tasks.status="done") marshal ratio ~1.46x (item.updated payload mean 3,518 B against an item body mean of 2,416 B) So renaming one status option on that instance emits ~11 MiB in a single row today, from a user clicking rename in the collection editor. That kills the obvious design: a write cap tight enough to bound the drain's memory refuses routine work, and one loose enough for routine work bounds nothing. The two therefore became two numbers. WRITE CAP (128 MiB) in writeOutboxTx, the single INSERT INTO event_outbox in the tree, so all seven emit paths inherit it rather than an enumerated site list. Refusing FAILS the mutation: the hop bound in the same function drops the event and keeps the mutation, correctly, because there the cascade is what is illegitimate, while here the mutation and the event are the same fact. 128 rather than 64 MiB so it clears MaxItemRenameCascadeBytes — a cascade squeaking under that 64 MiB bound marshals to ~96 MiB, and a 64 MiB cap here would let the vaguer refusal preempt rename_cascade_too_large on the very renames that bound describes. Surfaces as 413 event_payload_too_large, following that precedent rather than inventing a second spelling. CLAIM BUDGET (64 MiB per pass), spent by the primary candidates AND by batch siblings. Spending it on siblings knowingly relaxes "batches are claimed whole": read literally that rule makes the budget bypassable by construction, since one large batch is an unbounded read no row limit touches, and groupOutboxDeliveries already defines the split. The first candidate is always taken whatever it costs, so the bound meant to keep the drain alive cannot starve a row instead. CLAIM CEILING (2x the write cap), and it is NOT the write cap — a distinction the Postgres leg had to teach. The cap measures the Go bytes json.Marshal produced; the claim measures what the driver hands to Scan, which on Postgres is the JSONB ::text rendering, one space inserted after every colon and comma. A 40,000-byte payload reads back as 40,001. Thresholded at the same number, a payload written at exactly the cap was admitted by the guard and then permanently excluded by the claim: delivered to nobody, reported as nothing, reaped seven days later. TestARowWrittenAtTheCapIsStillClaimable fails on Postgres and passes on SQLite against the 1x version. Rows above the ceiling are excluded IN THE PREDICATE, not filtered in Go — filtering in Go leaves them occupying candidate slots and starves everything behind them, which is the same jam wearing different clothes. They are logged every tick and left pending for the existing 7-day undispatched retention, not stamped dispatched_at, which would record that an event went out when it did not. SCRUB. ScrubOutboxUserRefsTx collected every LIKE-matching payload at once — the same unbounded read through a different door. Now batched by a keyset cursor on id. READ FULLY THEN WRITE is preserved PER BATCH, and the cursor keeps the per-row UPDATEs in ascending id order across batches, so batching does not quietly reintroduce the BUG-2409 deadlock it was written to avoid. Prose sweep: MaxItemRenameCascadeBytes' comment asserted this vector had no bound, and emitBulkItemEventTx's asserted the payload was unbounded by decision. Both now say what is true, the second keeping its original reasoning because it still explains what the bound does NOT do. Mutation matrix, each mutant compiled and run unfiltered: removing the row-cap check, > to >=, removing the budget break, removing always-take-one, removing the SQL size exclusion, removing the sibling budget, removing the scrub cursor advance, and 2x to 1x — all die, the last on Postgres only. One survivor recorded as unfaithful rather than as a gap: setting the scrub cursor to the first row read still advances monotonically over an ascending id > cursor query, so termination holds and only the pass count degrades. Not addressed, named rather than left to be found: a workspace that outgrows the write cap cannot shrink its own cascade, so the refusal leaves no recourse. The answer is chunking one bulk event across rows sharing a batch_id, which the drain's fold already supports. Claude-Session: https://claude.ai/code/session_01XLtX4dbjBpApbAv3SuBcTm	2026-09-01 23:15:05 +00:00
xarmian	054b192e5d	feat(store): make the NUL invariant a property of the DATABASE (DOC-2823 S2) ... Closes BUG-2813's old-binary half. S1 put the rule in the binary, which cannot help in the window this bug is about: an older binary serving the same SQLite file has no guard, so a rollback, a staged rollout or a second instance writes rows the invariant forbids. A trigger is enforced by the FILE. 194 BEFORE INSERT/UPDATE triggers over 97 columns — TASK-2825's 86-column census plus the second ring Ruling 2 admitted wholesale. ONE LIST, THREE CONSUMERS, which is the shape TASK-2825 asked for. The migration is GENERATED from internal/store/nulcolumns.go, a guard test compares that list against the LIVE migrated schema, and a second test pins the generated SQL against the list so neither can drift from the other by hand. The census guard is a BASELINE rather than a demand that all 405 text columns be classified. The census classified the 86 that can carry caller text and left ids, timestamps and hashes unenumerated; requiring an entry for each would be 300 lines nobody reads. The 301 known-outside columns are recorded, and ANY change to that set — added, renamed, removed — fails and asks for a decision. The predicate is TASK-2824's, measured on the driver Pad actually embeds: instr for the raw byte (length() C-truncates and cannot be used), and json_tree for the decoded escape, in values AND keys, with a json_valid guard because json_tree raises on a non-document. The doubled-backslash case stays literal, which is the false positive this predicate family exists to avoid. Trigger aborts are classified into the SAME typed error Layer A produces, so a caller cannot tell which layer refused and the handler's existing 400 mapping covers both. That discharges Ruling 2's condition for admitting the second ring: a header-derived user agent or IP hitting a trigger must not surface as a 500 or a broken login. THE FOURTH LEG IS LIT. S1 built the differential harness with three legs and left this one dark; all 17 corpus cases now agree across the HTTP gate, Layer A at the driver, Layer B via a real UNGUARDED SQLite write, and native Postgres. Four independent enforcers, one corpus — the property DOC-2823 named as the actual deliverable. Two consumers needed changes, both legitimate. The FTS trigger census excludes the new triggers by prefix, so adding a protected column does not require editing that test. And a timeline test that INJECTS a NUL to build its fixture now brackets the injection by dropping and restoring the two triggers — it needs a row that violates the invariant, because what it tests is the handling of legacy data, and Layer B exists to stop such a row being written. SQLite only. Postgres refuses a NUL in text natively and an escape decoding to one in jsonb, so it already owns the rule there. Full Go suite green on SQLite and Postgres 17; lint 0 issues. Claude-Session: https://claude.ai/code/session_01XLtX4dbjBpApbAv3SuBcTm	2026-09-01 21:16:17 +00:00
xarmian	cc8bf888d2	fix(store,server): close codex round 3 — inspect what is actually bound ... Eight findings, four P1. The three parameter ones share a cause: the guard inspected one value and forwarded another. [P1] Value() was called for the CHECK and the original driver.Valuer was forwarded, so pgx called it again — a stateful valuer could show the guard clean text and the database a NUL. Resolution now happens once and the resolved value is written back into the argument. [P1] A typed-nil valuer, (*sql.NullString)(nil), was called directly and panicked. database/sql special-cases it as SQL NULL; so does the guard now. [P1] Only an exact `string` was recognised. pgx implements NamedValueChecker and binds *string, named string types and json.RawMessage unconverted, so each carried text the guard never saw. Resolution is by reflected KIND now, not by a list of types. [P1] Registration opened a probe connection for BOTH drivers, so creating a SQLite store attempted a live PostgreSQL connection against whatever host the environment happened to name — network access as a side effect of opening a local file — and swallowed the error, skipping the guarantees in exactly the case the check could not run. The assertions moved to connect and prepare time, where the object being asserted about is the one in hand. [P2] The statement-interface assertions were claimed in a COMMENT and existed nowhere. guardStmt forwards neither NamedValueChecker nor ColumnConverter, so a driver gaining one would silently lose its own argument conversion. Asserted now, and the parity test covers the statement level too. [P2] The cross-workspace copy gave a guard refusal its ambiguous "may or may not have landed" message. That refusal fires at parameter binding, before any statement executes — nothing landed, and telling the caller to reconcile invites exactly the manual work DR-13's wording exists to prevent. [P2] The Valuer integration test passed for the wrong reason by default: SQLite's driver lacks NamedValueChecker, so database/sql unwrapped the NullString before the guard ran. The three properties are unit-tested now, independent of which driver is present. And the NULL control asserted only that the write succeeded — it reads the row back and asserts SQL NULL, which an incorrect conversion to "" would otherwise have passed. Full Go suite green on SQLite and Postgres 17; lint 0 issues. Claude-Session: https://claude.ai/code/session_01XLtX4dbjBpApbAv3SuBcTm	2026-09-01 14:29:07 +00:00
xarmian	74c786d590	fix(store,server): close codex round 2 — the wrapper must mirror its base ... Seven findings; two were P1 and one of them was a real hole on Postgres only. [P1] driver.Valuer bypassed the guard. checkParams type-asserted `string`, and pgx implements NamedValueChecker — so it ACCEPTS a sql.NullString unchanged rather than letting database/sql's converter unwrap it, and the guard never saw the text. Measured before the fix: a NUL-bearing sql.NullString on Postgres passed Layer A entirely and was refused by the server as SQLSTATE 22021, i.e. a 500, while the identical value on SQLite got the typed 400 — the dialect split reappearing in the response shape. wiki_links.go binds sql.NullString today. [P1] driver.DriverContext was dropped, so sql.Open used a legacy connector that ignores the context and a cancelled request could leave a pgx dial running to its 60-second timeout. The rest share one cause, and it is the thing to remember: database/sql BRANCHES on whether an optional interface is present, so a wrapper advertising one the base lacks CHANGES behaviour rather than adding a no-op. Measured, the two drivers differ — pgx has no Validator, sqlite has no conn NamedValueChecker and no DriverContext — and the single wrapper type claimed all of them. So the wrapper now MIRRORS its base: four conn variants over the two interfaces that vary, a separate driver type for DriverContext, and the non-varying ones asserted at registration so a driver bump fails loudly instead of degrading. I made the same mistake inside the fix — implementing OpenConnector unconditionally, which broke every SQLite open — and then a third time, where guardConnector.Driver() returned the inner wrapper and a pgx pool reported no DriverContext. The third was caught by a new parity test on its first run, not by review: it asserts wrapped and base advertise EXACTLY the same interfaces. Also: both gap guardrails SKIPPED when their slice was empty, so deleting an entry made the suite green — the opposite of their purpose. They assert counts now. And the oracle test compared the two walkers without pinning any answer, which is how round 27 left both wrong about scalars; known answers are pinned against Postgres, and doing that caught me re-pinning one from a stale comment. Full Go suite green on SQLite and Postgres 17; lint 0 issues. Claude-Session: https://claude.ai/code/session_01XLtX4dbjBpApbAv3SuBcTm	2026-09-01 13:57:07 +00:00
xarmian	ef792bdf35	fix(store,server,textguard): close codex round 1 on S1 — seven findings ... All seven verified before acting; two were production regressions I had introduced and one was a defect older than this branch. [High] The wrapper HID the optional connection interfaces. Measured: the raw modernc conn implements Pinger, SessionResetter and Validator; the wrapped one implemented none, so Ping succeeded without pinging, pooled connections stopped being reset, and dead connections stayed in the pool. Each is now forwarded, falling back to exactly what database/sql does for a conn lacking it. NamedValueChecker is forwarded too, or the wrapper silently narrowed the argument types pgx accepts. [High] Statement-level driver.ErrSkip was wrong. At the CONN level ErrSkip is the documented fallback signal; at the STATEMENT level database/sql propagates it, so a base statement without the context interface would have failed rather than degraded. Both now fall back to the positional form, refusing a NAMED argument rather than binding it by position. [High] Scalar JSON documents bypassed the check — and the database settles it: SELECT ('"a<escape>b"')::jsonb; ERROR: unsupported Unicode escape sequence A bare JSON string is a complete jsonb document. The object/array-only shape test is right for the HTTP gate's "is this a nested document" question and wrong for the store's "will a jsonb parser read this". Both now use a widened form, so the gate refuses it at the door instead of leaving it to the store. That widening broke the gate's independent oracle, which had been NARROWED in BUG-2803 round 27 to make the two walkers agree. The disagreement was real; the direction of the fix was not — nobody measured which one matched Postgres, and it was the oracle. Restored, with the measurement in the comment. Two implementations made to agree are not thereby correct. [High] The 400 mapping was not the single funnel I claimed. createItemChecked, bulk ops and the cross-workspace copy carry their own error envelopes by design. One shared CLASSIFIER now serves all of them; the envelopes stay different, the classification and wording do not. Copy keeps its retry-discouraging message deliberately (PLAN-2357 DR-13). [Medium] The over-refusal justification was FALSE as written. It claimed a JSON-shaped text value carrying a live escape "is a value Postgres would refuse the moment anything parsed it". Nothing parses a text column; Postgres stores it fine. The trade is now stated honestly, and pinned: textguard. StoreOverRefusals records the case, and a test fails when it is paid down. [Medium] Duplicate JSON keys are a real under-refusal, inherited from the shared predicate's map model. DOC-2823 requires Layer A NOT to fix it alone, so it is recorded in textguard.KnownGaps with a test that fails when it CLOSES — the signal that BUG-2812's token-walk landed. [Medium] Test weaknesses, and the fixes caught a real one. The census now sees ALTER TABLE ... ADD COLUMN. The store corpus leg now requires accepted cases to SUCCEED, which immediately exposed a case passing for the wrong reason: a non-JSON value written to the fields column failed on SQLite's own JSON parser, never reaching the guard. Full Go suite green on SQLite and Postgres 17; lint 0 issues. Claude-Session: https://claude.ai/code/session_01XLtX4dbjBpApbAv3SuBcTm	2026-09-01 12:54:19 +00:00
xarmian	3baf984b41	feat(server,store): the differential test's three live legs + the 400 mapping ... Completes S1's deliverable except Layer B, which stays dark until S2. THE THREE LEGS, all driving the SAME corpus: - HTTP gate (internal/server): each case enters as a request body, classed the way a request is — by KEY NAME, "fields" for JSON, "content" for text. - Layer A at the driver (internal/store): each case is a real write, classed by column. - Native Postgres (internal/store): each case is a real INSERT through the RAW pgx driver, no guard in the path, so what is measured is Postgres's own verdict rather than ours reflected back. All 15 cases agree across all three. That is the calibration result the design needed: the guard neither over-refuses (breaking writes Postgres accepts) nor under-refuses (leaving the dialect split BUG-2831 was about). The Postgres leg additionally asserts the SQLSTATE, because "some error occurred" would have been satisfied by a typo in the test's own SQL. THE 400 MAPPING is in writeInternalError — the single funnel every 500 already passes through, so one insertion covers every handler present and future. The item-title unit had to find the same error block three times in one function before a structural test caught the third; this avoids the enumeration entirely. 400 rather than 500 because the request is understood and will be refused identically on retry, and 400 specifically because it is what the gate answers for the SAME value refused at the door — two statuses for one rule would be the layers disagreeing in the response instead of the predicate. The residual is stated rather than hidden: a value can reach the store from something the SERVER composed (BUG-2814's re-emit population), and for those a 400 misattributes the fault. It is still better than 500, and if that case ever needs its own status it needs its own error type first. Mutation matrix extended: 3 more mutants, all killed — mapping removed (500 returns), mapping swallowing every error (the control leg), and the gate ceasing to delegate to the shared core, which is the layers-diverge scenario the corpus exists to catch. Full Go suite green on SQLite and Postgres 17; lint 0 issues. Claude-Session: https://claude.ai/code/session_01XLtX4dbjBpApbAv3SuBcTm	2026-09-01 12:06:08 +00:00
xarmian	52c480c6d2	feat(store): enforce the NUL invariant at the driver (DOC-2823 S1 Layer A) ... Closes BUG-2814 and the current-binary half of BUG-2813: the fixed binary can no longer WRITE a decoded NUL from any path, request or re-emit. The guard is a database/sql driver wrapper, not a seam in this package, and the two measurements that decided it are in nulguard.go's header. The design named the Queryer seam; Queryer is Query + QueryRow, its own doc says "the read-only subset", and no store write passes through it. Writes reach the driver by four receivers — db.Exec (139 sites), tx.Exec (99), stmt.Exec (2 prepared), and a passed executor — and a wrapper at the *sql.DB level cannot see the prepared ones at all. The deeper reason is this cluster's own lesson: a seam every write site must be EDITED to route through is an enumeration wearing a seam's costume, and nothing stops the next site taking the raw handle. The wrapper never reads SQL text, only bound parameters, so Sprintf-built statements are covered without being understood and a new call site is covered before it is written. It covers Query as well as Exec, which is not symmetry: three writes ride the Query path today (UPDATE ... RETURNING in password_resets and email_verification, INSERT ... RETURNING in yjs_updates). A single-line regex for that shape found ZERO; a structural scan found all three, each a multi-line raw-string literal. CLASSING: string parameters are checked, []byte is exempt. The invariant is about text and JSON columns, and in this store []byte binds BINARY — item_yjs_updates.update_data, the only BLOB/BYTEA column in either schema. The first version checked []byte and refused every Yjs op-log append; the existing collab suite caught it immediately. The exemption is pinned by a census test that fails when a new binary column appears, because the supporting sweep was source-level and this unit is a catalogue of source-level sweeps missing things. Mutation matrix: 13 mutants, 12 killed by named tests. Four survived first and each taught something rather than needing a weaker claim — the Query path, both prepared routes, and a Prepare branch that turned out to be DEAD for both drivers (database/sql routes to PrepareContext when the conn implements it), so that duplicate was removed rather than tested. Full Go suite green on SQLite and Postgres 17. Claude-Session: https://claude.ai/code/session_01XLtX4dbjBpApbAv3SuBcTm	2026-09-01 04:47:57 +00:00
xarmian	d0644a60fc	refactor(textguard): extract the one decoded-NUL predicate and its corpus (DOC-2823 S1) ... Groundwork for S1's store-layer guard. The predicate lived in internal/server and could not be reached from internal/store, so a store guard would have had to reimplement it — two implementations of "does this decode to a NUL", which is the disagreement class this whole cluster exists because of. internal/textguard is a leaf package holding the predicate and the adversarial corpus. The HTTP gate now DELEGATES to it rather than carrying its own copy: stringIsJSONDocument and nestedDocumentDecodesNUL become one-line forwards, and valueDecodesNUL's unclassed arm is textguard's walk verbatim. Its existing 418-line corpus passes unchanged, which is the extraction's proof. What is deliberately NOT shared is named in the package doc: how a layer decides a value IS JSON. The gate derives that from request-body KEY NAMES; the store derives it from the COLUMN a parameter is bound to. Those cannot be merged, so every entry point takes the classification as an argument and the differential test pins that both derivations agree on one corpus. The corpus ships in the package rather than in a _test.go because three packages measure against it and three copies would drift. Fifteen cases, each carrying the reasoning or measurement that put it there — including the doubled-backslash false positive that made the raw-byte parity filter unsound, the raw-NUL-inside-a-JSON-value regression codex round 9 caught, and the nested-document asymmetry where the same escape is fatal one layer up. The escape text and the NUL character are constants, and a test asserts they did not decay into each other by comparing against a value it BUILDS rather than one it types. That mistake was made three times during BUG-2803 and twice more while writing this file. Claude-Session: https://claude.ai/code/session_01XLtX4dbjBpApbAv3SuBcTm	2026-09-01 04:24:45 +00:00
xarmian	0d0f1c9125	fix(items): require and bound item titles at every write door (BUG-2833, BUG-2831) ... `PATCH {"title": ""}` was accepted and applied while `POST` refused the same input with 400 "Title is required": the guard was an inline literal inside handleCreateItem, so the sibling handler on the same field never had it. Item titles were also unbounded, and the slug derives from the title with no truncation, so the same input was accepted on SQLite and refused by Postgres at the UNIQUE(workspace_id, slug) btree with an unmapped SQLSTATE 54000 — a latent `pad db migrate` failure as well as a create-path one. One models.NormalizeItemTitle / models.ValidateItemTitle pair now backs every door, enforced authoritatively in store.CreateItem and store.UpdateItem so a future door inherits the rule rather than having to repeat it. The handlers keep a pre-lock copy that REFUSES ONLY: it may answer 400 early and must not alter the input, because its view of the row predates the write lock. - trim: whitespace-only titles are refused, widening the create door. Artifact import already trimmed while create tested == "" exactly, and its comment claimed to mirror the gate it was stricter than. - bound: 255 runes, matching MaxDocumentTitleRunes but justified for items — slugify emits only [a-z0-9-] at one byte per rune and truncates nothing, so 255 runes bounds the slug well under the btree index-tuple cap. That cap is 2704 bytes in practice, not the 8191 the filing quoted; both figures and the readings behind them are in the constant's comment. - non-retroactive: a title identical to the stored one is not a rename, is not validated, and is dropped rather than re-applied — so rows predating the bound stay editable and a no-op echo cannot move an item's slug. - import coerces rather than refuses (empty -> "Untitled", over-long -> truncated, both logged, colliding truncations resolved), matching coerceJSONForImport's recorded disposition three lines away. Refusing would break restoring archives of data this product already accepted. - cross-workspace copy propagates a legacy source title, by ruling. It takes no title from the caller, so it cannot mint one. The guarantee that holds across every path is narrower than "every stored title satisfies the bound", and the comments say so: no CALLER-SUPPLIED title is stored without being validated. Seven codex rounds, 23 findings, ending CLEAN. Two of the findings were defects introduced by earlier fixes in this same unit — an empty-title hole opened through the legacy-protection clause, and a handler-side decision that dropped a concurrent rename — both recorded on BUG-2833's trail. 38 mutants; every behavioural fix has a mutant that is the defect at its site, killed by a named test. Prose sweep per CONVE-23: three comments asserting item titles are unbounded, and a cost model resting on a ~2 MiB single-request title, corrected in place — the guards they document still hold, because the bound is non-retroactive and the cascade charges STORED titles. Filed rather than bundled: BUG-2836, BUG-2839, BUG-2840, BUG-2842. Closes BUG-2833, BUG-2831. Claude-Session: https://claude.ai/code/session_01XLtX4dbjBpApbAv3SuBcTm	2026-08-31 22:45:46 -04:00
xarmian	21b4d8d9c4	fix(store,links): remove both quadratics in the item rename cascade and bound it (BUG-2804) (#1224) ... * test(store): measure the item-side rename cascade amplification (BUG-2804) Measurement only — no production code changes, no cap. The dispatch is measure-first, and BUG-2798's arithmetic is explicitly non-transferable to this path, so nothing here reuses its numbers. internal/store/items_rename_probe_test.go sweeps three axes with a k=0 negative control and a precondition that fails if the cascade would not be exercised (an empty cascade would otherwise report a flat line and read as "no amplification"). Instrument is MemStats.TotalAlloc, the same counter documents_rename_bounds_test.go uses, so the two paths compare directly. What it establishes, all measured on this machine: - 2.01x the body allocated per bracket, linear across four intervals (1 -> 8 -> 64 -> 512 -> 4096 brackets, agreeing to three significant figures). Bracket count is not independent of body size: the cheapest link is 5 bytes and nothing caps links per item, so the cost is O(C^2) in ONE linker's body, in ONE request, with no accumulation needed. Confirmed directly: doubling C multiplies allocation by 3.83 / 3.92 / 3.95, converging on 4. - 21.3x-22.2x the body per added linker at 8 brackets each, linear in k. - The outbox payload carries every rewritten body in a single row, at exactly 1.00x the body set, persisted. emitBulkItemEventTx documents this as deliberately unbounded in v1 and invites the measurement. Peak residency is reported as a coarse floor and carries no claim. The sampler sleeps between reads deliberately: ReadMemStats stops the world, and a spin loop turned a sub-second cascade into minutes of wall clock on this probe's first run. * test(store,links): decompose the per-bracket constant into its two sites (BUG-2804) Still measurement only. Checkpoint 2 reported a marginal cost of 2.01x the body per bracket without establishing what the 2 was made of. These probes account for it exactly, and the answer changes the fix shape. - internal/links/probe_alloc_test.go isolates RewriteBracketAt: 1.00x the body per call. So the rewriter is only HALF the per-bracket cost. - items_rename_probe_test.go's SELECT probe finds the other half, and it is not in the rewriter at all. cascadeTitleRename's SELECT joins item_wiki_links to items and projects s.content, returning one row per LINK, each carrying a full copy of the source body. The scan loop de-duplicates by source id only AFTER rows.Scan has allocated a fresh string per row. Measured dead flat at 1.00x the body per row against a SINGLE distinct source: 4096 rows on one 256 KiB item allocate 1.07 GB. 1.00 + 1.00 accounts for the measured 2.01. That makes TWO independent quadratics with different fixes. A single-pass rewriter would leave the SELECT's copy untouched and halve the cost rather than remove it. A third probe tests, and REFUTES, an earlier hypothesis that the second copy was the cascade's `rewritten != newContent` full-body comparison: a different-length new title measures 2.05x against the same-length 2.02x, no drop. The comparison allocates nothing, so it could never have appeared in an allocation counter. Kept as a negative result so the refuted reading is not re-derived. * refactor(links): single-pass RewriteBracketsAt, RewriteBracketAt delegates (BUG-2804 M2) The per-bracket decision moves into one unexported helper; RewriteBracketAt becomes the one-element case so the two cannot drift. Differential tests pin behaviour against the PRE-REFACTOR implementation, frozen verbatim in the test file as an oracle. Comparing RewriteBracketAt to RewriteBracketsAt-of-one would be vacuous now that the former is defined as the latter, so the oracle is what makes the test able to fail. * fix(store): bound the item rename cascade and remove both quadratics (BUG-2804) Three mechanisms, landing together because the first two are factors of one product — either alone still leaves O(C^2) while measuring "2x better". M1: cascadeTitleRename's SELECT no longer projects s.content. It returns one row per LINK, so projecting the body made the driver materialise a full copy per bracket, de-duplicated only after rows.Scan had allocated each one. Measured at 1.00x the body per row against a SINGLE source. Each source's content is now read once. M2: the rewrite is a single pass. links.RewriteBracketsAt splices every recorded bracket with one strings.Builder; RewriteBracketAt is reimplemented as the one-element case so the per-bracket decision has exactly one copy and the public contract is unchanged. M3: MaxItemRenameCascadeBytes (64 MiB) bounds the total linking-item content one rename may process, charged before each body is built so a refusal never allocates the thing it refuses. The number is derived from the live workspace's measured distribution, not inherited from documents.go, and the receipt is in the constant's doc comment. Measured, before -> after: doubling C 3.83/3.92/3.95x -> 2.04/2.21/2.06x (quadratic gone) per bracket 2.01x body -> 0.01x body per linker 21.3x body -> 7.3x body 4096-row SELECT 1,074,850,200 B -> 1,180,856 B (910x) store suite 180.9s -> 86.8s Equivalence is pinned against the PRE-REFACTOR implementation, frozen in the test file as an oracle, over a hand-enumerated behaviour corpus plus 20,000 randomised inputs; and the single pass is pinned against the descending fold the cascade used to perform, over 5,000 randomised inputs with shuffled, duplicated and corrupted offsets. The k-linear retention that remains is in the outbox member snapshots, filed separately as BUG-2827. * fix(store,server): codex R1 findings + a dialect-dependent test instrument (BUG-2804) P1 — ProjectRewrittenLen built a replacement string per bracket to measure it, reintroducing allocate-then-refuse one layer below the cap it feeds. The per-bracket helper now returns the SEGMENTS and never concatenates; projection does length arithmetic only, and the rewrite writes the parts straight into its builder, removing an allocation from the hot path too. P2 — the item rename cascade's refusal had no errors.As arm in handleUpdateItem, so a deliberate, permanent decline reached clients as a 500 implying a retry might help. Now 413 rename_cascade_too_large, composed from the error's typed fields so the internal call path is not published. Pinned by an exact-status test plus a counterfactual that a blanket-413 handler would fail. Checked the sibling write paths rather than assuming the reported one was the population: bulk update and version restore never set Title, so the cascade cannot fire there and this handler is the whole surface. P2 — `applied` counted brackets MATCHED, not brackets CHANGED, so a rewrite reproducing content byte-for-byte would still write the row, bump seq and emit events. Counting changes restores what the pre-fix whole-body comparison did. Cascade reachability is NOT established and the test says so. Separately, found by make test-pg: the refuse-before-building guard used a TotalAlloc ceiling, and the same refusal allocates ~69 MB on SQLite against ~242 MB on Postgres — the ceiling was measuring the driver. Replaced with an exact count of bodies built via a store test seam. The first version of that seam ALSO failed to discriminate: a mutation moving the cap check between the build and the seam call went undetected, so the build and the count are now one function with no third position for the defect to hide in. Gates on this tree: gofmt, go vet, go build ./... clean; go test ./... PASS on SQLite AND on Postgres (own container, private port 5446 — never the shared 5445, whose teardown would kill a sibling seat's container mid-run). * fix(links,server): codex R2 findings (BUG-2804) R2-1 — ProjectRewrittenLen advanced its cursor past a no-op bracket while RewriteBracketsAt did not, so the two disagreed about which overlapping rewrites the guard skips. On `[[A[[B]]]]` with a no-op at 0 and a change at 3, projection reported 10 bytes / 0 applied while the pass produced 13 / 1 — the cascade charged the read and never charged the rewrite it then performed, so the bound leaked on exactly the corrupt and duplicated offsets the defensive paths exist for. Projection now follows the pass, not the reverse: the pass's behaviour is pinned to the descending fold the cascade used to perform, so moving it would have changed cascade semantics under cover of a bug fix. The lockstep property test that pins this needed MIXED per-position target titles — the first version shared one title across a call and passed against the broken code, because the reproducing shape needs a no-op bracket overlapping a changing one, which a shared title cannot express. Cascade rows carry per-row target_title, so mixed is the realistic case. R2-2 — the 413 mapping covered one of THREE places handleUpdateItem reaches UpdateItemWithParentLink. The post-R1 population sweep asked whether other HANDLERS reach the store call and never asked whether this handler reaches it more than once. All three now share writeItemRenameCascadeTooLarge. The collab-edit half had a deeper cause than a missing arm. In applyContentViaCollabOnce's prune-and-direct-write fallback, a failing directWrite() had its error DISCARDED and the original collab error returned in its place, so the caller read a deterministic refusal as a recoverable routing problem and fell through to its own direct write — re-deriving the identical refusal from scratch. Measured 64 rewritten bodies built for one request against an expected 32. Deterministic failures now survive that branch, scoped to the three errors every call site already treats as final (open-children rejection, update conflict, cascade refusal). Everything else still returns the collab error, preserving the graceful-degradation contract the branch exists for. Status alone cannot discriminate the double-work fix — both behaviours end in 413, since the fall-through reaches the plain path's arm — so the store's build observer is now reachable from the server package via an exported test-support setter, and the test asserts the work count. Gates: gofmt, vet, go build ./... clean; go test ./... PASS on SQLite and Postgres; go test -race PASS on internal/links and internal/store with zero data races. Note -race on store runs 705s, past the 600s default timeout. * test(links): make the multi-bracket fold oracle independent (BUG-2804, codex R3) The descending-fold equivalence test built its oracle by folding RewriteBracketAt — which now delegates to RewriteBracketsAt, the function under test. The comparison was therefore circular: it could agree with itself while both diverged from the pre-refactor behaviour it exists to pin. This is the same vacuity I flagged and avoided for the ONE-element assertion, missed one case over in the multi-bracket one. The fold now uses rewriteBracketAtV0, the already-frozen pre-refactor primitive, which is the only genuinely independent oracle available. The test still passes, so the single pass does match true pre-refactor behaviour on every generated input including the overlapping ones — that agreement is now evidence rather than a tautology. Mutation-verified after the change: the fold test kills the dropped-sort and dropped-overlap-guard mutants, two of which it had previously left to other tests. Test-only. `git diff 8ddfdfd6 -- . ':!*_test.go'` is empty, so the Postgres and store -race results recorded for 8ddfdfd6 carry over unchanged. * fix(store): bound the cascade SCAN, not just the rewrite loop (BUG-2804, codex R4) `works` holds one entry per matching LINK ROW, each carrying that row's target_title, and item titles have no length bound. So the scan retained rows x title bytes before the per-source loop fired, and the content-bytes cap could not see it. The doc comment on MaxItemRenameCascadeBytes claimed the opposite — that the cascade's retention was O(1) in the linker count, with the k-linear remainder attributed to the outbox (BUG-2827). Both halves were wrong: the outbox is a different vector, and this function had k-linear retention of its own. That comment was mine, and it is the claim this unit is named after. The scan is now charged against the same budget and refuses DURING the scan, so at the moment of refusal the process holds only rows already counted. One budget covers both phases deliberately: index rows and content bytes are different quantities, but both are memory this one rename makes the server hold, and a second constant would be a second thing to tune with no separate meaning. Measured: refusal fires at 67,113,728 bytes against the 67,108,864 cap — 4,864 past crossing — with zero bodies built, on a fixture carrying 64 rows of 2 MiB titles whose full scan would have reached ~134 MB. Three test corrections, all mine: - The attack I first designed is impossible. A link to a T-byte title costs T bytes of body text, so content and retained-title bytes are COUPLED and content is always larger. The fixture guard caught it. The property that separates the two halves is WHEN the refusal fires, not size. - Two tests pinned an exact admitted count, which the new scan charge shifts by a source; one of them also used a body size that divided the cap evenly, so refuse-after-build totalled exactly the cap and could not have been detected. Both now assert that the work actually done fits the budget. - The scan test is SQLite-scoped with a measured reason: on Postgres the huge-title fixture cannot be built at all, because items carries UNIQUE(workspace_id, slug), the slug is derived from the title without truncation, and the btree tuple limit is 8191 bytes. Filed separately as BUG-2831. The bound itself is NOT dialect-specific — row count is unbounded on both backends. Gates: gofmt, vet, go build ./... clean; go test ./... PASS on SQLite and Postgres; go test -race PASS on internal/links and internal/store; zero failures, zero data races. * perf(links): never build the replacement in the projection path (BUG-2804, codex R5) bracketRewriteAt composed `collSlug + "/" + newTitle` BEFORE the match check, so every bracket paid for it including the leave-alone exit — and ProjectRewrittenLen calls it once per rewrite BEFORE the cascade's cap can fire. R1's shape, one layer down. It does not document away. BUG-2831 established that item titles carry no validation bound, so a multi-megabyte newTitle is admissible and a bracket-dense body projected gigabytes of immediately-discarded allocation ahead of the refusal that exists to prevent exactly that. The helper now MATCHES FIRST and returns a DESCRIPTION — a `qualified` flag for the slug prefix plus a display-suffix slice of the body — so nothing proportional to newTitle is built anywhere in the projection path. The real pass writes "[[", collSlug, "/", newTitle, suffix, "]]" as separate segments. bracketUnchanged compares segment-wise rather than concatenating a candidate, which would have reintroduced the same allocation to answer a comparison. The `ttLower + "|"` concatenation is gone too, spelled out as the same predicate on the same two already-lowered strings. Measured, 4096 brackets against a 1 MiB newTitle: pre-R5 shape 4,328,782,464 bytes 1.0079x newTitle per bracket now 196,704 bytes 0.0000x The second row is a new probe; the first is that probe run against a mutant carrying the old shape, not arithmetic. No semantic change, and the frozen pre-refactor oracle is what establishes that: the full differential suite passes, including 20,000 randomised inputs and the projection/pass lockstep property. The slice-growth transient stays as documented. Gates: gofmt, vet, go build ./... clean; go test ./... PASS on SQLite and Postgres; go test -race PASS on internal/links and internal/store; zero failures, zero data races.	2026-08-31 12:59:14 -04:00
xarmian	ba1255881d	fix(server): refuse a decoded NUL in a JSON request body (BUG-2803) (#1220) ... * fix(server): refuse a decoded NUL in a JSON request body (BUG-2803) The body half of BUG-2782 (path) and BUG-2784 (query). A caller-supplied string reached a Postgres text parameter, Postgres refused it, and the handler answered 500 — the honest answer is 400. WHY THE TRANSPORT RULE CANNOT BE EXTENDED, which is the whole reason this is a different fix rather than a wider middleware. ValidateQuery works because a decoded query value is a substring of the raw query with ASCII substitutions: the bad byte in the raw text IS the bad byte in the value. That property fails for a JSON body — the reachable NUL arrives as the six-character escape, all ordinary ASCII — so no request middleware can find it without decoding the body, which is the handler's job. MECHANISM, each premise measured against encoding/json rather than reasoned about: raw NUL inside a string -> decode ERR (invalid character in string literal) raw NUL after the value -> decode ERR the escape in a value -> decodes to a string CONTAINING a NUL the escape in a KEY -> same a DOUBLED backslash -> decodes to literal text, NO NUL the uppercase spelling -> not a JSON escape at all So the escape is the only vector and its substring is a sound FAST PATH (absent -> no NUL possible), but not a sufficient test: a doubled backslash carries the same six characters and decodes to text. In this product that is not hypothetical — items and documents store markdown, and a document about JSON escapes is an ordinary thing to write. The exact step is json.Decoder.Token(), which returns DECODED strings, covers object keys and arbitrary nesting (an item's fields blob), and needs no knowledge of the destination type. NOT REFLECTION over the decoded value, the other obvious design: it sees []byte fields AFTER base64 decoding, so a body carrying legitimate binary ({"b":"AQAC"} -> bytes 01 00 02) would be refused for a NUL that is not text. A token walk sees the base64 characters. No request struct has such a field today (searched: []byte with a json tag in internal/server and internal/models, non-test — only models.YjsUpdate.UpdateData, which no handler decodes from a body); the token walk is chosen so adding one later cannot silently start rejecting valid requests. BUFFERING IS NOT A COST. json.Decoder.Decode already holds the whole top-level value in memory — refill accumulates into dec.buf and grows it by doubling (encoding/json/stream.go) — so streaming never avoided the copy. Measured on the 64 MiB workspace-import shape, total allocation: stream+Decode 354.7 MiB, ReadAll+Unmarshal 256.5 MiB, ReadAll+Decode 512.5 MiB. Peak heap is order-dependent and does not discriminate; the first run of that measurement showed a 0.77x peak win that vanished when the legs were swapped, so only the allocation figure is claimed. POPULATION, measured on Postgres 17 through the real router with a control leg on every endpoint (92 mutating routes enumerated via chi.Walk; 13 probed): before: 12 of 13 DOOR (control 201 / NUL 500, SQLSTATE 22021) after: 0 of 13 — every NUL leg 400, every control leg unchanged Confirmed doors: workspace name, collection name, item title, item content, item fields value, item title via PATCH, comment body, agent role name, view name, document title, webhook secret, workspace import. workspace-token name is UNMEASURED, not clean — its control leg 500s on an unrelated FK in this fixture. The other 79 routes are unprobed, not claimed clean; the completeness argument is structural instead, and enforced by a test rather than asserted. SECOND DEFECT, named rather than slipped in: the six handlers that decoded straight off r.Body had no http.MaxBytesReader either — the cap decodeJSON has always applied — so each was an unbounded body read. Routing them through decodeJSON closes that too. COMPATIBILITY: json.Unmarshal refuses trailing non-whitespace after the JSON value where Decode ignored it. Deliberate, same direction as this fix, and the only behaviour change beyond the refusal. Trailing whitespace still passes. An EMPTY body still returns a wrapped io.EOF, because handlers_playbooks.go reads errors.Is(err, io.EOF) as "no arguments supplied" — caught by TestPlaybookRunAcceptsEmptyBody, which is exactly the wiring a helper-level change is blind to. No call site changed for the refusal itself: all 65 decodeJSON callers already turn a decode error into a 400 carrying err.Error(). Release note: a NUL character in a JSON request body now returns 400 instead of 500 on Postgres deployments. Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN * fix(server): follow the NUL refusal into JSON-encoded string fields (BUG-2803) Codex round 1 on #1220: the check scanned ONE JSON layer, and several fields cross the wire as JSON-ENCODED STRINGS rather than nested objects — an item's fields, a collection's schema, a workspace's settings. The OUTER decode of {"fields":"{...}"} yields the inner document as literal text, in which the escape is still six ordinary characters and no NUL exists, so the single-layer token walk passed it. MEASURED on Postgres 17 with a control leg on each, after the single-layer check was already in place: item.fields as a JSON-encoded string 500 control 201 collection.schema as a string 500 control 201 workspace.settings as a string 500 control 201 The error is DIFFERENT from the rest of this family, which is why it is worth reading rather than assuming: insert collection: ERROR: unsupported Unicode escape sequence (SQLSTATE 22P05) 22P05, not the 22021 the path and query halves produce. The outer string is pure ASCII so it never trips the text-encoding check; this is Postgres's own JSON parser refusing the escape inside a document bound for jsonb, which cannot represent a NUL. After this change all three answer 400 with their control legs unchanged. THE FIX: when a decoded string is itself a complete JSON object or array — the class this API re-parses downstream — walk it too, to a depth bound of 8. Recursion terminates on its own (each level is a strict substring of the one above); the bound keeps a hostile body from buying many full re-parses, and AT the bound the body is refused rather than passed uninspected, since the escape is known to be present and the walk has stopped looking. WHAT THIS OVER-REFUSES, by design and pinned by a test: the rule is structural, not destination-typed, so a plain TEXT field whose ENTIRE value is a valid JSON document carrying the escape is refused too, even though its column would have stored it. Prose ABOUT a JSON escape does not parse as a bare document, so the case is narrow, and a value of that shape breaks any consumer that parses it. The destination-typed alternative — an allow-list of the fields that arrive JSON-encoded — is exactly correct and goes stale in silence, which is the failure mode ValidateQuery's comment rejects when it explains why per-site query validators could not be written. Tests: nested documents (fields/schema/settings/array/twice-encoded), with controls for ordinary content, a doubled backslash INSIDE the nested document, a string that starts like JSON but does not parse, and prose that merely mentions the escape; the over-refusal pinned as a decision rather than left as an accident; the depth bound; and a wiring leg through the real router on SQLite, where the write would otherwise SUCCEED so a green cannot be the database doing the work. Fixtures build their JSON-encoded strings with encoding/json rather than hand-written backslashes, since the escaping rules are the subject under test. All four new tests fail with the recursion removed. Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN * test(server): build the NUL-bearing timeline fixture through the store (BUG-2803) TestTimeline_NeverEmitsACursorItWouldRefuse built its fixture through the API on a premise its own comment stated: "a structured id comes from the item's fields blob, which nothing validates on write". BUG-2803 made that false — decodeJSON now refuses a body whose strings decode to a NUL, including one nested inside a JSON-encoded `fields` string — so the API can no longer produce the row and the test 400'd on its fixture. Repaired rather than deleted, because the DEFENCE it covers is still live: rows in this shape can predate the rule, and the store has no such check of its own, so a migration, an import or any future non-HTTP writer can still produce one. The timeline must keep refusing to hand out a cursor it would then reject. The fixture now writes the blob directly, injecting the six-character JSON escape rather than a raw NUL — the blob is JSON text and both backends reject a raw NUL in it; the NUL comes into existence when Go DECODES the blob, which is exactly how the timeline ends up with one inside an entry id. The test is not vacuous under the change: it asserts the NUL-bearing id took the positional fallback, so an injection that failed to produce a NUL fails the test rather than passing quietly. This is the CONVE-23 case — a change that falsifies existing prose owes a sweep for that prose. The stale sentence was found by the test failing, not by the sweep, which is the weaker of the two ways to find it. Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN * docs(server): correct the timeline comment BUG-2803 falsified (CONVE-23) The entryID fallback's comment said note and decision ids "come from the item's fields blob and nothing validates them on write". BUG-2803 made that half false: the HTTP API now refuses a request body whose strings decode to a NUL, including one nested inside a JSON-encoded `fields` string. The sentence was true when written and nothing in this branch's diff pointed at it. The fallback still has to exist, and the corrected comment says why: the STORE has no such check, so rows predating the rule — and anything writing a blob by another path, a migration, an import, a future non-HTTP writer — can still carry one. SWEPT AND DELIBERATELY LEFT: two nearby comments (handlers_timeline_id_collision_test.go, handlers_timeline_structured_test.go) also say "nothing validates them on write". Both are about id FORMAT and DUPLICATION — an imported artifact carrying a UUID-shaped id, a hand-written blob repeating one — and this change validates neither. In context those sentences remain true, so they are left alone rather than edited into noise. Sweep command: grep -rniE "nothing validates|not validated on write|no validation on write|unvalidated" --include=*.go internal/ cmd/ — six further hits, all about other subjects (github_pr raw writes, terminal schema keys, push payload format, decodeJSON's size bound). Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN * fix(server): scope the nested-NUL walk to JSON-encoded fields (BUG-2803) Codex round 2 on #1220. The nesting check from the previous commit recursed into ANY string that parsed as a JSON document, on the argument that a structural test beats a destination-typed one. That argument was wrong in a way I had written down as an accepted trade and should have weighed as a defect: a plain-text `content` value holding a JSON snippet that merely MENTIONS the escape was accepted before this branch, is stored in a text column that has no problem with it, and was newly refused — including on RE-IMPORT of an export carrying it. Refusing input the server itself produced is a worse failure than the door the unscoped recursion was closing. Measured before the fix: a workspace whose item content held such a snippet exported 200 and re-imported 400. The walk now descends only under keys whose STRING value is a JSON document something downstream re-parses: config, events, fields, metadata, phase_data, plan_overrides, schema, settings, tags, traits. WHY A LIST IS SAFE HERE, when ValidateQuery's comment rejects exactly this shape for query parameters: there the set of names is unbounded by design (parseItemListParams turns any unrecognised parameter into a field filter), so no list could be complete. Here the set is a closed property of the wire model — a field is JSON-encoded because a Go struct declares it as a string holding JSON — and TestJSONEncodedFieldKeysCoversTheModels derives it from internal/models and fails when a new one appears. The list cannot go stale in silence. Over-inclusion is the safe direction and the list takes it: a listed key that is not really JSON-encoded costs one parse attempt and can only refuse a complete JSON document carrying the escape, while a missing key reopens a door. `traits` is listed for that reason — it carries JSON but its declaration has no comment saying so, which is exactly how the derivation test would have missed it, so the test asserts coverage in one direction only and the list is allowed to be a superset. The walk also changed shape: decoding into `any` and walking the value, rather than a token stream, because key context is needed to know which subtree is JSON-encoded. The []byte reasoning is unchanged and still holds — decoding into `any` never produces a []byte, so a base64 field is seen as its ASCII text rather than as decoded bytes that might contain a legitimate 0x00. Tests: text fields carrying a JSON document are ACCEPTED (five keys), with a leg proving the same document under a JSON-encoded key is still refused, so the pair differs only in the key; the derivation test; and the depth-bound fixture now nests under a JSON-encoded key at every level, since nesting under an ordinary key would never start the recursion and would have passed for the wrong reason. STILL OPEN, and the lead holds it: a LEGACY row whose stored fields blob already carries the escape still exports 200 and re-imports 400. That is data this fix cannot make importable without weakening the write-side refusal, and the disposition (repair sweep, flagged import, or documented acceptance) is a product ruling. Recorded on the item. Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN * fix(server): close the three body doors codex round 3 found (BUG-2803) All three verified before fixing, none taken on the reviewer's word. 1. BUNDLE IMPORT BYPASSED THE REFUSAL (P1). handlers_import_bundle.go parses pad-export.json itself rather than through decodeJSON, so the SAME workspace import — reached with Content-Type application/gzip instead of application/json — walked straight past the NUL check into Postgres. The bundle's export blob is now checked with bodyDecodesNUL before ImportWorkspace, answering the same 400. Test drives a real tar.gz through the router with a clean-bundle control leg, because this path answers 400 for a dozen unrelated reasons (bad gzip, out-of-order tar, duplicate entries) and a bare 400 would prove nothing. 2. ONE CALLER SWALLOWED THE NEW ERROR (P2). handlers_admin.go's test-email endpoint read `if err := decodeJSON(...); err != nil || input.To == ""` and fell back to the admin's own address, so a body carrying a NUL answered 200. An ABSENT body legitimately means "send it to me"; a body that is present and REFUSED is a different thing, and collapsing the two turns a validation error into a success. The two cases are now separated on errors.Is(err, io.EOF). 3. THE COMPLETENESS TEST COULD NOT SEE PAST TWO CALL SHAPES (P2). It scanned for json.NewDecoder(r.Body) and io.ReadAll(r.Body), so it was blind to io.ReadAll(io.LimitReader(r.Body, n)) — a shape ALREADY in the package — and to any alias or helper. A completeness test that misses a live example is worse than none, because it reads as coverage. It now scans for the thing that cannot be spelled around, a reference to the request body at all, and requires every FILE touching one to be accounted for with a written reason. Both directions are asserted: an unaccounted file fails because a door may have opened, and an accounted file that no longer touches a body ALSO fails, so the list cannot rot into stale excuses that quietly cover a future reader. Verified with a positive control (an added body reference in an unlisted file fails) and a negative one (a stale entry fails). FOUND BY THAT WIDENED SWEEP, and fixed here rather than filed: the raw artifact import (POST /workspaces/{ws}/import-artifact) takes TEXT, not JSON, so it never went through decodeJSON and inherited neither the NUL refusal nor the path/query rule — a body is neither. A raw NUL or invalid UTF-8 reached the store and Postgres answered 22021, which the handler turned into a 500 for what is a client error. It now applies bindableText, the same predicate ValidatePath and ValidateQuery use, and answers 400 invalid_body. Note the shape difference from the JSON half: there the ESCAPE is the vector because a decoder rejects a raw NUL; here the RAW BYTE is, because nothing is in the way. Each fix has a mutation run against it: disabling the bundle guard fails the bundle test, disabling the artifact guard fails the artifact test, and both controls still pass. Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN * fix(server): the escape gate was unsound, and YAML has its own (BUG-2803) Codex round 4, two P1s, both reproduced before fixing. 1. THE FAST PATH LET A REAL NUL THROUGH. bodyDecodesNUL gated on "does the raw body contain the six-character escape". That is unsound: the BACKSLASH itself can be written as an escape, so a body carrying \u0000 contains no literal six-character sequence anywhere in its raw bytes, while the OUTER decode manufactures one inside the string — and if that string is re-parsed as a JSON document (jsonEncodedFieldKeys) the second parse turns it into a real NUL. Measured through the real router before the fix: the oblique spelling answered 201 where the direct one answered 400. The mistake was applying a fact about how a NUL is spelled INSIDE a decoded string to the RAW BYTES, where the backslash can itself be an escape. That is the same layer-confusion this whole bug is made of, for the third round running. The gate is now a BACKSLASH. Every JSON escape mechanism requires one, so a body with no backslash has decoded strings byte-identical to its raw bytes, and a raw NUL cannot survive the decoder — no backslash therefore means no NUL, at any depth, however spelled. Bodies WITH one pay for an exact answer, a larger set than before (any nested JSON carries a backslash-quote), which is the cost of being correct. The same correction applies to the per-string pre-filter one level down. 2. YAML HAS ITS OWN ESCAPE VOCABULARY. The raw bindableText check added last commit passes a double-quoted scalar `title: "a\0b"` — no NUL in the request bytes — and the YAML decode manufactures one. Measured before the fix: that artifact imported 201 with a NUL in the item title. The decoded artifact is now checked too: title, body, and every frontmatter field value, walked because a playbook's `arguments` is a nested structure rather than a scalar. Keys are checked as well as values, on the same precautionary grounds ValidateQuery states for query parameter names. Same shape as the JSON half in both cases: a value that is harmless until a SECOND parse, checked at the layer that can see it. Tests: the oblique spelling joins the nested-document table, and the YAML escape joins the artifact table. Each is mutation-verified — reverting the gate to the substring fails the oblique case only, and disabling the post-decode artifact check fails the YAML case only, with the raw-byte cases still killed by the raw check. That per-leg discrimination is the point: it shows each check earns its own keep rather than being covered by its neighbour. Prose corrected where this falsified it: jsonNULEscape's "it is the ONLY spelling" is true of the escape and was being used to justify a filter on the raw bytes, which is a different claim. Both now say so explicitly. Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN * fix(server): multipart text fields and the bundle manifest (BUG-2803) Codex round 4's two P2s. Both are the same shape as the rest: a caller-supplied string reaching a text comparison through a door the earlier fixes did not cover. 1. MULTIPART TEXT FIELDS. The multipart body is deliberately exempt from the JSON rule — its payload is binary blob content and must not be scanned for text validity — but its TEXT fields are a different thing. `item_id` goes to ResolveItem and into a database comparison exactly as the query-string channel does, and that channel has been validated at the transport since BUG-2784; the form channel was not. multipartValues now drops values that are not bindable text, which makes an unusable value indistinguishable from an absent one — the disposition resolveUploadItemID already applies to empty values. The uploaded FILENAME gets the same predicate, with a fallback to a generic name rather than a refusal: the bytes are fine, only the label is unusable. A NEGATIVE RESULT worth recording, because it changed the test: a RAW NUL in the multipart header is NOT the vector. Go's multipart reader refuses it as a malformed MIME header line before any handler sees it (measured: 400, "malformed MIME header line"). The reachable spelling is the RFC 5987 encoded form, filename*=UTF-8''sh%00ot.png, which the header parser accepts and percent-decodes afterwards. The first version of this test used the raw form and was testing a vector that does not exist. 2. THE BUNDLE ATTACHMENT MANIFEST. A second JSON document inside the tar.gz, parsed directly like pad-export.json was, so it needed the same check. Without it a NUL in a manifest string reached rehydrateAttachment, whose failure is logged and SKIPPED — so the import reported success while silently dropping the attachment. The skip-on-failure behaviour is pre-existing and deliberate (a partial restore beats none); refusing the bad INPUT is what stops it being reached this way. Left as it is, and named rather than quietly changed. A VACUOUS ASSERTION THE MUTATION CAUGHT, recorded because the test would otherwise have shipped as coverage: the filename leg first asserted `!strings.ContainsRune(body, 0)` on the RESPONSE, which is JSON — a NUL in the filename comes back as the six-character escape, not as a 0x00, so the check passed whether or not the fix was present. It did pass with the fallback disabled. Now it decodes the response and asserts the replacement name. The item_id leg had the mirror-image weakness: it asserted "not a 500", which is the Postgres-only symptom, so on SQLite it would have passed either way; it now asserts the request behaves exactly like the no-value control. Every fix in this commit has a mutation against it, and each kills only its own leg. Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN * refactor(server): drop the now-unused escape constant (BUG-2803) The gate became a backslash check, which was the last production use of jsonNULEscape; golangci-lint's unused check failed on the next run. Its documentation was load-bearing, so the explanation moved into bodyDecodesNUL's comment rather than being deleted with the variable — including the distinction that made the old gate wrong (the escape has one spelling INSIDE a decoded string, which is not a claim about the raw bytes). Caught by re-running lint on the tip after the previous commit rather than trusting the run from the tip before it. Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN * fix(server): rune-safe truncation and User-Agent sanitising (BUG-2803) Codex round 5 was asked for the POPULATION rather than a confirmation — "enumerate every remaining way a caller-supplied string can reach a database text or jsonb parameter without passing a validity check" — and returned three residual classes with their sinks. Two are fixed here; the third is filed, because measuring it needs a fixture this unit should not grow. 1. TRUNCATION CAN UNDO THE VALIDATION. Four sites cut a caller string with a plain byte slice (name[:120], input.Name[:200]). If the boundary lands inside a multi-byte rune the result ends in a partial sequence and is no longer valid UTF-8 — so a value that PASSED the body check a few frames earlier arrives at the store unbindable, and Postgres answers 22021 for a request the server already accepted. This is the interesting one, because no input-side round could have found it: the defect is downstream of validation, and it is invisible with ASCII fixtures, which is what every test in that area used. truncateBindableText walks back off continuation bytes and drops the straddling rune. Tested with 2-, 3- and 4-byte runes so an off-by-one walk-back cannot pass them all, and with a counterfactual leg asserting the naive slice really does produce unbindable output for the same input — without it the cases would pass against an implementation that did nothing. 2. USER-AGENT REACHES TEXT COLUMNS. It lands in activities.user_agent (three document paths, the connected-apps revoke) and sessions.user_agent (three login paths), and no rule here sees a header. The disposition is SANITISE, not refuse, and that is deliberate: a header is metadata this server chose to record, not something the caller asked for, so a malformed one must not turn an otherwise fine request into a 400. The two sites that HASH the header are left alone — sha256 over arbitrary bytes is well defined, and changing what is hashed would invalidate every stored UAHash. The filing's own earlier probe had recorded User-Agent as NOT reproducing on the item-create path. That was true and did not generalise; these are different sinks. 3. NOT FIXED, FILED: the OAuth form-encoded bodies (/oauth/token, /oauth/authorize/decide, /oauth/revoke, /oauth/introspect) parse url-encoded form data outside the shared body validator, with connection_name reaching oauth_connections.name and client_id reaching the oauth_clients.id lookup. This was the ORIGINAL subject of BUG-2803 before the filing was re-scoped, and it was recorded then as unreachable without a fosite-backed fixture. That is still true, and round 5's sink list is far more than the filing had. Filed rather than guessed at. Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN * fix(server): narrow the gate, stop refusing natural-shape fields (BUG-2803) Codex round 6 plus one measurement of my own. Three changes, one of them a revert of something I got wrong in the previous commit. 1. THE GATE COST TOO MUCH, so it is narrower and still sound. The previous commit gated the walk on "does the raw body contain a backslash", which is correct but catches every body carrying nested JSON (each `\"` is a backslash). Measured on a ~377 KB import-shaped body: 60106 allocs/op with that gate versus 30073 with the walk disabled — the walk was running on ordinary traffic. The gate is now the four bytes that begin any \u escape for a character below U+0100. The argument: to manufacture the six-character NUL escape inside a decoded string, each of its characters arrives either literally from the raw bytes — in which case the raw contains the escape, which begins with that prefix — or from a \u escape of its own, and the three characters involved (backslash U+005C, 'u' U+0075, '0' U+0030) all sit below U+0100, so those escapes begin with it too. Back to 30073 allocs/op, identical to the walk-disabled build. That argument is the same KIND of reasoning that was wrong two rounds ago, so it does not stand on its own: a differential test runs the gated function against an UNGATED walk over a corpus built to attack it — oblique backslash, upper-case hex, an escaped 'u', an escaped '0', a doubled backslash — and fails on any disagreement. It also asserts the corpus contains both answers, since agreement over a one-sided corpus would be vacuous. Reverting the gate to the old substring fails it. 2. THE CHECK REFUSED THE NATURAL SHAPE OF ITS OWN FIELDS. `tags` and `fields` accept both a JSON-encoded STRING and their natural array/object form, and the walk propagated "this subtree is JSON-encoded" into containers — so a free-form tag whose whole value happened to be a JSON document was refused, though nothing re-parses it. Measured: refused before, accepted now, while the JSON-encoded spelling of the same field is still refused. The flag now marks only a direct STRING child of a listed key. 3. REVERTED: I wired the three LOGIN paths to the User-Agent sanitiser last commit, before reading store.CreateSession. It HASHES the header and stores no text — the round-5 enumeration named "sessions.user_agent" and I took the name for a column. The change would have been actively harmful: login would store sha256(sanitised) while middleware_auth still compares sha256(RAW), so every session from a client with a non-UTF-8 User-Agent would fail validation. A sink named in a review is a pointer to verify, not a finding. The real sink is activities.user_agent, from three document paths and the connected-apps revoke. 4. And the wiring leg codex asked for, on that real sink: a request through the router with a malformed header, reading the STORED value out of the activities row, with a control asserting an ordinary header is kept VERBATIM. Unwiring the production call site fails it; the helper's unit test does not notice. Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN * fix(server): apply the key rule at every level, not once (BUG-2803) Codex round 7, both findings, the first confirmed by measurement. 1. THE RECURSION WENT ONE LEVEL TOO DEEP. Once the walk descended into a JSON-encoded string it treated the WHOLE subtree below as JSON-encoded, so a value nested two levels down — an ordinary string inside a `fields` blob that happens to hold JSON text — was refused. That is a false rejection, and the measurement says so plainly. With the depth-2 check disabled, on Postgres 17: depth 1 (the fields blob itself) -> 400 (correct: Postgres parses it) depth 2 (a string INSIDE the blob) -> 201 (accepted, no error) control -> 201 The handler parses `fields` ONCE. The inner text is re-escaped when the blob is written, so what Postgres receives has a doubled backslash and no escape at all. Only the document Postgres itself parses can carry a fatal one. The nested call now passes false rather than true, which makes this a KEY RULE APPLIED AT EVERY LEVEL rather than a depth limit: a JSON-encoded key INSIDE a document still recurses (pinned by a test), an ordinary one does not. Same correction as round 6's natural-shape fix, one level further in — I fixed the sibling case and left this one, which is CONVE-18's lesson about my own enumeration being a sample too. I checked whether anything re-parses a value inside the blob before loosening this, rather than assuming: `arguments` was the candidate, and parsePlaybookArguments asserts it is a native ARRAY (raw.([]any)) rather than a JSON string, so it is covered by the natural-shape rule and needs no second parse. 2. AN ERROR MESSAGE THAT SENT CLIENTS THE WRONG WAY. The OAuth dynamic client registration handler prefixed every decode failure with "Request body must be JSON". A body carrying a NUL is valid JSON, so that message sends a client hunting a syntax error it does not have. The two failures are now distinguished. Round 7 also reports no break in normal CLI, MCP or web-client request generation — they marshal JSON and encode paths and query parameters — which is the first thing any round has said about the client surface. Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN * fix(server): complete the artifact check, make the walk path-aware (BUG-2803) Codex round 8. It confirmed round 7's two fixes, then found two real defects and two inaccurate comments — the comment half being the angle the round was asked for. 1. THE ARTIFACT CHECK MISSED TWO REACHABLE FIELDS. artifactIsBindableText walked the decoded artifact by TYPE, so it never covered Provenance — whose strings are rendered into a Markdown footer appended to the stored content — and never matched Arguments, declared []map[string]any, a concrete slice type the walk's []any case does not match. A YAML NUL escape in either reached storage. It now MARSHALS the artifact and searches the output for the escape encoding/json produces. A type switch over a struct that grows is a list that goes stale in silence; marshalling covers every exported field, including ones added later. The one thing it cannot see is invalid UTF-8 (which marshals to U+FFFD), and it does not need to: step 2 rejects that in the request bytes, and YAML cannot manufacture it from valid input — its escapes name code points, where \0 names a NUL. Both new cases fail with the check disabled; the raw-byte cases still pass, killed by the raw check, so each leg is discriminating. 2. THE WALK WAS NOT PATH-AWARE. A collection may declare a user field literally named `schema` or `tags`. The walk consulted the wire-key list at every level, so `{"fields":{"schema":"..."}}` treated a user field name as a wire key and refused valid text holding a JSON example. The key list is now consulted only OUTSIDE caller data — not under a natural `fields` object, not inside an element of a `tags` array, not inside a re-parsed document. Combined with round 7's fix that makes the descent exactly one level deep BY CONSTRUCTION, which is why the depth counter is gone: with the flag no longer inherited, a bound could never fire, and dead protection reads as protection. The depth-bound test is replaced by one that pins the property directly — an escape IN the parsed document is refused, one BELOW it is accepted, and a wire-key-shaped user field does not restart the descent. 3. THREE COMMENTS CORRECTED, all mine, all of the kind a reader would believe without checking: - MaxBytesReader: Close FORWARDS to the underlying body rather than being a no-op, and with a nil writer there is no automatic 413 — the cap surfaces as a read error the callers turn into 400. Behaviour unchanged; only the claim was wrong. - parseArtifactRequest said "three checks" while implementing five, and its returns list omitted ErrArtifactUnbindableText. Both added by this branch, which is exactly the prose a change is most likely to falsify (CONVE-23). - errJSONBodyNUL claimed all 65 callers surface its message. The STATUS is uniform; the wording is not — several substitute a generic string. 4. And one in a test: the timeline fixture said both backends hold a CHECK constraint a raw NUL violates. items.fields is a plain TEXT column with no CHECK on SQLite. What was OBSERVED is "SQL logic error: malformed JSON"; the likely source is an expression index over json_extract, and that attribution is recorded as NOT verified rather than asserted. Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN * fix(server): a regression this branch introduced, and the same trap again (BUG-2803) Codex round 9. Both findings are mine, one of them a regression from the round-8 restructure two commits ago. 1. THE ROUND-8 RESTRUCTURE REOPENED THE ORIGINAL DOOR. Taking the JSON-encoded branch for a listed key skipped the plain "does this string contain a NUL" check and asked only "does the document this string carries hold an escape". Those are different questions. So {"fields":"a<NUL escape>b"} — a direct NUL in the fields value, the very first case this whole change closed — was accepted again. Both checks now run. The test pins all three legs: a direct NUL in the fields string, an escape inside the fields document, and an ordinary fields string that must still be accepted, so the first two cannot pass merely because everything under a listed key is refused. 2. THE ARTIFACT CHECK FELL INTO THE TRAP IT WAS WRITTEN AGAINST. It searched the MARSHALLED bytes for the escape sequence, and a value holding the six LITERAL characters marshals to a doubled backslash which still contains that sequence as a substring — so valid content was refused. Artifacts are documentation; text about a JSON escape is exactly what one carries. Worse than the bug: the comment I wrote asserted the ambiguity "cannot arise here". It was the same doubled-backslash case bodyDecodesNUL exists to resolve, one function away, and I wrote a sentence explaining why it did not apply instead of checking. The marshalled form is now decoded again and walked with the same machinery — the round trip is what makes every field reachable without a type switch, the walk is what makes the answer exact. Its test asserts literal escape TEXT is accepted in title, body and a field value, with a counterfactual leg asserting a real NUL in each of those places is still refused, so acceptance cannot come from the check doing nothing. Reverting either fix fails its test and only its test. Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN * docs(backup): the one case where an export is not importable (BUG-2803) Codex round 12, an operational pass. It found no migration or config requirement, and two documentation gaps. docs/backup.md promises that application-level export/import is portable across SQLite and PostgreSQL. Since BUG-2803 that has one exception: a workspace whose stored data contains a NUL exports fine and is refused on import. It can only affect data written before the rule existed and only on SQLite, which accepted it — a PostgreSQL instance never stored one. `pad db migrate-to-pg` has the SAME problem and reports it worse: it copies rows directly and never passes through the import guard, so a legacy row fails against PostgreSQL's JSONB parser partway through the copy rather than being refused up front. That is the likelier way an operator meets this, since it is the operation that puts an entire old SQLite database in front of PostgreSQL for the first time. Recorded on BUG-2810, which owns the preflight and repair. Round 12's other finding — that the PR's stated release note covered the JSON 500-to-400 change and none of the rest — is fixed in the PR body rather than in the tree. Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN * test(server): close two blind spots the tests themselves had (BUG-2803) Codex round 13, asked whether the new TESTS are sound. Five findings; these are the two that were self-contained. The other three are recorded on the item with what each needs. 1. THE COMPLETENESS SCAN WAS BLIND TO FORM BODIES. It matched only `.Body`, so FormValue / ParseForm / MultipartForm — which read the request body just as surely — were invisible. It therefore reported full coverage while the OAuth form-encoded handlers were entirely outside its view. Widened, and it immediately failed on handlers_oauth.go, which is the instrument working. That file is now ACCOUNTED FOR AS A KNOWN GAP rather than as safe: the OAuth handlers read form-encoded bodies that no rule in this family covers (the transport rules see the query half of r.Form, not the body half), tracked as BUG-2811 and needing a fosite-backed fixture to measure. The test now STATES the gap instead of being blind to it, which is the difference between a completeness claim and a completeness appearance. 2. THE TRUNCATION TEST ADMITTED AN IMPLEMENTATION THAT RETURNED "". Its assertions were: within the limit, bindable text, a prefix of the input. An empty string satisfies all three. It now also asserts that an input fitting the limit comes back UNCHANGED, and that no more than one rune (4 bytes) is lost to the boundary — so a truncator that drops too much fails, not just one that keeps too much. Both were found by asking whether a broken implementation would pass, which is the question CONVE-12 is about and which I had applied to the production code and not to these two tests. Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN * test(server): the three remaining round-13 gaps (BUG-2803) Codex round 13's other three findings, all of the same shape: a test that would stay green with the production change reverted. 1. THE MANIFEST CHECK WAS UNTESTED. The bundle test built archives containing only pad-export.json, so disabling the INDEPENDENT attachment- manifest check left the suite green. The new test builds a bundle with both entries, differing only in the manifest, so a refusal cannot come from the export half. Verified by disabling each check separately: only the matching test fails, so the two are independently covered. 2. THE TEST-EMAIL CHANGE HAD NO HANDLER-LEVEL TEST. Every existing leg exercised decodeJSON, so reverting handlers_admin.go to default EVERY decode failure to the admin's own address passed them all. The new test drives the real endpoint with a wired mock sender and pins the distinction that used to collapse: an ABSENT body still means "send it to me" (control), an ordinary body still sends (control), and a body that is present and refused answers 400 rather than being reinterpreted as the default recipient. 3. THE MULTIPART LEG CHECKED ONE BYTE CLASS. A filter rejecting NULs while letting malformed UTF-8 through would have passed it. It now drives both, which matters because invalid UTF-8 is the class that reaches Postgres as 22021 on a UTF8 database. Round 13 was asked whether the new TESTS are sound — deterministic, order-independent, and failing on broken code. It reported the fixtures isolated and found five ways they were not discriminating. Two were fixed in the previous commit; these are the rest. Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN * test(server): pin the wiring at every call site, not one (BUG-2803) Codex round 14 confirmed round 13's five, then found the same shape one level out: reverting a SINGLE call site back to the unsafe form left the whole suite green, because the surviving fixtures are ASCII and a helper's unit test does not care who calls it. TestTextSafeHelpersAreUsedAtEveryCallSite asserts the wiring STATICALLY rather than adding a fixture per site (an OAuth connection, a cloud login, four audit paths). A byte-slice truncation of a caller string fails it, and so does a raw User-Agent read outside the exempt set. Both directions are checked: finding none of the SAFE form also fails, so a scan that silently matched nothing cannot pass forever. The User-Agent exemptions carry counts rather than being blanket, so a NEW raw read in an exempt file still fails. All four reads in handlers_auth.go are exempt because they feed a HASH — CreateSession hashes the header and stores no text — and sanitising before hashing would be actively harmful: login would store sha256(sanitised) while the session check still hashes the RAW header, failing validation for every client with a non-UTF-8 User-Agent. middleware_request_text.go's one raw read is requestUserAgent itself. Verified by reverting one truncation call site and one User-Agent call site independently; each fails the test. Round 14's third finding is fixed behaviourally rather than statically, because the static scan cannot see it — handlers_oauth.go is already listed for its form-body reads. TestOAuthRegisterRefusesNULBody drives the real dynamic-registration endpoint with cloud mode and an OAuth server wired, with a control leg registering successfully, and pins both the refusal and the message split: the body IS valid JSON, so the answer must not send a client hunting a syntax error. Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN * fix(server): match wire keys the way the decoder does (BUG-2803) Codex round 16, asked whether this change is consistent with its siblings in the same file and extensible by someone who did not write it. It found a live bypass instead. encoding/json matches an incoming key to a struct field by an exact match first and a CASE-INSENSITIVE one otherwise, so {"Fields":...} and {"FIELDS":...} land in ItemCreate.Fields exactly as {"fields":...} does. The walk looked the key up case-SENSITIVELY, so it skipped the nested document for a body the handler went on to accept, and the database answered the original 500. Measured before the fix: `fields` refused, `Fields` and `FIELDS` accepted. This is the same defect shape as everything else in this unit — a check that agrees with one layer's rules while the layer that actually consumes the value uses different ones — which is why the fix is a PREDICATE rather than a wider map: the map is the vocabulary, and the matching RULE belongs to the consumer. Someone adding a key should not also have to remember to add its spellings. The test drives six spellings including mixed case, with a control asserting an unlisted key stays caller data in any casing, so this is case-insensitive matching rather than matching everything. Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN * fix(server): fold keys the way encoding/json folds them (BUG-2803) Codex round 17, first of five findings. The previous commit fixed the ASCII half of key matching and left the Unicode half, which is this bug's own pattern one more time. encoding/json matches with Unicode SIMPLE FOLDING, not lower-casing. U+017F LATIN SMALL LETTER LONG S folds to 's', so "ſchema" reaches the `schema` struct field while strings.ToLower("ſchema") is unchanged and missed the allowlist — a nested NUL under that spelling reached the handler undetected. Matching is now strings.EqualFold against each canonical key. The test carries both a lower-case fold spelling and an upper-case one alongside the ASCII cases, and keeps its control asserting an unlisted key stays caller data in any casing. The other four round-17 findings are recorded on the item rather than patched here: they are genuine layer disagreements (duplicate keys merging differently in a typed decode than in a map, a scan-failure disposition on inputs the typed decode tolerates, and unknown-field policy) whose fixes are design decisions rather than corrections, and this seat is near its context bar. Each is written up with the measurement it needs. Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN * fix(server): pin that a NUL-bearing manifest refusal keeps the partial workspace (BUG-2803) Codex round 18. The comment on the manifest NUL branch said refusing the input "stops it from being reached this way" and stopped there, which reads as though the refusal undoes the import. It does not. A plain error with a non-nil workspace keeps the partial workspace, exactly as every other manifest failure in this loop does — the rollback branch fires only for *importStatusError, and mid-stream manifest failures intentionally keep what was imported (TASK-896). Returning a rollback-shaped error here would give NUL-bearing manifests different semantics from malformed ones, which is a change to the bundle-import contract rather than a fix to this bug. So the behaviour is unchanged and now DELIBERATE: the comment states it, and the test asserts the persisted state rather than only the HTTP answer. Mutation: routing the branch through *importStatusError makes the refusal roll back, and the new assertion fails naming the release note it would falsify. The pre-existing status/body assertions do not notice. Claude-Session: https://claude.ai/code/session_01AUvLoXsKdS5sdpYju6rj4p * docs(server): record the four map-model disagreements as dispositions, and pin them (BUG-2803) Lead ruling day-68 is land-and-follow: this branch lands on its measured commits, and the token-stream rewrite is the BUG-2812 unit's spec rather than a late restructure of an 18-commit branch under review pressure. That makes the four open findings from rounds 16-17 something to WRITE DOWN precisely, not something to leave in a trail comment. The doc comment on bodyDecodesNUL now carries all four, with the one root cause named: this scan decodes into map[string]any and the typed decode does not agree with that model about keys. Two under-refuse (duplicate-key merge; scan-failure passthrough) and are BUG-2812's spec - both dissolve under a walk that never builds values. Two over-refuse (unknown fields; case-variant duplicates) and are ACCEPTED, because refusing is the safe direction. The asymmetry is stated rather than smoothed over: within the map model, (1) and (4) are one defect seen from two sides and only one of them fails safe. Finding (3) is an observable compatibility change - a forward-compatible field carrying a NUL escape now gets a 400 where it got a 200 - so it goes in the release note as well as here. A qualification only protects where the actor meets it. All four are pinned by a test, measured on this tip rather than carried over from the round-16/17 write-up. The two known-gap legs assert the WRONG answer on purpose: when BUG-2812 lands they FAIL, naming the doc comment and the release note as what to update. Both gap legs carry a premise assertion - the same bodies with the disagreement mechanism removed ARE detected - without which they would pass against a check that detected nothing. Claude-Session: https://claude.ai/code/session_01AUvLoXsKdS5sdpYju6rj4p * test(server): wire release-note item 10 to the router, with its before-state measured (BUG-2803) The disposition test proves bodyDecodesNUL RETURNS true for an unknown field carrying a NUL escape. The release note claims the API answers 400. Those are different claims and only the second one is what an operator or client author reads - CONVE-19, my own convention: a direct-call test vouches for the component, not its binding. Two legs, and the control is the load-bearing one. An unknown field with an ordinary value must still be ACCEPTED, so this pins "refused for the NUL" rather than "refused for being unknown". The handler does not reject unknown fields; if it ever started to, the note's explanation would be wrong while its status code stayed right, and no status-code-only assertion could see that. The before-state is measured rather than asserted from memory. Disabling the check makes the same request answer 201 - which is main's behaviour, since decodeJSONWithLimit there unmarshals straight into the typed value and the key is dropped. So "answers 400 where it answered 200" is a measurement in both directions, not a recollection of one. Claude-Session: https://claude.ai/code/session_01AUvLoXsKdS5sdpYju6rj4p * docs(backup): the NUL rule lives in the binary, not the database (BUG-2803, BUG-2813) Codex round 19, the fresh-angle deploy/rollback/mixed-version pass. docs/backup.md said a NUL-bearing row "can only affect data written before that rule existed, and only on SQLite". The second half is true. The first half is false, and the reason is the interesting part: the guard is in decodeJSONWithLimit, so the invariant is a property of the running BINARY, not of the database. On SQLite any window where an older binary serves the same database can still write one - a rollback after upgrading, a staged rollout with an old and a new instance sharing a database, a second older instance on the same file. The window closes, the guard returns, and the rows are already stored, behaving exactly like genuinely old ones. A rollback is an ordinary operational move, so this is not an exotic path. The doc now states the binary-version dependence, says which dialect is affected and why PostgreSQL is not (it refuses a NUL itself, at every version), and gives the operational answer: drain writes from older binaries before the new one serves, or roll forward rather than back. Store-layer enforcement - so the running build stops mattering - is filed as BUG-2813 rather than added here. It is a dialect-level change and the day-68 ruling on this unit is land-and-follow. The same false implication was carried by the PR's release note calling such a workspace "legacy"; corrected there too. Claude-Session: https://claude.ai/code/session_01AUvLoXsKdS5sdpYju6rj4p * docs(server): cite the ruling in house style, not the team-room day counter (BUG-2803) "lead ruling day-68" is the internal day counter, which means nothing to anyone reading this repo and is inconsistent with every other citation in it - the codebase cites a lead ruling by DATE or by BUG ref, never by day-N. Replaced with the bug ref, which is the part a reader can actually follow. Claude-Session: https://claude.ai/code/session_01AUvLoXsKdS5sdpYju6rj4p * docs(server): drop a commit count I had already measured as wrong, and stop asserting a cause I borrowed (BUG-2803) Two defects in a comment I wrote an hour ago, both of the kind this unit's trail keeps recording. "an 18-commit branch" - the branch was 20 commits at b0192871 when I counted it this session, and is more now. 18 came from the previous checkpoint's own miscount, which I had ALREADY identified and written up before I typed it again here. A number that arrives inside a sentence about something else does not feel like a claim, which is exactly why it survives. The count is incidental to the argument, so it is gone rather than corrected - a figure that has to be maintained to stay true is a liability in a doc comment. "this branch's one regression came from exactly that" - the ruling's reasoning, restated by me as a verified fact. The regression I know about came from wiring a fix off a reviewer-named sink list without reading the mechanism, which is adjacent to "restructuring late under review pressure" but is not the same mechanism, and I did not check whether it is the one the ruling meant. Now attributed to the ruling and stated as its reasoning, with the part I can defend - the review loop finding something in nearly every round indicates a design problem - carrying the argument. Claude-Session: https://claude.ai/code/session_01AUvLoXsKdS5sdpYju6rj4p * fix(server): sanitise the MCP audit tool_name, and correct three claims wider than their evidence (BUG-2803) Codex round 20, asked for a POPULATION rather than a confirmation (CONVE-24). It returned a covered list AND four findings; this commit carries the two that belong to this unit plus the doc corrections. ## The door: MCP audit is a second reader, not a pass-through parseMCPRequestBody runs its OWN json.Unmarshal and binds the decoded method / params.name to mcp_audit_log.tool_name, TEXT NOT NULL. A six-character NUL escape therefore arrives as a real NUL: PostgreSQL refuses the audit INSERT with 22021 - the exact symptom this unit exists to remove - and SQLite stores an unprintable tool name. Nothing upstream catches it; the /mcp transport decodes the JSON-RPC envelope itself rather than through decodeJSON, so the body rule never sees the request. Measured before fixing: the decoded name reached the column intact. This unit's own completeness map had CERTIFIED that reader as safe, on the grounds that "decoding still happens in the MCP dispatcher". That is true and it does not bear on what this middleware persists - a correct description of a mechanism, with no question asked about what it does, sitting in the one artifact whose job is to say the population is covered. Corrected there too. Disposition is SANITISE, not refuse, following the User-Agent precedent from earlier in this unit, and the rule now lives in one extracted helper (sanitiseStoredText) with the reasoning attached: the body rule refuses because the caller asked to store that value; this serves metadata the SERVER elected to record, where failing the write would lose the audit row for precisely the request most worth auditing. Both caller-derived returns are cleaned inside parseMCPRequestBody, so both call sites - the ok path and the denied path - are covered at the choke point rather than at either caller. Both are tested: params.name AND the method path. Mutations un-sanitising each one compile and kill only their own leg. ## Three claims corrected, all wider than their evidence - "all 65 call sites" in server.go: measured 70. Removed rather than corrected, because the number has to be maintained to stay true and says nothing the sentence needs. - docs/backup.md said a NUL "cannot be stored in a text or JSON column" absolutely, two paragraphs above my own text explaining that SQLite accepts one. Now stated as what it is: an application rule Pad enforces on both dialects, which is exactly why it has to be enforced. - artifact_import.go said such a value "cannot be stored under any encoding this product supports". Refuses, not cannot - stating a policy as a capability tells the next reader SQLite enforces something it does not. ## Filed, not fixed BUG-2814 - guarded writes re-emit at-rest NULs (move/copy/restore/ fields-patch), propagating a legacy value to rows that never had one. Distinct from BUG-2813: that one is about writing a NUL while an old binary serves, this is the fixed binary SPREADING one already present. Both dissolve under the same store-layer enforcement, so they are filed to be designed together rather than patched at each of a long and moving list of re-emit sites. Declined: round 20 also reported the release-note assertions as unsupported. They live in the PR body, which a read-only sandbox cannot see - the claim is about the reviewer's visibility, not the diff. Claude-Session: https://claude.ai/code/session_01AUvLoXsKdS5sdpYju6rj4p * fix(server): sanitise before testing for emptiness, so the audit fallback survives (BUG-2803) Codex round 21 ranked this the most dangerous un-probed lens, and it is a boundary my own round-20 fix created. parseMCPRequestBody tested env.Method == "" and p.Name == "" BEFORE sanitising. A value made entirely of NUL escapes is non-empty as decoded and empty once cleaned, so it passed over the fallback and was then blanked - storing an empty tool_name in a TEXT NOT NULL column. That is exactly the silent drop the "(unknown)" / "tools/call" fallbacks exist to prevent; the function's own doc comment says so. Measured before fixing: both shapes returned an empty tool_name. Fixed by ordering rather than by adding guards - clean first, then test - so the invariant is structural instead of something each return has to remember. Same by-construction preference as the symmetric-gate fix earlier in this unit. Worth recording that my first patch was WRONG in a way that compiled: I put the sanitise above the json.Unmarshal that populates env, so the method would always have been empty. Caught by printing the patched function and reading it, not by trusting the script saying "patched". Claude-Session: https://claude.ai/code/session_01AUvLoXsKdS5sdpYju6rj4p * fix(server): classify MCP audit on the raw method, and trim only JSON whitespace (BUG-2803) Codex round 22. Two P2s, both measured before fixing. ## A forgeable audit row - my own regression from the round-21 fix The round-21 change reordered sanitise-before-compare so the fallback would survive an all-NUL value. That reorder made the CLASSIFICATION read the sanitised method, so "tools/<NUL>call" cleaned up INTO the literal "tools/call" and the parser then lifted params.name and hashed the arguments for a method that was never tools/call. Measured: tool_name="pad_item" with a full 64-character args_hash - an audit row indistinguishable from a genuine pad_item call, mintable by anyone who can send a request. Worse than the review described it. Fixed by splitting the two jobs, which were never the same job: dispatch decisions read what the client actually SENT; sanitising is for the value that gets STORED. The round-21 boundary is preserved - a method empty only after cleaning still falls back to "(unknown)". Fixing one boundary and creating another in the same function is worth naming: the reorder was correct for the case it addressed and I did not ask what else read that value. ## Go whitespace is not JSON whitespace The empty-body shortcut used bytes.TrimSpace, i.e. unicode.IsSpace, which strips \v, \f, U+00A0 and more. encoding/json accepts none of them. So a body of just \v trimmed to empty, returned io.EOF, and an EOF-tolerant caller - playbook run treats errors.Is(err, io.EOF) as "no arguments supplied" and runs anyway - took a syntactically invalid body for an ABSENT one. Now trims exactly the four bytes JSON calls whitespace. The test drives both directions, because only the pair discriminates: real JSON whitespace must still shortcut to EOF or the playbook contract breaks, and non-JSON whitespace must not or the divergence survives. Reverting to TrimSpace compiles and fails three legs. Claude-Session: https://claude.ai/code/session_01AUvLoXsKdS5sdpYju6rj4p * test(server): give the walker an independent oracle, not one that shares its code (BUG-2803) Codex round 22, finding 3. TestBodyDecodesNULGateAgreesWithAnUngatedWalk compares the gated function against an "ungated" reference that calls the SAME production valueDecodesNUL. That is valid for what the test claims - it pins the raw-prefix GATE - but it structurally cannot see a defect in the WALKER, because such a defect is present identically on both sides and cancels. That matters here specifically: every walker defect this unit has had lived in traversal, descent, or key matching (rounds 1, 2, 4, 16, 17), which is exactly the part the differential cannot check. Added a second implementation of the contract, written in the test and deliberately not calling the production walker. It shares encoding/json and jsonEncodedFieldKeys; it does NOT share traversal, descent, or key-matching. It is iterative with an explicit stack rather than recursive, so a recursion-shaped bug cannot reproduce in it by accident. Demonstrated rather than argued. With the nested-document descent removed from the production walker - a mutant that reopens the exact door this unit exists to close, and which compiles: differential (gate vs ungated) ok <- blind, as the finding said independent oracle FAIL <- catches it The corpus is also asserted to contain BOTH answers, since two walkers that always answer false agree perfectly. Claude-Session: https://claude.ai/code/session_01AUvLoXsKdS5sdpYju6rj4p * test(server): make the body-reader inventory type-aware, and state what it still cannot see (BUG-2803) Codex round 22, finding 4. The inventory that claims every request-body reader is accounted for was lexical, and wrong in three ways - all in the direction that matters for a test whose job is to say nothing is invisible: - it recognised only the variable names r and req, so a handler holding its request as httpReq or orig was INVISIBLE; - it matched inside COMMENTS, so prose could make a file look scanned; - the manually-listed traits field was already evidence of the model-regex blind spot. My first fix broadened the pattern to any identifier. That was worse, and worth recording: it matched every unrelated .Body field - input.Body in comments, fetched.Body in url import, comment.Body, art.Body, sidecarErr.Body - flagging five files that read no request body at all. The only route to green would have been listing those five as accounted, and an accounting entry HIDES future readers in its file. A false entry is worse than a missing one, so I abandoned that approach rather than tuning the regex. Now keyed on the TYPE via go/ast: collect identifiers declared *http.Request in a function signature, then find reader selectors on exactly those identifiers. Names stop mattering, comments are not in the AST, and .Body on anything else is not a match. Positive control, run rather than argued: a handler taking httpReq *http.Request and reading httpReq.Body is FLAGGED by the new scan, and matched zero times by the old regex. Two limits now stated in the test, because an unqualified completeness claim is exactly how the MCP audit reader got certified safe while persisting a decoded NUL: accounting is per FILE rather than per call site, and only signature-declared requests are seen - one stashed in a struct field or captured by a closure is not a parameter. Claude-Session: https://claude.ai/code/session_01AUvLoXsKdS5sdpYju6rj4p * fix(server): mark a cleaned audit identity, and repair two vacuous tests of my own (BUG-2803) Codex round 23, plus a defect in my own instruments that the mutation matrix found and the tests hid. ## Cleaning is lossy, so a cleaned identity was forgeable Round 22 closed the coarse version: sanitising before classifying let "tools/<NUL>call" become a genuine tools/call. Classifying on the raw method fixed that. But sanitising still COLLAPSES distinct inputs onto one output, so "pad_<NUL>item" stored exactly what "pad_item" stores - same tool_name, same args_hash - and anyone able to send a request could mint an audit row and a Prometheus label attributed to a real call. Cleaning and identity are different jobs. sanitiseStoredTextChanged now reports whether anything was removed, and an identity that only became well-formed by cleaning is marked. The cleaned text is kept, so the row stays diagnosable; the marker keeps it distinguishable. Descriptive text (User-Agent) keeps the unmarked helper - nothing decides anything on it. The parenthesised form is what this file already uses for a synthesised value, and a real method or tool name does not begin with "(", so the marker cannot itself be forged by choosing a clever name. ## Two of my own tests were vacuous, found by a surviving mutant I wrote nul := "\u0000" in the round-21 and round-23 tests, which in Go is the NUL CHARACTER, not the six-character escape text. Those bodies were malformed JSON that encoding/json rejected, so neither test ever reached the path it named. The comment on the line said "the escape, not the character"; the code did the opposite, and the correct form was already three lines away in the round-20 test. Nothing in the test output showed this. It surfaced only because the marker mutation SURVIVED, and because a surviving mutant was treated as a question - does the test not discriminate, or did it not run - rather than as either answer. Both repaired and both now kill their mutants: removing the marker fails with tool_name="pad_item" and a matching 64-character hash; removing the emptiness guard fails with "(sanitised) " instead of "(unknown)". Correction for the record: the round-21 checkpoint said that fix was measured failing before the fix. That measurement used the broken literal. The finding was real and the fix is right, but it is only properly established as of this commit. Claude-Session: https://claude.ai/code/session_01AUvLoXsKdS5sdpYju6rj4p * test(server): use the canonical escNULLiteral helper, not a local literal (BUG-2803) The helper is assembled from bytes precisely so this escape cannot decay into the NUL character it describes, and its comment says so: written as a Go literal it is one backslash away from being the NUL itself. I rolled a local one in three tests anyway, and two of them decayed exactly as that comment predicted - vacuous until the mutation matrix caught them. The safeguard existed, was documented, and I walked past it; using it is the only version of this fix that cannot recur. Claude-Session: https://claude.ai/code/session_01AUvLoXsKdS5sdpYju6rj4p * fix(metrics): bound the cleaned-identity marker as a metric label, and correct a false cardinality claim (BUG-2803, BUG-2817) Codex round 24, which enumerated CONSUMERS of the values this unit changed rather than asking again whether the guard is right. Most of that enumeration came back FINE, which is the useful half; three findings did not. ## The marker must not reach Prometheus as part of a name The cleaned-identity marker is right for the audit ROW - an operator reading one row needs to know which tool it resembles. It is wrong for a metric SERIES: "(sanitised) pad_item" and "pad_item" would be two series per user and per status, for a distinction no aggregate query asks. metricsToolLabel collapses the marked form to the bare marker, so it costs exactly ONE extra label value in total and that value is a constant rather than anything a caller supplies. Two tests, and the second exists because the first is not enough. The direct-call test proves the collapse function collapses. The WIRING test proves the emit path calls it - CONVE-19, my own convention. Measured: with the call removed from recordMCPCallMetrics, the direct-call test stays green and the wiring test fails naming the leaked label. ## A cardinality claim that was never true internal/metrics documented the tool label as "bounded by the catalog (~7 tools today)" with arithmetic resting on that. The value is whatever the caller put in params.name, recorded even for requests that dispatch later rejects, so an authenticated caller can mint a series per request. The comment now says so and points at BUG-2817, filed with the fix shape and the two wrinkles it has to decide - the catalog lives in internal/mcp, and legitimate JSON-RPC methods are not catalog tools. That unboundedness is PRE-EXISTING and not this unit's to fix; bounding the marker's own contribution is, which is why the collapse is here and the rest is filed. Also corrected: I wrote BUG-2815 into two comments before filing, and the filing came back BUG-2817. Predicting an identifier is the same class of claim as predicting a count. Claude-Session: https://claude.ai/code/session_01AUvLoXsKdS5sdpYju6rj4p * fix: keep a storable extension in the filename fallback, and sync the rename draft (BUG-2803) Codex round 24, the two remaining consumer findings. Both trace to this unit, and both are cases where a value was made SAFE without asking what reads it. ## The filename fallback was lossier than its sibling An unstorable upload name became a bare "upload" - no extension - while the empty-name fallback two lines below has always produced "upload.bin". The unusable part of "sh<NUL>ot.png" is the STEM; ".png" is ordinary text, and it is what consumers dispatch on: Content-Disposition, the web download anchor, bundle export naming, and , whose documented contract is handing a path to something that opens files by extension. That command was measurably affected - it treats any non-empty stored name as authoritative, so its MIME-based extension fallback never ran and the temp file was extensionless. Fixed at the source: a storable extension survives the fallback, bounded to 16 bytes so a hostile name cannot smuggle a long tail through. The CLI keeps a defensive extension fallback for any extensionless stored name, which also covers rows written before this. Both directions are tested: "sh<NUL>ot.png" now stores "upload.png", and "shot.p<NUL>ng" - where the EXTENSION is the unusable part - still stores bare "upload". Without the second leg, "keep the extension" could quietly become "keep whatever trails the last dot" and reintroduce the value the fallback exists to remove. Dropping the extension again fails the first leg. ## A rename that could never come clean saveName replaced the app object but never updated the draft, so when the server normalised the name the draft stayed as typed, the equality check never matched, Save stayed enabled, and each press re-sent the same request. The server caps at 120 BYTES via rune-safe truncation while the input allows 120 CHARACTERS, so any multibyte name near the limit diverges. The draft is now assigned the value the server actually STORED rather than compared for length, which stays correct for any future normalisation. svelte-check: 0 errors. Claude-Session: https://claude.ai/code/session_01AUvLoXsKdS5sdpYju6rj4p * fix: reserve the synthesised-value namespace, and stop a fallback carrying an unvetted extension (BUG-2803, BUG-2818, BUG-2819) Codex round 25, which probed whether the values this unit SYNTHESISES can themselves be attacked. Earlier rounds asked whether the guard refuses bad input; this asked what the substitutes are worth. ## A fallback must not carry an extension the product would refuse Preserving a storable extension was right; bindableText was the wrong bar for it. Control characters are valid UTF-8 and not NUL, so they are storable - and they are STRIPPED when the name is written into Content-Disposition. So ".s<VT>vg" passes the extension blocklist, which sees no known extension, and reaches the client as ".svg". attachments.SafeFallbackExtension now requires a KNOWN, ALLOWED extension, so a synthesised name can only carry a suffix the product already accepts on the ordinary path. Tested both ways: an obfuscated .svg and an unknown .foo are both dropped to bare "upload", while .png still survives. That divergence is PRE-EXISTING on the ordinary path, where the caller's name is stored as given and no fallback is involved - filed as BUG-2818 with the fix shape. This change only declines to add a second door. ## A mutation exposed a guard that could not fire I first wrote an explicit alphanumeric loop in that predicate as well. Removing it changed nothing: no key in extMIMEMap contains a non-alphanumeric character, so the map lookup already excluded every obfuscated suffix. Keeping an unreachable guard whose comment claims it stops control characters would have misdescribed which line does the work - so the loop is gone, and TestExtMIMEMapKeysArePlain enforces the property it was relying on. A guard that survives its own mutation is a question, not a clearance. ## The marker was forgeable, so the namespace is reserved Marking only what cleaning changed was not enough. A caller may name a tool "(unknown)" - what the parser returns for a malformed body - or "(sanitised) pad_item", and a genuine request then records the same identity as a substituted one. The older sentinels always had this; the new marker inherited it. A leading "(" is now reserved for values this server synthesises, and any caller value entering that namespace is marked too, so the two never collide. Cost stated: an MCP tool genuinely named with a leading "(" is recorded marked; tool names are identifiers in every catalog this server knows. The principled fix for the whole class is a provenance FIELD rather than sentinel strings in a caller-controlled namespace. That is BUG-2819 - it is a migration on two tables, and the same trick cannot rescue attachment filenames, which are legitimately named with parentheses. Claude-Session: https://claude.ai/code/session_01AUvLoXsKdS5sdpYju6rj4p * test(server): fix the independent oracle, which was wrong in a branch its corpus omitted (BUG-2803) Codex round 26, finding 5, and it lands on the instrument I introduced two rounds ago to check the walker. The oracle descended into a listed key's JSON document whenever it met one - including when that key appeared INSIDE a natural object that was itself under a listed key. Production does not: a natural object or array under a listed key is USER DATA, because the server marshals it and nothing re-parses it, so a listed key appearing inside it is an ordinary field name rather than a document marker. Measured on {"fields":{"schema":"<escape text>"}}: production=false, oracle=true. Production is RIGHT and the oracle was wrong, so had that body been in the corpus the test would have failed and pointed at the production walker. It was not in the corpus. That is the part worth keeping: the test already asserted its corpus was not one-sided - that BOTH answers appear - and that check passed while a whole branch of the contract went unexercised. Both answers appearing is not the same property as every branch being covered, and I had treated it as though it were. Fixed by giving the oracle the same user-data rule, and both bodies are now in the corpus - the natural-object case that must answer false, and its string-valued counterpart that must answer true. Re-verified that the correction did not blunt it: with the production nested descent removed, the oracle still fails. Claude-Session: https://claude.ai/code/session_01AUvLoXsKdS5sdpYju6rj4p * fix: refuse path-component filenames, and single-source the MIME extension table (BUG-2803) Codex round 26, findings 2 and 3. ## ".." is not a filename, it is a path component The server guard listed "", "." and "/" but not "..", which survives bindableText. filepath.Ext("..") is "." - non-empty - so an extension check waves it through too, and a consumer joining it onto a directory gets that directory's PARENT. The CLI builds its temp path exactly that way. Both ends fixed, deliberately independently. The server now rejects any name that is only dots or carries a separator, checked on the trimmed form so "..." and "./" do not each need a case. The CLI sanitises the name it receives regardless: a client that builds a local path out of a remote string should not depend on the remote end having sanitised it, and this CLI talks to whatever instance it is pointed at. Tested with "..", "...", "./" and "a/b", with an ordinary name as the premise leg. Restoring the old narrow guard fails it. ## Two tables for one relationship The CLI kept its own MIME-to-extension table and it had drifted: images and video but not gzip, tar, XML, YAML, TOML, HTML, JavaScript or several documents the server has always allowed. So the extension fallback added in round 24 silently did nothing for exactly the types whose viewers most depend on it. The CLI now delegates to attachments.ExtensionForMIME, and the second table is gone. Measured after: gzip .gz, tar .tar, html .html, js .js, pdf .pdf. The reverse map needs one choice per type where several extensions share one, and those preferences are asserted to name types the forward map actually uses - because the first version listed "text/yaml", which this map does not use (it says application/yaml), so that preference could never fire. Same class as the alphanumeric guard removed in the previous commit, caught the same way. Also recorded against myself: I destroyed both new functions mid-edit by running git checkout on a file with uncommitted work, to "revert an approach". That is a documented trap I have hit before and had written down. The committed function survived; the uncommitted ones did not. Claude-Session: https://claude.ai/code/session_01AUvLoXsKdS5sdpYju6rj4p * test(server): make the body-reader scan scope-aware, wrong in both directions before (BUG-2803) Codex round 26, finding 4. The scan used ONE flat name-set per top-level function, which is wrong in both directions at once: - a function literal inside a handler was scanned with the OUTER function's request names, so an unrelated inner variable that happened to be called r was FALSELY flagged; - a request arriving only as a function literal's own parameter was INVISIBLE, because literals were never given names of their own. A false flag in this test is not harmless. The only way to green is to add the file to the accounted list, and an accounting entry HIDES every future reader in that file - so a false positive here converts directly into a blind spot later. That is the same trap that made me abandon the broadened regex two commits ago. Now walks a SCOPE at a time. Each scope inherits its parent's request names, drops any it shadows with a parameter of a different type, and adds its own. Local aliases (req := r) are picked up as well, since that is an ordinary thing for a handler to do and the alias reads the same body. Three controls, run rather than argued: closure parameter reader -> FLAGGED local alias reader -> FLAGGED shadowed inner variable -> not flagged The first two were invisible to the previous scanner, which never gave literals their own names, and the third is the false positive it produced - both by reading the code this replaces. Limits restated honestly rather than left as they were, since two of them are now closed. Still invisible: a request in a struct field, one from a context, and one whose type reaches http.Request through an alias or embedded field. This matches the literal spelling rather than resolving types; closing those means the type checker, not the parser. Claude-Session: https://claude.ai/code/session_01AUvLoXsKdS5sdpYju6rj4p * fix: stop the reverse MIME map emitting BLOCKED extensions, and close four instrument gaps (BUG-2803) Codex round 27 returned "do not merge yet" with three P1s. All of them are mine, from the previous two commits. ## The reverse map turned a refusal list into a source of extensions extMIMEMap is the FORWARD table used to REFUSE uploads - it deliberately lists .svg, .exe, .com so those extensions can be recognised and rejected. Reversing it wholesale meant ExtensionForMIME("image/svg+xml") answered ".svg", where the old CLI table answered nothing, and names a local file with that. So I closed an SVG door two commits ago and reopened one through the MIME helper. Blocked types now get no reverse mapping at all, and the test asserts it with a premise leg (the map must CONTAIN a blocked type, or the assertion never runs). Removing the exclusion fails naming .svg, .com and .msi. ## The oracle was closer, not identical Production descends only into a JSON DOCUMENT - a string whose trimmed form starts with { or [. The oracle unmarshalled any valid JSON, so a SCALAR under a listed key made it answer true where production answers false. Closer to production is not a usable oracle; only identical is. Aligned, and the scalar case is in the corpus. ## The scan was still not scope-aware, and could now MISS a reader A nested block shared the enclosing name-set, so { r := &http.Response{}; r.Body.Read(nil) } was FALSELY flagged. And the shadowing rule deleted a name rebound to http.Request BY VALUE - which still shares the Body, since it is an interface holding the same reader - so that read became invisible. Blocks are now their own scope and a value request counts. ## The controls I claimed were not in the suite Round 27 was right: I had run them as throwaway probes and deleted them, so nothing held the scanner to them. The scanner is now a package-level helper and TestBodyReaderScanDiscriminates drives it over ten synthetic files - six that must be detected (plain, unconventional name, closure parameter, alias, value copy, form reader) and four that must not (no request, shadowed by a closure parameter, rebound in a nested block, mentioned only in a comment). ## And an over-refusal of my own making The filename guard rejected any dot-only name and anything containing a separator. Only "." and ".." are path components; "..." is an ordinary POSIX filename, and filepath.Base has already reduced "a/b" to "b", so the separator test was dead on this platform and removed rather than left looking load-bearing. Preservation controls now pin that legitimate names survive. Also removed: an unused id parameter on safeLocalFilename. Claude-Session: https://claude.ai/code/session_01AUvLoXsKdS5sdpYju6rj4p * test(server): only a DEFINE can rebind a name, and cover the idiomatic reassignment (BUG-2803) Found by probing my own previous commit rather than by a review round - the first time in this sequence I have caught the adjacent breakage before the next round did. The scope rules deleted a request name on ANY assignment whose right side was not a request identifier. That is wrong in the dangerous direction, and it fires on the most idiomatic line in Go HTTP code: r = r.WithContext(ctx) io.ReadAll(r.Body) // <- invisible to the scan WithContext is a call, so the name was dropped and every later read went unseen. Measured before the fix: MISSED. The correct rule is type-sound. Go is statically typed, so a plain cannot change a variable's type: if it held a request before, it holds one after. Only a DEFINE introduces a new binding that can be something else. So the delete is now gated on token.DEFINE, which is both more correct and simpler than what it replaces. Three controls added, and the two that would have caught this are the ones I had not written: a WithContext reassignment, and readers inside an if body and a for body - the last two because making every nested block its own scope is exactly the kind of change that could have started missing them. Thirteen controls now, six negative. Reverting to delete-on-any-assignment compiles and fails the WithContext leg. Claude-Session: https://claude.ai/code/session_01AUvLoXsKdS5sdpYju6rj4p * fix: make the body-reader scan conservative by design, and reduce filenames cross-platform (BUG-2803) Codex round 28. Two findings, and the first is the fifth consecutive round to find a FALSE NEGATIVE in the same instrument. ## Stop modelling scopes; change the error direction instead Rounds 24 through 28 each found another way the scope-modelling scan missed a real reader: a value-copied request, a plain r = r.WithContext(ctx), a mixed r, ok := ... that reuses an existing variable, and if/for/switch initialisers and case clauses whose scopes it did not model. Each fix closed one case and left another. That is a design telling me something, not a run of bad luck. The two error directions are not symmetric here. A false NEGATIVE hides a body reader, which is the entire thing this test exists to prevent. A false POSITIVE costs one human review and an accounting entry with a reason attached. So the scanner now OVER-APPROXIMATES on purpose: any name bound to an http.Request anywhere in the file counts for the whole file, aliases are followed to a fixed point, and names are never un-bound. Every scope-shaped false negative becomes structurally impossible. The cost is real and is now asserted rather than discovered: two controls that previously expected "not flagged" - a name shadowed by a closure parameter, and one rebound in a nested block - now assert CONSERVATIVELY FLAGGED, so the bias is on the record. Three of round 28's named misses are added as controls and pass: mixed short declaration, switch case, if-initialiser shadow. Sixteen controls, and the accounting test still passes against the real package - so the over-approximation costs nothing today. Exactness needs go/types with a real package load, which is a bigger instrument than this test warrants. The comment says so, and names the signal that would justify building it: an accounted entry whose reason is "the scan over-flagged". ## A filename safe on this OS is not safe on the consumer's filepath.Base is platform-specific, so on Unix it leaves a backslash alone - and the stored name is consumed cross-platform. A Windows client joining a stored "..\evil.png" onto a directory traverses upward. Reduced to the leaf under BOTH separator conventions. This normalises rather than refuses, which is less lossy than replacing the whole name and keeps round 27's point that a backslash is legitimate on Unix. Removing the reduction compiles and fails the new test. Claude-Session: https://claude.ai/code/session_01AUvLoXsKdS5sdpYju6rj4p * test(server): count readers per accounted file, add two missing reader methods, correct a false reason (BUG-2803) Codex round 29, and its central point was aimed at my REASONING, not my code. It was right. ## The over-approximation argument was wrong for per-file accounting I justified a deliberately conservative scanner by saying a false positive costs one review and one accounting entry. That is not what it costs. Once a file is listed, a NEW reader added to it is covered by the existing entry and the test stays green - so a false positive does not cost a review, it permanently blinds the list for that file. My own comment already recorded that hazard two commits earlier, and I argued past it anyway. The fix is to make the entry carry a COUNT of reader expressions rather than a yes/no. Adding a reader to an accounted file now changes the number and fails, so the entry must be re-read and its reason re-justified. It churns exactly when a body reader is added or removed, which is when a human should look. Demonstrated: inserting r.PostFormValue into handlers_tokens.go - an already-accounted file - is FLAGGED. Before this it was absorbed silently. The conservative bias stays, because the false-negative classes it eliminates are real and the count now removes the reason it was expensive. ## Two real reader methods were missing MultipartReader STREAMS the body and FormFile triggers multipart parsing of it. Neither was in the selector list, and FormFile is used in production in handlers_attachments.go - so the list was incomplete against code that exists, not hypothetically. ## A reason in the list was simply false handlers_tokens.go was accounted as "a nil/ContentLength check only - it never reads the body". It guards on those and then calls decodeJSON. A wrong reason is the same defect as a missing entry: both let a reader pass as reviewed. ## And I guessed the counts I wrote plausible numbers for the per-file counts and every one was wrong; the test reported the real ones on its first run. Same habit this branch keeps catching - a figure written from expectation reads exactly like a figure that was counted. They are measured now and the comment says so. Also recorded: my first verification script failed to apply its mutation and still printed a verdict, which I nearly banked. It now aborts unless the mutation is present in the file. Claude-Session: https://claude.ai/code/session_01AUvLoXsKdS5sdpYju6rj4p * fix(attachments,cli): close the closing round's two view defects, correct two overclaiming comments (BUG-2803) The closing enumeration (successor seat, per the lead's convergence ruling) returned two real attachment-view defects and two comments claiming more than their code delivers. Fixed here; the round's two design-scale findings are filed instead (BUG-2820 scanner precision via go/types, BUG-2822 Windows-unstorable filename forms). - Reverse MIME map: four ALLOWED spellings (text/xml, text/yaml, application/javascript, audio/webm) had no reverse extension because no extMIMEMap entry uses them as its value — `pad attachment view` wrote an extensionless temp file for exactly the types the delegation was built to fix. Population measured against the whole allowlist: these four, no more. An alias table closes them; TestEveryAllowedMIMEHasAnExtension asserts the class property over the allowlist (a future allowlist entry with no reverse extension fails), plus alias hygiene (allowed keys only, no forward-derived collisions, alias extensions must map to ALLOWED types so the table can never mint a refused extension). Mutation-verified: removing the alias application fails the test on all four types. - safeLocalFilename: a trailing dot survived every check and filepath.Ext("photo.") is "." — non-empty — so the MIME-extension fallback never fired and the temp file dispatched on no extension. Trailing dots are now stripped (cannot empty the name; dots-only names already returned early). The CLI guard also gains its first direct tests, including the backslash and traversal refusals that previously rode untested. Mutation-verified: removing the TrimRight fails both trailing-dot cases. - Two comment corrections, same defect class the accounting list itself names (a wrong reason reads as review): the handlers_cloud.go entry said bodyHasCloudSecret "restores" the body — it restores the first 64 KiB and drops the tail, a bound that file documents; and the accounting test's header said its scan "cannot be spelled around" while its own KNOWN LIMITS block lists the spellings that get around it (struct field, context value, type alias). The header now matches the limits block. * fix(server,attachments,cli): close closing-round-2's scanner blind spot and four stale comments (BUG-2803) Closing round 2 (successor seat) found no product defects; all four findings were in instruments and comments. Each verified against the code, then fixed: - The alias fixed-point resolved only identifier RHS (`req := r`), so a dereferenced copy (`c := *r; io.ReadAll(c.Body)`) was an invisible body reader — and unlike the disclosed type-level classes, this one was not in the KNOWN LIMITS block. The copy shares the Body (an interface holding the same reader). StarExpr operands now join the alias set; a new control pins the case. Mutation-verified: reverting the StarExpr handling fails the control. The type-level classes (struct field, context value, type alias) remain disclosed and are BUG-2820's territory. - The KNOWN LIMITS block said shadowed request names are "correctly ignored" while the controls deliberately assert they are conservatively OVER-FLAGGED — stale prose from the scope-aware era, falsified by the round-28 conservative flip that never touched those lines. - The reverse-map stability loop compared only PREFERRED entries across rebuilds; it now compares the entire map (sizes and every mapping) against the first build. Boundary stated honestly: every multi-spelling type today is preference-pinned, so the full-map comparison discriminates only when a future non-preferred multi-spelling entry appears — that future entry is what it guards. - Three orphaned/wrong comments: a `mimeForExt` doc block glued above ExtensionForMIME (the function it described is gone); the old hardcoded extension-table doc glued above safeLocalFilename (falsified by the delegation it predates); and two "120 chars" claims where the cap is 120 BYTES rune-safe via truncateBindableText — the consent form's maxlength=120 counts characters, so a multibyte name passes the client and is still truncated server-side, which is now what the comments say. * fix(cli,server,docs): close the attachment-view path escape, and closing-round-3's instrument and prose findings (BUG-2803) Closing round 3 found the branch's first product defect since round 17, in BRANCH-ADJACENT code the round-24 fallback extension work made reachable: the `pad attachment view` id fallback joined the RAW id onto its temp dir, and the client sent the id into the URL path UNESCAPED. An id is a CLI argument, but the documented agent flow harvests it from item content ("pad-attachment:" refs other workspace members write), so a traversal-shaped "id" could re-route the HEAD/GET to a different endpoint whose 200 then vouched for it, and the write escaped the temp dir. Both halves fixed and both mutation-verified through a new command-level test: reverting the fallback sanitize demonstrably wrote OUTSIDE the sandboxed TMPDIR; reverting the PathEscape put a raw "../../" on the recorded wire. - internal/cli: url.PathEscape(attachmentID) at both id-bearing client sites (HeadAttachment, DownloadAttachment — the enumerated population). - cmd/pad: the id fallback runs through safeLocalFilename, generic "attachment" when nothing survives; view's long help no longer claims the filename is used "without rewriting the extension" — it describes the reduction and the MIME-extension append, and says why the CLI is stricter than the server (the name is written to YOUR filesystem). - cmd/pad: attachmentViewCmd gets its first command-level test (CONVE-19 — the helper tests vouched for the component, not its wiring): disposition name, extensionless+MIME append, id fallback, traversal containment with a wire-escaping control, generic fallback. Instrument and prose findings, each verified before fixing: - The KNOWN LIMITS disclosure now names the ordinary alias forms the fixed-point does not walk (var-spec, call-derived, named results, range bindings) — they were in BUG-2820's filing but not in the in-file disclosure, which is what let the round read them as unfiled. The scanner itself deliberately does NOT grow another parser patch; go/types is the filed fix. - TestTextSafeHelpersAreUsedAtEveryCallSite pins EXACT occurrence counts (measured: 1 declaration + 4 call sites each) instead of a >=4 floor a removed call site could hide under. - middleware_mcp_audit: two stacked comment copies rested non-forgeability on "real names do not begin with (" — the exact reasoning round 25 retired; the const doc now points at auditLabel's namespace-reservation rule, which is what actually makes the marker non-forgeable. - docs/backup.md said repair is needed before "the export or migration" goes through, contradicting its own "exports fine" three paragraphs up — it is the IMPORT or migration that fails; the export succeeds either way. * fix(server,cli): decode chunked watch bodies, refuse dot-segment attachment ids, correct two texts (BUG-2803) Closing round 4 found one PRE-EXISTING product defect and one residue of the round-3 fix, plus two wrong texts. Each verified before fixing: - Watch creation gated its body decode on `ContentLength > 0`, so a CHUNKED request (ContentLength == -1) had its body silently DROPPED — the caller's predicate ignored, an unconditional watch created, 200 returned. The population of ContentLength gates in the package is exactly two: handlers_tokens.go already used the `!= 0` form, watches now matches it, with io.EOF tolerated so the documented no-body-is-valid contract holds for an empty chunked body too. Three handler-level tests discriminate the cases; the mutation (condition back to `> 0`) fails the two it should and passes the empty-body control. The accounting instrument then flagged the new `r.Body != nil` reference in the file — its exact job — and the file is now accounted with a measured reader count of 1. - url.PathEscape leaves exact "." and ".." UNCHANGED, so those two ids still reached the wire as live dot segments for a proxy or server to normalize — the escaping added in round 3 did not cover them. Both id-bearing client sites now share attachmentIDPathSegment, which refuses exactly those two values before any request (a real id is a UUID; the refusal cannot fire on one). Mutation-verified: removing the refusal fails the new subtest, which also asserts zero requests reach a recording stub. - The artifact rejection text said "NUL byte"; the same refusal fires for a NUL manufactured by a YAML escape during parsing, where no raw NUL byte exists — now "NUL character", in the handler message and the error var. - A test comment claimed the User-Agent reaches sessions.user_agent as text; sessions store only ua_hash, as the accounting list's own exemption states two hundred lines up. The sentence now agrees with it. * fix(attachments,server): remove a can't-fire MIME preference and a stale filename-guard sentence (BUG-2803) Closing round 5 is down to two P3 comment defects; both verified and fixed: - preferredExtensions "preferred" .md over a .markdown that has never been in the forward map — a line that cannot fire, the exact class this branch's own instruments hunt (the alphanumeric guard, the text/yaml preference, the charset loop). Entry removed; shortest-wins picks .md as the only candidate, unchanged. The preference-hygiene test now asserts every entry has a real competitor (>= 2 forward-map spellings), and the counterfactual — re-adding the entry — fails it. - The upload filename guard still carried round 26's "checking the trimmed form rather than listing spellings" sentence directly above round 27's code that does the opposite (exact "." / ".." comparisons, longer dot runs deliberately preserved). The stale layer is gone; the surviving paragraph already records why. * fix(server,docs): drop a dead test fixture, stop claiming the failing row is named (BUG-2803) Closing round 6 returned one P3 — TestDecodeJSONTrimsOnlyJSONWhitespace booted a full testServer it never used (`_ = srv`), dressing a direct decodeJSON test in router coverage it does not have. Removed. Its enumeration also re-read docs/backup.md against the code: "the failing row is named in the error" is true of neither leg — the import answers 400 naming the RULE it refused on (the NUL check is body-wide and knows no row), and `pad db migrate-to-pg` reports which WORKSPACE's copy failed. The doc now says exactly that, and that locating the value is manual until BUG-2810's preflight lands. * fix(server,cli): retire a stale byte-search claim, close two instrument gaps from closing round 7 (BUG-2803) Round 7 found no production defects; three instrument/comment findings: - artifactIsBindableText's doc comment still asserted the round-8 byte-search approach and that "the ambiguity cannot arise here" — directly above the round-9 body comment recording that assertion as simply wrong and doing the round-trip walk instead. The doc paragraph now describes the round trip and points at the body's history. - TestBodyReaderScanDiscriminates listed MultipartReader and FormFile in the scanner's selector set but had no control for either, so their removal from that list was undetectable. Two controls added. - The attachment-view test proved nothing about the MIME delegation: every case used image/png, which the OLD hand-rolled table also knew, so a stale local table passed. Two cases added — application/gzip (a type round 26 found missing from that table) must gain .gz, and blocked image/svg+xml must gain nothing. Mutation-verified: a stale-table mutant that answers only for png fails the gzip case. (First mutant attempt didn't build — unused import — and was not counted as a detection.) The per-file same-count substitution gap round 7 restated is declined as filed, not fixed: BUG-2820's filing already specifies per-call-site accounting via go/types as the fix that retires the per-file count workaround; the KNOWN LIMITS closing line now carries that ref. * fix(attachments,server): sweep two pre-BUG-2413 disposition comments, pin the manifest refusal to 400 (BUG-2803) Closing round 8 found no production defects; two evidence findings, verified then fixed: - Two comments still described the PRE-BUG-2413 disposition policy: the RenderChip mode doc said the HTTP layer serves every chip inline, and the read-path doc derived Content-Disposition from RenderMode. The live policy is the explicit fail-closed ServeInline allowlist — most chip types are served as "attachment". Both now say so and record the history. - TestImportBundle_RefusesNULInManifest accepted any status >= 400, so the documented 400 could decay into a 500 unnoticed. Pinned to http.StatusBadRequest.	2026-08-30 21:30:19 -04:00
xarmian	50499bceb8	fix(server): charge workspace imports against the plan limit (BUG-2793) (#1219) ... * fix(server): charge workspace imports against the plan limit (BUG-2793) `POST /workspaces` enforces the user-scoped `workspaces` plan limit before creating. `POST /workspaces/import` did not, and it mints a workspace through the same store.CreateWorkspace — so a user at their plan's limit could exceed it by exporting any workspace and importing it back. Cloud only; enforceUserPlanLimit is a no-op when cloudMode is off, so self-hosted was never affected. Dave ruled the shape on day 63: an import IS a new workspace and counts, with no exemption for re-importing something you previously owned. Export provenance is not trustworthy enough to gate billing on, and the at-limit case that deserves relief — undoing a delete — is served by the restore endpoint, which mints nothing. The call is one line; the PLACEMENT is the fix. It sits beside the #1212 consent gate, ABOVE the Content-Type dispatch, for two reasons: - handleImportWorkspaceBundle is reachable only through that dispatch, so a gate below it would cover the JSON path and leave the tar.gz path — the one that carries attachments, and the one a real export produces — wide open. - Above either body read, so a refused caller never uploads. The two paths have very different size bounds; the gate precedes both. That is not a hypothetical: the mutation matrix includes it. Moving the gate below the dispatch fails ONLY the bundle test and leaves the JSON test green, which is exactly the false confidence a placement-blind fix would have shipped. Five tests, four of them controls, because a gate is easy to get green and hard to get right: the JSON path refuses at the limit, the bundle path refuses at the limit, under-the-limit is NOT refused (a gate wired to the wrong feature key would pass the first two), self-hosted is unaffected (this must not introduce a limit where there are no plans), and a request with no resolved user is not charged — mirroring the create side's `userID != ""` guard, which is not defensive padding but the difference between "no limit applies" and a nil lookup. This is the SECOND gate on workspace creation the import door skipped; the first was the OAuth consent gate (IDEA-2756, PR #1212). Two have now diverged this way, which is the argument for the shared pre-step helper — tracked separately rather than folded in here. Gates: `go test ./...` under Postgres 17 EXIT=0; gofmt clean; `make lint` 0 issues. BUG-2793 Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN * test(server): make the import controls assert success, not merely non-refusal (BUG-2793) Codex round 1. The three control tests were vacuous and I would have shipped them. WorkspaceExport.Version defaults to 0 and the import requires 1, so the under-the-limit, self-hosted, and no-resolved-user cases were all failing with a 500 long before they reached anything this change is about. They passed because they asserted only "not 403" — and a 500 is not a 403. That made all three useless in the same direction: a fix that broke imports outright, or a gate wired to refuse everything with a non-403 status, would have sailed through them while the two refusal tests stayed green. The controls existed precisely to catch that, and could not. Fixed by setting Version: 1 and asserting the real success status, 201. A control that cannot tell success from a server error controls nothing. Mutation matrix re-run after the change, because a matrix over vacuous tests proves nothing either: removing the gate still fails exactly the two refusal tests, and the three controls now pass on genuine imports rather than on identical 500s. Gates: `go test ./...` under Postgres 17 EXIT=0; gofmt clean; `make lint` 0 issues. BUG-2793 Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN * docs(server): scope what the no-resolved-user import test actually pins (BUG-2793) Codex round 2 pointed out that this test locks in a 201 for a caller with no resolved user, and that a reader will take that as approval of userless workspace creation. It is not. The test pins the GUARD — that import behaves as create does when no user resolves — and it drives the handler directly, so it does not prove a real legacy workspace token reaches this code at all. Said so in the test rather than leaving the 201 to speak for itself, and pointed at BUG-2809 for the question it does not answer. Round 2's three findings are all real and all filed rather than folded, because this unit's ruling is specifically the plan limit on the import door: - BUG-2808 — enforceUserPlanLimit is check-then-act, so concurrent requests can exceed any cap. A property of the helper, shared with the create door and every other feature it gates; this change inherits it rather than introducing it. - BUG-2809 — import and create still enforce different preconditions on the same mint: required-name validation (a live defect — an empty name yields an empty SLUG, which is a routing key), settings normalization, source attribution, and the userless case above. Filed as a class because the mechanism is one thing and the record now shows it failing twice. The reviewer also confirmed two non-doors, which is the useful negative: autoCreateWorkspace is intentional first-workspace provisioning, and `pad db migrate-to-pg` calls ImportWorkspace directly as an operator-only migration outside HTTP entirely. BUG-2793 Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN * test(server): pin the JSON-path placement and assert "not charged" as data (BUG-2793) Codex round 3, on the tests. Two ways they could pass without proving what their names say. 1. The JSON refusal test sends VALID json, so a gate placed after the decode would still return 403 and it would stay green. The bundle test covers the gzip half of the placement claim; nothing covered the JSON half, and the code comment claims the gate sits above EITHER body read. Added an at-limit case with an undecodable body: reaching 403 rather than a decode error is only possible if nothing read the body first. 2. The no-resolved-user test asserted only a 201. A regression that quietly attributed the import to the at-limit fixture user would also return 201 and pass. It now asserts the fact instead of inferring it — the user's workspace count is unchanged across the request, and the created workspace has no owner. The mutation matrix now separates the two placements, which is the point of having both tests: - gate below the Content-Type dispatch (still above the decode) -> only the BUNDLE test fails. - gate below the JSON decode -> the bundle test AND the new JSON test fail. Neither mutation is caught by the original refusal test, which is what "passes for the wrong reason" looked like here. Gates: `go test ./...` under Postgres 17 EXIT=0; gofmt clean; `make lint` 0 issues. BUG-2793 Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN	2026-08-27 23:56:55 -04:00
xarmian	427540706c	fix(documents): bound the rename cascade at the title and at the projected total (BUG-2798, BUG-2796) (#1218) ... * fix(documents): bound the rename cascade at the title and at the projected total (BUG-2798, BUG-2796) A document rename rewrites [[oldTitle]] into every linking document. Neither factor of the output size was bounded: titles had no length validation, and the cascade holds every rewritten body in memory before writing any of them. One rename could project 10 GB from a 500 KB input -- 20,000x, measured -- and OOM while holding the workspace rename lock. Two walls, per Dave's day-63 ruling. 1. Title length, bounded at write time (models.MaxDocumentTitleRunes = 255). Runes, not bytes: "255 characters" is what a user and a UI counter mean. Existing over-limit titles stay valid until their next rename -- no retro-breakage of stored data. 2. The cascade's projected TOTAL, bounded at 16 MiB (store.MaxRenameCascadeProjectedBytes), accumulated across the linking set and refused before the first rewrite is built. The total is the right quantity and a per-document cap would not have been. Measured, with the title bound already in place: one linker holding the largest body a 2 MiB request can carry projects 108,632,370 bytes -- 51.8x -- and the aggregate is linear in the number of linkers (108.6 / 217.3 / 434.5 MB at k = 1/2/4, allocation tracking output at ~1.02x). A per-document cap of C still admits k * C, which is the same unbounded shape one level up. The 16 MiB figure has a receipt in the constant's doc comment: it sits above the absolute ceiling of any cascade this development instance could produce (its entire wiki-linking corpus is 10,077,476 bytes) and 6.5x below the single-document attack. The refusal is permanent-shaped and deliberately NOT in ErrLinkCascadeContention's family: 413 with the projection in the message and no Retry-After. Contention means "someone got there first, try again"; this means "this rename cannot be performed as asked". Answering it from the retryable family would tell a client to retry forever. BUG-2796 folds in at the same validation point, as ruled -- a title containing wiki-link syntax is emitted raw by links.ReplaceTitle, so renaming to `A]] [[A` produced two broken links and reported success. The rule is derived from the two mechanisms that consume a stored bracket (the grammar at markdown.ts:327 and the unescaper at markdown.ts:753) rather than from a character blacklist: the first version of this fix banned `]`, `\` and `|` because all three "look like wiki-link syntax", and the round-trip test refuted two thirds of that. `|` in particular is a title shape resolveWikiBody contains a dedicated branch to support, and `[` passes the grammar untouched. Doors enumerated rather than assumed (CONVE-24): store.CreateDocument and UpdateDocument have exactly two callers between them, both HTTP handlers. No CLI, import, or seed path writes a document title. Update previously validated doc_type and status and NOT title -- the one field that drives the cascade -- so the handler tests drive real requests through both doors (CONVE-19). BUG-2798, BUG-2796 Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN * fix(documents): count retained bytes, bound the retry path, escape the cascade's LIKE pattern (BUG-2798) Codex round 1 on #1218. Three findings, all real, all fixed here. 1. The guard bounded projected OUTPUT, which bounds nothing when the new title is SHORTER than the old one. Renaming a 255-character title to a one-character title makes each 2 MiB linker project ~40 KiB while the cascade still retains its 2 MiB read for the compare-and-set, so hundreds of linkers exhaust memory while the counter reports well under the cap. The counter now sums RETAINED bytes — read plus written, both alive at once — so the cap is a statement about resident memory rather than about output. MaxRenameCascadeProjectedBytes becomes MaxRenameCascadeRetainedBytes and moves 16 -> 32 MiB, because the legitimate ceiling it clears doubles under the new metric (that instance's whole wiki-linking corpus retains ~20,154,952 bytes); the single-document attack retains 110,729,522, so it is still refused by 3.3x. 2. The compare-and-set's retry path bypassed the guard entirely. On contention it re-reads the linker and calls ReplaceTitle on whatever the winner wrote — a NEW input, bounded by nothing the scan had checked — so a content edit landing inside the cascade's window could grow a linker from harmless to enormous and walk the rename back into the amplification it would have been refused for. Each document's compare-and-set now carries the cap less what the other linkers hold, and re-checks the grown body against it. 3. The cascade's `content LIKE ?` search term went in unescaped, so a document TITLE decided how the pattern was read. `\` is the default LIKE escape character on Postgres and NOT on SQLite, so `[[Alpha\Beta]]` was searched for as itself on one dialect and as `[[AlphaBeta]]` on the other: linkers not found, cascade rewrites nothing, rename reports success, every link left stale. Silent and dialect-dependent. Codex named the backslash; `%` and `_` are the rest of the class (CONVE-18) — wildcards on both dialects, so a title carrying them selects documents that do not link it. An explicit `ESCAPE '\'` clause plus escapeLikePattern makes both dialects agree, rather than leaving SQLite correct by accident. Finding 3 also constrains finding 3 of the ORIGINAL fix: models' validator allows a lone backslash in a title on the grounds that both renderers handle it, which was true of rendering and false of cascading. That comment now records the dependency — allowing it is only correct while the cascade's pattern stays escaped. Tests, four new, each mutation-verified against the code it guards: - CountsRetainedBytesNotJustOutput — the shrinking rename. Asserts as a PRECONDITION that the projected-output total stays under the cap, so the test cannot pass for the old reason. - RetryRecheckesTheBudgetAgainstTheGrownBody — drives the real race through the afterLinkCascadeRead seam. POSTGRES ONLY and skipped loudly elsewhere: SQLite's BEGIN IMMEDIATE closes the window structurally, so a green run there would be a property of the DSN. - FindsLinkersWhoseTitleContainsABackslash — Postgres only, same reasoning inverted: SQLite is the dialect that was accidentally right. - DoesNotSpendTheBudgetOnDocumentsThatDoNotLinkTheTitle — `%` and `_`. Its first version asserted the decoy's content was untouched and passed against the unescaped pattern, because over-matched rows rewrite to themselves. The observable harm is that they spend the caller's budget, so that is what it now asserts. Mutation matrix for this round: output-only counter -> only the shrinking test fails; retry check removed -> only the retry test fails (PG); LIKE unescaped -> the budget legs fail on SQLite and the backslash test fails on PG. Gates: `go test ./...` under Postgres 17 EXIT=0; SQLite packages EXIT=0; gofmt clean; `make lint` 0 issues. BUG-2798, BUG-2796 Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN * fix(documents): tighten the retry budget, stop charging no-op rewrites, order the typed check first (BUG-2798) Codex round 3 on #1218, an edge-case angle over the new arithmetic and control flow. Three findings fixed, one declined. 1. The retry budget credited back this document's own share, on the reasoning that the retry replaces it. It does not: the original read and rewritten bodies stay reachable through `updates` while the write loop runs, so the re-read and its rewrite are allocated ON TOP of them. The bound could be exceeded by up to one document's share while the arithmetic still reported it satisfied. The budget is now the genuine headroom, `cap - retained`. 2. A concurrent edit that REMOVES the link left a body with no occurrences, which cascadeRetainedBytes still charged twice — once for the read and once for a rewritten copy that does not exist, because strings.Replace returns its input unchanged when there is nothing to replace. That could refuse an otherwise valid rename for memory the cascade never allocates. 3. The handler classified this error by PROSE before testing it by identity. The UNIQUE-constraint arm matches a substring, and the refusal error embeds the caller's title verbatim, so renaming a document to a title containing the words "UNIQUE constraint" came back as a 409 name collision — advice to pick a different name, for a rename that was refused for size and would fail identically under any name. Typed sentinel now tested first. DECLINED: unchecked int64 arithmetic in the projection. The multiplicands are derived from the length of a string already resident in memory, so overflowing int64 needs a single document body of roughly nine exabytes; and the accumulator returns as soon as it passes the cap, so it cannot run away either. Saturating arithmetic here would be guarding a state the machine cannot reach. Tests, three new, each mutation-verified: - RetryBudgetExcludesThisDocumentsOwnStrings — deliberately separate from the existing retry test, because that one catches the check being ABSENT and this one catches it being too GENEROUS. The grown body is sized to fall BETWEEN the two budgets; a body far over the cap cannot tell them apart. - ConcurrentEditThatRemovesTheLinkDoesNotRefuseTheRename — its first version sized the link-free body against the CAP rather than against the retry's real headroom, so the refusal it caught was correct behaviour and the test was wrong, not the code. Re-sized against the headroom: fits when charged once, does not when charged twice. - IsNotMisreportedAsATitleCollision — at the handler, since the defect is entirely in its classification order. Mutation matrix for this round: credit the share back -> only the tight-budget test fails; charge the no-op body twice -> only the link-removed test fails; order the substring arm first -> only the misclassification test fails. Gates: `go test ./...` under Postgres 17 EXIT=0; touched packages re-run after the lint fix EXIT=0; gofmt clean; `make lint` 0 issues. CI green on b09aca12. BUG-2798, BUG-2796 Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN * test(documents): close four ways the cascade-bound tests could pass for the wrong reason (BUG-2798) Codex round 4, aimed at the TESTS rather than the code. No production behaviour changes here; four instruments that were weaker than they read. 1. Every retained-byte case exceeded the cap under `max(read, rewritten)` as well as under `read + rewritten`, so none of them could tell the two arithmetics apart — and taking the larger would hold twice the cap. Added CountsBothStringsNotTheLargerOne: an ordinary same-length rename over content totalling ~60% of the cap, which the sum refuses and the max admits. Its preconditions assert both halves of that gap. 2. Nothing pinned the 255 itself. Every length case derived its inputs from MaxDocumentTitleRunes, so changing the constant to 512 left them all green. That is fine for arithmetic and wrong for this number: it is a product decision Dave ruled, and a silent change to it silently changes how much amplification the cheap door lets through. Deliberately NOT done for MaxRenameCascadeRetainedBytes, which is mine and carries a measured receipt that is expected to be re-measured. 3. The oversize tests asserted only THAT a rename is refused, never that it is refused BEFORE the amplified string is built — which is the entire point of the guard. Moving links.ReplaceTitle above it would have kept them green. Added RefusesBeforeBuildingTheRewrittenBody, measuring cumulative allocation with a ~20x margin: refusing costs the one body it had to scan, building first costs ~108 MB. Verified by mutation — with the guard moved after the rewrite it reports 110,748,144 bytes against a 52,428,800 ceiling. This filing warned that measuring memory to prove the ABSENCE of amplification is flaky by construction. That still holds for the shape it described, a peak-RSS floor. This is the opposite: a generous ceiling on a deterministic counter, with the two outcomes twenty times apart. 4. The "reports the projection" assertions checked for the words `maximum` and `bytes`, which a message saying "maximum bytes exceeded" would satisfy while telling a caller nothing. They now require the cap's actual value and a real byte count. Mutation matrix: charge only the larger string -> only CountsBothStrings fails; move the rewrite above the guard -> only RefusesBeforeBuilding fails. Seventeen mutations across four rounds, each detected by the test that should catch it. Gates: `go test ./...` under Postgres 17 EXIT=0; gofmt clean; `make lint` 0 issues. BUG-2798 Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN * fix(documents): compose the 413 from typed fields instead of splicing err.Error() (BUG-2798) Codex round 5, on the side effects of a REFUSED rename. Side effects were otherwise clean — rollback removes versions, link rewrites, attachment stamps and the title change, and no activity row, SSE event or webhook is emitted — but the response body was built by appending err.Error() to a public sentence. That published whatever any layer had wrapped around the error on its way up. Today that is "update links: store: ", which is a call path clients have no business seeing; tomorrow it is whatever the next wrapper adds, with no decision point in between. The response is now composed from typed fields on a new store.RenameCascadeTooLargeError (NewTitle, Retained, Max), reached with errors.As. Unwrap keeps errors.Is(err, ErrRenameCascadeTooLarge) true, so every existing sentinel check is unaffected. Both FIGURES stay in the message, deliberately: Dave's day-63 ruling asked the refusal to state what it would hold and what the cap is, so "split the rename" is actionable advice rather than a shrug. The reviewer read those numbers as a content-size oracle; that framing does not survive the trust boundary — a rename requires `editor`, documents are readable at `viewer`, so the caller can already read every document the figure summarises and learns nothing from it. What they had no business receiving was the internal call path, and that is what changed. This is round 3's lesson applied in the other direction. There, prose was being used to CLASSIFY an error and should have been identity. Here, prose was being used to REPORT one and should have been data. The test now asserts both halves: the real byte counts are present (not merely the word "maximum"), and the strings "update links:" and "store:" are ABSENT. Mutation-verified — splicing err.Error() back in fails on both leaked prefixes. Gates: `go test ./...` under Postgres 17 EXIT=0; gofmt clean; `make lint` 0 issues. BUG-2798 Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN * docs(documents): correct eight claims the prose made that the code does not (BUG-2798) Codex round 6, aimed at the comments rather than the code. Eight findings, all mine, all real, no behaviour changed. This is the failure mode my own trail keeps naming — code right, prose broader than the sweep, always in the same direction — so they are corrected individually rather than smoothed over. Stale after the round-1 rename: - models cited store.MaxRenameCascadeProjectedBytes, which no longer exists. - the constant's own hostile figure read 110,729,522; it is 110,729,520. - the HTTP test said each body is ~135 KB; the formula produces 264,790 bytes. Claims wider than what is true: - The round-trip test's header stated a biconditional over the whole validator. False: a 300-rune title round-trips perfectly and is still refused, for the unrelated reason that it is an amplification factor. The property is about the SYNTAX rule, over titles inside the length bound, and now says so. - The mirrored grammar/unescaper comment claimed that a TypeScript change would make this test start disagreeing. It cannot — they are static copies, and nothing in the repository fails when the two drift. Replaced with what the duplication actually buys and what it does not. - The cap's receipt used `items` measurements to conclude the guard "cannot fire on honest use" for DOCUMENTS, having itself noted that instance's documents table is empty. The proxy is reasonable and it is an assumption, not a measurement of the guarded path; the inference is now named, with the narrower claim that survives without it. - The 413's comment cited the image_too_large precedent as a bound on OUTPUT while this guard bounds retained read-plus-write. What carries across is the shape — a small request refused for what handling it would cost — not the quantity. - TestRenameCascade_RefusesTheSingleDocumentAttack described the 2 MiB / 110,729,520-byte shape it does not build; it sizes from the cap (1,271,000 bytes retaining 67,108,800). The full-strength shape is exercised by the allocation test, and the comment now points there instead of describing a body that is not in the function. One figure was replaced by measurement rather than corrected by arithmetic: the allocation test claimed a "~20x margin". Both sides are now measured and stated separately — refusing allocates 2,114,624 bytes, ~24.8x under the 52,428,800 ceiling; building the rewrite first allocates 110,748,144, ~2.1x OVER it, taken from running the test against the mutation rather than computed. The smaller margin is the binding one, since it is the gap a regression must cross to be caught, and saying "~20x" hid that. Gates: `go test` on the three touched packages under Postgres 17 EXIT=0; gofmt clean; `make lint` 0 issues. BUG-2798 Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN * fix(documents): stop charging the cascade budget for rows the rewriter cannot touch (BUG-2798) Codex round 7. Third instance of one class, and the one I stopped short of when I extended the previous two. The SELECT that finds candidate linkers is a LIKE; the thing that rewrites them is links.ReplaceTitle. They do not agree on what counts as a match, so the SELECT returns a SUPERSET — and every row in the difference was charged to the caller's retained-byte budget and then handed a no-op UPDATE. Instances one and two were `%` and `_`, wildcards on both dialects, closed by the ESCAPE clause. This is the case half, and it splits the OTHER way from the backslash bug: SQLite's LIKE is ASCII case-insensitive by default while Postgres's is case-sensitive, so renaming `Alpha` scans every body holding `[[alpha]]` on SQLite only. ReplaceTitle is case-sensitive on both and will never touch them, so enough case-variant content could push an otherwise valid rename to a 413 — on one dialect, for content that was never in scope. The fix is to skip a row with no case-sensitive occurrence outright, which closes both halves: no budget is spent, and no pointless UPDATE is issued for a body the cascade was never going to change. The authority on what is a linker is the rewriter's own count, not the pattern that proposed the candidate. The test runs on BOTH dialects deliberately, unlike its Postgres-only backslash sibling: on Postgres it asserts the behaviour was already correct, which is what makes it a regression test rather than a SQLite quirk shim. It also asserts the case variants are left byte-identical — `[[alpha]]` is a different link, not a missed one. Mutation-verified: restoring the charge fails it with 33,554,790 bytes against the 33,554,432 cap. Gates: `go test ./...` under Postgres 17 EXIT=0; gofmt clean; `make lint` 0 issues. An earlier attempt at that gate died on host disk exhaustion, not on this diff — 373 stale go-tmp directories from crashed runs, cleared, re-run green. BUG-2798 Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN * fix(documents): report the whole operation's size when a retry is refused (BUG-2798) Codex round 8, on concurrency and the rest of the rename transaction. The advisory-lock lifetime, the CAS loop's termination, the ordering against attachment stamps and version writes, and the new skip's effect on the transaction's invariants all came back clean. One P2 stood. The retry path bounded correctly and REPORTED wrongly. It compared the re-read body against the headroom — right — and then named only that body in the error. Everything the scan counted is still held, so the operation's real size is the scan total plus the re-read, and a refusal could therefore say it would hold 16,777,200 bytes against a limit of 33,554,432: a refusal whose own figures do not justify it, which reads as a server bug rather than as advice you can act on. The compare-and-set is now handed the scan TOTAL instead of a pre-computed budget, so the same number both bounds and explains: refuse when scanTotal + grown exceeds the cap, and report scanTotal + grown. The test now asserts the refusal justifies itself — the figure reported must exceed the cap it cites — via the typed error added in round 5, which is what makes that property checkable at all rather than a string comparison. Mutation-verified: reporting the re-read alone fails it with exactly the 16,777,200-against-33,554,432 shape. Gates: `go test ./...` under Postgres 17 EXIT=0; gofmt clean; `make lint` 0 issues. BUG-2798 Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN * fix(documents): refuse titles whose links the cascade cannot find; count retry buffers (BUG-2798) Codex round 9, asking what is MISSING rather than what is wrong. Three findings: two fixed, one already filed. ## Titles validated against the wrong layer The validator ACCEPTED `Alpha|Beta` and `Alpha\Beta`, and I defended that choice with a test. Both round-trip perfectly — the renderer reads each back as exactly the title it started from — and the first version of this fix banned them on vibes, so being shown that was a real correction. It was still the wrong call, for a reason that test could not see. A link to such a title can be STORED escaped, as `[[Alpha\|Beta]]`, and the rename cascade searches for the raw `[[Alpha|Beta]]` only. It does not find those links, so the rename succeeds and leaves them pointing at a title that no longer exists — silently, which is BUG-2796's defect wearing different syntax. So the property a title has to satisfy is stricter than the one I tested: not "the renderer reads it back", but "the renderer reads it back AND the cascade can find its links". Validating against the layer that DISPLAYS a title while the layer that MAINTAINS it disagrees is the same mistake as the unescaped LIKE, met from the other side — twice in one unit, which is the part worth noticing. `[` stays accepted: the cascade's search term matches it literally, so it passes the stricter property too. The two characters get their own test rather than a row in the round-trip table, because the table asserts a biconditional and these are refused for a reason that predicate deliberately does not model. That test asserts its own premise — each title must still round-trip — so if that ever stops being true it fails rather than passing for a new reason. ## Retry buffers were not counted rewriteLinkerCAS bounded each retry against the scan total plus THAT attempt. Earlier attempts' buffers become unreachable when expected/next are reassigned, but unreachable is not reclaimed, so a run of failures could hold several copies while the arithmetic counted one. Now accumulated across attempts, which is conservative — it counts garbage as if live — and errs toward refusing, which is the safe direction for a memory bound. The loop is capped at cascadeRewriteAttempts, so it cannot grow without end. ## Already filed Item renames remaining unbounded, and item titles lacking this validation, are real and out of this unit's scope: BUG-2804 and BUG-2805, filed after round 2 with the code verified rather than taken on report. Gates: `go test ./...` under Postgres 17 EXIT=0; gofmt clean; `make lint` 0 issues. BUG-2798, BUG-2796 Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN * fix(documents): validate a title only when the rename actually changes it (BUG-2798) Codex round 10, on back-compatibility. This one is a regression THIS fix introduced, not one it inherited, and it falsified a promise the fix makes about itself in three places. "Enforced at write time; existing titles stay valid until their next rename" is Dave's ruling, and it is repeated in the constant's doc comment and in two commit messages. Validating every SUPPLIED title broke it for the most ordinary shape of an edit there is: a client that PATCHes the whole object, title included, to change the content. Under that, a document with a legacy title became uneditable rather than merely un-renameable — the opposite of grandfathering. Grandfathering is not something you get by validating at write time. It is something you get by not validating a write that is not a rename. The check now fires only when the supplied title DIFFERS from the stored one, which is also exactly the test the store already applies before cascading, so the validation and the work it guards now agree on what counts as a rename. The regression test seeds its legacy document through the store, because the title it needs can no longer be created through the API — which is precisely the population the grandfathering clause exists for. Three legs: the echoed-title content edit succeeds, a title-less content PATCH succeeds, and renaming to another invalid title is still refused. The last is the control; without it, deleting the validation entirely would pass. Filed rather than folded: BUG-2806, existing documents whose links are stored in escaped form are still orphaned by a rename. Round 9 stopped NEW titles of that shape being created; it did not repair the ones already stored, and the asymmetry — the product refusing to create a shape it still mishandles — belongs in the record rather than in this PR, which is nine commits deep on a bound it has already outgrown. Gates: `go test ./...` under Postgres 17 EXIT=0; gofmt clean; `make lint` 0 issues. BUG-2798 Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN * fix(documents): validate the rename under the lock, not against a pre-lock read (BUG-2798) Codex round 11. Round 10 moved title validation behind "only when the title actually changes", which is the right rule and was applied at the wrong place. The handler compares the supplied title against a document it read BEFORE the rename lock. UpdateDocument re-reads under the lock. Those can disagree: echo a legacy title back on a content edit while another request renames the document, and the handler sees "unchanged, skip validation" while the store sees a genuine rename — and writes the legacy title through with nothing having checked it. The rule is unchanged; the enforcement point moved to where the rename is actually decided. The store validates inside the transaction, on the same branch that triggers the cascade, and returns a typed InvalidDocumentTitleError carrying the reason. The handler keeps its pre-lock check, which is still worth having — it gives the common case a fast 400 without opening a transaction — and gains an arm that surfaces the store's refusal for the case its own check could not see. Grandfathering survives intact, because the store's check sits on the title-actually-changed branch, which is the same condition the handler uses. The test calls the store directly rather than reproducing the race: driving the interleaving would test the scheduler, while the property worth pinning is that the store refuses regardless of what a caller did. Its control leg is the grandfathering case — a content edit echoing the unchanged legacy title must still succeed — so validating everything here would fail it, which is round 10's regression restated as a guard. Mutation-verified: removing the store-side check leaves the handler tests green and fails this one with a nil error. Gates: `go test ./...` under Postgres 17 EXIT=0; gofmt clean; `make lint` 0 issues. BUG-2798 Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN * perf(documents): stop counting every linker's occurrences twice (BUG-2798) Codex round 13, on what the guard costs the SUCCESS path rather than what it blocks on the failure path. The cascade counts occurrences because the size guard needs that number before it is willing to build anything. It then handed the same content to links.ReplaceTitle, whose strings.Replace with n < 0 counts it again. Every ordinary rename therefore paid a second full pass over every linking document for a number it already had. links.ReplaceTitleN takes the count the caller already computed. It is a separate function rather than an optional parameter because the obligation is real and silent when broken: passing a number that is too small does not error, it leaves later occurrences unrewritten, which on this path means links left pointing at a title that no longer exists. A name at the call site is cheaper than a comment nobody reads. NO measured speedup is claimed, and the doc comment says so. This removes one linear pass from a path that also allocates a full copy of the same content and issues a write per linker, so the saving is real but not obviously significant. It is here because doing the same work twice needs a reason and there was not one — not because a benchmark asked for it. The test asserts equivalence with ReplaceTitle across several shapes, including the new-title-embeds-old case, and its counterfactual leg asserts that an under-count visibly DIVERGES — if it did not, the caller's obligation would be imaginary and the API misleading. Round 13 also confirmed two things worth recording: no quadratic scan across linkers, and the ESCAPE clause does not materially change the query plan because the leading `%` already forced a content scan. Gates: `go test ./...` under Postgres 17 EXIT=0; gofmt clean; `make lint` 0 issues. BUG-2798 Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN * docs(store): record the consequence the cascade cap necessarily has (BUG-2798) Codex round 14, on authorization and abuse. The refusal paths came back authorization-clean: reachable only after the workspace-access and `editor` checks, with viewers, guests, non-members and cross-workspace document IDs stopped first, and the reported byte figure exposing nothing an editor cannot already read. One consequence stands, and it is a property of having a cap at all rather than a defect in this one: once a workspace's documents linking a title exceed 32 MiB, that title can no longer be renamed, and any editor can put it in that state. Recorded in the constant's doc comment rather than fixed here, because the comparison that matters is with what it replaces. The same input previously took the server down for everyone; it now denies one operation to a role that can already delete every document in the workspace. Trading an unbounded OOM for a bounded, legible refusal is the point of the guard, not a gap in it. What IS missing is that the state has no exit but manual cleanup, with nothing telling an operator which documents to clean. That is a quota-and-recovery question rather than a cascade question, and it is filed as IDEA-2807 with three candidate shapes and an argument for the cheapest one — a state you can get out of is a different severity from one you cannot. The filing also says what is NOT established: no real workspace is known to approach the cap, and this fix's own receipt suggests none does, so it is a trap that exists rather than one anyone has fallen into. The adjacent concern the review raised — repeated near-cap renames contending for the rename lock and the connection pool — is noted there too, with the observation that the pre-existing behaviour was strictly worse, since each attempt was unbounded. Gates: `go test ./...` under Postgres 17 EXIT=0; gofmt clean; `make lint` 0 issues. BUG-2798 Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN * revert(links): remove ReplaceTitleN — it optimised nothing (BUG-2798) Codex round 15 caught a claim of mine that was simply false. Reverting the functional half of c00606b0. I added ReplaceTitleN so the cascade could hand strings.Replace the occurrence count it had already computed, and wrote that this "removes one linear pass from a path that also allocates a full copy". It does not. strings.Replace calls Count UNCONDITIONALLY, before it looks at n: func Replace(s, old, new string, n int) string { if old == new || n == 0 { return s } // Compute number of replacements. if m := Count(s, old); m == 0 { return s } else if n < 0 || m < n { n = m } Read from this machine's GOROOT this turn, rather than recalled. Passing n constrains how many replacements are APPLIED; it does not skip the count. So the function bought nothing and cost something: a second way to do the same thing, carrying an obligation that fails SILENTLY when broken — an under-count leaves later occurrences unrewritten, which on this path means links pointing at a title that no longer exists. API surface with a silent failure mode and no payoff is worse than no API, so it goes rather than getting a corrected comment. The failure is the one my own trail keeps naming: I asserted a mechanism without reading it. What makes this instance worse than the earlier ones is that I wrote a careful hedge — "no measured speedup is claimed" — which reads as rigour while the sentence beside it stated the mechanism as fact. Declining to measure a claim is not the same as checking it, and the hedge made the unchecked claim look examined. The occurrence count stays where it is: the guard genuinely needs it before it will build anything, and computing it there is not redundant with anything the guard can avoid. Also declined this round, as already filed: renaming a legacy title with `|`, `\` or `]` leaves escaped links stale — that is BUG-2806, filed at round 10 with the mechanism verified in code. Gates: `go test ./...` under Postgres 17 EXIT=0; gofmt clean; `make lint` 0 issues. BUG-2798 Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN	2026-08-27 22:45:27 -04:00
xarmian	771ec5bbaa	fix(server): refuse invalid-UTF-8 and NUL query values at the transport (BUG-2784) (#1217) ... The query-string half of BUG-2782. A caller-supplied query value reached a Postgres text comparison, Postgres refused the parameter, and the handler answered 500 — the honest answer is 400, because the caller asked about something that cannot exist. ValidateQuery is a root-router middleware beside ValidatePath, refusing 400 invalid_query when a decoded query key or value is invalid UTF-8 or carries a NUL. Both share one predicate, bindableText. Measured on Postgres 17 (server_encoding UTF8) at 19330410: 8 GET endpoints x 54 parameter names — every name any handler reads — each probe from its own source IP because the api limiter is keyed on ip:. invalid-UTF-8 value: 276 x 200, 56 x 400, 100 x 500 -> 432 x 400 NUL value: 276 x 200, 56 x 400, 100 x 500 -> 432 x 400 control: 376 x 200, 56 x 400, 0 x 500 -> unchanged Zero 500s in the control, so the 100 are attributable to the value. The error is `invalid byte sequence for encoding "UTF8": 0xff (SQLSTATE 22021)`. WHY A TRANSPORT RULE, when BUG-2782 planned per-site validators on BUG-2774's validCursorID model. Reading the mechanism retired that plan: parseItemListParams folds every parameter it does not recognise into a field filter, so ?email=, ?type= and ?anything-at-all= reach a text comparison exactly as ?search= does — 98 of the 100 failures are those two endpoints. The set of names is unbounded by design, so there is no finite list of points to validate. WHY IT IS NOT A NARROWING of what callers may send, which was BUG-2782's objection. That objection is sound against a charset rule and does not reach this one: bindableText requires only valid UTF-8 with no NUL, every legitimate value here is text, and text is valid UTF-8 in any language. CONTRACT CHANGE: the timeline's before_id answers invalid_query rather than invalid_cursor, since the transport rule runs first. Same 400, same client-error contract, less specific code. validCursorID is NOT dead — two of its three call sites read ids from the item's own fields blob, which no request middleware sees — and that reasoning is recorded at the function definition where someone would land before deleting it. Keys are validated precautionarily: an invalid-UTF-8 parameter NAME did not reproduce a 500 in the sweep (7 x 200, 1 x 400), and why it survives is unread, so they are checked rather than assumed safe. Gates: go test ./... green; full Postgres suite -timeout=45m green (28 packages, own container, not the shared port); -race on internal/server green; lint 0 issues; vuln clean; gofmt clean. Mutation matrix 14/14, including the unwiring mutation run against the Postgres leg to confirm it fails with the ORIGINAL 500 rather than merely failing. Ten adversarial review rounds. The first two found a vacuous test (one item in the workspace meant a handler ignoring ?search passed it) and a self-contradicting proof. Later rounds found false statements, including one where my own sweep CLAIM was false. Two real defects were found AFTER the first MERGE verdict, which is why the rounds continued. Filed not folded: BUG-2803, an escaped NUL in a JSON body reaching the store on every JSON write path — a different surface needing decode-time rather than transport-level validation. Release note: invalid UTF-8 or NUL bytes in query parameters now return 400 instead of 500 on Postgres deployments. Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN	2026-08-27 10:35:39 -04:00
xarmian	1933041027	fix(events): surface the SUBSCRIBE error and refuse callers instead of admitting a dead stream (BUG-2764) (#1215) ... * fix(events): surface the SUBSCRIBE error and refuse callers instead of admitting a dead stream (BUG-2764) * fix(watchevents): surface the SUBSCRIBE error at construction and on resubscribe (BUG-2764) * docs(events): the idle-cycle prose and metric Help now name the third install-nothing reason (BUG-2764) * fix(events): retry uncovered workspaces, wait on in-flight records, refuse after Close (BUG-2764 codex round 1) * fix(events): post-loop check waits on an in-flight record before trusting a live entry (BUG-2764 codex round 2) * test(watchevents): assert what the failed subscribe leaves, not how long it takes (BUG-2764 codex round 3) * docs(events): prose says what the code guarantees — delivery failure or shutdown, replacement refusals do not count as cycles (BUG-2764 codex round 4) * test(server): the 503 mapping is asserted on both subscribe branches; docs scope the refusal to the activity stream (BUG-2764 codex round 5, BUG-2800) * fix(events): uncovered-workspace retry logs are quiet once the bus is closing (BUG-2764 codex round 6) * fix(events): the uncovered-retry log promises a retry only while subscribers remain (BUG-2764 codex round 7)	2026-08-27 00:13:04 -04:00
xarmian	cade263fe5	fix(store): compare-and-set the wiki-link cascade's writes (BUG-2785) (#1213) ... * fix(store): compare-and-set the wiki-link cascade's writes (BUG-2785) A document rename cascades through every document linking the old title as a read-modify-write across two statements: SELECT each linker's content, rewrite the string in Go, UPDATE the row. A content edit to a linker committing between those two statements was silently overwritten — the cascade wrote the body it built from the version it read, with no error and no version row for the loss. Same lost-update shape as BUG-2770's activity-metadata merge, one table over, and fixed the same way: the UPDATE now carries `content = ?` with the body the cascade read, plus a bounded retry that re-reads the row and re-applies the rewrite. DIALECT SCOPE, which decides what the tests can prove. Reachable on POSTGRES only. SQLite's DSN sets `_txlock=immediate`, so UpdateDocument's db.Begin() takes the write lock at BEGIN and holds it across the whole read→write window; a concurrent edit cannot commit inside it and serializes on busy_timeout instead. On Postgres under READ COMMITTED each statement takes a fresh snapshot and the stale body wins. The CAS is therefore a no-op on SQLite by construction — the predicate always matches, because nobody else can have written. Two consequences, both acted on rather than noted: the new tests SKIP loudly on SQLite instead of passing for a reason unrelated to this fix, and the mutation matrix was run under Postgres, where removing the CAS leaves a SQLite suite entirely green. The retry rewrites the WINNER's body rather than replaying the original rewrite — replaying it would reintroduce exactly the text this bug loses. A mutation that replays instead is in the matrix. THE ZERO-ROW RESULT NEEDS A PROBE. The UPDATE now has two predicates that can each refuse it, and RowsAffected cannot say which did: `deleted_at IS NULL` (the linker was archived — a documented normal outcome, stop) or `content = ?` (a concurrent edit landed — re-read and retry). Treating them alike either retries forever against a deleted row or discards a live linker's rewrite, so a probe distinguishes them, as BUG-2770 needed for the same reason. The probe reads through tx, never the pool (BUG-2409). On retry exhaustion the RENAME fails rather than leaving one linker holding a title that no longer exists. The alternative — log and continue — was considered and rejected: it trades a loud retryable failure for a silent inconsistency, and a rename is atomic in intent. Exhausting three attempts needs three consecutive commits to the same linker inside one cascade. Adds afterLinkCascadeRead, the seam between the cascade's read and its writes. No existing seam reaches that gap: afterDocumentPreLockRead fires before the transaction, afterDocumentPreWrite before the renamed document's own update. That new seam also closes a gap a previous unit recorded as permanently open: the `deleted_at IS NULL` guard carried a note calling itself UNTESTED because reaching its window "would cost a fifth seam". This is that fifth seam, so the note is removed and replaced by a record of the closure, and the archived-linker test now drives exactly that window. Mutation matrix, run under Postgres — 5 mutants, 5 detected, including one that is literally the pre-fix code: M1 CAS predicate removed (the unfixed behaviour) → lost-update test M2 soft-delete probe arm removed → archived-linker test M3 retry budget cut to one attempt → lost-update test M4 retry replays the original rewrite → lost-update test M5 seam removed (control: tests must notice they never raced) → lost-update test Gates: gofmt clean, lint 0 issues, govulncheck clean, full suite green on SQLite (28 pkgs) and on Postgres 17 (28 pkgs, internal/store 346s vs 79s — the positive control that the PG legs ran rather than skipped). Not fixed here, filed as BUG-2795: cascadeTitleRename, the ITEM-side cascade, has the identical defect and no lock closes its window either. Its fix does not transfer — it rewrites by POSITION from item_wiki_links offsets, so a retry must re-derive positions from the winner's body and re-run replaceWikiLinks, which is a redesign of the retry unit rather than a predicate on an UPDATE. wiki_links.go's claim that it "matches the document rename behavior" is corrected to say which half no longer matches. * style(store): gofmt the CONVE-23 sweep comment (BUG-2785) Same failure as the previous unit, same cause, and worth naming rather than quietly fixing: gofmt wants a blank line between list items once one item grows a second paragraph, which the note about the previously- untested deleted_at guard made true. I re-ran build and the full Postgres suite after those comment edits but not lint, because the change was 'only a comment' — the exact reasoning the previous unit's fix commit warned about in writing, one unit earlier. The PR body's claim that all gates were re-run after the prose edits was false and has been corrected there too. The rule as remembered does not work. The mechanical form does: gofmt and lint are the last action before a push, comment-only changes included. * fix(links,server): stop a rename hanging the server, and report cascade contention honestly (BUG-2785) Three findings from Codex round 2 on this PR. The first is a server hang. 1. ReplaceTitle could never terminate. replaceAll looped "find old in result, splice new in" — re-searching the string it was building, including the text it had just inserted. When the NEW title contains the OLD link token it grows without bound. Measured, not argued: ReplaceTitle("x [[A]] y", "A", "A]] [[A") builds `[[A]] [[A]]`, which still contains `[[A]]`; a probe against the old implementation ran 3s without terminating before being killed. Document titles have no validation, so this is reachable from user input — and the caller is inside the rename transaction holding the workspace rename advisory lock (BUG-2778), so the hang would take every other rename in that workspace down with it while exhausting memory. strings.Replace with n = -1 has the semantics that were wanted: non-overlapping, left-to-right, over the input. Three-case regression test, all three of which fail against the old implementation, plus a control that catches a "fix" which terminates by doing nothing. Pre-existing, and folded in rather than filed: three lines against a server hang, and this PR's retry calls the helper again per attempt, which makes it reachable more often than before. 2. Retry exhaustion surfaced as an opaque 500. The rename rolls back cleanly and retrying can succeed, so "an internal error occurred" tells the caller the opposite of the truth. Adds the exported ErrLinkCascadeContention sentinel; the handler now answers 503 lock_contention with Retry-After, reusing the disposition BUG-2778 already established for 55P03/40P01. 3. That 503's message claimed the workspace was "busy with another rename". 55P03 there is just as likely to be an ordinary content edit holding the row, and the new arm is definitely one. It no longer names a cause the server has not established. Also closes the coverage gap round 2 named around this unit's own decision: exhaustion now has a test asserting the rename ROLLS BACK (target keeps its title) and that the concurrent editor's text survives that rollback. cascadeRewriteAttempts becomes a var so the test can force exhaustion at 1 rather than arranging three consecutive commits, which would need a per-attempt hook in production code — the divergence from its const sibling is noted where it lives. Records two limitations in the code rather than leaving "the cascade is safe now" to rot: the MIRROR direction is still open (a content writer that read before this transaction can commit afterwards and reinstate the old title — fixing it means giving ordinary content writes a CAS too), and delete-then-restore of a linker mid-rename brings back the old title. Both pre-existing, neither worsened here. Mutation matrix now 7 mutants, 7 detected, run under Postgres: M6 (%w -> %v, sentinel lost) and M7 (exhaustion swallowed) cover the new mechanisms. Gates re-run on the tree being pushed, tests included, after the final comment edit rather than before it: gofmt clean, lint 0 issues, SQLite 28/28, Postgres 17 28/28 (internal/store 345s vs 79s). gofmt caught an unformatted test file locally this time, which is the point. * fix(store,links): correct three prose claims and close the SQLite coverage gap (BUG-2785) Codex round 4, on SQLite semantics and prose accuracy. One finding says a bug I FILED is wrong; that is the important one. 1. BUG-2795's premise was false, and this PR repeated it in a comment. I filed that item claiming the item-side cascade has "the identical defect" and that "no lock closes its window either", on the strength of a grep for pg_advisory_xact_lock in items.go that turned up only the parent-link locks. It missed acquireWorkspaceSeqLock (items.go:2136), taken UNCONDITIONALLY by every UpdateItem — content-only edits included — immediately after Begin and long before cascadeTitleRename, and held to COMMIT. So two item updates in a workspace fully serialize on Postgres and an ordinary content edit CANNOT commit inside that cascade's window. The scenario I filed is not reachable. Not fully invalid: sweeping every `SET content` writer in internal/store finds RemapAttachmentReferencesInWorkspace, which rewrites items.content in its own transaction without that lock. So a real but far narrower window survives — attachment remap versus cascade, not user-edit versus cascade. BUG-2795 corrected on its trail and dropped to low; the comment here now states the lock, the one surviving writer, and stops claiming parity with the document cascade. I searched for the locks I expected rather than for what serializes that path, then wrote a sentence broader than the search. Same failure this PR's review has produced repeatedly. 2. "A title nobody should be able to write" was false — the document API validates doc_type and status, never the title. Correcting it surfaced a real second defect: renaming to `A]] [[A` now terminates (round 2's fix) but writes `[[A]] [[A]]`, two links to nothing. Filed as BUG-2796. The termination test deliberately still asserts the COUNT rather than the output, so it does not freeze today's broken rendering as intended behaviour. 3. The hang's blast-radius claim named the workspace rename advisory lock without qualifying the dialect. That lock is a no-op on SQLite, where the equivalent damage is the database-wide write lock the transaction already holds under BEGIN IMMEDIATE. Different mechanism, same outcome for everyone else. Also closes the last coverage gap round 2 named. The CAS predicate runs on SQLite in production and every concurrency test skips there, so TestUpdateDocument_CascadeRewritesEveryLinkOnBothDialects does not skip: two linkers, multiple links per body, plus a document that merely contains the word and must be left alone. Verified it earns its place — a mutant comparing against the rewritten body instead of the body that was read compiles, and dies to this test ON SQLITE, where nothing else would have caught it. Documents the parallel-test constraint on the now-mutable cascadeRewriteAttempts, with its boundary: no current t.Parallel test in internal/store reaches this cascade, but that is a fact about today's corpus rather than an invariant. Gates on the pushed tree: gofmt clean, lint 0 issues, govulncheck clean, SQLite 28/28, Postgres 17 28/28 (internal/store 349s vs 79s). * docs(store,links): point the cascade comment at the root cause, qualify a claim in its second location (BUG-2785) Codex round 5, probing cross-connection visibility and transaction boundaries. Two findings, both mine, neither changing behaviour. 1. The "one surviving writer" framing was too comfortable, twice over. RemapAttachmentReferencesInWorkspace is not a rare non-interactive writer: bundle import reaches it on an ordinary user-triggered import, and the workspace is ALREADY VISIBLE to its owner while it runs — store.ImportWorkspace commits the workspace row (with owner_id) in its own transaction before opening the one that inserts items, and the bundle handler runs the remap as Phase 3 afterwards. Both verified in code. So that writer races ordinary item edits, not just the rename cascade, and it is missing a guard outright rather than being an exotic pairing. Filed as BUG-2797, which covers the remap itself; BUG-2795 is now a consequence of it and says so on its trail. The comment here points at the root cause instead of implying the cascade's pairing is the whole story. 2. The dialect-unqualified advisory-lock claim survived in a SECOND location. Round 4 caught it in links.go and I fixed it there; the same sentence sat in the termination test's comment, and I never enumerated the sites. That is CONVE-23's verify half failing exactly as it warns: I fixed the instance I was shown rather than the population. Both now qualified, and a sweep for remaining unqualified copies leaves only BUG-2778's own comment, which already carries its no-op-on-SQLite note. Worth recording that this is the second time on BUG-2795 that a scope sentence of mine ran ahead of the sweep supporting it — first "no lock closes its window either" (acquireWorkspaceSeqLock did), now "not user triggered" (import is). Both found by a review round rather than by me. Gates on the pushed tree: gofmt clean, lint 0 issues, SQLite 28/28, Postgres 17 28/28 (internal/store 348s vs 79s).	2026-08-26 22:04:15 -04:00
xarmian	91d92f184f	feat(server): refuse workspace creation without may_create_workspaces consent (IDEA-2756) (#1212) ... * feat(server): refuse workspace creation without may_create_workspaces consent (IDEA-2756) The OAuth consent screen's "Let this app create new workspaces" checkbox gated only the post-creation auto-add. A connection whose user left it unticked could still create workspaces; it simply could not then see them. A permission that does not prevent the action it names is a consent mismatch. Dave ruled it: the checkbox is a permission on whether the connected token may CREATE, and it has to be true to what a user would honestly expect from the option. The behaviour-change-for-existing-connections argument loses to honest consent semantics. Adds Server.requireWorkspaceCreationConsent, a shared gate at the top of both endpoints that mint a workspace under the caller's account: POST /api/v1/workspaces handleCreateWorkspace POST /api/v1/workspaces/import handleImportWorkspace Import reaches CreateWorkspace via store.ImportWorkspace, so it is the same permission at a second door — lead-ruled as an application of the same rationale, not a new decision. The gate sits above the Content-Type dispatch, so it covers the tar.gz bundle path (whose only route is that handler) and refuses before the 64 MiB body read. Refusal is a 403, mirroring handleAuditLog's consent refusal (BUG-2102): a hard decline rather than a narrowed response, because there is no narrower version of creating a workspace. Three non-refusal cases and one refusal, all but the last with a test: - not an OAuth grant (PAT, CLI session, local stdio) — creation rides on ordinary account authority - ErrOAuthConnectionNotFound (pre-Phase-C grant) — ALLOW, matching the backfill's may_create_workspaces=ON default. Deliberately asymmetric with maybeAutoAddCreatorConnection's not-found branch, which declines a convenience where this one would invent a refusal - flag set — proceeds; the auto-add is unchanged - a store I/O error — REFUSED, failing closed with a 500, because allowing the create when the deciding state could not be read grants a declined permission on the strength of a database blip. This is the one branch with no test: injecting a store read failure needs a fault-injecting store the package does not have, so it is reasoned rather than measured Population enumerated before the fix (CONVE-18): five CreateWorkspace call sites, two of them HTTP endpoints reachable by an OAuth token (both gated). Excluded with reasons: autoCreateWorkspace (signup-time, no connection in context), workspace restore (un-deletes an existing workspace), /oauth/claim (grants access, does not mint), cmd_db.go (local store copy, no HTTP). Search boundary: the sweep traced Store.CreateWorkspace callers and did not look for a path that inserts a workspace by raw SQL. Ten tests, all driving the real router rather than calling handlers directly (CONVE-19). Every refusal leg asserts that no workspace of that name exists afterwards, not merely the status code (CONVE-12) — a guard that 403s after the write passes a status-only assertion. Seven mutants, seven detected, including both guard-placement mutations. MCP tool surface 0.25 -> 0.26. Behaviour bump on the v0.9/v0.16/v0.25 grounds: no tool name, action enum or param shape changed, but pad_workspace.create now refuses a call it used to permit. Closest precedent is v0.10; unlike v0.10 there is deliberately no escape-hatch param, because the gate encodes a decision the USER made at consent time and a bypass flag would be the app overriding its own grant. CONVE-23 sweep for prose the change falsified: instructions.md told agents the create still succeeds and to use the claim flow (it would have sent them to claim something that was never created); the TASK-2753 allow-list guard entry asserted the same and posed IDEA-2756 as open; the MCP catalog and CLI help described only the flag=true path; maybeAutoAddCreatorConnection's flag-off branch is now unreachable from its sole caller and is documented as dead code kept for contract, to be deleted only with the guard. CLAUDE.md was already stale at v0.24 (v0.25 bumped the constant without it) — brought to v0.26 with a backfilled v0.25 line. The consent screen and console copy are unchanged: they were the misleading half of this bug, and the fix makes them true. * docs(server): state the import gate's reachability precisely (IDEA-2756) The import-side gate is correct but currently unexercised in production, and the first framing of this change did not say so. WithMCPTokenIdentity is stashed by exactly one middleware, MCPBearerAuth, mounted on /mcp alone. An OAuth connection reaches an /api/v1 handler only through the in-process MCP dispatcher, and that dispatcher's route table has a workspace create action but no workspace import. So no OAuth-bound caller can reach handleImportWorkspace today. The gate stays, and the comment now says why: adding that action later must not silently reopen the door, which is the state a create-only fix would have left armed. Found on a verify pass reading the middleware mount points, not by the tests — they synthesize the OAuth identity into the request context, so they prove the handler's behaviour GIVEN an identity and have no opinion about which routes supply one (CONVE-19). Codex round 2 reached the same conclusion independently. * fix(server): correct five overstated claims from Codex round 3 (IDEA-2756) All five were mine, all P2, none changing the gate's behaviour — four are claims that were broader than the code, one is a test that proved less than its name. 1. "Only re-authorization lifts it" was wrong in five places (version.go, README, CLAUDE.md, the MCP catalog description, CLI help). A user can also enable the flag on the EXISTING connection via PATCH /connected-apps/{id}/flags, which the console page drives — instructions.md said so and contradicted the others. All five now name both remedies, and both are still the user's, which is the part that matters: neither is reachable by the app. 2. "This branch is UNREACHABLE ... it is dead code" on maybeAutoAddCreatorConnection's flag-off branch was false. The gate reads the connection and that function reads it AGAIN after creation; a user revoking creation power from the console between those two reads lands exactly there. It is a real second check across a real TOCTOU window, failing in the safe direction. The claim was written from the call graph, which cannot see a concurrent write between two reads. 3. handlers_import_bundle.go's "Auth: any authenticated user" was made false by this change and the concept sweep never had a chance at it — it greps may_create / auto-add / creation power, and that sentence contains none of them. Corrected in place. 4. The two NonOAuthCallerUnaffected tests claimed PAT, CLI session and local stdio; each drives one PAT. The comments now state the fixture's real scope and why one caller stands for the class (the guard branches on an identity only MCPBearerAuth sets, so callers that skipped it are indistinguishable) rather than implying three fixtures. 5. The JSON import refusal leg would have passed with the gate below decodeJSONWithLimit — only the bundle leg pinned placement, and only for gzip. Adds TestImportWorkspace_ConsentRefusalPrecedesBodyDecode (malformed body: 400 if the gate is late, 403 if it is early), mirroring the create-side ordering legs. Mutation matrix now 9 mutants, 9 detected. M8 (guard below the JSON decode) is killed by the bundle leg too, so it shows the new test is covered rather than necessary; M9 gates the bundle path and moves only the JSON path's guard, and dies to the new leg ALONE. That is the mutant that justifies the test. * docs(server): the second consent check narrows the race, it does not close it (BUG-2792) Round 3 caught me calling maybeAutoAddCreatorConnection's flag-off branch dead code. The replacement comment then claimed the branch means a revoked grant cannot silently gain a workspace — which is more safety than the code delivers, and round 4 caught that. The read and the AddConnectionWorkspace insert below it are separate unconditional statements, so a revocation landing BETWEEN them still adds the workspace. The check narrows the window; it does not close it. Filed as BUG-2792 rather than folded in: the race is pre-existing and unchanged by IDEA-2756, and closing it needs an atomic check-and-insert at the store layer, written and gated for both dialects — materially more diff and risk than this handler-level guard. Both mistakes were the same shape in opposite directions: a claim about concurrency derived from reading the call graph, which cannot see a concurrent write between two reads. * style(server): gofmt the doc comment (IDEA-2756) gofmt wants blank lines between list items once one item spans multiple paragraphs, which the BUG-2792 note made true. My error, and worth naming exactly: I ran build, vet and the targeted tests on this commit but not lint, because lint had passed on the PREVIOUS commit and the change was 'only a comment'. The gate has to run on the tree being pushed, not on an earlier one that resembles it. CI's golangci-lint is pinned to the same v2.11.4 the Makefile installs, so there was no version skew to blame — the local gate would have caught this in 51 seconds. * docs(server): correct ten overstated prose claims from Codex round 8 (IDEA-2756) Round 8 reviewed only the prose this change adds. Ten claims were broader than the code. All ten are mine; none changes behaviour. Rounds 3, 4 and 7 each caught one of these, which is why round 8 was pointed at the class rather than at a new dimension. The substantive ones: - "gates every endpoint that MINTS a workspace" — autoCreateWorkspace mints from registration, bootstrap and oauth-login and is deliberately outside this gate. The helper doc and the test header now name the two callers and the exclusion instead of claiming universality. - "the agent was handed a workspace it could not then see" (version.go, README, CLAUDE.md) — only true for a connection with an EXPLICIT allow-list. An all_current_workspaces=true connection is not gated per slug and could see what it made. The consent mismatch is the constant; the invisibility was its most visible symptom, not its definition. - "ErrOAuthConnectionNotFound — a pre-Phase-C grant" asserted a cause the code cannot know: ANY missing row takes that branch. Now stated as the expected cause, with the limit of what the code can tell. - "above the 64 MiB body read" conflated the two import paths. 64 MiB is the JSON decode's bound; the bundle path has its own, much larger. The gate precedes both, which is the property that actually matters. - "the request context is decorated AFTER TokenAuth runs" was false, and inherited verbatim from the sibling helper this was modelled on (handlers_oauth_claim_test.go's doClaim), where it is also false. The wrapper sets the identity BEFORE ServeHTTP; it survives because nothing on the /api/v1 chain writes that key. - "lets CreateWorkspace normalize it" — CreateWorkspace slugifies only when the supplied slug is EMPTY, and import supplies a non-empty one, so an imported workspace keeps the ?name= value verbatim. - "The PAT needs a workspace to bind to" — CreateAPIToken takes WorkspaceID as optional. And one where the first fix was worse than the finding: - "Every refusal leg asserts no workspace exists afterwards" was false — the two ordering legs assert status only. My first correction ADDED those assertions, which is the trap the finding was pointing at: a malformed body and an empty name are rejected before creation under every guard placement, so "no such workspace exists" is true of broken and working code alike. Reverted; the header now states which legs carry the counterfactual, and why the ordering legs discriminate on status instead. Gates re-run on the tree being pushed, not an earlier one: gofmt clean, lint 0 issues, internal/server and internal/mcp green, mutation matrix still 9/9. * ci: re-trigger CI after a GitHub startup_failure (IDEA-2756) No code change. The Go job on cb47c763 failed on BUG-2786 (the recurring internal/events subscribe-confirm guard, which fails by asserting its own premise: 'the acknowledgement never landed before the mark; this test could not have discriminated'). CONVE-11 owes that failure a re-run before it can be called a flake. rerun-failed-jobs produced attempt 2 = startup_failure with the Go job stuck in 'queued' — a GitHub infrastructure fault, not a test result — after which the run refuses further retries ('This workflow run cannot be retried'). The CI workflow has no workflow_dispatch trigger, so a push is the only way to get a fresh run. Evidence the failure is unrelated to this branch, gathered before re-running rather than after: the branch touches 0 files under internal/events (11 files total, none in that package), and the parent tip c6818500 had Go: SUCCESS with the only non-comment Go difference being one added assertion in this branch's own test file. Go (PostgreSQL) also passed on cb47c763, exercising the same package.	2026-08-26 13:23:56 -04:00