182 Commits

Author	SHA1	Message	Date
xarmian	b437cc582d	feat: item reminders — the fire-at-an-instant primitive, and one overdue rule for all four surfaces (IDEA-2641, closes #1010) (#1244) ... * feat(store): item reminders — the fire-at-an-instant primitive (IDEA-2641) Adds the storage, the scheduler tick, and the canonical event for one-shot item reminders (GitHub #1010). Nothing in Pad acted at a target time before this: a due_date makes an item show up as overdue once somebody asks the dashboard, so "revisit TASK-X on the 1st" had to live in an external cron. A TABLE, NOT A SCHEMA-FIELD ANNOTATION. The design sketch proposed marking schema date fields with a `reminds: true` key on models.FieldDef; recon overturned it. Such a key does not survive an ordinary collection edit, two independent ways: the web editor destructures each field into an EditableField and rebuilds a fresh definition key-by-key on save, so unknown keys are dropped (`pattern` and `unique_scope` survive only because two lines were hand-added for them), and models.CollectionSchema has fixed fields with no catch-all, so any Go unmarshal+marshal round-trip strips unknown properties — the hazard retargetRelationFieldsTx mutates raw JSON to avoid. Both failures are silent and both disarm a whole collection's reminders at once. It is the same defect class that moved traits out of the schema column in TASK-2657. The table also gives the lifecycle a home. A reminder is armed, then fired, then acknowledged, and a re-arm returns it to armed — per-reminder state a field definition has nowhere to keep. remind_at is an RFC3339 UTC instant, deliberately not a `date` schema value: those admit both YYYY-MM-DD and full RFC3339 and are compared against the SERVER'S LOCAL calendar day. A fire-at time cannot carry that ambiguity. The remaining timezone question for due_date is filed separately. Firing is one transaction per reminder carrying BOTH the fired_at write and the outbox insert. That pairing is the point: a fired_at committed without its event is a reminder that silently notifies nobody and can never be retried, because the row has left the armed set; an event without fired_at fires every tick forever. The UPDATE's own `fired_at IS NULL` predicate is the arbiter, so two instances ticking at once produce exactly one winner. item.reminder_due is admitted to the closed events/1 set as v1.2, with a new PayloadReminder family and no SSE name. The subject is the REMINDER, not the item: two reminders can be armed on one item, so an item-subject event could not say which fired, and the reminder id is what an acknowledgement addresses. A new payload family rather than reusing the item snapshot for the same reason — a snapshot would validate and still not answer the only question the event exists to answer. No SSE name in v1 because the poll surface is the contract; adding one later is additive, removing one is not. Ack is explicit and nothing else acks. An item reaching a terminal status deliberately does NOT ack: that would make every status write a reminder mutation, and it would silently consume a reminder set to fire after the work was done. * feat(server): reminder surfaces, and one shared overdue rule for all four Second half of IDEA-2641: the HTTP surface, the scheduler tick's wiring, and the fix for the finding that justified the unit — `ready` / `next` did no date handling at all. OVERDUE NOW HAS ONE IMPLEMENTATION. It used to live inline in the dashboard's attention loop, which meant `pad project stale` inherited it (it filters that very list) and the recommendation surface never saw it. So a deadline reached the two surfaces that REPORT on work and never the one an agent PULLS from. overdue.go is now the only place that decides, and all four call it. Two behaviour changes fall out, both deliberate: - An overdue item bypasses the orphan branch's high/critical priority gate. That gate was where a deadline quietly stopped: a low-priority item three weeks late was reported by `stale` and never suggested by `next`. - Overdue sorts above in-progress. The list is capped at three, so a rank below in-progress would not merely order the deadline lower — on any workspace with three things in flight it would keep an overdue item off the surface entirely, which is indistinguishable from not shipping this. The server-local-today comparison is UNCHANGED and known to be wrong for multi-timezone deployments; it is filed as its own item with the cloud case stated. Changing what "overdue" means on every existing instance inside a change about where the rule LIVES is the kind of behaviour change nobody reviews. Fired reminders reach `next` / `ready` two ways, from one filtered list: PendingReminders is the addressable form (it carries the id an ack needs), and a prepended suggestion is the rendered form. They are prepended AFTER the cap rather than entered as ranking candidates — a reminder is not a task competing on priority, and whether it appeared should not depend on how busy the workspace is. Terminal-item reminders are FILTERED from the surface, never acked. Acking on terminal status would couple every status write to reminder state and would consume a reminder armed to fire after the work was done. The row stays exactly as the user left it; the distinction is observable, and asserted. Three guard tests caught this change and each was answered rather than silenced: - The request-body reader guard was right: the handlers now go through decodeJSON, inheriting the NUL refusal and the size cap. - The canonical-events guard was right: item.reminder_due is admitted to the duplicated contract table as SPEC-3 v1.7, with the reminder subject kind and the new payload family. SPEC-3's own text owes the same amendment. - The NUL census asked for a decision on eight new columns. None carries caller text: ids and FKs are server-generated, four are the server clock, and remind_at is now re-parsed and re-formatted in the STORE as well as at the edge — so the stored value is always machine-produced from a parsed time and no caller bytes reach the column. The doc comment that used to say "the caller normalizes" protected nothing. Regenerating the baseline also found that GEN_NUL_BASELINE=1, which the test's own instructions name, was never implemented — the flag did nothing, so the documented path was hand-editing the file. Implemented, so the next reader gets the mechanism the instructions promise. * test(reminders): the lifecycle, the four surfaces, and 22 killed mutants Every test here was designed against a specific mutation and the mutation was RUN. A green suite proves nothing about a suite nobody tried to break, and three of the mutants I first wrote were not experiments at all. Store (10 mutants, all killed): candidate predicate <= flipped to >=; the event emission lifted out of the fire transaction; the fire UPDATE's `fired_at IS NULL` arbiter removed; the RowsAffected check ignored; re-arm clearing fired_at but not acked_at; ack losing `fired_at IS NOT NULL`; the poll surface losing `acked_at IS NULL`; normalizeRemindAt no longer refusing; it dropping .UTC(); GetReminder losing its workspace scope. Surfaces (12, all killed): the priority gate no longer bypassing on overdue; the sort no longer ranking overdue first; attention leaving the shared helper; the reason losing its OVERDUE prefix; the comparison flipped to >; terminal items no longer skipped; terminal reminders no longer filtered; the filter ACKING instead of hiding; reminders appended instead of prepended; the tick running on a far-future clock; ack answering 200 for an unfired reminder; parseRemindAt accepting a bare date. THREE MUTANTS DID NOT COUNT ON THE FIRST PASS and were rewritten. Two failed to compile (`if false` orphaned a variable; deleting a parse orphaned an import) and one had an anchor matching two call sites. A non-compiling mutant emits zero FAIL lines and reads exactly like a surviving one — it invents a hole that is not there — so the harness reports BUILD-FAIL and ANCHOR-BAD as outcomes distinct from SURVIVED. It also restores files from an in-memory copy rather than `git checkout`, which would delete uncommitted work in the tree. ONE MUTANT GENUINELY SURVIVED and the test was at fault, not the mutant: appending rather than prepending reminder suggestions was undetectable because the fixture had a single item, so the reminder sat at index 0 either way. The fixture now fills the three-item cap with in-progress work, where an appended reminder lands fourth and vanishes. Faithful mutant, weak test — checked in that order. The same lesson shapes the four-surface fixture: it is a LOW-priority open orphan, because that is the case the old code handled worst. A high-priority task would have made the ready/next leg pass against the unfixed tree, which is a green that measures nothing. Negative controls throughout: a future deadline is not overdue and does not reach the gate bypass; a tick with nothing due fires nothing; a completed item is neither overdue nor suggested. Without them a helper that reported every date, or a tick that fired everything, would satisfy every positive leg. The lead's pin is asserted in both directions: a fired reminder on a done item is ABSENT from the surface and PRESENT and still unacknowledged in the table. Asserting only the absence would pass against an implementation that consumed the row, which is the behaviour the pin exists to forbid. * feat(mcp): pad_item.remind + ack-reminder, ToolSurfaceVersion 0.28 An agent that can RECEIVE a reminder but not set one has half the primitive. The poll surface is pad_project.next / ready, both long exposed, so reminders already reached agents — what was missing is the other half: deferring a piece of work is exactly the moment an agent knows when it wants to be asked again, and it had no way to say so. Two additive actions, two optional params. Nothing existing moved, so a v0.27 consumer enumerating neither is unaffected — the v0.13 / v0.11 / v0.8 disposition, which likewise wired existing CLI verbs onto the catalog. remind_at REFUSES a bare date rather than reading it as midnight. Worth stating because the `date` schema type accepts YYYY-MM-DD and a caller will reasonably try it here: a bare date names a 24-hour span, and choosing an hour inside it would fire at a time nobody picked. Re-arm and disarm stay CLI-only. Both address a reminder by an id the agent would have to list first, and no listing action exists on this surface — a door with no handle. Adding them later is additive. Five guards had to be taught, and each was answered on its merits rather than excluded: the HTTP parity test (route mappers added, so the actions work on the remote transport rather than being advertised and unrouted), the read-only catalog's cmdhelp fixture and expected cmdPath map, the field- conflict classifier (remind_at / reminder_id are NOT field writers — a reminder is a row in its own table addressed by its own id, so listing them as classified sources would have pointed detectFieldConflicts at something that is not a field source), and the instructions.md / README action tables. That machinery is why the version bump is safe to make now, and it earned its keep on this change: every one of the five failed on the first build after the catalog entry landed. CONVE-23 sweep for prose this falsifies: - SPEC-3 (DOC-2653) amended to v1.7 in the room, recording item.reminder_due with its new subject kind and payload family — the first canonical event with no user mutation behind it, since a scheduler tick produces it. - CLAUDE.md gains the reminder routes, the CLI verbs, and the v0.28 entry. It was also stale at 0.26 with NO v0.27 entry at all: the 0.27 unit swept instructions.md and README.md and missed this file. Both added. - skills/pad/SKILL.md gains the verbs and a routing entry, including the two things an agent will get wrong — the time is an instant, so ask for a time of day rather than picking one, and finishing the item does not acknowledge the reminder. * fix(reminders): codex round 1 — four findings, all real, all with a pin Round 1 found four defects and refuted none of them. Each fix carries a test that fails against the code as it was, and each of those was mutation-checked. **P1 — pending reminders bypassed item-level visibility.** Every other dashboard section reads `allItems`, which the store already scoped to the caller's collections AND their granted item ids. The pending-reminder list is a direct workspace-wide query and inherited none of that, so a guest holding a grant on ONE item could read the refs and titles of every other item in the collection through its reminders — an item-level leak wearing a notification's clothes. Now filtered with the same `isItemVisibleToGuest` call the sibling sections use. The test's two items share a COLLECTION on purpose: a collection-level filter was already applied, so separate collections would have made it pass against the unfixed code. **P1 — soft-deleted items could starve the queue permanently.** Candidate selection ignored `deleted_at`, and `fireOneReminder` rolls back when it finds the item gone — which leaves the reminder ARMED and therefore a candidate again on the next pass. Candidates are ordered oldest-first and bounded by a limit, so enough archived reminders fill every batch and no live reminder ever fires. Silent, too: the tick reports zero fired and looks idle. Excluded in the candidate query rather than skipped downstream, so those rows never occupy a slot; the reminders themselves are kept, so restoring an item restores its reminder with it — asserted, because a fix that reaped them would pass the starvation test alone. **P2 — the pass stopped at the first failing reminder.** The per-reminder transaction exists precisely so one unfireable row cannot hold back the rest, and `return fired, err` made that comment false — with candidates oldest-first, one persistently broken old reminder blocks every newer one forever. Now continues and joins the errors, so a pass that fired seven and failed three reports both halves rather than reading as clean. The loop is split behind an injected seam because a real mid-transaction failure is not reachable from outside: the database refuses the corrupt rows that would cause one (verified — invalid JSON in items.fields is rejected by the schema). **P2 — suggestions dropped the reminder id.** The docs tell an agent to acknowledge what it sees in next/ready, and the payload carried no handle: a stateless poller could read the reminder and had no way to retire it, so it would be shown the same item forever. `DashboardSuggestion` now carries `reminder_id` (omitempty), `pad project next` prints the exact ack command, and the test acks with the id the surface handed out rather than merely checking the field is populated — a wrong-but-present id satisfies equality with itself. Four mutants, four killed; one was rewritten first because its anchor matched two call sites and was therefore not an experiment. * fix(reminders): codex round 2 — four findings, all real **`--rearm` was unusable.** `ExactArgs(1)` forced an item ref that the rearm branch then ignored, so the flag could not be reached without supplying a ref that was silently discarded. Now `MaximumNArgs(1)`, with each mode checked explicitly: a ref is required to arm, and a ref supplied ALONGSIDE `--rearm` is refused rather than ignored — it names an item the reminder may not even belong to, and quietly dropping it is how a user learns nothing about the reminder they just moved. **`unremind --format json` emitted plain text**, breaking the parseable-output contract every sibling command honours. **The MCP `ref` param did not list `remind`.** Agents read that flat description to decide what to send, so an action missing from it is an invalid call waiting to happen. It now also says what `ack-reminder` takes instead, and why: a reminder is addressed by its own id because an item can carry several. **Fractional seconds fired early.** `time.Parse` accepts `09:00:00.900Z` and `Format(RFC3339)` drops the fraction, so it was stored as `09:00:00Z` and fired 900ms BEFORE the moment the caller named — silently, having rewritten their value on the way in. Seconds are genuinely the stored resolution (the column is compared as a string against a whole-second clock, and the tick runs every 30s), so the only question was which way to resolve it, and truncation resolved it the wrong way. `NormalizeInstant` now rounds UP: at most a second of lateness, in exchange for a guarantee that can be stated — a reminder never fires before the instant it was set for. Late is a reminder; early is a wrong answer. Whole seconds round-trip exactly, which is asserted, because an implementation that added a second unconditionally would otherwise pass. Three mutants for this round, three killed (round-up→truncate, round-up→unconditional-add, MaximumNArgs→ExactArgs). Thirty across the unit. Two fixes carry no dedicated test and it is worth being explicit rather than implying coverage: the `--format json` branch on `unremind` is a one-line output change with no server-free way to drive it, and the MCP `ref` description is prose the drift tests do not read — they assert an action is DOCUMENTED, not that a param's sentence lists it. * docs(reminders): the ack id is on the surface an agent polls, not only on the arm response CONVE-23 follow-through on the round-1 fix. Both agent-facing docs told a caller to acknowledge a reminder with the id "returned when you armed it" — true, and useless to the caller that matters: a poller reading next/ready never armed anything. The suggestion now carries reminder_id and `pad project next` prints the exact ack command, so the docs say that instead. The prose was written before the fix existed, which is exactly the case CONVE-23 is about: a change that makes an instruction stale without touching the file the instruction lives in. * test(reminders): bind the tick LOOP to the work, not just the pass (CONVE-19) Every other test in this file calls runReminderTick directly. That vouches for the component and says nothing about whether anything ever calls it — a tick that is never started is indistinguishable, from those tests, from one that is. It is the convention's exact case, and the failure I recorded on my own identity doc three times in one unit: I test the component and not the binding. Driven through the injectable tick channel so the assertion pins a SPECIFIC pass instead of racing a 30-second ticker, and polled to a bounded deadline so a loop that never runs FAILS rather than hanging the suite. Mutant: drop `s.runReminderTick()` from the select and this goes red while every direct-call test stays green. Killed. The idempotence leg exists because a second Start spawning a second loop would leave one running after Stop, making the BUG-842 drain invariant false for this sweeper specifically — the one property a copied lifecycle is most likely to get right by accident and least likely to be checked. The cmd/pad call site (cmd_server.go, alongside StartTokenReaper) stays verified by inspection: a source-scanning guard for it would be an instrument asserting facts about source, which is code with an adversary and not worth it for one line that sits in the middle of five identical neighbours. * fix(reminders): codex round 3 — a deferred reminder fired anyway, and the poll surface was unbounded **A re-arm mid-pass did not stop the fire.** The candidate scan selects an id; before the UPDATE runs, a `--rearm` can move that reminder into the future. Re-arm clears `fired_at`, so a predicate checking only `fired_at IS NULL` still matched — the pass fired a reminder the user had just deferred and emitted its event. The re-arm cannot undo that: it can clear the mark, but the event is already on the outbox and at-least-once means a consumer has seen it. The fire UPDATE now revalidates `remind_at <= nowTS` against the SAME nowTS the candidate scan used. Same-value deliberately: the arbiter and the scan must agree about when this pass is, or a reminder could pass one and fail the other for no reason but clock drift inside a single pass. **The poll surface was unbounded.** Every fired-and-unacknowledged reminder was loaded and turned into a suggestion prepended to a list that is otherwise capped at three, so a workspace with five hundred unacknowledged reminders returned five hundred suggestions — in the dashboard response, the hottest read in the product, growing until somebody acknowledged them. Two bounds, because they are two different guarantees: the query takes a window (default 50, oldest-fired first, so it holds what has waited longest), and the prepended suggestions are capped at 5 so `suggested_next` stays a recommendation rather than a second inbox. The full set stays addressable in `pending_reminders`. Truncation is REPORTED as a boolean, not a count. A count would have to be post-visibility-filter to be true for the caller reading it, and the store cannot compute that — the filter runs per item, above. "There are more than you can see here" is the strongest claim the data supports, so it is the one made. Four mutants; two killed outright, two survived and were run down under CONVE-28: - **Uncapped suggestions survived because the fixture had ONE reminder** — capped and uncapped are the same list at n=1. That is the SECOND time a single-item fixture hid a count-or-order property in this file. Fixture now arms eight; it also asserts all eight remain in `pending_reminders`, so the cap is pinned to the recommendation and not to the data. - **Removing the SQL LIMIT survived, correctly, and the test comment now says so.** The Go slice cap bounds the PAYLOAD; the SQL LIMIT bounds the DATABASE'S work. Only the first is observable at this level — with the LIMIT gone the response is still bounded, while the query silently goes back to materialising every pending row before discarding most of them. That is a memory and I/O property with no assertion available here, so it is stated as a coverage boundary rather than papered over with a green that would not have measured it. * docs(reminders): the fire predicate arbitrates against two actors, not one CONVE-23 inside the file the round-3 fix touched. The comment described the UPDATE as an arbiter for concurrent TICKS, which is what it was written for and is why I did not re-read it when asked whether a user edit could race the pass. It now says what it actually defends against, and names the general shape: an arbiter is only an arbiter with respect to the writers it can see. * fix(reminders): codex round 4 — the round-3 bound recreated the round-1 starvation Round 3 bounded the poll surface. Round 4 caught what that bound did: the query took the first N rows and the dashboard then discarded the ones it could not show — hidden items, unauthorised items, completed items — so N such rows hide a visible reminder behind them indefinitely, with no continuation to reach it. That is the SAME defect I had removed from the fire path one round earlier, reintroduced in the read path within the hour. The general form is worth stating because I clearly did not hold it: **a bounded window is only safe when the discarding happens BEFORE the bound.** Filtering above a limit is a starvation every time, and it does not matter what the filter is for. Two halves, because the two filters are not the same kind of thing: **Visibility is now scoped IN SQL**, using the same collection-id / item-id sets every other dashboard section gets through `allItems` — the same three-way shape as ItemListParams, where holding both collection grants and item grants is an OR. Invisible rows no longer occupy the window at all, which is strictly better than filtering them out afterwards and is what the sibling sections have always done. **Terminality is paged**, because SQL cannot evaluate it — a collection's schema defines which statuses are terminal. The collector refills from the next page when a page comes back short, bounded by a max scan so a workspace full of completed items cannot turn a dashboard read into a table scan. The bound is 10x the window: the common shape fills on the first page, and the pathological shape terminates in a fixed number of indexed reads. Stopping at the scan bound reports truncation, which is honest — there may be more, and we did not look. The empty-scope case is a THIRD state that reads like the second: nil CollectionIDs means unrestricted, a non-nil EMPTY slice means this caller sees no collections. Without an explicit guard they collapse, because the switch matches none of its cases at length zero and adds no clause at all — so "nothing visible" would return the whole workspace. Three mutants, one survived: the empty-scope guard, because no dashboard-level test produces that state (callers that would are refused earlier by workspace access). Faithful mutant, missing test — it now has a direct one, with a sanity leg so a build returning nothing cannot pass it by accident. A guard for a state nothing exercises is exactly the one that rots. * fix(reminders): codex round 5 — the MCP action I shipped did not work over stdio **P1: local stdio MCP `remind` was unusable.** cmdhelp derives positionals by regex from a command's `Use` string, and `<instant>` inside `remind <ref> --remind-at <instant>` matched — it became a second REQUIRED positional, so dispatch failed with `missing required argument "instant"`. The action was advertised on a transport where it could not run. **The MCP catalog's own tests did not catch it, and the reason is the finding.** That suite builds its cmdhelp document BY HAND: I wrote `Args: mkArgs("ref")` in it, so the fixture agreed with what I meant rather than with what the CLI says. Five parity and drift tests passed against a document I authored to match my own intention — the "a test that agrees with whatever the table says is not a test of the table" shape, which the canonical-events test warns about in its own comment two packages away. The new test reads the REAL command tree via cmdhelp.Build, which is the only thing in this repo that can disagree with me about what the CLI declares. **P2: `pad project ready` withheld the ack handle** that `next` prints. Showing a fired reminder on the surface an agent polls while withholding the id it needs to retire it means the same entry comes back on every poll, forever. **P2: suggestions asserted a collection they did not have.** The orphan branch admits ANY collection — its own comment claimed it gated on tasks "mirroring the active-plan branch", and that comment was simply false — while the output hardcoded `Collection: "tasks"` and the reason said "Open task". Pre-existing for high-priority items since BUG-1082; my overdue bypass widened it to any overdue item, which is how it surfaced. Fixed by carrying the item's REAL collection rather than by narrowing the branch: narrowing would silently drop the non-task items this has surfaced for a year, and the defect is the mislabelling, not the inclusion. The false comment is replaced with what the code actually does. The first version of that test used an overdue IDEA and SKIPPED — ideas use `new`, and the branch requires `open` or an active status, so it never became a candidate. A test that cannot fire is a failed reconstruction, not a pass; the fixture is now a bug-like collection whose vocabulary contains `open`, which is the population the defect can actually reach. Three mutants, three killed. Forty-one across the unit. * fix(reminders): codex round 6 — reminders fired from soft-deleted workspaces **P1, and the only defect in this unit whose consequence leaves the process.** Workspace soft-delete deliberately keeps items for the 30-day restore window, so the candidate query's filter on the ITEM's deleted_at found nothing wrong — and the tick kept firing, emitting outbound webhook events for a workspace whose owner had deleted it, possibly while deleting their account. Both queries now join workspaces and require `w.deleted_at IS NULL`. Nothing is destroyed: a restored workspace resumes firing, which the test asserts, because "stops firing" and "is destroyed" are very different answers to someone who restores a workspace and only one of them is right. That test first failed for the WRONG REASON and the fixture was at fault: it counted every outbox row in the workspace, and item creation writes its own, so the assertion was satisfiable by the fixture itself and discriminated nothing. Scoped to the reminder event type. **`Use: "remind <ref>"` declared a requirement the command contradicts.** cmdhelp derives the machine-readable arg spec from that string, and `--rearm` takes no ref — so the published contract said "required" for something optional. The requirement is CONDITIONAL, which cmdhelp cannot express, so the honest declaration is `[ref]` plus the explicit check that names both call shapes. The round-5 test grew a `required` column, which is what makes this observable at all: asserting only the arg NAMES would have passed. **The pad_item tool description omitted both new actions.** The params were declared and the actions dispatched, but the prose an agent reads to decide what a tool can do did not mention them — discoverable only by someone who already knew to look. It now describes both, including the two things an agent gets wrong: remind_at is an instant, and nothing but an explicit ack retires a fired reminder. Three mutants, three killed. Forty-four across the unit. * fix(reminders): codex round 7 — one predicate for the scan and the arbiter Third instance of one class, so this fixes the SHAPE rather than the instance. The class: the candidate scan filters on something the fire transaction does not revalidate, so a change committed between them fires a reminder that no longer qualifies. Round 3 was a re-armed instant. Round 1's soft-deleted item was the same thing caught from the other side. Round 7 is a workspace deleted between the scan and the fire — the round-6 fix added the condition to the SCAN only, and the arbiter went on not knowing about it. Fixing those one at a time is what let the third happen. `reminderFireable` is now a single string that both sites reference: the scan asks it and the fire UPDATE re-asks it, so they cannot disagree, and a fourth condition is one edit in one place rather than two edits someone has to remember are paired. Written as a correlated EXISTS on item_reminders.item_id rather than a JOIN precisely so the identical text is valid in both a SELECT and an UPDATE, and the scan drops its table alias so the two uses are the same characters. What deliberately stays outside it: `fired_at IS NULL` and `remind_at <= ?` live on the reminder row itself, are already spelled identically at both sites, and folding them in would need a parameter order the shared form cannot express. Said in the comment so the omission reads as a decision. Both directions are now tested at the arbiter — a workspace deleted mid-pass and an item deleted mid-pass — because the item case previously relied on the item load coming back nil, and someone simplifying the EXISTS down to the workspace check alone would otherwise still see green. Three mutants, three killed: the arbiter dropping the shared predicate, and the predicate dropping each of its two halves. Forty-seven across the unit. * fix(reminders): codex round 8 — workspace export silently dropped every reminder WorkspaceExport is a hand-maintained field list, so a new table joins it only if someone remembers. Reminders did not: a backup/restore, or a SQLite→Postgres migration via `pad db migrate-to-pg`, dropped every pending reminder with nothing in the destination to show anything had gone. The line that list has always drawn is item-scoped workspace CONTENT (comments, links, versions — exported) versus per-user state (stars, watches — not). A reminder has no user column and hangs off an item, which puts it on the exported side. Stating the rule rather than just adding the field, because the next person adding a table needs to know which side they are on. LIFECYCLE MARKS ARE CARRIED, not reset. A fired-and-unacknowledged reminder is still owed to whoever armed it, so it arrives pending; an armed one whose instant has passed fires once on the destination's first tick, which is what would have happened had the workspace never moved. Re-arming everything on import would invent a schedule the user did not set. NULL rather than empty string for the unset marks — the lifecycle is defined by NULL-ness, and "" would make a never-fired reminder read as fired at "". TestMigratedTablesCoversTheExport caught the second half, which I would have missed: `pad db migrate-to-pg`'s NUL preflight decides what to REFUSE on from MigratedTables, so a table the migration copies and the preflight does not know about is a gap in exactly the guard that exists to prevent one. Added there too, with the reason it can never actually fire — every column is machine-produced, so it is listed for coverage rather than expectation — and the "six tables" prose it falsified is now seven. Two mutants, two killed: export dropping the block, and import discarding the marks. Forty-nine across the unit. * test(reminders): state the fire-path invariant and pin it from the invariant The lead's read on why rounds 4 and 7 were the same class: the fire path had no stated invariant, so each fix defended an instance. This states it, and derives the pin from the paragraph rather than from the bug history. THE INVARIANT: the candidate scan is a hint and may be assumed to prove nothing. Every condition that made a row a candidate is re-asserted inside the transaction that marks it fired, in the same statement that does the marking, so checking and writing are one atomic act. Worded as "the scan proves nothing" rather than as a list on purpose — a list invites the next person to add a condition to the scan and stop, which is exactly what happened four times here. TestFirePathInvariant is the pin: one table, one row per scan-side condition, each invalidating that condition in the window between the scan and the fire and asserting the same three things — nothing fires, no event leaves, the reminder is not consumed. The earlier per-defect tests are folded in as rows; they said the same thing one instance at a time, which is how four of these shipped. Adding a fifth condition to the scan without a row here should feel like an omission. It carries a positive control, because four cases that all assert nothing happens would pass against a build that never fires at all. The matrix immediately falsified a claim in the paragraph I had just written. I wrote that the item load inside the transaction is "for the payload, not for the check"; removing the item half of reminderFireable alone changes no observable behaviour, because the load then returns nil and the deferred rollback undoes the write. Item liveness is defended TWICE and a single-mutant experiment cannot say which guard is carrying it — removing both is what kills the test. Both are kept, the predicate is named as primary (the row never matches, so no write happens at all), and the asymmetry is stated: workspace liveness has no second line, which is why dropping ITS half does fail the pin. Six mutants: five singles plus the pair. Five killed alone; the item single survives by design and is documented as such rather than left as an unexplained green. Fifty-five across the unit. * fix(reminders): codex round 9 — one legacy row could hide every reminder **P1: items.item_number is NULLABLE and I scanned it into an int.** Migration 006 added the column to existing rows, so a pre-numbering item still carries NULL — and scanning NULL into an int fails the Scan, which fails the QUERY, which degrades the whole pending-reminder section. One old row, and the feature is dark for everyone in that workspace. ListWatchesForUser, which this query was modelled on, uses sql.NullInt64 for exactly this column. I copied its shape and dropped the part that handles the column's actual nullability — the same way of being wrong as the round-5 cmdhelp fixture: borrowing a form without borrowing what it knows. The legacy row now carries no ref rather than a fabricated "PREFIX-0", which would name a different item. **P1: export shipped reminders that import could only discard.** The items section filters on deleted_at IS NULL, so a soft-deleted item is not in the bundle and its reminder can never be reunited with it. My comment claimed the item_links rationale — round-trip the raw graph so a restore reunites them — which is true for links and false here, because links keep soft-deleted endpoints in the bundle and items do not. A link is a row ABOUT two items; a reminder whose item is absent is a dangling schedule. **P2: import wrote remind_at raw.** Import is a writer, and a bundle is not necessarily one this server produced — hand-edited, or from another instance. A local offset or a bare date would land in the one column every comparison downstream treats as a UTC instant, firing early, late, or never. It now normalizes like every other door. An unparseable value is SKIPPED with a warning rather than failing the restore, matching the lenient import-side precedent already in this file, and the raw value's LENGTH is logged rather than its content. Three mutants, three killed; two needed rewriting because the single-line form did not compile — reverting the nullable scan also requires reverting the render, and dropping the normalization orphans a variable. PROCESS FAULT, recorded because it makes this round's findings weaker than they look: I edited the tree while this review was reading it — committed the invariant work and ran five mutation experiments, which write and restore source, over the same files. A review binds to the tree it read and I moved it underneath. Every finding above was re-verified against the current tree before being acted on, and the next round runs with no concurrent edits. * fix(reminders): codex round 10 — one orphaned item aborted a whole restore An ORPHANED item — one whose collection is missing from the bundle — still gets an itemMap entry. It has to: the entry is written before the skip because parent resolution inside the same loop reads the map for items it has not reached yet. So `itemMap[x] != ""` is satisfied by an id that names no row, and inserting a foreign key to it fails (SQLite enforces FKs here via the DSN's `_pragma=foreign_keys(on)`; Postgres always does). The pre-existing mapping is the sharp edge. The aggravating half was mine: this loop treated a failed reminder insert as FATAL, where item_links and item_versions both skip, so one orphaned item carrying a reminder rolled back an entire 900-item workspace restore. A reminder is the least critical thing in a bundle and it had the strictest failure handling in the file. Both halves fixed: the loop gates on items that actually landed, and a failed insert warns and skips like its siblings. TWO GUARDS THAT ONLY DIE TOGETHER, and this is measured rather than assumed. Reverting either alone leaves the test green — with the map gate restored the skip survives the FK failure, and with the fatal return restored the gate means the insert never fails. Removing both is what fails it. They are kept as a pair because they defend the same failure at different depths (prevent the bad write / survive a bad write arriving some other way), and the pair is recorded in the code so a future reader does not delete one as dead after watching its mutant survive. Second time this shape appeared today; the first was item liveness on the fire path. The bundle in the test is hand-built, because ExportWorkspace cannot produce an orphan — which is the reason it needed a test. That shape only arrives from a hand-edited or foreign bundle, and surviving those is what import is for. Three mutants: two singles that survive by design, plus the pair that kills. Sixty-one across the unit. * fix(reminders): codex round 11 — four contract slips, one of them another unit's **suggested_next returned up to eight entries against a cap of three.** Round 3 prepended reminders PAST the list's own cap, reasoning they should not compete for slots. Every consumer — the web dashboard, `pad project next`, `pad project ready` — is written for three. Worse, it silently falsified a decision recorded elsewhere: BootstrapDashboard deliberately has no suggested_next_overflow_count BECAUSE this list is capped at three upstream, and its comment names raising that cap as the moment to add one. My change made another unit's reasoning wrong in a file I never opened. The combined list is now trimmed back to three, reminders still leading — a reminder can push a task suggestion out, which is the right way round, and the full set stays addressable in pending_reminders. My first version of that trim used `limit`, which is REASSIGNED above to len(candidates) — so on a workspace whose only entries are reminders it would have truncated to zero, killing precisely the case the surface exists for. Caught by reading the surrounding lines before running anything; it has its own test now. **pending_reminders was uncapped in the bootstrap projection.** BootstrapDashboard embeds *DashboardResponse, so every new field joins the boot payload automatically — here, a window of up to 50, which is the budget PLAN-1410 spent a unit trimming. Capped at 5 with an overflow count, under its own constant rather than borrowing bootstrapAttentionCap: they answer different questions and a future change to one must not silently move the other. **Truncation was reported from the wrong question.** The collector used the store's `more` flag, which answers "is there another PAGE", not "did I read all of THIS one" — so a window filling part way through the final page reported that the caller had seen everything while unread rows sat behind the fill point. The paging bounds are now injectable so the case is testable at all: building it with a window of 50 needs ~75 rows in a specific pattern, with a window of 3 it is four. **Import accepted acked-without-fired**, which is not one of the lifecycle's three states. Such a row fires, is excluded from the pending surface because it is already acked, and can never be acknowledged because AckReminder requires acked_at IS NULL — an event emitted into permanent invisibility. The acknowledgement is dropped and the schedule kept, since an ack of something that never fired means nothing. Five mutants, five killed (one rewritten — removing the flag orphans a variable). Sixty-six across the unit. * fix(reminders): codex round 12 — a read is not a hold; scope the arm; ack from the ack Four P2s from round 12 (two independent runs, both landing on the same line of the fire path), each closed at the layer where it lives: - fireOneReminder pins the item and workspace rows FOR NO KEY UPDATE on Postgres before the arbiter UPDATE. reminderFireable re-asserted liveness at the predicate's instant and nothing held it to the commit instant; under READ COMMITTED an archival could commit in between and the event left the process about a deleted resource. Same idiom and same lock strength as CreateAttachmentForLiveItem; SQLite is excluded by its BEGIN IMMEDIATE, not skipped for convenience. Two PG-only pins verify "blocked" in pg_stat_activity, not by elapsed time; the pin-removed mutant fails both. - CreateReminder asserts "live item of THIS workspace" in the INSERT's own SELECT and returns ErrReminderItemGone otherwise. The table had an FK and no same-workspace constraint; a mismatched pair fed another workspace's title to this one's dashboard and webhooks. Handler maps it to 404. - AckReminder matches every fired row (COALESCE keeps the first ack, updated_at moves only when acked_at does), so a no-match means exactly "not fired at the instant of the ack". The handler no longer decides 409-vs-200 from the row it read before the UPDATE. - The invariant paragraph gains its missing sentence: "at that instant" means the commit instant, and the pin is what makes the predicate's instant and the commit instant the same one. Round-12 caveat carried: both runs were static reads (sandbox blocked Go's build cache), so "four" is a floor, not a measurement. Refs IDEA-2641 * fix(reminders): codex round 13 — a reminder's workspace must agree with its item's, at every read Every reader scoped by r.workspace_id and then joined the item without asserting the two agree. No door writes a disagreeing row today (CreateReminder derives the pair from the item; import maps within the workspace), and the table has nothing that forbids one — so a hand-edited bundle, a future move door, or a direct write would carry one workspace's item into another's dashboard, export, and webhooks. The identity goes into reminderFireable (scan + arbiter), the Postgres row pin, ListPendingReminders and the export query. One test writes the row raw — the only way one can exist — and asserts it is inert at each site; the predicate-removed mutant scans and fires it. Refs IDEA-2641 * fix(reminders): codex round 14 — the by-id and by-item reads assert the same identity as every other read GetReminder scoped by the row's own workspace_id and ListRemindersForItem by item_id alone, so a row whose two columns disagree — the class rounds 12 and 13 closed at the scan, the arbiter, the pin, the pending surface and the export — was still readable through the two reads that reach a single row. reminderOwned is that identity on its own, without the liveness half those two reads must not have (a fired reminder on an archived item is history worth showing). The write paths reach a row only through GetReminder, so scoping it scopes them; a row no door can write needs no door to delete it. ListRemindersForItem now takes the workspace its caller already resolved the item in. The raw-row test asserts both reads refuse the row from both sides; the reminderOwned-removed mutant surfaces it through GetReminder. Refs IDEA-2641 * fix(reminders): codex round 16 — an archived item's reminders are readable, and its verbs say "archived" The doors resolved the item live. Listing an archived item's reminders answered 409 from a GET, and ack/re-arm/delete answered a bare 404 for a reminder that exists on an item that exists — while the store, since round 14, deliberately keeps that history readable. The API already has a posture for archived items: GET reads them, mutations answer 409 "archived … restore it before editing" (writeItemResolveError). The list now follows handleGetItem; the lifecycle verbs load the item include-deleted, run the visibility check first, and then answer the same 409 every other item mutation does. One test walks archive → list 200 / ack 409 / arm 409 → restore → ack 200 on the same rows. Refs IDEA-2641 * fix(reminders): codex round 17 — one suggestion per item, the archived 409 by slug, and the door courtesy named Three findings on the server pass. (1) An item that was both a fired reminder and an ordinary candidate appeared in suggested_next twice; the ordinary entry is dropped, the reminder entry (which carries the ack id) stays, and two reminders on one item remain two entries. (2) Round 16's 409 for an archived item's reminder was written by re-resolving item.Ref, which is derived and empty for a legacy item with no item_number — so the class most likely to be legacy fell through to a bare 404. The slug is handed over instead. (3) The archived check in resolveReminderForWrite is check-then-write, and an archive landing in between lets the verb through: accepted and documented — it is the posture of every item mutation here (UpdateItem's UPDATE has no liveness clause), the outcome is benign, and putting liveness in AckReminder's WHERE would re-create the no-match ambiguity round 12 removed. Refs IDEA-2641	2026-09-04 11:36:11 -04:00
xarmian	178b6b5010	fix(server,store): two more from codex rounds 3 and 4 (BUG-2810) ... **The import repair could silently change what gets imported.** It decodes into map[string]any, where a repeated object member keeps only the LAST value. The TYPED decode that runs next does not agree: encoding/json unmarshals members in order into the same struct field, so two `"workspace"` objects MERGE there and collapse here. A body with duplicate members would therefore import differently with --repair-nul than without, which is outside what a flag by that name may do. It now DECLINES such a body: returns it untouched, lets the gate judge it exactly as it would without the flag, and says why in the refusal — "the payload repeats the member X, and repairing it would change which value is imported". Detection is a token walk, because a decode is what loses the information: by the time there is a map the duplicate is gone. The detector's own test carries the false positive that matters — the same member name in SIBLING objects is not a duplicate, and a single shared set of names would decline every real export, since items all carry `id`, `title`, `slug`. Rewriting such a body faithfully wants a token-preserving pass, which is BUG-2812's token-walk and not a rider on this. A real export cannot contain duplicate members (json.Marshal does not emit them), so declining costs nothing an operator meets by accident. The tally now owns the repair — decodeJSONRepairingNUL takes it and calls Apply — so the count and the declined reason come back through one object instead of a return value a caller has to remember to record. That is the same mistake this branch already made once, when the JSON path dropped the count and the header reported 0 for an import that had rewritten a value. **A row the repair could not address was reported as a failure.** A NUL in a key column the list does not protect, on a row whose violation is elsewhere, makes the address unbindable: Layer A inspects every bound parameter, including a WHERE clause's, so the lookup is refused before SQLite is asked to find the row. It landed in Failed carrying "invalid text parameter: parameter 2" — the same information phrased as a fault in the repair rather than a property of the row. Now detected up front and reported as a skip with the reason, alongside the two skips that already existed. **One finding NOT fixed, deliberately, and recorded instead.** Round 3 raised that the scan misses a NUL in a value shadowed by a LITERAL duplicate key, so such a row passes the migrate-to-pg preflight and then fails during the copy — the exact failure the preflight replaces, surviving for one shape. That is textguard.KnownGaps: a blind spot every layer shares on purpose, which DOC-2823 forbids closing in one layer alone, because layers disagreeing about one value is the defect this cluster is made of. So it is named in ScanNUL's doc comment, in the preflight's, and in docs/backup.md for the operator, and TestScanNULInheritsTheRecordedKnownGaps pins the miss and FAILS when it stops being one — the notification that BUG-2812 has landed and those three prose sites need updating. The consequence is recorded on BUG-2812's trail. Round 2's single finding was refuted rather than fixed: it predicted TestRepairFlagReachesTheNestedAndObliqueForms would fail, on a mechanism that describes the raw-byte scanner this branch had already replaced. The test passes; the outer decode resolves the oblique spelling before the walk sees it.	2026-09-02 16:24:25 +00:00
xarmian	49bd342e4c	fix(store,server,cli): three defects from codex round 1 (BUG-2810) ... **The import flag could not repair the column it exists for.** `--repair-nul` scanned the RAW body for a live escape, which is right for a value the gate reads at the top level and wrong for the one that actually matters. An item's `fields` blob travels through an export as a STRING: a NUL escape in the stored blob marshals into the body with a DOUBLED backslash, which a raw scan must leave alone because at that layer it is literal text — while the gate refuses it anyway, since it decodes the body and re-parses that string as the document it is. So the repair now walks the DECODED body with the same classing bodyDecodesNUL uses, one verb changed: where the gate asks textguard whether a value decodes to a NUL, this asks textguard to repair it. Two walks of one shape in one package is a real risk, and the mitigation is that they are measured against the same corpus in both directions rather than reviewed for similarity — TestBodyRepairMirrorsTheGateOverTheCorpus drives every case through the body shape and asserts refused-becomes-accepted and accepted-stays-byte-identical. Two consequences worth stating. The walk also reaches the OBLIQUE spelling — the backslash written as its own escape, so the six characters never appear in the raw bytes at all — which the scanner could not, so the test that pinned that limit is replaced by one asserting the capability. And re-encoding is now possible, so it is bounded: UseNumber, so an integer wider than float64 is not silently re-emitted in scientific notation; SetEscapeHTML(false); and a body with nothing to repair is returned byte-identical rather than round-tripped. The mutation that removes UseNumber turns 9007199254740993 into ...992, and a test says so. The header is now X-Pad-Repaired-NUL-Values, because at the decoded layer an escape is not a thing that exists any more and one nested document may have carried several. **The scan could not run on the databases it exists for.** Several protected tables carry a NULLABLE workspace_id — activities, api_tokens, mcp_audit_log — and the scan selected it into a plain *string, which fails with "converting NULL to string is unsupported" and takes the scan, the repair and the migrate-to-pg preflight down with it. Every column is now scanned as sql.NullString: SQLite also permits NULL in a declared PRIMARY KEY that is neither INTEGER PRIMARY KEY nor NOT NULL, which no other engine does, and a NULL key cannot address a row for an UPDATE — such rows are reported and skipped with the reason rather than handed a WHERE that matches nothing. Verified against the unfixed code: the scan returned `scan activities.actor row: sql: Scan error ... converting NULL to string`. It needed a VIOLATING row in such a table, which is why every fixture that planted its rows in `items` missed it. **--force by accident.** The repair skipped the running-server check whenever --from was given — and the most natural --from an operator types is the path `pad db scan-nul` just printed, which IS the live database. The check is now on the resolved path (Abs + EvalSymlinks, so a symlinked data directory or a relative path still matches), and a --from naming an unrelated backup stays unguarded, which is correct: nothing is writing it. The ordering moved with it. `store.New` runs pending migrations, so the refusal now happens BEFORE the database is opened; opening first and refusing second made the guard arrive after the thing it guards against.	2026-09-02 15:15:11 +00:00
xarmian	63da2f4f5f	feat(store,server,cli): count and repair the legacy NUL population (BUG-2810) ... Layers A and B stop the value being written. Neither makes a row that already carries one go away, and BUG-2810's filing is what that costs: an affected workspace exports with a 200 and re-imports with a 400, so a self-hoster restoring their own backup is blocked with no path forward in the product, and `pad db migrate-to-pg` fails partway through the copy against PostgreSQL's jsonb parser rather than up front. This is DOC-2823's S3, on Dave's day-54 rulings: U+FFFD as the replacement, repair standalone only with a migrate-to-pg preflight that refuses and prints the command, `--repair-nul` on import shipping default-strict. ONE REPAIR, beside the one predicate. textguard.Repair lives next to ParameterRefused because four layers that agree about what is REFUSED and disagree about what a repair PRODUCES is this bug family arriving one step later. Its contract is a property over the same corpus, in both directions: every refused value becomes one all four layers accept, and every accepted value comes back IDENTICAL. The second half is the load-bearing one — a repair that tidies values nobody complained about rewrites `{"a":"x\\u0000y"}`, six literal characters after a doubled backslash, and corrupts it. The JSON arm is a string-literal SCANNER, not decode-walk-remarshal, which is what the recon write-up proposed before it was written. Re-marshalling changes four things nobody asked to change — object key order, insignificant whitespace, integers wider than float64, HTML-ish characters — and silently drops one of a document's LITERAL duplicate keys, which is a gap BUG-2812 owns and the last thing a repair should do. Scanning copies every byte it does not deliberately rewrite, so an untouched document is byte-identical without that having to be argued. A substring replace is not equivalent and the test that proves it took a mutation to find: a doubled-backslash literal ALONE never reaches the scanner, so the discriminating fixture is one document carrying a live escape AND a literal. THE COUNT IS COMPUTED IN GO. Measured on the read path in this worktree: a row planted with `bad<NUL>name` reads back into a Go string with all 8 bytes and the NUL intact, while `length(name)` in the same database answers 3. TASK-2824 found that C-truncation and concluded no DB-side REPAIR could be trusted; the same measurement on the read path says no DB-side COUNT can be either. SQL narrows — `instr(col, char(0))`, plus the escape prefix on JSON-classed columns, which is textguard's own pre-filter — and never decides. The decision stays ParameterRefused with isJSON from the shared 86-column list, i.e. Layer B's classing. Row addressing is read from the live schema rather than a hand-kept map: 39 tables carry protected columns, one (item_wiki_links) declares no primary key and is addressed by rowid, two have composite keys, and five have a single key that is not `id`. The repair checks RowsAffected because an address that stopped selecting its row would otherwise commit an UPDATE that touched nothing and report it as repaired — the one failure an operator cannot see in the output. `email_optouts(email)` is both a protected column and its own primary key. Repairing it changes the row's identity and can collide with an existing row, which in that table means somebody starts receiving mail again. It is reported and skipped, with the reason. The import flag is NOT an exemption from the gate. `--repair-nul` buys the body one repair attempt and then runs the same `bodyDecodesNUL` on the repaired bytes, which still decides — a decode path that skipped the check is the door BUG-2803 spent thirty rounds closing, on the endpoint carrying the largest attacker-controlled body in the product. Only the ESCAPE form is repaired: a raw NUL byte makes the document invalid JSON, and widening what parses is not this flag's job. Both doors are covered, JSON and tar.gz, because giving them different answers is how one of them keeps being forgotten. Postgres is settled with evidence rather than sent up as a ruling: it cannot hold either defect (22021, 22P05) and the four-way differential test already pins that, so the scan reports not-applicable WITH the reason rather than returning a zero a reader could mistake for a clean database. Spellings settled here, per the dispatch: `pad db scan-nul` and `pad db repair-nul` as siblings rather than `repair --nul`, matching `migrate-to-pg`'s hyphenated compound — a repair verb that errors when given no flag is a worse shape, and there is no second repair to share it with. scan-nul IS the dry run, so repair-nul grows no --dry-run. It refuses while the server is running unless --force, on the `pad db restore` precedent: the report is a claim about a database, and one somebody else is concurrently writing makes it a claim about a moment that has passed. docs/backup.md's section on this is rewritten. It still said the rule lives in the binary and not the database, which S2 made false, and it pointed at this item for a preflight and a repair that now exist. Its import examples also showed `pad workspace import < file`, which has never worked — the file is an argument. Closes BUG-2810.	2026-09-02 14:53:31 +00:00
xarmian	213142c4b5	fix(store,server): honest refusal figures and a throttled oversized scan (BUG-2827) ... Codex round 3. Two of three findings acted on, one rejected with an invariant test in place of the change it asked for. REFUSAL FIGURES SAID SOMETHING FALSE. The store refuses on two different measurements against two different limits — the member content before marshalling, against the write cap, and the row exactly as stored, against the claim ceiling — and both reported the number as "a %d-byte payload". For the first that is untrue: projectedBulkPayloadBytes is explicitly a lower bound, so the message named a size the payload did not have. A caller seeing two different numbers for one mutation had no way to reconcile them. OversizedOutboxPayloadError now carries what it measured, and both the error and the 413 say so. THE DIAGNOSTIC WAS THE MOST EXPENSIVE THING THE DRAIN DID, and it was most expensive when it found nothing. OversizedPendingOutbox has a non-sargable size predicate and no index to help it, so an empty result means evaluating octet_length over every pending row — on Postgres, detoasting and serializing each JSONB payload — and it ran every 5s tick. Now throttled to once every 5 minutes. Latency is the cheap thing to spend here: the rows it reports are permanently unclaimable and sit until the 7-day retention takes them, so a five-minute alarm delay changes no decision anyone makes about them. The first tick after a restart still scans, so an existing oversized row is reported promptly. REJECTED: that the claim needs to revalidate size, because a concurrent scrub could grow a payload between candidate selection and the claim UPDATE. It cannot. scrubOutboxRowTx is the only UPDATE of payload in the tree and it removes keys and re-marshals compactly, so a rewrite is strictly smaller — and shrinking is harmless, since a row judged claimable stays claimable. That is load-bearing for the claim needing no revalidation, so it is now stated at the function and pinned by TestScrubOnlyEverShrinksAPayload rather than left as an assumption for the next person adding a payload rewrite to break silently. The throttle test found its own gap on the way in. Written first against the helper, it stayed green when the call site was mutated to `if true` — a helper nothing calls is still correct in isolation. TestOutboxDrainTickConsultsTheThrottle covers the call site through the stamp the tick leaves behind. Also caught by the gate rather than by review: the field carrying the throttle clock landed on outboxDrainSettings as well as outboxDrainConfig, because the edit matched a line both structs have. Tests passed with both; lint named the dead one. Gates: gofmt clean, golangci-lint 0 issues, full SQLite suite exit 0, full Postgres suite exit 0. Mutants: growing the written payload kills TestScrubOnlyEverShrinksAPayload, removing the throttle call site kills TestOutboxDrainTickConsultsTheThrottle. One mutant discarded as unfaithful — corrupting the payload BEFORE the compare-and-swap is neutralised by the retry, which re-reads and redoes the work correctly, so it tests the retry rather than the invariant. Claude-Session: https://claude.ai/code/session_01XLtX4dbjBpApbAv3SuBcTm	2026-09-02 01:41:47 +00:00
xarmian	5b12d5eeeb	fix(store,server): bound the outbox's unbounded reads and writes (BUG-2827) ... item.bulk_updated marshals every cascaded member body into ONE event_outbox row, and nothing bounded either the row or the drain's reading of it. The v1 doc called the size deliberately unbounded and named a follow-up condition; measurement met it. MEASURED FIRST, against an 8,434-item instance: widest wiki-title cascade 23 members / 175.6 KiB of bodies widest wiki-ref cascade 50 members / 365.5 KiB worst option rename 4,429 members / 7.34 MiB (tasks.status="done") marshal ratio ~1.46x (item.updated payload mean 3,518 B against an item body mean of 2,416 B) So renaming one status option on that instance emits ~11 MiB in a single row today, from a user clicking rename in the collection editor. That kills the obvious design: a write cap tight enough to bound the drain's memory refuses routine work, and one loose enough for routine work bounds nothing. The two therefore became two numbers. WRITE CAP (128 MiB) in writeOutboxTx, the single INSERT INTO event_outbox in the tree, so all seven emit paths inherit it rather than an enumerated site list. Refusing FAILS the mutation: the hop bound in the same function drops the event and keeps the mutation, correctly, because there the cascade is what is illegitimate, while here the mutation and the event are the same fact. 128 rather than 64 MiB so it clears MaxItemRenameCascadeBytes — a cascade squeaking under that 64 MiB bound marshals to ~96 MiB, and a 64 MiB cap here would let the vaguer refusal preempt rename_cascade_too_large on the very renames that bound describes. Surfaces as 413 event_payload_too_large, following that precedent rather than inventing a second spelling. CLAIM BUDGET (64 MiB per pass), spent by the primary candidates AND by batch siblings. Spending it on siblings knowingly relaxes "batches are claimed whole": read literally that rule makes the budget bypassable by construction, since one large batch is an unbounded read no row limit touches, and groupOutboxDeliveries already defines the split. The first candidate is always taken whatever it costs, so the bound meant to keep the drain alive cannot starve a row instead. CLAIM CEILING (2x the write cap), and it is NOT the write cap — a distinction the Postgres leg had to teach. The cap measures the Go bytes json.Marshal produced; the claim measures what the driver hands to Scan, which on Postgres is the JSONB ::text rendering, one space inserted after every colon and comma. A 40,000-byte payload reads back as 40,001. Thresholded at the same number, a payload written at exactly the cap was admitted by the guard and then permanently excluded by the claim: delivered to nobody, reported as nothing, reaped seven days later. TestARowWrittenAtTheCapIsStillClaimable fails on Postgres and passes on SQLite against the 1x version. Rows above the ceiling are excluded IN THE PREDICATE, not filtered in Go — filtering in Go leaves them occupying candidate slots and starves everything behind them, which is the same jam wearing different clothes. They are logged every tick and left pending for the existing 7-day undispatched retention, not stamped dispatched_at, which would record that an event went out when it did not. SCRUB. ScrubOutboxUserRefsTx collected every LIKE-matching payload at once — the same unbounded read through a different door. Now batched by a keyset cursor on id. READ FULLY THEN WRITE is preserved PER BATCH, and the cursor keeps the per-row UPDATEs in ascending id order across batches, so batching does not quietly reintroduce the BUG-2409 deadlock it was written to avoid. Prose sweep: MaxItemRenameCascadeBytes' comment asserted this vector had no bound, and emitBulkItemEventTx's asserted the payload was unbounded by decision. Both now say what is true, the second keeping its original reasoning because it still explains what the bound does NOT do. Mutation matrix, each mutant compiled and run unfiltered: removing the row-cap check, > to >=, removing the budget break, removing always-take-one, removing the SQL size exclusion, removing the sibling budget, removing the scrub cursor advance, and 2x to 1x — all die, the last on Postgres only. One survivor recorded as unfaithful rather than as a gap: setting the scrub cursor to the first row read still advances monotonically over an ascending id > cursor query, so termination holds and only the pass count degrades. Not addressed, named rather than left to be found: a workspace that outgrows the write cap cannot shrink its own cascade, so the refusal leaves no recourse. The answer is chunking one bulk event across rows sharing a batch_id, which the drain's fold already supports. Claude-Session: https://claude.ai/code/session_01XLtX4dbjBpApbAv3SuBcTm	2026-09-01 23:15:05 +00:00
xarmian	ef792bdf35	fix(store,server,textguard): close codex round 1 on S1 — seven findings ... All seven verified before acting; two were production regressions I had introduced and one was a defect older than this branch. [High] The wrapper HID the optional connection interfaces. Measured: the raw modernc conn implements Pinger, SessionResetter and Validator; the wrapped one implemented none, so Ping succeeded without pinging, pooled connections stopped being reset, and dead connections stayed in the pool. Each is now forwarded, falling back to exactly what database/sql does for a conn lacking it. NamedValueChecker is forwarded too, or the wrapper silently narrowed the argument types pgx accepts. [High] Statement-level driver.ErrSkip was wrong. At the CONN level ErrSkip is the documented fallback signal; at the STATEMENT level database/sql propagates it, so a base statement without the context interface would have failed rather than degraded. Both now fall back to the positional form, refusing a NAMED argument rather than binding it by position. [High] Scalar JSON documents bypassed the check — and the database settles it: SELECT ('"a<escape>b"')::jsonb; ERROR: unsupported Unicode escape sequence A bare JSON string is a complete jsonb document. The object/array-only shape test is right for the HTTP gate's "is this a nested document" question and wrong for the store's "will a jsonb parser read this". Both now use a widened form, so the gate refuses it at the door instead of leaving it to the store. That widening broke the gate's independent oracle, which had been NARROWED in BUG-2803 round 27 to make the two walkers agree. The disagreement was real; the direction of the fix was not — nobody measured which one matched Postgres, and it was the oracle. Restored, with the measurement in the comment. Two implementations made to agree are not thereby correct. [High] The 400 mapping was not the single funnel I claimed. createItemChecked, bulk ops and the cross-workspace copy carry their own error envelopes by design. One shared CLASSIFIER now serves all of them; the envelopes stay different, the classification and wording do not. Copy keeps its retry-discouraging message deliberately (PLAN-2357 DR-13). [Medium] The over-refusal justification was FALSE as written. It claimed a JSON-shaped text value carrying a live escape "is a value Postgres would refuse the moment anything parsed it". Nothing parses a text column; Postgres stores it fine. The trade is now stated honestly, and pinned: textguard. StoreOverRefusals records the case, and a test fails when it is paid down. [Medium] Duplicate JSON keys are a real under-refusal, inherited from the shared predicate's map model. DOC-2823 requires Layer A NOT to fix it alone, so it is recorded in textguard.KnownGaps with a test that fails when it CLOSES — the signal that BUG-2812's token-walk landed. [Medium] Test weaknesses, and the fixes caught a real one. The census now sees ALTER TABLE ... ADD COLUMN. The store corpus leg now requires accepted cases to SUCCEED, which immediately exposed a case passing for the wrong reason: a non-JSON value written to the fields column failed on SQLite's own JSON parser, never reaching the guard. Full Go suite green on SQLite and Postgres 17; lint 0 issues. Claude-Session: https://claude.ai/code/session_01XLtX4dbjBpApbAv3SuBcTm	2026-09-01 12:54:19 +00:00
xarmian	3baf984b41	feat(server,store): the differential test's three live legs + the 400 mapping ... Completes S1's deliverable except Layer B, which stays dark until S2. THE THREE LEGS, all driving the SAME corpus: - HTTP gate (internal/server): each case enters as a request body, classed the way a request is — by KEY NAME, "fields" for JSON, "content" for text. - Layer A at the driver (internal/store): each case is a real write, classed by column. - Native Postgres (internal/store): each case is a real INSERT through the RAW pgx driver, no guard in the path, so what is measured is Postgres's own verdict rather than ours reflected back. All 15 cases agree across all three. That is the calibration result the design needed: the guard neither over-refuses (breaking writes Postgres accepts) nor under-refuses (leaving the dialect split BUG-2831 was about). The Postgres leg additionally asserts the SQLSTATE, because "some error occurred" would have been satisfied by a typo in the test's own SQL. THE 400 MAPPING is in writeInternalError — the single funnel every 500 already passes through, so one insertion covers every handler present and future. The item-title unit had to find the same error block three times in one function before a structural test caught the third; this avoids the enumeration entirely. 400 rather than 500 because the request is understood and will be refused identically on retry, and 400 specifically because it is what the gate answers for the SAME value refused at the door — two statuses for one rule would be the layers disagreeing in the response instead of the predicate. The residual is stated rather than hidden: a value can reach the store from something the SERVER composed (BUG-2814's re-emit population), and for those a 400 misattributes the fault. It is still better than 500, and if that case ever needs its own status it needs its own error type first. Mutation matrix extended: 3 more mutants, all killed — mapping removed (500 returns), mapping swallowing every error (the control leg), and the gate ceasing to delegate to the shared core, which is the layers-diverge scenario the corpus exists to catch. Full Go suite green on SQLite and Postgres 17; lint 0 issues. Claude-Session: https://claude.ai/code/session_01XLtX4dbjBpApbAv3SuBcTm	2026-09-01 12:06:08 +00:00
xarmian	ba1255881d	fix(server): refuse a decoded NUL in a JSON request body (BUG-2803) (#1220) ... * fix(server): refuse a decoded NUL in a JSON request body (BUG-2803) The body half of BUG-2782 (path) and BUG-2784 (query). A caller-supplied string reached a Postgres text parameter, Postgres refused it, and the handler answered 500 — the honest answer is 400. WHY THE TRANSPORT RULE CANNOT BE EXTENDED, which is the whole reason this is a different fix rather than a wider middleware. ValidateQuery works because a decoded query value is a substring of the raw query with ASCII substitutions: the bad byte in the raw text IS the bad byte in the value. That property fails for a JSON body — the reachable NUL arrives as the six-character escape, all ordinary ASCII — so no request middleware can find it without decoding the body, which is the handler's job. MECHANISM, each premise measured against encoding/json rather than reasoned about: raw NUL inside a string -> decode ERR (invalid character in string literal) raw NUL after the value -> decode ERR the escape in a value -> decodes to a string CONTAINING a NUL the escape in a KEY -> same a DOUBLED backslash -> decodes to literal text, NO NUL the uppercase spelling -> not a JSON escape at all So the escape is the only vector and its substring is a sound FAST PATH (absent -> no NUL possible), but not a sufficient test: a doubled backslash carries the same six characters and decodes to text. In this product that is not hypothetical — items and documents store markdown, and a document about JSON escapes is an ordinary thing to write. The exact step is json.Decoder.Token(), which returns DECODED strings, covers object keys and arbitrary nesting (an item's fields blob), and needs no knowledge of the destination type. NOT REFLECTION over the decoded value, the other obvious design: it sees []byte fields AFTER base64 decoding, so a body carrying legitimate binary ({"b":"AQAC"} -> bytes 01 00 02) would be refused for a NUL that is not text. A token walk sees the base64 characters. No request struct has such a field today (searched: []byte with a json tag in internal/server and internal/models, non-test — only models.YjsUpdate.UpdateData, which no handler decodes from a body); the token walk is chosen so adding one later cannot silently start rejecting valid requests. BUFFERING IS NOT A COST. json.Decoder.Decode already holds the whole top-level value in memory — refill accumulates into dec.buf and grows it by doubling (encoding/json/stream.go) — so streaming never avoided the copy. Measured on the 64 MiB workspace-import shape, total allocation: stream+Decode 354.7 MiB, ReadAll+Unmarshal 256.5 MiB, ReadAll+Decode 512.5 MiB. Peak heap is order-dependent and does not discriminate; the first run of that measurement showed a 0.77x peak win that vanished when the legs were swapped, so only the allocation figure is claimed. POPULATION, measured on Postgres 17 through the real router with a control leg on every endpoint (92 mutating routes enumerated via chi.Walk; 13 probed): before: 12 of 13 DOOR (control 201 / NUL 500, SQLSTATE 22021) after: 0 of 13 — every NUL leg 400, every control leg unchanged Confirmed doors: workspace name, collection name, item title, item content, item fields value, item title via PATCH, comment body, agent role name, view name, document title, webhook secret, workspace import. workspace-token name is UNMEASURED, not clean — its control leg 500s on an unrelated FK in this fixture. The other 79 routes are unprobed, not claimed clean; the completeness argument is structural instead, and enforced by a test rather than asserted. SECOND DEFECT, named rather than slipped in: the six handlers that decoded straight off r.Body had no http.MaxBytesReader either — the cap decodeJSON has always applied — so each was an unbounded body read. Routing them through decodeJSON closes that too. COMPATIBILITY: json.Unmarshal refuses trailing non-whitespace after the JSON value where Decode ignored it. Deliberate, same direction as this fix, and the only behaviour change beyond the refusal. Trailing whitespace still passes. An EMPTY body still returns a wrapped io.EOF, because handlers_playbooks.go reads errors.Is(err, io.EOF) as "no arguments supplied" — caught by TestPlaybookRunAcceptsEmptyBody, which is exactly the wiring a helper-level change is blind to. No call site changed for the refusal itself: all 65 decodeJSON callers already turn a decode error into a 400 carrying err.Error(). Release note: a NUL character in a JSON request body now returns 400 instead of 500 on Postgres deployments. Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN * fix(server): follow the NUL refusal into JSON-encoded string fields (BUG-2803) Codex round 1 on #1220: the check scanned ONE JSON layer, and several fields cross the wire as JSON-ENCODED STRINGS rather than nested objects — an item's fields, a collection's schema, a workspace's settings. The OUTER decode of {"fields":"{...}"} yields the inner document as literal text, in which the escape is still six ordinary characters and no NUL exists, so the single-layer token walk passed it. MEASURED on Postgres 17 with a control leg on each, after the single-layer check was already in place: item.fields as a JSON-encoded string 500 control 201 collection.schema as a string 500 control 201 workspace.settings as a string 500 control 201 The error is DIFFERENT from the rest of this family, which is why it is worth reading rather than assuming: insert collection: ERROR: unsupported Unicode escape sequence (SQLSTATE 22P05) 22P05, not the 22021 the path and query halves produce. The outer string is pure ASCII so it never trips the text-encoding check; this is Postgres's own JSON parser refusing the escape inside a document bound for jsonb, which cannot represent a NUL. After this change all three answer 400 with their control legs unchanged. THE FIX: when a decoded string is itself a complete JSON object or array — the class this API re-parses downstream — walk it too, to a depth bound of 8. Recursion terminates on its own (each level is a strict substring of the one above); the bound keeps a hostile body from buying many full re-parses, and AT the bound the body is refused rather than passed uninspected, since the escape is known to be present and the walk has stopped looking. WHAT THIS OVER-REFUSES, by design and pinned by a test: the rule is structural, not destination-typed, so a plain TEXT field whose ENTIRE value is a valid JSON document carrying the escape is refused too, even though its column would have stored it. Prose ABOUT a JSON escape does not parse as a bare document, so the case is narrow, and a value of that shape breaks any consumer that parses it. The destination-typed alternative — an allow-list of the fields that arrive JSON-encoded — is exactly correct and goes stale in silence, which is the failure mode ValidateQuery's comment rejects when it explains why per-site query validators could not be written. Tests: nested documents (fields/schema/settings/array/twice-encoded), with controls for ordinary content, a doubled backslash INSIDE the nested document, a string that starts like JSON but does not parse, and prose that merely mentions the escape; the over-refusal pinned as a decision rather than left as an accident; the depth bound; and a wiring leg through the real router on SQLite, where the write would otherwise SUCCEED so a green cannot be the database doing the work. Fixtures build their JSON-encoded strings with encoding/json rather than hand-written backslashes, since the escaping rules are the subject under test. All four new tests fail with the recursion removed. Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN * test(server): build the NUL-bearing timeline fixture through the store (BUG-2803) TestTimeline_NeverEmitsACursorItWouldRefuse built its fixture through the API on a premise its own comment stated: "a structured id comes from the item's fields blob, which nothing validates on write". BUG-2803 made that false — decodeJSON now refuses a body whose strings decode to a NUL, including one nested inside a JSON-encoded `fields` string — so the API can no longer produce the row and the test 400'd on its fixture. Repaired rather than deleted, because the DEFENCE it covers is still live: rows in this shape can predate the rule, and the store has no such check of its own, so a migration, an import or any future non-HTTP writer can still produce one. The timeline must keep refusing to hand out a cursor it would then reject. The fixture now writes the blob directly, injecting the six-character JSON escape rather than a raw NUL — the blob is JSON text and both backends reject a raw NUL in it; the NUL comes into existence when Go DECODES the blob, which is exactly how the timeline ends up with one inside an entry id. The test is not vacuous under the change: it asserts the NUL-bearing id took the positional fallback, so an injection that failed to produce a NUL fails the test rather than passing quietly. This is the CONVE-23 case — a change that falsifies existing prose owes a sweep for that prose. The stale sentence was found by the test failing, not by the sweep, which is the weaker of the two ways to find it. Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN * docs(server): correct the timeline comment BUG-2803 falsified (CONVE-23) The entryID fallback's comment said note and decision ids "come from the item's fields blob and nothing validates them on write". BUG-2803 made that half false: the HTTP API now refuses a request body whose strings decode to a NUL, including one nested inside a JSON-encoded `fields` string. The sentence was true when written and nothing in this branch's diff pointed at it. The fallback still has to exist, and the corrected comment says why: the STORE has no such check, so rows predating the rule — and anything writing a blob by another path, a migration, an import, a future non-HTTP writer — can still carry one. SWEPT AND DELIBERATELY LEFT: two nearby comments (handlers_timeline_id_collision_test.go, handlers_timeline_structured_test.go) also say "nothing validates them on write". Both are about id FORMAT and DUPLICATION — an imported artifact carrying a UUID-shaped id, a hand-written blob repeating one — and this change validates neither. In context those sentences remain true, so they are left alone rather than edited into noise. Sweep command: grep -rniE "nothing validates|not validated on write|no validation on write|unvalidated" --include=*.go internal/ cmd/ — six further hits, all about other subjects (github_pr raw writes, terminal schema keys, push payload format, decodeJSON's size bound). Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN * fix(server): scope the nested-NUL walk to JSON-encoded fields (BUG-2803) Codex round 2 on #1220. The nesting check from the previous commit recursed into ANY string that parsed as a JSON document, on the argument that a structural test beats a destination-typed one. That argument was wrong in a way I had written down as an accepted trade and should have weighed as a defect: a plain-text `content` value holding a JSON snippet that merely MENTIONS the escape was accepted before this branch, is stored in a text column that has no problem with it, and was newly refused — including on RE-IMPORT of an export carrying it. Refusing input the server itself produced is a worse failure than the door the unscoped recursion was closing. Measured before the fix: a workspace whose item content held such a snippet exported 200 and re-imported 400. The walk now descends only under keys whose STRING value is a JSON document something downstream re-parses: config, events, fields, metadata, phase_data, plan_overrides, schema, settings, tags, traits. WHY A LIST IS SAFE HERE, when ValidateQuery's comment rejects exactly this shape for query parameters: there the set of names is unbounded by design (parseItemListParams turns any unrecognised parameter into a field filter), so no list could be complete. Here the set is a closed property of the wire model — a field is JSON-encoded because a Go struct declares it as a string holding JSON — and TestJSONEncodedFieldKeysCoversTheModels derives it from internal/models and fails when a new one appears. The list cannot go stale in silence. Over-inclusion is the safe direction and the list takes it: a listed key that is not really JSON-encoded costs one parse attempt and can only refuse a complete JSON document carrying the escape, while a missing key reopens a door. `traits` is listed for that reason — it carries JSON but its declaration has no comment saying so, which is exactly how the derivation test would have missed it, so the test asserts coverage in one direction only and the list is allowed to be a superset. The walk also changed shape: decoding into `any` and walking the value, rather than a token stream, because key context is needed to know which subtree is JSON-encoded. The []byte reasoning is unchanged and still holds — decoding into `any` never produces a []byte, so a base64 field is seen as its ASCII text rather than as decoded bytes that might contain a legitimate 0x00. Tests: text fields carrying a JSON document are ACCEPTED (five keys), with a leg proving the same document under a JSON-encoded key is still refused, so the pair differs only in the key; the derivation test; and the depth-bound fixture now nests under a JSON-encoded key at every level, since nesting under an ordinary key would never start the recursion and would have passed for the wrong reason. STILL OPEN, and the lead holds it: a LEGACY row whose stored fields blob already carries the escape still exports 200 and re-imports 400. That is data this fix cannot make importable without weakening the write-side refusal, and the disposition (repair sweep, flagged import, or documented acceptance) is a product ruling. Recorded on the item. Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN * fix(server): close the three body doors codex round 3 found (BUG-2803) All three verified before fixing, none taken on the reviewer's word. 1. BUNDLE IMPORT BYPASSED THE REFUSAL (P1). handlers_import_bundle.go parses pad-export.json itself rather than through decodeJSON, so the SAME workspace import — reached with Content-Type application/gzip instead of application/json — walked straight past the NUL check into Postgres. The bundle's export blob is now checked with bodyDecodesNUL before ImportWorkspace, answering the same 400. Test drives a real tar.gz through the router with a clean-bundle control leg, because this path answers 400 for a dozen unrelated reasons (bad gzip, out-of-order tar, duplicate entries) and a bare 400 would prove nothing. 2. ONE CALLER SWALLOWED THE NEW ERROR (P2). handlers_admin.go's test-email endpoint read `if err := decodeJSON(...); err != nil || input.To == ""` and fell back to the admin's own address, so a body carrying a NUL answered 200. An ABSENT body legitimately means "send it to me"; a body that is present and REFUSED is a different thing, and collapsing the two turns a validation error into a success. The two cases are now separated on errors.Is(err, io.EOF). 3. THE COMPLETENESS TEST COULD NOT SEE PAST TWO CALL SHAPES (P2). It scanned for json.NewDecoder(r.Body) and io.ReadAll(r.Body), so it was blind to io.ReadAll(io.LimitReader(r.Body, n)) — a shape ALREADY in the package — and to any alias or helper. A completeness test that misses a live example is worse than none, because it reads as coverage. It now scans for the thing that cannot be spelled around, a reference to the request body at all, and requires every FILE touching one to be accounted for with a written reason. Both directions are asserted: an unaccounted file fails because a door may have opened, and an accounted file that no longer touches a body ALSO fails, so the list cannot rot into stale excuses that quietly cover a future reader. Verified with a positive control (an added body reference in an unlisted file fails) and a negative one (a stale entry fails). FOUND BY THAT WIDENED SWEEP, and fixed here rather than filed: the raw artifact import (POST /workspaces/{ws}/import-artifact) takes TEXT, not JSON, so it never went through decodeJSON and inherited neither the NUL refusal nor the path/query rule — a body is neither. A raw NUL or invalid UTF-8 reached the store and Postgres answered 22021, which the handler turned into a 500 for what is a client error. It now applies bindableText, the same predicate ValidatePath and ValidateQuery use, and answers 400 invalid_body. Note the shape difference from the JSON half: there the ESCAPE is the vector because a decoder rejects a raw NUL; here the RAW BYTE is, because nothing is in the way. Each fix has a mutation run against it: disabling the bundle guard fails the bundle test, disabling the artifact guard fails the artifact test, and both controls still pass. Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN * fix(server): the escape gate was unsound, and YAML has its own (BUG-2803) Codex round 4, two P1s, both reproduced before fixing. 1. THE FAST PATH LET A REAL NUL THROUGH. bodyDecodesNUL gated on "does the raw body contain the six-character escape". That is unsound: the BACKSLASH itself can be written as an escape, so a body carrying \u0000 contains no literal six-character sequence anywhere in its raw bytes, while the OUTER decode manufactures one inside the string — and if that string is re-parsed as a JSON document (jsonEncodedFieldKeys) the second parse turns it into a real NUL. Measured through the real router before the fix: the oblique spelling answered 201 where the direct one answered 400. The mistake was applying a fact about how a NUL is spelled INSIDE a decoded string to the RAW BYTES, where the backslash can itself be an escape. That is the same layer-confusion this whole bug is made of, for the third round running. The gate is now a BACKSLASH. Every JSON escape mechanism requires one, so a body with no backslash has decoded strings byte-identical to its raw bytes, and a raw NUL cannot survive the decoder — no backslash therefore means no NUL, at any depth, however spelled. Bodies WITH one pay for an exact answer, a larger set than before (any nested JSON carries a backslash-quote), which is the cost of being correct. The same correction applies to the per-string pre-filter one level down. 2. YAML HAS ITS OWN ESCAPE VOCABULARY. The raw bindableText check added last commit passes a double-quoted scalar `title: "a\0b"` — no NUL in the request bytes — and the YAML decode manufactures one. Measured before the fix: that artifact imported 201 with a NUL in the item title. The decoded artifact is now checked too: title, body, and every frontmatter field value, walked because a playbook's `arguments` is a nested structure rather than a scalar. Keys are checked as well as values, on the same precautionary grounds ValidateQuery states for query parameter names. Same shape as the JSON half in both cases: a value that is harmless until a SECOND parse, checked at the layer that can see it. Tests: the oblique spelling joins the nested-document table, and the YAML escape joins the artifact table. Each is mutation-verified — reverting the gate to the substring fails the oblique case only, and disabling the post-decode artifact check fails the YAML case only, with the raw-byte cases still killed by the raw check. That per-leg discrimination is the point: it shows each check earns its own keep rather than being covered by its neighbour. Prose corrected where this falsified it: jsonNULEscape's "it is the ONLY spelling" is true of the escape and was being used to justify a filter on the raw bytes, which is a different claim. Both now say so explicitly. Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN * fix(server): multipart text fields and the bundle manifest (BUG-2803) Codex round 4's two P2s. Both are the same shape as the rest: a caller-supplied string reaching a text comparison through a door the earlier fixes did not cover. 1. MULTIPART TEXT FIELDS. The multipart body is deliberately exempt from the JSON rule — its payload is binary blob content and must not be scanned for text validity — but its TEXT fields are a different thing. `item_id` goes to ResolveItem and into a database comparison exactly as the query-string channel does, and that channel has been validated at the transport since BUG-2784; the form channel was not. multipartValues now drops values that are not bindable text, which makes an unusable value indistinguishable from an absent one — the disposition resolveUploadItemID already applies to empty values. The uploaded FILENAME gets the same predicate, with a fallback to a generic name rather than a refusal: the bytes are fine, only the label is unusable. A NEGATIVE RESULT worth recording, because it changed the test: a RAW NUL in the multipart header is NOT the vector. Go's multipart reader refuses it as a malformed MIME header line before any handler sees it (measured: 400, "malformed MIME header line"). The reachable spelling is the RFC 5987 encoded form, filename*=UTF-8''sh%00ot.png, which the header parser accepts and percent-decodes afterwards. The first version of this test used the raw form and was testing a vector that does not exist. 2. THE BUNDLE ATTACHMENT MANIFEST. A second JSON document inside the tar.gz, parsed directly like pad-export.json was, so it needed the same check. Without it a NUL in a manifest string reached rehydrateAttachment, whose failure is logged and SKIPPED — so the import reported success while silently dropping the attachment. The skip-on-failure behaviour is pre-existing and deliberate (a partial restore beats none); refusing the bad INPUT is what stops it being reached this way. Left as it is, and named rather than quietly changed. A VACUOUS ASSERTION THE MUTATION CAUGHT, recorded because the test would otherwise have shipped as coverage: the filename leg first asserted `!strings.ContainsRune(body, 0)` on the RESPONSE, which is JSON — a NUL in the filename comes back as the six-character escape, not as a 0x00, so the check passed whether or not the fix was present. It did pass with the fallback disabled. Now it decodes the response and asserts the replacement name. The item_id leg had the mirror-image weakness: it asserted "not a 500", which is the Postgres-only symptom, so on SQLite it would have passed either way; it now asserts the request behaves exactly like the no-value control. Every fix in this commit has a mutation against it, and each kills only its own leg. Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN * refactor(server): drop the now-unused escape constant (BUG-2803) The gate became a backslash check, which was the last production use of jsonNULEscape; golangci-lint's unused check failed on the next run. Its documentation was load-bearing, so the explanation moved into bodyDecodesNUL's comment rather than being deleted with the variable — including the distinction that made the old gate wrong (the escape has one spelling INSIDE a decoded string, which is not a claim about the raw bytes). Caught by re-running lint on the tip after the previous commit rather than trusting the run from the tip before it. Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN * fix(server): rune-safe truncation and User-Agent sanitising (BUG-2803) Codex round 5 was asked for the POPULATION rather than a confirmation — "enumerate every remaining way a caller-supplied string can reach a database text or jsonb parameter without passing a validity check" — and returned three residual classes with their sinks. Two are fixed here; the third is filed, because measuring it needs a fixture this unit should not grow. 1. TRUNCATION CAN UNDO THE VALIDATION. Four sites cut a caller string with a plain byte slice (name[:120], input.Name[:200]). If the boundary lands inside a multi-byte rune the result ends in a partial sequence and is no longer valid UTF-8 — so a value that PASSED the body check a few frames earlier arrives at the store unbindable, and Postgres answers 22021 for a request the server already accepted. This is the interesting one, because no input-side round could have found it: the defect is downstream of validation, and it is invisible with ASCII fixtures, which is what every test in that area used. truncateBindableText walks back off continuation bytes and drops the straddling rune. Tested with 2-, 3- and 4-byte runes so an off-by-one walk-back cannot pass them all, and with a counterfactual leg asserting the naive slice really does produce unbindable output for the same input — without it the cases would pass against an implementation that did nothing. 2. USER-AGENT REACHES TEXT COLUMNS. It lands in activities.user_agent (three document paths, the connected-apps revoke) and sessions.user_agent (three login paths), and no rule here sees a header. The disposition is SANITISE, not refuse, and that is deliberate: a header is metadata this server chose to record, not something the caller asked for, so a malformed one must not turn an otherwise fine request into a 400. The two sites that HASH the header are left alone — sha256 over arbitrary bytes is well defined, and changing what is hashed would invalidate every stored UAHash. The filing's own earlier probe had recorded User-Agent as NOT reproducing on the item-create path. That was true and did not generalise; these are different sinks. 3. NOT FIXED, FILED: the OAuth form-encoded bodies (/oauth/token, /oauth/authorize/decide, /oauth/revoke, /oauth/introspect) parse url-encoded form data outside the shared body validator, with connection_name reaching oauth_connections.name and client_id reaching the oauth_clients.id lookup. This was the ORIGINAL subject of BUG-2803 before the filing was re-scoped, and it was recorded then as unreachable without a fosite-backed fixture. That is still true, and round 5's sink list is far more than the filing had. Filed rather than guessed at. Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN * fix(server): narrow the gate, stop refusing natural-shape fields (BUG-2803) Codex round 6 plus one measurement of my own. Three changes, one of them a revert of something I got wrong in the previous commit. 1. THE GATE COST TOO MUCH, so it is narrower and still sound. The previous commit gated the walk on "does the raw body contain a backslash", which is correct but catches every body carrying nested JSON (each `\"` is a backslash). Measured on a ~377 KB import-shaped body: 60106 allocs/op with that gate versus 30073 with the walk disabled — the walk was running on ordinary traffic. The gate is now the four bytes that begin any \u escape for a character below U+0100. The argument: to manufacture the six-character NUL escape inside a decoded string, each of its characters arrives either literally from the raw bytes — in which case the raw contains the escape, which begins with that prefix — or from a \u escape of its own, and the three characters involved (backslash U+005C, 'u' U+0075, '0' U+0030) all sit below U+0100, so those escapes begin with it too. Back to 30073 allocs/op, identical to the walk-disabled build. That argument is the same KIND of reasoning that was wrong two rounds ago, so it does not stand on its own: a differential test runs the gated function against an UNGATED walk over a corpus built to attack it — oblique backslash, upper-case hex, an escaped 'u', an escaped '0', a doubled backslash — and fails on any disagreement. It also asserts the corpus contains both answers, since agreement over a one-sided corpus would be vacuous. Reverting the gate to the old substring fails it. 2. THE CHECK REFUSED THE NATURAL SHAPE OF ITS OWN FIELDS. `tags` and `fields` accept both a JSON-encoded STRING and their natural array/object form, and the walk propagated "this subtree is JSON-encoded" into containers — so a free-form tag whose whole value happened to be a JSON document was refused, though nothing re-parses it. Measured: refused before, accepted now, while the JSON-encoded spelling of the same field is still refused. The flag now marks only a direct STRING child of a listed key. 3. REVERTED: I wired the three LOGIN paths to the User-Agent sanitiser last commit, before reading store.CreateSession. It HASHES the header and stores no text — the round-5 enumeration named "sessions.user_agent" and I took the name for a column. The change would have been actively harmful: login would store sha256(sanitised) while middleware_auth still compares sha256(RAW), so every session from a client with a non-UTF-8 User-Agent would fail validation. A sink named in a review is a pointer to verify, not a finding. The real sink is activities.user_agent, from three document paths and the connected-apps revoke. 4. And the wiring leg codex asked for, on that real sink: a request through the router with a malformed header, reading the STORED value out of the activities row, with a control asserting an ordinary header is kept VERBATIM. Unwiring the production call site fails it; the helper's unit test does not notice. Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN * fix(server): apply the key rule at every level, not once (BUG-2803) Codex round 7, both findings, the first confirmed by measurement. 1. THE RECURSION WENT ONE LEVEL TOO DEEP. Once the walk descended into a JSON-encoded string it treated the WHOLE subtree below as JSON-encoded, so a value nested two levels down — an ordinary string inside a `fields` blob that happens to hold JSON text — was refused. That is a false rejection, and the measurement says so plainly. With the depth-2 check disabled, on Postgres 17: depth 1 (the fields blob itself) -> 400 (correct: Postgres parses it) depth 2 (a string INSIDE the blob) -> 201 (accepted, no error) control -> 201 The handler parses `fields` ONCE. The inner text is re-escaped when the blob is written, so what Postgres receives has a doubled backslash and no escape at all. Only the document Postgres itself parses can carry a fatal one. The nested call now passes false rather than true, which makes this a KEY RULE APPLIED AT EVERY LEVEL rather than a depth limit: a JSON-encoded key INSIDE a document still recurses (pinned by a test), an ordinary one does not. Same correction as round 6's natural-shape fix, one level further in — I fixed the sibling case and left this one, which is CONVE-18's lesson about my own enumeration being a sample too. I checked whether anything re-parses a value inside the blob before loosening this, rather than assuming: `arguments` was the candidate, and parsePlaybookArguments asserts it is a native ARRAY (raw.([]any)) rather than a JSON string, so it is covered by the natural-shape rule and needs no second parse. 2. AN ERROR MESSAGE THAT SENT CLIENTS THE WRONG WAY. The OAuth dynamic client registration handler prefixed every decode failure with "Request body must be JSON". A body carrying a NUL is valid JSON, so that message sends a client hunting a syntax error it does not have. The two failures are now distinguished. Round 7 also reports no break in normal CLI, MCP or web-client request generation — they marshal JSON and encode paths and query parameters — which is the first thing any round has said about the client surface. Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN * fix(server): complete the artifact check, make the walk path-aware (BUG-2803) Codex round 8. It confirmed round 7's two fixes, then found two real defects and two inaccurate comments — the comment half being the angle the round was asked for. 1. THE ARTIFACT CHECK MISSED TWO REACHABLE FIELDS. artifactIsBindableText walked the decoded artifact by TYPE, so it never covered Provenance — whose strings are rendered into a Markdown footer appended to the stored content — and never matched Arguments, declared []map[string]any, a concrete slice type the walk's []any case does not match. A YAML NUL escape in either reached storage. It now MARSHALS the artifact and searches the output for the escape encoding/json produces. A type switch over a struct that grows is a list that goes stale in silence; marshalling covers every exported field, including ones added later. The one thing it cannot see is invalid UTF-8 (which marshals to U+FFFD), and it does not need to: step 2 rejects that in the request bytes, and YAML cannot manufacture it from valid input — its escapes name code points, where \0 names a NUL. Both new cases fail with the check disabled; the raw-byte cases still pass, killed by the raw check, so each leg is discriminating. 2. THE WALK WAS NOT PATH-AWARE. A collection may declare a user field literally named `schema` or `tags`. The walk consulted the wire-key list at every level, so `{"fields":{"schema":"..."}}` treated a user field name as a wire key and refused valid text holding a JSON example. The key list is now consulted only OUTSIDE caller data — not under a natural `fields` object, not inside an element of a `tags` array, not inside a re-parsed document. Combined with round 7's fix that makes the descent exactly one level deep BY CONSTRUCTION, which is why the depth counter is gone: with the flag no longer inherited, a bound could never fire, and dead protection reads as protection. The depth-bound test is replaced by one that pins the property directly — an escape IN the parsed document is refused, one BELOW it is accepted, and a wire-key-shaped user field does not restart the descent. 3. THREE COMMENTS CORRECTED, all mine, all of the kind a reader would believe without checking: - MaxBytesReader: Close FORWARDS to the underlying body rather than being a no-op, and with a nil writer there is no automatic 413 — the cap surfaces as a read error the callers turn into 400. Behaviour unchanged; only the claim was wrong. - parseArtifactRequest said "three checks" while implementing five, and its returns list omitted ErrArtifactUnbindableText. Both added by this branch, which is exactly the prose a change is most likely to falsify (CONVE-23). - errJSONBodyNUL claimed all 65 callers surface its message. The STATUS is uniform; the wording is not — several substitute a generic string. 4. And one in a test: the timeline fixture said both backends hold a CHECK constraint a raw NUL violates. items.fields is a plain TEXT column with no CHECK on SQLite. What was OBSERVED is "SQL logic error: malformed JSON"; the likely source is an expression index over json_extract, and that attribution is recorded as NOT verified rather than asserted. Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN * fix(server): a regression this branch introduced, and the same trap again (BUG-2803) Codex round 9. Both findings are mine, one of them a regression from the round-8 restructure two commits ago. 1. THE ROUND-8 RESTRUCTURE REOPENED THE ORIGINAL DOOR. Taking the JSON-encoded branch for a listed key skipped the plain "does this string contain a NUL" check and asked only "does the document this string carries hold an escape". Those are different questions. So {"fields":"a<NUL escape>b"} — a direct NUL in the fields value, the very first case this whole change closed — was accepted again. Both checks now run. The test pins all three legs: a direct NUL in the fields string, an escape inside the fields document, and an ordinary fields string that must still be accepted, so the first two cannot pass merely because everything under a listed key is refused. 2. THE ARTIFACT CHECK FELL INTO THE TRAP IT WAS WRITTEN AGAINST. It searched the MARSHALLED bytes for the escape sequence, and a value holding the six LITERAL characters marshals to a doubled backslash which still contains that sequence as a substring — so valid content was refused. Artifacts are documentation; text about a JSON escape is exactly what one carries. Worse than the bug: the comment I wrote asserted the ambiguity "cannot arise here". It was the same doubled-backslash case bodyDecodesNUL exists to resolve, one function away, and I wrote a sentence explaining why it did not apply instead of checking. The marshalled form is now decoded again and walked with the same machinery — the round trip is what makes every field reachable without a type switch, the walk is what makes the answer exact. Its test asserts literal escape TEXT is accepted in title, body and a field value, with a counterfactual leg asserting a real NUL in each of those places is still refused, so acceptance cannot come from the check doing nothing. Reverting either fix fails its test and only its test. Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN * docs(backup): the one case where an export is not importable (BUG-2803) Codex round 12, an operational pass. It found no migration or config requirement, and two documentation gaps. docs/backup.md promises that application-level export/import is portable across SQLite and PostgreSQL. Since BUG-2803 that has one exception: a workspace whose stored data contains a NUL exports fine and is refused on import. It can only affect data written before the rule existed and only on SQLite, which accepted it — a PostgreSQL instance never stored one. `pad db migrate-to-pg` has the SAME problem and reports it worse: it copies rows directly and never passes through the import guard, so a legacy row fails against PostgreSQL's JSONB parser partway through the copy rather than being refused up front. That is the likelier way an operator meets this, since it is the operation that puts an entire old SQLite database in front of PostgreSQL for the first time. Recorded on BUG-2810, which owns the preflight and repair. Round 12's other finding — that the PR's stated release note covered the JSON 500-to-400 change and none of the rest — is fixed in the PR body rather than in the tree. Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN * test(server): close two blind spots the tests themselves had (BUG-2803) Codex round 13, asked whether the new TESTS are sound. Five findings; these are the two that were self-contained. The other three are recorded on the item with what each needs. 1. THE COMPLETENESS SCAN WAS BLIND TO FORM BODIES. It matched only `.Body`, so FormValue / ParseForm / MultipartForm — which read the request body just as surely — were invisible. It therefore reported full coverage while the OAuth form-encoded handlers were entirely outside its view. Widened, and it immediately failed on handlers_oauth.go, which is the instrument working. That file is now ACCOUNTED FOR AS A KNOWN GAP rather than as safe: the OAuth handlers read form-encoded bodies that no rule in this family covers (the transport rules see the query half of r.Form, not the body half), tracked as BUG-2811 and needing a fosite-backed fixture to measure. The test now STATES the gap instead of being blind to it, which is the difference between a completeness claim and a completeness appearance. 2. THE TRUNCATION TEST ADMITTED AN IMPLEMENTATION THAT RETURNED "". Its assertions were: within the limit, bindable text, a prefix of the input. An empty string satisfies all three. It now also asserts that an input fitting the limit comes back UNCHANGED, and that no more than one rune (4 bytes) is lost to the boundary — so a truncator that drops too much fails, not just one that keeps too much. Both were found by asking whether a broken implementation would pass, which is the question CONVE-12 is about and which I had applied to the production code and not to these two tests. Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN * test(server): the three remaining round-13 gaps (BUG-2803) Codex round 13's other three findings, all of the same shape: a test that would stay green with the production change reverted. 1. THE MANIFEST CHECK WAS UNTESTED. The bundle test built archives containing only pad-export.json, so disabling the INDEPENDENT attachment- manifest check left the suite green. The new test builds a bundle with both entries, differing only in the manifest, so a refusal cannot come from the export half. Verified by disabling each check separately: only the matching test fails, so the two are independently covered. 2. THE TEST-EMAIL CHANGE HAD NO HANDLER-LEVEL TEST. Every existing leg exercised decodeJSON, so reverting handlers_admin.go to default EVERY decode failure to the admin's own address passed them all. The new test drives the real endpoint with a wired mock sender and pins the distinction that used to collapse: an ABSENT body still means "send it to me" (control), an ordinary body still sends (control), and a body that is present and refused answers 400 rather than being reinterpreted as the default recipient. 3. THE MULTIPART LEG CHECKED ONE BYTE CLASS. A filter rejecting NULs while letting malformed UTF-8 through would have passed it. It now drives both, which matters because invalid UTF-8 is the class that reaches Postgres as 22021 on a UTF8 database. Round 13 was asked whether the new TESTS are sound — deterministic, order-independent, and failing on broken code. It reported the fixtures isolated and found five ways they were not discriminating. Two were fixed in the previous commit; these are the rest. Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN * test(server): pin the wiring at every call site, not one (BUG-2803) Codex round 14 confirmed round 13's five, then found the same shape one level out: reverting a SINGLE call site back to the unsafe form left the whole suite green, because the surviving fixtures are ASCII and a helper's unit test does not care who calls it. TestTextSafeHelpersAreUsedAtEveryCallSite asserts the wiring STATICALLY rather than adding a fixture per site (an OAuth connection, a cloud login, four audit paths). A byte-slice truncation of a caller string fails it, and so does a raw User-Agent read outside the exempt set. Both directions are checked: finding none of the SAFE form also fails, so a scan that silently matched nothing cannot pass forever. The User-Agent exemptions carry counts rather than being blanket, so a NEW raw read in an exempt file still fails. All four reads in handlers_auth.go are exempt because they feed a HASH — CreateSession hashes the header and stores no text — and sanitising before hashing would be actively harmful: login would store sha256(sanitised) while the session check still hashes the RAW header, failing validation for every client with a non-UTF-8 User-Agent. middleware_request_text.go's one raw read is requestUserAgent itself. Verified by reverting one truncation call site and one User-Agent call site independently; each fails the test. Round 14's third finding is fixed behaviourally rather than statically, because the static scan cannot see it — handlers_oauth.go is already listed for its form-body reads. TestOAuthRegisterRefusesNULBody drives the real dynamic-registration endpoint with cloud mode and an OAuth server wired, with a control leg registering successfully, and pins both the refusal and the message split: the body IS valid JSON, so the answer must not send a client hunting a syntax error. Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN * fix(server): match wire keys the way the decoder does (BUG-2803) Codex round 16, asked whether this change is consistent with its siblings in the same file and extensible by someone who did not write it. It found a live bypass instead. encoding/json matches an incoming key to a struct field by an exact match first and a CASE-INSENSITIVE one otherwise, so {"Fields":...} and {"FIELDS":...} land in ItemCreate.Fields exactly as {"fields":...} does. The walk looked the key up case-SENSITIVELY, so it skipped the nested document for a body the handler went on to accept, and the database answered the original 500. Measured before the fix: `fields` refused, `Fields` and `FIELDS` accepted. This is the same defect shape as everything else in this unit — a check that agrees with one layer's rules while the layer that actually consumes the value uses different ones — which is why the fix is a PREDICATE rather than a wider map: the map is the vocabulary, and the matching RULE belongs to the consumer. Someone adding a key should not also have to remember to add its spellings. The test drives six spellings including mixed case, with a control asserting an unlisted key stays caller data in any casing, so this is case-insensitive matching rather than matching everything. Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN * fix(server): fold keys the way encoding/json folds them (BUG-2803) Codex round 17, first of five findings. The previous commit fixed the ASCII half of key matching and left the Unicode half, which is this bug's own pattern one more time. encoding/json matches with Unicode SIMPLE FOLDING, not lower-casing. U+017F LATIN SMALL LETTER LONG S folds to 's', so "ſchema" reaches the `schema` struct field while strings.ToLower("ſchema") is unchanged and missed the allowlist — a nested NUL under that spelling reached the handler undetected. Matching is now strings.EqualFold against each canonical key. The test carries both a lower-case fold spelling and an upper-case one alongside the ASCII cases, and keeps its control asserting an unlisted key stays caller data in any casing. The other four round-17 findings are recorded on the item rather than patched here: they are genuine layer disagreements (duplicate keys merging differently in a typed decode than in a map, a scan-failure disposition on inputs the typed decode tolerates, and unknown-field policy) whose fixes are design decisions rather than corrections, and this seat is near its context bar. Each is written up with the measurement it needs. Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN * fix(server): pin that a NUL-bearing manifest refusal keeps the partial workspace (BUG-2803) Codex round 18. The comment on the manifest NUL branch said refusing the input "stops it from being reached this way" and stopped there, which reads as though the refusal undoes the import. It does not. A plain error with a non-nil workspace keeps the partial workspace, exactly as every other manifest failure in this loop does — the rollback branch fires only for *importStatusError, and mid-stream manifest failures intentionally keep what was imported (TASK-896). Returning a rollback-shaped error here would give NUL-bearing manifests different semantics from malformed ones, which is a change to the bundle-import contract rather than a fix to this bug. So the behaviour is unchanged and now DELIBERATE: the comment states it, and the test asserts the persisted state rather than only the HTTP answer. Mutation: routing the branch through *importStatusError makes the refusal roll back, and the new assertion fails naming the release note it would falsify. The pre-existing status/body assertions do not notice. Claude-Session: https://claude.ai/code/session_01AUvLoXsKdS5sdpYju6rj4p * docs(server): record the four map-model disagreements as dispositions, and pin them (BUG-2803) Lead ruling day-68 is land-and-follow: this branch lands on its measured commits, and the token-stream rewrite is the BUG-2812 unit's spec rather than a late restructure of an 18-commit branch under review pressure. That makes the four open findings from rounds 16-17 something to WRITE DOWN precisely, not something to leave in a trail comment. The doc comment on bodyDecodesNUL now carries all four, with the one root cause named: this scan decodes into map[string]any and the typed decode does not agree with that model about keys. Two under-refuse (duplicate-key merge; scan-failure passthrough) and are BUG-2812's spec - both dissolve under a walk that never builds values. Two over-refuse (unknown fields; case-variant duplicates) and are ACCEPTED, because refusing is the safe direction. The asymmetry is stated rather than smoothed over: within the map model, (1) and (4) are one defect seen from two sides and only one of them fails safe. Finding (3) is an observable compatibility change - a forward-compatible field carrying a NUL escape now gets a 400 where it got a 200 - so it goes in the release note as well as here. A qualification only protects where the actor meets it. All four are pinned by a test, measured on this tip rather than carried over from the round-16/17 write-up. The two known-gap legs assert the WRONG answer on purpose: when BUG-2812 lands they FAIL, naming the doc comment and the release note as what to update. Both gap legs carry a premise assertion - the same bodies with the disagreement mechanism removed ARE detected - without which they would pass against a check that detected nothing. Claude-Session: https://claude.ai/code/session_01AUvLoXsKdS5sdpYju6rj4p * test(server): wire release-note item 10 to the router, with its before-state measured (BUG-2803) The disposition test proves bodyDecodesNUL RETURNS true for an unknown field carrying a NUL escape. The release note claims the API answers 400. Those are different claims and only the second one is what an operator or client author reads - CONVE-19, my own convention: a direct-call test vouches for the component, not its binding. Two legs, and the control is the load-bearing one. An unknown field with an ordinary value must still be ACCEPTED, so this pins "refused for the NUL" rather than "refused for being unknown". The handler does not reject unknown fields; if it ever started to, the note's explanation would be wrong while its status code stayed right, and no status-code-only assertion could see that. The before-state is measured rather than asserted from memory. Disabling the check makes the same request answer 201 - which is main's behaviour, since decodeJSONWithLimit there unmarshals straight into the typed value and the key is dropped. So "answers 400 where it answered 200" is a measurement in both directions, not a recollection of one. Claude-Session: https://claude.ai/code/session_01AUvLoXsKdS5sdpYju6rj4p * docs(backup): the NUL rule lives in the binary, not the database (BUG-2803, BUG-2813) Codex round 19, the fresh-angle deploy/rollback/mixed-version pass. docs/backup.md said a NUL-bearing row "can only affect data written before that rule existed, and only on SQLite". The second half is true. The first half is false, and the reason is the interesting part: the guard is in decodeJSONWithLimit, so the invariant is a property of the running BINARY, not of the database. On SQLite any window where an older binary serves the same database can still write one - a rollback after upgrading, a staged rollout with an old and a new instance sharing a database, a second older instance on the same file. The window closes, the guard returns, and the rows are already stored, behaving exactly like genuinely old ones. A rollback is an ordinary operational move, so this is not an exotic path. The doc now states the binary-version dependence, says which dialect is affected and why PostgreSQL is not (it refuses a NUL itself, at every version), and gives the operational answer: drain writes from older binaries before the new one serves, or roll forward rather than back. Store-layer enforcement - so the running build stops mattering - is filed as BUG-2813 rather than added here. It is a dialect-level change and the day-68 ruling on this unit is land-and-follow. The same false implication was carried by the PR's release note calling such a workspace "legacy"; corrected there too. Claude-Session: https://claude.ai/code/session_01AUvLoXsKdS5sdpYju6rj4p * docs(server): cite the ruling in house style, not the team-room day counter (BUG-2803) "lead ruling day-68" is the internal day counter, which means nothing to anyone reading this repo and is inconsistent with every other citation in it - the codebase cites a lead ruling by DATE or by BUG ref, never by day-N. Replaced with the bug ref, which is the part a reader can actually follow. Claude-Session: https://claude.ai/code/session_01AUvLoXsKdS5sdpYju6rj4p * docs(server): drop a commit count I had already measured as wrong, and stop asserting a cause I borrowed (BUG-2803) Two defects in a comment I wrote an hour ago, both of the kind this unit's trail keeps recording. "an 18-commit branch" - the branch was 20 commits at b0192871 when I counted it this session, and is more now. 18 came from the previous checkpoint's own miscount, which I had ALREADY identified and written up before I typed it again here. A number that arrives inside a sentence about something else does not feel like a claim, which is exactly why it survives. The count is incidental to the argument, so it is gone rather than corrected - a figure that has to be maintained to stay true is a liability in a doc comment. "this branch's one regression came from exactly that" - the ruling's reasoning, restated by me as a verified fact. The regression I know about came from wiring a fix off a reviewer-named sink list without reading the mechanism, which is adjacent to "restructuring late under review pressure" but is not the same mechanism, and I did not check whether it is the one the ruling meant. Now attributed to the ruling and stated as its reasoning, with the part I can defend - the review loop finding something in nearly every round indicates a design problem - carrying the argument. Claude-Session: https://claude.ai/code/session_01AUvLoXsKdS5sdpYju6rj4p * fix(server): sanitise the MCP audit tool_name, and correct three claims wider than their evidence (BUG-2803) Codex round 20, asked for a POPULATION rather than a confirmation (CONVE-24). It returned a covered list AND four findings; this commit carries the two that belong to this unit plus the doc corrections. ## The door: MCP audit is a second reader, not a pass-through parseMCPRequestBody runs its OWN json.Unmarshal and binds the decoded method / params.name to mcp_audit_log.tool_name, TEXT NOT NULL. A six-character NUL escape therefore arrives as a real NUL: PostgreSQL refuses the audit INSERT with 22021 - the exact symptom this unit exists to remove - and SQLite stores an unprintable tool name. Nothing upstream catches it; the /mcp transport decodes the JSON-RPC envelope itself rather than through decodeJSON, so the body rule never sees the request. Measured before fixing: the decoded name reached the column intact. This unit's own completeness map had CERTIFIED that reader as safe, on the grounds that "decoding still happens in the MCP dispatcher". That is true and it does not bear on what this middleware persists - a correct description of a mechanism, with no question asked about what it does, sitting in the one artifact whose job is to say the population is covered. Corrected there too. Disposition is SANITISE, not refuse, following the User-Agent precedent from earlier in this unit, and the rule now lives in one extracted helper (sanitiseStoredText) with the reasoning attached: the body rule refuses because the caller asked to store that value; this serves metadata the SERVER elected to record, where failing the write would lose the audit row for precisely the request most worth auditing. Both caller-derived returns are cleaned inside parseMCPRequestBody, so both call sites - the ok path and the denied path - are covered at the choke point rather than at either caller. Both are tested: params.name AND the method path. Mutations un-sanitising each one compile and kill only their own leg. ## Three claims corrected, all wider than their evidence - "all 65 call sites" in server.go: measured 70. Removed rather than corrected, because the number has to be maintained to stay true and says nothing the sentence needs. - docs/backup.md said a NUL "cannot be stored in a text or JSON column" absolutely, two paragraphs above my own text explaining that SQLite accepts one. Now stated as what it is: an application rule Pad enforces on both dialects, which is exactly why it has to be enforced. - artifact_import.go said such a value "cannot be stored under any encoding this product supports". Refuses, not cannot - stating a policy as a capability tells the next reader SQLite enforces something it does not. ## Filed, not fixed BUG-2814 - guarded writes re-emit at-rest NULs (move/copy/restore/ fields-patch), propagating a legacy value to rows that never had one. Distinct from BUG-2813: that one is about writing a NUL while an old binary serves, this is the fixed binary SPREADING one already present. Both dissolve under the same store-layer enforcement, so they are filed to be designed together rather than patched at each of a long and moving list of re-emit sites. Declined: round 20 also reported the release-note assertions as unsupported. They live in the PR body, which a read-only sandbox cannot see - the claim is about the reviewer's visibility, not the diff. Claude-Session: https://claude.ai/code/session_01AUvLoXsKdS5sdpYju6rj4p * fix(server): sanitise before testing for emptiness, so the audit fallback survives (BUG-2803) Codex round 21 ranked this the most dangerous un-probed lens, and it is a boundary my own round-20 fix created. parseMCPRequestBody tested env.Method == "" and p.Name == "" BEFORE sanitising. A value made entirely of NUL escapes is non-empty as decoded and empty once cleaned, so it passed over the fallback and was then blanked - storing an empty tool_name in a TEXT NOT NULL column. That is exactly the silent drop the "(unknown)" / "tools/call" fallbacks exist to prevent; the function's own doc comment says so. Measured before fixing: both shapes returned an empty tool_name. Fixed by ordering rather than by adding guards - clean first, then test - so the invariant is structural instead of something each return has to remember. Same by-construction preference as the symmetric-gate fix earlier in this unit. Worth recording that my first patch was WRONG in a way that compiled: I put the sanitise above the json.Unmarshal that populates env, so the method would always have been empty. Caught by printing the patched function and reading it, not by trusting the script saying "patched". Claude-Session: https://claude.ai/code/session_01AUvLoXsKdS5sdpYju6rj4p * fix(server): classify MCP audit on the raw method, and trim only JSON whitespace (BUG-2803) Codex round 22. Two P2s, both measured before fixing. ## A forgeable audit row - my own regression from the round-21 fix The round-21 change reordered sanitise-before-compare so the fallback would survive an all-NUL value. That reorder made the CLASSIFICATION read the sanitised method, so "tools/<NUL>call" cleaned up INTO the literal "tools/call" and the parser then lifted params.name and hashed the arguments for a method that was never tools/call. Measured: tool_name="pad_item" with a full 64-character args_hash - an audit row indistinguishable from a genuine pad_item call, mintable by anyone who can send a request. Worse than the review described it. Fixed by splitting the two jobs, which were never the same job: dispatch decisions read what the client actually SENT; sanitising is for the value that gets STORED. The round-21 boundary is preserved - a method empty only after cleaning still falls back to "(unknown)". Fixing one boundary and creating another in the same function is worth naming: the reorder was correct for the case it addressed and I did not ask what else read that value. ## Go whitespace is not JSON whitespace The empty-body shortcut used bytes.TrimSpace, i.e. unicode.IsSpace, which strips \v, \f, U+00A0 and more. encoding/json accepts none of them. So a body of just \v trimmed to empty, returned io.EOF, and an EOF-tolerant caller - playbook run treats errors.Is(err, io.EOF) as "no arguments supplied" and runs anyway - took a syntactically invalid body for an ABSENT one. Now trims exactly the four bytes JSON calls whitespace. The test drives both directions, because only the pair discriminates: real JSON whitespace must still shortcut to EOF or the playbook contract breaks, and non-JSON whitespace must not or the divergence survives. Reverting to TrimSpace compiles and fails three legs. Claude-Session: https://claude.ai/code/session_01AUvLoXsKdS5sdpYju6rj4p * test(server): give the walker an independent oracle, not one that shares its code (BUG-2803) Codex round 22, finding 3. TestBodyDecodesNULGateAgreesWithAnUngatedWalk compares the gated function against an "ungated" reference that calls the SAME production valueDecodesNUL. That is valid for what the test claims - it pins the raw-prefix GATE - but it structurally cannot see a defect in the WALKER, because such a defect is present identically on both sides and cancels. That matters here specifically: every walker defect this unit has had lived in traversal, descent, or key matching (rounds 1, 2, 4, 16, 17), which is exactly the part the differential cannot check. Added a second implementation of the contract, written in the test and deliberately not calling the production walker. It shares encoding/json and jsonEncodedFieldKeys; it does NOT share traversal, descent, or key-matching. It is iterative with an explicit stack rather than recursive, so a recursion-shaped bug cannot reproduce in it by accident. Demonstrated rather than argued. With the nested-document descent removed from the production walker - a mutant that reopens the exact door this unit exists to close, and which compiles: differential (gate vs ungated) ok <- blind, as the finding said independent oracle FAIL <- catches it The corpus is also asserted to contain BOTH answers, since two walkers that always answer false agree perfectly. Claude-Session: https://claude.ai/code/session_01AUvLoXsKdS5sdpYju6rj4p * test(server): make the body-reader inventory type-aware, and state what it still cannot see (BUG-2803) Codex round 22, finding 4. The inventory that claims every request-body reader is accounted for was lexical, and wrong in three ways - all in the direction that matters for a test whose job is to say nothing is invisible: - it recognised only the variable names r and req, so a handler holding its request as httpReq or orig was INVISIBLE; - it matched inside COMMENTS, so prose could make a file look scanned; - the manually-listed traits field was already evidence of the model-regex blind spot. My first fix broadened the pattern to any identifier. That was worse, and worth recording: it matched every unrelated .Body field - input.Body in comments, fetched.Body in url import, comment.Body, art.Body, sidecarErr.Body - flagging five files that read no request body at all. The only route to green would have been listing those five as accounted, and an accounting entry HIDES future readers in its file. A false entry is worse than a missing one, so I abandoned that approach rather than tuning the regex. Now keyed on the TYPE via go/ast: collect identifiers declared *http.Request in a function signature, then find reader selectors on exactly those identifiers. Names stop mattering, comments are not in the AST, and .Body on anything else is not a match. Positive control, run rather than argued: a handler taking httpReq *http.Request and reading httpReq.Body is FLAGGED by the new scan, and matched zero times by the old regex. Two limits now stated in the test, because an unqualified completeness claim is exactly how the MCP audit reader got certified safe while persisting a decoded NUL: accounting is per FILE rather than per call site, and only signature-declared requests are seen - one stashed in a struct field or captured by a closure is not a parameter. Claude-Session: https://claude.ai/code/session_01AUvLoXsKdS5sdpYju6rj4p * fix(server): mark a cleaned audit identity, and repair two vacuous tests of my own (BUG-2803) Codex round 23, plus a defect in my own instruments that the mutation matrix found and the tests hid. ## Cleaning is lossy, so a cleaned identity was forgeable Round 22 closed the coarse version: sanitising before classifying let "tools/<NUL>call" become a genuine tools/call. Classifying on the raw method fixed that. But sanitising still COLLAPSES distinct inputs onto one output, so "pad_<NUL>item" stored exactly what "pad_item" stores - same tool_name, same args_hash - and anyone able to send a request could mint an audit row and a Prometheus label attributed to a real call. Cleaning and identity are different jobs. sanitiseStoredTextChanged now reports whether anything was removed, and an identity that only became well-formed by cleaning is marked. The cleaned text is kept, so the row stays diagnosable; the marker keeps it distinguishable. Descriptive text (User-Agent) keeps the unmarked helper - nothing decides anything on it. The parenthesised form is what this file already uses for a synthesised value, and a real method or tool name does not begin with "(", so the marker cannot itself be forged by choosing a clever name. ## Two of my own tests were vacuous, found by a surviving mutant I wrote nul := "\u0000" in the round-21 and round-23 tests, which in Go is the NUL CHARACTER, not the six-character escape text. Those bodies were malformed JSON that encoding/json rejected, so neither test ever reached the path it named. The comment on the line said "the escape, not the character"; the code did the opposite, and the correct form was already three lines away in the round-20 test. Nothing in the test output showed this. It surfaced only because the marker mutation SURVIVED, and because a surviving mutant was treated as a question - does the test not discriminate, or did it not run - rather than as either answer. Both repaired and both now kill their mutants: removing the marker fails with tool_name="pad_item" and a matching 64-character hash; removing the emptiness guard fails with "(sanitised) " instead of "(unknown)". Correction for the record: the round-21 checkpoint said that fix was measured failing before the fix. That measurement used the broken literal. The finding was real and the fix is right, but it is only properly established as of this commit. Claude-Session: https://claude.ai/code/session_01AUvLoXsKdS5sdpYju6rj4p * test(server): use the canonical escNULLiteral helper, not a local literal (BUG-2803) The helper is assembled from bytes precisely so this escape cannot decay into the NUL character it describes, and its comment says so: written as a Go literal it is one backslash away from being the NUL itself. I rolled a local one in three tests anyway, and two of them decayed exactly as that comment predicted - vacuous until the mutation matrix caught them. The safeguard existed, was documented, and I walked past it; using it is the only version of this fix that cannot recur. Claude-Session: https://claude.ai/code/session_01AUvLoXsKdS5sdpYju6rj4p * fix(metrics): bound the cleaned-identity marker as a metric label, and correct a false cardinality claim (BUG-2803, BUG-2817) Codex round 24, which enumerated CONSUMERS of the values this unit changed rather than asking again whether the guard is right. Most of that enumeration came back FINE, which is the useful half; three findings did not. ## The marker must not reach Prometheus as part of a name The cleaned-identity marker is right for the audit ROW - an operator reading one row needs to know which tool it resembles. It is wrong for a metric SERIES: "(sanitised) pad_item" and "pad_item" would be two series per user and per status, for a distinction no aggregate query asks. metricsToolLabel collapses the marked form to the bare marker, so it costs exactly ONE extra label value in total and that value is a constant rather than anything a caller supplies. Two tests, and the second exists because the first is not enough. The direct-call test proves the collapse function collapses. The WIRING test proves the emit path calls it - CONVE-19, my own convention. Measured: with the call removed from recordMCPCallMetrics, the direct-call test stays green and the wiring test fails naming the leaked label. ## A cardinality claim that was never true internal/metrics documented the tool label as "bounded by the catalog (~7 tools today)" with arithmetic resting on that. The value is whatever the caller put in params.name, recorded even for requests that dispatch later rejects, so an authenticated caller can mint a series per request. The comment now says so and points at BUG-2817, filed with the fix shape and the two wrinkles it has to decide - the catalog lives in internal/mcp, and legitimate JSON-RPC methods are not catalog tools. That unboundedness is PRE-EXISTING and not this unit's to fix; bounding the marker's own contribution is, which is why the collapse is here and the rest is filed. Also corrected: I wrote BUG-2815 into two comments before filing, and the filing came back BUG-2817. Predicting an identifier is the same class of claim as predicting a count. Claude-Session: https://claude.ai/code/session_01AUvLoXsKdS5sdpYju6rj4p * fix: keep a storable extension in the filename fallback, and sync the rename draft (BUG-2803) Codex round 24, the two remaining consumer findings. Both trace to this unit, and both are cases where a value was made SAFE without asking what reads it. ## The filename fallback was lossier than its sibling An unstorable upload name became a bare "upload" - no extension - while the empty-name fallback two lines below has always produced "upload.bin". The unusable part of "sh<NUL>ot.png" is the STEM; ".png" is ordinary text, and it is what consumers dispatch on: Content-Disposition, the web download anchor, bundle export naming, and , whose documented contract is handing a path to something that opens files by extension. That command was measurably affected - it treats any non-empty stored name as authoritative, so its MIME-based extension fallback never ran and the temp file was extensionless. Fixed at the source: a storable extension survives the fallback, bounded to 16 bytes so a hostile name cannot smuggle a long tail through. The CLI keeps a defensive extension fallback for any extensionless stored name, which also covers rows written before this. Both directions are tested: "sh<NUL>ot.png" now stores "upload.png", and "shot.p<NUL>ng" - where the EXTENSION is the unusable part - still stores bare "upload". Without the second leg, "keep the extension" could quietly become "keep whatever trails the last dot" and reintroduce the value the fallback exists to remove. Dropping the extension again fails the first leg. ## A rename that could never come clean saveName replaced the app object but never updated the draft, so when the server normalised the name the draft stayed as typed, the equality check never matched, Save stayed enabled, and each press re-sent the same request. The server caps at 120 BYTES via rune-safe truncation while the input allows 120 CHARACTERS, so any multibyte name near the limit diverges. The draft is now assigned the value the server actually STORED rather than compared for length, which stays correct for any future normalisation. svelte-check: 0 errors. Claude-Session: https://claude.ai/code/session_01AUvLoXsKdS5sdpYju6rj4p * fix: reserve the synthesised-value namespace, and stop a fallback carrying an unvetted extension (BUG-2803, BUG-2818, BUG-2819) Codex round 25, which probed whether the values this unit SYNTHESISES can themselves be attacked. Earlier rounds asked whether the guard refuses bad input; this asked what the substitutes are worth. ## A fallback must not carry an extension the product would refuse Preserving a storable extension was right; bindableText was the wrong bar for it. Control characters are valid UTF-8 and not NUL, so they are storable - and they are STRIPPED when the name is written into Content-Disposition. So ".s<VT>vg" passes the extension blocklist, which sees no known extension, and reaches the client as ".svg". attachments.SafeFallbackExtension now requires a KNOWN, ALLOWED extension, so a synthesised name can only carry a suffix the product already accepts on the ordinary path. Tested both ways: an obfuscated .svg and an unknown .foo are both dropped to bare "upload", while .png still survives. That divergence is PRE-EXISTING on the ordinary path, where the caller's name is stored as given and no fallback is involved - filed as BUG-2818 with the fix shape. This change only declines to add a second door. ## A mutation exposed a guard that could not fire I first wrote an explicit alphanumeric loop in that predicate as well. Removing it changed nothing: no key in extMIMEMap contains a non-alphanumeric character, so the map lookup already excluded every obfuscated suffix. Keeping an unreachable guard whose comment claims it stops control characters would have misdescribed which line does the work - so the loop is gone, and TestExtMIMEMapKeysArePlain enforces the property it was relying on. A guard that survives its own mutation is a question, not a clearance. ## The marker was forgeable, so the namespace is reserved Marking only what cleaning changed was not enough. A caller may name a tool "(unknown)" - what the parser returns for a malformed body - or "(sanitised) pad_item", and a genuine request then records the same identity as a substituted one. The older sentinels always had this; the new marker inherited it. A leading "(" is now reserved for values this server synthesises, and any caller value entering that namespace is marked too, so the two never collide. Cost stated: an MCP tool genuinely named with a leading "(" is recorded marked; tool names are identifiers in every catalog this server knows. The principled fix for the whole class is a provenance FIELD rather than sentinel strings in a caller-controlled namespace. That is BUG-2819 - it is a migration on two tables, and the same trick cannot rescue attachment filenames, which are legitimately named with parentheses. Claude-Session: https://claude.ai/code/session_01AUvLoXsKdS5sdpYju6rj4p * test(server): fix the independent oracle, which was wrong in a branch its corpus omitted (BUG-2803) Codex round 26, finding 5, and it lands on the instrument I introduced two rounds ago to check the walker. The oracle descended into a listed key's JSON document whenever it met one - including when that key appeared INSIDE a natural object that was itself under a listed key. Production does not: a natural object or array under a listed key is USER DATA, because the server marshals it and nothing re-parses it, so a listed key appearing inside it is an ordinary field name rather than a document marker. Measured on {"fields":{"schema":"<escape text>"}}: production=false, oracle=true. Production is RIGHT and the oracle was wrong, so had that body been in the corpus the test would have failed and pointed at the production walker. It was not in the corpus. That is the part worth keeping: the test already asserted its corpus was not one-sided - that BOTH answers appear - and that check passed while a whole branch of the contract went unexercised. Both answers appearing is not the same property as every branch being covered, and I had treated it as though it were. Fixed by giving the oracle the same user-data rule, and both bodies are now in the corpus - the natural-object case that must answer false, and its string-valued counterpart that must answer true. Re-verified that the correction did not blunt it: with the production nested descent removed, the oracle still fails. Claude-Session: https://claude.ai/code/session_01AUvLoXsKdS5sdpYju6rj4p * fix: refuse path-component filenames, and single-source the MIME extension table (BUG-2803) Codex round 26, findings 2 and 3. ## ".." is not a filename, it is a path component The server guard listed "", "." and "/" but not "..", which survives bindableText. filepath.Ext("..") is "." - non-empty - so an extension check waves it through too, and a consumer joining it onto a directory gets that directory's PARENT. The CLI builds its temp path exactly that way. Both ends fixed, deliberately independently. The server now rejects any name that is only dots or carries a separator, checked on the trimmed form so "..." and "./" do not each need a case. The CLI sanitises the name it receives regardless: a client that builds a local path out of a remote string should not depend on the remote end having sanitised it, and this CLI talks to whatever instance it is pointed at. Tested with "..", "...", "./" and "a/b", with an ordinary name as the premise leg. Restoring the old narrow guard fails it. ## Two tables for one relationship The CLI kept its own MIME-to-extension table and it had drifted: images and video but not gzip, tar, XML, YAML, TOML, HTML, JavaScript or several documents the server has always allowed. So the extension fallback added in round 24 silently did nothing for exactly the types whose viewers most depend on it. The CLI now delegates to attachments.ExtensionForMIME, and the second table is gone. Measured after: gzip .gz, tar .tar, html .html, js .js, pdf .pdf. The reverse map needs one choice per type where several extensions share one, and those preferences are asserted to name types the forward map actually uses - because the first version listed "text/yaml", which this map does not use (it says application/yaml), so that preference could never fire. Same class as the alphanumeric guard removed in the previous commit, caught the same way. Also recorded against myself: I destroyed both new functions mid-edit by running git checkout on a file with uncommitted work, to "revert an approach". That is a documented trap I have hit before and had written down. The committed function survived; the uncommitted ones did not. Claude-Session: https://claude.ai/code/session_01AUvLoXsKdS5sdpYju6rj4p * test(server): make the body-reader scan scope-aware, wrong in both directions before (BUG-2803) Codex round 26, finding 4. The scan used ONE flat name-set per top-level function, which is wrong in both directions at once: - a function literal inside a handler was scanned with the OUTER function's request names, so an unrelated inner variable that happened to be called r was FALSELY flagged; - a request arriving only as a function literal's own parameter was INVISIBLE, because literals were never given names of their own. A false flag in this test is not harmless. The only way to green is to add the file to the accounted list, and an accounting entry HIDES every future reader in that file - so a false positive here converts directly into a blind spot later. That is the same trap that made me abandon the broadened regex two commits ago. Now walks a SCOPE at a time. Each scope inherits its parent's request names, drops any it shadows with a parameter of a different type, and adds its own. Local aliases (req := r) are picked up as well, since that is an ordinary thing for a handler to do and the alias reads the same body. Three controls, run rather than argued: closure parameter reader -> FLAGGED local alias reader -> FLAGGED shadowed inner variable -> not flagged The first two were invisible to the previous scanner, which never gave literals their own names, and the third is the false positive it produced - both by reading the code this replaces. Limits restated honestly rather than left as they were, since two of them are now closed. Still invisible: a request in a struct field, one from a context, and one whose type reaches http.Request through an alias or embedded field. This matches the literal spelling rather than resolving types; closing those means the type checker, not the parser. Claude-Session: https://claude.ai/code/session_01AUvLoXsKdS5sdpYju6rj4p * fix: stop the reverse MIME map emitting BLOCKED extensions, and close four instrument gaps (BUG-2803) Codex round 27 returned "do not merge yet" with three P1s. All of them are mine, from the previous two commits. ## The reverse map turned a refusal list into a source of extensions extMIMEMap is the FORWARD table used to REFUSE uploads - it deliberately lists .svg, .exe, .com so those extensions can be recognised and rejected. Reversing it wholesale meant ExtensionForMIME("image/svg+xml") answered ".svg", where the old CLI table answered nothing, and names a local file with that. So I closed an SVG door two commits ago and reopened one through the MIME helper. Blocked types now get no reverse mapping at all, and the test asserts it with a premise leg (the map must CONTAIN a blocked type, or the assertion never runs). Removing the exclusion fails naming .svg, .com and .msi. ## The oracle was closer, not identical Production descends only into a JSON DOCUMENT - a string whose trimmed form starts with { or [. The oracle unmarshalled any valid JSON, so a SCALAR under a listed key made it answer true where production answers false. Closer to production is not a usable oracle; only identical is. Aligned, and the scalar case is in the corpus. ## The scan was still not scope-aware, and could now MISS a reader A nested block shared the enclosing name-set, so { r := &http.Response{}; r.Body.Read(nil) } was FALSELY flagged. And the shadowing rule deleted a name rebound to http.Request BY VALUE - which still shares the Body, since it is an interface holding the same reader - so that read became invisible. Blocks are now their own scope and a value request counts. ## The controls I claimed were not in the suite Round 27 was right: I had run them as throwaway probes and deleted them, so nothing held the scanner to them. The scanner is now a package-level helper and TestBodyReaderScanDiscriminates drives it over ten synthetic files - six that must be detected (plain, unconventional name, closure parameter, alias, value copy, form reader) and four that must not (no request, shadowed by a closure parameter, rebound in a nested block, mentioned only in a comment). ## And an over-refusal of my own making The filename guard rejected any dot-only name and anything containing a separator. Only "." and ".." are path components; "..." is an ordinary POSIX filename, and filepath.Base has already reduced "a/b" to "b", so the separator test was dead on this platform and removed rather than left looking load-bearing. Preservation controls now pin that legitimate names survive. Also removed: an unused id parameter on safeLocalFilename. Claude-Session: https://claude.ai/code/session_01AUvLoXsKdS5sdpYju6rj4p * test(server): only a DEFINE can rebind a name, and cover the idiomatic reassignment (BUG-2803) Found by probing my own previous commit rather than by a review round - the first time in this sequence I have caught the adjacent breakage before the next round did. The scope rules deleted a request name on ANY assignment whose right side was not a request identifier. That is wrong in the dangerous direction, and it fires on the most idiomatic line in Go HTTP code: r = r.WithContext(ctx) io.ReadAll(r.Body) // <- invisible to the scan WithContext is a call, so the name was dropped and every later read went unseen. Measured before the fix: MISSED. The correct rule is type-sound. Go is statically typed, so a plain cannot change a variable's type: if it held a request before, it holds one after. Only a DEFINE introduces a new binding that can be something else. So the delete is now gated on token.DEFINE, which is both more correct and simpler than what it replaces. Three controls added, and the two that would have caught this are the ones I had not written: a WithContext reassignment, and readers inside an if body and a for body - the last two because making every nested block its own scope is exactly the kind of change that could have started missing them. Thirteen controls now, six negative. Reverting to delete-on-any-assignment compiles and fails the WithContext leg. Claude-Session: https://claude.ai/code/session_01AUvLoXsKdS5sdpYju6rj4p * fix: make the body-reader scan conservative by design, and reduce filenames cross-platform (BUG-2803) Codex round 28. Two findings, and the first is the fifth consecutive round to find a FALSE NEGATIVE in the same instrument. ## Stop modelling scopes; change the error direction instead Rounds 24 through 28 each found another way the scope-modelling scan missed a real reader: a value-copied request, a plain r = r.WithContext(ctx), a mixed r, ok := ... that reuses an existing variable, and if/for/switch initialisers and case clauses whose scopes it did not model. Each fix closed one case and left another. That is a design telling me something, not a run of bad luck. The two error directions are not symmetric here. A false NEGATIVE hides a body reader, which is the entire thing this test exists to prevent. A false POSITIVE costs one human review and an accounting entry with a reason attached. So the scanner now OVER-APPROXIMATES on purpose: any name bound to an http.Request anywhere in the file counts for the whole file, aliases are followed to a fixed point, and names are never un-bound. Every scope-shaped false negative becomes structurally impossible. The cost is real and is now asserted rather than discovered: two controls that previously expected "not flagged" - a name shadowed by a closure parameter, and one rebound in a nested block - now assert CONSERVATIVELY FLAGGED, so the bias is on the record. Three of round 28's named misses are added as controls and pass: mixed short declaration, switch case, if-initialiser shadow. Sixteen controls, and the accounting test still passes against the real package - so the over-approximation costs nothing today. Exactness needs go/types with a real package load, which is a bigger instrument than this test warrants. The comment says so, and names the signal that would justify building it: an accounted entry whose reason is "the scan over-flagged". ## A filename safe on this OS is not safe on the consumer's filepath.Base is platform-specific, so on Unix it leaves a backslash alone - and the stored name is consumed cross-platform. A Windows client joining a stored "..\evil.png" onto a directory traverses upward. Reduced to the leaf under BOTH separator conventions. This normalises rather than refuses, which is less lossy than replacing the whole name and keeps round 27's point that a backslash is legitimate on Unix. Removing the reduction compiles and fails the new test. Claude-Session: https://claude.ai/code/session_01AUvLoXsKdS5sdpYju6rj4p * test(server): count readers per accounted file, add two missing reader methods, correct a false reason (BUG-2803) Codex round 29, and its central point was aimed at my REASONING, not my code. It was right. ## The over-approximation argument was wrong for per-file accounting I justified a deliberately conservative scanner by saying a false positive costs one review and one accounting entry. That is not what it costs. Once a file is listed, a NEW reader added to it is covered by the existing entry and the test stays green - so a false positive does not cost a review, it permanently blinds the list for that file. My own comment already recorded that hazard two commits earlier, and I argued past it anyway. The fix is to make the entry carry a COUNT of reader expressions rather than a yes/no. Adding a reader to an accounted file now changes the number and fails, so the entry must be re-read and its reason re-justified. It churns exactly when a body reader is added or removed, which is when a human should look. Demonstrated: inserting r.PostFormValue into handlers_tokens.go - an already-accounted file - is FLAGGED. Before this it was absorbed silently. The conservative bias stays, because the false-negative classes it eliminates are real and the count now removes the reason it was expensive. ## Two real reader methods were missing MultipartReader STREAMS the body and FormFile triggers multipart parsing of it. Neither was in the selector list, and FormFile is used in production in handlers_attachments.go - so the list was incomplete against code that exists, not hypothetically. ## A reason in the list was simply false handlers_tokens.go was accounted as "a nil/ContentLength check only - it never reads the body". It guards on those and then calls decodeJSON. A wrong reason is the same defect as a missing entry: both let a reader pass as reviewed. ## And I guessed the counts I wrote plausible numbers for the per-file counts and every one was wrong; the test reported the real ones on its first run. Same habit this branch keeps catching - a figure written from expectation reads exactly like a figure that was counted. They are measured now and the comment says so. Also recorded: my first verification script failed to apply its mutation and still printed a verdict, which I nearly banked. It now aborts unless the mutation is present in the file. Claude-Session: https://claude.ai/code/session_01AUvLoXsKdS5sdpYju6rj4p * fix(attachments,cli): close the closing round's two view defects, correct two overclaiming comments (BUG-2803) The closing enumeration (successor seat, per the lead's convergence ruling) returned two real attachment-view defects and two comments claiming more than their code delivers. Fixed here; the round's two design-scale findings are filed instead (BUG-2820 scanner precision via go/types, BUG-2822 Windows-unstorable filename forms). - Reverse MIME map: four ALLOWED spellings (text/xml, text/yaml, application/javascript, audio/webm) had no reverse extension because no extMIMEMap entry uses them as its value — `pad attachment view` wrote an extensionless temp file for exactly the types the delegation was built to fix. Population measured against the whole allowlist: these four, no more. An alias table closes them; TestEveryAllowedMIMEHasAnExtension asserts the class property over the allowlist (a future allowlist entry with no reverse extension fails), plus alias hygiene (allowed keys only, no forward-derived collisions, alias extensions must map to ALLOWED types so the table can never mint a refused extension). Mutation-verified: removing the alias application fails the test on all four types. - safeLocalFilename: a trailing dot survived every check and filepath.Ext("photo.") is "." — non-empty — so the MIME-extension fallback never fired and the temp file dispatched on no extension. Trailing dots are now stripped (cannot empty the name; dots-only names already returned early). The CLI guard also gains its first direct tests, including the backslash and traversal refusals that previously rode untested. Mutation-verified: removing the TrimRight fails both trailing-dot cases. - Two comment corrections, same defect class the accounting list itself names (a wrong reason reads as review): the handlers_cloud.go entry said bodyHasCloudSecret "restores" the body — it restores the first 64 KiB and drops the tail, a bound that file documents; and the accounting test's header said its scan "cannot be spelled around" while its own KNOWN LIMITS block lists the spellings that get around it (struct field, context value, type alias). The header now matches the limits block. * fix(server,attachments,cli): close closing-round-2's scanner blind spot and four stale comments (BUG-2803) Closing round 2 (successor seat) found no product defects; all four findings were in instruments and comments. Each verified against the code, then fixed: - The alias fixed-point resolved only identifier RHS (`req := r`), so a dereferenced copy (`c := *r; io.ReadAll(c.Body)`) was an invisible body reader — and unlike the disclosed type-level classes, this one was not in the KNOWN LIMITS block. The copy shares the Body (an interface holding the same reader). StarExpr operands now join the alias set; a new control pins the case. Mutation-verified: reverting the StarExpr handling fails the control. The type-level classes (struct field, context value, type alias) remain disclosed and are BUG-2820's territory. - The KNOWN LIMITS block said shadowed request names are "correctly ignored" while the controls deliberately assert they are conservatively OVER-FLAGGED — stale prose from the scope-aware era, falsified by the round-28 conservative flip that never touched those lines. - The reverse-map stability loop compared only PREFERRED entries across rebuilds; it now compares the entire map (sizes and every mapping) against the first build. Boundary stated honestly: every multi-spelling type today is preference-pinned, so the full-map comparison discriminates only when a future non-preferred multi-spelling entry appears — that future entry is what it guards. - Three orphaned/wrong comments: a `mimeForExt` doc block glued above ExtensionForMIME (the function it described is gone); the old hardcoded extension-table doc glued above safeLocalFilename (falsified by the delegation it predates); and two "120 chars" claims where the cap is 120 BYTES rune-safe via truncateBindableText — the consent form's maxlength=120 counts characters, so a multibyte name passes the client and is still truncated server-side, which is now what the comments say. * fix(cli,server,docs): close the attachment-view path escape, and closing-round-3's instrument and prose findings (BUG-2803) Closing round 3 found the branch's first product defect since round 17, in BRANCH-ADJACENT code the round-24 fallback extension work made reachable: the `pad attachment view` id fallback joined the RAW id onto its temp dir, and the client sent the id into the URL path UNESCAPED. An id is a CLI argument, but the documented agent flow harvests it from item content ("pad-attachment:" refs other workspace members write), so a traversal-shaped "id" could re-route the HEAD/GET to a different endpoint whose 200 then vouched for it, and the write escaped the temp dir. Both halves fixed and both mutation-verified through a new command-level test: reverting the fallback sanitize demonstrably wrote OUTSIDE the sandboxed TMPDIR; reverting the PathEscape put a raw "../../" on the recorded wire. - internal/cli: url.PathEscape(attachmentID) at both id-bearing client sites (HeadAttachment, DownloadAttachment — the enumerated population). - cmd/pad: the id fallback runs through safeLocalFilename, generic "attachment" when nothing survives; view's long help no longer claims the filename is used "without rewriting the extension" — it describes the reduction and the MIME-extension append, and says why the CLI is stricter than the server (the name is written to YOUR filesystem). - cmd/pad: attachmentViewCmd gets its first command-level test (CONVE-19 — the helper tests vouched for the component, not its wiring): disposition name, extensionless+MIME append, id fallback, traversal containment with a wire-escaping control, generic fallback. Instrument and prose findings, each verified before fixing: - The KNOWN LIMITS disclosure now names the ordinary alias forms the fixed-point does not walk (var-spec, call-derived, named results, range bindings) — they were in BUG-2820's filing but not in the in-file disclosure, which is what let the round read them as unfiled. The scanner itself deliberately does NOT grow another parser patch; go/types is the filed fix. - TestTextSafeHelpersAreUsedAtEveryCallSite pins EXACT occurrence counts (measured: 1 declaration + 4 call sites each) instead of a >=4 floor a removed call site could hide under. - middleware_mcp_audit: two stacked comment copies rested non-forgeability on "real names do not begin with (" — the exact reasoning round 25 retired; the const doc now points at auditLabel's namespace-reservation rule, which is what actually makes the marker non-forgeable. - docs/backup.md said repair is needed before "the export or migration" goes through, contradicting its own "exports fine" three paragraphs up — it is the IMPORT or migration that fails; the export succeeds either way. * fix(server,cli): decode chunked watch bodies, refuse dot-segment attachment ids, correct two texts (BUG-2803) Closing round 4 found one PRE-EXISTING product defect and one residue of the round-3 fix, plus two wrong texts. Each verified before fixing: - Watch creation gated its body decode on `ContentLength > 0`, so a CHUNKED request (ContentLength == -1) had its body silently DROPPED — the caller's predicate ignored, an unconditional watch created, 200 returned. The population of ContentLength gates in the package is exactly two: handlers_tokens.go already used the `!= 0` form, watches now matches it, with io.EOF tolerated so the documented no-body-is-valid contract holds for an empty chunked body too. Three handler-level tests discriminate the cases; the mutation (condition back to `> 0`) fails the two it should and passes the empty-body control. The accounting instrument then flagged the new `r.Body != nil` reference in the file — its exact job — and the file is now accounted with a measured reader count of 1. - url.PathEscape leaves exact "." and ".." UNCHANGED, so those two ids still reached the wire as live dot segments for a proxy or server to normalize — the escaping added in round 3 did not cover them. Both id-bearing client sites now share attachmentIDPathSegment, which refuses exactly those two values before any request (a real id is a UUID; the refusal cannot fire on one). Mutation-verified: removing the refusal fails the new subtest, which also asserts zero requests reach a recording stub. - The artifact rejection text said "NUL byte"; the same refusal fires for a NUL manufactured by a YAML escape during parsing, where no raw NUL byte exists — now "NUL character", in the handler message and the error var. - A test comment claimed the User-Agent reaches sessions.user_agent as text; sessions store only ua_hash, as the accounting list's own exemption states two hundred lines up. The sentence now agrees with it. * fix(attachments,server): remove a can't-fire MIME preference and a stale filename-guard sentence (BUG-2803) Closing round 5 is down to two P3 comment defects; both verified and fixed: - preferredExtensions "preferred" .md over a .markdown that has never been in the forward map — a line that cannot fire, the exact class this branch's own instruments hunt (the alphanumeric guard, the text/yaml preference, the charset loop). Entry removed; shortest-wins picks .md as the only candidate, unchanged. The preference-hygiene test now asserts every entry has a real competitor (>= 2 forward-map spellings), and the counterfactual — re-adding the entry — fails it. - The upload filename guard still carried round 26's "checking the trimmed form rather than listing spellings" sentence directly above round 27's code that does the opposite (exact "." / ".." comparisons, longer dot runs deliberately preserved). The stale layer is gone; the surviving paragraph already records why. * fix(server,docs): drop a dead test fixture, stop claiming the failing row is named (BUG-2803) Closing round 6 returned one P3 — TestDecodeJSONTrimsOnlyJSONWhitespace booted a full testServer it never used (`_ = srv`), dressing a direct decodeJSON test in router coverage it does not have. Removed. Its enumeration also re-read docs/backup.md against the code: "the failing row is named in the error" is true of neither leg — the import answers 400 naming the RULE it refused on (the NUL check is body-wide and knows no row), and `pad db migrate-to-pg` reports which WORKSPACE's copy failed. The doc now says exactly that, and that locating the value is manual until BUG-2810's preflight lands. * fix(server,cli): retire a stale byte-search claim, close two instrument gaps from closing round 7 (BUG-2803) Round 7 found no production defects; three instrument/comment findings: - artifactIsBindableText's doc comment still asserted the round-8 byte-search approach and that "the ambiguity cannot arise here" — directly above the round-9 body comment recording that assertion as simply wrong and doing the round-trip walk instead. The doc paragraph now describes the round trip and points at the body's history. - TestBodyReaderScanDiscriminates listed MultipartReader and FormFile in the scanner's selector set but had no control for either, so their removal from that list was undetectable. Two controls added. - The attachment-view test proved nothing about the MIME delegation: every case used image/png, which the OLD hand-rolled table also knew, so a stale local table passed. Two cases added — application/gzip (a type round 26 found missing from that table) must gain .gz, and blocked image/svg+xml must gain nothing. Mutation-verified: a stale-table mutant that answers only for png fails the gzip case. (First mutant attempt didn't build — unused import — and was not counted as a detection.) The per-file same-count substitution gap round 7 restated is declined as filed, not fixed: BUG-2820's filing already specifies per-call-site accounting via go/types as the fix that retires the per-file count workaround; the KNOWN LIMITS closing line now carries that ref. * fix(attachments,server): sweep two pre-BUG-2413 disposition comments, pin the manifest refusal to 400 (BUG-2803) Closing round 8 found no production defects; two evidence findings, verified then fixed: - Two comments still described the PRE-BUG-2413 disposition policy: the RenderChip mode doc said the HTTP layer serves every chip inline, and the read-path doc derived Content-Disposition from RenderMode. The live policy is the explicit fail-closed ServeInline allowlist — most chip types are served as "attachment". Both now say so and record the history. - TestImportBundle_RefusesNULInManifest accepted any status >= 400, so the documented 400 could decay into a 500 unnoticed. Pinned to http.StatusBadRequest.	2026-08-30 21:30:19 -04:00
xarmian	771ec5bbaa	fix(server): refuse invalid-UTF-8 and NUL query values at the transport (BUG-2784) (#1217) ... The query-string half of BUG-2782. A caller-supplied query value reached a Postgres text comparison, Postgres refused the parameter, and the handler answered 500 — the honest answer is 400, because the caller asked about something that cannot exist. ValidateQuery is a root-router middleware beside ValidatePath, refusing 400 invalid_query when a decoded query key or value is invalid UTF-8 or carries a NUL. Both share one predicate, bindableText. Measured on Postgres 17 (server_encoding UTF8) at 19330410: 8 GET endpoints x 54 parameter names — every name any handler reads — each probe from its own source IP because the api limiter is keyed on ip:. invalid-UTF-8 value: 276 x 200, 56 x 400, 100 x 500 -> 432 x 400 NUL value: 276 x 200, 56 x 400, 100 x 500 -> 432 x 400 control: 376 x 200, 56 x 400, 0 x 500 -> unchanged Zero 500s in the control, so the 100 are attributable to the value. The error is `invalid byte sequence for encoding "UTF8": 0xff (SQLSTATE 22021)`. WHY A TRANSPORT RULE, when BUG-2782 planned per-site validators on BUG-2774's validCursorID model. Reading the mechanism retired that plan: parseItemListParams folds every parameter it does not recognise into a field filter, so ?email=, ?type= and ?anything-at-all= reach a text comparison exactly as ?search= does — 98 of the 100 failures are those two endpoints. The set of names is unbounded by design, so there is no finite list of points to validate. WHY IT IS NOT A NARROWING of what callers may send, which was BUG-2782's objection. That objection is sound against a charset rule and does not reach this one: bindableText requires only valid UTF-8 with no NUL, every legitimate value here is text, and text is valid UTF-8 in any language. CONTRACT CHANGE: the timeline's before_id answers invalid_query rather than invalid_cursor, since the transport rule runs first. Same 400, same client-error contract, less specific code. validCursorID is NOT dead — two of its three call sites read ids from the item's own fields blob, which no request middleware sees — and that reasoning is recorded at the function definition where someone would land before deleting it. Keys are validated precautionarily: an invalid-UTF-8 parameter NAME did not reproduce a 500 in the sweep (7 x 200, 1 x 400), and why it survives is unread, so they are checked rather than assumed safe. Gates: go test ./... green; full Postgres suite -timeout=45m green (28 packages, own container, not the shared port); -race on internal/server green; lint 0 issues; vuln clean; gofmt clean. Mutation matrix 14/14, including the unwiring mutation run against the Postgres leg to confirm it fails with the ORIGINAL 500 rather than merely failing. Ten adversarial review rounds. The first two found a vacuous test (one item in the workspace meant a handler ignoring ?search passed it) and a self-contradicting proof. Later rounds found false statements, including one where my own sweep CLAIM was false. Two real defects were found AFTER the first MERGE verdict, which is why the rounds continued. Filed not folded: BUG-2803, an escaped NUL in a JSON body reaching the store on every JSON write path — a different surface needing decode-time rather than transport-level validation. Release note: invalid UTF-8 or NUL bytes in query parameters now return 400 instead of 500 on Postgres deployments. Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN	2026-08-27 10:35:39 -04:00
xarmian	e4e914d399	fix(server): reject path segments the database cannot be asked about (BUG-2782) (#1207) ... * fix(server): reject path segments the database cannot be asked about (BUG-2782) Every handler that resolves a workspace, collection, item, comment or attachment from a URL path segment passes that segment to the store verbatim, and the store binds it into a text comparison. Postgres refuses a text parameter that is not valid UTF-8 or that contains a NUL (SQLSTATE 22021 / 22P05); the driver surfaces that as a query error and the handler answers 500. SQLite accepts both bytes and matches nothing, so the same request is a clean 404 there — a dialect divergence that leaves the defect invisible to self-hosted installs and live on Pad Cloud. Measured before the fix, driving every route that carries a path parameter with one segment set to "bad-%FF-x" (247 probes, one per parameter position per method, real values elsewhere): Postgres answered 500 to 191 of them, SQLite to 0. After: 0 and 0, all 247 answered 400. Fixed with one root-level middleware rather than at ~112 chi.URLParam call sites, because this is a transport-level input rule and per-call-site fixes rely on every future route remembering. ValidatePath rejects a request whose percent-DECODED path is not valid UTF-8 or contains a NUL, before routing. It validates r.URL.Path rather than what chi hands the handler. chi routes on RawPath when non-empty and Path otherwise, and Go populates RawPath only when the client's escaping is not already canonical — Go escapes 0xff as uppercase "%FF", so the CANONICAL form any ordinary client emits is exactly the one that reaches the store decoded, and the lowercase "%ff" oddity is the harmless one. Validating the decoded path answers both identically and does not depend on chi continuing to prefer RawPath. It cannot refuse Pad's own URLs: store.slugify emits only [a-z0-9-], ids are UUIDs or hex, refs are a prefix plus digits. Valid non-ASCII segments pass through untouched — the database accepts them and they may legitimately name something. 400 rather than 404 because the request is malformed as a URI and the answer does not depend on whether anything exists, so it is not an existence oracle. Scope is the path; the query string is validated at its points of use, per BUG-2774's validCursorID. Claude-Session: https://claude.ai/code/session_01JVDBKbgn3Xt7ndW1YoYd8X * fix(server): the invalid-path rejection must look like every other API error (BUG-2782) Codex round 1, verified before acting on: ValidatePath runs on the root router, so its rejection short-circuits ABOVE the /api/v1 group's cors.Handler and jsonContentType and inherited neither. Measured — the 400 carried a JSON body sniffed as text/plain and no CORS headers at all, while a normal 404 on the same route carried Content-Type: application/json plus the full CORS set. On a cross-origin deployment (PAD_CORS_ORIGINS set) the browser refuses to let the page read a response with no Access-Control- Allow-Origin, so a debuggable 400 arrives as an opaque network error. Fixed without duplicating the CORS configuration: the group's cors.Handler is hoisted into one shared instance, the group mounts it as before, and ValidatePath serves its rejection THROUGH the same instance. Content-Type is set explicitly, since jsonContentType is mounted below and never runs for a rejection. Moving ValidatePath down into the group instead was rejected: two covered routes live outside it — the SPA catch-all and /api/v1/collab/{itemID} — and the mutant that makes that move is caught by exactly those two subtests. A genuine preflight (Origin + Access-Control-Request-Method) to an invalid path is answered 200 by the shared handler, the same as for any other path: a preflight asks whether the method and headers are permitted, not whether the resource exists. The real request that follows still gets the 400, and can now be read. Asserted rather than described. The new test compares each header on the 400 against the SAME route answered normally, for an allowed origin AND a disallowed one, so it pins parity with the API's own errors rather than a header list copied from a spec — and the disallowed-origin leg is what would fail if the rejection echoed origins the shared handler refuses. Mutation matrix, all nine verified to COMPILE first: dropping the CORS decoration and dropping the explicit Content-Type are each detected, and only by this new test. Claude-Session: https://claude.ai/code/session_01JVDBKbgn3Xt7ndW1YoYd8X * test(server): pin the ordering decision the path check makes (BUG-2782) Codex round 2, angle rotated to middleware contracts: a rejected request never reaches TokenAuth, SessionAuth, RateLimit or CSRFProtect, because ValidatePath sits on the root router above that group. The finding is factually right and the ordering is deliberate, but nothing in the diff said so and no test held it — which is the same defect shape as an undocumented invariant: true today, unenforced tomorrow. Verified rather than argued, because "bypasses the rate limiter" reads as a weakening and here the direction is inverted. Before this middleware, the same request ran SessionAuth — a store.ValidateSession round trip — then the limiter, then a handler whose query the database refused, and answered 500. It now costs a UTF-8 scan and a short JSON write with no database contact, so the unmetered path is strictly cheaper than every path the limiter protects. The answer is also constant for all inputs of this shape, independent of auth and of existence, so a flood learns nothing. And the limiter is a plain token bucket per key — no escalating ban, no durable block — so skipping it defeats no state that outlives the request. The alternative, metering it inside the /api/v1 group, trades this for a real coverage hole: the SPA catch-all and /api/v1/collab/{itemID} are mounted outside that group. The test floods 80 invalid paths from one IP (burst is 60), requires all 80 to be 400 and none 429, then requires a VALID request from the same IP to still get the resolver's 404 — proving the budget was untouched. It then asserts its own premise: the same volume of valid requests from a second IP must actually hit the limiter, because an inert limiter would produce an identical reading for the first half. Both mutants land where they should: metering the rejection fails at request 61 (burst 60 + 1, which independently confirms the constant cited above), and disabling the limiter fails the premise check rather than passing quietly. Claude-Session: https://claude.ai/code/session_01JVDBKbgn3Xt7ndW1YoYd8X * docs(server): correct four claims this branch's own measurements refute (BUG-2782) Codex round 3, angle rotated onto prose accuracy. No behaviour changes — every finding is a sentence that was stronger than what was verified, and in two cases stronger than data already sitting in this branch. 1. "It rejects exactly what the DATABASE rejects." Too strong, and inherited verbatim from validCursorID. Postgres refuses these two classes under a UTF8 database encoding; SQL_ASCII accepts the same bytes, and SQLite's sqlite3_bind_text accepts arbitrary sequences with NUL undefined rather than erroring. Pad neither creates nor configures that database — nothing issues CREATE DATABASE or sets client_encoding — so the encoding is the operator's. Now stated as what it is: the strictest reading, applied uniformly so the two backends stop disagreeing about the same request, measured against postgres:17-alpine at its defaults. A first draft of this correction replaced the overstatement with a NEW unverified claim ("the encoding Pad's migrations create"). Grepping for CREATE DATABASE found it only in test helpers. Fixing an unchecked sentence with another unchecked sentence is the same defect wearing the repair's clothes. 2. "Against unfixed code these are 500 on Postgres and 404 on SQLite." False for 56 of the 247 pre-fix probes, and my own sweep output said so — routes whose authorization or configuration gate answers before any store call (admin user lookup; attachments with no storage configured). Replaced with the pasted distribution: 500:191 404:34 403:12 401:4 503:4 400:2. 3. "Passed through untouched" oversold what this middleware guarantees. It does not touch a valid path, but chi still hands the handler the ESCAPED text whenever RawPath is populated: "caf%C3%A9" arrives as "café", the non-canonical "caf%c3%a9" arrives literally, and "%2F" never becomes a separator. Pre-existing chi behaviour, unaffected by this change, written down because the obvious reading is stronger than the truth. 4. "The request is malformed as a URI." It is not — "%FF" and "%00" are syntactically valid percent-encoded octets. The 400 is because the DECODED value cannot be a resource identifier here, which is the actual reason and a different one. Also reconciled the two probe counts that appear in this branch's history (111/94 GET-only, 247/191 all methods) so a reader meeting both does not have to guess which is wrong; they are one sweep at two widths. CONVE-23 sweep: finding 1 falsifies the same sentence in validCursorID (handlers_timeline.go, BUG-2774), which is where this branch inherited it. Corrected there too rather than left standing — the rule that comment describes is unchanged and still right; only its claim about the database was wrong. Claude-Session: https://claude.ai/code/session_01JVDBKbgn3Xt7ndW1YoYd8X * test(mcp): drive the in-process transport seam the path check walks past (BUG-2782) Codex round 4's exploration pointed at the door I had asserted rather than driven: the remote /mcp transport does not reach the server over a socket. HTTPHandlerDispatcher SYNTHESIZES an *http.Request and calls Handler.ServeHTTP in-process, so "the middleware covers every route" was a claim about a path this transport bypasses on its face. Driven, it is covered — and for a chain nothing in the tree stated: Handler is the *server.Server, chi's Mux.ServeHTTP runs mx.handler (middlewares + routeHTTP) on BOTH branches, and buildAuthedRequest forces the fresh-routing branch with a typed-nil RouteCtxKey. Every link is load-bearing and none was written down; this test is what notices if one changes. The counterfactual was worth more than the confirmation. Unfixed, an MCP agent that put an invalid byte in a ref got upstream_error on Postgres — whose hint says the failure is "usually transient, retry" — for an input that can never succeed. An agent obeying that hint retries forever. That is the retry-hostile misclassification family BUG-2675 added a code for, and this change removes an instance of it that nobody had noticed. Now validation_failed: the agent is told its INPUT is wrong. The first version of this test named upstream_error in its comment while running on SQLite, where unfixed gives item_not_found instead — an assertion that would have failed for a reason other than the one it named. The comment now states both backends separately and the fixture takes Postgres when PAD_TEST_POSTGRES_URL is set, so under make test-pg the dangerous half is what actually runs. Control leg included: a valid-but-absent ref must still return item_not_found, or a dispatcher that refused every ref would pass. Claude-Session: https://claude.ai/code/session_01JVDBKbgn3Xt7ndW1YoYd8X * chore(mcp): remove a throwaway probe that was committed by accident (BUG-2782) The probe that established the MCP seam behaviour was meant to be deleted once dispatch_http_invalid_path_test.go replaced it. The 'rm' was written as the first half of a compound command whose second half the tool layer REJECTED, so the whole command never executed — and a later 'git add -A' swept the file in. It duplicates the real test with printf-style output and no assertions. The rule this breaks is one I already hold: verify the mutation, not the report of it. I read 'rm -f X && cat > Y' as having removed X because I wrote it, when the command never ran at all. A rejected command and a successful one look identical in a transcript if you do not look. Caught by a Codex file listing showing an A for a file I believed gone. Claude-Session: https://claude.ai/code/session_01JVDBKbgn3Xt7ndW1YoYd8X * test: five corrections from the final review pass (BUG-2782) Codex round 5, judging the whole change. All five are mine; none needed a behaviour change. **A flake I built in.** The rate-limit test flooded 80 requests against a bucket of burst 60 — but a token bucket REFILLS while the loop runs, at 10/s here, so 20 tokens of headroom is 2 seconds of tolerance and a slow or -race'd run would admit all 80 and fail spuriously. The margin that matters is not flood-vs-burst but how long the loop must take for refill to cover the excess. At 400 requests that is (400-60)/10 = 34 seconds against in-process calls measured in microseconds: four orders of magnitude. The constant now carries that derivation, including the rate and burst it depends on. Both mutants still land, and metering the rejection still fails at request 61 — burst 60 + 1, unchanged by the larger flood. **A claim about MCP that JSON does not support.** The seam test's comment said an agent could put "a raw invalid byte" into a ref. Measured with encoding/json instead of assumed: raw 0xff / lone surrogate / truncated sequence → U+FFFD, valid UTF-8 raw 0x00 → JSON parse error the u0000 ESCAPE → a real NUL So exactly one of the five cases is reachable end to end over a JSON transport, and it is the one only the NUL half of validPathText refuses. The raw-byte cases stay — Dispatch is a Go API and the JSON decode is upstream of that boundary, so they assert the seam holds for callers that do not launder their strings through encoding/json — but the comment no longer offers them as evidence a JSON client can send them. **Two prose overstatements the earlier sweep missed.** The control test still said the rule rejects "only what the database rejects", which the previous commit had already established is false in the permissive direction. And TestValidatePathPostgresNoInternalError was described as reproducing the original 500 when it runs the FIXED server and can only ever observe a 400; the 500 lives in the counterfactual sweep and in the mutation matrix, and a test cannot both apply a fix and witness the bug. **One dead construction**, plus a smaller instance of the same habit: the MCP fixture built a SQLite store and discarded it in Postgres mode. My first attempt replaced the comment with one claiming the branch had been hoisted, and left the code as it was — writing the fix into the prose instead of the code, in the same hour I committed a message about not doing exactly that. Now actually branched. Claude-Session: https://claude.ai/code/session_01JVDBKbgn3Xt7ndW1YoYd8X * docs+test: seven more corrections, and one the emoji route earns (BUG-2782) Codex round 6, reading the three files as they now stand. All prose or assertion strength; one of them changes what the tests cover. **Pad DOES emit a non-ASCII path segment, and I said it never does.** `DELETE /workspaces/{ws}/comments/{commentID}/reactions/{emoji}` — the web client sends `encodeURIComponent(emoji)`. So the justification I gave for "it cannot refuse Pad's own URLs" was false in its premise while true in its conclusion, which is the worst combination: a reader checking the premise finds a counterexample and has no reason to trust the rest. It is also the best possible illustration of why the rule permits valid non-ASCII, so the control test now drives that ACTUAL route with a real emoji rather than relying on an emoji-shaped item slug — the claim is true by construction instead of by careful wording. A new mutant confirms the leg discriminates: a rule that rejects all non-ASCII (the plausible wrong version, not the absurd one) is caught there. **"The handler answers 500" was universal and is not.** Handlers that collapse a resolution error into not-found already answer 404 — the timeline handler's `err != nil || item == nil` is the example. My own measured distribution said so; the sentence did not. **"Self-hosted installs never see it" was wrong about the axis.** The split is by BACKEND, not deployment: a SQLite install never sees it, any Postgres install does — Pad Cloud and a self-hoster on Postgres alike. **A stale cross-reference of my own making.** The previous commit corrected TestValidatePathPostgresNoInternalError's claim to reproduce the 500, and left the sentence POINTING at it still saying it does. Fixing a claim at one site and leaving its pointer false is the CONVE-23 case in miniature. **The MCP test asserted too little.** validation_failed is how the dispatcher classifies ANY 400, so the test could have passed on a mapper-level refusal without ValidatePath running at all. It now pins the middleware's own message, which rides through on the hint. **Two overstatements in the same file.** "Exactly one case is reachable" should be one input CLASS (two cases carry a NUL). And the raw-byte cases do not cover "the stdio path": local stdio MCP is ExecDispatcher, which shells out to the binary and never touches this in-process door. Scope now says HTTPHandlerDispatcher and says what it does not speak for. Claude-Session: https://claude.ai/code/session_01JVDBKbgn3Xt7ndW1YoYd8X * docs(server): two qualifications the file already owed itself (BUG-2782) Codex round 7. Two, both narrow — the review is converging (7 findings last round, 2 this one), and both are internal inconsistencies rather than new ground. "Any Postgres install does" contradicted a qualification made forty lines lower in the same file, where validPathText spells out that SQL_ASCII Postgres accepts these bytes. Now says a Postgres install whose database encoding is UTF8, notes that this is initdb's default, and points at the place the qualification lives so the two cannot drift apart again. validCursorID's paragraph still described the 500 in the present tense, though BUG-2774 fixed it — it is the behaviour the guard PREVENTS, not what the endpoint does. My first attempt at this appended "past tense throughout this paragraph" and left the following sentence in the present tense, which is annotating a problem instead of fixing it. Rewritten so the tense carries the meaning without a note telling the reader to read it differently. Claude-Session: https://claude.ai/code/session_01JVDBKbgn3Xt7ndW1YoYd8X * docs(server): the SQLite half of the claim needed the same narrowing (BUG-2782) Codex round 8, one P3 and it is the mirror image of round 6's. I qualified "the handler answers 500" on the Postgres side and left the symmetric sentence — "the same request is a clean 404 there" — universal on the SQLite side, in the same paragraph. Not every request reaches a store resolution on either backend; a gate that answers first keeps its own status, and my own GET-only sweep recorded 102 x 404 alongside 5 x 403, 2 x 200, 1 x 401 and 1 x 503 on SQLite. Fixing one direction of a symmetric claim and leaving the other is a shape I have hit before and evidently do not catch by intention. The paragraph now says the divergence is in what happens once a value REACHES the store, which is the true and symmetric statement, with the distribution pasted. Claude-Session: https://claude.ai/code/session_01JVDBKbgn3Xt7ndW1YoYd8X	2026-08-25 23:20:53 -04:00
xarmian	effea01666	fix(server): diff the activity change list against the store's pre-image (BUG-2776) (#1206) ... handleUpdateItem built the activity's human-readable change list by diffing the item it read at the TOP of the request — before the permission checks, before the store's locks — against the row the store wrote. Anything a concurrent writer committed inside that window appeared in the difference and was stamped, with this request's actor and agent name, onto whoever sent the PATCH. Agent A sets status, agent B sets priority, and B's entry reads "status: open → done; priority: low → high" over B's name. Unlike BUG-2770, where a change went MISSING, here the timeline gains a confident false statement about who did what — and BUG-2770's debounce then merges that statement forward into the coalesced row, so it outlives the request that invented it. The fix is a carrier, not new machinery. The store has re-read the row under its own locks since TASK-2533, unconditionally, and diffs THAT snapshot for its status and assignment signals; the handler simply had no way to reach it. models.Item.PreUpdate hands the same snapshot back on the returned item — `json:"-"`, transient, populated only by the update path, following the LastMutation precedent. The title, role and assignment arms move onto it too, which makes the whole list committed-vs-committed: what this transaction wrote over, versus what it wrote. A missing pre-image DROPS the change list rather than falling back to the handler's stale read. The fallback is the defect wearing a warning: an entry that says nothing is recoverable, one that names the wrong author is not. The activity row is still written; a slog.Warn names the invariant. Also fixed, same lines: the title arm only recorded a rename when the field diff had produced nothing, so a PATCH that renamed AND edited a field silently dropped the rename. Two test seams are added (Server.afterItemPreRead, Store.afterItemPreLockRead), both nil in production and both documenting their reentrancy requirement. The second exists because of a mutation that SURVIVED the first matrix: with the rival's write landing before the store call, a pre-image taken from the store's pre-lock read is indistinguishable from one taken under the lock — the instrument could not see the difference the fix is about. That mutation now dies. Seven mutations aimed, six die; the seventh (aliasing the pre-image instead of copying it) survives by design and says so in the code. The Postgres leg earned its keep again: two store assertions compared `fields` blobs byte-wise, which passes on SQLite (TEXT, exact bytes) and fails on Postgres (JSONB, re-serialised) while proving nothing either way about which snapshot the blob came from. They compare by value now.	2026-08-25 20:16:18 -04:00
xarmian	b3c5ba95f5	style: gofmt the struct-field alignment the new Server field broke ... Caught by make lint, after I chained the commit onto the same line as the gate and shipped it on a failing exit code. Same shape as the rule about never piping a gate: read the exit status before the commit runs, not alongside it.	2026-08-23 01:39:48 +00:00
xarmian	a82bbd6b4f	test(server): the rate limit has to be tested where it is BOUND (BUG-2730, codex round 11) ... A one-survivor pass on the added tests: the announcer was tested directly and each handler test injected a single gap, so a handler that bypassed the limiter entirely and emitted sync_required straight from `case <-gaps:` passed everything — reopening the exact feedback loop the limiter exists to prevent. The same CONVE-19 shape as the wrapper: the component was vouched for, the binding was not. Both handlers now drive a burst through one connection and assert both halves of the bound: exactly ONE announcement inside the window, and one MORE after it. The second leg matters as much as the first — a handler that discarded the extras rather than latching them would satisfy the first and be this fix's own defect one layer up. The cooldown becomes a Server field so the test can narrow it. An integration test that waited out five real seconds per assertion would not have been written, which is how the gap got here.	2026-08-23 01:39:18 +00:00
xarmian	d936464736	fix(events): bound the mid-stream signal, and stop it moving existing alerts (BUG-2730, codex round 4) ... Three findings from the operator-at-3am angle, all real. A pub/sub outage on a workspace with a subscriber but NO replay buffer yet was silent. dropWorkspaceCoverage returned early before telling anyone, on the reasoning that there was no coverage to end — true of the BUFFER, and beside the point for the SUBSCRIBER, which has the largest possible hole and the least evidence of it. Live subscribers are now signalled on that path while the reset metric stays suppressed: the metric measures coverage endings, the signal measures clients who may have missed something, and those are different questions. The gap channel coalesces, which bounds the queue but not the loop: once the handler consumes a signal the next drop re-arms it, so a slow client could be answered with a delta sync, made slower, and answered again. Both handlers now share a gapAnnouncer that allows one announcement per connection per 5 seconds — a delta-sync round trip, not a tuning knob — and LATCHES rather than drops, so a gap inside the window is announced when the window closes. Suppressing it would be this fix's own defect one layer up. Folding mid-stream signals into pad_*_resume_gaps_total silently changed what every existing alert on those counters measures, and a mixed-version fleet would have reported two populations under one name for the length of a rollout. They go back to counting resumes; the new population gets pad_event_midstream_resyncs_total and pad_watchevents_midstream_resyncs_total, which count CLIENTS TOLD rather than causes — one instance-wide coverage loss moves them once per subscriber while the reset counter moves once, and that ratio is the fan-out an operator wants when judging a storm.	2026-08-23 00:56:23 +00:00
xarmian	9f88e94832	fix(events): a resume must not be answered from coverage we never had (BUG-2731) ... internal/events answered a Last-Event-ID resume with an empty-but-non-nil slice whenever the workspace's replay buffer could not speak to the span being asked about. The SSE handler reads that as "caught up", so the client sat on a live stream believing it was current while everything between its cursor and now was silently gone. COVERAGE. replayBuffer gains knownFrom: the lowest event ID from which this instance's coverage of a workspace can be vouched for. A resume from below it answers nil, which the handler already turns into sync_required. Covers a buffer that does not exist (cold start, restart, scale-up, or simply the first connection to a workspace on this instance), a buffer that exists but starts above the cursor — NOT full and NOT empty, reachable on any multi-instance deployment with no eviction and no restart — and a non-zero cursor from a previous incarnation of a single process. knownFrom here means RECEIVING-continuity, never ID-contiguity, and the defining comment says so with the measurement attached. internal/watchevents has a field of the same name that ALSO detects holes by noticing a non-consecutive ID; porting that would have been a serious regression, because this bus has a global counter and per-workspace buffers, so a workspace's buffer holds non-consecutive IDs by construction (four publishes alternating across two workspaces measure as W=[1 4], X=[2 3]). An ID-contiguity check would fire on nearly every append and turn every resume into sync_required — the false-positive inversion of this bug. LIFECYCLE. Coverage now ends where it really ends: - a stopped workspace subscription drops its replay buffer. Keeping it "in case they come back" looks like a free win and is the bug: events published elsewhere never enter it while it goes on looking complete. - subscriptions are generation-numbered, so a straggler from an ended subscription cannot re-create a buffer and vouch for coverage that ended with it — including the case where the workspace has already been resubscribed under the stale goroutine. - a pub/sub reconnect ends that workspace's coverage. PubSub.Channel resubscribes transparently, so a Redis failover left a hole the buffer had no idea about; the loop reads pubsub.Receive instead. It must RECOVER rather than exit — returning on a transient error would leave an instance publishing fine and receiving nothing — and it drops ONE workspace's buffer, since a dropped subscription says nothing about any other channel. Subscribers are indexed by workspace because the replay buffers moved under the same mutex (necessary for the straggler race): scanning every local subscriber under that lock would make one hot workspace the serialization point for every other workspace's fan-out and every resume. Also removes Publish's local-counter fallback on a failed INCR, which minted an ID from a process-local space and published it — every receiving instance reads that as the counter having been reset. It bought nothing: this bus has no local fan-out path, so an event that does not reach Redis reaches no subscriber here either. SIBLING. internal/watchevents had the identical cold-resume defect on its MemoryBus — its RedisBus guards it, MemoryBus reached the buffer directly — so a single-process instance answered a post-restart resume as caught up. Found by a cross-artifact review pass; the guard goes in `since` so both implementations inherit it, and is tested through SubscribeAndReplaySince as well as EventsSince because that is the path the handler uses. Refs BUG-2731	2026-08-22 14:23:40 +00:00
xarmian	461c5a3e3d	refactor: prune the claim surface, and turn one prose claim into a test ... The review could not converge on this diff's comments because each round of corrections re-expanded the surface it was reviewing — rounds 16 and 17 found errors inside 15 and 16's fixes. That is a production rate being measured, not a backlog being drained, so the treatment is to write fewer claims rather than review the same ones again. PRUNED, ~135 comment lines: process narration. "An earlier version said X", "found by mutation testing", "codex round N caught this", the scoreboards. Every one of those is already in a commit message, which is where the archaeology belongs; in the source they are claims a future reader has to verify, about a past that no longer exists. KEPT, because they earn it and a reader would otherwise re-derive them: metric semantics, reachability boundaries, what a test does and does not discriminate, why the obvious alternative was rejected, and the hazards that cannot be enforced in code. MOVED TO A TEST, per the rule this run earned the hard way: a comment asserting countable behaviour belongs in the suite. Two test comments in internal/watchevents relied on "this constructor waits for its SUBSCRIBE to be confirmed" — prose, and the same assumption applied to the OTHER bus (which subscribes asynchronously) is what made a namespace test flake. It is now asserted with no polling and no sleep, and the mutation that removes the wait fails it. That rule generalises and is why round 17's find mattered: "counts every unservable resume" was prose, so its falseness could hide a real metric gap. Prose is for claims that cannot be asserted. Claude-Session: https://claude.ai/code/session_01JVDBKbgn3Xt7ndW1YoYd8X	2026-08-22 05:09:29 +00:00
xarmian	a167005654	fix(server): register the stream gauge per metrics instance, and drop the comments the last refactor falsified (BUG-2726) ... Codex round 11, reviewing the collector conversion: - The registration guard was a sync.Once per Server, which is wrong in the direction that hides: SetMetrics can install a DIFFERENT registry, and the second one would silently never get pad_stream_connections_active. An absent metric is worse than a wrong one, because a dashboard with no data reads as a deployment with no traffic. Now tracked per metrics instance, which still cannot double-register (MustRegister panics) and follows a replacement. - Two comments and a test comment still explained the deleted observer machinery — "the discarded gate keeps its observer, so its releases keep driving the gauge" — which stopped being true when that machinery was removed one commit earlier. The reasoning they were making still holds for a different reason (a replaced gate is still reachable through Server.admission(), so the scrape reads the new gate and looks plausible), so they say that instead. And heldTotal's doc still opened with the old name. Falsifying your own comments while fixing something else is the failure this diff has hit repeatedly; catching it one round later is the loop working, not the comments being unimportant. Claude-Session: https://claude.ai/code/session_01JVDBKbgn3Xt7ndW1YoYd8X	2026-08-22 04:01:33 +00:00
xarmian	a51320c861	refactor(metrics,server): make pad_stream_connections_active a scrape-time collector (BUG-2726) ... Asking the round-21 question of my own work, because the signal was there: rounds 9 and 10 each found a defect inside this one mechanism, and my own record says several consecutive rounds finding defects inside one addition is a signal about the ADDITION, not a run of bad luck. The pushed gauge fired a callback on every admit and release. Round 9 found those callbacks running under the gate's lock — a deadlock for any callback that touched the gate. Round 10 found the fix reorderable: one admission could capture 1, a concurrent one capture 2, and the stale 1 land last, leaving the gauge permanently BELOW the real total, which reads to an operator as spare capacity that is not there. Each fix was correct. The mechanism was the problem. A GaugeFunc reading the total on scrape has neither failure mode and deletes both fixes with it: the observer field, the notify mutex, the snapshot closure, and the deferred-notify plumbing in acquire and release. The repo already had the pattern — RegisterDBCollector — and the argument that ruled it out for pad_redis_up does not apply here: reading an in-memory counter has no I/O, no side effects, and cannot make the value depend on who is asking, which is exactly what a Redis PING on scrape would have done. The tests got simpler and stronger in the same move. The concurrent one no longer needs a deliberately widened race window to be an instrument — that version's first attempt survived five runs against a broken build, which is the shape of a test that cannot fail — and it now scrapes WHILE the churn runs rather than only at rest. Claude-Session: https://claude.ai/code/session_01JVDBKbgn3Xt7ndW1YoYd8X	2026-08-22 03:55:14 +00:00
xarmian	640c5a02a1	fix(watchevents,server): four concurrency findings from codex round 9 (BUG-2727, BUG-2726) ... None of these can bite the shipped wiring today. All four are hazards on EXPORTED seams whose safety depended on a caller reading a comment, which is a deadlock with a delay on it. 1. Observer callbacks fired while the bus held its mutex, on both implementations. An observer that subscribed or published would deadlock the receive loop. Reports are now collected inside the locked section and flushed after it, via a defer registered before the unlock, so re-entrancy is a property of the BUS rather than a rule for implementers. The new test drives an observer that calls back into the bus and bounds itself, because the failure is a hang rather than a wrong value. Writing that test found a second hazard the fix cannot remove: an observer must not call a bus method that itself REPORTS (SubscribeAndReplaySince can raise a resume gap), because that is unbounded mutual recursion. Found by the test hanging when it tried exactly that. Now named in the contract. 2. RedisHealth's lifecycle raced: cancel was written by Start and read by Stop with no synchronisation, a repeated Start leaked the first loop and left Stop waiting forever, and Stop was unsafe on a prober that never started. All three closed, and the onProbe contract now says it must not call Stop — that waits on the goroutine it runs on. 3. The admission gauge callback ran under the gate's lock. Same fix, same reason. 4. SetSSELimits' doc oversold what it supports. It also writes plain Server fields that handlers read, so a live call races regardless of how careful the gate is. It is config-time; the in-place gate update exists so a late call does not silently over-grant capacity, not as a live-reconfiguration feature. Mutation-verified, and the first attempt at that matrix was wrong in a way worth recording: it moved the flush defer to a position that STILL ran after the unlock (defers are LIFO), so both mutations "survived" and looked like weak tests. The corrected mutation deadlocks the package binary, which is the shape the fix prevents. Also added the missing lifecycle test — Start's idempotency had no instrument at all, which is why that mutation survived honestly. Race detector clean: go test -race across internal/events, internal/watchevents, internal/metrics, internal/redisns, and the new internal/server suites. Claude-Session: https://claude.ai/code/session_01JVDBKbgn3Xt7ndW1YoYd8X	2026-08-22 03:43:15 +00:00
xarmian	9a94398bc2	fix(server): restore the readiness route I broke, and stop the tests bypassing it (BUG-2727) ... A scripted comment edit in the previous commit replaced every "/health/ready" in server.go — including the ROUTE REGISTRATION, inside the /api/v1 group. The real endpoint became /api/v1/api/v1/health/ready, so readiness 404'd at the path the docs, the k8s manifest and every runbook name. Codex round 7 found it. Two failures, and the second is the one worth fixing: - A blanket string replace on a file where the same literal appears as both prose and code. I verified the diff of the comments I meant to change, which is the half that was correct. - The health tests called srv.handleHealthReady directly, so the suite had no opinion about the URL at all and stayed green through a broken route. They now go through srv.ServeHTTP, and a new test pins all three health paths as mounted — with a negative control asserting the double-prefixed path is NOT served, so a router that answered everything could not pass. Mutation-verified: reintroducing the exact defect fails both. Also from round 7, both pre-existing and neither fixed here: - pad_eventbus_publish_total counts publish ATTEMPTS. Publish returns nothing, so a failed Redis publish is logged and still counted, and the counter climbs at its normal rate through an outage. The Help string and the wrapper now say so; the real fix is Publish reporting acceptance, which is BUG-2699's change one bus over. Filed as BUG-2732. - waitForDrain leaks its waiter goroutine when the drain times out. Accepted and now documented at the function: it runs only from Close, so it is one goroutine seconds from process exit, and a cancellable wait would mean tracking every renewal for a benefit that expires with the process. And the admission gate is now reconfigured IN PLACE rather than replaced (round 7 P2). Replacing it left the old gate holding every open connection's slot while the new one started at zero, so those connections stopped counting and the process silently over-granted capacity. Worth recording how the test for it landed: the first version asserted the GAUGE and survived the mutation, because the discarded gate keeps its observer and its releases keep the gauge looking right while the budget is wrong. The visible signal stayed correct and the invisible one went wrong — so the test now asserts that a held slot still refuses the next connection. Claude-Session: https://claude.ai/code/session_01JVDBKbgn3Xt7ndW1YoYd8X	2026-08-22 03:08:50 +00:00
xarmian	a790810bd6	docs: close the cross-artifact gaps codex round 6 found (BUG-2724, BUG-2726, BUG-2727) ... The angle no earlier round probed: which artifacts a human or agent CONSUMES should have changed and did not. Five, and the pattern is the one my own record keeps naming — the caveat existed in the artifacts I was editing and not in the ones that get read. - .env.example had neither new variable and still described PAD_SSE_MAX_CONNECTIONS with its old single-endpoint meaning. It is the file an operator copies; docs/deployment.md being right does not help someone who never opens it. - docs/deployment.md called the readiness endpoint /health/ready. The route is /api/v1/health/ready, so every instruction to go read the new redis block pointed at a 404. Corrected there and in four code comments, and the Health Check section now actually shows the three endpoints, the healthy payload, and the degraded one — it previously demonstrated only /api/v1/health, which is the build-info endpoint and says nothing about readiness. - CLAUDE.md listed /api/v1/events and not /api/v1/events/stream at all, so the endpoint this unit bounds was undocumented in the file agents read first. Added, with the limits and the 429 contract. - `pad watch --stream --help` said silence means "no workspace linked or padd unreachable". A capacity refusal now produces the same silence through the same backoff, so the help was enumerating a set that had quietly grown. - The plugin skill told agents "silence means nothing changed" — now false in the same way, and worse, because an agent repeats it to a user as though the quiet were evidence. Rewritten to say what silence does and does not prove. The plugin monitor description had the same enumeration and got the same fix. Checked rather than assumed: there are two SKILL.md files, and only the plugin copy carries a notifications section — the embedded one has no monitor guidance to correct. NOT changed, and raised with the lead instead: deploy/k8s/deployment.yaml points both probes at /api/v1/health, so the readiness endpoint is never consumed. Fixing it is right but it changes rollout behaviour for anyone using the shipped manifest (a database blip would start pulling pods from the load balancer), which is a deployment-posture call rather than part of this unit. Claude-Session: https://claude.ai/code/session_01JVDBKbgn3Xt7ndW1YoYd8X	2026-08-22 02:45:53 +00:00
xarmian	03518466ab	fix(server,metrics,docs): five codex round-2 findings (BUG-2724, BUG-2726) ... Round 2 probed angles round 1 did not: rolling upgrade and rollback, request cancellation, and whether any operator-facing text now contradicts the code. Four of the five were the latter. 1. The admission slot was held through the Redis presence cleanup. Defers run LIFO, so the acquire-site release ran LAST — after Remove's round trip, bounded by presenceOpTimeout (5s) and a wait on the renewal goroutine. A reconnect arriving inside that window could be refused by a bound the connection had already stopped consuming, and the window is widest during a Redis outage, which is when clients reconnect most. A second deferred release, registered later so it runs first, closes it; the acquire-site defer stays as the safety net for early returns, and release is idempotent so deferring twice releases once. 2. pad_sse_connections_active is written by the events.EventBus wrapper, so it has only ever counted the workspace stream. That was every SSE connection Pad had a limit for until this branch; it no longer is, so an operator watching it against the global limit would be reading one endpoint's share of a two-endpoint budget. Adds pad_stream_connections_active, driven by the admission gate itself, and both Help strings now name their population. Wired from either SetMetrics or SetSSELimits (either can land first) and from the lazily-built gate, each covered by a test — a gauge stuck at zero while streams are held is the same shape of lie as a metric that is not registered at all. 3. The limits are enforced in-process and the docs called them "Global". With the shipped k8s manifest's two replicas, 1000 admits ~2000 and a user can hold 50 per pod. Documented as per-instance, with the multiply-by-replicas note and a pointer at the new gauge. 4. A namespace cutover partitions a rolling upgrade — namespaced and un-namespaced replicas are two installations for the length of the rollout — and rolling back with the variable still set silently restores the split. Both now stated, with the env var and the binary having to move together in both directions. 5. Client resync across that cutover is honest on the watch stream (the epoch key detects the changed id space) and SILENT on the activity stream, whose cold replay buffer answers a resume as "caught up". Documented, and filed as BUG-2731 rather than fixed here: it is pre-existing, fires on any replica restart, and the minimal fix changes reconnect behaviour for every deployment, which wants a ruling rather than a quiet patch. Claude-Session: https://claude.ai/code/session_01JVDBKbgn3Xt7ndW1YoYd8X	2026-08-22 02:05:01 +00:00
xarmian	720b792176	feat(server,config): bound the watch-events stream, with one budget across both SSE endpoints (BUG-2726) ... GET /api/v1/events/stream had no concurrent-connection limit of any kind. PAD_SSE_MAX_* gated only /api/v1/events, and the API rate limiter caps how FAST connections are opened, not how many are HELD — so one authenticated user could hold arbitrarily many streams, each costing a goroutine, a bus subscription and, since BUG-2698, a presence registration in shared Redis. The bound is a process-wide admission gate rather than a second per-bus limit. Each bus can bound its own subscribers atomically and events.EventBus already does, but neither can bound the two together, and a held connection costs the same process resources whichever endpoint opened it. A global limit on one bus would have let a user exhaust the machine through the other while every configured limit still read as satisfied. So PAD_SSE_MAX_CONNECTIONS now covers BOTH endpoints and is passed to the events bus as 0. That is a deliberate re-point of an existing knob, ruled rather than assumed: an operator who tuned it for one endpoint is now bounding both and may reach the limit sooner. A knob that silently bounded half the connections it named is the worse failure — invisible — where this one announces itself and is tunable. A startup log line reports the effective limits and which endpoints each covers, so the change is visible without reading release notes. New PAD_SSE_MAX_PER_USER (default 50) applies to both endpoints. The global bound alone lets one user exhaust the process for everyone, which the per-workspace limit cannot prevent — the watch stream has no workspace to count against. Per-workspace stays /api/v1/events-only for the same reason. Refusal is 429 sse_limit_exceeded, matching the existing endpoint. The CLI monitor folds any non-200 into its backoff ladder (linear, 5s base, 5min cap, reset on connect), verified rather than assumed, so a refused stream backs off instead of spinning. Deliberately NOT a registry-side cap: PR #1175 added one in review round 17 and removed it in round 21, because it bounded one of three resources a held stream consumes, was never hard (admitted renewals must bypass it), and cost delivered_sessions its honesty. The admission check is upstream of all of that — refusing costs one connection instead of making a live session untargetable. Claude-Session: https://claude.ai/code/session_01JVDBKbgn3Xt7ndW1YoYd8X	2026-08-22 01:31:33 +00:00
xarmian	8dea9abca3	feat(watchevents,metrics): operational observability for the Redis notification bus (BUG-2727) ... The watch bus detects four conditions an operator would want to alert on — a notification dropped for a slow local subscriber, a gap in the received id sequence, an id-space reset, and the receive loop stopping — and until now reported all four to slog and nowhere else. Log lines are not alertable without someone already looking, and the last of the four was not even logged: the loop returned silently, leaving an instance that publishes fine and receives nothing indistinguishable from a quiet workspace. Adds watchevents.Observer, an adapter seam rather than a bus wrapper. The events.EventBus wrapper shape does not work here: every condition is detected on the RECEIVE path, inside the bus, and is invisible at the Bus interface — a wrapper can count publishes and subscribers, but not a notification that never arrived. Two corrections to BUG-2727's filing, both verified against go-redis v9.22.0 rather than assumed: - Its proposed fix — "re-subscribe rather than exiting where the cause is recoverable" — would be dead code. PubSub.Channel's message channel is closed ONLY on pool.ErrClosed; every other receive error is retried indefinitely, and a health-check goroutine pings every 3s and reconnects on failure. So go-redis already does the re-subscribing. The exit gets an ERROR log and a counter instead, which is what the condition actually needs. - The genuinely silent path is go-redis DROPPING messages when a subscription's 100-deep buffer stays full past its 60s send timeout, logged only through go-redis's own logger. Pad cannot count that directly, so it is reported by its CONSEQUENCE (a sequence gap) and its cause is made visible by routing go-redis's logger into slog. Observer's doc comment states that boundary, so a gap is not misread as evidence of any particular cause. Session presence gets the same treatment for the same reason: it is fail-soft everywhere by design, so its failures have no user-visible signal beyond a push that quietly reaches fewer sessions than it should. The renew counter is deliberately NOT throttled where its log line is — throttling the metric would make it under-report during the incident it exists for. Tests assert the CONDITION increments the counter, not that the counter exists, and each asserts its own premise first (a healthy subscriber reports nothing; contiguous ids report nothing; a cold start reports nothing) so a bus that reported on every notification could not pass. The receive-loop test drives the real closed-client condition rather than calling the reporter. Claude-Session: https://claude.ai/code/session_01JVDBKbgn3Xt7ndW1YoYd8X	2026-08-22 01:21:50 +00:00
xarmian	6a37512227	feat(server): outbox drain — webhooks delivered from the choke point (TASK-2714) (#1173) ... * test(store): pin the events/1 taxonomy as an independent copy (TASK-2714) TestCanonicalEventsAreFullyDeclared iterated kernelevents.Canonical() and asserted each entry resolved something non-empty. That check cannot fail for any table the compiler accepts: eventSpec requires both fields, so a corrupted table — an entry deleted, an entry added, item.deleted quietly rebased onto the ref-only payload — passed its own validation. A test that agrees with whatever the table says is not a test of the table. The sixteen name/subject/family triples are now written out as literals, so the test DISAGREES with the table when the table moves. The wire strings behind the name constants are pinned separately, because the triple map is keyed on literals and a renamed constant would otherwise slip through as long as the constant and the table moved together. Ordered as this unit's first commit because TASK-2714 edits that table (the handler-path bulk mapping): an independent copy earns its keep at the moment of the edit, not before. Mutation matrix, 4/4 caught: drop member.joined (17 -> 15 count mismatch and a missing-name error), rehome item.deleted onto ref_only (family mismatch), rename ItemMoved's wire string to item.move (constant leg), add an undeclared item.frobnicated entry (count + undeclared-name + non-canonical legs). The fourth reported "survived" on its first run because the sed never matched the table's alignment — the mutation was verified present in the file before the result was believed. TASK-2714 requirement 4 (lead pass on #1172). * feat(store): max-age prune for undispatched outbox rows (TASK-2714) Requirement 3's missing half. PruneDispatchedOutbox filters on dispatched_at IS NOT NULL, so a row that can never be delivered — a workspace whose only webhook was deleted, an endpoint that 4xxs forever — is unreachable by it and keeps its frozen payload indefinitely. That matters because SPEC-3 makes payload privacy TEMPORAL. An outbox payload is a frozen snapshot and account deletion's de-identify posture reaches only live rows, so the retention window is the whole privacy claim; a window only one of its two halves can close is not a window. The trade is stated in the doc comment rather than left to be inferred: at-least-once holds WITHIN the retention window and not past it, which is why the caller's max-age must be far larger than any retry schedule. Deleting rather than stamping the rows dispatched is deliberate — a dispatched stamp would be a lie in the durable record, and this table is the only evidence of what the kernel emitted. Mutation matrix, 2/2 caught: drop the dispatched_at IS NULL clause (prunes the aged DISPATCHED row too, handing retention two owners with different windows), drop the occurred_at cutoff (prunes a young pending row a retry is still owed). The test asserts its own premise — all three seeded rows are confirmed present before the survivor checks, which would otherwise pass for a reason unrelated to the prune. No caller yet: the drain loop wires it up in the next commit. * feat(events): derive SSE names from the taxonomy; retire item.updated_with_comment (TASK-2714) SPEC-3 §"the choke point owns the canonical→surface name mapping". SSE's snake_case vocabulary and the webhook dot-form vocabulary drifted because nothing tied them together — each was hand-passed at its own call sites. This ties them. v1.5 pins what "derive" means: NAME derivation, not delivery path. SSE stays direct-published at the mutation site, because it carries request-scoped attribution (Actor / ActorName / Source) that a frozen outbox payload deliberately does not hold; only its NAME now comes from the taxonomy. Moving SSE behind the drain is TASK-2722. - eventSpec gains an `sse` field — ONE table, not a second map, for the reason round 11 of the last unit established: a separate map can disagree with the first and fails open exactly when it matters. Empty is a real value (attachment, member and pack events have no SSE surface) and SurfaceSSE reports false for it, so silence can't be mistaken for a name. - Several canonical events derive the SAME SSE name — status_changed and moved both surface as item_updated — because the SSE vocabulary is coarser than events/1 and the UI never distinguished them. The finer name is what the webhook wire and bindings get. - The 12 canonical SSE publish sites take their names from derived package vars, resolved AT INIT. Every call site is a compile-time constant, so a missing surface is a startup panic rather than a per-request decision between "log and drop" and "publish under an empty name". - handlers_item_links.go keeps the events.ItemUpdated literal, commented: link mutations are silent in events/1 (v1.5), so there is no canonical name to derive from. TASK-2723 carries link.created / link.removed. - item.updated_with_comment retired (v1.2, Dave's ruling). One producer deleted; the events.ItemUpdatedWithComment constant deleted with it — it had no producer and no web consumer (grepped .go/.ts/.svelte), so leaving it would leave a name a future publisher could reach for. The compat guard is what makes this a refactor rather than a wire change: TestDerivedSSENamesMatchTheLegacyWireVocabulary asserts each derived name equals the events.* constant clients are pinned to. A derivation producing "item.created" or "item_deleted" would break the live UI while every other Go test still passed. Mutation matrix, 3/3 caught: rename item.deleted's SSE surface to item_deleted (both the taxonomy test and the compat guard fail), split item.moved onto its own SSE name (same), make SurfaceSSE return (spec.sse, ok) so no-surface events fail open (the taxonomy test's silence leg names all four). Running total 9/9. go test ./internal/server ./internal/store ./internal/events: all green. * feat(webhooks): synchronous DeliverEvent seam with per-endpoint outcome (TASK-2714) Requirements 1 and 2. Dispatch returns once its per-hook goroutines are spawned and reports nothing, so a drain built on it would stamp rows dispatched while the HTTP requests were still in flight — losing exactly the events the outbox exists to make unlosable. DeliverEvent blocks and tallies. - Delivery carries WorkspaceID / EventID / Event / OccurredAt / Payload. OccurredAt is the EVENT's timestamp, not dispatch time: SPEC-3 pins time-relative binding predicates to it, so stamping time.Now() would make every consumer's notion of when a mutation happened depend on how backed up the queue was. Payload is json.RawMessage — []byte would base64 the snapshot into a string that is valid JSON and completely unusable. - WebhookPayload gains ID, the consumer dedupe key SPEC-3 §Delivery guarantees already told consumers to use. Before this, that instruction named a field nobody could see. omitempty, because the "webhook.test" ping is not a kernel event, has no outbox row, and must not invent an id. - DeliveryOutcome counts rather than a status, because one event fans out to N endpoints and the answers differ. Three distinctions the drain branches on: Matched==0 is SUCCESS (a webhook-less workspace is owed nothing; reading it as undelivered would back up every event in every such workspace until retention deleted it); Permanent does not hold the event pending (re-sending to an endpoint that will reject it again costs the queue its progress); Transient does. Retryable() states the ack rule once instead of letting each caller re-derive it. - A returned error is reserved for the SERVER's failures — listing hooks, marshalling. Those must not ack: nothing was attempted, so the event is still owed in full. - Dispatch keeps its async shape for its one remaining caller and says so. deliver() now returns the outcome it always computed; the async path discards it. Mutation matrix, 6/6 caught: stamp dispatch time instead of occurred_at; drop the envelope id; pass the payload as []byte (base64); deliver asynchronously and assume success (the synchronous leg names it exactly); count a permanent rejection as transient; swallow a store failure into a zero outcome (the test prints the outcome that would have acked an undelivered event). Running total 15/15. go test ./internal/webhooks green. * feat(store): batch_id correlation for handler-path bulk mutations (TASK-2714) F2's write half. A lane-wide bulk action is a handler LOOP over per-item store mutations with no enclosing transaction, so each member writes its own canonical outbox row — which is what keeps SPEC-3's per-member binding evaluation free, and also means that without a marker the drain would put 200 item.deleted events on the webhook wire for a 200-item lane archive: exactly the flood TASK-1668's batch event exists to prevent. RECORDED, NEVER INFERRED (SPEC-3 v1.5). The schema-free alternative was grouping pending rows by workspace and a time window, which would fold two unrelated single updates into somebody's bulk event whenever they landed in the same tick. A wire event saying "these five items changed together" is only true if something recorded that they did. - migrations 082 / pgmigrations 060: nullable event_outbox.batch_id, no FK (a batch is not a row anywhere, it is a name the handler minted), plus a partial index on the pending set. - store.MutationOption / WithEventBatch: variadic, because every existing call site is a single-item mutation with nothing to declare and making all of them pass a zero value would bury the one case that matters. - The handler mints one id per bulk OPERATION, before the loop and unconditionally — deciding mid-loop whether a run "counts as" a batch would make the correlation depend on how far the loop got. POPULATION CORRECTED: my escalation said four store methods; it is FIVE. archive (DeleteItem), restore (RestoreItem), move (MoveItemWithPreCheck), field update (UpdateItemWithPreCheck) and assign (UpdateItem) are the complete set of mutating store calls handlers_items_bulk.go makes — restore was the one I missed, which is CONVE-18's exact lesson arriving one level up. The test drives all five rather than sampling, because the failure is per-method: a signature that accepts the option and never threads it compiles, passes everything else, and silently un-batches one of the six bulk verbs. Mutation matrix, 5/5 caught across the four distinct emit sites: drop the stamp on the update path (both Update legs fail), on delete, on restore, on move. The delete mutation first read as SURVIVED — it had made the package fail to BUILD (opt then unused), and the grep for test-level FAIL lines printed nothing. The compiler catch is the stronger result, but the instrument mis-reported it, so it was re-run with opt kept alive and the test named it directly. go test ./internal/store ./internal/server green. * test(server): anchor the SSE compat guard to the client's literal strings (TASK-2714) The guard compared the derivation against events.* — the Go side. A coordinated rename of the taxonomy AND the constants passes that, and is exactly the change that breaks the browser: the client is pinned to the STRINGS, in web/src/lib/services/sse.svelte.ts's ITEM_EVENTS. The wanted column is now a literal copy of what the client listens for, with the file named. events.* is asserted alongside as a second leg, so a drift between the Go constants and the client is attributed rather than merely reported. Same disagree-with-the-table principle as the taxonomy test, one layer out: this file has to be edited by hand when the wire vocabulary intentionally changes, and that edit is when someone goes and changes the client too. Mutation matrix, 2/2, each hitting only its own leg: rename events.ItemCreated to the dot-form with the taxonomy untouched (drift leg fires), and make the taxonomy publish the dot-form on SSE (browser leg fires). Running total 22/22. Lead's catch on the day-49 review of commit 33662da0. * feat(store): outbox claim protocol with lease and whole-batch claiming (TASK-2714) F3. Every instance of a cloud deployment runs the drain, so an unclaimed pending row is delivered once PER INSTANCE by construction. SPEC-3 permits duplicates — consumers dedupe on the event id — but "occasionally, after a crash" and "always, once per instance" are different promises, and only the first is one a consumer can budget for. - migrations 083 / pgmigrations 061: claimed_at + claimed_by, dialect-uniform conditional UPDATE (BUG-2415's orphan-GC protocol). Postgres FOR UPDATE SKIP LOCKED plus a separate SQLite path would be two implementations of one behaviour, only one of which runs where it matters. - claimed_at doubles as the lease: an instance that dies between claiming and dispatching must not strand its rows, and at-least-once is exactly what makes re-claiming safe. - BATCHES ARE CLAIMED WHOLE, past the limit. The limit is a throughput knob; letting it split a batch would make one bulk operation arrive as two wire events each reporting a partial member count. - MarkOutboxAttemptFailed RELEASES the claim rather than letting it expire. A transient failure means the event is owed and nothing is in flight; on a single-instance deployment the lease would otherwise be the only reason a retry ever waited. THE EXCLUSIVITY TEST WAS VACUOUS AND THE MATRIX CAUGHT IT. Removing the availability predicate from the claim UPDATE left it green: the candidate query already filters held rows, so single-threaded the end state is identical (CONVE-12 — another mechanism produces it). That implementation double-claims every row two instances select in the same moment, which is the entire bug. claimOutboxIDs is now split out so a test can drive the arbiter with a deliberately STALE candidate list, and the same mutation fails it by name. Mutation matrix, 4/4: drop the UPDATE's availability predicate (survived the first test, named by the race test); drop the batch expansion; keep the claim on a failed attempt; and the vacuity finding above. Running total 26/26. go test ./internal/store green. * feat(server): the outbox drain — claim, fold, deliver, retain (TASK-2714) The half of SPEC-3's choke point that turns stored events into delivered ones. 2a built the fill side; until this, the table filled and nothing read it. NOT STARTED YET, deliberately: the hand-called dispatchWebhook sites are still in place, so wiring the loop here would double-deliver every canonical event. Starting it is the next commit, together with deleting them — the unit's behaviour edge, kept as one reviewable diff. - Two declared payload shapes for item.bulk_updated (SPEC-3 v1.6). The store-side single-tx producers know every member at write time and embed snapshots; the handler-path HEADER knows the operation, the shared delta and the member refs, with snapshots living on the members' own rows. Declared rather than loosened: stuffing placeholder snapshots to satisfy a single-shape check would be a lie in the durable record, and dropping the gate would drop it on the one event with two producers. - EmitBulkHeaderEvent + bulkEventDelta: the delta is captured where it is KNOWN. By the time the drain sees member rows they carry post-mutation snapshots, and a diff of a snapshot against nothing is not a delta. - The fold: header plus whatever member rows of that batch are still undispatched. Members whose header is not in this claim deliver individually — not a fallback, the defined behaviour for the window between the loop committing and the header landing. batch_id is on the wire so a consumer can tie the singles to the batch. - Per-unit acking: a folded batch is many rows and ONE delivery, so a partially acked batch would re-deliver. - Retention runs every tick, both halves. The undispatched one is the privacy bound; PruneDispatchedOutbox looks like it covers retention until you notice which rows it can never see. TWO REAL BUGS THE TESTS FOUND, both in this commit's own code: 1. DEFAULTS APPLIED ONLY IN StartOutboxDrain. A tick reached directly ran with a ZERO undispatched max age, making the retention cutoff `now` and deleting the entire pending set on its first pass. Every test does this, and so would any future admin-triggered drain. Fixed by construction — one resolver both entry points call — with a refusal guard behind it. 2. THE GUARD'S FIRST TEST WAS VACUOUS AND PASSED WITH THE GUARD REMOVED. RFC3339 is second-granular, so a row written in the same second as a zero-window cutoff survives `occurred_at < cutoff` either way: the end state was reachable by another mechanism, and that mechanism was the clock. runOutboxRetention now returns its refusal so the test asserts the refusal rather than the survival, plus a positive control. Mutation matrix, 7/7 after the instrument fix: ack regardless of outcome; ack only when something succeeded (permanent failures would wedge the queue); fold without acking its members; drop members that have no header instead of delivering them; remove the retention guard; remove the resolver's max-age default. Two mutations initially read as survivors — one had failed to build, one met the vacuous test — and both are recorded above rather than counted as passes. Running total 33/33. go test ./internal/server ./internal/store green. * feat(server): deliver canonical webhooks from the drain, not from the handlers (TASK-2714) The unit's behaviour edge, kept as its own commit. The drain starts, and the nine remaining hand-called dispatchWebhook sites go: comment.created, comment.updated, item.created (x2 — plain and copy), item.updated, item.deleted (x2), item.moved, item.bulk_updated. Each was verified to have an outbox producer before its deletion, not assumed to. The Server.dispatchWebhook helper goes with them — it had no production callers left. Three copy tests used it as a probe and now call s.webhooks.Dispatch directly, which is what it did. WHAT CHANGES ON THE WIRE, stated plainly because "no behaviour change" would be false here: - TIMING. Deliveries were post-commit and inline; they are now up to one drain interval (5s default) later. In exchange a delivery survives a crash: the event is committed with the mutation it describes. - THE DISJOINT-DELTA RULE ARRIVES (SPEC-3 v1.3, ruled in 2a). A bare status flip now emits item.status_changed ONLY, where the hand-call always emitted item.updated. A mixed update emits both. This was ruled while the webhook surface has no known consumers; it is the same grounding as the v1.2 fold. - PAYLOADS. The envelope gains `id` (the dedupe key SPEC-3 already told consumers to use), and `timestamp` is now the event's occurred_at rather than dispatch time. Item snapshots come from the in-transaction read-back and are PII-scrubbed — the joined assignee name and email are gone, deliberately (see scrubItemPII: a frozen payload outlives account de-identification). - item.bulk_updated carries batch_id, the shared delta, and the member snapshots folded in from the member rows. Two copy tests needed real changes, not cosmetic ones: the DR-14 emission matrix they assert (which workspace hears what) is unchanged, but nothing arrives until a drain pass runs, and the fixture's own backlog — member joins, filler items — would otherwise be reported as the copy's output. The observer now drains once before the receivers are registered, which is what its "baseline" has always meant, and drainWebhooks runs a pass before collecting. go test ./internal/server ./internal/store ./internal/webhooks green. * fix: codex round 1 — unbatched bulk verbs, member dedup, comment overclaims (TASK-2714) THE P1, and it is CONVE-18 for the third time in this unit: batchID was threaded into the bulk helpers' SIGNATURES but not passed at three of the six store CALLS (set-priority/move-status via UpdateItemWithPreCheck, tag/untag via UpdateItem, move via MoveItemWithPreCheck). Those verbs' member rows stayed unbatched while the header was still written — N individual wire deliveries plus a header claiming they were a batch. My store-level test could not see it. It called the five store methods directly with the option, so it proved the option WORKS and said nothing about whether the handler passes it. TestBulkItems_EveryVerbStampsOneBatchID drives all seven legs through the HTTP handler and asserts every row of the operation shares one non-empty batch id with exactly one header. Reverting one stamp fails it by name (set-priority and move-status both). Writing that test also surfaced two legs that asserted nothing: untag and assign were no-ops in the fixture (no such tag; nothing assigned), so no member events existed at all. Both now perform real mutations, and the leg fails if fewer than two rows appear. Also from round 1: - FOLD DEDUPS MEMBERS. The disjoint-delta rule means one member can write two or three rows (a move that also changes status emits item.moved AND item.status_changed), so the folded payload listed the same item repeatedly while `count` reported ITEMS — the wire event contradicting itself. Keeps the LAST snapshot per id; an unreadable snapshot is kept rather than dropped. - BULK MOVE DELTA carries both collection and status when both were sent. bulkMoveCollection applies req.Status as a field override, so a move-with-status changes two things. - FIVE COMMENT OVERCLAIMS, all mine or inherited and all now matching the code: the taxonomy package doc still said nothing drains the outbox; "every event produces exactly one payload shape" predates the batch event's second shape; two places said "the dispatcher runs item-level selectors against each member snapshot" when no binding engine exists and the dispatcher filters on event NAME only; my own retirement comment said this path emits "item.updated + comment.created" transactionally, when the item half is whichever slice moved (a status-only update emits status_changed) and the comment is a separate transaction; migration 083 described the claim as one statement doing both the select and the mark. One finding recorded rather than fixed: affectedIDs counts rows TOUCHED, not rows semantically changed, so an all-no-op operation writes a header with a count and no members. Verified against origin/main — the webhook this replaces fired on the identical condition with the identical count, so it is inherited, and narrowing it is a wire change to count/item_ids that belongs with a contract version rather than a delivery refactor. Gates: build clean, make lint 0 issues, go test ./internal/... green, make test-pg exit 0 / 3463 PASS / 0 FAIL with the new outbox tests verified present in the Postgres run. * fix: codex round 2 — batch correlation on the wire, prior_status survival (TASK-2714) Round 2 was aimed at round 1's own fixes, and that is where both P1s were. - BATCH_ID REACHED ONLY THE FOLDED HALF. A member delivered individually — the window this whole design accounts for — carried an item snapshot with no batch anywhere in it, while three comments claimed consumers could correlate the singles with the batch. They could not. batch_id is now an ENVELOPE field on every delivery of a batched event, singles included, which is the only place a consumer can read it for a member. - FOLD DEDUP COULD DROP prior_status. A mixed update writes item.status_changed (carrying the transition) and item.updated (not); round 1's last-wins kept the later row and silently lost the one field a "nonterminal → terminal" binding needs, in exactly the case that produces both rows. The snapshot is still last-wins — every field IS fresher on the later row — but prior_status is carried forward, because it is envelope metadata only one of the two events ever has. - Sibling scans deduped: a 100-row candidate slice from one batch ran the same query 100 times. - The "only when something actually changed" comment on the bulk emission condition is corrected rather than left to be re-derived: the condition is that a row was TOUCHED without erroring. Untagging a tag nobody has succeeds on every row and changes nothing, so the header fires with a count while the store writes no member events. Same inherited asymmetry round 1 recorded; now the comment says it where the code is. - MY OWN COUNTS WERE WRONG IN THREE PLACES, which is the number-discipline lesson landing on documentation instead of a report: the handler test said "six verbs" while driving seven legs and "three of the six verbs" for what was three CALL SITES across four verbs; the store test implied it covered the verbs when it covers entry points, and now says out loud that it is not sufficient alone — round 1's bug lived one layer above it. Mutations, 2/2 on the new fixes: deliver singles with an empty batch id (the member leg names it twice, once per member); revert the dedupe to plain last-wins (the prior_status leg names it). go test ./internal/... green. * fix: codex round 3 — ack and release are conditioned on the claim (TASK-2714) The P1, and it is round 2's area again: claim tokens were minted and never checked. MarkOutboxDispatched and MarkOutboxAttemptFailed matched on the row id alone, so once a lease expired, a slow pass could still reach rows a newer pass legitimately owned — a late ack stamping a row the new holder is mid-delivery on, and a late release CLEARING a live claim and handing the event to a third pass. Reachable, not theoretical: a workspace's endpoints are delivered sequentially, each with three attempts and a 10s timeout, and the "well under a minute" estimate behind the lease default is an estimate rather than a bound. Both writes now carry the token and condition on claimed_by, and an empty token is refused outright rather than matching NULL. OutboxEvent carries ClaimToken so the drain never has to track it separately. A stale ack matches zero rows, which is exactly right — the event has become the new claim's problem. Also round 3, all P3: - The fold's "embedded VERBATIM" claim now names its one exception: a deduped survivor is re-encoded to carry prior_status across. Non-duplicate members are untouched bytes. - Four stale comments corrected where they live, not just where they were introduced: migration 081 still said nothing drained the table and webhooks fired from hand-calls; createItemChecked's summary still ended in "webhook dispatch"; handlers_watch_notify still called publishBulkItemsEvent "the SSE/webhook bulk path"; and two copies of the PII rationale said nothing drains or prunes, when the window is now bounded (bounded is not zero, which is why the scrub still does the work). Mutations, 2/2: drop claimed_by from the ack (the stale-ack leg fires), drop it from the release (the stale-release leg fires, naming the instance that took the freed row). Both mutations were verified present in the file first — the initial pair silently failed to apply and reported green, which is the third instrument mis-report this unit. Gates: go test ./internal/... green, make lint 0 issues. * fix: codex round 4 — retention spares live claims, token refusal is unconditional (TASK-2714) - RETENTION COULD DELETE A ROW MID-DELIVERY. Every instance runs retention, so one instance's prune could remove an old undispatched row another instance was actively delivering: the delivery would succeed while the ack matched zero rows, and a crash in that window loses an event the outbox had already committed. Live claims are now exempt, using the same lease predicate the claim itself uses. An EXPIRED claim stays fair game — that is what expiry means — and the test asserts both directions. - THE EMPTY-TOKEN REFUSAL SAT BEHIND THE EMPTY-ID SHORT-CIRCUIT, so MarkOutboxDispatched("", nil) returned nil: a contract that depended on the argument it was not about. Token check first. - The taxonomy comment claimed per-member events for ALL bulk mutations. True only of the handler path; the store-side single-transaction producers have no loop, and for those the snapshots INSIDE the payload are the only per-member view there is. Both mechanisms now named, since the distinction is visible in the payloads. - ListPendingOutboxEvents is documented as the diagnostic reader. The drain claims; a reader finding two pending-row queries should not have to guess which one production uses. Mutation, 1/1: drop the claim predicate from the prune (the new test names the count). Verified applied before the result was read. Round 4 also found a REGRESSION I am not fixing here because it is a fork: create-with-parent webhooks carry a pre-link snapshot. CreateItem writes the item.created row in its own transaction, SetParentLink runs in a separate one, and main's hand-called webhook dispatched the RE-READ item — so the parent and the post-link seq were visible then and are not now. Escalated with a recommendation (emit item.updated from SetParentLink's own transaction, which also covers the general case); it sits close enough to SPEC-3 v1.5's link-silence ruling that it is not mine to infer. go test ./internal/... green. * fix: SetParentLink emits item.updated on its own transaction (TASK-2714) Codex round 4's regression, ruled (a) by the lead with the F4 boundary made mechanical rather than inferred (SPEC-3 v1.6): a mutation that writes the ITEM'S OWN ROW emits item.updated; a relationship-graph link, which writes only the links table, stays silent. A parent link advances seq and flips the is_unparented bit, so it is on the emitting side of that line. The regression it closes: createItemChecked calls SetParentLink AFTER CreateItem has already committed item.created with a pre-link snapshot, then re-reads the item for its response. Main's hand-called webhook dispatched that re-read, so a consumer saw the parent and the post-link seq; under the drain the frozen created row was all there was, with nothing to correct it. created(pre-link) then updated(post-link) is a true history. Placed in setParentLinkOnce, not in the shared setParentLinkTx: UpdateItemWithParentLink reuses that core inside the item-update transaction and already emits from the field diff, so the shared site would double-emit. The snapshot comes from getItemTx, and the test enforces that rather than the comment doing it alone — mutating the read to the pool's GetItem fails on the seq assertion, because a different connection cannot see the uncommitted write and would emit the pre-link row under a post-link event. Mutations, 2/2: delete the emit (no event after linking); read the snapshot from the pool (seq is the create's). Plus a control leg asserting the PARENT emits nothing — the link does not write its row. go test ./internal/... green. * fix: codex round 5 — parent-only updates emit; the parent-emit claim is narrowed to the truth (TASK-2714) Round 5 aimed at round 4's own fix and found two P1s in it. Four-for-four on that angle now. 1. THE PARENT-ONLY UPDATE PATH STILL EMITTED NOTHING. SetParentLink's fix covers its own transaction; UpdateItemWithParentLink writes the hierarchy inside the ITEM-UPDATE transaction and emits from a snapshot DIFF — and a parent write leaves nothing in a snapshot to diff, since items.parent_id is legacy and untouched, the link lives in its own table, and seq/updated_at are excluded as metadata. So a fields_patch carrying only `parent` mutated the row and emitted zero events. The emitter now takes hierarchyChanged from the caller, which knows what it wrote; the diff cannot see it and must not have to. Covers set AND clear, with a control leg asserting a genuinely empty update still emits nothing — without it the fix could be "always emit", which would undo the disjoint-delta rule. 2. MY OWN ROUND-4 COMMENT AND TEST OVERCLAIMED. Both said the event carries a "post-link snapshot"; the payload is the item ROW, and the parent EDGE is not on it — IsUnparented is populated only by the local-first index queries, so the test's is_unparented assertion passed VACUOUSLY against an absent field. What the emit actually restores is the row change (a fresh seq and updated_at), which is exactly what main's hand-called webhook carried: it dispatched the handler's post-link re-read, the same scan. The comment now says that, and the test asserts the ABSENCE so the next reader cannot infer linkage data that has never been on this wire. Third instance this unit of the same shape: a partial verification written up as a complete one. Also round 5, both comment-level: - handlers_item_links.go said item-link mutations are silent in events/1. True per link TYPE, not per handler: parent crosses the "writes the item's own row" line and emits, blocks/blocked-by and implements do not. The comment now states the criterion and names the consequence — this handler publishes SSE for both kinds, so the SSE and events/1 pictures deliberately differ. - EmitBulkHeaderEvent's guard said "a bulk operation that changed nothing is not an event" while being an empty-LIST guard. Corrected in place with the reason it stays: the webhook it replaced fired on the identical condition with the identical count, so narrowing it is a wire change for a contract version, not a fix. Mutation, 1/1: drop the hierarchy force (the parent-only leg fails by name). Verified applied before the result was read. go test ./internal/... green. * fix: codex round 6 — parent DETACH emits on every route (TASK-2714) Round 6 aimed at round 5's fix and found two more in it. Five-for-five. 1. DETACH WAS SILENT ON TWO OF THREE ROUTES. Attach emitted from SetParentLink and from the update path, but ClearParentLink (its own transaction) and DeleteItemLink on a parent row (what DELETE /links/{id} actually calls) wrote the item's row and emitted nothing. A consumer's model would keep a parent the user had removed, with every attach route observable — the worst shape for this kind of gap, because the wire looks healthy. Routes are now enumerated in one test rather than sampled. 2. hierarchyChanged MEANT "PROVIDED", NOT "CHANGED". Clearing an already-unparented item deletes zero rows; round 5's flag still forced item.updated, putting an event on a public wire for a mutation that did not happen. clearParentLinkTx now reports whether it removed a link and the flag comes from that. The set branch stays unconditional — it is a DELETE-then-INSERT and bumps the row either way. IMPLEMENTS IS FLAGGED, NOT DECIDED. It bumps the same row (so the v1.6 mechanical criterion would include it) but it is a relationship-graph link (so v1.5's silence would exclude it). The contract does not resolve that case, and inventing an answer inside a delivery refactor is how a public wire acquires an event nobody ruled on. Recorded at the call site and raised with the lead. Mutations, 2/2: drop the parent-detach emit from DeleteItemLink (the route's leg fails); treat provided as changed on the clear branch (the no-op leg fails by name). The second mutation first failed to BUILD — `removed` then unused, which is the compiler catching it and my grep reading the empty result as a pass — so it was re-run with the variable kept alive. Fourth instrument mis-report this unit; all four are on the record. Gates: go test ./internal/... green, make lint 0 issues, make test-pg exit 0 on the pre-round-6 tip (re-run pending on the final tip). * fix: codex round 7 — the batch delta matches the mutation (TASK-2714) Round 7 returned no P1s; the parent-detach work from round 6 came back clean on transaction scope, lock ordering, error paths and duplicate emissions. - BULK DELTA REPORTED THE REQUEST, NOT THE COMMIT. bulkEventDelta echoed raw request values while the mutation normalizes: bulkTagUpdate trims added tags and skips ones that go empty, and the store's assignment SET clause gives a NON-EMPTY id precedence over the clear flag (BUG-2566). So a request with both an id and clear=true announced a clear while the row was assigned — the delta describing the opposite of what committed. This is the one field of the batch payload the drain cannot derive, so nothing downstream corrects it: whatever it says is what a consumer believes. Now normalized the same way, including untag matching RAW because the mutation removes by exact match. - THE implements COMMENT WAS WRONG, and it was mine from round 5: it listed implements with the silent link types when implements DOES bump the source row, exactly as parent does. Corrected in both places, and the case is stated as UNRESOLVED rather than settled — the mechanical criterion (writes the item's row) would have it emit, v1.5's relationship-link silence would not, and deciding it inside a delivery refactor would put an event nobody ruled on onto a public wire. With the lead. Mutations, 2/2: give the clear flag precedence over a non-empty id (the precedence leg fails, naming the row it would misdescribe); stop trimming added tags (the trim leg fails). The first mutation initially failed to build and was rewritten to compile before its result was believed. Gates on the round-6 tip: go test ./internal/... green, make lint 0 issues, make test-pg exit 0 / 3445 PASS / 0 FAIL with 13 of this unit's new test legs verified present in the Postgres output. Re-run pending on the final tip. * fix: codex round 8 — tag delta dedups, link SSE name derives (TASK-2714) Two P2s, both small and both the same shape: a claim in a comment that the code did not quite meet. - THE TAG DELTA TRIMMED BUT DID NOT DEDUPE, while bulkTagUpdate does both — it skips a tag already in its `seen` set. So tag ["foo", " foo "] added one tag and advertised two, under a comment saying the delta is normalized "the same way the mutation does". Round 7 fixed half of that sentence; this fixes the other half. - handlers_item_links.go PUBLISHED UNDER THE events.ItemUpdated LITERAL. The wire value happens to match, which is exactly why it was worth changing: it recreates the drift the central mapping exists to prevent, one rename away from being wrong. The name now derives like every other SSE site, and the comment separates the two facts a reader has to keep apart — the NAME derives, the events/1 EVENT still does not exist for relationship links. Mutation, 1/1: drop the `seen` check from the delta's tag loop (the dedup test names the duplicated value). go test ./internal/... green. * fix: codex round 9 — untag delta dedups, version restore derives its SSE name (TASK-2714) Both are the same shape as round 8's, one layer further out. - THE UNTAG DELTA STILL ECHOED DUPLICATES. bulkTagUpdate builds a removal SET, so ["foo","foo"] removes one tag; the delta advertised two. The two verbs normalize DIFFERENTLY — tag trims and dedups, untag dedups but matches raw, because removal is by exact string — and the delta now mirrors each side's own rule rather than applying one of them to both. - handlers_item_versions.go PUBLISHED A RAW "item_updated" STRING. A second source of SSE vocabulary, and the harder kind to find: it does not even reference the events package, so a grep for events.ItemUpdated misses it. Now derived like every other site. go test ./internal/... green. Gates on the round-8 tip: make test-pg exit 0, 3447 PASS, 0 FAIL, with 150 lines of this unit's own test legs verified present in the Postgres output.	2026-08-20 23:33:26 -04:00
xarmian	25c7cd20f5	feat(watchevents): Redis-backed bus so watch notifications cross instances (BUG-2651) (#1167) ... * feat(watchevents): Redis-backed bus so watch notifications cross instances (BUG-2651) internal/watchevents shipped MemoryBus only, so in a multi-instance deployment a notification published on instance A never reached a stream held open on instance B — watches appeared to work and silently dropped. Bus was an interface from day one for exactly this; adding RedisBus changed no producer and no consumer. NOT A MECHANICAL PORT of internal/events.RedisBus. Three deliberate divergences, each documented at the point someone diffing the two files would call it a mistake: - ONE channel and ONE replay buffer, because this package has exactly one logical stream by contract (DOC-2479 DR-2: all per-caller filtering happens in the consumer). Most of the template's bookkeeping — per- workspace counts, subscriptions, buffers — has nothing to key on here. - EAGER subscription for the bus's lifetime, not lazily on first local subscriber. The replay buffer fills from the RECEIVE path, so a lazily torn-down subscription stops filling it at precisely the moment before a Last-Event-ID resume — for one harness monitor holding one stream, that makes resume structurally useless. The template can afford lazy because per-workspace means N idle subscriptions; here it is one. - ONE mutex across subscriber membership and the replay buffer, held through the whole local fan-out. The template uses two and offers only separate Subscribe + EventsSince, which cannot provide SubscribeAndReplaySince's guarantee. Copying its locking would have handed back the double-delivery window this package's interface exists to close. Publish fails CLOSED when INCR fails, where the template falls back to a local counter. Two instances falling back at once mint ids from independent counters into a shared stream, and replayBuffer.since() reasons on monotonicity — so the damage is silent replay corruption, not a visible error. INCR and PUBLISH share a connection anyway, so the fallback mostly lets a doomed publish proceed carrying a poisoned id. Both load-bearing tests were VACUOUS as first written; the mutation matrix is the only reason I know: - the concurrency test's producer finished before the subscriber joined, so the channel leg was never exercised and a split-lock mutant survived 50 iterations. Now paced, with a both-legs-non-empty precondition that fails a run which never approached the boundary, plus a dedicated detector (600 attempts, 8/8 kills, 0.02s after switching the drain to non-blocking — exact, because the duplicate is already buffered when the call returns). - the fail-closed test asserted nothing was delivered, which is true of the fallback too: Publish never delivers locally, so with Redis down neither policy delivers. Rewritten around a go-redis ProcessHook that records attempted commands, which is where the policies actually differ (INCR-then-stop vs INCR-then-PUBLISH). Also corrects session_presence.go, which told the next person these two had to be fixed together. Delivery is now cross-instance; the registry's under-report is unchanged, so the remaining defect is a picker that under-reports rather than a push that lies. The PLAN-2558 S3 gate stays, for that reason instead of the old one. Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V * fix(watchevents): make id assignment and publish atomic; close the bus on shutdown (Codex round 1) P1 — INCR and PUBLISH as two client calls are not order-preserving, and the failure is concrete: A gets id 1 and is descheduled, B gets id 2 and publishes, A publishes 1. Every subscriber receives 2 before 1, the replay buffer appends in ARRIVAL order, and replayBuffer.since() reasons on monotonicity — so a resume from 2 hits the sinceID > newestID branch and answers 'gap too large', turning a healthy reconnect into a spurious sync_required, while a resume from 1 silently skips the late arrival. Fixed at the source with a Lua script: Redis runs it atomically on its single thread, so INCR and PUBLISH for one instance both complete before another's script begins, and publish order equals id order globally with no coordination on our side. The id rides as a '<id>|<json>' prefix rather than being edited into the JSON from Lua; the id is digits and the FIRST '|' separates, so a '|' in the body is unambiguous. A pleasant consequence: there is no longer a window where an id exists but the publish has not happened, so the fail-closed decision and the publish decision became the same decision. P2 — Stop() never closed the watch bus. That was survivable for MemoryBus, whose Close only drops channels; RedisBus holds a receive goroutine and a Redis subscription from construction, so it leaked both for the process's life. Closed after bg.Wait(), so a background producer cannot publish into a bus already tearing down. nits, all real, all in artifacts someone reads: - 'exactly-once delivery' was simply wrong. Redis pub/sub is at-most-once and the local send is deliberately non-blocking. The property the round trip actually buys is NO DOUBLE DELIVERY to the publishing instance; the comment now says that and names the replay buffer as the bounded recovery mechanism for the rest. - the Bus interface comment still said only MemoryBus existed. - cmd_server.go's session-presence note still claimed the same caveat as 'the watch bus directly above', which had just stopped applying. - session_presence.go now says delivery is fixed WHEN PAD_REDIS_URL is set, rather than unconditionally. Tests: the fail-closed assertion moved from 'nothing was delivered' — still true under the two-call version — to 'no bare INCR or PUBLISH was issued', which is what distinguishes atomic from not. Mutation-verified by splitting the script back into two calls. Added a decode round-trip test covering the new wire format, a '|' inside the body, and four malformed payloads, since that decoder consumes bytes from a channel any holder of the Redis credentials can publish to. Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V * fix(watchevents,server): correct the targeted-push claim; close the bus before HTTP shutdown (Codex round 2) P2 — I claimed cross-instance DELIVERY was fixed. Half true, and the false half was mine to catch: handlers_push.go gates a session-targeted push on the LOCAL presence registry and skips the publish entirely when the id is not there, so a POST landing on A for a session held on B still delivers nothing. The bus would carry it; the gate means it never reaches the bus. Broadcast pushes and every other notification kind ARE fixed. I asserted that behaviour from reading the bus and session_presence.go without reading the push handler — the exact thing I hold myself to not doing. Corrected in all three places the claim was made (the package doc, session_presence.go, and the KindPush comment), with the correction recorded rather than quietly overwritten. The gate's own justification is now stale too, and worth more than a tweak: 'a target this instance cannot see is a guaranteed no-op' was TRUE under MemoryBus and is FALSE under RedisBus, where another instance may hold that session. Left in place deliberately — publishing unconditionally would fix delivery and immediately make delivered_sessions=0 a lie in the other direction, which is a question about what that field promises. It belongs with the shared-state SessionPresence that PLAN-2558 S3 already gates on: fixing the registry makes the snapshot right, and then the skip is correct again for its original reason. Both open halves collapse into that one implementation. P2 — the watch bus was closed only in Server.Stop(), which runs AFTER http.Server.Shutdown. The event bus is closed before Shutdown precisely so its SSE handlers unblock; the watch stream is the same shape, so an open one would have held Shutdown to its full 30s deadline. Now closed alongside eventBus, with the Stop() close kept as the path for other callers — both implementations are idempotent. nit — MemoryBus and RedisBus disagreed after Close: RedisBus handed a late Subscribe an already-closed channel, MemoryBus registered one nobody would ever close, so a consumer racing shutdown blocked forever. MemoryBus now matches, and its Close is idempotent, which the CLI's double close relies on. Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V * fix(watchevents): report a missed notification as a replay gap (Codex round 3) P2 — a divergence MemoryBus structurally cannot have. It assigns every id itself, so its replay buffer is contiguous and the only gap it can report is eviction. RedisBus receives ids over at-most-once pub/sub, so a blipped subscription can miss 101 and receive 102: the buffer holds a hole, is nowhere near full, and replayBuffer.since() answers a resume from 100 with just [102]. The consumer loses a nudge and is never told. RedisBus now tracks the id at which the sequence resumed after the most recent hole, and answers nil — the same signal eviction already gives, which the SSE handler already turns into sync_required — for a resume that would have to span it. Resumes that do not span it still replay normally, and sinceID=0 is treated as a fresh subscriber rather than a resume, so a hole nobody spanned is not turned into a spurious resync. The atomic publish script is what makes this readable: publish order is id order globally, so a non-consecutive id means MISSED, not reordered. Mutation-verified by disabling the check; the test fails on both the spanning resumes and would have failed the over-broad version too (it asserts the non-spanning resumes still work). Two residuals documented rather than fixed, both because the fix is the same shared-state SessionPresence that PLAN-2558 S3 gates on: - delivered_sessions is now wrong in BOTH directions for a broadcast push — the count is local while delivery is global, so a replica can report 1 while two sessions receive it, or 0 while a remote one does. No local arithmetic fixes that; it is asking one replica what all of them are doing. - the Redis channel and counter names are not deployment-scoped, so two installations sharing a Redis endpoint cross-feed (and picking different logical DBs does not help — pub/sub ignores them). Left flat to match internal/events rather than giving one of the two buses a prefix the other lacks; the rule is one Redis endpoint per installation, and relaxing it should cover both buses at once. Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V * fix(watchevents): a cold-started replica must report a gap too (Codex round 4) P1 — the round-3 hole check only fired BETWEEN two received messages, so it never fired for the first one. A replica restarting while Redis is already at 101 has an empty buffer; its first received message is 102, nothing looks like a hole, and a client reconnecting to that replica with Last-Event-ID 100 was handed [102] — skipping 101 exactly as silently as the case round 3 fixed, by a different route. Replaced contiguousFrom with knownFrom: the lowest id from which this instance's buffer is contiguous. SET on the first append (before which this instance knows nothing) and RESET on every hole (before which it no longer knows anything usable). One variable, both failures. The boundary is pinned in both directions, which is what stops this being an over-broad 'always gap after a restart': a resume from exactly the id before our first (101 when we started at 102) IS contiguous with our view and replays normally. Mutation-verified by disabling the cold-start arm. Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V * fix(watchevents): idempotent publish, confirmed subscription, and real Redis tests (Codex round 5) P2 — go-redis retries a command whose reply is lost to a network error, and the publish script was not idempotent: the same notification would be published twice under two different ids. Both copies look valid — ordered, distinct — so nothing downstream could tell them apart, and on the push path a duplicate is a duplicate DISPATCH into an agent harness. The script now takes a caller-generated token and SET NX's it, so a retry carrying the same arguments returns 0 without publishing. TWO THINGS THIS UNIT OWES ITS TESTS, both found within minutes of each other and both invisible to the hermetic ones: 1. The idempotency script shipped indexing ARGV[3] while Publish passed two arguments. Caught by re-reading, which is not a control worth relying on for the next Lua edit. 2. NewRedisBus returned before go-redis had established the subscription, so notifications published in that window were lost to this instance, silently. Surfaced as a test flake; the production shape is a rolling deploy, where a replica takes traffic before its subscription is live. The constructor now waits for the confirmation (bounded, and a failure is logged rather than fatal since Channel() re-subscribes on reconnect). So miniredis is now a test dependency, and the round-trip tests it enables cover what fanOutLocally-driven tests structurally cannot: the channel name, the KEYS/ARGV mapping, the id prefix wire format, the shared counter across two buses, cross-instance delivery (the actual bug), the dedupe token, and Close tearing down the SERVER-side subscription rather than just local channels. Verified by restoring the ARGV[3] bug: the round-trip test fails on it. The two findings I am NOT fixing here are unchanged and documented where the reasoning is met — the targeted-push gate and delivered_sessions are both consequences of the per-process presence registry, and both are closed by the shared-state SessionPresence that PLAN-2558 S3 gates on, not by anything in this package. make vuln: 0 vulnerabilities in imported packages. Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V * fix(watchevents): survive a Redis counter reset without replaying stale ids (Codex round 6) P2 — pad:watchevents_seq has no TTL but can still vanish: evicted under maxmemory, dropped by a FLUSHDB, or restored from an older snapshot. Ids then restart at 1 while this instance's ring still holds the hundreds. Keeping both is what corrupts replay — the two id spaces are not comparable, so a resume from 2 in the NEW space would be handed the stale 99/100/101 as though they were newer. A backwards id now drops the replay buffer and re-anchors knownFrom. Every resume from the old space then exceeds the newest id held and gets nil — the resync signal that is the only honest answer once the ids stopped meaning what the client thinks they mean — while clients in the new space keep working immediately. The test asserts BOTH halves, which is what makes it a detector rather than a description: a build that logged the reset and kept the buffer passes 'the old resume reports a gap' and fails 'the new resume never returns a pre-reset entry'. Mutation-verified on exactly that. Hardened while I was here: the epoch-reset path REBUILDS the buffer at runtime, so a bus constructed with a non-positive replay size would have turned a counter reset into a panic (newReplayBuffer(0)'s first append indexes a zero-length slice) rather than a resync. The constructor now normalizes. MemoryBus has the same trap for a caller passing 0; left alone as pre-existing and off this path, but named in the comment rather than silently fixed or silently ignored. nit — this file's header still claimed there was no miniredis dependency and no round-trip coverage, which the previous commit made false. Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V * docs(watchevents): actually correct the hermetic test header (Codex round 7) The previous commit's message claimed this fix. It did not contain it: the edit ran as one of two scripts in a single command, its assertion failed with a traceback, and the second script's success is what I read. The header kept saying there was no miniredis dependency and no round-trip coverage — both false since two commits ago, in the file a reader consults to find out what IS covered. That is the adjacent-success-signal failure exactly: a success line from the step next to the one I cared about. The tell was in the output and I walked past it, then asserted the change in a commit message. Recording it here rather than quietly fixing, because a commit that claims a change it does not make is worse than one that omits it. Verified this time by reading the file back and grepping for the stale phrases: zero. Round 7's other three findings are the documented residuals re-raised for the third time — the targeted-push gate, delivered_sessions, and the unnamespaced Redis keys. All three are dispositioned at the line a reader meets them, all three are consequences of the per-process SessionPresence registry or of matching internal/events' existing convention, and none is fixable inside this package. They stay open, on the record, and with the lead. Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V * docs(watchevents,cli): correct pad push --help; document the reset-window residual (Codex round 8) nit, and the one that stings — cmd_push.go's Long help still said pushes go over the 'in-memory watch-events bus'. That is the text a user reads when they run pad push --help, and it has been false since this branch's first commit. I have a standing pre-push step to grep the artifacts a CONSUMER reads for exactly this, and I ran it as a code search (watchevents.New) rather than a prose search, so --help never came up. The help now distinguishes broadcast (reaches every instance) from session-targeted (still resolved against the handling server) and names the bug. P2 — the counter-reset handling fires when the first post-reset notification ARRIVES, so there is a window between Redis losing the counter and the next publish in which this instance still replays old ids to a reconnecting client. Documented as accepted rather than closed: nothing local can detect the reset earlier (the counter is in Redis and we learn of it by receiving something), and the two shapes that would — a GET per resume, or a background poller — put network I/O on a latency-sensitive path or spend a goroutine and a round trip per tick forever against a condition measured in years. The exposure is redelivery of notifications the client already has, bounded by the window and self-healing on the next publish. Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V * fix(watchevents): a replica that has received nothing must not answer 'caught up' (Codex round 9) P1 — the coverage check was skipped entirely while knownFrom was still 0, so a bus that had received NOTHING answered any cursor with an empty-but-non-nil replay, which the SSE handler reads as caught-up. The scenario is a restart, not an exotic one: replica B comes up while Redis is at 100, id 101 is published before B's subscription is live, and a client reconnects to B with Last-Event-ID 100 before 102 arrives. B says caught-up, then delivers 102 live, and 101 is gone with nothing to tell anyone. The principle the code now follows: having received nothing is strictly LESS knowledge than 'contiguous from X', so it must produce at least as strong a signal. A non-zero cursor against an empty bus is a gap. Both sides pinned, because the over-broad version is a real risk here — answering every fresh connection with a resync would be its own bug. A sinceID of 0 is not a resume and still gets an empty replay rather than a gap. Mutation-verified on the new arm. Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V * docs(watchevents,cli): name the trailing-gap and shutdown trades (Codex round 10) Two findings that are decisions rather than defects, so both are documented at the line where the reasoning is met and taken to the plan instead of being settled unilaterally after ten review rounds. P1 as reported — the TRAILING gap. Everything the coverage bookkeeping does reasons about what this instance HAS received; it cannot see a notification missed at the END of the sequence. Hold 100, miss 101 to a disconnect, and a client resuming from 100 before 102 arrives is told caught-up. The hole only becomes visible when 102 lands, which is too late for that connection. What would reveal it is a GET of the sequence key: a value above lastAppendedID means ids exist we never saw, and a value BELOW it reveals the counter reset documented last round — one mechanism, both open windows. It is not done here because it is product-visible in the other direction: INCR happens before the message propagates, so the counter legitimately runs ahead of every instance for microseconds after each publish, and a strict comparison turns ordinary in-flight traffic into spurious sync_required responses with no principled tolerance to pick. A resync is recoverable and a lost nudge is not, which is the argument for doing it — but that is a call about how chatty the resync path should be. P2 — closing the watch bus before Shutdown drains handlers means a push already in flight can publish into a closed bus and still return 200 with pushed:true. Closing after would instead hold every shutdown to its 30s deadline on any open stream. eventBus already makes the same trade the same way; naming it rather than inheriting it silently. The honest fix is Bus.Publish reporting the drop so the handler can, which is an interface change and a different unit. Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V * feat(watchevents): close the trailing gap with a settle-window authority check (lead ruling) Lead's ruling on BUG-2651: a silently lost nudge is unbounded staleness, a spurious resync costs one redundant fetch, so the gap must not survive — and don't pick a magnitude tolerance, because the reason the counter legitimately runs ahead is in-flight propagation, which is TIME-bounded while a genuinely missed message never arrives. So the discriminator is time. On a resume (and only on a resume), read the shared counter: if it disagrees with this instance's high-water mark, wait out one settle window and read again. In-flight ids land during the beat and the resume proceeds normally; missed ones never do and the resume is answered with a gap. That converts an unprincipled 'how many ids behind is too many' threshold into a principled propagation bound. The same read also catches the counter having gone BACKWARDS, so the counter-reset window documented last round is closed by the same mechanism rather than needing its own — the arrival-time reset handling stays, because it is what repairs the instance's own state and what covers a bus with no reconnecting clients. Ordering matters and is documented at the call: the check runs WITHOUT the mutex (it sleeps and does network I/O, neither of which may happen inside the lock fan-out needs) and BEFORE subscribing rather than between subscribe and replay, which would reopen the double-delivery window SubscribeAndReplaySince exists to close. Nothing is lost by waiting first — fanOutLocally buffers regardless of subscribers. An unreadable counter falls back to local knowledge rather than failing closed: turning a Redis hiccup into a resync for every reconnecting client at once is a worse failure than the one being guarded against. EventsSince deliberately does NOT do this and says so — it is the local primitive the Bus interface already describes as being for tests and non-resuming callers, and making it sleep and hit the network would surprise every one of them. Five tests, each pinning a different half: the missed tail reports a gap; a current instance does NOT (the control that stops this being 'always resync'); an id arriving mid-settle is tolerated; an unreadable counter falls back; a fresh subscriber neither waits nor gets a gap. Mutation-verified twice — disabling the check, and removing the settle beat — each killed by the test that names it. Also filed at the lead's direction, so the two remaining cross-instance defects have tracked homes rather than only comments: BUG-2698 (targeted push resolved against local presence, plus the delivered_sessions inaccuracy — one shared-state SessionPresence closes both) and BUG-2699 (push returns 200 pushed:true for a dropped publish; Bus.Publish reports nothing, and fixing it is an interface change). Every disposition comment now cites its item. Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V * fix(watchevents): compare two FRESH reads, not one stale snapshot (Codex round 11) P1 — the settle beat re-read only the local side, so the comparison was against a counter SNAPSHOT taken before the wait. Id 2 arrives during the beat while id 3 is published and missed: the stale remote is still 2, the check declares convergence, and 3 is silently lost — the exact failure this whole mechanism exists to prevent, reintroduced inside it. P2 — the same staleness in the other direction. A GET can land just before a publish completes and report a value BELOW what this instance already holds; that never matches, so a client who had missed nothing got a full resync. Both are one defect: agreement between the authority and this instance has to be evaluated on two FRESH reads or it is not agreement. Now re-reads both sides after the beat, and treats any remaining disagreement as a gap in either direction — still behind means ids never reached us, still ahead means the counter was reset under us and our buffer belongs to a dead id space. Two tests, one per direction, each mutation-verified against the re-read-locally-only version: the second counter advance must produce a gap, and the raced read must NOT produce a resync. Without the second test the fix could have been 'always report a gap', which passes the first. Documented the cost side of the lead's ruling while I was in here: the condition is agreement, so a resume during CONTINUOUS publishing across the whole settle window can disagree every time and resync. Bounded by this stream being low-volume by design and resumes only happening on reconnect; if a workload makes it chatty, the answer is a longer window, not a magnitude threshold. Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V * fix(watchevents): an absent sequence key is zero, not unreadable (Codex round 12) P2 — the counter key can DISAPPEAR after this bus has seen ids (FLUSHDB, eviction). Reading redis.Nil as 'unreadable' meant falling back to local knowledge and cheerfully replaying an id space the authority no longer has — while the next publish starts again at 1 and collides with it. Absent is a VALUE. Returning zero-and-readable makes the case fall out of the ordinary comparison with no special branch: an instance holding 101 disagrees with an authority at 0, does not converge, and the resume is answered with a gap. A genuinely fresh deployment still agrees at zero and is not resynced — which is the control leg, and the reason 'absent means gap' would have been the wrong fix: it passes the first test while resyncing every first connection on a new install. P1 as reported — the equality fast path returning without settling — is not closed, and the comment now says why rather than leaving it to be re-found. A notification published AFTER that read and missed by this instance is invisible to any check made here, and settling anyway would not close it: the same race exists in the instant after the function returns. The check's honest scope is what was missed BEFORE the resume. A message missed after it is a property of at-most-once pub/sub with no per-connection ack, and the real answer is a durable stream (Redis Streams with consumer groups), not a longer wait. Mutation-verified: restoring redis.Nil to the unreadable branch fails the disappearing-counter test. Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V * feat(watchevents): epoch marker, so a reset that caught up is still a reset (Codex round 13) P2 — numeric detection is blind to a reset that has already climbed past this instance's high-water mark. Hold 100, lose the connection, the counter resets and ids 1-101 are published, and the only one that reaches us is 101 — the perfect contiguous successor of 100. Every arithmetic check passes, the buffer quietly mixes two id spaces, and a client resuming from OLD 100 is handed NEW 101 having silently missed the new space's 1-100. No amount of comparing numbers fixes that, because the question is not 'is this bigger' but 'is this the same sequence'. The publish script now mints an epoch once per id space (SET NX, so every publisher can offer one and the first wins) and carries it on every message; a change drops the buffer and re-anchors. The subtle half, and the one the first attempt got wrong: after an epoch change the cold-start rule must NOT admit its usual contiguous-with-our-view cursor. Within an epoch, a client at n.ID-1 is genuinely adjacent to our first id. Across one it is ambiguous — id spaces overlap, so that cursor may be the OLD sequence's n.ID-1, a different notification entirely — and admitting it hands them the new epoch's id as though it followed theirs, which is exactly the failure the epoch exists to prevent. Letting it back in one line later would have been a poor joke. The test caught it; the control leg (a cursor genuinely inside the new epoch is still served) is what stops the fix becoming 'resync everyone forever after any reset'. Wire format changed to <epoch>|<id>|<json>. Free of compat cost, checked rather than assumed: redis_bus.go does not exist on origin/main, so no released build produces or consumes the old shape. The numeric backward check stays — it covers a counter reset where the epoch key survived (eviction picks keys individually), and it is what repairs an instance with no reconnecting clients at all. Mutation-verified: ignoring the epoch change fails the new test. Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V * docs(watchevents): the wire format comments say <epoch>|<id>|<json> (Codex round 14) Three comments still described the pre-epoch format. Worth more than a tidy-up: a maintainer following them would conclude the epoch prefix is vestigial and remove it, which reintroduces exactly the cross-epoch replay corruption round 13 existed to fix. The publishScript comment now also says outright that the epoch is not decoration and points at redisWatchEpochKey before anyone considers it removable. Verified by grepping for the old shape rather than by trusting the edits — zero remaining, which is the check I owed after getting this wrong in round 7. Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V * chore(nix): update vendorHash for the miniredis test dependency (BUG-2651) CI's Nix job failed on a fixed-output hash mismatch, and it is neither a flake nor a surprise once seen: nix/package.nix pins the vendored module set, and adding miniredis (plus gopher-lua, its Lua interpreter) to go.mod changed it. Regenerated per the procedure the file itself documents — build and read the 'got:' line. Run on CI rather than locally because this box has no nix; the hash is a content hash of the module set determined by go.mod/go.sum, so the same inputs produce it in either place. Worth naming as a gate lesson rather than just fixing: my pre-merge matrix had build, lint, test, test-pg, vuln and Codex, and none of them can see this. A dependency change has a SEVENTH consumer — the Nix packaging — and the only thing that checks it is the CI job that just did. Adding a dependency means checking the packaging, not only the security scan. Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V	2026-08-20 02:54:53 -04:00
xarmian	b5f0cd3963	feat(web,server): render embedded image attachments on share pages via a variants-only byte endpoint (BUG-2389 2b, TASK-2637) (#1153) ... Merged by lead on accepted infra cancellation, Dave-approved in chat (day-44). Evidence basis: E2E tests demonstrably pass at pinned SHA d3c8234e — run 1: 191 passed (3.7m), job cancelled by CI/Nix twin-run concurrency race; run 2: 190 passed + 1 flaky (pane-controller, PLAN-2154's known flake, unrelated to this diff), job cancelled by the 10-minute cap after the flaky retry; run 3: rerun expired inside terminal-cancelled parent at 38s, no test signal. All six code-testing checks green (Go, Go-PG, Web, Nix, Smoke×2). The gate defect is filed as BUG-2645 (cap breachable by one flaky retry + twin-run race); the fix ships as its own reviewed workflow unit. Lead spot-check of the diff and full pin inventory: TASK-2637 trail.	2026-08-18 12:26:00 -04:00
xarmian	08dfbdb318	fix(server): rowless-blob GC sweep — reclaim blobs no attachments row references (BUG-2406) (#1137) ... * fix(server): rowless-blob GC sweep — reclaim blobs no attachments row references (BUG-2406) Every attachment write path calls AttachmentStore.Put BEFORE inserting the attachments row, so a failure (or crash) between the two leaves a blob on disk that nothing references — and the row-driven orphan sweep, which walks Store.OrphanedAttachments, can never see it. Disk that is never returned; the upload handler's failure comment even claimed the GC would reclaim it. Fix: a rowless-blob sweep that runs after the row sweep on the same GC tick. attachments.Lister is a new OPTIONAL backend capability (ListBlobs → key/hash/size/mtime); FSStore implements it via one WalkDir of the sharded tree with a base-name validHash gate (excludes Put's dot-prefixed temp files and anything the store didn't write). Backends without the capability are skipped with a once-per-process notice. Candidate = blob whose content hash has ZERO rows in ANY state (soft-deleted rows still own their bytes under the row sweep's row-before-bytes claim protocol, BUG-2415) AND whose mtime predates the same operator-configured GC grace the row sweep uses — a young rowless blob is just an upload whose insert hasn't happened yet. Delete-time guards run under inFlightHashesMu: the in-flight fence plus a single-hash row RE-CHECK that closes the subtraction-to-delete TOCTOU (the writer that marked, inserted, and released entirely inside the gap). Cost: O(blobs) per tick, 24h cadence, never on a request path. Also retro-reclaims blobs stranded by past row-sweep delete failures. The wrong claim in handleUploadAttachment's failure path is corrected to point at this sweep. Tests: FSStore.ListBlobs impostor coverage; five sweep legs (aged-rowless reclaimed with a row-sweep-can't-see-it counterfactual, young kept, live/soft-deleted-row kept, in-flight kept then reclaimed after release, hook-injected delete-time row kept) — mutation-verified: removing the re-check, the age gate, or the in-flight fence each fails its leg; the store-level subtraction contract is pinned separately. Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V * docs(store): state the any-row rule's real rationale per Codex review (round 1) Codex flagged the thumbnail refusal-cleanup's grace-window protection as inconsistent with the sweep comment's claim that deleting bytes under any existing row violates the claim protocol. The cleanup (and the row sweep itself) deliberately end a row's hash-protection when its own grace expires — CountProtectingAttachmentsForHash documents exactly that, and the row machinery may do it because its claim protocol coordinates row and blob fates within a sweep. The overstatement was mine: the rowless sweep's any-row rule is chosen because it holds no claim on any row and has no such coordination, not because past-grace stranding is forbidden to the machinery that does. Comment corrected; no behavior change on either path. Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V	2026-08-17 09:03:44 -04:00
xarmian	31075d996a	fix(store): route cross-workspace copy's lock-held reads through the copy transaction (BUG-2409) (#1136) ... * fix(store): route cross-workspace copy's lock-held reads through the copy transaction (BUG-2409) The copy transaction holds advisory locks on BOTH workspaces, but the attachment planner (PlanAttachmentCopy) and the server's per-row attachment authorizer read through the connection pool. Under enough concurrent copies every pooled connection can be occupied by a lock-waiter while the lock holder waits for a spare connection — starvation presenting as a hang. Fix: a store.Queryer interface (satisfied by *sql.DB and *sql.Tx) threaded through the planner and the AttachmentAuthorizer callback, so the mutating copy plans and authorizes on its own transaction's connection while the preflight keeps planning through the pool — one implementation, two executors, preserving TASK-2354's no-drift shape. Mechanical *Q variants added for the store reads the authorizer transitively needs (GetItem, GetUser, GetWorkspaceMember, VisibleCollectionIDs, GetMemberCollectionAccess, ListSystemCollectionIDs, GuestVisibleCollectionIDs, GuestVisibleResources(+IncludeDeleted), ResolveBacklinksVisibility) and Q-cores behind existing-signature server wrappers (checkItemVisible, guestResourceFilterCore, resolveAttachmentParentItem, attachmentCallerIsRestricted). No decision logic changed anywhere — executor threading only. GetItem/getItemTx/GetItemIncludeDeleted's three duplicate scan bodies collapse into one getItemScanQ. Regression test: TestCopyItemAcrossWorkspaces_NoPoolIOUnderLocks pins the invariant deterministically — with MaxOpenConns(1) the transaction owns the only connection, so ANY pool read under the locks deadlocks. Fails by timeout on the pre-fix executor (verified); passes in 0.16s fixed. The test's authorizer performs a real read through the handed Queryer, pinning the callback leg too. Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V * fix(store): quota check reads through the copy transaction too, per Codex review (round 2) Codex's targeted round found the third lock-held pool-read leg: CheckLimitTx routed only the feature COUNT through the caller's transaction while checkLimitOn's owner lookup, GetUser, and resolveLimit's platform-setting read stayed on the pool — the same starvation shape under the copy's advisory locks. checkLimitOn is now parameterized over a single Queryer for every read (CheckLimit passes the pool, CheckLimitTx the transaction), with resolveLimitQ / GetPlatformSettingQ variants behind existing-signature wrappers. The regression test now arms this leg deliberately: a FREE-plan owner with EnforceItemLimit and no plan override drives the full quota read chain under MaxOpenConns(1) — verified deadlocking before this commit, 0.16s after. Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V	2026-08-17 08:16:02 -04:00
xarmian	79b3220c61	test(server): drive the reval-fault test off the fault seam instead of a sleep (BUG-2570) (#1102) ... * test(server): drive the reval-fault test off the fault seam instead of a sleep (BUG-2570) The reload-fault closure now narrows the member's access on faulting tick 1 and lifts the fault on faulting tick 2, so consecutive reload failures stop at exactly 2 — strictly below the clear-the-watch-set bound — and the green path carries no timing bet at any load. The 300ms sleep is gone; readiness is signaled by the tick sequence itself. Codex round 1 on this fix surfaced that regression DETECTION still has a window (a successful tick 3 masks a hypothetical reset-skipped-on- fault regression), so the interval is set to 500ms to give the revoked PATCH ~10x headroom over measured loaded-runner request latency, and the control-leg wait — the one that timed out in both CI instances — is widened to 10s since it asserts delivery-at-all, not latency. Verified: 5x -race green at both 50ms and 500ms; counterfactual mutant (reset moved to the reload success path) leaks 3/3; full suite + lint green; Postgres leg 2x -race green. Claude-Session: https://claude.ai/code/session_01BhQoeaWXxJbvw86ezzK8dt * test(server): drive reval ticks through a seam — deterministic in both directions (BUG-2570) Codex rounds on the first fix found two regression-DETECTION windows the interval-tuned shape could not close: a stray successful tick before fault installation or after the tick-2 lift resets visCache / reloads the watch list, masking the reset-skipped-on-fault regression this test exists to catch. Interval tuning trades green-determinism against detection-determinism; a free-running ticker cannot give both. So the handler gains watchRevalTickOverride — a test seam mirroring watchPredicatesLoadFault (atomic pointer, read once at stream setup) that lets a test substitute the reval tick source. The test now drives exactly ONE tick, after the access change, with the reload fault active: no early tick can mask via a pre-fault reset, no late tick can mask via a post-lift reload, and one faulting tick can never reach the clear-the-watch-set bound. No sleeps, no interval mutation, no wall- clock bets in either direction. Claude-Session: https://claude.ai/code/session_01BhQoeaWXxJbvw86ezzK8dt	2026-08-15 09:33:29 -04:00
xarmian	599fdbd3f4	feat(watch): drop assignment from the addressed-to-you stream (IDEA-2544 Phase 2, TASK-2551) (#1092) ... * feat(watch): drop assignment from the addressed-to-you stream (TASK-2551) IDEA-2544 Phase 2. Assignment is bookkeeping (who owns this); push is dispatch (where attention goes now). Conflating them meant one triage session assigning N items sprayed N notifications into every open session of the assignee, so Dave's product call (day-33) was to drop assignment from addressed-to-you entirely — no opt-in flag, no config key. watchNotificationVisible loses its KindAssignment early-return; an assignment notification now falls through to the watch-map check like any other item-level fact, which is what an unconditional watch already promises to deliver. Producers are untouched and AssignedUserID is still populated, so a future opt-in re-addressing would be a consumer-side change only. KindPush is now the only addressed kind. Tests: six tests rode the deleted path and are reworked, not deleted. The two mid-stream visibility tests needed new vehicles — the persistent-reload-failure test uses a push (same watch-map-independent property), and the reval-ordering test uses collection-access revocation with a still-granted control item, since push is self-addressed only and its subject is a user losing access. That test's reval interval goes 50ms -> 200ms: at 50ms the clear-the-watch-set bound landed ~30ms behind the assertion and the control leg lost the race. New coverage for the asymmetry the change creates: a push stays exclusive of watch-matched delivery, an assignment does not — a watcher is entitled to see who an item was assigned to. Mutation-tested three ways (restore the old branch; make assignment exclusive addressed-only; couple visCache.reset() to reload success); each is caught by the intended test and each revert was grep-verified. Live: assigning a fresh unwatched item to the connected user leaves the plugin monitor silent, pushing the same item prints one line, and assigning a WATCHED item still delivers — verified end to end against a sandboxed server, not just in tests. Refs TASK-2551, IDEA-2544 * docs(watch): note the deferred plugin wording per Codex review (round 1) Codex's only finding: plugin/monitors/monitors.json and plugin/skills/pad/SKILL.md still describe assignment as addressed-to-you traffic. Correct observation, deliberately out of scope — installed plugins are version-pinned at install, so plugin-visible text reaches nobody without a version bump, and TASK-2564 (PLAN-2558 S6) owns the wording and the bump together. Recording it in code next to the deleted branch rather than leaving a reader to discover the mismatch, and on TASK-2564 with the exact line refs so the follow-up does not have to re-find them.	2026-08-14 17:16:10 -04:00
xarmian	21001bc4c3	feat(sessions): session-presence registry + GET /api/v1/sessions (PLAN-2558 S1) (#1091) ... * feat(sessions): session-presence registry + GET /api/v1/sessions (PLAN-2558 S1) Slice 1 of PLAN-2558 (IDEA-2544 Phase 3, web-UI push). The server can now answer "is anything actually listening right now?" for the calling user. WHY. `pad push` (Phase 1, da6ce642) is fire-and-forget with no "no session connected" warning. That's a defensible contract for a CLI verb typed by someone who knows whether their own session is running. It is not a defensible contract for a web-UI button: "Push to Claude" that silently goes nowhere is worse than the clipboard ferry it replaces, because the user cannot tell the two outcomes apart. Presence lets the UI answer the question before the click, and — once sessions carry a label (S2) — turns the same data into the target picker S5 needs. This also closes the substrate half of PLAN-2469 Phase 3 ("presence surface: SessionStart hook -> live-sessions view", IDEA-2464). The two Phase 3s were the same work; see PLAN-2558's opening section. - internal/server/session_presence.go: SessionPresence interface + MemorySessionPresence. Registered from handleWatchEventsStream, bracketed to the SUBSCRIPTION's lifetime (defer pairs with Unsubscribe's on the adjacent line) so every exit path — ctx.Done, a failed SSE write, the replay-loop returns, the reval-tick paths — releases both or neither. A leaked entry is the failure that matters: it makes the UI promise a listener that is gone, i.e. the same silent nowhere-push with a confident label on it. - internal/server/handlers_sessions.go: GET /api/v1/sessions, self-scoped. No ?user_id=, no admin bypass — who has an agent session open is a presence signal about a person, and the same reasoning that made push self-addressed only applies. 503 (not 200-with-empty-list) when no registry is wired: "I can't tell" and "nobody is listening" must not look the same to the UI, since collapsing them is exactly the dishonesty this slice exists to remove. Interface from day one because MemorySessionPresence is per-process. Its doc comment states the boundary precisely rather than hand-waving it: watchevents.Bus is blind in the SAME direction (a push published on instance A never reaches a stream on instance B), so per-instance presence is as accurate as per-instance delivery and both stop being trustworthy at the same boundary — except that a load balancer may route a POST and a GET to different instances, at which point they disagree. A Redis-backed presence must therefore land WITH the Redis watchevents.Bus that package already anticipates, not separately. No pad-cloud change required (checked, not assumed): /api/v1/sessions is a plain JSON GET served by nginx-router.conf's default `location /` pass-through — the special long-lived-connection blocks are for /api/v1/events and /api/v1/collab/ only. Verified: go test ./... (SQLite) clean; make test-pg clean (25 pkgs, exit 0); make lint 0 issues; new tests pass under -race. Live, on the installed binary: 0 sessions with nothing connected -> 1 with one stream open -> 2 with two, oldest-first -> back to 0 after both disconnect, with a second user's list staying empty throughout. * fix(sessions): no-store the presence response; document two lifetime constraints (PLAN-2558 S1) Codex round 2 findings, both verified against source before acting. P2 — Cache-Control. GET /api/v1/sessions set no cache header: writeJSON sets none and the jsonContentType middleware only sets Content-Type, so the response was heuristically cacheable. Now `private, no-store`, matching the house pattern for per-user sensitive responses (handlers_attachments.go:585). Wrong two ways without it: a shared cache could serve one user's presence to another (the same boundary this endpoint's absent admin view exists to hold), and a cached liveness answer is exactly the confident-but-wrong "1 session connected" the slice exists to prevent. Pinned by a test. P2 — Shutdown, REFINED rather than adopted as reported. Server.Shutdown delegates to http.Server.Shutdown, which does not cancel an in-flight handler's context; SSE handlers therefore hang until their own ctx.Done or a failed write. True, but for MemorySessionPresence it is HARMLESS, and that is the useful half: the registry lives in the process that is going away, so its entries die with it. There is nothing to reap. A Redis-backed implementation does not inherit that — its entries outlive the writing process, so a crash strands them permanently rather than for 30 seconds. Recorded as a hard constraint on the interface: any out-of-process implementation must carry its own reaping story (TTL plus heartbeat renewal, or instance-keyed ownership swept at startup). Also documents the staleness window neither codex round surfaced, found in my own pass: a clean disconnect deregisters immediately, an ungraceful one is invisible until the next keepalive write fails, and the keepalive is 30s. So the list can name a dead listener for up to ~30 seconds. That bound is fine for a fire-and-forget channel — a push to a session that died 5 seconds ago loses a message that was lost anyway — but consumers must not upgrade it into a delivery guarantee. Shortening it means shortening the keepalive, which taxes every idle connection; the right answer for a consumer that needs delivery confidence is an ack, not a faster heartbeat. Verified: go build, go vet, make lint 0 issues, presence tests green under -race.	2026-08-14 17:16:07 -04:00
xarmian	da6ce642da	feat(push): pad push — user-authored instruction dispatch to agent sessions (IDEA-2544 Phase 1) (#1090) ... * feat(push): add pad push <ref> -m vertical (IDEA-2544 Phase 1) Self-addressed, human-to-harness dispatch over the existing watch-events bus/stream: CLI -> POST .../items/{itemSlug}/push -> a new KindPush Notification (carrying the generalized TargetUserID addressed-to field KindAsk will later share) -> watchNotificationVisible delivers it back to the pushing user's own connected monitor sessions. Transient, fire-and-forget by design (no migration, no durable inbox) since assignment already covers the durable-notification case and this is meant to be the explicit, no-inference dispatch verb instead. * docs(plugin): document the push notification contract (IDEA-2544 Phase 1) Push is the one notification kind that IS an instruction rather than a passive fact, so it gets its own lead bullet in the plugin skill's notification-etiquette section (ahead of the read-only/park default, which it explicitly lifts) and a mention in the monitor's description. The embed-source skills/pad/SKILL.md has no notification section to mirror this into (the two files diverge by design) and is left untouched. * fix(push): reject over-long push messages instead of unbounded Summary Comments truncate their notification Summary to a preview (the full body is still fetchable), but a push message IS the payload — silently truncating it would corrupt the instruction with nothing to recover it from. Add maxPushMessageLen (4096, measured post-collapse) and reject anything over it with a 400 rather than truncating; state the same bound in `pad push --help` so it's discoverable before a 400, not only from one. * fix(push): close the watch-fallthrough leak, disambiguate SKILL.md exceptions Codex round 1 P1: watchNotificationVisible's push branch only returned early on a MATCH — a non-target caller fell through to the watch-map check below it, so anyone holding an unconditional (or predicated) watch on the item received every push addressed to every OTHER user, instruction text included. Push is addressed private dispatch, not an item-level fact watchers have a legitimate claim on (unlike assignment, which watchers are expected to see per `pad watch --help`) — the branch now returns unconditionally for KindPush, gating strictly on TargetUserID and never reaching the watch-map fallback either way. Pinned explicitly since Phase 4's session targeting is expected to inherit this same exclusivity. Also (codex P2): reworded the SKILL.md notification-etiquette bullets — the new push exception and the pre-existing assignment/ask exception literally contradicted each other ("the ONE narrow exception" claimed singularity after push had already claimed exception status). Now explicitly enumerated as the first and second exceptions to the never-write rule. * test(cli): pin that PushItem inherits X-Pad-Agent (BUG-2542 rebase) Verified, not assumed: PushItem builds its request via c.post -> c.newRequest like every other mutating client method (CreateWatch included), so the attribution fix's client.agentName wiring covers it for free with zero code changes needed on this branch. Adds a live httptest assertion rather than trusting the code-path read alone — the same shape as TestClientSendsResolvedAgentHeader, scoped to PushItem specifically since that's the one method this PR added. * fix(push): disambiguate workspace in the monitor line and skill contract Codex round 2 P1: the watch-events stream is user-scoped ACROSS every workspace a caller has watches in, but formatMonitorLine printed only ItemRef/Kind/Actor/Summary and dropped the Workspace field the wire payload already carried — a session linked to workspace A receiving a notification for workspace B would resolve the wrong item (or 404) with no signal in the line that anything was off. Fixed universally, not push-only: grepped plugin/ and skills/ for anything parsing "PAD ..." lines and found none — the Claude Code plugin host ingests the stdout line as free-text notification prose, formatMonitorLine's only real consumer is its own fmt.Println, so there is no wire-format consumer a workspace prefix could break. The ambiguity predates push (any watched item across workspaces already had it); push just makes the consequence sharper because it carries an instruction rather than a passive fact. SKILL.md's push bullet now tells the agent to resolve with `pad --workspace <workspace> item show <ref>` using the slug read off the notification line, not a bare `pad item show <ref>`. * fix(push): respect --format json instead of hardcoding plain text Codex round 2 P2: pushCmd's RunE ignored the global format flag and always printed "Pushed <ref>", silently discarding --format json. - server.pushResponse replaces the bare map the handler wrote before — a typed {ref, workspace, pushed, message} shape, with workspace resolved to the CANONICAL slug via s.getWorkspace (not merely echoed from whatever the URL contained), matching the same disambiguation need the round-2 P1 fix addressed for the monitor line. - cli.PushItem now returns (*PushResult, error) instead of discarding the response body. - pushCmd checks formatFlag == "json" and calls cli.PrintJSON, mirroring runCreateWatch's existing pattern. internal/cli/agent_identity_test.go's TestPushItemSendsResolvedAgentHeader needed a one-line update for PushItem's new two-value return — caught by `go vet ./...`, not `go build ./...` (which doesn't compile test files); folding vet into my own pre-flight going forward.	2026-08-13 18:41:46 -04:00
xarmian	ec7fd027fc	feat(server,cli): watches, user-scoped event stream, plugin monitor command — PLAN-2469 Phase 1 (TASK-2533) (#1082) ... * feat(store): race-free status/assignment mutation signal (TASK-2533) Adds models.Item.LastMutation (ItemMutationSignal), populated inside the SAME transaction that already writes status_transitions / assigned_user_id in UpdateItemWithParentLink and MoveItemWithPreCheck. This is the foundation for TASK-2533's watch-notification pipeline: a before/after snapshot taken in the HTTP handler layer would race concurrent writers of the same item, so the signal is computed where the authoritative diff already happens, in-transaction. * feat(store): watches table migration, both drivers (TASK-2533) watches(id, workspace_id, user_id, item_id, predicate, created_at) per DOC-2479's subscription-table design: durable, server-side subscriptions that survive both the plugin-monitor process and a padd restart. uq_watches_user_item makes `pad watch <ref>` idempotent (re-watching upserts the predicate). Wires watches into the workspace-purge child-delete list, mirroring item_stars. * feat(watchevents): add in-process notification bus (TASK-2533) New package: a global (not per-workspace) in-process pub/sub bus carrying watch-worthy Notifications (status-change / assignment / comment; ask reserved in the enum with no producer yet — see the follow-up server commit). Bus is an interface specifically so a Redis-backed implementation can slot in later without touching any caller; only MemoryBus exists today. Package doc comment states the single-process/multi-instance limitation explicitly, mirroring internal/events' shape. * feat(store): watches CRUD (TASK-2533) models.Watch + Store.CreateWatch (upsert on user+item)/GetWatchByUserItem/ ListWatchesForUser (unscoped by workspace — a watch is personal, and the event-stream handler needs every watch a caller holds across all their workspaces)/DeleteWatch. * feat(server): watch/nudge event stream + CRUD endpoints (TASK-2533) GET /api/v1/events/stream (DOC-2479): a user-scoped, cross-workspace SSE stream, filtered server-side to the caller's watches (with optional --until field=value predicate) plus "addressed to you" — narrowed to assignment-to-you only for Phase 1, confirmed with the dispatcher: this codebase has neither a Collection.Kind field nor any user->active-role binding to ground DOC-2479's "human-gate-shaped collection targets your active role" half mechanically. watchevents.KindAsk stays in the wire enum with no producer. `pad session register` is the natural future hook for a session-carried role identity. POST/DELETE .../items/{slug}/watch, GET /api/v1/watches (unscoped, mirrors /auth/tokens' shape for a personal, not workspace, resource). Producer wiring (TASK-2533 audit) publishes from every live mutation path that can produce a LastMutation signal or a new comment: handleUpdateItem (incl. its collab sub-paths and the comment-attached-to-update path, which bypasses handleCreateComment entirely), handleMoveItem, handleCreateComment, item creation with an initial assignee, and the bulk-items loop (covers archive/restore/move/set-priority/tag/untag/assign uniformly via one call site). Named, not silent, bypasses: import bundle, status_transitions backfill, workspace restore/purge — none are live human-facing mutations. Known Phase-1 tradeoff, flagged not fixed: bulk mutations are NOT batched into one notification the way the existing SSE/webhook bulk path is — a bulk-assign of N items surfaces N individual notifications. Each is still correctly scoped by the recipient's own watches/addressed-to-you filter (a narrower audience than the workspace-wide SSE firehose the existing batching protects), so this is a noise-discipline tradeoff, not a leak. * feat(cli): pad watch + pad session register (TASK-2533) pad watch <ref> [--until field=value] creates/upserts a durable watch; pad watch list / pad watch remove <ref> are the hygiene companions the dispatcher asked to be included explicitly rather than silently added. pad watch --stream --for-session is the plugin-monitor command: one stdout line per matching event ("PAD TASK-214 -> kind (actor): summary"), silent on startup with no .pad.toml (hourly retry) or an unreachable padd (backoff retry) per DOC-2479's noise-discipline contract. The retry/ backoff math and line formatting are pure, unit-tested functions; the actual sleep loop is not (per the dispatcher's ask). pad session register writes ~/.pad/sessions/<pid>.json (pid, cwd, CLAUDE_CODE_MESSAGING_SOCKET when set) -- forward-looking infra for Phase 3's live-sessions/presence surface; nothing consumes it yet in Phase 1/2. * fix(server): comment replies never published a watch notification (TASK-2533) Codex round 1 finding 2 (verified real, not a false positive): handleCreateReply is a SEPARATE code path from handleCreateComment — it calls store.CreateComment directly via POST .../comments/{id}/replies, not POST .../comments — and was missing the watch-notification hook entirely. A reply to a comment on a watched item produced zero notification. Same kind=comment publish as the top-level path, plus a regression test covering the reply route specifically. * fix(server): re-check current access before serving/delivering watches (TASK-2533) Codex round 1 finding 1: ListWatchesForUser filtered only by user_id — a watch row survives a revoked workspace membership or grant (nothing deletes it), so GET /api/v1/watches and the event-stream's notification filter could keep leaking item title/ref, workspace slug, actor, and summary for access the caller no longer has. Adds Store.ListWatchesForUser's ItemCollectionID column (needed for the visibility check) and server.filterWatchesByCurrentAccess, which mirrors computeSSEVisibility's RBAC resolution (handlers_events.go) — admin bypass, VisibleCollectionIDs for member/guest full-collection access, GuestVisibleResources for item-level grants — grouped by workspace since a caller's watches can span many, unlike a single SSE connection scoped to one. Fails closed on any lookup error. Wired into handleListWatches here; the event-stream's loadWatchPredicates call site picks up the same filter in the next commit, which also restructures that function's Subscribe/replay sequence and therefore touches the same lines. * fix(watchevents): atomic ID assignment + subscribe-and-replay (TASK-2533) Codex round 1, findings 3 and 4 (same subsystem, fixed together): Finding 4 — sequence assignment and replay-buffer insertion happened under SEPARATE locks in MemoryBus.Publish. Two concurrent Publish calls could append to the ring buffer out of ID order, corrupting since()'s ordering assumptions (it walks the ring oldest→newest assuming monotonic IDs). Fixed by unifying seq assignment, buffer append, and the subscriber-list snapshot under one lock; the (already non-blocking) fan-out send still happens after releasing it. Finding 3 — GET /api/v1/events/stream called Subscribe() and, later (when resuming via Last-Event-ID), EventsSince() as two separate calls. A Notification published in the window between them landed in BOTH the replay result and the live channel, double-delivering it. Bus gains SubscribeAndReplaySince(sinceID), which atomically subscribes and reads the replay buffer under the SAME lock; the stream handler now uses it whenever a Last-Event-ID is present (this commit carries that call-site change, plus the finding-1 loadWatchPredicates filter wiring from the previous commit — both land in the same lines of this function). Adds a concurrent-publish ID-ordering test and a subscribe-then- concurrent-publish no-duplicate test, both run with -race. * fix(cli): monitor silent-start ordering + sync_required handling (TASK-2533) Codex round 1, findings 5 and 6: Finding 5 (P1) — runWatchMonitor called getClient() once, before the loop and before the .pad.toml check. getClient() -> getConfiguredConfig() os.Exit(1)s when unconfigured with no TTY, or launches an INTERACTIVE configuration wizard when one is attached — either way a direct violation of DOC-2479's silent-start contract, which requires "not ready yet" to be a silent retry, never a crash or a prompt. Adds monitorClient(), which builds the client the same way but returns a plain error instead of exiting or prompting; client construction now happens INSIDE the loop, after the .pad.toml gate, on every iteration, and its failure folds into the existing padd-unreachable backoff path. Finding 6 (P2) — streamWatchEvents ignored "sync_required" (the server's signal that the requested Last-Event-ID was evicted from its replay buffer), so a stale cursor got resent on every reconnect forever. Now clears the cursor on sync_required so the next reconnect is a fresh, non-resuming subscription instead. Both covered by tests that assert the goroutine returns promptly on context cancellation (proving no os.Exit / no blocking prompt was hit, since the test process itself is still running to observe the return) and that streamWatchEvents clears/re-tracks the cursor correctly around sync_required. * fix(server): uniform current-access gate for watch AND addressed-to-you delivery (TASK-2533) Codex round 2, findings 1 and 2 — same subsystem (watch/nudge delivery access control), fixed together; finding 2 explicitly falsifies finding 1's fix's own admin-bypass argument, so this replaces that reasoning rather than patching around it. Finding 1 (confirmed real): VisibleCollectionIDs / GuestVisibleCollectionIDs deliberately over-widen for navigation — a collection ID is included if the caller has an item grant on ANY item inside it, explicitly leaving item-level narrowing to the caller (their own doc comments say so). computeWatchAccessVisibility used that over-wide set directly as the "fully visible" gate, so a guest granted item A was treated as having full access to A's WHOLE collection, including an ungranted sibling item B. Fixed by building the "genuinely full access" set from GuestVisibleResources' fullCollectionIDs (populated only from direct collection_grants, never widened by an item grant) + GetMemberCollectionAccess / ListSystemCollectionIDs for an actual member — exactly computeSSEVisibility's own fullCollSet construction, not an approximation of it. Finding 2 (confirmed real): the addressed-to-you (KindAssignment) branch in watchNotificationVisible returned true unconditionally, with NO access check. validateAssignmentScope (internal/store/items.go) only checks WORKSPACE membership, never collection access, so an item can be assigned to a "specific"-access member whose granted collections don't include it at all — an ordinary assignment, no revocation timing required. Fixed by gating EVERY notification kind — watch-matched and addressed-to-you alike — through the SAME watchAccessVisibility check before either branch runs. watchevents.Notification gains CollectionID so the check has what it needs without a second lookup; the stream handler resolves it lazily per workspace via a small connection-scoped cache (workspaces aren't known in advance for addressed-to-you the way watch workspaces are), cleared on the same reval tick that reloads the watches map. This also required replacing computeWatchAccessVisibility's admin-bypass argument, not just its code: "every call site filters the caller's OWN watches" stopped being a sufficient justification once addressed-to-you (which is fundamentally about *this* caller's own assignment activity across every workspace) shares the same gate — a bearer-borne admin token unconditionally trusted for that is exactly BUG-1616's blast radius. Now mirrors computeSSEVisibility's cookie-vs-bearer distinction exactly. Tests: guest-with-item-grant no longer sees a sibling item's watch or stream notification (filter-level and HTTP/SSE-level); an assignment outside a restricted member's granted collections is denied at both levels; addressed-to-you is proven still gated (denied with no access, visible once granted) as a pure unit test. * fix(store): always re-read existing under lock, not just for precheck/patch updates (TASK-2533) Codex round 2 finding 4, verified real: updateItemWithParentLinkOnce's `existing` snapshot was only refreshed under the write lock when precheck != nil, ExpectedUpdatedAt != "", or FieldsPatch != nil — any update touching none of those (e.g. a plain title-only PATCH) kept the STALE pre-tx `existing` for the rest of the function, including the LastMutation assignment-delta comparison added in TASK-2533's first round. A concurrent OTHER transaction's assignment change landing between this transaction's pre-tx read and its lock acquisition would get misattributed to THIS transaction: a title-only update could report a spurious, wrongly-attributed AssignmentChanged for a transition it never made, duplicating the one the other transaction already reported correctly (or missing a real one, depending on interleaving). The status-transition capture already defended against exactly this with its own separate conditional re-read; the assignment-delta capture added later did not replicate that guard. Fixed by making the re-read unconditional — once, right after the locks are held, before any SET- clause building or the UPDATE itself — so every existing.* comparison in this function is race-free by construction, not by each caller remembering to guard itself. Also removes the now-redundant duplicate re-read the status code had of its own. Reproduces the exact race deterministically using UpdateItemWithPreCheck's precheck hook as a synchronization point (TX2's assignment change blocks mid-transaction while TX1's title-only update races its own pre-tx read against it) — the new test fails reliably against the pre-fix code and passes reliably (including under -race, and in Postgres mode) against the fix. * fix(watchevents): send under the same lock Unsubscribe/Close use (TASK-2533) Codex round 2 finding 3, confirmed real and high-severity: Publish snapshotted subscriber channels under the lock, released it, and only then sent to them. A concurrent Unsubscribe or Close could close one of those channels in the window between the snapshot and the send — a send on a closed channel PANICS in Go, which crashes the whole padd process, not just one subscriber's connection. The reasoning for releasing the lock before sending ("a slow subscriber would stall everyone else") didn't hold up: the send is already non-blocking (select/default — a full channel is dropped-and-logged, never awaited), so holding the lock through it costs nothing and closes the window structurally. Adds a hammer test (many iterations of concurrent Publish / Subscribe / Unsubscribe / Close, short-lived churned channels, recover()-wrapped so a regression fails cleanly instead of crashing the whole `go test` run) that reproduces "send on closed channel" dozens of times per run against the pre-fix code (plus an independent -race detection) and passes cleanly, repeatedly, against the fix. * fix(server): re-fetch the user, not just the vis map, on each reval tick (TASK-2533) Codex round 3, confirmed real: watchVisCache captured *models.User ONCE at connect time (newWatchVisCache) and never re-fetched it; reset() cleared only the per-workspace visibility map. computeSSEVisibility's own doc comment explains why it re-fetches the user fresh on every call — "so mid-stream role changes (admin demotion, user.disabled flips) take effect on the next tick" — and the round-2 commit claimed to mirror that "exactly," but only carried over the collection/bearer logic, not the re-fetch itself. Net effect: a demoted or disabled admin kept fullAccess on an open stream (both watch-matched and addressed-to-you delivery, since both go through this same cache) until reconnect. Adds watchVisCache.refreshUser, called by both the constructor and reset() so the cadence matches computeSSEVisibility's actual cadence in handlers_events.go (that function is invoked once at connect and again only on each membershipCheck tick — never per event — so "per cache reset" here is the same cadence, not a narrower one). Deliberately fails CLOSED (not open-to-stale like computeSSEVisibility's own transient-error fallback) on a fetch error, a deleted user, or a disabled user — a nudge stream's wrong failure mode is delivering a fact to someone who shouldn't see it, not a dropped UI update, so this trades computeSSEVisibility's availability-leaning fallback for a stricter one and says so in the comment rather than repeating the "mirrors exactly" claim the fix falsified. Tests: a unit-level pair (mirroring handlers_events_revalidation_test.go's existing admin-demotion/disable coverage of the analogous SSE gap exactly) proves an admin loses fullAccess after a demotion + reset(), and a disabled user is denied outright; an HTTP/SSE-level test proves a live stream stops delivering entirely once its connected user is disabled and a reval tick passes. All three reproduce the bug reliably against the pre-fix code and pass cleanly against the fix. The HTTP-level test deliberately runs serially (not t.Parallel()): it mutates the package-level watchListRevalInterval var, which every other parallel watch-stream test in this package also reads via its own ticker — writing to it from a t.Parallel() test raced against those reads under -race (misattributed by the race detector to a whole cluster of unrelated concurrently-running tests before this was diagnosed). Full server package -race pass is clean after the fix. * fix(server): decouple vis-cache reset from watch-list reload success (TASK-2533) Codex round 4, confirmed real: on a reval tick, if ListWatchesForUser errored, the handler's `continue` skipped visCache.reset() entirely — the two were coupled, with reset() only reachable on the reload's success path. A demoted or disabled user's stale identity/visibility (round 3's fix) stayed live for exactly as long as that UNRELATED query kept failing, so the round-3 leak reopens for the duration of any watch-list reload error. Fixed by running visCache.reset() first, unconditionally, before attempting the watch-list reload. On a reload failure, the stale watch list is kept (its own staleness is already bounded by watchListRevalInterval's "eventually consistent" contract) but is now gated by the FRESH visCache regardless — a demoted/disabled user is denied via visCache even while the watch list itself lags a tick. Chose this over dropping all delivery for the tick (the other option the finding offered) because tying stream availability to an unrelated query's transient health seemed like the wrong tradeoff; the comment at the call site states this choice explicitly. Adds a watchPredicatesLoadFault test seam on *Server (mirrors the existing restoreAckFault pattern) so the reload failure can be forced deterministically without breaking the DB connection for the whole test. Reproduces the exact bug: forces the reload to fail on every tick while concurrently disabling the connected user, and asserts addressed-to-you delivery (which depends only on visCache, never the watch list) is denied anyway. Fails reliably against a reverted (pre-fix, coupled) version of the reval branch and passes cleanly against the fix. Full server package -race pass, full suite (SQLite + Postgres) pass, lint clean — this is the pre-PR verification matrix; round 5 will be a narrow re-verify of this fix only. * fix(server): bound stale watch set under persistent reload failure; atomic test seam (TASK-2533) Codex round 5, two P2s, both confirmed real: Finding 1 — `watches = fresh` only ran on the reload's success path, so under a PERSISTENT (not single-tick) reload failure the watch set stayed live indefinitely: a dead watch (removed, item deleted) kept matching forever, and a watch created during the outage was silently missed forever — visCache (round 4) gates current ACCESS, not whether a watch still legitimately exists, so it couldn't catch this on its own. Fixed by tracking consecutive reload failures and clearing the watch set once maxConsecutiveWatchReloadFailures (3 ticks) is crossed, failing closed on watch-matched delivery specifically while addressed-to-you delivery (visCache-only, unaffected either way) continues throughout. Updated the tradeoff comment at the call site so the "eventually-consistent" claim now matches the bounded, not unbounded, behavior it actually describes. Finding 2 — the watchPredicatesLoadFault test seam was a plain `func() error` field, written by a test AFTER the SSE stream's background goroutine was already running and reading it on every reval tick: genuinely racy, unlike restoreAckFault's own use of the identical field shape, which is set once, synchronously, before the single HTTP request that reads it — goroutine creation's happens-before edge makes THAT usage safe without any extra synchronization. Verified restoreAckFault does not share the flaw and left it untouched. Fixed the watch seam with atomic.Pointer[func() error] instead. Test for finding 1: forces maxConsecutiveWatchReloadFailures+1 consecutive reload failures via the (now-atomic) fault seam and asserts watch-matched delivery is suppressed once the bound is crossed while addressed-to-you keeps delivering, then clears the fault and confirms watch-matched delivery resumes on the next successful reload — a bounded outage response, not a one-way ratchet. Fails reliably against the bound disabled, passes cleanly restored. This is the (re-run) pre-PR verification matrix per the dispatcher: SQLite + Postgres + full-suite -race + lint + gofmt, all clean. Round 6 is a narrow re-verify of these two fixes only. * test(store): bound the concurrent mutation-signal test's wait (TASK-2533) CI-triage follow-up: PR #1082's plain Postgres step hit go test's default 10-minute per-binary timeout. Investigated whether any store test added by this branch scales with runner slowness (lock-wait defaults, sleep-based polling, transaction-hold durations): - Watches CRUD tests (8): 0.63-0.80s each under Postgres, isolated and in the full 741-test package run. - Mutation-signal tests (6), including the precheck-hook two-transaction race test: 0.48-0.80s each; the race test held at 0.48-0.51s across 10 consecutive runs (no variance) and across the full-package run. - Full store package under Postgres: 279.17s and 277.12s across two runs on this branch, matching the ~275s/297s baseline team-lead measured locally and on PR #1081 — no reproducible slowdown from anything this branch adds. No pathological test found locally. The one test with genuine cross-goroutine DB lock contention (TestLastMutation_AssignmentDelta_ NotMisattributedUnderConcurrentWrite) had an unbounded wg.Wait() as its only unbounded wait — TX2's release was already unconditional (fixed 50ms sleep, not gated on TX1's progress), so there's no deadlock risk, but there was no ceiling on how long legitimate lock contention could stretch it under a slow/shared runner. Replaced with a bounded 10s wait that fails fast with a diagnostic instead of silently consuming test-binary budget if it's ever exceeded. Verified the regression test still fails reliably (5/5) against a revert of the round-2 fix it guards. Could not reproduce the CI timeout locally; likely the pre-existing ~297s CI baseline (already noted as close to the 10-minute ceiling) plus environmental variance on the shared runner, not a specific test this branch adds.	2026-08-12 15:50:41 -04:00
xarmian	f8ff5742e5	feat(server): add cross-workspace copy endpoint with post-commit fanout (TASK-2365) ... Claude-Session: https://claude.ai/code/session_01E2fRi12n8rARczvdEa2LYT	2026-07-31 00:59:25 +00:00
xarmian	01d640978c	feat(server): add cross-workspace copy dry-run preflight endpoint (TASK-2364) ... Claude-Session: https://claude.ai/code/session_01E2fRi12n8rARczvdEa2LYT	2026-07-30 20:15:00 +00:00
xarmian	5804c80146	feat(server): add cross-workspace authorization helper (TASK-2358)	2026-07-30 12:20:38 +00:00
xarmian	e601f2b368	fix(collab): reconcile Postgres commit-ack-loss on version restore instead of treating it as rollback (BUG-2276 residual 1) ... On Postgres, a version-restore commit that durably lands but whose ack is lost surfaced as an error and wrongly resumed peers on a stale Y.Doc. ForceRefreshRoom now runs a Postgres-only reconcile after a commit error: two durable signals (content == restored version AND last_restore_seq advanced past a lock-captured baseline) must agree → LANDED (publish fences + reseed, return the restored item + SSE); both false → rolled back (unfreeze); disagree/read-error → UNCERTAIN (invalidate in-memory fences so durable state governs, then plain-close sockets so peers reconnect + re-evaluate). SQLite path unchanged. Confirming Codex (high effort): 3 rounds — false-404, frozen-forever, archive-nil, stale-baseline, stale-in-memory-fence-clobber all closed; real Postgres end-to-end ack-loss + SSE test. make test-pg green. Residual 2 (applier-ack rollback race) follows separately. Go CI red only on the pre-existing govulncheck advisory (BUG-2278). https://claude.ai/code/session_01EZ6yr6pAUFb1uffan912ra	2026-07-21 18:00:49 -04:00
xarmian	c8492db29f	fix(e2e): disable rate limiting on the E2E server to stop 429 flakes (BUG-2089) (#922) ... The E2E harness runs the real pad binary with the real rate limiter, and every Playwright test shares one loopback IP (127.0.0.1). The auth limiter (5 logins/min/IP, burst 5) trips as soon as a spec logs in a couple of browser clients — collab-persistence.spec.ts logs in two per test — so browserLogin fails with "in-page login failed with status 429". This was deterministic, not flaky: it failed on TASK-2058's own PR and its push to main, and on every downstream PR since. Add a test-only env knob PAD_DISABLE_RATE_LIMITS: when truthy, New() leaves Server.rateLimiters nil, which RateLimit() already treats as a pass-through (Stop() and the MCP path are already nil-safe). Wire it into the Playwright webServer.env; run-pad.mjs spawns the binary with inherited env so it reaches the pad process. Limiters stay fully active in prod/self-host — the knob is an explicit opt-in only the E2E server sets. Verified: collab-persistence.spec.ts passes locally with the fix; the existing limiter tests still pass (limiters on when the env is unset); new TestRateLimit_DisabledByEnv pins the bypass. Claude-Session: https://claude.ai/code/session_015yuBJQYfDj95cgX3DaD8SF	2026-07-11 14:30:32 -04:00
xarmian	3f69b76b06	feat(security): enforce session UA binding under strict mode (TASK-2056) (#912) ... Session IP/User-Agent binding was log-only by default, so a stolen session token granted durable any-origin access. IP-change enforcement already existed behind PAD_IP_CHANGE_ENFORCE=strict; this extends the same single toggle to also enforce the User-Agent-hash binding. When strict enforce is ON, a request whose client IP OR User-Agent hash no longer matches the session's stored binding now revokes the session (DeleteSessionIfExists) and rejects the request (401 for API, revoked-passthrough for public/browser paths), killing the stolen token. When enforce is OFF (default), behavior is unchanged: UA mismatch is logged (slog only, no new audit row) and the request proceeds, so existing self-host users see no behavior change and routine client churn (browser/WebView updates, DevTools emulation, mobile-app rebuilds) is tolerated. The UA hash is stable within a real session, so UA-mismatch enforce carries fewer false positives than IP enforce (mobile roaming, VPN toggles, carrier NAT) — documented in the handler comment. Adds the ActionSessionUAChanged audit action, emitted only in strict mode. No DB migration: reuses the existing IPChangeEnforce config flag and the existing session store primitives. Claude-Session: https://claude.ai/code/session_015yuBJQYfDj95cgX3DaD8SF	2026-07-10 23:32:22 -04:00
xarmian	aeb80883f0	fix(webhooks): track delivery goroutines + bounded retry (BUG-2012) (#864) ... * fix(webhooks): track delivery goroutines + bounded retry (BUG-2012) Webhook deliveries ran in untracked `go d.deliver(...)` goroutines that write to the store — the BUG-842 shutdown-race class that goAsync was built to prevent — and had no retry. - Inject a `spawn func(func())` into Dispatcher (SetSpawn). Server wires s.goAsync via SetWebhookDispatcher so deliveries are tracked on s.bg (Stop() waits for in-flight deliveries) and inherit goAsync's panic recovery (BUG-2011). Nil spawn falls back to a plain goroutine, so standalone Dispatcher usage is unchanged. - Add a bounded in-goroutine retry: up to 3 attempts with linear backoff on transient failures (network error / timeout / 5xx). Permanent failures (4xx, SSRF block, malformed URL) stop immediately. The final outcome is recorded once via UpdateWebhookFailure. - Tests: delivery runs on the injected spawn; transient 5xx retries to the cap; permanent 4xx does not; a recovered transient records success. Claude-Session: https://claude.ai/code/session_019knGmnHcx5rrgWXQ8V8DZS * fix(webhooks): classify redirect-block + non-5xx as permanent (Codex review) Second Codex review of PR #864 found two retry-classification gaps: - P2: an SSRF-blocked (or looping) redirect surfaces as an error from client.Do (via CheckRedirect), which the retry loop treated as transient — so a redirect to an internal target was retried 3x with backoff. Wrap a sentinel (errRedirectRejected) in checkRedirect and match it with errors.Is (url.Error unwraps to it) to classify these as permanent — attempted once, no retries. - P3: the status switch treated every non-2xx/non-4xx as transient. Narrow transient to 5xx only; 4xx/3xx-no-Location/1xx are permanent, matching the stated "network error / timeout / 5xx" retry policy. Adds TestDispatcher_RedirectBlockIsPermanent. Claude-Session: https://claude.ai/code/session_019knGmnHcx5rrgWXQ8V8DZS	2026-07-08 12:02:49 -04:00
xarmian	a04fd861dc	fix(server): add panic recovery to background sweeper goroutines (BUG-2071) (#865) ... The four long-running sweeper loops (orphan GC, op-log GC, token reaper, workspace purge) spawn their own s.bg-tracked goroutine with a stop-channel lifecycle, so they can't route through goAsync (a fire-and-forget helper that owns the whole goroutine) without breaking shutdown or double-counting s.bg. As a result they had NO recover(): a panic in any sweeper body crashed the single-binary server for every tenant. Add a shared Server.recoverSweeper(name) firewall — mirroring goAsync's recover + debug.Stack slog style — and defer it inside each sweeper goroutine. A panic is now logged with a stack and the goroutine unwinds cleanly; its own deferred s.bg.Done() still fires (recover stops the unwind), so Stop() still drains. No change to any sweeper's loop cadence or stop-signal shutdown. Adds TestTokenReaper_RecoversPanic, which drives a real reaper tick to panic (nil store → nil-pointer deref in the first cleaner) and asserts the panic is logged+recovered and Stop() returns. Claude-Session: https://claude.ai/code/session_019knGmnHcx5rrgWXQ8V8DZS	2026-07-08 11:58:41 -04:00
xarmian	f7c4cb3287	fix(server): add panic recovery to goAsync background tasks (#863) ... goAsync wrapped fn in a bare goroutine with no recover(); chi's Recoverer only covers request goroutines, not these detached ones. A panic in a background task (e.g. deriveThumbnails hitting a Go image-decoder panic on a crafted upload, or an email send) would unwind past the goroutine and crash the whole single-binary server for every tenant. Add a single deferred recover() inside the goAsync goroutine that logs the panic + stack via slog, covering all 15+ call sites at once. The recover defer is registered after `defer s.bg.Done()`, so it runs first on unwind and Done() still fires — Stop() continues to drain the WaitGroup even when fn panics. Adds TestServer_goAsync_RecoversPanic asserting the process survives a panicking fn and Stop() returns. Fixes BUG-2011. Claude-Session: https://claude.ai/code/session_019knGmnHcx5rrgWXQ8V8DZS	2026-07-08 11:49:01 -04:00
xarmian	f5b437a65f	feat(server): workspace restore + deleted-list endpoints (TASK-1970) (#827) ... Foundation for PLAN-1969 (user-recoverable workspace soft-delete). A workspace delete only stamps workspaces.deleted_at; items/collections/ members are untouched, hidden transitively. Restore clears deleted_at so everything re-surfaces intact. Store (internal/store/workspaces.go): - RestoreWorkspace(slug): UPDATE ... SET deleted_at = NULL WHERE slug=? AND deleted_at IS NOT NULL. Returns sql.ErrNoRows (-> 404) when no soft-deleted row matched (already live or purged). - ListDeletedWorkspaces(userID, cutoff): owner-scoped, deleted_at within the window, ordered deleted_at DESC. Account-deleted workspaces have no live owner, so they never leak. - GetDeletedWorkspaceBySlug(slug): resolves a soft-deleted row (the normal resolvers filter deleted_at IS NULL) so the handler can tell 403 from 404. - Dual-dialect via s.q/s.dialect; no migration (deleted_at already exists). Handlers (internal/server/handlers_workspaces.go): - POST /api/v1/workspaces/{slug}/restore: owner-only; 404 not-restorable, 403 non-owner, 200 + restored workspace; logs a "restored" activity. - GET /api/v1/workspaces/deleted: owner-scoped list with per-entry purge_at + days_left, both derived from workspacePurgeRetention so restore and the purge sweeper share ONE 30-day window (no drift). - Both routed outside the /{slug} RequireWorkspaceAccess subrouter (which resolves only live workspaces); restore enforces owner authz inline. CLI client (internal/cli/client.go): RestoreWorkspace + ListDeletedWorkspaces. TS type (web/src/lib/types/index.ts): Workspace.deleted_at + DeletedWorkspace. Tests: store (resurface-intact; double-restore/live -> ErrNoRows; window boundary 29d IN / 31d OUT + owner-scoping) and handler (owner-only 403, 404 live/unknown, 200 restore, owner-scoped deleted-list). Green on SQLite and Postgres (make test-pg); golangci-lint clean. Closes TASK-1970 Claude-Session: https://claude.ai/code/session_01HxBkAMiFBtCRJ2tKSCt3ST	2026-07-06 10:07:34 -04:00
xarmian	b73ba63752	feat(store): hard-purge soft-deleted workspaces after 30 days (TASK-1966) (#825) ... The /privacy policy promises owned workspaces are removed from live systems within 30 days, but DeleteAccountAtomic and DeleteWorkspace only SOFT-delete (workspaces.deleted_at) and nothing ever expunged them — a right-to-erasure gap. Add a scheduled sweeper that hard-purges workspaces soft-deleted longer than a named 30-day retention constant. - Store: ListPurgeableWorkspaces (soft-deleted + past cutoff; never touches live rows), WorkspaceAttachmentBlobs, CountAttachmentsForHash- OutsideWorkspace (content-addressed dedupe guard), and PurgeWorkspace- Data — a transactional cascade that deletes every workspace-scoped child row in FK-dependency order (items/comments/versions/links/ reactions/stars/yjs op-log/wiki-links/grants/transitions/moves/views/ collections/documents+versions/agent_roles/webhooks/invitations/ templates/share_links+views/oauth join rows/report layouts/members/ member access/api tokens/attachments/activities), de-identifies mcp_audit_log, and refuses to touch a non-soft-deleted workspace. - Server: a periodic sweeper modeled on the orphan GC — captures blob keys before the purge, cascades the DB rows, then reclaims blobs through the attachment store abstraction (FS + S3 safe) with the orphan GC's cross-workspace dedupe + in-flight-upload guards. Failure isolated per workspace; idempotent. - Dual-dialect (SQLite + Postgres); partial index on workspaces(deleted_at) — migrations/073 + pgmigrations/051. Both delete paths (account + manual workspace delete) purge on the same 30-day clock: identical deleted_at mechanism, both owner-initiated, and the orphan GC already reclaims their attachment blobs at 30 days. Claude-Session: https://claude.ai/code/session_01HxBkAMiFBtCRJ2tKSCt3ST	2026-07-05 22:23:12 -04:00
xarmian	f34e66254b	feat(admin): admin force-verify email override (TASK-1939) (#809) ... Wave 4 of PLAN-1933 (DR-7). Adds a web-console-only admin override to force-verify a locked-out unverified account. No CLI, no MCP (matches the no-auth-mutation-on-MCP rule). Server: - POST /api/v1/admin/users/{userID}/verify-email — admin-only, mirrors handleAdminEnableUser. Reuses the existing SetUserEmailVerified store method (added in Wave 3b) and audits with the distinct ActionEmailVerifiedByAdmin action (separate from the self-serve ActionEmailVerified — a force-verify is an operator security action). Idempotent (already-verified returns 200 no-op). - Surface email_verified_at in the admin list + get-user JSON so the console knows verified state (Wave 1 only added the store-level scan). Web: - adminVerifyEmail client method (api.admin.verifyEmail) confined to the admin section of client.ts. - "Mark email verified" action in the admin user panel (UserSettingsForm), shown only when the target user is unverified. - email_verified_at added to the AdminUser type. Tests: admin force-verifies an unverified user (flips email_verified_at + audits ActionEmailVerifiedByAdmin, not the self-serve action) and the now-verified session is unblocked; non-admin -> 403 with no side-effect. Claude-Session: https://claude.ai/code/session_01HxBkAMiFBtCRJ2tKSCt3ST	2026-07-04 04:15:09 -04:00
xarmian	4a7c054223	feat(server): cloud email self-registration + verify-email/resend endpoints (TASK-1938) (#808) ... Wave 3b of PLAN-1933 turns ON Pad Cloud email/password self-registration with mandatory email verification. - DR-6: relax handleRegister to allow self-serve signup when cloudMode && emailConfigured. emailConfigured = s.email != nil AND a USABLE public base URL (non-empty, not a 0.0.0.0/:: bind-all host), so no unverifiable user is ever created. Self-serve is the ONLY path that writes email_verified_at = NULL (UserCreate.Unverified); admin-created and invited signups stay verified. Mints + sends a verification email. - DR-5: POST /auth/verify-email (ConsumeEmailVerification → flips email_verified_at → returns fresh user) and POST /auth/resend-verification (always-200, enumeration-safe; minting a new token invalidates the prior one). Both wired into the rate-limiter path switch (PasswordReset bucket). - DR-1: handleAcceptInvitation verifies an unverified account on accept (email-bound invite proves email control), via new store method SetUserEmailVerified. - DR-11: keep the existing clear 409 on duplicate email at signup. Session freshness: currentUser is re-read fresh from the DB per request (ValidateSession → GetUser), so flipping email_verified_at unblocks the same session's subsequent mutations immediately under RequireVerifiedEmail (Wave 3a) — no session-row rewrite needed. Test covers verify → same-session mutation succeeds. Claude-Session: https://claude.ai/code/session_01HxBkAMiFBtCRJ2tKSCt3ST	2026-07-04 03:32:48 -04:00
xarmian	31053086ee	feat(server): RequireVerifiedEmail enforcement across all mutation perimeters (TASK-1937) (#807) ... PLAN-1933 DR-4 (Wave 3a). Enforce, on Pad Cloud only, that an authenticated-but-unverified user cannot mutate content or mint credentials. Cloud-only + unauthenticated + verified are all no-ops, so this is inert in production until Wave 3b starts creating unverified users; the tests drive the state directly via the store's explicit UserCreate.Unverified control. Core rule: block only when cloudMode && currentUser != nil && !IsEmailVerified(), on mutating methods (POST/PATCH/PUT/DELETE). Returns 403 email_not_verified. The middleware does NOT inherit CSRFProtect's / RequireAuth's blanket /api/v1/auth/* exemption — it decides for itself. Perimeters gated (systematic DR-4 audit — one test each): - /api/v1 core writes (session AND PAT) — RequireVerifiedEmail method gate mounted after RequireAuth (server.go). - Authenticated /auth/* mutations — token create/rotate/delete, PATCH /me, 2FA setup/disable, OAuth link/unlink, and cli-session approve — all fall through the method gate (no auth exemption); logout, verify-email, resend-verification, delete-account allowlisted. - Collab WS GET-upgrade — authorizeCollabAccess (a GET the method gate can't catch; it persists Yjs edits). - Remote MCP write path — dispatcher RequireVerifiedEmail hook fired in buildAuthedRequest (the single chokepoint every synthesized write passes through), wired in cmd/pad/main.go. - OAuth-provider authorize + authorize/decide — emailUnverifiedBlocked checks (mounted outside /api/v1; decide mints the auth code). - POST /api/v1/import/url — SSRF/abuse surface, method gate. Carve-outs: POST /api/v1/invitations/{code}/accept stays open for unverified invitees (DR-1); legacy no-user workspace PATs are intentionally ungated (currentUser==nil). Self-host (!cloudMode) is a full no-op. Claude-Session: https://claude.ai/code/session_01HxBkAMiFBtCRJ2tKSCt3ST	2026-07-04 02:06:58 -04:00
xarmian	b0eeef16ce	feat(store): email_verification_tokens + SendEmailVerification + token reaper (TASK-1936) (#806) ... Wave 2 of PLAN-1933 — verification-token infrastructure (pure infra; no endpoint consumes it until Wave 3). - Migration 071 (SQLite) / 049 (Postgres): email_verification_tokens table, cloning the password_resets shape (id/user_id FK/token_hash/expires_at/ used_at/created_at + token_hash + user_id indexes), per-dialect created_at default. - Store email_verification.go: 256-bit crypto/rand token, padver_ prefix, SHA-256-at-rest, non-destructive Lookup, atomic UPDATE...RETURNING Consume. Deltas from password_resets (DR-2): 24h TTL, keep invalidate-prior-on-mint (resend burns the old link), consume side-effect sets users.email_verified_at (RFC3339-with-Z, same format Wave 1's migration used) in one transaction — no password reset, no session mint. - Email SendEmailVerification: clones SendPasswordReset, "1 hour" -> "24 hours". - Token reaper (DR-5): lifecycle-safe background sweep (mirrors orphanGC/opLogGC — self-registers on Server.bg, context-cancellable via stop channel, started only from cmd/pad/main.go so unit tests don't leak goroutines) calling the four previously-unwired CleanExpired* methods (email verifications, password resets, sessions, CLI auth sessions) hourly. Adds CleanExpiredEmailVerifications. - Audit consts ActionEmailVerified + ActionEmailVerifiedByAdmin. Gates: make check + make test-pg green (store + migration on both dialects). Claude-Session: https://claude.ai/code/session_01HxBkAMiFBtCRJ2tKSCt3ST	2026-07-04 01:19:41 -04:00