mirror of
https://github.com/PerpetualSoftware/pad.git
synced 2026-09-10 15:05:40 +00:00
main
911 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 xarmian
|
dc70ff3d7f |
fix(server): write first, apply second on the collab applier path (TASK-2989 / BUG-2840 half A) (#1318)
* test(server): measure BUG-2840 half A's premise before designing a fix Half A's plan makes step one an experiment, not a design: the claim that a refused PATCH still lands its content was a reading of the snapshot branch rather than an observation, and the shape of the fix depends on which half actually bites. Measured, on the applier path with a live room: a PATCH carrying content and a stale expected_updated_at answers 409, leaves items.content untouched, and adds an op-log row that outlives the request. The caller's refusal is true of the row and false of the collaborative document. The first version of this test was CIRCULAR and reported the premise confirmed. It drove a ?source=collab-snapshot PATCH carrying the refused string, which proves only that a snapshot write writes what it is given. The server cannot close that loop at all: collab here is a dumb relay that persists opaque Yjs updates and never parses them, so nothing server-side derives markdown from a room's document — in production that markdown comes from a live tab's Y.Doc. What IS observable server-side is durable collab state created by a request that was refused, which is what this now measures. Two details that make the harness faithful rather than convenient: - The fake applier emits a binary op as well as the ack. A real applier is a browser tab that applies the markdown and broadcasts the resulting update; acking alone would leave no durable trace, so the experiment would have been measuring a peer that does not exist. - Readiness is detected by the observable difference between the two paths — a succeeding probe PATCH that leaves items.content untouched proves the applier answered — because no exported accessor for electable connections exists and the manager's state is not reachable from this package. The test asserts today's behaviour, defect included, so the fix has a baseline to move. It skips with an explicit "premise NOT established" message if the harness ever stops reproducing the applier writing durable state, rather than passing vacuously. Refs: BUG-2840 * feat(server): write first, apply second on the collab applier path (TASK-2989 / BUG-2840 half A) PLAN-2975 decisions 2-4. A refused PATCH no longer changes the item. The applier path used to push content into the live Y.Doc before the row write, so any of the four typed refusals answered 4xx while the collaborative document had already moved and the next collab-snapshot flush carried the refused content into items.content. The reorder is possible because TASK-2987's HasElectableApplier answers which path the request is on without taking it. routeContentUpdate owns the re-decision deliberately: the predecessor retried ErrRoomActiveDuringPrune inside applyContentViaCollab and re-called ApplyExternalContent, which could succeed through a freshly joined applier and return nil, after which the row write still ran last and reproduced the defect. Re-deciding before anything is written makes that impossible rather than unlikely. Two typed 409s join the structured family. content_not_applied answers the hybrid the reorder creates - row write committed, content not in the document - naming the landed fields and the new updated_at so a content-only retry does not trip OCC. room_settling answers the standoff where PruneAndApply blocks on any writer while election also demands unfrozen and replay-done: the predecessor gave up after three tries and wrote past the live peer, losing the write on its next flush. applier_ambiguous is untouched; its outcome is unknown and a claim either way would be false. The measurement harness is inverted rather than deleted: it asserted the defect and would have become a SKIP, which reads as a pass. Claude-Session: https://claude.ai/code/session_01GqaEDuCtRiSJfa7eppWecn * refactor(server): retire the route-flipping helper chain the reorder replaced (TASK-2989) applyContentViaCollab, applyContentViaCollabOnce, directWriteFn, applyContentMaxRetries and isDeterministicWriteFailure are dead once the router owns the decision, and golangci-lint said so. Removing them is the point rather than tidying: that chain retried ErrRoomActiveDuringPrune internally and re-called ApplyExternalContent, which could succeed through a freshly joined applier and let the row write run last after all. Two things ported rather than dropped. isDeterministicWriteFailure's closed-set warning moves onto writeTypedItemRefusal, which inherits the job of recognising every typed permanent refusal. Its regression test is ported too, unchanged in property: a refusal the handler does not recognise is treated as recoverable and the request re-derives it by another route, which BUG-2804 measured as a rename cascade run twice. CONVE-23 sweep: my own comment on HasElectableApplier, merged four hours ago, said the fallback could write content past live peers. This unit made that false. It now states what the sentence was true of and what replaced it, rather than being quietly deleted. The structural guard needed teaching, not weakening: it counts the handler's refusal blocks and failed closed when one moved into a shared function. It now scans both files and says why three is still three. Claude-Session: https://claude.ai/code/session_01GqaEDuCtRiSJfa7eppWecn * test(server): pin the settle budget itself, which every other test bypassed (TASK-2989) Found by mutation: applierSettleBudget = 0 survived the whole suite. The decision tests pass their own budget, so the constant had no coverage at all — and a zero budget makes the retryable refusal the normal answer for any room with a writer still anchoring. The floor is the measurement the constant was sized from rather than a number: 47ms, just above the 46.41ms worst anchoring time measured for this deployment. Claude-Session: https://claude.ai/code/session_01GqaEDuCtRiSJfa7eppWecn * test(server): bound the standoff subtest so a broken deadline fails instead of hanging (TASK-2989) The only exit from the standoff branch is the deadline, so the mutant that makes it unreachable spins and the failure arrives as a package timeout with no --- FAIL line — which a mutation harness reads as 'the package broke' rather than as a detection. Measured: that is exactly what M5 produced. Same shape as the waiter rule: a failure mode indistinguishable from the waiting mode is not a signal. Claude-Session: https://claude.ai/code/session_01GqaEDuCtRiSJfa7eppWecn * fix(server): content_not_applied must not assert a timeout did not land (TASK-2989) Codex round 1, and the sharpest finding in it. ApplyExternalContent returns ErrAllAppliersTimedOut only AFTER an applier_request has gone out on the wire, so the elected peer may have applied the markdown and persisted its ops while the ack was lost or merely late. Answering content_landed:false there states as fact something the server cannot know — the same overclaim the ruling avoided by leaving applier_ambiguous alone, arriving one door over. The discriminator already existed upstream and needed no new machinery: electAndApply returns ErrNoApplierAvailable when anyWriteSucceeded is false (nothing reached a peer) and ErrAllAppliersTimedOut when something did. The envelope now carries content_outcome, and content_landed is ABSENT rather than false when the outcome is unknown, because a caller that reads false may act on a premise nothing supports. Three smaller round-1 items. The settle budget's comment now says it bounds how long the route keeps ASKING, not how long the request takes — the deadline is only consulted between attempts and PruneAndApply can block on the per-item lock. A comment on fullWriteHandled still named applyContentViaCollab, which this unit deleted; my own sweep missed it. The ported classifier test now inspects the recorder rather than only the boolean, since a mutant could return true while writing the wrong status. Verified and NOT changed: nil-ing content on the row write does not newly suppress version bracketing. main already set input.Content = nil on the applier path before its row write, so that behaviour is identical before and after the reorder. Claude-Session: https://claude.ai/code/session_01GqaEDuCtRiSJfa7eppWecn * fix(collab,server): the not-applied claim was still false on two post-wire paths (TASK-2989) Codex round 2, and it refuted the reasoning I gave in round 1's fix rather than just finding another case. I said the discriminator already existed upstream because electAndApply tracks anyWriteSucceeded. It does — PER ELECTION — and two paths escape it: - a restore storm returns ErrNoApplierAvailable after up to applierMaxRestartsAfterRestore elections, each of which may have put an applier_request on the wire, with the per-election flag discarded at every restart; - a registerPendingAck failure on a retry attempt returns a raw error after an earlier attempt had already sent one. Both would have answered content_landed:false about content that may have landed. Same shape as the finding they follow: a reason that was sufficient-sounding and one file short of true. Fixed at the source where the source can know it — ApplyExternalContent now carries sentAny across restarts, so ErrNoApplierAvailable means what its callers read it to mean — and by construction everywhere else: classifyApplyOutcome is a whitelist, so only the two sentinels that mean nothing reached a peer may make the claim and every other error, including ones nobody has written yet, degrades to unknown. Cancellation: the re-decision wait is the only new blocking wait this branch adds, and it now ends when the caller goes away. The rest of the path was context-blind on main and stays that way; threading a context into the store and the applier round-trip is a different change. Not fixed here, deliberately: the ambiguous-commit double-write. Codex confirmed against main that it has the identical shape there, so it is pre-existing and gets filed rather than folded into this unit. Claude-Session: https://claude.ai/code/session_01GqaEDuCtRiSJfa7eppWecn * docs(collab): sweep the prose my own round-2 fix falsified (TASK-2989) Codex round 3, one P3, and it is CONVE-23 arriving for the third time in this unit. Carrying sentAny across the restart loop changed which sentinel a restore storm returns, and left two comments describing the old behaviour: the cap's doc still said exhaustion falls back with ErrNoApplierAvailable, and the sentinel's own doc still said every attempt timed out. Both now say what the sentinel MEANS rather than how it usually arises — bytes reached a peer and the outcome is unknown — because that is the half two callers depend on: the op-log prune stays suppressed, and the PATCH handler reports the content outcome as unknown rather than not-applied. Claude-Session: https://claude.ai/code/session_01GqaEDuCtRiSJfa7eppWecn * fix(server): restore the UNIQUE-constraint 409 the applier path used to inherit (TASK-2989) Codex round 5, and a regression rather than a gap. The ordinary error block maps a UNIQUE-constraint race — two updates that both pass checkUniqueFields and then both hit the partial unique index on invocation_slug — to a 409. Before the reorder the applier path's row write ran through that block and inherited the mapping. Routing it through a helper built from 'the four typed refusals' dropped the arm and turned a benign race into a 500 on that route alone. The irony is the lesson, and it belongs on the record: writeTypedItemRefusal exists BECAUSE this handler's refusal set has been under-counted three times, and I under-counted it again while building the thing meant to stop that — by taking the population from the errors that have a Go type rather than from the block that actually answers them. The new arm's first version panicked on a nil error, since it dereferences where the typed arms use errors.As. The existing nil control leg caught it immediately, which is the entire reason that leg is there. The structural guard now DERIVES its file set — every non-test file in the package that calls UpdateItemWithParentLink — instead of listing two names, so a future block in a third file cannot sit unmapped while the test passes. Claude-Session: https://claude.ai/code/session_01GqaEDuCtRiSJfa7eppWecn * test(server): the guard now requires the fifth arm, scoped to the block's own function (TASK-2989) Codex round 6. Two gaps in the guard as it stood: it verified only the four typed arms, so removing the UNIQUE-constraint mapping from either ordinary block still passed; and its file set matched on the store call text, so a file reaching the store through a wrapper would not be scanned at all. The file set is now the UNION of files calling UpdateItemWithParentLink and files calling any of the arms — a refusal block lives where the arms are called, whatever it calls the store through. The fifth-arm check is scoped to the ENCLOSING FUNCTION, and that is the part worth reading. The first version asked whether a UNIQUE literal appeared between one block's start and the next block's start in token.Pos. Those windows span whole files, so the gap between the last block of one file and the first block of the next swallowed every literal in between — two in handlers_items.go belonging to the create and restore paths, one in handlers_items_bulk.go. All three mutation controls survived it. It asserted nothing, and it passed, which is the only reason I looked. Committed BEFORE the controls run this time. The previous round's controls used git checkout -- internal/ against uncommitted guard work and deleted it; the tree read clean afterwards, which is the ambiguity — clean means the mutation was reverted OR the mutation and my work both were. Claude-Session: https://claude.ai/code/session_01GqaEDuCtRiSJfa7eppWecn * test(server): scope the fifth-arm check to the block statement, not the function (TASK-2989) Per-function was the second wrong containment and the controls said so: handleUpdateItem holds TWO refusal blocks with a UNIQUE arm each, so neutralising either hid behind the other and survived. Only the writeTypedItemRefusal control was detected — the check covered one of the three blocks it claimed to cover. Innermost enclosing BlockStmt is the containment that matches what the sentence means by 'the block's own arm'. Claude-Session: https://claude.ai/code/session_01GqaEDuCtRiSJfa7eppWecn * test(server): the fifth-arm check reads if-conditions, not any literal in the block (TASK-2989) Codex round 7. Scanning the whole BlockStmt for a matching string literal let an unrelated nested closure — or a message string quoting the phrase — satisfy the guard after the real mapping had been deleted. That is the guard passing for a reason unrelated to what it asserts, which is the failure this whole check exists to prevent one level down. It now reads only IfStmt conditions, which is the shape the arm actually has. Claude-Session: https://claude.ai/code/session_01GqaEDuCtRiSJfa7eppWecn |
||
|
xarmian
|
4a4a912fa6 |
docs: pad token create needs a session, and one code comment said otherwise (TASK-2984) (#1316)
Since #1267 the server refuses a mint authenticated by an API token with HTTP 403 `session_required` — "Creating or rotating API tokens requires an interactive session, not an API token" — because the tokens such a mint produces outlive the revocation of the token that made them: each has its own name and expiry, and nothing in `pad token list` records which token minted which. `list` and `revoke` stay reachable by a PAT deliberately, since revocation is the response to a compromised credential and should not need a fresh login. Verified against `handlers_tokens.go::requireInteractiveSession` before writing it, as the item asked: the code and the message are quoted from there, and the guard is `isAPITokenAuth`, which is false for a session cookie AND for a saved `padsess_` CLI bearer — a CLI session IS an interactive session. Three artifacts a reader consumes carried the now-false implication; two are fixed here. - The README's PAD_TOKEN section said the override authenticates "without `pad auth login`" and named `pad token create` as where tokens come from, which together read as "you can mint under the override". It now names the one exception and links to the paragraph. - The token-management section gets that paragraph: which subcommands need a session, the exact code and message, why, and that `pad auth login -i` is the headless path. - `internal/cli/client_tokens.go` claimed the PAD_TOKEN override was "usable end-to-end without a browser". True when written and false after the gate. Corrected in place with the reason and a pointer to the guard, rather than deleted — a comment that was true once is worth more as a dated correction than as a gap. THREE CLAIMS I HAD WRONG, all caught before merge and all by checking rather than by rereading: - "needs a terminal but not a browser" — `pad auth login` DEFAULTS to browser-based auth; `-i` is the email/password prompt. The README and the comment name the flag now. - "fails when `PAD_TOKEN` is set" — too broad. `PAD_TOKEN` accepts a `padsess_` session token as well as a `pad_` API token (`env_token.go:21-23`), and the session form mints normally. The gate is on the CREDENTIAL KIND, not on the variable. - The history was compressed into "#879, before #1267". #879 added the override and left minting web-only; #1237 ( |
||
|
xarmian
|
cece78a703 |
feat(collab): applier-availability pre-check on RoomManager (TASK-2987 / PLAN-2975 unit 1) (#1314)
* feat(collab): applier-availability pre-check on RoomManager (TASK-2987) PLAN-2975 decision 1. HasElectableApplier answers "would an external content update for this item route through a designated applier right now?", so handleUpdateItem can learn it is on the applier path BEFORE it writes the row — today the only way to learn that is to apply, and by then the content is already in the live Y.Doc (BUG-2840 half A). It is a hint, not a lock: it registers nothing, takes no admission, and enters no gate, so a concurrent ApplyExternalContent neither blocks it nor sees it. Eligibility is delegated to pickApplier rather than restated, so the hint and the elector cannot drift. The error direction is asymmetric on purpose. A false negative is the defect (it sends the caller back to apply-then-write); a false positive costs a typed partial answer, which PLAN-2975 decision 2 owes anyway. So an in-progress version restore answers TRUE rather than consulting the conns: ForceRefreshRoom freezes every conn and pickApplier skips frozen conns, yet a restore ROLLBACK unfreezes them and elects an applier — so a bare pickApplier check answers false for exactly the window this is required to compose with. Eight legs, including a negative control: the pickApplier-only formulation must answer false during a restore, or the table would be evidence about neither implementation. Claude-Session: https://claude.ai/code/session_01GqaEDuCtRiSJfa7eppWecn * test(collab): pin the m.closed guard, which Close makes unreachable through the public API (TASK-2987) The manager-closed leg passed with the guard deleted: Close sets m.closed and empties m.rooms in one m.mu critical section, so the room lookup already answers nil. Measured as a surviving mutant, not assumed. The new leg constructs closed-with-rooms-populated — a state Close does not produce — because that is the coupling the guard breaks. Claude-Session: https://claude.ai/code/session_01GqaEDuCtRiSJfa7eppWecn * docs(collab): narrow two claims in HasElectableApplier's contract that the code does not support (TASK-2987) Codex round 2, adversarial on the unit's own claims. 'A false positive costs a partial answer, not corruption' was a claim about the whole path stated as if it were about this return value. The fallback that exists today can write content past live peers: when applyContentViaCollab exhausts its ErrRoomActiveDuringPrune retries the handler falls through to a plain direct write while a live writer may hold a Y.Doc that outvotes it. That predates this function; the comment now says so rather than implying it cannot happen. 'The hint and the elector cannot drift' reads as 'cannot disagree'. They cannot disagree about the RULE, since the predicate is shared. They can disagree about the ANSWER, and the routes are now enumerated: a fresh writer joins, an unanchored conn finishes replay, a view-only conn is promoted by the periodic revalidation, a restore rolls back. Claude-Session: https://claude.ai/code/session_01GqaEDuCtRiSJfa7eppWecn |
||
|
xarmian
|
793fad959c |
docs(mcp): the reason given for the protocol restriction was false — replace it with the true one (TASK-2977) (#1312)
TASK-2977 step 1 restricted the remote transport to the handshake era and gave this reason: pad_set_workspace pins a session default workspace that the stateless era has nowhere to keep, so pad "is not known to be able to serve" that era. It is plausible and it is false for THIS transport, and I wrote it reasoning from the tool's purpose rather than from its remote behaviour. cmd/pad builds the cloud dispatcher with a SHARED workspace state whose ResolveDefault() returns "" by construction — BUG-1865, the cross-user workspace bleed — so the pin is recorded and never consulted here. Resolution on /mcp is the explicit workspace argument, else a default derived per request from the caller's own OAuth identity and token allow-list. Every input comes from the request. This transport has been stateless with respect to workspace resolution since that bug was fixed, and the fix for a cross-user bug turns out to be most of the work a stateless era would need. THE TRUE REASON IS BETTER AND WAS ONE FILE AWAY. The mcp-active-sessions gauge is keyed on the Mcp-Session-Id header, and the generate-only session-id manager at this transport's call site exists so that header is always minted and the gauge stays observable (PR #400 round 1). SEP-2567 REMOVES session IDs in 2026-07-28 — a server serving that revision never mints or echoes one — so in that era nothing pad mints is available to key on. Ruled day 62: that is an accepted cost, not a blocker, and it is recorded where the key is CHOSEN rather than only where the era is refused — middleware_mcp_session.go now carries the obligation on whoever opens that era to re-key the gauge first, and says why the era's arrival is exactly the moment a silently-flat gauge gets read as "no MCP traffic" instead of "no measurement". The superseded reason is kept in the comment as superseded, four lines of it, because the false reason is the PLAUSIBLE one: the next person to reason about the stateless era from pad_set_workspace's name will reach for it, and the comment now meets them with the shared-state mechanism instead. TWO CODEX ROUNDS, TWO FINDINGS, both about this change's own prose and both verified in the code before accepting: - The server-package comment named mcp.ServedProtocolVersions as a symbol. It is not reachable from there — internal/mcp imports internal/server and not the reverse, which is also why the transport reaches the router as a plain http.Handler. Named by path now, with the direction stated. - "Modern-era traffic would be invisible to the gauge" was OVERSTATED, which is this unit's own defect class arriving inside the fix for it. trackMCPSession resolves the id from the response header and FALLS BACK to the request header, so a modern-era client that volunteers an Mcp-Session-Id is still tracked. The accurate claim, now in both files: the gauge stops depending on anything pad mints and starts depending on whether clients keep sending a header the spec removed — under-counting by a margin nobody controls, rather than a flat zero. Comment-only; no behaviour changes. The restriction, its four tests and the derived version set are untouched. Claude-Session: https://claude.ai/code/session_01GqaEDuCtRiSJfa7eppWecn |
||
|
xarmian
|
d83632e258 |
feat(attachments): extension trust — the audio/video split, the text family, the CFB office trio and RTF (TASK-2976 / BUG-2963 PR B) (#1309)
* test(attachments): record the Void-beyond-window limit, and correct three comments that claimed more than their evidence (TASK-2976) The first commit of BUG-2963 PR B, before any extension-trust work: round 6's three items, deliberately kept out of PR A so its verified tip would not move, plus the nix terminator-comment correction folded in from BUG-2974's trail. 1. TestMatroskaDocTypeBeyondTheWindowIsWebM records a LIMITATION. A Void element is legal anywhere in an EBML header and may be any size; make one larger than the 512 bytes this door reads and the DocType behind it is not in the input at all, so the parse finds nothing and the stdlib's video/webm stands. Nothing got worse — the same file was video/webm before the DocType read existed, and video/webm's own allowlist entry permits inline serving — and no larger window fixes it, since Void may be larger still. The fixture is the ordinary FFmpeg Matroska with a 560-byte Void spliced into its header and the header size widened to match, the same construction as matroska-void-padded.head512; the complete file reads as matroska,webm under ffprobe, and what is committed is its first 512 bytes, so the DocType is absent by construction rather than by truncation accident. 2. TestTarWinsAPrefixCollision no longer calls the collision "asymmetric". That was the round-4 premise round 5 refuted with flac-ustar-in-comment.head512 — a FLAC's Vorbis COMMENT tags are arbitrary UTF-8, so real audio carries "ustar" at offset 257 as readily as a real tar carries an audio marker at offset zero. The implementation and the fixtures already said so; the stale word survived in the place a reader looks first. The comment now gives the real reason tar leads the default order, which is weaker: its magic sits at a fixed offset rather than at a prefix. 3. The seven-byte textual-AAC input's comment says which of the two things it is. It is a valid ADTS sync and layer signature that the stdlib reads as text — not a decodable AAC, and nothing in the test establishes that it is. What review established is the case it stands in for, and the comment now separates the two. 4. The nix loop-terminator comments named the wrong mechanism (BUG-2974, observed day 62). The heal push cannot loop because a push made with the default GITHUB_TOKEN creates no workflow runs at all — `gh run list --branch main` spans |
||
|
xarmian
|
a357b9f609 |
fix(mcp): restrict the remote transport to the protocol era pad can serve (TASK-2977) (#1310)
* fix(mcp): restrict the remote transport to the era pad can serve (TASK-2977)
mcp-go 1.0 implements the stateless protocol core from 2026-07-28 — no
handshake, no sessions, per-request identity in _meta — and its
Streamable HTTP transport advertises EVERY revision it implements by
default, serving both eras concurrently on one endpoint and deciding the
era per request. pad's construction site passed no version restriction,
so the library bump alone had main answering modern-era traffic through
server/discover while pad://_meta/version still published 2025-11-25 as
the maximum revision this server can negotiate.
NOT A PRODUCTION DEFECT, checked rather than assumed: app.getpad.dev and
mcp.getpad.dev both report commit
|
||
|
David Barkhausen
|
f262449b18 |
feat(cli): pad token create/list/revoke — CLI mint path for API tokens (#1237)
Contributed by b4rk13 (#879 follow-up). Reviewed under DECIS-212-style read: sits on the existing user-scoped /auth/tokens endpoints, no server changes; create requires a login session per #1267 and answers 403 session_required under PAD_TOKEN, list/revoke stay PAT-reachable. Claude-Session: https://claude.ai/code/session_01W71Y4K5hGbbqqAhbVFnjB4 |
||
|
dependabot[bot]
|
31d11e76ea |
chore(deps)(deps): bump github.com/mark3labs/mcp-go from 0.58.0 to 1.0.0 (#1275)
* chore(deps)(deps): bump github.com/mark3labs/mcp-go from 0.58.0 to 1.0.0 Bumps [github.com/mark3labs/mcp-go](https://github.com/mark3labs/mcp-go) from 0.58.0 to 1.0.0. - [Release notes](https://github.com/mark3labs/mcp-go/releases) - [Commits](https://github.com/mark3labs/mcp-go/compare/v0.58.0...v1.0.0) --- updated-dependencies: - dependency-name: github.com/mark3labs/mcp-go dependency-version: 1.0.0 dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> * fix(mcp): pad owns the protocol revision it advertises, not the library (TASK-2972) mcp-go 1.0 moves LATEST_PROTOCOL_VERSION to 2026-07-28. `MetaPayload. MCPProtocolVersion` was sourced from that constant, on the reasoning — written in the comment — that doing so meant the value "never drifts from what NewMCPServer actually advertises in the handshake". 1.0 falsified that, and in the direction the comment was guarding against. The handshake answers through `mcp.NegotiateLegacyVersion`, which returns at most LATEST_LEGACY_PROTOCOL_VERSION and CANNOT return the modern revision at all: measured, a client asking for 2026-07-28 is told 2025-11-25, and a client that sends nothing is told 2025-03-26. So the bump would have left the handshake where it was and moved the meta document alone — publishing a claim to negotiate a revision this server cannot negotiate. The advertised revision is now a pad-owned literal. Moving it means reading the new revision's delta against this server's surface first; a library bump must not move it on its own. The test that should have caught this was a tautology: it compared the payload against the same constant the payload was built from, so it could not fail, and it would have passed through this bump. Replaced with two assertions that each catch what the other cannot — against the LITERAL, so moving pad's claim is a deliberate edit visible in a diff, and against what the library's handshake ACTUALLY answers, which is the property the old comment claimed and never had. Both legs verified to fail when the constant is moved. Refs: TASK-2972 --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: xarmian <xarmian@gmail.com> |
||
|
xarmian
|
cbfc073ef1 |
fix(attachments): make allowlisted formats reachable by recognising their magic (BUG-2963 PR A) (#1304)
* fix(attachments): types the allowlist names are reachable from their own bytes (BUG-2963 PR A) BUG-2961 fixed one member of a class. The class, measured over 44 real files covering 41 extensions: 25 of 48 upload-allowlist entries could never be the type an upload was stored under, so the door refused — or silently retyped — files every surface advertises as supported. This is the sniff-side half. Nothing here trusts a filename to introduce a type; the trust decisions (the text family, the audio/video category split, the CFB office trio) are a separate change. - F1: two spelling aliases, the shape audio/wave and application/x-gzip already have. video/avi is a pure spelling difference. application/ogg is not: Ogg is a container and the allowlist has no video/ogg, so aliasing types an Ogg video as audio. Ruled the better of two answers, since the alternative is refusing every Ogg, and the reason is on the alias line. - F2: magic-byte pre-checks for tar (ustar at offset 257, which is why no prefix matcher finds it), bzip2, 7z and FLAC. Consulted ONLY where the stdlib returned application/octet-stream, so they can add a detection and never replace one. Raw AAC is resolved in ValidateUpload instead: twelve bits of sync is too weak to act on alone, so it is gated on the .aac extension as well — the bytes must still carry the sync, the extension only decides whether a weak signature may speak. - F3: the one that was accepted rather than refused, and so survived the first pass. The mimesniff table maps the bare EBML magic to video/webm with no DocType check, so a Matroska file uploaded fine and was stored as WebM. A DocType read separates them; an EBML header carrying neither string keeps the stdlib verdict, so the fallback is the behaviour that shipped before. - F6: application/javascript, text/yaml and application/xml leave the allowlist — no extension reaches those spellings and no sniff emits them. That deletion has a trap: extMIMEMap's values are looked up in `allowed`, and an extension naming a removed type is the mechanism that refuses .svg and .exe, so .xml now names text/xml. A test covers the .xml upload, not just the three lookups. audio/webm is equally unreachable and STAYS by ruling, with the comment a future tidy-up will read. Fixtures are real encoder output truncated to the 512 bytes the door reads, with provenance and one recorded gap in the testdata README. The measurement harness ships in neither PR. Refs: BUG-2963, BUG-2961 * docs(attachments): bring prose that F6 falsified back to true (BUG-2963) Three client comments and one store filter described the server's category for `application/javascript`, which stopped having one when F6 removed it from the allowlist. Nothing in the previous commit's diff points at these lines, which is the whole reason the sweep exists (team CONVE-23). Behaviour is unchanged in all four. The web allowlists still exclude `application/javascript` deliberately — the string can reach the client from somewhere that is not our upload door — and the store's category filter still matches it, because it buckets rows that EXIST rather than deciding what may be created: a filter that stops matching a type costs a row nobody can find, while one that matches a type no row carries costs nothing. Refs: BUG-2963 * fix(attachments): admit a format by its own integrity check, not by its magic (BUG-2963) Round 1 of adversarial review walked straight through the first version of these signatures. A prefix match is not a format, and on a default-deny door the difference is the whole point: - a real, EXECUTING ELF binary with "ustar" in unused padding at offset 257 was stored as application/x-tar - the six-byte 7z signature, alone, was a 7z archive - "fLaC\x00" was a FLAC stream; "BZh9\x00" was a bzip2 stream - FF F1 00 — three bytes — was an AAC frame - a WebM carrying "matroska" inside a legal Void element was stored as Matroska, and a Matroska with 40 bytes of Void padding was stored as WebM, leaving the mistyping this change exists to fix live for any padded file - a real VP8-in-Ogg video was accepted as audio/ogg, category audio, INLINE Every one was refused before the change and accepted after it. So each format is now admitted only by the integrity check the format itself defines: tar's header checksum, bzip2's block magic, 7z's start-header CRC, FLAC's mandatory 34-byte STREAMINFO, ADTS's reserved sampling-frequency index and layer bits. The EBML DocType is PARSED — a real element walk that skips Void — rather than searched for as a string in a fixed window. The Ogg alias is gated on the first packet naming an audio codec (Vorbis, Opus, FLAC, Speex); Theora and VP8 in Ogg stay refused exactly as before, and video/ogg is deliberately not added, because admitting a format is a review and not a side effect. Three comments asserted things that were false and are corrected rather than softened: that these formats always sniff as octet-stream (a tar whose first member is BM.txt sniffs image/bmp), that a .aac without a sync is refused (it is not, if its bytes are another audio type), and that deleting one switch case would fail one named test (this project's own mutation matrix already showed otherwise, and the comment was written against that data). Tests gained the cases that decide it: every near-miss body above as an exact upload, the four EBML directions including both adversarial files, the ADTS guards isolated so exactly one mutation kills each, and length boundaries at the offsets each check indexes past — the previous negatives padded everything to 512 bytes, which hid every length guard. Refs: BUG-2963 * test(attachments): kill two mutants the round-1 tests left alive (BUG-2963) Both survived the expanded matrix, and both for the same reason round 1 kept finding: a negative case that fails for a reason other than the guard it names. - The bzip2 near-miss was five bytes, so the LENGTH guard refused it and the block-magic check was never reached; removing that check left the suite green. The new sibling is long enough to reach it and NUL-padded so the stdlib still says octet-stream — without the padding the body sniffs as text and the magic table is never consulted at all, which the first attempt at this test demonstrated by passing for the wrong reason. - The ADTS octet-stream gate had no discriminating input: the two-byte case fails validADTSHeader on length before the gate matters. FF F1 40 41 41 41 41 is a structurally valid ADTS header — sync, layer 00, sampling index 0, frame length 2570 — whose every byte the stdlib reads as text, so it answers text/plain. That is the only shape that separates the gate from the structural check, and with the gate removed the file is stored as audio/aac. Refs: BUG-2963 * test(attachments): fuzz the sniff path, since this change added a parser (BUG-2963) Every other check in this package reads fixed offsets and is bounded by construction. sniffEBMLDocType walks caller-supplied length fields, which is the one shape here that can index out of range or fail to advance, so the question 'does it survive malformed input' deserved an answer from running it rather than from reading it. Asserts the two properties a sniffer owes its caller — it returns, and it does not panic — across sniffOpaqueMagic, sniffEBMLDocType, sniffOggAudio, validADTSHeader, SniffMIME and ValidateUpload. What it returns for nonsense is left to the table tests. Seeds are the real fixtures, each truncated at fourteen offsets, plus the shapes a walk breaks on first: unknown-size elements (the reserved all-ones VINT), a child whose declared size exceeds the data, the invalid all-zero VINT marker, a zero-length child that must still advance, and a truncated Ogg page header. 5.56M executions, no panic and no hang. Also collapses a duplicate fixture loader this file had grown alongside mime_isobmff_test.go's readFixture. Refs: BUG-2963 * fix(attachments): use the real parsers, drop Ogg, and stop overclaiming (BUG-2963) Round 2 found 15 issues, two of them P1, and the important one is not any single check — it is that round 1's answer was wrong in the same WAY round 1's defect was. Round 1 defeated prefix matching with an ELF carrying "ustar" at offset 257. The response was a checksum. Round 2 defeated the checksum with an ELF carrying a CORRECT one. Measured while responding, and it settles the question: archive/tar's own Reader.Next accepts that file too. A 512-byte tar header is exactly those fields, and nothing forbids another format's padding from containing them. The two are not distinguishable at this size by anything in the standard library, so this is a property of the formats and not a defect to fix. Adding a third round of field checks would have been the same mistake a third time. So the change is to the CLAIMS as much as to the code. These checks RECOGNISE a format; they do not establish one. The safety property lives elsewhere and is now stated once, at the top of mime_magic.go: whatever the bytes are, they are stored opaquely, never executed or decompressed here, and served under a reviewed type with nosniff, as an attachment for every type recognised here. The checksum-correct ELF ships as a fixture and a test asserting it is ACCEPTED — a limitation recorded where it cannot be rediscovered as a bug. Where a real parser exists, it is now used: archive/tar and compress/bzip2, which cannot drift from the parsers a consumer would use and handle GNU/pax variants for free. bzip2 keeps its magic check alongside the decode because neither subsumes the other — the magic refuses a five-byte stream the decoder can only call truncated, the decode refuses an empty stream whose combined CRC is wrong. OGG IS REMOVED. The alias was ruled in, and review showed the question it has to answer — is this container audio — cannot be answered from the head of the file: an Ogg with Opus first and VP8 second passes a first-packet codec gate, because Ogg multiplexes and the video pages come later. The gate was also wrong in the other direction, refusing legitimate Skeleton-prefixed audio. Ogg is refused exactly as before this branch; making it work means adding video/ogg as a reviewed entry or demuxing, and neither belongs in a change whose premise is that it adds no new trust. The reasoning is a comment where the next person to reach for an alias will meet it. Other round-2 correctness fixes: a DocType value ends at its first NUL, so a real Matroska padded "matroska\x00junk" is no longer stored as WebM; reserved all-ones EBML IDs are refused rather than acting as zero-length children; FLAC checks the STREAMINFO BODY rather than its declaration; 7z checks that the start header's arithmetic is representable, not only that its CRC agrees; ADTS accounts for the two CRC bytes a protected frame declares. Tests gained the direction they lacked entirely — legal variants that must NOT be refused (an empty bzip2 stream, STREAMINFO carrying the last-block flag, a CRC-protected ADTS frame) — and the truncation loop now asserts refusal below each check's minimum instead of discarding its return value, which is why it had been passing while flac.head512[:8] was accepted. Refs: BUG-2963 * test(attachments): cover the five guards the rebuilt matrix found untested (BUG-2963) Rebuilding the mutation matrix against the round-2 code turned up seven survivors. Six were real gaps; each now has the exact input a review round used, or the smallest one that separates the guard from its neighbours, and two carry a control leg so the negative cannot pass for the wrong reason. - 7z: a start header whose next-header offset is 2^64-1, with a recomputed valid CRC. The CRC proves the bytes are intended, not that they are possible. - FLAC: STREAMINFO declaring a zero sample rate. - bzip2: the head of a 200KB archive, which must be RECOGNISED — truncation is what a 512-byte head of any real archive looks like, and the guard that distinguishes it from a structural error had nothing testing it. - EBML: a real Matroska whose DocType payload is 'matroska\x00junk'; and a header with a reserved all-ones ID before a valid DocType. The seventh is an EQUIVALENT mutant and is recorded as one rather than fixed: lowering validTarHeader's length guard from 512 to 262 changes no outcome, because archive/tar refuses a short block by itself. Measured, then written into the comment so the guard reads as clarifying rather than load-bearing. Refs: BUG-2963 * fix(attachments): delete the bzip2 magic comparison a mutation proved dead (BUG-2963) The check compared the block magic AND decoded, under a comment claiming each caught what the other could not. A repaired mutation run refuted it: with the comparison removed, every input the comment credited it with stopping is still refused — the short ones by the length guard, the rest by the decoder. So the line could not change an outcome while its comment said it did, which is the failure this package's own SafeFallbackExtension comment warns about. Deleted rather than demoted, and the length guard now says what it is for. The mutants that found this had been scoring BUILD-FAIL, which is nothing at all rather than a survivor — repairing them to compile is what surfaced three untested guards, two of them FLAC's STREAMINFO block-size floor and ordering. Refs: BUG-2963 * fix(attachments): recognise by magic, to the standard this door already uses (BUG-2963) Three review rounds walked the structural-validation path to its end and the ruling is to stop: - Round 1 defeated magic matching with a real, executing ELF carrying "ustar" at offset 257. - Round 2 defeated the checksum that answered round 1, with an ELF carrying a CORRECT one — and archive/tar's own Reader.Next accepts that file too. A 512-byte tar header is exactly those fields; the two are not distinguishable at this size by anything in the standard library. - Round 3 found the accumulated validation refusing REAL files: PAX and long-name GNU tars, legal randomized bzip2 blocks, FLAC declaring the zero sample rate RFC 9639 permits. That is this bug's own defect — refusing files people legitimately have — reintroduced by the fix for it. Validation could not narrow what the door accepts and had started refusing what it should take, so it is gone. Recognition is by defining magic: ustar at 257, BZh plus its digit, the 7z six bytes, fLaC, and for ADTS the syncword plus the layer bits, which keeps its .aac extension gate because fourteen bits is weaker than the rest. The fact that settles the width, read from the stdlib rather than assumed: http.DetectContentType recognises audio/mpeg from the three bytes "ID3" (net/http/sniff.go), audio/mpeg is on the allowlist, and this door already serves it inline. Every signature here is at least as wide, so this is the door's EXISTING standard rather than a relaxation of it. A test asserts that premise so it cannot rot. Two refusals are deliberate and say so in the code: V7 tar has no magic anywhere and cannot be recognised by this kind of check at all, and Ogg stays refused for the reason a previous commit records. bzip2 is no longer decompressed anywhere in the door. The safety paragraph is rewritten from what the code does, for the third time and the last: nothing is executed, nothing is decompressed, archives download while FLAC and AAC play inline exactly as every other allowlisted audio type does, and nosniff means recognition can move a file between reviewed types but never outside them. Tests now assert the contract that exists. The widening is asserted rather than described — every input the validation used to refuse is accepted, each with its SERVING BUCKET checked, because that is the property that makes it tolerable. The files validation used to refuse are asserted accepted. What remains of the negatives is the only thing still true: the magic has to be there, in the right place, in full. Refs: BUG-2963 * test(attachments): restore two EBML properties the bulk cut dropped (BUG-2963) Cutting the structural-validation tests wholesale took the DocType NUL-termination and reserved-ID cases with them, though the DocType walk they cover is untouched by the magic-only ruling. A mutation run is what noticed: both mutations had gone from detected to surviving. Worth recording as a shape rather than a slip — deleting a test file's worth of obsolete assertions is exactly when live coverage leaves with them, and the matrix is the only thing that says so. Refs: BUG-2963 * fix(attachments): tar wins a prefix collision, and the AAC gate stops refusing real files (BUG-2963) Round 4's two blocking findings were both REAL files of listed types turned away — the direction the ruling's convergence bar names first. **A tar whose first member is named fLaC.txt was refused.** Every recogniser except tar's is a prefix test, and a tar header's first 100 bytes are its member's FILENAME — arbitrary text a user chooses. So an ordinary archive carrying "fLaC" or "BZh9" in a name was recognised as that format and then refused for a category mismatch against its own .tar extension. Tar is tested first now, and the order is load-bearing rather than arbitrary: the collision is asymmetric. A real tar carrying a foreign prefix needs only a filename; a real FLAC carrying "ustar" needs those five bytes at exactly offset 257 in compressed data. Losing the first case costs ordinary uploads. **Real AAC files were refused when their leading bytes looked textual.** The gate ran on application/octet-stream alone, but a raw AAC frame whose ancillary payload is printable makes the first 512 bytes read as text, so the stdlib answers text/plain and a genuine, ffmpeg-decodable .aac was rejected for a category mismatch. Neither verdict is a format detection; both mean "nothing here identifies this", which is the condition under which a weak signature may speak. A type the stdlib DOES recognise is still untouched, and a test pins that with PNG bytes named .aac. Also, the leftovers that keep being mine: comments still describing structural validation that is gone, a test comment claiming production delegates to archive/tar when it no longer does, a README row saying the ELF fixture exists to be refused when it is now accepted, an unused fixture, and two fixtures the README never listed. Tests strengthened where a review round showed one example was standing in for a whole signature: the EBML legs now ask sniffEBMLDocType directly, because routed through SniffMIME the stdlib fallback supplied the same WebM answer and the explicit mapping could be deleted with the suite green; the ADTS signature is walked byte by byte; the bzip2 digit range is tested at both bounds and the FLAC marker at its width and case. Refs: BUG-2963 * test(attachments): give the ADTS verdict gate a control that actually controls (BUG-2963) The PNG leg could not establish the gate it was named for: PNG bytes fail validADTSHeader on the first byte, so they are refused with the gate removed too. A mutation run said so — dropping the stdlib-verdict gate survived the whole suite. The only shape that separates the verdict gate from the structural check is an input that PASSES validADTSHeader and is ALSO identified as something else: a buffer opening with a valid ADTS header and carrying 'ustar' at offset 257, named .aac. It is identified as a tar and must be refused for the category mismatch it is; without the gate the AAC branch overwrites that and accepts it. Recording the process failure alongside it, because it is one I have written down before: this test was lost once between writing and committing, because the mutation runner's restore is 'git checkout -- internal/attachments/' and the work was still uncommitted. Committing is step one of running a control, not step one of the unit. Refs: BUG-2963 * fix(attachments): let the extension arbitrate a magic collision, both ways (BUG-2963) Round 5 found the mirror image of round 4's finding, which is the useful part: my fix for round 4 created it. Round 4 showed a tar whose first member is named fLaC.txt being refused, so tar was ordered first. Round 5 then built complete, decodable FLAC and AAC files carrying "ustar" at offset 257 in ordinary metadata — a Vorbis COMMENT tag is arbitrary UTF-8 (RFC 9639 §8.6) — and those were refused instead. The premise I wrote into the ordering comment was false. I argued the collision was asymmetric, that real audio could not plausibly carry "ustar" at a fixed offset. It can, in a tag a user typed. Any total order refuses somebody. So there is no winner by order. sniffOpaqueCandidates returns EVERY matching type, and when the filename's extension names one of them it breaks the tie. That is a narrower thing than extension trust and the code says so: every candidate is a type the BYTES already matched, so the extension chooses among readings rather than casting a vote, and a name for a type whose magic is absent can never appear in the list. An extension naming none of them changes nothing — asserted, including that a .zip name does not make colliding bytes a zip. The AAC branch is fixed by the same finding from the other direction: it was gated on the REFINED sniff, so a real AAC with "ustar" in its payload — which this package refines to application/x-tar — was refused under its own .aac name. The gate now reads the standard library's verdict, which for those bytes is "nothing identifies this". Kept rather than dropped as unfalsifiable, because no mimesniff signature begins with 0xFF today and this is what stops a fourteen-bit match overriding one that does. Fixtures: the FLAC is real and decodes under libsndfile. The AAC counterpart is NOT shipped — overwriting a real frame's bytes produces a file ffmpeg rejects, so the test uses a synthetic buffer that reproduces the condition and says so rather than claiming to be audio. Also cleared, and reliably my own: comments still describing structural validation, a test's why-strings still citing checksums and CRCs, a comment claiming PAX and GNU archives are refused when they are recognised, and a README gap note that no longer covered all the fixtures. Two test premises tightened where they proved less than they said: the bzip2 prefix case also failed the digit check, and the three-byte ID3 claim was tested with seven. Refs: BUG-2963 * test(attachments): pin the default candidate order (BUG-2963) Reordering sniffOpaqueCandidates changed no test, because extension arbitration settles both known collisions whichever way the list runs. The order still decides one case — colliding bytes whose extension names neither candidate — and nothing asserted it, so the rationale in the comment was unenforced. Refs: BUG-2963 * docs(attachments): the fuzz comment still named the structural validators (BUG-2963) They were removed by the magic-only ruling. Comment-only. Refs: BUG-2963 |
||
|
xarmian
|
734b53f33c |
fix(attachments): a HEIC embed is decided by what can be served and painted, not by an image/ prefix (BUG-2964)
A pure-Go build derives no HEIC thumbnail, and the byte endpoint SILENTLY serves the original when the requested variant is missing — so an editor that chose <img> on a MIME PREFIX handed the browser HEIC bytes, which Chrome and Firefox render as the broken-image icon. The server now says what it did: X-Pad-Attachment-Variant names what the BYTES are (a fallback still reports `original`), and X-Pad-Attachment-Derived names which variants EXIST, answered only on the no-variant path so the hot image path pays nothing. `none` is a sentinel rather than an empty value, because ABSENCE has to keep meaning "server predates this fix". The rule, in both renderers: embed as <img> iff THE VARIANT THIS RENDER WILL REQUEST exists, OR the browser paints the original. Not a prefix (the bug); not availability alone (the same build derives no AVIF thumbnail, and browsers decode AVIF). The second disjunct is a new fourth predicate beside display.ts's three, NOT canOpenInViewer — that one excludes image/svg+xml for active-content reasons, and an SVG inside an <img> runs no script, so reusing it would have flipped every existing SVG embed to a chip. Verified on a REAL HEIF against a pure-Go build, with a PNG positive control on the same instance: HEIF reads `none` and answers ?variant=thumb-md with 200 + Content-Type image/heif — the defect itself; PNG reads `thumb-sm,thumb-md` and answers with the actual thumbnail. Three counterfactuals and a negative control on the new TS/Go lock-step test, which replaces a lock-step that had been asserted in a comment only. Codex CLEAN after 3 rounds; rounds 1 and 2 each found a real defect in the fix (a per-variant fact collapsed to a boolean, and an async fact cached as durable). The share-link 404 is deliberately unchanged — that path serves variants only because the variant pipeline is the privacy boundary, and serving an original to an anonymous viewer would trade a broken image for an EXIF/GPS leak. The limitation is documented per surface beside the capabilities endpoint. Claude-Session: https://claude.ai/code/session_01Xk9M5UVPdc84xL5E1mZkm8 |
||
|
xarmian
|
4618876e3e |
fix(cli): pad server stop signals only a process it can prove is ours (BUG-2969) (#1299)
fix(cli): `pad server stop` signals only a process it can prove is ours (BUG-2969) Measured on the merged binary before this change: a `sleep 600` whose pid had been written into the PID file was SIGTERMed, and stop printed "Server stopped." No pad server was running anywhere near that config. Three things had to be true at once for that. os.FindProcess succeeds for ANY pid on Unix. Nothing asked whether the pid belonged to a pad server. And the confirmation loop polled the PORT — which is unhealthy from the first poll when nothing was ever serving, so the success check was satisfied by the failure case. Liveness is the wrong question, and this is the trap the obvious fix falls into: the stranger WAS alive. The question is whether the pid is OUR server. ## The discriminator Unix takes an advisory flock on the PID file, held for the server's lifetime. `stop` probes it non-blockingly: acquiring it proves nobody holds the file, so the record is stale whatever the pid now names; failing to acquire proves a live pad server holds THIS file. One implementation for Linux and macOS, no new dependency, and the same primitive session_lock_unix.go has used since TASK-2767. Windows has no flock in that pattern, so it compares the process creation time from GetProcessTimes against the one recorded at start — the attribute that survives pid reuse, since a reused pid belongs to a process that started later. The lead first ruled start-time comparison on every platform; I objected with the cost (three implementations — /proc, a macOS sysctl promoting x/sys to a direct dependency, and GetProcessTimes) and the ruling changed to this hybrid. The cost table is on the item so the next reader sees why the shape moved. The PID file gains a fingerprint on both platforms — pid, start time, executable path — as JSON, with the legacy bare-integer form still parsed. A legacy record carries no proof, which reads as UNPROVABLE, and unprovable means nothing is signalled. ## Three races, each found by codex and each the same shape 1. Reading the record and checking ownership were separate steps, so a successor could claim the file between them: the lock then reported "held" — truthfully, about the successor — while the pid handed back was the predecessor's. pidFileOwner now returns the record it read from the descriptor it probed. 2. Removing the PID file after a successful stop could delete a fast successor's live record. It no longer removes at all there: the server removes its own on the way down, and a file left by a crash is handled by the next stop. 3. Removing a STALE file after the probe released the lock had the same window. The removal now happens inside the ownership check, while the lock is held — the only moment at which no replacement can have claimed the path. A claim arriving during that instant retries for half a second rather than losing its claim for the life of the process. Windows deliberately does NOT delete a stale file: with no atomic primitive, a check-then-remove would race a successor, and a stale file that the next start overwrites is recoverable where a wrongly deleted record is not. ## Verified Negative control, and it is the literal one: with the ownership check bypassed, `go test` reports `signal: terminated` — the test binary is SIGTERMed by the code under test, because the stale record names the test process itself. Live, in throwaway HOMEs: a stale record naming a live `sleep` is refused and the sleep survives (it was killed before this change); a stale record with a HEALTHY port answering is still refused, nothing signalled, and both the stranger and the real server survive; a server stopped through its own held record stops, and its file is gone. The CI smoke on windows-latest now stops the server with `pad server stop` instead of Stop-Process, because that is the only place the Windows ownership check runs — a smoke that killed the process directly would leave the GetProcessTimes path unexercised on every platform. make lint, make test green; codex CLEAN in round 4. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR |
||
|
xarmian
|
5ec17a7c92 |
fix(cli): pad server stop stops the server that is running, or says it is (BUG-2965) (#1298)
fix(cli): `pad server stop` stops the server that is running, or says it is (BUG-2965)
`StopServer` read the PID file and, on any read error, answered "server not
running (no PID file)" — without asking whether anything was listening. The file
was written in exactly one place, EnsureServer's auto-start branch, so a server
started any other way held the port with no file to find: a service unit, a
human running `pad server start`, the seats' refresh recipe relaunching with the
killed process's argv. A stop command that says "not running" about a running
process leaves the caller believing they stopped something, and the next thing
they do rests on that belief.
Two halves, per the item's property and corollary:
- A missing PID file now asks the port. Only an unhealthy address earns "not
running"; a healthy one earns a message naming the address, the missing
file, and what to do instead. Deliberately NOT "find the listener and kill
it" — resolving a pid from a port is platform-specific, and the process
holding it may not be ours. A stop that kills by port can kill a stranger.
- `pad server start` claims the PID file itself, so the file exists for every
start path rather than only the auto-started one.
The second half took four codex rounds to get right, and each round found the
previous shape reintroducing the defect it was fixing:
1. Write-then-defer-remove let a duplicate start overwrite a running server's
entry and then delete it on the way out, leaving a healthy server
unaddressable.
2. Refusing to replace a live pid fixed that and opened its mirror: the start
that LOST the port could still own the file, so the winner was unaddressable.
The fix is ordering, not arbitration — BIND FIRST, then claim, so the file
always names the process that owns the address. internal/server grows
Listen and Serve for that; ListenAndServe is now the two together.
3. With the bind first, EnsureServer's parent-side write became the stale
mechanism (it records a child that may never bind) and the live-pid refusal
became actively wrong (no live process can be serving an address we just
bound). Both removed, along with processIsAlive, whose only remaining
callers were its own tests.
4. Cleanup is a read-then-remove, so running it AFTER the listener closes let
a successor bind and claim between the two steps and lose its file to us.
It now runs before the listener closes, while nothing else can legitimately
own the file. The cost is a drain-window where a healthy server has no PID
file and `stop` says so — a true message in place of a silent wrong one.
Verified live against the built binary, in a throwaway HOME, in both shapes:
start writes the file naming the serving process; a second start against the
held port fails at bind and leaves the first server's file intact; stop then
stops it and removes the file; a further stop reports "not running". The first
live run also caught a flaw in my own method — `stop` reads the config's port,
so the probe answered about 127.0.0.1:7777 (this box's dev server) until it was
re-run with PAD_PORT set. Re-checked after the restructure.
Mutants: the health check removed, the health branch still answering "not
running", an empty PID file, a cleanup that does not remove, and a cleanup that
removes a successor's file are each killed by a named test. The call site itself
is wiring a unit test cannot vouch for (CONVE-19) — that is what the live runs
cover, and the Listen/Serve split is pinned in internal/server.
make lint, make test green.
Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR
|
||
|
xarmian
|
a2bab75c73 |
fix(server): the op-log prune rides inside the write it justifies (BUG-2840 half B) (#1295)
fix(server): the op-log prune rides inside the write it justifies (BUG-2840 half B) On the no-room / no-applier path a content PATCH pruned the item's Yjs op-log and then wrote items.content. The prune ran FIRST, in its own statement, on this justification: "any prior collab state is strictly older than the items.content the caller is about to write". That premise holds when the write LANDS. Four typed refusals can come out of that write — the open-children guard, the optimistic-concurrency conflict, the rename-cascade byte refusal and the item-title refusal — and on every one of them the caller wrote nothing, so the pruned ops were superseded by nothing. They were simply gone. It is not hypothetical on this branch. It fires on ErrNoApplierAvailable, i.e. a room inside its 60s grace TTL with zero connections: exactly the state where the op-log holds a closed tab's edits that never reached items.content. Those edits exist nowhere else, and a request that wrote nothing destroyed them. The prune now runs INSIDE the write's own transaction, composed onto the precheck hook that UpdateItemWithParentLink already runs there, so a refusal rolls it back. directWriteFn takes the prune as a hook rather than performing it, which keeps the choice of transaction with the caller that owns the write. This is the shape version-restore already uses: PruneItemOpLogTx's own comment says a split prune/commit "leaves a divergent state on any failure" in EITHER order, and closes that split by running the wipe in the update's transaction. This path was the remaining split, and it also closes the opposite window a plain reorder would have left — a crash between a successful write and a later prune, leaving stale ops to be replayed over fresh content. One deliberate behaviour change: a prune failure now rolls the content write back, where before it was logged and the write proceeded. That leniency assumed the prune was optional cleanup; it is not. A write that commits with a stale op-log is the "resurrect stale content on the next flush" hazard the prune exists to prevent, and it is the same trade the restore path made. Also removes internal/server's distantFuture, whose only use this was; the collab package keeps its own copy for its own prune. Grepped the repo before deleting rather than inferring deadness from the edit in front of me. Verified by a negative control — restoring the prune-first ordering fails the refusal test — and by three mutants: never invoking the hook and passing nil for it are both killed by the success test. A fourth, swapping the guard and the prune inside the hook, SURVIVES, and the comment now says so: both orders are equivalent while they share a transaction, so that ordering is a preference and not a rule anything enforces. make lint, make test and make test-pg green; PG run because this moves a DELETE into a transaction. Codex CLEAN. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR |
||
|
xarmian
|
696b477b12 |
test(server): the mobile shells' unversioned contracts fail CI when they move (TASK-2053) (#1294)
test(server): the mobile shells' unversioned contracts fail CI when they move (TASK-2053)
The native shells talk to this server over string contracts nothing protects:
route paths, cookie names, JSON keys, a header shape. None sits behind the MCP
tool-surface version or any other gate, and the min-server-version warning the
app shows covers the opposite direction — it protects a NEW app against an OLD
server and says nothing when the server moves forward under a shipped build. A
shipped build cannot be patched on our schedule, so a renamed key is a silent
break for every installed copy until its owner updates.
One file, one table, six contracts, each naming the shell surface that breaks:
- Both session cookie spellings. __Host-pad_session is the one that matters —
every deployment a phone talks to is TLS — and it appeared in NO Go test
assertion, only in a config-test comment. The insecure spelling is in
nineteen test files as a helper building a request, which is a mention and
not a claim about the name.
- The six auth routes, asserted by WALKING the route table rather than by
firing requests: a request-based check answers "did something handle this",
which a catch-all or a redirect satisfies while the route itself is gone.
- The /auth/session keys the app branches on before showing any UI, version
included. A rename there reads on a phone as a blank screen or a login form
on an instance that needs setup, not as an error.
- The full two-step 2FA sign-in. requires_2fa, challenge_token and
recovery_code were live request/response keys appearing in ZERO test files;
the test drives login → login-verify with a real recovery code, so a rename
on either side fails here.
- Content-Disposition carrying a filename on BOTH dispositions (BUG-2910).
The existing download tests assert the inline;/attachment; prefix and say
nothing about the parameter after it, so the filename could be dropped from
either branch with the suite green.
Scope is stated in the file rather than left for a reader to re-derive.
/auth/apple/native is a pad-cloud route; the app-scheme redirect allowlist does
not exist in this repo (greps for the scheme forms and for app_scheme/appScheme
return nothing in Go, TypeScript or Svelte); and the OAuth error codes this repo
emits belong to Dynamic Client Registration for MCP clients, not to mobile
sign-in — pinning them here would look like mobile coverage while protecting a
different client entirely.
Verified by an eight-mutant matrix: renaming either cookie name, the
login-verify route, requires_2fa, challenge_token or recovery_code, dropping
version from the session payload, or dropping the filename from
Content-Disposition each fails a named test. Every mutant is a change someone
could plausibly make in an afternoon, and every one of them was silent before.
Two of my own assumptions were caught by running it rather than by reading:
chi's router is built lazily, so walking it on a server that has served nothing
panicked; and the "downloaded document" case used a .txt, which is on the read
path's inline-safe allowlist and came back inline — the case would have tested
the same branch twice under a name claiming otherwise. It uses a real zip now.
Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR
|
||
|
xarmian
|
89a9fb0241 |
fix(server): a refused collection prefix reaches the caller as a 400 naming the rule (BUG-2951) (#1293)
fix(server): a refused collection prefix reaches the caller as a 400 naming the rule (BUG-2951)
BUG-2943 made the store refuse a prefix outside the grammar, with a message
naming the rule and an example. Three of the four doors onto that refusal threw
the message away: they mapped conflict shapes and sent everything else to
writeInternalError, so `pad collection update docs --prefix "ab1"` answered
"An internal error occurred". The refusal kept its data-protection value and
lost its entire teaching value — a 500 with no text reads as an outage, so the
honest user response is to retry or report one.
An MCP agent was told something worse than nothing. internal/mcp classifies a
stdio failure by matching CLI stderr prose; the generic message matches none of
the validation patterns, so a permanently-invalid prefix arrived as the
RETRYABLE server_error code and the correct agent response was to retry a call
that can never succeed. Nothing in internal/mcp changes here: the store's own
message now reaches stderr and the existing `invalid` pattern recognises it.
Both directions are pinned by tests, including the negative control that the
old generic string still classifies as server_error — which is right for a real
internal failure, and is why the refusal had to stop wearing that message.
The population, read door by door rather than grepped:
- CREATE and UPDATE lost the message entirely (500, no text).
- DELETE kept it via strings.Contains on the store's error text — the right
status by the wrong mechanism: a reworded refusal became a 500 silently.
- Workspace IMPORT kept the text under a 500, while its sibling bundle-import
door already answered 400 for the same class.
store.ValidationError carries the caller-facing Reason, with AsValidationError
for the doors, following the InvalidDocumentTitleError precedent in the same
package. Constructing it is the per-site DECISION that a message is safe to
show; the alternative — returning err.Error() from the generic path — makes
that decision by default for every error any layer may later add. Doors render
Reason, never Error(), because Error() carries the sentinel prefix and whatever
a call path wrapped around it; a test helper asserts no response leaks that
prefix, after a mutant swapping Reason for Error() survived every message
assertion (Reason is a substring of Error(), so a contains-check cannot see it).
Two sites are deliberately NOT converted, both read and left:
- The template-seeding trait validation (collections.go) checks FIRST-PARTY
template code, not caller input. A 500 is the honest answer there.
- The two expected_updated_at refusals are converted for uniformity but are
unreachable through HTTP — both doors validate the token at the boundary.
They are defence in depth, not live paths.
WIRE CHANGE: POST /workspaces/import now answers 400 for a caller-input refusal
where it answered 500. The code string (import_failed) and the message are
unchanged, and its sibling bundle-import door has always answered 400 for this
class, so this aligns two doors onto one refusal. Ruled by the lead rather than
decided here. "Cannot delete a default collection" stays 400.
Codex round 1 caught the consumer this change created: the tar.gz import door
renders a bundle failure through its own envelope and its fallback wraps
err.Error(), so the very edit that made the JSON door actionable moved the
sentinel prefix into the bundle door's message. It now detects the type and
renders Reason in its own "Bundle pad-export.json is not importable" envelope,
with a test and a mutant. A producer change is not finished until its consumers
have been read; round 2 was CLEAN.
Verified by negative control (each door arm removed in turn) and a ten-mutant
matrix in which every arm removal, every store site reverted to fmt.Errorf, and
both Reason→Error() swaps are killed by a named test. One matrix attribution
was wrong on first run — a store test appeared as a casualty of a server-side
mutant — and re-running it in isolation showed the mutant does not affect it;
the runner had attributed every FAIL line in a two-package run to the mutation.
Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR
|
||
|
xarmian
|
6a5eb3dee0 |
fix(attachments): HEIC, HEIF and AVIF are recognised from their bytes, not refused (BUG-2961) (#1291)
fix(attachments): HEIC, HEIF and AVIF are recognised from their bytes, not refused (BUG-2961)
image/heic, image/heif and image/avif have been on the upload allowlist all
along and every real file of those types was rejected 415 mime_not_allowed:
http.DetectContentType implements the WHATWG mimesniff table, which has no
signature for ISO base media file format still images, so they sniffed as
application/octet-stream and ValidateUpload's first rule refused them before
the extension cross-check was ever consulted. HEIC is the iPhone camera
default, so that was every photo shared straight from an Apple device, through
the web UI today and the mobile share sheet being built.
The fix stays inside rule 1 rather than around it. sniffISOBMFFImage reads the
ftyp box and returns the allowlisted MIME, so the sniffed type is still what
the allowlist is consulted with; it returns "" for everything else, so it can
only add detections. The alternative was to trust the filename for these three,
the way ValidateUpload already trusts it for zip-based Office documents, and
that was refused in triage because a renamed .heic would then pass on a name
alone. Both tests layers pin that: an executable named .heic is still refused,
and real HEIC bytes named .jpg are still stored as image/heic.
Four decisions, each argued at its site:
- Every brand in the box, not just the major one. The measured Apple HEIC
carries major heic with compatible mif1 MiPr miaf MiHB heic; the measured
Apple AVIF carries major avif with compatible MiPr avif miaf mif1; libheif
writes major mif1. A check on bytes 8..12 alone is right for one encoder
and wrong for the next. A specific brand wins wherever it appears; mif1
alone means image/heif.
- Sequence brands (hevc/hevx/hevm/hevs, avis, msf1) are deliberately absent:
they name image/*-sequence types, none of which is on the allowlist, so
recognising them would produce a sniff ValidateUpload then refuses with a
stranger message.
- A container carrying an mp4 brand stays a video and is left to the stdlib.
- A size-1 (64-bit largesize) ftyp box is declined, because the brands shift
eight bytes and the fixed offsets would be reading the size field.
Fixtures are real encoder output with provenance recorded in testdata/README.md
(sips on macOS 26 for the Apple pair, libheif 1.20.2 here for the other two);
hand-built byte vectors appear only for shapes no available encoder produces.
Verified by negative control — with the pre-check disabled all four new
acceptance tests fail — and by a nine-mutant matrix in which the major-only
scan, the dropped mp4 yield, generic-over-specific preference, the minor
version read as a brand, the missing box bound, the removed largesize bail, a
dropped brand table entry and an unconsulted sniffer are each killed by a named
test. The one survivor is equivalent: an 8..11-byte buffer yields no whole
brand either way.
Codex found two more refusals this owed, both now guarded and mutation-checked:
a declared box size of 2..15 is impossible for an ftyp box, and a buffer that
stops inside the 16-byte header is a truncated file — in either case the bytes
at offset 8 are not brands of a box that exists, and classifying from them let
a caller-controlled payload be stored and served as an image.
Also corrects processor.go's claim that "display always works (browsers handle
WebP / AVIF / HEIC natively)", which the client's own viewer table contradicts
and which this change would have made load-bearing (CONVE-23). What it means
for rendering on builds that cannot derive a thumbnail is filed as BUG-2964;
the wider class of allowlist entries the sniffer cannot produce is BUG-2963.
Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR
|
||
|
xarmian
|
367aae8e18 |
fix: one prefix grammar, and the ref parser widens to it (BUG-2943) (#1286)
* fix(collections): a DERIVED prefix is A-Z only (BUG-2943) DerivePrefix took the first BYTE of each word, so a collection named 'TEMP Rook A 2870' got the prefix 'TRA2'. parseItemRef resolves a PREFIX-NUMBER ref only when every prefix character is A-Z and otherwise falls through to a slug lookup, so every item in that collection printed an issue ID the CLI then refused: 'pad item show TRA2-2942' answered 'item not found' while the slug resolved fine. Two functions, each locally reasonable, disagreeing about what a prefix may contain — and the generator was the permissive one, so the failure surfaced at read time on an identifier the product itself minted and printed. The first-BYTE bug had a second half: a word starting with a multi-byte rune contributed a UTF-8 lead byte, so a collection named in most non-Latin scripts produced a prefix that is not even valid text. Non-letters are SKIPPED rather than mapped — there is no honest A-Z substitute for '2' or 'Omega', and inventing one puts a character in the ID that is in nobody's collection name. A name with no ASCII letters yields the empty string, which store.CreateCollection already turns into its ITEM fallback. SCOPE, stated because the first draft of this message overstated it (codex round 1 [P2]): DERIVED prefixes are safe now; the INVARIANT IS NOT ENFORCED. Three other doors store a prefix verbatim and unvalidated — CreateCollection with an explicit input.Prefix, UpdateCollection, and workspace import — so the same unresolvable-ID defect is still reachable through the API, the --prefix flag and a restore. Named on the trail with their call sites, held for a ruling rather than swept into this commit, because the import door wants a different answer from the other two: refusing a restore is not obviously right. The parity test lives in internal/store, where parseItemRef is: it asserts the generator against the RESOLVER rather than against a restatement of the resolver's rule, which is how these two drifted apart in the first place. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR * fix(store): one prefix grammar at all four doors, and the parser widens to it (BUG-2943) The ruled shape, which dissolves the import dilemma rather than choosing a side of it: collections.IsValidPrefix is the single definition — an uppercase letter followed by uppercase letters or digits — and parseItemRef now asks it instead of carrying its own stricter A-Z rule. Because the PARSER widened, a workspace already carrying a prefix like AB1 resolves every item by its printed ID the moment this ships. No migration, no rewrite of an identifier a user's other records may reference. The four doors: - derive: unchanged from the previous commit, still letters-only, still within the grammar; - create with an explicit prefix: REFUSED if outside the grammar, with a message naming the rule. The caller typed it, so a refusal is actionable; - update: same, and it matters more here — update is the door someone reaches for to FIX a bad prefix, so it must not accept another one; - import: the most permissive door that can still be honest. Anything the parser resolves is accepted (which now includes digits); only a prefix NO surface could resolve is refused, naming the collection and saying the export can be edited. Carrying that verbatim would restore a workspace whose items print IDs the CLI answers 'not found' to, which is this item's defect rather than a compatibility owed. An ABSENT prefix on import is not an unresolvable one. Old exports and every fixture in the suite carry "", and the first version of this check refused them — turning a fix for unresolvable IDs into one that cannot restore an old bundle at all (caught by three server tests). It now takes the same derive-then-ITEM fallback CreateCollection applies, which also upgrades it: an empty prefix is itself unresolvable, since the ref would begin with a dash. A prefix accepted only because the parser widened is logged at WARN, so an operator can see an id-space that would have been rejected before rather than inferring it from a resolve failure that no longer happens. Tests for each door follow in the next commit. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR * test(store): one test per prefix door, plus the parser round trip (BUG-2943) Each door asserted separately: 'they all call the same helper' is a claim about the code, not about behaviour, and the bug was two definitions disagreeing. - create with an explicit prefix: AB1 accepted and resolves; ab1, 1AB, 'A B', A-B, A!, a non-Latin letter and a bare digit refused, with the rule named; - update: AB1 accepted, a bad replacement refused AND the stored prefix unchanged after the refusal — update is the door someone uses to FIX a bad prefix, so it must not swap one unresolvable id-space for the next; - import: a digit-bearing prefix restores unrewritten and resolves; one no surface can resolve is refused naming the collection and the export; an ABSENT prefix takes the create-path fallback and comes back resolvable; - the parser: every prefix the doors accept round-trips, and 1AB / 9 / 'A B' / A! / a trailing dash / a bare prefix stay refused. One correction: my first version of the parser test asserted that 'ab1-42' is refused. It is not, and the code is right — parseItemRef upper-cases before splitting, which is what makes "pad item show task-5" work. Case-insensitivity is now PINNED rather than mis-asserted, because a later reader working from the grammar comment alone would otherwise 'fix' it. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR (This message was rewritten once: the sentence above lost its example because the original was written with backticks inside a double-quoted shell string, which the shell EXECUTED and replaced with the command's empty output. The span was blank in the commit as first written.) * fix: the widened grammar reaches its consumers too (BUG-2943) Codex round 2. Widening parseItemRef without widening what CONSUMES a ref would have left the same two-definitions bug this unit is about, introduced by its own fix: - cmd/pad/cmd_github.go matched [A-Z]+-\d+, so 'pad github link' on a branch carrying a digit-bearing ref silently found nothing; - web localSearch's palette Enter fast-path could not recognise one either. Both now match collections.IsValidPrefix. Tests strengthened, both on codex's reading: - the import fallback pinned the VALUE, not just resolvability — asserting 'non-empty and parseable' passes an implementation that stamps ITEM on every absent prefix, giving every collection in a restored workspace the same id-space. Two legs now: an ordinary name derives TASK, a letterless name falls through to ITEM, which is what makes it 'derive, THEN ITEM'; - the WARN the ruling asked for had no test, so it was a line nobody would notice was gone. Now asserted, with a control that an ordinary prefix does NOT warn — a log everything trips is a log an operator learns to skip. Three comments still described the parser's A-Z rule as current, including one in the file that changed it. STILL OPEN, on the trail for a ruling: web paneTarget.ts keeps the narrow grammar on PURPOSE — its comment argues a digit-permitting shape would misclassify a slug like 'roadmap2-5' as a ref — and that argument cited the server rule this unit just widened. Whether the guard follows or stays is a question about the widening's blast radius, not a line to change quietly. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR * fix(web,store): the pane guard follows the server, and the precedence is written down (BUG-2943) Both lead-ruled after codex round 2 surfaced them. paneTarget's REF_SHAPE kept a LETTERS-ONLY grammar deliberately, citing the server's A-Z loop as its warrant. The server dropped that rule, so the guard was holding a grammar nothing else holds — which does not avoid a wrong answer, it produces a different one. It now matches IsValidPrefix. The cost is real and is stated in the test rather than buried: an HREF whose last segment is ref-shaped under the wider grammar is compared by NUMBER with the prefix discarded, so a genuine slug like 'roadmap2-5' now counts as the same pane target as TASK-5. The existing test pinned the opposite and is REPLACED, naming what changed and why. The prefix is dropped because a moved item keeps a stale one (the server's own number-only fallback) and PaneGuardItem carries no prefix to compare; tightening that means widening that type and its callers, which is a separate change and is on the trail. The SLUG-channel leg is kept as its own test: provenance, not grammar, is what protects it — a target naming an item by slug is judged only as a slug. ResolveItem's ref-before-slug precedence is now documented on the function and pinned in both directions: 'ab1-42' resolves as a SLUG when no AB1-42 exists, and a live ref wins when it does (case-insensitively). The widening made more strings ref-shaped, so 'is my slug still findable' needed an answer that does not depend on reading the resolver. Web unit tests run here via a node_modules SYMLINK to the main checkout, which CLAUDE.md permits; npm ci was not run and must not be. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR * fix(web): the self-pane guard was the fourth copy of the ref grammar (BUG-2943) Codex round 3 [P1]. The route-level refNumber() still matched [A-Za-z]+, so a master whose ref is R2-1 parsed as null, its item_number fell back to 0, and the same-item guard stopped recognising ?item=R2-1 as the master — mounting a second provider for the item already on screen. That is the FOURTH consumer found carrying its own copy of this grammar (github branch extraction, the search palette, the pane target guard, and now this). Four independent copies is the argument for the shared definition rather than for four careful edits, and it is why the widening had to be swept rather than applied where it was noticed. Also from round 3: comments saying these client regexes 'match collections.IsValidPrefix' were imprecise — the validator accepts uppercase only, while the client patterns accept either case on purpose, because a user types a ref however they like and the server upper-cases before splitting. They mirror the ref GRAMMAR, and now say so. Web gates run here through a node_modules symlink to the main checkout (permitted; npm ci is not): vitest 2195 passed, svelte-check 0 errors. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR * fix(collections,e2e): the fifth and sixth copies, and the docs that taught the old rule (BUG-2943) Codex round 4, after I claimed the sweep was complete twice. - Two SEEDED PLAYBOOK BODIES carry their own ref grammar and instruct agents with it: playbook_library_plan.go and templates_sdd_spec.go both said a ref 'matches ^[A-Z]+-\d+$'. An agent following those literally would refuse to treat AB1-42 as a ref — a grammar copy that lives in PROSE and is executed by a reader rather than a regexp engine, which is why two sweeps of the code missed it. - Three e2e comments taught the defect as a rule: one of them carries the empirical confirmation ('GET /items/BS1-10 404'd while the slug worked'), which is precisely this bug. They now say the by-ref 404 is fixed and that the explicit prefix those suites pass buys DETERMINISM rather than dodging it. Counting honestly: six live copies of one grammar, found in four rounds of review, two of which I opened by asserting there were no more. The shared definition is the fix; every one of these was a place that had quietly made its own. Gates: go test ./... 0, make lint 0 issues, vitest 2195 passed, svelte-check 0 errors (web run through a node_modules symlink to the main checkout — permitted; npm ci is not). Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR |
||
|
xarmian
|
3a00ea9c8c |
fix(cli,mcp): item update --parent "" is refused, not silently ignored (BUG-2941) (#1285)
* fix(cli): item update --parent "" is refused, not silently ignored (BUG-2941) It exited 0 and printed the updated item while doing nothing: hasFieldChanges tests parentRef != "", so the empty value built no patch and the key the server's clear-path needs (parent present, empty) never reached the wire. The two representations of the link then disagreed — parent_id read null while parent_ref and the child listing still named the parent, and the parent still could not be closed for open_children. BUG-2078 shipped --clear-parent as the working route and left this one looking like it worked. Refusing rather than aliasing it: two spellings for one operation is what produced the confusion, and naming the flag that does the job is the actionable answer. UPDATE only. On create an empty --parent expresses nothing to ignore and --parent "$MAYBE_EMPTY" is a normal shell idiom; a test pins that asymmetry as a decision rather than a gap. Compat note for review: a script passing --parent "$P" with P empty gets a loud failure where it used to get a silent no-op. That is the point of the change, but it is a real behaviour change for callers who were relying on the no-op, and it is the one thing here worth a second opinion. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR * fix(cli,mcp): refuse before the request, and keep MCP's empty-parent convention (BUG-2941) Codex round 1 found two P1s in the first shape of this fix. 1. THE REFUSAL WAS TOO LATE. It sat beside the parent handling, which is after the item fetch, so a refused call still made a GET. My test only watched for writes, so it passed a version that refuses after fetching — the test was not an instrument for the claim it was named for. The guard now runs first in RunE, before the client exists, and the test counts EVERY request rather than ignoring GETs. 2. THE FIX WOULD HAVE SPLIT THE TRANSPORTS. Stdio MCP shells out to the CLI and BuildCLIArgs emits a flag for any key that is PRESENT, so the catalog's `parent: ""` — documented inert since v0.19 — became `--parent ""` and would now be refused on stdio while the remote door went on ignoring it. A transport divergence created by a fix for a transport-independent bug, which is the class BUG-2870 exists to close. dropInertEmptyParent removes the key before dispatch. Dropped there, refused at the CLI: same input, opposite dispositions, because the two surfaces have opposite conventions about what an empty declared string means. At the CLI a human typing it means "detach"; in the catalog it means "not provided", and `clear_parent` is the documented way. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR * test(mcp): the empty-parent drop is about parent, not about emptiness (BUG-2941) Codex round 2 [P2]: the test could not tell 'drops an empty parent' from 'drops every empty-valued key', which would be a much larger and undiscussed change to the tool's input handling. It now carries an empty `comment` alongside and asserts that one still reaches the CLI. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR * docs: two wording corrections from codex round 2 (BUG-2941) BuildCLIArgs emits THIS STRING FLAG whenever its key is present — booleans and hidden flags are handled differently, so the broader claim was wrong even though it held for the case at hand. And '--parent "" reads as detach' described the caller's intent as though it were the code's behaviour; it never was, which is the whole bug. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR |
||
|
xarmian
|
ee0d945863 |
fix(cli,mcp): one --field key=value entry means one thing at every door (BUG-2870) (#1283)
* feat(items): one shared parse for a --field key=value entry (BUG-2870) Six sites parsed that entry independently — item create, list, update, move and copy in cmd/pad, plus ingestFieldKVP on the remote /mcp door — in four spellings, and they disagreed about what it meant. The CLI sites used both halves verbatim, so `--field " effort=l"` stored an undeclared field named " effort" and left the declared `effort` untouched; the remote door trimmed both halves and wrote `effort`. Same call, two stored keys, decided by which transport the caller was on. This is the helper only; the call sites move over in the commits that follow. Two rules, deliberately asymmetric, per the day-60 ruling: - a KEY whose trimmed form differs from what was written is REFUSED at every door, rather than silently retargeted to a different field; - a VALUE is carried VERBATIM at every door, because trimming reinterprets a caller's bytes and on a text field the space is content. A padded value against a typed field is refused one layer down by validation, naming the field — measured, not assumed. ErrFieldEntryMalformed is returned rather than handled because the six sites deliberately disagree about a malformed entry (four skip it, copy hard-errors) and unifying that is a separate decision. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR * fix(cli,mcp): all six --field parse sites go through the one helper (BUG-2870) item create, list, update, move and copy in cmd/pad, plus ingestFieldKVP on the remote /mcp door, now call items.SplitFieldEntry instead of each rolling its own split. A padded key is refused at every door; a value reaches every door verbatim. Two sites keep something specific to them, both documented in place: - `item list` is a READ filter, and it takes the same key rule deliberately: a padded key there filters on a field nobody declared and returns empty, which is indistinguishable from "no rows match". - `item move` gets KEY normalisation only. Its values stay strings because the server types a declared field on that path too, so a clean `--field n=3` already stores the number 3 — measured before the change. Each site keeps its historical disposition toward a MALFORMED entry (four skip silently, copy hard-errors), which is why the helper classifies that case rather than deciding it. NOT YET EVIDENCE: ./internal/mcp, ./cmd/pad and ./internal/items all pass, and that green does not show the divergence closed — the three BUG-2850 pinned tests exercise the catalog conflict pass, which never reaches ingestFieldKVP. The door-level test and the re-grounding of that pass are the next commits. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR * test(cli,mcp): pin the door-parity claim at both doors (BUG-2870) Nothing in the suite asserted what the remote door STORED for a padded entry — the three BUG-2850 tests that cite its trimming all exercise the catalog conflict pass, which never reaches ingestFieldKVP. So the previous commit's green was not evidence for the thing it changed. Three files now hold the claim: internal/items pins the rule, internal/mcp pins the remote door, cmd/pad pins the CLI door, and each cites the other two. Padded key refused at both; padded value carried verbatim at both; a refusal aborts the call rather than dropping one entry, and on the CLI it happens before any request reaches the server. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR * fix(mcp): re-ground the conflict pass on the new door behaviour (BUG-2870) The pass's rules were derived from ingestFieldKVP trimming, so changing the door without changing the layer built on it would have been the same one-door lapse a level up. - parseFieldArray splits through items.SplitFieldEntry: a padded key is REFUSED before dispatch on both transports, and values are indexed RAW, because raw is now what both doors write. - Both comparison sites compare raw for the same reason. The round-19 "COMPARED TRIMMED" rule is superseded and its comment says so. - detectFieldConflicts PROPAGATES the parse refusal instead of returning nil. It swallowed it as "the caller owns this error surface", which was true when the only possible error was a shape error — reshapeItemFields returns early with no `fields` object, so on the no-`fields` path (this bug's path) nobody owned it and a padded entry turned back into a success. - A padded entry is refused in the pass rather than skipped. Skipping dropped it from conflict detection entirely, turning four existing refusals into successes. The last two were caught by the BUG-2850 tests, not by reasoning: the first shape of this commit passed a full package build and turned four guards off. Seven tests still fail. They assert the OLD door behaviour and are the specification being changed; each gets read on its own next, and is either kept because the behaviour survives or replaced by a test stating the new behaviour that cites the old name. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR * test(mcp): restate the seven BUG-2850 pins on the new rule (BUG-2870) Each was read on its own and either kept or replaced; every replacement names the test it replaces and why the old assertion was right at the time, so the deletion is traceable rather than a green that appeared. - padded value is not a conflict → IS a disagreement now that no door trims (" done" and "done" are two values), with an equal-values control leg. - padded entries still caught (hierarchy) → refused EARLIER, by the padded-key rule, before the alias pass observes both keys. The alias guard keeps its three unpadded cases, which is what stops this being a hole. - PaddedEqualDuplicateIsCanonicalized → IsRefused, plus a canonical control that still emits --field exactly once. - MixedCanonicalAndPaddedDuplicatesCollapse → Refused. The round-8 finding survives: one canonical entry still does not make its padded sibling harmless, it is refused rather than swallowed. - PaddedEntryAloneIsUntouched → IsRefused. That test pinned a DEFERRAL, in its own words "BUG-2870's business, not this PR's". This is that business. - "fields carries the key — canonicalized, so accepted" → still refused, since nothing canonicalizes now; the per-key question it defended is still tested by the two legs beside it, and a canonical control was added. - ReEmittedValueKeepsItsWhitespace → the re-emission path is gone, so it becomes a refusal test that also asserts the ADVISED form is accepted with its value untouched. The property it defended is pinned at both doors. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR * test(server): pin that a move override is typed server-side (BUG-2870) The fact the ruling turned on, and the easiest one in this unit to lose: it is invisible from cmd/pad, where moveCmd plainly sends a string. - a declared number field given the STRING "3" through field_overrides ends up as the NUMBER 3, which is why move needs the shared KEY parse and no client-side typing; - a padded " 3" is REFUSED with a 400 and the item does not move, which is the answer the remote door will now give too instead of trimming and succeeding. t.Parallel per CONVE-2086 — both build their own server through testServer, so each has its own database, limiter and bus. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR * chore(mcp): bump tool surface to 0.30 and sync the docs the guards enforce (BUG-2870) Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR * refactor(mcp): remove the canonicalization the door change made unreachable (BUG-2870) Two mechanisms existed to make a padded entry reach both doors as the same write: the nonCanonical conflict guard (round 16) and the re-emission path that rewrote a padded entry to canonical form (rounds 7/8). Both are dead now — items.SplitFieldEntry refuses a padded key, so every entry that parses satisfies `entry == key + "=" + value` BY CONSTRUCTION. Removing each changed no test. That is consistent with "dead" and with "untested" alike, so the construction argument above is what settles it — recorded in the comments that replace them, along with what the removed guard was defending and where that premise is enforced now. Rewriting a caller's key was also the behaviour this bug is about, applied by us rather than by a door: canonicalization silently changed the key the caller wrote. Refusing says so instead. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR * docs(mcp): put the trimming narrations in the past tense (BUG-2870, CONVE-23) Six comments described the old door behaviour in the present tense ("HTTP trims and writes effort"), which reads as a claim about the code as it stands. The rounds they narrate still explain why the surrounding rules exist, so they are re-tensed rather than deleted. Two references were checked and left alone because they are still true: ingestFieldKVP does still store every field value as a STRING (coerce.go's BUG-2850 note, and the github_pr hint in dispatch_http.go). This change stopped it TRIMMING, not stringifying. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR * docs: sync CLAUDE.md to tool surface 0.30 (BUG-2870) The drift guards cover instructions.md and README.md but not this file, and its own 0.27 entry records the consequence: 'This entry was missing from CLAUDE.md — the 0.27 unit swept instructions.md and README.md and not this file.' The unit that makes a version line stale is the unit that owes it. Both markers updated, and the entry states the two behaviour changes in the terms they were ruled: /mcp refuses what it silently accepted, and the swallowed parseFieldArray refusal that was landing four refusals as successes on the no-fields path. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR * fix(mcp): finish the removal, and correct a claim I made twice (BUG-2870) Codex round 1: no P1/P2, two nits, both real. 1. The re-emission removal was incomplete. `reEmitFields` and the branch that appended its entries survived with nothing populating the map, and two comments still described canonical re-emission as something this code does. Unreachable, but my own commit message had said the path was removed, so the code contradicted the claim. Removed, and the round-16/17 paragraphs that decided WHEN to canonicalize go with it — they answered a question that no longer arises. 2. "The only behaviour change is /mcp refusing what it silently accepted" is WRONG, and it was in version.go, README.md and CLAUDE.md. Every door refuses a padded key now; they were merely accepting it differently — /mcp trimmed it and wrote the declared field, the CLI stored a ghost field beside it. What is /mcp-only is the VALUE half. Corrected in all three, with the correction itself recorded in the version.go entry so the next reader sees the claim was checked rather than a sentence that quietly changed shape. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR * docs(mcp): rename the predicate to the question it asks (BUG-2870) Codex round 2: no P1/P2, three nits, all naming and prose. - `canonicalized` is renamed `coveredByFieldsObject`. Nothing canonicalizes anything any more, and the only thing that predicate ever asked was whether the `fields` object carries THIS key — it kept the old name only because the guard it used to feed had been removed a commit earlier. - parseFieldKVP's doc said invalid entries are skipped silently. True of a MALFORMED entry, false of a padded key, which now aborts the call. - Three test comments still described re-emission as live, and version.go described this door's trimming in the present tense. Nothing in these two rounds was a defect in the change itself; both rounds found prose describing a version of the code that stopped existing partway through the unit, which is the failure mode a re-grounding pass invites. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR * docs(mcp): last of the prose that outlived the code (BUG-2870) Codex round 3: no P1/P2, prose only. - the predicate's own comment still asked 'will anything canonicalize THIS key'; it asks whether the fields object carries the key, and always did; - two test comments described re-emission and trimmed comparison as current. Both tests are kept — what they pin is narrower now and still worth pinning — with the change in what they mean written down. Deliberately NOT changed: the comments and replacement-test names that cite the OLD test names. Codex reads them as stale terminology; they are the traceability the restatement commit was asked for, so a reader can find what each replacement replaced. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR * docs(items): the coercion note names what the door does now (BUG-2870) Codex round 4. The paragraph described ingestFieldKVP as doing `dst[key] = val` unconditionally. Its CLAIM — every value arrives at the server as a string — is still true and is the reason this file exists; the description of the line is not, since that door now parses through items.SplitFieldEntry. Restated so the still-true part is not carried by a sentence a reader can falsify. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR |
||
|
xarmian
|
110045578c |
fix(build): make install proves what it installed and what it restarted (BUG-2897, TASK-2787) (#1272)
`make install` made three claims it did not check.
1. It brought the server back BY SIDE EFFECT -- `pad auth whoami` triggers
an auto-start, which does not know the killed process's argv. A server
running `--host 0.0.0.0` came back bound to the default host alone:
curl 127.0.0.1:7777 -> 000 while the LAN address -> 200, with the
process count and the version both reading correct (BUG-2897).
2. `cp` copied whatever was at the repo path, not what the invocation
built. Two sessions sharing the checkout interleave and the loser's
build is installed by the winner, every exit code green (TASK-2787).
3. "Server restarted." was printed after a command ending in `|| true`,
with no probe of any kind. Not "the wrong address went unverified" --
nothing was verified. Found reading the recipe; neither filing names it,
and it is what made the other two invisible.
The logic moves to scripts/install-refresh.sh for one reason above
readability: a script can be TESTED. internal/buildtools drives it against
a compiled stub `pad`, including the branches where a check must FAIL.
Recipe-inline logic is only exercisable by running `make install`, which
stops the developer's server -- a test nobody runs twice, which is how this
target accumulated three unverified claims.
The script checks OUTCOMES rather than steps: what got installed, and what
is answering afterwards. Two commit checks, deliberately, answering
different questions -- the ARTIFACT before the kill, so a wrong build costs
an error instead of an outage, and the DESTINATION after the copy, which is
the shared-path race TASK-2787 names. The restart uses the argv read from
/proc before the kill, and nothing is printed about a restart until the
server answers on BOTH 127.0.0.1 and the configured host (`--host 0.0.0.0`
resolving to loopback plus the primary LAN address, since 0.0.0.0 is a bind
spec, not something to curl).
Three defects were found in the fix itself, by its own tests and by the
first real run:
- The restart redirected to $HOME/.pad/server.log with nothing creating
that directory. On a fresh HOME the redirect fails, the server never
starts, and the probe reports "did not answer" -- true, and three steps
downstream. Invisible to every hand-run this script could have had,
because a developer's box has ~/.pad by luck of history.
- The post-copy check SURVIVED its mutant: both fixtures reported the same
version from source and destination, so the guard and its absence were
indistinguishable. Fixed by making the stub's version depend on its path.
- Comparing commits by equality rejected a healthy build. `git rev-parse
--short` returns the shortest UNAMBIGUOUS prefix, so its width grows with
the object database: the binary embedded `a3a1d58` and the Makefile
produced `a3a1d586` minutes later. Now a prefix comparison in either
direction, with a negative leg pinning that a genuinely different commit
of the same width is still refused.
Five mutants, each verified to compile, each detected by its own leg.
Verified end to end on the real box: captured `--host 0.0.0.0`, installed
|
||
|
Matt Faltyn
|
a3a1d5862b | fix(store): preserve valid item references across workspace import (#1271) | ||
|
xarmian
|
bb8ec04ef1 |
fix(server,store): both workspace mint doors enforce their preconditions from one place (BUG-2809) (#1268)
handleCreateWorkspace and handleImportWorkspace mint the same thing
through the same store.CreateWorkspace, and enforced preconditions in two
places. Two had already diverged and been fixed one at a time, each found
by a reviewer rather than by the door that lacked it: the OAuth consent
grant (IDEA-2756) and the user-scoped plan limit (BUG-2793). A third was
live.
The shared place is internal/server/workspace_mint.go, split by WHEN a
precondition can run, and the split is load-bearing rather than tidy:
beginWorkspaceMint — everything that does not need the body (consent,
plan limit, and the owner/source attributions). Runs before the body
read, so a refused caller never uploads a bundle and a refusal cannot
be probed by body shape; on the import route it sits above the
Content-Type dispatch, so one line covers both body shapes.
validateWorkspaceMintPayload — the payload-shaped rules. Returns an
error rather than writing one, because the JSON doors answer 400
bad_request and the bundle door answers 400 bad_bundle through
importStatusError. The rule is shared; the envelope stays each door's.
Callers: handleCreateWorkspace, handleImportWorkspace, and importBundle.
The mint context reaches the bundle path as an ARGUMENT rather than on the
Server, because it is per-request state and the two things it carries are
exactly what two concurrent requests would differ on.
THE LIVE DEFECT. Import accepted an empty workspace name. Measured before
the fix: it created a workspace with name="" and slug="", and a second
such import landed on slug "-2" -- the first had taken the empty slug,
globally, and a slug is a routing key. Both import doors now refuse it,
checking the EFFECTIVE name (the ?name= override when given, the bundle's
own otherwise) because that is what becomes the slug. A control leg covers
the override, or the rule would be indistinguishable from "reject any
bundle whose payload name is empty" and would break rename-on-import.
SETTINGS: the item's premise was wrong and this corrects it rather than
fixing it. Malformed settings never reached the store unnormalized --
createWorkspaceQ calls NormalizeWorkspaceSettings itself and refuses. What
diverged was the STATUS: create answers 400, import answered 500
import_failed because handleImportWorkspace maps every store error that
way. Validating in the shared payload step makes both 400. Context stays
create-only: an export carries none, so applying it on import would be
inventing input.
SOURCE: imported workspaces got no attribution at all (BUG-1557).
store.ImportWorkspace now takes a source parameter, derived by the caller
from the request's auth shape exactly as create derives it -- a parameter
rather than an export field, because a bundle says what the workspace WAS
and where this copy is minted from is a fact about this request. The
operator path (pad db migrate-to-pg) passes "": it is a copy, not a
creation surface, and inventing "cli" would relabel every migrated
workspace's origin.
Userless callers (the inventory's fourth item) are deliberately unchanged.
beginWorkspaceMint preserves the userID != "" guard exactly as both doors
had it rather than changing behaviour under cover of a refactor; the
measurement and the ruling are on BUG-2914.
Five mutants, each verified to COMPILE first and each detected by its own
leg: either import door skipping the payload check, the create door
skipping it, checking the payload name instead of the effective name, and
passing "" for source. Two of them initially did not compile, and go test
answers a build failure with FAIL <pkg> [build failed], which in a
filtered run reads exactly like detection -- a false DETECTED, the mirror
of the false SURVIVED. Re-run with the orphaned variable kept alive.
Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR
|
||
|
xarmian
|
cdc5b301e2 |
fix(server): minting or rotating an API token requires an interactive session (BUG-2890) (#1267)
A PAT that could reach POST /auth/tokens minted further tokens with
independent names and expiries. Those survive the revocation of the token
that created them, and nothing in the token list records which token
minted which -- so revoking a leaked credential did not end the access it
had been used to establish.
Ruled on the item's trail day 57: create and rotate require session auth
and refuse a PAT-authenticated call with 403 session_required; list and
revoke stay PAT-reachable.
Population is THREE doors, not the two the filing named. Enumerating every
route that reaches store.CreateAPIToken / RotateAPIToken turns up the
workspace-scoped mint at POST /workspaces/{ws}/tokens, which had only
requireMinRole("owner") in front of it -- and a user-owned PAT held by an
owner satisfies that. Measured, not read: with the fixture's owner also
written into workspace_members, that door returned 201 with a live token
on the unfixed tree. All three doors are gated; list and revoke on both
the user and workspace routes are deliberately untouched, because neither
extends access and revocation is the compromised-credential response.
The gate is on the CREDENTIAL, not on the door: isAPITokenAuth is false
for a session cookie AND for a padsess_ CLI bearer, a distinction that
predates this fix (see ctxValidatedSessionBearer's note in
middleware_auth.go) because a CLI session IS an interactive session. At
the workspace door the check runs before requireMinRole, so a PAT-borne
caller cannot learn its own membership status from the difference between
the two 403s.
Five mutants run, each detected by its own leg: dropping the gate at each
of the three doors fails that door's test; gating on Authorization != ""
instead of the credential kind fails the CLI-session leg AND NOTHING ELSE;
over-applying the gate to list and revoke fails both PAT-still-works
controls. The fourth is why the CLI leg exists -- a header-shaped gate
passes every PAT assertion and breaks every logged-in CLI.
Also documents the refusal on the CLAUDE.md route line: the routes were
listed without saying anything about credential kind, so an agent holding
a PAT would have met an unexplained 403.
Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR
|
||
|
xarmian
|
e32d4f468e |
fix(store): a relation value pointing at an orphaned item is carried, not rewritten into a minted id (BUG-2895) (#1265)
ImportWorkspace's second pass handed remapFieldIDs the unfiltered itemMap. That map holds an entry for EVERY item in the bundle including orphans -- items whose collection the bundle does not carry -- because the entry is written before the orphan skip and parent resolution inside the same loop reads it for items the loop has not reached yet. So a relation field pointing at an orphan was rewritten to the id that orphan WOULD have received: an id that names no row in the destination and has never identified anything in any workspace. That is worse than dangling. TASK-2878's carry rule imports an unresolvable relation value verbatim precisely so nothing is invented -- the source id is evidence a human or a repair tool can act on -- and ImportWorkspace runs no MigrateRelationReferents pass afterwards to re-examine what this wrote. The fix is the same correction BUG-2884 made for parent_id: filter the map to items that actually landed. It arrives separately because every site that unit fixed writes a FOREIGN KEY, so the database objected and the sweep found them. This one writes into a JSON blob, so nothing objected. Population, re-verified at this tip rather than carried from the recon: itemMap has eight consumers inside ImportWorkspace and seven already check insertedItems; this call was the eighth. collMap needs no equivalent -- the collections loop has no `continue`, so every collection either INSERTs or the import returns an error, and no collMap entry can name a row that does not exist. Repo-wide, remapFieldIDs has one caller. Search boundary: internal/store. Also drops remapFieldIDs' collMap parameter, which was never read. A relation's target collection travels in the SCHEMA as a slug, not as an id in the fields blob, so an unread parameter there reads as a claim that collection ids are remapped -- they are not. Test: TestImportWorkspace_DoesNotMintIDsForOrphanRelationReferents. The bundle is hand-built because ExportWorkspace cannot produce an orphan, which is the population the carry rule exists for. Three mutants run, all detected: restoring the defect fails the orphan leg with a fresh UUID; deleting the second-pass remap fails the CONTROL leg; filtering on the wrong key space (itemMap is old->new, insertedItems is keyed by NEW) fails the control too. Under the first mutant the pre-existing carry test stays GREEN -- its unresolvable ids are absent from the bundle, so they are absent from itemMap, which is why no existing test could see this. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR |
||
|
xarmian
|
3c93472613 |
perf(store): run the store suite in parallel (TASK-2900 deliverable 3) (#1263)
The SQLite leg was CPU-bound and serial: 154.9s of CPU inside 185.6s of wall on an eight-core box, which is 0.84 cores. Unlike the Postgres leg there was no setup cost to reclaim — the unattributed gap between package wall-clock and the sum of its tests measured MINUS 1.4s, so per-binary overhead was already nil — and no fat test to find beyond one 48s outlier. The only lever left was the seven idle cores. t.Parallel() on 844 of 906 tests. Every test in this package builds its own isolated store (a SQLite file copy, or since |
||
|
xarmian
|
7659ad3cd3 |
feat(server,web,cli): say when a relation's copy target is unusable, instead of offering a picker that cannot answer (IDEA-2899) (#1262)
* feat(server): the copy preflight says when a relation's target is not usable (IDEA-2899) TASK-2869 made a `needs_value` relation row collectable as soon as it names a target collection. Naming one is not having one: the slug can name a collection that has been DELETED, or one this caller cannot READ. The dialog then mounts a picker that can return nothing and, because the row is not blocked, Confirm stays disabled carrying only the generic required-field message — the user is told a value is missing and never told that no value is reachable. `collection_unavailable` on the needs_value row is the server saying so. THE CLIENT CANNOT COMPUTE THIS, which is why it belongs here. The dialog's destination collection list is filtered through `canEditCollection`, because it drives the copy-INTO picker; a relation TARGET needs only READ access, so a perfectly usable target routinely does not appear in that list. Testing against it would refuse rows the user could have filled in — over-blocking, which is the worse failure and invisible to whoever hits it. `visibleCollectionIDs` is the read-scoped view, and its NAV-LENIENT shape is right here rather than merely tolerable: it includes a collection reachable only through an item-level grant, and the question is "could a picker here return anything at all". One granted item is a picker with one row. DELETED and UNREADABLE are deliberately not distinguished. Same consequence, no client branch would differ — and separating them would tell a caller who cannot read a collection that it nonetheless exists. `omitempty` on a BOOL drops `false`, so the field is phrased NEGATIVELY. Present-and-true means the server checked and the target is unusable; ABSENT means available, or a server that does not report. A client must block only on an explicit true, so absence stays "no information" rather than becoming a value — the rule `access_epoch` follows on the item doors, and the one whose violation cost two review rounds on IDEA-2898 this morning. Costs nothing on the common path: a destination schema declaring no relation field runs no query at all. Claude-Session: https://claude.ai/code/session_01Xk9M5UVPdc84xL5E1mZkm8 * test(server): pin the type gate on collection_unavailable (IDEA-2899) Found by a surviving mutant rather than by inspection: dropping the `def.Type == "relation"` gate left every other test in the file green. Nothing stops a schema declaring `collection` on a field of another type — the validator does not police keys it has no use for — and such a field would then pick up a flag whose meaning is defined only for relations. The dialog would block a perfectly collectable `select` because some relation elsewhere in the same schema points at a collection that happens to be gone. The fixture is the discriminating one: ONE deleted collection, TWO required rows that name it, and only one of them means anything by it. Six mutants on this half, all killed: flag never set, flag always set, deleted target not flagged, unreadable target not flagged, type gate dropped, and the nil-visible-set case (an admin's "no filtering" read as "nothing visible", which would flag every target for the callers who can see everything). Claude-Session: https://claude.ai/code/session_01Xk9M5UVPdc84xL5E1mZkm8 * feat(web): block a relation whose target is unavailable, and stop advising a command that cannot work (IDEA-2899) The client half. `isCollectable` now refuses a relation row the server has flagged, so the row lands in `blockedFields`, Confirm is disabled with a reason, and no picker mounts that could only come back empty. `collection_unavailable !== true` is STRICT on purpose. The field is absent when the target is fine and absent from a server that predates it, so absence must read as "no information". (Over the domain the type admits — `boolean | undefined` — the truthiness spelling is EQUIVALENT and a mutant swapping it in survives; that is recorded in the source rather than papered over with an off-contract fixture. The strict form is kept because it states the contract where the next edit will read it, and the inverse spelling would block every row against an older server.) THE PART THAT IS NOT WIRING: the existing blocked-field notice said the field "is a required <type> field. This dialog can't collect a value for that type safely" and then printed `pad item copy … --field key=value`. Both halves are FALSE here. The type is perfectly collectable; the TARGET is gone. And the CLI runs as the same user against the same referent validation, so the command it prints is refused for exactly the reason the user is already stuck — advice that sends someone to do work that cannot succeed is worse than no advice. So the message branches on `uncollectableReason`, names the collection and the destination workspace, and the CLI line is now gated on `cliFillableField` — the first blocked row the CLI can ACTUALLY fill. `blockedFields[0]` was correct while every blocked row was type-shaped; with an unavailable relation sorted first it named the one field `--field` cannot set either. Eleven unit tests on `copyNeedsValue`, plus a source pin on the dialog whose own measured limit is in its docblock. Client mutants: 7 real, 6 killed, 1 recorded as equivalent with the domain argument that makes it equivalent. Claude-Session: https://claude.ai/code/session_01Xk9M5UVPdc84xL5E1mZkm8 * feat(cli): the copy preview marks an unavailable relation target and stops suggesting it (IDEA-2899) Caught by `TestItemCopyMirrorsMatchServerShapes`, not by me. The CLI keeps a mirror of the preflight response, and adding a field server-side without mirroring it fails that test by design — a mirror that silently lags is a mirror that lies. Working exactly as intended, and the reason this half exists at all. Mirroring the field turned out to be the smaller part. The CLI already prints `target collection: people` for a relation row, and it builds an `Add: --field owner_ref=<value>` suggestion from every unsupplied row. Both are wrong when the target is unavailable: the first sends a user looking for a ref in a collection they cannot read, and the second hands them a command the referent validation refuses for exactly the reason they are already stuck. So the target line is marked NOT AVAILABLE, and the row is excluded from the suggestion with a sentence saying why — modelled on the empty-key branch, which was written for the identical reason (a `--field =<value>` nobody can run) and is three lines away. That the same defect had to be fixed in two places is the shape worth naming: the dialog and the CLI independently built "here is how to supply it" from "here is a field needing a value", and neither had a notion of a field that CANNOT be supplied. The empty-key case was the first instance and was fixed locally; this is the second. Five mutants on this half, all killed: suppression removed, suppression applied to everything, the unavailable label dropped, the explanation dropped, and the mirror field ignored. The available-target control leg is a separate test so the omitempty contract is exercised on this surface too. Claude-Session: https://claude.ai/code/session_01Xk9M5UVPdc84xL5E1mZkm8 * fix(cli): route all three "how to supply it" sites through one predicate (IDEA-2899) Review found the fix applied at one door and not its siblings — my own recurring shape, arriving again. THREE places tell a CLI user how to resolve an unsatisfied field: the detailed `renderItemCopyNeedsValue`, the `--dry-run` summary, and the error the command returns. The first commit fixed the render. The other two went on printing `--field key=value` at someone for whom no value exists — and the ERROR is the line a script or a hurried reader actually sees, so it was the worst of the three to leave. `itemCopyUnfillable` is now the single definition all three consult. Not because three call sites are tidier than one, but because three sites independently answering "how do I supply this" is exactly how they diverged in the first place. The dry-run summary branches three ways rather than two, because the MIXED case is the one a boolean gets wrong: some fields can be supplied and some cannot, and collapsing that either suppresses advice the user needs or offers advice they cannot use. The error hint is suppressed only when NO field can be supplied — with one fillable field left, `--field key=value` is still true. Also pins the BOUNDARY the same review probed: a target collection that is live and readable but EMPTY is deliberately not flagged. The symptom looks identical — an empty picker — but the cases differ where it matters. An unavailable target is unfixable from inside the dialog, so blocking costs the user nothing they had; an empty collection is resolved by creating the item and retrying, and blocking would refuse a copy they were about to complete. It would also cost a live-visible-item count per relation target on a dry run the UI calls on every keystroke. The weaker case — an empty picker that says nothing about WHY — is filed as IDEA-2905 and belongs to the picker. Ten mutants across this round, all killed, including both directions on the error hint and both directions on the dry-run branch. Claude-Session: https://claude.ai/code/session_01Xk9M5UVPdc84xL5E1mZkm8 * fix: unfillable means EITHER reason, and a select never names a relation target (IDEA-2899) Review round 2, two findings, both real and both about a rule stated in one place and enforced in another. **"Unfillable" answered for one of two reasons.** An EMPTY KEY cannot be supplied either — `--field =value` is rejected by this command's own parser, and the detailed render has explained that since Codex round 6. Only that render knew: the --dry-run summary and the returned error went on advising `--field` for those rows, because the predicate I extracted last commit covered the relation reason alone. A predicate named "unfillable" that answers for half its name is a worse trap than no predicate — right at the site that defined it, wrong everywhere it was reused, which is precisely what extracting it was meant to prevent. Two functions now: `itemCopyUnfillable` (either reason — advice), and `itemCopyUnavailableTarget` (the relation half — the render's own sentence, since the two explanations are not interchangeable to a reader). `itemCopyUnavailableTarget` deliberately does NOT also exclude empty keys, though my first version did. A row can carry both faults, and a mutant removing that exclusion survived every test — correctly, because all it changes is printing two sentences that are both TRUE about such a row. The guard was tidiness dressed as a rule; a condition nothing can distinguish is one the next reader has to re-derive. **`Collection` was emitted for non-relation fields**, while its own doc said it is empty for every other type. That was a claim about the schemas people write, not a property of the code: a `select` carrying `"collection": "people"` is storable — field validation has no use for the key and does not police it — and the value was copied straight through, so the CLI printed "target collection: people" beneath a select. A relation fact asserted about a field that has none. `relationTargetSlug` makes the documented contract true at the only place that can make it true; my own type-gate test had created exactly that shape and asserted only the FLAG, not the slug. Three mutants on these fixes, all killed. Claude-Session: https://claude.ai/code/session_01Xk9M5UVPdc84xL5E1mZkm8 * fix(cli): the explanation now names the reason that actually applies (IDEA-2899) Review round 3, and the sharpest miss of this unit — my own, one commit old. Broadening what a predicate ACTS on silently broadened what a sentence SAYS. Once `itemCopyUnfillable` counted empty keys as well as unavailable relation targets, a set of empty-key rows selected the all-unfillable branch and was explained as "the relation target is not available to you" — a false statement about rows that contain no relation at all. Same in the returned error, which is the line a script sees. The tell was there to be read: a sentence that was TRUE while the predicate was narrower is a sentence to re-read the moment it widens. I broadened the predicate deliberately, wrote a commit message about how a half-answering predicate is a trap, and left the sentence describing the half. `itemCopyUnfillableWhy` names the reasons actually present — relation targets, empty keys, or both — and the two one-sentence sites consult it. The detailed render is unchanged: it explains each reason where the row is printed, which is why it uses the narrower count. Four mutants, all killed, including the two that matter: the explanation always saying "relation" (the defect) and never saying it (the same defect pointing the other way). The test carries a mixed-reason leg, because a sentence that picks one of two true reasons is the failure a single-reason fixture cannot see. Also corrected: three comments claiming `itemCopyUnfillable` is relation-only or that the detailed render consults it. Both stopped being true last commit. Claude-Session: https://claude.ai/code/session_01Xk9M5UVPdc84xL5E1mZkm8 * fix: one row can carry both faults, and four docs said this was simpler than it is (IDEA-2899) Review round 4. Four findings, no P1s, and the first is the one worth the round. **A `continue` between the two counts.** `itemCopyUnfillableWhy` counted a row as an unavailable relation target and then skipped the empty-key check, so ONE row carrying both faults reported only the first. My mixed-case test used TWO rows with one fault each — a different input, and the only one it exercised. Two rows with one fault each and one row with two are not the same fixture, and I built the weaker one while writing a commit message about fixtures that cannot discriminate. **The dialog could still print `--field =value`.** `cliFillableField` excluded unavailable relation targets and not empty keys, so a required `json` field the destination reported with no key was type-shaped, blocked, and still offered a command the CLI's own parser rejects. The CLI has refused those since Codex round 6; the web side had never learned it. Same defect, other surface — which is the third time this unit has fixed one door and not its sibling. **Cardinality.** "no --field can supply it" for several fields, and "reported them with an empty key" for one. Both sites now agree with their counts, and the empty-key phrase is neutral on number so it reads correctly after either. **Four documents claimed every needs_value row is resolvable with an override** — the CLI renderer's docblock, the server's `NeedsValue` field, the CLI mirror type, and the dialog's collectability comment. That was true when each was written and this unit falsified all four; a reader following any of them would conclude the CLI had simply forgotten to print a flag. Two mutants on the fixes, both killed: the `continue` restored, and the dialog's empty-key exclusion removed. Claude-Session: https://claude.ai/code/session_01Xk9M5UVPdc84xL5E1mZkm8 * refactor(cli): one tally, because the rounds said the branching was the problem (IDEA-2899) Four review rounds returned 2, 2, 2 and 4 findings. The counts looked like slow convergence; the DISTRIBUTION was the finding. Every defect after round 1 lived in this one layer — how the CLI and the dialog say "here is how to supply it" — while the server half that computes availability stayed clean throughout. The layer had accreted exactly the way IDEA-2898's cold path did: a count, then a second count for the other reason, then a phrase function, then a `continue` between two counters that made a dual-fault row report half of itself. Round 4 fixed something round 3 introduced to fix something round 2 introduced. That is not a run of bad luck, it is a shape. So this round removes branches instead of adding a seventh guard. `itemCopyTally` walks the rows once and returns what every caller needs; `AllUnfillable()` is the condition both one-sentence sites test, and `Why()` is the phrase both interpolate. Three helpers become one type. There is no second definition of "unfillable" to drift from the first, and no sentence describing a subset of what a predicate counts, because the sentence and the count come from the same walk. `Unfillable` is deliberately NOT `UnavailableTarget + EmptyKey`: one row can carry both, and double-counting makes `Unfillable == Total` false for a set that is entirely unfillable — the comparison every caller makes. A mutant does the addition and dies. Five mutants, all killed. The last needed a new test rather than a new fixture: `AllUnfillable`'s `Total > 0` guard is unreachable from both current callers, so a mutant removing it survived every command-level test. Keeping an unreachable guard and calling it defence is how a promise becomes a lie, so the tally is now unit-tested directly — an empty set is not "entirely unfillable", and a future caller outside the `len() > 0` gate would otherwise be told silently that nothing can be supplied. Claude-Session: https://claude.ai/code/session_01Xk9M5UVPdc84xL5E1mZkm8 |
||
|
xarmian
|
00d650a861 |
feat(server,web): detect a revocation that writes no item, and evict the cache it left stale (IDEA-2898) (#1261)
* feat(server): fingerprint the caller's visible set on the item doors (IDEA-2898)
The delta stream can only express changes to ROWS. A revocation that writes
no item — the ordinary shape of revocation — therefore produces no signal at
all, and a client's warm local index goes on serving rows for a collection
the caller can no longer see. `ItemPicker` lists those titles.
`computeAccessEpoch` hashes the caller's EFFECTIVE visible set (collection
ids + item grants, canonicalised by sorting, separated by a byte no id can
contain) and `/items-index`, `/items-changes` and the 60s SSE tick all carry
it. The value is opaque, derived only from the caller's own access, and
costs no extra query — both lists are already resolved before the response
is written.
ONE DEFINITION, deliberately. The delta door computes its epoch from the
LIVE grant set rather than from its own include-deleted query set: a caller
holding a grant on a soft-deleted item would otherwise get a different epoch
from each door, forever, and the client would resync to its page cap on
every poll. `handlers_items_access_epoch_test.go` pins the two doors against
each other on exactly that fixture — a grant on a soft-deleted item is the
one input that discriminates, and an earlier version of this test agreed for
the wrong reason because its fixture had no grants at all.
The unrestricted caller gets a SENTINEL ("all") rather than a hash of the
empty set, because the empty set is a real and opposite state: a restricted
member with zero visible collections. Hashing both the same would make the
widest and narrowest access indistinguishable.
Claude-Session: https://claude.ai/code/session_01Xk9M5UVPdc84xL5E1mZkm8
* feat(web): compare the access fingerprint and evict the cache when it changes (IDEA-2898)
The client half of the signal. `WorkspaceState.accessEpoch` is the baseline
the cache was built under; it is persisted beside the cursor because the
revocation this closes can land while the tab is CLOSED, and a cache that
forgot its scope on reload could not detect that at all.
`ensureAccessScope` is the comparison, and it is ONE function with two call
sites — the page's `/items-changes` poll and bootstrap's own reconcile loop.
The first draft inlined it in the loop and had already diverged: the inline
copy skipped the null-baseline case, which is precisely the
offline-revocation cache the change exists for.
Three properties the tests pin, each of which was wrong at some point:
- ABSENCE IS NOT A VALUE. A server that does not send the field is a
mid-deploy older build, not a changed scope; and a snapshot carrying no
epoch must not erase a baseline we already know, which sent the reconcile
loop resyncing to its 50-page cap.
- THE RESYNC MUST TERMINATE. When the snapshot carries no epoch the
baseline would be unchanged and the next poll would ask again forever, so
the epoch we were TOLD goes in as the fallback — and again after the
await, for the case where the resync was JOINED rather than started and
the fallback was never seen.
- RAM AND DISK AGREE. `persistReplace` runs inside the resync, so the
baseline is set before it, not patched after: a durable meta row a
version behind the in-memory one makes the next warm boot resync for a
scope that never changed.
SCOPE, stated plainly because it is a real limitation and not a rounding
error: this closes the single-tab case and the offline case. A write from a
tab that has not yet learned the new scope can still reinsert a row into the
durable cache while the cache advertises the current epoch, and nothing
re-fires until the next access change. That is F2, and PLAN-2903 owns it
together with the cross-tab cache-coherence work. Strictly better than the
pre-change state, where no revocation without a row change was detected at
all.
`LOCAL_INDEX_SCHEMA_VERSION` 3 -> 4: a cache written before this has no
baseline, and adopting the incoming epoch for it would be exactly the silent
adopt the change exists to prevent. Two fixtures that hard-coded 3 now read
the constant, so the next bump does not turn them into stale-cache fixtures
by accident.
Claude-Session: https://claude.ai/code/session_01Xk9M5UVPdc84xL5E1mZkm8
* test(web): close the four seams a mutation run on the reduced tip found open (IDEA-2898)
The gates that ran on the full branch describe a different tree. A fresh
25-mutant run scoped to what this branch actually ships found four seams with
no coverage, three of them the same shape: the baseline being ADOPTED is
untested everywhere the adoption is silent.
That shape is why they survived a suite written for the eviction. Dropping an
adopt line does not stop a revocation being detected — the cache simply has no
baseline, and a null baseline over a populated cache resyncs, which LOOKS like
the change working. The discriminating case is the QUIET one: an unchanged
scope must cost nothing. All three new tests assert the absence of a resync.
- warm hydrate adopting the PERSISTED epoch (the offline-revocation case;
needs the mocked persistence module, since jsdom has no IndexedDB and the
warm branch is otherwise unreachable)
- the cold snapshot's epoch becoming the first-ever baseline
- `applyDelta` handing `persistDelta` the baseline its rows were applied
under, so the durable meta row is not stamped null after every delta
The fourth is the collection route's `ensureAccessScope` call, whose source
pin moved to PLAN-2903 with the pairing-guard argument it also covered. Half
of what it pinned still ships here, and the mutation run proved that half
uncovered — so the pin comes back NARROWED to the one surviving call site.
Its measured limit is in its docblock rather than assumed: deleting the call
kills it; short-circuiting the call (`if (false && await ...)`) SURVIVES,
because the text it matches is still on the line. A source pin cannot see
reachability. That survivor is reported, not hidden.
Instrument corrections made before any of this counted, both caught by the
runner's own controls rather than by inspection:
- `go vet` was in the Go build gate. It flags unreachable code, so the
positive control — an early `return` — scored BUILD-FAIL while compiling
perfectly. A vet failure is not a build failure.
- the web runs passed `--reporter=basic`, which this vitest does not have.
Every web mutant failed to START and the classifier read that as a
verdict. Fixed, and the classifier now requires the FULL baseline
population (50 tests across 7 files) to have run before it will call
anything killed — a mutant that stops a file loading also prints a
failing summary.
Claude-Session: https://claude.ai/code/session_01Xk9M5UVPdc84xL5E1mZkm8
* fix(web): an empty RAM state is not an empty cache, and a joined resync owes a durable epoch (IDEA-2898)
Two real defects from the review round on the reduced tip. Both are the
SILENT ADOPT class the change exists to prevent, arriving in the two places
the change itself created.
**A cache that has not answered yet is not an empty cache.** `bootstrap`
awaits `hydrate` before merging anything, but the SSE-driven `deltaSync` runs
on its own subscription rather than behind that await — so it can reach
`ensureAccessScope` with RAM empty and IDB holding rows from a scope nobody
has checked. The "nothing to evict, adopt silently" branch then stamps the
new epoch onto the durable cache through the delta that follows, and the
stale row hydrates under an epoch that agrees with the server FOREVER. Not
merely wrong once: permanently inert, which is worse than the defect this
change closes.
`cacheRead` makes the distinction the code was eliding. Declining to adopt
costs nothing and fails in the safe direction — the delta persists a null
epoch, and a null baseline over a populated cache resyncs on the next
reconcile.
**A joined resync updates RAM and leaves the disk behind.** Resyncs are
deduplicated per workspace. When `ensureAccessScope` JOINS one, that resync
already ran its own `persistReplace` under its own baseline, so assigning the
told epoch afterwards leaves the meta row recording the old one. The session
converges and every RELOAD hydrates the stale baseline and pays a full resync
for a scope that has not changed since. The code's own comment claimed the
epoch "lands in RAM and IDB together", which was true of the started path and
false of the joined one three lines below it.
`persistAccessEpoch` repairs just the epoch on an existing cache, on BOTH
branches — the first version of the fix had it on one, and no test in the
file could tell them apart until a mutant did.
Two tests changed because a PROPERTY changed, said out loud rather than
quietly rewritten: "adopts silently when there is no baseline and nothing
cached" held for a workspace that had never been bootstrapped, and no longer
does. It is now "…and the cache is known empty", and the unbootstrapped case
is its own test asserting the opposite. The `seedUnder` helper acquires its
baseline through a cold bootstrap, which is how a real session gets one
anyway.
Mutation matrix re-run whole on this tip: 31 real mutants, 30 killed. The
survivor is the page pin's short-circuit case, documented in its docblock.
Six of the mutants target these two fixes; one of them (`persistAccessEpoch`
minting a meta row) had to be rewritten after it survived for the wrong
reason — the naive version put a keyless row that IDB rejects, so the error
path compensated for the defect and the test never had to.
Claude-Session: https://claude.ai/code/session_01Xk9M5UVPdc84xL5E1mZkm8
* fix(web): a failed cache read is not an empty cache, and the epoch patch is a compare-and-set (IDEA-2898)
Round 2 on the reduced tip, and both findings are the previous round's fixes
being not quite finished.
**`hydrate` returns the same empty payload for a FAILURE as for an empty
cache** — deliberately, since a best-effort cache should not take the app
down. `cacheRead` was set unconditionally after the await, so a transient IDB
failure read as "there is nothing stored" and re-opened the exact silent
adopt round 1 closed: the durable cache may hold rows from a scope nobody
checked, and adopting stamps the new epoch onto them through the next delta.
`HydrateResult.durableRead` carries the difference the payload cannot. True
when the read succeeded, true when IndexedDB is unsupported (nothing durable
can contradict anything later), true when the read found an incompatible
cache and wiped it — false only when a database that might hold rows could
not be opened or read. Failing that way costs a resync and hides nothing.
**`persistAccessEpoch` was a blind read-modify-write on a row other writers
own.** IDB serializes transactions, but a `persistDelta` or `persistReplace`
carrying a NEWER epoch can commit between the resync this caller joined and
the patch — and the overwrite would then stamp the older epoch onto rows
fetched under the newer one, so the next comparison reports a change that
never happened and pays a full resync for it. Now a compare-and-set against
the epoch the caller believes it is repairing.
Both hydrate failure paths are covered, and they needed different fixtures:
a database at a HIGHER format version (open fails outright) and one whose
`items` store is missing (the open succeeds, the read throws). They are one
statement written twice, and a mutant on either is invisible to a test of the
other — which is how the second one was found, by a mutant surviving a test
written for the first.
The mutation matrix is 36 real mutants on this tip, 35 killed; the survivor
remains the page pin's short-circuit case, documented in that file. One
mutant is worth naming because it survived for the WRONG reason twice: the
naive removal of `persistAccessEpoch`'s "no meta row" guard cannot be
detected, because without it the code dereferences `undefined` and the catch
swallows the throw, so the outcome is identical. The faithful version — a
guard replaced by code that actually mints a valid row — is killed. A mutant
has to be the defect, not a crash that happens to look like it.
Also corrected: `accessEpoch`'s doc claimed null exists only before the first
response of a session. Two things falsify that now — a server that sends no
epoch, and the pre-hydration guard.
Claude-Session: https://claude.ai/code/session_01Xk9M5UVPdc84xL5E1mZkm8
* docs,test(web): the epoch patch's expected value is asserted, and null is not unreachable (IDEA-2898)
Round 3, two P3s and no behavioural findings.
The `HydrateResult.accessEpoch` doc said the version bump to 4 made a null
epoch unreachable for caches this build writes. It does not: a server that
predates `access_epoch` omits it, and `persistDelta`/`persistReplace` record
that absence honestly rather than inventing a value — a case the tests
already cover. What the bump actually rules out is a PRE-IDEA-2898 cache
being READ as though it had a baseline. Comment corrected to say the thing
that is true.
The two joined-resync tests asserted the epoch `persistAccessEpoch` is given
but not the value it expects to be REPLACING. The patch is a compare-and-set,
so a wrong `expectedPrevious` makes it a silent no-op that still looks right
in RAM, and the disagreement surfaces only as a resync on the next reload —
invisible to a single-session test. Both call sites now have their expected
value asserted, and two mutants (each call site given the wrong expectation)
are killed by them.
Claude-Session: https://claude.ai/code/session_01Xk9M5UVPdc84xL5E1mZkm8
|
||
|
xarmian
|
6f8b6eb958 |
perf(store): clone Postgres test databases from a migrated template (TASK-2900) (#1260)
`storetest.NewPostgres` and its in-package twin `testStorePostgres` ran the
full 87-file migration chain for EVERY test that asked for a store. The schema
is identical for every one of them, so building it per test bought nothing.
Measured on this box, at
|
||
|
xarmian
|
8910ad0679 |
fix(store): mint the imported workspace inside the import transaction (BUG-2892) (#1259)
ImportWorkspace called CreateWorkspace — its own committed write — before opening the transaction that carries every other row. Eight error returns sit between that INSERT and the commit (begin, de-duplicate declarations, import collection, item slug-after-truncation, import item, remap item, import comment, and the commit itself), and each returned an error while leaving the workspace row behind: named, slugged, owned by the caller, holding no collections and no items. The husk was not only clutter. uniqueWorkspaceSlug probes `WHERE slug = ? AND deleted_at IS NULL`, and a husk is not soft-deleted, so it kept the slug: an operator who fixed the bundle and retried landed on `name-2`. Measured on the unfixed build — the retry leg returns `retry-slug-2`. The attempt that stored nothing took the name from the one that worked. CreateWorkspace is now a thin wrapper over createWorkspaceQ, which takes the caller's executor, and ImportWorkspace opens its transaction first and mints on it. Both of createWorkspaceQ's reads — the slug probe and the read-back — take that executor too, which is load-bearing rather than tidy: a read routed through the pool while the caller's transaction holds its connection can wait for a free one, and under MaxOpenConns(1) there is none (BUG-2778 / BUG-2409). workspaces.slug has been globally UNIQUE since 001_initial, so the constraint still covers the race the in-transaction probe cannot. The filing said this needed a look rather than a one-liner because CreateWorkspace "does more than one INSERT (owner membership, seeding hooks)". That premise was wrong, and reading the function is what retired it: it does one INSERT. Owner membership and template seeding are both handler-level, and neither is on the import path inside the store — handleImportWorkspace calls AddWorkspaceMember after ImportWorkspace returns, and import never seeds because the bundle carries the collections. Tests: two legs, both red on the unfixed build for the reasons named above and green here. The negative fixture is a bundle whose two collections share a slug and differ in everything else, so it fails for exactly one reason, asserted on by message. CONVE-23 sweep: two comments said the workspace "survives as a husk" when the de-duplication rolls an import back (export.go's trait-dedupe note and TestImportDeduplicatesAConflictingArchive). Both were true when written and are now false in that clause only; corrected without disturbing the argument they were making, which is unchanged. CONVE-18 sweep: the class is a committed pool write followed by a transaction whose failure orphans it. One instance across the 64 non-test files of internal/store — this one. The one other hit is a false positive (CreateItemLink's early `return s.SetParentLink(...)`, a mutually exclusive path). Search boundary: internal/store non-test files, by syntax; the instrument recognises `s.db.Exec(` and s.<write-verb> calls, does not model control flow, and does not follow writes into helpers named otherwise. It was run against the pre-fix export.go as a positive control and fired on exactly the defect. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR |
||
|
xarmian
|
07b2e439f2 |
TASK-2869 (U2b): the preflight names a relation's target collection, and the copy dialog scopes its picker to the destination (#1258)
* feat: the preflight's needs-value row names its relation target, and the copy dialog scopes its picker to the destination (TASK-2869)
U2b, per the day-55 ruling. Both blockers (U1 referent validation, U2 the
FieldEditor branch + picker) are in.
THE DEFECT. `ItemCopyPreflightNeedsValue` carried `type: "relation"` and no
target. FieldEditor gates its relation branch on `wsSlug` AND
`field.collection`, so a required relation in the destination reached the copy
dialog as a field it knew was a relation with no idea what to point at, and
rendered as FREE TEXT. Before U1 the copy stored whatever was typed.
SERVER. `collection` is added to the needs-value row, populated from the
DESTINATION schema's `def.Collection` — the only place it is known, since the
row is built from that schema. Additive and `omitempty`: a client that does not
read it is unaffected, and a row for a non-relation field is byte-identical to
before. Not a wire-version question, for the same reason
`models.ItemWriteWarnings` was not.
CLIENT. `toFieldDef` carries the collection through, and the FieldEditor call
passes `wsSlug={destWs}` — the DESTINATION, never the source. A relation
resolves at the destination, so the picker must list items the copy can
actually point at; that is same-workspace resolution AT the destination, not
the cross-workspace case PLAN-2857 rules out.
`relation` becomes collectable ONLY IF THE ROW NAMES ITS TARGET. Without a
collection, FieldEditor's gate renders the non-editable state, so offering the
row would produce a control that cannot be filled and a Confirm that cannot be
satisfied. Such a row now lands in the blocked list and the user is told which
field and why — the same disposition `multi_select` gets, for the same reason:
a control that silently cannot do its job is worse than an honest refusal.
TWO THINGS THIS UNIT TAUGHT ME THAT ARE NOT IN THE RULING.
1. THE WEB MUTANT SURVIVED, AND THAT IS WHY `isCollectable` MOVED. My first
version left the predicate inline in `CopyItemDialog.svelte`. Making
`relation` unconditionally collectable — the exact defect the negative leg
of the proving test is about — passed EVERY suite in the repo. That is
IDEA-2894's lesson arriving one unit later in the same file, so the
predicate now lives in `$lib/items/copyNeedsValue` with tests. Two mutants
die there: relation-always-collectable, and `multi_select` slipped into the
collectable set.
The Go half was pinned from the start (drop `Collection: def.Collection` ->
FAIL naming the empty value and the expected slug). Only the client half was
unpinned, and only because of where the code lived.
2. A U2-ERA TEST ASSERTED THE ABSENCE THIS UNIT CLOSES, and asserted it
CORRECTLY. `fieldEditorRelationCallers.test.ts` required that the dialog
pass no `wsSlug` and build its FieldDef from a shape with no `collection` —
which was the behaviour, and withholding `wsSlug` was what kept an unscoped
picker out. It also named this task by ref and told its successor to revisit
the gate WITH the change rather than let it drift. Inverted here: the block
now asserts the destination slug is passed, that the collection reaches the
FieldDef, and — the half that is easy to lose — that the dialog still
DELEGATES the collectability decision, so a future inlined predicate would
pass the unit tests and fail this.
Worth keeping: a test that pins a temporary absence should name what would
make it wrong. This one did, and that is the only reason its inversion was a
five-minute job instead of an argument about whether it was load-bearing.
Gates: `internal/server` ok 158.335s, `go vet` and `gofmt` clean,
`npm run check` 0 errors (6 pre-existing warnings), `make web-test` 127 files /
2123 tests. Postgres and CI are owed on this tip.
Claude-Session: https://claude.ai/code/session_01Xk9M5UVPdc84xL5E1mZkm8
* fix(cli): mirror the needs-value collection, and name the relation target in the CLI (TASK-2869)
TWO CONSUMERS I DID NOT SWEEP. The previous commit added `collection` to the
preflight's needs-value row and updated the server struct and the TypeScript
type — "both sides", as its own message put it. There are THREE sides.
`internal/cli` keeps a mirror of the preflight shape and
`TestItemCopyMirrorsMatchServerShapes` requires it to match the server field
for field. It failed in the Postgres gate, in a package the change did not
touch.
That is the third time this session a producer change broke a consumer I had
not enumerated, and the shape is always the same: I name the surfaces I edited
and call that the population. The instrument that would have caught it is not
"run more tests" but "grep for the type's name before claiming the sweep is
done" — `ItemCopyPreflightNeedsValue` appears in exactly three files and I
looked at two.
Mirrored, with a comment saying why it exists: a mirror that silently lags is a
mirror that lies, and the CLI renders these rows.
AND THE CLI NOW NAMES THE TARGET COLLECTION, which is the point of the unit on
the surface that has no picker at all. A row reading
owner_ref (Owner, relation) required — required, with no value…
tells a user a value is needed and nothing about what kind of value exists.
The dialog answers that with a scoped picker; the CLI had no answer. It now
prints the relation analogue of the `options:` line a select already gets:
owner_ref (Owner, relation) required — …
target collection: people
Test asserts the line appears for the relation row, appears EXACTLY ONCE with a
select row rendered alongside — so it cannot pass by printing unconditionally —
and that the select's own `options:` line still renders, so this did not
displace it.
Gates: `internal/cli` and `cmd/pad` green, build and gofmt clean. The full
Postgres run and CI are owed on this tip; the earlier PG run is the one that
caught the mirror and is superseded.
Claude-Session: https://claude.ai/code/session_01Xk9M5UVPdc84xL5E1mZkm8
* fix(web): the relation picker searches the preflight's canonical destination slug (TASK-2869)
Codex review, finding 3 of 3, and the only one of the three that belongs in
this unit.
`wsSlug={destWs}` handed `FieldEditor` a value that is NOT always a workspace
slug. An item can be opened through a workspace-UUID URL, the route parameter
is passed straight through as `sourceWsSlug`, and a same-workspace copy then
puts that UUID in `destWs`. `/search` resolves a workspace by SLUG only, so a
picker handed a UUID searches nothing and returns no results — a control that
looks usable, is not, and says nothing about why.
Now `pickerWsSlug`, which is the preflight response's own
`destination.workspace_slug`. The preflight IS the canonicalising round-trip:
the server resolved whatever it was given and answered with the real slug.
Falls back to `destWs` only before the first preflight returns, at which point
no needs-value row is rendered anyway.
The caller test asserts the prop AND the derivation, because asserting only the
prop would pass against a `pickerWsSlug` that was just `destWs` renamed.
THE OTHER TWO FINDINGS ARE REAL AND ARE FILED, NOT FIXED HERE.
IDEA-2898 — `ItemPicker` serves warm local-index results without re-authorising
them, so a collection whose access was revoked can still be listed. The cold
`/search` path is visibility-filtered and correct; the warm path is not. This
is PRE-EXISTING and applies to every caller of the picker, `ItemDetail`
included — last touched by TASK-2877, not by this unit. U2b widened the
exposure by adding a caller; it did not create the defect, and rewriting the
picker's cache-authorisation model inside a feature branch would be an
unrelated change riding along. The fix needs a decision about where the client
learns its access set from, which no current signal provides.
IDEA-2899 — a relation row can name a target collection that is DELETED or
UNREADABLE, so `isCollectable` says yes on a non-empty string and the user
meets a picker with nothing in it. I could not fix this correctly here, and the
reason is measured rather than assumed: the obvious test is to check the target
against `destCollections`, and that list is filtered by `canEditCollection` —
it is what the user may copy INTO. A relation TARGET needs only READ access, so
a perfectly usable target routinely is not in it. Using it would OVER-BLOCK,
refusing rows the user could have filled, which is a worse failure than the one
being fixed and invisible to whoever hits it. The right shape is probably the
server reporting the target's availability on the row it already builds — an
additive field on the same row this unit just changed, worth doing deliberately
rather than bolted on at the end of a branch.
Gates: `npm run check` 0 errors (6 pre-existing warnings), `make web-test` 127
files / 2123 tests. Postgres is running on this tip; CI is owed.
Claude-Session: https://claude.ai/code/session_01Xk9M5UVPdc84xL5E1mZkm8
|
||
|
xarmian
|
47d2b15e10 |
feat(store,server): make collection trait uniqueness a database invariant — de-dup pass + partial unique indexes (TASK-2710) (#1257)
* feat(store): make one-collection-per-trait a database invariant (TASK-2710) Partial unique indexes on both drivers over the artifact_kind and invocation_field declarations, with the de-duplication pass that has to precede them. The de-dup is Go, not SQL, and runs BEFORE migrate(). The ruling requires every resolution to be REPORTED, because it silently changes which collection owns a kernel behaviour, and a SQL migration cannot log — Postgres RAISE NOTICE goes nowhere here and SQLite has no equivalent. Splitting decide-in-SQL from report-in-Go would give one rule two spellings to keep in step, which is the defect class IDEA-2883 closed. It runs before migrate() because the CREATE statements fail on exactly the databases needing repair. The rule, after three proposals each retired by a measurement: most user-written items wins; ties break on lowest (created_at, id), reported as ARBITRARY rather than as age, because created_at is second-resolution and newID() is a random uuid v4, so same-second rows carry no age at all. The loser keeps every item and loses only the declaration. Routing MAY change on affected deployments; there is no current behaviour to preserve, since with two declarations live the winner was measured flipping between runs on Postgres. SeedCollectionsFromTemplate now skips a definition whose artifact kind is already declared. Renaming re-slugs, so a workspace whose conventions became house-rules looked slug-empty while its kind was still claimed; seeding used to mint the duplicate and would now fail the whole seed instead. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR * test(server): keep the shadowing tests under the new trait invariant (TASK-2710) TestResolvePlaybookIgnoresInvisibleCollections and TestCollectionIDForKindIgnoresInvisibleCollections build two collections declaring one trait, which the partial unique indexes now forbid. They are not obsolete and I did not weaken them. They guard the round-2 shadowing fix: when two collections declare one kind and the first-sorting one is invisible to the caller, resolution must return the visible one instead of failing. TASK-2710 makes that state unrepresentable going forward and repairs it at startup on databases holding it, but the resolver is what stands between a legacy database and a wrong answer in the window before that repair, and on any deployment where an operator dropped the index. So the fixture now constructs the state the way it exists in the wild — with the constraint suspended — and the assertions are untouched. Same reasoning as IDEA-2883's disagreeing-reminder fixture. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR * test(store): SuspendTraitUniquenessForTesting beside its neighbour, with a restore (TASK-2710) Lead ruling: shape it like SetBcryptCostForTesting — same file, ForTesting suffix, returns a restore the caller defers, so a suspended constraint cannot outlive the test that suspended it. The restore recreates the indexes from the SHIPPED migration text rather than a hand-copied approximation, which would drift and then attest to an index the product does not have. Its failure is information, not noise: recreating a unique index while a duplicate is live is exactly what the migration would hit. The de-dup tests therefore assert the restore SUCCEEDS, which is the migration's precondition checked rather than assumed; the server shadowing tests leave the duplicate live for their whole duration and ignore the error explicitly rather than by omission. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR * fix(store): de-dup uses the index's own extraction, strips every declaration once, counts NULL source as user-written (TASK-2710) Three P1s from codex round 1, each verified before accepting. A collection can lose BOTH declarations — the playbooks definition declares artifact_kind and invocation_field — and resolving them in two passes, each re-parsing the row's ORIGINAL traits, made the second write restore what the first stripped. The duplicate survived and the migration would still have failed on it, surfacing as a broken upgrade rather than a test. Now one write per collection accumulating every strip, with a regression test. Detection asked Go what a declaration is while the indexes ask json_extract / ->>, so the two could disagree: a row the Go parser rejects still carries a value the index sees, and the de-dup would leave a pair the CREATE then refuses. It now asks the database the same question the index asks, which is the same one-rule-one-spelling reasoning that put the report in Go. source IS NULL now counts as user-written. The column is nullable and legacy rows predate it; 'source <> template' alone is NULL for those, which SQL treats as not-true, so a workspace whose only user content is old would have had it ignored when picking the winner. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR * fix(server,store): seed skips either held declaration; unique violations map to 409 on both drivers (TASK-2710) Two P2s from codex round 1. The seeder checked only artifact_kind, but the playbooks definition also declares invocation_field and TASK-2710 adds an index for each — so a workspace whose invocation-routing collection had been renamed would still have failed its seed. It now skips when EITHER declaration is held, and says which. Collection create recognised only SQLite's "UNIQUE constraint" text, so the identical race answered 409 on SQLite and 500 on Postgres; the update path recognised neither. Both now use one named isUniqueViolation covering both drivers, matching what every item handler already did. The conflict message also distinguishes the indexes: "a collection with this name already exists" is actively misleading for a trait conflict, where the name is fine and the declaration is taken — a user told to rename would rename forever. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR * chore(store): drop the unused order slice from the de-dup pass (TASK-2710) Leftover scaffolding from the one-write-per-collection rewrite; staticcheck caught it (SA4010). Mine to catch earlier — I ran lint at the tip BEFORE that rewrite and not after it, so the gate found what a re-run would have. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR * fix(store): de-duplicate a conflicting archive on import; guard json_extract against malformed traits (TASK-2710) Both from codex round 2, both real. Import warned about a duplicate declaration and inserted both, which was right while nothing forbade the pair. With the unique indexes the second INSERT is refused, the whole transaction rolls back, and the workspace minted beforehand survives as a husk (BUG-2892) — so an archive carrying a duplicate would become unimportable, and those archives are exactly the ones this release repairs. This is the task's item 4, which I had not done. The first declaring collection in bundle order keeps it and later ones are stripped and reported; the rule cannot use user-item counts here because items are inserted after collections, so bundle order IS the terminator and the log says so. SQLite's json_extract RAISES on malformed JSON rather than returning NULL, so the unguarded expressions in the index predicates and the de-dup scan would have failed STARTUP on any database holding one bad blob. json_valid now guards both, matching what every other reader does with malformed traits — treat the row as declaring nothing. Postgres needs no equivalent: traits is JSONB, so the column type makes malformed content unrepresentable at rest. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR * test(store): assert the malformed-traits asymmetry per driver (TASK-2710) My own Postgres gate caught this: the test planted a malformed traits blob and Postgres refused it — invalid input syntax for type json — because traits is JSONB there. That refusal IS the reason migration 064 carries no json_valid guard while 087 does, and it was prose in the migration until the gate turned it into an observation. The test now asserts it per driver: on Postgres the plant must be REFUSED, on SQLite it must succeed and the guard must keep both the de-dup pass and index creation working. It therefore also catches someone 'fixing' the asymmetry later — adding a guard Postgres does not need, or dropping the one SQLite does. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR * fix(store,server): de-duplicate on the bytes being written; stop claiming a name conflict for item-index violations (TASK-2710) Round 3, two findings. P1, a regression I introduced: the import de-duplication pre-computed its strips from the traits the BUNDLE carries, which is not what gets written — coercion, validation-discard and canonical inference all run afterwards. A pre-traits archive carrying conventions with an empty blob has its declaration INFERRED from the slug (BUG-2702), so a bundle pairing that with an explicit declarer showed the pre-pass one declaration and the database two, and the index aborted the whole import. The check now runs immediately before the INSERT, on the final bytes, which turns the question from 'what did the file say' into 'what am I about to write'. Reproduced first, then fixed. P2: a collection UPDATE can migrate item field values, and an item-level unique index can fail there — invocation_slug is the live example. My catch-all reported that as 'a collection with this name already exists', sending the caller to rename something that is not the problem. The name message now requires the error to name the collections table; otherwise it says what it knows. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR * test(store): fail when an archived collection takes the live one's declaration (TASK-2710) The test for the rebase onto BUG-2884, written before the fix and failing against the naive resolution: live collections declaring convention = [], want exactly [conventions] BUG-2884 made the bundle carry soft-deleted collections. This branch moved import's duplicate-declaration check out of a pre-pass and into the insert loop, so it operates on the bytes actually being written (round 3's P1) — but `dropDuplicateImportDeclarations` has no notion of liveness, which the pre-pass had gained on main. An archived collection travelling ahead of the live one that replaced it therefore CLAIMS the kind, and the live collection is stripped of it. Every resolver filters deleted_at IS NULL, so the workspace imports with no live convention routing at all. The second assertion is the other direction: the archived collection must KEEP its declaration. Nothing routes to it, both partial unique indexes exclude it, and stripping it would edit data the operator archived rather than deleted. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR * fix(store): an archived collection neither takes nor loses a trait declaration on import (TASK-2710) The rebase fix for the test in the previous commit. dropDuplicateImportDeclarations now returns an archived collection's traits untouched. A soft-deleted row sits outside both partial unique indexes (each carries `AND deleted_at IS NULL`) and outside every trait resolver, so it can neither create the conflict this function prevents nor be harmed by holding a stale declaration. Letting it take a claim was the real damage: the live collection later in the bundle lost the declaration and the workspace imported with no routing for that kind at all. BUG-2884's pre-pass had grown the same condition; this branch replaced that pre-pass with an in-loop check on the final bytes (round 3's P1) and the condition did not come with it. Keeping both is what the rebase owes. TestImportRoutingIgnoresSoftDeletedCollections builds its fixture in a new order — declare, archive, then seed — because the unique index refuses two LIVE collections declaring one kind. The order is not a workaround: it is the production path that mints this state (delete the conventions collection, seed again), every step legal under the invariant, and it needs no test-only suspension of the constraint. Its assertions are unchanged. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR * docs(server): checkTraitConflicts no longer claims the index it now has (TASK-2710) CONVE-23 sweep. The doc comment describing that gate was written when the invariant did not exist and this branch falsified three of its sentences: "workspace IMPORT bypasses it entirely by design" (import de-duplicates on the way in now), "the database-level version is deliberately NOT added in phase 0" (migration 087 adds it), and the closing paragraph handing duplicates back to the resolvers' order-dependent behaviour. Rewritten to say what the division of labour actually is — the pre-check survives for the MESSAGE, because a unique violation is a 409 about a name unless something tells the handler otherwise and "rename your collection" is useless when the name is fine and the declaration is taken; the index is what holds. It also states the two things the invariant genuinely does not cover: import (which de-duplicates rather than refusing a restore) and archived collections (outside both indexes and every resolver, so a soft-deleted row may hold a declaration a live one also holds). Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR * fix(store): take the pre-migration snapshot before the trait repair writes (TASK-2710) Codex round 5, P2, verified. dedupeTraitDeclarations ran from the two constructors, ahead of migrate() — and snapshotBeforeMigrate() runs INSIDE migrate(). The repair changes data: it strips a declaration, moving which collection owns a kernel behavior. So the altered ownership was already committed when the snapshot was copied, and `<db>.pre-<version>` — the operator's rollback for a bad upgrade — contained it. Restoring after a failed migration handed back the old schema with the repair applied and unrecorded: the one thing the rollback could not undo was the only thing that had silently changed routing. The call moves into migrate(), immediately after the snapshot and before the migration loop, which satisfies both constraints at once — 087 / 064 still cannot run against a database holding duplicates, and the snapshot now precedes the write. The Postgres path takes the same position for symmetry; there is no snapshot there to sit after, so the ordering argument is one-sided on that dialect and the comment says so. TestTheSnapshotIsTakenBeforeTheRepairWrites pins it end to end: plant a duplicate, un-apply 087 so a migration is genuinely pending, reopen, then read the snapshot with the RAW driver — New() would migrate and repair the snapshot too, destroying the thing being measured — and assert it still holds both declarations while the live database holds one. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR |
||
|
xarmian
|
d6771abeea |
IDEA-2886: state the drop-vs-refuse rule once, and route all three sites through it (#1256)
* refactor(store): state drop-vs-refuse once, and route all three sites through it (IDEA-2886)
The rule — a relation value the CALLER ASSERTED is refused if it does not
resolve; one asserted by NOBODY is dropped and reported — was stated three
times, each in its own control flow and its own prose:
- the origin split in MigrateRelationReferentsQ (supplied -> refusals, the
other two -> dropped);
- ResolveLateRelationDefaultsQ, whose return is named `dropped` because
everything it sees is a default;
- IssuesForCallerInput, which kept an issue if its key was present before
validation.
Three statements of one rule is the shape that invites a fourth door to state
it slightly differently, which is the drift TASK-2878 existed to remove. I
filed this out of that unit after codex round 10 stated the rule for the third
time.
The polarity now lives in ONE place, `RelationOrigin.Refuses()`, and the three
sites consult it:
- the supplied branch asks the rule instead of assuming its own identity.
Its `else` is unreachable while Refuses() answers as it does, and that IS
the point: if the rule changes, this site follows it rather than
contradicting it.
- the late-default pass opens with an assertion that its origin does not
refuse, so an edit that starts refusing there fails loudly instead of
quietly forking the rule. Its prose now says "this is
RelationOriginDestinationDefault.Refuses() being false", not "this is a
decision this function makes".
- IssuesForCallerInput CLASSIFIES to an origin and then asks, rather than
restating the rule as "keep it if it was there before".
BEHAVIOUR IS UNCHANGED AT EVERY DOOR, which is the spec and not a caveat.
WHAT THE MUTANTS CAN AND CANNOT SHOW, stated plainly because a behaviour-
preserving refactor is exactly where a mutation matrix lies:
- PER-SITE mutants are impossible BY CONSTRUCTION. Restoring any one site's
old form is behaviour-identical, so no test can distinguish it. Reporting
"3/3 survived" would be true and would mean nothing.
- The MASTER mutant is the real one. Flipping Refuses() to `!=` compiles and
kills 116 server subtests and 1 store subtest, so the rule is load-bearing
rather than decorative.
- PER-SITE REACHABILITY is the instrument that answers the question the
mutants cannot: does each site actually consult the rule, or keep a private
copy? Making Refuses() panic and running one test per site:
TestRelationDoors_Move -> panics
TestRelationDoors_Create -> panics
TestRelationDoors_MoveDropsInvisibleDestinationDefault -> panics
TestRelationDoors_LateDefaultWrongCollectionDoesNotDisclose -> panics
TestRelationDoors_RequiredInvisibleDefaultRefuses -> panics
My first run of that probe reported ZERO panics for the late-default site
and I nearly wrote it up as unreached. The filter I used
(`-run TestResolveLateRelationDefaults`) matched NO TESTS — the reading was
my instrument, not the code. I have the `-run`-narrower-than-the-population
trap in my own notes from a previous unit and walked into it again, so the
re-run above counts `=== RUN` lines alongside the panics: an instrument
that cannot say how many tests it ran cannot tell silence from absence.
Gates: `internal/store` ok 188.174s, `internal/server` ok 206.077s (SQLite),
`gofmt` and `go vet` clean. Postgres and CI are owed on this tip.
Claude-Session: https://claude.ai/code/session_01Xk9M5UVPdc84xL5E1mZkm8
* fix(store): make the supplied write-back survivor-guarded, as its sibling already is
Review finding on the previous commit, and the finding is exactly the one I
asked the round to look for: the new `else` in the supplied path is
UNREACHABLE, and it was WRONG.
It deleted each unresolvable key from `fieldMap` and then fell into a
write-back loop that copied every entry of `suppliedRelations` back in —
restoring the value it had just dropped. The result would report a drop while
retaining the dropped value, which is worse than either outcome alone. An
unreachable branch that is wrong is a trap for whoever makes it reachable, and
the whole reason that branch exists is that someone might.
FIXED BY CONSTRUCTION RATHER THAN BY BOOKKEEPING. My first fix deleted the key
from `suppliedRelations` as well, which works and leaves two loops that have to
agree with each other. The write-back is now guarded on survival —
if _, survived := fieldMap[k]; !survived { continue }
— which is character-for-character what the DESTINATION-DEFAULT branch twenty
lines below already does, and for the same reason its comment gives. The two
branches differ in their disposition; they no longer differ in their
write-back. When I notice I am writing a second loop to compensate for the
first, the compensation is usually the sign that the structure is wrong.
BEHAVIOUR IS UNCHANGED ON THE REACHABLE PATH, and here is why rather than an
assertion that it is: in the refusing path nothing is deleted from `fieldMap`,
`suppliedRelations` was built from `fieldMap`'s own keys, and
`ResolveRelationReferentsQ` canonicalises the copy it is handed without
touching `fieldMap` — so every supplied key survives the guard and the loop
does what it did before.
WHY THERE IS NO TEST FOR THE BRANCH, named rather than left looking covered:
exercising it requires inverting `Refuses()`, which inverts the rule for every
door at once and fails 116 subtests for unrelated reasons. A probe that cannot
isolate what it is probing proves nothing, so I did not keep the one I tried.
The reviewer's exhibit — schema `{Key:"r", Type:"relation", Collection:""}`,
`fieldMap={"r":"bad"}`, `supplied={"r":"bad"}` — is recorded here instead, and
the structural fix means the branch is now correct without needing to be run.
Gates on this tip: `internal/store` ok 183.002s, `internal/server` ok 160.804s,
`gofmt` clean. Postgres was green on the parent (29 packages, EXIT=0) and is
owed again here.
Claude-Session: https://claude.ai/code/session_01Xk9M5UVPdc84xL5E1mZkm8
|
||
|
xarmian
|
47cc106ab8 |
IDEA-2893 + IDEA-2894: record the accepted carry disclosure, and make the drop-reason mapper testable (#1254)
* refactor(web): extract the copy dialog's drop-reason mapper so it can be tested (IDEA-2894)
The mapping from a server drop reason to the sentence a user reads lived inline
in `CopyItemDialog.svelte`, unexported, with no test file for the component at
all. Two separate review rounds found defects in it and NEITHER FIX WAS PINNED
BY ANYTHING:
- round 12: the UI asserted NON-EXISTENCE from `not_found`, which the server
also emits for a target the caller merely cannot see. Telling those apart
is the existence oracle the collapse exists to prevent.
- round 18: `referent_not_portable` read "it points at something in the
source workspace", claiming both existence and location for a reason
emitted WITHOUT resolving the target — and which `github_pr` reaches too,
where the referent is in no workspace at all.
A third defect of the same shape would have been found the same way, by a
reviewer happening to read it, or not at all.
Moved to `$lib/items/copyDropReasons` with the reason vocabulary as an explicit
exported list, which makes two tests possible that could not be written before:
1. EVERY reason the server can emit has a sentence. This is round 12's other
finding as a test: BUG-2674 added `referent_not_portable` server-side,
nothing here learned it, and it rendered through the fallback as a raw enum
string in front of a user. A reason that maps to itself IS that defect.
Mutant: delete `referent_not_portable`'s message -> FAIL.
2. THE TWO HAZARDOUS REASONS STAY NEUTRAL. `not_found` and
`referent_not_portable` must not claim a target exists, does not exist, or
say where it is. Asserted over those two rather than all ten:
`wrong_collection` legitimately says the target is outside the field's
collection, and it may, because the server only emits it to a caller who
can SEE the target.
Mutant: restore round 18's wording -> FAIL, naming the sentence and why.
The unknown-reason fallback returns the raw string, and a third test pins that
deliberately: a reason this build has never heard of means the server is ahead
of the client, and showing the enum is more honest than inventing a sentence or
hiding the row. It is also what keeps test 1 from being vacuous.
WHAT THIS DOES NOT FIX, stated because the list is the thing a future reader
will trust: the reason vocabulary is DUPLICATED from Go (five constants in
`handlers_items_copy_preflight.go`, five in `internal/store/relation_referents.go`)
rather than generated, so it can still go stale in the one direction that
matters — a reason added to Go and not added here. The test cannot see that.
What it can see is a reason listed here without a sentence, and the list is now
the single place to update. Generating it from the Go constants would close the
gap properly and is a bigger change than this one.
Gates: `npm run check` 0 errors (6 pre-existing warnings in unrelated files),
`make web-test` 126 files / 2118 tests passed. Frontend-only; no Go touched.
Claude-Session: https://claude.ai/code/session_01Xk9M5UVPdc84xL5E1mZkm8
* docs(store): record why a carried relation's survival is observable and accepted (IDEA-2893)
Comment only; no behaviour change.
A carried relation value naming a live item in a collection the mover cannot
see resolves and survives a same-workspace move, while one naming nothing is
dropped — so a mover can tell those apart, and on a stored REF they also learn
the target's canonical id. The lead's ruling is ACCEPT AND DOCUMENT, and this
is the documentation, placed at the branch that produces the behaviour rather
than in a doc nobody reading that code will open.
Four measurements decided it, and the comment carries the two that matter so a
future reviewer reaches the reasoning instead of re-deriving it:
- NOT ENUMERABLE. A caller cannot choose what to test: create, update and
fields_patch all refuse a caller-supplied ref naming an item they cannot
see, with the COLLAPSED `not_found` wording, so no door turns a chosen
value into a carried one. It can only confirm a value already sitting in
an item the caller can read and did not put there.
- THEY ALREADY HAVE THE VALUE. An ordinary GET returns the raw stored
relation value verbatim; reads apply no redaction. The increment is "it
currently resolves" plus the ref-to-id mapping — not the target's
existence in any general sense, not its title, and backlinks do not widen
it either.
The comment also records why each candidate fix costs more than the increment,
because "we considered it" is worth nothing to a successor without the reasons:
redacting the response closes nothing (the id is in the blob and comes back
from a plain GET); not canonicalising removes only the id half and makes a
relation value stop meaning one thing everywhere; dropping by the MOVER's
visibility silently destroys a valid relation because of who moved the item;
and canonicalising only for movers who can see the target would make the
STORED BYTES depend on who performed the move.
The last paragraph is the one I most want read. The ONE change that would close
this is carrying unresolvable values verbatim instead of dropping them — which
is precisely the drop-and-report rule three lines below, the rule #1246 exists
to enforce. So the comment doubles as a warning: changing that rule changes
this, in the other direction. A design decision that is load-bearing for a
second decision should say so at the place where someone would change it.
I also WITHDREW my own earlier proposal rather than leaving it standing:
"evaluate the carry against the ITEM's access rather than the mover's" was
confused, because an item has no access identity and resolving without a
requester IS the status quo. A plausible-sounding option left in a trail is a
successor's wasted afternoon.
Gates: `gofmt` clean, `go build ./internal/store/`, `go vet ./internal/store/`,
`go test ./internal/store -run 'TestRelation|TestMigrateRelation'` ok. Scoped
to the package because this worktree has no `web/build` for the root embed, and
the change is a comment in one file.
Claude-Session: https://claude.ai/code/session_01Xk9M5UVPdc84xL5E1mZkm8
* fix(test): repoint the drop-reason parity gate at the extracted module (IDEA-2894)
`f2c4a722` moved the drop-reason mapper out of `CopyItemDialog.svelte` and
broke `TestCopyPreflightDropReasonsAreRenderedByTheDialog`, which reads the
component for that function. CI caught it; my local gates did not, because this
worktree had no `web/build` for the root embed so I had scoped the Go run to
`internal/store` — and I stated that boundary in the commit message while it
was hiding a real failure. Naming a gate's scope is not the same as the scope
being adequate.
THE GATE FAILED THE RIGHT WAY, and that is worth recording. It does not search
the file for `case 'not_found':` and shrug when the file changes; it looks for
the declaration by name and calls `t.Fatalf` if it is gone, saying "this gate
is reading for a function that moved or was renamed, so its green means nothing
until it is repointed". A parity gate that cannot tell "no such reason" from
"no such function" is worse than none, because the second reads as the first
passing.
Repointed at `web/src/lib/items/copyDropReasons.ts` and STRENGTHENED, because
the extraction split the thing it was checking in two. It now requires each
server reason to appear in BOTH:
- `COPY_DROP_REASONS`, the exported list;
- the `MESSAGES` map.
They fail differently, and the first is the one that matters. The module's own
completeness test ITERATES that list, so a reason missing from the list is
invisible to that test as well — this gate is the only place it shows. A test
driven by a list cannot notice something absent from the list.
I ALSO HAVE A CORRECTION TO MAKE, to my own prose in `f2c4a722`. That commit
message and the PR body say the Go-to-TypeScript direction "can still go stale
in the one direction that matters — a reason added to Go and not added here.
The test cannot see that." That is FALSE, and I wrote it without checking: this
parity gate has enumerated the Go vocabulary and required a renderer for every
entry since BUG-2674, which is precisely that direction. My TS test cannot see
it; the repo already had a test that could, and I asserted its absence rather
than looking. Same failure as the fixture I designed around a hazard yesterday
instead of asking whether the product had it — a claim about what is NOT
covered owes a grep exactly as much as a claim about what is.
The PR body is corrected in the same push.
Mutants, all three arms, each restored after:
- remove a reason from the LIST -> FAIL, naming the list and why the module's
own test cannot see it;
- remove its entry from the MESSAGES map -> FAIL, naming the fallthrough;
- rename the `MESSAGES` declaration -> FATAL with the repoint message, so the
fail-safe itself is exercised rather than assumed.
Gates: `go test ./internal/server` ok **279.535s** — the full package this
time, with `web/build` populated so the root embed resolves. `gofmt` clean.
Claude-Session: https://claude.ai/code/session_01Xk9M5UVPdc84xL5E1mZkm8
* docs(store): point the carry comment at the trail rather than at a person (IDEA-2893)
Comment wording only.
The attribution now reads `(IDEA-2893, lead ruling day 58; the measurements it
rests on are on that idea's trail, which is where to check this reasoning
rather than take it)`.
The lead asked for `confirmed by Dave in chat` and I declined to write it: Dave
had said nothing to me about this disposition, so the only evidence was a relay
through a channel BUG-2542 proved cannot carry provenance, and the artifact is
a permanent comment asserting what a specific person decided. The lead withdrew
the line and agreed the hold was right. Dave then ruled the general case — no
code comment needs to name him — which is the wording above and is better than
either version, because a ref is CHECKABLE and a name is not. A reader who
doubts this comment can open the idea and read the measurement; a reader who
meets a name can only take it or leave it.
Recorded team-side as CONVE-32 so the successor does not relearn it: code
comments cite the trail, never a person by name. Its scope is source comments
only — commit messages, PR bodies and trail comments are where naming who
decided something is often the entire content, and those artifacts sit beside
their own evidence.
Two things the convention says out loud rather than gloss:
- The rule reached me as a RELAY, and I acted on it because it only ever
REMOVES a claim about a person. Acting on a relay to stop asserting
something is safe in a way that acting on a relay to start asserting it is
not — which is the same distinction that made the hold correct an hour
earlier. Read as general licence to act on relayed instruction it would be
a misreading; the direction is the whole point.
- About nineteen comments already in `internal/` name a person. They are NOT
rewritten. Churning merged history to apply a new rule retroactively costs
more than it returns and a sibling rebasing onto it pays the bill. Fix one
only while editing that comment for another reason.
Gates: `gofmt` clean, `go build ./internal/store/`, and the parity gate green
(`TestCopyPreflightDropReasonsAreRenderedByTheDialog` ok) since this touches
the same file the previous commit repointed it away from.
Claude-Session: https://claude.ai/code/session_01Xk9M5UVPdc84xL5E1mZkm8
|
||
|
xarmian
|
5cb2297f67 |
fix(store): export soft-deleted collections so their live items survive a round trip (BUG-2884) (#1255)
* test(store): BUG-2884 regression tests — items under a soft-deleted collection
Measured against
|
||
|
xarmian
|
5aa4bbe319 |
fix(build): give each worktree its own Postgres test port, and refuse to run when it is unreachable (TASK-2708) (#1253)
* fix(build): give each worktree its own Postgres test port, and refuse to run when it is unreachable (TASK-2708) docker-compose.test.yml bound the host port to 5445, so exactly one worktree could run make test-pg at a time. With concurrent worktrees the normal operating mode that produced three incidents in an afternoon: a port-already-allocated collision, a container dying mid-run under concurrent suites, and a stack orphaned by a removed worktree blocking the port for everyone. The worst of the three forged a gate leg: go test exited 2 having executed NO TESTS because the database was unreachable, and exit 2 with zero FAIL lines reads like a pass at a glance. Docker now assigns the host port and the Makefile reads it back with docker compose port. Before running anything the target probes the HOST path the tests will use, from a throwaway container, and refuses with a banner saying no tests executed rather than letting an unreachable database look like a result. If the suite fails and the database is gone afterwards, it says the failures are infrastructure. The compose project name was already per-directory, so teardown never could reach a sibling; the orphan recovery command is now documented where someone looking for it will be. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR * docs: record that make test-pg is now safe from concurrent worktrees (TASK-2708) The worktree section is where a reader learns what is safe to run alongside a sibling, so it is where this belongs — including that a privately-started container is no longer needed, and the recovery command for a stack orphaned by a deleted worktree. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR * docs(store): the mutation-harness recipe reads the port back instead of hardcoding it (TASK-2708) A paste-ready recipe in a comment is a consumed artifact: it said 5445, and after the ephemeral-port change pasting it would connect to whatever else is on that port, or to nothing. Found by re-running the prose sweep with a path-scoped exclusion — the first pass piped through 'grep -v node_modules', which filters by LINE CONTENT and had silently eaten the hits in files whose matching line mentions node_modules. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR * docs(build): say that the banner discriminates, not the exit code (TASK-2708) Measured while building the counterfactual matrix: make collapses every failed recipe to exit 2, so the infrastructure refusals and an ordinary test failure are indistinguishable by status. The banners are the only discriminator, and a reader who assumed otherwise would build automation on a difference that does not exist. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR * fix(build): unique compose project, guarded startup, loud teardown failure (TASK-2708) All four from codex round 1, each verified in the recipe before accepting. Compose defaults the project name to the directory BASENAME, so two checkouts sharing a basename share a stack and one down -v tears down the other's database mid-run — the cross-worktree teardown this task exists to prevent, reached through a second door. The project name is now explicit and keyed to the absolute path. My compose comment had claimed the default was already sufficient, in the place the next reader would believe it. up --wait now runs inside the guarded block: a health-check timeout used to abort the recipe before teardown, leaving the stack behind and creating exactly the orphan this task was filed about. A failed teardown is announced with the command to reap the stack instead of being swallowed. It does NOT fail the build: the tests genuinely ran and their status is honest; the leak is a separate fact and is now a loud one. make test-pg-project prints the name so an orphan can be reaped without re-deriving it. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR * fix(build): portable readiness probe, safe project derivation, honest recovery docs (TASK-2708) Five from codex round 2, each verified before accepting. The readiness probe used `docker run --network host`, which is Linux-only by default. On Docker Desktop a perfectly healthy database would have been reported unreachable and the target would have REFUSED TO RUN where it used to work — a guard against false greens turned into a false red. It now uses the host's pg_isready when present and falls back to an in-container check, which is weaker but never lies about the platform. The project name interpolated CURDIR into shell command text, so a checkout path containing a quote would have broken the quoting. The shell now reads its own working directory instead. Teardown failures on the three guard exits were silenced by >/dev/null, contradicting the loud-teardown promise those same guards make. Two docs were falsified by my own earlier commit in this branch: CLAUDE.md still told the reader to reap a stack by directory name, and the mutation recipe in the store test omitted -p entirely, which is exactly the same-basename collision the change exists to prevent. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR * fix(build): tear the stack down on interrupt; record why a post-run outage is not reported (TASK-2708) Round 3, one finding accepted and one refused. ACCEPTED: Ctrl-C during go test killed the recipe shell before down -v, leaving an orphaned stack — the exact failure this task was filed about. An INT/TERM trap set before up covers startup as well. REFUSED, with the premise checked rather than argued: the reviewer asked for the post-run banner's EXIT_CODE gate to be dropped so a database dying after a passing run is reported. storetest.NewPostgres skips only when the env var is EMPTY; a database that is gone produces t.Fatalf, not a skip. So exit 0 means every Postgres-backed test completed against a live database, and failing the leg because the container stopped afterwards would convert honest greens into reds. Written into the Makefile so the next reviewer does not re-raise it. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR * fix(build): probe the host-published port on every platform; interrupt reports teardown honestly (TASK-2708) Round 4, both findings real. The readiness fallback ran 'compose exec pg_isready', which answers whether the server is alive INSIDE the container — a broken host port mapping passes it and the guard is bypassed. Not a rarely-exercised path either: this box has no host pg_isready, so the fallback is the branch that has been running all along. It now reaches back through host.docker.internal from a throwaway container, which is native on Docker Desktop and resolves on Linux via --add-host=...:host-gateway. Verified against a live stack, with a negative control on a port nothing listens on. That is the third version of this probe. --network host was Linux-only and would have falsely refused on Desktop; compose exec was portable but asked a narrower question than the claim it carried. The interrupt trap announced 'stack torn down' unconditionally, so an interrupted run whose teardown failed reported successful cleanup. It now reports what happened and names the command to reap the stack. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR |
||
|
xarmian
|
14cb97593f |
WIP feat(server,store): referent validation for relation values — 6 of 8 doors (TASK-2878) (#1246)
* feat(store): referent resolution for relation values (TASK-2878)
PLAN-2857 U1, first slice: the rule itself, with no door wired to it yet.
`ResolveRelationReferents` canonicalises every `relation` value in a field
map to the target item's ID and reports the ones that cannot be resolved —
same workspace, and the collection the field DECLARES.
WHERE IT LIVES was forced, not chosen. `internal/items` is DB-free by
construction and keeps the shape check only. `internal/server` cannot own
it either: six of the eight coercion doors live there, but the eighth is
`store.migrateFieldsForCopy`, and `store` does not import `server`. Putting
it here is what lets the cross-workspace copy door and the preflight door
reach the SAME function instead of two implementations of one rule — those
two already carry a comment saying they sit in different packages and that
is how they drift unnoticed.
VISIBILITY IS NOT HERE, deliberately. "Can this requester see that item" is
request-scoped and needs the user, role and auth mode; the server layer
adds it via `checkItemVisible`, which already exists as the context-free
predicate for exactly this reason.
NO SLUG FALLBACK, which is a deliberate divergence from `ResolveItem`
(UUID, then ref, then slug). Found by a test failing rather than by
reading: "red" resolved, because it is the slug of the live Red colour. A
relation field's contract is that it stores an item ID; a slug is neither
an ID nor stable, so the same stored value could point elsewhere tomorrow.
Worse, "red" is exactly the free-text value the pre-U2 editor wrote into
these fields, so accepting it makes the corruption this unit exists to
stop indistinguishable from a legitimate write. The client refuses the
same match for the same reason (TASK-2868). Exact-TITLE resolution is U6.
Issues are reported in SCHEMA order, not map order, because the copy
preflight is one of the callers and is specified to be safe to call
repeatedly and return identical results.
Unresolvable values are left EXACTLY as supplied: the caller quotes them
back, and a half-canonicalised map would make a drop report lie about what
the source held.
Verified rather than asserted: both lookups exclude soft-deleted rows
(`ResolveItem` by contrast with `ResolveItemIncludeDeleted`; `GetItem` via
`getItemScanQ`, which appends `AND i.deleted_at IS NULL`). That is what
keeps "target was deleted" distinguishable from "never resolved" — the
read half U2 shipped.
* feat(server): refuse unresolvable relation values at the four write doors (TASK-2878)
PLAN-2857 U1, second slice: the doors that take CALLER-SUPPLIED field
values now refuse a relation value that does not name a live item in the
declared target collection — create, update (full fields), update
(fields_patch), and bulk update.
The server half adds the one thing the store resolver deliberately does
not: visibility. It folds into the SAME `not_found` reason rather than
getting its own, because "that item exists but you may not see it" is an
existence oracle, and this codebase has a standing rule against handing
one out.
Ordering at every door is after the shape check and after coercion, so one
bad value produces one error rather than two describing it differently,
and so the value is in its final form when it is resolved.
`fields_patch` examines only the keys the patch carries — the resolver
skips absent keys — so an unresolvable value already stored on an item is
not re-litigated by an update that does not touch it. That mirrors the
undeclared-key rule immediately above it, and it is what stops this
turning every edit of a legacy item into a failure.
Refusals use the ORDINARY `validation_error` shape with no new details
key. The MCP stdio transport classifies errors by matching CLI stderr
prose, so a structured field it cannot see would help nobody there, and a
new error shape is a contract change for every client.
Existing suites unchanged: internal/server ok (224.5s), internal/store ok
(258.0s), internal/items ok. Nothing in the tree was writing a bogus
relation value through these doors, which is what made this slice safe to
land before the per-door pins.
* feat(store): one migrate decision for all four carrying doors (TASK-2878)
PLAN-2857 U1, third slice, on the lead's refined ruling: PROVENANCE
decides, not which door you came through.
* SUPPLIED (an explicit `--field` override on a move or copy) is a write
like any other, so an unresolvable value REFUSES.
* CARRIED (everything the source item already held) was asserted by
nobody. `internal/items` has accepted any string for a relation all
along, so most stored values are legacy — refusing them would make
those items unmovable and uncopyable. Dropped and REPORTED instead.
And carried values are not all alike, which is the refinement that keeps
this from being one rule wearing four coats:
* WITHIN a workspace (move, bulk move) the targets are still here, so a
valid relation SURVIVES the move and only an unresolvable one is
dropped, through the `dropped_fields` channel BUG-2674 established.
* ACROSS workspaces (copy, and its preflight) every carried relation is
dropped WITHOUT a lookup: the value names a source-workspace row and
v1 excludes cross-workspace targets, so no amount of resolving in the
destination changes what it means. Reported as `referent_not_portable`
— the same reason `github_pr` uses, because it is the same fact about
the same kind of value.
`MigrateRelationReferents` is one function because the four doors sharing
it is the point, not tidiness: the preflight lives in `internal/server`
and the copy in `internal/store`, and the code already carries a comment
saying those two sit in different packages and that is how they drift
unnoticed. A preflight that says "carried" while the copy drops is one
request answered two ways.
Tests drive both provenances against both modes, because the same bad
value must be a drop when carried and a refusal when supplied — a suite
that only drove carried values would pass against a build that never
refuses anything.
* feat(server): the two same-workspace migrate doors resolve and report (TASK-2878)
PLAN-2857 U1: `handleMoveItem` and `bulkMoveCollection` now take their
relation decision from `store.MigrateRelationReferents` — the same
function the two copy doors will call, which is the point of it existing.
Within a workspace the targets are still present, so a correctly-related
item KEEPS its relation across a move; only an unresolvable value is
dropped, and it joins the `dropped_fields` report BUG-2674 established
rather than failing the move. Refusing carried values here would make
every legacy item permanently unmovable, and `internal/items` has accepted
any string for a relation all along, so "legacy" is most of them.
The bulk path carries no per-field overrides — only `status` — so every
relation value reaching it is CARRIED and nothing there can refuse. It
passes nil for `supplied` to say so, and keeps the refusal branch: it is
unreachable today and stops being a silent no-op the day that path grows
overrides.
internal/server ok (270.8s), internal/store ok.
KNOWN GAP, recorded rather than half-built: the two CROSS-workspace doors
are not wired yet, and the reason is a real constraint rather than
running out of road. `migrateCopyFields` is called from
`copyItemAcrossWorkspacesTx` with a transaction already open
(`s.db.Begin()` at the top of that function), so resolving a SUPPLIED
override there would issue POOL reads while holding a tx — the deadlock
shape this repo keeps a deterministic test for. The carried half needs no
lookup at all and is safe; the supplied half needs
`GetCollectionBySlugQ` / `GetItemByRefQ` so the resolver can run on the
tx's connection, which is exactly the `...Q` convention the store already
uses (`GetItemQ`, `getCollectionInWorkspaceTx`, `uniqueSlugQ`). Adding
those two is the remaining work, and it is what makes one function
genuinely serve all four doors.
* refactor(store): thread a Queryer through referent resolution (TASK-2878)
Preparation for the two cross-workspace copy doors, landed on its own
because it is independently correct and the doors are not.
`migrateCopyFields` runs inside `copyItemAcrossWorkspacesTx`, which opens
a transaction as its second statement. A resolver reading from the POOL
there would issue pool reads while holding a tx — the deadlock this repo
keeps a deterministic test for. So `ResolveRelationReferentsQ` and
`MigrateRelationReferentsQ` take the executor, following the store's own
convention (`GetItemQ`, `uniqueSlugQ`, `getCollectionInWorkspaceTx`); the
pool-backed names stay as one-line shims for the six wired doors.
Two small read helpers come with it. `collectionIDBySlugQ` returns the ID
only — the referent check compares `item.CollectionID`, and the full model
would pull in per-collection counts nothing here uses. `itemByRefQ` keeps
`GetItemByRef`'s fallback to a bare item-number lookup, because a relation
written as COLO-3 must keep resolving after its target collection is
renamed, which is exactly what BUG-2873 made possible.
internal/store ok (341.7s), vet and gofmt clean.
WHY THE COPY DOORS ARE NOT IN THIS COMMIT. They were written and building,
and I reverted them. Team CONVE-29 and the lead's condition both say the
copy pair lands WITH its pin — one case driving BOTH doors, asserting
identical drop-and-report for a carried relation and refusal for a
supplied override — and I measured 58.9% context against a 65% ceiling,
which is not enough for that pin plus the 270s server suite plus the
commit. Landing the behaviour change unpinned would have been worse than
landing nothing: the preflight and the store copy are the pair the code
already warns will drift unnoticed, so they are the last place to accept
an untested agreement.
The design is complete and on the trail: derive the carry mode from the
existing `items.MigrateScope` rather than a second flag, pass `tx` on the
store side and the pool on the preflight side, refusals through the copy's
existing validation-error channel, drops appended to `migrated.Dropped`.
* feat(copy): the two cross-workspace doors resolve referents, with their pin (TASK-2878)
PLAN-2857 U1, doors seven and eight. `migrateCopyFields` and
`handleCopyItemPreflight` now take their relation decision from
`store.MigrateRelationReferents` — the same function the four write doors
and the two move doors already call, which is the entire reason it exists.
The defect this closes: MigrateFields matches on key and TYPE, so a
same-named `relation` field carried a SOURCE-workspace item id across the
boundary and the preflight reported it as a clean carry. What landed in
workspace B was a value naming a row in workspace A — unrenderable, and
indistinguishable on read from a legitimate reference.
Provenance decides, as at the move doors. A CARRIED value on a
cross-workspace copy is dropped without a lookup (no id from A can mean
anything in B) and reported through the `dropped_fields` channel BUG-2674
established; a SUPPLIED override is an ordinary write and an unresolvable
one is refused, 400 validation_error on both doors, rendered by the same
`store.RelationIssuesMessage` so one refusal cannot acquire two phrasings.
`internal/server`'s `relationIssuesMessage` now delegates to it: the eighth
door refuses from inside `store`, so the sentence had to be reachable there.
Two things are threaded rather than re-derived, and both are load-bearing:
- The TRANSACTION, not the pool. `migrateCopyFields` becomes a method
taking a Queryer, and `copyItemAcrossWorkspacesTx` passes its `tx`. That
function has held a transaction since its second statement, so a pool read
from inside it can wait for a free connection while every pooled
connection is blocked on this transaction's locks — the starvation shape
BUG-2409 fixed for the attachment planner and this repo keeps a
deterministic test for. This is what the day-70 handoff named as the
reason these two doors were not wired with the other six.
- The MODE comes from the `scope` MigrateFields was already given, not from
a second boundary test. Two independent answers to "is this crossing a
workspace" is how one request gets migrated one way and validated the
other, and this path also serves a copy whose target IS the source
workspace, where relations resolve and survive exactly as on a move.
The destination workspace id is the resolution scope: a supplied override
is a write into B and must name something that exists there.
THE PIN, and why it is not a per-door table. These two doors sit in
different PACKAGES and the code at both sites says so is how they drift
unnoticed. A table with a row per door can be fully green while the two
disagree about one request, which is the defect rather than a gap in
coverage of it. So every case sends ONE body to BOTH endpoints:
- carried relation — must drop on both, and the preflight must say
referent_not_portable rather than the generic no_target_field, which is
false here (the destination DOES declare the key, so that answer sends
the reader to fix a schema that is fine);
- supplied + unresolvable — both refuse, same status, same code, both name
the offending field and value, and nothing is written;
- supplied + resolvable — the positive control, supplied as a REF so
resolution is visible in the result. Without it the first two legs are
equally consistent with "relations always fail".
Negative controls run, all three mutants BUILD-CHECKED first (a
non-compiling mutant produces no `--- FAIL` lines and reads as survived):
both doors unwired = DETECTED; preflight unwired alone = DETECTED; store
unwired alone = DETECTED. Each single-door mutant failing is the pin's
whole claim — neither door can be wired without the other.
CONVE-23 sweep: the preflight's LIMITATION comment said this gap belonged
in MigrateFields "for both callers at once". That is now false in its
prescription as well as its premise — `internal/items` is DB-free by
construction and cannot ask whether a string names a live item — so the
comment records where the fix actually went and what of it remains open
(`computed`, `terminal_options`, `unique_scope`).
Gates: internal/server ok 170.0s · internal/store ok 296.1s · internal/items
ok · go vet clean · gofmt clean · make lint 0 issues.
* test(server): the per-door x per-provenance table, and the defect it found (TASK-2878)
PLAN-2857 U1. `internal/store` already tests the resolver exhaustively, but
those tests call it DIRECTLY: they vouch for the component and say nothing
about whether any door is bound to it. A door that never calls the resolver
passes every one of them. This table is the binding claim — one leg per
(door, provenance) pair, driven through the handler a client reaches.
PROVENANCE IS THE SECOND AXIS BECAUSE THE ANSWER DEPENDS ON IT, not for
symmetry. A SUPPLIED value is the caller's assertion and an unresolvable one
is refused; a CARRIED value was asserted by nobody, and refusing it makes
legacy items un-updatable, un-movable and un-copyable — the failure this
unit would otherwise CAUSE while fixing another. Which provenance a door
sees is a property OF THE DOOR, and getting it wrong is invisible until a
legacy item meets it.
THE TABLE FOUND ONE, ON ITS FIRST RUN. `bulkFieldUpdate` merges the item's
STORED fields blob with the caller's `changes` before validating, and the
resolver was pointed at the MERGED map — so a bulk status move or
set-priority re-litigated every stored relation value and REFUSED the item.
An item carrying a legacy relation value had its status and priority frozen
by a field the operation never mentioned. Fixed the way the fields_patch
door already handles it: resolve only the keys the operation CHANGES, read
out of the coerced map so the value is final, written back so a supplied ref
is still canonicalised. Verified by prediction before the run and by the leg
failing against the unfixed code.
THE DISPATCH MUTANT, which is what makes the table's coverage a measurement
rather than a hope. Wire ONLY the two `extractParentLink` doors (update
fields, update fields_patch) and neuter the other six by swapping their
field-map argument for an empty one — types unchanged, so the mutant
compiles and its verdict means something:
build OK · failing legs: create (both), move (both), bulk move,
bulk update supplied, copy, preflight. Update and update-patch pass.
Exactly the six unwired doors, and only those. Then each door alone, eight
runs: every one detected by its own legs and no other door's. That is the
claim the table exists to make — each leg reaches its OWN door rather than
being satisfied by a neighbour's check.
ONE DOOR NEEDS A WEAKER INSTRUMENT, AND THE TEST SAYS SO. No bulk op puts a
relation key into `changes` — `op` is a closed list and the only field
values any of them set are `status` and `priority` — so that door's SUPPLIED
branch is unreachable from outside. The first mutant run proved it: unwiring
door 4 alone left every black-box leg passing. It gets a direct-call leg,
labelled as vouching for the FUNCTION and not for a binding that does not
exist yet, and kept for the same reason `bulkMoveCollection`'s refusal
branch is kept: the day the bulk path grows per-field overrides, the branch
must already refuse rather than be a silent no-op nobody notices is missing.
Every refusing leg has a resolvable counterpart. Without them the table is
equally consistent with a build that refuses every relation value.
Gates: internal/server ok 156.1s · go vet clean · gofmt clean · make lint 0
issues.
* feat(mcp): ToolSurfaceVersion 0.29, and the drop-reason renderer it exposed (TASK-2878)
PLAN-2857 U1. The bump, its documentation sweep, and the consumer this
change turned from a rare wart into a routine one.
THE BUMP, at 0.29 rather than 0.28. Rebasing onto main found
|
||
|
xarmian
|
a0e1bbf53c |
feat(store): composite FK making a reminder's workspace agree with its item's (IDEA-2883) (#1252)
* feat(store): composite FK making a reminder's workspace agree with its item's (IDEA-2883) item_reminders.workspace_id is denormalized from the item, and until now nothing but a read-side predicate repeated at five call sites stopped the two from disagreeing — a class IDEA-2641 closed one read at a time across two codex rounds. The composite foreign key makes the disagreeing row unrepresentable; the predicates stay as belt on top of braces. SQLite rebuilds the table (it cannot ADD a constraint), Postgres alters in place. Both add a unique index on items(id, workspace_id) as the referenceable parent key, and both repair before constraining — a pre-existing disagreeing row would otherwise fail a deployment's startup. Such a row is DROPPED rather than repaired: every read predicate already filters it, so it can never fire, and pointing it back at its item would resurrect it and deliver a notification for an instant long past. No ON UPDATE CASCADE: items.workspace_id is never updated anywhere, and a future feature that moves an item between workspaces should be refused here rather than silently rewriting its reminders. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR * test(store): build the disagreeing-reminder fixture with enforcement suspended (IDEA-2883) The IDEA-2641 test that proves a workspace-mismatched reminder is inert everywhere could no longer construct its row: the composite foreign key refuses it, on both drivers. Kept rather than deleted, for a specific reason rather than a general nod to defence in depth. The constraint protects rows written through a connection enforcing it, and SQLite's enforcement is a per-connection pragma that table-rebuild migrations legitimately turn off — so a row can still arrive from a restored pre-086 backup, from an operator who dropped the constraint, or through a future rebuild's window. Those five reads are what keep it inert, and this is the only test that says so. Its comment said the table has nothing that forbids such a row. That is now false and it says so. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR * docs(store): the table now forbids the disagreement the predicate compensated for (IDEA-2883) reminderFireable's note said the table has an FK to the item and no constraint tying the two columns. That was the premise for the predicate and it is now false. It also says why the predicate stays: enforcement binds to the connection doing the write, and SQLite's is a per-connection pragma that table-rebuild migrations legitimately turn off. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR * fix(store): drop EVERY legacy item_id foreign key, not just the first (IDEA-2883) Self-review: plpgsql's SELECT ... INTO takes the first row and does not error on several, so a table carrying two single-column FKs on item_id would keep one alongside the composite key while the migration reported success. A loop costs two lines. Nothing creates a duplicate today, which is exactly why the branch needed a test rather than a comment. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR |
||
|
xarmian
|
5c4fa22999 |
fix(store): collection slug races — allocate under the row lock, serialize CreateCollection with rename (TASK-2885) (#1249)
* fix(store): allocate collection slugs under the row lock; serialize CreateCollection with rename (TASK-2885) IDEA-2874: UpdateCollection allocated its new slug on the pool before its transaction opened, so two renames deriving the same base both chose it and the loser failed on the UNIQUE index. Allocation now runs on the tx after the workspace lock and the FOR UPDATE re-read, matching documents.go and items.go. BUG-2875: CreateCollection took no lock, so a create could land between a rename's FOR UPDATE scan of existing siblings and its commit. It now opens a transaction, takes the workspace advisory lock first (the order item-create and rename share), and allocates its slug under it. Population: every collection rename and every collection create; the import door inserts into a workspace nobody else holds yet and is out by construction. * test(store): pool-I/O guard for the collection slug writers under the workspace lock (TASK-2885) |
||
|
xarmian
|
02846a6785 |
fix(cli): the session's registered agent is the name its writes carry (BUG-2882) (#1248)
* fix(cli): the session's registered agent is the name its writes carry (BUG-2882) Two seats booted under one name; one re-registered under the right one with `pad session register --agent`; every write it made afterwards still carried the wrong name. The registry row and $PAD_AGENT were two self-declarations of "the same value" — the row could be rewritten, the environment could not, and nothing reconciled them or warned. Night 10 read both live seats inverted against each other. ResolveAgentName now consults the registry record for the session that owns this process FIRST: a stat-and-read of one file, no MkdirAll, no lock, ignored when malformed, legacy, or carrying a different process-start token (pid reuse). A non-empty registered name wins over .pad.toml and $PAD_AGENT; an anonymous row leaves an environment name in force. `pad session register` without --agent keeps the current name, as before, because its default is the resolver. Help text, the record's doc comment and README's precedence list say what is now true. Test: register as rook under PAD_AGENT=wren and a .pad.toml name → rook; default re-register → rook; anonymous → the .pad.toml name; a record for this pid with another process's start token → ignored. The registry-step mutant fails the first two assertions. Fixes BUG-2882 * fix(cli): a registry record names this session only when it is verifiably this session's, and the identity tests stop reading the real registry Codex round 1 on #1248. (1) registeredAgentForThisSession compared process-start tokens only when both sides had one, so a stale row with no token under a reused pid — a dead session's — would have named a live one. Fail closed: when this process can read a token, the record must carry the same one; and the record must pass the same OwnerLiveness verdict `pad session list` applies. The token-less case is now a test row, with a positive control after it. (2) The pre-existing resolver and header tests cleared PAD_AGENT/CLAUDECODE but not HOME or the session-pid variables, so run inside a registered seat they would have read that seat's row as the resolver's first answer. They now run from a scratch HOME with no session identity. Refs BUG-2882 * fix(cli): a registry row names this session only if its owner is this process or an ancestor, where that can be checked Codex round 2 on #1248. (1) TestPushItemSendsResolvedAgentHeader was the one identity test round 1's hermeticity fix missed; it now isolates HOME and the session env like its neighbours. (2) The registry step accepted a row whose owner pid was alive and token-matched but NOT this process or an ancestor — a misconfigured CLAUDE_PID pointing at a sibling session could borrow that session's name. Refused where the platform can walk ancestry. Not gated on PIDVerified: CaptureSessionOwner records "cannot check" and "checked and wrong" as the same false, and the flag alone would have disabled the step on every non-Linux platform. The check is re-run and only a checked-and-wrong answer refuses. Test uses a live non-ancestor child as the claimed owner; the refusal-dropped mutant lets it name us. Refs BUG-2882 * docs(cli): session list help — a row's name outranks PAD_AGENT; change it with --agent, not by re-registering (BUG-2882, codex round 3) |
||
|
xarmian
|
b437cc582d |
feat: item reminders — the fire-at-an-instant primitive, and one overdue rule for all four surfaces (IDEA-2641, closes #1010) (#1244)
* feat(store): item reminders — the fire-at-an-instant primitive (IDEA-2641) Adds the storage, the scheduler tick, and the canonical event for one-shot item reminders (GitHub #1010). Nothing in Pad acted at a target time before this: a due_date makes an item show up as overdue once somebody asks the dashboard, so "revisit TASK-X on the 1st" had to live in an external cron. A TABLE, NOT A SCHEMA-FIELD ANNOTATION. The design sketch proposed marking schema date fields with a `reminds: true` key on models.FieldDef; recon overturned it. Such a key does not survive an ordinary collection edit, two independent ways: the web editor destructures each field into an EditableField and rebuilds a fresh definition key-by-key on save, so unknown keys are dropped (`pattern` and `unique_scope` survive only because two lines were hand-added for them), and models.CollectionSchema has fixed fields with no catch-all, so any Go unmarshal+marshal round-trip strips unknown properties — the hazard retargetRelationFieldsTx mutates raw JSON to avoid. Both failures are silent and both disarm a whole collection's reminders at once. It is the same defect class that moved traits out of the schema column in TASK-2657. The table also gives the lifecycle a home. A reminder is armed, then fired, then acknowledged, and a re-arm returns it to armed — per-reminder state a field definition has nowhere to keep. remind_at is an RFC3339 UTC instant, deliberately not a `date` schema value: those admit both YYYY-MM-DD and full RFC3339 and are compared against the SERVER'S LOCAL calendar day. A fire-at time cannot carry that ambiguity. The remaining timezone question for due_date is filed separately. Firing is one transaction per reminder carrying BOTH the fired_at write and the outbox insert. That pairing is the point: a fired_at committed without its event is a reminder that silently notifies nobody and can never be retried, because the row has left the armed set; an event without fired_at fires every tick forever. The UPDATE's own `fired_at IS NULL` predicate is the arbiter, so two instances ticking at once produce exactly one winner. item.reminder_due is admitted to the closed events/1 set as v1.2, with a new PayloadReminder family and no SSE name. The subject is the REMINDER, not the item: two reminders can be armed on one item, so an item-subject event could not say which fired, and the reminder id is what an acknowledgement addresses. A new payload family rather than reusing the item snapshot for the same reason — a snapshot would validate and still not answer the only question the event exists to answer. No SSE name in v1 because the poll surface is the contract; adding one later is additive, removing one is not. Ack is explicit and nothing else acks. An item reaching a terminal status deliberately does NOT ack: that would make every status write a reminder mutation, and it would silently consume a reminder set to fire after the work was done. * feat(server): reminder surfaces, and one shared overdue rule for all four Second half of IDEA-2641: the HTTP surface, the scheduler tick's wiring, and the fix for the finding that justified the unit — `ready` / `next` did no date handling at all. OVERDUE NOW HAS ONE IMPLEMENTATION. It used to live inline in the dashboard's attention loop, which meant `pad project stale` inherited it (it filters that very list) and the recommendation surface never saw it. So a deadline reached the two surfaces that REPORT on work and never the one an agent PULLS from. overdue.go is now the only place that decides, and all four call it. Two behaviour changes fall out, both deliberate: - An overdue item bypasses the orphan branch's high/critical priority gate. That gate was where a deadline quietly stopped: a low-priority item three weeks late was reported by `stale` and never suggested by `next`. - Overdue sorts above in-progress. The list is capped at three, so a rank below in-progress would not merely order the deadline lower — on any workspace with three things in flight it would keep an overdue item off the surface entirely, which is indistinguishable from not shipping this. The server-local-today comparison is UNCHANGED and known to be wrong for multi-timezone deployments; it is filed as its own item with the cloud case stated. Changing what "overdue" means on every existing instance inside a change about where the rule LIVES is the kind of behaviour change nobody reviews. Fired reminders reach `next` / `ready` two ways, from one filtered list: PendingReminders is the addressable form (it carries the id an ack needs), and a prepended suggestion is the rendered form. They are prepended AFTER the cap rather than entered as ranking candidates — a reminder is not a task competing on priority, and whether it appeared should not depend on how busy the workspace is. Terminal-item reminders are FILTERED from the surface, never acked. Acking on terminal status would couple every status write to reminder state and would consume a reminder armed to fire after the work was done. The row stays exactly as the user left it; the distinction is observable, and asserted. Three guard tests caught this change and each was answered rather than silenced: - The request-body reader guard was right: the handlers now go through decodeJSON, inheriting the NUL refusal and the size cap. - The canonical-events guard was right: item.reminder_due is admitted to the duplicated contract table as SPEC-3 v1.7, with the reminder subject kind and the new payload family. SPEC-3's own text owes the same amendment. - The NUL census asked for a decision on eight new columns. None carries caller text: ids and FKs are server-generated, four are the server clock, and remind_at is now re-parsed and re-formatted in the STORE as well as at the edge — so the stored value is always machine-produced from a parsed time and no caller bytes reach the column. The doc comment that used to say "the caller normalizes" protected nothing. Regenerating the baseline also found that GEN_NUL_BASELINE=1, which the test's own instructions name, was never implemented — the flag did nothing, so the documented path was hand-editing the file. Implemented, so the next reader gets the mechanism the instructions promise. * test(reminders): the lifecycle, the four surfaces, and 22 killed mutants Every test here was designed against a specific mutation and the mutation was RUN. A green suite proves nothing about a suite nobody tried to break, and three of the mutants I first wrote were not experiments at all. Store (10 mutants, all killed): candidate predicate <= flipped to >=; the event emission lifted out of the fire transaction; the fire UPDATE's `fired_at IS NULL` arbiter removed; the RowsAffected check ignored; re-arm clearing fired_at but not acked_at; ack losing `fired_at IS NOT NULL`; the poll surface losing `acked_at IS NULL`; normalizeRemindAt no longer refusing; it dropping .UTC(); GetReminder losing its workspace scope. Surfaces (12, all killed): the priority gate no longer bypassing on overdue; the sort no longer ranking overdue first; attention leaving the shared helper; the reason losing its OVERDUE prefix; the comparison flipped to >; terminal items no longer skipped; terminal reminders no longer filtered; the filter ACKING instead of hiding; reminders appended instead of prepended; the tick running on a far-future clock; ack answering 200 for an unfired reminder; parseRemindAt accepting a bare date. THREE MUTANTS DID NOT COUNT ON THE FIRST PASS and were rewritten. Two failed to compile (`if false` orphaned a variable; deleting a parse orphaned an import) and one had an anchor matching two call sites. A non-compiling mutant emits zero FAIL lines and reads exactly like a surviving one — it invents a hole that is not there — so the harness reports BUILD-FAIL and ANCHOR-BAD as outcomes distinct from SURVIVED. It also restores files from an in-memory copy rather than `git checkout`, which would delete uncommitted work in the tree. ONE MUTANT GENUINELY SURVIVED and the test was at fault, not the mutant: appending rather than prepending reminder suggestions was undetectable because the fixture had a single item, so the reminder sat at index 0 either way. The fixture now fills the three-item cap with in-progress work, where an appended reminder lands fourth and vanishes. Faithful mutant, weak test — checked in that order. The same lesson shapes the four-surface fixture: it is a LOW-priority open orphan, because that is the case the old code handled worst. A high-priority task would have made the ready/next leg pass against the unfixed tree, which is a green that measures nothing. Negative controls throughout: a future deadline is not overdue and does not reach the gate bypass; a tick with nothing due fires nothing; a completed item is neither overdue nor suggested. Without them a helper that reported every date, or a tick that fired everything, would satisfy every positive leg. The lead's pin is asserted in both directions: a fired reminder on a done item is ABSENT from the surface and PRESENT and still unacknowledged in the table. Asserting only the absence would pass against an implementation that consumed the row, which is the behaviour the pin exists to forbid. * feat(mcp): pad_item.remind + ack-reminder, ToolSurfaceVersion 0.28 An agent that can RECEIVE a reminder but not set one has half the primitive. The poll surface is pad_project.next / ready, both long exposed, so reminders already reached agents — what was missing is the other half: deferring a piece of work is exactly the moment an agent knows when it wants to be asked again, and it had no way to say so. Two additive actions, two optional params. Nothing existing moved, so a v0.27 consumer enumerating neither is unaffected — the v0.13 / v0.11 / v0.8 disposition, which likewise wired existing CLI verbs onto the catalog. remind_at REFUSES a bare date rather than reading it as midnight. Worth stating because the `date` schema type accepts YYYY-MM-DD and a caller will reasonably try it here: a bare date names a 24-hour span, and choosing an hour inside it would fire at a time nobody picked. Re-arm and disarm stay CLI-only. Both address a reminder by an id the agent would have to list first, and no listing action exists on this surface — a door with no handle. Adding them later is additive. Five guards had to be taught, and each was answered on its merits rather than excluded: the HTTP parity test (route mappers added, so the actions work on the remote transport rather than being advertised and unrouted), the read-only catalog's cmdhelp fixture and expected cmdPath map, the field- conflict classifier (remind_at / reminder_id are NOT field writers — a reminder is a row in its own table addressed by its own id, so listing them as classified sources would have pointed detectFieldConflicts at something that is not a field source), and the instructions.md / README action tables. That machinery is why the version bump is safe to make now, and it earned its keep on this change: every one of the five failed on the first build after the catalog entry landed. CONVE-23 sweep for prose this falsifies: - SPEC-3 (DOC-2653) amended to v1.7 in the room, recording item.reminder_due with its new subject kind and payload family — the first canonical event with no user mutation behind it, since a scheduler tick produces it. - CLAUDE.md gains the reminder routes, the CLI verbs, and the v0.28 entry. It was also stale at 0.26 with NO v0.27 entry at all: the 0.27 unit swept instructions.md and README.md and missed this file. Both added. - skills/pad/SKILL.md gains the verbs and a routing entry, including the two things an agent will get wrong — the time is an instant, so ask for a time of day rather than picking one, and finishing the item does not acknowledge the reminder. * fix(reminders): codex round 1 — four findings, all real, all with a pin Round 1 found four defects and refuted none of them. Each fix carries a test that fails against the code as it was, and each of those was mutation-checked. **P1 — pending reminders bypassed item-level visibility.** Every other dashboard section reads `allItems`, which the store already scoped to the caller's collections AND their granted item ids. The pending-reminder list is a direct workspace-wide query and inherited none of that, so a guest holding a grant on ONE item could read the refs and titles of every other item in the collection through its reminders — an item-level leak wearing a notification's clothes. Now filtered with the same `isItemVisibleToGuest` call the sibling sections use. The test's two items share a COLLECTION on purpose: a collection-level filter was already applied, so separate collections would have made it pass against the unfixed code. **P1 — soft-deleted items could starve the queue permanently.** Candidate selection ignored `deleted_at`, and `fireOneReminder` rolls back when it finds the item gone — which leaves the reminder ARMED and therefore a candidate again on the next pass. Candidates are ordered oldest-first and bounded by a limit, so enough archived reminders fill every batch and no live reminder ever fires. Silent, too: the tick reports zero fired and looks idle. Excluded in the candidate query rather than skipped downstream, so those rows never occupy a slot; the reminders themselves are kept, so restoring an item restores its reminder with it — asserted, because a fix that reaped them would pass the starvation test alone. **P2 — the pass stopped at the first failing reminder.** The per-reminder transaction exists precisely so one unfireable row cannot hold back the rest, and `return fired, err` made that comment false — with candidates oldest-first, one persistently broken old reminder blocks every newer one forever. Now continues and joins the errors, so a pass that fired seven and failed three reports both halves rather than reading as clean. The loop is split behind an injected seam because a real mid-transaction failure is not reachable from outside: the database refuses the corrupt rows that would cause one (verified — invalid JSON in items.fields is rejected by the schema). **P2 — suggestions dropped the reminder id.** The docs tell an agent to acknowledge what it sees in next/ready, and the payload carried no handle: a stateless poller could read the reminder and had no way to retire it, so it would be shown the same item forever. `DashboardSuggestion` now carries `reminder_id` (omitempty), `pad project next` prints the exact ack command, and the test acks with the id the surface handed out rather than merely checking the field is populated — a wrong-but-present id satisfies equality with itself. Four mutants, four killed; one was rewritten first because its anchor matched two call sites and was therefore not an experiment. * fix(reminders): codex round 2 — four findings, all real **`--rearm` was unusable.** `ExactArgs(1)` forced an item ref that the rearm branch then ignored, so the flag could not be reached without supplying a ref that was silently discarded. Now `MaximumNArgs(1)`, with each mode checked explicitly: a ref is required to arm, and a ref supplied ALONGSIDE `--rearm` is refused rather than ignored — it names an item the reminder may not even belong to, and quietly dropping it is how a user learns nothing about the reminder they just moved. **`unremind --format json` emitted plain text**, breaking the parseable-output contract every sibling command honours. **The MCP `ref` param did not list `remind`.** Agents read that flat description to decide what to send, so an action missing from it is an invalid call waiting to happen. It now also says what `ack-reminder` takes instead, and why: a reminder is addressed by its own id because an item can carry several. **Fractional seconds fired early.** `time.Parse` accepts `09:00:00.900Z` and `Format(RFC3339)` drops the fraction, so it was stored as `09:00:00Z` and fired 900ms BEFORE the moment the caller named — silently, having rewritten their value on the way in. Seconds are genuinely the stored resolution (the column is compared as a string against a whole-second clock, and the tick runs every 30s), so the only question was which way to resolve it, and truncation resolved it the wrong way. `NormalizeInstant` now rounds UP: at most a second of lateness, in exchange for a guarantee that can be stated — a reminder never fires before the instant it was set for. Late is a reminder; early is a wrong answer. Whole seconds round-trip exactly, which is asserted, because an implementation that added a second unconditionally would otherwise pass. Three mutants for this round, three killed (round-up→truncate, round-up→unconditional-add, MaximumNArgs→ExactArgs). Thirty across the unit. Two fixes carry no dedicated test and it is worth being explicit rather than implying coverage: the `--format json` branch on `unremind` is a one-line output change with no server-free way to drive it, and the MCP `ref` description is prose the drift tests do not read — they assert an action is DOCUMENTED, not that a param's sentence lists it. * docs(reminders): the ack id is on the surface an agent polls, not only on the arm response CONVE-23 follow-through on the round-1 fix. Both agent-facing docs told a caller to acknowledge a reminder with the id "returned when you armed it" — true, and useless to the caller that matters: a poller reading next/ready never armed anything. The suggestion now carries reminder_id and `pad project next` prints the exact ack command, so the docs say that instead. The prose was written before the fix existed, which is exactly the case CONVE-23 is about: a change that makes an instruction stale without touching the file the instruction lives in. * test(reminders): bind the tick LOOP to the work, not just the pass (CONVE-19) Every other test in this file calls runReminderTick directly. That vouches for the component and says nothing about whether anything ever calls it — a tick that is never started is indistinguishable, from those tests, from one that is. It is the convention's exact case, and the failure I recorded on my own identity doc three times in one unit: I test the component and not the binding. Driven through the injectable tick channel so the assertion pins a SPECIFIC pass instead of racing a 30-second ticker, and polled to a bounded deadline so a loop that never runs FAILS rather than hanging the suite. Mutant: drop `s.runReminderTick()` from the select and this goes red while every direct-call test stays green. Killed. The idempotence leg exists because a second Start spawning a second loop would leave one running after Stop, making the BUG-842 drain invariant false for this sweeper specifically — the one property a copied lifecycle is most likely to get right by accident and least likely to be checked. The cmd/pad call site (cmd_server.go, alongside StartTokenReaper) stays verified by inspection: a source-scanning guard for it would be an instrument asserting facts about source, which is code with an adversary and not worth it for one line that sits in the middle of five identical neighbours. * fix(reminders): codex round 3 — a deferred reminder fired anyway, and the poll surface was unbounded **A re-arm mid-pass did not stop the fire.** The candidate scan selects an id; before the UPDATE runs, a `--rearm` can move that reminder into the future. Re-arm clears `fired_at`, so a predicate checking only `fired_at IS NULL` still matched — the pass fired a reminder the user had just deferred and emitted its event. The re-arm cannot undo that: it can clear the mark, but the event is already on the outbox and at-least-once means a consumer has seen it. The fire UPDATE now revalidates `remind_at <= nowTS` against the SAME nowTS the candidate scan used. Same-value deliberately: the arbiter and the scan must agree about when this pass is, or a reminder could pass one and fail the other for no reason but clock drift inside a single pass. **The poll surface was unbounded.** Every fired-and-unacknowledged reminder was loaded and turned into a suggestion prepended to a list that is otherwise capped at three, so a workspace with five hundred unacknowledged reminders returned five hundred suggestions — in the dashboard response, the hottest read in the product, growing until somebody acknowledged them. Two bounds, because they are two different guarantees: the query takes a window (default 50, oldest-fired first, so it holds what has waited longest), and the prepended suggestions are capped at 5 so `suggested_next` stays a recommendation rather than a second inbox. The full set stays addressable in `pending_reminders`. Truncation is REPORTED as a boolean, not a count. A count would have to be post-visibility-filter to be true for the caller reading it, and the store cannot compute that — the filter runs per item, above. "There are more than you can see here" is the strongest claim the data supports, so it is the one made. Four mutants; two killed outright, two survived and were run down under CONVE-28: - **Uncapped suggestions survived because the fixture had ONE reminder** — capped and uncapped are the same list at n=1. That is the SECOND time a single-item fixture hid a count-or-order property in this file. Fixture now arms eight; it also asserts all eight remain in `pending_reminders`, so the cap is pinned to the recommendation and not to the data. - **Removing the SQL LIMIT survived, correctly, and the test comment now says so.** The Go slice cap bounds the PAYLOAD; the SQL LIMIT bounds the DATABASE'S work. Only the first is observable at this level — with the LIMIT gone the response is still bounded, while the query silently goes back to materialising every pending row before discarding most of them. That is a memory and I/O property with no assertion available here, so it is stated as a coverage boundary rather than papered over with a green that would not have measured it. * docs(reminders): the fire predicate arbitrates against two actors, not one CONVE-23 inside the file the round-3 fix touched. The comment described the UPDATE as an arbiter for concurrent TICKS, which is what it was written for and is why I did not re-read it when asked whether a user edit could race the pass. It now says what it actually defends against, and names the general shape: an arbiter is only an arbiter with respect to the writers it can see. * fix(reminders): codex round 4 — the round-3 bound recreated the round-1 starvation Round 3 bounded the poll surface. Round 4 caught what that bound did: the query took the first N rows and the dashboard then discarded the ones it could not show — hidden items, unauthorised items, completed items — so N such rows hide a visible reminder behind them indefinitely, with no continuation to reach it. That is the SAME defect I had removed from the fire path one round earlier, reintroduced in the read path within the hour. The general form is worth stating because I clearly did not hold it: **a bounded window is only safe when the discarding happens BEFORE the bound.** Filtering above a limit is a starvation every time, and it does not matter what the filter is for. Two halves, because the two filters are not the same kind of thing: **Visibility is now scoped IN SQL**, using the same collection-id / item-id sets every other dashboard section gets through `allItems` — the same three-way shape as ItemListParams, where holding both collection grants and item grants is an OR. Invisible rows no longer occupy the window at all, which is strictly better than filtering them out afterwards and is what the sibling sections have always done. **Terminality is paged**, because SQL cannot evaluate it — a collection's schema defines which statuses are terminal. The collector refills from the next page when a page comes back short, bounded by a max scan so a workspace full of completed items cannot turn a dashboard read into a table scan. The bound is 10x the window: the common shape fills on the first page, and the pathological shape terminates in a fixed number of indexed reads. Stopping at the scan bound reports truncation, which is honest — there may be more, and we did not look. The empty-scope case is a THIRD state that reads like the second: nil CollectionIDs means unrestricted, a non-nil EMPTY slice means this caller sees no collections. Without an explicit guard they collapse, because the switch matches none of its cases at length zero and adds no clause at all — so "nothing visible" would return the whole workspace. Three mutants, one survived: the empty-scope guard, because no dashboard-level test produces that state (callers that would are refused earlier by workspace access). Faithful mutant, missing test — it now has a direct one, with a sanity leg so a build returning nothing cannot pass it by accident. A guard for a state nothing exercises is exactly the one that rots. * fix(reminders): codex round 5 — the MCP action I shipped did not work over stdio **P1: local stdio MCP `remind` was unusable.** cmdhelp derives positionals by regex from a command's `Use` string, and `<instant>` inside `remind <ref> --remind-at <instant>` matched — it became a second REQUIRED positional, so dispatch failed with `missing required argument "instant"`. The action was advertised on a transport where it could not run. **The MCP catalog's own tests did not catch it, and the reason is the finding.** That suite builds its cmdhelp document BY HAND: I wrote `Args: mkArgs("ref")` in it, so the fixture agreed with what I meant rather than with what the CLI says. Five parity and drift tests passed against a document I authored to match my own intention — the "a test that agrees with whatever the table says is not a test of the table" shape, which the canonical-events test warns about in its own comment two packages away. The new test reads the REAL command tree via cmdhelp.Build, which is the only thing in this repo that can disagree with me about what the CLI declares. **P2: `pad project ready` withheld the ack handle** that `next` prints. Showing a fired reminder on the surface an agent polls while withholding the id it needs to retire it means the same entry comes back on every poll, forever. **P2: suggestions asserted a collection they did not have.** The orphan branch admits ANY collection — its own comment claimed it gated on tasks "mirroring the active-plan branch", and that comment was simply false — while the output hardcoded `Collection: "tasks"` and the reason said "Open task". Pre-existing for high-priority items since BUG-1082; my overdue bypass widened it to any overdue item, which is how it surfaced. Fixed by carrying the item's REAL collection rather than by narrowing the branch: narrowing would silently drop the non-task items this has surfaced for a year, and the defect is the mislabelling, not the inclusion. The false comment is replaced with what the code actually does. The first version of that test used an overdue IDEA and SKIPPED — ideas use `new`, and the branch requires `open` or an active status, so it never became a candidate. A test that cannot fire is a failed reconstruction, not a pass; the fixture is now a bug-like collection whose vocabulary contains `open`, which is the population the defect can actually reach. Three mutants, three killed. Forty-one across the unit. * fix(reminders): codex round 6 — reminders fired from soft-deleted workspaces **P1, and the only defect in this unit whose consequence leaves the process.** Workspace soft-delete deliberately keeps items for the 30-day restore window, so the candidate query's filter on the ITEM's deleted_at found nothing wrong — and the tick kept firing, emitting outbound webhook events for a workspace whose owner had deleted it, possibly while deleting their account. Both queries now join workspaces and require `w.deleted_at IS NULL`. Nothing is destroyed: a restored workspace resumes firing, which the test asserts, because "stops firing" and "is destroyed" are very different answers to someone who restores a workspace and only one of them is right. That test first failed for the WRONG REASON and the fixture was at fault: it counted every outbox row in the workspace, and item creation writes its own, so the assertion was satisfiable by the fixture itself and discriminated nothing. Scoped to the reminder event type. **`Use: "remind <ref>"` declared a requirement the command contradicts.** cmdhelp derives the machine-readable arg spec from that string, and `--rearm` takes no ref — so the published contract said "required" for something optional. The requirement is CONDITIONAL, which cmdhelp cannot express, so the honest declaration is `[ref]` plus the explicit check that names both call shapes. The round-5 test grew a `required` column, which is what makes this observable at all: asserting only the arg NAMES would have passed. **The pad_item tool description omitted both new actions.** The params were declared and the actions dispatched, but the prose an agent reads to decide what a tool can do did not mention them — discoverable only by someone who already knew to look. It now describes both, including the two things an agent gets wrong: remind_at is an instant, and nothing but an explicit ack retires a fired reminder. Three mutants, three killed. Forty-four across the unit. * fix(reminders): codex round 7 — one predicate for the scan and the arbiter Third instance of one class, so this fixes the SHAPE rather than the instance. The class: the candidate scan filters on something the fire transaction does not revalidate, so a change committed between them fires a reminder that no longer qualifies. Round 3 was a re-armed instant. Round 1's soft-deleted item was the same thing caught from the other side. Round 7 is a workspace deleted between the scan and the fire — the round-6 fix added the condition to the SCAN only, and the arbiter went on not knowing about it. Fixing those one at a time is what let the third happen. `reminderFireable` is now a single string that both sites reference: the scan asks it and the fire UPDATE re-asks it, so they cannot disagree, and a fourth condition is one edit in one place rather than two edits someone has to remember are paired. Written as a correlated EXISTS on item_reminders.item_id rather than a JOIN precisely so the identical text is valid in both a SELECT and an UPDATE, and the scan drops its table alias so the two uses are the same characters. What deliberately stays outside it: `fired_at IS NULL` and `remind_at <= ?` live on the reminder row itself, are already spelled identically at both sites, and folding them in would need a parameter order the shared form cannot express. Said in the comment so the omission reads as a decision. Both directions are now tested at the arbiter — a workspace deleted mid-pass and an item deleted mid-pass — because the item case previously relied on the item load coming back nil, and someone simplifying the EXISTS down to the workspace check alone would otherwise still see green. Three mutants, three killed: the arbiter dropping the shared predicate, and the predicate dropping each of its two halves. Forty-seven across the unit. * fix(reminders): codex round 8 — workspace export silently dropped every reminder WorkspaceExport is a hand-maintained field list, so a new table joins it only if someone remembers. Reminders did not: a backup/restore, or a SQLite→Postgres migration via `pad db migrate-to-pg`, dropped every pending reminder with nothing in the destination to show anything had gone. The line that list has always drawn is item-scoped workspace CONTENT (comments, links, versions — exported) versus per-user state (stars, watches — not). A reminder has no user column and hangs off an item, which puts it on the exported side. Stating the rule rather than just adding the field, because the next person adding a table needs to know which side they are on. LIFECYCLE MARKS ARE CARRIED, not reset. A fired-and-unacknowledged reminder is still owed to whoever armed it, so it arrives pending; an armed one whose instant has passed fires once on the destination's first tick, which is what would have happened had the workspace never moved. Re-arming everything on import would invent a schedule the user did not set. NULL rather than empty string for the unset marks — the lifecycle is defined by NULL-ness, and "" would make a never-fired reminder read as fired at "". TestMigratedTablesCoversTheExport caught the second half, which I would have missed: `pad db migrate-to-pg`'s NUL preflight decides what to REFUSE on from MigratedTables, so a table the migration copies and the preflight does not know about is a gap in exactly the guard that exists to prevent one. Added there too, with the reason it can never actually fire — every column is machine-produced, so it is listed for coverage rather than expectation — and the "six tables" prose it falsified is now seven. Two mutants, two killed: export dropping the block, and import discarding the marks. Forty-nine across the unit. * test(reminders): state the fire-path invariant and pin it from the invariant The lead's read on why rounds 4 and 7 were the same class: the fire path had no stated invariant, so each fix defended an instance. This states it, and derives the pin from the paragraph rather than from the bug history. THE INVARIANT: the candidate scan is a hint and may be assumed to prove nothing. Every condition that made a row a candidate is re-asserted inside the transaction that marks it fired, in the same statement that does the marking, so checking and writing are one atomic act. Worded as "the scan proves nothing" rather than as a list on purpose — a list invites the next person to add a condition to the scan and stop, which is exactly what happened four times here. TestFirePathInvariant is the pin: one table, one row per scan-side condition, each invalidating that condition in the window between the scan and the fire and asserting the same three things — nothing fires, no event leaves, the reminder is not consumed. The earlier per-defect tests are folded in as rows; they said the same thing one instance at a time, which is how four of these shipped. Adding a fifth condition to the scan without a row here should feel like an omission. It carries a positive control, because four cases that all assert nothing happens would pass against a build that never fires at all. The matrix immediately falsified a claim in the paragraph I had just written. I wrote that the item load inside the transaction is "for the payload, not for the check"; removing the item half of reminderFireable alone changes no observable behaviour, because the load then returns nil and the deferred rollback undoes the write. Item liveness is defended TWICE and a single-mutant experiment cannot say which guard is carrying it — removing both is what kills the test. Both are kept, the predicate is named as primary (the row never matches, so no write happens at all), and the asymmetry is stated: workspace liveness has no second line, which is why dropping ITS half does fail the pin. Six mutants: five singles plus the pair. Five killed alone; the item single survives by design and is documented as such rather than left as an unexplained green. Fifty-five across the unit. * fix(reminders): codex round 9 — one legacy row could hide every reminder **P1: items.item_number is NULLABLE and I scanned it into an int.** Migration 006 added the column to existing rows, so a pre-numbering item still carries NULL — and scanning NULL into an int fails the Scan, which fails the QUERY, which degrades the whole pending-reminder section. One old row, and the feature is dark for everyone in that workspace. ListWatchesForUser, which this query was modelled on, uses sql.NullInt64 for exactly this column. I copied its shape and dropped the part that handles the column's actual nullability — the same way of being wrong as the round-5 cmdhelp fixture: borrowing a form without borrowing what it knows. The legacy row now carries no ref rather than a fabricated "PREFIX-0", which would name a different item. **P1: export shipped reminders that import could only discard.** The items section filters on deleted_at IS NULL, so a soft-deleted item is not in the bundle and its reminder can never be reunited with it. My comment claimed the item_links rationale — round-trip the raw graph so a restore reunites them — which is true for links and false here, because links keep soft-deleted endpoints in the bundle and items do not. A link is a row ABOUT two items; a reminder whose item is absent is a dangling schedule. **P2: import wrote remind_at raw.** Import is a writer, and a bundle is not necessarily one this server produced — hand-edited, or from another instance. A local offset or a bare date would land in the one column every comparison downstream treats as a UTC instant, firing early, late, or never. It now normalizes like every other door. An unparseable value is SKIPPED with a warning rather than failing the restore, matching the lenient import-side precedent already in this file, and the raw value's LENGTH is logged rather than its content. Three mutants, three killed; two needed rewriting because the single-line form did not compile — reverting the nullable scan also requires reverting the render, and dropping the normalization orphans a variable. PROCESS FAULT, recorded because it makes this round's findings weaker than they look: I edited the tree while this review was reading it — committed the invariant work and ran five mutation experiments, which write and restore source, over the same files. A review binds to the tree it read and I moved it underneath. Every finding above was re-verified against the current tree before being acted on, and the next round runs with no concurrent edits. * fix(reminders): codex round 10 — one orphaned item aborted a whole restore An ORPHANED item — one whose collection is missing from the bundle — still gets an itemMap entry. It has to: the entry is written before the skip because parent resolution inside the same loop reads the map for items it has not reached yet. So `itemMap[x] != ""` is satisfied by an id that names no row, and inserting a foreign key to it fails (SQLite enforces FKs here via the DSN's `_pragma=foreign_keys(on)`; Postgres always does). The pre-existing mapping is the sharp edge. The aggravating half was mine: this loop treated a failed reminder insert as FATAL, where item_links and item_versions both skip, so one orphaned item carrying a reminder rolled back an entire 900-item workspace restore. A reminder is the least critical thing in a bundle and it had the strictest failure handling in the file. Both halves fixed: the loop gates on items that actually landed, and a failed insert warns and skips like its siblings. TWO GUARDS THAT ONLY DIE TOGETHER, and this is measured rather than assumed. Reverting either alone leaves the test green — with the map gate restored the skip survives the FK failure, and with the fatal return restored the gate means the insert never fails. Removing both is what fails it. They are kept as a pair because they defend the same failure at different depths (prevent the bad write / survive a bad write arriving some other way), and the pair is recorded in the code so a future reader does not delete one as dead after watching its mutant survive. Second time this shape appeared today; the first was item liveness on the fire path. The bundle in the test is hand-built, because ExportWorkspace cannot produce an orphan — which is the reason it needed a test. That shape only arrives from a hand-edited or foreign bundle, and surviving those is what import is for. Three mutants: two singles that survive by design, plus the pair that kills. Sixty-one across the unit. * fix(reminders): codex round 11 — four contract slips, one of them another unit's **suggested_next returned up to eight entries against a cap of three.** Round 3 prepended reminders PAST the list's own cap, reasoning they should not compete for slots. Every consumer — the web dashboard, `pad project next`, `pad project ready` — is written for three. Worse, it silently falsified a decision recorded elsewhere: BootstrapDashboard deliberately has no suggested_next_overflow_count BECAUSE this list is capped at three upstream, and its comment names raising that cap as the moment to add one. My change made another unit's reasoning wrong in a file I never opened. The combined list is now trimmed back to three, reminders still leading — a reminder can push a task suggestion out, which is the right way round, and the full set stays addressable in pending_reminders. My first version of that trim used `limit`, which is REASSIGNED above to len(candidates) — so on a workspace whose only entries are reminders it would have truncated to zero, killing precisely the case the surface exists for. Caught by reading the surrounding lines before running anything; it has its own test now. **pending_reminders was uncapped in the bootstrap projection.** BootstrapDashboard embeds *DashboardResponse, so every new field joins the boot payload automatically — here, a window of up to 50, which is the budget PLAN-1410 spent a unit trimming. Capped at 5 with an overflow count, under its own constant rather than borrowing bootstrapAttentionCap: they answer different questions and a future change to one must not silently move the other. **Truncation was reported from the wrong question.** The collector used the store's `more` flag, which answers "is there another PAGE", not "did I read all of THIS one" — so a window filling part way through the final page reported that the caller had seen everything while unread rows sat behind the fill point. The paging bounds are now injectable so the case is testable at all: building it with a window of 50 needs ~75 rows in a specific pattern, with a window of 3 it is four. **Import accepted acked-without-fired**, which is not one of the lifecycle's three states. Such a row fires, is excluded from the pending surface because it is already acked, and can never be acknowledged because AckReminder requires acked_at IS NULL — an event emitted into permanent invisibility. The acknowledgement is dropped and the schedule kept, since an ack of something that never fired means nothing. Five mutants, five killed (one rewritten — removing the flag orphans a variable). Sixty-six across the unit. * fix(reminders): codex round 12 — a read is not a hold; scope the arm; ack from the ack Four P2s from round 12 (two independent runs, both landing on the same line of the fire path), each closed at the layer where it lives: - fireOneReminder pins the item and workspace rows FOR NO KEY UPDATE on Postgres before the arbiter UPDATE. reminderFireable re-asserted liveness at the predicate's instant and nothing held it to the commit instant; under READ COMMITTED an archival could commit in between and the event left the process about a deleted resource. Same idiom and same lock strength as CreateAttachmentForLiveItem; SQLite is excluded by its BEGIN IMMEDIATE, not skipped for convenience. Two PG-only pins verify "blocked" in pg_stat_activity, not by elapsed time; the pin-removed mutant fails both. - CreateReminder asserts "live item of THIS workspace" in the INSERT's own SELECT and returns ErrReminderItemGone otherwise. The table had an FK and no same-workspace constraint; a mismatched pair fed another workspace's title to this one's dashboard and webhooks. Handler maps it to 404. - AckReminder matches every fired row (COALESCE keeps the first ack, updated_at moves only when acked_at does), so a no-match means exactly "not fired at the instant of the ack". The handler no longer decides 409-vs-200 from the row it read before the UPDATE. - The invariant paragraph gains its missing sentence: "at that instant" means the commit instant, and the pin is what makes the predicate's instant and the commit instant the same one. Round-12 caveat carried: both runs were static reads (sandbox blocked Go's build cache), so "four" is a floor, not a measurement. Refs IDEA-2641 * fix(reminders): codex round 13 — a reminder's workspace must agree with its item's, at every read Every reader scoped by r.workspace_id and then joined the item without asserting the two agree. No door writes a disagreeing row today (CreateReminder derives the pair from the item; import maps within the workspace), and the table has nothing that forbids one — so a hand-edited bundle, a future move door, or a direct write would carry one workspace's item into another's dashboard, export, and webhooks. The identity goes into reminderFireable (scan + arbiter), the Postgres row pin, ListPendingReminders and the export query. One test writes the row raw — the only way one can exist — and asserts it is inert at each site; the predicate-removed mutant scans and fires it. Refs IDEA-2641 * fix(reminders): codex round 14 — the by-id and by-item reads assert the same identity as every other read GetReminder scoped by the row's own workspace_id and ListRemindersForItem by item_id alone, so a row whose two columns disagree — the class rounds 12 and 13 closed at the scan, the arbiter, the pin, the pending surface and the export — was still readable through the two reads that reach a single row. reminderOwned is that identity on its own, without the liveness half those two reads must not have (a fired reminder on an archived item is history worth showing). The write paths reach a row only through GetReminder, so scoping it scopes them; a row no door can write needs no door to delete it. ListRemindersForItem now takes the workspace its caller already resolved the item in. The raw-row test asserts both reads refuse the row from both sides; the reminderOwned-removed mutant surfaces it through GetReminder. Refs IDEA-2641 * fix(reminders): codex round 16 — an archived item's reminders are readable, and its verbs say "archived" The doors resolved the item live. Listing an archived item's reminders answered 409 from a GET, and ack/re-arm/delete answered a bare 404 for a reminder that exists on an item that exists — while the store, since round 14, deliberately keeps that history readable. The API already has a posture for archived items: GET reads them, mutations answer 409 "archived … restore it before editing" (writeItemResolveError). The list now follows handleGetItem; the lifecycle verbs load the item include-deleted, run the visibility check first, and then answer the same 409 every other item mutation does. One test walks archive → list 200 / ack 409 / arm 409 → restore → ack 200 on the same rows. Refs IDEA-2641 * fix(reminders): codex round 17 — one suggestion per item, the archived 409 by slug, and the door courtesy named Three findings on the server pass. (1) An item that was both a fired reminder and an ordinary candidate appeared in suggested_next twice; the ordinary entry is dropped, the reminder entry (which carries the ack id) stays, and two reminders on one item remain two entries. (2) Round 16's 409 for an archived item's reminder was written by re-resolving item.Ref, which is derived and empty for a legacy item with no item_number — so the class most likely to be legacy fell through to a bare 404. The slug is handed over instead. (3) The archived check in resolveReminderForWrite is check-then-write, and an archive landing in between lets the verb through: accepted and documented — it is the posture of every item mutation here (UpdateItem's UPDATE has no liveness clause), the outcome is benign, and putting liveness in AckReminder's WHERE would re-create the no-match ambiguity round 12 removed. Refs IDEA-2641 |
||
|
xarmian
|
e94e9afbea |
Merge pull request #1240 from PerpetualSoftware/fix/bug-2850-field-coercion
fix(server,mcp,cli): type field values server-side; carry the fields object natively (BUG-2850) |
||
|
xarmian
|
80be76a3ce |
docs(mcp): the detectFieldConflicts header stated the pre-round-14 reach (BUG-2850)
Comment-only. The lead caught it in the package review. The "SCOPE:" paragraph still said the pass runs only when a `fields` object is present, and that a top-level-vs-`field:[]` collision without one is outside it. Round 14 falsified both halves — the pass runs from both action entry points on every call — and the body's own comment said so while the header contradicted it. On the one boundary this loop spent eighteen rounds on, the header is what a future reader trusts. CONVE-23 is exactly this and I missed it: the round-14 commit swept the version.go prose and the test comments, and left the header of the function it had just changed. The paragraph now states the actual reach: both entry points regardless of `fields`; alias collisions adjudicated always and refused even on equal values; same-name collisions adjudicated only when the `fields` object carries THAT key, which is a per-key question; the round-7 exemption as the sole carve-out, itself narrowed to keys the CLI can express (the compat IDs refuse, but only when a top-level compat value is present), with padded entries outside it. It closes with the instruction the loop earned: say a new condition's QUANTIFIER out loud before writing it. Rounds 15, 16, 17, 19, 20 and 21 were each that question answered by assumption. gofmt clean · go vet clean · go test ./internal/mcp/ ./cmd/pad/ green |
||
|
xarmian
|
9ecc59af1e |
fix(store): refuse a schema with trailing content instead of truncating it (BUG-2873)
Codex round 3, one P2. `json.Decoder.Decode` stops at the end of the FIRST value and ignores whatever follows, where `json.Unmarshal` refuses it — so a stored schema with junk after the object would be silently truncated by the rewrite. It is now treated as unparseable and left alone, the same posture as any other schema this migration cannot faithfully reproduce. **The Postgres gate then failed the new test, and the failure is the finding.** It failed at the SEED, not the assertion: `ERROR: invalid input syntax for type json (SQLSTATE 22P02)`. `collections.schema` is TEXT on SQLite (`005_collections.sql:10`) and JSONB on Postgres (`pgmigrations/001_initial.sql:114`), so a value with trailing content cannot be STORED on Postgres at all. The state this guard defends against is reachable on one dialect and forbidden by the column type on the other. So the test skips on Postgres with that reason recorded. Asserting there would be asserting about a state that cannot exist — and reading WHICH LINE failed is what separated "my test is not portable" from "the product is broken on PG". **Second instance of the mutation harness reporting a false survivor**, same cause as the last: deleting the guard leaves `io` unused, the mutant fails to compile, and counting `--- FAIL` lines sees zero. With `_ = err` in place of the return it dies immediately. Twice in one unit makes it a harness defect, not bad luck: a runner that counts test failures must check the BUILD separately, or every non-compiling mutant reads as a hole in the tests. Mutation matrix 7 of 7 killed. Gates: `gofmt` clean, `go vet ./...` ok, full `go test ./...` green on SQLite, full `internal/store` green on Postgres (502.2s, private container at 127.0.0.1:5473, detached with a sentinel). |
||
|
xarmian
|
a1b8e63a29 |
fix(mcp): a nil top-level value is absence, for every key (BUG-2850)
Codex round 21, one P2 and no P1 — a false refusal, and the finding
named a strict subset of it.
topLevelValueProvided returned true unconditionally for the compat IDs,
and fell through to true for everything else, so a nil counted as a
supplied value and refused against a `fields` entry for the same key.
Nothing writes a nil: the HTTP mapper's `.(string)` assertion drops it
and BuildCLIArgs has no flag value to emit, so both doors resolve to the
`fields` value.
The finding named `assigned_user_id` / `agent_role_id`. Probing the
population first — the habit this unit has been beating into me — showed
all five top-level keys behaving identically, because the non-compat
path fell through to `return true` as well. Fixing the named pair alone
would have left `status: null` refusing.
Mutation matrix, three directions:
remove the nil check -> all five key legs fail
fix ONLY the named compat pair -> status / priority / parent fail
treat the empty compat clear
as absence too -> both clear-semantics tests fail
The middle mutant is the population-versus-instance distinction made
executable: a fix that satisfies the reviewer's example and nothing else
is red, by name, in three legs.
Gates: gofmt clean · go vet clean · go test ./... green (29 packages) ·
contract-drift gate green
|
||
|
xarmian
|
499387e99d |
fix(store): never regress a migrated token; keep large integers intact (BUG-2873)
Codex round 2: four findings, two fixed here and two filed as their own items.
**Migrated siblings' OCC tokens could REGRESS.** A sibling updated between this
rename's timestamp and the scan already holds a newer `updated_at`; stamping the
rename's value on it moved the token BACKWARDS — breaking the strictly-increasing
invariant the transaction above exists to maintain, and re-validating a token the
client should have lost. Each rewritten row now takes `max(current + 1ns,
renameToken)`, computed in Go from the value read under the row lock rather than
by comparing timestamp TEXT, which the existing comment warns is never safe.
**Large integers in unknown properties were corrupted.** Round 1 fixed the typed
round-trip dropping unknown keys, but decoding into `interface{}` turns every
JSON number into float64, so `9007199254740993` came back CHANGED. A rename would
silently damage a property it exists only to carry through. `UseNumber` keeps the
literal text.
## Filed, not absorbed — both because their dependents are not the relation feature
- **BUG-2875** — a collection CREATED during a rename escapes the scan's
`FOR UPDATE` and keeps a relation aimed at the old slug. Closing it means
`CreateCollection` takes the workspace lock, which changes the concurrency
behaviour of every collection creation on the instance. Same reasoning that
split IDEA-2874 out; this unit's reviewability rests on affecting zero live rows.
- **IDEA-2876** — migrated siblings emit no `collection_updated` event, so an open
page keeps the pre-rename schema until reload. Handler/event layer; the store
publishes nothing.
## The mutation harness was reporting a false survivor
Counting `--- FAIL` lines treats a mutant that FAILS TO COMPILE as one that
survived — zero failures either way. Removing the token guard leaves
`rowUpdatedAt` and `renameToken` unused, so that is exactly what happened, and it
read as "the guard is untested". With a compiling mutant (`_ = rowUpdatedAt`) it
dies immediately. Worth stating because the failure mode is silent and points the
wrong way: it invents doubt about code that is fine, and would equally hide a
real survivor behind an unrelated build break.
Mutation matrix 6 of 6 killed.
Gates: `gofmt` clean, `go vet ./...` ok, full `go test ./...` green on SQLite,
full `internal/store` green on Postgres — 460.9s, private container at
127.0.0.1:5473, run detached with a sentinel.
|
||
|
xarmian
|
a9f2405903 |
fix(mcp): the compat exception turns on a top-level value, not on the key (BUG-2850)
Codex round 20, one P2 and no P1 — another false refusal from a reason
of mine applied past the source it was verified on.
Round 15's reason was specific: a TOP-LEVEL compat param has no CLI
flag, so BuildCLIArgs drops it while HTTP reads it, and the doors
receive different writes. I then keyed the exception on the KEY being a
compat one, which caught `field:["assigned_user_id=A",
"assigned_user_id=B"]` — two array entries, no top-level value, no
asymmetry: both doors keep the last and lift the same column. Refused a
call that resolves deterministically.
The gate now asks whether a top-level compat value is actually present,
which is the condition the reason describes.
Mutation matrix, both directions:
broaden it back to any compat-keyed contribution -> only the two-entry legs fail
drop the exception entirely -> only the top-level legs fail
(round 15's defect returns)
Round 15's own case is a leg of the new test deliberately: without it
this pin would pass on a build that dropped the compat exception
altogether, which is the defect round 15 existed to fix.
Gates: gofmt clean · go vet clean · go test ./... green (29 packages) ·
contract-drift gate green
|
||
|
xarmian
|
6428e7db31 |
fix(store): lock, stamp and preserve on the relation retarget (BUG-2873)
Codex round 1: four findings, three P1, all real. **The migrated siblings' concurrency token was not advanced.** `collections.updated_at` doubles as the OCC token (BUG-2265), so rewriting a sibling's schema without touching it left a client holding the PRE-rename schema — and a token that still matched — able to write it straight back and undo the migration. Every rewritten row now takes the rename's own token, so the whole rename shares one instant. Pinned by asserting the stale token now 409s. **The scan did not lock the rows it rewrites.** A concurrent schema update to a sibling could commit between the SELECT and the UPDATE, and this transaction would then overwrite the newer schema with its stale copy. `FOR UPDATE` on Postgres, ordered by id so the multi-row acquisition is deterministic; SQLite is covered by its BEGIN IMMEDIATE write lock. **The old slug came from the pre-transaction snapshot.** Two tokenless concurrent renames of the same collection both read the ORIGINAL slug outside the lock; the loser would migrate `original -> its own new slug` while the relations already said the WINNER's, matching nothing and stranding them at a name no collection holds. The slug is now re-read alongside the token under the row lock. This is the READ — the ALLOCATION of the new slug is still outside the transaction and still IDEA-2874's, deliberately. **Re-marshaling through `models.CollectionSchema` dropped unknown properties.** That struct has fixed fields, so unmarshal+marshal silently erased anything it does not declare — a rename would quietly strip forward-compatible metadata from every relation-bearing schema in the workspace. It now edits the raw decoded JSON, touching only `fields[i].collection`. ## Two instruments that were not instruments Both found by mutation, not by reading: - **The deadlock test passed against its own mutant in 0.44s.** Two unsynchronised goroutines never collided. With a start barrier and 40 rounds it now fails in 1.4s with `ERROR: deadlock detected (SQLSTATE 40P01)` — so Rook's hazard was reproducible, not theoretical. - **The pre-tx-slug mutant survived the first matrix**, because nothing forced the interleaving. Rather than call it untestable, it is pinned by an end-state invariant that holds under ANY interleaving — whatever slug the collection ends up with, every relation aimed at it points there — over 40 concurrent rounds. It fails at round 1 under the mutant. Mutation matrix 4 of 4 killed. Gates: `gofmt` clean, `go vet ./...` ok, full `go test ./...` green on SQLite, and the full `internal/store` suite green on **Postgres** — 448.6s on a private container at 127.0.0.1:5473, never the shared 5445 a sibling seat may tear down. Run detached with a sentinel after the first attempt was killed at a turn boundary; the harness kills backgrounded tasks, it does not kill disowned ones. |
||
|
xarmian
|
052850f613 |
fix(mcp): both gates ask the per-key question; compare like with like (BUG-2850)
Codex round 19, two P2 and no P1. Both were FALSE REFUSALS my own fixes
introduced — the first is round 17's mistake in the sibling gate.
[P2] THE SAME-NAME GATE WAS STILL PER-REQUEST. Round 17 made the padded
gate per-key and left this one asking whether the request has any
`fields` object. With `fields:{"other":"x"}` and
`field:["effort=l","effort=s"]`, `effort` is not in the object, nothing
arbitrates it but the doors themselves, and both keep the last entry —
so a call that resolves deterministically was refused. The predicate is
now `canonicalized`, the same per-key question the other gate asks.
`fieldsPresent` no longer exists anywhere in the pass, and its absence
is commented as the fix's shape: a future gate reaching for "does the
request have a fields object" is almost certainly this mistake a third
time.
[P2] TRIMMED AND UNTRIMMED VALUES WERE COMPARED. Entry values are
trimmed for comparison because ingestFieldKVP trims them; the `fields`
object's value was compared raw. `fields:{"note":" x "}` with
`field:["note= x "]` read as " x " vs "x" and refused, though both doors
write " x ". Only the COMPARISON key is trimmed now — `raw` and the
re-emitted wire value keep the caller's whitespace.
Mutation matrix:
same-name gate back to per-request -> only the unrelated-key leg fails
stop trimming for comparison -> only the whitespace-equal leg fails
trim the EMITTED value too -> only the re-emission leg fails
THE THIRD MUTANT SURVIVED AT FIRST, and it was unreachable rather than
unobserved: the whitespace test's entry is already canonical, so the
re-emission path never ran and a mutant trimming the emitted value
changed nothing it could see. Added a leg whose KEY is padded, which
forces the re-emission, and asserted the emitted value still carries the
caller's whitespace. Third time this loop that asking "is the mutant
faithful" before "is the test weak" found a real hole (CONVE-28).
Control legs, both directions: a `fields` object that DOES carry the key
still refuses differing values, and genuinely different values are still
refused however they are padded.
Gates: gofmt clean · go vet clean · go test ./... green (29 packages) ·
contract-drift gate green
|
||
|
xarmian
|
4687c46f94 |
fix(store): migrate relation fields when their target collection is renamed (BUG-2873)
`models.FieldDef.Collection` holds the target's SLUG, and it is the ONLY pointer a relation field carries — there is no id beside it to fall back on. `UpdateCollection` re-slugifies on rename and nothing migrated the definitions aimed at the renamed collection, so every relation field pointing at it was stranded: the picker filters on a slug that resolves to nothing and the field silently stops being fillable. `retargetRelationFieldsTx` re-points them in the SAME transaction as the rename, for the reason the field-value migrations already run there: a failure must roll the rename back rather than commit collections pointing at a slug that no longer exists. **It parses instead of string-replacing.** A schema's JSON contains the old slug in places that must not move — a text field's `default`, a select's `options`, a label. Only `FieldDef.Collection` on a `relation` field is a reference. Export's `remapFieldIDs` gets away with a blind replace because it substitutes UUIDs, which cannot collide with prose; a slug is a word. A control test pins that. **The renamed collection is included deliberately** — a relation targeting ITSELF needs the same rewrite — and the rewrite lands after the caller's own `schema` write in the transaction, so a simultaneous schema edit composes rather than being reverted. Both have tests. ## The deadlock hazard, and why the existing comment does not cover it The lock-order comment above this transaction is a Codex P1 fix that orders the workspace lock against ONE collection row lock, because until now nothing took more than one. This change writes SIBLING collection rows, so two concurrent renames of mutually-referencing collections take those locks in opposite orders. **Reproduced, not theorised:** with the serialization removed, the test fails in 1.4s with `ERROR: deadlock detected (SQLSTATE 40P01)` on Postgres. Renames now take the workspace lock — previously acquired only when `len(input.Migrations) > 0` — BEFORE the row lock, which closes it without inventing a second ordering rule to keep in sync with the first. **The first version of that test was not an instrument.** Two unsynchronised goroutines passed against the same mutant in 0.44s, having simply never collided. It takes a start barrier and 40 rounds to be evidence. ## Scope The out-of-tx slug allocation (`uniqueSlugExcluding(s.db, …)` at :420, before `s.db.Begin()` at :503) is deliberately NOT touched — filed as IDEA-2874. Its dependents are every collection rename in every workspace, not the relation feature, so it does not belong in a change whose reviewability rests on affecting zero live rows. `UNIQUE(workspace_id, slug)` makes today's behaviour loud rather than lossy, so it can wait. A census found ZERO relation fields across all 11 accessible workspaces on this instance, and no shipped template declares one — this repairs the rename path before PLAN-2857 creates the population, which is why a migration is not needed. Gates: `gofmt` clean, `go vet ./...` ok, `go build ./...` ok, full `go test ./...` green on SQLite, and the full `internal/store` suite green on **Postgres** (private container on 127.0.0.1:5473, never the shared 5445 a sibling seat may tear down). Pin written and run BEFORE the fix per team CONVE-29: 2 propagation tests failed, the control passed. |
||
|
xarmian
|
21a3057389 |
fix(mcp): keep per-entry multiplicity in the conflict pass (BUG-2850)
Codex round 18, one P2 and no P1 — and the fix is upstream of the rules rather than another rule. parseFieldArray indexes by NORMALIZED key, so two entries naming one key collapsed into a single index slot, and this pass walked that index. Its own input was lossy: `field:["effort=l", " effort=l"]` arrived as ONE contribution, fell under the len < 2 early exit, and passed unchecked — HTTP trims both to `effort` while stdio writes `effort` AND a junk `" effort"`. The pass claims to adjudicate one canonical key offered by multiple sources; two array entries ARE multiple sources, and it could not see them. It now walks the raw entries, so multiplicity survives and the existing rules apply unchanged — no new branch. The index is deliberately discarded here and the discard is commented, because reaching for it is the natural thing to do next. I NEARLY CHANGED THE CODE TO SATISFY A WRONG TEST. The third leg was first written asserting that two canonical entries with DIFFERING values are refused. The code disagreed, and the code was right: ingestFieldKVP (HTTP) and the --field loop in cmd_item.go (CLI) both do `map[key] = val` in entry order, so each door keeps the LAST entry and they agree. That is the round-7 boundary exactly — a visible duplicate with a resolution the caller can predict — and refusing it would have contradicted the boundary the lead confirmed, on two doors pinned to agree. Verified by reading both loops before touching anything. The leg is kept, inverted, because it is the one that stops a future "refuse every repeated key" simplification from looking correct. Mutation: restoring the collapse (walk one contribution per key) fails exactly the padded-twin leg, and leaves the two control legs green. Gates: gofmt clean · go vet clean · go test ./... green (29 packages) · contract-drift gate green (go test ./internal/mcp/ -run 'CoversEveryCatalogAction|VersionMatchesToolSurface') |