mirror of
https://github.com/PerpetualSoftware/pad.git
synced 2026-10-26 14:30:13 +00:00
9e4ffe5cee674446fb4e927577c95121b64a00a0
119 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 xarmian
|
d4ec587de9 |
fix(cli): a CLI call during an install-refresh bounce waits for the refresh instead of auto-starting a server over it (BUG-3215) (#1544)
install-refresh.sh stops the server, installs, and restarts it with the
captured argv. Any pad CLI call in between found nothing answering and
auto-started a server of its own: the caller's binary, cwd and default
flags. That server held the port against the refresh's restart (day 79:
127.0.0.1:7777 went dark and the script exited 1).
- The script writes <DataDir>/refresh.lock {pid, port, deadline} by rename
before the stop and removes it in its EXIT trap, only while it is still
its own. The deadline is 10s + PAD_PROBE_TIMEOUT (30s by default).
- EnsureServer defers to a live marker for its own port (writer pid alive
AND deadline not passed): it polls health and never spawns. If the
writer is still alive past the deadline it errors naming the pid. A dead
writer, a passed deadline, another port or an unparseable file means
the marker is ignored.
- The script names a holder that took the port before the restart spawns
(and does not spawn it), and refuses success when the listener after
the probe is not the restart it spawned. A health probe answers for
whoever holds the address.
- The script-test dependency guard now reads every install_refresh*_test.go.
Claude-Session: https://claude.ai/code/session_01A3rXV7GDLAhBe9mL4V6z2m
|
||
|
xarmian
|
fd95b79694 |
fix(BUG-2367): moves and copies never carry into a computed field, and a carried unique-field collision is dropped and named instead of 500ing or storing a duplicate (#1514)
MigrateFields matched fields by key and type only. On every migrate door (single move, bulk move, cross-workspace copy and its preflight): - a value was carried into a destination field marked `computed`. It is now dropped, with preflight reason `target_computed`. - a value colliding on a destination `unique_scope` field answered 500 on single move (invocation_slug, which has an index), leaked the SQL error through bulk move, and stored a duplicate on an unindexed unique field. Provenance decides now, as TASK-2878 did for relation referents: a CARRIED collision is dropped and reported as `not_unique`, and an override or schema default that collides is refused with 409 conflict. A required field emptied by the drop is refused as required. One store check (UniqueFieldConflictsQ / DropCarriedUniqueCollisionsQ) serves all four doors, including the tx-bound copy. The report names the value, and names the item holding it only when the caller may see that item. The activity row always uses the holder-free sentence. Reporting, all additive and omitempty: warnings.not_unique on the move response, the copy response and bulk updated[] rows; `detail` on the preflight's dropped row; CLI copy dry run and result, plus ` dropped:` lines on `item move` stdout; web copy dialog, move and copy toasts, and the timeline move card. ToolSurfaceVersion 0.51 (behaviour bump). The relation leg of the original report was already closed by TASK-2878. Item 4 (terminal/abandoned state change) is PR 2. Out of scope and filed separately: BUG-3200 (missing needs_value routes) and BUG-3202 (float64 precision; the migrateCopyFields comment now points there). Claude-Session: https://claude.ai/code/session_01RYiqbqXmxQMoKHsGm8B77N |
||
|
xarmian
|
df5531b815 |
fix(cli): pad server stop keeps a live legacy PID file instead of deleting it, and clears a dead one (BUG-2970) (#1485)
* fix(cli): pad server stop keeps a legacy bare-pid file instead of deleting a live server's record (BUG-2970) On Unix, pidFileOwner read a free lock as proof that a record was stale, and removed it. That holds for records this build writes, because their server holds the lock for its whole life. It does not hold for a bare-pid file written before BUG-2969: no build that wrote that form took a lock, so a free lock says nothing about its server, which may be live. The day-19 refresh deleted exactly such a record. A record without a fingerprint (zero StartedAt, which is what a bare pid parses to) is now unprovable and left in place, the answer Windows already gave for the same shape. Fingerprinted records are unchanged. Three existing fixtures modelled fingerprinted records without a StartedAt, which no build writes; they now carry one. Claude-Session: https://claude.ai/code/session_01RYiqbqXmxQMoKHsGm8B77N * fix(cli): a legacy PID file naming a dead process is stale and removed; start is never blocked by a kept one (BUG-2970) The kept, unprovable state now applies only while the legacy record's pid exists. A pid with no process behind it cannot be anyone's server, so that record is stale by the same proof as an unheld fingerprinted one, and is removed under the lock. Stop then says "server not running" instead of refusing forever. Tests pin both halves of the way out. Start is never blocked: EnsureServer probes the port, and the new server's ClaimPIDFile takes over a kept legacy file and writes a fingerprinted record stop can address. And the kept refusal names its manual remedy (remove <path>). Claude-Session: https://claude.ai/code/session_01RYiqbqXmxQMoKHsGm8B77N |
||
|
xarmian
|
71de56f6de |
fix(relations): a relation value stored as text reads as text, not as an unavailable target (BUG-3014, ToolSurfaceVersion 0.50) (#1484)
* fix(relations): a relation value stored as text reads as text, not as an unavailable target (BUG-3014, ToolSurfaceVersion 0.50)
HydrateRelationTargetsQ treated every stored relation value as an id. A value
that is not UUID-shaped (a title a bundle import carries verbatim, which is that
door's contract; or legacy free text) resolved to nothing and hydrated as an
id-only entry, which means "the target is gone or not visible". So an imported
workspace showed relations as broken when the title might name a live item.
Such an entry now carries an additive, omitempty `stored_as_text: true`. The
predicate is syntactic ("not UUID-shaped"), decided from the stored bytes alone
and never from a lookup, so the flag discloses nothing about any item, and it is
deliberately not coupled to the resolver's ladder (lead ruling). A UUID-shaped
value, resolved or not, marshals byte-identically to 0.49, and a UUID that fails
to resolve is never marked.
The CLI renders `"<text>" (text, not a reference)`. The web chip, which resolves
locally from its index rather than from relation_targets, uses the same
syntactic predicate and says the same. The MCP instructions tell agents to
render a marked entry as text.
Claude-Session: https://claude.ai/code/session_01RYiqbqXmxQMoKHsGm8B77N
* test(relations): one shared vector table pins the Go and web stored_as_text predicates together (BUG-3014)
The web carries its own copy of the not-UUID predicate, because it resolves
relation chips from its local index rather than from relation_targets. Two
implementations of one contract rule drift. Both now read
internal/store/testdata/relation_stored_as_text_cases.json (21 cases: upper
and mixed case, braces, no dashes, URN form, wrong dash positions, length off
by one, titles, refs, blanks, and padding), and their KEEP IN SYNC comments
point at each other.
The table found a real divergence: the web trimmed with String.prototype.trim,
which disagrees with Go's strings.TrimSpace on U+0085 (Go only) and U+FEFF (JS
only). The web now trims with serverTrimmedTitle, the Go-parity helper the title
limit already uses. Swapping JS trim back in fails exactly those two vectors,
and dropping Go's TrimSpace fails the five padded ones.
Claude-Session: https://claude.ai/code/session_01RYiqbqXmxQMoKHsGm8B77N
|
||
|
xarmian
|
919c2975c2 |
fix(items): note and decide append server-side under the write lock, so they no longer revert a concurrent field write (BUG-3056) (#1458)
* wip(BUG-3056): server-side note/decide append, capability-gated CLI * test(BUG-3056): server append tests * test(BUG-3056): CLI capability gate tests * test(BUG-3056): store append tests on both backends; CLAUDE.md API note * fix(BUG-3056): keep the remote /mcp door's created_by label (codex r1) * docs(BUG-3056): CLAUDE.md names the remote door's author label |
||
|
xarmian
|
969955c810 |
fix(mcp): a 413 is too_large on both transports, not server_error (BUG-2829) (#1455)
* fix(mcp): a 413 is too_large on both transports, not server_error (BUG-2829) Claude-Session: https://claude.ai/code/session_01RYiqbqXmxQMoKHsGm8B77N * docs(mcp): tool surface 0.45 in instructions, README and CLAUDE.md (BUG-2829) Claude-Session: https://claude.ai/code/session_01RYiqbqXmxQMoKHsGm8B77N |
||
|
xarmian
|
88a5af1a74 |
fix(mcp): stdio passes "-"-leading free text as text, and reports argv refusals and 429s with the right code (BUG-3142, BUG-3147) (#1444)
* fix(mcp): stdio sends positionals behind "--", so free text starting with "-" is no longer parsed as a flag (BUG-3142) BuildCLIArgs emitted positionals first with no terminator, so cobra read `item search "-ship it"` as an unknown shorthand flag and refused the call before the command ran, on all 12 free-text positionals the stdio transport carries. It now emits every flag (root flags, --workspace, --format included) and then `--` and the positionals; the remote door ignores this vector. Pinned against the real cobra tree through each command's own flag parser, including a positional that is literally "--workspace". Claude-Session: https://claude.ai/code/session_01RYiqbqXmxQMoKHsGm8B77N * fix(mcp): stdio reports an argv refusal as validation_failed, with its message, instead of a retryable server_error (BUG-3142) Two defects on one path. The root command's flag-error hook prints cobra's usage block BEFORE the "Error:" line, and stripCobraUsageBlock truncated at "Usage:", so a flag-parse refusal reached the classifier as an empty string. And no stderr pattern matched cobra's or pflag's argv refusals at all. The strip now stops at a following "Error:" line, and one anchored pattern covers the parse-time family enumerated from cobra v1.10.2 and pflag v1.0.10. Pinned end to end: the test binary re-executes as the real CLI under ExecDispatcher, so the real stderr ordering is what gets classified. Claude-Session: https://claude.ai/code/session_01RYiqbqXmxQMoKHsGm8B77N * fix(mcp): a 429 reaches a stdio caller as rate_limited with the server's wait, and the tool surface is 0.42 (BUG-3147, BUG-3142) The CLI printed only the limiter's message, which no stderr pattern matched, so every 429 reached a stdio MCP caller as a retryable server_error while the remote transport said rate_limited. The CLI now writes the structured marker line at its root for a 429 on any command, carrying Retry-After as details.retry_after_seconds, and stdio lifts it. ToolSurfaceVersion 0.42 covers this and BUG-3142's classification change in one entry: both change the code a stdio caller receives, and nothing else. Claude-Session: https://claude.ai/code/session_01RYiqbqXmxQMoKHsGm8B77N * test(cmd/pad): the stdio argv test restores the root flag globals it parses into (BUG-3142) Claude-Session: https://claude.ai/code/session_01RYiqbqXmxQMoKHsGm8B77N |
||
|
xarmian
|
6fdd8816f3 |
feat(playbook): match free text to an active playbook via the decision provider (TASK-3120) (#1429)
* feat(server): POST /playbooks/match — typed-decision Choice over active playbooks (TASK-3120)
PLAN-3114 unit 5. A synchronous, read-only endpoint: given free text, asks
the workspace's typed-decision provider which active (non-draft,
non-deprecated) playbook it best matches, with a reserved "none" option so
unrelated text gets an honest answer instead of a forced pick — and, as a
side effect, keeps a workspace with exactly one active playbook askable
(decision.Question.Validate needs >=2 Choice options).
- decision.Runner gains a nil-safe Provider() accessor: this endpoint needs
a plain synchronous Ask, not the owed-jobs pipeline Evaluate drives.
- No provider configured -> 404 decision_provider_unavailable (callers fall
back to slug/trigger routing). A provider error or an out-of-set answer
the provider was never offered -> 502 decision_provider_error, distinct
from the 404 case so a caller can tell "try again" from "not configured".
Zero active playbooks -> 200 with a stated reason, no provider call.
Over the provider's 255-option ceiling (including "none") -> refused
rather than silently dropping playbooks and answering from an
incomplete set.
- listPlaybookItems is factored out of handleListPlaybooks so list and
match share one visibility-scoped item source and can't drift on which
playbooks a caller can see.
- 30s timeout on the synchronous provider call, tighter than the
provider's own worst-case retry budget (~330s) since an HTTP caller
blocks on this one directly, unlike the async decision tick.
- content_state_population_test.go's BUG-3033 sweep gets a new ruling: the
PlaybookSummary call here feeds the provider prompt only, never the
response, so there's no reader-facing text to mark.
Claude-Session: https://claude.ai/code/session_01ByMjAhvdWgbAWWeGE9jwWd
* feat(cli): pad playbook match "<text>" (TASK-3120)
CLI + client-method half of PLAN-3114 unit 5 — POST /playbooks/match.
Table output prints the matched ref/title (or "none"), confidence, model,
and a probabilities table sorted descending; --format json passes the
response through unmodified. Read-only, side-effect-free.
Claude-Session: https://claude.ai/code/session_01ByMjAhvdWgbAWWeGE9jwWd
* docs(skill): route to playbook match as rung 3, offer-only (TASK-3120)
PLAN-3114 unit 5's skill half. Exact invocation-slug match and trigger
match stay rungs 1-2; match is consulted only when the user is plainly
asking for a procedure/workflow and both of those missed — never on
ordinary conversation, since every call spends provider money. Any match
returned is offered, never auto-run, at any confidence: TASK-3137's replay
of this same provider's attention question set found no validated
threshold, so nothing here invents one for this question either. A 404
(no provider configured), a "none" choice, or any other error falls back
silently to today's slug/trigger-only behavior.
Claude-Session: https://claude.ai/code/session_01ByMjAhvdWgbAWWeGE9jwWd
* chore(mcp): de-indent padItemToolDescription's continuation lines (TASK-3120)
Whitespace-only — git diff -w on this file is empty. The description's
continuation lines carried 18 leading spaces purely for visual alignment
under each action's " name — " bullet in the Go source; that padding is
emitted verbatim into the tools/list JSON, where it conveys nothing an
LLM reading the tool description gets any value from.
De-indented by exactly 14 spaces on every line that had >= 18 (all 116 of
them, uniformly, since the file carries only three indentation levels:
0, 2, 18 — no deeper nesting to preserve a relative offset for).
Continuations now land at 4 (bullet's 2 + 2), so a reader can still tell
which action a line belongs to; a 1-space collapse would have inverted
that (continuation LESS indented than its own bullet).
tools/list: 55290 -> 53666 bytes (-1624). initialize: unchanged at 24530
(it carries no tool descriptions). Frees headroom against the 55296
budget for the pad_playbook.match action landing in the next commit —
budget constants themselves are untouched, per standing instruction.
Claude-Session: https://claude.ai/code/session_01ByMjAhvdWgbAWWeGE9jwWd
* feat(mcp): pad_playbook.match action, ToolSurfaceVersion 0.39 -> 0.41 (TASK-3120)
PLAN-3114 unit 5's MCP half, landing on top of the whitespace-only
catalog_item.go trim (previous commit) so this stays within the
tools/list budget with room to spare.
- catalog_playbook.go: new `text` param, new `match` action
(passThrough — the CLI's positional arg is named "text", matching
this tool's param, so no custom shaping is needed) plus a
description bullet in padPlaybookToolDescription.
- tool_surface.go: `match` marked read-only (nothing stored, no job
enqueued).
- dispatch_http_routes.go: `playbook match` routeSpec (POST, flat
{text} body) so the remote /mcp transport can dispatch it in-process.
- version.go: ToolSurfaceVersion 0.39 -> 0.41. 0.40 is reserved for
PR #1427 (the 0.32/0.33 precedent — a gap is cheaper than a
collision). 0.41's own entry: pure ADDITIVE bump, same disposition
as 0.13/0.28/0.11 — nothing existing moved.
- instructions.md, README.md, CLAUDE.md: version headings + the
pad_playbook bullet/table-row/changelog-sentence updated per the
drift tests (TestInstructionsMDCoversEveryCatalogAction,
TestReadmeTableCoversEveryCatalogAction, the two version-match
tests). instructions.md's addition is intentionally terse ("`match`
finds a playbook from text.") — it's embedded in the `initialize`
handshake payload, which has its own 24576-byte budget and had only
46 bytes of headroom; a fuller explanation lives in the tools/list
param/action docs instead, which have budget to spare after the trim.
- catalog_readonly_test.go, dispatch_http_parity_test.go: new
`pad_playbook.match` entries in three hand-built fixture maps
(expected cmdPath ×2, a hand-rolled cmdhelp stub, and the shared
parity-drive fixture input) so the existing bijection/parity/dispatch
guards actually exercise the new action instead of skipping it.
Bytes measured (go test ./internal/mcp -run Budget -v):
tools/list: 55290 (baseline) -> 53666 (after the trim commit) ->
54805 (after this commit) — 491 bytes of headroom left
against the 55296 budget.
initialize: 24530 (baseline, unchanged by the trim) -> 24574 (after
this commit) — 2 bytes of headroom against the 24576
budget. Flagged in the implementer report: this is now
as tight as tools/list was before the trim, and nothing
in this unit addresses it.
Claude-Session: https://claude.ai/code/session_01ByMjAhvdWgbAWWeGE9jwWd
* fix(server,decision): don't leak provider errors, classify oversized requests, fix none's wording (TASK-3120)
Lead's shallow-pass findings on cd7a41ed, applied after the MCP commit.
F1 (disclosure): the provider-error path answered with err.Error()
verbatim. typesafe.go's client surfaces the provider's raw 4xx body in
that string (APIError.Error()), which is the INSTANCE's own account
state with typesafe.ai — key validity, quota, billing wording — not
anything about the caller's request. This endpoint requires only
viewer access, so any workspace member could read it. Now logged
server-side in full (slog, workspace + err) and answered with a fixed
message plus a coarse details.reason ("timeout" for a context
deadline, "upstream" otherwise) via a new writeDecisionProviderError
helper. Test constructs a fake error containing a secret-looking
string and asserts it never reaches the response body.
F2 (misclassification): the request budget is 32000 tokens
(~80000 chars). At a couple hundred active playbooks with realistic
summaries, the Choice question's options alone can exceed it before
any state is even added — or leave no room for the input text. Both
were plain fmt.Errorf with no sentinel, so they fell into the generic
"provider failed" 502 path, telling a caller "try again" for a
request that will never fit no matter how many times it's retried.
decision/typesafe.go gains ErrRequestTooLarge (wrapped with %w onto
both pre-send refusals; existing message text unchanged, just
appended-to) alongside the existing ErrMaxTokensExceeded (the
provider's own after-the-fact refusal of a request fitState let
through) — both now map to a distinct 400 match_request_too_large.
New decision-package unit test builds a realistic 254-option Choice
(title + near-cap summary + trigger + slug per option, mirroring what
playbook-match could plausibly send) and confirms fitState refuses it
before the fake HTTP handler is ever called; server-side table test
covers both sentinels through the fake provider.
F3 (nit): the "none" option's description read "...listed above",
but Options is an unordered map — nothing puts them "above" it.
Reworded to "...any of the other options' procedures"; the pinned
exact-prompt test updated to match.
Also (lead's separate MCP-round instruction): decision.go's
maxChoiceOptions is now exported as MaxChoiceOptions, and the
server's over-limit refusal reads that constant instead of a copied
`255` literal, so the two ceilings cannot drift apart.
go test ./internal/decision/... ./internal/mcp/... ./internal/server/...
./cmd/pad/... ./internal/cli/... — all green. golangci-lint run ./...
— 0 issues. Full `go test ./...` run separately; final line in the
report.
Claude-Session: https://claude.ai/code/session_01ByMjAhvdWgbAWWeGE9jwWd
* chore(mcp): cut pad_playbook.match's tools/list text to the approved +292 variant (TASK-3120)
The lead's shape-(a) approval was for the measured "+292 documented"
prototype, not the fuller wording ae38353c actually shipped
(+1139 bytes) — the trim commit exists to pay for the approved
action text, not to fund arbitrary new prose. Cut the `text` param to
one sentence (the 404-fallback advice lives once, in the action
bullet, not repeated in the param) and the `match` action bullet down
to four lines at consistent bullet+2 continuation indentation,
matching the rest of padPlaybookToolDescription's style (the previous
wording had drifted to 20-space continuation on its last two lines).
tools/list: 53666 (post-trim baseline) -> 54023 (+357, close to the
~300 target — wording differs from the original prototype's exact
text, so an exact match isn't expected). initialize: unchanged at
24574 (this file's bullet is tools/list-only; instructions.md is a
separate file, untouched here).
Claude-Session: https://claude.ai/code/session_01ByMjAhvdWgbAWWeGE9jwWd
* fix(server): don't overclaim writeDecisionProviderError's timeout reason (TASK-3120)
The doc comment said "timeout" covered a cancelled caller request too,
but context.Canceled is a distinct error from context.DeadlineExceeded
and errors.Is(err, context.DeadlineExceeded) does not match it — that
case actually falls through to "upstream", contradicting what the
comment claimed. Narrowed the comment to deadline-only (the preferred
fix per review): a cancelled caller's own request also never reads
this response body, so the distinction was moot as well as wrong.
Code was already correct; only the comment overclaimed.
Claude-Session: https://claude.ai/code/session_01ByMjAhvdWgbAWWeGE9jwWd
* fix(cli,skill): pad playbook match needs -- before dash-led text (TASK-3120)
Codex round 2 finding on f31d5d64. Text starting with "-" (a dash-led
sentence, or literal text like "-ship it") is parsed as a flag by
cobra's default scanner, on both the CLI and stdio MCP (which shells
out to this same command). Pre-existing and class-wide — `pad item
search "-x"` fails the same way — so the general stdio fix across
every command is filed as BUG-3142 rather than done here; BuildCLIArgs
is untouched.
This command's own help now shows the workaround: an Example section
with `pad playbook match -- "-ship it"`. skills/pad/SKILL.md's rung-3
routing entry calls it the same way, with a one-line reason, and notes
the MCP form is unaffected (a structured `text` param, not a CLI
token).
New CLI test asserts BOTH directions: `playbook match "-ship it"`
(no --) fails at cobra's flag parser and never reaches the server;
`playbook match -- "-ship it"` succeeds and the server receives the
text with its leading dash intact. Ran both legs — see the test.
Claude-Session: https://claude.ai/code/session_01ByMjAhvdWgbAWWeGE9jwWd
* fix(skill): local stdio MCP is not exempt from the leading-dash flag parse (TASK-3120)
Codex round 4 finding. The rung-3 routing entry claimed the MCP form of
match was "unaffected" by BUG-3142, but that's only true for remote
/mcp, which keeps `text` a structured param. Local stdio MCP shells
out to `pad playbook match`, turning `text` into a CLI positional via
BuildCLIArgs (dispatch.go:435) — the same flag parser, the same bug.
Corrected: remote /mcp is safe as-is; local stdio MCP still hits it and
should prefer the CLI's `--` form when the text starts with `-`.
Claude-Session: https://claude.ai/code/session_01ByMjAhvdWgbAWWeGE9jwWd
* fix(cmdhelp): the example validator's flag scan must stop at "--" (TASK-3120)
CI red on #1429 (
|
||
|
xarmian
|
5fe478b7d0 |
fix: refuse a token-guarded content write over unflushed tab edits (BUG-3133) (#1423)
expected_seq guards the row, and a browser tab's typing is not in the row until the tab flushes. So a content PATCH carrying a current token replaced those edits: through the applier's setContent with a tab open, and with no tab open the direct path's PruneItemOpLogTx deleted them, and they existed nowhere else. A content write that carries expected_seq or expected_updated_at is now refused with 409 content_pending_flush while content-bearing op-log rows sit above the flush watermark. The check is composed into the existing precheck, so the direct, applier and collab-disabled paths all inherit it, and runs in the write's own tx after the token check (a stale token still answers update_conflict). It has its own code because re-reading returns the same row and seq. overwrite_pending_edits lifts it on the API, the CLI (--overwrite-pending-edits; item edit --force sets it on the save) and MCP on both transports, with a shared hint that does not say re-read and retry. A tokenless write is unchanged, except that a direct write now reports warnings.pruned_pending_edits when its prune deleted such rows (the CLI prints it to stderr). MCP ToolSurfaceVersion 0.39: a behavior bump with one additive param. Claude-Session: https://claude.ai/code/session_01X4uYKs5ZPF28eCWfm8VPvE |
||
|
xarmian
|
304673faa1 |
feat(decision): typed-decision storage and async runner (TASK-3117) (#1417)
* feat(store): item_decisions + decision_jobs schema (TASK-3117) Answers keyed on (item_id, question_set, question_key, state_hash) with item_seq kept for the audit; a private per-(item, question_set) job queue with generation-based coalescing and lease reclaim, in place of the public event outbox. NUL census: three columns excluded with reasons, fifteen id/timestamp/enum columns baselined. Claude-Session: https://claude.ai/code/session_01ByMjAhvdWgbAWWeGE9jwWd * feat(decision): job queue store layer + Runner with state-hash idempotency (TASK-3117) Item create, item update and comment create enqueue a coalesced job per applicable question set in their own transaction; nothing waits on the network. The Runner builds one serialised state (title, collection, fields, bounded body, last RecentTrailWindow comments), hashes those exact bytes, and sends the same bytes as json.RawMessage. Claude-Session: https://claude.ai/code/session_01ByMjAhvdWgbAWWeGE9jwWd * test(decision): queue, runner and read pins; move enqueues (TASK-3117) Move is a door (ruling 3, keyed on the target collection's sets). The three system fan-out rewrites are named as non-doors at their sites. Tests pin: each door enqueues and coalesces; nil resolver writes no row; lease reclaim; mid-evaluation re-enqueue survives completion; backoff; latest-per-key read; hashed bytes == wire bytes; a comment re-evaluates at an unchanged seq; reorder / parent link / role assignment move seq but do not re-evaluate; current flips false after a comment. Claude-Session: https://claude.ai/code/session_01ByMjAhvdWgbAWWeGE9jwWd * feat(server): decision tick, read API, decisions on item GET (TASK-3117) Boot builds the provider from one shared resolver (also used by `pad server info`); a nil provider means no resolver, no job rows, no tick. GET /items/{ref}/decisions returns the latest answer per question with current=true only when computed from the item's present state; the single-item GET and the agent view carry `decisions` when non-empty. A shutdown releases in-flight claims without counting a failed attempt. Claude-Session: https://claude.ai/code/session_01ByMjAhvdWgbAWWeGE9jwWd * test(decision): cancelled pass releases; agent view carries decisions (TASK-3117) Claude-Session: https://claude.ai/code/session_01ByMjAhvdWgbAWWeGE9jwWd * feat(store): comment edit/delete enqueue; mid-call cancel test (TASK-3117) Comment edit (when the body changed) and comment delete change the hashed trail, so they are doors under ruling 3's rule. Adds the mid-call shutdown test that mutant M20 showed was missing, and the decisions endpoint to CLAUDE.md. Claude-Session: https://claude.ai/code/session_01ByMjAhvdWgbAWWeGE9jwWd * fix(decision): never send an item in a soft-deleted workspace or collection (TASK-3117) GetItem filters only the item's own soft delete, so a job owed for an item whose workspace or collection was soft-deleted would have sent that content to the third-party provider during the restore window. The runner now checks three-way liveness (the reminder tick's rule) before building state, and drops such a job. Claude-Session: https://claude.ai/code/session_01ByMjAhvdWgbAWWeGE9jwWd * fix(decision): drop is generation-matched; inapplicable sets are not asked (TASK-3117) Codex round 2. P1: DropDecisionJob deleted the row whatever its generation, so a write that landed while a run was hitting its retry limit (or a gone item) was lost; it now deletes only the claimed generation and otherwise releases, as CompleteDecisionJob does. P2: a job owed before a move still named the old collection's scoped set; Evaluate re-checks applicability against the item's current collection and the job is dropped. Claude-Session: https://claude.ai/code/session_01ByMjAhvdWgbAWWeGE9jwWd * fix(store): idempotency counts only the asked question keys (TASK-3117) Codex round 3. The completeness check counted every stored key at the state, so a RENAMED key kept the total equal and the set read complete; the new key was never asked at an unchanged state. The comment claiming over-counting was only possible in the safe direction was wrong and is replaced. Claude-Session: https://claude.ai/code/session_01ByMjAhvdWgbAWWeGE9jwWd * fix(decision): claim one job at a time; restore enqueues; recheck before ask (TASK-3117) Codex round 4. Claiming a whole pass up front let the last job wait out its lease behind earlier provider calls (worst case ~330s per job) and be re-claimed elsewhere; jobs are now claimed one at a time and the lease comment carries the arithmetic. Restore is a door: a job dropped while the item was deleted left it stale with nothing scheduled. Liveness is re-checked as the last statement before the send, with the residual window stated. Collection rename / field migration not enqueuing is the ruled fan-out gap and stays. Claude-Session: https://claude.ai/code/session_01ByMjAhvdWgbAWWeGE9jwWd * fix(decision): hold jobs of a soft-deleted workspace for its restore (TASK-3117) Codex round 5: a job processed while its workspace was soft-deleted was dropped, and a workspace restore (a fan-out, so it enqueues nothing) never brought the work back. Such jobs are now held: the claim scan skips them and they resume on restore; a purge cascades them away. Collections have no restore path and still drop. Liveness is one guard, the last statement before the send; the earlier copy in State was redundant (mutant M26 survived its removal) and is removed. Claude-Session: https://claude.ai/code/session_01ByMjAhvdWgbAWWeGE9jwWd * fix(decision): answers are keyed on question+model; outages never drop work (TASK-3117) Codex round 6. An answer is now current only for the question definition and model that produced it (question_hash, per key): rewording a question or re-pinning the model at an unchanged item state re-asks. Jobs are dropped only after repeated PERMANENT refusals (a 4xx other than 429); transient failures back off to an hourly cap and are never dropped, so a provider outage loses no owed evaluation. Adds the RunOnce-level test for a workspace deleted mid-job (mutant M35 survived without it). The empty production registry is the contract ("ships the rail with an empty registry"), not a defect. Claude-Session: https://claude.ai/code/session_01ByMjAhvdWgbAWWeGE9jwWd * docs(store): name workspace import as a non-door for decisions (TASK-3117) Codex round 7. Import is the one item-creation path that bypasses createItemTxWithID; it stays off the decision queue for the same fan-out reason SPEC-3 keeps it off events/1. Named at the site and in CLAUDE.md, and raised to the lead as an open policy question. Claude-Session: https://claude.ai/code/session_01ByMjAhvdWgbAWWeGE9jwWd |
||
|
David Barkhausen
|
828c9ba2cf |
feat(items): execution lease — atomic claim/checkout with expiry (#1238)
Implements #1221 (IDEA-2815). Contributor: b4rk13. Closes #1221. |
||
|
Matt Faltyn
|
dba74e8bc0 |
Fix macOS session test portability
Signed-off-by: Matt Faltyn <faltyn.matthew@gmail.com> |
||
|
Matt Faltyn
|
07c00e008e | feat(agent): add compact item view | ||
|
Matt Faltyn
|
d6e4053aca | perf(agent): reuse session context | ||
|
Matt Faltyn
|
8e5b43ded5 | fix(agent): align compact dispatcher guidance | ||
|
Matt Faltyn
|
cacf492d95 | feat(agent): install compact shared skill | ||
|
Matt Faltyn
|
d89587b4a8 | test(agent): add prompt payload budgets | ||
|
xarmian
|
5e869b285e |
fix(export): mark bundle bodies that were stale when written, and refuse the migration that would abandon them (BUG-3032) (#1356)
* wip(items): mark bundle items whose body is behind the live doc (BUG-3032 doors 1-2) * wip: doors 4 and 6 (BUG-3032) * wip: store test for the gate predicate (BUG-3032) * wip: door 3 import stale-body count + tests (BUG-3032) * docs(backup): the migration refuses on unflushed collab edits; bundles mark stale bodies (BUG-3032) * docs: discharge BUG-3032's open-decision conditional in the artifact door and the population guard * fix(export): the migration gate's population cannot diverge from the bundle's (BUG-3032) * docs(claude): Go build caches never go under /tmp — it is a tmpfs, so it is RAM * wip: codex round 1 P1 + P2 fixes (BUG-3032) * fix(db): refuse a migration whose bundle went stale after the pre-flight check (BUG-3032, codex r1 P1); print the import's stale-body count (P2) * fix(db): refuse to migrate while a server can append, and parse the stale-body header (BUG-3032, codex r2 P1+P2) * docs: narrow the running-server claim to what the probe measures (BUG-3032) * wip: codex r3 fixes + web parity test (BUG-3032) * fix(db): fail closed when the running-server check cannot run; match Go's header parsing on the web (BUG-3032, codex r3) * fix(cli): the Go and web readers of the stale-body header accept exactly the same values (BUG-3032, codex r4) * docs: correct two comment claims about integer representability and Atoi's leniency (BUG-3032, codex r5 nits) |
||
|
xarmian
|
d2973b24d7 |
feat(items): expected_seq — an OCC token that can tell two same-second writes apart (BUG-3037) (#1351)
* feat(items): expected_seq — the strong OCC token (BUG-3037) * test(items): the same-second race, with each token (BUG-3037) * test(cli): the summary shape carries the seq token (BUG-3037, matrix hole) * test(web): the pane sends the seq token, with the cached-row arm (BUG-3037) * test(web): the OCC retry re-reads the token through tokenOf (BUG-3037, matrix hole) * docs: CLAUDE.md carries the v0.34 tool-surface entry (BUG-3037) * test(e2e): the pane's field write is guarded by seq (BUG-3037) * fix(mcp,webmcp): forward the OCC token and refuse a fractional seq (codex round 1) * test(items): the reproduction no longer races the wall clock (BUG-3037) * fix(items): seq decides when both OCC tokens are sent (codex round 2) * fix(cli): --expected-seq is sent by presence, so an explicit 0 is not dropped (codex round 3) * fix(cli): render a seq conflict with the token the caller sent (codex round 4) * docs(mcp): agent instructions recommend expected_seq (codex round 5) |
||
|
xarmian
|
d5085bba00 |
feat(items): multi_relation — an ordered list of references (TASK-2999 / PLAN-2857 U4) (#1342)
* feat(items): multi_relation shape validation, DSL target, and rename propagation (TASK-2999 / PLAN-2857 U4, rows 1/17/18)
The first three rows of U4's population table, chosen because two of them are
the rows the task's own 5-row scope table did not contain and both fail
SILENTLY.
The table came first (posted on TASK-2999). A mechanical enumeration of every
site keyed on the literal "relation" finds 37 decision points in 21 files — 21
Go sites in 9 files, 16 web sites in 12 — against a scope table naming 5. By
CONVE-35 that is a measurement of the enumeration, not a longer to-do list, so
the deliverable under review is the table.
ROW 1 — internal/items/validate.go, `case "multi_relation"`.
Shape only; this package is DB-free, so whether an element names a live item
stays the store's question, exactly as for a scalar relation. Two rules
DIVERGE from the scalar case, ruled day 64 as NEW rules rather than inherited
ones: an empty or whitespace-only element is REFUSED (the scalar resolver skips
such a value), and `[]` is accepted as a shape while `required` is enforced
against RESOLVED elements elsewhere. Skipping is how scalar relation ended up
with three stored spellings for "no target"; inside an ORDERED array it would
also silently change the element count.
There was nothing coherent to inherit, which is BUG-3028, filed with line
references: a required scalar relation is satisfied by "", because
validateFieldType only asserts val.(string) and the resolver then skips the
empty value at relation_referents.go:232-235 and 1504-1507. Its fix is
convergence on U4's rule, deliberately ordered AFTER U4.
ROW 18 — internal/collections/dsl.go.
`multi_relation` joins the relation branch rather than falling to the Options
arm. Cardinality changes how many targets a VALUE names, never how many a FIELD
declares. Left out, `tags:multi_relation` parses happily into Options=["people"]
with no target — the exact unwritable field the v0.31 bump existed to stop the
DSL minting, one type later. Error messages now name the type the caller typed.
ROW 17 — internal/store/collections.go, retargetRelationsInSchemaJSON.
BUG-2873 rename propagation. A multi_relation declares its target the same way,
so a rename that skipped it left the field pointing at a slug that names
nothing: no error, the schema still parses, and every later write refused with
`target_missing` for a rename nobody would connect to it. This function had NO
test coverage before now (verified by grep across internal/**/*_test.go), and
it is the highest-consequence row in the table.
EVERY new test was run against the unfixed code and fails there:
* dsl.go reverted to `relation` only -> 3 of 4 new DSL tests FAIL
* the `case "multi_relation"` block deleted -> all 7 refusal legs FAIL
* the retarget condition reverted -> MovesBothRelationTypes FAILS,
naming the orphaned field
The accept-legs deliberately carry no weight on their own: an unknown field
type falls through validateFieldType with no check, so "accepts an array" stays
green against a tree that has never heard of multi_relation. The refusal legs
are the instrument, and the counterfactual above is what says so.
Three counterfactual legs guard the other direction, where a fix goes too far:
multi_select still takes its options list and gets no Collection; scalar
relation still accepts "" (BUG-3028's territory, not U4's to change here) and
still refuses an array; and a non-relation field carrying a `collection` key is
NOT retargeted, which would otherwise pass every assertion in that test.
Verified by reading rather than assumed: U5 already did U4's index half.
relation_links.go handles `case []any:` with ordinals and its schema matcher
already names multi_relation; migration 065/088 carries `ordinal` explicitly so
U4 needs no ALTER. Two rows there stay OPEN in the table with that stated — the
SQL backfill's array handling, and what a scalar->array retype reindexes.
Gates on this tip: make test 0 FAIL / 30 ok packages, make lint 0 issues,
gofmt clean. web/build generated in this worktree by `npx vite build` so the
Go gates embed this tree (CONVE-34 corollary); no web/src change in this
commit.
Claude-Session: https://claude.ai/code/session_019EmDd3Wy8b3cPG1TZBSzBu
* feat(store): multi_relation resolution by extracting the one relation ladder (TASK-2999 / PLAN-2857 U4, rows 2/14/15)
Rows 2, 14 and 15 of U4's population table. The shape of the change is an
EXTRACTION, not a second implementation: `ResolveRelationReferentsQ`'s
per-value UUID→ref→title ladder becomes `resolveRelationValueQ`, and both the
scalar path and the new `resolveMultiRelationField` call it.
One place knows the ladder, one place decides `VisibilityChecked`, one place
tells `ambiguous` from `wrong_collection` from `not_found`. That function holds
two separate rulings on `VisibilityChecked` — one of them a window deliberately
left to BUG-3012 — and a second copy of it would look correct while diverging
the moment either gained a case.
Rows 14 and 15, the two server doors, need no change of their own: they call
the store function and inherit the array handling. Stated rather than left
implicit, because "no diff" and "not covered" look identical in a table.
THREE DECISIONS, none of them visible from the code alone:
* ORDER AND LENGTH ARE PRESERVED. An unresolvable element stays VERBATIM in
the output; only resolved elements are rewritten to their canonical id.
Dropping one would make the stored length depend on which elements were
malformed, and the ratified model is an ordered list whose identity IS the
array — the same thing U5's `ordinal` column already commits to.
* NOTHING IS WRITTEN BACK UNLESS EVERY ELEMENT RESOLVED. The doors turn any
issue into a refusal, so a partially canonicalised blob is a write the
caller never asked for on a request that is about to fail.
* A FIELD WITH NO TARGET COLLECTION REPORTS ONCE FOR THE FIELD, not once per
element. Every element fails for the identical reason, and N copies bury
the one thing the caller can fix.
DUPLICATES get a new reason, `duplicate_referent`, with its `Message()` case
and its `RelationIssueReasons()` entry. Detected AFTER resolution, which is why
it cannot be a shape check: the duplicated elements are usually DIFFERENT
STRINGS naming one item — a UUID and a ref, a ref and an exact title — so
nothing that cannot resolve them can see the duplication. The issue names the
SECOND occurrence, the one the caller can remove. Refused rather than silently
de-duplicated: a caller who sent a target twice either erred or thinks
multiplicity means something, and storing one copy answers neither honestly.
THE PARITY GATE DID ITS JOB. `TestCopyPreflightDropReasonsAreRenderedByTheDialog`
went red on `duplicate_referent` the moment the constant existed, and stayed red
until the reason was in the TS type union, in `COPY_DROP_REASONS`, and in the
`MESSAGES` map. That is BUG-2674's failure — a reason shipping and rendering to
a user as a raw enum — prevented instead of repeated.
`copyDropReasons.ts`'s header says "The test cannot catch a reason added to Go
and not added here". True of THAT module's vitest test, and it reads as though
the direction is uncovered: I believed it and was part-way into building a
second instrument before finding the Go gate. A paragraph now names the gate.
MUTANTS — four, each killed by exactly the test that should kill it:
* multi_relation routing removed -> all four resolver tests FAIL
* duplicate check removed -> DuplicateAfterResolution only
* write-back made unconditional -> OneBadElementRefusesAndWritesNothing only
* output order reversed -> CanonicalisesEveryElementAndKeepsOrder only
`EmptyArrayIsNotAnIssue` survives all four. Recorded rather than counted: it is
a CONTROL against a resolver that invents an issue for `[]`, not a detector of
any of these.
The risk this commit carries is the extraction moving SCALAR behaviour, and the
U1/U6 suites are the instrument for it: full `internal/store` 0 failures, full
`internal/server` ok in 436s, 0 failures.
Gates: make test 0 FAIL / 30 ok packages · make lint 0 issues · gofmt clean ·
svelte-check 0 errors · copyDropReasons vitest 3 passed.
Claude-Session: https://claude.ai/code/session_019EmDd3Wy8b3cPG1TZBSzBu
* refactor(models): IsRelation / IsMultiRelation predicates, and the field-level sites that join them (TASK-2999 / PLAN-2857 U4, rows 8/12/13 + 16's predicate)
Rows 8, 12 and 13 of U4's population table, plus the instrument that makes the
table maintainable.
WHY A PREDICATE AT ALL. The contract for this unit found 37 decision points
keyed on the literal "relation" with NO constant behind any of them, so "which
code decides relation behaviour?" was a question only a string search could
answer — and the string-search answer is wrong the moment a second relation
type exists, which is what this unit adds. `FieldDef.IsRelation()` and
`FieldDef.IsMultiRelation()` make it a symbol search, and `RelationFieldTypes()`
makes the vocabulary enumerable rather than greppable.
The two predicates are deliberately different questions. FIELD-level code asks
IsRelation: does this field have a target collection, should a rename touch it,
is it worth hydrating. VALUE-walking code has to ask IsMultiRelation too,
because the shapes differ — and a single predicate answering both the same way
would make that distinction inexpressible.
WIDENED (field-level, body already shape-agnostic):
* relationTargetsUnavailable — availability is a property of the FIELD
* relationTargetSlug — returns the declared target; cardinality is
not part of that question
* relationSchemasForWorkspace — which schemas are worth hydrating at all
* RelationKeysPresent — presence only; it never reads the value
NOT WIDENED, and this is the more interesting half. `HydrateRelationTargetsQ`
keeps `def.Type != "relation"` with the reason written at the site. Its body
assumes ONE value per field — `perItem[item][key]` holds a single
`relationWant`, and `raw.(string)` skips an array silently — so widening the
predicate there would read as "multi handled" while hydrating nothing. I made
that widening, read the body, and reverted it.
Hydrating a `multi_relation` needs a WIRE-SHAPE decision first: whether
`relation_targets[key]` becomes a list or gains a parallel key. That is a v0.31
contract change, not a predicate edit, so it stays an OPEN row on the table
rather than being smuggled in as a one-liner.
The same trap is why `models.RelationFieldTypes`'s doc comment says explicitly
that the migration and default-resolution sites name their types by hand on
purpose: widening them with IsRelation alone would treat an array as a scalar.
TESTS. The predicates are pinned against every other field type this repo
ships, listed explicitly rather than derived so a new type forces a decision
instead of inheriting one, plus a leg asserting RelationFieldTypes agrees with
IsRelation — two expressions of one fact drift, and nothing else can notice.
Gates: make test 0 FAIL / 30 ok packages · make lint 0 issues · gofmt clean ·
full internal/store ok (157s), internal/items, internal/collections,
internal/models all ok.
Claude-Session: https://claude.ai/code/session_019EmDd3Wy8b3cPG1TZBSzBu
* feat(mcp): multi_relation hydration, migrate/default sites, and the v0.32 surface bump (TASK-2999 / PLAN-2857 U4, rows 4-7/9-11)
Closes the rest of U4's Go half. Rows 4-7 (MigrateRelationReferentsQ), 9-10 (the
two default-resolution passes) and 11 (read-side hydration), plus the surface
bump the read shape owes.
ROWS 4-7 AND 9-10 — the four migrate sites and the default passes asked
`_, isStr := raw.(string)`, which is the right question for exactly one of the
two types. Asked of a `multi_relation` it calls every legitimate array
malformed — and since those sites DELETE what they judge malformed, the answer
decides whether a valid value survives. Three small helpers now carry the
type-aware questions:
* relationValueShapeIsValid — string for `relation`, array of strings for
`multi_relation`
* relationValueIsCleared — the "no referent" form, which for an array is
`[]` or an array of blanks. The origin split skips a cleared value so it
lands in no bucket; without this an array the caller deliberately emptied
would be bucketed as a referent, resolve to nothing, and be reported as a
drop nobody could act on
* relationIssueValue — which value a FIELD-level issue names
`relationIssueValue` returns a SCALAR value UNTRIMMED, deliberately. The call
site it replaced read `raw.(string)` and reported it verbatim, and BUG-2870
settled that a caller's value bytes are carried as sent — so trimming would
quietly change what an existing scalar drop reports, a behaviour change
belonging to no row of this table. The multi case trims per element because it
is CHOOSING which element to name, not reporting one it was handed.
The cross-workspace carry now drops a `multi_relation` once for the whole field,
never per element, because the ruling is that the value drops WHOLE — so an
element count is not something the destination learns and an import cannot
change it.
ROW 11 — hydration, on the lead's ruling: `relation_targets[key]` carries EITHER
the scalar object v0.31 emitted OR a JSON ARRAY of those objects in stored order.
ONE map, not a parallel key: a consumer rendering "what does this field point
at?" should not have to ask two questions and merge them, and the field's type
already says which shape to expect.
`models.RelationTargetSet` holds either shape with a custom marshaller. THE
COMPATIBILITY CLAIM IS THAT A SCALAR ENTRY IS BYTE-IDENTICAL, and the test pins
it against a LITERAL rather than against another call into this code — comparing
it to itself would pass against any self-consistent change, including one that
renamed every key. It unmarshals both shapes too, because the CLI decodes item
JSON the server produced, and a marshaller without a matching unmarshaller is how
a shape change reaches a consumer as a decode error instead of as data.
Position is preserved: every stored element gets an entry, including one that
names nothing, which hydrates ID-only — the same honest shape a dangling scalar
gets. Dropping it would silently shorten the list for exactly the legacy rows
that need explaining, and a consumer could no longer line the list up against
`fields`. Visibility redaction is element-wise: a list with one visible and one
invisible target redacts only the second, since redacting the whole list
withholds what the caller may see and redacting none discloses what they may not.
An EMPTY list marshals as `[]`, distinct from the key being absent: absent means
"nothing hydrated this field", `[]` means "this field holds no references".
v0.32, AND A VERSION COLLISION STATED RATHER THAN RESOLVED SILENTLY. PR #1337
also claims 0.32 and was opened first; `gh pr view 1337` says OPEN and main is
still 0.31, and the standing instruction is to take 0.33 only once it HAS merged.
So this takes 0.32 and flags the contest: whichever merges SECOND owes a
renumber, in the constant and in the entry. 0.33 is free if the reviewer would
rather not have the contest at all.
The bump is owed by the read shape, not by the type. Every value rule here is
reachable only through a field whose schema says `multi_relation`, which no
existing schema can contain — but a strictly-typed client parsing
`relation_targets` will fail to decode an array where it expects an object, the
moment its workspace acquires such a field.
DOCS SWEPT, all three consumers: `internal/mcp/instructions.md` (the agent-facing
contract, with the four rules that are NOT guessable from scalar behaviour),
`README.md` (heading, `tool_surface_version`, changelog sentence), and CLAUDE.md —
which the 0.27 unit missed while sweeping the other two, and which says so in its
own text.
Test adaptations: `scalarHydrated` / `scalarTarget` helpers in the store and
server suites unwrap a scalar entry and FAIL on a list rather than returning a
zero value, because a scalar field hydrating as a list is precisely the
regression this type makes possible and a quiet zero would surface it as a
confusing assertion two lines later.
Gates: make test 0 FAIL / 30 ok packages · make lint 0 issues (it caught an
unformatted file in internal/cli that would otherwise have been pushed) ·
internal/mcp drift gates green on all three documents.
Claude-Session: https://claude.ai/code/session_019EmDd3Wy8b3cPG1TZBSzBu
* test(store): pin the reverse index's array handling, and renumber to v0.33 (TASK-2999 / PLAN-2857 U4, rows 20-22)
Closes the last three rows of U4's Go half — all three as QUESTIONS rather than
work, and the answer in each case is that U5 already built it right. That is not
a reason to leave them unpinned: reading a `case []any:` arm is not the same as
knowing the index holds the rows, and the whole point of the population table is
that a row's state is recorded rather than assumed.
ROW 20 — a `multi_relation` write indexes one row per element with the array
position in `ordinal`. The control is the part worth having: reversing the stored
array must reverse the ordinals, so an extractor emitting a constant 0 cannot
pass by the fixture happening to be sorted its way.
ROW 21 — the BACKFILL covers arrays, not just the write hook. Worth separating
because the two paths reach the index differently (per-write vs every live item
from a clean table) and because migration 065's own comment says a derived query
"on a multi_relation array silently matches nothing", which is easy to misread as
being about this path. It is not — the backfill reaches the same Go extractor via
`relationKeysFromSchemaJSON`, which has matched `multi_relation` since U5.
The test clears the rows by hand AND clears the once-only
`platform_settings` marker first. Both are preconditions without which it proves
nothing: the write hook has already populated those rows, so a backfill that did
nothing at all would pass, and a backfill that reported `Skipped` would too. It
asserts `!res.Skipped` for the same reason.
ROW 22 — a scalar->array RETYPE reindexes against what is STORED, not against
what the new schema says the shape should be. The ruling called a garbage retype
a U4 defect; it is not one, and this records WHY rather than leaving the row open.
The extractor is VALUE-shaped — it switches on the JSON type it finds — so after
a retype the still-scalar blob indexes as one row at ordinal 0, which honestly
describes what the item holds. Indexing by the DECLARED cardinality instead would
invent rows the blob does not contain.
MUTANT: disabling the extractor's `case []any:` arm kills rows 20 and 21. The
retype test SURVIVES it, correctly — its blob is a scalar, so that arm is not on
its path. Stated rather than counted, because a surviving mutant is worth
explaining either way.
VERSION RENUMBERED TO 0.33 on a lead ruling, in all four documents (version.go,
instructions.md, README.md, CLAUDE.md). 0.32 is now RESERVED for PR #1337, which
claims it and was opened first. The reasoning is in the changelog entry: version
numbers are not scarce, and two branches claiming one means whichever merges
second renumbers under merge pressure — which is exactly when a version bump is
least likely to be checked. A gap in the sequence is cheaper than that, so 0.32
may simply be skipped if #1337 is ever abandoned.
Gates: make test 0 FAIL / 30 ok packages · make lint 0 issues · gofmt clean ·
internal/mcp drift gates green on all three versioned documents.
Claude-Session: https://claude.ai/code/session_019EmDd3Wy8b3cPG1TZBSzBu
* feat(web): the type union, the schema editor, and the five field-level web sites (TASK-2999 / PLAN-2857 U4, web rows 1-5 + 11-15)
First batch of U4's web half: the sites where the right answer is "this field
points at other items, cardinality is irrelevant". Value-shaped surfaces — the
item-pane picker, the filter bar, board/list grouping — are NOT in this commit,
because each needs a decision about what a multi-valued answer means and a
predicate widening would paper over that.
`$lib/items/relationFieldTypes.ts` mirrors the Go predicates with the same split:
`isRelationType` for field-level questions, `isMultiRelationType` for value-level
ones, `isResolvableRelation` for the type+target pairing that appears at five
sites. Placed in `lib/items/` rather than under `components/` because
non-component code (`shareView`, `copyNeedsValue`) asks the same questions, and a
predicate living under `components/` invites a second copy for everything that
is not one.
SITES, and what each would have done left alone:
* `types/index.ts` — the FieldDef union gains `multi_relation`. Without it
every other change here is a type error.
* `field-editor-types.ts` FIELD_TYPES — the schema editor can now DECLARE one.
A type the API accepts and the editor cannot offer is a half-shipped
surface: the only route in would be the CLI's DSL or a hand-written schema,
which is a gap with a workaround, not a feature.
* Create/EditCollectionModal (3 sites) — these preserve `def.collection` only
when the type matched. Left alone, a `multi_relation` saved with NO target,
and every later write to it was refused with `target_missing` — silently
from the user's side, since the editor had shown them the target they
picked.
* `shareView.isPublicGroupable` — a public share refuses to group by a
relation because the payload carries values with no index behind them. A
`multi_relation` is strictly worse, each value being an ARRAY of ids, so a
shared view grouped by one would render a group per stringified array.
* `copyNeedsValue` (2 sites) — the copy dialog's collectable/uncollectable
split, including IDEA-2899's distinction between naming NO target and naming
an unavailable one, which had to survive the widening.
* `[collection]/+page.svelte` (2 sites) — the relation-keys set, and the
DESTRUCTIVE lane-order guard that would otherwise write item ids into the
schema as `options`.
TESTS are at two levels on purpose. The predicate is pinned against every other
field type the app ships, listed explicitly so a new type forces a decision. The
SITES are then asserted through their own functions, because a test on the
predicate proves nothing about whether a given site calls it (CONVE-19) — each
with the scalar control that would catch a widening gone too far: `multi_select`
still groupable on a share, a non-relation copy row unaffected, FIELD_TYPES
still carrying what it had.
Counterfactual, run: reverting all three widenings turns 4 of the 8 wiring legs
red. The other 4 are controls and correctly survive.
ONE TEST REPOINTED, and it is worth reading. TASK-2998's source guard on
`handleGroupReorder` read for the literal `type === 'relation'`. Replacing that
with the predicate made the guard STRONGER — it now covers both types — and the
test went red for a spelling change. It now reads for the predicate CALL, plus a
leg requiring the guard to test the REORDERED field (`s.fields[idx].type`) rather
than merely calling the predicate about something. A source guard reading for a
spelling fails whenever the code improves; that is the cost of the technique, and
the comment now says so at the site.
Gates: web 2546 passed / 180 files (2545 before, +1 new file) · svelte-check 0
errors · vite build clean.
Claude-Session: https://claude.ai/code/session_019EmDd3Wy8b3cPG1TZBSzBu
* feat(web): filtering a multi_relation is MEMBERSHIP (TASK-2999 / PLAN-2857 U4, web rows 8 + 12)
U7 deferred this to U4 in as many words, in both the function's doc comment and a
test, rather than guess: a `multi_relation` stores an ARRAY and no amount of
trimming makes `===` match one, so equality would have returned EMPTY for every
filter, forever, and silently.
Defined now as MEMBERSHIP: an item matches when its list CONTAINS the target. It
is the only reading of "filter by this target" that is true of a list.
Membership, not POSITION. An item whose second reference is Red matches a filter
for Red. Order is part of the stored value — it round-trips, and the reverse index
keeps it in `ordinal` — but it is not part of this question.
Elements are trimmed individually, on the scalar case's own reasoning: the write
path has refused padded values since TASK-2878, so there is nothing to migrate,
only old rows to read correctly. A non-string element is ignored rather than
thrown on; a corrupt blob must not take out the whole view, and an element that
is not a reference cannot match anyway.
`FilterBar` now offers BOTH relation types. Its picker needs no change — it still
returns ONE target id; what differs is the question asked of the stored value.
A U7 TEST WAS REWRITTEN, not deleted. `refuses a non-string, including the
multi_relation ARRAY shape` asserted that an array matched nothing, and explained
that the deferral was deliberate. That assertion now states the OPPOSITE of the
shipped behaviour. It keeps its non-string legs under an honest name, and the
comment records that U4 closed the deferral — the deferral was real and the record
of it belongs somewhere, which is the U4 block the array legs moved into. This is
the shape where a prior unit's prose asserts the absence of the thing you just
built; the prose has to move when the absence does.
Counterfactual, run: removing the membership arm turns 3 of the 6 new legs red.
The other three are the scalar control and the negative cases, which correctly
survive — a change that only ever looked at arrays would pass everything else and
break every existing relation filter, which is what that control is for.
NOT in this commit, and it needs a ruling rather than a guess: GROUPING by a
`multi_relation`. The same "membership" answer would put one item in N lanes,
which breaks the invariant U7 worked to preserve — `bucketByColumn` puts every
item in EXACTLY ONE lane, and U7's proving row asserts it by placing six items and
counting. So grouping is either a different rule (an item appears once per
target, which is a rendering change, not a predicate widening) or a refusal like
the one public shares already give. Raised on the trail; BoardView and ListView
are untouched meanwhile.
Gates: web 2551 passed / 180 files · svelte-check 0 errors · vite build clean.
Claude-Session: https://claude.ai/code/session_019EmDd3Wy8b3cPG1TZBSzBu
* feat(web): grouping by a multi_relation is REFUSED, with the reason said out loud (TASK-2999 / PLAN-2857 U4, web rows 6 + 9 + 10)
Lead ruling, option (2) of the three I put up: refuse, same disposition a public
share already gives relation grouping, and never fall back silently.
WHY REFUSE rather than render. `bucketByColumn` puts every item in EXACTLY ONE
lane, and U7's proving row asserts that by placing six items and counting. An
item with three references belongs to three. The alternatives were rendering the
item once per target — which leaves a drag between lanes meaning something
undefined (add? move? replace?) — or grouping by the FIRST element, which
silently answers a different question than the user asked. The multi-lane
rendering is IDEA-3034 if it is ever wanted.
`relationGroupingRefusal` returns two distinct reasons, and the distinction is
the point: `no_target` is fixable by editing the schema, `multi_valued` is not.
One message for both would make a distinction the type draws invisible to the
person reading it, so a test asserts the two sentences DIFFER.
THE "NEVER SILENTLY" HALF IS MOST OF THE WORK. Before this, a `multi_relation`
group field fell through both views' scalar predicate, was treated as an ordinary
field, and — having no `options` — put every item in UNCATEGORIZED. An
Uncategorized lane with no explanation reads as "none of these items has a
value", which is false and unactionable.
* ListView already had the mechanism: `relationWithoutTarget` drops it to
ungrouped. That flag now derives from the refusal, so it covers both reasons
instead of one, and a notice renders above the list.
* BoardView cannot be "ungrouped" — a board IS lanes — so its fallback stays
the single lane it already produced. What changes is that it says why.
The notice is plain and inline rather than a dismissible toast, deliberately: it
describes a standing property of the view's CONFIGURATION, not an event, so it
has to still be there after a reload.
ROW 6 also lands here: the schema editor's per-field "Relates to" picker now
shows for both relation types. FIELD_TYPES already offered `multi_relation` in
the type dropdown, so without this the editor let a user pick the type and then
gave them no way to say what it points at — every field declared through the UI
would have been refused on its first write with `target_missing`.
ONE THING WORTH RECORDING that needed no change: the board/list GROUP-BY PICKER
(`DisplaySettingsEditor`, fed by `selectFieldKeys`) only ever offered `select` and
`multi_select` fields. It never offered relation fields at all, so "the picker
does not offer a multi_relation" was already true by construction, and U7's
relation grouping is reachable only through a saved view or a hand-set
`board_group_by`. Verified rather than assumed — I went looking for the picker to
change it and found it already correct, which is the more useful half of the
ruling to have checked.
Counterfactual, run: removing the `multi_valued` arm turns exactly the leg that
should go red, red. Two control legs — a relation WITH a target still groups,
every ordinary type still groups — guard the opposite failure, which would
silently un-group every relation board U7 shipped.
Gates: web 2556 passed / 180 files · svelte-check 0 errors · vite build clean.
Claude-Session: https://claude.ai/code/session_019EmDd3Wy8b3cPG1TZBSzBu
* feat(web): the item-pane picker edits a multi_relation (TASK-2999 / PLAN-2857 U4, web row W7)
W7 is the last of the 29 population rows and the one the rest of U4's web half
exists to make usable: a type the schema editor can declare but no item pane can
EDIT is still half-shipped.
A PARAMETERISATION, not a widening. The four resolution steps — resolve the row,
name its state, format its ref, build its href — were $derived values closing
over the single scalar `value`. They are now functions of ONE raw reference, and
the scalar path calls them with its single element, so there is one
implementation rather than a scalar one and a multi copy that drift. The chip
snippet takes the reference the same way. `relationValues` is the list both types
render from: one element for a `relation`, N for a `multi_relation`, none when
empty.
The 40 existing scalar tests are the instrument that this moved nothing; they
were not touched. Negative control run before trusting them: forcing
`relationHrefFor` to null turns one red, so they do read the parameterised path.
What the list behaviour is, all of it ruled in U4 and none re-litigated here:
* ORDERED. Chips render in stored order, a choice APPENDS to the end, and
Remove drops by POSITION — the user pointed at a row, not at an id.
* `[]` is the one spelling of none. The last Remove writes the empty array, not
the scalar's `''` (which this type's validator refuses); normalising it to an
absent key is the write door's job and stays there.
* DUPLICATES are prevented by not offering them: the picker is handed the
field's own elements to hide. The exclusion set is the RAW stored elements
because the write door canonicalises every value to its target's UUID, so a
stored element IS the id — which also covers the element `localIndex` cannot
resolve, the one whose target the cold picker can still hand back from the
server.
A write-side duplicate guard was written and removed. It compared against the
same set the picker is given, so nothing the picker could offer could reach it
and no mutant could kill it — an unfalsifiable branch reading as rigour. The
server refuses a duplicate outright (`RelationTargetDuplicate`) rather than
de-duplicating, so the backstop behind the exclusion is a refusal, not a silent
one-copy write. A resolved-id half of the exclusion set went the same way for the
same reason: under canonicalisation it is the same set, and the mutant that
narrowed it away survived.
Affordances are per-element (Remove) plus one for the list (+ Add). The scalar's
Change/Clear pair does not translate — "Change" would have to mean "replace
everything". The picker's open/closed rule IS the scalar's, unchanged: open when
the field is empty or when asked for, closed once a choice lands.
Instrument: 22 new tests in `FieldEditor.multiRelation.svelte.test.ts`. Run
against the PARENT COMMIT's FieldEditor: 21 fail, and the one that passes is the
scalar CONTROL, which is what a control is for. That run is also why one leg
grew an assertion — "an empty field opens the picker with no Add button" passed
unfixed, because an empty value used to fall into the SCALAR branch, which also
opens a picker and also has no Add button. It now asserts the SHAPE the picker
writes, which is what tells the two branches apart.
Mutation matrix, each mutant compiled and each killed by the test that should
kill it:
| mutant | died in |
|---|---|
| append -> replace | APPENDS + inline-create-appends |
| last Remove writes `''` | the `[]` leg only |
| `excludeIds` dropped | all three duplicate legs |
| element order sorted | the order leg + 5 others |
| blank/non-string elements kept | the dropped-elements leg only |
| chip handler ignores its own row | the pane-open leg only |
| non-array value read as one reference | that leg only |
| empty list no longer auto-opens the picker | both picker-visibility legs |
| multi falls through to the scalar branch | 10 legs |
Two mutants SURVIVED a first pass and are the reason two things above are
smaller than they were: the blanks filter (the titles assertion could not see
extra EMPTY elements — the test now counts rendered units) and the write-side
duplicate guard (unreachable, removed). Both are recorded rather than patched
around.
TASK-2999 / PLAN-2857 U4
Claude-Session: https://claude.ai/code/session_019EmDd3Wy8b3cPG1TZBSzBu
* test(server): the Q2 round-trip proving test for multi_relation (TASK-2999 / PLAN-2857 U4)
The design row asks for one property end to end: write a `multi_relation` value
as a list of REFS through the write door, read it back hydrated, export the
workspace and re-import it, and assert the stored array is identical and every
element still resolves. Negative leg: one unresolvable element refuses the WHOLE
write, naming that element, never a partial store.
AT THE DOOR, not in internal/store, and that is the point of the row. Every
component underneath already has direct tests — the resolver's canonicalisation
and ordering, the hydrator's list shape, the export remap. Those vouch for the
components and say nothing about whether the doors are BOUND to them: a create
handler that never reaches the resolver, or an import that never rewrites an id
inside an array, passes every one of them. A round trip is a claim about the
seams, which is exactly what no component test can make.
The export/import half is the VERIFICATION the design asked for rather than a
rebuild: `remapFieldIDs` substitutes quoted JSON tokens, so it reaches ids inside
arrays without knowing the schema. Nothing was added there. This is the test that
would go red if someone replaced that substitution with a scalar-shaped walk.
Three targets, not two, and supplied in an order that is neither sorted nor
reverse-sorted by id, ref or title — with two elements a reversed array and a
sorted one are the same observation, and "order preserved" would be satisfied by
a resolver that sorted.
Mutants, each compiled, each killed by the test that should kill it:
| mutant | died in |
|---|---|
| export remap skips values containing an array | the round-trip test |
| resolved array reversed before write-back | the round-trip test |
| a multi_relation hydrates as a SCALAR (first element) | the round-trip test |
| an unresolvable element reported as no issue | the refusal test |
Stated rather than counted, because the difference matters: the "no item was
created" assertion in the refusal leg is a REGRESSION GUARD, not evidence. The
refusal happens before any insert today, so no mutant short of reordering the
handler turns it red and I did not build one. It guards a future
create-then-validate ordering, which the 400 alone would not distinguish. The
comment at the site says so.
The third test is a CONTROL: a door that refused EVERY array satisfies both
assertions of the refusal leg, and this file would then read as proof of a
working feature.
Gates: `make lint` 0 issues · gofmt clean · `internal/server` ok (364s) ·
`internal/store` ok (165s) · `internal/items` ok. All on SQLite — `testServer`
calls `storetest.NewSQLite` unconditionally, so nothing built on it is
Postgres-backed regardless of which make target runs it.
TASK-2999 / PLAN-2857 U4
Claude-Session: https://claude.ai/code/session_019EmDd3Wy8b3cPG1TZBSzBu
* fix(items): every site that dispatches on field type now knows multi_relation (TASK-2999 / PLAN-2857 U4)
Codex round 1 on the U4 tip returned five findings. All five were real, and
enumerating the class they belong to found NINE. They are one defect wearing
nine costumes: a site that dispatches on FIELD TYPE and was never extended for
the new one. I built the WEB population table for exactly this reason and never
built the GO one.
The instrument is two greps, and the second is load-bearing:
`grep -rn '"relation"' --include=*.go internal/ cmd/` finds 13 sites, but two of
the gaps live in switches that enumerate field TYPES and never mention
`relation` at all, so a relation-literal sweep cannot see them. The class is
"dispatches on field type", not "mentions relation".
## The nine
**Visibility (the P1).** A `multi_relation` element was never visibility-checked.
The store resolver takes `canSee`, but only its TITLE rung consulted it; the
UUID and REF rungs were judged by the SERVER's post-resolution pass, which is
gated on the field type. So an editor with no access to the target collection
could put a hidden item's ref inside an array and receive 201 with its canonical
UUID — the existence oracle U1 exists to close, reopened by the array door.
Per the lead's ruling the check moved INTO the shared per-element resolver,
where both types already resolve, rather than being duplicated in the server
pass — gating a security check on a type was the defect, so the fix is one site
and not two. Order matters there and has its own test: the visibility answer
comes BEFORE the wrong-collection answer, because that reason's message names a
live item and would disclose existence first. `VisibilityChecked` is now set on
both arms, which supersedes the note saying the server re-resolves a ref-derived
`wrong_collection`; that also closes BUG-3012's TOCTOU window for values
reaching this rung, since there is no longer a second lookup to race.
**The empty list** (lead ruling, P17). `validate.go` said normalising `[]` to an
absent key was "the write door's job". No door did it. It now happens at the one
entry every write passes through, in the two spellings the two write shapes
need: a FULL write's `[]` becomes an absent key, and the ordinary required
machinery refuses it for free; a PARTIAL write's `[]` becomes the patch path's
nil deletion sentinel, because deleting the key from a patch would turn "empty
this field" into "do nothing" while reporting success.
**Required** was enforced for scalars only, so a required `multi_relation` had
no enforcement anywhere: the key is deleted after validation has passed, and
nothing re-checks it.
**Defaults**, twice. `ValidateFields` assigns a schema default and skips its own
type check, so the late pass is the ONLY thing between an injected default and
the blob — and it skipped arrays, so a schema defaulting to `42` stored 42, and
one defaulting to refs stored refs. The invisible-default drop skipped them
twice over: once on the type gate and again on a `.(string)` that fails on an
array, which is an accident rather than a decision.
**The CLI could not write the type at all.** Both coercion sites JSON-parse only
`json` and `multi_select`, so `--field 'members=["PEOPL-1"]'` arrived as a raw
string and the validator refused a shape the caller never sent.
**The copy preflight** reported `collection_unavailable: false` for a
multi_relation aimed at a deleted collection, so the dialog offered a picker it
could never fill and explained nothing. Codex did not name this one.
**A refused board ate every drag.** `isRelationGroup` is `type === 'relation'`,
so a multi_relation board took the scalar arm: `currentValue` is the stored
ARRAY, `targetColumn` a lane string, never equal, so every drop fired a write
that would replace the list with a scalar. The server refuses it and the reorder
is silently dropped — while the refusal notice rendered correctly the whole
time, which is what hid it.
## Instruments
25 new tests across four packages, each fix with its own. Mutation matrix, one
mutant per fix, every mutant compiled, every one KILLED by the test that should
kill it:
| mutant | died in |
|---|---|
| visibility block removed | all three visibility legs |
| visibility moved AFTER the collection check | the seen-target control only |
| required gate reverted | the required leg only |
| full-write normalisation removed | both full-write legs |
| partial spelling made a delete | both partial legs |
| late-default gate reverted | all four default legs |
| invisible-default multi branch removed | that leg only |
| preflight gate reverted | that leg only |
| coerce case reverted | the coercion leg |
| CLI case reverted | the CLI leg |
| board refusal gate removed | the board guard |
Controls throughout, because most of these legs assert a refusal and a fix that
refused EVERYTHING would satisfy them: a visible element still resolves, a nil
predicate still means "not asked" rather than "deny", a `wrong_collection` a
caller may see keeps its specific message, a key present before validation is
still the caller's value and not a default, an empty `multi_select` is untouched,
and a scalar relation still takes a bare string at both coercion sites.
Prose swept where this falsified it (CONVE-23): validate.go's arm no longer
claims a door does the normalising, `ValidateFieldsDetailed`'s doc now says it
runs one mutation before its traversal, and `cmd_item.go`'s "all other types …
relation — string is correct" no longer names relation flatly.
## Gates
`make test` 0 FAIL / 30 ok packages · `make lint` 0 issues · gofmt clean ·
`svelte-check` 0 errors · web vitest 2580 passed · `vite build` clean, and
`web/build` regenerated so the Go gates embed this tree. `make test-pg` NOT run
for this commit: no SQL changed, only Go-side type dispatch.
TASK-2999 / PLAN-2857 U4
Claude-Session: https://claude.ai/code/session_019EmDd3Wy8b3cPG1TZBSzBu
* fix(store): three defects in the previous commit's own fixes (TASK-2999 / PLAN-2857 U4, codex round 2)
Round 2 found three, and all three were in round 1's CORRECTIONS rather than in
the original code. That is the shape worth naming: I was reviewing the defect
and trusting the fix.
**A lenient element reader.** My `relationDefaultList` mapped a non-string
element to `""` on the reasoning that one bad element should not make the value
un-listable — and the resolver SKIPS a blank element, so `default: [42]` was
accepted, stored as `[""]`, and reported to nobody. A list is ONE value: an
element nobody can resolve makes the default unusable, exactly as a non-array
does. It is now strict about both non-string and blank elements, which is the
same shape rule `ValidateFields` applies to a caller's array, differing only in
disposition (dropped, not refused).
**An empty default on a required field, deleted in silence.** Validation had
just been satisfied by the PRESENCE of that default; deleting it afterwards with
no issue left a required field absent with nothing to refuse it.
`RequiredRelationIssues` promotes by key, so an issue is the only way a door
hears about it. Reported only when the field is REQUIRED: on an optional one an
empty default is a sensible way to say "starts with nothing", and a warning
there would be a false alarm on a schema doing nothing wrong.
**The order bug, and the sharpest of the three.** Every door captures relation
provenance with `RelationKeysPresent` BEFORE `ValidateFields` runs, so that
function saw the `[]` the new normalisation was about to remove. Counting it as
caller input marked the key "not a default"; validation then injected the schema
default into the hole normalisation had made; and the late-default pass skipped
the key on the strength of that mark. A caller sending `members: []` against
`default: 42` stored 42 — a value nobody sent, through the seam between two
correct-looking passes. `RelationKeysPresent` now reads an empty list as absent,
by the same rule, which is the one-site fix: the key is then a DEFAULT to every
later pass, which is exactly what it became.
Instruments: 5 new tests. Mutants, each killed by the test that should kill it —
lenient elements, blanks tolerated, the required report removed, the empty-list
rule reverted, and the empty-list rule LEAKING onto the scalar type (whose empty
spelling has its own meaning, BUG-3028).
One mutant SURVIVED a first pass and is recorded rather than quietly re-rolled:
"map a non-string element to empty string" is not a faithful negation of the
non-string rule, because its own output then trips the BLANK guard two lines
later and dies for the wrong reason. The faithful mutant stringifies instead —
`42` becomes `"42"`, which is neither blank nor resolvable — and that one is
killed by the non-string leg alone.
Gates: `make test` 0 FAIL / 30 ok · `make lint` 0 issues · gofmt clean.
TASK-2999 / PLAN-2857 U4
Claude-Session: https://claude.ai/code/session_019EmDd3Wy8b3cPG1TZBSzBu
* fix(store): three more seams the empty-list rule opened (TASK-2999 / PLAN-2857 U4, codex round 3)
Two of round 3's three were again in the previous round's fixes, and they are
the same class as round 1's: a SECOND path that does not share the rule. The
lead's ruling put the empty-list normalisation at the one validation entry every
write passes through; these are the passes that run BESIDE it and had their own
idea of what a value is.
**A cleared list survived a move.** `MigrateRelationReferents` SKIPPED a cleared
relation value so it would be neither refused nor reported — correct for a
scalar `""`, which validation accepts and which is a legal stored spelling. For
a LIST it is not enough: the migrate doors validate AFTER resolving, so an array
of blanks stayed in the map and failed the shape check. Reachable as a
destination-schema DEFAULT, since `ValidateFields` assigns one without
type-checking it — so the same schema default returned 400 on an ordinary move
and was quietly dropped on a create. The list is now DELETED there; the scalar
is untouched, and a control pins that asymmetry.
**The bulk door resolved a default as though it were typed.** `applyBulkOp`
snapshots relation provenance and then validates, which injects defaults; its
relation pass collected "the keys `changes` names, read out of the
post-validation map". Those are different sets the moment normalisation removes
an empty list before the injection: the map then held a DEFAULT under a name the
caller had used. An unresolvable default must be dropped and reported, never
refused — one bad default in a schema would otherwise fail every bulk update
into that collection. It now gates on the snapshot that was already being taken
three lines above.
**Hydration depended on the rest of the batch.** A stored `multi_relation`
holding no resolvable reference got no `relation_targets` entry when read alone,
and an empty list when read beside an item that carried one — identical stored
bytes, two answers, decided by the batch. An absent key says "nothing hydrated
this field" and `[]` says "this field points at nothing"; those are different
statements and a caller cannot be handed either at random. The early return is
gone rather than special-cased: falling through costs nothing (the chunk loop
does not run on an empty id list) and duplicates nothing, where a second render
path is exactly how the two answers would drift apart again.
Instruments: 5 new tests. Mutants, each killed by the test that should kill it —
the cleared list skipped rather than deleted, the deletion LEAKING onto the
scalar spelling, the hydration early return restored, the bulk gate removed, and
the bulk gate widened to skip every relation key. The last two are a pair: one
proves the gate fires, the other proves it still lets a genuinely supplied value
be refused.
Gates: `make test` 0 FAIL / 30 ok · `make lint` 0 issues · gofmt clean · web
vitest 2580 passed.
TASK-2999 / PLAN-2857 U4
Claude-Session: https://claude.ai/code/session_019EmDd3Wy8b3cPG1TZBSzBu
* refactor(store): one shape reader composes the next, and the three questions are named (TASK-2999 / PLAN-2857 U4)
CONVE-35's move rather than another review round: rounds 1, 2 and 3 each found a
new member of a class, which measures the enumeration and not the code. The
class rounds 2 and 3 belong to is THREE READERS OF ONE SHAPE — `relationArray-
Elements`, `relationDefaultList` and `relationListShape`, plus a fourth across
the package boundary in `normalizeEmptyRelationLists`. Both rounds found the
wrong one being asked in the wrong place.
`relationDefaultList` was a second parser with the same two cases as the general
reader. It now COMPOSES it and adds exactly one rule — no blank elements —
because that one rule is the entire difference and a line saying so cannot drift
from a copy that repeats the parsing.
`relationListShape` stays separate and that is deliberate, not an oversight: it
must admit an array with non-string elements, since a caller's malformed array is
still a value they supplied and only an EMPTY one is the absent spelling. The
three questions are now written down together where the first reader is defined,
so a fifth reader has to declare its question before it gets written.
Mutants on the composition, both killed: the blank rule dropped, and the shape
result ignored.
Gates: `make lint` 0 issues · `make test` 0 FAIL.
TASK-2999 / PLAN-2857 U4
Claude-Session: https://claude.ai/code/session_019EmDd3Wy8b3cPG1TZBSzBu
* fix(store): a caller's malformed override is refused, a schema's is dropped (TASK-2999 / PLAN-2857 U4, codex round 4)
Both of round 4's findings were in round 3's fix, which is the third round
running where the review found my CORRECTION rather than the original code.
Round 3 added a deletion for a cleared `multi_relation` on the migrate path, and
put it BEFORE the origin split — so it treated a caller's malformed override
exactly like a schema's malformed default. Those get opposite dispositions on
purpose: a value someone ASSERTED is refused, a value nobody asserted is dropped,
and "silently clear the field" is neither.
**`[" "]` is not a cleared field.** It is a list of ONE reference that happens to
be blank, a shape every write door refuses. Counting it as cleared made a move
with `field_overrides: {"members":[" "]}` return 200 with the field absent: the
caller's own malformed value vanished instead of being refused. "Cleared" now
means the EMPTY list and nothing else, which is the same spelling the validators
normalise — the analogy with the scalar `" "` was the mistake.
**A mixed default `[validID, " "]` still 400'd a copy.** The destination-default
bucket used the LENIENT shape reader ("is this an array of strings"), so it
passed, resolution skipped the blank, and the shape check afterwards refused it —
on an optional field, for a default the caller never wrote. That bucket now uses
a strict reader, and the strict/lenient split has a named helper explaining why
the two origins are held to different standards: a refusal is visible to the
person writing the request, a drop is not.
Instruments: 6 new tests. Mutants: blanks counted as cleared again (killed by the
supplied-override leg), the default bucket back to the lenient check (killed by
both default legs), and the empty list no longer clearing (killed by its own leg).
ONE MUTANT SURVIVED and it found a coverage hole older than this unit. Applying
the strict list reader to the SCALAR type as well — which would drop EVERY scalar
relation default on a copy or move — survived the entire `internal/store` suite,
first under a `-run` filter narrower than its population and then unfiltered too.
Nothing covered the scalar arm of that bucket at all. Two tests now do, and the
mutant dies in the first of them.
Gates: `make test` 0 FAIL / 30 ok · `make lint` 0 issues · gofmt clean.
TASK-2999 / PLAN-2857 U4
Claude-Session: https://claude.ai/code/session_019EmDd3Wy8b3cPG1TZBSzBu
* fix(store,web): the third origin and the second view (TASK-2999 / PLAN-2857 U4, codex round 5)
Both findings were incomplete ENUMERATIONS of mine — the same failure round 1
was about, twice more, on smaller sets.
**Three origins, and I had settled two.** Rounds 1-4 gave SUPPLIED the refuse
disposition and DESTINATION DEFAULT the drop disposition, and left CARRIED on
the lenient shape check. A carried value is asserted by nobody either —
TASK-2878's rule is that it is dropped, never refused — so a carried
`[validID, " "]` passing the lenient check meant resolution skipped the blank
and the shape check afterwards REFUSED an ordinary same-workspace move on an
optional field, which is the exact outcome that rule exists to prevent. The
helper is renamed from `relationDefaultShapeIsUsable` to
`relationDroppableShapeIsUsable`, because naming it for ONE of its two origins
is what let the other keep the wrong check for a round.
**Two views implement grouping, and I fixed one.** ListView had both of the
defects BoardView had: the drop handler compared the stored ARRAY against a lane
string and fired `onStatusChange(item, "")` on every reorder, and the status chip
was withheld only for `isRelationGroup`, so a refused multi_relation board left
it live to write a scalar into a list.
THE BOARD'S CHIP GATE WAS REMOVED AFTER BEING WRITTEN. A mutant reverting it
survived, and the reason is structural rather than a weak test: a refused
grouping takes the `field?.options ?? []` branch of `columns`, a multi_relation
declares no options, so `ItemCard.statusCyclable` (which needs
`statusOptions.length > 1`) is already false. The gate was unreachable by
construction — the same disposition W7's write-side duplicate guard got. ListView
genuinely needs its gate, because its `statusOptions` is a CALLER PROP rather
than its own lanes, and its mutant dies. Both facts are written at their sites.
TableView needs neither: no grouping, and its chip is gated on
`field.key === 'status' && field.options`, which a multi_relation cannot satisfy.
Instruments: 8 new tests. Mutants: carried bucket back to lenient (killed),
carried bucket dropping every list (killed by the control), ListView drag gate
reverted (killed), ListView chip gate reverted — BOTH halves, since either half
alone suppresses the chip and a one-half mutant dies for the wrong reason
(killed), and the chip withheld ALWAYS (killed by both views' ordinary-grouping
controls).
A VACUOUS TEST was caught by its own control, which is why the control was
written first: my ListView chip leg omitted `statusOptions`, which is a prop of
that view and not something it derives, so the chip never rendered and the
assertion held against any build. The control failed, which is the only reason I
looked.
Gates: `make test` 0 FAIL / 30 ok · `make lint` 0 issues (run three times — one
earlier run exited 3 inside a piped command that swallowed its message; three
clean unpiped runs since, and no explanation offered for the one, since I cannot
reconstruct it) · gofmt clean · web vitest 2585 passed · `svelte-check` 0 errors
· `vite build` clean.
TASK-2999 / PLAN-2857 U4
Claude-Session: https://claude.ai/code/session_019EmDd3Wy8b3cPG1TZBSzBu
* fix(web,server): the guard I deleted was reachable, and an empty override is not "your value" (TASK-2999 / PLAN-2857 U4, codex round 6)
**Round 5's reasoning was wrong and this restores what it removed.** I added a
refusal gate to BoardView's status chip, saw the mutant reverting it SURVIVE, and
concluded the gate was unreachable: a refused grouping takes the
`field?.options ?? []` branch of `columns`, and a `multi_relation` declares no
options, so the chip cannot cycle anyway.
Every step of that is true of the schemas anyone would WRITE and none of it is
enforced. Nothing strips `options` when a field's type changes, so a
`multi_select` retyped to `multi_relation` in the schema editor keeps them,
`columns` is non-empty, and the chip cycles — writing a scalar into a list. A
claim about schemas is not a claim about code; the preflight's own
`collection`-on-a-`select` comment says exactly that about a neighbouring field,
and I had read it earlier the same day.
So the surviving mutant was evidence about my FIXTURE, not about the guard. The
test now builds the retained-options schema and the mutant dies. The lesson is
narrower than "mutants lie": a survivor is a question, and the first question is
whether the fixture can reach the case — which is the same
check-the-faithfulness step that caught two survivors earlier in this unit and
which I skipped on this one because the conclusion was convenient.
**An empty override is not the caller's value.** The preflight already treats an
explicit `null` override as "leave this unset": it drops the key from `final` AND
from `origin`, so the destination default validation injects is reported as a
default. An empty `multi_relation` list is the same statement in the other
spelling — normalisation removes it, the default lands in its place — and it took
the other branch, so the preflight labelled that default `from: "override"`. On
the one surface whose entire job is to predict what a copy will do, attributing a
value to a caller who never sent it is the whole failure.
`items.IsEmptyRelationList` is now exported and both the normalisation and the
preflight ask it, rather than the preflight re-deriving a rule that lives in
another package.
Instruments: 4 new tests. Mutants: the preflight branch reverted (killed), the
preflight dropping EVERY override (killed by the control), and the board chip
gate reverted in BOTH halves (killed — where round 5's identical mutant survived,
which is the measurement that matters here).
Gates, each run unpiped after an earlier piped one swallowed a lint failure:
`make lint` exit 0 / 0 issues · `make test` exit 0, 0 FAIL / 30 ok · gofmt 0
files · `svelte-check` exit 0 / 0 errors · web vitest 2585 passed · `vite build`
exit 0.
TASK-2999 / PLAN-2857 U4
Claude-Session: https://claude.ai/code/session_019EmDd3Wy8b3cPG1TZBSzBu
* fix(items,server,web): every spelling of none, every refusal-gated affordance, and a lost update in my own picker (TASK-2999 / PLAN-2857 U4, codex round 7)
Round 7 was asked to ENUMERATE rather than confirm, and returned five. Three
were siblings of round 6's fix — which is what an enumeration round is for, and
what six confirmation rounds had not produced.
**One sweep replaces round 6's point fix.** An empty `multi_relation` is about to
be removed by normalisation, and the destination default lands in its place, so
whatever `origin` says about the removed value gets said about the default
instead. Round 6 put that check in the preflight's OVERRIDE loop. That loop sees
the caller's RAW bytes, so `"[]"` is still a string there and only becomes an
empty list once `CoerceFields` runs; and it sees only overrides, so a CARRIED
empty list kept `from: "migrated"` by the same mechanism. One sweep over the
coerced map covers the override, the carried value and both string spellings,
because by that line they are all the same value.
**NULL is the third spelling.** On a full write the traversal treats a
present-but-nil key as absent for required-ness and defaults but does not REMOVE
it, so `{"members": null}` was stored — a third representation of "no targets"
beside the absent key and the empty list. Normalised on the full path only: on a
patch, nil is the deletion sentinel and the spelling being normalised TO.
**The lane create controls are refusal-gated too.** A `multi_relation` that
retained `options` produces NAMED lanes, so neither `isUncategorized` nor
`isRelationGroup` withheld the lane "+" or the lane menu's add: creating there
sent the lane string as the relation value. Third member of the affordance class
after the drag and the status chip — everything that writes the GROUP VALUE has
to ask the refusal.
**A lost update in W7's own picker.** Every list edit is a WHOLE-LIST write, and
they all derived from `value`, which only catches up after the server round trip:
remove A then B from [A,B,C] sent [B,C] and then [A,C], and the parent's 409
refetch-and-retry can persist the second, putting A back. The component now holds
the list it last SENT as the base for the next edit, released as soon as the prop
agrees by CONTENT.
THE FIRST VERSION OF THAT FENCE ONLY WORKED IN THE TEST. I cleared the hold from
an `$effect` reading `wsSlug` and `field.key` directly, which re-runs on every
prop assignment whether or not anything changed — so an ordinary parent
re-render mid-write released the hold and the stale prop came back as the base.
The two-click test never saw it, because nothing re-renders between two clicks.
A surviving mutant ("never release the hold") is what pointed at it, and the leg
that now pins it re-renders with IDENTICAL props mid-write. The hold is stamped
with the identity it was sent for and compared, which asks the question the fence
is for instead of one that merely correlates with it.
Instruments: 12 new tests. 12 mutants, all killed, including the four that
survived earlier passes and were each chased to a cause: a fixture that could not
reach the case, a fixture whose two outcomes coincided, a guard masked by its
neighbour, and the effect-timing one above.
Gates, unpiped: `make lint` exit 0 / 0 issues · `make test` exit 0, 0 FAIL / 30
ok · gofmt 0 files · `svelte-check` exit 0 / 0 errors · web vitest 2592 passed ·
`vite build` exit 0.
TASK-2999 / PLAN-2857 U4
Claude-Session: https://claude.ai/code/session_019EmDd3Wy8b3cPG1TZBSzBu
* fix(store,server): two more spellings, and a shape answer that depended on the route (TASK-2999 / PLAN-2857 U4, codex round 8)
Round 8 returned five. Two are members of classes already closed in this unit and
are fixed here; the other three open NEW classes and are deliberately NOT fixed —
they are the first rows of the resume table on TASK-2999, per the lead's
disposition for this round.
**The coerced NIL spelling.** `CoerceFields` JSON-parses a `multi_relation`
string, so the override `"null"` reaches the preflight's post-coercion sweep as a
nil — a spelling `IsEmptyRelationList` deliberately does not claim, since nil is
the PATCH path's deletion sentinel and that rule belongs to the validators. At
the sweep the question is only "will this value survive to be described", and a
nil will not, so the destination default that replaces it was again labelled
`from: "override"`.
Scoped to the list type, and a CONTROL pins that: a `json` field's `"null"`
coerces to nil too, and there nil is a legitimate stored value rather than a
spelling of absence. A mutant dropping every coerced nil survived until that
control existed.
**A shape answer that depended on which door asked.** `CoerceFields` runs BEFORE
`ValidateFields` injects a default, so an injected default is uncoerced at the
create door and coerced at the migrate door, where the copy coerces after
merging. A default written as the JSON TEXT `["<id>"]` was therefore stored by
one path and dropped as `invalid_shape` by the other — the same schema, the same
value, two answers chosen by the route. Droppable-shape judgements now run the
value through the real coercion first; the control is that text parsing to a
NUMBER is still not a list.
Mutants, all killed: the raw-value judgement restored, the shape check made
unconditional, the nil arm removed from the sweep, and the nil arm widened past
multi_relation.
Gates, unpiped: `make lint` exit 0 / 0 issues · `make test` exit 0, 0 FAIL / 30
ok · gofmt 0 files · `svelte-check` exit 0 / 0 errors · web vitest 2592 passed ·
`vite build` exit 0.
TASK-2999 / PLAN-2857 U4
Claude-Session: https://claude.ai/code/session_019EmDd3Wy8b3cPG1TZBSzBu
* fix(web): two whole-list writes to one field can no longer lose the newer one
Codex round 8, R8-1 and R8-2, fixed as one unit because they are two halves
of one question: which write to this field is current.
R8-1, the lost update. Every multi_relation edit is a WHOLE-LIST write, so
two of them in flight are not deltas the server can merge — each asserts the
complete value. Remove A then B from [A,B,C]: both PATCHes carry the same
expected_updated_at, the second commits first, and the first then 409s,
refetches and REPLAYS [B,C]. A is back on the row, with no error raised
anywhere. Round 7 fixed where the second edit got its BASE; this is the
ordering, which is a different mechanism.
R8-2, the hold outliving its write. The hold released only on prop
AGREEMENT, and a value the server REFUSES never comes back — so a rejected
clear of a required field kept showing the rejected list forever and ignored
every later server value.
The shared model is `$lib/items/fieldWriteOrder`: two counters per target, a
ticket at dispatch and a high-water mark of what WROTE, because each
single-counter version answers a different question than the one being asked
(harness note ticket-plus-high-water-request-ordering). The dispatch counter
gates the RETRY — the one place counting dispatches is right, since a retry
is a new send of an OLD body and nothing here discards a response. The
applied mark gates every write of `item`, giving newest-wins among responses
that resolve out of order. The field editor takes the same tickets so an
older write settling cannot release a newer one's hold, and the hold now ends
when the write is OVER: prop agreement, or the consumer's own settlement
where it gives one (`onchange` may now return a promise; returning nothing is
unchanged and still supported, which is what the consumers with no server
behind them do).
The pane's OCC retry loop is EXTRACTED rather than guarded in place:
ItemDetail is ~7,900 lines and not renderable, so the defect site would
otherwise only ever be grepped for spellings. It is now driven directly,
conflict by conflict, through the interleaving that loses the update.
Instruments — 23 tests, three files:
- fieldWriteOrder.test.ts drives the real loop, with a NEGATIVE CONTROL per
single-counter alternative: no supersession question at all, and counting
applies instead of dispatches, each of which puts A back.
- FieldEditor.multiRelation: a refused write stops holding and shows server
truth, the next edit bases on the prop again, an older write settling does
not release a newer one's hold, and a CONTROL that a consumer answering
nothing still holds (round 7 preserved).
- itemDetailOrdersFieldWrites.test.ts guards the binding (CONVE-19), with its
own limits written down.
Mutation matrix, 7 mutants. Killed: settlement release disabled (3 legs),
release without the ticket check (1), the catch's supersession check, one
claim gate, the `order` wiring, a per-item instead of per-field target.
SURVIVED and each chased rather than re-rolled: moving `take` to just before
the first send is behaviour-preserving and not a defect, so that leg was
REMOVED rather than kept as rigour nothing can falsify; dropping the flush
before release survives the whole file and is KEPT and labelled, because
unlike W7's duplicate guard it is not unreachable by construction — the
signature permits a consumer that resolves before it assigns.
The proximity bound in the binding guard has its receipt: assignments sit at
35/35/60/156/207 characters from their gate, deleting one gate pushes two to
359 and 410, so the bound is 300. It was 400 first, which caught only the
farther of the two and by ten characters.
TASK-2999 / PLAN-2857 U4
Claude-Session: https://claude.ai/code/session_019EmDd3Wy8b3cPG1TZBSzBu
* fix(web): a board that refuses to group stops grouping (R8-3)
Codex round 8, R8-3. Round 7 gated every affordance that WRITES the group
value and left the LANES reading it, so the board went on bucketing under a
notice reading "Showing everything ungrouped."
The route is the same false premise round 6 corrected one derivation away:
nothing strips `options` when a field's type changes, so a `multi_select`
retyped to `multi_relation` keeps them — and a lane key is the STRINGIFIED
array, so `['id-red']` and `['id-red','id-blue']` match options `id-red` and
`id-red,id-blue` and the two cards land in two scalar-labelled lanes.
`columns` is now empty when grouping is refused, which sends every item to
UNCATEGORIZED and leaves exactly the one lane the notice describes. One lane
rather than keeping the notice and dropping it: it is the smaller change, it
makes the sentence true, and it matches the day-64 ruling that this unit
refuses grouping rather than reinventing it. IDEA-3034 holds the multi-lane
alternative. Taken as the lead's stated DEFAULT — no ruling on the product
question had reached this window.
WHAT THIS DID TO THE ROUND 5/6/7 GUARDS, measured rather than assumed. With
no named lanes, three of the four affordance gates are subsumed by the lane
they now cannot reach: deleting `groupingRefusal` from `cardStatusOptions`,
from the lane "+", or from the lane menu's add leaves all 63 tests in that
directory green. The drag one is NOT subsumed — its mutant still dies. The
gates are KEPT, because each affordance asking the refusal directly is what
made round 7's class enumerable; what is not kept is the pretence, so both
legs that stopped discriminating say so in place, naming the leg that does
fail when this regresses.
A forcing term that showed the lane even when empty was written and then
REMOVED: `items.length === 0` renders EmptyState instead of the lanes, so no
mutant could reach it. W7's disposition, not a comment explaining a guard
nothing can falsify.
Instruments: one leg with the finding's own fixture — two cards, retained
options matching their stringified values — asserting ONE lane holding BOTH,
with the refusal and both cards as preconditions so "one lane" is neither
"not refused" nor "a dropped card". Reverting `columns` turns it and the
round-7 lane-control leg red.
Gates on the tip, each run unpiped: `make lint` exit 0 / 0 issues · `make
test` exit 0, 0 FAIL / 30 ok · gofmt 0 files · `svelte-check` exit 0 / 0
errors · web vitest 2616 passed · `vite build` exit 0 with web/build
regenerated. `make test-pg` not run and not owed — no SQL in any commit.
TASK-2999 / PLAN-2857 U4
Claude-Session: https://claude.ai/code/session_019EmDd3Wy8b3cPG1TZBSzBu
* fix(web): the round-8 enumeration's in-class findings, including two this unit introduced
ONE enumeration-prompted codex round on the frozen tip, per the lead's
disposition: not "is this right?" but "enumerate every remaining way a write
can leave the client and the server disagreeing". It returned 36 items across
four categories and said no category was empty — which is CONVE-24 working:
an enumeration is the only round that can return "none left" as a finding,
and this one did not.
Fixed here: the members of classes this unit had already closed, and the two
defects the unit's OWN round-8 fix introduced.
MINE, both in the open-children path, both from gating too widely:
1. The DECLINED branch took a claim. Declining writes nothing — it re-props
the value the client already had — but claiming advanced the applied mark,
so a write that had genuinely COMMITTED and was still in flight lost its
claim on return. Server changed, pane showing the old value, nothing left
to correct it. A branch that asserts no new server truth must not take the
mark for one.
2. A successful FORCED write was discarded on supersession. If the newer
write then failed without writing, the server held the confirmed value and
the pane held the old one. `claim` asks the question that matters — has
anything newer actually WRITTEN — and supersession does not.
IN-CLASS, four more:
3. The confirm callback SENDS, and asked about supersession only after it
returned. The dialog can sit open indefinitely; checking afterwards
suppresses the display of a write that already reached the server.
4. `submitOrderedOCC` asked only about supersession after its refetch, so
switching items during that await still produced another PATCH at the item
the pane had left — invisible, because the caller discards the response,
and landing on the row all the same. Both questions now, both times.
5. The hold's release: prop AGREEMENT is a coincidence test, not an answer.
Remove C from [A,B,C] and add it straight back, and the second write holds
a list equal to the prop nobody has changed — the effect read that as
confirmation and released mid-write, the first write's answer became the
base, and C was lost by the mechanism built to stop that. Settlement now
owns the release where it exists; agreement keeps it where it does not.
A synchronously-throwing consumer no longer strands the hold, and the
agreement path checks the identity stamp it had been ignoring.
6. The lane menu's bulk "Move all to" is the FIFTH affordance that writes the
group value; round 7 enumerated four. It offers `statusField.options` as
destinations, so a `status` retyped to `multi_relation` with its options
retained offers lanes that do not exist and sends a scalar into a list.
A SOURCE FILE WAS BINARY, and that is worth its own paragraph because it made
this unit's own verification lie. `fieldWriteTarget` was written with a real
NUL BYTE where the source should carry the two-character escape for one —
identical at runtime, but `file` reports "data" and grep then returns NOTHING
AND EXIT 1 for every pattern, which is indistinguishable from "not present".
A restore-and-verify grep over that file came back empty for a symbol four
lines below it. It shipped in a9f58ccb; git still shows that path as `Bin` in
this diff.
INSTRUMENTS — 9 new legs, every one shown red against the unfixed code:
the agreement release (its first fixture gave the same answer either way and
was rebuilt so it discriminates), the retarget stamp, the synchronous throw,
the post-refetch item switch, the bulk move (its first version passed against
a menu that never opened — it now proves the menu opened, with an entry that
is not gated on the refusal), and four source-guard legs pinning the two
dispositions above, the claim-as-GATE spelling codex showed was passable with
the `if` removed, and the callback's guard-before-send order. Two error
assertions now compare IDENTITY rather than class, which they were shown to
need.
NOT FIXED, filed as the lead ruled — new classes go into the package as
follow-up items, not another round: BUG-3036 (whole-item responses unordered
against each other), BUG-3037 (OCC cannot separate two writes inside one
second — the client guard never even runs), BUG-3038 (a whole-list retry
replays instead of re-deriving), BUG-3039 (a debounced edit cancelled by the
previous keystroke's response), BUG-3040 (a refused write leaves the rejected
value in scalar inputs), BUG-3041 (a conventional key typed multi_relation
crashes card rendering), BUG-3042, BUG-3043, BUG-3044, BUG-3045 (`npm test`
can exit 0 having run no .svelte.test.ts at all), BUG-3046, BUG-3047,
BUG-3049 (full-fields writes without OCC undo a field patch ON THE SERVER),
TASK-3048.
One existing test now describes a KNOWN LOSS and says so: the
ordinary-conflict leg asserts that a third party's addition disappears, which
is what the code does and not what it should do (BUG-3038). Invert it when
that lands.
Gates on the tip, each run unpiped: `make lint` exit 0 / 0 issues · `make
test` exit 0, 0 FAIL / 30 ok · gofmt 0 files · `svelte-check` exit 0 / 0
errors · web vitest 2625 passed · `vite build` exit 0 with web/build
regenerated. `make test-pg` not run and not owed — no SQL in any commit.
TASK-2999 / PLAN-2857 U4
Claude-Session: https://claude.ai/code/session_019EmDd3Wy8b3cPG1TZBSzBu
* test(web): stop the deliberate uncaught error failing the whole vitest run
CI's Web check went red at
|
||
|
xarmian
|
185194caf6 |
feat(items): content_state marks a body the server knows is stale (BUG-3000) (#1343)
* feat(items): content_state marks a body the server knows is stale (BUG-3000) A read of items.content could serve a body the server KNEW was behind the item's live collaborative document, with no way for the caller to tell. BUG-2995 fixed the write response; this is the read side. The window is UNBOUNDED, which is what makes this a marker rather than a documentation note. Nothing server-side moves applier content into items.content: PruneSweep prunes and never flushes, and its candidate query requires MAX(op-log.id) <= content_flushed_op_log_id, so an item in exactly this state is EXCLUDED from the sweep. The content is durable (those rows are retained BECAUSE they are unflushed) and the row stays behind until a tab next opens the item. ON THE ITEM, NOT THE DOOR. Reads return a bare item or array — there is no warnings envelope on the read side — so a per-door marker would be N changes across a population that grows whenever someone adds a handler returning a models.Item. An additive omitempty field on the struct is inherited by every door by construction, and a consumer that does not know it sees byte-identical responses. EXISTS, not MAX: the op-log's (item_id, id) index makes this a seek that stops at the first qualifying row, where the sweeper has to aggregate across the whole per-item range. Same index, different cost class — which is why the sweeper is the wrong precedent to copy here even though it is the obvious one. The marker tracks the CONTENT COLUMN, not the row. Four queries select an empty-string literal for content (dashboard and child batching, the index and changes readers); marking those would be a false signal about a body they never return, so they are exempt and say so at the site. ListItems builds both columns together for the same reason, so NoContent can never mark an omitted body. The population is 15 SELECT sites, not the 10 an items.go-only sweep finds: search.go and item_stars.go carry content too, and ListStarredItems shares the scanItems helper across that boundary — which is how the first pass failed, at runtime rather than at compile time. Summaries carry content_preview, so ItemSummary gets the field too; a preview is content. RECEIPT in the code: measured with a counterfactual arm, because a one-armed timing cannot attribute cost. 200 items on SQLite, best-of-5: with the predicate 2.44 / 4.00 / 3.81ms at 0 / 2k / 20k op-log rows, without it 3.58 / 2.86 / 3.27ms. The arms are indistinguishable and at 0 rows the arm WITHOUT was slower, so the honest statement is "below the noise floor at this scale", not "free". Untested scales are named in the comment. Tests assert both directions on the same item — a marker that is always on is indistinguishable from one that is always right. Mutation-tested: an always-true predicate is killed only by the clearing leg, a NoContent leak only by the omitted-body leg, and a never-set marker by all three. Both backends; the Postgres leg verified to RUN, not skip. Also corrected under CONVE-23: warnContentPendingFlush's comment said the flush question was "not established". It now is, and it says so. Claude-Session: https://claude.ai/code/session_01PDYjP67KTAcqpFpo3ijAAA * docs(items): restore an orphaned doc comment; name the export door (BUG-3000, codex r1) Two corrections from round 1, one of them mine to the population claim. The contentStateSQL const was inserted between getItemScanQ's doc comment and its func, orphaning that comment and attaching it to the const. Moved above it. POPULATION CORRECTION. Codex flagged content reads outside the 15 projections, and one is real: Store.ExportWorkspace selects content straight into models.ItemExport, so an export taken during a pending-flush window captures the stale body — and ImportWorkspace writes it back as canonical in the destination, where the op-log that held the real content does not exist. Filed as BUG-3032 rather than folded in: ItemExport is a portable bundle schema, not models.Item, so marking it is a format change the import path has to tolerate, and its query does not even use the i. alias contentStateSQL requires. Checked and correctly OUT: wiki_links.go's cascade re-read and items.go's in-tx re-read (internal rewrites, never returned to a caller), RemapAttachmentReferencesInWorkspace (same), the documents/templates/versions tables (different subjects), and item_versions (BUG-3031, split out by ruling). Claude-Session: https://claude.ai/code/session_01PDYjP67KTAcqpFpo3ijAAA * test(items): cover every spliced door; mark the public share DTO (BUG-3000, codex r2) Round 2's P1 and the coverage gap behind it, plus a correction to my own comment. THE COVERAGE GAP IS THE IMPORTANT HALF. The content-state tests exercised two of the fifteen spliced queries, so "the suite is green" was never evidence the other thirteen were column/Scan aligned — and a mismatch there is a RUNTIME error that leaves go build green, which is exactly how the first implementation pass failed. TestEveryContentBearingReadDoorCarriesTheMarker now drives all thirteen reachable store doors against one stale item. It is an alignment guard first and a behaviour test second: a door returning the right marker necessarily has a matching destination. ListStarredItems gets its own leg by name — it reaches the shared scanItems helper from outside items.go, which is the boundary that broke. P1, folded in rather than filed: publicShareItemDTO is an explicit allow-list, so it does not inherit the marker. A public share is the door whose reader is least able to notice on their own — an anonymous viewer has no editor, no op-log, nothing to compare against. Added only when set, so the key set stays byte-identical in the common case and TestMovedTo_ShareLinkNeverCarriesPointer's pin is undisturbed. MY OWN ERROR, corrected: the scripted exemption comment landed on ItemsModifiedSince instead of GetChildItemsForParents, so a comment reading "NO contentStateSQL here, this query selects an empty-string literal" sat directly above a query that has the predicate and selects i.content. A non-unique anchor edited the wrong match. Both exemptions are now on the functions they describe, verified by a check that reads what is actually below each one. Four further doors build their own shape and are filed as BUG-3033 rather than folded in, because each needs a rendering decision rather than a field copy: artifact export, playbook run, bootstrap convention bodies, and the MCP item resource (verified directly — the JSON it fetches now carries the field, and formatItemAsMarkdown re-renders without it). The playbook and bootstrap halves rank first there: an agent ACTS on those bodies. make test 0 FAIL; make test-pg exit 0, 0 FAIL, no banner, all 13 door legs PASS on Postgres too; make lint 0 issues. Claude-Session: https://claude.ai/code/session_01PDYjP67KTAcqpFpo3ijAAA * test(items): close two coverage holes in the coverage guard (BUG-3000, codex r3) Both P2s are holes in the test built to close holes, which is the part worth saying. ResolveItemIncludeDeleted was called with a SLUG. That falls through to GetItemBySlugIncludeDeleted, so the leg passed while measuring a different function and this one's own ref SELECT — a separately spliced query — stayed unexercised. Now driven by REF, with ResolveItem added alongside. The enumeration ran SQLite only. The FTS branches are DIALECT-SPECIFIC: ListItems (search) and Search take different SELECTs per backend, so a SQLite-only run left the Postgres variants of those spliced queries uncovered — exactly the runtime-mismatch hazard the test exists for. The whole enumeration is now parameterised over both backends: 14 door legs each. Two P3s: The CLI warning fired inside the empty-body check, so an item whose STORED body is empty while its document holds the real text — this bug's own shape — printed nothing and warned nothing. It now warns before the check, which is the case where a reader has least to go on. The TypeScript type was `string`, broader than the server's vocabulary. It is the literal 'applied_pending_flush', matching ItemWriteWarnings.content_outcome. Gates VERIFIED at this commit: make test 0 FAIL, make lint 0 issues, gofmt clean, and all 14 door legs passing on BOTH backends. make test-pg is being re-run alone: the previous attempt overlapped a second run of the same target in this worktree, which shares a compose project, so its teardown killed the other's database mid-run — 47 connection-refused lines and failures confined to pre-existing Postgres tests. That run is INDETERMINATE, not red, and is not being reported as a pass either. Claude-Session: https://claude.ai/code/session_01PDYjP67KTAcqpFpo3ijAAA * fix(share): the public COLLECTION share carries the staleness marker too (BUG-3000) Night 19's P2, and it is the same door class I folded in for the single-item share one round earlier — I fixed one allow-list and missed its sibling twelve lines away. Both public-share doors build an explicit allow-list rather than serialising models.Item, so neither inherits the marker the way a struct-serialising door does. Both serve an ANONYMOUS reader, who is the least able to notice staleness unaided: no editor, no op-log, nothing to compare the body against. Set only when non-empty, mirroring the sibling, so the key set stays byte-identical whenever the row is current — which is what the existing exact-shape pin on the item DTO depends on, and what a marker that were always present would break while satisfying every positive assertion. The test covers BOTH doors in BOTH directions on one item: absence asserted FIRST so the presence assertions cannot be read as an always-on marker. Mutation-tested independently — removing the collection half kills only the collection leg, removing the item half kills only the item leg. No cross-talk, so neither door is riding on the other's coverage, which is exactly how this one was missed. A note left at the site rather than in this message alone: if a third allow-list over item bodies ever appears, these should become one helper instead of a third copy. make test 0 FAIL; make test-pg exit 0, 0 FAIL, no banner (run alone — an overlapping run of the same target in one worktree shares a compose project and its teardown kills the other's database mid-run); make lint 0 issues; gofmt clean. Claude-Session: https://claude.ai/code/session_01PDYjP67KTAcqpFpo3ijAAA |
||
|
David Barkhausen
|
1a041b33ff |
feat(cli): pad token rotate — new secret in place, old one dies with the write (#1237 follow-up) (#1330)
The rotate endpoint (POST /auth/tokens/{id}/rotate) existed server-side
with no CLI in front of it; the #1237 review asked for it as its own PR.
- exact id only, same discipline as revoke: a typo is the server's 404
surfaced, never a wrong token rotated (which kills a live secret
somewhere)
- the new secret prints exactly once (count-pinned) with the store-it-now
notice, and the output says plainly that the old secret already stopped
working — RotateAPIToken replaces the hash in place, no grace window,
so the wording must not imply one
- --expires-in <days> posts a new expiry (server-capped by max lifetime);
without it no expires_in is posted and the original expiry is preserved
(test-pinned both ways)
- README documents rotate beside create/list/revoke, including that
rotate shares create's session_required gate (#1267)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
||
|
xarmian
|
162eccb42b |
feat(relations): exact-title resolution and hydrated reads for relation fields (PLAN-2857 U6 / TASK-2996) (#1326)
PLAN-2857 unit U6, closes TASK-2996.
U1 made a relation value have to name a live item. U6 adds the third
spelling a human actually knows — the item's TITLE — and the read side
that lets a client render any of them.
- a value may be a UUID, an issue ref, or an EXACT TITLE, in that
order, so an item literally titled COLO-3 is unreachable by title
while the ref COLO-3 resolves;
- title matching is scoped to the collection the field declares: a
title unique only workspace-wide REFUSES naming the collection
searched, and two or more matches inside it are `ambiguous` rather
than `not_found`, which would state the opposite of what happened;
- reads carry `relation_targets` — field key to {id, ref, title} —
BESIDE `fields` rather than inside it, so read-modify-write keeps
working;
- the --fields DSL takes a relation's target collection. It used to put
the third part into Options and build a field with NO target, which
every subsequent write then refused with target_missing; a bare
`owner:relation` is refused at parse time now;
- ToolSurfaceVersion 0.31 — a behaviour bump, because that DSL refusal
is a call that used to be accepted.
THE RULED PROPERTY: among live exact-title matches in the declared
collection, only VISIBLE ones count — zero is not_found, exactly one
RESOLVES, two or more is ambiguous — and a hidden match never changes the
answer a caller gets.
It took three attempts to put that in the right place. It was a
SERVER-SIDE second pass first, which left the cross-workspace copy —
running the resolver inside its own transaction — deciding without it, so
preflight and copy disagreed about one input. It lives in the resolver
now, taking the store.RelationVisibilityFunc this codebase already had
for the shape, and which resolveRelationsForWrite's own comment had named
as the missing piece. nil means no requester and every live match counts.
CARRIED values are exempt by provenance: nobody asserted them, and
judging them by the mover's visibility is what MigrateRelationReferentsQ's
comment calls out as making the stored bytes depend on who moved the
item. Supplied values and injected defaults are judged.
EIGHT CODEX ROUNDS; seven found something real. The ones worth keeping:
a CAP on the candidate list was itself an oracle (one visible plus cap
hidden read differently from one visible plus cap+1); the paging cursor
used the NULLABLE item_number and excluded legacy rows the deciding count
included; hydration filtered on COLLECTION visibility, which is
documented as deliberately nav-lenient, so an item-level grant on one row
exposed every sibling's ref and title; and hydration scanned that
nullable item_number into an int, so one pre-006 row erased
relation_targets from the ENTIRE response.
Round 7's "P1" about changed exported signatures was REJECTED, not fixed:
internal/store cannot be imported outside this module, so the compiler
had already proved every caller updated — which is why a required
parameter is right rather than a variadic a caller can silently omit.
TESTING: every fix carries a mutation. Three verdicts were thrown away
rather than counted — two mutants did not COMPILE (a build failure
produces FAIL lines identical to a real kill) and one mutation never
applied. In one sub-task three successive tests passed for reasons that
had not been established: a field that never arrived, a fixture whose
encoder filtered the key before the wire, and a schema reshape that
silently failed. One mutation is recorded UNCOVERED rather than dropped:
the canSee==nil count-before-nil-check is a race with no seam to force,
so that fix is argued from the code path and not demonstrated.
Filed, not fixed: BUG-3012 (ref-path TOCTOU), BUG-3013 (U2 chip wording),
BUG-3014 (bundle-import title renders as dangling), BUG-3015 (the
relationsCarry posture drops silently, medium).
Gates on
|
||
|
xarmian
|
524469b7a3 |
test: no test request goes through the process-global default transport (BUG-3008, closes BUG-2949) (#1324)
Closes BUG-3008. Closes BUG-2949, which filed this failure on 09-07 with the
mechanism explicitly undiagnosed and stopped in the right place: it refuted the
obvious candidate, noted a repo-wide grep found no other caller, and listed
three hypotheses. This is hypothesis (1), and the caller is the standard
library rather than this repo.
httptest.Server.Close() calls CloseIdleConnections() on the PROCESS-WIDE
http.DefaultTransport — deliberately, and the standard library says so in its
own comment, calling it "not part of httptest.Server's correctness". So in a
package where many tests each stand up a server, every `defer ts.Close()`
mutates state every other test's requests depend on. CI observed a request
failing in one test while the server that closed belonged to another:
client_stream_identity_test.go:135: request not sendable:
Get "http://127.0.0.1:45521/api/v1/events/stream?armed=true":
net/http: HTTP/1.x transport connection broken: http: CloseIdleConnections called
SCOPE is every test request in internal/server, not just the parallel ones.
Three revisions were needed to get the class right, and each earlier boundary
was wrong in a way worth recording:
- Direct call sites in test bodies missed SHARED HELPERS. connectSSE,
apiRequest and readRawSSEFramesAuthed sent through the default transport
and have parallel callers; connectSSE holds its request open for a whole
test. A helper cannot know which of its callers is exposed, so it no
longer tries to.
- "Serial tests are safe" is true but is the wrong boundary. Serial-ness is
one t.Parallel() away, and CONVE-2086 tells the next author to add exactly
that line.
- A nil Transport IS http.DefaultTransport. listen_serve_test.go's health
probe was exposed identically and appears in no grep for DefaultClient.
internal/cli is deliberately different: its tests drive the PRODUCTION client
from NewClientFromURL, which has a nil Transport by design, so those sends
cannot be isolated without changing the shipped CLI. There the boundary really
is parallel-vs-serial, and the two parallel tests that build their own requests
use srv.Client(). The asymmetry is written into isolatedTestClient's doc
comment rather than left to be inferred.
WHAT IS NOT CLAIMED: Transport.CloseIdleConnections documents that it "does not
interrupt any connections currently in use", and the idle pool is mutated under
idleMu. The interleaving by which errCloseIdleConns reached a caller's Do() is
NOT reconstructed, and the comment says so instead of inventing one. That
unknown is the argument for isolation over a narrower repair: a client with its
own transport is out of reach whichever window it was.
CONVE-2086 amended with the rule, so the convention that creates the exposure
also carries it.
14 files, all _test.go. Codex: four rounds, CLEAN on the fourth; rounds 1-3
each found a real defect in the class boundary or in the comment's claims about
net/http. Gates on
|
||
|
xarmian
|
4a4a912fa6 |
docs: pad token create needs a session, and one code comment said otherwise (TASK-2984) (#1316)
Since #1267 the server refuses a mint authenticated by an API token with HTTP 403 `session_required` — "Creating or rotating API tokens requires an interactive session, not an API token" — because the tokens such a mint produces outlive the revocation of the token that made them: each has its own name and expiry, and nothing in `pad token list` records which token minted which. `list` and `revoke` stay reachable by a PAT deliberately, since revocation is the response to a compromised credential and should not need a fresh login. Verified against `handlers_tokens.go::requireInteractiveSession` before writing it, as the item asked: the code and the message are quoted from there, and the guard is `isAPITokenAuth`, which is false for a session cookie AND for a saved `padsess_` CLI bearer — a CLI session IS an interactive session. Three artifacts a reader consumes carried the now-false implication; two are fixed here. - The README's PAD_TOKEN section said the override authenticates "without `pad auth login`" and named `pad token create` as where tokens come from, which together read as "you can mint under the override". It now names the one exception and links to the paragraph. - The token-management section gets that paragraph: which subcommands need a session, the exact code and message, why, and that `pad auth login -i` is the headless path. - `internal/cli/client_tokens.go` claimed the PAD_TOKEN override was "usable end-to-end without a browser". True when written and false after the gate. Corrected in place with the reason and a pointer to the guard, rather than deleted — a comment that was true once is worth more as a dated correction than as a gap. THREE CLAIMS I HAD WRONG, all caught before merge and all by checking rather than by rereading: - "needs a terminal but not a browser" — `pad auth login` DEFAULTS to browser-based auth; `-i` is the email/password prompt. The README and the comment name the flag now. - "fails when `PAD_TOKEN` is set" — too broad. `PAD_TOKEN` accepts a `padsess_` session token as well as a `pad_` API token (`env_token.go:21-23`), and the session form mints normally. The gate is on the CREDENTIAL KIND, not on the variable. - The history was compressed into "#879, before #1267". #879 added the override and left minting web-only; #1237 ( |
||
|
David Barkhausen
|
f262449b18 |
feat(cli): pad token create/list/revoke — CLI mint path for API tokens (#1237)
Contributed by b4rk13 (#879 follow-up). Reviewed under DECIS-212-style read: sits on the existing user-scoped /auth/tokens endpoints, no server changes; create requires a login session per #1267 and answers 403 session_required under PAD_TOKEN, list/revoke stay PAT-reachable. Claude-Session: https://claude.ai/code/session_01W71Y4K5hGbbqqAhbVFnjB4 |
||
|
xarmian
|
4618876e3e |
fix(cli): pad server stop signals only a process it can prove is ours (BUG-2969) (#1299)
fix(cli): `pad server stop` signals only a process it can prove is ours (BUG-2969) Measured on the merged binary before this change: a `sleep 600` whose pid had been written into the PID file was SIGTERMed, and stop printed "Server stopped." No pad server was running anywhere near that config. Three things had to be true at once for that. os.FindProcess succeeds for ANY pid on Unix. Nothing asked whether the pid belonged to a pad server. And the confirmation loop polled the PORT — which is unhealthy from the first poll when nothing was ever serving, so the success check was satisfied by the failure case. Liveness is the wrong question, and this is the trap the obvious fix falls into: the stranger WAS alive. The question is whether the pid is OUR server. ## The discriminator Unix takes an advisory flock on the PID file, held for the server's lifetime. `stop` probes it non-blockingly: acquiring it proves nobody holds the file, so the record is stale whatever the pid now names; failing to acquire proves a live pad server holds THIS file. One implementation for Linux and macOS, no new dependency, and the same primitive session_lock_unix.go has used since TASK-2767. Windows has no flock in that pattern, so it compares the process creation time from GetProcessTimes against the one recorded at start — the attribute that survives pid reuse, since a reused pid belongs to a process that started later. The lead first ruled start-time comparison on every platform; I objected with the cost (three implementations — /proc, a macOS sysctl promoting x/sys to a direct dependency, and GetProcessTimes) and the ruling changed to this hybrid. The cost table is on the item so the next reader sees why the shape moved. The PID file gains a fingerprint on both platforms — pid, start time, executable path — as JSON, with the legacy bare-integer form still parsed. A legacy record carries no proof, which reads as UNPROVABLE, and unprovable means nothing is signalled. ## Three races, each found by codex and each the same shape 1. Reading the record and checking ownership were separate steps, so a successor could claim the file between them: the lock then reported "held" — truthfully, about the successor — while the pid handed back was the predecessor's. pidFileOwner now returns the record it read from the descriptor it probed. 2. Removing the PID file after a successful stop could delete a fast successor's live record. It no longer removes at all there: the server removes its own on the way down, and a file left by a crash is handled by the next stop. 3. Removing a STALE file after the probe released the lock had the same window. The removal now happens inside the ownership check, while the lock is held — the only moment at which no replacement can have claimed the path. A claim arriving during that instant retries for half a second rather than losing its claim for the life of the process. Windows deliberately does NOT delete a stale file: with no atomic primitive, a check-then-remove would race a successor, and a stale file that the next start overwrites is recoverable where a wrongly deleted record is not. ## Verified Negative control, and it is the literal one: with the ownership check bypassed, `go test` reports `signal: terminated` — the test binary is SIGTERMed by the code under test, because the stale record names the test process itself. Live, in throwaway HOMEs: a stale record naming a live `sleep` is refused and the sleep survives (it was killed before this change); a stale record with a HEALTHY port answering is still refused, nothing signalled, and both the stranger and the real server survive; a server stopped through its own held record stops, and its file is gone. The CI smoke on windows-latest now stops the server with `pad server stop` instead of Stop-Process, because that is the only place the Windows ownership check runs — a smoke that killed the process directly would leave the GetProcessTimes path unexercised on every platform. make lint, make test green; codex CLEAN in round 4. Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR |
||
|
xarmian
|
5ec17a7c92 |
fix(cli): pad server stop stops the server that is running, or says it is (BUG-2965) (#1298)
fix(cli): `pad server stop` stops the server that is running, or says it is (BUG-2965)
`StopServer` read the PID file and, on any read error, answered "server not
running (no PID file)" — without asking whether anything was listening. The file
was written in exactly one place, EnsureServer's auto-start branch, so a server
started any other way held the port with no file to find: a service unit, a
human running `pad server start`, the seats' refresh recipe relaunching with the
killed process's argv. A stop command that says "not running" about a running
process leaves the caller believing they stopped something, and the next thing
they do rests on that belief.
Two halves, per the item's property and corollary:
- A missing PID file now asks the port. Only an unhealthy address earns "not
running"; a healthy one earns a message naming the address, the missing
file, and what to do instead. Deliberately NOT "find the listener and kill
it" — resolving a pid from a port is platform-specific, and the process
holding it may not be ours. A stop that kills by port can kill a stranger.
- `pad server start` claims the PID file itself, so the file exists for every
start path rather than only the auto-started one.
The second half took four codex rounds to get right, and each round found the
previous shape reintroducing the defect it was fixing:
1. Write-then-defer-remove let a duplicate start overwrite a running server's
entry and then delete it on the way out, leaving a healthy server
unaddressable.
2. Refusing to replace a live pid fixed that and opened its mirror: the start
that LOST the port could still own the file, so the winner was unaddressable.
The fix is ordering, not arbitration — BIND FIRST, then claim, so the file
always names the process that owns the address. internal/server grows
Listen and Serve for that; ListenAndServe is now the two together.
3. With the bind first, EnsureServer's parent-side write became the stale
mechanism (it records a child that may never bind) and the live-pid refusal
became actively wrong (no live process can be serving an address we just
bound). Both removed, along with processIsAlive, whose only remaining
callers were its own tests.
4. Cleanup is a read-then-remove, so running it AFTER the listener closes let
a successor bind and claim between the two steps and lose its file to us.
It now runs before the listener closes, while nothing else can legitimately
own the file. The cost is a drain-window where a healthy server has no PID
file and `stop` says so — a true message in place of a silent wrong one.
Verified live against the built binary, in a throwaway HOME, in both shapes:
start writes the file naming the serving process; a second start against the
held port fails at bind and leaves the first server's file intact; stop then
stops it and removes the file; a further stop reports "not running". The first
live run also caught a flaw in my own method — `stop` reads the config's port,
so the probe answered about 127.0.0.1:7777 (this box's dev server) until it was
re-run with PAD_PORT set. Re-checked after the restructure.
Mutants: the health check removed, the health branch still answering "not
running", an empty PID file, a cleanup that does not remove, and a cleanup that
removes a successor's file are each killed by a named test. The call site itself
is wiring a unit test cannot vouch for (CONVE-19) — that is what the live runs
cover, and the Listen/Serve split is pinned in internal/server.
make lint, make test green.
Claude-Session: https://claude.ai/code/session_01HeChkgZVYb3NTgTcckF5KR
|
||
|
xarmian
|
7659ad3cd3 |
feat(server,web,cli): say when a relation's copy target is unusable, instead of offering a picker that cannot answer (IDEA-2899) (#1262)
* feat(server): the copy preflight says when a relation's target is not usable (IDEA-2899) TASK-2869 made a `needs_value` relation row collectable as soon as it names a target collection. Naming one is not having one: the slug can name a collection that has been DELETED, or one this caller cannot READ. The dialog then mounts a picker that can return nothing and, because the row is not blocked, Confirm stays disabled carrying only the generic required-field message — the user is told a value is missing and never told that no value is reachable. `collection_unavailable` on the needs_value row is the server saying so. THE CLIENT CANNOT COMPUTE THIS, which is why it belongs here. The dialog's destination collection list is filtered through `canEditCollection`, because it drives the copy-INTO picker; a relation TARGET needs only READ access, so a perfectly usable target routinely does not appear in that list. Testing against it would refuse rows the user could have filled in — over-blocking, which is the worse failure and invisible to whoever hits it. `visibleCollectionIDs` is the read-scoped view, and its NAV-LENIENT shape is right here rather than merely tolerable: it includes a collection reachable only through an item-level grant, and the question is "could a picker here return anything at all". One granted item is a picker with one row. DELETED and UNREADABLE are deliberately not distinguished. Same consequence, no client branch would differ — and separating them would tell a caller who cannot read a collection that it nonetheless exists. `omitempty` on a BOOL drops `false`, so the field is phrased NEGATIVELY. Present-and-true means the server checked and the target is unusable; ABSENT means available, or a server that does not report. A client must block only on an explicit true, so absence stays "no information" rather than becoming a value — the rule `access_epoch` follows on the item doors, and the one whose violation cost two review rounds on IDEA-2898 this morning. Costs nothing on the common path: a destination schema declaring no relation field runs no query at all. Claude-Session: https://claude.ai/code/session_01Xk9M5UVPdc84xL5E1mZkm8 * test(server): pin the type gate on collection_unavailable (IDEA-2899) Found by a surviving mutant rather than by inspection: dropping the `def.Type == "relation"` gate left every other test in the file green. Nothing stops a schema declaring `collection` on a field of another type — the validator does not police keys it has no use for — and such a field would then pick up a flag whose meaning is defined only for relations. The dialog would block a perfectly collectable `select` because some relation elsewhere in the same schema points at a collection that happens to be gone. The fixture is the discriminating one: ONE deleted collection, TWO required rows that name it, and only one of them means anything by it. Six mutants on this half, all killed: flag never set, flag always set, deleted target not flagged, unreadable target not flagged, type gate dropped, and the nil-visible-set case (an admin's "no filtering" read as "nothing visible", which would flag every target for the callers who can see everything). Claude-Session: https://claude.ai/code/session_01Xk9M5UVPdc84xL5E1mZkm8 * feat(web): block a relation whose target is unavailable, and stop advising a command that cannot work (IDEA-2899) The client half. `isCollectable` now refuses a relation row the server has flagged, so the row lands in `blockedFields`, Confirm is disabled with a reason, and no picker mounts that could only come back empty. `collection_unavailable !== true` is STRICT on purpose. The field is absent when the target is fine and absent from a server that predates it, so absence must read as "no information". (Over the domain the type admits — `boolean | undefined` — the truthiness spelling is EQUIVALENT and a mutant swapping it in survives; that is recorded in the source rather than papered over with an off-contract fixture. The strict form is kept because it states the contract where the next edit will read it, and the inverse spelling would block every row against an older server.) THE PART THAT IS NOT WIRING: the existing blocked-field notice said the field "is a required <type> field. This dialog can't collect a value for that type safely" and then printed `pad item copy … --field key=value`. Both halves are FALSE here. The type is perfectly collectable; the TARGET is gone. And the CLI runs as the same user against the same referent validation, so the command it prints is refused for exactly the reason the user is already stuck — advice that sends someone to do work that cannot succeed is worse than no advice. So the message branches on `uncollectableReason`, names the collection and the destination workspace, and the CLI line is now gated on `cliFillableField` — the first blocked row the CLI can ACTUALLY fill. `blockedFields[0]` was correct while every blocked row was type-shaped; with an unavailable relation sorted first it named the one field `--field` cannot set either. Eleven unit tests on `copyNeedsValue`, plus a source pin on the dialog whose own measured limit is in its docblock. Client mutants: 7 real, 6 killed, 1 recorded as equivalent with the domain argument that makes it equivalent. Claude-Session: https://claude.ai/code/session_01Xk9M5UVPdc84xL5E1mZkm8 * feat(cli): the copy preview marks an unavailable relation target and stops suggesting it (IDEA-2899) Caught by `TestItemCopyMirrorsMatchServerShapes`, not by me. The CLI keeps a mirror of the preflight response, and adding a field server-side without mirroring it fails that test by design — a mirror that silently lags is a mirror that lies. Working exactly as intended, and the reason this half exists at all. Mirroring the field turned out to be the smaller part. The CLI already prints `target collection: people` for a relation row, and it builds an `Add: --field owner_ref=<value>` suggestion from every unsupplied row. Both are wrong when the target is unavailable: the first sends a user looking for a ref in a collection they cannot read, and the second hands them a command the referent validation refuses for exactly the reason they are already stuck. So the target line is marked NOT AVAILABLE, and the row is excluded from the suggestion with a sentence saying why — modelled on the empty-key branch, which was written for the identical reason (a `--field =<value>` nobody can run) and is three lines away. That the same defect had to be fixed in two places is the shape worth naming: the dialog and the CLI independently built "here is how to supply it" from "here is a field needing a value", and neither had a notion of a field that CANNOT be supplied. The empty-key case was the first instance and was fixed locally; this is the second. Five mutants on this half, all killed: suppression removed, suppression applied to everything, the unavailable label dropped, the explanation dropped, and the mirror field ignored. The available-target control leg is a separate test so the omitempty contract is exercised on this surface too. Claude-Session: https://claude.ai/code/session_01Xk9M5UVPdc84xL5E1mZkm8 * fix(cli): route all three "how to supply it" sites through one predicate (IDEA-2899) Review found the fix applied at one door and not its siblings — my own recurring shape, arriving again. THREE places tell a CLI user how to resolve an unsatisfied field: the detailed `renderItemCopyNeedsValue`, the `--dry-run` summary, and the error the command returns. The first commit fixed the render. The other two went on printing `--field key=value` at someone for whom no value exists — and the ERROR is the line a script or a hurried reader actually sees, so it was the worst of the three to leave. `itemCopyUnfillable` is now the single definition all three consult. Not because three call sites are tidier than one, but because three sites independently answering "how do I supply this" is exactly how they diverged in the first place. The dry-run summary branches three ways rather than two, because the MIXED case is the one a boolean gets wrong: some fields can be supplied and some cannot, and collapsing that either suppresses advice the user needs or offers advice they cannot use. The error hint is suppressed only when NO field can be supplied — with one fillable field left, `--field key=value` is still true. Also pins the BOUNDARY the same review probed: a target collection that is live and readable but EMPTY is deliberately not flagged. The symptom looks identical — an empty picker — but the cases differ where it matters. An unavailable target is unfixable from inside the dialog, so blocking costs the user nothing they had; an empty collection is resolved by creating the item and retrying, and blocking would refuse a copy they were about to complete. It would also cost a live-visible-item count per relation target on a dry run the UI calls on every keystroke. The weaker case — an empty picker that says nothing about WHY — is filed as IDEA-2905 and belongs to the picker. Ten mutants across this round, all killed, including both directions on the error hint and both directions on the dry-run branch. Claude-Session: https://claude.ai/code/session_01Xk9M5UVPdc84xL5E1mZkm8 * fix: unfillable means EITHER reason, and a select never names a relation target (IDEA-2899) Review round 2, two findings, both real and both about a rule stated in one place and enforced in another. **"Unfillable" answered for one of two reasons.** An EMPTY KEY cannot be supplied either — `--field =value` is rejected by this command's own parser, and the detailed render has explained that since Codex round 6. Only that render knew: the --dry-run summary and the returned error went on advising `--field` for those rows, because the predicate I extracted last commit covered the relation reason alone. A predicate named "unfillable" that answers for half its name is a worse trap than no predicate — right at the site that defined it, wrong everywhere it was reused, which is precisely what extracting it was meant to prevent. Two functions now: `itemCopyUnfillable` (either reason — advice), and `itemCopyUnavailableTarget` (the relation half — the render's own sentence, since the two explanations are not interchangeable to a reader). `itemCopyUnavailableTarget` deliberately does NOT also exclude empty keys, though my first version did. A row can carry both faults, and a mutant removing that exclusion survived every test — correctly, because all it changes is printing two sentences that are both TRUE about such a row. The guard was tidiness dressed as a rule; a condition nothing can distinguish is one the next reader has to re-derive. **`Collection` was emitted for non-relation fields**, while its own doc said it is empty for every other type. That was a claim about the schemas people write, not a property of the code: a `select` carrying `"collection": "people"` is storable — field validation has no use for the key and does not police it — and the value was copied straight through, so the CLI printed "target collection: people" beneath a select. A relation fact asserted about a field that has none. `relationTargetSlug` makes the documented contract true at the only place that can make it true; my own type-gate test had created exactly that shape and asserted only the FLAG, not the slug. Three mutants on these fixes, all killed. Claude-Session: https://claude.ai/code/session_01Xk9M5UVPdc84xL5E1mZkm8 * fix(cli): the explanation now names the reason that actually applies (IDEA-2899) Review round 3, and the sharpest miss of this unit — my own, one commit old. Broadening what a predicate ACTS on silently broadened what a sentence SAYS. Once `itemCopyUnfillable` counted empty keys as well as unavailable relation targets, a set of empty-key rows selected the all-unfillable branch and was explained as "the relation target is not available to you" — a false statement about rows that contain no relation at all. Same in the returned error, which is the line a script sees. The tell was there to be read: a sentence that was TRUE while the predicate was narrower is a sentence to re-read the moment it widens. I broadened the predicate deliberately, wrote a commit message about how a half-answering predicate is a trap, and left the sentence describing the half. `itemCopyUnfillableWhy` names the reasons actually present — relation targets, empty keys, or both — and the two one-sentence sites consult it. The detailed render is unchanged: it explains each reason where the row is printed, which is why it uses the narrower count. Four mutants, all killed, including the two that matter: the explanation always saying "relation" (the defect) and never saying it (the same defect pointing the other way). The test carries a mixed-reason leg, because a sentence that picks one of two true reasons is the failure a single-reason fixture cannot see. Also corrected: three comments claiming `itemCopyUnfillable` is relation-only or that the detailed render consults it. Both stopped being true last commit. Claude-Session: https://claude.ai/code/session_01Xk9M5UVPdc84xL5E1mZkm8 * fix: one row can carry both faults, and four docs said this was simpler than it is (IDEA-2899) Review round 4. Four findings, no P1s, and the first is the one worth the round. **A `continue` between the two counts.** `itemCopyUnfillableWhy` counted a row as an unavailable relation target and then skipped the empty-key check, so ONE row carrying both faults reported only the first. My mixed-case test used TWO rows with one fault each — a different input, and the only one it exercised. Two rows with one fault each and one row with two are not the same fixture, and I built the weaker one while writing a commit message about fixtures that cannot discriminate. **The dialog could still print `--field =value`.** `cliFillableField` excluded unavailable relation targets and not empty keys, so a required `json` field the destination reported with no key was type-shaped, blocked, and still offered a command the CLI's own parser rejects. The CLI has refused those since Codex round 6; the web side had never learned it. Same defect, other surface — which is the third time this unit has fixed one door and not its sibling. **Cardinality.** "no --field can supply it" for several fields, and "reported them with an empty key" for one. Both sites now agree with their counts, and the empty-key phrase is neutral on number so it reads correctly after either. **Four documents claimed every needs_value row is resolvable with an override** — the CLI renderer's docblock, the server's `NeedsValue` field, the CLI mirror type, and the dialog's collectability comment. That was true when each was written and this unit falsified all four; a reader following any of them would conclude the CLI had simply forgotten to print a flag. Two mutants on the fixes, both killed: the `continue` restored, and the dialog's empty-key exclusion removed. Claude-Session: https://claude.ai/code/session_01Xk9M5UVPdc84xL5E1mZkm8 * refactor(cli): one tally, because the rounds said the branching was the problem (IDEA-2899) Four review rounds returned 2, 2, 2 and 4 findings. The counts looked like slow convergence; the DISTRIBUTION was the finding. Every defect after round 1 lived in this one layer — how the CLI and the dialog say "here is how to supply it" — while the server half that computes availability stayed clean throughout. The layer had accreted exactly the way IDEA-2898's cold path did: a count, then a second count for the other reason, then a phrase function, then a `continue` between two counters that made a dual-fault row report half of itself. Round 4 fixed something round 3 introduced to fix something round 2 introduced. That is not a run of bad luck, it is a shape. So this round removes branches instead of adding a seventh guard. `itemCopyTally` walks the rows once and returns what every caller needs; `AllUnfillable()` is the condition both one-sentence sites test, and `Why()` is the phrase both interpolate. Three helpers become one type. There is no second definition of "unfillable" to drift from the first, and no sentence describing a subset of what a predicate counts, because the sentence and the count come from the same walk. `Unfillable` is deliberately NOT `UnavailableTarget + EmptyKey`: one row can carry both, and double-counting makes `Unfillable == Total` false for a set that is entirely unfillable — the comparison every caller makes. A mutant does the addition and dies. Five mutants, all killed. The last needed a new test rather than a new fixture: `AllUnfillable`'s `Total > 0` guard is unreachable from both current callers, so a mutant removing it survived every command-level test. Keeping an unreachable guard and calling it defence is how a promise becomes a lie, so the tally is now unit-tested directly — an empty set is not "entirely unfillable", and a future caller outside the `len() > 0` gate would otherwise be told silently that nothing can be supplied. Claude-Session: https://claude.ai/code/session_01Xk9M5UVPdc84xL5E1mZkm8 |
||
|
xarmian
|
07b2e439f2 |
TASK-2869 (U2b): the preflight names a relation's target collection, and the copy dialog scopes its picker to the destination (#1258)
* feat: the preflight's needs-value row names its relation target, and the copy dialog scopes its picker to the destination (TASK-2869)
U2b, per the day-55 ruling. Both blockers (U1 referent validation, U2 the
FieldEditor branch + picker) are in.
THE DEFECT. `ItemCopyPreflightNeedsValue` carried `type: "relation"` and no
target. FieldEditor gates its relation branch on `wsSlug` AND
`field.collection`, so a required relation in the destination reached the copy
dialog as a field it knew was a relation with no idea what to point at, and
rendered as FREE TEXT. Before U1 the copy stored whatever was typed.
SERVER. `collection` is added to the needs-value row, populated from the
DESTINATION schema's `def.Collection` — the only place it is known, since the
row is built from that schema. Additive and `omitempty`: a client that does not
read it is unaffected, and a row for a non-relation field is byte-identical to
before. Not a wire-version question, for the same reason
`models.ItemWriteWarnings` was not.
CLIENT. `toFieldDef` carries the collection through, and the FieldEditor call
passes `wsSlug={destWs}` — the DESTINATION, never the source. A relation
resolves at the destination, so the picker must list items the copy can
actually point at; that is same-workspace resolution AT the destination, not
the cross-workspace case PLAN-2857 rules out.
`relation` becomes collectable ONLY IF THE ROW NAMES ITS TARGET. Without a
collection, FieldEditor's gate renders the non-editable state, so offering the
row would produce a control that cannot be filled and a Confirm that cannot be
satisfied. Such a row now lands in the blocked list and the user is told which
field and why — the same disposition `multi_select` gets, for the same reason:
a control that silently cannot do its job is worse than an honest refusal.
TWO THINGS THIS UNIT TAUGHT ME THAT ARE NOT IN THE RULING.
1. THE WEB MUTANT SURVIVED, AND THAT IS WHY `isCollectable` MOVED. My first
version left the predicate inline in `CopyItemDialog.svelte`. Making
`relation` unconditionally collectable — the exact defect the negative leg
of the proving test is about — passed EVERY suite in the repo. That is
IDEA-2894's lesson arriving one unit later in the same file, so the
predicate now lives in `$lib/items/copyNeedsValue` with tests. Two mutants
die there: relation-always-collectable, and `multi_select` slipped into the
collectable set.
The Go half was pinned from the start (drop `Collection: def.Collection` ->
FAIL naming the empty value and the expected slug). Only the client half was
unpinned, and only because of where the code lived.
2. A U2-ERA TEST ASSERTED THE ABSENCE THIS UNIT CLOSES, and asserted it
CORRECTLY. `fieldEditorRelationCallers.test.ts` required that the dialog
pass no `wsSlug` and build its FieldDef from a shape with no `collection` —
which was the behaviour, and withholding `wsSlug` was what kept an unscoped
picker out. It also named this task by ref and told its successor to revisit
the gate WITH the change rather than let it drift. Inverted here: the block
now asserts the destination slug is passed, that the collection reaches the
FieldDef, and — the half that is easy to lose — that the dialog still
DELEGATES the collectability decision, so a future inlined predicate would
pass the unit tests and fail this.
Worth keeping: a test that pins a temporary absence should name what would
make it wrong. This one did, and that is the only reason its inversion was a
five-minute job instead of an argument about whether it was load-bearing.
Gates: `internal/server` ok 158.335s, `go vet` and `gofmt` clean,
`npm run check` 0 errors (6 pre-existing warnings), `make web-test` 127 files /
2123 tests. Postgres and CI are owed on this tip.
Claude-Session: https://claude.ai/code/session_01Xk9M5UVPdc84xL5E1mZkm8
* fix(cli): mirror the needs-value collection, and name the relation target in the CLI (TASK-2869)
TWO CONSUMERS I DID NOT SWEEP. The previous commit added `collection` to the
preflight's needs-value row and updated the server struct and the TypeScript
type — "both sides", as its own message put it. There are THREE sides.
`internal/cli` keeps a mirror of the preflight shape and
`TestItemCopyMirrorsMatchServerShapes` requires it to match the server field
for field. It failed in the Postgres gate, in a package the change did not
touch.
That is the third time this session a producer change broke a consumer I had
not enumerated, and the shape is always the same: I name the surfaces I edited
and call that the population. The instrument that would have caught it is not
"run more tests" but "grep for the type's name before claiming the sweep is
done" — `ItemCopyPreflightNeedsValue` appears in exactly three files and I
looked at two.
Mirrored, with a comment saying why it exists: a mirror that silently lags is a
mirror that lies, and the CLI renders these rows.
AND THE CLI NOW NAMES THE TARGET COLLECTION, which is the point of the unit on
the surface that has no picker at all. A row reading
owner_ref (Owner, relation) required — required, with no value…
tells a user a value is needed and nothing about what kind of value exists.
The dialog answers that with a scoped picker; the CLI had no answer. It now
prints the relation analogue of the `options:` line a select already gets:
owner_ref (Owner, relation) required — …
target collection: people
Test asserts the line appears for the relation row, appears EXACTLY ONCE with a
select row rendered alongside — so it cannot pass by printing unconditionally —
and that the select's own `options:` line still renders, so this did not
displace it.
Gates: `internal/cli` and `cmd/pad` green, build and gofmt clean. The full
Postgres run and CI are owed on this tip; the earlier PG run is the one that
caught the mirror and is superseded.
Claude-Session: https://claude.ai/code/session_01Xk9M5UVPdc84xL5E1mZkm8
* fix(web): the relation picker searches the preflight's canonical destination slug (TASK-2869)
Codex review, finding 3 of 3, and the only one of the three that belongs in
this unit.
`wsSlug={destWs}` handed `FieldEditor` a value that is NOT always a workspace
slug. An item can be opened through a workspace-UUID URL, the route parameter
is passed straight through as `sourceWsSlug`, and a same-workspace copy then
puts that UUID in `destWs`. `/search` resolves a workspace by SLUG only, so a
picker handed a UUID searches nothing and returns no results — a control that
looks usable, is not, and says nothing about why.
Now `pickerWsSlug`, which is the preflight response's own
`destination.workspace_slug`. The preflight IS the canonicalising round-trip:
the server resolved whatever it was given and answered with the real slug.
Falls back to `destWs` only before the first preflight returns, at which point
no needs-value row is rendered anyway.
The caller test asserts the prop AND the derivation, because asserting only the
prop would pass against a `pickerWsSlug` that was just `destWs` renamed.
THE OTHER TWO FINDINGS ARE REAL AND ARE FILED, NOT FIXED HERE.
IDEA-2898 — `ItemPicker` serves warm local-index results without re-authorising
them, so a collection whose access was revoked can still be listed. The cold
`/search` path is visibility-filtered and correct; the warm path is not. This
is PRE-EXISTING and applies to every caller of the picker, `ItemDetail`
included — last touched by TASK-2877, not by this unit. U2b widened the
exposure by adding a caller; it did not create the defect, and rewriting the
picker's cache-authorisation model inside a feature branch would be an
unrelated change riding along. The fix needs a decision about where the client
learns its access set from, which no current signal provides.
IDEA-2899 — a relation row can name a target collection that is DELETED or
UNREADABLE, so `isCollectable` says yes on a non-empty string and the user
meets a picker with nothing in it. I could not fix this correctly here, and the
reason is measured rather than assumed: the obvious test is to check the target
against `destCollections`, and that list is filtered by `canEditCollection` —
it is what the user may copy INTO. A relation TARGET needs only READ access, so
a perfectly usable target routinely is not in it. Using it would OVER-BLOCK,
refusing rows the user could have filled, which is a worse failure than the one
being fixed and invisible to whoever hits it. The right shape is probably the
server reporting the target's availability on the row it already builds — an
additive field on the same row this unit just changed, worth doing deliberately
rather than bolted on at the end of a branch.
Gates: `npm run check` 0 errors (6 pre-existing warnings), `make web-test` 127
files / 2123 tests. Postgres is running on this tip; CI is owed.
Claude-Session: https://claude.ai/code/session_01Xk9M5UVPdc84xL5E1mZkm8
|
||
|
xarmian
|
02846a6785 |
fix(cli): the session's registered agent is the name its writes carry (BUG-2882) (#1248)
* fix(cli): the session's registered agent is the name its writes carry (BUG-2882) Two seats booted under one name; one re-registered under the right one with `pad session register --agent`; every write it made afterwards still carried the wrong name. The registry row and $PAD_AGENT were two self-declarations of "the same value" — the row could be rewritten, the environment could not, and nothing reconciled them or warned. Night 10 read both live seats inverted against each other. ResolveAgentName now consults the registry record for the session that owns this process FIRST: a stat-and-read of one file, no MkdirAll, no lock, ignored when malformed, legacy, or carrying a different process-start token (pid reuse). A non-empty registered name wins over .pad.toml and $PAD_AGENT; an anonymous row leaves an environment name in force. `pad session register` without --agent keeps the current name, as before, because its default is the resolver. Help text, the record's doc comment and README's precedence list say what is now true. Test: register as rook under PAD_AGENT=wren and a .pad.toml name → rook; default re-register → rook; anonymous → the .pad.toml name; a record for this pid with another process's start token → ignored. The registry-step mutant fails the first two assertions. Fixes BUG-2882 * fix(cli): a registry record names this session only when it is verifiably this session's, and the identity tests stop reading the real registry Codex round 1 on #1248. (1) registeredAgentForThisSession compared process-start tokens only when both sides had one, so a stale row with no token under a reused pid — a dead session's — would have named a live one. Fail closed: when this process can read a token, the record must carry the same one; and the record must pass the same OwnerLiveness verdict `pad session list` applies. The token-less case is now a test row, with a positive control after it. (2) The pre-existing resolver and header tests cleared PAD_AGENT/CLAUDECODE but not HOME or the session-pid variables, so run inside a registered seat they would have read that seat's row as the resolver's first answer. They now run from a scratch HOME with no session identity. Refs BUG-2882 * fix(cli): a registry row names this session only if its owner is this process or an ancestor, where that can be checked Codex round 2 on #1248. (1) TestPushItemSendsResolvedAgentHeader was the one identity test round 1's hermeticity fix missed; it now isolates HOME and the session env like its neighbours. (2) The registry step accepted a row whose owner pid was alive and token-matched but NOT this process or an ancestor — a misconfigured CLAUDE_PID pointing at a sibling session could borrow that session's name. Refused where the platform can walk ancestry. Not gated on PIDVerified: CaptureSessionOwner records "cannot check" and "checked and wrong" as the same false, and the flag alone would have disabled the step on every non-Linux platform. The check is re-run and only a checked-and-wrong answer refuses. Test uses a live non-ancestor child as the claimed owner; the refusal-dropped mutant lets it name us. Refs BUG-2882 * docs(cli): session list help — a row's name outranks PAD_AGENT; change it with --agent, not by re-registering (BUG-2882, codex round 3) |
||
|
xarmian
|
b437cc582d |
feat: item reminders — the fire-at-an-instant primitive, and one overdue rule for all four surfaces (IDEA-2641, closes #1010) (#1244)
* feat(store): item reminders — the fire-at-an-instant primitive (IDEA-2641) Adds the storage, the scheduler tick, and the canonical event for one-shot item reminders (GitHub #1010). Nothing in Pad acted at a target time before this: a due_date makes an item show up as overdue once somebody asks the dashboard, so "revisit TASK-X on the 1st" had to live in an external cron. A TABLE, NOT A SCHEMA-FIELD ANNOTATION. The design sketch proposed marking schema date fields with a `reminds: true` key on models.FieldDef; recon overturned it. Such a key does not survive an ordinary collection edit, two independent ways: the web editor destructures each field into an EditableField and rebuilds a fresh definition key-by-key on save, so unknown keys are dropped (`pattern` and `unique_scope` survive only because two lines were hand-added for them), and models.CollectionSchema has fixed fields with no catch-all, so any Go unmarshal+marshal round-trip strips unknown properties — the hazard retargetRelationFieldsTx mutates raw JSON to avoid. Both failures are silent and both disarm a whole collection's reminders at once. It is the same defect class that moved traits out of the schema column in TASK-2657. The table also gives the lifecycle a home. A reminder is armed, then fired, then acknowledged, and a re-arm returns it to armed — per-reminder state a field definition has nowhere to keep. remind_at is an RFC3339 UTC instant, deliberately not a `date` schema value: those admit both YYYY-MM-DD and full RFC3339 and are compared against the SERVER'S LOCAL calendar day. A fire-at time cannot carry that ambiguity. The remaining timezone question for due_date is filed separately. Firing is one transaction per reminder carrying BOTH the fired_at write and the outbox insert. That pairing is the point: a fired_at committed without its event is a reminder that silently notifies nobody and can never be retried, because the row has left the armed set; an event without fired_at fires every tick forever. The UPDATE's own `fired_at IS NULL` predicate is the arbiter, so two instances ticking at once produce exactly one winner. item.reminder_due is admitted to the closed events/1 set as v1.2, with a new PayloadReminder family and no SSE name. The subject is the REMINDER, not the item: two reminders can be armed on one item, so an item-subject event could not say which fired, and the reminder id is what an acknowledgement addresses. A new payload family rather than reusing the item snapshot for the same reason — a snapshot would validate and still not answer the only question the event exists to answer. No SSE name in v1 because the poll surface is the contract; adding one later is additive, removing one is not. Ack is explicit and nothing else acks. An item reaching a terminal status deliberately does NOT ack: that would make every status write a reminder mutation, and it would silently consume a reminder set to fire after the work was done. * feat(server): reminder surfaces, and one shared overdue rule for all four Second half of IDEA-2641: the HTTP surface, the scheduler tick's wiring, and the fix for the finding that justified the unit — `ready` / `next` did no date handling at all. OVERDUE NOW HAS ONE IMPLEMENTATION. It used to live inline in the dashboard's attention loop, which meant `pad project stale` inherited it (it filters that very list) and the recommendation surface never saw it. So a deadline reached the two surfaces that REPORT on work and never the one an agent PULLS from. overdue.go is now the only place that decides, and all four call it. Two behaviour changes fall out, both deliberate: - An overdue item bypasses the orphan branch's high/critical priority gate. That gate was where a deadline quietly stopped: a low-priority item three weeks late was reported by `stale` and never suggested by `next`. - Overdue sorts above in-progress. The list is capped at three, so a rank below in-progress would not merely order the deadline lower — on any workspace with three things in flight it would keep an overdue item off the surface entirely, which is indistinguishable from not shipping this. The server-local-today comparison is UNCHANGED and known to be wrong for multi-timezone deployments; it is filed as its own item with the cloud case stated. Changing what "overdue" means on every existing instance inside a change about where the rule LIVES is the kind of behaviour change nobody reviews. Fired reminders reach `next` / `ready` two ways, from one filtered list: PendingReminders is the addressable form (it carries the id an ack needs), and a prepended suggestion is the rendered form. They are prepended AFTER the cap rather than entered as ranking candidates — a reminder is not a task competing on priority, and whether it appeared should not depend on how busy the workspace is. Terminal-item reminders are FILTERED from the surface, never acked. Acking on terminal status would couple every status write to reminder state and would consume a reminder armed to fire after the work was done. The row stays exactly as the user left it; the distinction is observable, and asserted. Three guard tests caught this change and each was answered rather than silenced: - The request-body reader guard was right: the handlers now go through decodeJSON, inheriting the NUL refusal and the size cap. - The canonical-events guard was right: item.reminder_due is admitted to the duplicated contract table as SPEC-3 v1.7, with the reminder subject kind and the new payload family. SPEC-3's own text owes the same amendment. - The NUL census asked for a decision on eight new columns. None carries caller text: ids and FKs are server-generated, four are the server clock, and remind_at is now re-parsed and re-formatted in the STORE as well as at the edge — so the stored value is always machine-produced from a parsed time and no caller bytes reach the column. The doc comment that used to say "the caller normalizes" protected nothing. Regenerating the baseline also found that GEN_NUL_BASELINE=1, which the test's own instructions name, was never implemented — the flag did nothing, so the documented path was hand-editing the file. Implemented, so the next reader gets the mechanism the instructions promise. * test(reminders): the lifecycle, the four surfaces, and 22 killed mutants Every test here was designed against a specific mutation and the mutation was RUN. A green suite proves nothing about a suite nobody tried to break, and three of the mutants I first wrote were not experiments at all. Store (10 mutants, all killed): candidate predicate <= flipped to >=; the event emission lifted out of the fire transaction; the fire UPDATE's `fired_at IS NULL` arbiter removed; the RowsAffected check ignored; re-arm clearing fired_at but not acked_at; ack losing `fired_at IS NOT NULL`; the poll surface losing `acked_at IS NULL`; normalizeRemindAt no longer refusing; it dropping .UTC(); GetReminder losing its workspace scope. Surfaces (12, all killed): the priority gate no longer bypassing on overdue; the sort no longer ranking overdue first; attention leaving the shared helper; the reason losing its OVERDUE prefix; the comparison flipped to >; terminal items no longer skipped; terminal reminders no longer filtered; the filter ACKING instead of hiding; reminders appended instead of prepended; the tick running on a far-future clock; ack answering 200 for an unfired reminder; parseRemindAt accepting a bare date. THREE MUTANTS DID NOT COUNT ON THE FIRST PASS and were rewritten. Two failed to compile (`if false` orphaned a variable; deleting a parse orphaned an import) and one had an anchor matching two call sites. A non-compiling mutant emits zero FAIL lines and reads exactly like a surviving one — it invents a hole that is not there — so the harness reports BUILD-FAIL and ANCHOR-BAD as outcomes distinct from SURVIVED. It also restores files from an in-memory copy rather than `git checkout`, which would delete uncommitted work in the tree. ONE MUTANT GENUINELY SURVIVED and the test was at fault, not the mutant: appending rather than prepending reminder suggestions was undetectable because the fixture had a single item, so the reminder sat at index 0 either way. The fixture now fills the three-item cap with in-progress work, where an appended reminder lands fourth and vanishes. Faithful mutant, weak test — checked in that order. The same lesson shapes the four-surface fixture: it is a LOW-priority open orphan, because that is the case the old code handled worst. A high-priority task would have made the ready/next leg pass against the unfixed tree, which is a green that measures nothing. Negative controls throughout: a future deadline is not overdue and does not reach the gate bypass; a tick with nothing due fires nothing; a completed item is neither overdue nor suggested. Without them a helper that reported every date, or a tick that fired everything, would satisfy every positive leg. The lead's pin is asserted in both directions: a fired reminder on a done item is ABSENT from the surface and PRESENT and still unacknowledged in the table. Asserting only the absence would pass against an implementation that consumed the row, which is the behaviour the pin exists to forbid. * feat(mcp): pad_item.remind + ack-reminder, ToolSurfaceVersion 0.28 An agent that can RECEIVE a reminder but not set one has half the primitive. The poll surface is pad_project.next / ready, both long exposed, so reminders already reached agents — what was missing is the other half: deferring a piece of work is exactly the moment an agent knows when it wants to be asked again, and it had no way to say so. Two additive actions, two optional params. Nothing existing moved, so a v0.27 consumer enumerating neither is unaffected — the v0.13 / v0.11 / v0.8 disposition, which likewise wired existing CLI verbs onto the catalog. remind_at REFUSES a bare date rather than reading it as midnight. Worth stating because the `date` schema type accepts YYYY-MM-DD and a caller will reasonably try it here: a bare date names a 24-hour span, and choosing an hour inside it would fire at a time nobody picked. Re-arm and disarm stay CLI-only. Both address a reminder by an id the agent would have to list first, and no listing action exists on this surface — a door with no handle. Adding them later is additive. Five guards had to be taught, and each was answered on its merits rather than excluded: the HTTP parity test (route mappers added, so the actions work on the remote transport rather than being advertised and unrouted), the read-only catalog's cmdhelp fixture and expected cmdPath map, the field- conflict classifier (remind_at / reminder_id are NOT field writers — a reminder is a row in its own table addressed by its own id, so listing them as classified sources would have pointed detectFieldConflicts at something that is not a field source), and the instructions.md / README action tables. That machinery is why the version bump is safe to make now, and it earned its keep on this change: every one of the five failed on the first build after the catalog entry landed. CONVE-23 sweep for prose this falsifies: - SPEC-3 (DOC-2653) amended to v1.7 in the room, recording item.reminder_due with its new subject kind and payload family — the first canonical event with no user mutation behind it, since a scheduler tick produces it. - CLAUDE.md gains the reminder routes, the CLI verbs, and the v0.28 entry. It was also stale at 0.26 with NO v0.27 entry at all: the 0.27 unit swept instructions.md and README.md and missed this file. Both added. - skills/pad/SKILL.md gains the verbs and a routing entry, including the two things an agent will get wrong — the time is an instant, so ask for a time of day rather than picking one, and finishing the item does not acknowledge the reminder. * fix(reminders): codex round 1 — four findings, all real, all with a pin Round 1 found four defects and refuted none of them. Each fix carries a test that fails against the code as it was, and each of those was mutation-checked. **P1 — pending reminders bypassed item-level visibility.** Every other dashboard section reads `allItems`, which the store already scoped to the caller's collections AND their granted item ids. The pending-reminder list is a direct workspace-wide query and inherited none of that, so a guest holding a grant on ONE item could read the refs and titles of every other item in the collection through its reminders — an item-level leak wearing a notification's clothes. Now filtered with the same `isItemVisibleToGuest` call the sibling sections use. The test's two items share a COLLECTION on purpose: a collection-level filter was already applied, so separate collections would have made it pass against the unfixed code. **P1 — soft-deleted items could starve the queue permanently.** Candidate selection ignored `deleted_at`, and `fireOneReminder` rolls back when it finds the item gone — which leaves the reminder ARMED and therefore a candidate again on the next pass. Candidates are ordered oldest-first and bounded by a limit, so enough archived reminders fill every batch and no live reminder ever fires. Silent, too: the tick reports zero fired and looks idle. Excluded in the candidate query rather than skipped downstream, so those rows never occupy a slot; the reminders themselves are kept, so restoring an item restores its reminder with it — asserted, because a fix that reaped them would pass the starvation test alone. **P2 — the pass stopped at the first failing reminder.** The per-reminder transaction exists precisely so one unfireable row cannot hold back the rest, and `return fired, err` made that comment false — with candidates oldest-first, one persistently broken old reminder blocks every newer one forever. Now continues and joins the errors, so a pass that fired seven and failed three reports both halves rather than reading as clean. The loop is split behind an injected seam because a real mid-transaction failure is not reachable from outside: the database refuses the corrupt rows that would cause one (verified — invalid JSON in items.fields is rejected by the schema). **P2 — suggestions dropped the reminder id.** The docs tell an agent to acknowledge what it sees in next/ready, and the payload carried no handle: a stateless poller could read the reminder and had no way to retire it, so it would be shown the same item forever. `DashboardSuggestion` now carries `reminder_id` (omitempty), `pad project next` prints the exact ack command, and the test acks with the id the surface handed out rather than merely checking the field is populated — a wrong-but-present id satisfies equality with itself. Four mutants, four killed; one was rewritten first because its anchor matched two call sites and was therefore not an experiment. * fix(reminders): codex round 2 — four findings, all real **`--rearm` was unusable.** `ExactArgs(1)` forced an item ref that the rearm branch then ignored, so the flag could not be reached without supplying a ref that was silently discarded. Now `MaximumNArgs(1)`, with each mode checked explicitly: a ref is required to arm, and a ref supplied ALONGSIDE `--rearm` is refused rather than ignored — it names an item the reminder may not even belong to, and quietly dropping it is how a user learns nothing about the reminder they just moved. **`unremind --format json` emitted plain text**, breaking the parseable-output contract every sibling command honours. **The MCP `ref` param did not list `remind`.** Agents read that flat description to decide what to send, so an action missing from it is an invalid call waiting to happen. It now also says what `ack-reminder` takes instead, and why: a reminder is addressed by its own id because an item can carry several. **Fractional seconds fired early.** `time.Parse` accepts `09:00:00.900Z` and `Format(RFC3339)` drops the fraction, so it was stored as `09:00:00Z` and fired 900ms BEFORE the moment the caller named — silently, having rewritten their value on the way in. Seconds are genuinely the stored resolution (the column is compared as a string against a whole-second clock, and the tick runs every 30s), so the only question was which way to resolve it, and truncation resolved it the wrong way. `NormalizeInstant` now rounds UP: at most a second of lateness, in exchange for a guarantee that can be stated — a reminder never fires before the instant it was set for. Late is a reminder; early is a wrong answer. Whole seconds round-trip exactly, which is asserted, because an implementation that added a second unconditionally would otherwise pass. Three mutants for this round, three killed (round-up→truncate, round-up→unconditional-add, MaximumNArgs→ExactArgs). Thirty across the unit. Two fixes carry no dedicated test and it is worth being explicit rather than implying coverage: the `--format json` branch on `unremind` is a one-line output change with no server-free way to drive it, and the MCP `ref` description is prose the drift tests do not read — they assert an action is DOCUMENTED, not that a param's sentence lists it. * docs(reminders): the ack id is on the surface an agent polls, not only on the arm response CONVE-23 follow-through on the round-1 fix. Both agent-facing docs told a caller to acknowledge a reminder with the id "returned when you armed it" — true, and useless to the caller that matters: a poller reading next/ready never armed anything. The suggestion now carries reminder_id and `pad project next` prints the exact ack command, so the docs say that instead. The prose was written before the fix existed, which is exactly the case CONVE-23 is about: a change that makes an instruction stale without touching the file the instruction lives in. * test(reminders): bind the tick LOOP to the work, not just the pass (CONVE-19) Every other test in this file calls runReminderTick directly. That vouches for the component and says nothing about whether anything ever calls it — a tick that is never started is indistinguishable, from those tests, from one that is. It is the convention's exact case, and the failure I recorded on my own identity doc three times in one unit: I test the component and not the binding. Driven through the injectable tick channel so the assertion pins a SPECIFIC pass instead of racing a 30-second ticker, and polled to a bounded deadline so a loop that never runs FAILS rather than hanging the suite. Mutant: drop `s.runReminderTick()` from the select and this goes red while every direct-call test stays green. Killed. The idempotence leg exists because a second Start spawning a second loop would leave one running after Stop, making the BUG-842 drain invariant false for this sweeper specifically — the one property a copied lifecycle is most likely to get right by accident and least likely to be checked. The cmd/pad call site (cmd_server.go, alongside StartTokenReaper) stays verified by inspection: a source-scanning guard for it would be an instrument asserting facts about source, which is code with an adversary and not worth it for one line that sits in the middle of five identical neighbours. * fix(reminders): codex round 3 — a deferred reminder fired anyway, and the poll surface was unbounded **A re-arm mid-pass did not stop the fire.** The candidate scan selects an id; before the UPDATE runs, a `--rearm` can move that reminder into the future. Re-arm clears `fired_at`, so a predicate checking only `fired_at IS NULL` still matched — the pass fired a reminder the user had just deferred and emitted its event. The re-arm cannot undo that: it can clear the mark, but the event is already on the outbox and at-least-once means a consumer has seen it. The fire UPDATE now revalidates `remind_at <= nowTS` against the SAME nowTS the candidate scan used. Same-value deliberately: the arbiter and the scan must agree about when this pass is, or a reminder could pass one and fail the other for no reason but clock drift inside a single pass. **The poll surface was unbounded.** Every fired-and-unacknowledged reminder was loaded and turned into a suggestion prepended to a list that is otherwise capped at three, so a workspace with five hundred unacknowledged reminders returned five hundred suggestions — in the dashboard response, the hottest read in the product, growing until somebody acknowledged them. Two bounds, because they are two different guarantees: the query takes a window (default 50, oldest-fired first, so it holds what has waited longest), and the prepended suggestions are capped at 5 so `suggested_next` stays a recommendation rather than a second inbox. The full set stays addressable in `pending_reminders`. Truncation is REPORTED as a boolean, not a count. A count would have to be post-visibility-filter to be true for the caller reading it, and the store cannot compute that — the filter runs per item, above. "There are more than you can see here" is the strongest claim the data supports, so it is the one made. Four mutants; two killed outright, two survived and were run down under CONVE-28: - **Uncapped suggestions survived because the fixture had ONE reminder** — capped and uncapped are the same list at n=1. That is the SECOND time a single-item fixture hid a count-or-order property in this file. Fixture now arms eight; it also asserts all eight remain in `pending_reminders`, so the cap is pinned to the recommendation and not to the data. - **Removing the SQL LIMIT survived, correctly, and the test comment now says so.** The Go slice cap bounds the PAYLOAD; the SQL LIMIT bounds the DATABASE'S work. Only the first is observable at this level — with the LIMIT gone the response is still bounded, while the query silently goes back to materialising every pending row before discarding most of them. That is a memory and I/O property with no assertion available here, so it is stated as a coverage boundary rather than papered over with a green that would not have measured it. * docs(reminders): the fire predicate arbitrates against two actors, not one CONVE-23 inside the file the round-3 fix touched. The comment described the UPDATE as an arbiter for concurrent TICKS, which is what it was written for and is why I did not re-read it when asked whether a user edit could race the pass. It now says what it actually defends against, and names the general shape: an arbiter is only an arbiter with respect to the writers it can see. * fix(reminders): codex round 4 — the round-3 bound recreated the round-1 starvation Round 3 bounded the poll surface. Round 4 caught what that bound did: the query took the first N rows and the dashboard then discarded the ones it could not show — hidden items, unauthorised items, completed items — so N such rows hide a visible reminder behind them indefinitely, with no continuation to reach it. That is the SAME defect I had removed from the fire path one round earlier, reintroduced in the read path within the hour. The general form is worth stating because I clearly did not hold it: **a bounded window is only safe when the discarding happens BEFORE the bound.** Filtering above a limit is a starvation every time, and it does not matter what the filter is for. Two halves, because the two filters are not the same kind of thing: **Visibility is now scoped IN SQL**, using the same collection-id / item-id sets every other dashboard section gets through `allItems` — the same three-way shape as ItemListParams, where holding both collection grants and item grants is an OR. Invisible rows no longer occupy the window at all, which is strictly better than filtering them out afterwards and is what the sibling sections have always done. **Terminality is paged**, because SQL cannot evaluate it — a collection's schema defines which statuses are terminal. The collector refills from the next page when a page comes back short, bounded by a max scan so a workspace full of completed items cannot turn a dashboard read into a table scan. The bound is 10x the window: the common shape fills on the first page, and the pathological shape terminates in a fixed number of indexed reads. Stopping at the scan bound reports truncation, which is honest — there may be more, and we did not look. The empty-scope case is a THIRD state that reads like the second: nil CollectionIDs means unrestricted, a non-nil EMPTY slice means this caller sees no collections. Without an explicit guard they collapse, because the switch matches none of its cases at length zero and adds no clause at all — so "nothing visible" would return the whole workspace. Three mutants, one survived: the empty-scope guard, because no dashboard-level test produces that state (callers that would are refused earlier by workspace access). Faithful mutant, missing test — it now has a direct one, with a sanity leg so a build returning nothing cannot pass it by accident. A guard for a state nothing exercises is exactly the one that rots. * fix(reminders): codex round 5 — the MCP action I shipped did not work over stdio **P1: local stdio MCP `remind` was unusable.** cmdhelp derives positionals by regex from a command's `Use` string, and `<instant>` inside `remind <ref> --remind-at <instant>` matched — it became a second REQUIRED positional, so dispatch failed with `missing required argument "instant"`. The action was advertised on a transport where it could not run. **The MCP catalog's own tests did not catch it, and the reason is the finding.** That suite builds its cmdhelp document BY HAND: I wrote `Args: mkArgs("ref")` in it, so the fixture agreed with what I meant rather than with what the CLI says. Five parity and drift tests passed against a document I authored to match my own intention — the "a test that agrees with whatever the table says is not a test of the table" shape, which the canonical-events test warns about in its own comment two packages away. The new test reads the REAL command tree via cmdhelp.Build, which is the only thing in this repo that can disagree with me about what the CLI declares. **P2: `pad project ready` withheld the ack handle** that `next` prints. Showing a fired reminder on the surface an agent polls while withholding the id it needs to retire it means the same entry comes back on every poll, forever. **P2: suggestions asserted a collection they did not have.** The orphan branch admits ANY collection — its own comment claimed it gated on tasks "mirroring the active-plan branch", and that comment was simply false — while the output hardcoded `Collection: "tasks"` and the reason said "Open task". Pre-existing for high-priority items since BUG-1082; my overdue bypass widened it to any overdue item, which is how it surfaced. Fixed by carrying the item's REAL collection rather than by narrowing the branch: narrowing would silently drop the non-task items this has surfaced for a year, and the defect is the mislabelling, not the inclusion. The false comment is replaced with what the code actually does. The first version of that test used an overdue IDEA and SKIPPED — ideas use `new`, and the branch requires `open` or an active status, so it never became a candidate. A test that cannot fire is a failed reconstruction, not a pass; the fixture is now a bug-like collection whose vocabulary contains `open`, which is the population the defect can actually reach. Three mutants, three killed. Forty-one across the unit. * fix(reminders): codex round 6 — reminders fired from soft-deleted workspaces **P1, and the only defect in this unit whose consequence leaves the process.** Workspace soft-delete deliberately keeps items for the 30-day restore window, so the candidate query's filter on the ITEM's deleted_at found nothing wrong — and the tick kept firing, emitting outbound webhook events for a workspace whose owner had deleted it, possibly while deleting their account. Both queries now join workspaces and require `w.deleted_at IS NULL`. Nothing is destroyed: a restored workspace resumes firing, which the test asserts, because "stops firing" and "is destroyed" are very different answers to someone who restores a workspace and only one of them is right. That test first failed for the WRONG REASON and the fixture was at fault: it counted every outbox row in the workspace, and item creation writes its own, so the assertion was satisfiable by the fixture itself and discriminated nothing. Scoped to the reminder event type. **`Use: "remind <ref>"` declared a requirement the command contradicts.** cmdhelp derives the machine-readable arg spec from that string, and `--rearm` takes no ref — so the published contract said "required" for something optional. The requirement is CONDITIONAL, which cmdhelp cannot express, so the honest declaration is `[ref]` plus the explicit check that names both call shapes. The round-5 test grew a `required` column, which is what makes this observable at all: asserting only the arg NAMES would have passed. **The pad_item tool description omitted both new actions.** The params were declared and the actions dispatched, but the prose an agent reads to decide what a tool can do did not mention them — discoverable only by someone who already knew to look. It now describes both, including the two things an agent gets wrong: remind_at is an instant, and nothing but an explicit ack retires a fired reminder. Three mutants, three killed. Forty-four across the unit. * fix(reminders): codex round 7 — one predicate for the scan and the arbiter Third instance of one class, so this fixes the SHAPE rather than the instance. The class: the candidate scan filters on something the fire transaction does not revalidate, so a change committed between them fires a reminder that no longer qualifies. Round 3 was a re-armed instant. Round 1's soft-deleted item was the same thing caught from the other side. Round 7 is a workspace deleted between the scan and the fire — the round-6 fix added the condition to the SCAN only, and the arbiter went on not knowing about it. Fixing those one at a time is what let the third happen. `reminderFireable` is now a single string that both sites reference: the scan asks it and the fire UPDATE re-asks it, so they cannot disagree, and a fourth condition is one edit in one place rather than two edits someone has to remember are paired. Written as a correlated EXISTS on item_reminders.item_id rather than a JOIN precisely so the identical text is valid in both a SELECT and an UPDATE, and the scan drops its table alias so the two uses are the same characters. What deliberately stays outside it: `fired_at IS NULL` and `remind_at <= ?` live on the reminder row itself, are already spelled identically at both sites, and folding them in would need a parameter order the shared form cannot express. Said in the comment so the omission reads as a decision. Both directions are now tested at the arbiter — a workspace deleted mid-pass and an item deleted mid-pass — because the item case previously relied on the item load coming back nil, and someone simplifying the EXISTS down to the workspace check alone would otherwise still see green. Three mutants, three killed: the arbiter dropping the shared predicate, and the predicate dropping each of its two halves. Forty-seven across the unit. * fix(reminders): codex round 8 — workspace export silently dropped every reminder WorkspaceExport is a hand-maintained field list, so a new table joins it only if someone remembers. Reminders did not: a backup/restore, or a SQLite→Postgres migration via `pad db migrate-to-pg`, dropped every pending reminder with nothing in the destination to show anything had gone. The line that list has always drawn is item-scoped workspace CONTENT (comments, links, versions — exported) versus per-user state (stars, watches — not). A reminder has no user column and hangs off an item, which puts it on the exported side. Stating the rule rather than just adding the field, because the next person adding a table needs to know which side they are on. LIFECYCLE MARKS ARE CARRIED, not reset. A fired-and-unacknowledged reminder is still owed to whoever armed it, so it arrives pending; an armed one whose instant has passed fires once on the destination's first tick, which is what would have happened had the workspace never moved. Re-arming everything on import would invent a schedule the user did not set. NULL rather than empty string for the unset marks — the lifecycle is defined by NULL-ness, and "" would make a never-fired reminder read as fired at "". TestMigratedTablesCoversTheExport caught the second half, which I would have missed: `pad db migrate-to-pg`'s NUL preflight decides what to REFUSE on from MigratedTables, so a table the migration copies and the preflight does not know about is a gap in exactly the guard that exists to prevent one. Added there too, with the reason it can never actually fire — every column is machine-produced, so it is listed for coverage rather than expectation — and the "six tables" prose it falsified is now seven. Two mutants, two killed: export dropping the block, and import discarding the marks. Forty-nine across the unit. * test(reminders): state the fire-path invariant and pin it from the invariant The lead's read on why rounds 4 and 7 were the same class: the fire path had no stated invariant, so each fix defended an instance. This states it, and derives the pin from the paragraph rather than from the bug history. THE INVARIANT: the candidate scan is a hint and may be assumed to prove nothing. Every condition that made a row a candidate is re-asserted inside the transaction that marks it fired, in the same statement that does the marking, so checking and writing are one atomic act. Worded as "the scan proves nothing" rather than as a list on purpose — a list invites the next person to add a condition to the scan and stop, which is exactly what happened four times here. TestFirePathInvariant is the pin: one table, one row per scan-side condition, each invalidating that condition in the window between the scan and the fire and asserting the same three things — nothing fires, no event leaves, the reminder is not consumed. The earlier per-defect tests are folded in as rows; they said the same thing one instance at a time, which is how four of these shipped. Adding a fifth condition to the scan without a row here should feel like an omission. It carries a positive control, because four cases that all assert nothing happens would pass against a build that never fires at all. The matrix immediately falsified a claim in the paragraph I had just written. I wrote that the item load inside the transaction is "for the payload, not for the check"; removing the item half of reminderFireable alone changes no observable behaviour, because the load then returns nil and the deferred rollback undoes the write. Item liveness is defended TWICE and a single-mutant experiment cannot say which guard is carrying it — removing both is what kills the test. Both are kept, the predicate is named as primary (the row never matches, so no write happens at all), and the asymmetry is stated: workspace liveness has no second line, which is why dropping ITS half does fail the pin. Six mutants: five singles plus the pair. Five killed alone; the item single survives by design and is documented as such rather than left as an unexplained green. Fifty-five across the unit. * fix(reminders): codex round 9 — one legacy row could hide every reminder **P1: items.item_number is NULLABLE and I scanned it into an int.** Migration 006 added the column to existing rows, so a pre-numbering item still carries NULL — and scanning NULL into an int fails the Scan, which fails the QUERY, which degrades the whole pending-reminder section. One old row, and the feature is dark for everyone in that workspace. ListWatchesForUser, which this query was modelled on, uses sql.NullInt64 for exactly this column. I copied its shape and dropped the part that handles the column's actual nullability — the same way of being wrong as the round-5 cmdhelp fixture: borrowing a form without borrowing what it knows. The legacy row now carries no ref rather than a fabricated "PREFIX-0", which would name a different item. **P1: export shipped reminders that import could only discard.** The items section filters on deleted_at IS NULL, so a soft-deleted item is not in the bundle and its reminder can never be reunited with it. My comment claimed the item_links rationale — round-trip the raw graph so a restore reunites them — which is true for links and false here, because links keep soft-deleted endpoints in the bundle and items do not. A link is a row ABOUT two items; a reminder whose item is absent is a dangling schedule. **P2: import wrote remind_at raw.** Import is a writer, and a bundle is not necessarily one this server produced — hand-edited, or from another instance. A local offset or a bare date would land in the one column every comparison downstream treats as a UTC instant, firing early, late, or never. It now normalizes like every other door. An unparseable value is SKIPPED with a warning rather than failing the restore, matching the lenient import-side precedent already in this file, and the raw value's LENGTH is logged rather than its content. Three mutants, three killed; two needed rewriting because the single-line form did not compile — reverting the nullable scan also requires reverting the render, and dropping the normalization orphans a variable. PROCESS FAULT, recorded because it makes this round's findings weaker than they look: I edited the tree while this review was reading it — committed the invariant work and ran five mutation experiments, which write and restore source, over the same files. A review binds to the tree it read and I moved it underneath. Every finding above was re-verified against the current tree before being acted on, and the next round runs with no concurrent edits. * fix(reminders): codex round 10 — one orphaned item aborted a whole restore An ORPHANED item — one whose collection is missing from the bundle — still gets an itemMap entry. It has to: the entry is written before the skip because parent resolution inside the same loop reads the map for items it has not reached yet. So `itemMap[x] != ""` is satisfied by an id that names no row, and inserting a foreign key to it fails (SQLite enforces FKs here via the DSN's `_pragma=foreign_keys(on)`; Postgres always does). The pre-existing mapping is the sharp edge. The aggravating half was mine: this loop treated a failed reminder insert as FATAL, where item_links and item_versions both skip, so one orphaned item carrying a reminder rolled back an entire 900-item workspace restore. A reminder is the least critical thing in a bundle and it had the strictest failure handling in the file. Both halves fixed: the loop gates on items that actually landed, and a failed insert warns and skips like its siblings. TWO GUARDS THAT ONLY DIE TOGETHER, and this is measured rather than assumed. Reverting either alone leaves the test green — with the map gate restored the skip survives the FK failure, and with the fatal return restored the gate means the insert never fails. Removing both is what fails it. They are kept as a pair because they defend the same failure at different depths (prevent the bad write / survive a bad write arriving some other way), and the pair is recorded in the code so a future reader does not delete one as dead after watching its mutant survive. Second time this shape appeared today; the first was item liveness on the fire path. The bundle in the test is hand-built, because ExportWorkspace cannot produce an orphan — which is the reason it needed a test. That shape only arrives from a hand-edited or foreign bundle, and surviving those is what import is for. Three mutants: two singles that survive by design, plus the pair that kills. Sixty-one across the unit. * fix(reminders): codex round 11 — four contract slips, one of them another unit's **suggested_next returned up to eight entries against a cap of three.** Round 3 prepended reminders PAST the list's own cap, reasoning they should not compete for slots. Every consumer — the web dashboard, `pad project next`, `pad project ready` — is written for three. Worse, it silently falsified a decision recorded elsewhere: BootstrapDashboard deliberately has no suggested_next_overflow_count BECAUSE this list is capped at three upstream, and its comment names raising that cap as the moment to add one. My change made another unit's reasoning wrong in a file I never opened. The combined list is now trimmed back to three, reminders still leading — a reminder can push a task suggestion out, which is the right way round, and the full set stays addressable in pending_reminders. My first version of that trim used `limit`, which is REASSIGNED above to len(candidates) — so on a workspace whose only entries are reminders it would have truncated to zero, killing precisely the case the surface exists for. Caught by reading the surrounding lines before running anything; it has its own test now. **pending_reminders was uncapped in the bootstrap projection.** BootstrapDashboard embeds *DashboardResponse, so every new field joins the boot payload automatically — here, a window of up to 50, which is the budget PLAN-1410 spent a unit trimming. Capped at 5 with an overflow count, under its own constant rather than borrowing bootstrapAttentionCap: they answer different questions and a future change to one must not silently move the other. **Truncation was reported from the wrong question.** The collector used the store's `more` flag, which answers "is there another PAGE", not "did I read all of THIS one" — so a window filling part way through the final page reported that the caller had seen everything while unread rows sat behind the fill point. The paging bounds are now injectable so the case is testable at all: building it with a window of 50 needs ~75 rows in a specific pattern, with a window of 3 it is four. **Import accepted acked-without-fired**, which is not one of the lifecycle's three states. Such a row fires, is excluded from the pending surface because it is already acked, and can never be acknowledged because AckReminder requires acked_at IS NULL — an event emitted into permanent invisibility. The acknowledgement is dropped and the schedule kept, since an ack of something that never fired means nothing. Five mutants, five killed (one rewritten — removing the flag orphans a variable). Sixty-six across the unit. * fix(reminders): codex round 12 — a read is not a hold; scope the arm; ack from the ack Four P2s from round 12 (two independent runs, both landing on the same line of the fire path), each closed at the layer where it lives: - fireOneReminder pins the item and workspace rows FOR NO KEY UPDATE on Postgres before the arbiter UPDATE. reminderFireable re-asserted liveness at the predicate's instant and nothing held it to the commit instant; under READ COMMITTED an archival could commit in between and the event left the process about a deleted resource. Same idiom and same lock strength as CreateAttachmentForLiveItem; SQLite is excluded by its BEGIN IMMEDIATE, not skipped for convenience. Two PG-only pins verify "blocked" in pg_stat_activity, not by elapsed time; the pin-removed mutant fails both. - CreateReminder asserts "live item of THIS workspace" in the INSERT's own SELECT and returns ErrReminderItemGone otherwise. The table had an FK and no same-workspace constraint; a mismatched pair fed another workspace's title to this one's dashboard and webhooks. Handler maps it to 404. - AckReminder matches every fired row (COALESCE keeps the first ack, updated_at moves only when acked_at does), so a no-match means exactly "not fired at the instant of the ack". The handler no longer decides 409-vs-200 from the row it read before the UPDATE. - The invariant paragraph gains its missing sentence: "at that instant" means the commit instant, and the pin is what makes the predicate's instant and the commit instant the same one. Round-12 caveat carried: both runs were static reads (sandbox blocked Go's build cache), so "four" is a floor, not a measurement. Refs IDEA-2641 * fix(reminders): codex round 13 — a reminder's workspace must agree with its item's, at every read Every reader scoped by r.workspace_id and then joined the item without asserting the two agree. No door writes a disagreeing row today (CreateReminder derives the pair from the item; import maps within the workspace), and the table has nothing that forbids one — so a hand-edited bundle, a future move door, or a direct write would carry one workspace's item into another's dashboard, export, and webhooks. The identity goes into reminderFireable (scan + arbiter), the Postgres row pin, ListPendingReminders and the export query. One test writes the row raw — the only way one can exist — and asserts it is inert at each site; the predicate-removed mutant scans and fires it. Refs IDEA-2641 * fix(reminders): codex round 14 — the by-id and by-item reads assert the same identity as every other read GetReminder scoped by the row's own workspace_id and ListRemindersForItem by item_id alone, so a row whose two columns disagree — the class rounds 12 and 13 closed at the scan, the arbiter, the pin, the pending surface and the export — was still readable through the two reads that reach a single row. reminderOwned is that identity on its own, without the liveness half those two reads must not have (a fired reminder on an archived item is history worth showing). The write paths reach a row only through GetReminder, so scoping it scopes them; a row no door can write needs no door to delete it. ListRemindersForItem now takes the workspace its caller already resolved the item in. The raw-row test asserts both reads refuse the row from both sides; the reminderOwned-removed mutant surfaces it through GetReminder. Refs IDEA-2641 * fix(reminders): codex round 16 — an archived item's reminders are readable, and its verbs say "archived" The doors resolved the item live. Listing an archived item's reminders answered 409 from a GET, and ack/re-arm/delete answered a bare 404 for a reminder that exists on an item that exists — while the store, since round 14, deliberately keeps that history readable. The API already has a posture for archived items: GET reads them, mutations answer 409 "archived … restore it before editing" (writeItemResolveError). The list now follows handleGetItem; the lifecycle verbs load the item include-deleted, run the visibility check first, and then answer the same 409 every other item mutation does. One test walks archive → list 200 / ack 409 / arm 409 → restore → ack 200 on the same rows. Refs IDEA-2641 * fix(reminders): codex round 17 — one suggestion per item, the archived 409 by slug, and the door courtesy named Three findings on the server pass. (1) An item that was both a fired reminder and an ordinary candidate appeared in suggested_next twice; the ordinary entry is dropped, the reminder entry (which carries the ack id) stays, and two reminders on one item remain two entries. (2) Round 16's 409 for an archived item's reminder was written by re-resolving item.Ref, which is derived and empty for a legacy item with no item_number — so the class most likely to be legacy fell through to a bare 404. The slug is handed over instead. (3) The archived check in resolveReminderForWrite is check-then-write, and an archive landing in between lets the verb through: accepted and documented — it is the posture of every item mutation here (UpdateItem's UPDATE has no liveness clause), the outcome is benign, and putting liveness in AckReminder's WHERE would re-create the no-match ambiguity round 12 removed. Refs IDEA-2641 |
||
|
xarmian
|
63da2f4f5f |
feat(store,server,cli): count and repair the legacy NUL population (BUG-2810)
Layers A and B stop the value being written. Neither makes a row that
already carries one go away, and BUG-2810's filing is what that costs: an
affected workspace exports with a 200 and re-imports with a 400, so a
self-hoster restoring their own backup is blocked with no path forward in
the product, and `pad db migrate-to-pg` fails partway through the copy
against PostgreSQL's jsonb parser rather than up front.
This is DOC-2823's S3, on Dave's day-54 rulings: U+FFFD as the replacement,
repair standalone only with a migrate-to-pg preflight that refuses and
prints the command, `--repair-nul` on import shipping default-strict.
ONE REPAIR, beside the one predicate. textguard.Repair lives next to
ParameterRefused because four layers that agree about what is REFUSED and
disagree about what a repair PRODUCES is this bug family arriving one step
later. Its contract is a property over the same corpus, in both directions:
every refused value becomes one all four layers accept, and every accepted
value comes back IDENTICAL. The second half is the load-bearing one — a
repair that tidies values nobody complained about rewrites
`{"a":"x\\u0000y"}`, six literal characters after a doubled backslash, and
corrupts it.
The JSON arm is a string-literal SCANNER, not decode-walk-remarshal, which
is what the recon write-up proposed before it was written. Re-marshalling
changes four things nobody asked to change — object key order, insignificant
whitespace, integers wider than float64, HTML-ish characters — and silently
drops one of a document's LITERAL duplicate keys, which is a gap BUG-2812
owns and the last thing a repair should do. Scanning copies every byte it
does not deliberately rewrite, so an untouched document is byte-identical
without that having to be argued. A substring replace is not equivalent and
the test that proves it took a mutation to find: a doubled-backslash literal
ALONE never reaches the scanner, so the discriminating fixture is one
document carrying a live escape AND a literal.
THE COUNT IS COMPUTED IN GO. Measured on the read path in this worktree: a
row planted with `bad<NUL>name` reads back into a Go string with all 8 bytes
and the NUL intact, while `length(name)` in the same database answers 3.
TASK-2824 found that C-truncation and concluded no DB-side REPAIR could be
trusted; the same measurement on the read path says no DB-side COUNT can be
either. SQL narrows — `instr(col, char(0))`, plus the escape prefix on
JSON-classed columns, which is textguard's own pre-filter — and never
decides. The decision stays ParameterRefused with isJSON from the shared
86-column list, i.e. Layer B's classing.
Row addressing is read from the live schema rather than a hand-kept map:
39 tables carry protected columns, one (item_wiki_links) declares no primary
key and is addressed by rowid, two have composite keys, and five have a
single key that is not `id`. The repair checks RowsAffected because an
address that stopped selecting its row would otherwise commit an UPDATE that
touched nothing and report it as repaired — the one failure an operator
cannot see in the output.
`email_optouts(email)` is both a protected column and its own primary key.
Repairing it changes the row's identity and can collide with an existing
row, which in that table means somebody starts receiving mail again. It is
reported and skipped, with the reason.
The import flag is NOT an exemption from the gate. `--repair-nul` buys the
body one repair attempt and then runs the same `bodyDecodesNUL` on the
repaired bytes, which still decides — a decode path that skipped the check
is the door BUG-2803 spent thirty rounds closing, on the endpoint carrying
the largest attacker-controlled body in the product. Only the ESCAPE form is
repaired: a raw NUL byte makes the document invalid JSON, and widening what
parses is not this flag's job. Both doors are covered, JSON and tar.gz,
because giving them different answers is how one of them keeps being
forgotten.
Postgres is settled with evidence rather than sent up as a ruling: it cannot
hold either defect (22021, 22P05) and the four-way differential test already
pins that, so the scan reports not-applicable WITH the reason rather than
returning a zero a reader could mistake for a clean database.
Spellings settled here, per the dispatch: `pad db scan-nul` and
`pad db repair-nul` as siblings rather than `repair --nul`, matching
`migrate-to-pg`'s hyphenated compound — a repair verb that errors when given
no flag is a worse shape, and there is no second repair to share it with.
scan-nul IS the dry run, so repair-nul grows no --dry-run. It refuses while
the server is running unless --force, on the `pad db restore` precedent: the
report is a claim about a database, and one somebody else is concurrently
writing makes it a claim about a moment that has passed.
docs/backup.md's section on this is rewritten. It still said the rule lives
in the binary and not the database, which S2 made false, and it pointed at
this item for a preflight and a repair that now exist. Its import examples
also showed `pad workspace import < file`, which has never worked — the file
is an argument.
Closes BUG-2810.
|
||
|
xarmian
|
ba1255881d |
fix(server): refuse a decoded NUL in a JSON request body (BUG-2803) (#1220)
* fix(server): refuse a decoded NUL in a JSON request body (BUG-2803)
The body half of BUG-2782 (path) and BUG-2784 (query). A caller-supplied
string reached a Postgres text parameter, Postgres refused it, and the
handler answered 500 — the honest answer is 400.
WHY THE TRANSPORT RULE CANNOT BE EXTENDED, which is the whole reason this
is a different fix rather than a wider middleware. ValidateQuery works
because a decoded query value is a substring of the raw query with ASCII
substitutions: the bad byte in the raw text IS the bad byte in the value.
That property fails for a JSON body — the reachable NUL arrives as the
six-character escape, all ordinary ASCII — so no request middleware can
find it without decoding the body, which is the handler's job.
MECHANISM, each premise measured against encoding/json rather than
reasoned about:
raw NUL inside a string -> decode ERR (invalid character in string literal)
raw NUL after the value -> decode ERR
the escape in a value -> decodes to a string CONTAINING a NUL
the escape in a KEY -> same
a DOUBLED backslash -> decodes to literal text, NO NUL
the uppercase spelling -> not a JSON escape at all
So the escape is the only vector and its substring is a sound FAST PATH
(absent -> no NUL possible), but not a sufficient test: a doubled
backslash carries the same six characters and decodes to text. In this
product that is not hypothetical — items and documents store markdown,
and a document about JSON escapes is an ordinary thing to write. The
exact step is json.Decoder.Token(), which returns DECODED strings, covers
object keys and arbitrary nesting (an item's fields blob), and needs no
knowledge of the destination type.
NOT REFLECTION over the decoded value, the other obvious design: it sees
[]byte fields AFTER base64 decoding, so a body carrying legitimate binary
({"b":"AQAC"} -> bytes 01 00 02) would be refused for a NUL that is not
text. A token walk sees the base64 characters. No request struct has such
a field today (searched: []byte with a json tag in internal/server and
internal/models, non-test — only models.YjsUpdate.UpdateData, which no
handler decodes from a body); the token walk is chosen so adding one
later cannot silently start rejecting valid requests.
BUFFERING IS NOT A COST. json.Decoder.Decode already holds the whole
top-level value in memory — refill accumulates into dec.buf and grows it
by doubling (encoding/json/stream.go) — so streaming never avoided the
copy. Measured on the 64 MiB workspace-import shape, total allocation:
stream+Decode 354.7 MiB, ReadAll+Unmarshal 256.5 MiB, ReadAll+Decode
512.5 MiB. Peak heap is order-dependent and does not discriminate; the
first run of that measurement showed a 0.77x peak win that vanished when
the legs were swapped, so only the allocation figure is claimed.
POPULATION, measured on Postgres 17 through the real router with a
control leg on every endpoint (92 mutating routes enumerated via
chi.Walk; 13 probed):
before: 12 of 13 DOOR (control 201 / NUL 500, SQLSTATE 22021)
after: 0 of 13 — every NUL leg 400, every control leg unchanged
Confirmed doors: workspace name, collection name, item title, item
content, item fields value, item title via PATCH, comment body, agent
role name, view name, document title, webhook secret, workspace import.
workspace-token name is UNMEASURED, not clean — its control leg 500s on
an unrelated FK in this fixture. The other 79 routes are unprobed, not
claimed clean; the completeness argument is structural instead, and
enforced by a test rather than asserted.
SECOND DEFECT, named rather than slipped in: the six handlers that
decoded straight off r.Body had no http.MaxBytesReader either — the cap
decodeJSON has always applied — so each was an unbounded body read.
Routing them through decodeJSON closes that too.
COMPATIBILITY: json.Unmarshal refuses trailing non-whitespace after the
JSON value where Decode ignored it. Deliberate, same direction as this
fix, and the only behaviour change beyond the refusal. Trailing
whitespace still passes. An EMPTY body still returns a wrapped io.EOF,
because handlers_playbooks.go reads errors.Is(err, io.EOF) as "no
arguments supplied" — caught by TestPlaybookRunAcceptsEmptyBody, which is
exactly the wiring a helper-level change is blind to.
No call site changed for the refusal itself: all 65 decodeJSON callers
already turn a decode error into a 400 carrying err.Error().
Release note: a NUL character in a JSON request body now returns 400
instead of 500 on Postgres deployments.
Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN
* fix(server): follow the NUL refusal into JSON-encoded string fields (BUG-2803)
Codex round 1 on #1220: the check scanned ONE JSON layer, and several
fields cross the wire as JSON-ENCODED STRINGS rather than nested objects
— an item's fields, a collection's schema, a workspace's settings. The
OUTER decode of {"fields":"{...}"} yields the inner document as literal
text, in which the escape is still six ordinary characters and no NUL
exists, so the single-layer token walk passed it.
MEASURED on Postgres 17 with a control leg on each, after the
single-layer check was already in place:
item.fields as a JSON-encoded string 500 control 201
collection.schema as a string 500 control 201
workspace.settings as a string 500 control 201
The error is DIFFERENT from the rest of this family, which is why it is
worth reading rather than assuming:
insert collection: ERROR: unsupported Unicode escape sequence (SQLSTATE 22P05)
22P05, not the 22021 the path and query halves produce. The outer string
is pure ASCII so it never trips the text-encoding check; this is
Postgres's own JSON parser refusing the escape inside a document bound
for jsonb, which cannot represent a NUL. After this change all three
answer 400 with their control legs unchanged.
THE FIX: when a decoded string is itself a complete JSON object or array
— the class this API re-parses downstream — walk it too, to a depth
bound of 8. Recursion terminates on its own (each level is a strict
substring of the one above); the bound keeps a hostile body from buying
many full re-parses, and AT the bound the body is refused rather than
passed uninspected, since the escape is known to be present and the walk
has stopped looking.
WHAT THIS OVER-REFUSES, by design and pinned by a test: the rule is
structural, not destination-typed, so a plain TEXT field whose ENTIRE
value is a valid JSON document carrying the escape is refused too, even
though its column would have stored it. Prose ABOUT a JSON escape does
not parse as a bare document, so the case is narrow, and a value of that
shape breaks any consumer that parses it. The destination-typed
alternative — an allow-list of the fields that arrive JSON-encoded — is
exactly correct and goes stale in silence, which is the failure mode
ValidateQuery's comment rejects when it explains why per-site query
validators could not be written.
Tests: nested documents (fields/schema/settings/array/twice-encoded),
with controls for ordinary content, a doubled backslash INSIDE the
nested document, a string that starts like JSON but does not parse, and
prose that merely mentions the escape; the over-refusal pinned as a
decision rather than left as an accident; the depth bound; and a wiring
leg through the real router on SQLite, where the write would otherwise
SUCCEED so a green cannot be the database doing the work. Fixtures build
their JSON-encoded strings with encoding/json rather than hand-written
backslashes, since the escaping rules are the subject under test.
All four new tests fail with the recursion removed.
Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN
* test(server): build the NUL-bearing timeline fixture through the store (BUG-2803)
TestTimeline_NeverEmitsACursorItWouldRefuse built its fixture through the
API on a premise its own comment stated: "a structured id comes from the
item's fields blob, which nothing validates on write". BUG-2803 made that
false — decodeJSON now refuses a body whose strings decode to a NUL,
including one nested inside a JSON-encoded `fields` string — so the API
can no longer produce the row and the test 400'd on its fixture.
Repaired rather than deleted, because the DEFENCE it covers is still
live: rows in this shape can predate the rule, and the store has no such
check of its own, so a migration, an import or any future non-HTTP writer
can still produce one. The timeline must keep refusing to hand out a
cursor it would then reject.
The fixture now writes the blob directly, injecting the six-character
JSON escape rather than a raw NUL — the blob is JSON text and both
backends reject a raw NUL in it; the NUL comes into existence when Go
DECODES the blob, which is exactly how the timeline ends up with one
inside an entry id. The test is not vacuous under the change: it asserts
the NUL-bearing id took the positional fallback, so an injection that
failed to produce a NUL fails the test rather than passing quietly.
This is the CONVE-23 case — a change that falsifies existing prose owes a
sweep for that prose. The stale sentence was found by the test failing,
not by the sweep, which is the weaker of the two ways to find it.
Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN
* docs(server): correct the timeline comment BUG-2803 falsified (CONVE-23)
The entryID fallback's comment said note and decision ids "come from the
item's fields blob and nothing validates them on write". BUG-2803 made
that half false: the HTTP API now refuses a request body whose strings
decode to a NUL, including one nested inside a JSON-encoded `fields`
string. The sentence was true when written and nothing in this branch's
diff pointed at it.
The fallback still has to exist, and the corrected comment says why:
the STORE has no such check, so rows predating the rule — and anything
writing a blob by another path, a migration, an import, a future
non-HTTP writer — can still carry one.
SWEPT AND DELIBERATELY LEFT: two nearby comments
(handlers_timeline_id_collision_test.go, handlers_timeline_structured_test.go)
also say "nothing validates them on write". Both are about id FORMAT and
DUPLICATION — an imported artifact carrying a UUID-shaped id, a
hand-written blob repeating one — and this change validates neither. In
context those sentences remain true, so they are left alone rather than
edited into noise.
Sweep command: grep -rniE "nothing validates|not validated on write|no
validation on write|unvalidated" --include=*.go internal/ cmd/ — six
further hits, all about other subjects (github_pr raw writes, terminal
schema keys, push payload format, decodeJSON's size bound).
Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN
* fix(server): scope the nested-NUL walk to JSON-encoded fields (BUG-2803)
Codex round 2 on #1220. The nesting check from the previous commit
recursed into ANY string that parsed as a JSON document, on the argument
that a structural test beats a destination-typed one. That argument was
wrong in a way I had written down as an accepted trade and should have
weighed as a defect: a plain-text `content` value holding a JSON snippet
that merely MENTIONS the escape was accepted before this branch, is
stored in a text column that has no problem with it, and was newly
refused — including on RE-IMPORT of an export carrying it.
Refusing input the server itself produced is a worse failure than the
door the unscoped recursion was closing. Measured before the fix: a
workspace whose item content held such a snippet exported 200 and
re-imported 400.
The walk now descends only under keys whose STRING value is a JSON
document something downstream re-parses: config, events, fields,
metadata, phase_data, plan_overrides, schema, settings, tags, traits.
WHY A LIST IS SAFE HERE, when ValidateQuery's comment rejects exactly
this shape for query parameters: there the set of names is unbounded by
design (parseItemListParams turns any unrecognised parameter into a field
filter), so no list could be complete. Here the set is a closed property
of the wire model — a field is JSON-encoded because a Go struct declares
it as a string holding JSON — and
TestJSONEncodedFieldKeysCoversTheModels derives it from internal/models
and fails when a new one appears. The list cannot go stale in silence.
Over-inclusion is the safe direction and the list takes it: a listed key
that is not really JSON-encoded costs one parse attempt and can only
refuse a complete JSON document carrying the escape, while a missing key
reopens a door. `traits` is listed for that reason — it carries JSON but
its declaration has no comment saying so, which is exactly how the
derivation test would have missed it, so the test asserts coverage in one
direction only and the list is allowed to be a superset.
The walk also changed shape: decoding into `any` and walking the value,
rather than a token stream, because key context is needed to know which
subtree is JSON-encoded. The []byte reasoning is unchanged and still
holds — decoding into `any` never produces a []byte, so a base64 field is
seen as its ASCII text rather than as decoded bytes that might contain a
legitimate 0x00.
Tests: text fields carrying a JSON document are ACCEPTED (five keys),
with a leg proving the same document under a JSON-encoded key is still
refused, so the pair differs only in the key; the derivation test; and
the depth-bound fixture now nests under a JSON-encoded key at every
level, since nesting under an ordinary key would never start the
recursion and would have passed for the wrong reason.
STILL OPEN, and the lead holds it: a LEGACY row whose stored fields blob
already carries the escape still exports 200 and re-imports 400. That is
data this fix cannot make importable without weakening the write-side
refusal, and the disposition (repair sweep, flagged import, or documented
acceptance) is a product ruling. Recorded on the item.
Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN
* fix(server): close the three body doors codex round 3 found (BUG-2803)
All three verified before fixing, none taken on the reviewer's word.
1. BUNDLE IMPORT BYPASSED THE REFUSAL (P1). handlers_import_bundle.go
parses pad-export.json itself rather than through decodeJSON, so the
SAME workspace import — reached with Content-Type application/gzip
instead of application/json — walked straight past the NUL check into
Postgres. The bundle's export blob is now checked with bodyDecodesNUL
before ImportWorkspace, answering the same 400. Test drives a real
tar.gz through the router with a clean-bundle control leg, because this
path answers 400 for a dozen unrelated reasons (bad gzip, out-of-order
tar, duplicate entries) and a bare 400 would prove nothing.
2. ONE CALLER SWALLOWED THE NEW ERROR (P2). handlers_admin.go's
test-email endpoint read `if err := decodeJSON(...); err != nil ||
input.To == ""` and fell back to the admin's own address, so a body
carrying a NUL answered 200. An ABSENT body legitimately means "send it
to me"; a body that is present and REFUSED is a different thing, and
collapsing the two turns a validation error into a success. The two
cases are now separated on errors.Is(err, io.EOF).
3. THE COMPLETENESS TEST COULD NOT SEE PAST TWO CALL SHAPES (P2). It
scanned for json.NewDecoder(r.Body) and io.ReadAll(r.Body), so it was
blind to io.ReadAll(io.LimitReader(r.Body, n)) — a shape ALREADY in the
package — and to any alias or helper. A completeness test that misses a
live example is worse than none, because it reads as coverage. It now
scans for the thing that cannot be spelled around, a reference to the
request body at all, and requires every FILE touching one to be
accounted for with a written reason. Both directions are asserted: an
unaccounted file fails because a door may have opened, and an accounted
file that no longer touches a body ALSO fails, so the list cannot rot
into stale excuses that quietly cover a future reader. Verified with a
positive control (an added body reference in an unlisted file fails) and
a negative one (a stale entry fails).
FOUND BY THAT WIDENED SWEEP, and fixed here rather than filed: the raw
artifact import (POST /workspaces/{ws}/import-artifact) takes TEXT, not
JSON, so it never went through decodeJSON and inherited neither the NUL
refusal nor the path/query rule — a body is neither. A raw NUL or
invalid UTF-8 reached the store and Postgres answered 22021, which the
handler turned into a 500 for what is a client error. It now applies
bindableText, the same predicate ValidatePath and ValidateQuery use, and
answers 400 invalid_body. Note the shape difference from the JSON half:
there the ESCAPE is the vector because a decoder rejects a raw NUL;
here the RAW BYTE is, because nothing is in the way.
Each fix has a mutation run against it: disabling the bundle guard fails
the bundle test, disabling the artifact guard fails the artifact test,
and both controls still pass.
Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN
* fix(server): the escape gate was unsound, and YAML has its own (BUG-2803)
Codex round 4, two P1s, both reproduced before fixing.
1. THE FAST PATH LET A REAL NUL THROUGH. bodyDecodesNUL gated on "does
the raw body contain the six-character escape". That is unsound: the
BACKSLASH itself can be written as an escape, so a body carrying
\u0000 contains no literal six-character sequence anywhere in its
raw bytes, while the OUTER decode manufactures one inside the string —
and if that string is re-parsed as a JSON document (jsonEncodedFieldKeys)
the second parse turns it into a real NUL.
Measured through the real router before the fix: the oblique spelling
answered 201 where the direct one answered 400.
The mistake was applying a fact about how a NUL is spelled INSIDE a
decoded string to the RAW BYTES, where the backslash can itself be an
escape. That is the same layer-confusion this whole bug is made of, for
the third round running.
The gate is now a BACKSLASH. Every JSON escape mechanism requires one, so
a body with no backslash has decoded strings byte-identical to its raw
bytes, and a raw NUL cannot survive the decoder — no backslash therefore
means no NUL, at any depth, however spelled. Bodies WITH one pay for an
exact answer, a larger set than before (any nested JSON carries a
backslash-quote), which is the cost of being correct. The same
correction applies to the per-string pre-filter one level down.
2. YAML HAS ITS OWN ESCAPE VOCABULARY. The raw bindableText check added
last commit passes a double-quoted scalar `title: "a\0b"` — no NUL in
the request bytes — and the YAML decode manufactures one. Measured
before the fix: that artifact imported 201 with a NUL in the item title.
The decoded artifact is now checked too: title, body, and every
frontmatter field value, walked because a playbook's `arguments` is a
nested structure rather than a scalar. Keys are checked as well as
values, on the same precautionary grounds ValidateQuery states for
query parameter names.
Same shape as the JSON half in both cases: a value that is harmless
until a SECOND parse, checked at the layer that can see it.
Tests: the oblique spelling joins the nested-document table, and the
YAML escape joins the artifact table. Each is mutation-verified —
reverting the gate to the substring fails the oblique case only, and
disabling the post-decode artifact check fails the YAML case only, with
the raw-byte cases still killed by the raw check. That per-leg
discrimination is the point: it shows each check earns its own keep
rather than being covered by its neighbour.
Prose corrected where this falsified it: jsonNULEscape's "it is the ONLY
spelling" is true of the escape and was being used to justify a filter on
the raw bytes, which is a different claim. Both now say so explicitly.
Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN
* fix(server): multipart text fields and the bundle manifest (BUG-2803)
Codex round 4's two P2s. Both are the same shape as the rest: a
caller-supplied string reaching a text comparison through a door the
earlier fixes did not cover.
1. MULTIPART TEXT FIELDS. The multipart body is deliberately exempt from
the JSON rule — its payload is binary blob content and must not be
scanned for text validity — but its TEXT fields are a different thing.
`item_id` goes to ResolveItem and into a database comparison exactly as
the query-string channel does, and that channel has been validated at
the transport since BUG-2784; the form channel was not. multipartValues
now drops values that are not bindable text, which makes an unusable
value indistinguishable from an absent one — the disposition
resolveUploadItemID already applies to empty values.
The uploaded FILENAME gets the same predicate, with a fallback to a
generic name rather than a refusal: the bytes are fine, only the label
is unusable.
A NEGATIVE RESULT worth recording, because it changed the test: a RAW
NUL in the multipart header is NOT the vector. Go's multipart reader
refuses it as a malformed MIME header line before any handler sees it
(measured: 400, "malformed MIME header line"). The reachable spelling is
the RFC 5987 encoded form, filename*=UTF-8''sh%00ot.png, which the
header parser accepts and percent-decodes afterwards. The first version
of this test used the raw form and was testing a vector that does not
exist.
2. THE BUNDLE ATTACHMENT MANIFEST. A second JSON document inside the
tar.gz, parsed directly like pad-export.json was, so it needed the same
check. Without it a NUL in a manifest string reached
rehydrateAttachment, whose failure is logged and SKIPPED — so the import
reported success while silently dropping the attachment. The
skip-on-failure behaviour is pre-existing and deliberate (a partial
restore beats none); refusing the bad INPUT is what stops it being
reached this way. Left as it is, and named rather than quietly changed.
A VACUOUS ASSERTION THE MUTATION CAUGHT, recorded because the test would
otherwise have shipped as coverage: the filename leg first asserted
`!strings.ContainsRune(body, 0)` on the RESPONSE, which is JSON — a NUL
in the filename comes back as the six-character escape, not as a 0x00,
so the check passed whether or not the fix was present. It did pass with
the fallback disabled. Now it decodes the response and asserts the
replacement name. The item_id leg had the mirror-image weakness: it
asserted "not a 500", which is the Postgres-only symptom, so on SQLite it
would have passed either way; it now asserts the request behaves exactly
like the no-value control.
Every fix in this commit has a mutation against it, and each kills only
its own leg.
Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN
* refactor(server): drop the now-unused escape constant (BUG-2803)
The gate became a backslash check, which was the last production use of
jsonNULEscape; golangci-lint's unused check failed on the next run. Its
documentation was load-bearing, so the explanation moved into
bodyDecodesNUL's comment rather than being deleted with the variable —
including the distinction that made the old gate wrong (the escape has
one spelling INSIDE a decoded string, which is not a claim about the raw
bytes).
Caught by re-running lint on the tip after the previous commit rather
than trusting the run from the tip before it.
Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN
* fix(server): rune-safe truncation and User-Agent sanitising (BUG-2803)
Codex round 5 was asked for the POPULATION rather than a confirmation —
"enumerate every remaining way a caller-supplied string can reach a
database text or jsonb parameter without passing a validity check" — and
returned three residual classes with their sinks. Two are fixed here;
the third is filed, because measuring it needs a fixture this unit
should not grow.
1. TRUNCATION CAN UNDO THE VALIDATION. Four sites cut a caller string
with a plain byte slice (name[:120], input.Name[:200]). If the boundary
lands inside a multi-byte rune the result ends in a partial sequence and
is no longer valid UTF-8 — so a value that PASSED the body check a few
frames earlier arrives at the store unbindable, and Postgres answers
22021 for a request the server already accepted.
This is the interesting one, because no input-side round could have
found it: the defect is downstream of validation, and it is invisible
with ASCII fixtures, which is what every test in that area used.
truncateBindableText walks back off continuation bytes and drops the
straddling rune. Tested with 2-, 3- and 4-byte runes so an off-by-one
walk-back cannot pass them all, and with a counterfactual leg asserting
the naive slice really does produce unbindable output for the same
input — without it the cases would pass against an implementation that
did nothing.
2. USER-AGENT REACHES TEXT COLUMNS. It lands in activities.user_agent
(three document paths, the connected-apps revoke) and
sessions.user_agent (three login paths), and no rule here sees a header.
The disposition is SANITISE, not refuse, and that is deliberate: a
header is metadata this server chose to record, not something the caller
asked for, so a malformed one must not turn an otherwise fine request
into a 400. The two sites that HASH the header are left alone — sha256
over arbitrary bytes is well defined, and changing what is hashed would
invalidate every stored UAHash.
The filing's own earlier probe had recorded User-Agent as NOT
reproducing on the item-create path. That was true and did not
generalise; these are different sinks.
3. NOT FIXED, FILED: the OAuth form-encoded bodies
(/oauth/token, /oauth/authorize/decide, /oauth/revoke,
/oauth/introspect) parse url-encoded form data outside the shared body
validator, with connection_name reaching oauth_connections.name and
client_id reaching the oauth_clients.id lookup. This was the ORIGINAL
subject of BUG-2803 before the filing was re-scoped, and it was recorded
then as unreachable without a fosite-backed fixture. That is still true,
and round 5's sink list is far more than the filing had. Filed rather
than guessed at.
Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN
* fix(server): narrow the gate, stop refusing natural-shape fields (BUG-2803)
Codex round 6 plus one measurement of my own. Three changes, one of them
a revert of something I got wrong in the previous commit.
1. THE GATE COST TOO MUCH, so it is narrower and still sound. The
previous commit gated the walk on "does the raw body contain a
backslash", which is correct but catches every body carrying nested JSON
(each `\"` is a backslash). Measured on a ~377 KB import-shaped body:
60106 allocs/op with that gate versus 30073 with the walk disabled — the
walk was running on ordinary traffic.
The gate is now the four bytes that begin any \u escape for a character
below U+0100. The argument: to manufacture the six-character NUL escape
inside a decoded string, each of its characters arrives either literally
from the raw bytes — in which case the raw contains the escape, which
begins with that prefix — or from a \u escape of its own, and the three
characters involved (backslash U+005C, 'u' U+0075, '0' U+0030) all sit
below U+0100, so those escapes begin with it too. Back to 30073
allocs/op, identical to the walk-disabled build.
That argument is the same KIND of reasoning that was wrong two rounds
ago, so it does not stand on its own: a differential test runs the gated
function against an UNGATED walk over a corpus built to attack it —
oblique backslash, upper-case hex, an escaped 'u', an escaped '0', a
doubled backslash — and fails on any disagreement. It also asserts the
corpus contains both answers, since agreement over a one-sided corpus
would be vacuous. Reverting the gate to the old substring fails it.
2. THE CHECK REFUSED THE NATURAL SHAPE OF ITS OWN FIELDS. `tags` and
`fields` accept both a JSON-encoded STRING and their natural array/object
form, and the walk propagated "this subtree is JSON-encoded" into
containers — so a free-form tag whose whole value happened to be a JSON
document was refused, though nothing re-parses it. Measured: refused
before, accepted now, while the JSON-encoded spelling of the same field
is still refused. The flag now marks only a direct STRING child of a
listed key.
3. REVERTED: I wired the three LOGIN paths to the User-Agent sanitiser
last commit, before reading store.CreateSession. It HASHES the header
and stores no text — the round-5 enumeration named "sessions.user_agent"
and I took the name for a column. The change would have been actively
harmful: login would store sha256(sanitised) while middleware_auth still
compares sha256(RAW), so every session from a client with a non-UTF-8
User-Agent would fail validation. A sink named in a review is a pointer
to verify, not a finding. The real sink is activities.user_agent, from
three document paths and the connected-apps revoke.
4. And the wiring leg codex asked for, on that real sink: a request
through the router with a malformed header, reading the STORED value out
of the activities row, with a control asserting an ordinary header is
kept VERBATIM. Unwiring the production call site fails it; the helper's
unit test does not notice.
Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN
* fix(server): apply the key rule at every level, not once (BUG-2803)
Codex round 7, both findings, the first confirmed by measurement.
1. THE RECURSION WENT ONE LEVEL TOO DEEP. Once the walk descended into
a JSON-encoded string it treated the WHOLE subtree below as
JSON-encoded, so a value nested two levels down — an ordinary string
inside a `fields` blob that happens to hold JSON text — was refused.
That is a false rejection, and the measurement says so plainly. With the
depth-2 check disabled, on Postgres 17:
depth 1 (the fields blob itself) -> 400 (correct: Postgres parses it)
depth 2 (a string INSIDE the blob) -> 201 (accepted, no error)
control -> 201
The handler parses `fields` ONCE. The inner text is re-escaped when the
blob is written, so what Postgres receives has a doubled backslash and no
escape at all. Only the document Postgres itself parses can carry a fatal
one.
The nested call now passes false rather than true, which makes this a KEY
RULE APPLIED AT EVERY LEVEL rather than a depth limit: a JSON-encoded key
INSIDE a document still recurses (pinned by a test), an ordinary one does
not. Same correction as round 6's natural-shape fix, one level further in
— I fixed the sibling case and left this one, which is CONVE-18's lesson
about my own enumeration being a sample too.
I checked whether anything re-parses a value inside the blob before
loosening this, rather than assuming: `arguments` was the candidate, and
parsePlaybookArguments asserts it is a native ARRAY (raw.([]any)) rather
than a JSON string, so it is covered by the natural-shape rule and needs
no second parse.
2. AN ERROR MESSAGE THAT SENT CLIENTS THE WRONG WAY. The OAuth dynamic
client registration handler prefixed every decode failure with "Request
body must be JSON". A body carrying a NUL is valid JSON, so that message
sends a client hunting a syntax error it does not have. The two failures
are now distinguished.
Round 7 also reports no break in normal CLI, MCP or web-client request
generation — they marshal JSON and encode paths and query parameters —
which is the first thing any round has said about the client surface.
Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN
* fix(server): complete the artifact check, make the walk path-aware (BUG-2803)
Codex round 8. It confirmed round 7's two fixes, then found two real
defects and two inaccurate comments — the comment half being the angle
the round was asked for.
1. THE ARTIFACT CHECK MISSED TWO REACHABLE FIELDS. artifactIsBindableText
walked the decoded artifact by TYPE, so it never covered Provenance —
whose strings are rendered into a Markdown footer appended to the stored
content — and never matched Arguments, declared []map[string]any, a
concrete slice type the walk's []any case does not match. A YAML NUL
escape in either reached storage.
It now MARSHALS the artifact and searches the output for the escape
encoding/json produces. A type switch over a struct that grows is a list
that goes stale in silence; marshalling covers every exported field,
including ones added later. The one thing it cannot see is invalid UTF-8
(which marshals to U+FFFD), and it does not need to: step 2 rejects that
in the request bytes, and YAML cannot manufacture it from valid input —
its escapes name code points, where \0 names a NUL.
Both new cases fail with the check disabled; the raw-byte cases still
pass, killed by the raw check, so each leg is discriminating.
2. THE WALK WAS NOT PATH-AWARE. A collection may declare a user field
literally named `schema` or `tags`. The walk consulted the wire-key list
at every level, so `{"fields":{"schema":"..."}}` treated a user field
name as a wire key and refused valid text holding a JSON example.
The key list is now consulted only OUTSIDE caller data — not under a
natural `fields` object, not inside an element of a `tags` array, not
inside a re-parsed document. Combined with round 7's fix that makes the
descent exactly one level deep BY CONSTRUCTION, which is why the depth
counter is gone: with the flag no longer inherited, a bound could never
fire, and dead protection reads as protection. The depth-bound test is
replaced by one that pins the property directly — an escape IN the
parsed document is refused, one BELOW it is accepted, and a
wire-key-shaped user field does not restart the descent.
3. THREE COMMENTS CORRECTED, all mine, all of the kind a reader would
believe without checking:
- MaxBytesReader: Close FORWARDS to the underlying body rather than
being a no-op, and with a nil writer there is no automatic 413 — the
cap surfaces as a read error the callers turn into 400. Behaviour
unchanged; only the claim was wrong.
- parseArtifactRequest said "three checks" while implementing five, and
its returns list omitted ErrArtifactUnbindableText. Both added by this
branch, which is exactly the prose a change is most likely to falsify
(CONVE-23).
- errJSONBodyNUL claimed all 65 callers surface its message. The STATUS
is uniform; the wording is not — several substitute a generic string.
4. And one in a test: the timeline fixture said both backends hold a
CHECK constraint a raw NUL violates. items.fields is a plain TEXT column
with no CHECK on SQLite. What was OBSERVED is "SQL logic error:
malformed JSON"; the likely source is an expression index over
json_extract, and that attribution is recorded as NOT verified rather
than asserted.
Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN
* fix(server): a regression this branch introduced, and the same trap again (BUG-2803)
Codex round 9. Both findings are mine, one of them a regression from the
round-8 restructure two commits ago.
1. THE ROUND-8 RESTRUCTURE REOPENED THE ORIGINAL DOOR. Taking the
JSON-encoded branch for a listed key skipped the plain "does this string
contain a NUL" check and asked only "does the document this string
carries hold an escape". Those are different questions. So
{"fields":"a<NUL escape>b"} — a direct NUL in the fields value, the very
first case this whole change closed — was accepted again.
Both checks now run. The test pins all three legs: a direct NUL in the
fields string, an escape inside the fields document, and an ordinary
fields string that must still be accepted, so the first two cannot pass
merely because everything under a listed key is refused.
2. THE ARTIFACT CHECK FELL INTO THE TRAP IT WAS WRITTEN AGAINST. It
searched the MARSHALLED bytes for the escape sequence, and a value
holding the six LITERAL characters marshals to a doubled backslash which
still contains that sequence as a substring — so valid content was
refused. Artifacts are documentation; text about a JSON escape is
exactly what one carries.
Worse than the bug: the comment I wrote asserted the ambiguity "cannot
arise here". It was the same doubled-backslash case bodyDecodesNUL exists
to resolve, one function away, and I wrote a sentence explaining why it
did not apply instead of checking. The marshalled form is now decoded
again and walked with the same machinery — the round trip is what makes
every field reachable without a type switch, the walk is what makes the
answer exact.
Its test asserts literal escape TEXT is accepted in title, body and a
field value, with a counterfactual leg asserting a real NUL in each of
those places is still refused, so acceptance cannot come from the check
doing nothing.
Reverting either fix fails its test and only its test.
Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN
* docs(backup): the one case where an export is not importable (BUG-2803)
Codex round 12, an operational pass. It found no migration or config
requirement, and two documentation gaps.
docs/backup.md promises that application-level export/import is portable
across SQLite and PostgreSQL. Since BUG-2803 that has one exception: a
workspace whose stored data contains a NUL exports fine and is refused on
import. It can only affect data written before the rule existed and only
on SQLite, which accepted it — a PostgreSQL instance never stored one.
`pad db migrate-to-pg` has the SAME problem and reports it worse: it
copies rows directly and never passes through the import guard, so a
legacy row fails against PostgreSQL's JSONB parser partway through the
copy rather than being refused up front. That is the likelier way an
operator meets this, since it is the operation that puts an entire old
SQLite database in front of PostgreSQL for the first time. Recorded on
BUG-2810, which owns the preflight and repair.
Round 12's other finding — that the PR's stated release note covered the
JSON 500-to-400 change and none of the rest — is fixed in the PR body
rather than in the tree.
Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN
* test(server): close two blind spots the tests themselves had (BUG-2803)
Codex round 13, asked whether the new TESTS are sound. Five findings;
these are the two that were self-contained. The other three are recorded
on the item with what each needs.
1. THE COMPLETENESS SCAN WAS BLIND TO FORM BODIES. It matched only
`.Body`, so FormValue / ParseForm / MultipartForm — which read the
request body just as surely — were invisible. It therefore reported full
coverage while the OAuth form-encoded handlers were entirely outside its
view. Widened, and it immediately failed on handlers_oauth.go, which is
the instrument working.
That file is now ACCOUNTED FOR AS A KNOWN GAP rather than as safe: the
OAuth handlers read form-encoded bodies that no rule in this family
covers (the transport rules see the query half of r.Form, not the body
half), tracked as BUG-2811 and needing a fosite-backed fixture to
measure. The test now STATES the gap instead of being blind to it, which
is the difference between a completeness claim and a completeness
appearance.
2. THE TRUNCATION TEST ADMITTED AN IMPLEMENTATION THAT RETURNED "". Its
assertions were: within the limit, bindable text, a prefix of the input.
An empty string satisfies all three. It now also asserts that an input
fitting the limit comes back UNCHANGED, and that no more than one rune
(4 bytes) is lost to the boundary — so a truncator that drops too much
fails, not just one that keeps too much.
Both were found by asking whether a broken implementation would pass,
which is the question CONVE-12 is about and which I had applied to the
production code and not to these two tests.
Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN
* test(server): the three remaining round-13 gaps (BUG-2803)
Codex round 13's other three findings, all of the same shape: a test
that would stay green with the production change reverted.
1. THE MANIFEST CHECK WAS UNTESTED. The bundle test built archives
containing only pad-export.json, so disabling the INDEPENDENT attachment-
manifest check left the suite green. The new test builds a bundle with
both entries, differing only in the manifest, so a refusal cannot come
from the export half. Verified by disabling each check separately: only
the matching test fails, so the two are independently covered.
2. THE TEST-EMAIL CHANGE HAD NO HANDLER-LEVEL TEST. Every existing leg
exercised decodeJSON, so reverting handlers_admin.go to default EVERY
decode failure to the admin's own address passed them all. The new test
drives the real endpoint with a wired mock sender and pins the
distinction that used to collapse: an ABSENT body still means "send it
to me" (control), an ordinary body still sends (control), and a body that
is present and refused answers 400 rather than being reinterpreted as
the default recipient.
3. THE MULTIPART LEG CHECKED ONE BYTE CLASS. A filter rejecting NULs
while letting malformed UTF-8 through would have passed it. It now drives
both, which matters because invalid UTF-8 is the class that reaches
Postgres as 22021 on a UTF8 database.
Round 13 was asked whether the new TESTS are sound — deterministic,
order-independent, and failing on broken code. It reported the fixtures
isolated and found five ways they were not discriminating. Two were
fixed in the previous commit; these are the rest.
Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN
* test(server): pin the wiring at every call site, not one (BUG-2803)
Codex round 14 confirmed round 13's five, then found the same shape one
level out: reverting a SINGLE call site back to the unsafe form left the
whole suite green, because the surviving fixtures are ASCII and a
helper's unit test does not care who calls it.
TestTextSafeHelpersAreUsedAtEveryCallSite asserts the wiring STATICALLY
rather than adding a fixture per site (an OAuth connection, a cloud
login, four audit paths). A byte-slice truncation of a caller string
fails it, and so does a raw User-Agent read outside the exempt set. Both
directions are checked: finding none of the SAFE form also fails, so a
scan that silently matched nothing cannot pass forever.
The User-Agent exemptions carry counts rather than being blanket, so a
NEW raw read in an exempt file still fails. All four reads in
handlers_auth.go are exempt because they feed a HASH — CreateSession
hashes the header and stores no text — and sanitising before hashing
would be actively harmful: login would store sha256(sanitised) while the
session check still hashes the RAW header, failing validation for every
client with a non-UTF-8 User-Agent. middleware_request_text.go's one raw
read is requestUserAgent itself.
Verified by reverting one truncation call site and one User-Agent call
site independently; each fails the test.
Round 14's third finding is fixed behaviourally rather than statically,
because the static scan cannot see it — handlers_oauth.go is already
listed for its form-body reads. TestOAuthRegisterRefusesNULBody drives
the real dynamic-registration endpoint with cloud mode and an OAuth
server wired, with a control leg registering successfully, and pins both
the refusal and the message split: the body IS valid JSON, so the answer
must not send a client hunting a syntax error.
Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN
* fix(server): match wire keys the way the decoder does (BUG-2803)
Codex round 16, asked whether this change is consistent with its siblings
in the same file and extensible by someone who did not write it. It found
a live bypass instead.
encoding/json matches an incoming key to a struct field by an exact match
first and a CASE-INSENSITIVE one otherwise, so {"Fields":...} and
{"FIELDS":...} land in ItemCreate.Fields exactly as {"fields":...} does.
The walk looked the key up case-SENSITIVELY, so it skipped the nested
document for a body the handler went on to accept, and the database
answered the original 500.
Measured before the fix: `fields` refused, `Fields` and `FIELDS`
accepted.
This is the same defect shape as everything else in this unit — a check
that agrees with one layer's rules while the layer that actually consumes
the value uses different ones — which is why the fix is a PREDICATE
rather than a wider map: the map is the vocabulary, and the matching RULE
belongs to the consumer. Someone adding a key should not also have to
remember to add its spellings.
The test drives six spellings including mixed case, with a control
asserting an unlisted key stays caller data in any casing, so this is
case-insensitive matching rather than matching everything.
Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN
* fix(server): fold keys the way encoding/json folds them (BUG-2803)
Codex round 17, first of five findings. The previous commit fixed the
ASCII half of key matching and left the Unicode half, which is this
bug's own pattern one more time.
encoding/json matches with Unicode SIMPLE FOLDING, not lower-casing.
U+017F LATIN SMALL LETTER LONG S folds to 's', so "ſchema" reaches the
`schema` struct field while strings.ToLower("ſchema") is unchanged and
missed the allowlist — a nested NUL under that spelling reached the
handler undetected.
Matching is now strings.EqualFold against each canonical key. The test
carries both a lower-case fold spelling and an upper-case one alongside
the ASCII cases, and keeps its control asserting an unlisted key stays
caller data in any casing.
The other four round-17 findings are recorded on the item rather than
patched here: they are genuine layer disagreements (duplicate keys
merging differently in a typed decode than in a map, a scan-failure
disposition on inputs the typed decode tolerates, and unknown-field
policy) whose fixes are design decisions rather than corrections, and
this seat is near its context bar. Each is written up with the
measurement it needs.
Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN
* fix(server): pin that a NUL-bearing manifest refusal keeps the partial workspace (BUG-2803)
Codex round 18. The comment on the manifest NUL branch said refusing the
input "stops it from being reached this way" and stopped there, which
reads as though the refusal undoes the import. It does not.
A plain error with a non-nil workspace keeps the partial workspace,
exactly as every other manifest failure in this loop does — the rollback
branch fires only for *importStatusError, and mid-stream manifest
failures intentionally keep what was imported (TASK-896). Returning a
rollback-shaped error here would give NUL-bearing manifests different
semantics from malformed ones, which is a change to the bundle-import
contract rather than a fix to this bug.
So the behaviour is unchanged and now DELIBERATE: the comment states it,
and the test asserts the persisted state rather than only the HTTP
answer. Mutation: routing the branch through *importStatusError makes
the refusal roll back, and the new assertion fails naming the release
note it would falsify. The pre-existing status/body assertions do not
notice.
Claude-Session: https://claude.ai/code/session_01AUvLoXsKdS5sdpYju6rj4p
* docs(server): record the four map-model disagreements as dispositions, and pin them (BUG-2803)
Lead ruling day-68 is land-and-follow: this branch lands on its measured
commits, and the token-stream rewrite is the BUG-2812 unit's spec rather
than a late restructure of an 18-commit branch under review pressure.
That makes the four open findings from rounds 16-17 something to WRITE
DOWN precisely, not something to leave in a trail comment.
The doc comment on bodyDecodesNUL now carries all four, with the one
root cause named: this scan decodes into map[string]any and the typed
decode does not agree with that model about keys. Two under-refuse
(duplicate-key merge; scan-failure passthrough) and are BUG-2812's spec
- both dissolve under a walk that never builds values. Two over-refuse
(unknown fields; case-variant duplicates) and are ACCEPTED, because
refusing is the safe direction. The asymmetry is stated rather than
smoothed over: within the map model, (1) and (4) are one defect seen
from two sides and only one of them fails safe.
Finding (3) is an observable compatibility change - a forward-compatible
field carrying a NUL escape now gets a 400 where it got a 200 - so it
goes in the release note as well as here. A qualification only protects
where the actor meets it.
All four are pinned by a test, measured on this tip rather than carried
over from the round-16/17 write-up. The two known-gap legs assert the
WRONG answer on purpose: when BUG-2812 lands they FAIL, naming the doc
comment and the release note as what to update. Both gap legs carry a
premise assertion - the same bodies with the disagreement mechanism
removed ARE detected - without which they would pass against a check
that detected nothing.
Claude-Session: https://claude.ai/code/session_01AUvLoXsKdS5sdpYju6rj4p
* test(server): wire release-note item 10 to the router, with its before-state measured (BUG-2803)
The disposition test proves bodyDecodesNUL RETURNS true for an unknown
field carrying a NUL escape. The release note claims the API answers
400. Those are different claims and only the second one is what an
operator or client author reads - CONVE-19, my own convention: a
direct-call test vouches for the component, not its binding.
Two legs, and the control is the load-bearing one. An unknown field with
an ordinary value must still be ACCEPTED, so this pins "refused for the
NUL" rather than "refused for being unknown". The handler does not
reject unknown fields; if it ever started to, the note's explanation
would be wrong while its status code stayed right, and no
status-code-only assertion could see that.
The before-state is measured rather than asserted from memory. Disabling
the check makes the same request answer 201 - which is main's behaviour,
since decodeJSONWithLimit there unmarshals straight into the typed value
and the key is dropped. So "answers 400 where it answered 200" is a
measurement in both directions, not a recollection of one.
Claude-Session: https://claude.ai/code/session_01AUvLoXsKdS5sdpYju6rj4p
* docs(backup): the NUL rule lives in the binary, not the database (BUG-2803, BUG-2813)
Codex round 19, the fresh-angle deploy/rollback/mixed-version pass.
docs/backup.md said a NUL-bearing row "can only affect data written
before that rule existed, and only on SQLite". The second half is true.
The first half is false, and the reason is the interesting part: the
guard is in decodeJSONWithLimit, so the invariant is a property of the
running BINARY, not of the database.
On SQLite any window where an older binary serves the same database can
still write one - a rollback after upgrading, a staged rollout with an
old and a new instance sharing a database, a second older instance on
the same file. The window closes, the guard returns, and the rows are
already stored, behaving exactly like genuinely old ones. A rollback is
an ordinary operational move, so this is not an exotic path.
The doc now states the binary-version dependence, says which dialect is
affected and why PostgreSQL is not (it refuses a NUL itself, at every
version), and gives the operational answer: drain writes from older
binaries before the new one serves, or roll forward rather than back.
Store-layer enforcement - so the running build stops mattering - is
filed as BUG-2813 rather than added here. It is a dialect-level change
and the day-68 ruling on this unit is land-and-follow.
The same false implication was carried by the PR's release note calling
such a workspace "legacy"; corrected there too.
Claude-Session: https://claude.ai/code/session_01AUvLoXsKdS5sdpYju6rj4p
* docs(server): cite the ruling in house style, not the team-room day counter (BUG-2803)
"lead ruling day-68" is the internal day counter, which means nothing to
anyone reading this repo and is inconsistent with every other citation
in it - the codebase cites a lead ruling by DATE or by BUG ref, never by
day-N. Replaced with the bug ref, which is the part a reader can
actually follow.
Claude-Session: https://claude.ai/code/session_01AUvLoXsKdS5sdpYju6rj4p
* docs(server): drop a commit count I had already measured as wrong, and stop asserting a cause I borrowed (BUG-2803)
Two defects in a comment I wrote an hour ago, both of the kind this
unit's trail keeps recording.
"an 18-commit branch" - the branch was 20 commits at
|
||
|
xarmian
|
e747a1610c |
feat(session): registry keyed on the harness session, carrying the agent name; pad session list / prune (TASK-2767) (#1200)
## Summary TASK-2767 (IDEA-2750 part 2, with part 3 riding along — the keying fix and the reaping are one mechanism). The local session registry (`~/.pad/sessions`) was keyed on the pid of the `pad session register` subprocess, which is dead before anyone reads the file. One session left a new file per call and its own pid appeared in none of them; the only live identifier was the harness pid a reader could parse out of the socket path's basename. In practice nothing wrote it (zero callers in `plugin/`, `skills/`, or hooks) and nothing read it. Now: - **One record per session, keyed on the harness session pid** — `$PAD_SESSION_PID` (harness-agnostic override), else `$CLAUDE_PID` (verified present in both the tool shell and a live plugin monitor's `/proc/<pid>/environ`), else the calling process. A set-but-invalid value is an error, not a silent fall-through. - **The record carries the agent name** the session's writes are attributed to (`ResolveAgentName`: `.pad.toml agent_name` → `$PAD_AGENT` → detected runtime; `--agent` overrides, `--agent ""` is anonymous), the harness session id, and the messaging socket's identity (inode/device/mtime — the same binding the arm-state file uses). - **One owner-identity type, one verdict.** `internal/cli/session_owner.go`: `SessionOwner` + tri-state `OwnerLiveness` (`alive` / `dead` / `unknown`). `armStateOwnerAlive` is now `OwnerLiveness(...) == alive` with its file contract preserved (socket identity else mtime; headless pid + start token; fail closed). The registry pruner takes the opposite posture on `unknown`: on Windows `pidAlive` reports dead for every pid, and a reaper built on that would delete every live session's record. - **Verbs:** `pad session register [--agent]` (writes/refreshes; prunes dead records), `pad session list [--agent] [--cwd] [--all] [--format json]` (liveness per row, newest first; dead hidden unless `--all`), `pad session prune [--older-than DUR]` (dead always; unknown only under an explicit bound; alive never). Nothing on MCP — host-local filesystem state. - **Who registers:** `plugin/scripts/pad-monitor.sh` runs `pad session register` on start, BEFORE the consent gate — presence is a fact, consent is a grant, and the record is local/0600/never on the wire. - **Legacy v1 files** list as `legacy` rows: owner = socket-basename pid (else registrar pid), liveness by pid only (v1 recorded no socket identity, and the socket-without-identity rule would have judged every legacy record dead while its session ran). A legacy row can say a session exists, never who it is. Lead rulings on the four open decisions, all as built: `agent`/`--agent` vocabulary; no server-presence merge in `list`; register from the monitor script before the gate; wire follow-on (agent name on the stream) filed separately as IDEA-2750 part 2b. One ordering change from the plan's section A: pid precedence is `PAD_SESSION_PID` > `CLAUDE_PID` > self (explicit override beats detection, mirroring `PAD_AGENT` over runtime detection); the plan listed `CLAUDE_PID` first. ## Behaviour changes for existing users of `~/.pad/sessions` / `pad session register` - Registry files are keyed on the **harness session pid** (`PAD_SESSION_PID` → `CLAUDE_PID` → self), not the `pad` command's pid; repeated registrations overwrite one record instead of accumulating. - `pad session register` records the agent name, harness session id and socket identity; stores the **real path** of the cwd; prints a different text line and a different JSON shape (the full `SessionRecord`); and **rejects** an invalid `PAD_SESSION_PID` / `CLAUDE_PID` instead of silently keying on itself. - Existing v1 files are read as `legacy` rows (owner = socket-basename pid, no agent name) and dead ones are pruned by the next register. - The plugin monitor now registers (and prunes) on every start, before the consent gate. - `armStateOwnerAlive` now delegates to the shared `OwnerLiveness`; the consent gate's observable behaviour is unchanged on every platform and key type (codex round 4 traced every caller; matrix M29 pins the socket-keyed mapping). https://claude.ai/code/session_016zc6oxBvpax6Z3iQMsAJno |
||
|
xarmian
|
99ffad1bca |
feat(server): timeline comment rows carry the agent name (TASK-2760) (#1196)
* feat(server): carry the agent name onto comment rows in the timeline (TASK-2760) An agent's comment rendered under the human's name: the name is stamped only on the linked 'commented' activity, which the timeline suppresses because the comment card stands in for it. The comment list queries now LEFT JOIN that activity and surface the name as Comment.AgentName (top-level and nested replies, on the timeline and the comments endpoint alike, through one scan helper), mirrored onto comment-kind TimelineEntry.agent_name to match the actor_name idiom. The web comment card renders it verbatim in an isolated <bdi>, separate from the human author. Store join rather than a handler-side match: the two lists are paginated independently, so a handler join misses at page edges and reads as intermittently-correct attribution. Metadata is parsed in Go, not SQL, to keep the query free of a SQLite/Postgres dialect fork. * test(store): make the activity-window premise strict, not a same-second coin flip (TASK-2760) * fix(server): replies log + link their commented activity so the agent name reaches them (TASK-2760, codex r1) The dedicated reply route wrote no 'commented' activity, and the activity is the only row that carries the writing agent's name — so a reply through the web UI rendered under a generic chip no matter what the client sent. Also rewrites the README + SKILL.md claim that comments never show the name, moves the reply test onto the real route, and asserts order/limit under the join. * fix(store): exclude comment-linked activities in the timeline's activity query (TASK-2760, codex r2) buildTimeline suppressed a comment's linked activity only when that comment was on the same page; the two sources are paginated separately, so an activity could slip through as a standalone 'commented' card. The query now excludes linked rows via NOT EXISTS on idx_comments_activity (both dialects), exact regardless of either window, and the page-local guard is removed rather than kept as a dead one that reads as load-bearing. * fix(store): item-scope the comment/activity link and freeze comment-linked activities against debounce merges (TASK-2760, codex r3) The join keyed on activity id alone while nothing in the schema ties a comment's activity to its item — scope both the LEFT JOIN and the NOT EXISTS to the item. And CreateActivityDebounced could merge a later update into the 'updated' row a comment links to, overlaying its agent stamp and bumping created_at, so two agents under one set of credentials would silently re-attribute an earlier comment; comment-linked rows are no longer merge targets. Prose corrected: the linked row is a 'commented' row OR the 'updated' row of an update that carried the comment. * fix(server,web): keep the read-skew guard beside the SQL exclusion; nowrap on every 24ch agent label (TASK-2760, codex r4) The page-local guard covers a distinct failure from the query exclusion — a comment fetched then hard-deleted before the activity query runs — so it returns with that reason written down. Sweep: of the seven 24ch agent-label rules, three lacked white-space: nowrap (both timeline cards and EpisodeFeed), so a name with spaces wrapped instead of ellipsizing; the other four already had it. Prose nits corrected; the pre-link debounce race on update-with-comment is recorded on BUG-2716 with a pointer in the handler. * docs(server,cli): state the reverse read-skew at the guard and the CLI non-rendering decision (TASK-2760, codex r5) * fix(store): debounce merge refuses a comment-linked row inside the UPDATE itself (TASK-2760, codex r6) The read-then-write left a window in which a comment could link the chosen row before the merge overwrote its agent stamp. The merge is now one statement whose predicate re-checks the link under the row write, and a zero-row merge falls through to a fresh insert. Prose corrected: a later update looks past a frozen row, to an older unlinked one or a fresh one. * fix(store,test): one freeze mechanism, and the window-edge leak proven end to end (TASK-2760, matrix survivors) The debounce SELECT-side exclusion became redundant once the UPDATE's own predicate refused linked rows, and its 'look past to an older unlinked row' semantics folded a later change into an earlier entry — a linked row now simply ends the coalescing run. And the server suite could no longer tell the SQL exclusion from the restored in-memory guard, because it only exercised the same-page case; a test now drives the page-edge case codex found (comment outside its window, activity inside), where only the query can help. * fix(web): drop a duplicate nowrap in EpisodeFeed — the rule already had it (TASK-2760, codex r7) Corrects the round-4 sweep count: of seven 24ch agent-label rules, two lacked white-space: nowrap (both timeline cards), not three. |
||
|
xarmian
|
ea139272ce |
fix(server,watchevents): shared session presence + honest push acceptance (BUG-2698, BUG-2699) (#1175)
Two coupled defects in the push path, fixed as one unit because 2699's honest-acceptance signature is the substrate 2698's fix reports through. BUG-2699 — Bus.Publish reports acceptance. The endpoint returned 200 pushed:true for a publish that was dropped, because Publish returned nothing and swallowed every failure. An error is two outcomes and they are kept apart: ErrBusClosed proves nothing was published (503 unavailable, safe to resend), while any other error means UNCONFIRMED — go-redis retries a command whose reply was lost, which is why the publish script already carries a dedupe token — and gets 502 push_unconfirmed, deliberately off the web client's safe-to-resend list. MemoryBus was the worse case, not the exempt one: neither implementation checked `closed`, and the in-process one dropped silently with no log at all. Seven production call sites, not the six the item named; the six best-effort producers discard through one named helper, and an AST-based test fails when a new producer publishes directly. BUG-2698 — RedisSessionPresence. A session-targeted push was resolved against the answering replica's presence registry, and the handler skips the publish when the target is absent, so a POST landing on A for a session held on B dropped the instruction and answered delivered_sessions:0. Fixed at the REGISTRY rather than the gate: a shared registry makes the snapshot right, which makes the picker complete and restores the gate's original premise, so the existing skip becomes correct for the reason it was written. Entry and index are written atomically under a TTL renewed by a goroutine that lives exactly as long as the connection; a crashed process stops renewing and Redis clears it. Staleness is unchanged and now stated in full: ~30s for a dropped client, ~90s for a dead instance. delivered_sessions becomes nullable — null means published-but-uncountable, never zero — documented as three states at every consumer. 35 Codex review rounds. Notable: a per-user registry cap was added and then removed after three consecutive rounds found defects inside it and a fourth was asked whether it belonged in this PR at all; a context bound was documented, disproved by its own test (go-redis does not apply a command context to connection establishment — 5.0s measured against a 150ms ctx), and rewritten to say what is true. Every fix was mutation-checked; one instrument was deleted for passing on broken code and one for not asserting its own premise. Filed rather than folded in: BUG-2724 (Redis keyspace namespacing + Cluster), BUG-2725 (delivered_sessions is an estimate with error in both directions), BUG-2726 (no concurrent-connection limit on the watch stream), BUG-2727 (Redis absent from readiness/metrics; silent subscriber loss), BUG-2728 (epoch-reset resume lead). Gates: build · make lint 0 issues · go test ./... (25 pkgs) · svelte-check 0 errors · vitest 1738 passed · CI 7/7 including Go (PostgreSQL) and Nix. |
||
|
David Barkhausen
|
5784d907c0 |
feat(cli): PAD_TOKEN environment override for stored credentials (#879) (#1160)
* feat(cli): PAD_TOKEN environment override for stored credentials (#879) Layer 1 of #879: if PAD_TOKEN is set, the CLI uses it as the bearer token and skips the credential-store lookup — gh's GH_TOKEN convention. Reads never write credentials.json, so a read-only override sidesteps the multi-agent identity contention completely; the store is never touched under the override. Per the acceptance grounding notes: - NewClientFromURL resolves PAD_TOKEN before the per-server store lookup (the single token-attachment chokepoint). - whoami no longer lies under the override: it skips the store short-circuit and reports the effective identity via a real /me fetch, with an 'Auth: PAD_TOKEN environment override' line. - auth login/logout print a gh-style stderr notice when the override is active. logout additionally pins its server-side session invalidation to the STORED token — an unpinned Logout() after the constructor change would have invalidated the env token's session — and skips the server call when there is no stored session. - pad init's status line and server info's report disclose the override (env_token_override field; the auth probe uses the token every other command would use). Zero behaviour change when PAD_TOKEN is unset. Token minting stays web-only; a minimal 'pad token' CLI is offered as a follow-up. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(cli): review round 1 — init fails on a rejected PAD_TOKEN; login shortcut skipped under the override; logout asymmetry documented Per the PR #1160 round-1 review: - Bug 1: pad init's auth step no longer falls back to stored credentials when a set PAD_TOKEN is rejected — it fails with the distinct rejected-token message (mirroring whoami), which also makes the status line's override disclosure truthful. Test drives the real padInitCmd flow and asserts the stored identity is never consulted. - Bug 2: login's 'Already logged in as <stored user>' shortcut is skipped when the override is active — it reads the store, and firing it right after envTokenNotice contradicted the notice. A second test pins the unchanged no-override shortcut behaviour. - Doc ask: the deliberate logout asymmetry (the env token's own session is never invalidated; its lifecycle belongs to the minter, GH_TOKEN posture) is now stated in env_token.go's doc comment and the README PAD_TOKEN section. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
xarmian
|
449ac109e9 |
fix(server,mcp,cli): refuse system-metadata keys in fields_patch + retry-hostile error code (BUG-2627 part 2, BUG-2675) (#1166)
* fix(server,mcp,cli): refuse system-metadata keys in fields_patch + retry-hostile error code (BUG-2627 part 2, BUG-2675) Part 2 of BUG-2627 closes the door that mints the defect parts 1 and 3 dealt with: `--field implementation_notes=<json>` stored the entries as a JSON-ENCODED STRING, which is invisible to every reader and — since part 3's guard — disables `pad item note` on that item until the row is repaired. Refused SERVER-SIDE in `fields_patch`, not at the CLI as the item's scope line proposed. The deviation is deliberate and recorded on the trail: the CLI is one of three clients, and all three lower a user field-setter into the same key (`pad item update --field` at cmd_item.go, the MCP `field` param via dispatch_http_advanced.go on remote, and stdio by shelling out to that CLI). One gate closes all three; a CLI-only refusal would have left remote MCP writing the key. Both call sites were read, and the CLI's lowering is now pinned by a test rather than left as an assumption. Scope, stated because it is deliberate: this closes UPDATE only. The full `fields` blob stays open because that door is SHARED — `pad item note` / `decide` / `github link` send one, and so does convention activation via BuildConventionItemFields -> ItemCreate. Closing it would break the system writers the gate exists to protect. Item create therefore remains a mint site, tracked with the rest of that surface in BUG-2685. The refusal message is per-key: implementation_notes -> `pad item note`, decision_log -> `pad item decide`, github_pr -> the GitHub link flow, and `convention` refuses WITHOUT naming a command, because none writes it. PATTE-135 wants a remedy that works in the failing state; a single "use pad item note" line would have been wrong for three of the four keys. BUG-2675 rides along on one ToolSurfaceVersion bump, as ruled. The append refusal from part 3 reached MCP agents as `server_error` — not our fault, and not transient, so agents could reasonably retry a failure that is deterministic forever. New closed-set code `stored_state_unreadable`, emitted on BOTH transports: HTTP classifies the sentinel error directly, stdio via a `pad-structured-error/v1:` marker the CLI now writes for its own local refusal (the first marker generated without an upstream APIError). v0.16-then-v0.17 is what a one-transport fix costs. Also here: - items.ReservedOverrideKeys -> ReservedFieldKeysIn. The second caller passes a patch, not an override map, and the old doc comment said fields_patch was an open exposure — true until this commit. - `Extract* returns nil for THREE reasons` -> FOUR. The comment listed four; the count was corrected everywhere except the code. - Consumer-read artifacts updated where the claim is ACTED on, not only where it is documented: instructions.md (incl. a "do not retry this code" section), the catalog `field` param description, `pad item update --help`, README. Gates: build · make lint · go test ./... · make test-pg · Codex. Eleven-mutation matrix run against the new tests; every one killed by an assertion (two were rewritten after killing by compile error / surviving, which proves nothing). Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V * fix(server,mcp): honest remedy when the stored value is already unreadable; name the MCP-facing code (Codex round 1) Three findings from the pre-push review, all real: P2 — the refusal named `pad item note` unconditionally, but on an item whose stored value is ALREADY undecodable that command refuses too (part 3's guard). The caller was routed in a circle: field write refused -> run the note -> refused -> back again. That is exactly the failure PATTE-135 exists to prevent, and my own trail had reasoned the remedy was safe on the strength of the HEALTHY case only. The message now inspects the item's stored value and, when the key is unparseable, says so and points at the one action that works in that state (inspection), noting that the repair needs a full `fields` write no CLI flag exposes. P2 — two doc claims were false where an actor reads them. The catalog said reserved keys are refused "on every action that accepts field", which includes CREATE, and create is deliberately NOT gated; and both the catalog and instructions.md named `validation_error` (the HTTP code) where an MCP client actually receives `validation_failed`. Both corrected, and the create exception is now stated rather than implied by omission — an agent that reads only "refused on update" will otherwise assume create is fine, which is how a hole gets used. nit — the destructive-downstream sentence claimed every reserved key becomes unreadable and trips an append guard. True only for the two append-backed keys; github_pr and convention are simply overwritten. The clause is now per-key, because a confident wrong explanation is worse than a vague right one. Two more mutations run against the new branch: always-readable (the circular remedy returns) and never-readable (the working remedy disappears) — both killed by assertions. Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V * fix(models,mcp,cli): one appendability predicate, per-key docs, stdio hint parity (Codex round 2) Five findings, all real. P2 — the message's readability check and the guard it describes were two different decodes. Mine unmarshalled into []json.RawMessage; the guard uses []ItemImplementationNote. A stored `[1]` passed mine and fails the guard, so the message would again have prescribed a command that refuses — the same circularity round 1 caught, through a narrower door. Replaced with models.StructuredFieldIsAppendable, which ASKS the guard rather than re-deriving it, plus an agreement test over 12 shapes x 2 keys that compares the predicate against the real Append* helpers. Verified by restoring the RawMessage version: the table catches it on `[1]`. P2 — stdio lost the new code's hint. Remote MCP told the agent retrying is pointless and how to inspect; stdio got the code with an empty hint, because the CLI's marker envelope carried none and the classifier parsed none. Both fixed, with the hint hoisted into paired constants (the same duplication StructuredErrorMarker already uses) and the test comparing the two TRANSPORTS' envelopes rather than either against a literal. P2 — doc text was still false for `convention`: the catalog, the instructions and `--help` all said reserved keys are maintained by note/decide/the GitHub flow, which is true of three of the four. Each key now names its own writer, and `convention` names library activation. Also dropped the `malformed_override` advertisement — that is the SERVER's code; an MCP client sees validation_failed for both refusals. nit — the classification test called structuredAppendErrorResult directly, so deleting either dispatcher call site left it green. Added dispatcher-level tests driving the real server + store, asserting the code, the hint, and that the item's stored fields are byte-identical afterwards. Mutation-verified by reverting the note call site. Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V * fix(items,models,mcp): github_pr stays writable through fields_patch; no nil-map panic (Codex round 3) P1 — the gate refused `github_pr`, and that was wrong. My model was "system writers use the full fields blob, user setters use fields_patch", which holds for three of the four reserved keys and fails for this one: `pad github link` needs a local git checkout and the `gh` CLI, so it is excluded from remote MCP BY NAME, and internal/mcp/dispatch_http.go's noRemoteEquivalent map tells remote agents in so many words to use `item update --field github_pr=...` instead. For that audience the patch door is not a bypass of the writer — it IS the writer. So the refusal deleted a documented capability from remote agents, and answered with a message naming a command they cannot run: the same circular remedy round 1 caught, aimed this time at the people the gate was meant to help. items.PatchRefusedFieldKeysIn now exempts the key and records the rule being applied — refuse a raw write where a real writer exists — rather than the list it produces. Whether remote agents should get a proper PR-link action, so the key can be closed too, is a product question and is left as one. P2 — the hint told agents to read the bad value with `pad_item action=get`. They cannot: stripDuplicatedFieldsKeys removes implementation_notes and decision_log from every MCP response's fields blob, and the top-level arrays come from the extractor, which returns nil for exactly this shape. The value is invisible on the whole surface. The hint now says so and routes to a human, who can read it with `pad item show --format json`. P2 — `fields` holding a literal `null` unmarshals into a NIL map with no error, and both Append* helpers assign into what they get back, so `pad item note` PANICKED ("assignment to entry in nil map") instead of appending. Reproduced, fixed in parseMutableItemFields, and pinned by a test that fails on a panic rather than taking the process down. An absent blob and a null blob mean the same thing to every caller. Pre-existing, but it sits in the function family this bug is about and the message was about to recommend the command that panics. nit — README claimed a "closed eight-code taxonomy" (17 codes, and I had just added one) and read as if create lowers into fields_patch. Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V * fix(models,mcp): predicate matches the append on malformed blobs; stop promising a broken workaround (Codex round 4) P1 — round 3 exempted `github_pr` from the update gate on the strength of noRemoteEquivalent's documented workaround. That workaround does not work: ingestFieldKVP (remote) and parseFieldFlag (CLI, and so stdio) both store a `field` value as a STRING, so the PR data lands double-encoded and no link appears — the BUG-2627 shape one key over. Filed as BUG-2696 with the three candidate fixes; NOT folded in, because the narrowest of them changes how every field value is typed. The exemption stands regardless: refusing would leave remote agents with strictly less than a broken door. What changes is what we may PROMISE. The catalog, instructions.md, version.go and README said "this is how you link a PR"; they now say the door is open and broken, and to hand PR linking to a human. Advertising a capability that isn't there is the failure mode this whole unit keeps circling. P2 — StructuredFieldIsAppendable returned TRUE when the whole fields blob was unparseable, on the reasoning that a broken outer blob is a different problem. True of the cause, irrelevant to the caller: the Append* helpers bail on that same parse, so the message again named a command that fails. It now returns false, which is simply the honest answer to the question asked, and the agreement table grew a malformed-outer-blob leg — the gap that let the disagreement through. P2 — the message claimed a raw field write always stores something Pad cannot read back. That holds for the CLI and MCP (a `--field` value is typed by schema lookup and these keys are in no schema) but not for a direct REST caller sending a valid array, who is refused for ownership reasons alone. Reworded to say both parts. nit — a misplaced parenthetical in the README read as if item CREATE lowers into fields_patch. It does not; it sends the full blob. Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V * fix(mcp,models): stop the remote hint advertising the broken PR workaround; classify an unparseable blob as retry-hostile (Codex round 5) P1 — I corrected four artifacts that pointed agents at the github_pr field write and missed the fifth: noRemoteEquivalent's own text, which IS the message a remote agent receives when it calls `github link`, and which Codex had quoted at me in round 3 to establish the workaround existed. The nearest artifact to the actor was the one I did not open. Both entries now say there is no working remote path and name BUG-2696, with a test pinning the negative so a future edit cannot quietly reinstate the advice while the write is still broken. P2 — a fields blob that will not parse at all produced a bare parse error, so `note` / `decide` reached agents as `server_error`: transient- looking, and therefore retried, for a failure that is as deterministic as the per-key one BUG-2675 exists for. Both Append* helpers now wrap that parse failure in ErrStructuredFieldUnreadable, which both transports already classify, and the malformed-blob test asserts the sentinel rather than just an error. Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V * docs(mcp,cli): qualify what an agent can actually see when the state is unreadable (Codex round 6 nit) Round 5 widened stored_state_unreadable to cover a fields blob that fails to parse outright, which made half of its own hint false: MCP's normalization strips a broken structured KEY (so `get` hides it), but leaves an unparseable BLOB as a raw string (so `get` shows it). The hint and instructions.md asserted the first case for both. Now stated per layer, in the two paired constants and the instructions. The reason it is worth the words rather than being cut: an agent told 'you cannot see this' does not look, and would have missed a value that was in fact right there in the response it already had. Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V * fix(mcp): classify the move/copy reserved-key refusal as validation on stdio too (Codex round 7) P2 — carried over from v0.22, surfaced because THIS bump documents the two reserved-key refusals as agreeing across transports. The move/copy message ("Field(s) reserved for system metadata and not settable here") matched none of the stdio validation patterns, so the same deterministic 400 arrived as validation_failed on remote and server_error on stdio — and server_error reads as transient, so an agent retries a refusal that can never pass. One pattern added, plus a test that drives both real classifiers with the real server message text for both refusals, so a reworded message that stops matching fails here rather than in the field. nit — the github_pr exemption is UPDATE-only; move and copy still refuse it, because there the argument is BUG-2674's (an override reintroduces the key the migration just dropped), not this one's. The catalog and instructions said "not refused" without that qualifier. Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V * fix(mcp): cover the copy path's own refusal wording in the stdio classifier (Codex round 8) P2 — round 7 fixed the MOVE wording; the copy path words the same class of refusal differently ("Destination collection has no field(s): ..."), so it kept arriving as server_error on stdio and validation_failed on remote. Third message in one family, and the round-7 test used the move text for every case, which is why it missed this. The parity table now carries all three real messages plus a control leg using one the pattern list already covered — without it the table could pass by matching everything. Recorded in the pattern list's comment rather than left implicit: matching prose is a stopgap, the structural fix is the pad-structured-error/v1 marker that carries the code instead of inferring it, and until a refusal emits one, this test is where a new wording has to be added. Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V * test(mcp): use the real upstream codes in the parity fixtures (Codex round 9 nit) The copy legs carried `validation_error` where the handlers actually emit `malformed_override` and `invalid_override`. The 400 branch ignores the body code today, so the test passed either way — which is exactly why the fixture mattered: it was quietly recording a wrong contract, and a future code-aware classifier would regress against a table that agrees with it. Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V * docs(mcp): the upstream code is not forwarded to MCP clients (Codex round 10 nit) The catalog said the server's own code (validation_error / malformed_override) appears in the MCP message. It does not: the 400 branch emits code=validation_failed with a fixed "Validation failed." message and the server's text in the HINT, discarding the finer-grained code. Reworded to say what an agent actually receives, and to say that telling the two refusals apart means reading the message. Also carried the update-only qualifier on the github_pr exemption into the README, matching the catalog and instructions. Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V * docs(items): state the exemption predicate, not the exemption list (lead ruling) The lead's ruling on the github_pr reversal: make the REASON what the code says, so the next key added to reserved metadata is evaluated against 'does this audience have a real writer?' rather than pattern-matched onto a list that happened to be wrong for one key. Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V |
||
|
xarmian
|
bc68b84848 |
fix(cli,mcp): send raw collection slug so an alias can't shadow a real collection (BUG-2630) (#1162)
* fix(cli,mcp): send raw collection slug so an alias can't shadow a real collection (BUG-2630)
The client-side alias map (collections.NormalizeSlug) rewrote seven hardcoded
singulars ("task", "plan", …) to their plurals BEFORE the request. In a
workspace whose collection slug IS one of those singulars, the user's exact
name was rewritten away and their create/list/move landed in a DIFFERENT
collection — silently, with a success message naming the wrong one.
Fix, per lead ruling on the BUG-2630 trail, split by transport:
CLI (real HTTP, may hit a pre-resolver server) — Option 2, one shared helper
cli.WithCollectionAliasFallback: send the RAW slug first (the server's
exact-match-first resolver from BUG-2578 wins, so an exact name is never
shadowed), and retry with the alias ONLY on a collection-not-found error, only
when the alias differs. Keying on collection-not-found is load-bearing: a
request to a collection that exists but fails for another reason is never
retried into the alias (that would recreate the bug). Both the schema fetch and
the create funnel through the helper so typed --field values parse against — and
the item lands in — one collection. On a genuine double-miss the error names the
RAW slug the user typed (collection "widget" not found), not the alias.
MCP remote transport (in-process ServeHTTP against the SAME binary, which always
carries the resolver — no version skew) — drop client-side normalization
entirely and send raw. Also removed the dormant expandPath collection
normalization: no routeSpec uses a {collection}/{target_collection} path
placeholder, so the branch was dead code in the area this fixes.
Search is deliberately out of scope (filed BUG-2659): its collection is a global
c.slug=? FILTER, not a path — a miss returns 200 + zero results, not
collection-not-found, so the retry can't key on it; and handleSearch is
cross-workspace, so the per-workspace resolver has no single workspace to run
against. Cross-workspace copy is excluded too (DR-13 forbids auto-retrying the
copy mutation).
Verified live against a real server: create/list/move into a singular collection
that collides with its plural now land in the named singular; shorthand still
resolves; genuine misses error naming the raw slug. New MCP integration test
reproduces the original shadow (item → PLANS-1) when normalization is restored.
Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V
* fix(server,cli): own collection resolution server-side + capability-gate the CLI retry (BUG-2630 Codex r1)
Addresses all three Codex round-1 findings, via the lead's ruling that
dissolves the earlier "retry vs archived-protection" tension by making the
server the sole owner of resolution semantics.
Finding #2 (MCP lost the legacy abbreviations t/i/p/d and phase/phases -> plans,
which the server's ±s resolver did not cover): fold the legacy alias map into
collectionSlugCandidates as a LAST-resort candidate. Exact-match-first and the
archived-claims refusal run for the input and every structural candidate before
the alias is reached, so it never shadows or redirects around a real/archived
collection. Now every client can send the raw slug — including the MCP transport
that can't retry — and lose nothing.
Finding #1 (the client retry re-opened the archived/hidden redirect the server
deliberately refused, because not_found can't be told from absent): add a
collection_resolution capability flag to GET /server/capabilities and gate the
CLI retry on it. Happy path unchanged (raw slug, one request). On
collection-not-found ONLY, the client probes capabilities once (cached): if the
server advertises resolution, its not-found is authoritative — the slug is
absent, archived, or hidden — so the client does NOT retry. Only an older server
that lacks the flag (or 404s the endpoint) triggers the legacy alias retry,
which is non-regressive there since old servers never had the protection. The
probe fails safe toward retry. This makes the follow-up distinct-error-code bug
unnecessary.
Finding #3 (double-fail masked a substantive alias error as "collection not
found"): the helper now surfaces a substantive alias-attempt error verbatim, and
only collapses to the raw-named not-found when the alias ALSO 404s.
Verified live against a resolving server: create/list/move into a singular that
collides with its plural land in the named singular; the abbreviation `i`
resolves to `ideas`; and after archiving `plan`, `create plan` honestly fails
("collection \"plan\" not found") instead of being retried into a live `plans`.
Gates: make lint 0 issues; go test ./... green; make test-pg green.
Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V
* fix(cli): fail-closed capability probe + always-retry the schema lookup (BUG-2630 Codex r2)
P1: the capability probe cached ANY failure as "no resolver", so a single
transient blip (timeout/5xx) permanently re-enabled the alias retry and could
bypass the archived/hidden protection on a resolving server. Now the probe
distinguishes a DEFINITIVE verdict (HTTP 200 with the flag, or a clean 404 =
legacy build) from an INDETERMINATE one (transport error / 5xx): only definitive
verdicts are cached, and an indeterminate probe fails CLOSED (trusts the
not-found, no retry) without caching, so the next call re-probes. A genuine old
server still returns a clean 404, so its retry is unaffected. Renamed the
predicate to CollectionNotFoundIsAuthoritative to name what it actually decides.
P2: the create schema lookup hits exact-match-only GetCollection, which does NOT
resolve slugs server-side, so capability-gating it made `create task
--field amount=3` 404 the schema fetch, skip the retry, and send amount as the
string "3". The schema lookup now always retries the alias (nil gate),
restoring typed-field parsing against an aliased collection's schema. Best-effort
as before: a genuine miss still degrades to string fields.
New client test covers the probe: definitive verdicts cache (one probe), and a
transient failure fails closed AND re-probes on the next call (mutation-verified).
Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V
* docs(cli): note fail-closed-on-indeterminate as a deliberate safety asymmetry (BUG-2630)
Per lead review: make explicit in CollectionNotFoundIsAuthoritative's doc that
failing closed on an indeterminate capability probe is deliberate — a recoverable
alias-shorthand miss is the safer side of the trade vs a retry doing an
un-undoable wrong-write. Comment-only.
Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V
|
||
|
xarmian
|
052c971785 |
feat(plugin): consent-gated push monitor + tri-state arm/disarm + envelope (PLAN-2613 S3, TASK-2618) (#1150)
* feat(plugin): consent-gated push monitor + tri-state arm/disarm + envelope (PLAN-2613 S3, TASK-2618) The plugin layer of the push-consent gate. S2 built the CLI arm/disarm/status verbs and the arm-state file; S3 makes the monitor existence itself the gate (D1) and adds the tri-state, the envelope, and the connect ritual. - Tri-state arm-state file: a session can be explicitly ARMED, explicitly DISARMED, or absent. `pad session disarm` now writes a session-scoped OFF marker (not a file removal), so a within-session disconnect wins even in an auto_arm=true repo — the disconnect verb must not be a lie there. The marker dies with the session (same liveness), so across sessions auto_arm remains the standing contract. ResolveAnnouncedArmed folds the tri-state over auto_arm; the monitor announces its result. - Gated monitors (monitors.json): the single always-on monitor is replaced by two — an `always` auto-arm monitor and an `on-skill-invoke:connect` manual monitor — both running scripts/pad-monitor.sh. The wrapper gates on a new hidden `pad session should-arm`, dedupes concurrent monitors with a liveness-aware per-session lockfile, and carries the reconnect loop so an in-session disarm stops the stream on its next reconnect. No consent → the monitor exits → nothing listening. - D5 envelope: a push notification carries the verbatim direction-with-authority framing (confirm in-session before anything destructive/irreversible); item- change kinds stay a light informational label. - /pad:connect + /pad:disconnect skills; /pad:status gains a one-line connection header from `pad session status`. /pad:connect runs the workspace's on-session-start playbooks on the first connect only (D8), tracked by a Booted flag carried forward across arm/disarm. plugin 0.2.1 → 0.3.0. Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V * fix(plugin): address Codex R1 on S3 (disarm stops active stream, fail-closed local state) - HIGH-1: a within-session disarm now stops an ACTIVE stream, not just the next reconnect. The monitor re-checks consent every 2s while streaming and cancels the connection when it flips to not-armed, then exits (D1's whole- stream-behind-consent gate at the top of the loop), so the plugin wrapper keeps it dead. Fixes /pad:disconnect being a lie for an idle SSE that might never naturally reconnect. - HIGH-2: a corrupt/unreadable local arm-state file now fails CLOSED (LocalArmError -> not armed) instead of falling through to auto_arm, so a corrupted disarm marker can't silently re-arm an auto_arm repo. It is not reaped (reaping would re-arm on the next read); it is session-keyed and a re-arm overwrites it. - Shell wrapper: an empty (mid-startup) lock pid is treated as live so two monitors can't both steal the lock; INT/TERM now exit (a trap otherwise resumes the loop and reconnects without a lock). - Docs: plugin/skills/pad describes the new push-envelope line format; connect/status skills distinguish "consent set (armed)" from the server's observed connection counts rather than claiming "Connected". Bounded/safe-direction residuals documented in code: the reap TOCTOU and the Booted carry-forward race (both fail-closed / benign), and lock pid-reuse (dedupe only, fails toward not-streaming). Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V * fix(plugin): address Codex R2 on S3 (disarm-watcher timing, semantic corruption fail-closed) - HIGH-1: the disarm-watcher now starts BEFORE the connection is opened, so a disarm during connection/header negotiation cancels the request too (the request is built on streamCtx). streamWatchEvents also re-checks consent before delivering each notification and stops the stream if it was withdrawn, so no push is printed after a disarm even within the poll window. - HIGH-2: a syntactically-valid but semantically-garbage arm-state file (e.g. {} or {"pid":1}) now fails CLOSED via a well-formedness check (StartedAt + PID must be present, as our writer always stamps them) before liveness or reaping — so it can't be judged owner-dead, reaped, and re-armed through auto_arm, nor mistaken for a live headless arm naming init. - LOW: the cleanup trap uses condition 0 (portable) rather than the EXIT name. The disconnect skill note reflects the ~2s active-stream drop. Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V * fix(plugin): /pad:disconnect always disarms, never gated on a linked workspace (Codex R3) Consent is session-scoped (keyed by the messaging socket, not the workspace), so a session that connected in one repo must be able to disconnect from anywhere — including a directory with no .pad.toml. The old precondition let a session move to an unlinked directory, "disconnect", and keep receiving pushes. Verified: `pad session disarm` from an unlinked cwd disarms the socket-keyed session state; should-arm then reports not-armed back in the original repo. Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V * fix(cli): enforce the Armed != Disarmed writer invariant in arm-state validation (Codex R4) armStateWellFormed checked only StartedAt + PID, so a well-stamped file that violated the writer invariant — both armed and disarmed false (or both true) — passed validation and, since SessionArmState only branches on Disarmed, resolved to LocalArmOn and armed. The writer always sets exactly one of the two; require it, so a neither/both file fails closed (LocalArmError). Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V |
||
|
xarmian
|
e40df6b31c |
feat(cli): pad session arm/disarm/status + consent config resolution (PLAN-2613 S2, TASK-2617) (#1149)
* feat(cli): pad session arm/disarm/status + consent config resolution (PLAN-2613 S2, TASK-2617) The S2 CLI contract S3's plugin skills and S4's web composer build against. S1 gated push delivery on a server-side armed bit declared at stream connect; nothing decided WHETHER to arm or sent the declaration. S2 adds both, defaulting off everywhere. - ResolveAutoArm (internal/cli/arm_consent.go): pure consent resolver. .pad.toml [push] auto_arm is the only per-repo enabler (D4); a per-user config auto_arm=false vetoes it (deny-wins); default off. Config surfaces: PadToml.Push.AutoArm + config.Config.Push.AutoArm (*bool, unset != false), both nil-safe. - Wire contract: StreamSessionIdentity.Armed sends ?armed=true on the event stream — S1's server gate finally has a sender. The monitor announces armed = live local arm OR resolved auto_arm, so a repo opt-in works end to end with a safe default-off skew. - Verbs pad session arm/disarm/status: arm/disarm manage a per-session local arm-state file; status reports the resolved local/auto decision plus the server's own armed/connected counts (new Client.ListSessions), degrading gracefully when padd is unreachable. - Arm-state file (session_arm_state.go): keyed per session by CLAUDE_CODE_MESSAGING_SOCKET (cwd fallback for headless, secondary to auto_arm). Mandatory liveness — a dead-owner file (socket vanished / pid gone) reads as disarmed and is reaped, so a crashed session can never arm a future monitor. Local client state only; the server's armed bit stays the sole delivery authority. Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V * fix(cli): address Codex R1 on push-consent (fail-closed config, owner-identity liveness) - HIGH-1: user config.toml read now fails CLOSED. config.LoadPushConfigAutoArm reads the [push] auto_arm value strictly — absent → no opinion, but present-but-unparseable → error — and ResolveAutoArmFromDisk refuses to auto-arm when it can't confirm the user's veto (was: swallowed by the lenient config.Load and treated as no-opinion). - HIGH-2: arm-state liveness now checks owner IDENTITY, not just presence. Socket-keyed files record the socket's mtime and require an exact match, so a reused socket path can't revive a stale file. Headless files record a Linux /proc start-time token (portable fallback documented) to reject a reused pid. - MED-1: arm-state writes are atomic (temp + rename) and reaping is non-destructive (re-checks staleness before removing) — a concurrent re-arm is never clobbered. - MED-2: pad session status applies the .pad.toml URL override, so it queries the same server the monitor connects to. - LOW: malformed arm-state files are now reaped (safe now that writes are atomic — a corrupt file can't be a torn in-progress write). Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V * fix(cli): address Codex R2 on push-consent (atomic config write, stronger owner identity) - HIGH-1: Config.Save() is now atomic (temp + rename), so a monitor reconnecting while `pad configure` rewrites config.toml can't read a truncated/partial file, miss a [push] auto_arm=false veto, and arm. - finding 2: socket owner identity now uses inode+device (unix) as the primary signal, with mtime as the non-unix fallback — a rebound socket or a lingering stale node at the same path gets a new inode and is rejected, closing the mtime-collision / reused-node gaps. - finding 3: headless liveness fails closed when a proc-start token was recorded but can't be re-verified (was: fell back to bare pid-liveness, which a reused pid passes); zombies (state 'Z') now report not-alive. - finding 5: `pad session status` applies an explicit --url override too, not just the .pad.toml one. - finding 4 (connect-time TOCTOU): documented as an accepted, bounded residual — a disarm racing an in-flight connect is corrected on the next reconnect; fully closing it needs S3's server-side disarm-on-open signal. Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V |
||
|
xarmian
|
625cab9984 |
fix: bound item history and stop resolving bodies nobody reads (BUG-2608) (#1147)
* fix: bound item history and stop resolving bodies nobody reads (BUG-2608)
Item history was unbounded on every surface, and summary mode paid for what it
discarded: the endpoint resolved EVERY version by walking the item's whole
reverse-patch chain, and both the CLI and the MCP dispatcher then projected
that away to metadata. An item edited under collab records a version every few
seconds while someone types, so this is routinely hundreds of full-content
reconstructions per history call, for output that shows none of them.
Two independent fixes, because they address different costs.
SUMMARY SKIPS THE WALK. `?summary=true` returns metadata from the raw rows and
never resolves a patch. That is the dominant win: the resolution was pure waste
for every caller except --full. Content and is_diff are cleared TOGETHER — an
empty body still claiming to be a reverse patch would tell a consumer to
resolve something that is not there.
LIMIT BOUNDS THE WINDOW, newest-first. That direction is not a preference: with
reverse patches, reconstructing any version means walking back from current
content, so a newest-end window is the cheap prefix of that walk while an older
one still pays for everything above it. That is also why there is deliberately
no offset — it would advertise a pagination whose later pages cost the same as
no bound at all.
Absent limit stays UNBOUNDED on the endpoint, following the item-list
precedent (maxItemListQueryLimit: "a zero/absent limit is left unbounded — this
only clamps an explicit oversized request"). The defaults live on the CLIENTS,
where a token budget is actually known: `pad item history` defaults to 50 with
--limit to change it, and the MCP catalog action injects 50 (max 300, the same
pair list and backlinks already use). A server that truncates a request nobody
bounded is a silent-truncation trap for third-party API consumers.
The MCP default goes in the CATALOG action rather than either dispatcher, so it
reaches BOTH transports — HTTP reads it off the input, and stdio receives it as
the CLI's new --limit through BuildCLIArgs. ToolSurfaceVersion 0.20 -> 0.21
with a changelog entry, plus instructions.md and README, per the 2304-family
contract discipline. Additive param bump: `limit` already existed, nothing
changed shape, and a v0.20 consumer that sends no limit now gets the newest 50
instead of all — which is the fix, not a break in it.
The restore and single-version-expand paths still resolve the FULL chain, and a
test pins that: bounding their walk would strand exactly the old versions those
paths exist to reach.
Eight mutations, each failing only the leg it targets. Three fixture problems
surfaced that way and are worth naming, because each made a test that could not
fail:
- force_version in a PATCH body does nothing (`json:"-"` on ItemUpdate), so
the throttle collapsed six edits into one version; varying the source per
edit is what actually records them.
- an 8-byte body is cheaper stored whole than as a patch, so no version was
ever is_diff=true and the is_diff assertion was inert. The fixture now uses
a body large enough that the store really stores patches.
- the cmdhelp test fixture lacked the new --limit flag, so BuildCLIArgs
silently dropped it. Verified against the REAL cmdhelp tree that the flag
is present and typed int, so the fixture mirrors the CLI rather than
flattering it.
* docs: bring CLAUDE.md to v0.21 and name why the two result caps differ (BUG-2608)
Codex round 1, both findings.
CLAUDE.md still described the MCP surface as v0.20 — stale because of my own
bump, in the document every agent working this repo reads first. README and
instructions.md are held to the version by a test; CLAUDE.md is not, which is
exactly why it drifts.
The cap "mismatch" (MCP max 300, endpoint clamp 500) is deliberate layering,
not an oversight — item lists have the identical split (300 in the catalog,
1000 at the endpoint) because the two answer different questions: an agent
token budget is only knowable in the catalog, while the endpoint's clamp is a
server-resource ceiling on what any caller may ask for. But nothing said so
anywhere, so a reader comparing the two numbers had no way to tell design from
accident — which is precisely the report Codex filed. Now stated at the
constant and in CLAUDE.md, including why the versions ceiling is LOWER than the
list one (resolving a version can cost a patch application per row, not just a
row read) and why an absent limit is left unbounded at the endpoint.
* fix+test: honest truncation notice, armed fixtures, and the residual named (BUG-2608)
Codex round 2, both findings, and the second is the more useful one.
CLI TRUNCATION NOTICE was wrong in both directions: it compared the response
length against the requested limit, so an item with exactly N versions was
reported as truncated, and a --limit above the server's ceiling was clamped
there and reported as complete. It now asks for ONE MORE row than it shows and
reports truncation only when that extra row comes back. The one case this still
cannot detect — an ask above the server's own ceiling, where the probe row is
clamped away with everything else — is stated in the code rather than papered
over by hardcoding the server's constant in the CLI.
UNDER-ARMED FIXTURES. The unbounded test seeded 5 versions, so a server quietly
defaulting to 50 would have passed the assertion that denies exactly that; it
now seeds 60. The clamp test seeded 2 and could not observe a clamp at all;
the clamp is now asserted directly against a parseItemVersionsLimit function
extracted for the purpose, over the inputs a URL can really carry (absent, 0,
negative, unparseable, either side of the ceiling).
That extraction replaced my own first attempt, which was worse than no test: it
re-implemented the clamp arithmetic in the test body and asserted the result
against itself. It could not have failed.
THE RESIDUAL, NAMED RATHER THAN IMPLIED. Codex's sharpest point is that the
summary tests cannot detect "resolve everything, then clear the fields" —
verified by mutation: pointing the summary branch at the resolving reader
leaves every handler test green, because the response is byte-identical either
way. So the performance claim does not rest on them. It rests on the handler's
summary branch calling ListItemVersionsPage (one reviewable line) plus a new
store test proving that reader really returns unresolved rows rather than
quietly resolving them — mutation-verified from the other side by making the
resolver a passthrough. The test file says all of this, including that an
end-to-end assertion would need a patch-application counter in the production
path, and why that is not worth it when the cost of being wrong is performance
rather than correctness.
* fix(cli): don't resolve for table output, guard the probe overflow, finish the CLAUDE.md bump (BUG-2608)
Codex round 3, four findings.
--full was treated as "content needed" regardless of output format, but the
table path prints no bodies at any setting — so `pad item history --full`
without --format json made the server walk the entire patch chain to build
content the CLI then dropped. That is the exact waste this bug is about,
reintroduced through the flag meant to opt into it. Content is now resolved
only when it will actually be shown.
The limit+1 probe overflowed at MaxInt: it wrapped negative, the client omitted
the parameter, and a request the user bounded came back unbounded — the
opposite of the ask. Guarded.
The truncation notice's documented blind spot was understated: it is AT the
server ceiling as well as above it, since the probe row is clamped away with
everything else. Wording corrected rather than resolved — the CLI still does
not duplicate the server's constant, because a copied ceiling goes stale
silently and asking for hundreds of versions is already opting out of a bound.
Two more CLAUDE.md sites still called v0.19 current; I fixed only the first on
the previous pass. That document describes the contract in three places and I
updated one, which is its own small lesson about grepping for every instance of
a claim rather than the first.
Live-verified against a real server: --limit 2 truncates and says so, --limit 4
on a 4-version item says nothing (the false positive Codex found), --limit 0
returns everything, --full --format json carries bodies, and the default JSON
shape carries metadata only.
* fix(server): clamp an out-of-range limit instead of treating it as unbounded (BUG-2608)
Codex round 4. `?limit=9223372036854775808` made strconv.Atoi return ErrRange,
which my parser lumped in with 'unparseable' and mapped to 0 — unbounded. So an
absurdly large number defeated the very ceiling the function exists to impose,
which is worse than no clamp at all: the bound looked enforced and was not.
Atoi hands back the saturated value alongside ErrRange, so a range-positive
input is now clamped to the ceiling — it is an oversized ASK, not a malformed
one. Range-negative still falls through to unbounded, matching a plain
negative.
Regression cases cover both overflow directions, and the mutation that
reinstates the old behaviour fails on the exact input.
This is the input-domain lesson again: I enumerated absent, zero, negative and
non-numeric, and stopped at what a person would plausibly type rather than at
what the TYPE admits.
* docs(cli): say in --help that a large --limit is capped server-side (BUG-2608)
Codex round 6, and the third time this ceiling came up — which is the tell that
the disposition was in the wrong place. I had documented it twice in CODE
comments, where the person affected never looks. `--help` says what the flag
does, and it said 'show only the newest N versions (0 = all)', which is false
for an N above the server's cap.
Now stated in both the flag help and the command's long description, including
that the truncation notice cannot detect that case and that --limit 0 is the
way to ask for a complete history.
Deliberately not naming the number: a constant duplicated into help text goes
stale silently, and 'capped server-side' is the part a user needs in order to
act. Behaviour unchanged — this is the artifact catching up with it, which is
the same correction the BUG-2301 sentinel comment and the instructions.md
overclaim both came down to.
* test: seed versions through the store so the fixture stops tripping the rate limiter (BUG-2608)
CI caught this and it is my defect, not a flake. Arming the unbounded test
above the plausible default meant seeding 60 versions, and the fixture did that
with 60 HTTP PATCHes in a burst — which trips the server's rate limiter. It
passed locally and in the Go job and failed under Nix, which is the signature
of a test that was always going to fail intermittently rather than one that
broke.
Seeding now goes through the store. That is not a weakening: versions are
recorded by the store on either path, and the endpoint under test is the READ
side, so seeding underneath the transport costs the assertions nothing while
removing a burst the server is entitled to refuse.
The three things that make this fixture work are now stated where someone would
otherwise undo them by accident — the large body (a small one is stored whole,
so no version is ever is_diff and every diff assertion goes vacuous), the
rotating source (the throttle collapses same-(actor, source) bursts into one
version), and the store-not-HTTP seeding with the rate-limit reason attached.
Re-verified after the change: the fixture still records more than 50 versions
and still produces reverse-patch rows, and the default-cap mutation now uses
the REALISTIC default of 50 rather than the 3 I first tested with — the old
5-version fixture could only have caught an implausibly small cap.
|
||
|
David Barkhausen
|
b9381bf5f1 |
feat(cli): markdown output on the remaining list surfaces; broaden ANSI stripping (#1080)
Completes #898 and fixes #1076. Markdown on the seven surfaces left out of #1070, so `--format markdown` is now honestly global and the flag help collapses to "table, json, markdown": - `item comments`, `item deps`, `project activity`, `attachment list`, `library list`, `role list`, `workspace members`. Two of those are not tabular, and markdown follows the terminal shape rather than forcing a table onto them: - `item comments` keeps the attribution-line-then-body form, and the body is emitted VERBATIM. A comment body is authored as markdown; escaping it would turn its lists and code fences into literal text. Only the attribution line, which we construct, is sanitized. - `item deps` keeps its two sections as `## Blocks` / `## Blocked by` lists. Colour carried the direction in the terminal (yellow out, red in); headings carry it here. New shared spine: `cli.RenderMarkdownTable(w, headers, rows)`. Every cell is escaped, and ragged rows are padded or truncated to the header width so a short or long row can't shift the column count and break the table. Wiring a surface is now naming columns and mapping rows. #1076 — ANSI stripping covered only SGR (`ESC[…m`), so non-SGR CSI sequences, OSC-8 hyperlinks, and stray C0 controls survived, both in the table width maths and in markdown output whose doc comment promised escape-free text. Replaced `sgrPattern` with `ansiPattern` + `stripANSI` covering OSC, CSI, two-character Fe escapes, and stray C0/DEL, with TAB/LF/CR deliberately preserved for callers that normalize them. `displayWidth` now uses it too: a control sequence is zero-width, so counting it was a column-alignment bug of the same family. Tests: 12 stripping cases, 4 table-helper cases (including ragged rows), 4 renderer cases for the two non-tabular surfaces, and the routing test extended to 8 subtests — one per surface, driven through cobra against an httptest server. Also covers the two gaps named in #1076: `item starred` and the scoped `item list <collection>` path. Each new guard was proven by mutating the source and watching it fail, not just by passing. Gates: go build ./... PASS; go vet PASS; gofmt clean; golangci-lint 0 issues. Both touched packages show the same 6+2 pre-existing Windows failures as clean main under an identical sandboxed run. |
||
|
xarmian
|
c84cf7437c |
feat(sessions): announce session identity on the event stream (PLAN-2558 S2, TASK-2560) (#1094)
* feat(sessions): announce session identity on the event stream (TASK-2560) PLAN-2558 S2. S1 gave the presence registry a count of anonymous uuids; this makes each row nameable, which is what S3 needs for an honest empty state and S5 needs for a target picker. A monitor now announces itself when it opens the stream: X-Pad-Session-Label (the working directory's basename) and X-Pad-Session-Pid. The server sanitizes both and stores them on the LiveSession; GET /api/v1/sessions returns them. TRANSPORT. The task body sketched "the stream connect carries it" without picking a mechanism and explicitly left the call open. Headers, because a query param would put the label and pid into every access-log line (this server logs path= for each request) and any proxy log in front of it — which is the same "don't let local detail travel further than it needs to" the privacy line below is about — and a separate registration POST would need its own correlation to the connection it describes, plus a matching lifecycle, when the registry entry already lives and dies with the stream. Headers ride the request that exists and sit alongside Last-Event-ID, already doing this job on this endpoint. Cost, written into the code rather than discovered later: a browser EventSource cannot set headers, so a future web-tab consumer needs a deliberate query-param fallback or a fetch-based SSE reader. PRIVACY. The basename crosses, never the full cwd — "/home/dave/Dev/ docapp" additionally hands over a home directory and usually an account name for no gain — and messaging_socket_path never leaves the machine. Pinned by a test rather than by the implementation being one line. WHAT THIS DELIBERATELY DOESN'T DO: read ~/.pad/sessions/. The task framed S2 as giving `pad session register` its first consumer, and the monitor cannot honestly be one. Registry entries are written by whatever process ran that command — a different pid — and the only matchable fields are pid and cwd, so two agent sessions in one checkout are indistinguishable and "pick the newest" is a coin flip that would put a confident wrong name in the S5 picker. Process ancestry settles it exactly and is platform-specific (this binary ships for macOS and Windows). The monitor's own cwd basename and pid are never wrong and answer the question the label exists to answer; correlating a stream to the agent session that spawned it needs an identifier the harness passes down, which is worth doing when something needs it and worth not faking until then. Also moves S1's STALENESS doc block, which sat above LiveSession.Label where it read as documenting the name rather than the whole entry. Tests: sanitizer units (whitespace collapse, control-char stripping, rune-not-byte truncation), header wiring, the end-to-end labelled session, the unannounced-client compatibility leg (a pre-S2 monitor must still register and still stream), a hostile-input leg over the wire, the client's omit-when-unset behaviour, and the basename promise. Measured rather than assumed: Go's server answers 400 to a header value containing a control byte before any handler runs (verified with a raw socket, since Go's own client refuses to send one and the two refusals are indistinguishable from a normal client test). So that arm of the sanitizer is unreachable over HTTP; it stays as defence in depth for the next caller in, and both the comment and the wire test say so instead of the test quietly passing because the transport refused the input. Mutation-tested four ways, each revert grep-verified: handler ignoring the parsed identity, monitor sending the full cwd, dropping the truncation, and the client always setting the headers. Refs TASK-2560, PLAN-2558 * fix(cli): sanitize the session label client-side per Codex review (round 1) Codex round 1's only finding, and it is a bigger deal than a missing label. Unix directory names may contain control bytes — "doc\napp" is a legal directory — and Go's http.Client REFUSES to send a request whose header value holds one: Do returns "invalid header field value" and nothing is transmitted. In the monitor that is indistinguishable from an unreachable padd, so the retry loop backs off and tries again, forever, printing nothing by contract. A user who named a directory that way would simply stop receiving notifications, with no signal anywhere. The server cannot defend against a request that never arrives. Reproduced before fixing, with a real directory and a real client, rather than reasoned about from the error message. Sanitizing in NewWatchEventsStreamRequest rather than in monitorSessionIdentity: the invariant is "this function never builds an unsendable request", which belongs at the point where a value becomes a header, not at one caller. The client's cap (256 runes) is deliberately looser than and independent of the server's (64): the server decides what a label should look like, the client only has to keep the request sane, and neither has to track the other to stay correct. The regression test does the ROUND TRIP instead of inspecting the header, because the header contents were never the bug — http.Header.Set stores anything, so an assertion on the value passes against the broken version too. Only attempting the request tells the two apart. Mutation-verified: reverting the sanitizer fails the test with exactly the "invalid header field value" error from the field report. |
||
|
xarmian
|
da6ce642da |
feat(push): pad push — user-authored instruction dispatch to agent sessions (IDEA-2544 Phase 1) (#1090)
* feat(push): add pad push <ref> -m vertical (IDEA-2544 Phase 1)
Self-addressed, human-to-harness dispatch over the existing watch-events
bus/stream: CLI -> POST .../items/{itemSlug}/push -> a new KindPush
Notification (carrying the generalized TargetUserID addressed-to field
KindAsk will later share) -> watchNotificationVisible delivers it back
to the pushing user's own connected monitor sessions. Transient,
fire-and-forget by design (no migration, no durable inbox) since
assignment already covers the durable-notification case and this is
meant to be the explicit, no-inference dispatch verb instead.
* docs(plugin): document the push notification contract (IDEA-2544 Phase 1)
Push is the one notification kind that IS an instruction rather than a
passive fact, so it gets its own lead bullet in the plugin skill's
notification-etiquette section (ahead of the read-only/park default,
which it explicitly lifts) and a mention in the monitor's description.
The embed-source skills/pad/SKILL.md has no notification section to
mirror this into (the two files diverge by design) and is left
untouched.
* fix(push): reject over-long push messages instead of unbounded Summary
Comments truncate their notification Summary to a preview (the full
body is still fetchable), but a push message IS the payload — silently
truncating it would corrupt the instruction with nothing to recover it
from. Add maxPushMessageLen (4096, measured post-collapse) and reject
anything over it with a 400 rather than truncating; state the same
bound in `pad push --help` so it's discoverable before a 400, not only
from one.
* fix(push): close the watch-fallthrough leak, disambiguate SKILL.md exceptions
Codex round 1 P1: watchNotificationVisible's push branch only returned
early on a MATCH — a non-target caller fell through to the watch-map
check below it, so anyone holding an unconditional (or predicated)
watch on the item received every push addressed to every OTHER user,
instruction text included. Push is addressed private dispatch, not an
item-level fact watchers have a legitimate claim on (unlike assignment,
which watchers are expected to see per `pad watch --help`) — the branch
now returns unconditionally for KindPush, gating strictly on
TargetUserID and never reaching the watch-map fallback either way.
Pinned explicitly since Phase 4's session targeting is expected to
inherit this same exclusivity.
Also (codex P2): reworded the SKILL.md notification-etiquette bullets —
the new push exception and the pre-existing assignment/ask exception
literally contradicted each other ("the ONE narrow exception" claimed
singularity after push had already claimed exception status). Now
explicitly enumerated as the first and second exceptions to the
never-write rule.
* test(cli): pin that PushItem inherits X-Pad-Agent (BUG-2542 rebase)
Verified, not assumed: PushItem builds its request via c.post ->
c.newRequest like every other mutating client method (CreateWatch
included), so the attribution fix's client.agentName wiring covers it
for free with zero code changes needed on this branch. Adds a live
httptest assertion rather than trusting the code-path read alone —
the same shape as TestClientSendsResolvedAgentHeader, scoped to
PushItem specifically since that's the one method this PR added.
* fix(push): disambiguate workspace in the monitor line and skill contract
Codex round 2 P1: the watch-events stream is user-scoped ACROSS every
workspace a caller has watches in, but formatMonitorLine printed only
ItemRef/Kind/Actor/Summary and dropped the Workspace field the wire
payload already carried — a session linked to workspace A receiving a
notification for workspace B would resolve the wrong item (or 404) with
no signal in the line that anything was off.
Fixed universally, not push-only: grepped plugin/ and skills/ for
anything parsing "PAD ..." lines and found none — the Claude Code
plugin host ingests the stdout line as free-text notification prose,
formatMonitorLine's only real consumer is its own fmt.Println, so there
is no wire-format consumer a workspace prefix could break. The
ambiguity predates push (any watched item across workspaces already had
it); push just makes the consequence sharper because it carries an
instruction rather than a passive fact.
SKILL.md's push bullet now tells the agent to resolve with
`pad --workspace <workspace> item show <ref>` using the slug read off
the notification line, not a bare `pad item show <ref>`.
* fix(push): respect --format json instead of hardcoding plain text
Codex round 2 P2: pushCmd's RunE ignored the global format flag and
always printed "Pushed <ref>", silently discarding --format json.
- server.pushResponse replaces the bare map the handler wrote before —
a typed {ref, workspace, pushed, message} shape, with workspace
resolved to the CANONICAL slug via s.getWorkspace (not merely echoed
from whatever the URL contained), matching the same disambiguation
need the round-2 P1 fix addressed for the monitor line.
- cli.PushItem now returns (*PushResult, error) instead of discarding
the response body.
- pushCmd checks formatFlag == "json" and calls cli.PrintJSON, mirroring
runCreateWatch's existing pattern.
internal/cli/agent_identity_test.go's TestPushItemSendsResolvedAgentHeader
needed a one-line update for PushItem's new two-value return — caught by
`go vet ./...`, not `go build ./...` (which doesn't compile test files);
folding vet into my own pre-flight going forward.
|
||
|
xarmian
|
212d59e7c6 |
fix(cli,server): make agent attribution actually happen (BUG-2542) (#1088)
* fix(cli,server): make agent attribution actually happen (BUG-2542)
Agent CLI writes were recorded as the human whose credentials they used.
Three independent defects, each verified by reading the path AND by
probing a live instance — the item deliberately held the mechanism open,
so none of this is inherited.
1. THE HEADER WAS NEVER SENT. actorFromRequest sets actor="agent" on one
signal: the X-Pad-Agent header. The only code that sets it took the
value from `agent_name` in .pad.toml and nowhere else — no
environment detection, no session detection. This repo's .pad.toml
has only `workspace`, so the header has never been sent from here and
every agent write has looked human. ResolveAgentName now resolves
.pad.toml → $PAD_AGENT → detected runtime.
2. ITEM CREATE DISCARDED THE ACTOR. createItemChecked called
actorFromRequest and kept only the source (`_, src :=`), never
setting input.CreatedBy, so store.CreateItem fell through to its
"user" default — even for an agent that DID send the header.
Comments have always stamped it correctly; item creation silently did
not, which made the skill's own contract false on its own terms.
3. SINGLE-ITEM PATCH NEVER STAMPED LastModifiedBy. Bulk ops do
(handlers_items_bulk.go); the single-item path did not, so an item
edited only by agents read as human-edited.
Only entries VERIFIED against a live session belong in the runtime
detection table, so it has exactly one: Claude Code exports CLAUDECODE=1
to child processes, confirmed by reading a pad subprocess's environment
inside one. Guessing at Cursor/Windsurf/Aider variable names would put
unverified claims in a shipped binary and misattribute silently when
wrong; those set $PAD_AGENT until someone confirms a signature.
WHAT THIS DOES NOT DO, stated in the code and the skill rather than left
for someone to assume: the header is client-supplied and self-declared.
An agent that omits it is indistinguishable from the human it borrows
credentials from, and a human running `! pad ...` inside an agent's
terminal inherits that environment and is attributed to the agent. This
makes the trail HONEST, not VERIFIED — it is not a basis for
machine-verifiable human-approval provenance, which needs a channel the
agent cannot author at all. The incident behind this item is exactly
that distinction: an agent's relay of a human's words was recorded
indistinguishably from the human typing them.
Contract corrected in both skill copies, since the item's first question
was which of contract and behavior was wrong. It was the contract: it
promised automatic agent attribution that only ever applied to
workspaces that had opted in.
Tests, each mutation-tested against its own defect reverted alone:
- TestResolveAgentName — precedence plus the negative that makes it mean
something: a plain human shell must still resolve to "". Fails 2/5
reverted.
- TestItemAttribution_AgentVsHuman — agent and human legs for create,
update and the create-stamp-survives-edit invariant. Fails on the
create stamp reverted; fails 2/2 on the update stamp reverted.
The update leg deliberately uses the OTHER writer: insertItemTx seeds
last_modified_by FROM created_by, so a same-writer edit passes whether
or not the PATCH stamps anything — the first version of this test did
exactly that and passed its own counterfactual. Caught only because
each fix was reverted separately.
- TestItemAttribution_ExplicitBodyValueWins — an explicit body value
still beats the header.
End-to-end on a live instance through the real CLI, no .pad.toml opt-in:
agent session → created_by/last_modified_by/comment all `agent`; same
binary with CLAUDECODE stripped → all `user`.
Claude-Session: https://claude.ai/code/session_01QGbUKZBAZoWdEgiTNWsXag
* fix(server): artifact import wrote a UUID into created_by (BUG-2542)
Found by Codex while reviewing the attribution fix. handleImportArtifact
set `input.CreatedBy = u.ID`, which is the wrong DOMAIN for the field
rather than merely the wrong value: created_by holds the role — "user"
or "agent" — and consumers compare it against those literals
(CommentThread.svelte, TimelineVersionCard.svelte). An imported item
matched neither and rendered as neither.
It also would have defeated the fix in the parent commit at this path: a
non-empty CreatedBy suppresses the actor stamp, so imports would have
kept a UUID while every other create path started recording the actor.
The line contradicted the comment directly above it, which said Source
was being left blank precisely so createItemChecked could stamp it "like
every other create path". Now both fields are left blank and stamped
together.
The user's identity has its own home — the items.created_by_user_id
column — which no create path currently populates. That is a separate
gap and is not widened into this change.
Claude-Session: https://claude.ai/code/session_01QGbUKZBAZoWdEgiTNWsXag
* fix: close the remaining attribution bypasses Codex found (BUG-2542)
Review found no P1s and three P2 families beyond the artifact-import bug
already fixed in
|
||
|
xarmian
|
ec7fd027fc |
feat(server,cli): watches, user-scoped event stream, plugin monitor command — PLAN-2469 Phase 1 (TASK-2533) (#1082)
* feat(store): race-free status/assignment mutation signal (TASK-2533)
Adds models.Item.LastMutation (ItemMutationSignal), populated inside the
SAME transaction that already writes status_transitions / assigned_user_id
in UpdateItemWithParentLink and MoveItemWithPreCheck. This is the
foundation for TASK-2533's watch-notification pipeline: a before/after
snapshot taken in the HTTP handler layer would race concurrent writers of
the same item, so the signal is computed where the authoritative diff
already happens, in-transaction.
* feat(store): watches table migration, both drivers (TASK-2533)
watches(id, workspace_id, user_id, item_id, predicate, created_at) per
DOC-2479's subscription-table design: durable, server-side subscriptions
that survive both the plugin-monitor process and a padd restart.
uq_watches_user_item makes `pad watch <ref>` idempotent (re-watching
upserts the predicate). Wires watches into the workspace-purge child-delete
list, mirroring item_stars.
* feat(watchevents): add in-process notification bus (TASK-2533)
New package: a global (not per-workspace) in-process pub/sub bus carrying
watch-worthy Notifications (status-change / assignment / comment; ask
reserved in the enum with no producer yet — see the follow-up server
commit). Bus is an interface specifically so a Redis-backed implementation
can slot in later without touching any caller; only MemoryBus exists today.
Package doc comment states the single-process/multi-instance limitation
explicitly, mirroring internal/events' shape.
* feat(store): watches CRUD (TASK-2533)
models.Watch + Store.CreateWatch (upsert on user+item)/GetWatchByUserItem/
ListWatchesForUser (unscoped by workspace — a watch is personal, and the
event-stream handler needs every watch a caller holds across all their
workspaces)/DeleteWatch.
* feat(server): watch/nudge event stream + CRUD endpoints (TASK-2533)
GET /api/v1/events/stream (DOC-2479): a user-scoped, cross-workspace SSE
stream, filtered server-side to the caller's watches (with optional
--until field=value predicate) plus "addressed to you" — narrowed to
assignment-to-you only for Phase 1, confirmed with the dispatcher: this
codebase has neither a Collection.Kind field nor any user->active-role
binding to ground DOC-2479's "human-gate-shaped collection targets your
active role" half mechanically. watchevents.KindAsk stays in the wire
enum with no producer. `pad session register` is the natural future hook
for a session-carried role identity.
POST/DELETE .../items/{slug}/watch, GET /api/v1/watches (unscoped,
mirrors /auth/tokens' shape for a personal, not workspace, resource).
Producer wiring (TASK-2533 audit) publishes from every live mutation path
that can produce a LastMutation signal or a new comment: handleUpdateItem
(incl. its collab sub-paths and the comment-attached-to-update path, which
bypasses handleCreateComment entirely), handleMoveItem, handleCreateComment,
item creation with an initial assignee, and the bulk-items loop (covers
archive/restore/move/set-priority/tag/untag/assign uniformly via one call
site). Named, not silent, bypasses: import bundle, status_transitions
backfill, workspace restore/purge — none are live human-facing mutations.
Known Phase-1 tradeoff, flagged not fixed: bulk mutations are NOT batched
into one notification the way the existing SSE/webhook bulk path is — a
bulk-assign of N items surfaces N individual notifications. Each is still
correctly scoped by the recipient's own watches/addressed-to-you filter
(a narrower audience than the workspace-wide SSE firehose the existing
batching protects), so this is a noise-discipline tradeoff, not a leak.
* feat(cli): pad watch + pad session register (TASK-2533)
pad watch <ref> [--until field=value] creates/upserts a durable watch;
pad watch list / pad watch remove <ref> are the hygiene companions the
dispatcher asked to be included explicitly rather than silently added.
pad watch --stream --for-session is the plugin-monitor command: one
stdout line per matching event ("PAD TASK-214 -> kind (actor): summary"),
silent on startup with no .pad.toml (hourly retry) or an unreachable padd
(backoff retry) per DOC-2479's noise-discipline contract. The retry/
backoff math and line formatting are pure, unit-tested functions; the
actual sleep loop is not (per the dispatcher's ask).
pad session register writes ~/.pad/sessions/<pid>.json (pid, cwd,
CLAUDE_CODE_MESSAGING_SOCKET when set) -- forward-looking infra for
Phase 3's live-sessions/presence surface; nothing consumes it yet in
Phase 1/2.
* fix(server): comment replies never published a watch notification (TASK-2533)
Codex round 1 finding 2 (verified real, not a false positive):
handleCreateReply is a SEPARATE code path from handleCreateComment — it
calls store.CreateComment directly via POST .../comments/{id}/replies,
not POST .../comments — and was missing the watch-notification hook
entirely. A reply to a comment on a watched item produced zero
notification. Same kind=comment publish as the top-level path, plus a
regression test covering the reply route specifically.
* fix(server): re-check current access before serving/delivering watches (TASK-2533)
Codex round 1 finding 1: ListWatchesForUser filtered only by user_id — a
watch row survives a revoked workspace membership or grant (nothing
deletes it), so GET /api/v1/watches and the event-stream's notification
filter could keep leaking item title/ref, workspace slug, actor, and
summary for access the caller no longer has.
Adds Store.ListWatchesForUser's ItemCollectionID column (needed for the
visibility check) and server.filterWatchesByCurrentAccess, which mirrors
computeSSEVisibility's RBAC resolution (handlers_events.go) — admin
bypass, VisibleCollectionIDs for member/guest full-collection access,
GuestVisibleResources for item-level grants — grouped by workspace since
a caller's watches can span many, unlike a single SSE connection scoped
to one. Fails closed on any lookup error.
Wired into handleListWatches here; the event-stream's loadWatchPredicates
call site picks up the same filter in the next commit, which also
restructures that function's Subscribe/replay sequence and therefore
touches the same lines.
* fix(watchevents): atomic ID assignment + subscribe-and-replay (TASK-2533)
Codex round 1, findings 3 and 4 (same subsystem, fixed together):
Finding 4 — sequence assignment and replay-buffer insertion happened
under SEPARATE locks in MemoryBus.Publish. Two concurrent Publish calls
could append to the ring buffer out of ID order, corrupting since()'s
ordering assumptions (it walks the ring oldest→newest assuming monotonic
IDs). Fixed by unifying seq assignment, buffer append, and the
subscriber-list snapshot under one lock; the (already non-blocking)
fan-out send still happens after releasing it.
Finding 3 — GET /api/v1/events/stream called Subscribe() and, later
(when resuming via Last-Event-ID), EventsSince() as two separate calls.
A Notification published in the window between them landed in BOTH the
replay result and the live channel, double-delivering it. Bus gains
SubscribeAndReplaySince(sinceID), which atomically subscribes and reads
the replay buffer under the SAME lock; the stream handler now uses it
whenever a Last-Event-ID is present (this commit carries that call-site
change, plus the finding-1 loadWatchPredicates filter wiring from the
previous commit — both land in the same lines of this function).
Adds a concurrent-publish ID-ordering test and a subscribe-then-
concurrent-publish no-duplicate test, both run with -race.
* fix(cli): monitor silent-start ordering + sync_required handling (TASK-2533)
Codex round 1, findings 5 and 6:
Finding 5 (P1) — runWatchMonitor called getClient() once, before the
loop and before the .pad.toml check. getClient() -> getConfiguredConfig()
os.Exit(1)s when unconfigured with no TTY, or launches an INTERACTIVE
configuration wizard when one is attached — either way a direct violation
of DOC-2479's silent-start contract, which requires "not ready yet" to be
a silent retry, never a crash or a prompt. Adds monitorClient(), which
builds the client the same way but returns a plain error instead of
exiting or prompting; client construction now happens INSIDE the loop,
after the .pad.toml gate, on every iteration, and its failure folds into
the existing padd-unreachable backoff path.
Finding 6 (P2) — streamWatchEvents ignored "sync_required" (the server's
signal that the requested Last-Event-ID was evicted from its replay
buffer), so a stale cursor got resent on every reconnect forever. Now
clears the cursor on sync_required so the next reconnect is a fresh,
non-resuming subscription instead.
Both covered by tests that assert the goroutine returns promptly on
context cancellation (proving no os.Exit / no blocking prompt was hit,
since the test process itself is still running to observe the return)
and that streamWatchEvents clears/re-tracks the cursor correctly around
sync_required.
* fix(server): uniform current-access gate for watch AND addressed-to-you delivery (TASK-2533)
Codex round 2, findings 1 and 2 — same subsystem (watch/nudge delivery
access control), fixed together; finding 2 explicitly falsifies finding
1's fix's own admin-bypass argument, so this replaces that reasoning
rather than patching around it.
Finding 1 (confirmed real): VisibleCollectionIDs / GuestVisibleCollectionIDs
deliberately over-widen for navigation — a collection ID is included if the
caller has an item grant on ANY item inside it, explicitly leaving
item-level narrowing to the caller (their own doc comments say so).
computeWatchAccessVisibility used that over-wide set directly as the
"fully visible" gate, so a guest granted item A was treated as having full
access to A's WHOLE collection, including an ungranted sibling item B.
Fixed by building the "genuinely full access" set from
GuestVisibleResources' fullCollectionIDs (populated only from direct
collection_grants, never widened by an item grant) + GetMemberCollectionAccess
/ ListSystemCollectionIDs for an actual member — exactly computeSSEVisibility's
own fullCollSet construction, not an approximation of it.
Finding 2 (confirmed real): the addressed-to-you (KindAssignment) branch in
watchNotificationVisible returned true unconditionally, with NO access
check. validateAssignmentScope (internal/store/items.go) only checks
WORKSPACE membership, never collection access, so an item can be assigned
to a "specific"-access member whose granted collections don't include it
at all — an ordinary assignment, no revocation timing required. Fixed by
gating EVERY notification kind — watch-matched and addressed-to-you alike —
through the SAME watchAccessVisibility check before either branch runs.
watchevents.Notification gains CollectionID so the check has what it needs
without a second lookup; the stream handler resolves it lazily per
workspace via a small connection-scoped cache (workspaces aren't known in
advance for addressed-to-you the way watch workspaces are), cleared on the
same reval tick that reloads the watches map.
This also required replacing computeWatchAccessVisibility's admin-bypass
argument, not just its code: "every call site filters the caller's OWN
watches" stopped being a sufficient justification once addressed-to-you
(which is fundamentally about *this* caller's own assignment activity
across every workspace) shares the same gate — a bearer-borne admin token
unconditionally trusted for that is exactly BUG-1616's blast radius. Now
mirrors computeSSEVisibility's cookie-vs-bearer distinction exactly.
Tests: guest-with-item-grant no longer sees a sibling item's watch or
stream notification (filter-level and HTTP/SSE-level); an assignment
outside a restricted member's granted collections is denied at both
levels; addressed-to-you is proven still gated (denied with no access,
visible once granted) as a pure unit test.
* fix(store): always re-read existing under lock, not just for precheck/patch updates (TASK-2533)
Codex round 2 finding 4, verified real: updateItemWithParentLinkOnce's
`existing` snapshot was only refreshed under the write lock when precheck
!= nil, ExpectedUpdatedAt != "", or FieldsPatch != nil — any update
touching none of those (e.g. a plain title-only PATCH) kept the STALE
pre-tx `existing` for the rest of the function, including the
LastMutation assignment-delta comparison added in TASK-2533's first
round. A concurrent OTHER transaction's assignment change landing between
this transaction's pre-tx read and its lock acquisition would get
misattributed to THIS transaction: a title-only update could report a
spurious, wrongly-attributed AssignmentChanged for a transition it never
made, duplicating the one the other transaction already reported
correctly (or missing a real one, depending on interleaving).
The status-transition capture already defended against exactly this with
its own separate conditional re-read; the assignment-delta capture added
later did not replicate that guard. Fixed by making the re-read
unconditional — once, right after the locks are held, before any SET-
clause building or the UPDATE itself — so every existing.* comparison in
this function is race-free by construction, not by each caller
remembering to guard itself. Also removes the now-redundant duplicate
re-read the status code had of its own.
Reproduces the exact race deterministically using UpdateItemWithPreCheck's
precheck hook as a synchronization point (TX2's assignment change blocks
mid-transaction while TX1's title-only update races its own pre-tx read
against it) — the new test fails reliably against the pre-fix code and
passes reliably (including under -race, and in Postgres mode) against
the fix.
* fix(watchevents): send under the same lock Unsubscribe/Close use (TASK-2533)
Codex round 2 finding 3, confirmed real and high-severity: Publish
snapshotted subscriber channels under the lock, released it, and only
then sent to them. A concurrent Unsubscribe or Close could close one of
those channels in the window between the snapshot and the send — a send
on a closed channel PANICS in Go, which crashes the whole padd process,
not just one subscriber's connection. The reasoning for releasing the
lock before sending ("a slow subscriber would stall everyone else") didn't
hold up: the send is already non-blocking (select/default — a full
channel is dropped-and-logged, never awaited), so holding the lock
through it costs nothing and closes the window structurally.
Adds a hammer test (many iterations of concurrent Publish / Subscribe /
Unsubscribe / Close, short-lived churned channels, recover()-wrapped so a
regression fails cleanly instead of crashing the whole `go test` run) that
reproduces "send on closed channel" dozens of times per run against the
pre-fix code (plus an independent -race detection) and passes cleanly,
repeatedly, against the fix.
* fix(server): re-fetch the user, not just the vis map, on each reval tick (TASK-2533)
Codex round 3, confirmed real: watchVisCache captured *models.User ONCE
at connect time (newWatchVisCache) and never re-fetched it; reset()
cleared only the per-workspace visibility map. computeSSEVisibility's own
doc comment explains why it re-fetches the user fresh on every call —
"so mid-stream role changes (admin demotion, user.disabled flips) take
effect on the next tick" — and the round-2 commit claimed to mirror that
"exactly," but only carried over the collection/bearer logic, not the
re-fetch itself. Net effect: a demoted or disabled admin kept fullAccess
on an open stream (both watch-matched and addressed-to-you delivery,
since both go through this same cache) until reconnect.
Adds watchVisCache.refreshUser, called by both the constructor and
reset() so the cadence matches computeSSEVisibility's actual cadence in
handlers_events.go (that function is invoked once at connect and again
only on each membershipCheck tick — never per event — so "per cache
reset" here is the same cadence, not a narrower one). Deliberately fails
CLOSED (not open-to-stale like computeSSEVisibility's own transient-error
fallback) on a fetch error, a deleted user, or a disabled user — a nudge
stream's wrong failure mode is delivering a fact to someone who
shouldn't see it, not a dropped UI update, so this trades
computeSSEVisibility's availability-leaning fallback for a stricter one
and says so in the comment rather than repeating the "mirrors exactly"
claim the fix falsified.
Tests: a unit-level pair (mirroring handlers_events_revalidation_test.go's
existing admin-demotion/disable coverage of the analogous SSE gap
exactly) proves an admin loses fullAccess after a demotion + reset(),
and a disabled user is denied outright; an HTTP/SSE-level test proves a
live stream stops delivering entirely once its connected user is
disabled and a reval tick passes. All three reproduce the bug reliably
against the pre-fix code and pass cleanly against the fix.
The HTTP-level test deliberately runs serially (not t.Parallel()): it
mutates the package-level watchListRevalInterval var, which every other
parallel watch-stream test in this package also reads via its own
ticker — writing to it from a t.Parallel() test raced against those
reads under -race (misattributed by the race detector to a whole
cluster of unrelated concurrently-running tests before this was
diagnosed). Full server package -race pass is clean after the fix.
* fix(server): decouple vis-cache reset from watch-list reload success (TASK-2533)
Codex round 4, confirmed real: on a reval tick, if ListWatchesForUser
errored, the handler's `continue` skipped visCache.reset() entirely —
the two were coupled, with reset() only reachable on the reload's
success path. A demoted or disabled user's stale identity/visibility
(round 3's fix) stayed live for exactly as long as that UNRELATED query
kept failing, so the round-3 leak reopens for the duration of any
watch-list reload error.
Fixed by running visCache.reset() first, unconditionally, before
attempting the watch-list reload. On a reload failure, the stale watch
list is kept (its own staleness is already bounded by
watchListRevalInterval's "eventually consistent" contract) but is now
gated by the FRESH visCache regardless — a demoted/disabled user is
denied via visCache even while the watch list itself lags a tick.
Chose this over dropping all delivery for the tick (the other option the
finding offered) because tying stream availability to an unrelated
query's transient health seemed like the wrong tradeoff; the comment at
the call site states this choice explicitly.
Adds a watchPredicatesLoadFault test seam on *Server (mirrors the
existing restoreAckFault pattern) so the reload failure can be forced
deterministically without breaking the DB connection for the whole test.
Reproduces the exact bug: forces the reload to fail on every tick while
concurrently disabling the connected user, and asserts addressed-to-you
delivery (which depends only on visCache, never the watch list) is
denied anyway. Fails reliably against a reverted (pre-fix, coupled)
version of the reval branch and passes cleanly against the fix.
Full server package -race pass, full suite (SQLite + Postgres) pass,
lint clean — this is the pre-PR verification matrix; round 5 will be a
narrow re-verify of this fix only.
* fix(server): bound stale watch set under persistent reload failure; atomic test seam (TASK-2533)
Codex round 5, two P2s, both confirmed real:
Finding 1 — `watches = fresh` only ran on the reload's success path, so
under a PERSISTENT (not single-tick) reload failure the watch set stayed
live indefinitely: a dead watch (removed, item deleted) kept matching
forever, and a watch created during the outage was silently missed
forever — visCache (round 4) gates current ACCESS, not whether a watch
still legitimately exists, so it couldn't catch this on its own. Fixed
by tracking consecutive reload failures and clearing the watch set once
maxConsecutiveWatchReloadFailures (3 ticks) is crossed, failing closed
on watch-matched delivery specifically while addressed-to-you delivery
(visCache-only, unaffected either way) continues throughout. Updated the
tradeoff comment at the call site so the "eventually-consistent" claim
now matches the bounded, not unbounded, behavior it actually describes.
Finding 2 — the watchPredicatesLoadFault test seam was a plain `func()
error` field, written by a test AFTER the SSE stream's background
goroutine was already running and reading it on every reval tick:
genuinely racy, unlike restoreAckFault's own use of the identical field
shape, which is set once, synchronously, before the single HTTP request
that reads it — goroutine creation's happens-before edge makes THAT
usage safe without any extra synchronization. Verified restoreAckFault
does not share the flaw and left it untouched. Fixed the watch seam with
atomic.Pointer[func() error] instead.
Test for finding 1: forces maxConsecutiveWatchReloadFailures+1
consecutive reload failures via the (now-atomic) fault seam and asserts
watch-matched delivery is suppressed once the bound is crossed while
addressed-to-you keeps delivering, then clears the fault and confirms
watch-matched delivery resumes on the next successful reload — a bounded
outage response, not a one-way ratchet. Fails reliably against the
bound disabled, passes cleanly restored.
This is the (re-run) pre-PR verification matrix per the dispatcher:
SQLite + Postgres + full-suite -race + lint + gofmt, all clean. Round 6
is a narrow re-verify of these two fixes only.
* test(store): bound the concurrent mutation-signal test's wait (TASK-2533)
CI-triage follow-up: PR #1082's plain Postgres step hit go test's default
10-minute per-binary timeout. Investigated whether any store test added by
this branch scales with runner slowness (lock-wait defaults, sleep-based
polling, transaction-hold durations):
- Watches CRUD tests (8): 0.63-0.80s each under Postgres, isolated and in
the full 741-test package run.
- Mutation-signal tests (6), including the precheck-hook two-transaction
race test: 0.48-0.80s each; the race test held at 0.48-0.51s across 10
consecutive runs (no variance) and across the full-package run.
- Full store package under Postgres: 279.17s and 277.12s across two runs
on this branch, matching the ~275s/297s baseline team-lead measured
locally and on PR #1081 — no reproducible slowdown from anything this
branch adds.
No pathological test found locally. The one test with genuine
cross-goroutine DB lock contention (TestLastMutation_AssignmentDelta_
NotMisattributedUnderConcurrentWrite) had an unbounded wg.Wait() as its
only unbounded wait — TX2's release was already unconditional (fixed 50ms
sleep, not gated on TX1's progress), so there's no deadlock risk, but
there was no ceiling on how long legitimate lock contention could
stretch it under a slow/shared runner. Replaced with a bounded 10s wait
that fails fast with a diagnostic instead of silently consuming
test-binary budget if it's ever exceeded. Verified the regression test
still fails reliably (5/5) against a revert of the round-2 fix it guards.
Could not reproduce the CI timeout locally; likely the pre-existing
~297s CI baseline (already noted as close to the 10-minute ceiling)
plus environmental variance on the shared runner, not a specific test
this branch adds.
|
||
|
David Barkhausen
|
f900b0aefb |
fix(cli): address review on markdown list output
All three requested changes from @xarmian's review of #1070, plus both nits. 1. Escape backslashes before pipes in escapeMarkdownCell. A title containing "\|" became "\|", which GFM reads as an escaped backslash followed by a LIVE pipe, so the row still gained a column. Backslash-first turns it into "\\|". Confirmed the bug with a failing test before fixing it. 2. Sanitize the group headings. Extracted SanitizeMarkdownText (SGR strip + newline collapse) and ran the collection icon and name through it, so a newline in a collection name can no longer inject a second "## " heading. Sanitizing happens per part, before joining, because it trims and would otherwise eat the separating space. Pipes are deliberately not escaped outside a table. 3. Tightened the --format help to the precise enumeration: "markdown on: item list/starred, collection list, item show, project changelog" per option (a) on #898. Nits: - `item starred --format markdown` on an empty result now says "No starred items." rather than the shared renderer's "No items found."; the empty check moved above the format branch so both paths agree. - Added format_markdown_routing_test.go: three end-to-end tests driving `item list` and `collection list` through cobra against an httptest server, asserting the markdown branch is actually reached and that the table and markdown paths don't leak into each other. Proven by disabling the markdown branch and watching the test fail. Follows the item_open_test.go pattern, with USERPROFILE set alongside HOME since os.UserHomeDir reads USERPROFILE on Windows — worth noting, as tests that set only HOME are why part of the credential-store suite fails there. Gates: go build ./... PASS; go vet PASS; gofmt clean; golangci-lint 0 issues; all markdown tests PASS. Both touched packages show the same 6+2 pre-existing Windows failures as clean main under an identical sandboxed run. |
||
|
David Barkhausen
|
4ab7b10b35 |
feat(cli): markdown output for the list commands
Implements `--format markdown` on the list commands that lacked it, so the format is honestly global rather than honestly-partial (#898, the option (a) follow-up to #851). - `pad item list` — grouped `## Icon Name (N)` sections with a table each when listing across collections (mirroring the table layout), a single table when scoped to one collection. Heading style matches `project changelog`. - `pad item starred` — single table. - `pad collection list` — Name / Slug / Items / Default. - `--format` help no longer carries the "markdown on select commands" caveat. The markdown renderers deliberately do NOT reuse the colorized helpers (ColorizedStatus, PriorityColor, Dim): markdown goes to a file, a PR body or an agent's context, never a terminal, so raw values go in and the reader's renderer styles them. Every cell is escaped — an unescaped `|` in a title silently adds a column and corrupts the row. Refs #898 |