3 Commits

Author	SHA1	Message	Date
xarmian	cb99879d53	feat(materialize): recover unsaved editor sessions into items.content (TASK-2198 U4) (#1683) ... * feat(materialize): recover unsaved editor sessions into items.content in the background (TASK-2198 U4) Wires the U3 supervisor in. A collab room reclaimed at the end of its grace TTL with content-bearing rows above the flush watermark (T3), or a dormant pending item found by a one-minute sweep (T1), is queued; one worker goroutine materializes the item's whole content-bearing op-log with the whole-workspace link index and lands it through store.MaterializeFlush, under the collab item lock with no room open. The write is a flush: cursor == MAX(op-log id), rows pending, no set-aside rows, checked in the write's own transaction; watermark to the cursor only; seq bumped; one version row attributed to system/recovery; the op-log is never pruned. PAD_MATERIALIZE=off disables it entirely. A per-item in-memory failure budget (3 failures, doubling backoff, reset by a new op-log row) keeps untrusted input from being retried in a loop. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * test(store): recovery watermark is the cursor, not MAX at UPDATE time (TASK-2198 U4) Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * test(store): a second recovery still writes its own version row (TASK-2198 U4) Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * docs(mcp): op-log recovery bounds the stale-body window; tool surface v0.60 (TASK-2198 U4) content_state applied_pending_flush now clears on its own after the item's room closes, unless recovery fails or PAD_MATERIALIZE=off. Every sentence that called the window unbounded, or said nothing server-side moves op-log content into items.content, is corrected (CLAUDE.md, instructions.md, README, docs/deployment.md, docs/backup.md, code comments). BEHAVIOR bump to 0.60; goldens regenerated. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * fix(materialize): count ErrNoMemoryCap silently against the per-item budget (TASK-2198 U4) U3 now refuses every job with ErrNoMemoryCap when no memory cap can be established, and warns once itself. The recovery worker counts it like any failure but logs it at debug, not per item. Its log lines go through a logger seam so the test can tell. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * style: gofmt (TASK-2198 U4) Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * feat(materialize): one log line per budget exhaustion; count set-aside skips on the sweep line (TASK-2198 U4) An item that uses up its failure budget logs exactly one WARN naming the item, the failure count and the last error kind (DEBUG when the kind is no_memory_cap, which the supervisor already warned about once), and logs again only after op-log growth resets the budget and it is exhausted anew. The schema-version skip goes through the same line. Items skipped for set-aside rows are counted and reported on a new single sweep summary line, written only when it has news; never per item. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * fix(materialize): keyset-page the recovery sweep so no candidate starves (TASK-2198 U4, codex r1) The sweep read the oldest 200 candidates from the start every tick, and the room check and the failure budget filtered them afterwards. Once 200 sat in backoff or exhausted, every sweep returned the same 200 and the 201st was never queued: filtering above a bound is starvation. Each tick now reads one page strictly after an in-memory keyset cursor (newest op-log created_at, item id) and wraps to the start after a short page, so every candidate is offered within ceil(N/limit)+1 ticks, whatever sorts ahead of it. Keyset over NOT IN: no parameter-count limit, and it also covers the room check, which an exclusion list would not. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * test(materialize): the sweep cursor wraps; docs for the exhaustion line, the set-aside count and paging (TASK-2198 U4) Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * docs(deployment): reflow the recovery budget paragraph (TASK-2198 U4) Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * test(materialize): recovery strips the follows-title marker as a tab's save does (TASK-2198 U4, BUG-3315) Since #1688 a tab loads a link that follows its target's title with the follows-title marker in the link's title attribute, so the marker lives in Y.Docs that recovery replays. Three fixture cases (a follows-title link; the same link after a rename; a URL edited to an external address) are generated from documents that really hold the marker, and the existing 8 are unchanged. A new leg recovers each through the production path (whole-workspace index) and through the runner with a restricted and an empty index, and asserts no 'pad-follows-title' reaches the body. It goes red in all 7 ways when the bundle's marker strips are disabled. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * test(store): a recovery version row carries #1691's history data (TASK-2198 U4) Rebased onto PLAN-2348 U2 (#1691), which adds user_id, line counts and a create marker to version rows. Recovery writes its row through the same UpdateItem insert, so it gets them automatically: no user_id (a system row), not a create, and line counts equal to diff.LineCounts of the stale and recovered bodies. The leg pins that. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC	2026-09-30 11:03:07 -04:00
xarmian	8ee68e9b4e	test(server): the body-reader accounting asks the type checker, so a request reached through a struct field, context value, alias or embedding is counted (BUG-2820) (#1546) ... TestEveryRequestBodyReaderIsAccountedFor scanned internal/server with a parser that matched the spelling `http.Request` and names bound to it. A request reached through a struct field, a context value, a type alias, an embedded field, an import alias, a dot import, a var declaration, a call-derived alias, a named result or a range binding was invisible, and accounting was per file with a reader count as a workaround. - collectRequestBodyReads loads the package with go/packages and counts every selector whose resolved object is the SAME object that looking the member up on net/http.Request yields (Body, FormValue, PostFormValue, ParseForm, ParseMultipartForm, MultipartForm, PostForm, MultipartReader, FormFile). Accounting is per call site: file::function::member -> count, with the per-file reasons kept. - A fixture package (testdata/bodyreaders) pins the 12 forms the parser could not see, the 9 its own controls held it to, and 6 non-readers, including the two shadowing cases the parser over-flagged on purpose, which are now exact. - New: TestEveryRequestHandoffIsClassified. A request passed to code outside the module must be classified. fosite's five New*Request calls read the OAuth form body (BUG-2811) and are pinned per site; interface dispatch (http.Handler.ServeHTTP) is named as statically unresolvable. - Measured on this tree: the type checker finds exactly the readers the parser found, file for file. The blind spots were real, but nothing used them. - The parser (scanFileForRequestBodyReads) and its controls (TestBodyReaderScanDiscriminates) are retired; their cases live in the fixture. Claude-Session: https://claude.ai/code/session_01A3rXV7GDLAhBe9mL4V6z2m	2026-09-25 14:22:55 -04:00
xarmian	e03ba45b5c	feat(web): push-to-agent composer in the item view (TASK-2561) (#1099) ... * feat(web): push-to-agent composer in the item view (TASK-2561) PLAN-2558 S3 — the web half of IDEA-2544's push-to-harness. Adds `api.items.push`, a new `api.sessions.list`, and a "Push to agent…" row in the item pane's ⋯ menu that opens a small composer. The deliverable is the presence line, not the textarea. `pad push` is fire-and-forget — no durable inbox, no ack, no "nobody was listening" warning — which is defensible for a CLI verb typed by someone who knows their own session is running, and indefensible for a button. So the dialog answers "is anything listening?" before the click, and keeps three states apart rather than two: N > 0 send, worded "N session connected", never "will be delivered" — the registry can name a session that died up to ~30s ago and no push gets a receipt N == 0 send DISABLED. Nothing listening means the message is lost, not queued; the empty state offers the clipboard instead (the fallback S4 rules for quick actions) can't tell send ENABLED, uncertainty stated. A 503/401/network failure is not zero — rendering it as zero is the exact lie handleListSessions returns 503 rather than an empty list to avoid The menu row is gated on a resolved user, not on canEdit: push is self-addressed, so a viewer pushing an item into their own session is a read. Without a user the endpoint 401s. $lib/push/message mirrors the server's rune-after-collapse accounting so an over-length message is caught in the composer instead of coming back as a 400. It deliberately does not use JS `\s`: Go's unicode.IsSpace and `\s` disagree in both directions (U+0085 is whitespace to Go only, U+FEFF to JS only), so a `\s` client under-counts a pasted BOM and over-counts a pasted NEL. The agreement is pinned by a shared fixture (internal/server/testdata/push_message_cases.json) read by BOTH internal/server/push_message_collapse_test.go and the web unit test — a TS-only table would assert a belief about Go rather than Go's behaviour. Claude-Session: https://claude.ai/code/session_01QCMLhHQBrMHVML3YKdm4Cd * fix(web): close the push composer's races and ambiguity gaps (codex review) Round-1 review findings on the S3 composer, all real: - ItemDetail did not reset `pushDialogOpen` on an item switch. The dialog is {#key itemSlug}-remounted while `open` is owned by the parent, so a stale `true` silently REOPENED the composer pointed at the new item. The reset block's existing comment (written for copyDialogOpen) describes this exact failure. Verified live, with the counterfactual: reverting the one-line fix reopens the dialog on item B after a client-side navigation. (The typed draft does NOT carry over — the {#key} remount clears it — so the defect is the silent reopen, not a retargeted message.) - Presence polls shared one generation counter, which fences OPENINGS, not requests. A stalled poll could resolve after a later one and overwrite a fresh count with a stale one, re-arming Push against a session list already known to be empty. Added a per-request sequence; only a strictly newer response is applied. - Nothing bounded a `/sessions` read, and 'checking' disables Push, so a request that never settled stranded the composer with a dead button and no explanation. It now degrades to the honest "can't tell" state after 5s; a later response still lands and upgrades the answer. - A failed send re-armed Push unconditionally. The handler publishes BEFORE writing its response, so an unstructured failure (rejected fetch, non-JSON 502) leaves the outcome genuinely unknown and a second click can deliver the instruction twice on an endpoint with no idempotency key. Split on the same line CopyItemDialog draws (DR-13): a structured PadApiError means the server refused before publishing — re-arm; anything else latches an outcome-unknown state. - `willCollapse` compared against `String.trim()`, reintroducing the very JS-vs-Go whitespace mismatch $lib/push/message exists to avoid (JS trims a leading U+FEFF the server keeps; it leaves a U+0085 the server strips). Added `trimPushMessage`, which trims with Go's class. - The textarea described only the counter, so the collapse note and the over-length error reached no screen reader. Both now live in one stable referenced node that swaps text rather than mounting and unmounting — an aria-describedby pointing at an absent id resolves to nothing. - Positive presence wording implied the count was current. It now says "as of the last check" and names the ~30s window. Test changes: the Go fixture test duplicated `strings.Fields` rather than invoking the handler, so a change to the handler's normalization would have left BOTH suites green — demonstrated by mutating the join separator, which the copied-expression test did not notice and the new handler-driven test caught on 22 cases. The bound is likewise now asserted through the endpoint at 4096/4097 instead of comparing the constant to a copy of itself. Claude-Session: https://claude.ai/code/session_01QCMLhHQBrMHVML3YKdm4Cd * fix(web): fence the push composer against destroyed instances, unrecognised errors, and a frozen count (codex round 2) Three findings, one of them introduced by round 1's own fix: - The send/copy continuation fence used the generation counter, which cannot see a keyed REMOUNT. `{#key itemSlug}` gives item B a fresh instance with its own counter, so item A's in-flight send still saw its own `gen` unchanged and called the SHARED parent `onclose` — closing the composer the user had just opened for B. Added a per-instance `destroyed` flag, which is what actually distinguishes "still mine to close" from "I no longer exist". - The outcome-unknown split treated any PadApiError as proof the server refused before publishing. It isn't: the API client turns EVERY JSON error envelope into one, including a gateway 5xx invented after the handler published. Replaced with a whitelist of codes the handler and its middleware actually emit pre-publish; everything unrecognised is now ambiguous. The asymmetry is deliberate — an unnecessary "we can't tell" costs the user a check, a wrong re-arm delivers twice. - PRESENCE_STALL_MS only rescued the FIRST read. A later poll that hung froze the count at its last value indefinitely while the UI kept rendering "1 session connected" as fact. A known answer now expires to "can't tell" after 30s without a refresh — the server's own presence staleness bound, so past it our answer carries no more authority. Also dropped the status→alert role swap on the composer's live region: changing a live region's role and its text together is not reliably honoured, so the escalation was a promise the markup couldn't keep. The blocking condition rides `aria-invalid` on the textarea instead. The "latest ARRIVED, not latest ISSUED" behaviour of the sequence fence is kept and now documented as a choice: dropping an early-arriving response because a newer request exists strands the UI when that newer request is the one that never settles. Each fix mutation-tested 1:1 against its new test. Claude-Session: https://claude.ai/code/session_01QCMLhHQBrMHVML3YKdm4Cd * fix(web): complete the pre-publish whitelist and retire in-flight polls on expiry (codex round 3) Two of round 3's three findings were real: - `csrf_error` and `email_not_verified` are middleware refusals, written strictly before the handler runs, so they belong in PRE_PUBLISH_ERROR_CODES. Without them a CSRF mismatch told the user we couldn't tell whether their message was sent, when nothing had been. - The 30s staleness expiry didn't fence requests already in flight. A poll issued before the expiry could land after it and reinstate the very count we had just declared too old to trust. Expiry now advances `presenceAppliedSeq` to the current `presenceSeq`, retiring those responses; the poll issued in the same tick carries a newer seq and still applies. The third finding — that `archived` belongs in the whitelist, and that the launcher should be hidden for archived items because "the endpoint always rejects them" — is REFUTED. handlePushToItem has no archived gate (`requireItemVisible` admits archived items), and pushing to an archived item against a running server returns 200 with `pushed: true`. There is no `archived` error code on this path to whitelist, and hiding the launcher would remove a capability that works. Recorded rather than silently skipped so the next reader doesn't re-derive it. Claude-Session: https://claude.ai/code/session_01QCMLhHQBrMHVML3YKdm4Cd	2026-08-15 08:22:09 -04:00