11 Commits

Author	SHA1	Message	Date
xarmian	b2c303c4bb	docs(links,store,models): cite markdown.ts by symbol, and check it (BUG-2832) ... Go comments describe the web renderer constantly and cite it by LINE NUMBER. Nothing verifies those citations — they cross a language boundary, so no compiler, test or linter has ever checked one — and they had drifted onto unrelated code. This converts all 32 to `markdown.ts::symbolName` form and adds the check that makes the conversion worth something. Scope note, because this is wider than the rider it was dispatched as. The BUG-2834 commit added the pattern constant near the top of markdown.ts, shifting the file by +45 lines and invalidating EVERY line citation into it — including the three BUG-2832 had confirmed were still accurate. Leaving 13 knowingly-wrong citations because they sit outside the files this unit otherwise touched is not the neutral option when this branch is what broke them. Happy to split this commit back out if the lead would rather hold the rider to its stated bound. While converting, five of the filing's six "suspect, not established" citations were settled by reading the shifted positions: :307, :478-481, :485, :513 and :516 point at a @param doc line, unescapeDocLinks, REF_PATTERN, the tail of parseCrossWorkspaceBody, and findItemByRef respectively. All substantively stale, not merely off-by-lines. That answers the filing's open question. Two guard tests, per the filing's own proposed fix shape: TestMarkdownCitationsNameLiveSymbols verifies every cited symbol is really declared in markdown.ts. This is the check a line number could never have. TestMarkdownCitationsAreNotLineNumbers bans the line-number form, so the fix cannot erode the next time someone reads a number off their editor gutter. The first version of the symbol check FAILED its negative control and that is the part worth reading. It asked strings.Contains(ts, "function "+sym) — a PREFIX match. Renaming resolveWikiBody to resolveWikiBodyRENAMED leaves "function resolveWikiBody" a substring of the renamed declaration, so the guard stayed green through precisely the rename it exists to catch. It passed its first real run and would have shipped as coverage. Fixed by requiring the following character to be one that cannot continue a JS identifier; the control now fires and names the symbol. Both guards are non-vacuity-asserted: the sweep fails if it finds fewer than 50 Go files, and the symbol check fails if it finds no citations at all. Currently verifying 7 distinct symbols across 29 citation sites. The line-number guard earned its keep before being committed — it caught three citations silently reverted when a file was restored from a snapshot taken before the conversion.	2026-08-31 23:24:27 +00:00
xarmian	6114954236	fix(links,store): decide bracket qualification by the old title (BUG-2830) ... An item whose LITERAL title starts with its own collection's slug plus a slash — say "tasks/Setup" in collection "tasks" — stores an index row byte-identical to a genuinely collection-qualified reference to an item titled "Setup". Both read target_title = "tasks/Setup", collSlug = "tasks". Renaming to "Renamed", the first must become [[Renamed]] and the second [[tasks/Renamed]]. bracketRewriteAt inferred "this was qualified" from targetTitle merely STARTING WITH collSlug + "/", which answers the second case for both. So renaming the literal-titled item emitted [[tasks/Renamed]] — converting a literal-title reference into a qualified one, resolved by a different rule. What that costs depends on what else the workspace holds, and both cases are measured: - With an item LITERALLY titled `tasks/Renamed` (slash included), the link is STOLEN outright. resolveTitleTx tries an exact full-title match before the qualified fallback, so that item wins and the renamed item loses its backlink. Under the unfixed code the decoy gains 1 backlink. Silent retarget, exactly as filed. - With no such item, the emitted bracket still finds the renamed item, because collSlug is that item's own collection and it now carries the new title. The cost there is ambiguity wherever a same-titled sibling exists, plus a later collection move breaking `[[tasks/X]]` where `[[X]]` would have followed. Repro delivered before the fix, as the filing required: RewriteBracketAt("see [[tasks/Setup]] here", 4, "tasks/Setup", "Renamed", "tasks") => "see [[tasks/Renamed]] here" with the correct answer depending on information the function did not have. The discriminator is the renamed item's OLD title, and the cascade has had it all along — cascadeTitleRename takes oldTitle and simply never passed it down. It now rides on TitleEscaper (per-cascade, like everything else there), and qualifiedFor decides by COMPARISON: targetTitle == oldTitle -> literal targetTitle == collSlug + "/" + oldTitle -> qualified neither -> index drift, refuse Literal wins when both could apply, and that is the correct precedence rather than a convenient tiebreak: the renderer's stage 1 beats stage 2, so a row pointing at this item resolved literally. resolveBrokenTitleLinks already makes the same stage-1-over-stage-2 ruling for the same reason — the discriminator existed in the codebase and was thrown away before reaching the rewriter. oldTitle is a required parameter of NewTitleEscaper rather than an optional setter, so a caller that forgets it fails to compile instead of silently getting the old behaviour back. NO byte-length precondition guards the fold comparisons. strings.EqualFold is Unicode simple case folding and case-equivalent strings can differ in byte length — EqualFold("K", "K") (KELVIN SIGN) is true at 1 byte vs 3 — so a length check is not a cheap pre-filter but a strictly narrower predicate, and it made a qualified bracket whose title folds across lengths read as index drift, leaving the link stale. The slug boundary is still located by byte offset, which IS sound: collection slugs are ASCII-lowercase by construction (store.slugify). The frozen pre-refactor oracle is deliberately NOT updated — it is an oracle, not live code. BUG-2830 is added to the named list of intentional divergences from it, and the guarded corpus reaches the new function through v0OldTitle, which states what the old implementation implicitly assumed. Inputs where that assumption was WRONG cannot be produced by the derivation and are pinned by name instead. TestProjectRewrittenLen_IsLockstepWithTheRealPass grew a totalApplied assertion: lockstep is trivially true when both sides refuse everything, and this change makes the rewriter refuse more. It applies 5499 rewrites, so it is measuring something. The codex-R2 overlap fixture was respelled for the same reason — its two unrelated target titles are a shape a real cascade cannot produce, so it would have decayed into two no-ops and lost the regression; `[[A[[A]]]]` reproduces the no-op-then-overlapping-change shape with reachable inputs and asserts it applies exactly one. Negative-controlled four ways: reverting qualifiedFor to the prefix rule kills the case-A regression in both its homes while the twin correctly survives; making it accept drift kills the drift test (added because the first mutation run showed that branch was unreachable by the whole suite); the fold-length regression fails without the EqualFold fix; and passing the WRONG oldTitle at the store call site kills five tests including three pre-existing ones — the binding control, since the unit tests pin qualifiedFor and only that shows the cascade hands it the right value, which was the entire bug. The severity above took two wrong turns before it was measured, and both are recorded in the tests rather than quietly corrected. I first asserted the retarget with a decoy that could not be stolen; then, finding that decoy inert, concluded retargeting was impossible and wrote that into a production doc comment. Neither conclusion came from reading resolveTitleTx — both generalised one fixture's result. The two store fixtures now split along exactly that line and each says which case it pins.	2026-08-31 23:24:27 +00:00
xarmian	b6afbb8fca	fix(links,web): align the JS wiki-link grammar with Go's . (BUG-2834) ... The Go and JS wiki-link patterns were byte-identical source text and did not mean the same thing. Both spelled the escape alternative `\\.`, but Go's RE2 `.` excludes only LF while ECMAScript's also excludes CR, U+2028 and U+2029. A body with a backslash immediately before one of those three was INDEXED by the server and NOT RENDERED by the client: the backlink panel claimed a link the document refused to draw. CRLF line endings make the CR case the plausible one. Measured on both sides before deciding anything, over nine code points. Exactly three diverge; VT, FF and U+0085 agree, which bounds the divergence at precisely ECMAScript's LineTerminator set minus LF rather than leaving it open at "some whitespace controls". Aligns JS UP to Go (`\\[^\n]`) rather than narrowing Go, for three independent reasons: the grammar's other alternative already admits RAW CR/LS/PS in both languages, so narrowing Go would make `[[A<CR>B]]` legal and `[[A\<CR>B]]` illegal; narrowing Go would stop ExtractWikiLinks returning rows it currently returns, and the next reconcile would DELETE them, which is BUG-2805's damage shape; and markdown.ts's own splitWikiBody already treats `\<CR>` as an escape pair, so only its regex disagreed. LF stays excluded on both sides — scanBracketBody depends on that and is untouched. Also collapses the two duplicate regex literals in markdown.ts into one exported WIKI_LINK_PATTERN_SOURCE. Exported as source text, not a RegExp object, because a `/g` regex carries lastIndex and sharing one between a replace() and a test's exec() would couple them through it. The harness is the part meant to outlive the fix. testdata/wiki_grammar_corpus.json is read by BOTH languages, and carries expectations derived from the grammar spec rather than from either implementation — comparing the two implementations to each other would have reproduced the exact blind spot that hid this, since looking identical is what they already did. Pure ASCII with every control character as a \uXXXX escape: an early probe typed U+2028/U+2029 into a shell heredoc, silently lost them, and would have "confirmed" the bug on two cases that were actually spaces. JS assertions are split across the node and jsdom projects because renderMarkdown finishes through DOMPurify and returns '' without a DOM, where it would fail for a reason unrelated to the grammar. Same split, same reason, as the existing markdown.shareAttachments pair; the jsdom file carries a leg proving the DOM path is live, since '' satisfies every not.toContain assertion. Negative-controlled both ways: reverting the JS pattern fails exactly 9 assertions (3 corpus + 3 per call site) and nothing else; narrowing the GO pattern to JS semantics fails the same 3 cases from the other side, so both halves are live instruments rather than tests that cannot fail.	2026-08-31 23:23:56 +00:00
xarmian	69a6ebfe69	fix(links,store): make the rename cascade escape-aware in both directions (BUG-2805) (#1225) ... * fix(links,store): make the rename cascade escape-aware in both directions (BUG-2805) Item titles containing `]`, `|` or `\` did not round-trip through a rename. Repro through the real API on TASK-2826 (Rook) with hex receipts; this fixes both directions plus a third half that repro did not name. Direction 1 — MATCHING. The rewriter compared the RAW bracket body against the unescaped title, so a link stored as `[[Weird \] Title]]` never matched. The link was left naming the OLD title while the reparse flipped its index row to broken: content and index disagreeing, with the staleness visible to the user. Matching now runs on the UNESCAPED body, full-body first and then the split-on-unescaped-pipe form — the same preference order the parser and the renderer use, which is what keeps literal-pipe titles like "A|B" working. Direction 2 — EMISSION. The rewriter emitted the new title by plain concatenation. A title containing `]` produced a bracket the grammar cannot parse, so the reparse found no link and DELETED the index row — permanent damage from an ordinary rename, since a later rename has no row left to cascade. A title containing `\]` produced valid syntax for a DIFFERENT title. Emission now escapes, byte-for-byte the same rule as the editor's escapeWikiBody (markdown.ts:748). Third half, not in the filing or the repro — the close-scan. The rewriter found its closing `]]` with strings.Index, which is not the grammar: in `[[A\]]]` it stops at the `]` belonging to the `\]` escape. That was latent only while the rewriter never emitted escapes; this fix makes escaped bodies routine, so each rename would have corrupted what the last one wrote. scanBracketBody now implements the parser's body production exactly. Allocation discipline carried forward from BUG-2804 R5: escaping happens ONCE per cascade via a TitleEscaper, not per source and not per bracket. A cascade renames one item to one title, but the projection and rewrite run once per source and the scan bound admits very many sources when their titles are short — escaping inside those calls would multiply an unbounded title by an unbounded source count, which is the R5 defect in a new costume. The projection path measures with escapedWikiBodyLen and builds nothing. The frozen V0 oracle is now scoped rather than blanket: it stays authoritative for bodies with no escape characters, and every deliberate departure is listed by name with its reason in TestRewriteBracketAt_IntentionalDivergencesFromV0. One departure is outside BUG-2805's scope and called out there: an empty body `[[]]` is not a link under the parser's `+` production, so the scanner refuses it where V0 would have MINTED a link from a non-link on a drifted offset. Two accidental compatibilities the repro identified as non-reproducing directions are pinned so the fix cannot trade two silent successes for two silent failures: the legacy full-body-with-pipe fallback, and unescapeWikiBody's leniency toward a stray backslash. Instruments: a 4000-iteration property that every emitted title parses back to itself; a scanner/parser parity property; end-to-end cascade tests for both directions including a PERMANENCE check, since a fix that merely delayed the index-row destruction by one rename would otherwise pass. Gates: gofmt, vet, go build ./... clean; go test ./... PASS on SQLite and Postgres; go test -race PASS on internal/links and internal/store; zero failures, zero data races. * test(links): lock the scanner's delimiter edges + correct a stale renderer citation (BUG-2805) Codex R1 named scanBracketBody's edges as a review priority. Probing them against the parser as oracle found no disagreement — all ten shapes agree, including trailing backslash at EOF, `]]]` runs, an escape consuming the would-be close, and the two where BOTH refuse (`[[]]`, and a bare `]` mid-body). Locked in one assertion per shape so a failure names WHICH edge broke rather than printing a random string. Separately, and more substantive: extract.go carried a stale cross-file claim. It said renderMarkdown at markdown.ts:300 uses a simpler `[^\]]+` regex and therefore REJECTS escaped bodies, making escaped links index-only and never clickable. That is no longer true. markdown.ts:300 is inside a doc comment now, and BOTH wiki-link regexes in that file — renderMarkdown at :326 and wikiLinksToMarkdown at :625 — use this package's exact escape-aware production; the renderer comment at :323-325 names BUG-1744 as the change that aligned them. The staleness is load-bearing for BUG-2805 in the direction that makes the bug WORSE: the stale link a rename left behind was a working, clickable link rather than an invisible one. It had also been quoted forward into TASK-2826's repro as a live constraint, which is how a stale comment turns into a shared wrong premise. Verified by reading both regexes rather than by re-citing the comment. * fix(links): refuse empty-title emission; agree with the parser on backslash-LF (BUG-2805, codex R2) Codex R2, triaged per finding with a receipt. The orchestrator did not spot-check these, so each was confirmed or refuted here first. P1 — CONFIRMED as a test weakness, REFUTED as irrecoverable. The added test counted index rows, so it would have passed with the row present and target_item_id cleared. Measured: after a rename to "" the content survives, the row survives, target_item_id goes NULL, and renaming back RESTORES it. So the broken state is correct rather than damage — no title-form link can resolve to an item with no title, and NULL is exactly what a renderer would resolve — and it is recoverable precisely because the CONTENT was never destroyed. The test now asserts the full post-state plus the recovery leg, which is what distinguishes "broken but honest" from "irrecoverable". Separately and genuinely destructive, found while probing the new TitleEscaper's zero value: an empty new title emitted `[[]]`, which is not a link under the parser's `+` production, so the reparse DELETED the row. Reachable two ways — a zero-value escaper, and UpdateItem accepting a rename to "" because the empty-title guard lives only in handleCreateItem. The rewriter now refuses to emit an empty title segment in both the build and the projection path, kept in lockstep. The door-level validation gap is BUG-2833. P2 backslash-LF — CONFIRMED and introduced by this diff. Go's regexp `.` does not match a newline, so `\\.` cannot consume one and the parser rejects the body; scanBracketBody treated backslash-ANY as a pair and accepted it. Now excludes LF only, which is what the Go parser does. Measured both ways before and after. Establishing that surfaced a PRE-EXISTING divergence, filed as BUG-2834: the Go and JS wiki-link regexes are byte-identical source text, but `.` excludes only LF in Go and all four line terminators in JS (measured with node), so Go indexes a backslash-CR link the renderer will not render. P2 qualified-vs-literal — PRE-EXISTING. Byte-identical output between the frozen V0 oracle and current for both slug variants, so this diff neither introduced nor changed it. Matches BUG-2830's mechanism, already filed. P2 cross-workspace parity — PRE-EXISTING. Two receipts: extract.go's diff against main is comment-only, and no line of this diff touches cross-workspace parsing. Gates: gofmt, vet, go build ./... clean; go test ./... PASS on SQLite and Postgres; go test -race PASS on internal/links and internal/store; zero failures, zero data races.	2026-08-31 15:33:40 -04:00
xarmian	21b4d8d9c4	fix(store,links): remove both quadratics in the item rename cascade and bound it (BUG-2804) (#1224) ... * test(store): measure the item-side rename cascade amplification (BUG-2804) Measurement only — no production code changes, no cap. The dispatch is measure-first, and BUG-2798's arithmetic is explicitly non-transferable to this path, so nothing here reuses its numbers. internal/store/items_rename_probe_test.go sweeps three axes with a k=0 negative control and a precondition that fails if the cascade would not be exercised (an empty cascade would otherwise report a flat line and read as "no amplification"). Instrument is MemStats.TotalAlloc, the same counter documents_rename_bounds_test.go uses, so the two paths compare directly. What it establishes, all measured on this machine: - 2.01x the body allocated per bracket, linear across four intervals (1 -> 8 -> 64 -> 512 -> 4096 brackets, agreeing to three significant figures). Bracket count is not independent of body size: the cheapest link is 5 bytes and nothing caps links per item, so the cost is O(C^2) in ONE linker's body, in ONE request, with no accumulation needed. Confirmed directly: doubling C multiplies allocation by 3.83 / 3.92 / 3.95, converging on 4. - 21.3x-22.2x the body per added linker at 8 brackets each, linear in k. - The outbox payload carries every rewritten body in a single row, at exactly 1.00x the body set, persisted. emitBulkItemEventTx documents this as deliberately unbounded in v1 and invites the measurement. Peak residency is reported as a coarse floor and carries no claim. The sampler sleeps between reads deliberately: ReadMemStats stops the world, and a spin loop turned a sub-second cascade into minutes of wall clock on this probe's first run. * test(store,links): decompose the per-bracket constant into its two sites (BUG-2804) Still measurement only. Checkpoint 2 reported a marginal cost of 2.01x the body per bracket without establishing what the 2 was made of. These probes account for it exactly, and the answer changes the fix shape. - internal/links/probe_alloc_test.go isolates RewriteBracketAt: 1.00x the body per call. So the rewriter is only HALF the per-bracket cost. - items_rename_probe_test.go's SELECT probe finds the other half, and it is not in the rewriter at all. cascadeTitleRename's SELECT joins item_wiki_links to items and projects s.content, returning one row per LINK, each carrying a full copy of the source body. The scan loop de-duplicates by source id only AFTER rows.Scan has allocated a fresh string per row. Measured dead flat at 1.00x the body per row against a SINGLE distinct source: 4096 rows on one 256 KiB item allocate 1.07 GB. 1.00 + 1.00 accounts for the measured 2.01. That makes TWO independent quadratics with different fixes. A single-pass rewriter would leave the SELECT's copy untouched and halve the cost rather than remove it. A third probe tests, and REFUTES, an earlier hypothesis that the second copy was the cascade's `rewritten != newContent` full-body comparison: a different-length new title measures 2.05x against the same-length 2.02x, no drop. The comparison allocates nothing, so it could never have appeared in an allocation counter. Kept as a negative result so the refuted reading is not re-derived. * refactor(links): single-pass RewriteBracketsAt, RewriteBracketAt delegates (BUG-2804 M2) The per-bracket decision moves into one unexported helper; RewriteBracketAt becomes the one-element case so the two cannot drift. Differential tests pin behaviour against the PRE-REFACTOR implementation, frozen verbatim in the test file as an oracle. Comparing RewriteBracketAt to RewriteBracketsAt-of-one would be vacuous now that the former is defined as the latter, so the oracle is what makes the test able to fail. * fix(store): bound the item rename cascade and remove both quadratics (BUG-2804) Three mechanisms, landing together because the first two are factors of one product — either alone still leaves O(C^2) while measuring "2x better". M1: cascadeTitleRename's SELECT no longer projects s.content. It returns one row per LINK, so projecting the body made the driver materialise a full copy per bracket, de-duplicated only after rows.Scan had allocated each one. Measured at 1.00x the body per row against a SINGLE source. Each source's content is now read once. M2: the rewrite is a single pass. links.RewriteBracketsAt splices every recorded bracket with one strings.Builder; RewriteBracketAt is reimplemented as the one-element case so the per-bracket decision has exactly one copy and the public contract is unchanged. M3: MaxItemRenameCascadeBytes (64 MiB) bounds the total linking-item content one rename may process, charged before each body is built so a refusal never allocates the thing it refuses. The number is derived from the live workspace's measured distribution, not inherited from documents.go, and the receipt is in the constant's doc comment. Measured, before -> after: doubling C 3.83/3.92/3.95x -> 2.04/2.21/2.06x (quadratic gone) per bracket 2.01x body -> 0.01x body per linker 21.3x body -> 7.3x body 4096-row SELECT 1,074,850,200 B -> 1,180,856 B (910x) store suite 180.9s -> 86.8s Equivalence is pinned against the PRE-REFACTOR implementation, frozen in the test file as an oracle, over a hand-enumerated behaviour corpus plus 20,000 randomised inputs; and the single pass is pinned against the descending fold the cascade used to perform, over 5,000 randomised inputs with shuffled, duplicated and corrupted offsets. The k-linear retention that remains is in the outbox member snapshots, filed separately as BUG-2827. * fix(store,server): codex R1 findings + a dialect-dependent test instrument (BUG-2804) P1 — ProjectRewrittenLen built a replacement string per bracket to measure it, reintroducing allocate-then-refuse one layer below the cap it feeds. The per-bracket helper now returns the SEGMENTS and never concatenates; projection does length arithmetic only, and the rewrite writes the parts straight into its builder, removing an allocation from the hot path too. P2 — the item rename cascade's refusal had no errors.As arm in handleUpdateItem, so a deliberate, permanent decline reached clients as a 500 implying a retry might help. Now 413 rename_cascade_too_large, composed from the error's typed fields so the internal call path is not published. Pinned by an exact-status test plus a counterfactual that a blanket-413 handler would fail. Checked the sibling write paths rather than assuming the reported one was the population: bulk update and version restore never set Title, so the cascade cannot fire there and this handler is the whole surface. P2 — `applied` counted brackets MATCHED, not brackets CHANGED, so a rewrite reproducing content byte-for-byte would still write the row, bump seq and emit events. Counting changes restores what the pre-fix whole-body comparison did. Cascade reachability is NOT established and the test says so. Separately, found by make test-pg: the refuse-before-building guard used a TotalAlloc ceiling, and the same refusal allocates ~69 MB on SQLite against ~242 MB on Postgres — the ceiling was measuring the driver. Replaced with an exact count of bodies built via a store test seam. The first version of that seam ALSO failed to discriminate: a mutation moving the cap check between the build and the seam call went undetected, so the build and the count are now one function with no third position for the defect to hide in. Gates on this tree: gofmt, go vet, go build ./... clean; go test ./... PASS on SQLite AND on Postgres (own container, private port 5446 — never the shared 5445, whose teardown would kill a sibling seat's container mid-run). * fix(links,server): codex R2 findings (BUG-2804) R2-1 — ProjectRewrittenLen advanced its cursor past a no-op bracket while RewriteBracketsAt did not, so the two disagreed about which overlapping rewrites the guard skips. On `[[A[[B]]]]` with a no-op at 0 and a change at 3, projection reported 10 bytes / 0 applied while the pass produced 13 / 1 — the cascade charged the read and never charged the rewrite it then performed, so the bound leaked on exactly the corrupt and duplicated offsets the defensive paths exist for. Projection now follows the pass, not the reverse: the pass's behaviour is pinned to the descending fold the cascade used to perform, so moving it would have changed cascade semantics under cover of a bug fix. The lockstep property test that pins this needed MIXED per-position target titles — the first version shared one title across a call and passed against the broken code, because the reproducing shape needs a no-op bracket overlapping a changing one, which a shared title cannot express. Cascade rows carry per-row target_title, so mixed is the realistic case. R2-2 — the 413 mapping covered one of THREE places handleUpdateItem reaches UpdateItemWithParentLink. The post-R1 population sweep asked whether other HANDLERS reach the store call and never asked whether this handler reaches it more than once. All three now share writeItemRenameCascadeTooLarge. The collab-edit half had a deeper cause than a missing arm. In applyContentViaCollabOnce's prune-and-direct-write fallback, a failing directWrite() had its error DISCARDED and the original collab error returned in its place, so the caller read a deterministic refusal as a recoverable routing problem and fell through to its own direct write — re-deriving the identical refusal from scratch. Measured 64 rewritten bodies built for one request against an expected 32. Deterministic failures now survive that branch, scoped to the three errors every call site already treats as final (open-children rejection, update conflict, cascade refusal). Everything else still returns the collab error, preserving the graceful-degradation contract the branch exists for. Status alone cannot discriminate the double-work fix — both behaviours end in 413, since the fall-through reaches the plain path's arm — so the store's build observer is now reachable from the server package via an exported test-support setter, and the test asserts the work count. Gates: gofmt, vet, go build ./... clean; go test ./... PASS on SQLite and Postgres; go test -race PASS on internal/links and internal/store with zero data races. Note -race on store runs 705s, past the 600s default timeout. * test(links): make the multi-bracket fold oracle independent (BUG-2804, codex R3) The descending-fold equivalence test built its oracle by folding RewriteBracketAt — which now delegates to RewriteBracketsAt, the function under test. The comparison was therefore circular: it could agree with itself while both diverged from the pre-refactor behaviour it exists to pin. This is the same vacuity I flagged and avoided for the ONE-element assertion, missed one case over in the multi-bracket one. The fold now uses rewriteBracketAtV0, the already-frozen pre-refactor primitive, which is the only genuinely independent oracle available. The test still passes, so the single pass does match true pre-refactor behaviour on every generated input including the overlapping ones — that agreement is now evidence rather than a tautology. Mutation-verified after the change: the fold test kills the dropped-sort and dropped-overlap-guard mutants, two of which it had previously left to other tests. Test-only. `git diff 8ddfdfd6 -- . ':!*_test.go'` is empty, so the Postgres and store -race results recorded for 8ddfdfd6 carry over unchanged. * fix(store): bound the cascade SCAN, not just the rewrite loop (BUG-2804, codex R4) `works` holds one entry per matching LINK ROW, each carrying that row's target_title, and item titles have no length bound. So the scan retained rows x title bytes before the per-source loop fired, and the content-bytes cap could not see it. The doc comment on MaxItemRenameCascadeBytes claimed the opposite — that the cascade's retention was O(1) in the linker count, with the k-linear remainder attributed to the outbox (BUG-2827). Both halves were wrong: the outbox is a different vector, and this function had k-linear retention of its own. That comment was mine, and it is the claim this unit is named after. The scan is now charged against the same budget and refuses DURING the scan, so at the moment of refusal the process holds only rows already counted. One budget covers both phases deliberately: index rows and content bytes are different quantities, but both are memory this one rename makes the server hold, and a second constant would be a second thing to tune with no separate meaning. Measured: refusal fires at 67,113,728 bytes against the 67,108,864 cap — 4,864 past crossing — with zero bodies built, on a fixture carrying 64 rows of 2 MiB titles whose full scan would have reached ~134 MB. Three test corrections, all mine: - The attack I first designed is impossible. A link to a T-byte title costs T bytes of body text, so content and retained-title bytes are COUPLED and content is always larger. The fixture guard caught it. The property that separates the two halves is WHEN the refusal fires, not size. - Two tests pinned an exact admitted count, which the new scan charge shifts by a source; one of them also used a body size that divided the cap evenly, so refuse-after-build totalled exactly the cap and could not have been detected. Both now assert that the work actually done fits the budget. - The scan test is SQLite-scoped with a measured reason: on Postgres the huge-title fixture cannot be built at all, because items carries UNIQUE(workspace_id, slug), the slug is derived from the title without truncation, and the btree tuple limit is 8191 bytes. Filed separately as BUG-2831. The bound itself is NOT dialect-specific — row count is unbounded on both backends. Gates: gofmt, vet, go build ./... clean; go test ./... PASS on SQLite and Postgres; go test -race PASS on internal/links and internal/store; zero failures, zero data races. * perf(links): never build the replacement in the projection path (BUG-2804, codex R5) bracketRewriteAt composed `collSlug + "/" + newTitle` BEFORE the match check, so every bracket paid for it including the leave-alone exit — and ProjectRewrittenLen calls it once per rewrite BEFORE the cascade's cap can fire. R1's shape, one layer down. It does not document away. BUG-2831 established that item titles carry no validation bound, so a multi-megabyte newTitle is admissible and a bracket-dense body projected gigabytes of immediately-discarded allocation ahead of the refusal that exists to prevent exactly that. The helper now MATCHES FIRST and returns a DESCRIPTION — a `qualified` flag for the slug prefix plus a display-suffix slice of the body — so nothing proportional to newTitle is built anywhere in the projection path. The real pass writes "[[", collSlug, "/", newTitle, suffix, "]]" as separate segments. bracketUnchanged compares segment-wise rather than concatenating a candidate, which would have reintroduced the same allocation to answer a comparison. The `ttLower + "|"` concatenation is gone too, spelled out as the same predicate on the same two already-lowered strings. Measured, 4096 brackets against a 1 MiB newTitle: pre-R5 shape 4,328,782,464 bytes 1.0079x newTitle per bracket now 196,704 bytes 0.0000x The second row is a new probe; the first is that probe run against a mutant carrying the old shape, not arithmetic. No semantic change, and the frozen pre-refactor oracle is what establishes that: the full differential suite passes, including 20,000 randomised inputs and the projection/pass lockstep property. The slice-growth transient stays as documented. Gates: gofmt, vet, go build ./... clean; go test ./... PASS on SQLite and Postgres; go test -race PASS on internal/links and internal/store; zero failures, zero data races.	2026-08-31 12:59:14 -04:00
xarmian	cade263fe5	fix(store): compare-and-set the wiki-link cascade's writes (BUG-2785) (#1213) ... * fix(store): compare-and-set the wiki-link cascade's writes (BUG-2785) A document rename cascades through every document linking the old title as a read-modify-write across two statements: SELECT each linker's content, rewrite the string in Go, UPDATE the row. A content edit to a linker committing between those two statements was silently overwritten — the cascade wrote the body it built from the version it read, with no error and no version row for the loss. Same lost-update shape as BUG-2770's activity-metadata merge, one table over, and fixed the same way: the UPDATE now carries `content = ?` with the body the cascade read, plus a bounded retry that re-reads the row and re-applies the rewrite. DIALECT SCOPE, which decides what the tests can prove. Reachable on POSTGRES only. SQLite's DSN sets `_txlock=immediate`, so UpdateDocument's db.Begin() takes the write lock at BEGIN and holds it across the whole read→write window; a concurrent edit cannot commit inside it and serializes on busy_timeout instead. On Postgres under READ COMMITTED each statement takes a fresh snapshot and the stale body wins. The CAS is therefore a no-op on SQLite by construction — the predicate always matches, because nobody else can have written. Two consequences, both acted on rather than noted: the new tests SKIP loudly on SQLite instead of passing for a reason unrelated to this fix, and the mutation matrix was run under Postgres, where removing the CAS leaves a SQLite suite entirely green. The retry rewrites the WINNER's body rather than replaying the original rewrite — replaying it would reintroduce exactly the text this bug loses. A mutation that replays instead is in the matrix. THE ZERO-ROW RESULT NEEDS A PROBE. The UPDATE now has two predicates that can each refuse it, and RowsAffected cannot say which did: `deleted_at IS NULL` (the linker was archived — a documented normal outcome, stop) or `content = ?` (a concurrent edit landed — re-read and retry). Treating them alike either retries forever against a deleted row or discards a live linker's rewrite, so a probe distinguishes them, as BUG-2770 needed for the same reason. The probe reads through tx, never the pool (BUG-2409). On retry exhaustion the RENAME fails rather than leaving one linker holding a title that no longer exists. The alternative — log and continue — was considered and rejected: it trades a loud retryable failure for a silent inconsistency, and a rename is atomic in intent. Exhausting three attempts needs three consecutive commits to the same linker inside one cascade. Adds afterLinkCascadeRead, the seam between the cascade's read and its writes. No existing seam reaches that gap: afterDocumentPreLockRead fires before the transaction, afterDocumentPreWrite before the renamed document's own update. That new seam also closes a gap a previous unit recorded as permanently open: the `deleted_at IS NULL` guard carried a note calling itself UNTESTED because reaching its window "would cost a fifth seam". This is that fifth seam, so the note is removed and replaced by a record of the closure, and the archived-linker test now drives exactly that window. Mutation matrix, run under Postgres — 5 mutants, 5 detected, including one that is literally the pre-fix code: M1 CAS predicate removed (the unfixed behaviour) → lost-update test M2 soft-delete probe arm removed → archived-linker test M3 retry budget cut to one attempt → lost-update test M4 retry replays the original rewrite → lost-update test M5 seam removed (control: tests must notice they never raced) → lost-update test Gates: gofmt clean, lint 0 issues, govulncheck clean, full suite green on SQLite (28 pkgs) and on Postgres 17 (28 pkgs, internal/store 346s vs 79s — the positive control that the PG legs ran rather than skipped). Not fixed here, filed as BUG-2795: cascadeTitleRename, the ITEM-side cascade, has the identical defect and no lock closes its window either. Its fix does not transfer — it rewrites by POSITION from item_wiki_links offsets, so a retry must re-derive positions from the winner's body and re-run replaceWikiLinks, which is a redesign of the retry unit rather than a predicate on an UPDATE. wiki_links.go's claim that it "matches the document rename behavior" is corrected to say which half no longer matches. * style(store): gofmt the CONVE-23 sweep comment (BUG-2785) Same failure as the previous unit, same cause, and worth naming rather than quietly fixing: gofmt wants a blank line between list items once one item grows a second paragraph, which the note about the previously- untested deleted_at guard made true. I re-ran build and the full Postgres suite after those comment edits but not lint, because the change was 'only a comment' — the exact reasoning the previous unit's fix commit warned about in writing, one unit earlier. The PR body's claim that all gates were re-run after the prose edits was false and has been corrected there too. The rule as remembered does not work. The mechanical form does: gofmt and lint are the last action before a push, comment-only changes included. * fix(links,server): stop a rename hanging the server, and report cascade contention honestly (BUG-2785) Three findings from Codex round 2 on this PR. The first is a server hang. 1. ReplaceTitle could never terminate. replaceAll looped "find old in result, splice new in" — re-searching the string it was building, including the text it had just inserted. When the NEW title contains the OLD link token it grows without bound. Measured, not argued: ReplaceTitle("x [[A]] y", "A", "A]] [[A") builds `[[A]] [[A]]`, which still contains `[[A]]`; a probe against the old implementation ran 3s without terminating before being killed. Document titles have no validation, so this is reachable from user input — and the caller is inside the rename transaction holding the workspace rename advisory lock (BUG-2778), so the hang would take every other rename in that workspace down with it while exhausting memory. strings.Replace with n = -1 has the semantics that were wanted: non-overlapping, left-to-right, over the input. Three-case regression test, all three of which fail against the old implementation, plus a control that catches a "fix" which terminates by doing nothing. Pre-existing, and folded in rather than filed: three lines against a server hang, and this PR's retry calls the helper again per attempt, which makes it reachable more often than before. 2. Retry exhaustion surfaced as an opaque 500. The rename rolls back cleanly and retrying can succeed, so "an internal error occurred" tells the caller the opposite of the truth. Adds the exported ErrLinkCascadeContention sentinel; the handler now answers 503 lock_contention with Retry-After, reusing the disposition BUG-2778 already established for 55P03/40P01. 3. That 503's message claimed the workspace was "busy with another rename". 55P03 there is just as likely to be an ordinary content edit holding the row, and the new arm is definitely one. It no longer names a cause the server has not established. Also closes the coverage gap round 2 named around this unit's own decision: exhaustion now has a test asserting the rename ROLLS BACK (target keeps its title) and that the concurrent editor's text survives that rollback. cascadeRewriteAttempts becomes a var so the test can force exhaustion at 1 rather than arranging three consecutive commits, which would need a per-attempt hook in production code — the divergence from its const sibling is noted where it lives. Records two limitations in the code rather than leaving "the cascade is safe now" to rot: the MIRROR direction is still open (a content writer that read before this transaction can commit afterwards and reinstate the old title — fixing it means giving ordinary content writes a CAS too), and delete-then-restore of a linker mid-rename brings back the old title. Both pre-existing, neither worsened here. Mutation matrix now 7 mutants, 7 detected, run under Postgres: M6 (%w -> %v, sentinel lost) and M7 (exhaustion swallowed) cover the new mechanisms. Gates re-run on the tree being pushed, tests included, after the final comment edit rather than before it: gofmt clean, lint 0 issues, SQLite 28/28, Postgres 17 28/28 (internal/store 345s vs 79s). gofmt caught an unformatted test file locally this time, which is the point. * fix(store,links): correct three prose claims and close the SQLite coverage gap (BUG-2785) Codex round 4, on SQLite semantics and prose accuracy. One finding says a bug I FILED is wrong; that is the important one. 1. BUG-2795's premise was false, and this PR repeated it in a comment. I filed that item claiming the item-side cascade has "the identical defect" and that "no lock closes its window either", on the strength of a grep for pg_advisory_xact_lock in items.go that turned up only the parent-link locks. It missed acquireWorkspaceSeqLock (items.go:2136), taken UNCONDITIONALLY by every UpdateItem — content-only edits included — immediately after Begin and long before cascadeTitleRename, and held to COMMIT. So two item updates in a workspace fully serialize on Postgres and an ordinary content edit CANNOT commit inside that cascade's window. The scenario I filed is not reachable. Not fully invalid: sweeping every `SET content` writer in internal/store finds RemapAttachmentReferencesInWorkspace, which rewrites items.content in its own transaction without that lock. So a real but far narrower window survives — attachment remap versus cascade, not user-edit versus cascade. BUG-2795 corrected on its trail and dropped to low; the comment here now states the lock, the one surviving writer, and stops claiming parity with the document cascade. I searched for the locks I expected rather than for what serializes that path, then wrote a sentence broader than the search. Same failure this PR's review has produced repeatedly. 2. "A title nobody should be able to write" was false — the document API validates doc_type and status, never the title. Correcting it surfaced a real second defect: renaming to `A]] [[A` now terminates (round 2's fix) but writes `[[A]] [[A]]`, two links to nothing. Filed as BUG-2796. The termination test deliberately still asserts the COUNT rather than the output, so it does not freeze today's broken rendering as intended behaviour. 3. The hang's blast-radius claim named the workspace rename advisory lock without qualifying the dialect. That lock is a no-op on SQLite, where the equivalent damage is the database-wide write lock the transaction already holds under BEGIN IMMEDIATE. Different mechanism, same outcome for everyone else. Also closes the last coverage gap round 2 named. The CAS predicate runs on SQLite in production and every concurrency test skips there, so TestUpdateDocument_CascadeRewritesEveryLinkOnBothDialects does not skip: two linkers, multiple links per body, plus a document that merely contains the word and must be left alone. Verified it earns its place — a mutant comparing against the rewritten body instead of the body that was read compiles, and dies to this test ON SQLITE, where nothing else would have caught it. Documents the parallel-test constraint on the now-mutable cascadeRewriteAttempts, with its boundary: no current t.Parallel test in internal/store reaches this cascade, but that is a fact about today's corpus rather than an invariant. Gates on the pushed tree: gofmt clean, lint 0 issues, govulncheck clean, SQLite 28/28, Postgres 17 28/28 (internal/store 349s vs 79s). * docs(store,links): point the cascade comment at the root cause, qualify a claim in its second location (BUG-2785) Codex round 5, probing cross-connection visibility and transaction boundaries. Two findings, both mine, neither changing behaviour. 1. The "one surviving writer" framing was too comfortable, twice over. RemapAttachmentReferencesInWorkspace is not a rare non-interactive writer: bundle import reaches it on an ordinary user-triggered import, and the workspace is ALREADY VISIBLE to its owner while it runs — store.ImportWorkspace commits the workspace row (with owner_id) in its own transaction before opening the one that inserts items, and the bundle handler runs the remap as Phase 3 afterwards. Both verified in code. So that writer races ordinary item edits, not just the rename cascade, and it is missing a guard outright rather than being an exotic pairing. Filed as BUG-2797, which covers the remap itself; BUG-2795 is now a consequence of it and says so on its trail. The comment here points at the root cause instead of implying the cascade's pairing is the whole story. 2. The dialect-unqualified advisory-lock claim survived in a SECOND location. Round 4 caught it in links.go and I fixed it there; the same sentence sat in the termination test's comment, and I never enumerated the sites. That is CONVE-23's verify half failing exactly as it warns: I fixed the instance I was shown rather than the population. Both now qualified, and a sweep for remaining unqualified copies leaves only BUG-2778's own comment, which already carries its no-op-on-SQLite note. Worth recording that this is the second time on BUG-2795 that a scope sentence of mine ran ahead of the sweep supporting it — first "no lock closes its window either" (acquireWorkspaceSeqLock did), now "not user triggered" (import is). Both found by a review round rather than by me. Gates on the pushed tree: gofmt clean, lint 0 issues, SQLite 28/28, Postgres 17 28/28 (internal/store 348s vs 79s).	2026-08-26 22:04:15 -04:00
xarmian	905876af04	feat(backlinks): cross-workspace wiki-links + request-independent ACL (Phase 2b) (#622) ... * feat(backlinks): cross-workspace wiki-links + request-independent ACL (Phase 2b) Phase 2b of PLAN-1593 (TASK-1597). Completes the wiki-link reverse index by indexing and surfacing `[[workspace::REF]]` cross-workspace references. Builds on Phase 2a's title work (PR #621). Phase 3 (TASK-1596) owns the UI/MCP/CLI rendering changes. What changed - internal/store/backlinks_visibility.go (new): request-independent ACL helper `Store.ResolveBacklinksVisibility(userID, workspaceID, includeDeletedItems)`. Mirrors the role-determination + collection- merge logic from server.guestResourceFilterCore but doesn't depend on a request context, so cross-ws traversal can compute per-source- workspace ACLs without a `workspaceRole(r)` lookup. The Codex planning-round review caught the prior plan reusing the request- scoped helper as a hidden architectural cost; this is the resolution. - internal/server/server.go: guestResourceFilterCore refactored to delegate to the new store helper. Keeps the request-scoped wrapper signature stable for all existing handler call sites; only the internals move. - internal/links/extract.go: lift the Phase-2a workspace_ref emit gate. WikiLinkKindWorkspaceRef now flows through ExtractWikiLinks alongside ref and title kinds. parseBody recognition was already in place from earlier rounds. - internal/store/wiki_links.go: WikiLinkKindWorkspaceRef branch in replaceWikiLinks stores (target_workspace_id, target_ref) verbatim, resolving the slug→ID via new resolveWorkspaceSlugTx (with per-call cache so repeated `[[ws::X]]` in one body don't re-query). Unknown slugs persist with target_workspace_id=NULL — broken-link semantics, identical to existing ref/title patterns. - internal/store/wiki_links.go: new `Store.GetCrossWorkspaceBacklinks` enumerates accessible workspaces via Store.GetUserWorkspaces (which includes guest-only access — broader than membership query), then per-workspace computes visibility via ResolveBacklinksVisibility and runs the SQL backlinks query with the per-ws (FullCollectionIDs, GrantedItemIDs) predicate inline. Results sorted by updated_at DESC in Go, paginated globally. Per-workspace safety cap (offset+limit) prevents one workspace from dominating the global slice. - internal/store/wiki_links.go: new `Store.CountBacklinks` for same-ws pagination boundary detection. Needed so the handler knows where the cross-ws tier begins for pages 2+. - internal/models/backlink.go: new `SourceWorkspaceSlug string` (omitempty) field. Populated only by cross-ws rows; same-ws rows leave it empty so the existing wire shape is preserved. - internal/server/handlers_backlinks.go: union pagination across same-ws and cross-ws tiers. Same-ws first (matches the renderer's UI mental model — your own workspace's links at the top of the panel). Count-based slice math handles pages 2+ correctly when same-ws is exhausted. Tests - internal/links/extract_test.go: workspace_ref forms emit correctly (bare, display alias, mixed case, invalid-slug fallback to title). - internal/store/wiki_links_xws_test.go (new): six cross-ws scenarios plus a role-matrix test: - end-to-end cross-ws index + query - non-member sees nothing - guest with collection grant sees only that collection - guest with item grant sees only the granted item - unknown workspace slug → broken row, no query results - same-ws rows leave SourceWorkspaceSlug empty - ResolveBacklinksVisibility role matrix (admin/full member/guest with grants/non-member non-grant) Out of scope (Phase 3 / TASK-1596) UI rendering of cross-ws backlinks (workspace badge + workspace- prefixed ref), MCP `pad_item.action: backlinks` cross-ws fields, CLI display tweaks. PLAN-1593 / TASK-1597. * fix(backlinks): admin enumeration + cross-prefix ref fallback + unbounded perWsCap (Codex round 1) Three P2 findings from Codex round 1 against PR #622: Finding 1 — admin users miss cross-ws backlinks. `GetUserWorkspaces` returns only memberships + grant-only guest workspaces, but RequireWorkspaceAccess (middleware_auth.go:481) gives admins implicit access to every workspace. An admin querying for backlinks would silently miss links from workspaces they're not explicitly a member of. Fix: in GetCrossWorkspaceBacklinks, branch on user.Role: - admin → s.ListWorkspaces() (every non-deleted workspace) - non-admin → s.GetUserWorkspaces (memberships + grants) Stale user IDs return empty result rather than erroring. Finding 2 — cross-ws ref matching doesn't handle cross-prefix moves. Same-ws is immune because target_item_id is resolved at parse time and survives renames/moves; cross-ws resolves at query time, so a `[[other-ws::OLD-42]]` row written before the target moved from OLD→NEW collection wouldn't match a query under the NEW ref. Fix: in queryCrossWorkspaceBacklinksForWorkspace, dual ref-match clause: exact `LOWER(wl.target_ref) = LOWER(?)` OR `LOWER(wl.target_ref) LIKE LOWER('%-N')` where N is the item_number from the target ref. Pad prefixes are alphanumeric with no internal `-`, so trailing `-N` uniquely identifies the number suffix — no false positives like "TASK-142" matching "%-42" (LIKE anchors to the trailing literal). Finding 3 — per-workspace cap of 1000 silently broke pagination beyond offset>=1000. The 1000 ceiling was defensive paranoia; the correct math is offset+limit per workspace (worst case all rows come from one workspace and the global slice still needs that many). Fix: drop the 1000 ceiling. perWsCap = offset+limit unconditionally. For runaway offsets the per-workspace transfer cost is proportional; documented as a known characteristic (callers shouldn't be paging past offset=10000 anyway). Regression tests: - TestWikiLinks_CrossWorkspaceAdminSeesAllWorkspaces: admin sees cross-ws backlink without being a workspace member. - TestWikiLinks_CrossWorkspaceRefNumberFallback: move target to new collection, query under new ref, old-ref-stored row still surfaces. PLAN-1593 / TASK-1597. * fix(backlinks): honor OAuth/MCP token workspace allow-list (Codex round 2) Codex round 2 P1: cross-workspace backlinks bypassed the OAuth/MCP token's workspace allow-list (TASK-952). A token consented for workspace A but with the underlying user having access to B would still surface source rows from B via the cross-ws query — leaking data outside the token's consent scope. Fix: thread `allowedWorkspaceSlugs []string` through GetCrossWorkspaceBacklinks. Handler populates it from TokenAllowedWorkspacesFromContext(r.Context()): - nil → no token gate (PAT or pre-TASK-952 token, allow all) - "*" wildcard → allow all - explicit list → strict slug membership Workspace enumeration skips any source workspace whose slug isn't in the allowlist. The same-ws path is unchanged because RequireWorkspaceAccess already gated the target workspace against the allow-list (so we only reach this handler when the target IS in the list). Regression test in wiki_links_xws_test.go covers four shapes: nil, wildcard, target-only (blocks cross-ws), explicit source-workspace (allows cross-ws). PLAN-1593 / TASK-1597. * fix(backlinks): normalize limit at handler boundary (Codex round 3) Codex round 3 P2: the backlinks handler parsed ?limit=N but didn't normalize it before computing the same-ws/cross-ws pagination split. GetBacklinks and GetCrossWorkspaceBacklinks each clamp >300 internally, but the handler's 'remaining := limit - len(sameWs)' used the original (potentially huge) value. With ?limit=301 and more than 50 same-ws backlinks, the first page would mix cross-ws in before same-ws was exhausted, violating the documented tier order. Fix: clamp 'limit' to <=300 at the handler boundary, before any pagination math runs. PLAN-1593 / TASK-1597. * fix(backlinks): normalize same-workspace [[ws::REF]] to ref-kind (Codex round 4) Codex round 4 P2: `[[<current-ws>::TASK-1]]` was being indexed as a workspace_ref row with target_workspace_id = current workspace. But the same-ws GetBacklinks query requires target_item_id (workspace_ref rows leave it NULL), AND GetCrossWorkspaceBacklinks explicitly skips the target workspace — so the link rendered and navigated correctly in the UI but no backlink ever surfaced. The renderer's L307 short-circuits same-workspace fully-qualified form to behave identically to `[[REF]]`; the index must follow. Fix: in replaceWikiLinks, normalize a workspace_ref link to ref-kind when its slug resolves to the current workspace. The promotion canonicalizes the ref (via new links.CanonicalizeRef exported alias) so `[[ws::task-5]]` stores the same canonical shape as `[[TASK-5]]`. Tests: - TestWikiLinks_CrossWorkspaceSameWorkspaceQualifiedNormalized: same-ws fully-qualified `[[ws::REF]]` surfaces in same-ws backlinks and is absent from cross-ws backlinks. PLAN-1593 / TASK-1597. * fix(backlinks): same-ws qualified ref miss doesn't title-fallback (Codex round 5) Codex round 5 P2: my round-4 normalization was too aggressive. It promoted `[[<current-ws>::REF]]` to ref-kind and let the regular ref branch handle it — including the title-fallback path that runs on ref miss. But the renderer's same-ws qualified branch (markdown.ts:472-481) does NOT title-fallback: a ref miss in that path returns the wiki-link verbatim (broken). Only the bare `[[REF]]` path (markdown.ts:513) falls through to title lookup. So my normalization could create ghost backlinks for source bodies like `[[ws::ISO-9001]]` when an item titled "ISO-9001" exists but no ISO collection — the renderer renders broken text, but the index would point at the title-matching item. Fix: handle same-ws qualified refs inline at the top of the loop, BEFORE the switch dispatches. Insert as ref-kind row (resolved or NULL) and `continue` past the switch. Bypasses the title-fallback path entirely, mirroring the renderer's behavior. Regression test in wiki_links_xws_test.go pairs same-ws qualified miss (must NOT title-fallback) with bare ref miss (SHOULD title-fallback) to lock the asymmetry in. PLAN-1593 / TASK-1597.	2026-05-24 13:27:40 -04:00
xarmian	c67c167c43	feat(backlinks): title-form wiki-links + rename cascade (Phase 2a) (#621) ... * feat(backlinks): title-form wiki-links + rename cascade (Phase 2a) Phase 2a of PLAN-1593 (TASK-1595). Extends the server-side wiki-link reverse index from Phase 1's `[[REF-N]]` coverage to also handle `[[Title]]` and `[[collection/Title]]`. Cross-workspace `[[ws::REF]]` forms stay gated until TASK-1597 (Phase 2b) ships the request- independent ACL helper. What changed - internal/links/extract.go: lift the Phase-1 emit gate for WikiLinkKindTitle; keep WikiLinkKindWorkspaceRef gated. - internal/store/wiki_links.go: title branch in replaceWikiLinks (verbatim target_title storage + case-insensitive resolution), resolveTitleTx with full-key match first / `/`-split fallback (mirrors renderer order — Codex review caught the inverse on the planning round), cascadeTitleRename + resolveBrokenTitleLinks. - internal/store/items.go: rename cascade fires in-tx on title change; create path flips pre-existing broken title rows. - internal/store/migrations/062 + pgmigrations/041: one-shot `DELETE FROM item_wiki_links` so the startup backfill repopulates with the Phase 2a vocabulary. - Tests cover title round-trip, case-insensitive resolution, broken- link persistence + later resolution, full rename cascade with content rewrite, collection-qualified form, full-key-beats-split precedence (regresses Codex finding #3), broken-row flip on rename, no-op rename, and self-reference filtering. PLAN-1593 / TASK-1595. * fix(backlinks): rename cascade preserves display aliases (Codex round 1) Codex round 1 against PR #621 caught: the title-rename cascade selects sources via target_item_id (correctly hitting all rows that resolve to the renamed item — including aliased and mixed-case shapes), but the literal `strings.ReplaceAll` rewrite step only matched `[[Old Title]]` / `[[<slug>/Old Title]]`. Rows from `[[Old Title|alias]]`, `[[old title]]` (mixed case), or `[[<slug>/Old Title|alias]]` slipped past the rewrite; the trailing replaceWikiLinks re-parse then saw the same body, failed to resolve under the new title, and converted the row to broken — exactly the regression the cascade exists to prevent. Fix: new internal/links/RewriteWikiTitle helper handles all four title-form shapes with case-insensitive title matching and verbatim display-alias preservation. Cascade swaps from ReplaceAll to this helper. Unit tests in links_test.go cover the matrix; integration test in wiki_links_test.go regresses the original failure mode by mixing all four shapes in one source and asserting all four stay resolved after rename. Known limitation documented in the helper: titles containing wiki- link escape characters (`]`, `|`, `\`) — stored escaped in source content — don't match the regex's literal old-title segment. Same limitation exists in the legacy ReplaceTitle helper; promotable if a real user hits it. PLAN-1593 / TASK-1595. * fix(backlinks): retarget qualified-fallback rows on literal-title arrival (Codex round 2) Codex round 2 against PR #621 (P2): resolveBrokenTitleLinks only flipped target_item_id IS NULL rows, missing the arrival-order case where a literal `[[tasks/Setup]]` should win stage 1 over a row previously resolved via stage 2 (qualified fallback to item "Setup" in collection "tasks"). The index would stay stale until the source's content was rewritten — a latent inconsistency between the persisted backlink and what the renderer would actually show. Fix: drop the IS NULL constraint on the stage-1 UPDATE. Stage 1 ALWAYS wins per the renderer's order at markdown.ts:541, so any row with a matching literal title flips to the new item — including rows currently pointing at a qualified-fallback target. Added a target_item_id != ? guard so we don't churn rows that already point at us. Stage 2 keeps the NULL constraint so already-resolved qualified rows don't churn when another fallback candidate appears. Regression test in wiki_links_test.go reproduces the exact scenario Codex described: fallback resolves first, literal arrival steals the row from the fallback target. PLAN-1593 / TASK-1595. * fix(backlinks): renderer parity for [[A|B]] + scope arrival retarget (Codex round 3) Two findings from Codex round 3 against PR #621: P1 — parser/renderer parity for [[A|B]] matching literal title "A|B". The renderer (web/src/lib/utils/markdown.ts:516-525) tries the FULL body as a title FIRST when a pipe is present, only falling through to the split interpretation on miss. Our parseBody always split on the first unescaped pipe, so an item literally titled "A|B" was indexed as title="A" with display="B" — the index would miss backlinks the UI shows, or point them at a different "A" item. Fix: in replaceWikiLinks for title kind, when HasDisplay, try the full body (Title+"|"+Display) as a title FIRST via resolveTitleTx; on hit, store target_title=fullBody and drop the display override (the display segment was actually part of the title). On miss, fall back to the existing split-key interpretation. The cascade's RewriteWikiTitle regex correctly handles both storage shapes via QuoteMeta on the title. P2 — stage-1 UPDATE was too aggressive after the round-2 fix. The broad UPDATE (no IS NULL constraint) silently stole backlinks from legitimately-resolved rows when a SECOND item was created/renamed to the same title. Titles aren't unique, the renderer's Array.find() is order-dependent, and silent churn is worse than no-op stability. Fix: split resolveBrokenTitleLinks into three updates: (1) Plain literal flip — NULL only. (2) Qualified literal flip — NULL only. (3) Literal-arrival retarget — gated on title containing `/`. Only fires for `[[<slug>/Title]]` rows, which are the only ones that COULD have been stage-2 qualified-fallback resolved. Rows with target_title='Foo' (no slash) can only have been stage-1 literal — we don't steal those. Regression tests: - TestWikiLinks_LiteralPipeInTitleResolves — `[[A|B]]` resolves to item literally titled "A|B". - TestWikiLinks_LiteralPipeInTitleFallsThroughToSplit — `[[A|B]]` falls back to item "A" when no "A|B" item exists. - TestWikiLinks_SecondItemSameTitleDoesNotStealBacklinks — adding a second "Foo" item doesn't redirect the existing backlink. PLAN-1593 / TASK-1595. * fix(backlinks): broken pipe-in-body rows keyed on full body (Codex round 4) Codex round 4 against PR #621: when a source body `[[A|B]]` is written before any matching item exists, the broken row was stored with target_title="A" (the split key). If an item literally titled "A|B" was later created, resolveBrokenTitleLinks looking for target_title="A|B" couldn't find the row — index went stale while the renderer's preferred full-body interpretation would correctly resolve the link. Fix: when nothing resolves AND a pipe was present (HasDisplay), key the broken row on the FULL body (Title+"|"+Display) instead of the split key. resolveBrokenTitleLinks then naturally finds it via the literal-arrival path. The remaining asymmetry — a broken row keyed on full body won't pick up a future split-fallback resolution to a new item titled "A" — is documented in the code as a v3-promotable limitation. The full-body path is the renderer's PREFERRED interpretation (markdown.ts:516), so prioritizing it is the right tradeoff in the rare case both interpretations could apply. Regression test in wiki_links_test.go reproduces the scenario: source written first with `[[A|B]]`, item titled "A|B" created later, backlink should resolve. PLAN-1593 / TASK-1595. * fix(backlinks): scope stage-3 retarget + cascade self-refs (Codex round 5) Two findings from Codex round 5 against PR #621: Finding 1 — stage-3 literal-arrival retarget could still steal backlinks from a legitimate stage-1 literal-match row when a SECOND item with the same slash-containing title is created. The previous fix (round 3) gated stage-3 on title containing `/`, which let through the qualified-fallback retarget case correctly but didn't distinguish stage-1-resolved rows pointing at a literal twin from stage-2-resolved rows pointing at the fallback target. Fix: add an EXISTS check that scopes the flip to rows whose CURRENT target has a title NOT matching ours. Stage-1 (literal) resolutions point at items literally titled the same as the row's target_title; stage-2 (qualified-fallback) resolutions point at items titled just the trailing segment. The EXISTS clause picks out only the latter. Finding 2 — self-references on the renamed item went stale on title-only renames. The cascade's `s.id != renamedItemID` filter excluded self, but items.go only re-indexes content when input.Content != nil. So a title-only rename of an item whose body mentions itself by its old title kept the body's now-broken `[[Old Title]]` literal in place while the index still recorded a "working" backlink — drift between renderer state and index state. Fix: drop the self-exclusion from the cascade SELECT. RewriteWikiTitle rewrites the renamed item's own content along with everyone else's; GetBacklinks still hides self-links at query time, so the backlinks panel behavior is unchanged. Tests: - TestWikiLinks_DuplicateSlashTitleNoTheft regresses Finding 1. - TestWikiLinks_TitleRenameRewritesSelfReferences asserts Finding 2's new correct behavior (replaces the prior test that asserted the old buggy behavior). PLAN-1593 / TASK-1595. * fix(backlinks): ref→title fallback + reorder cascade self-ref (Codex round 6) Two findings from Codex round 6 against PR #621: Finding 1 — ref-shaped title fallback missing. parseBody returns WikiLinkKindRef for `[[ISO-9001]]` (matches refPattern), but if no ISO-9001 ref-item exists, the renderer falls through to legacy title lookup (markdown.ts:513) and resolves to an item literally titled "ISO-9001". The store inserted only a broken ref-kind row with target_title=NULL, so GetBacklinks never surfaced the backlink even when the renderer rendered it. Fix: in replaceWikiLinks' WikiLinkKindRef branch, when resolveRefTx misses, try resolveTitleTx on the same body. If title resolves, INSERT as title-kind row with target_title=ref-shaped-body. The rename cascade catches these correctly via target_kind='title' + target_item_id. The asymmetry — a future ref-item creation can't auto-retarget these title-stored rows — is documented as a v3 limitation. Finding 2 — combined title+content update broke self-ref cascade. The original order (main UPDATE → replaceWikiLinks(self) → cascade) wiped self's `target_item_id=renamedItemID` row before cascade ran: when input.Content contains `[[Old Title]]`, re-indexing self resolved it as broken (target_item_id=NULL), so cascade's SELECT missed self for the title+content path. Fix: reorder so cascade runs BEFORE the self re-index — the pre-existing wl rows are still intact at cascade time. The final re-index re-reads items.content from the DB (since cascade may have rewritten it in-band) rather than using *input.Content directly; otherwise the re-index would undo the cascade's self-ref rewrite. Tests: - TestWikiLinks_RefShapedFallsThroughToTitle — `[[ISO-9001]]` resolves to an item titled "ISO-9001". - TestWikiLinks_TitleAndContentRenameCascadesSelfRef — combined title+content rename with self-ref in new content gets the self-ref rewritten by cascade. PLAN-1593 / TASK-1595. * fix(backlinks): ref+pipe→title parity, position-based cascade, scoped self-rewrite (Codex round 7) Three intertwined fixes addressing Codex round 7 findings against PR #621: Finding 1 — ref→title fallback missed pipe-bodies. For `[[ISO-9001|Spec]]`, renderer tries full-body title "ISO-9001|Spec" BEFORE falling to bare "ISO-9001" (markdown.ts:516). Our ref-fallback only tried bare. Extended ref-branch's fallback to try full body first when HasDisplay, then bare — same order as the title-branch's stage (a)/(b) pattern. Finding 2 — cascade corrupted UNRELATED literal-pipe titles. Items A "Old Title" and B "Old Title|alias" both referenced from one source; renaming A previously triggered RewriteWikiTitle's regex `(?i:Old Title)((?:\|...)?)` which matched BOTH A's `[[Old Title]]` AND B's `[[Old Title|alias]]` — corrupting the B link. Refactored cascadeTitleRename to be POSITION-BASED: SELECT each individual wl row with its position + target_title (no longer DISTINCT sources). Per-row, rewrite the bracket AT THAT EXACT POSITION via new links.RewriteBracketAt helper. Process rows in descending position order per source so earlier offsets don't shift. Brackets whose wl row doesn't resolve to the renamed item are never visited. Scoped self-rewrite — title-only renames cascade self (input.Content == nil); combined title+content renames EXCLUDE self (input.Content != nil). User-supplied content is authoritative; auto-rewriting their just-submitted brackets would surprise them. Mirrors documents.go::updateLinksInTx, which also leaves the renamed entity's own content alone. Codex round 6 finding 2 is fully addressed: title-only path still rewrites self-refs, combined path respects user submission and the index correctly records the bracket as broken (matching what the renderer would render). New links.RewriteBracketAt helper with full unit-test coverage: plain/aliased/qualified/qualified+aliased shapes, case-insensitive matching, slug-prefix preservation, full-body vs split-key target disambiguation, out-of-bounds defensive guards. Integration test TestWikiLinks_CascadeDoesNotCorruptLiteralPipeNeighbor reproduces Codex round 7 finding 2's scenario. PLAN-1593 / TASK-1595. * fix(backlinks): retarget rows pointing at soft-deleted targets (Codex round 8) Codex round 8 P2: resolveBrokenTitleLinks only considered target_item_id IS NULL rows as eligible for flip. A row that resolved to item A and then had A soft-deleted stayed pointing at deleted A; creating a new item B titled the same as A wouldn't flip the row, so GetBacklinks(B) missed the backlink the renderer would actually show (renderer hides deleted-target links). Fix: introduce a "broken-in-practice" predicate (target_item_id IS NULL OR NOT EXISTS ( SELECT 1 FROM items t WHERE t.id = item_wiki_links.target_item_id AND t.deleted_at IS NULL )) applied to stages 1 (plain literal flip) and 2 (qualified literal flip). Stage 3 (slash-title literal-arrival retarget) already considered "current target deleted" implicitly via its title- mismatch EXISTS check. Regression test in wiki_links_test.go covers the exact scenario: A "Foo" resolves a backlink → A soft-deleted → B "Foo" created → backlink flips to B. Known limitation deferred to v3: the symmetric case (delete A while B with same title already exists) doesn't fire any hook that re-resolves the row. A dedicated DeleteItem hook would close that gap; not blocking Phase 2a. PLAN-1593 / TASK-1595. * fix(backlinks): preserve title whitespace to match renderer (Codex round 9) Codex round 9 P2: parseBody trimmed whitespace from the body BEFORE deciding it was a title kind. The renderer doesn't trim before title matching (markdown.ts:541-543) — `[[ Foo ]]` is matched against items.title with the surrounding spaces intact, so an item titled "Foo" wouldn't match. Trimming server-side created index entries the UI couldn't click — backlinks showed in the panel for links that the renderer rendered as broken. Fix: keep an UNTRIMMED unescaped body for title-kind fallthrough, and a trimmed copy only for ref / workspace_ref SHAPE detection (refs are whitespace-free by construction, the renderer's key.trim() at L503 is just typing forgiveness for the ref form). Tests in extract_test.go: - `[[ Foo ]]` emits title-kind with Title=" Foo " (whitespace preserved). - `[[ TASK-5 ]]` still parses as ref (shape detection trims). - `[[Project Goals]]` (two spaces inside) preserves internal whitespace. PLAN-1593 / TASK-1595. * fix(backlinks): ref→title fallback uses raw untrimmed key (Codex round 10) Codex round 10 P2: after round 9's title-kind whitespace fix, the ref→title FALLBACK path still used canonical-trimmed link.Ref for its title lookup. The renderer's fallback at markdown.ts:541-543 uses the UNTRIMMED key (no .trim() on the title-lookup path), so `[[ TASK-5 ]]` falling through to title would search " TASK-5 " (with whitespace) — an item literally titled " TASK-5 " resolves in the UI but not in our index. Fix: add a RawKey field to WikiLinkRef capturing the untrimmed unescaped key for ref kinds. parseBody populates it alongside the canonical Ref. replaceWikiLinks' ref→title fallback path uses RawKey (with defensive fallback to Ref for old rows) when constructing title candidates. Mirrors the renderer's untrimmed title lookup across both bare and pipe forms. Regression test in wiki_links_test.go covers the exact scenario: item titled " TASK-5 " (with whitespace), source body `[[ TASK-5 ]]`, backlink resolves via the untrimmed ref→title fallback. PLAN-1593 / TASK-1595.	2026-05-24 12:01:05 -04:00
xarmian	8e7d4040fd	feat(backlinks): server-side reverse index for [[...]] (Phase 1) (#620) ... * feat(backlinks): server-side reverse index for [[...]] wiki-links (Phase 1) First phase of PLAN-1593. Today [[REF]] is parsed only at render time on the client and there's no way to ask "who links to TASK-5?" without a full-text scan. This change adds a materialized reverse index (item_wiki_links) that's written every time an item's content changes and exposes it via REST + CLI. Phase 1 covers ref-form links only (`[[TASK-5]]` / `[[TASK-5|Display]]`). Phase 2 (TASK-1595) will extend to titles + cross-workspace; Phase 3 (TASK-1596) adds the web UI panel + MCP action. What lands here: * Migrations 061 (SQLite) and 040 (Postgres) create item_wiki_links with partial indexes on target_item_id, (target_workspace_id, target_ref), and target_title — the schema accommodates all 5 wiki-link forms up-front so Phase 2 doesn't ALTER. * internal/links/extract.go is the canonical parser. It strips fenced and inline code regions before extracting [[...]] occurrences, so example refs in docs / code blocks don't pollute the index. Phase 1 emits only WikiLinkKindRef rows; title and workspace_ref kinds parse successfully but are gated out until Phase 2. * internal/store/wiki_links.go (replaceWikiLinks + GetBacklinks + helpers) handles write-time bookkeeping and the read query. Resolution to target_item_id happens at parse time inside the same transaction as the items INSERT/UPDATE, so partial state never lands. Broken refs (target_item_id IS NULL) intentionally persist — they feed a future broken-links report. * internal/store/wiki_links_backfill.go + cmd/pad/main.go hook the idempotent backfill into server startup. Existing items get indexed on first boot after the migration; subsequent boots are near-no-ops via an EXISTS short-circuit. * internal/store/items.go is amended in two places: tryCreateItem always calls replaceWikiLinks (empty content → no-op DELETE), and UpdateItemWithPreCheck re-parses whenever input.Content was supplied. * internal/server/handlers_backlinks.go serves `GET /api/v1/workspaces/{ws}/items/{itemSlug}/backlinks` with visibility + guest-grant filtering on the source items. * internal/cli/client.go adds GetBacklinks; cmd/pad/main.go adds the `pad item backlinks <ref>` command (registered in groups.go). Behavior decisions (per PLAN-1593): - code blocks excluded (fenced + inline) - self-links filtered at query time (kept in storage) - repeated mentions stored as separate rows by position - ordering: source updated_at DESC, position ASC Tests: - internal/links/extract_test.go: 26 sub-cases covering ref/title/ workspace-ref discrimination, code-block exclusion (fenced + inline + unclosed fence), position-is-byte-offset (UTF-8 safety), and edge inputs. - internal/store/wiki_links_test.go: 8 integration tests covering the create/update/delete/self-link/broken-ref/repeated/code-block scenarios plus backfill idempotence. All pass. `make check` clean (lint + go test + web build). Refs: TASK-1594, PLAN-1593, IDEA-1577 * fix(backlinks): visibility-aware pagination + case-insensitive refs per Codex review (round 1) Two fixes from Codex code review: P1 — GetBacklinks now takes a visibleCollectionIDs []string argument that's applied INSIDE the SQL WHERE clause. Previously the handler fetched LIMIT raw rows and filtered visible ones in Go, so a restricted user asking for limit=50 could receive an empty page even when later visible backlinks existed. Pushing visibility into SQL makes LIMIT/OFFSET count visible rows. nil → no restriction (owners, editors, root tokens) [] → see nothing (returns early, no SQL) [..] → AND s.collection_id IN (?, ?, ...) Item-level guest grants still apply post-fetch — they're rare enough that the residual page shrink is acceptable and pushing them into SQL would balloon the query. P2 — refPattern now accepts mixed/lowercase refs and parseBody canonicalizes the prefix to uppercase at the single chokepoint. Previously the renderer accepted `[[task-5]]` as a real link (its REF_PATTERN is case-insensitive) but the indexer's ^[A-Z]... pattern silently dropped it — divergent parsing on the same input. Storage shape is canonical uppercase so the (workspace, prefix, number) lookup against collections.prefix (also uppercase) has one shape. New helper: canonicalizeRef("task-5") → "TASK-5". Regressions: internal/links/extract_test.go + TestCanonicalizeRef — helper unit tests + TestExtractWikiLinks_RefVsTitleFallback updated to assert mixed/lowercase parses-as-ref-and-uppercases + edge-case test renamed from "lowercase ref" to "number-led not a ref" (lowercase IS a ref now per Codex P2) internal/store/wiki_links_test.go + TestWikiLinks_MixedCaseRefIndexed — `[[task-5]]` produces a backlink row whose target_ref is "TASK-5" + TestWikiLinks_VisibilityAwarePagination — three sub-cases: nil → all 3, visible-only limit=2 → 2 visible rows (not 1 with hidden one consuming a slot), empty → 0 All call sites updated (8 in tests + 1 in handler). `make check` clean (lint + tests + web build). Refs: TASK-1594, PLAN-1593 * fix(backlinks): SQL-level item-grant filter per Codex review (round 2) Round 1 fixed pagination for collection-level visibility but Codex round 2 correctly flagged the same class of bug at the item-grant layer: `visibleCollectionIDs` returns the UNION (full grants ∪ collections containing granted items), and the handler then filtered each row's item-level visibility in Go AFTER fetching — letting hidden rows in a granted-item's collection consume LIMIT slots. The refactor moves the precise predicate into SQL. New shape: type BacklinksVisibility struct { Unrestricted bool // admin / full-access member FullCollectionIDs []string // direct collection grants GrantedItemIDs []string // item-level grants } // SQL predicate when Unrestricted=false: // AND (s.collection_id IN (?...) OR s.id IN (?...)) This matches `guestResourceFilter` (which returns the precise primitives), so the handler now passes them straight through and drops the post-fetch filter loop entirely. Pagination is correct for guests, restricted members, and unrestricted users alike. New test: TestWikiLinks_ItemGrantPagination — guest with item-grant on ONE item in an otherwise-hidden collection sees exactly that one item; hidden siblings in the same collection do NOT leak in, and limit=2 returns 1 row (not silently shrunken). Other call sites updated: - TestWikiLinks_VisibilityAwarePagination → uses BacklinksVisibility{FullCollectionIDs: ...} and BacklinksVisibility{} for the no-access case. - 8 existing tests → BacklinksVisibility{Unrestricted: true}. - handlers_backlinks.go → no longer calls visibleCollectionIDs; uses guestResourceFilter exclusively and skips the Go-side filter. Verification: - make check clean - All TestWikiLinks_* pass Refs: TASK-1594, PLAN-1593 * fix(backlinks): scan EXISTS into bool not int for Postgres parity (Codex round 3) `SELECT EXISTS(...)` returns boolean on Postgres but integer 0/1 on SQLite. Scanning into `int` happened to work on SQLite (the modernc.org driver coerces) but would fail on Postgres — silently disabling the backfill short-circuit there and meaning upgraded Postgres installs wouldn't populate backlinks for pre-existing content until each item got edited. Fix: scan into bool. Both database/sql drivers in use (modernc.org/ sqlite and lib/pq) coerce their native representation into Go's bool, so this single shape works on both engines. make check clean. Refs: TASK-1594, PLAN-1593 * fix(backlinks): allow CommonMark 0-3 space indented fences per Codex (round 5) Round 5 flagged two edge cases in the code-stripping pass: 1. Multi-backtick inline code (``see [[X]]``) — traced through the parser; my permissive close-on-next-backtick logic already covers it correctly (range = [opener-start, after-closer-run]). Added a regression test to lock this in: TestExtractWikiLinks_CodeBlocksExcluded / "multi-backtick inline code excludes ref" 2. Indented fenced blocks — CommonMark allows 0-3 leading spaces of indentation before a fence opener (4+ spaces makes it an indented code block, a different construct). My fencedCodeRanges only matched fences at column 0, so ` ```\n[[X]]\n```` ` would render as code in the UI but leak a false backlink. Fixed both fencedCodeRanges (opener) and findFenceCloser (closer) to skip up to 3 leading spaces, with a hard cap at 4 (which would be indented-code, not a fence). Regression test: TestExtractWikiLinks_CodeBlocksExcluded / "indented fenced block (CommonMark 0-3 spaces)" Not addressed: - Round-4 escape-body parity finding. extract.go mirrors renderMarkdown's regex (web/src/lib/utils/markdown.ts:300), which is the actual render-time link parser; wikiLinksToMarkdown's more permissive escape grammar is editor-serializer-side and the renderer can't even consume its escaped output. Indexing what the user actually sees as a link is the correct invariant. make check clean. Refs: TASK-1594, PLAN-1593 * fix(backlinks): tilde fences + strict closer lines per Codex (round 6) Two CommonMark conformance gaps in the code-block stripping pass: 1. Tilde-fenced code blocks (~~~) were ignored. marked() treats them the same as backtick fences, so a [[REF]] inside a tilde block would render as code in the UI but leak as a false backlink. Fixed by parameterizing fenceChar across fencedCodeRanges and findFenceCloser, with separate handling for the backtick-specific "no backtick in info string" rule (CommonMark §4.5). 2. Closer-line strictness — CommonMark requires the closing fence line to contain only the fence + optional trailing spaces. The previous accept-any-fence-prefixed-line check would terminate a still-open fence prematurely on a line like ```not-closed, leaking later refs in the still-rendered code block. Refs reside in 4 new sub-tests under TestExtractWikiLinks_CodeBlocksExcluded: - tilde fence excludes refs inside - tilde fence with language tag - mixed fence types don't pair - closer-line strictness — backticks plus other text is not a closer - closer-line strictness — trailing spaces OK make check clean. Refs: TASK-1594, PLAN-1593 * fix(backlinks): inline code closer must match opener length per Codex (round 7) CommonMark §6.1 requires an inline-code span opened with N backticks to close on a run of EXACTLY N backticks. The previous "close on next backtick run of any length" logic would prematurely end the excluded range on a stray single backtick inside a ``...`` span, leaking any [[REF]] in the latter half of the code text as a false backlink. Concrete failure case: ``has ` inside [[X-1]] and more`` → old: range [0, 7], [[X-1]] indexed (bug) → new: range [0, end-of-closer], [[X-1]] excluded (correct) Fix: track the opener-run length and scan only for matching-length closer runs. Wrong-length runs in between are code text. Two new regression sub-cases under TestExtractWikiLinks_CodeBlocksExcluded: - inline code closer matches opener length — the main case - single-backtick span unaffected by adjacent multi-backtick run — asserts the opposite direction (opener=1 doesn't close on ``) Not addressed: - Re-flagged round-4/round-7 escape-body parity finding. extract.go intentionally mirrors renderMarkdown's regex (markdown.ts:300), not wikiLinksToMarkdown's more permissive escape grammar (markdown.ts:461). renderMarkdown is the actual link parser at display time; its regex rejects escaped-`]` bodies, so any link with an escaped `]` in its body is NOT shown as a clickable link in the UI. Indexing it would produce phantom backlinks the user can't see. The wikiLinksToMarkdown permissive grammar is paranoid serialization that the renderer can't consume — that's a pre-existing inconsistency in the editor pipeline, not a backlinks bug. make check clean (lint + tests + web build). Refs: TASK-1594, PLAN-1593 * fix(backlinks): rune-align snippet end-edge to keep UTF-8 valid (Codex round 8) The previous snippetAround() trimmed `start` to a rune boundary (so the leading edge of the snippet was always at a valid codepoint) but left `end` as a raw +40-byte clamp. When that landed in the middle of a multi-byte rune — common around emoji or accented text — the resulting slice was invalid UTF-8 and the JSON encoder would emit replacement characters in backlink snippets. Fix: same forward-advance pattern at the end as at the start. Continuation bytes (10xxxxxx) get skipped until we land on a leading byte. Going forward keeps the snippet anchored slightly past the match rather than slightly before it, which is a small UX win (emoji or accented text right after the link survives intact). Regression test: TestWikiLinks_SnippetIsValidUTF8 — pads body with enough 4-byte emoji on each side that the ±40-byte window cuts through one; asserts utf8.ValidString on the resulting snippet. make check clean (lint + tests + web build). Refs: TASK-1594, PLAN-1593 * fix(backlinks): inline code spans cross newlines, break on blank lines (Codex round 9) CommonMark §6.1: an inline-code span can cross single newlines but terminates at a blank line (a line containing no chars or only whitespace, which ends the enclosing paragraph). My previous scanner broke at every newline, so multi-line spans like `pre [[INSIDE-1]] post` would treat the opener as unclosed and leak [[INSIDE-1]] as a false backlink. Fixed by: 1. The newline branch in the closer scan now peeks ahead via the new isBlankLineAt() helper. Same-paragraph newlines are traversed; blank-line breaks terminate the span unmatched. 2. isBlankLineAt() treats any line with only space/tab as blank (mirroring CommonMark's blank-line definition). Three new regression sub-cases under TestExtractWikiLinks_CodeBlocksExcluded: - inline code spans single newline (CommonMark §6.1) - inline code breaks at blank line (paragraph boundary) - inline code breaks at whitespace-only blank line Trade-off: a truly-unclosed inline backtick now consumes from the opener up to the next blank line instead of just the rest of the line. False-positive on wiki-links in that span, but the surface area is small (unclosed backticks are rare in published prose) and matches the renderer's behavior. make check clean. Refs: TASK-1594, PLAN-1593 * fix(backlinks): accept escaped wiki-link bodies per editor grammar (Codex round 10) After 3 rounds of disagreement, capitulating on the escape-body parity finding. My position was technically correct for the CURRENT renderMarkdown behavior (which uses [^\]]+ and can't parse escaped- bracket bodies), but the editor's wikiLinksToMarkdown grammar at markdown.ts:461 explicitly produces such bodies — making the renderer's regex the inconsistent half of the pipeline, not mine. Mirroring the editor's grammar in the extractor makes the index forward-compatible: when the renderer eventually gets fixed, no change here is needed. The cost is a few "phantom" rows in the interim (indexed links the renderer doesn't currently display as clickable), but those are harmless and aligned with author intent. Changes: - wikiLinkPattern now uses `\[\[((?:\\.|[^\]\\])+)\]\]` — mirrors markdown.ts:461 verbatim. - New splitOnUnescapedPipe() helper — scans for the first `|` that isn't preceded by `\`. Mirrors splitWikiBody at markdown.ts:664. - New unescapeWikiBody() helper — undoes `\]`, `\|`, `\\` escapes in display text and key. Mirrors unescapeWikiBody at markdown.ts:657. - parseBody() now uses both helpers — split on unescaped `|`, unescape both sides. Regression coverage: - TestExtractWikiLinks_EscapedBodyChars (5 sub-cases): escaped `]`, escaped `|`, escaped `\`, non-escape backslash passes through, Position still points at opening `[[` despite escapes. - TestSplitOnUnescapedPipe + TestUnescapeWikiBody: direct unit tests for the helpers (round-trip safety vs the editor's escape/unescape pair). make check clean. Refs: TASK-1594, PLAN-1593 * fix(backlinks): preserve display text verbatim per Codex round 11 P3 The previous parseBody trimmed the display side of [[X|Display]] but the WikiLinkRef.Display contract promises verbatim storage and the renderer at markdown.ts doesn't trim either. Trimming would silently diverge on padded display text like [[TASK-1| spaces ]] (renderer keeps the spaces, extractor stripped them). Fix: drop TrimSpace from the suffix half of the split. Keep trimming the key/ref side because refPattern is anchored — a leading or trailing space in the key would force the body to fall through to the title kind even though the renderer resolves it as a ref. Regression test: TestExtractWikiLinks_EscapedBodyChars / "display text preserved verbatim (no TrimSpace)" make check clean. Refs: TASK-1594, PLAN-1593 * fix(backlinks): distinguish empty display override from no-pipe per Codex round 12 [[REF|]] (explicit empty display) and [[REF]] (no display) are distinct shapes in the editor: splitWikiBody returns displayOverride="" for the former, null for the latter, and the renderer uses `displayOverride ?? title` (nullish coalescing, NOT empty-string fallback) so "" is preserved. The previous extractor collapsed both into display_text=NULL, violating verbatim-display preservation for the empty-string edge case. Fix: - WikiLinkRef gains a HasDisplay bool. parseBody sets HasDisplay=true iff splitOnUnescapedPipe found a pipe; downstream uses HasDisplay (not Display!="") to decide whether to persist the override. - replaceWikiLinks in store: NullString.Valid is keyed off HasDisplay. display_text='' for explicit empty, NULL for no override. Regression coverage: - internal/links/extract_test.go: "explicit empty display override is distinguished from no pipe" - internal/store/wiki_links_test.go: TestWikiLinks_EmptyDisplayDistinct (two-source assert: NOT NULL for [[REF|]], NULL for [[REF]]) make check clean. Refs: TASK-1594, PLAN-1593 * fix(backlinks): pointer-typed DisplayText to preserve empty distinction over JSON (Codex round 13) Round 12 added HasDisplay on the parser side and made the store preserve display_text='' vs NULL on the DB row, but the wire model collapsed the distinction at JSON-serialization time: DisplayText string `json:"display_text,omitempty"` `omitempty` drops empty strings, so [[REF|]] (empty override) and [[REF]] (no override) serialized identically on the API and CLI JSON output. The end-to-end goal of round 12 wasn't reached. Fix: change DisplayText to *string. nil → no override (field omitted from JSON via omitempty), pointer to "" → explicit empty override (field present with empty value). The store's NullString.Valid drives the assignment, so the SQL round-trip matches the JSON shape. Knock-on: the CLI's `pad item backlinks` now dereferences the pointer and prints both populated and empty overrides ("displayed as: "). Regression coverage: - TestWikiLinks_EmptyDisplayDistinct extended to assert withBL.DisplayText is non-nil-pointing-at-"" and noBL.DisplayText is nil after a GetBacklinks round-trip. make check clean. Refs: TASK-1594, PLAN-1593	2026-05-23 23:09:44 -04:00
xarmian	dd381e1066	chore: delete 5 unwired document handlers (TASK-769) (#252) ... * chore: delete 5 unwired document handlers (TASK-769) internal/server/handlers_documents.go had 5 dead HTTP handlers that were drafted as Documents-v1 extensions but never wired into the router (server.go:509 already labels Documents itself as "v1, will be replaced by items in Phase 2"): - handleQuickSave (POST /documents/quick-save) — title-based upsert - handleBulkRead (POST /documents/bulk-read) — multi-doc fetch by IDs - handleGetBacklinks (GET /documents/{id}/backlinks) - handleGetLinks (GET /documents/{id}/links) - handleGetContext (GET /documents/context?type=) Investigation confirmed zero consumers: - Not registered in setupRouter (`grep -n "QuickSave\|BulkRead\|Backlinks\|GetLinks\|GetContext" server.go` → empty). - Not used by the SvelteKit frontend (`web/src/`). - Not used by the CLI (`internal/cli/`). - Pre-launch repo, no fork or downstream that could be relying on them. Delete scope is intentionally limited to the HTTP handlers. The underlying `Store.QuickSave / BulkRead / GetBacklinks / GetLinks / GetContext` methods stay — they're tested at the store level (internal/store/store_test.go) and preserve optionality if Phase 2 work needs to revive any of these features. `models.QuickSave` stays for the same reason. After this lands, IDEA-732's lint catalog is fully cleared on main (staticcheck SA* + U1000 returns zero). TASK-771 (flip CI only-new-issues=false) becomes safe. Verified: - `go build ./...` clean - `go vet ./...` clean - `go test ./...` all pass - `staticcheck -checks "SA*,U1000" ./...` clean - All `import "strings"` etc. still used elsewhere in file Parent: PLAN-644. * chore: also delete now-test-only document store helpers (TASK-769) Codex round 1 on PR #252 flagged that the document-store helpers retained for "Phase 2 optionality" are now exclusively kept alive by their own store tests — Store.QuickSave, BulkRead, GetBacklinks, GetLinks, GetContext are not called by any production code path after the handler deletions in the previous commit. Same for the models.QuickSave struct. Pre-launch with no external consumers, optionality preservation has a real cost (dead code on main, ongoing test maintenance). When Phase 2 needs any of these capabilities it is cheaper to re-derive them against the Items model than to drag dead Documents-v1 plumbing forward. So delete them now. Removed: - internal/store/documents.go: QuickSave (38 lines), BulkRead (28), GetBacklinks (15), GetLinks (28), GetContext (41). - internal/models/document.go: QuickSave struct. - internal/store/store_test.go: TestQuickSave (38 lines), TestBulkRead (16), TestDocumentLinking (29), TestContext (23). Kept: - TestDocumentLinkRename — exercises UpdateDocument's internal link-rewriting path, not any of the deleted helpers. - GetDocumentByTitle — still used by TestDocumentLinkRename. - The full CRUD/restore handlers and their store methods — these are still wired into setupRouter and have their own coverage. Verified: - `go build ./...` clean - `go vet ./...` clean - `go test ./...` all pass (TestDocumentLinkRename and the wider doc CRUD/version/activity tests still cover the surviving paths). - `staticcheck -checks "SA*,U1000" ./...` clean - No new unused imports introduced (links package is still used by documents.go for ReplaceTitle in UpdateDocument). Parent: PLAN-644. * chore: drop GetDocumentByTitle and refactor TestDocumentLinkRename (TASK-769) Codex round 2 caught the chain — after deleting QuickSave/BulkRead/ GetBacklinks/GetLinks/GetContext, Store.GetDocumentByTitle was kept alive by exactly one test (TestDocumentLinkRename), which was re-fetching by title only because the test ignored the *Document already returned by createTestDoc. Use the createTestDoc return value instead, then drop GetDocumentByTitle from the store. Same idea, cleaner test, one fewer test-only API on the store. The rename behaviour (the actual thing under test) is unchanged. Verified: - `go build ./...` clean - `go test ./internal/store` and `./internal/server` pass - `staticcheck -checks "SA*,U1000" ./...` still clean Parent: PLAN-644. * chore: drop now-orphaned links.Extract (TASK-769) Codex round 3 caught the next link in the chain: after Store.GetLinks was deleted, links.Extract had no remaining callers — links.ReplaceTitle is the only Extract-package function still used (by UpdateDocument's rename rewrite). The linkPattern regex was only used by Extract. Drop linkPattern, the regexp import, and Extract itself. Leaves ReplaceTitle and its private string helpers (replaceAll, indexOf) intact. The cleanup chain ends here: ReplaceTitle is still wired into a live production path (Documents-v1 rename), and the supporting helpers have no other roles to inherit. Verified: - `go build ./...` clean - `go test ./internal/store` and `./internal/server` pass - `staticcheck -checks "SA*,U1000" ./...` clean Parent: PLAN-644.	2026-04-25 12:22:53 -04:00
xarmian	81579847c6	Initial release ... Pad — project management for developers and AI agents. Single Go binary with embedded SvelteKit web UI, SQLite storage, CLI, and Claude Code /pad skill integration. https://getpad.dev	2026-03-26 01:52:36 +00:00