fix(documents): count retained bytes, bound the retry path, escape the cascade's LIKE pattern (BUG-2798)

Codex round 1 on #1218. Three findings, all real, all fixed here.

1. The guard bounded projected OUTPUT, which bounds nothing when the new title
   is SHORTER than the old one. Renaming a 255-character title to a
   one-character title makes each 2 MiB linker project ~40 KiB while the
   cascade still retains its 2 MiB read for the compare-and-set, so hundreds
   of linkers exhaust memory while the counter reports well under the cap.

   The counter now sums RETAINED bytes — read plus written, both alive at once
   — so the cap is a statement about resident memory rather than about output.
   MaxRenameCascadeProjectedBytes becomes MaxRenameCascadeRetainedBytes and
   moves 16 -> 32 MiB, because the legitimate ceiling it clears doubles under
   the new metric (that instance's whole wiki-linking corpus retains
   ~20,154,952 bytes); the single-document attack retains 110,729,522, so it
   is still refused by 3.3x.

2. The compare-and-set's retry path bypassed the guard entirely. On
   contention it re-reads the linker and calls ReplaceTitle on whatever the
   winner wrote — a NEW input, bounded by nothing the scan had checked — so a
   content edit landing inside the cascade's window could grow a linker from
   harmless to enormous and walk the rename back into the amplification it
   would have been refused for. Each document's compare-and-set now carries
   the cap less what the other linkers hold, and re-checks the grown body
   against it.

3. The cascade's `content LIKE ?` search term went in unescaped, so a document
   TITLE decided how the pattern was read. `\` is the default LIKE escape
   character on Postgres and NOT on SQLite, so `[[Alpha\Beta]]` was searched
   for as itself on one dialect and as `[[AlphaBeta]]` on the other: linkers
   not found, cascade rewrites nothing, rename reports success, every link
   left stale. Silent and dialect-dependent.

   Codex named the backslash; `%` and `_` are the rest of the class (CONVE-18)
   — wildcards on both dialects, so a title carrying them selects documents
   that do not link it. An explicit `ESCAPE '\'` clause plus escapeLikePattern
   makes both dialects agree, rather than leaving SQLite correct by accident.

Finding 3 also constrains finding 3 of the ORIGINAL fix: models' validator
allows a lone backslash in a title on the grounds that both renderers handle
it, which was true of rendering and false of cascading. That comment now
records the dependency — allowing it is only correct while the cascade's
pattern stays escaped.

Tests, four new, each mutation-verified against the code it guards:

- CountsRetainedBytesNotJustOutput — the shrinking rename. Asserts as a
  PRECONDITION that the projected-output total stays under the cap, so the
  test cannot pass for the old reason.
- RetryRecheckesTheBudgetAgainstTheGrownBody — drives the real race through
  the afterLinkCascadeRead seam. POSTGRES ONLY and skipped loudly elsewhere:
  SQLite's BEGIN IMMEDIATE closes the window structurally, so a green run
  there would be a property of the DSN.
- FindsLinkersWhoseTitleContainsABackslash — Postgres only, same reasoning
  inverted: SQLite is the dialect that was accidentally right.
- DoesNotSpendTheBudgetOnDocumentsThatDoNotLinkTheTitle — `%` and `_`. Its
  first version asserted the decoy's content was untouched and passed against
  the unescaped pattern, because over-matched rows rewrite to themselves. The
  observable harm is that they spend the caller's budget, so that is what it
  now asserts.

Mutation matrix for this round: output-only counter -> only the shrinking test
fails; retry check removed -> only the retry test fails (PG); LIKE unescaped ->
the budget legs fail on SQLite and the backslash test fails on PG.

Gates: `go test ./...` under Postgres 17 EXIT=0; SQLite packages EXIT=0;
gofmt clean; `make lint` 0 issues.

BUG-2798, BUG-2796

Claude-Session: https://claude.ai/code/session_011T365kP1N9V88y15HxL4YN
This commit is contained in:
xarmian
2026-08-27 18:16:06 +00:00
parent e5131a9583
commit b09aca12ea
4 changed files with 381 additions and 78 deletions
+6 -1
View File
@@ -138,7 +138,12 @@ const MaxDocumentTitleRunes = 255
// "stored legacy titles that contain a literal `|`". Banning it would
// refuse what that branch was written to support.
// - a lone `\` not followed by `\`, `]` or `|` — passes the grammar as an
// escape pair and survives the unescaper unchanged.
// escape pair and survives the unescaper unchanged. Note this one depends
// on store.escapeLikePattern: the rename cascade finds linking documents
// with `content LIKE`, where Postgres reads `\` as an escape character, so
// before that escaping landed a backslash title rendered fine and then
// silently failed to cascade on one dialect. Allowing it here is only
// correct while the cascade's pattern stays escaped.
//
// Boundary, stated rather than papered over: this is derived from the SHARED
// stored-syntax path in markdown.ts. The legacy documents surface has no
@@ -109,8 +109,8 @@ func TestRenameCascadeTooLarge_IsPermanentShapedNotRetryable(t *testing.T) {
// request cap, which is the point: the hostile input is cheap to deliver.
const newTitleLen = 255
const linkers = 3
perDoc := (store.MaxRenameCascadeProjectedBytes / linkers) + (store.MaxRenameCascadeProjectedBytes / (linkers * 4))
occurrences := perDoc / (5 + (newTitleLen - 1))
perDoc := (store.MaxRenameCascadeRetainedBytes / linkers) + (store.MaxRenameCascadeRetainedBytes / (linkers * 4))
occurrences := perDoc / (5 + 5 + (newTitleLen - 1))
body := strings.Repeat("[[A]]", occurrences)
for i := 0; i < linkers; i++ {
+128 -45
View File
@@ -564,13 +564,40 @@ func (s *Store) acquireWorkspaceDocumentRenameLock(tx *sql.Tx, workspaceID strin
return nil
}
// escapeLikePattern escapes the three characters that carry meaning inside a
// LIKE pattern, for use with an explicit `ESCAPE '\'` clause.
//
// Without this the cascade's own search term is interpreted as a pattern, and
// a document TITLE decides how (BUG-2798, codex round 1 P2 — plus the rest of
// the class it was an instance of):
//
// - `_` and `%` are wildcards in BOTH dialects, so a title containing them
// selects documents that do not link it. Those extra rows rewrite to
// themselves, so the damage is not corruption — it is that the guard below
// is computed from this result set, so an over-broad pattern spends a
// caller's budget on rows that were never going to change.
// - `\` is where the two dialects DISAGREE, which is the dangerous half.
// Postgres LIKE treats backslash as the default escape character; SQLite
// LIKE has no default escape character at all. So `[[Alpha\Beta]]` is
// searched for as the literal it is on SQLite and as `[[AlphaBeta]]` on
// Postgres — the linking documents are simply not found, the cascade
// rewrites nothing, and the rename succeeds leaving every link stale. A
// silent, dialect-dependent data defect.
//
// The explicit ESCAPE clause makes both dialects agree, rather than leaving
// SQLite correct by accident and Postgres wrong by default.
func escapeLikePattern(s string) string {
r := strings.NewReplacer(`\`, `\\`, `%`, `\%`, `_`, `\_`)
return r.Replace(s)
}
func (s *Store) updateLinksInTx(tx *sql.Tx, workspaceID, oldTitle, newTitle string) error {
// Find all documents in the workspace that contain [[oldTitle]]
searchTerm := "[[" + oldTitle + "]]"
rows, err := tx.Query(s.q(`
SELECT id, content FROM documents
WHERE workspace_id = ? AND deleted_at IS NULL AND content LIKE ?
`), workspaceID, "%"+searchTerm+"%")
WHERE workspace_id = ? AND deleted_at IS NULL AND content LIKE ? ESCAPE '\'
`), workspaceID, "%"+escapeLikePattern(searchTerm)+"%")
if err != nil {
return err
}
@@ -583,39 +610,48 @@ func (s *Store) updateLinksInTx(tx *sql.Tx, workspaceID, oldTitle, newTitle stri
// the column handed us and never a normalized form of it.
read string
rewritten string
// retained is what this row contributed to the running total, kept so
// the compare-and-set below can be given ITS share of the budget when
// a concurrent edit forces it to re-read and re-rewrite.
retained int64
}
var updates []docUpdate
var projected int64
var retained int64
for rows.Next() {
var du docUpdate
if err := rows.Scan(&du.id, &du.read); err != nil {
return err
}
// Project this linker's rewritten size BEFORE building it, and refuse
// on the running TOTAL across the linking set (BUG-2798).
//
// The quantity is exact rather than an estimate: strings.Replace
// substitutes every non-overlapping occurrence, so the output is
// len(read) + occurrences * (len(new) - len(old)) to the byte.
// Project what this linker will make the cascade HOLD, before building
// it, and refuse on the running TOTAL across the linking set
// (BUG-2798).
//
// The total is the right thing to bound, and a per-document cap would
// not be. Measured: with the title bound in place, one linker holding
// the largest body a 2 MiB request can carry projects 108,632,370
// bytes — 51.8x — and the cascade holds EVERY rewritten body in
// `updates` before it writes any of them, so k linkers hold k times
// that (measured linear at k = 1/2/4). A per-document cap of C still
// admits k * C, which is the same unbounded shape one level up.
// bytes of output — 51.8x — and the loop below holds EVERY rewritten
// body in `updates` before it writes any of them, so k linkers hold k
// times that (measured linear at k = 1/2/4). A per-document cap of C
// still admits k * C, which is the same unbounded shape one level up.
//
// RETAINED bytes, not output bytes. An earlier version of this guard
// summed only the projected output, which bounds nothing when the new
// title is SHORTER than the old one: renaming a 255-character title to
// a one-character title makes each 2 MiB linker project about 40 KiB
// while the cascade still retains its 2 MiB read for the
// compare-and-set, so hundreds of linkers exhaust memory while the
// counter reports well under the cap (codex round 1 P1). Both strings
// are alive at once, so both are counted.
//
// Refusing here rather than after the loop is what makes the bound
// real: at the moment of refusal the process holds the linkers already
// projected (under the cap by construction) plus this one row's body,
// counted (under the cap by construction) plus this one row's body,
// and none of the amplified output.
occurrences := int64(strings.Count(du.read, searchTerm))
projected += int64(len(du.read)) + occurrences*int64(len(newTitle)-len(oldTitle))
if projected > MaxRenameCascadeProjectedBytes {
return fmt.Errorf("%w: renaming to %q projects at least %d bytes across linked documents, maximum %d",
ErrRenameCascadeTooLarge, newTitle, projected, MaxRenameCascadeProjectedBytes)
du.retained = cascadeRetainedBytes(du.read, searchTerm, oldTitle, newTitle)
retained += du.retained
if retained > MaxRenameCascadeRetainedBytes {
return newRenameCascadeTooLargeError(newTitle, retained)
}
du.rewritten = links.ReplaceTitle(du.read, oldTitle, newTitle)
@@ -636,13 +672,40 @@ func (s *Store) updateLinksInTx(tx *sql.Tx, workspaceID, oldTitle, newTitle stri
}
for _, du := range updates {
if err := s.rewriteLinkerCAS(tx, du.id, du.read, du.rewritten, oldTitle, newTitle); err != nil {
// Budget handed to this document's compare-and-set: the cap, less
// everything the OTHER documents are holding. A retry that re-reads a
// body grown by a concurrent edit is checked against it, so the
// aggregate bound survives the retry path as well as the scan (codex
// round 1 P1 — the guard used to cover only the scan, and the retry
// called ReplaceTitle on an unbounded re-read).
budget := MaxRenameCascadeRetainedBytes - (retained - du.retained)
if err := s.rewriteLinkerCAS(tx, du.id, du.read, du.rewritten, oldTitle, newTitle, searchTerm, budget); err != nil {
return err
}
}
return nil
}
// cascadeRetainedBytes is what one linking document makes the cascade hold:
// the body it read (kept verbatim as the compare-and-set token) plus the body
// it will write.
//
// Exact rather than an estimate. strings.Replace substitutes every
// non-overlapping occurrence, so the rewritten length is
// len(read) + occurrences * (len(new) - len(old)) to the byte, and this
// function is the only place that arithmetic lives — the scan and the retry
// path must not be allowed to drift apart on it.
func cascadeRetainedBytes(read, searchTerm, oldTitle, newTitle string) int64 {
occurrences := int64(strings.Count(read, searchTerm))
rewritten := int64(len(read)) + occurrences*int64(len(newTitle)-len(oldTitle))
return int64(len(read)) + rewritten
}
func newRenameCascadeTooLargeError(newTitle string, retained int64) error {
return fmt.Errorf("%w: renaming to %q would hold at least %d bytes of linked-document content, maximum %d",
ErrRenameCascadeTooLarge, newTitle, retained, MaxRenameCascadeRetainedBytes)
}
// ErrLinkCascadeContention reports that a rename's link cascade lost its
// compare-and-set on the same linking document too many times in a row.
//
@@ -653,9 +716,8 @@ func (s *Store) updateLinksInTx(tx *sql.Tx, workspaceID, oldTitle, newTitle stri
// opposite of the truth here (codex round 2 on BUG-2785).
var ErrLinkCascadeContention = errors.New("store: link cascade lost the compare-and-set")
// ErrRenameCascadeTooLarge reports that a rename was refused because the work
// it projects across linking documents exceeds
// MaxRenameCascadeProjectedBytes.
// ErrRenameCascadeTooLarge reports that a rename was refused because the
// linked-document content it would hold exceeds MaxRenameCascadeRetainedBytes.
//
// Deliberately NOT in ErrLinkCascadeContention's family, and the distinction
// is the caller-visible one: contention means "someone else got there first,
@@ -665,37 +727,47 @@ var ErrLinkCascadeContention = errors.New("store: link cascade lost the compare-
// the projection so the caller can see what it asked for. Blurring the two
// vocabularies would tell a client to retry forever (BUG-2798, lead ruling
// day-63).
var ErrRenameCascadeTooLarge = errors.New("store: rename cascade exceeds the projected-output bound")
var ErrRenameCascadeTooLarge = errors.New("store: rename cascade exceeds the retained-content bound")
// MaxRenameCascadeProjectedBytes bounds the TOTAL bytes a single rename may
// project across every document linking the renamed title.
// MaxRenameCascadeRetainedBytes bounds the TOTAL linked-document content a
// single rename may hold in memory: for every linking document, the body read
// plus the body written.
//
// 16 MiB, and the basis is measured rather than picked:
// RETAINED rather than merely projected-output, because output alone is not
// the resource. A rename to a SHORTER title projects less output than its
// input while still holding every read body for the compare-and-set — so an
// output-only counter reports ~40 KiB per 2 MiB linker and bounds nothing in
// that direction (codex round 1). Counting both strings makes the cap a
// statement about resident memory, which is what actually runs out.
//
// 32 MiB, and both bounds of the gap are measured rather than picked:
//
// - Legitimate ceiling. In this development instance's database — a mature
// workspace set, 206 MB on disk — the ENTIRE corpus of wiki-linking
// content is 2,949 items totalling 10,077,476 bytes (largest single body
// 86,147 bytes). That is the absolute ceiling on any conceivable single
// cascade there: it assumes every wiki-linking document links the one
// title being renamed, which no real workspace does. 16 MiB is above
// 86,147 bytes). A cascade over all of it would retain read + rewritten,
// so ~20,154,952 bytes. That is the absolute ceiling on any conceivable
// single cascade there: it assumes every wiki-linking document links the
// one title being renamed, which no real workspace does. 32 MiB is ~1.6x
// that impossible worst case, so the guard cannot fire on honest use.
// (Measured on `items`, the live surface; the `documents` table in that
// instance is empty, which is why the proxy — the two carry the same kind
// of content through the same kind of cascade.)
// (Measured on `items`, the live surface; that instance's `documents`
// table is empty, which is why the proxy — the two carry the same kind of
// content through the same kind of cascade.)
// - Hostile floor. A single linking document holding the largest body a
// 2 MiB request can carry projects 108,632,370 bytes once the title bound
// is in place — 6.5x this cap — so the attack is refused at k = 1 and
// 2 MiB request can carry retains 110,729,522 bytes once the title bound
// is in place — 3.3x this cap — so the attack is refused at k = 1 and
// every k above it, rather than at some threshold count of documents.
// - Cost of the bound itself. The cascade holds read and rewritten bodies
// concurrently, so the cap is a promise about resident memory: at most
// ~2x this per in-flight rename, which is a bounded, budgetable number
// for a server that previously had none.
//
// The gap between the two figures is deliberate and wide: a cap has to be far
// enough above real use that nobody meets it by accident, and far enough
// below the hazard that meeting it costs nothing. 16 MiB is ~1.6x the former
// and ~0.15x the latter.
const MaxRenameCascadeProjectedBytes = 16 << 20
// The gap is deliberate and wide: a cap has to be far enough above real use
// that nobody meets it by accident, and far enough below the hazard that
// meeting it costs nothing.
//
// What it does NOT cover, stated so the next reader does not over-read it:
// this bounds ONE rename's linked-document content, not concurrent renames (N
// of them may each hold up to this), and not the base cost of a workspace
// whose linking documents are legitimately large — a cascade under the cap
// still allocates whatever it holds.
const MaxRenameCascadeRetainedBytes = 32 << 20
// cascadeRewriteAttempts bounds rewriteLinkerCAS's retry loop.
//
@@ -789,7 +861,7 @@ var cascadeRewriteAttempts = 3
// Both are pre-existing and neither is made worse here. They are recorded
// because the next reader's question is "is the cascade correct now", and the
// honest answer is "for the direction this bug named".
func (s *Store) rewriteLinkerCAS(tx *sql.Tx, id, read, rewritten, oldTitle, newTitle string) error {
func (s *Store) rewriteLinkerCAS(tx *sql.Tx, id, read, rewritten, oldTitle, newTitle, searchTerm string, budget int64) error {
expected := read
next := rewritten
for attempt := 0; attempt < cascadeRewriteAttempts; attempt++ {
@@ -821,6 +893,17 @@ func (s *Store) rewriteLinkerCAS(tx *sql.Tx, id, read, rewritten, oldTitle, newT
return err
}
// The re-read body is a NEW input, supplied by whoever won the race,
// and it is bounded by nothing this cascade has already checked. Its
// budget is the cap less what the other linkers are holding, so the
// aggregate bound holds across retries too — without this, an editor
// could grow a linker between the scan and the retry and walk the
// rename straight back into the amplification it was refused for
// (BUG-2798, codex round 1 P1).
if grown := cascadeRetainedBytes(current, searchTerm, oldTitle, newTitle); grown > budget {
return newRenameCascadeTooLargeError(newTitle, grown)
}
// A concurrent edit won. Rewrite ITS body rather than ours: replaying
// the original rewrite would reintroduce the very content this bug is
// about losing. If that edit already removed the link, ReplaceTitle is
+245 -30
View File
@@ -3,6 +3,7 @@ package store
import (
"errors"
"strings"
"sync"
"testing"
"github.com/PerpetualSoftware/pad/internal/models"
@@ -11,30 +12,33 @@ import (
// BUG-2798. A document rename rewrites `[[oldTitle]]` → `[[newTitle]]` in
// every linking document, and the cascade holds every rewritten body in memory
// before it writes any of them. Neither the title length nor the number of
// linking documents was bounded, so one rename could project more output than
// the process could hold — measured at 20,000x on the filing, and still 51.8x
// per document after the title bound alone.
// linking documents was bounded, so one rename could hold more content than the
// process could carry — measured at 20,000x amplification on the filing, and
// still 51.8x per document after the title bound alone.
//
// The title bound (models.MaxDocumentTitleRunes) is the cheap door. This file
// covers the wall: the cascade refuses when its projected TOTAL exceeds
// MaxRenameCascadeProjectedBytes.
// covers the wall: the cascade refuses when the TOTAL it would retain across
// the linking set — every read body plus every written body — exceeds
// MaxRenameCascadeRetainedBytes.
// linkerBody returns a body containing exactly n `[[A]]` occurrences, and the
// number of bytes renaming "A" to a title of length newLen would project for
// it: len(content) + occurrences * (len(new) - len(old)).
// number of bytes the cascade would RETAIN for it when renaming "A" to a title
// of length newLen: the body it reads plus the body it writes.
//
// Exact, not an estimate — strings.Replace substitutes every non-overlapping
// occurrence, so this is the output size to the byte.
// Deliberately computed here rather than by calling cascadeRetainedBytes — a
// test that reuses the implementation's arithmetic cannot catch that
// arithmetic being wrong.
func linkerBody(n, newLen int) (string, int) {
body := strings.Repeat("[[A]]", n)
return body, len(body) + n*(newLen-1)
rewritten := len(body) + n*(newLen-1)
return body, len(body) + rewritten
}
// TestRenameCascade_RefusesOnProjectedTOTAL_NotPerDocument is the load-bearing
// test, and its shape is the finding it encodes: a per-document cap would not
// close this bug.
//
// Every linking document here projects comfortably UNDER the cap on its own.
// Every linking document here retains comfortably UNDER the cap on its own.
// Only the total is over. A guard that tested each document in isolation would
// admit all three, allocate the sum, and pass a test that merely asserted "a
// huge single document is refused" — which is why this test asserts the
@@ -47,20 +51,20 @@ func TestRenameCascade_RefusesOnProjectedTOTAL_NotPerDocument(t *testing.T) {
// Size each linker so that linkers-1 of them fit under the cap and all of
// them do not. Derived from the cap rather than hardcoded, so the test
// keeps discriminating if the cap moves.
perDocTarget := (MaxRenameCascadeProjectedBytes / linkers) + (MaxRenameCascadeProjectedBytes / (linkers * 4))
occurrences := perDocTarget / (5 + (newTitleLen - 1))
body, perDocProjected := linkerBody(occurrences, newTitleLen)
perDocTarget := (MaxRenameCascadeRetainedBytes / linkers) + (MaxRenameCascadeRetainedBytes / (linkers * 4))
occurrences := perDocTarget / (5 + 5 + (newTitleLen - 1))
body, perDocRetained := linkerBody(occurrences, newTitleLen)
// Preconditions — these are what make the test discriminate. If either
// fails the test is no longer testing what its name says.
if perDocProjected >= MaxRenameCascadeProjectedBytes {
t.Fatalf("precondition: per-document projection %d must be UNDER the cap %d, "+
if perDocRetained >= MaxRenameCascadeRetainedBytes {
t.Fatalf("precondition: per-document retention %d must be UNDER the cap %d, "+
"otherwise a per-document guard would also pass this test",
perDocProjected, MaxRenameCascadeProjectedBytes)
perDocRetained, MaxRenameCascadeRetainedBytes)
}
if total := perDocProjected * linkers; total <= MaxRenameCascadeProjectedBytes {
t.Fatalf("precondition: total projection %d must EXCEED the cap %d",
total, MaxRenameCascadeProjectedBytes)
if total := perDocRetained * linkers; total <= MaxRenameCascadeRetainedBytes {
t.Fatalf("precondition: total retention %d must EXCEED the cap %d",
total, MaxRenameCascadeRetainedBytes)
}
s := testStore(t)
@@ -83,9 +87,9 @@ func TestRenameCascade_RefusesOnProjectedTOTAL_NotPerDocument(t *testing.T) {
}
// 2. Refused with the projection in the message. The only actionable
// information for a caller is what was projected against what is
// allowed; an error that says "too large" and nothing else sends them
// guessing.
// information for a caller is what the rename would hold against what
// is allowed; an error that says "too large" and nothing else sends
// them guessing.
if msg := err.Error(); !strings.Contains(msg, "maximum") || !strings.Contains(msg, "bytes") {
t.Errorf("error message lacks the projection: %q", msg)
}
@@ -142,19 +146,19 @@ func TestRenameCascade_AllowsAnOrdinaryRename(t *testing.T) {
// TestRenameCascade_RefusesTheSingleDocumentAttack covers the k=1 case
// directly: one linking document holding the largest body a 2 MiB request can
// carry still projects 108,632,370 bytes once the title bound is in place
// (measured), which is 6.5x the cap. The attack is refused at every k, not
// only at a threshold count of documents.
// carry still retains 110,729,522 bytes once the title bound is in place
// (108,632,370 written plus the 2,097,152 read), which is 3.3x the cap. The
// attack is refused at every k, not only at a threshold count of documents.
//
// Kept separate from the total-versus-per-document test because it is the one
// case a per-document guard WOULD catch — asserting both makes it explicit
// that the total guard is a superset, not a replacement of unclear scope.
func TestRenameCascade_RefusesTheSingleDocumentAttack(t *testing.T) {
const newTitleLen = 255
occurrences := (MaxRenameCascadeProjectedBytes / (5 + (newTitleLen - 1))) * 2 // 2x the cap
body, projected := linkerBody(occurrences, newTitleLen)
if projected <= MaxRenameCascadeProjectedBytes {
t.Fatalf("precondition: single-document projection %d must exceed the cap %d", projected, MaxRenameCascadeProjectedBytes)
occurrences := (MaxRenameCascadeRetainedBytes / (5 + 5 + (newTitleLen - 1))) * 2 // 2x the cap
body, retained := linkerBody(occurrences, newTitleLen)
if retained <= MaxRenameCascadeRetainedBytes {
t.Fatalf("precondition: single-document retention %d must exceed the cap %d", retained, MaxRenameCascadeRetainedBytes)
}
s := testStore(t)
@@ -168,3 +172,214 @@ func TestRenameCascade_RefusesTheSingleDocumentAttack(t *testing.T) {
t.Fatalf("rename: got %v, want ErrRenameCascadeTooLarge", err)
}
}
// TestRenameCascade_CountsRetainedBytesNotJustOutput pins codex round 1's P1
// against the guard that shipped in the first commit, which summed only the
// PROJECTED OUTPUT.
//
// The counterexample is a rename to a SHORTER title. Every linker here holds a
// large body, but the rewrite shrinks it, so an output-only counter reports a
// small number while the cascade still retains every read body for its
// compare-and-set. Under the old guard the total below reported far under the
// cap and the rename proceeded; the retained-bytes guard refuses it.
//
// This is why the direction of the rename matters and why the constant is
// named for retention rather than for projection.
func TestRenameCascade_CountsRetainedBytesNotJustOutput(t *testing.T) {
// Old title long, new title short — the shrinking direction.
oldTitle := strings.Repeat("O", 200)
newTitle := "n"
// Each linker is ~2 MiB of `[[<200-char title>]]`, the shape a single 2 MiB
// request can deliver.
occurrencesPerDoc := (2 << 20) / (len(oldTitle) + 4)
body := strings.Repeat("[["+oldTitle+"]]", occurrencesPerDoc)
// Enough linkers that the RETAINED total is over the cap while the
// projected OUTPUT total stays under it. That gap is the finding.
linkers := (MaxRenameCascadeRetainedBytes / len(body)) + 2
outputTotal := 0
retainedTotal := 0
for i := 0; i < linkers; i++ {
rewritten := len(body) + occurrencesPerDoc*(len(newTitle)-len(oldTitle))
outputTotal += rewritten
retainedTotal += len(body) + rewritten
}
if outputTotal > MaxRenameCascadeRetainedBytes {
t.Fatalf("precondition: projected OUTPUT total %d must stay UNDER the cap %d, "+
"otherwise an output-only guard would also refuse this and the test proves nothing",
outputTotal, MaxRenameCascadeRetainedBytes)
}
if retainedTotal <= MaxRenameCascadeRetainedBytes {
t.Fatalf("precondition: RETAINED total %d must exceed the cap %d", retainedTotal, MaxRenameCascadeRetainedBytes)
}
s := testStore(t)
ws := createTestWorkspace(t, s, "CascadeShrinking")
target := createTestDoc(t, s, ws.ID, oldTitle, "the document being renamed")
for i := 0; i < linkers; i++ {
createTestDoc(t, s, ws.ID, "Linker"+string(rune('a'+i)), body)
}
if _, err := s.UpdateDocument(target.ID, models.DocumentUpdate{Title: &newTitle}); !errors.Is(err, ErrRenameCascadeTooLarge) {
t.Fatalf("shrinking rename: got %v, want ErrRenameCascadeTooLarge — an output-only guard admits this", err)
}
}
// TestRenameCascade_FindsLinkersWhoseTitleContainsABackslash pins codex round
// 1's P2. POSTGRES ONLY, and skipped loudly elsewhere rather than passing
// quietly — the defect is a DIALECT DIVERGENCE, and SQLite is the dialect that
// was accidentally right.
//
// The cascade finds linkers with `content LIKE ?`. Postgres LIKE treats
// backslash as the default escape character; SQLite LIKE has no default escape
// character at all. So an unescaped search term for a title containing `\` was
// searched for as the literal it is on SQLite, and as a DIFFERENT literal on
// Postgres — `[[Alpha\Beta]]` became `[[AlphaBeta]]`, the linking documents
// were not found, the cascade rewrote nothing, and the rename reported success
// leaving every link pointing at a title that no longer exists.
//
// A green run on SQLite is therefore a property of the DSN, not evidence about
// this fix, which is why this skips instead.
func TestRenameCascade_FindsLinkersWhoseTitleContainsABackslash(t *testing.T) {
s := testStore(t)
if s.dialect.Driver() != DriverPostgres {
t.Skip("asserts a Postgres LIKE-escape property; SQLite LIKE has no default escape character, so the unescaped pattern is accidentally correct there")
}
ws := createTestWorkspace(t, s, "CascadeLikeBackslash")
title := `Alpha\Beta`
target := createTestDoc(t, s, ws.ID, title, "the document being renamed")
linker := createTestDoc(t, s, ws.ID, "RealLinker", "see [["+title+"]] here")
newTitle := "Renamed"
if _, err := s.UpdateDocument(target.ID, models.DocumentUpdate{Title: &newTitle}); err != nil {
t.Fatalf("rename: %v", err)
}
got, err := s.GetDocument(linker.ID)
if err != nil {
t.Fatalf("read back linker: %v", err)
}
if want := "see [[Renamed]] here"; got.Content != want {
t.Errorf("the linker was not rewritten — the cascade's LIKE pattern did not find it:\n got: %q\nwant: %q",
got.Content, want)
}
}
// TestRenameCascade_DoesNotSpendTheBudgetOnDocumentsThatDoNotLinkTheTitle is
// the rest of the class codex's backslash finding was an instance of
// (CONVE-18): `%` and `_` are LIKE wildcards in BOTH dialects, so an unescaped
// search term for a title containing them selects documents that do not link
// it.
//
// Over-matching cannot be caught by asserting content — the extra rows rewrite
// to themselves, because ReplaceTitle looks for the literal. It is observable
// through the guard, which is computed from this result set: unrelated
// documents inflate the retained total, and a rename that fits the cap is
// refused because of content it was never going to touch. That is the harm,
// and it is what this asserts.
//
// The first version of this test asserted the decoy's content was untouched
// and passed against the unescaped pattern — a vacuous green. Recorded here
// because the fix was to find the observable consequence, not to trust the
// mechanism.
func TestRenameCascade_DoesNotSpendTheBudgetOnDocumentsThatDoNotLinkTheTitle(t *testing.T) {
for _, tc := range []struct {
name string
title string
decoy string // matches the title read as a PATTERN, not as a literal
}{
{"percent", `Alpha%Beta`, `[[AlphaXYZBeta]]`},
{"underscore", `Alpha_Beta`, `[[AlphaZBeta]]`},
} {
t.Run(tc.name, func(t *testing.T) {
s := testStore(t)
ws := createTestWorkspace(t, s, "CascadeLike"+tc.name)
target := createTestDoc(t, s, ws.ID, tc.title, "the document being renamed")
linker := createTestDoc(t, s, ws.ID, "RealLinker", "see [["+tc.title+"]] here")
// Decoys big enough that INCLUDING them blows the cap, while the
// real linker alone is negligible. With the pattern escaped they
// are not selected and the rename is comfortably under budget.
decoyBody := strings.Repeat("x", 1<<20) + " " + tc.decoy
decoys := (MaxRenameCascadeRetainedBytes / len(decoyBody)) + 2
for i := 0; i < decoys; i++ {
createTestDoc(t, s, ws.ID, "Decoy"+string(rune('a'+i)), decoyBody)
}
newTitle := "Renamed"
if _, err := s.UpdateDocument(target.ID, models.DocumentUpdate{Title: &newTitle}); err != nil {
t.Fatalf("a rename well under the cap was refused because of documents that do not link it: %v", err)
}
got, err := s.GetDocument(linker.ID)
if err != nil {
t.Fatalf("read back linker: %v", err)
}
if want := "see [[Renamed]] here"; got.Content != want {
t.Errorf("the real linker was not rewritten:\n got: %q\nwant: %q", got.Content, want)
}
})
}
}
// TestRenameCascade_RetryRecheckesTheBudgetAgainstTheGrownBody pins codex round
// 1's other P1: the guard used to cover only the cascade's SCAN, while the
// compare-and-set's retry path re-read a linker's body and called ReplaceTitle
// on it with no bound at all.
//
// The re-read body is a NEW input supplied by whoever won the race, so a
// content edit landing inside the cascade's window could grow a linker from
// harmless to enormous and walk the rename straight back into the
// amplification it would have been refused for.
//
// POSTGRES ONLY, for the same structural reason as
// TestUpdateDocument_CascadeDoesNotOverwriteConcurrentEdit: SQLite's
// `_txlock=immediate` DSN takes the write lock at BEGIN and holds it across the
// cascade's whole read→write window, so a concurrent edit cannot commit inside
// it. A green run there would be a property of the DSN, not evidence about this
// guard.
func TestRenameCascade_RetryRecheckesTheBudgetAgainstTheGrownBody(t *testing.T) {
s := testStore(t)
if s.dialect.Driver() != DriverPostgres {
t.Skip("needs a concurrent edit to commit inside the cascade's read→write window; SQLite's BEGIN IMMEDIATE closes it structurally")
}
ws := createTestWorkspace(t, s, "CascadeRetryBudget")
target := createTestDoc(t, s, ws.ID, "A", "the document being renamed")
// Small at scan time — the cascade counts a few bytes and proceeds.
linker := createTestDoc(t, s, ws.ID, "Linker", "before [[A]] after")
// The winner's body is over the cap on its own, so the retry's re-read is
// the first and only place this can be caught.
newTitleLen := 255
occurrences := (MaxRenameCascadeRetainedBytes / (5 + 5 + (newTitleLen - 1))) * 2
grownBody, grownRetained := linkerBody(occurrences, newTitleLen)
if grownRetained <= MaxRenameCascadeRetainedBytes {
t.Fatalf("precondition: the grown body's retention %d must exceed the cap %d", grownRetained, MaxRenameCascadeRetainedBytes)
}
var once sync.Once
var editErr error
s.afterLinkCascadeRead = func(string) {
once.Do(func() {
// Content-only, so it takes no rename lock and can commit inside
// the cascade's window (BUG-2785's seam, same mechanism).
_, editErr = s.UpdateDocument(linker.ID, models.DocumentUpdate{Content: &grownBody})
})
}
defer func() { s.afterLinkCascadeRead = nil }()
newTitle := strings.Repeat("T", newTitleLen)
_, err := s.UpdateDocument(target.ID, models.DocumentUpdate{Title: &newTitle})
if editErr != nil {
t.Fatalf("the concurrent edit itself failed, so this run never exercised the retry path: %v", editErr)
}
if !errors.Is(err, ErrRenameCascadeTooLarge) {
t.Fatalf("rename: got %v, want ErrRenameCascadeTooLarge — the retry re-read an unbounded body", err)
}
}