ProseMirror lets a mark sit on an inline atom (a hard break, an inline
attachment image or chip). Bolding, striking or linking a range that held
one marked it, and markdown or HTML parsing marked it when the source
wrapped it (**a\
b**). y-tiptap never writes marks on non-text inline nodes to the Y.Doc,
so that tab's editor held a document its own Y.Doc did not. It flushed one
body while every other tab, a reload and the materializer derived another.
This is the ~0.7% tab divergence materializerParity found. Measured: both
Y.Docs identical, and only the marking tab's ProseMirror doc differs; a
one-tab repro with no relay reproduces it 4 of 4.
InlineAtomNoMarks (extensions/inlineAtomNoMarks.ts) is an appendTransaction
plugin that strips marks from inline non-text nodes in the same update. It
fires only on AddMarkStep, AddNodeMarkStep, or a replace whose slice holds a
marked atom, so typing pays nothing. It is registered in Editor.svelte and
the headless materializer. No node spec or persisted shape changes, so no
SCHEMA_VERSION question; NodeSpec.marks cannot do this, since it governs a
node's content (measured: it left the marks in place).
One-time normalisation: a body holding a mark across a hard break
(**a\
b**) flushes as **a**\
**b**, the form every Y-derived door already produced, so this touches only
bodies that already flip-flopped. It is stable from then on.
materializerParity: MAX_TAB_DISAGREEMENTS 2 -> 0. The corpus is regenerated;
seed-300922 is now a case. The generator is not deterministic run to run
(Yjs client ids), so a regen also rewrites a couple of other cases'
expectations; each corpus is self-consistent.
Claude-Session: https://claude.ai/code/session_01Q9WssCjFuHjSfLABoW4tmS
The editor parsed markdown with html enabled, so `<Name>` in prose was a
tag the schema could not hold: dropped on the next flush (one keystroke,
or an agent's write applied by an open tab, which answered 200 while the
text vanished). `<Table>` became an empty table that was then removed.
htmlProse.ts derives the allowlist from the schema's own parse rules, so
the HTML the editor itself writes (tables with spans, HTML-only marks)
still reads back:
- an inline tag passes only if written lower case, matching an inline
node's or a mark's rule, and closing one that passed; anything else
is literal text;
- an HTML block passes only if every tag in it matches a rule; otherwise
it is kept whole as an HtmlBlock (stored as a ```html fence);
- comments are visible text.
PadText escapes `<` only where the parse would read a tag, so
`Promise<T>` round-trips byte for byte. Both the editor and the headless
materializer take the same extensions, right after StarterKit.
materializerParity now skips and counts cases where the two live tabs
converge to different documents (BUG-3568, pre-existing: 2 of 300 seeds
on main), at most 2 of 50, and still requires the materializer to match
one of them; such cases stay out of the Go corpus (lead ruling).
Claude-Session: https://claude.ai/code/session_01FAVXA5mJ5BNhKmv95jxQkQ
* chore(web): adopt Tiptap 3.31.4 and regenerate the materializer corpus (TASK-3421)
3.31.4's TableCell/TableHeader renderHTML stop rendering a default
colspan="1" / rowspan="1". parseHTML is unchanged, so a stored body
carrying them parses to the same document. Replaying all 50 old corpus
op-logs through the 3.31.4 bundle reproduces the old output exactly once
those attributes are removed (and fails on 28 cases without removing them).
Claude-Session: https://claude.ai/code/session_01Q9WssCjFuHjSfLABoW4tmS
* test(web): pin that a pre-3.31.4 table body still dedupes on open (TASK-3421)
Claude-Session: https://claude.ai/code/session_01Q9WssCjFuHjSfLABoW4tmS
A wiki link's display text was mangled on its way through the editor, and
the first save of any item linking such a title pinned the mangled text as
a display override, with no rename involved (a census found 76 stored
overrides of this shape among 107 that differ from their target's title).
Load: escapeMarkdownLinkText escaped only brackets and backslashes, so the
html-enabled parser read the title as markdown. `*real*` became emphasis,
`<topic>` was dropped as a tag, `>` was decoded. It now backslash-escapes
\ [ ] * _ ~ ` < > &, all ASCII punctuation CommonMark lets a backslash
escape.
Save: unescapeMarkdownLinkText undid only bracket and backslash escapes,
while the serializer also backslash-escapes other punctuation and writes
< > & as entities. It now undoes both, in one pass.
One residue is intrinsic and pinned by a test: the serializer writes a
literal `>` and a literal `>` as the same bytes, so a title containing
entity-shaped text still becomes an override (0 of 9,620 live titles do).
The materializer corpus: two expectations change (a backtick and `<b>` in
link text, both this fix), taken from the goja bundle's output. A full
regeneration was not used because the generator is nondeterministic: with
unchanged code it rewrites all 50 cases' rows and 1 expected value.
Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC
* feat(materialize): headless op-log materializer bundle, goja runner and worker (TASK-2198 U2)
Builds the editor's own JS (Yjs -> y-tiptap -> Tiptap schema -> tiptap-markdown
-> the tab's flush pipeline) into one IIFE, embeds it, and runs it in goja. Not
wired into the server yet (U3 supervisor, U4 write path).
- web/src/lib/collab/materializer/: bundle entry (extension list mirrors
Editor.svelte's collab branch via the U1 shared modules), op-log replay
mirroring wsProvider's MESSAGE_SYNC arm, linkedom DOM shim and polyfills
from the spike, and a vendored patch of prosemirror-markdown
MarkdownSerializerState.atBlank (quadratic under goja; equivalent).
- web/scripts/build-materializer.mjs (esbuild) -> web/build-materializer/,
run by `npm run build`; embedded as pad.MaterializerJS.
- internal/materialize: Runner (Go-native atob/btoa/TextEncoder/TextDecoder/
getRandomValues, schema-version check, ctx interrupt with a clean VM after),
and RunWorker's length-prefixed JSON protocol behind the hidden
`pad __materialize-worker` command.
- Parity: vitest drives two real Editor.svelte mounts and compares
materialize() with the tab's flush on 50 seeded cases; the same cases are
committed as testdata/corpus.json and replayed in goja (TestCorpusParity).
- CI Go jobs, release tests, Docker and Nix produce the bundle; linkedom and
esbuild are new devDependencies.
Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC
* test(materialize): pin the atBlank patch's equivalence under V8 too (TASK-2198 U2, codex r1)
Codex r1 claimed upstream's /(^|\n)$/ is true for a trailing CR in JS. It is
not (checked in V8: false for "\r" and "a\u2028", as the patch returns),
but the only equivalence test ran under goja and the patch header cited a
V8-side test that did not exist. This adds it, with the line-terminator edge
cases, and it fails when the patch checks CR instead of LF.
Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC
* build(nix): vendorHash for goja (TASK-2198 U2)
Taken from the Nix job's recompute on c7fa1931 (specified sha256-iEqv…,
got sha256-P1bQ…), so main does not depend on the heal job after merge.
Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC
xarmian