* test(attachments): record the Void-beyond-window limit, and correct three comments that claimed more than their evidence (TASK-2976) The first commit of BUG-2963 PR B, before any extension-trust work: round 6's three items, deliberately kept out of PR A so its verified tip would not move, plus the nix terminator-comment correction folded in from BUG-2974's trail. 1. TestMatroskaDocTypeBeyondTheWindowIsWebM records a LIMITATION. A Void element is legal anywhere in an EBML header and may be any size; make one larger than the 512 bytes this door reads and the DocType behind it is not in the input at all, so the parse finds nothing and the stdlib's video/webm stands. Nothing got worse — the same file was video/webm before the DocType read existed, and video/webm's own allowlist entry permits inline serving — and no larger window fixes it, since Void may be larger still. The fixture is the ordinary FFmpeg Matroska with a 560-byte Void spliced into its header and the header size widened to match, the same construction as matroska-void-padded.head512; the complete file reads as matroska,webm under ffprobe, and what is committed is its first 512 bytes, so the DocType is absent by construction rather than by truncation accident. 2. TestTarWinsAPrefixCollision no longer calls the collision "asymmetric". That was the round-4 premise round 5 refuted with flac-ustar-in-comment.head512 — a FLAC's Vorbis COMMENT tags are arbitrary UTF-8, so real audio carries "ustar" at offset 257 as readily as a real tar carries an audio marker at offset zero. The implementation and the fixtures already said so; the stale word survived in the place a reader looks first. The comment now gives the real reason tar leads the default order, which is weaker: its magic sits at a fixed offset rather than at a prefix. 3. The seven-byte textual-AAC input's comment says which of the two things it is. It is a valid ADTS sync and layer signature that the stdlib reads as text — not a decodable AAC, and nothing in the test establishes that it is. What review established is the case it stands in for, and the comment now separates the two. 4. The nix loop-terminator comments named the wrong mechanism (BUG-2974, observed day 62). The heal push cannot loop because a push made with the default GITHUB_TOKEN creates no workflow runs at all — `gh run list --branch main` spanscbfc073e->31d11e76->0b16be49and skips the heal commit249a8f87entirely. The corrected-tree argument the comments gave is true and UNEXERCISED; it is now written as the backstop, with the tripwire that it becomes load-bearing the moment that push uses a PAT or app token. The per-commit query is not the instrument for this and the comment says so: it also returned nothing for0b16be49, which has runs. Also swept: the fixture README's "the two EBML entries" was already false at four, and nix.yml's "THAT DISTINCTION" lost its referent once paragraphs landed above it. The new fixture joins the FuzzSniffMIME seeds. go test ./internal/attachments/ green; nix/heal-vendor-hash_test.sh 20 passed, 0 failed. Claude-Session: https://claude.ai/code/session_01GqaEDuCtRiSJfa7eppWecn * feat(attachments): the ISO-BMFF audio/video split — .mov and .m4a stop being refused (TASK-2976, BUG-2963 F4) Both iPhone-default containers were refused at upload while their types sat on the allowlist. Measured on06ccabddbefore the change: clip.mov (qt brand) -> sniff application/octet-stream -> mime_not_allowed clip.m4a (isom brand) -> sniff video/mp4 -> mime_extension_mismatch clip.m4a (M4A brand) -> sniff video/mp4 -> mime_extension_mismatch Three legs, per the day-62 per-family ruling on BUG-2963: 1. Major brand "qt " -> video/quicktime. The stdlib's mp4 matcher wants a brand beginning "mp4" and a QuickTime file carries none, so it had no opinion at all. This ADDS a detection, exactly as the still-image brands do. 2. Major brand "M4A " -> audio/mp4. This is the one place this package OVERRIDES a type the standard library named: an .m4a's compatible brands routinely include "mp41", so the stdlib answers video/mp4 from a compatible brand, having no way to weigh it against the major one. The major brand is the file's own statement about itself. The override is bounded — both types are allowlisted, both render inline, and the only thing that moves is the CATEGORY, which decides whether the UI offers an audio player or a video one. It is read from the MAJOR brand alone and BEFORE the mp4-yield, which is the whole mechanism: the yield would otherwise return "" on that same mp41 brand before the check could speak. Both facts are stated in the code. 3. isom-branded bytes named .m4a -> audio/mp4, in ValidateUpload beside the zip+document branch and as the same kind of trust one family over: the BYTES establish the container, the FILENAME chooses only the spelling within it. No track reads, per the ruling. sniffISOBMFFImage is now sniffISOBMFF. It answers for audio and video now, and a name saying Image while it returns audio/mp4 is the exact defect class this bug keeps producing. The controls are the test. An accept-only test is satisfied by a rule that trusts .m4a outright, which is a much larger claim than the one granted, so each leg carries what the STDLIB said about the same bytes (the thing separating "adds a detection" from "overrides one"), and the extension leg carries four: the same bytes named .mp4 stay video, the same bytes with no extension stay video, a WebM named .m4a is still refused (the trust does not reach outside the MP4 family), and PNG bytes named .m4a are still refused (the extension introduces no type). A synthetic ftyp box with major isom and "M4A " among its COMPATIBLE brands is refused too — reading a category out of the compatible list is a wider rule than the one granted. Fixtures: quicktime.head512, m4a-brand.head512, m4a-isom.head512 — real FFmpeg n7.1 output, provenance in the fixture README. Swept prose that said this pre-check is still-images-only (SniffMIME's doc comment, the mime_isobmff.go header). go test ./internal/attachments/ green. Mutation matrix next, then F5. Claude-Session: https://claude.ai/code/session_01GqaEDuCtRiSJfa7eppWecn * refactor(attachments): the .m4a branch's category guard was dead — assert the invariant it stood for instead (TASK-2976) A mutation run says the guard survives removal, and it is right: the branch answers for exactly .m4a, so extEntry.Category == CategoryAudio can never differ from the ext test beside it. (The zip branch's identical-looking guard IS load-bearing, because that branch answers for many extensions at once.) What the branch actually depends on is that .m4a maps to an allowlisted audio entry, since it returns that entry. That is now asserted directly in TestBUG2963F4M4AExtensionTrust, so a remap of .m4a fails a test instead of quietly retyping MP4 bytes. Dead code replaced by a tested invariant, rather than kept as defence in depth it cannot provide. Claude-Session: https://claude.ai/code/session_01GqaEDuCtRiSJfa7eppWecn * feat(attachments): the text family — the extension chooses which text, category unchanged (TASK-2976, BUG-2963 F5) text/plain is the stdlib saying "these bytes are text" and nothing more; it has no signature separating Markdown from YAML from JavaScript because at the byte level there is none to have. So when the sniff is text/plain and the extension maps to an allowlisted TEXT entry, that entry is what gets stored. Per the day-62 ruling. Category-preserving is the whole safety argument: nothing becomes an image, an archive or a document by being renamed, and a mapping that is not an allowlisted text entry does not fire — .svg still reaches extension_blocked rather than being quietly stored as text. TestTextFamilyStillStoresAsPlain is now TestBUG2963F5TextFamily. It recorded this boundary and said in its own failure message that F5 landing should move it; it kept the same three inputs and changed the expectation, which is what "move it" meant. THE SERVING BUCKET MOVES FOR EVERY TYPE F5 TOUCHES, NOT ONLY .js. text/plain is the only member of the text family in inlineSafe, so choosing any other text spelling takes the file out of inline serving and into Content-Disposition: attachment. The ruling weighed that for .js, where the move to RenderForceDownload is the point. It applies equally to .md, .csv, .json, .yaml and .toml, where it is a side effect — and it turns out to be the honest one: the client's own DR-5 already says every text/* subtype but plain text "is downloaded or rendered inconsistently across browsers", and those types only reached the browser-preview affordance because the server was storing them as text/plain. Whether any of the five belongs in inlineSafe is an allowlist decision and is not taken here. The in-app markdown preview is unaffected: it fetch()es the bytes and renders them itself, which an attachment disposition does not impede, and it now matches on the MIME rather than falling back to the extension. Its prose in web/src/lib/attachments/display.ts said an uploaded .md "is stored as text/plain" as a present-tense measured fact; that is now history, and the fallback it justifies is what PRE-EXISTING rows still depend on, so the comment says that instead (CONVE-23, and its mirror in display.test.ts). Controls, since an accept-only test is satisfied by "trust text extensions outright": .txt / an unmapped extension / no extension at all stay text/plain; PNG bytes named .md are still a mismatch; text bytes named .mp3 are still a mismatch (an allowlisted mapping in another category is the dangerous shape and the one the category guard is for); .svg still fails extension_blocked; and the .js leg asserts RenderMode == RenderForceDownload against the same bytes named .txt, not merely that the type changed. go test ./internal/attachments/ green. Mutation matrix next. Claude-Session: https://claude.ai/code/session_01GqaEDuCtRiSJfa7eppWecn * feat(attachments): the legacy Office trio and RTF — two more families the door refused (TASK-2976, BUG-2963) CFB. The container legacy Office wrote .doc/.xls/.ppt into has no stdlib signature, so every one of those files sniffed application/octet-stream and was refused while all three types sat on the allowlist. The branch reads eight fixed bytes and says only "this is a CFB container", because that is all eight bytes can say: .msi installers and Visio files wear the same header, and telling them apart is a directory-stream walk this door does not do. So the extension chooses which of the three REVIEWED types is stored, exactly as the zip+document branch does one container family over. The extension list is written out rather than derived from extMIMEMap. The question is not "does this extension map to a document" — .vsd and .msi are CFB too — it is "is this one of the three the allowlist reviewed", and that is a list, not a predicate. .docx is the sharp control: a document extension whose type is allowlisted but whose container is zip, refused on a CFB header. RTF. Printable ASCII, so the stdlib answers text/plain — an OPINION, not the absence of one — and a .rtf was refused as a mismatch against its own type. This one needs no extension: five fixed bytes at offset zero, one of them a version digit the spec fixes, is a stronger signature than the three-byte "ID3" the stdlib uses for audio/mpeg. So it is a refinement in SniffMIME rather than extension trust in ValidateUpload, and the tests assert it under three filenames including .bin, which is what says so. text/plain therefore joins the refinement switch, and the comment there says what makes that case narrower than the octet-stream one: it admits exactly one signature, and the bar for a second is the argument at validRTFStream rather than the list's existence. One behaviour change beyond the refusals: an EXTENSIONLESS RTF used to store as text/plain and serve inline; it now stores as application/rtf and serves as an attachment. More conservative, and named here because it is the one leg that changes a file that was previously accepted. CONVE-23 sweep, three sites, all of them prose this change made false: SniffMIME's "two refinements", and the F6 comment claiming text/javascript "is not reachable EITHER" — F5 made it reachable, which is the whole point of including .js. The CFB fixture is hand-built and the test says why that is sound here and not elsewhere: the branch reads eight fixed bytes, so eight bytes exercise all of it, unlike the ISO-BMFF and EBML fixtures where the code walks structure an encoder produces. go test ./internal/attachments/ green. Claude-Session: https://claude.ai/code/session_01GqaEDuCtRiSJfa7eppWecn * fix(attachments): gate the CFB branch on the stdlib verdict, not the refined one (TASK-2976, BUG-2963) A surviving mutant found this. Dropping the octet-stream gate changed no test, which said the gate was either dead or untested; it is untested, and it was also gating on the wrong value. A CFB file's 512-byte header is sector data, so one can carry "ustar" at offset 257 and be refined to application/x-tar by this package's own magic table. Gated on the REFINED verdict, that file is refused under its own .doc name — the false refusal BUG-2963 exists to remove. Gated on the stdlib's, the question is whether anything IDENTIFIED the bytes, and nothing did. This is the reasoning the .aac branch already carries, in the same function, for the same collision from the other side. The test is a CFB header with ustar spliced into its sector data, and it asserts the premise (these bytes really do refine to x-tar) before asserting the outcome, so it cannot pass by the collision quietly failing to happen. Claude-Session: https://claude.ai/code/session_01GqaEDuCtRiSJfa7eppWecn
Attachment fixtures
Real files, not hand-built headers. A sniffer's whole job is to recognise what
encoders actually emit, so a fixture I typed from a spec would test my reading
of the spec (BUG-2961's tests, mime_isobmff_test.go).
| file | produced by | ftyp brands |
|---|---|---|
apple-still-heic.head512 |
sips -s format heic on macOS 26 — first 512 bytes only |
major heic; compatible mif1 MiPr miaf MiHB heic |
apple-still.avif |
sips -s format avif on macOS 26, complete file (509 B) |
major avif; compatible MiPr avif miaf mif1 |
still.avif |
heif-enc -A -q 60 (libheif 1.20.2 / aom 3.12.1) on Linux, complete file |
major avif; compatible mif1 avif miaf |
still-mif1.heif |
heif-enc -U (libheif 1.20.2, uncompressed codec) on Linux, complete file |
major mif1; compatible mif1 miaf |
The .head512 file is a prefix because that is the entire input domain of
SniffMIME — it reads at most 512 bytes and the ftyp box is the first box in
the file. It is named for what it is so nobody mistakes it for a loadable image.
Two gaps, recorded rather than papered over:
- No
sips-produced HEIC beyond 512 bytes and no iPhone-camera capture. Both Apple fixtures come fromsips, not from a camera: same container and brand set, different encoder pipeline. - No file whose MAJOR brand is
mif1from Apple.sips -s format heifwrites no file on that machine. The libheifstill-mif1.heifcovers the major-mif1shape from a different encoder instead of a guessed one.
BUG-2963 fixtures — formats the sniffer learned to recognise
Same rule as above: real encoder output, truncated to the 512 bytes
SniffMIME actually reads. bzip2.head512 and sevenzip.head512 are shorter
than 512 because the whole file is.
| file | produced by | what it proves |
|---|---|---|
tar.head512 |
GNU tar -cf (tar 1.34, Linux) |
ustar at offset 257 — the magic is not at the start, which is why no prefix matcher finds it |
bzip2.head512 |
bzip2 -c (1.0.8), complete file (88 B) |
BZh plus the block-size digit |
sevenzip.head512 |
py7zr 1.1.3, complete file (192 B) |
the six-byte 7z signature |
flac.head512 |
libsndfile 1.2.2 via soundfile, FLAC format |
the fLaC stream marker |
aac-adts.head512 |
FFmpeg 7.1 -c:a aac -f adts |
an ADTS sync (ff f1) — the one signature weak enough to need its extension |
matroska.head512 |
FFmpeg 7.1 -f matroska |
EBML magic with DocType matroska at offset 24 |
webm.head512 |
FFmpeg 7.1 -f webm |
EBML magic with DocType webm at offset 24 — the control that stops the DocType read from answering Matroska for everything |
avi.head512 |
FFmpeg 7.1 -f avi |
RIFF/AVI, which the stdlib names video/avi against the allowlist's video/x-msvideo |
PR B fixtures — the audio/video split (F4)
| file | produced by | what it proves |
|---|---|---|
quicktime.head512 |
FFmpeg n7.1 -f mov |
major brand qt . The stdlib's mp4 matcher wants a brand beginning mp4 and this carries none, so it sniffs application/octet-stream — recognising it ADDS a detection rather than overriding one |
m4a-brand.head512 |
FFmpeg n7.1 -f mp4 -brand "M4A " |
major brand M4A , compatible M4A iso2 mp41. The stdlib answers video/mp4 off that mp41, so this is the one fixture whose recognition OVERRIDES the standard library, on the major brand |
m4a-isom.head512 |
FFmpeg n7.1 -f mp4 |
major brand isom, compatible isom iso2 mp41 — an audio-only MP4 whose brands name no category at all. It is the input for the extension-trust leg: the bytes reach video/mp4 and only the .m4a filename can choose the audio spelling |
All three are one second of sine=frequency=440 encoded to AAC, truncated to
the 512 bytes SniffMIME reads.
Fixtures for files the recogniser must ACCEPT
Real files that three rounds of structural validation refused. Each is a file a user legitimately has, and refusing them is the defect BUG-2963 exists to fix — reintroduced, for a while, by its own fix.
| file | produced by | what it proves |
|---|---|---|
tar-pax.head512 |
tar --format=pax -cf (tar 1.34) |
a PAX archive leads with a metadata header, so a full parse needs more blocks than this door ever reads |
tar-gnu-longname.head512 |
tar --format=gnu -cf with a 144-character member name |
same shape, via GNU's long-name header |
flac-zero-sample-rate.head512 |
flac.head512 with the sample-rate field zeroed |
RFC 9639 permits a zero sample rate for non-audio samples and still registers the result as audio/flac |
Adversarial fixtures — the round-1 findings, kept as tests
These exist to be REFUSED or to be typed correctly against a check that used to get them wrong. Each was produced by an adversarial review round that broke the first version of these signatures.
| file | produced by | what it proves |
|---|---|---|
tar-bmp-firstmember.head512 |
GNU tar -cf on a file named BM.txt |
a structurally valid tar the stdlib reads as image/bmp — a real file exercising competing detections, refused before this change and still refused |
elf-with-ustar-magic.head512 |
hand-built | an ELF header with ustar at offset 257. It is ACCEPTED now — recognition is by magic — and the test says so. It was written when it was refused, and is kept because the finding it comes from used a real, executing binary |
webm-void-says-matroska.head512 |
FFmpeg WebM with a Void element containing the string matroska, header size widened to match |
a substring search calls this Matroska; its DocType is webm |
matroska-void-padded.head512 |
FFmpeg Matroska with 40 bytes of Void padding, header size widened to match | DocType moves to offset 66, past any fixed leading window — the mistyping this change fixes, still live under a search |
elf-with-valid-tar-checksum.head512 |
hand-built | an ELF header carrying a well-formed tar header in its padding, checksum included. ACCEPTED, deliberately — and when this package still verified checksums, archive/tar's own reader accepted it too. It records a limitation rather than a defect |
tar-flac-named-member.head512 |
GNU tar -cf on a file named fLaC.txt |
an ordinary archive carrying another format's prefix magic in its member NAME |
flac-ustar-in-comment.head512 |
a real FLAC with a Vorbis COMMENT block whose vendor string places ustar at offset 257; decodes under libsndfile |
the same collision from the other side — Vorbis tags are arbitrary UTF-8, so audio can carry the tar magic as easily as a tar can carry an audio marker |
matroska-void-beyond-window.head512 |
FFmpeg Matroska with a 560-byte Void spliced into its header, header size widened to match; the complete file reads as matroska,webm under ffprobe |
a Void larger than the sniff window puts DocType outside the input entirely, so the parse finds nothing and the stdlib's video/webm stands. A limitation, recorded — no window size fixes it, since Void may be larger still |
matroska-nul-terminated-doctype.head512 |
sample.mkv with its DocType payload rewritten to matroska\0junk, declared length 13 |
a DocType value ends at its first NUL; trimming instead stored a real Matroska as WebM |
Void elements are legal anywhere in an EBML header and their contents are meaningless by specification, which is why only a parse can tell payload from padding.
Where an EBML entry above says a muxer or ffprobe read a file, that is said of the file it was MADE FROM. What is committed is the first 512 bytes, as with every fixture here, so running ffprobe on the committed prefix fails with a premature EOF — that is the truncation, not the file. (This paragraph said "the two EBML entries" while there were four of them; it is a property of the whole family, not of a fixed pair.)
Ogg fixtures — kept for a format the sniffer does NOT recognise
Both are refused, and both are kept so the next person to reach for an
application/ogg alias meets the evidence before writing one.
| file | produced by | what it proves |
|---|---|---|
ogg-opus.head512 |
FFmpeg 7.1 -c:a libopus -f ogg |
ordinary Ogg audio, refused — audio/ogg stays unreachable |
ogg-vp8-video.head512 |
FFmpeg 7.1 -c:v libvpx -f ogg |
a real Ogg file whose first packet is OVP80: video in an Ogg container, which an unconditional alias accepted as inline audio |
An alias was written, ruled in, and removed: whether an Ogg container is audio-only cannot be decided from its first page, because Ogg multiplexes and a second video stream's pages come later than any 512-byte sniff can see.
The FFmpeg used is the one bundled with Remotion
(@remotion/compositor-linux-x64-gnu), transcoded from real project assets;
there is no system FFmpeg on the machine these were made on.
Gap, recorded rather than papered over: the media fixtures come from one
FFmpeg build, apart from flac-ustar-in-comment.head512, which is a FLAC
assembled by hand from a real one and verified decodable by libsndfile. A second encoder would be worth having for the EBML pair in
particular, since the DocType read is the only check here that depends on where
a muxer places a string rather than on a fixed prefix.