Problem: the Matroska demuxer mapped A_MPEG/L2 and A_MPEG/L3 only, so a
Layer I track was reported as an unsupported codec even though the MP1/MP2
decoder (#809) reads Layer I.
Fix: add Codec::Mp1 for A_MPEG/L1 and route it through the same packet
duration and decoder path as Mp2. Test remuxes synthetic Layer I frames
into Matroska with ffmpeg and compares our decode (max error 3e-5).
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Both were offered for import but could never be imported.
`.y4m` is in VIDEO_EXTENSIONS (and file-format-parity.md lists Y4M as
read), but nothing read it. `codecs::y4m` is a YUV4MPEG2 source written
from the format's public documentation (yuv4mpeg(5)): the W/H/F/A/I/C/X
stream header, FRAME lines with their own parameters, 8-bit 4:2:0 / 4:2:2
/ 4:4:4 / 4:4:4 + alpha / mono and 9–16-bit little-endian
(`420p10`, `422p12`, `mono16`, …) as Yuv8 / Yuv16 frames, XCOLORRANGE=FULL
and the pixel aspect. Frames are indexed once from their header lines and
read on demand (a reader opener too), so large files aren't loaded whole.
Sizes, rates and offsets are checked; 4:1:1 and unknown colour spaces are
an "unsupported" error.
The Media Browser listed `.ale` among project files, but FilmCraft only
writes ALE (File ▸ Export; Premiere doesn't import it either, per
file-format-parity.md), so importing one always failed. It is no longer
listed.
Tests: hand-built Y4M files (4:2:0 frame by frame, 10-bit 4:2:2 with frame
parameters, aspect and full range, mono, the reader path), truncated /
bit-flipped / hostile headers under catch_unwind, and an ffmpeg oracle
(8-bit 4:2:0 and 10-bit 4:2:2 files equal ffmpeg's rawvideo output byte
for byte); the browser no longer classifies `.ale`.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
File ▸ Import and the Media Browser offer `.aac` (it is in AUDIO_EXTENSIONS),
and Export ▸ AAC Audio writes ADTS `.aac` (#814), but no opener read them:
`audio::opener` only accepted mp3/mp2/flac/ogg/aif by extension, so every
`.aac` failed with "unrecognised format".
`AudioFileSource` now decodes a standalone ADTS stream with our AAC decoder
(ISO/IEC 13818-7 §6.2): frame after frame from the first header (after an
ID3v2 tag), resyncing past damage, skipping frames with another
configuration, keeping a frame that doesn't decode as silence, and
upsampling HE-AAC to its output rate. Decoding stops with an error past
4 GiB of samples instead of allocating without bound.
Tests encode a stereo tone with our encoder's ADTS output and import it
(rate, channels, duration, level), and run truncated, bit-flipped and
bad-length streams under catch_unwind.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
vui_color took the VUI matrix, transfer and range but not the primaries,
so every decoded H.264 / HEVC frame reported BT.709 primaries. Stream info
read them, so a BT.2020 file without a colr box said BT.2020 in its info
and BT.709 on its frames (Rec709 instead of Rec2020 for the renderer).
Pass colour_primaries through, for the software and hardware decoders.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The MP4 reader took the AudioSpecificConfig's channel configuration as a
count, but it is an index into ISO/IEC 14496-3 Table 1.19: configuration 7
(7.1) reported seven channels and dropped the last decoded one, and 12
(7.1 back, what ffmpeg 9 writes for 7.1) reported twelve. Count from the
table (with Amd 4: 11 → 7, 12 and 14 → 8, 13 → 24), and decode
configurations 11 and 12, reordered to the WAV/SMPTE order used for 5.1.
Checked against ffmpeg's decode channel by channel.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ffmpeg's CLI handles Matroska priming differently by version: 9.x trims
the 481-sample CodecDelay, 2023-24 static builds keep the priming
samples, so comparing from sample 2 put the priming ramp against our
tone (max error = peak 0.125). The .mkv case now searches lags within
two samples of 0 (trimmed) and of the stream's own CodecDelay (kept),
over the interior of both decodes; the plain .mp2 file stays at lag 0.
Review nits: the ID3v2 skip offset uses saturating_add (test with the
largest tag size); every Layer II table's sblimit is checked <= 32 at
compile time and clamped to SUBBANDS before indexing the per-subband
arrays (row() is a match, so it cannot index out of range).
Verified with ffmpeg 9.0.2 and the static N-71064 (2024-06) build.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ffmpeg's CLI handles Matroska priming differently by version: 9.x trims
CodecDelay, 2023-24 static builds keep the priming samples (AAC 1024,
AC-3 / E-AC-3 256, MP3 1105), so a +-40-sample best-lag search against
its decode landed on a spurious minimum (4 or 24 samples).
matroska_audio_oracle now has ffmpeg encode a reference signal we
generate (low-passed xorshift noise) and asserts our decode lines up
with that signal to the sample, the stream's own CodecDelay trimmed:
version-independent, and it fails on the old ms-rounded trim (-16).
ffmpeg's decode is compared after a best-lag search bounded by the
CodecDelay; its lag is reported, the waveform must match (< 1e-3).
E-AC-3 joins the cases. eac3_oracle's .mkv case accepts lag 0 or the
256-sample priming; MP4 / MOV / TS stay strictly at lag 0.
Verified with ffmpeg 9.0.2 and the static N-71064 (2024-06) build.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Use ds64.dataSize for an RF64 data-size sentinel instead of the rest of
the file, so chunks after the audio are no longer decoded as samples.
Fixes#925
Problem: DPX import rejected the 10/12-bit packed layout (packing 0)
and the Y'CbCr descriptors (100 CbYCrY, 101 CbYACrYA, 102 CbYCr,
103 CbYCrA) with an "unsupported" error.
Fix: unpack packed components back to back, low bits first, each scan
line on a 32-bit boundary (as ffmpeg reads 12-bit packed files), and
convert Y'CbCr to RGB with the matrix the colorimetric field names
(BT.709; BT.601 for the 601 and composite codes; by picture size when
unspecified), in video range unless the reference codes span the full
code range; 4:2:2 chroma is shared by each pixel pair. Unit tests for
every new layout and hostile headers; an ffmpeg oracle test builds
packed 12-bit RGB and 8-bit 4:2:2 / 4:4:4 / 4:4:4:4 files from
ffmpeg's test picture and matches its decode within one 8-bit step.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Problem: Premiere imports Targa and DPX as stills and numbered sequences.
FilmCraft exports both (#779) but could not read either: the extensions
were not importable and the image crate has no DPX reader, so even our
own Targa / DPX renders could not come back in.
Fix: clean-room decoders in filmcraft-media. Targa from the TGA 2.0 spec:
image types 1/2/3 and RLE 9/10/11, colour maps, 15/16/24/32-bit, 8/16-bit
grayscale, any origin. DPX from SMPTE ST 268: first element, luma / RGB /
RGBA / ABGR, 8/16-bit, filled 10/12-bit (methods A and B), either byte
order, padded lines, orientations 0-3. Both bound allocations by the
data present and reject hostile headers with errors. Stills and image
sequences share one decode_raster entry point; "tga" and "dpx" join the
importable still extensions.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Problem: symphonia rejects every Layer II frame whose bit rate / mode
combination ISO/IEC 11172-3 does not allow (mono above 192 kbit/s, stereo
below 64 kbit/s). ffmpeg writes mono MP2 at 384 kbit/s by default, so such
tracks played as silence in MPEG-TS / PS, Matroska and raw .mp2 files.
Fix: new L0 crate filmcraft-mpa, a clean-room Layer I / II decoder written
from ISO/IEC 11172-3 and 13818-3 (all modes incl. joint stereo, MPEG-1/2/2.5
rates, the disallowed combinations decoded with the allocation table their
bit rate per channel selects). PacketDecoder::mpeg_audio uses it for layers
I and II (packets of several frames too); raw .mp2 files decode with it,
resyncing after damage. Oracle tests vs ffmpeg: 73-84 dB SNR on ffmpeg /
twolame streams and synthetic Layer I / joint stereo frames; fuzzed damaged
frames never panic.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Problem: MP4 and QuickTime name MPEG audio without its layer (ffmpeg
stores MP2 as `mp4a` + esds object type 0x6B), and we always made the
layer III decoder for it. symphonia's decoder reads only the layer it
was made for, so every MP2 frame was rejected and the track played as
silence; MOV's `.mp2` sample entry was not recognised at all.
Fix: the MPEG audio decoder for MP4 / MOV is made on the first packet,
for the layer its frame header names (ISO/IEC 11172-3 2.4.2.3);
`.mp2` maps to the MPEG audio codec like `.mp3`; the track is labelled
"MPEG Audio" for layers I/II, as Matroska's is. Decoded tracks now match
ffmpeg sample-exactly (edit list trims the 481-sample encoder delay).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Problem: ffmpeg stores the encoder priming as CodecDelay in ns (1024 AAC
samples at 48 kHz = 21.333 ms, 256 AC-3 samples at 44.1 kHz = 5.805 ms)
and block timestamps in ms. The demuxer subtracts the delay rounded to
1 ms, and fixed-length packets were anchored on that rounded stamp, so
the whole track played up to 16 samples late against ffmpeg's decode.
Only Opus undid the rounding.
Fix: one stamp-to-samples conversion for every Matroska audio codec
that undoes the demuxer's rounding and removes the exact CodecDelay
(rounded to the nearest sample; the element holds whole ns), with
checked math against hostile timebases.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Problem: files with E-AC-3 audio (MP4 ec-3, MOV, Matroska A_EAC3, MPEG-TS)
imported silent: filmcraft-ac3 refused bsid 11-16.
Fix: clean-room E-AC-3 decoding from ATSC A/52:2012 Annex E in filmcraft-ac3:
bsi and audfrm syntax, 1/2/3/6-block syncframes, reduced sample rates,
frame-based exponent strategies (exponents reused across short syncframes),
SNR offset strategies, default band structures, spectral extension
(translation, notch filter, noise blending). Independent substream 0 is
decoded; dependent substreams and other programs are skipped (7.1 plays as
its 5.1 core). AHT and enhanced coupling are refused as unsupported. The
audio blocks share the AC-3 path; AC-3 output is unchanged. filmcraft-codecs
routes E-AC-3 in MP4/MOV, Matroska and TS to the decoder; the TS splitter
joins dependent substream frames to their access unit.
Tests: ffmpeg oracle for raw E-AC-3 (64.8-91.0 dB SNR) and in MP4, MOV,
Matroska and TS (79.5-91.3 dB, random access identical), mutation fuzz,
synthetic frames for headers, substream skipping, refusals and exponent
reuse across 1-block syncframes, spectral extension unit tests.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
In an open GOP (iPhone HEVC; x265 `open-gop`) each CRA after the first is
followed in the file by RASL pictures shown before it, predicted from the
GOP before. GopCache started a decoding run at `sync_before(i)`, the sync
sample at or before the wanted sample in decoding order: for a RASL
picture that is the CRA after it, and the decoders (software and
VideoToolbox) rightly leave a run's leading RASL pictures out. The frame
was never produced, so the request failed ("frame not produced") and the
clip's layer was left out: a black frame (or the track below) wherever a
seek landed on one of the 3-in-24 RASL frames of iPhone video, such as a
clip trimmed to start on one, the parked playhead, and parallel export.
- `decode_start`: a sample shown before its sync sample starts the run at
the sync sample before (the MPEG and MXF readers already did this in
their own `sync_before`; this covers MP4/MOV and Matroska too).
- A running decoder that started after that point is not continued for
it: it left the picture out.
Tests: a synthetic open-GOP stream in gop.rs, and an x265 open-GOP oracle
test (MP4 and Matroska, every frame from a fresh source and each CRA
before its leading pictures) against ffmpeg. Both fail without the fix.
Drop-frame counting never emits the first base/15 frame labels of a minute
except each tenth (frames 00-01 at 29.97, 00-03 at 59.94). Accepting them let
a damaged 00:01:00;00 normalize to a different start time, so fields_to_frames
now rejects those labels and the start timecode is left unset.
Address review of #460:
- Read only the fixed-size rtmd header (18 bytes) instead of read_sample,
which allocated the whole untrusted stsz size (a corrupt multi-GiB size
could exhaust memory during import).
- Validate hours/minutes/seconds/frames against the frame base and drop the
start timecode when they are out of range, instead of normalizing damaged
metadata into a plausible but wrong offset.
Sony XAVC cameras such as the FX3 and a6400 keep the clip's start timecode
in an rtmd (real-time metadata) data track rather than a QuickTime tmcd
track, so probe reported start_timecode: null and timecode sync could not
use the footage. Read the hours/minutes/seconds/drop-frame/frames block from
the first rtmd sample and expose it as MediaInfo::start_timecode, falling
back only when no tmcd track is present (#460).
The MP4 source reported the sample entry's size, only swapped for the display rotation, so two
crops were lost: the codec's own (H.264 SPS frame cropping, HEVC conformance window), which the
decoder applies but the reported size and sequences made from the clip didn't, and the clean
aperture (`clap`), parsed but never used. An iPhone clip (1920x1440 coded, 1744x1308 clean
aperture, -90 degree display matrix) showed 1440x1920 instead of 1308x1744.
H.264 / HEVC sizes now come from the stream's parameter sets; the clean aperture is a rectangle
inside that picture (ignored when it makes no sense), reported as the size and cut from every
frame by the GOP cache before the display rotation, through the new VideoFrame::cropped.
Tests: bit-exact against ffmpeg (which applies both crops) for an H.264 SPS crop, an HEVC
conformance window, centred and off-centre clean apertures and a clean aperture with a rotation;
clean apertures that make no sense are ignored without a panic; VideoFrame::cropped on every pixel
format with odd sizes and misaligned corners.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Keeps #359's contiguous_starts audio timing, now per stream in Mp4Audio /
MkvAudio (starts and preroll in source sample frames). MP4 audio streams and
their decoders are built from the same (track, info) pairs so they stay
index-aligned. Adapts new main code to audio_streams / TrackItem::audio_stream.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Resolve inherited physical key collisions when loading built-in presets,
while preserving established explicit winners and deliberate custom conflicts.
Read every ffprobe display-frame timestamp and register the requested credit.
Verify exact frame-plane reuse without depending on concurrent global metrics.
Co-authored-by: Codex <codex@openai.com>
- Update MediaInfo to store all discovered audio streams in audio_streams
- Retain independent stream metadata and decoder state for MP4/MOV and Matroska/WebM
- Update audio_placement_specs to allocate successive tracks via ensure_audio_tracks
- Isolate clip rendering to read assigned audio stream rather than replaying stream 0
- Bump project schema to v13 with backwards compatibility for legacy v12 projects
Resolves#312Fixes#200Fixes#315
Matroska timestamps are whole milliseconds and a 1024-sample AAC frame at 48 kHz lasts
21.33 ms, so frames are stamped at 0, 1008, 2064, 3072… samples; OBS's remux to MP4 carries
that rounding into the sample durations (1008, 1008, 1056…). The MKV and MP4 readers put each
decoded packet at its own timestamp, overwriting the end of one packet and leaving a hole of
silence after another: a click every few packets, crackling throughout playback and export.
Packets of codecs with a fixed frame length (AAC, MPEG audio, AC-3) now run on from one
another and resynchronise to the timestamp only across gaps of more than half a packet, the
rule the MPEG-TS/PS reader already uses. Laced Matroska frames without a duration (which
repeat the block's timestamp) follow on too, and MKV Opus uses the same helper. MP4 audio
packet starts are kept in samples instead of track timescale units.
Regression tests mux our own AAC into MKV (millisecond stamps, plain and laced, a real gap)
and into an MP4 with OBS-remux durations: block-by-block playback and seeks must equal the
packets decoded back to back (all fail before the fix). An ffmpeg oracle test plays
ffmpeg-made MKVs with AAC, MP3, MP2 and AC-3 and requires them as smooth as ffmpeg's decode.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
make_sequence had ffmpeg write numbered stills straight into the shared seq_<ext> directory and
treated it as ready once frames 1 and 12 existed. Two tests generating it at once (png_sequence_exact
and missing_frames_hold_the_previous_frame) read each other's half-written PNGs; one that fails to
decode is held as missing, so frame 6 equalled frame 3. Six parallel processes on a fresh fixture
dir failed 18 of 120 runs before, 0 of 120 after.
ffmpeg now writes into a testkit temp_path directory that is renamed into place (with a .complete
marker) only when all twelve frames are there; the gap copy is private to its test.
Fixes#125
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The clip duration came from the media duration (`mdhd`), which runs on
the decode timeline. ffmpeg / x264 MP4s with B-frames skip the reorder
delay with an edit, so every one imported a frame too long (a 120-frame
file became 121 frames, the last a held repeat). A video track delayed
by a leading empty edit was a frame short, and its last frame could not
be reached.
Use the sum of the track's edits (movie timescale, what `tkhd` records)
when there is an edit list. Fragmented files, and edit lists that add up
to nothing or overflow a Tick, keep the media duration.
Tests: a unit test with crafted edit lists (skip, empty edit, hostile
sizes) and an ffmpeg oracle test (libx264 with B-frames: 60 frames,
was 61).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
trim_decoders dropped the decoders it took from idle caches while the requesting cache's lock was held. Dropping a decoder can block (a VideoToolbox session waits for its frames in flight), so the render path paid that latency under the lock. trim_decoders now returns the released decoders and frame_late keeps them in a local declared before its lock guard, so they drop after the guard on every return path; the cap is enforced exactly as before.
New tests/pool_bitexact.rs: primes the frame pool with 0xAA-filled plane buffers, then decodes an H.264 fixture into them; every frame must match ffmpeg bit for bit, and the pool must report the reuse.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Problem: a clip with AAC audio in MP4 whose sample rate differs from the
sequence's (48 kHz clip in a 44.1 kHz sequence) exported with loud
low-frequency thumps, several times full scale, recurring through the clip.
Every exported container was affected (AAC clips them into thumps, PCM
clips at full scale); a WAV export had a single one at the clip's start.
Cause: Mp4Source::audio decodes a window starting at s0, the requested
presentation sample minus the edit-list offset, then resamples it. The
resampler derived its sub-sample phase from s0, so the edit-list offset
(-1024 samples of AAC priming) was applied a second time: the read
position started about 1024 samples before the window, was clamped to
index 0 with a large negative fraction, and linear interpolation
extrapolated a ramp over the first ~940 samples of every request. Export
asks for audio one batch of video frames at a time, so the ramp repeated
at each request. Sources at the sequence rate take the copy path and
were unaffected, as were tracks without an edit list.
Fix: take the phase from the presentation sample the window starts at
(before the edit-list offset is subtracted).
Test: aac_mp4_resampled_reads_follow_the_edit_list
(crates/codecs/tests/mp4_oracle.rs, with `require_ffmpeg!` and the
shared fixture helper as docs/testing.md asks of new oracle tests) reads
an ffmpeg AAC fixture at 44.1 kHz in frame-sized pieces and compares it
with the 48 kHz decode interpolated at the same times (peak was 7.35
against a source level of 0.15 before the fix). docs/testing.md lists it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Rust/clippy 1.99 added `chunks_exact_to_as_chunks` and tightened
`needless_late_init` and `drain_collect`, failing `clippy -D warnings`
on main.
- `chunks_exact(N)` / `chunks_exact_mut(N)` with a constant N become
`as_chunks::<N>()` / `as_chunks_mut::<N>()` (fixed-size arrays, so
several byte reads drop their per-element indexing). relink's
fingerprint uses the `(chunks, remainder)` split; its hash output is
unchanged.
- Late-initialised `let`s in ac3 bit allocation and opus CELT
bands/rate become `let x = if/match …`.
- `dpb.drain(..).collect()` becomes `std::mem::take(&mut dpb)`.
No behaviour change; `cargo xtask ci` passes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sub-pixel filter passes run over fixed block widths; frame / region planes are recycled
through a bounded pool (zeroed on reuse) instead of fresh calloc / munmap per frame; the
palette mode-info arrays (40 bytes per 4x4) exist only when screen content tools are allowed;
CDF adaptation is branch-free. Draft mode skips deblocking, CDEF and loop restoration of shown
frames that refresh no reference slot. Media-stack test: VP9 / HEVC / AV1 draft playback only
reaches draft requests, and an export afterwards is bit-exact with ffmpeg.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Brandon Thomas