Adds a second Linux decoder backend, crates/platform/src/nvdec, that talks to
NVIDIA's driver directly (libnvcuvid + the CUDA driver, both loaded at run
time). It is registered in front of VA-API: what NVDEC declines still reaches
VA-API, then the software decoders. H.264 8-bit and HEVC 8/10-bit, 4:2:0
progressive.
NVIDIA's parser takes Annex B access units, so there is no host-side DPB.
Pictures are read back as NV12 / P016 through the existing biplanar path and
are bit-exact with the software decoders. The CUDA context is the one
nvenc::device already retains; it gains push/pop and copy-to-host. A seek
recreates the parser and keeps the decoder.
Also fixes tests/setting.rs, which expected the Media Foundation backend name
on Linux, and updates the Hardware decoding tooltip and its Spanish row.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
main gained NVENC H.265 Main / Main 10 (#328, #360), export stage timings
(#370) and VA-API decoding (#358). Resolution: main's NVENC session, FFI and
export factory, with this PR's changes ported onto them:
- ffi: the portable Guid (now also for the HEVC GUIDs), NV_ENC_DEVICE_TYPE_CUDA;
- session: device.rs (Direct3D 11 / retained CUDA primary context), the
Linux libnvidia-encode.so.1 loader, Device::KIND;
- export: factory_with and its tests, the even-size check, the saturating
max bitrate; H.265 through NVENC stays Windows-only (the factory declines
it on Linux, where nothing probed or tested it);
- Cargo.toml: main's Linux libloading dependency covers both users;
- docs: README / ROADMAP / architecture now say NVENC H.264 runs on Windows
and Linux; ADR and platform README keep both addenda / test rows;
- abi_tests: note why the MSVC-generated expectations hold on LP64 Linux.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Runtime-load the NVIDIA encode and CUDA driver libraries, retaining the primary context through session destruction. Preserve the Windows Direct3D path and default-off selection, and test driver rejection and deterministic software fallback.
Co-Authored-By: OpenAI Codex <noreply@openai.com>
HDR sequences export as real 10-bit HEVC Main 10 through NVENC where the GPU
reports 10-bit HEVC: float BT.2020 R'G'B' -> rgbf_to_yuv420_10 -> P010 input;
VUI BT.2020 + PQ/HLG; HDR10 mastering-display / content-light as SEI on every
IDR (PQ); colr/mdcv/clli in the sample entry. New register_hdr_probe /
hdr_available in the export crate; macOS and GPUs without 10-bit keep
tone-mapping HDR sequences to 8-bit SDR HEVC. ABI asserts extended and
re-verified against nvEncodeAPI.h 12.1 (0 drift). 8-bit SDR HEVC unchanged;
the two #328 tests that asserted HDR is declined now expect Main 10.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
HEVC through VA-API the same way as H.264: the software HEVC decoder's
parsers and DPB (now generic over the picture type in filmcraft-hevc:
POC, reference picture sets, reference lists, output order, RASL
handling) drive the stateless hardware. Main, Main 10 and Main Still
Picture, 4:2:0 8/10-bit (P010 read-back); scaling lists, tiles, WPP,
dependent slices, weighted prediction. Missing references and range
extensions go to the software decoder.
Tests: hardware parity on HEVC Main / Main 10 incl. open GOP, scaling
lists, slices+WPP, weighted prediction, tskip+AMP, 1080p/2160p (streamed
comparison to stay within memory); HEVC front end against a recording
stand-in on every OS; ABI checks extended to va_dec_hevc.h. Benchmark on
Iris Xe: 4K HEVC 203 -> 10 ms CPU per frame, 4K Main 10 212 -> 18 ms.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Stateless VA-API decoding with FilmCraft's own H.264 host side: the
software decoder's SPS/PPS/slice-header parsers and its DPB (now generic
over the picture type in filmcraft-h264) drive the GPU, so reference
marking, reference lists and output order are shared and the pictures are
bit-exact with the software decoder. libva is loaded at run time; no libva
or driver means software decoding as before. frame_num gaps (open-GOP
seeks) get the software decoder's stand-in frames. HybridDecoder keeps the
transparent software fallback.
Tests: hardware parity on 7 H.264 streams incl. 1080p/2160p, seeks,
flushes, forced failures, fuzzing, concurrency; front end against a
recording stand-in on every OS; ABI checks against va.h. Docs: platform
README, ADR 0001 addendum, architecture, performance (H.264 2160p CPU per
frame 417 -> 11 ms on Iris Xe), roadmap.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
One NVENC session, ring and NV12 path now serve H.264 and HEVC (Codec enum,
Profile::HevcMain). HEVC Main 8-bit 4:2:0 SDR BT.709, MP4/MOV hvc1, parameter
sets only in an hvcC built from the SPS the encoder wrote. The H.265 format is
available through a cached NVENC probe; choosing it is the opt-in, declines are
counted errors. ABI asserts regenerated from nvEncodeAPI.h 12.1 (no drift) and
extended for HEVC; tests, docs and roadmap updated.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Merge origin/main into the NVENC branch. #183 already provides the
HardwareEncoding enum, ExportSettings.hardware_encoding, the
hardwareEncoding export param and the Export > Video > Hardware Encoding
checkbox, so the duplicates from this branch are dropped and NVENC uses
them. filmcraft-export is now an all-platform dependency of
filmcraft-platform (VideoToolbox and NVENC both register encoders).
Never-crash fixes in the NVENC backend:
- submit() rejects a driver input pitch narrower than a row (and unlocks
the buffer); fill_nv12 uses get_mut instead of slicing rows.
- composition offsets use saturating math and i32::try_from; frame-rate,
size and timestamp casts are checked; sizes above u16::MAX decline.
- Annex B scanning and NAL type checks use get()/first().
- A failed nvEncOpenEncodeSessionEx that returns a handle destroys it.
ffi.rs carries NVIDIA's MIT notice for nvEncodeAPI.h, listed in
ATTRIBUTION.md under a new "Third-party source code" section.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Adds `Format::Hevc` (Export > Format > H.265 (HEVC); `"format": "hevc"` in
`file.exportMedia`): HEVC Main, 8-bit 4:2:0, in MP4 (`hvc1`) or QuickTime, with AAC audio,
encoded by the macOS hardware encoder. There is no built-in H.265 encoder (one in pure Rust
is a project of its own, and x265 is GPL), so the format exists only where the OS has one.
- `videotoolbox_encode`: the session wrapper creates HEVC sessions too (`VtProfile::HevcMain`;
the profile also says which codec). Keyframes are the HEVC random access points (NAL types
16 to 21); the parameter sets are VPS / SPS / PPS; the `hvcC` record in the MP4 sample
entry is the one VideoToolbox writes (sample description extension atoms), so profile,
level and flags are the encoder's own. `VtEncoder::uses_hardware()` asks VideoToolbox
whether the session runs on a hardware encoder (a test checks H.264 and HEVC), so a
requirement dropped by accident could never become a silent software encode.
- `hardware_encode`: one adapter for both codecs. Choosing the format is the opt-in (the
`hardware_encoding` setting is for H.264). `hevc_available()` (one small hardware session,
created once) is registered with the new `filmcraft_export::register_format_probe`, so
`available(Format::Hevc)`, the format list of Export mode and `exportFormats` follow the
machine. `Format::has_builtin_encoder` says which formats need such a probe.
- `export`: `Format::is_h26x` where H.264 and H.265 behave alike (MP4 or QuickTime, AAC, size
estimate, summary line). Two-pass VBR is refused up front ("H.265 export has no two-pass
mode") and HDR sequences export as SDR, because there is no encoder to fall back to.
- Export mode: H.265 shows H.264's multiplexer, audio, metadata and bitrate controls, without
profile, level, two-pass or the hardware checkbox, and says "Encoder: Hardware".
Apple M1 (8 cores), 1080p25, a 9:29 timeline (camera clip, ProRes 4444 overlays, AAC,
loudness): hardware H.265 at 10 Mb/s 195 s, hardware H.264 at 20 Mb/s 204 s (CPU 1029 s and
1042 s; the same H.264 export took 193 s and 200 s on other runs). It is not faster: the codec
that compresses further is the more expensive one (raw encoders: 166 and 189 frames/s), and
the export is bound by the CPU compositor, not by the media engine.
Quality per bit, 30 s of the same timeline against a ProRes 422 HQ export (average PSNR over
the three planes, SSIM): 20 Mb/s 46.88 dB / 0.9902 (H.264) and 46.91 dB / 0.9903 (H.265);
10 Mb/s 44.96 / 0.9863 and 45.09 / 0.9863; 6 Mb/s 43.72 / 0.9833 and 44.31 / 0.9843; 4 Mb/s
42.91 / 0.9807 and 43.64 / 0.9825; 3 Mb/s 42.37 / 0.9786 and 43.05 / 0.9807. The same picture
at 10 Mb/s and above; at 3 to 6 Mb/s H.265 needs 22 to 31 % less bitrate for the same PSNR.
Both formats therefore keep the same default bitrate. Details in docs/performance.md (HW3).
Tests: tests/hardware_encode.rs gets the HEVC counterparts of the H.264 ones (what is taken
and declined, a round trip through our HEVC decoder with luma PSNR above 30 dB, keyframes, MP4
and QuickTime exports with AAC, ffmpeg / ffprobe reading `hevc` / `Main` / `hvc1` /
`yuv420p` / BT.709, exact sizes, hostile configurations, encoders dropped at any point, the
format list agreeing with the probe, sessions on the hardware encoder); settings_tests.rs
checks names, ids, defaults, the summary, two-pass refusal and the probe; export_ui.rs the
controls H.265 shows and hides.
Stacked on the hardware H.264 encoder (the first commit of this branch).
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Export encodes H.264 on the CPU with the built-in encoder, which is most of the time
of an export (about two thirds of the busy CPU in a profile of a 20 s export). On
macOS the media engine can do it: a VideoToolbox compression session behind
`filmcraft_export::VideoEncoder`, registered by `filmcraft_platform::register()`.
- `ExportSettings::hardware_encoding` (`HardwareEncoding::Off` | `Auto`; Export > Video >
Hardware Encoding, `"hardwareEncoding": "off|auto"` in `file.exportMedia`, a boolean is
understood too), off by default: the built-in encoder's output is byte-identical on
every machine, a hardware encoder's is not.
- `videotoolbox_encode` (FFI, same containment as the decoder: SAFETY comments, no
panic across the callback, safe API) and `hardware_encode` (safe adapter and factory).
- The factory declines, so the built-in encoder is used, for other formats, MXF, two-pass
VBR, HDR, odd sizes, non-square pixels and any configuration VideoToolbox cannot create a
hardware session for. An encoder that fails mid-export stops it with an error: unlike a
decoder, it cannot hand a half-written stream to another one.
- No B-frames: on real 1080p footage the quality is the same with and without them
(+-0.3 dB at equal bitrate) and the bitrate lands closer to the target, and without
them every frame is presented at its decode time (no composition offsets, no edit list).
- `export::register_encoder` ignores a factory registered twice; `encoder_registered`.
Apple M1, 8 cores, single runs: a 9:29 timeline (camera clip, ProRes 4444 overlays with
alpha, AAC, loudness) exports in 311 s instead of 793 s (2.5x faster); its densest 134 s
in 84 s instead of 395 s (4.7x). The outputs compare at SSIM 0.990 / PSNR 46.5 dB. The encoder
alone runs 200 fps at 1080p (docs/performance.md).
Tests (tests/hardware_encode.rs): what is taken and declined, a round trip through our
software decoder, an export that decodes in our decoder and in ffmpeg, the fallback for
everything hardware declines, exact output sizes, an empty range, hostile configurations
and encoders dropped at any point; the engine's `hardwareEncoding` parameter.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
- New crates/platform (filmcraft-platform, L5): OS media FFI only. The one
crate allowed `unsafe` (own [lints] with unsafe_code = "deny",
#[allow(unsafe_code)] only on the videotoolbox module, SAFETY comment on
every block, safe Result-returning public API). AGENTS.md §0.3 names the
exception; docs/adr/0001-platform-ffi.md records why and the rules.
- VideoToolbox decoder for avcC / hvcC, 8/10-bit 4:2:0 and 4:2:2: session from
the sample entry's parameter sets with hardware required, asynchronous
decode (2 in flight), biplanar CVPixelBuffer copied to planar Yuv8/Yuv16,
reorder buffer of the stream's own depth, RASL skipped after a starting CRA.
Output callback under catch_unwind.
- HybridDecoder: transparent mid-stream fallback to the software decoder
(replays samples since the last restart point, drops already-returned
pictures). New codecs::software_video_decoder (built-in factories only) and
codecs::hw (switch, counters, NalStreamInfo shared with software decoders;
vui_color / sar_par helpers now used by both).
- Settings ▸ Playback ▸ Hardware decoding: Auto / Off (engine applies it to
the codecs switch); desktop app, CLI/MCP and bench call register().
- perf.stats decode.framesDecoded and decode.hardware counters.
- Tests: bit-exact parity with the software decoders (H.264 High, HEVC Main
open GOP, Main 10; resets, reseeks, flushes), forced-failure fallback,
mutation fuzz of samples and parameter sets, cross-platform fallback with a
stand-in decoder, setting Off, HEVC 4:2:2 vs ffmpeg.
- Bench --hw auto|off; results in docs/performance.md (4K CPU/frame
~86-119 ms -> ~4 ms).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
SilentYeti