Commit Graph

372 Commits

Author SHA1 Message Date
aiongg 21903294bf Merge track r1-rawcat into next
crates/ui-egui/locales/{cs,it,pl}-formats.json: kept the translations and added the track's new keys.
docs/parity-checklist.md: newer header line; revision rows from both sides.
docs/whats-new.md: both tracks' 2026-10-15 entries.
2026-10-21 14:58:55 +00:00
aiongg 83ac0bca70 Merge PR #1923: LR-IMP-FORMATS: CRX high-frequency band width from the tile geometry … 2026-10-21 14:53:02 +00:00
aiongg 4cf0b3472e Integration: #2003 decode test for real lossy tiles (from #1994)
Co-authored-by: EmpressAthleticus <153782305+EmpressAthleticus@users.noreply.github.com>
2026-10-21 14:52:59 +00:00
aiongg 1a4d81fd5a Merge PR #2180: raw, codecs: checked dimension products so wasm32 cannot wrap the size gates (#2138) 2026-10-21 14:49:01 +00:00
Matthew James Sivak 0360f940e3 Merge main into lightroom-jxl-dng
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 08:33:20 +02:00
Brandon Thomas 56c9b49634 Merge PR #1919: Hasselblad FFF: colour matrix from the embedded 3FR header
# Conflicts:
#	docs/parity-checklist.md
#	docs/raw-parity.md
2026-10-18 09:22:57 -04:00
Brandon Thomas 363dbbdd9f Merge PR #2224: X-Trans CMOS 4 bodies take the X-T3's spectral matrices 2026-10-18 09:20:17 -04:00
Brandon Thomas f819c8439b Merge PR #2151: Make TIFF chunk-length inference and header-plan merging fast 2026-10-18 09:19:42 -04:00
Yorgos S. 863adcac6f X-Trans CMOS 4 bodies take the X-T3's spectral matrices
An X-T30 RAF shot in Acros embeds a black-and-white camera JPEG, so the
per-file look fits only the tone curve, and the colour came from the
neutral "camera RGB ~ sRGB" fallback: a grey sky and muted colours. The
raw decode (lossless compressed) was correct.

Add a hand-kept SAME_SENSOR alias list next to the generated
rawtoaces-data table: the 26.1 MP X-Trans CMOS 4 bodies (X-T30, X-T30 II,
X-Pro3, X100V, X-S10, X-E4) use the X-T3's measured matrices, and their
white balance reads in Kelvin through them.
2026-10-18 11:46:07 +02:00
eliets2 9f2052f56b raw: audit fixes — DEFLATE row bytes in u64, checked is_packed12, cap invariant documented (#2138)
The reviewer audit of the first commit found the DEFLATE arm computed row_bytes in
wrapping usize before the gate, so on wasm32 a hostile chunk header (width 2^29+1,
16-bit) could wrap cw * cpp * bits to a small value and slip a garbage chunk past the
new gate. row_bytes is now computed in u64 (the gate caps at MAX_SAMPLES, so the usize
cast is lossless on 32-bit too), checked_pixel_count takes u64 directly so no call site
can truncate before the check, is_packed12 uses a checked_mul chain instead of comparing
against a possibly-wrapped length, and the arw decode head documents the invariant the
cap provides: with w*h <= 2^30 every downstream usize product stays inside range on
32-bit targets.

Behaviour is unchanged for every input the previous commit accepted; the only shrunk
set is chunks that passed because wrapping made row_bytes small.
2026-10-17 22:20:48 +03:00
eliets2 36d22a2745 raw, codecs: checked dimension products so wasm32 cannot wrap the size gates (#2138)
On wasm32 usize is 32-bit, so 65536 x 65536 wraps to 0 and products like w * h,
w * h * cpp, fw * fh * nc and row_bytes * ch wrapped past their own validation and let
later true-dimension indexing panic (profile.web sets panic = abort). All the enumerated
sites now validate through a checked u64 gate:

- lightcraft-raw gains checked_pixel_count(width, height) (u64, capped at MAX_SAMPLES),
  used by the ARW decode head (the strip-length equality now compares the true sample
  count, not a wrapped one) and by RawImage::check's data-length validation;
- the CR2 lossless-JPEG frame size is checked in u64 on both the Full and Header paths
  (it used to be clamped AFTER wrapping), and the no-slices width fallback is checked;
- the TIFF DEFLATE chunk's inflate length is checked and capped before the cast;
- lightcraft-codecs gains checked_sample_buf_len for the finish() buffer bound, compared
  in u64 via try_from so no cast truncates on 32-bit.

Tests pin the gate arithmetic with values that would wrap 32-bit usize (65536 x 65536,
u32::MAX x 2, 2^30 x 4). cargo check --target wasm32-unknown-unknown passes for both
crates. The issue's suggested wasm32 arithmetic-side-effects clippy pass is a follow-up.
2026-10-17 22:20:48 +03:00
NetterSilver f6d465c72c Make TIFF chunk-length inference and header-plan merging fast
Fixes #2128

Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-17 14:50:59 +02:00
Matthew James Sivak ed5afb7eb0 LR-IMP-FORMATS: document why lossy JPEG XL DNG tiles keep their declared curve
Lightroom Classic's lossy LinearRaw DNGs linearise the sRGB-coded samples with a
cubic MapPolynomial in OpcodeList2; measured against Lightroom's preview, the
declared-curve reading tracks its rendering and the linear one is 3.6 EV dark.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 10:46:01 +02:00
Matthew James Sivak 2d57a1466d LR-IMP-FORMATS: decode lossy JPEG XL DNG tiles with a non-linear declared transfer curve
Lightroom Classic 15.1 writes lossy LinearRaw DNGs (e.g. Denoise output) whose
JPEG XL tiles declare Rec. 2100 primaries with the sRGB transfer curve. These
were rejected ("declared encoding that is not linear RGB") and fell back to the
embedded preview. The tile's samples mean what they would in a lossless tile with
the same header, so the declared curve (sRGB, BT.709, DCI, gamma) is now applied
after the primaries matrix, mirrored and unclipped. PQ / HLG / unknown stay
unsupported.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 10:46:01 +02:00
naryzach dde403946b raw: alias Canon EOS R6 Mark II to EOS R6m2
Canon EOS R6 Mark II writes its Exif model as 'Canon EOS R6 Mark II', which failed the spectral camera table lookup because the table registers the model as 'EOS R6m2'. Add the alias mapping and regression unit tests in lightcraft_raw and lightcraft_catalog.
2026-10-15 12:17:25 -07:00
Claude b5dc500c54 Hasselblad 3FR / FFF: uncompressed RGB embedded previews
These files carry no JPEG: their preview is an uncompressed 8-bit RGB
IFD (1217 x 913 in the H4D-40 FFF, 218 x 218 in the CFV 3FR). The
largest such image is handed on by embedded_preview as a small baseline
TIFF, which lightcraft_codecs::decode reads, so 3FR / FFF show a preview
in the grid and loupe before they are developed. No Orientation tag is
written: the raw file's applies, as for JPEG previews without one. The
FFF's 16-bit copy (with a TransferFunction of unknown meaning) is not
used.

decode_raw_preview applied a container-declared colour space by forcing
the JPEG decoder on any owned preview; it now checks for JPEG bytes, so
a wrapped TIFF (or a JPEG XL preview) goes to its own decoder.

Checked on the CC0 H4D-40 FFF and CFV 3FR through
embedded_preview_srgb: right colours, the 3FR upright through its
Rotate90. Tests cover the wrap, the choice among RGB images, a short
strip and truncated files.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J2NDVXA8jQQWeJb4YVTb6b
2026-10-15 18:58:05 +00:00
Claude 0f67225e83 Hasselblad FFF: colour matrix from the embedded 3FR header
An FFF (a 3FR converted on the desktop) has no colour matrix of its own:
the camera's stays in the original 3FR header the FFF carries as a
complete TIFF in private IFD0 tag 0xC51B. The new vendor::hasselblad
reader decodes 3FR / FFF through dng::decode_as and, when the file has
no matrix, takes the matrices from that header, so an FFF no longer
develops with the "camera RGB = sRGB" fallback (H4D-40 as-shot white
moves from xy 0.279, 0.356 to 0.325, 0.336).

RGGB checked against the camera preview on the CC0 H4D-40 FFF and CFV
3FR (raw.pixls.us 931 and 709). Synthetic tests cover the embedded
matrix, an own matrix winning, a missing or damaged header, and
truncated files.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J2NDVXA8jQQWeJb4YVTb6b
2026-10-15 18:19:15 +00:00
Pierre-Edouard Honnet c806debada LR-IMP-FORMATS: CRX high-frequency band width from the tile geometry (#899)
The decoder tried each high-frequency band at its synthesis width and up to
two more columns, keeping the first width whose entropy stream ended exactly.
Two widths can both do that with different coefficients (#899).

Measured on every C-RAW plane in cr3_corpus.rs (CC0 raw.pixls.us files):
each band decodes at exactly one width. A tile with a neighbour on its right
codes bands 5 and 8 (the vertical high-pass of the level-2 and level-1
low-pass columns) at an odd width, so an even band gains one unused column;
no other band is extended. The width now comes from that rule, and a stream
that does not fit it is corrupt (the CR3 falls back to its preview).

All corpus sensor hashes are unchanged. The property-test encoder now codes
bands 5 and 8 the same way. Middle tiles (three or more C-RAW tiles) follow
the left-tile rule; no public file has one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 20:12:46 +02:00
Brandon Thomas 51a784cd95 Merge PR #1475 (rebased)
# Conflicts:
#	crates/raw/examples/rawinfo.rs
#	crates/raw/src/lib.rs
#	crates/raw/src/vendor/arw.rs
#	crates/raw/src/vendor/erf.rs
#	crates/raw/src/vendor/mef.rs
#	crates/raw/src/vendor/nef.rs
#	crates/raw/tests/lossy_dng.rs
#	crates/raw/tests/robust.rs
2026-10-15 01:13:36 -04:00
Brandon Thomas 66a0cd826c Merge PR #1475
# Conflicts:
#	crates/merge/src/output.rs
#	crates/raw/src/vendor/nef.rs
2026-10-15 01:05:31 -04:00
Brandon Thomas 9ca113defa Merge PR #1135 (rebased)
# Conflicts:
#	xtask/src/main.rs
2026-10-15 00:53:45 -04:00
Brandon Thomas 76881c977e Merge PR #1327 (rebased)
# Conflicts:
#	ROADMAP.md
#	docs/gaps.md
#	docs/parity-checklist.md
#	docs/target-app-parity.md
2026-10-15 00:53:22 -04:00
Brandon Thomas 0af2d36442 Merge PR #1320 (rebased) 2026-10-15 00:52:32 -04:00
Brandon Thomas f7204aa8dd Merge PR #1176 (rebased) 2026-10-15 00:52:31 -04:00
Brandon Thomas f27ca795c7 Merge PR #1327
# Conflicts:
#	ROADMAP.md
#	crates/raw/tests/corpus.rs
#	docs/gaps.md
#	docs/parity-checklist.md
#	docs/raw-parity.md
#	docs/raw/coverage.tsv
#	docs/raw/evidence.tsv
#	docs/target-app-parity.md
2026-10-15 00:44:42 -04:00
Brandon Thomas 372f39f8f7 Merge PR #1320
# Conflicts:
#	crates/engine/src/files.rs
#	crates/raw/src/demosaic/ahd.rs
2026-10-15 00:38:16 -04:00
Brandon Thomas 77dc170ede Merge PR #1053 (updated head)
# Conflicts:
#	crates/engine/examples/raw_frame.rs
#	crates/raw/src/vendor/nef.rs
#	docs/gaps.md
#	docs/parity-checklist.md
#	docs/raw-parity.md
#	docs/whats-new.md
#	xtask/src/main.rs
2026-10-15 00:10:37 -04:00
Brandon Thomas f613aeba59 Merge PR #1176
# Conflicts:
#	crates/raw/src/vendor/arw.rs
#	docs/parity-checklist.md
#	docs/sony-lens-corrections.md
2026-10-14 23:54:06 -04:00
Brandon Thomas 851aedc64d Batch 36: tests and tracker after the merges
- Visualize HDR (#1555) has no default key: ⇧H cycles the Tool Overlay (#1473, issue #224)
- CR3 tests (#1461, #1462) accept adaptive QP 131–175, as #1694 decodes; cr3.md says the R6 Mark II files decode
- English fallbacks in the Czech and Polish catalogs use → for the menu triangle; Polish in the keyword-label test
- canvas handle tests show pins (Tool Overlay Always); a focused handle keeps its pins shown in Auto
- the model-install test and demo deadlock guards wait on deadlines long enough for a busy machine

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 23:25:29 -04:00
Brandon Thomas 9ca3cd0926 Merge PR #1135
# Conflicts:
#	Cargo.lock
#	Cargo.toml
#	README.md
#	crates/raw/Cargo.toml
#	crates/ui-egui/src/theme.rs
#	xtask/src/main.rs
2026-10-14 23:23:18 -04:00
Brandon Thomas 56022ffc2a Batch 36: D600/D610 measured areas listed once (#1053 and #1770 both added them, with the same crops)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 22:51:38 -04:00
robmdunn 2b3b6e7871 LR-IMP-CAMERA-COVERAGE: measured image area for Nikon bodies without CropArea
Nikon bodies from before ~2016 record no CropArea, so the decoder kept
their whole sensor data, less trailing dark columns. Where that data
holds masked lines above black or optical black, they were developed as
a coloured or dark bar on that edge: the D7000 and D5100's 36 columns at
~600 (#976), the D800's 14 (#914, the top in portrait), the D600 / D610's
8 (#1236), the D3200's 46, and others.

MEASURED_BODIES (crates/raw/src/vendor/nef.rs) now gives 44 more model /
data size / depth groups the frame of their camera JPEG inside the data,
next to the D4, D7100 and D90 entries already there: D7000, D5100, D800,
D800E, Df, the D4's 12-bit mode, D3, D3S, D700, D300, D300S, D600, D610,
D810, D4S, D5600, Coolpix A, D3000, D40X, D60, D80, D2Xs, D200, D3200
and Nikon 1 V1; and, on best-guess frames short of the evidence rule,
each with a comment naming its evidence, the D3100 (3 of 4 clean files
agree), D5000 (4 of 5, and the D90's sensor), D40 (2 of 2), D50 and D70
(one file each, agreeing) and D70s (next to them, between pixels).
Every best-guess frame keeps the masked lines out by several pixels.
An entry matches model, data size and depth exactly. Bodies whose data
holds only picture beyond the camera frame get no entry and render as
before.

`cargo xtask raw-frame FILE...` (crates/engine/examples/raw_frame.rs)
measures a body: it aligns each raw's embedded camera JPEG with the
sensor data (half-size luminance, gradient correlation, sub-pixel peak;
weak, ambiguous or between-pixel peaks are flagged), counts the edge
lines that hold no picture and reports what the decoder's crop keeps
beyond the camera frame. A masked line is flat and its two CFA phases
agree within the noise or within 0.4% of white, so filler with a small
fixed offset between them (D90, D5000) counts. It prints entries only
with evidence: at least 3 clean files agreeing exactly, of the model
(12- and 14-bit pooled) or, with at least one of its own, with sibling
models of the same readout (data and JPEG size) that agree on the same
frame. On the D7500, which records its CropArea, it finds that area
exactly; it declines the Z f, whose JPEG is distortion-corrected. It
works for any make whose raw embeds a full-size camera JPEG: on Sony,
Canon CR2 / CR3, Pentax and Samsung files it finds the decoder's own
crop (relative to the active area).

Every Nikon raw on raw.pixls.us without CropArea was surveyed this way
(its 138 CC0 samples, plus 124 other publicly posted sample files; only
the measurements are committed). Bodies whose data holds only picture
beyond the frame need no entry (D750, D5300, D5500, D3300, D2X, D2Hs,
D3X, D5200, Coolpix P6000, Nikon 1 AW1 / J3 / V2 / V3, the D800's crop
modes); raw-parity.md lists them, and no longer claims every NEF
coding but HE decodes. measured_bodies_are_consistent checks every entry lies
inside its data and is listed once; measured_older_dx_bodies_image_area
covers the D200, D3200, Nikon 1 V1 and best-guess entries.

Fixes #976
Fixes #914
Fixes #1236

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 22:40:06 -04:00
Brandon Thomas 211255770f Batch 36: #1510's rename follows the optional name #1665 gave person_from; #1780's test sets #974's preview_checked; rustfmt
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 22:35:34 -04:00
Brandon Thomas 32e870d589 Merge PR #1694
# Conflicts:
#	docs/cr3.md
#	docs/whats-new.md
2026-10-14 22:33:20 -04:00
Brandon Thomas 133d575b8a Merge PR #1770
# Conflicts:
#	crates/raw/src/vendor/nef.rs
#	docs/parity-checklist.md
#	docs/raw-parity.md
#	docs/whats-new.md
2026-10-14 22:33:17 -04:00
Brandon Thomas c809c249d1 Merge PR #1792 2026-10-14 22:31:46 -04:00
Brandon Thomas 2c4390e2d0 Merge PR #1776
# Conflicts:
#	docs/whats-new.md
2026-10-14 22:31:40 -04:00
Brandon Thomas 141aec6ddf Merge PR #1731 2026-10-14 22:29:36 -04:00
Brandon Thomas a8f0f5d60b Merge PR #1715
# Conflicts:
#	docs/gaps.md
#	docs/parity-checklist.md
#	docs/target-app-parity.md
2026-10-14 22:29:33 -04:00
Brandon Thomas 1aea595527 Merge PR #1696
# Conflicts:
#	contributors/people.toml
2026-10-14 22:29:28 -04:00
Brandon Thomas d9508ff909 Merge PR #1467 2026-10-14 22:29:15 -04:00
Brandon Thomas 60696fe7c1 Merge PR #1462 2026-10-14 22:29:12 -04:00
Brandon Thomas 0f842c8f87 Merge PR #1461
# Conflicts:
#	crates/raw/tests/robust.rs
2026-10-14 22:29:11 -04:00
Brandon Thomas b6eb559243 Merge PR #1053
# Conflicts:
#	crates/raw/src/vendor/nef.rs
#	docs/gaps.md
#	docs/parity-checklist.md
#	docs/raw-parity.md
#	docs/whats-new.md
2026-10-14 22:01:32 -04:00
Brandon Thomas d8b52d9405 Merge PR #1105
# Conflicts:
#	ROADMAP.md
#	docs/gaps.md
#	docs/parity-checklist.md
#	docs/raw-parity.md
2026-10-14 21:59:32 -04:00
Brandon Thomas aba04aaf5c Merge PR #1087
# Conflicts:
#	ROADMAP.md
#	docs/gaps.md
#	docs/parity-checklist.md
#	docs/raw-parity.md
#	docs/raw/coverage.tsv
#	docs/raw/evidence.tsv
#	xtask/src/main.rs
2026-10-14 21:59:31 -04:00
Brandon Thomas d3edca8000 Merge PR #1081
# Conflicts:
#	ROADMAP.md
#	docs/gaps.md
#	docs/parity-checklist.md
#	docs/raw-parity.md
#	docs/raw/coverage.tsv
#	xtask/src/main.rs
2026-10-14 21:59:29 -04:00
Brandon Thomas 6ab5e2b0d4 Merge PR #1108
# Conflicts:
#	ROADMAP.md
#	docs/gaps.md
#	docs/parity-checklist.md
#	docs/raw-parity.md
#	docs/raw/evidence.tsv
#	xtask/src/main.rs
2026-10-14 21:59:28 -04:00
Brandon Thomas 381b57dd0b Merge PR #1096
# Conflicts:
#	ROADMAP.md
#	docs/gaps.md
#	docs/parity-checklist.md
#	docs/raw-parity.md
#	xtask/src/main.rs
2026-10-14 21:59:10 -04:00
Brandon Thomas a19cbdb17b Merge PR #1045
# Conflicts:
#	ROADMAP.md
#	docs/gaps.md
#	docs/parity-checklist.md
#	docs/raw-parity.md
#	docs/target-app-parity.md
2026-10-14 21:59:07 -04:00