Merge pr/zoom-detail (with upstream/main) into pr/overlay-windows

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Andrew Frahn
2026-10-14 19:30:44 +11:00
148 changed files with 6648 additions and 662 deletions
+5
View File
@@ -58,6 +58,11 @@ jobs:
'Key="Software\LightCraft" Name="InstallFolder"' 'Value="LaunchLightcraft"'; do
grep -qF "$want" packaging/windows/lightcraft.wxs || { echo "::error::lightcraft.wxs lost: $want"; exit 1; }
done
# The completion page the Publish events point at must actually exist: a Publish alone
# does not pull the dialog fragment into the MSI, and without the DialogRef the
# installer finishes silently (issues #790, #892).
grep -qF '<DialogRef Id="ExitDialog" />' packaging/windows/lightcraft.wxs \
|| { echo "::error::ExitDialog is not referenced; installs and removes end without a completion dialog"; exit 1; }
desktop-file-validate packaging/linux/ai.storyteller.lightcraft.desktop
sed -e 's/@VERSION@/0.0.0/' -e 's/@DATE@/2026-01-01/' packaging/linux/ai.storyteller.lightcraft.metainfo.xml.in >"$RUNNER_TEMP/ai.storyteller.lightcraft.metainfo.xml"
appstreamcli validate --no-net --explain "$RUNNER_TEMP/ai.storyteller.lightcraft.metainfo.xml"
+2 -1
View File
@@ -16,7 +16,8 @@ LightCraft is a clean-room, open-source, pure-Rust photo library + non-destructi
When you land a feature, update its row(s) and the gap entry in the same commit, and the numbers in `ROADMAP.md` /
`docs/target-app-parity.md` when a listed gap closes (bump each doc's *Last updated* line and revision history, per
craftrules `standards/progress-docs.md`); `cargo xtask parity` (part of `ci`) checks that every `cmd:`/`ctl:` id and
path the tracker cites still exists, and `cargo xtask parity --write` refreshes its summary.
path the tracker cites still exists and that `docs/gaps.md` lists each gap once, and `cargo xtask parity --write`
refreshes its summary.
5. Follow the autonomous operation protocol (`plan/execution-plan.md` §7). Don't stop to ask unless §7 lists the decision as the user's.
`plan/` is gitignored (local-only).
+29 -2
View File
@@ -337,7 +337,7 @@ LightCraft is young and moving fast: **alpha**. **Where we honestly stand** (det
| Import: Add in place / Copy / Move, rename and folder templates, devices, duplicate detection, watched folders; Local folder browsing | ✅ |
| MCP server (headless or live app, persistent libraries), CLI, control channel | ✅ |
| XMP sidecars (read/write, auto-write), reading `crs:` develop settings, preset files (`.lcpreset`, XMP presets) | ✅ |
| Optics (distortion, vignetting, auto + manual CA, defringe, lens corrections embedded in DNG files and Panasonic / Leica RW2 / RWL distortion data), Geometry (transforms, Constrain Crop), Upright (Auto/Level/Vertical/Full/Guided) | ✅ · camera lens profiles (our own) ⬜ |
| Optics (distortion, vignetting, auto + manual CA, defringe, lens corrections embedded in DNG files, Panasonic / Leica RW2 / RWL distortion data and Nikon Z 6II / Z 8 / Z f distortion data), Geometry (transforms, Constrain Crop), Upright (Auto/Level/Vertical/Full/Guided) | ✅ · camera lens profiles (our own) ⬜ |
| Photo Merge: HDR (auto-align, deghost), Panorama (spherical/cylindrical/perspective, boundary warp, auto crop), HDR Panorama → DNG | ✅ |
| GPU pipeline (wgpu compute, CPU-exact within 1/255), CPU fallback on device limits / errors | ✅ · WebGPU in the browser 🚧 |
| AI: segmentation masks, AI denoise, super resolution, faces; video | 🚧 (see [gaps](docs/gaps.md)) |
@@ -370,6 +370,31 @@ per CPU, otherwise exhaust RAM before any test runs. GPU coverage is unchanged.
`CARGO_PROFILE_DEV_DEBUG`, `CARGO_BUILD_JOBS` and `RUST_TEST_THREADS` settings
override these defaults. Ordinary development commands keep their own settings.
**Release builds on memory-constrained machines:** [#1196](https://github.com/storytold/lightcraft/issues/1196)
reports Arch/AUR release-link OOM kills on a 16 GB machine when the desktop and CLI build concurrently
with full debug information, thin LTO and one codegen unit. The CI defaults above do not apply to
ordinary `cargo build --release`. Start with one job; two may be suitable if enough memory is available:
```sh
CARGO_BUILD_JOBS=1 cargo build --release -p lightcraft -p lightcraft-cli
# Alternatively: CARGO_BUILD_JOBS=2 cargo build --release -p lightcraft -p lightcraft-cli
```
If a packaging environment enables full debug information or memory-heavy LTO settings, use
command-local overrides for a lower-memory source build:
```sh
CARGO_BUILD_JOBS=1 CARGO_PROFILE_RELEASE_DEBUG=0 CARGO_PROFILE_RELEASE_LTO=false CARGO_PROFILE_RELEASE_CODEGEN_UNITS=16 cargo build --release -p lightcraft -p lightcraft-cli
```
These POSIX-shell assignments affect only this command, not global Cargo configuration or the
repository's release profiles. Disabling LTO may change performance and binary size; disabling debug
information reduces debugging detail. This is build guidance, not a guarantee that any machine has
enough memory. Downstream packagers should serialize large binary builds and avoid full debug
information combined with LTO and one codegen unit on low-memory builders. A PKGBUILD that exports
its own profile settings can override inherited variables; adjust the packaging recipe locally or
contact its maintainer rather than assuming an environment prefix on `makepkg` will take precedence.
**Chinese and Japanese text** need the shared font repo, an optional build input (official releases always include it):
```sh
@@ -385,7 +410,9 @@ Language**; the choice applies immediately and persists. See [docs/localization.
**Logs:** the desktop app writes its log to standard error and to `logs/lightcraft.log` in its settings folder
(Linux `$XDG_CONFIG_HOME/lightcraft/logs/`, by default `~/.config/lightcraft/logs/`; macOS
`~/Library/Application Support/LightCraft/logs/`; Windows `%APPDATA%\LightCraft\logs\`), never in the library. A
`~/Library/Application Support/LightCraft/logs/`; Windows `%APPDATA%\LightCraft\logs\`), never in the library.
**Portable mode:** a file named `portable.txt` (or `LightCraft.portable`) next to the executable moves the settings
folder to `LightCraftData` beside it. A
launch from a desktop menu or the Dock has no terminal, so attach this file to a bug report (**Help ▸ Open Log
Folder** shows it in the file manager). Each start moves the
previous log to `lightcraft.1.log` and that one to `lightcraft.2.log`, so the log of a run that crashed survives the
+24
View File
@@ -359,6 +359,30 @@ CI 默认使用行表调试信息,并按启动时可用内存每 1.5 GB 开一
`CARGO_PROFILE_DEV_DEBUG`、`CARGO_BUILD_JOBS` 和 `RUST_TEST_THREADS` 会覆盖这些默认值。
日常开发命令保持它们各自的设置。
**内存受限机器上的 release 构建:** [#1196](https://github.com/storytold/lightcraft/issues/1196)
报告在 16 GB 的 Arch/AUR 机器上,同时构建桌面应用和 CLI,并启用完整调试信息、thin LTO
和单个代码生成单元时,链接过程被 OOM 终止。上述 CI 默认设置不适用于普通的
`cargo build --release`。建议先使用一个构建任务;内存充足时可以尝试两个:
```sh
CARGO_BUILD_JOBS=1 cargo build --release -p lightcraft -p lightcraft-cli
# 或者:CARGO_BUILD_JOBS=2 cargo build --release -p lightcraft -p lightcraft-cli
```
如果打包环境启用了完整调试信息或占用较多内存的 LTO 设置,可以使用仅对本条命令生效的
覆盖设置,降低源码构建的内存需求:
```sh
CARGO_BUILD_JOBS=1 CARGO_PROFILE_RELEASE_DEBUG=0 CARGO_PROFILE_RELEASE_LTO=false CARGO_PROFILE_RELEASE_CODEGEN_UNITS=16 cargo build --release -p lightcraft -p lightcraft-cli
```
这些 POSIX shell 赋值只影响本条命令,不修改全局 Cargo 配置或仓库的 release profile。
关闭 LTO 可能改变性能和二进制大小;关闭调试信息会减少可用的调试细节。这是构建建议,
并不保证任何机器都有足够内存。下游打包者应串行构建大型二进制文件,并避免在低内存
构建机器上同时使用完整调试信息、LTO 和单个代码生成单元。PKGBUILD 自己导出的 profile
设置可以覆盖继承的环境变量;应在本地调整打包配方或联系维护者,而不要假定给
`makepkg` 添加环境变量前缀就能优先生效。
**中文和日文文字**需要共享字体仓库,这是一个可选的构建输入(所有正式发布版都会包含它):
```sh
+10 -1
View File
@@ -462,6 +462,14 @@ fn services(cc: &eframe::CreationContext<'_>, log_file: Option<&std::path::Path>
pick_folder: Some(Box::new(|| {
file_dialog().set_title(lightcraft_ui_egui::i18n::tr("Open Library")).pick_folder().map(|p| p.to_string_lossy().to_string())
})),
#[cfg(target_os = "linux")]
open_model_folder: Some(std::sync::Arc::new(|path: &str| {
std::process::Command::new("xdg-open")
.arg(path)
.status()
.map_err(|e| e.to_string())
.and_then(|s| if s.success() { Ok(()) } else { Err(format!("opening folder failed: {s}")) })
})),
open_with: Some(Box::new(editors::open_with)),
open_all_with: Some(Box::new(editors::open_all_with)),
check_editor: Some(Box::new(editors::check)),
@@ -666,7 +674,8 @@ OPTIONS:
--version, --help
ENVIRONMENT:
LIGHTCRAFT_CONFIG_DIR=DIR use this settings folder instead of the platform default (empty = no settings folder)
LIGHTCRAFT_CONFIG_DIR=DIR use this settings folder instead of the platform default (empty = no settings folder).
Without it, a portable.txt file next to the program puts the folder beside it (LightCraftData)
LIGHTCRAFT_GPU_BACKEND=dx12|vulkan|metal|auto|off graphics backend (default: DX12 on Windows, Metal on macOS,
Vulkan on Linux; off = render on the CPU); else WGPU_BACKEND. LIGHTCRAFT_GPU=0: CPU rendering.
LIGHTCRAFT_SAM3_DIR=DIR the SAM 3 model for Object / Describe masks (default: <settings folder>/models/sam3;
+4
View File
@@ -27,3 +27,7 @@ sync_github_name = true
[people.sanyamsingh001]
display_name = "sanyamsingh001"
[people.ArekatlaNishanthchowdary]
real_name = "Arekatla Nishanth Chowdary"
sync_github_name = true
+8 -1
View File
@@ -113,6 +113,8 @@ pub struct LoadReport {
/// The library was in an older catalog format (`0` = before format versions: a log without a
/// snapshot) and was rewritten in this version's format ([`VERSION`]).
pub upgraded_from: Option<u32>,
/// What [`Catalog::repair`] changed in a damaged snapshot (empty for a sound one).
pub repaired: Vec<String>,
}
/// Where persistence time goes (reported by `library.info` → `persistence`; printed to stderr
@@ -328,7 +330,12 @@ impl Journal {
return Err(newer(format!("{SNAPSHOT} is catalog format v{}", h.version)));
}
let s: SnapFile = serde_json::from_slice(&bytes).map_err(|e| CatalogError::Corrupt(format!("{SNAPSHOT}: {e}")))?;
(s.catalog, s.seq, Some(h.version))
let mut catalog = s.catalog;
report.repaired = catalog.repair();
for r in &report.repaired {
log::warn!("{SNAPSHOT}: {r}");
}
(catalog, s.seq, Some(h.version))
}
None => (Catalog::new(), 0, None),
};
+102 -9
View File
@@ -290,19 +290,21 @@ impl Catalog {
// ---- ids
// (saturating: a counter a damaged snapshot left at the top hands out its last id again, which
// the op that adds it then refuses as taken, rather than overflowing)
pub fn alloc_photo_id(&mut self) -> PhotoId {
let id = PhotoId(self.next_photo.max(1));
self.next_photo = id.0 + 1;
self.next_photo = id.0.saturating_add(1);
id
}
pub fn alloc_album_id(&mut self) -> AlbumId {
let id = AlbumId(self.next_album.max(1));
self.next_album = id.0 + 1;
self.next_album = id.0.saturating_add(1);
id
}
pub fn alloc_stack_id(&mut self) -> StackId {
let id = StackId(self.next_stack.max(1));
self.next_stack = id.0 + 1;
self.next_stack = id.0.saturating_add(1);
id
}
@@ -497,7 +499,7 @@ impl Catalog {
return Err(CatalogError::Invalid(format!("photo {:?} exists", photo.id)));
}
let id = photo.id;
self.next_photo = self.next_photo.max(id.0 + 1);
self.next_photo = self.next_photo.max(id.0.saturating_add(1));
self.photos.insert(id, Arc::new(*photo));
Op::RemovePhoto { id }
}
@@ -587,7 +589,7 @@ impl Catalog {
self.validate_rules(rules)?;
}
let id = album.id;
self.next_album = self.next_album.max(id.0 + 1);
self.next_album = self.next_album.max(id.0.saturating_add(1));
self.albums.insert(id, album);
Op::RemoveAlbum { id }
}
@@ -607,13 +609,15 @@ impl Catalog {
if p == id || !self.albums.get(&p).is_some_and(|a| a.folder) {
return Err(CatalogError::Invalid("parent must be another folder".into()));
}
// no cycles
// no cycles (bounded: a loop elsewhere in a damaged catalog must not hang this)
let mut cur = Some(p);
let mut steps = 0usize;
while let Some(c) = cur {
if c == id {
if c == id || steps > self.albums.len() {
return Err(CatalogError::Invalid("cannot move a folder into itself".into()));
}
cur = self.albums.get(&c).and_then(|a| a.parent);
steps += 1;
}
}
let a = self.album_mut(id)?;
@@ -659,7 +663,7 @@ impl Catalog {
}
self.validate_stack(stack.id, &stack.photos)?;
let id = stack.id;
self.next_stack = self.next_stack.max(id.0 + 1);
self.next_stack = self.next_stack.max(id.0.saturating_add(1));
self.stacks.insert(id, stack);
Op::RemoveStack { id }
}
@@ -823,7 +827,94 @@ impl Catalog {
}
pub fn from_snapshot(s: &str) -> Result<Catalog> {
serde_json::from_str(s).map_err(|e| CatalogError::Corrupt(e.to_string()))
let mut c: Catalog = serde_json::from_str(s).map_err(|e| CatalogError::Corrupt(e.to_string()))?;
c.repair();
Ok(c)
}
/// Make a catalog read from a snapshot hold together where a damaged or hand-edited file left
/// it inconsistent, so nothing that relies on it can fail (code indexes stacks by the ids it
/// finds in them, and walks folders up to the top): every photo, album and stack carries the
/// id it is filed under; the id counters lie past every id in use; a stack keeps only photos
/// in the catalog, once each, and goes when fewer than two are left; an album whose parent is
/// missing, not a folder or inside itself moves to the top level. Returns what was changed
/// (nothing, for a catalog this build wrote).
pub fn repair(&mut self) -> Vec<String> {
let mut changed = Vec::new();
for (id, p) in self.photos.iter_mut() {
if p.id != *id {
changed.push(format!("photo {} was filed as {}", p.id.0, id.0));
Arc::make_mut(p).id = *id;
}
}
for (id, a) in self.albums.iter_mut() {
if a.id != *id {
changed.push(format!("album {} was filed as {}", a.id.0, id.0));
a.id = *id;
}
}
let photos = &self.photos;
self.stacks.retain(|id, st| {
if st.id != *id {
changed.push(format!("stack {} was filed as {}", st.id.0, id.0));
st.id = *id;
}
let mut seen = std::collections::HashSet::new();
let before = st.photos.len();
st.photos.retain(|p| photos.contains_key(p) && seen.insert(*p));
if st.photos.len() != before {
changed.push(format!("stack {}: {} missing or repeated photo(s) left out", id.0, before - st.photos.len()));
}
let keep = st.photos.len() >= 2;
if !keep {
changed.push(format!("stack {} had fewer than two photos: removed", id.0));
}
keep
});
// a photo in two stacks stays in the first
let mut stacked = std::collections::HashSet::new();
self.stacks.retain(|id, st| {
let before = st.photos.len();
st.photos.retain(|p| stacked.insert(*p));
if st.photos.len() != before {
changed.push(format!("stack {}: {} photo(s) already in another stack left out", id.0, before - st.photos.len()));
}
st.photos.len() >= 2
});
let parents: Vec<(AlbumId, Option<AlbumId>)> = self.albums.values().map(|a| (a.id, a.parent)).collect();
for (id, parent) in parents {
let Some(p) = parent else { continue };
let mut cur = Some(p);
let mut steps = 0usize;
let ok = loop {
match cur {
None => break true,
Some(c) if c == id || steps > self.albums.len() => break false,
Some(c) => match self.albums.get(&c) {
Some(a) if a.folder => cur = a.parent,
_ => break false,
},
}
steps += 1;
};
if !ok && let Some(a) = self.albums.get_mut(&id) {
changed.push(format!("album {}: its folder {} is missing, not a folder or inside it: moved to the top level", id.0, p.0));
a.parent = None;
}
}
let past = |last: Option<u64>| last.map_or(1, |v| v.saturating_add(1));
let (photo, album, stack) = (
past(self.photos.keys().last().map(|k| k.0)),
past(self.albums.keys().last().map(|k| k.0)),
past(self.stacks.keys().last().map(|k| k.0)),
);
if self.next_photo < photo || self.next_album < album || self.next_stack < stack {
changed.push("id counters behind the ids in use: moved past them".into());
self.next_photo = self.next_photo.max(photo);
self.next_album = self.next_album.max(album);
self.next_stack = self.next_stack.max(stack);
}
changed
}
/// Replay an op log (JSON lines) on top of `self`. A torn final line (crash mid-write) is ignored.
@@ -878,4 +969,6 @@ mod tests_local;
#[cfg(test)]
mod tests_lock;
#[cfg(test)]
mod tests_repair;
#[cfg(test)]
mod tests_torn_append;
+2 -2
View File
@@ -253,8 +253,8 @@ impl Filter {
return false;
}
match &self.folder {
// browsed photos only show in folder views
None if p.local => return false,
// browsed photos only show in folder views, and in Recently Deleted once deleted (#1430)
None if p.local && !p.deleted => return false,
None => {}
Some(dir) => {
let crate::Source::File { path } = &p.source else { return false };
+33 -9
View File
@@ -20,8 +20,8 @@
//! it is then simply exported again.
//! - [`write_new`] / [`write_new_unique`] write a file that must not exist yet: the same temp file
//! and sync, plus a read-back check, and the file then appears under its final name **without
//! replacing anything** (a hard link, which fails on a taken name; a check-then-rename where the
//! volume has no hard links).
//! replacing anything** (a hard link, which fails on a taken name; where the volume has no hard
//! links, an exclusive create that claims the name, then a rename over that placeholder).
//! - [`same_file`] tells whether two paths name the same file on disk (device + inode on Unix,
//! resolved paths elsewhere), for "is this target an original?" checks.
@@ -226,16 +226,24 @@ fn publish_new(tmp: &Path, path: &Path) -> io::Result<()> {
match fs::hard_link(tmp, path) {
Ok(()) => Ok(()),
Err(e) if e.kind() == io::ErrorKind::AlreadyExists => Err(e),
// no hard links on this volume (FAT/exFAT, some shares): check, then rename
Err(_) => {
if fs::symlink_metadata(path).is_ok() {
return Err(io::Error::new(io::ErrorKind::AlreadyExists, format!("{} exists", path.display())));
}
fs::rename(tmp, path)
}
// no hard links on this volume (FAT/exFAT, some shares)
Err(_) => publish_new_by_rename(tmp, path, &|| {}),
}
}
/// Fallback of [`publish_new`]: reserve the name with an exclusive create (atomic, so a file that
/// appears at the same time is never replaced), then rename the complete temp file over our own
/// empty placeholder. `before_claim` runs just before the name is claimed (a test hook).
fn publish_new_by_rename(tmp: &Path, path: &Path, before_claim: &dyn Fn()) -> io::Result<()> {
before_claim();
OpenOptions::new().write(true).create_new(true).open(path)?;
if let Err(e) = fs::rename(tmp, path) {
let _ = fs::remove_file(path);
return Err(e);
}
Ok(())
}
/// Does the file at `path` hold exactly `data`?
fn same_content(path: &Path, data: &[u8]) -> io::Result<bool> {
use std::io::Read;
@@ -296,6 +304,22 @@ mod tests {
v
}
#[test]
fn fallback_publish_never_replaces_a_file_that_appears_at_the_same_time() {
let d = temp("nolink-race");
let tmp = d.join(".tmp-new");
let dest = d.join("IMG_1.dng");
fs::write(&tmp, b"new").unwrap();
let e = publish_new_by_rename(&tmp, &dest, &|| fs::write(&dest, b"other writer").unwrap()).unwrap_err();
assert_eq!(e.kind(), io::ErrorKind::AlreadyExists);
assert_eq!(fs::read(&dest).unwrap(), b"other writer");
fs::remove_file(&dest).unwrap();
publish_new_by_rename(&tmp, &dest, &|| {}).unwrap();
assert_eq!(fs::read(&dest).unwrap(), b"new");
assert!(!tmp.exists());
let _ = fs::remove_dir_all(&d);
}
#[test]
fn atomic_write_replaces_and_leaves_no_temp() {
let d = temp("replace");
+6 -3
View File
@@ -25,7 +25,7 @@ impl Catalog {
}
/// Photo → (stack, position in the stack), for every stacked photo.
pub fn stack_index(&self) -> HashMap<PhotoId, (StackId, usize)> {
self.stacks.values().flat_map(|s| s.photos.iter().enumerate().map(move |(i, p)| (*p, (s.id, i)))).collect()
self.stacks.iter().flat_map(|(id, s)| s.photos.iter().enumerate().map(move |(i, p)| (*p, (*id, i)))).collect()
}
pub(crate) fn validate_stack(&self, id: StackId, photos: &[PhotoId]) -> Result<()> {
@@ -70,7 +70,7 @@ impl Catalog {
}
}
for sid in &touched {
for p in self.stacks[sid].photos.clone() {
for p in self.stacks.get(sid).map(|s| s.photos.clone()).unwrap_or_default() {
push(&mut order, p);
}
}
@@ -186,7 +186,10 @@ impl Catalog {
if !done.insert(*sid) {
continue;
}
let s = &self.stacks[sid];
let Some(s) = self.stacks.get(sid) else {
out.push(*id);
continue;
};
let members = s.photos.iter().copied().filter(|p| present.contains(p));
if s.collapsed {
out.extend(members.take(1));
+17
View File
@@ -237,6 +237,23 @@ fn recently_deleted_shows_the_deleted_photos_of_the_chosen_folder() {
assert!(!f.matches(c.photo(kept).unwrap(), &c));
}
#[test]
fn a_deleted_browsed_photo_shows_in_recently_deleted_and_comes_back() {
let mut c = Catalog::new();
let browsed = add_with(&mut c, "/pics/trip/a.jpg", |p| p.local = true);
let recently_deleted = Filter { deleted: true, ..Default::default() };
let library = Filter::default();
assert!(!recently_deleted.matches(c.photo(browsed).unwrap(), &c));
assert!(!library.matches(c.photo(browsed).unwrap(), &c), "browsed photos stay out of the library view");
c.apply(Op::SetDeleted { id: browsed, deleted: true }).unwrap();
assert_eq!(c.query(&recently_deleted, &Sort::default()), vec![browsed], "#1430: it was invisible everywhere");
assert!(c.query(&library, &Sort::default()).is_empty());
c.apply(Op::SetDeleted { id: browsed, deleted: false }).unwrap();
assert!(c.query(&recently_deleted, &Sort::default()).is_empty());
}
#[test]
fn an_empty_folder_choice_is_no_choice() {
let c = library(&["/pics/a.jpg"]);
+152
View File
@@ -0,0 +1,152 @@
//! A damaged or hand-edited snapshot that still parses is made to hold together on load
//! ([`Catalog::repair`]): found by fuzzing snapshots of a real library.
use crate::journal::SNAPSHOT;
use crate::*;
fn album(id: u64, parent: Option<u64>, folder: bool, photos: Vec<PhotoId>) -> Album {
Album {
id: AlbumId(id),
name: format!("A{id}"),
parent: parent.map(AlbumId),
folder,
photos,
cover: None,
smart: None,
quick: false,
order: None,
}
}
/// A catalog of four photos, a stack of two, a folder with an album in it.
fn sound() -> (Catalog, Vec<PhotoId>) {
let mut c = Catalog::new();
let mut ids = Vec::new();
for i in 0..4 {
let id = c.alloc_photo_id();
c.apply(Op::AddPhoto {
photo: Box::new(Photo::new(id, Source::File { path: format!("/p/{i}.jpg") }, &format!("{i}.jpg"), "JPEG", 60, 40, "2026-01-01")),
})
.unwrap();
ids.push(id);
}
let sid = c.alloc_stack_id();
c.apply(Op::AddStack { stack: Stack { id: sid, photos: vec![ids[0], ids[1]], collapsed: true } }).unwrap();
let (f, a) = (c.alloc_album_id(), c.alloc_album_id());
c.apply(Op::AddAlbum { album: album(f.0, None, true, vec![]) }).unwrap();
c.apply(Op::AddAlbum { album: album(a.0, Some(f.0), false, vec![ids[2]]) }).unwrap();
(c, ids)
}
/// The snapshot file of `c` with `edit` applied to its JSON, loaded through the journal.
fn load_edited(c: &Catalog, edit: impl FnOnce(&mut serde_json::Value)) -> (Catalog, LoadReport) {
let m = MemStore::new();
let (mut j, _, _) = Journal::open(Box::new(m.clone())).unwrap();
j.snapshot(c).unwrap();
let mut v: serde_json::Value = serde_json::from_slice(&m.get(SNAPSHOT).unwrap()).unwrap();
edit(&mut v["catalog"]);
m.set(SNAPSHOT, serde_json::to_vec(&v).unwrap());
drop(j);
let (_, c, report) = Journal::open(Box::new(m)).unwrap();
(c, report)
}
/// A stack whose `id` differs from the key it is filed under made `arrange_stacks` index the map
/// with an id it doesn't hold, which panics: the grid of any library with such a stack.
#[test]
fn a_stack_filed_under_another_id_is_arranged() {
let (c, ids) = sound();
let (c, report) = load_edited(&c, |v| {
let key = v["stacks"].as_object().unwrap().keys().next().unwrap().clone();
v["stacks"][&key]["id"] = 77.into();
});
assert!(report.repaired.iter().any(|r| r.contains("stack 77 was filed as")), "{:?}", report.repaired);
assert_eq!(c.arrange_stacks(&ids), vec![ids[0], ids[2], ids[3]], "the collapsed stack shows its top");
assert!(c.stacks().all(|s| c.stack_of(s.photos[0]).is_some_and(|t| t.id == s.id)));
}
/// Stacks of missing, repeated or too few photos (or one photo in two stacks) are put right: a
/// stack of none had no top (`photos[0]`).
#[test]
fn stacks_keep_only_photos_in_the_catalog() {
let (c, ids) = sound();
let (c, report) = load_edited(&c, |v| {
let key = v["stacks"].as_object().unwrap().keys().next().unwrap().clone();
v["stacks"][&key]["photos"] = serde_json::json!([ids[0].0, 999, ids[0].0, ids[1].0]);
v["stacks"]["8"] = serde_json::json!({"id": 8, "photos": []});
v["stacks"]["9"] = serde_json::json!({"id": 9, "photos": [ids[1].0, ids[3].0, ids[2].0]});
});
let stacks: Vec<(StackId, Vec<PhotoId>)> = c.stacks().map(|s| (s.id, s.photos.clone())).collect();
assert_eq!(stacks.len(), 2, "{stacks:?} {:?}", report.repaired);
assert_eq!(stacks[0].1, vec![ids[0], ids[1]], "missing and repeated photos left out");
assert_eq!(stacks[1].1, vec![ids[3], ids[2]], "a photo already in a stack stays in the first");
assert!(!c.arrange_stacks(&ids).is_empty());
}
/// Folders inside each other (a loop), or under an album or a missing folder, move to the top
/// level: walking up from them never ends otherwise, and moving any folder there hung.
#[test]
fn folder_loops_and_missing_parents_move_to_the_top() {
let (c, _) = sound();
let (mut c, report) = load_edited(&c, |v| {
v["albums"]["10"] = serde_json::to_value(album(10, Some(11), true, vec![])).unwrap();
v["albums"]["11"] = serde_json::to_value(album(11, Some(10), true, vec![])).unwrap();
v["albums"]["12"] = serde_json::to_value(album(12, Some(12), true, vec![])).unwrap();
v["albums"]["13"] = serde_json::to_value(album(13, Some(404), false, vec![])).unwrap();
});
assert!(!report.repaired.is_empty());
for id in [10, 11, 12, 13] {
let a = c.album(AlbumId(id)).unwrap();
assert!(
a.parent.is_none() || a.parent.is_some_and(|p| c.album(p).is_some_and(|f| f.folder && f.parent.is_none())),
"album {id}: {:?}",
a.parent
);
}
// moving a folder around terminates either way
let _ = c.apply(Op::MoveAlbum { id: AlbumId(10), parent: Some(AlbumId(11)) });
let _ = c.apply(Op::MoveAlbum { id: AlbumId(11), parent: Some(AlbumId(10)) });
}
/// Id counters left at the top (or behind the ids in use) by a damaged file: allocating no longer
/// overflows (a panic in debug builds) and never hands out an id already in use.
#[test]
fn id_counters_are_moved_past_the_ids_in_use() {
let (c, ids) = sound();
let (mut c, report) = load_edited(&c, |v| {
v["next_photo"] = 1.into();
v["next_album"] = 1.into();
v["next_stack"] = 1.into();
});
assert!(report.repaired.iter().any(|r| r.contains("id counters")), "{:?}", report.repaired);
assert!(c.alloc_photo_id().0 > ids[3].0);
let (mut c, _) = load_edited(&c, |v| v["next_photo"] = u64::MAX.into());
assert_eq!((c.alloc_photo_id().0, c.alloc_photo_id().0), (u64::MAX, u64::MAX), "saturates");
let p = Photo::new(PhotoId(u64::MAX), Source::Demo { scene: 1 }, "a", "JPEG", 1, 1, "t");
c.apply(Op::AddPhoto { photo: Box::new(p.clone()) }).unwrap();
assert!(c.apply(Op::AddPhoto { photo: Box::new(p) }).is_err(), "the id is taken: refused, not overwritten");
}
/// A sound catalog loads unchanged.
#[test]
fn a_sound_snapshot_needs_no_repair() {
let (c, _) = sound();
let (loaded, report) = load_edited(&c, |_| {});
assert!(report.repaired.is_empty(), "{:?}", report.repaired);
assert_eq!(loaded.to_snapshot(), c.to_snapshot());
}
/// Found by fuzzing: an album filed under one id but naming another (here: the folder's) put
/// itself inside itself, and listing the folder tree overflowed the stack (an abort, which no
/// panic guard catches, every time the library opened).
#[test]
fn an_album_filed_under_another_id_gets_its_own() {
let (c, _) = sound();
let folder = c.albums().find(|a| a.folder).unwrap().id;
let inner = c.albums().find(|a| !a.folder).unwrap().id;
let (c, report) = load_edited(&c, |v| v["albums"][inner.0.to_string()]["id"] = folder.0.into());
assert!(report.repaired.iter().any(|r| r.contains(&format!("album {} was filed as {}", folder.0, inner.0))), "{:?}", report.repaired);
let a = c.album(inner).unwrap();
assert_eq!((a.id, a.parent), (inner, Some(folder)));
assert_eq!(c.album_children(Some(folder)).iter().map(|a| a.id).collect::<Vec<_>>(), vec![inner]);
}
+5
View File
@@ -5,3 +5,8 @@ No weights or inference backend. See [the user workflow](../../docs/denoise.md).
`lightcraft-denoise` re-exports these types for CPU/GPU users; the engine uses this
crate directly when inference is disabled. The default download catalog is empty.
Run `cargo test -p lightcraft-denoise-core`; optional catalog checks use `--features rawnind-model`.
The Bayer runner rejects non-finite source samples before inference and non-finite
results after gain matching, highlight protection and tile blending. Finite negative
and HDR source samples remain supported. These guards do not implement Raw Details
or Super Resolution.
+56
View File
@@ -95,6 +95,9 @@ pub fn denoise_bayer(
if mosaic.len() != pixels {
return Err(Error::Input("the mosaic does not have width × height samples".into()));
}
if mosaic.iter().any(|v| !v.is_finite()) {
return Err(Error::Input("the mosaic has samples that are not finite".into()));
}
if !(16..=2048).contains(&p.tile) || p.overlap >= p.tile.div_ceil(2) || layout.dx > 1 || layout.dy > 1 {
return Err(Error::Input("the tile size and overlap do not fit".into()));
}
@@ -148,6 +151,9 @@ pub fn denoise_bayer(
return Err(Error::Model("numbers that are not finite".into()));
}
finish_tile(&input, &mut out, p, (bx, ax, by, ay));
if out.iter().any(|v| !v.is_finite()) {
return Err(Error::Model("numbers that are not finite after scale matching or highlight protection".into()));
}
Ok(out)
})
.collect()
@@ -201,6 +207,9 @@ pub fn denoise_bayer(
}
}
});
if acc.iter().flatten().any(|v| !v.is_finite()) {
return Err(Error::Model("numbers that are not finite after tile blending".into()));
}
if std::env::var_os("LIGHTCRAFT_PROFILE").is_some() {
eprintln!(
"[denoise] {total} tiles, {} at once: tiles {:.0} ms (blending, {:.0} ms of it, overlaps the model), normalise {:.0} ms",
@@ -388,6 +397,53 @@ mod tests {
Params { tile, overlap, gain, clip: None, parallel: 2 }
}
#[test]
fn nonfinite_mosaics_are_rejected_before_inference() {
struct Counting(std::sync::atomic::AtomicUsize);
impl TileRunner for Counting {
fn run(&self, _: &[f32]) -> Result<Vec<f32>, Error> {
self.0.fetch_add(1, Ordering::Relaxed);
Ok(vec![0.0; 3 * 32 * 32])
}
}
let model = Counting(std::sync::atomic::AtomicUsize::new(0));
let layout = Layout { dx: 0, dy: 0 };
for sample in [f32::NAN, f32::INFINITY, f32::NEG_INFINITY] {
assert!(matches!(denoise_bayer(&[sample], 1, 1, layout, &model, &params(16, 0, Gain::None), &Control::default()), Err(Error::Input(_))));
}
assert_eq!(model.0.load(Ordering::Relaxed), 0);
// Finite scene-linear samples can be negative or exceed SDR white.
for sample in [-0.25, 2.0] {
assert!(denoise_bayer(&[sample], 1, 1, layout, &model, &params(16, 0, Gain::None), &Control::default()).is_ok());
}
}
#[test]
fn finite_model_output_that_overflows_during_gain_matching_is_rejected() {
struct Extreme;
impl TileRunner for Extreme {
fn run(&self, _: &[f32]) -> Result<Vec<f32>, Error> {
Ok(vec![f32::MAX; 3 * 32 * 32])
}
}
let p = params(16, 0, Gain::MatchMean { nominal: 1e-6, max_deviation: 0.0 });
assert!(matches!(denoise_bayer(&[0.5], 1, 1, Layout { dx: 0, dy: 0 }, &Extreme, &p, &Control::default()), Err(Error::Model(_))));
}
#[test]
fn overlapping_finite_tiles_that_overflow_the_accumulator_are_rejected() {
struct Extreme;
impl TileRunner for Extreme {
fn run(&self, _: &[f32]) -> Result<Vec<f32>, Error> {
Ok(vec![f32::MAX; 3 * 32 * 32])
}
}
assert!(matches!(
denoise_bayer(&vec![0.5; 48 * 48], 48, 48, Layout { dx: 0, dy: 0 }, &Extreme, &params(16, 4, Gain::None), &Control::default()),
Err(Error::Model(_))
));
}
#[test]
fn overflowing_tiles_and_nan_gain_are_errors_before_the_model_runs() {
let layout = Layout::from_cell([0, 1, 1, 2]).unwrap();
+25 -1
View File
@@ -97,7 +97,12 @@ fn read_bytes(bytes: &[u8]) -> Result<Net, NetError> {
if next.input.len() != 1 || next.output.len() != 1 || next.attribute.iter().any(|a| a.name != "alpha") {
return invalid("malformed LeakyRelu");
}
let alpha = attr(next, "alpha").map_or(0.01, |a| a.f);
// only an omitted alpha gets the ONNX default; one that is not a single float must not read as 0
let alpha = match next.attribute.as_slice() {
[] => 0.01,
[AttributeProto { f: Some(f), kind: 0 | 1, .. }] => *f,
_ => return invalid("LeakyRelu alpha is not a single float"),
};
i += 1;
let name = next.output.first().cloned().ok_or_else(|| NetError::Invalid("LeakyRelu without an output".into()))?;
return Ok((Some(alpha), name));
@@ -319,6 +324,25 @@ mod tests {
let _ = std::fs::remove_file(p);
}
#[test]
fn a_leaky_relu_alpha_that_is_not_a_float_is_rejected() {
let good = unet_onnx(4, 3, 1, 7);
assert!(read_bytes(&good).is_ok());
// an alpha attribute: float field 2, then the declared type (field 20) FLOAT
let mut alpha = vec![0x15];
alpha.extend_from_slice(&0.2f32.to_le_bytes());
let at = good.windows(alpha.len()).position(|w| w == alpha.as_slice()).unwrap() + alpha.len();
assert_eq!(good[at..at + 3], [0xa0, 0x01, 1]);
// declared INT: the float must not be taken as the slope
let mut wrong_type = good.clone();
wrong_type[at + 2] = 2;
assert!(matches!(read_bytes(&wrong_type), Err(NetError::Invalid(_))));
// declared FLOAT but no float value (the float field becomes an int field 3 = 0 of the same length): must not read as 0
let mut no_float = good.clone();
no_float[at - 5..at].copy_from_slice(&[0x18, 0x80, 0x80, 0x80, 0x00]);
assert!(matches!(read_bytes(&no_float), Err(NetError::Invalid(_))));
}
#[test]
fn an_explicit_empty_or_nonvector_bias_is_not_treated_as_missing() {
let w = TensorProto { name: "w".into(), dims: vec![12, 4, 1, 1], data_type: 1, float_data: vec![0.1; 48], ..Default::default() };
+6 -3
View File
@@ -113,7 +113,10 @@ fn ints(w: Wire<'_>, out: &mut Vec<i64>, cap: usize) -> Result<()> {
#[derive(Default)]
pub(super) struct AttributeProto {
pub name: String,
pub f: f32,
/// `None` when the file has no float value for the attribute.
pub f: Option<f32>,
/// The declared attribute type (1 float, 2 int, 3 string, 7 ints); 0 when the file does not say.
pub kind: i64,
pub i: i64,
pub s: Vec<u8>,
pub ints: Vec<i64>,
@@ -151,7 +154,7 @@ fn attribute(b: &[u8]) -> Result<AttributeProto> {
while let Some((n, w)) = fields.next()? {
match (n, w) {
(1, Wire::Bytes(b)) => a.name = string(b)?,
(2, Wire::F32(v)) => a.f = v,
(2, Wire::F32(v)) if a.f.is_none() => a.f = Some(v),
(3, Wire::Int(v)) => a.i = v as i64,
(4, Wire::Bytes(b)) => {
if b.len() > MAX_STRING {
@@ -160,7 +163,7 @@ fn attribute(b: &[u8]) -> Result<AttributeProto> {
a.s = b.to_vec();
}
(8, w) => ints(w, &mut a.ints, 16)?,
(20, Wire::Int(1 | 2 | 3 | 7)) => {}
(20, Wire::Int(v @ (1 | 2 | 3 | 7))) if a.kind == 0 => a.kind = v as i64,
_ => return Err(NetError::Unsupported("unknown attribute value".into())),
}
}
+42
View File
@@ -0,0 +1,42 @@
//! UI-thread work one slider-drag tick costs outside the render: `develop.set` inside an
//! interaction, the filmstrip's `visible()` after the catalog revision moved, and the settings
//! hash the loupe takes every frame. `cargo run --release -p lightcraft-engine --example
//! drag_tick_bench -- [photos]` (default 20000).
#![allow(clippy::unwrap_used, clippy::expect_used)]
use lightcraft_catalog::{Op, Photo, PhotoId, Source};
use lightcraft_engine::Session;
use serde_json::json;
use std::time::Instant;
fn main() {
let n: u64 = std::env::args().nth(1).and_then(|a| a.parse().ok()).unwrap_or(20_000);
let mut s = Session::new();
for i in 1..=n {
let mut p = Photo::new(PhotoId(i), Source::File { path: format!("/x/{i}.ARW") }, &format!("{i}.ARW"), "ARW", 9984, 6656, "2026-01-01");
p.captured = Some(format!("2025-{:02}-{:02}T10:{:02}:00", 1 + i % 12, 1 + i % 28, i % 60));
s.catalog.apply(Op::AddPhoto { photo: Box::new(p) }).unwrap();
}
s.execute("library.select", &json!({"ids": [1]})).unwrap();
let ms = |t: Instant| t.elapsed().as_secs_f64() * 1e3;
s.visible_cloned();
s.execute("develop.beginInteraction", &json!({"label": "Exposure"})).unwrap();
let ticks = 30;
let (mut set, mut vis, mut hash) = (0.0, 0.0, 0.0);
for k in 0..ticks {
let t = Instant::now();
s.execute("develop.set", &json!({"control": "light.exposure", "value": k as f64 / 30.0})).unwrap();
set += ms(t);
let t = Instant::now();
let v = s.visible_cloned();
vis += ms(t);
assert_eq!(v.len() as u64, n);
let d = s.develop_of(PhotoId(1)).unwrap();
let t = Instant::now();
std::hint::black_box(d.hash64());
hash += ms(t);
}
s.execute("develop.endInteraction", &json!({})).unwrap();
let f = ticks as f64;
println!("{n} photos, per tick: develop.set {:.3} ms, visible() {:.3} ms, settings hash64 {:.3} ms", set / f, vis / f, hash / f);
}
+136 -3
View File
@@ -9,7 +9,15 @@
//! export of each (decode + render + encode). Prints minimum wall-clock and minimum process CPU
//! time: on a shared machine the CPU time shows the work done, wall-clock also the wait for cores.
//! Each scenario runs on the CPU pipeline and, when a GPU adapter exists, on `lightcraft-gpu`
//! (second column; `LIGHTCRAFT_GPU=0` for the CPU only).
//! (second column; `LIGHTCRAFT_GPU=0` for the CPU only, `GPU_ONLY=1` for the GPU only).
//!
//! `ONLY=window`: a 1:1 zoom window cut from the full-size original (the loupe's `Slot::Region`):
//! the first window of a photo, a pan, and exposure / draft / NR changes with a warm stage cache;
//! `DUMP=file` writes the GPU window's RGBA bytes, for pixel A/B comparisons between builds.
//! Opening a photo, per file: `ONLY=source` (decode steps), `ONLY=load` (what the app's loaders
//! cost: file read, decode, binning, embedded stand-ins, `load_bytes(2560)`, the fs loader),
//! `ONLY=load1` (decode checksum + `load_bytes(2560)`; `LOADONLY=1`: only that, for a profiler),
//! `ONLY=jpeg` (a JPEG decoded to fit 256–2560 px boxes).
use std::sync::Arc;
use std::time::Instant;
@@ -88,6 +96,63 @@ fn main() {
let n: usize = std::env::var("N").ok().and_then(|v| v.parse().ok()).unwrap_or(5);
let only = std::env::var("ONLY").unwrap_or_default();
let run = |name: &str| only.is_empty() || only.split(',').any(|o| name.contains(o));
if only == "jpeg" {
// a JPEG (e.g. an embedded preview pulled out with exiftool) decoded to fit a few boxes
for path in &args {
let bytes = std::fs::read(path).expect("read");
for edge in [256u32, 512, 1600, 2560] {
let o = lightcraft_codecs::DecodeOptions::fit(edge, edge);
let d = lightcraft_codecs::decode(&bytes, o).expect("decode");
println!(
" {path} fit {edge}: {}×{} {}",
d.image.width,
d.image.height,
best(n, || drop(lightcraft_codecs::decode(&bytes, o).expect("decode")))
);
}
}
return;
}
if only == "load" || only == "load1" {
// opening a photo in the loupe, step by step: what the app's loaders cost (per file)
for path in &args {
let bytes = std::fs::read(path).expect("read");
println!("{path}");
if std::env::var("LOADONLY").is_ok() {
// only the preview load (for a profiler)
println!(" {:<36} {}", "load_bytes(2560):", best(n, || drop(lightcraft_engine::files::load_bytes(&bytes, 2560).expect("load"))));
continue;
}
println!(" {:<36} {}", "file read:", best(n, || drop(std::fs::read(path).expect("read"))));
println!(" {:<36} {}", "raw decode:", best(n, || drop(lightcraft_raw::decode(&bytes).expect("decode"))));
let mut raw = lightcraft_raw::decode(&bytes).expect("decode");
if let lightcraft_raw::RawData::U16(d) = &raw.data {
let sum = d.iter().enumerate().fold(0u64, |a, (i, &v)| a.wrapping_mul(31).wrapping_add(v as u64 ^ i as u64));
println!(" decode checksum: {sum:016x}");
}
if only == "load1" {
println!(" {:<36} {}", "load_bytes(2560):", best(n, || drop(lightcraft_engine::files::load_bytes(&bytes, 2560).expect("load"))));
continue;
}
raw.opcodes.list3.retain(|op| !op.is_lens_correction());
if let Some(k) = lightcraft_engine::files::bin_factor(&raw, 2560) {
println!(" {:<36} {}", format!("binned ×{k} masked:"), best(n, || drop(raw.develop_binned_masked(k, 0.99))));
}
for edge in [2560usize, 1600] {
println!(
" {:<36} {}",
format!("embedded srgb, whole file ({edge}):"),
best(n, || drop(lightcraft_engine::files::embedded_preview_srgb(&bytes, edge)))
);
let quick = lightcraft_engine::files::fs_preview_loader();
println!(" {:<36} {}", format!("fs_preview_loader ({edge}):"), best(n, || drop(quick(path, edge))));
}
println!(" {:<36} {}", "load_bytes(2560):", best(n, || drop(lightcraft_engine::files::load_bytes(&bytes, 2560).expect("load"))));
let (loader, _) = lightcraft_engine::files::fs_hooks();
println!(" {:<36} {}", "fs loader (2560, read+load):", best(n, || drop(loader(path, 2560).expect("load"))));
}
return;
}
if only == "source" {
// opening a photo: where the time goes before the first sharp loupe (per file)
for path in &args {
@@ -152,10 +217,16 @@ fn main() {
Some(a) => println!("gpu: {a} (device + kernels: {:.0} ms)", t.elapsed().as_secs_f64() * 1e3),
None => println!("gpu: none"),
}
let backends: &[bool] = if gpu { &[false, true] } else { &[false] };
// `GPU_ONLY=1`: skip the CPU column (profiling the GPU path)
let backends: &[bool] = match (gpu, std::env::var_os("GPU_ONLY").is_some()) {
(true, true) => &[true],
(true, false) => &[false, true],
_ => &[false],
};
let row = |name: &str, f: &mut dyn FnMut(bool)| {
let cols: Vec<String> = backends.iter().map(|&g| format!("{}", best(n, || f(g)))).collect();
println!("{name:<38} cpu: {}{}", cols[0], cols.get(1).map(|g| format!(" | gpu: {g}")).unwrap_or_default());
let first = if backends[0] { "gpu" } else { "cpu" };
println!("{name:<38} {first}: {}{}", cols[0], cols.get(1).map(|g| format!(" | gpu: {g}")).unwrap_or_default());
};
let preview = Arc::new(fit(&full, 2560, 2560, Filter::Box));
let s = typical();
@@ -214,6 +285,68 @@ fn main() {
drop(develop(&preview, &info, &t, &view, Some(&caches[g as usize]), g));
});
}
if run("window") {
// a 1:1 zoom window (the loupe's `Slot::Region`): visible 1920×1200 + margin, snapped to the
// 256 px grid, cut from the full-resolution original (`ui-egui` `region::window_for`)
use lightcraft_pipeline::PixelWindow;
let (fw, fh) = (full.width, full.height);
let win = |cx: usize, cy: usize| {
let snap = |v: usize| v / 256 * 256;
let (x, y) = (snap(cx.saturating_sub(960 + 512)), snap(cy.saturating_sub(600 + 512)));
PixelWindow { x, y, w: 3072.min(fw - x), h: 2304.min(fh - y) }
};
let at = |w: PixelWindow, q: Quality| RenderRequest { window: Some(w), quality: q, ..RenderRequest::fit(fw, fh) };
let w0 = win(fw / 2, fh / 2);
println!("window {}×{} at ({}, {}) of {fw}×{fh}", w0.w, w0.h, w0.x, w0.y);
// first window of a photo: nothing cached (the GPU uploads the original: `shared_source`)
row("window first (upload original):", &mut |g| {
lightcraft_engine::gpu::trim_pool(0);
drop(develop(&full, &info, &s, &at(w0, Quality::Full), Some(&StageCache::default()), g))
});
let wc = [StageCache::default(), StageCache::default()];
for g in backends {
drop(develop(&full, &info, &s, &at(w0, Quality::Full), Some(&wc[*g as usize]), *g));
}
let mut step = 0usize;
row("window pan (new window, warm src):", &mut |g| {
step += 1;
let w = win(fw / 2 + 512 * (step % 4 + 1), fh / 2);
drop(develop(&full, &info, &s, &at(w, Quality::Full), Some(&wc[g as usize]), g))
});
for g in backends {
drop(develop(&full, &info, &s, &at(w0, Quality::Full), Some(&wc[*g as usize]), *g));
}
row("window exposure change (warm):", &mut |g| {
let mut t = s.clone();
k += 1.0;
t.light.exposure = 0.3 + k * 0.01;
drop(develop(&full, &info, &t, &at(w0, Quality::Full), Some(&wc[g as usize]), g));
});
row("window draft exposure drag (warm):", &mut |g| {
let mut t = s.clone();
k += 1.0;
t.light.exposure = 0.3 + k * 0.01;
drop(develop(&full, &info, &t, &at(w0, Quality::Draft), Some(&wc[g as usize]), g));
});
row("window NR change (warm):", &mut |g| {
let mut t = s.clone();
k += 1.0;
t.detail.nr_luminance = 30.0 + k % 50.0;
drop(develop(&full, &info, &t, &at(w0, Quality::Full), Some(&wc[g as usize]), g));
});
// `DUMP=file`: the GPU window's raw RGBA bytes (A/B pixel comparisons between builds)
if let (true, Some(path)) = (gpu, std::env::var_os("DUMP")) {
let g = develop(&full, &info, &s, &at(w0, Quality::Full), Some(&StageCache::default()), true).image;
let bytes: Vec<u8> = g.data.iter().flatten().copied().collect();
std::fs::write(path, bytes).expect("dump");
}
if gpu && std::env::var_os("GPU_ONLY").is_none() {
let c = develop(&full, &info, &s, &at(w0, Quality::Full), None, false).image;
let g = develop(&full, &info, &s, &at(w0, Quality::Full), None, true).image;
let (max, mean) = diff(&c, &g);
println!("gpu vs cpu, window: max {max} LSB, mean {mean:.4} LSB");
}
}
if run("finish") {
// per-pixel stage cost by feature (warm cache: exposure changes only)
let variants: [(&str, fn(&mut DevelopSettings)); 6] = [
@@ -0,0 +1,63 @@
//! What a scan, an import and Synchronize Folder cost before they read the disk, and what
//! naming one photo's sidecar costs, by the size of the library (`N`, default 100,000 photos; the
//! folder looked at holds 500 of them).
//!
//! `N=3500000 cargo run --release -p lightcraft-engine --example scan_setup_bench`
use std::time::Instant;
use lightcraft_catalog::{Op, Photo, Source};
use lightcraft_engine::Session;
fn main() {
let n: u64 = std::env::var("N").ok().and_then(|v| v.parse().ok()).unwrap_or(100_000);
let mut s = Session::new();
let t = Instant::now();
for chunk in (0..n).collect::<Vec<_>>().chunks(10_000) {
for i in chunk {
let id = s.catalog.alloc_photo_id();
let name = format!("IMG_{i:08}.jpg");
let path = format!("/Volumes/Photos/Library/Archive/2019/2019-07 Summer/Trip to the coast/Day {:05}/extra/{name}", i / 500);
let mut p = Photo::new(id, Source::File { path }, &name, "JPEG", 4032, 3024, "2026-09-30T12:00:00");
p.file_size = 3_400_000 + i;
p.captured = Some(format!("2019-07-{:02}T10:{:02}:{:02}", 1 + i % 28, i % 60, (i / 60) % 60));
p.content_hash = Some(format!("{:032x}", i.wrapping_mul(0x9e37_79b9_7f4a_7c15)));
s.catalog.apply(Op::AddPhoto { photo: Box::new(p) }).unwrap();
}
}
eprintln!("{n} photos built in {:.2} s", t.elapsed().as_secs_f64());
let folder = "/Volumes/Photos/Library/Archive/2019/2019-07 Summer/Trip to the coast/Day 00000".to_string();
for run in 0..3 {
let ms = |t: Instant| t.elapsed().as_secs_f64() * 1e3;
let t = Instant::now();
let f = lightcraft_catalog::Filter { library_folder: Some(folder.clone()), ..Default::default() };
let ids = s.catalog.query(&f, &lightcraft_catalog::Sort::default());
let query = ms(t);
let t = Instant::now();
let namings = s.sidecar_namings(&ids);
let naming = ms(t);
let t = Instant::now();
let one = ids.first().map(|id| s.sidecar_naming(*id));
let naming_one = ms(t);
let t = Instant::now();
let (input, paths) = lightcraft_engine::import::ScanInput::new(&mut s, std::slice::from_ref(&folder));
let scan = ms(t);
// (the folder is not on this disk: the scan finds no file, and its cost is what it makes of the library)
let t = Instant::now();
let out = lightcraft_engine::import::scan_with(input, &paths, &Default::default());
let scanned = ms(t);
let t = Instant::now();
let job = lightcraft_engine::import::ImportJob::new(&mut s, Default::default());
let import = ms(t);
let t = Instant::now();
let sync = lightcraft_engine::sync::SyncInput::new(&mut s, &folder, false);
let whole = ms(t);
println!(
"N={n} run {run}: query {query:.0} ms ({} ids) | sidecar_namings {naming:.0} ms | sidecar_naming of one {naming_one:.0} ms ({one:?}) | ScanInput::new {scan:.0} ms + scan_with {scanned:.0} ms ({} found) | ImportJob::new {import:.0} ms (ok {}) | SyncInput::new {whole:.0} ms (ok {})",
ids.len(),
out.candidates.len(),
job.is_ok(),
sync.is_ok()
);
let _ = namings;
}
}
+14 -6
View File
@@ -308,9 +308,12 @@ pub enum Outcome {
Imported(Imported),
/// Photos (or osascript) failed: why, for the user.
Failed(String),
/// Photos wasn't asked: the export that reserved the import was cancelled, wrote no files or
/// stopped early.
Skipped(String),
/// Photos wasn't asked: the export that reserved the import was cancelled (`cancelled`), wrote
/// no files or stopped early. `why` is for people; code decides on `cancelled`.
Skipped {
why: String,
cancelled: bool,
},
}
/// One import: reserved by an export still writing its files, running, or finished.
@@ -402,9 +405,12 @@ impl ImportJob {
v["running"] = json!(false);
v["error"] = json!(e);
}
Some(Outcome::Skipped(why)) => {
Some(Outcome::Skipped { why, cancelled }) => {
v["running"] = json!(false);
v["skipped"] = json!(why);
if cancelled {
v["cancelled"] = json!(true);
}
}
}
v
@@ -660,7 +666,7 @@ impl Reservation {
None
};
if let Some(why) = skipped {
job.finish(Outcome::Skipped(why.into()));
job.finish(Outcome::Skipped { why: why.into(), cancelled });
return Err(job.json());
}
if let Err(e) = check(&paths, &self.album) {
@@ -700,7 +706,7 @@ impl Reservation {
impl Drop for Reservation {
fn drop(&mut self) {
if let Some(job) = self.job.take() {
job.finish(Outcome::Skipped("the export ended before writing its files".into()));
job.finish(Outcome::Skipped { why: "the export ended before writing its files".into(), cancelled: false });
}
}
}
@@ -1009,8 +1015,10 @@ mod tests {
// nothing written, or cancelled: Photos isn't asked, and the slot is free again
let out = imports.reserve(runner.clone(), "").unwrap().run(&[json!({"skipped": "x"})], false);
assert!(out["skipped"].as_str().unwrap().contains("wrote no files"), "{out}");
assert_eq!(out.get("cancelled"), None, "{out}");
let out = imports.reserve(runner.clone(), "").unwrap().run(&exported, true);
assert!(out["skipped"].as_str().unwrap().contains("cancelled"), "{out}");
assert_eq!(out["cancelled"], true, "the app decides on the flag, not on the words: {out}");
assert_eq!(calls.lock().unwrap().len(), 1);
assert!(imports.running().is_none());
let _ = std::fs::remove_dir_all(dir);
+1
View File
@@ -345,6 +345,7 @@ fn proxies(raw: &RawImage, bytes: &[u8], transform: &CameraTransform, size: usiz
let decoded = crate::files::decode_raw_preview(
bytes,
lightcraft_codecs::DecodeOptions { max_size: Some((edge, edge)), max_pixels: FIT_PREVIEW_MAX_PIXELS },
None,
)?;
let mut reference = decoded.to_working();
let (a, crop) = (raw.active_area, raw.crop.clipped(raw.active_area.width, raw.active_area.height));
+22 -13
View File
@@ -26,8 +26,11 @@ fn section_param(p: &Value, c: &str) -> Result<Section> {
serde_json::from_value(v).map_err(|e| bad(c, e.to_string()))
}
fn groups_param(p: &Value) -> Option<Vec<SettingsGroup>> {
p.get("groups").and_then(|g| serde_json::from_value(g.clone()).ok())
fn groups_param(p: &Value, c: &str) -> Result<Option<Vec<SettingsGroup>>> {
match p.get("groups") {
None | Some(Value::Null) => Ok(None),
Some(v) => serde_json::from_value(v.clone()).map(Some).map_err(|e| bad(c, format!("invalid `groups`: {e}"))),
}
}
fn crop_dims(s: &Session, id: PhotoId) -> (f64, f64) {
@@ -497,13 +500,15 @@ pub fn specs() -> Vec<CommandSpec> {
}),
cmd!("develop.curve", "Set Point Curve", [], None, "{channel: master|red|green|blue, points: [[x,y],...] in 0..1}", has_active, |s, p| {
let ch = str_param(p, "channel").unwrap_or("master").to_string();
let pts: Vec<Point> = p
let mut pts: Vec<Point> = p
.get("points")
.and_then(Value::as_array)
.ok_or_else(|| bad("develop.curve", "missing points"))?
.iter()
.filter_map(|q| Some(Point::new(q.get(0)?.as_f64()?.clamp(0.0, 1.0), q.get(1)?.as_f64()?.clamp(0.0, 1.0))))
.collect();
// stored in the order the curve runs, as the editor draws and drags them
pts.sort_by(|a, b| a.x.total_cmp(&b.x));
edit(s, "develop.curve", "Tone Curve", |d| {
match ch.as_str() {
"master" | "rgb" => d.curve.master = pts,
@@ -782,8 +787,9 @@ pub fn specs() -> Vec<CommandSpec> {
"{groups?: [settingsGroup]} (default: all but crop, masks, remove)",
has_active,
|s, p| {
let id = active(s, "develop.copy")?;
if let Some(g) = groups_param(p) {
let c = "develop.copy";
let id = active(s, c)?;
if let Some(g) = groups_param(p, c)? {
s.copy_groups = g;
}
let d = s.develop_of(id).unwrap_or_default();
@@ -802,7 +808,7 @@ pub fn specs() -> Vec<CommandSpec> {
let c = "develop.pastePrevious";
let prev =
s.previous_active.filter(|id| s.catalog.photo(*id).is_some_and(|ph| !ph.deleted)).ok_or_else(|| bad(c, "no previous photo"))?;
let groups = groups_param(p).unwrap_or_else(|| s.copy_groups.clone());
let groups = groups_param(p, c)?.unwrap_or_else(|| s.copy_groups.clone());
let partial = lightcraft_develop::extract_groups(&s.develop_of(prev).unwrap_or_default(), &groups);
let ids: Vec<PhotoId> = s.targets(p).into_iter().filter(|id| *id != prev).collect();
let ops = ids
@@ -823,8 +829,9 @@ pub fn specs() -> Vec<CommandSpec> {
"{ids?, groups?: [settingsGroup] (paste only these of the copied groups)}",
has_clipboard,
|s, p| {
let c = "develop.paste";
let mut clip = s.clipboard.clone().unwrap_or_default();
if let Some(only) = groups_param(p) {
if let Some(only) = groups_param(p, c)? {
// the clipboard holds whole groups: rebuild it with just the chosen (copied) ones
let only: Vec<_> = only.into_iter().filter(|g| s.copy_groups.contains(g)).collect();
let full = lightcraft_develop::apply_partial(&DevelopSettings::default(), &clip, 1.0);
@@ -849,8 +856,9 @@ pub fn specs() -> Vec<CommandSpec> {
"{groups?} — copy the active photo's settings to all selected photos",
has_selection,
|s, p| {
let id = active(s, "develop.sync")?;
let groups = groups_param(p).unwrap_or_else(SettingsGroup::default_copy);
let c = "develop.sync";
let id = active(s, c)?;
let groups = groups_param(p, c)?.unwrap_or_else(SettingsGroup::default_copy);
let d = s.develop_of(id).unwrap_or_default();
let partial = lightcraft_develop::extract_groups(&d, &groups);
let ops = s
@@ -979,10 +987,11 @@ pub fn specs() -> Vec<CommandSpec> {
ok()
}),
cmd!("preset.create", "Create Preset", ["Photo"], None, "{name, group?, groups?: [settingsGroup]}", has_active, |s, p| {
let id = active(s, "preset.create")?;
let name = str_param(p, "name").ok_or_else(|| bad("preset.create", "missing name"))?;
let c = "preset.create";
let id = active(s, c)?;
let name = str_param(p, "name").ok_or_else(|| bad(c, "missing name"))?;
let group = str_param(p, "group").unwrap_or("User Presets");
let groups = groups_param(p).unwrap_or_else(SettingsGroup::default_copy);
let groups = groups_param(p, c)?.unwrap_or_else(SettingsGroup::default_copy);
let d = s.develop_of(id).unwrap_or_default();
let pid = format!("user.{}.{}", s.presets.len(), name.to_lowercase().replace(' ', "-"));
s.presets.push(Preset::from_settings(&pid, name, group, &d, &groups));
@@ -1026,7 +1035,7 @@ pub fn specs() -> Vec<CommandSpec> {
let c = "preset.update";
let id = active(s, c)?;
let current = s.develop_of(id).unwrap_or_default().to_json();
let groups = groups_param(p);
let groups = groups_param(p, c)?;
let pr = user_preset(s, p, c)?;
pr.settings = match groups {
Some(g) => lightcraft_develop::extract_groups(&serde_json::from_value(current).unwrap_or_default(), &g),
+35 -9
View File
@@ -18,7 +18,7 @@ use lightcraft_meta::{Region, RegionKind};
use serde_json::{Value, json};
use super::{CommandSpec, bad, cmd, has_selection};
use crate::{Result, Session};
use crate::{AiModel, EngineError, Result, Session};
const C: &str = "faces.detect";
/// Regions made by this command say so in their description; only those are replaced on a new run.
@@ -34,15 +34,39 @@ const EXISTING_IOU: f64 = 0.3;
/// What to tell someone who has not downloaded the detector yet.
const NOT_INSTALLED: &str = "finding faces needs the YuNet face detector (232 KB, MIT): download it in Settings > Faces";
/// Why [`detector`] has no detector to give.
#[derive(Debug)]
pub(crate) enum NoDetector {
/// Not downloaded yet: a frontend offers the download ([`crate::ErrorKind::ModelMissing`]).
NotInstalled,
/// Nowhere to keep it, or there but unusable: says what to do.
Unusable(String),
}
impl NoDetector {
/// Command `cmd`'s error for it. The message reads as before kinds existed (an invalid-parameters
/// error), so control-channel and MCP clients see the same `error` text.
pub(crate) fn into_error(self, cmd: &str) -> EngineError {
match self {
Self::NotInstalled => EngineError::ModelMissing { model: AiModel::FaceDetector, msg: bad(cmd, NOT_INSTALLED).to_string() },
Self::Unusable(why) => bad(cmd, why),
}
}
}
/// The installed YuNet, loaded once and kept (a new one is loaded when the model folder changes).
/// (also used by the background scan, which aligns faces by the detector's landmarks and finds faces in photos that have none)
pub(crate) fn detector(s: &Session) -> std::result::Result<Arc<Detector>, String> {
pub(crate) fn detector(s: &Session) -> std::result::Result<Arc<Detector>, NoDetector> {
static LOADED: Mutex<Option<(PathBuf, Arc<Detector>)>> = Mutex::new(None);
let home = s.face_models_dir.as_deref().ok_or("this build has nowhere to keep face models (the desktop app does)")?.join(YUNET_ID);
let home = s
.face_models_dir
.as_deref()
.ok_or_else(|| NoDetector::Unusable("this build has nowhere to keep face models (the desktop app does)".into()))?
.join(YUNET_ID);
let path = home.join("model.onnx");
// installing writes the manifest last, so a folder without one is not (yet) a model
if !path.is_file() || !home.join("face-model.json").is_file() {
return Err(NOT_INSTALLED.into());
return Err(NoDetector::NotInstalled);
}
let mut loaded = LOADED.lock().unwrap_or_else(PoisonError::into_inner);
if let Some((p, d)) = loaded.as_ref()
@@ -50,12 +74,14 @@ pub(crate) fn detector(s: &Session) -> std::result::Result<Arc<Detector>, String
{
return Ok(d.clone());
}
let len = std::fs::metadata(&path).map_err(|e| format!("could not read the detector: {e}"))?.len();
let len = std::fs::metadata(&path).map_err(|e| NoDetector::Unusable(format!("could not read the detector: {e}")))?.len();
if len > MAX_MODEL_BYTES {
return Err("the detector's file is far bigger than YuNet: remove it in Settings > Faces and download it again".into());
return Err(NoDetector::Unusable("the detector's file is far bigger than YuNet: remove it in Settings > Faces and download it again".into()));
}
let bytes = std::fs::read(&path).map_err(|e| format!("could not read the detector: {e}"))?;
let d = Arc::new(Detector::new(&bytes).map_err(|e| format!("{e}: remove the detector in Settings > Faces and download it again"))?);
let bytes = std::fs::read(&path).map_err(|e| NoDetector::Unusable(format!("could not read the detector: {e}")))?;
let d = Arc::new(
Detector::new(&bytes).map_err(|e| NoDetector::Unusable(format!("{e}: remove the detector in Settings > Faces and download it again")))?,
);
*loaded = Some((path, d.clone()));
Ok(d)
}
@@ -103,7 +129,7 @@ fn detect(s: &mut Session, p: &Value) -> Result<Value> {
nms_iou: p.get("nmsIou").and_then(Value::as_f64).map_or(defaults.nms_iou, |v| v as f32),
max_faces: p.get("maxFaces").and_then(Value::as_u64).map_or(defaults.max_faces, |v| usize::try_from(v).unwrap_or(defaults.max_faces)),
};
let detector = detector(s).map_err(|e| bad(C, e))?;
let detector = detector(s).map_err(|e| e.into_error(C))?;
let started = Instant::now();
let (mut results, mut ops) = (Vec::new(), Vec::new());
for id in s.targets(p) {
+23 -15
View File
@@ -84,29 +84,29 @@ fn resolve_ai(s: &mut Session, shape: &mut MaskShape, c: &str) -> Result<Later>
match shape {
MaskShape::Object { seg: Some(_), .. } | MaskShape::Prompt { seg: Some(_), .. } => {}
MaskShape::Object { hint, exclude, .. } if !hint.is_empty() && background => {
s.segmenter.model_dir().map_err(ai_err)?;
s.segmenter.require_model()?;
return Ok(Later::Clicks(std::mem::take(hint), std::mem::take(exclude)));
}
MaskShape::Object { hint, exclude, seg, .. } if !hint.is_empty() => {
*seg = Some(s.segment_clicks(id, &clicks_of(hint, exclude)).map_err(ai_err)?);
*seg = Some(segment(s, |s| s.segment_clicks(id, &clicks_of(hint, exclude)))?);
}
MaskShape::Object { .. } => {
// the clicks come later: refuse an Object mask that could never be computed (no
// model) rather than leave an empty one; in the app, analyze the photo meanwhile
s.segmenter.model_dir().map_err(ai_err)?;
s.segmenter.require_model()?;
if background {
s.segment_prepare(id).map_err(ai_err)?;
segment(s, |s| s.segment_prepare(id))?;
}
}
MaskShape::Prompt { text, .. } if background => {
s.segmenter.model_dir().map_err(ai_err)?;
s.segmenter.require_model()?;
if text.trim().is_empty() {
return Err(ai_err("describe what to select"));
}
return Ok(Later::Text(text.clone()));
}
MaskShape::Prompt { text, seg, .. } => {
let found = s.segment_text(id, text).map_err(ai_err)?;
let found = segment(s, |s| s.segment_text(id, text))?;
*seg = Some(found.ok_or_else(|| ai_err(format!("nothing matching “{text}” was found in this photo")))?);
}
_ => {}
@@ -118,7 +118,7 @@ fn resolve_ai(s: &mut Session, shape: &mut MaskShape, c: &str) -> Result<Later>
fn queue_later(s: &mut Session, later: Later, mask: u32, comp: usize, mut out: Value) -> Result<Value> {
if let Later::Clicks(hint, exclude) = later {
let id = s.active().ok_or_else(|| ai_err("no active photo"))?;
s.segment_clicks_later(id, mask, comp, hint, exclude).map_err(ai_err)?;
segment(s, |s| s.segment_clicks_later(id, mask, comp, hint, exclude))?;
out["pending"] = json!(true);
}
Ok(out)
@@ -129,6 +129,13 @@ fn ai_err(msg: impl Into<String>) -> crate::EngineError {
crate::EngineError::Other(msg.into())
}
/// A segmentation call: the model is checked first, so a missing one is [`crate::EngineError::ModelMissing`]
/// (the segmenter's own errors are text).
fn segment<T>(s: &mut Session, f: impl FnOnce(&mut Session) -> std::result::Result<T, String>) -> Result<T> {
s.segmenter.require_model()?;
f(s).map_err(ai_err)
}
pub(crate) fn clicks_of(include: &[Point], exclude: &[Point]) -> Vec<Click> {
include.iter().map(|p| Click { at: *p, include: true }).chain(exclude.iter().map(|p| Click { at: *p, include: false })).collect()
}
@@ -167,7 +174,7 @@ pub fn specs() -> Vec<CommandSpec> {
let name = str_param(p, "name").map(str::to_string);
if let Later::Text(text) = &later {
let id = s.active().ok_or_else(|| bad("mask.add", "no active photo"))?;
s.segment_text_later(id, None, "add", name, text).map_err(ai_err)?;
segment(s, |s| s.segment_text_later(id, None, "add", name, text))?;
return Ok(json!({"pending": true}));
}
let next = s.active().and_then(|id| s.develop_of(id)).map(|d| d.next_mask_id()).unwrap_or(1);
@@ -205,7 +212,7 @@ pub fn specs() -> Vec<CommandSpec> {
let later = resolve_ai(s, &mut shape, "mask.addComponent")?;
if let Later::Text(text) = &later {
let op = p.get("op").and_then(Value::as_str).unwrap_or("add");
s.segment_text_later(id, Some(mid), op, None, text).map_err(ai_err)?;
segment(s, |s| s.segment_text_later(id, Some(mid), op, None, text))?;
return Ok(json!({"pending": true}));
}
let mut comp = 0;
@@ -365,10 +372,10 @@ pub fn specs() -> Vec<CommandSpec> {
}
if s.segmenter.background && !hint.is_empty() {
let out = json!({"include": hint.len(), "exclude": ex.len(), "pending": true});
s.segment_clicks_later(id, mid, k, hint, ex).map_err(ai_err)?;
segment(s, |s| s.segment_clicks_later(id, mid, k, hint, ex))?;
return Ok(out);
}
let seg = if hint.is_empty() { None } else { Some(s.segment_clicks(id, &clicks_of(&hint, &ex)).map_err(ai_err)?) };
let seg = if hint.is_empty() { None } else { Some(segment(s, |s| s.segment_clicks(id, &clicks_of(&hint, &ex)))?) };
let out = json!({"include": hint.len(), "exclude": ex.len()});
masks_edit(s, c, "Object Mask", |masks, _| {
let i = find(masks, mid, c)?;
@@ -401,7 +408,7 @@ pub fn specs() -> Vec<CommandSpec> {
.rposition(|x| matches!(x.shape, MaskShape::Object { .. } | MaskShape::Prompt { .. }))
.ok_or_else(|| bad(c, "the mask has no Object or Describe selection"))?,
};
let started = s.segment_detail(id, mid, k).map_err(ai_err)?;
let started = segment(s, |s| s.segment_detail(id, mid, k))?;
Ok(json!({"started": started}))
}
),
@@ -415,7 +422,7 @@ pub fn specs() -> Vec<CommandSpec> {
|s, _| {
let c = "segment.prepare";
let id = s.active().ok_or_else(|| bad(c, "no active photo"))?;
s.segment_prepare(id).map_err(ai_err)?;
segment(s, |s| s.segment_prepare(id))?;
Ok(json!({"busy": s.segmenter.busy()}))
}
),
@@ -639,7 +646,8 @@ pub fn specs() -> Vec<CommandSpec> {
let i = find(&mut masks, mid, "mask.move")?;
let to = match (p.get("to").and_then(Value::as_i64), p.get("delta").and_then(Value::as_i64)) {
(Some(t), _) => t,
(None, Some(d)) => i as i64 + d,
// (an agent's delta can be anything: saturate rather than overflow)
(None, Some(d)) => (i as i64).saturating_add(d),
(None, None) => return Err(bad("mask.move", "give `to` (index) or `delta`")),
}
.clamp(0, masks.len() as i64 - 1) as usize;
@@ -700,7 +708,7 @@ pub fn specs() -> Vec<CommandSpec> {
|s, p| {
let c = "spot.findBlemishes";
let id = s.active().ok_or_else(|| bad(c, "no active photo"))?;
let detector = super::face_detect::detector(s).map_err(|e| bad(c, e))?;
let detector = super::face_detect::detector(s).map_err(|e| e.into_error(c))?;
// the uncropped photo, so positions are the spots' own coordinates
let job = s.render_job(id, 1600, 1600, false, false).ok_or_else(|| bad(c, "no photo"))?;
let img = job.run().rendered.map_err(|e| bad(c, e))?.image;
+13 -3
View File
@@ -31,14 +31,24 @@ pub fn photo_summary(p: &Photo) -> Value {
})
}
fn album_json(a: &Album, all: &[Album], cat: &lightcraft_catalog::Catalog) -> Value {
/// How deep [`album_json`] lists folders inside folders: a catalog that loops (an album inside
/// itself, which [`lightcraft_catalog::Catalog::repair`] puts right on load) lists no further
/// instead of overflowing the stack.
const MAX_ALBUM_DEPTH: usize = 64;
fn album_json(a: &Album, all: &[Album], cat: &lightcraft_catalog::Catalog, depth: usize) -> Value {
let children: Vec<Value> = if depth < MAX_ALBUM_DEPTH {
all.iter().filter(|c| c.parent == Some(a.id)).map(|c| album_json(c, all, cat, depth + 1)).collect()
} else {
Vec::new()
};
let mut v = json!({
"id": a.id.0,
"name": a.name,
"folder": a.folder,
"count": cat.album_count(a.id),
"cover": a.cover.map(|c| c.0),
"children": all.iter().filter(|c| c.parent == Some(a.id)).map(|c| album_json(c, all, cat)).collect::<Vec<_>>(),
"children": children,
});
if let Some(rules) = &a.smart {
v["smart"] = json!(true);
@@ -139,7 +149,7 @@ pub fn specs() -> Vec<CommandSpec> {
}),
cmd!(query "albums.list", "List Albums", [], None, "{}", always, |s, _| {
let all: Vec<Album> = s.catalog.albums().cloned().collect();
Ok(Value::Array(all.iter().filter(|a| a.parent.is_none()).map(|a| album_json(a, &all, &s.catalog)).collect()))
Ok(Value::Array(all.iter().filter(|a| a.parent.is_none()).map(|a| album_json(a, &all, &s.catalog, 0)).collect()))
}),
cmd!(query "photo.inspect", "Inspect Photo", [], None, "{id?}", always, |s, p| {
let id = photo_arg(s, p, "photo.inspect")?;
+3 -2
View File
@@ -12,8 +12,9 @@ fn save(s: &mut Session, p: &Value) -> Result<Value> {
let mut merged = Vec::new();
let mut backups = Vec::new();
let mut failed = Vec::new();
let owners = crate::sidecar::StemOwners::of(&s.catalog);
for id in s.targets(p) {
let ids = s.targets(p);
let owners = crate::sidecar::StemOwners::among(&s.catalog, &ids);
for id in ids {
match s.save_sidecar_with(id, &owners) {
Ok(r) => {
let path = r.path.display().to_string();
+129 -16
View File
@@ -1,16 +1,31 @@
//! Where LightCraft keeps per-user data on this machine: the config folder (app settings, camera
//! profiles, face and denoise models).
//!
//! [`config_dir`] picks it, in order:
//! 1. `$LIGHTCRAFT_CONFIG_DIR`, when set (tests, agents, custom setups).
//! 2. **Portable mode** (issue #1200, as in PhotoCraft): a marker file ([`PORTABLE_MARKERS`]) beside the
//! executable puts everything in `<exe dir>/LightCraftData`, so a copy on a USB stick leaves nothing in
//! `%APPDATA%`. If that folder can't be written (read-only medium, Program Files), a warning is logged
//! and the platform folder is used.
//! 3. The platform convention: macOS `~/Library/Application Support/LightCraft`, Windows
//! `%APPDATA%\LightCraft`, Linux `$XDG_CONFIG_HOME/lightcraft` or `~/.config/lightcraft`.
//!
//! Hosts that have a file system (the desktop app, the CLI, the MCP server) share these defaults so a model
//! installed from one is there for the others. Nothing here is applied automatically: a [`Session`] has no
//! face-, denoise- or SAM 3 model folder until a host asks for one ([`Session::with_default_face_models`],
//! [`Session::with_default_denoise_models`], [`Session::with_default_sam3_model`]), so tests stay hermetic.
use std::ffi::OsString;
use std::path::PathBuf;
use std::path::{Path, PathBuf};
use std::sync::OnceLock;
use crate::Session;
/// Files beside the executable that switch on portable mode (either one; contents are ignored).
pub const PORTABLE_MARKERS: [&str; 2] = ["portable.txt", "LightCraft.portable"];
/// The data folder created beside the executable in portable mode.
pub const PORTABLE_DATA_DIR: &str = "LightCraftData";
#[derive(Clone, Copy)]
enum Platform {
Mac,
@@ -18,9 +33,9 @@ enum Platform {
Other,
}
/// LightCraft's settings folder. A nonempty `$LIGHTCRAFT_CONFIG_DIR` overrides the platform default.
/// The override is the complete directory; no application-name suffix is appended. An explicitly
/// empty value disables the settings directory instead of falling back to a personal profile.
/// LightCraft's settings folder. A nonempty `$LIGHTCRAFT_CONFIG_DIR` overrides portable mode and the
/// platform default. The override is the complete directory; no application-name suffix is appended. An
/// explicitly empty value disables the settings directory instead of falling back to a personal profile.
pub fn config_dir() -> Option<PathBuf> {
let platform = if cfg!(target_os = "macos") {
Platform::Mac
@@ -29,13 +44,17 @@ pub fn config_dir() -> Option<PathBuf> {
} else {
Platform::Other
};
config_dir_with(platform, |name| std::env::var_os(name))
config_dir_with(platform, |name| std::env::var_os(name), portable_dir)
}
fn config_dir_with(platform: Platform, env: impl Fn(&str) -> Option<OsString>) -> Option<PathBuf> {
/// `portable` is asked only without an override, so an isolated run never probes beside the executable.
fn config_dir_with(platform: Platform, env: impl Fn(&str) -> Option<OsString>, portable: impl FnOnce() -> Option<PathBuf>) -> Option<PathBuf> {
if let Some(dir) = env("LIGHTCRAFT_CONFIG_DIR") {
return (!dir.is_empty()).then(|| PathBuf::from(dir));
}
if let Some(dir) = portable() {
return Some(dir);
}
match platform {
Platform::Mac => env("HOME").map(|h| PathBuf::from(h).join("Library/Application Support/LightCraft")),
Platform::Windows => env("APPDATA").map(|a| PathBuf::from(a).join("LightCraft")),
@@ -45,6 +64,50 @@ fn config_dir_with(platform: Platform, env: impl Fn(&str) -> Option<OsString>) -
}
}
/// This process's portable data folder, if a marker sits beside its executable and the folder can be
/// written. Resolved once: the check creates the folder and writes a probe file.
fn portable_dir() -> Option<PathBuf> {
static DIR: OnceLock<Option<PathBuf>> = OnceLock::new();
DIR.get_or_init(|| {
let exe = std::env::current_exe().ok()?;
match portable_dir_in(exe.parent()?) {
Ok(dir) => dir,
Err(why) => {
log::warn!("{why}");
None
}
}
})
.clone()
}
/// `<exe_dir>/LightCraftData` when `exe_dir` holds a portable marker; `Err` says why a marked folder
/// can't be used (the caller falls back to the platform folder).
fn portable_dir_in(exe_dir: &Path) -> Result<Option<PathBuf>, String> {
let Some(marker) = PORTABLE_MARKERS.iter().map(|m| exe_dir.join(m)).find(|p| p.is_file()) else {
return Ok(None);
};
let data = exe_dir.join(PORTABLE_DATA_DIR);
match ensure_writable(&data) {
Ok(()) => Ok(Some(data)),
Err(e) => Err(format!(
"portable mode ({} found) but {} isn't writable ({e}); settings are kept in the usual folder instead",
marker.display(),
data.display()
)),
}
}
/// Create `dir` and prove a file can be written in it (a folder can exist yet be read-only).
fn ensure_writable(dir: &Path) -> std::io::Result<()> {
std::fs::create_dir_all(dir)?;
let probe = dir.join(format!(".write-test-{}", std::process::id()));
std::fs::write(&probe, b"")?;
// A leftover probe is harmless; don't fail portable mode over it.
let _ = std::fs::remove_file(&probe);
Ok(())
}
/// Where face models are kept: `$LIGHTCRAFT_FACE_MODELS` if set, else `<config>/models`.
pub fn default_face_models_dir() -> Option<PathBuf> {
std::env::var_os("LIGHTCRAFT_FACE_MODELS").filter(|v| !v.is_empty()).map(PathBuf::from).or_else(|| config_dir().map(|d| d.join("models")))
@@ -101,16 +164,62 @@ mod config_tests {
use super::*;
fn resolve(platform: Platform, vars: &[(&str, &str)]) -> Option<PathBuf> {
config_dir_with(platform, |name| vars.iter().find_map(|(key, value)| (*key == name).then(|| OsString::from(value))))
config_dir_with(platform, |name| vars.iter().find_map(|(key, value)| (*key == name).then(|| OsString::from(value))), || None)
}
/// An override is used as is: portable mode is not even looked for.
fn no_portable_probe() -> Option<PathBuf> {
panic!("an override must not probe for portable mode")
}
fn temp(tag: &str) -> PathBuf {
let d = std::env::temp_dir().join(format!("lc-config-{tag}-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&d);
std::fs::create_dir_all(&d).unwrap();
d
}
/// A marker beside the executable moves the settings folder next to it, ahead of the platform folder (issue #1200).
#[test]
fn portable_marker_puts_settings_beside_the_executable() {
for marker in PORTABLE_MARKERS {
let exe = temp(&format!("portable-{marker}"));
assert_eq!(portable_dir_in(&exe), Ok(None), "no marker, no portable folder");
assert!(!exe.join(PORTABLE_DATA_DIR).exists());
std::fs::write(exe.join(marker), b"").unwrap();
let data = exe.join(PORTABLE_DATA_DIR);
assert_eq!(portable_dir_in(&exe), Ok(Some(data.clone())));
assert!(data.is_dir());
assert_eq!(std::fs::read_dir(&data).unwrap().count(), 0, "the write probe is removed");
let found = portable_dir_in(&exe).unwrap();
let actual = config_dir_with(Platform::Windows, |name| (name == "APPDATA").then(|| OsString::from("/roaming")), || found);
assert_eq!(actual, Some(data));
let _ = std::fs::remove_dir_all(&exe);
}
}
/// A marked folder that can't hold the data folder (here: a file is in its place) is reported, not used.
#[test]
fn portable_folder_that_cannot_be_written_is_refused() {
let exe = temp("portable-blocked");
std::fs::write(exe.join("portable.txt"), b"").unwrap();
std::fs::write(exe.join(PORTABLE_DATA_DIR), b"").unwrap();
let why = portable_dir_in(&exe).unwrap_err();
assert!(why.contains("portable.txt") && why.contains(PORTABLE_DATA_DIR), "{why}");
let _ = std::fs::remove_dir_all(&exe);
}
#[test]
fn config_override_wins_without_reading_user_profile() {
for platform in [Platform::Mac, Platform::Windows, Platform::Other] {
let actual = config_dir_with(platform, |name| {
assert_eq!(name, "LIGHTCRAFT_CONFIG_DIR");
Some(OsString::from("isolated/settings"))
});
let actual = config_dir_with(
platform,
|name| {
assert_eq!(name, "LIGHTCRAFT_CONFIG_DIR");
Some(OsString::from("isolated/settings"))
},
no_portable_probe,
);
assert_eq!(actual, Some(PathBuf::from("isolated/settings")));
}
}
@@ -119,10 +228,14 @@ mod config_tests {
fn config_empty_override_disables_profile_without_reading_defaults() {
for platform in [Platform::Mac, Platform::Windows, Platform::Other] {
assert_eq!(
config_dir_with(platform, |name| {
assert_eq!(name, "LIGHTCRAFT_CONFIG_DIR");
Some(OsString::new())
}),
config_dir_with(
platform,
|name| {
assert_eq!(name, "LIGHTCRAFT_CONFIG_DIR");
Some(OsString::new())
},
no_portable_probe
),
None
);
}
@@ -145,7 +258,7 @@ mod config_tests {
fn config_override_preserves_non_unicode_paths() {
use std::os::unix::ffi::OsStringExt as _;
let value = OsString::from_vec(b"isolated/\xff".to_vec());
assert_eq!(config_dir_with(Platform::Mac, |_| Some(value.clone())), Some(PathBuf::from(value)));
assert_eq!(config_dir_with(Platform::Mac, |_| Some(value.clone()), no_portable_probe), Some(PathBuf::from(value)));
}
}
+15 -1
View File
@@ -173,7 +173,8 @@ fn curve(props: &Props, k: &str) -> Option<Value> {
let mut pts = Vec::new();
for it in items {
let (x, y) = it.split_once(',')?;
let (x, y) = (x.trim().parse::<f64>().ok()?, y.trim().parse::<f64>().ok()?);
// (`parse` reads "NaN" and "inf" too: a curve with such a point is not a curve)
let (x, y) = (x.trim().parse::<f64>().ok().filter(|v| v.is_finite())?, y.trim().parse::<f64>().ok().filter(|v| v.is_finite())?);
pts.push(json!({"x": (x / 255.0).clamp(0.0, 1.0), "y": (y / 255.0).clamp(0.0, 1.0)}));
}
if pts.len() < 2 {
@@ -617,6 +618,19 @@ mod tests {
assert_eq!((c.rect.x0, c.rect.y0, c.rect.x1, c.rect.y1, c.angle), (0.05, 0.1, 0.8, 0.9, 1.5));
}
/// A sidecar curve with a `NaN` or `inf` point (Rust's `parse` reads both) is not a curve:
/// it is left out, rather than stored with a point the curve editor can't place.
#[test]
fn curves_with_non_finite_points_are_left_out() {
for bad in ["NaN, 0", "64, inf", "-inf, 3"] {
let x = SIDECAR.replace("<rdf:li>64, 52</rdf:li>", &format!("<rdf:li>{bad}</rdf:li>"));
let partial = to_partial(&props(&x), Target::RawAbsolute);
let s = apply_partial(&DevelopSettings::for_raw(5000.0, 0.0), &partial, 1.0);
assert!(s.curve.master.is_empty(), "{bad}: {:?}", s.curve.master);
assert!(s.curve.master.iter().all(|p| p.x.is_finite() && p.y.is_finite()));
}
}
#[test]
fn only_present_fields_are_in_the_partial() {
let x = r#"<rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"><rdf:Description
+68
View File
@@ -0,0 +1,68 @@
//! What kind of error a command returned, for code that decides on it (#1046).
//!
//! A frontend that reacts to an error (offers a model download, says "cancelled" in its own words
//! instead of showing a failure) matches [`EngineError::kind`], never the message: messages are
//! for people, get reworded, and will be translated.
use crate::EngineError;
/// A model a feature needs and LightCraft downloads only when the person agrees.
#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)]
pub enum AiModel {
/// YuNet, which finds faces (`faces.detect`; installed in Settings ▸ Faces).
FaceDetector,
/// SAM 3, behind Object and Describe masks (`segment.model.download`).
Sam3,
}
/// The kind of an [`EngineError`] (or of an error a frontend raised itself: [`ErrorKind::Other`]).
#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)]
pub enum ErrorKind {
UnknownCommand,
Disabled,
BadParams,
Catalog,
NotSaved,
LibraryInUse,
LightroomCatalog,
/// The work was cancelled: not a failure.
Cancelled,
/// The feature needs this model, which is not installed (or still downloading).
ModelMissing(AiModel),
Other,
}
impl EngineError {
/// What kind of error this is: decide on this, not on the message.
pub fn kind(&self) -> ErrorKind {
match self {
Self::UnknownCommand(_) => ErrorKind::UnknownCommand,
Self::Disabled(..) => ErrorKind::Disabled,
Self::BadParams { .. } => ErrorKind::BadParams,
Self::Catalog(_) => ErrorKind::Catalog,
Self::NotSaved(_) => ErrorKind::NotSaved,
Self::LibraryInUse(_) => ErrorKind::LibraryInUse,
Self::LightroomCatalog { .. } => ErrorKind::LightroomCatalog,
Self::Cancelled(_) => ErrorKind::Cancelled,
Self::ModelMissing { model, .. } => ErrorKind::ModelMissing(*model),
Self::Other(_) => ErrorKind::Other,
}
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn the_kind_does_not_depend_on_the_message() {
let cancelled = EngineError::Cancelled("Lightroom-Import abgebrochen".into());
assert_eq!(cancelled.kind(), ErrorKind::Cancelled);
assert_eq!(cancelled.to_string(), "Lightroom-Import abgebrochen", "the message is shown as is");
let missing = EngineError::ModelMissing { model: AiModel::Sam3, msg: "any wording".into() };
assert_eq!(missing.kind(), ErrorKind::ModelMissing(AiModel::Sam3));
assert_eq!(missing.to_string(), "any wording");
// an `Other` whose text reads like a cancellation or a missing model is still `Other`
assert_eq!(EngineError::Other("The SAM 3 model is not installed; cancelled".into()).kind(), ErrorKind::Other);
}
}
+98 -3
View File
@@ -108,6 +108,10 @@ impl Default for Resize {
}
}
/// The most pixels an export enlarged past its photo's own size can have: 512 megapixels, also
/// Lightroom's ceiling. A full-size or smaller export is never held to it.
pub const MAX_ENLARGED_PIXELS: f64 = 512e6;
impl Resize {
pub fn long_edge(px: u32) -> Self {
Self { value: px as f32, ..Default::default() }
@@ -131,8 +135,12 @@ impl Resize {
ResizeMode::Percent => v / 100.0,
};
let k = if self.dont_enlarge { k.min(1.0) } else { k };
// at least one pixel, at most the encoders' 65535 limit
k.clamp(1.0 / short, 65_535.0 / long)
// an enlargement stops at 512 megapixels (Lightroom's ceiling too): 20000 typed for 2000
// with Don't Enlarge off asked for gigapixels, more memory than a computer has
let k = k.min((MAX_ENLARGED_PIXELS / (w * h)).sqrt().max(1.0));
// at least one pixel, at most the encoders' 65535 limit; the limit wins for a photo more
// than 65535 times as long as it is short (where `clamp` would panic: min above max)
k.max(1.0 / short).min(65_535.0 / long)
}
/// Output size for a `w × h` (cropped, full-resolution) photo.
@@ -1670,10 +1678,16 @@ fn prepare_guarded(
let (full_px, long) = ((p.width as usize).saturating_mul(p.height as usize), p.width.max(p.height) as usize);
let (work, weight) = if o.format.is_rendered() {
let (w, h) = output_size(p, o);
let (nw, nh) = lightcraft_pipeline::native_output_size(p.width.max(1) as usize, p.height.max(1) as usize, &p.develop);
let native_px = (nw.round().max(1.0) * nh.round().max(1.0)) as usize;
let label = p.file_name.clone();
let meta = export_metadata(p, &session.catalog, o);
let gain_map = o.hdr_output() && o.format == ExportFormat::Jpeg;
let job = session.export_job(id, w, h, o.effective_space(), o.effective_depth())?;
let weight = render_weight(full_px, long, job.level, w.saturating_mul(h));
if let Some(why) = too_big_for_memory((w, h), native_px, weight, crate::memory::total_ram()) {
return Err(format!("{label}: {why}"));
}
let hdr_job = if gain_map { Some(session.export_job(id, w, h, o.effective_space(), OutputDepth::F32Hdr)?) } else { None };
(Work::Render(Box::new(RenderWork { job, hdr_job, meta, opts: o.clone() })), weight)
} else {
@@ -1697,6 +1711,26 @@ fn prepare_guarded(
Ok(PreparedExport { photo: id, file_name, work, guard, weight })
}
/// Why an export of `out` pixels (`w`, `h`), from a photo of `native_px` pixels at its own size,
/// can't be made on a computer with `ram` bytes, given its working memory `weight`: an export
/// enlarged past the photo's size that needs more memory than the computer has would fail to
/// allocate, which ends the process rather than the export. Full-size and smaller exports, and
/// computers whose memory isn't known, are never refused.
fn too_big_for_memory(out: (usize, usize), native_px: usize, weight: usize, ram: Option<usize>) -> Option<String> {
let ram = ram?;
if out.0.saturating_mul(out.1) <= native_px || weight <= ram {
return None;
}
let gb = |b: usize| b as f64 / (1u64 << 30) as f64;
Some(format!(
"exporting at {} × {} needs about {:.0} GB of memory, more than this computer has ({:.0} GB); choose a smaller size",
out.0,
out.1,
gb(weight).ceil(),
gb(ram)
))
}
/// Working memory of rendering a photo of `full_px` pixels (long edge `long`) from source level
/// `level` into `out_px` output pixels: the decoded source (linear RGB f32, 12 B/px) plus the
/// pipeline's planes and the encoded output (~36 B per output pixel). Measured: a 24 MP export
@@ -2738,10 +2772,44 @@ mod tests {
assert_eq!(ExportOptions::from_json(&ExportOptions::resize_json(None)).resize, None);
assert_eq!(ExportOptions::from_json(&json!({"ppi": 300})).ppi, 300);
// never larger than the encoders' limit, never empty
assert_eq!(size(json!({"percent": 10000, "dontEnlarge": false}), 6000.0, 4000.0), Some((65535, 43690)));
assert_eq!(size(json!({"percent": 10000, "dontEnlarge": false}), 6000.0, 4000.0), Some((27713, 18475)), "512 MP at most");
assert_eq!(size(json!({"width": 70000, "dontEnlarge": false}), 60000.0, 100.0), Some((65535, 109)), "65535 px an edge");
assert_eq!(size(json!({"width": 1}), 6000.0, 40.0).map(|s| s.1), Some(1));
}
/// Typing 20000 for 2000 (or percent 10000) with Don't Enlarge off asked for gigapixels: the
/// render's first plane alone (34 GB at 65535 × 43690) failed to allocate, which aborts the
/// process. An enlargement stops at 512 megapixels; full-size and smaller exports of photos
/// larger than that are untouched.
#[test]
fn enlargements_stop_at_512_megapixels() {
let size = |p: serde_json::Value, w: f64, h: f64| ExportOptions::from_json(&p).resize.map(|r| r.apply(w, h)).unwrap();
for p in [json!({"longEdge": 60000, "dontEnlarge": false}), json!({"megapixels": 5000}), json!({"percent": 10000, "dontEnlarge": false})] {
let (w, h) = size(p.clone(), 6000.0, 4000.0);
assert!((w * h) as f64 <= MAX_ENLARGED_PIXELS * 1.001, "{p}: {w}×{h}");
assert!((w as f64 / h as f64 - 1.5).abs() < 0.001, "{p}: keeps the shape");
}
// below the ceiling an enlargement is as asked
assert_eq!(size(json!({"longEdge": 12000, "dontEnlarge": false}), 6000.0, 4000.0), (12000, 8000));
// a 600 MP panorama exports at full size, and smaller, as before; it is never enlarged
assert_eq!(size(json!({"percent": 100, "dontEnlarge": false}), 30000.0, 20000.0), (30000, 20000));
assert_eq!(size(json!({"percent": 50}), 30000.0, 20000.0), (15000, 10000));
assert_eq!(size(json!({"percent": 200, "dontEnlarge": false}), 30000.0, 20000.0), (30000, 20000));
}
/// An enlarged export that needs more memory than the computer has is refused with a message,
/// before it starts; everything else goes ahead.
#[test]
fn enlarged_exports_must_fit_the_computers_memory() {
let gb = 1usize << 30;
let why = too_big_for_memory((27713, 18475), 24_000_000, 20 * gb, Some(16 * gb)).unwrap();
assert!(why.contains("27713 × 18475") && why.contains("20 GB") && why.contains("16 GB") && why.contains("smaller size"), "{why}");
assert_eq!(too_big_for_memory((27713, 18475), 24_000_000, 20 * gb, Some(32 * gb)), None, "fits");
assert_eq!(too_big_for_memory((6000, 4000), 24_000_000, 20 * gb, Some(gb)), None, "full size is never refused");
assert_eq!(too_big_for_memory((3000, 2000), 24_000_000, 20 * gb, Some(gb)), None, "nor a smaller export");
assert_eq!(too_big_for_memory((27713, 18475), 24_000_000, 20 * gb, None), None, "unknown memory");
}
#[test]
fn resize_size_survives_the_render() {
// the render fits the photo into the size `apply` gives; that must not cost a pixel (#1312)
@@ -2783,6 +2851,33 @@ mod tests {
}
}
/// A photo more than 65535 times as long as it is short (a 70000 × 1 strip) made `scale`
/// clamp with its lower bound above its upper one, which panics, in the export and in the
/// Export dialog's preview on the UI thread. The encoders' limit wins, the short edge keeps a
/// pixel.
#[test]
fn resize_survives_photos_too_long_for_the_encoders() {
let modes = [
json!({"longEdge": 2000}),
json!({"shortEdge": 1}),
json!({"width": 500, "height": 500}),
json!({"megapixels": 1}),
json!({"percent": 50}),
json!({"percent": 1000, "dontEnlarge": false}),
];
for p in modes {
for (w, h) in [(70_000.0, 1.0), (1.0, 70_000.0), (1e9, 3.0), (65_536.0, 1.0)] {
let r = ExportOptions::from_json(&p).resize.unwrap();
let (ow, oh) = r.apply(w, h);
assert!((1..=65_535).contains(&ow) && (1..=65_535).contains(&oh), "{p} on {w}×{h}: {ow}×{oh}");
assert!(r.scale(w, h).is_finite());
}
}
// (a short edge under a pixel would be scaled up to one, which the limit then caps)
assert_eq!(Resize::long_edge(2000).apply(70_000.0, 1.0), (65_535, 1));
assert_eq!(Resize::long_edge(2000).apply(1.0, 70_000.0), (1, 65_535));
}
/// A flat field of a saturated green inside Display P3 but outside sRGB, rendered into `space`.
fn p3_green(space: OutputSpace) -> Rgba8 {
use lightcraft_color::{DISPLAY_P3, REC2020};
+143
View File
@@ -0,0 +1,143 @@
//! Face tiles cut from a raw's embedded preview (issue #1440).
//!
//! A People card's face picture ([`crate::Session::face_job`]) is a small square around the face.
//! Developing the raw for it reads and demosaics most of a 24–50 MB file for a ~170 px picture; the
//! camera's own JPEG inside the file (read through the planned preview path,
//! [`crate::files::preview_window`], which reads the header and the preview and not the sensor
//! data) usually holds that square at the tile's size already. A [`PreviewCut`] says which preview
//! to read and how large it must be; [`crate::media::RenderJob::run`] tries it before the job's own
//! source and falls back to that source when the preview can't serve.
//!
//! Only a raw that still looks the way it was imported gets one: that is the photo whose grid
//! thumbnail is the same embedded JPEG ([`crate::Session::embedded_thumb_is_final`]). An edited raw
//! needs its edits rendered, which the camera's display-referred JPEG can't take the way the raw
//! does, so it keeps the full develop render.
use lightcraft_geom::Rect;
use lightcraft_raster::Rgba8;
use lightcraft_raster::resample::{Filter, resize};
use crate::media::PreviewLoader;
/// How much smaller than asked an embedded preview may be and still serve a face tile: at least
/// three quarters of the pixels the tile asks for across the face. The square is then enlarged by
/// at most 4⁄3 on its way to the tile, which at a 56–180 pt tile is not visible (the camera's JPEG
/// is sharpened and denoised, so its pixels hold more detail than a binned raw's), and it is the
/// same bound the raw path already accepts for an X-Trans sensor binned 3× for a preview
/// ([`crate::files::bin_factor`]). Below it the tile would look soft: the job's own source renders it.
pub const SOFT_LIMIT: f64 = 0.75;
/// Largest relative difference between the preview's and the photo's aspect ratios: a preview of
/// another shape (letterboxed, 16:9 from a 3:2 sensor, a 4:3 sensor's 3:2 JPEG) does not show the
/// same frame, and the face box would land beside the face. Real previews of the same frame differ
/// by rounding only (Sony A7S III: 1616 × 1080 against 4240 × 2832, 0.06 %).
const ASPECT_TOLERANCE: f64 = 0.02;
/// A face tile to cut from the photo's embedded preview.
#[derive(Clone)]
pub struct PreviewCut {
/// The raw file.
pub path: String,
/// Reads its largest embedded preview, display-ready and upright, no larger than the edge given.
pub loader: PreviewLoader,
/// The preview's long edge that gives the tile its own size across the face.
pub want: usize,
/// The smallest long edge accepted ([`SOFT_LIMIT`] of `want`).
pub min: usize,
/// The square around the face, normalized on the upright photo.
pub rect: Rect,
/// The photo's width / height, upright.
pub aspect: f64,
/// The tile's edge, pixels.
pub edge: usize,
}
impl PreviewCut {
/// The tile, or `None` when the preview can't give it: none could be read (no preview, an
/// offline or damaged file), it is smaller than [`Self::min`], or it shows another frame.
pub fn cut(&self) -> Option<Rgba8> {
let preview = (self.loader)(&self.path, self.want)?;
self.cut_from(&preview)
}
/// [`Self::cut`] from a preview already read.
pub fn cut_from(&self, img: &Rgba8) -> Option<Rgba8> {
let (w, h) = (img.width, img.height);
if w == 0 || h == 0 || w.max(h) < self.min || img.data.len() != w.checked_mul(h)? {
return None;
}
let aspect = w as f64 / h as f64;
if !(self.aspect.is_finite() && self.aspect > 0.0) || ((aspect - self.aspect) / self.aspect).abs() > ASPECT_TOLERANCE {
return None;
}
let r = self.rect;
if ![r.x0, r.y0, r.x1, r.y1].iter().all(|v| v.is_finite()) {
return None;
}
// pixel bounds inside the preview (the clamps keep every value in 0..=w / 0..=h)
let px = |v: f64, n: usize| (v.clamp(0.0, 1.0) * n as f64).round() as usize;
let (x0, x1, y0, y1) = (px(r.x0, w), px(r.x1, w), px(r.y0, h), px(r.y1, h));
if x1 <= x0 || y1 <= y0 {
return None;
}
let edge = self.edge.clamp(1, crate::media::SourceLevel::Thumb.max_edge());
let square = img.crop(x0, y0, x1 - x0, y1 - y0).to_linear();
Some(resize(&square, edge, edge, Filter::Mitchell).to_srgb8())
}
}
#[cfg(test)]
mod tests {
use super::*;
use std::sync::Arc;
fn cut(rect: Rect, aspect: f64, min: usize) -> PreviewCut {
PreviewCut { path: String::new(), loader: Arc::new(|_, _| None), want: 300, min, rect, aspect, edge: 32 }
}
fn picture(w: usize, h: usize) -> Rgba8 {
let mut img = Rgba8::new(w, h);
for (i, p) in img.data.iter_mut().enumerate() {
*p = [(i % w * 255 / w) as u8, (i / w * 255 / h) as u8, 90, 255];
}
img
}
/// The square comes out the tile's size, from the place the box names.
#[test]
fn cuts_the_square_at_the_tiles_size() {
let img = picture(300, 200);
let out = cut(Rect { x0: 0.5, y0: 0.5, x1: 0.6, y1: 0.65 }, 1.5, 200).cut_from(&img).unwrap();
assert_eq!((out.width, out.height), (32, 32));
// red follows x: the box starts at half the width
let red = out.data[16 * 32 + 16][0];
assert!((125..=160).contains(&red), "{red}");
}
/// A preview too small, of another shape, empty, or a box that is no box: no tile, no panic.
#[test]
fn declines_what_it_cannot_cut() {
let img = picture(300, 200);
let ok = Rect { x0: 0.4, y0: 0.4, x1: 0.5, y1: 0.55 };
assert!(cut(ok, 1.5, 301).cut_from(&img).is_none(), "below the soft limit");
assert!(cut(ok, 16.0 / 9.0, 200).cut_from(&img).is_none(), "another frame");
assert!(cut(ok, f64::NAN, 200).cut_from(&img).is_none());
assert!(cut(ok, 0.0, 200).cut_from(&img).is_none());
assert!(cut(ok, 1.5, 0).cut_from(&Rgba8::new(0, 0)).is_none(), "empty preview");
let short = Rgba8 { width: 300, height: 200, data: vec![[0; 4]; 10] };
assert!(cut(ok, 1.5, 0).cut_from(&short).is_none(), "a buffer shorter than its size");
for bad in [
Rect { x0: f64::NAN, y0: 0.4, x1: 0.5, y1: 0.5 },
Rect { x0: 0.5, y0: 0.5, x1: 0.5, y1: 0.5 },
Rect { x0: 0.6, y0: 0.6, x1: 0.4, y1: 0.4 },
Rect { x0: 2.0, y0: 2.0, x1: 3.0, y1: 3.0 },
] {
assert!(cut(bad, 1.5, 0).cut_from(&img).is_none(), "{bad:?}");
}
// a box past the edges is clamped to the picture
let edge = cut(Rect { x0: -1.0, y0: -1.0, x1: 0.1, y1: 0.15 }, 1.5, 0).cut_from(&img).unwrap();
assert_eq!((edge.width, edge.height), (32, 32));
// and a loader with nothing to give leaves the job to its source
assert!(cut(ok, 1.5, 0).cut().is_none());
}
}
+152 -48
View File
@@ -450,7 +450,8 @@ fn load_bytes_now(
stages.push(("highlights", t0.elapsed()));
colour.apply(&mut img);
stages.push(("colour", t0.elapsed()));
let img = fit(&img, max_edge, max_edge, Filter::Box);
// a full-size render keeps the picture (fitting it to its own size would copy it, ~390 MB at 33 MP)
let img = if img.width.max(img.height) > max_edge { fit(&img, max_edge, max_edge, Filter::Box) } else { img };
stages.push(("fit", t0.elapsed()));
let img = img.into_oriented(raw.orientation);
stages.push(("orient", t0.elapsed()));
@@ -622,11 +623,16 @@ fn preview_orientation(raw_bytes: &[u8], jpeg_orientation: u16) -> Orientation {
lightcraft_meta::extract(raw_bytes).orientation.unwrap_or(Orientation::Normal)
}
/// What stands in for a raw file we can't decode: its largest embedded JPEG, else, for a TIFF-based
/// raw, the file's own first image when the TIFF reader can show it (a reduced RGB copy: the only
/// preview some containers carry). Returns image bytes for [`lightcraft_codecs::decode`].
fn stand_in_image(bytes: &[u8]) -> Option<std::borrow::Cow<'_, [u8]>> {
if let Some(jpeg) = lightcraft_raw::embedded_preview(bytes) {
/// What stands in for a raw file we can't decode: its largest embedded JPEG (with `covering`, the smallest one that is
/// at least that big, [`lightcraft_raw::embedded_preview_covering`]), else, for a TIFF-based raw, the file's own first
/// image when the TIFF reader can show it (a reduced RGB copy: the only preview some containers carry). Returns image
/// bytes for [`lightcraft_codecs::decode`].
fn stand_in_image(bytes: &[u8], covering: Option<usize>) -> Option<std::borrow::Cow<'_, [u8]>> {
let jpeg = match covering {
Some(edge) => lightcraft_raw::embedded_preview_covering(bytes, u32::try_from(edge).unwrap_or(u32::MAX)),
None => lightcraft_raw::embedded_preview(bytes),
};
if let Some(jpeg) = jpeg {
return Some(std::borrow::Cow::Owned(jpeg));
}
let undecodable = lightcraft_raw::probe(bytes).is_some_and(|f| !f.is_supported());
@@ -634,9 +640,14 @@ fn stand_in_image(bytes: &[u8]) -> Option<std::borrow::Cow<'_, [u8]>> {
.then_some(std::borrow::Cow::Borrowed(bytes))
}
/// Shared colour interpretation for quick previews, unsupported RAW fallback and camera-look fitting.
pub(crate) fn decode_raw_preview(bytes: &[u8], opts: lightcraft_codecs::DecodeOptions) -> Option<lightcraft_codecs::Decoded> {
let jpeg = stand_in_image(bytes)?;
/// Shared colour interpretation for quick previews, unsupported RAW fallback and camera-look fitting (`covering`: see
/// [`stand_in_image`]).
pub(crate) fn decode_raw_preview(
bytes: &[u8],
opts: lightcraft_codecs::DecodeOptions,
covering: Option<usize>,
) -> Option<lightcraft_codecs::Decoded> {
let jpeg = stand_in_image(bytes, covering)?;
// the colour space is a property of the JPEG; a TIFF stand-in is read by the TIFF reader
let is_jpeg = matches!(jpeg, std::borrow::Cow::Owned(_));
match lightcraft_raw::embedded_preview_color_space(bytes).filter(|_| is_jpeg) {
@@ -653,7 +664,7 @@ pub(crate) fn decode_raw_preview(bytes: &[u8], opts: lightcraft_codecs::DecodeOp
/// Oriented size of the embedded preview of a raw file.
fn embedded_preview_size(bytes: &[u8]) -> Option<(u32, u32)> {
let jpeg = stand_in_image(bytes)?;
let jpeg = stand_in_image(bytes, None)?;
// headers only, as for other images (issue #367): decoding a full-size preview just for its
// size took about a second per file in an import; a decode remains the fallback
let (mut w, mut h, orientation) = match lightcraft_codecs::read_header(&jpeg) {
@@ -671,16 +682,18 @@ fn embedded_preview_size(bytes: &[u8]) -> Option<(u32, u32)> {
/// The embedded preview of a raw file as a working-space image no larger than `max_edge`, oriented.
pub fn load_embedded_preview(bytes: &[u8], max_edge: usize) -> Option<(Rgb32f, SourceInfo)> {
let d = decode_raw_preview(bytes, fit_box(max_edge))?;
// always the largest JPEG: a source's geometry must not depend on its size level (previews match exports)
let d = decode_raw_preview(bytes, fit_box(max_edge), None)?;
let img = d.to_working();
let img = if img.width.max(img.height) > max_edge { fit(&img, max_edge, max_edge, Filter::Mitchell) } else { img };
Some((img.oriented(preview_orientation(bytes, d.orientation)), SourceInfo::default()))
}
/// The embedded preview of a raw file for display (sRGB, oriented, no larger than `max_edge`): the
/// loupe and grid show it until the raw itself has been developed ([`crate::media::QuickJob`]).
/// loupe and grid show it until the raw itself has been developed ([`crate::media::QuickJob`]). The smallest
/// embedded JPEG that covers `max_edge` is decoded, not the largest.
pub fn embedded_preview_srgb(bytes: &[u8], max_edge: usize) -> Option<lightcraft_raster::Rgba8> {
let mut d = decode_raw_preview(bytes, fit_box(max_edge))?;
let mut d = decode_raw_preview(bytes, fit_box(max_edge), Some(max_edge))?;
if d.image.width.max(d.image.height) > max_edge {
d.image = fit(&d.image, max_edge, max_edge, Filter::Box);
d.alpha = None;
@@ -708,7 +721,11 @@ pub fn fs_preview_loader() -> PreviewLoader {
let len = std::fs::metadata(path).map(|m| usize::try_from(m.len()).unwrap_or(usize::MAX)).unwrap_or(0);
let gate = crate::memory::work_gate();
let _permit = if crate::memory::is_background() { gate.acquire(len.saturating_mul(2)) } else { gate.acquire_urgent(len.saturating_mul(2)) };
if let Some(image) = preview_window(path).ok().flatten().and_then(|(header, window, _)| quick_image_srgb(&header, &window, max_edge)) {
if let Some(image) = preview_window(path, max_edge)
.ok()
.flatten()
.and_then(|(header, window, read)| quick_image_srgb(&header, &window, read.adobe_rgb, max_edge))
{
return Some(image);
}
embedded_preview_srgb(&std::fs::read(path).ok()?, max_edge)
@@ -729,6 +746,9 @@ pub struct PreviewRead {
/// path a Canon CR3 and the TIFF raws whose preview tag sits past the prefix take (`false` on
/// the header path, and on the whole-file fallback).
pub planned: bool,
/// Adobe RGB when the container says the preview is in it and the JPEG doesn't say otherwise
/// (`decode_raw_preview` applies the same declaration): the quick decode applies it too.
pub adobe_rgb: bool,
}
impl PreviewRead {
@@ -743,7 +763,9 @@ impl PreviewRead {
/// The file's header and its embedded preview, read without the rest of the file: `(header bytes,
/// preview bytes, what was read)`. `None` when the header points at no preview this build can find,
/// which is the caller's cue to read the file whole.
/// which is the caller's cue to read the file whole. `max_edge` is the size the preview is for: where a file keeps
/// several JPEGs (the plan route), the window is the one [`embedded_preview_srgb`] decodes for that size, the smallest
/// that covers it ([`lightcraft_raw::embedded_preview_covering`]).
///
/// Why it exists: a 56 MB Fuji RAF keeps its JPEG in the first 5.6 MB (measured: `DSCF9492.RAF`),
/// so reading the file whole to extract it costs ten times the bytes — and on a network share ten
@@ -751,7 +773,7 @@ impl PreviewRead {
/// still needs the whole container, so the header comes back with the window; for a preview taken
/// out of a container that carries no preview colour space of its own (Nikon's maker note) the
/// whole-file fallback stays in charge.
pub fn preview_window(path: &str) -> Result<Option<(Vec<u8>, Vec<u8>, PreviewRead)>, String> {
pub fn preview_window(path: &str, max_edge: usize) -> Result<Option<(Vec<u8>, Vec<u8>, PreviewRead)>, String> {
use std::io::{Read, Seek, SeekFrom};
let mut file = std::fs::File::open(path).map_err(|e| format!("{path}: {e}"))?;
@@ -773,7 +795,7 @@ pub fn preview_window(path: &str) -> Result<Option<(Vec<u8>, Vec<u8>, PreviewRea
// preview tag sits past the prefix, the containers whose preview only a scan finds: ask the
// sparse planner which ranges `embedded_preview` reads before handing the caller the
// whole-file fallback.
return planned_preview(&mut file, path, total);
return planned_preview(&mut file, path, total, max_edge);
};
// A preview at the file's end may have been padded past it: clamp rather than fail.
let end = range.offset.saturating_add(range.window_len).min(total);
@@ -796,7 +818,7 @@ pub fn preview_window(path: &str) -> Result<Option<(Vec<u8>, Vec<u8>, PreviewRea
if lightcraft_raw::embedded_preview_from_range(&window, range).is_none() {
return Ok(None);
}
let read = PreviewRead { read: bytes_read, total, header_path: true, planned: false };
let read = PreviewRead { read: bytes_read, total, header_path: true, planned: false, adobe_rgb: false };
// On a network share the bytes are the wait, so say what this preview actually cost. A window
// taken from the prefix cost nothing beyond the prefix itself.
if lightcraft_pipeline::profiling() {
@@ -838,7 +860,7 @@ const PLAN_PREFIX: u64 = 64 * 1024;
/// below). The planner's contract is the one its corpus test asserts over real files (what it loaded
/// gives what the whole file gives), so a window that still fails to decode ends in the same
/// whole-file fallback in [`fs_preview_loader`], never in a wrong picture.
fn planned_preview(file: &mut std::fs::File, path: &str, total: u64) -> Result<Option<(Vec<u8>, Vec<u8>, PreviewRead)>, String> {
fn planned_preview(file: &mut std::fs::File, path: &str, total: u64, max_edge: usize) -> Result<Option<(Vec<u8>, Vec<u8>, PreviewRead)>, String> {
use std::io::{Read, Seek, SeekFrom};
if total > MAX_PREVIEW_PLAN_BYTES {
@@ -867,19 +889,20 @@ fn planned_preview(file: &mut std::fs::File, path: &str, total: u64) -> Result<O
}
}
lightcraft_raw::sparse::HeaderPlan::Ready => {
let Some(window) = lightcraft_raw::embedded_preview(&buf) else { return Ok(None) };
// the plan loads every preview's strips, so the JPEG picked for `max_edge` is the whole file's pick
let edge = u32::try_from(max_edge).unwrap_or(u32::MAX);
let Some(window) = lightcraft_raw::embedded_preview_covering(&buf, edge) else { return Ok(None) };
// The quick path hands the JPEG to the decoder alone, so what the *container* says
// about the preview has to be something the bytes it is handed can show. Two container
// facts change the pixels and are not in the JPEG: the preview's colour space (which
// the whole-file path applies through `decode_raw_preview`) and, when the JPEG carries
// no orientation of its own, the file's EXIF orientation. Where either is present and
// the header cannot answer for it, the whole-file path keeps the preview — exactly as
// it already does for a container-declared colour space. A container that says sRGB
// agrees with the JPEG-only decode (its own declaration, else sRGB), so only the other
// space declines.
if lightcraft_raw::embedded_preview_color_space(&buf) == Some(lightcraft_raw::PreviewColorSpace::AdobeRgb) {
return Ok(None);
}
// no orientation of its own, the file's EXIF orientation. The colour space is read
// here and handed on ([`PreviewRead::adobe_rgb`]: a Sony α1 writes Adobe RGB previews,
// which used to send each of them to the whole-file path, issue #1440); a container
// that says sRGB agrees with the JPEG-only decode (its own declaration, else sRGB).
// Where the orientation is present and the header cannot answer for it, the
// whole-file path keeps the preview.
let adobe_rgb = lightcraft_raw::embedded_preview_color_space(&buf) == Some(lightcraft_raw::PreviewColorSpace::AdobeRgb);
// `extract` copies a TIFF's bytes to hand on its Exif block, so ask the cheap side
// first: a JPEG that carries its own orientation needs nothing from the container.
if lightcraft_meta::extract(&window).orientation.is_none()
@@ -888,7 +911,7 @@ fn planned_preview(file: &mut std::fs::File, path: &str, total: u64) -> Result<O
{
return Ok(None);
}
let read = PreviewRead { read: loaded.total(), total, header_path: true, planned: true };
let read = PreviewRead { read: loaded.total(), total, header_path: true, planned: true, adobe_rgb };
// A plan-driven preview is the one that used to cost the whole file, so it is worth
// saying what it cost now too.
if lightcraft_pipeline::profiling() {
@@ -937,9 +960,15 @@ fn raf_preview_extent(prefix: &[u8]) -> Option<u64> {
/// The display-ready preview of a raw whose embedded JPEG was already located: the counterpart of
/// [`embedded_preview_srgb`] for [`preview_window`], which does not re-read the container.
/// `None` when the bytes are not a decodable image.
fn quick_image_srgb(header: &[u8], window: &[u8], max_edge: usize) -> Option<lightcraft_raster::Rgba8> {
let mut d = lightcraft_codecs::decode(window, fit_box(max_edge)).ok()?;
/// `adobe_rgb`: the container declares the preview Adobe RGB ([`PreviewRead::adobe_rgb`]), applied
/// as [`decode_raw_preview`] applies it. `None` when the bytes are not a decodable image.
fn quick_image_srgb(header: &[u8], window: &[u8], adobe_rgb: bool, max_edge: usize) -> Option<lightcraft_raster::Rgba8> {
let decoded = if adobe_rgb && window.starts_with(&[0xff, 0xd8]) {
lightcraft_codecs::decode_jpeg_with_fallback(window, fit_box(max_edge), lightcraft_codecs::NamedSpace::AdobeRgb)
} else {
lightcraft_codecs::decode(window, fit_box(max_edge))
};
let mut d = decoded.ok()?;
if d.image.width.max(d.image.height) > max_edge {
d.image = fit(&d.image, max_edge, max_edge, Filter::Box);
d.alpha = None;
@@ -1125,7 +1154,7 @@ mod tests {
let path = path.to_string_lossy().into_owned();
let max_edge = 512;
let (header, window, read) = preview_window(&path).expect("the file reads").expect("it has a preview");
let (header, window, read) = preview_window(&path, max_edge).expect("the file reads").expect("it has a preview");
assert_eq!(header.len(), lightcraft_raw::PREVIEW_PREFIX, "the header prefix is bounded");
assert!(read.header_path, "the header path is what answered");
assert!(!read.planned, "a RAF pointer in the header needs no planner");
@@ -1134,7 +1163,7 @@ mod tests {
assert!(window.len() < file.len() / 2, "the raw tail must not be read: {} of {} bytes", window.len(), file.len());
// The quick path and the whole-file path agree byte for byte.
let quick = quick_image_srgb(&header, &window, max_edge).expect("the window holds a decodable preview");
let quick = quick_image_srgb(&header, &window, read.adobe_rgb, max_edge).expect("the window holds a decodable preview");
let whole = embedded_preview_srgb(&file, max_edge).expect("the whole file decodes too");
assert_eq!((quick.width, quick.height), (whole.width, whole.height), "same preview size");
assert_eq!(quick.data, whole.data, "byte for byte");
@@ -1142,8 +1171,8 @@ mod tests {
// A file with no locatable preview declines (the caller then reads it whole), a missing file errors.
let plain = dir.join("plain.jpg");
std::fs::write(&plain, &jpeg).unwrap();
assert!(preview_window(&plain.to_string_lossy()).expect("a plain file reads").is_none());
assert!(preview_window(&dir.join("nope.raf").to_string_lossy()).is_err());
assert!(preview_window(&plain.to_string_lossy(), max_edge).expect("a plain file reads").is_none());
assert!(preview_window(&dir.join("nope.raf").to_string_lossy(), max_edge).is_err());
let _ = std::fs::remove_dir_all(&dir);
}
@@ -1163,6 +1192,12 @@ mod tests {
/// `tail` bytes. The writer puts each IFD's table after its own chunk, so a `strip` bigger than
/// the plan's bounded header puts IFD0 (and any orientation tag in it) out of that header's reach.
fn tiff_with_preview_strip(strip: &[u8], tail: usize, what: Fixture) -> Vec<u8> {
tiff_with_preview_strips(strip, &[], tail, what)
}
/// [`tiff_with_preview_strip`] that also keeps the `smaller` JPEGs in sub-IFDs of IFD0, as an ARW keeps its
/// 1616 px preview beside the full-size one.
fn tiff_with_preview_strips(strip: &[u8], smaller: &[&[u8]], tail: usize, what: Fixture) -> Vec<u8> {
use lightcraft_tiff::{IfdBuilder, ImageData, TiffWriter, Value, tags as t};
let mut preview = IfdBuilder::new();
if what.dng {
@@ -1189,6 +1224,15 @@ mod tests {
preview.set(t::IMAGE_WIDTH, Value::Long(vec![48]));
preview.set(t::IMAGE_LENGTH, Value::Long(vec![32]));
preview.set_image(ImageData::Strips { rows_per_strip: 32, strips: vec![strip.to_vec()] });
for jpeg in smaller {
let mut sub = IfdBuilder::new();
sub.set(t::COMPRESSION, Value::Short(vec![7]));
sub.set(t::PHOTOMETRIC, Value::Short(vec![t::photometric::RGB]));
sub.set(t::IMAGE_WIDTH, Value::Long(vec![48]));
sub.set(t::IMAGE_LENGTH, Value::Long(vec![32]));
sub.set_image(ImageData::Strips { rows_per_strip: 32, strips: vec![jpeg.to_vec()] });
preview.add_sub_ifd(sub);
}
let mut raw = IfdBuilder::new();
raw.set(t::COMPRESSION, Value::Short(vec![1]));
raw.set(t::PHOTOMETRIC, Value::Short(vec![t::photometric::CFA]));
@@ -1216,7 +1260,7 @@ mod tests {
let path = path.to_string_lossy().into_owned();
let max_edge = 512;
let (header, window, read) = preview_window(&path).expect("the file reads").expect("the plan locates its preview");
let (header, window, read) = preview_window(&path, max_edge).expect("the file reads").expect("the plan locates its preview");
assert_eq!(header.len(), PLAN_PREFIX as usize, "the plan's header is bounded");
assert!(read.planned, "the planner is what answered");
assert!(read.header_path, "and it is still a bounded read");
@@ -1225,7 +1269,7 @@ mod tests {
assert!(window.len() < file.len() / 2, "the sensor strip must not be read: {} of {} bytes", window.len(), file.len());
// The plan path and the whole-file path agree byte for byte.
let quick = quick_image_srgb(&header, &window, max_edge).expect("the window holds a decodable preview");
let quick = quick_image_srgb(&header, &window, read.adobe_rgb, max_edge).expect("the window holds a decodable preview");
let whole = embedded_preview_srgb(&file, max_edge).expect("the whole file decodes too");
assert_eq!((quick.width, quick.height), (whole.width, whole.height), "same preview size");
assert_eq!(quick.data, whole.data, "byte for byte");
@@ -1234,7 +1278,39 @@ mod tests {
// caller's whole-file read is what answers (no guessing at what a file might hold).
let not_a_raw = tiff_with_preview_strip(&jpeg, 400_000, Fixture::default());
std::fs::write(&path, &not_a_raw).unwrap();
assert!(preview_window(&path).expect("the file reads").is_none(), "an unknown container falls back");
assert!(preview_window(&path, max_edge).expect("the file reads").is_none(), "an unknown container falls back");
let _ = std::fs::remove_dir_all(&dir);
}
/// A raw that keeps several JPEGs is shown, on the plan route as on the whole-file path, from the smallest one that
/// covers the box (`embedded_preview_covering`). The plan route used to hand on the largest whatever the size: an
/// A7R VI ARW's 9984 × 6656 JPEG for a 512 px thumbnail instead of its 1616 × 1080 one.
#[test]
fn preview_loading_plans_the_smallest_covering_jpeg() {
let jpeg = |w: u32, h: u32| {
let rgb: Vec<u8> = (0..w * h * 3).map(|i| (i * 31 % 247) as u8).collect();
encode_jpeg(&EncodeImage::new(w, h, 3, Samples::U8(&rgb)), 90, ChromaSubsampling::S420, &EncodeMeta::default()).unwrap()
};
let (big, small) = (jpeg(96, 64), jpeg(48, 32));
assert!(big.len() > small.len(), "the full-size JPEG is the largest");
let file = tiff_with_preview_strips(&big, &[&small], 400_000, Fixture { dng: true, ..Default::default() });
let dir = crate::tests_xmp::temp_dir("preview-plan-covering");
let path = dir.join("two-previews.dng");
std::fs::write(&path, &file).unwrap();
let path = path.to_string_lossy().into_owned();
for (edge, want, which) in [(48, &small, "small"), (49, &big, "full-size"), (usize::MAX, &big, "full-size")] {
let (header, window, read) = preview_window(&path, edge).expect("the file reads").expect("the plan locates its preview");
assert!(read.planned, "the planner is what answered");
assert!(&window == want, "a {edge} px box takes the {which} JPEG");
let quick = quick_image_srgb(&header, &window, read.adobe_rgb, edge).expect("the window holds a decodable preview");
let whole = embedded_preview_srgb(&file, edge).expect("the whole file decodes too");
assert_eq!(
(quick.width, quick.height, quick.data),
(whole.width, whole.height, whole.data),
"{edge} px: the plan and the whole file agree"
);
}
let _ = std::fs::remove_dir_all(&dir);
}
@@ -1262,17 +1338,17 @@ mod tests {
assert_eq!(lightcraft_meta::extract(&file).orientation, Some(Orientation::from_exif(6)), "the fixture states an orientation");
assert!(lightcraft_meta::extract(&file[..PLAN_PREFIX as usize]).orientation.is_none(), "which the bounded header cannot see");
assert!(lightcraft_meta::extract(&jpeg).orientation.is_none(), "and the preview JPEG carries none of its own");
assert!(preview_window(&path).expect("the file reads").is_none(), "the whole-file path keeps it");
assert!(preview_window(&path, usize::MAX).expect("the file reads").is_none(), "the whole-file path keeps it");
let _ = std::fs::remove_dir_all(&dir);
}
/// A container can declare what its preview JPEG is in, and the whole-file path applies that
/// declaration (`decode_raw_preview`). A JPEG-only decode has the JPEG's own space, else sRGB, so
/// a container that declares Adobe RGB must keep the whole-file path — while one that declares
/// sRGB (what most bodies write) must **not** take the fast path away: that half is a regression
/// test, since a guard that declined every declared space did exactly that to a real Sony ARW.
/// declaration (`decode_raw_preview`). The planned path reads it too and the quick decode applies
/// it, so a container that declares Adobe RGB (a Sony α1's ARW) is read in part like any other and
/// shows the same pixels as the whole-file path (issue #1440; it used to be read whole) — and one
/// that declares sRGB (what most bodies write) keeps the fast path as before.
#[test]
fn preview_loading_declines_only_a_container_colour_space_it_cannot_apply() {
fn preview_loading_applies_a_container_colour_space() {
use lightcraft_raw::PreviewColorSpace;
let (w, h) = (48u32, 32u32);
let rgb: Vec<u8> = (0..w * h * 3).map(|i| (i * 23 % 233) as u8).collect();
@@ -1285,13 +1361,20 @@ mod tests {
let adobe = tiff_with_preview_strip(&jpeg, 400_000, Fixture { sony: true, color_space: Some(0xFFFF), ..Default::default() });
assert_eq!(lightcraft_raw::embedded_preview_color_space(&adobe), Some(PreviewColorSpace::AdobeRgb), "the fixture declares Adobe RGB");
std::fs::write(&path, &adobe).unwrap();
assert!(preview_window(&path).expect("the file reads").is_none(), "the whole-file path applies the space");
let (header, window, read) = preview_window(&path, usize::MAX).expect("the file reads").expect("the plan answers for Adobe RGB too");
assert!(read.planned && read.adobe_rgb, "{read:?}");
assert!(read.fraction() < 0.5, "the sensor strip is not read: {read:?}");
let quick = quick_image_srgb(&header, &window, read.adobe_rgb, 512).expect("the window decodes");
let whole = embedded_preview_srgb(&adobe, 512).expect("the whole file decodes too");
assert_eq!(quick.data, whole.data, "the space is applied as the whole-file path applies it");
let plain = quick_image_srgb(&header, &window, false, 512).expect("the window decodes");
assert_ne!(plain.data, whole.data, "and it changes the pixels (the fixture's colours are saturated enough)");
let srgb = tiff_with_preview_strip(&jpeg, 400_000, Fixture { sony: true, color_space: Some(1), ..Default::default() });
assert_eq!(lightcraft_raw::embedded_preview_color_space(&srgb), Some(PreviewColorSpace::Srgb), "the fixture declares sRGB");
std::fs::write(&path, &srgb).unwrap();
let (_, window, read) = preview_window(&path).expect("the file reads").expect("sRGB is what the JPEG-only decode assumes");
assert!(read.planned, "the plan still answers");
let (_, window, read) = preview_window(&path, usize::MAX).expect("the file reads").expect("sRGB is what the JPEG-only decode assumes");
assert!(read.planned && !read.adobe_rgb, "the plan still answers");
assert!(window.starts_with(&[0xff, 0xd8]), "and hands on the JPEG");
}
@@ -2296,4 +2379,25 @@ mod tests {
assert_eq!(s.quick_view_job(id, 1600, true).unwrap().run().quick, Some(QuickSource::Small));
let _ = std::fs::remove_dir_all(&dir);
}
/// Issue #1448: the grid renders nothing behind an unedited raw's quick thumbnail, so when the
/// embedded preview can't be read (and nothing is cached) the quick job itself must render the
/// raw, or the cell stays empty for good ("no quick preview").
#[test]
fn a_quick_thumbnail_without_an_embedded_preview_renders_the_raw() {
use crate::media::QuickSource;
let dir = std::env::temp_dir().join(format!("lc-quick-none-{}", std::process::id()));
std::fs::create_dir_all(&dir).unwrap();
let path = dir.join("a.cr3");
std::fs::write(&path, cr3_with_preview(96, 64)).unwrap();
let mut s = crate::Session::new().with_fs();
let r = s.execute("library.import", &serde_json::json!({"paths": [path.to_string_lossy()]})).unwrap();
let id = lightcraft_catalog::PhotoId(r["imported"][0].as_u64().unwrap());
s.media.preview_loader = Some(std::sync::Arc::new(|_: &str, _| None));
let job = s.thumb_job(id, 128).unwrap();
let q = s.quick_thumb_job(&job).unwrap().run();
assert_eq!((q.quick, q.key), (Some(QuickSource::Small), job.key));
assert!(q.rendered.is_ok(), "{:?}", q.rendered.err());
let _ = std::fs::remove_dir_all(&dir);
}
}
+103 -45
View File
@@ -548,12 +548,62 @@ fn copy_into(dir: &Path, src: &str, name: Option<&str>, probe_hash: Option<&str>
pub struct ScanInput {
probe: Option<crate::media::FileProbe>,
skip: Option<PathBuf>,
/// path → (photo, its summary) for files already in the library.
by_path: HashMap<PathBuf, (PhotoId, ImportCandidate)>,
by_hash: HashMap<String, PhotoId>,
/// The library as it is now (photos are shared with the session, not copied): looked up for
/// the files the scan finds ([`library_files`]) and the contents it probes
/// ([`library_hashes`]), on the scan's thread.
library: std::sync::Arc<lightcraft_catalog::Catalog>,
cache: HashMap<String, ProbeInfo>,
}
/// The library's photos of `files`: path → (photo, its summary as a candidate that is there
/// already). One pass over the library, and a record only for a file that was found: a folder of
/// a hundred files in a library of millions costs a hundred records.
fn library_files(library: &lightcraft_catalog::Catalog, files: &[String]) -> HashMap<PathBuf, (PhotoId, ImportCandidate)> {
let found: std::collections::HashSet<&Path> = files.iter().map(Path::new).collect();
let mut by_path = HashMap::new();
if found.is_empty() {
return by_path;
}
for p in library.photos() {
if let Source::File { path } = &p.source
&& found.contains(Path::new(path))
{
let c = ImportCandidate {
path: path.clone(),
format: p.format.clone(),
kind: p.kind,
width: p.width,
height: p.height,
file_size: p.file_size,
captured: p.captured.clone(),
duplicate: Some("path".into()),
existing: Some(p.id.0),
..Default::default()
};
by_path.insert(PathBuf::from(path), (p.id, c));
}
}
by_path
}
/// Which photo of the library has each of the contents `hashes` (the later one where several
/// have it). One pass over the library, and an entry only for a content that was asked about.
fn library_hashes<'a>(library: &lightcraft_catalog::Catalog, hashes: impl Iterator<Item = &'a str>) -> HashMap<String, PhotoId> {
let wanted: std::collections::HashSet<&str> = hashes.collect();
let mut by_hash = HashMap::new();
if wanted.is_empty() {
return by_hash;
}
for p in library.photos() {
if let Some(h) = &p.content_hash
&& wanted.contains(h.as_str())
{
by_hash.insert(h.clone(), p.id);
}
}
by_hash
}
/// Progress and cancellation of a running [`scan_with`].
#[derive(Default)]
pub struct ScanProgress {
@@ -573,31 +623,17 @@ pub struct ScanOutput {
impl ScanInput {
pub fn new(s: &mut Session, paths: &[String]) -> (Self, Vec<String>) {
let skip = s.library.as_ref().filter(|l| l.on_disk).map(|l| l.dir.clone());
let mut by_path = HashMap::new();
let mut by_hash = HashMap::new();
for p in s.catalog.photos() {
if let Source::File { path } = &p.source {
let c = ImportCandidate {
path: path.clone(),
format: p.format.clone(),
kind: p.kind,
width: p.width,
height: p.height,
file_size: p.file_size,
captured: p.captured.clone(),
duplicate: Some("path".into()),
existing: Some(p.id.0),
..Default::default()
};
by_path.insert(PathBuf::from(path), (p.id, c));
}
if let Some(h) = &p.content_hash {
by_hash.insert(h.clone(), p.id);
}
}
let input = ScanInput { probe: s.media.file_probe.clone(), skip, by_path, by_hash, cache: std::mem::take(&mut s.import_probes) };
// (no table of the library here: this is the session's thread, and the library may hold
// millions of photos. The scan looks up what it finds)
let library = std::sync::Arc::new(s.catalog.clone());
let input = ScanInput { probe: s.media.file_probe.clone(), skip, library, cache: std::mem::take(&mut s.import_probes) };
(input, paths.to_vec())
}
/// The library as the scan sees it (shared, for [`crate::sync`], which looks at the same one).
pub(crate) fn library(&self) -> std::sync::Arc<lightcraft_catalog::Catalog> {
std::sync::Arc::clone(&self.library)
}
}
/// Find and probe what importing `paths` would add, marking duplicates (by path, or by content
@@ -615,7 +651,8 @@ pub fn scan(s: &mut Session, paths: &[String]) -> Vec<ImportCandidate> {
pub fn scan_with(mut input: ScanInput, paths: &[String], progress: &ScanProgress) -> ScanOutput {
use std::sync::atomic::Ordering::Relaxed;
let files = expand(paths, input.skip.as_deref());
let todo: Vec<String> = files.iter().filter(|f| !input.by_path.contains_key(Path::new(f))).cloned().collect();
let by_path = library_files(&input.library, &files);
let todo: Vec<String> = files.iter().filter(|f| !by_path.contains_key(Path::new(f))).cloned().collect();
progress.total.store(todo.len(), Relaxed);
// probes from a preceding `scan` are reused when the file is unchanged (same size)
let cached: Vec<Option<ProbeInfo>> = todo
@@ -633,11 +670,14 @@ pub fn scan_with(mut input: ScanInput, paths: &[String], progress: &ScanProgress
cached.into_iter().map(|c| c.map(Ok).unwrap_or_else(|| fresh.next().unwrap_or_else(|| Err("not probed".into())))).collect();
crate::memory::release();
let mut probes: HashMap<String, Result<ProbeInfo, String>> = todo.into_iter().zip(probed).collect();
let mut seen_hash: HashMap<String, Option<u64>> = input.by_hash.into_iter().map(|(h, id)| (h, Some(id.0))).collect();
// (who in the library has a content that was probed: asked once the probes are in)
let probed_hashes = probes.values().filter_map(|r| r.as_ref().ok()).filter_map(|info| info.content_hash.as_deref());
let mut seen_hash: HashMap<String, Option<u64>> =
library_hashes(&input.library, probed_hashes).into_iter().map(|(h, id)| (h, Some(id.0))).collect();
let mut out = Vec::with_capacity(files.len());
let mut kept = HashMap::new();
for f in files {
if let Some((_, c)) = input.by_path.get(Path::new(&f)) {
if let Some((_, c)) = by_path.get(Path::new(&f)) {
let mut c = c.clone();
c.name = Path::new(&f).file_name().map(|n| n.to_string_lossy().to_string()).unwrap_or_else(|| f.clone());
out.push(c);
@@ -701,6 +741,12 @@ pub struct ImportJob {
mode: ImportMode,
lib_dir: Option<PathBuf>,
copy_root: Option<PathBuf>,
/// The library as it was when the import was set up, until it is looked up
/// ([`ImportJob::look_up_library`], with the first files): setting an import up is the
/// session's thread, and a table of a library of millions of photos takes seconds. With it,
/// whether a photo in Recently Deleted is passed over (an import that replaces such photos,
/// as the options said when the import was set up).
library: Option<(lightcraft_catalog::Catalog, bool)>,
/// path → (photo, a Local browse record?)
by_path: HashMap<String, (PhotoId, bool)>,
/// content hash → the photo that has it (`None`: a file earlier in this import)
@@ -842,28 +888,16 @@ impl ImportJob {
if matches!(mode, ImportMode::Copy | ImportMode::Move) && copy_root.is_none() {
return Err(crate::EngineError::Other("copying or moving needs a destination folder or an open library".into()));
}
let mut by_path = HashMap::new();
let mut by_hash = HashMap::new();
for p in s.catalog.photos() {
// a trashed photo is replaced by a fresh import, so it doesn't count as a duplicate
if p.deleted && opts.on_deleted == OnDeleted::Fresh {
continue;
}
if let Source::File { path } = &p.source {
by_path.insert(path.clone(), (p.id, p.local));
}
if let Some(h) = &p.content_hash {
by_hash.insert(h.clone(), Some(p.id));
}
}
let fresh = opts.on_deleted == OnDeleted::Fresh;
Ok(ImportJob {
seq: opts.rename_start.max(1),
opts,
mode,
lib_dir,
copy_root,
by_path,
by_hash,
library: Some((s.catalog.clone(), fresh)),
by_path: HashMap::new(),
by_hash: HashMap::new(),
probe: s.media.file_probe.clone(),
file_bytes: s.media.file_bytes.clone(),
cache: std::mem::take(&mut s.import_probes),
@@ -898,9 +932,28 @@ impl ImportJob {
&self.now
}
/// What the library held when the import was set up, by path and by content: made once, with
/// the first files (on the thread that prepares them), from the library as it was then.
fn look_up_library(&mut self) {
let Some((library, fresh)) = self.library.take() else { return };
for p in library.photos() {
// a trashed photo is replaced by a fresh import, so it doesn't count as a duplicate
if p.deleted && fresh {
continue;
}
if let Source::File { path } = &p.source {
self.by_path.insert(path.clone(), (p.id, p.local));
}
if let Some(h) = &p.content_hash {
self.by_hash.insert(h.clone(), Some(p.id));
}
}
}
/// [`ImportJob::prepare`] for files [`ImportJob::expand`] returned.
pub fn prepare_files(&mut self, files: Vec<String>, cancel: &std::sync::atomic::AtomicBool) -> Prepared {
use std::sync::atomic::Ordering::Relaxed;
self.look_up_library();
let opts = self.opts.clone();
let mode = self.mode;
let mut out = Prepared { scanned: files.len(), items: Vec::new() };
@@ -1479,6 +1532,7 @@ impl Session {
stages: None,
view_cache: None,
display: None,
preview_cut: None,
};
let embedded = match (&self.media.preview_loader, c.kind) {
(Some(l), MediaKind::Raw) => Some((c.path.clone(), l.clone(), edge)),
@@ -1494,6 +1548,10 @@ impl Session {
}
}
#[cfg(test)]
#[path = "import/tests_known.rs"]
mod tests_known;
#[cfg(test)]
mod prepared_tests {
use super::*;
+134
View File
@@ -0,0 +1,134 @@
//! What a scan or an import knows of the library: worked out for the files it meets, and not on
//! the session's thread. A library can hold millions of photos; looking at one folder must not
//! cost a table of them all before the first file is read.
use super::*;
use lightcraft_catalog::{Catalog, Op};
/// A library of `folders` folders of three photos each, the third a copy (by content) of a
/// photo in the folder before.
fn library(folders: usize) -> Catalog {
let mut c = Catalog::new();
for folder in 0..folders {
for (n, name) in ["a.jpg", "b.jpg", "c.jpg"].into_iter().enumerate() {
let id = c.alloc_photo_id();
let mut p = Photo::new(id, Source::File { path: format!("/pics/{folder:03}/{name}") }, name, "JPEG", 6, 4, "2026-01-01");
p.content_hash = Some(format!("{:032x}", if n == 2 { folder.saturating_sub(1) * 3 } else { folder * 3 + n }));
c.apply(Op::AddPhoto { photo: Box::new(p) }).unwrap();
}
}
c
}
/// Given a library of many folders, when a scan meets the files of one of them (and one the
/// library does not have), then what it knows of the library is those files' photos, each as the
/// library has it, and nothing of the other folders.
#[test]
fn a_scan_knows_the_library_photos_of_the_files_it_found() {
let c = library(100);
let files: Vec<String> = ["/pics/007/a.jpg", "/pics/007/b.jpg", "/pics/007/new.jpg", "/pics/007//c.jpg"].map(String::from).to_vec();
let known = library_files(&c, &files);
assert_eq!(known.len(), 3, "the three the library has (a path is compared as a path, not as text)");
let (id, as_found) = known.get(Path::new("/pics/007/b.jpg")).expect("b");
let p = c.photo(*id).unwrap();
assert_eq!((as_found.path.as_str(), as_found.existing, as_found.duplicate.as_deref()), ("/pics/007/b.jpg", Some(id.0), Some("path")));
assert_eq!((as_found.format.as_str(), as_found.width, as_found.height, as_found.file_size), (p.format.as_str(), p.width, p.height, p.file_size));
assert!(library_files(&c, &[]).is_empty());
// two photos of one file (a virtual copy): the later one is the one named, as a table of the
// whole library named it
let mut c = c;
let copy = c.alloc_photo_id();
c.apply(Op::AddPhoto { photo: Box::new(Photo::new(copy, Source::File { path: "/pics/007/a.jpg".into() }, "a.jpg", "JPEG", 6, 4, "2026-01-02")) })
.unwrap();
assert_eq!(library_files(&c, &files).get(Path::new("/pics/007/a.jpg")).map(|(id, _)| *id), Some(copy));
}
/// Given a library and a folder on disk that holds a file the library has by its path, one it
/// has by its content under another name, and a new one, when the folder is scanned (the whole
/// scan, as the import review runs it), then each is found for what it is.
#[test]
fn a_scan_finds_what_the_library_has_by_path_and_by_content() {
let dir = std::env::temp_dir().join(format!("lc-known-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&dir);
std::fs::create_dir_all(&dir).unwrap();
let at = |name: &str| dir.join(name).to_string_lossy().to_string();
for name in ["here.jpg", "copy.jpg", "new.jpg"] {
std::fs::write(dir.join(name), name.as_bytes()).unwrap();
}
let mut s = Session::new();
s.catalog = library(20);
let here = s.catalog.alloc_photo_id();
s.catalog
.apply(Op::AddPhoto { photo: Box::new(Photo::new(here, Source::File { path: at("here.jpg") }, "here.jpg", "JPEG", 6, 4, "2026-01-01")) })
.unwrap();
// (the probe: every file is a picture whose content is its name; `copy.jpg` has the content of a library photo)
let had = s.catalog.photos().find(|p| p.file_name == "b.jpg").map(|p| (p.id, p.content_hash.clone().unwrap())).unwrap();
let shared = had.1.clone();
s.media.file_probe = Some(std::sync::Arc::new(move |path: &str| {
let name = Path::new(path).file_name().map(|n| n.to_string_lossy().to_string()).unwrap_or_default();
let content_hash = Some(if name == "copy.jpg" { shared.clone() } else { format!("hash-of-{name}") });
Ok(ProbeInfo { format: "JPEG".into(), width: 6, height: 4, file_size: 8, content_hash, ..Default::default() })
}));
let found = scan(&mut s, &[dir.to_string_lossy().to_string()]);
let of = |name: &str| found.iter().find(|c| c.name == name).map(|c| (c.duplicate.as_deref(), c.existing)).unwrap();
assert_eq!(of("here.jpg"), (Some("path"), Some(here.0)));
assert_eq!(of("copy.jpg"), (Some("content"), Some(had.0.0)));
assert_eq!(of("new.jpg"), (None, None));
let _ = std::fs::remove_dir_all(&dir);
}
/// Given the hashes of the files a scan probed, when the library is asked who has them, then it
/// answers for those hashes only; and where two photos have one content, the same one of them
/// as a table of the whole library would name (the later one).
#[test]
fn a_scan_knows_who_has_the_contents_it_probed() {
let c = library(100);
// (the content of /pics/006/a.jpg, which /pics/007/c.jpg has too; and one nobody has)
let shared = format!("{:032x}", 6 * 3);
let nobody = format!("{:032x}", 999_999);
let known = library_hashes(&c, [shared.as_str(), nobody.as_str()].into_iter());
assert_eq!(known.len(), 1);
let whole: HashMap<String, PhotoId> = c.photos().filter_map(|p| p.content_hash.clone().map(|h| (h, p.id))).collect();
assert_eq!(known.get(&shared), whole.get(&shared));
assert!(matches!(&c.photo(known[&shared]).unwrap().source, Source::File { path } if path == "/pics/007/c.jpg"));
}
/// Given a session with a library, when an import is set up, then no table of the library is
/// made on the spot (that is the session's thread); it is made when the first files are
/// prepared, and then finds a file the library has by its path and one it has by its content.
#[test]
fn an_import_looks_the_library_up_when_it_prepares_files() {
let mut s = Session::new();
s.catalog = library(50);
let mut job = ImportJob::new(&mut s, ImportOptions { mode: ImportMode::Add, ..ImportOptions::default() }).unwrap();
assert!(job.by_path.is_empty() && job.by_hash.is_empty() && job.library.is_some(), "nothing is looked up yet");
let out = job.prepare_files(vec!["/pics/007/a.jpg".into()], &std::sync::atomic::AtomicBool::new(false));
assert!(job.library.is_none(), "looked up once");
assert_eq!((job.by_path.len(), job.by_hash.len()), (150, 100));
assert!(matches!(out.items.as_slice(), [PreparedItem::Duplicate { reason: "path", existing: Some(_), .. }]), "found by its path");
// what the options said when the import was set up decides, whatever is made of them after
let id = s.catalog.photos().next().unwrap().id;
s.catalog.apply(Op::SetDeleted { id, deleted: true }).unwrap();
let mut job = ImportJob::new(&mut s, ImportOptions { mode: ImportMode::Add, on_deleted: OnDeleted::Fresh, ..ImportOptions::default() }).unwrap();
job.opts.on_deleted = OnDeleted::Skip;
job.prepare_files(Vec::new(), &std::sync::atomic::AtomicBool::new(false));
assert_eq!(job.by_path.len(), 149, "the trashed photo was passed over, as set up");
}
/// Given a photo in Recently Deleted and an import that replaces such photos with a fresh
/// import, when the library is looked up, then that photo is no duplicate (as before the lookup
/// moved to the first files).
#[test]
fn a_trashed_photo_is_not_looked_up_when_imports_replace_it() {
let mut s = Session::new();
s.catalog = library(2);
// (a photo whose content is its own: nobody else in this library has it)
let id = s.catalog.photos().find(|p| p.file_name == "b.jpg").unwrap().id;
s.catalog.apply(Op::SetDeleted { id, deleted: true }).unwrap();
for (on_deleted, expected) in [(OnDeleted::Fresh, 5), (OnDeleted::Skip, 6)] {
let mut job = ImportJob::new(&mut s, ImportOptions { mode: ImportMode::Add, on_deleted, ..ImportOptions::default() }).unwrap();
job.prepare_files(Vec::new(), &std::sync::atomic::AtomicBool::new(false));
assert_eq!(job.by_path.len(), expected, "{on_deleted:?}");
let its_content = s.catalog.photo(id).and_then(|p| p.content_hash.clone()).unwrap();
assert_eq!(job.by_hash.contains_key(&its_content), on_deleted == OnDeleted::Skip, "{on_deleted:?}");
}
}
+3 -2
View File
@@ -223,11 +223,12 @@ pub(crate) fn rollback(p: &Placed) {
}
/// Remove a placed photo's source (its catalog record is committed and saved): only when the
/// destination is still there with the same size. Then its sidecars, unless another photo still
/// destination is still there with the same bytes. Then its sidecars, unless another photo still
/// uses a shared one. `Err` = the source was kept (why); `Ok` lists sidecars kept (path, why).
pub(crate) fn finish(p: &Placed) -> Result<Vec<(String, String)>, String> {
// compare the bytes now: either file may have changed since the copy was verified
let same = match (fs::metadata(&p.src), fs::metadata(&p.dst)) {
(Ok(s), Ok(d)) => s.is_file() && d.is_file() && s.len() == d.len(),
(Ok(s), Ok(d)) => s.is_file() && d.is_file() && same_bytes(&p.src, &p.dst).unwrap_or(false),
_ => false,
};
if !same {
+13
View File
@@ -29,8 +29,10 @@ pub mod demo;
pub mod denoise;
pub mod devices;
pub mod display;
mod errors;
pub mod export;
pub mod face_download;
pub mod face_tiles;
#[cfg(not(target_arch = "wasm32"))]
mod faces_index;
#[cfg(not(target_arch = "wasm32"))]
@@ -68,6 +70,7 @@ pub mod walk;
use std::sync::Arc;
pub use cmd::{CommandInfo, CommandSpec, command_specs, find_command};
pub use errors::{AiModel, ErrorKind};
pub use fonts::{CRAFT_FONTS, CraftFont};
use lightcraft_catalog::{Catalog, Filter, Op, PhotoId, Sort};
use lightcraft_develop::DevelopSettings;
@@ -98,6 +101,13 @@ pub enum EngineError {
/// to import with `library.importLightroom`, when one was found.
#[error("{} is a Lightroom Classic catalog, not a LightCraft library; import it into a LightCraft library with File ▸ Import Lightroom Catalog…{}", path.display(), catalog.as_ref().map_or(String::new(), |c| format!(" ({})", c.display())))]
LightroomCatalog { path: std::path::PathBuf, catalog: Option<std::path::PathBuf> },
/// The work was cancelled (by the person, an agent or a newer request): not a failure.
#[error("{0}")]
Cancelled(String),
/// The feature needs `model`, which is not installed (or still downloading): a frontend
/// offers its download (terms first) on [`ErrorKind::ModelMissing`], not on the message.
#[error("{msg}")]
ModelMissing { model: AiModel, msg: String },
#[error("{0}")]
Other(String),
}
@@ -1001,6 +1011,9 @@ mod tests_face_models;
#[cfg(not(target_arch = "wasm32"))]
mod tests_face_recognize;
#[cfg(test)]
#[cfg(not(target_arch = "wasm32"))]
mod tests_face_tiles;
#[cfg(test)]
mod tests_filesystem_history;
#[cfg(test)]
mod tests_folders;
+29 -7
View File
@@ -24,6 +24,12 @@ fn error(message: impl Into<String>) -> crate::EngineError {
crate::EngineError::BadParams { cmd: COMMAND.into(), msg: message.into() }
}
/// The import was cancelled ([`crate::ErrorKind::Cancelled`]: not a failure). The message is the
/// one it always had, so control-channel and MCP clients see the same `error` text.
fn cancelled() -> crate::EngineError {
crate::EngineError::Cancelled(error("Lightroom import cancelled").to_string())
}
fn stage_archive(archive_dir: Option<&Path>, data: &mut CatalogImport) -> crate::Result<Option<PathBuf>> {
let archive_path = archive_dir.map(|dir| crate::lightroom_archive::store_best_effort(dir, data).map_err(error)).transpose()?.flatten();
if archive_dir.is_some() && archive_path.is_none() {
@@ -232,13 +238,14 @@ impl LightroomJob {
/// Read/probe/stage all source data off the owner thread.
pub fn prepare(&mut self, cancel: &AtomicBool) -> crate::Result<PreparedLightroom> {
if cancel.load(Ordering::Relaxed) {
return Err(error("Lightroom import cancelled"));
return Err(cancelled());
}
let mut data =
lightroom_catalog::read_with_progress(&self.path, cancel, &self.progress.catalog_total, &self.progress.catalog_done).map_err(error)?;
// (reading stops with a message when `cancel` is set: the flag says it was cancelled)
let mut data = lightroom_catalog::read_with_progress(&self.path, cancel, &self.progress.catalog_total, &self.progress.catalog_done)
.map_err(|e| if cancel.load(Ordering::Relaxed) { cancelled() } else { error(e) })?;
lightroom_catalog::validate_for_job(&data).map_err(error)?;
if cancel.load(Ordering::Relaxed) {
return Err(error("Lightroom import cancelled"));
return Err(cancelled());
}
let paths: Vec<String> = data
.photos
@@ -259,7 +266,7 @@ impl LightroomJob {
let normal = self.import.prepare(&paths, cancel);
if cancel.load(Ordering::Relaxed) {
normal.rollback();
return Err(error("Lightroom import cancelled"));
return Err(cancelled());
}
let archive_path = stage_archive(self.archive_dir.as_deref(), &mut data)?;
let index_path = self.archive_dir.as_ref().map(|dir| dir.join("lightroom-index.json"));
@@ -282,7 +289,7 @@ impl LightroomJob {
pub fn commit_prepared(s: &mut crate::Session, mut prepared: PreparedLightroom) -> crate::Result<LightroomCompletion> {
if prepared.cancelled {
prepared.normal.rollback();
return Err(error("Lightroom import cancelled"));
return Err(cancelled());
}
if !prepared.token.matches(s) {
prepared.normal.rollback();
@@ -435,11 +442,26 @@ mod tests {
let mut session = crate::Session::new();
let before = session.catalog.to_snapshot();
let prepared = cancelled_prepared(&session);
assert!(commit_prepared(&mut session, prepared).is_err());
let Err(e) = commit_prepared(&mut session, prepared) else { panic!("a cancelled import commits nothing") };
assert_eq!(e.kind(), crate::ErrorKind::Cancelled, "{e}");
assert_eq!(session.catalog.to_snapshot(), before);
assert!(session.undo.is_empty());
}
/// Cancelling while the catalog is read is [`crate::ErrorKind::Cancelled`] whatever the message
/// says, and a failure that is not a cancellation is not one, even when its text says "cancelled".
#[test]
fn a_cancelled_read_is_cancelled_by_kind() {
let mut session = crate::Session::new();
let mut job = LightroomJob::new(&mut session, PathBuf::from("/nonexistent/cancelled.lrcat"), false).unwrap();
let Err(e) = job.prepare(&AtomicBool::new(true)) else { panic!("cancelled before it started") };
assert_eq!(e.kind(), crate::ErrorKind::Cancelled, "{e}");
// the `error` text clients see is unchanged
assert_eq!(e.to_string(), "invalid parameters for `library.importLightroom`: Lightroom import cancelled");
let Err(e) = job.prepare(&AtomicBool::new(false)) else { panic!("no such catalog") };
assert_ne!(e.kind(), crate::ErrorKind::Cancelled, "{e}");
}
#[test]
fn library_identity_rejects_late_completion_after_reopen() {
let mut session = crate::Session::new();
+31 -1
View File
@@ -395,8 +395,10 @@ fn validate_header_fields(header: &[u8], page_size: usize) -> Result<usize, Stri
if usable < 480 {
return Err("SQLite usable page size is below 480 bytes".into());
}
// 0 = no schema yet (file format §1.3.10): a catalog whose tables are all still in its WAL
// has that in the main file, and the WAL's own page 1 is checked again with the overlay
let schema_format = be_u32(header, 44)?;
if !(1..=4).contains(&schema_format) {
if !(0..=4).contains(&schema_format) {
return Err("unsupported SQLite schema format".into());
}
Ok(reserved)
@@ -941,6 +943,34 @@ mod tests {
assert_eq!(table_alias, Some(0));
}
/// A database file with no schema yet has schema format 0 in its header (file format
/// §1.3.10): a catalog created in WAL mode whose tables are all still in its `-wal` file (one
/// Lightroom has open and hasn't checkpointed) was refused as "unsupported SQLite schema
/// format" before the WAL was looked at. It opens, with no tables; other formats stay refused.
#[test]
fn schema_format_zero_is_an_empty_database() {
let header = |format: u32| {
let mut main = vec![0u8; 512];
main[..16].copy_from_slice(b"SQLite format 3\0");
main[16..18].copy_from_slice(&512u16.to_be_bytes());
main[18] = 2;
main[19] = 2;
main[21] = 64;
main[22] = 32;
main[23] = 32;
main[28..32].copy_from_slice(&1u32.to_be_bytes());
main[44..48].copy_from_slice(&format.to_be_bytes());
main[56..60].copy_from_slice(&1u32.to_be_bytes());
main[100] = 0x0d;
main[105..107].copy_from_slice(&512u16.to_be_bytes());
main
};
let db = Database::open_with_wal(header(0), &[]).unwrap();
assert!(db.live_tables().unwrap().is_empty());
assert!(Database::open_with_wal(header(4), &[]).is_ok());
assert_eq!(Database::open_with_wal(header(5), &[]).err().as_deref(), Some("unsupported SQLite schema format"));
}
#[test]
fn unsupported_unrelated_schema_entries_are_skipped() {
let mut main = vec![0u8; 2 * 512];
+68 -6
View File
@@ -67,6 +67,17 @@ pub const RENDER_CACHE_VERSION: u64 = 62;
/// Thumbnails render at one of these long edges (so window/cell size changes reuse the cache).
pub const THUMB_SIZES: [usize; 4] = [128, 256, 384, 512];
/// Face tiles are rendered at a multiple of this many pixels ([`crate::Session::face_job`]): the
/// tile's size follows the thumbnail slider and the screen's scale, and without steps every size
/// in between would render (and cache) every face again.
pub const FACE_EDGE_STEP: usize = 16;
/// The edge a face tile asked to be `edge` pixels across is rendered at: the next
/// [`FACE_EDGE_STEP`], 16..=512.
pub fn face_edge(edge: usize) -> usize {
edge.clamp(16, SourceLevel::Thumb.max_edge()).div_ceil(FACE_EDGE_STEP).saturating_mul(FACE_EDGE_STEP).min(SourceLevel::Thumb.max_edge())
}
pub fn thumb_bucket(long_edge: usize) -> usize {
THUMB_SIZES.iter().copied().find(|s| *s >= long_edge).unwrap_or(512)
}
@@ -741,6 +752,9 @@ pub struct RenderJob {
pub view_cache: Option<(Arc<PreviewCache>, Hash128)>,
/// Render for this monitor profile ([`Self::with_display`]).
pub display: Option<Arc<crate::display::Display>>,
/// A face tile that can be cut from the raw's embedded preview, tried before `source`
/// ([`crate::Session::face_job`]; `None` for every other job).
pub preview_cut: Option<crate::face_tiles::PreviewCut>,
}
pub struct RenderResult {
@@ -820,6 +834,7 @@ impl RenderJob {
self.settings = Arc::new(s);
self.cache = None;
self.view_cache = None;
self.preview_cut = None;
self
}
@@ -887,6 +902,26 @@ impl RenderJob {
display: None,
};
}
// a face tile of an unedited raw: cut from the camera's embedded preview when it is big enough
// (the sensor data is never read for it), else rendered from `source` below
if let Some(image) = self.preview_cut.as_ref().and_then(|c| c.cut()) {
if let Some((cache, key)) = &self.cache {
cache.put_at(self.cache_generation, *key, Arc::new(image.clone()));
}
let histogram = Histogram::of_srgb8(&image);
return RenderResult {
request_id: self.request_id,
source_key: self.source_key,
photo: self.photo,
level: self.level,
key: self.key,
rendered: Ok(Rendered { image, histogram, deep: None }),
gpu_fallback: None,
loaded: None,
quick: None,
display: None,
};
}
let was_loaded = matches!(self.source, SourceRef::Loaded(_));
// The app is closing (issue #620): nobody is left to show this, and quitting waits for the
// jobs that are running. Without this, one that lost the GPU mid-way would render on the CPU.
@@ -1192,6 +1227,7 @@ impl crate::Session {
stages: None,
view_cache: None,
display: None,
preview_cut: None,
})
}
@@ -1243,13 +1279,19 @@ impl crate::Session {
stages: None,
view_cache: None,
display: None,
preview_cut: None,
})
}
/// A square close-up of `face` (normalized, in the photo's upright frame) for a People card: the
/// photo's own look with its crop replaced by a square around the face (room for hair and chin),
/// `edge` pixels across (≤ 512). The source level is the smallest that keeps the face sharp, up
/// to the preview. Cached like [`Self::variant_job`], and `key` likewise follows the content.
/// at least `edge` pixels across (rounded up to [`FACE_EDGE_STEP`], ≤ 512, so a tile size
/// between two steps reuses the cached one). An unedited raw's tile is cut from the camera's
/// embedded preview when that holds the square at [`crate::face_tiles::SOFT_LIMIT`] of the
/// size asked or more ([`crate::face_tiles`]); else, and for every other photo, the source
/// level is the smallest that keeps the face sharp, up to the preview. Cached (memory and the
/// library's thumbnail cache on disk) like [`Self::variant_job`], and `key` likewise follows
/// the content: a later launch shows the tile without reading the original.
pub fn face_job(&mut self, id: PhotoId, face: lightcraft_geom::Rect, edge: usize) -> Option<RenderJob> {
let p = self.catalog.photo(id)?.clone();
// `face` is on the upright (EXIF-oriented) photo; the crop below is in the frame after the
@@ -1267,12 +1309,13 @@ impl crate::Session {
let rect = lightcraft_geom::Rect { x0: x0 / w, y0: y0 / h, x1: (x0 + side) / w, y1: (y0 + side) / h };
let mut settings = (*p.develop).clone();
settings.crop = lightcraft_develop::Crop { geometry: lightcraft_geom::CropGeometry { rect, angle: 0.0 }, ..Default::default() };
let edge = edge.clamp(16, SourceLevel::Thumb.max_edge());
let edge = face_edge(edge);
// pixels along the long edge that leave `edge` across the crop
let needed = (edge as f64 * w.max(h) / side).ceil().min(SourceLevel::Preview.max_edge() as f64);
let level = SourceLevel::for_size(needed as usize);
let source = self.media.source_ref(&p, level);
let ck = Self::variant_key(&p, &settings, edge);
let preview_cut = self.face_preview_cut(&p, rect, edge, needed as usize);
Some(RenderJob {
request_id: 0,
cache_generation: self.media.rendered.generation(),
@@ -1289,9 +1332,26 @@ impl crate::Session {
stages: None,
view_cache: None,
display: None,
preview_cut,
})
}
/// How a face tile of `p` can come from its embedded preview ([`crate::face_tiles`]): only an
/// unedited raw (whose grid thumbnail is that preview too) not known to be offline, upright as
/// imported, with a preview loader installed. `rect` is the square around the face, `edge` the
/// tile, `needed` the source's long edge the develop render would ask for.
fn face_preview_cut(&self, p: &Photo, rect: lightcraft_geom::Rect, edge: usize, needed: usize) -> Option<crate::face_tiles::PreviewCut> {
let (path, loader) = self.embedded_of(p)?;
if p.develop.orientation != lightcraft_geom::Orientation::Normal || self.media.availability.is_offline(&path) || p.width == 0 || p.height == 0
{
return None;
}
let want = needed.max(edge).max(1);
let min = (want as f64 * crate::face_tiles::SOFT_LIMIT).ceil() as usize;
let aspect = f64::from(p.width) / f64::from(p.height);
Some(crate::face_tiles::PreviewCut { path, loader, want, min, rect, aspect, edge })
}
/// Size-independent cache key of a photo's view render (loupe) for its current settings.
fn view_key(p: &Photo, apply_crop: bool) -> Hash128 {
Hasher128::new()
@@ -1427,8 +1487,10 @@ impl crate::Session {
/// A first grid thumbnail for an unedited raw: the cached render for `job` (the real
/// thumbnail job, from [`Self::thumb_job`]) if there is one — then the result carries
/// [`QuickSource::Cached`] and `job.key` and is final — else its embedded preview. `None` for
/// other photos (render them directly).
/// [`QuickSource::Cached`] and `job.key` and is final — else its embedded preview, else `job`
/// itself: the grid queues no render behind an unedited raw's quick thumbnail
/// ([`Self::embedded_thumb_is_final`]), so a preview that can't be read must not leave the
/// cell empty (issue #1448). `None` for other photos (render them directly).
pub fn quick_thumb_job(&mut self, job: &RenderJob) -> Option<QuickJob> {
let p = self.catalog.photo(job.photo)?.clone();
let (path, l) = self.embedded_of(&p)?;
@@ -1439,7 +1501,7 @@ impl crate::Session {
key: job.key,
cached: job.cache.clone().into_iter().collect(),
embedded: Some((path, l, edge)),
small: None,
small: Some(Box::new(job.clone())),
})
}
+1 -1
View File
@@ -34,7 +34,7 @@ pub fn default_budget() -> usize {
}
/// Physical memory (bytes), when the platform tells us without native calls.
fn total_ram() -> Option<usize> {
pub(crate) fn total_ram() -> Option<usize> {
#[cfg(target_os = "linux")]
{
let s = std::fs::read_to_string("/proc/meminfo").ok()?;
+6 -2
View File
@@ -130,11 +130,14 @@ impl MergeJob {
return Err("cancelled".into());
}
let sub = |a: f32, b: f32| move |f: f32, s: &str| progress(a + (b - a) * f, s);
// the reference frame's lens corrections (HDR); panoramas carry none
let mut lens = Vec::new();
let (image, color, orientation, metadata, baseline, samples, crop, mut info, ghost) = match &self.kind {
MergeKind::Hdr { hdr } => {
let frames = self.load(false, progress, (0.0, 0.3))?;
let p = sub(0.3, 0.9);
let r = lightcraft_merge::merge_hdr(frames, hdr, &p).map_err(err)?;
let mut r = lightcraft_merge::merge_hdr(frames, hdr, &p).map_err(err)?;
lens = std::mem::take(&mut r.lens);
let info = json!({
"reference": self.sources[r.reference].0.0,
"ev": r.ev, "evExif": r.ev_exif,
@@ -183,6 +186,7 @@ impl MergeJob {
orientation: r.orientation,
metadata: r.metadata,
baseline_exposure: r.baseline_exposure,
lens: Vec::new(),
};
f.orient();
merged.push(f);
@@ -214,7 +218,7 @@ impl MergeJob {
metadata.caption = Some(format!("Merged from {}", names.join(", ")));
}
metadata.software = Some("LightCraft Photo Merge".into());
let dng = lightcraft_merge::write_linear_dng(&image, &color, orientation, &metadata, baseline, samples).map_err(err)?;
let dng = lightcraft_merge::write_linear_dng(&image, &color, orientation, &metadata, baseline, samples, &lens).map_err(err)?;
info["width"] = json!(image.width);
info["height"] = json!(image.height);
let preview = match self.preview {
+64 -13
View File
@@ -28,28 +28,45 @@ pub const PRESET_EXTS: &[&str] = &["xmp", "lrtemplate", "zip", "dng", "lmp", "mp
// ------------------------------------------------------------------------------------- zip
/// The most a preset zip entry may hold unpacked (a "DNG preset" photo is the largest kind, tens
/// of MB): its stated size is the file's word, so a small archive could otherwise ask for 8 GB.
pub const MAX_ZIP_ENTRY: usize = 256 << 20;
/// The most a whole preset zip may hold unpacked.
pub const MAX_ZIP_TOTAL: usize = 1 << 30;
/// The files of a zip archive: (path inside the archive, bytes). Stored and deflated entries;
/// directories, macOS resource forks and hidden files are skipped.
/// directories, macOS resource forks and hidden files are skipped. An entry that unpacks past
/// [`MAX_ZIP_ENTRY`], or an archive past [`MAX_ZIP_TOTAL`], is an error (a zip bomb would make
/// the allocation fail, which ends the app).
pub fn read_zip(bytes: &[u8]) -> Result<Vec<(String, Vec<u8>)>, String> {
let u16_at = |o: usize| bytes.get(o..o + 2).map(|b| u16::from_le_bytes([b[0], b[1]]) as usize);
let u32_at = |o: usize| bytes.get(o..o + 4).map(|b| u32::from_le_bytes([b[0], b[1], b[2], b[3]]) as usize);
read_zip_limited(bytes, MAX_ZIP_ENTRY, MAX_ZIP_TOTAL)
}
fn read_zip_limited(bytes: &[u8], max_entry: usize, max_total: usize) -> Result<Vec<(String, Vec<u8>)>, String> {
let u16_at = |o: usize| bytes.get(o..o.checked_add(2)?).map(|b| u16::from_le_bytes([b[0], b[1]]) as usize);
let u32_at = |o: usize| bytes.get(o..o.checked_add(4)?).map(|b| u32::from_le_bytes([b[0], b[1], b[2], b[3]]) as usize);
// offsets from the file, added up without overflowing (a 32-bit build reads them too)
let add = |a: usize, b: usize| a.checked_add(b).ok_or("bad zip");
// end of central directory: the last "PK\x05\x06" within the trailing 64 KiB + 22 bytes
let tail = bytes.len().saturating_sub(65_557);
let eocd = (tail..bytes.len().saturating_sub(21)).rev().find(|&i| bytes[i..].starts_with(b"PK\x05\x06")).ok_or("not a zip file")?;
let count = u16_at(eocd + 10).ok_or("bad zip")?;
let mut at = u32_at(eocd + 16).ok_or("bad zip")?;
let mut out = Vec::new();
let mut total = 0usize;
for _ in 0..count {
if !bytes.get(at..).is_some_and(|b| b.starts_with(b"PK\x01\x02")) {
return Err("bad zip central directory".into());
}
let method = u16_at(at + 10).ok_or("bad zip")?;
let csize = u32_at(at + 20).ok_or("bad zip")?;
let usize_ = u32_at(at + 24).ok_or("bad zip")?;
let (nlen, xlen, clen) = (u16_at(at + 28).ok_or("bad zip")?, u16_at(at + 30).ok_or("bad zip")?, u16_at(at + 32).ok_or("bad zip")?);
let local = u32_at(at + 42).ok_or("bad zip")?;
let name = String::from_utf8_lossy(bytes.get(at + 46..at + 46 + nlen).ok_or("bad zip")?).replace('\\', "/");
at += 46 + nlen + xlen + clen;
let method = u16_at(add(at, 10)?).ok_or("bad zip")?;
let csize = u32_at(add(at, 20)?).ok_or("bad zip")?;
let usize_ = u32_at(add(at, 24)?).ok_or("bad zip")?;
let (nlen, xlen, clen) =
(u16_at(add(at, 28)?).ok_or("bad zip")?, u16_at(add(at, 30)?).ok_or("bad zip")?, u16_at(add(at, 32)?).ok_or("bad zip")?);
let local = u32_at(add(at, 42)?).ok_or("bad zip")?;
let name_at = add(at, 46)?;
let name = String::from_utf8_lossy(bytes.get(name_at..add(name_at, nlen)?).ok_or("bad zip")?).replace('\\', "/");
at = add(add(add(name_at, nlen)?, xlen)?, clen)?;
let skip = name.ends_with('/') || name.split('/').any(|p| p.starts_with('.') || p == "__MACOSX");
if skip {
continue;
@@ -57,13 +74,20 @@ pub fn read_zip(bytes: &[u8]) -> Result<Vec<(String, Vec<u8>)>, String> {
if !bytes.get(local..).is_some_and(|b| b.starts_with(b"PK\x03\x04")) {
return Err(format!("{name}: bad local header"));
}
let data_at = local + 30 + u16_at(local + 26).ok_or("bad zip")? + u16_at(local + 28).ok_or("bad zip")?;
let data = bytes.get(data_at..data_at + csize).ok_or_else(|| format!("{name}: truncated"))?;
let data_at = add(add(add(local, 30)?, u16_at(add(local, 26)?).ok_or("bad zip")?)?, u16_at(add(local, 28)?).ok_or("bad zip")?)?;
let data = bytes.get(data_at..add(data_at, csize)?).ok_or_else(|| format!("{name}: truncated"))?;
let left = max_total.saturating_sub(total);
let limit = max_entry.min(left);
let too_big = || format!("{name}: unpacks to more than {} MB, too much for a preset", limit >> 20);
let content = match method {
0 if data.len() > limit => return Err(too_big()),
0 => data.to_vec(),
8 => miniz_oxide::inflate::decompress_to_vec_with_limit(data, usize_.max(1) * 2 + 1024).map_err(|e| format!("{name}: {e:?}"))?,
8 => miniz_oxide::inflate::decompress_to_vec_with_limit(data, usize_.max(1).saturating_mul(2).saturating_add(1024).min(limit)).map_err(
|e| if matches!(e.status, miniz_oxide::inflate::TINFLStatus::HasMoreOutput) { too_big() } else { format!("{name}: {e:?}") },
)?,
m => return Err(format!("{name}: unsupported compression method {m}")),
};
total = total.saturating_add(content.len());
out.push((name, content));
}
Ok(out)
@@ -695,6 +719,33 @@ text]], z = ZSTR "loc" }"#,
assert!(read_presets("plain.jpg", b"\xff\xd8 no xmp", None).is_err());
}
#[test]
fn zip_bombs_are_refused() {
// an entry unpacking past the limit (3 MB of zeros pack into a few kB): refused, named
let zeros = vec![0u8; 3 << 20];
let z = zip(&[("Pack/huge.xmp", &zeros)], true);
assert!(z.len() < 64 << 10, "a small archive: {} bytes", z.len());
let e = read_zip_limited(&z, 1 << 20, 1 << 30).unwrap_err();
assert!(e.contains("Pack/huge.xmp") && e.contains("more than 1 MB"), "{e}");
// the whole archive counts too
let two = zip(&[("a.xmp", &zeros[..1 << 20]), ("b.xmp", &zeros[..1 << 20])], true);
assert!(read_zip_limited(&two, 2 << 20, (3 << 20) / 2).is_err());
assert_eq!(read_zip_limited(&two, 2 << 20, 3 << 20).unwrap().len(), 2);
// and stored entries
let stored = zip(&[("s.xmp", &zeros[..2 << 20])], false);
assert!(read_zip_limited(&stored, 1 << 20, 1 << 30).is_err());
// the size an entry states (here 4 GB) no longer sets how much may be unpacked
let mut lying = zip(&[("a.xmp", b"<x:xmpmeta/>")], true);
let cd = lying.windows(4).position(|w| w == b"PK\x01\x02").unwrap();
lying[cd + 24..cd + 28].copy_from_slice(&u32::MAX.to_le_bytes());
assert_eq!(read_zip(&lying).unwrap()[0].1, b"<x:xmpmeta/>");
// offsets near the top of a u32 are an error, not an overflow
let mut far = zip(&[("a.xmp", b"x")], false);
let cd = far.windows(4).position(|w| w == b"PK\x01\x02").unwrap();
far[cd + 42..cd + 46].copy_from_slice(&u32::MAX.to_le_bytes());
assert!(read_zip(&far).is_err());
}
#[test]
fn zip_bundles_with_folders_as_groups() {
for deflate in [false, true] {
+11 -2
View File
@@ -20,6 +20,8 @@ pub struct Status {
pub file: String,
/// The last download's error (cleared when a new one starts).
pub error: Option<String>,
/// The last download was cancelled (its `error` says so in words): decide on this, not on the text.
pub cancelled: bool,
/// The last download finished and the files are in place.
pub finished: bool,
}
@@ -28,6 +30,7 @@ pub struct Status {
struct Text {
file: String,
error: Option<String>,
cancelled: bool,
}
#[derive(Default)]
@@ -68,15 +71,17 @@ impl Downloader {
{
last.file.clone_from(&t.file);
last.error.clone_from(&t.error);
last.cancelled = t.cancelled;
}
// (only the session's thread locks `last`)
let (file, error) = self.last.try_lock().map(|l| (l.file.clone(), l.error.clone())).unwrap_or_default();
let (file, error, cancelled) = self.last.try_lock().map(|l| (l.file.clone(), l.error.clone(), l.cancelled)).unwrap_or_default();
Status {
running: s.running.load(Ordering::SeqCst),
done: s.done.load(Ordering::SeqCst),
total: s.total.load(Ordering::SeqCst),
file,
error,
cancelled,
finished: s.finished.load(Ordering::SeqCst),
}
}
@@ -135,6 +140,7 @@ impl Downloader {
};
let r =
std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| fetch::download(files, &mirrors, &dir, &opts, &s.cancel, &mut on_progress)));
let cancelled = matches!(r, Ok(Err(fetch::DownloadError::Cancelled)));
let error = match r {
Ok(Ok(())) => {
s.finished.store(true, Ordering::SeqCst);
@@ -147,7 +153,10 @@ impl Downloader {
if let Some(e) = &error {
log::warn!("SAM 3 download: {e}");
}
lock(&s.text).error = error;
let mut text = lock(&s.text);
text.cancelled = cancelled;
text.error = error;
drop(text);
drop(guard);
});
match spawned {
+31 -12
View File
@@ -35,7 +35,7 @@ pub const MODEL_BYTES: u64 = 3_439_938_512;
/// The model's licence (not LightCraft's): shown before downloading.
pub const LICENSE_NAME: &str = "SAM License (Meta)";
pub const LICENSE_URL: &str = "https://github.com/facebookresearch/sam3/blob/main/LICENSE";
/// How errors about a missing model start (the UI offers the download on it).
/// How errors about a missing model start (frontends decide on [`crate::ErrorKind::ModelMissing`], not on this text).
pub const NOT_INSTALLED: &str = "The SAM 3 model is not installed";
/// Longest a waiting command (CLI, MCP) waits for the model (loading on a CPU can take minutes).
#[cfg_attr(not(feature = "sam"), allow(dead_code))]
@@ -111,6 +111,8 @@ pub struct DownloadStatus {
pub total: u64,
pub file: String,
pub error: Option<String>,
/// The last download was cancelled (it resumes next time): decide on this, not on `error`'s text.
pub cancelled: bool,
pub finished: bool,
}
@@ -215,7 +217,15 @@ impl Segmenter {
#[cfg(feature = "sam")]
{
let s = self.download.status();
DownloadStatus { running: s.running, done: s.done, total: s.total, file: s.file, error: s.error, finished: s.finished }
DownloadStatus {
running: s.running,
done: s.done,
total: s.total,
file: s.file,
error: s.error,
cancelled: s.cancelled,
finished: s.finished,
}
}
#[cfg(not(feature = "sam"))]
DownloadStatus::default()
@@ -234,27 +244,36 @@ impl Segmenter {
/// The model folder, if the model is there; otherwise an error that says what to do.
pub fn model_dir(&self) -> Result<PathBuf, String> {
self.require_model().map_err(|e| e.to_string())
}
/// [`Segmenter::model_dir`] for commands: a model that is not installed (or still downloading) is
/// [`crate::EngineError::ModelMissing`], so a frontend offers the download without reading the message.
pub fn require_model(&self) -> crate::Result<PathBuf> {
let other = |why: &str| crate::EngineError::Other(why.into());
if !Self::AVAILABLE {
return Err("AI masks are not available in this build".into());
return Err(other("AI masks are not available in this build"));
}
#[cfg(feature = "sam")]
if lightcraft_segment::remote::configured().is_some() {
return Ok(PathBuf::new());
}
let dir = self.dir.clone().ok_or("no folder is set for the SAM 3 model")?;
let dir = self.dir.clone().ok_or_else(|| other("no folder is set for the SAM 3 model"))?;
if self.installed() {
return Ok(dir);
}
let d = self.download_status();
if d.running {
let msg = if d.running {
let pct = d.done.saturating_mul(100).checked_div(d.total).unwrap_or(0);
return Err(format!("{NOT_INSTALLED} yet: it is downloading ({pct} %)."));
}
Err(format!(
"{NOT_INSTALLED}. Download it (about {:.1} GB, {LICENSE_NAME}) when LightCraft offers it, with `segment.model.download {{\"acknowledged\": true}}`, or put model.safetensors, vocab.json and merges.txt in {}.",
MODEL_BYTES as f64 / 1e9,
dir.display()
))
format!("{NOT_INSTALLED} yet: it is downloading ({pct} %).")
} else {
format!(
"{NOT_INSTALLED}. Download it (about {:.1} GB, {LICENSE_NAME}) when LightCraft offers it, with `segment.model.download {{\"acknowledged\": true}}`, or put model.safetensors, vocab.json and merges.txt in {}.",
MODEL_BYTES as f64 / 1e9,
dir.display()
)
};
Err(crate::EngineError::ModelMissing { model: crate::AiModel::Sam3, msg })
}
/// Start downloading the model (on a background thread; see `segment.model.download`).
+86 -13
View File
@@ -450,6 +450,33 @@ fn stem_key(original: &str) -> String {
sidecar_path(original, SidecarNaming::Stem).to_string_lossy().to_lowercase()
}
/// Where a plain path is cut to make its [`stem_key`]: at the dot of its extension, which is
/// what `with_extension` replaces. A plain path is ASCII and ends in its file's name with an
/// extension, which is what a library's paths are; `None` for any other shape.
fn plain_cut(original: &str) -> Option<usize> {
let path = Path::new(original);
let (name, stem, ext) = (path.file_name()?.to_str()?, path.file_stem()?.to_str()?, path.extension()?.to_str()?);
// (the path ends in its file's name, as it stands: no `/.` or separator after it; and the
// name has a stem and an extension of its own, which `..jpg` and `a.` have not)
let plain = original.is_ascii() && original.ends_with(name) && !ext.is_empty() && !stem.is_empty() && stem != "." && stem != "..";
let cut = original.len().checked_sub(ext.len().saturating_add(1))?;
(plain && original.as_bytes().get(cut) == Some(&b'.')).then_some(cut)
}
/// [`stem_key`] written into `key`, which is reused: nothing is allocated for a plain path
/// ([`plain_cut`]). Any other shape goes the long way.
fn stem_key_into(key: &mut String, original: &str) {
key.clear();
match plain_cut(original) {
Some(cut) => {
key.push_str(original.get(..cut).unwrap_or_default());
key.make_ascii_lowercase();
key.push_str(".xmp");
}
None => key.push_str(&stem_key(original)),
}
}
fn file_path(p: &Photo) -> Option<&str> {
match &p.source {
Source::File { path } => Some(path),
@@ -490,8 +517,11 @@ pub(crate) fn op_photos(op: &Op, out: &mut Vec<PhotoId>) {
pub(crate) struct StemOwners(HashMap<String, PhotoId>);
impl StemOwners {
/// The owners of every shared stem in the library, each file's key made the long way: what
/// [`StemOwners::among`] must agree with for the photos it is asked about (this is how the
/// owners were worked out before, for every write).
#[cfg(test)]
pub(crate) fn of(cat: &lightcraft_catalog::Catalog) -> StemOwners {
// per stem: (raw first, then file name, then id) of the best candidate; and how many
let mut best: HashMap<String, ((bool, String, u64), PhotoId, usize)> = HashMap::new();
for p in cat.photos().filter(|p| p.copy_of.is_none()) {
let Some(path) = file_path(p) else { continue };
@@ -505,6 +535,39 @@ impl StemOwners {
StemOwners(best.into_iter().filter(|(_, v)| v.2 > 1).map(|(k, v)| (k, v.1)).collect())
}
/// The owners of the stems photos `ids` map to: all that naming those photos needs. The
/// library is passed over once, and only files of those stems are looked at closer, so a
/// sidecar written for one photo does not cost a table of the whole library.
pub(crate) fn among(cat: &lightcraft_catalog::Catalog, ids: &[PhotoId]) -> StemOwners {
let wanted: std::collections::HashSet<String> =
ids.iter().filter_map(|id| cat.photo(*id)).filter_map(|p| file_path(p)).map(stem_key).collect();
if wanted.is_empty() {
return StemOwners(HashMap::new());
}
Self::where_wanted(cat, &wanted)
}
/// The owners of the shared stems in `wanted`.
fn where_wanted(cat: &lightcraft_catalog::Catalog, wanted: &std::collections::HashSet<String>) -> StemOwners {
// per stem: (raw first, then file name, then id) of the best candidate; and how many
let mut best: HashMap<String, ((bool, String, u64), PhotoId, usize)> = HashMap::new();
let mut key = String::new();
for p in cat.photos().filter(|p| p.copy_of.is_none()) {
let Some(path) = file_path(p) else { continue };
stem_key_into(&mut key, path);
if !wanted.contains(key.as_str()) {
continue;
}
let rank = (p.kind != MediaKind::Raw, path.to_lowercase(), p.id.0);
let e = best.entry(key.clone()).or_insert_with(|| (rank.clone(), p.id, 0));
e.2 += 1;
if rank < e.0 {
(e.0, e.1) = (rank, p.id);
}
}
StemOwners(best.into_iter().filter(|(_, v)| v.2 > 1).map(|(k, v)| (k, v.1)).collect())
}
/// `naming` for photo `p`, except that a stem sidecar another file owns becomes Full.
fn naming(&self, p: &Photo, naming: SidecarNaming) -> SidecarNaming {
match (naming, file_path(p)) {
@@ -514,33 +577,39 @@ impl StemOwners {
}
}
/// How the sidecar of each of photos `ids` is named in library `cat`, whose naming is `naming`
/// ([`Session::sidecar_namings`], for a thread that has the library and not the session).
pub(crate) fn namings_in(cat: &lightcraft_catalog::Catalog, naming: SidecarNaming, ids: &[PhotoId]) -> Vec<SidecarNaming> {
let owners = (naming == SidecarNaming::Stem).then(|| StemOwners::among(cat, ids));
ids.iter()
.map(|id| match (cat.photo(*id), &owners) {
(Some(p), Some(o)) => o.naming(p, naming),
_ => naming,
})
.collect()
}
impl Session {
/// The sidecar naming for photo `id`: the library's, except under Stem naming when another
/// catalogued file shares the stem (`IMG_0001.CR3` + `IMG_0001.JPG`) and owns the stem
/// sidecar — a raw first, else the first by file name. The others use Full naming
/// (`IMG_0001.JPG.xmp`), so their metadata never overwrites each other.
/// [`Session::sidecar_naming`] for many photos, the catalog's sidecar owners worked out once
/// (per photo that is a pass over the whole catalog).
/// [`Session::sidecar_naming`] for many photos, their stems' owners worked out once (per
/// photo that is a pass over the whole catalog).
pub fn sidecar_namings(&self, ids: &[PhotoId]) -> Vec<SidecarNaming> {
let owners = (self.xmp.naming == SidecarNaming::Stem).then(|| StemOwners::of(&self.catalog));
ids.iter()
.map(|id| match (self.catalog.photo(*id), &owners) {
(Some(p), Some(o)) => o.naming(p, self.xmp.naming),
_ => self.xmp.naming,
})
.collect()
namings_in(&self.catalog, self.xmp.naming, ids)
}
pub fn sidecar_naming(&self, id: PhotoId) -> SidecarNaming {
match self.catalog.photo(id) {
Some(p) if self.xmp.naming == SidecarNaming::Stem => StemOwners::of(&self.catalog).naming(p, self.xmp.naming),
Some(p) if self.xmp.naming == SidecarNaming::Stem => StemOwners::among(&self.catalog, &[id]).naming(p, self.xmp.naming),
_ => self.xmp.naming,
}
}
/// Write the photo's XMP sidecar, merged into an existing one (see the module docs).
pub fn save_sidecar(&mut self, id: PhotoId) -> Result<SidecarSaved> {
self.save_sidecar_with(id, &StemOwners::of(&self.catalog))
self.save_sidecar_with(id, &StemOwners::among(&self.catalog, &[id]))
}
pub(crate) fn save_sidecar_with(&mut self, id: PhotoId, owners: &StemOwners) -> Result<SidecarSaved> {
@@ -642,7 +711,7 @@ impl Session {
if ids.is_empty() {
return;
}
let owners = StemOwners::of(&self.catalog);
let owners = StemOwners::among(&self.catalog, &ids);
for id in ids {
if self.catalog.photo(id).is_some_and(|p| file_path(p).is_some() && p.copy_of.is_none())
&& let Err(e) = self.save_sidecar_with(id, &owners)
@@ -764,3 +833,7 @@ mod tests_repair_companions;
#[cfg(test)]
#[path = "sidecar/tests_repair_provenance.rs"]
mod tests_repair_provenance;
#[cfg(test)]
#[path = "sidecar/tests_stem_owners.rs"]
mod tests_stem_owners;
@@ -0,0 +1,163 @@
//! Who owns a stem sidecar, worked out for the photos asked about only (`StemOwners::among`).
//!
//! A library can hold millions of photos; a sidecar that is written for one of them, or a folder
//! that is synchronized, must not cost a table of them all.
use super::*;
use lightcraft_catalog::{Catalog, Op, Photo, Source};
/// Paths that try the key: case in folders, names and endings, no ending, a dot file, endings
/// inside folder names, non-ASCII letters (one that lower-cases to ASCII: the Kelvin sign).
const PATHS: [&str; 26] = [
"/pics/trip/IMG_1.CR3",
"/pics/trip/IMG_1.JPG",
"/pics/trip/img_1.jpeg",
"/Pics/Trip/IMG_1.tif",
"/pics/trip/IMG_2.JPG",
"/pics/trip/IMG_3.CR3",
"/pics/trip/IMG_3.cr3.JPG",
"/pics/trip/IMG_3.cr3",
"/pics/other/IMG_1.JPG",
"/pics/other/IMG_1",
"/pics/other/.hidden",
"/pics/other/.hidden.jpg",
"/pics/v1.2/a.b.JPG",
"/pics/v1.2/a.b.cr3",
"/pics/v1.2/a",
"/pics/straße/Größe.JPG",
"/pics/STRASSE/GRÖSSE.cr3",
"/pics/straße/größe.CR3",
"/pics/kelvin/\u{212a}1.JPG",
"/pics/kelvin/k1.cr3",
"/pics/ΟΔΟΣ/ΟΔΟΣ.JPG",
"/pics/οδος/οδος.CR3",
// (names that are all dots before their ending: the ending is no extension to replace)
"/pics/dots/..cr3",
"/pics/dots/..jpg",
"/pics/dots/a./.",
"/pics/dots/a.jpg",
];
fn library() -> (Catalog, Vec<PhotoId>) {
let mut c = Catalog::new();
let mut ids = Vec::new();
for path in PATHS {
let id = c.alloc_photo_id();
let name = path.rsplit('/').next().unwrap_or(path);
let mut p = Photo::new(id, Source::File { path: path.to_string() }, name, "X", 6, 4, "2026-01-01");
if path.to_lowercase().ends_with("cr3") {
p.kind = MediaKind::Raw;
}
c.apply(Op::AddPhoto { photo: Box::new(p) }).unwrap();
ids.push(id);
}
// a virtual copy of the first (it owns nothing) and a photo that is no file
let copy = c.alloc_photo_id();
let mut p = Photo::new(copy, Source::File { path: PATHS[0].to_string() }, "IMG_1.CR3", "X", 6, 4, "2026-01-01");
p.copy_of = Some(ids[0]);
c.apply(Op::AddPhoto { photo: Box::new(p) }).unwrap();
ids.push(copy);
let demo = c.alloc_photo_id();
c.apply(Op::AddPhoto { photo: Box::new(Photo::new(demo, Source::Demo { scene: 1 }, "demo", "X", 6, 4, "2026-01-01")) }).unwrap();
ids.push(demo);
(c, ids)
}
fn namings(owners: &StemOwners, c: &Catalog, ids: &[PhotoId]) -> Vec<SidecarNaming> {
ids.iter().map(|id| owners.naming(c.photo(*id).unwrap(), SidecarNaming::Stem)).collect()
}
/// Given a library where files share stems in every way a name can, when the owners are worked
/// out for some of its photos only, then each of those photos is named exactly as the table of
/// the whole library names it: whichever photos are asked about, alone or together.
#[test]
fn owners_among_some_photos_name_them_as_the_whole_library_does() {
let (c, ids) = library();
let whole = StemOwners::of(&c);
let expected = namings(&whole, &c, &ids);
assert!(expected.contains(&SidecarNaming::Full) && expected.contains(&SidecarNaming::Stem), "both cases are there: {expected:?}");
// all at once
assert_eq!(namings(&StemOwners::among(&c, &ids), &c, &ids), expected);
// one at a time, and every pair
for (i, a) in ids.iter().enumerate() {
assert_eq!(namings(&StemOwners::among(&c, &[*a]), &c, &[*a]), [expected[i]], "{:?}", PATHS.get(i));
for (j, b) in ids.iter().enumerate() {
assert_eq!(namings(&StemOwners::among(&c, &[*a, *b]), &c, &[*a, *b]), [expected[i], expected[j]]);
}
}
// an id that is not in the library is passed over
assert_eq!(StemOwners::among(&c, &[PhotoId(9_999)]).0.len(), 0);
}
/// Given a library of many folders, when the owners are worked out for the photos of one, then
/// the table holds that folder's shared stems and nothing of the others: what it costs follows
/// the photos asked about, not the library.
#[test]
fn owners_among_some_photos_hold_nothing_of_the_rest() {
let mut c = Catalog::new();
let mut first = Vec::new();
for folder in 0..200 {
for name in ["IMG_1.CR3", "IMG_1.JPG", "IMG_2.JPG"] {
let id = c.alloc_photo_id();
let mut p = Photo::new(id, Source::File { path: format!("/pics/{folder:03}/{name}") }, name, "X", 6, 4, "2026-01-01");
if name.ends_with("CR3") {
p.kind = MediaKind::Raw;
}
c.apply(Op::AddPhoto { photo: Box::new(p) }).unwrap();
if folder == 0 {
first.push(id);
}
}
}
assert_eq!(StemOwners::of(&c).0.len(), 200, "one shared stem a folder");
let some = StemOwners::among(&c, &first);
assert_eq!(some.0.len(), 1, "the one shared stem of the folder asked about");
assert_eq!(namings(&some, &c, &first), [SidecarNaming::Stem, SidecarNaming::Full, SidecarNaming::Stem]);
}
/// The key written into a buffer (no allocation for a plain path) is the key: for every path
/// above, for shapes a path from a file can have, and for every string of up to six of the
/// characters that decide a key (a letter in each case, a dot, both separators, a letter that is
/// not ASCII).
#[test]
fn the_key_in_a_buffer_is_the_key() {
let odd = [
"",
"/",
"a",
"a.",
".a",
"..a",
"...a",
"a.b",
"a/b.c/",
"a/b.c//",
"a/.b",
"a/..b",
"a/b.",
"a/b..c",
"a./.",
"a\\b.c",
"C:\\x\\y.JPG",
"a/b.c/.",
"a/b.c/..",
"x.JPG ",
"É.JPG",
];
let mut buf = String::new();
for path in PATHS.iter().chain(&odd) {
stem_key_into(&mut buf, path);
assert_eq!(buf, stem_key(path), "{path:?}");
}
let letters = ['a', 'B', '.', '/', '\\', 'é'];
let mut tried = 0usize;
let mut todo: Vec<String> = vec![String::new()];
while let Some(path) = todo.pop() {
stem_key_into(&mut buf, &path);
assert_eq!(buf, stem_key(&path), "{path:?}");
tried += 1;
if path.chars().count() < 6 {
todo.extend(letters.iter().map(|c| format!("{path}{c}")));
}
}
assert_eq!(tried, (0..=6).map(|n| 6usize.pow(n)).sum::<usize>());
}
+17 -9
View File
@@ -74,10 +74,14 @@ impl SyncProgress {
pub struct SyncInput {
folder: String,
scan: ScanInput,
/// The library's photos in the folder, each with how its sidecar is named.
photos: Vec<(Arc<Photo>, SidecarNaming)>,
/// For the colour label names a sidecar may use.
labels: Catalog,
/// The library's photos in the folder.
ids: Vec<PhotoId>,
/// How the library names sidecars; how each photo's is named is worked out by the scan (a
/// pass over the library, which the session's thread does not wait for).
naming: SidecarNaming,
/// The library as it is now (the one the scan looks files up in): for those photos, their
/// sidecars' names and the colour label names a sidecar may use.
labels: Arc<Catalog>,
}
/// What changed in a folder since it came into the library.
@@ -176,10 +180,9 @@ impl SyncInput {
if !disk && (lightcraft_catalog::folders::is_disk_root(path) || crate::cmd::library::covers_other_disks(s, path, &ids)) {
return Err(invalid(format!("{path} is a whole disk or holds disks: pass `disk: true` to synchronize it")));
}
let namings = s.sidecar_namings(&ids);
let photos = ids.iter().zip(namings).filter_map(|(id, naming)| s.catalog.photo(*id).map(|p| (Arc::clone(p), naming))).collect();
let (scan, _) = ScanInput::new(s, &[path.to_string()]);
Ok(SyncInput { folder: path.to_string(), scan, photos, labels: s.catalog.clone() })
let labels = scan.library();
Ok(SyncInput { folder: path.to_string(), scan, ids, naming: s.xmp.naming, labels })
}
}
@@ -190,11 +193,16 @@ impl SyncInput {
/// library's photos, which run on several threads too. Stops early when `progress.files.cancel`
/// is set.
pub fn scan_with(input: SyncInput, progress: &SyncProgress) -> FolderChanges {
let SyncInput { folder, scan, photos, labels } = input;
let seen: Vec<Arc<Photo>> = photos.iter().map(|(p, _)| Arc::clone(p)).collect();
let SyncInput { folder, scan, ids, naming, labels } = input;
let seen: Vec<Arc<Photo>> = ids.iter().filter_map(|id| labels.photo(*id)).map(Arc::clone).collect();
if !cfg!(target_arch = "wasm32") && !Path::new(&folder).is_dir() {
return FolderChanges { path: folder, seen, offline: true, ..Default::default() };
}
// (how each photo's sidecar is named: a pass over the library, made here and not while the
// session waits, and not at all for a folder that is not there)
let namings = crate::sidecar::namings_in(&labels, naming, &ids);
let photos: Vec<(Arc<Photo>, SidecarNaming)> =
ids.iter().zip(namings).filter_map(|(id, naming)| labels.photo(*id).map(|p| (Arc::clone(p), naming))).collect();
// (no threads on the web: one after the other there)
#[cfg(target_arch = "wasm32")]
let (out, checks) = (crate::import::scan_with(scan, std::slice::from_ref(&folder), &progress.files), check_photos(&photos, &labels, progress));
+308
View File
@@ -402,6 +402,48 @@ fn paste_selected_settings_pastes_only_chosen_copied_groups() {
assert_eq!(s.develop_of(ids[1]).unwrap().color.vibrance, 30.0);
}
/// A crop of no size (or a hair wide), as `crop.set` or a sidecar can set it, made the render's
/// samplers read at infinite coordinates and its blurs ask for sigmas of millions of pixels: with a
/// healing spot and masks on, render workers panicked (an i32 square in the blur, a bilinear
/// neighbour past isize::MAX). Such a photo renders, quickly.
#[test]
fn degenerate_crops_render_with_spots_and_masks() {
for rect in [[0.5, 0.5, 0.5, 0.5], [0.0, 0.0, 1e-9, 1.0]] {
let mut s = demo();
let id = s.active().unwrap();
s.execute("spot.add", &json!({"points": [[0.5, 0.5]], "size": 0.2, "mode": "heal"})).unwrap();
s.execute("mask.add", &json!({"kind": "radial", "rx": 0.3, "ry": 0.3})).unwrap();
s.execute("mask.add", &json!({"kind": "linear"})).unwrap();
s.execute("develop.set", &json!({"control": "effects.clarity", "value": 80})).unwrap();
s.execute("develop.set", &json!({"control": "effects.dehaze", "value": 50})).unwrap();
s.execute("crop.set", &json!({"rect": rect, "angle": 30})).unwrap();
let t = std::time::Instant::now();
let r = s.render_now(id, 700, 450).unwrap();
assert!(r.image.width >= 1 && r.image.height >= 1, "{rect:?}");
assert!(t.elapsed().as_secs_f64() < 60.0, "{rect:?}: {:?}", t.elapsed());
}
}
/// `mask.move` with a delta near `i64::MAX` overflowed `index + delta` (a panic wherever overflow
/// checks are on); extreme deltas and indices now move the mask to the bottom or the top.
#[test]
fn mask_move_with_extreme_deltas() {
let mut s = demo();
for kind in ["linear", "radial", "brush"] {
s.execute("mask.add", &json!({"kind": kind})).unwrap();
}
let ids = |s: &Session| active_dev(s).masks.iter().map(|m| m.id).collect::<Vec<_>>();
let [a, b, c] = ids(&s)[..] else { panic!("{:?}", ids(&s)) };
s.execute("mask.move", &json!({"id": b, "delta": i64::MAX})).unwrap();
assert_eq!(ids(&s), [a, c, b], "to the bottom");
s.execute("mask.move", &json!({"id": b, "delta": i64::MIN})).unwrap();
assert_eq!(ids(&s), [b, a, c], "to the top");
s.execute("mask.move", &json!({"id": c, "to": i64::MIN})).unwrap();
assert_eq!(ids(&s), [c, b, a]);
s.execute("mask.move", &json!({"id": c, "to": i64::MAX})).unwrap();
assert_eq!(ids(&s), [b, a, c]);
}
#[test]
fn masks_reorder_and_duplicate_and_invert() {
let mut s = demo();
@@ -1178,3 +1220,269 @@ fn hdr_edits_render_with_their_hdr_histogram() {
assert!(r.histogram.above_sdr() > 0.0);
assert!(r.deep.is_none(), "the preview itself stays the 8-bit SDR rendition");
}
/// Found by fuzzing catalog snapshots: an album filed under one id but naming its own folder's
/// listed itself inside itself, and `albums.list` recursed until the stack overflowed (an abort:
/// no guard catches it). Loading repairs such a catalog (`Catalog::repair`); one that skipped
/// that still lists, the loop cut off.
#[test]
fn albums_list_survives_a_catalog_that_loops() {
let mut s = Session::with_demo();
let f = s.execute("album.create", &json!({"name": "F", "folder": true})).unwrap()["id"].as_u64().unwrap();
let a = s.execute("album.create", &json!({"name": "A", "parent": f})).unwrap()["id"].as_u64().unwrap();
let mut v: serde_json::Value = serde_json::from_str(&s.catalog.to_snapshot()).unwrap();
v["albums"][a.to_string()]["id"] = f.into();
// as read, without the repair a load does
s.catalog = serde_json::from_value(v).unwrap();
let r = s.execute("albums.list", &json!({})).unwrap();
assert!(r.to_string().contains("\"F\""), "{r}");
}
#[test]
fn develop_paste_validates_groups_and_rejects_malformed_with_zero_mutations() {
let mut s = demo();
let ids: Vec<_> = s.visible().iter().copied().take(2).collect();
let photo_a = ids[0];
let photo_b = ids[1];
// Photo A has exposure = 1.0, vibrance = 30.0
s.execute("library.select", &json!({"ids": [photo_a.0]})).unwrap();
s.execute("develop.set", &json!({"values": {"light.exposure": 1.0, "color.vibrance": 30.0}})).unwrap();
s.execute("develop.copy", &json!({"groups": ["light", "color"]})).unwrap();
let initial_b_dev = (*s.develop_of(photo_b).unwrap()).clone();
let initial_b_hist_len = s.catalog.photo(photo_b).unwrap().history.len();
// 1. Invalid group names are rejected
for invalid_group in ["bogusGroup", "LIGHT", "Exposure", "unknown"] {
let err = s.execute("develop.paste", &json!({"ids": [photo_b.0], "groups": [invalid_group]})).unwrap_err();
assert!(matches!(err, crate::EngineError::BadParams { .. }));
assert_eq!(*s.develop_of(photo_b).unwrap(), initial_b_dev);
assert_eq!(s.catalog.photo(photo_b).unwrap().history.len(), initial_b_hist_len);
}
// 2. Incorrect JSON types are rejected
for invalid_param in [
json!("light"), // string instead of sequence
json!(42), // number
json!(true), // boolean
json!({"name": "light"}), // object
json!([123]), // array of non-string
json!(["light", 456]), // mixed array with invalid type
json!([true]), // array of bool
] {
let err = s.execute("develop.paste", &json!({"ids": [photo_b.0], "groups": invalid_param})).unwrap_err();
assert!(matches!(err, crate::EngineError::BadParams { .. }));
assert_eq!(*s.develop_of(photo_b).unwrap(), initial_b_dev);
assert_eq!(s.catalog.photo(photo_b).unwrap().history.len(), initial_b_hist_len);
}
// 3. Valid groups remain supported: pasting only "light" does not paste "color"
s.execute("develop.paste", &json!({"ids": [photo_b.0], "groups": ["light"]})).unwrap();
let dev_after_light = (*s.develop_of(photo_b).unwrap()).clone();
assert_eq!(dev_after_light.light.exposure, 1.0);
assert_eq!(dev_after_light.color.vibrance, initial_b_dev.color.vibrance);
// Reset photo B
s.execute("library.select", &json!({"ids": [photo_b.0]})).unwrap();
s.execute("develop.reset", &json!({})).unwrap();
assert_eq!(s.develop_of(photo_b).unwrap().light.exposure, 0.0);
// 4. Omitted groups preserves default behavior (all copied groups pasted)
s.execute("develop.paste", &json!({"ids": [photo_b.0]})).unwrap();
let dev_after_omitted = (*s.develop_of(photo_b).unwrap()).clone();
assert_eq!(dev_after_omitted.light.exposure, 1.0);
assert_eq!(dev_after_omitted.color.vibrance, 30.0);
// Reset photo B
s.execute("develop.reset", &json!({})).unwrap();
// 5. Explicit null behaves as omitted
s.execute("develop.paste", &json!({"ids": [photo_b.0], "groups": null})).unwrap();
let dev_after_null = (*s.develop_of(photo_b).unwrap()).clone();
assert_eq!(dev_after_null.light.exposure, 1.0);
assert_eq!(dev_after_null.color.vibrance, 30.0);
}
#[test]
fn develop_sync_validates_groups_and_rejects_malformed_with_zero_mutations() {
let mut s = demo();
let ids: Vec<_> = s.visible().iter().copied().take(2).collect();
let photo_a = ids[0];
let photo_b = ids[1];
// Setup: Photo A has exposure = 1.0, vibrance = 30.0
s.execute("library.select", &json!({"ids": [photo_a.0]})).unwrap();
s.execute("develop.set", &json!({"values": {"light.exposure": 1.0, "color.vibrance": 30.0}})).unwrap();
// Select Photo A (active) and Photo B
s.execute("library.select", &json!({"ids": [photo_a.0, photo_b.0], "active": photo_a.0})).unwrap();
let initial_b_dev = (*s.develop_of(photo_b).unwrap()).clone();
let initial_b_hist_len = s.catalog.photo(photo_b).unwrap().history.len();
// 1. Invalid group names are rejected
for invalid_group in ["bogusGroup", "LIGHT", "WhiteBalance", "invalid_name"] {
let err = s.execute("develop.sync", &json!({"groups": [invalid_group]})).unwrap_err();
assert!(matches!(err, crate::EngineError::BadParams { .. }));
assert_eq!(*s.develop_of(photo_b).unwrap(), initial_b_dev);
assert_eq!(s.catalog.photo(photo_b).unwrap().history.len(), initial_b_hist_len);
}
// 2. Incorrect JSON types are rejected
for invalid_param in [
json!("light"), // string instead of sequence
json!(100), // number
json!(false), // boolean
json!({"groups": "light"}), // object
json!([999]), // array of non-string
json!(["color", false]), // mixed array with invalid type
json!([null]), // array with null
] {
let err = s.execute("develop.sync", &json!({"groups": invalid_param})).unwrap_err();
assert!(matches!(err, crate::EngineError::BadParams { .. }));
assert_eq!(*s.develop_of(photo_b).unwrap(), initial_b_dev);
assert_eq!(s.catalog.photo(photo_b).unwrap().history.len(), initial_b_hist_len);
}
// 3. Valid groups remain supported: syncing only "light" does not sync "color"
s.execute("develop.sync", &json!({"groups": ["light"]})).unwrap();
let dev_after_light = (*s.develop_of(photo_b).unwrap()).clone();
assert_eq!(dev_after_light.light.exposure, 1.0);
assert_eq!(dev_after_light.color.vibrance, initial_b_dev.color.vibrance);
// Reset photo B
s.execute("library.select", &json!({"ids": [photo_b.0]})).unwrap();
s.execute("develop.reset", &json!({})).unwrap();
assert_eq!(s.develop_of(photo_b).unwrap().light.exposure, 0.0);
// Reselect Photo A (active) and Photo B
s.execute("library.select", &json!({"ids": [photo_a.0, photo_b.0], "active": photo_a.0})).unwrap();
// 4. Omitted groups preserves default behavior (all default copy groups synced)
s.execute("develop.sync", &json!({})).unwrap();
let dev_after_omitted = (*s.develop_of(photo_b).unwrap()).clone();
assert_eq!(dev_after_omitted.light.exposure, 1.0);
assert_eq!(dev_after_omitted.color.vibrance, 30.0);
// Reset photo B
s.execute("library.select", &json!({"ids": [photo_b.0]})).unwrap();
s.execute("develop.reset", &json!({})).unwrap();
// Reselect Photo A (active) and Photo B
s.execute("library.select", &json!({"ids": [photo_a.0, photo_b.0], "active": photo_a.0})).unwrap();
// 5. Explicit null behaves as omitted
s.execute("develop.sync", &json!({"groups": null})).unwrap();
let dev_after_null = (*s.develop_of(photo_b).unwrap()).clone();
assert_eq!(dev_after_null.light.exposure, 1.0);
assert_eq!(dev_after_null.color.vibrance, 30.0);
}
#[test]
fn develop_copy_and_paste_previous_validate_groups() {
let mut s = demo();
let ids: Vec<_> = s.visible().iter().copied().take(2).collect();
let photo_a = ids[0];
let photo_b = ids[1];
s.execute("library.select", &json!({"ids": [photo_a.0]})).unwrap();
s.execute("develop.set", &json!({"values": {"light.exposure": 1.5}})).unwrap();
// develop.copy rejects invalid group
let copy_err = s.execute("develop.copy", &json!({"groups": ["notAGroup"]})).unwrap_err();
assert!(matches!(copy_err, crate::EngineError::BadParams { .. }));
// develop.copy accepts valid groups
s.execute("develop.copy", &json!({"groups": ["light"]})).unwrap();
// Switch active photo to photo_b so photo_a becomes previous_active
s.execute("library.select", &json!({"ids": [photo_b.0]})).unwrap();
let initial_b_dev = (*s.develop_of(photo_b).unwrap()).clone();
// develop.pastePrevious rejects invalid group
let prev_err = s.execute("develop.pastePrevious", &json!({"groups": ["badGroup"]})).unwrap_err();
assert!(matches!(prev_err, crate::EngineError::BadParams { .. }));
assert_eq!(*s.develop_of(photo_b).unwrap(), initial_b_dev);
// develop.pastePrevious accepts valid group
s.execute("develop.pastePrevious", &json!({"groups": ["light"]})).unwrap();
assert_eq!(s.develop_of(photo_b).unwrap().light.exposure, 1.5);
}
#[test]
fn presets_validate_groups_on_create_and_update() {
let mut s = demo();
s.execute("develop.set", &json!({"values": {"light.exposure": 1.0, "color.vibrance": 25}})).unwrap();
let initial_preset_count = s.presets.len();
// 1. preset.create rejects invalid group names and incorrect JSON types without creating a preset
for invalid_groups in [json!(["bogusGroup"]), json!("light"), json!(123), json!([true]), json!(["light", 99])] {
let err = s.execute("preset.create", &json!({"name": "Test", "groups": invalid_groups})).unwrap_err();
assert!(matches!(err, crate::EngineError::BadParams { .. }));
assert_eq!(s.presets.len(), initial_preset_count);
}
// 2. preset.create with valid groups creates a preset with only those groups
let res = s.execute("preset.create", &json!({"name": "ValidPreset", "groups": ["light"]})).unwrap();
let pid = res["id"].as_str().unwrap().to_string();
assert_eq!(s.presets.len(), initial_preset_count + 1);
let p = s.presets.iter().find(|p| p.id == pid).unwrap();
assert!(p.settings.get("light").is_some());
assert!(p.settings.get("color").is_none());
// 3. preset.create with omitted groups uses default copy groups
let res_omitted = s.execute("preset.create", &json!({"name": "DefaultPreset"})).unwrap();
let pid_omitted = res_omitted["id"].as_str().unwrap().to_string();
let p_omitted = s.presets.iter().find(|p| p.id == pid_omitted).unwrap();
assert!(p_omitted.settings.get("light").is_some());
assert!(p_omitted.settings.get("color").is_some());
// Change develop settings
s.execute("develop.set", &json!({"values": {"light.exposure": 2.0, "color.vibrance": 50}})).unwrap();
let preset_before_update = s.presets.iter().find(|p| p.id == pid).unwrap().clone();
// 4. preset.update rejects invalid group names and incorrect JSON types without modifying the preset
for invalid_groups in [json!(["bogusGroup"]), json!("color"), json!(999), json!([false]), json!(["color", "bad"])] {
let err = s.execute("preset.update", &json!({"id": pid, "groups": invalid_groups})).unwrap_err();
assert!(matches!(err, crate::EngineError::BadParams { .. }));
let current = s.presets.iter().find(|p| p.id == pid).unwrap();
assert_eq!(*current, preset_before_update);
}
// 5. preset.update with valid groups updates specified groups
s.execute("preset.update", &json!({"id": pid, "groups": ["light", "color"]})).unwrap();
let p_after_valid = s.presets.iter().find(|p| p.id == pid).unwrap();
assert_eq!(p_after_valid.settings["light"]["exposure"], 2.0);
assert_eq!(p_after_valid.settings["color"]["vibrance"], 50.0);
// 6. preset.update with omitted groups updates existing preset keys
s.execute("develop.set", &json!({"values": {"light.exposure": -1.0, "color.vibrance": -20}})).unwrap();
s.execute("preset.update", &json!({"id": pid})).unwrap();
let p_after_omitted = s.presets.iter().find(|p| p.id == pid).unwrap();
assert_eq!(p_after_omitted.settings["light"]["exposure"], -1.0);
assert_eq!(p_after_omitted.settings["color"]["vibrance"], -20.0);
}
/// A Heal or Remove spot whose size (from a sidecar, which isn't held to the slider's range) is far past the photo
/// made the render's radius infinite: its heal blur and the source it reads went with it, overflowing the blur's box
/// radii and the bilinear sampler (issue #1519). Such spots render, quickly, as a spot over the whole frame.
#[test]
fn oversized_spots_render() {
use lightcraft_develop::{Spot, SpotMode};
for size in [10.0, 1e30, f64::MAX, f64::INFINITY, f64::NAN] {
for mode in [SpotMode::Heal, SpotMode::Remove, SpotMode::Clone] {
let mut s = demo();
let id = s.active().unwrap();
let mut dd = (*s.develop_of(id).unwrap_or_default()).clone();
dd.spots.push(Spot { mode, points: vec![lightcraft_geom::Point::new(0.5, 0.5)], size, ..Default::default() });
s.set_develop(id, dd, "Remove").unwrap();
let t = std::time::Instant::now();
let r = s.render_now(id, 300, 200).unwrap();
assert!(r.image.width >= 1 && r.image.height >= 1, "{size} {mode:?}");
assert!(t.elapsed().as_secs_f64() < 60.0, "{size} {mode:?}: {:?}", t.elapsed());
}
}
}
+17 -3
View File
@@ -205,10 +205,24 @@ fn detect_without_the_model_says_where_to_get_it() {
let d = temp("nodetector");
let mut s = Session::with_demo();
// no folder for models at all, then a folder that has none yet
assert!(s.execute("faces.detect", &json!({})).is_err());
let e = s.execute("faces.detect", &json!({})).unwrap_err();
assert_ne!(e.kind(), crate::ErrorKind::ModelMissing(crate::AiModel::FaceDetector), "nothing to download into: {e}");
s.face_models_dir = Some(d.join("models"));
let e = s.execute("faces.detect", &json!({})).unwrap_err().to_string();
assert!(e.contains("Settings > Faces"), "{e}");
let e = s.execute("faces.detect", &json!({})).unwrap_err();
// the app offers the download on the kind (#1046), the person reads the message
assert_eq!(e.kind(), crate::ErrorKind::ModelMissing(crate::AiModel::FaceDetector), "{e}");
assert!(
e.to_string().starts_with("invalid parameters for `faces.detect`: ") && e.to_string().contains("Settings > Faces"),
"the `error` text clients see is unchanged: {e}"
);
// a detector that is there but unusable also points to Settings > Faces, and is not a missing one
let home = d.join("models").join(lightcraft_faces::known::YUNET_ID);
std::fs::create_dir_all(&home).unwrap();
std::fs::write(home.join("model.onnx"), b"not a model").unwrap();
std::fs::write(home.join("face-model.json"), b"{}").unwrap();
let e = s.execute("faces.detect", &json!({})).unwrap_err();
assert!(e.to_string().contains("Settings > Faces"), "{e}");
assert_eq!(e.kind(), crate::ErrorKind::BadParams, "{e}");
let _ = std::fs::remove_dir_all(&d);
}
+275
View File
@@ -0,0 +1,275 @@
//! Face tiles from the embedded preview, kept in the thumbnail cache (issue #1440).
//!
//! The fixture is a TIFF-family raw with two embedded JPEG previews (a small flat red one and a
//! larger picture) and a sensor strip, as #1020's planned-read tests build them. Counting wrappers
//! around the session's file loader (the path that reads the whole original and develops the raw)
//! and its preview loader say which source a face tile came from.
use std::path::{Path, PathBuf};
use std::sync::Arc;
use std::sync::atomic::{AtomicUsize, Ordering};
use lightcraft_catalog::PhotoId;
use lightcraft_codecs::{ChromaSubsampling, EncodeImage, EncodeMeta, Samples, encode_jpeg};
use lightcraft_geom::Rect;
use lightcraft_raster::Rgba8;
use serde_json::json;
use crate::Session;
const W: u32 = 600;
const H: u32 = 400;
fn temp_dir(tag: &str) -> PathBuf {
let d = std::env::temp_dir().join(format!("lc-face-tiles-{tag}-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&d);
std::fs::create_dir_all(&d).unwrap();
d
}
/// The big preview's pixel at (x, y): smooth ramps and a soft disc, so a resampled square of it
/// compares well between two resamplers.
fn big_pixel(x: u32, y: u32) -> [u8; 3] {
let (dx, dy) = (x as f64 - 300.0, y as f64 - 200.0);
let disc = (-(dx * dx + dy * dy) / (2.0 * 40.0 * 40.0)).exp();
[(40.0 + 150.0 * x as f64 / W as f64) as u8, (60.0 + 120.0 * y as f64 / H as f64) as u8, (50.0 + 160.0 * disc) as u8]
}
fn jpeg(w: u32, h: u32, px: impl Fn(u32, u32) -> [u8; 3]) -> Vec<u8> {
let rgb: Vec<u8> = (0..h).flat_map(|y| (0..w).map(move |x| (x, y))).flat_map(|(x, y)| px(x, y)).collect();
let img = EncodeImage::new(w, h, 3, Samples::U8(&rgb));
encode_jpeg(&img, 95, ChromaSubsampling::S444, &EncodeMeta::default()).unwrap()
}
/// A raw (DNG-tagged) whose IFD0 is a small flat red preview, IFD1 the big picture and IFD2 a CFA
/// "sensor" strip of `tail` bytes. The sensor data can't be developed, so the import describes the
/// photo by its largest preview (600 × 400) and the develop path renders from it, after reading
/// the whole file.
fn raw_with_two_previews(tail: usize) -> Vec<u8> {
use lightcraft_tiff::{IfdBuilder, ImageData, TiffWriter, Value, tags as t};
let preview = |w: u32, h: u32, bytes: Vec<u8>, first: bool| {
let mut ifd = IfdBuilder::new();
if first {
ifd.set(t::DNG_VERSION, Value::Byte(vec![1, 4, 0, 0]));
}
ifd.set(t::NEW_SUBFILE_TYPE, Value::Long(vec![1]));
ifd.set(t::COMPRESSION, Value::Short(vec![7]));
ifd.set(t::PHOTOMETRIC, Value::Short(vec![t::photometric::RGB]));
ifd.set(t::IMAGE_WIDTH, Value::Long(vec![w]));
ifd.set(t::IMAGE_LENGTH, Value::Long(vec![h]));
ifd.set_image(ImageData::Strips { rows_per_strip: h, strips: vec![bytes] });
ifd
};
let small = preview(120, 80, jpeg(120, 80, |_, _| [200, 40, 40]), true);
let big = preview(W, H, jpeg(W, H, big_pixel), false);
let mut raw = IfdBuilder::new();
raw.set(t::COMPRESSION, Value::Short(vec![1]));
raw.set(t::PHOTOMETRIC, Value::Short(vec![t::photometric::CFA]));
raw.set(t::IMAGE_WIDTH, Value::Long(vec![6000]));
raw.set(t::IMAGE_LENGTH, Value::Long(vec![4000]));
raw.set_image(ImageData::Strips { rows_per_strip: 4000, strips: vec![vec![0x33u8; tail]] });
TiffWriter::default().write(&[small, big, raw]).unwrap()
}
/// How often each loader ran.
#[derive(Clone, Default)]
struct Loads {
/// The original read whole and developed (for a raw: its sensor data decoded).
files: Arc<AtomicUsize>,
/// The embedded preview read (planned read: header and preview only).
previews: Arc<AtomicUsize>,
}
impl Loads {
fn get(&self) -> (usize, usize) {
(self.files.load(Ordering::SeqCst), self.previews.load(Ordering::SeqCst))
}
}
/// Count the session's loads from now on.
fn count_loads(s: &mut Session) -> Loads {
let loads = Loads::default();
let file = s.media.file_loader.clone().unwrap();
let n = loads.files.clone();
s.media.file_loader = Some(Arc::new(move |p: &str, e: usize| {
n.fetch_add(1, Ordering::SeqCst);
file(p, e)
}));
let preview = s.media.preview_loader.clone().unwrap();
let n = loads.previews.clone();
s.media.preview_loader = Some(Arc::new(move |p: &str, e: usize| {
n.fetch_add(1, Ordering::SeqCst);
preview(p, e)
}));
loads
}
/// A session on the library in `dir` (created on first use).
fn open(dir: &Path) -> Session {
let mut s = Session::new().with_fs();
s.open_library(dir.join("library"), false).unwrap();
s
}
/// A library holding the fixture: (dir, session, the photo, the file).
fn setup(tag: &str) -> (PathBuf, Session, PhotoId, PathBuf) {
let dir = temp_dir(tag);
let file = dir.join("faces.dng");
std::fs::write(&file, raw_with_two_previews(400_000)).unwrap();
let mut s = open(&dir);
s.execute("library.import", &json!({"paths": [file.to_string_lossy()]})).unwrap();
let p = s.catalog.photos().next().unwrap();
assert_eq!((p.kind, p.width, p.height), (lightcraft_catalog::MediaKind::Raw, W, H), "described by its largest preview");
let id = p.id;
(dir, s, id, file)
}
/// A face of a tenth of the frame, centred: a 64 px tile needs a 377 px source, which the 600 px
/// preview has.
const FACE: Rect = Rect { x0: 0.45, y0: 0.45, x1: 0.55, y1: 0.55 };
/// A face of 2 %: a 64 px tile would need a 1883 px source; the preview is under ¾ of that.
const TINY_FACE: Rect = Rect { x0: 0.49, y0: 0.49, x1: 0.51, y1: 0.51 };
fn tile(s: &mut Session, id: PhotoId, face: Rect, edge: usize) -> Result<Rgba8, String> {
s.face_job(id, face, edge).unwrap().run().rendered.map(|r| r.image)
}
fn mean_abs_diff(a: &Rgba8, b: &Rgba8) -> f64 {
assert_eq!((a.width, a.height), (b.width, b.height));
let sum: u64 = a.data.iter().zip(&b.data).flat_map(|(p, q)| (0..3).map(move |c| u64::from(p[c].abs_diff(q[c])))).sum();
sum as f64 / (a.data.len() * 3) as f64
}
/// An unedited raw's face tile is cut from its largest embedded preview: the original is never read
/// whole nor its raw developed, and the tile matches the develop path's at tile size.
#[test]
fn an_unedited_raws_face_comes_from_its_largest_preview() {
let (dir, mut s, id, file) = setup("preview");
// (the preview loader's planned read: the header and the preview, not the sensor strip)
let (_, _, read) = crate::files::preview_window(&file.to_string_lossy(), usize::MAX).unwrap().unwrap();
assert!(read.fraction() < 0.5, "{read:?}");
let loads = count_loads(&mut s);
let from_preview = tile(&mut s, id, FACE, 64).unwrap();
assert_eq!((from_preview.width, from_preview.height), (64, 64));
assert_eq!(loads.get(), (0, 1), "one preview read, no whole-file read or raw decode");
// the big preview, not the small flat red one
let centre = from_preview.data[32 * 64 + 32];
assert!(centre[2] > 150 && centre[0] < 150, "the face's centre is the big preview's disc: {centre:?}");
// the develop path (no preview loader: what every face used before), from the memory cache's reach
s.media.preview_loader = None;
let mut job = s.face_job(id, FACE, 64).unwrap();
job.cache = None;
let developed = job.run().rendered.unwrap().image;
assert_eq!(loads.files.load(Ordering::SeqCst), 1, "the develop path reads the original");
let d = mean_abs_diff(&from_preview, &developed);
eprintln!("mean difference preview/develop: {d:.3}");
assert!(d < 4.0, "the preview's tile looks like the developed one: mean difference {d:.2} of 255");
let _ = std::fs::remove_dir_all(dir);
}
/// A preview under the soft limit (¾ of the pixels the tile needs across the face) leaves the tile to
/// the develop path; so does an edited photo, whose look the camera's JPEG doesn't have.
#[test]
fn a_preview_too_small_or_an_edited_photo_falls_back_to_the_develop_path() {
let (dir, mut s, id, _) = setup("fallback");
let loads = count_loads(&mut s);
let img = tile(&mut s, id, TINY_FACE, 64).unwrap();
assert_eq!((img.width, img.height), (64, 64));
assert_eq!(loads.get(), (1, 1), "the preview was tried, then the original developed");
s.execute("library.select", &json!({"ids": [id.0]})).unwrap();
s.execute("develop.set", &json!({"control": "light.exposure", "value": 1.0})).unwrap();
let edited = tile(&mut s, id, FACE, 64).unwrap();
assert_eq!(loads.get(), (2, 1), "an edited photo's face is developed; its preview isn't read");
let _ = std::fs::remove_dir_all(dir);
drop(edited);
}
/// Rendered tiles are kept in the library's thumbnail cache: a new session (the next launch) shows
/// them without reading the original, from the preview or the develop path alike; and a size
/// between two steps reuses the tile of its step.
#[test]
fn face_tiles_are_kept_on_disk_for_the_next_launch() {
let (dir, mut s, id, _) = setup("disk");
let first = tile(&mut s, id, FACE, 64).unwrap();
let _ = tile(&mut s, id, TINY_FACE, 64).unwrap();
drop(s);
let mut s = open(&dir);
let loads = count_loads(&mut s);
let again = tile(&mut s, id, FACE, 64).unwrap();
let tiny = tile(&mut s, id, TINY_FACE, 64).unwrap();
assert_eq!(loads.get(), (0, 0), "both tiles come from the disk cache: the original is not touched");
assert_eq!((again.width, tiny.width), (64, 64));
assert!(mean_abs_diff(&first, &again) < 3.0, "the cached tile is the rendered one (as a JPEG)");
// 60 px asks for the 64 px step: the same tile, no new render
assert_eq!(s.face_job(id, FACE, 60).unwrap().key, s.face_job(id, FACE, 64).unwrap().key);
let _ = tile(&mut s, id, FACE, 60).unwrap();
assert_eq!(loads.get(), (0, 0));
let _ = std::fs::remove_dir_all(dir);
}
/// What can go wrong never panics: a preview loader that fails or answers nonsense, a damaged
/// preview, face boxes at the edge or degenerate, an offline original, a damaged cache entry.
#[test]
fn face_tiles_never_crash() {
let (dir, mut s, id, file) = setup("crash");
// a damaged cache entry: dropped, and the tile rendered again (from the preview)
let job = s.face_job(id, FACE, 64).unwrap();
let key = job.cache.as_ref().unwrap().1.to_string();
assert!(job.run().rendered.is_ok());
let thumbs = s.library.as_ref().unwrap().thumbs_dir();
drop(s);
let entry = thumbs.join(&key[..2]).join(format!("{key}.jpg"));
assert!(entry.exists(), "the tile was written to the thumbnail cache");
std::fs::write(&entry, b"not a jpeg").unwrap();
let mut s = open(&dir);
let loads = count_loads(&mut s);
assert_eq!(tile(&mut s, id, FACE, 64).unwrap().width, 64);
assert_eq!(loads.get(), (0, 1), "rendered again from the preview");
// face boxes at the corners, oversized, empty, not numbers: a tile or an error, never a panic
for face in [
Rect { x0: 0.0, y0: 0.0, x1: 0.05, y1: 0.05 },
Rect { x0: 0.95, y0: 0.95, x1: 1.0, y1: 1.0 },
Rect { x0: -5.0, y0: -5.0, x1: 5.0, y1: 5.0 },
Rect { x0: 0.5, y0: 0.5, x1: 0.5, y1: 0.5 },
Rect { x0: f64::NAN, y0: 0.5, x1: 0.6, y1: f64::INFINITY },
Rect { x0: f64::NEG_INFINITY, y0: f64::NAN, x1: f64::NAN, y1: f64::NAN },
] {
for edge in [0, 1, 64, usize::MAX] {
let img = tile(&mut s, id, face, edge).unwrap_or_else(|e| panic!("{face:?} at {edge}: {e}"));
assert_eq!(img.width, img.height, "{face:?} at {edge}");
}
}
// a preview loader that answers nonsense or nothing: the develop path renders the tile
let id2 = id;
for answer in [None, Some(Rgba8 { width: W as usize, height: H as usize, data: Vec::new() }), Some(Rgba8::new(1, 1))] {
s.media.preview_loader = Some(Arc::new(move |_: &str, _: usize| answer.clone()));
let mut job = s.face_job(id2, FACE, 64).unwrap();
job.cache = None;
assert_eq!(job.run().rendered.unwrap().image.width, 64);
}
// a damaged preview in the file: no tile from it, and no panic either way
s.media.preview_loader = Some(crate::files::fs_preview_loader());
let mut bytes = std::fs::read(&file).unwrap();
let n = bytes.len();
for b in bytes.iter_mut().take(n - 400_000).skip(1000) {
*b = 0x5a;
}
std::fs::write(&file, &bytes).unwrap();
let mut job = s.face_job(id, FACE, 64).unwrap();
job.cache = None;
let _ = job.run();
// the original gone (an offline drive): an error, no panic
std::fs::remove_file(&file).unwrap();
let mut job = s.face_job(id, FACE, 64).unwrap();
job.cache = None;
assert!(job.run().rendered.is_err());
let _ = std::fs::remove_dir_all(dir);
}
+27
View File
@@ -503,3 +503,30 @@ fn copies_run_side_by_side_with_the_same_outcome() {
assert_eq!(first.file_name, "IMG_1.png");
let _ = std::fs::remove_dir_all(&base);
}
#[test]
fn finish_keeps_the_original_when_the_copy_changed_to_the_same_length() {
use crate::import_move::{finish, place};
let base = temp_dir("finish-recheck");
let (src, dest) = (base.join("card/IMG_1.jpg"), base.join("dest"));
std::fs::create_dir_all(src.parent().unwrap()).unwrap();
std::fs::write(&src, b"original bytes").unwrap();
inject(Fault::NoLink);
let placed = place(&src, &dest, "IMG_1.jpg").unwrap();
// unchanged copy: the original goes
assert!(finish(&placed).is_ok());
assert!(!src.exists());
// a copy replaced by other bytes of the same length: the original stays
std::fs::write(&src, b"original bytes").unwrap();
let placed = place(&src, &dest, "IMG_1.jpg").unwrap();
std::fs::write(&placed.dst, b"different byte").unwrap();
assert!(finish(&placed).is_err());
assert_eq!(std::fs::read(&src).unwrap(), b"original bytes");
// an original edited meanwhile to other bytes of the same length: it stays too
let placed = place(&src, &dest, "IMG_1.jpg").unwrap();
std::fs::write(&src, b"edited bytes!").unwrap();
assert!(finish(&placed).is_err());
assert_eq!(std::fs::read(&src).unwrap(), b"edited bytes!");
inject(Fault::None);
let _ = std::fs::remove_dir_all(&base);
}
+51
View File
@@ -67,6 +67,57 @@ fn hdr_merge_command_creates_and_imports_a_dng() {
let _ = std::fs::remove_dir_all(&dir);
}
/// An HDR merge keeps the reference frame's lens corrections. The sources carry an off-centre `WarpRectilinear` and
/// `FixVignetteRadial` for their active area and have a default crop; the merged DNG must give "Enable Profile
/// Corrections" the same correction as the reference photo, upright and for a vertical bracket (whose DNG keeps the
/// pixels unturned).
#[test]
fn hdr_merge_keeps_the_reference_frames_lens_corrections() {
use lightcraft_geom::{Orientation, Point};
use lightcraft_pipeline::optics::{embedded_vignette_gain, embedded_warp};
use lightcraft_raw::Opcode;
for orientation in [Orientation::Normal, Orientation::Rotate90] {
let dir = temp_dir(&format!("hdr-lens-{orientation:?}"));
let mut paths = Vec::new();
for (i, b) in lightcraft_merge::synth::bracket_dngs(480, 320, &[-2.0, 0.0, 2.0]).unwrap().into_iter().enumerate() {
let mut raw = lightcraft_raw::decode(&b).unwrap();
raw.crop = lightcraft_raw::Rect::new(10, 6, 452, 304);
raw.orientation = orientation;
raw.metadata.orientation = Some(orientation);
raw.opcodes.list3 = vec![
Opcode::WarpRectilinear { planes: vec![[0.96, 0.05, -0.012, 0.002, 0.001, -0.0005]], center: [0.48, 0.53] },
Opcode::FixVignetteRadial { k: [0.35, -0.05, 0.01, 0.0, 0.0], center: [0.48, 0.53] },
];
let p = dir.join(format!("IMG_{i}.dng"));
std::fs::write(&p, lightcraft_raw::write_dng(&raw, &Default::default()).unwrap()).unwrap();
paths.push(p.to_string_lossy().to_string());
}
let mut s = Session::new().with_fs();
let ids = import(&mut s, paths);
let r = s.execute("merge.hdr", &json!({"ids": ids})).unwrap();
assert_eq!(r["reference"].as_u64(), Some(ids[1]), "{r}");
let merged = s.catalog.photo(lightcraft_catalog::PhotoId(r["id"].as_u64().unwrap())).unwrap().clone();
let reference = s.catalog.photo(lightcraft_catalog::PhotoId(ids[1])).unwrap().clone();
let expect = if orientation == Orientation::Normal { (452, 304) } else { (304, 452) };
assert_eq!(((merged.width, merged.height), (reference.width, reference.height)), (expect, expect));
let a = reference.embedded_lens.expect("the source's correction");
let b = merged.embedded_lens.expect("the merged DNG carries the reference frame's correction");
let (wa, wb) = (a.warp.unwrap(), b.warp.unwrap());
let (va, vb) = (a.vignette.unwrap(), b.vignette.unwrap());
let (w, h) = (expect.0 as f64, expect.1 as f64);
for (x, y) in [(0.0, 0.0), (w, h), (w, 0.0), (0.0, h), (w * 0.3, h * 0.7), (w * 0.5, h * 0.5)] {
let p = Point::new(x, y);
for ch in 0..3 {
let (pa, pb) = (embedded_warp(&wa, p, ch, w, h), embedded_warp(&wb, p, ch, w, h));
assert!((pa.x - pb.x).abs() < 1e-6 && (pa.y - pb.y).abs() < 1e-6, "{orientation:?} {p:?}: {pa:?} vs {pb:?}");
}
let (ga, gb) = (embedded_vignette_gain(&va, p, w, h), embedded_vignette_gain(&vb, p, w, h));
assert!((ga - gb).abs() < 1e-9, "{orientation:?} {p:?}: {ga} vs {gb}");
}
let _ = std::fs::remove_dir_all(&dir);
}
}
/// Raws without colour matrices of their own (CR2, NEF, ARW…) merge into a DNG with the colour they open with,
/// not with no colour at all (issue #965: an HDR merge of CR2s rendered green and dark).
#[test]
+28 -1
View File
@@ -115,6 +115,7 @@ fn a_failing_download_ends_with_an_error_and_never_blocks() {
}
let st = s.segmenter.download_status();
assert!(!st.finished);
assert!(!st.cancelled, "a failure is not a cancel");
let e = st.error.unwrap();
assert!(e.contains("not found"), "{e}");
assert!(!s.segmenter.installed());
@@ -272,9 +273,35 @@ fn a_running_model_download_shows_in_activity_and_cancels() {
assert!(t.elapsed() < Duration::from_secs(10), "the download never stopped");
std::thread::sleep(Duration::from_millis(10));
}
let e = s.segmenter.download_status().error.unwrap_or_default();
let st = s.segmenter.download_status();
assert!(st.cancelled, "the app tells a cancel by the flag (#1046): {st:?}");
let e = st.error.unwrap_or_default();
assert!(e.contains("cancelled"), "{e}");
assert!(!s.segmenter.installed());
let _ = std::fs::remove_dir_all(&dir);
let _ = std::fs::remove_file(&mirrors);
}
/// Frontends decide on the error's kind, not its words (#1046): a missing model is
/// `ModelMissing(Sam3)` whether the app queues requests (background) or a command waits (CLI, MCP);
/// a build without AI masks is not a missing model (there is nothing to download).
#[test]
fn a_missing_model_is_model_missing_by_kind() {
use crate::{AiModel, ErrorKind};
for background in [true, false] {
let mut s = Session::with_demo();
s.segmenter.background = background;
s.segmenter.dir = Some(tmp(&format!("kind-{background}")));
for (cmd, p) in
[("mask.add", json!({"kind": "object"})), ("mask.add", json!({"kind": "prompt", "text": "sky"})), ("segment.prepare", json!({}))]
{
let e = s.execute(cmd, &p).unwrap_err();
if Segmenter::AVAILABLE {
assert_eq!(e.kind(), ErrorKind::ModelMissing(AiModel::Sam3), "{cmd} (background {background}): {e}");
assert!(e.to_string().starts_with(NOT_INSTALLED), "the message is unchanged: {e}");
} else {
assert_eq!(e.kind(), ErrorKind::Other, "{cmd}: {e}");
}
}
}
}
+3 -2
View File
@@ -20,8 +20,9 @@ fn a_preview_reads_only_the_header_and_the_preview() {
assert!(file_len > 0, "empty file");
let started = Instant::now();
let window =
lightcraft_engine::files::preview_window(&path).expect("the file reads").expect("it carries an embedded preview the header can point at");
let window = lightcraft_engine::files::preview_window(&path, usize::MAX)
.expect("the file reads")
.expect("it carries an embedded preview the header can point at");
let elapsed = started.elapsed();
let (_, preview, read) = window;
+1 -1
View File
@@ -2,6 +2,6 @@
Native model downloads in pure Rust (L0, Apache-2.0): HTTP/1.1 over std sockets and rustls with the RustCrypto provider, without ring, aws-lc, C or assembly build dependencies. The wasm32 crate is empty.
Call `download` with bounded file specifications and ordered mirrors, or `download_url` for an exact catalog URL (including its query). Each file has a local filename independent of the URL, a size cap and optional pinned size/SHA-256. Transfers resume pinned `.part` files, verify before renaming, report progress and check cancellation during reads. Run on a worker thread. No proxy support; DNS resolution uses the platform resolver. Tokens are off by default and never forwarded to a different redirect host.
Call `download` with bounded file specifications and ordered mirrors, or `download_url` for an exact catalog URL (including its query). Each file has a local filename independent of the URL, a size cap and optional pinned size/SHA-256. Transfers resume pinned `.part` files, verify before renaming, report progress and check cancellation during reads. Run on a worker thread. No proxy support; DNS resolution uses the platform resolver. Tokens are off by default and never forwarded to a different redirect host. A redirect from https to http is refused.
Tests use local synthetic HTTP servers for redirects, resume, truncation, cancellation, stalls, hashes, size limits and hostile specifications; they make no internet requests.
+10 -1
View File
@@ -110,7 +110,11 @@ impl Url {
pub fn join(&self, location: &str) -> Result<Url, HttpError> {
let l = location.trim();
if l.starts_with("http://") || l.starts_with("https://") {
return Url::parse(l);
let url = Url::parse(l)?;
if self.tls && !url.tls {
return Err(HttpError::BadUrl("redirect from https to http refused".into()));
}
return Ok(url);
}
// another scheme (`ftp:`, `file:`, `data:`…)
if let Some((scheme, _)) = l.split_once(':')
@@ -541,6 +545,11 @@ mod tests {
assert_eq!(u.join("other").unwrap().path, "/x/y/other");
assert_eq!(u.join("//c.example/p").unwrap().host, "c.example");
assert!(u.join("ftp://nope").is_err());
// no downgrade from https to http, however the redirect is written
assert!(u.join("http://a.example/z").is_err());
let plain = Url::parse("http://127.0.0.1:8080/a").unwrap();
assert!(!plain.join("http://127.0.0.1:9/b").unwrap().tls);
assert!(plain.join("https://b.example/z").unwrap().tls);
}
#[test]
+2 -3
View File
@@ -245,10 +245,9 @@ fn box_pass(cx: &mut Cx<'_>, horizontal: bool, src: &Buf, dst: &Buf, (w, h): (us
/// Gaussian blur of a `w × h` image of `nc` interleaved channels: three box passes each way
/// (`lightcraft_raster::blur::gaussian`).
pub(crate) fn gaussian(cx: &mut Cx<'_>, src: &Buf, w: usize, h: usize, nc: usize, sigma: f32) -> Buf {
if sigma <= 0.3 || w * h == 0 {
let Some(radii) = lightcraft_raster::blur::radii_for(sigma, w, h) else {
return cx.copy(src);
}
let radii = lightcraft_raster::blur::box_radii(sigma);
};
let passes: Vec<(bool, usize)> = radii.iter().map(|r| (true, *r)).chain(radii.iter().map(|r| (false, *r))).filter(|p| p.1 > 0).collect();
if passes.is_empty() {
return cx.copy(src);
+109 -12
View File
@@ -65,22 +65,46 @@ impl Remote {
Err(last)
}
fn roundtrip(&mut self, line: &str) -> std::io::Result<String> {
/// Send one request line and read the reply line. On failure the flag says whether the request
/// may have reached the app (written in full); only an unsent request is safe to resend.
fn roundtrip(&mut self, line: &str) -> Result<String, (std::io::Error, bool)> {
// A connection the app already closed (it restarted, or closed it after an error reply) is
// replaced before anything is sent on it, so that case still reconnects without a resend.
if self.conn.as_ref().is_some_and(|(reader, writer)| !reader.buffer().is_empty() || peer_closed(writer)) {
self.conn = None;
}
if self.conn.is_none() {
self.reconnect()?;
self.reconnect().map_err(|e| (e, false))?;
}
let Some((reader, writer)) = self.conn.as_mut() else {
return Err(std::io::Error::new(std::io::ErrorKind::NotConnected, "not connected"));
return Err((std::io::Error::new(std::io::ErrorKind::NotConnected, "not connected"), false));
};
writer.write_all(line.as_bytes())?;
writer.write_all(b"\n")?;
writer.flush()?;
let mut data = Vec::with_capacity(line.len() + 1);
data.extend_from_slice(line.as_bytes());
data.push(b'\n');
writer.write_all(&data).map_err(|e| (e, false))?;
writer.flush().map_err(|e| (e, true))?;
let mut reply = String::new();
if reader.read_line(&mut reply)? == 0 {
return Err(std::io::Error::new(std::io::ErrorKind::UnexpectedEof, "control channel closed"));
if reader.read_line(&mut reply).map_err(|e| (e, true))? == 0 {
return Err((std::io::Error::new(std::io::ErrorKind::UnexpectedEof, "control channel closed"), true));
}
Ok(reply)
}
fn unknown_outcome(&self, method: &str, e: &std::io::Error) -> String {
let addr = &self.addr;
format!("lost the LightCraft app at {addr} after sending `{method}` ({e}); the outcome is unknown and it was not resent, check the app")
}
}
/// True when the app has closed `stream` (or it is unusable), checked without blocking or consuming data.
fn peer_closed(stream: &TcpStream) -> bool {
if stream.set_nonblocking(true).is_err() {
return true;
}
let mut byte = [0u8; 1];
let open = matches!(stream.peek(&mut byte), Err(ref e) if e.kind() == std::io::ErrorKind::WouldBlock);
stream.set_nonblocking(false).is_err() || !open
}
impl Backend for Remote {
@@ -88,16 +112,26 @@ impl Backend for Remote {
let id = self.next_id;
self.next_id += 1;
let line = json!({"id": id, "method": method, "params": params}).to_string();
// One retry with a fresh connection (the app may have restarted).
// Retry once with a fresh connection (the app may have restarted), but only when the request
// never reached the app. After a full write the command may have run, so resending it could
// apply a mutation twice (toggle, undo, ...): report the unknown outcome instead.
let reply = match self.roundtrip(&line) {
Ok(r) => r,
Err(_) => {
Err((_, false)) => {
self.conn = None;
self.roundtrip(&line).map_err(|e| {
self.roundtrip(&line).map_err(|(e, sent)| {
self.conn = None;
format!("LightCraft app at {} is not reachable: {e} (start it with `lightcraft --control PORT`)", self.addr)
if sent {
self.unknown_outcome(method, &e)
} else {
format!("LightCraft app at {} is not reachable: {e} (start it with `lightcraft --control PORT`)", self.addr)
}
})?
}
Err((e, true)) => {
self.conn = None;
return Err(self.unknown_outcome(method, &e));
}
};
let v: Value = serde_json::from_str(reply.trim()).map_err(|e| format!("bad reply from app: {e}"))?;
if v.get("ok").and_then(Value::as_bool) == Some(true) {
@@ -115,3 +149,66 @@ impl Backend for Remote {
format!("remote {}", self.addr)
}
}
#[cfg(test)]
mod tests {
use super::*;
use std::net::TcpListener;
use std::sync::Arc;
use std::sync::atomic::{AtomicUsize, Ordering};
#[test]
fn lost_reply_is_not_resent() {
let listener = TcpListener::bind("127.0.0.1:0").unwrap();
let addr = listener.local_addr().unwrap().to_string();
let seen = Arc::new(AtomicUsize::new(0));
let seen2 = seen.clone();
std::thread::spawn(move || {
// Every connection: read the request, count it, drop without replying.
while let Ok((s, _)) = listener.accept() {
let mut line = String::new();
if BufReader::new(s).read_line(&mut line).unwrap_or(0) > 0 {
seen2.fetch_add(1, Ordering::SeqCst);
}
}
});
let mut remote = Remote::lazy(&addr);
let err = remote.call("engine.execute", json!({})).unwrap_err();
assert!(err.contains("outcome is unknown"), "{err}");
std::thread::sleep(Duration::from_millis(200));
assert_eq!(seen.load(Ordering::SeqCst), 1);
}
#[test]
fn connection_closed_by_app_reconnects_before_sending() {
let listener = TcpListener::bind("127.0.0.1:0").unwrap();
let addr = listener.local_addr().unwrap().to_string();
let seen = Arc::new(AtomicUsize::new(0));
let seen2 = seen.clone();
std::thread::spawn(move || {
// Every connection: answer one request, then close (as after an app restart).
while let Ok((s, _)) = listener.accept() {
let mut writer = s.try_clone().unwrap();
let mut line = String::new();
if BufReader::new(s).read_line(&mut line).unwrap_or(0) > 0 {
seen2.fetch_add(1, Ordering::SeqCst);
writer.write_all(b"{\"ok\": true, \"result\": 7}\n").unwrap();
}
}
});
let mut remote = Remote::lazy(&addr);
assert_eq!(remote.call("engine.commands", json!({})).unwrap(), json!(7));
std::thread::sleep(Duration::from_millis(200));
assert_eq!(remote.call("engine.commands", json!({})).unwrap(), json!(7));
assert_eq!(seen.load(Ordering::SeqCst), 2);
}
#[test]
fn unreachable_app_is_reported() {
let listener = TcpListener::bind("127.0.0.1:0").unwrap();
let addr = listener.local_addr().unwrap().to_string();
drop(listener);
let err = Remote::lazy(&addr).call("engine.commands", json!({})).unwrap_err();
assert!(err.contains("not reachable"), "{err}");
}
}
+25 -3
View File
@@ -6,7 +6,7 @@ use lightcraft_geom::Orientation;
use lightcraft_meta::Metadata;
use lightcraft_raster::Rgb32f;
use lightcraft_raster::resample::{Filter, fit};
use lightcraft_raw::ColorData;
use lightcraft_raw::{ColorData, Opcode};
use crate::{MergeError, Result};
@@ -36,6 +36,10 @@ pub struct Frame {
pub metadata: Metadata,
/// Raw files: the source's `BaselineExposure` (EV), carried into the output.
pub baseline_exposure: f64,
/// Raw files: the lens corrections the source applies (`WarpRectilinear`, `WarpRectilinear2`, `FixVignetteRadial`,
/// in order), re-expressed for `image` (the default crop, not yet turned), for the merged DNG to carry. Empty for
/// other sources and once [`Frame::orient`] has turned the pixels.
pub lens: Vec<Opcode>,
}
impl Frame {
@@ -53,6 +57,7 @@ impl Frame {
orientation: Orientation::Normal,
metadata: Metadata::default(),
baseline_exposure: 0.0,
lens: Vec::new(),
}
}
@@ -63,11 +68,12 @@ impl Frame {
self.image.height
}
/// Apply the display orientation to the pixels.
/// Apply the display orientation to the pixels. The lens corrections, written for the unturned pixels, are dropped.
pub fn orient(&mut self) {
if self.orientation != Orientation::Normal {
self.image = self.image.oriented(self.orientation);
self.orientation = Orientation::Normal;
self.lens.clear();
}
}
}
@@ -120,7 +126,21 @@ pub fn load_frame_with(bytes: &[u8], max_edge: Option<usize>, orient: bool, colo
let exposure = exif_exposure(&metadata);
if lightcraft_raw::probe(bytes).is_some() {
let mut raw = lightcraft_raw::decode(bytes).map_err(|e| MergeError::Decode(e.to_string()))?;
// geometric lens corrections stay with the develop settings ("profile corrections")
// Lens corrections stay with the develop settings ("profile corrections"): frames are merged uncorrected, and
// the output DNG carries the reference frame's corrections, re-expressed for the default crop `develop` keeps.
// The DNG 1.6 skip rule is applied here, so no `WarpRectilinear2` is left optional.
let (aw, ah) = (raw.active_area.width, raw.active_area.height);
let crop = raw.develop_crop(aw, ah);
let lens = lightcraft_raw::opcodes::applied_ops(&raw.opcodes.list3)
.filter(|op| op.is_lens_correction())
.filter_map(|op| op.cropped(aw, ah, crop))
.map(|op| match op {
Opcode::WarpRectilinear2 { planes, center, reciprocal, .. } => {
Opcode::WarpRectilinear2 { planes, center, reciprocal, optional: false }
}
op => op,
})
.collect();
raw.opcodes.list3.retain(|op| !op.is_lens_correction());
if let Some(color) = colour(&mut raw, bytes) {
raw.color = color;
@@ -146,6 +166,7 @@ pub fn load_frame_with(bytes: &[u8], max_edge: Option<usize>, orient: bool, colo
orientation: raw.orientation,
metadata: if raw.metadata == Metadata::default() { metadata } else { raw.metadata.clone() },
baseline_exposure: raw.color.baseline_exposure,
lens,
};
if f.exposure.is_none() {
f.exposure = exif_exposure(&f.metadata);
@@ -176,6 +197,7 @@ pub fn load_frame_with(bytes: &[u8], max_edge: Option<usize>, orient: bool, colo
orientation: Orientation::from_exif(d.orientation),
metadata,
baseline_exposure: 0.0,
lens: Vec::new(),
};
if orient {
f.orient();
+3
View File
@@ -83,6 +83,8 @@ pub struct HdrResult {
pub orientation: lightcraft_geom::Orientation,
pub metadata: lightcraft_meta::Metadata,
pub baseline_exposure: f64,
/// The reference frame's lens corrections, for `radiance` ([`Frame::lens`]).
pub lens: Vec<lightcraft_raw::Opcode>,
}
#[inline]
@@ -277,6 +279,7 @@ pub fn merge_hdr(mut frames: Vec<Frame>, opts: &HdrOptions, progress: &Progress)
orientation: rf.orientation,
metadata: rf.metadata.clone(),
baseline_exposure: rf.baseline_exposure,
lens: rf.lens.clone(),
})
}
+9 -5
View File
@@ -8,12 +8,14 @@
//! still camera RGB.
//! - Other sources: the "camera" is the file's linear RGB space; `ColorMatrix1` = XYZ(D65) → RGB,
//! `ForwardMatrix1` = RGB → XYZ(D50), `AsShotNeutral` = (1, 1, 1).
//! - Lens corrections (an HDR merge's: the reference frame's, [`crate::Frame::lens`]) go in `OpcodeList3`, where the
//! develop settings' profile corrections find them as they do in the sources.
use lightcraft_color::{D50, D65, Mat3, bradford};
use lightcraft_geom::Orientation;
use lightcraft_meta::Metadata;
use lightcraft_raster::Rgb32f;
use lightcraft_raw::{BlackLevel, ColorData, DngCompression, DngWriteOptions, OpcodeLists, RawData, RawFormat, RawImage, Rect};
use lightcraft_raw::{BlackLevel, ColorData, DngCompression, DngWriteOptions, Opcode, OpcodeLists, RawData, RawFormat, RawImage, Rect};
use crate::frame::FrameColor;
use crate::{MergeError, Result};
@@ -59,7 +61,8 @@ pub fn color_data(color: &FrameColor) -> ColorData {
}
}
/// Encode `img` (scene-linear, 1.0 = source white) as a LinearRaw DNG.
/// Encode `img` (scene-linear, 1.0 = source white) as a LinearRaw DNG, with `lens` (lens corrections for `img`, not
/// yet turned by `orientation`) as its `OpcodeList3`.
pub fn write_linear_dng(
img: &Rgb32f,
color: &FrameColor,
@@ -67,6 +70,7 @@ pub fn write_linear_dng(
metadata: &Metadata,
baseline_exposure: f64,
samples: DngSamples,
lens: &[Opcode],
) -> Result<Vec<u8>> {
let (w, h) = (img.width, img.height);
if w == 0 || h == 0 {
@@ -107,7 +111,7 @@ pub fn write_linear_dng(
color: cd,
wb_multipliers: None,
linearized: false,
opcodes: OpcodeLists::default(),
opcodes: OpcodeLists { list3: lens.to_vec(), ..Default::default() },
metadata: meta,
};
let compression = DngCompression::Deflate { tile: 256, half: samples == DngSamples::Half };
@@ -122,7 +126,7 @@ mod tests {
fn linear_dng_round_trips_through_the_raw_decoder() {
let img = Rgb32f::from_fn(80, 60, |x, y| [x as f32 / 20.0, y as f32 / 30.0, 0.3]);
let color = FrameColor::Linear { to_xyz_d50: bradford(D65, D50).mul(&lightcraft_color::SRGB.to_xyz()) };
let bytes = write_linear_dng(&img, &color, Orientation::Normal, &Metadata::default(), 0.0, DngSamples::Half).unwrap();
let bytes = write_linear_dng(&img, &color, Orientation::Normal, &Metadata::default(), 0.0, DngSamples::Half, &[]).unwrap();
let raw = lightcraft_raw::decode(&bytes).unwrap();
assert_eq!((raw.width, raw.height, raw.cpp), (80, 60, 3));
let gain = 2f32.powf(raw.color.baseline_exposure as f32);
@@ -145,7 +149,7 @@ mod tests {
for c in 0..3 {
assert!((red[c] / k - want[c]).abs() < 5e-3, "{red:?} vs {want:?}");
}
let u16 = write_linear_dng(&img, &color, Orientation::Normal, &Metadata::default(), 0.0, DngSamples::U16).unwrap();
let u16 = write_linear_dng(&img, &color, Orientation::Normal, &Metadata::default(), 0.0, DngSamples::U16, &[]).unwrap();
assert!(lightcraft_raw::decode(&u16).is_ok());
}
}
+1 -1
View File
@@ -48,7 +48,7 @@ pub fn bracket_dngs(w: usize, h: usize, evs: &[f64]) -> Result<Vec<Vec<u8>>> {
iso: Some(100),
..Default::default()
};
write_linear_dng(&img, &srgb_color(), Orientation::Normal, &meta, 0.0, DngSamples::U16)
write_linear_dng(&img, &srgb_color(), Orientation::Normal, &meta, 0.0, DngSamples::U16, &[])
})
.collect()
}
+24 -3
View File
@@ -16,7 +16,7 @@ use crate::optics::{Warp, reorient_lens};
pub const BLANK: [f32; 3] = [0.55, 0.55, 0.55];
/// The geometric frame of a render.
#[derive(Clone, Debug)]
#[derive(Clone, Debug, PartialEq)]
pub struct Frame {
/// Source (EXIF-oriented) dimensions in pixels.
pub src_w: usize,
@@ -167,10 +167,12 @@ impl Frame {
}
}
/// Aspect ratio (w/h) of the output.
/// Aspect ratio (w/h) of the output. A crop narrower than a source pixel (or of no size, from
/// an agent or a sidecar) counts each side as one pixel, so the aspect stays finite: 0/0 made
/// it NaN, and the loupe laid the photo out at a NaN size.
pub fn aspect(&self) -> f64 {
let r = self.crop.rect_px(self.ow, self.oh);
r.width() / r.height()
r.width().max(1.0) / r.height().max(1.0)
}
/// Size of the (cropped) output at the source's own resolution, in pixels.
@@ -383,6 +385,25 @@ mod tests {
assert!((back.x - 50.0).abs() < 1e-6 && (back.y - 50.0).abs() < 1e-6);
}
/// A crop of no size (or under a pixel wide) gave a NaN or infinite aspect; each side now
/// counts as at least one source pixel.
#[test]
fn degenerate_crops_have_a_finite_aspect() {
let mut s = DevelopSettings::default();
for (rect, want) in [
(Rect::new(0.5, 0.5, 0.5, 0.5), 1.0),
(Rect::new(0.0, 0.0, 1e-9, 1.0), 1.0 / 200.0),
(Rect::new(0.0, 0.5, 1.0, 0.5), 400.0),
(Rect::new(0.25, 0.0, 0.75, 1.0), 1.0),
] {
s.crop.geometry.rect = rect;
let f = Frame::new(400, 200, &s, true);
assert!((f.aspect() - want).abs() < 1e-9, "{rect:?}: {}", f.aspect());
let (w, h) = f.fit(1000, 1000);
assert!((1..=1000).contains(&w) && (1..=1000).contains(&h), "{rect:?}: {w}×{h}");
}
}
#[test]
fn flip_mirrors() {
let mut s = DevelopSettings::default();
+54 -1
View File
@@ -543,12 +543,35 @@ pub(crate) fn prepare(
/// The airlight of the whole output frame, estimated on a small render of it (`proxy_w` px wide):
/// a windowed render must dehaze with the frame's airlight, not that of the pixels it holds.
pub(crate) fn frame_airlight(src: &Rgb32f, info: &SourceInfo, s: &DevelopSettings, frame: &Frame, proxy_w: usize, proxy_h: usize) -> f32 {
let mut img = frame.sample(src, proxy_w, proxy_h);
let mut img = Rgb32f::clone(&frame_proxy(src, frame, proxy_w, proxy_h));
white_balance(&mut img, info, s);
let sigma = (0.02 * frame.px_per_long(proxy_w)).max(1.0) as f32;
airlight(&gaussian(&img.map(dark_of), sigma))
}
/// How many airlight proxies [`frame_proxy`] keeps (one per photo and frame being zoomed into; each ≤ 1.8 MB).
const FRAME_PROXIES_KEPT: usize = 4;
/// `frame.sample(src, w, h)`, kept for the last few (source, frame, size): sampling a 64 MP original down to the
/// airlight proxy reads all of it (~190 ms of CPU), and every window of a zoomed view with Dehaze asked for it again,
/// on each pan and slider tick. The proxy is taken before white balance, so a WB change still hits.
fn frame_proxy(src: &Rgb32f, frame: &Frame, w: usize, h: usize) -> Arc<Rgb32f> {
type Entry = (u64, Frame, usize, usize, Arc<Rgb32f>);
static CACHE: std::sync::Mutex<Vec<Entry>> = std::sync::Mutex::new(Vec::new());
let key = crate::optics::fingerprint(src);
let found = |c: &[Entry]| c.iter().find(|e| e.0 == key && e.1 == *frame && (e.2, e.3) == (w, h)).map(|e| e.4.clone());
if let Some(img) = found(&CACHE.lock().unwrap_or_else(std::sync::PoisonError::into_inner)) {
return img;
}
let img = Arc::new(frame.sample(src, w, h));
let mut c = CACHE.lock().unwrap_or_else(std::sync::PoisonError::into_inner);
c.push((key, frame.clone(), w, h, img.clone()));
if c.len() > FRAME_PROXIES_KEPT {
c.remove(0);
}
img
}
/// The airlight is estimated from every `AIRLIGHT_STEP`-th value of the dark channel.
pub const AIRLIGHT_STEP: usize = 7;
@@ -600,6 +623,36 @@ mod tests {
assert!((luminance_2020(c) - 0.3).abs() < 0.01);
}
// A windowed render's airlight proxy is kept per (source, frame, size): the airlight is the one sampling anew gives,
// and another source or frame of the same size gets its own, never the proxy kept for the last one
#[test]
fn the_kept_airlight_proxy_matches_a_fresh_one_and_misses_on_another_source_or_frame() {
let (info, s) = (SourceInfo::default(), DevelopSettings::default());
let hazy = Rgb32f::from_fn(300, 200, |x, y| {
let v = 0.05 + 0.6 * x as f32 / 300.0;
[v, v * 0.95, v * (0.8 + 0.2 * y as f32 / 200.0)]
});
let other = Rgb32f::from_fn(300, 200, |x, y| [0.02 + 0.1 * ((x + y) % 7) as f32 / 7.0; 3]);
let fresh = |src: &Rgb32f, frame: &Frame| {
let mut img = frame.sample(src, 48, 32);
white_balance(&mut img, &info, &s);
airlight(&gaussian(&img.map(dark_of), (0.02 * frame.px_per_long(48)).max(1.0) as f32))
};
let frame = crate::frame_for(&hazy, &info, &s, true);
let kept = frame_proxy(&hazy, &frame, 48, 32);
assert!(Arc::ptr_eq(&kept, &frame_proxy(&hazy, &frame, 48, 32)), "the second ask is served from the cache");
assert_eq!(frame_airlight(&hazy, &info, &s, &frame, 48, 32).to_bits(), fresh(&hazy, &frame).to_bits());
assert_eq!(frame_airlight(&hazy, &info, &s, &frame, 48, 32).to_bits(), fresh(&hazy, &frame).to_bits());
// another source of the same size
assert_ne!(fresh(&other, &frame), fresh(&hazy, &frame));
assert_eq!(frame_airlight(&other, &info, &s, &frame, 48, 32).to_bits(), fresh(&other, &frame).to_bits());
// the same source, cropped to its darker half
let mut cropped = frame.clone();
cropped.crop.rect.x1 = 0.5;
assert_ne!(fresh(&hazy, &cropped), fresh(&hazy, &frame));
assert_eq!(frame_airlight(&hazy, &info, &s, &cropped, 48, 32).to_bits(), fresh(&hazy, &cropped).to_bits());
}
#[test]
fn exposure_doubles() {
let mut s = DevelopSettings::default();
+3 -1
View File
@@ -398,7 +398,9 @@ fn box_down(m: &Image<u8>, f: usize) -> Plane {
/// Bilinear sample at continuous pixel coordinates (pixel centres at +0.5), edges extended.
fn bilinear(p: &Plane, x: f32, y: f32) -> f32 {
let (fx, fy) = (x - 0.5, y - 0.5);
// (held a pixel outside, which reads the same edge: an infinite coordinate reads the edge
// instead of a NaN weight and a neighbour index past isize::MAX)
let (fx, fy) = (x.clamp(-1.0, p.width as f32 + 1.0) - 0.5, y.clamp(-1.0, p.height as f32 + 1.0) - 0.5);
let (x0, y0) = (fx.floor(), fy.floor());
let (tx, ty) = (fx - x0, fy - y0);
let (x0, y0) = (x0 as isize, y0 as isize);
+44 -5
View File
@@ -407,8 +407,9 @@ pub(crate) fn fingerprint(img: &Rgb32f) -> u64 {
let mut h: u64 = 0xcbf2_9ce4_8422_2325 ^ ((img.width as u64) << 32 | img.height as u64);
let n = img.data.len();
for i in 0..512usize {
let p = img.data[(i.wrapping_mul(2_654_435_761) ^ i) % n.max(1)];
for c in p {
// (an empty image has no pixels to sample: its dimensions are the fingerprint)
let Some(p) = img.data.get((i.wrapping_mul(2_654_435_761) ^ i) % n.max(1)) else { break };
for c in *p {
h ^= c.to_bits() as u64;
h = h.wrapping_mul(0x0100_0000_01b3);
}
@@ -578,9 +579,13 @@ pub fn defringe(img: &mut Rgb32f, s: &DevelopSettings, scale: f64) {
return;
}
let (w, h) = (img.width, img.height);
let max_amt = o.defringe_purple_amount.max(o.defringe_green_amount);
// fringe width in source px grows with the amount (1..7 px), in output px at least one
let radius = ((1.0 + max_amt * 0.3) * scale).round().max(1.0) as usize;
// as far as the sliders go (0–20): an amount from a damaged catalog or sidecar made the
// radius overflow
let max_amt = o.defringe_purple_amount.max(o.defringe_green_amount).min(20.0);
// fringe width in source px grows with the amount (1..7 px), in output px at least one (and
// never wider than the picture)
let radius = ((1.0 + max_amt * 0.3) * scale).round();
let radius = if radius.is_finite() { radius.clamp(1.0, w.max(h).max(1) as f64) as usize } else { 1 };
// perceptual lightness and its local range (dilated contrast)
let lab: Vec<[f32; 3]> = img.data.iter().map(|p| oklab_from_2020([p[0].max(0.0), p[1].max(0.0), p[2].max(0.0)])).collect();
let l = Plane { width: w, height: h, data: lab.iter().map(|p| p[0]).collect() };
@@ -648,6 +653,13 @@ fn min_filter(p: &Plane, r: usize, max: bool) -> Plane {
mod tests {
use super::*;
// An empty image (no pixel to sample) has a fingerprint, its dimensions' (indexing its first pixel panicked)
#[test]
fn an_empty_image_has_a_fingerprint() {
assert_eq!(fingerprint(&Rgb32f::new(0, 0)), fingerprint(&Rgb32f::new(0, 0)));
assert_ne!(fingerprint(&Rgb32f::new(0, 3)), fingerprint(&Rgb32f::new(3, 0)));
}
/// splitmix64: deterministic pseudo-random numbers for the warp tests.
struct Rng(u64);
@@ -1095,4 +1107,31 @@ mod tests {
assert!(chroma(img.get(20, 10)) < chroma(purple) * 0.3);
assert!((chroma(flat2.get(20, 10)) - chroma(purple)).abs() < 1e-4);
}
/// Found by fuzzing catalog snapshots: a defringe amount far past the sliders' 20 (from a
/// damaged catalog or sidecar) made the filter's radius overflow (`x + r`), which panics.
/// It acts as the sliders' end.
#[test]
fn defringe_amounts_past_the_sliders_act_as_their_end() {
let purple = [0.25f32, 0.05, 0.45];
let scene = Rgb32f::from_fn(40, 20, |x, _| {
if x < 20 {
[0.9; 3]
} else if x == 20 {
purple
} else {
[0.02; 3]
}
});
let at = |amount: f64, scale: f64| {
let mut s = DevelopSettings::default();
s.optics.defringe_purple_amount = amount;
s.optics.defringe_green_amount = amount / 2.0;
let mut img = scene.clone();
defringe(&mut img, &s, scale);
img
};
assert_eq!(at(1e300, 1.0).data, at(20.0, 1.0).data);
assert_eq!(at(f64::MAX, 1e300).data.len(), 800, "a huge scale too");
}
}
+65 -3
View File
@@ -18,10 +18,27 @@ use crate::geometry::Frame;
#[inline]
fn smooth(e0: f32, e1: f32, x: f32) -> f32 {
// Zero feather is a hard edge, including when the pixel lies exactly on the radius.
// Dividing 0 by 0 here would give that pixel a NaN alpha, even at zero opacity.
if e0 >= e1 {
return if x < e1 { 0.0 } else { 1.0 };
}
let t = ((x - e0) / (e1 - e0)).clamp(0.0, 1.0);
t * t * (3.0 - 2.0 * t)
}
/// Most a spot's size counts for, in long edges: a size read from a sidecar isn't held to the slider's range, and one
/// far past the picture (or infinite) made the radius, the blur it heals with and the patch it reads from unbounded
/// (issue #1519). Two long edges cover the whole frame from anywhere in it.
const MAX_SPOT_SIZE: f64 = 2.0;
/// A spot's radius in output pixels, `ppl` output pixels to the long edge: at least a pixel (a size that is not a
/// number too), at most [`MAX_SPOT_SIZE`] long edges.
fn radius_px(size: f64, ppl: f64) -> f64 {
let size = if size.is_nan() { 0.0 } else { size.min(MAX_SPOT_SIZE) };
(size * ppl).max(1.0)
}
/// Mean colour of `img` in a disc/annulus.
fn ring_stats(img: &Rgb32f, c: (f32, f32), r0: f32, r1: f32) -> Option<Vec<[f32; 3]>> {
let mut out = Vec::new();
@@ -99,7 +116,7 @@ pub fn pick_source(
let img = frame.sample(src, w, h);
let (to_out, to_norm) = (frame.norm_to_out(w, h), frame.out_to_norm(w, h));
let t = to_out.apply(*spot.points.first()?);
let r = (spot.size * frame.px_per_long(w)).max(1.0) as f32;
let r = radius_px(spot.size, frame.px_per_long(w)) as f32;
let avoid = avoid.map(|o| {
let (a, b) = (to_out.apply(Point::new(0.0, 0.0)), to_out.apply(o));
((b.x - a.x) as f32, (b.y - a.y) as f32)
@@ -165,7 +182,7 @@ pub fn window_for_reads_checked(
let boxes: Vec<([f64; 4], [f64; 4])> = spots
.iter()
.filter_map(|spot| {
let r = (spot.size * ppl).max(1.0);
let r = radius_px(spot.size, ppl);
let pts: Vec<_> = spot.points.iter().map(|p| to_out.apply(*p)).collect();
let first = pts.first()?;
let (mut x0, mut y0, mut x1, mut y1) = (first.x, first.y, first.x, first.y);
@@ -261,7 +278,7 @@ pub fn apply(img: &mut Rgb32f, spots: &[Spot], frame: &Frame, ppl: f64) {
let (w, h) = (img.width, img.height);
let to_out = frame.norm_to_out(w, h);
for spot in spots {
let r = (spot.size * ppl).max(1.0) as f32;
let r = radius_px(spot.size, ppl) as f32;
let pts: Vec<(f32, f32)> = spot
.points
.iter()
@@ -369,6 +386,51 @@ mod tests {
use super::*;
use lightcraft_develop::DevelopSettings;
#[test]
fn hard_edge_spots_at_zero_opacity_leave_every_pixel_unchanged() {
for mode in [SpotMode::Clone, SpotMode::Heal, SpotMode::Remove, SpotMode::Fill] {
for (cx, cy) in [(32.5, 24.5), (0.5, 0.5)] {
let original = Rgb32f::from_fn(64, 48, |x, y| [0.2 + x as f32 / 128.0, 0.3 + y as f32 / 128.0, 0.4]);
let mut img = original.clone();
let frame = Frame::new(64, 48, &DevelopSettings::default(), true);
let spot = Spot {
mode,
points: vec![Point::new(cx / 64.0, cy / 48.0)],
size: 5.0 / 64.0,
feather: 0.0,
opacity: 0.0,
source_offset: Some(Point::new(8.0 / 64.0, 0.0)),
};
apply(&mut img, &[spot], &frame, frame.px_per_long(64));
for (i, (actual, expected)) in img.data.iter().zip(&original.data).enumerate() {
assert_eq!(actual, expected, "zero-opacity {mode:?} at ({cx}, {cy}), pixel {i}");
}
}
}
}
#[test]
fn hard_edge_clone_copies_inside_and_preserves_pixels_on_the_radius() {
for (w, h, cx, cy, radius) in [(64, 48, 32.5, 24.5, 5.0), (128, 96, 64.5, 48.5, 10.0)] {
let original = Rgb32f::from_fn(w, h, |x, y| [0.2 + x as f32 / w as f32, y as f32 / h as f32, 0.4]);
let mut img = original.clone();
let frame = Frame::new(w, h, &DevelopSettings::default(), true);
let spot = Spot {
mode: SpotMode::Clone,
points: vec![Point::new(cx / w as f64, cy / h as f64)],
size: radius / w as f64,
feather: 0.0,
opacity: 100.0,
source_offset: Some(Point::new(0.125, 0.0)),
};
apply(&mut img, &[spot], &frame, frame.px_per_long(w));
let (x, y) = (cx as usize, cy as usize);
assert_eq!(img.get(x, y), original.get(x + w / 8, y), "inside copies the source at {w}px");
assert_eq!(img.get(x + radius as usize, y), original.get(x + radius as usize, y), "radius stays unchanged at {w}px");
assert!(img.data.iter().flatten().all(|v| v.is_finite()));
}
}
#[test]
fn remove_erases_a_dot() {
// flat grey with a gentle gradient and a black dot in the middle
+24
View File
@@ -711,3 +711,27 @@ fn stored_process_numbers_render_with_a_known_process() {
}
}
}
/// A crop a hair wide (or of no size, as a command or a sidecar can set) shown to fit made the
/// tone, clarity and dehaze stages blur with sigmas of millions of pixels (infinite for no size):
/// the box width's square overflowed (a panic in debug builds, a radius of billions of pixels and
/// a hung render in release ones). The blur is held to twice the picture's size: the render is
/// quick and its pixels are numbers.
#[test]
fn sliver_crops_render_quickly_with_wide_stages_on() {
let src = scene();
for (rect, angle) in [([0.0, 0.0, 1e-9, 1.0], 45.0), ([0.5, 0.5, 0.5, 0.5], 0.0), ([0.0, 0.0, 1e-5, 1.0], -30.0)] {
let mut s = DevelopSettings::default();
s.crop.geometry.rect = lightcraft_geom::Rect::new(rect[0], rect[1], rect[2], rect[3]);
s.crop.geometry.angle = angle;
s.effects.clarity = 80.0;
s.effects.dehaze = 50.0;
s.effects.texture = 40.0;
s.light.highlights = -60.0;
s.light.shadows = 60.0;
let t = std::time::Instant::now();
let r = render(&src, &SourceInfo::default(), &s, &RenderRequest::fit(280, 180));
assert!(t.elapsed().as_secs_f64() < 20.0, "{rect:?}: {:?}", t.elapsed());
assert!(r.image.width >= 1 && r.image.height >= 1, "{rect:?}");
}
}
+11 -7
View File
@@ -199,13 +199,18 @@ pub fn apply(img: &mut Rgba8, o: Overlay, plan: &Plan<'_>, mask: Option<&Plane>,
}
}
Overlay::AllMasks { view, color, opacity } => {
if let Some(planes) = masks {
for a in planes {
if (a.width, a.height) == (img.width, img.height) {
mask_view(img, a, view, color, opacity);
}
// one view of the union of the masks: painting the planes one after another
// overwrites (White on Black, Image on Black…) or compounds (Color) the earlier ones
let mut union: Option<Plane> = None;
for a in masks.unwrap_or_default().iter().filter(|a| (a.width, a.height) == (img.width, img.height)) {
match &mut union {
None => union = Some(a.clone()),
Some(u) => u.data.iter_mut().zip(&a.data).for_each(|(u, a)| *u = u.max(*a)),
}
}
if let Some(u) = &union {
mask_view(img, u, view, color, opacity);
}
}
}
}
@@ -425,7 +430,6 @@ mod tests {
components: vec![MaskComponent { name: None, op: MaskOp::Add, invert: false, shape: right }],
..Default::default()
});
let plain = render(&src, &info, &s, &RenderRequest::fit(80, 40)).image;
let o = Overlay::AllMasks { view: MaskView::WhiteOnBlack, color: [255, 0, 0], opacity: 100 };
let img = render(&src, &info, &s, &RenderRequest { overlay: o, ..RenderRequest::fit(80, 40) }).image;
let (in_l, in_r, mid) = (20 * 80 + 12, 20 * 80 + 60, 20 * 80 + 40);
@@ -435,7 +439,7 @@ mod tests {
// a hidden mask is skipped
s.masks[1].visible = false;
let img2 = render(&src, &info, &s, &RenderRequest { overlay: o, ..RenderRequest::fit(80, 40) }).image;
assert_eq!(img2.data[in_r], plain.data[in_r], "hidden right mask not painted");
assert_eq!(img2.data[in_r], [0, 0, 0, 255], "hidden right mask not painted: black like everything outside");
assert_eq!(img2.data[in_l], [255; 4], "visible left mask still painted");
// parts round trip and a stable, distinct cache key
let (k, v) = o.to_parts();
+12 -2
View File
@@ -9,6 +9,7 @@
//! (`<2 hex>/<32 hex>.jpg` and the `<32 hex>.tmp…` temp files of [`DiskCache::put`], see
//! [`is_cache_file`]): the directory may be shared with other files (e.g. a library opened on a
//! photo folder that already has a `thumbs/` folder), and those are never touched (issue #98).
//! A shard directory that is a symlink or junction is not followed (issue #1504).
use std::path::{Path, PathBuf};
use std::sync::Mutex;
@@ -65,7 +66,7 @@ impl DiskCache {
pub fn get(&self, key: Hash128) -> Option<Rgba8> {
let p = self.path(key);
let Ok(bytes) = std::fs::read(&p) else {
let Some(bytes) = p.parent().filter(|d| is_real_dir(d)).and_then(|_| std::fs::read(&p).ok()) else {
self.misses.fetch_add(1, Ordering::Relaxed);
return None;
};
@@ -91,7 +92,7 @@ impl DiskCache {
let Some(bytes) = encode_jpeg(img) else { return };
let p = self.path(key);
let Some(parent) = p.parent() else { return };
if std::fs::create_dir_all(parent).is_err() {
if std::fs::create_dir_all(parent).is_err() || !is_real_dir(parent) {
return;
}
// Serialize the file replacement and its size accounting. A repeated render of
@@ -124,6 +125,10 @@ impl DiskCache {
if !is_shard_name(sub_name) {
continue;
}
// a shard that is a symlink (or junction) leads outside the cache: never walk it
if !is_real_dir(&sub.path()) {
continue;
}
let Ok(files) = std::fs::read_dir(sub.path()) else { continue };
for f in files.flatten() {
let name = f.file_name();
@@ -173,6 +178,11 @@ impl DiskCache {
}
}
/// Whether `p` is a directory itself, not a symlink or Windows reparse point to one.
fn is_real_dir(p: &Path) -> bool {
std::fs::symlink_metadata(p).is_ok_and(|m| m.is_dir())
}
fn is_lower_hex(s: &str) -> bool {
s.bytes().all(|b| b.is_ascii_digit() || (b'a'..=b'f').contains(&b))
}
+27
View File
@@ -380,6 +380,33 @@ fn disk_cache_never_touches_foreign_files() {
let _ = std::fs::remove_dir_all(&dir);
}
/// A shard directory that is a symlink to a folder outside the cache is neither listed, cleared,
/// pruned, read nor written (issue #1504).
#[cfg(unix)]
#[test]
fn disk_cache_ignores_symlinked_shard_directories() {
let dir = temp_dir("symlink-shard");
let outside = temp_dir("symlink-outside");
std::fs::create_dir_all(&dir).unwrap();
std::fs::create_dir_all(&outside).unwrap();
let key = hash_bytes(&[9, 9]);
let hex = key.to_string();
let sentinel = outside.join(format!("{hex}.jpg"));
std::fs::write(&sentinel, vec![7u8; 50_000]).unwrap();
std::os::unix::fs::symlink(&outside, dir.join(&hex[..2])).unwrap();
let d = DiskCache::new(&dir, 10);
assert_eq!(d.size(), 0, "files behind the link are not the cache's");
d.prune();
assert!(sentinel.exists(), "prune followed the link");
assert!(d.get(key).is_none(), "get read through the link");
d.put(key, &gradient(16, 16, 1));
assert_eq!(std::fs::metadata(&sentinel).unwrap().len(), 50_000, "put wrote through the link");
d.clear();
assert!(sentinel.exists(), "clear followed the link");
let _ = std::fs::remove_dir_all(&dir);
let _ = std::fs::remove_dir_all(&outside);
}
/// [`JobPool::wait`] returns once a job finishes, which [`JobPool::try_recv`] then hands out, or
/// after the timeout.
#[test]
+76 -5
View File
@@ -3,8 +3,16 @@
use crate::resample::Pixel;
use crate::{Image, par_rows};
/// Box radii approximating a Gaussian of `sigma` with 3 passes (Kovesi / Wells).
/// The largest blur sigma (pixels) the box radii are made for. A box wider than any image (the
/// encoders' limit is 65535 px) blurs to the same flat result, and a larger sigma made the box
/// width's square overflow (from about 23 170 px: a panic where overflow is checked, a garbage
/// radius of billions of pixels, i.e. a hung render, where it isn't).
pub const MAX_SIGMA: f32 = 65_536.0;
/// Box radii approximating a Gaussian of `sigma` with 3 passes (Kovesi / Wells). A sigma that is
/// not a number counts as 0 (no blur); one above [`MAX_SIGMA`] (infinity too) as `MAX_SIGMA`.
pub fn box_radii(sigma: f32) -> [usize; 3] {
let sigma = if sigma.is_nan() { 0.0 } else { sigma.clamp(0.0, MAX_SIGMA) };
let n = 3.0f32;
let w_ideal = (12.0 * sigma * sigma / n + 1.0).sqrt();
let mut wl = w_ideal.floor() as i32;
@@ -12,7 +20,8 @@ pub fn box_radii(sigma: f32) -> [usize; 3] {
wl -= 1;
}
let wu = wl + 2;
let m_ideal = (12.0 * sigma * sigma - n * (wl * wl) as f32 - 4.0 * n * wl as f32 - 3.0 * n) / (-4.0 * wl as f32 - 4.0);
// (the square in i64: it passes i32 from a box 46 341 px wide)
let m_ideal = (12.0 * sigma * sigma - n * (i64::from(wl) * i64::from(wl)) as f32 - 4.0 * n * wl as f32 - 3.0 * n) / (-4.0 * wl as f32 - 4.0);
let m = m_ideal.round() as i32;
std::array::from_fn(|i| (((if (i as i32) < m { wl } else { wu }) - 1) / 2).max(0) as usize)
}
@@ -71,12 +80,24 @@ fn box_v<T: Pixel>(src: &Image<T>, dst: &mut Image<T>, r: usize) {
});
}
/// The box radii of [`gaussian`] on a `w × h` image (the GPU renderer's twin uses them too), or
/// `None` when the blur leaves the image as it is: `sigma` at most 0.3 or not a number, or an
/// empty image. A sigma wider than twice the image's long side counts as that: a crop a hair
/// wide shown to fit asked the tone and dehaze stages for sigmas of millions of pixels (infinite
/// for a crop of no size), which blur no flatter than that but cost boxes that wide on every row.
pub fn radii_for(sigma: f32, w: usize, h: usize) -> Option<[usize; 3]> {
if sigma.is_nan() || sigma <= 0.3 || w == 0 || h == 0 {
return None;
}
let cap = (w.max(h) as f32 * 2.0).max(1.0);
Some(box_radii(sigma.min(cap)))
}
/// Blur with Gaussian `sigma` (pixels). `sigma <= 0.3` returns a copy.
pub fn gaussian<T: Pixel>(img: &Image<T>, sigma: f32) -> Image<T> {
if sigma <= 0.3 || img.is_empty() {
let Some(radii) = radii_for(sigma, img.width, img.height) else {
return img.clone();
}
let radii = box_radii(sigma);
};
let mut a = img.clone();
par_rows(&mut a.data, img.width, |_, row| {
let mut tmp = Vec::with_capacity(row.len());
@@ -97,6 +118,56 @@ pub fn gaussian<T: Pixel>(img: &Image<T>, sigma: f32) -> Image<T> {
#[cfg(test)]
mod tests {
/// The box radii of the formula as it was, for the check below.
fn box_radii_before(sigma: f32) -> [usize; 3] {
let n = 3.0f32;
let w_ideal = (12.0 * sigma * sigma / n + 1.0).sqrt();
let mut wl = w_ideal.floor() as i32;
if wl % 2 == 0 {
wl -= 1;
}
let wu = wl + 2;
let m_ideal = (12.0 * sigma * sigma - n * (wl * wl) as f32 - 4.0 * n * wl as f32 - 3.0 * n) / (-4.0 * wl as f32 - 4.0);
let m = m_ideal.round() as i32;
std::array::from_fn(|i| (((if (i as i32) < m { wl } else { wu }) - 1) / 2).max(0) as usize)
}
/// A blur sigma from a degenerate edit or a deep zoom (found by a UI campaign: a render
/// worker's blur with a huge sigma) squared the box width past i32, which panics in a debug
/// build and gave a radius of billions of pixels in a release one. Huge, infinite and NaN
/// sigmas give bounded radii, and a blur with them is quick and stays finite; every sigma that
/// worked before keeps exactly its radii.
#[test]
fn huge_and_non_finite_sigmas_give_bounded_radii() {
for sigma in [26_800.0, 46_341.0, 1e5, 1e9, f32::MAX, f32::INFINITY, f32::NEG_INFINITY, f32::NAN, -5.0] {
let r = box_radii(sigma);
assert!(r.iter().all(|&r| r <= MAX_SIGMA as usize * 2), "{sigma}: {r:?}");
}
assert_eq!(box_radii(f32::NAN), [0, 0, 0]);
assert_eq!(box_radii(f32::NEG_INFINITY), [0, 0, 0]);
assert_eq!(box_radii(1e9), box_radii(MAX_SIGMA));
assert_eq!(box_radii(f32::INFINITY), box_radii(MAX_SIGMA));
// on an image, the blur is at most twice its long side; NaN and tiny sigmas don't blur
assert_eq!(radii_for(3.1e7, 1400, 900), Some(box_radii(2800.0)));
assert_eq!(radii_for(f32::INFINITY, 1400, 900), Some(box_radii(2800.0)));
assert_eq!(radii_for(40.0, 1400, 900), Some(box_radii(40.0)), "an ordinary sigma is as asked");
assert_eq!(radii_for(f32::NAN, 1400, 900), None);
assert_eq!(radii_for(0.2, 1400, 900), None);
assert_eq!(radii_for(5.0, 0, 900), None);
let p = crate::Plane { width: 4, height: 3, data: (0..12).map(|i| i as f32).collect() };
let t = std::time::Instant::now();
for sigma in [1e9, f32::INFINITY, f32::NAN] {
let b = gaussian(&p, sigma);
assert!(b.data.iter().all(|v| v.is_finite()), "{sigma}");
}
assert!(t.elapsed().as_secs_f64() < 30.0, "a huge sigma's blur is bounded");
let mut s = 0.0f32;
while s < 23_000.0 {
assert_eq!(box_radii(s), box_radii_before(s), "sigma {s}");
s = s * 1.013 + 0.07;
}
}
use super::*;
#[test]
+17 -2
View File
@@ -206,8 +206,11 @@ impl Rgb32f {
/// Bilinear sample at continuous pixel coordinates (pixel centres at +0.5).
#[inline]
pub fn sample_bilinear(&self, x: f32, y: f32) -> [f32; 3] {
let fx = x - 0.5;
let fy = y - 0.5;
// a pixel past the edge reads the edge pixel, so a coordinate held a pixel outside gives
// the same sample; an infinite one (a crop of no size's inverse mapping) then reads the
// edge too, instead of a NaN weight and a neighbour index past isize::MAX
let fx = x.clamp(-1.0, self.width as f32 + 1.0) - 0.5;
let fy = y.clamp(-1.0, self.height as f32 + 1.0) - 0.5;
let x0 = fx.floor();
let y0 = fy.floor();
let tx = fx - x0;
@@ -258,6 +261,18 @@ mod tests {
use super::*;
use lightcraft_geom::Orientation;
/// A sample at an infinite coordinate (the inverse mapping of a crop of no size) cast to
/// isize::MAX, and its right / lower neighbour overflowed (a panic where overflow is checked):
/// the neighbours saturate and the edge pixel comes back.
#[test]
fn bilinear_samples_at_infinite_coordinates_read_the_edge() {
let img = Rgb32f::from_fn(3, 2, |x, y| [x as f32, y as f32, 1.0]);
assert_eq!(img.sample_bilinear(f32::INFINITY, f32::INFINITY), [2.0, 1.0, 1.0]);
assert_eq!(img.sample_bilinear(f32::MAX, 0.0), [2.0, 0.0, 1.0]);
assert_eq!(img.sample_bilinear(f32::NEG_INFINITY, f32::NEG_INFINITY), [0.0, 0.0, 1.0]);
assert_eq!(img.sample_bilinear(1.5, 0.5), [1.0, 0.0, 1.0], "an ordinary sample is as before");
}
#[test]
fn crop_and_rotate() {
let img = Image::<u32>::from_fn(4, 3, |x, y| (y * 10 + x) as u32);
+14 -1
View File
@@ -79,7 +79,20 @@ pub fn demosaic(n: &Normalized, method: Method) -> Rgb32f {
}
match (&n.cfa, n.cpp) {
(_, 3) => Rgb32f { width: w, height: h, data: n.data.as_chunks::<3>().0.iter().map(|c| [c[0], c[1], c[2]]).collect() },
(_, cpp) if cpp != 1 => Rgb32f { width: w, height: h, data: n.data.chunks_exact(cpp).map(|c| [c[0], c[1], c[2]]).collect() },
// four colours develop from their first three; narrower data (refused by `RawImage::validate`, issue #1511)
// repeats its last sample instead of reading past the pixel
(_, cpp) if cpp != 1 => Rgb32f {
width: w,
height: h,
data: n
.data
.chunks_exact(cpp.max(1))
.map(|c| {
let s = |i: usize| c.get(i).or(c.last()).copied().unwrap_or(0.0);
[s(0), s(1), s(2)]
})
.collect(),
},
(None, _) => Rgb32f { width: w, height: h, data: n.data.iter().map(|&v| [v; 3]).collect() },
(Some(cfa), _) => {
let m = Mosaic { w, h, data: &n.data, cfa };
+11 -4
View File
@@ -207,7 +207,8 @@ pub(crate) fn decode_as(bytes: &[u8], mode: Mode, format: RawFormat) -> Result<R
}
let info = raw.image()?;
let (w, h, cpp) = (info.width as usize, info.height as usize, info.samples_per_pixel as usize);
if !(1..=4).contains(&cpp) {
// one sample (CFA or monochrome), RGB or four colours; two are neither and can't be developed (issue #1511)
if !matches!(cpp, 1 | 3 | 4) {
return Err(RawError::Unsupported(format!("{cpp} samples per pixel")));
}
let mut data = read_image_in(mode, bytes, &info, tiff.order, Packing::Msb)?;
@@ -251,12 +252,14 @@ pub(crate) fn decode_as(bytes: &[u8], mode: Mode, format: RawFormat) -> Result<R
[r, c] if (1..=16).contains(r) && (1..=16).contains(c) => (*r as usize, *c as usize),
_ => (1, 1),
};
let mut values: Vec<f32> = raw.f64s(t::BLACK_LEVEL).unwrap_or_default().into_iter().map(|v| v as f32).collect();
// (a FLOAT / DOUBLE tag can hold NaN or infinity: such a level reads as 0, in its place)
let level = |v: f64| Some(v as f32).filter(|v| v.is_finite()).unwrap_or(0.0);
let mut values: Vec<f32> = raw.f64s(t::BLACK_LEVEL).unwrap_or_default().into_iter().map(level).collect();
if values.len() != br * bc * cpp {
values = vec![values.first().copied().unwrap_or(0.0)];
}
let (br, bc) = if values.len() == 1 { (1, 1) } else { (br, bc) };
let floats = |tag| raw.f64s(tag).unwrap_or_default().into_iter().map(|v| v as f32).collect::<Vec<f32>>();
let floats = |tag| raw.f64s(tag).unwrap_or_default().into_iter().map(level).collect::<Vec<f32>>();
let black =
BlackLevel { repeat_rows: br, repeat_cols: bc, values, delta_h: floats(t::BLACK_LEVEL_DELTA_H), delta_v: floats(t::BLACK_LEVEL_DELTA_V) };
@@ -303,7 +306,11 @@ pub(crate) fn decode_as(bytes: &[u8], mode: Mode, format: RawFormat) -> Result<R
}
// the tag is anchored at the top-left of the ActiveArea (DNG spec); `RawImage::cfa` is anchored at the
// sensor origin (the crop to the active area shifts it back), so move the anchor out by the area's offset
Some(Cfa { width: cols, height: rows, pattern }.shifted(cols - active_area.x % cols, rows - active_area.y % rows))
// (LightCraft's own DNGs from before the writer followed the spec carry the sensor-origin pattern; they are marked by
// a bare "LightCraft" Software tag, the writer now says [`crate::dngwrite::SOFTWARE`], and keep their original reading)
let at_origin = Cfa { width: cols, height: rows, pattern };
let legacy_writer = [raw, ifd0].iter().any(|i| i.string(t::SOFTWARE).as_deref().map(str::trim) == Some("LightCraft"));
Some(if legacy_writer { at_origin } else { at_origin.shifted(cols - active_area.x % cols, rows - active_area.y % rows) })
} else {
None
};
+19 -6
View File
@@ -25,6 +25,10 @@ pub enum DngCompression {
},
}
/// Software tag of the DNGs this writer makes. A bare "LightCraft" marks older files, whose CFAPattern is anchored at
/// the sensor origin instead of the ActiveArea (the reader keeps their original reading).
pub const SOFTWARE: &str = "LightCraft DNG 2";
/// IEEE 754 binary32 → binary16 (round to nearest even; overflow → ±inf, NaN kept).
pub fn f32_to_f16(v: f32) -> u16 {
let b = v.to_bits();
@@ -108,18 +112,27 @@ pub fn write_dng(raw: &RawImage, opts: &DngWriteOptions) -> Result<Vec<u8>> {
ifd.set(t::PHOTOMETRIC, Value::Short(vec![if raw.cfa.is_some() { photometric::CFA } else { photometric::LINEAR_RAW }]));
ifd.set(t::DNG_VERSION, Value::Byte(vec![1, 4, 0, 0]));
ifd.set(t::DNG_BACKWARD_VERSION, Value::Byte(vec![1, 3, 0, 0]));
let make = raw.metadata.make.clone().unwrap_or_default();
let model = raw.metadata.model.clone().unwrap_or_default();
let make = raw.metadata.make.as_deref().unwrap_or_default();
let model = raw.metadata.model.as_deref().unwrap_or_default();
if !make.is_empty() {
ifd.set(t::MAKE, Value::Ascii(make.clone()));
ifd.set(t::MAKE, Value::Ascii(make.to_owned()));
}
if !model.is_empty() {
ifd.set(t::MODEL, Value::Ascii(model.clone()));
ifd.set(t::MODEL, Value::Ascii(model.to_owned()));
}
let unique = format!("{make} {model}").trim().to_string();
let model_includes_make = !make.is_empty()
&& model.get(..make.len()).is_some_and(|prefix| prefix.eq_ignore_ascii_case(make))
&& model.get(make.len()..).is_some_and(|suffix| suffix.chars().next().is_none_or(|c| !c.is_alphanumeric() && c != '_'));
let unique = if model_includes_make || make.is_empty() {
model.to_owned()
} else if model.is_empty() {
make.to_owned()
} else {
format!("{make} {model}")
};
ifd.set(t::UNIQUE_CAMERA_MODEL, Value::Ascii(if unique.is_empty() { "LightCraft".into() } else { unique }));
ifd.set(t::ORIENTATION, Value::Short(vec![raw.orientation.to_exif()]));
ifd.set(t::SOFTWARE, Value::Ascii("LightCraft".into()));
ifd.set(t::SOFTWARE, Value::Ascii(SOFTWARE.into()));
let xmp = opts.xmp.clone().unwrap_or_else(|| lightcraft_meta::write_xmp(&raw.metadata, None));
ifd.set(t::XMP, Value::Byte(xmp.into_bytes()));
+9 -2
View File
@@ -56,7 +56,7 @@ pub use lightcraft_meta::Metadata;
pub use lightcraft_raster::Rgb32f;
pub use opcodes::{Opcode, OpcodeLists};
pub use preview::{
PREVIEW_PREFIX, PreviewColorSpace, PreviewRange, bounded_preview, embedded_preview, embedded_preview_color_space,
PREVIEW_PREFIX, PreviewColorSpace, PreviewRange, bounded_preview, embedded_preview, embedded_preview_color_space, embedded_preview_covering,
embedded_preview_dynamic_range_optimized, embedded_preview_from_range,
};
pub use semantic::{SemanticMask, semantic_masks};
@@ -715,6 +715,10 @@ impl RawImage {
if self.width == 0 || self.height == 0 || !(1..=4).contains(&self.cpp) {
return Err(RawError::Corrupt("bad dimensions".into()));
}
// two samples a pixel are neither one colour nor RGB: nothing downstream can develop them (issue #1511)
if self.cpp == 2 {
return Err(RawError::Unsupported("2 samples per pixel".into()));
}
if samples && self.data.len() != self.width * self.height * self.cpp {
return Err(RawError::Corrupt("data length mismatch".into()));
}
@@ -768,7 +772,10 @@ impl RawImage {
for s in 0..cpp {
let v = src(base + x * cpp + s);
let b = self.black.at(x, y, s, cpp);
row[x * cpp + s] = (v - b) * scale[s];
// a NaN or infinite sample (a float DNG's, or a black level a file got wrong)
// reads as black, so the demosaic and every stage after it see numbers
let n = (v - b) * scale[s];
row[x * cpp + s] = if n.is_finite() { n } else { 0.0 };
}
}
});
+126
View File
@@ -129,6 +129,76 @@ impl Opcode {
pub fn is_lens_correction(&self) -> bool {
matches!(self, Opcode::WarpRectilinear { .. } | Opcode::WarpRectilinear2 { .. } | Opcode::FixVignetteRadial { .. })
}
/// This lens correction re-expressed for the part `crop` of the `width × height` image it was written for: applied
/// to the cropped image it gives the crop of the corrected whole. The centre moves into the crop and the
/// coefficients are rescaled for the crop's farthest corner, the radius every coordinate is normalised by. Centres
/// use the warps' convention (0 and 1 at the first and last pixel), also for `FixVignetteRadial`, as the develop
/// pipeline reads them. `None` for other opcodes, an image or crop under 2 × 2, or a crop outside the image.
pub fn cropped(&self, width: usize, height: usize, crop: crate::Rect) -> Option<Opcode> {
let fits = crop.x.checked_add(crop.width).is_some_and(|r| r <= width) && crop.y.checked_add(crop.height).is_some_and(|b| b <= height);
if width < 2 || height < 2 || crop.width < 2 || crop.height < 2 || !fits {
return None;
}
let (aw, ah) = (width as f64 - 1.0, height as f64 - 1.0);
let (cw, ch) = (crop.width as f64 - 1.0, crop.height as f64 - 1.0);
// the centre in the crop, and the ratio of the crop's normalising radius to the image's
let reframe = |c: [f64; 2]| -> Option<([f64; 2], f64)> {
let (px, py) = (c[0] * aw, c[1] * ah);
let (qx, qy) = (px - crop.x as f64, py - crop.y as f64);
let s = max_radius(qx, qy, cw, ch) / max_radius(px, py, aw, ah);
(s.is_finite() && s > 0.0 && qx.is_finite() && qy.is_finite()).then_some(([qx / cw, qy / ch], s))
};
match self {
// source radius r·(kr0 + kr1 r² + kr2 r⁴ + kr3 r⁶) and tangential terms quadratic in r: with r = r'·s,
// kr_i scales by s^2i and kt by s
Opcode::WarpRectilinear { planes, center } => {
let (center, s) = reframe(*center)?;
let planes = planes
.iter()
.map(|k| {
let mut k = *k;
for (i, v) in k[..4].iter_mut().enumerate() {
*v *= s.powi(2 * i as i32);
}
k[4] *= s;
k[5] *= s;
k
})
.collect();
Some(Opcode::WarpRectilinear { planes, center })
}
// radial factor Σ kr_i r^i between the valid radii: kr_i scales by s^i, the radii by 1/s
Opcode::WarpRectilinear2 { planes, center, reciprocal, optional } => {
let (center, s) = reframe(*center)?;
let planes = planes
.iter()
.map(|k| {
let mut k = *k;
for (i, v) in k[..15].iter_mut().enumerate() {
*v *= s.powi(i as i32);
}
k[15] *= s;
k[16] *= s;
k[17] /= s;
k[18] /= s;
k
})
.collect();
Some(Opcode::WarpRectilinear2 { planes, center, reciprocal: *reciprocal, optional: *optional })
}
// gain 1 + Σ k_i r^(2i+2)
Opcode::FixVignetteRadial { k, center } => {
let (center, s) = reframe(*center)?;
let mut k = *k;
for (i, v) in k.iter_mut().enumerate() {
*v *= s.powi(2 * i as i32 + 2);
}
Some(Opcode::FixVignetteRadial { k, center })
}
_ => None,
}
}
}
/// The opcodes of a list that a reader applies, in order: all of them, minus the DNG 1.6 skip rule (specification
@@ -1032,4 +1102,60 @@ pub(crate) mod tests {
let bad = Opcode::Unknown { id: 14, flags: 1, params: vec![] };
assert!(applied_ops(&[bad, wr]).any(|o| matches!(o, Opcode::WarpRectilinear { .. })));
}
/// `cropped` re-expresses a lens correction for a crop: the corrected crop is the crop of the corrected whole. A
/// linear ramp makes bilinear sampling exact, so only the borders (where the warp reaches past the crop) differ.
#[test]
fn cropped_lens_corrections_correct_the_crop_like_the_whole() {
let (w, h) = (90usize, 64usize);
let crop = crate::Rect::new(7, 5, 70, 50);
let ramp = Rgb32f::from_fn(w, h, |x, y| [0.002 * x as f32 + 0.001 * y as f32 + 0.1, 0.3 + 0.0015 * y as f32, 0.2]);
let mut wr2 = [0.0; WR2_LEN];
wr2[..5].copy_from_slice(&[1.0, 0.01, -0.03, 0.0, 0.01]);
wr2[15] = 0.002;
wr2[16] = -0.001;
wr2[17] = 0.05;
wr2[18] = 0.9;
let warps = [
Opcode::WarpRectilinear { planes: vec![[0.97, 0.04, -0.01, 0.002, 0.003, -0.002]], center: [0.47, 0.53] },
Opcode::WarpRectilinear2 { planes: vec![wr2], center: [0.52, 0.46], reciprocal: false, optional: true },
Opcode::WarpRectilinear2 { planes: vec![wr2], center: [0.52, 0.46], reciprocal: true, optional: false },
];
for op in &warps {
let mut whole = ramp.clone();
apply_list3(std::slice::from_ref(op), &mut whole);
let re = op.cropped(w, h, crop).unwrap();
let mut part = ramp.clone().into_crop(crop.x, crop.y, crop.width, crop.height);
apply_list3(std::slice::from_ref(&re), &mut part);
let mut worst = 0f32;
for y in 6..crop.height - 6 {
for x in 6..crop.width - 6 {
let (a, b) = (whole.data[(y + crop.y) * w + x + crop.x], part.data[y * crop.width + x]);
worst = (0..3).map(|c| (a[c] - b[c]).abs()).fold(worst, f32::max);
}
}
assert!(worst < 1e-4, "{op:?} → {re:?}: {worst}");
}
// the vignette, in the warps' centre convention the develop pipeline reads it with
let gain = |k: &[f64; 5], c: [f64; 2], w: usize, h: usize, x: f64, y: f64| {
let (aw, ah) = (w as f64 - 1.0, h as f64 - 1.0);
let (px, py) = (c[0] * aw, c[1] * ah);
let r2 = ((x - px).powi(2) + (y - py).powi(2)) / max_radius(px, py, aw, ah).powi(2);
1.0 + (0..5).map(|i| k[i] * r2.powi(i as i32 + 1)).sum::<f64>()
};
let (k, center) = ([-0.3, 0.1, -0.05, 0.02, -0.01], [0.45, 0.55]);
let Some(Opcode::FixVignetteRadial { k: k2, center: c2 }) = (Opcode::FixVignetteRadial { k, center }).cropped(w, h, crop) else {
panic!("vignette")
};
for (x, y) in [(0.0, 0.0), (69.0, 49.0), (33.5, 20.0), (69.0, 0.0), (12.0, 40.0)] {
let (a, b) = (gain(&k, center, w, h, x + crop.x as f64, y + crop.y as f64), gain(&k2, c2, crop.width, crop.height, x, y));
assert!((a - b).abs() < 1e-12, "({x}, {y}): {a} vs {b}");
}
// no change for a full-frame "crop"; nothing for other opcodes, tiny images or a crop past the edge
assert_eq!(warps[0].cropped(w, h, crate::Rect::new(0, 0, w, h)).unwrap(), warps[0]);
assert!(Opcode::TrimBounds { top: 0, left: 0, bottom: 2, right: 2 }.cropped(w, h, crop).is_none());
assert!(warps[0].cropped(1, h, crate::Rect::new(0, 0, 1, h)).is_none());
assert!(warps[0].cropped(w, h, crate::Rect::new(30, 0, 70, 50)).is_none());
assert!(warps[0].cropped(w, h, crate::Rect::new(usize::MAX, 0, 70, 50)).is_none());
}
}
+114 -19
View File
@@ -65,31 +65,43 @@ pub fn embedded_preview_dynamic_range_optimized(bytes: &[u8]) -> Option<bool> {
crate::vendor::arw::dynamic_range_optimizer(&mn)
}
/// Whether `b` looks like a displayable (DCT) JPEG: SOI, and the first SOF marker is not lossless.
fn is_dct_jpeg(b: &[u8]) -> bool {
if b.len() < 4 || b[0] != 0xff || b[1] != 0xd8 {
return false;
/// The first frame (SOF) marker of a JPEG and its position, from the marker segments before the first scan.
fn first_frame(b: &[u8]) -> Option<(u8, usize)> {
if b.len() < 4 || !b.starts_with(&[0xff, 0xd8]) {
return None;
}
let mut i = 2;
while i + 4 <= b.len() {
if b[i] != 0xff {
return false;
let mut i = 2usize;
while i.checked_add(4)? <= b.len() {
if *b.get(i)? != 0xff {
return None;
}
let m = b[i + 1];
let m = *b.get(i + 1)?;
if m == 0xff {
i += 1;
continue;
}
match m {
0xc0..=0xc2 => return true,
0xc3 | 0xc5..=0xc7 | 0xcb | 0xcd..=0xcf => return false,
0xda | 0xd9 => return false,
0xc0..=0xc3 | 0xc5..=0xc7 | 0xcb | 0xcd..=0xcf => return Some((m, i)),
0xda | 0xd9 => return None,
_ => {}
}
let len = u16::from_be_bytes([b[i + 2], b[i + 3]]) as usize;
i += 2 + len;
let len = usize::from(u16::from_be_bytes([*b.get(i + 2)?, *b.get(i + 3)?]));
i = i.checked_add(2 + len)?;
}
false
None
}
/// Whether `b` looks like a displayable (DCT) JPEG: SOI, and the first SOF marker is not lossless.
fn is_dct_jpeg(b: &[u8]) -> bool {
first_frame(b).is_some_and(|(m, _)| matches!(m, 0xc0..=0xc2))
}
/// Width, height and component count of a displayable (DCT) JPEG, from its frame header.
fn jpeg_frame(b: &[u8]) -> Option<(u32, u32, u8)> {
let (_, i) = first_frame(b).filter(|(m, _)| matches!(m, 0xc0..=0xc2))?;
let at = |k: usize| b.get(i.checked_add(k)?).copied();
let be16 = |k: usize| Some(u32::from(u16::from_be_bytes([at(k)?, at(k + 1)?])));
Some((be16(7)?, be16(5)?, at(9)?))
}
/// Whether `b` is a JPEG XL file: a bare codestream or the ISO-BMFF container's signature box.
@@ -139,6 +151,33 @@ fn candidates<'a>(data: &'a [u8], ifd: &Ifd, base: u64, out: &mut Vec<&'a [u8]>)
/// The largest embedded preview, if any: a JPEG, or (DNG 1.7) a JPEG XL file — both decode with
/// `lightcraft_codecs::decode`.
pub fn embedded_preview(bytes: &[u8]) -> Option<Vec<u8>> {
preview_covering(bytes, None)
}
/// The smallest embedded JPEG whose long edge is at least `min_edge` and that looks like a copy of the largest preview
/// (its aspect ratio within 1 %, so not a letterboxed or cropped thumbnail, and as many colour components, so not a
/// grey mask or depth map), else [`embedded_preview`]. Only TIFF-based raws keep several: an A7R VI's 1616 × 1080
/// preview is 0.3 MB beside its 9984 × 6656 one (5.6 MB), so a stand-in up to 1616 px decodes about a twentieth of the data.
pub fn embedded_preview_covering(bytes: &[u8], min_edge: u32) -> Option<Vec<u8>> {
preview_covering(bytes, Some(min_edge))
}
/// Of the DCT JPEGs in `found`, the one with the shortest long edge of at least `min_edge` with `largest`'s aspect
/// ratio (within 1 %) and component count.
fn smallest_covering<'a>(found: &[&'a [u8]], largest: &[u8], min_edge: u32) -> Option<&'a [u8]> {
let (bw, bh, bc) = jpeg_frame(largest)?;
let like = |(w, h, c): (u32, u32, u8)| {
let (a, b) = (u64::from(w) * u64::from(bh), u64::from(h) * u64::from(bw));
w > 0 && h > 0 && c == bc && a.abs_diff(b) * 100 <= a
};
found
.iter()
.filter_map(|&s| jpeg_frame(s).filter(|&f| like(f) && f.0.max(f.1) >= min_edge).map(|(w, h, _)| (s, w.max(h))))
.min_by_key(|&(_, long)| long)
.map(|(s, _)| s)
}
fn preview_covering(bytes: &[u8], min_edge: Option<u32>) -> Option<Vec<u8>> {
if bytes.starts_with(b"FUJIFILMCCD-RAW") {
let j = crate::vendor::raf::header(bytes).ok()?.jpeg?;
return is_dct_jpeg(j).then(|| trim_eoi(j).to_vec());
@@ -173,10 +212,11 @@ pub fn embedded_preview(bytes: &[u8]) -> Option<Vec<u8>> {
if let Some(p) = crate::vendor::orf::preview(bytes) {
found.push(p);
}
let best = found
.into_iter()
.filter(|s| is_dct_jpeg(s) || is_jxl(s))
.max_by_key(|s| s.len())
let largest = found.iter().copied().filter(|s| is_dct_jpeg(s) || is_jxl(s)).max_by_key(|s| s.len());
let best = min_edge
.zip(largest)
.and_then(|(min, big)| smallest_covering(&found, big, min))
.or(largest)
.map(|s| if is_jxl(s) { s.to_vec() } else { trim_eoi(s).to_vec() });
// a TIFF raw we can't decode whose preview no tag points to (Leaf MOS, Epson ERF)
best.or_else(|| format.filter(|f| !f.is_supported()).and_then(|_| scan_for_jpeg(bytes)))
@@ -543,6 +583,61 @@ mod tests {
}
}
/// A JPEG like [`fake_jpeg`] whose frame header says `w` x `h` with `nc` components.
fn sized_jpeg(w: u16, h: u16, nc: u8, n: usize) -> Vec<u8> {
let mut j = vec![0xff, 0xd8, 0xff, 0xc0, 0x00, 8 + 3 * nc, 8];
j.extend_from_slice(&h.to_be_bytes());
j.extend_from_slice(&w.to_be_bytes());
j.push(nc);
for c in 0..nc {
j.extend_from_slice(&[c + 1, 0x11, 0]);
}
j.extend(std::iter::repeat_n(0x55u8, n));
j.extend_from_slice(&[0xff, 0xd9]);
j
}
/// A TIFF raw holding `jpegs` as JPEG-compressed strips: the first in IFD0, the others in sub-IFDs.
fn tiff_with_jpegs(jpegs: &[&[u8]]) -> Vec<u8> {
let ifd = |j: &[u8]| {
let mut ifd = IfdBuilder::new();
ifd.set(t::COMPRESSION, Value::Short(vec![6]));
ifd.set(t::IMAGE_WIDTH, Value::Long(vec![1]));
ifd.set(t::IMAGE_LENGTH, Value::Long(vec![1]));
ifd.set_image(ImageData::Strips { rows_per_strip: 1, strips: vec![j.to_vec()] });
ifd
};
let mut ifd0 = ifd(jpegs[0]);
for j in &jpegs[1..] {
ifd0.add_sub_ifd(ifd(j));
}
TiffWriter::default().write(&[ifd0]).unwrap()
}
/// A stand-in decodes the smallest JPEG that covers its box and shows the same picture as the largest: not a
/// thumbnail of another shape (a letterboxed one) or a grey mask, and the largest when no smaller one is big enough.
#[test]
fn a_stand_in_takes_the_smallest_covering_jpeg_of_the_same_shape() {
let (big, small, boxed) = (sized_jpeg(9984, 6656, 3, 500), sized_jpeg(1616, 1080, 3, 50), sized_jpeg(160, 120, 3, 10));
// a ProRAW DNG keeps grey semantic masks of the picture's shape as JPEGs too
let mask = sized_jpeg(768, 512, 1, 20);
let file = tiff_with_jpegs(&[&small, &big, &boxed, &mask]);
assert_eq!(embedded_preview(&file).unwrap(), big, "unchanged: the largest");
assert_eq!(embedded_preview_covering(&file, 1600).unwrap(), small, "1616 px covers a 1600 px box");
assert_eq!(embedded_preview_covering(&file, 1616).unwrap(), small);
assert_eq!(embedded_preview_covering(&file, 1617).unwrap(), big, "too small: the largest");
assert_eq!(embedded_preview_covering(&file, u32::MAX).unwrap(), big);
assert_eq!(embedded_preview_covering(&file, 100).unwrap(), small, "the 4:3 thumbnail and the grey mask are not this picture");
// a small copy of the same shape (within 1 %) is taken
let thumb = sized_jpeg(160, 107, 3, 10);
let file = tiff_with_jpegs(&[&small, &big, &thumb]);
assert_eq!(embedded_preview_covering(&file, 100).unwrap(), thumb);
assert_eq!(embedded_preview_covering(&tiff_with_jpegs(&[&big]), 100).unwrap(), big, "one preview: that one");
for n in (0..file.len()).step_by(7) {
let _ = embedded_preview_covering(&file[..n], 100);
}
}
/// CR3 with a full-size JPEG track: that JPEG wins over the smaller `PRVW` box.
#[test]
fn cr3_prefers_the_full_size_jpeg_track() {
+18 -3
View File
@@ -13,9 +13,9 @@
//! tracks (keeping the raw track's coding headers and quantization maps, and the timed-metadata
//! track where Canon puts the as-shot white balance), a RAF's CFA strip (keeping its compression
//! header) and a Panasonic RW2's raw data. Formats whose header probe reads samples (NEF: masked
//! columns; ORF: bit depth and CFA phase; PEF: dark borders) and anything else get
//! [`HeaderPlan::Whole`], as does a structure that is incomplete, inconsistent or keeps metadata
//! inside its sample data: the plan never guesses.
//! columns; ORF: bit depth and CFA phase; PEF: dark borders; ARW6, for its header plan only: the
//! start of every tile) and anything else get [`HeaderPlan::Whole`], as does a structure that is
//! incomplete, inconsistent or keeps metadata inside its sample data: the plan never guesses.
use std::collections::HashSet;
@@ -236,6 +236,8 @@ fn tiff_plan(bytes: &[u8], loaded: &Ranges, preview: bool) -> HeaderPlan {
let chunks: Vec<(u64, u64)> = match ifd.image() {
// a preview's strips are read when the preview is wanted: only the raw image is skipped
Ok(info) if preview && !matches!(info.photometric, t::photometric::CFA | t::photometric::LINEAR_RAW) => continue,
// (Sony Compressed RAW 2: the header probe decodes the start of every tile; a preview reads none)
Ok(info) if !preview && info.compression == 32766 => return HeaderPlan::Whole("an ARW6 header probe reads its tiles"),
Ok(info) => info.chunks(len).iter().map(|c| (c.offset, c.len)).collect(),
// a raw IFD without the tags a TIFF image needs (an older CR2's) still lists its strips
Err(_) => match (ifd.u64s(t::STRIP_OFFSETS), ifd.u64s(t::STRIP_BYTE_COUNTS)) {
@@ -550,4 +552,17 @@ mod tests {
raf[104..108].copy_from_slice(&1000u32.to_be_bytes());
assert_eq!(run(&raf, 160, 0).0, HeaderPlan::Whole("RAF raw container"));
}
/// An ARW6 header probe reads every tile's headers and the first lines of its data, which lie in the strip a
/// plan would skip: here the second tile, past a first one of bulk size.
#[test]
fn arw6_is_described_from_the_whole_file() {
use crate::vendor::arw6_testenc::{Qis, arw6_file, one_tile_quantised};
let (tile, _) = one_tile_quantised(64, 48, 0, Qis::REAL);
let mut first = tile.clone();
first.resize(BULK as usize, 0);
let file = arw6_file(&[(0, 0, 64, 48, first), (64, 0, 64, 48, tile)], 128, 48);
assert!(crate::probe_info(&file).is_ok());
assert_eq!(run(&file, 16, 0).0, HeaderPlan::Whole("an ARW6 header probe reads its tiles"));
}
}
+76 -19
View File
@@ -47,6 +47,13 @@ pub fn check_image(data: &[u8], info: &ImageInfo) -> Result<usize> {
if chunks.is_empty() {
return Err(RawError::Corrupt("no image data chunks".into()));
}
// every visible sample needs a chunk: a short offsets table would leave zero-filled (black) regions
let (across, down) = info.grid();
let planes = if info.planar == 2 { cpp } else { 1 };
let expected = (across as usize).saturating_mul(down as usize).saturating_mul(planes);
if chunks.len() < expected {
return Err(RawError::Corrupt(format!("{} image data chunks, expected {expected}", chunks.len())));
}
// Plausibility bound before allocating: no supported coding stores more than ~2000 samples per byte
// (Deflate of constant data is the extreme), so tiny files cannot trigger huge allocations.
let available: u64 = chunks.iter().map(|c| chunk_bytes(data, c).map_or(0, |s| s.len() as u64)).sum();
@@ -102,21 +109,9 @@ pub fn read_image(data: &[u8], info: &ImageInfo, order: ByteOrder, packing: Pack
chunks.par_iter().map(|c| decode_chunk(data, info, c, order, packing, ccpp, bits, float).map(|p| (*c, p))).collect();
let mut out_u16 = if float { Vec::new() } else { vec![0u16; total] };
let mut out_f32 = if float { vec![0f32; total] } else { Vec::new() };
let mut ok = 0usize;
let mut first_err = None;
for r in decoded {
let (c, px) = match r {
Ok(v) => v,
Err(e) => {
// a JPEG XL tile that cannot be decoded is not a partial image: the file stays preview-only
if info.compression == comp::JPEG_XL {
return Err(e);
}
first_err.get_or_insert(e);
continue;
}
};
ok += 1;
// a chunk that cannot be decoded is not a partial image: the file stays preview-only
let (c, px) = r?;
let (cw, ch) = (c.width as usize, c.height as usize);
let (x0, y0) = (c.x as usize, c.y as usize);
if x0 >= w || y0 >= h {
@@ -149,9 +144,6 @@ pub fn read_image(data: &[u8], info: &ImageInfo, order: ByteOrder, packing: Pack
}
}
}
if ok == 0 {
return Err(first_err.unwrap_or_else(|| RawError::Corrupt("no decodable chunks".into())));
}
Ok(if float { RawData::F32(out_f32) } else { RawData::U16(out_u16) })
}
@@ -168,8 +160,8 @@ fn decode_chunk(data: &[u8], info: &ImageInfo, c: &Chunk, order: ByteOrder, pack
comp::JPEG => {
let f = ljpeg::decode(src, n.saturating_mul(2).max(1 << 16))?;
if f.data.len() < n {
// Some writers encode edge tiles smaller than nominal: accept a frame that covers the rows it has.
if f.data.is_empty() || f.data.len() % (cw * cpp) != 0 {
// Some writers encode edge tiles smaller than nominal: accept a frame that covers the rows inside the image.
if f.data.is_empty() || f.data.len() % (cw * cpp) != 0 || f.data.len() / (cw * cpp) < vh {
return Err(RawError::Corrupt(format!("lossless JPEG tile has {} samples, expected {n}", f.data.len())));
}
}
@@ -377,4 +369,69 @@ mod lossy_tests {
assert!((px(20, 3)[0] as i32 - 220).abs() <= 3, "second tile: {:?}", px(20, 3));
assert!((px(10, 5)[1] as i32 - 150).abs() <= 6, "green ramp: {:?}", px(10, 5));
}
fn strip_info(compression: u16, offsets: Vec<u64>, byte_counts: Vec<u64>) -> ImageInfo {
ImageInfo {
width: 4,
height: 4,
bits_per_sample: vec![8],
samples_per_pixel: 1,
compression,
photometric: 1,
planar: 1,
predictor: 1,
sample_format: 1,
new_subfile_type: 0,
layout: Layout::Strips { rows_per_strip: 2 },
offsets,
byte_counts,
}
}
/// A missing strip or a strip that fails to decode is an error, not a black band in a "successful" image.
#[test]
fn missing_or_failed_chunks_are_errors() {
let file = vec![9u8; 8 + 8];
// both strips present
let ok = strip_info(comp::NONE, vec![0, 8], vec![8, 8]);
assert!(read_image(&file, &ok, ByteOrder::Little, Packing::Msb).is_ok());
// the second strip has no offset
let missing = strip_info(comp::NONE, vec![0], vec![8]);
assert!(read_image(&file, &missing, ByteOrder::Little, Packing::Msb).is_err());
// the header-only read rejects it too, as the full decode will
assert!(read_image_in(crate::Mode::Header, &file, &missing, ByteOrder::Little, Packing::Msb).is_err());
// the second strip starts past the end of the file
let past_end = strip_info(comp::NONE, vec![0, 100], vec![8, 8]);
assert!(read_image(&file, &past_end, ByteOrder::Little, Packing::Msb).is_err());
}
/// A lossless-JPEG tile may be coded only as tall as the part inside the image, never shorter.
#[test]
fn short_lossless_jpeg_tiles_are_errors() {
let tile = |rows: usize| ljpeg::encode(&vec![300u16; 8 * rows], 8, rows, 1, 12, 1, 0);
let decode = |height: u32, rows: usize| {
let t = tile(rows);
let info = ImageInfo {
width: 8,
height,
bits_per_sample: vec![12],
samples_per_pixel: 1,
compression: comp::JPEG,
photometric: 32803,
planar: 1,
predictor: 1,
sample_format: 1,
new_subfile_type: 0,
layout: Layout::Tiles { tile_width: 8, tile_height: 8 },
offsets: vec![0],
byte_counts: vec![t.len() as u64],
};
read_image(&t, &info, ByteOrder::Little, Packing::Msb)
};
// an edge tile coded at its in-image height, and a full tile
assert_eq!(decode(6, 6), Ok(RawData::U16(vec![300; 8 * 6])));
assert_eq!(decode(8, 8), Ok(RawData::U16(vec![300; 8 * 8])));
// half of the visible rows are missing
assert!(decode(8, 4).is_err());
}
}
+4
View File
@@ -1115,6 +1115,10 @@ mod tests {
("cr3-canon-m50-craw.cr3", 0x9aacbfe66f505e1bu64),
("cr3-canon-r8-raw.cr3", 0x63c1c6d8d1eb2312u64),
("cr3-canon-r8-craw.cr3", 0xf836a663a795a375u64),
("cr3-canon-rp-raw.cr3", 0x5cb0f46404b8b663u64),
("cr3-canon-rp-craw.cr3", 0xf48df06ca1d67382u64),
("cr3-canon-rp-raw-crop.cr3", 0x46808a9de3b3dcc7u64),
("cr3-canon-rp-craw-crop.cr3", 0x39a60c4efed4e1f4u64),
] {
let path = std::path::Path::new(&directory).join(name);
let bytes = std::fs::read(path).unwrap();
+1
View File
@@ -12,6 +12,7 @@ pub mod crx;
pub(crate) mod crx_wavelet;
pub mod kodak;
pub mod nef;
mod nef_lens;
pub mod nefc;
pub mod orf;
pub mod pef;
+50 -2
View File
@@ -177,7 +177,8 @@ pub(crate) fn decode(bytes: &[u8], mode: Mode) -> Result<RawImage> {
(Some([2, 2]), Some(p)) if p.len() == 4 && p.iter().all(|&c| c <= 2) => Cfa { width: 2, height: 2, pattern: p.to_vec() },
_ => Cfa::bayer_static("RGGB"),
};
let body = measured_body(&ifd0.string(t::MODEL).unwrap_or_default(), w, h, bits);
let model = ifd0.string(t::MODEL).unwrap_or_default();
let body = measured_body(&model, w, h, bits);
let active_w = trailing_masked_columns(samples, w, bits);
// the white level is measured from the whole image (or known for a measured body); the probe (no samples in its
// result) takes the nominal one
@@ -212,7 +213,11 @@ pub(crate) fn decode(bytes: &[u8], mode: Mode) -> Result<RawImage> {
color: ColorData::default(),
wb_multipliers: wb,
linearized: false,
opcodes: OpcodeLists::default(),
// Nikon's in-file distortion correction, about the optical centre (which follows the sensor shift)
opcodes: OpcodeLists {
list3: super::nef_lens::distortion(&model, raw, Rect::new(0, 0, active_w, h), crop).into_iter().collect(),
..Default::default()
},
metadata,
};
img.validate_for(mode)?;
@@ -316,6 +321,49 @@ mod tests {
TiffWriter::new(order, false).write(&[ifd0]).unwrap()
}
/// A Z 6II NEF carrying NikonNEFInfo gets Nikon's distortion correction as one warp, in the header probe and
/// the full decode alike, and keeps it through DNG export; another body with the same data gets none.
#[test]
fn nikon_in_file_distortion_becomes_a_warp() {
use super::super::nef_lens::tests::{distortion_block, nef_info_blob, offset_block};
let (w, h) = (48u32, 32u32);
let file = |model: &str| {
let words: Vec<u8> = (0..w * h).flat_map(|i| (600 + (i % 97) as u16 * 50).to_be_bytes()).collect();
let mut raw = IfdBuilder::new();
raw.set(t::NEW_SUBFILE_TYPE, Value::Long(vec![0]));
raw.set(t::IMAGE_WIDTH, Value::Long(vec![w]));
raw.set(t::IMAGE_LENGTH, Value::Long(vec![h]));
raw.set(t::BITS_PER_SAMPLE, Value::Short(vec![14]));
raw.set(t::COMPRESSION, Value::Short(vec![1]));
raw.set(t::PHOTOMETRIC, Value::Short(vec![photometric::CFA]));
raw.set(t::CFA_REPEAT_PATTERN_DIM, Value::Short(vec![2, 2]));
raw.set(t::CFA_PATTERN_EP, Value::Byte(vec![0, 1, 1, 2]));
let info = nef_info_blob(&[
(0x0005, distortion_block(3, 21.6, 4, [0.039895, 0.019211, -0.127218], -0.025043)),
(0x0008, offset_block(1.5, -0.75)),
]);
raw.set(0xc7d5, Value::Undefined(info));
raw.set_image(ImageData::Strips { rows_per_strip: h, strips: vec![words] });
let mut ifd0 = IfdBuilder::new();
ifd0.set(t::MAKE, Value::Ascii("NIKON CORPORATION".into()));
ifd0.set(t::MODEL, Value::Ascii(model.into()));
ifd0.add_sub_ifd(raw);
TiffWriter::new(ByteOrder::Little, false).write(&[ifd0]).unwrap()
};
let bytes = file("NIKON Z 6_2");
let header = decode(&bytes, Mode::Header).unwrap();
let full = decode(&bytes, Mode::Full).unwrap();
assert_eq!(header.info(), full.info());
assert_eq!(header.opcodes.list3, full.opcodes.list3);
let [crate::Opcode::WarpRectilinear { planes, center }] = full.opcodes.list3.as_slice() else { panic!("{:?}", full.opcodes.list3) };
assert!((planes[0][0] - 1.0 / (1.0 - 0.025043)).abs() < 1e-5, "{planes:?}");
assert!((center[0] * 47.0 - (23.5 + 1.5)).abs() < 0.01 && (center[1] * 31.0 - (15.5 - 0.75)).abs() < 0.01, "{center:?}");
let dng = crate::write_dng(&full, &Default::default()).unwrap();
assert_eq!(crate::decode(&dng).unwrap().opcodes.list3, full.opcodes.list3);
full.develop(crate::Method::Bilinear).unwrap();
assert!(decode(&file("NIKON D850"), Mode::Full).unwrap().opcodes.list3.is_empty());
}
#[test]
fn preview_color_space_is_read_from_the_nikon_note() {
for (value, expected) in [(1, Some(crate::PreviewColorSpace::Srgb)), (2, Some(crate::PreviewColorSpace::AdobeRgb)), (3, None)] {
+259
View File
@@ -0,0 +1,259 @@
//! Nikon's in-file distortion correction (raw SubIFD tag `0xc7d5`, NikonNEFInfo), expressed as a DNG warp for
//! the shared optics pipeline.
//!
//! Tag identity: the ExifTool Nikon tag-name documentation (NikonNEFInfo `0xc7d5`; its DistortionInfo `0x0005`
//! with DistortionCorrection and RadialDistortionCoefficient1..3). The model below (which stored value multiplies
//! which power, the normalisation, the fill-frame zoom and the optical-centre offset in entry `0x0008`) was inferred
//! independently by matching the raw against the camera's own corrected JPEG; see docs/nikon-lens-corrections.md.
//! No lens database or external decoder code.
use crate::{Opcode, Rect};
use lightcraft_tiff::{ByteOrder, Ifd, ParseOptions, parse_ifd_at};
const NEF_INFO: u16 = 0xc7d5;
const DISTORTION_INFO: u16 = 0x0005;
const CENTRE_OFFSET: u16 = 0x0008;
/// Bodies whose files were checked against the camera's JPEG (docs/nikon-lens-corrections.md). Matching tags on
/// another body do not establish its pixel pitch or image area; add a body only after checking its files.
const VALIDATED: &[&str] = &["NIKON Z 6_2", "NIKON Z 8", "NIKON Z f"];
/// The normalisation radius every checked file stores: the FX half-diagonal in millimetres.
const FX_HALF_DIAGONAL_MM: f64 = 21.6;
const SAMPLES: usize = 256;
/// `r_src = r_out * (1 + k3 rho^2 + k2 rho^4 + k1 rho^6) / (1 + zoom)` around the optical centre, `rho` the radius
/// over the half-diagonal of the image area (`k1..k3` as ExifTool names them, in the order they are stored).
#[derive(Clone, Copy, Debug, PartialEq)]
struct Distortion {
k: [f64; 3],
zoom: f64,
}
/// A signed rational at `at` (`i32 / i32`), when it is one.
fn rational(b: &[u8], at: usize, order: ByteOrder) -> Option<f64> {
let n = order.u32(b.get(at..at + 4)?.try_into().ok()?) as i32;
let d = order.u32(b.get(at + 4..at + 8)?.try_into().ok()?) as i32;
(d > 0).then(|| f64::from(n) / f64::from(d)).filter(|v| v.is_finite())
}
/// The NikonNEFInfo entries: the blob is `"Nikon\0"`, a 4-byte version, then a little TIFF stream.
fn nef_info(raw: &Ifd) -> Option<(Ifd, ByteOrder)> {
let tiff = raw.bytes(NEF_INFO)?.strip_prefix(b"Nikon\0")?.get(4..)?;
let (order, false) = lightcraft_tiff::Tiff::sniff(tiff)? else { return None };
let first = order.u32(tiff.get(4..8)?.try_into().ok()?);
let opts = ParseOptions { follow_children: false, ..Default::default() };
let (ifd, _) = parse_ifd_at(tiff, u64::from(first), order, 0, false, &opts).ok()?;
Some((ifd, order))
}
/// DistortionInfo: `"0100"`, the correction mode (0 none), the normalisation radius (mm) at 8, the term count (4)
/// at 16, the coefficients at 20 / 28 / 36 and the zoom at 64, each a signed rational.
fn distortion_info(b: &[u8], order: ByteOrder) -> Option<Distortion> {
if b.get(0..4)? != b"0100" || *b.get(4)? == 0 {
return None;
}
let norm = rational(b, 8, order)?;
let terms = order.u32(b.get(16..20)?.try_into().ok()?);
if (norm - FX_HALF_DIAGONAL_MM).abs() > 0.05 || terms != 4 {
return None;
}
let k = [rational(b, 20, order)?, rational(b, 28, order)?, rational(b, 36, order)?];
let zoom = rational(b, 64, order)?;
// real lenses stay far inside these (the checked files: |k| < 0.13, |zoom| < 0.09)
(k.iter().all(|v| v.abs() < 1.0) && zoom.abs() < 0.5).then_some(Distortion { k, zoom })
}
/// Entry `0x0008`: `"0100"`, then the optical centre's offset from the image-area centre in pixels (x right,
/// y down), two signed rationals. Zero when the lens stabilises (the sensor stays centred).
fn centre_offset(b: &[u8], order: ByteOrder) -> Option<(f64, f64)> {
(b.get(0..4)? == b"0100").then_some(())?;
Some((rational(b, 4, order)?, rational(b, 12, order)?))
}
/// The correction of a `model` NEF whose raw IFD is `raw`, for the image area `crop` inside `active` (both in
/// sensor pixels), as an `OpcodeList3` warp in active-area coordinates. `None` for unvalidated bodies, other
/// image areas, missing or implausible data, and maps that fold.
pub(super) fn distortion(model: &str, raw: &Ifd, active: Rect, crop: Rect) -> Option<Opcode> {
if !VALIDATED.contains(&model.trim()) {
return None;
}
let crop = crop.clipped(active.width, active.height);
if active.width < 2 || active.height < 2 || crop.width < 2 || crop.height < 2 {
return None;
}
// the stored radius is the FX half-diagonal: only the full 3:2 image area (not a DX or other crop of the
// sensor) is normalised by it
if (crop.width as f64 / crop.height as f64 - 1.5).abs() > 0.005 || (crop.width as f64) < active.width as f64 * 0.95 {
return None;
}
let (info, order) = nef_info(raw)?;
let d = distortion_info(info.bytes(DISTORTION_INFO)?, order)?;
let (ox, oy) = info.bytes(CENTRE_OFFSET).and_then(|b| centre_offset(b, order)).unwrap_or((0.0, 0.0));
let (cw, ch) = (crop.width as f64 - 1.0, crop.height as f64 - 1.0);
if ox.abs() > cw * 0.05 || oy.abs() > ch * 0.05 {
return None;
}
let half_diagonal = cw.hypot(ch) / 2.0;
let poly = |rho: f64| {
let x = rho * rho;
1.0 + x * (d.k[2] + x * (d.k[1] + x * d.k[0]))
};
// DNG normalises by the farthest active-area corner from the centre; the image area reaches at most there
let (ax, ay) = (active.width as f64 - 1.0, active.height as f64 - 1.0);
let (cx, cy) = (crop.x as f64 + cw / 2.0 + ox, crop.y as f64 + ch / 2.0 + oy);
let farthest = cx.max(ax - cx).hypot(cy.max(ay - cy));
let a = farthest / half_diagonal;
// the source radius must grow with the output radius out to the farthest corner, or the map folds
let mut previous = -1.0;
for j in 0..=SAMPLES {
let rho = a * j as f64 / SAMPLES as f64;
let source = rho * poly(rho) / (1.0 + d.zoom);
if !source.is_finite() || source <= previous {
return None;
}
previous = source;
}
let s = 1.0 / (1.0 + d.zoom);
let a2 = a * a;
let planes = vec![[s, d.k[2] * a2 * s, d.k[1] * a2 * a2 * s, d.k[0] * a2 * a2 * a2 * s, 0.0, 0.0]];
Some(Opcode::WarpRectilinear { planes, center: [cx / ax, cy / ay] })
}
#[cfg(test)]
pub(in crate::vendor) mod tests {
use super::*;
use lightcraft_tiff::{Entry, IfdBuilder, TiffWriter, Value};
const AREA: Rect = Rect { x: 0, y: 0, width: 6064, height: 4040 };
const CROP: Rect = Rect { x: 8, y: 8, width: 6048, height: 4024 };
fn rat(v: f64, den: i32) -> [u8; 8] {
let mut b = [0u8; 8];
b[..4].copy_from_slice(&((v * f64::from(den)).round() as i32).to_le_bytes());
b[4..].copy_from_slice(&den.to_le_bytes());
b
}
/// DistortionInfo as the checked files lay it out (84 bytes, little-endian).
pub(in crate::vendor) fn distortion_block(mode: u8, norm: f64, terms: u32, k: [f64; 3], zoom: f64) -> Vec<u8> {
let mut b = vec![0u8; 84];
b[..4].copy_from_slice(b"0100");
b[4] = mode;
b[5] = 1;
b[8..16].copy_from_slice(&rat(norm, 4096));
b[16..20].copy_from_slice(&terms.to_le_bytes());
for (i, v) in k.iter().enumerate() {
b[20 + 8 * i..28 + 8 * i].copy_from_slice(&rat(*v, 1 << 20));
}
b[44..52].copy_from_slice(&rat(0.0, 1));
b[52..60].copy_from_slice(&rat(0.0, 1));
b[64..72].copy_from_slice(&rat(zoom, 1 << 20));
b
}
pub(in crate::vendor) fn offset_block(x: f64, y: f64) -> Vec<u8> {
let mut b = b"0100".to_vec();
b.extend_from_slice(&rat(x, 256));
b.extend_from_slice(&rat(y, 256));
b.extend_from_slice(&[0, 0, 0, 0]);
b
}
/// A NikonNEFInfo (`0xc7d5`) blob with the given entries.
pub(in crate::vendor) fn nef_info_blob(entries: &[(u16, Vec<u8>)]) -> Vec<u8> {
let mut info = IfdBuilder::new();
for (tag, data) in entries {
info.set(*tag, Value::Undefined(data.clone()));
}
let tiff = TiffWriter::new(ByteOrder::Little, false).write(&[info]).unwrap();
let mut blob = b"Nikon\0\x01\x03\0\0".to_vec();
blob.extend_from_slice(&tiff);
blob
}
/// A raw IFD carrying a NikonNEFInfo blob with the given entries.
pub(super) fn raw_with(entries: &[(u16, Vec<u8>)]) -> Ifd {
Ifd { entries: vec![Entry { tag: NEF_INFO, value: Value::Undefined(nef_info_blob(entries)), offset: 0 }], ..Default::default() }
}
// a Z 24-70 f/4 S at 24 mm and at 70 mm, as stored in two of the checked files
const WIDE: ([f64; 3], f64) = ([0.039895, 0.019211, -0.127218], -0.025043);
const TELE: ([f64; 3], f64) = ([0.01681, -0.010601, 0.056758], 0.088301);
fn warp(model: &str, raw: &Ifd) -> Option<([f64; 6], [f64; 2])> {
match distortion(model, raw, AREA, CROP)? {
Opcode::WarpRectilinear { planes, center } => Some((planes[0], center)),
_ => None,
}
}
/// The source radius the warp gives for an output radius `rho` (in image-area half-diagonals) about the centre.
fn source_rho(p: [f64; 6], centre: [f64; 2], rho: f64) -> f64 {
let (ax, ay) = (AREA.width as f64 - 1.0, AREA.height as f64 - 1.0);
let (cx, cy) = (centre[0] * ax, centre[1] * ay);
let m = cx.max(ax - cx).hypot(cy.max(ay - cy));
let hd = (CROP.width as f64 - 1.0).hypot(CROP.height as f64 - 1.0) / 2.0;
let t2 = (rho * hd / m).powi(2);
rho * (p[0] + t2 * (p[1] + t2 * (p[2] + t2 * p[3])))
}
#[test]
fn the_warp_is_the_stored_polynomial_over_the_zoom() {
for (k, zoom) in [WIDE, TELE] {
let raw = raw_with(&[(DISTORTION_INFO, distortion_block(3, 21.6, 4, k, zoom))]);
let (p, c) = warp("NIKON Z 6_2", &raw).unwrap();
for rho in [0.0, 0.25, 0.5, 0.75, 1.0] {
let x = rho * rho;
let want = rho * (1.0 + k[2] * x + k[1] * x * x + k[0] * x * x * x) / (1.0 + zoom);
assert!((source_rho(p, c, rho) - want).abs() < 2e-6, "{k:?} at {rho}");
}
// centred on the image area, which sits 8 px in from the sensor's top-left
assert!((c[0] - (8.0 + 6047.0 / 2.0) / 6063.0).abs() < 1e-12 && (c[1] - (8.0 + 4023.0 / 2.0) / 4039.0).abs() < 1e-12);
}
}
#[test]
fn the_optical_centre_follows_the_sensor_shift() {
let raw = raw_with(&[(DISTORTION_INFO, distortion_block(3, 21.6, 4, TELE.0, TELE.1)), (CENTRE_OFFSET, offset_block(-90.1, 18.59))]);
let (_, c) = warp("NIKON Z 6_2", &raw).unwrap();
assert!((c[0] * 6063.0 - (8.0 + 6047.0 / 2.0 - 90.1)).abs() < 0.01, "{c:?}");
assert!((c[1] * 4039.0 - (8.0 + 4023.0 / 2.0 + 18.59)).abs() < 0.01, "{c:?}");
}
#[test]
fn refuses_unvalidated_bodies_and_unknown_layouts() {
let good = distortion_block(3, 21.6, 4, WIDE.0, WIDE.1);
assert!(warp("NIKON Z 6_2", &raw_with(&[(DISTORTION_INFO, good.clone())])).is_some());
assert!(warp("NIKON Z f", &raw_with(&[(DISTORTION_INFO, good.clone())])).is_some());
assert!(warp("NIKON Z 8", &raw_with(&[(DISTORTION_INFO, good.clone())])).is_some());
for model in ["", "NIKON Z 6", "NIKON Z 6_3", "NIKON Z 9", "NIKON D850", "NIKON Z 6_2 x"] {
assert!(warp(model, &raw_with(&[(DISTORTION_INFO, good.clone())])).is_none(), "{model}");
}
// correction off, a DX radius, another term count, a wild coefficient or zoom, no NEFInfo at all
for bad in [
distortion_block(0, 21.6, 4, WIDE.0, WIDE.1),
distortion_block(3, 14.2, 4, WIDE.0, WIDE.1),
distortion_block(3, 21.6, 3, WIDE.0, WIDE.1),
distortion_block(3, 21.6, 4, [0.0, 0.0, 1.5], 0.0),
distortion_block(3, 21.6, 4, WIDE.0, 0.9),
b"0200".iter().copied().chain([0u8; 80]).collect(),
vec![0u8; 12],
] {
assert!(warp("NIKON Z 6_2", &raw_with(&[(DISTORTION_INFO, bad)])).is_none());
}
assert!(warp("NIKON Z 6_2", &Ifd::default()).is_none());
// a map that folds (the source radius turns back) is refused
assert!(warp("NIKON Z 6_2", &raw_with(&[(DISTORTION_INFO, distortion_block(3, 21.6, 4, [-0.9, 0.0, 0.0], 0.0))])).is_none());
}
#[test]
fn only_the_full_3_2_image_area_is_corrected() {
let raw = raw_with(&[(DISTORTION_INFO, distortion_block(3, 21.6, 4, WIDE.0, WIDE.1))]);
// a DX or 1:1 crop of the sensor would need another normalisation
assert!(distortion("NIKON Z 6_2", &raw, AREA, Rect::new(1016, 676, 4032, 2688)).is_none(), "DX");
assert!(distortion("NIKON Z 6_2", &raw, AREA, Rect::new(1016, 8, 4024, 4024)).is_none());
assert!(distortion("NIKON Z 6_2", &raw, AREA, Rect::new(0, 0, 1, 1)).is_none());
// a sensor-shift offset beyond 5 % of the frame is not trusted
let far = raw_with(&[(DISTORTION_INFO, distortion_block(3, 21.6, 4, WIDE.0, WIDE.1)), (CENTRE_OFFSET, offset_block(900.0, 0.0))]);
assert!(warp("NIKON Z 6_2", &far).is_none());
}
}
+35 -3
View File
@@ -279,14 +279,17 @@ pub(crate) fn decode_rows(src: &[u8], w: usize, h: usize, rows: usize, bits: u32
} else {
diff(&mut stream, &huff).ok_or_else(|| RawError::Corrupt(format!("NEF: invalid Huffman code at row {y}")))?
};
// a run of large differences in a corrupt stream can drive a predictor past i32 (issue #1515)
let overflow = || RawError::Corrupt(format!("NEF: predictor out of range at row {y}"));
let p = if x < 2 {
let s = seeds.get_mut(x).ok_or(RawError::Limit("NEF column"))?;
*s += d;
*s = s.checked_add(d).ok_or_else(overflow)?;
hpred[x & 1] = *s;
*s
} else {
hpred[x & 1] += d;
hpred[x & 1]
let p = hpred[x & 1].checked_add(d).ok_or_else(overflow)?;
hpred[x & 1] = p;
p
};
let v = p.clamp(0, max);
clipped += (v != p && (table.split == 0 || y < table.split)) as usize;
@@ -728,6 +731,35 @@ mod tests {
assert!(decode_rows(&src[..w * h / 4 - 1], w, h, 1, 14, &t).is_err(), "a strip too short for the image is corrupt in any mode");
}
/// A long run of the largest difference drove a predictor past i32: a panic in builds with overflow checks,
/// wrapped values otherwise (issue #1515). Both the row and the column predictors now stop with an error.
#[test]
fn predictor_overflow_is_an_error_not_a_panic() {
let codes = LOSSLESS_14.codes();
let k = (0..codes.len()).rev().find(|&k| codes[k].1 > 0).unwrap();
let d = (1u32 << k) - 1;
let stream = |n: usize| {
let mut wr = Writer::default();
let (code, len) = codes[k];
for _ in 0..n {
wr.put(code as u32 >> (12 - len), len as u32);
wr.put(d, k as u32);
}
wr.finish()
};
// differences one predictor needs to pass i32::MAX
let steps = (i32::MAX as u32 / d) as usize + 2;
let table = DecodeTable { encoding: Encoding::Lossless, seeds: [[0; 2]; 2], split: 0 };
// along a row: every other sample adds to the same predictor
let w = 2 * steps + 2;
let err = decode(&stream(w), w, 1, 14, &table).unwrap_err();
assert!(matches!(err, RawError::Corrupt(ref m) if m.contains("predictor")), "{err}");
// down two columns: each row adds to the seed of its parity
let h = 2 * steps + 2;
let err = decode(&stream(2 * h), 2, h, 14, &table).unwrap_err();
assert!(matches!(err, RawError::Corrupt(ref m) if m.contains("predictor")), "{err}");
}
#[test]
fn truncated_and_corrupted_streams_error_never_panic() {
let (w, h) = (64, 16);
+55 -6
View File
@@ -103,8 +103,10 @@ impl<'a> Bits<'a> {
pub fn consumed_bits(&self) -> usize {
self.pos * 8 - self.n as usize
}
/// Whether more bits were handed out than the source holds (the zeros read past its end are lookahead padding,
/// not data; using them would fabricate samples).
pub fn overrun(&self) -> bool {
self.pos > self.src.len() + 8
self.consumed_bits() > self.src.len() * 8
}
}
@@ -150,13 +152,17 @@ pub(crate) fn decode_huffman(src: &[u8], h: &Huffman, w: usize, hgt: usize, bits
let row = &mut out[y * w..(y + 1) * w];
for (x, o) in row.iter_mut().enumerate() {
let d = diff(&mut bits, h).ok_or_else(|| RawError::Corrupt(format!("PEF: invalid Huffman code at row {y}")))?;
// a run of large differences in a corrupt stream can drive a predictor past i32 (issue #1515)
let overflow = || RawError::Corrupt(format!("PEF: predictor out of range at row {y}"));
let p = if x < 2 {
vpred[y & 1][x] += d;
hpred[x] = vpred[y & 1][x];
hpred[x]
let v = vpred[y & 1][x].checked_add(d).ok_or_else(overflow)?;
vpred[y & 1][x] = v;
hpred[x] = v;
v
} else {
hpred[x & 1] += d;
hpred[x & 1]
let v = hpred[x & 1].checked_add(d).ok_or_else(overflow)?;
hpred[x & 1] = v;
v
};
*o = p.clamp(0, max) as u16;
}
@@ -390,6 +396,49 @@ mod tests {
assert!(table(&tb[..20], ByteOrder::Big).is_err());
}
/// Zero-padded lookahead past the end must not be decoded as samples: with no entropy at all, the zero bits
/// are a valid code (category 1) in this table, so a reader that only limits the refill position accepts them.
#[test]
fn symbols_decoded_from_padding_past_the_end_are_rejected() {
let (tb, codes) = test_table();
let h = table(&tb, ByteOrder::Big).unwrap();
assert!(decode_huffman(&[], &h, 2, 1, 14).is_err());
// the same image with its real entropy still decodes, including the last byte's padding bits
let px = [5u16, 9];
let src = encode(&px, 2, &codes);
assert_eq!(decode_huffman(&src, &h, 2, 1, 14).unwrap(), px);
// one byte short of the real entropy is rejected
assert!(decode_huffman(&src[..src.len() - 1], &h, 2, 1, 14).is_err());
}
/// A long run of the largest difference drove a predictor past i32: a panic in builds with overflow checks,
/// wrapped values otherwise (issue #1515). Both the row and the column predictors now stop with an error.
#[test]
fn predictor_overflow_is_an_error_not_a_panic() {
let (tb, codes) = test_table();
let h = table(&tb, ByteOrder::Big).unwrap();
let (c, l) = codes[14];
let d = (1u32 << 14) - 1;
let stream = |n: usize| {
let mut bits: Vec<bool> = Vec::with_capacity(n * (l as usize + 14));
for _ in 0..n {
(0..l).rev().for_each(|i| bits.push((c >> (12 - l)) >> i & 1 == 1));
(0..14).rev().for_each(|i| bits.push(d >> i & 1 == 1));
}
bits.chunks(8).map(|c| c.iter().enumerate().fold(0u8, |a, (i, &b)| a | (b as u8) << (7 - i))).collect::<Vec<u8>>()
};
// differences one predictor needs to pass i32::MAX
let steps = (i32::MAX as u32 / d) as usize + 2;
// along a row: every other sample adds to the same predictor
let w = 2 * steps + 2;
let err = decode_huffman(&stream(w), &h, w, 1, 14).unwrap_err();
assert!(matches!(err, RawError::Corrupt(ref m) if m.contains("predictor")), "{err}");
// down two columns: each row adds to the predictor of its parity
let rows = 2 * steps + 2;
let err = decode_huffman(&stream(2 * rows), &h, 2, rows, 14).unwrap_err();
assert!(matches!(err, RawError::Corrupt(ref m) if m.contains("predictor")), "{err}");
}
/// A big-endian one-strip PEF-style TIFF of `w × h` 12-bit samples with the given compression tag and strip
/// byte count, around `payload`.
fn tiny_pef(w: u16, h: u16, compression: u16, strip_len: u32, payload: &[u8]) -> Vec<u8> {
+35
View File
@@ -1018,6 +1018,41 @@ fn sony_embedded_distortion_is_limited_to_validated_models() {
eprintln!("Sony distortion model boundary: {checked} corpus files checked");
}
/// Nikon's in-file distortion correction (docs/nikon-lens-corrections.md) reaches only the validated bodies: the
/// Z f sample (a Tamron 35-150 at 150 mm) gets one warp, the DSLR and Coolpix samples none, and the header probe
/// describes every file as its full decode does.
#[test]
fn nikon_in_file_distortion_is_limited_to_validated_bodies() {
let root = corpus_root().join("raw");
let mut checked = 0;
for (name, model, expected) in [
("nef-nikon-zf-lossless.nef", "NIKON Z f", 1),
("nef-nikon-d5100-lossless.nef", "NIKON D5100", 0),
("nef-nikon-d5100-uncompressed.nef", "NIKON D5100", 0),
("nef-nikon-d7000-lossy12.nef", "NIKON D7000", 0),
("nef-nikon-d7500-lossless12.nef", "NIKON D7500", 0),
("nef-nikon-d7500-lossless14.nef", "NIKON D7500", 0),
("nrw-nikon-b700-uncompressed.nrw", "COOLPIX B700", 0),
] {
let path = root.join(name);
if !path.exists() {
eprintln!("skip: {} absent", path.display());
continue;
}
let bytes = std::fs::read(&path).unwrap();
let full = decode(&bytes).unwrap_or_else(|e| panic!("{name}: {e}"));
assert_eq!(probe_info(&bytes).unwrap(), full.info(), "{name}");
assert_eq!(full.metadata.model.as_deref(), Some(model), "{name}");
assert_eq!(full.opcodes.list3.len(), expected, "{name}");
if let Some(lightcraft_raw::Opcode::WarpRectilinear { planes, .. }) = full.opcodes.list3.first() {
// the stored zoom (0.0557) frames the corrected picture as the camera does
assert!((planes[0][0] - 1.0 / 1.055719).abs() < 1e-4, "{name}: {planes:?}");
}
checked += 1;
}
eprintln!("Nikon distortion boundary: {checked} corpus files checked");
}
/// Kodak DCS520C (`crates/raw/src/vendor/kodak.rs`): the two-component lossless JPEG strip decodes through the
/// file's response curve to a 1736 x 1160 GRBG mosaic with 12-bit white, per-position black levels and the
/// white balance of the vendor IFD. Looks for the corpus copy, then the raw.pixls.us folder; skips without either.
+28
View File
@@ -124,3 +124,31 @@ fn canon_m50_craw_crop_is_the_shot_aspect() {
let raw = decode(&bytes).unwrap();
assert_eq!((raw.crop.x, raw.crop.y, raw.crop.width, raw.crop.height), (1012, 12, 4000, 4000));
}
#[test]
fn canon_rp_lossless_sensor_is_exact() {
// raw.pixls.us/getfile.php/4674; SHA256
// dcbd0f6f9929de2e2331b0fd10a9d01e7cfd04c5a5e4038456de1647d714cdf2
check("cr3-canon-rp-raw.cr3", 0x5cb0_f464_04b8_b663, (6384, 4224), (6240, 4160));
}
#[test]
fn canon_rp_craw_sensor_is_exact() {
// raw.pixls.us/getfile.php/4672; SHA256
// bd3270610593d37ff8a399908bb9406a4e32f9151d55bf19ac1cf8f10d787b46
check("cr3-canon-rp-craw.cr3", 0xf48d_f06c_a1d6_7382, (6384, 4224), (6240, 4160));
}
#[test]
fn canon_rp_crop_mode_lossless_sensor_is_exact() {
// 1.6x crop mode: a single tile. raw.pixls.us/getfile.php/4673; SHA256
// 7d4ba45dc9264d598336133593c677b09774bcdf13b602ee770e72415a652a16
check("cr3-canon-rp-raw-crop.cr3", 0x4680_8a9d_e3b3_dcc7, (4032, 2656), (3888, 2592));
}
#[test]
fn canon_rp_crop_mode_craw_sensor_is_exact() {
// raw.pixls.us/getfile.php/4671; SHA256
// 4a859ab137f792aedaf3416b0aad1497ebaab02052a49453902e226699154aa7
check("cr3-canon-rp-craw-crop.cr3", 0x39a6_0c4e_fed4_e1f4, (4032, 2656), (3888, 2592));
}
+82
View File
@@ -90,6 +90,45 @@ fn assert_same(a: &RawImage, b: &RawImage) {
assert_eq!(b.metadata.rating, Some(3));
}
#[test]
fn writer_camera_identity_tags_roundtrip() {
let cases = [
(Some("Canon"), Some("Canon EOS R6"), "Canon EOS R6"),
(Some("CANON"), Some("cAnOn EOS R6"), "cAnOn EOS R6"),
(Some("Canon"), Some("Canon"), "Canon"),
(Some("Canon"), Some("Canon-EOS R6"), "Canon-EOS R6"),
(Some("Canon"), Some("Canon_EOS R6"), "Canon Canon_EOS R6"),
(Some("Canon"), Some("Canonic R6"), "Canon Canonic R6"),
(Some("Canon"), Some("Canon2 R6"), "Canon Canon2 R6"),
(Some("Canon"), Some("Canoné R6"), "Canon Canoné R6"),
(Some("Canon"), Some("EOS R6"), "Canon EOS R6"),
(Some("Eastman Kodak"), Some("eAsTmAn kOdAk DCS Pro"), "eAsTmAn kOdAk DCS Pro"),
(Some("Eastman Kodak"), Some("Eastman Kodakish DCS Pro"), "Eastman Kodak Eastman Kodakish DCS Pro"),
(Some("Eastman Kodak"), Some("Eastman DCS Pro"), "Eastman Kodak Eastman DCS Pro"),
(Some(""), Some("EOS R6"), "EOS R6"),
(Some("Canon"), Some(""), "Canon"),
(Some(""), Some(""), "LightCraft"),
(None, Some("EOS R6"), "EOS R6"),
(Some("Canon"), None, "Canon"),
(None, None, "LightCraft"),
];
for (make, model, unique) in cases {
let mut raw = synthetic(16, 16, Some(Cfa::bayer("RGGB").unwrap()), 1);
raw.metadata.make = make.map(str::to_owned);
raw.metadata.model = model.map(str::to_owned);
for order in [ByteOrder::Little, ByteOrder::Big] {
let bytes = write_dng(&raw, &DngWriteOptions { compression: DngCompression::Uncompressed, order, xmp: None }).unwrap();
let tiff = lightcraft_tiff::Tiff::parse(&bytes).unwrap();
let ifd = &tiff.ifds[0];
assert_eq!(ifd.value(t::UNIQUE_CAMERA_MODEL), Some(&Value::Ascii(unique.into())), "{make:?} / {model:?}, {order:?}");
for (tag, original) in [(t::MAKE, make), (t::MODEL, model)] {
let expected = original.filter(|s| !s.is_empty()).map(|s| Value::Ascii(s.into()));
assert_eq!(ifd.value(tag), expected.as_ref(), "tag {tag}, {make:?} / {model:?}, {order:?}");
}
}
}
}
#[test]
fn writer_roundtrip_all_layouts() {
for (w, h) in [(64, 48), (70, 45), (33, 17)] {
@@ -1047,3 +1086,46 @@ fn cfa_pattern_is_relative_to_the_active_area() {
assert_eq!(back.normalized().unwrap().cfa.unwrap().name(), "GRBG");
}
}
/// Two samples a pixel are neither one colour nor RGB. A DNG with them decoded, and developing it read a third sample
/// past each pixel, a panic in release builds too (issue #1511). Such data is refused with a reason, and the demosaic
/// itself no longer reads past a pixel; one, three and four samples develop as before.
#[test]
fn two_samples_per_pixel_are_refused_not_a_panic() {
let (w, h) = (8, 6);
let file = |cpp: usize| {
let mut raw = base_ifd(w, h, 16, cpp, false);
raw.set_image(ImageData::Strips { rows_per_strip: h as u32, strips: vec![u16_bytes(&pattern(w, h, cpp, 16), ByteOrder::Little)] });
dng(raw, ByteOrder::Little)
};
let err = decode(&file(2)).unwrap_err();
assert!(matches!(err, lightcraft_raw::RawError::Unsupported(ref m) if m.contains("2 samples")), "{err}");
for cpp in [1, 3, 4] {
decode(&file(cpp)).unwrap().develop(Method::Bilinear).unwrap();
}
// a raw image built with two samples is refused when it develops
let raw = synthetic(16, 12, None, 2);
let err = raw.develop(Method::Ahd).unwrap_err();
assert!(matches!(err, lightcraft_raw::RawError::Unsupported(_)), "{err}");
// the demosaic of such data repeats the last sample instead of reading past the pixel
let n = lightcraft_raw::Normalized { width: 2, height: 1, cpp: 2, data: vec![0.1, 0.2, 0.3, 0.4], cfa: None };
assert_eq!(lightcraft_raw::demosaic(&n, Method::Ahd).data, vec![[0.1, 0.2, 0.2], [0.3, 0.4, 0.4]]);
}
/// DNGs LightCraft wrote before the writer anchored the pattern at the ActiveArea carry a bare "LightCraft" Software
/// tag and a sensor-origin pattern; they keep that reading, while the current writer marks its files differently.
#[test]
fn legacy_lightcraft_dng_keeps_the_sensor_anchored_pattern() {
let (w, h) = (20, 12);
let build = |software: &str| {
let mut raw = base_ifd(w, h, 16, 1, true);
raw.set_image(ImageData::Strips { rows_per_strip: h as u32, strips: vec![u16_bytes(&pattern(w, h, 1, 16), ByteOrder::Little)] });
raw.set(t::WHITE_LEVEL, Value::Short(vec![65535]));
raw.set(t::ACTIVE_AREA, Value::Long(vec![1, 0, h as u32 - 1, w as u32 - 1]));
raw.set(t::SOFTWARE, Value::Ascii(software.into()));
decode(&dng(raw, ByteOrder::Little)).unwrap()
};
assert_eq!(build("LightCraft").cfa.unwrap().name(), "GRBG");
assert_eq!(build("Adobe Photoshop Lightroom").cfa.unwrap().name(), "BGGR");
assert_ne!(lightcraft_raw::dngwrite::SOFTWARE, "LightCraft");
}
+65
View File
@@ -102,3 +102,68 @@ fn hostile_dimensions_are_rejected_quickly() {
assert!(lightcraft_raw::decode(&bytes).is_err());
assert!(t0.elapsed().as_secs_f64() < 2.0);
}
/// A Bayer DNG whose `BlackLevel` is a FLOAT holding NaN (or infinity), and a float CFA DNG with
/// NaN samples: every normalised sample came out NaN, and AHD's green estimate clamped between two
/// NaN neighbours, which panics (the 1:1 view and every export of such a file). Non-finite levels
/// read as 0 and non-finite samples as black: the photo develops to finite numbers.
#[test]
fn non_finite_black_levels_and_samples_develop() {
use lightcraft_tiff::tags as t;
use lightcraft_tiff::{IfdBuilder, ImageData, TiffWriter, Value};
let (w, h) = (32u32, 16u32);
for black in [Value::Float(vec![f32::NAN]), Value::Double(vec![f64::INFINITY]), Value::Float(vec![64.0, f32::NAN, 64.0, 64.0])] {
let mut ifd = IfdBuilder::new();
ifd.set(t::DNG_VERSION, Value::Byte(vec![1, 4, 0, 0]));
ifd.set(t::NEW_SUBFILE_TYPE, Value::Long(vec![0]));
ifd.set(t::IMAGE_WIDTH, Value::Long(vec![w]));
ifd.set(t::IMAGE_LENGTH, Value::Long(vec![h]));
ifd.set(t::BITS_PER_SAMPLE, Value::Short(vec![16]));
ifd.set(t::COMPRESSION, Value::Short(vec![1]));
ifd.set(t::PHOTOMETRIC, Value::Short(vec![t::photometric::CFA]));
ifd.set(t::CFA_REPEAT_PATTERN_DIM, Value::Short(vec![2, 2]));
ifd.set(t::CFA_PATTERN_EP, Value::Byte(vec![0, 1, 1, 2]));
ifd.set(t::BLACK_LEVEL_REPEAT_DIM, Value::Short(vec![2, 2]));
ifd.set(t::BLACK_LEVEL, black.clone());
ifd.set(t::WHITE_LEVEL, Value::Long(vec![4095]));
let strip: Vec<u8> = (0..w * h).flat_map(|i| (100 + (i * 37 % 3000) as u16).to_le_bytes()).collect();
ifd.set_image(ImageData::Strips { rows_per_strip: h, strips: vec![strip] });
let bytes = TiffWriter::default().write(&[ifd]).unwrap();
let img = lightcraft_raw::decode(&bytes).unwrap_or_else(|e| panic!("{black:?}: {e}"));
let rgb = img.develop(Method::Ahd).unwrap();
assert!(rgb.data.iter().flatten().all(|v| v.is_finite()), "{black:?}: finite output");
let _ = img.develop(Method::Ppg);
}
// float samples: a block of NaN and infinities in the middle of the mosaic
let (w, h) = (24usize, 16usize);
let data: Vec<f32> = (0..w * h)
.map(|i| match (i % w, i / w) {
(8..=13, 5..=10) => f32::NAN,
(2, _) => f32::INFINITY,
_ => (i * 97 % 1000) as f32 / 1000.0,
})
.collect();
let raw = RawImage {
format: RawFormat::Dng,
width: w,
height: h,
cpp: 1,
data: RawData::F32(data),
cfa: Some(Cfa::bayer("RGGB").unwrap()),
bits: 32,
black: BlackLevel::uniform(0.0),
white: vec![1.0],
active_area: Rect::new(0, 0, w, h),
crop: Rect::new(0, 0, w, h),
orientation: Orientation::Normal,
color: ColorData::default(),
wb_multipliers: None,
linearized: false,
opcodes: OpcodeLists::default(),
metadata: Default::default(),
};
for img in [raw.clone(), lightcraft_raw::decode(&lightcraft_raw::write_dng(&raw, &DngWriteOptions::default()).unwrap()).unwrap()] {
let rgb = img.develop(Method::Ahd).unwrap();
assert!(rgb.data.iter().flatten().all(|v| v.is_finite()), "float NaN samples: finite output");
}
}
+16 -1
View File
@@ -68,7 +68,7 @@ impl Scene {
paint::render(self.kind, self.seed, w, h)
}
/// Render fitting within `max_edge` on the long side.
/// Render fitting within `max_edge` on the long side (at most the scene's own size).
pub fn render_fit(&self, max_edge: usize) -> Rgb32f {
let (w, h) = self.fit(max_edge);
self.render(w, h)
@@ -81,6 +81,9 @@ impl Scene {
}
fn fit(&self, max_edge: usize) -> (usize, usize) {
// never past the scene's own size: that is a demo photo's full resolution, and a size
// from a damaged catalog (a width of 4294967295) asked for more memory than exists
let max_edge = max_edge.min(self.width.max(self.height).max(1) as usize);
let a = self.aspect();
let (w, h) = if a >= 1.0 { (max_edge, (max_edge as f32 / a).round() as usize) } else { ((max_edge as f32 * a).round() as usize, max_edge) };
(w.max(1), h.max(1))
@@ -205,6 +208,18 @@ mod tests {
}
}
/// Found by fuzzing catalog snapshots: a demo photo whose width read 4294967295 asked its scene
/// for a render that size, and the pixel buffer's size overflowed (a panic on the render
/// thread). A scene renders at most at its own size, which is the photo's full resolution.
#[test]
fn a_scene_renders_at_most_at_its_own_size() {
let mut s = demo_library().remove(0);
(s.width, s.height) = (40, 30);
let img = s.render_fit(usize::MAX);
assert_eq!((img.width, img.height), (40, 30));
assert_eq!(s.fit(20), (20, 15), "smaller sizes as asked");
}
#[test]
fn sunsets_have_highlight_headroom() {
let lib = demo_library();
+24 -4
View File
@@ -76,6 +76,11 @@ impl Weights {
self.entries.contains_key(name)
}
/// The shape stored in the header for `name`, without reading its data.
pub fn shape(&self, name: &str) -> Option<&[usize]> {
self.entries.get(name).map(|e| e.shape.as_slice())
}
/// Tensor `name` as stored, on the CPU.
pub fn read(&self, name: &str) -> Result<Tensor> {
let e = self.entries.get(name).ok_or_else(|| Error::Model(format!("missing tensor `{name}`")))?;
@@ -97,11 +102,13 @@ pub struct Backend(pub Arc<Weights>);
impl SimpleBackend for Backend {
fn get(&self, s: Shape, name: &str, _h: candle_nn::Init, dtype: DType, dev: &Device) -> candle_core::Result<Tensor> {
let t = self.get_unchecked(name, dtype, dev)?;
if t.shape() != &s {
candle_core::bail!("shape mismatch for {name}: expected {s:?}, got {:?}", t.shape())
// compare against the header first, so a wrong-shaped tensor is never read or copied to the device
if let Some(stored) = self.0.shape(name)
&& stored != s.dims()
{
candle_core::bail!("shape mismatch for {name}: expected {s:?}, got {stored:?}")
}
Ok(t)
self.get_unchecked(name, dtype, dev)
}
fn get_unchecked(&self, name: &str, dtype: DType, dev: &Device) -> candle_core::Result<Tensor> {
@@ -127,6 +134,19 @@ mod tests {
p
}
#[test]
fn wrong_expected_shape_fails_before_the_data_is_read() {
let p = write("shape", r#"{"a":{"dtype":"F32","shape":[2,2],"data_offsets":[0,16]}}"#, 16);
let backend = Backend(Weights::open(&p).unwrap());
let ok = backend.get(Shape::from((2, 2)), "a", candle_nn::Init::Const(0.0), DType::F32, &Device::Cpu).unwrap();
assert_eq!(ok.dims(), &[2, 2]);
// cut the data off after opening: a read would now fail with an I/O error instead of a shape error
std::fs::OpenOptions::new().write(true).open(&p).unwrap().set_len(8).unwrap();
let err = backend.get(Shape::from((3, 3)), "a", candle_nn::Init::Const(0.0), DType::F32, &Device::Cpu).unwrap_err().to_string();
assert!(err.contains("shape mismatch"), "{err}");
let _ = std::fs::remove_file(p);
}
#[test]
fn tensors_must_lie_inside_the_file() {
let ok = write("ok", r#"{"a":{"dtype":"F32","shape":[2,2],"data_offsets":[0,16]}}"#, 16);
+4 -3
View File
@@ -140,9 +140,10 @@ impl<'a> Ctx<'a> {
}
entries.sort_by_key(|e| e.tag);
entries.dedup_by_key(|e| e.tag);
let next_pos = table + (n as usize).saturating_mul(entry_size);
let next = if n as usize == n_read {
match self.read_off(next_pos) {
// the next-IFD offset follows the whole table, so only a table read in full has one: a count far past the
// file (a BigTIFF's is 64-bit) must not reach the arithmetic (issue #1516)
let next = if n == n_read as u64 {
match n_read.checked_mul(entry_size).and_then(|len| table.checked_add(len)).and_then(|p| self.read_off(p)) {
Some(0) | None => 0,
Some(v) => base.checked_add(v).unwrap_or(0),
}
+34
View File
@@ -393,3 +393,37 @@ proptest! {
prop_assert_eq!(t.ifds[0].f64s(501).unwrap().len(), v.len());
}
}
/// A BigTIFF whose IFD says it has an enormous number of entries, with one readable, computed where the next-IFD
/// offset would be from the advertised count: an add overflow, a panic in builds with overflow checks (issue
/// #1516). The readable entry is kept and there is no next IFD; complete chains still link.
#[test]
fn bigtiff_with_an_enormous_entry_count_does_not_panic() {
for order in ORDERS {
for count in [u64::MAX, 1 << 62, 2] {
// little-endian bytes, reversed for big-endian files
let e = |le: &[u8]| -> Vec<u8> { if order == ByteOrder::Little { le.to_vec() } else { le.iter().rev().copied().collect() } };
let mut f = if order == ByteOrder::Little { b"II".to_vec() } else { b"MM".to_vec() };
for part in [e(&43u16.to_le_bytes()), e(&8u16.to_le_bytes()), e(&0u16.to_le_bytes()), e(&16u64.to_le_bytes()), e(&count.to_le_bytes())] {
f.extend(part);
}
// one entry: ImageWidth, LONG, 1 value (inline in the 8-byte field)
for part in [e(&tags::IMAGE_WIDTH.to_le_bytes()), e(&4u16.to_le_bytes()), e(&1u64.to_le_bytes()), e(&640u32.to_le_bytes())] {
f.extend(part);
}
f.extend([0; 4]);
let (ifd, next) = parse_ifd_at(&f, 16, order, 0, true, &ParseOptions::default()).unwrap();
assert_eq!(ifd.value(tags::IMAGE_WIDTH), Some(&Value::Long(vec![640])), "{order:?} {count}");
assert_eq!(next, 0, "{order:?} {count}");
let t = Tiff::parse(&f).unwrap();
assert_eq!(t.ifds.len(), 1);
}
// a complete chain of two IFDs still links
let mut a = IfdBuilder::new();
a.set(1000, Value::Short(vec![1]));
let mut b = IfdBuilder::new();
b.set(1000, Value::Short(vec![2]));
let t = Tiff::parse(&TiffWriter::new(order, true).write(&[a, b]).unwrap()).unwrap();
assert_eq!(t.ifds.len(), 2, "{order:?}");
}
}
+229
View File
@@ -0,0 +1,229 @@
//! Slider-drag latency through the whole app: the real session, renderer, job pool and GPU (when
//! there is one), driven by a headless host at 60 frames a second, as a window with vsync would be.
//!
//! `cargo run --release -p lightcraft-ui-egui --example drag_latency -- LIBRARY_DIR PHOTO [CONTROL…]`
//! Env: `SIZE=1920x1200` (points), `SCALE=1`, `TICKS=30`, `GPU=0` for the CPU pipeline.
//!
//! Per control: a drag of `TICKS` value changes, one per frame; for each tick the frames (and ms)
//! until its draft became the loupe texture, how many ticks were never shown, the UI thread's
//! frame time, and after release the time until the full-quality render is the texture. (A window
//! presents the texture one frame later: add ~16.7 ms for photons.)
#![allow(clippy::unwrap_used, clippy::expect_used)]
use std::collections::HashMap;
use std::time::{Duration, Instant};
use lightcraft_engine::Session;
use lightcraft_ui_egui::headless::Headless;
use lightcraft_ui_egui::render::Slot;
use lightcraft_ui_egui::{LightcraftApp, Services};
use serde_json::json;
const FRAME: Duration = Duration::from_micros(16_667);
const CONTROLS: &[(&str, f64, f64)] = &[
("light.exposure", 0.0, 1.0),
("light.contrast", 0.0, 50.0),
("light.highlights", 0.0, -60.0),
("light.shadows", 0.0, 60.0),
("light.whites", 0.0, 40.0),
("light.blacks", 0.0, -40.0),
("wb.temp", 5000.0, 6500.0),
("wb.tint", 0.0, 20.0),
("color.vibrance", 0.0, 40.0),
("color.saturation", 0.0, 30.0),
("curve.shadows", 0.0, 30.0),
("mixer.orange.hue", 0.0, 30.0),
("mixer.blue.sat", 0.0, -40.0),
("grading.shadows.sat", 0.0, 30.0),
("calibration.redHue", 0.0, 30.0),
("effects.texture", 0.0, 40.0),
("effects.clarity", 0.0, 40.0),
("effects.dehaze", 0.0, 30.0),
("vignette.amount", 0.0, -40.0),
("grain.amount", 0.0, 30.0),
("detail.sharpenAmount", 40.0, 90.0),
("detail.nrLuminance", 0.0, 40.0),
("detail.nrColor", 25.0, 60.0),
("optics.distortion", 0.0, 30.0),
("optics.vignetting", 0.0, 50.0),
("optics.defringePurple", 0.0, 5.0),
("geometry.vertical", 0.0, 20.0),
];
/// Step one frame at the 60 Hz cadence that started at `t0`.
fn frame(h: &mut Headless, t0: Instant, n: &mut u32) -> f64 {
*n += 1;
let due = t0 + FRAME * *n;
if let Some(d) = due.checked_duration_since(Instant::now()) {
std::thread::sleep(d);
}
let t = Instant::now();
h.step();
t.elapsed().as_secs_f64() * 1e3
}
/// The view slot measured: the zoom window with `ZOOM=<percent>` (set after opening), else the loupe.
fn watch() -> Slot {
if ZOOMED.load(std::sync::atomic::Ordering::Relaxed) { Slot::Region } else { Slot::Main }
}
static ZOOMED: std::sync::atomic::AtomicBool = std::sync::atomic::AtomicBool::new(false);
fn settle(h: &mut Headless, id: lightcraft_catalog::PhotoId) {
let t = Instant::now();
while t.elapsed() < Duration::from_secs(120) {
h.step();
let r = &h.app.renderer;
let done = r.textures.get(&watch()).is_some_and(|t| t.photo == id && Some(t.key) == r.wanted(watch())) && !r.is_pending(watch());
if done && !h.busy() {
return;
}
std::thread::sleep(Duration::from_millis(2));
}
eprintln!("settle: timed out");
}
fn pct(v: &mut [f64], p: f64) -> f64 {
if v.is_empty() {
return f64::NAN;
}
v.sort_by(|a, b| a.total_cmp(b));
v[((v.len() - 1) as f64 * p).round() as usize]
}
fn main() {
let args: Vec<String> = std::env::args().skip(1).collect();
let (lib, file) = (args.first().expect("LIBRARY_DIR"), args.get(1).expect("PHOTO"));
let only: Vec<&str> = args[2..].iter().map(String::as_str).collect();
let env = |k: &str, d: &str| std::env::var(k).unwrap_or_else(|_| d.to_string());
let (w, hgt) = env("SIZE", "1920x1200").split_once('x').map(|(a, b)| (a.parse::<f32>().unwrap(), b.parse::<f32>().unwrap())).unwrap();
let scale: f32 = env("SCALE", "1").parse().unwrap();
let ticks: u32 = env("TICKS", "30").parse().unwrap();
let gpu = env("GPU", "1") != "0";
let mut s = Session::new().with_fs().with_system_clock();
s.open_library(lib, false).unwrap();
let r = s.execute("library.import", &json!({"paths": [file]})).unwrap();
let id = r["imported"].as_array().and_then(|a| a.first()).and_then(|v| v.as_u64()).map(lightcraft_catalog::PhotoId).unwrap_or_else(|| {
// already in the library
let name = std::path::Path::new(file).file_name().unwrap().to_string_lossy().to_string();
s.catalog.photos().find(|p| p.file_name == name).map(|p| p.id).expect("photo in library")
});
s.execute("library.select", &json!({"ids": [id.0]})).unwrap();
let mut app = LightcraftApp::new(s, Services::default());
app.ui.settings.gpu = gpu;
let mut h = Headless::new(app, [w, hgt], scale);
let t = Duration::from_secs(60);
h.request("ui.set", json!({"view": "detail", "right": "edit", "filmstrip": true}), t);
let t_open = Instant::now();
settle(&mut h, id);
if let Ok(z) = std::env::var("ZOOM") {
let pct: f64 = z.parse().unwrap();
h.app.run("view.navigate", json!({"zoom": {"percent": pct}, "pan": [0.5, 0.5]})).unwrap();
for _ in 0..5 {
h.step();
}
ZOOMED.store(true, std::sync::atomic::Ordering::Relaxed);
let t = Instant::now();
settle(&mut h, id);
eprintln!("zoomed to {pct} % in {:.0} ms", t.elapsed().as_secs_f64() * 1e3);
}
let tex = h.app.renderer.textures.get(&watch()).map(|t| t.size);
eprintln!("opened in {:.0} ms, loupe texture {tex:?}, gpu {gpu}", t_open.elapsed().as_secs_f64() * 1e3);
println!("control,shown/ticks,lat_frames_med,lat_ms_med,lat_ms_p90,lat_ms_max,frame_ms_med,frame_ms_max,final_ms,draft_px,final_px");
// the controls named (in that order, repeats allowed), else all of them
let list: Vec<(&str, f64, f64)> =
if only.is_empty() { CONTROLS.to_vec() } else { only.iter().filter_map(|n| CONTROLS.iter().find(|c| c.0 == *n).copied()).collect() };
for (control, a, b) in list {
let mut asked: HashMap<u64, (u32, Instant)> = HashMap::new();
let mut lat_frames = Vec::new();
let mut lat_ms = Vec::new();
let mut frame_ms = Vec::new();
let mut shown_key = h.app.renderer.textures.get(&watch()).map(|t| t.key);
let mut draft_px = None;
let t0 = Instant::now();
let mut n = 0u32;
h.app.run("develop.beginInteraction", json!({"label": control})).unwrap();
#[allow(clippy::too_many_arguments)]
fn observe(
h: &Headless,
n: u32,
asked: &HashMap<u64, (u32, Instant)>,
shown_key: &mut Option<u64>,
lat_frames: &mut Vec<f64>,
lat_ms: &mut Vec<f64>,
draft_px: &mut Option<[usize; 2]>,
) {
let r = &h.app.renderer;
if let Some(tex) = r.textures.get(&watch())
&& Some(tex.key) != *shown_key
{
*shown_key = Some(tex.key);
if let Some((k, at)) = asked.get(&tex.key) {
lat_frames.push((n - k) as f64);
lat_ms.push(at.elapsed().as_secs_f64() * 1e3);
*draft_px = Some(tex.size);
}
}
}
for i in 1..=ticks {
let v = a + (b - a) * i as f64 / ticks as f64;
// the event arrives at the start of the frame (as input does)
let due = t0 + FRAME * (n + 1);
if let Some(d) = due.checked_duration_since(Instant::now()) {
std::thread::sleep(d);
}
let start = Instant::now();
h.app.run("develop.set", json!({"control": control, "value": v})).unwrap();
n += 1;
h.step();
frame_ms.push(start.elapsed().as_secs_f64() * 1e3);
if let Some(k) = h.app.renderer.wanted(watch()) {
asked.entry(k).or_insert((n, start));
}
observe(&h, n, &asked, &mut shown_key, &mut lat_frames, &mut lat_ms, &mut draft_px);
}
// release
let due = t0 + FRAME * (n + 1);
if let Some(d) = due.checked_duration_since(Instant::now()) {
std::thread::sleep(d);
}
let release = Instant::now();
h.app.run("develop.endInteraction", json!({})).unwrap();
let mut final_ms = f64::NAN;
let mut final_px = None;
while release.elapsed() < Duration::from_secs(60) {
let fm = frame(&mut h, t0, &mut n);
frame_ms.push(fm);
observe(&h, n, &asked, &mut shown_key, &mut lat_frames, &mut lat_ms, &mut draft_px);
let r = &h.app.renderer;
if h.app.session.interaction.is_none()
&& let Some(tex) = r.textures.get(&watch())
&& Some(tex.key) == r.wanted(watch())
&& !r.is_pending(watch())
{
final_ms = release.elapsed().as_secs_f64() * 1e3;
final_px = Some(tex.size);
break;
}
}
let shown = lat_ms.len();
let fmax = frame_ms.iter().cloned().fold(0.0, f64::max);
let lmax = lat_ms.iter().cloned().fold(0.0, f64::max);
println!(
"{control},{shown}/{ticks},{:.0},{:.1},{:.1},{lmax:.1},{:.1},{fmax:.1},{final_ms:.1},{:?},{:?}",
pct(&mut lat_frames, 0.5),
pct(&mut lat_ms, 0.5),
pct(&mut lat_ms, 0.9),
pct(&mut frame_ms, 0.5),
draft_px.unwrap_or([0, 0]),
final_px.unwrap_or([0, 0]),
);
// back to where it was (one undo step), settled
h.app.run("develop.set", json!({"control": control, "value": a})).unwrap();
settle(&mut h, id);
std::thread::sleep(Duration::from_millis(200));
}
}
+74
View File
@@ -0,0 +1,74 @@
//! Opening raw photos in the loupe, timed through the real app logic, headless (no compositor, so no frame
//! throttling) but with the GPU on: import FILES into an in-memory library, open the first in Detail, then step to
//! the next one once its neighbour prefetch is done and once more at once.
//! `cargo run --release -p lightcraft-ui-egui --example open_timeline -- a.ARW b.ARW c.ARW`
//! (`LIGHTCRAFT_PROFILE=1` adds the engine's stage timings; every line carries ms since start on stderr).
use lightcraft_engine::Session;
use lightcraft_ui_egui::{LightcraftApp, Services, headless::Headless};
use serde_json::{Value, json};
use std::time::{Duration, Instant};
fn main() {
let files: Vec<String> = std::env::args().skip(1).collect();
let t0 = Instant::now();
let ms = move || t0.elapsed().as_secs_f64() * 1e3;
let mut s = Session::new().with_fs();
let r = s.execute("library.import", &json!({"paths": files})).expect("import");
eprintln!("{:9.1} imported {}", ms(), r["imported"]);
let mut h = Headless::new(LightcraftApp::new(s, Services::default()), [1600.0, 1000.0], 1.0);
let req = |h: &mut Headless, m: &str, p: Value| h.request(m, p, Duration::from_secs(60));
let cmd = |h: &mut Headless, c: &str| req(h, "engine.execute", json!({"command": c}))["result"].clone();
cmd(&mut h, "view.photoGrid");
h.settle(Duration::from_secs(60));
cmd(&mut h, "library.selectNone");
let r = cmd(&mut h, "library.next");
eprintln!("{:9.1} first: {r}", ms());
let first = r["active"].as_u64().unwrap_or(1);
h.settle(Duration::from_secs(60));
let gpu = req(&mut h, "ui.inspect", json!({}))["result"]["perf"]["gpu"].clone();
eprintln!("{:9.1} grid settled, gpu {gpu}", ms());
// what the loupe shows, every frame, until `photo` is rendered for good
let watch = |h: &mut Headless, label: &str, photo: u64| {
let start = ms();
let mut last = String::new();
loop {
let i = req(h, "ui.inspect", json!({}))["result"].clone();
let l = &i["loupe"];
let state = format!("photo={} source={} pending={}", l["photo"], l["source"], l["pending"]);
if state != last {
eprintln!(
"{:9.1} [{label}] +{:7.1} ms {state} (queue {}, in flight {})",
ms(),
ms() - start,
i["perf"]["renderQueue"],
i["perf"]["rendersInFlight"]
);
last = state;
}
let done = l["photo"].as_u64() == Some(photo) && matches!(l["source"].as_str(), Some("render" | "cached")) && l["pending"] == false;
if done || ms() - start > 30_000.0 {
return ms() - start;
}
}
};
eprintln!("{:9.1} open {first}", ms());
cmd(&mut h, "view.detail");
let open = watch(&mut h, "open", first);
let t = ms();
h.settle(Duration::from_secs(60));
eprintln!("{:9.1} neighbours prefetched after {:.0} ms", ms(), ms() - t);
// step towards the side that has photos (the sort may put the first one last)
let mut dir = "library.next";
let mut r = cmd(&mut h, dir);
if r["active"].as_u64() == Some(first) {
dir = "library.previous";
r = cmd(&mut h, dir);
}
eprintln!("{:9.1} {dir}: {r}", ms());
let n1 = r["active"].as_u64().unwrap_or(0);
let step1 = watch(&mut h, "next (prefetched)", n1);
let n2 = cmd(&mut h, dir)["active"].as_u64().unwrap_or(0);
let step2 = watch(&mut h, "next at once", n2);
h.settle(Duration::from_secs(60));
eprintln!("{:9.1} summary: open {open:.0} ms, next prefetched {step1:.0} ms, next at once {step2:.0} ms", ms());
}
+42 -42
View File
@@ -1,22 +1,22 @@
{
" ({} in Recently Deleted)": " ({} em Excluídos recentemente)",
" · {skipped} skipped (file exists)": " · {skipped} ignorada(s) (arquivo existente)",
" · {} duplicate{} skipped": " · {} duplicada{:.0} ignorada",
" · {} duplicate{} skipped": " · {} duplicada(s) ignorada(s){:.0}",
" · {} failed: {}": " · {} falhou: {}",
" · {} moved": " · {} movida(s)",
" · {} not readable": " · {} não legível",
" · {} restored from Recently Deleted": " · {} restaurada(s) de Excluídos recentemente",
", {failed} file{} not readable": ", {failed} arquivo{:.0} não legível",
", {failed} file{} not readable": ", {failed} arquivo(s) não legível(is){:.0}",
"Add {local_n} to My Photos": "Adicionar {local_n} a Minhas fotos",
"Add {} Photo{}": "Adicionar {} foto{:.0}",
"Add {} Photo{}, restore {}": "Adicionar {} foto{:.0}, restaurar {}",
"Add {} Photo{}": "Adicionar {} foto{}",
"Add {} Photo{}, restore {}": "Adicionar {} foto{}, restaurar {}",
"Add “{}” to Favorites": "Adicionar “{}” aos favoritos",
"Added {n} photo{} to My Photos": "Adicionada {n} foto{:.0} a Minhas fotos",
"Added {n} photo{} to “{}”": "Adicionada {n} foto{:.0} a “{}”",
"Added {n} photo{} to My Photos": "{n} foto{0} adicionada{0} a Minhas fotos",
"Added {n} photo{} to “{}”": "{n} foto{0} adicionada{0} a “{1}”",
"Adding photos… {} of {}": "Adicionando fotos… {} de {}",
"All {n} selected photos move by the same amount.": "Todas as {n} fotos selecionadas se movem pela mesma quantidade.",
"Already in the library, in Recently Deleted: {} photo{} selected · right-click ▸ Restore, or Delete Permanently to import afresh": "Já na biblioteca, em Excluídos recentemente: {} foto{:.0} selecionada · clique com o botão direito → Restaurar, ou Excluir permanentemente para importar de novo",
"Already in the library: {} photo{} selected in All Photos": "Já na biblioteca: {} foto{:.0} selecionada em Todas as fotos",
"Already in the library, in Recently Deleted: {} photo{} selected · right-click ▸ Restore, or Delete Permanently to import afresh": "Já na biblioteca, em Excluídos recentemente: {0} foto{1} selecionada{1} · clique com o botão direito → Restaurar, ou Excluir permanentemente para importar de novo",
"Already in the library: {} photo{} selected in All Photos": "Já na biblioteca: {0} foto{1} selecionada{1} em Todas as fotos",
"Auto ({})": "Automático ({})",
"Auto Sync: edits apply to {n} photos": "Sincronização automática: edições aplicadas a {n} fotos",
"Automatic ({} MB)": "Automático ({} MB)",
@@ -33,18 +33,18 @@
"Discard Smart Previews ({scope})": "Descartar pré-visualizações inteligentes ({scope})",
"Duplicate \"{label}\"": "Duplicar \"{label}\"",
"Example: {example}": "Exemplo: {example}",
"Exported {n} photo{}": "Exportada {n} foto{:.0}",
"Exported {ok} of {total} photo{}": "Exportadas {ok} de {total} foto{:.0}",
"Exported {} point curve preset{}": "Exportadas {} predefinição{:.0} de curva de pontos",
"Exported {} preset{}": "Exportadas {} predefinição{:.0}",
"Exported {n} photo{}": "{n} foto{0} exportada{0}",
"Exported {ok} of {total} photo{}": "Fotos exportadas: {ok} de {total}{:.0}",
"Exported {} point curve preset{}": "{} predefinição(ões) de curva de pontos exportada(s){:.0}",
"Exported {} preset{}": "{} predefinição(ões) exportada(s){:.0}",
"Exporting {} of {total}": "Exportando {} de {total}",
"Filter bar — {active} active filter{}": "Barra de filtros — {active} filtro{:.0} ativo",
"Healed {n} dust spot{}": "Recuperadas {n} mancha{:.0} de poeira",
"Import cancelled · {} photo{} added": "Importação cancelada · {} foto{:.0} adicionada",
"Imported {n} point curve preset{}": "Importadas {n} predefinição{:.0} de curva de pontos",
"Imported {n} preset{}": "Importadas {n} predefinição{:.0}",
"Imported {n} preset{}, {f} file{} not readable": "Importadas {n} predefinição{:.0}, {f} arquivo{:.0} não legível",
"Imported {} photo{}": "Importadas {} foto{:.0}",
"Filter bar — {active} active filter{}": "Barra de filtros — {active} filtro{0} ativo{0}",
"Healed {n} dust spot{}": "{n} mancha{0} de poeira recuperada{0}",
"Import cancelled · {} photo{} added": "Importação cancelada · {0} foto{1} adicionada{1}",
"Imported {n} point curve preset{}": "{n} predefinição(ões) de curva de pontos importada(s){:.0}",
"Imported {n} preset{}": "{n} predefinição(ões) importada(s){:.0}",
"Imported {n} preset{}, {f} file{} not readable": "{n} predefinição(ões) importada(s){:.0}, {f} arquivo(s) não legível(is){:.0}",
"Imported {} photo{}": "{0} foto{1} importada{1}",
"Lightroom catalog inspected: {photos} photos, {collections} collections, {missing} missing": "Catálogo do Lightroom inspecionado: {photos} fotos, {collections} coleções, {missing} ausentes",
"Lightroom import complete: {imported} photos added": "Importação do Lightroom concluída: {imported} fotos adicionadas",
"Lightroom import complete: {imported} photos added, {failed} failed": "Importação do Lightroom concluída: {imported} fotos adicionadas, {failed} com falha",
@@ -58,7 +58,7 @@
"Merging… {stage} {:.0}%": "Mesclando… {stage} {:.0}%",
"Move {what} to Recently Deleted?": "Mover {what} para Excluídos recentemente?",
"Named ({named_n})": "Nomeadas ({named_n})",
"Paste into {n} photo{} — only settings that were copied are pasted": "Colar em {n} foto{:.0} — somente os ajustes copiados são colados",
"Paste into {n} photo{} — only settings that were copied are pasted": "Colar em {n} foto{} — somente os ajustes copiados são colados",
"Preset: {name}": "Predefinição: {name}",
"Preview only — editing the camera's embedded JPEG ({})": "Somente pré-visualização — editando o JPEG incorporado da câmera ({})",
"Preview only — {}": "Somente pré-visualização — {}",
@@ -81,10 +81,10 @@
"Reset {name}": "Redefinir {name}",
"Reset {} Channel": "Redefinir canal {}",
"Restore {}": "Restaurar {}",
"Restore {} Photo{}": "Restaurar {} foto{:.0}",
"Restored {} photo{} from Recently Deleted": "Restauradas {} foto{:.0} de Excluídos recentemente",
"Restore {} Photo{}": "Restaurar {} foto{}",
"Restored {} photo{} from Recently Deleted": "{0} foto{1} restaurada{1} de Excluídos recentemente",
"Saved export preset “{}”": "Predefinição de exportação “{}” salva",
"Show {hidden} hidden location{}": "Mostrar {hidden} local{:.0} oculto",
"Show {hidden} hidden location{}": "Mostrar {hidden} local(is) oculto(s){:.0}",
"Showing {n} of {}": "Mostrando {n} de {}",
"Smart album: {}": "Álbum inteligente: {}",
"Undo {}": "Desfazer {}",
@@ -95,7 +95,7 @@
"the {n} selected photos": "as {n} fotos selecionadas",
"the {} photos in view": "as {} fotos em exibição",
"these {count} photos": "estas {count} fotos",
"{count} smart preview{} · {:.1} MB": "{count} pré-visualizações{:.0} · {:.1} MB",
"{count} smart preview{} · {:.1} MB": "{count} pré-visualização(ões) inteligente(s){:.0} · {:.1} MB",
"{f} folders (and their subfolders)": "{f} pastas (e suas subpastas)",
"{label} Grading": "Graduação de {label}",
"{label} {d:+} on every selected photo": "{label} {d:+} em cada foto selecionada",
@@ -114,12 +114,12 @@
"{n} found · {sel} selected": "{n} encontradas · {sel} selecionadas",
"{n} photos": "{n} fotos",
"{n} photos in {} are hidden by the filters above": "{n} fotos em {} estão ocultas pelos filtros acima",
"{n} photo{}": "{n} foto{:.0}",
"{n} photo{} match · updates automatically as photos change": "{n} foto{:.0} correspondente(s) · atualiza automaticamente conforme as fotos mudam",
"{n} photo{} now · updates automatically as photos change": "{n} foto{:.0} agora · atualiza automaticamente conforme as fotos mudam",
"{n} photo{}": "{n} foto{}",
"{n} photo{} match · updates automatically as photos change": "{n} foto{0} correspondente{0} · atualiza automaticamente conforme as fotos mudam",
"{n} photo{} now · updates automatically as photos change": "{n} foto{} agora · atualiza automaticamente conforme as fotos mudam",
"{sel_n} selected · {counted}": "{sel_n} selecionadas · {counted}",
"{stage}… {:.0}%": "{stage}… {:.0}%",
"{verb} {n} Photo{}": "{verb} {n} foto{:.0}",
"{verb} {n} Photo{}": "{verb} {n} foto{}",
"{weekday}, {day}": "{weekday}, {day}",
"{weekday}, {day} {month} {year}": "{weekday}, {day}/{month}/{year}",
"{what} merge added": "{what} mesclagem adicionada",
@@ -127,13 +127,13 @@
"{year}": "{year}",
"{} and higher": "{} ou superior",
"{} files": "{} arquivos",
"{} files and {f} folder{}": "{} arquivos e {f} pasta{:.0}",
"{} files and {f} folder{}": "{} arquivos e {f} pasta{}",
"{} of 4 guides — drag along lines that should be vertical or horizontal.": "{} de 4 guias — arraste ao longo das linhas que devem ficar verticais ou horizontais.",
"{} photos": "{} fotos",
"{} photo{} not readable": "{} foto{:.0} não legível",
"{} photo{} not readable": "{} foto(s) não legível(is){:.0}",
"{} spot(s) on this photo": "{} ponto(s) nesta foto",
"· {dups} already in the library": "· {dups} já na biblioteca",
"· {n} photo{}": "· {n} foto{:.0}",
"· {n} photo{}": "· {n} foto{}",
"· {trashed} in Recently Deleted": "· {trashed} em Excluídos recentemente",
"… and {more} more": "… e mais {more}",
" ({unsure} with several look-alike files: use Locate)": " ({unsure} com vários arquivos parecidos: use Localizar)",
@@ -197,23 +197,23 @@
"{} label (click more labels to show any of them)": "Rótulo {} (clique em mais rótulos para mostrar qualquer um deles)",
", {} star{}": ", {} estrela{}",
", {} label": ", rótulo {}",
"Downloads from {host}. Once it has arrived and checked out, LightCraft installs it, {after}.": "Downloads from {host}. Once it has arrived and checked out, LightCraft installs it, {after}.",
"Downloads from {host}. Once it has arrived and checked out, LightCraft installs it, {after}.": "Faz o download de {host}. Depois que chegar e for verificado, o LightCraft o instala e {after}.",
"{:.1} MB": "{:.1} MB",
"Running on the processor: {}.": "Running on the processor: {}.",
"{} · {}. Kept in the library's “denoise” folder; the oldest ones that no photo uses go first when the limit is reached.": "{} · {}. Kept in the library's “denoise” folder; the oldest ones that no photo uses go first when the limit is reached.",
"Denoising {name}: {done} of {total} tiles{}": "Denoising {name}: {done} of {total} tiles{}",
"Running on the processor: {}.": "Executando no processador: {}.",
"{} · {}. Kept in the library's “denoise” folder; the oldest ones that no photo uses go first when the limit is reached.": "{} · {}. Mantidas na pasta “denoise” da biblioteca; as mais antigas que nenhuma foto usa são removidas primeiro quando o limite é atingido.",
"Denoising {name}: {done} of {total} tiles{}": "Reduzindo o ruído de {name}: {done} de {total} partes{}",
"{size} GB": "{size} GB",
"{} of {} from {host}": "{} of {} from {host}",
"{} of {} from {host}": "{} de {} de {host}",
"{} KB": "{} KB",
"{} MB": "{} MB",
"Making the denoised picture… {}%": "Making the denoised picture… {}%",
"Making the denoised picture… {}%": "Gerando a imagem com ruído reduzido… {}%",
"{name} · {terms}": "{name} · {terms}",
"{n} {noun}": "{n} {noun}",
"Trained on: {provenance}": "Trained on: {provenance}",
" · {queued} waiting": " · {queued} waiting",
"Shows the model's terms, then downloads {} from {host}, installs it and starts using it. Nothing else is sent.": "Shows the model's terms, then downloads {} from {host}, installs it and starts using it. Nothing else is sent.",
"In line for AI Denoise ({ahead} ahead)…": "In line for AI Denoise ({ahead} ahead)…",
"Running on {}.": "Running on {}.",
"Trained on: {provenance}": "Treinado com: {provenance}",
" · {queued} waiting": " · {queued} aguardando",
"Shows the model's terms, then downloads {} from {host}, installs it and starts using it. Nothing else is sent.": "Mostra os termos do modelo, depois baixa {} de {host}, instala e começa a usá-lo. Nada mais é enviado.",
"In line for AI Denoise ({ahead} ahead)…": "Na fila da redução de ruído com IA ({ahead} à frente)…",
"Running on {}.": "Executando em {}.",
"{file} · {size}": "{file} · {size}",
"• {assumption}": "• {assumption}",
"Put inside “{parent}”": "Colocar dentro de “{parent}”",
+93 -93
View File
@@ -578,8 +578,8 @@
"Green": "Verde",
"Green Amount": "Quantidade de verde",
"Green Hue": "Matiz do verde",
"Green Hue High": "Matiz do verde alta",
"Green Hue Low": "Matiz do verde baixa",
"Green Hue High": "Matiz do verde alto",
"Green Hue Low": "Matiz do verde baixo",
"Green Luminance": "Luminância do verde",
"Green Primary": "Primária verde",
"Green Saturation": "Saturação do verde",
@@ -667,13 +667,13 @@
"Importing scans the source for photos; it doesn't add the folder to Local (use Local → Browse Folder… to work in a folder without importing).": "A importação examina a origem em busca de fotos; ela não adiciona a pasta a Local (use Local → Procurar pasta… para trabalhar em uma pasta sem importar).",
"In Photo Grid and Square Grid: Flag as Pick and advance": "Na Grade de fotos e na Grade quadrada: marcar como seleção e avançar",
"In Photo Grid and Square Grid: expand or collapse the stack": "Na Grade de fotos e na Grade quadrada: expandir ou recolher a pilha",
"In Recently Deleted": "Em Recentemente excluídos",
"In Recently Deleted": "Em Excluídos recentemente",
"In library": "Na biblioteca",
"In memory — nothing is saved": "Na memória — nada é salvo",
"In use": "Em uso",
"In-memory session: these settings last until LightCraft quits.": "Sessão na memória: estes ajustes duram até o LightCraft ser encerrado.",
"Include subfolders": "Incluir subpastas",
"Increase Brush Feather": "Aumentar suavização do pincel",
"Increase Brush Feather": "Aumentar esfumado do pincel",
"Increase Brush Size": "Aumentar tamanho do pincel",
"Info": "Informações",
"Info (I)": "Informações (I)",
@@ -1066,8 +1066,8 @@
"Purple": "Roxo",
"Purple Amount": "Quantidade de roxo",
"Purple Hue": "Matiz do roxo",
"Purple Hue High": "Matiz do roxo alta",
"Purple Hue Low": "Matiz do roxo baixa",
"Purple Hue High": "Matiz do roxo alto",
"Purple Hue Low": "Matiz do roxo baixo",
"Purple Luminance": "Luminância do roxo",
"Purple Saturation": "Saturação do roxo",
"Put the locations you removed from Local back (no files change)": "Devolve os locais removidos de Local (nenhum arquivo muda)",
@@ -1663,96 +1663,96 @@
"≥ at least": "≥ no mínimo",
"日本語": "日本語",
"繁體中文(台灣)": "繁體中文(台灣)",
"Back to this person": "Back to this person",
"AI Denoise": "AI Denoise",
"AI Denoise Model": "AI Denoise Model",
"AI Denoise is off: press Use on a model.": "AI Denoise is off: press Use on a model.",
"AI Denoise needs a model. Install one in Settings.": "AI Denoise needs a model. Install one in Settings.",
"Accept & Download": "Accept & Download",
"Already installed. Remove it before installing a replacement.": "Already installed. Remove it before installing a replacement.",
"Automatic uses the graphics card where it is faster than the processor: both are timed on this computer the first time a photo is made.": "Automatic uses the graphics card where it is faster than the processor: both are timed on this computer the first time a photo is made.",
"Back to this person": "Voltar para esta pessoa",
"AI Denoise": "Redução de ruído com IA",
"AI Denoise Model": "Modelo de redução de ruído com IA",
"AI Denoise is off: press Use on a model.": "A redução de ruído com IA está desativada: clique em Usar em um modelo.",
"AI Denoise needs a model. Install one in Settings.": "A redução de ruído com IA precisa de um modelo. Instale um em Configurações.",
"Accept & Download": "Aceitar e baixar",
"Already installed. Remove it before installing a replacement.": "Já instalado. Remova-o antes de instalar um substituto.",
"Automatic uses the graphics card where it is faster than the processor: both are timed on this computer the first time a photo is made.": "Automático usa a placa de vídeo quando ela é mais rápida que o processador: ambos são cronometrados neste computador na primeira vez que uma foto é gerada.",
"Cache": "Cache",
"Checking and installing the model…": "Checking and installing the model…",
"Checking…": "Checking…",
"Clear cache": "Clear cache",
"Denoise Models": "Denoise Models",
"Denoise Settings": "Denoise Settings",
"Denoise all photos with an Amount": "Denoise all photos with an Amount",
"Downloaded and checked. Installing…": "Downloaded and checked. Installing…",
"Failed its last test": "Failed its last test",
"Graphics card": "Graphics card",
"I have read these terms and accept them for my own use": "I have read these terms and accept them for my own use",
"Inspect Denoise Model": "Inspect Denoise Model",
"Install Denoise Model": "Install Denoise Model",
"Install from file…": "Install from file…",
"Install your own ONNX model with denoise-model.json beside it. This build offers no model downloads.": "Install your own ONNX model with denoise-model.json beside it. This build offers no model downloads.",
"Its denoised picture is not made yet, and pictures are not made on their own (Settings, AI Denoise).": "Its denoised picture is not made yet, and pictures are not made on their own (Settings, AI Denoise).",
"LightCraft does not know this model, so it assumed:": "LightCraft does not know this model, so it assumed:",
"Limit": "Limit",
"Make it now": "Make it now",
"Make the denoised picture of the photos I am looking at": "Make the denoised picture of the photos I am looking at",
"Next up for AI Denoise…": "Next up for AI Denoise…",
"Nothing is fetched until you accept. The model is kept on this computer only; LightCraft never uploads or shares it.": "Nothing is fetched until you accept. The model is kept on this computer only; LightCraft never uploads or shares it.",
"Only photos whose AI Denoise amount is above 0 are made. Exports always make theirs if it is missing.": "Only photos whose AI Denoise amount is above 0 are made. Exports always make theirs if it is missing.",
"Open page": "Open page",
"Open the model's page": "Open the model's page",
"Opens the model's own page in your browser, to read about it or get the file yourself.": "Opens the model's own page in your browser, to read about it or get the file yourself.",
"Processor": "Processor",
"Set up AI Denoise…": "Set up AI Denoise…",
"Speed": "Speed",
"Starting…": "Starting…",
"Tested and works on this computer": "Tested and works on this computer",
"The denoise model is installed and in use: the AI Denoise Amount slider under Detail now works": "The denoise model is installed and in use: the AI Denoise Amount slider under Detail now works",
"The graphics card is set up when the next photo is made.": "The graphics card is set up when the next photo is made.",
"The model is kept on this computer only. LightCraft never uploads or shares it.": "The model is kept on this computer only. LightCraft never uploads or shares it.",
"The processor does the work.": "The processor does the work.",
"There is no model file to install": "There is no model file to install",
"This build cannot run denoise models.": "This build cannot run denoise models.",
"This build has nowhere to keep denoise models, so AI Denoise is not available. (The desktop app has.)": "This build has nowhere to keep denoise models, so AI Denoise is not available. (The desktop app has.)",
"This file cannot be used yet": "This file cannot be used yet",
"Tick the box to accept the model's terms first": "Tick the box to accept the model's terms first",
"Try again": "Try again",
"Try the graphics card again": "Try the graphics card again",
"Turn off": "Turn off",
"Unknown licence": "Unknown licence",
"Use": "Use",
"Waiting to start…": "Waiting to start…",
"Works on raw files from cameras with a standard (Bayer) sensor. Other files keep their normal noise reduction.": "Works on raw files from cameras with a standard (Bayer) sensor. Other files keep their normal noise reduction.",
"cached picture": "cached picture",
"cached pictures": "cached pictures",
"commercial use allowed": "commercial use allowed",
"Checking and installing the model…": "Verificando e instalando o modelo…",
"Checking…": "Verificando…",
"Clear cache": "Limpar cache",
"Denoise Models": "Modelos de redução de ruído",
"Denoise Settings": "Configurações de redução de ruído",
"Denoise all photos with an Amount": "Reduzir o ruído de todas as fotos com uma Quantidade",
"Downloaded and checked. Installing…": "Baixado e verificado. Instalando…",
"Failed its last test": "Falhou no último teste",
"Graphics card": "Placa de vídeo",
"I have read these terms and accept them for my own use": "Li estes termos e os aceito para meu uso pessoal",
"Inspect Denoise Model": "Inspecionar modelo de redução de ruído",
"Install Denoise Model": "Instalar modelo de redução de ruído",
"Install from file…": "Instalar de arquivo…",
"Install your own ONNX model with denoise-model.json beside it. This build offers no model downloads.": "Instale seu próprio modelo ONNX com o denoise-model.json ao lado dele. Esta versão não oferece downloads de modelos.",
"Its denoised picture is not made yet, and pictures are not made on their own (Settings, AI Denoise).": "A imagem com ruído reduzido ainda não foi gerada, e as imagens não são geradas automaticamente (Configurações, Redução de ruído com IA).",
"LightCraft does not know this model, so it assumed:": "O LightCraft não conhece este modelo, então presumiu:",
"Limit": "Limite",
"Make it now": "Gerar agora",
"Make the denoised picture of the photos I am looking at": "Gerar a imagem com ruído reduzido das fotos que estou vendo",
"Next up for AI Denoise…": "Próxima na fila da redução de ruído com IA…",
"Nothing is fetched until you accept. The model is kept on this computer only; LightCraft never uploads or shares it.": "Nada é baixado até você aceitar. O modelo fica somente neste computador; o LightCraft nunca o envia nem o compartilha.",
"Only photos whose AI Denoise amount is above 0 are made. Exports always make theirs if it is missing.": "Somente fotos com quantidade de redução de ruído com IA acima de 0 são geradas. Ao exportar, a imagem é sempre gerada se estiver faltando.",
"Open page": "Abrir página",
"Open the model's page": "Abrir a página do modelo",
"Opens the model's own page in your browser, to read about it or get the file yourself.": "Abre a página do próprio modelo no seu navegador, para ler sobre ele ou baixar o arquivo você mesmo.",
"Processor": "Processador",
"Set up AI Denoise…": "Configurar redução de ruído com IA…",
"Speed": "Velocidade",
"Starting…": "Iniciando…",
"Tested and works on this computer": "Testado e funcionando neste computador",
"The denoise model is installed and in use: the AI Denoise Amount slider under Detail now works": "O modelo de redução de ruído está instalado e em uso: o controle Quantidade da redução de ruído com IA em Detalhe agora funciona",
"The graphics card is set up when the next photo is made.": "A placa de vídeo é configurada quando a próxima foto for gerada.",
"The model is kept on this computer only. LightCraft never uploads or shares it.": "O modelo fica somente neste computador. O LightCraft nunca o envia nem o compartilha.",
"The processor does the work.": "O processador faz o trabalho.",
"There is no model file to install": "Não há arquivo de modelo para instalar",
"This build cannot run denoise models.": "Esta versão não consegue executar modelos de redução de ruído.",
"This build has nowhere to keep denoise models, so AI Denoise is not available. (The desktop app has.)": "Esta versão não tem onde guardar modelos de redução de ruído, então a redução de ruído com IA não está disponível. (O aplicativo para desktop tem.)",
"This file cannot be used yet": "Este arquivo ainda não pode ser usado",
"Tick the box to accept the model's terms first": "Marque a caixa para aceitar primeiro os termos do modelo",
"Try again": "Tentar novamente",
"Try the graphics card again": "Tentar a placa de vídeo novamente",
"Turn off": "Desativar",
"Unknown licence": "Licença desconhecida",
"Use": "Usar",
"Waiting to start…": "Aguardando para iniciar…",
"Works on raw files from cameras with a standard (Bayer) sensor. Other files keep their normal noise reduction.": "Funciona com arquivos RAW de câmeras com sensor padrão (Bayer). Outros arquivos mantêm a redução de ruído normal.",
"cached picture": "imagem em cache",
"cached pictures": "imagens em cache",
"commercial use allowed": "uso comercial permitido",
"denoiseModel.accept": "denoiseModel.accept",
"in use": "in use",
"installed": "installed",
"makes AI Denoise available": "makes AI Denoise available",
"non-commercial use only": "non-commercial use only",
"photo has its denoised picture.": "photo has its denoised picture.",
"photo waiting": "photo waiting",
"photos have their denoised picture.": "photos have their denoised picture.",
"photos waiting": "photos waiting",
"starts using it for AI Denoise": "starts using it for AI Denoise",
"terms unclear": "terms unclear",
"the graphics card": "the graphics card",
"the graphics card is not used": "the graphics card is not used",
"Cancel Denoise": "Cancel Denoise",
"Cancel Denoise Model Download": "Cancel Denoise Model Download",
"Choose Denoise Model": "Choose Denoise Model",
"Clear Denoise Cache": "Clear Denoise Cache",
"Denoise Model Downloads": "Denoise Model Downloads",
"Denoise Photos": "Denoise Photos",
"in use": "em uso",
"installed": "instalado",
"makes AI Denoise available": "disponibiliza a redução de ruído com IA",
"non-commercial use only": "somente uso não comercial",
"photo has its denoised picture.": "foto já está com o ruído reduzido.",
"photo waiting": "foto aguardando",
"photos have their denoised picture.": "fotos já estão com o ruído reduzido.",
"photos waiting": "fotos aguardando",
"starts using it for AI Denoise": "começa a usá-lo para a redução de ruído com IA",
"terms unclear": "termos pouco claros",
"the graphics card": "a placa de vídeo",
"the graphics card is not used": "a placa de vídeo não é usada",
"Cancel Denoise": "Cancelar redução de ruído",
"Cancel Denoise Model Download": "Cancelar download do modelo de redução de ruído",
"Choose Denoise Model": "Escolher modelo de redução de ruído",
"Clear Denoise Cache": "Limpar cache de redução de ruído",
"Denoise Model Downloads": "Downloads de modelos de redução de ruído",
"Denoise Photos": "Reduzir ruído das fotos",
"Denoise Pump": "Denoise Pump",
"Denoise Status": "Denoise Status",
"Download Denoise Model": "Download Denoise Model",
"Remove Denoise Model": "Remove Denoise Model",
"Test Denoise Model": "Test Denoise Model",
"AI Denoise is enabled on the requesting photo": "AI Denoise is enabled on the requesting photo",
"After installation, AI Denoise will turn on for the requesting photo.": "After installation, AI Denoise will turn on for the requesting photo.",
"Cancel pending action": "Cancel pending action",
"Install a denoise model to enable AI Denoise on this photo": "Install a denoise model to enable AI Denoise on this photo",
"Pending photo action cancelled": "Pending photo action cancelled",
"Pending photo action cancelled because its library or photo changed": "Pending photo action cancelled because its library or photo changed",
"ONNX model": "ONNX model",
"commercial use subject to GPL conditions": "commercial use subject to GPL conditions",
"Denoise Status": "Status da redução de ruído",
"Download Denoise Model": "Baixar modelo de redução de ruído",
"Remove Denoise Model": "Remover modelo de redução de ruído",
"Test Denoise Model": "Testar modelo de redução de ruído",
"AI Denoise is enabled on the requesting photo": "A redução de ruído com IA está ativada na foto que a solicitou",
"After installation, AI Denoise will turn on for the requesting photo.": "Após a instalação, a redução de ruído com IA será ativada na foto que a solicitou.",
"Cancel pending action": "Cancelar ação pendente",
"Install a denoise model to enable AI Denoise on this photo": "Instale um modelo de redução de ruído para ativar a redução de ruído com IA nesta foto",
"Pending photo action cancelled": "Ação pendente da foto cancelada",
"Pending photo action cancelled because its library or photo changed": "Ação pendente da foto cancelada porque a biblioteca ou a foto foi alterada",
"ONNX model": "Modelo ONNX",
"commercial use subject to GPL conditions": "uso comercial sujeito às condições da GPL",
"Decrease Rating": "Diminuir avaliação",
"Increase Rating": "Aumentar avaliação",
"Click to type a value": "Clique para digitar um valor",

Some files were not shown because too many files have changed in this diff Show More