3 Commits

Author SHA1 Message Date
Dylan M. Taylor 4ae3089420 security: bounded keyword, repair and journal-recovery work (CodeRabbit) (#2763)
* security: harden dev-tool scripts and the testkit skip-log test (CodeRabbit)

- tools/install-sam3.sh: the HF bearer token went through curl's argv, where
  `ps` exposes it to other local accounts (CWE-214); it now travels through
  a private 0600 temp config file removed on exit.
- packaging/macos/import-cert.sh: the decoded .p12 was written to a
  predictable path in a shared TMPDIR, where a precreated attacker-readable
  file would receive it (CWE-377); both cert and keychain are created inside
  a mktemp -d 0700 directory now.
- docs/showcase/run.py: the reply reader copied whole responses with no size
  cap (obvious quadratic-copy and unbounded-memory DoS). It accumulates in a
  bytearray now and closes the connection past 128 MiB.
- tools/spectral_camera_table.py: alias/make/model strings and comments from
  the rawtoaces-data checkout moved directly into Rust source; one newline in
  a comment became Rust code. External text is now escaped as Rust literal
  content and comments are normalized to one printable line.
- crates/testkit: appended_lines_accumulate reused a pip-predictable temp
  dir by creating it with create_dir_all and carried on after failing to
  remove its log path, so the append would follow an attacker-planted
  symlink (CWE-59); it now fails the test if the directory exists instead of
  testing through it, and removes any existing log path only after clearly
  checking it, always failing the test instead of proceeding when removal fails.

Findings #10080/#10078 PR (rows 2, 6, 8, 9, 15 of the
storytold-lightcraft-agent-findings report).

🤖 Generated with Codebuff
Co-Authored-By: Codebuff <noreply@codebuff.com>

* security: bounded catalog keyword, repair, journal recovery work (CodeRabbit)

- keywords: from_file searched every hierarchical level for each flat keyword
  and `same` lower-cased two strings per comparison — one lowered level set
  instead (CWE-407, row 97).
- keywords: parse_keyword_list scanned each keyword's growing synonym list per
  line — a lowered-synonym set per keyword instead (CWE-407, row 100).
- keywords: keyword_tree stored every level's prefix twice, so a photo XMP
  keyword nested deeper than a list file accepts made stored bytes quadratic
  in the level count; the tree caps at MAX_LEVELS, and keyword_list_text names
  whatever lies deeper from the photos' and list's own spellings, so the file
  still reads back (CWE-400, row 103).
- catalog: repair validated a folder chain once per album — memoized by chain
  now, so a valid chain in a flat snapshot costs one walk overall (CWE-407,
  row 101).
- store: journal append and truncate follow a planted catalog.log symlink;
  both refuse any non-regular file at the name before writing through it
  (CWE-59, row 107), like the staging writes since #1502.
- journal: resync re-validated the whole remaining suffix UTF-8 for every
  candidate record start; a whole line is validated once (covering every
  candidate in it), and an invalid line's re-checks stop at a 64 MiB budget,
  with the tail then staying a torn one the damaged path keeps and reports
  (CWE-407, row 109).

New tests: from_file case-insensitive level match, synonyms kept once, a
4096-level keyword cut at MAX_LEVELS, a link at catalog.log refused, and
resync over a fragment of 1024 record starts.

Findings rows 97, 100, 101, 103, 107, 109 of the
storytold-lightcraft-agent-findings report.

🤖 Generated with Codebuff
Co-Authored-By: Codebuff <noreply@codebuff.com>

---------

Co-authored-by: Codebuff <noreply@codebuff.com>
2026-10-23 22:45:21 -04:00
aiongg f3bb7a9f30 CI: one test binary per crate, line tables only, one WASM check (#2710)
* CI: dev profile keeps line tables for workspace crates and no debug info for dependencies

* Control server tests: persistent_requests_and_slow_replies_outlive_the_idle_read_timeout no longer races the idle timeout

Each request (and the closing half-shutdown) reaches the socket before the
worker reads, so on a slow machine only the reply waits outlast the 100 ms
idle timeout, as the test intends. The idle test sends its input the same
way, and the no-frames test allows 5 s instead of 2 s to give up.

* CI: one integration-test binary per crate

Each crate's tests/*.rs become modules of tests/it/main.rs (51 test files
into 13 binaries, plus 6 that stay their own). Inner #![cfg] attributes
move to the mod lines. Files that change state for the whole process keep
a binary of their own: codecs jxl_tail_heap and raw sr2_heap (dhat global
allocator), gpu shutdown, shutdown_first and memory, ui-egui shutdown.
Docs, comments and the GPU probe name the new paths and targets
(--test it <module>::); the CI report names an it binary's skips after
the crate.

* CI: tests under nextest, doctests apart, one wasm check

xtask ci runs the tests with cargo nextest run --lib --bins --tests when
cargo-nextest is installed, else cargo test with the same targets and a
one-line notice; doctests run after them with cargo test --workspace --doc.
The heif and hardware GPU steps use the same runner. Examples stay out of
the test build (clippy --all-targets still checks them). Concurrent tests
under nextest follow the RUST_TEST_THREADS cap (NEXTEST_TEST_THREADS).
The wasm check is one cargo check --keep-going over the same crates, and
clippy runs with --keep-going.

* CI: tests run under cargo test by default, nextest with LIGHTCRAFT_NEXTEST=1 (per-process scene cache)
2026-10-23 13:34:30 -04:00
Chad Elliott 3e42f2e05c feat(testkit): #1037 tests record why they skip
libtest has no skipped status and hides the output of passing tests, so a
test that returns early without a GPU, a corpus file, a craft-fonts face or
a model passed silently. `lightcraft-testkit` (the testkit slot the layering
table already reserves, dev-dependency only, std only) adds
`skip!(Need, "reason")`: it prints the note as before and, when
LIGHTCRAFT_SKIP_LOG names a file, appends one JSON line (need, reason, test,
binary, file:line). Plain `cargo test` is unchanged. The 61 existing skip
sites use it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 16:33:54 -05:00