mirror of
https://github.com/storytold/pdfcraft.git
synced 2026-10-25 02:40:09 +00:00
Clear replacement text (#1350)
* M8: Clear marked-content replacement text * M8: Move roadmap note * M8: Keep inherited resource guard clean under Rust 1.96 --------- Co-authored-by: Codex <codex@openai.com> Co-authored-by: skylar taylor-barrick <hi@skylar.cc>
This commit is contained in:
@@ -2,6 +2,7 @@
|
||||
|
||||
**Stage: alpha** · next: beta, ~25% (ready for real work 50% → 75%) and ~400–700 h away
|
||||
|
||||
> **Last reviewed:** 2026-10-18 · **Last updated:** 2026-10-18 · **Change:** minor (M8, #1350: named replacement descriptions retired during Sanitize and across shared resources; estimates unchanged)
|
||||
> **Last reviewed:** 2026-10-15 · **Last updated:** 2026-10-15 · **Change:** minor (M6: AcroForm XML text; estimates unchanged)
|
||||
> **Last reviewed:** 2026-10-15 · **Last updated:** 2026-10-15 · **Change:** minor (M2/gap 12: the Annex D.2 alias codes read as the ink they draw in both tables; the text oracle scores reading order; estimates unchanged) · **Target:** Adobe Acrobat Pro (Acrobat DC, continuous track 26.002.21931, macOS)
|
||||
> **Last reviewed:** 2026-10-13 · **Last updated:** 2026-10-13 · **Change:** minor (M6: preserve XML/XFDF attribute whitespace; estimates unchanged) · **Target:** Adobe Acrobat Pro (Acrobat DC, continuous track 26.002.21931, macOS)
|
||||
@@ -160,6 +161,7 @@ Ranked; detail and the full milestone table in [docs/roadmap.md](docs/roadmap.md
|
||||
| 8 | 1.0 polish: performance, right-to-left, missing languages, keyboard-only (M14) | 55–95 |
|
||||
|
||||
## Progress log
|
||||
- **2026-10-18 (M8, #1350):** Integrated current main while preserving content-stream comment boundaries. Hidden-text cleanup retires unused named replacement descriptions on rewritten pages and forms, including inherited/shared resource holders, without removing descriptions still used by surviving content. Synthetic saved-byte and preservation regressions cover the cleanup. Estimates unchanged.
|
||||
- **2026-10-15 (M6, AcroForm XML text):** Plain AcroForm XML imports retain all direct text around comments and processing instructions, including CDATA and character references. Synthetic regressions verify exact values and unknown catalog data after incremental and full save/reopen. Nested-element and XFA interpretation are unchanged. Estimates unchanged.
|
||||
- **2026-10-15 (M6, #1917 refresh):** Integrated upstream `99941daa` while retaining the XML/XFDF attribute whitespace correction and all upstream progress. Only independent documentation conflicts required resolution; focused production and test files are unchanged from the previously green CI head. Estimates unchanged.
|
||||
- **2026-10-15 (M1, #2267 refresh):** Integrated upstream `99941daa` while retaining the raw-deflate fallback correction and all upstream progress. Only independent documentation conflicts required resolution; focused production and test files are unchanged from the previously green CI head. Estimates unchanged.
|
||||
@@ -528,7 +530,10 @@ Newest first: the date, what moved, and the new overall estimate.
|
||||
| 2026-10-10 | major | Restructured to the progress-docs standard (stage, two numbers, dimensions, features, languages, upcoming, log); full re-measure; assessment, gaps and milestones moved to `docs/`; log sorted newest first and a stray merged entry dated |
|
||||
| 2026-10-07 | minor | 806 tracked features, 51.0% shipped, effort-weighted ≈ 30–35%, 600–1,100 h |
|
||||
| 2026-10-05 | major | Honest assessment and "Where we're lacking" added |
|
||||
| 2026-10-12 | minor | M8: cleanup removes replacement descriptions only from changed marked-content spans, preserving surviving accessibility text. Estimates unchanged. |
|
||||
| 2026-10-12 | minor | M8: image redaction removes alternate images and unsupported masks only on affected images. Estimates unchanged. |
|
||||
|
||||
- **2026-10-12 (M8):** Redaction cleans indirect and form structure references, preserving surviving form tags.
|
||||
- **2026-10-13 (M7):** Edit a PDF ▸ Add content: Header and footer, Watermark and Background open an Add / Update / Remove menu (Update and Remove enabled only when the document has that mark), as Acrobat's Watermark control does. The catalogue gains `ToolItem::actions`, so any tool can list more commands (#1192).
|
||||
|
||||
- **2026-10-18 (M8, #1350):** Published the history-preserving integration; simplified the inherited-form resource guard for Rust 1.96 Clippy. Formatting, 64 redaction tests, scoped Clippy, layering, assets and the scoped wasm check pass locally. CI remains pending; estimates unchanged.
|
||||
|
||||
@@ -44,6 +44,11 @@ This limits appearance generation; it is not a limit on the input PDF or the who
|
||||
Structure cleanup resolves named MCIDs and indirect content references. Form content
|
||||
references are updated to the cleaned copy; unchanged sibling descriptions remain.
|
||||
|
||||
Replacement descriptions are removed only from marked-content spans changed by cleanup.
|
||||
Unchanged spans retain their descriptions; unused property entries are retired during redaction
|
||||
and hidden-content cleanup, including aliases in shared/inherited resources. Descriptions still
|
||||
used by surviving pages or forms are kept.
|
||||
|
||||
A redacted page that inherits `/Resources` from the page tree gets them pinned (by reference;
|
||||
a direct inherited dictionary is made one shared object first) before it is rewritten. The parent's
|
||||
`/Resources` is then retired unless another page still inherits it, so it cannot keep a removed
|
||||
|
||||
+236
-18
@@ -21,7 +21,7 @@ use std::collections::{HashMap, HashSet};
|
||||
use std::rc::Rc;
|
||||
|
||||
use pdfcraft_content::{Matrix, Op, Pieces, contains, num, overlaps, serialize_ops, string};
|
||||
use pdfcraft_cos::{Dict, Document, ObjRef, Object, Stream};
|
||||
use pdfcraft_cos::{CosError, Dict, Document, ObjRef, Object, Stream};
|
||||
|
||||
use crate::{RedactError, Report, image, page_streams, tags};
|
||||
use pdfcraft_fonts::pdf::Metrics;
|
||||
@@ -66,12 +66,16 @@ pub(crate) struct Output {
|
||||
/// dictionaries may name them too (an inherited or shared /Resources, or the caller's when a
|
||||
/// form has none of its own): [`unlink_doomed`] removes those entries.
|
||||
pub doomed: Vec<ObjRef>,
|
||||
/// Original property values retired by this scope or a nested form. Other resource holders
|
||||
/// may still name them; unlink only entries no surviving content uses.
|
||||
pub retired_properties: Vec<Object>,
|
||||
/// Glyphs / inline images still overlapping a region (verify mode).
|
||||
pub residue: usize,
|
||||
/// Apply mode met vertical-writing text (which it cannot remove) while a redaction region was
|
||||
/// in force: the operation must fail closed rather than leave that text under the mark.
|
||||
pub vertical: bool,
|
||||
pub tag_changes: Vec<(ObjRef, ObjRef, HashSet<i64>, HashSet<i64>)>,
|
||||
pub error: Option<CosError>,
|
||||
}
|
||||
|
||||
pub(crate) struct Scope<'a> {
|
||||
@@ -246,9 +250,10 @@ fn content_draws(doc: &Document, content: &[Vec<u8>], xo: &Dict, name: &[u8], de
|
||||
/// An entry goes unless a page that draws through that dictionary still draws the name (the same
|
||||
/// image shown on another page, or again outside a mark). A dictionary that loses an entry is
|
||||
/// replaced by a private copy on its node, so a dictionary object shared with other nodes is
|
||||
/// left as it is for them. Fails closed when a page's use of a name cannot be told.
|
||||
pub(crate) fn unlink_doomed(doc: &mut Document, doomed: &[ObjRef]) -> Result<(), RedactError> {
|
||||
if doomed.is_empty() {
|
||||
/// left as it is for them. Retired properties are likewise unlinked only when none of the
|
||||
/// holder's pages or inheriting forms names them. Fails closed when use cannot be determined.
|
||||
pub(crate) fn unlink_doomed(doc: &mut Document, doomed: &[ObjRef], retired_properties: &[Object]) -> Result<(), RedactError> {
|
||||
if doomed.is_empty() && retired_properties.is_empty() {
|
||||
return Ok(());
|
||||
}
|
||||
let doomed: HashSet<ObjRef> = doomed.iter().copied().collect();
|
||||
@@ -278,11 +283,16 @@ pub(crate) fn unlink_doomed(doc: &mut Document, doomed: &[ObjRef]) -> Result<(),
|
||||
}
|
||||
}
|
||||
let index: HashMap<ObjRef, usize> = pdfcraft_model::pages(doc).iter().enumerate().map(|(i, p)| (p.obj, i)).collect();
|
||||
let holder_nodes: Vec<ObjRef> = holders.iter().map(|(node, _)| *node).collect();
|
||||
for (node, users) in holders {
|
||||
let Some(res) = doc.get(node).as_dict().and_then(|d| d.get(b"Resources")).and_then(|r| doc.resolve(r).as_dict().cloned()) else { continue };
|
||||
let Some(mut res) = doc.get(node).as_dict().and_then(|d| d.get(b"Resources")).and_then(|r| doc.resolve(r).as_dict().cloned()) else {
|
||||
continue;
|
||||
};
|
||||
let mut xo = res_dict(doc, &res, b"XObject");
|
||||
let names: Vec<Vec<u8>> = xo.iter().filter(|(_, v)| v.as_ref().is_some_and(|r| doomed.contains(&r))).map(|(n, _)| n.to_vec()).collect();
|
||||
if names.is_empty() {
|
||||
let properties = res_dict(doc, &res, b"Properties");
|
||||
let property_names: Vec<Vec<u8>> = properties.iter().filter(|(_, v)| retired_properties.contains(v)).map(|(n, _)| n.to_vec()).collect();
|
||||
if names.is_empty() && property_names.is_empty() {
|
||||
continue;
|
||||
}
|
||||
let mut contents = Vec::with_capacity(users.len());
|
||||
@@ -310,11 +320,84 @@ pub(crate) fn unlink_doomed(doc: &mut Document, doomed: &[ObjRef]) -> Result<(),
|
||||
}
|
||||
}
|
||||
if pruned {
|
||||
let mut res = res;
|
||||
res.set(b"XObject".to_vec(), Object::Dict(xo));
|
||||
}
|
||||
pruned |= prune_retired_properties(doc, &mut res, &property_names, &contents)?;
|
||||
if pruned {
|
||||
doc.update_dict(node, |d| d.set(b"Resources".to_vec(), Object::Dict(res)))?;
|
||||
}
|
||||
}
|
||||
// A surviving form can own the same old resource dictionary as a rewritten page. Its
|
||||
// unused descriptions remain reachable too, even after every page holder is cleaned.
|
||||
unlink_form_properties(doc, &holder_nodes, retired_properties)?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Retire only properties removed from a rewritten scope, preserving names still used by any
|
||||
/// content sharing this holder. Different pages' streams must never be concatenated together.
|
||||
fn prune_retired_properties(doc: &Document, res: &mut Dict, names: &[Vec<u8>], contents: &[Vec<Vec<u8>>]) -> Result<bool, CosError> {
|
||||
if names.is_empty() {
|
||||
return Ok(false);
|
||||
}
|
||||
let mut used = HashSet::new();
|
||||
for content in contents {
|
||||
for op in resource_ops(doc, res, content)? {
|
||||
if matches!(op.op.as_slice(), b"BDC" | b"DP")
|
||||
&& let Some(name) = op.name(1)
|
||||
{
|
||||
used.insert(name.to_vec());
|
||||
}
|
||||
}
|
||||
}
|
||||
let mut properties = res_dict(doc, res, b"Properties");
|
||||
let mut pruned = false;
|
||||
for name in names {
|
||||
if !used.contains(name) {
|
||||
pruned |= properties.remove(name).is_some();
|
||||
}
|
||||
}
|
||||
if pruned {
|
||||
if properties.is_empty() {
|
||||
res.remove(b"Properties");
|
||||
} else {
|
||||
res.set(b"Properties".to_vec(), Object::Dict(properties));
|
||||
}
|
||||
}
|
||||
Ok(pruned)
|
||||
}
|
||||
|
||||
fn unlink_form_properties(doc: &mut Document, holders: &[ObjRef], retired: &[Object]) -> Result<(), CosError> {
|
||||
if retired.is_empty() {
|
||||
return Ok(());
|
||||
}
|
||||
let mut pending: Vec<Dict> = holders
|
||||
.iter()
|
||||
.filter_map(|r| doc.get(*r).as_dict().and_then(|d| d.get(b"Resources")).and_then(|r| doc.resolve(r).as_dict().cloned()))
|
||||
.collect();
|
||||
let mut seen = HashSet::new();
|
||||
while let Some(resources) = pending.pop() {
|
||||
for (_, value) in res_dict(doc, &resources, b"XObject").iter() {
|
||||
let Some(reference) = value.as_ref() else { continue };
|
||||
if !seen.insert(reference) {
|
||||
continue;
|
||||
}
|
||||
if seen.len() > 1_000_000 {
|
||||
return Err(CosError::Syntax { offset: 0, detail: "redaction could not fully inspect shared form resources".into() });
|
||||
}
|
||||
let object = doc.get(reference);
|
||||
let Object::Stream(form) = &*object else { continue };
|
||||
if form.dict.name(b"Subtype") != Some(b"Form") {
|
||||
continue;
|
||||
}
|
||||
let Some(mut res) = form.dict.get(b"Resources").and_then(|r| doc.resolve(r).as_dict().cloned()) else { continue };
|
||||
let names: Vec<Vec<u8>> =
|
||||
res_dict(doc, &res, b"Properties").iter().filter(|(_, value)| retired.contains(value)).map(|(name, _)| name.to_vec()).collect();
|
||||
if !names.is_empty() && prune_retired_properties(doc, &mut res, &names, &[vec![form.decoded()?]])? {
|
||||
doc.update_dict(reference, |d| d.set(b"Resources".to_vec(), Object::Dict(res.clone())))?;
|
||||
}
|
||||
pending.push(res);
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
@@ -341,6 +424,71 @@ fn clip_out(rects: &[[f64; 4]], ctm: &Matrix, around: [f64; 4]) -> Option<Vec<Op
|
||||
Some(ops)
|
||||
}
|
||||
|
||||
fn resource_ops(doc: &Document, res: &Dict, content: &[Vec<u8>]) -> Result<Vec<Op>, CosError> {
|
||||
let parsed = Pieces::join(content).parse();
|
||||
if parsed.skipped > 0 || parsed.ops.len() > 2_000_000 {
|
||||
return Err(CosError::Syntax { offset: 0, detail: "redaction could not fully inspect property resources".into() });
|
||||
}
|
||||
let mut ops = parsed.ops;
|
||||
let xobjects = res_dict(doc, res, b"XObject");
|
||||
let mut seen = HashSet::new();
|
||||
let mut cursor = 0;
|
||||
while cursor < ops.len() {
|
||||
let op = &ops[cursor];
|
||||
if op.is("Do")
|
||||
&& let Some(r) = op.name(0).and_then(|n| xobjects.get(n)).and_then(Object::as_ref)
|
||||
&& seen.insert(r)
|
||||
&& let Object::Stream(s) = &*doc.get(r)
|
||||
&& s.dict.name(b"Subtype") == Some(b"Form")
|
||||
&& s.dict.get(b"Resources").is_none_or(|r| doc.resolve(r).as_dict().is_none())
|
||||
{
|
||||
let parsed = pdfcraft_content::parse(&s.decoded()?);
|
||||
if seen.len() > 2_000_000 || ops.len().saturating_add(parsed.ops.len()) > 2_000_000 || parsed.skipped > 0 {
|
||||
return Err(CosError::Syntax { offset: 0, detail: "redaction could not fully inspect inherited form resources".into() });
|
||||
}
|
||||
ops.extend(parsed.ops);
|
||||
}
|
||||
cursor += 1;
|
||||
}
|
||||
Ok(ops)
|
||||
}
|
||||
|
||||
pub(crate) fn prune_properties(doc: &Document, res: &mut Dict, content: &[Vec<u8>]) -> Result<Vec<Object>, CosError> {
|
||||
let mut properties = res_dict(doc, res, b"Properties");
|
||||
if properties.is_empty() {
|
||||
return Ok(Vec::new());
|
||||
}
|
||||
let mut retired = Vec::new();
|
||||
let ops = resource_ops(doc, res, content)?;
|
||||
let names: Vec<Vec<u8>> = properties.iter().map(|(n, _)| n.to_vec()).collect();
|
||||
for name in names {
|
||||
if !ops.iter().any(|op| matches!(op.op.as_slice(), b"BDC" | b"DP") && op.name(1) == Some(&name))
|
||||
&& let Some(value) = properties.remove(&name)
|
||||
{
|
||||
retired.push(value);
|
||||
}
|
||||
}
|
||||
if properties.is_empty() {
|
||||
res.remove(b"Properties");
|
||||
} else {
|
||||
res.set(b"Properties".to_vec(), Object::Dict(properties));
|
||||
}
|
||||
Ok(retired)
|
||||
}
|
||||
|
||||
fn prune_form_resources(doc: &Document, res: &mut Dict, content: &[Vec<u8>]) -> Result<(), CosError> {
|
||||
let ops = resource_ops(doc, res, content)?;
|
||||
let mut xo = res_dict(doc, res, b"XObject");
|
||||
let names: Vec<Vec<u8>> = xo.iter().map(|(n, _)| n.to_vec()).collect();
|
||||
for name in names {
|
||||
if !ops.iter().any(|op| op.is("Do") && op.name(0) == Some(&name)) {
|
||||
xo.remove(&name);
|
||||
}
|
||||
}
|
||||
res.set(b"XObject".to_vec(), Object::Dict(xo));
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// One glyph of a shown string.
|
||||
struct Glyph {
|
||||
bytes: std::ops::Range<usize>,
|
||||
@@ -383,6 +531,9 @@ pub(crate) fn process(doc: &mut Document, scope: &mut Scope<'_>, streams: &[Vec<
|
||||
// A rewritten piece loses its comments, so keep where they began and ended: a comment that
|
||||
// ran on into the next piece still hides that piece's first line.
|
||||
let comment_edges = joined.comment_edges(&parsed);
|
||||
// Layer removal also changes enclosing replacement spans, so remember the operators before
|
||||
// any cleanup, not just the survivors of the layer pass.
|
||||
let original = parsed.ops.clone();
|
||||
if !scope.hidden_layers.is_empty() {
|
||||
let props = res_dict(doc, resources, b"Properties");
|
||||
let (kept, removed) = strip_hidden_layers(doc, scope, parsed.ops, &props);
|
||||
@@ -408,7 +559,8 @@ pub(crate) fn process(doc: &mut Document, scope: &mut Scope<'_>, streams: &[Vec<
|
||||
}
|
||||
}
|
||||
|
||||
for ((piece, mut changed), (comment_in, comment_out)) in pieces.into_iter().zip(rewrite).zip(comment_edges) {
|
||||
let mut processed = Vec::new();
|
||||
for (piece, mut changed) in pieces.into_iter().zip(rewrite) {
|
||||
let mut ops: Vec<Op> = Vec::with_capacity(piece.len());
|
||||
for op in piece {
|
||||
let o = op.op.as_slice();
|
||||
@@ -625,6 +777,12 @@ pub(crate) fn process(doc: &mut Document, scope: &mut Scope<'_>, streams: &[Vec<
|
||||
if !path.is_empty() {
|
||||
ops.append(&mut path);
|
||||
}
|
||||
processed.push((ops, changed));
|
||||
}
|
||||
if scope.mode == Mode::Apply {
|
||||
clean_replacements(doc, resources, &original, &mut processed, scope.report);
|
||||
}
|
||||
for ((ops, changed), (comment_in, comment_out)) in processed.into_iter().zip(comment_edges) {
|
||||
out.streams.push(changed.then(|| {
|
||||
// The piece before still ends inside a comment: end it before the first operator.
|
||||
let mut bytes = if comment_in { b"\n".to_vec() } else { Vec::new() };
|
||||
@@ -639,6 +797,48 @@ pub(crate) fn process(doc: &mut Document, scope: &mut Scope<'_>, streams: &[Vec<
|
||||
out
|
||||
}
|
||||
|
||||
fn clean_replacements(doc: &Document, resources: &Dict, original: &[Op], processed: &mut [(Vec<Op>, bool)], report: &mut Report) {
|
||||
let originals: HashMap<usize, &Op> = original.iter().map(|op| (op.span.start, op)).collect();
|
||||
let unchanged: HashSet<usize> =
|
||||
processed.iter().flat_map(|(ops, _)| ops).filter(|op| originals.get(&op.span.start) == Some(op)).map(|op| op.span.start).collect();
|
||||
let mut stack = Vec::new();
|
||||
let mut affected = HashSet::new();
|
||||
for op in original {
|
||||
match op.op.as_slice() {
|
||||
b"BDC" | b"BMC" => stack.push(op.span.start),
|
||||
b"EMC" => {
|
||||
stack.pop();
|
||||
}
|
||||
_ if !unchanged.contains(&op.span.start) => affected.extend(stack.iter().copied()),
|
||||
_ => {}
|
||||
}
|
||||
}
|
||||
let properties = res_dict(doc, resources, b"Properties");
|
||||
for (ops, changed) in processed {
|
||||
for op in ops {
|
||||
if !op.is("BDC") || !affected.contains(&op.span.start) {
|
||||
continue;
|
||||
}
|
||||
let property = match op.operands.get(1) {
|
||||
Some(Object::Name(name)) => properties.get(name).map(|p| doc.resolve(p)),
|
||||
Some(p) => Some(doc.resolve(p)),
|
||||
None => None,
|
||||
};
|
||||
if let Some(mut dict) = property.and_then(|p| p.as_dict().cloned()) {
|
||||
let mut stripped = false;
|
||||
for key in [&b"ActualText"[..], b"Alt", b"E", b"T"] {
|
||||
stripped |= dict.remove(key).is_some();
|
||||
}
|
||||
if stripped && let Some(property) = op.operands.get_mut(1) {
|
||||
*property = Object::Dict(dict);
|
||||
*changed = true;
|
||||
report.replacement_text += 1;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Lay out a shown string (or `TJ` array) glyph by glyph, advancing `tm`. Returns the `TJ`
|
||||
/// items with the glyphs under a region replaced by displacements, and how many were removed.
|
||||
fn show(scope: &Scope<'_>, gs: &Gs, tm: &mut Matrix, items: &[Object]) -> (Vec<Object>, usize) {
|
||||
@@ -810,7 +1010,9 @@ fn xobject(
|
||||
out.residue += inner.residue;
|
||||
out.vertical |= inner.vertical;
|
||||
out.doomed.extend_from_slice(&inner.doomed);
|
||||
out.retired_properties.extend(inner.retired_properties);
|
||||
out.tag_changes.extend(inner.tag_changes);
|
||||
out.error = inner.error.or(out.error.take());
|
||||
if scope.mode == Mode::Verify {
|
||||
return None;
|
||||
}
|
||||
@@ -820,19 +1022,35 @@ fn xobject(
|
||||
}
|
||||
let mut dict = s.dict.clone();
|
||||
dict.remove(b"Length");
|
||||
// Rebuild the copy's /Resources whenever inner content gained or lost an XObject, so a
|
||||
// `Do` the copy no longer makes stops keeping the original image or form reachable.
|
||||
let mut cleaned_res = res.clone();
|
||||
// Install rewritten nested forms before finding live property names: a new form may
|
||||
// still inherit its caller's resources, and its new Do name must resolve here.
|
||||
if !inner.xobjects.is_empty() || !inner.replaced.is_empty() {
|
||||
let mut res = res.clone();
|
||||
let mut xo = res_dict(doc, &res, b"XObject");
|
||||
if let Some(d) = &new_data {
|
||||
prune(&mut xo, &inner.replaced, std::slice::from_ref(d));
|
||||
let mut xo = res_dict(doc, &cleaned_res, b"XObject");
|
||||
if let Some(data) = &new_data {
|
||||
prune(&mut xo, &inner.replaced, std::slice::from_ref(data));
|
||||
}
|
||||
for (n, r) in &inner.xobjects {
|
||||
xo.set(n.clone(), Object::Ref(*r));
|
||||
for (name, reference) in &inner.xobjects {
|
||||
xo.set(name.clone(), Object::Ref(*reference));
|
||||
}
|
||||
res.set(b"XObject".to_vec(), Object::Dict(xo));
|
||||
dict.set(b"Resources".to_vec(), Object::Dict(res));
|
||||
cleaned_res.set(b"XObject".to_vec(), Object::Dict(xo));
|
||||
}
|
||||
if let Some(data) = &new_data {
|
||||
let content = std::slice::from_ref(data);
|
||||
match prune_properties(doc, &mut cleaned_res, content) {
|
||||
Ok(retired) => out.retired_properties.extend(retired),
|
||||
Err(error) => {
|
||||
out.error = Some(error);
|
||||
return None;
|
||||
}
|
||||
}
|
||||
if let Err(error) = prune_form_resources(doc, &mut cleaned_res, content) {
|
||||
out.error = Some(error);
|
||||
return None;
|
||||
}
|
||||
}
|
||||
if cleaned_res != res {
|
||||
dict.set(b"Resources".to_vec(), Object::Dict(cleaned_res));
|
||||
}
|
||||
let (changed, empty) = tags::touched(doc, &res, &data, new_data.as_deref().unwrap_or(&data));
|
||||
let stream = match new_data {
|
||||
|
||||
@@ -119,6 +119,7 @@ pub struct Report {
|
||||
pub forms_removed: usize,
|
||||
pub annotations: usize,
|
||||
pub fields: usize,
|
||||
pub replacement_text: usize,
|
||||
/// Structure elements that lost alternate/actual text or emptied marked content.
|
||||
pub tags: usize,
|
||||
/// The form was an AcroForm+XFA form and its `/XFA` packets (which still held redacted field
|
||||
@@ -334,6 +335,7 @@ pub fn apply(doc: &mut Document, pages: Option<&[usize]>) -> Result<Report, Reda
|
||||
let mut doomed_fields: Vec<String> = Vec::new();
|
||||
// The original image and form XObjects the rewrite drops or replaces, on any page.
|
||||
let mut doomed_xobjects: Vec<ObjRef> = Vec::new();
|
||||
let mut retired_properties = Vec::new();
|
||||
let widget_owner: Vec<(ObjRef, String)> =
|
||||
pdfcraft_forms::fields(doc).into_iter().flat_map(|f| f.widgets.iter().map(|w| (w.obj, f.name.clone())).collect::<Vec<_>>()).collect();
|
||||
|
||||
@@ -353,10 +355,14 @@ pub fn apply(doc: &mut Document, pages: Option<&[usize]>) -> Result<Report, Reda
|
||||
// leave the text in place, refuse the whole operation (the caller keeps the previous
|
||||
// document). The error does not claim the text was under the mark. Full vertical-text
|
||||
// removal is a future enhancement.
|
||||
if let Some(error) = out.error {
|
||||
return Err(error.into());
|
||||
}
|
||||
if out.vertical {
|
||||
return Err(RedactError::VerticalText(pi + 1));
|
||||
}
|
||||
doomed_xobjects.extend_from_slice(&out.doomed);
|
||||
retired_properties.extend(out.retired_properties);
|
||||
let mut new_list = list.clone();
|
||||
let mut changed = false;
|
||||
for (i, new) in out.streams.into_iter().enumerate() {
|
||||
@@ -393,6 +399,16 @@ pub fn apply(doc: &mut Document, pages: Option<&[usize]>) -> Result<Report, Reda
|
||||
}
|
||||
res.set(b"XObject".to_vec(), Object::Dict(xo));
|
||||
}
|
||||
let (_, cleaned_data) = page_streams(
|
||||
doc,
|
||||
&{
|
||||
let mut cleaned_page = page.dict.clone();
|
||||
cleaned_page.set(b"Contents".to_vec(), Object::Array(new_list.clone()));
|
||||
cleaned_page
|
||||
},
|
||||
pi,
|
||||
)?;
|
||||
retired_properties.extend(interp::prune_properties(doc, &mut res, &cleaned_data)?);
|
||||
doc.update_dict(page.obj, |d| {
|
||||
d.set(b"Contents".to_vec(), Object::Array(new_list));
|
||||
d.set(b"Resources".to_vec(), Object::Dict(res));
|
||||
@@ -488,7 +504,7 @@ pub fn apply(doc: &mut Document, pages: Option<&[usize]>) -> Result<Report, Reda
|
||||
|
||||
// A removed image or form must not stay reachable through another resource dictionary (an
|
||||
// inherited or shared /Resources): a full save keeps whatever is still referenced.
|
||||
interp::unlink_doomed(doc, &doomed_xobjects)?;
|
||||
interp::unlink_doomed(doc, &doomed_xobjects, &retired_properties)?;
|
||||
|
||||
// 5. Verify: nothing readable may remain under a region.
|
||||
let mut residue = 0;
|
||||
|
||||
@@ -146,6 +146,7 @@ fn content_pass(doc: &mut Document, text: bool, layers: bool, write: bool) -> Re
|
||||
let mut total = 0;
|
||||
// The original image and form XObjects the rewrite drops or replaces, on any page.
|
||||
let mut doomed = Vec::new();
|
||||
let mut retired_properties = Vec::new();
|
||||
let n = pdfcraft_model::pages(doc).len();
|
||||
for pi in 0..n {
|
||||
let page = pdfcraft_model::pages(doc).swap_remove(pi);
|
||||
@@ -165,12 +166,16 @@ fn content_pass(doc: &mut Document, text: bool, layers: bool, write: bool) -> Re
|
||||
}
|
||||
scope.hidden_layers = off.clone();
|
||||
let out = process(doc, &mut scope, &data, &resources, Matrix::IDENTITY);
|
||||
if let Some(error) = out.error {
|
||||
return Err(error.into());
|
||||
}
|
||||
let blocks = scope.layer_blocks;
|
||||
total += out.residue + blocks + report.glyphs;
|
||||
if !write {
|
||||
continue;
|
||||
}
|
||||
doomed.extend_from_slice(&out.doomed);
|
||||
retired_properties.extend(out.retired_properties);
|
||||
let mut new_list = list.clone();
|
||||
let mut changed = false;
|
||||
for (i, new) in out.streams.into_iter().enumerate() {
|
||||
@@ -184,24 +189,24 @@ fn content_pass(doc: &mut Document, text: bool, layers: bool, write: bool) -> Re
|
||||
changed = true;
|
||||
}
|
||||
if changed || !out.xobjects.is_empty() || !out.replaced.is_empty() {
|
||||
// Pin inherited resources before replacing them. When no page still inherits the
|
||||
// original holder, retiring it also retires descriptions no longer used anywhere.
|
||||
crate::localize_resources(doc, &[pi])?;
|
||||
let mut cleaned_page = page.dict.clone();
|
||||
cleaned_page.set(b"Contents".to_vec(), Object::Array(new_list.clone()));
|
||||
let (_, page_content) = page_streams(doc, &cleaned_page, pi)?;
|
||||
let mut res = resources;
|
||||
if !out.xobjects.is_empty() || !out.replaced.is_empty() {
|
||||
let mut xo = res.get(b"XObject").and_then(|x| doc.resolve(x).as_dict().cloned()).unwrap_or_default();
|
||||
// A full save keeps whatever is still referenced: unlink the originals this page no
|
||||
// longer draws so the removed form or image bytes cannot survive in the saved file.
|
||||
let page_content: Vec<Vec<u8>> = new_list
|
||||
.iter()
|
||||
.map(|o| match &*doc.resolve(o) {
|
||||
Object::Stream(s) => s.decoded().unwrap_or_default(),
|
||||
_ => Vec::new(),
|
||||
})
|
||||
.collect();
|
||||
crate::interp::prune(&mut xo, &out.replaced, &page_content);
|
||||
for (n, r) in &out.xobjects {
|
||||
xo.set(n.clone(), Object::Ref(*r));
|
||||
}
|
||||
res.set(b"XObject".to_vec(), Object::Dict(xo));
|
||||
}
|
||||
retired_properties.extend(crate::interp::prune_properties(doc, &mut res, &page_content)?);
|
||||
doc.update_dict(page.obj, |d| {
|
||||
d.set(b"Contents".to_vec(), Object::Array(new_list));
|
||||
d.set(b"Resources".to_vec(), Object::Dict(res));
|
||||
@@ -210,7 +215,7 @@ fn content_pass(doc: &mut Document, text: bool, layers: bool, write: bool) -> Re
|
||||
}
|
||||
// What a page no longer draws must not stay reachable through an inherited or shared
|
||||
// /Resources either.
|
||||
crate::interp::unlink_doomed(doc, &doomed)?;
|
||||
crate::interp::unlink_doomed(doc, &doomed, &retired_properties)?;
|
||||
Ok(total)
|
||||
}
|
||||
|
||||
|
||||
@@ -1056,6 +1056,100 @@ fn redacting_every_inheriting_page_retires_the_parent_resources() {
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn replacement_text_and_property_resources_do_not_survive_redaction() {
|
||||
let mut doc = one_page(
|
||||
b"/Span /P0 BDC /Span <</ActualText (LEAK_INLINE)>> BDC BT /F1 10 Tf 10 200 Td (SECRET)",
|
||||
"/Properties << /P0 7 0 R >>",
|
||||
vec![
|
||||
b"<< /MCID 0 /ActualText (LEAK_PROPERTY) /Alt (LEAK_ALT) /E (LEAK_EXPANSION) >>".to_vec(),
|
||||
stream("", b"Tj ET EMC EMC /Span <</ActualText (Public point)>> DP BT /F1 10 Tf 10 100 Td (Public) Tj ET"),
|
||||
],
|
||||
);
|
||||
doc.update_dict(ObjRef::new(3, 0), |d| {
|
||||
d.set(b"Contents".to_vec(), Object::Array(vec![Object::Ref(ObjRef::new(4, 0)), Object::Ref(ObjRef::new(8, 0))]));
|
||||
})
|
||||
.unwrap();
|
||||
mark(&mut doc, 0, &[[5.0, 195.0, 60.0, 212.0]], "");
|
||||
apply(&mut doc, None).unwrap();
|
||||
assert_no_secret(&doc, b"LEAK_");
|
||||
let saved = reopen(&doc);
|
||||
assert!(content(&saved, 0).contains("Public"));
|
||||
let p = pdfcraft_model::pages(&saved).swap_remove(0);
|
||||
assert!(!saved.resolve(p.dict.get(b"Resources").unwrap()).as_dict().unwrap().contains(b"Properties"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn inherited_form_properties_lose_replacement_text() {
|
||||
let mut doc = one_page(
|
||||
b"/Fm Do",
|
||||
"/XObject << /Fm 7 0 R >> /Properties << /P0 8 0 R >>",
|
||||
vec![
|
||||
stream(
|
||||
"/Type /XObject /Subtype /Form /BBox [0 0 300 300]",
|
||||
b"/Span /P0 BDC BT /F1 10 Tf 10 200 Td (SECRET) Tj ET EMC BT /F1 10 Tf 10 100 Td (Public) Tj ET",
|
||||
),
|
||||
b"<< /ActualText (LEAK_FORM_PROPERTY) >>".to_vec(),
|
||||
],
|
||||
);
|
||||
mark(&mut doc, 0, &[[5.0, 195.0, 60.0, 212.0]], "");
|
||||
apply(&mut doc, None).unwrap();
|
||||
assert_no_secret(&doc, b"LEAK_");
|
||||
assert_no_secret(&doc, b"SECRET");
|
||||
let mut saved = reopen(&doc);
|
||||
assert_eq!(under(&mut saved, 0, &[[0.0, 90.0, 100.0, 110.0]]), 6);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn properties_used_by_unmodified_inheriting_forms_stay() {
|
||||
let mut doc = one_page(
|
||||
b"BT /F1 10 Tf 10 200 Td (SECRET) Tj ET /Fm Do",
|
||||
"/XObject << /Fm 7 0 R >> /Properties << /P0 8 0 R >>",
|
||||
vec![stream("/Type /XObject /Subtype /Form /BBox [0 0 10 10]", b"/Span /P0 BDC 0 0 10 10 re f EMC"), b"<< /ActualText (Public) >>".to_vec()],
|
||||
);
|
||||
mark(&mut doc, 0, &[[5.0, 195.0, 60.0, 212.0]], "");
|
||||
apply(&mut doc, None).unwrap();
|
||||
let saved = reopen(&doc);
|
||||
let page = pdfcraft_model::pages(&saved).swap_remove(0);
|
||||
let resources = saved.resolve(page.dict.get(b"Resources").unwrap());
|
||||
let properties = saved.resolve(resources.as_dict().unwrap().get(b"Properties").unwrap());
|
||||
let property = saved.resolve(properties.as_dict().unwrap().get(b"P0").unwrap());
|
||||
assert_eq!(property.as_dict().unwrap().get(b"ActualText").unwrap().as_string().unwrap().to_text(), "Public");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn untouched_replacement_descriptions_survive_redaction_and_hidden_text_cleanup() {
|
||||
for hidden in [false, true] {
|
||||
let mode = if hidden { 3 } else { 0 };
|
||||
let data = format!(
|
||||
"/Span <</ActualText (LEAK_REMOVED)>> BDC BT /F1 10 Tf {mode} Tr 10 200 Td (SECRET) Tj ET EMC \
|
||||
/Span /Public BDC BT /F1 10 Tf 0 Tr 10 100 Td (Public) Tj ET EMC"
|
||||
);
|
||||
let mut doc = one_page(
|
||||
data.as_bytes(),
|
||||
"/Properties << /Public 7 0 R >>",
|
||||
vec![b"<< /ActualText (PUBLIC_ACTUAL) /Alt (PUBLIC_ALT) /E (PUBLIC_EXPANSION) /T (PUBLIC_TITLE) >>".to_vec()],
|
||||
);
|
||||
if hidden {
|
||||
sanitize::remove_hidden(&mut doc, &[sanitize::Hidden::HiddenText]).unwrap();
|
||||
} else {
|
||||
mark(&mut doc, 0, &[[5.0, 195.0, 60.0, 212.0]], "");
|
||||
apply(&mut doc, None).unwrap();
|
||||
}
|
||||
assert_no_secret(&doc, b"LEAK_");
|
||||
assert_no_secret(&doc, b"SECRET");
|
||||
let saved = reopen(&doc);
|
||||
let page = pdfcraft_model::pages(&saved).swap_remove(0);
|
||||
let resources = saved.resolve(page.dict.get(b"Resources").unwrap());
|
||||
let properties = saved.resolve(resources.as_dict().unwrap().get(b"Properties").unwrap());
|
||||
let property = saved.resolve(properties.as_dict().unwrap().get(b"Public").unwrap());
|
||||
let dict = property.as_dict().unwrap();
|
||||
for key in [&b"ActualText"[..], b"Alt", b"E", b"T"] {
|
||||
assert!(dict.contains(key), "surviving description removed: {key:?}, hidden={hidden}");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_hostile_icc_component_count_is_refused_not_multiplied_into_a_crash() {
|
||||
// /N comes from the file. 2^60 × 8 bpc × width 2 overflows the row size; a negative /N wrapped
|
||||
@@ -1156,3 +1250,86 @@ fn rewriting_a_piece_keeps_its_leading_comment_terminator() {
|
||||
assert!(ops.iter().any(|o| o.operands.iter().any(|v| v.as_string().is_some_and(|s| s.bytes == b"KEEP"))));
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn named_hidden_text_descriptions_are_removed_from_saved_bytes() {
|
||||
let mut doc = one_page(
|
||||
b"/Span /P0 BDC BT /F1 10 Tf 3 Tr 10 200 Td (SECRET) Tj ET EMC",
|
||||
"/Properties << /P0 7 0 R >>",
|
||||
vec![b"<< /ActualText (LEAK_ACTUAL) /Alt (LEAK_ALT) /E (LEAK_EXPANSION) /T (LEAK_TITLE) >>".to_vec()],
|
||||
);
|
||||
sanitize::remove_hidden(&mut doc, &[sanitize::Hidden::HiddenText]).unwrap();
|
||||
assert_no_secret(&doc, b"SECRET");
|
||||
assert_no_secret(&doc, b"LEAK_");
|
||||
let saved = reopen(&doc);
|
||||
let page = pdfcraft_model::pages(&saved).swap_remove(0);
|
||||
assert!(!saved.resolve(page.dict.get(b"Resources").unwrap()).as_dict().unwrap().contains(b"Properties"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn unused_descriptions_in_shared_or_inherited_resources_are_retired() {
|
||||
for hidden in [false, true] {
|
||||
for inherited in [false, true] {
|
||||
let mode = if hidden { 3 } else { 0 };
|
||||
let data = format!("/Span /P0 BDC BT /F1 10 Tf {mode} Tr 10 200 Td (SECRET) Tj ET EMC /Fm Do");
|
||||
let mut doc = one_page(
|
||||
data.as_bytes(),
|
||||
"/Properties << /P0 7 0 R /Public 8 0 R >> /XObject << /Fm 9 0 R >>",
|
||||
vec![
|
||||
b"<< /ActualText (LEAK_ACTUAL) /Alt (LEAK_ALT) /E (LEAK_EXPANSION) /T (LEAK_TITLE) >>".to_vec(),
|
||||
b"<< /ActualText (PUBLIC_ACTUAL) /Alt (PUBLIC_ALT) /E (PUBLIC_EXPANSION) /T (PUBLIC_TITLE) >>".to_vec(),
|
||||
stream("/Type /XObject /Subtype /Form /BBox [0 0 100 120]", b"/Span /Public BDC BT /F1 10 Tf 0 Tr 10 100 Td (Public) Tj ET EMC"),
|
||||
],
|
||||
);
|
||||
let first = doc.get(ObjRef::new(3, 0)).as_dict().unwrap().clone();
|
||||
let resources = doc.add(first.get(b"Resources").unwrap().clone());
|
||||
let second_content = doc.add(Object::Stream(Stream::from_raw(Dict::new(), b"/Fm Do".to_vec())));
|
||||
let mut second = first;
|
||||
second.set(b"Contents".to_vec(), Object::Ref(second_content));
|
||||
if inherited {
|
||||
second.remove(b"Resources");
|
||||
} else {
|
||||
second.set(b"Resources".to_vec(), Object::Ref(resources));
|
||||
}
|
||||
let second = doc.add(Object::Dict(second));
|
||||
doc.update_dict(ObjRef::new(3, 0), |d| {
|
||||
if inherited {
|
||||
d.remove(b"Resources");
|
||||
} else {
|
||||
d.set(b"Resources".to_vec(), Object::Ref(resources));
|
||||
}
|
||||
})
|
||||
.unwrap();
|
||||
doc.update_dict(ObjRef::new(2, 0), |d| {
|
||||
d.set(b"Kids".to_vec(), Object::Array(vec![Object::Ref(ObjRef::new(3, 0)), Object::Ref(second)]));
|
||||
d.set(b"Count".to_vec(), Object::Int(2));
|
||||
if inherited {
|
||||
d.set(b"Resources".to_vec(), Object::Ref(resources));
|
||||
}
|
||||
})
|
||||
.unwrap();
|
||||
if hidden {
|
||||
sanitize::remove_hidden(&mut doc, &[sanitize::Hidden::HiddenText]).unwrap();
|
||||
} else {
|
||||
mark(&mut doc, 0, &[[5.0, 195.0, 60.0, 212.0]], "");
|
||||
apply(&mut doc, Some(&[0])).unwrap();
|
||||
}
|
||||
assert_no_secret(&doc, b"LEAK_");
|
||||
assert_no_secret(&doc, b"SECRET");
|
||||
let mut saved = reopen(&doc);
|
||||
for page_index in 0..2 {
|
||||
assert_eq!(under(&mut saved, page_index, &[[0.0, 90.0, 100.0, 110.0]]), 6);
|
||||
let page = pdfcraft_model::pages(&saved).swap_remove(page_index);
|
||||
let resources = saved.resolve(page.dict.get(b"Resources").unwrap());
|
||||
let properties = saved.resolve(resources.as_dict().unwrap().get(b"Properties").unwrap());
|
||||
assert!(!properties.as_dict().unwrap().contains(b"P0"));
|
||||
let property = saved.resolve(properties.as_dict().unwrap().get(b"Public").unwrap());
|
||||
for key in [&b"ActualText"[..], b"Alt", b"E", b"T"] {
|
||||
assert!(property.as_dict().unwrap().contains(key), "shared form lost {key:?}: hidden={hidden}, inherited={inherited}");
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
mod property_retirement;
|
||||
|
||||
@@ -0,0 +1,212 @@
|
||||
use super::*;
|
||||
|
||||
const SECRET: &[u8] = b"/Span /P0 BDC BT /F1 10 Tf 10 200 Td (SECRET) Tj ET EMC";
|
||||
const DESCRIPTION: &[u8] = b"<< /ActualText (LEAK_ACTUAL) /Alt (LEAK_ALT) /E (LEAK_EXPANSION) /T (LEAK_TITLE) >>";
|
||||
const PUBLIC: &[u8] = b"<< /ActualText (PUBLIC_ACTUAL) /Alt (PUBLIC_ALT) /E (PUBLIC_EXPANSION) /T (PUBLIC_TITLE) >>";
|
||||
|
||||
fn resources(doc: &Document, page: usize) -> Dict {
|
||||
let page = pdfcraft_model::pages(doc).swap_remove(page);
|
||||
doc.resolve(page.dict.get(b"Resources").unwrap()).as_dict().unwrap().clone()
|
||||
}
|
||||
|
||||
fn property(doc: &Document, resources: &Dict, name: &[u8]) -> Option<Dict> {
|
||||
let properties = doc.resolve(resources.get(b"Properties")?);
|
||||
doc.resolve(properties.as_dict()?.get(name)?).as_dict().cloned()
|
||||
}
|
||||
|
||||
fn assert_description(dict: &Dict) {
|
||||
for key in [&b"ActualText"[..], b"Alt", b"E", b"T"] {
|
||||
assert!(dict.contains(key), "surviving description lost {key:?}");
|
||||
}
|
||||
}
|
||||
|
||||
fn add_second_page(doc: &mut Document, data: &[u8], resources: Object) -> ObjRef {
|
||||
let mut page = doc.get(ObjRef::new(3, 0)).as_dict().unwrap().clone();
|
||||
let contents = doc.add(Object::Stream(Stream::from_raw(Dict::new(), data.to_vec())));
|
||||
page.set(b"Contents".to_vec(), Object::Ref(contents));
|
||||
page.set(b"Resources".to_vec(), resources);
|
||||
let second = doc.add(Object::Dict(page));
|
||||
doc.update_dict(ObjRef::new(2, 0), |d| {
|
||||
d.set(b"Kids".to_vec(), Object::Array(vec![Object::Ref(ObjRef::new(3, 0)), Object::Ref(second)]));
|
||||
d.set(b"Count".to_vec(), Object::Int(2));
|
||||
})
|
||||
.unwrap();
|
||||
second
|
||||
}
|
||||
|
||||
fn redact_first(doc: &mut Document) -> Result<Report, RedactError> {
|
||||
mark(doc, 0, &[[5.0, 195.0, 60.0, 212.0]], "");
|
||||
apply(doc, Some(&[0]))
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn surviving_form_own_resources_do_not_retain_unused_shared_descriptions() {
|
||||
for hidden in [false, true] {
|
||||
let mode = if hidden { 3 } else { 0 };
|
||||
let data = format!("/Span /P0 BDC BT /F1 10 Tf {mode} Tr 10 200 Td (SECRET) Tj ET EMC /Fm Do");
|
||||
let mut doc = one_page(
|
||||
data.as_bytes(),
|
||||
"/Properties << /P0 7 0 R /Public 8 0 R >> /XObject << /Fm 9 0 R >>",
|
||||
vec![
|
||||
DESCRIPTION.to_vec(),
|
||||
PUBLIC.to_vec(),
|
||||
stream("/Type /XObject /Subtype /Form /BBox [0 0 100 120]", b"/Span /Public BDC BT /F1 10 Tf 0 Tr 10 100 Td (Public) Tj ET EMC"),
|
||||
],
|
||||
);
|
||||
// The page and its unchanged Form both reference the same resource object. The
|
||||
// dictionary names the Form itself, but the Form's content never invokes that name.
|
||||
let shared = doc.add(Object::Dict(resources(&doc, 0)));
|
||||
for owner in [ObjRef::new(3, 0), ObjRef::new(9, 0)] {
|
||||
doc.update_dict(owner, |d| d.set(b"Resources".to_vec(), Object::Ref(shared))).unwrap();
|
||||
}
|
||||
if hidden {
|
||||
sanitize::remove_hidden(&mut doc, &[sanitize::Hidden::HiddenText]).unwrap();
|
||||
} else {
|
||||
redact_first(&mut doc).unwrap();
|
||||
}
|
||||
assert_no_secret(&doc, b"LEAK_");
|
||||
assert_no_secret(&doc, b"SECRET");
|
||||
let mut saved = reopen(&doc);
|
||||
assert_eq!(under(&mut saved, 0, &[[0.0, 90.0, 100.0, 110.0]]), 6);
|
||||
let page_resources = resources(&saved, 0);
|
||||
let xobjects = saved.resolve(page_resources.get(b"XObject").unwrap());
|
||||
let form = saved.resolve(xobjects.as_dict().unwrap().get(b"Fm").unwrap());
|
||||
let form_resources = saved.resolve(form.as_dict().unwrap().get(b"Resources").unwrap());
|
||||
let form_resources = form_resources.as_dict().unwrap();
|
||||
assert!(property(&saved, form_resources, b"P0").is_none());
|
||||
assert_description(&property(&saved, form_resources, b"Public").unwrap());
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn unused_property_aliases_are_retired_by_value_not_resource_name() {
|
||||
for direct in [false, true] {
|
||||
let value = if direct { std::str::from_utf8(DESCRIPTION).unwrap() } else { "7 0 R" };
|
||||
let mut doc = one_page(SECRET, &format!("/Properties << /P0 {value} >>"), vec![DESCRIPTION.to_vec()]);
|
||||
let mut second_resources = resources(&doc, 0);
|
||||
let original_properties = doc.resolve(second_resources.get(b"Properties").unwrap());
|
||||
let mut alias = Dict::new();
|
||||
alias.set(b"Alias".to_vec(), original_properties.as_dict().unwrap().get(b"P0").unwrap().clone());
|
||||
second_resources.set(b"Properties".to_vec(), Object::Dict(alias));
|
||||
add_second_page(&mut doc, b"q Q", Object::Dict(second_resources));
|
||||
redact_first(&mut doc).unwrap();
|
||||
assert_no_secret(&doc, b"LEAK_");
|
||||
let saved = reopen(&doc);
|
||||
assert!(property(&saved, &resources(&saved, 1), b"Alias").is_none(), "direct={direct}");
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn unrelated_unused_property_with_the_same_name_is_not_changed() {
|
||||
let mut doc = one_page(SECRET, "/Properties << /P0 7 0 R >>", vec![DESCRIPTION.to_vec(), PUBLIC.to_vec()]);
|
||||
let mut second_resources = resources(&doc, 0);
|
||||
let mut properties = Dict::new();
|
||||
properties.set(b"P0".to_vec(), Object::Ref(ObjRef::new(8, 0)));
|
||||
second_resources.set(b"Properties".to_vec(), Object::Dict(properties));
|
||||
let second = add_second_page(&mut doc, b"q Q", Object::Dict(second_resources.clone()));
|
||||
let before = doc.get(second);
|
||||
redact_first(&mut doc).unwrap();
|
||||
assert_eq!(doc.get(second), before, "an unrelated page was rewritten");
|
||||
assert_no_secret(&doc, b"LEAK_");
|
||||
let saved = reopen(&doc);
|
||||
assert_description(&property(&saved, &resources(&saved, 1), b"P0").unwrap());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn null_resources_forms_preserve_a_retired_property_still_used_by_nested_dp() {
|
||||
let mut doc = one_page(
|
||||
SECRET,
|
||||
"/Properties << /P0 7 0 R >> /XObject << /Outer 8 0 R /Inner 9 0 R >>",
|
||||
vec![
|
||||
PUBLIC.to_vec(),
|
||||
stream("/Type /XObject /Subtype /Form /BBox [0 0 100 120] /Resources null", b"/Inner Do"),
|
||||
stream("/Type /XObject /Subtype /Form /BBox [0 0 100 120]", b"/Span /P0 DP BT /F1 10 Tf 10 100 Td (Public) Tj ET"),
|
||||
],
|
||||
);
|
||||
let shared = doc.add(Object::Dict(resources(&doc, 0)));
|
||||
doc.update_dict(ObjRef::new(3, 0), |d| d.set(b"Resources".to_vec(), Object::Ref(shared))).unwrap();
|
||||
add_second_page(&mut doc, b"/Outer Do", Object::Ref(shared));
|
||||
redact_first(&mut doc).unwrap();
|
||||
assert_no_secret(&doc, b"SECRET");
|
||||
let mut saved = reopen(&doc);
|
||||
assert!(property(&saved, &resources(&saved, 0), b"P0").is_none());
|
||||
assert_description(&property(&saved, &resources(&saved, 1), b"P0").unwrap());
|
||||
assert_eq!(under(&mut saved, 1, &[[0.0, 90.0, 100.0, 110.0]]), 6);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn uninspectable_forms_with_candidate_properties_fail_closed() {
|
||||
for own in [false, true] {
|
||||
for malformed in [false, true] {
|
||||
let filter = if malformed { "" } else { "/Filter /UnsupportedPropertyTest" };
|
||||
let data = if malformed { b")".as_slice() } else { b"opaque" };
|
||||
if malformed {
|
||||
assert!(pdfcraft_content::parse(data).skipped > 0, "fixture must need parser recovery");
|
||||
}
|
||||
let mut doc = one_page(
|
||||
SECRET,
|
||||
"/Properties << /P0 7 0 R >> /XObject << /Fm 8 0 R >>",
|
||||
vec![DESCRIPTION.to_vec(), stream(&format!("/Type /XObject /Subtype /Form /BBox [0 0 100 120] {filter}"), data)],
|
||||
);
|
||||
let shared = doc.add(Object::Dict(resources(&doc, 0)));
|
||||
doc.update_dict(ObjRef::new(3, 0), |d| d.set(b"Resources".to_vec(), Object::Ref(shared))).unwrap();
|
||||
if own {
|
||||
doc.update_dict(ObjRef::new(8, 0), |d| d.set(b"Resources".to_vec(), Object::Ref(shared))).unwrap();
|
||||
}
|
||||
add_second_page(&mut doc, b"/Fm Do", Object::Ref(shared));
|
||||
assert_eq!(under(&mut doc, 0, &[[5.0, 195.0, 60.0, 212.0]]), 6);
|
||||
let error = redact_first(&mut doc).unwrap_err();
|
||||
assert!(matches!(error, RedactError::Cos(_)), "own Resources={own}, malformed={malformed}: {error:?}");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn hidden_text_in_nested_forms_retires_named_descriptions_and_keeps_public_text() {
|
||||
let mut doc = one_page(
|
||||
b"/Outer Do",
|
||||
"/Properties << /P0 7 0 R /Public 8 0 R >> /XObject << /Outer 9 0 R /Inner 10 0 R >>",
|
||||
vec![
|
||||
DESCRIPTION.to_vec(),
|
||||
PUBLIC.to_vec(),
|
||||
stream("/Type /XObject /Subtype /Form /BBox [0 0 300 300]", b"/Inner Do"),
|
||||
stream(
|
||||
"/Type /XObject /Subtype /Form /BBox [0 0 300 300]",
|
||||
b"/Span /P0 BDC BT /F1 10 Tf 3 Tr 10 200 Td (SECRET) Tj ET EMC \
|
||||
/Span /Public BDC BT /F1 10 Tf 0 Tr 10 100 Td (Public) Tj ET EMC",
|
||||
),
|
||||
],
|
||||
);
|
||||
sanitize::remove_hidden(&mut doc, &[sanitize::Hidden::HiddenText]).unwrap();
|
||||
assert_no_secret(&doc, b"SECRET");
|
||||
assert_no_secret(&doc, b"LEAK_");
|
||||
let mut saved = reopen(&doc);
|
||||
assert_eq!(under(&mut saved, 0, &[[0.0, 90.0, 100.0, 110.0]]), 6);
|
||||
// The rewritten inner Form owns the surviving /Public description after resources are
|
||||
// localized; no page-level property lookup is sufficient to check it.
|
||||
assert!(saved.scan_objects().any(|(_, object)| {
|
||||
object.as_dict().and_then(|d| d.get(b"ActualText")).and_then(Object::as_string).is_some_and(|s| s.to_text() == "PUBLIC_ACTUAL")
|
||||
}));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn removing_hidden_layer_content_cleans_its_surviving_outer_description() {
|
||||
let mut doc = one_page(
|
||||
b"/Span /P0 BDC /OC /Off BDC BT /F1 10 Tf 10 200 Td (SECRET) Tj ET EMC EMC \
|
||||
/Span /Public BDC BT /F1 10 Tf 10 100 Td (Public) Tj ET EMC",
|
||||
"/Properties << /P0 7 0 R /Public 8 0 R /Off 9 0 R >>",
|
||||
vec![DESCRIPTION.to_vec(), PUBLIC.to_vec(), b"<< /Type /OCG /Name (Hidden test layer) >>".to_vec()],
|
||||
);
|
||||
let mut defaults = Dict::new();
|
||||
defaults.set(b"OFF".to_vec(), Object::Array(vec![Object::Ref(ObjRef::new(9, 0))]));
|
||||
let mut optional_content = Dict::new();
|
||||
optional_content.set(b"OCGs".to_vec(), Object::Array(vec![Object::Ref(ObjRef::new(9, 0))]));
|
||||
optional_content.set(b"D".to_vec(), Object::Dict(defaults));
|
||||
doc.update_dict(ObjRef::new(1, 0), |d| d.set(b"OCProperties".to_vec(), Object::Dict(optional_content))).unwrap();
|
||||
sanitize::remove_hidden(&mut doc, &[sanitize::Hidden::HiddenLayers]).unwrap();
|
||||
assert_no_secret(&doc, b"SECRET");
|
||||
assert_no_secret(&doc, b"LEAK_");
|
||||
let mut saved = reopen(&doc);
|
||||
assert_description(&property(&saved, &resources(&saved, 0), b"Public").unwrap());
|
||||
assert_eq!(under(&mut saved, 0, &[[0.0, 90.0, 100.0, 110.0]]), 6);
|
||||
}
|
||||
+3
-1
@@ -1,5 +1,6 @@
|
||||
# PdfCraft roadmap: milestones and what's next
|
||||
|
||||
> **Last reviewed:** 2026-10-18 · **Last updated:** 2026-10-18 · **Change:** minor (M8, #1350: named replacement descriptions retired during Sanitize and across shared resources; estimates unchanged)
|
||||
> **Last reviewed:** 2026-10-15 · **Last updated:** 2026-10-15 · **Change:** minor (M6: AcroForm XML text; estimates unchanged)
|
||||
> **Last reviewed:** 2026-10-15 · **Last updated:** 2026-10-15 · **Change:** minor (M6: preserve XML/XFDF attribute whitespace; refreshed upstream; estimates unchanged) · **Target:** Adobe Acrobat Pro (Acrobat DC, continuous track 26.002.21931, macOS)
|
||||
> **Last reviewed:** 2026-10-15 · **Last updated:** 2026-10-15 · **Change:** minor (M1: raw-deflate fallback preserves decoding-limit errors; refreshed upstream; estimates unchanged) · **Target:** Adobe Acrobat Pro (Acrobat DC, continuous track 26.002.21931, macOS)
|
||||
@@ -128,7 +129,7 @@ total matches the parity estimate.
|
||||
| M6 | Forms + JavaScript | 160–320 | 50% | 40–75 | Done: filling all field types with regenerated appearances; Prepare a form (every field type, multi-select/shared properties, move/resize/delete, Field Properties General/Appearance/Position/Options/Format/Validate/Calculate); Acrobat's AF format/keystroke/validate/calculate functions and simplified field notation run natively in Acrobat's event order; tab order; push-button actions; flatten; agent tools. Form data exchange (FDF, XFDF, XML, CSV, text) done; XFDF values retain text across XML comments and processing instructions.<br>Done since: JavaScript engine (boa, sandboxed) running custom keystroke/validate/calculate/format and button scripts, document JavaScripts, console. Missing: Actions tab, wider object model (annotations, layers, dialogs), XFA<br>Done since: Existing simple field-font encodings and Differences are respected when generating appearances; an unencodable value uses a separate Helvetica appearance font without changing the source font. Synthetic reopen regressions cover punctuation, subset fallback and multiline text. Exact font metrics/layout and broader Unicode fallback remain incomplete. |
|
||||
| M6 | Forms + JavaScript | 160–320 | 50% | 40–75 | Done: filling all field types with regenerated appearances; Prepare a form (every field type, multi-select/shared properties, move/resize/delete, Field Properties General/Appearance/Position/Options/Format/Validate/Calculate); Acrobat's AF format/keystroke/validate/calculate functions and simplified field notation run natively in Acrobat's event order; tab order; push-button actions; flatten; agent tools. Form data exchange (FDF, XFDF, XML, CSV, text) done; XFDF values retain text across XML comments and processing instructions.<br>Done since: JavaScript engine (boa, sandboxed) running custom keystroke/validate/calculate/format and button scripts, document JavaScripts, console. Missing: Actions tab, wider object model (annotations, layers, dialogs), XFA<br>Done since: Tab and Shift-Tab wrapping to the same field or another widget of that field refresh the newly opened editor from the accepted value before Save. Synthetic UI regressions cover both directions, same-frame Save, validation refusal and undo/redo. |
|
||||
| M7 | Content editing (text, images, header/footer, watermark) | 250–500 | 20% | 100–180 | Done: header & footer, watermarks, backgrounds, Bates; added text and images that stay editable (move, resize, format, rotate, flip, crop, replace); links (Link tool, Link Properties, create from URLs, remove all). Missing: editing existing text and images in place (the longest pole), image/PDF watermarks |
|
||||
| M8 | Security + redaction | 100–180 | 68% | 15–30 | Done: opening protected documents, permissions, Protect Using Password, Remove security; redaction (mark text/areas/pages, Search & Redact with patterns, apply removing glyphs, image pixels, vectors, XObject content, annotations and fields, verification, full rewrite on save); Remove hidden information and Sanitize.<br>Done since: redaction codes (U.S. FOIA and Privacy Act sets as overlay text, in Redaction properties and `redact_mark`). Redaction cleans indirect and form structure references, preserving surviving form tags. Redaction retires inherited resource dictionaries, so a parent `/Pages` can't keep a removed XObject reachable; unredacted pages are left untouched. Image redaction removes alternate images and unsupported masks only on affected images. Missing: certificate security, custom redaction code sets and pattern locales, DCT re-encoding |
|
||||
| M8 | Security + redaction | 100–180 | 68% | 15–30 | Done: opening protected documents, permissions, Protect Using Password, Remove security; redaction (mark text/areas/pages, Search & Redact with patterns, apply removing glyphs, image pixels, vectors, XObject content, annotations and fields, verification, full rewrite on save); Remove hidden information and Sanitize.<br>Done since: redaction codes (U.S. FOIA and Privacy Act sets as overlay text, in Redaction properties and `redact_mark`). Redaction cleans indirect and form structure references, preserving surviving form tags. Redaction retires inherited resource dictionaries, so a parent `/Pages` can't keep a removed XObject reachable; unredacted pages are left untouched. Image redaction removes alternate images and unsupported masks only on affected images. Changed marked-content spans lose replacement descriptions; untouched spans keep theirs. Hidden-text cleanup retires unused named descriptions across inherited/shared page and form resources (64 focused tests and scoped Rust 1.96 Clippy/wasm checks pass; publication CI pending). Missing: certificate security, custom redaction code sets and pattern locales, DCT re-encoding |
|
||||
| M9 | Signatures (PAdES, validation) | 160–280 | 60% | 40–70 | Done: new `sign` crate (DER, X.509, CMS, PKCS #12 on RustCrypto; aws-lc-rs for RSA private keys); PAdES B-B signing (visible, invisible, existing fields, certification with DocMDP); validation with trust store and changes-after-signing classification; self-signed digital IDs; Signatures panel, message bar, sign dialogs; agent tools. Checked with pdfsig and OpenSSL.<br>Done since: macOS Keychain and Windows Current User Personal store signing (software-backed CNG keys), certificate viewer, signed documents protected from rewrites, page validation overlays separate from original signature artwork (baked Not Verified captions remain; #79 open), snapshot tabs that reuse instead of multiplying. Missing: timestamps (B-T), LTV (DSS, OCSP, CRL), FieldMDP, smart cards, PKCS #11 |
|
||||
| M10 | OCR, create, export, print | 200–350 | 33% | 120–215 | Done: create from blank/text/PNG/JPEG/TIFF (multi-page)/GIF/BMP; export PNG/JPEG/TIFF and text; Print (Acrobat's sizing, n-up, booklet, poster, comments & forms, preview, the CUPS and Windows spoolers, print-ready PDF).<br>Done since: embedded/72/custom DPI choices for image imports; export all images; OCR (searchable image for pages, ranges and multiple files); single-sided cut-and-stack imposition with cut marks, through Print and doc_print; architectural/ANSI/large ISO sheet presets, Match page at 100% and PDF-scale preview (#599 follow-up); Create PDF from multiple files (PDFs, images and text, edited as pages in the grid); OCR can also read pictures on pages that have text, without adding that text twice (#1028). Done since: Word invoice export uses borderless inferred tables, floating positioned text/artwork only where text overlaps an image or is rotated, and per-page sizes, covered by synthetic invoice automation regressions (#773); printing on the web through the browser's print dialog, sheets drawn by PdfCraft (gap 4). Missing: OCR languages beyond Latin, editable-text OCR output, full Office layout fidelity/import |
|
||||
| M11 | Optimize, preflight, PDF/A/X/UA, print production | 200–350 | 18% | 80–140 | Done: new `optimize` crate: Reduce File Size and the PDF Optimizer (images measured where drawn, bicubic downsampling, JPEG/ZIP recompression only when smaller, discard objects and user data, Flate clean-up, resource merging, object streams). Missing: fonts and transparency panels, space audit, preflight, PDF/A/X/UA |
|
||||
@@ -240,5 +241,6 @@ M2 (renderer), M7 (content editing), M10 (OCR and Office), M11 (Preflight and st
|
||||
| 2026-10-10 | minor | M3 records the System appearance completion after #881; estimates unchanged |
|
||||
| 2026-10-10 | minor | Re-judged convert and share with the cross-app gate rule (only a workflow that can't be completed at all blocks): partial, not blocking → stage alpha |
|
||||
| 2026-10-10 | minor | Added the alpha gate (six core workflows; convert and share fails → pre-alpha, 40–70 h to alpha) and rank 0 in the upcoming milestones |
|
||||
| 2026-10-12 | minor | M8: cleanup removes replacement descriptions only from changed marked-content spans, preserving surviving accessibility text. Estimates unchanged. |
|
||||
| 2026-10-12 | minor | M8: image redaction removes alternate images and unsupported masks only on affected images. Estimates unchanged. |
|
||||
| 2026-10-10 | major | Created from ROADMAP.md §Milestones, §Critical path, §Risks and §Where we're lacking. Merged the duplicated M1, M2, M3 and M14 rows, re-measured Done and Remaining, added Current focus and the ranked upcoming milestones |
|
||||
|
||||
Reference in New Issue
Block a user