Files
rustfs/crates/heal
Zhengchao An c3f54f95ed fix(heal): preserve resume state and retry when heal objects fail (backlog#855) (#4308)
fix(heal): don't mark set healed / discard resume state when objects failed (backlog#855)

`execute_heal_with_resume` unconditionally called `mark_completed()` and
returned `Ok(())` after the bucket loop, even when objects failed. The caller
then treats `Ok` as success and cleans up the resume + checkpoint state, so a
heal run with per-object failures was reported as a clean completion and its
state (including the failed set) was destroyed with no retry.

Finalize based on the failure count instead:

- failed_objects == 0 -> mark completed (unchanged);
- failed_objects > 0 with retry budget left -> `schedule_retry()` (bump the
  bounded retry counter + reset per-pass progress for a full re-scan) and
  return Err, so `heal_erasure_set` preserves the resume/checkpoint state and
  the caller keeps the healing markers for the next run;
- failed_objects > 0 with retries exhausted -> drop the resume state (no
  zombie task) but still return Err so the markers survive for a later heal
  cycle / the background scanner. Never silently claim a clean completion.

The failure identities are not persisted across pages (the per-page sets are
pruned in `complete_page`), so a retry is a bounded full re-scan; healed
objects are skipped quickly on the normal-scan pass.

Adds `ResumeState::reset_for_retry`, `ResumeManager::schedule_retry`,
`ResumeCheckpoint::reset_for_retry`, `CheckpointManager::reset_for_retry`, and
regression tests. This activates the failure path the caller already documents
at task.rs ("Keep the markers on failure ... the next run re-marks and
eventually clears them"), which was previously dead because heal never
returned Err on object failures.

Refs backlog#799 (B6).
2026-07-06 20:47:29 +08:00
..