gsadmin/rustfs
1
0
Fork 0
mirror of https://github.com/rustfs/rustfs.git synced 2026-08-24 05:06:28 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat(ecstore): reclaim orphaned data dirs during heal (#4356)

Browse Source 
* feat(ecstore): reclaim orphaned data dirs during heal (backlog #3231/#3191)

Pre-#3510, an unversioned overwrite leaked one UUID-named data dir per PUT.
The write path now cleans up going forward, but pre-existing strays stay on
disk forever: heal's dangling logic only removes whole objects whose data is
missing, never surplus data dirs of an otherwise-healthy object. That leaked
space is what eventually made ListObjects scan tens of GB and time out.

Add SetDisks::reclaim_orphan_data_dirs, a fail-closed, quorum-safe sweep that
mirrors purge_orphan_dir_object:

- referenced set is the UNION of get_data_dirs() across every online replica's
  xl.meta, so a dir named by any replica is kept;
- if a disk holds the object dir but its xl.meta is missing or unparseable the
  object is treated as degraded and nothing is removed;
- only UUID-named subdirectories are ever considered.

Wire it into heal_object's healthy tail (under the object write lock) as a
best-effort step so the background heal scanner and admin heal recover the
leaked space automatically; a reclaim failure never fails the heal.

Covered by four unit tests: removal of an unreferenced dir, no-op when all dirs
are referenced, fail-closed abort on missing metadata, and cross-replica union
preservation.

Co-Authored-By: heihutu <heihutu@gmail.com>

* docs(ecstore): fix typo unparseable -> unparsable

Satisfies the repository typos CI check.

Co-Authored-By: heihutu <heihutu@gmail.com>

---------

Co-authored-by: heihutu <heihutu@gmail.com>
This commit is contained in:
houseme
2026-07-07 13:06:22 +08:00
committed by GitHub
parent 0ce0388fc0
commit c9b976ad46
3 changed files with 317 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
crates/ecstore/src/set_disk/ops/heal.rs
+14
View File
@@ -624,6 +624,20 @@ impl SetDisks {
record_capacity_scope_if_needed(None, &out_dated_disks);
// The object is healthy here; sweep any data dirs left behind
// by pre-#3510 unversioned overwrites, which the dangling paths
// above never touch (issues #3231, #3191). Best effort — a
// failure must not fail the heal.
match self.reclaim_orphan_data_dirs(bucket, object).await {
Ok(removed) if removed > 0 => {
info!(bucket, object, removed, "heal_object: reclaimed orphaned data directories");
}
Ok(_) => {}
Err(e) => {
warn!(bucket, object, error = %e, "heal_object: orphan data-dir reclaim failed");
}
}
Ok((result, None))
}
Err(err) => Ok((result, Some(err))),