gsadmin/rustfs
1
0
Fork 0
mirror of https://github.com/rustfs/rustfs.git synced 2026-07-26 16:28:15 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix(heal): clean up .rustfs/tmp healed shards on all failure paths (backlog#799 B20) (#4319)

Browse Source 
heal_object wrote healed shards to .rustfs/tmp/<uuid>/ and only removed that tmp
dir on the success path (the final delete_all). Three early exits after the tmp
shards were written leaked the dir:

- `erasure.heal(...)?` failing midway,
- the `disks_to_heal_count == 0` early return, and
- the `?` on the post-rename remote data-dir delete.

Add the tmp cleanup before the first two, and downgrade the remote data-dir
cleanup failure to a warning (the healed shard is already renamed into place, so
that cleanup failing must not abort the heal or leak the tmp shards).

Refs backlog#799 (B20), tracked in rustfs/backlog#863.
This commit is contained in:
Zhengchao An
2026-07-06 22:16:58 +08:00
committed by GitHub
parent 849ea9a122
commit 1b3727e2c4
1 changed files with 32 additions and 13 deletions
Download Patch File Download Diff File Expand all files Collapse all files
crates/ecstore/src/set_disk/ops/heal.rs
+32 -13
View File
@@ -484,7 +484,12 @@ impl SetDisks {
// Heal each part. erasure.Heal() will write the healed
// part to .rustfs/tmp/uuid/ which needs to be renamed
// later to the final location.
erasure.heal(&mut writers, readers, part.size, &prefer).await?;
if let Err(e) = erasure.heal(&mut writers, readers, part.size, &prefer).await {
// Don't leak the partially-written healed shards in
// .rustfs/tmp when heal fails midway (backlog#799 B20).
let _ = self.delete_all(RUSTFS_META_TMP_BUCKET, &tmp_id).await;
return Err(e.into());
}
// close_bitrot_writers(&mut writers).await?;
for (index, disk_op) in out_dated_disks.iter_mut().enumerate() {
@@ -523,6 +528,8 @@ impl SetDisks {
}
if disks_to_heal_count == 0 {
// Clean up healed shards written to .rustfs/tmp before bailing (B20).
let _ = self.delete_all(RUSTFS_META_TMP_BUCKET, &tmp_id).await;
return Ok((
result,
Some(DiskError::other(format!(
@@ -570,18 +577,30 @@ impl SetDisks {
let d_path = Path::new(&encode_dir_object(object)).join(rm_data_dir);
disk.delete(
bucket,
d_path.to_str().expect("operation should succeed"),
DeleteOptions {
immediate: true,
recursive: true,
..Default::default()
},
)
.await?;
if let Err(e) = disk
.delete(
bucket,
d_path.to_str().expect("operation should succeed"),
DeleteOptions {
immediate: true,
recursive: true,
..Default::default()
},
)
.await
{
// The healed shard has already been renamed into place; a
// failure cleaning up the old remote data dir must not abort
// the heal and leak the tmp shards (backlog#799 B20).
warn!(
component = LOG_COMPONENT_ECSTORE,
subsystem = LOG_SUBSYSTEM_HEAL,
bucket,
object,
error = %e,
"Heal remote data-dir cleanup failed"
);
}
}
for (i, v) in result.before.drives.iter().enumerate() {