fix(ecstore): retry decommission entries safely on source changes

A single object's SourceChanged during decommission cleanup no longer
cancels the shared worker token and fails the whole pool operation.
Cleanup preflight and source-cleanup outcomes now retry per entry with
bounded attempts and cancellation-aware backoff, applied uniformly to
ordinary versions, delete markers, and tiered copies (removing the
try-once-only branches); every retry re-lists the entry and redoes
version multiset validation before touching the source. Only quorum
loss, unrecoverable system errors, or exceeding a pool-level
SourceChanged exhaustion threshold still fails the decommission, and
exhausted entries never delete their source versions.

Retry attempts, backoff, and deferred/exhausted reasons are logged per
entry for observability. Heavy regression tests spawn on dedicated
32MiB stacks following the existing store-test pattern.

Fixes rustfs/backlog#1913
This commit is contained in:
overtrue
2026-08-23 06:46:28 +08:00
parent 20d1266496
commit 693800db8e
3 changed files with 1246 additions and 113 deletions
+9 -1
View File
@@ -4736,7 +4736,15 @@ impl SetDisks {
achieved: 0,
});
}
let parts_metadata = vec![fi.clone(); disks.len()];
// Rebuilt tiered metadata starts with index zero, but shuffling validates
// each source slot before assigning the shuffled index below.
let parts_metadata: Vec<FileInfo> = (0..disks.len())
.map(|disk_index| {
let mut part = fi.clone();
part.erasure.index = fi.erasure.distribution[disk_index];
part
})
.collect();
let (shuffle_disks, parts_metadata) = Self::shuffle_disks_and_parts_metadata(&disks, &parts_metadata, &fi);
let mut errs = Vec::with_capacity(shuffle_disks.len());