gsadmin/rustfs
1
0
Fork 0
mirror of https://github.com/rustfs/rustfs.git synced 2026-08-27 15:37:02 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix(object-capacity): drive scan progress checks by visited entries and remove unreachable stall detector (#4557)

Browse Source 
Progress/timeout checks only ran when file_count advanced, so trees of
directory-only or error-dense entries (dirs, traversal errors, symlinks
never increment file_count) bypassed the entire cooperative time budget
and held the blocking thread for unbounded time, while each error entry
logged its own warn — permission-dense trees flooded the log (S13).
The stall detector was structurally unreachable: it fired on 'no file
progress between checks', but checks themselves only ran when file
progress happened, so RUSTFS_CAPACITY_STALL_TIMEOUT never did anything
since its introduction (S09).

- Progress checks (timeout + early-sampling entry) are now driven by a
  visited-entry counter that also advances on directories and errors, so
  every tree shape reaches the budget checks.
- Remove the unreachable stall detector and its plumbing end to end
  (ProgressMonitor fields, ScanLimits, config getter, env const, stall
  metric). Genuine walker wedges are handled by the hard outer
  wall-clock budget from backlog#1017; no decorative protection is left.
- Cap per-entry error warns at 10 per scan with an explicit suppression
  notice; had_partial_errors still records the condition.

Ref: rustfs/backlog#1016 (S09+S13 from audit rustfs/backlog#1010)
This commit is contained in:
Zhengchao An
2026-07-09 02:47:17 +08:00
committed by GitHub
parent d232a46b4d
commit 0ebe00b383
5 changed files with 145 additions and 167 deletions
Download Patch File Download Diff File Expand all files Collapse all files
crates/config/src/constants/capacity.rs
-8
View File
@@ -57,9 +57,6 @@ pub const ENV_CAPACITY_MIN_TIMEOUT: &str = "RUSTFS_CAPACITY_MIN_TIMEOUT";
/// Environment variable for maximum capacity calculation timeout
pub const ENV_CAPACITY_MAX_TIMEOUT: &str = "RUSTFS_CAPACITY_MAX_TIMEOUT";
/// Environment variable for progress stall detection timeout
pub const ENV_CAPACITY_STALL_TIMEOUT: &str = "RUSTFS_CAPACITY_STALL_TIMEOUT";
// ============================================================================
// Default Values
// ============================================================================
@@ -116,10 +113,6 @@ pub const DEFAULT_CAPACITY_MIN_TIMEOUT_SECS: u64 = 2;
/// Default: 15 seconds
pub const DEFAULT_CAPACITY_MAX_TIMEOUT_SECS: u64 = 15;
/// Progress stall detection timeout in seconds
/// Default: 20 seconds
pub const DEFAULT_CAPACITY_STALL_TIMEOUT_SECS: u64 = 20;
// ============================================================================
// Tests
// ============================================================================
@@ -143,6 +136,5 @@ mod tests {
assert_eq!(ENV_CAPACITY_ENABLE_DYNAMIC_TIMEOUT, "RUSTFS_CAPACITY_ENABLE_DYNAMIC_TIMEOUT");
assert_eq!(ENV_CAPACITY_MIN_TIMEOUT, "RUSTFS_CAPACITY_MIN_TIMEOUT");
assert_eq!(ENV_CAPACITY_MAX_TIMEOUT, "RUSTFS_CAPACITY_MAX_TIMEOUT");
assert_eq!(ENV_CAPACITY_STALL_TIMEOUT, "RUSTFS_CAPACITY_STALL_TIMEOUT");
}
}