mirror of
https://github.com/rustfs/rustfs.git
synced 2026-08-13 16:46:55 +00:00
fix(object-data-cache): make memory container-aware and bound cache config (#4655)
* fix(object-data-cache): bound cache config and make memory container-aware Harden the object data cache configuration path so a single bad input degrades to the disabled adapter instead of OOM-killing pods, panicking at boot, or silently mis-sizing the cache (backlog#1110/1113/1114/1115/ 1127/1140/1130). - ODC-05: resolve capacity and the fill gate from the effective (container-aware) memory. Prefer sysinfo cgroup_limits() and use min(host, cgroup) as the total plus cgroup-derived availability, falling back to host values when no cgroup limit constrains. Log the resolved capacity and its basis once at startup. - ODC-08: cap ttl/time_to_idle at 30 days in validate() so an unbounded Duration can no longer trip moka's ~1000-year builder assertion. - ODC-09: drop the max_entry_bytes floor from the derived-capacity clamp so MAX_ENTRY_BYTES can no longer inflate total capacity above the safety clamp; reject an entry larger than the resolved capacity instead. - ODC-10: require an explicit max_bytes to clear max_entry_bytes plus the weigher overhead so a fillable-but-unretainable cache is rejected. - ODC-22: reject max_entry_bytes at/above the u32 weigher boundary. - ODC-35: warn (not reject) when time_to_idle exceeds ttl and document the min(ttl, time_to_idle) expiry interaction. - ODC-25: give numeric env overrides two-valued semantics via a new rustfs_utils::get_env_parse_outcome; a malformed value now disables the whole cache with one aggregated warning instead of silently keeping defaults. Co-Authored-By: heihutu <heihutu@gmail.com> * fix(object-data-cache): require identity_keys_max of at least 2 The identity index admits a new key by evicting the oldest one, so a budget of 1 evicts the previous key on every fill and can never hold two live keys of one object at once — a versioned bucket alternating two versions then hits a permanent 0% hit rate. Reject the degenerate value at config validation time, where the adapter turns it into a disabled cache with a warning, since the bounded-eviction policy cannot rescue it at runtime. Handed off from the identity-index batch (backlog#1128), which changed the overflow policy but could not touch validate(). Refs: backlog#1115, backlog#1128 Co-Authored-By: heihutu <heihutu@gmail.com> * fix(object-data-cache): let a zero free-memory floor opt out of the gate Making the memory gate container-aware turned it live in CI: the runners are Kubernetes pods, so the gate now reads the pod's cgroup free memory, finds it below the 20% floor, and refuses the fill. Tests that assert a fill succeeds then failed on CI while passing on a developer host, which has no cgroup and falls back to host memory. The gate is behaving correctly — the tests were the ones depending on a live memory reading. Treat min_free_memory_percent = 0 as a deliberate opt-out rather than an invalid value: allows_fill returns early before it touches any snapshot, so admission becomes independent of where the suite runs. Operators gain the same escape hatch. Every test that requires a fill to succeed now sets the floor to 0. The tests that exercise the gate keep it enabled via memory_gated_config, so the ODC-05 coverage they provide is preserved rather than short-circuited. Refs: backlog#1110 Co-Authored-By: heihutu <heihutu@gmail.com> * fix(object-data-cache): opt the usecase fill tests out of the memory gate The previous commit exempted the fill-dependent tests it could find, but missed the six adapters built inside object_usecase.rs. Fixing the first batch moved nextest's fail-fast point forward and CI surfaced them: build_get_object_body_with_cache_materializes_once_and_hits_later and ..._uses_cached_body_without_reader_preread both assert a fill lands, and both ran with the default 20% free-memory floor. Set the floor to 0 on all six, matching the sibling test modules. Verified by pinning the gate's snapshot to 0% available — harsher than any CI pod — and confirming these tests still pass, which shows the exemption path never reads memory at all. An exhaustive sweep over every fill-enabled ObjectDataCacheConfig in the tree now shows no remaining site: the only unexempted ones are fill_enabled()'s matches! arm and two adapter tests that assert the adapter is disabled and therefore never fill. Refs: backlog#1110 Co-Authored-By: heihutu <heihutu@gmail.com> --------- Co-authored-by: heihutu <heihutu@gmail.com>
This commit is contained in:
@@ -533,6 +533,49 @@ pub fn get_env_opt_u64(key: &str) -> Option<u64> {
|
||||
parse_env_value(key)
|
||||
}
|
||||
|
||||
/// Outcome of reading and parsing an environment variable.
|
||||
///
|
||||
/// Unlike the `get_env_opt_*` helpers, this distinguishes an absent variable
|
||||
/// from one that is present but fails to parse, so callers can treat a
|
||||
/// malformed value as a hard configuration error instead of silently falling
|
||||
/// back to a default.
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub enum EnvParseOutcome<T> {
|
||||
/// The variable (and any alias) is not set.
|
||||
Absent,
|
||||
/// The variable is set but its value failed to parse into `T`.
|
||||
Invalid,
|
||||
/// The variable is set and parsed successfully.
|
||||
Parsed(T),
|
||||
}
|
||||
|
||||
/// Read an environment variable and report whether it is absent, present but
|
||||
/// malformed, or parsed successfully.
|
||||
///
|
||||
/// #Parameters
|
||||
/// - `key`: The environment variable key to look up (deprecated aliases apply).
|
||||
///
|
||||
/// #Returns
|
||||
/// - `EnvParseOutcome<T>`: `Absent`, `Invalid`, or `Parsed(value)`.
|
||||
pub fn get_env_parse_outcome<T>(key: &str) -> EnvParseOutcome<T>
|
||||
where
|
||||
T: std::str::FromStr,
|
||||
{
|
||||
let Some((used_key, value)) = resolve_env_with_aliases(key, &[]) else {
|
||||
return EnvParseOutcome::Absent;
|
||||
};
|
||||
|
||||
match value.parse::<T>() {
|
||||
Ok(parsed) => EnvParseOutcome::Parsed(parsed),
|
||||
Err(_) => {
|
||||
log_once(&format!("env_invalid_value:{used_key}"), || {
|
||||
format!("Invalid {} value for {used_key}: {value}. Treating as unset.", type_name::<T>())
|
||||
});
|
||||
EnvParseOutcome::Invalid
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Retrieve an environment variable as a specific type, with a default value if not set or parsing fails.
|
||||
///
|
||||
/// #Parameters
|
||||
@@ -687,8 +730,8 @@ pub fn apply_external_env_compat() -> ExternalEnvCompatReport {
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::{
|
||||
apply_external_env_compat, build_external_env_compat_report_from_entries, get_env_bool_with_aliases, get_env_f64,
|
||||
get_env_i32_with_aliases, get_env_opt_f64, get_env_opt_u64, get_env_str, get_env_u64,
|
||||
EnvParseOutcome, apply_external_env_compat, build_external_env_compat_report_from_entries, get_env_bool_with_aliases,
|
||||
get_env_f64, get_env_i32_with_aliases, get_env_opt_f64, get_env_opt_u64, get_env_parse_outcome, get_env_str, get_env_u64,
|
||||
};
|
||||
|
||||
fn source_key(suffix: &str) -> String {
|
||||
@@ -828,6 +871,19 @@ mod tests {
|
||||
});
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn env_parse_outcome_distinguishes_absent_invalid_and_parsed() {
|
||||
temp_env::with_var_unset("RUSTFS_TEST_OUTCOME", || {
|
||||
assert_eq!(get_env_parse_outcome::<u64>("RUSTFS_TEST_OUTCOME"), EnvParseOutcome::Absent);
|
||||
});
|
||||
temp_env::with_var("RUSTFS_TEST_OUTCOME", Some("not-a-u64"), || {
|
||||
assert_eq!(get_env_parse_outcome::<u64>("RUSTFS_TEST_OUTCOME"), EnvParseOutcome::Invalid);
|
||||
});
|
||||
temp_env::with_var("RUSTFS_TEST_OUTCOME", Some("42"), || {
|
||||
assert_eq!(get_env_parse_outcome::<u64>("RUSTFS_TEST_OUTCOME"), EnvParseOutcome::Parsed(42));
|
||||
});
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn apply_external_env_compat_copies_missing_rustfs_keys() {
|
||||
temp_env::with_var("MINIO_ROOT_USER", Some("compat-admin"), || {
|
||||
|
||||
Reference in New Issue
Block a user