gsadmin/rustfs
1
0
Fork 0
mirror of https://github.com/rustfs/rustfs.git synced 2026-08-06 05:17:42 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix(cache): key the object body cache on data_dir for write-uniqueness (#4703)

Browse Source 
* refactor(object-data-cache): derive Default for ObjectDataCacheGetRequest

The GET request literal is hand-listed field-by-field across ~13 test sites in
two crates. Adding `mod_time_unix_nanos` in backlog#1111 had to touch every one
and still missed a literal, producing a compile error caught only in a later
CI lane. Derive `Default` and spread the engine-crate literals so the next
field addition is absorbed rather than fanned out.

Co-Authored-By: heihutu <heihutu@gmail.com>

* fix(cache): key the object body cache on data_dir for write-uniqueness

The cache key's correctness rested on being write-unique, but its only
write-scoped component was `mod_time` — a wall-clock timestamp that is not
monotonic and can be absent. Two writes that collide on MD5 (same etag+size)
and land on an equal mod_time (clock skew, same-tick, or absent) derived the
same key, so a node that never saw the overwrite could serve the previous body
for up to the TTL, and the same collision turned the fill-after-invalidation
race into a serving bug. This is the store's strong-read-after-write guarantee
leaning on a probabilistic argument.

Add `data_dir` — the xl.meta directory UUID ecstore regenerates on every body
write — as the primary write-unique anchor:
- surface `data_dir: Option<Uuid>` on ObjectInfo, copied from FileInfo in the
  single GET-path constructor `from_file_info`;
- carry it into ObjectDataCacheKey as `data_dir_u128` (held as u128 to keep the
  engine crate free of a uuid dependency), derived in the one planner site both
  the ecstore hook and the usecase layer share, so both produce an identical
  key by construction;
- keep `mod_time` as a second anchor and `etag+size` as belt-and-braces; an
  absent data_dir falls back to the prior behavior — strict improvement, no
  regression.

Two writes distinct only by data_dir now derive different keys even under an
MD5 collision with identical mod_time — the case mod_time alone cannot cover.

Blast radius is compiler-guarded: ObjectInfo derives Default and every real
construction site uses `..Default::default()`, so only from_file_info and one
full-literal test needed the field. The three P0 body_cache_hook_e2e
regressions, engine (80), and app (36) suites pass unchanged; a mutation that
severs the planner wiring fails planner_key_changes_with_data_dir.

Refs: backlog#1111, backlog#1118

Co-Authored-By: heihutu <heihutu@gmail.com>

* fix(cache): thread data_dir field through the merged mutation-hook test

Merging main (which landed the object-mutation-hook work, backlog#1131) brought
in a GetRequest test literal that predates the data_dir field. Spread it via
`..Default::default()` — the derive(Default) added here means this is the last
such hand-listed literal to need touching.

Co-Authored-By: heihutu <heihutu@gmail.com>

---------

Co-authored-by: heihutu <heihutu@gmail.com>
This commit is contained in:
houseme
2026-07-11 08:00:52 +08:00
committed by GitHub
parent a35ebb92b8
commit 7437f99c45
10 changed files with 161 additions and 40 deletions
Download Patch File Download Diff File Expand all files Collapse all files
crates/object-data-cache/src/cache.rs
+15 -13
View File
@@ -144,12 +144,13 @@ impl ObjectDataCache {
record_plan_decision(self.backend.as_metric_label(), self.config.mode, "cacheable", "eligible", request.size);
ObjectDataCacheGetPlan::Cacheable {
key: ObjectDataCacheKey::with_mod_time(
key: ObjectDataCacheKey::with_write_anchors(
request.bucket,
request.object,
request.version_id.as_deref(),
request.etag,
request.size,
request.data_dir_u128,
request.mod_time_unix_nanos,
request.body_variant,
),
@@ -393,7 +394,11 @@ const fn invalidation_outcome(result: &ObjectDataCacheInvalidationResult) -> &'s
}
/// Protocol-neutral GET request metadata for cache planning.
#[derive(Debug, Clone)]
///
/// Derives `Default` so tests construct it with `..Default::default()`; a new
/// field then does not have to be added to every literal (which is how the
/// `mod_time` seam bug arose during backlog#1111).
#[derive(Debug, Clone, Default)]
pub struct ObjectDataCacheGetRequest<'a> {
/// Bucket name.
pub bucket: &'a str,
@@ -405,9 +410,11 @@ pub struct ObjectDataCacheGetRequest<'a> {
pub etag: &'a str,
/// Object size in bytes.
pub size: u64,
/// Resolved version's `data_dir` UUID as a `u128`, or `None`. Primary
/// write-unique key component (backlog#1111 / ODC-06).
pub data_dir_u128: Option<u128>,
/// Resolved version's modification time as Unix nanoseconds, or `0` when
/// absent. Carried into the key so it is write-unique (backlog#1111 /
/// ODC-06).
/// absent. Second write-unique anchor.
pub mod_time_unix_nanos: i128,
/// Supported response body variant.
pub body_variant: ObjectDataCacheBodyVariant,
@@ -546,7 +553,7 @@ mod tests {
ObjectDataCacheInvalidationReason, ObjectDataCacheInvalidationResult, ObjectDataCacheLookup,
};
use crate::config::{ObjectDataCacheConfig, ObjectDataCacheMode};
use crate::key::{ObjectDataCacheBodyVariant, ObjectDataCacheIdentity};
use crate::key::ObjectDataCacheIdentity;
use bytes::Bytes;
use metrics_util::MetricKind;
use metrics_util::debugging::{DebugValue, DebuggingRecorder};
@@ -576,11 +583,9 @@ mod tests {
ObjectDataCacheGetRequest {
bucket,
object,
version_id: None,
etag,
size,
mod_time_unix_nanos: 0,
body_variant: ObjectDataCacheBodyVariant::FullObjectPlainV1,
..Default::default()
}
}
@@ -846,11 +851,9 @@ mod tests {
let plan = cache.plan_get(ObjectDataCacheGetRequest {
bucket: "bucket",
object: "object",
version_id: None,
etag: "etag",
size: 5,
mod_time_unix_nanos: 0,
body_variant: ObjectDataCacheBodyVariant::FullObjectPlainV1,
..Default::default()
});
let fill = cache.fill_body(&plan, Bytes::from_static(b"oops")).await;
@@ -900,11 +903,10 @@ mod tests {
let request = ObjectDataCacheGetRequest {
bucket: "bucket",
object: "object",
version_id: None,
etag: "etag",
size: 5,
mod_time_unix_nanos: 42,
body_variant: ObjectDataCacheBodyVariant::FullObjectPlainV1,
..Default::default()
};
let ObjectDataCacheGetPlan::Cacheable { key } = cache.plan_get(request) else {
panic!("plan should be cacheable");