docs(object-data-cache): state the correctness boundary and timing channel (#4695)

docs(object-data-cache): state the correctness boundary and the timing channel

The crate doc still described itself as "a minimal skeleton for the initial
rollout phase", which stopped being true several releases ago, and neither
the crate nor the operator-facing env constant said anything about what the
cache does or does not guarantee.

Record two things a reader has to know.

The correctness boundary: a hit is sound because the key matched metadata the
caller just resolved from a read quorum, not because invalidation ran.
Process-local invalidation is hygiene that frees capacity; the key is what
keeps a stale body from being served. Anyone tempted to weaken the key should
meet this sentence first.

The timing side channel: a hit skips the erasure read, bitrot verify and
decode, so it is reliably faster than a miss. A principal authorized to read
an object can time a single GET and learn whether someone read that object
within the entry's lifetime. This crosses no authorization boundary — the
probe needs read access to that exact object, checked before the cache is
consulted — but it does disclose a co-tenant's recent access pattern. Say so
where operators look: on RUSTFS_OBJECT_DATA_CACHE_ENABLE. Timing noise would
cost precisely the latency the cache exists to save, so the mitigation is to
leave the cache off for buckets where access-pattern confidentiality matters.

Refs: backlog#1139

Co-authored-by: heihutu <heihutu@gmail.com>

crates/config/src/constants/runtime.rs

@@ -118,6 +118,25 @@ pub const ENV_OBJECT_SEEK_SUPPORT_THRESHOLD: &str = "RUSTFS_OBJECT_SEEK_SUPPORT_
 pub const DEFAULT_OBJECT_SEEK_SUPPORT_THRESHOLD: usize = 10 * 1024 * 1024;
 
 // Object data cache configuration
+
+/// Enables the in-memory object body cache, which serves whole-object GET
+/// responses without an erasure read.
+///
+/// # Timing side channel
+///
+/// A cache hit skips the erasure read, bitrot verification and decode, so it is
+/// reliably faster than a miss. Any principal authorized to read an object can
+/// therefore infer, by timing a single GET, whether *someone* read that object
+/// within the entry's lifetime (see `RUSTFS_OBJECT_DATA_CACHE_TTL_SECS` and
+/// `..._TIME_TO_IDLE_SECS`).
+///
+/// This never crosses an authorization boundary — the probe requires read
+/// access to the exact bucket and object, checked before the cache is consulted
+/// — but it does disclose co-tenants' recent access patterns on objects the
+/// observer may already read. Leave the cache disabled where access-pattern
+/// confidentiality matters, such as a bucket shared read-only between competing
+/// tenants. Timing noise is not a viable mitigation: it would cost exactly the
+/// latency the cache exists to save.
 pub const ENV_OBJECT_DATA_CACHE_ENABLE: &str = "RUSTFS_OBJECT_DATA_CACHE_ENABLE";
 pub const ENV_OBJECT_DATA_CACHE_MODE: &str = "RUSTFS_OBJECT_DATA_CACHE_MODE";
 pub const ENV_OBJECT_DATA_CACHE_MAX_BYTES: &str = "RUSTFS_OBJECT_DATA_CACHE_MAX_BYTES";

crates/object-data-cache/src/lib.rs

@@ -12,11 +12,34 @@
 // See the License for the specific language governing permissions and
 // limitations under the License.
 
-//! Engine-only object body cache contracts for RustFS.
+//! Engine-only object body cache for RustFS.
 //!
-//! This crate intentionally contains only a minimal skeleton for the initial
-//! rollout phase. App-layer semantics and storage-specific read paths stay in
-//! the `rustfs` crate.
+//! App-layer semantics and storage-specific read paths stay in the `rustfs`
+//! crate; this crate owns the key, the eviction backend and the metrics.
+//!
+//! # Correctness boundary
+//!
+//! An entry is keyed by object identity — bucket, object, version, etag, size
+//! and body variant — and every lookup runs after the caller has resolved fresh
+//! metadata from a read quorum. Serving a hit is therefore sound because the
+//! key matched metadata that was just read, not because the entry was recently
+//! invalidated. Invalidation is process-local and is **hygiene**: it frees
+//! capacity promptly, but the key is what keeps a stale body from being served.
+//!
+//! # Timing side channel
+//!
+//! A cache hit skips the erasure read, bitrot verification and decode, so it is
+//! reliably faster than a miss. Any principal authorized to read an object can
+//! therefore infer, by timing a single GET, whether *someone* read that object
+//! within the entry's lifetime (`ttl` / `time_to_idle`).
+//!
+//! This never crosses an authorization boundary: the probe requires read access
+//! to the exact bucket and object, which is checked before the cache is
+//! consulted. It does leak co-tenants' recent access patterns on objects the
+//! observer may already read. Deployments where access-pattern confidentiality
+//! matters — a bucket shared read-only between competing tenants — should leave
+//! the cache disabled for those buckets. Adding timing noise is not a viable
+//! mitigation: it would cost exactly the latency the cache exists to save.
 
 pub mod backend;
 pub mod cache;