feat(cache): add object data cache engine and app flow (#4187)

* feat(cache): add object data cache engine

* feat(cache): wire app-layer object cache flow

* refactor(cache): streamline app-layer cache flow

* refactor(cache): tighten cache flow internals

* refactor: address final clippy cleanup

* chore(deps): update quick-xml to 0.41.0

* feat(cache): wire object data cache env config

* fix(cache): gate materialize fill by cache plan

* chore(cache): add object data cache benchmark gate

* fix(cache): guard object cache fill size mismatches

* refactor(cache): streamline object cache body planning

* fix(cache): align object cache rollout config

* test(cache): cover buffered object cache benchmark

* test(cache): isolate object cache benchmark metrics

* test(cache): mark materialize rollout experimental

* test(cache): tighten object cache benchmark gate

* fix(cache): address review findings for object data cache

- singleflight: clean up leader entry on cancellation (Drop impl) so a dropped GET future can no longer wedge all subsequent fills for the same key; switch the fill map to a std Mutex and add a regression test

- adapter: honor RUSTFS_OBJECT_DATA_CACHE_ENABLE=true by defaulting to hit_only when no explicit mode is set (explicit mode still wins)

- planner: treat nil version UUIDs as "no value" per repo convention so unversioned objects key under the canonical "null" instead of fragmenting the key space

- multipart: invalidate the object cache on the quota-exceeded rollback delete after complete-multipart, closing a stale-cache window

- layering: move the disabled-cache fallback into app::context and drop the new infra->app layer-dependency baseline entry

* fix(cache): close invalidation races and drop full-cache scan on writes

- index: make identity-index insert/remove/prune atomic via starshard compute_if_present/compute_if_absent so concurrent fills can no longer drop each other's keys (lost keys made entries unreachable to invalidation until TTL); add a concurrency regression test

- fill: register the key in the identity index before the entry becomes visible in the cache and re-check the index afterwards, undoing the fill when an invalidation raced in between (new skipped_invalidation_race fill result)

- invalidate: with the index now authoritative, remove the full-cache iter() fallback that made every PUT/DELETE of a never-cached object O(total cache entries) (two scans per PUT, 2N per batch delete)

- materialize-fill: fail the GET instead of falling back to the partially consumed stream after a mid-read error (the fallback would send a body missing its prefix under a full-length Content-Length), and log the same size-mismatch warning as the sibling buffering paths

Co-Authored-By: heihutu <heihutu@gmail.com>

* test(storage): fix media-dependent buffer clamp expectation

test_concurrency_manager_multi_factor_strategy_buffer_clamp asserted media_cap.min(MI_B), but the implementation's final safety clamp is [32KiB, media_cap.max(MI_B)] — deliberately so a media cap above 1MiB (NVMe's 2MiB default) stays effective. The test only passed on machines detected as SSD/Unknown (cap == 1MiB) and failed on NVMe-backed CI runners with 2MiB != 1MiB. Assert the media cap itself, which is what the strategy actually guarantees on every environment.

Co-Authored-By: heihutu <heihutu@gmail.com>

* test(storage): format buffer clamp assertion

* chore(logging): update tier guardrail path

---------

Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: cxymds <cxymds@gmail.com>
Co-authored-by: overtrue <anzhengchao@gmail.com>
Co-authored-by: heihutu <heihutu@gmail.com>
This commit is contained in:
houseme
2026-07-03 18:11:14 +08:00
committed by GitHub
parent 25d80d7c60
commit eebd16d8a4
37 changed files with 5701 additions and 14 deletions
+40
View File
@@ -0,0 +1,40 @@
// Copyright 2024 RustFS Team
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
use crate::moka_backend::MokaBackend;
use crate::noop::NoopBackend;
/// Backend dispatch for the object data cache.
#[derive(Debug)]
pub enum ObjectDataCacheBackendKind {
/// No-op backend used while the feature is disabled.
Noop(NoopBackend),
/// Moka backend used when cache lookups are enabled.
Moka(Box<MokaBackend>),
}
impl Default for ObjectDataCacheBackendKind {
fn default() -> Self {
Self::Noop(NoopBackend)
}
}
impl ObjectDataCacheBackendKind {
pub(crate) const fn as_metric_label(&self) -> &'static str {
match self {
Self::Noop(_) => "noop",
Self::Moka(_) => "moka",
}
}
}
+374
View File
@@ -0,0 +1,374 @@
// Copyright 2024 RustFS Team
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
use crate::backend::ObjectDataCacheBackendKind;
use crate::config::ObjectDataCacheConfig;
use crate::error::ObjectDataCacheConfigError;
use crate::key::{ObjectDataCacheBodyVariant, ObjectDataCacheIdentity, ObjectDataCacheKey};
use crate::metrics::{
describe_metrics_once, publish_cache_state, record_fill_result, record_hit_bytes, record_invalidation,
record_request_decision,
};
use crate::moka_backend::MokaBackend;
use crate::noop::NoopBackend;
use crate::stats::{ObjectDataCacheStats, ObjectDataCacheStatsSnapshot};
use bytes::Bytes;
use std::sync::Arc;
use std::time::Instant;
/// Protocol-neutral cache facade for object body reuse.
#[derive(Debug)]
pub struct ObjectDataCache {
backend: ObjectDataCacheBackendKind,
config: Arc<ObjectDataCacheConfig>,
stats: Arc<ObjectDataCacheStats>,
}
impl ObjectDataCache {
/// Creates a disabled cache facade without requiring configuration parsing.
pub fn disabled() -> Self {
let config = Arc::new(ObjectDataCacheConfig::default());
let stats = Arc::new(ObjectDataCacheStats::default());
Self {
backend: ObjectDataCacheBackendKind::Noop(NoopBackend),
config,
stats,
}
}
/// Creates a new cache facade.
pub fn new(config: ObjectDataCacheConfig) -> Result<Self, ObjectDataCacheConfigError> {
describe_metrics_once();
config.validate()?;
let stats = Arc::new(ObjectDataCacheStats::default());
let backend = if config.is_disabled() {
ObjectDataCacheBackendKind::Noop(NoopBackend)
} else {
ObjectDataCacheBackendKind::Moka(Box::new(MokaBackend::new(&config, Arc::clone(&stats))?))
};
Ok(Self {
backend,
config: Arc::new(config),
stats,
})
}
/// Produces a lightweight GET plan from request metadata.
pub fn plan_get(&self, request: ObjectDataCacheGetRequest<'_>) -> ObjectDataCacheGetPlan {
if self.config.is_disabled() {
record_request_decision(
self.backend.as_metric_label(),
self.config.mode,
"disabled",
"mode_disabled",
request.size,
);
return ObjectDataCacheGetPlan::Disabled;
}
if request.size > self.config.max_entry_bytes {
record_request_decision(self.backend.as_metric_label(), self.config.mode, "skip", "too_large", request.size);
return ObjectDataCacheGetPlan::SkipTooLarge;
}
record_request_decision(self.backend.as_metric_label(), self.config.mode, "cacheable", "eligible", request.size);
ObjectDataCacheGetPlan::Cacheable {
key: ObjectDataCacheKey::new(
request.bucket,
request.object,
request.version_id.as_deref(),
request.etag,
request.size,
request.body_variant,
),
}
}
/// Looks up an object body from the configured backend.
pub async fn lookup_body(&self, plan: &ObjectDataCacheGetPlan) -> ObjectDataCacheLookup {
let lookup = match &self.backend {
ObjectDataCacheBackendKind::Noop(backend) => backend.lookup_body(plan).await,
ObjectDataCacheBackendKind::Moka(backend) => backend.lookup_body(plan).await,
};
self.stats.record_lookup(matches!(lookup, ObjectDataCacheLookup::Hit(_)));
self.refresh_entry_count();
match &lookup {
ObjectDataCacheLookup::Hit(bytes) => {
let size_bytes = u64::try_from(bytes.len()).unwrap_or(u64::MAX);
record_request_decision(self.backend.as_metric_label(), self.config.mode, "hit", "cache_hit", size_bytes);
record_hit_bytes(self.backend.as_metric_label(), self.config.mode, size_bytes);
}
ObjectDataCacheLookup::Miss => {
let size_bytes = match plan {
ObjectDataCacheGetPlan::Cacheable { key } => key.size,
_ => 0,
};
record_request_decision(self.backend.as_metric_label(), self.config.mode, "miss", "cache_miss", size_bytes);
}
ObjectDataCacheLookup::SkipDisabled => {
record_request_decision(self.backend.as_metric_label(), self.config.mode, "skip", "lookup_disabled", 0);
}
ObjectDataCacheLookup::SkipNotCacheable => {
record_request_decision(self.backend.as_metric_label(), self.config.mode, "skip", "lookup_not_cacheable", 0);
}
}
lookup
}
/// Attempts to fill the cache body for the current plan.
pub async fn fill_body(&self, plan: &ObjectDataCacheGetPlan, bytes: Bytes) -> ObjectDataCacheFillResult {
let fill_bytes = u64::try_from(bytes.len()).unwrap_or(u64::MAX);
if !self.config.fill_enabled() {
record_fill_result(self.backend.as_metric_label(), self.config.mode, "skipped_by_mode", fill_bytes, 0.0);
return ObjectDataCacheFillResult::SkippedByMode;
}
if let ObjectDataCacheGetPlan::Cacheable { key } = plan
&& fill_bytes != key.size
{
let result = ObjectDataCacheFillResult::SkippedSizeMismatch;
record_fill_result(
self.backend.as_metric_label(),
self.config.mode,
result.as_metric_label(),
fill_bytes,
0.0,
);
return result;
}
let fill_start = Instant::now();
let result = match &self.backend {
ObjectDataCacheBackendKind::Noop(backend) => backend.fill_body(plan).await,
ObjectDataCacheBackendKind::Moka(backend) => backend.fill_body(plan, bytes).await,
};
if matches!(result, ObjectDataCacheFillResult::Inserted) {
self.stats.record_fill();
}
self.refresh_entry_count();
record_fill_result(
self.backend.as_metric_label(),
self.config.mode,
result.as_metric_label(),
fill_bytes,
fill_start.elapsed().as_secs_f64(),
);
result
}
/// Invalidates all cache entries associated with the object identity.
pub async fn invalidate_object(
&self,
_identity: ObjectDataCacheIdentity,
_reason: ObjectDataCacheInvalidationReason,
) -> ObjectDataCacheInvalidationResult {
let result = match &self.backend {
ObjectDataCacheBackendKind::Noop(backend) => backend.invalidate_object().await,
ObjectDataCacheBackendKind::Moka(backend) => backend.invalidate_object(&_identity).await,
};
self.stats.record_invalidation();
self.refresh_entry_count();
record_invalidation(self.backend.as_metric_label(), _reason.as_metric_label());
result
}
/// Returns the current stats snapshot.
pub fn stats(&self) -> ObjectDataCacheStatsSnapshot {
self.stats.snapshot()
}
/// Returns true when the cache facade is fully disabled.
pub fn is_disabled(&self) -> bool {
self.config.is_disabled()
}
/// Returns true when the cache mode allows materialize fill.
pub fn materialize_fill_enabled(&self) -> bool {
matches!(self.config.mode, crate::config::ObjectDataCacheMode::FillMaterializeEnabled)
}
fn refresh_entry_count(&self) {
let (entries, weighted_bytes) = match &self.backend {
ObjectDataCacheBackendKind::Noop(_) => (0, 0),
ObjectDataCacheBackendKind::Moka(backend) => (backend.entry_count(), backend.weighted_size()),
};
self.stats.set_entries(entries);
publish_cache_state(self.backend.as_metric_label(), entries, weighted_bytes);
}
}
/// Protocol-neutral GET request metadata for cache planning.
#[derive(Debug, Clone)]
pub struct ObjectDataCacheGetRequest<'a> {
/// Bucket name.
pub bucket: &'a str,
/// Object key.
pub object: &'a str,
/// Optional version id.
pub version_id: Option<String>,
/// Object ETag.
pub etag: &'a str,
/// Object size in bytes.
pub size: u64,
/// Supported response body variant.
pub body_variant: ObjectDataCacheBodyVariant,
}
/// Planning result for a cache-aware GET.
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum ObjectDataCacheGetPlan {
/// Cache is globally disabled.
Disabled,
/// Object body exceeds the configured cacheable entry size.
SkipTooLarge,
/// Request is eligible for cache lookup and fill.
Cacheable {
/// Stable cache key for the request.
key: ObjectDataCacheKey,
},
}
/// Result of a cache lookup attempt.
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum ObjectDataCacheLookup {
/// Cache is disabled.
SkipDisabled,
/// Request was not cacheable under the current plan.
SkipNotCacheable,
/// Cache did not contain a matching object body.
Miss,
/// Cache returned a reusable object body.
Hit(Bytes),
}
/// Result of a cache fill attempt.
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum ObjectDataCacheFillResult {
/// Cache is disabled.
SkippedDisabled,
/// Cache mode does not currently allow fill.
SkippedByMode,
/// Request was not cacheable under the current plan.
SkippedNotCacheable,
/// Fill was skipped because the local memory gate rejected it.
SkippedMemoryPressure,
/// Fill was skipped because the per-identity key budget overflowed and was conservatively cleared.
SkippedIdentityOverflow,
/// Fill was skipped because the provided body length did not match the cache key identity.
SkippedSizeMismatch,
/// Fill waiters were released without a published leader result.
SkippedSingleflightClosed,
/// Fill was undone because an invalidation raced with the insert.
SkippedInvalidationRace,
/// The cache entry was inserted successfully.
Inserted,
}
impl ObjectDataCacheFillResult {
pub(crate) const fn as_metric_label(&self) -> &'static str {
match self {
Self::SkippedDisabled => "skipped_disabled",
Self::SkippedByMode => "skipped_by_mode",
Self::SkippedNotCacheable => "skipped_not_cacheable",
Self::SkippedMemoryPressure => "skipped_memory_pressure",
Self::SkippedIdentityOverflow => "skipped_identity_overflow",
Self::SkippedSizeMismatch => "skipped_size_mismatch",
Self::SkippedSingleflightClosed => "skipped_singleflight_closed",
Self::SkippedInvalidationRace => "skipped_invalidation_race",
Self::Inserted => "inserted",
}
}
}
/// Invalidation reason placeholder for the skeleton.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum ObjectDataCacheInvalidationReason {
/// Conservative invalidation before a mutating write or delete begins.
BeforeMutation,
/// Invalidation after a successful PutObject write.
AfterPutSuccess,
/// Invalidation after a successful delete.
AfterDeleteSuccess,
/// Invalidation after a successful copy destination write.
AfterCopySuccess,
/// Invalidation after a successful complete multipart upload.
AfterCompleteMultipartSuccess,
/// Manual invalidation requested by the caller.
Manual,
}
impl ObjectDataCacheInvalidationReason {
pub(crate) const fn as_metric_label(self) -> &'static str {
match self {
Self::BeforeMutation => "before_mutation",
Self::AfterPutSuccess => "after_put_success",
Self::AfterDeleteSuccess => "after_delete_success",
Self::AfterCopySuccess => "after_copy_success",
Self::AfterCompleteMultipartSuccess => "after_complete_multipart_success",
Self::Manual => "manual",
}
}
}
/// Result of an invalidation request.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum ObjectDataCacheInvalidationResult {
/// Invalidation completed successfully.
Success,
}
#[cfg(test)]
mod tests {
use super::{ObjectDataCache, ObjectDataCacheFillResult, ObjectDataCacheGetRequest, ObjectDataCacheLookup};
use crate::config::{ObjectDataCacheConfig, ObjectDataCacheMode};
use crate::key::ObjectDataCacheBodyVariant;
use bytes::Bytes;
fn fill_enabled_cache() -> ObjectDataCache {
let config = ObjectDataCacheConfig {
mode: ObjectDataCacheMode::FillBufferedOnly,
max_bytes: 8_388_608,
..ObjectDataCacheConfig::default()
};
ObjectDataCache::new(config).expect("fill-enabled cache config should initialize")
}
#[tokio::test]
async fn fill_body_rejects_size_mismatch() {
let cache = fill_enabled_cache();
let plan = cache.plan_get(ObjectDataCacheGetRequest {
bucket: "bucket",
object: "object",
version_id: None,
etag: "etag",
size: 5,
body_variant: ObjectDataCacheBodyVariant::FullObjectPlainV1,
});
let fill = cache.fill_body(&plan, Bytes::from_static(b"oops")).await;
let lookup = cache.lookup_body(&plan).await;
assert_eq!(fill, ObjectDataCacheFillResult::SkippedSizeMismatch);
assert!(matches!(lookup, ObjectDataCacheLookup::Miss));
}
}
+311
View File
@@ -0,0 +1,311 @@
// Copyright 2024 RustFS Team
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
use crate::error::ObjectDataCacheConfigError;
use std::time::Duration;
use sysinfo::System;
const DEFAULT_DERIVED_MAX_MEMORY_PERCENT_CAP: u64 = 10;
const DEFAULT_DERIVED_MAX_BYTES_CAP: u64 = 64 * 1024 * 1024 * 1024;
/// Runtime mode for the object data cache.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
pub enum ObjectDataCacheMode {
/// Cache is completely disabled.
#[default]
Disabled,
/// Cache lookups are allowed, but cache fill remains disabled.
HitOnly,
/// Cache fill is only allowed from an existing buffered body.
FillBufferedOnly,
/// Cache fill may materialize the final body stream exactly once.
FillMaterializeEnabled,
}
impl ObjectDataCacheMode {
pub(crate) const fn as_metric_label(self) -> &'static str {
match self {
Self::Disabled => "disabled",
Self::HitOnly => "hit_only",
Self::FillBufferedOnly => "fill_buffered_only",
Self::FillMaterializeEnabled => "fill_materialize_enabled",
}
}
}
/// Object data cache configuration.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct ObjectDataCacheConfig {
/// Runtime mode gate for the cache engine.
pub mode: ObjectDataCacheMode,
/// Explicit byte-capacity override. Zero means derive from memory percent.
pub max_bytes: u64,
/// Memory percent used when `max_bytes` is zero.
pub max_memory_percent: u8,
/// Maximum cacheable entry size in bytes.
pub max_entry_bytes: u64,
/// Time-to-live for a cache entry.
pub ttl: Duration,
/// Time-to-idle for a cache entry.
pub time_to_idle: Duration,
/// Minimum free memory percent before fill is paused.
pub min_free_memory_percent: u8,
/// Fill concurrency multiplier applied to CPU count.
pub fill_concurrency_per_cpu: u16,
/// Absolute fill concurrency cap.
pub fill_concurrency_max: u16,
/// Conservative cap for keys attached to one object identity.
pub identity_keys_max: u16,
}
impl Default for ObjectDataCacheConfig {
fn default() -> Self {
Self {
mode: ObjectDataCacheMode::Disabled,
max_bytes: 0,
max_memory_percent: 5,
max_entry_bytes: 1_048_576,
ttl: Duration::from_secs(60),
time_to_idle: Duration::from_secs(30),
min_free_memory_percent: 20,
fill_concurrency_per_cpu: 1,
fill_concurrency_max: 32,
identity_keys_max: 16,
}
}
}
impl ObjectDataCacheConfig {
/// Returns true when the cache is effectively disabled.
pub const fn is_disabled(&self) -> bool {
matches!(self.mode, ObjectDataCacheMode::Disabled)
}
/// Returns true when the cache mode allows lookups.
pub const fn lookup_enabled(&self) -> bool {
!self.is_disabled()
}
/// Returns true when the cache mode allows fills.
pub const fn fill_enabled(&self) -> bool {
matches!(
self.mode,
ObjectDataCacheMode::FillBufferedOnly | ObjectDataCacheMode::FillMaterializeEnabled
)
}
/// Resolves the effective max capacity in bytes for the cache.
pub fn resolved_max_bytes(&self) -> Result<u64, ObjectDataCacheConfigError> {
if self.max_bytes > 0 {
return Ok(self.max_bytes);
}
let mut system = System::new();
system.refresh_memory();
let total_memory = system.total_memory();
let derived = total_memory.saturating_mul(u64::from(self.max_memory_percent)) / 100;
let resolved = clamp_derived_max_bytes(derived, total_memory, self.max_entry_bytes);
if resolved == 0 {
return Err(ObjectDataCacheConfigError::ZeroResolvedMaxBytes);
}
Ok(resolved)
}
/// Validates the configuration for internal consistency.
pub fn validate(&self) -> Result<(), ObjectDataCacheConfigError> {
if self.max_bytes == 0 && (self.max_memory_percent == 0 || self.max_memory_percent > 100) {
return Err(ObjectDataCacheConfigError::InvalidMaxMemoryPercent);
}
if self.max_entry_bytes == 0 {
return Err(ObjectDataCacheConfigError::ZeroMaxEntryBytes);
}
if self.ttl.is_zero() {
return Err(ObjectDataCacheConfigError::ZeroTimeToLiveSecs);
}
if self.time_to_idle.is_zero() {
return Err(ObjectDataCacheConfigError::ZeroTimeToIdleSecs);
}
if self.min_free_memory_percent == 0 || self.min_free_memory_percent > 100 {
return Err(ObjectDataCacheConfigError::InvalidMinFreeMemoryPercent);
}
if self.fill_concurrency_per_cpu == 0 {
return Err(ObjectDataCacheConfigError::ZeroFillConcurrencyPerCpu);
}
if self.fill_concurrency_max == 0 {
return Err(ObjectDataCacheConfigError::ZeroFillConcurrencyMax);
}
if self.fill_concurrency_max < self.fill_concurrency_per_cpu {
return Err(ObjectDataCacheConfigError::FillConcurrencyMaxTooSmall);
}
if self.identity_keys_max == 0 {
return Err(ObjectDataCacheConfigError::ZeroIdentityKeysMax);
}
Ok(())
}
}
fn clamp_derived_max_bytes(derived: u64, total_memory: u64, max_entry_bytes: u64) -> u64 {
let percent_cap = total_memory.saturating_mul(DEFAULT_DERIVED_MAX_MEMORY_PERCENT_CAP) / 100;
let safe_cap = percent_cap.min(DEFAULT_DERIVED_MAX_BYTES_CAP).max(max_entry_bytes);
derived.min(safe_cap).max(max_entry_bytes)
}
#[cfg(test)]
mod tests {
use super::{DEFAULT_DERIVED_MAX_BYTES_CAP, ObjectDataCacheConfig, ObjectDataCacheMode, clamp_derived_max_bytes};
use crate::error::ObjectDataCacheConfigError;
use std::time::Duration;
#[test]
fn default_config_matches_v3_baseline() {
let config = ObjectDataCacheConfig::default();
assert!(matches!(config.mode, ObjectDataCacheMode::Disabled));
assert_eq!(config.max_bytes, 0);
assert_eq!(config.max_memory_percent, 5);
assert_eq!(config.max_entry_bytes, 1_048_576);
assert_eq!(config.ttl, Duration::from_secs(60));
assert_eq!(config.time_to_idle, Duration::from_secs(30));
assert_eq!(config.min_free_memory_percent, 20);
assert_eq!(config.fill_concurrency_per_cpu, 1);
assert_eq!(config.fill_concurrency_max, 32);
assert_eq!(config.identity_keys_max, 16);
}
#[test]
fn validate_rejects_invalid_memory_percent_when_capacity_is_derived() {
let config = ObjectDataCacheConfig {
max_memory_percent: 0,
..ObjectDataCacheConfig::default()
};
let err = config
.validate()
.expect_err("derived capacity requires a non-zero memory percent");
assert_eq!(err, ObjectDataCacheConfigError::InvalidMaxMemoryPercent);
}
#[test]
fn validate_rejects_zero_entry_size() {
let config = ObjectDataCacheConfig {
max_entry_bytes: 0,
..ObjectDataCacheConfig::default()
};
let err = config.validate().expect_err("entry size must stay positive");
assert_eq!(err, ObjectDataCacheConfigError::ZeroMaxEntryBytes);
}
#[test]
fn validate_rejects_zero_ttl() {
let config = ObjectDataCacheConfig {
ttl: Duration::ZERO,
..ObjectDataCacheConfig::default()
};
let err = config.validate().expect_err("ttl must stay positive");
assert_eq!(err, ObjectDataCacheConfigError::ZeroTimeToLiveSecs);
}
#[test]
fn validate_rejects_zero_time_to_idle() {
let config = ObjectDataCacheConfig {
time_to_idle: Duration::ZERO,
..ObjectDataCacheConfig::default()
};
let err = config.validate().expect_err("time-to-idle must stay positive");
assert_eq!(err, ObjectDataCacheConfigError::ZeroTimeToIdleSecs);
}
#[test]
fn validate_rejects_invalid_fill_concurrency_bounds() {
let config = ObjectDataCacheConfig {
fill_concurrency_per_cpu: 2,
fill_concurrency_max: 1,
..ObjectDataCacheConfig::default()
};
let err = config
.validate()
.expect_err("max fill concurrency must not be smaller than per-cpu factor");
assert_eq!(err, ObjectDataCacheConfigError::FillConcurrencyMaxTooSmall);
}
#[test]
fn validate_accepts_explicit_byte_cap() {
let config = ObjectDataCacheConfig {
mode: ObjectDataCacheMode::HitOnly,
max_bytes: 4_194_304,
max_memory_percent: 0,
..ObjectDataCacheConfig::default()
};
assert!(config.validate().is_ok());
}
#[test]
fn resolved_max_bytes_prefers_explicit_cap() {
let config = ObjectDataCacheConfig {
max_bytes: 4_194_304,
..ObjectDataCacheConfig::default()
};
let resolved = config
.resolved_max_bytes()
.expect("explicit max_bytes should be returned directly");
assert_eq!(resolved, 4_194_304);
}
#[test]
fn resolved_max_bytes_is_at_least_max_entry_bytes() {
let config = ObjectDataCacheConfig {
max_bytes: 0,
max_memory_percent: 1,
max_entry_bytes: 8_388_608,
..ObjectDataCacheConfig::default()
};
let resolved = config.resolved_max_bytes().expect("derived capacity should stay positive");
assert!(resolved >= config.max_entry_bytes);
}
#[test]
fn derived_max_bytes_clamps_to_v3_safe_cap() {
let one_tib = 1024_u64 * 1024 * 1024 * 1024;
let derived = one_tib / 2;
let resolved = clamp_derived_max_bytes(derived, one_tib, 1_048_576);
assert_eq!(resolved, DEFAULT_DERIVED_MAX_BYTES_CAP);
}
}
+113
View File
@@ -0,0 +1,113 @@
// Copyright 2024 RustFS Team
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
use bytes::Bytes;
use std::cmp;
use std::sync::Arc;
use std::time::Instant;
use crate::key::ObjectDataCacheKey;
const ENTRY_OVERHEAD_BYTES: usize = 64;
/// Cached object body entry.
#[derive(Debug, Clone)]
pub struct ObjectDataCacheEntry {
bytes: Bytes,
content_length: u64,
etag: Arc<str>,
inserted_at: Instant,
}
impl ObjectDataCacheEntry {
/// Creates a new cached entry.
pub fn new(bytes: Bytes, content_length: u64, etag: Arc<str>) -> Self {
Self {
bytes,
content_length,
etag,
inserted_at: Instant::now(),
}
}
/// Returns a clone of the cached body bytes.
pub fn bytes(&self) -> Bytes {
self.bytes.clone()
}
/// Returns the recorded content length.
pub const fn content_length(&self) -> u64 {
self.content_length
}
/// Returns the cached etag reference.
pub fn etag(&self) -> &Arc<str> {
&self.etag
}
/// Returns the insertion timestamp.
pub const fn inserted_at(&self) -> Instant {
self.inserted_at
}
/// Returns the estimated weighted size for capacity accounting.
pub fn estimated_weight(&self, key: &ObjectDataCacheKey) -> u32 {
let key_bytes = key.bucket.len() + key.object.len() + key.version_id.len() + key.etag.len();
let body_bytes = self.bytes.len();
let total = key_bytes.saturating_add(body_bytes).saturating_add(ENTRY_OVERHEAD_BYTES);
let clamped = cmp::min(total, u32::MAX as usize);
u32::try_from(clamped).unwrap_or(u32::MAX)
}
}
#[cfg(test)]
mod tests {
use super::ObjectDataCacheEntry;
use crate::key::{ObjectDataCacheBodyVariant, ObjectDataCacheKey};
use bytes::Bytes;
use std::sync::Arc;
#[test]
fn estimated_weight_includes_body_and_key_bytes() {
let key =
ObjectDataCacheKey::new("bucket", "object", Some("vid"), "etag", 5, ObjectDataCacheBodyVariant::FullObjectPlainV1);
let entry = ObjectDataCacheEntry::new(Bytes::from_static(b"hello"), 5, Arc::<str>::from("etag"));
let weight = entry.estimated_weight(&key);
assert!(weight >= 5);
}
#[test]
fn estimated_weight_clamps_to_u32_max() {
let huge_bucket = "b".repeat(1024);
let huge_object = "o".repeat(1024);
let huge_etag = "e".repeat(1024);
let key = ObjectDataCacheKey::new(
huge_bucket.as_str(),
huge_object.as_str(),
Some("version"),
huge_etag.as_str(),
1,
ObjectDataCacheBodyVariant::FullObjectPlainV1,
);
let huge = vec![0u8; (u32::MAX as usize).saturating_add(1024)];
let entry = ObjectDataCacheEntry::new(Bytes::from(huge), 1, Arc::<str>::from("etag"));
let weight = entry.estimated_weight(&key);
assert_eq!(weight, u32::MAX);
}
}
+59
View File
@@ -0,0 +1,59 @@
// Copyright 2024 RustFS Team
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
use thiserror::Error;
/// Configuration errors for the object data cache engine.
#[derive(Debug, Error, PartialEq, Eq)]
pub enum ObjectDataCacheConfigError {
/// The configured memory percentage exceeded the supported range.
#[error("object data cache max_memory_percent must be in 1..=100")]
InvalidMaxMemoryPercent,
/// The configured entry size exceeded the supported range.
#[error("object data cache max_entry_bytes must be greater than 0")]
ZeroMaxEntryBytes,
/// The configured time-to-live cannot be zero.
#[error("object data cache ttl_secs must be greater than 0")]
ZeroTimeToLiveSecs,
/// The configured time-to-idle cannot be zero.
#[error("object data cache time_to_idle_secs must be greater than 0")]
ZeroTimeToIdleSecs,
/// The configured minimum free memory percentage exceeded the supported range.
#[error("object data cache min_free_memory_percent must be in 1..=100")]
InvalidMinFreeMemoryPercent,
/// The configured fill concurrency per CPU exceeded the supported range.
#[error("object data cache fill_concurrency_per_cpu must be greater than 0")]
ZeroFillConcurrencyPerCpu,
/// The configured fill concurrency maximum exceeded the supported range.
#[error("object data cache fill_concurrency_max must be greater than 0")]
ZeroFillConcurrencyMax,
/// The configured fill concurrency bounds are internally inconsistent.
#[error("object data cache fill_concurrency_max must be at least fill_concurrency_per_cpu")]
FillConcurrencyMaxTooSmall,
/// The configured identity key-set cap exceeded the supported range.
#[error("object data cache identity_keys_max must be greater than 0")]
ZeroIdentityKeysMax,
/// Failed to resolve a non-zero cache capacity from the runtime environment.
#[error("object data cache could not resolve a positive max capacity")]
ZeroResolvedMaxBytes,
}
+127
View File
@@ -0,0 +1,127 @@
// Copyright 2024 RustFS Team
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
use crate::key::ObjectDataCacheKey;
use crate::starshard_index::StarshardIdentityIndex;
/// Result of inserting a cache key into the identity index.
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum ObjectDataCacheIndexInsertResult {
/// The key was inserted into the identity set.
Inserted,
/// The key was already tracked for the identity.
Duplicate,
/// The identity exceeded its configured key budget and was conservatively cleared.
Overflow {
/// Keys removed while clearing the identity.
cleared_keys: Vec<ObjectDataCacheKey>,
},
}
#[derive(Debug, Clone, Default, PartialEq, Eq)]
pub(crate) struct ObjectDataCacheKeySet {
keys: Vec<ObjectDataCacheKey>,
}
impl ObjectDataCacheKeySet {
pub(crate) fn insert(&mut self, key: ObjectDataCacheKey, max_keys: usize) -> ObjectDataCacheIndexInsertResult {
if self.keys.iter().any(|existing| existing == &key) {
return ObjectDataCacheIndexInsertResult::Duplicate;
}
if self.keys.len() >= max_keys {
let cleared_keys = self.drain();
return ObjectDataCacheIndexInsertResult::Overflow { cleared_keys };
}
self.keys.push(key);
ObjectDataCacheIndexInsertResult::Inserted
}
pub(crate) fn remove_key(&mut self, key: &ObjectDataCacheKey) -> bool {
let original_len = self.keys.len();
self.keys.retain(|existing| existing != key);
original_len != self.keys.len()
}
pub(crate) fn retain<F>(&mut self, mut keep: F)
where
F: FnMut(&ObjectDataCacheKey) -> bool,
{
self.keys.retain(|key| keep(key));
}
pub(crate) fn is_empty(&self) -> bool {
self.keys.is_empty()
}
pub(crate) fn contains(&self, key: &ObjectDataCacheKey) -> bool {
self.keys.iter().any(|existing| existing == key)
}
#[cfg(test)]
pub(crate) fn len(&self) -> usize {
self.keys.len()
}
pub(crate) fn cloned(&self) -> Vec<ObjectDataCacheKey> {
self.keys.clone()
}
pub(crate) fn drain(&mut self) -> Vec<ObjectDataCacheKey> {
std::mem::take(&mut self.keys)
}
}
/// Public identity-index façade used by the cache backend.
pub type ObjectDataCacheIdentityIndex = StarshardIdentityIndex;
#[cfg(test)]
mod tests {
use super::{ObjectDataCacheIndexInsertResult, ObjectDataCacheKeySet};
use crate::key::{ObjectDataCacheBodyVariant, ObjectDataCacheKey};
fn make_key(id: &str) -> ObjectDataCacheKey {
ObjectDataCacheKey::new("bucket", "object", Some(id), "etag", 1, ObjectDataCacheBodyVariant::FullObjectPlainV1)
}
#[test]
fn key_set_deduplicates_existing_key() {
let mut set = ObjectDataCacheKeySet::default();
let key = make_key("v1");
let first = set.insert(key.clone(), 4);
let second = set.insert(key, 4);
assert_eq!(first, ObjectDataCacheIndexInsertResult::Inserted);
assert_eq!(second, ObjectDataCacheIndexInsertResult::Duplicate);
assert_eq!(set.len(), 1);
}
#[test]
fn key_set_overflow_clears_existing_keys() {
let mut set = ObjectDataCacheKeySet::default();
let key_a = make_key("v1");
let key_b = make_key("v2");
let _ = set.insert(key_a.clone(), 1);
let result = set.insert(key_b, 1);
assert!(matches!(
result,
ObjectDataCacheIndexInsertResult::Overflow { cleared_keys } if cleared_keys == vec![key_a]
));
assert!(set.is_empty());
}
}
+118
View File
@@ -0,0 +1,118 @@
// Copyright 2024 RustFS Team
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
use std::sync::Arc;
/// Canonical synthetic version id for unversioned or latest-only object bodies.
pub const NULL_VERSION_ID: &str = "null";
/// Response body variant supported by the cache.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, Default)]
pub enum ObjectDataCacheBodyVariant {
/// Full plain object body.
#[default]
FullObjectPlainV1,
}
/// Stable cache key for a reusable object body.
#[derive(Debug, Clone, PartialEq, Eq, Hash)]
pub struct ObjectDataCacheKey {
/// Bucket name.
pub bucket: Arc<str>,
/// Object key.
pub object: Arc<str>,
/// Canonical version id, using `"null"` for unversioned bodies.
pub version_id: Arc<str>,
/// Object ETag.
pub etag: Arc<str>,
/// Object size in bytes.
pub size: u64,
/// Cached body semantics.
pub body_variant: ObjectDataCacheBodyVariant,
}
/// Identity used for conservative invalidation.
#[derive(Debug, Clone, PartialEq, Eq, Hash)]
pub struct ObjectDataCacheIdentity {
/// Bucket name.
pub bucket: Arc<str>,
/// Object key.
pub object: Arc<str>,
}
impl ObjectDataCacheKey {
/// Creates a new stable object data cache key.
pub fn new(
bucket: impl Into<Arc<str>>,
object: impl Into<Arc<str>>,
version_id: Option<&str>,
etag: impl Into<Arc<str>>,
size: u64,
body_variant: ObjectDataCacheBodyVariant,
) -> Self {
Self {
bucket: bucket.into(),
object: object.into(),
version_id: version_id.map_or_else(|| Arc::<str>::from(NULL_VERSION_ID), Arc::<str>::from),
etag: etag.into(),
size,
body_variant,
}
}
/// Returns true when this key targets the canonical unversioned body variant.
pub fn is_null_version(&self) -> bool {
self.version_id.as_ref() == NULL_VERSION_ID
}
}
impl ObjectDataCacheIdentity {
/// Creates a conservative invalidation identity.
pub fn new(bucket: impl Into<Arc<str>>, object: impl Into<Arc<str>>) -> Self {
Self {
bucket: bucket.into(),
object: object.into(),
}
}
}
#[cfg(test)]
mod tests {
use super::{NULL_VERSION_ID, ObjectDataCacheBodyVariant, ObjectDataCacheIdentity, ObjectDataCacheKey};
#[test]
fn key_uses_canonical_null_version_for_missing_version_id() {
let key = ObjectDataCacheKey::new("bucket", "object", None, "etag", 42, ObjectDataCacheBodyVariant::FullObjectPlainV1);
assert_eq!(key.version_id.as_ref(), NULL_VERSION_ID);
assert!(key.is_null_version());
}
#[test]
fn key_distinguishes_explicit_version_ids() {
let latest = ObjectDataCacheKey::new("bucket", "object", None, "etag", 42, ObjectDataCacheBodyVariant::FullObjectPlainV1);
let versioned =
ObjectDataCacheKey::new("bucket", "object", Some("3d2"), "etag", 42, ObjectDataCacheBodyVariant::FullObjectPlainV1);
assert_ne!(latest, versioned);
}
#[test]
fn identity_new_preserves_bucket_and_object() {
let identity = ObjectDataCacheIdentity::new("bucket", "object");
assert_eq!(identity.bucket.as_ref(), "bucket");
assert_eq!(identity.object.as_ref(), "object");
}
}
+43
View File
@@ -0,0 +1,43 @@
// Copyright 2024 RustFS Team
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//! Engine-only object body cache contracts for RustFS.
//!
//! This crate intentionally contains only a minimal skeleton for the initial
//! rollout phase. App-layer semantics and storage-specific read paths stay in
//! the `rustfs` crate.
pub mod backend;
pub mod cache;
pub mod config;
pub mod entry;
pub mod error;
pub mod index;
pub mod key;
pub mod memory;
pub mod metrics;
pub mod moka_backend;
pub mod noop;
pub mod singleflight;
pub mod starshard_index;
pub mod stats;
pub use cache::{
ObjectDataCache, ObjectDataCacheFillResult, ObjectDataCacheGetPlan, ObjectDataCacheGetRequest,
ObjectDataCacheInvalidationReason, ObjectDataCacheInvalidationResult, ObjectDataCacheLookup,
};
pub use config::{ObjectDataCacheConfig, ObjectDataCacheMode};
pub use error::ObjectDataCacheConfigError;
pub use key::{NULL_VERSION_ID, ObjectDataCacheBodyVariant, ObjectDataCacheIdentity, ObjectDataCacheKey};
pub use stats::{ObjectDataCacheStats, ObjectDataCacheStatsSnapshot};
+184
View File
@@ -0,0 +1,184 @@
// Copyright 2024 RustFS Team
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
use crate::config::ObjectDataCacheConfig;
use crate::metrics::record_memory_pressure;
use crate::stats::ObjectDataCacheStats;
use std::sync::atomic::{AtomicU64, Ordering};
use std::sync::{Arc, Mutex};
use std::time::{Duration, Instant};
use sysinfo::System;
const DEFAULT_REFRESH_INTERVAL: Duration = Duration::from_secs(5);
/// Immutable memory snapshot used by the cache fill gate.
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)]
pub struct ObjectDataCacheMemorySnapshot {
/// Total system memory in bytes.
pub total_bytes: u64,
/// Available system memory in bytes.
pub available_bytes: u64,
}
impl ObjectDataCacheMemorySnapshot {
/// Returns the available memory percentage.
pub fn available_percent(&self) -> u8 {
if self.total_bytes == 0 {
return 0;
}
let percent = self.available_bytes.saturating_mul(100) / self.total_bytes;
u8::try_from(percent.min(100)).unwrap_or(100)
}
}
/// Memory gate that keeps a cheap snapshot for fill-path checks.
#[derive(Debug)]
pub struct ObjectDataCacheMemoryGate {
system: Mutex<System>,
last_refresh: Mutex<Instant>,
snapshot_total_bytes: AtomicU64,
snapshot_available_bytes: AtomicU64,
min_free_memory_percent: u8,
refresh_interval: Duration,
stats: Arc<ObjectDataCacheStats>,
#[cfg(test)]
test_override: Mutex<Option<ObjectDataCacheMemorySnapshot>>,
}
impl ObjectDataCacheMemoryGate {
/// Creates a new memory gate.
pub fn new(config: &ObjectDataCacheConfig, stats: Arc<ObjectDataCacheStats>) -> Self {
let mut system = System::new();
system.refresh_memory();
let snapshot = ObjectDataCacheMemorySnapshot {
total_bytes: system.total_memory(),
available_bytes: system.available_memory(),
};
Self {
system: Mutex::new(system),
last_refresh: Mutex::new(Instant::now()),
snapshot_total_bytes: AtomicU64::new(snapshot.total_bytes),
snapshot_available_bytes: AtomicU64::new(snapshot.available_bytes),
min_free_memory_percent: config.min_free_memory_percent,
refresh_interval: DEFAULT_REFRESH_INTERVAL,
stats,
#[cfg(test)]
test_override: Mutex::new(None),
}
}
/// Returns the current atomic memory snapshot.
pub fn snapshot(&self) -> ObjectDataCacheMemorySnapshot {
#[cfg(test)]
{
if let Some(snapshot) = *lock_or_recover(&self.test_override) {
return snapshot;
}
}
ObjectDataCacheMemorySnapshot {
total_bytes: self.snapshot_total_bytes.load(Ordering::Relaxed),
available_bytes: self.snapshot_available_bytes.load(Ordering::Relaxed),
}
}
/// Refreshes the snapshot if it is stale.
pub fn refresh_if_stale(&self) {
{
let last_refresh = lock_or_recover(&self.last_refresh);
if last_refresh.elapsed() < self.refresh_interval {
return;
}
}
let mut system = lock_or_recover(&self.system);
system.refresh_memory();
let snapshot = ObjectDataCacheMemorySnapshot {
total_bytes: system.total_memory(),
available_bytes: system.available_memory(),
};
self.snapshot_total_bytes.store(snapshot.total_bytes, Ordering::Relaxed);
self.snapshot_available_bytes
.store(snapshot.available_bytes, Ordering::Relaxed);
*lock_or_recover(&self.last_refresh) = Instant::now();
}
/// Returns true when the fill path may proceed under current memory pressure.
pub fn allows_fill(&self, required_bytes: u64) -> bool {
self.refresh_if_stale();
let snapshot = self.snapshot();
if snapshot.total_bytes == 0 {
return true;
}
let min_free = u64::from(self.min_free_memory_percent);
let has_percent_budget = snapshot.available_bytes.saturating_mul(100) >= snapshot.total_bytes.saturating_mul(min_free);
let has_entry_budget = snapshot.available_bytes >= required_bytes;
let allowed = has_percent_budget && has_entry_budget;
if !allowed {
record_memory_pressure(&self.stats, "moka");
}
allowed
}
#[cfg(test)]
pub fn set_test_snapshot(&self, snapshot: Option<ObjectDataCacheMemorySnapshot>) {
*lock_or_recover(&self.test_override) = snapshot;
}
}
fn lock_or_recover<T>(mutex: &Mutex<T>) -> std::sync::MutexGuard<'_, T> {
match mutex.lock() {
Ok(guard) => guard,
Err(poisoned) => poisoned.into_inner(),
}
}
#[cfg(test)]
mod tests {
use super::{ObjectDataCacheMemoryGate, ObjectDataCacheMemorySnapshot};
use crate::config::ObjectDataCacheConfig;
use crate::stats::ObjectDataCacheStats;
use std::sync::Arc;
#[test]
fn allows_fill_when_memory_snapshot_has_headroom() {
let stats = Arc::new(ObjectDataCacheStats::default());
let gate = ObjectDataCacheMemoryGate::new(&ObjectDataCacheConfig::default(), Arc::clone(&stats));
gate.set_test_snapshot(Some(ObjectDataCacheMemorySnapshot {
total_bytes: 1_000,
available_bytes: 500,
}));
assert!(gate.allows_fill(100));
}
#[test]
fn blocks_fill_under_memory_pressure() {
let stats = Arc::new(ObjectDataCacheStats::default());
let gate = ObjectDataCacheMemoryGate::new(&ObjectDataCacheConfig::default(), Arc::clone(&stats));
gate.set_test_snapshot(Some(ObjectDataCacheMemorySnapshot {
total_bytes: 1_000,
available_bytes: 100,
}));
assert!(!gate.allows_fill(128));
assert_eq!(stats.snapshot().memory_pressure_events, 1);
}
}
+156
View File
@@ -0,0 +1,156 @@
// Copyright 2024 RustFS Team
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
use crate::config::ObjectDataCacheMode;
use crate::stats::ObjectDataCacheStats;
use metrics::{counter, describe_counter, describe_gauge, describe_histogram, gauge, histogram};
use std::sync::{Arc, OnceLock};
const METRIC_REQUESTS_TOTAL: &str = "rustfs_object_data_cache_requests_total";
const METRIC_FILLS_TOTAL: &str = "rustfs_object_data_cache_fill_total";
const METRIC_FILL_DURATION_SECONDS: &str = "rustfs_object_data_cache_fill_duration_seconds";
const METRIC_FILL_BYTES_TOTAL: &str = "rustfs_object_data_cache_fill_bytes_total";
const METRIC_HIT_BYTES_TOTAL: &str = "rustfs_object_data_cache_hit_bytes_total";
const METRIC_ENTRIES: &str = "rustfs_object_data_cache_entries";
const METRIC_WEIGHTED_BYTES: &str = "rustfs_object_data_cache_weighted_bytes";
const METRIC_INFLIGHT_FILLS: &str = "rustfs_object_data_cache_inflight_fills";
const METRIC_INVALIDATIONS_TOTAL: &str = "rustfs_object_data_cache_invalidations_total";
const METRIC_MEMORY_PRESSURE_TOTAL: &str = "rustfs_object_data_cache_memory_pressure_total";
pub(crate) fn describe_metrics_once() {
static DESCRIBED: OnceLock<()> = OnceLock::new();
let _ = DESCRIBED.get_or_init(|| {
describe_counter!(
METRIC_REQUESTS_TOTAL,
"Object data cache request decisions labeled by backend, mode, decision, reason, and size class."
);
describe_counter!(
METRIC_FILLS_TOTAL,
"Object data cache fill outcomes by backend, mode, result, and size class."
);
describe_histogram!(METRIC_FILL_DURATION_SECONDS, "Object data cache fill duration in seconds.");
describe_counter!(METRIC_FILL_BYTES_TOTAL, "Total bytes submitted to object data cache fill operations.");
describe_counter!(METRIC_HIT_BYTES_TOTAL, "Total bytes served from object data cache hits.");
describe_gauge!(METRIC_ENTRIES, "Current object data cache entry count.");
describe_gauge!(METRIC_WEIGHTED_BYTES, "Approximate weighted bytes held by the object data cache.");
describe_gauge!(METRIC_INFLIGHT_FILLS, "Current number of in-flight object data cache fills.");
describe_counter!(
METRIC_INVALIDATIONS_TOTAL,
"Object data cache invalidation attempts by backend and reason."
);
describe_counter!(METRIC_MEMORY_PRESSURE_TOTAL, "Object data cache fill skips caused by memory pressure.");
});
}
pub(crate) const fn size_class(size_bytes: u64) -> &'static str {
if size_bytes <= 4 * 1024 {
"le_4k"
} else if size_bytes <= 64 * 1024 {
"le_64k"
} else if size_bytes <= 256 * 1024 {
"le_256k"
} else if size_bytes <= 1024 * 1024 {
"le_1m"
} else if size_bytes <= 4 * 1024 * 1024 {
"le_4m"
} else {
"gt_4m"
}
}
pub(crate) fn record_request_decision(
backend: &'static str,
mode: ObjectDataCacheMode,
decision: &'static str,
reason: &'static str,
size_bytes: u64,
) {
counter!(
METRIC_REQUESTS_TOTAL,
"backend" => backend,
"mode" => mode.as_metric_label(),
"decision" => decision,
"reason" => reason,
"size_class" => size_class(size_bytes),
)
.increment(1);
}
pub(crate) fn record_fill_result(
backend: &'static str,
mode: ObjectDataCacheMode,
result: &'static str,
size_bytes: u64,
duration_seconds: f64,
) {
let size_class = size_class(size_bytes);
counter!(
METRIC_FILLS_TOTAL,
"backend" => backend,
"mode" => mode.as_metric_label(),
"result" => result,
"size_class" => size_class,
)
.increment(1);
histogram!(
METRIC_FILL_DURATION_SECONDS,
"backend" => backend,
"mode" => mode.as_metric_label(),
"result" => result,
"size_class" => size_class,
)
.record(duration_seconds);
counter!(
METRIC_FILL_BYTES_TOTAL,
"backend" => backend,
"mode" => mode.as_metric_label(),
"result" => result,
"size_class" => size_class,
)
.increment(size_bytes);
}
pub(crate) fn record_hit_bytes(backend: &'static str, mode: ObjectDataCacheMode, size_bytes: u64) {
counter!(
METRIC_HIT_BYTES_TOTAL,
"backend" => backend,
"mode" => mode.as_metric_label(),
"size_class" => size_class(size_bytes),
)
.increment(size_bytes);
}
pub(crate) fn publish_cache_state(backend: &'static str, entries: u64, weighted_bytes: u64) {
gauge!(METRIC_ENTRIES, "backend" => backend).set(entries as f64);
gauge!(METRIC_WEIGHTED_BYTES, "backend" => backend).set(weighted_bytes as f64);
}
pub(crate) fn set_inflight_fills(stats: &Arc<ObjectDataCacheStats>, backend: &'static str, count: usize) {
stats.set_inflight_fills(count);
let count_u64 = u64::try_from(count).unwrap_or(u64::MAX);
gauge!(METRIC_INFLIGHT_FILLS, "backend" => backend).set(count_u64 as f64);
}
pub(crate) fn record_singleflight_join(stats: &Arc<ObjectDataCacheStats>) {
stats.record_singleflight_join();
}
pub(crate) fn record_memory_pressure(stats: &Arc<ObjectDataCacheStats>, backend: &'static str) {
stats.record_memory_pressure();
counter!(METRIC_MEMORY_PRESSURE_TOTAL, "backend" => backend).increment(1);
}
pub(crate) fn record_invalidation(backend: &'static str, reason: &'static str) {
counter!(METRIC_INVALIDATIONS_TOTAL, "backend" => backend, "reason" => reason).increment(1);
}
@@ -0,0 +1,288 @@
// Copyright 2024 RustFS Team
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
use crate::cache::{ObjectDataCacheFillResult, ObjectDataCacheGetPlan, ObjectDataCacheInvalidationResult, ObjectDataCacheLookup};
use crate::config::ObjectDataCacheConfig;
use crate::entry::ObjectDataCacheEntry;
use crate::index::{ObjectDataCacheIdentityIndex, ObjectDataCacheIndexInsertResult};
use crate::key::ObjectDataCacheIdentity;
use crate::memory::ObjectDataCacheMemoryGate;
use crate::singleflight::{ObjectDataCacheSingleflight, ObjectDataCacheSingleflightAcquire};
use crate::stats::ObjectDataCacheStats;
use bytes::Bytes;
use moka::future::Cache;
use std::sync::Arc;
/// Weighted Moka backend for reusable object bodies.
#[derive(Debug)]
pub struct MokaBackend {
cache: Cache<crate::key::ObjectDataCacheKey, Arc<ObjectDataCacheEntry>>,
index: ObjectDataCacheIdentityIndex,
singleflight: ObjectDataCacheSingleflight,
memory_gate: ObjectDataCacheMemoryGate,
}
impl MokaBackend {
/// Creates a new backend from the validated configuration.
pub fn new(
config: &ObjectDataCacheConfig,
stats: Arc<ObjectDataCacheStats>,
) -> Result<Self, crate::error::ObjectDataCacheConfigError> {
let max_capacity = config.resolved_max_bytes()?;
let ttl = config.ttl;
let time_to_idle = config.time_to_idle;
let cache = Cache::builder()
.max_capacity(max_capacity)
.weigher(|key, value: &Arc<ObjectDataCacheEntry>| value.estimated_weight(key))
.time_to_live(ttl)
.time_to_idle(time_to_idle)
.build();
Ok(Self {
cache,
index: ObjectDataCacheIdentityIndex::new(usize::from(config.identity_keys_max)),
singleflight: ObjectDataCacheSingleflight::new(Arc::clone(&stats)),
memory_gate: ObjectDataCacheMemoryGate::new(config, stats),
})
}
/// Returns the current cache entry count.
pub fn entry_count(&self) -> u64 {
self.cache.entry_count()
}
/// Returns the approximate weighted size of cached entries.
pub fn weighted_size(&self) -> u64 {
self.cache.weighted_size()
}
/// Looks up a cached body for the supplied plan.
pub async fn lookup_body(&self, plan: &ObjectDataCacheGetPlan) -> ObjectDataCacheLookup {
let ObjectDataCacheGetPlan::Cacheable { key } = plan else {
return ObjectDataCacheLookup::SkipNotCacheable;
};
match self.cache.get(key).await {
Some(entry) => ObjectDataCacheLookup::Hit(entry.bytes()),
None => ObjectDataCacheLookup::Miss,
}
}
/// Inserts a cached body for the supplied plan.
pub async fn fill_body(&self, plan: &ObjectDataCacheGetPlan, bytes: Bytes) -> ObjectDataCacheFillResult {
let ObjectDataCacheGetPlan::Cacheable { key } = plan else {
return ObjectDataCacheFillResult::SkippedNotCacheable;
};
let fill_state = self.singleflight.acquire(key.clone()).await;
let ObjectDataCacheSingleflightAcquire::Leader(leader) = fill_state else {
let ObjectDataCacheSingleflightAcquire::Waiter(waiter) = fill_state else {
unreachable!();
};
return waiter.wait().await;
};
if !self.memory_gate.allows_fill(u64::try_from(bytes.len()).unwrap_or(u64::MAX)) {
return leader.finish(ObjectDataCacheFillResult::SkippedMemoryPressure).await;
}
let identity = ObjectDataCacheIdentity::new(Arc::clone(&key.bucket), Arc::clone(&key.object));
self.index
.prune_missing(&identity, |candidate| self.cache.contains_key(candidate))
.await;
// Register the key in the identity index BEFORE the entry becomes
// visible in the cache, so a concurrent invalidation always finds it.
let result = match self.index.insert(identity.clone(), key.clone()).await {
ObjectDataCacheIndexInsertResult::Inserted | ObjectDataCacheIndexInsertResult::Duplicate => {
let entry = Arc::new(ObjectDataCacheEntry::new(bytes, key.size, Arc::clone(&key.etag)));
self.cache.insert(key.clone(), entry).await;
// An invalidation may have raced between the index and cache
// inserts; re-check the index and undo the fill so the stale
// body cannot outlive the invalidation.
if self.index.contains_key(&identity, key).await {
ObjectDataCacheFillResult::Inserted
} else {
self.cache.remove(key).await;
ObjectDataCacheFillResult::SkippedInvalidationRace
}
}
ObjectDataCacheIndexInsertResult::Overflow { cleared_keys } => {
for stale_key in cleared_keys {
self.cache.remove(&stale_key).await;
}
ObjectDataCacheFillResult::SkippedIdentityOverflow
}
};
leader.finish(result).await
}
/// Conservatively invalidates all cached keys matching the object identity.
///
/// The identity index is authoritative: fills register the key in the
/// index before the entry becomes visible in the cache (and undo the fill
/// if an invalidation raced in between), so no full-cache scan fallback is
/// needed when the index has no entry for the identity.
pub async fn invalidate_object(&self, identity: &ObjectDataCacheIdentity) -> ObjectDataCacheInvalidationResult {
let keys_to_remove = self.index.remove_identity(identity).await;
for key in keys_to_remove {
self.cache.remove(&key).await;
}
ObjectDataCacheInvalidationResult::Success
}
}
#[cfg(test)]
mod tests {
use super::MokaBackend;
use crate::cache::{
ObjectDataCacheFillResult, ObjectDataCacheGetPlan, ObjectDataCacheInvalidationResult, ObjectDataCacheLookup,
};
use crate::config::{ObjectDataCacheConfig, ObjectDataCacheMode};
use crate::key::{ObjectDataCacheBodyVariant, ObjectDataCacheIdentity, ObjectDataCacheKey};
use crate::stats::ObjectDataCacheStats;
use bytes::Bytes;
use std::sync::Arc;
use std::time::Duration;
fn enabled_config() -> ObjectDataCacheConfig {
ObjectDataCacheConfig {
mode: ObjectDataCacheMode::FillMaterializeEnabled,
max_bytes: 8_388_608,
max_memory_percent: 5,
max_entry_bytes: 1_048_576,
ttl: Duration::from_millis(100),
time_to_idle: Duration::from_millis(100),
min_free_memory_percent: 20,
fill_concurrency_per_cpu: 1,
fill_concurrency_max: 32,
identity_keys_max: 16,
}
}
fn cacheable_plan(object: &str, etag: &str) -> ObjectDataCacheGetPlan {
ObjectDataCacheGetPlan::Cacheable {
key: ObjectDataCacheKey::new("bucket", object, None, etag, 5, ObjectDataCacheBodyVariant::FullObjectPlainV1),
}
}
#[tokio::test]
async fn moka_backend_round_trips_cached_body() {
let backend =
MokaBackend::new(&enabled_config(), Arc::new(ObjectDataCacheStats::default())).expect("moka backend should build");
let plan = cacheable_plan("object", "etag-a");
let fill = backend.fill_body(&plan, Bytes::from_static(b"hello")).await;
let lookup = backend.lookup_body(&plan).await;
assert!(matches!(fill, ObjectDataCacheFillResult::Inserted));
assert!(matches!(lookup, ObjectDataCacheLookup::Hit(ref bytes) if bytes.as_ref() == b"hello"));
}
#[tokio::test]
async fn moka_backend_invalidates_matching_identity() {
let backend =
MokaBackend::new(&enabled_config(), Arc::new(ObjectDataCacheStats::default())).expect("moka backend should build");
let plan_a = cacheable_plan("object-a", "etag-a");
let plan_b = cacheable_plan("object-b", "etag-b");
let _ = backend.fill_body(&plan_a, Bytes::from_static(b"aaaaa")).await;
let _ = backend.fill_body(&plan_b, Bytes::from_static(b"bbbbb")).await;
let result = backend
.invalidate_object(&ObjectDataCacheIdentity::new("bucket", "object-a"))
.await;
let lookup_a = backend.lookup_body(&plan_a).await;
let lookup_b = backend.lookup_body(&plan_b).await;
assert!(matches!(result, ObjectDataCacheInvalidationResult::Success));
assert!(matches!(lookup_a, ObjectDataCacheLookup::Miss));
assert!(matches!(lookup_b, ObjectDataCacheLookup::Hit(_)));
}
#[tokio::test]
async fn moka_backend_expires_entries_by_ttl() {
let backend =
MokaBackend::new(&enabled_config(), Arc::new(ObjectDataCacheStats::default())).expect("moka backend should build");
let plan = cacheable_plan("object", "etag-a");
let _ = backend.fill_body(&plan, Bytes::from_static(b"hello")).await;
tokio::time::sleep(Duration::from_millis(150)).await;
let lookup = backend.lookup_body(&plan).await;
assert!(matches!(lookup, ObjectDataCacheLookup::Miss));
}
#[tokio::test]
async fn moka_backend_expires_entries_by_tti() {
let mut config = enabled_config();
config.ttl = Duration::from_secs(30);
config.time_to_idle = Duration::from_millis(100);
let backend = MokaBackend::new(&config, Arc::new(ObjectDataCacheStats::default())).expect("moka backend should build");
let plan = cacheable_plan("object", "etag-a");
let _ = backend.fill_body(&plan, Bytes::from_static(b"hello")).await;
tokio::time::sleep(Duration::from_millis(150)).await;
let lookup = backend.lookup_body(&plan).await;
assert!(matches!(lookup, ObjectDataCacheLookup::Miss));
}
#[tokio::test]
async fn moka_backend_skips_fill_under_memory_pressure() {
let stats = Arc::new(ObjectDataCacheStats::default());
let backend = MokaBackend::new(&enabled_config(), Arc::clone(&stats)).expect("moka backend should build");
backend
.memory_gate
.set_test_snapshot(Some(crate::memory::ObjectDataCacheMemorySnapshot {
total_bytes: 1_000,
available_bytes: 100,
}));
let plan = cacheable_plan("object", "etag-a");
let result = backend.fill_body(&plan, Bytes::from_static(b"hello")).await;
let lookup = backend.lookup_body(&plan).await;
assert_eq!(result, ObjectDataCacheFillResult::SkippedMemoryPressure);
assert!(matches!(lookup, ObjectDataCacheLookup::Miss));
assert_eq!(stats.snapshot().memory_pressure_events, 1);
}
#[tokio::test]
async fn moka_backend_singleflight_waiter_observes_leader_result() {
let stats = Arc::new(ObjectDataCacheStats::default());
let backend = Arc::new(MokaBackend::new(&enabled_config(), Arc::clone(&stats)).expect("moka backend should build"));
let plan = cacheable_plan("object", "etag-a");
let leader_plan = plan.clone();
let plan_clone = plan.clone();
let first = Arc::clone(&backend);
let second = Arc::clone(&backend);
let leader = tokio::spawn(async move { first.fill_body(&leader_plan, Bytes::from_static(b"hello")).await });
let waiter = tokio::spawn(async move { second.fill_body(&plan_clone, Bytes::from_static(b"hello")).await });
let leader_result = leader.await.expect("leader task should complete");
let waiter_result = waiter.await.expect("waiter task should complete");
let lookup = backend.lookup_body(&plan).await;
assert_eq!(leader_result, ObjectDataCacheFillResult::Inserted);
assert_eq!(waiter_result, ObjectDataCacheFillResult::Inserted);
assert!(matches!(lookup, ObjectDataCacheLookup::Hit(ref bytes) if bytes.as_ref() == b"hello"));
}
}
+58
View File
@@ -0,0 +1,58 @@
// Copyright 2024 RustFS Team
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
use crate::cache::{ObjectDataCacheFillResult, ObjectDataCacheGetPlan, ObjectDataCacheInvalidationResult, ObjectDataCacheLookup};
/// Lightweight no-op backend for the initial skeleton.
#[derive(Debug, Default)]
pub struct NoopBackend;
impl NoopBackend {
/// Returns a disabled lookup result.
pub async fn lookup_body(&self, _plan: &ObjectDataCacheGetPlan) -> ObjectDataCacheLookup {
ObjectDataCacheLookup::SkipDisabled
}
/// Returns a skipped fill result.
pub async fn fill_body(&self, _plan: &ObjectDataCacheGetPlan) -> ObjectDataCacheFillResult {
ObjectDataCacheFillResult::SkippedDisabled
}
/// Returns a successful no-op invalidation result.
pub async fn invalidate_object(&self) -> ObjectDataCacheInvalidationResult {
ObjectDataCacheInvalidationResult::Success
}
}
#[cfg(test)]
mod tests {
use super::NoopBackend;
use crate::cache::{
ObjectDataCacheFillResult, ObjectDataCacheGetPlan, ObjectDataCacheInvalidationResult, ObjectDataCacheLookup,
};
#[tokio::test]
async fn noop_backend_returns_disabled_results() {
let backend = NoopBackend;
let plan = ObjectDataCacheGetPlan::Disabled;
let lookup = backend.lookup_body(&plan).await;
let fill = backend.fill_body(&plan).await;
let invalidation = backend.invalidate_object().await;
assert!(matches!(lookup, ObjectDataCacheLookup::SkipDisabled));
assert!(matches!(fill, ObjectDataCacheFillResult::SkippedDisabled));
assert!(matches!(invalidation, ObjectDataCacheInvalidationResult::Success));
}
}
@@ -0,0 +1,198 @@
// Copyright 2024 RustFS Team
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
use crate::cache::ObjectDataCacheFillResult;
use crate::key::ObjectDataCacheKey;
use crate::metrics::{record_singleflight_join, set_inflight_fills};
use crate::stats::ObjectDataCacheStats;
use std::collections::HashMap;
use std::sync::{Arc, Mutex};
use tokio::sync::watch;
type FillMap = Mutex<HashMap<ObjectDataCacheKey, watch::Sender<Option<ObjectDataCacheFillResult>>>>;
fn lock_fills(
fills: &FillMap,
) -> std::sync::MutexGuard<'_, HashMap<ObjectDataCacheKey, watch::Sender<Option<ObjectDataCacheFillResult>>>> {
fills.lock().unwrap_or_else(|poisoned| poisoned.into_inner())
}
/// Shared singleflight controller for cache fill operations.
#[derive(Debug)]
pub struct ObjectDataCacheSingleflight {
fills: FillMap,
stats: Arc<ObjectDataCacheStats>,
}
impl ObjectDataCacheSingleflight {
/// Creates a new singleflight controller.
pub fn new(stats: Arc<ObjectDataCacheStats>) -> Self {
Self {
fills: Mutex::new(HashMap::new()),
stats,
}
}
/// Acquires a leader-or-waiter role for the supplied cache key.
pub async fn acquire(&self, key: ObjectDataCacheKey) -> ObjectDataCacheSingleflightAcquire<'_> {
let mut fills = lock_fills(&self.fills);
if let Some(sender) = fills.get(&key) {
record_singleflight_join(&self.stats);
return ObjectDataCacheSingleflightAcquire::Waiter(ObjectDataCacheSingleflightWaiter { rx: sender.subscribe() });
}
let (tx, _rx) = watch::channel(None);
fills.insert(key.clone(), tx.clone());
set_inflight_fills(&self.stats, "moka", fills.len());
drop(fills);
ObjectDataCacheSingleflightAcquire::Leader(ObjectDataCacheSingleflightLeader {
key,
tx,
fills: &self.fills,
stats: Arc::clone(&self.stats),
finished: false,
})
}
}
/// Leader or waiter result from the singleflight map.
pub enum ObjectDataCacheSingleflightAcquire<'a> {
/// Caller is responsible for performing the fill operation.
Leader(ObjectDataCacheSingleflightLeader<'a>),
/// Caller must wait for the leader's result.
Waiter(ObjectDataCacheSingleflightWaiter),
}
/// Leader handle for a singleflight fill operation.
pub struct ObjectDataCacheSingleflightLeader<'a> {
key: ObjectDataCacheKey,
tx: watch::Sender<Option<ObjectDataCacheFillResult>>,
fills: &'a FillMap,
stats: Arc<ObjectDataCacheStats>,
finished: bool,
}
impl<'a> ObjectDataCacheSingleflightLeader<'a> {
/// Completes the leader operation and publishes the shared result.
pub async fn finish(mut self, result: ObjectDataCacheFillResult) -> ObjectDataCacheFillResult {
self.remove_entry();
self.finished = true;
let _ = self.tx.send(Some(result.clone()));
result
}
fn remove_entry(&self) {
let mut fills = lock_fills(self.fills);
fills.remove(&self.key);
set_inflight_fills(&self.stats, "moka", fills.len());
}
}
impl<'a> Drop for ObjectDataCacheSingleflightLeader<'a> {
fn drop(&mut self) {
// A leader dropped without finish() was cancelled mid-fill (e.g. the
// GET request future was aborted). Remove the map entry so its sender
// clone is released and waiters observe the closed channel instead of
// blocking forever, and so a later fill can become the new leader.
if !self.finished {
self.remove_entry();
}
}
}
/// Waiter handle for a singleflight fill operation.
pub struct ObjectDataCacheSingleflightWaiter {
rx: watch::Receiver<Option<ObjectDataCacheFillResult>>,
}
impl ObjectDataCacheSingleflightWaiter {
/// Waits for the shared fill result.
pub async fn wait(mut self) -> ObjectDataCacheFillResult {
if self.rx.wait_for(|value| value.is_some()).await.is_err() {
return ObjectDataCacheFillResult::SkippedSingleflightClosed;
}
self.rx
.borrow()
.as_ref()
.cloned()
.unwrap_or(ObjectDataCacheFillResult::SkippedSingleflightClosed)
}
}
#[cfg(test)]
mod tests {
use super::{ObjectDataCacheSingleflight, ObjectDataCacheSingleflightAcquire};
use crate::cache::ObjectDataCacheFillResult;
use crate::key::{ObjectDataCacheBodyVariant, ObjectDataCacheKey};
use crate::stats::ObjectDataCacheStats;
use std::sync::Arc;
fn key() -> ObjectDataCacheKey {
ObjectDataCacheKey::new("bucket", "object", None, "etag", 1, ObjectDataCacheBodyVariant::FullObjectPlainV1)
}
#[tokio::test]
async fn singleflight_returns_waiter_for_second_caller() {
let stats = Arc::new(ObjectDataCacheStats::default());
let singleflight = ObjectDataCacheSingleflight::new(Arc::clone(&stats));
let first = singleflight.acquire(key()).await;
let second = singleflight.acquire(key()).await;
let leader = match first {
ObjectDataCacheSingleflightAcquire::Leader(leader) => leader,
ObjectDataCacheSingleflightAcquire::Waiter(_) => panic!("first caller must become leader"),
};
let waiter = match second {
ObjectDataCacheSingleflightAcquire::Leader(_) => panic!("second caller must become waiter"),
ObjectDataCacheSingleflightAcquire::Waiter(waiter) => waiter,
};
let waiter_task = tokio::spawn(async move { waiter.wait().await });
let leader_result = leader.finish(ObjectDataCacheFillResult::Inserted).await;
let waiter_result = waiter_task.await.expect("waiter task should complete");
assert_eq!(leader_result, ObjectDataCacheFillResult::Inserted);
assert_eq!(waiter_result, ObjectDataCacheFillResult::Inserted);
assert_eq!(stats.snapshot().singleflight_joins, 1);
}
#[tokio::test]
async fn cancelled_leader_releases_key_and_unblocks_waiters() {
let stats = Arc::new(ObjectDataCacheStats::default());
let singleflight = ObjectDataCacheSingleflight::new(Arc::clone(&stats));
let leader = match singleflight.acquire(key()).await {
ObjectDataCacheSingleflightAcquire::Leader(leader) => leader,
ObjectDataCacheSingleflightAcquire::Waiter(_) => panic!("first caller must become leader"),
};
let waiter = match singleflight.acquire(key()).await {
ObjectDataCacheSingleflightAcquire::Leader(_) => panic!("second caller must become waiter"),
ObjectDataCacheSingleflightAcquire::Waiter(waiter) => waiter,
};
// Dropping without finish() simulates the leader future being cancelled.
drop(leader);
let waiter_result = waiter.wait().await;
assert_eq!(waiter_result, ObjectDataCacheFillResult::SkippedSingleflightClosed);
assert!(
matches!(singleflight.acquire(key()).await, ObjectDataCacheSingleflightAcquire::Leader(_)),
"key must be released after a cancelled leader"
);
}
}
@@ -0,0 +1,215 @@
// Copyright 2024 RustFS Team
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
use crate::index::{ObjectDataCacheIndexInsertResult, ObjectDataCacheKeySet};
use crate::key::{ObjectDataCacheIdentity, ObjectDataCacheKey};
use starshard::{AsyncShardedHashMap, DEFAULT_SHARDS, SnapshotMode};
use std::collections::hash_map::RandomState;
use std::sync::Arc;
/// Async starshard-backed identity -> keys index.
#[derive(Clone)]
pub struct StarshardIdentityIndex {
by_object: Arc<AsyncShardedHashMap<ObjectDataCacheIdentity, ObjectDataCacheKeySet, RandomState>>,
max_keys_per_identity: usize,
}
impl std::fmt::Debug for StarshardIdentityIndex {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("StarshardIdentityIndex")
.field("max_keys_per_identity", &self.max_keys_per_identity)
.finish()
}
}
impl StarshardIdentityIndex {
/// Creates a new identity index.
pub fn new(max_keys_per_identity: usize) -> Self {
Self {
by_object: Arc::new(AsyncShardedHashMap::with_shards_and_hasher_and_snapshot_mode(
DEFAULT_SHARDS,
RandomState::new(),
SnapshotMode::Cached,
)),
max_keys_per_identity,
}
}
/// Inserts a key into the identity index.
///
/// Runs the read-modify-write under the shard write lock so concurrent
/// fills for the same identity cannot drop each other's keys.
pub async fn insert(&self, identity: ObjectDataCacheIdentity, key: ObjectDataCacheKey) -> ObjectDataCacheIndexInsertResult {
let max_keys = self.max_keys_per_identity;
loop {
let mut outcome = None;
{
let outcome = &mut outcome;
let key = key.clone();
let _ = self
.by_object
.compute_if_present(&identity, move |mut key_set| {
let result = key_set.insert(key, max_keys);
let keep = !matches!(result, ObjectDataCacheIndexInsertResult::Overflow { .. }) && !key_set.is_empty();
*outcome = Some(result);
keep.then_some(key_set)
})
.await;
}
if let Some(result) = outcome {
return result;
}
// Identity not tracked yet: publish a fresh single-key set.
let mut fresh = ObjectDataCacheKeySet::default();
let result = fresh.insert(key.clone(), max_keys);
if !matches!(result, ObjectDataCacheIndexInsertResult::Inserted) {
return result;
}
let final_set = self.by_object.compute_if_absent(identity.clone(), move || fresh).await;
if final_set.contains(&key) {
return ObjectDataCacheIndexInsertResult::Inserted;
}
// Lost the race to a concurrent insert; retry against the now
// present entry.
}
}
/// Removes all keys tracked for an identity.
pub async fn remove_identity(&self, identity: &ObjectDataCacheIdentity) -> Vec<ObjectDataCacheKey> {
self.by_object
.remove(identity)
.await
.map_or_else(Vec::new, |set| set.cloned())
}
/// Removes a single key tracked under an identity.
pub async fn remove_key(&self, identity: &ObjectDataCacheIdentity, key: &ObjectDataCacheKey) -> bool {
let mut removed = false;
{
let removed = &mut removed;
let _ = self
.by_object
.compute_if_present(identity, move |mut key_set| {
*removed = key_set.remove_key(key);
(!key_set.is_empty()).then_some(key_set)
})
.await;
}
removed
}
/// Returns whether the identity currently tracks the supplied key.
pub async fn contains_key(&self, identity: &ObjectDataCacheIdentity, key: &ObjectDataCacheKey) -> bool {
self.by_object
.get(identity)
.await
.is_some_and(|key_set| key_set.contains(key))
}
/// Removes index keys that no longer exist in the cache.
pub async fn prune_missing<F>(&self, identity: &ObjectDataCacheIdentity, mut key_exists: F)
where
F: FnMut(&ObjectDataCacheKey) -> bool,
{
let _ = self
.by_object
.compute_if_present(identity, move |mut key_set| {
key_set.retain(|key| key_exists(key));
(!key_set.is_empty()).then_some(key_set)
})
.await;
}
}
#[cfg(test)]
mod tests {
use super::StarshardIdentityIndex;
use crate::index::ObjectDataCacheIndexInsertResult;
use crate::key::{ObjectDataCacheBodyVariant, ObjectDataCacheIdentity, ObjectDataCacheKey};
fn identity() -> ObjectDataCacheIdentity {
ObjectDataCacheIdentity::new("bucket", "object")
}
fn key(id: &str) -> ObjectDataCacheKey {
ObjectDataCacheKey::new("bucket", "object", Some(id), "etag", 1, ObjectDataCacheBodyVariant::FullObjectPlainV1)
}
#[tokio::test]
async fn identity_index_removes_all_keys_for_identity() {
let index = StarshardIdentityIndex::new(4);
let identity = identity();
let key_a = key("v1");
let key_b = key("v2");
let _ = index.insert(identity.clone(), key_a.clone()).await;
let _ = index.insert(identity.clone(), key_b.clone()).await;
let removed = index.remove_identity(&identity).await;
assert_eq!(removed, vec![key_a, key_b]);
}
#[tokio::test]
async fn identity_index_prunes_stale_keys() {
let index = StarshardIdentityIndex::new(4);
let identity = identity();
let key_a = key("v1");
let key_b = key("v2");
let _ = index.insert(identity.clone(), key_a.clone()).await;
let _ = index.insert(identity.clone(), key_b.clone()).await;
index.prune_missing(&identity, |candidate| candidate == &key_b).await;
let removed = index.remove_identity(&identity).await;
assert_eq!(removed, vec![key_b]);
}
#[tokio::test]
async fn identity_index_concurrent_inserts_keep_all_keys() {
let index = StarshardIdentityIndex::new(64);
let identity = identity();
let mut handles = Vec::new();
for i in 0..32 {
let index = index.clone();
let identity = identity.clone();
handles.push(tokio::spawn(async move { index.insert(identity, key(&format!("v{i}"))).await }));
}
for handle in handles {
handle.await.expect("insert task should complete");
}
let removed = index.remove_identity(&identity).await;
assert_eq!(removed.len(), 32, "no concurrent insert may drop another fill's key");
}
#[tokio::test]
async fn identity_index_overflow_clears_identity() {
let index = StarshardIdentityIndex::new(1);
let identity = identity();
let key_a = key("v1");
let key_b = key("v2");
let _ = index.insert(identity.clone(), key_a.clone()).await;
let result = index.insert(identity.clone(), key_b).await;
let removed = index.remove_identity(&identity).await;
assert!(matches!(
result,
ObjectDataCacheIndexInsertResult::Overflow { cleared_keys } if cleared_keys == vec![key_a]
));
assert!(removed.is_empty());
}
}
+133
View File
@@ -0,0 +1,133 @@
// Copyright 2024 RustFS Team
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
use std::sync::atomic::{AtomicU64, Ordering};
/// Cache statistics holder for the engine skeleton.
#[derive(Debug, Default)]
pub struct ObjectDataCacheStats {
entries: AtomicU64,
lookups: AtomicU64,
hits: AtomicU64,
fills: AtomicU64,
invalidations: AtomicU64,
inflight_fills: AtomicU64,
singleflight_joins: AtomicU64,
memory_pressure_events: AtomicU64,
}
/// Immutable snapshot of cache statistics.
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)]
pub struct ObjectDataCacheStatsSnapshot {
/// Number of cached entries.
pub entries: u64,
/// Total lookup attempts.
pub lookups: u64,
/// Total cache hits.
pub hits: u64,
/// Total fill attempts.
pub fills: u64,
/// Total invalidation attempts.
pub invalidations: u64,
/// Current number of in-flight fills.
pub inflight_fills: u64,
/// Number of singleflight joiners.
pub singleflight_joins: u64,
/// Number of times fill was skipped under memory pressure.
pub memory_pressure_events: u64,
}
impl ObjectDataCacheStats {
/// Updates the current entry count snapshot.
pub fn set_entries(&self, entries: u64) {
self.entries.store(entries, Ordering::Relaxed);
}
/// Records a cache lookup attempt.
pub fn record_lookup(&self, hit: bool) {
self.lookups.fetch_add(1, Ordering::Relaxed);
if hit {
self.hits.fetch_add(1, Ordering::Relaxed);
}
}
/// Records a cache fill attempt.
pub fn record_fill(&self) {
self.fills.fetch_add(1, Ordering::Relaxed);
}
/// Records a cache invalidation attempt.
pub fn record_invalidation(&self) {
self.invalidations.fetch_add(1, Ordering::Relaxed);
}
/// Sets the current number of in-flight fills.
pub fn set_inflight_fills(&self, inflight_fills: usize) {
let value = u64::try_from(inflight_fills).unwrap_or(u64::MAX);
self.inflight_fills.store(value, Ordering::Relaxed);
}
/// Records a singleflight join.
pub fn record_singleflight_join(&self) {
self.singleflight_joins.fetch_add(1, Ordering::Relaxed);
}
/// Records a memory pressure event.
pub fn record_memory_pressure(&self) {
self.memory_pressure_events.fetch_add(1, Ordering::Relaxed);
}
/// Returns the current immutable stats snapshot.
pub fn snapshot(&self) -> ObjectDataCacheStatsSnapshot {
ObjectDataCacheStatsSnapshot {
entries: self.entries.load(Ordering::Relaxed),
lookups: self.lookups.load(Ordering::Relaxed),
hits: self.hits.load(Ordering::Relaxed),
fills: self.fills.load(Ordering::Relaxed),
invalidations: self.invalidations.load(Ordering::Relaxed),
inflight_fills: self.inflight_fills.load(Ordering::Relaxed),
singleflight_joins: self.singleflight_joins.load(Ordering::Relaxed),
memory_pressure_events: self.memory_pressure_events.load(Ordering::Relaxed),
}
}
}
#[cfg(test)]
mod tests {
use super::ObjectDataCacheStats;
#[test]
fn snapshot_reflects_recorded_counters() {
let stats = ObjectDataCacheStats::default();
stats.set_entries(3);
stats.record_lookup(true);
stats.record_lookup(false);
stats.record_fill();
stats.record_invalidation();
stats.set_inflight_fills(2);
stats.record_singleflight_join();
stats.record_memory_pressure();
let snapshot = stats.snapshot();
assert_eq!(snapshot.entries, 3);
assert_eq!(snapshot.lookups, 2);
assert_eq!(snapshot.hits, 1);
assert_eq!(snapshot.fills, 1);
assert_eq!(snapshot.invalidations, 1);
assert_eq!(snapshot.inflight_fills, 2);
assert_eq!(snapshot.singleflight_joins, 1);
assert_eq!(snapshot.memory_pressure_events, 1);
}
}