gsadmin/rustfs
1
0
Fork 0
mirror of https://github.com/rustfs/rustfs.git synced 2026-08-10 23:26:53 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat(object-data-cache): close write-side invalidation gaps and add an admin surface (#4694)

Browse Source 
* feat(object-data-cache): close write/delete-side invalidation gaps

The object data cache exposed only a single per-(bucket,object)
invalidation primitive and no write-side ecstore hook, so several
delete paths left dead bodies resident until TTL (hygiene/capacity, not
stale-serving: lookups follow a fresh metadata quorum and cannot serve a
gone object). This adds the missing primitives and wires them in.

ODC-26 (backlog#1131): add an `ObjectMutationHook` trait beside the GET
body hook, registered next to it at startup, and call it from the
ecstore-internal delete paths (`apply_expiry_on_non_transitioned_objects`,
`expire_transitioned_object` including the restored-copy branch, and
`delete_object_versions`). The app impl is one `invalidate_object` call
under a new `AfterLifecycleExpiry` reason.

ODC-27 (backlog#1132): force prefix delete now invalidates the whole
prefix, not just the prefix string. `store.delete_object(delete_prefix)`
returns no deleted-name list, so this uses a new prefix primitive rather
than the batch path.

ODC-28 (backlog#1133): DeleteBucket now flushes the bucket via a new
bucket-scope primitive (covers force and non-force, which share the
delete_bucket call).

ODC-C2 (backlog#1143): add `ObjectDataCache::clear()` and two admin
handlers (GET stats, POST flush) routed through admin runtime_sources.

The starshard identity index gains a single `remove_matching` full-scan
API backing prefix/bucket/clear; it is documented as admin/delete-path
only and never runs on the GET or fill hot path. New invalidation
reasons and metric labels added; outcome (removed/noop) labelling kept
correct for every new primitive.

Also fixes a pre-existing broken intra-doc link in memory.rs.

Co-Authored-By: heihutu <heihutu@gmail.com>

* refactor(ecstore): extract the shared HookSlot behind both cache hooks

This PR introduced object_mutation_hook.rs by mirroring body_cache_hook.rs,
which left two process-global registration slots whose register/get/clear
bodies were line-for-line identical except the trait type and the WARN string:
a RwLock<Option<Arc<dyn _>>>, an Arc::ptr_eq "different instance" warning, the
poison-recovery closure, and the same read-lock-and-clone read. Two copies of
the same swap-vs-warn logic can drift apart under maintenance.

Hoist it into a generic HookSlot<T: ?Sized> that owns the logic once. Each hook
module keeps its `static HOOK: HookSlot<dyn XxxHook>` and its thin, unchanged
public wrappers (register_/get_/clear_), so the crate's public surface and
every call site are untouched — this is an internal consolidation, not a
contract change.

The load-bearing #1126 guarantee (newest registration wins, so a rebuilt
AppContext is never stranded on a first-wins slot) previously had no direct
test — the hook tests only covered register-then-notify. HookSlot now has its
own unit tests including re_registration_swaps_to_the_latest_instance;
mutation-testing confirms a first-wins regression fails exactly that test.

No behavior change: the two hooks' existing tests, the P0 body_cache_hook_e2e
regressions, and the app-layer mutation-hook tests all pass unchanged.

Refs: backlog#1126, backlog#1131

Co-Authored-By: heihutu <heihutu@gmail.com>

* fix(admin): register the object-data-cache routes in the policy inventory

This PR added GET /object-data-cache/stats and POST /object-data-cache/flush
but did not list them in the two registries that must account for every
admin route: the route-policy inventory (route_policy.rs) and the route
matrix (route_registration_test.rs). Their coverage tests —
route_policy_inventory_covers_registered_routes and
test_admin_route_matrix_matches_registered_routes — failed on CI because a
registered route had no policy/matrix entry.

These two tests are not part of `make pre-commit` (which runs fmt + arch +
quick-check, not the full suite), so the gap passed local pre-commit and
only surfaced in the CI Test-and-Lint lane.

stats is a read (ServerInfoAdminAction, Sensitive); flush mutates
(ConfigUpdateAdminAction, High) — matching the actions the handlers already
enforce. The MinIO-alias matrix test is unaffected: these are native rustfs
endpoints with no MinIO equivalent.

Refs: backlog#1143

Co-Authored-By: heihutu <heihutu@gmail.com>

---------

Co-authored-by: heihutu <heihutu@gmail.com>
This commit is contained in:
houseme
2026-07-11 04:04:05 +08:00
committed by GitHub
parent f9874b591a
commit 85fd824581
28 changed files with 1173 additions and 53 deletions
Download Patch File Download Diff File Expand all files Collapse all files
crates/ecstore/src/object_api/body_cache_hook.rs
+18 -25
View File
@@ -21,8 +21,9 @@
//! (after the reader is built) means a hit no longer saves any disk I/O.
use crate::object_api::ObjectInfo;
use crate::object_api::hook_slot::HookSlot;
use bytes::Bytes;
use std::sync::{Arc, RwLock};
use std::sync::Arc;
/// Serves full-object GET bodies from a cache keyed by object identity.
///
@@ -35,14 +36,15 @@ pub trait GetObjectBodyCacheHook: Send + Sync + 'static {
async fn lookup(&self, bucket: &str, object: &str, info: &ObjectInfo) -> Option<Bytes>;
}
// `RwLock<Option<Arc<dyn ...>>>` rather than `ArcSwapOption`: arc-swap's
// `RefCnt` is implemented only for the sized `Arc<T>` (it stores a thin
// `*mut T`), so it cannot hold an `Arc<dyn GetObjectBodyCacheHook>` without a
// sized newtype wrapper. The probe reads this slot once per full-object GET,
// but the read guard only clones an `Arc`, which is negligible next to the
// metadata quorum fan-out already completed before the probe. Registration is
// a startup / config-reload event, so writer contention is a non-issue.
static GET_OBJECT_BODY_CACHE_HOOK: RwLock<Option<Arc<dyn GetObjectBodyCacheHook>>> = RwLock::new(None);
// A `HookSlot` (RwLock<Option<Arc<dyn ...>>>) rather than `ArcSwapOption`:
// arc-swap's `RefCnt` is implemented only for the sized `Arc<T>` (it stores a
// thin `*mut T`), so it cannot hold an `Arc<dyn GetObjectBodyCacheHook>`
// without a sized newtype wrapper. The probe reads this slot once per
// full-object GET, but the read guard only clones an `Arc`, which is negligible
// next to the metadata quorum fan-out already completed before the probe.
// Registration is a startup / config-reload event, so writer contention is a
// non-issue.
static GET_OBJECT_BODY_CACHE_HOOK: HookSlot<dyn GetObjectBodyCacheHook> = HookSlot::new();
/// Register (or re-register) the process-wide GET body cache hook.
///
@@ -52,31 +54,22 @@ static GET_OBJECT_BODY_CACHE_HOOK: RwLock<Option<Arc<dyn GetObjectBodyCacheHook>
/// to the original adapter while every usecase-layer fill and invalidation
/// targeted the replacement, silently degrading the feature to a 0% hit rate
/// and stranding entries in the unreachable cache until their TTL (backlog#1126).
///
/// Replacing a *different* hook instance is logged at WARN: in production the
/// hook is installed exactly once per process, so a swap to a distinct instance
/// signals an unexpected re-init and orphans the previous adapter's cache.
pub fn register_get_object_body_cache_hook(hook: Arc<dyn GetObjectBodyCacheHook>) {
let mut slot = GET_OBJECT_BODY_CACHE_HOOK.write().unwrap_or_else(|e| e.into_inner());
if let Some(previous) = slot.as_ref()
&& !Arc::ptr_eq(previous, &hook)
{
tracing::warn!(
"GET object body cache hook re-registered with a different instance; \
the previous adapter's cache is now unreachable by ecstore's GET probe"
);
}
*slot = Some(hook);
GET_OBJECT_BODY_CACHE_HOOK.register(
hook,
"GET object body cache hook re-registered with a different instance; \
the previous adapter's cache is now unreachable by ecstore's GET probe",
);
}
/// The registered hook, if any.
pub(crate) fn get_object_body_cache_hook() -> Option<Arc<dyn GetObjectBodyCacheHook>> {
GET_OBJECT_BODY_CACHE_HOOK.read().unwrap_or_else(|e| e.into_inner()).clone()
GET_OBJECT_BODY_CACHE_HOOK.get()
}
/// Test-only: unregister the hook so tests can register and clear the slot
/// deterministically without leaking a hook into unrelated tests.
#[cfg(test)]
pub(crate) fn clear_get_object_body_cache_hook() {
*GET_OBJECT_BODY_CACHE_HOOK.write().unwrap_or_else(|e| e.into_inner()) = None;
GET_OBJECT_BODY_CACHE_HOOK.clear();
}