mirror of
https://github.com/rustfs/rustfs.git
synced 2026-08-06 05:17:42 +00:00
Zhengchao An
92402a3bde
perf(get,heal): land verified fixes for backlog #800-#804 Five fixes from the GET-path performance audit and scanner/heal completeness audit (rustfs/backlog#800..#804), each verified locally: - backlog#800 (heal checkpoint O(N^2)): ResumeCheckpoint object sets are now HashSet (Vec::contains was O(n) per healed object; 1.5ms at n=1M vs 11ns measured), and per-object checkpoint/resume-state persistence is batched (1000 mutations / 5s) instead of rewriting the whole file per object. complete_page() prunes the sets at page boundaries so memory stays bounded; positions still persist unconditionally and legacy Vec-format checkpoints still deserialize. - backlog#801 (DiskInfo.healing never set): erasure-set heal now writes a healing marker (.rustfs.sys/healing.bin) on the disks it rebuilds (endpoints plumbed via HealRequest/HealTask.heal_endpoints from the auto disk scanner) and clears it on success. LocalDisk::disk_info surfaces the marker, so scanner heal coordination, lock selection and admin/metrics healing counts see the rebuild. - backlog#802 (cache probe after data read): new GetObjectBodyCacheHook in ecstore lets the app-layer object data cache serve the body inside get_object_reader, after metadata quorum resolution (etag known) but before the erasure shard read/decode. Previously a hit still paid the full disk read. Hook is None/no-op when the cache is disabled. - backlog#803 (GET hot-path redundant work): ObjectInfo is cloned for event notification only when an event will actually be built (GET and HEAD paths; events are currently suppressed so the clone was pure waste); get_opts/put_opts/del_opts resolve bucket versioning with one metadata-sys lookup instead of two; skip_verify_bitrot and get_lock_acquire_timeout env reads are cached via OnceLock; the io-priority metric is no longer double-counted; GetObject input fields are cloned selectively instead of cloning the whole input. - backlog#804 (disk permit starvation): the disk-read permit wait is now bounded (RUSTFS_OBJECT_DISK_PERMIT_WAIT_TIMEOUT, default 5s, 0 = previous unbounded behavior); on timeout the GET proceeds without a permit and the bypass is counted. DiskReadPermitReader also releases the permit at body EOF instead of holding it until the client drops the stream. Verification: make pre-commit; cargo clippy -D warnings on the four changed crates; full rustfs lib suite (2096 tests) green; rustfs-heal lib suite green with new unit tests for checkpoint pruning/legacy format/throttle, permit EOF release, and the cache hook (hit + SSE skip). The heal_integration_test and one set_disk listing test fail identically on unmodified main (pre-existing global-state ordering flakes, verified via git stash A/B). Co-authored-by: houseme <housemecn@gmail.com>
50 lines
2.2 KiB
Rust
50 lines
2.2 KiB
Rust
// Copyright 2024 RustFS Team
|
|
//
|
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
|
// you may not use this file except in compliance with the License.
|
|
// You may obtain a copy of the License at
|
|
//
|
|
// http://www.apache.org/licenses/LICENSE-2.0
|
|
//
|
|
// Unless required by applicable law or agreed to in writing, software
|
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
// See the License for the specific language governing permissions and
|
|
// limitations under the License.
|
|
|
|
//! Hook that lets the application layer serve a GET body from its object data
|
|
//! cache after metadata resolution but before the erasure data read.
|
|
//!
|
|
//! The cache itself lives above ecstore (it needs app-level config, metrics
|
|
//! and invalidation), but the lookup must happen inside `get_object_reader`:
|
|
//! probing earlier would require a second metadata fan-out, and probing later
|
|
//! (after the reader is built) means a hit no longer saves any disk I/O.
|
|
|
|
use crate::object_api::ObjectInfo;
|
|
use bytes::Bytes;
|
|
use std::sync::{Arc, OnceLock};
|
|
|
|
/// Serves full-object GET bodies from a cache keyed by object identity.
|
|
///
|
|
/// Implementations must validate identity (etag/version/size) against the
|
|
/// provided `ObjectInfo`, which reflects the just-resolved metadata quorum,
|
|
/// and must return `None` for anything they cannot serve byte-identically
|
|
/// (encrypted objects, remote/transitioned objects, size mismatches, ...).
|
|
#[async_trait::async_trait]
|
|
pub trait GetObjectBodyCacheHook: Send + Sync + 'static {
|
|
async fn lookup(&self, bucket: &str, object: &str, info: &ObjectInfo) -> Option<Bytes>;
|
|
}
|
|
|
|
static GET_OBJECT_BODY_CACHE_HOOK: OnceLock<Arc<dyn GetObjectBodyCacheHook>> = OnceLock::new();
|
|
|
|
/// Register the process-wide GET body cache hook. First registration wins;
|
|
/// later calls are ignored so tests and re-inits cannot swap the hook midway.
|
|
pub fn register_get_object_body_cache_hook(hook: Arc<dyn GetObjectBodyCacheHook>) {
|
|
let _ = GET_OBJECT_BODY_CACHE_HOOK.set(hook);
|
|
}
|
|
|
|
/// The registered hook, if any.
|
|
pub(crate) fn get_object_body_cache_hook() -> Option<&'static Arc<dyn GetObjectBodyCacheHook>> {
|
|
GET_OBJECT_BODY_CACHE_HOOK.get()
|
|
}
|