mirror of
https://github.com/rustfs/rustfs.git
synced 2026-07-26 08:18:18 +00:00
houseme
8039a4ceae
fix(ecstore): make body-cache hook re-registrable + add e2e regressions ODC-21 (backlog#1126): the GET body-cache hook lived in a first-wins OnceLock. When AppContext is rebuilt (config reload, test re-init) a fresh ObjectDataCacheAdapter is constructed and re-registered, but the OnceLock kept ecstore's GET probe pointed at adapter #1 while every usecase-layer fill and invalidation targeted adapter #2 — silently degrading the feature to a 0% hit rate with no error, log, or metric, and stranding entries in the unreachable cache until their TTL. Replace the slot with RwLock<Option<Arc<dyn GetObjectBodyCacheHook>>> so re-registration atomically swaps to the newest adapter, and log at WARN when a swap replaces a *different* instance (Arc::ptr_eq). RwLock over ArcSwapOption because arc-swap's RefCnt is impl<T> (Sized, thin *mut T) and cannot hold an Arc<dyn Trait> without a sized newtype wrapper; the probe reads the slot once per full-object GET but only clones an Arc, negligible next to the metadata quorum fan-out already done before the probe. Add a test-only clear_get_object_body_cache_hook so tests register/unregister deterministically. With the hook now re-registrable, add true end-to-end regressions that drive get_object_reader (not the full_object_plaintext_len predicate) against a real erasure-coded, genuinely-compressed object via the blackbox make_local_set_disks harness, with a stand-in hook playing the app-layer cache (the injection point production uses; the adapter itself lives above ecstore). These close the gap the predicate-only tests left — a caller that opens a new shortcut serving the cached body directly, the original form of both P0s: - backlog#1108: a raw_data_movement_read must yield the STORED (compressed) bytes, never the cached plaintext. - backlog#1109: a compressed cache hit must publish the DECOMPRESSED length as object_info.size (the UploadPartCopy invariant), with the streamed length matching. - backlog#1146: a restore read (restore_request.days) must serve STORED bytes, not the cache. Mutation-verified each e2e test bites: dropping the raw_data_movement_read gate serves plaintext (fails #1108); removing the hit-site size republication publishes 2972 vs 660000 (fails #1109); dropping the restore gate serves plaintext (fails #1146). Co-authored-by: heihutu <heihutu@gmail.com>
65 lines
2.5 KiB
Rust
65 lines
2.5 KiB
Rust
// Copyright 2024 RustFS Team
|
|
//
|
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
|
// you may not use this file except in compliance with the License.
|
|
// You may obtain a copy of the License at
|
|
//
|
|
// http://www.apache.org/licenses/LICENSE-2.0
|
|
//
|
|
// Unless required by applicable law or agreed to in writing, software
|
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
// See the License for the specific language governing permissions and
|
|
// limitations under the License.
|
|
|
|
// #730: object API readers keep staged compatibility paths during facade migration.
|
|
#![allow(dead_code)]
|
|
|
|
use crate::bucket::metadata_sys::get_versioning_config;
|
|
use crate::bucket::replication::{
|
|
ReplicateDecision, ReplicationState, ReplicationStatusType, VersionPurgeStatusType, replication_status_from_filemeta,
|
|
replication_statuses_map, version_purge_status_from_filemeta, version_purge_statuses_map,
|
|
};
|
|
use crate::bucket::versioning::VersioningApi as _;
|
|
use crate::config::storageclass;
|
|
use crate::error::{Error, Result};
|
|
use crate::io_support::rio::{HashReader, LimitReader};
|
|
use crate::storage_api_contracts::{
|
|
lifecycle::{ExpirationOptions, TransitionedObject},
|
|
range::HTTPRangeSpec,
|
|
};
|
|
use crate::store::utils::clean_metadata;
|
|
use crate::{bucket::lifecycle::bucket_lifecycle_audit::LcAuditEvent, bucket::lifecycle::lifecycle::TransitionOptions};
|
|
use bytes::Bytes;
|
|
use http::{HeaderMap, HeaderValue};
|
|
use rustfs_filemeta::{FileInfo, MetaCacheEntriesSorted, ObjectPartInfo, RestoreStatusOps as _, parse_restore_obj_status};
|
|
use rustfs_rio::Checksum;
|
|
use rustfs_utils::CompressionAlgorithm;
|
|
use rustfs_utils::http::headers::AMZ_OBJECT_TAGGING;
|
|
use rustfs_utils::http::{AMZ_BUCKET_REPLICATION_STATUS, AMZ_RESTORE, AMZ_STORAGE_CLASS};
|
|
use rustfs_utils::path::decode_dir_object;
|
|
use std::collections::HashMap;
|
|
use std::fmt::Debug;
|
|
use std::io::Cursor;
|
|
use std::pin::Pin;
|
|
use std::sync::Arc;
|
|
use std::task::{Context, Poll};
|
|
use time::OffsetDateTime;
|
|
use tokio::io::{AsyncRead, AsyncReadExt, ReadBuf};
|
|
use tracing::warn;
|
|
use uuid::Uuid;
|
|
|
|
pub const ERASURE_ALGORITHM: &str = "rs-vandermonde";
|
|
pub const BLOCK_SIZE_V2: usize = 1024 * 1024; // 1M
|
|
|
|
mod body_cache_hook;
|
|
mod readers;
|
|
mod types;
|
|
|
|
#[cfg(test)]
|
|
pub(crate) use body_cache_hook::clear_get_object_body_cache_hook;
|
|
pub(crate) use body_cache_hook::get_object_body_cache_hook;
|
|
pub use body_cache_hook::{GetObjectBodyCacheHook, register_get_object_body_cache_hook};
|
|
pub use readers::*;
|
|
pub use types::*;
|