Compare commits

...

35 Commits

Author SHA1 Message Date
overtrue 28cc45be87 chore: merge origin/main into feat/dynamic-block-size-layouts-v2
Picks up main through 77ce76fde, including the #9353 revert. No conflicts.
2026-11-13 19:08:31 +08:00
overtrue 58eca99ca7 chore: merge origin/main into feat/dynamic-block-size-layouts-v2
Picks up c83f47871 (box the legacy GET producer read pipeline). No
conflicts.
2026-11-13 14:08:03 +08:00
overtrue 7a43230f36 test(ecstore): pin the exact key set of a replication status write-back
The default-layout write-back test now asserts the precise keys main adds
(the status keys, X-Amz-Replication-Status and the data-moved markers)
instead of a subset check.
2026-11-13 12:08:42 +08:00
overtrue c2feca6794 fix(ecstore): split bounded frames on the coalesced shard write path
The encoder writes shards through BitrotWriterWrapper::write_shard_bytes
since #9346. That path wrote a shard as one bitrot frame, so a bounded read
quantum (frame smaller than the shard) failed with "data size exceeds shard
size" on every non-inline write. The defect exists only on the tree that
merges this branch with main, and only with RUSTFS_EC_READ_QUANTUM_ENABLE on;
the default layout never selects a bounded frame.

Shards larger than the frame now go through the same frame splitting as
write(). Protected writes never select a bounded quantum; a shard-integrity
spec together with bounded frames is refused instead of dropping the digest.

Red before, green after: bounded_quantum_heal_restores_exact_frames_and_range_bytes,
bounded_quantum_shards_pass_disk_verification_and_deep_heal,
version_only_copy_preserves_quantized_shared_data.
2026-11-13 12:08:41 +08:00
overtrue 0cf1449af9 chore: merge origin/main into feat/dynamic-block-size-layouts-v2
Picks up #9304 (erasure read path for large GETs).

Conflicts:
- crates/ecstore/src/api/mod.rs: export list union.
- crates/ecstore/src/disk/local.rs: FileCacheReclaimReader::new takes
  main's read_ahead argument and keeps the branch's set_max_buf_size; the
  branch's large-frame test passes read_ahead 0.
- crates/ecstore/src/erasure/coding/bitrot.rs: read_appending keeps the
  branch's rollback on error and main's arena stage metrics.

Semantic follow-ups of main's ShardBuf shard buffers:
- read_quantized_appending is generic over ShardAppend like its caller.
- the quantized hedged range read wraps its fallback Vec in ShardBuf.
2026-11-13 11:24:17 +08:00
overtrue 7b61e59dfa fix(ecstore): keep the EC block size projection out of user_defined
ObjectInfo::from_file_info_parts projected the persisted erasure block size
into user_defined as x-rustfs-internal-ec-block-size and
x-minio-internal-ec-block-size for every local version, including objects
written with the default layout. Writers that turn user_defined back into
stored metadata then persisted both keys: a metadata-replacing copy_object
(set_disk/ops/object.rs replacement_metadata) and put_object_metadata, used
by the replication status write-back. The same-version restore metadata
update also stopped matching its protected-metadata check, which made
restore_metadata_copy_preserves_tags_expiry_and_version_order fail.

The projection now lives in ObjectInfo::ec_block_size, read through
effective_ec_block_size(), so response headers are unchanged and stored
metadata keys stay what main writes. The put_object and new_multipart_upload
strips stay: internal PUT origins can still supply internal keys verbatim.

Changed assertions:
- from_file_info_projects_persisted_ec_block_size_for_responses and
  from_owned_file_info_preserves_restored_ec_block_size_projection asserted
  the key inside user_defined; they now assert the accessor and that neither
  spelling is in user_defined.

Tests added:
- default_layout_metadata_copy_persists_no_ec_block_size_projection
- default_layout_replication_status_writeback_persists_no_ec_block_size_projection
2026-11-13 10:33:18 +08:00
overtrue 5b96e7ba34 test(ecstore): satisfy needless_range_loop in deferred shard open test
Rust 1.100 clippy rejects indexing setup.attempted with the loop variable.
Iterate the same two entries instead; the assertions are unchanged.
2026-11-13 10:19:59 +08:00
overtrue 14156e5be7 chore: merge origin/main into feat/dynamic-block-size-layouts-v2
Conflicts:
- crates/ecstore/src/erasure/coding/bitrot.rs: keep the branch's read
  dispatch counter in fill() and main's throttled read-error log.
- crates/ecstore/src/set_disk/ops/object.rs: import list union (main's
  SUFFIX_RESTORE_COPYBACK, the branch's frame-size reader builders).
- rustfs/src/app/multipart_usecase.rs: main split the use case per
  operation; the branch's CreateMultipartUpload block-size hint and
  CompleteMultipartUpload effective block-size header move into
  multipart_usecase/create.rs and complete.rs unchanged.
2026-11-13 09:24:59 +08:00
marshawcoco 963556e14c test(ecstore): align read_appending rollback assertions on error 2026-11-11 12:44:28 +08:00
marshawcoco 77d4b7c3ff chore: sync with 1.0.5-preview.1 (34e6449ed) 2026-11-11 11:47:21 +08:00
marshawcoco d0ae2b87a8 fix(ecstore): run delete-recovery discovery off the metadata lane
Bucket-wide delete-recovery discovery starts with the server and walks the
whole bucket tree in pages of up to 4096 entries, one lstat per entry. Each
page ran on the per-disk namespace queue's foreground lane, which has a
single worker and also serves GET metadata reads. On a cold tree a page
holds that worker for hundreds of milliseconds, so every GET on the disk
waits behind it until the walk finishes.

Run the page on the queue's sync lane instead, as the delete-transaction
inspection already does for the same reason. The namespace lease and the
page bounds are unchanged.

The regression test occupies the foreground lane and requires a discovery
page to complete within ten seconds; it times out without this change.
2026-11-10 16:27:43 +08:00
marshawcoco a9c897a986 chore(ecstore): remove unused schedule_bitrot_reader_task wrapper 2026-11-10 14:30:31 +08:00
marshawcoco d1cfe2e91e perf(ecstore): enable direct-memory GET and coalesce cross-shard range reads for bounded-quantum blocks 2026-11-10 13:39:48 +08:00
marshawcoco c4719563bf fix(ecstore): preserve legacy full-shard framing for default 1 MiB layout 2026-11-10 12:18:11 +08:00
marshawcoco 5d13e43135 chore: merge upstream/main into feat/dynamic-block-size-layouts-v2 2026-11-10 12:06:49 +08:00
Chris An 34e6449ed9 revert(release): defer certification rollout for 1.0.5 (#9254)
Revert e5563e6004 only on release/1.0.5. Its trusted verifier workflow and immutable-candidate integration are not available yet. Restore the previous publication path under the approved release policy; retain the completed product regression and preview acceptance requirements.
2026-11-09 22:44:06 -05:00
marshawcoco 34fea3fbe4 perf(ecstore): use half-shard read quantum at 1 MiB EC layout 2026-11-09 11:13:13 +08:00
marshawcoco 2809f5a999 perf(ecstore): read local batch windows with an exact readahead hint
Kernel readahead sizes its first request from the read size. A cold
16 KiB window at offset 0 pulled 64-128 KiB, and a single 2 MiB read was
split into serial readahead chunks that cached almost twice the window,
which is why neither the single positioned read nor growing sequential
reads won for every window size on block storage.

Announce each window with POSIX_FADV_WILLNEED, which submits exactly the
window at once, and then fill the exact spare capacity with positioned
reads. This replaces the size-based choice between the two read methods.
The hint is advisory and its failure does not fail the read. A residency
test checks that cold reads cache only their window; it failed with the
hint disabled (a 5-page window cached 32 pages).
2026-11-01 22:11:17 +08:00
marshawcoco ff001942c7 perf(ecstore): choose the batch read method by window size
Measured on ESSD-backed cloud storage, a cold window of about 1 MiB is
fastest as one positioned read, which completes as a single device
request, while windows of 2 MiB and more are faster with growing
sequential reads that keep readahead pipelined (one large read waits on
each readahead chunk in turn). Use one positioned read up to 1.5 MiB and
a seek plus length-limited read_to_end above it. Both paths read into
uninitialized spare capacity; per-shard dispatch for large windows is
unchanged.
2026-11-01 20:14:57 +08:00
marshawcoco 149950d01e Revert "perf(ecstore): keep readahead pipelined for local batch windows"
This reverts commit c252fca525.

On the ESSD-backed cloud footprint, the growing read_to_end reads were
slower than one window-sized positioned read for cold windows at nonzero
offsets (cross-block -27% on 4 MiB layouts, cross-shard -21% on 64 KiB
layouts versus the previous batch reader). The local benchmark that
motivated the change ran on a WSL virtual disk and did not represent the
target storage. Keep the single positioned read into spare capacity and
the per-shard dispatch for large windows.
2026-11-01 16:26:35 +08:00
marshawcoco c252fca525 perf(ecstore): keep readahead pipelined for local batch windows
A single window-sized pread on cold pages waits on each kernel readahead
chunk in turn. Seek to the window start and fill the buffer with
read_to_end over a length-limited reader instead: it still reads into
uninitialized spare capacity, but its growing sequential reads keep
readahead pipelined. Warm windows cost the same.
2026-11-01 12:36:22 +08:00
marshawcoco cc64a8fe54 perf(ecstore): avoid zero-fill and serial copies in local batch reads
The local batch reader zero-initialized every window at a nonzero offset
before reading it, and copied every shard serially inside one blocking
task. Two-segment q-window range reads (cross-shard, cross-block) pay
both costs twice per request.

Read each window with a positioned read into uninitialized Vec capacity,
and dispatch each shard separately when every window is at least 512 KiB,
where copies dominate blocking-pool round trips. Smaller windows keep one
coalesced dispatch.
2026-11-01 11:28:39 +08:00
marshawcoco aefc536983 fix(ecstore): preserve typed quantized read errors 2026-10-31 21:56:01 +08:00
marshawcoco 86a3095d3d perf(ecstore): batch q reads and retain shared copy framing 2026-10-31 17:30:39 +08:00
marshawcoco 39b913169f feat(ecstore): integrate dynamic block layouts with mainline 2026-10-31 15:21:02 +08:00
marshawcoco 645220bc62 perf(ecstore): reuse owned quantized frame batches 2026-10-31 09:46:15 +08:00
marshawcoco 3ea712255c fix(ecstore): preserve bounded quantum frames during heal 2026-10-31 09:46:15 +08:00
Xiaoyang Han a87e8f94a5 perf(ecstore): include shard checksums in file buffer cap 2026-10-25 15:41:39 +00:00
Xiaoyang Han 56756650d0 perf(ecstore): compact deferred shard reader state 2026-10-25 15:41:39 +00:00
Xiaoyang Han 948d1d7707 fix(ecstore): avoid prefetching conservative short range tails 2026-10-25 15:41:39 +00:00
Xiaoyang Han c60dec7167 perf(ecstore): reduce dynamic layout read overhead 2026-10-25 15:41:39 +00:00
marshawcoco bd97fdc7d1 chore: checkpoint quantized read metrics draft
Save work in progress for collaboration. Counts omit retired readers and transient or cancelled hedges. Frame semantics, async recorder coverage and disabled-metrics overhead remain unresolved; these counters are not ready for performance conclusions.

Validation: rustfmt and ecstore check passed; io-metrics 120, quantized 4 and hashed-read 3 tests passed. No new live or performance validation.
2026-10-25 15:41:39 +00:00
marshawcoco d443b2ff28 perf: measure quantized range read shape 2026-10-25 15:41:39 +00:00
marshawcoco 314b8bcd5d test(s3): cover dynamic block layouts over HTTP 2026-10-25 15:41:39 +00:00
Xiaoyang Han a6bdb5d819 perf(ecstore): reduce allocations for bounded bitrot frames 2026-10-25 15:41:39 +00:00
35 changed files with 6371 additions and 144 deletions
+27
View File
@@ -152,6 +152,33 @@ pub const DEFAULT_SHARD_INTEGRITY_WRITE: bool = false;
pub const ENV_SHARD_INTEGRITY_FLEET_CONFIRMED: &str = "RUSTFS_SHARD_INTEGRITY_FLEET_CONFIRMED";
pub const DEFAULT_SHARD_INTEGRITY_FLEET_CONFIRMED: bool = false;
/// Request parsing switch for the per-object erasure-code block-size hint.
/// The default keeps the feature inert during rolling upgrades.
pub const ENV_EC_BLOCK_SIZE_HINT_ENABLE: &str = "RUSTFS_EC_BLOCK_SIZE_HINT_ENABLE";
pub const DEFAULT_EC_BLOCK_SIZE_HINT_ENABLE: bool = false;
/// Operator confirmation that every writer and reader in the fleet understands
/// the persisted erasure block-size geometry.
pub const ENV_EC_BLOCK_SIZE_HINT_FLEET_CONFIRMED: &str = "RUSTFS_EC_BLOCK_SIZE_HINT_FLEET_CONFIRMED";
pub const DEFAULT_EC_BLOCK_SIZE_HINT_FLEET_CONFIRMED: bool = false;
/// Write new EC shards with a bounded bitrot frame size so Range GETs do not
/// inherit the full persisted layout block as their physical read quantum.
/// The marker is only emitted after the fleet confirmation gate is enabled;
/// readers without this feature continue to consume legacy objects unchanged.
pub const ENV_EC_READ_QUANTUM_ENABLE: &str = "RUSTFS_EC_READ_QUANTUM_ENABLE";
pub const DEFAULT_EC_READ_QUANTUM_ENABLE: bool = false;
/// Operator confirmation that every reader and writer understands the
/// persisted EC read-quantum marker and its per-frame bitrot layout.
pub const ENV_EC_READ_QUANTUM_FLEET_CONFIRMED: &str = "RUSTFS_EC_READ_QUANTUM_FLEET_CONFIRMED";
pub const DEFAULT_EC_READ_QUANTUM_FLEET_CONFIRMED: bool = false;
const _: () = assert!(!DEFAULT_EC_BLOCK_SIZE_HINT_ENABLE);
const _: () = assert!(!DEFAULT_EC_BLOCK_SIZE_HINT_FLEET_CONFIRMED);
const _: () = assert!(!DEFAULT_EC_READ_QUANTUM_ENABLE);
const _: () = assert!(!DEFAULT_EC_READ_QUANTUM_FLEET_CONFIRMED);
const _: () = assert!(!DEFAULT_SHARD_INTEGRITY_WRITE);
const _: () = assert!(!DEFAULT_SHARD_INTEGRITY_FLEET_CONFIRMED);
+13 -7
View File
@@ -609,15 +609,21 @@ pub mod object {
};
pub use crate::erasure::codec::shard_emit::ShardEmitter;
pub use crate::object_api::{
BLOCK_SIZE_V2, ERASURE_ALGORITHM, EncryptionResolutionError, EncryptionResolutionErrorKind, GetObjectBodyCacheHook,
GetObjectBodyCacheHookLookup, GetObjectBodySource, GetObjectReadCondition, GetObjectReadMetadata, GetObjectReader,
GetObjectSink, GetObjectSinkSegmentFactory, GetObjectSinkSegments, GetObjectSinkStream, GetObjectSinkWriter,
NamespaceLockFence, ObjectEncryptionResolver, ObjectInfo, ObjectLockConfigSnapshot, ObjectMutationHook, ObjectOptions,
PutObjReader, QuotaAdmission, RangedDecompressReader, ReadEncryptionMaterial, ReadEncryptionMode, ReadEncryptionRequest,
BLOCK_SIZE_V2, ClientEcBlockSizeHint, DEFAULT_EC_BLOCK_SIZE, EC_BLOCK_SIZE_CANDIDATES, EC_BLOCK_SIZE_HINT_INTERNAL_KEY,
EC_BLOCK_SIZE_HINT_INTERNAL_SUFFIX, EC_BLOCK_SIZE_HINT_METADATA_KEY, EC_BLOCK_SIZE_HINT_REASON_HEADER,
EC_BLOCK_SIZE_HINT_STATUS_HEADER, EC_BLOCK_SIZE_INFO_INTERNAL_KEY, EC_BLOCK_SIZE_INFO_INTERNAL_SUFFIX,
EC_BLOCK_SIZE_RESPONSE_HEADER, EC_READ_QUANTUM_INFO_INTERNAL_KEY, EC_READ_QUANTUM_INFO_INTERNAL_SUFFIX,
ERASURE_ALGORITHM, EcBlockSizeHintError, EcBlockSizeHintIgnoreReason, EcBlockSizeHintOutcome, EncryptionResolutionError,
EncryptionResolutionErrorKind, GetObjectBodyCacheHook, GetObjectBodyCacheHookLookup, GetObjectBodySource,
GetObjectReadCondition, GetObjectReadMetadata, GetObjectReader, GetObjectSink, GetObjectSinkSegmentFactory,
GetObjectSinkSegments, GetObjectSinkStream, GetObjectSinkWriter, NamespaceLockFence, ObjectEncryptionResolver,
ObjectInfo, ObjectLockConfigSnapshot, ObjectMutationHook, ObjectOptions, PutObjReader, QuotaAdmission,
RangedDecompressReader, ReadEncryptionMaterial, ReadEncryptionMode, ReadEncryptionRequest, ResolvedEcBlockSize,
SCANNER_PUBLICATION_LEASE_FENCE_METADATA_KEY, ScannerPublicationCommitScope, ScannerPublicationCommitStartError,
ScannerPublicationCommitState, ShardIntegrityWriteMode, StreamConsumer, WriteCommitGuard, WriteCompletion,
get_object_body_cache_plaintext_len, lookup_get_object_body_cache_hook, register_get_object_body_cache_hook,
register_object_mutation_hook, unregister_get_object_body_cache_hook, unregister_object_mutation_hook,
get_object_body_cache_plaintext_len, is_supported_ec_block_size, lookup_get_object_body_cache_hook,
parse_ec_block_size_hint, register_get_object_body_cache_hook, register_object_mutation_hook, resolve_ec_block_size_hint,
take_ec_block_size_hint, unregister_get_object_body_cache_hook, unregister_object_mutation_hook,
with_get_object_read_condition,
};
pub use crate::store::{
+1
View File
@@ -3507,6 +3507,7 @@ mod tests {
checksum: None,
multipart_completion_replayed: false,
backfilled_old_current_size: None,
ec_block_size: None,
}
}
}
+263 -25
View File
@@ -3852,6 +3852,9 @@ enum SyncMode {
FileOnly,
}
// Keep 2 MiB of shard data plus up to 128 32-byte hashes in one file buffer.
const SHARD_FILE_MAX_BUF_SIZE: usize = 2 * 1024 * 1024 + 4096;
#[allow(
dead_code,
reason = "reclaim bookkeeping fields written by Drop but never read back (backlog#1823)"
@@ -4200,7 +4203,8 @@ async fn read_dir_entry_batch_with_walk_stall(
}
impl FileCacheReclaimReader {
fn new(inner: File, reclaim_offset: u64, reclaim_len: usize, reclaim_on_drop: bool, read_ahead: usize) -> Self {
fn new(mut inner: File, reclaim_offset: u64, reclaim_len: usize, reclaim_on_drop: bool, read_ahead: usize) -> Self {
inner.set_max_buf_size(SHARD_FILE_MAX_BUF_SIZE);
#[cfg(target_os = "macos")]
if reclaim_on_drop {
let _ = set_fd_nocache(&inner);
@@ -5043,7 +5047,8 @@ impl AsyncWrite for LazyLocalFileWriter {
#[cfg(test)]
impl FileCacheReclaimWriter {
fn new(inner: File, reclaim_len: usize, reclaim_on_shutdown: bool) -> Self {
fn new(mut inner: File, reclaim_len: usize, reclaim_on_shutdown: bool) -> Self {
inner.set_max_buf_size(SHARD_FILE_MAX_BUF_SIZE);
#[cfg(target_os = "macos")]
if reclaim_on_shutdown {
let _ = set_fd_nocache(&inner);
@@ -14238,12 +14243,38 @@ impl LocalDisk {
}
}
/// Batch positioned reads for local EC shard files in a single `spawn_blocking`.
/// Shard windows at least this large are copy-bound: reading them concurrently
/// beats one serial blocking pass, while smaller windows stay coalesced to save
/// blocking-pool round trips.
#[cfg(unix)]
const LOCAL_BATCH_PARALLEL_MIN_SHARD_BYTES: usize = 512 * 1024;
/// Positioned reads for local EC shard files, returned in request order.
///
/// Collapses per-shard blocking-pool round-trips that dominate warm GET
/// fan-out on single-node multi-disk topologies.
/// Small windows share a single `spawn_blocking`, collapsing per-shard
/// blocking-pool round-trips that dominate warm GET fan-out on single-node
/// multi-disk topologies. When every window reaches
/// `LOCAL_BATCH_PARALLEL_MIN_SHARD_BYTES`, each shard gets its own dispatch so
/// copies and device reads proceed concurrently.
#[cfg(unix)]
pub(crate) async fn batch_shard_pread(requests: Vec<(std::path::PathBuf, usize, usize)>) -> Vec<Result<Bytes>> {
if requests.len() < 2
|| requests
.iter()
.any(|(_, _, length)| *length < LOCAL_BATCH_PARALLEL_MIN_SHARD_BYTES)
{
return batch_shard_pread_blocking(requests).await;
}
futures::future::join_all(requests.into_iter().map(|request| batch_shard_pread_blocking(vec![request])))
.await
.into_iter()
.flatten()
.collect()
}
/// Read every request in turn inside one `spawn_blocking`.
#[cfg(unix)]
async fn batch_shard_pread_blocking(requests: Vec<(std::path::PathBuf, usize, usize)>) -> Vec<Result<Bytes>> {
let n = requests.len();
let metrics = rustfs_io_metrics::LocalBatchReadMetrics::capture();
let submitted = metrics.as_ref().map(|_| std::time::Instant::now());
@@ -14251,7 +14282,7 @@ pub(crate) async fn batch_shard_pread(requests: Vec<(std::path::PathBuf, usize,
let diagnostic = crate::diagnostics::batch_read_diagnostics::Sample::capture();
tokio::task::spawn_blocking(move || {
use rustfs_io_metrics::LocalBatchReadStage;
use std::os::unix::fs::{FileExt, OpenOptionsExt};
use std::os::unix::fs::OpenOptionsExt;
// The disabled build expands directly to the original operation.
macro_rules! diagnostic_stage {
@@ -14319,26 +14350,36 @@ pub(crate) async fn batch_shard_pread(requests: Vec<(std::path::PathBuf, usize,
metrics.as_deref(),
LocalBatchReadStage::Read,
|| {
if offset == 0 {
use std::io::Read as _;
total = (&file)
.take(u64::try_from(length).unwrap_or(u64::MAX))
.read_to_end(&mut buf)
.map_err(DiskError::from)?;
if total < length {
// Announce the exact window before reading it. Kernel
// readahead otherwise sizes its first request from the
// read size: a small window at offset 0 pulls several
// times its size, and a multi-MiB read is split into
// serial readahead chunks. WILLNEED submits exactly the
// window at once. It is advisory, so an unsupported hint
// must not fail the read.
#[cfg(target_os = "linux")]
if let (Ok(start), Ok(len)) = (u64::try_from(offset), u64::try_from(length)) {
let _ = rustix::fs::fadvise(
&file,
start,
std::num::NonZeroU64::new(len),
rustix::fs::Advice::WillNeed,
);
}
// `with_capacity` reserves exactly `length` bytes, so each
// positioned read fills only the remaining requested span
// without zero-initializing it first.
while total < length {
let position = u64::try_from(offset + total).map_err(|_| DiskError::FileCorrupt)?;
let nbytes = match rustix::io::pread(&file, rustix::buffer::spare_capacity(&mut buf), position) {
Ok(nbytes) => nbytes,
Err(rustix::io::Errno::INTR) => continue,
Err(err) => return Err(DiskError::from(std::io::Error::from(err))),
};
if nbytes == 0 {
return Err(DiskError::FileCorrupt);
}
} else {
buf.resize(length, 0);
while total < length {
let nbytes = file
.read_at(&mut buf[total..], u64::try_from(offset + total).unwrap_or(u64::MAX))
.map_err(DiskError::from)?;
if nbytes == 0 {
return Err(DiskError::FileCorrupt);
}
total += nbytes;
}
total += nbytes;
}
Ok::<_, DiskError>(())
}
@@ -14889,6 +14930,14 @@ impl DiskAPI for LocalDisk {
)
.map_err(DiskError::from)?;
fi.validate(ValidationMode::RequireErasure)?;
let frame_size = crate::object_api::persisted_ec_read_quantum(
&fi.metadata,
codec_erasure.block_size,
codec_erasure.data_shards,
codec_erasure.shard_size(),
)
.map_err(|_| DiskError::FileCorrupt)?
.unwrap_or_else(|| codec_erasure.shard_size());
for (i, part) in fi.parts.iter().enumerate() {
let checksum_info = erasure.get_checksum_info(part.number);
let checksum_algo = if fi.uses_legacy_checksum && checksum_info.algorithm == HashAlgorithm::HighwayHash256S {
@@ -14910,7 +14959,7 @@ impl DiskAPI for LocalDisk {
&part_path,
codec_erasure.shard_file_size(part.size as i64) as usize,
checksum_algo,
codec_erasure.shard_size(),
frame_size,
)
.await
.err();
@@ -32973,6 +33022,80 @@ mod test {
assert!(body.ends_with(b"abcdef"));
}
#[tokio::test]
async fn file_cache_reclaim_wrappers_round_trip_checksum_sized_file_buffers() {
use sha2::{Digest, Sha256};
use std::io::IoSlice;
let dir = tempfile::tempdir().expect("create wrapper test directory");
for checksum_bytes in [32, 64] {
let wire_len = 2 * 1024 * 1024 + checksum_bytes;
let content: Vec<_> = (0u8..=250).cycle().take(wire_len).collect();
let path = dir.path().join(format!("frame-{checksum_bytes}"));
let file = File::create(&path).await.expect("create writer file");
let mut writer = FileCacheReclaimWriter::new(file, wire_len, true);
let mut bufs = Vec::new();
for frame in content.chunks(wire_len / (checksum_bytes / 32)) {
bufs.push(IoSlice::new(&frame[..32]));
bufs.push(IoSlice::new(&frame[32..]));
}
let accepted = writer.write_vectored(&bufs).await.expect("accept checksum and data slices");
assert_eq!(accepted, wire_len, "checksum overhead must fit alongside 2 MiB of data");
let tail = b"short final tail!";
writer.write_all(tail).await.expect("write after the accepted frame");
writer.flush().await.expect("flush pending writes");
writer.shutdown().await.expect("finish writes before cache reclaim");
writer.inner.sync_all().await.expect("sync the completed test file");
let file = File::open(&path).await.expect("reopen completed file");
let mut reader = FileCacheReclaimReader::new(file, 0, wire_len + tail.len(), true, 0);
let mut actual = vec![0; wire_len];
let read = reader
.read(&mut actual)
.await
.expect("read the complete large frame in one request");
assert_eq!(read, wire_len, "the file buffer must read checksum overhead alongside 2 MiB of data");
assert_eq!(actual, content, "large reads must preserve every accepted byte");
assert_eq!(Sha256::digest(&actual), Sha256::digest(&content), "frame digest must survive file I/O");
let mut actual_tail = [0; 32];
assert_eq!(
reader
.read_exact(&mut actual_tail)
.await
.expect_err("short tail must report EOF")
.kind(),
std::io::ErrorKind::UnexpectedEof
);
assert_eq!(&actual_tail[..tail.len()], tail, "short read must preserve the available tail");
assert_eq!(reader.read(&mut [0; 1]).await.expect("read at EOF"), 0);
}
}
#[cfg(target_os = "linux")]
#[tokio::test]
async fn file_cache_reclaim_writer_propagates_shutdown_error_after_acceptance() {
use std::io::IoSlice;
let file = fs::OpenOptions::new()
.write(true)
.open("/dev/full")
.await
.expect("open full device");
let payload = vec![31; 2 * 1024 * 1024 + 64];
let mut writer = FileCacheReclaimWriter::new(file, payload.len(), true);
let accepted = writer
.write_vectored(&[IoSlice::new(&payload[..32]), IoSlice::new(&payload[32..])])
.await
.expect("Tokio may accept bytes before the blocking write fails");
assert_eq!(accepted, payload.len(), "the expanded buffer must accept the complete frame");
let error = writer
.shutdown()
.await
.expect_err("accepted bytes must not hide a failed blocking write");
assert_eq!(error.raw_os_error(), Some(libc::ENOSPC));
assert!(!writer.reclaimed, "failed shutdown must not run successful-write cache reclaim");
}
#[cfg(target_os = "macos")]
#[tokio::test]
async fn macos_nocache_helpers_accept_tokio_and_std_files() {
@@ -38296,6 +38419,46 @@ mod test {
assert_eq!(recorder.histogram_values(stages, &[("stage", "queue_wait")]).len(), 1);
}
#[cfg(unix)]
#[test]
fn local_batch_reads_large_windows_per_shard_and_coalesces_small_windows() {
use crate::test_metrics::CapturingRecorder;
let dir = tempfile::tempdir().expect("dispatch shape directory");
let large = LOCAL_BATCH_PARALLEL_MIN_SHARD_BYTES;
let paths: Vec<_> = (0..2)
.map(|index| {
let path = dir.path().join(format!("shard-{index}"));
std::fs::write(&path, vec![u8::try_from(index).expect("small shard index"); large + 64])
.expect("write dispatch shape shard");
path
})
.collect();
let runtime = tokio::runtime::Builder::new_current_thread()
.enable_all()
.build()
.expect("dispatch shape runtime");
let previous = rustfs_io_metrics::get_stage_metrics_enabled();
rustfs_io_metrics::set_get_stage_metrics_enabled(true);
let dispatches = |length: usize| {
let recorder = CapturingRecorder::default();
let requests = paths.iter().map(|path| (path.clone(), 64, length)).collect();
let results = metrics::with_local_recorder(&recorder, || runtime.block_on(batch_shard_pread(requests)));
for (index, result) in results.iter().enumerate() {
let bytes = result.as_ref().expect("window inside the shard");
assert_eq!(bytes.len(), length);
assert!(bytes.iter().all(|byte| usize::from(*byte) == index), "results keep request order");
}
recorder.histogram_values("rustfs_io_get_object_local_batch_shards", &[])
};
let large_shape = dispatches(large);
let small_shape = dispatches(large - 1);
rustfs_io_metrics::set_get_stage_metrics_enabled(previous);
assert_eq!(large_shape, vec![1.0, 1.0], "copy-bound windows must not serialize in one worker");
assert_eq!(small_shape, vec![2.0], "small windows must share one blocking dispatch");
}
#[cfg(unix)]
#[test]
fn local_batch_read_metrics_survive_caller_cancellation() {
@@ -38412,6 +38575,81 @@ mod test {
}
}
#[cfg(unix)]
#[tokio::test]
async fn test_batch_shard_pread_returns_exact_multi_page_windows() {
let dir = tempfile::tempdir().expect("multi-page window fixture");
let path = dir.path().join("shard");
let window = 1024 * 1024 + 32;
let content: Vec<u8> = (0..3 * window).map(|i| (i % 251) as u8).collect();
std::fs::write(&path, &content).expect("write multi-page shard");
let unaligned = 2 * window + 7;
// Windows of one and two q frames, at zero and nonzero offsets.
let windows = [
(0, window),
(window, window),
(unaligned, content.len() - unaligned),
(0, 2 * window),
(window - 5, 2 * window),
];
let results = batch_shard_pread(
windows
.iter()
.map(|&(offset, length)| (path.clone(), offset, length))
.collect(),
)
.await;
for (result, (offset, length)) in results.iter().zip(windows) {
let bytes = result.as_ref().expect("window inside the shard");
assert_eq!(bytes.len(), length, "window at {offset} must not over-read");
assert_eq!(bytes.as_ref(), &content[offset..offset + length], "window at {offset}");
}
}
/// A cold batch read must bring in only its window. Without an exact hint,
/// kernel readahead sizes the first request from the read size, so a 16 KiB
/// window at offset 0 caches several times its pages and a 2 MiB window
/// almost twice its pages.
#[cfg(target_os = "linux")]
#[tokio::test]
async fn test_batch_shard_pread_cold_read_caches_only_its_window() {
use rustix::fs::{Advice, fadvise};
let dir = tempfile::tempdir().expect("cold window fixture");
let path = dir.path().join("shard");
std::fs::write(&path, vec![5u8; 8 << 20]).expect("write cold window shard");
std::fs::File::open(&path)
.expect("open shard for sync")
.sync_all()
.expect("clean pages before eviction");
let page = mmap_page_size().expect("page size should be available") as usize;
let evict = || {
let file = std::fs::File::open(&path).expect("open shard for eviction");
fadvise(&file, 0, None, Advice::DontNeed).expect("evict shard pages");
};
evict();
assert_eq!(
resident_pages(&path),
0,
"the temporary filesystem must be able to drop page cache, otherwise residency is unobservable"
);
for length in [16 * 1024 + 32, 2 * 1024 * 1024 + 64] {
evict();
let results = batch_shard_pread(vec![(path.clone(), 0, length)]).await;
assert_eq!(results[0].as_ref().expect("cold window read").len(), length);
let window_pages = length.div_ceil(page);
let resident = resident_pages(&path);
assert!(
resident >= window_pages,
"the window itself must be resident: {resident} < {window_pages}"
);
assert!(
resident <= window_pages + window_pages / 8 + 4,
"a {length}-byte cold read cached {resident} pages for a {window_pages}-page window"
);
}
}
#[cfg(unix)]
#[tokio::test]
async fn test_batch_shard_pread_basic() {
@@ -1661,7 +1661,10 @@ impl LocalDisk {
) -> Result<(Vec<DeleteTransactionId>, Option<DiscoverySession>)> {
let lease = os::acquire_namespace_mutation_lease_with_owner(path, None).await;
self.validate_delete_discovery_owner(&session)?;
os::run_blocking_namespace_operation(lease, move || session.advance(limit))
// A discovery page can lstat thousands of entries on a cold tree. The
// one-worker foreground lane also serves GET metadata reads, so a page
// there stalls every read on this disk until the walk finishes.
os::run_blocking_namespace_operation_with_sync(lease, true, move || session.advance(limit))
.await
.map_err(|error| to_file_error(error).into())
}
@@ -4340,6 +4343,38 @@ mod tests {
}
}
#[tokio::test]
async fn delete_recovery_discovery_runs_while_foreground_metadata_lane_is_busy() {
let (_temp, disk, _info, _original) = fixture().await;
let (entered_tx, entered_rx) = tokio::sync::oneshot::channel();
let (release_tx, release_rx) = std::sync::mpsc::channel::<()>();
let queue = disk.blocking_queue.clone();
let foreground = tokio::spawn(async move {
queue
.run_cancellable(move |_| {
let _ = entered_tx.send(());
let _ = release_rx.recv();
Ok(())
})
.await
});
entered_rx.await.expect("foreground metadata worker entered");
let page = tokio::time::timeout(
std::time::Duration::from_secs(10),
disk.delete_recovery_discover("recovery", None, None, 128),
)
.await;
release_tx.send(()).expect("release foreground metadata worker");
foreground
.await
.expect("foreground metadata job joins")
.expect("foreground metadata job succeeds");
let page = page
.expect("bucket discovery must not wait behind the foreground metadata lane")
.expect("bucket discovery succeeds");
assert!(page.transactions.is_empty(), "fixture holds no delete transactions");
}
#[tokio::test]
async fn delete_recovery_live_write_backup_preserves_delete_evidence() {
let (_temp, disk, info, original) = fixture().await;
File diff suppressed because it is too large Load Diff
+735
View File
@@ -53,6 +53,8 @@ pub(crate) struct DecodeOutcome {
pub(crate) error: Option<std::io::Error>,
pub(crate) exact_quorum: bool,
pub(crate) repair: ShardReadRepair,
pub(crate) frame_count: usize,
pub(crate) read_dispatch_count: usize,
}
type ShardIndexes = SmallVec<[usize; INLINE_SHARD_SLOTS]>;
@@ -835,6 +837,58 @@ where
}
}
/// Build a reconstruction-verifying reader for one q-aligned slice of a
/// logical EC block. The underlying erasure geometry still determines the
/// number of data and parity readers, while `read_shard_size` describes the
/// smaller byte vector that Reed-Solomon should reconstruct for this range.
///
/// The q-range caller opens every reader at the same logical shard offset,
/// so this reader only needs one local stripe and no global shard cursor.
fn new_for_quantized_decode(
readers: Vec<Option<BitrotReader<R>>>,
e: Erasure,
read_shard_size: usize,
metrics_path: Option<&'static str>,
read_costs: Option<Vec<ShardReadCost>>,
) -> Self {
let mut reader = if let Some(read_costs) = read_costs {
Self::new_with_metrics_path_read_costs_timeout_and_reconstruction_verification(
readers,
e,
0,
0,
metrics_path,
read_costs,
get_object_disk_read_timeout(),
true,
)
} else {
Self::new_with_metrics_path_read_timeout_and_reconstruction_verification(
readers,
e,
0,
0,
metrics_path,
get_object_disk_read_timeout(),
true,
)
};
reader.offset = 0;
reader.shard_size = read_shard_size;
reader.shard_file_size = read_shard_size;
// A q-range is already bounded to the exact shard frames needed by the
// client request. Keep healthy reads demand-bound as well: read the
// data quorum and retain parity as stripe-aligned reserves for a failed
// data reader. The ordinary GET rollout remains controlled by its
// deployment gate, while this bounded path avoids paying parity I/O for
// every small Range request.
reader.demand_bound_lockstep = true;
for (index, engaged) in reader.engaged.iter_mut().enumerate() {
*engaged = index < reader.data_shards;
}
reader
}
pub(crate) fn with_repair_evidence(mut self, evidence: Arc<ShardReadRepair>) -> Self {
self.repair_evidence = Some(evidence);
self
@@ -999,6 +1053,15 @@ where
state.into_parts()
}
pub(crate) fn read_shape(&self) -> (usize, usize) {
self.readers.iter().flatten().fold((0, 0), |(frames, dispatches), reader| {
(
frames.saturating_add(reader.last_frame_count()),
dispatches.saturating_add(reader.last_read_dispatch_count()),
)
})
}
async fn read_into_state(&mut self, state: &mut StripeReadState) {
// On the reconstruction-verifying GET path, read every live shard reader
// in lockstep so all readers advance one block per stripe and stay
@@ -1834,6 +1897,9 @@ where
// fail a stripe that later reaches a real quorum.
self.engaged[i] = false;
} else {
if let Some(evidence) = &self.repair_evidence {
evidence.observe(&[Some(e.clone())]);
}
errs[i] = Some(e);
// Lockstep cannot safely reuse a reader after any
// error, even when the low-level classifier called it
@@ -2443,6 +2509,301 @@ impl Erasure {
.await
}
/// Decode a range that was opened as one q-aligned slice of every data
/// shard. The caller supplies the output offset within the concatenated
/// q-slices (`shard_index * read_shard_size + offset_in_shard`). Keeping
/// this path separate from `decode_inner` preserves the normal full-shard
/// reader geometry for legacy and full-object GETs.
/// `allow_direct_data_shard` requires the caller to have established its
/// metadata/open quorum and enabled frame authentication; policies requiring
/// reconstruction evidence must leave it false.
#[allow(clippy::too_many_arguments)]
pub(crate) async fn decode_quantized_range_with_stripe_handles_and_reopeners<W, R>(
&self,
writer: &mut W,
mut readers: Vec<Option<BitrotReader<R>>>,
output_offset: usize,
length: usize,
read_shard_size: usize,
read_costs: Option<Vec<ShardReadCost>>,
mut deferred_handles: Vec<Option<DeferredReaderStripeHandle>>,
mut deferred_reopeners: Vec<Option<DeferredReaderReopener<R>>>,
allow_direct_data_shard: bool,
) -> DecodeOutcome
where
W: AsyncWrite + Send + Sync + Unpin,
R: crate::erasure::coding::ShardSource,
{
if readers.len() != self.data_shards + self.parity_shards {
record_get_object_pipeline_failure(GET_STAGE_RANGE, GetObjectFailureReason::RangeOrLengthInvalid);
return DecodeOutcome {
error: Some(io::Error::new(ErrorKind::InvalidInput, "Invalid number of readers")),
..Default::default()
};
}
if read_shard_size == 0 {
record_get_object_pipeline_failure(GET_STAGE_RANGE, GetObjectFailureReason::RangeOrLengthInvalid);
return DecodeOutcome {
error: Some(io::Error::new(ErrorKind::InvalidInput, "quantized shard size must be non-zero")),
..Default::default()
};
}
let Some(required_len) = output_offset.checked_add(length) else {
record_get_object_pipeline_failure(GET_STAGE_RANGE, GetObjectFailureReason::RangeOrLengthInvalid);
return DecodeOutcome {
error: Some(io::Error::new(ErrorKind::InvalidInput, "quantized output range overflows")),
..Default::default()
};
};
let Some(max_len) = self.data_shards.checked_mul(read_shard_size) else {
record_get_object_pipeline_failure(GET_STAGE_RANGE, GetObjectFailureReason::RangeOrLengthInvalid);
return DecodeOutcome {
error: Some(io::Error::new(ErrorKind::InvalidInput, "quantized shard geometry overflows")),
..Default::default()
};
};
if required_len > max_len {
record_get_object_pipeline_failure(GET_STAGE_RANGE, GetObjectFailureReason::RangeOrLengthInvalid);
return DecodeOutcome {
error: Some(io::Error::new(ErrorKind::InvalidInput, "quantized output range exceeds shard geometry")),
..Default::default()
};
}
if length == 0 {
return DecodeOutcome::default();
}
let metrics_path = Some(GET_OBJECT_PATH_LEGACY_DUPLEX);
let target_shard = output_offset / read_shard_size;
let within_shard = output_offset % read_shard_size;
// The caller has already established the normal metadata and reader-
// open quorum. A healthy systematic shard can serve a single-shard
// range after authenticating every frame in its opened q-slice. Other
// data shards contain no requested bytes; read them only if this shard
// fails and reconstruction becomes necessary. Protected layouts and
// policies requiring reconstruction evidence keep the quorum decoder.
if allow_direct_data_shard
&& !self.uses_legacy_codec()
&& length <= read_shard_size - within_shard
&& readers.iter().flatten().all(|reader| reader.integrity_proof().is_none())
&& let Some(mut direct_reader) = readers[target_shard].take()
{
let stripe_read_stage_start = get_stage_timer_if_enabled(rustfs_io_metrics::get_stage_metrics_enabled());
let read_cost = read_costs
.as_ref()
.and_then(|costs| costs.get(target_shard))
.copied()
.unwrap_or(ShardReadCost::Unknown);
let read_timeout = get_object_disk_read_timeout();
let mut direct_read = Box::pin(read_shard_result(
target_shard,
read_cost,
&mut direct_reader,
None,
read_shard_size,
self.data_shards,
read_timeout,
metrics_path,
));
let initial_result = if let Some(delay) = shard_read_hedge_delay(read_timeout) {
tokio::select! {
biased;
result = &mut direct_read => Some(result.0),
_ = tokio::time::sleep(delay) => None,
}
} else {
Some((&mut direct_read).await.0)
};
let repair = Arc::new(ShardReadRepair::default());
let mut outcome = DecodeOutcome::default();
let selected_data;
match initial_result {
Some(Ok(data)) => selected_data = Some((data, within_shard)),
initial_result => {
// Keep a pending target alive through its real deadline.
// The hedge only admits competing reconstruction; it is
// not evidence that this healthy source is missing.
if let Some(handle) = deferred_handles.get_mut(target_shard) {
*handle = None;
}
if let Some(reopener) = deferred_reopeners.get_mut(target_shard) {
*reopener = None;
}
if let Some(Err(error)) = &initial_result {
repair.observe(&[Some(error.clone())]);
}
// Buffer at most one requested data-shard slice. Neither
// contender writes to the downstream until authenticated
// direct bytes or a complete verified quorum has won.
let mut fallback_data = Vec::with_capacity(length);
let mut fallback = Box::pin(self.decode_quantized_range_quorum(
&mut fallback_data,
readers,
output_offset,
length,
read_shard_size,
read_costs,
deferred_handles,
deferred_reopeners,
Some(target_shard),
Arc::clone(&repair),
));
if initial_result.is_some() {
outcome = (&mut fallback).await;
drop(fallback);
selected_data = (outcome.written == length).then_some((ShardBuf::from(fallback_data), 0));
} else {
let race = tokio::select! {
biased;
result = &mut direct_read => Ok(result.0),
outcome = &mut fallback => Err(outcome),
};
match race {
Ok(Ok(data)) => {
drop(fallback);
selected_data = Some((data, within_shard));
}
Ok(Err(error)) => {
repair.observe(&[Some(error)]);
outcome = (&mut fallback).await;
drop(fallback);
selected_data = (outcome.written == length).then_some((ShardBuf::from(fallback_data), 0));
}
Err(fallback_outcome) => {
drop(fallback);
if fallback_outcome.written == length {
outcome = fallback_outcome;
selected_data = Some((ShardBuf::from(fallback_data), 0));
} else {
// A failed alternative quorum must not vote
// away a still-pending healthy target.
match (&mut direct_read).await.0 {
Ok(data) => selected_data = Some((data, within_shard)),
Err(error) => {
repair.observe(&[Some(error)]);
outcome = fallback_outcome;
selected_data = None;
}
}
}
}
}
}
}
}
drop(direct_read);
outcome.frame_count = outcome.frame_count.saturating_add(direct_reader.last_frame_count());
outcome.read_dispatch_count = outcome
.read_dispatch_count
.saturating_add(direct_reader.last_read_dispatch_count());
outcome.repair = repair.snapshot();
record_get_stage_duration_if_enabled(GET_OBJECT_PATH_LEGACY_DUPLEX, GET_STAGE_STRIPE_READ, stripe_read_stage_start);
if let Some((data, start)) = selected_data {
let emit_stage_start = get_stage_timer_if_enabled(rustfs_io_metrics::get_stage_metrics_enabled());
let result = writer.write_all(&data[start..start + length]).await;
record_get_stage_duration_if_enabled(GET_OBJECT_PATH_LEGACY_DUPLEX, GET_STAGE_EMIT, emit_stage_start);
if let Err(error) = result {
outcome.written = 0;
outcome.error = Some(error);
} else {
outcome.written = length;
}
}
return outcome;
}
self.decode_quantized_range_quorum(
writer,
readers,
output_offset,
length,
read_shard_size,
read_costs,
deferred_handles,
deferred_reopeners,
None,
Arc::new(ShardReadRepair::default()),
)
.await
}
#[allow(clippy::too_many_arguments)]
async fn decode_quantized_range_quorum<W, R>(
&self,
writer: &mut W,
readers: Vec<Option<BitrotReader<R>>>,
output_offset: usize,
length: usize,
read_shard_size: usize,
read_costs: Option<Vec<ShardReadCost>>,
deferred_handles: Vec<Option<DeferredReaderStripeHandle>>,
deferred_reopeners: Vec<Option<DeferredReaderReopener<R>>>,
pending_target: Option<usize>,
repair: Arc<ShardReadRepair>,
) -> DecodeOutcome
where
W: AsyncWrite + Send + Sync + Unpin,
R: crate::erasure::coding::ShardSource,
{
let metrics_path = Some(GET_OBJECT_PATH_LEGACY_DUPLEX);
let mut reader =
ParallelReader::new_for_quantized_decode(readers, self.clone(), read_shard_size, metrics_path, read_costs)
.with_deferred_parity_handles(deferred_handles)
.with_deferred_parity_reopeners(deferred_reopeners)
.with_repair_evidence(Arc::clone(&repair));
let integrity = reader.integrity.clone();
// The q-range reader still verifies each bitrot frame. Requiring an
// additional reconstruction source for a healthy range would defeat
// demand-bound parity and turn every request back into an all-shard
// read; parity is admitted by the lockstep reader only when data is
// missing or fails verification.
let require_surplus_source = false;
let stripe_read_stage_start = get_stage_timer_if_enabled(rustfs_io_metrics::get_stage_metrics_enabled());
let (mut shards, mut errs) = reader.read().await;
if let Some(target_shard) = pending_target {
// This slot is owned by the direct contender, rather than missing
// from storage. Clear only its synthetic error; actual successful
// shards still determine the reconstruction quorum below.
errs[target_shard] = None;
}
let (frame_count, read_dispatch_count) = reader.read_shape();
record_get_stage_duration_if_enabled(GET_OBJECT_PATH_LEGACY_DUPLEX, GET_STAGE_STRIPE_READ, stripe_read_stage_start);
let exact_quorum = shards.iter().filter(|shard| shard.is_some()).count() == self.data_shards;
let mut written = 0;
let mut ret_err = None;
let flow = self
.emit_decoded_stripe(
writer,
&mut shards,
&errs,
output_offset,
length,
&mut written,
&mut ret_err,
&repair,
rustfs_io_metrics::get_stage_metrics_enabled(),
require_surplus_source,
integrity.as_ref(),
0,
)
.await;
if matches!(flow, StripeFlow::Continue) {
reader.recycle_shards(&mut shards);
}
if ret_err.is_none() && written < length {
ret_err = Some(Error::LessData.into());
}
DecodeOutcome {
written,
error: ret_err,
exact_quorum,
repair: repair.snapshot(),
frame_count,
read_dispatch_count,
}
}
/// Reconstruct and emit one already-read stripe.
///
/// This is the shared per-stripe body used by both the serial legacy loop
@@ -2893,6 +3254,8 @@ impl Erasure {
error: ret_err,
exact_quorum,
repair,
frame_count: 0,
read_dispatch_count: 0,
}
}
}
@@ -3046,6 +3409,372 @@ mod tests {
}
}
async fn quantized_counting_readers(
erasure: &Erasure,
data: &[u8],
frame_size: usize,
slice_offset: usize,
slice_length: usize,
corrupt_shard: Option<usize>,
) -> (Vec<Option<BitrotReader<CountingShardReader>>>, Vec<Arc<AtomicUsize>>) {
let shards = erasure.encode_data(data).expect("quantized fixture must encode");
let mut readers = Vec::with_capacity(shards.len());
let mut counters = Vec::with_capacity(shards.len());
for shard in shards {
let mut writer = BitrotWriter::new(Vec::new(), frame_size, HashAlgorithm::HighwayHash256S);
for frame in shard[slice_offset..slice_offset + slice_length].chunks(frame_size) {
writer.write(frame).await.expect("quantized fixture frame must write");
}
let mut encoded = writer.into_inner();
if corrupt_shard == Some(readers.len()) {
*encoded.last_mut().expect("fixture has a frame payload") ^= 0x80;
}
let bytes_read = Arc::new(AtomicUsize::new(0));
readers.push(Some(BitrotReader::new_with_frame_size(
CountingShardReader {
inner: Cursor::new(encoded),
bytes_read: Arc::clone(&bytes_read),
},
erasure.shard_size(),
frame_size,
HashAlgorithm::HighwayHash256S,
false,
)));
counters.push(bytes_read);
}
(readers, counters)
}
#[tokio::test]
async fn quantized_direct_reads_only_the_requested_authenticated_data_shard() {
let data = b"abcdefghijklmnop";
let erasure = Erasure::new(2, 2, data.len());
let (readers, counters) = quantized_counting_readers(&erasure, data, 4, 0, 4, None).await;
let mut output = Vec::new();
let outcome = erasure
.decode_quantized_range_with_stripe_handles_and_reopeners(
&mut output,
readers,
5,
2,
4,
None,
Vec::new(),
Vec::new(),
true,
)
.await;
assert!(outcome.error.is_none(), "authenticated direct range must succeed: {:?}", outcome.error);
assert_eq!(outcome.written, 2);
assert_eq!(output, &data[9..11]);
assert_eq!(outcome.frame_count, 1);
assert_eq!(outcome.read_dispatch_count, 1);
for (index, counter) in counters.iter().enumerate() {
assert_eq!(
counter.load(Ordering::SeqCst),
if index == 1 {
HashAlgorithm::HighwayHash256S.size() + 4
} else {
0
},
"healthy direct range must not read unrequested shard {index}"
);
}
}
#[tokio::test]
async fn quantized_direct_can_be_disabled_without_changing_range_bytes() {
let data = b"abcdefghijklmnop";
let erasure = Erasure::new(2, 2, data.len());
let (readers, counters) = quantized_counting_readers(&erasure, data, 4, 0, 4, None).await;
let mut output = Vec::new();
let outcome = erasure
.decode_quantized_range_with_stripe_handles_and_reopeners(
&mut output,
readers,
5,
2,
4,
None,
Vec::new(),
Vec::new(),
false,
)
.await;
assert!(outcome.error.is_none(), "quorum decoder range must succeed: {:?}", outcome.error);
assert_eq!(output, &data[9..11]);
assert!(counters[0].load(Ordering::SeqCst) > 0, "disabled direct path must read the data quorum");
assert!(
counters[1].load(Ordering::SeqCst) > 0,
"disabled direct path must read the requested data shard"
);
assert_eq!(counters[2].load(Ordering::SeqCst), 0);
assert_eq!(counters[3].load(Ordering::SeqCst), 0);
}
#[tokio::test]
async fn quantized_direct_verifies_every_frame_before_emitting_and_recovers_corruption() {
let data = b"abcdefghijklmnop";
let erasure = Erasure::new(2, 2, data.len());
for corrupt in [false, true] {
let (readers, counters) = quantized_counting_readers(&erasure, data, 4, 0, 8, corrupt.then_some(0)).await;
let mut output = Vec::new();
let outcome = erasure
.decode_quantized_range_with_stripe_handles_and_reopeners(
&mut output,
readers,
2,
6,
8,
None,
Vec::new(),
Vec::new(),
true,
)
.await;
assert_eq!(
outcome.written, 6,
"corrupt={corrupt}: authenticated or reconstructed range must complete"
);
assert_eq!(output, &data[2..8], "corrupt={corrupt}: no corrupt prefix may reach the writer");
assert_eq!(
outcome.repair.needed(false),
corrupt,
"observed corruption must survive successful reconstruction"
);
assert_eq!(
counters[0].load(Ordering::SeqCst),
2 * (HashAlgorithm::HighwayHash256S.size() + 4),
"target must authenticate both frames exactly once, including the failing second frame"
);
if !corrupt {
assert_eq!(outcome.frame_count, 2);
assert_eq!(counters[1].load(Ordering::SeqCst), 0);
} else {
assert!(
counters[1].load(Ordering::SeqCst) > 0,
"reconstruction must read the remaining data shard"
);
assert!(counters[2].load(Ordering::SeqCst) + counters[3].load(Ordering::SeqCst) > 0);
}
}
}
#[tokio::test]
async fn quantized_direct_missing_target_reconstructs_the_same_slice() {
let data = b"abcdefghijklmnop";
let erasure = Erasure::new(2, 2, data.len());
let (mut readers, counters) = quantized_counting_readers(&erasure, data, 4, 4, 4, None).await;
readers[1] = None;
let mut output = Vec::new();
let outcome = erasure
.decode_quantized_range_with_stripe_handles_and_reopeners(
&mut output,
readers,
5,
2,
4,
None,
Vec::new(),
Vec::new(),
true,
)
.await;
assert_eq!(outcome.written, 2, "missing target must reconstruct from a read quorum");
assert_eq!(output, &data[13..15], "fallback must remain aligned to the nonzero q slice");
assert_eq!(counters[1].load(Ordering::SeqCst), 0);
assert!(counters[0].load(Ordering::SeqCst) > 0);
assert!(counters[2].load(Ordering::SeqCst) + counters[3].load(Ordering::SeqCst) > 0);
}
#[tokio::test]
async fn quantized_direct_unrecoverable_corruption_emits_no_bytes() {
let data = b"abcdefghijklmnop";
let erasure = Erasure::new(2, 2, data.len());
let (mut readers, _) = quantized_counting_readers(&erasure, data, 4, 0, 4, Some(0)).await;
readers[1] = None;
readers[3] = None;
let mut output = Vec::new();
let outcome = erasure
.decode_quantized_range_with_stripe_handles_and_reopeners(
&mut output,
readers,
1,
2,
4,
None,
Vec::new(),
Vec::new(),
true,
)
.await;
assert!(outcome.error.is_some(), "insufficient reconstruction quorum must fail");
assert_eq!(outcome.written, 0);
assert!(output.is_empty(), "unverified data must never be emitted");
assert!(outcome.repair.needed(false), "failed direct read must retain corruption evidence");
}
#[tokio::test]
async fn quantized_direct_preserves_writer_failure() {
let data = b"abcdefghijklmnop";
let erasure = Erasure::new(2, 2, data.len());
let (readers, _) = quantized_counting_readers(&erasure, data, 4, 0, 4, None).await;
let outcome = erasure
.decode_quantized_range_with_stripe_handles_and_reopeners(
&mut FailingEmitWriter,
readers,
1,
2,
4,
None,
Vec::new(),
Vec::new(),
true,
)
.await;
assert_eq!(outcome.error.expect("writer failure must escape").kind(), ErrorKind::BrokenPipe);
assert_eq!(outcome.written, 0);
}
#[tokio::test(start_paused = true)]
#[serial_test::serial]
async fn quantized_direct_stalled_target_recovers_without_waiting_for_disk_timeout() {
let data = b"abcdefghijklmnop";
let erasure = Erasure::new(2, 2, data.len());
let (readers, _) = quantized_counting_readers(&erasure, data, 4, 4, 4, None).await;
let slow_until = TokioInstant::now() + Duration::from_secs(60);
let readers = readers
.into_iter()
.enumerate()
.map(|(index, reader)| {
let bytes = reader.expect("fixture reader").inner_ref().inner.get_ref().clone();
let source = if index == 1 {
TestShardReader::ReadyAt {
cursor: Cursor::new(bytes),
ready_at: slow_until,
sleep: None,
}
} else {
TestShardReader::Ready(Cursor::new(bytes))
};
Some(BitrotReader::new_with_frame_size(
source,
erasure.shard_size(),
4,
HashAlgorithm::HighwayHash256S,
false,
))
})
.collect();
let mut output = Vec::new();
let started = TokioInstant::now();
let outcome = temp_env::async_with_vars([(rustfs_config::ENV_OBJECT_DISK_READ_TIMEOUT, Some("30"))], async {
erasure
.decode_quantized_range_with_stripe_handles_and_reopeners(
&mut output,
readers,
5,
2,
4,
None,
Vec::new(),
Vec::new(),
true,
)
.await
})
.await;
assert_eq!(outcome.written, 2, "fast parity must recover the stalled q slice");
assert_eq!(output, &data[13..15], "timeout recovery must retain the original q offset");
assert!(!outcome.repair.needed(true), "a hedged healthy target is not a missing storage shard");
assert!(
started.elapsed() <= Duration::from_millis(150),
"direct attempt must admit reconstruction by the existing 100 ms hedge, elapsed={:?}",
started.elapsed()
);
}
#[tokio::test(start_paused = true)]
#[serial_test::serial]
async fn quantized_direct_pending_healthy_target_survives_missing_or_corrupt_parity() {
let data = b"abcdefghijklmnop";
let erasure = Erasure::new(2, 2, data.len());
for corrupt_parity in [false, true] {
let (readers, _) = quantized_counting_readers(&erasure, data, 4, 4, 4, None).await;
let started = TokioInstant::now();
let ready_at = started + Duration::from_millis(200);
let readers = readers
.into_iter()
.enumerate()
.map(|(index, reader)| {
let mut bytes = reader.expect("fixture reader").inner_ref().inner.get_ref().clone();
if index >= erasure.data_shards {
if !corrupt_parity {
return None;
}
*bytes.last_mut().expect("parity payload") ^= 0x80;
}
let source = if index == 1 {
TestShardReader::ReadyAt {
cursor: Cursor::new(bytes),
ready_at,
sleep: None,
}
} else {
TestShardReader::Ready(Cursor::new(bytes))
};
Some(BitrotReader::new_with_frame_size(
source,
erasure.shard_size(),
4,
HashAlgorithm::HighwayHash256S,
false,
))
})
.collect();
let mut output = Vec::new();
let outcome = temp_env::async_with_vars([(rustfs_config::ENV_OBJECT_DISK_READ_TIMEOUT, Some("30"))], async {
erasure
.decode_quantized_range_with_stripe_handles_and_reopeners(
&mut output,
readers,
5,
2,
4,
None,
Vec::new(),
Vec::new(),
true,
)
.await
})
.await;
assert!(
outcome.error.is_none(),
"corrupt_parity={corrupt_parity}: healthy target must remain eligible"
);
assert_eq!(outcome.written, 2);
assert_eq!(output, &data[13..15], "a failed competing quorum must not emit or duplicate range bytes");
assert!(
started.elapsed() >= Duration::from_millis(200) && started.elapsed() <= Duration::from_millis(250),
"corrupt_parity={corrupt_parity}: wait for the healthy target, not the 30 s disk deadline"
);
assert_eq!(
outcome.repair.needed(false),
corrupt_parity,
"actual observed parity corruption must survive a successful direct winner"
);
}
}
/// Build a production-shaped reader set: data shards as opened stream
/// readers, parity shards as *unopened* deferred readers carrying stripe
/// handles (mirroring `fill_deferred_bitrot_readers`). `truncate` shortens
@@ -3305,6 +4034,12 @@ mod tests {
assert_eq!(defaulted.read_costs, vec![ShardReadCost::Unknown; 3]);
assert!(defaulted.verify_reconstruction);
let quantized: ParallelReader<Cursor<Vec<u8>>> =
ParallelReader::new_for_quantized_decode(vec![None, None, None], Erasure::new(2, 1, 64), 16, None, None);
assert!(quantized.verify_reconstruction);
assert!(quantized.demand_bound_lockstep);
assert_eq!(quantized.engaged.as_slice(), &[true, true, false]);
}
#[tokio::test]
+365 -9
View File
@@ -39,7 +39,7 @@ use std::pin::Pin;
use std::sync::{Arc, Mutex};
use std::task::{Context, Poll};
use std::time::Instant;
use tokio::io::{AsyncRead, ReadBuf};
use tokio::io::{AsyncRead, AsyncReadExt, ReadBuf};
use tracing::debug;
#[cfg(all(test, feature = "hotpath"))]
@@ -142,6 +142,7 @@ struct BitrotReaderSource {
path: String,
offset: usize,
length: usize,
small_shard: bool,
use_mmap_read: bool,
stage_metrics: Option<BitrotReaderStageMetrics>,
}
@@ -155,6 +156,7 @@ impl BitrotReaderSource {
&self.path,
self.offset,
self.length,
self.small_shard,
self.use_mmap_read,
self.stage_metrics.map(|metrics| metrics.path),
)
@@ -170,6 +172,7 @@ async fn open_reader_source(
path: &str,
offset: usize,
length: usize,
small_shard: bool,
use_mmap_read: bool,
metrics_path: Option<&'static str>,
) -> disk::error::Result<Option<BoxedObjectReader>> {
@@ -182,14 +185,40 @@ async fn open_reader_source(
} else if let Some(disk) = disk {
#[cfg(test)]
crate::set_disk::disk_call_counters::record_shard_open(path, false);
open_disk_reader(disk, bucket, path, offset, length, use_mmap_read, metrics_path)
.await
.map(Some)
let buffer_stream = small_shard && length >= MIN_BUFFERED_STREAM_BYTES;
let reader = open_disk_reader(disk, bucket, path, offset, length, use_mmap_read, metrics_path).await?;
let reader = if buffer_stream && matches!(reader, ShardReader::Stream(_)) && disk.is_local() {
buffer_local_shard_stream(reader, length)?
} else {
reader
};
Ok(Some(reader))
} else {
Ok(None)
}
}
const MIN_BUFFERED_STREAM_BYTES: usize = 1024 * 1024;
fn buffer_local_shard_stream(reader: ShardReader, length: usize) -> disk::error::Result<ShardReader> {
if length < MIN_BUFFERED_STREAM_BYTES {
return Ok(reader);
}
let ShardReader::Stream(reader) = reader else {
return Ok(reader);
};
let limit = u64::try_from(length).map_err(|_| DiskError::FileCorrupt)?;
// Small layout stripes otherwise dispatch a blocking file read per frame.
// Short ranges keep direct reads: the caller's conservative stream span
// can include an extra stripe beyond the final requested block. Long
// streams stay bounded to that span; prefetched frames still authenticate
// individually when consumed by the bitrot reader.
Ok(ShardReader::Stream(Box::new(tokio::io::BufReader::with_capacity(
256 * 1024,
reader.take(limit),
))))
}
struct FirstReadMetricsReader {
inner: FileReader,
metrics_path: &'static str,
@@ -555,6 +584,29 @@ fn bitrot_encoded_range(offset: usize, length: usize, shard_size: usize, checksu
adjust_shard_read_params(offset, length, shard_size, &checksum_algo)
}
pub(crate) fn bitrot_encoded_range_with_frame_size(
offset: usize,
length: usize,
shard_size: usize,
frame_size: usize,
checksum_algo: HashAlgorithm,
) -> (usize, usize) {
if frame_size == shard_size {
return bitrot_encoded_range(offset, length, shard_size, checksum_algo);
}
// A frame-aligned range may begin inside a logical shard. Skip the hashes
// in preceding logical shards as well as those before this frame.
let frames_per_shard = shard_size.div_ceil(frame_size);
let hash_size = checksum_algo.size();
let logical_shards = offset / shard_size;
let within_shard = offset % shard_size;
let frame_index = within_shard / frame_size;
let physical_offset = logical_shards * (shard_size + frames_per_shard * hash_size) + within_shard + frame_index * hash_size;
let physical_length = length + length.div_ceil(frame_size) * hash_size;
(physical_offset, physical_length)
}
/// Adjusts a raw (offset, length) pair to account for per-shard checksum overhead.
/// Returns (adjusted_offset, adjusted_length).
pub(crate) fn adjust_shard_read_params(
@@ -686,12 +738,44 @@ pub(crate) async fn create_bitrot_reader_from_bytes_with_stage_metrics(
skip_verify: bool,
use_mmap_read: bool,
stage_metrics: Option<BitrotReaderStageMetrics>,
) -> disk::error::Result<Option<BitrotReader<ShardReader>>> {
create_bitrot_reader_from_bytes_with_frame_size_and_stage_metrics(
inline_data,
disk,
bucket,
path,
offset,
length,
shard_size,
shard_size,
checksum_algo,
skip_verify,
use_mmap_read,
stage_metrics,
)
.await
}
#[allow(clippy::too_many_arguments)]
pub(crate) async fn create_bitrot_reader_from_bytes_with_frame_size_and_stage_metrics(
inline_data: Option<Bytes>,
disk: Option<&DiskStore>,
bucket: &str,
path: &str,
offset: usize,
length: usize,
shard_size: usize,
frame_size: usize,
checksum_algo: HashAlgorithm,
skip_verify: bool,
use_mmap_read: bool,
stage_metrics: Option<BitrotReaderStageMetrics>,
) -> disk::error::Result<Option<BitrotReader<ShardReader>>> {
let stage_metrics = stage_metrics.filter(|_| rustfs_io_metrics::get_stage_metrics_enabled());
let stage_metrics_enabled = stage_metrics.is_some();
let reader_construction_start = stage_metrics_enabled.then(Instant::now);
let (offset, length) = bitrot_encoded_range(offset, length, shard_size, checksum_algo.clone());
let (offset, length) = bitrot_encoded_range_with_frame_size(offset, length, shard_size, frame_size, checksum_algo.clone());
if let Some(metrics) = stage_metrics {
record_get_stage_duration_if_enabled(metrics.path, metrics.reader_construction_stage, reader_construction_start);
}
@@ -704,6 +788,7 @@ pub(crate) async fn create_bitrot_reader_from_bytes_with_stage_metrics(
path,
offset,
length,
shard_size != 0 && shard_size <= 128 * 1024,
use_mmap_read,
stage_metrics.map(|metrics| metrics.path),
)
@@ -713,7 +798,8 @@ pub(crate) async fn create_bitrot_reader_from_bytes_with_stage_metrics(
}
let bitrot_reader_init_start = stage_metrics_enabled.then(Instant::now);
let reader = reader.map(|reader| BitrotReader::new(reader, shard_size, checksum_algo, skip_verify));
let reader =
reader.map(|reader| BitrotReader::new_with_frame_size(reader, shard_size, frame_size, checksum_algo, skip_verify));
if let Some(metrics) = stage_metrics {
record_get_stage_duration_if_enabled(metrics.path, metrics.bitrot_reader_init_stage, bitrot_reader_init_start);
}
@@ -766,8 +852,38 @@ pub(crate) fn create_deferred_bitrot_reader_with_stripe_handle(
skip_verify: bool,
use_mmap_read: bool,
) -> (BitrotReader<ShardReader>, DeferredReaderStripeHandle) {
let stripe_stride = shard_size + checksum_algo.size();
let (offset, length) = bitrot_encoded_range(offset, length, shard_size, checksum_algo.clone());
create_deferred_bitrot_reader_with_stripe_handle_and_frame_size(
inline_data,
disk,
bucket,
path,
offset,
length,
shard_size,
shard_size,
checksum_algo,
skip_verify,
use_mmap_read,
)
}
#[allow(clippy::too_many_arguments)]
pub(crate) fn create_deferred_bitrot_reader_with_stripe_handle_and_frame_size(
inline_data: Option<Bytes>,
disk: Option<DiskStore>,
bucket: &str,
path: &str,
offset: usize,
length: usize,
shard_size: usize,
frame_size: usize,
checksum_algo: HashAlgorithm,
skip_verify: bool,
use_mmap_read: bool,
) -> (BitrotReader<ShardReader>, DeferredReaderStripeHandle) {
let frames_per_shard = shard_size.div_ceil(frame_size);
let stripe_stride = shard_size + frames_per_shard * checksum_algo.size();
let (offset, length) = bitrot_encoded_range_with_frame_size(offset, length, shard_size, frame_size, checksum_algo.clone());
let inline_source = inline_data.is_some();
let source = BitrotReaderSource {
inline_data,
@@ -776,6 +892,7 @@ pub(crate) fn create_deferred_bitrot_reader_with_stripe_handle(
path: if inline_source { String::new() } else { path.to_string() },
offset,
length,
small_shard: shard_size != 0 && shard_size <= 128 * 1024,
use_mmap_read,
stage_metrics: None,
};
@@ -785,7 +902,13 @@ pub(crate) fn create_deferred_bitrot_reader_with_stripe_handle(
// The deferred parity reader opens its source lazily, so it cannot hand out an
// in-memory block up front; it stays on the streaming path. Parity shards are
// only read when a data shard fails, so the fast path is not needed here.
let reader = BitrotReader::new(ShardReader::Stream(Box::new(deferred)), shard_size, checksum_algo, skip_verify);
let reader = BitrotReader::new_with_frame_size(
ShardReader::Stream(Box::new(deferred)),
shard_size,
frame_size,
checksum_algo,
skip_verify,
);
(reader, handle)
}
@@ -864,6 +987,184 @@ mod tests {
use super::*;
use rustfs_rio::ChunkReader;
use std::collections::VecDeque;
use std::sync::atomic::{AtomicUsize, Ordering};
struct CountingWindowReader {
source: Cursor<Vec<u8>>,
reads: Arc<AtomicUsize>,
bytes: Arc<AtomicUsize>,
}
impl AsyncRead for CountingWindowReader {
fn poll_read(mut self: Pin<&mut Self>, cx: &mut Context<'_>, out: &mut ReadBuf<'_>) -> Poll<io::Result<()>> {
let before = out.filled().len();
let result = Pin::new(&mut self.source).poll_read(cx, out);
self.reads.fetch_add(1, Ordering::Relaxed);
self.bytes.fetch_add(out.filled().len() - before, Ordering::Relaxed);
result
}
}
#[tokio::test]
async fn buffered_local_shard_stream_coalesces_reads_without_crossing_window() {
const SHARD: usize = 32 * 1024;
const TAIL: usize = 12_001;
let algo = HashAlgorithm::HighwayHash256S;
let data = vec![37; 37 * SHARD + TAIL];
let mut encoded = Vec::new();
{
let mut writer = crate::erasure::coding::BitrotWriter::new(&mut encoded, SHARD, algo.clone());
for chunk in data.chunks(SHARD) {
writer.write(chunk).await.expect("encode full shards and short tail");
}
}
let window = encoded.len();
encoded.extend_from_slice(&vec![99; 100_000]);
let reads = Arc::new(AtomicUsize::new(0));
let bytes = Arc::new(AtomicUsize::new(0));
let source = ShardReader::Stream(Box::new(CountingWindowReader {
source: Cursor::new(encoded),
reads: Arc::clone(&reads),
bytes: Arc::clone(&bytes),
}));
let source = buffer_local_shard_stream(source, window).expect("buffer the exact local window");
let mut reader = BitrotReader::new(source, SHARD, algo, false);
let mut actual = Vec::new();
for chunk in data.chunks(SHARD) {
reader
.read_appending(&mut actual, chunk.len())
.await
.expect("verify each buffered shard");
}
assert_eq!(actual, data);
assert_eq!(
bytes.load(Ordering::Relaxed),
window,
"prefetch must not consume the sentinel beyond the window"
);
assert_eq!(
reads.load(Ordering::Relaxed),
5,
"thirty-eight logical reads should share five bounded source reads"
);
let mut extra = vec![0; 1];
assert_eq!(
reader.read(&mut extra).await.expect_err("window ended").kind(),
io::ErrorKind::UnexpectedEof
);
assert_eq!(bytes.load(Ordering::Relaxed), window);
}
#[tokio::test]
async fn buffered_local_shard_stream_still_rejects_corrupt_prefetched_frame() {
const SHARD: usize = 32 * 1024;
let algo = HashAlgorithm::HighwayHash256S;
let payload = vec![73; SHARD];
let mut encoded = Vec::new();
{
let mut writer = crate::erasure::coding::BitrotWriter::new(&mut encoded, SHARD, algo.clone());
for _ in 0..33 {
writer.write(&payload).await.expect("encode fixture shards");
}
}
encoded[SHARD + algo.size() + algo.size() + 7] ^= 1;
let window = encoded.len();
let reads = Arc::new(AtomicUsize::new(0));
let bytes = Arc::new(AtomicUsize::new(0));
let source = buffer_local_shard_stream(
ShardReader::Stream(Box::new(CountingWindowReader {
source: Cursor::new(encoded),
reads: Arc::clone(&reads),
bytes: Arc::clone(&bytes),
})),
window,
)
.expect("buffer corruption fixture");
let mut reader = BitrotReader::new(source, SHARD, algo, false);
let mut output = vec![0xa5; SHARD];
reader.read(&mut output).await.expect("first frame is valid");
assert_eq!(output, payload);
assert_eq!(reads.load(Ordering::Relaxed), 1);
assert_eq!(
bytes.load(Ordering::Relaxed),
256 * 1024,
"long stream must prefetch the corrupt second frame"
);
output.fill(0xa5);
assert_eq!(
reader
.read(&mut output)
.await
.expect_err("corrupt second frame must fail")
.kind(),
io::ErrorKind::InvalidData
);
assert!(
output.iter().all(|byte| *byte == 0xa5),
"corrupt prefetched data must not escape verification"
);
assert_eq!(
reads.load(Ordering::Relaxed),
1,
"corrupt prefetched frame must fail without another source read"
);
}
#[tokio::test]
async fn buffered_local_shard_stream_does_not_prefetch_conservative_short_range_tail() {
const BLOCK: usize = 256 * 1024;
const SHARD: usize = BLOCK / 2;
let algo = HashAlgorithm::HighwayHash256S;
let erasure = crate::erasure::coding::Erasure::new(2, 2, BLOCK);
// An aligned 1 MiB Range consumes four data-shard frames, while the
// existing reader planner conservatively opens five logical shards.
let logical_window = erasure.shard_file_offset(0, 4 * BLOCK, 6 * BLOCK);
assert_eq!(logical_window, 5 * SHARD);
let (_, wire_window) = adjust_shard_read_params(0, logical_window, SHARD, &algo);
let payload = vec![73; SHARD];
let mut encoded = Vec::new();
{
let mut writer = crate::erasure::coding::BitrotWriter::new(&mut encoded, SHARD, algo.clone());
for _ in 0..4 {
writer.write(&payload).await.expect("encode requested range frames");
}
writer
.write(&vec![99; SHARD])
.await
.expect("encode unrequested conservative tail frame");
}
assert_eq!(encoded.len(), wire_window);
let reads = Arc::new(AtomicUsize::new(0));
let bytes = Arc::new(AtomicUsize::new(0));
let source = buffer_local_shard_stream(
ShardReader::Stream(Box::new(CountingWindowReader {
source: Cursor::new(encoded),
reads: Arc::clone(&reads),
bytes: Arc::clone(&bytes),
})),
wire_window,
)
.expect("open conservative short range stream");
let mut reader = BitrotReader::new(source, SHARD, algo.clone(), false);
let mut output = Vec::new();
for _ in 0..4 {
reader
.read_appending(&mut output, SHARD)
.await
.expect("read and authenticate requested frame");
}
assert_eq!(output, payload.repeat(4));
assert_eq!(
bytes.load(Ordering::Relaxed),
4 * (SHARD + algo.size()),
"four decoded frames must not prefetch the unrequested fifth frame"
);
assert_eq!(
reads.load(Ordering::Relaxed),
4,
"short range should retain direct per-frame source reads"
);
}
#[test]
fn bitrot_create_file_size_grows_known_length_by_checksums() {
@@ -1038,6 +1339,61 @@ mod tests {
(disk, dir)
}
#[tokio::test]
async fn deferred_local_opener_applies_buffer_gate_to_current_window() {
use tokio::io::AsyncReadExt;
const SHARD: usize = 128 * 1024;
const WINDOW: usize = 1024 * 1024;
let (disk, _dir) = local_test_disk().await;
let payload: Vec<_> = (0u8..=250).cycle().take(2 * WINDOW + 1).collect();
disk.make_volume("test-bucket").await.expect("create test volume");
disk.write_all("test-bucket", "obj/buffer-gate", Bytes::from(payload.clone()))
.await
.expect("write source with bytes beyond every requested window");
for (shard_size, length, advance, expected_end) in [
(SHARD, WINDOW, 0, WINDOW),
(SHARD + 1, WINDOW, 0, payload.len()),
(SHARD, WINDOW + SHARD, 1, WINDOW + SHARD),
(SHARD, WINDOW, 1, payload.len()),
] {
let (_reader, handle) = create_deferred_bitrot_reader_with_stripe_handle(
None,
Some(disk.clone()),
"test-bucket",
"obj/buffer-gate",
0,
length,
shard_size,
HashAlgorithm::None,
false,
false,
);
assert!(handle.advance_stripes(advance), "advance the unopened source");
let source = {
let mut state = handle.state.lock().expect("lock pending reader state");
let DeferredObjectReaderState::Pending(source) = &mut *state else {
panic!("advancing stripes must leave the source unopened");
};
source.take().expect("pending source must be available")
};
let mut source = source
.open()
.await
.expect("open advanced local source")
.expect("disk source must produce a reader");
let mut actual = Vec::new();
source.read_to_end(&mut actual).await.expect("read the opened source");
// Unbuffered local streams expose the file remainder; buffered streams
// must stop at the current encoded window, including after an advance.
assert_eq!(
actual.as_slice(),
&payload[advance * shard_size..expected_end],
"wrong buffering boundary for shard {shard_size}, advance {advance}"
);
}
}
#[cfg(feature = "hotpath")]
async fn remote_test_disk(transport: TestRemoteDataTransport) -> (DiskStore, TestRemoteDataTransport) {
use crate::disk::endpoint::Endpoint;
@@ -0,0 +1,441 @@
//! Request and storage contracts for the per-object erasure-code block size.
//!
//! The request hint is deliberately kept separate from persisted file
//! metadata. Application code resolves the hint and writes the selected
//! value to the transient marker below; the set layer consumes that marker
//! before constructing `FileInfo`. Readers use the block size persisted in
//! `FileInfo::erasure`, never the request metadata.
use std::collections::HashMap;
use std::fmt;
/// The historical RustFS erasure block size.
pub const DEFAULT_EC_BLOCK_SIZE: usize = 1024 * 1024;
/// The only block sizes accepted for new objects.
pub const EC_BLOCK_SIZE_CANDIDATES: [usize; 4] = [64 * 1024, 256 * 1024, 1024 * 1024, 4 * 1024 * 1024];
/// User metadata key after the `x-amz-meta-` prefix has been removed.
pub const EC_BLOCK_SIZE_HINT_METADATA_KEY: &str = "rustfs-ec-block-size-hint";
/// Internal marker passed from the request layer to the storage writer.
/// It is always removed before user metadata is persisted.
pub const EC_BLOCK_SIZE_HINT_INTERNAL_SUFFIX: &str = "ec-block-size-hint";
pub const EC_BLOCK_SIZE_HINT_INTERNAL_KEY: &str = "x-rustfs-internal-ec-block-size-hint";
/// Internal projection copied into `ObjectInfo` for response construction.
/// The projection is filtered from S3 user metadata.
pub const EC_BLOCK_SIZE_INFO_INTERNAL_SUFFIX: &str = "ec-block-size";
pub const EC_BLOCK_SIZE_INFO_INTERNAL_KEY: &str = "x-rustfs-internal-ec-block-size";
pub const EC_BLOCK_SIZE_RESPONSE_HEADER: &str = "x-rustfs-effective-ec-block-size";
pub const EC_BLOCK_SIZE_HINT_STATUS_HEADER: &str = "x-rustfs-layout-hint-status";
pub const EC_BLOCK_SIZE_HINT_REASON_HEADER: &str = "x-rustfs-layout-hint-reason";
/// Persisted marker for the per-shard bitrot frame size used by new EC
/// objects. The value is internal metadata and is always written through the
/// RustFS/MinIO compatibility-key pair.
pub const EC_READ_QUANTUM_INFO_INTERNAL_SUFFIX: &str = "ec-read-quantum";
pub const EC_READ_QUANTUM_INFO_INTERNAL_KEY: &str = "x-rustfs-internal-ec-read-quantum";
/// Keep the physical read quantum bounded even when an object chooses a large
/// layout block. The smaller entries cover the candidate layouts whose shard
/// size is below the target quantum.
pub const EC_READ_QUANTUM_CANDIDATES: [usize; 9] = [
1024 * 1024,
512 * 1024,
256 * 1024,
128 * 1024,
64 * 1024,
32 * 1024,
16 * 1024,
8 * 1024,
4 * 1024,
];
const EC_READ_QUANTUM_LEGACY_CANDIDATES: [usize; 7] =
[256 * 1024, 128 * 1024, 64 * 1024, 32 * 1024, 16 * 1024, 8 * 1024, 4 * 1024];
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum EcReadQuantumMetadataError {
ConflictingValues,
InvalidValue,
UnsupportedGeometry,
}
impl fmt::Display for EcReadQuantumMetadataError {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
f.write_str(match self {
Self::ConflictingValues => "EC read quantum metadata has conflicting compatibility values",
Self::InvalidValue => "EC read quantum metadata is not a supported decimal value",
Self::UnsupportedGeometry => "EC read quantum metadata does not match the object geometry",
})
}
}
impl std::error::Error for EcReadQuantumMetadataError {}
/// A syntactically valid request value. It may still be outside the
/// supported candidate set.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub struct ClientEcBlockSizeHint {
pub requested_bytes: u64,
}
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum EcBlockSizeHintError {
Empty,
TooLong,
NonAsciiDecimal,
Zero,
Overflow,
}
impl fmt::Display for EcBlockSizeHintError {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
let message = match self {
Self::Empty => "value is empty",
Self::TooLong => "value has more than 20 digits",
Self::NonAsciiDecimal => "value must contain only ASCII decimal digits",
Self::Zero => "value must be greater than zero",
Self::Overflow => "value does not fit in an unsigned 64-bit integer",
};
f.write_str(message)
}
}
impl std::error::Error for EcBlockSizeHintError {}
/// Parse the strict wire representation required by the SPEC.
pub fn parse_ec_block_size_hint(value: &str) -> Result<ClientEcBlockSizeHint, EcBlockSizeHintError> {
if value.is_empty() {
return Err(EcBlockSizeHintError::Empty);
}
if value.len() > 20 {
return Err(EcBlockSizeHintError::TooLong);
}
if !value.bytes().all(|byte| byte.is_ascii_digit()) {
return Err(EcBlockSizeHintError::NonAsciiDecimal);
}
let requested_bytes = value.parse::<u64>().map_err(|_| EcBlockSizeHintError::Overflow)?;
if requested_bytes == 0 {
return Err(EcBlockSizeHintError::Zero);
}
Ok(ClientEcBlockSizeHint { requested_bytes })
}
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum EcBlockSizeHintIgnoreReason {
Disabled,
FleetNotConfirmed,
UnsupportedSize,
IneligibleWrite,
}
impl EcBlockSizeHintIgnoreReason {
pub const fn as_str(self) -> &'static str {
match self {
Self::Disabled => "disabled",
Self::FleetNotConfirmed => "fleet-unconfirmed",
Self::UnsupportedSize => "unsupported-size",
Self::IneligibleWrite => "unsupported-write-mode",
}
}
}
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum EcBlockSizeHintOutcome {
NoHint,
Applied,
Ignored(EcBlockSizeHintIgnoreReason),
}
impl EcBlockSizeHintOutcome {
pub const fn status_header(self) -> Option<&'static str> {
match self {
Self::NoHint => None,
Self::Applied => Some("applied"),
Self::Ignored(_) => Some("ignored"),
}
}
pub const fn reason_header(self) -> Option<&'static str> {
match self {
Self::Ignored(reason) => Some(reason.as_str()),
Self::NoHint | Self::Applied => None,
}
}
}
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub struct ResolvedEcBlockSize {
pub effective_block_size: usize,
pub outcome: EcBlockSizeHintOutcome,
}
/// Resolve a request hint after authentication and eligibility checks.
///
/// Disabled and fleet-unconfirmed requests deliberately do not parse the
/// value. In those rollout phases the header remains ordinary user metadata,
/// including when its value would be malformed.
pub fn resolve_ec_block_size_hint(
raw_hint: Option<&str>,
feature_enabled: bool,
fleet_confirmed: bool,
eligible_write: bool,
) -> Result<ResolvedEcBlockSize, EcBlockSizeHintError> {
let Some(raw_hint) = raw_hint else {
return Ok(ResolvedEcBlockSize {
effective_block_size: DEFAULT_EC_BLOCK_SIZE,
outcome: EcBlockSizeHintOutcome::NoHint,
});
};
if !feature_enabled {
return Ok(ResolvedEcBlockSize {
effective_block_size: DEFAULT_EC_BLOCK_SIZE,
outcome: EcBlockSizeHintOutcome::Ignored(EcBlockSizeHintIgnoreReason::Disabled),
});
}
if !fleet_confirmed {
return Ok(ResolvedEcBlockSize {
effective_block_size: DEFAULT_EC_BLOCK_SIZE,
outcome: EcBlockSizeHintOutcome::Ignored(EcBlockSizeHintIgnoreReason::FleetNotConfirmed),
});
}
let hint = parse_ec_block_size_hint(raw_hint)?;
let Some(effective_block_size) = usize::try_from(hint.requested_bytes)
.ok()
.filter(|size| EC_BLOCK_SIZE_CANDIDATES.contains(size))
else {
return Ok(ResolvedEcBlockSize {
effective_block_size: DEFAULT_EC_BLOCK_SIZE,
outcome: EcBlockSizeHintOutcome::Ignored(EcBlockSizeHintIgnoreReason::UnsupportedSize),
});
};
if !eligible_write {
return Ok(ResolvedEcBlockSize {
effective_block_size: DEFAULT_EC_BLOCK_SIZE,
outcome: EcBlockSizeHintOutcome::Ignored(EcBlockSizeHintIgnoreReason::IneligibleWrite),
});
}
Ok(ResolvedEcBlockSize {
effective_block_size,
outcome: EcBlockSizeHintOutcome::Applied,
})
}
pub fn is_supported_ec_block_size(value: usize) -> bool {
EC_BLOCK_SIZE_CANDIDATES.contains(&value)
}
/// Return whether the storage writer rollout is enabled and fleet-confirmed.
/// Read it on the write path so each new object uses the active process
/// environment values.
pub fn ec_read_quantum_write_enabled() -> bool {
rustfs_utils::get_env_bool(rustfs_config::ENV_EC_READ_QUANTUM_ENABLE, rustfs_config::DEFAULT_EC_READ_QUANTUM_ENABLE)
&& rustfs_utils::get_env_bool(
rustfs_config::ENV_EC_READ_QUANTUM_FLEET_CONFIRMED,
rustfs_config::DEFAULT_EC_READ_QUANTUM_FLEET_CONFIRMED,
)
}
/// Select a frame size that preserves exact EC geometry. A candidate must be
/// a divisor of both the physical shard size and the logical layout block
/// divided across data shards; this keeps every complete layout stripe aligned
/// and leaves only the final object stripe short.
pub fn select_ec_read_quantum(block_size: usize, data_shards: usize, shard_size: usize) -> Option<usize> {
if block_size == 0 || data_shards == 0 || shard_size == 0 {
return None;
}
// Small layouts benefit from half-shard frames because their common range
// requests otherwise pull a complete data-plus-parity stripe. Larger
// layouts use a coarser target to cap authenticated-frame count during
// upload: the 1 MiB layout fits one frame per shard and the 4 MiB layout
// uses two 1 MiB frames per shard. A 4 MiB range still avoids the legacy
// multi-megabyte stripe read while PUT avoids paying for eight hashes per
// shard.
let target = if block_size <= 256 * 1024 {
shard_size / 2
} else {
1024 * 1024
};
EC_READ_QUANTUM_CANDIDATES.into_iter().find(|candidate| {
*candidate <= target && ec_read_quantum_matches_geometry(*candidate, block_size, data_shards, shard_size)
})
}
fn select_legacy_ec_read_quantum(block_size: usize, data_shards: usize, shard_size: usize) -> Option<usize> {
if block_size == 0 || data_shards == 0 || shard_size == 0 {
return None;
}
EC_READ_QUANTUM_LEGACY_CANDIDATES
.into_iter()
.find(|candidate| ec_read_quantum_matches_geometry(*candidate, block_size, data_shards, shard_size))
}
fn ec_read_quantum_matches_geometry(quantum: usize, block_size: usize, data_shards: usize, shard_size: usize) -> bool {
quantum > 0
&& quantum <= shard_size
&& EC_READ_QUANTUM_CANDIDATES.contains(&quantum)
&& shard_size.is_multiple_of(quantum)
&& quantum
.checked_mul(data_shards)
.is_some_and(|logical_quantum| block_size.is_multiple_of(logical_quantum))
}
/// Resolve a persisted marker. Absence is the legacy layout-frame format;
/// presence with a malformed or conflicting value is an error so a damaged
/// metadata record cannot silently reinterpret the shard stream.
pub fn persisted_ec_read_quantum(
metadata: &HashMap<String, String>,
block_size: usize,
data_shards: usize,
shard_size: usize,
) -> Result<Option<usize>, EcReadQuantumMetadataError> {
let present = rustfs_utils::http::metadata_compat::contains_key_str(metadata, EC_READ_QUANTUM_INFO_INTERNAL_SUFFIX);
let Some(value) = rustfs_utils::http::metadata_compat::get_consistent_str(metadata, EC_READ_QUANTUM_INFO_INTERNAL_SUFFIX)
else {
return if present {
Err(EcReadQuantumMetadataError::ConflictingValues)
} else {
Ok(None)
};
};
let quantum = value.parse::<usize>().map_err(|_| EcReadQuantumMetadataError::InvalidValue)?;
// Accept markers written by the previous selector as well as the current
// smaller target. Existing 64 KiB and 256 KiB objects may carry a
// full-shard marker and must remain readable after the rollout changes.
let current = select_ec_read_quantum(block_size, data_shards, shard_size);
let legacy = select_legacy_ec_read_quantum(block_size, data_shards, shard_size);
if current != Some(quantum) && legacy != Some(quantum) {
return Err(EcReadQuantumMetadataError::UnsupportedGeometry);
}
Ok(Some(quantum))
}
pub fn insert_ec_read_quantum(metadata: &mut HashMap<String, String>, quantum: usize) {
rustfs_utils::http::metadata_compat::insert_str(metadata, EC_READ_QUANTUM_INFO_INTERNAL_SUFFIX, quantum.to_string());
}
/// Consume a trusted transient marker from storage options.
pub fn take_ec_block_size_hint(metadata: &mut std::collections::HashMap<String, String>) -> Option<usize> {
let value = rustfs_utils::http::metadata_compat::get_consistent_str(metadata, EC_BLOCK_SIZE_HINT_INTERNAL_SUFFIX)
.map(ToOwned::to_owned);
rustfs_utils::http::metadata_compat::remove_str(metadata, EC_BLOCK_SIZE_HINT_INTERNAL_SUFFIX);
let value = value?;
let parsed = value.parse::<usize>().ok()?;
is_supported_ec_block_size(parsed).then_some(parsed)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn parser_accepts_decimal_values_and_rejects_wire_variants() {
assert_eq!(parse_ec_block_size_hint("65536").unwrap().requested_bytes, 65_536);
for value in ["", "0", "+65536", "65_536", "65536 ", "1.0", "128"] {
assert!(parse_ec_block_size_hint(value).is_err(), "{value:?} must be rejected");
}
assert_eq!(parse_ec_block_size_hint("18446744073709551615").unwrap().requested_bytes, u64::MAX);
assert!(matches!(
parse_ec_block_size_hint("18446744073709551616"),
Err(EcBlockSizeHintError::Overflow)
));
}
#[test]
fn resolver_applies_each_candidate_only_when_rollout_and_write_are_eligible() {
for candidate in EC_BLOCK_SIZE_CANDIDATES {
let result = resolve_ec_block_size_hint(Some(&candidate.to_string()), true, true, true).unwrap();
assert_eq!(result.effective_block_size, candidate);
assert_eq!(result.outcome, EcBlockSizeHintOutcome::Applied);
}
assert_eq!(
resolve_ec_block_size_hint(Some("131072"), true, true, true).unwrap().outcome,
EcBlockSizeHintOutcome::Ignored(EcBlockSizeHintIgnoreReason::UnsupportedSize)
);
let ineligible = resolve_ec_block_size_hint(Some("65536"), true, true, false).unwrap();
assert_eq!(
ineligible.outcome,
EcBlockSizeHintOutcome::Ignored(EcBlockSizeHintIgnoreReason::IneligibleWrite)
);
assert_eq!(ineligible.outcome.reason_header(), Some("unsupported-write-mode"));
}
#[test]
fn disabled_and_unconfirmed_rollouts_do_not_turn_malformed_metadata_into_errors() {
for (enabled, fleet, reason) in [
(false, false, EcBlockSizeHintIgnoreReason::Disabled),
(true, false, EcBlockSizeHintIgnoreReason::FleetNotConfirmed),
] {
let result = resolve_ec_block_size_hint(Some("not-a-number"), enabled, fleet, true).unwrap();
assert_eq!(result.effective_block_size, DEFAULT_EC_BLOCK_SIZE);
assert_eq!(result.outcome, EcBlockSizeHintOutcome::Ignored(reason));
}
}
#[test]
fn transient_marker_is_consumed_only_for_supported_candidates() {
let mut metadata = std::collections::HashMap::new();
rustfs_utils::http::metadata_compat::insert_str(&mut metadata, EC_BLOCK_SIZE_HINT_INTERNAL_SUFFIX, "262144".to_string());
assert_eq!(take_ec_block_size_hint(&mut metadata), Some(262_144));
assert!(!metadata.contains_key(EC_BLOCK_SIZE_HINT_INTERNAL_KEY));
assert!(!metadata.contains_key("x-minio-internal-ec-block-size-hint"));
let mut unsupported =
std::collections::HashMap::from([(EC_BLOCK_SIZE_HINT_INTERNAL_KEY.to_string(), "131072".to_string())]);
assert_eq!(take_ec_block_size_hint(&mut unsupported), None);
assert!(!unsupported.contains_key(EC_BLOCK_SIZE_HINT_INTERNAL_KEY));
let mut conflicting = std::collections::HashMap::from([
(EC_BLOCK_SIZE_HINT_INTERNAL_KEY.to_string(), "65536".to_string()),
("x-minio-internal-ec-block-size-hint".to_string(), "262144".to_string()),
]);
assert_eq!(take_ec_block_size_hint(&mut conflicting), None);
assert!(conflicting.is_empty());
}
#[test]
fn read_quantum_stays_aligned_with_layout_geometry() {
// Two data shards produce the same compact geometry used by the
// persisted 64 KiB, 256 KiB, 1 MiB and 4 MiB candidates. Large
// layouts deliberately use coarser frames to keep PUT overhead bounded.
assert_eq!(select_ec_read_quantum(64 * 1024, 2, 32 * 1024), Some(16 * 1024));
assert_eq!(select_ec_read_quantum(256 * 1024, 2, 128 * 1024), Some(64 * 1024));
assert_eq!(select_ec_read_quantum(1024 * 1024, 2, 512 * 1024), Some(512 * 1024));
assert_eq!(select_ec_read_quantum(4 * 1024 * 1024, 2, 2 * 1024 * 1024), Some(1024 * 1024));
}
#[test]
fn persisted_read_quantum_rejects_conflicts_and_bad_geometry() {
let mut metadata = HashMap::new();
insert_ec_read_quantum(&mut metadata, 256 * 1024);
assert_eq!(
persisted_ec_read_quantum(&metadata, 4 * 1024 * 1024, 2, 2 * 1024 * 1024),
Ok(Some(256 * 1024))
);
metadata.clear();
insert_ec_read_quantum(&mut metadata, 1024 * 1024);
assert_eq!(
persisted_ec_read_quantum(&metadata, 4 * 1024 * 1024, 2, 2 * 1024 * 1024),
Ok(Some(1024 * 1024))
);
metadata.insert("x-minio-internal-ec-read-quantum".to_owned(), "128".to_owned());
assert_eq!(
persisted_ec_read_quantum(&metadata, 4 * 1024 * 1024, 2, 2 * 1024 * 1024),
Err(EcReadQuantumMetadataError::ConflictingValues)
);
metadata.clear();
insert_ec_read_quantum(&mut metadata, 32 * 1024);
assert_eq!(persisted_ec_read_quantum(&metadata, 64 * 1024, 2, 32 * 1024), Ok(Some(32 * 1024)));
}
}
+2
View File
@@ -99,6 +99,7 @@ pub(crate) fn legacy_encrypted_range_seek_enabled() -> bool {
}
mod body_cache_hook;
pub mod ec_block_layout;
mod encryption;
mod hook_slot;
mod object_mutation_hook;
@@ -122,6 +123,7 @@ pub use body_cache_hook::{
pub(crate) use body_cache_hook::{
get_object_body_cache_hook, get_object_body_cache_hook_suppressed, without_get_object_body_cache_hook,
};
pub use ec_block_layout::*;
pub use encryption::{
EncryptionResolutionError, EncryptionResolutionErrorKind, ObjectEncryptionResolver, ReadEncryptionMaterial,
ReadEncryptionMode, ReadEncryptionRequest,
+73 -1
View File
@@ -1840,6 +1840,12 @@ pub struct ObjectInfo {
pub multipart_completion_replayed: bool,
/// Previous current version's size observed during `rename_data` commit.
pub backfilled_old_current_size: Option<crate::disk::OldCurrentSize>,
/// Persisted EC block size of a local version, projected from its erasure
/// layout for response headers. It is deliberately not part of
/// `user_defined`: every writer that turns `user_defined` back into stored
/// metadata would otherwise persist it. Read it through
/// [`ObjectInfo::effective_ec_block_size`].
pub ec_block_size: Option<usize>,
}
impl Default for ObjectInfo {
@@ -1883,6 +1889,7 @@ impl Default for ObjectInfo {
checksum: None,
multipart_completion_replayed: false,
backfilled_old_current_size: None,
ec_block_size: None,
}
}
}
@@ -1927,12 +1934,19 @@ impl Clone for ObjectInfo {
checksum: self.checksum.clone(),
multipart_completion_replayed: self.multipart_completion_replayed,
backfilled_old_current_size: self.backfilled_old_current_size,
ec_block_size: self.ec_block_size,
expires: self.expires,
}
}
}
impl ObjectInfo {
/// Return the persisted EC block size exposed by the response-side
/// projection. Callers must not infer this value from request metadata.
pub fn effective_ec_block_size(&self) -> Option<usize> {
self.ec_block_size
}
pub(crate) fn shard_integrity_write_mode(&self) -> ShardIntegrityWriteMode {
// Any declaration requires protection. Malformed declarations remain
// errors in the source reader and must never select the legacy path.
@@ -2246,9 +2260,10 @@ impl ObjectInfo {
let inlined = fi.inline_data();
let free_version = fi.tier_free_version();
let is_remote = fi.is_remote();
let metadata = std::mem::take(&mut fi.metadata);
let parts = std::mem::take(&mut fi.parts);
Self::from_file_info_parts(&fi, inlined, free_version, metadata, parts, bucket, object, version_id)
Self::from_file_info_parts(&fi, inlined, free_version, is_remote, metadata, parts, bucket, object, version_id)
}
pub(crate) fn from_file_info_with_version_id(
@@ -2261,6 +2276,7 @@ impl ObjectInfo {
fi,
fi.inline_data(),
fi.tier_free_version(),
fi.is_remote(),
fi.metadata.clone(),
fi.parts.clone(),
bucket,
@@ -2274,6 +2290,7 @@ impl ObjectInfo {
fi: &FileInfo,
inlined: bool,
free_version: bool,
is_remote: bool,
mut metadata: HashMap<String, String>,
parts: Vec<ObjectPartInfo>,
bucket: &str,
@@ -2349,6 +2366,11 @@ impl ObjectInfo {
};
clean_metadata(&mut metadata);
// The frame quantum is a storage-layout detail. Keep it in
// FileInfo for the reader, but never expose the internal marker as
// ordinary S3 metadata.
rustfs_utils::http::metadata_compat::remove_str(&mut metadata, crate::object_api::EC_READ_QUANTUM_INFO_INTERNAL_SUFFIX);
let ec_block_size = (!fi.deleted && !is_remote && fi.erasure.block_size > 0).then_some(fi.erasure.block_size);
let storage_class = Some(
storageclass::effective_class(
@@ -2408,6 +2430,7 @@ impl ObjectInfo {
version_purge_status_internal,
version_purge_status,
replication_decision,
ec_block_size,
..Default::default()
}
}
@@ -3322,6 +3345,55 @@ mod tests {
assert_eq!(info.version_id, None, "a normalized absent version must not be rewritten to nil");
}
#[test]
fn from_file_info_projects_persisted_ec_block_size_for_responses() {
let mut fi = FileInfo::new("bucket/object", 4, 2);
fi.size = 1;
fi.erasure.block_size = 64 * 1024;
let info = ObjectInfo::from_file_info(&fi, "bucket", "object", false);
assert_eq!(info.effective_ec_block_size(), Some(64 * 1024));
for key in ["x-rustfs-internal-ec-block-size", "x-minio-internal-ec-block-size"] {
assert!(
!info.user_defined.contains_key(key),
"the projection must not enter metadata that writers persist: {key}"
);
}
fi.erasure.block_size = 131_072;
let info = ObjectInfo::from_file_info(&fi, "bucket", "object", false);
assert_eq!(info.effective_ec_block_size(), Some(131_072));
fi.transition_status = rustfs_filemeta::TRANSITION_COMPLETE.to_string();
let info = ObjectInfo::from_file_info(&fi, "bucket", "remote-object", false);
assert_eq!(info.effective_ec_block_size(), None);
}
#[test]
fn from_owned_file_info_preserves_restored_ec_block_size_projection() {
let mut fi = FileInfo::new("bucket/object", 4, 2);
fi.size = 1;
fi.erasure.block_size = 64 * 1024;
fi.transition_status = rustfs_filemeta::TRANSITION_COMPLETE.to_string();
let restore = rustfs_filemeta::RestoreStatus {
is_restore_in_progress: Some(false),
restore_expiry_date: Some(OffsetDateTime::now_utc() + time::Duration::days(1)),
};
fi.metadata.insert(metadata_keys::RESTORE.to_string(), restore.to_string());
assert!(!fi.is_remote(), "the restored source has a local copy");
let borrowed = ObjectInfo::from_file_info(&fi, "bucket", "object", false);
let owned = ObjectInfo::from_owned_file_info(fi, "bucket", "object", false);
assert_eq!(owned.effective_ec_block_size(), Some(64 * 1024));
assert_eq!(borrowed.effective_ec_block_size(), Some(64 * 1024));
assert_eq!(owned.user_defined, borrowed.user_defined);
for key in ["x-rustfs-internal-ec-block-size", "x-minio-internal-ec-block-size"] {
assert!(!owned.user_defined.contains_key(key), "{key}");
}
}
#[test]
fn from_file_info_reports_effective_storage_class_for_legacy_metadata() {
for legacy_label in [
File diff suppressed because it is too large Load Diff
@@ -677,7 +677,11 @@ fn verify_inline_body_from_observations<'a>(
candidate.uses_legacy_checksum,
);
let shard_size = inline_erasure_shard_size(candidate.erasure.block_size, data_shards, candidate.uses_legacy_checksum);
if object_size == 0 || read_length == 0 || read_length > shard_size || data_files.len() < data_shards {
let frame_size =
crate::object_api::persisted_ec_read_quantum(&candidate.metadata, candidate.erasure.block_size, data_shards, shard_size)
.map_err(|_| GET_METADATA_EARLY_STOP_REASON_DATA_READ_INLINE_GEOMETRY)?
.unwrap_or(shard_size);
if object_size == 0 || read_length == 0 || frame_size == 0 || read_length > shard_size || data_files.len() < data_shards {
return Err(GET_METADATA_EARLY_STOP_REASON_DATA_READ_INLINE_BODY_VERIFY);
}
let shards_needed = object_size.div_ceil(read_length);
@@ -698,20 +702,32 @@ fn verify_inline_body_from_observations<'a>(
}
let hash_size = checksum_algo.size();
let need = hash_size + read_length;
let mut body = Vec::with_capacity(encoded_capacity);
for file in data_files.iter().take(shards_needed) {
let Some(raw) = file.data.as_deref() else {
return Err(GET_METADATA_EARLY_STOP_REASON_DATA_READ_INLINE_BODY_VERIFY);
};
let Some(block) = raw.get(..need) else {
return Err(GET_METADATA_EARLY_STOP_REASON_DATA_READ_INLINE_BODY_VERIFY);
};
let (hash, data) = block.split_at(hash_size);
if hash_size > 0 && checksum_algo.hash_encode(data).as_ref() != hash {
return Err(GET_METADATA_EARLY_STOP_REASON_DATA_READ_INLINE_BODY_VERIFY);
let mut remaining = read_length;
let mut cursor = 0usize;
while remaining > 0 {
let payload_len = remaining.min(frame_size);
let Some(frame_need) = hash_size.checked_add(payload_len) else {
return Err(GET_METADATA_EARLY_STOP_REASON_DATA_READ_INLINE_BODY_VERIFY);
};
let Some(end) = cursor.checked_add(frame_need) else {
return Err(GET_METADATA_EARLY_STOP_REASON_DATA_READ_INLINE_BODY_VERIFY);
};
let Some(block) = raw.get(cursor..end) else {
return Err(GET_METADATA_EARLY_STOP_REASON_DATA_READ_INLINE_BODY_VERIFY);
};
let (hash, data) = block.split_at(hash_size);
if hash_size > 0 && checksum_algo.hash_encode(data).as_ref() != hash {
return Err(GET_METADATA_EARLY_STOP_REASON_DATA_READ_INLINE_BODY_VERIFY);
}
body.extend_from_slice(data);
cursor = end;
remaining -= payload_len;
}
body.extend_from_slice(data);
if body.len() >= object_size {
body.truncate(object_size);
return Ok(bytes::Bytes::from(body));
+67 -10
View File
@@ -29,11 +29,21 @@ use sha2::Digest;
#[derive(Clone, Copy)]
struct FileInfoIdentityGroup {
hash: [u8; 32],
key: FileInfoQuorumKey,
count: usize,
mod_time: Option<OffsetDateTime>,
}
/// Metadata identity used when selecting a quorum member. The historical
/// content hash remains unchanged for compatibility with diagnostics and
/// external callers; this second digest prevents two objects with the same
/// metadata but different decode geometry from sharing a quorum.
#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)]
struct FileInfoQuorumKey {
content_hash: [u8; 32],
decode_layout_hash: [u8; 32],
}
impl SetDisks {
/// Read one key's complete version stack without enumerating its parent.
pub(crate) async fn read_object_versions_for_lifecycle(
@@ -670,7 +680,7 @@ impl SetDisks {
continue;
}
let key = Self::file_info_quorum_hash(meta);
let key = Self::file_info_quorum_key(meta);
let count = identity_counts.entry(key).or_insert(0);
*count += 1;
@@ -922,6 +932,26 @@ impl SetDisks {
key
}
fn file_info_quorum_key(meta: &FileInfo) -> FileInfoQuorumKey {
let mut hasher = Sha256::new();
Self::update_hash_str(&mut hasher, &meta.erasure.algorithm);
hasher.update([u8::from(meta.uses_legacy_checksum)]);
hasher.update(meta.erasure.data_blocks.to_le_bytes());
hasher.update(meta.erasure.parity_blocks.to_le_bytes());
hasher.update(meta.erasure.block_size.to_le_bytes());
hasher.update(meta.erasure.distribution.len().to_le_bytes());
for disk_index in &meta.erasure.distribution {
hasher.update(disk_index.to_le_bytes());
}
let digest = hasher.finalize();
let mut decode_layout_hash = [0u8; 32];
decode_layout_hash.copy_from_slice(digest.as_slice());
FileInfoQuorumKey {
content_hash: Self::file_info_quorum_hash(meta),
decode_layout_hash,
}
}
/// Hash the per-target delete-marker versions through their normalized form
/// so the dual internal prefixes carrying the same mapping share one
/// identity, while a genuine disagreement between disks still changes the
@@ -1012,14 +1042,14 @@ impl SetDisks {
continue;
}
let hash = Self::file_info_quorum_hash(meta);
if let Some(group) = groups.iter_mut().find(|group| group.hash == hash) {
let key = Self::file_info_quorum_key(meta);
if let Some(group) = groups.iter_mut().find(|group| group.key == key) {
group.count += 1;
continue;
}
groups.push(FileInfoIdentityGroup {
hash,
key,
count: 1,
mod_time: meta.mod_time,
});
@@ -1032,7 +1062,7 @@ impl SetDisks {
disks: &[Option<DiskStore>],
parts_metadata: &[FileInfo],
errs: &[Option<DiskError>],
hash: [u8; 32],
key: FileInfoQuorumKey,
quorum: usize,
) -> disk::error::Result<(Vec<Option<DiskStore>>, FileInfo)> {
let mut online_disks = vec![None; disks.len()];
@@ -1040,7 +1070,7 @@ impl SetDisks {
let mut count = 0;
for (i, ((meta, err), disk)) in parts_metadata.iter().zip(errs.iter()).zip(disks.iter()).enumerate() {
if err.is_some() || !file_info_is_valid_for_metadata(meta) || Self::file_info_quorum_hash(meta) != hash {
if err.is_some() || !file_info_is_valid_for_metadata(meta) || Self::file_info_quorum_key(meta) != key {
continue;
}
@@ -1081,7 +1111,7 @@ impl SetDisks {
let mut older_start = 0;
while older_start < groups.len() && groups[older_start].mod_time == latest_mod_time {
if groups[older_start].count >= write_quorum {
return Self::pick_fileinfo_identity(disks, parts_metadata, errs, groups[older_start].hash, write_quorum);
return Self::pick_fileinfo_identity(disks, parts_metadata, errs, groups[older_start].key, write_quorum);
}
older_start += 1;
}
@@ -1092,7 +1122,7 @@ impl SetDisks {
for group in groups.iter().skip(older_start) {
if group.count >= read_quorum {
return Self::pick_fileinfo_identity(disks, parts_metadata, errs, group.hash, read_quorum);
return Self::pick_fileinfo_identity(disks, parts_metadata, errs, group.key, read_quorum);
}
}
@@ -1186,7 +1216,7 @@ impl SetDisks {
let mod_valid = mod_time == &meta.mod_time;
if etag_only || mod_valid {
meta_hashes[i] = Some(Self::file_info_quorum_hash(meta));
meta_hashes[i] = Some(Self::file_info_quorum_key(meta));
} else {
debug!(
index = i,
@@ -1607,6 +1637,33 @@ mod tests {
assert_ne!(SetDisks::file_info_quorum_hash(&left), SetDisks::file_info_quorum_hash(&right));
}
#[test]
fn decode_geometry_is_required_for_fileinfo_quorum_selection() {
let mod_time = OffsetDateTime::from_unix_timestamp(1_705_312_300).expect("valid timestamp");
let mut base = metadata_quorum_test_fileinfo(mod_time, 1);
base.erasure.block_size = 64 * 1024;
let mut same_layout = vec![base; 3];
for (index, fi) in same_layout.iter_mut().enumerate() {
fi.erasure.index = index + 1;
}
let mut different_layout = same_layout[2].clone();
different_layout.erasure.block_size = 256 * 1024;
assert_eq!(
SetDisks::file_info_quorum_hash(&same_layout[0]),
SetDisks::file_info_quorum_hash(&different_layout),
"the compatibility hash remains unchanged when only decode geometry differs"
);
same_layout[2] = different_layout;
let selected = SetDisks::find_file_info_in_quorum(&same_layout, &Some(mod_time), &None, 2)
.expect("the two matching geometries should form a quorum");
assert_eq!(selected.erasure.block_size, 64 * 1024);
let err = SetDisks::find_file_info_in_quorum(&same_layout, &Some(mod_time), &None, 3)
.expect_err("mixed block sizes must never be combined into a quorum");
assert_eq!(err, DiskError::ErasureReadQuorum);
}
#[test]
fn purge_pending_quorum_hash_keeps_erasure_layouts_separate() {
let mod_time = OffsetDateTime::from_unix_timestamp(1_705_312_300).expect("valid timestamp");
+59 -3
View File
@@ -80,7 +80,7 @@ use crate::erasure::codec::bridge::{
use crate::erasure::coding;
use crate::error::{Error, Result, is_err_version_not_found};
use crate::error::{GenericError, ObjectApiError, is_err_object_not_found};
use crate::io_support::bitrot::{create_bitrot_reader_from_bytes, create_bitrot_writer, create_bitrot_writer_with_orig_volume};
use crate::io_support::bitrot::{create_bitrot_writer, create_bitrot_writer_with_orig_volume};
use crate::object_api::ObjectOptions;
use crate::object_api::get_object_body_cache_hook;
use crate::object_api::object_api_utils::get_raw_etag;
@@ -6164,6 +6164,7 @@ fn inline_admission_shard_size(erasure: &coding::Erasure, stored_size: i64, actu
HashReader::SIZE_PRESERVE_LAYER
}
#[allow(dead_code, reason = "compatibility wrapper for the legacy shard geometry")]
#[allow(clippy::too_many_arguments)]
async fn build_inline_bitrot_readers(
files: &[FileInfo],
@@ -6174,11 +6175,37 @@ async fn build_inline_bitrot_readers(
shard_size: usize,
checksum_algo: &HashAlgorithm,
skip_verify_bitrot: bool,
) -> disk::error::Result<Vec<Option<InlineBitrotReader>>> {
build_inline_bitrot_readers_with_frame_size(
files,
total_shards,
bucket,
object,
read_length,
shard_size,
shard_size,
checksum_algo,
skip_verify_bitrot,
)
.await
}
#[allow(clippy::too_many_arguments)]
async fn build_inline_bitrot_readers_with_frame_size(
files: &[FileInfo],
total_shards: usize,
bucket: &str,
object: &str,
read_length: usize,
shard_size: usize,
frame_size: usize,
checksum_algo: &HashAlgorithm,
skip_verify_bitrot: bool,
) -> disk::error::Result<Vec<Option<InlineBitrotReader>>> {
let mut readers = Vec::with_capacity(total_shards);
for file in files.iter().take(total_shards) {
let reader = if let Some(data) = &file.data {
create_bitrot_reader_from_bytes(
crate::io_support::bitrot::create_bitrot_reader_from_bytes_with_frame_size_and_stage_metrics(
Some(data.clone()),
None,
bucket,
@@ -6186,9 +6213,11 @@ async fn build_inline_bitrot_readers(
0,
read_length,
shard_size,
frame_size,
checksum_algo.clone(),
skip_verify_bitrot,
false,
None,
)
.await?
} else {
@@ -6199,6 +6228,7 @@ async fn build_inline_bitrot_readers(
Ok(readers)
}
#[allow(dead_code, reason = "compatibility wrapper for the legacy shard geometry")]
#[allow(clippy::too_many_arguments)]
async fn build_inline_bitrot_readers_from_refs(
files: &[&FileInfo],
@@ -6208,11 +6238,35 @@ async fn build_inline_bitrot_readers_from_refs(
shard_size: usize,
checksum_algo: &HashAlgorithm,
skip_verify_bitrot: bool,
) -> disk::error::Result<Vec<Option<InlineBitrotReader>>> {
build_inline_bitrot_readers_from_refs_with_frame_size(
files,
bucket,
object,
read_length,
shard_size,
shard_size,
checksum_algo,
skip_verify_bitrot,
)
.await
}
#[allow(clippy::too_many_arguments)]
async fn build_inline_bitrot_readers_from_refs_with_frame_size(
files: &[&FileInfo],
bucket: &str,
object: &str,
read_length: usize,
shard_size: usize,
frame_size: usize,
checksum_algo: &HashAlgorithm,
skip_verify_bitrot: bool,
) -> disk::error::Result<Vec<Option<InlineBitrotReader>>> {
let mut readers = Vec::with_capacity(files.len());
for file in files {
let reader = if let Some(data) = &file.data {
create_bitrot_reader_from_bytes(
crate::io_support::bitrot::create_bitrot_reader_from_bytes_with_frame_size_and_stage_metrics(
Some(data.clone()),
None,
bucket,
@@ -6220,9 +6274,11 @@ async fn build_inline_bitrot_readers_from_refs(
0,
read_length,
shard_size,
frame_size,
checksum_algo.clone(),
skip_verify_bitrot,
false,
None,
)
.await?
} else {
+242 -4
View File
@@ -52,7 +52,7 @@ fn pin_object_heal_future<F: std::future::Future>(build: impl FnOnce() -> F) ->
use crate::disk::DataDirDeleteStatus;
use crate::disk::DiskAPI;
use crate::disk::local::{DELETE_DATA_DIR_MARKER_PREFIX, metadata_less_part_file};
use crate::io_support::bitrot::{create_bitrot_reader_from_bytes, object_mmap_read_enabled};
use crate::io_support::bitrot::{create_bitrot_reader_from_bytes_with_frame_size_and_stage_metrics, object_mmap_read_enabled};
use crate::storage_api_contracts::namespace::NamespaceLocking as _;
use rustfs_common::mrf_channel::MrfDeleteMarkerPurge;
use rustfs_common::trace_bus::{TraceEvent, TraceFunc, TraceKind, trace_emit};
@@ -1703,6 +1703,14 @@ impl SetDisks {
if !latest_meta.deleted && !latest_meta.is_remote() {
let stage_result: disk::error::Result<Option<DiskError>> = async {
let erasure_info = latest_meta.erasure.clone();
let frame_size = crate::object_api::persisted_ec_read_quantum(
&latest_meta.metadata,
erasure.block_size,
erasure.data_shards,
erasure.shard_size(),
)
.map_err(|_| DiskError::FileCorrupt)?
.unwrap_or_else(|| erasure.shard_size());
let mut writer_failures = HealWriterFailures::new(disks_to_heal_count);
for (part_index, part) in latest_meta.parts.iter().enumerate() {
@@ -1745,7 +1753,7 @@ impl SetDisks {
&format!("part.{}", part.number),
]),
erasure.shard_file_size(part.size as i64),
erasure.shard_size(),
frame_size,
HashAlgorithm::HighwayHash256S,
)
.await
@@ -1812,7 +1820,7 @@ impl SetDisks {
checksum_info.algorithm
};
match create_bitrot_reader_from_bytes(
match create_bitrot_reader_from_bytes_with_frame_size_and_stage_metrics(
metadata.data.clone(),
Some(disk),
bucket,
@@ -1820,9 +1828,11 @@ impl SetDisks {
0,
till_offset,
erasure.shard_size(),
frame_size,
checksum_algo.clone(),
false,
use_mmap_read,
None,
)
.await
{
@@ -3975,7 +3985,10 @@ mod heal_result_report_tests {
use crate::disk::error::DiskError;
use crate::disk::format::FormatV3;
use crate::disk::local::RESERVED_DELETE_DATA_DIR_MARKER_PREFIX;
use crate::disk::{DiskAPI as _, DiskOption, DiskStore, RUSTFS_META_TMP_BUCKET, ReadOptions, STORAGE_FORMAT_FILE, new_disk};
use crate::disk::{
CHECK_PART_SUCCESS, DiskAPI as _, DiskOption, DiskStore, RUSTFS_META_TMP_BUCKET, ReadOptions, STORAGE_FORMAT_FILE,
new_disk,
};
use crate::error::Error;
use crate::object_api::{ObjectOptions, PutObjReader};
use crate::set_disk::ops::object::hermetic_set_disks_support::{
@@ -5653,6 +5666,231 @@ mod heal_result_report_tests {
}
}
#[tokio::test]
#[serial_test::serial]
async fn bounded_quantum_shards_pass_disk_verification_and_deep_heal() {
let (_dirs, disks, set) = hermetic_set_disks_isolated(4).await;
let bucket = "bounded-quantum-healthy";
for disk in &disks {
disk.make_volume(bucket).await.expect("create bounded-quantum test bucket");
}
for (block_size, quantum) in [
(64 * 1024, Some(16 * 1024)),
(1024 * 1024, None),
(4 * 1024 * 1024, Some(1024 * 1024)),
] {
let object = format!("block-{block_size}");
let mut options = ObjectOptions {
no_lock: true,
..Default::default()
};
options
.user_defined
.insert(crate::object_api::EC_BLOCK_SIZE_HINT_INTERNAL_KEY.to_owned(), block_size.to_string());
temp_env::async_with_vars(
[
(rustfs_config::ENV_EC_READ_QUANTUM_ENABLE, Some("true")),
(rustfs_config::ENV_EC_READ_QUANTUM_FLEET_CONFIRMED, Some("true")),
(rustfs_config::ENV_SHARD_INTEGRITY_WRITE, Some("false")),
],
async {
set.put_object(bucket, &object, &mut PutObjReader::from_vec(vec![0x6d; 4 * 1024 * 1024 + 123]), &options)
.await
.expect("write non-inline bounded-quantum fixture");
},
)
.await;
for (disk_index, disk) in disks.iter().enumerate() {
let metadata = disk
.read_version("", bucket, &object, "", &ReadOptions::default())
.await
.expect("read persisted bounded-quantum layout");
assert!(metadata.data.is_none(), "verification must open physical shards");
let erasure = &metadata.erasure;
assert_eq!(
crate::object_api::persisted_ec_read_quantum(
&metadata.metadata,
erasure.block_size,
erasure.data_blocks,
erasure.shard_size(),
)
.expect("stored frame geometry must be valid"),
quantum,
);
let verified = disk
.verify_file(bucket, &object, &metadata)
.await
.expect("verify healthy physical shard");
assert_eq!(verified.results, vec![CHECK_PART_SUCCESS], "block size {block_size}");
if disk_index == 0 && block_size == 64 * 1024 {
let mut invalid = metadata.clone();
crate::object_api::insert_ec_read_quantum(&mut invalid.metadata, 1);
assert_eq!(
disk.verify_file(bucket, &object, &invalid)
.await
.expect_err("unsupported frame geometry must fail closed"),
DiskError::FileCorrupt,
);
invalid = metadata.clone();
invalid
.metadata
.insert("x-minio-internal-ec-read-quantum".to_owned(), "32768".to_owned());
assert_eq!(
disk.verify_file(bucket, &object, &invalid)
.await
.expect_err("conflicting frame markers must not select a plausible layout"),
DiskError::FileCorrupt,
);
}
}
let (result, error) = set
.heal_object(
bucket,
&object,
"",
&HealOpts {
no_lock: true,
scan_mode: HealScanMode::Deep,
..Default::default()
},
)
.await
.expect("deep scan should accept persisted bounded-quantum data");
assert!(error.is_none(), "healthy bounded-quantum object must not require recovery: {error:?}");
assert_eq!(result.drives_healed(), Some(0));
assert!(
result
.after
.drives
.iter()
.all(|drive| drive.state == DriveState::Ok.to_string())
);
}
}
#[tokio::test]
#[serial_test::serial]
async fn bounded_quantum_heal_restores_exact_frames_and_range_bytes() {
use crate::storage_api_contracts::range::HTTPRangeSpec;
use tokio::io::AsyncReadExt;
let (dirs, disks, set) = hermetic_set_disks_isolated(4).await;
let bucket = "bounded-quantum-recovery";
for disk in &disks {
disk.make_volume(bucket)
.await
.expect("create bounded-quantum recovery bucket");
}
let payload: Vec<u8> = (0..4 * 1024 * 1024 + 123)
.map(|index| u8::try_from(index % 251).expect("fixture byte"))
.collect();
for (block_size, quantum) in [(64 * 1024, 16 * 1024), (4 * 1024 * 1024, 1024 * 1024)] {
for scan_mode in [HealScanMode::Normal, HealScanMode::Deep] {
let object = format!("block-{block_size}-{}", scan_mode.as_str());
let mut options = ObjectOptions {
no_lock: true,
..Default::default()
};
options
.user_defined
.insert(crate::object_api::EC_BLOCK_SIZE_HINT_INTERNAL_KEY.to_owned(), block_size.to_string());
temp_env::async_with_vars(
[
(rustfs_config::ENV_EC_READ_QUANTUM_ENABLE, Some("true")),
(rustfs_config::ENV_EC_READ_QUANTUM_FLEET_CONFIRMED, Some("true")),
(rustfs_config::ENV_SHARD_INTEGRITY_WRITE, Some("false")),
],
async {
set.put_object(bucket, &object, &mut PutObjReader::from_vec(payload.clone()), &options)
.await
.expect("write bounded-quantum recovery fixture");
},
)
.await;
let original = disks[0]
.read_version("", bucket, &object, "", &ReadOptions::default())
.await
.expect("read original shard layout");
let path = dirs[0]
.path()
.join(bucket)
.join(&object)
.join(original.data_dir.expect("external shard data directory").to_string())
.join("part.1");
let original_bytes = tokio::fs::read(&path).await.expect("snapshot encoded shard");
tokio::fs::remove_file(&path).await.expect("remove task-owned shard");
let (result, error) = set
.heal_object(
bucket,
&object,
"",
&HealOpts {
no_lock: true,
scan_mode,
..Default::default()
},
)
.await
.expect("bounded-quantum heal should finish");
assert!(error.is_none(), "bounded-quantum recovery must retain a read quorum: {error:?}");
assert_eq!(result.drives_healed(), Some(1));
let repaired = disks[0]
.read_version("", bucket, &object, "", &ReadOptions::default())
.await
.expect("read repaired shard metadata");
assert_eq!(
crate::object_api::persisted_ec_read_quantum(
&repaired.metadata,
repaired.erasure.block_size,
repaired.erasure.data_blocks,
repaired.erasure.shard_size(),
)
.expect("healed frame marker must match its geometry"),
Some(quantum),
);
let repaired_path = dirs[0]
.path()
.join(bucket)
.join(&object)
.join(repaired.data_dir.expect("repaired shard data directory").to_string())
.join("part.1");
assert_eq!(tokio::fs::read(repaired_path).await.expect("read restored frames"), original_bytes);
assert_eq!(
disks[0]
.verify_file(bucket, &object, &repaired)
.await
.expect("verify repaired shard")
.results,
vec![CHECK_PART_SUCCESS],
);
for (range, expected) in [
(None, payload.as_slice()),
(
Some(HTTPRangeSpec {
start: 16 * 1024 - 37,
end: 64 * 1024 + 85,
is_suffix_length: false,
}),
&payload[16 * 1024 - 37..64 * 1024 + 86],
),
] {
let mut reader = set
.get_object_reader(bucket, &object, range, Default::default(), &options)
.await
.expect("healed object must serve full and cross-frame range reads");
let mut body = Vec::new();
reader.stream.read_to_end(&mut body).await.expect("read healed object body");
assert_eq!(body, expected);
}
}
}
}
#[cfg(unix)]
#[tokio::test]
async fn deep_heal_legacy_verification_error_cannot_certify_healthy() {
+81 -4
View File
@@ -122,14 +122,14 @@ async fn create_part_writers(
disks: &[Option<DiskStore>],
path: &str,
length: i64,
shard_size: usize,
frame_size: usize,
) -> (Vec<Option<BitrotWriterWrapper>>, Vec<Option<DiskError>>) {
join_all(disks.iter().map(|disk| async move {
let Some(disk) = disk else {
return (None, Some(DiskError::DiskNotFound));
};
#[cfg(test)]
if let Some(result) = multipart_writer_test_hook::create_writer(disk, path, length, shard_size).await {
if let Some(result) = multipart_writer_test_hook::create_writer(disk, path, length, frame_size).await {
return match result {
Ok(writer) => (Some(writer), None),
Err(error) => (None, Some(error)),
@@ -142,7 +142,7 @@ async fn create_part_writers(
RUSTFS_META_TMP_BUCKET,
path,
length,
shard_size,
frame_size,
HashAlgorithm::HighwayHash256S,
)
.await
@@ -2592,13 +2592,21 @@ impl crate::storage_api_contracts::multipart::MultipartOperations for SetDisks {
let erasure =
Arc::new(coding::Erasure::try_new(fi.erasure.data_blocks, fi.erasure.parity_blocks, fi.erasure.block_size)
.map_err(Error::from)?);
let frame_size = crate::object_api::persisted_ec_read_quantum(
&fi.metadata,
erasure.block_size,
erasure.data_shards,
erasure.shard_size(),
)
.map_err(|error| Error::other(error.to_string()))?
.unwrap_or_else(|| erasure.shard_size());
let writer_setup_stage_start = rustfs_io_metrics::put_stage_metrics_enabled().then(Instant::now);
let (mut writers, errors) = create_part_writers(
&shuffle_disks,
&tmp_part_path,
erasure.shard_file_size(data.size()),
erasure.shard_size(),
frame_size,
)
.await;
@@ -3043,6 +3051,10 @@ impl crate::storage_api_contracts::multipart::MultipartOperations for SetDisks {
user_defined: {
let mut metadata = fi.metadata.clone();
strip_internal_multipart_metadata(&mut metadata);
rustfs_utils::http::metadata_compat::remove_str(
&mut metadata,
crate::object_api::EC_READ_QUANTUM_INFO_INTERNAL_SUFFIX,
);
metadata
},
..Default::default()
@@ -3220,8 +3232,10 @@ impl crate::storage_api_contracts::multipart::MultipartOperations for SetDisks {
let disks = self.get_disks_internal().await;
let mut user_defined = opts.user_defined.clone();
let requested_ec_block_size = crate::object_api::take_ec_block_size_hint(&mut user_defined);
strip_native_controls(&mut user_defined);
rustfs_utils::http::remove_str(&mut user_defined, SUFFIX_INLINE_DATA);
rustfs_utils::http::metadata_compat::remove_str(&mut user_defined, crate::object_api::EC_BLOCK_SIZE_INFO_INTERNAL_SUFFIX);
rustfs_filemeta::shard_integrity::clear_integrity_metadata(&mut user_defined);
if protect_upload {
rustfs_utils::http::insert_str(
@@ -3267,6 +3281,20 @@ impl crate::storage_api_contracts::multipart::MultipartOperations for SetDisks {
)?;
let mut fi = FileInfo::new([bucket, object].join("/").as_str(), data_drives, parity_drives);
if let Some(block_size) = requested_ec_block_size {
fi.erasure.block_size = block_size;
}
if !protect_upload && crate::object_api::ec_read_quantum_write_enabled() {
let erasure = coding::Erasure::try_new(fi.erasure.data_blocks, fi.erasure.parity_blocks, fi.erasure.block_size)
.map_err(Error::from)?;
if let Some(frame_size) =
crate::object_api::select_ec_read_quantum(fi.erasure.block_size, erasure.data_shards, erasure.shard_size())
&& frame_size != erasure.shard_size()
{
crate::object_api::insert_ec_read_quantum(&mut user_defined, frame_size);
}
}
fi.version_id = if let Some(vid) = &opts.version_id {
Some(Uuid::parse_str(vid)?)
@@ -3389,6 +3417,9 @@ impl crate::storage_api_contracts::multipart::MultipartOperations for SetDisks {
upload_id,
checksum_algo: user_defined.get(rustfs_rio::RUSTFS_MULTIPART_CHECKSUM).cloned(),
checksum_type: user_defined.get(rustfs_rio::RUSTFS_MULTIPART_CHECKSUM_TYPE).cloned(),
effective_ec_block_size: Some(
u64::try_from(fi.erasure.block_size).map_err(|_| Error::other("invalid EC block size"))?,
),
})
}
@@ -3418,6 +3449,10 @@ impl crate::storage_api_contracts::multipart::MultipartOperations for SetDisks {
upload_id: upload_id.to_owned(),
user_defined: {
strip_internal_multipart_metadata(&mut fi.metadata);
rustfs_utils::http::metadata_compat::remove_str(
&mut fi.metadata,
crate::object_api::EC_READ_QUANTUM_INFO_INTERNAL_SUFFIX,
);
fi.metadata.clone()
},
..Default::default()
@@ -5109,6 +5144,48 @@ mod tests {
use tempfile::TempDir;
use tokio::sync::{Notify, RwLock};
#[tokio::test]
async fn multipart_upload_persists_block_size_for_later_parts_and_completion() {
use crate::storage_api_contracts::multipart::MultipartOperations as _;
let (_temp_dirs, disks, set_disks) = hermetic_set_disks(4).await;
let bucket = "ec-block-size-multipart";
for disk in &disks {
disk.make_volume(bucket).await.expect("bucket volume should be created");
}
let mut create_opts = ObjectOptions::default();
create_opts
.user_defined
.insert(crate::object_api::EC_BLOCK_SIZE_HINT_INTERNAL_KEY.to_string(), "65536".to_string());
let upload = set_disks
.new_multipart_upload(bucket, "object", &create_opts)
.await
.expect("multipart upload should be created");
assert_eq!(upload.effective_ec_block_size, Some(65_536));
let mut part_reader = PutObjReader::from_vec(vec![0x61; 5 * 1024 * 1024]);
let part = set_disks
.put_object_part(bucket, "object", &upload.upload_id, 1, &mut part_reader, &ObjectOptions::default())
.await
.expect("part must use the block size persisted at upload creation");
let completed = set_disks
.complete_multipart_upload(
bucket,
"object",
&upload.upload_id,
vec![CompletePart {
part_num: 1,
etag: part.etag.clone(),
..Default::default()
}],
&ObjectOptions::default(),
)
.await
.expect("upload should complete using its stored layout");
assert_eq!(completed.effective_ec_block_size(), Some(65_536));
}
#[tokio::test]
async fn multipart_writer_setup_opens_disks_concurrently_and_preserves_error_slots() {
use crate::cluster::rpc::internode_data_transport::{
+390 -5
View File
@@ -43,8 +43,8 @@ use super::super::{
ReplicationObjectBridge, Result, SET_DISK_COMMIT_TAIL_WARN_THRESHOLD_MS, SLASH_SEPARATOR, SUFFIX_ACTUAL_SIZE,
SUFFIX_COMPRESSION, SUFFIX_COMPRESSION_SIZE, SUFFIX_RESTORE_COPYBACK, SUFFIX_RESTORE_OPERATION_ID,
SUFFIX_RESTORE_WORKER_LOCK, SetDisks, SmallWritePath, StorageError, TRANSITION_COMPLETE, UpdateMetadataOpts, Uuid,
WriteLayout, adaptive_duplex_buffer_size, build_get_object_info, build_inline_bitrot_readers,
build_inline_bitrot_readers_from_refs, can_try_inline_data_shards_direct, check_object_lock_delete,
WriteLayout, adaptive_duplex_buffer_size, build_get_object_info, build_inline_bitrot_readers_from_refs_with_frame_size,
build_inline_bitrot_readers_with_frame_size, can_try_inline_data_shards_direct, check_object_lock_delete,
check_object_lock_for_deletion_with_state, check_object_lock_retention_update, classify_get_codec_streaming_object_class,
classify_put_write_path, classify_storage_error, collect_inline_data_shard_fileinfos_by_index, contains_key_str,
create_bitrot_writer_with_orig_volume, debug, delete_file_info_version_id, disk, ensure_delete_commit_locks_held, error,
@@ -3172,6 +3172,10 @@ impl crate::storage_api_contracts::object::ObjectIO for SetDisks {
fi.uses_legacy_checksum,
);
let shard_size = inline_erasure_shard_size(fi.erasure.block_size, data_shards, fi.uses_legacy_checksum);
let frame_size =
crate::object_api::persisted_ec_read_quantum(&fi.metadata, fi.erasure.block_size, data_shards, shard_size)
.map_err(|error| to_object_err(Error::other(error.to_string()), vec![bucket, object]))?
.unwrap_or(shard_size);
if let Some(inline_prepare_stage_start) = inline_prepare_stage_start.take() {
rustfs_io_metrics::record_get_object_stage_duration_by_size(
GET_OBJECT_PATH_INLINE_DIRECT,
@@ -3182,12 +3186,13 @@ impl crate::storage_api_contracts::object::ObjectIO for SetDisks {
);
}
let reader_setup_stage_start = rustfs_io_metrics::get_stage_metrics_enabled().then(Instant::now);
let mut readers = build_inline_bitrot_readers_from_refs(
let mut readers = build_inline_bitrot_readers_from_refs_with_frame_size(
&data_files,
bucket,
object,
read_length,
shard_size,
frame_size,
&checksum_algo,
opts.skip_verify_bitrot,
)
@@ -3230,6 +3235,8 @@ impl crate::storage_api_contracts::object::ObjectIO for SetDisks {
}
let erasure = self.erasure_cache.get_for_file_info(fi)?;
let frame_size = crate::set_disk::read::read_frame_size_for_file_info(fi, &erasure)
.map_err(|error| to_object_err(error, vec![bucket, object]))?;
let read_length = erasure.shard_file_offset(0, object_size, object_size);
let total_shards = data_shards + fi.erasure.parity_blocks;
let (_disks, files) = Self::shuffle_disks_and_parts_metadata_by_index(disks, files, fi);
@@ -3252,13 +3259,14 @@ impl crate::storage_api_contracts::object::ObjectIO for SetDisks {
);
}
let reader_setup_stage_start = rustfs_io_metrics::get_stage_metrics_enabled().then(Instant::now);
let readers = build_inline_bitrot_readers(
let readers = build_inline_bitrot_readers_with_frame_size(
&files,
total_shards,
bucket,
object,
read_length,
erasure.shard_size(),
frame_size,
&checksum_algo,
opts.skip_verify_bitrot,
)
@@ -4670,6 +4678,11 @@ impl SetDisks {
let expected_restore_operation_id = restore_commit_operation_id_from_metadata(&opts.user_defined)?;
let mut user_defined = opts.user_defined.clone();
let requested_ec_block_size = crate::object_api::take_ec_block_size_hint(&mut user_defined);
// This is a request-to-writer marker. It must never become ordinary
// object metadata, even when the caller supplied it through an
// internal path.
rustfs_utils::http::metadata_compat::remove_str(&mut user_defined, crate::object_api::EC_BLOCK_SIZE_INFO_INTERNAL_SUFFIX);
rustfs_filemeta::shard_integrity::clear_integrity_metadata(&mut user_defined);
if let Some(eval_metadata) = &opts.eval_metadata {
merge_evaluated_metadata(&mut user_defined, eval_metadata)?;
@@ -4715,6 +4728,9 @@ impl SetDisks {
// }
let mut fi = FileInfo::new([bucket, object].join("/").as_str(), data_drives, parity_drives);
if let Some(block_size) = requested_ec_block_size {
fi.erasure.block_size = block_size;
}
fi.version_id = {
if let Some(ref vid) = opts.version_id {
@@ -4807,6 +4823,21 @@ impl SetDisks {
let collect_stage_timing = rustfs_io_metrics::put_stage_metrics_enabled() || issue3031_diag_enabled();
let shard_file_size = shard_file_size_raw;
let shard_size = erasure.shard_size();
// Large layout blocks are still useful for sequential writes, but
// they should not force every Range GET to verify/read that whole
// shard. New unprotected, non-inline objects may opt into the
// bounded per-frame format; legacy and protected writes retain the
// historical framing so their existing readers and proofs remain
// byte-for-byte compatible.
let frame_size = if !is_inline_buffer && !protect_write && crate::object_api::ec_read_quantum_write_enabled() {
crate::object_api::select_ec_read_quantum(fi.erasure.block_size, erasure.data_shards, shard_size)
.unwrap_or(shard_size)
} else {
shard_size
};
if frame_size != shard_size {
crate::object_api::insert_ec_read_quantum(&mut user_defined, frame_size);
}
let write_path = classify_put_write_path(is_inline_buffer, put_object_size, fi.erasure.block_size);
let direct_inline_commit = matches!(write_path, SmallWritePath::Inline);
rustfs_io_metrics::record_put_object_path(write_path.metric_label());
@@ -4846,7 +4877,7 @@ impl SetDisks {
RUSTFS_META_TMP_BUCKET,
&tmp_obj,
shard_file_size,
shard_size,
frame_size,
HashAlgorithm::HighwayHash256S,
)
.await
@@ -10135,6 +10166,15 @@ impl crate::storage_api_contracts::object::ObjectOperations for SetDisks {
};
let mut replacement_metadata = (*src_info.user_defined).clone();
rustfs_filemeta::shard_integrity::clear_integrity_metadata(&mut replacement_metadata);
if src_info.version_only {
// Shared shard files retain the selected version's persisted framing.
let suffix = crate::object_api::EC_READ_QUANTUM_INFO_INTERNAL_SUFFIX;
rustfs_utils::http::remove_str(&mut replacement_metadata, suffix);
if rustfs_utils::http::contains_key_str(&fi.metadata, suffix) {
let quantum = rustfs_utils::http::get_consistent_str(&fi.metadata, suffix).ok_or(Error::FileCorrupt)?;
rustfs_utils::http::insert_str(&mut replacement_metadata, suffix, quantum.to_owned());
}
}
for suffix in [
rustfs_filemeta::shard_integrity::SUFFIX_SHARD_INTEGRITY,
rustfs_filemeta::shard_integrity::SUFFIX_INLINE_INTEGRITY,
@@ -13930,6 +13970,61 @@ mod replication_quota_safety_tests {
use super::*;
use std::io::Cursor;
#[tokio::test]
async fn put_object_persists_transient_ec_block_size_without_leaking_marker_metadata() {
let (_temp_dirs, disks, set_disks) = hermetic_set_disks(4).await;
let bucket = "ec-block-size-marker";
for disk in &disks {
disk.make_volume(bucket).await.expect("bucket volume should be created");
}
for block_size in crate::object_api::EC_BLOCK_SIZE_CANDIDATES {
let object = format!("object-{block_size}");
let mut opts = ObjectOptions::default();
opts.user_defined
.insert(crate::object_api::EC_BLOCK_SIZE_HINT_INTERNAL_KEY.to_string(), block_size.to_string());
let payload = vec![0x61; 4096];
let mut reader = PutObjReader::from_vec(payload.clone());
set_disks
.put_object(bucket, &object, &mut reader, &opts)
.await
.expect("object with a supported block-size marker should be written");
let info = set_disks
.get_object_info(bucket, &object, &ObjectOptions::default())
.await
.expect("written object metadata should be readable");
assert_eq!(info.effective_ec_block_size(), Some(block_size));
assert!(
!info
.user_defined
.contains_key(crate::object_api::EC_BLOCK_SIZE_HINT_INTERNAL_KEY)
);
let mut object_reader = set_disks
.get_object_reader(
bucket,
&object,
Some(HTTPRangeSpec {
is_suffix_length: false,
start: 128,
end: 511,
}),
HeaderMap::new(),
&ObjectOptions::default(),
)
.await
.expect("persisted block size should decode range reads");
let mut restored = Vec::new();
object_reader
.stream
.read_to_end(&mut restored)
.await
.expect("range read should complete");
assert_eq!(restored, payload[128..512]);
}
}
#[tokio::test]
async fn quota_put_future_keeps_commit_state_off_the_caller_stack() {
let (_temp_dirs, _disks, set_disks) = hermetic_set_disks(4).await;
@@ -16085,6 +16180,134 @@ mod metadata_mutation_generation_tests {
assert!(metadata.contains_key(&format!("x-minio-internal-{suffix}")));
}
const EC_BLOCK_SIZE_PROJECTION_KEYS: [&str; 2] = ["x-rustfs-internal-ec-block-size", "x-minio-internal-ec-block-size"];
/// Metadata keys of the latest version as persisted on every disk.
async fn stored_metadata_keys(disks: &[DiskStore], bucket: &str, object: &str) -> Vec<HashSet<String>> {
let mut stored = Vec::with_capacity(disks.len());
for disk in disks {
let bytes = disk
.read_all(bucket, &format!("{object}/xl.meta"))
.await
.expect("persisted metadata");
let fi = FileMeta::load(&bytes)
.expect("decode xl.meta")
.into_fileinfo(bucket, object, "", false, false, false)
.expect("latest version");
stored.push(fi.metadata.into_keys().collect());
}
stored
}
#[tokio::test]
async fn default_layout_metadata_copy_persists_no_ec_block_size_projection() {
let (_temp_dirs, disks, set) = hermetic_set_disks(4).await;
let bucket = "default-layout-copy";
let object = "object";
for disk in &disks {
disk.make_volume(bucket).await.expect("create bucket");
}
set.put_object(
bucket,
object,
&mut PutObjReader::from_vec(b"default layout".to_vec()),
&ObjectOptions {
write_completion: crate::object_api::WriteCompletion::TailDrained,
user_defined: HashMap::from([("x-amz-meta-color".to_string(), "red".to_string())]),
..Default::default()
},
)
.await
.expect("write object");
let before = stored_metadata_keys(&disks, bucket, object).await;
let opts = ObjectOptions::default();
let mut source = set.get_object_info(bucket, object, &opts).await.expect("read metadata");
assert_eq!(source.effective_ec_block_size(), Some(crate::object_api::DEFAULT_EC_BLOCK_SIZE));
source.metadata_only = true;
Arc::make_mut(&mut source.user_defined).insert("x-amz-meta-color".to_string(), "blue".to_string());
set.copy_object(bucket, object, bucket, object, &mut source, &opts, &opts)
.await
.expect("metadata-replacing copy");
let after = stored_metadata_keys(&disks, bucket, object).await;
for keys in &after {
for key in EC_BLOCK_SIZE_PROJECTION_KEYS {
assert!(!keys.contains(key), "a default-layout copy must not persist {key}");
}
}
assert_eq!(after, before, "a metadata-replacing copy keeps the stored key set");
let copied = set
.get_object_info(bucket, object, &opts)
.await
.expect("read copied metadata");
assert_eq!(copied.user_defined.get("x-amz-meta-color").map(String::as_str), Some("blue"));
assert_eq!(copied.effective_ec_block_size(), Some(crate::object_api::DEFAULT_EC_BLOCK_SIZE));
}
#[tokio::test]
async fn default_layout_replication_status_writeback_persists_no_ec_block_size_projection() {
let (_temp_dirs, disks, set) = hermetic_set_disks(4).await;
let bucket = "default-layout-writeback";
let object = "object";
for disk in &disks {
disk.make_volume(bucket).await.expect("create bucket");
}
set.put_object(
bucket,
object,
&mut PutObjReader::from_vec(b"default layout".to_vec()),
&ObjectOptions {
write_completion: crate::object_api::WriteCompletion::TailDrained,
..Default::default()
},
)
.await
.expect("write object");
let before = stored_metadata_keys(&disks, bucket, object).await;
let snapshot = set
.get_object_info(bucket, object, &ObjectOptions::default())
.await
.expect("read metadata")
.replication_generation_snapshot();
let status = replication_metadata(None, None, "arn:test=COMPLETED;");
set.put_object_metadata(
bucket,
object,
&ObjectOptions {
eval_metadata: Some(status.clone()),
replication_status_writeback: Some(Box::new(ReplicationStatusWritebackCondition {
expected_generation: snapshot,
..Default::default()
})),
..Default::default()
},
)
.await
.expect("replication status write-back");
let after = stored_metadata_keys(&disks, bucket, object).await;
for (keys, before) in after.iter().zip(&before) {
for key in EC_BLOCK_SIZE_PROJECTION_KEYS {
assert!(!keys.contains(key), "a status write-back must not persist {key}");
}
// The write-back adds exactly the status keys `main` writes.
let added: HashSet<&str> = keys.difference(before).map(String::as_str).collect();
let expected: HashSet<&str> = status
.keys()
.map(String::as_str)
.chain([
"X-Amz-Replication-Status",
"x-rustfs-internal-replication-status",
"x-minio-internal-replication-status",
"x-rustfs-internal-data-moved",
"x-minio-internal-data-moved",
])
.collect();
assert_eq!(added, expected);
}
}
#[tokio::test]
async fn replication_status_writeback_cas_accepts_suspended_null_version_snapshot() {
let (_temp_dirs, disk_stores, set_disks) = hermetic_set_disks(4).await;
@@ -25517,6 +25740,168 @@ mod put_object_tmp_cleanup_tests {
.expect("version delete must not be wedged by cleared lock metadata");
}
#[tokio::test]
#[serial_test::serial]
async fn version_only_copy_preserves_quantized_shared_data() {
assert_version_only_copy_frame_metadata(true).await;
}
#[tokio::test]
#[serial_test::serial]
async fn version_only_copy_ignores_injected_frame_markers() {
assert_version_only_copy_frame_metadata(false).await;
}
async fn assert_version_only_copy_frame_metadata(quantized: bool) {
const BLOCK: usize = 64 * 1024;
const FRAME: usize = 16 * 1024;
let (_dirs, disks, set) = hermetic_set_disks(4).await;
let bucket = "copy-version-frame-metadata";
let object = if quantized {
"quantized"
} else {
"legacy-with-injected-marker"
};
for disk in &disks {
disk.make_volume(bucket).await.expect("create version-only copy bucket");
}
let payload: Vec<u8> = (0..2 * BLOCK + 17)
.map(|index| u8::try_from(index % 251).expect("bounded fixture byte"))
.collect();
let mut put_options = ObjectOptions {
versioned: true,
..Default::default()
};
put_options
.user_defined
.insert(crate::object_api::EC_BLOCK_SIZE_HINT_INTERNAL_KEY.to_owned(), BLOCK.to_string());
let enabled = if quantized { "true" } else { "false" };
let source = temp_env::async_with_vars(
[
(rustfs_config::ENV_EC_READ_QUANTUM_ENABLE, Some(enabled)),
(rustfs_config::ENV_EC_READ_QUANTUM_FLEET_CONFIRMED, Some(enabled)),
(rustfs_config::ENV_SHARD_INTEGRITY_WRITE, Some("false")),
],
async {
set.put_object(bucket, object, &mut PutObjReader::from_vec(payload.clone()), &put_options)
.await
.expect("write physical source version")
},
)
.await;
let source_version = source.version_id.expect("source version ID").to_string();
let source_options = ObjectOptions {
version_id: Some(source_version.clone()),
versioned: true,
..Default::default()
};
let mut source_info = set
.get_object_info(bucket, object, &source_options)
.await
.expect("project source metadata");
assert!(!rustfs_utils::http::contains_key_str(
&source_info.user_defined,
crate::object_api::EC_READ_QUANTUM_INFO_INTERNAL_SUFFIX
));
let mut source_data_dirs = Vec::new();
for disk in &disks {
let fi = disk
.read_version("", bucket, object, &source_version, &crate::disk::ReadOptions::default())
.await
.expect("read source physical layout");
assert!(!fi.inline_data(), "fixture must share real shard files");
assert_eq!(fi.erasure.block_size, BLOCK);
assert_eq!(
crate::object_api::persisted_ec_read_quantum(
&fi.metadata,
BLOCK,
fi.erasure.data_blocks,
fi.erasure.shard_size(),
)
.expect("source frame geometry"),
quantized.then_some(FRAME)
);
source_data_dirs.push(fi.data_dir);
}
if !quantized {
// Replacement metadata cannot change the format of reused bytes.
crate::object_api::insert_ec_read_quantum(Arc::make_mut(&mut source_info.user_defined), FRAME);
}
source_info.version_only = true;
let copied = temp_env::async_with_vars(
[
(rustfs_config::ENV_EC_READ_QUANTUM_ENABLE, Some("false")),
(rustfs_config::ENV_EC_READ_QUANTUM_FLEET_CONFIRMED, Some("false")),
],
async {
set.copy_object(
bucket,
object,
bucket,
object,
&mut source_info,
&source_options,
&ObjectOptions {
versioned: true,
..Default::default()
},
)
.await
.expect("version-only copy must commit a new version")
},
)
.await;
let copy_version = copied.version_id.expect("copied version ID").to_string();
assert_ne!(copy_version, source_version);
assert!(!rustfs_utils::http::contains_key_str(
&copied.user_defined,
crate::object_api::EC_READ_QUANTUM_INFO_INTERNAL_SUFFIX
));
for version in [&source_version, &copy_version] {
let mut reader = set
.get_object_reader(
bucket,
object,
None,
HeaderMap::new(),
&ObjectOptions {
version_id: Some(version.clone()),
versioned: true,
..Default::default()
},
)
.await
.expect("source and copied version must open");
let mut actual = Vec::new();
reader
.stream
.read_to_end(&mut actual)
.await
.expect("shared frames must remain readable");
assert_eq!(actual, payload);
}
for (disk, source_data_dir) in disks.iter().zip(source_data_dirs) {
let fi = disk
.read_version("", bucket, object, &copy_version, &crate::disk::ReadOptions::default())
.await
.expect("read copied version layout");
assert_eq!(fi.data_dir, source_data_dir, "version-only copy must reuse source data");
assert_eq!(
crate::object_api::persisted_ec_read_quantum(
&fi.metadata,
BLOCK,
fi.erasure.data_blocks,
fi.erasure.shard_size(),
)
.expect("copied frame geometry"),
quantized.then_some(FRAME)
);
for key in ["x-rustfs-internal-ec-read-quantum", "x-minio-internal-ec-read-quantum"] {
assert_eq!(fi.metadata.get(key).map(String::as_str), quantized.then_some("16384"));
}
}
}
#[tokio::test]
async fn version_only_copy_checks_the_destination_version_object_lock() {
let (_temp_dirs, disk_stores, set_disks) = hermetic_set_disks(4).await;
+655 -13
View File
@@ -17,11 +17,11 @@ use super::{
GetObjectFileInfo, GetObjectMetadataCacheCapability, GetObjectMetadataCacheEntry, GetObjectMetadataCacheGeneration,
GetObjectMetadataCacheKey, GetObjectMetadataCacheLookupKey, GetObjectReadPolicy, HashAlgorithm, LOG_COMPONENT_ECSTORE,
LOG_SUBSYSTEM_SET_DISK, OBJECT_OP_IGNORED_ERRS, ObjectInfo, ObjectOptions, RUSTFS_META_BUCKET, ReadOptions, Result, SetDisks,
StorageError, adaptive_duplex_buffer_size, build_get_codec_streaming_decode_engine, build_inline_bitrot_readers_from_refs,
collect_inline_data_shard_fileinfos_by_index, debug, error, get_codec_streaming_metrics_path,
get_codec_streaming_multipart_max_parts, get_object_read_policy, is_codec_streaming_multipart_enabled,
is_multipart_reader_setup_prefetch_enabled, object_fits_single_block, reduce_read_quorum_errs, to_object_err,
try_read_inline_data_shards_direct, warn,
StorageError, adaptive_duplex_buffer_size, build_get_codec_streaming_decode_engine,
build_inline_bitrot_readers_from_refs_with_frame_size, collect_inline_data_shard_fileinfos_by_index, debug, error,
get_codec_streaming_metrics_path, get_codec_streaming_multipart_max_parts, get_object_read_policy,
is_codec_streaming_multipart_enabled, is_multipart_reader_setup_prefetch_enabled, object_fits_single_block,
reduce_read_quorum_errs, to_object_err, try_read_inline_data_shards_direct, warn,
};
use crate::diagnostics::get::{
GET_DIRECT_MEMORY_SUBPATH_DISK_DATA_BLOCKS, GET_DIRECT_MEMORY_SUBPATH_INLINE_BUFFERED, GET_METADATA_CACHE_DECISION_HIT,
@@ -119,16 +119,17 @@ use super::core::io_primitives::{
BitrotReaderSetup, BitrotReaderSetupAttribution, BitrotReaderSetupMode, DeferredReaderReopener, EVENT_SET_DISK_READ,
GetCodecStreamingReaderBuildOutcome, MetadataCacheLookup, ObjectBitrotReader, ReadRepairAdmissionSubmitter,
ReadRepairHealSubmission, SLOW_OBJECT_READ_LOG_THRESHOLD, codec_streaming_reader_setup_fallback_reason,
create_bitrot_readers_until_quorum_with_preference, create_data_block_bitrot_readers, resolved_read_repair_version_id,
send_read_repair_heal_request, shard_read_costs_for_disks, submit_read_repair_heal, submit_read_repair_heal_with_submitter,
create_bitrot_readers_until_quorum_with_preference_and_frame_size, create_data_block_bitrot_readers_with_frame_size,
resolved_read_repair_version_id, send_read_repair_heal_request, shard_read_costs_for_disks, submit_read_repair_heal,
submit_read_repair_heal_with_submitter,
};
#[cfg(test)]
use super::core::io_primitives::{
ENV_RUSTFS_GET_CODEC_STREAMING_DATA_BLOCKS_FIRST_READER_SETUP, ENV_RUSTFS_GET_DATA_BLOCKS_FIRST_READER_SETUP,
MultipartCodecStreamingReader, ReadRepairAdmissionFuture, ReadRepairAdmissionOutcome, collect_read_multiple_results,
collect_read_parts_results, create_bitrot_readers_until_quorum, create_bitrot_readers_until_quorum_all_shards,
release_read_repair_heal_reservation, reserve_read_repair_heal, resolve_read_part_from_responses, shard_read_cost_for_disk,
shard_read_cost_for_endpoint,
create_bitrot_readers_until_quorum_with_preference, release_read_repair_heal_reservation, reserve_read_repair_heal,
resolve_read_part_from_responses, shard_read_cost_for_disk, shard_read_cost_for_endpoint,
};
#[cfg(test)]
use super::core::metadata_quorum::{MetadataEarlyStopDecision, MetadataQuorumAccumulator};
@@ -1371,6 +1372,7 @@ impl SetDisks {
}
let erasure = erasure_cache.get_for_file_info(fi)?;
let frame_size = read_frame_size_for_file_info(fi, &erasure)?;
let checksum_info = fi.erasure.get_checksum_info(part.number);
let checksum_algo = if fi.uses_legacy_checksum && checksum_info.algorithm == HashAlgorithm::HighwayHash256S {
@@ -1392,12 +1394,13 @@ impl SetDisks {
};
let reader_setup_stage_start = Instant::now();
let mut readers = build_inline_bitrot_readers_from_refs(
let mut readers = build_inline_bitrot_readers_from_refs_with_frame_size(
&data_files,
bucket,
object,
read_length,
erasure.shard_size(),
frame_size,
&checksum_algo,
skip_verify_bitrot,
)
@@ -1450,7 +1453,7 @@ impl SetDisks {
let use_mmap_read = object_mmap_read_enabled();
let reader_setup_stage_start = Instant::now();
let mut reader_setup = create_data_block_bitrot_readers(
let mut reader_setup = create_data_block_bitrot_readers_with_frame_size(
&files,
&disks,
bucket,
@@ -1459,6 +1462,7 @@ impl SetDisks {
0,
read_length,
erasure.shard_size(),
frame_size,
checksum_algo,
skip_verify_bitrot,
use_mmap_read,
@@ -1610,6 +1614,7 @@ impl SetDisks {
);
let erasure = erasure_cache.get_for_file_info(&fi)?;
let frame_size = read_frame_size_for_file_info(&fi, &erasure)?;
let part_indices: Vec<usize> = (part_index..=last_part_index).collect();
debug!(bucket, object, ?part_indices, "Multipart part indices to stream");
@@ -1622,6 +1627,39 @@ impl SetDisks {
let use_mmap_read = object_mmap_read_enabled();
let files = Arc::new(files);
let disks = Arc::new(disks);
// A small Range GET only needs the authenticated q-aligned slice that
// contains each requested data-shard segment. Permit one complete
// logical block as well as a single physical shard: the 64 KiB layout
// has 32 KiB shards, so a normal 64 KiB range otherwise bypasses the
// bounded reader and reopens parity on the legacy path.
if frame_size != erasure.shard_size() && length > 0 && length <= erasure.block_size {
return Self::read_quantized_range(
bucket,
object,
writer,
&fi,
files.as_ref(),
disks.as_ref(),
&erasure,
frame_size,
length,
part_index,
last_part_index,
part_offset,
skip_verify_bitrot,
suppress_read_repair,
prefer_data_blocks_first_reader_setup,
require_reconstruction_surplus,
pool_index,
set_index,
metrics_path,
metrics_object_class,
metrics_size_bucket,
)
.await;
}
let prefetch_enabled = multipart_reader_setup_prefetch_enabled(get_object_read_policy());
let mut prefetched: Option<(usize, PrefetchedReaderSetup)> = None;
@@ -1674,6 +1712,7 @@ impl SetDisks {
part_number,
read_offset,
read_length,
frame_size,
checksum_algo,
};
let mut setup_result = None;
@@ -1729,6 +1768,7 @@ impl SetDisks {
part_number: next_number,
read_offset: 0,
read_length: erasure.shard_file_offset(0, next_length, next_size),
frame_size,
checksum_algo: multipart_part_checksum_algo(&fi, next_number),
};
let files = Arc::clone(&files);
@@ -1911,7 +1951,9 @@ impl SetDisks {
error: err,
exact_quorum,
repair,
..
} = erasure
.clone()
.decode_with_stripe_handles_and_reopeners_with_diagnostics(
writer,
readers,
@@ -2047,6 +2089,284 @@ impl SetDisks {
Ok(())
}
/// Read a small q-enabled range one data-shard segment at a time. Each setup
/// opens the same q-aligned logical slice on every shard, so Reed-Solomon sees
/// equal-length vectors while the caller receives only the requested bytes.
#[allow(clippy::too_many_arguments)]
async fn read_quantized_range<W>(
bucket: &str,
object: &str,
writer: &mut W,
fi: &FileInfo,
files: &[FileInfo],
disks: &[Option<DiskStore>],
erasure: &coding::Erasure,
frame_size: usize,
length: usize,
part_index: usize,
last_part_index: usize,
mut part_offset: usize,
skip_verify_bitrot: bool,
suppress_read_repair: bool,
prefer_data_blocks_first_reader_setup: bool,
require_reconstruction_surplus: bool,
pool_index: usize,
set_index: usize,
metrics_path: &'static str,
metrics_object_class: &'static str,
metrics_size_bucket: &'static str,
) -> Result<()>
where
W: AsyncWrite + Send + Sync + Unpin + 'static,
{
let logical_shard_size = erasure.shard_size();
if frame_size == 0
|| logical_shard_size == 0
|| erasure.block_size == 0
|| erasure.data_shards == 0
|| !logical_shard_size.is_multiple_of(frame_size)
{
return Err(Error::other("invalid EC read quantum geometry"));
}
let use_mmap_read = object_mmap_read_enabled();
let quantum_label = quantized_range_quantum_label(frame_size);
rustfs_io_metrics::record_get_object_quantized_range_request(
metrics_path,
metrics_object_class,
metrics_size_bucket,
quantum_label,
u64::try_from(length).unwrap_or(u64::MAX),
);
let mut total_read = 0usize;
for current_part in part_index..=last_part_index {
if total_read == length {
break;
}
let part_number = fi.parts[current_part].number;
let part_size = fi.parts[current_part].size;
let part_remaining = quantized_part_remaining(part_size, part_offset)?;
let part_length = part_remaining.min(length - total_read);
let mut segment_offset = part_offset;
let mut segment_remaining = part_length;
while segment_remaining > 0 {
rustfs_io_metrics::record_get_object_quantized_range_segment(
metrics_path,
metrics_object_class,
metrics_size_bucket,
quantum_label,
);
let block_index = segment_offset / erasure.block_size;
let within_block = segment_offset % erasure.block_size;
let block_data_start = block_index
.checked_mul(erasure.block_size)
.ok_or_else(|| Error::other("EC read quantum block offset overflows"))?;
if block_data_start >= part_size {
return Err(Error::other("EC read quantum block offset exceeds part size"));
}
let block_data_len = (part_size - block_data_start).min(erasure.block_size);
let block_shard_size = coding::calc_shard_size(block_data_len, erasure.data_shards);
if block_shard_size == 0 {
return Err(Error::other("EC read quantum block has zero shard length"));
}
let shard_index = within_block / block_shard_size;
if shard_index >= erasure.data_shards {
return Err(Error::other("EC read quantum data shard index is out of range"));
}
let within_shard = within_block % block_shard_size;
let block_remaining = erasure.block_size - within_block;
if within_shard >= block_shard_size {
return Err(Error::other("EC read quantum shard offset exceeds encoded block"));
}
let shard_remaining = block_shard_size - within_shard;
let block_data_remaining = block_data_len - within_block;
let block_segment_length = segment_remaining.min(block_data_remaining);
let read_full_block = shard_index == 0 && within_shard < frame_size && block_segment_length > shard_remaining;
let (segment_length, read_shard_size, read_offset, output_offset) = if read_full_block {
// A range that begins in the first q-frame of shard 0 and
// crosses into shard 1 already requires all frames of
// shard 0. Group its data-shard segments into one decode so
// we do not reopen the block's shards a second time.
let read_offset = block_index
.checked_mul(logical_shard_size)
.ok_or_else(|| Error::other("EC read quantum offset overflows"))?;
(block_segment_length, block_shard_size, read_offset, within_block)
} else {
let segment_length = segment_remaining.min(block_remaining).min(shard_remaining);
if segment_length == 0 {
return Err(Error::other("EC read quantum segment is empty"));
}
let q_base = (within_shard / frame_size) * frame_size;
let relative_offset = within_shard - q_base;
let covered = relative_offset
.checked_add(segment_length)
.ok_or_else(|| Error::other("EC read quantum segment overflows"))?;
let frame_count = covered.div_ceil(frame_size);
let frame_aligned_size = frame_count
.checked_mul(frame_size)
.ok_or_else(|| Error::other("EC read quantum frame size overflows"))?;
let read_shard_size = frame_aligned_size.min(block_shard_size - q_base);
let read_offset = block_index
.checked_mul(logical_shard_size)
.and_then(|base| base.checked_add(q_base))
.ok_or_else(|| Error::other("EC read quantum offset overflows"))?;
let output_offset = shard_index
.checked_mul(read_shard_size)
.and_then(|base| base.checked_add(relative_offset))
.ok_or_else(|| Error::other("EC read quantum output offset overflows"))?;
(segment_length, read_shard_size, read_offset, output_offset)
};
let spec = PartReaderSetupSpec {
integrity: fi.parts[current_part].integrity.clone(),
part_number,
read_offset,
read_length: read_shard_size,
frame_size,
checksum_algo: multipart_part_checksum_algo(fi, part_number),
};
let (reader_setup, reader_setup_elapsed) = setup_multipart_part_readers(
files,
disks,
bucket,
object,
spec,
logical_shard_size,
erasure.data_shards,
erasure.parity_shards,
skip_verify_bitrot,
use_mmap_read,
prefer_data_blocks_first_reader_setup,
metrics_path,
metrics_object_class,
metrics_size_bucket,
)
.await;
rustfs_io_metrics::record_get_object_quantized_range_reader_setup(
metrics_path,
metrics_object_class,
metrics_size_bucket,
quantum_label,
);
rustfs_io_metrics::record_get_object_shard_reader_setup_duration(reader_setup_elapsed.as_secs_f64());
rustfs_io_metrics::record_get_object_stage_duration_by_size(
metrics_path,
GET_STAGE_READER_SETUP,
metrics_object_class,
metrics_size_bucket,
reader_setup_elapsed.as_secs_f64(),
);
let available_shards = reader_setup.available_shards();
if available_shards < erasure.data_shards {
return Err(DiskError::ErasureReadQuorum.into());
}
let quorum_window_bytes = read_shard_size
.saturating_mul(available_shards)
.try_into()
.unwrap_or(u64::MAX);
rustfs_io_metrics::record_get_object_quantized_range_quorum_window_bytes(
metrics_path,
metrics_object_class,
metrics_size_bucket,
quantum_label,
quorum_window_bytes,
);
let missing_shards = reader_setup.completed_failed_shards();
if !suppress_read_repair && missing_shards > 0 && available_shards >= erasure.data_shards {
let version_id = fi.version_id.as_ref().map(ToString::to_string);
submit_read_repair_heal(
bucket,
object,
version_id.as_deref(),
pool_index,
set_index,
Some(part_number),
"missing_shards",
)
.await;
}
let unattempted_data_shards = !reader_setup.data_shards_attempted(erasure.data_shards);
let read_costs = coding::decode::should_collect_shard_read_costs().then(|| shard_read_costs_for_disks(disks));
let readers = reader_setup.readers;
let deferred_stripe_handles = reader_setup.deferred_stripe_handles;
let deferred_reopeners = reader_setup.deferred_reopeners;
let outcome = erasure
.clone()
.decode_quantized_range_with_stripe_handles_and_reopeners(
writer,
readers,
output_offset,
segment_length,
read_shard_size,
read_costs,
deferred_stripe_handles,
deferred_reopeners,
!skip_verify_bitrot
&& !require_reconstruction_surplus
&& multipart_part_checksum_algo(fi, part_number).size() > 0,
)
.await;
rustfs_io_metrics::record_get_object_quantized_range_frames(
metrics_path,
metrics_object_class,
metrics_size_bucket,
quantum_label,
u64::try_from(outcome.frame_count).unwrap_or(u64::MAX),
);
rustfs_io_metrics::record_get_object_quantized_range_read_dispatches(
metrics_path,
metrics_object_class,
metrics_size_bucket,
quantum_label,
u64::try_from(outcome.read_dispatch_count).unwrap_or(u64::MAX),
);
let mut repair_needed = outcome.repair.needed(!unattempted_data_shards);
if require_reconstruction_surplus && outcome.exact_quorum && outcome.error.is_none() {
return Err(Error::other("two-phase read completed with exact reconstruction quorum"));
}
if let Some(error) = outcome.error {
let disk_error: DiskError = error.into();
let mut has_error = true;
if outcome.written == segment_length {
let should_enqueue_heal = matches!(disk_error, DiskError::FileCorrupt)
|| (matches!(disk_error, DiskError::FileNotFound) && !unattempted_data_shards);
if should_enqueue_heal {
repair_needed = true;
has_error = false;
}
}
if has_error {
return Err(disk_error.into());
}
}
if outcome.written != segment_length {
return Err(Error::other("quantized range returned fewer bytes than requested"));
}
if !suppress_read_repair && repair_needed {
DecodeReadRepairContext::new(bucket, object, fi.version_id, pool_index, set_index, part_number, false)
.submit()
.await;
}
total_read += segment_length;
segment_offset += segment_length;
segment_remaining -= segment_length;
}
part_offset = 0;
}
if total_read != length {
return Err(StorageError::LessData);
}
Ok(())
}
#[allow(clippy::too_many_arguments)]
#[hotpath::measure(impl_type = "SetDisks")]
@@ -2326,6 +2646,7 @@ impl SetDisks {
if part_length > part_size {
return Err(Error::other("codec streaming reader part length exceeds part size"));
}
let frame_size = read_frame_size_for_file_info(fi, erasure)?;
let checksum_info = fi.erasure.get_checksum_info(part_number);
let checksum_algo = if fi.uses_legacy_checksum && checksum_info.algorithm == HashAlgorithm::HighwayHash256S {
HashAlgorithm::HighwayHash256SLegacy
@@ -2346,7 +2667,7 @@ impl SetDisks {
});
let reader_setup_stage_start = get_stage_timer_if_enabled(stage_metrics_enabled);
let read_costs = coding::decode::should_collect_shard_read_costs().then(|| shard_read_costs_for_disks(disks));
let mut reader_setup = create_bitrot_readers_until_quorum_with_preference(
let mut reader_setup = create_bitrot_readers_until_quorum_with_preference_and_frame_size(
files,
disks,
bucket,
@@ -2355,6 +2676,7 @@ impl SetDisks {
read_offset,
read_length,
erasure.shard_size(),
frame_size,
checksum_algo,
skip_verify_bitrot,
use_mmap_read,
@@ -2617,6 +2939,7 @@ struct PartReaderSetupSpec {
part_number: usize,
read_offset: usize,
read_length: usize,
frame_size: usize,
checksum_algo: HashAlgorithm,
}
@@ -2631,6 +2954,55 @@ fn multipart_part_checksum_algo(fi: &FileInfo, part_number: usize) -> HashAlgori
}
}
/// Resolve the physical authenticated frame size for a persisted object while
/// keeping the original EC layout geometry for offsets and reconstruction.
pub(in crate::set_disk) fn read_frame_size_for_file_info(fi: &FileInfo, erasure: &coding::Erasure) -> Result<usize> {
let Some(quantum) =
crate::object_api::persisted_ec_read_quantum(&fi.metadata, erasure.block_size, erasure.data_shards, erasure.shard_size())
.map_err(|error| Error::other(error.to_string()))?
else {
return Ok(erasure.shard_size());
};
if erasure.uses_legacy_codec() {
return Err(Error::other("EC read quantum marker is unsupported for legacy erasure objects"));
}
if fi.parts.iter().any(|part| part.integrity.is_some()) {
return Err(Error::other("EC read quantum marker cannot be combined with protected shard integrity"));
}
Ok(quantum)
}
fn quantized_range_quantum_label(frame_size: usize) -> &'static str {
const Q_4KIB: usize = 4 * 1024;
const Q_8KIB: usize = 8 * 1024;
const Q_16KIB: usize = 16 * 1024;
const Q_32KIB: usize = 32 * 1024;
const Q_64KIB: usize = 64 * 1024;
const Q_128KIB: usize = 128 * 1024;
const Q_256KIB: usize = 256 * 1024;
const Q_512KIB: usize = 512 * 1024;
const Q_1MIB: usize = 1024 * 1024;
match frame_size {
Q_4KIB => "4kib",
Q_8KIB => "8kib",
Q_16KIB => "16kib",
Q_32KIB => "32kib",
Q_64KIB => "64kib",
Q_128KIB => "128kib",
Q_256KIB => "256kib",
Q_512KIB => "512kib",
Q_1MIB => "1mib",
_ => "other",
}
}
fn quantized_part_remaining(part_size: usize, part_offset: usize) -> Result<usize> {
part_size
.checked_sub(part_offset)
.ok_or_else(|| Error::other("EC read quantum part offset exceeds part size"))
}
fn multipart_reader_setup_prefetch_enabled(policy: GetObjectReadPolicy) -> bool {
policy.allows_multipart_setup_prefetch() && is_multipart_reader_setup_prefetch_enabled()
}
@@ -2756,7 +3128,7 @@ async fn setup_multipart_part_readers(
metrics_size_bucket: &'static str,
) -> (BitrotReaderSetup, Duration) {
let started = Instant::now();
let mut setup = create_bitrot_readers_until_quorum_with_preference(
let mut setup = create_bitrot_readers_until_quorum_with_preference_and_frame_size(
files,
disks,
bucket,
@@ -2765,6 +3137,7 @@ async fn setup_multipart_part_readers(
spec.read_offset,
spec.read_length,
shard_size,
spec.frame_size,
spec.checksum_algo,
skip_verify_bitrot,
use_mmap_read,
@@ -6040,6 +6413,54 @@ mod tests {
);
}
#[test]
fn quantized_part_remaining_rejects_offsets_past_part_end() {
assert_eq!(quantized_part_remaining(1024, 768).expect("valid part offset"), 256);
assert!(quantized_part_remaining(1024, 1025).is_err());
}
#[tokio::test]
async fn quantized_range_preserves_typed_quorum_and_short_read_errors() {
let erasure = coding::Erasure::new(2, 2, 64 * 1024);
let disks = vec![None; erasure.total_shard_count()];
for (part_size, expected) in [(1, StorageError::ErasureReadQuorum), (0, StorageError::LessData)] {
let mut fi = FileInfo::new(CODEC_STREAMING_TEST_OBJECT, 2, 2);
fi.size = 1;
fi.erasure.block_size = erasure.block_size;
fi.erasure.distribution = (1..=erasure.total_shard_count()).collect();
fi.add_object_part(1, "fixture-etag".to_string(), part_size, None, 1, None, None);
let files = vec![fi.clone(); erasure.total_shard_count()];
let mut output = Vec::new();
let error = SetDisks::read_quantized_range(
CODEC_STREAMING_TEST_BUCKET,
CODEC_STREAMING_TEST_OBJECT,
&mut output,
&fi,
&files,
&disks,
&erasure,
16 * 1024,
1,
0,
0,
0,
false,
true,
true,
false,
0,
0,
GET_OBJECT_PATH_SET_DISK,
"plain",
"small",
)
.await
.expect_err("unreadable quantized range must fail");
assert_eq!(error, expected);
assert!(output.is_empty());
}
}
fn capture_read_repair_submitter(
_request: rustfs_heal_contracts::heal_channel::HealChannelRequest,
) -> ReadRepairAdmissionFuture {
@@ -6150,6 +6571,227 @@ mod tests {
(dirs, disks)
}
#[tokio::test]
#[serial]
async fn quantized_range_preserves_frame_shard_block_and_odd_tail_boundaries() {
const BLOCK_SIZE: usize = 64 * 1024;
const FRAME_SIZE: usize = 16 * 1024;
const TAIL_SIZE: usize = 17_409;
let payload: Vec<u8> = (0..BLOCK_SIZE + TAIL_SIZE)
.map(|index| u8::try_from(index % 251).expect("fixture byte fits u8"))
.collect();
let erasure = coding::Erasure::new(2, 2, BLOCK_SIZE);
let mut encoded = vec![Vec::new(); erasure.total_shard_count()];
for block in payload.chunks(BLOCK_SIZE) {
for (index, shard) in erasure
.encode_data(block)
.expect("fixture block encodes")
.into_iter()
.enumerate()
{
let mut writer = BitrotWriter::new(Vec::new(), FRAME_SIZE, HashAlgorithm::HighwayHash256S);
for frame in shard.chunks(FRAME_SIZE) {
writer.write(frame).await.expect("fixture frame writes");
}
encoded[index].extend(writer.into_inner());
}
}
let mut fi = FileInfo::new(CODEC_STREAMING_TEST_OBJECT, 2, 2);
fi.volume = CODEC_STREAMING_TEST_BUCKET.to_string();
fi.size = i64::try_from(payload.len()).expect("fixture size fits i64");
fi.erasure.block_size = BLOCK_SIZE;
fi.erasure.distribution = (1..=erasure.total_shard_count()).collect();
crate::object_api::insert_ec_read_quantum(&mut fi.metadata, FRAME_SIZE);
fi.add_object_part(1, "fixture-etag".to_string(), payload.len(), None, fi.size, None, None);
let files: Vec<FileInfo> = encoded
.iter()
.enumerate()
.map(|(index, bytes)| {
let mut file = fi.clone();
file.erasure.index = index + 1;
file.data = Some(Bytes::copy_from_slice(bytes));
file
})
.collect();
let (_dirs, disks) = local_test_disks(erasure.total_shard_count(), CODEC_STREAMING_TEST_BUCKET).await;
for damage in ["healthy", "corrupt-tail", "missing-data"] {
let mut files = files.clone();
let mut disks = disks.clone();
if damage == "corrupt-tail" {
let mut bytes = files[0].data.take().expect("tail shard fixture").to_vec();
*bytes.last_mut().expect("odd tail byte") ^= 0x80;
files[0].data = Some(Bytes::from(bytes));
} else if damage == "missing-data" {
disks[0] = None;
}
for (offset, length) in [
(3, 29),
(FRAME_SIZE - 3, 9),
(erasure.shard_size() - 3, 9),
(BLOCK_SIZE - 3, 9),
(BLOCK_SIZE + TAIL_SIZE.div_ceil(2) - 3, 9),
(BLOCK_SIZE, TAIL_SIZE),
(payload.len() - 7, 7),
(0, payload.len()),
] {
let mut output = Vec::new();
SetDisks::get_object_with_fileinfo(
CODEC_STREAMING_TEST_BUCKET,
CODEC_STREAMING_TEST_OBJECT,
Arc::new(ErasureCache::new()),
offset,
i64::try_from(length).expect("range length fits i64"),
&mut output,
fi.clone(),
files.clone(),
&disks,
0,
0,
false,
true,
true,
false,
GET_OBJECT_PATH_SET_DISK,
"plain",
"small",
)
.await
.unwrap_or_else(|error| panic!("{damage}, offset={offset}, length={length}: range must recover: {error}"));
assert_eq!(
output,
payload[offset..offset + length],
"{damage}, offset={offset}, length={length}: frame/shard/block/tail order must stay exact"
);
}
}
}
#[tokio::test]
#[serial]
async fn quantized_range_direct_memory_and_cross_shard_header_offset_coalesce() {
const BLOCK_SIZE: usize = 64 * 1024;
const FRAME_SIZE: usize = 16 * 1024;
let payload: Vec<u8> = (0..BLOCK_SIZE)
.map(|index| u8::try_from(index % 251).expect("fixture byte fits u8"))
.collect();
let erasure = coding::Erasure::new(2, 2, BLOCK_SIZE);
let mut encoded = vec![Vec::new(); erasure.total_shard_count()];
for (index, shard) in erasure
.encode_data(&payload)
.expect("fixture block encodes")
.into_iter()
.enumerate()
{
let mut writer = BitrotWriter::new(Vec::new(), FRAME_SIZE, HashAlgorithm::HighwayHash256S);
for frame in shard.chunks(FRAME_SIZE) {
writer.write(frame).await.expect("fixture frame writes");
}
encoded[index].extend(writer.into_inner());
}
let mut fi = FileInfo::new(CODEC_STREAMING_TEST_OBJECT, 2, 2);
fi.volume = CODEC_STREAMING_TEST_BUCKET.to_string();
fi.name = CODEC_STREAMING_TEST_OBJECT.to_string();
fi.size = i64::try_from(payload.len()).expect("fixture size fits i64");
fi.is_latest = true;
fi.version_id = Some(Uuid::nil());
fi.data_dir = Some(Uuid::nil());
fi.mod_time = Some(time::OffsetDateTime::UNIX_EPOCH);
fi.metadata.insert("etag".to_string(), "fixture-etag".to_string());
fi.erasure.block_size = BLOCK_SIZE;
fi.erasure.distribution = (1..=erasure.total_shard_count()).collect();
crate::object_api::insert_ec_read_quantum(&mut fi.metadata, FRAME_SIZE);
fi.add_object_part(1, "fixture-etag".to_string(), payload.len(), fi.mod_time, fi.size, None, None);
fi.set_inline_data();
let files: Vec<FileInfo> = encoded
.iter()
.enumerate()
.map(|(index, bytes)| {
let mut file = fi.clone();
file.erasure.index = index + 1;
file.data = Some(Bytes::copy_from_slice(bytes));
file
})
.collect();
let head_fi = files[0].clone();
let (_dirs, disks) = local_test_disks(erasure.total_shard_count(), CODEC_STREAMING_TEST_BUCKET).await;
let body = SetDisks::try_get_object_direct_data_shards_with_fileinfo(
CODEC_STREAMING_TEST_BUCKET,
CODEC_STREAMING_TEST_OBJECT,
Arc::new(ErasureCache::new()),
&head_fi,
&files,
&disks,
false,
"plain",
"small",
)
.await
.expect("bounded-quantum direct-memory read should not error")
.expect("healthy bounded-quantum single-block object should use direct-memory fast path");
assert_eq!(body.as_ref(), payload.as_slice());
let mut corrupted_files = files.clone();
let mut shard0 = corrupted_files[0].data.take().expect("shard 0 fixture").to_vec();
*shard0.last_mut().expect("second frame byte") ^= 0x80;
corrupted_files[0].data = Some(Bytes::from(shard0));
let corrupted_head = corrupted_files[0].clone();
let fallback = SetDisks::try_get_object_direct_data_shards_with_fileinfo(
CODEC_STREAMING_TEST_BUCKET,
CODEC_STREAMING_TEST_OBJECT,
Arc::new(ErasureCache::new()),
&corrupted_head,
&corrupted_files,
&disks,
false,
"plain",
"small",
)
.await
.expect("corrupted bounded-quantum direct-memory read should return Ok(None) for parity fallback");
assert!(fallback.is_none(), "corrupted second frame must fail closed in direct-memory path");
// A 32 KiB Range GET starting at offset 4 (skipping a 4-byte framing header)
// starts in the first q-frame of shard 0 and crosses into shard 1. It must
// coalesce into 1 quantized block decode instead of opening the shards twice.
let calls = crate::set_disk::disk_call_counters::observe(CODEC_STREAMING_TEST_OBJECT);
let mut range_out = Vec::new();
SetDisks::read_quantized_range(
CODEC_STREAMING_TEST_BUCKET,
CODEC_STREAMING_TEST_OBJECT,
&mut range_out,
&fi,
&files,
&disks,
&erasure,
FRAME_SIZE,
32 * 1024,
0,
0,
4,
false,
true,
true,
false,
0,
0,
GET_OBJECT_PATH_SET_DISK,
"plain",
"small",
)
.await
.expect("coalesced cross-shard range read must succeed");
assert_eq!(range_out, payload[4..4 + 32 * 1024]);
assert_eq!(
calls.total(crate::set_disk::disk_call_counters::KIND_INLINE_SHARD_OPEN),
2,
"cross-shard range starting in frame 0 of shard 0 must open each data shard only once"
);
}
#[test]
fn shard_read_cost_for_endpoint_maps_topology_classes() {
let local_hosts = vec!["node-a:9000".to_string()];
@@ -36,6 +36,12 @@ pub(crate) struct ShardReadRepair {
}
impl ShardReadRepair {
pub(crate) fn snapshot(&self) -> Self {
Self {
observed: AtomicU8::new(self.observed.load(Ordering::Acquire)),
}
}
pub(crate) fn observe(&self, errors: &[Option<Error>]) {
let flags = errors.iter().flatten().fold(0, |flags, error| {
flags
+179
View File
@@ -1120,6 +1120,155 @@ pub fn record_get_object_stage_duration_by_size(
.record(duration_secs);
}
/// Record the shape of a quantized Range request.
///
/// These counters deliberately use bounded labels. The requested-byte and
/// quorum-window totals estimate program-visible read amplification without
/// pretending to measure physical media or RPC bytes.
#[inline(always)]
pub fn record_get_object_quantized_range_request(
path: &'static str,
object_class: &'static str,
size_bucket: &'static str,
quantum: &'static str,
requested_bytes: u64,
) {
if !get_stage_metrics_enabled() {
return;
}
counter!(
"rustfs_io_get_object_quantized_range_requests_total",
"path" => path,
"object_class" => object_class,
"size_bucket" => size_bucket,
"quantum" => quantum
)
.increment(1);
counter!(
"rustfs_io_get_object_quantized_range_requested_bytes_total",
"path" => path,
"object_class" => object_class,
"size_bucket" => size_bucket,
"quantum" => quantum
)
.increment(requested_bytes);
}
/// Record one q-aligned segment planned for a quantized Range request.
#[inline(always)]
pub fn record_get_object_quantized_range_segment(
path: &'static str,
object_class: &'static str,
size_bucket: &'static str,
quantum: &'static str,
) {
if !get_stage_metrics_enabled() {
return;
}
counter!(
"rustfs_io_get_object_quantized_range_segments_total",
"path" => path,
"object_class" => object_class,
"size_bucket" => size_bucket,
"quantum" => quantum
)
.increment(1);
}
/// Record one reader setup performed for a quantized Range segment.
#[inline(always)]
pub fn record_get_object_quantized_range_reader_setup(
path: &'static str,
object_class: &'static str,
size_bucket: &'static str,
quantum: &'static str,
) {
if !get_stage_metrics_enabled() {
return;
}
counter!(
"rustfs_io_get_object_quantized_range_reader_setups_total",
"path" => path,
"object_class" => object_class,
"size_bucket" => size_bucket,
"quantum" => quantum
)
.increment(1);
}
/// Record the q-window bytes made available by one quantized Range setup.
#[inline(always)]
pub fn record_get_object_quantized_range_quorum_window_bytes(
path: &'static str,
object_class: &'static str,
size_bucket: &'static str,
quantum: &'static str,
bytes: u64,
) {
if !get_stage_metrics_enabled() {
return;
}
counter!(
"rustfs_io_get_object_quantized_range_quorum_window_bytes_total",
"path" => path,
"object_class" => object_class,
"size_bucket" => size_bucket,
"quantum" => quantum
)
.increment(bytes);
}
/// Record authenticated bitrot frames consumed by one quantized Range decode.
///
/// This is a program-visible read-shape counter. It does not represent physical
/// disk I/O, network RPCs, or the number of returned bytes.
#[inline(always)]
pub fn record_get_object_quantized_range_frames(
path: &'static str,
object_class: &'static str,
size_bucket: &'static str,
quantum: &'static str,
frames: u64,
) {
if !get_stage_metrics_enabled() {
return;
}
counter!(
"rustfs_io_get_object_quantized_range_frames_total",
"path" => path,
"object_class" => object_class,
"size_bucket" => size_bucket,
"quantum" => quantum
)
.increment(frames);
}
/// Record underlying AsyncRead dispatches used by one quantized Range decode.
///
/// A dispatch is one call into the shard reader. It is intentionally separate
/// from frames because a partial reader can require multiple dispatches, while
/// an in-memory/chunk handoff can require none.
#[inline(always)]
pub fn record_get_object_quantized_range_read_dispatches(
path: &'static str,
object_class: &'static str,
size_bucket: &'static str,
quantum: &'static str,
dispatches: u64,
) {
if !get_stage_metrics_enabled() {
return;
}
counter!(
"rustfs_io_get_object_quantized_range_read_dispatches_total",
"path" => path,
"object_class" => object_class,
"size_bucket" => size_bucket,
"quantum" => quantum
)
.increment(dispatches);
}
/// Record GetObject metadata fanout duration.
#[inline(always)]
pub fn record_get_object_metadata_fanout_duration(path: &'static str, duration_secs: f64) {
@@ -3729,6 +3878,16 @@ mod tests {
fn test_record_get_object_stage_metrics() {
record_get_object_stage_duration("s3_handler", "request_context", 0.001);
record_get_object_stage_duration_by_size("legacy_duplex", "metadata", "plain_single_part", "le_4kib", 0.001);
record_get_object_quantized_range_request("legacy_duplex", "plain_single_part", "le_1mib", "1mib", 64 * 1024);
record_get_object_quantized_range_segment("legacy_duplex", "plain_single_part", "le_1mib", "1mib");
record_get_object_quantized_range_reader_setup("legacy_duplex", "plain_single_part", "le_1mib", "1mib");
record_get_object_quantized_range_quorum_window_bytes(
"legacy_duplex",
"plain_single_part",
"le_1mib",
"1mib",
2 * 1024 * 1024,
);
record_get_object_reader_path("codec_streaming");
record_get_object_reader_path_by_size("codec_streaming", "plain_single_part", "le_1mib");
record_get_object_codec_streaming_fallback("range");
@@ -3815,6 +3974,26 @@ mod tests {
assert!(0.005_f64.is_sign_positive());
}
#[test]
fn quantized_range_shape_metrics_capture_frame_and_dispatch_totals() {
let _guard = METRICS_FLAG_LOCK.lock().unwrap_or_else(|e| e.into_inner());
let recorder = DebuggingRecorder::new();
let snapshotter = recorder.snapshotter();
metrics::with_local_recorder(&recorder, || {
set_get_stage_metrics_enabled(true);
record_get_object_quantized_range_frames("legacy_duplex", "plain_single_part", "le_1mib", "256kib", 8);
record_get_object_quantized_range_read_dispatches("legacy_duplex", "plain_single_part", "le_1mib", "256kib", 11);
set_get_stage_metrics_enabled(false);
});
let rows = snapshotter.snapshot().into_vec();
assert_eq!(counter_total(&rows, "rustfs_io_get_object_quantized_range_frames_total"), Some(8));
assert_eq!(
counter_total(&rows, "rustfs_io_get_object_quantized_range_read_dispatches_total"),
Some(11)
);
}
#[test]
fn test_get_object_shard_locality_observe_only_metrics_smoke() {
let remote_scheduled = 2;
+4
View File
@@ -39,6 +39,10 @@ pub struct MultipartUploadResult {
pub upload_id: String,
pub checksum_algo: Option<String>,
pub checksum_type: Option<String>,
/// The block size persisted in the multipart session's authoritative
/// `FileInfo`. The application uses this value for the Create-MPU
/// response; later parts never re-select the layout.
pub effective_ec_block_size: Option<u64>,
}
#[derive(Debug, Default, Clone)]
+50
View File
@@ -120,6 +120,56 @@ distribution[i-1] = 1 + ((start + i) % N) for i in 1..=N // a cyclic rotatio
### 4.1 Block size and shard sizing
- **INVARIANT — block size.** New objects use `BLOCK_SIZE_V2 = 1 MiB` ([object_api/mod.rs](../../crates/ecstore/src/object_api/mod.rs)); it is stored per version in `ErasureInfo.block_size` and must be read back from metadata, never assumed.
#### 4.1.1 Client block-size hint (RustFS V1)
The default remains 1 MiB. During the staged rollout, a direct `PutObject` or
`CreateMultipartUpload` may request one of the supported sizes (64 KiB, 256
KiB, 1 MiB, or 4 MiB) with the user metadata key
`x-amz-meta-rustfs-ec-block-size-hint`. The feature is applied only when both
`RUSTFS_EC_BLOCK_SIZE_HINT_ENABLE=true` and
`RUSTFS_EC_BLOCK_SIZE_HINT_FLEET_CONFIRMED=true`; both flags default to
`false`. Restart server processes after changing either environment flag.
Values are ASCII decimal digits, 1–20 digits, and greater than zero.
Malformed values or duplicate raw headers/case-variant metadata names return
`InvalidArgument` after authentication. An unsupported numeric value is
ignored and uses the default. Before both rollout flags are true, the value is
ordinary user metadata and is not parsed for layout selection.
The hint is ignored for compression, server-side encryption, protected shard
integrity, replication/data-movement, recovery, and other internal writes.
Successful and ignored writes expose the decision through
`x-rustfs-layout-hint-status` and, when ignored,
`x-rustfs-layout-hint-reason`; all committed writes expose the actual value in
`x-rustfs-effective-ec-block-size`. The selected value is persisted in
`ErasureInfo.block_size` and is authoritative for every later read, range, and
healing operation. Multipart uploads select the value once at creation;
`UploadPart` hints are ignored and all parts use the stored upload geometry.
Metadata quorum identity includes the decode geometry, including block size,
while the legacy compatibility hash remains available for callers that need
it. No performance target is implied without measurements on the target
workload.
#### 4.1.2 Bounded bitrot read quantum (RustFS V1)
New, eligible EC writes may split each shard into smaller authenticated bitrot
frames to reduce the amount of shard data read for small ranges. The selected
frame size is recorded in the internal `ec-read-quantum` metadata pair; an
absent marker means the historical one-frame-per-shard layout. Enable writes
only after every reader and writer in the fleet understands this marker and
frame format, using both `RUSTFS_EC_READ_QUANTUM_ENABLE=true` and
`RUSTFS_EC_READ_QUANTUM_FLEET_CONFIRMED=true`. Both flags default to `false`.
Restart server processes after changing either environment flag.
Protected-integrity and inline writes retain their existing framing.
This changes the physical bitrot frame boundaries. Older RustFS readers do not
interpret the marker and cannot read objects written with bounded frames.
Disabling the write gate prevents additional objects from using that format;
it does not make existing bounded-frame objects readable by an older binary.
Keep compatible readers available or rewrite/export those objects before a
rollback to a version without this reader support.
- **INVARIANT — shard-size formulas (frozen for on-disk compatibility).** Two formulas exist and must both be preserved:
- Modern ([erasure.rs](../../crates/ecstore/src/erasure/coding/erasure.rs)): `calc_shard_size(block_size, data_shards) = block_size.div_ceil(data_shards)` — plain ceiling division, no even rounding. This is the MinIO-compatible sizing (MinIO's `Erasure.ShardSize` is the same plain `ceil(block_size / data_shards)`); the modern GF(2⁸) path uses it for all new and MinIO-migrated data.
- Legacy ([erasure.rs](../../crates/ecstore/src/erasure/coding/erasure.rs)): `calc_shard_size_legacy = (block_size.div_ceil(data_shards) + 1) & !1` — round the ceiling **up to the nearest even number**. This even-padded form is **RustFS-legacy-only** and matches `filemeta`'s own even-padded `calc_shard_size` ([fileinfo.rs](../../crates/filemeta/src/fileinfo.rs)); it is **not** MinIO's sizing and the two differ by one byte at typical geometries (for `block_size = 1 MiB`, `data_shards = 6`, plain `div_ceil` yields `174763` bytes whereas the legacy even-padded form yields `174764`). Because MinIO-migrated data is decoded by the modern path (§1), this legacy even-padding never applies to MinIO objects.
@@ -1,13 +1,8 @@
# Sampled local batch CPU diagnostics
**Use this when:** separating sampled local EC batch allocation/initialization
and filesystem wall time from worker CPU time in a default-off diagnostic build.
**Use this when:** separating sampled local EC batch allocation and filesystem wall time from worker CPU time in a default-off diagnostic build.
This diagnostic build separates allocation/initialization, metadata, open and
positioned-read elapsed time from Linux thread CPU time. It preserves the
single blocking dispatch, buffer initialization, offsets, integrity checks and
error handling. It does not implement a rustix optimization or qualify
observation overhead.
This diagnostic build separates allocation, metadata, open and positioned-read elapsed time from Linux thread CPU time. It preserves offsets, integrity checks and error handling. Windows below 512 KiB share one blocking dispatch per batch; when every window is at least 512 KiB, each shard is a separate dispatch and appears as its own diagnostic batch. On Linux each window is announced with `POSIX_FADV_WILLNEED` and then read with positioned reads into uninitialized buffer capacity, so the allocation stage contains no buffer initialization. The build does not qualify observation overhead.
Build separately from any frozen performance binary:
+91
View File
@@ -0,0 +1,91 @@
# Dynamic block size HTTP regression
**Use this when:** validating S3 block hints, multipart layout persistence, or
reads after disabling the bounded-frame write rollout.
Before performance measurements, verify physical shard checks and recovery:
```bash
cargo test -p rustfs-ecstore --lib bounded_quantum_ -- --test-threads=1
```
These regressions cover healthy deep verification, normal/deep repair after a
missing shard, exact restored frame bytes, and Range reads with the write gates
disabled. They also retain the legacy framing control and reject invalid or
conflicting persisted q metadata. HTTP GET success alone does not validate heal.
`scripts/dynamic_block_size_e2e.py` checks S3 PUT and multipart initiation hints
against a disposable RustFS endpoint. Install `boto3` in a test virtualenv and
export `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` for that endpoint. The SDK
performs SigV4 signing; the helper bypasses proxies and disables retries.
Start a freshly built RustFS binary with four isolated local volumes, compression
disabled, and these process environment variables:
```bash
RUSTFS_EC_BLOCK_SIZE_HINT_ENABLE=true
RUSTFS_EC_BLOCK_SIZE_HINT_FLEET_CONFIRMED=true
RUSTFS_EC_READ_QUANTUM_ENABLE=true
RUSTFS_EC_READ_QUANTUM_FLEET_CONFIRMED=true
```
For four test directories on the same physical disk, the existing local-test
configuration `RUSTFS_UNSAFE_BYPASS_DISK_CHECK=true` is also necessary. Use it
only for this disposable functional test; it does not validate disk independence.
Four volumes use two data shards and two parity shards. This geometry supports
the bounded bitrot read quantum; a geometry with six or twelve data shards does
not currently select a smaller quantum for the allowed block sizes.
```bash
python scripts/dynamic_block_size_e2e.py --endpoint http://127.0.0.1:19180 \
--bucket dynamic-block-restart-test --keep
```
The helper writes non-inline deterministic payloads using 64 KiB, 256 KiB, 1 MiB,
and 4 MiB hints, plus default and unsupported-size probes. It checks actual B
and hint status/reason in write responses, then validates HEAD, full GET, and
Ranges spanning EC block and multipart boundaries. A malformed enabled hint
must return `InvalidArgument`. JSON output contains no credentials. Use a fresh
bucket for write runs; existing probe objects are never overwritten.
Stop only that test server, restart the same binary with the same volumes and
all four gates false, then verify persisted layouts without rewriting objects:
```bash
python scripts/dynamic_block_size_e2e.py --endpoint http://127.0.0.1:19180 \
--bucket dynamic-block-restart-test --verify-only
```
Separately run against a server with block hint enable false using
`--write-policy disabled`, or enable true and fleet confirmation false using
`--write-policy fleet-unconfirmed`. Both must report ignored hints and actual
B of 1 MiB. Without `--keep`, ordinary write runs remove only the objects they
created and their disposable bucket. Verify-only runs never delete data.
B is exposed by `x-rustfs-effective-ec-block-size`. The internal q marker is
deliberately filtered from public metadata. For isolated local volumes, decode
an object's `xl.meta` using the existing filemeta example:
```bash
cargo run -p rustfs-filemeta --example dump_fileinfo -- /path/to/object/xl.meta
```
Confirm both `x-rustfs-internal-ec-read-quantum` and
`x-minio-internal-ec-read-quantum` agree. With two data shards, non-inline 4 MiB
objects use q=1048576; 64 KiB and 256 KiB use q=16384 and q=65536 respectively.
The 1 MiB layout keeps the historical full shard frame and has no q marker.
For a cross-repository probe, write one task-prefixed object with BrewFS's
`object-put-bench`, then check its actual layout and Range bytes:
```bash
python scripts/dynamic_block_size_e2e.py --endpoint http://127.0.0.1:19180 \
--bucket brewfs-task-bucket --external-prefix dynamic-block-task/ \
--external-block-size 4194304 --external-size 6291713 \
--external-sha256 "$EXPECTED_SHA256"
```
External verification reads only the selected prefix and never deletes it. The
optional size and SHA-256 checks validate the intended payload before comparing
Range bytes to full GET bytes. SHA-256 must be exactly 64 hexadecimal characters.
@@ -15,6 +15,7 @@
//! CompleteMultipartUpload.
use super::*;
use crate::app::storage_api::multipart_usecase::EC_BLOCK_SIZE_RESPONSE_HEADER;
use crate::app::storage_api::s3::dto::{
CompleteMultipartUploadInput, CompleteMultipartUploadOutput, CompletedPart, ETag, ServerSideEncryption,
};
@@ -583,6 +584,7 @@ impl DefaultMultipartUsecase {
version_id: mpu_version,
..Default::default()
};
let effective_ec_block_size = obj_info.effective_ec_block_size();
// Set object info for event notification
helper = helper.object(obj_info);
if let Some(version_id) = &mpu_version_for_event {
@@ -591,6 +593,11 @@ impl DefaultMultipartUsecase {
let mut response = S3Response::new(output);
crate::app::object_usecase::inject_additional_checksum_headers(&mut response.headers, &complete_extra_checksum_headers);
if let Some(block_size) = effective_ec_block_size
&& let Ok(value) = HeaderValue::from_str(&block_size.to_string())
{
response.headers.insert(EC_BLOCK_SIZE_RESPONSE_HEADER, value);
}
if let Some(algorithm) = ssec_algorithm.as_deref() {
let value = HeaderValue::from_str(algorithm)
.map_err(|_| s3_error!(InternalError, "Invalid stored SSE-C algorithm metadata"))?;
+159 -3
View File
@@ -15,6 +15,10 @@
//! CreateMultipartUpload.
use super::*;
use crate::app::storage_api::multipart_usecase::{
DEFAULT_EC_BLOCK_SIZE, EC_BLOCK_SIZE_HINT_INTERNAL_SUFFIX, EC_BLOCK_SIZE_HINT_METADATA_KEY, EC_BLOCK_SIZE_HINT_REASON_HEADER,
EC_BLOCK_SIZE_HINT_STATUS_HEADER, EC_BLOCK_SIZE_RESPONSE_HEADER, EcBlockSizeHintOutcome, resolve_ec_block_size_hint,
};
use crate::app::storage_api::s3::dto::{
ChecksumAlgorithm, ChecksumType, CreateMultipartUploadInput, CreateMultipartUploadOutput,
};
@@ -40,6 +44,61 @@ fn create_multipart_upload_metadata(
metadata
}
fn extract_multipart_ec_block_size_hint(headers: &HeaderMap, metadata: &HashMap<String, String>) -> S3Result<Option<String>> {
let enabled = rustfs_utils::get_env_bool(
rustfs_config::ENV_EC_BLOCK_SIZE_HINT_ENABLE,
rustfs_config::DEFAULT_EC_BLOCK_SIZE_HINT_ENABLE,
);
let fleet_confirmed = rustfs_utils::get_env_bool(
rustfs_config::ENV_EC_BLOCK_SIZE_HINT_FLEET_CONFIRMED,
rustfs_config::DEFAULT_EC_BLOCK_SIZE_HINT_FLEET_CONFIRMED,
);
let strict = enabled && fleet_confirmed;
let header_name = http::HeaderName::from_static("x-amz-meta-rustfs-ec-block-size-hint");
let raw_values = headers
.get_all(header_name)
.iter()
.map(|value| {
if strict {
value.to_str().map(str::to_owned).map_err(|_| {
S3Error::with_message(
S3ErrorCode::InvalidArgument,
"rustfs-ec-block-size-hint must be ASCII decimal digits".to_string(),
)
})
} else {
Ok(String::from_utf8_lossy(value.as_bytes()).into_owned())
}
})
.collect::<Result<Vec<_>, _>>()?;
if strict && raw_values.len() > 1 {
return Err(S3Error::with_message(
S3ErrorCode::InvalidArgument,
"duplicate rustfs-ec-block-size-hint metadata is not allowed".to_string(),
));
}
let metadata_values = metadata
.iter()
.filter(|(key, _)| key.eq_ignore_ascii_case(EC_BLOCK_SIZE_HINT_METADATA_KEY))
.map(|(_, value)| value.clone())
.collect::<Vec<_>>();
if strict && metadata_values.len() > 1 {
return Err(S3Error::with_message(
S3ErrorCode::InvalidArgument,
"duplicate rustfs-ec-block-size-hint metadata is not allowed".to_string(),
));
}
let hint = raw_values.into_iter().next().or_else(|| metadata_values.into_iter().next());
if strict && let Some(value) = hint.as_deref() {
resolve_ec_block_size_hint(Some(value), true, true, true).map_err(|error| {
S3Error::with_message(S3ErrorCode::InvalidArgument, format!("invalid rustfs-ec-block-size-hint: {error}"))
})?;
}
Ok(hint)
}
/// A multipart session advertises disk compression only when the staged-rollout
/// switch (`RUSTFS_COMPRESSION_MULTIPART_ENABLED`) is on, the object key/headers
/// qualify, AND the session is not an SSE-C ciphertext-passthrough replication
@@ -131,6 +190,7 @@ impl DefaultMultipartUsecase {
)?;
let mut metadata = create_multipart_upload_metadata(input_metadata, &req.headers, tagging, storage_class.as_ref());
let ec_block_size_hint = extract_multipart_ec_block_size_hint(&req.headers, &metadata)?;
if let Some(limit) = multipart_max_total_object_size {
insert_str(&mut metadata, SUFFIX_MAX_TOTAL_OBJECT_SIZE, limit.to_string());
}
@@ -223,11 +283,12 @@ impl DefaultMultipartUsecase {
None => (None, None),
};
if should_advertise_session_compression(
let session_compression = should_advertise_session_compression(
is_multipart_disk_compression_enabled(),
ciphertext_passthrough,
is_disk_compressible(&req.headers, &key),
) {
);
if session_compression {
rustfs_utils::http::insert_str(
&mut metadata,
rustfs_utils::http::SUFFIX_COMPRESSION,
@@ -241,6 +302,47 @@ impl DefaultMultipartUsecase {
.await
.map_err(ApiError::from)?;
apply_bucket_generation_guard(&req, &bucket, &mut opts)?;
let shard_integrity_rollout =
rustfs_utils::get_env_bool(rustfs_config::ENV_SHARD_INTEGRITY_WRITE, rustfs_config::DEFAULT_SHARD_INTEGRITY_WRITE)
&& rustfs_utils::get_env_bool(
rustfs_config::ENV_SHARD_INTEGRITY_FLEET_CONFIRMED,
rustfs_config::DEFAULT_SHARD_INTEGRITY_FLEET_CONFIRMED,
);
let ec_block_size_hint_resolution = resolve_ec_block_size_hint(
ec_block_size_hint.as_deref(),
rustfs_utils::get_env_bool(
rustfs_config::ENV_EC_BLOCK_SIZE_HINT_ENABLE,
rustfs_config::DEFAULT_EC_BLOCK_SIZE_HINT_ENABLE,
),
rustfs_utils::get_env_bool(
rustfs_config::ENV_EC_BLOCK_SIZE_HINT_FLEET_CONFIRMED,
rustfs_config::DEFAULT_EC_BLOCK_SIZE_HINT_FLEET_CONFIRMED,
),
!session_compression
&& effective_sse.is_none()
&& sse_customer_algorithm.is_none()
&& effective_kms_key_id.is_none()
&& !ciphertext_passthrough
&& !replication_authorized
&& !opts.data_movement
&& opts.shard_integrity_write_mode.is_none()
&& !shard_integrity_rollout,
)
.map_err(|err| {
S3Error::with_message(S3ErrorCode::InvalidArgument, format!("invalid rustfs-ec-block-size-hint: {err}"))
})?;
// A 1 MiB hint selects the historical storage default, so avoid
// writing a redundant transient marker on the multipart hot path.
if matches!(ec_block_size_hint_resolution.outcome, EcBlockSizeHintOutcome::Applied)
&& ec_block_size_hint_resolution.effective_block_size != DEFAULT_EC_BLOCK_SIZE
{
rustfs_utils::http::metadata_compat::insert_str(
&mut opts.user_defined,
EC_BLOCK_SIZE_HINT_INTERNAL_SUFFIX,
ec_block_size_hint_resolution.effective_block_size.to_string(),
);
}
let dsc =
must_replicate_object(&bucket, &key, &mt2, "".to_string(), opts.delete_marker_replication_status(), opts.clone())
@@ -292,6 +394,7 @@ impl DefaultMultipartUsecase {
upload_id,
checksum_algo,
checksum_type,
effective_ec_block_size,
} = store
.new_multipart_upload(&bucket, &key, &opts)
.await
@@ -310,7 +413,23 @@ impl DefaultMultipartUsecase {
..Default::default()
};
let result = Ok(S3Response::new(output));
let mut response = S3Response::new(output);
if let Some(block_size) = effective_ec_block_size
&& let Ok(value) = HeaderValue::from_str(&block_size.to_string())
{
response.headers.insert(EC_BLOCK_SIZE_RESPONSE_HEADER, value);
}
if let Some(status) = ec_block_size_hint_resolution.outcome.status_header() {
response
.headers
.insert(EC_BLOCK_SIZE_HINT_STATUS_HEADER, HeaderValue::from_static(status));
}
if let Some(reason) = ec_block_size_hint_resolution.outcome.reason_header() {
response
.headers
.insert(EC_BLOCK_SIZE_HINT_REASON_HEADER, HeaderValue::from_static(reason));
}
let result = Ok(response);
let _ = helper.complete(&result);
result
}
@@ -321,6 +440,43 @@ mod tests {
use super::super::test_support::*;
use super::*;
#[test]
#[serial_test::serial(ec_block_size_hint)]
fn multipart_ec_block_size_hint_validates_create_upload_metadata() {
temp_env::with_vars(
[
(rustfs_config::ENV_EC_BLOCK_SIZE_HINT_ENABLE, Some("true")),
(rustfs_config::ENV_EC_BLOCK_SIZE_HINT_FLEET_CONFIRMED, Some("true")),
],
|| {
let header_name = http::HeaderName::from_static("x-amz-meta-rustfs-ec-block-size-hint");
let mut headers = HeaderMap::new();
headers.insert(header_name, HeaderValue::from_static("65_536"));
let error =
extract_multipart_ec_block_size_hint(&headers, &HashMap::new()).expect_err("malformed create-upload hint");
assert_eq!(error.code(), &S3ErrorCode::InvalidArgument);
},
);
temp_env::with_vars(
[
(rustfs_config::ENV_EC_BLOCK_SIZE_HINT_ENABLE, Some("false")),
(rustfs_config::ENV_EC_BLOCK_SIZE_HINT_FLEET_CONFIRMED, Some("false")),
],
|| {
let header_name = http::HeaderName::from_static("x-amz-meta-rustfs-ec-block-size-hint");
let mut headers = HeaderMap::new();
headers.insert(header_name, HeaderValue::from_static("not-a-number"));
assert_eq!(
extract_multipart_ec_block_size_hint(&headers, &HashMap::new())
.unwrap()
.as_deref(),
Some("not-a-number")
);
},
);
}
#[test]
fn session_compression_is_advertised_only_for_non_passthrough_compressible_uploads() {
// (multipart_enabled, ciphertext_passthrough, disk_compressible, expected)
+13
View File
@@ -398,6 +398,7 @@ struct GetObjectStrategyContext {
struct GetObjectOutputContext {
output: GetObjectOutput,
event_info: Option<ObjectInfo>,
effective_ec_block_size: Option<usize>,
response_content_length: i64,
optimal_buffer_size: usize,
extra_checksum_headers: Vec<(&'static str, String)>,
@@ -4006,6 +4007,7 @@ impl DefaultObjectUsecase {
method: &hyper::Method,
headers: &HeaderMap,
event_info: Option<ObjectInfo>,
effective_ec_block_size: Option<usize>,
version_id_for_event: String,
output: GetObjectOutput,
extra_checksum_headers: Vec<(&'static str, String)>,
@@ -4023,6 +4025,11 @@ impl DefaultObjectUsecase {
// Emit XXHash3/64/128 and SHA-512 checksums that s3s GetObjectOutput cannot
// carry (#1257). This is the download-side integrity path AWS SDKs verify.
inject_additional_checksum_headers(&mut response.headers, &extra_checksum_headers);
if let Some(block_size) = effective_ec_block_size
&& let Ok(value) = HeaderValue::from_str(&block_size.to_string())
{
response.headers.insert(EC_BLOCK_SIZE_RESPONSE_HEADER, value);
}
record_get_object_s3_handler_stage_duration(GET_OBJECT_STAGE_RESPONSE_HEADER_INJECT, header_inject_start);
let result = Ok(response);
let event_complete_start = rustfs_io_metrics::get_stage_metrics_enabled().then(std::time::Instant::now);
@@ -4067,6 +4074,7 @@ impl DefaultObjectUsecase {
where
F: FnOnce(&ObjectInfo) -> Option<GetObjectResumeControl<DynReader>>,
{
let effective_ec_block_size = info.effective_ec_block_size();
let strategy_start = rustfs_io_metrics::get_stage_metrics_enabled().then(std::time::Instant::now);
let strategy = self.finalize_get_object_strategy(
manager,
@@ -4177,6 +4185,7 @@ impl DefaultObjectUsecase {
Ok(GetObjectOutputContext {
output,
event_info,
effective_ec_block_size,
response_content_length,
optimal_buffer_size,
extra_checksum_headers: checksums.extra,
@@ -4717,6 +4726,7 @@ impl DefaultObjectUsecase {
let GetObjectOutputContext {
output,
event_info,
effective_ec_block_size,
response_content_length,
optimal_buffer_size,
extra_checksum_headers,
@@ -4738,6 +4748,7 @@ impl DefaultObjectUsecase {
&req.method,
&req.headers,
event_info,
effective_ec_block_size,
version_id_for_event,
output,
extra_checksum_headers,
@@ -6491,6 +6502,7 @@ mod tests {
&req.method,
&req.headers,
None,
Some(262_144),
String::new(),
GetObjectOutput::default(),
Vec::new(),
@@ -6499,6 +6511,7 @@ mod tests {
.expect("finalize response");
assert_eq!(response.headers.get(http::header::ACCEPT_RANGES).unwrap(), ACCEPT_RANGES_BYTES);
assert_eq!(response.headers.get(EC_BLOCK_SIZE_RESPONSE_HEADER).unwrap(), "262144");
}
#[test]
+6
View File
@@ -601,6 +601,7 @@ impl DefaultObjectUsecase {
} else {
0
};
let effective_ec_block_size = info.effective_ec_block_size();
let output = HeadObjectOutput {
content_length: Some(content_length),
content_type,
@@ -654,6 +655,11 @@ impl DefaultObjectUsecase {
// Emit additional-checksum headers (XXHash3/64/128, SHA-512) that s3s cannot
// carry on the typed HeadObjectOutput (#1257).
inject_additional_checksum_headers(&mut response.headers, &extra_checksum_headers);
if let Some(block_size) = effective_ec_block_size
&& let Ok(value) = HeaderValue::from_str(&block_size.to_string())
{
response.headers.insert(EC_BLOCK_SIZE_RESPONSE_HEADER, value);
}
// Add x-amz-tagging-count header if object has tags
// Per S3 API spec, this header should be present in HEAD object response when tags exist
+1
View File
@@ -313,6 +313,7 @@ impl DefaultObjectUsecase {
sse_customer_key_md5: None,
},
user_metadata,
ec_block_size_hint: None,
internal_metadata,
content: internal_put_content_input(&headers, tags),
object_lock: PutObjectLockInput {
+4
View File
@@ -115,6 +115,10 @@ use super::storage_api::object_usecase::sse::{
};
use super::storage_api::object_usecase::storage_class as storageclass;
use super::storage_api::object_usecase::timeout_wrapper::{GetObjectTimeoutPolicy, RequestTimeoutWrapper};
pub(crate) use super::storage_api::object_usecase::{
DEFAULT_EC_BLOCK_SIZE, EC_BLOCK_SIZE_HINT_INTERNAL_SUFFIX, EC_BLOCK_SIZE_HINT_METADATA_KEY, EC_BLOCK_SIZE_HINT_REASON_HEADER,
EC_BLOCK_SIZE_HINT_STATUS_HEADER, EC_BLOCK_SIZE_RESPONSE_HEADER, EcBlockSizeHintOutcome, resolve_ec_block_size_hint,
};
use super::storage_api::object_usecase::{ECStore, OldCurrentSize};
use super::storage_api::object_usecase::{
RFC1123, check_preconditions, parse_object_lock_legal_hold, parse_object_lock_retention, parse_part_number_i32_to_usize,
+188
View File
@@ -27,6 +27,7 @@ use futures::FutureExt;
use crate::app::trailer_adapter::trailer_source;
use crate::auth::{RUSTFS_MAX_CONTENT_LENGTH_QUERY, VerifiedPresignedRequest, parse_presigned_put_max_content_length};
use crate::error::UploadLimitExceeded;
use http::HeaderName;
static PUT_FAILURE_LOGS: rustfs_utils::LogThrottle = rustfs_utils::LogThrottle::new(5_000);
#[cfg(test)]
@@ -769,6 +770,87 @@ pub(super) fn apply_put_request_metadata(
Ok(())
}
/// Read the block-size hint from the raw request metadata without losing
/// duplicate headers. Once rollout is enabled and fleet-confirmed, multiple
/// values or case-variant metadata names are rejected before the body is read.
fn extract_ec_block_size_hint(headers: &HeaderMap, metadata: Option<&HashMap<String, String>>) -> S3Result<Option<String>> {
let enabled = rustfs_utils::get_env_bool(
rustfs_config::ENV_EC_BLOCK_SIZE_HINT_ENABLE,
rustfs_config::DEFAULT_EC_BLOCK_SIZE_HINT_ENABLE,
);
let fleet_confirmed = rustfs_utils::get_env_bool(
rustfs_config::ENV_EC_BLOCK_SIZE_HINT_FLEET_CONFIRMED,
rustfs_config::DEFAULT_EC_BLOCK_SIZE_HINT_FLEET_CONFIRMED,
);
let strict = enabled && fleet_confirmed;
let header_name = HeaderName::from_static("x-amz-meta-rustfs-ec-block-size-hint");
let raw_values = headers
.get_all(header_name)
.iter()
.map(|value| {
if strict {
value.to_str().map(str::to_owned).map_err(|_| {
S3Error::with_message(
S3ErrorCode::InvalidArgument,
"rustfs-ec-block-size-hint must be ASCII decimal digits".to_string(),
)
})
} else {
Ok(String::from_utf8_lossy(value.as_bytes()).into_owned())
}
})
.collect::<Result<Vec<_>, _>>()?;
if strict && raw_values.len() > 1 {
return Err(S3Error::with_message(
S3ErrorCode::InvalidArgument,
"duplicate rustfs-ec-block-size-hint metadata is not allowed".to_string(),
));
}
let metadata_values = metadata
.into_iter()
.flat_map(|entries| entries.iter())
.filter(|(key, _)| key.eq_ignore_ascii_case(EC_BLOCK_SIZE_HINT_METADATA_KEY))
.map(|(_, value)| value.clone())
.collect::<Vec<_>>();
if strict && metadata_values.len() > 1 {
return Err(S3Error::with_message(
S3ErrorCode::InvalidArgument,
"duplicate rustfs-ec-block-size-hint metadata is not allowed".to_string(),
));
}
let hint = raw_values.into_iter().next().or_else(|| metadata_values.into_iter().next());
if strict && let Some(value) = hint.as_deref() {
resolve_ec_block_size_hint(Some(value), true, true, true).map_err(|error| {
S3Error::with_message(S3ErrorCode::InvalidArgument, format!("invalid rustfs-ec-block-size-hint: {error}"))
})?;
}
Ok(hint)
}
fn ec_block_size_hint_write_eligible(
should_compress: bool,
server_side_encryption_requested: bool,
inbound_replication_put: bool,
opts: &ObjectOptions,
) -> bool {
let shard_integrity_rollout =
rustfs_utils::get_env_bool(rustfs_config::ENV_SHARD_INTEGRITY_WRITE, rustfs_config::DEFAULT_SHARD_INTEGRITY_WRITE)
&& rustfs_utils::get_env_bool(
rustfs_config::ENV_SHARD_INTEGRITY_FLEET_CONFIRMED,
rustfs_config::DEFAULT_SHARD_INTEGRITY_FLEET_CONFIRMED,
);
!should_compress
&& !server_side_encryption_requested
&& !inbound_replication_put
&& !opts.preserve_ciphertext
&& !opts.data_movement
&& opts.shard_integrity_write_mode.is_none()
&& !shard_integrity_rollout
}
pub(super) fn apply_put_request_object_lock_opts(
bucket: &str,
object_lock_config_state: &metadata_sys::ObjectLockConfigState,
@@ -1042,6 +1124,9 @@ pub(super) struct PutObjectWriteRequest<'a> {
pub(super) sse: PutObjectSseInput,
/// User metadata keyed the way s3s delivers it (`x-amz-meta-` stripped).
pub(super) user_metadata: HashMap<String, String>,
/// Raw client hint retained separately so the request metadata remains
/// ordinary user metadata and storage only receives a trusted marker.
pub(super) ec_block_size_hint: Option<String>,
/// Internal `x-rustfs-internal-*` / `x-minio-internal-*` keys written
/// verbatim onto the object; empty for S3 requests.
pub(super) internal_metadata: HashMap<String, String>,
@@ -1107,6 +1192,7 @@ pub(super) struct PutObjectCommitted {
pub(super) sse_customer_algorithm: Option<SSECustomerAlgorithm>,
pub(super) sse_customer_key_md5: Option<SSECustomerKeyMD5>,
pub(super) put_extra_checksum_headers: Vec<(&'static str, String)>,
pub(super) ec_block_size_hint_outcome: EcBlockSizeHintOutcome,
completion: Option<PutObjectCompletion>,
put_request_guard: PutObjectGuard,
bucket: String,
@@ -1274,6 +1360,15 @@ impl DefaultObjectUsecase {
..
} = input;
// POST form uploads share the storage path but are a distinct S3
// operation. The V1 hint is defined for direct PutObject only, so a
// POST field remains ordinary user metadata and never selects layout.
let ec_block_size_hint = if req.extensions.get::<PostObjectRequestMarker>().is_some() {
None
} else {
extract_ec_block_size_hint(&req.headers, metadata.as_ref())?
};
// Merge SSE-C params from headers (fallback when S3 layer does not populate input)
let (h_algo, h_key, h_md5) = extract_ssec_params_from_headers(&req.headers)?;
let sse_customer_algorithm = sse_customer_algorithm.or(h_algo);
@@ -1341,6 +1436,7 @@ impl DefaultObjectUsecase {
sse_customer_key_md5,
},
user_metadata: metadata.unwrap_or_default(),
ec_block_size_hint,
internal_metadata: HashMap::new(),
content: PutObjectContentInput {
cache_control,
@@ -1411,6 +1507,21 @@ impl DefaultObjectUsecase {
// Echo XXHash3/64/128 / SHA-512 checksums that s3s PutObjectOutput has no typed
// field for (#1256).
inject_additional_checksum_headers(&mut response.headers, &committed.put_extra_checksum_headers);
if let Some(block_size) = committed.obj_info.effective_ec_block_size()
&& let Ok(value) = HeaderValue::from_str(&block_size.to_string())
{
response.headers.insert(EC_BLOCK_SIZE_RESPONSE_HEADER, value);
}
if let Some(status) = committed.ec_block_size_hint_outcome.status_header() {
response
.headers
.insert(EC_BLOCK_SIZE_HINT_STATUS_HEADER, HeaderValue::from_static(status));
}
if let Some(reason) = committed.ec_block_size_hint_outcome.reason_header() {
response
.headers
.insert(EC_BLOCK_SIZE_HINT_REASON_HEADER, HeaderValue::from_static(reason));
}
rustfs_io_metrics::record_put_object_stage_duration_from("app_response_build", response_build_stage_start);
let result = Ok(response);
committed.finish(&result);
@@ -1445,6 +1556,7 @@ impl DefaultObjectUsecase {
sse,
user_metadata,
internal_metadata,
ec_block_size_hint,
content,
object_lock,
content_md5,
@@ -2051,6 +2163,35 @@ impl DefaultObjectUsecase {
let mt2 = metadata.clone();
opts.user_defined.extend(metadata);
let ec_block_size_hint_resolution = resolve_ec_block_size_hint(
ec_block_size_hint.as_deref(),
rustfs_utils::get_env_bool(
rustfs_config::ENV_EC_BLOCK_SIZE_HINT_ENABLE,
rustfs_config::DEFAULT_EC_BLOCK_SIZE_HINT_ENABLE,
),
rustfs_utils::get_env_bool(
rustfs_config::ENV_EC_BLOCK_SIZE_HINT_FLEET_CONFIRMED,
rustfs_config::DEFAULT_EC_BLOCK_SIZE_HINT_FLEET_CONFIRMED,
),
ec_block_size_hint_write_eligible(should_compress, server_side_encryption_requested, inbound_replication_put, &opts),
)
.map_err(|err| {
S3Error::with_message(S3ErrorCode::InvalidArgument, format!("invalid rustfs-ec-block-size-hint: {err}"))
})?;
// The historical 1 MiB layout is already the storage default. Do not
// carry a redundant transient marker for that value: it only adds
// compatibility-metadata work to the hot PUT path. The persisted
// FileInfo still records the authoritative block size and response
// projection remains unchanged.
if matches!(ec_block_size_hint_resolution.outcome, EcBlockSizeHintOutcome::Applied)
&& ec_block_size_hint_resolution.effective_block_size != DEFAULT_EC_BLOCK_SIZE
{
rustfs_utils::http::metadata_compat::insert_str(
&mut opts.user_defined,
EC_BLOCK_SIZE_HINT_INTERNAL_SUFFIX,
ec_block_size_hint_resolution.effective_block_size.to_string(),
);
}
let request_id = request_context
.as_ref()
.map(|ctx| ctx.request_id.clone())
@@ -2343,6 +2484,7 @@ impl DefaultObjectUsecase {
sse_customer_algorithm,
sse_customer_key_md5,
put_extra_checksum_headers,
ec_block_size_hint_outcome: ec_block_size_hint_resolution.outcome,
completion: Some(completion),
put_request_guard,
bucket,
@@ -2714,6 +2856,52 @@ mod tests {
assert!(limited.next().await.is_none());
}
#[test]
#[serial_test::serial(ec_block_size_hint)]
fn put_ec_block_size_hint_enforces_strict_wire_rules_only_after_rollout() {
let vars = [
(rustfs_config::ENV_EC_BLOCK_SIZE_HINT_ENABLE, Some("true")),
(rustfs_config::ENV_EC_BLOCK_SIZE_HINT_FLEET_CONFIRMED, Some("true")),
];
temp_env::with_vars(vars, || {
let header_name = HeaderName::from_static("x-amz-meta-rustfs-ec-block-size-hint");
let mut duplicate_headers = HeaderMap::new();
duplicate_headers.append(header_name.clone(), HeaderValue::from_static("65536"));
duplicate_headers.append(header_name.clone(), HeaderValue::from_static("262144"));
let duplicate_error = extract_ec_block_size_hint(&duplicate_headers, None).expect_err("duplicate header");
assert_eq!(duplicate_error.code(), &S3ErrorCode::InvalidArgument);
let duplicate_metadata = HashMap::from([
(EC_BLOCK_SIZE_HINT_METADATA_KEY.to_string(), "65536".to_string()),
(EC_BLOCK_SIZE_HINT_METADATA_KEY.to_uppercase(), "262144".to_string()),
]);
let metadata_error = extract_ec_block_size_hint(&HeaderMap::new(), Some(&duplicate_metadata))
.expect_err("case-variant metadata names");
assert_eq!(metadata_error.code(), &S3ErrorCode::InvalidArgument);
let mut malformed_headers = HeaderMap::new();
malformed_headers.insert(header_name, HeaderValue::from_static("65_536"));
let malformed_error = extract_ec_block_size_hint(&malformed_headers, None).expect_err("malformed value");
assert_eq!(malformed_error.code(), &S3ErrorCode::InvalidArgument);
});
temp_env::with_vars(
[
(rustfs_config::ENV_EC_BLOCK_SIZE_HINT_ENABLE, Some("false")),
(rustfs_config::ENV_EC_BLOCK_SIZE_HINT_FLEET_CONFIRMED, Some("false")),
],
|| {
let header_name = HeaderName::from_static("x-amz-meta-rustfs-ec-block-size-hint");
let mut ordinary_metadata = HeaderMap::new();
ordinary_metadata.insert(header_name, HeaderValue::from_static("not-a-number"));
assert_eq!(
extract_ec_block_size_hint(&ordinary_metadata, None).unwrap().as_deref(),
Some("not-a-number")
);
},
);
}
#[test]
fn put_request_user_metadata_cannot_suppress_bucket_default_retention() {
let mut metadata =
+10 -5
View File
@@ -1789,12 +1789,14 @@ pub(crate) mod object_usecase {
object_utils, options, request_context, s3_api, set_disk, sse, storage_class, timeout_wrapper,
};
pub(crate) use crate::storage::storage_api::{
ECStore, GetObjectReader, OldCurrentSize, RFC1123, StorageDeletedObject, StorageObjectInfo,
DEFAULT_EC_BLOCK_SIZE, EC_BLOCK_SIZE_HINT_INTERNAL_SUFFIX, EC_BLOCK_SIZE_HINT_METADATA_KEY,
EC_BLOCK_SIZE_HINT_REASON_HEADER, EC_BLOCK_SIZE_HINT_STATUS_HEADER, EC_BLOCK_SIZE_RESPONSE_HEADER, ECStore,
EcBlockSizeHintOutcome, GetObjectReader, OldCurrentSize, RFC1123, StorageDeletedObject, StorageObjectInfo,
StorageObjectLockDeleteOptions, StorageObjectOptions, StorageObjectToDelete, StoragePutObjReader, check_preconditions,
parse_object_lock_legal_hold, parse_object_lock_retention, parse_part_number_i32_to_usize,
remove_object_lock_metadata_for_copy, strip_managed_encryption_metadata, validate_bucket_exists, validate_new_object_key,
validate_object_key, validate_sse_headers_for_read, validate_sse_headers_for_write, validate_ssec_for_read,
wrap_response_with_cors,
remove_object_lock_metadata_for_copy, resolve_ec_block_size_hint, strip_managed_encryption_metadata,
validate_bucket_exists, validate_new_object_key, validate_object_key, validate_sse_headers_for_read,
validate_sse_headers_for_write, validate_ssec_for_read, wrap_response_with_cors,
};
}
@@ -1827,7 +1829,10 @@ pub(crate) mod multipart_usecase {
access, bucket, compression, data_usage, error, helper, io, object_utils, options, request_context, s3_api, set_disk, sse,
};
pub(crate) use crate::storage::storage_api::{
ECStore, StorageObjectInfo, StorageObjectOptions, StoragePutObjReader, validate_new_object_key,
DEFAULT_EC_BLOCK_SIZE, EC_BLOCK_SIZE_HINT_INTERNAL_SUFFIX, EC_BLOCK_SIZE_HINT_METADATA_KEY,
EC_BLOCK_SIZE_HINT_REASON_HEADER, EC_BLOCK_SIZE_HINT_STATUS_HEADER, EC_BLOCK_SIZE_RESPONSE_HEADER, ECStore,
EcBlockSizeHintOutcome, StorageObjectInfo, StorageObjectOptions, StoragePutObjReader, resolve_ec_block_size_hint,
validate_new_object_key,
};
}
+12 -4
View File
@@ -680,15 +680,23 @@ pub(crate) mod ecstore_object {
#[cfg(test)]
pub(crate) use rustfs_ecstore::api::object::GetObjectBodySource;
pub(crate) use rustfs_ecstore::api::object::{
EncryptionResolutionError, EncryptionResolutionErrorKind, GetObjectBodyCacheHook, GetObjectBodyCacheHookLookup,
GetObjectReadCondition, GetObjectReadMetadata, ObjectEncryptionResolver, ObjectMutationHook,
PrepareSelectObjectSnapshotError, ReadEncryptionMaterial, ReadEncryptionMode, ReadEncryptionRequest,
DEFAULT_EC_BLOCK_SIZE, EC_BLOCK_SIZE_HINT_INTERNAL_SUFFIX, EC_BLOCK_SIZE_HINT_METADATA_KEY,
EC_BLOCK_SIZE_HINT_REASON_HEADER, EC_BLOCK_SIZE_HINT_STATUS_HEADER, EC_BLOCK_SIZE_RESPONSE_HEADER,
EcBlockSizeHintOutcome, EncryptionResolutionError, EncryptionResolutionErrorKind, GetObjectBodyCacheHook,
GetObjectBodyCacheHookLookup, GetObjectReadCondition, GetObjectReadMetadata, ObjectEncryptionResolver,
ObjectMutationHook, PrepareSelectObjectSnapshotError, ReadEncryptionMaterial, ReadEncryptionMode, ReadEncryptionRequest,
SelectObjectSnapshot, WriteCommitGuard, WriteCompletion, get_object_body_cache_plaintext_len,
lookup_get_object_body_cache_hook, register_get_object_body_cache_hook, register_object_mutation_hook,
unregister_get_object_body_cache_hook, unregister_object_mutation_hook, with_get_object_read_condition,
resolve_ec_block_size_hint, unregister_get_object_body_cache_hook, unregister_object_mutation_hook,
with_get_object_read_condition,
};
}
pub(crate) use ecstore_object::{
DEFAULT_EC_BLOCK_SIZE, EC_BLOCK_SIZE_HINT_INTERNAL_SUFFIX, EC_BLOCK_SIZE_HINT_METADATA_KEY, EC_BLOCK_SIZE_HINT_REASON_HEADER,
EC_BLOCK_SIZE_HINT_STATUS_HEADER, EC_BLOCK_SIZE_RESPONSE_HEADER, EcBlockSizeHintOutcome, resolve_ec_block_size_hint,
};
#[cfg(all(test, feature = "rio-v2"))]
pub(crate) mod ecstore_test_support {
pub(crate) use rustfs_ecstore::api::bitrot::create_bitrot_reader;
+381
View File
@@ -0,0 +1,381 @@
#!/usr/bin/env python3
"""Verify RustFS dynamic EC block hints against a disposable S3 endpoint.
Requires boto3 and AWS_ACCESS_KEY_ID/AWS_SECRET_ACCESS_KEY. See
docs/testing/dynamic-block-size.md for rollout and restart probes.
"""
import argparse
import hashlib
import json
import os
import re
import uuid
import boto3
from botocore.config import Config
from botocore.exceptions import ClientError
MIB = 1024 * 1024
CANDIDATES = (64 * 1024, 256 * 1024, MIB, 4 * MIB)
HINT = "rustfs-ec-block-size-hint"
ACTUAL = "x-rustfs-effective-ec-block-size"
STATUS = "x-rustfs-layout-hint-status"
REASON = "x-rustfs-layout-hint-reason"
SIZE = 4 * MIB + 123
PART_SIZES = (5 * MIB + 123, MIB + 77)
def require(condition, message):
if not condition:
raise AssertionError(message)
def headers(response):
return response["ResponseMetadata"]["HTTPHeaders"]
def check_layout(response, expected, label, status=None, reason=None):
values = headers(response)
require(
values.get(ACTUAL) == str(expected),
f"{label}: actual B {values.get(ACTUAL)!r}, expected {expected}",
)
require(
values.get(STATUS) == status,
f"{label}: hint status {values.get(STATUS)!r}, expected {status!r}",
)
require(
values.get(REASON) == reason,
f"{label}: hint reason {values.get(REASON)!r}, expected {reason!r}",
)
require(
not any("internal-ec-read-quantum" in key for key in values),
f"{label}: internal q leaked over S3",
)
def payload(key, size):
return hashlib.shake_256(key.encode()).digest(size)
def sha256_argument(value):
if re.fullmatch(r"[0-9a-fA-F]{64}", value) is None:
raise argparse.ArgumentTypeError(
"SHA-256 must contain exactly 64 hexadecimal characters"
)
return value.lower()
def read_body(response):
try:
return response["Body"].read()
finally:
response["Body"].close()
def verify_object(client, bucket, key, expected, block_size, part_boundary=None):
head = client.head_object(Bucket=bucket, Key=key)
check_layout(head, block_size, f"HEAD {key}")
require(head["ContentLength"] == len(expected), f"HEAD {key}: size mismatch")
full = client.get_object(Bucket=bucket, Key=key)
check_layout(full, block_size, f"GET {key}")
require(read_body(full) == expected, f"GET {key}: payload mismatch")
ranges = []
if expected:
ranges.extend(
[
(0, min(122, len(expected) - 1)),
(max(0, len(expected) - 123), len(expected) - 1),
]
)
if len(expected) > block_size:
ranges.append((block_size - 37, min(block_size + 85, len(expected) - 1)))
if part_boundary is not None:
ranges.append((part_boundary - 37, min(part_boundary + 85, len(expected) - 1)))
for start, end in ranges:
response = client.get_object(
Bucket=bucket, Key=key, Range=f"bytes={start}-{end}"
)
check_layout(response, block_size, f"Range {key} {start}-{end}")
require(
response["ResponseMetadata"]["HTTPStatusCode"] == 206,
f"Range {key}: expected HTTP 206",
)
require(
response["ContentRange"] == f"bytes {start}-{end}/{len(expected)}",
f"Range {key}: content-range mismatch",
)
require(
read_body(response) == expected[start : end + 1],
f"Range {key}: payload mismatch",
)
print(
json.dumps(
{
"key": key,
"actual_b": block_size,
"size": len(expected),
"range_checks": len(ranges),
"result": "passed",
}
)
)
def ensure_absent(client, bucket, key):
try:
client.head_object(Bucket=bucket, Key=key)
except ClientError as error:
if error.response["ResponseMetadata"]["HTTPStatusCode"] == 404:
return
raise
raise RuntimeError(f"refusing to overwrite existing object {bucket}/{key}")
def put_multipart(client, bucket, key, block_size, status, reason):
initiated = client.create_multipart_upload(
Bucket=bucket, Key=key, Metadata={HINT: str(block_size)}
)
upload_id = initiated["UploadId"]
parts = []
committed = False
try:
check_layout(
initiated,
block_size if status == "applied" else MIB,
f"MPU initiate {key}",
status,
reason,
)
for number, size in enumerate(PART_SIZES, 1):
response = client.upload_part(
Bucket=bucket,
Key=key,
UploadId=upload_id,
PartNumber=number,
Body=payload(f"{key}:{number}", size),
)
parts.append({"PartNumber": number, "ETag": response["ETag"]})
completed = client.complete_multipart_upload(
Bucket=bucket, Key=key, UploadId=upload_id, MultipartUpload={"Parts": parts}
)
committed = True
check_layout(
completed, block_size if status == "applied" else MIB, f"MPU complete {key}"
)
except BaseException:
if not committed:
client.abort_multipart_upload(Bucket=bucket, Key=key, UploadId=upload_id)
raise
def cases(policy):
status = "applied" if policy == "applied" else "ignored"
reason = None if status == "applied" else policy
for block_size in CANDIDATES:
actual = block_size if status == "applied" else MIB
yield f"put-{block_size}", block_size, actual, status, reason, False
yield f"mpu-{block_size}", block_size, actual, status, reason, True
yield "put-default", None, MIB, None, None, False
yield (
"put-unsupported",
131072,
MIB,
"ignored",
"unsupported-size" if policy == "applied" else policy,
False,
)
def verify_external(client, args):
require(
args.external_block_size is not None,
"--external-prefix requires --external-block-size",
)
found = 0
for page in client.get_paginator("list_objects_v2").paginate(
Bucket=args.bucket, Prefix=args.external_prefix
):
for item in page.get("Contents", []):
key = item["Key"]
response = client.get_object(Bucket=args.bucket, Key=key)
expected = read_body(response)
require(
len(expected) == item["Size"],
f"external {key}: full GET length mismatch",
)
if args.external_size is not None:
require(
len(expected) == args.external_size,
f"external {key}: intended payload size mismatch",
)
if args.external_sha256 is not None:
require(
hashlib.sha256(expected).hexdigest() == args.external_sha256,
f"external {key}: intended payload checksum mismatch",
)
verify_object(client, args.bucket, key, expected, args.external_block_size)
found += 1
require(found > 0, f"no external objects found under {args.external_prefix!r}")
print(json.dumps({"external_objects": found, "result": "passed"}))
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--endpoint", required=True)
parser.add_argument("--bucket", help="Disposable bucket; generated by default")
parser.add_argument("--prefix", default="dynamic-block-e2e/")
parser.add_argument(
"--write-policy",
choices=("applied", "disabled", "fleet-unconfirmed"),
default="applied",
help="Policy during object creation, including when verifying after restart",
)
parser.add_argument(
"--keep",
action="store_true",
help="Keep this script's bucket and objects for restart verification",
)
parser.add_argument(
"--verify-only",
action="store_true",
help="Read previously written objects; never delete them",
)
parser.add_argument(
"--external-prefix",
help="Only verify objects written by a client such as BrewFS",
)
parser.add_argument("--external-block-size", type=int, choices=CANDIDATES)
parser.add_argument(
"--external-size",
type=int,
help="Expected size in bytes of every external object",
)
parser.add_argument(
"--external-sha256",
type=sha256_argument,
help="Expected SHA-256 of every external object",
)
args = parser.parse_args()
if not os.environ.get("AWS_ACCESS_KEY_ID") or not os.environ.get(
"AWS_SECRET_ACCESS_KEY"
):
parser.error(
"set AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY for the disposable endpoint"
)
if (args.verify_only or args.external_prefix is not None) and not args.bucket:
parser.error("--verify-only and --external-prefix require --bucket")
if args.external_prefix is not None and not args.external_prefix:
parser.error("--external-prefix must identify a nonempty task prefix")
if args.external_size is not None and args.external_size < 0:
parser.error("--external-size must be nonnegative")
if (
args.external_size is not None or args.external_sha256 is not None
) and args.external_prefix is None:
parser.error("--external-size and --external-sha256 require --external-prefix")
args.bucket = args.bucket or "dynamic-block-" + uuid.uuid4().hex[:20]
client = boto3.client(
"s3",
endpoint_url=args.endpoint,
region_name="us-east-1",
config=Config(
s3={"addressing_style": "path"},
signature_version="s3v4",
proxies={},
connect_timeout=5,
read_timeout=120,
retries={"total_max_attempts": 1},
request_checksum_calculation="when_required",
response_checksum_validation="when_required",
),
)
created = []
created_bucket = False
try:
if args.external_prefix is not None:
verify_external(client, args)
return
if not args.verify_only:
try:
client.head_bucket(Bucket=args.bucket)
except ClientError as error:
require(
error.response["ResponseMetadata"]["HTTPStatusCode"] == 404,
"test bucket lookup failed",
)
else:
raise RuntimeError(f"refusing to reuse existing bucket {args.bucket}")
client.create_bucket(Bucket=args.bucket)
created_bucket = True
for name, hint, actual, status, reason, multipart in cases(args.write_policy):
key = args.prefix + name
if multipart:
expected = b"".join(
payload(f"{key}:{number}", size)
for number, size in enumerate(PART_SIZES, 1)
)
else:
expected = payload(key, SIZE)
if not args.verify_only:
ensure_absent(client, args.bucket, key)
created.append(key)
if multipart:
put_multipart(client, args.bucket, key, hint, status, reason)
else:
response = client.put_object(
Bucket=args.bucket,
Key=key,
Body=expected,
Metadata={} if hint is None else {HINT: str(hint)},
)
check_layout(response, actual, f"PUT {key}", status, reason)
verify_object(
client,
args.bucket,
key,
expected,
actual,
PART_SIZES[0] if multipart else None,
)
if not args.verify_only and args.write_policy == "applied":
key = args.prefix + "invalid-hint"
ensure_absent(client, args.bucket, key)
try:
client.put_object(
Bucket=args.bucket,
Key=key,
Body=b"invalid-hint-probe",
Metadata={HINT: "+65536"},
)
except ClientError as error:
require(
error.response["Error"]["Code"] == "InvalidArgument",
"malformed hint must fail with InvalidArgument",
)
else:
created.append(key)
raise AssertionError("malformed hint unexpectedly succeeded")
print(
json.dumps(
{
"bucket": args.bucket,
"prefix": args.prefix,
"write_policy": args.write_policy,
"verify_only": args.verify_only,
"objects": 10,
"result": "passed",
}
)
)
finally:
if not args.keep and not args.verify_only and args.external_prefix is None:
for key in created:
client.delete_object(Bucket=args.bucket, Key=key)
if created_bucket:
client.delete_bucket(Bucket=args.bucket)
client.close()
if __name__ == "__main__":
main()