* refactor(sse): decouple encryption from ecstore
* feat(kms): enhance KMS service manager with runtime state and persistence support
* feat(kms): add local key export functionality for SSE-S3 migration tests
* fix(kms): keep local key export narrowly scoped
* fix(sse): validate copy source customer algorithm
---------
Co-authored-by: Zhengchao An <anzhengchao@gmail.com>
* fix(filemeta): add state-aware file info validation
* fix(filemeta): validate shard arithmetic and delete paths
* fix(ecstore): add fallible erasure construction
* fix(ecstore): resolve storage parity per pool
* fix(storage): report heterogeneous erasure layouts
* fix(admin): publish prepared storage config atomically
* fix(storage): harden per-pool parity boundaries
* fix(storage): address pre-PR validation findings
* test(ci): fix strict-topology validation fixtures
* fix(heal): preserve delete markers during repair
* refactor(filemeta): drop unused ValidatedFileInfo witness
ValidatedFileInfo wrapped an unread `_file_info` reference alongside an `Option<ValidatedErasureLayout>`, but only the layout was ever consumed. Return the layout directly from `FileInfo::validate` so the sole production consumer (`LocalDisk::check_parts`) and the two unit tests read it without the extra witness type and lifetime.
No behavior change.
* fix(filemeta): keep compressed and MinIO-migrated tiered objects readable
The new decode-path validation rejected several legitimate on-disk shapes that older RustFS and MinIO-migrated data carry, turning readable objects into FileCorrupt:
- Compressed objects written with an unknown upload size persist a negative per-part actual_size (the documented "unknown size" sentinel that ObjectInfo::get_actual_size already tolerates). validate_collection_contents rejected it via usize::try_from; now a negative actual_size skips shard validation and only real, non-negative sizes are checked.
- MinIO-migrated objects transitioned to a versioned remote tier store the tier version id as a UUID string, not 16 raw bytes. MetaObject::into_fileinfo returned FileCorrupt (main tolerated it as None), making all versions of the object unreadable; MetaDeleteMarker free-version records took a Some(nil) sentinel path with the same effect, which also breaks free-version expiry (remote-tier leak). Both now decode through a shared transitioned_version_id_from_meta_sys helper: 16 raw bytes or a UUID string are accepted, anything else is tolerated as None instead of failing the read.
Regression tests updated to assert the readable/compat behavior, with new tests covering MinIO string-form recovery.
* fix(scanner): build the delete-marker test fixture without erasure geometry
get_size_counts_delete_markers_separately_from_versions built its delete marker with `FileInfo::new(object, 1, 1)`, which attaches erasure geometry (data=1/parity=1/distribution). This PR classifies versions by shape via `is_storage_delete_marker()` (no geometry) rather than the raw `deleted` flag, so a geometry-bearing "delete marker" is correctly serialized as a purge-pending payload Object and counted as a version — CI saw summary.versions=3, expected 2.
Real delete markers carry no erasure geometry (delete paths build them as `FileInfo { deleted: true, ..Default::default() }`), so construct the fixture the same way. It then classifies as a storage delete marker and the counts (versions=2, delete_markers=1) hold. This keeps the PR's more-correct classification, which prevents a purge-pending object's geometry from being dropped when serialized as a bare delete marker.
* docs(changelog): note per-pool parity fix and storage-class startup upgrade caveat
Records the #4801 per-pool erasure parity fix under Fixed, and documents the upgrade behavior where a persisted storage class that a small or heterogeneous pool cannot satisfy now fails startup — with the RUSTFS_STORAGE_CLASS_STANDARD recovery steps. Docs-only; covers R4 from the on-disk compatibility audit.
* fix(heal): report parity from erasure geometry, not is_valid()
heal_object set HealResultItem.parity_blocks via `if lfi.is_valid()`, which was missed by the migration of the other quorum/metadata predicates. With the new `is_valid()` semantics (full payload validation; delete markers now return false), a delete marker or a geometry-bearing version with a benign collection quirk would misreport parity as the pool default instead of its own. Use `has_valid_erasure_geometry()` — the narrow "does this carry erasure geometry" predicate the rest of the migration uses — so reporting matches the object's actual layout. Reporting-only; no data-path change.
* fix(filemeta): do not silently serialize a non-canonical deleted FileInfo as an Object
`From<FileInfo> for FileMetaVersion` classifies by `is_storage_delete_marker()` (shape), which correctly routes canonical delete markers to Delete and purge-pending payloads (deleted=true with real erasure geometry) to Object. But a `deleted` FileInfo that is neither a canonical marker nor a valid erasure payload would silently serialize as a zero-geometry MetaObject that later fails `validate_for_metadata_read`. Write paths validate first (`validate_for_erasure_write` / `validate_for_metadata_read`), so this is a caller bug; `From` is infallible, so surface it with a structured `warn!` on the malformed branch instead of writing corrupt metadata silently. Legitimate purge-pending objects (valid geometry) are unaffected — the guard only fires for `deleted && !has_valid_erasure_geometry()`.
* test(filemeta): assert real historical xl.meta versions pass metadata-read validation
Empirical companion to the code-reasoned decode-tolerance invariants (docs/architecture/erasure-coding.md §11) and the rolling-upgrade / MinIO-migration compatibility concern: the tightened `validate_for_metadata_read` runs on every local disk read and peer-RPC-decoded FileInfo, so it must accept every version of real historically-written xl.meta, never reject it as FileCorrupt.
Loads five real fixtures — MinIO small-inline, MinIO versioned (two object versions + a delete marker), MinIO large multipart, a legacy V1 (xl.json-derived) object, and a legacy meta_ver 2 object — decodes every version with parts materialized, and asserts validate_for_metadata_read() is Ok for each. Reverting the tolerant handling (delete-marker shape, legacy per-part checksums, string/short transitioned-versionID, negative actual_size) turns this red.
* fix(ci): remove duplicate storage test re-exports
---------
Co-authored-by: overtrue <anzhengchao@gmail.com>
fix(ecstore): validate erasure distribution values to avoid shuffle index panic (backlog#949)
The element values of `erasure.distribution` read from `xl.meta` were never
range-checked. `FileInfo::is_valid()` and `MetaObjectV1::valid()` only verified
`distribution.len()` and the `erasure.index` bound, not that each distribution
value is a valid 1-based slot in `[1, N]`. The metadata shuffle helpers then use
these values directly as `distribution[k] - 1` indices, so a corrupt or
adversarial `xl.meta` carrying a `0` (usize underflow) or a value greater than N
(out-of-bounds) triggers a panic in the shuffle path, turning bad-disk metadata
that erasure coding is meant to tolerate into a request/task crash.
Fix, two layers:
- Validate distribution values at metadata acceptance: `is_valid_distribution`
now requires the distribution to be a permutation of `1..=N` (correct length,
every value in range, no duplicates). `FileInfo::is_valid()` and
`MetaObjectV1::valid()` use it, so `find_file_info_in_quorum` rejects corrupt
metadata and it surfaces as a clean `ErasureReadQuorum` error instead of an
index path.
- Defensive indexing in the shuffle helpers (`shuffle_disks_and_parts_metadata`,
`_by_index`, `_by_index_owned`, `shuffle_parts_metadata`, `shuffle_disks`,
`shuffle_check_parts`): out-of-range distribution values are skipped via
`checked_sub(1)` + bounds-checked slot access instead of a bare `idx - 1`
index, matching the existing pattern in
`collect_inline_data_shard_fileinfos_by_index`.
Regression tests: `is_valid_distribution`/`is_valid`/`valid` reject distributions
containing `0`, values greater than N, duplicates, and wrong length while
accepting valid permutations; the shuffle helpers no longer panic on corrupt
distributions and preserve output length.
Refs: https://github.com/rustfs/backlog/issues/949
* perf(ecstore): replace linear merge scan with k-way heap and drop clone-to-parse (backlog#874 backlog#875)
merge_entry_channels advanced the k-way merge with a linear scan over all
channel heads (O(entries x channels)) and allocated two fresh Strings per
pairwise comparison via path::clean. Every step also cloned MetaCacheEntry
values, including entry.clone().xl_meta() clone-to-parse calls.
- Introduce MergeHead with a cached cleaned name (allocated only when the
raw name is not already clean) and drive the merge with a BinaryHeap of
boxed heads: O(log channels) per entry, allocation-free comparisons.
- Move entries through the merge instead of cloning; the winner is sent
without an intermediate copy.
- Remove the dead merge_file_meta_versions block: it only ran for
prefix-dir groups whose entries have empty metadata, so xl_meta() always
failed; cross-drive version merging happens in the resolve path.
- Keep legacy same-name semantics (dir groups collapse, objects shadow
prefix dirs, later object candidate wins) and add regression tests for
interleaved ordering, dir/object precedence, uncleaned-name grouping,
and prefix-dir collapse.
Co-Authored-By: heihutu <heihutu@gmail.com>
* fix(ecstore): honor ascending versions_sort in ListObjects walk (#4348)
* fix(ecstore): honor ascending versions_sort in walk and document ordering invariant (backlog#876)
The walk loop carried a bare `//TODO: SORT` inside the
`WalkVersionsSortOrder::Ascending` branch, so the requested ascending
order was silently ignored and versions streamed newest-first (the raw
FileMeta order). WalkOptions defaults to Ascending, so every default
walker -- notably replication resync, which replays versions and needs
oldest-first to preserve the version-stack order -- received the exact
opposite of the contract.
FileMeta maintains versions newest-first (sort_by_mod_time is
descending) and into_file_info_versions preserves that order, so
ascending emission is the exact reverse of file_info_versions output.
Reverse in place when ascending is requested and add a regression test
locking the newest-first invariant plus the reversal contract.
Key-ordering audit result (no gap found): per-disk walkers emit sorted
streams, merge_entry_channels performs an ordered k-way merge, and
gather_results only filters by marker/limit, so ListObjects key order is
guaranteed upstream and needs no post-sort.
Co-Authored-By: heihutu <heihutu@gmail.com>
* perf(ecstore): enable GET metadata early-stop by default (#4349)
* perf(ecstore): enable GET metadata early-stop by default with env opt-out (backlog#872)
The metadata early-stop fanout (read_all_fileinfo_early_stop) has been
implemented and instrumented for a while but stayed behind an opt-in
flag, so default GETs always waited for every disk to answer the
metadata read even after quorum agreement was reached.
Flip RUSTFS_GET_METADATA_EARLY_STOP_ENABLE to default-on. The gate stays
conservative: should_allow_metadata_early_stop only admits metadata-only
reads (read_data=false) without version_id, healing, or free-version
requirements, everything else falls back to the full-wait fanout, and
setting the env var to false restores the old behavior entirely. The
version-aware gate (RUSTFS_GET_METADATA_VERSION_EARLY_STOP_ENABLE)
remains opt-in because versioned reads carry a higher stale-selection
risk profile.
Also replace the stale "optimize concurrency" TODO in
get_object_fileinfo with a pointer to the early-stop implementation and
add regression tests for the new default plus the explicit opt-out path.
Co-Authored-By: heihutu <heihutu@gmail.com>
* perf(ecstore): lazily construct codec streaming multipart readers (#4350)
* perf(ecstore): lazily construct codec streaming multipart part readers (backlog#871)
get_object_decode_reader_with_fileinfo opened shard readers for every
part of a multipart object before returning the streaming reader, so
TTFB paid for parts x disks file opens up front and an early client
disconnect wasted the setup work for every unread part.
Replace the eager loop with LazyMultipartCodecStreamingReader: the first
part is still built eagerly so the dominant fallback conditions (missing
shards / read quorum) are detected before any byte is streamed and the
whole request can fall back to the legacy duplex path exactly as before.
Each subsequent part is built on demand -- when the previous part hits
EOF -- via a spawned task handle owned by the reader; dropping the
reader aborts an in-flight build so disconnects stop all further IO.
If a later part hits a fallback condition mid-stream (a shard vanished
after the request started), the reader surfaces an explicit read error
with a pipeline-failure metric instead of silently degrading; the
client's retry then detects the condition on the eager first-part setup
and takes the legacy path cleanly.
Adds unit tests for in-order streaming across lazy boundaries, deferred
construction (no build when the client stops within part 1), and the
mid-stream fallback error path.
Co-Authored-By: heihutu <heihutu@gmail.com>
* perf(ecstore): prefetch next multipart part reader setup during decode (#4351)
* perf(ecstore): prefetch next multipart part reader setup during decode (backlog#870)
get_object_with_fileinfo processed multipart parts strictly serially:
the next part's bitrot reader setup (file opens + read-quorum wait
across all disks) only started after the current part finished
decoding, so large multipart reads paid full setup latency between
every part.
Overlap the two stages with a depth-one pipeline: right after the
current part's readers are obtained, the next part's setup is spawned
(shared inputs behind Arc) and joined when the loop reaches that part.
The shared setup_multipart_part_readers helper keeps stage-duration
metrics semantics identical for both paths; a failed or stale prefetch
falls back to the synchronous setup, and the PrefetchedReaderSetup
guard aborts the in-flight task on error returns, early breaks, or
caller drop so disconnects stop background disk IO.
Gate: RUSTFS_GET_MULTIPART_READER_SETUP_PREFETCH (default on, env
opt-out). Adds a three-part end-to-end read test covering the prefetch
hit path and cross-part content ordering.
Co-Authored-By: heihutu <heihutu@gmail.com>
* perf(ecstore): move FileInfo through GET shuffle instead of cloning (#4352)
perf(ecstore): move FileInfo entries through the GET shuffle instead of cloning (backlog#873)
shuffle_disks_and_parts_metadata_by_index deep-cloned every valid
FileInfo (parts, erasure info, metadata map) once per disk on each GET.
Add an ownership-taking variant that runs the same by-index consistency
check as a read-only first pass and then moves entries into their
shuffled slots with mem::take, and switch get_object_with_fileinfo to
it -- that call site already owned the parts metadata vector. Disk
handles are Arc clones and stay cheap.
Scope notes from the backlog#873 audit:
- get_object_fileinfo's disks.clone() stays: DiskStore is Arc<Disk>, so
the clone is per-slot refcounting and correctly avoids holding the
RwLock read guard across the metadata fanout awaits.
- get_object_decode_reader_with_fileinfo keeps the borrowing shuffle:
its caller must retain files/disks for the legacy fallback path, so an
owned variant would just shift the same clone upstream.
- The metadata-cache hit path still clones parts_metadata; sharing the
cached entry via Arc changes the read-path return types and is left
as a follow-up.
Equivalence tests cover both the by-index placement and the mod-time
fallback against the borrowing variant.
Co-authored-by: heihutu <heihutu@gmail.com>
---------
Co-authored-by: heihutu <heihutu@gmail.com>
---------
Co-authored-by: heihutu <heihutu@gmail.com>
---------
Co-authored-by: heihutu <heihutu@gmail.com>
---------
Co-authored-by: heihutu <heihutu@gmail.com>
* fix(ecstore): gate merge emission on cleaned key and clear clippy redundant_clone
Address review + CI findings on the ListObjects/GET optimization PR:
- merge_entry_channels gated emission on the raw entry name while the heap
orders by the cleaned key, so entries whose cleaned order and raw byte order
disagree (e.g. redundant slashes) could be dropped. Gate on the same cleaned
sort key the heap uses; add a regression test (`a//c` after `a/b`).
- Drop three redundant `.clone()` calls in test code flagged by
clippy::redundant_clone (owned-shuffle equivalence tests and the walk
ascending-versions contract test) that failed the CI clippy gate.
- Document the known mid-stream fallback limitation of the opt-in multipart
codec streaming reader (default off) and mark the in-place per-part legacy
degradation as a follow-up.
Co-Authored-By: heihutu <heihutu@gmail.com>
* fix(ecstore): force full metadata fanout for object tagging writes (backlog#872)
put_object_tags reads the object fileinfo with read_data=false and then
writes the updated tags to the online-disk set that read returned. With
metadata early-stop enabled by default, that read now returns as soon as
read quorum is reached, so the online-disk set is only a read-quorum
subset. Writing tags to that subset fails write quorum -> ErasureWriteQuorum
-> S3 SlowDown, which is exactly the s3-tests tagging failures
(PutObjectTagging/DeleteObjectTagging, reached max retries).
Thread a caller-controlled `allow_early_stop` gate through
read_all_fileinfo_observed/_inner and add get_object_fileinfo_gated;
put_object_tags calls it with allow_early_stop=false so the metadata read
does the full quorum fanout and returns the complete online-disk set as
the write target. Pure-read callers (GET/HEAD/tag read) keep the
early-stop fast path unchanged.
Extract metadata_early_stop_permitted() as the single gate and add a unit
test locking the invariant: caller opt-out (and observe=false, and data
reads) never early-stop even with the env flags on.
Co-Authored-By: heihutu <heihutu@gmail.com>
---------
Co-authored-by: heihutu <heihutu@gmail.com>
唐小鸭