MRF delete replay reconstructed the delete with `..Default::default()`, leaving
`replication_state = None`. `replicate_delete` derives its target set purely
from `replication_state.replicate_decision_str`, so an empty state produced an
empty decision -> zero targets: the replayed delete contacted no remote at all,
a silent no-op that left replicas permanently diverged after a restart.
The MRF entry doesn't persist the decision and the source object is already
gone, so re-derive it from the live bucket config at replay time via
`check_replicate_delete` — mirroring the object heal path
(`get_heal_replicate_object_info`) — and set it on the reconstructed delete's
`replication_state`. This is a contained fix with no change to the on-disk MRF
format.
Refs backlog#799 (B9).
Note: the source delete-marker mtime is still not persisted in the MRF entry,
so a replayed delete marker is stamped with the replay time on the target. That
is a separate, minor consistency nuance (the delete now propagates correctly)
and can be addressed by extending the MRF entry format in a follow-up.
The MRF persister accumulated overflow entries in `pending`, flushed them with
`flush_mrf_to_disk`, and cleared `pending` on success. But `flush_mrf_to_disk`
*overwrites* the whole MRF file with exactly the entries passed. After flushing
batch A (file = A) and clearing, the next flush wrote batch B and thereby
overwrote the file to contain only B — and the MRF file is only replayed (and
cleared) at startup, never during the run, so batch A's entries were silently
lost. A crash after the B flush lost all of batch A's pending replications.
Keep `pending` cumulative (the file must hold the full set of overflow entries
for the run) and rewrite the whole set on each flush instead of clearing after
success:
- flush eagerly once 1 000 *new* entries accumulate since the last write
(measured against the flushed length, so a large backlog isn't rewritten on
every add), and on the 10s tick when dirty;
- bound the in-memory/on-disk backlog with `MRF_PENDING_CAP` (200 000) and log
once when the cap is hit rather than growing without limit.
Refs backlog#799 (B10).
The resync result verification HEADed the target after replicating and counted
the outcome with inverted error handling:
- for a delete marker, ANY HEAD error (timeout, 5xx, auth, malformed) was
counted as replicated (success);
- for a versioned object against an AWS-style target, HEAD was sent with the
RustFS UUID versionId, which AWS rejects with 400, so a well-replicated
object was counted as failed.
Classify the error before counting:
- delete marker: only a definitive 404/NoSuchKey or 405/MethodNotAllowed
confirms the marker propagated (`is_retryable_delete_replication_head_error`
== false); any retryable/ambiguous error now counts as failed;
- versioned object with a version-id-format rejection: re-verify via
`head_object_fallback` (versionId-less HEAD) before deciding — present ->
replicated, absent/error -> failed;
- all other errors: failed, as before.
Reuses the existing, unit-tested classifier helpers. Verified against the
existing resyncer suite (24 tests).
Refs backlog#799 (B13).
During resync against an AWS-style target that rejects RustFS UUID versionIds,
the code retries HEAD without a versionId and compares ETags. On a match it set
`replication_action = None` ("already in sync, nothing to copy") but did not
return, so control fell through to `if replication_action != ReplicationAction::All`
— a branch meant only for the unsupported metadata-only case — and stamped the
object FAILED with "metadata-only replication is not implemented". The target
already held an identical object, yet the source recorded FAILED forever, so
AWS-style targets never converged and the MRF kept re-queuing.
Handle `ReplicationAction::None` explicitly before that branch: record it as
Completed (with the resync timestamp/`replication_resynced` bookkeeping for
ExistingObject + reset_id, mirroring the HEAD-success None path) and return.
Only `ReplicationAction::Metadata` now takes the metadata-unsupported failure
branch; `All` still proceeds to the copy. This path is the only way `None`
reaches that point (the HEAD-success None case already returns earlier).
Refs backlog#799 (B11).
`MultiWriter` recorded per-writer failures only in a private `errs` vector and
nulled the failed writer locally, but `put_object` never used that to prune the
disk set: `rename_data` ran over the full `shuffle_disks`, so a disk that took a
short/failed write still had its truncated shard renamed into place and counted
as an online disk. The object then claimed N good shards while one was
short/corrupt, so a single later disk failure could drop it below reconstructable
quorum — silent data loss.
- `write_shard`: null the writer on a generic write error too (not only on
`ShortWrite`), so a failed writer is uniformly represented as `None` — matching
`shutdown_writer`, which already does this.
- `put_object`: after encode, drop every disk whose writer failed
(`drop_failed_writer_disks`) from `shuffle_disks` before `rename_data`, and
re-check write quorum over the survivors. `rename_data` already re-checks
quorum and rolls back if too few disks remain, so excluded disks are neither
committed nor counted (MinIO sets failed writers to nil before `renameData`).
Adds unit tests for the exclusion/quorum accounting.
Refs backlog#799 (B3).
fix(heal): don't mark set healed / discard resume state when objects failed (backlog#855)
`execute_heal_with_resume` unconditionally called `mark_completed()` and
returned `Ok(())` after the bucket loop, even when objects failed. The caller
then treats `Ok` as success and cleans up the resume + checkpoint state, so a
heal run with per-object failures was reported as a clean completion and its
state (including the failed set) was destroyed with no retry.
Finalize based on the failure count instead:
- failed_objects == 0 -> mark completed (unchanged);
- failed_objects > 0 with retry budget left -> `schedule_retry()` (bump the
bounded retry counter + reset per-pass progress for a full re-scan) and
return Err, so `heal_erasure_set` preserves the resume/checkpoint state and
the caller keeps the healing markers for the next run;
- failed_objects > 0 with retries exhausted -> drop the resume state (no
zombie task) but still return Err so the markers survive for a later heal
cycle / the background scanner. Never silently claim a clean completion.
The failure identities are not persisted across pages (the per-page sets are
pruned in `complete_page`), so a retry is a bounded full re-scan; healed
objects are skipped quickly on the normal-scan pass.
Adds `ResumeState::reset_for_retry`, `ResumeManager::schedule_retry`,
`ResumeCheckpoint::reset_for_retry`, `CheckpointManager::reset_for_retry`, and
regression tests. This activates the failure path the caller already documents
at task.rs ("Keep the markers on failure ... the next run re-marks and
eventually clears them"), which was previously dead because heal never
returned Err on object failures.
Refs backlog#799 (B6).
PR #4220 added a purge for orphan empty-directory trees (folder keys with
no xl.meta) on the delete path, but the guard only accepted
object-not-found. Over the real HTTP DELETE path the guard is never
reached: `del_opts` pins `version_id = Uuid::nil()` for directory keys, so
the missing dir object fails the specific-version lookup with
version-not-found (FileVersionNotFound), not object-not-found. The guard
short-circuits, the store returns the error, and the API layer turns it
into a fake 204 — the ghost folder survives, exactly the #4189 symptom.
The existing unit tests passed because they call
`purge_orphan_dir_object` directly, bypassing the nil-version lookup.
Accept both misses in the guard (extracted as
`should_purge_orphan_dir_on_missing`) so the folder delete actually takes
effect. Non-directory keys and non-miss errors (e.g. quorum failures) are
unaffected; the cross-disk data-safety refusal in
`purge_orphan_dir_object` is unchanged.
Verified end-to-end against a running 4-disk erasure server: DELETE of a
planted orphan `ghost/` tree now purges it on all drives and clears the
listing, a real object under the same prefix is untouched, and a prefix
holding real data on any drive is refused (all drives preserved).
Adds predicate regression tests covering version-not-found /
object-not-found on directory keys, and the negative cases.
fix(heal): classify heal_object errors by type, not "not found" substring (backlog#856)
The heal page loop classified heal_object errors with a substring test
(`message.contains("not found")`). `StorageError::DiskNotFound` renders as
"disk not found", so an offline drive during a deep-scan heal matched the
"object absent" branch: the object was returned as `Ok(false)`, recorded into
the checkpoint's processed set, counted as a success, and permanently skipped
on resume — the object silently never got healed.
Replace the substring test with a typed classifier over the wrapped
`StorageError`:
- genuine object/version absence (FileNotFound / FileVersionNotFound /
ObjectNotFound / VersionNotFound) -> Absent (treated as handled);
- transient infrastructure conditions (quorum errors, DiskNotFound,
VolumeNotFound, SlowDown, OperationCanceled) -> TransientSkip, so the object
is retried on a later pass instead of recorded as processed;
- everything else -> Failed (recorded as failed).
Deliberately does NOT use `StorageError::is_not_found()`, which lumps
DiskNotFound/VolumeNotFound with object absence — the exact conflation that
caused the bug. Applied to both the deep-scan and normal-scan heal_object call
sites. Adds regression tests for the classifier.
Refs backlog#799 (B7).
P6 of the SetDisks God-Object split (tracking backlog#815, issue backlog#821;
depends on P5 #4288). Relocate the core object read/write hot-path contract
impls out of the set_disk/mod.rs God-Object into their own module home:
- impl ObjectIO for SetDisks (~1,010 lines) -> set_disk/ops/object.rs
- impl ObjectOperations for SetDisks (~1,137 lines) -> set_disk/ops/object.rs
- Registered pub(crate) mod object; under set_disk/ops.
Pure move — zero logic change. Both contracts stay implemented for SetDisks, so
their EcstoreObjectIO / EcstoreObjectOperations associated-type bounds are
unchanged and the contract-compat tests still guard them. Method bodies are
moved verbatim: a whitespace-insensitive token-stream diff of the two impl
blocks (with their #[async_trait::async_trait] attributes) against the pre-move
mod.rs source is byte-identical (5,754 tokens each) — NO visibility widening was
required, because the impls reach SetDisks helpers and the P5 io_primitives
through inherent self./Self:: calls that resolve across modules unchanged. The
two inherent impl SetDisks blocks that sat between the trait impls (lock batch
helpers) remain in mod.rs, verified present exactly once and uncorrupted.
The issue's borrow/Arc-clone-avoidance optimization is intentionally deferred:
it is a perf-sensitive change requiring the #738 benchmark and would risk the
'byte-level behavior unchanged' acceptance; this PR delivers the relocation.
Verification:
- cargo check / clippy -D warnings -p rustfs-ecstore --all-targets: clean
- cargo test -p rustfs-ecstore --lib: 1841 passed, 0 failed
- all five arch guard scripts: pass
- token-stream diff of moved impls vs original: identical (mod.rs diff is a pure
relocation; ObjectIO/ObjectOperations each defined exactly once post-move)
fix(ecstore): lockstep stripe read to stop EC GET desync truncation
On the reconstruction-verifying GET path, ParallelReader::read used a
data-first schedule: it read only `data_shards` readers per stripe and
pulled in a parity reader as a substitute on demand. Because the shard
readers are streaming (advanced only by being read, no seek), a parity
reader first used mid-object was still positioned at its stream start
(block 0) and returned an earlier stripe than the surviving data shards.
Every shard passed its own bitrot hash, yet the set was mutually
misaligned, so decode_data_with_reconstruction_verification correctly
rejected it with "inconsistent read source shards" and the large-object
GET truncated mid-stream (client "unexpected EOF").
Add read_lockstep(): on the verify_reconstruction path, read every live
shard reader once per stripe and wait for all of them, so all readers
advance one block per stripe and stay mutually aligned; any reader that
errors is retired for the rest of the object (a stream that failed
mid-block can no longer be trusted to be aligned). The adaptive
data-first path is unchanged for non-verifying callers (e.g. heal).
Adds a regression test reproducing a data shard that dies partway through
a multi-stripe object; it must still reconstruct byte-exact output.
Refs backlog#832.
* refactor(ecstore): sink write/rename/delete primitives into set_disk::core::io_primitives (backlog#820)
P5 step 2 of the SetDisks God-Object split (tracking backlog#815, issue
backlog#820; follows step 1 #4285). Relocate the entire set_disk/write.rs
primitive family into the core/io_primitives.rs module home established by
step 1:
- Module items: dangling_delete_grace() + its env consts, and the
OrphanDirScan scan-result enum.
- impl SetDisks write/rename/delete primitives shared across mod.rs and the
ops/ operation families: rename_data, commit_rename_data_dir,
cleanup_multipart_path, rename_part, eval_disks, write_unique_file_info,
update_object_meta(_with_opts), delete_if_dangling, delete_prefix,
scan_orphan_dir, purge_orphan_dir_object, check_write_precondition,
default_read_quorum, default_write_quorum.
- The dangling_delete_grace unit tests move with their subject.
set_disk/write.rs is removed and 'mod write;' dropped from mod.rs.
Pure move + visibility adjustment, zero logic change. Method bodies are moved
verbatim; pub(super) items are widened to pub(in crate::set_disk) so mod.rs /
ops still reach them (write.rs's super was set_disk; the deeper module needs the
explicit path to preserve identical reach). Callers use inherent self./Self::
calls, so no call sites change.
Verification:
- cargo check / clippy -D warnings -p rustfs-ecstore --all-targets: clean
- cargo test -p rustfs-ecstore --lib: 1841 passed, 0 failed (moved
dangling_delete_grace tests run as set_disk::core::io_primitives::tests::*)
- all five arch guard scripts: pass
- token-stream diff of moved block vs original write.rs: identical modulo
visibility tokens (2713 tokens each)
* refactor(ecstore): sink shared read primitives into set_disk::core::io_primitives (backlog#820)
P5 step 3 (final) of the SetDisks God-Object split (tracking backlog#815, issue
backlog#820; follows steps 1 #4285 and 2). Relocate the shared, low-level
metadata/erasure READ PRIMITIVE methods out of set_disk/read.rs into the
core/io_primitives.rs module home, leaving the object-read operation itself in
read.rs.
Moved into a new impl SetDisks block in io_primitives.rs (verbatim bodies):
- read_parts, read_all_fileinfo (+ _observed / _inner / _full_wait /
_early_stop variants), read_all_xl, load_file_info_versions_exact,
read_all_raw_file_info, pick_latest_quorum_files_info, read_multiple_files.
- The should_allow_metadata_early_stop free helper (called by the moved
read_all_fileinfo_observed).
Kept in read.rs: the object-read operation and its private helpers
(read_version_optimized, get_object_fileinfo, get_object_info_and_quorum,
try_get_object_direct_data_shards_with_fileinfo, get_object_with_fileinfo,
get_object_decode_reader_with_fileinfo, build_codec_streaming_part_reader) and
the metadata-cache helpers/tests. read.rs reaches the moved primitives through
the SetDisks core (inherent self./Self:: calls; the sole cross-boundary edge,
get_object_fileinfo -> read_all_fileinfo_observed, is handled by widening that
one method to pub(in crate::set_disk)).
Pure move + visibility adjustment, zero logic change. pub(super) primitives are
widened to pub(in crate::set_disk); the three internal-only fanout variants
(_inner/_full_wait/_early_stop) stay private; load_file_info_versions_exact
stays pub(crate). Method bodies are moved verbatim.
Verification:
- cargo check / clippy -D warnings -p rustfs-ecstore --all-targets: clean
- cargo test -p rustfs-ecstore --lib: 1841 passed, 0 failed
- all five arch guard scripts: pass
- token-stream diff of the three moved regions vs original read.rs: identical
modulo visibility tokens, the new impl-block scaffolding, and rustfmt
signature re-wrapping (read.rs diff is pure deletion, zero added lines)
* refactor(ecstore): extract read IO primitives to set_disk::core::io_primitives (backlog#820)
P5 step 1 of the SetDisks God-Object split (tracking backlog#815, issue
backlog#820). Relocate the module-level low-level read/erasure primitives out of
the 5,898-line set_disk/read.rs into a new core/io_primitives.rs module home:
- Metadata-fanout quorum machinery (MetadataQuorumAccumulator,
MetadataFanoutDiagnostics/Observation, MetadataEarlyStopDecision,
MetadataCacheLookup).
- Bitrot reader scheduling/creation (BitrotReaderSetup and the
create_bitrot_readers_* / schedule / fill helpers).
- Shard-cost classification, read-repair heal dedup, and codec-streaming
reader helpers.
Pure move + visibility adjustment, zero logic change. The moved block is
byte-identical to the pre-move read.rs source modulo the module header swap
(use super::* -> use super::super::*) and item visibility widening
(module-private / pub(super) -> pub(in crate::set_disk) so read.rs still
reaches them). read.rs's impl SetDisks body and imports are byte-identical to
before; mod.rs only gains 'mod core;' and repoints two
GetCodecStreamingReaderBuildOutcome references to the new path.
Verification:
- cargo check / clippy -D warnings -p rustfs-ecstore --all-targets: clean
- cargo test -p rustfs-ecstore --lib: 1840 passed, 0 failed
- normalized + token-stream diff of moved block vs original: identical modulo
visibility tokens and rustfmt signature re-wrapping
* fix(arch): allow io_primitives as READ_REPAIR_HEAL_CACHE owner (backlog#820)
The READ_REPAIR_HEAL_CACHE owner-path guard in
check_architecture_migration_rules.sh pinned the cache to
set_disk/read.rs. P5 step 1 relocated the cache (and its accessor
helpers) to set_disk/core/io_primitives.rs, so allow that path too.
Guard intent is unchanged: the cache stays behind the ECStore set-disk
read-owner helpers.
---------
Co-authored-by: houseme <housemecn@gmail.com>
fix(internode): relax aggressive HTTP/2 keepalive & RPC timeouts to stop large-object GET truncation
The internode gRPC channel used a 3s HTTP/2 keepalive timeout and a 10s
overall RPC timeout. Under high-concurrency large-object reads a saturated
peer's PING ACK is legitimately delayed past 3s, so the whole channel (and
every RPC/stream on it) is torn down as a 'dead peer'. In-flight peer shard
reads then fail mid-object and large GETs truncate after headers+Content-Length
are already sent, surfacing to clients as 'download error: unexpected EOF'.
Reproduced on a 4-node erasure cluster with pure GET-only warp (no concurrent
writes) at 64 concurrency: 10MiB GET ~31 unexpected-EOF / 3min. Raising the
internode keepalive timeout (3s->30s via env) alone cut that to ~7; also raising
the RPC timeout cut it to ~3.
- Raise DEFAULT_INTERNODE_HTTP2_KEEPALIVE_TIMEOUT_SECS 3 -> 20
- Raise DEFAULT_INTERNODE_RPC_TIMEOUT_SECS 10 -> 30
- Wire the data-plane rio HttpReader keepalive timeout (was hardcoded 3s) to the
same env/default so control- and data-plane stay consistent.
Refs backlog#832.
P1 of the SetDisks split (tracking backlog#815, depends on P0 backlog#816).
Give the Heal operation family its own module home: relocate set_disk/heal.rs
to set_disk/ops/heal.rs and move the HealOperations storage-api contract impl
beside its inherent helpers. The contract stays implemented for SetDisks so its
associated-type bounds are unchanged (ecstore_contract_compat_test still
covers it).
Method bodies are moved unchanged. The four inherent helpers widen from
pub(super) to pub(in crate::set_disk) to preserve their exact prior visibility
from the deeper module. get_pool_and_set now reads topology through
SetDisksCtx to keep the Heal family aligned with the P0 borrow pattern; the
read is provably identical (ctx.format()/pool_index() alias the core fields).
Runtime behavior is unchanged.
P0 of the SetDisks split (tracking backlog#815). Introduce SetDisksCtx<'a>,
a Copy borrow handle over the shared SetDisks core state (topology/config,
disks, locker trio), and validate the pattern by routing the List family's
delete_all through a ListOperations<'a> service unit.
No trait impl is moved and runtime behavior is byte-for-byte unchanged; the
public SetDisks::delete_all entry point is preserved and delegates to the
borrow-based unit.
* fix(rio): reject corrupted short compressed/encrypted blocks instead of panicking
DecompressReader::poll_read and DecryptReader::poll_read sliced the block
body with a fixed `[0..16]` index to read the length varint. The body length
comes from an untrusted 24-bit header field, so a corrupted/truncated block
shorter than 16 bytes made the slice panic and crash the request task — a
read-path DoS on GET of tiered/corrupted data.
Pass the whole (arbitrary-length-safe) slice to uvarint and reject a
non-positive or out-of-range length prefix with InvalidData. Adds a repro
test for each reader; all existing round-trip tests still pass.
Refs rustfs/backlog#812
* fix(utils): close SSRF bypass via IPv4-mapped IPv6 addresses
validate_outbound_ip branched on the IpAddr variant, and the V6 branch's
is_loopback/is_unicast_link_local/is_unique_local checks never inspect the
embedded IPv4 of an IPv4-mapped address (::ffff:a.b.c.d). The metadata guard
also only matched the plain V4 169.254.169.254. So ::ffff:127.0.0.1,
::ffff:10.0.0.5 and ::ffff:169.254.169.254 all passed the outbound guard,
letting an attacker reach loopback/private/metadata endpoints.
Normalize IPv4-mapped IPv6 to its embedded IPv4 (via to_ipv4_mapped, which
matches only the true mapped form) before classification. Adds reject tests
for mapped loopback/private/metadata and an allow test for public IPv6.
Refs rustfs/backlog#813
* fix(ecstore): streaming last-part loss, GCS tier Range/remove, stat_all_dirs alignment
Four confirmed data-reliability defects:
- put_object_multipart_stream: the CompleteMultipartUpload part-collection loop
used exclusive `1..total_parts_count`, dropping the final part (and collecting
zero parts for a single-part object) — silently truncating the completed object.
Extracted collect_complete_parts (1..=total_parts_count) with unit tests.
- GCS warm backend get() ignored the requested byte range, returning the whole
object for a Range GET; now applies ReadRange::segment like the other backends.
- GCS warm backend remove() was an empty stub, so deleting a tiered object left
it on GCS forever; now deletes via StorageControl (added a control-plane client),
and in_use() actually lists (prefix-scoped) instead of always returning false.
- stat_all_dirs skipped None disk slots and dropped JoinErrors, returning a
compressed, misaligned error vector; heal_object_dir then zipped it against the
full disks array and could make_volume on the WRONG disk. Now returns one
index-aligned entry per slot (None -> DiskNotFound), and heal no longer
pre-fills the drive report (which would double it). Added an alignment test.
Refs rustfs/backlog#807
* fix(kms): stop Vault backend from destroying/reviving keys on failure
Two confirmed key-safety defects in the Vault KV2 backend:
- get_key_material() 'self-healed' a decrypt or wrong-length failure by minting a
fresh random master key and overwriting the stored value. That destroys the
original key material, making every DEK ever wrapped by it permanently
undecryptable. Decryption must never mutate the stored key: both branches now
return a cryptographic_error instead. (The empty-material bootstrap path, which
only fills a never-initialized key, is intentionally left intact.)
- cancel_key_deletion() reset key_state to Enabled only in the returned response
and never persisted it, so the key stayed PendingDeletion in storage and would
still be reaped. It now writes the state back via update_key_metadata_in_storage
and fails the request if the write fails.
Adds ignored (Vault-requiring) integration tests documenting both behaviours.
The third item (VaultTransit key state only in memory -> revived as Enabled after
restart) is deferred: a fail-closed guard would break restart availability for all
transit keys; the correct fix needs a persistent metadata store + Vault integration
testing. Tracked in rustfs/backlog#808.
Refs rustfs/backlog#808
* fix(admin): clamp STS AssumeRole duration; persist ImportBucketMetadata to disk
Two confirmed admin-API defects:
- Standard AssumeRole used the raw client-supplied DurationSeconds with no upper
bound, so a caller could mint near-permanent temporary credentials. Clamp it to
the AWS/MinIO STS window [900, 43200] (with 0 -> default 3600) via a shared
clamp_assume_role_duration helper, and build the exp claim with saturating_add.
This matches the existing AssumeRoleWithWebIdentity path.
- ImportBucketMetadata only mutated an in-memory map and returned 200, silently
dropping every imported config. It now persists each non-empty config via
metadata_sys::update (which merges onto existing on-disk metadata) and returns
InternalError if a write fails. Mapping extracted to imported_configs_to_persist
with unit tests.
Refs rustfs/backlog#809
* fix(heal): enqueue displacing request in release builds
push_displacing_lower_priority folded the real enqueue call into
debug_assert_eq!(self.push(request), Accepted). In release builds
(debug_assertions off) the whole macro — including its argument — is compiled
out, so after evicting a lower-priority queued item the new high-priority
request was silently dropped and never healed. Hoist self.push(request) out of
the assertion so the side effect runs in all builds. Adds a --release regression
test.
Refs rustfs/backlog#811
* fix(iam): propagate real delete_policy backend errors instead of swallowing them
delete_policy's is_from_notify path had its error handling inverted: a real
backend failure (disk IO / insufficient quorum) evicted the cache and returned
Ok(()), reporting a phantom success while policy.json survived on disk (to be
reloaded on the next full IAM reload); NoSuchPolicy — which should be idempotent
success — returned Err. Propagate real errors and let NoSuchPolicy fall through
to the idempotent cache-evict + Ok, matching delete_user / the notification
handler in the same file. Adds a backend-error-injection regression test.
Refs rustfs/backlog#810
* fix(utils): also normalize IPv4-compatible IPv6 in the SSRF guard
The initial fix only unwrapped IPv4-mapped (::ffff:a.b.c.d) addresses; the
deprecated IPv4-compatible form (::a.b.c.d, e.g. ::127.0.0.1 / ::169.254.169.254)
still bypassed the guard. Reject pure-IPv6 specials (::, ::1, fe80::, fc00::)
first, then normalize BOTH embedded-IPv4 forms before the IPv4 rules. Adds tests
for compatible-form loopback/metadata and confirms ::1 / :: stay rejected.
Found by adversarial review of the initial fix. Refs rustfs/backlog#813
* fix(ecstore): fix the same last-part loss in the parallel streaming path
put_object_multipart_stream_parallel had the identical off-by-one
(1..total_parts_count) that truncated the last part / produced zero parts for a
single-part upload — reachable when concurrent stream parts are enabled. Reuse
collect_complete_parts, which now returns an error instead of panicking on a gap
in the parts map. Adds a missing-part error test.
Found by adversarial review of the initial fix. Refs rustfs/backlog#807
* fix(kms): local backend must preserve key material on status change
LocalKmsClient (the default KMS backend) regenerated the master key material on
enable_key/disable_key/schedule_key_deletion/cancel_key_deletion — a pure status
change. A single disable+enable cycle therefore destroyed the original key,
making every DEK ever wrapped by it permanently undecryptable (silent data loss,
no network needed). Preserve the existing material via get_key_material and
re-save with only the status changed. Adds a hermetic regression test that wraps
a DEK, cycles all four status methods, and asserts the DEK still decrypts.
Found by adversarial review of the Vault fix. Refs rustfs/backlog#808
* test(rio): cover the length-prefix guard; correct its comment
Add a DecompressReader test that feeds an unterminated length varint so uvarint
returns 0 and the new guard (not the downstream codec) produces the InvalidData
error, and reword the guard comment which overclaimed that the > len bound
prevents a reachable panic (it is belt-and-suspenders). No behavior change.
Found by adversarial review. Refs rustfs/backlog#812
* test(rio): build test block headers via vec! to satisfy clippy
The new corrupted-block tests built the header with Vec::new() + repeated push,
tripping clippy::vec_init_then_push (-D warnings in CI). Construct the fixed
header bytes with vec![] instead. No behavior change.
---------
Co-authored-by: houseme <housemecn@gmail.com>
Zhengchao An