* fix(rio): reject corrupted short compressed/encrypted blocks instead of panicking
DecompressReader::poll_read and DecryptReader::poll_read sliced the block
body with a fixed `[0..16]` index to read the length varint. The body length
comes from an untrusted 24-bit header field, so a corrupted/truncated block
shorter than 16 bytes made the slice panic and crash the request task — a
read-path DoS on GET of tiered/corrupted data.
Pass the whole (arbitrary-length-safe) slice to uvarint and reject a
non-positive or out-of-range length prefix with InvalidData. Adds a repro
test for each reader; all existing round-trip tests still pass.
Refs rustfs/backlog#812
* fix(utils): close SSRF bypass via IPv4-mapped IPv6 addresses
validate_outbound_ip branched on the IpAddr variant, and the V6 branch's
is_loopback/is_unicast_link_local/is_unique_local checks never inspect the
embedded IPv4 of an IPv4-mapped address (::ffff:a.b.c.d). The metadata guard
also only matched the plain V4 169.254.169.254. So ::ffff:127.0.0.1,
::ffff:10.0.0.5 and ::ffff:169.254.169.254 all passed the outbound guard,
letting an attacker reach loopback/private/metadata endpoints.
Normalize IPv4-mapped IPv6 to its embedded IPv4 (via to_ipv4_mapped, which
matches only the true mapped form) before classification. Adds reject tests
for mapped loopback/private/metadata and an allow test for public IPv6.
Refs rustfs/backlog#813
* fix(ecstore): streaming last-part loss, GCS tier Range/remove, stat_all_dirs alignment
Four confirmed data-reliability defects:
- put_object_multipart_stream: the CompleteMultipartUpload part-collection loop
used exclusive `1..total_parts_count`, dropping the final part (and collecting
zero parts for a single-part object) — silently truncating the completed object.
Extracted collect_complete_parts (1..=total_parts_count) with unit tests.
- GCS warm backend get() ignored the requested byte range, returning the whole
object for a Range GET; now applies ReadRange::segment like the other backends.
- GCS warm backend remove() was an empty stub, so deleting a tiered object left
it on GCS forever; now deletes via StorageControl (added a control-plane client),
and in_use() actually lists (prefix-scoped) instead of always returning false.
- stat_all_dirs skipped None disk slots and dropped JoinErrors, returning a
compressed, misaligned error vector; heal_object_dir then zipped it against the
full disks array and could make_volume on the WRONG disk. Now returns one
index-aligned entry per slot (None -> DiskNotFound), and heal no longer
pre-fills the drive report (which would double it). Added an alignment test.
Refs rustfs/backlog#807
* fix(kms): stop Vault backend from destroying/reviving keys on failure
Two confirmed key-safety defects in the Vault KV2 backend:
- get_key_material() 'self-healed' a decrypt or wrong-length failure by minting a
fresh random master key and overwriting the stored value. That destroys the
original key material, making every DEK ever wrapped by it permanently
undecryptable. Decryption must never mutate the stored key: both branches now
return a cryptographic_error instead. (The empty-material bootstrap path, which
only fills a never-initialized key, is intentionally left intact.)
- cancel_key_deletion() reset key_state to Enabled only in the returned response
and never persisted it, so the key stayed PendingDeletion in storage and would
still be reaped. It now writes the state back via update_key_metadata_in_storage
and fails the request if the write fails.
Adds ignored (Vault-requiring) integration tests documenting both behaviours.
The third item (VaultTransit key state only in memory -> revived as Enabled after
restart) is deferred: a fail-closed guard would break restart availability for all
transit keys; the correct fix needs a persistent metadata store + Vault integration
testing. Tracked in rustfs/backlog#808.
Refs rustfs/backlog#808
* fix(admin): clamp STS AssumeRole duration; persist ImportBucketMetadata to disk
Two confirmed admin-API defects:
- Standard AssumeRole used the raw client-supplied DurationSeconds with no upper
bound, so a caller could mint near-permanent temporary credentials. Clamp it to
the AWS/MinIO STS window [900, 43200] (with 0 -> default 3600) via a shared
clamp_assume_role_duration helper, and build the exp claim with saturating_add.
This matches the existing AssumeRoleWithWebIdentity path.
- ImportBucketMetadata only mutated an in-memory map and returned 200, silently
dropping every imported config. It now persists each non-empty config via
metadata_sys::update (which merges onto existing on-disk metadata) and returns
InternalError if a write fails. Mapping extracted to imported_configs_to_persist
with unit tests.
Refs rustfs/backlog#809
* fix(heal): enqueue displacing request in release builds
push_displacing_lower_priority folded the real enqueue call into
debug_assert_eq!(self.push(request), Accepted). In release builds
(debug_assertions off) the whole macro — including its argument — is compiled
out, so after evicting a lower-priority queued item the new high-priority
request was silently dropped and never healed. Hoist self.push(request) out of
the assertion so the side effect runs in all builds. Adds a --release regression
test.
Refs rustfs/backlog#811
* fix(iam): propagate real delete_policy backend errors instead of swallowing them
delete_policy's is_from_notify path had its error handling inverted: a real
backend failure (disk IO / insufficient quorum) evicted the cache and returned
Ok(()), reporting a phantom success while policy.json survived on disk (to be
reloaded on the next full IAM reload); NoSuchPolicy — which should be idempotent
success — returned Err. Propagate real errors and let NoSuchPolicy fall through
to the idempotent cache-evict + Ok, matching delete_user / the notification
handler in the same file. Adds a backend-error-injection regression test.
Refs rustfs/backlog#810
* fix(utils): also normalize IPv4-compatible IPv6 in the SSRF guard
The initial fix only unwrapped IPv4-mapped (::ffff:a.b.c.d) addresses; the
deprecated IPv4-compatible form (::a.b.c.d, e.g. ::127.0.0.1 / ::169.254.169.254)
still bypassed the guard. Reject pure-IPv6 specials (::, ::1, fe80::, fc00::)
first, then normalize BOTH embedded-IPv4 forms before the IPv4 rules. Adds tests
for compatible-form loopback/metadata and confirms ::1 / :: stay rejected.
Found by adversarial review of the initial fix. Refs rustfs/backlog#813
* fix(ecstore): fix the same last-part loss in the parallel streaming path
put_object_multipart_stream_parallel had the identical off-by-one
(1..total_parts_count) that truncated the last part / produced zero parts for a
single-part upload — reachable when concurrent stream parts are enabled. Reuse
collect_complete_parts, which now returns an error instead of panicking on a gap
in the parts map. Adds a missing-part error test.
Found by adversarial review of the initial fix. Refs rustfs/backlog#807
* fix(kms): local backend must preserve key material on status change
LocalKmsClient (the default KMS backend) regenerated the master key material on
enable_key/disable_key/schedule_key_deletion/cancel_key_deletion — a pure status
change. A single disable+enable cycle therefore destroyed the original key,
making every DEK ever wrapped by it permanently undecryptable (silent data loss,
no network needed). Preserve the existing material via get_key_material and
re-save with only the status changed. Adds a hermetic regression test that wraps
a DEK, cycles all four status methods, and asserts the DEK still decrypts.
Found by adversarial review of the Vault fix. Refs rustfs/backlog#808
* test(rio): cover the length-prefix guard; correct its comment
Add a DecompressReader test that feeds an unterminated length varint so uvarint
returns 0 and the new guard (not the downstream codec) produces the InvalidData
error, and reword the guard comment which overclaimed that the > len bound
prevents a reachable panic (it is belt-and-suspenders). No behavior change.
Found by adversarial review. Refs rustfs/backlog#812
* test(rio): build test block headers via vec! to satisfy clippy
The new corrupted-block tests built the header with Vec::new() + repeated push,
tripping clippy::vec_init_then_push (-D warnings in CI). Construct the fixed
header bytes with vec![] instead. No behavior change.
---------
Co-authored-by: houseme <housemecn@gmail.com>
Remove or consolidate 57 test cases that cannot catch regressions
(literal-constant asserts, construct-then-assert, derived-serde
round-trips, near-duplicate env/getter matrices) in common, config,
iam, madmin, and object-capacity, keeping all wire-format and
error-path guards. Add 13 tests for previously uncovered high-risk
behavior: filemeta version-sort determinism and merge resilience to
garbage headers, zip extraction path-traversal rejection and exact
limit boundaries, JWT tampered-signature rejection, and the bytes
variant of dual-key (rustfs/minio) metadata fallback and precedence.
Test-only change; no production code touched.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* refactor: move delete object contracts to storage api
* refactor: narrow store api compatibility exports
* refactor: route table catalog test through storage compat
Persist decommission progress after either the existing time interval or a migrated-item threshold, and flush progress baselines after bucket and terminal-state saves.
Also stabilize the OIDC discovery mock used by the pre-commit gate.
`is_safe_claim_policy_name` rejected any character other than
`[a-zA-Z0-9_-]`, silently dropping policy names containing colons.
This breaks Kubernetes workload identity where `claim_name=sub`
resolves to `system:serviceaccount:<namespace>:<sa-name>` — a valid
policy name that can be created via the admin API but is then
unreachable during STS session authorization.
Add `:` and `.` to the allowed character set. These characters are:
- Used in K8s service account `sub` claims (colons)
- Used in Java/DNS-style group names from OIDC providers (dots)
- Already accepted by the `add-canned-policy` admin API endpoint
Require at least one alphanumeric character to prevent meaningless
names (`.`, `..`, `-`, `_`, `:`, etc.) from resolving.
Still rejected: `/`, `\`, whitespace, `$`, `;`, `{`, `}` and other
chars that could enable path traversal or injection.
Signed-off-by: Alexander Kharkevich <alex@mara.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: GatewayJ <835269233@qq.com>
Add `RUSTFS_IDENTITY_OPENID_HIDE_FROM_UI[_<SUFFIX>]` setting that
removes a provider from the login page while keeping it fully
functional for STS AssumeRoleWithWebIdentity and site-replication.
Changes:
- Add `hide_from_ui: bool` to `OidcProviderConfig`
- Add `list_visible_providers()` that filters hidden providers
(used by console login and /v3/oidc/providers endpoint)
- Keep `list_providers()` unfiltered for site-replication/admin config
- Extract `normalize_provider_config(config) -> config` to deduplicate
field normalization (accepts the struct directly, not 18 parameters)
- Add `parse_enable_state()` helper for consistent EnableState parsing
- Plumb through admin API request structs (`#[serde(default)]`)
- Expose in `OidcConfigView` for admin GET config round-trip
- Persist via `upsert_persisted_provider_config()`
Note: adding `hide_from_ui` to the public `OidcProviderConfig` struct
is a source-level change for code constructing it with struct literals.
This is acceptable for the current pre-1.0 release cycle.
Signed-off-by: Alexander Kharkevich <alex@mara.com>
Co-authored-by: GatewayJ <835269233@qq.com>
GatewayJ