Commit Graph

139 Commits

Author SHA1 Message Date
houseme 13bdca6762 build(toolchain): switch Rust channel to stable (#4775)
* Change Rust toolchain channel to stable

Signed-off-by: houseme <housemecn@gmail.com>

* style: apply clippy --fix and cargo fix lint suggestions

Run `cargo clippy --fix --all-targets --all-features` and
`cargo fix --lib --all-targets` across the workspace, then resolve the
remaining warnings by hand:

- collapse needless borrows in `format!` args, prefer `?` over explicit
  early returns, and use `.values()` / `.flatten()` iterator adapters
- rewrite the `Md5` scan loop via `manual_flatten` and re-indent the
  `select!` macro body (rustfmt skips macro interiors)
- annotate the intentional dead-code `Md5` inherent methods (constructed
  only by the test factory) with `#[allow(dead_code)]`

Behavior is unchanged.

Co-Authored-By: heihutu <heihutu@gmail.com>

---------

Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: heihutu <heihutu@gmail.com>
2026-07-12 18:59:43 +08:00
Zhengchao An 008b872414 refactor(iam): hand the built IAM system to the AppContext explicitly (#4624)
backlog#1052 S3, fourth slice. The AppContext's IamHandle already owned an
Arc<IamSys>, but the value it held was read back from the process
singleton after init — so a future second server's context would silently
bind to the first server's IAM domain (credentials, policies, users all
belong to a specific store's .rustfs.sys).

- rustfs_iam gains build_iam_sys(store): construct an IAM system bound to
  a store without touching the singleton. init_iam_sys wraps it (build,
  publish first-wins, return the handle — previously Result<()>).
- Startup threads the built handle: the inline bootstrap path passes the
  Arc it just created into ensure_startup_after_iam, which constructs the
  AppContext from the passed value instead of re-resolving the global.
  The deferred-recovery path resolves the freshly published global
  directly (context-first resolution cannot be used there: the AppContext
  does not exist yet — creating it is the finalizer's job; caught by the
  embedded deferred-IAM e2e).
- IamHandle::is_ready() reports the held system's readiness instead of
  consulting the process singleton; the dead global re-resolver is
  removed. token_signing_key stays on the credentials global until S4.

157 iam tests + embedded e2e (basic + deferred recovery) green.
2026-07-09 22:35:00 +08:00
Zhengchao An 476352106e refactor(app): per-context server-config handle and Arc-owned notification system (#4621)
* refactor(app): give each context's server-config handle its own copy

backlog#1052 S3, first slice: establish the per-context state pattern on
the smallest subsystem. ServerConfigHandle was a unit struct forwarding
both get and set to the process-global GLOBAL_SERVER_CONFIG, so every
AppContext shared one config regardless of which context a caller held.

The handle now keeps its own copy: a set lands on the owning context and
on the process global (ambient readers — the config loader path, the
scanner — keep working), and a get prefers the owned copy, falling back
to the global while the initial load still publishes there. Single
instance: the owned copy and the global are written together, so reads
are unchanged. Two contexts that both published stay isolated even though
the shared global only remembers the last write (covered by a new test).

The global write in set() is the transitional bridge for ambient readers;
it goes away when those readers migrate and multi-instance flips on
(backlog#1052 S5).

* refactor(notify): hand out the notification system as an owned Arc

backlog#1052 S3, second slice. GLOBAL_NOTIFICATION_SYS stored the
NotificationSys by value and every accessor returned
Option<&'static NotificationSys> — a process-lifetime borrow that a
per-server AppContext can never hold. The global now stores an Arc and
the accessor chain (ecstore runtime sources, ECStore::notification_system,
the iam forwarders, the rustfs shims, NotificationSystemInterface and the
AppContext resolvers) hands out owned Arcs instead.

Call sites are unchanged apart from two borrow adjustments in the
rebalance admin handler (pass &Arc where a reference is expected; borrow
with as_ref() so the handle survives to the second use). Behavior is
identical — same single global instance, first init still wins — but the
type now permits a future per-server context to own its notification
system (backlog#1052 S3 follow-up).
2026-07-09 20:07:09 +08:00
Zhengchao An 16a91c35ec perf(iam): load IAM lists in fixed chunks to avoid O(n^2) startup load (backlog#806) (#4537)
load_all fetched policies/users/user-policies by passing the FULL remaining
list to load_*_concurrent every iteration and then split_off(32), so each item
was re-fetched once per preceding chunk — O(n^2) redundant loads on startup for
>32 items. Replace the split_off loops with chunks(32) so each item is fetched
exactly once. Behavior-preserving (cache inserts were already idempotent).
2026-07-09 02:01:36 +08:00
Henry Guo 506cd156bb fix(scanner): scope long walk timeouts (#4376)
* fix(scanner): scope long walk timeouts

* fix(scanner): bound IAM config walks

---------

Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
2026-07-08 17:06:51 +08:00
houseme 7001373316 fix(iam): load IAM bootstrap snapshot without namespace locks (#4363)
* fix(iam): load IAM bootstrap snapshot without namespace locks

IAM bootstrap (init_iam_sys -> load_all) read every config object with
default ObjectOptions (no_lock=false), so each read acquired a
distributed namespace read lock. Lock quorum is counted over cluster
nodes and unreachable peers are hard failures, so during a sequential
restart the very first read failed with
"Quorum not reached: required 2, achieved 0" and IAM could not come up
until enough peers' lock RPC surfaces converged - even when the storage
read quorum was already satisfiable (rustfs#4304).

Extend the startup contract from rustfs#4056 ("startup metadata I/O must
not require namespace locks") to the IAM bootstrap path:

- Introduce LoadMode {Locked, BootstrapNoLock} and plumb it through the
  load_all chain (groups, users, policies, mapped policies and their
  concurrent variants) down to the storage read options.
- load_all now performs all reads with no_lock=true; on-line
  single-object loads via the Store trait keep locked semantics.
- Fail-closed behavior is unchanged: any loader error still aborts the
  whole snapshot load.

Safety: config objects are atomic whole-object writes, so a lock-free
read only observes an old or a new value; staleness is bounded by the
existing periodic IAM reload. Listing (walk) never took namespace locks,
and maybe_schedule_lazy_rewrite stays a best-effort background task.

Verification:
- cargo test -p rustfs-iam --lib (153 passed, incl. new LoadMode tests)
- cargo clippy -p rustfs-iam --all-targets
- cargo check -p rustfs
- make pre-commit

Ref: rustfs#4304; tracking rustfs/backlog#884, rustfs/backlog#885

Co-Authored-By: heihutu <heihutu@gmail.com>

* test(iam): sequential-restart regression test for lock-free bootstrap

Add an integration test reproducing the rustfs#4304 failure mode against
a real 4-disk temp-dir ECStore:

- Seed IAM group data in single-node mode, then flip the runtime into
  distributed-erasure mode. new_ns_lock now builds a distributed lock
  over the set's (empty) lock-client list, so every namespace-locked
  read fails exactly like a sequential restart with unreachable peers
  (lock quorum unavailable, storage read quorum healthy).
- Assert the locked load_group path fails in that state, while the
  bulk snapshot load_all (no_lock plumbing from the previous commit)
  succeeds, and the data survives intact once single-node mode is
  restored.

Reverse-verified: temporarily switching load_all back to the locked
mode makes the test fail, so it genuinely guards the contract.

Verification:
- cargo test -p rustfs-iam --test iam_bootstrap_no_lock_test
- cargo test -p rustfs-iam --lib

Ref: rustfs#4304; tracking rustfs/backlog#886

Co-Authored-By: heihutu <heihutu@gmail.com>

* test(iam): route test ECStore imports through ecstore_test_compat boundary

The new integration test imported rustfs_ecstore facade paths directly,
tripping three architecture migration rules. Move every ECStore import
behind crates/iam/tests/ecstore_test_compat/mod.rs (the sanctioned
test-compat pattern), and register that module as a reviewed test-only
global-facade boundary in check_architecture_migration_rules.sh: the
sequential-restart regression test needs api::global::update_erasure_type
to flip into distributed-erasure mode for lock-quorum fault injection.

Verification:
- ./scripts/check_architecture_migration_rules.sh
- cargo test -p rustfs-iam --test iam_bootstrap_no_lock_test
- make pre-commit

Co-Authored-By: heihutu <heihutu@gmail.com>

* feat(server): expose readiness blocking reason + rolling-restart runbook

Operators hitting the rustfs#4304 sequential cold start could not tell
from the outside why a node stayed unavailable. Three additions:

- The readiness gate's 503 now names the blocking dependency in both the
  body ("Service not ready: waiting for storage_quorum") and a new
  x-rustfs-readiness-pending header (storage_quorum | iam |
  startup_finalization), derived from the current startup stage.
  /health/ready already returned details + degradedReasons; this covers
  the plain S3 requests that hit the gate.
- IAM bootstrap retry logs now carry an actionable `hint` field that
  classifies the failure (storage read quorum vs lock quorum vs
  uninitialized metadata) instead of only echoing the storage error.
- New docs/operations/rolling-restart.md runbook: correct rolling
  restart procedure, sequential cold-start expectations (degraded ->
  auto-recovery), readiness signal reference, and
  RUSTFS_STARTUP_READINESS_MAX_WAIT_SECS guidance.

Verification:
- cargo test -p rustfs --lib -- hint_tests service_not_ready readiness_pending
- make pre-commit

Ref: rustfs#4304; tracking rustfs/backlog#887

Co-Authored-By: heihutu <heihutu@gmail.com>

* upgrade deps version and improve import

* feat(iam): notification-path cache refreshes read without namespace locks (#4368)

P3 step 1 of rustfs/backlog#884 (scoped down from full MinIO readConfig
alignment after review): cross-node notification handlers
(group/policy/policy-mapping/user) refresh the local IAM cache with
single-object reads that previously took distributed namespace read
locks. These refreshes are asynchronous, best-effort, and already
stale-tolerant (the periodic reload converges them), so a node-counted
lock quorum failure or lock RPC hiccup on a peer must not fail them —
the same rationale as the lock-free bootstrap load_all (rustfs#4304).

- Store trait: add load_user_no_lock / load_group_no_lock /
  load_policy_doc_no_lock / load_mapped_policy_no_lock with defaults
  forwarding to the locked variants, so existing implementations and
  test mocks keep their behavior.
- ObjectStore overrides them via the existing LoadMode::BootstrapNoLock
  plumbing. Deletions triggered by the handlers keep locked writes.
- manager.rs: the four *_notification_handler paths (8 call sites)
  switch to the lock-free variants.
- Integration test: while the lock quorum is unavailable (DistErasure
  with empty lockers), load_group_no_lock must succeed exactly where
  the locked load_group fails.

Request-path loads (check_key, verify_temp_user_persistence) and admin
write-then-reload paths intentionally stay locked: load_user_identity
embeds expiry deletions, so those need the side-effect extraction
tracked in rustfs/backlog#884 before going lock-free.

Verification:
- cargo test -p rustfs-iam --lib (156 passed)
- cargo test -p rustfs-iam --test iam_bootstrap_no_lock_test
- make pre-commit

Ref: rustfs/backlog#884, rustfs#4304

Co-authored-by: heihutu <heihutu@gmail.com>

* fix(server): drop unused iam_bootstrap_failure_hint import in tests

The hint tests live in their own hint_tests module with a local import;
the stale re-import in mod tests failed clippy's -D warnings on the
Test and Lint CI variants.

Verification:
- cargo clippy -p rustfs --all-targets

Co-Authored-By: heihutu <heihutu@gmail.com>

* fix

---------

Co-authored-by: heihutu <heihutu@gmail.com>
2026-07-07 20:07:09 +08:00
houseme 717cdd2abd fix(migration): decrypt MinIO IAM & server config on drop-in migration (#4358)
* fix(migration): decrypt MinIO IAM & server config on drop-in migration

MinIO encrypts IAM identity/service-account files and the server config at
rest with a key derived from the root credentials. The drop-in migration
paths read those blobs from the legacy `.minio.sys` bucket and parsed them
as plaintext JSON, so any encrypted blob failed to parse and was silently
skipped with "incompatible format". This is why users migrating from MinIO
kept their buckets/objects/policies but lost users and access keys (#2212).

The IAM load path already knows how to decrypt these blobs (RustFS master
keys plus MinIO-compatible legacy keys derived from the root credentials),
but that logic lived behind a private method and was never used by the
migration paths. Expose it as `rustfs_iam::try_decrypt_iam_blob` and inject
it into both migration paths via a `LegacyBlobDecryptFn` callback (ecstore
cannot depend on the IAM crate, so the closure is wired in the binary crate).
When a blob cannot be decrypted the raw bytes are used as-is, preserving the
previous plaintext-only behavior with no regression.

Also improve object-layer migration observability without changing control
flow: `try_migrate_format` now distinguishes "no legacy format" (a normal
fresh install) from "legacy format present but incompatible", and the caller
logs a loud error before initializing a fresh format that would leave the
existing MinIO objects unreadable. Topology/version skip reasons are promoted
from debug to warn.

Fixes a pre-existing test isolation race by marking
`test_recovery_falls_back_to_default_config_when_blob_stays_corrupt` serial,
since it reads a process-wide env var toggled by a sibling test.

Co-Authored-By: heihutu <heihutu@gmail.com>

* fix(migration): box FormatV3 in LegacyFormatOutcome to satisfy clippy

Co-Authored-By: heihutu <heihutu@gmail.com>

* test(ecstore): stabilize concurrent multipart resend lock timeout

concurrent_resend_same_part_commits_one_generation spawns 6 same-part
resends whose cross-disk commits serialize on the per-uploadId commit
lock. Under the full nextest suite the parallel disk load pushes those
serialized commits past the small default lock-acquire timeout (5s),
producing a spurious `Lock(Timeout ...)` unrelated to the property under
test (observed on CI at 5.775s vs ~0.5s in isolation).

Raise RUSTFS_OBJECT_LOCK_ACQUIRE_TIMEOUT to the production default (30s)
for the concurrent-commit section via temp_env, so the regression guard
reflects correctness (exactly one intact generation) rather than disk
latency under CI load. The meaningful assertions are unchanged, and
#[serial] keeps the process-wide env override isolated.

Co-Authored-By: heihutu <heihutu@gmail.com>

* fix(lock): bound fast-lock notification wait to prevent lost-wakeup stall

The real cause of the concurrent_resend_same_part_commits_one_generation
failures was a lost wakeup in the fast-lock slow path, not disk latency:
raising the acquire timeout to 30s only delayed the failure (it then timed
out at 30s), proving a genuine stall rather than overload.

In acquire_lock_slow_path a waiter that reaches the notification phase did a
single `timeout(remaining, wait_for_write())` spanning the whole acquire
budget, and treated that wait's elapse as a hard `Timeout`. But the release
path only notifies when `writer_waiters > 0`, so if the holder releases in
the gap after the waiter's `try_acquire` fails and before it registers as a
waiter, no notification (and no stored permit, since the pooled `Notify` is
gated) is produced. The waiter then blocks until the deadline even though the
lock is free and stays free — a spurious lock-acquire timeout. The shared
process-wide notify pool makes it worse: a wakeup can be consumed by a waiter
of a different lock hashing to the same slot.

Bound each notification wait (NOTIFY_WAIT_CAP = 50ms) and, on elapse, loop
back and re-`try_acquire` instead of returning `Timeout`; the deadline check
at the top of the loop is the single source of truth for timing out. A
lost/stolen wakeup now degrades to bounded re-polling (acquire within ~50ms
of the lock becoming free) instead of stalling for the whole timeout.
Correctness (mutual exclusion) is unchanged — acquisition still only happens
via `try_acquire_*`.

Add a regression test that reproduces the stall (holder + late waiter across
many keys): it times out without the fix and passes in ~1s with it. Revert
the earlier acquire-timeout workaround in the multipart test now that the
underlying stall is fixed, so it runs under the default timeout again.

Co-Authored-By: heihutu <heihutu@gmail.com>

---------

Co-authored-by: heihutu <heihutu@gmail.com>
2026-07-07 16:36:05 +08:00
Zhengchao An 8f11222a63 feat(admin): add MinIO-compatible IAM and IDP endpoints (#4334)
feat(admin): add MinIO-compatible IAM/IDP admin endpoints

Register and implement MinIO admin API compatibility for IAM/IDP:
- PUT /v3/import-iam-v2 and POST /v3/revoke-tokens/{userProvider}
- generic /v3/idp-config/{type}[/{name}] CRUD mapped onto existing config
- LDAP/OpenID service-account, policy-entities, and list-access-keys flows

Adds IamSys::delete_temp_account primitive to back STS token revocation.
revoke-tokens requires the broader ListUsers admin capability for
cross-user revocation (self-revocation only needs RemoveServiceAccount),
mirroring the cross-user guard used by the service-account handlers.
Registers admin route-policy inventory entries for every new route.
Unsupported LDAP/OpenID backends return honest compatibility errors.

Refs rustfs/backlog#609 #610 #616
2026-07-07 05:12:30 +08:00
Zhengchao An 31c6859965 chore: converge stale TODOs and apply safe fills (backlog#646) (#4322)
Second TODO-convergence round over the current tree (backlog#646). All
line numbers in the old inventory had gone stale after the set_disk /
diagnostics / cluster refactors, so this re-scans and reduces the marker
count from 144 to 99.

STALE removals (comment describes already-implemented behavior, or dead
commented-out blocks) across ecstore (set_disk ops/core, store,
cluster/rpc, bucket/metadata_sys, services), iam, filemeta, s3select and
rustfs auth/object_usecase. No behavior change.

Safe fills, each verified:
- filemeta: replication_info_equals now also compares
  replication_state_internal (function currently has no callers; adds a
  regression test).
- bitrot: drop the confirmed-unused `_want` parameter from bitrot_verify
  and the now-unused `sum` on LocalDisk::bitrot_verify, removing a
  Bytes::copy_from_slice allocation. Streaming verify uses the file's
  embedded per-shard hash, never the passed sum.
- signer: rename v4_ignored_headers -> V4_IGNORED_HEADERS and drop the
  non_upper_case_globals allow.
- admin/heal: test_decode was #[ignore]d and used serde_urlencoded on a
  JSON body (would panic); rewire to serde_json::from_slice to match the
  production decode path, add assertions, un-ignore.

Verified: cargo fmt; cargo check on touched crates; tests pass
(filemeta, signer, bitrot, heal::test_decode); arch guardrail scripts
pass.
2026-07-06 22:43:32 +08:00
GatewayJ 9cf211930d fix(iam): expand OIDC auth diagnostics (#4281)
* fix(iam): expand OIDC auth diagnostics

* fix(iam): accept RFC3339 OIDC timestamps

* chore(iam): log OIDC policy mapping diagnostics

* chore(iam): log OIDC claim and policy details

* chore(iam): lower OIDC diagnostic log verbosity

* fix(iam): gate OIDC diagnostics behind debug

* chore: update yanked num-bigint lockfile
2026-07-05 18:05:23 +08:00
Zhengchao An e3a8234bc9 fix: 12 P1 reliability/security defects from the full-repo audit (backlog#806) (#4256)
* fix(rio): reject corrupted short compressed/encrypted blocks instead of panicking

DecompressReader::poll_read and DecryptReader::poll_read sliced the block
body with a fixed `[0..16]` index to read the length varint. The body length
comes from an untrusted 24-bit header field, so a corrupted/truncated block
shorter than 16 bytes made the slice panic and crash the request task — a
read-path DoS on GET of tiered/corrupted data.

Pass the whole (arbitrary-length-safe) slice to uvarint and reject a
non-positive or out-of-range length prefix with InvalidData. Adds a repro
test for each reader; all existing round-trip tests still pass.

Refs rustfs/backlog#812

* fix(utils): close SSRF bypass via IPv4-mapped IPv6 addresses

validate_outbound_ip branched on the IpAddr variant, and the V6 branch's
is_loopback/is_unicast_link_local/is_unique_local checks never inspect the
embedded IPv4 of an IPv4-mapped address (::ffff:a.b.c.d). The metadata guard
also only matched the plain V4 169.254.169.254. So ::ffff:127.0.0.1,
::ffff:10.0.0.5 and ::ffff:169.254.169.254 all passed the outbound guard,
letting an attacker reach loopback/private/metadata endpoints.

Normalize IPv4-mapped IPv6 to its embedded IPv4 (via to_ipv4_mapped, which
matches only the true mapped form) before classification. Adds reject tests
for mapped loopback/private/metadata and an allow test for public IPv6.

Refs rustfs/backlog#813

* fix(ecstore): streaming last-part loss, GCS tier Range/remove, stat_all_dirs alignment

Four confirmed data-reliability defects:

- put_object_multipart_stream: the CompleteMultipartUpload part-collection loop
  used exclusive `1..total_parts_count`, dropping the final part (and collecting
  zero parts for a single-part object) — silently truncating the completed object.
  Extracted collect_complete_parts (1..=total_parts_count) with unit tests.
- GCS warm backend get() ignored the requested byte range, returning the whole
  object for a Range GET; now applies ReadRange::segment like the other backends.
- GCS warm backend remove() was an empty stub, so deleting a tiered object left
  it on GCS forever; now deletes via StorageControl (added a control-plane client),
  and in_use() actually lists (prefix-scoped) instead of always returning false.
- stat_all_dirs skipped None disk slots and dropped JoinErrors, returning a
  compressed, misaligned error vector; heal_object_dir then zipped it against the
  full disks array and could make_volume on the WRONG disk. Now returns one
  index-aligned entry per slot (None -> DiskNotFound), and heal no longer
  pre-fills the drive report (which would double it). Added an alignment test.

Refs rustfs/backlog#807

* fix(kms): stop Vault backend from destroying/reviving keys on failure

Two confirmed key-safety defects in the Vault KV2 backend:

- get_key_material() 'self-healed' a decrypt or wrong-length failure by minting a
  fresh random master key and overwriting the stored value. That destroys the
  original key material, making every DEK ever wrapped by it permanently
  undecryptable. Decryption must never mutate the stored key: both branches now
  return a cryptographic_error instead. (The empty-material bootstrap path, which
  only fills a never-initialized key, is intentionally left intact.)
- cancel_key_deletion() reset key_state to Enabled only in the returned response
  and never persisted it, so the key stayed PendingDeletion in storage and would
  still be reaped. It now writes the state back via update_key_metadata_in_storage
  and fails the request if the write fails.

Adds ignored (Vault-requiring) integration tests documenting both behaviours.

The third item (VaultTransit key state only in memory -> revived as Enabled after
restart) is deferred: a fail-closed guard would break restart availability for all
transit keys; the correct fix needs a persistent metadata store + Vault integration
testing. Tracked in rustfs/backlog#808.

Refs rustfs/backlog#808

* fix(admin): clamp STS AssumeRole duration; persist ImportBucketMetadata to disk

Two confirmed admin-API defects:

- Standard AssumeRole used the raw client-supplied DurationSeconds with no upper
  bound, so a caller could mint near-permanent temporary credentials. Clamp it to
  the AWS/MinIO STS window [900, 43200] (with 0 -> default 3600) via a shared
  clamp_assume_role_duration helper, and build the exp claim with saturating_add.
  This matches the existing AssumeRoleWithWebIdentity path.
- ImportBucketMetadata only mutated an in-memory map and returned 200, silently
  dropping every imported config. It now persists each non-empty config via
  metadata_sys::update (which merges onto existing on-disk metadata) and returns
  InternalError if a write fails. Mapping extracted to imported_configs_to_persist
  with unit tests.

Refs rustfs/backlog#809

* fix(heal): enqueue displacing request in release builds

push_displacing_lower_priority folded the real enqueue call into
debug_assert_eq!(self.push(request), Accepted). In release builds
(debug_assertions off) the whole macro — including its argument — is compiled
out, so after evicting a lower-priority queued item the new high-priority
request was silently dropped and never healed. Hoist self.push(request) out of
the assertion so the side effect runs in all builds. Adds a --release regression
test.

Refs rustfs/backlog#811

* fix(iam): propagate real delete_policy backend errors instead of swallowing them

delete_policy's is_from_notify path had its error handling inverted: a real
backend failure (disk IO / insufficient quorum) evicted the cache and returned
Ok(()), reporting a phantom success while policy.json survived on disk (to be
reloaded on the next full IAM reload); NoSuchPolicy — which should be idempotent
success — returned Err. Propagate real errors and let NoSuchPolicy fall through
to the idempotent cache-evict + Ok, matching delete_user / the notification
handler in the same file. Adds a backend-error-injection regression test.

Refs rustfs/backlog#810

* fix(utils): also normalize IPv4-compatible IPv6 in the SSRF guard

The initial fix only unwrapped IPv4-mapped (::ffff:a.b.c.d) addresses; the
deprecated IPv4-compatible form (::a.b.c.d, e.g. ::127.0.0.1 / ::169.254.169.254)
still bypassed the guard. Reject pure-IPv6 specials (::, ::1, fe80::, fc00::)
first, then normalize BOTH embedded-IPv4 forms before the IPv4 rules. Adds tests
for compatible-form loopback/metadata and confirms ::1 / :: stay rejected.

Found by adversarial review of the initial fix. Refs rustfs/backlog#813

* fix(ecstore): fix the same last-part loss in the parallel streaming path

put_object_multipart_stream_parallel had the identical off-by-one
(1..total_parts_count) that truncated the last part / produced zero parts for a
single-part upload — reachable when concurrent stream parts are enabled. Reuse
collect_complete_parts, which now returns an error instead of panicking on a gap
in the parts map. Adds a missing-part error test.

Found by adversarial review of the initial fix. Refs rustfs/backlog#807

* fix(kms): local backend must preserve key material on status change

LocalKmsClient (the default KMS backend) regenerated the master key material on
enable_key/disable_key/schedule_key_deletion/cancel_key_deletion — a pure status
change. A single disable+enable cycle therefore destroyed the original key,
making every DEK ever wrapped by it permanently undecryptable (silent data loss,
no network needed). Preserve the existing material via get_key_material and
re-save with only the status changed. Adds a hermetic regression test that wraps
a DEK, cycles all four status methods, and asserts the DEK still decrypts.

Found by adversarial review of the Vault fix. Refs rustfs/backlog#808

* test(rio): cover the length-prefix guard; correct its comment

Add a DecompressReader test that feeds an unterminated length varint so uvarint
returns 0 and the new guard (not the downstream codec) produces the InvalidData
error, and reword the guard comment which overclaimed that the > len bound
prevents a reachable panic (it is belt-and-suspenders). No behavior change.

Found by adversarial review. Refs rustfs/backlog#812

* test(rio): build test block headers via vec! to satisfy clippy

The new corrupted-block tests built the header with Vec::new() + repeated push,
tripping clippy::vec_init_then_push (-D warnings in CI). Construct the fixed
header bytes with vec![] instead. No behavior change.

---------

Co-authored-by: houseme <housemecn@gmail.com>
2026-07-04 14:24:02 +08:00
GatewayJ b1582b3391 fix(iam): improve OIDC token exchange diagnostics (#4232) 2026-07-03 14:15:58 +08:00
Zhengchao An 9dfeffc4c1 test: prune redundant cases and add high-risk coverage across crates (#4208)
Remove or consolidate 57 test cases that cannot catch regressions
(literal-constant asserts, construct-then-assert, derived-serde
round-trips, near-duplicate env/getter matrices) in common, config,
iam, madmin, and object-capacity, keeping all wire-format and
error-path guards. Add 13 tests for previously uncovered high-risk
behavior: filemeta version-sort determinism and merge resilience to
garbage headers, zip extraction path-traversal rejection and exact
limit boundaries, JWT tampered-signature rejection, and the bytes
variant of dual-key (rustfs/minio) metadata fallback and precedence.

Test-only change; no production code touched.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 00:35:37 +08:00
Zhengchao An 1b3dea012e refactor: route ecstore runtime globals through facade (#3941) 2026-06-27 12:27:03 +08:00
Zhengchao An 72ae43cb90 refactor: segment external storage contract imports (#3908) 2026-06-26 16:44:59 +08:00
Zhengchao An 1f7e159388 refactor: segment external storage api boundaries (#3903) 2026-06-26 15:10:49 +08:00
Zhengchao An c9614eb7cb refactor: route ecstore storage api boundaries (#3892) 2026-06-26 09:35:18 +08:00
Zhengchao An e37e367390 refactor: route remaining external storage boundaries (#3889) 2026-06-26 06:22:51 +08:00
Zhengchao An ad6184b126 refactor: centralize iam runtime source helpers (#3828) 2026-06-24 22:49:24 +08:00
Zhengchao An 5c60f0cae9 refactor: centralize owner server config reads (#3793) 2026-06-23 21:27:30 +08:00
Zhengchao An 70a2441407 refactor: route notify dispatch through app context (#3789)
* refactor: route notify dispatch through app context

* refactor: route admin IAM globals through app context (#3791)

* refactor: centralize IAM root credential access (#3792)
2026-06-23 20:05:28 +08:00
houseme 583a23bdf2 fix(ecstore): replace panic-driven pool and set stubs (#3753)
* fix(ecstore): replace panic-driven pool and set stubs

* test(runtime): tolerate restricted local bind checks

* fix(ecstore): remove remaining trait stub placeholders

* fix(ecstore): tighten trait stub follow-up semantics

* chore: ignore local worktrees

* chore: update layer dependency baseline for resolve_* context entries

Add 7 accepted infra->app dependency entries introduced by recent
refactoring PRs (#3770, #3771, #3772) that route global state lookups
through app::context::resolve_* functions.

Co-Authored-By: heihutu <heihutu@gmail.com>

---------

Co-authored-by: heihutu <heihutu@gmail.com>
2026-06-23 12:31:17 +08:00
Zhengchao An 30559f7e1b refactor: collapse test fuzz ecstore thin bridges (#3765) 2026-06-23 07:04:51 +08:00
Zhengchao An 198fd4f150 refactor(runtime): route RustFS runtime consumers through storage owner (#3756) 2026-06-23 05:17:56 +08:00
Zhengchao An 6da61b44e5 refactor: expose remaining external owner symbols (#3751) 2026-06-22 23:35:50 +08:00
Zhengchao An 88a87b3e8f refactor: centralize external ECStore facade aliases (#3746) 2026-06-22 20:37:13 +08:00
Zhengchao An d3796d6c10 refactor: remove external owner compat bridges (#3741) 2026-06-22 18:28:35 +08:00
Zhengchao An 539c68778d refactor: remove standalone compat bridge modules (#3740) 2026-06-22 17:35:58 +08:00
安正超 cca9e83a8b refactor: use relative standalone compat consumers (#3734) 2026-06-22 15:07:58 +08:00
houseme cc6909b08b fix(iam): verify sts temp-user persistence (#3722) 2026-06-22 13:54:57 +08:00
安正超 de158743c3 refactor: split compat facade imports (#3716) 2026-06-22 09:53:22 +08:00
安正超 d39815470e refactor: prune consumer storage compat paths (#3704) 2026-06-22 02:31:30 +08:00
安正超 0985225448 refactor: consolidate ecstore public facade (#3678)
* refactor: expand ecstore compatibility facade

* test: enforce ecstore api facade imports

* refactor: hide legacy ecstore layout modules

* refactor: hide ecstore facade root modules
2026-06-21 09:09:11 +08:00
安正超 9b3fda9e30 refactor: route ecstore public surfaces through api (#3673) 2026-06-21 05:23:23 +08:00
安正超 08b8f58e64 refactor: prune notify ecstore object alias (#3620) 2026-06-19 20:11:56 +08:00
安正超 54ffbedbc8 refactor: remove ecstore operation compat facades (#3608) 2026-06-19 17:01:12 +08:00
安正超 b14e49e84e refactor: narrow IAM and Swift compatibility surfaces (#3587)
* refactor: narrow IAM and Swift compatibility surfaces

* refactor: narrow heal and scanner compatibility surfaces (#3588)

* refactor: narrow RustFS runtime compatibility surfaces (#3591)
2026-06-19 03:06:57 +08:00
安正超 205f964dc5 refactor: tighten storage compat store_api aliases (#3582)
* refactor: collapse storage compat store_api modules

* chore: guard storage compat store_api aliases
2026-06-18 22:50:02 +08:00
安正超 c28fee0013 refactor: continue storage api contract cleanup (#3580)
* refactor: move delete object contracts to storage api

* refactor: narrow store api compatibility exports

* refactor: route table catalog test through storage compat
2026-06-18 22:42:02 +08:00
安正超 7b0cb9e725 refactor: prune storage compatibility re-export allowances (#3579)
* refactor: prune storage compatibility re-export allowances

* fix: reject whitespace-padded dot path segments
2026-06-18 21:14:24 +08:00
安正超 f3fcdd4ba2 refactor: narrow remaining storage compatibility exports (#3578) 2026-06-18 19:18:32 +08:00
安正超 08371a6e09 refactor: narrow storage compatibility exports (#3576) 2026-06-18 18:31:14 +08:00
安正超 c098184c49 refactor: clean runtime and test storage boundaries (#3573)
* refactor: clean runtime observability select boundaries

* refactor: clean test harness fuzz storage boundaries
2026-06-18 18:09:25 +08:00
安正超 0e7e2660bb refactor: clean remaining storage DTO imports (#3568) 2026-06-18 14:44:46 +08:00
安正超 24443c829d refactor: clean shared list operation consumers (#3563) 2026-06-18 11:57:46 +08:00
安正超 3fb4cb3d65 refactor: move list operations contract (#3550) 2026-06-18 08:48:18 +08:00
安正超 919deeb816 refactor: move object option helper contracts (#3521) 2026-06-17 22:56:10 +08:00
cxymds 5b36ef5556 fix(decommission): persist progress adaptively (#3497)
Persist decommission progress after either the existing time interval or a migrated-item threshold, and flush progress baselines after bucket and terminal-state saves.

Also stabilize the OIDC discovery mock used by the pre-commit gate.
2026-06-17 08:17:59 +08:00
cxymds d094d91925 fix(site-replication): harden service account sync (#3500) 2026-06-16 21:20:13 +08:00
houseme e8012bd1ba refactor(logging): normalize admin telemetry and error messages (#3430) 2026-06-14 13:27:10 +08:00