perf(ecstore): parallelize bulk-delete lock and metadata fan-out (#9155)

* perf(ecstore): probe bulk-delete write locks in parallel waves

Bulk delete acquired one distributed write lock per key in serialized
order, paying a full quorum round-trip per key. Probe the unique keys in
bounded parallel waves with a short timeout instead; any contention drops
the whole wave and the canonical serial order resolves it as before, so
two overlapping batches can never deadlock.

* perf(ecstore): probe bulk-delete transaction locks in parallel

The set-layer delete-transaction locks were also acquired one quorum
round-trip per key after the raw write locks. Apply the same optimistic
all-or-nothing probe wave with a canonical serial fallback, and keep the
per-key error reporting the fallback already provided.

* perf(ecstore): pre-fetch bulk-delete source metadata in parallel

The admission loop ran one metadata quorum fan-out per key in sequence
on top of the per-key lock round-trips. Pre-fetch the source reads the
loop needs with bounded parallelism under the locks already held, and
derive the per-key read inputs from one shared helper so the pre-fetch
and the loop cannot disagree.

* test(ecstore): drop unused import in batch-delete probe test

* refactor(ecstore,lock): await wave releases and unify probe waves

Adversarial review of the bulk-delete lock waves found three real
issues. Distributed guard release is fire-and-forget on drop, so the
serial fallback could re-acquire keys while unlock RPCs were still in
flight and block on locks it already owned: guards are now released
with an awaited unlock before the fallback runs. The wave also probed
every key after the first failure, adding up to one probe timeout per
chunk of pure overhead under contention: probing now stops at the
first failing chunk. Finally the two copy-pasted wave implementations
are replaced by one shared probe_lock_wave, and the per-key delete
read inputs are resolved once instead of being recomputed.

Tests now assert the wave/fallback counters so they prove which path
ran, and an overlapping-batch test pins the no-deadlock property.

* refactor(ecstore,lock): batch bulk-delete lock rounds per key

Replace the optimistic probe waves with the bulk lock primitive the lock
layer already supports on the wire: one lock_batch RPC per lock client
for a whole DeleteObjects key set, per-key quorum guards with the usual
TTL heartbeat, and per-key outcomes. The batched round is deadlock-free
by construction (every lock server orders concurrent batches by the
same global (shard, key) order), so the probe timeout, the wave
release-and-wait dance, and the canonical serial fallback all go away.

Per-key lock outcomes are now reported per key end to end: a locked-out
key fails alone in the response while the rest of the batch proceeds,
matching the S3 DeleteObjects contract and the set layer's existing
behavior. The set layer's raw-write-lock and delete-transaction-lock
waves fold into the same primitive via one shared resource-folding
helper, and the fast-lock batch fast path now counts acquisitions so
shard metrics keep their meaning.

* perf(ecstore): move journal publish off the one-worker namespace lane

A bulk delete publishes one recovery transaction per key, and every
journal inspection and settle-plan closure ran on the per-disk
foreground namespace lane (one worker), serializing the whole publish
phase per drive behind every other mutation. Route those closures onto
the durable-mutation lane the fused delete prepare already uses (eight
workers per disk, fsyncs coalesce in the filesystem journal), and keep
32 journal publications in flight so a 16-drive set can feed those
lanes.

* test(ecstore): flatten lock-error iteration in overlap test

* fix(ecstore,lock): refresh duplicate keys after recovery, bound unlock

Review findings on the batched lock rounds. A batch entry whose recovery
rolls back an interrupted prepared delete can bring a retained version
back; a later duplicate of the same key still held its pre-fetched
source-missing read, skipped the object-lock check, and deleted the
restored version. Recovery now evicts the pre-fetched reads of every
other entry naming the same object, so those entries take the loop's
live re-read and see the restored version.

The batch rollback release awaited the full unlock retry schedule,
including the deferred background pass, parking the request behind an
unreachable peer. The synchronous phase is now a single release pass
with the deferred retries spawned in the background.

* fix(ecstore): release local batch locks before deferred recovery

A set-layer batch delete that runs deferred key recovery re-acquires a
write lock on the same keys through recover_delete_object, but the raw
local batch guards were only ever released at function exit — the
distributed path releases its batch ids before the deferred loop and the
local path missed the mirror release, so the recovery deadlocked against
the batch's own locks and surfaced as a spurious Lock(Timeout) for every
deferred key.

A regression test plants the interrupted prepared delete this needs and
pins the end-to-end contract: a batch naming a GOVERNANCE-retained
version twice must deny both entries after the first entry's recovery
rolls the prepared delete back, and the restored version must survive.

* test(lock): cover batch acquire partial-grant rollback

The distributed acquire_guard_batch rollback path — a key granted by
fewer than quorum clients must fail alone and release its partial grants
before the call returns — was only reachable in distributed mode and had
no coverage. A scripted per-object LockClient pins both halves: the
below-quorum key reports QuorumNotReached with its partial vote, and the
orphaned grant is released synchronously while the quorate key's guard
stays held.

.github/workflows/nightly-gnu.yml

@@ -87,18 +87,206 @@ jobs:
           install-build-packaging-tools: 'false'
           install-test-tools: 'false'
 
+      - name: Download static console assets
+        shell: bash
+        env:
+          GITHUB_TOKEN: ${{ github.token }}
+        run: |
+          # Packaging tools are off on this lane, so setup does not install unzip.
+          if ! command -v unzip >/dev/null 2>&1; then
+            sudo apt-get update -qq
+            sudo apt-get install -y -qq unzip
+          fi
+          if ! command -v unzip >/dev/null 2>&1; then
+            echo "unzip is required to extract console assets" >&2
+            exit 1
+          fi
+
+          mkdir -p ./rustfs/static
+
+          verify_sha256() {
+            local expected="$1"
+            local file="$2"
+            local actual=""
+            local actual_line=""
+
+            if command -v sha256sum >/dev/null 2>&1; then
+              actual_line=$(sha256sum -- "$file") || return 1
+            elif command -v shasum >/dev/null 2>&1; then
+              actual_line=$(shasum -a 256 -- "$file") || return 1
+            else
+              echo "No SHA-256 verification tool found" >&2
+              return 1
+            fi
+            actual="${actual_line%%[[:space:]]*}"
+
+            if [ "$actual" = "$expected" ]; then
+              echo "SHA256 verified OK: $actual"
+              return 0
+            else
+              echo "SHA256 mismatch: expected=$expected actual=$actual" >&2
+              return 1
+            fi
+          }
+
+          download_console_assets() {
+            local curl_bin="curl"
+            local python_bin="python3"
+            local console_api="https://api.github.com/repos/rustfs/console/releases/latest"
+            local console_json="console-release.json"
+            local console_url
+            local console_sha256
+            local curl_auth_args=()
+            console_tag=""
+
+            chmod +w ./rustfs/static/LICENSE || true
+
+            if ! command -v "$python_bin" >/dev/null 2>&1; then
+              python_bin="python"
+            fi
+            if ! command -v "$python_bin" >/dev/null 2>&1; then
+              echo "No Python interpreter found for release metadata parsing" >&2
+              return 1
+            fi
+
+            if [[ -n "${GITHUB_TOKEN:-}" ]]; then
+              curl_auth_args=(-H "Authorization: Bearer ${GITHUB_TOKEN}" -H "X-GitHub-Api-Version: 2022-11-28")
+            fi
+
+            "$curl_bin" "${curl_auth_args[@]}" --fail -L "$console_api" \
+              -o "$console_json" --retry 3 --retry-delay 5 --max-time 300 || return 1
+
+            read -r console_tag console_url console_sha256 < <("$python_bin" - "$console_json" <<'PY'
+          import json
+          import re
+          import sys
+
+          with open(sys.argv[1], encoding="utf-8") as handle:
+              release = json.load(handle)
+
+          tag = release.get("tag_name", "") or "unknown"
+
+          for asset in release.get("assets", []):
+              name = asset.get("name", "")
+              digest = asset.get("digest", "")
+              url = asset.get("browser_download_url", "")
+              if name.startswith("rustfs-console-") and name.endswith(".zip") and digest.startswith("sha256:") and url:
+                  sha256 = digest.split(":", 1)[1]
+                  if not re.fullmatch(r"[0-9a-fA-F]{64}", sha256):
+                      raise SystemExit(f"console zip asset has invalid sha256 digest: {sha256}")
+                  sys.stdout.buffer.write(f"{tag} {url} {sha256}\n".encode("utf-8"))
+                  break
+          else:
+              raise SystemExit("no console zip asset with sha256 digest found")
+          PY
+            ) || return 1
+
+            if [[ -z "$console_url" || -z "$console_sha256" ]]; then
+              echo "Console release metadata is missing URL or SHA-256 digest" >&2
+              return 1
+            fi
+
+            echo "Console release: ${console_tag}"
+            echo "Downloading console asset: ${console_url}"
+            "$curl_bin" --fail -L "$console_url" -o console.zip --retry 3 --retry-delay 5 --max-time 300 || return 1
+            verify_sha256 "$console_sha256" console.zip || return 2
+            unzip -o console.zip -d ./rustfs/static || return 2
+          }
+
+          if download_console_assets; then
+            rm -f console.zip console-release.json
+          else
+            status=$?
+            rm -f console.zip console-release.json
+            if [[ "$status" -eq 2 ]]; then
+              echo "Console asset integrity verification failed" >&2
+            fi
+            echo "Failed to download verified console assets" >&2
+            exit 1
+          fi
+
+          if [[ ! -s ./rustfs/static/index.html ]]; then
+            echo "Console asset archive is missing static/index.html" >&2
+            exit 1
+          fi
+
+          asset_count=$(find ./rustfs/static -type f | wc -l | tr -d '[:space:]')
+          echo "Console assets ready: version=${console_tag:-unknown}, ${asset_count} files extracted to ./rustfs/static"
+
       - name: Build RustFS
         # Pin the glibc baseline (`<target>.<glibc>` for cargo-zigbuild) so
         # nightlies run on older distros such as RHEL/Rocky 9 (glibc 2.34); #8062.
-        run: cargo zigbuild --release --locked --target x86_64-unknown-linux-gnu.2.31 -p rustfs --bins
+        shell: bash
+        run: |
+          # Force rebuild by touching build.rs so a restored cache cannot
+          # reuse a rustfs artifact compiled before these console assets existed.
+          touch rustfs/build.rs
+          cargo zigbuild --release --locked --target x86_64-unknown-linux-gnu.2.31 -p rustfs --bin rustfs
 
       - name: Verify glibc symbol baseline
         run: |
           python3 scripts/release/check_glibc.py 2.31 \
-            target/x86_64-unknown-linux-gnu/release/rustfs \
-            target/x86_64-unknown-linux-gnu/release/rustfs-cli
+            target/x86_64-unknown-linux-gnu/release/rustfs
           target/x86_64-unknown-linux-gnu/release/rustfs --version | grep -Fx 'build target : x86_64-unknown-linux-gnu'
 
+      - name: Verify console
+        shell: bash
+        run: |
+          set -euo pipefail
+
+          find_free_port() {
+            python3 - <<'PY'
+          import socket
+
+          with socket.socket() as sock:
+              sock.bind(("127.0.0.1", 0))
+              print(sock.getsockname()[1])
+          PY
+          }
+
+          binary="target/x86_64-unknown-linux-gnu/release/rustfs"
+          data_dir="$(mktemp -d)"
+          log_file="${RUNNER_TEMP}/rustfs-console-smoke.log"
+          server_pid=""
+          trap '
+            if [[ -n "${server_pid:-}" ]]; then
+              kill "$server_pid" 2>/dev/null || true
+              wait "$server_pid" 2>/dev/null || true
+            fi
+            rm -rf "$data_dir"
+          ' EXIT
+
+          if [[ ! -x "$binary" ]]; then
+            echo "rustfs binary not found: ${binary}" >&2
+            exit 1
+          fi
+
+          api_port="$(find_free_port)"
+          console_port="$(find_free_port)"
+          console_url="http://127.0.0.1:${console_port}/rustfs/console/"
+
+          "$binary" server \
+            --address "127.0.0.1:${api_port}" \
+            --console-enable \
+            --console-address "127.0.0.1:${console_port}" \
+            --access-key console-smoke \
+            --secret-key console-smoke-secret \
+            "$data_dir" >"$log_file" 2>&1 &
+          server_pid=$!
+
+          response=""
+          for _ in {1..40}; do
+            response="$(curl --silent --output /dev/null --write-out '%{http_code} %{content_type}' "$console_url" || true)"
+            if [[ "$response" == 200\ text/html* ]]; then
+              exit 0
+            fi
+            sleep 0.25
+          done
+
+          echo "Console endpoint did not return 200 text/html: $response" >&2
+          cat "$log_file"
+          exit 1
+
       - name: Build DEB package
         id: deb
         shell: bash

Cargo.lock

@@ -4731,7 +4731,6 @@ version = "1.4.5"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "337d46834ee672ab3e48caca2cb0c78cc174fb12b3a68d0d88f99a0519a5e36e"
 dependencies = [
- "generic-array 0.14.7",
  "rustversion",
  "typenum",
 ]
@@ -9349,9 +9348,9 @@ dependencies = [
 
 [[package]]
 name = "russh"
-version = "0.65.0"
+version = "0.65.1"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "3bb77a4b259312b2abbc541bca1c13cb4e9b61c560a7fd894068c3b92e0c9dc3"
+checksum = "94be3d86c71b9ed220a675c405e35c7a02565a009bf8da2e9cc13a700320a5c5"
 dependencies = [
  "aes 0.9.4",
  "aws-lc-rs",
@@ -10115,9 +10114,9 @@ dependencies = [
 
 [[package]]
 name = "rustfs-kafka"
-version = "1.4.0"
+version = "1.4.1"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "bd2f8b46ec4dd462c2235948d1efa2103957230c171e118c6ff8e5d1a845ab48"
+checksum = "82dc39ba1a5a9f6b2e49decb0b54c23ebaebdfb08a899307c266694bfd269bf9"
 dependencies = [
  "base64 0.23.1",
  "bytes",
@@ -10140,9 +10139,9 @@ dependencies = [
 
 [[package]]
 name = "rustfs-kafka-async"
-version = "1.4.0"
+version = "1.4.1"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "4b3b98705f6f9f5071b8793d29935de764c31668c6e9877553b229eb67127af6"
+checksum = "684f5e78ea1421bbf915e4cd0515bdefd587cfa281da000e41ae4a434f90195a"
 dependencies = [
  "base64 0.23.1",
  "bytes",
@@ -10535,6 +10534,7 @@ dependencies = [
  "md-5 0.11.0",
  "percent-encoding",
  "proptest",
+ "rcgen",
  "regex",
  "russh",
  "russh-sftp",
@@ -11024,6 +11024,7 @@ version = "1.0.2"
 dependencies = [
  "async-trait",
  "axum",
+ "futures",
  "hotpath",
  "http 1.5.0",
  "ipnetwork",
@@ -11199,15 +11200,14 @@ dependencies = [
 
 [[package]]
 name = "rustls-connector"
-version = "0.23.9"
+version = "0.23.10"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "524994fd6957827c775abc941f41a146ce351bbd0a9b8cf632ee37f43a13c205"
+checksum = "f799ef81c26fda57609b9456ae1e6750e2ca3dafc2dc5478232c0705f1458fbf"
 dependencies = [
  "futures-io",
  "futures-rustls",
  "log",
  "rustls",
- "rustls-pki-types",
  "rustls-platform-verifier",
  "rustls-webpki",
 ]
@@ -12858,9 +12858,9 @@ dependencies = [
 
 [[package]]
 name = "toml"
-version = "1.1.8+spec-1.1.0"
+version = "1.2.0+spec-1.1.0"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "20489e00e4d8741d6be680764cc12e270655e375a20d1011e844a9c3379e678d"
+checksum = "c5ffcdf8f50436dbbfd675c1fe1b8e0854f42b7defea8584bdebe1aee900e561"
 dependencies = [
  "indexmap 2.14.2",
  "serde_core",
@@ -12882,9 +12882,9 @@ dependencies = [
 
 [[package]]
 name = "toml_edit"
-version = "0.25.17+spec-1.1.0"
+version = "0.25.18+spec-1.1.0"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "e3641d5bbb5349a79e1020a242d251efbc546ad8048d133958323ce9c40a9c9c"
+checksum = "1ec6e78bd70db9420d8d212fa9dc97f116448a1ceff5d00eabb991d407e21396"
 dependencies = [
  "indexmap 2.14.2",
  "toml_datetime",
@@ -12903,9 +12903,9 @@ dependencies = [
 
 [[package]]
 name = "toml_writer"
-version = "1.1.3+spec-1.1.0"
+version = "1.2.0+spec-1.1.0"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "06bdbd8cfc056b8d2e2e85f29b56a3bdbecb527cef81eb39e3e7b98af4652770"
+checksum = "009dbb7700f11c55ef83aebbe4c0e9cf1f1ac0844bd8d6f00dc38f302026e13a"
 
 [[package]]
 name = "tonic"

Cargo.toml

@@ -168,7 +168,7 @@ http-body-util = "0.1.5"
 memchr = "2.8.3"
 minlz = "1.2.3"
 reqwest = "0.13.5"
-rustfs-kafka-async = { version = "1.4.0" }
+rustfs-kafka-async = { version = "1.4.1" }
 socket2 = { version = "0.6.5" }
 tokio = { version = "1.54.1" }
 tokio-rustls = { default-features = false, version = "0.26.6" }
@@ -212,7 +212,7 @@ serde_urlencoded = "0.7.1"
 # releases.
 aes-gcm = { version = "0.11.1" }
 argon2 = { version = "0.6.0" }
-aws-lc-rs = { version = "1.18.0", default-features = false }
+aws-lc-rs = { version = "1.19.0", default-features = false }
 blake2 = "0.11.0"
 chacha20poly1305 = { version = "0.11.0" }
 crc-fast = "1.10.0"
@@ -378,7 +378,7 @@ libunftp = { version = "0.24.0" }
 unftp-core = "0.1.0"
 suppaftp = { version = "12.2.1" }
 rcgen = { version = "0.14.10", default-features = false, features = ["aws_lc_rs", "crypto", "pem"] }
-russh = { version = "0.65.0" }
+russh = { version = "0.65.1" }
 russh-sftp = "3.0.1"
 
 # WebDAV

crates/config/src/constants/proxy.rs

@@ -30,10 +30,10 @@ pub const ENV_TRUSTED_PROXY_VALIDATION_MODE: &str = "RUSTFS_TRUSTED_PROXY_VALIDA
 /// Default validation mode is "hop_by_hop".
 pub const DEFAULT_TRUSTED_PROXY_VALIDATION_MODE: &str = "hop_by_hop";
 
-/// Environment variable to enable RFC 7239 "Forwarded" header support.
+/// Select RFC 7239 "Forwarded" instead of "X-Forwarded-For" in legacy mode.
 pub const ENV_TRUSTED_PROXY_ENABLE_RFC7239: &str = "RUSTFS_TRUSTED_PROXY_ENABLE_RFC7239";
-/// RFC 7239 support is enabled by default.
-pub const DEFAULT_TRUSTED_PROXY_ENABLE_RFC7239: bool = true;
+/// Legacy mode uses X-Forwarded-For unless RFC 7239 is explicitly selected.
+pub const DEFAULT_TRUSTED_PROXY_ENABLE_RFC7239: bool = false;
 
 /// Environment variable for the maximum allowed proxy hops.
 pub const ENV_TRUSTED_PROXY_MAX_HOPS: &str = "RUSTFS_TRUSTED_PROXY_MAX_HOPS";

crates/config/src/constants/targets.rs

@@ -146,3 +146,9 @@ pub const ENV_TARGET_STORE_COMPRESS: &str = "RUSTFS_TARGET_STORE_COMPRESS";
 
 /// Queue-store compression is enabled by default to reduce disk footprint.
 pub const DEFAULT_TARGET_STORE_COMPRESS: bool = true;
+
+/// Maximum on-disk bytes in each target queue, including failed entries and pending writes.
+pub const ENV_TARGET_STORE_MAX_BYTES: &str = "RUSTFS_TARGET_STORE_MAX_BYTES";
+
+/// Queue writes fail at one GiB unless the operator sets another positive byte budget.
+pub const DEFAULT_TARGET_STORE_MAX_BYTES: u64 = 1024 * 1024 * 1024;

crates/ecstore/src/disk/fd_cache_tests.rs

@@ -183,3 +183,459 @@ async fn uring_fd_cache_invalidation_keeps_borrowed_old_inode_alive_and_allows_r
         assert_eq!(fresh.len, 11);
     }
 }
+
+#[tokio::test]
+async fn fd_cache_tracker_skips_unmatched_invalidations_and_handles_deep_prefixes() {
+    let (_directory, file) = fixture();
+    let mut cache = FdCache::new();
+
+    // Unmatched invalidations on an empty cache must not spawn the maintenance task.
+    cache.invalidate_under("v", "missing/prefix");
+    cache.invalidate_volume("v");
+    cache.invalidate_exact("v", "missing/prefix/part.1").await;
+    cache.clear();
+    assert!(
+        cache.maintenance_task.get().is_none(),
+        "empty-cache invalidations must not spawn maintenance"
+    );
+
+    // Populate a normal entry and verify matched vs unmatched filter gates directly,
+    // and ensure unmatched invalidations do not enqueue pending removals.
+    let cached_key = key("v", "obj-1/0198a3f0/part.1", false);
+    cache.insert(cached_key.clone(), Arc::clone(&file)).await;
+    assert_eq!(cache.tracker.live_entries(), 1);
+    assert!(cache.tracker.may_match_volume("v"));
+    assert!(!cache.tracker.may_match_volume("other-volume"));
+    assert!(cache.tracker.may_match_prefix("v", "obj-1"));
+    assert!(cache.tracker.may_match_prefix("v", "obj-1/0198a3f0"));
+    assert!(!cache.tracker.may_match_prefix("v", "obj-2/0198a3f0"));
+    assert!(cache.tracker.may_match_exact("v", "obj-1/0198a3f0/part.1"));
+    assert!(!cache.tracker.may_match_exact("v", "obj-1/0198a3f0/xl.meta"));
+
+    cache.invalidate_volume("other-volume");
+    cache.invalidate_under("v", "obj-2/0198a3f0");
+    cache.invalidate_exact("v", "obj-1/0198a3f0/xl.meta").await;
+    assert!(
+        !cache
+            .tracker
+            .has_pending_invalidations
+            .load(std::sync::atomic::Ordering::SeqCst),
+        "unmatched invalidations must not enqueue maintenance removals"
+    );
+    assert!(cache.get(&cached_key).await.is_some());
+
+    // Deeply nested path (> FD_CACHE_MAX_INDEXED_PREFIXES slashes) must fall back
+    // to per-volume unindexed_counts without degrading other volumes.
+    let other_vol_key = key("v2", "shallow/part.1", false);
+    cache.insert(other_vol_key.clone(), Arc::clone(&file)).await;
+    let deep_path = "c1/c2/c3/c4/c5/c6/c7/c8/c9/c10/c11/c12/c13/c14/part.1";
+    let deep_key = key("v", deep_path, false);
+    cache.insert(deep_key.clone(), Arc::clone(&file)).await;
+    assert_eq!(
+        cache
+            .tracker
+            .unindexed_counts
+            .iter()
+            .map(|c| c.load(std::sync::atomic::Ordering::Acquire))
+            .sum::<u32>(),
+        1
+    );
+    assert!(
+        !cache.tracker.may_match_prefix("v2", "unmatched/prefix"),
+        "deep key in volume v must not disable prefix filtering in volume v2"
+    );
+    cache.invalidate_under("v", "c1/c2/c3/c4/c5/c6/c7/c8/c9/c10/c11/c12/c13");
+    assert!(cache.get(&deep_key).await.is_none());
+    cache.invalidate_volume("v2");
+    assert!(cache.get(&other_vol_key).await.is_none());
+
+    // Replacing an invalidated key before maintenance drains pending_invalidations
+    // must release the old descriptor immediately and keep the replacement live
+    // even when a strong reference to the old CachedFdEntry shell survives until
+    // maintenance runs its Arc::ptr_eq check.
+    let old_file = Arc::new(tempfile::tempfile().expect("create replaced descriptor"));
+    let old_weak = Arc::downgrade(&old_file);
+    cache.insert(cached_key.clone(), old_file).await;
+    let held_old_shell = cache.cache.get(&cached_key).expect("cached shell must exist");
+    cache.invalidate_under("v", "obj-1/0198a3f0");
+    assert!(cache.get(&cached_key).await.is_none());
+    cache.insert(cached_key.clone(), Arc::clone(&file)).await;
+    assert!(
+        old_weak.upgrade().is_none(),
+        "replacing an entry must retire and drop the previous descriptor immediately"
+    );
+    tokio::time::timeout(std::time::Duration::from_secs(5), async {
+        while cache
+            .tracker
+            .has_pending_invalidations
+            .load(std::sync::atomic::Ordering::SeqCst)
+        {
+            tokio::task::yield_now().await;
+        }
+    })
+    .await
+    .expect("maintenance should drain pending invalidations");
+    drop(held_old_shell);
+    assert!(
+        cache.get(&cached_key).await.is_some(),
+        "stale pending invalidation must not evict a newer replacement"
+    );
+    assert_eq!(
+        cache
+            .tracker
+            .unindexed_counts
+            .iter()
+            .map(|c| c.load(std::sync::atomic::Ordering::Acquire))
+            .sum::<u32>(),
+        0
+    );
+
+    // Dropping a cache while a live non-invalidated entry is present (even if a
+    // cloned Moka handle outlives FdCache and the maintenance task is aborted)
+    // must retire the descriptor and release FdCacheTracker without cycles.
+    let live_drop_file = Arc::new(tempfile::tempfile().expect("create live drop descriptor"));
+    let live_drop_weak = Arc::downgrade(&live_drop_file);
+    let shutdown_file = Arc::new(tempfile::tempfile().expect("create shutdown descriptor"));
+    let shutdown_weak = Arc::downgrade(&shutdown_file);
+    let tracker_weak = Arc::downgrade(&cache.tracker);
+    let cloned_moka = cache.cache.clone();
+    cache.insert(key("v", "live-on-drop/part.1", false), live_drop_file).await;
+    cache.insert(key("v", "shutdown/part.1", false), shutdown_file).await;
+    cache.invalidate_under("v", "shutdown");
+    if let Some(handle) = cache.maintenance_task.take() {
+        handle.abort();
+        let _ = handle.await;
+    }
+    drop(cache);
+    assert!(
+        live_drop_weak.upgrade().is_none(),
+        "Drop for FdCache must retire live descriptors even if a cloned Moka handle survives"
+    );
+    assert!(
+        shutdown_weak.upgrade().is_none(),
+        "dropping cache when maintenance is aborted must still release descriptors"
+    );
+    assert!(
+        tracker_weak.upgrade().is_none(),
+        "dropping cache must release FdCacheTracker without reference cycles"
+    );
+    drop(cloned_moka);
+
+    // Short-TTL expired entries (including mixed expired and fresh entries) must
+    // be retired by invalidate_under and clear() even though Moka's iterator skips
+    // expired entries.
+    let ttl_cache = FdCache::with_ttl(std::time::Duration::from_millis(15));
+    let expired_file = Arc::new(tempfile::tempfile().expect("create short-ttl descriptor"));
+    let expired_weak = Arc::downgrade(&expired_file);
+    ttl_cache.insert(key("v", "ttl-obj/part.1", false), expired_file).await;
+    tokio::time::sleep(std::time::Duration::from_millis(30)).await;
+    let fresh_file = Arc::new(tempfile::tempfile().expect("create fresh descriptor"));
+    let fresh_weak = Arc::downgrade(&fresh_file);
+    ttl_cache.insert(key("v", "ttl-obj/part.2", false), fresh_file).await;
+    ttl_cache.invalidate_under("v", "ttl-obj");
+    tokio::time::timeout(std::time::Duration::from_secs(5), async {
+        while expired_weak.upgrade().is_some() || fresh_weak.upgrade().is_some() {
+            tokio::task::yield_now().await;
+        }
+    })
+    .await
+    .expect("invalidate_under must retire both expired and fresh matching descriptors");
+    assert_eq!(ttl_cache.tracker.live_entries(), 0);
+
+    let clear_expired_file = Arc::new(tempfile::tempfile().expect("create clear short-ttl descriptor"));
+    let clear_expired_weak = Arc::downgrade(&clear_expired_file);
+    ttl_cache
+        .insert(key("v", "ttl-clear/part.1", false), clear_expired_file)
+        .await;
+    tokio::time::sleep(std::time::Duration::from_millis(30)).await;
+    let clear_live_file = Arc::new(tempfile::tempfile().expect("create clear live descriptor"));
+    let clear_live_weak = Arc::downgrade(&clear_live_file);
+    ttl_cache.insert(key("v", "ttl-clear/part.2", false), clear_live_file).await;
+    ttl_cache.clear();
+    assert!(
+        clear_expired_weak.upgrade().is_none() && clear_live_weak.upgrade().is_none(),
+        "clear() must synchronously retire both TTL-expired and live descriptors"
+    );
+    assert_eq!(ttl_cache.tracker.live_entries(), 0);
+}
+
+#[tokio::test]
+async fn fd_cache_lockstep_oracle_equivalence_and_counter_conservation() {
+    use std::collections::HashMap;
+    use std::sync::atomic::Ordering;
+
+    let (_directory, file) = fixture();
+    let cache = FdCache::with_ttl(std::time::Duration::from_secs(3600));
+    let mut oracle: HashMap<FdKey, Arc<FdCacheEntry>> = HashMap::new();
+
+    let volumes = ["vol-a", "vol-b", "vol-c", ".rustfs.sys/multipart"];
+    let paths = [
+        "a/b",
+        "a/b/part.1",
+        "a/b/dir/part.2",
+        "a/bc",
+        "a/bc/part.1",
+        "prefix/obj-0001.bin/0198a3f0/part.1",
+        "prefix/obj-0002.bin/0198a3f0/part.1",
+        "d1/d2/d3/d4/d5/d6/d7/d8/d9/d10/d11/d12/d13/part.1",
+    ];
+    let prefixes = [
+        "a",
+        "a/",
+        "a/b",
+        "a/b/",
+        "a/b/dir",
+        "a/bc",
+        "prefix",
+        "prefix/obj-0001.bin",
+        "prefix/obj-0001.bin/0198a3f0",
+        "d1/d2/d3/d4/d5/c_missing",
+        "d1/d2/d3/d4/d5/d6/d7/d8/d9/d10/d11/d12/d13",
+        "nonexistent/prefix",
+    ];
+
+    let mut candidate_keys = Vec::new();
+    for v in volumes {
+        for p in paths {
+            for direct in [false, true] {
+                candidate_keys.push(key(v, p, direct));
+            }
+        }
+    }
+
+    let mut rng = 0x9e37_79b9_7f4a_7c15u64;
+    let mut next_u64 = || {
+        rng ^= rng << 13;
+        rng ^= rng >> 7;
+        rng ^= rng << 17;
+        rng
+    };
+
+    for step in 0..1000 {
+        let op = next_u64() % 100;
+        if op < 45 {
+            // Insert or replace (either fresh or stale generation)
+            let k = candidate_keys[(next_u64() as usize) % candidate_keys.len()].clone();
+            let entry = Arc::new(FdCacheEntry {
+                file: Arc::clone(&file),
+                len: step,
+                invalidated: AtomicBool::new(false),
+            });
+            let cur_gen = cache.generation();
+            let stale = (next_u64() % 5) == 0 && cur_gen > 0;
+            let gen_to_use = if stale { cur_gen - 1 } else { cur_gen };
+            cache.insert_if_fresh(k.clone(), Arc::clone(&entry), gen_to_use).await;
+            if !stale {
+                oracle.insert(k, entry);
+            }
+        } else if op < 65 {
+            // Exact invalidation
+            let v = volumes[(next_u64() as usize) % volumes.len()];
+            let p = paths[(next_u64() as usize) % paths.len()];
+            cache.invalidate_exact(v, p).await;
+            oracle.remove(&key(v, p, false));
+            oracle.remove(&key(v, p, true));
+        } else if op < 88 {
+            // Prefix invalidation
+            let v = volumes[(next_u64() as usize) % volumes.len()];
+            let raw_prefix = prefixes[(next_u64() as usize) % prefixes.len()];
+            cache.invalidate_under(v, raw_prefix);
+            let trimmed = raw_prefix.trim_end_matches('/');
+            oracle.retain(|k, _| {
+                !(k.volume == v && (k.path == trimmed || k.path.strip_prefix(trimmed).is_some_and(|r| r.starts_with('/'))))
+            });
+        } else if op < 97 {
+            // Volume invalidation
+            let v = volumes[(next_u64() as usize) % volumes.len()];
+            cache.invalidate_volume(v);
+            oracle.retain(|k, _| k.volume != v);
+        } else {
+            // Full clear
+            cache.clear();
+            oracle.clear();
+        }
+
+        // Verify lockstep equivalence against the reference oracle after every operation.
+        for k in &candidate_keys {
+            let got = cache.get(k).await;
+            let expected = oracle.get(k);
+            match (got, expected) {
+                (Some(g), Some(e)) => assert!(
+                    Arc::ptr_eq(&g, e),
+                    "step {step}: pointer mismatch for key ({}, {}, {})",
+                    k.volume,
+                    k.path,
+                    k.direct
+                ),
+                (None, None) => {}
+                (g, e) => panic!(
+                    "step {step}: oracle mismatch for key ({}, {}, {}): got={}, expected={}",
+                    k.volume,
+                    k.path,
+                    k.direct,
+                    g.is_some(),
+                    e.is_some()
+                ),
+            }
+        }
+    }
+
+    // Clear and wait for maintenance to drain all entries, then verify exact counter conservation.
+    cache.clear();
+    tokio::time::timeout(std::time::Duration::from_secs(5), async {
+        while cache.tracker.live_entries() != 0 || cache.tracker.has_pending_invalidations.load(Ordering::SeqCst) {
+            tokio::task::yield_now().await;
+        }
+    })
+    .await
+    .expect("clear must drain all live entries and pending invalidations");
+
+    assert_eq!(cache.tracker.live_entries(), 0);
+    assert!(cache.tracker.unindexed_counts.iter().all(|c| c.load(Ordering::SeqCst) == 0));
+    assert!(cache.tracker.volume_counts.iter().all(|c| c.load(Ordering::SeqCst) == 0));
+    assert!(cache.tracker.exact_counts.iter().all(|c| c.load(Ordering::SeqCst) == 0));
+    assert!(cache.tracker.prefix_counts.iter().all(|c| c.load(Ordering::SeqCst) == 0));
+    assert!(cache.tracker.prefix_shards.iter().all(|s| s.lock().is_empty()));
+    assert!(cache.tracker.entry_shards.iter().all(|s| s.lock().is_empty()));
+}
+
+#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
+async fn fd_cache_concurrent_insert_and_invalidation_stress() {
+    use std::sync::atomic::{AtomicU64, Ordering};
+
+    let (_directory, file) = fixture();
+    let cache = Arc::new(FdCache::new());
+
+    // Deterministic Dekker-partner interleaving check: pause `CachedFdEntry::new`
+    // at `entry_shards[shard].lock()` after it has passed the initial generation
+    // check and incremented `volume_counts`, bump generation via an unmatched
+    // invalidation, and verify the post-`new()` recheck refuses publication and
+    // preserves the existing entry.
+    let seam_key = key("v-seam", "obj-1/part.1", false);
+    cache.insert(seam_key.clone(), Arc::clone(&file)).await;
+    let initial_entry = cache.get(&seam_key).await.expect("initial entry must exist");
+    let (vol_hash, vol_slot) = super::fd_cache_volume_hash_and_slot("v-seam");
+    let entry_shard = (vol_hash as usize) & (super::FD_CACHE_PREFIX_SHARDS - 1);
+    let stale_gen = cache.generation();
+    let stale_entry = Arc::new(FdCacheEntry {
+        file: Arc::clone(&file),
+        len: 999_999,
+        invalidated: AtomicBool::new(false),
+    });
+    let blocked_insert = {
+        let hold = cache.tracker.entry_shards[entry_shard].lock();
+        let cache_clone = Arc::clone(&cache);
+        let seam_key_clone = seam_key.clone();
+        let task = tokio::spawn(async move {
+            cache_clone.insert_if_fresh(seam_key_clone, stale_entry, stale_gen).await;
+        });
+        let deadline = std::time::Instant::now() + std::time::Duration::from_secs(5);
+        while cache.tracker.volume_counts[usize::from(vol_slot)].load(Ordering::SeqCst) < 2 {
+            assert!(
+                std::time::Instant::now() < deadline,
+                "insert never reached CachedFdEntry::new with the previous entry still live"
+            );
+            std::thread::yield_now();
+        }
+        cache.invalidate_under("v-seam", "unmatched-prefix");
+        drop(hold);
+        task
+    };
+    blocked_insert.await.expect("blocked insert task should complete");
+    let preserved = cache
+        .get(&seam_key)
+        .await
+        .expect("refused stale insert must not retire a valid previous entry");
+    assert!(Arc::ptr_eq(&preserved, &initial_entry));
+    cache.invalidate_volume("v-seam");
+
+    let floor_v1 = Arc::new(AtomicU64::new(0));
+    let floor_v2 = Arc::new(AtomicU64::new(0));
+    let mut handles = Vec::new();
+
+    for t in 0..8usize {
+        let cache = Arc::clone(&cache);
+        let file = Arc::clone(&file);
+        let floor_v1 = Arc::clone(&floor_v1);
+        let floor_v2 = Arc::clone(&floor_v2);
+        handles.push(tokio::spawn(async move {
+            for i in 0..250u64 {
+                let is_v1 = (t + i as usize).is_multiple_of(2);
+                let vol = if is_v1 { "v1" } else { "v2" };
+                let path = format!("dir-{}/obj-{}/part.1", t % 4, i % 16);
+                let k = key(vol, &path, i.is_multiple_of(2));
+                let g = cache.generation();
+                let entry = Arc::new(FdCacheEntry {
+                    file: Arc::clone(&file),
+                    len: g,
+                    invalidated: AtomicBool::new(false),
+                });
+                cache.insert_if_fresh(k.clone(), entry, g).await;
+                let min_gen = if is_v1 {
+                    floor_v1.load(Ordering::SeqCst)
+                } else {
+                    floor_v2.load(Ordering::SeqCst)
+                };
+                if let Some(hit) = cache.get(&k).await {
+                    assert!(
+                        hit.len >= min_gen,
+                        "stale descriptor served after volume invalidation: opened_gen={}, floor_gen={min_gen}",
+                        hit.len
+                    );
+                }
+            }
+        }));
+    }
+
+    for t in 0..4usize {
+        let cache = Arc::clone(&cache);
+        let floor_v1 = Arc::clone(&floor_v1);
+        let floor_v2 = Arc::clone(&floor_v2);
+        handles.push(tokio::spawn(async move {
+            for i in 0..250u64 {
+                let is_v1 = (t + i as usize).is_multiple_of(2);
+                let vol = if is_v1 { "v1" } else { "v2" };
+                match i % 4 {
+                    0 => cache.invalidate_under(vol, &format!("dir-{}/obj-{}", t % 4, i % 16)),
+                    1 => {
+                        let p = format!("dir-{}/obj-{}/part.1", t % 4, i % 16);
+                        cache.invalidate_exact(vol, &p).await;
+                    }
+                    2 => {
+                        let before = cache.generation();
+                        cache.invalidate_volume(vol);
+                        let floor = before + 1;
+                        if is_v1 {
+                            floor_v1.fetch_max(floor, Ordering::SeqCst);
+                        } else {
+                            floor_v2.fetch_max(floor, Ordering::SeqCst);
+                        }
+                    }
+                    _ => cache.invalidate_under("uncached-vol", "uncached/prefix"),
+                }
+                tokio::task::yield_now().await;
+            }
+        }));
+    }
+
+    for h in handles {
+        h.await.expect("worker task should succeed");
+    }
+
+    cache.clear();
+    tokio::time::timeout(std::time::Duration::from_secs(5), async {
+        while cache.tracker.live_entries() != 0 || cache.tracker.has_pending_invalidations.load(Ordering::SeqCst) {
+            tokio::task::yield_now().await;
+        }
+    })
+    .await
+    .expect("clear after concurrent stress must drain all live entries");
+
+    assert_eq!(cache.tracker.live_entries(), 0);
+    assert!(cache.tracker.unindexed_counts.iter().all(|c| c.load(Ordering::SeqCst) == 0));
+    assert!(cache.tracker.volume_counts.iter().all(|c| c.load(Ordering::SeqCst) == 0));
+    assert!(cache.tracker.exact_counts.iter().all(|c| c.load(Ordering::SeqCst) == 0));
+    assert!(cache.tracker.prefix_counts.iter().all(|c| c.load(Ordering::SeqCst) == 0));
+    assert!(cache.tracker.prefix_shards.iter().all(|s| s.lock().is_empty()));
+    assert!(cache.tracker.entry_shards.iter().all(|s| s.lock().is_empty()));
+}

crates/ecstore/src/disk/local.rs

@@ -5710,6 +5710,401 @@ struct FdCacheEntry {
     invalidated: AtomicBool,
 }
 
+#[cfg(any(target_os = "linux", all(unix, test)))]
+const FD_CACHE_VOLUME_FILTER_SLOTS: usize = 256;
+#[cfg(any(target_os = "linux", all(unix, test)))]
+const FD_CACHE_EXACT_FILTER_SLOTS: usize = 8192;
+#[cfg(any(target_os = "linux", all(unix, test)))]
+const FD_CACHE_PREFIX_FILTER_SLOTS: usize = 16384;
+#[cfg(any(target_os = "linux", all(unix, test)))]
+const FD_CACHE_PREFIX_SHARDS: usize = 64;
+#[cfg(any(target_os = "linux", all(unix, test)))]
+const FD_CACHE_MAX_INDEXED_PREFIXES: usize = 12;
+
+#[cfg(any(target_os = "linux", all(unix, test)))]
+#[inline]
+fn fd_cache_volume_hash_and_slot(volume: &str) -> (u64, u8) {
+    let hash = xxhash_rust::xxh3::xxh3_64(volume.as_bytes());
+    (hash, (hash as usize & (FD_CACHE_VOLUME_FILTER_SLOTS - 1)) as u8)
+}
+
+#[cfg(any(target_os = "linux", all(unix, test)))]
+#[inline]
+fn fd_cache_hash_and_quad_slots(volume_hash: u64, text: &str, slots: usize) -> (u64, [u16; 4]) {
+    let h = xxhash_rust::xxh3::xxh3_64_with_seed(text.as_bytes(), volume_hash);
+    let mask = (slots - 1) as u16;
+    (
+        h,
+        [
+            (h as u16) & mask,
+            ((h >> 16) as u16) & mask,
+            ((h >> 32) as u16) & mask,
+            ((h >> 48) as u16) & mask,
+        ],
+    )
+}
+
+#[cfg(any(target_os = "linux", all(unix, test)))]
+type FdTrackerEntryList = Vec<(u64, Arc<FdKey>, std::sync::Weak<CachedFdEntry>)>;
+
+/// Tracks active descriptors and their volume/prefix hashes so unmatched
+/// `invalidate_under`, `invalidate_volume`, and `invalidate_exact` calls return
+/// without scanning `moka` or waking the background maintenance task.
+///
+/// Memory-ordering invariant:
+/// - `CachedFdEntry::new` increments all volume/exact/prefix counters and then,
+///   while holding `entry_shards[entry_shard]`, executes
+///   `live_entries.fetch_add(1, Ordering::SeqCst)` before pushing the entry into
+///   `entry_shards`. No thread can discover or retire the entry before it is counted.
+/// - `insert_if_fresh` rechecks `generation.load(Ordering::SeqCst)` after
+///   `CachedFdEntry::new` (`live_entries++`) and before publishing the entry into
+///   `moka`, retiring and aborting on mismatch.
+/// - Every `invalidate_*` bumps `generation.fetch_add(1, Ordering::SeqCst)` and then
+///   calls `may_match_*`, which executes `live_entries.load(Ordering::SeqCst)` as its
+///   first load. This Dekker `SeqCst` pair guarantees that an invalidation and a
+///   concurrent `insert_if_fresh` cannot both miss each other.
+///
+/// Lock acquisition order:
+/// 1. `invalidation_locks[i]` (outermost per-key shard lock, serializes cache insert vs removal)
+/// 2. `CachedFdEntry::state` (per-entry RwLock)
+/// 3. `prefix_shards[i]` and `entry_shards[i]` (leaf locks, acquired inside `CachedFdEntry::new` and `CachedFdLiveState::drop`)
+///
+/// `pending_invalidations` is an independent leaf lock and is never held while
+/// acquiring `invalidation_locks`, `state`, `prefix_shards`, or `entry_shards`.
+#[cfg(any(target_os = "linux", all(unix, test)))]
+struct FdCacheTracker {
+    live_entries: std::sync::atomic::AtomicUsize,
+    next_entry_id: std::sync::atomic::AtomicU64,
+    has_pending_invalidations: AtomicBool,
+    idle: AtomicBool,
+    invalidation_locks: [parking_lot::Mutex<()>; FD_CACHE_PREFIX_SHARDS],
+    pending_invalidations: parking_lot::Mutex<Vec<(Arc<FdKey>, std::sync::Weak<CachedFdEntry>)>>,
+    volume_counts: [std::sync::atomic::AtomicU32; FD_CACHE_VOLUME_FILTER_SLOTS],
+    unindexed_counts: [std::sync::atomic::AtomicU32; FD_CACHE_VOLUME_FILTER_SLOTS],
+    exact_counts: [std::sync::atomic::AtomicU32; FD_CACHE_EXACT_FILTER_SLOTS],
+    prefix_counts: [std::sync::atomic::AtomicU32; FD_CACHE_PREFIX_FILTER_SLOTS],
+    prefix_shards: [parking_lot::Mutex<Vec<(u64, usize)>>; FD_CACHE_PREFIX_SHARDS],
+    entry_shards: [parking_lot::Mutex<FdTrackerEntryList>; FD_CACHE_PREFIX_SHARDS],
+}
+
+#[cfg(any(target_os = "linux", all(unix, test)))]
+impl FdCacheTracker {
+    fn new() -> Self {
+        Self {
+            live_entries: std::sync::atomic::AtomicUsize::new(0),
+            next_entry_id: std::sync::atomic::AtomicU64::new(1),
+            has_pending_invalidations: AtomicBool::new(false),
+            idle: AtomicBool::new(false),
+            invalidation_locks: std::array::from_fn(|_| parking_lot::Mutex::new(())),
+            pending_invalidations: parking_lot::Mutex::new(Vec::new()),
+            volume_counts: std::array::from_fn(|_| std::sync::atomic::AtomicU32::new(0)),
+            unindexed_counts: std::array::from_fn(|_| std::sync::atomic::AtomicU32::new(0)),
+            exact_counts: std::array::from_fn(|_| std::sync::atomic::AtomicU32::new(0)),
+            prefix_counts: std::array::from_fn(|_| std::sync::atomic::AtomicU32::new(0)),
+            prefix_shards: std::array::from_fn(|_| parking_lot::Mutex::new(Vec::new())),
+            entry_shards: std::array::from_fn(|_| parking_lot::Mutex::new(Vec::new())),
+        }
+    }
+
+    #[inline]
+    fn lock_key(&self, volume: &str, path: &str) -> parking_lot::MutexGuard<'_, ()> {
+        let vol_hash = xxhash_rust::xxh3::xxh3_64(volume.as_bytes());
+        let h = xxhash_rust::xxh3::xxh3_64_with_seed(path.as_bytes(), vol_hash);
+        self.invalidation_locks[(h as usize) & (FD_CACHE_PREFIX_SHARDS - 1)].lock()
+    }
+
+    #[inline]
+    fn live_entries(&self) -> usize {
+        self.live_entries.load(Ordering::SeqCst)
+    }
+
+    #[inline]
+    fn is_idle_candidate(&self) -> bool {
+        self.live_entries.load(Ordering::SeqCst) == 0 && !self.has_pending_invalidations.load(Ordering::SeqCst)
+    }
+
+    fn insert_prefix_hash(&self, hash: u64) {
+        let mut shard = self.prefix_shards[(hash as usize) & (FD_CACHE_PREFIX_SHARDS - 1)].lock();
+        if let Some((_, count)) = shard.iter_mut().find(|(h, _)| *h == hash) {
+            *count += 1;
+        } else {
+            shard.push((hash, 1));
+        }
+    }
+
+    fn remove_prefix_hash(&self, hash: u64) {
+        let mut shard = self.prefix_shards[(hash as usize) & (FD_CACHE_PREFIX_SHARDS - 1)].lock();
+        if let Some(pos) = shard.iter().position(|(h, _)| *h == hash) {
+            if shard[pos].1 > 1 {
+                shard[pos].1 -= 1;
+            } else {
+                shard.swap_remove(pos);
+            }
+        }
+    }
+
+    #[inline]
+    fn has_prefix_hash(&self, hash: u64) -> bool {
+        let shard = self.prefix_shards[(hash as usize) & (FD_CACHE_PREFIX_SHARDS - 1)].lock();
+        shard.iter().any(|(h, _)| *h == hash)
+    }
+
+    fn matching_entries_in_volume(
+        &self,
+        volume: &str,
+        matches: impl Fn(&FdKey) -> bool,
+    ) -> Vec<(Arc<FdKey>, Arc<CachedFdEntry>)> {
+        let (vol_hash, _) = fd_cache_volume_hash_and_slot(volume);
+        let shard = self.entry_shards[(vol_hash as usize) & (FD_CACHE_PREFIX_SHARDS - 1)].lock();
+        let mut out = Vec::new();
+        for (_, key, weak) in shard.iter() {
+            if matches(key)
+                && let Some(cached) = weak.upgrade()
+            {
+                out.push((Arc::clone(key), cached));
+            }
+        }
+        out
+    }
+
+    fn all_live_entries(&self) -> Vec<Arc<CachedFdEntry>> {
+        let mut out = Vec::new();
+        for shard in &self.entry_shards {
+            let guard = shard.lock();
+            for (_, _, weak) in guard.iter() {
+                if let Some(cached) = weak.upgrade() {
+                    out.push(cached);
+                }
+            }
+        }
+        out
+    }
+
+    #[inline]
+    fn may_match_volume(&self, volume: &str) -> bool {
+        if self.live_entries.load(Ordering::SeqCst) == 0 {
+            return false;
+        }
+        let (_, vol_slot) = fd_cache_volume_hash_and_slot(volume);
+        self.volume_counts[usize::from(vol_slot)].load(Ordering::Acquire) != 0
+    }
+
+    #[inline]
+    fn may_match_exact(&self, volume: &str, path: &str) -> bool {
+        if self.live_entries.load(Ordering::SeqCst) == 0 {
+            return false;
+        }
+        let (vol_hash, vol_slot) = fd_cache_volume_hash_and_slot(volume);
+        if self.volume_counts[usize::from(vol_slot)].load(Ordering::Acquire) == 0 {
+            return false;
+        }
+        let (hash, [s0, s1, s2, s3]) = fd_cache_hash_and_quad_slots(vol_hash, path, FD_CACHE_EXACT_FILTER_SLOTS);
+        if self.exact_counts[usize::from(s0)].load(Ordering::Acquire) == 0
+            || self.exact_counts[usize::from(s1)].load(Ordering::Acquire) == 0
+            || self.exact_counts[usize::from(s2)].load(Ordering::Acquire) == 0
+            || self.exact_counts[usize::from(s3)].load(Ordering::Acquire) == 0
+        {
+            return false;
+        }
+        self.unindexed_counts[usize::from(vol_slot)].load(Ordering::Acquire) != 0 || self.has_prefix_hash(hash)
+    }
+
+    #[inline]
+    fn may_match_prefix(&self, volume: &str, prefix: &str) -> bool {
+        if self.live_entries.load(Ordering::SeqCst) == 0 {
+            return false;
+        }
+        let (vol_hash, vol_slot) = fd_cache_volume_hash_and_slot(volume);
+        if self.volume_counts[usize::from(vol_slot)].load(Ordering::Acquire) == 0 {
+            return false;
+        }
+        if self.unindexed_counts[usize::from(vol_slot)].load(Ordering::Acquire) != 0 {
+            return true;
+        }
+        let (hash, [s0, s1, s2, s3]) = fd_cache_hash_and_quad_slots(vol_hash, prefix, FD_CACHE_PREFIX_FILTER_SLOTS);
+        self.prefix_counts[usize::from(s0)].load(Ordering::Acquire) != 0
+            && self.prefix_counts[usize::from(s1)].load(Ordering::Acquire) != 0
+            && self.prefix_counts[usize::from(s2)].load(Ordering::Acquire) != 0
+            && self.prefix_counts[usize::from(s3)].load(Ordering::Acquire) != 0
+            && self.has_prefix_hash(hash)
+    }
+
+    fn enqueue_invalidations(&self, entries: Vec<(Arc<FdKey>, std::sync::Weak<CachedFdEntry>)>) {
+        let mut pending = self.pending_invalidations.lock();
+        if pending.len() >= (FD_CACHE_CAPACITY as usize).saturating_mul(2) {
+            pending.retain(|(_, weak)| weak.strong_count() > 0);
+        }
+        pending.extend(entries);
+        self.has_pending_invalidations.store(true, Ordering::SeqCst);
+    }
+
+    fn take_pending_invalidations(&self) -> Vec<(Arc<FdKey>, std::sync::Weak<CachedFdEntry>)> {
+        let mut pending = self.pending_invalidations.lock();
+        let taken = std::mem::take(&mut *pending);
+        self.has_pending_invalidations.store(false, Ordering::SeqCst);
+        taken
+    }
+
+    fn clear_pending_invalidations(&self) {
+        let mut pending = self.pending_invalidations.lock();
+        pending.clear();
+        self.has_pending_invalidations.store(false, Ordering::SeqCst);
+    }
+}
+
+#[cfg(any(target_os = "linux", all(unix, test)))]
+struct CachedFdLiveState {
+    entry: Arc<FdCacheEntry>,
+    tracker: Arc<FdCacheTracker>,
+    entry_id: u64,
+    entry_shard: u8,
+    volume_slot: u8,
+    prefix_slot_len: u8,
+    exact_slots: [u16; 4],
+    prefix_hashes: [u64; FD_CACHE_MAX_INDEXED_PREFIXES],
+    prefix_slots: [[u16; 4]; FD_CACHE_MAX_INDEXED_PREFIXES],
+}
+
+#[cfg(any(target_os = "linux", all(unix, test)))]
+impl Drop for CachedFdLiveState {
+    fn drop(&mut self) {
+        {
+            let mut shard = self.tracker.entry_shards[usize::from(self.entry_shard)].lock();
+            if let Some(pos) = shard.iter().position(|(id, _, _)| *id == self.entry_id) {
+                shard.swap_remove(pos);
+            }
+        }
+        if self.prefix_slot_len == u8::MAX {
+            self.tracker.unindexed_counts[usize::from(self.volume_slot)].fetch_sub(1, Ordering::AcqRel);
+        } else {
+            for i in 0..usize::from(self.prefix_slot_len) {
+                for &s in &self.prefix_slots[i] {
+                    self.tracker.prefix_counts[usize::from(s)].fetch_sub(1, Ordering::AcqRel);
+                }
+                self.tracker.remove_prefix_hash(self.prefix_hashes[i]);
+            }
+        }
+        for s in self.exact_slots {
+            self.tracker.exact_counts[usize::from(s)].fetch_sub(1, Ordering::AcqRel);
+        }
+        self.tracker.volume_counts[usize::from(self.volume_slot)].fetch_sub(1, Ordering::AcqRel);
+        self.tracker.live_entries.fetch_sub(1, Ordering::SeqCst);
+    }
+}
+
+#[cfg(any(target_os = "linux", all(unix, test)))]
+struct CachedFdEntry {
+    active: AtomicBool,
+    enqueued_for_removal: AtomicBool,
+    state: std::sync::RwLock<Option<CachedFdLiveState>>,
+}
+
+#[cfg(any(target_os = "linux", all(unix, test)))]
+impl CachedFdEntry {
+    fn new(tracker: Arc<FdCacheTracker>, key: &FdKey, entry: Arc<FdCacheEntry>) -> Arc<Self> {
+        let (vol_hash, volume_slot) = fd_cache_volume_hash_and_slot(&key.volume);
+        let entry_shard = (vol_hash as usize) & (FD_CACHE_PREFIX_SHARDS - 1);
+        let entry_id = tracker.next_entry_id.fetch_add(1, Ordering::Relaxed);
+        let (_, exact_slots) = fd_cache_hash_and_quad_slots(vol_hash, &key.path, FD_CACHE_EXACT_FILTER_SLOTS);
+        let mut prefix_hashes = [0u64; FD_CACHE_MAX_INDEXED_PREFIXES];
+        let mut prefix_slots = [[0u16; 4]; FD_CACHE_MAX_INDEXED_PREFIXES];
+        let mut prefix_slot_len = 0usize;
+        let mut overflowed = false;
+
+        for (idx, byte) in key.path.bytes().enumerate() {
+            if byte == b'/' {
+                if prefix_slot_len == FD_CACHE_MAX_INDEXED_PREFIXES {
+                    overflowed = true;
+                    break;
+                }
+                let (h, slots) = fd_cache_hash_and_quad_slots(vol_hash, &key.path[..idx], FD_CACHE_PREFIX_FILTER_SLOTS);
+                prefix_hashes[prefix_slot_len] = h;
+                prefix_slots[prefix_slot_len] = slots;
+                prefix_slot_len += 1;
+            }
+        }
+        if !overflowed {
+            if prefix_slot_len == FD_CACHE_MAX_INDEXED_PREFIXES {
+                overflowed = true;
+            } else {
+                let (h, slots) = fd_cache_hash_and_quad_slots(vol_hash, &key.path, FD_CACHE_PREFIX_FILTER_SLOTS);
+                prefix_hashes[prefix_slot_len] = h;
+                prefix_slots[prefix_slot_len] = slots;
+                prefix_slot_len += 1;
+            }
+        }
+
+        tracker.volume_counts[usize::from(volume_slot)].fetch_add(1, Ordering::AcqRel);
+        for s in exact_slots {
+            tracker.exact_counts[usize::from(s)].fetch_add(1, Ordering::AcqRel);
+        }
+        let encoded_len = if overflowed {
+            tracker.unindexed_counts[usize::from(volume_slot)].fetch_add(1, Ordering::AcqRel);
+            u8::MAX
+        } else {
+            for i in 0..prefix_slot_len {
+                tracker.insert_prefix_hash(prefix_hashes[i]);
+                for &s in &prefix_slots[i] {
+                    tracker.prefix_counts[usize::from(s)].fetch_add(1, Ordering::AcqRel);
+                }
+            }
+            prefix_slot_len as u8
+        };
+
+        let tracker_for_state = Arc::clone(&tracker);
+        let cached = Arc::new(Self {
+            active: AtomicBool::new(true),
+            enqueued_for_removal: AtomicBool::new(false),
+            state: std::sync::RwLock::new(Some(CachedFdLiveState {
+                entry,
+                tracker: tracker_for_state,
+                entry_id,
+                entry_shard: entry_shard as u8,
+                volume_slot,
+                prefix_slot_len: encoded_len,
+                exact_slots,
+                prefix_hashes,
+                prefix_slots,
+            })),
+        });
+        {
+            let mut shard = tracker.entry_shards[entry_shard].lock();
+            // Count the entry before publishing it to `entry_shards` so no thread
+            // can discover and retire it before `live_entries` is incremented.
+            // Pairs with `live_entries.load(Ordering::SeqCst)` in `may_match_*`.
+            tracker.live_entries.fetch_add(1, Ordering::SeqCst);
+            shard.push((entry_id, Arc::new(key.clone()), Arc::downgrade(&cached)));
+        }
+
+        cached
+    }
+
+    #[inline]
+    fn get_entry(&self) -> Option<Arc<FdCacheEntry>> {
+        if !self.active.load(Ordering::Acquire) {
+            return None;
+        }
+        let guard = self.state.read().unwrap_or_else(|e| e.into_inner());
+        let live = guard.as_ref()?;
+        (!live.entry.invalidated.load(Ordering::Acquire)).then(|| Arc::clone(&live.entry))
+    }
+
+    fn mark_invalidated(&self) {
+        self.active.store(false, Ordering::Release);
+        if let Some(live) = self.state.read().unwrap_or_else(|e| e.into_inner()).as_ref() {
+            live.entry.invalidated.store(true, Ordering::Release);
+        }
+    }
+
+    fn retire(&self) {
+        self.active.store(false, Ordering::Release);
+        let live = self.state.write().unwrap_or_else(|e| e.into_inner()).take();
+        if let Some(live) = live {
+            live.entry.invalidated.store(true, Ordering::Release);
+        }
+    }
+}
+
 /// Per-disk cache of open descriptors for io_uring reads (backlog#1145).
 ///
 /// Why this exists: both `pread_uring` and the native O_DIRECT path used to open
@@ -5740,10 +6135,8 @@ struct FdCacheEntry {
 /// the descriptor once no in-flight read still holds it.
 #[cfg(any(target_os = "linux", all(unix, test)))]
 struct FdCache {
-    cache: moka::future::Cache<FdKey, Arc<FdCacheEntry>>,
-    // Moka's entry count can lag inserts. Only a cache that has never accepted
-    // an insert can safely skip the foreground invalidation scan.
-    has_inserted_entries: AtomicBool,
+    cache: moka::sync::Cache<FdKey, Arc<CachedFdEntry>>,
+    tracker: Arc<FdCacheTracker>,
     // The only sender belongs to this cache. Dropping it lets the worker finish
     // maintenance and release its cache clone without retaining the backend.
     maintenance_tx: tokio::sync::watch::Sender<()>,
@@ -5755,6 +6148,18 @@ struct FdCache {
     generation: std::sync::atomic::AtomicU64,
 }
 
+#[cfg(any(target_os = "linux", all(unix, test)))]
+impl Drop for FdCache {
+    fn drop(&mut self) {
+        self.tracker.clear_pending_invalidations();
+        for cached in self.tracker.all_live_entries() {
+            cached.retire();
+        }
+        self.cache.invalidate_all();
+        self.cache.run_pending_tasks();
+    }
+}
+
 #[cfg(any(target_os = "linux", all(unix, test)))]
 impl FdCache {
     fn new() -> Self {
@@ -5767,20 +6172,26 @@ impl FdCache {
     fn with_ttl(ttl: std::time::Duration) -> Self {
         let (maintenance_tx, _) = tokio::sync::watch::channel(());
         Self {
-            cache: moka::future::Cache::builder()
+            cache: moka::sync::Cache::builder()
                 .max_capacity(FD_CACHE_CAPACITY)
                 .time_to_live(ttl)
+                .eviction_listener(|_key, cached: Arc<CachedFdEntry>, cause| {
+                    if !matches!(cause, moka::notification::RemovalCause::Explicit) {
+                        cached.retire();
+                    }
+                })
                 .build(),
-            has_inserted_entries: AtomicBool::new(false),
+            tracker: Arc::new(FdCacheTracker::new()),
             maintenance_tx,
             maintenance_task: std::sync::OnceLock::new(),
             generation: std::sync::atomic::AtomicU64::new(0),
         }
     }
 
-    fn schedule_maintenance(&self) {
+    fn ensure_maintenance_task(&self) {
         self.maintenance_task.get_or_init(|| {
             let cache = self.cache.clone();
+            let tracker = Arc::clone(&self.tracker);
             let mut notifications = self.maintenance_tx.subscribe();
             // Invalidation runs on the disk API's Tokio runtime. Starting lazily
             // keeps construction runtime-independent and idle caches task-free.
@@ -5791,58 +6202,102 @@ impl FdCache {
                 );
                 interval.set_missed_tick_behavior(tokio::time::MissedTickBehavior::Skip);
                 loop {
-                    let stopping = tokio::select! {
-                        changed = notifications.changed() => changed.is_err(),
-                        _ = interval.tick() => false,
-                    };
-                    if stopping {
-                        cache.invalidate_all();
-                    }
-                    cache.run_pending_tasks().await;
-                    if stopping {
-                        break;
-                    }
-                    if cache.entry_count() != 0 {
-                        for (key, entry) in cache.iter() {
-                            if entry.invalidated.load(Ordering::Acquire) {
-                                // Concurrent replacement may evict a fresh fd;
-                                // a later read safely reopens it.
-                                cache.invalidate(key.as_ref()).await;
+                    let stopping = if tracker.is_idle_candidate() {
+                        tracker.idle.store(true, Ordering::SeqCst);
+                        if tracker.is_idle_candidate() {
+                            let res = notifications.changed().await.is_err();
+                            tracker.idle.store(false, Ordering::SeqCst);
+                            interval.reset();
+                            res
+                        } else {
+                            tracker.idle.store(false, Ordering::SeqCst);
+                            tokio::select! {
+                                changed = notifications.changed() => changed.is_err(),
+                                _ = interval.tick() => false,
                             }
                         }
-                        cache.run_pending_tasks().await;
+                    } else {
+                        tokio::select! {
+                            changed = notifications.changed() => changed.is_err(),
+                            _ = interval.tick() => false,
+                        }
+                    };
+                    if stopping {
+                        tracker.clear_pending_invalidations();
+                        let live = tracker.all_live_entries();
+                        cache.invalidate_all();
+                        cache.run_pending_tasks();
+                        for cached in live {
+                            cached.retire();
+                        }
+                        break;
+                    }
+                    let pending = tracker.take_pending_invalidations();
+                    let mut retired_entries = Vec::with_capacity(pending.len());
+                    for (key, weak_entry) in pending {
+                        let Some(invalidated_entry) = weak_entry.upgrade() else {
+                            continue;
+                        };
+                        {
+                            let _guard = tracker.lock_key(&key.volume, &key.path);
+                            if cache
+                                .get(key.as_ref())
+                                .is_some_and(|current| Arc::ptr_eq(&current, &invalidated_entry))
+                            {
+                                cache.invalidate(key.as_ref());
+                            }
+                        }
+                        retired_entries.push(invalidated_entry);
+                    }
+                    cache.run_pending_tasks();
+                    for entry in retired_entries {
+                        entry.retire();
                     }
                 }
             })
         });
+    }
+
+    fn schedule_maintenance(&self) {
+        self.ensure_maintenance_task();
         self.maintenance_tx.send_replace(());
     }
 
-    fn invalidate_matching(&self, matches: impl Fn(&FdKey) -> bool) {
-        if !self.has_inserted_entries.load(Ordering::Acquire) {
-            return;
+    fn activate_maintenance(&self) {
+        if self.maintenance_task.get().is_none() {
+            self.ensure_maintenance_task();
         }
-        // Entries already present are visited unless concurrently removed. The
-        // generation fence rejects an older open inserted after this scan.
-        for (key, entry) in self.cache.iter() {
-            if matches(&key) {
-                entry.invalidated.store(true, Ordering::Release);
+        if self.tracker.idle.load(Ordering::SeqCst) {
+            self.maintenance_tx.send_replace(());
+        }
+    }
+
+    fn invalidate_matching(&self, volume: &str, matches: impl Fn(&FdKey) -> bool) {
+        for lock in &self.tracker.invalidation_locks {
+            drop(lock.lock());
+        }
+        let matched = self.tracker.matching_entries_in_volume(volume, matches);
+        let mut newly_invalidated = Vec::with_capacity(matched.len());
+        for (key, cached) in matched {
+            cached.mark_invalidated();
+            if !cached.enqueued_for_removal.swap(true, Ordering::AcqRel) {
+                newly_invalidated.push((key, Arc::downgrade(&cached)));
             }
         }
-        self.schedule_maintenance();
+        if !newly_invalidated.is_empty() {
+            self.tracker.enqueue_invalidations(newly_invalidated);
+            self.schedule_maintenance();
+        }
     }
 
     async fn get(&self, key: &FdKey) -> Option<Arc<FdCacheEntry>> {
-        self.cache
-            .get(key)
-            .await
-            .filter(|entry| !entry.invalidated.load(Ordering::Acquire))
+        self.cache.get(key).and_then(|cached| cached.get_entry())
     }
 
     /// Current invalidation generation, snapshotted by the miss path before it
     /// opens a descriptor (rustfs/backlog#1176).
     fn generation(&self) -> u64 {
-        self.generation.load(Ordering::Acquire)
+        self.generation.load(Ordering::SeqCst)
     }
 
     /// Insert a freshly-opened descriptor only if no invalidation happened since
@@ -5851,13 +6306,39 @@ impl FdCache {
     /// cached. The post-insert re-check closes the tiny window where an
     /// invalidate races the insert itself, by removing the entry we just added.
     async fn insert_if_fresh(&self, key: FdKey, entry: Arc<FdCacheEntry>, gen_at_open: u64) {
-        if self.generation.load(Ordering::Acquire) != gen_at_open {
+        if self.generation.load(Ordering::SeqCst) != gen_at_open {
             return;
         }
-        self.has_inserted_entries.store(true, Ordering::Release);
-        self.cache.insert(key.clone(), entry).await;
-        if self.generation.load(Ordering::Acquire) != gen_at_open {
-            self.cache.invalidate(&key).await;
+        let cached = {
+            let _guard = self.tracker.lock_key(&key.volume, &key.path);
+            if self.generation.load(Ordering::SeqCst) != gen_at_open {
+                return;
+            }
+            let cached = CachedFdEntry::new(Arc::clone(&self.tracker), &key, entry);
+            // Dekker partner: must stay after `CachedFdEntry::new` (`live_entries++`)
+            // and before `previous.retire()` / `self.cache.insert`.
+            if self.generation.load(Ordering::SeqCst) != gen_at_open {
+                cached.retire();
+                return;
+            }
+            if let Some(previous) = self.cache.get(&key) {
+                previous.retire();
+            }
+            self.cache.insert(key.clone(), Arc::clone(&cached));
+            if self.generation.load(Ordering::SeqCst) != gen_at_open {
+                cached.retire();
+                self.cache.invalidate(&key);
+                return;
+            }
+            cached
+        };
+        self.activate_maintenance();
+        if self.generation.load(Ordering::SeqCst) != gen_at_open {
+            cached.retire();
+            let _guard = self.tracker.lock_key(&key.volume, &key.path);
+            if self.cache.get(&key).is_some_and(|current| Arc::ptr_eq(&current, &cached)) {
+                self.cache.invalidate(&key);
+            }
         }
     }
 
@@ -5867,15 +6348,33 @@ impl FdCache {
     async fn invalidate_exact(&self, volume: &str, path: &str) {
         // Bump BEFORE the moka invalidation so a concurrent miss-path insert
         // that snapshotted the old generation is refused (rustfs/backlog#1176).
-        self.generation.fetch_add(1, Ordering::AcqRel);
+        self.generation.fetch_add(1, Ordering::SeqCst);
+        if !self.tracker.may_match_exact(volume, path) {
+            return;
+        }
         let mut key = FdKey {
             volume: volume.to_owned(),
             path: path.to_owned(),
             direct: false,
         };
-        self.cache.invalidate(&key).await;
-        key.direct = true;
-        self.cache.invalidate(&key).await;
+        let mut removed_any = false;
+        {
+            let _guard = self.tracker.lock_key(volume, path);
+            if let Some(cached) = self.cache.get(&key) {
+                cached.retire();
+                removed_any = true;
+            }
+            self.cache.invalidate(&key);
+            key.direct = true;
+            if let Some(cached) = self.cache.get(&key) {
+                cached.retire();
+                removed_any = true;
+            }
+            self.cache.invalidate(&key);
+        }
+        if removed_any {
+            self.schedule_maintenance();
+        }
     }
 
     /// Drop every descriptor for `volume` whose path is `prefix` or lies under it.
@@ -5887,9 +6386,12 @@ impl FdCache {
     fn invalidate_under(&self, volume: &str, prefix: &str) {
         // Bump before scanning the cache so a concurrent miss-path insert
         // that snapshotted the old generation is refused (rustfs/backlog#1176).
-        self.generation.fetch_add(1, Ordering::AcqRel);
+        self.generation.fetch_add(1, Ordering::SeqCst);
         let prefix = prefix.trim_end_matches('/');
-        self.invalidate_matching(|k| {
+        if !self.tracker.may_match_prefix(volume, prefix) {
+            return;
+        }
+        self.invalidate_matching(volume, |k| {
             k.volume == volume && (k.path == prefix || k.path.strip_prefix(prefix).is_some_and(|r| r.starts_with('/')))
         });
     }
@@ -5897,16 +6399,34 @@ impl FdCache {
     /// Drop every descriptor for `volume` — the whole bucket is gone
     /// (rustfs/backlog#1177).
     fn invalidate_volume(&self, volume: &str) {
-        self.generation.fetch_add(1, Ordering::AcqRel);
-        self.invalidate_matching(|k| k.volume == volume);
+        self.generation.fetch_add(1, Ordering::SeqCst);
+        if !self.tracker.may_match_volume(volume) {
+            return;
+        }
+        self.invalidate_matching(volume, |k| k.volume == volume);
     }
 
     /// Drop ALL descriptors — this disk instance is being retired
     /// (rustfs/backlog#1177).
     fn clear(&self) {
-        self.generation.fetch_add(1, Ordering::AcqRel);
+        self.generation.fetch_add(1, Ordering::SeqCst);
+        for lock in &self.tracker.invalidation_locks {
+            drop(lock.lock());
+        }
+        let mut had_entries = self.tracker.live_entries() != 0 || self.tracker.has_pending_invalidations.load(Ordering::SeqCst);
+        self.tracker.clear_pending_invalidations();
+        let live = self.tracker.all_live_entries();
+        if !live.is_empty() {
+            had_entries = true;
+        }
         self.cache.invalidate_all();
-        self.schedule_maintenance();
+        self.cache.run_pending_tasks();
+        for cached in live {
+            cached.retire();
+        }
+        if had_entries || self.maintenance_task.get().is_some() {
+            self.schedule_maintenance();
+        }
     }
 
     /// Unconditional insert. Production reads go through `insert_if_fresh` (the
@@ -5915,22 +6435,25 @@ impl FdCache {
     #[cfg(test)]
     async fn insert(&self, key: FdKey, file: Arc<std::fs::File>) {
         let len = file.metadata().map(|metadata| metadata.len()).unwrap_or_default();
-        self.has_inserted_entries.store(true, Ordering::Release);
-        self.cache
-            .insert(
-                key,
-                Arc::new(FdCacheEntry {
-                    file,
-                    len,
-                    invalidated: AtomicBool::new(false),
-                }),
-            )
-            .await;
+        let entry = Arc::new(FdCacheEntry {
+            file,
+            len,
+            invalidated: AtomicBool::new(false),
+        });
+        {
+            let _guard = self.tracker.lock_key(&key.volume, &key.path);
+            if let Some(previous) = self.cache.get(&key) {
+                previous.retire();
+            }
+            let cached = CachedFdEntry::new(Arc::clone(&self.tracker), &key, entry);
+            self.cache.insert(key, cached);
+        }
+        self.activate_maintenance();
     }
 
     #[cfg(test)]
     async fn entry_count(&self) -> u64 {
-        self.cache.run_pending_tasks().await;
+        self.cache.run_pending_tasks();
         self.cache.entry_count()
     }
 }
@@ -11222,8 +11745,16 @@ impl LocalDisk {
 
                 if entry.ends_with(SLASH_SEPARATOR) {
                     if entry.ends_with(GLOBAL_DIR_SUFFIX_WITH_SLASH) {
-                        let entry =
-                            format!("{}{}", entry.as_str().trim_end_matches(GLOBAL_DIR_SUFFIX_WITH_SLASH), SLASH_SEPARATOR);
+                        let marker_name = entry.as_str().trim_end_matches(GLOBAL_DIR_SUFFIX_WITH_SLASH);
+                        // An empty marker name would schedule the directory being
+                        // scanned as its own child and recurse until the stack
+                        // overflows. Ignore malformed markers left by older writers
+                        // or corrupt disks instead of decoding them.
+                        if marker_name.is_empty() {
+                            item.clear();
+                            continue;
+                        }
+                        let entry = format!("{marker_name}{SLASH_SEPARATOR}");
                         dir_objes.insert(entry.clone());
                         *item = entry;
                         continue;
@@ -26834,6 +27365,49 @@ mod test {
         assert_eq!(objs_returned, 1);
     }
 
+    #[tokio::test]
+    async fn test_scan_dir_ignores_empty_directory_markers() {
+        use rustfs_filemeta::MetacacheReader;
+        use tempfile::tempdir;
+
+        let dir = tempdir().expect("create disk directory");
+        let bucket = "test-bucket";
+        let bucket_dir = dir.path().join(bucket);
+        for object in ["__XLDIR__", "nested/__XLDIR__", "nested/healthy", "folder__XLDIR__"] {
+            let object_dir = bucket_dir.join(object);
+            fs::create_dir_all(&object_dir).await.expect("create object directory");
+            fs::write(object_dir.join("xl.meta"), b"meta")
+                .await
+                .expect("write object metadata");
+        }
+        let endpoint = Endpoint::try_from(dir.path().to_str().expect("UTF-8 disk path")).expect("valid disk endpoint");
+        let disk = LocalDisk::new(&endpoint, false).await.expect("create local disk");
+        let (reader, mut writer) = tokio::io::duplex(4096);
+        let mut out = MetacacheWriter::new(&mut writer);
+        let opts = WalkDirOptions {
+            bucket: bucket.to_owned(),
+            recursive: true,
+            ..Default::default()
+        };
+        let mut count = 0;
+        tokio::time::timeout(
+            std::time::Duration::from_secs(5),
+            disk.scan_dir(String::new(), String::new(), &opts, &mut out, &mut count, false, None),
+        )
+        .await
+        .expect("malformed directory marker must not recurse into its parent")
+        .expect("scan succeeds despite malformed markers");
+        out.close().await.expect("close metacache stream");
+        let entries = MetacacheReader::new(reader).read_all().await.expect("read listed objects");
+        let names: Vec<_> = entries
+            .into_iter()
+            .filter(|entry| !entry.metadata.is_empty())
+            .map(|entry| entry.name)
+            .collect();
+        assert_eq!(names, vec!["folder/", "nested/healthy"]);
+        assert_eq!(count, 2);
+    }
+
     #[tokio::test]
     async fn test_scan_dir_deduplicates_explicit_dir_marker_recursion() {
         use rustfs_filemeta::MetacacheReader;

crates/ecstore/src/disk/local/delete_recovery.rs

@@ -1287,9 +1287,15 @@ impl LocalDisk {
         let volume = volume.to_owned();
         let object = object.to_owned();
         let lease = os::acquire_namespace_mutation_lease_with_owner(&root, owner).await;
-        os::run_blocking_namespace_operation(lease, move || Ok(inspection_with_markers(&root, &volume, &object, transaction)))
-            .await
-            .map_err(to_file_error)?
+        // Journal inspections ride the durable-mutation lane (the fused delete
+        // prepare already does): a bulk delete publishes one transaction per
+        // key, and the one-worker foreground namespace lane would serialize
+        // those per-drive publishes behind every other mutation.
+        os::run_blocking_namespace_operation_with_sync(lease, true, move || {
+            Ok(inspection_with_markers(&root, &volume, &object, transaction))
+        })
+        .await
+        .map_err(to_file_error)?
     }
 
     async fn activate_delete_preparation(
@@ -1685,7 +1691,10 @@ impl LocalDisk {
             let plan_root = root.clone();
             let missing = observed.current_digest.is_none();
             let lease = os::acquire_namespace_mutation_lease_with_owner(&root, owner.clone()).await;
-            let plan = os::run_blocking_namespace_operation(lease, move || {
+            // Same durable-mutation lane as the journal inspection above: this
+            // plan closure runs inside every settle on the bulk-delete publish
+            // path.
+            let plan = os::run_blocking_namespace_operation_with_sync(lease, true, move || {
                 Ok((|| {
                     let before =
                         regular_metadata_bytes(&plan_root.join(transaction.to_string()).join(STORAGE_FORMAT_FILE_BACKUP))?

crates/ecstore/src/set_disk/ops/locking.rs

@@ -95,13 +95,8 @@ impl SetDisks {
         bucket: &str,
         object: &str,
     ) -> Result<Arc<rustfs_lock::NamespaceLockGuard>> {
-        let resource = ObjectKey::with_version(
-            bucket,
-            object_name_disk_identity(object).as_ref(),
-            format!("rustfs:delete-transaction:{}:{}", self.pool_index, self.set_index),
-        );
         let guard = self
-            .namespace_lock_for_key(resource)
+            .namespace_lock_for_key(self.delete_transaction_resource(bucket, object))
             .await
             .get_write_lock(super::super::get_lock_acquire_timeout())
             .await
@@ -111,6 +106,94 @@ impl SetDisks {
         Ok(Arc::new(guard))
     }
 
+    fn delete_transaction_resource(&self, bucket: &str, object: &str) -> ObjectKey {
+        ObjectKey::with_version(
+            bucket,
+            object_name_disk_identity(object).as_ref(),
+            format!("rustfs:delete-transaction:{}:{}", self.pool_index, self.set_index),
+        )
+    }
+
+    /// Batched write-lock acquisition for the whole key set of one bulk
+    /// mutation: one lock round instead of one quorum round-trip per key.
+    ///
+    /// Bulk delete is the only multi-key write-lock holder. The batched round
+    /// is deadlock-free by construction (a single `lock_batch` RPC per lock
+    /// client, and every lock server orders concurrent batches by the same
+    /// global (shard, key) order), so unlike an optimistic probe wave it
+    /// needs no canonical serial fallback. Outcomes are per key and use the
+    /// same error shape as the single-key paths.
+    async fn namespace_write_locks_batch(
+        &self,
+        bucket: &str,
+        resources: &[ObjectKey],
+        timeout: std::time::Duration,
+        mode: &'static str,
+    ) -> Vec<Result<rustfs_lock::NamespaceLockGuard>> {
+        if resources.is_empty() {
+            return Vec::new();
+        }
+        let lock = self.namespace_lock_for_key(resources[0].clone()).await;
+        lock.get_write_locks_batch(resources, timeout)
+            .await
+            .into_iter()
+            .zip(resources)
+            .map(|(result, resource)| {
+                result.map_err(|error| self.map_namespace_lock_error(bucket, resource.object.as_ref(), mode, error))
+            })
+            .collect()
+    }
+
+    /// Batched raw namespace write locks for a bulk delete's key set. The
+    /// resource keys use the same disk-identity folding as `new_ns_lock`, so
+    /// aliases (`a\b` vs `a/b`) still share one lock.
+    pub(crate) async fn acquire_object_write_locks_batch(
+        &self,
+        bucket: &str,
+        objects: &[&str],
+        timeout: std::time::Duration,
+    ) -> Vec<Result<rustfs_lock::NamespaceLockGuard>> {
+        let resources: Vec<ObjectKey> = objects
+            .iter()
+            .map(|object| ObjectKey {
+                bucket: Arc::from(bucket),
+                object: Arc::from(object_name_disk_identity(object).as_ref()),
+                version: None,
+            })
+            .collect();
+        self.namespace_write_locks_batch(bucket, &resources, timeout, "write").await
+    }
+
+    /// Batched delete-transaction locks for a bulk delete's key set; see
+    /// [`Self::acquire_delete_transaction_lock`] for the resource shape.
+    pub(in crate::set_disk) async fn acquire_delete_transaction_locks_batch(
+        &self,
+        bucket: &str,
+        objects: &[&str],
+        timeout: std::time::Duration,
+    ) -> Vec<Result<Arc<rustfs_lock::NamespaceLockGuard>>> {
+        let resources: Vec<ObjectKey> = objects
+            .iter()
+            .map(|object| self.delete_transaction_resource(bucket, object))
+            .collect();
+        let mut guards = Vec::with_capacity(resources.len());
+        for (object, result) in objects.iter().zip(
+            self.namespace_write_locks_batch(bucket, &resources, timeout, "delete_transaction")
+                .await,
+        ) {
+            match result {
+                Ok(guard) => {
+                    #[cfg(test)]
+                    super::delete_recovery::test_hooks::run_after_transaction_lock_acquired(bucket, object, &guard);
+                    let _ = object;
+                    guards.push(Ok(Arc::new(guard)));
+                }
+                Err(error) => guards.push(Err(error)),
+            }
+        }
+        guards
+    }
+
     #[allow(dead_code, reason = "lock diagnostics formatter with no caller in this port (backlog#1823)")]
     pub(in crate::set_disk) fn format_lock_error(&self, bucket: &str, object: &str, mode: &str, err: &LockResult) -> String {
         match err {

crates/ecstore/src/set_disk/ops/multipart.rs

@@ -57,7 +57,6 @@ use super::object::{
     object_transaction_fencing_requested, read_object_transaction_epoch_fence, verify_object_transaction_epoch_fence,
 };
 use crate::api::config::storageclass;
-#[cfg(test)]
 use crate::bucket::metadata_sys::ObjectLockConfigState;
 use crate::bucket::quota::reservation;
 use crate::bucket::replication::ReplicationStatusType;
@@ -4114,6 +4113,40 @@ impl crate::storage_api_contracts::multipart::MultipartOperations for SetDisks {
         )
         .await?;
 
+        if !opts.versioned
+            && !crate::bucket::utils::is_meta_bucketname(bucket)
+            && let Some(snapshot) = opts.object_lock_config_snapshot.as_deref()
+            && !matches!(snapshot.state(), ObjectLockConfigState::ConfirmedAbsent)
+        {
+            // Completion replaces the null version, not necessarily the latest
+            // version inspected by the request-level Object Lock check.
+            let current = self
+                .get_object_info(
+                    bucket,
+                    object,
+                    &ObjectOptions {
+                        version_id: Some(Uuid::nil().to_string()),
+                        no_lock: true,
+                        metadata_cache_safe: false,
+                        versioned: opts.versioned,
+                        version_suspended: opts.version_suspended,
+                        ..Default::default()
+                    },
+                )
+                .await;
+            match current {
+                Ok(existing) => {
+                    if check_object_lock_for_deletion_with_state(snapshot.state(), &existing, false)?.is_some() {
+                        return Err(StorageError::PrefixAccessDenied(bucket.to_string(), object.to_string()));
+                    }
+                }
+                // A null delete marker has no protected payload to replace.
+                Err(StorageError::MethodNotAllowed) => {}
+                Err(err) if is_err_object_not_found(&err) || is_err_version_not_found(&err) => {}
+                Err(err) => return Err(err),
+            }
+        }
+
         if opts.data_movement
             && opts.http_preconditions.is_some()
             && let Some(err) = self.check_write_precondition(bucket, object, opts).await

crates/ecstore/src/set_disk/ops/object.rs

@@ -4958,7 +4958,20 @@ impl SetDisks {
                 );
             }
 
-            if let Some(version_id) = opts.version_id.as_deref()
+            // A write without a version id to a key that is not versioning-Enabled
+            // replaces the key's null version in place, so the WORM gate must judge
+            // that version too. The request-layer pre-check reads the latest
+            // version, which is not the one replaced when a UUID version or a
+            // delete marker sits above a retained null version.
+            let worm_gate_version_id = opts.version_id.clone().or_else(|| {
+                (!opts.versioned
+                    && opts
+                        .object_lock_config_snapshot
+                        .as_ref()
+                        .is_some_and(|snapshot| !matches!(snapshot.state(), ObjectLockConfigState::ConfirmedAbsent)))
+                .then(|| Uuid::nil().to_string())
+            });
+            if let Some(version_id) = worm_gate_version_id.as_deref()
                 && !is_meta_bucketname(bucket)
             {
                 let explicit_version_lookup_started = rustfs_io_metrics::put_stage_timer();
@@ -4978,9 +4991,10 @@ impl SetDisks {
                     .await;
                 match current {
                     Ok(existing) => {
-                        let object_lock_config = opts.object_lock_config_snapshot.as_deref().ok_or_else(|| {
-                            Error::other("explicit-version PUT is missing its Object Lock configuration snapshot")
-                        })?;
+                        let object_lock_config = opts
+                            .object_lock_config_snapshot
+                            .as_deref()
+                            .ok_or_else(|| Error::other("destructive PUT is missing its Object Lock configuration snapshot"))?;
                         // The WORM gate protects the locked version from local
                         // overwrites; an authorized replication write passes it
                         // only when the LWW merge below will judge every
@@ -5011,6 +5025,8 @@ impl SetDisks {
                             }
                         }
                     }
+                    // A null delete marker has no protected payload to replace.
+                    Err(StorageError::MethodNotAllowed) if version_id == Uuid::nil().to_string() => {}
                     Err(err) if is_err_object_not_found(&err) || is_err_version_not_found(&err) => {}
                     Err(err) => return Err(err),
                 }
@@ -9304,6 +9320,60 @@ impl SetDisks {
 }
 
 impl SetDisks {
+    /// Judge the version a versionless destructive delete removes.
+    ///
+    /// For a key that is not versioning-`Enabled`, a delete without a version
+    /// id removes the `null` version, while the preceding Object Lock check
+    /// judged the key's latest entry. When a UUID version or delete marker
+    /// sits above a retained `null` version, the two differ and the retained
+    /// version must be judged on its own. The caller holds the object's write
+    /// lock. Plain unversioned keys and versioning-`Enabled` keys keep a single
+    /// lookup: their latest entry is the version removed.
+    ///
+    /// The lookup future is boxed from a synchronous wrapper so it leaves the
+    /// delete futures' frames: `delete_object` already nests the boxed batch
+    /// path, and inlining a second `get_object_info` overflowed the 4 MiB test
+    /// stack on Linux.
+    fn check_null_version_object_lock_delete<'a>(
+        &'a self,
+        bucket: &'a str,
+        object: &'a str,
+        latest: &'a ObjectInfo,
+        opts: &'a ObjectOptions,
+    ) -> impl Future<Output = Result<()>> + Send + 'a {
+        Box::pin(self.check_null_version_object_lock_delete_inner(bucket, object, latest, opts))
+    }
+
+    async fn check_null_version_object_lock_delete_inner(
+        &self,
+        bucket: &str,
+        object: &str,
+        latest: &ObjectInfo,
+        opts: &ObjectOptions,
+    ) -> Result<()> {
+        if is_meta_bucketname(bucket)
+            || opts.version_id.is_some()
+            || set_disk_delete_creates_delete_marker(opts)
+            || opts.transition.expire_restored
+            || latest.version_id.is_none_or(|version| version.is_nil())
+        {
+            return Ok(());
+        }
+        let null_opts = ObjectOptions {
+            version_id: Some(Uuid::nil().to_string()),
+            no_lock: true,
+            metadata_cache_safe: false,
+            ..opts.clone()
+        };
+        match self.get_object_info(bucket, object, &null_opts).await {
+            Ok(existing) => check_object_lock_delete(&self.ctx, bucket, object, &existing, &null_opts).await,
+            // A null delete marker has no protected payload to remove.
+            Err(StorageError::MethodNotAllowed) => Ok(()),
+            Err(err) if is_err_object_not_found(&err) || is_err_version_not_found(&err) => Ok(()),
+            Err(err) => Err(err),
+        }
+    }
+
     /// Read current metadata without cache admission or read-side repair. The
     /// outer batch owner already holds the sorted namespace write-lock set.
     pub(crate) async fn conditional_delete_current_snapshot(
@@ -9338,6 +9408,76 @@ where
     futures::future::try_join_all(probes).await.map(|_| ()).map_err(Error::from)
 }
 
+/// Per-key source-read inputs for the bulk-delete admission loop. The bounded
+/// metadata pre-fetch and the loop itself must resolve these identically, so
+/// both sides derive them here.
+struct BatchDeleteSourceInputs {
+    replication_object_name: String,
+    explicit_null_version: bool,
+    version_id: Option<Uuid>,
+    versioned: bool,
+    version_suspended: bool,
+    check_opts: ObjectOptions,
+    object_lock_check_required: bool,
+    replicate_delete: bool,
+    needs_source_read: bool,
+}
+
+fn batch_delete_source_inputs(
+    bucket: &str,
+    dobj: &ObjectToDelete,
+    opts: &ObjectOptions,
+    delete_config_snapshot: &DeleteReplicationConfigSnapshot,
+    object_lock_config: &Arc<ObjectLockConfigSnapshot>,
+    guarded: bool,
+) -> BatchDeleteSourceInputs {
+    let replication_object_name = decode_dir_object(&dobj.object_name);
+    let explicit_null_version = is_explicit_null_version(dobj.version_id);
+    let version_id = delete_file_info_version_id(dobj.version_id);
+    let (versioned, version_suspended) = delete_config_snapshot
+        .versioning_config()
+        .delete_state(replication_object_name.as_str());
+    let check_opts = ObjectOptions {
+        version_id: dobj.version_id.map(|version_id| version_id.to_string()),
+        versioned,
+        version_suspended,
+        object_lock_delete: opts.object_lock_delete.clone(),
+        object_lock_config_snapshot: Some(Arc::clone(object_lock_config)),
+        no_lock: true,
+        suppress_read_repair: guarded,
+        ..Default::default()
+    };
+    // A missing bucket Object Lock configuration does not prove that
+    // persisted object metadata is clean: migrated or corrupt data may
+    // still carry an explicit retention or legal hold. Only a real
+    // delete-marker creation can skip the object-level WORM check.
+    let object_lock_check_required = !is_meta_bucketname(bucket) && !set_disk_delete_creates_delete_marker(&check_opts);
+    let replicate_delete = delete_config_snapshot.has_active_rule(&replication_object_name);
+    let marker_delete = dobj.version_id.is_none() || dobj.synthetic_version_id;
+    let replication_needs_source = replicate_delete
+        && (!marker_delete || delete_config_snapshot.active_delete_marker_rules_require_tags(&replication_object_name));
+    // Guarded explicit-version deletes are rejected before any read and must
+    // not join the pre-fetch wave.
+    let admission_error = guarded && dobj.version_id.is_some();
+    let needs_source_read = !admission_error
+        && (object_lock_check_required
+            || replication_needs_source
+            || opts.tier_delete_journal_api.is_some()
+            || dobj.expected_identity.is_some()
+            || dobj.version_id.is_some());
+    BatchDeleteSourceInputs {
+        replication_object_name,
+        explicit_null_version,
+        version_id,
+        versioned,
+        version_suspended,
+        check_opts,
+        object_lock_check_required,
+        replicate_delete,
+        needs_source_read,
+    }
+}
+
 #[async_trait::async_trait]
 impl crate::storage_api_contracts::object::ObjectOperations for SetDisks {
     type Error = Error;
@@ -9736,7 +9876,7 @@ impl crate::storage_api_contracts::object::ObjectOperations for SetDisks {
         let unique_lock_count = batch.requests.len();
 
         let mut failed_map = HashMap::new();
-        let mut _local_batch_guards: Vec<FastLockGuard> = Vec::with_capacity(batch.requests.len());
+        let mut local_batch_guards: Vec<FastLockGuard> = Vec::with_capacity(batch.requests.len());
         let mut locked_objects = HashSet::new();
 
         // Instance-scoped, not the ambient facade (backlog#1052) — see
@@ -9753,7 +9893,7 @@ impl crate::storage_api_contracts::object::ObjectOperations for SetDisks {
             (failed_map, locked_objects, dist_batch_lock_ids) = self.acquire_dist_delete_object_locks_batch(&batch).await;
         } else {
             let batch_result = self.local_lock_manager.acquire_locks_batch(batch).await;
-            _local_batch_guards = batch_result.guards;
+            local_batch_guards = batch_result.guards;
 
             for key in batch_result.successful_locks {
                 locked_objects.insert(key.object.as_ref().to_string());
@@ -9795,8 +9935,17 @@ impl crate::storage_api_contracts::object::ObjectOperations for SetDisks {
             // Every raw batch lock is acquired first. Canonical sorting also
             // gives aliases of one disk key exactly one transaction owner.
             let keys: std::collections::BTreeSet<_> = locked_objects.iter().cloned().collect();
-            for key in keys {
-                match self.acquire_delete_transaction_lock(bucket, &key).await {
+            // Bulk delete is the only multi-key transaction-lock holder; one
+            // batched lock round replaces one quorum round-trip per key and is
+            // deadlock-free by construction (a single `lock_batch` RPC per lock
+            // client, global (shard, key) order at each server). Per-key
+            // failures keep the set layer's per-key error reporting.
+            let key_names: Vec<&str> = keys.iter().map(|key| key.as_str()).collect();
+            let acquired = self
+                .acquire_delete_transaction_locks_batch(bucket, &key_names, super::super::get_lock_acquire_timeout())
+                .await;
+            for (key, result) in keys.into_iter().zip(acquired) {
+                match result {
                     Ok(guard) => {
                         opts.add_namespace_lock_guard(&guard);
                         transaction_guards.insert(key, guard);
@@ -9832,6 +9981,54 @@ impl crate::storage_api_contracts::object::ObjectOperations for SetDisks {
                 },
             }
         };
+        // Per-key read inputs are resolved exactly once; the pre-fetch and the
+        // admission loop below consume this table so the two cannot disagree
+        // on eligibility or read options.
+        let mut source_inputs: Vec<Option<BatchDeleteSourceInputs>> = Vec::with_capacity(objects.len());
+        for dobj in &objects {
+            let guarded = current_generations.is_some_and(|generations| generations.contains_key(&dobj.object_name));
+            source_inputs.push(Some(batch_delete_source_inputs(
+                bucket,
+                dobj,
+                &opts,
+                &delete_config_snapshot,
+                &object_lock_config,
+                guarded,
+            )));
+        }
+        // Bounded-parallel pre-fetch of the source metadata each admitted key
+        // needs below. These reads are pure quorum lookups under the locks
+        // already held; sequencing one metadata fan-out per key made a
+        // 1000-key batch pay a full round of drive reads at a time.
+        const DELETE_OBJECTS_SOURCE_READ_CONCURRENCY: usize = 16;
+        let mut source_reads: HashMap<usize, (ObjectInfo, FileInfo, Option<Error>)> = HashMap::with_capacity(objects.len());
+        {
+            let mut pending = Vec::new();
+            for (i, source) in source_inputs.iter().enumerate() {
+                if del_errs[i].is_some() {
+                    continue;
+                }
+                let Some(source) = source else {
+                    continue;
+                };
+                if !source.needs_source_read {
+                    continue;
+                }
+                let object_name = objects[i].object_name.clone();
+                let check_opts = &source.check_opts;
+                pending.push(async move {
+                    let (goi, file_info, _write_quorum, gerr) = self
+                        .get_object_info_fileinfo_and_quorum(bucket, &object_name, check_opts)
+                        .await;
+                    (i, (goi, file_info, gerr))
+                });
+            }
+            let reads = futures::stream::iter(pending)
+                .buffer_unordered(DELETE_OBJECTS_SOURCE_READ_CONCURRENCY)
+                .collect::<Vec<_>>()
+                .await;
+            source_reads.extend(reads);
+        }
         let mut vers_map: HashMap<&String, FileInfoVersions> = HashMap::new();
         let mut tier_reference_leases: Vec<(usize, String, Option<TierDestinationId>)> = Vec::new();
         let mut tier_free_version_receipt_candidates: HashMap<usize, TierFreeVersionReceiptCandidate> = HashMap::new();
@@ -9853,41 +10050,30 @@ impl crate::storage_api_contracts::object::ObjectOperations for SetDisks {
                 ));
                 continue;
             }
-            let replication_object_name = decode_dir_object(&dobj.object_name);
-            let explicit_null_version = is_explicit_null_version(dobj.version_id);
-            let version_id = delete_file_info_version_id(dobj.version_id);
-            let (versioned, version_suspended) = delete_config_snapshot
-                .versioning_config()
-                .delete_state(replication_object_name.as_str());
-            let check_opts = ObjectOptions {
-                version_id: dobj.version_id.map(|version_id| version_id.to_string()),
+            let Some(source) = source_inputs[i].take() else {
+                continue;
+            };
+            let BatchDeleteSourceInputs {
+                replication_object_name,
+                explicit_null_version,
+                version_id,
                 versioned,
                 version_suspended,
-                object_lock_delete: opts.object_lock_delete.clone(),
-                object_lock_config_snapshot: Some(Arc::clone(&object_lock_config)),
-                no_lock: true,
-                suppress_read_repair: guarded,
-                ..Default::default()
-            };
-            // A missing bucket Object Lock configuration does not prove that
-            // persisted object metadata is clean: migrated or corrupt data may
-            // still carry an explicit retention or legal hold. Only a real
-            // delete-marker creation can skip the object-level WORM check.
-            let object_lock_check_required = !is_meta_bucketname(bucket) && !set_disk_delete_creates_delete_marker(&check_opts);
-            let replicate_delete = delete_config_snapshot.has_active_rule(&replication_object_name);
-            let marker_delete = dobj.version_id.is_none() || dobj.synthetic_version_id;
-            let replication_needs_source = replicate_delete
-                && (!marker_delete || delete_config_snapshot.active_delete_marker_rules_require_tags(&replication_object_name));
-            let (mut goi, mut authoritative_file_info, mut gerr) = if object_lock_check_required
-                || replication_needs_source
-                || opts.tier_delete_journal_api.is_some()
-                || dobj.expected_identity.is_some()
-                || dobj.version_id.is_some()
-            {
-                let (goi, file_info, _write_quorum, gerr) = self
-                    .get_object_info_fileinfo_and_quorum(bucket, &dobj.object_name, &check_opts)
-                    .await;
-                (goi, file_info, gerr)
+                check_opts,
+                object_lock_check_required,
+                replicate_delete,
+                needs_source_read,
+            } = source;
+            let (mut goi, mut authoritative_file_info, mut gerr) = if needs_source_read {
+                match source_reads.remove(&i) {
+                    Some(read) => read,
+                    None => {
+                        let (goi, file_info, _write_quorum, gerr) = self
+                            .get_object_info_fileinfo_and_quorum(bucket, &dobj.object_name, &check_opts)
+                            .await;
+                        (goi, file_info, gerr)
+                    }
+                }
             } else {
                 (ObjectInfo::default(), FileInfo::default(), None)
             };
@@ -9904,6 +10090,17 @@ impl crate::storage_api_contracts::object::ObjectOperations for SetDisks {
                 && !guarded
                 && goi.name.is_empty()
             {
+                // Recovery can restore or retire versions of this key
+                // (rolling back an interrupted prepared delete brings a
+                // retained version back). A pre-fetched read for a later
+                // duplicate of the same name would still report the stale
+                // miss and skip its object-lock check — bypassing retention —
+                // so evict those entries and let the loop re-read them live.
+                for (other_index, other) in objects.iter().enumerate() {
+                    if other_index != i && other.object_name == dobj.object_name {
+                        source_reads.remove(&other_index);
+                    }
+                }
                 let recovery = self
                     .recover_delete_key(
                         bucket,
@@ -10022,6 +10219,17 @@ impl crate::storage_api_contracts::object::ObjectOperations for SetDisks {
                 del_errs[i] = Some(err);
                 continue;
             }
+            // Not gated on `source_missing`: with a delete marker on top the
+            // latest-version lookup reports the key as missing while the null
+            // version below it may still be retained.
+            if object_lock_check_required
+                && let Err(err) = self
+                    .check_null_version_object_lock_delete(bucket, &dobj.object_name, &goi, &check_opts)
+                    .await
+            {
+                del_errs[i] = Some(err);
+                continue;
+            }
 
             if opts.tier_delete_journal_api.is_some()
                 && !source_missing
@@ -10477,6 +10685,10 @@ impl crate::storage_api_contracts::object::ObjectOperations for SetDisks {
             // response can hide a persisted decision and must never roll back.
             let groups = groups.collect::<Vec<_>>();
             published_groups.extend(groups.iter().map(|index| vers[*index].name.clone()));
+            // Journal publish fans out per object per drive; keep enough
+            // publications in flight to feed the drives' durable-mutation
+            // lanes (8 workers each) across a 16-drive set.
+            const DELETE_JOURNAL_PUBLISH_CONCURRENCY: usize = 32;
             let publications = futures::stream::iter(groups.into_iter().map(|index: usize| {
                 let group = &vers[index];
                 let recovery_ticket = recovery_tickets[index].clone();
@@ -10512,7 +10724,7 @@ impl crate::storage_api_contracts::object::ObjectOperations for SetDisks {
                     (index, result)
                 }
             }))
-            .buffer_unordered(16)
+            .buffer_unordered(DELETE_JOURNAL_PUBLISH_CONCURRENCY)
             .collect::<Vec<_>>()
             .await;
             for (index, result) in publications {
@@ -10652,9 +10864,14 @@ impl crate::storage_api_contracts::object::ObjectOperations for SetDisks {
         // Deferred recovery takes fresh raw and transaction locks. Discard the
         // completed leases' deadlines while retaining every caller-owned fence.
         opts.namespace_lock_fence = outer_namespace_lock_fence;
-        // Release inner owners before reacquiring any raw key lock.
+        // Release inner owners before reacquiring any raw key lock. The raw
+        // batch guards must go too: deferred recovery takes a fresh write lock
+        // on the same keys below, and a still-held local batch guard would
+        // deadlock it against itself (the distributed path releases its batch
+        // ids in the same region).
         drop(namespace_owner);
         drop(transaction_guards);
+        drop(local_batch_guards);
 
         // TODO(backlog): support partial object deletion for multi-part objects
 
@@ -11094,6 +11311,8 @@ impl crate::storage_api_contracts::object::ObjectOperations for SetDisks {
             opts.precondition_check(&goi)?;
             check_object_lock_delete(&self.ctx, bucket, object, &goi, &opts).await?;
         }
+        self.check_null_version_object_lock_delete(bucket, object, &goi, &opts)
+            .await?;
         let delete_marker_purge_candidate = opts
             .version_id
             .as_deref()
@@ -24992,6 +25211,131 @@ mod delete_objects_lock_gating_tests {
             .expect("plain object should be written");
     }
 
+    /// Regression for the PR9155 review finding: a pre-fetched source-missing
+    /// read of a later duplicate must not skip the object-lock check after an
+    /// earlier entry's recovery rolls back an interrupted prepared delete and
+    /// restores a GOVERNANCE-retained version. Without the eviction the later
+    /// duplicate deletes the restored version, bypassing retention.
+    #[tokio::test]
+    async fn duplicate_entries_recheck_object_lock_after_recovery_restores_retained_version() {
+        let (_dirs, disk_stores, set_disks) = hermetic_set_disks(4).await;
+        let bucket = "recovery-retention-dup";
+        let object = "retained.bin";
+        for disk in &disk_stores {
+            disk.make_volume(bucket).await.expect("create bucket");
+        }
+
+        // A versioned put, so the batch can address the version explicitly.
+        let mut reader = PutObjReader::from_vec(vec![7u8; 256]);
+        set_disks
+            .put_object(
+                bucket,
+                object,
+                &mut reader,
+                &ObjectOptions {
+                    versioned: true,
+                    no_lock: true,
+                    ..Default::default()
+                },
+            )
+            .await
+            .expect("versioned object should be written");
+        let version = set_disks
+            .get_object_info(bucket, object, &ObjectOptions::default())
+            .await
+            .expect("read back the version identity")
+            .version_id
+            .expect("versioned put assigns a version id");
+
+        // GOVERNANCE retention on that version, persisted the way
+        // PutObjectRetention persists it.
+        let retain_until = OffsetDateTime::now_utc() + time::Duration::days(60);
+        set_disks
+            .put_object_metadata(
+                bucket,
+                object,
+                &ObjectOptions {
+                    no_lock: true,
+                    version_id: Some(version.to_string()),
+                    eval_metadata: Some(HashMap::from([
+                        (metadata_keys::OBJECT_LOCK_MODE.to_string(), "GOVERNANCE".to_string()),
+                        (
+                            metadata_keys::OBJECT_LOCK_RETAIN_UNTIL_DATE.to_string(),
+                            retain_until
+                                .format(&time::format_description::well_known::Rfc3339)
+                                .expect("retention date formats"),
+                        ),
+                    ])),
+                    ..Default::default()
+                },
+            )
+            .await
+            .expect("governance retention should persist");
+
+        // Interrupted authorized delete: the recoverable per-disk delete
+        // applies the version removal and leaves the prepared journal
+        // undecided — exactly a crash between apply and publish.
+        let transaction = Uuid::new_v4();
+        for disk in &disk_stores {
+            let fi = disk
+                .read_version("", bucket, object, &version.to_string(), &crate::disk::ReadOptions::default())
+                .await
+                .expect("version exists before the interrupted delete");
+            let errors = disk
+                .delete_versions_recoverable(
+                    bucket,
+                    vec![FileInfoVersions {
+                        name: object.to_string(),
+                        versions: vec![fi],
+                        ..Default::default()
+                    }],
+                    DeleteOptions {
+                        recoverable_delete: true,
+                        old_data_dir: Some(transaction),
+                        ..Default::default()
+                    },
+                )
+                .await;
+            assert!(errors.iter().all(Option::is_none), "the plant must apply: {errors:?}");
+        }
+
+        // A no-bypass batch naming the retained version twice: the first
+        // entry rolls the prepared delete back (restoring the retained
+        // version) and must be denied; the second must re-read and stay
+        // denied too instead of reusing the stale pre-fetched miss.
+        let victim = ObjectToDelete {
+            object_name: object.to_string(),
+            version_id: Some(version),
+            ..Default::default()
+        };
+        let (_deleted, errors) = set_disks
+            .delete_objects(
+                bucket,
+                vec![victim.clone(), victim],
+                ObjectOptions {
+                    versioned: true,
+                    object_lock_config_snapshot: Some(Arc::new(ObjectLockConfigSnapshot::new(
+                        ObjectLockConfigState::ConfirmedAbsent,
+                    ))),
+                    ..Default::default()
+                },
+            )
+            .await;
+        assert!(
+            errors.iter().all(|error| error
+                .as_ref()
+                .is_some_and(|error| matches!(error, StorageError::PrefixAccessDenied(_, _)))),
+            "both duplicates must be denied by the restored retention: {errors:?}"
+        );
+
+        // The retained version survives the batch.
+        for disk in &disk_stores {
+            disk.read_version("", bucket, object, &version.to_string(), &crate::disk::ReadOptions::default())
+                .await
+                .expect("the retained version must survive the batch");
+        }
+    }
+
     #[tokio::test]
     #[serial_test::serial(shard_integrity_rollout)]
     async fn conditional_delete_snapshot_preserves_multipart_etag() {

crates/ecstore/src/store/init.rs

@@ -27352,4 +27352,282 @@ mod tests {
             .expect("recovery must preserve the local source object");
         assert_eq!(source_after.size, i64::try_from(payload.len()).expect("payload length should fit i64"));
     }
+
+    #[cfg(feature = "test-util")]
+    async fn seed_retained_null_version(
+        store: &Arc<crate::store::ECStore>,
+        bucket: &str,
+        object: &str,
+        protection: &str,
+        latest_is_marker: bool,
+    ) {
+        let metadata = if protection == "legal-hold" {
+            HashMap::from([("x-amz-object-lock-legal-hold".to_string(), "ON".to_string())])
+        } else {
+            HashMap::from([
+                ("x-amz-object-lock-mode".to_string(), protection.to_string()),
+                ("x-amz-object-lock-retain-until-date".to_string(), "2099-01-01T00:00:00Z".to_string()),
+            ])
+        };
+        let mut original = PutObjReader::from_vec(b"protected null version".to_vec());
+        store
+            .put_object(
+                bucket,
+                object,
+                &mut original,
+                &ObjectOptions {
+                    user_defined: metadata,
+                    mod_time: Some(OffsetDateTime::now_utc() - time::Duration::days(1)),
+                    ..Default::default()
+                },
+            )
+            .await
+            .expect("seed retained null version");
+        let mut latest = PutObjReader::from_vec(b"unprotected latest version".to_vec());
+        store
+            .put_object(
+                bucket,
+                object,
+                &mut latest,
+                &ObjectOptions {
+                    versioned: true,
+                    ..Default::default()
+                },
+            )
+            .await
+            .expect("seed newer unprotected UUID version");
+        if latest_is_marker {
+            store
+                .delete_object(
+                    bucket,
+                    object,
+                    ObjectOptions {
+                        versioned: true,
+                        ..Default::default()
+                    },
+                )
+                .await
+                .expect("create a UUID delete marker above the retained null version");
+        }
+    }
+
+    #[cfg(feature = "test-util")]
+    async fn assert_retained_null_version_body(store: &Arc<crate::store::ECStore>, bucket: &str, object: &str) {
+        let mut reader = store
+            .get_object_reader(
+                bucket,
+                object,
+                None,
+                HeaderMap::new(),
+                &ObjectOptions {
+                    version_id: Some(Uuid::nil().to_string()),
+                    ..Default::default()
+                },
+            )
+            .await
+            .expect("retained null version must remain readable");
+        let mut body = Vec::new();
+        reader
+            .stream
+            .read_to_end(&mut body)
+            .await
+            .expect("read retained null version");
+        assert_eq!(body, b"protected null version");
+    }
+
+    /// A versionless delete removes the null version, so the Object Lock check
+    /// must judge that version even when a UUID version or delete marker is
+    /// the key's latest entry (rustfs/backlog#3017).
+    #[cfg(feature = "test-util")]
+    #[tokio::test]
+    #[serial_test::serial(storage_class_env)]
+    async fn null_version_object_lock_delete_checks_actual_target() {
+        use crate::bucket::replication::DeleteReplicationConfigSnapshot;
+        use s3s::dto::{BucketVersioningStatus, ExcludedPrefix, VersioningConfiguration};
+
+        let temp = tempfile::tempdir().expect("create null-version delete store");
+        let (_ctx, store, _shutdown) =
+            without_storage_class_env(build_isolated_test_store(temp.path(), "null-version-delete-lock", &[4])).await;
+        crate::bucket::metadata_sys::init_bucket_metadata_sys(store.clone(), Vec::new()).await;
+        let bucket = format!("null-delete-lock-{}", Uuid::new_v4());
+        store
+            .make_bucket(
+                &bucket,
+                &MakeBucketOptions {
+                    lock_enabled: true,
+                    ..Default::default()
+                },
+            )
+            .await
+            .expect("create Object Lock bucket");
+        let configs = [
+            VersioningConfiguration::default(),
+            VersioningConfiguration {
+                status: Some(BucketVersioningStatus::from_static(BucketVersioningStatus::SUSPENDED)),
+                ..Default::default()
+            },
+            VersioningConfiguration {
+                status: Some(BucketVersioningStatus::from_static(BucketVersioningStatus::ENABLED)),
+                excluded_prefixes: Some(vec![ExcludedPrefix {
+                    prefix: Some("archive/".to_string()),
+                }]),
+                ..Default::default()
+            },
+        ];
+        for (case, config) in configs.into_iter().enumerate() {
+            let snapshot = Arc::new(DeleteReplicationConfigSnapshot::from_configs_for_test(config, None));
+            for protection in ["COMPLIANCE", "GOVERNANCE", "legal-hold"] {
+                for latest_is_marker in [false, true] {
+                    let object = format!("archive/{case}-{protection}-{latest_is_marker}");
+                    seed_retained_null_version(&store, &bucket, &object, protection, latest_is_marker).await;
+                    let (_, errors) = store
+                        .delete_objects(
+                            &bucket,
+                            vec![ObjectToDelete {
+                                object_name: object.clone(),
+                                ..Default::default()
+                            }],
+                            ObjectOptions {
+                                delete_replication_config_snapshot: Some(Arc::clone(&snapshot)),
+                                ..Default::default()
+                            },
+                        )
+                        .await;
+                    assert!(
+                        matches!(errors.as_slice(), [Some(StorageError::PrefixAccessDenied(_, _))]),
+                        "batch delete must judge the retained null version: {object}: {errors:?}"
+                    );
+                    let error = store
+                        .delete_object(
+                            &bucket,
+                            &object,
+                            ObjectOptions {
+                                version_suspended: true,
+                                ..Default::default()
+                            },
+                        )
+                        .await
+                        .expect_err("suspended single delete must also judge the retained null version");
+                    assert!(matches!(error, StorageError::PrefixAccessDenied(_, _)), "{object}: {error:?}");
+                    assert_retained_null_version_body(&store, &bucket, &object).await;
+                    if protection == "GOVERNANCE" {
+                        let (_, errors) = store
+                            .delete_objects(
+                                &bucket,
+                                vec![ObjectToDelete {
+                                    object_name: object.clone(),
+                                    ..Default::default()
+                                }],
+                                ObjectOptions {
+                                    delete_replication_config_snapshot: Some(Arc::clone(&snapshot)),
+                                    object_lock_delete: Some(crate::storage_api_contracts::object::ObjectLockDeleteOptions {
+                                        bypass_governance: true,
+                                    }),
+                                    ..Default::default()
+                                },
+                            )
+                            .await;
+                        assert!(errors.iter().all(Option::is_none), "authorized governance bypass: {errors:?}");
+                        assert!(
+                            store
+                                .get_object_reader(
+                                    &bucket,
+                                    &object,
+                                    None,
+                                    HeaderMap::new(),
+                                    &ObjectOptions {
+                                        version_id: Some(Uuid::nil().to_string()),
+                                        ..Default::default()
+                                    }
+                                )
+                                .await
+                                .is_err(),
+                            "authorized deletion must make the retained null payload unreadable"
+                        );
+                    }
+                }
+            }
+        }
+    }
+
+    /// A versionless PUT or multipart completion replaces the null version in
+    /// place, so the commit-time WORM gate must judge that version even when a
+    /// UUID version or delete marker is the key's latest entry
+    /// (rustfs/backlog#3019).
+    #[cfg(feature = "test-util")]
+    #[tokio::test]
+    #[serial_test::serial(storage_class_env)]
+    async fn null_version_object_lock_put_and_multipart_preserve_retained_target() {
+        let temp = tempfile::tempdir().expect("create null-version overwrite store");
+        let (_ctx, store, _shutdown) =
+            without_storage_class_env(build_isolated_test_store(temp.path(), "null-version-overwrite-lock", &[4])).await;
+        crate::bucket::metadata_sys::init_bucket_metadata_sys(store.clone(), Vec::new()).await;
+        let bucket = format!("null-overwrite-lock-{}", Uuid::new_v4());
+        store
+            .make_bucket(
+                &bucket,
+                &MakeBucketOptions {
+                    lock_enabled: true,
+                    ..Default::default()
+                },
+            )
+            .await
+            .expect("create Object Lock bucket");
+        for protection in ["COMPLIANCE", "GOVERNANCE", "legal-hold"] {
+            for latest_is_marker in [false, true] {
+                let object = format!("{protection}-{latest_is_marker}");
+                seed_retained_null_version(&store, &bucket, &object, protection, latest_is_marker).await;
+                let mut replacement = PutObjReader::from_vec(b"replacement".to_vec());
+                let error = store
+                    .put_object(&bucket, &object, &mut replacement, &ObjectOptions::default())
+                    .await
+                    .expect_err("PUT must reject replacement of a retained null version");
+                assert!(matches!(error, StorageError::PrefixAccessDenied(_, _)), "{object}: {error:?}");
+
+                let upload = store
+                    .clone()
+                    .new_multipart_upload(&bucket, &object, &ObjectOptions::default())
+                    .await
+                    .expect("initiate multipart upload");
+                let mut part_reader = PutObjReader::from_vec(b"multipart replacement".to_vec());
+                let part = store
+                    .clone()
+                    .put_object_part(&bucket, &object, &upload.upload_id, 1, &mut part_reader, &ObjectOptions::default())
+                    .await
+                    .expect("stage multipart replacement");
+                let error = store
+                    .clone()
+                    .complete_multipart_upload(
+                        &bucket,
+                        &object,
+                        &upload.upload_id,
+                        vec![crate::storage_api_contracts::multipart::CompletePart {
+                            part_num: part.part_num,
+                            etag: part.etag,
+                            ..Default::default()
+                        }],
+                        &ObjectOptions::default(),
+                    )
+                    .await
+                    .expect_err("multipart completion must reject replacement of a retained null version");
+                assert!(matches!(error, StorageError::PrefixAccessDenied(_, _)), "{object}: {error:?}");
+                assert_retained_null_version_body(&store, &bucket, &object).await;
+
+                let mut next_version = PutObjReader::from_vec(b"new UUID version".to_vec());
+                store
+                    .put_object(
+                        &bucket,
+                        &object,
+                        &mut next_version,
+                        &ObjectOptions {
+                            versioned: true,
+                            ..Default::default()
+                        },
+                    )
+                    .await
+                    .expect("a new UUID version must remain allowed");
+                assert_retained_null_version_body(&store, &bucket, &object).await;
+            }
+        }
+    }
 }

crates/ecstore/src/store/multipart.rs

@@ -1210,7 +1210,34 @@ impl ECStore {
         opts: &ObjectOptions,
     ) -> Result<ObjectInfo> {
         check_complete_multipart_args(bucket, object, upload_id)?;
+        let encoded_object = encode_dir_object_ref(object);
+        let object = encoded_object.as_ref();
         let (mut opts, _bucket_lifecycle_guard) = self.guard_multipart_bucket_incarnation(bucket, opts).await?;
+        if opts.overwrites_existing_version() && !is_meta_bucketname(bucket) {
+            // The set-level commit judges the null version this completion
+            // replaces; give it the same authoritative snapshot a PUT carries.
+            let expected_incarnation_id = opts
+                .expected_bucket_incarnation_id
+                .ok_or_else(|| Error::other("multipart completion is missing its bucket incarnation"))?;
+            let lifecycle_fence = opts
+                .bucket_lifecycle_lock_fence
+                .as_ref()
+                .ok_or_else(|| Error::other("multipart completion is missing its bucket lifecycle fence"))?;
+            let snapshot = match opts.object_lock_config_snapshot.as_ref() {
+                Some(snapshot) => Arc::clone(snapshot),
+                None => {
+                    self.object_lock_config_snapshot_under_lifecycle_fence(bucket, lifecycle_fence)
+                        .await?
+                }
+            };
+            if !snapshot.is_valid_for_destructive_put(self.id, bucket, expected_incarnation_id) {
+                return Err(Error::other(
+                    "Object Lock snapshot does not hold valid target bucket generation and configuration fences",
+                ));
+            }
+            snapshot.add_lock_fences(&mut opts);
+            opts.object_lock_config_snapshot = Some(snapshot);
+        }
         if !self.single_pool() {
             opts.decommission_capacity_admission = crate::bucket::metadata_sys::object_store_if_initialized_in(&self.ctx).await;
         }

crates/ecstore/src/store/object.rs

@@ -1352,8 +1352,8 @@ impl RestoreWorkerGuard {
     }
 }
 
-impl Drop for ObjectLockDiagGuard {
-    fn drop(&mut self) {
+impl ObjectLockDiagGuard {
+    fn record_hold_diag(&self) {
         if !self.enabled || self.guard.as_ref().is_released() {
             return;
         }
@@ -1378,6 +1378,12 @@ impl Drop for ObjectLockDiagGuard {
     }
 }
 
+impl Drop for ObjectLockDiagGuard {
+    fn drop(&mut self) {
+        self.record_hold_diag();
+    }
+}
+
 /// A failure to preserve one object generation for a SelectObjectContent read.
 #[derive(Debug, thiserror::Error)]
 #[non_exhaustive]
@@ -3467,22 +3473,53 @@ impl ECStore {
         }
     }
 
+    /// Acquire the per-object write locks a bulk delete needs before any
+    /// mutation, one batched lock round for the whole key set. Bulk delete is
+    /// the only multi-key write-lock holder, and the batched round is
+    /// deadlock-free by construction (one `lock_batch` RPC per lock client,
+    /// global key order at each server), so no optimistic wave or serial
+    /// fallback is needed. Outcomes are per key: a locked-out key fails alone
+    /// and the rest of the batch proceeds.
     async fn acquire_delete_objects_write_locks(
         &self,
         bucket: &str,
         objects: &[ObjectToDelete],
         opts: &mut ObjectOptions,
-    ) -> Result<Vec<ObjectLockDiagGuard>> {
+    ) -> Result<(Vec<ObjectLockDiagGuard>, HashMap<String, StorageError>)> {
         if opts.no_lock || objects.is_empty() {
-            return Ok(Vec::new());
+            return Ok((Vec::new(), HashMap::new()));
         }
 
         let object_names = sorted_unique_delete_object_names(objects);
-        // Lock order: encoded object names are acquired in ascending order, then
-        // the set-layer calls receive no_lock so they do not reacquire them.
+        let acquired = self.pools[0].disk_set[0]
+            .acquire_object_write_locks_batch(bucket, &object_names, get_lock_acquire_timeout())
+            .await;
+
+        let diag_enabled = is_object_lock_diag_enabled();
         let mut guards = Vec::with_capacity(object_names.len());
-        for object in object_names {
-            guards.push(self.acquire_object_write_lock("delete_objects", bucket, object).await?);
+        let mut failures = HashMap::new();
+        for (object, result) in object_names.iter().zip(acquired) {
+            match result {
+                Ok(guard) => {
+                    #[cfg(any(test, feature = "test-util"))]
+                    notify_delete_namespace_pending(bucket);
+                    guards.push(
+                        ObjectLockDiagGuard::new(
+                            guard,
+                            diag_enabled,
+                            "delete_objects",
+                            diag_enabled.then(|| bucket.to_string()),
+                            diag_enabled.then(|| (*object).to_string()),
+                            diag_enabled.then(|| self.pools[0].disk_set[0].locker_owner.clone()),
+                            ObjectLockDiagMode::Write,
+                        )
+                        .with_write_key(bucket, object),
+                    );
+                }
+                Err(error) => {
+                    failures.insert((*object).to_owned(), error);
+                }
+            }
         }
         opts.no_lock = true;
         for signal in guards.iter().filter_map(ObjectLockDiagGuard::lock_lost_signal) {
@@ -3490,7 +3527,7 @@ impl ECStore {
         }
         opts.ensure_namespace_lock_fence();
 
-        Ok(guards)
+        Ok((guards, failures))
     }
 
     pub(super) async fn acquire_object_read_lock_if_needed(
@@ -6165,10 +6202,21 @@ impl ECStore {
         if current_bucket_incarnation_id.is_some() {
             pause_delete_after_object_lock_snapshot(bucket).await;
         }
-        let _object_lock_guards = Arc::new(match self.acquire_delete_objects_write_locks(bucket, &objects, &mut opts).await {
-            Ok(guards) => guards,
-            Err(err) => return return_batch_delete_lock_error_with_accounting(objects.as_slice(), err),
-        });
+        let (object_lock_guards, object_lock_failures) =
+            match self.acquire_delete_objects_write_locks(bucket, &objects, &mut opts).await {
+                Ok(result) => result,
+                Err(err) => return return_batch_delete_lock_error_with_accounting(objects.as_slice(), err),
+            };
+        // Lock outcomes are per key: a locked-out key fails alone and the rest
+        // of the batch proceeds, matching the set layer's per-key lock errors.
+        if !object_lock_failures.is_empty() {
+            for (index, object) in objects.iter().enumerate() {
+                if let Some(error) = object_lock_failures.get(&object.object_name) {
+                    del_errs[index] = Some(error.clone());
+                }
+            }
+        }
+        let _object_lock_guards = Arc::new(object_lock_guards);
         opts.delete_mutation_owner = Some(Arc::new((opts.delete_mutation_owner.take(), _object_lock_guards.clone())));
         #[cfg(any(test, feature = "test-util"))]
         if !_object_lock_guards.is_empty() {
@@ -6288,8 +6336,11 @@ impl ECStore {
         inject_conditional_delete_post_admission_mutations(self, bucket).await;
 
         if !has_conditions {
-            let indices = (0..objects.len()).collect::<Vec<_>>();
-            (del_objects, del_errs, accounting) = self
+            // Keys whose write lock failed are already reported in `del_errs`;
+            // the phase only runs for the keys this batch actually holds, and
+            // only its outcomes replace the outer per-key state.
+            let indices = (0..objects.len()).filter(|idx| del_errs[*idx].is_none()).collect::<Vec<_>>();
+            let (phase_deleted, mut phase_errors, mut phase_accounting) = self
                 .run_batch_delete_phase(
                     bucket,
                     &objects,
@@ -6301,6 +6352,11 @@ impl ECStore {
                     },
                 )
                 .await;
+            for idx in indices {
+                del_objects[idx] = phase_deleted[idx].clone();
+                del_errs[idx] = phase_errors[idx].take();
+                accounting[idx] = phase_accounting[idx].take();
+            }
         } else {
             let mut conditional_actions = HashMap::with_capacity(objects.len());
             let mut conditional_indices = Vec::with_capacity(objects.len());
@@ -12944,11 +13000,12 @@ mod tests {
         ];
         let mut opts = ObjectOptions::default();
 
-        let guards = store
+        let (guards, failures) = store
             .acquire_delete_objects_write_locks("bucket", &objects, &mut opts)
             .await
             .expect("delete object locks should be acquired");
 
+        assert!(failures.is_empty(), "uncontended keys must all lock: {failures:?}");
         assert_eq!(guards.len(), 2, "duplicate object names should share one namespace lock");
         assert!(opts.no_lock, "set layer should not reacquire locks already held by ECStore");
         assert!(
@@ -13580,3 +13637,129 @@ mod missing_key_delete_quorum_tests {
         .await;
     }
 }
+
+/// The bulk-delete lock rounds are batched and per key: one `lock_batch` wave
+/// per lock layer replaces one quorum round-trip per key, a locked-out key
+/// fails alone, and the rest of the batch proceeds (rustfs/backlog#3146).
+#[cfg(test)]
+mod batch_delete_lock_tests {
+    use super::*;
+    use crate::bucket::metadata_sys::{init_bucket_metadata_sys, test_support::isolated_store_over_temp_disks};
+    use crate::storage_api_contracts::bucket::{BucketOperations, MakeBucketOptions};
+
+    async fn seeded_store(bucket: &str) -> (Vec<tempfile::TempDir>, Arc<ECStore>) {
+        let (dirs, store) = isolated_store_over_temp_disks().await;
+        init_bucket_metadata_sys(store.clone(), Vec::new()).await;
+        store
+            .make_bucket(bucket, &MakeBucketOptions::default())
+            .await
+            .expect("fixture bucket");
+        for key in ["a-object", "b-object"] {
+            let mut reader = PutObjReader::from_vec(vec![0x53; 4096]);
+            store
+                .put_object(
+                    bucket,
+                    key,
+                    &mut reader,
+                    &ObjectOptions {
+                        write_completion: crate::object_api::WriteCompletion::TailDrained,
+                        ..Default::default()
+                    },
+                )
+                .await
+                .expect("seed object");
+        }
+        (dirs, store)
+    }
+
+    fn two_key_batch() -> Vec<ObjectToDelete> {
+        ["b-object", "a-object"]
+            .into_iter()
+            .map(|object_name| ObjectToDelete {
+                object_name: object_name.to_owned(),
+                ..Default::default()
+            })
+            .collect()
+    }
+
+    #[tokio::test]
+    async fn uncontended_batch_deletes_every_key() {
+        let (_dirs, store) = seeded_store("lock-batch-success").await;
+
+        let (deleted, errors, _) = store
+            .delete_objects_with_tier_delete_journal_and_accounting(
+                "lock-batch-success",
+                two_key_batch(),
+                ObjectOptions::default(),
+            )
+            .await;
+        assert!(errors.iter().all(Option::is_none), "batch outcome: {errors:?}");
+        assert_eq!(deleted.len(), 2);
+    }
+
+    #[tokio::test]
+    async fn contended_key_fails_alone_and_the_rest_of_the_batch_proceeds() {
+        let (_dirs, store) = seeded_store("lock-batch-partial").await;
+
+        // Hold one key's write lock: the batched round must fail that key
+        // alone (per-key outcome) while the other key still deletes.
+        let held = store
+            .acquire_object_write_lock("delete_objects", "lock-batch-partial", "a-object")
+            .await
+            .expect("hold contended key");
+        let (deleted, errors, _) = tokio::time::timeout(
+            Duration::from_secs(30),
+            store.delete_objects_with_tier_delete_journal_and_accounting(
+                "lock-batch-partial",
+                two_key_batch(),
+                ObjectOptions::default(),
+            ),
+        )
+        .await
+        .expect("per-key lock outcomes must not block the whole batch");
+
+        assert!(errors[1].is_some(), "contended a-object must fail alone: {errors:?}");
+        assert!(errors[0].is_none(), "b-object must proceed: {errors:?}");
+        assert!(deleted[1].object_name.is_empty(), "failed key must not report success");
+        assert_eq!(deleted[0].object_name, "b-object");
+        drop(held);
+    }
+
+    #[tokio::test]
+    async fn overlapping_batches_reach_per_key_order_without_deadlock() {
+        let (_dirs, store) = seeded_store("lock-batch-overlap").await;
+
+        let first = store.clone();
+        let second = store.clone();
+        let run = |store: Arc<ECStore>| {
+            tokio::spawn(async move {
+                store
+                    .delete_objects_with_tier_delete_journal_and_accounting(
+                        "lock-batch-overlap",
+                        two_key_batch(),
+                        ObjectOptions::default(),
+                    )
+                    .await
+            })
+        };
+        let first = run(first);
+        let second = run(second);
+
+        // Whatever mix of per-key lock outcomes the overlap produces, both
+        // batches must finish: the batched round is deadlock-free by
+        // construction (global key order at each lock server).
+        for batch in [first, second] {
+            let (_, errors, _) = tokio::time::timeout(Duration::from_secs(30), batch)
+                .await
+                .expect("overlapping batches must not deadlock")
+                .expect("delete task");
+            for error in errors.iter().flatten() {
+                let message = error.to_string();
+                assert!(
+                    message.contains("lock") || message.contains("Lock") || message.contains("namespace"),
+                    "overlap may only surface lock outcomes: {message}"
+                );
+            }
+        }
+    }
+}

crates/iam/src/cache.rs

@@ -15,7 +15,6 @@
 use std::{
     collections::{HashMap, HashSet},
     ops::{Deref, DerefMut},
-    ptr,
     sync::{Arc, Mutex, Weak},
 };
 
@@ -64,21 +63,46 @@ impl Default for CacheState {
 pub struct Cache {
     state: ArcSwap<CacheState>,
     write_lock: Mutex<()>,
+    reload_lock: AsyncMutex<()>,
+    reload_changes: Mutex<Weak<Mutex<ReloadChanges>>>,
     service_account_mutation_lock: AsyncMutex<()>,
-    sts_account_mutation_locks: Arc<StsMutationLockRegistry>,
+    sts_account_mutation_locks: Arc<MutationLockRegistry>,
+    parent_mutation_locks: Arc<MutationLockRegistry>,
 }
 
-struct StsMutationLockRegistry {
-    locks: Mutex<HashMap<String, Weak<AsyncMutex<StsMutationLockState>>>>,
+#[derive(Default)]
+struct MutationLockRegistry {
+    locks: Mutex<HashMap<String, Weak<AsyncMutex<MutationLockState>>>>,
 }
 
-pub(crate) struct StsMutationLockState {
+pub(crate) struct MutationLockState {
     access_key: String,
-    registry: Weak<StsMutationLockRegistry>,
-    lock: Weak<AsyncMutex<StsMutationLockState>>,
+    registry: Weak<MutationLockRegistry>,
+    lock: Weak<AsyncMutex<MutationLockState>>,
 }
 
-impl Drop for StsMutationLockState {
+impl MutationLockRegistry {
+    fn lock_for(self: &Arc<Self>, access_key: &str) -> Arc<AsyncMutex<MutationLockState>> {
+        let mut locks = self.locks.lock().unwrap_or_else(|poisoned| poisoned.into_inner());
+        if let Some(lock) = locks.get(access_key).and_then(Weak::upgrade) {
+            return lock;
+        }
+
+        let registry = Arc::downgrade(self);
+        let access_key_owned = access_key.to_string();
+        let lock = Arc::new_cyclic(|lock| {
+            AsyncMutex::new(MutationLockState {
+                access_key: access_key_owned,
+                registry,
+                lock: lock.clone(),
+            })
+        });
+        locks.insert(access_key.to_string(), Arc::downgrade(&lock));
+        lock
+    }
+}
+
+impl Drop for MutationLockState {
     fn drop(&mut self) {
         let Some(registry) = self.registry.upgrade() else {
             return;
@@ -95,10 +119,52 @@ impl Default for Cache {
         Self {
             state: ArcSwap::new(Arc::new(CacheState::default())),
             write_lock: Mutex::new(()),
+            reload_lock: AsyncMutex::new(()),
+            reload_changes: Mutex::new(Weak::new()),
             service_account_mutation_lock: AsyncMutex::new(()),
-            sts_account_mutation_locks: Arc::new(StsMutationLockRegistry {
-                locks: Mutex::new(HashMap::new()),
-            }),
+            sts_account_mutation_locks: Arc::new(MutationLockRegistry::default()),
+            parent_mutation_locks: Arc::new(MutationLockRegistry::default()),
+        }
+    }
+}
+
+#[derive(Clone, Copy, Hash, PartialEq, Eq)]
+pub(crate) enum CachePartition {
+    PolicyDocs,
+    Users,
+    UserPolicies,
+    StsAccounts,
+    StsPolicies,
+    Groups,
+    UserGroupMemberships,
+    GroupPolicies,
+}
+
+type ReloadChanges = HashMap<CachePartition, HashSet<String>>;
+
+/// A reload journals only keys changed during its own lifetime, including
+/// deletions. Unrelated STS traffic cannot suppress a persisted revocation.
+pub(crate) struct CacheReload<'a> {
+    _guard: tokio::sync::MutexGuard<'a, ()>,
+    changes: Arc<Mutex<ReloadChanges>>,
+}
+
+impl CacheReload<'_> {
+    pub(crate) fn merge_partition<T: Clone>(
+        &self,
+        partition: CachePartition,
+        loaded: &mut CacheEntity<T>,
+        current: &CacheEntity<T>,
+    ) {
+        let changes = self.changes.lock().unwrap_or_else(|poisoned| poisoned.into_inner());
+        if let Some(keys) = changes.get(&partition) {
+            for key in keys {
+                if let Some(value) = current.get(key) {
+                    loaded.insert(key.clone(), value.clone());
+                } else {
+                    loaded.remove(key);
+                }
+            }
         }
     }
 }
@@ -106,31 +172,27 @@ impl Default for Cache {
 pub(crate) type CacheSnapshot = Guard<Arc<CacheState>>;
 
 impl Cache {
+    pub(crate) async fn begin_reload(&self) -> CacheReload<'_> {
+        // Registration and publication both take write_lock before the journal
+        // lock, so no mutation can fall between the reload and its journal.
+        let guard = self.reload_lock.lock().await;
+        let changes = Arc::new(Mutex::new(HashMap::new()));
+        let _write = self.write_lock.lock().unwrap_or_else(|poisoned| poisoned.into_inner());
+        *self.reload_changes.lock().unwrap_or_else(|poisoned| poisoned.into_inner()) = Arc::downgrade(&changes);
+        CacheReload { _guard: guard, changes }
+    }
+
     pub(crate) fn service_account_mutation_lock(&self) -> &AsyncMutex<()> {
         &self.service_account_mutation_lock
     }
 
-    pub(crate) fn sts_account_mutation_lock(&self, access_key: &str) -> Arc<AsyncMutex<StsMutationLockState>> {
-        let mut locks = self
-            .sts_account_mutation_locks
-            .locks
-            .lock()
-            .unwrap_or_else(|poisoned| poisoned.into_inner());
-        if let Some(lock) = locks.get(access_key).and_then(Weak::upgrade) {
-            return lock;
-        }
+    pub(crate) fn sts_account_mutation_lock(&self, access_key: &str) -> Arc<AsyncMutex<MutationLockState>> {
+        self.sts_account_mutation_locks.lock_for(access_key)
+    }
 
-        let registry = Arc::downgrade(&self.sts_account_mutation_locks);
-        let access_key_owned = access_key.to_string();
-        let lock = Arc::new_cyclic(|lock| {
-            AsyncMutex::new(StsMutationLockState {
-                access_key: access_key_owned,
-                registry,
-                lock: lock.clone(),
-            })
-        });
-        locks.insert(access_key.to_string(), Arc::downgrade(&lock));
-        lock
+    /// Orders derived-credential publication with changes to its regular parent.
+    pub(crate) fn parent_mutation_lock(&self, access_key: &str) -> Arc<AsyncMutex<MutationLockState>> {
+        self.parent_mutation_locks.lock_for(access_key)
     }
 
     pub(crate) fn snapshot(&self) -> CacheSnapshot {
@@ -142,7 +204,13 @@ impl Cache {
         let current = self.state.load_full();
         let mut locked = LockedCache {
             state: CacheState::clone(&current),
+            #[cfg(test)]
             current_ptr: Arc::as_ptr(&current),
+            reload_changes: self
+                .reload_changes
+                .lock()
+                .unwrap_or_else(|poisoned| poisoned.into_inner())
+                .upgrade(),
             dirty: false,
         };
         let ret = f(&mut locked);
@@ -215,7 +283,9 @@ impl Cache {
 
 pub(crate) struct LockedCache {
     state: CacheState,
+    #[cfg(test)]
     current_ptr: *const CacheState,
+    reload_changes: Option<Arc<Mutex<ReloadChanges>>>,
     dirty: bool,
 }
 
@@ -224,11 +294,19 @@ impl LockedCache {
         &self.state
     }
 
+    #[cfg(test)]
     pub(crate) fn matches_snapshot(&self, snapshot: &CacheSnapshot) -> bool {
-        ptr::eq(self.current_ptr, Arc::as_ptr(snapshot))
+        std::ptr::eq(self.current_ptr, Arc::as_ptr(snapshot))
     }
 
-    fn exec<T: Clone>(target: &mut Arc<CacheEntity<T>>, t: OffsetDateTime, mut op: impl FnMut(&mut CacheEntity<T>)) -> bool {
+    fn exec<T: Clone>(
+        target: &mut Arc<CacheEntity<T>>,
+        t: OffsetDateTime,
+        changes: Option<&Arc<Mutex<ReloadChanges>>>,
+        partition: CachePartition,
+        key: &str,
+        mut op: impl FnMut(&mut CacheEntity<T>),
+    ) -> bool {
         if target.load_time >= t {
             return false;
         }
@@ -236,6 +314,14 @@ impl LockedCache {
         let mut new = CacheEntity::clone(target);
         op(&mut new);
         *target = Arc::new(new);
+        if let Some(changes) = changes {
+            changes
+                .lock()
+                .unwrap_or_else(|poisoned| poisoned.into_inner())
+                .entry(partition)
+                .or_default()
+                .insert(key.to_string());
+        }
         true
     }
 
@@ -244,139 +330,315 @@ impl LockedCache {
     }
 
     pub(crate) fn replace_policy_docs(&mut self, value: CacheEntity<PolicyDoc>) {
+        if let Some(changes) = &self.reload_changes {
+            changes
+                .lock()
+                .unwrap_or_else(|poisoned| poisoned.into_inner())
+                .entry(CachePartition::PolicyDocs)
+                .or_default()
+                .extend(self.state.policy_docs.keys().chain(value.keys()).cloned());
+        }
         self.state.policy_docs = Self::replaced(value);
         self.dirty = true;
     }
 
     pub(crate) fn replace_users(&mut self, value: CacheEntity<UserIdentity>) {
+        if let Some(changes) = &self.reload_changes {
+            changes
+                .lock()
+                .unwrap_or_else(|poisoned| poisoned.into_inner())
+                .entry(CachePartition::Users)
+                .or_default()
+                .extend(self.state.users.keys().chain(value.keys()).cloned());
+        }
         self.state.users = Self::replaced(value);
         self.dirty = true;
     }
 
     pub(crate) fn replace_user_policies(&mut self, value: CacheEntity<MappedPolicy>) {
+        if let Some(changes) = &self.reload_changes {
+            changes
+                .lock()
+                .unwrap_or_else(|poisoned| poisoned.into_inner())
+                .entry(CachePartition::UserPolicies)
+                .or_default()
+                .extend(self.state.user_policies.keys().chain(value.keys()).cloned());
+        }
         self.state.user_policies = Self::replaced(value);
         self.dirty = true;
     }
 
     pub(crate) fn replace_sts_accounts(&mut self, value: CacheEntity<UserIdentity>) {
+        if let Some(changes) = &self.reload_changes {
+            changes
+                .lock()
+                .unwrap_or_else(|poisoned| poisoned.into_inner())
+                .entry(CachePartition::StsAccounts)
+                .or_default()
+                .extend(self.state.sts_accounts.keys().chain(value.keys()).cloned());
+        }
         self.state.sts_accounts = Self::replaced(value);
         self.dirty = true;
     }
 
     pub(crate) fn replace_sts_policies(&mut self, value: CacheEntity<MappedPolicy>) {
+        if let Some(changes) = &self.reload_changes {
+            changes
+                .lock()
+                .unwrap_or_else(|poisoned| poisoned.into_inner())
+                .entry(CachePartition::StsPolicies)
+                .or_default()
+                .extend(self.state.sts_policies.keys().chain(value.keys()).cloned());
+        }
         self.state.sts_policies = Self::replaced(value);
         self.dirty = true;
     }
 
     pub(crate) fn replace_groups(&mut self, value: CacheEntity<GroupInfo>) {
+        if let Some(changes) = &self.reload_changes {
+            changes
+                .lock()
+                .unwrap_or_else(|poisoned| poisoned.into_inner())
+                .entry(CachePartition::Groups)
+                .or_default()
+                .extend(self.state.groups.keys().chain(value.keys()).cloned());
+        }
         self.state.groups = Self::replaced(value);
         self.dirty = true;
     }
 
     pub(crate) fn replace_group_policies(&mut self, value: CacheEntity<MappedPolicy>) {
+        if let Some(changes) = &self.reload_changes {
+            changes
+                .lock()
+                .unwrap_or_else(|poisoned| poisoned.into_inner())
+                .entry(CachePartition::GroupPolicies)
+                .or_default()
+                .extend(self.state.group_policies.keys().chain(value.keys()).cloned());
+        }
         self.state.group_policies = Self::replaced(value);
         self.dirty = true;
     }
 
     pub(crate) fn replace_user_group_memberships(&mut self, value: CacheEntity<HashSet<String>>) {
+        if let Some(changes) = &self.reload_changes {
+            changes
+                .lock()
+                .unwrap_or_else(|poisoned| poisoned.into_inner())
+                .entry(CachePartition::UserGroupMemberships)
+                .or_default()
+                .extend(self.state.user_group_memberships.keys().chain(value.keys()).cloned());
+        }
         self.state.user_group_memberships = Self::replaced(value);
         self.dirty = true;
     }
 
     pub(crate) fn add_or_update_policy_doc(&mut self, key: &str, value: &PolicyDoc, t: OffsetDateTime) {
-        self.dirty |= Self::exec(&mut self.state.policy_docs, t, |map| {
-            map.insert(key.to_string(), value.clone());
-        });
+        self.dirty |= Self::exec(
+            &mut self.state.policy_docs,
+            t,
+            self.reload_changes.as_ref(),
+            CachePartition::PolicyDocs,
+            key,
+            |map| {
+                map.insert(key.to_string(), value.clone());
+            },
+        );
     }
 
     pub(crate) fn add_or_update_user(&mut self, key: &str, value: &UserIdentity, t: OffsetDateTime) {
-        self.dirty |= Self::exec(&mut self.state.users, t, |map| {
-            map.insert(key.to_string(), value.clone());
-        });
+        self.dirty |= Self::exec(
+            &mut self.state.users,
+            t,
+            self.reload_changes.as_ref(),
+            CachePartition::Users,
+            key,
+            |map| {
+                map.insert(key.to_string(), value.clone());
+            },
+        );
     }
 
     pub(crate) fn add_or_update_user_policy(&mut self, key: &str, value: &MappedPolicy, t: OffsetDateTime) {
-        self.dirty |= Self::exec(&mut self.state.user_policies, t, |map| {
-            map.insert(key.to_string(), value.clone());
-        });
+        self.dirty |= Self::exec(
+            &mut self.state.user_policies,
+            t,
+            self.reload_changes.as_ref(),
+            CachePartition::UserPolicies,
+            key,
+            |map| {
+                map.insert(key.to_string(), value.clone());
+            },
+        );
     }
 
     pub(crate) fn add_or_update_sts_account(&mut self, key: &str, value: &UserIdentity, t: OffsetDateTime) {
-        self.dirty |= Self::exec(&mut self.state.sts_accounts, t, |map| {
-            map.insert(key.to_string(), value.clone());
-        });
+        self.dirty |= Self::exec(
+            &mut self.state.sts_accounts,
+            t,
+            self.reload_changes.as_ref(),
+            CachePartition::StsAccounts,
+            key,
+            |map| {
+                map.insert(key.to_string(), value.clone());
+            },
+        );
     }
 
     pub(crate) fn add_or_update_sts_policy(&mut self, key: &str, value: &MappedPolicy, t: OffsetDateTime) {
-        self.dirty |= Self::exec(&mut self.state.sts_policies, t, |map| {
-            map.insert(key.to_string(), value.clone());
-        });
+        self.dirty |= Self::exec(
+            &mut self.state.sts_policies,
+            t,
+            self.reload_changes.as_ref(),
+            CachePartition::StsPolicies,
+            key,
+            |map| {
+                map.insert(key.to_string(), value.clone());
+            },
+        );
     }
 
     pub(crate) fn add_or_update_group(&mut self, key: &str, value: &GroupInfo, t: OffsetDateTime) {
-        self.dirty |= Self::exec(&mut self.state.groups, t, |map| {
-            map.insert(key.to_string(), value.clone());
-        });
+        self.dirty |= Self::exec(
+            &mut self.state.groups,
+            t,
+            self.reload_changes.as_ref(),
+            CachePartition::Groups,
+            key,
+            |map| {
+                map.insert(key.to_string(), value.clone());
+            },
+        );
     }
 
     pub(crate) fn add_or_update_user_group_membership(&mut self, key: &str, value: &HashSet<String>, t: OffsetDateTime) {
-        self.dirty |= Self::exec(&mut self.state.user_group_memberships, t, |map| {
-            map.insert(key.to_string(), value.clone());
-        });
+        self.dirty |= Self::exec(
+            &mut self.state.user_group_memberships,
+            t,
+            self.reload_changes.as_ref(),
+            CachePartition::UserGroupMemberships,
+            key,
+            |map| {
+                map.insert(key.to_string(), value.clone());
+            },
+        );
     }
 
     pub(crate) fn add_or_update_group_policy(&mut self, key: &str, value: &MappedPolicy, t: OffsetDateTime) {
-        self.dirty |= Self::exec(&mut self.state.group_policies, t, |map| {
-            map.insert(key.to_string(), value.clone());
-        });
+        self.dirty |= Self::exec(
+            &mut self.state.group_policies,
+            t,
+            self.reload_changes.as_ref(),
+            CachePartition::GroupPolicies,
+            key,
+            |map| {
+                map.insert(key.to_string(), value.clone());
+            },
+        );
     }
 
     pub(crate) fn delete_policy_doc(&mut self, key: &str, t: OffsetDateTime) {
-        self.dirty |= Self::exec(&mut self.state.policy_docs, t, |map| {
-            map.remove(key);
-        });
+        self.dirty |= Self::exec(
+            &mut self.state.policy_docs,
+            t,
+            self.reload_changes.as_ref(),
+            CachePartition::PolicyDocs,
+            key,
+            |map| {
+                map.remove(key);
+            },
+        );
     }
 
     pub(crate) fn delete_user(&mut self, key: &str, t: OffsetDateTime) {
-        self.dirty |= Self::exec(&mut self.state.users, t, |map| {
-            map.remove(key);
-        });
+        self.dirty |= Self::exec(
+            &mut self.state.users,
+            t,
+            self.reload_changes.as_ref(),
+            CachePartition::Users,
+            key,
+            |map| {
+                map.remove(key);
+            },
+        );
     }
 
     pub(crate) fn delete_user_policy(&mut self, key: &str, t: OffsetDateTime) {
-        self.dirty |= Self::exec(&mut self.state.user_policies, t, |map| {
-            map.remove(key);
-        });
+        self.dirty |= Self::exec(
+            &mut self.state.user_policies,
+            t,
+            self.reload_changes.as_ref(),
+            CachePartition::UserPolicies,
+            key,
+            |map| {
+                map.remove(key);
+            },
+        );
     }
 
     pub(crate) fn delete_user_group_membership(&mut self, key: &str, t: OffsetDateTime) {
-        self.dirty |= Self::exec(&mut self.state.user_group_memberships, t, |map| {
-            map.remove(key);
-        });
+        self.dirty |= Self::exec(
+            &mut self.state.user_group_memberships,
+            t,
+            self.reload_changes.as_ref(),
+            CachePartition::UserGroupMemberships,
+            key,
+            |map| {
+                map.remove(key);
+            },
+        );
     }
 
     pub(crate) fn delete_sts_account(&mut self, key: &str, t: OffsetDateTime) {
-        self.dirty |= Self::exec(&mut self.state.sts_accounts, t, |map| {
-            map.remove(key);
-        });
+        self.dirty |= Self::exec(
+            &mut self.state.sts_accounts,
+            t,
+            self.reload_changes.as_ref(),
+            CachePartition::StsAccounts,
+            key,
+            |map| {
+                map.remove(key);
+            },
+        );
     }
 
     pub(crate) fn delete_sts_policy(&mut self, key: &str, t: OffsetDateTime) {
-        self.dirty |= Self::exec(&mut self.state.sts_policies, t, |map| {
-            map.remove(key);
-        });
+        self.dirty |= Self::exec(
+            &mut self.state.sts_policies,
+            t,
+            self.reload_changes.as_ref(),
+            CachePartition::StsPolicies,
+            key,
+            |map| {
+                map.remove(key);
+            },
+        );
     }
 
     pub(crate) fn delete_group(&mut self, key: &str, t: OffsetDateTime) {
-        self.dirty |= Self::exec(&mut self.state.groups, t, |map| {
-            map.remove(key);
-        });
+        self.dirty |= Self::exec(
+            &mut self.state.groups,
+            t,
+            self.reload_changes.as_ref(),
+            CachePartition::Groups,
+            key,
+            |map| {
+                map.remove(key);
+            },
+        );
     }
 
     pub(crate) fn delete_group_policy(&mut self, key: &str, t: OffsetDateTime) {
-        self.dirty |= Self::exec(&mut self.state.group_policies, t, |map| {
-            map.remove(key);
-        });
+        self.dirty |= Self::exec(
+            &mut self.state.group_policies,
+            t,
+            self.reload_changes.as_ref(),
+            CachePartition::GroupPolicies,
+            key,
+            |map| {
+                map.remove(key);
+            },
+        );
     }
 
     pub(crate) fn build_user_group_memberships(&mut self) {
@@ -494,6 +756,50 @@ mod tests {
     use crate::cache::Cache;
     use crate::store::MappedPolicy;
 
+    #[tokio::test]
+    async fn reload_journal_preserves_same_key_mutations_and_expires_on_cancellation() {
+        use super::{CacheEntity, CachePartition};
+        let cache = Cache::default();
+        let stale = UserIdentity {
+            version: 1,
+            ..Default::default()
+        };
+        let current = UserIdentity {
+            version: 2,
+            ..Default::default()
+        };
+        let reload = cache.begin_reload().await;
+        cache.add_or_update_user("updated", &current, OffsetDateTime::now_utc());
+        cache.add_or_update_user("removed", &current, OffsetDateTime::now_utc());
+        cache.delete_user("removed", OffsetDateTime::now_utc());
+        // Even an absent key's deletion is recorded: storage may have returned
+        // an entry created and deleted entirely during the reload's reads.
+        cache.delete_user("absent", OffsetDateTime::now_utc());
+        let mut loaded = CacheEntity::new(std::collections::HashMap::from([
+            ("updated".to_string(), stale.clone()),
+            ("removed".to_string(), stale.clone()),
+            ("absent".to_string(), stale.clone()),
+            ("disk-only".to_string(), stale),
+        ]));
+        cache.with_write_lock(|locked| {
+            reload.merge_partition(CachePartition::Users, &mut loaded, &locked.state().users);
+            locked.replace_users(loaded);
+        });
+        let snapshot = cache.snapshot();
+        assert_eq!(snapshot.users["updated"].version, 2);
+        assert!(!snapshot.users.contains_key("removed"));
+        assert!(!snapshot.users.contains_key("absent"));
+        assert!(snapshot.users.contains_key("disk-only"));
+        drop(reload);
+        assert!(cache.reload_changes.lock().expect("journal lock").upgrade().is_none());
+        let cancelled = cache.begin_reload().await;
+        cache.delete_user("disk-only", OffsetDateTime::now_utc());
+        drop(cancelled);
+        assert!(cache.reload_changes.lock().expect("journal lock").upgrade().is_none());
+        let next = cache.begin_reload().await;
+        assert!(next.changes.lock().expect("new journal").is_empty());
+    }
+
     #[test]
     fn sts_mutation_locks_are_keyed_and_prune_unused_entries() {
         let cache = Cache::default();

crates/iam/src/federation/mapper.rs

@@ -16,6 +16,7 @@ use super::{FederatedClaims, VerifiedFederatedIdentity};
 use crate::{EVENT_OIDC_DIAGNOSTICS, LOG_COMPONENT_IAM, LOG_SUBSYSTEM_OIDC};
 use rustfs_policy::policy::{ClaimLookup, get_claim_case_insensitive};
 use rustfs_utils::HashAlgorithm;
+use serde_json::Value;
 use std::{collections::HashMap, sync::Arc};
 use tracing::debug;
 
@@ -66,6 +67,15 @@ impl FederatedAuthorization {
         &self.provider_id
     }
 
+    /// Issuer from the verified upstream identity.
+    pub fn issuer(&self) -> Option<&str> {
+        self.claims
+            .raw
+            .get("iss")
+            .and_then(Value::as_str)
+            .filter(|issuer| !issuer.is_empty())
+    }
+
     pub fn subject(&self) -> &str {
         &self.claims.sub
     }

crates/iam/src/manager.rs

@@ -415,6 +415,16 @@ where
 
         self.api.save_iam_config(IAMFormat::new_version_1(), path).await
     }
+    /// Current built-in IAM group membership, independent of credential snapshots.
+    pub fn user_group_memberships(&self, name: &str) -> Vec<String> {
+        self.cache
+            .snapshot()
+            .user_group_memberships
+            .get(name)
+            .map(|groups| groups.iter().cloned().collect())
+            .unwrap_or_default()
+    }
+
     pub async fn get_user(&self, access_key: &str) -> Option<UserIdentity> {
         let cache = self.cache.snapshot();
         cache
@@ -1005,6 +1015,17 @@ where
     }
 
     pub async fn sts_policy_db_get(&self, name: &str, groups: &Option<Vec<String>>) -> Result<Vec<String>> {
+        self.sts_policy_db_get_with_identity_mapping(name, groups)
+            .await
+            .map(|(policies, _)| policies)
+    }
+
+    /// Resolve STS policies and whether that same lookup found a nonempty identity mapping.
+    pub async fn sts_policy_db_get_with_identity_mapping(
+        &self,
+        name: &str,
+        groups: &Option<Vec<String>>,
+    ) -> Result<(Vec<String>, bool)> {
         if name.is_empty() {
             return Err(Error::InvalidArgument);
         }
@@ -1031,7 +1052,10 @@ where
             }
         };
 
-        self.policy_db_get_with_groups(mapped_policy.to_slice(), groups).await
+        let direct_policies = mapped_policy.to_slice();
+        let has_identity_mapping = !direct_policies.is_empty();
+        let policies = self.policy_db_get_with_groups(direct_policies, groups).await?;
+        Ok((policies, has_identity_mapping))
     }
 
     async fn policy_db_get_with_groups(&self, mut policies: Vec<String>, groups: &Option<Vec<String>>) -> Result<Vec<String>> {
@@ -1158,7 +1182,7 @@ where
         if let Some(groups) = u.credentials.groups.as_ref() {
             for group in groups.iter() {
                 if groups_cache.get(group).filter(|v| v.status == STATUS_DISABLED).is_some() {
-                    return Ok((Vec::new(), OffsetDateTime::now_utc()));
+                    continue;
                 }
 
                 let mp = match group_policies.get(group) {
@@ -1189,7 +1213,7 @@ where
 
         for group in user_group_memberships.get(name).cloned().unwrap_or_default().iter() {
             if groups_cache.get(group).filter(|v| v.status == STATUS_DISABLED).is_some() {
-                return Ok((Vec::new(), OffsetDateTime::now_utc()));
+                continue;
             }
 
             let mp = match group_policies.get(group) {
@@ -1486,7 +1510,27 @@ where
         args: &AddOrUpdateUserReq,
         updated_at: OffsetDateTime,
     ) -> Result<OffsetDateTime> {
+        self.add_user_at_with_secret_change(access_key, args, updated_at)
+            .await
+            .map(|(updated_at, _, _guard)| updated_at)
+    }
+
+    /// Return whether this write replaced a different regular-user secret.
+    pub(crate) async fn add_user_at_with_secret_change(
+        &self,
+        access_key: &str,
+        args: &AddOrUpdateUserReq,
+        updated_at: OffsetDateTime,
+    ) -> Result<(OffsetDateTime, bool, tokio::sync::OwnedMutexGuard<crate::cache::MutationLockState>)> {
+        let parent_lock = self.cache.parent_mutation_lock(access_key);
+        let parent_guard = parent_lock.lock_owned().await;
+        let _service_guard = self.cache.service_account_mutation_lock().lock().await;
+        let sts_lock = self.cache.sts_account_mutation_lock(access_key);
+        let _sts_guard = sts_lock.lock().await;
         let cache = self.cache.snapshot();
+        if cache.sts_accounts.contains_key(access_key) {
+            return Err(Error::AccessKeyAlreadyExists);
+        }
         let users = Arc::clone(&cache.users);
         if let Some(x) = users.get(access_key) {
             debug!(
@@ -1496,10 +1540,13 @@ where
                 state = "existing",
                 "Updating existing IAM user"
             );
-            if x.credentials.is_temp() {
+            if x.credentials.is_temp() || x.credentials.is_service_account() {
                 return Err(Error::AccessKeyAlreadyExists);
             }
         }
+        let secret_changed = users
+            .get(access_key)
+            .is_some_and(|user| user.credentials.secret_key != args.secret_key);
         drop(cache);
         drop(users);
 
@@ -1523,7 +1570,7 @@ where
 
         self.update_user_with_claims(access_key, user_entry)?;
 
-        Ok(updated_at)
+        Ok((updated_at, secret_changed, parent_guard))
     }
 
     pub async fn delete_user(&self, access_key: &str, utype: UserType) -> Result<()> {
@@ -1540,12 +1587,17 @@ where
             return Err(Error::InvalidArgument);
         }
 
+        let parent_lock = (utype == UserType::Reg).then(|| self.cache.parent_mutation_lock(access_key));
+        let _parent_guard = match &parent_lock {
+            Some(lock) => Some(lock.lock().await),
+            None => None,
+        };
         let sts_mutation_lock = (utype == UserType::Sts).then(|| self.cache.sts_account_mutation_lock(access_key));
         let _sts_mutation_guard = match &sts_mutation_lock {
             Some(lock) => Some(lock.lock().await),
             None => None,
         };
-        let _service_account_guard = if utype == UserType::Svc {
+        let _service_account_guard = if matches!(utype, UserType::Svc | UserType::Reg) {
             Some(self.cache.service_account_mutation_lock().lock().await)
         } else {
             None
@@ -1640,17 +1692,46 @@ where
     }
 
     pub async fn update_user_secret_key(&self, access_key: &str, secret_key: &str) -> Result<(OffsetDateTime, AccountStatus)> {
+        self.update_user_secret_key_with_guard(access_key, secret_key, None)
+            .await
+            .map(|(updated_at, status, _guard)| (updated_at, status))
+    }
+
+    pub(crate) async fn update_user_secret_key_with_guard(
+        &self,
+        access_key: &str,
+        secret_key: &str,
+        expected_secret: Option<&str>,
+    ) -> Result<(
+        OffsetDateTime,
+        AccountStatus,
+        tokio::sync::OwnedMutexGuard<crate::cache::MutationLockState>,
+    )> {
         if access_key.is_empty() || secret_key.is_empty() {
             return Err(Error::InvalidArgument);
         }
 
+        let parent_lock = self.cache.parent_mutation_lock(access_key);
+        let parent_guard = parent_lock.lock_owned().await;
         let cache = self.cache.snapshot();
         let users = Arc::clone(&cache.users);
         let u = match users.get(access_key) {
             Some(u) => u,
+            None if expected_secret.is_some() => return Err(Error::IAMActionNotAllowed),
             None => return Err(Error::NoSuchUser(access_key.to_string())),
         };
 
+        if let Some(expected) = expected_secret {
+            use subtle::ConstantTimeEq as _;
+            if !u.credentials.is_valid()
+                || u.credentials.is_temp()
+                || u.credentials.is_service_account()
+                || !bool::from(u.credentials.secret_key.as_bytes().ct_eq(expected.as_bytes()))
+            {
+                return Err(Error::IAMActionNotAllowed);
+            }
+        }
+
         let mut cred = u.credentials.clone();
         cred.secret_key = secret_key.to_string();
 
@@ -1672,7 +1753,7 @@ where
             .await?;
 
         self.update_user_with_claims(access_key, u)?;
-        Ok((updated_at, status))
+        Ok((updated_at, status, parent_guard))
     }
 
     /// Add SSH public key for a user (for SFTP authentication)
@@ -1681,6 +1762,8 @@ where
             return Err(Error::InvalidArgument);
         }
 
+        let parent_lock = self.cache.parent_mutation_lock(access_key);
+        let _parent_guard = parent_lock.lock().await;
         let cache = self.cache.snapshot();
         let users = Arc::clone(&cache.users);
         let u = match users.get(access_key) {
@@ -1720,6 +1803,8 @@ where
             return Err(Error::InvalidArgument);
         }
 
+        let parent_lock = self.cache.parent_mutation_lock(access_key);
+        let _parent_guard = parent_lock.lock().await;
         let cache = self.cache.snapshot();
         let users = Arc::clone(&cache.users);
         let u = match users.get(access_key) {
@@ -2238,6 +2323,11 @@ where
         Ok(())
     }
     pub async fn user_notification_handler(&self, name: &str, user_type: UserType) -> Result<()> {
+        let parent_lock = (user_type == UserType::Reg).then(|| self.cache.parent_mutation_lock(name));
+        let _parent_guard = match &parent_lock {
+            Some(lock) => Some(lock.lock().await),
+            None => None,
+        };
         let sts_mutation_lock = (user_type == UserType::Sts).then(|| self.cache.sts_account_mutation_lock(name));
         let _sts_mutation_guard = match &sts_mutation_lock {
             Some(lock) => Some(lock.lock().await),
@@ -2985,6 +3075,58 @@ mod tests {
         }
     }
 
+    #[tokio::test]
+    async fn disabled_group_preserves_direct_and_other_group_policies() {
+        for credential_groups in [false, true] {
+            let iam = build_test_iam_cache(FailingInitialLoadStore);
+            let groups = vec!["disabled".to_string(), "enabled".to_string()];
+            let user = UserIdentity::new(Credentials {
+                access_key: "alice".to_string(),
+                secret_key: "alice-secret".to_string(),
+                status: STATUS_ENABLED.to_string(),
+                groups: credential_groups.then(|| groups.clone()),
+                ..Default::default()
+            });
+            let now = OffsetDateTime::now_utc();
+            iam.cache.add_or_update_user("alice", &user, now);
+            iam.cache
+                .add_or_update_user_policy("alice", &MappedPolicy::new("direct-deny"), now);
+            if !credential_groups {
+                iam.cache
+                    .add_or_update_user_group_membership("alice", &groups.iter().cloned().collect(), now);
+            }
+            for (group, status, policy) in [
+                ("disabled", STATUS_DISABLED, "disabled-grant"),
+                ("enabled", STATUS_ENABLED, "other-group-deny"),
+            ] {
+                iam.cache.add_or_update_group(
+                    group,
+                    &GroupInfo {
+                        status: status.to_string(),
+                        members: vec!["alice".to_string()],
+                        ..Default::default()
+                    },
+                    now,
+                );
+                iam.cache.add_or_update_group_policy(group, &MappedPolicy::new(policy), now);
+            }
+            let policies: HashSet<_> = iam
+                .policy_db_get("alice", &None)
+                .await
+                .expect("user policy lookup")
+                .into_iter()
+                .collect();
+            assert_eq!(policies, HashSet::from(["direct-deny".to_string(), "other-group-deny".to_string()]));
+            assert!(
+                iam.policy_db_get_internal("disabled", true, false)
+                    .await
+                    .expect("disabled group lookup")
+                    .0
+                    .is_empty()
+            );
+        }
+    }
+
     #[tokio::test]
     async fn sts_policy_lookup_ignores_colliding_regular_user_mapping() {
         let iam = build_test_iam_cache(FailingInitialLoadStore);

crates/iam/src/oidc/provider.rs

@@ -18,18 +18,19 @@ use super::{
 use crate::{EVENT_OIDC_DIAGNOSTICS, LOG_COMPONENT_IAM, LOG_SUBSYSTEM_OIDC};
 use openidconnect::core::{CoreIdToken, CoreIdTokenClaims, CoreIdTokenVerifier, CoreJsonWebKeySet, CoreJwsSigningAlgorithm};
 use openidconnect::{
-    AsyncHttpClient, ClaimsVerificationError, ClientId, ClientSecret, DiscoveryError, IssuerUrl, JsonWebKeySetUrl, NonceVerifier,
+    AsyncHttpClient, ClaimsVerificationError, ClientId, DiscoveryError, IssuerUrl, JsonWebKeySetUrl, NonceVerifier,
     ProviderMetadataWithLogout, SignatureVerificationError, TokenUrl,
 };
 use serde::Deserialize;
 use std::collections::HashMap;
-use std::sync::RwLock;
+use std::sync::{Arc, Mutex, RwLock};
 use std::time::{Duration as StdDuration, Instant};
 use tokio::time::sleep;
 use tracing::{debug, warn};
 use url::Url;
 
 pub(super) const OIDC_JWKS_REFRESH_INTERVAL: StdDuration = StdDuration::from_secs(24 * 60 * 60);
+const OIDC_JWKS_REFRESH_COOLDOWN: StdDuration = StdDuration::from_secs(60);
 pub(super) const OIDC_DISCOVERY_TRANSPORT_RETRIES: usize = 3;
 pub(super) const OIDC_DISCOVERY_TRANSPORT_RETRY_DELAY: StdDuration = StdDuration::from_millis(50);
 pub(super) const OIDC_JWKS_BLOCKED_BY_OUTBOUND_POLICY: &str = "JWKS request blocked by outbound policy";
@@ -97,9 +98,14 @@ pub(super) struct ProviderState {
     pub(super) discovered_at: Instant,
 }
 
+type RefreshAttempt = Option<(Instant, Result<ProviderState, String>)>;
+type ProviderRefreshLock = Arc<tokio::sync::Mutex<RefreshAttempt>>;
+
+#[derive(Clone)]
 pub(super) struct ProviderRuntime {
-    states: RwLock<HashMap<String, ProviderState>>,
-    http_client: ReqwestHttpClient,
+    states: Arc<RwLock<HashMap<String, ProviderState>>>,
+    http_client: Arc<ReqwestHttpClient>,
+    refreshes: Arc<Mutex<HashMap<String, ProviderRefreshLock>>>,
 }
 
 // Workload issuers do not implement the browser authorization flow. Keep their
@@ -200,16 +206,25 @@ impl DiscoveredProviderMetadata {
 
     pub(super) fn verifier(&self, config: &OidcProviderConfig) -> CoreIdTokenVerifier<'static> {
         let client_id = ClientId::new(config.client_id.clone());
-        let secret = config.client_secret.as_ref().map(|secret| ClientSecret::new(secret.clone()));
         let (issuer, jwks, algorithms) = match self {
             Self::Console(metadata) => (metadata.issuer(), metadata.jwks(), metadata.id_token_signing_alg_values_supported()),
             Self::Workload(metadata) => (&metadata.issuer, &metadata.jwks, &metadata.id_token_signing_alg_values_supported),
         };
-        let verifier = match secret {
-            Some(secret) => CoreIdTokenVerifier::new_confidential_client(client_id, secret, issuer.clone(), jwks.clone()),
-            None => CoreIdTokenVerifier::new_public_client(client_id, issuer.clone(), jwks.clone()),
-        };
-        verifier.set_allowed_algs(algorithms.clone())
+        // Caller-supplied tokens must prove possession of the issuer's signing
+        // key. A client secret is shared with clients and cannot provide that proof.
+        let algorithms = algorithms
+            .iter()
+            .filter(|algorithm| {
+                !matches!(
+                    algorithm,
+                    CoreJwsSigningAlgorithm::HmacSha256
+                        | CoreJwsSigningAlgorithm::HmacSha384
+                        | CoreJwsSigningAlgorithm::HmacSha512
+                )
+            })
+            .cloned()
+            .collect::<Vec<_>>();
+        CoreIdTokenVerifier::new_public_client(client_id, issuer.clone(), jwks.clone()).set_allowed_algs(algorithms)
     }
 }
 
@@ -273,8 +288,9 @@ impl ProviderState {
 impl ProviderRuntime {
     pub(super) fn new(http_client: ReqwestHttpClient, states: HashMap<String, ProviderState>) -> Self {
         Self {
-            states: RwLock::new(states),
-            http_client,
+            states: Arc::new(RwLock::new(states)),
+            http_client: Arc::new(http_client),
+            refreshes: Arc::new(Mutex::new(HashMap::new())),
         }
     }
 
@@ -326,9 +342,38 @@ impl ProviderRuntime {
         provider_id: &str,
         config: &OidcProviderConfig,
     ) -> Result<ProviderState, String> {
-        self.recover_provider_state(provider_id, config)
+        let refresh = self
+            .refreshes
+            .lock()
+            .map_err(|_| "provider refresh registry is unavailable".to_string())?
+            .entry(provider_id.to_string())
+            .or_default()
+            .clone();
+        let mut attempt = refresh.lock_owned().await;
+        if let Some((completed, result)) = attempt.as_ref()
+            && completed.elapsed() < OIDC_JWKS_REFRESH_COOLDOWN
+        {
+            return result.clone();
+        }
+
+        // The admitted fetch owns the per-provider lock. A disconnected request
+        // cannot abandon discovery while consuming the provider's cooldown.
+        // Waiting callers share its completed result, including genuine failures.
+        let runtime = tokio::runtime::Handle::try_current().map_err(|err| err.to_string())?;
+        let provider_runtime = self.clone();
+        let provider_id = provider_id.to_string();
+        let config = config.clone();
+        runtime
+            .spawn(async move {
+                let result = provider_runtime
+                    .recover_provider_state(&provider_id, &config)
+                    .await
+                    .map_err(|err| err.to_string());
+                *attempt = Some((Instant::now(), result.clone()));
+                result
+            })
             .await
-            .map_err(|err| err.to_string())
+            .map_err(|err| err.to_string())?
     }
 
     pub(super) async fn recover_provider_state(

crates/iam/src/oidc/provider_tests.rs

@@ -180,7 +180,7 @@ fn workload_metadata_rejects_duplicate_fields() {
     }
 }
 #[test]
-fn workload_verifier_preserves_algorithm_secret_and_audience_policy() {
+fn caller_supplied_tokens_cannot_be_signed_with_the_client_secret() {
     let secret = Nonce::new_random().secret().to_string();
     let mut config = build_mocked_oidc_provider_config("workload", "https://issuer.example.com");
     config.client_secret = Some(secret.clone());
@@ -195,7 +195,8 @@ fn workload_verifier_preserves_algorithm_secret_and_audience_policy() {
     let token: CoreIdToken = signed.parse().unwrap();
     for workload in [false, true] {
         for (algorithms, accepted) in [
-            (serde_json::json!(["HS256", "unsupported-future-algorithm"]), true),
+            (serde_json::json!(["RS256", "HS256"]), false),
+            (serde_json::json!(["HS256", "unsupported-future-algorithm"]), false),
             (serde_json::json!(["ES256"]), false),
             (serde_json::json!(["unsupported-future-algorithm"]), false),
         ] {
@@ -216,21 +217,6 @@ fn workload_verifier_preserves_algorithm_secret_and_audience_policy() {
                 accepted,
                 "workload={workload}, algorithms={algorithms}"
             );
-            if accepted {
-                assert!(
-                    token.claims(&metadata.verifier(&config), |_: Option<&Nonce>| Ok(())).is_err(),
-                    "additional audiences require explicit trust"
-                );
-                let mut wrong_secret = config.clone();
-                wrong_secret.client_secret = Some(Nonce::new_random().secret().to_string());
-                let verifier = metadata
-                    .verifier(&wrong_secret)
-                    .set_other_audience_verifier_fn(|aud| trusted_aud(&config.other_audiences, aud));
-                assert!(
-                    token.claims(&verifier, |_: Option<&Nonce>| Ok(())).is_err(),
-                    "incorrect client secret must fail"
-                );
-            }
         }
     }
 }
@@ -274,6 +260,19 @@ fn configured_verifier_rejects_malformed_hmac_signature_lengths() {
             let metadata = DiscoveredProviderMetadata::parse(&serde_json::to_vec(&document).expect("encode metadata"), true)
                 .expect("parse configured provider metadata");
             let verifier = metadata.verifier(&config);
+            assert!(
+                verify_id_token_claims(&valid, &verifier, |_: Option<&Nonce>| Ok(())).is_err(),
+                "caller-supplied HMAC tokens must be rejected"
+            );
+            // The authorization-code flow obtains tokens directly from the
+            // provider and still supports confidential-client verification.
+            let verifier = openidconnect::core::CoreIdTokenVerifier::new_confidential_client(
+                openidconnect::ClientId::new(config.client_id.clone()),
+                openidconnect::ClientSecret::new(secret.to_owned()),
+                metadata.issuer().clone(),
+                openidconnect::core::CoreJsonWebKeySet::new(vec![]),
+            )
+            .set_allowed_algs(vec![serde_json::from_value(serde_json::json!(name)).expect("parse signing algorithm")]);
             assert!(
                 verify_id_token_claims(&valid, &verifier, |_: Option<&Nonce>| Ok(())).is_ok(),
                 "valid {name} token must pass for workload={workload}"
@@ -780,3 +779,149 @@ async fn test_validate_oidc_provider_config_returns_detailed_errors() {
     assert!(err.contains(&format!("issuer '{base}/application/o/rustfs/'")));
     assert!(handle.join().is_ok());
 }
+
+#[tokio::test]
+async fn jwks_refresh_is_single_flight_and_survives_request_cancellation() {
+    let (initial_key, initial_jwk) = oidc_es256_key_and_jwk("initial");
+    let (rotated_key, rotated_jwk) = oidc_es256_key_and_jwk("rotated");
+    let (wrong_key, _) = oidc_es256_key_and_jwk("wrong");
+    let requests = Arc::new(AtomicUsize::new(0));
+    let started = Arc::new(tokio::sync::Notify::new());
+    let (release_tx, release_rx) = std::sync::mpsc::channel();
+    let server_requests = Arc::clone(&requests);
+    let server_started = Arc::clone(&started);
+    let Some((base, server)) = start_mock_oidc_discovery_server_with_jwks(
+        |base| (base.to_string(), format!("{base}/jwks"), "/jwks".to_string()),
+        6,
+        "ES256",
+        false,
+        move |fetch| {
+            let request = server_requests.fetch_add(1, Ordering::SeqCst);
+            if request == 2 {
+                server_started.notify_one();
+                release_rx
+                    .recv_timeout(StdDuration::from_secs(5))
+                    .expect("release refresh response");
+            }
+            serde_json::json!({"keys": [if fetch == 0 { &initial_jwk } else { &rotated_jwk }]}).to_string()
+        },
+    ) else {
+        return;
+    };
+    let mut config = build_mocked_oidc_provider_config("bounded-refresh", &base);
+    config.issuer = Some(base.clone());
+    let http = ReqwestHttpClient::with_policy(OutboundPolicy::from_allowed_origins(&base).expect("allow test origin"));
+    let state = discover_provider(&config, &http).await.expect("initial discovery");
+    let sys = Arc::new(OidcSys {
+        configs: HashMap::from([(config.id.clone(), test_sourced_config(config.clone()))]),
+        provider_runtime: ProviderRuntime::new(http, HashMap::from([(config.id.clone(), state)])),
+        state_store: OidcStateStore::new(),
+        pending_discovery: Default::default(),
+    });
+    let now = time::OffsetDateTime::now_utc().unix_timestamp();
+    let payload = serde_json::json!({"iss": base, "sub": "user", "aud": config.client_id, "iat": now, "exp": now + 300});
+    let sign = |kid: &str, claims: &serde_json::Value, key: &EncodingKey| {
+        let mut header = Header::new(Algorithm::ES256);
+        header.kid = Some(kid.to_string());
+        jsonwebtoken::encode(&header, claims, key).expect("sign test token")
+    };
+    for (claim, value) in [
+        ("aud", serde_json::json!("wrong-audience")),
+        ("exp", serde_json::json!(now - 300)),
+    ] {
+        let mut invalid = payload.clone();
+        invalid[claim] = value;
+        assert!(
+            sys.verify_web_identity_token(&sign("initial", &invalid, &initial_key))
+                .await
+                .is_err()
+        );
+    }
+    assert_eq!(requests.load(Ordering::SeqCst), 2, "claim failures cannot refresh signing keys");
+    let invalid = sign("rotated", &payload, &wrong_key);
+    let attack = {
+        let sys = Arc::clone(&sys);
+        let invalid = invalid.clone();
+        tokio::spawn(async move { sys.verify_web_identity_token(&invalid).await })
+    };
+    tokio::time::timeout(StdDuration::from_secs(5), started.notified())
+        .await
+        .expect("refresh started");
+    attack.abort();
+    assert!(attack.await.expect_err("request canceled").is_cancelled());
+    let mut retries = Vec::new();
+    for _ in 0..16 {
+        let sys = Arc::clone(&sys);
+        let invalid = invalid.clone();
+        retries.push(tokio::spawn(async move { sys.verify_web_identity_token(&invalid).await }));
+    }
+    release_tx.send(()).expect("release provider");
+    for retry in retries {
+        assert!(
+            tokio::time::timeout(StdDuration::from_secs(5), retry)
+                .await
+                .expect("retry timeout")
+                .expect("retry join")
+                .is_err()
+        );
+    }
+    let rotated = sign("rotated", &payload, &rotated_key);
+    sys.verify_web_identity_token(&rotated)
+        .await
+        .expect("completed refresh must accept rotated keys after the original caller left");
+    assert_eq!(requests.load(Ordering::SeqCst), 4, "one discovery/JWKS pair for all rejected tokens");
+    sys.provider_runtime
+        .refresh_provider_state("independent-provider", &config)
+        .await
+        .expect("provider cooldowns must be independent");
+    assert_eq!(requests.load(Ordering::SeqCst), 6);
+    server.join().expect("mock server");
+}
+
+#[tokio::test]
+async fn jwks_refresh_cools_down_failures_and_recovers_after_the_interval() {
+    let (_, initial_jwk) = oidc_es256_key_and_jwk("initial");
+    let (_, recovered_jwk) = oidc_es256_key_and_jwk("recovered");
+    let requests = Arc::new(AtomicUsize::new(0));
+    let server_requests = Arc::clone(&requests);
+    let Some((base, server)) = start_mock_oidc_discovery_server_with_jwks(
+        |base| (base.to_string(), format!("{base}/jwks"), "/jwks".to_string()),
+        6,
+        "ES256",
+        false,
+        move |fetch| {
+            server_requests.fetch_add(1, Ordering::SeqCst);
+            match fetch {
+                0 => serde_json::json!({"keys": [initial_jwk]}).to_string(),
+                1 => "invalid JWKS".to_string(),
+                _ => serde_json::json!({"keys": [recovered_jwk]}).to_string(),
+            }
+        },
+    ) else {
+        return;
+    };
+    let mut config = build_mocked_oidc_provider_config("recovering-refresh", &base);
+    config.issuer = Some(base.clone());
+    let http = ReqwestHttpClient::with_policy(OutboundPolicy::from_allowed_origins(&base).expect("allow test origin"));
+    let initial = discover_provider(&config, &http).await.expect("initial discovery");
+    let runtime = ProviderRuntime::new(http, HashMap::from([(config.id.clone(), initial.clone())]));
+    assert!(runtime.refresh_provider_state(&config.id, &config).await.is_err());
+    assert_eq!(requests.load(Ordering::SeqCst), 4);
+    for _ in 0..8 {
+        assert!(runtime.refresh_provider_state(&config.id, &config).await.is_err());
+    }
+    assert_eq!(requests.load(Ordering::SeqCst), 4, "completed failures also consume a bounded cooldown");
+    assert_eq!(
+        runtime.get_provider_state(&config.id).expect("cached keys").discovered_at,
+        initial.discovered_at
+    );
+    let refresh = runtime.refreshes.lock().expect("registry")[&config.id].clone();
+    refresh.lock().await.as_mut().expect("completed attempt").0 = Instant::now() - OIDC_JWKS_REFRESH_COOLDOWN;
+    runtime
+        .refresh_provider_state(&config.id, &config)
+        .await
+        .expect("retry after cooldown");
+    assert_eq!(requests.load(Ordering::SeqCst), 6);
+    assert!(runtime.get_provider_state(&config.id).expect("recovered keys").discovered_at > initial.discovered_at);
+    server.join().expect("mock server");
+}

crates/iam/src/oidc/runtime.rs

@@ -33,8 +33,8 @@ use crate::{
 };
 use openidconnect::core::{CoreAuthenticationFlow, CoreClient, CoreIdToken};
 use openidconnect::{
-    Audience, AuthType, AuthorizationCode, ClientId, ClientSecret, CsrfToken, LogoutRequest, Nonce, PkceCodeChallenge,
-    PkceCodeVerifier, PostLogoutRedirectUrl, RedirectUrl, RequestTokenError, Scope,
+    Audience, AuthType, AuthorizationCode, ClaimsVerificationError, ClientId, ClientSecret, CsrfToken, LogoutRequest, Nonce,
+    PkceCodeChallenge, PkceCodeVerifier, PostLogoutRedirectUrl, RedirectUrl, RequestTokenError, Scope,
 };
 use rustfs_policy::policy::{ClaimLookup, get_claim_case_insensitive};
 use rustfs_utils::logging::LogThrottle;
@@ -537,6 +537,9 @@ impl OidcSys {
             .set_other_audience_verifier_fn(|aud| trusted_aud(&config.other_audiences, aud));
         let verified = verify_id_token_claims(id_token, &verifier, &Nonce::new(session.nonce.clone()));
         if let Err(e) = verified {
+            if !matches!(e, ClaimsVerificationError::SignatureVerification(_)) {
+                return Err(format!("ID token verification failed: {e}"));
+            }
             let verification_error = e.to_string();
             warn!(
                 event = EVENT_OIDC_DIAGNOSTICS,
@@ -767,6 +770,9 @@ impl OidcSys {
             .verifier(&config)
             .set_other_audience_verifier_fn(|aud| trusted_aud(&config.other_audiences, aud));
         if let Err(e) = verify_id_token_claims(&id_token, &verifier, |_: Option<&Nonce>| Ok(())) {
+            if !matches!(e, ClaimsVerificationError::SignatureVerification(_)) {
+                return Err(format!("ID token verification failed: {e}"));
+            }
             state = self
                 .provider_runtime
                 .refresh_provider_state(&provider_id, &config)

crates/iam/src/storage_api.rs

@@ -14,6 +14,20 @@
 
 use std::sync::Arc;
 
+pub(crate) fn is_unreadable_bucket_tags(error: &crate::error::Error) -> bool {
+    let crate::error::Error::Io(error) = error else {
+        return false;
+    };
+    let Some(storage_error) = error.get_ref().and_then(|inner| inner.downcast_ref::<IamStorageError>()) else {
+        return false;
+    };
+    rustfs_ecstore::api::bucket::metadata::unreadable_config_refusal(storage_error)
+        .is_some_and(|failure| failure.config_file == rustfs_ecstore::api::bucket::metadata::BUCKET_TAGGING_CONFIG)
+}
+
+#[cfg(test)]
+pub(crate) use rustfs_ecstore::api::bucket::metadata::UnreadableBucketConfig;
+
 pub(crate) use rustfs_ecstore::api::config::RUSTFS_CONFIG_PREFIX as IAM_CONFIG_ROOT_PREFIX;
 use rustfs_ecstore::api::config::com::{
     delete_config as ecstore_delete_config, read_config_no_lock as ecstore_read_config_no_lock,

crates/iam/src/store/object.rs

@@ -1154,7 +1154,7 @@ impl Store for ObjectStore {
         // node-counted lock quorum (rustfs#4304; startup contract rustfs#4056).
         const LOAD_ALL_MODE: LoadMode = LoadMode::BootstrapNoLock;
 
-        let cache_snapshot = cache.snapshot();
+        let reload = cache.begin_reload().await;
         let listed_config_items = self.list_all_iamconfig_items().await?;
 
         let mut policy_docs_cache = CacheEntity::new(get_default_policies());
@@ -1201,26 +1201,20 @@ impl Store for ObjectStore {
         }
 
         // groups
-        let mut groups_cache = None;
+        let mut groups_cache = CacheEntity::default();
         if let Some(item_name_list) = listed_config_items.get(GROUPS_LIST_KEY) {
-            let mut items_cache = CacheEntity::default();
-
             for item in item_name_list.iter() {
                 let name = rustfs_utils::path::dir(item);
                 debug!(group = %name, "IAM group loaded");
-                if let Err(err) = self.load_group_with(&name, &mut items_cache, LOAD_ALL_MODE).await {
+                if let Err(err) = self.load_group_with(&name, &mut groups_cache, LOAD_ALL_MODE).await {
                     return Err(Error::other(format!("load group failed: {err}")));
                 };
             }
-
-            groups_cache = Some(items_cache);
         }
 
         // user policies
-        let mut user_policies_cache = None;
+        let mut user_policies_cache = CacheEntity::default();
         if let Some(item_name_list) = listed_config_items.get(POLICY_DB_USERS_LIST_KEY) {
-            let mut items_cache = CacheEntity::default();
-
             // Load in fixed-size chunks so each mapping is fetched exactly once
             // (backlog#806 — the split_off loop was O(n^2)).
             for chunk in item_name_list.chunks(32) {
@@ -1235,32 +1229,26 @@ impl Store for ObjectStore {
 
                     let name = chunk[idx].trim_end_matches(".json").to_owned();
                     debug!(user = %name, "IAM user policy loaded");
-                    items_cache.insert(name, p);
+                    user_policies_cache.insert(name, p);
                 }
             }
-
-            user_policies_cache = Some(items_cache);
         }
 
         // group policy
-        let mut group_policies_cache = None;
+        let mut group_policies_cache = CacheEntity::default();
         if let Some(item_name_list) = listed_config_items.get(POLICY_DB_GROUPS_LIST_KEY) {
-            let mut items_cache = CacheEntity::default();
-
             for item in item_name_list.iter() {
                 let name = item.trim_end_matches(".json");
 
                 debug!(group = %name, "IAM group policy loaded");
                 if let Err(err) = self
-                    .load_mapped_policy_with(name, UserType::Reg, true, &mut items_cache, LOAD_ALL_MODE)
+                    .load_mapped_policy_with(name, UserType::Reg, true, &mut group_policies_cache, LOAD_ALL_MODE)
                     .await
                     && !is_err_no_such_policy(&err)
                 {
                     return Err(Error::other(format!("load group policy failed: {err}")));
                 };
             }
-
-            group_policies_cache = Some(items_cache);
         }
 
         let mut sts_policies_cache = CacheEntity::default();
@@ -1330,25 +1318,48 @@ impl Store for ObjectStore {
             }
         }
 
+        #[cfg(test)]
+        if let Ok(barrier) = LOAD_ALL_COMMIT_BARRIER.try_with(std::sync::Arc::clone) {
+            barrier.0.notify_one();
+            barrier.1.notified().await;
+        }
+
         cache.with_write_lock(|cache| {
-            if cache.matches_snapshot(&cache_snapshot) {
-                cache.replace_policy_docs(policy_docs_cache);
-                if let Some(groups_cache) = groups_cache {
-                    cache.replace_groups(groups_cache);
-                }
-                if let Some(user_policies_cache) = user_policies_cache {
-                    cache.replace_user_policies(user_policies_cache);
-                }
-                if let Some(group_policies_cache) = group_policies_cache {
-                    cache.replace_group_policies(group_policies_cache);
-                }
-                cache.replace_users(user_items_cache);
-                cache.replace_sts_accounts(sts_items_cache);
-                cache.replace_sts_policies(sts_policies_cache);
-                cache.build_user_group_memberships();
-            } else {
-                warn!("IAM full reload cache commit skipped due to concurrent cache changes");
-            }
+            reload.merge_partition(
+                crate::cache::CachePartition::PolicyDocs,
+                &mut policy_docs_cache,
+                &cache.state().policy_docs,
+            );
+            reload.merge_partition(crate::cache::CachePartition::Groups, &mut groups_cache, &cache.state().groups);
+            reload.merge_partition(
+                crate::cache::CachePartition::UserPolicies,
+                &mut user_policies_cache,
+                &cache.state().user_policies,
+            );
+            reload.merge_partition(
+                crate::cache::CachePartition::GroupPolicies,
+                &mut group_policies_cache,
+                &cache.state().group_policies,
+            );
+            reload.merge_partition(crate::cache::CachePartition::Users, &mut user_items_cache, &cache.state().users);
+            reload.merge_partition(
+                crate::cache::CachePartition::StsAccounts,
+                &mut sts_items_cache,
+                &cache.state().sts_accounts,
+            );
+            reload.merge_partition(
+                crate::cache::CachePartition::StsPolicies,
+                &mut sts_policies_cache,
+                &cache.state().sts_policies,
+            );
+            cache.replace_policy_docs(policy_docs_cache);
+            cache.replace_groups(groups_cache);
+            cache.replace_user_policies(user_policies_cache);
+            cache.replace_group_policies(group_policies_cache);
+            cache.replace_users(user_items_cache);
+            cache.replace_sts_accounts(sts_items_cache);
+            cache.replace_sts_policies(sts_policies_cache);
+            cache.build_user_group_memberships();
         });
 
         Ok(())
@@ -1363,6 +1374,11 @@ fn map_delete_user_identity_error(name: &str, err: Error) -> Error {
     }
 }
 
+#[cfg(test)]
+tokio::task_local! {
+    static LOAD_ALL_COMMIT_BARRIER: std::sync::Arc<(tokio::sync::Notify, tokio::sync::Notify)>;
+}
+
 #[cfg(test)]
 mod tests {
     use super::{DecryptSource, LoadMode, ObjectStore, map_delete_user_identity_error};
@@ -1372,6 +1388,99 @@ mod tests {
     use serial_test::serial;
     use temp_env::with_vars;
 
+    #[test]
+    #[serial]
+    fn test_load_all_clears_partitions_with_empty_prefix() {
+        use crate::cache::Cache;
+        use crate::store::{GroupInfo, MappedPolicy, Store, UserType};
+
+        std::thread::Builder::new()
+            .stack_size(8 * 1024 * 1024)
+            .spawn(|| {
+                let runtime = tokio::runtime::Builder::new_multi_thread()
+                    .worker_threads(2)
+                    .enable_all()
+                    .build()
+                    .expect("build IAM reload test runtime");
+                runtime.block_on(async {
+                    let _credentials = test_cred();
+                    let temp = tempfile::tempdir().expect("create IAM reload store directory");
+                    let env = Box::pin(
+                        rustfs_test_utils::TestECStoreEnv::builder()
+                            .base_dir(temp.path())
+                            .init_bucket_metadata(false)
+                            .build(),
+                    )
+                    .await;
+                    let store = ObjectStore::new(env.ecstore);
+                    store
+                        .save_group_info("reload-group", GroupInfo::new(vec!["reload-user".to_string()]))
+                        .await
+                        .expect("seed group membership");
+                    store
+                        .save_mapped_policy("reload-user", UserType::Reg, false, MappedPolicy::new("readwrite"), None)
+                        .await
+                        .expect("seed user policy");
+                    store
+                        .save_mapped_policy("reload-group", UserType::Reg, true, MappedPolicy::new("readonly"), None)
+                        .await
+                        .expect("seed group policy");
+
+                    let cache = Cache::default();
+                    store.load_all(&cache).await.expect("load nonempty IAM partitions");
+                    {
+                        let snapshot = cache.snapshot();
+                        assert!(snapshot.groups.contains_key("reload-group"));
+                        assert_eq!(snapshot.user_policies["reload-user"].policies, "readwrite");
+                        assert_eq!(snapshot.group_policies["reload-group"].policies, "readonly");
+                        assert!(snapshot.user_group_memberships["reload-user"].contains("reload-group"));
+                    }
+
+                    // Mutate durable state without notifying this cache, as on
+                    // a peer that missed the last mapping/group deletion.
+                    store
+                        .delete_mapped_policy("reload-user", UserType::Reg, false)
+                        .await
+                        .expect("remove last user mapping");
+                    store
+                        .delete_mapped_policy("reload-group", UserType::Reg, true)
+                        .await
+                        .expect("remove last group mapping");
+                    store.delete_group_info("reload-group").await.expect("remove last group");
+                    assert!(cache.snapshot().groups.contains_key("reload-group"));
+                    // Simulate a missed peer revocation while STS publication
+                    // and a separate user update overlap this full disk reload.
+                    let stale = rustfs_policy::auth::UserIdentity {
+                        version: 1,
+                        ..Default::default()
+                    };
+                    cache.add_or_update_user("revoked-user", &stale, time::OffsetDateTime::now_utc());
+                    let barrier = std::sync::Arc::new((tokio::sync::Notify::new(), tokio::sync::Notify::new()));
+                    let loading = super::LOAD_ALL_COMMIT_BARRIER.scope(std::sync::Arc::clone(&barrier), store.load_all(&cache));
+                    let concurrent = async {
+                        barrier.0.notified().await;
+                        cache.add_or_update_sts_account("new-session", &stale, time::OffsetDateTime::now_utc());
+                        cache.add_or_update_user("new-service", &stale, time::OffsetDateTime::now_utc());
+                        barrier.1.notify_one();
+                    };
+                    let (loaded, ()) = tokio::join!(loading, concurrent);
+                    loaded.expect("reconcile revoked identities despite STS traffic");
+                    let snapshot = cache.snapshot();
+                    assert!(!snapshot.users.contains_key("revoked-user"));
+                    assert!(snapshot.users.contains_key("new-service"));
+                    assert!(snapshot.sts_accounts.contains_key("new-session"));
+                    assert!(snapshot.groups.is_empty());
+                    assert!(snapshot.user_policies.is_empty());
+                    assert!(snapshot.group_policies.is_empty());
+                    assert!(snapshot.user_group_memberships.is_empty());
+                    assert!(!snapshot.policy_docs.is_empty(), "built-in policies must remain available");
+                });
+            })
+            .expect("spawn IAM reload test thread")
+            .join()
+            .expect("IAM reload test should complete");
+    }
+
     /// rustfs#4304 / startup contract rustfs#4056: the bulk snapshot load
     /// (`load_all`) must never depend on the node-counted namespace lock
     /// quorum, so its reads have to carry `no_lock = true` down to the

crates/keystone/src/auth.rs

@@ -85,6 +85,17 @@ impl KeystoneAuthProvider {
         Ok(self.keystone_token_to_credentials(&keystone_token))
     }
 
+    /// Revalidate a long-lived subscription directly with Keystone, bypassing
+    /// the request cache so token revocation terminates an existing stream.
+    /// Returns the same authentication and expiry errors as a new token login.
+    pub async fn revalidate_subscription_token(&self, token: &str) -> Result<Credentials> {
+        let validated = self.client.validate_token(token).await?;
+        if validated.is_expired() {
+            return Err(KeystoneError::TokenExpired);
+        }
+        Ok(self.keystone_token_to_credentials(&validated))
+    }
+
     /// Authenticate using EC2 credentials (S3 API with AWS SigV4)
     pub async fn authenticate_with_ec2(&self, access_key: &str, signature: &str, string_to_sign: &str) -> Result<Credentials> {
         // Check cache

crates/keystone/src/client.rs

@@ -121,7 +121,7 @@ impl KeystoneClient {
             .await
             .map_err(|e| {
                 error!("Failed to send token validation request: {}", e);
-                KeystoneError::HttpError(e.to_string())
+                KeystoneError::HttpError(e.without_url().to_string())
             })?;
 
         let status = response.status();
@@ -138,7 +138,10 @@ impl KeystoneClient {
             )));
         }
 
-        let body: serde_json::Value = response.json().await.map_err(|e| KeystoneError::ParseError(e.to_string()))?;
+        let body: serde_json::Value = response
+            .json()
+            .await
+            .map_err(|e| KeystoneError::ParseError(e.without_url().to_string()))?;
 
         self.parse_token_v3(&body)
     }
@@ -260,13 +263,16 @@ impl KeystoneClient {
             .header("X-Auth-Token", admin_token)
             .send()
             .await
-            .map_err(|e| KeystoneError::HttpError(e.to_string()))?;
+            .map_err(|e| KeystoneError::HttpError(e.without_url().to_string()))?;
 
         if !response.status().is_success() {
             return Ok(vec![]);
         }
 
-        let body: serde_json::Value = response.json().await.map_err(|e| KeystoneError::ParseError(e.to_string()))?;
+        let body: serde_json::Value = response
+            .json()
+            .await
+            .map_err(|e| KeystoneError::ParseError(e.without_url().to_string()))?;
 
         let credentials = body
             .get("credentials")
@@ -365,7 +371,7 @@ impl KeystoneClient {
             .json(&auth_payload)
             .send()
             .await
-            .map_err(|e| KeystoneError::HttpError(e.to_string()))?;
+            .map_err(|e| KeystoneError::HttpError(e.without_url().to_string()))?;
 
         if !response.status().is_success() {
             return Err(KeystoneError::AuthenticationFailed("Admin authentication failed".to_string()));
@@ -379,7 +385,10 @@ impl KeystoneClient {
             .to_string();
 
         // Parse expiration from response body
-        let body: serde_json::Value = response.json().await.map_err(|e| KeystoneError::ParseError(e.to_string()))?;
+        let body: serde_json::Value = response
+            .json()
+            .await
+            .map_err(|e| KeystoneError::ParseError(e.without_url().to_string()))?;
 
         let expires_at = body
             .get("token")
@@ -410,6 +419,133 @@ impl KeystoneClient {
 mod tests {
     use super::*;
 
+    #[tokio::test]
+    async fn subscription_token_refresh_bypasses_a_still_valid_login_cache() {
+        use std::io::{Read, Write};
+        let listener = std::net::TcpListener::bind("127.0.0.1:0").expect("fake Keystone");
+        let endpoint = format!("http://{}", listener.local_addr().unwrap());
+        let expires = (OffsetDateTime::now_utc() + time::Duration::hours(1))
+            .format(&time::format_description::well_known::Rfc3339)
+            .unwrap();
+        let body = json!({"token":{"user":{"id":"listener","name":"listener"},
+            "roles":[{"name":"Member"}],"expires_at":expires}})
+        .to_string();
+        let server = std::thread::spawn(move || {
+            for response in [
+                format!(
+                    "HTTP/1.1 200 OK\r\nContent-Type: application/json\r\nContent-Length: {}\r\nConnection: close\r\n\r\n{body}",
+                    body.len()
+                ),
+                "HTTP/1.1 401 Unauthorized\r\nContent-Length: 0\r\nConnection: close\r\n\r\n".to_string(),
+            ] {
+                let (mut socket, _) = listener.accept().expect("token validation request");
+                socket.set_read_timeout(Some(std::time::Duration::from_secs(5))).unwrap();
+                let mut request = [0; 4096];
+                let length = socket.read(&mut request).expect("request headers");
+                assert!(String::from_utf8_lossy(&request[..length]).contains("subscription-token"));
+                socket.write_all(response.as_bytes()).expect("validation response");
+            }
+        });
+        let mut client = KeystoneClient::new(
+            endpoint,
+            KeystoneVersion::V3,
+            None,
+            None,
+            None,
+            "Default".into(),
+            true,
+            std::time::Duration::from_secs(5),
+        );
+        client.client = reqwest::Client::builder()
+            .no_proxy()
+            .timeout(std::time::Duration::from_secs(5))
+            .build()
+            .unwrap();
+        let provider = crate::KeystoneAuthProvider::new(client, 10, std::time::Duration::from_secs(60), true);
+        assert!(provider.authenticate_with_token("subscription-token").await.is_ok());
+        assert!(
+            provider.authenticate_with_token("subscription-token").await.is_ok(),
+            "ordinary login still uses the valid cached token"
+        );
+        assert!(matches!(
+            provider.revalidate_subscription_token("subscription-token").await,
+            Err(KeystoneError::InvalidToken)
+        ));
+        server.join().expect("fake Keystone finished");
+    }
+
+    #[tokio::test]
+    async fn middleware_keystone_failures_do_not_disclose_upstream_locations() {
+        use http_body_util::{BodyExt, Full};
+        use std::io::{Read, Write};
+        use tower::{Layer, ServiceExt};
+
+        for (response, details) in [
+            (None, "Keystone token validation failed"),
+            (
+                Some("HTTP/1.1 200 OK\r\nContent-Length: 8\r\nConnection: close\r\n\r\nnot-json"),
+                "Keystone token validation failed",
+            ),
+            (
+                Some("HTTP/1.1 401 Unauthorized\r\nContent-Length: 0\r\nConnection: close\r\n\r\n"),
+                "Invalid token",
+            ),
+        ] {
+            for header in ["X-Auth-Token", "X-Storage-Token"] {
+                let listener = std::net::TcpListener::bind("127.0.0.1:0").expect("bind fake Keystone");
+                let endpoint = format!("http://{}/private-deployment", listener.local_addr().unwrap());
+                let server = response.map(|response| {
+                    std::thread::spawn(move || {
+                        let (mut stream, _) = listener.accept().expect("Keystone request");
+                        stream.set_read_timeout(Some(std::time::Duration::from_secs(5))).unwrap();
+                        let mut request = [0; 4096];
+                        assert!(stream.read(&mut request).expect("read token validation request") > 0);
+                        stream.write_all(response.as_bytes()).expect("send fake response");
+                    })
+                });
+                let mut client = KeystoneClient::new(
+                    endpoint.clone(),
+                    KeystoneVersion::V3,
+                    None,
+                    None,
+                    None,
+                    "Default".to_owned(),
+                    true,
+                    std::time::Duration::from_secs(5),
+                );
+                client.client = reqwest::Client::builder()
+                    .no_proxy()
+                    .timeout(std::time::Duration::from_secs(5))
+                    .build()
+                    .unwrap();
+                let provider = crate::KeystoneAuthProvider::new(client, 1, std::time::Duration::from_secs(1), false);
+                let inner = tower::service_fn(|_| async {
+                    Ok::<_, std::convert::Infallible>(http::Response::new(Full::new(bytes::Bytes::new())))
+                });
+                let response = crate::middleware::KeystoneAuthLayer::new(Some(Arc::new(provider)))
+                    .layer(inner)
+                    .oneshot(
+                        http::Request::builder()
+                            .header(header, "private-token")
+                            .body(Full::new(bytes::Bytes::new()))
+                            .unwrap(),
+                    )
+                    .await
+                    .unwrap();
+                assert_eq!(response.status(), http::StatusCode::UNAUTHORIZED);
+                let body = response.into_body().collect().await.unwrap().to_bytes();
+                let body = std::str::from_utf8(&body).unwrap();
+                assert!(body.contains(&format!("<Details>{details}</Details>")));
+                for secret in [&endpoint, "private-deployment", "private-token", "/v3/auth/tokens"] {
+                    assert!(!body.contains(secret), "upstream diagnostics must not appear in the response");
+                }
+                if let Some(server) = server {
+                    server.join().unwrap();
+                }
+            }
+        }
+    }
+
     #[test]
     fn test_client_creation() {
         let client = KeystoneClient::new(

crates/keystone/src/middleware.rs

@@ -173,7 +173,11 @@ where
     <Message>Invalid Keystone token</Message>
     <Details>{}</Details>
 </Error>"#,
-                            xml_escape(&e.to_string())
+                            xml_escape(match e {
+                                crate::KeystoneError::InvalidToken => "Invalid token",
+                                crate::KeystoneError::TokenExpired => "Token expired",
+                                _ => "Keystone token validation failed",
+                            })
                         );
 
                         let body: BoxBody = Full::new(Bytes::from(error_xml))

crates/kms/src/manager.rs

@@ -214,6 +214,9 @@ impl KmsManager {
     }
 
     async fn create_key_inner(&self, request: CreateKeyRequest) -> Result<CreateKeyResponse> {
+        if request.key_name.as_deref() == Some(crate::probe::PROBE_KEY_ID) {
+            return Err(KmsError::invalid_operation("the KMS probe key is reserved for internal health checks"));
+        }
         // A blank name is neither "generate one" (that is `None`) nor a usable
         // id: the Local backend would write a key file with an empty stem and
         // the Vault backends would address their mount root, each failing with
@@ -438,6 +441,9 @@ impl KmsManager {
     /// any backend sees the request. The backends repeat the window bound as a
     /// defensive assertion for callers that hold a backend handle directly.
     async fn delete_key_inner(&self, request: DeleteKeyRequest) -> Result<DeleteKeyResponse> {
+        if request.key_id == crate::probe::PROBE_KEY_ID {
+            return Err(KmsError::invalid_operation("the KMS probe key cannot be deleted through key management"));
+        }
         self.check_deletion_request(&request)?;
         if request.force_immediate.unwrap_or(false) {
             self.refuse_referenced_immediate_deletion(&request.key_id).await?;
@@ -591,7 +597,11 @@ impl KmsManager {
     /// Disable a key on behalf of `context`'s principal
     pub async fn disable_key_with_context(&self, key_id: &str, context: &OperationContext) -> Result<()> {
         let started = Instant::now();
-        let result = self.backend.disable_key(key_id).await;
+        let result = if key_id == crate::probe::PROBE_KEY_ID {
+            Err(KmsError::invalid_operation("the KMS probe key cannot be disabled through key management"))
+        } else {
+            self.backend.disable_key(key_id).await
+        };
         if result.is_ok() {
             self.invalidate_cached_metadata(key_id).await;
         }

crates/kms/src/probe.rs

@@ -700,6 +700,57 @@ mod tests {
         );
     }
 
+    #[tokio::test]
+    async fn key_management_cannot_preempt_or_disable_the_readiness_probe() {
+        let temp_dir = tempfile::tempdir().expect("key directory");
+        let backend = local_backend(&temp_dir).await;
+        let mut config = KmsConfig::local(temp_dir.path().to_path_buf()).with_insecure_development_defaults();
+        config.allow_immediate_deletion = true;
+        let manager = crate::manager::KmsManager::new(backend.clone(), config);
+        let request = CreateKeyRequest {
+            key_name: Some(PROBE_KEY_ID.to_owned()),
+            ..Default::default()
+        };
+        assert!(matches!(manager.create_key(request).await, Err(KmsError::InvalidOperation { .. })));
+        let worker = worker(backend.clone());
+        assert_eq!(worker.round().await, ProbeResult::Success, "the probe still provisions its reserved key");
+        assert!(matches!(manager.disable_key(PROBE_KEY_ID).await, Err(KmsError::InvalidOperation { .. })));
+        for immediate in [false, true] {
+            assert!(matches!(
+                manager
+                    .delete_key(DeleteKeyRequest {
+                        key_id: PROBE_KEY_ID.to_owned(),
+                        pending_window_in_days: Some(7),
+                        force_immediate: Some(immediate),
+                        confirm_key_id: Some(PROBE_KEY_ID.to_owned())
+                    })
+                    .await,
+                Err(KmsError::InvalidOperation { .. })
+            ));
+            assert_eq!(worker.round().await, ProbeResult::Success);
+        }
+        let customer = manager
+            .create_key(CreateKeyRequest {
+                key_name: Some("customer-probe-control".to_owned()),
+                ..Default::default()
+            })
+            .await
+            .expect("customer key");
+        manager
+            .disable_key(&customer.key_id)
+            .await
+            .expect("customer key can be disabled");
+        manager
+            .delete_key(DeleteKeyRequest {
+                key_id: customer.key_id,
+                pending_window_in_days: Some(7),
+                ..Default::default()
+            })
+            .await
+            .expect("customer deletion remains available");
+        assert_eq!(worker.round().await, ProbeResult::Success);
+    }
+
     #[test]
     fn read_only_backend_is_reported_unsupported_without_failures() {
         let (snapshot, result) = record_metrics(|| {

crates/lock/src/distributed_lock.rs

@@ -694,6 +694,132 @@ impl DistributedLock {
         self.acquire_guard(&req).await
     }
 
+    /// Acquire exclusive locks for many independent resources in one batched
+    /// round: one [`LockClient::acquire_locks_batch`] RPC per client instead of
+    /// one quorum round-trip per key. Returns one guard or error per resource.
+    ///
+    /// Bulk operations (S3 `DeleteObjects`) are the only multi-key write-lock
+    /// holders, and their keys are independent. Per-key outcomes are
+    /// deadlock-free without any serial fallback: the wire call is a single
+    /// round per client and every lock server orders concurrent batches by the
+    /// same global (shard, key) order, so no acquirer can hold one key while
+    /// waiting for another. A key that fails to reach quorum releases its
+    /// partial grants (awaited) before its error is returned.
+    ///
+    /// Guards keep the same TTL heartbeat and lock-lost semantics as
+    /// [`Self::acquire_guard`]. Contention is waited out server-side up to
+    /// `timeout` (each request carries it), matching the single-key path.
+    pub async fn acquire_guard_batch(
+        &self,
+        resources: &[ObjectKey],
+        owner: &str,
+        timeout: Duration,
+        ttl: Duration,
+    ) -> Vec<Result<DistributedLockGuard>> {
+        if resources.is_empty() {
+            return Vec::new();
+        }
+        if self.clients.is_empty() {
+            return resources
+                .iter()
+                .map(|_| Err(LockError::internal("No lock clients available")))
+                .collect();
+        }
+
+        let required_quorum = self.required_quorum(LockType::Exclusive);
+        let attempt_started = Instant::now();
+        let requests: Vec<LockRequest> = resources
+            .iter()
+            .map(|resource| {
+                LockRequest::new(resource.clone(), LockType::Exclusive, owner)
+                    .with_acquire_timeout(timeout)
+                    .with_ttl(ttl)
+            })
+            .collect();
+
+        // One batched round per client; a client-level failure is a per-key
+        // failure for every key that still needed that client's vote.
+        let rounds = join_all(self.clients.iter().map(|client| {
+            let requests = requests.clone();
+            async move { client.acquire_locks_batch(&requests).await }
+        }))
+        .await;
+
+        let mut results: Vec<Result<DistributedLockGuard>> = Vec::with_capacity(resources.len());
+        let mut orphaned_entries: Vec<(LockId, Arc<dyn LockClient>)> = Vec::new();
+        for (key_idx, resource) in resources.iter().enumerate() {
+            let mut entries: Vec<HeldLockEntry> = Vec::new();
+            let mut failure: Option<String> = None;
+            for (client_idx, client) in self.clients.iter().enumerate() {
+                match &rounds[client_idx] {
+                    Ok(responses) => match responses.get(key_idx) {
+                        Some(response) if response.success => {
+                            let lock_id = response
+                                .lock_info
+                                .as_ref()
+                                .map(|info| info.id.clone())
+                                .unwrap_or_else(|| LockId::new_unique(resource));
+                            entries.push(HeldLockEntry {
+                                lock_id,
+                                client: client.clone(),
+                                valid_until: attempt_started.checked_add(ttl).unwrap_or(attempt_started),
+                            });
+                        }
+                        Some(response) => {
+                            failure.get_or_insert_with(|| {
+                                response
+                                    .error
+                                    .clone()
+                                    .unwrap_or_else(|| "distributed lock acquisition failed".to_string())
+                            });
+                        }
+                        None => {
+                            failure.get_or_insert_with(|| "client returned incomplete batch lock response".to_string());
+                        }
+                    },
+                    Err(err) => {
+                        failure.get_or_insert_with(|| err.to_string());
+                    }
+                }
+            }
+
+            if entries.len() >= required_quorum {
+                let heartbeat = LockHeartbeatConfig {
+                    ttl,
+                    interval: derive_refresh_interval(entries.len(), ttl, None),
+                    refresh_quorum: required_quorum,
+                    owner: owner.to_string(),
+                    resource: resource.clone(),
+                };
+                results.push(Ok(DistributedLockGuard::new(
+                    LockId::new_unique(resource),
+                    entries,
+                    LockType::Exclusive,
+                    heartbeat,
+                )));
+            } else {
+                let achieved = entries.len();
+                orphaned_entries.extend(entries.iter().map(HeldLockEntry::release_entry));
+                if let Some(error) = failure {
+                    tracing::debug!(resource = %resource, owner, required_quorum, achieved, "batch lock acquire failed: {error}");
+                }
+                results.push(Err(LockError::QuorumNotReached {
+                    required: required_quorum,
+                    achieved,
+                }));
+            }
+        }
+
+        // One bounded synchronous pass so the caller cannot race these
+        // unlocks on an immediate retry; anything a slow peer still holds
+        // keeps retrying in the background instead of parking the request
+        // on the deferred unlock schedule.
+        if !orphaned_entries.is_empty() {
+            Self::release_entries_bounded(orphaned_entries, "distributed_lock_batch_quorum_rollback").await;
+        }
+        results
+    }
+
     fn spawn_lock_requests_until(
         &self,
         pending: &mut JoinSet<LockAcquireTaskResult>,
@@ -755,6 +881,19 @@ impl DistributedLock {
         Self::release_entries_deferred(pending, context).await;
     }
 
+    /// Release entries with one synchronous pass, then continue the retry
+    /// schedule in the background.
+    ///
+    /// The batch-acquire rollback path runs on the request: it must not park
+    /// the caller on the deferred unlock schedule when a peer is unreachable,
+    /// while still confirming the healthy quorum is free before returning.
+    async fn release_entries_bounded(entries: Vec<(LockId, Arc<dyn LockClient>)>, context: &'static str) {
+        let pending = Self::release_pending_once(entries, 1, context).await;
+        if !pending.is_empty() {
+            Self::spawn_release_cleanup(pending, context);
+        }
+    }
+
     async fn release_pending_once(
         pending: Vec<(LockId, Arc<dyn LockClient>)>,
         attempt: usize,
@@ -2856,4 +2995,120 @@ mod tests {
 
         drop(guard);
     }
+
+    /// Per-object scripted client: grants only the named objects and records
+    /// every released lock's resource.
+    #[derive(Debug)]
+    struct ScriptedBatchClient {
+        grants: Vec<String>,
+        releases: Arc<Mutex<Vec<String>>>,
+    }
+
+    impl ScriptedBatchClient {
+        fn build(grants: &[&str]) -> (Arc<dyn LockClient>, Arc<Mutex<Vec<String>>>) {
+            let releases = Arc::new(Mutex::new(Vec::new()));
+            let client = Arc::new(Self {
+                grants: grants.iter().map(|object| (*object).to_string()).collect(),
+                releases: Arc::clone(&releases),
+            }) as Arc<dyn LockClient>;
+            (client, releases)
+        }
+    }
+
+    #[async_trait::async_trait]
+    impl LockClient for ScriptedBatchClient {
+        async fn acquire_lock(&self, request: &LockRequest) -> crate::Result<LockResponse> {
+            if self.grants.iter().any(|object| object == request.resource.object.as_ref()) {
+                let info = LockInfo {
+                    id: request.lock_id.clone(),
+                    resource: request.resource.clone(),
+                    lock_type: request.lock_type,
+                    status: crate::types::LockStatus::Acquired,
+                    owner: request.owner.clone(),
+                    acquired_at: std::time::SystemTime::now(),
+                    expires_at: std::time::SystemTime::now() + request.ttl,
+                    last_refreshed: std::time::SystemTime::now(),
+                    metadata: request.metadata.clone(),
+                    priority: request.priority,
+                    wait_start_time: None,
+                };
+                Ok(LockResponse::success(info, Duration::ZERO))
+            } else {
+                Ok(LockResponse::failure("denied by test script", Duration::ZERO))
+            }
+        }
+
+        async fn release(&self, lock_id: &LockId) -> crate::Result<bool> {
+            self.releases
+                .lock()
+                .expect("release log")
+                .push(lock_id.resource.object.to_string());
+            Ok(true)
+        }
+
+        async fn refresh(&self, _lock_id: &LockId) -> crate::Result<bool> {
+            Ok(true)
+        }
+
+        async fn force_release(&self, lock_id: &LockId) -> crate::Result<bool> {
+            self.release(lock_id).await
+        }
+
+        async fn check_status(&self, _lock_id: &LockId) -> crate::Result<Option<LockInfo>> {
+            Ok(None)
+        }
+
+        async fn get_stats(&self) -> crate::Result<LockStats> {
+            Ok(LockStats::default())
+        }
+
+        async fn close(&self) -> crate::Result<()> {
+            Ok(())
+        }
+
+        async fn is_online(&self) -> bool {
+            true
+        }
+
+        async fn is_local(&self) -> bool {
+            false
+        }
+    }
+
+    /// A batch key granted by fewer than quorum clients fails alone, and its
+    /// partial grants are released synchronously before the call returns
+    /// (PR9155 review coverage for the distributed rollback path).
+    #[tokio::test]
+    async fn acquire_guard_batch_releases_partial_grants_below_quorum() {
+        let (c0, c0_releases) = ScriptedBatchClient::build(&["kept", "lost"]);
+        let (c1, _c1_releases) = ScriptedBatchClient::build(&["kept"]);
+        let (c2, _c2_releases) = ScriptedBatchClient::build(&[]);
+        let lock = DistributedLock::new("batch-test".to_string(), vec![c0, c1, c2], 2);
+
+        let resources = vec![ObjectKey::new("bucket", "kept"), ObjectKey::new("bucket", "lost")];
+        let results = lock
+            .acquire_guard_batch(&resources, "owner", Duration::from_millis(200), Duration::from_secs(30))
+            .await;
+
+        assert!(results[0].is_ok(), "kept reached quorum: {:?}", results[0].as_ref().err());
+        let error = results[1].as_ref().expect_err("lost must fail below quorum");
+        assert!(
+            matches!(
+                error,
+                LockError::QuorumNotReached {
+                    required: 2,
+                    achieved: 1
+                }
+            ),
+            "lost must report its partial vote: {error:?}"
+        );
+        // Exactly the orphaned grant is released before the call returns,
+        // while "kept" stays held by its live guard.
+        assert_eq!(
+            c0_releases.lock().expect("release log").clone(),
+            vec!["lost".to_string()],
+            "only the below-quorum key's partial grant is released synchronously"
+        );
+        drop(results);
+    }
 }

crates/lock/src/fast_lock/shard.rs

@@ -153,7 +153,15 @@ impl LockShard {
         {
             return false;
         }
-        self.try_fast_path(request).is_some()
+        // Count the acquisition like `acquire_lock` does: batch callers use
+        // this path for every uncontended key, and the metric promises the
+        // total of successful acquisitions.
+        if self.try_fast_path(request).is_some() {
+            self.metrics.record_fast_path_success();
+            true
+        } else {
+            false
+        }
     }
 
     /// Try fast path lock acquisition (lock-free when possible)

crates/lock/src/local_lock.rs

@@ -15,7 +15,7 @@
 use crate::{
     GlobalLockManager, ObjectKey,
     error::Result,
-    fast_lock::{FastLockGuard, LockManager, LockMode, ObjectLockRequest},
+    fast_lock::{FastLockGuard, LockManager, LockMode, LockResult, ObjectLockRequest},
     types::{LockPriority, LockRequest, LockType},
 };
 use std::sync::Arc;
@@ -112,4 +112,61 @@ impl LocalLock {
             .with_ttl(ttl);
         self.acquire_guard(&req).await
     }
+
+    /// Acquire exclusive locks for many independent resources in one batched
+    /// pass; see [`crate::distributed_lock::DistributedLock::acquire_guard_batch`].
+    /// The manager's batch acquire sorts every request into one global
+    /// (shard, key) order, so concurrent batches cannot deadlock. Returns one
+    /// guard or error per resource, in `resources` order.
+    pub async fn acquire_guard_batch(
+        &self,
+        resources: &[ObjectKey],
+        owner: &str,
+        timeout: Duration,
+        ttl: Duration,
+    ) -> Vec<Result<FastLockGuard>> {
+        if resources.is_empty() {
+            return Vec::new();
+        }
+        let batch = crate::fast_lock::BatchLockRequest {
+            requests: resources
+                .iter()
+                .map(|resource| {
+                    ObjectLockRequest::new_write(resource.clone(), owner)
+                        .with_acquire_timeout(timeout)
+                        .with_lock_timeout(ttl)
+                })
+                .collect(),
+            owner: owner.into(),
+            all_or_nothing: false,
+        };
+        let outcome = self.manager.acquire_locks_batch(batch).await;
+        let mut guards = outcome
+            .successful_locks
+            .into_iter()
+            .zip(outcome.guards)
+            .collect::<std::collections::HashMap<ObjectKey, FastLockGuard>>();
+        let mut failures = outcome
+            .failed_locks
+            .into_iter()
+            .collect::<std::collections::HashMap<ObjectKey, LockResult>>();
+        resources
+            .iter()
+            .map(|resource| {
+                if let Some(guard) = guards.remove(resource) {
+                    return Ok(guard);
+                }
+                match failures.remove(resource) {
+                    Some(LockResult::Timeout) => Err(crate::error::LockError::timeout(resource.to_string(), timeout)),
+                    Some(LockResult::Conflict { current_owner, .. }) => Err(crate::error::LockError::AlreadyLocked {
+                        resource: resource.to_string(),
+                        owner: current_owner.to_string(),
+                    }),
+                    Some(LockResult::Acquired) | None => {
+                        Err(crate::error::LockError::internal("batch lock acquire returned no outcome for key"))
+                    }
+                }
+            })
+            .collect()
+    }
 }

crates/lock/src/namespace/mod.rs

@@ -88,6 +88,17 @@ impl NamespaceLockWrapper {
     pub async fn get_read_lock(&self, timeout: Duration) -> std::result::Result<NamespaceLockGuard, crate::error::LockError> {
         self.lock.get_read_lock(self.resource.clone(), &self.owner, timeout).await
     }
+
+    /// Acquire write locks for `resources` in one batched round using this
+    /// wrapper's owner; see [`NamespaceLock::acquire_write_locks_batch`].
+    /// One guard or error per resource, in `resources` order.
+    pub async fn get_write_locks_batch(
+        &self,
+        resources: &[ObjectKey],
+        timeout: Duration,
+    ) -> Vec<std::result::Result<NamespaceLockGuard, crate::error::LockError>> {
+        self.lock.acquire_write_locks_batch(resources, &self.owner, timeout).await
+    }
 }
 
 impl NamespaceLockGuard {
@@ -349,6 +360,38 @@ impl NamespaceLock {
         }
     }
 
+    /// Acquire exclusive locks for many independent resources in one batched
+    /// round; see [`crate::distributed_lock::DistributedLock::acquire_guard_batch`].
+    /// Returns one guard or error per resource, in `resources` order.
+    ///
+    /// This is the lock primitive bulk mutations (S3 `DeleteObjects`) should
+    /// use: per-key quorum round-trips serialize a 1000-key batch, while the
+    /// batched form is one wire round per lock client and stays deadlock-free
+    /// without any canonical serial fallback (every lock server orders
+    /// concurrent batches by the same global key order).
+    pub async fn acquire_write_locks_batch(
+        &self,
+        resources: &[ObjectKey],
+        owner: &str,
+        timeout: Duration,
+    ) -> Vec<std::result::Result<NamespaceLockGuard, crate::error::LockError>> {
+        let ttl = crate::fast_lock::DEFAULT_LOCK_TIMEOUT;
+        match self {
+            Self::Distributed(lock) => lock
+                .acquire_guard_batch(resources, owner, timeout, ttl)
+                .await
+                .into_iter()
+                .map(|result| result.map(NamespaceLockGuard::Standard))
+                .collect(),
+            Self::Local(lock) => lock
+                .acquire_guard_batch(resources, owner, timeout, ttl)
+                .await
+                .into_iter()
+                .map(|result| result.map(NamespaceLockGuard::Fast))
+                .collect(),
+        }
+    }
+
     /// Get health information
     pub async fn get_health(&self) -> crate::types::HealthInfo {
         let lock_stats = self.get_stats().await;

crates/obs/src/telemetry/filter.rs

@@ -41,6 +41,16 @@ pub(super) fn pyroscope_log_filter() -> FilterFn {
     filter_fn(allows_non_pyroscope_dependency_event)
 }
 
+/// s3s verbose events include response bodies and expected signatures. Keep
+/// diagnostics spans, but never let these events reach logs or OTLP exporters.
+pub(super) fn s3s_log_filter() -> FilterFn {
+    filter_fn(|metadata| {
+        !(metadata.is_event()
+            && (metadata.target() == "s3s" || metadata.target().starts_with("s3s::"))
+            && *metadata.level() > tracing::Level::INFO)
+    })
+}
+
 /// Build an `EnvFilter` from the given log level string.
 ///
 /// If `default_level` is provided, it is used directly. Otherwise, the
@@ -339,6 +349,47 @@ mod tests {
         format!("{HTTP_SERVER_LOG_TARGET}={level}")
     }
 
+    #[test]
+    fn s3s_filter_blocks_credentials_despite_explicit_verbose_directives() {
+        use tracing_subscriber::prelude::*;
+        for directive in ["trace", concat!("trace,", "s3s::service=trace,", "s3s::ops::signature=debug")] {
+            let writer = SharedWriter::default();
+            let captured = Arc::clone(&writer.0);
+            let subscriber = tracing_subscriber::registry()
+                .with(EnvFilter::new(directive))
+                .with(s3s_log_filter())
+                .with(
+                    tracing_subscriber::fmt::layer()
+                        .without_time()
+                        .with_ansi(false)
+                        .with_writer(writer)
+                        .with_span_events(tracing_subscriber::fmt::format::FmtSpan::NEW),
+                );
+            with_default(subscriber, || {
+                let span = tracing::debug_span!(target: "s3s::service", "safe-operation-span");
+                let _entered = span.enter();
+                tracing::debug!(target: "s3s::service", response = "secret-response", "response dump");
+                tracing::trace!(target: "s3s", response = "secret-trace", "response dump");
+                tracing::debug!(target: "s3s::ops::signature", expected = "secret-signature", "signature mismatch");
+                tracing::info!(target: "s3s::service", "visible-info");
+                tracing::warn!(target: "s3s::service", "visible-warning");
+                tracing::debug!(target: "rustfs", "visible-rustfs-debug");
+                tracing::debug!(target: "s3s_other", "visible-other-target");
+            });
+            let output = String::from_utf8(captured.lock().expect("captured logs").clone()).expect("UTF-8 logs");
+            assert!(!output.contains("secret"), "{output}");
+            for expected in [
+                "safe-operation-span",
+                "visible-info",
+                "visible-warning",
+                "visible-rustfs-debug",
+                "visible-other-target",
+            ] {
+                assert!(output.contains(expected), "missing {expected}: {output}");
+            }
+        }
+    }
+
     #[test]
     fn test_is_verbose_level() {
         assert!(is_verbose_level("debug"));

crates/obs/src/telemetry/local.rs

@@ -36,7 +36,7 @@ use crate::cleaner::LogCleaner;
 use crate::cleaner::types::{CompressionAlgorithm, FileMatchMode};
 use crate::config::OtelConfig;
 use crate::global::{METRIC_LOG_CLEANER_RUN_FAILURES_TOTAL, METRIC_LOG_CLEANER_RUNS_TOTAL, set_observability_metric_enabled};
-use crate::telemetry::filter::{build_env_filter, effective_verbose_logging, pyroscope_log_filter};
+use crate::telemetry::filter::{build_env_filter, effective_verbose_logging, pyroscope_log_filter, s3s_log_filter};
 use crate::telemetry::rolling::{RollingAppender, Rotation};
 use metrics::counter;
 use rustfs_config::observability::{
@@ -369,6 +369,7 @@ fn init_stdout_only(_config: &OtelConfig, logger_level: &str, is_production: boo
     tracing_subscriber::registry()
         .with(env_filter)
         .with(pyroscope_log_filter())
+        .with(s3s_log_filter())
         .with(ErrorLayer::default())
         .with(fmt_layer)
         .try_init()
@@ -486,6 +487,7 @@ fn init_file_logging_internal(
     tracing_subscriber::registry()
         .with(env_filter)
         .with(pyroscope_log_filter())
+        .with(s3s_log_filter())
         .with(ErrorLayer::default())
         .with(file_layer)
         .with(stdout_layer)

crates/obs/src/telemetry/otel.rs

@@ -39,7 +39,7 @@
 use crate::cleaner::types::FileMatchMode;
 use crate::config::OtelConfig;
 use crate::global::set_observability_metric_enabled;
-use crate::telemetry::filter::{build_env_filter, pyroscope_log_filter};
+use crate::telemetry::filter::{build_env_filter, pyroscope_log_filter, s3s_log_filter};
 use crate::telemetry::guard::{OtelGuard, ProfilingAgent};
 use crate::telemetry::local::{build_json_log_layer, spawn_cleanup_task};
 use crate::telemetry::recorder::{Recorder, install_process_global_recorder};
@@ -296,6 +296,7 @@ pub(super) fn init_observability_http(
     tracing_subscriber::registry()
         .with(filter)
         .with(pyroscope_log_filter())
+        .with(s3s_log_filter())
         .with(ErrorLayer::default())
         .with(file_layer_opt)
         .with(stdout_layer_opt)
@@ -1196,6 +1197,7 @@ mod tests {
         let subscriber = tracing_subscriber::registry()
             .with(env_filter)
             .with(pyroscope_log_filter())
+            .with(s3s_log_filter())
             .with(
                 tracing_subscriber::fmt::layer()
                     .with_ansi(false)
@@ -1219,6 +1221,7 @@ mod tests {
             tracing_subscriber::registry()
                 .with(build_env_filter("info", None))
                 .with(pyroscope_log_filter())
+                .with(s3s_log_filter())
                 .with(tracing_subscriber::fmt::layer().with_ansi(false).without_time())
                 .try_init()
                 .expect("install isolated LogTracer subscriber");

crates/policy/src/arn.rs

@@ -20,7 +20,7 @@ const ARN_PARTITION_RUSTFS: &str = "rustfs";
 const ARN_SERVICE_IAM: &str = "iam";
 const ARN_RESOURCE_TYPE_ROLE: &str = "role";
 
-#[derive(Debug, PartialEq, Eq, Hash)]
+#[derive(Debug, Clone, PartialEq, Eq, Hash)]
 pub struct ARN {
     pub partition: String,
     pub service: String,

crates/policy/src/policy/action.rs

@@ -165,9 +165,10 @@ impl<'de> Deserialize<'de> for ActionSet {
     }
 }
 
-#[derive(Serialize, Deserialize, Hash, PartialEq, Eq, Clone, Debug, Copy)]
+#[derive(Deserialize, Hash, PartialEq, Eq, Clone, Debug, Copy)]
 #[serde(try_from = "&str", untagged)]
 pub enum Action {
+    All,
     S3Action(S3Action),
     AdminAction(AdminAction),
     StsAction(StsAction),
@@ -175,12 +176,29 @@ pub enum Action {
     None,
 }
 
+impl Serialize for Action {
+    fn serialize<S>(&self, serializer: S) -> std::result::Result<S::Ok, S::Error>
+    where
+        S: serde::Serializer,
+    {
+        match self {
+            Self::None => serializer.serialize_unit(),
+            _ => serializer.serialize_str(self.into()),
+        }
+    }
+}
+
 impl Action {
     pub fn is_match(&self, action: &Action) -> bool {
         self.is_match_for_effect(action, false)
     }
 
     pub fn is_match_for_effect(&self, action: &Action, deny: bool) -> bool {
+        if matches!(self, Self::All) {
+            // Preserve the historical S3-only Allow while allowing deny-all and
+            // NotAction exclusions to cover every action family.
+            return deny || Self::S3Action(S3Action::AllActions).is_match_for_effect(action, false);
+        }
         // Force-delete bypasses emptiness and version checks. A blanket `s3:*`
         // / `*` Allow (including canned consoleAdmin) must not confer it; the
         // statement has to name `s3:ForceDeleteBucket` or `s3:ForceDeleteObject`.
@@ -202,6 +220,7 @@ fn action_requires_explicit_grant(action: &Action) -> bool {
 impl From<&Action> for &str {
     fn from(value: &Action) -> &'static str {
         match value {
+            Action::All => "*",
             Action::S3Action(s) => s.into(),
             Action::AdminAction(s) => s.into(),
             Action::StsAction(s) => s.into(),
@@ -221,9 +240,8 @@ impl Action {
 impl TryFrom<&str> for Action {
     type Error = Error;
     fn try_from(value: &str) -> std::result::Result<Self, Self::Error> {
-        // Support wildcard "*" which matches all S3 actions (AWS S3 standard)
         if value == "*" {
-            return Ok(Self::S3Action(S3Action::AllActions));
+            return Ok(Self::All);
         }
         if value.starts_with(Self::S3_PREFIX) {
             Ok(Self::S3Action(
@@ -822,7 +840,7 @@ mod tests {
     #[test]
     fn blanket_s3_wildcard_does_not_grant_force_delete() {
         let wildcard = Action::try_from("s3:*").expect("s3:* parses");
-        let star = Action::try_from("*").expect("* parses as s3:*");
+        let star = Action::try_from("*").expect("* parses");
         let force_bucket = Action::try_from("s3:ForceDeleteBucket").expect("force bucket action parses");
         let force_object = Action::try_from("s3:ForceDeleteObject").expect("force object action parses");
         let delete_object = Action::try_from("s3:DeleteObject").expect("delete object parses");
@@ -841,9 +859,10 @@ mod tests {
 
     #[test]
     fn test_action_wildcard_parsing() {
-        // Test that "*" parses to S3Action::AllActions
+        // Preserve the bare wildcard through serialization.
         let action = Action::try_from("*").expect("Should parse wildcard");
-        assert!(matches!(action, Action::S3Action(S3Action::AllActions)));
+        assert!(matches!(action, Action::All));
+        assert_eq!(serde_json::to_string(&action).expect("serialize wildcard"), r#""*""#);
     }
 
     #[test]
@@ -929,15 +948,13 @@ mod tests {
 
     #[test]
     fn test_actionset_wildcard_serialization() {
-        // Wildcard action should serialize as array for S3 specification compliance
-        let actionset = ActionSet(vec![Action::try_from("*").expect("Should parse wildcard")]);
-
-        let json = serde_json::to_string(&actionset).expect("Should serialize");
-        let parsed: serde_json::Value = serde_json::from_str(&json).expect("Should parse");
-        assert!(parsed.is_array(), "Should serialize as array");
-        let arr = parsed.as_array().expect("Should be array");
-        assert_eq!(arr.len(), 1);
-        assert_eq!(arr[0].as_str().unwrap(), "s3:*");
+        for wildcard in ["*", "s3:*"] {
+            let actionset = ActionSet(vec![Action::try_from(wildcard).expect("parse wildcard")]);
+            let json = serde_json::to_string(&actionset).expect("serialize action set");
+            assert_eq!(json, format!(r#"["{wildcard}"]"#));
+            let round_trip: ActionSet = serde_json::from_str(&json).expect("deserialize action set");
+            assert_eq!(round_trip, actionset);
+        }
     }
 
     #[test]

crates/policy/src/policy/function/addr.rs

@@ -15,7 +15,7 @@
 use super::func::InnerFunc;
 use ipnetwork::IpNetwork;
 use serde::{Deserialize, Serialize, de::Visitor};
-use std::{borrow::Cow, collections::HashMap, net::IpAddr};
+use std::{collections::HashMap, net::IpAddr};
 
 pub type AddrFunc = InnerFunc<AddrFuncValue>;
 
@@ -30,7 +30,16 @@ impl AddrFunc {
                 };
 
                 for ip_net in inner.values.0.iter() {
-                    if ip_net.contains(ip) {
+                    let canonical = ip.to_canonical();
+                    let matches_mapped_network = match (ip_net, canonical) {
+                        (IpNetwork::V6(network), IpAddr::V4(address))
+                            if network.prefix() >= 96 && network.ip().to_ipv4_mapped().is_some() =>
+                        {
+                            network.contains(address.to_ipv6_mapped())
+                        }
+                        _ => false,
+                    };
+                    if ip_net.contains(ip) || ip_net.contains(canonical) || matches_mapped_network {
                         return true;
                     }
                 }
@@ -81,13 +90,7 @@ impl<'de> Deserialize<'de> for AddrFuncValue {
 
         impl AddrFuncValueVisitor {
             fn cidr<E: serde::de::Error>(v: &str) -> Result<IpNetwork, E> {
-                let mut cidr_str = Cow::from(v);
-                if v.find('/').is_none() {
-                    cidr_str.to_mut().push_str("/32");
-                }
-
-                cidr_str
-                    .parse::<IpNetwork>()
+                v.parse::<IpNetwork>()
                     .map_err(|_| E::custom(format!("{v} can not be parsed to CIDR")))
             }
         }
@@ -107,6 +110,40 @@ mod tests {
     };
     use test_case::test_case;
 
+    #[test]
+    fn source_ip_matches_ipv4_and_mapped_forms_across_protocols() {
+        for network in ["203.0.113.0/24", "::ffff:203.0.113.0/120"] {
+            let function: AddrFunc =
+                serde_json::from_str(&format!(r#"{{"aws:SourceIp":"{network}"}}"#)).expect("parse source IP condition");
+            for address in ["203.0.113.9", "::ffff:203.0.113.9"] {
+                assert!(
+                    function.evaluate(&std::collections::HashMap::from([("SourceIp".to_string(), vec![address.to_string()]),])),
+                    "{network} must match {address}"
+                );
+            }
+            for address in ["198.51.100.9", "::ffff:198.51.100.9", "2001:db8::9"] {
+                assert!(
+                    !function.evaluate(&std::collections::HashMap::from([("SourceIp".to_string(), vec![address.to_string()]),])),
+                    "{network} must not match {address}"
+                );
+            }
+        }
+    }
+
+    #[test]
+    fn bare_address_matches_only_the_named_host() {
+        for (host, neighbor, prefix) in [
+            ("2001:db8:85a3::8a2e:370:7334", "2001:db8:ffff::1", 128),
+            ("203.0.113.7", "203.0.113.8", 32),
+        ] {
+            let function: AddrFunc =
+                serde_json::from_str(&format!(r#"{{"aws:SourceIp":"{host}"}}"#)).expect("parse bare IP address condition");
+            assert_eq!(function.0[0].values.0[0].prefix(), prefix);
+            assert!(function.evaluate(&std::collections::HashMap::from([("SourceIp".to_string(), vec![host.to_string()])])));
+            assert!(!function.evaluate(&std::collections::HashMap::from([("SourceIp".to_string(), vec![neighbor.to_string()])])));
+        }
+    }
+
     fn new_func(name: KeyName, variable: Option<String>, value: Vec<&str>) -> AddrFunc {
         AddrFunc {
             0: vec![FuncKeyValue {
@@ -119,20 +156,20 @@ mod tests {
     #[test_case(r#"{"aws:SourceIp": "203.0.113.0/24"}"#, new_func(Aws(AWSSourceIP), None, vec!["203.0.113.0/24"]); "1")]
     #[test_case(r#"{"aws:SourceIp": "203.0.113.0"}"#, new_func(Aws(AWSSourceIP), None, vec!["203.0.113.0/32"]); "2")]
     #[test_case(r#"{"aws:SourceIp": "2001:DB8:1234:5678::/64"}"#, new_func(Aws(AWSSourceIP),None, vec!["2001:DB8:1234:5678::/64"]); "3")]
-    #[test_case(r#"{"aws:SourceIp": "2001:DB8:1234:5678::"}"#, new_func(Aws(AWSSourceIP), None, vec!["2001:DB8:1234:5678::/32"]); "4")]
+    #[test_case(r#"{"aws:SourceIp": "2001:DB8:1234:5678::"}"#, new_func(Aws(AWSSourceIP), None, vec!["2001:DB8:1234:5678::/128"]); "4")]
     #[test_case(r#"{"aws:SourceIp": ["203.0.113.0/24","203.0.113.0"]}"#, new_func(Aws(AWSSourceIP), None, vec!["203.0.113.0/24", "203.0.113.0/32"]); "5")]
     #[test_case(r#"{"aws:SourceIp": ["2001:DB8:1234:5678::/64","203.0.113.0/24"]}"#, new_func(Aws(AWSSourceIP), None, vec!["2001:DB8:1234:5678::/64", "203.0.113.0/24"]); "6")]
-    #[test_case(r#"{"aws:SourceIp": ["2001:DB8:1234:5678::/64", "2001:DB8:1234:5678::"]}"#, new_func(Aws(AWSSourceIP),None, vec!["2001:DB8:1234:5678::/64", "2001:DB8:1234:5678::/32"]); "7")]
-    #[test_case(r#"{"aws:SourceIp": ["2001:DB8:1234:5678::", "203.0.113.0"]}"#, new_func(Aws(AWSSourceIP), None, vec!["2001:DB8:1234:5678::/32", "203.0.113.0/32"]); "8")]
+    #[test_case(r#"{"aws:SourceIp": ["2001:DB8:1234:5678::/64", "2001:DB8:1234:5678::"]}"#, new_func(Aws(AWSSourceIP),None, vec!["2001:DB8:1234:5678::/64", "2001:DB8:1234:5678::/128"]); "7")]
+    #[test_case(r#"{"aws:SourceIp": ["2001:DB8:1234:5678::", "203.0.113.0"]}"#, new_func(Aws(AWSSourceIP), None, vec!["2001:DB8:1234:5678::/128", "203.0.113.0/32"]); "8")]
     #[test_case(r#"{"aws:SourceIp/a": "203.0.113.0/24"}"#, new_func(Aws(AWSSourceIP), Some("a".into()), vec!["203.0.113.0/24"]); "9")]
     #[test_case(r#"{"aws:SourceIp/a": "203.0.113.0/24"}"#, new_func(Aws(AWSSourceIP), Some("a".into()), vec!["203.0.113.0/24"]); "10")]
     #[test_case(r#"{"aws:SourceIp/a": "203.0.113.0"}"#, new_func(Aws(AWSSourceIP), Some("a".into()), vec!["203.0.113.0/32"]); "11")]
     #[test_case(r#"{"aws:SourceIp/a": "2001:DB8:1234:5678::/64"}"#, new_func(Aws(AWSSourceIP),Some("a".into()), vec!["2001:DB8:1234:5678::/64"]); "12")]
-    #[test_case(r#"{"aws:SourceIp/a": "2001:DB8:1234:5678::"}"#, new_func(Aws(AWSSourceIP), Some("a".into()), vec!["2001:DB8:1234:5678::/32"]); "13")]
+    #[test_case(r#"{"aws:SourceIp/a": "2001:DB8:1234:5678::"}"#, new_func(Aws(AWSSourceIP), Some("a".into()), vec!["2001:DB8:1234:5678::/128"]); "13")]
     #[test_case(r#"{"aws:SourceIp/a": ["203.0.113.0/24", "203.0.113.0"]}"#, new_func(Aws(AWSSourceIP), Some("a".into()), vec!["203.0.113.0/24", "203.0.113.0/32"]); "14")]
     #[test_case(r#"{"aws:SourceIp/a": ["2001:DB8:1234:5678::/64", "203.0.113.0/24"]}"#, new_func(Aws(AWSSourceIP), Some("a".into()), vec!["2001:DB8:1234:5678::/64", "203.0.113.0/24"]); "15")]
-    #[test_case(r#"{"aws:SourceIp/a": ["2001:DB8:1234:5678::/64", "2001:DB8:1234:5678::"]}"#, new_func(Aws(AWSSourceIP),Some("a".into()), vec!["2001:DB8:1234:5678::/64", "2001:DB8:1234:5678::/32"]); "16")]
-    #[test_case(r#"{"aws:SourceIp/a": ["2001:DB8:1234:5678::", "203.0.113.0"]}"#, new_func(Aws(AWSSourceIP), Some("a".into()), vec!["2001:DB8:1234:5678::/32", "203.0.113.0/32"]); "17")]
+    #[test_case(r#"{"aws:SourceIp/a": ["2001:DB8:1234:5678::/64", "2001:DB8:1234:5678::"]}"#, new_func(Aws(AWSSourceIP),Some("a".into()), vec!["2001:DB8:1234:5678::/64", "2001:DB8:1234:5678::/128"]); "16")]
+    #[test_case(r#"{"aws:SourceIp/a": ["2001:DB8:1234:5678::", "203.0.113.0"]}"#, new_func(Aws(AWSSourceIP), Some("a".into()), vec!["2001:DB8:1234:5678::/128", "203.0.113.0/32"]); "17")]
     fn test_deser(input: &str, expect: AddrFunc) -> Result<(), serde_json::Error> {
         let v: AddrFunc = serde_json::from_str(input)?;
         assert_eq!(v, expect);

crates/policy/src/policy/function/condition.rs

@@ -61,6 +61,11 @@ pub enum Condition {
 
 impl Condition {
     pub fn from_deserializer<'a, D: MapAccess<'a>>(key: &str, d: &mut D) -> Result<Self, D::Error> {
+        // Two suffixes are retained for existing serialized policies. Bound the
+        // chain before parsing so Clone, evaluation and Drop stay bounded too.
+        if key.ends_with("IfExistsIfExistsIfExists") {
+            return Err(Error::custom("condition operator has too many IfExists suffixes"));
+        }
         Ok(match key {
             "StringEquals" => Self::StringEquals(d.next_value()?),
             "StringNotEquals" => Self::StringNotEquals(d.next_value()?),
@@ -262,18 +267,18 @@ impl Condition {
                 Null(s) => s.evaluate_null(values),
                 Bool(s) => s.evaluate_bool(values),
                 NumericEquals(s) => s.evaluate(i64::eq, false, values),
-                NumericNotEquals(s) => s.evaluate(i64::ne, false, values),
+                NumericNotEquals(s) => s.evaluate(i64::ne, true, values),
                 NumericLessThan(s) => s.evaluate(i64::lt, false, values),
                 NumericLessThanEquals(s) => s.evaluate(i64::le, false, values),
                 NumericGreaterThan(s) => s.evaluate(i64::gt, false, values),
                 NumericGreaterThanIfExists(s) => s.evaluate(i64::ge, true, values),
                 NumericGreaterThanEquals(s) => s.evaluate(i64::ge, false, values),
-                DateEquals(s) => s.evaluate(OffsetDateTime::eq, values),
-                DateNotEquals(s) => s.evaluate(OffsetDateTime::ne, values),
-                DateLessThan(s) => s.evaluate(OffsetDateTime::lt, values),
-                DateLessThanEquals(s) => s.evaluate(OffsetDateTime::le, values),
-                DateGreaterThan(s) => s.evaluate(OffsetDateTime::gt, values),
-                DateGreaterThanEquals(s) => s.evaluate(OffsetDateTime::ge, values),
+                DateEquals(s) => s.evaluate(OffsetDateTime::eq, false, values),
+                DateNotEquals(s) => s.evaluate(OffsetDateTime::ne, true, values),
+                DateLessThan(s) => s.evaluate(OffsetDateTime::lt, false, values),
+                DateLessThanEquals(s) => s.evaluate(OffsetDateTime::le, false, values),
+                DateGreaterThan(s) => s.evaluate(OffsetDateTime::gt, false, values),
+                DateGreaterThanEquals(s) => s.evaluate(OffsetDateTime::ge, false, values),
                 IfExists(inner) => {
                     if !inner.has_any_key_in(values) {
                         return true;
@@ -486,6 +491,31 @@ mod tests {
         assert_eq!(functions, deserialized);
     }
 
+    #[test]
+    fn repeated_if_exists_suffixes_are_rejected_before_recursion() {
+        for count in [3, 10_000] {
+            let key = format!("StringEquals{}", "IfExists".repeat(count));
+            let document = serde_json::json!({key: {"s3:x-amz-server-side-encryption": "aws:kms"}});
+            let result = serde_json::from_slice::<Functions>(&serde_json::to_vec(&document).expect("serialize condition"));
+            assert!(result.is_err(), "{count} suffixes must be rejected without constructing a deep chain");
+        }
+    }
+
+    #[tokio::test]
+    async fn numeric_and_date_not_equals_match_absent_keys() {
+        for (operator, key, value, absent) in [
+            ("NumericNotEquals", "s3:max-keys", "30", true),
+            ("NumericEquals", "s3:max-keys", "30", false),
+            ("DateNotEquals", "s3:object-lock-retain-until-date", "2030-01-01T00:00:00Z", true),
+            ("DateEquals", "s3:object-lock-retain-until-date", "2030-01-01T00:00:00Z", false),
+        ] {
+            let document = serde_json::json!({operator: {key: value}});
+            let functions: Functions =
+                serde_json::from_slice(&serde_json::to_vec(&document).expect("serialize condition")).expect("parse condition");
+            assert_eq!(functions.evaluate(&HashMap::new()).await, absent, "{operator}");
+        }
+    }
+
     #[test]
     fn test_nested_if_exists_serde_round_trip() {
         // Verifies that nested IfExists(IfExists(StringEquals)) serializes with the

crates/policy/src/policy/function/date.rs

@@ -13,6 +13,7 @@
 // limitations under the License.
 
 use super::func::InnerFunc;
+use super::key_name::{AwsKeyName, KeyName};
 use serde::{Deserialize, Deserializer, Serialize, de};
 use std::{collections::HashMap, fmt};
 use time::{OffsetDateTime, format_description::well_known::Rfc3339};
@@ -20,14 +21,27 @@ use time::{OffsetDateTime, format_description::well_known::Rfc3339};
 pub type DateFunc = InnerFunc<DateFuncValue>;
 
 impl DateFunc {
-    pub fn evaluate(&self, op: impl Fn(&OffsetDateTime, &OffsetDateTime) -> bool, values: &HashMap<String, Vec<String>>) -> bool {
+    pub fn evaluate(
+        &self,
+        op: impl Fn(&OffsetDateTime, &OffsetDateTime) -> bool,
+        absent: bool,
+        values: &HashMap<String, Vec<String>>,
+    ) -> bool {
         for inner in self.0.iter() {
             let v = match values.get(inner.key.name().as_str()).and_then(|x| x.first()) {
                 Some(x) => x,
+                None if absent => continue,
                 None => return false,
             };
 
-            let Ok(rv) = OffsetDateTime::parse(v, &Rfc3339) else {
+            let parsed = if inner.key.is(&KeyName::Aws(AwsKeyName::AWSEpochTime)) {
+                v.parse::<i64>()
+                    .ok()
+                    .and_then(|seconds| OffsetDateTime::from_unix_timestamp(seconds).ok())
+            } else {
+                OffsetDateTime::parse(v, &Rfc3339).ok()
+            };
+            let Some(rv) = parsed else {
                 return false;
             };
 
@@ -88,7 +102,7 @@ impl<'de> Deserialize<'de> for DateFuncValue {
 
 #[cfg(test)]
 mod tests {
-    use super::{DateFunc, DateFuncValue};
+    use super::{AwsKeyName, DateFunc, DateFuncValue};
     use crate::policy::function::func::FuncKeyValue;
     use crate::policy::function::{
         key::Key,
@@ -124,15 +138,50 @@ mod tests {
         Ok(())
     }
 
+    #[test]
+    fn date_conditions_compare_epoch_seconds_only_for_epoch_time() {
+        let function = new_func(KeyName::Aws(AwsKeyName::AWSEpochTime), None, "2020-01-01T00:00:00Z");
+        for (value, equal, later) in [
+            ("1577836799", false, false),
+            ("1577836800", true, false),
+            ("1577836801", false, true),
+        ] {
+            let values = HashMap::from([("EpochTime".to_string(), vec![value.to_string()])]);
+            assert_eq!(function.evaluate(OffsetDateTime::eq, false, &values), equal);
+            assert_eq!(function.evaluate(OffsetDateTime::gt, false, &values), later);
+            assert_eq!(function.evaluate(OffsetDateTime::lt, false, &values), !equal && !later);
+        }
+        for invalid in ["2020-01-01T00:00:00Z", "invalid", "9223372036854775807"] {
+            let values = HashMap::from([("EpochTime".to_string(), vec![invalid.to_string()])]);
+            assert!(!function.evaluate(OffsetDateTime::ne, true, &values));
+        }
+        let header_date = new_func(S3(S3ObjectLockRetainUntilDate), None, "2020-01-01T00:00:00Z");
+        let values = HashMap::from([("object-lock-retain-until-date".to_string(), vec!["1577836800".to_string()])]);
+        assert!(!header_date.evaluate(OffsetDateTime::eq, false, &values));
+    }
+
+    #[test]
+    fn absent_date_key_does_not_skip_other_keys() {
+        let mut function = new_func(S3(S3ObjectLockRetainUntilDate), None, "2030-01-01T00:00:00Z");
+        function
+            .0
+            .extend(new_func(KeyName::Aws(AwsKeyName::AWSCurrentTime), None, "2030-01-01T00:00:00Z").0);
+        let values = HashMap::from([("CurrentTime".to_string(), vec!["2030-01-01T00:00:00Z".to_string()])]);
+        assert!(!function.evaluate(OffsetDateTime::ne, true, &values));
+        assert!(function.evaluate(OffsetDateTime::eq, true, &values));
+        function.0.reverse();
+        assert!(!function.evaluate(OffsetDateTime::ne, true, &values));
+    }
+
     #[test]
     fn evaluate_compares_request_date_to_policy_date() {
         let function = new_func(S3(S3ObjectLockRetainUntilDate), None, "2030-01-01T00:00:00Z");
         let later = HashMap::from([("object-lock-retain-until-date".to_string(), vec!["2099-01-01T00:00:00Z".to_string()])]);
         let earlier = HashMap::from([("object-lock-retain-until-date".to_string(), vec!["2029-01-01T00:00:00Z".to_string()])]);
 
-        assert!(function.evaluate(OffsetDateTime::gt, &later));
-        assert!(!function.evaluate(OffsetDateTime::gt, &earlier));
-        assert!(function.evaluate(OffsetDateTime::lt, &earlier));
-        assert!(!function.evaluate(OffsetDateTime::lt, &later));
+        assert!(function.evaluate(OffsetDateTime::gt, false, &later));
+        assert!(!function.evaluate(OffsetDateTime::gt, false, &earlier));
+        assert!(function.evaluate(OffsetDateTime::lt, false, &earlier));
+        assert!(!function.evaluate(OffsetDateTime::lt, false, &later));
     }
 }

crates/policy/src/policy/function/number.rs

@@ -26,11 +26,12 @@ pub type NumberFunc = InnerFunc<NumberFuncValue>;
 pub struct NumberFuncValue(i64);
 
 impl NumberFunc {
-    pub fn evaluate(&self, op: impl Fn(&i64, &i64) -> bool, if_exists: bool, values: &HashMap<String, Vec<String>>) -> bool {
+    pub fn evaluate(&self, op: impl Fn(&i64, &i64) -> bool, absent: bool, values: &HashMap<String, Vec<String>>) -> bool {
         for inner in self.0.iter() {
             let v = match values.get(inner.key.name().as_str()).and_then(|x| x.first()) {
                 Some(x) => x,
-                None => return if_exists,
+                None if absent => continue,
+                None => return false,
             };
 
             let Ok(rv) = v.parse::<i64>() else {
@@ -146,6 +147,20 @@ mod tests {
         assert_eq!(limit.evaluate(i64::eq, false, &values), equals);
     }
 
+    #[test]
+    fn absent_numeric_key_does_not_skip_other_keys() {
+        let mut function = new_func(S3(S3MaxKeys), None, 30);
+        function
+            .0
+            .extend(new_func(S3(S3ObjectLockRemainingRetentionDays), None, 30).0);
+        let values = HashMap::from([("object-lock-remaining-retention-days".to_string(), vec!["30".to_string()])]);
+        assert!(!function.evaluate(i64::ne, true, &values));
+        assert!(!function.evaluate(i64::gt, true, &values));
+        assert!(function.evaluate(i64::eq, true, &values));
+        function.0.reverse();
+        assert!(!function.evaluate(i64::ne, true, &values));
+    }
+
     #[test]
     fn remaining_retention_days_numeric_condition_without_value_uses_if_exists() {
         let limit = new_func(S3(S3ObjectLockRemainingRetentionDays), None, 30);

crates/policy/src/policy/policy.rs

@@ -3416,6 +3416,277 @@ mod test {
         assert_eq!(round_trip.statements[0].effect, policy.statements[0].effect);
     }
 
+    #[tokio::test]
+    async fn global_wildcard_deny_covers_all_families_without_broadening_allow() {
+        for action in [
+            "s3:GetObject",
+            "s3:ForceDeleteObject",
+            "admin:CreateUser",
+            "sts:AssumeRole",
+            "kms:Configure",
+            "kms:Decrypt",
+        ] {
+            let parsed_action = Action::try_from(action).expect("action parses");
+            let conditions = HashMap::new();
+            let claims = HashMap::new();
+            let args = Args {
+                account: "user",
+                groups: &None,
+                action: parsed_action,
+                bucket: if action.starts_with("s3:") { "bucket" } else { "" },
+                object: if action.starts_with("s3:") { "object" } else { "" },
+                conditions: &conditions,
+                claims: &claims,
+                is_owner: false,
+                deny_only: false,
+            };
+            for (effect, selector, wildcard, expected) in [
+                ("Deny", "Action", "*", false),
+                ("Deny", "Action", "s3:*", !action.starts_with("s3:")),
+                ("Deny", "NotAction", "*", true),
+            ] {
+                let document = serde_json::json!({
+                    "Version": "2012-10-17", "Statement": [
+                        {"Effect": "Allow", "Action": action, "Resource": "*"},
+                        {"Effect": effect, selector: wildcard, "Resource": "*"}
+                    ]
+                });
+                let policy = Policy::parse_config(&serde_json::to_vec(&document).expect("serialize policy"))
+                    .expect("parse wildcard policy");
+                assert_eq!(policy.is_allowed(&args).await, expected, "{effect} {selector} {wildcard} for {action}");
+                let serialized = serde_json::to_value(&policy).expect("serialize policy");
+                assert_eq!(serialized["Statement"][1][selector][0], wildcard);
+            }
+            for selector in ["Action", "NotAction"] {
+                let document = serde_json::json!({
+                    "Version": "2012-10-17", "Statement": [{"Effect": "Allow", selector: "*", "Resource": "*"}]
+                });
+                let policy = Policy::parse_config(&serde_json::to_vec(&document).expect("serialize policy"))
+                    .expect("parse wildcard Allow");
+                assert_eq!(policy.is_allowed(&args).await, selector == "Action" && action == "s3:GetObject");
+            }
+        }
+    }
+
+    #[tokio::test]
+    async fn global_wildcard_deny_still_obeys_resource_and_condition_scope() {
+        let document = serde_json::json!({
+            "Version": "2012-10-17", "Statement": [
+                {"Effect": "Allow", "Action": "admin:CreateUser", "Resource": "*"},
+                {"Effect": "Deny", "Action": "*", "Resource": "arn:aws:s3:::bucket/*"}
+            ]
+        });
+        let policy = Policy::parse_config(&serde_json::to_vec(&document).expect("serialize policy")).expect("parse scoped Deny");
+        let claims = HashMap::new();
+        let conditions = HashMap::new();
+        let mut args = Args {
+            account: "user",
+            groups: &None,
+            action: Action::try_from("admin:CreateUser").expect("admin action"),
+            bucket: "",
+            object: "",
+            conditions: &conditions,
+            claims: &claims,
+            is_owner: false,
+            deny_only: false,
+        };
+        assert!(policy.is_allowed(&args).await, "bucket-scoped Deny must not become a global admin Deny");
+        let mut document = document;
+        document["Statement"][1]["Resource"] = serde_json::json!("*");
+        document["Statement"][1]["Condition"] = serde_json::json!({"StringEquals": {"aws:username": "blocked"}});
+        let conditional =
+            Policy::parse_config(&serde_json::to_vec(&document).expect("serialize policy")).expect("parse conditional Deny");
+        assert!(conditional.is_allowed(&args).await);
+        let conditions = HashMap::from([("username".to_string(), vec!["blocked".to_string()])]);
+        args.conditions = &conditions;
+        assert!(!conditional.is_allowed(&args).await);
+    }
+
+    #[tokio::test]
+    async fn security_conditions_enforce_identity_and_bucket_denies() {
+        for (operator, key, policy_value, request_value, denied) in [
+            ("NumericNotEquals", "s3:object-lock-remaining-retention-days", "30", None, true),
+            ("NumericNotEquals", "s3:object-lock-remaining-retention-days", "30", Some("30"), false),
+            ("NumericNotEquals", "s3:object-lock-remaining-retention-days", "30", Some("10"), true),
+            ("DateNotEquals", "s3:object-lock-retain-until-date", "2030-01-01T00:00:00Z", None, true),
+            (
+                "DateNotEquals",
+                "s3:object-lock-retain-until-date",
+                "2030-01-01T00:00:00Z",
+                Some("2030-01-01T00:00:00Z"),
+                false,
+            ),
+            ("DateGreaterThan", "aws:EpochTime", "2020-01-01T00:00:00Z", Some("1577836801"), true),
+            ("DateGreaterThan", "aws:EpochTime", "2020-01-01T00:00:00Z", Some("1577836799"), false),
+            (
+                "NotIpAddress",
+                "aws:SourceIp",
+                "2001:db8:85a3::8a2e:370:7334",
+                Some("2001:db8:ffff::1"),
+                true,
+            ),
+            (
+                "NotIpAddress",
+                "aws:SourceIp",
+                "2001:db8:85a3::8a2e:370:7334",
+                Some("2001:db8:85a3::8a2e:370:7334"),
+                false,
+            ),
+            (
+                "IpAddress",
+                "aws:SourceIp",
+                "2001:db8:85a3::8a2e:370:7334",
+                Some("2001:db8:ffff::1"),
+                false,
+            ),
+            ("IpAddress", "aws:SourceIp", "2001:db8::/32", Some("2001:db8:ffff::1"), true),
+        ] {
+            let mut document = serde_json::json!({
+                "Version": "2012-10-17", "Statement": [
+                    {"Effect": "Allow", "Action": "s3:PutObject", "Resource": "arn:aws:s3:::bucket/*"},
+                    {"Effect": "Deny", "Action": "s3:PutObject", "Resource": "arn:aws:s3:::bucket/*",
+                     "Condition": {operator: {key: policy_value}}}
+                ]
+            });
+            let policy =
+                Policy::parse_config(&serde_json::to_vec(&document).expect("serialize policy")).expect("parse identity policy");
+            let conditions = request_value
+                .map(|value| {
+                    HashMap::from([(key.split_once(':').expect("condition prefix").1.to_string(), vec![value.to_string()])])
+                })
+                .unwrap_or_default();
+            let claims = HashMap::new();
+            let args = Args {
+                account: "user",
+                groups: &None,
+                action: Action::S3Action(S3Action::PutObjectAction),
+                bucket: "bucket",
+                object: "object",
+                conditions: &conditions,
+                claims: &claims,
+                is_owner: false,
+                deny_only: false,
+            };
+            assert_eq!(policy.is_allowed(&args).await, !denied, "identity {operator} {key} {request_value:?}");
+            for statement in document["Statement"].as_array_mut().expect("statements array") {
+                statement["Principal"] = serde_json::json!("*");
+            }
+            let policy: BucketPolicy = serde_json::from_slice(&serde_json::to_vec(&document).expect("serialize bucket policy"))
+                .expect("parse bucket policy");
+            assert_eq!(
+                policy
+                    .is_allowed(&BucketPolicyArgs {
+                        account: "user",
+                        groups: &None,
+                        action: args.action,
+                        bucket: "bucket",
+                        object: "object",
+                        conditions: &conditions,
+                        is_owner: false,
+                    })
+                    .await,
+                !denied,
+                "bucket {operator} {key} {request_value:?}"
+            );
+        }
+    }
+
+    #[tokio::test]
+    async fn service_principal_never_grants_access_to_a_named_access_key() {
+        for principal in [
+            serde_json::json!({"Service": "logging.s3.amazonaws.com"}),
+            serde_json::json!({"Service": "*"}),
+            serde_json::json!({"Service": "logging.s3.amazonaws.com", "AWS": "authorized-user"}),
+        ] {
+            let document = serde_json::json!({
+                "Version": "2012-10-17", "Statement": [{"Effect": "Allow", "Principal": principal,
+                "Action": ["s3:GetObject", "s3:PutObject"], "Resource": "arn:aws:s3:::bucket/*"}]
+            });
+            let policy: BucketPolicy = serde_json::from_slice(&serde_json::to_vec(&document).unwrap()).unwrap();
+            for action in [S3Action::GetObjectAction, S3Action::PutObjectAction] {
+                for account in ["logging.s3.amazonaws.com", "authorized-user"] {
+                    let allowed = policy
+                        .is_allowed(&BucketPolicyArgs {
+                            account,
+                            groups: &None,
+                            action: Action::S3Action(action),
+                            bucket: "bucket",
+                            object: "log",
+                            conditions: &HashMap::new(),
+                            is_owner: false,
+                        })
+                        .await;
+                    assert_eq!(allowed, account == "authorized-user" && principal.get("AWS").is_some());
+                }
+            }
+            assert_eq!(serde_json::to_value(&policy).unwrap()["Statement"][0]["Principal"], principal);
+        }
+    }
+
+    #[tokio::test]
+    async fn directory_objects_obey_prefix_denies_in_identity_and_bucket_policies() {
+        let mut document = serde_json::json!({
+            "Version": "2012-10-17", "Statement": [
+                {"Effect": "Allow", "Action": "s3:*", "Resource": "arn:aws:s3:::bucket/*"},
+                {"Effect": "Deny", "Action": "s3:*", "Resource": "arn:aws:s3:::bucket/private/*"}
+            ]
+        });
+        let identity = Policy::parse_config(&serde_json::to_vec(&document).unwrap()).unwrap();
+        for statement in document["Statement"].as_array_mut().unwrap() {
+            statement["Principal"] = serde_json::json!("*");
+        }
+        let bucket: BucketPolicy = serde_json::from_slice(&serde_json::to_vec(&document).unwrap()).unwrap();
+        for action in [
+            S3Action::GetObjectAction,
+            S3Action::PutObjectAction,
+            S3Action::DeleteObjectAction,
+            S3Action::DeleteObjectVersionAction,
+        ] {
+            for (object, allowed) in [
+                ("private/", false),
+                ("private/child", false),
+                ("public/", true),
+                ("private", true),
+            ] {
+                let conditions = HashMap::new();
+                assert_eq!(
+                    identity
+                        .is_allowed(&Args {
+                            account: "user",
+                            groups: &None,
+                            action: Action::S3Action(action),
+                            bucket: "bucket",
+                            object,
+                            conditions: &conditions,
+                            claims: &HashMap::new(),
+                            is_owner: false,
+                            deny_only: false,
+                        })
+                        .await,
+                    allowed,
+                    "identity {action:?} {object}"
+                );
+                for account in ["user", ""] {
+                    assert_eq!(
+                        bucket
+                            .is_allowed(&BucketPolicyArgs {
+                                account,
+                                groups: &None,
+                                action: Action::S3Action(action),
+                                bucket: "bucket",
+                                object,
+                                conditions: &conditions,
+                                is_owner: false,
+                            })
+                            .await,
+                        allowed,
+                        "bucket {account} {action:?} {object}"
+                    );
+                }
+            }
+        }
+    }
+
     #[tokio::test]
     async fn force_delete_actions_require_an_explicit_grant() {
         let wildcard = Policy::parse_config(

crates/policy/src/policy/principal.rs

@@ -19,7 +19,7 @@ use std::collections::HashSet;
 
 /// Principal that serializes AWS field as single string when containing only "*",
 /// or as an array otherwise (matching AWS S3 API format).
-/// Also supports Service principals (e.g., "logging.s3.amazonaws.com").
+/// Service principals round-trip for compatibility but do not match access keys.
 #[derive(Debug, Clone, Default, PartialEq, Eq)]
 pub struct Principal {
     aws: HashSet<String>,
@@ -131,11 +131,6 @@ impl Principal {
                 return true;
             }
         }
-        for pattern in self.service.iter() {
-            if wildcard::is_simple_match(pattern, principal) {
-                return true;
-            }
-        }
         false
     }
 }
@@ -225,7 +220,7 @@ mod test {
             service: HashSet::from(["logging.s3.amazonaws.com".to_string()]),
         };
 
-        assert!(principal.is_match("logging.s3.amazonaws.com"));
+        assert!(!principal.is_match("logging.s3.amazonaws.com"));
         assert!(!principal.is_match("replication.s3.amazonaws.com"));
         assert!(!principal.is_match("arn:aws:iam::123456789012:root"));
     }
@@ -238,7 +233,7 @@ mod test {
         };
 
         assert!(principal.is_match("arn:aws:iam::123456789012:root"));
-        assert!(principal.is_match("logging.s3.amazonaws.com"));
+        assert!(!principal.is_match("logging.s3.amazonaws.com"));
         assert!(!principal.is_match("other-principal"));
     }
 }

crates/policy/src/policy/resource.rs

@@ -17,7 +17,11 @@ use serde::{
     Deserialize, Deserializer, Serialize,
     de::{self, Error as DeError, Visitor},
 };
-use std::{collections::HashMap, fmt, ops::Deref};
+use std::{
+    collections::{HashMap, HashSet},
+    fmt,
+    ops::Deref,
+};
 
 use super::{
     Error as IamError, Validator,
@@ -50,12 +54,6 @@ impl ResourceSet {
         self.0.is_empty()
     }
 
-    fn push_unique(&mut self, resource: Resource) {
-        if !self.0.contains(&resource) {
-            self.0.push(resource);
-        }
-    }
-
     pub async fn is_match(&self, resource: &str, conditions: &HashMap<String, Vec<String>>) -> bool {
         self.is_match_with_resolver(resource, conditions, None).await
     }
@@ -130,9 +128,7 @@ impl<'de> Deserialize<'de> for ResourceSet {
                 E: de::Error,
             {
                 let resource = Resource::try_from(value).map_err(|e| E::custom(format!("invalid resource: {}", e)))?;
-                let mut resources = ResourceSet::default();
-                resources.push_unique(resource);
-                Ok(resources)
+                Ok(ResourceSet(vec![resource]))
             }
 
             fn visit_seq<A>(self, mut seq: A) -> std::result::Result<Self::Value, A::Error>
@@ -140,11 +136,14 @@ impl<'de> Deserialize<'de> for ResourceSet {
                 A: de::SeqAccess<'de>,
                 A::Error: DeError,
             {
-                let mut resources = ResourceSet(Vec::with_capacity(seq.size_hint().unwrap_or(0)));
+                let mut resources = ResourceSet::default();
+                let mut seen = HashSet::new();
                 while let Some(value) = seq.next_element::<String>()? {
                     match Resource::try_from(value.as_str()) {
                         Ok(resource) => {
-                            resources.push_unique(resource);
+                            if seen.insert(resource.clone()) {
+                                resources.0.push(resource);
+                            }
                         }
                         Err(e) => {
                             return Err(A::Error::custom(format!("invalid resource: {}", e)));
@@ -224,7 +223,14 @@ impl Resource {
                 return true;
             }
 
-            if cp != "." && wildcard::is_match(resolved_pattern, &cp) {
+            // Keep directory objects distinct from their bare key for prefix
+            // matching, while retaining path cleaning and exact-match compatibility.
+            let wildcard_resource = if matches!(self, Resource::S3(_)) && resource.ends_with('/') && cp.contains('/') {
+                format!("{cp}/")
+            } else {
+                cp.clone()
+            };
+            if cp != "." && wildcard::is_match(resolved_pattern, &wildcard_resource) {
                 return true;
             }
         }
@@ -314,6 +320,20 @@ mod tests {
     use std::collections::HashMap;
     use test_case::test_case;
 
+    #[test]
+    fn large_resource_sets_deduplicate_without_changing_first_seen_order() {
+        let unique = (0..40_000)
+            .map(|index| format!("arn:aws:s3:::bucket/key-{index}"))
+            .collect::<Vec<_>>();
+        let mut repeated = unique.clone();
+        repeated.extend(unique.iter().rev().cloned());
+        let resources: super::ResourceSet = serde_json::from_slice(&serde_json::to_vec(&repeated).expect("encode resources"))
+            .expect("large valid resource list parses");
+        assert_eq!(resources.len(), unique.len());
+        assert_eq!(serde_json::to_value(resources).expect("serialize resources"), serde_json::json!(unique));
+        assert!(serde_json::from_str::<super::ResourceSet>(r#"["arn:aws:s3:::bucket/key","invalid"]"#).is_err());
+    }
+
     #[test_case("*", "mybucket" => true; "all buckets")]
     #[test_case("*", "mybucket/myobject" => true; "all objects")]
     #[test_case("*", "key/mykey" => true; "all keys")]
@@ -335,6 +355,10 @@ mod tests {
     #[test_case("arn:aws:s3:::mybucket","mybucket/myobject" => false; "15")]
     #[test_case("arn:aws:s3:::attacker-bucket/*","attacker-bucket/../victim-bucket/evil.txt" => false; "16")]
     #[test_case("arn:aws:s3:::attacker-bucket/*","attacker-bucket/safe/../../victim-bucket/evil.txt" => false; "17")]
+    #[test_case("arn:aws:s3:::bucket/secret/*","bucket/secret/" => true; "directory prefix")]
+    #[test_case("arn:aws:s3:::bucket/secret","bucket/secret/" => true; "directory exact compatibility")]
+    #[test_case("arn:aws:s3:::bucket/*","bucket/" => false; "bucket is not a directory object")]
+    #[test_case("arn:aws:s3:::attacker-bucket/*","attacker-bucket/../victim-bucket/" => false; "directory traversal")]
     #[test_case("arn:aws:kms:::key/mykey","key/mykey" => true; "kms exact key")]
     #[test_case("arn:aws:kms:::key/mykey","key/otherkey" => false; "kms wrong key")]
     #[test_case("arn:aws:kms:::key/*","key/mykey" => true; "kms key wildcard")]

crates/policy/src/policy/statement.rs

@@ -144,7 +144,7 @@ impl Statement {
 
         for action in self.actions.iter() {
             match action {
-                Action::S3Action(_) => saw_s3 = true,
+                Action::All | Action::S3Action(_) => saw_s3 = true,
                 Action::AdminAction(_) => saw_admin = true,
                 Action::StsAction(_) => saw_sts = true,
                 Action::KmsAction(_) => saw_kms = true,

crates/policy/src/policy/variables.rs

@@ -259,35 +259,11 @@ async fn resolve_aws_variables_with_depth(pattern: &str, resolver: &dyn PolicyVa
     if depth >= 10 {
         return vec![pattern.to_string()];
     }
-    let mut results = vec![pattern.to_string()];
-
-    let mut changed = true;
-    let max_iterations = 10; // Prevent infinite loops
-    let mut iteration = 0;
-
-    while changed && iteration < max_iterations {
-        changed = false;
-        iteration += 1;
-
-        let mut new_results = Vec::new();
-        for result in &results {
-            let resolved = resolve_single_pass(result, resolver, depth).await;
-            if resolved.len() > 1 || (resolved.len() == 1 && &resolved[0] != result) {
-                changed = true;
-            }
-            new_results.extend(resolved);
-        }
-
-        // Remove duplicates while preserving order
-        results.clear();
-        let mut seen = std::collections::HashSet::new();
-        for result in new_results {
-            if seen.insert(result.clone()) {
-                results.push(result);
-            }
-        }
-    }
-
+    // Values from credentials and claims are data, not policy source. Expanding
+    // substituted placeholders again allows attacker-controlled exponential growth.
+    let mut results = resolve_single_pass(pattern, resolver, depth).await;
+    let mut seen = std::collections::HashSet::new();
+    results.retain(|result| seen.insert(result.clone()));
     results
 }
 
@@ -370,8 +346,8 @@ async fn resolve_single_pass(pattern: &str, resolver: &dyn PolicyVariableResolve
                             let suffix = &results[i].0[end_pos + 1..];
 
                             for value in values {
-                                // Resume after the substitution; a generated placeholder
-                                // belongs to the next bounded pass, not this one.
+                                // Resume after the substitution; generated placeholders
+                                // are literal data and must never be expanded.
                                 new_results.push((format!("{prefix}{value}{suffix}"), actual_pos + value.len()));
                             }
 
@@ -445,8 +421,20 @@ mod tests {
             calls: AtomicUsize::new(0),
         };
         let result = resolve_aws_variables("${aws:AccountId}{aws:username}", &resolver).await;
-        assert!(resolver.calls.load(Ordering::Relaxed) <= 10);
-        assert_eq!(result, vec!["${aws:AccountId}{aws:username}".to_string()]);
+        assert_eq!(resolver.calls.load(Ordering::Relaxed), 1);
+        assert_eq!(result, vec!["$${aws:username}".to_string()]);
+    }
+
+    #[tokio::test]
+    async fn substituted_account_id_placeholders_are_literal() {
+        let account = "${aws:AccountId}".repeat(2);
+        let resolver = VariableResolver::new(VariableContext {
+            account_id: Some(account.clone()),
+            username: Some("alice".to_string()),
+            ..Default::default()
+        });
+        let result = resolve_aws_variables("prefix/${aws:AccountId}/${aws:username}", &resolver).await;
+        assert_eq!(result, vec![format!("prefix/{account}/alice")]);
     }
 
     #[tokio::test]

crates/protocols/Cargo.toml

@@ -183,6 +183,7 @@ subtle = { workspace = true, optional = true }
 socket2 = { workspace = true, optional = true, features = ["all"] }
 
 [dev-dependencies]
+rcgen = { workspace = true }
 tempfile = { workspace = true }
 proptest = "1"
 rustfs-test-utils = { workspace = true }

crates/protocols/src/common/mod.rs

@@ -16,6 +16,9 @@ pub mod client;
 pub mod gateway;
 pub mod session;
 
+#[cfg(any(feature = "webdav", feature = "ftps"))]
+pub(crate) mod tls;
+
 #[cfg(test)]
 pub(crate) mod dummy_storage;
 

crates/protocols/src/common/tls.rs

@@ -0,0 +1,183 @@
+// Copyright 2026 RustFS Team
+// SPDX-License-Identifier: Apache-2.0
+
+use rustls::{ServerConfig, server::ResolvesServerCert};
+use std::{io, path::Path, sync::Arc};
+
+/// Shared listener TLS policy: a configured client CA always requires client authentication.
+pub(crate) fn server_config(resolver: Arc<dyn ResolvesServerCert>, ca_file: Option<&str>) -> io::Result<ServerConfig> {
+    let _ = rustls::crypto::aws_lc_rs::default_provider().install_default();
+    let builder = ServerConfig::builder();
+    let builder = if let Some(ca_file) = ca_file {
+        builder.with_client_cert_verifier(rustfs_tls_runtime::build_webpki_client_verifier_from_ca_file(Path::new(ca_file))?)
+    } else {
+        builder.with_no_client_auth()
+    };
+    Ok(builder.with_cert_resolver(resolver))
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+    use rcgen::{BasicConstraints, CertificateParams, CertifiedIssuer, ExtendedKeyUsagePurpose, IsCa, KeyPair, KeyUsagePurpose};
+    use rustls::{ClientConfig, ClientConnection, RootCertStore, ServerConnection, pki_types::PrivatePkcs8KeyDer};
+
+    fn authority(name: &str) -> CertifiedIssuer<'static, KeyPair> {
+        let mut params = CertificateParams::new(vec![name.to_owned()]).expect("CA names");
+        params.is_ca = IsCa::Ca(BasicConstraints::Unconstrained);
+        params.key_usages = vec![KeyUsagePurpose::KeyCertSign, KeyUsagePurpose::CrlSign];
+        CertifiedIssuer::self_signed(params, KeyPair::generate().expect("CA signing key")).expect("self-signed CA")
+    }
+
+    fn handshake(server: ServerConfig, client: ClientConfig) -> io::Result<()> {
+        let mut server = ServerConnection::new(Arc::new(server)).map_err(io::Error::other)?;
+        let mut client =
+            ClientConnection::new(Arc::new(client), "localhost".try_into().expect("server name")).map_err(io::Error::other)?;
+        for _ in 0..16 {
+            let mut wire = Vec::new();
+            client.write_tls(&mut wire)?;
+            if !wire.is_empty() {
+                server.read_tls(&mut std::io::Cursor::new(wire))?;
+                server.process_new_packets().map_err(io::Error::other)?;
+            }
+            let mut wire = Vec::new();
+            server.write_tls(&mut wire)?;
+            if !wire.is_empty() {
+                client.read_tls(&mut std::io::Cursor::new(wire))?;
+                client.process_new_packets().map_err(io::Error::other)?;
+            }
+            if !server.is_handshaking() && !client.is_handshaking() {
+                return Ok(());
+            }
+        }
+        Err(io::Error::other("TLS handshake did not finish"))
+    }
+
+    #[test]
+    fn protocol_listener_tls_requires_the_configured_client_ca() {
+        let _ = rustls::crypto::aws_lc_rs::default_provider().install_default();
+        let directory = tempfile::tempdir().expect("TLS directory");
+        let trusted = authority("trusted-ca");
+        let unrelated = authority("unrelated-ca");
+        let server_key = KeyPair::generate().expect("server key");
+        let mut params = CertificateParams::new(vec!["localhost".to_owned()]).expect("server names");
+        params.extended_key_usages = vec![ExtendedKeyUsagePurpose::ServerAuth];
+        let server_cert = params.signed_by(&server_key, &trusted).expect("server certificate");
+        std::fs::write(directory.path().join("rustfs_cert.pem"), server_cert.pem()).expect("write server certificate");
+        std::fs::write(directory.path().join("rustfs_key.pem"), server_key.serialize_pem()).expect("write server key");
+        let ca_file = directory.path().join("clients.pem");
+        std::fs::write(&ca_file, trusted.pem()).expect("write client CA");
+        let resolver =
+            rustfs_tls_runtime::ReloadableServerCertResolver::load_from_directory(directory.path().to_str().expect("directory"))
+                .expect("production certificate resolver");
+        for (client_issuer, allowed) in [(None, false), (Some(&unrelated), false), (Some(&trusted), true)] {
+            let mut roots = RootCertStore::empty();
+            roots.add(trusted.der().clone()).expect("trust server CA");
+            let builder = ClientConfig::builder().with_root_certificates(roots);
+            let client = if let Some(issuer) = client_issuer {
+                let key = KeyPair::generate().expect("client key");
+                let mut params = CertificateParams::new(Vec::new()).expect("client names");
+                params.extended_key_usages = vec![ExtendedKeyUsagePurpose::ClientAuth];
+                let cert = params.signed_by(&key, issuer).expect("client certificate");
+                builder
+                    .with_client_auth_cert(vec![cert.der().clone()], PrivatePkcs8KeyDer::from(key.serialize_der()).into())
+                    .expect("client identity")
+            } else {
+                builder.with_no_client_auth()
+            };
+            let server = server_config(resolver.clone(), ca_file.to_str()).expect("production listener TLS policy");
+            assert_eq!(handshake(server, client).is_ok(), allowed);
+        }
+        let mut roots = RootCertStore::empty();
+        roots.add(trusted.der().clone()).expect("trust server CA");
+        handshake(
+            server_config(resolver.clone(), None).expect("ordinary TLS"),
+            ClientConfig::builder().with_root_certificates(roots).with_no_client_auth(),
+        )
+        .expect("no CA preserves ordinary TLS");
+        for bad in [
+            "",
+            "not a certificate",
+            "-----BEGIN CERTIFICATE-----\nAAAA\n-----END CERTIFICATE-----",
+        ] {
+            std::fs::write(&ca_file, bad).expect("bad CA fixture");
+            assert!(server_config(resolver.clone(), ca_file.to_str()).is_err(), "bad CA must fail startup");
+        }
+        std::fs::remove_file(&ca_file).expect("remove CA fixture");
+        assert!(server_config(resolver, ca_file.to_str()).is_err());
+    }
+
+    #[tokio::test]
+    async fn protocol_listeners_refuse_startup_with_an_invalid_client_ca() {
+        let directory = tempfile::tempdir().expect("TLS directory");
+        let certified = rcgen::generate_simple_self_signed(vec!["localhost".to_owned()]).expect("server identity");
+        std::fs::write(directory.path().join("rustfs_cert.pem"), certified.cert.pem()).expect("server certificate");
+        std::fs::write(directory.path().join("rustfs_key.pem"), certified.signing_key.serialize_pem()).expect("server key");
+        let ca_file = directory.path().join("clients.pem");
+        std::fs::write(&ca_file, "not a CA certificate").expect("invalid client CA fixture");
+        let (_shutdown, shutdown) = tokio::sync::broadcast::channel(1);
+        #[cfg(feature = "webdav")]
+        {
+            let config = crate::webdav::config::WebDavConfig {
+                bind_addr: "127.0.0.1:0".parse().expect("bind"),
+                tls_enabled: true,
+                cert_dir: Some(directory.path().to_string_lossy().into_owned()),
+                ca_file: Some(ca_file.to_string_lossy().into_owned()),
+                ..Default::default()
+            };
+            let server = crate::webdav::server::WebDavServer::new(config, crate::common::dummy_storage::DummyBackend::default())
+                .await
+                .expect("path checks");
+            let error = tokio::time::timeout(std::time::Duration::from_secs(5), server.start(shutdown.resubscribe()))
+                .await
+                .expect("startup deadline")
+                .expect_err("invalid CA must not serve WebDAV");
+            assert!(matches!(error, crate::webdav::config::WebDavInitError::Tls(_)));
+        }
+        #[cfg(feature = "ftps")]
+        {
+            let config = crate::ftps::config::FtpsConfig {
+                bind_addr: "127.0.0.1:0".parse().expect("bind"),
+                tls_enabled: true,
+                cert_dir: Some(directory.path().to_string_lossy().into_owned()),
+                ca_file: Some(ca_file.to_string_lossy().into_owned()),
+                ..Default::default()
+            };
+            let server = crate::ftps::server::FtpsServer::new(config, crate::common::dummy_storage::DummyBackend::default())
+                .await
+                .expect("path checks");
+            let error = tokio::time::timeout(std::time::Duration::from_secs(5), server.start(shutdown.resubscribe()))
+                .await
+                .expect("startup deadline")
+                .expect_err("invalid CA must not serve FTPS");
+            assert!(matches!(error, crate::ftps::config::FtpsInitError::InvalidConfig(_)));
+        }
+    }
+
+    #[tokio::test]
+    async fn protocol_client_ca_configuration_cannot_allow_cleartext() {
+        let directory = tempfile::tempdir().expect("config directory");
+        let ca_file = directory.path().join("clients.pem");
+        std::fs::write(&ca_file, "test").expect("existing CA path");
+        #[cfg(feature = "webdav")]
+        {
+            let config = crate::webdav::config::WebDavConfig {
+                tls_enabled: false,
+                cert_dir: Some(directory.path().to_string_lossy().into_owned()),
+                ca_file: Some(ca_file.to_string_lossy().into_owned()),
+                ..Default::default()
+            };
+            assert!(config.validate().await.is_err());
+        }
+        #[cfg(feature = "ftps")]
+        {
+            let config = crate::ftps::config::FtpsConfig {
+                tls_enabled: false,
+                cert_dir: Some(directory.path().to_string_lossy().into_owned()),
+                ca_file: Some(ca_file.to_string_lossy().into_owned()),
+                ..Default::default()
+            };
+            assert!(config.validate().await.is_err());
+        }
+    }
+}

crates/protocols/src/ftps/config.rs

@@ -56,6 +56,12 @@ impl FtpsConfig {
             ));
         }
 
+        if self.ca_file.is_some() && (!self.tls_enabled || self.cert_dir.is_none()) {
+            return Err(FtpsInitError::InvalidConfig(
+                "A client CA requires TLS and a server certificate directory".to_owned(),
+            ));
+        }
+
         if let Some(path) = &self.cert_dir
             && !tokio::fs::try_exists(path).await.unwrap_or(false)
         {

crates/protocols/src/ftps/server.rs

@@ -121,7 +121,7 @@ where
             Arc::new(ConnectionUserDetailProvider),
         )
         .authenticator(Arc::new(ConnectionAuthenticator {
-            require_data_tls: self.config.tls_enabled && self.config.ftps_required,
+            require_data_tls: self.config.tls_enabled && (self.config.ftps_required || self.config.ca_file.is_some()),
         }));
 
         // Configure passive ports for data connections
@@ -193,14 +193,12 @@ where
                     reload_shutdown_rx.clone(),
                 );
 
-                // Build ServerConfig with SNI support
-                let server_config = rustls::ServerConfig::builder()
-                    .with_no_client_auth()
-                    .with_cert_resolver(resolver);
+                let server_config = crate::common::tls::server_config(resolver, self.config.ca_file.as_deref())
+                    .map_err(|error| FtpsInitError::InvalidConfig(format!("Invalid client CA configuration: {error}")))?;
 
                 server_builder = server_builder.ftps_manual::<std::path::PathBuf>(Arc::new(server_config));
 
-                if self.config.ftps_required {
+                if self.config.ftps_required || self.config.ca_file.is_some() {
                     debug!(
                         event = EVENT_FTPS_TLS_STATE,
                         component = LOG_COMPONENT_PROTOCOLS,
@@ -210,7 +208,7 @@ where
                     );
                     server_builder = server_builder.ftps_required(FtpsRequired::All, FtpsRequired::All);
                 }
-            } else if self.config.ftps_required {
+            } else if self.config.ftps_required || self.config.ca_file.is_some() {
                 return Err(FtpsInitError::InvalidConfig(
                     "FTPS required but certificate directory not provided".into(),
                 ));

crates/protocols/src/webdav/config.rs

@@ -65,6 +65,12 @@ impl WebDavConfig {
             ));
         }
 
+        if self.ca_file.is_some() && (!self.tls_enabled || self.cert_dir.is_none()) {
+            return Err(WebDavInitError::InvalidConfig(
+                "A client CA requires TLS and a server certificate directory".to_owned(),
+            ));
+        }
+
         if let Some(path) = &self.cert_dir
             && !tokio::fs::try_exists(path).await.unwrap_or(false)
         {

crates/protocols/src/webdav/server.rs

@@ -30,7 +30,6 @@ use hyper_util::rt::{TokioIo, TokioTimer};
 use rustfs_config::{DEFAULT_TLS_RELOAD_ENABLE, DEFAULT_TLS_RELOAD_INTERVAL, ENV_TLS_RELOAD_ENABLE, ENV_TLS_RELOAD_INTERVAL};
 use rustfs_tls_runtime::{ReloadableServerCertResolver, TlsReloadOptions, spawn_server_cert_reload_loop};
 use rustfs_utils::MaskedAccessKey;
-use rustls::ServerConfig;
 use std::convert::Infallible;
 use std::io;
 use std::net::IpAddr;
@@ -159,7 +158,8 @@ where
                     reload_shutdown_rx.clone(),
                 );
 
-                let server_config = ServerConfig::builder().with_no_client_auth().with_cert_resolver(resolver);
+                let server_config = crate::common::tls::server_config(resolver, self.config.ca_file.as_deref())
+                    .map_err(|error| WebDavInitError::Tls(format!("Invalid client CA configuration: {error}")))?;
 
                 Some(TlsAcceptor::from(Arc::new(server_config)))
             } else {

crates/targets/src/manifest.rs

@@ -17,11 +17,11 @@ use std::collections::HashMap;
 use std::sync::{Mutex, OnceLock};
 
 use rustfs_config::{
-    AMQP_PASSWORD, AMQP_TLS_CLIENT_CERT, AMQP_TLS_CLIENT_KEY, KAFKA_SASL_PASSWORD, KAFKA_TLS_CLIENT_CERT, KAFKA_TLS_CLIENT_KEY,
-    MQTT_PASSWORD, MQTT_TLS_CLIENT_CERT, MQTT_TLS_CLIENT_KEY, MYSQL_DSN_STRING, MYSQL_TLS_CLIENT_CERT, MYSQL_TLS_CLIENT_KEY,
-    NATS_CREDENTIALS_FILE, NATS_PASSWORD, NATS_TLS_CLIENT_CERT, NATS_TLS_CLIENT_KEY, NATS_TOKEN, POSTGRES_DSN_STRING,
-    POSTGRES_TLS_CLIENT_CERT, POSTGRES_TLS_CLIENT_KEY, PULSAR_AUTH_TOKEN, PULSAR_PASSWORD, REDIS_PASSWORD, REDIS_TLS_CLIENT_CERT,
-    REDIS_TLS_CLIENT_KEY, WEBHOOK_AUTH_TOKEN, WEBHOOK_CLIENT_CERT, WEBHOOK_CLIENT_KEY,
+    AMQP_PASSWORD, AMQP_TLS_CLIENT_CERT, AMQP_TLS_CLIENT_KEY, AMQP_URL, KAFKA_SASL_PASSWORD, KAFKA_TLS_CLIENT_CERT,
+    KAFKA_TLS_CLIENT_KEY, MQTT_PASSWORD, MQTT_TLS_CLIENT_CERT, MQTT_TLS_CLIENT_KEY, MYSQL_DSN_STRING, MYSQL_TLS_CLIENT_CERT,
+    MYSQL_TLS_CLIENT_KEY, NATS_CREDENTIALS_FILE, NATS_PASSWORD, NATS_TLS_CLIENT_CERT, NATS_TLS_CLIENT_KEY, NATS_TOKEN,
+    POSTGRES_DSN_STRING, POSTGRES_TLS_CLIENT_CERT, POSTGRES_TLS_CLIENT_KEY, PULSAR_AUTH_TOKEN, PULSAR_PASSWORD, REDIS_PASSWORD,
+    REDIS_TLS_CLIENT_CERT, REDIS_TLS_CLIENT_KEY, REDIS_URL, WEBHOOK_AUTH_TOKEN, WEBHOOK_CLIENT_CERT, WEBHOOK_CLIENT_KEY,
 };
 
 /// Shared plugin manifest metadata for a target implementation.
@@ -36,6 +36,51 @@ pub struct TargetPluginManifest {
     pub secret_fields: &'static [&'static str],
 }
 
+impl TargetPluginManifest {
+    /// Configuration fields that bind saved credentials to a destination and its TLS trust policy.
+    /// Changing these fields requires callers to supply credentials again instead of reusing hidden values.
+    pub fn credential_binding_fields(&self) -> &'static [&'static str] {
+        use rustfs_config::*;
+        match self.target_type {
+            "webhook" => &[WEBHOOK_ENDPOINT, WEBHOOK_CLIENT_CA, WEBHOOK_SKIP_TLS_VERIFY],
+            "mqtt" => &[
+                MQTT_BROKER,
+                MQTT_USERNAME,
+                MQTT_TLS_POLICY,
+                MQTT_TLS_CA,
+                MQTT_TLS_TRUST_LEAF_AS_CA,
+            ],
+            "kafka" => &[
+                KAFKA_BROKERS,
+                KAFKA_TLS_ENABLE,
+                KAFKA_TLS_CA,
+                KAFKA_SASL_ENABLE,
+                KAFKA_SASL_MECHANISM,
+                KAFKA_SASL_USERNAME,
+            ],
+            "amqp" => &[AMQP_URL, AMQP_USERNAME, AMQP_TLS_CA],
+            "nats" => &[NATS_ADDRESS, NATS_USERNAME, NATS_TLS_CA, NATS_TLS_REQUIRED],
+            "pulsar" => &[
+                PULSAR_BROKER,
+                PULSAR_USERNAME,
+                PULSAR_TLS_CA,
+                PULSAR_TLS_ALLOW_INSECURE,
+                PULSAR_TLS_HOSTNAME_VERIFICATION,
+            ],
+            "mysql" => &[MYSQL_DSN_STRING, MYSQL_TLS_CA],
+            "redis" => &[
+                REDIS_URL,
+                REDIS_USERNAME,
+                REDIS_TLS_POLICY,
+                REDIS_TLS_CA,
+                REDIS_TLS_ALLOW_INSECURE,
+            ],
+            "postgres" => &[POSTGRES_DSN_STRING, POSTGRES_TLS_CA, POSTGRES_TLS_REQUIRED],
+            _ => &[],
+        }
+    }
+}
+
 /// Declares how a plugin is packaged relative to the RustFS process boundary.
 #[derive(Debug, Clone, Copy, PartialEq, Eq)]
 pub enum TargetPluginPackaging {
@@ -117,7 +162,7 @@ const NO_SECRET_FIELDS: &[&str] = &[];
 const WEBHOOK_SECRET_FIELDS: &[&str] = &[WEBHOOK_AUTH_TOKEN, WEBHOOK_CLIENT_CERT, WEBHOOK_CLIENT_KEY];
 const MQTT_SECRET_FIELDS: &[&str] = &[MQTT_PASSWORD, MQTT_TLS_CLIENT_CERT, MQTT_TLS_CLIENT_KEY];
 const KAFKA_SECRET_FIELDS: &[&str] = &[KAFKA_TLS_CLIENT_CERT, KAFKA_TLS_CLIENT_KEY, KAFKA_SASL_PASSWORD];
-const AMQP_SECRET_FIELDS: &[&str] = &[AMQP_PASSWORD, AMQP_TLS_CLIENT_CERT, AMQP_TLS_CLIENT_KEY];
+const AMQP_SECRET_FIELDS: &[&str] = &[AMQP_URL, AMQP_PASSWORD, AMQP_TLS_CLIENT_CERT, AMQP_TLS_CLIENT_KEY];
 const NATS_SECRET_FIELDS: &[&str] = &[
     NATS_PASSWORD,
     NATS_TOKEN,
@@ -127,7 +172,7 @@ const NATS_SECRET_FIELDS: &[&str] = &[
 ];
 const PULSAR_SECRET_FIELDS: &[&str] = &[PULSAR_AUTH_TOKEN, PULSAR_PASSWORD];
 const MYSQL_SECRET_FIELDS: &[&str] = &[MYSQL_DSN_STRING, MYSQL_TLS_CLIENT_CERT, MYSQL_TLS_CLIENT_KEY];
-const REDIS_SECRET_FIELDS: &[&str] = &[REDIS_PASSWORD, REDIS_TLS_CLIENT_CERT, REDIS_TLS_CLIENT_KEY];
+const REDIS_SECRET_FIELDS: &[&str] = &[REDIS_URL, REDIS_PASSWORD, REDIS_TLS_CLIENT_CERT, REDIS_TLS_CLIENT_KEY];
 const POSTGRES_SECRET_FIELDS: &[&str] = &[POSTGRES_DSN_STRING, POSTGRES_TLS_CLIENT_CERT, POSTGRES_TLS_CLIENT_KEY];
 
 #[inline]
@@ -243,8 +288,18 @@ mod tests {
         installable_target_marketplace_manifest,
     };
     use crate::domain::TargetDomain;
+    use rustfs_config::{AMQP_URL, REDIS_URL};
     use rustfs_config::{KAFKA_SASL_PASSWORD, WEBHOOK_AUTH_TOKEN, WEBHOOK_CLIENT_CERT, WEBHOOK_CLIENT_KEY};
 
+    #[test]
+    fn broker_urls_are_secret_and_credentials_are_bound_to_connection_settings() {
+        for (target, url) in [("amqp", AMQP_URL), ("redis", REDIS_URL)] {
+            let manifest = builtin_target_manifest(target);
+            assert!(manifest.secret_fields.contains(&url));
+            assert!(manifest.credential_binding_fields().contains(&url));
+        }
+    }
+
     #[test]
     fn builtin_webhook_manifest_marks_secret_fields() {
         let manifest = builtin_target_manifest("webhook");

crates/targets/src/store.rs

@@ -14,7 +14,10 @@
 
 use crate::error::StoreError;
 use rustfs_config::notify::{COMPRESS_EXT, DEFAULT_EXT};
-use rustfs_config::{DEFAULT_LIMIT, DEFAULT_TARGET_STORE_COMPRESS, ENV_TARGET_STORE_COMPRESS, EnableState};
+use rustfs_config::{
+    DEFAULT_LIMIT, DEFAULT_TARGET_STORE_COMPRESS, DEFAULT_TARGET_STORE_MAX_BYTES, ENV_TARGET_STORE_COMPRESS,
+    ENV_TARGET_STORE_MAX_BYTES, EnableState,
+};
 use serde::{Serialize, de::DeserializeOwned};
 use snap::raw::{Decoder, Encoder};
 use std::{
@@ -76,9 +79,8 @@ const LEGACY_BATCH_COUNT_SEPARATOR: char = ':';
 const FAILED_STORE_SUBDIR: &str = "failed";
 
 /// Maximum number of entries retained in the failed store per target, bounded independently of the
-/// live queue so an accumulation of terminal failures cannot starve live events. At the bound the
-/// oldest failed entry is dropped to admit the newer one, so a newer failure is never lost in favour
-/// of an older one.
+/// live queue. Both stores share the byte budget. At the count bound the oldest failed entry is
+/// dropped to admit the newer one, provided the byte budget can also admit it.
 const FAILED_STORE_MAX_ENTRIES: usize = 10_000;
 
 /// Maximum age of a failed-store entry before it is removed as expired, measured from the file write
@@ -361,10 +363,13 @@ pub trait FailedEventStore: Send + Sync {
 /// A store that uses the filesystem to persist events in a queue
 pub struct QueueStore<T> {
     entry_limit: u64,
+    byte_limit: u64,
+    stored_bytes: Arc<AtomicU64>,
+    failed_bytes: Arc<AtomicU64>,
     directory: PathBuf,
     file_ext: String,
     compress: bool,
-    entries: Arc<RwLock<HashMap<String, i64>>>, // key -> modtime as unix nano
+    entries: Arc<RwLock<HashMap<String, u64>>>, // key -> persisted file bytes
     pending_entries: Arc<AtomicU64>,
     /// Cached count of complete failed-store entries, shared across clones. Initialized by one
     /// directory read in open() and kept current by the failed-store write, the capacity trim, and
@@ -383,6 +388,9 @@ impl<T> Clone for QueueStore<T> {
     fn clone(&self) -> Self {
         QueueStore {
             entry_limit: self.entry_limit,
+            byte_limit: self.byte_limit,
+            stored_bytes: Arc::clone(&self.stored_bytes),
+            failed_bytes: Arc::clone(&self.failed_bytes),
             directory: self.directory.clone(),
             file_ext: self.file_ext.clone(),
             compress: self.compress,
@@ -406,6 +414,22 @@ impl Drop for EntryReservation<'_> {
     }
 }
 
+struct ByteReservation<'a> {
+    counter: &'a AtomicU64,
+    bytes: u64,
+    committed: bool,
+}
+
+impl Drop for ByteReservation<'_> {
+    fn drop(&mut self) {
+        if !self.committed {
+            let _ = self
+                .counter
+                .try_update(Ordering::SeqCst, Ordering::SeqCst, |used| Some(used.saturating_sub(self.bytes)));
+        }
+    }
+}
+
 impl<T: Serialize + DeserializeOwned + Send + Sync> QueueStore<T> {
     /// Creates a new QueueStore
     pub fn new(directory: impl Into<PathBuf>, limit: u64, ext: &str) -> Self {
@@ -420,6 +444,13 @@ impl<T: Serialize + DeserializeOwned + Send + Sync> QueueStore<T> {
         QueueStore {
             directory: directory.into(),
             entry_limit,
+            byte_limit: std::env::var(ENV_TARGET_STORE_MAX_BYTES)
+                .ok()
+                .and_then(|value| value.parse::<u64>().ok())
+                .filter(|value| *value > 0)
+                .unwrap_or(DEFAULT_TARGET_STORE_MAX_BYTES),
+            stored_bytes: Arc::new(AtomicU64::new(0)),
+            failed_bytes: Arc::new(AtomicU64::new(0)),
             file_ext: file_ext.to_string(),
             compress,
             entries: Arc::new(RwLock::new(HashMap::with_capacity((entry_limit as usize).min(MAX_PREALLOC_CAPACITY)))),
@@ -531,6 +562,34 @@ impl<T: Serialize + DeserializeOwned + Send + Sync> QueueStore<T> {
             .map_err(|e| StoreError::Compression(e.to_string()))
     }
 
+    fn reserve_bytes(&self, bytes: u64) -> Result<ByteReservation<'_>, StoreError> {
+        self.stored_bytes
+            .try_update(Ordering::SeqCst, Ordering::SeqCst, |used| {
+                used.checked_add(bytes).filter(|total| *total <= self.byte_limit)
+            })
+            .map_err(|_| StoreError::LimitExceeded)?;
+        Ok(ByteReservation {
+            counter: &self.stored_bytes,
+            bytes,
+            committed: false,
+        })
+    }
+
+    fn release_bytes(&self, bytes: u64) {
+        let _ = self
+            .stored_bytes
+            .try_update(Ordering::SeqCst, Ordering::SeqCst, |used| Some(used.saturating_sub(bytes)));
+    }
+
+    fn reconcile_failed_bytes(&self, bytes: u64) {
+        let previous = self.failed_bytes.swap(bytes, Ordering::SeqCst);
+        if bytes >= previous {
+            self.stored_bytes.fetch_add(bytes - previous, Ordering::SeqCst);
+        } else {
+            self.release_bytes(previous - bytes);
+        }
+    }
+
     fn reserve_entry_slot(&self) -> Result<EntryReservation<'_>, StoreError> {
         loop {
             let entries = self
@@ -565,7 +624,7 @@ impl<T: Serialize + DeserializeOwned + Send + Sync> QueueStore<T> {
     /// one — never a half-written payload. On a crash mid-write the temp file is
     /// left behind and cleaned up by `open()`, so an acknowledged event is never
     /// lost and no ghost/truncated entry is ever indexed.
-    fn write_file(&self, key: &Key, data: &[u8]) -> Result<i64, StoreError> {
+    fn write_file(&self, key: &Key, data: &[u8]) -> Result<u64, StoreError> {
         let path = self.file_path(key);
         let parent = path
             .parent()
@@ -582,6 +641,8 @@ impl<T: Serialize + DeserializeOwned + Send + Sync> QueueStore<T> {
             std::borrow::Cow::Borrowed(data)
         };
 
+        let size = payload.len() as u64;
+        let mut budget = self.reserve_bytes(size)?;
         let tmp_path = {
             let mut file_name = key.to_key_string();
             file_name.push_str(TMP_SUFFIX);
@@ -589,13 +650,17 @@ impl<T: Serialize + DeserializeOwned + Send + Sync> QueueStore<T> {
         };
 
         // One implementation of the write-fsync-rename invariant, shared with the failed-store write.
-        // The temp file is removed on any error path so a failed write leaves no residue.
-        write_temp_then_rename(&tmp_path, &path, &payload)?;
+        // If cleanup itself fails, retain the reservation until reopen removes the orphan.
+        if let Err(error) = write_temp_then_rename(&tmp_path, &path, &payload) {
+            budget.committed =
+                !matches!(std::fs::metadata(&tmp_path), Err(error) if error.kind() == std::io::ErrorKind::NotFound);
+            return Err(error);
+        }
+        budget.committed = true;
 
         // Best-effort: persist the new directory entry created by rename.
         Self::fsync_dir(&self.directory);
 
-        let modified = SystemTime::now().duration_since(UNIX_EPOCH).unwrap_or_default().as_nanos() as i64;
         debug!(
             event = EVENT_TARGET_STORE_STATE,
             component = LOG_COMPONENT_TARGETS,
@@ -604,15 +669,15 @@ impl<T: Serialize + DeserializeOwned + Send + Sync> QueueStore<T> {
             key = %key,
             "target store state"
         );
-        Ok(modified)
+        Ok(size)
     }
 
-    fn insert_entry(&self, key: &Key, modified: i64) -> Result<(), StoreError> {
+    fn insert_entry(&self, key: &Key, size: u64) -> Result<(), StoreError> {
         let mut entries = self
             .entries
             .write()
             .map_err(|_| StoreError::Internal("Failed to acquire write lock on entries".to_string()))?;
-        entries.insert(key.to_key_string(), modified);
+        entries.insert(key.to_key_string(), size);
         Ok(())
     }
 
@@ -626,11 +691,12 @@ impl<T: Serialize + DeserializeOwned + Send + Sync> QueueStore<T> {
     }
 
     fn write_and_index(&self, key: &Key, data: &[u8]) -> Result<(), StoreError> {
-        let modified = self.write_file(key, data)?;
-        if let Err(err) = self.insert_entry(key, modified) {
+        let size = self.write_file(key, data)?;
+        if let Err(err) = self.insert_entry(key, size) {
             self.remove_file_if_present(key).map_err(|cleanup_err| {
                 StoreError::Internal(format!("Failed to index store entry {key}: {err}; cleanup failed: {cleanup_err}"))
             })?;
+            self.release_bytes(size);
             return Err(err);
         }
         Ok(())
@@ -644,23 +710,29 @@ impl<T: Serialize + DeserializeOwned + Send + Sync> QueueStore<T> {
     /// Reads the failed directory once and counts complete entries, skipping non-file entries and
     /// residual temp files from an interrupted write. Off the hot path, so it seeds the cached count
     /// at open and reconciles it on the maintenance scan rather than serving failed_len.
-    fn count_failed_entries_on_disk(&self) -> u64 {
+    fn failed_usage_on_disk(&self) -> Result<(u64, u64), StoreError> {
         let read_dir = match std::fs::read_dir(self.failed_dir()) {
             Ok(read_dir) => read_dir,
-            Err(_) => return 0,
+            Err(error) if error.kind() == std::io::ErrorKind::NotFound => return Ok((0, 0)),
+            Err(error) => return Err(StoreError::Io(error)),
         };
-
-        let mut count = 0u64;
-        for entry in read_dir.flatten() {
-            if entry.file_name().to_string_lossy().ends_with(TMP_SUFFIX) {
+        let (mut count, mut bytes) = (0_u64, 0_u64);
+        for entry in read_dir {
+            let entry = entry.map_err(StoreError::Io)?;
+            let Some(metadata) = Self::failed_scan_entry_metadata(entry.metadata())? else {
+                continue;
+            };
+            if !metadata.is_file() {
                 continue;
             }
-            // file_type reads the directory entry kind without a full stat where the platform records it.
-            if matches!(entry.file_type(), Ok(file_type) if file_type.is_file()) {
-                count += 1;
+            if entry.file_name().to_string_lossy().ends_with(TMP_SUFFIX) {
+                std::fs::remove_file(entry.path()).map_err(StoreError::Io)?;
+                continue;
             }
+            count += 1;
+            bytes = bytes.checked_add(metadata.len()).ok_or(StoreError::LimitExceeded)?;
         }
-        count
+        Ok((count, bytes))
     }
 
     /// Lowers the cached failed count by one removed entry, clamped at zero so a decrement can never
@@ -735,9 +807,13 @@ impl<T: Serialize + DeserializeOwned + Send + Sync> QueueStore<T> {
                 .file_name()
                 .map(|name| name.to_string_lossy().to_string())
                 .unwrap_or_default();
+            let size = std::fs::metadata(path).map(|metadata| metadata.len()).unwrap_or_default();
             match std::fs::remove_file(path) {
                 Ok(()) => {
                     self.decrement_failed_count();
+                    let remaining = self.failed_bytes.load(Ordering::SeqCst).saturating_sub(size);
+                    self.failed_bytes.store(remaining, Ordering::SeqCst);
+                    self.release_bytes(size);
                     warn!(
                         event = EVENT_TARGET_STORE_STATE,
                         component = LOG_COMPONENT_TARGETS,
@@ -777,6 +853,7 @@ where
             .write()
             .map_err(|_| StoreError::Internal("Failed to acquire write lock on entries".to_string()))?;
         self.pending_entries.store(0, Ordering::SeqCst);
+        let mut live_bytes = 0_u64;
         entries_map.clear();
         for entry in dir_entries {
             let entry = entry.map_err(|error| io_error_with_path(&self.directory, error))?;
@@ -791,7 +868,9 @@ where
             // Remove leftover temp files from an interrupted atomic write; they
             // are never valid committed entries.
             if file_name.ends_with(TMP_SUFFIX) {
-                let _ = std::fs::remove_file(entry.path());
+                if std::fs::remove_file(entry.path()).is_err() {
+                    live_bytes = live_bytes.checked_add(metadata.len()).ok_or(StoreError::LimitExceeded)?;
+                }
                 continue;
             }
 
@@ -809,14 +888,17 @@ where
                 continue;
             }
 
-            let modified = metadata.modified().map_err(|error| io_error_with_path(&entry_path, error))?;
-            let unix_nano = modified.duration_since(UNIX_EPOCH).unwrap_or_default().as_nanos() as i64;
-            entries_map.insert(file_name, unix_nano);
+            live_bytes = live_bytes.checked_add(metadata.len()).ok_or(StoreError::LimitExceeded)?;
+            entries_map.insert(file_name, metadata.len());
         }
 
         // Seed the cached failed count from one read of the failed directory, under the write guard
         // already held, so failed_len() serves an atomic load rather than a directory scan.
-        self.failed_count.store(self.count_failed_entries_on_disk(), Ordering::SeqCst);
+        let (failed_count, failed_bytes) = self.failed_usage_on_disk()?;
+        self.failed_count.store(failed_count, Ordering::SeqCst);
+        self.failed_bytes.store(failed_bytes, Ordering::SeqCst);
+        self.stored_bytes
+            .store(live_bytes.checked_add(failed_bytes).ok_or(StoreError::LimitExceeded)?, Ordering::SeqCst);
 
         debug!(
             event = EVENT_TARGET_STORE_STATE,
@@ -1005,12 +1087,14 @@ where
             .map_err(|_| StoreError::Internal("Failed to acquire write lock on entries".to_string()))?;
 
         // A batch entry indexed under the legacy separator is removed under both renderings so its slot is reclaimed.
-        let removed_current = entries.remove(&key.to_key_string()).is_some();
-        let removed_legacy = key.item_count > 1
-            && entries
-                .remove(&key.to_key_string_with(LEGACY_BATCH_COUNT_SEPARATOR))
-                .is_some();
-        if !removed_current && !removed_legacy {
+        let removed_current = entries.remove(&key.to_key_string());
+        let removed_legacy = if key.item_count > 1 {
+            entries.remove(&key.to_key_string_with(LEGACY_BATCH_COUNT_SEPARATOR))
+        } else {
+            None
+        };
+        self.release_bytes(removed_current.unwrap_or_default() + removed_legacy.unwrap_or_default());
+        if removed_current.is_none() && removed_legacy.is_none() {
             debug!(
                 event = EVENT_TARGET_STORE_STATE,
                 component = LOG_COMPONENT_TARGETS,
@@ -1042,15 +1126,18 @@ where
             .entries
             .write()
             .map_err(|_| StoreError::Internal("Failed to acquire write lock on entries".to_string()))?;
+        match std::fs::remove_dir_all(&self.directory) {
+            Ok(()) => {}
+            Err(err) if err.kind() == std::io::ErrorKind::NotFound => {}
+            Err(err) => return Err(StoreError::Io(err)),
+        }
         entries.clear();
         self.pending_entries.store(0, Ordering::SeqCst);
         self.failed_count.store(0, Ordering::SeqCst);
+        self.failed_bytes.store(0, Ordering::SeqCst);
+        self.stored_bytes.store(0, Ordering::SeqCst);
 
-        match std::fs::remove_dir_all(&self.directory) {
-            Ok(()) => Ok(()),
-            Err(err) if err.kind() == std::io::ErrorKind::NotFound => Ok(()),
-            Err(err) => Err(StoreError::Io(err)),
-        }
+        Ok(())
     }
 
     fn list(&self) -> Vec<Self::Key> {
@@ -1144,7 +1231,13 @@ where
 
         let entry_id = entry_name.to_string();
         let final_path = failed_dir.join(&entry_id);
-        let is_new_entry = !final_path.exists();
+        let old_size = match std::fs::metadata(&final_path) {
+            Ok(metadata) => Some(metadata.len()),
+            Err(error) if error.kind() == std::io::ErrorKind::NotFound => None,
+            Err(error) => return Err(StoreError::Io(error)),
+        };
+        let is_new_entry = old_size.is_none();
+        let mut budget = self.reserve_bytes(data.len() as u64)?;
 
         // A repeated move of the same live entry lands on the same filename and replaces the earlier
         // file. The count bound is enforced only when the write creates a new entry, since an
@@ -1160,7 +1253,22 @@ where
         let temp_path = failed_dir.join(format!("{entry_id}.{}{}", Uuid::new_v4(), TMP_SUFFIX));
         // rename replaces an existing destination file on the supported platforms, which the
         // idempotent overwrite relies on.
-        write_temp_then_rename(&temp_path, &final_path, data)?;
+        if let Err(error) = write_temp_then_rename(&temp_path, &final_path, data) {
+            budget.committed =
+                !matches!(std::fs::metadata(&temp_path), Err(error) if error.kind() == std::io::ErrorKind::NotFound);
+            if budget.committed {
+                self.failed_bytes.fetch_add(data.len() as u64, Ordering::SeqCst);
+            }
+            return Err(error);
+        }
+        budget.committed = true;
+        self.release_bytes(old_size.unwrap_or_default());
+        let failed_bytes = self
+            .failed_bytes
+            .load(Ordering::SeqCst)
+            .saturating_sub(old_size.unwrap_or_default())
+            .saturating_add(data.len() as u64);
+        self.failed_bytes.store(failed_bytes, Ordering::SeqCst);
 
         // A new failed file grows the cached count. An overwrite of an existing failed file leaves it
         // unchanged.
@@ -1232,6 +1340,8 @@ where
         // made outside the store drifts the value by at most one maintenance interval and no concurrent
         // write is overwritten.
         self.failed_count.store((materialized_len - pruned) as u64, Ordering::SeqCst);
+        let (_, bytes) = self.failed_usage_on_disk()?;
+        self.reconcile_failed_bytes(bytes);
         Ok(pruned)
     }
 
@@ -1259,6 +1369,107 @@ mod tests {
         std::env::temp_dir().join(format!("rustfs-targets-{name}-{}", Uuid::new_v4()))
     }
 
+    fn byte_limited_store(directory: &Path, limit: u64, compress: bool) -> QueueStore<serde_json::Value> {
+        let mut store = QueueStore::new_with_compression(directory, 100, ".event", compress);
+        store.byte_limit = limit;
+        store.open().expect("queue open");
+        store
+    }
+
+    #[test]
+    fn queue_byte_budget_is_shared_by_concurrent_clones() {
+        let directory = temp_store_dir("byte-concurrency");
+        let store = byte_limited_store(&directory, 96, false);
+        let barrier = Arc::new(Barrier::new(8));
+        let writers: Vec<_> = (0..8)
+            .map(|_| {
+                let store = store.clone();
+                let barrier = barrier.clone();
+                thread::spawn(move || {
+                    barrier.wait();
+                    store.put_raw(&[7; 32])
+                })
+            })
+            .collect();
+        let mut keys = Vec::new();
+        for writer in writers {
+            match writer.join().expect("writer") {
+                Ok(key) => keys.push(key),
+                Err(StoreError::LimitExceeded) => {}
+                Err(error) => panic!("unexpected write failure: {error}"),
+            }
+        }
+        assert_eq!(keys.len(), 3);
+        assert_eq!(store.stored_bytes.load(Ordering::SeqCst), 96);
+        for key in keys {
+            store.del(&key).expect("release queue bytes");
+        }
+        assert_eq!(store.stored_bytes.load(Ordering::SeqCst), 0);
+        assert!(store.put_raw(&[8; 96]).is_ok());
+        store.delete().expect("cleanup");
+    }
+
+    #[test]
+    fn queue_byte_budget_survives_restart_and_preserves_over_limit_entries() {
+        let directory = temp_store_dir("byte-restart");
+        let store = byte_limited_store(&directory, 128, false);
+        let key = store.put_raw(&[1; 40]).expect("live event");
+        store.put_failed_raw("failed-event", &[2; 30]).expect("failed event");
+        drop(store);
+        let store = byte_limited_store(&directory, 50, false);
+        assert_eq!(store.stored_bytes.load(Ordering::SeqCst), 70);
+        assert_eq!(store.get_raw(&key).unwrap(), [1; 40]);
+        assert_eq!(std::fs::read(store.failed_dir().join("failed-event")).unwrap(), [2; 30]);
+        assert!(matches!(store.put_raw(&[3]), Err(StoreError::LimitExceeded)));
+        store.del(&key).unwrap();
+        assert!(store.put_raw(&[4; 20]).is_ok());
+        assert!(matches!(store.put_failed_raw("another-event", &[5]), Err(StoreError::LimitExceeded)));
+        store.delete().unwrap();
+    }
+
+    #[test]
+    fn queue_byte_budget_counts_compressed_files_without_truncating_payloads() {
+        let directory = temp_store_dir("byte-compression");
+        let store = byte_limited_store(&directory, 128, true);
+        let payload = vec![b'x'; 1024];
+        let key = store.put_raw(&payload).expect("compressed event fits");
+        let bytes = std::fs::metadata(store.file_path(&key)).unwrap().len();
+        assert!(bytes < 128);
+        assert_eq!(store.stored_bytes.load(Ordering::SeqCst), bytes);
+        assert_eq!(store.get_raw(&key).unwrap(), payload);
+        drop(store);
+        let store = byte_limited_store(&directory, 128, false);
+        assert_eq!(store.stored_bytes.load(Ordering::SeqCst), bytes);
+        assert_eq!(store.get_raw(&key).unwrap(), payload);
+        assert!(matches!(store.put_raw(&payload), Err(StoreError::LimitExceeded)));
+        store.delete().unwrap();
+    }
+
+    #[test]
+    fn queue_byte_budget_reclaims_failed_overwrites_expiry_and_write_failures() {
+        let directory = temp_store_dir("byte-failed");
+        let store = byte_limited_store(&directory, 96, false);
+        let live = store.put_raw(&[1; 32]).unwrap();
+        store.put_failed_raw("failed-event", &[2; 32]).unwrap();
+        assert!(matches!(store.put_failed_raw("too-large", &[3; 64]), Err(StoreError::LimitExceeded)));
+        assert_eq!(store.get_raw(&live).unwrap(), [1; 32]);
+        store.put_failed_raw("failed-event", &[4; 16]).unwrap();
+        assert_eq!(store.stored_bytes.load(Ordering::SeqCst), 48);
+        let failed = store.failed_dir().join("failed-event");
+        File::open(&failed).unwrap().set_modified(UNIX_EPOCH).unwrap();
+        assert_eq!(store.prune_failed_store().unwrap(), 1);
+        assert_eq!(store.stored_bytes.load(Ordering::SeqCst), 32);
+        std::fs::create_dir(store.failed_dir().join("cannot-replace-directory")).unwrap();
+        assert!(store.put_failed_raw("cannot-replace-directory", &[5; 20]).is_err());
+        assert_eq!(
+            store.stored_bytes.load(Ordering::SeqCst),
+            32,
+            "failed temporary write must release its reservation"
+        );
+        assert!(store.put_raw(&[6; 64]).is_ok());
+        store.delete().unwrap();
+    }
+
     #[test]
     fn io_error_with_path_preserves_path_and_source_chain() {
         let error = io_error_with_path(

crates/targets/src/target/webhook.rs

@@ -536,10 +536,9 @@ where
         })?;
 
         let status = resp.status();
-        // Drain the response body so the underlying connection is returned to the
-        // pool and can be reused (keep-alive) instead of being closed mid-stream
-        // (backlog#983). The body content is not needed for delivery accounting.
-        let _ = resp.bytes().await;
+        // Reuse connections for ordinary replies without collecting an
+        // endpoint-controlled body. Delivery accounting only depends on status.
+        drain_webhook_response(resp).await;
         let result = classify_delivery_status(status);
         if result.is_ok() {
             debug!(
@@ -557,6 +556,24 @@ where
     }
 }
 
+const WEBHOOK_RESPONSE_DRAIN_LIMIT: usize = 64 * 1024;
+
+async fn drain_webhook_response(mut response: reqwest::Response) {
+    if response
+        .content_length()
+        .is_some_and(|length| length > WEBHOOK_RESPONSE_DRAIN_LIMIT as u64)
+    {
+        return;
+    }
+    let mut remaining = WEBHOOK_RESPONSE_DRAIN_LIMIT;
+    while let Ok(Some(chunk)) = response.chunk().await {
+        let Some(rest) = remaining.checked_sub(chunk.len()) else {
+            return;
+        };
+        remaining = rest;
+    }
+}
+
 fn outbound_policy() -> Result<&'static OutboundPolicy, TargetError> {
     OutboundPolicy::from_env_cached().map_err(|err| TargetError::Configuration(format!("invalid outbound policy: {err}")))
 }
@@ -786,6 +803,41 @@ mod tests {
     use url::Url;
     use url::form_urlencoded;
 
+    #[tokio::test]
+    async fn webhook_response_drain_stops_at_the_byte_limit() {
+        for (headers, body) in [
+            ("Content-Length: 65537", Vec::new()),
+            ("Transfer-Encoding: chunked", format!("10001\r\n{}\r\n", "x".repeat(65537)).into_bytes()),
+            ("Transfer-Encoding: chunked", b"2\r\nok\r\n0\r\n\r\n".to_vec()),
+        ] {
+            let listener = TcpListener::bind("127.0.0.1:0").await.expect("bind response server");
+            let address = listener.local_addr().expect("server address");
+            let server = tokio::spawn(async move {
+                let (mut stream, _) = listener.accept().await.expect("accept request");
+                let mut request = [0; 1024];
+                assert!(stream.read(&mut request).await.expect("read request") > 0);
+                stream
+                    .write_all(format!("HTTP/1.1 200 OK\r\n{headers}\r\n\r\n").as_bytes())
+                    .await
+                    .expect("write headers");
+                let _ = stream.write_all(&body).await;
+                // No EOF: an oversized response must be abandoned without waiting
+                // for its remainder, while a complete small chunked reply drains.
+                std::future::pending::<()>().await;
+            });
+            let client = reqwest::Client::builder().no_proxy().build().expect("direct test client");
+            let response = client
+                .get(format!("http://{address}/"))
+                .send()
+                .await
+                .expect("receive response headers");
+            tokio::time::timeout(std::time::Duration::from_secs(2), super::drain_webhook_response(response))
+                .await
+                .expect("bounded drain must finish without waiting for peer EOF");
+            server.abort();
+        }
+    }
+
     #[derive(Clone)]
     struct StaticResolver(IpAddr);
 

crates/tls-runtime/src/certs.rs

@@ -180,6 +180,13 @@ pub fn build_webpki_client_verifier(options: WebPkiClientVerifierOptions) -> io:
         ))
     })?;
 
+    build_webpki_client_verifier_from_ca_file(&ca_path).map(Some)
+}
+
+/// Load a client CA bundle and require every TLS peer to present a certificate signed by it.
+///
+/// Returns an error if the bundle is missing, unreadable, empty, or contains an invalid CA certificate.
+pub fn build_webpki_client_verifier_from_ca_file(ca_path: &std::path::Path) -> io::Result<Arc<dyn ClientCertVerifier>> {
     let ca_path = ca_path
         .to_str()
         .ok_or_else(|| Error::other(format!("Invalid UTF-8 in mTLS CA path: {ca_path:?}")))?;
@@ -197,7 +204,7 @@ pub fn build_webpki_client_verifier(options: WebPkiClientVerifierOptions) -> io:
         .build()
         .map_err(|e| Error::other(format!("Build client cert verifier failed: {e}")))?;
 
-    Ok(Some(verifier))
+    Ok(verifier)
 }
 
 fn mtls_ca_bundle_path(options: &WebPkiClientVerifierOptions) -> Option<PathBuf> {

crates/tls-runtime/src/lib.rs

@@ -28,9 +28,9 @@ pub mod state;
 
 pub use certs::{
     CertDirectoryLoadOptions, TlsCertPairInspection, TlsCertPairStatus, TlsDirectoryInspection, TlsDomainInspection,
-    WebPkiClientVerifierOptions, build_webpki_client_verifier, create_multi_cert_resolver, inspect_cert_directory,
-    load_all_certs_from_directory, load_all_certs_from_directory_strict, load_cert_bundle_der_bytes, load_certs,
-    load_private_key, validate_certified_key,
+    WebPkiClientVerifierOptions, build_webpki_client_verifier, build_webpki_client_verifier_from_ca_file,
+    create_multi_cert_resolver, inspect_cert_directory, load_all_certs_from_directory, load_all_certs_from_directory_strict,
+    load_cert_bundle_der_bytes, load_certs, load_private_key, validate_certified_key,
 };
 pub use config::{ReloadApplyHint, ReloadDetectMode, TlsReloadOptions};
 pub use coordinator::{TlsConsumer, TlsReloadCoordinator};

crates/trusted-proxies/Cargo.toml

@@ -40,6 +40,7 @@ hotpath-cpu = ["hotpath", "hotpath/hotpath-cpu", "rustfs-config/hotpath-cpu", "r
 hotpath.workspace = true
 async-trait = { workspace = true }
 axum = { workspace = true }
+futures = { workspace = true }
 http = { workspace = true }
 ipnetwork = { workspace = true, features = ["serde"] }
 metrics = { workspace = true }

crates/trusted-proxies/README.md

@@ -6,8 +6,7 @@ maintaining a zero-trust security model.
 
 ## Modes
 
-- **Simple default**: only trusts forwarding headers when the direct peer IP is
-  internal.
+- **Simple default**: trusts one internal proxy and uses the rightmost `X-Forwarded-For` address, including repeated header lines.
 - **Legacy full mode**: keeps the original proxy-chain validation, available
   via `legacy_*` helpers.
 
@@ -21,7 +20,7 @@ maintaining a zero-trust security model.
 - **High Performance**: Utilizes the `moka` cache for fast lookup of validation results and `axum` for a
   high-performance web interface.
 - **Observability**: Built-in support for Prometheus metrics and structured JSON logging via `tracing`.
-- **RFC 7239 Support**: Full support for the modern `Forwarded` header alongside legacy `X-Forwarded-For` headers.
+- **RFC 7239 Support**: Legacy mode can explicitly select `Forwarded` instead of `X-Forwarded-For`.
 
 ## Configuration
 
@@ -31,6 +30,7 @@ The module is configured primarily through environment variables:
 |-----------------------------------------------|---------------------|---------------------------------------------------------|
 | `RUSTFS_TRUSTED_PROXY_ENABLED`                | `true`              | Enable the trusted proxy middleware                     |
 | `RUSTFS_TRUSTED_PROXY_IMPLEMENTATION`         | `simple`            | Select `simple` or `legacy` implementation              |
+| `RUSTFS_TRUSTED_PROXY_ENABLE_RFC7239`         | `false`             | In legacy mode, select `Forwarded` exclusively instead of `X-Forwarded-For` |
 | `RUSTFS_TRUSTED_PROXY_VALIDATION_MODE`        | `hop_by_hop`        | Validation strategy (`strict`, `lenient`, `hop_by_hop`) |
 | `RUSTFS_TRUSTED_PROXY_NETWORKS`               | `127.0.0.1,::1`     | Comma-separated list of trusted CIDR ranges (loopback-only by default) |
 | `RUSTFS_TRUSTED_PROXY_MAX_HOPS`               | `10`                | Maximum allowed proxy hops                              |
@@ -68,7 +68,9 @@ let app = Router::new()
 
 ### Simple default mode
 
-The default mode only trusts forwarding headers from internal IPs.
+The default mode trusts one proxy whose direct socket address is internal. It uses the last address that proxy appended to `X-Forwarded-For`, even if that address is a private LAN client or another proxy. Client-written earlier addresses do not influence policy conditions or rate-limit identity. The proxy must append or overwrite `X-Forwarded-For` and overwrite `X-Forwarded-Proto` and `X-Forwarded-Host`; these last two headers use their rightmost values. `Forwarded`, `X-Real-IP`, and `X-Forwarded-Scheme` are not fallback sources.
+
+For multiple proxies, use legacy mode with `RUSTFS_TRUSTED_PROXY_NETWORKS` restricted to the actual proxy addresses. Simple mode reports the immediately preceding proxy in such deployments. Use the same explicit network configuration when untrusted clients can connect directly from an internal network.
 
 ```bash
 RUSTFS_TRUSTED_PROXY_IMPLEMENTATION=simple
@@ -89,6 +91,10 @@ Or switch the global default path:
 RUSTFS_TRUSTED_PROXY_IMPLEMENTATION=legacy
 ```
 
+Legacy mode uses `X-Forwarded-For` by default. Deployments whose proxy writes RFC 7239 headers must set `RUSTFS_TRUSTED_PROXY_ENABLE_RFC7239=true`; that selects `Forwarded` exclusively, including its last element's `proto` and `host`. There is no automatic fallback between families. Every intermediary must append or overwrite the selected header and strip or overwrite any protocol/host headers it does not populate. Hop-by-hop and lenient modes derive the client from the trusted suffix, not the client-written prefix. Strict mode continues to require every listed address to be trusted.
+
+Malformed forwarding headers, a missing selected family when another family is supplied, and rejected proxy chains now return HTTP 400 before the request reaches application handlers. They are never attributed to the proxy's own address. IPv4-mapped IPv6 peers and forwarding values are canonicalized to IPv4; policies still recognize explicitly mapped IPv6 networks. Upgrading a multi-proxy or `Forwarded`-only deployment requires the mode settings above before serving traffic. Rolling back restores the previous permissive header selection and validation fallback.
+
 ### Accessing Client Info
 
 Retrieve the verified client information in your handlers or other middleware:

crates/trusted-proxies/src/config/types.rs

@@ -31,7 +31,7 @@ use std::time::Duration;
 #[serde(rename_all = "snake_case")]
 #[derive(Default)]
 pub enum ValidationMode {
-    /// Lenient mode: Accepts the entire chain as long as the last proxy is trusted.
+    /// Lenient mode: Identifies the client immediately before the trusted suffix.
     Lenient,
     /// Strict mode: Requires all proxies in the chain to be trusted.
     Strict,
@@ -90,8 +90,8 @@ impl TrustedProxy {
     /// Checks if the given IP address matches this proxy configuration.
     pub fn contains(&self, ip: &IpAddr) -> bool {
         match self {
-            Self::Single(proxy_ip) => ip == proxy_ip,
-            Self::Cidr(network) => network.contains(*ip),
+            Self::Single(proxy_ip) => ip.to_canonical() == proxy_ip.to_canonical(),
+            Self::Cidr(network) => network.contains(*ip) || network.contains(ip.to_canonical()),
         }
     }
 }
@@ -103,7 +103,7 @@ pub struct TrustedProxyConfig {
     pub proxies: Vec<TrustedProxy>,
     /// The validation mode to use for verifying proxy chains.
     pub validation_mode: ValidationMode,
-    /// Whether to enable RFC 7239 "Forwarded" header support.
+    /// Select RFC 7239 `Forwarded` exclusively; false selects `X-Forwarded-For`.
     pub enable_rfc7239: bool,
     /// Maximum allowed proxy hops in the chain.
     pub max_hops: usize,

crates/trusted-proxies/src/middleware/service.rs

@@ -14,8 +14,9 @@
 
 //! Tower service implementation for the trusted proxy middleware.
 
-use crate::{ClientInfo, ProxyValidator};
-use http::Request;
+use crate::ProxyValidator;
+use futures::future::{Either, Ready, ready};
+use http::{Request, Response, StatusCode};
 use std::sync::Arc;
 use std::task::{Context, Poll};
 use tower::Service;
@@ -48,14 +49,15 @@ impl<S> TrustedProxyMiddleware<S> {
     }
 }
 
-impl<S, ReqBody> Service<Request<ReqBody>> for TrustedProxyMiddleware<S>
+impl<S, ReqBody, ResBody> Service<Request<ReqBody>> for TrustedProxyMiddleware<S>
 where
-    S: Service<Request<ReqBody>> + Clone + Send + 'static,
+    S: Service<Request<ReqBody>, Response = Response<ResBody>> + Clone + Send + 'static,
     S::Future: Send,
+    ResBody: Default,
 {
     type Response = S::Response;
     type Error = S::Error;
-    type Future = S::Future;
+    type Future = Either<S::Future, Ready<Result<S::Response, S::Error>>>;
 
     fn poll_ready(&mut self, cx: &mut Context<'_>) -> Poll<Result<(), Self::Error>> {
         self.inner.poll_ready(cx)
@@ -71,7 +73,7 @@ where
                 state = "disabled",
                 "trusted proxy middleware bypassed"
             );
-            return self.inner.call(req);
+            return Either::Left(self.inner.call(req));
         }
 
         let start_time = std::time::Instant::now();
@@ -107,46 +109,21 @@ where
                 req.extensions_mut().insert(client_info);
             }
             Err(err) => {
-                // If the error is recoverable, fallback to a direct connection info.
-                if err.is_recoverable() {
-                    let duration = start_time.elapsed();
-                    warn!(
-                        event = "proxy_validation.middleware",
-                        component = "trusted_proxies",
-                        subsystem = "middleware",
-                        result = "fallback",
-                        fallback = "socket_peer",
-                        peer_ip = peer_addr
-                            .map(|addr| addr.ip().to_string())
-                            .unwrap_or_else(|| "0.0.0.0".to_string()),
-                        error = %err,
-                        duration_ms = duration.as_millis(),
-                        "trusted proxy validation fell back to direct peer"
-                    );
-                    let client_info = ClientInfo::direct(
-                        peer_addr.unwrap_or_else(|| std::net::SocketAddr::new(std::net::IpAddr::from([0, 0, 0, 0]), 0)),
-                    );
-                    req.extensions_mut().insert(client_info);
-                } else {
-                    let duration = start_time.elapsed();
-                    warn!(
-                        event = "proxy_validation.middleware",
-                        component = "trusted_proxies",
-                        subsystem = "middleware",
-                        result = "error",
-                        fallback = "none",
-                        peer_ip = peer_addr
-                            .map(|addr| addr.ip().to_string())
-                            .unwrap_or_else(|| "0.0.0.0".to_string()),
-                        error = %err,
-                        duration_ms = duration.as_millis(),
-                        "trusted proxy validation failed"
-                    );
-                }
+                warn!(
+                    event = "proxy_validation.middleware",
+                    component = "trusted_proxies",
+                    subsystem = "middleware",
+                    result = "rejected",
+                    error = %err,
+                    "trusted proxy validation rejected request"
+                );
+                let mut response = Response::new(ResBody::default());
+                *response.status_mut() = StatusCode::BAD_REQUEST;
+                return Either::Right(ready(Ok(response)));
             }
         }
 
         // Call the inner service.
-        self.inner.call(req)
+        Either::Left(self.inner.call(req))
     }
 }

crates/trusted-proxies/src/proxy/chain.rs

@@ -135,21 +135,10 @@ impl ProxyChainAnalyzer {
         })
     }
 
-    /// Lenient mode: Accepts the entire chain if the last proxy is trusted.
+    /// Lenient mode still derives identity from the trusted suffix: accepting
+    /// a connection does not make client-written earlier hops authoritative.
     fn analyze_lenient(&self, chain: &[IpAddr]) -> (IpAddr, Vec<IpAddr>, usize) {
-        if chain.is_empty() {
-            return (IpAddr::from([0, 0, 0, 0]), Vec::new(), 0);
-        }
-
-        if let Some(last_proxy) = chain.last()
-            && self.is_ip_trusted(last_proxy)
-        {
-            let client_ip = chain.first().copied().unwrap_or(*last_proxy);
-            return (client_ip, chain.to_vec(), chain.len());
-        }
-
-        let client_ip = chain.first().copied().unwrap_or_else(|| IpAddr::from([0, 0, 0, 0]));
-        (client_ip, Vec::new(), 0)
+        self.analyze_hop_by_hop(chain)
     }
 
     /// Strict mode: Requires every IP in the chain to be trusted.

crates/trusted-proxies/src/proxy/validator.rs

@@ -49,7 +49,7 @@ impl ClientInfo {
     /// Creates a `ClientInfo` for a direct connection without any proxies.
     pub fn direct(addr: SocketAddr) -> Self {
         Self {
-            real_ip: addr.ip(),
+            real_ip: addr.ip().to_canonical(),
             forwarded_host: None,
             forwarded_proto: None,
             is_from_trusted_proxy: false,
@@ -71,11 +71,11 @@ impl ClientInfo {
         warnings: Vec<String>,
     ) -> Self {
         Self {
-            real_ip,
+            real_ip: real_ip.to_canonical(),
             forwarded_host,
             forwarded_proto,
             is_from_trusted_proxy: true,
-            proxy_ip: Some(proxy_ip),
+            proxy_ip: Some(proxy_ip.to_canonical()),
             proxy_hops,
             validation_mode,
             warnings,
@@ -152,7 +152,7 @@ impl ProxyValidator {
             return Ok(ClientInfo::direct(SocketAddr::new(IpAddr::from([0, 0, 0, 0]), 0)));
         };
 
-        let peer_ip = peer_addr.ip();
+        let peer_ip = peer_addr.ip().to_canonical();
         if peer_ip.is_unspecified() {
             debug!(
                 event = "proxy_validation.evaluate",
@@ -249,14 +249,35 @@ impl ProxyValidator {
 
     /// Validates a request that originated from a trusted proxy.
     fn validate_trusted_proxy_request(&self, proxy_addr: &SocketAddr, headers: &HeaderMap) -> Result<ClientInfo, ProxyError> {
-        let proxy_ip = proxy_addr.ip();
+        let proxy_ip = proxy_addr.ip().to_canonical();
 
-        // Prefer RFC 7239 "Forwarded" header if enabled, otherwise fallback to legacy headers.
-        let client_info = if self.config.enable_rfc7239 {
-            self.try_parse_rfc7239_headers(headers, proxy_ip)
-                .unwrap_or_else(|| self.parse_legacy_headers(headers))
+        // Select one header family explicitly. A client-provided alternative
+        // must never override the family appended by the trusted proxy.
+        let selected = if self.config.enable_rfc7239 {
+            "forwarded"
         } else {
-            self.parse_legacy_headers(headers)
+            "x-forwarded-for"
+        };
+        if !headers.contains_key(selected) {
+            if headers.contains_key("forwarded") || headers.contains_key("x-forwarded-for") || headers.contains_key("x-real-ip") {
+                return Err(ProxyError::HeaderParseError("configured forwarding header is missing".to_string()));
+            }
+            return Ok(ClientInfo::direct(*proxy_addr));
+        }
+        let value = headers
+            .get_all(selected)
+            .iter()
+            .map(|header| {
+                header
+                    .to_str()
+                    .map_err(|_| ProxyError::HeaderParseError("forwarding header is not text".to_string()))
+            })
+            .collect::<Result<Vec<_>, _>>()?
+            .join(",");
+        let client_info = if self.config.enable_rfc7239 {
+            Self::parse_forwarded_header(&value)?
+        } else {
+            self.parse_legacy_headers(headers, &value)?
         };
 
         // Analyze the integrity and continuity of the proxy chain.
@@ -299,37 +320,29 @@ impl ProxyValidator {
         ))
     }
 
-    /// Attempts to parse the RFC 7239 "Forwarded" header.
-    fn try_parse_rfc7239_headers(&self, headers: &HeaderMap, proxy_ip: IpAddr) -> Option<ParsedHeaders> {
-        headers
-            .get("forwarded")
-            .and_then(|h| h.to_str().ok())
-            .and_then(|s| Self::parse_forwarded_header(s, proxy_ip))
-    }
-
-    /// Parses legacy proxy headers (X-Forwarded-For, X-Forwarded-Host, X-Forwarded-Proto).
-    fn parse_legacy_headers(&self, headers: &HeaderMap) -> ParsedHeaders {
-        let forwarded_host = headers
-            .get("x-forwarded-host")
-            .and_then(|h| h.to_str().ok())
-            .map(String::from);
-
-        let forwarded_proto = headers
-            .get("x-forwarded-proto")
-            .and_then(|h| h.to_str().ok())
-            .map(String::from);
-
-        let proxy_chain = headers
-            .get("x-forwarded-for")
-            .and_then(|h| h.to_str().ok())
-            .map(Self::parse_x_forwarded_for)
-            .unwrap_or_default();
-
-        ParsedHeaders {
+    /// Parses only the configured legacy header family, including repeated lines.
+    fn parse_legacy_headers(&self, headers: &HeaderMap, value: &str) -> Result<ParsedHeaders, ProxyError> {
+        let last_value = |name| {
+            headers
+                .get_all(name)
+                .iter()
+                .next_back()?
+                .to_str()
+                .ok()?
+                .rsplit(',')
+                .next()
+                .map(str::trim)
+                .map(String::from)
+        };
+        let proxy_chain = value
+            .split(',')
+            .map(|token| Self::parse_forwarded_node(token.trim()).ok_or_else(|| ProxyError::invalid_xff("invalid address")))
+            .collect::<Result<Vec<_>, _>>()?;
+        Ok(ParsedHeaders {
             proxy_chain,
-            forwarded_host,
-            forwarded_proto,
-        }
+            forwarded_host: last_value("x-forwarded-host"),
+            forwarded_proto: last_value("x-forwarded-proto"),
+        })
     }
 
     /// Parses the RFC 7239 "Forwarded" header value.
@@ -344,12 +357,8 @@ impl ProxyValidator {
     /// which is appended by the directly connected trusted proxy — mirroring
     /// how the legacy path reads proxy-set `X-Forwarded-Host`/`Proto` headers.
     /// Values injected by the client in earlier elements are ignored.
-    fn parse_forwarded_header(header_value: &str, proxy_ip: IpAddr) -> Option<ParsedHeaders> {
-        let elements: Vec<&str> = header_value
-            .split(',')
-            .map(|element| element.trim())
-            .filter(|element| !element.is_empty())
-            .collect();
+    fn parse_forwarded_header(header_value: &str) -> Result<ParsedHeaders, ProxyError> {
+        let elements: Vec<&str> = header_value.split(',').map(|element| element.trim()).collect();
 
         let mut proxy_chain = Vec::new();
         let mut forwarded_host = None;
@@ -358,6 +367,7 @@ impl ProxyValidator {
         let last_index = elements.len().saturating_sub(1);
         for (index, element) in elements.iter().enumerate() {
             let is_last = index == last_index;
+            let mut client_ip = None;
             for part in element.split(';') {
                 let part = part.trim();
                 if let Some((key, value)) = part.split_once('=') {
@@ -366,9 +376,13 @@ impl ProxyValidator {
 
                     match key.as_str() {
                         "for" => {
-                            if let Some(ip) = Self::parse_forwarded_node(value) {
-                                proxy_chain.push(ip);
+                            if client_ip.is_some() {
+                                return Err(ProxyError::invalid_forwarded("duplicate for parameter"));
                             }
+                            client_ip = Some(
+                                Self::parse_forwarded_node(value)
+                                    .ok_or_else(|| ProxyError::invalid_forwarded("invalid for address"))?,
+                            );
                         }
                         // Only honor host/proto from the trusted proxy-appended node.
                         "host" if is_last => {
@@ -381,14 +395,10 @@ impl ProxyValidator {
                     }
                 }
             }
+            proxy_chain.push(client_ip.ok_or_else(|| ProxyError::invalid_forwarded("missing for address"))?);
         }
 
-        // Fallback to the proxy IP if no client IP was found in the header.
-        if proxy_chain.is_empty() {
-            proxy_chain.push(proxy_ip);
-        }
-
-        Some(ParsedHeaders {
+        Ok(ParsedHeaders {
             proxy_chain,
             forwarded_host,
             forwarded_proto,
@@ -428,7 +438,7 @@ impl ProxyValidator {
             token.rsplit_once(':').map(|(host, _)| host).unwrap_or(token)
         };
 
-        ip_str.parse::<IpAddr>().ok()
+        ip_str.parse::<IpAddr>().ok().map(|ip| ip.to_canonical())
     }
 
     /// Records the start of a validation attempt in metrics.

crates/trusted-proxies/src/simple.rs

@@ -19,7 +19,8 @@
 //! switch the global entrypoints to the legacy chain validator.
 
 use crate::{ClientInfo, LegacyTrustedProxyLayer, LegacyTrustedProxyMiddleware, ValidationMode, global};
-use axum::http::{HeaderMap, Request};
+use axum::http::{HeaderMap, Request, Response, StatusCode};
+use futures::future::{Either, Ready, ready};
 use rustfs_config::{
     DEFAULT_TRUSTED_PROXY_ENABLED, DEFAULT_TRUSTED_PROXY_IMPLEMENTATION, ENV_TRUSTED_PROXY_ENABLED,
     ENV_TRUSTED_PROXY_IMPLEMENTATION,
@@ -35,11 +36,14 @@ use tracing::debug;
 /// Constant switch for the crate's default integration path.
 pub const SIMPLE_INTERNAL_ONLY_DEFAULT: bool = true;
 
+#[cfg(test)]
 const HEADER_FORWARDED: &str = "forwarded";
 const HEADER_X_FORWARDED_FOR: &str = "x-forwarded-for";
 const HEADER_X_FORWARDED_HOST: &str = "x-forwarded-host";
 const HEADER_X_FORWARDED_PROTO: &str = "x-forwarded-proto";
+#[cfg(test)]
 const HEADER_X_FORWARDED_SCHEME: &str = "x-forwarded-scheme";
+#[cfg(test)]
 const HEADER_X_REAL_IP: &str = "x-real-ip";
 
 static ENABLED: OnceLock<bool> = OnceLock::new();
@@ -162,14 +166,15 @@ pub enum TrustedProxyMiddleware<S> {
     Legacy(LegacyTrustedProxyMiddleware<S>),
 }
 
-impl<S, ReqBody> Service<Request<ReqBody>> for TrustedProxyMiddleware<S>
+impl<S, ReqBody, ResBody> Service<Request<ReqBody>> for TrustedProxyMiddleware<S>
 where
-    S: Service<Request<ReqBody>> + Clone + Send + 'static,
+    S: Service<Request<ReqBody>, Response = Response<ResBody>> + Clone + Send + 'static,
     S::Future: Send,
+    ResBody: Default,
 {
     type Response = S::Response;
     type Error = S::Error;
-    type Future = S::Future;
+    type Future = Either<S::Future, Ready<Result<S::Response, S::Error>>>;
 
     fn poll_ready(&mut self, cx: &mut Context<'_>) -> Poll<Result<(), Self::Error>> {
         match self {
@@ -224,14 +229,15 @@ pub struct SimpleTrustedProxyMiddleware<S> {
     enabled: bool,
 }
 
-impl<S, ReqBody> Service<Request<ReqBody>> for SimpleTrustedProxyMiddleware<S>
+impl<S, ReqBody, ResBody> Service<Request<ReqBody>> for SimpleTrustedProxyMiddleware<S>
 where
-    S: Service<Request<ReqBody>> + Clone + Send + 'static,
+    S: Service<Request<ReqBody>, Response = Response<ResBody>> + Clone + Send + 'static,
     S::Future: Send,
+    ResBody: Default,
 {
     type Response = S::Response;
     type Error = S::Error;
-    type Future = S::Future;
+    type Future = Either<S::Future, Ready<Result<S::Response, S::Error>>>;
 
     fn poll_ready(&mut self, cx: &mut Context<'_>) -> Poll<Result<(), Self::Error>> {
         self.inner.poll_ready(cx)
@@ -240,13 +246,23 @@ where
     fn call(&mut self, mut req: Request<ReqBody>) -> Self::Future {
         if self.enabled {
             let peer_addr = req.extensions().get::<SocketAddr>().copied();
+            if peer_addr.is_some_and(|peer| is_internal_ip(peer.ip()))
+                && (req.headers().contains_key(HEADER_X_FORWARDED_FOR)
+                    || req.headers().contains_key("forwarded")
+                    || req.headers().contains_key("x-real-ip"))
+                && forwarded_client_ip(req.headers()).is_none_or(|ip| !is_usable_ip(ip))
+            {
+                let mut response = Response::new(ResBody::default());
+                *response.status_mut() = StatusCode::BAD_REQUEST;
+                return Either::Right(ready(Ok(response)));
+            }
             let client_info = resolve_client_info(peer_addr, req.headers());
             req.extensions_mut().insert(client_info);
         } else {
             debug!("Simple trusted proxy middleware is disabled");
         }
 
-        self.inner.call(req)
+        Either::Left(self.inner.call(req))
     }
 }
 
@@ -275,54 +291,30 @@ fn resolve_client_info(peer_addr: Option<SocketAddr>, headers: &HeaderMap) -> Cl
 
 fn forwarded_client_ip(headers: &HeaderMap) -> Option<IpAddr> {
     parse_x_forwarded_for(headers)
-        .or_else(|| parse_single_ip_header(headers, HEADER_X_REAL_IP))
-        .or_else(|| parse_forwarded_header(headers))
 }
 
 fn forwarded_host(headers: &HeaderMap) -> Option<String> {
-    parse_single_value_header(headers, HEADER_X_FORWARDED_HOST)
-        .and_then(sanitize_forwarded_host)
-        .or_else(|| parse_forwarded_header_value(headers, "host").and_then(sanitize_forwarded_host))
+    parse_single_value_header(headers, HEADER_X_FORWARDED_HOST).and_then(sanitize_forwarded_host)
 }
 
 fn forwarded_proto(headers: &HeaderMap) -> Option<String> {
-    parse_single_value_header(headers, HEADER_X_FORWARDED_PROTO)
-        .and_then(sanitize_forwarded_proto)
-        .or_else(|| parse_single_value_header(headers, HEADER_X_FORWARDED_SCHEME).and_then(sanitize_forwarded_proto))
-        .or_else(|| parse_forwarded_header_value(headers, "proto").and_then(sanitize_forwarded_proto))
+    parse_single_value_header(headers, HEADER_X_FORWARDED_PROTO).and_then(sanitize_forwarded_proto)
 }
 
 fn parse_single_value_header(headers: &HeaderMap, name: &str) -> Option<String> {
     headers
-        .get(name)?
+        .get_all(name)
+        .iter()
+        .next_back()?
         .to_str()
         .ok()?
-        .split(',')
+        .rsplit(',')
         .next()
         .map(str::trim)
         .filter(|value| !value.is_empty())
         .map(ToOwned::to_owned)
 }
 
-fn parse_forwarded_header_value(headers: &HeaderMap, name: &str) -> Option<String> {
-    let value = headers.get(HEADER_FORWARDED)?.to_str().ok()?;
-    let first = value.split(',').next()?.trim();
-
-    for part in first.split(';') {
-        let Some((key, value)) = part.trim().split_once('=') else {
-            continue;
-        };
-        if key.trim().eq_ignore_ascii_case(name) {
-            let value = value.trim().trim_matches('"');
-            if !value.is_empty() {
-                return Some(value.to_string());
-            }
-        }
-    }
-
-    None
-}
-
 fn sanitize_forwarded_host(value: String) -> Option<String> {
     sanitize_forwarded_value(&value).filter(|value| !value.chars().any(char::is_whitespace))
 }
@@ -346,30 +338,10 @@ fn sanitize_forwarded_value(value: &str) -> Option<String> {
 }
 
 fn parse_x_forwarded_for(headers: &HeaderMap) -> Option<IpAddr> {
-    let value = headers.get(HEADER_X_FORWARDED_FOR)?.to_str().ok()?;
-    let first = value.split(',').next()?.trim();
-    parse_ip_token(first)
-}
-
-fn parse_single_ip_header(headers: &HeaderMap, name: &str) -> Option<IpAddr> {
-    let value = headers.get(name)?.to_str().ok()?;
-    parse_ip_token(value)
-}
-
-fn parse_forwarded_header(headers: &HeaderMap) -> Option<IpAddr> {
-    let value = headers.get(HEADER_FORWARDED)?.to_str().ok()?;
-    let first_entry = value.split(',').next()?.trim();
-
-    for part in first_entry.split(';') {
-        let Some((key, raw_value)) = part.split_once('=') else {
-            continue;
-        };
-        if key.trim().eq_ignore_ascii_case("for") {
-            return parse_ip_token(raw_value.trim());
-        }
-    }
-
-    None
+    // Simple mode trusts exactly one proxy. The rightmost value is the peer
+    // that proxy observed, even when that peer is another private address.
+    let value = headers.get_all(HEADER_X_FORWARDED_FOR).iter().next_back()?.to_str().ok()?;
+    parse_ip_token(value.rsplit(',').next()?.trim())
 }
 
 fn parse_ip_token(value: &str) -> Option<IpAddr> {
@@ -381,22 +353,22 @@ fn parse_ip_token(value: &str) -> Option<IpAddr> {
     if let Some(bracketed) = value.strip_prefix('[')
         && let Some(end) = bracketed.find(']')
     {
-        return IpAddr::from_str(&bracketed[..end]).ok();
+        return IpAddr::from_str(&bracketed[..end]).ok().map(|ip| ip.to_canonical());
     }
 
     if let Ok(ip) = IpAddr::from_str(value) {
-        return Some(ip);
+        return Some(ip.to_canonical());
     }
 
     if let Ok(socket_addr) = SocketAddr::from_str(value) {
-        return Some(socket_addr.ip());
+        return Some(socket_addr.ip().to_canonical());
     }
 
     None
 }
 
 fn is_internal_ip(ip: IpAddr) -> bool {
-    match ip {
+    match ip.to_canonical() {
         IpAddr::V4(ip) => ip.is_private() || ip.is_loopback() || ip.is_link_local(),
         IpAddr::V6(ip) => ip.is_loopback() || ip.is_unique_local() || ip.is_unicast_link_local(),
     }
@@ -459,14 +431,14 @@ mod tests {
         headers.insert(HEADER_X_FORWARDED_FOR, HeaderValue::from_static("203.0.113.10, 10.0.0.5"));
         headers.insert(HEADER_X_REAL_IP, HeaderValue::from_static("198.51.100.10"));
         headers.insert(HEADER_FORWARDED, HeaderValue::from_static("for=192.0.2.60;proto=https"));
-        assert_eq!(forwarded_client_ip(&headers), Some(IpAddr::from([203, 0, 113, 10])));
+        assert_eq!(forwarded_client_ip(&headers), Some(IpAddr::from([10, 0, 0, 5])));
     }
 
     #[test]
-    fn test_forwarded_header_fallback() {
+    fn test_forwarded_header_does_not_select_another_address_family() {
         let mut headers = HeaderMap::new();
         headers.insert(HEADER_FORWARDED, HeaderValue::from_static("for=203.0.113.10;proto=https"));
-        assert_eq!(forwarded_client_ip(&headers), Some(IpAddr::from([203, 0, 113, 10])));
+        assert_eq!(forwarded_client_ip(&headers), None);
     }
 
     #[test]
@@ -494,18 +466,18 @@ mod tests {
     }
 
     #[test]
-    fn test_internal_peer_preserves_forwarded_scheme_fallback() {
+    fn test_internal_peer_ignores_unselected_forwarded_scheme() {
         let mut headers = HeaderMap::new();
         headers.insert(HEADER_X_FORWARDED_FOR, HeaderValue::from_static("203.0.113.10"));
         headers.insert(HEADER_X_FORWARDED_SCHEME, HeaderValue::from_static("https"));
 
         let client_info = resolve_client_info(Some(SocketAddr::from(([10, 0, 0, 5], 9000))), &headers);
 
-        assert_eq!(client_info.forwarded_proto.as_deref(), Some("https"));
+        assert_eq!(client_info.forwarded_proto, None);
     }
 
     #[test]
-    fn test_forwarded_header_preserves_proto_and_host() {
+    fn test_forwarded_header_cannot_supply_proto_or_host() {
         let mut headers = HeaderMap::new();
         headers.insert(
             HEADER_FORWARDED,
@@ -514,9 +486,9 @@ mod tests {
 
         let client_info = resolve_client_info(Some(SocketAddr::from(([10, 0, 0, 5], 9000))), &headers);
 
-        assert_eq!(client_info.real_ip, IpAddr::from([203, 0, 113, 10]));
-        assert_eq!(client_info.forwarded_host.as_deref(), Some("s3.example.test"));
-        assert_eq!(client_info.forwarded_proto.as_deref(), Some("https"));
+        assert_eq!(client_info.real_ip, IpAddr::from([10, 0, 0, 5]));
+        assert_eq!(client_info.forwarded_host, None);
+        assert_eq!(client_info.forwarded_proto, None);
     }
 
     #[test]
@@ -559,10 +531,12 @@ mod tests {
 
     #[test]
     fn test_forwarded_header_segment_without_equals() {
-        // A segment without '=' before 'for=' must not abort parsing.
+        // Simple mode never accepts the alternate Forwarded family.
         let mut headers = HeaderMap::new();
         headers.insert(HEADER_FORWARDED, HeaderValue::from_static("proto;for=203.0.113.10"));
-        assert_eq!(forwarded_client_ip(&headers), Some(IpAddr::from([203, 0, 113, 10])));
+        assert_eq!(forwarded_client_ip(&headers), None);
+        headers.insert(HEADER_X_FORWARDED_FOR, HeaderValue::from_static("203.0.113.20"));
+        assert_eq!(forwarded_client_ip(&headers), Some(IpAddr::from([203, 0, 113, 20])));
     }
 
     #[test]

crates/trusted-proxies/tests/integration/proxy_tests.rs

@@ -20,7 +20,7 @@ use tower::ServiceExt;
 #[tokio::test]
 async fn test_proxy_validation_flow() {
     let proxies = vec![TrustedProxy::Single("127.0.0.1".parse().unwrap())];
-    let config = TrustedProxyConfig::new(proxies, ValidationMode::HopByHop, true, 10, true, vec![]);
+    let config = TrustedProxyConfig::new(proxies, ValidationMode::HopByHop, false, 10, true, vec![]);
     let proxy_layer = LegacyTrustedProxyLayer::enabled(config, None);
 
     let app = Router::new().route("/test", get(|| async { "OK" })).layer(proxy_layer);

crates/trusted-proxies/tests/proxy_layer.rs

@@ -21,6 +21,167 @@ use std::convert::Infallible;
 use std::net::SocketAddr;
 use tower::{Layer, ServiceExt, service_fn};
 
+#[tokio::test]
+async fn simple_proxy_uses_only_the_rightmost_observed_peer() {
+    for (chain, expected) in [
+        ("203.0.113.7, 198.51.100.9", "198.51.100.9"),
+        ("203.0.113.7, 10.20.0.9", "10.20.0.9"),
+        ("203.0.113.7, ::ffff:198.51.100.9", "198.51.100.9"),
+        ("198.51.100.9, 10.20.0.8", "10.20.0.8"),
+    ] {
+        let service = service_fn(|request: Request<Body>| async move {
+            let info = request.extensions().get::<ClientInfo>().expect("client identity");
+            assert_eq!(info.forwarded_proto.as_deref(), Some("http"));
+            Ok::<_, Infallible>(Response::new(Body::from(info.real_ip.to_string())))
+        });
+        let service = TrustedProxyLayer::enabled().layer(service);
+        let mut request = Request::builder().body(Body::empty()).expect("request");
+        request
+            .extensions_mut()
+            .insert("[::ffff:10.0.0.5]:9000".parse::<SocketAddr>().expect("peer"));
+        request
+            .headers_mut()
+            .append("x-forwarded-for", "203.0.113.1".parse().expect("forged line"));
+        request
+            .headers_mut()
+            .append("x-forwarded-for", chain.parse().expect("proxy line"));
+        request
+            .headers_mut()
+            .append("x-forwarded-proto", "https".parse().expect("forged proto"));
+        request
+            .headers_mut()
+            .append("x-forwarded-proto", "http".parse().expect("proxy proto"));
+        request
+            .headers_mut()
+            .insert("forwarded", "for=127.0.0.1;proto=https".parse().expect("forged family"));
+        let response = service.oneshot(request).await.expect("proxy response");
+        assert_eq!(response.status(), 200);
+        let body = body::to_bytes(response.into_body(), 1024).await.expect("response body");
+        assert_eq!(body.as_ref(), expected.as_bytes());
+    }
+}
+
+#[tokio::test]
+async fn invalid_proxy_chains_never_reach_the_inner_service() {
+    use rustfs_trusted_proxies::{LegacyTrustedProxyLayer, TrustedProxy, TrustedProxyConfig, ValidationMode};
+    use std::sync::{
+        Arc,
+        atomic::{AtomicUsize, Ordering},
+    };
+
+    let peer: SocketAddr = "10.0.0.5:9000".parse().expect("peer");
+    for mode in [
+        None,
+        Some(ValidationMode::HopByHop),
+        Some(ValidationMode::Lenient),
+        Some(ValidationMode::Strict),
+    ] {
+        for invalid in ["203.0.113.7, not-an-ip", "203.0.113.7, 0.0.0.0", "203.0.113.7,", ""] {
+            let calls = Arc::new(AtomicUsize::new(0));
+            let called = Arc::clone(&calls);
+            let service = service_fn(move |_request: Request<Body>| {
+                called.fetch_add(1, Ordering::SeqCst);
+                async { Ok::<_, Infallible>(Response::new(Body::empty())) }
+            });
+            let layer = match mode {
+                None => TrustedProxyLayer::enabled(),
+                Some(mode) => TrustedProxyLayer::legacy(LegacyTrustedProxyLayer::enabled(
+                    TrustedProxyConfig::new(vec![TrustedProxy::Single(peer.ip())], mode, false, 3, false, vec![]),
+                    None,
+                )),
+            };
+            let mut request = Request::builder()
+                .header("x-forwarded-for", invalid)
+                .header("forwarded", "for=127.0.0.1;proto=https")
+                .header("x-real-ip", "127.0.0.1")
+                .body(Body::empty())
+                .expect("request");
+            request.extensions_mut().insert(peer);
+            let response = layer.layer(service).oneshot(request).await.expect("proxy rejection");
+            assert_eq!(response.status(), 400, "mode={mode:?}, header={invalid:?}");
+            assert_eq!(calls.load(Ordering::SeqCst), 0);
+        }
+    }
+}
+
+#[tokio::test]
+async fn legacy_chain_rejection_never_falls_back_to_proxy_identity() {
+    use rustfs_trusted_proxies::{LegacyTrustedProxyLayer, TrustedProxy, TrustedProxyConfig, ValidationMode};
+    use std::sync::{
+        Arc,
+        atomic::{AtomicUsize, Ordering},
+    };
+
+    let peer: SocketAddr = "10.0.0.5:9000".parse().expect("peer");
+    for mode in [ValidationMode::HopByHop, ValidationMode::Lenient, ValidationMode::Strict] {
+        for chain in [None, Some("203.0.113.1, 10.0.0.1, 10.0.0.2, 198.51.100.9")] {
+            let calls = Arc::new(AtomicUsize::new(0));
+            let called = Arc::clone(&calls);
+            let service = service_fn(move |_request: Request<Body>| {
+                called.fetch_add(1, Ordering::SeqCst);
+                async { Ok::<_, Infallible>(Response::new(Body::empty())) }
+            });
+            let layer = LegacyTrustedProxyLayer::enabled(
+                TrustedProxyConfig::new(vec![TrustedProxy::Single(peer.ip())], mode, false, 3, false, vec![]),
+                None,
+            );
+            let mut request = Request::builder()
+                .header("forwarded", "for=127.0.0.1;proto=https")
+                .body(Body::empty())
+                .expect("request");
+            request.extensions_mut().insert(peer);
+            if let Some(chain) = chain {
+                request
+                    .headers_mut()
+                    .insert("x-forwarded-for", chain.parse().expect("long chain"));
+            }
+            let response = layer.layer(service).oneshot(request).await.expect("proxy rejection");
+            assert_eq!(response.status(), 400);
+            assert_eq!(calls.load(Ordering::SeqCst), 0);
+        }
+    }
+}
+
+#[tokio::test]
+async fn legacy_proxy_selects_one_family_and_preserves_repeated_lines() {
+    use rustfs_trusted_proxies::{LegacyTrustedProxyLayer, TrustedProxy, TrustedProxyConfig, ValidationMode};
+    let peer: SocketAddr = "10.0.0.5:9000".parse().expect("peer");
+    for rfc7239 in [false, true] {
+        for mode in [ValidationMode::HopByHop, ValidationMode::Lenient] {
+            let service = service_fn(|request: Request<Body>| async move {
+                let info = request.extensions().get::<ClientInfo>().expect("client identity");
+                assert_eq!(info.real_ip.to_string(), "198.51.100.9");
+                assert_eq!(info.forwarded_proto.as_deref(), Some("http"));
+                Ok::<_, Infallible>(Response::new(Body::empty()))
+            });
+            let layer = LegacyTrustedProxyLayer::enabled(
+                TrustedProxyConfig::new(vec![TrustedProxy::Single(peer.ip())], mode, rfc7239, 5, false, vec![]),
+                None,
+            );
+            let mut request = Request::builder()
+                .header("x-forwarded-for", "127.0.0.1")
+                .header("forwarded", "for=127.0.0.1;proto=https")
+                .body(Body::empty())
+                .expect("request");
+            request.extensions_mut().insert(peer);
+            if rfc7239 {
+                request
+                    .headers_mut()
+                    .append("forwarded", "for=198.51.100.9;proto=http".parse().expect("proxy line"));
+            } else {
+                request
+                    .headers_mut()
+                    .append("x-forwarded-for", "198.51.100.9".parse().expect("proxy line"));
+                request
+                    .headers_mut()
+                    .insert("x-forwarded-proto", "http".parse().expect("proxy proto"));
+            }
+            let response = layer.layer(service).oneshot(request).await.expect("valid proxy request");
+            assert_eq!(response.status(), 200);
+        }
+    }
+}
+
 #[tokio::test]
 async fn test_layer_inserts_client_info_for_internal_proxy() {
     let peer_addr = SocketAddr::from(([10, 0, 0, 5], 9000));

crates/trusted-proxies/tests/unit/config_tests.rs

@@ -36,7 +36,7 @@ fn test_config_loader_default() {
             assert_eq!(config.server_addr.port(), 9000);
             assert!(!config.proxy.proxies.is_empty());
             assert_eq!(config.proxy.validation_mode, ValidationMode::HopByHop);
-            assert!(config.proxy.enable_rfc7239);
+            assert!(!config.proxy.enable_rfc7239);
             assert_eq!(config.proxy.max_hops, 10);
         },
     );

crates/trusted-proxies/tests/unit/validator_tests.rs

@@ -112,7 +112,7 @@ fn test_ipv6_client_behind_trusted_proxy_resolves_client() {
     let config = TrustedProxyConfig::new(
         vec![TrustedProxy::Cidr("fd00::/8".parse().unwrap())],
         ValidationMode::HopByHop,
-        true,
+        false,
         5,
         true,
         vec![],
@@ -176,11 +176,11 @@ fn test_validator_caches_trusted_direct_peer_decision() {
     assert_eq!(validator.cache_stats().size, 0);
 
     let first = validator.validate_request(peer_addr, &headers).unwrap();
-    assert!(first.is_from_trusted_proxy);
+    assert!(!first.is_from_trusted_proxy);
     assert_eq!(validator.cache_stats().size, 1);
 
     let second = validator.validate_request(peer_addr, &headers).unwrap();
-    assert!(second.is_from_trusted_proxy);
+    assert!(!second.is_from_trusted_proxy);
     assert_eq!(validator.cache_stats().size, 1);
 }
 

crates/zstd-compat/Cargo.toml

@@ -30,7 +30,7 @@ zstdmt = ["zstd_fixed/zstdmt"]
 
 [dev-dependencies]
 zstd-safe = { version = "8.2.0", features = ["seekable"] }
-toml = "1.1.8"
+toml = "1.2.0"
 semver = "1.0.28"
 apache-avro.workspace = true
 arrow-array = "60.1.0"

docs/operations/target-queue-storage.md

@@ -0,0 +1,9 @@
+# Target queue storage limits
+
+Disk queues for notification and audit targets enforce both their configured entry limit and a shared byte limit for each queue directory. `RUSTFS_TARGET_STORE_MAX_BYTES` sets that byte limit; the default is 1 GiB. The value must be a positive integer in bytes. Missing, zero, invalid, and overflowing values use the default.
+
+The byte budget counts persisted live entries, failed entries, and reserved temporary writes. Compressed entries count their compressed file size. Payload fields are preserved. Concurrent clones of the same queue share reservations, and opening the queue reconstructs usage from disk. A queue directory must have a single owning process, as with the existing entry limit.
+
+A write that cannot reserve space returns `StoreError::LimitExceeded`. Moving a live event to the failed store also needs space for its new representation before deleting the live entry; if that reservation fails, the live entry remains available for retry. Failed entries retain the existing count and expiry policies. Removal and expiry reclaim bytes. An I/O failure whose temporary file cannot be removed conservatively retains its reservation until cleanup or reopening succeeds.
+
+Upgrading or reducing the limit preserves existing entries, including queues already above the limit. New writes remain rejected until delivery, expiry, or operator cleanup frees enough space. Reopening does not rewrite event payloads or change their filename format. Rolling back preserves readability but removes the byte limit on an older writer.

rustfs/src/admin/auth.rs

@@ -293,6 +293,18 @@ pub async fn authenticate_request(
     result
 }
 
+/// The policy client address comes only from verified middleware context.
+/// Keep the original socket peer in request extensions for auditing.
+pub(crate) fn policy_source_addr(extensions: &http::Extensions) -> Option<std::net::SocketAddr> {
+    let peer = extensions
+        .get::<Option<RemoteAddr>>()
+        .and_then(|peer| peer.map(|addr| addr.0));
+    extensions
+        .get::<rustfs_trusted_proxies::ClientInfo>()
+        .map(|client| std::net::SocketAddr::new(client.real_ip, peer.map_or(0, |addr| addr.port())))
+        .or(peer)
+}
+
 /// Full admin gate over an `S3Request`: extract the request credentials,
 /// authenticate them ([`authenticate_request`]), then authorize the caller for
 /// `actions` ([`validate_admin_request`], allowing on the first permitted
@@ -306,7 +318,7 @@ pub async fn authorize_admin_request(req: &S3Request<Body>, actions: Vec<Action>
 
     let (cred, owner) = authenticate_request(&req.headers, &req.uri, input_cred).await?;
 
-    let remote_addr = req.extensions.get::<Option<RemoteAddr>>().and_then(|opt| opt.map(|a| a.0));
+    let remote_addr = policy_source_addr(&req.extensions);
     validate_admin_request(&req.headers, &cred, owner, false, actions, remote_addr).await?;
 
     Ok(cred)
@@ -547,6 +559,163 @@ mod tests {
         );
     }
 
+    #[test]
+    #[serial_test::serial]
+    fn admin_and_sts_handlers_use_verified_client_ip_and_bucket_scope() {
+        crate::app::gating_test_env::run_large_stack_test("admin-client-conditions", || async {
+            use crate::admin::router::Operation;
+            use crate::admin::runtime_sources::{AppContext, IamInterface, KmsInterface, publish_test_app_context};
+            use rustfs_iam::store::object::ObjectStore;
+            use rustfs_madmin::{AccountStatus, AddOrUpdateUserReq};
+            use std::sync::Arc;
+            struct ReadyIam(Arc<rustfs_iam::sys::IamSys<ObjectStore>>);
+            impl IamInterface for ReadyIam {
+                fn handle(&self) -> Arc<rustfs_iam::sys::IamSys<ObjectStore>> {
+                    self.0.clone()
+                }
+                fn is_ready(&self) -> bool {
+                    true
+                }
+            }
+            struct TestKms;
+            impl KmsInterface for TestKms {
+                fn handle(&self) -> Arc<rustfs_kms::KmsServiceManager> {
+                    Arc::new(rustfs_kms::KmsServiceManager::new())
+                }
+            }
+            let previous = crate::runtime_sources::current_app_context();
+            let (_directory, _paths, store) = crate::app::gating_test_env::isolated_multi_pool_ecstore().await;
+            ObjectStore::new(store.clone())
+                .save_iam_config(
+                    serde_json::json!({"version": 1}),
+                    format!("{}/format.json", *rustfs_iam::store::object::IAM_CONFIG_PREFIX),
+                )
+                .await
+                .unwrap();
+            let iam = rustfs_iam::build_iam_sys(store.clone()).await.unwrap();
+            for (key, document) in [
+                (
+                    "proxy-admin",
+                    serde_json::json!({"Version":"2012-10-17","Statement":[
+                        {"Effect":"Allow","Action":"admin:*"},
+                        {"Effect":"Allow","Action":"sts:AssumeRole"},
+                        {"Effect":"Deny","Action":"admin:*","Condition":{"NotIpAddress":{"aws:SourceIp":"10.0.0.0/8"}}},
+                        {"Effect":"Deny","Action":"sts:AssumeRole","Condition":{"NotIpAddress":{"aws:SourceIp":"10.0.0.0/8"}}}
+                    ]}),
+                ),
+                (
+                    "scoped-usage",
+                    serde_json::json!({"Version":"2012-10-17","Statement":[
+                        {"Effect":"Allow","Action":"s3:ListBucket","Resource":"arn:aws:s3:::usage-allowed"},
+                        {"Effect":"Deny","Action":"s3:ListBucket","NotResource":"arn:aws:s3:::usage-allowed"}
+                    ]}),
+                ),
+            ] {
+                iam.create_user(
+                    key,
+                    &AddOrUpdateUserReq {
+                        secret_key: "test-user-secret".to_owned(),
+                        policy: None,
+                        status: AccountStatus::Enabled,
+                    },
+                )
+                .await
+                .unwrap();
+                iam.set_policy(key, Policy::parse_config(&serde_json::to_vec(&document).unwrap()).unwrap())
+                    .await
+                    .unwrap();
+                iam.policy_db_set(key, UserType::Reg, false, key).await.unwrap();
+            }
+            let context = Arc::new(AppContext::new(store, Arc::new(ReadyIam(iam)), Arc::new(TestKms)));
+            assert!(context.publish_action_credentials(rustfs_credentials::Credentials {
+                access_key: "proxy-test-root".to_owned(),
+                secret_key: "proxy-root-secret".to_owned(),
+                status: "on".to_owned(),
+                ..Default::default()
+            }));
+            publish_test_app_context(context);
+            let request = |key: &str, client_ip: Option<&str>| {
+                let mut extensions = http::Extensions::new();
+                extensions.insert(Some(RemoteAddr("10.1.2.3:4567".parse().unwrap())));
+                if let Some(ip) = client_ip {
+                    extensions.insert(rustfs_trusted_proxies::ClientInfo::direct(format!("{ip}:7890").parse().unwrap()));
+                }
+                let mut headers = HeaderMap::new();
+                headers.insert("x-forwarded-for", "10.1.2.3".parse().unwrap());
+                headers.insert("x-real-ip", "10.1.2.3".parse().unwrap());
+                S3Request {
+                    input: Body::from(String::new()),
+                    method: http::Method::GET,
+                    uri: "/".parse().unwrap(),
+                    headers,
+                    extensions,
+                    credentials: Some(s3s::auth::Credentials {
+                        access_key: key.to_owned(),
+                        secret_key: s3s::auth::SecretKey::from("test-user-secret".to_owned()),
+                    }),
+                    region: None,
+                    service: None,
+                    trailing_headers: None,
+                }
+            };
+            for (client, allowed) in [(Some("203.0.113.17"), false), (Some("10.8.7.6"), true), (None, true)] {
+                let req = request("proxy-admin", client);
+                assert_eq!(policy_source_addr(&req.extensions).unwrap().port(), 4567);
+                let result = authorize_admin_request(&req, vec![admin_action()]).await;
+                if allowed {
+                    result.expect("verified private client is allowed");
+                } else {
+                    assert_eq!(result.unwrap_err().code(), &s3s::S3ErrorCode::AccessDenied);
+                }
+                let mut req = request("proxy-admin", client);
+                req.method = http::Method::POST;
+                req.input = Body::from("Action=AssumeRole&Version=invalid-version".to_owned());
+                let error = crate::admin::handlers::sts::AssumeRoleHandle {}
+                    .call(req, matchit::Params::new())
+                    .await
+                    .unwrap_err();
+                if allowed {
+                    assert_eq!(error.code(), &s3s::S3ErrorCode::InvalidArgument);
+                    assert_eq!(
+                        error.message(),
+                        Some("not support version"),
+                        "only an authorized caller reaches version validation"
+                    );
+                } else {
+                    assert_eq!(error.code(), &s3s::S3ErrorCode::AccessDenied);
+                }
+                let response = crate::admin::handlers::is_admin::IsAdminHandler {}
+                    .call(request("proxy-admin", client), matchit::Params::new())
+                    .await
+                    .unwrap();
+                let mut body = response.output.1;
+                let json: serde_json::Value = serde_json::from_slice(&body.store_all_limited(4096).await.unwrap()).unwrap();
+                assert_eq!(json["is_admin"], allowed);
+            }
+            let mut routes = matchit::Router::new();
+            routes.insert("/usage/{bucket}", ()).unwrap();
+            for (bucket, allowed) in [("usage-allowed", true), ("usage-secret", false)] {
+                let path = format!("/usage/{bucket}");
+                let matched = routes.at(&path).unwrap();
+                let mut req = request("scoped-usage", None);
+                req.uri = format!("{path}?unexpected=1").parse().unwrap();
+                let error = crate::admin::handlers::usage_prefix::BucketPrefixUsageHandler {}
+                    .call(req, matched.params)
+                    .await
+                    .unwrap_err();
+                if allowed {
+                    assert_eq!(error.code(), &s3s::S3ErrorCode::InvalidArgument);
+                    assert_eq!(error.message(), Some("unknown query parameter: unexpected"));
+                } else {
+                    assert_eq!(error.code(), &s3s::S3ErrorCode::AccessDenied);
+                }
+            }
+            if let Some(previous) = previous {
+                publish_test_app_context(previous);
+            }
+        });
+    }
+
     /// Owner (root/admin) credential is authorized for an admin action.
     #[tokio::test]
     async fn admin_owner_credential_is_allowed() {

rustfs/src/admin/handlers/account.rs

@@ -42,7 +42,6 @@ use crate::admin::service::caller_identity::{CallerIdentity, oidc_profile_fields
 use crate::admin::storage_api::s3::{self, Body, S3ErrorCode, S3Request, S3Response, S3Result};
 use crate::admin::utils::read_compatible_admin_body;
 use crate::auth::constant_time_eq;
-use crate::server::RemoteAddr;
 use http::StatusCode;
 use hyper::Method;
 use matchit::Params;
@@ -162,6 +161,13 @@ pub struct ChangeOwnPasswordHandler {}
 #[async_trait::async_trait]
 impl Operation for ChangeOwnPasswordHandler {
     async fn call(&self, req: S3Request<Body>, _params: Params<'_, '_>) -> S3Result<S3Response<(StatusCode, Body)>> {
+        let signing_secret = req
+            .credentials
+            .as_ref()
+            .ok_or_else(|| s3::error(S3ErrorCode::AccessDenied, "authentication required"))?
+            .secret_key
+            .expose()
+            .to_string();
         let caller = CallerIdentity::resolve(&req).await?;
         let audit = AccountAuditContext::from_request(&req);
 
@@ -203,7 +209,9 @@ impl Operation for ChangeOwnPasswordHandler {
             return Err(s3::error(S3ErrorCode::InvalidRequest, "the calling identity no longer exists"));
         };
 
-        if !constant_time_eq(&request.current_secret_key, &stored.credentials.secret_key) {
+        if !constant_time_eq(&request.current_secret_key, &stored.credentials.secret_key)
+            || !constant_time_eq(&request.current_secret_key, &signing_secret)
+        {
             warn!(
                 component = LOG_COMPONENT_ADMIN,
                 subsystem = LOG_SUBSYSTEM_ACCOUNT,
@@ -256,31 +264,16 @@ impl Operation for ChangeOwnPasswordHandler {
             let iam_store =
                 current_ready_iam_handle().map_err(|_| s3::error(S3ErrorCode::InternalError, "iam is not initialized"))?;
 
-            let (updated_at, status) = iam_store
-                .set_user_secret_key(&access_key, &new_secret_key)
+            let (updated_at, status, revoked) = iam_store
+                .change_own_secret_key(&access_key, &signing_secret, &new_secret_key)
                 .await
-                .map_err(iam_error_to_s3_error)?;
-
-            // Sessions minted under the old secret must not outlive it. A
-            // failure here is reported but does not undo the rotation: the new
-            // secret is already authoritative, and re-running the revocation is
-            // safe, whereas rolling the secret back would resurrect it.
-            let revoked = match iam_store.revoke_sts_sessions_for_parent(&access_key).await {
-                Ok(revoked) => revoked,
-                Err(err) => {
-                    warn!(
-                        component = LOG_COMPONENT_ADMIN,
-                        subsystem = LOG_SUBSYSTEM_ACCOUNT,
-                        event = EVENT_ADMIN_ACCOUNT_STATE,
-                        action = "change_own_password",
-                        access_key = %MaskedAccessKey(&access_key),
-                        result = "session_revocation_incomplete",
-                        error = ?err,
-                        "admin account state"
-                    );
-                    0
-                }
-            };
+                .map_err(|error| {
+                    if matches!(error, rustfs_iam::error::Error::IAMActionNotAllowed) {
+                        s3::error(S3ErrorCode::InvalidRequest, "the current secret key is incorrect")
+                    } else {
+                        iam_error_to_s3_error(error)
+                    }
+                })?;
 
             emit_audit(
                 &audit_for_task,
@@ -378,7 +371,7 @@ impl Operation for SetUserSecretKeyHandler {
             caller.is_owner,
             false,
             vec![Action::AdminAction(AdminAction::CreateUserAdminAction)],
-            req.extensions.get::<Option<RemoteAddr>>().and_then(|opt| opt.map(|a| a.0)),
+            crate::admin::auth::policy_source_addr(&req.extensions),
         )
         .await
         .inspect_err(|_| {
@@ -424,28 +417,11 @@ impl Operation for SetUserSecretKeyHandler {
                 ));
             }
 
-            let (updated_at, status) = iam_store
-                .set_user_secret_key(&target, &request.secret_key)
+            let (updated_at, status, revoked) = iam_store
+                .rotate_user_secret_key(&target, &request.secret_key)
                 .await
                 .map_err(iam_error_to_s3_error)?;
 
-            let revoked = match iam_store.revoke_sts_sessions_for_parent(&target).await {
-                Ok(revoked) => revoked,
-                Err(err) => {
-                    warn!(
-                        component = LOG_COMPONENT_ADMIN,
-                        subsystem = LOG_SUBSYSTEM_ACCOUNT,
-                        event = EVENT_ADMIN_ACCOUNT_STATE,
-                        action = "set_user_secret_key",
-                        access_key = %MaskedAccessKey(&target),
-                        result = "session_revocation_incomplete",
-                        error = ?err,
-                        "admin account state"
-                    );
-                    0
-                }
-            };
-
             emit_audit(
                 &audit_for_task,
                 AccountAuditRecord::success(AccountAuditOperation::ResetUserPassword, &target, identity_type)
@@ -648,16 +624,23 @@ mod tests {
     #[test]
     fn both_rotation_handlers_broadcast_after_revoking_sessions() {
         let src = include_str!("account.rs");
-        for marker in [
-            "impl Operation for ChangeOwnPasswordHandler",
-            "impl Operation for SetUserSecretKeyHandler",
+        // Session revocation now runs inside the IAM rotation call, under the
+        // per-key mutation lock, so the handler must reach that call before it
+        // broadcasts the new secret to peer sites.
+        for (marker, rotation) in [
+            ("impl Operation for ChangeOwnPasswordHandler", "change_own_secret_key("),
+            ("impl Operation for SetUserSecretKeyHandler", "rotate_user_secret_key("),
         ] {
             let start = src.find(marker).expect("handler should exist");
             let block = &src[start..];
             let block = &block[..block.find("\n}\n").expect("handler block end")];
             let revoke = block
-                .find("revoke_sts_sessions_for_parent")
-                .expect("handler must revoke sessions");
+                .find(rotation)
+                .expect("handler must rotate through the IAM call that revokes sessions");
+            assert!(
+                !block.contains("revoke_sts_sessions_for_parent"),
+                "{marker}: revocation belongs to the locked IAM rotation, not a separate handler step"
+            );
             let broadcast = block
                 .find("broadcast_secret_key_rotation(")
                 .expect("handler must broadcast the rotation to peer sites");

rustfs/src/admin/handlers/account_info.rs

@@ -22,7 +22,7 @@ use crate::admin::storage_api::contract::bucket::{BucketOperations, BucketOption
 use crate::admin::storage_api::data_usage::{apply_bucket_usage_memory_overlay, load_admin_data_usage_from_backend_cached};
 use crate::admin::storage_api::metadata_sys;
 use crate::auth::get_condition_values;
-use crate::server::{ADMIN_PREFIX, RemoteAddr};
+use crate::server::ADMIN_PREFIX;
 use http::{HeaderMap, HeaderValue};
 use hyper::{Method, StatusCode};
 use matchit::Params;
@@ -146,7 +146,7 @@ impl Operation for AccountInfoHandler {
         let claims = cred.claims.as_ref().unwrap_or(&default_claims);
 
         let cred_clone = cred.clone();
-        let remote_addr = req.extensions.get::<Option<RemoteAddr>>().and_then(|opt| opt.map(|a| a.0));
+        let remote_addr = crate::admin::auth::policy_source_addr(&req.extensions);
         let conditions = get_condition_values(&req.headers, &cred_clone, None, None, remote_addr);
         let cred_clone = Arc::new(cred_clone);
         let conditions = Arc::new(conditions);

rustfs/src/admin/handlers/durability.rs

@@ -28,7 +28,6 @@ use crate::admin::storage_api::bucket::metadata::BUCKET_DURABILITY_CONFIG;
 use crate::admin::storage_api::bucket::metadata_sys;
 use crate::auth::{check_key_valid, get_session_token};
 use crate::server::ADMIN_PREFIX;
-use crate::server::RemoteAddr;
 use hyper::{Method, StatusCode};
 use matchit::Params;
 use rustfs_policy::policy::action::{Action, AdminAction};
@@ -127,7 +126,7 @@ async fn authenticate_admin(req: &S3Request<Body>) -> S3Result<()> {
 
     let (cred, owner) = check_key_valid(get_session_token(&req.uri, &req.headers).unwrap_or_default(), &cred.access_key).await?;
 
-    let remote_addr = req.extensions.get::<Option<RemoteAddr>>().and_then(|opt| opt.map(|a| a.0));
+    let remote_addr = crate::admin::auth::policy_source_addr(&req.extensions);
     validate_admin_request(
         &req.headers,
         &cred,

rustfs/src/admin/handlers/group.rs

@@ -434,7 +434,7 @@ impl Operation for SetGroupStatus {
 pub struct UpdateGroupMembers {}
 #[async_trait::async_trait]
 impl Operation for UpdateGroupMembers {
-    async fn call(&self, req: S3Request<Body>, _params: Params<'_, '_>) -> S3Result<S3Response<(StatusCode, Body)>> {
+    async fn call(&self, mut req: S3Request<Body>, _params: Params<'_, '_>) -> S3Result<S3Response<(StatusCode, Body)>> {
         warn!(
             event = EVENT_ADMIN_GROUP_STATE,
             component = LOG_COMPONENT_ADMIN,
@@ -448,9 +448,16 @@ impl Operation for UpdateGroupMembers {
             return Err(s3_error!(InvalidRequest, "authentication required"));
         }
 
-        authorize_admin_request(&req, vec![Action::AdminAction(AdminAction::AddUserToGroupAdminAction)]).await?;
+        authorize_admin_request(
+            &req,
+            vec![
+                Action::AdminAction(AdminAction::AddUserToGroupAdminAction),
+                Action::AdminAction(AdminAction::RemoveUserFromGroupAdminAction),
+            ],
+        )
+        .await?;
 
-        let mut input = req.input;
+        let mut input = std::mem::take(&mut req.input);
         let body = match input.store_all_limited(MAX_ADMIN_REQUEST_BODY_SIZE).await {
             Ok(b) => b,
             Err(e) => {
@@ -470,6 +477,13 @@ impl Operation for UpdateGroupMembers {
         let args: GroupAddRemove = serde_json::from_slice(&body)
             .map_err(|e| S3Error::with_message(S3ErrorCode::InvalidRequest, format!("invalid JSON: {e}")))?;
 
+        let action = if args.is_remove {
+            AdminAction::RemoveUserFromGroupAdminAction
+        } else {
+            AdminAction::AddUserToGroupAdminAction
+        };
+        authorize_admin_request(&req, vec![Action::AdminAction(action)]).await?;
+
         warn!(
             event = EVENT_ADMIN_GROUP_STATE,
             component = LOG_COMPONENT_ADMIN,
@@ -719,6 +733,143 @@ mod tests {
         &block[..end]
     }
 
+    #[tokio::test]
+    #[serial_test::serial]
+    async fn group_members_handler_enforces_add_and_remove_permissions() {
+        use crate::admin::runtime_sources::{AppContext, IamInterface, KmsInterface, publish_test_app_context};
+        use rustfs_iam::store::{Store as _, UserType, object::ObjectStore};
+        use rustfs_madmin::{AccountStatus, AddOrUpdateUserReq};
+        use rustfs_policy::policy::Policy;
+        use std::sync::Arc;
+
+        struct ReadyIam(Arc<rustfs_iam::sys::IamSys<ObjectStore>>);
+        impl IamInterface for ReadyIam {
+            fn handle(&self) -> Arc<rustfs_iam::sys::IamSys<ObjectStore>> {
+                Arc::clone(&self.0)
+            }
+            fn is_ready(&self) -> bool {
+                true
+            }
+        }
+        struct TestKms;
+        impl KmsInterface for TestKms {
+            fn handle(&self) -> Arc<rustfs_kms::KmsServiceManager> {
+                Arc::new(rustfs_kms::KmsServiceManager::new())
+            }
+        }
+
+        let (_directory, _paths, store) = crate::app::gating_test_env::isolated_multi_pool_ecstore().await;
+        ObjectStore::new(store.clone())
+            .save_iam_config(
+                serde_json::json!({"version": 1}),
+                format!("{}/format.json", *rustfs_iam::store::object::IAM_CONFIG_PREFIX),
+            )
+            .await
+            .expect("seed IAM format");
+        let iam = rustfs_iam::build_iam_sys(store.clone()).await.expect("initialize IAM");
+        for key in ["group-adder", "group-remover", "group-member"] {
+            iam.create_user(
+                key,
+                &AddOrUpdateUserReq {
+                    secret_key: "group-user-secret".to_owned(),
+                    policy: None,
+                    status: AccountStatus::Enabled,
+                },
+            )
+            .await
+            .expect("create group test user");
+        }
+        for (key, allow, deny) in [
+            ("group-adder", "admin:AddUserToGroup", "admin:RemoveUserFromGroup"),
+            ("group-remover", "admin:RemoveUserFromGroup", "admin:AddUserToGroup"),
+        ] {
+            let document = serde_json::json!({"Version": "2012-10-17", "Statement": [
+                {"Effect": "Allow", "Action": [allow]}, {"Effect": "Deny", "Action": [deny]}
+            ]});
+            iam.set_policy(
+                key,
+                Policy::parse_config(&serde_json::to_vec(&document).expect("policy JSON")).expect("parse policy"),
+            )
+            .await
+            .expect("store delegated policy");
+            iam.policy_db_set(key, UserType::Reg, false, key)
+                .await
+                .expect("attach delegated policy");
+        }
+        iam.add_users_to_group("guardrail", vec!["group-member".to_owned()])
+            .await
+            .expect("seed protected group");
+        let context = Arc::new(AppContext::new(store, Arc::new(ReadyIam(Arc::clone(&iam))), Arc::new(TestKms)));
+        assert!(context.publish_action_credentials(rustfs_credentials::Credentials {
+            access_key: "group-test-root".to_owned(),
+            secret_key: "group-root-secret".to_owned(),
+            status: "on".to_owned(),
+            ..Default::default()
+        }));
+        publish_test_app_context(context);
+
+        for (key, remove, allowed) in [
+            ("group-adder", true, false),
+            ("group-remover", false, false),
+            ("group-remover", true, true),
+            ("group-adder", false, true),
+        ] {
+            let before = iam
+                .get_group_description("guardrail")
+                .await
+                .expect("read original membership")
+                .members;
+            let mut req = credential_less_request(Method::PUT, "/rustfs/admin/v3/update-group-members");
+            req.credentials = Some(s3s::auth::Credentials {
+                access_key: key.to_owned(),
+                secret_key: s3s::auth::SecretKey::from("group-user-secret".to_owned()),
+            });
+            req.input = Body::from(
+                serde_json::to_vec(&serde_json::json!({
+                    "group": "guardrail", "members": ["group-member"], "groupStatus": "enabled", "isRemove": remove
+                }))
+                .expect("group update JSON"),
+            );
+            let result = UpdateGroupMembers {}.call(req, Params::new()).await;
+            let after = iam
+                .get_group_description("guardrail")
+                .await
+                .expect("read resulting membership")
+                .members;
+            if allowed {
+                result.expect("matching delegated action permits the mutation");
+                assert_eq!(after.contains(&"group-member".to_owned()), !remove);
+            } else {
+                let error = result.expect_err("the opposite action must not authorize the mutation");
+                assert_eq!(error.code(), &S3ErrorCode::AccessDenied);
+                assert_eq!(after, before, "a rejected mutation must preserve the group");
+            }
+        }
+    }
+
+    #[test]
+    fn update_group_members_authorizes_the_decoded_mutation() {
+        let source = include_str!("group.rs")
+            .split("\n#[cfg(test)]\n")
+            .next()
+            .expect("production source");
+        let handler = source_block(source, "impl Operation for UpdateGroupMembers");
+        assert_eq!(handler.matches("authorize_admin_request(").count(), 2);
+        let decoded = handler
+            .find("let args: GroupAddRemove")
+            .expect("decode request before choosing action");
+        let action = handler
+            .find("let action = if args.is_remove")
+            .expect("body controls requested action");
+        let gate = handler.rfind("authorize_admin_request(").expect("mutation authorization");
+        assert!(decoded < action && action < gate);
+        assert!(handler[action..gate].contains("AdminAction::RemoveUserFromGroupAdminAction"));
+        assert!(handler[action..gate].contains("AdminAction::AddUserToGroupAdminAction"));
+        for mutation in [".remove_users_from_group(", ".add_users_to_group("] {
+            assert!(gate < handler.find(mutation).expect("group mutation"));
+        }
+    }
+
     #[test]
     fn group_handlers_use_the_shared_admin_gate_with_their_actions() {
         let production = include_str!("group.rs")
@@ -731,7 +882,6 @@ mod tests {
             ("GetGroup", "GetGroupAdminAction"),
             ("DeleteGroup", "RemoveUserFromGroupAdminAction"),
             ("SetGroupStatus", "EnableGroupAdminAction"),
-            ("UpdateGroupMembers", "AddUserToGroupAdminAction"),
         ] {
             let block = source_block(production, &format!("impl Operation for {handler}"));
             assert_eq!(

rustfs/src/admin/handlers/heal.rs

@@ -1474,7 +1474,7 @@ async fn authenticate_heal_admin_request(req: &S3Request<Body>) -> S3Result<Stri
         owner,
         false,
         vec![Action::AdminAction(AdminAction::HealAdminAction)],
-        req.extensions.get::<Option<RemoteAddr>>().and_then(|opt| opt.map(|a| a.0)),
+        crate::admin::auth::policy_source_addr(&req.extensions),
     )
     .await?;
     Ok(cred.access_key)

rustfs/src/admin/handlers/idp_compat.rs

@@ -48,7 +48,7 @@ use crate::admin::router::{AdminOperation, Operation, S3Router};
 use crate::admin::runtime_sources::{current_app_context, current_ready_iam_handle, current_server_config_for_context};
 use crate::admin::utils::is_compat_admin_request;
 use crate::auth::{check_key_valid, get_session_token};
-use crate::server::{ADMIN_PREFIX, RemoteAddr};
+use crate::server::ADMIN_PREFIX;
 use http::StatusCode;
 use hyper::Method;
 use matchit::Params;
@@ -981,7 +981,7 @@ async fn authorize_action(req: &S3Request<Body>, cred: &StoredCredentials, owner
         owner,
         false,
         vec![Action::AdminAction(action)],
-        req.extensions.get::<Option<RemoteAddr>>().and_then(|opt| opt.map(|a| a.0)),
+        crate::admin::auth::policy_source_addr(&req.extensions),
     )
     .await
 }

rustfs/src/admin/handlers/is_admin.rs

@@ -66,7 +66,8 @@ impl Operation for IsAdminHandler {
             let empty_claims = HashMap::new();
             let iam_store = crate::admin::runtime_sources::current_ready_iam_handle()
                 .map_err(|_| s3_error!(InternalError, "iam not init"))?;
-            let conditions = get_condition_values(&req.headers, &cred, None, None, None);
+            let conditions =
+                get_condition_values(&req.headers, &cred, None, None, crate::admin::auth::policy_source_addr(&req.extensions));
             iam_store
                 .is_allowed(&Args {
                     account: &cred.access_key,

rustfs/src/admin/handlers/kms_backup.rs

@@ -46,7 +46,7 @@ use crate::admin::router::{AdminOperation, Operation, S3Router};
 use crate::admin::runtime_sources::{current_deployment_id, current_kms_runtime_service_manager};
 use crate::admin::utils::json_response;
 use crate::auth::{check_key_valid, get_session_token};
-use crate::server::{ADMIN_PREFIX, RemoteAddr};
+use crate::server::ADMIN_PREFIX;
 use base64_simd::STANDARD as BASE64;
 use hyper::{Method, StatusCode};
 use matchit::Params;
@@ -923,7 +923,7 @@ async fn authorize(
             owner,
             false,
             actions,
-            req.extensions.get::<Option<RemoteAddr>>().and_then(|opt| opt.map(|a| a.0)),
+            crate::admin::auth::policy_source_addr(&req.extensions),
         )
         .await,
         operation,

rustfs/src/admin/handlers/kms_dynamic.rs

@@ -28,7 +28,7 @@ use crate::admin::storage_api::error::StorageError;
 use crate::admin::storage_api::runtime::ECStore;
 use crate::admin::storage_api::s3::{S3ErrorCode, error as admin_s3_error};
 use crate::auth::{check_key_valid, get_session_token};
-use crate::server::{ADMIN_PREFIX, RemoteAddr};
+use crate::server::ADMIN_PREFIX;
 use hyper::{Method, StatusCode};
 use matchit::Params;
 use rustfs_config::MAX_ADMIN_REQUEST_BODY_SIZE;
@@ -97,6 +97,118 @@ fn kms_service_control_actions() -> Vec<Action> {
     vec![Action::KmsAction(KmsAction::ServiceControlAction)]
 }
 
+// A remote KMS administrator may reuse operator-provisioned files, but cannot
+// select new files or redirect the credentials read from them.
+struct VaultConnection<'a> {
+    address: &'a str,
+    namespace: Option<&'a str>,
+    auth: &'a rustfs_kms::VaultAuthMethod,
+    tls: Option<&'a rustfs_kms::config::TlsConfig>,
+}
+
+fn vault_connection(config: &KmsConfig) -> Option<VaultConnection<'_>> {
+    match &config.backend_config {
+        rustfs_kms::BackendConfig::VaultKv2(vault) => Some(VaultConnection {
+            address: &vault.address,
+            namespace: vault.namespace.as_deref(),
+            auth: &vault.auth_method,
+            tls: vault.tls.as_ref(),
+        }),
+        rustfs_kms::BackendConfig::VaultTransit(vault) => Some(VaultConnection {
+            address: &vault.address,
+            namespace: vault.namespace.as_deref(),
+            auth: &vault.auth_method,
+            tls: vault.tls.as_ref(),
+        }),
+        _ => None,
+    }
+}
+
+fn vault_auth_uses_file(auth: &rustfs_kms::VaultAuthMethod) -> bool {
+    use rustfs_kms::VaultAuthMethod;
+    matches!(
+        auth,
+        VaultAuthMethod::AppRole {
+            secret_id_file: Some(_),
+            ..
+        } | VaultAuthMethod::Kubernetes { .. }
+            | VaultAuthMethod::TokenFile { .. }
+    )
+}
+
+fn same_vault_file_auth(requested: &rustfs_kms::VaultAuthMethod, existing: &rustfs_kms::VaultAuthMethod) -> bool {
+    use rustfs_kms::VaultAuthMethod;
+    match (requested, existing) {
+        (
+            VaultAuthMethod::AppRole {
+                role_id,
+                secret_id_file,
+                mount,
+                ..
+            },
+            VaultAuthMethod::AppRole {
+                role_id: old_role,
+                secret_id_file: old_file,
+                mount: old_mount,
+                ..
+            },
+        ) => role_id == old_role && secret_id_file == old_file && mount == old_mount,
+        (
+            VaultAuthMethod::Kubernetes {
+                role, jwt_path, mount, ..
+            },
+            VaultAuthMethod::Kubernetes {
+                role: old_role,
+                jwt_path: old_path,
+                mount: old_mount,
+                ..
+            },
+        ) => role == old_role && jwt_path == old_path && mount == old_mount,
+        (VaultAuthMethod::TokenFile { path, .. }, VaultAuthMethod::TokenFile { path: old_path, .. }) => path == old_path,
+        _ => false,
+    }
+}
+
+fn validate_vault_local_file_access(
+    requested: &KmsConfig,
+    existing: Option<&KmsConfig>,
+    inherited_auth: bool,
+) -> Result<(), String> {
+    let Some(requested) = vault_connection(requested) else {
+        return Ok(());
+    };
+    let auth_file = vault_auth_uses_file(requested.auth);
+    let tls_files = requested
+        .tls
+        .is_some_and(|tls| tls.ca_cert_path.is_some() || tls.client_cert_path.is_some() || tls.client_key_path.is_some());
+    if !auth_file && !tls_files && !inherited_auth {
+        return Ok(());
+    }
+    let error = "Vault server-local files and reused credentials must retain their operator-configured connection";
+    let existing = existing.and_then(vault_connection).ok_or_else(|| error.to_string())?;
+    if requested.address != existing.address || requested.namespace != existing.namespace {
+        return Err(error.to_string());
+    }
+    if auth_file && !same_vault_file_auth(requested.auth, existing.auth) {
+        return Err(error.to_string());
+    }
+    let tls_settings = |tls: Option<&rustfs_kms::config::TlsConfig>| {
+        tls.map(|tls| {
+            (
+                tls.ca_cert_path.clone(),
+                tls.client_cert_path.clone(),
+                tls.client_key_path.clone(),
+                tls.skip_verify,
+            )
+        })
+        .unwrap_or_default()
+    };
+    if tls_settings(requested.tls) != tls_settings(existing.tls) {
+        return Err(error.to_string());
+    }
+    Ok(())
+}
+
 fn normalize_configure_request_secrets(
     request: &mut ConfigureKmsRequest,
     existing_config: Option<&KmsConfig>,
@@ -142,7 +254,7 @@ fn normalize_configure_request_secrets(
     };
 
     if !needs_existing_auth {
-        return Ok(());
+        return validate_vault_local_file_access(&request.to_kms_config(), existing_config, false);
     }
 
     let existing_auth = existing_config
@@ -157,7 +269,7 @@ fn normalize_configure_request_secrets(
         ConfigureKmsRequest::Aws(_) => {}
     }
 
-    Ok(())
+    validate_vault_local_file_access(&request.to_kms_config(), existing_config, true)
 }
 
 fn ensure_kms_config_persistable(config: &KmsConfig) -> Result<(), String> {
@@ -747,7 +859,7 @@ impl Operation for ConfigureKmsHandler {
                 owner,
                 false,
                 kms_configure_actions(),
-                req.extensions.get::<Option<RemoteAddr>>().and_then(|opt| opt.map(|a| a.0)),
+                crate::admin::auth::policy_source_addr(&req.extensions),
             )
             .await,
             KmsAdminOperation::Configure,
@@ -900,7 +1012,7 @@ impl Operation for StartKmsHandler {
                 owner,
                 false,
                 kms_service_control_actions(),
-                req.extensions.get::<Option<RemoteAddr>>().and_then(|opt| opt.map(|a| a.0)),
+                crate::admin::auth::policy_source_addr(&req.extensions),
             )
             .await,
             KmsAdminOperation::Start,
@@ -1057,7 +1169,7 @@ impl Operation for StopKmsHandler {
                 owner,
                 false,
                 kms_service_control_actions(),
-                req.extensions.get::<Option<RemoteAddr>>().and_then(|opt| opt.map(|a| a.0)),
+                crate::admin::auth::policy_source_addr(&req.extensions),
             )
             .await,
             KmsAdminOperation::Stop,
@@ -1159,7 +1271,7 @@ impl Operation for ReloadKmsHandler {
                 owner,
                 false,
                 kms_service_control_actions(),
-                req.extensions.get::<Option<RemoteAddr>>().and_then(|opt| opt.map(|a| a.0)),
+                crate::admin::auth::policy_source_addr(&req.extensions),
             )
             .await,
             KmsAdminOperation::Reload,
@@ -1253,7 +1365,7 @@ impl Operation for GetKmsStatusHandler {
             owner,
             false,
             kms_service_control_actions(),
-            req.extensions.get::<Option<RemoteAddr>>().and_then(|opt| opt.map(|a| a.0)),
+            crate::admin::auth::policy_source_addr(&req.extensions),
         )
         .await?;
 
@@ -1347,7 +1459,7 @@ impl Operation for ReconfigureKmsHandler {
                 owner,
                 false,
                 kms_configure_actions(),
-                req.extensions.get::<Option<RemoteAddr>>().and_then(|opt| opt.map(|a| a.0)),
+                crate::admin::auth::policy_source_addr(&req.extensions),
             )
             .await,
             KmsAdminOperation::Reconfigure,
@@ -1954,6 +2066,147 @@ mod tests {
         }
     }
 
+    fn vault_request(backend: &str, auth: serde_json::Value) -> serde_json::Value {
+        serde_json::json!({
+            "backend_type": backend,
+            "address": "https://vault.internal:8200",
+            "auth_method": auth,
+        })
+    }
+
+    fn decode_vault_request(value: serde_json::Value) -> rustfs_kms::ConfigureKmsRequest {
+        serde_json::from_value(value).expect("valid Vault configure request")
+    }
+
+    fn vault_file_auth_methods() -> [serde_json::Value; 3] {
+        [
+            serde_json::json!({"TokenFile": {"path": "/etc/rustfs/vault-token"}}),
+            serde_json::json!({"AppRole": {
+                "role_id": "rustfs", "secret_id": "", "secret_id_file": "/etc/rustfs/secret-id"
+            }}),
+            serde_json::json!({"Kubernetes": {"role": "rustfs", "jwt_path": "/etc/rustfs/service-account"}}),
+        ]
+    }
+
+    #[test]
+    fn vault_local_files_require_existing_operator_configuration() {
+        for backend in ["VaultKV2", "VaultTransit"] {
+            for auth in vault_file_auth_methods() {
+                let mut request = decode_vault_request(vault_request(backend, auth));
+                normalize_configure_request_secrets(&mut request, None)
+                    .expect_err("remote requests cannot bootstrap server-local credential reads");
+                let existing = vault_config("https://vault.internal:8200", VAULT_TOKEN);
+                normalize_configure_request_secrets(&mut request, Some(&existing))
+                    .expect_err("inline-token configuration grants no local-file access");
+            }
+            // Omitting jwt_path selects the pod's projected token, which is
+            // still a server-local credential and must not bypass the guard.
+            let mut request = decode_vault_request(vault_request(backend, serde_json::json!({"Kubernetes": {"role": "rustfs"}})));
+            normalize_configure_request_secrets(&mut request, None)
+                .expect_err("default Kubernetes path also needs operator configuration");
+        }
+    }
+
+    #[test]
+    fn vault_local_files_allow_unchanged_connections_but_reject_redirects() {
+        for backend in ["VaultKV2", "VaultTransit"] {
+            for auth in vault_file_auth_methods() {
+                let value = vault_request(backend, auth);
+                let existing = decode_vault_request(value.clone()).to_kms_config();
+                let mut request = decode_vault_request(value.clone());
+                normalize_configure_request_secrets(&mut request, Some(&existing))
+                    .expect("existing operator-provisioned credentials remain usable");
+
+                for (field, replacement) in [
+                    ("address", serde_json::json!("https://attacker.invalid:8200")),
+                    ("namespace", serde_json::json!("attacker")),
+                    ("skip_tls_verify", serde_json::json!(true)),
+                    ("ca_cert_path", serde_json::json!("/tmp/attacker-ca.pem")),
+                ] {
+                    let mut changed = value.clone();
+                    changed[field] = replacement;
+                    normalize_configure_request_secrets(&mut decode_vault_request(changed), Some(&existing))
+                        .expect_err("file credentials cannot be redirected or lose their trust boundary");
+                }
+            }
+        }
+    }
+
+    #[test]
+    fn vault_local_files_reject_changed_auth_paths_and_routes() {
+        for backend in ["VaultKV2", "VaultTransit"] {
+            for (auth, variant, path) in [
+                (vault_file_auth_methods()[0].clone(), "TokenFile", "path"),
+                (vault_file_auth_methods()[1].clone(), "AppRole", "secret_id_file"),
+                (vault_file_auth_methods()[2].clone(), "Kubernetes", "jwt_path"),
+            ] {
+                let value = vault_request(backend, auth);
+                let existing = decode_vault_request(value.clone()).to_kms_config();
+                let mut changed = value.clone();
+                changed["auth_method"][variant][path] = serde_json::json!("/proc/self/environ");
+                normalize_configure_request_secrets(&mut decode_vault_request(changed), Some(&existing))
+                    .expect_err("the request cannot select an arbitrary server file");
+                if variant != "TokenFile" {
+                    let mut changed = value.clone();
+                    changed["auth_method"][variant]["mount"] = serde_json::json!("attacker-auth");
+                    normalize_configure_request_secrets(&mut decode_vault_request(changed), Some(&existing))
+                        .expect_err("file contents cannot be sent to a different authentication mount");
+                }
+            }
+        }
+    }
+
+    #[test]
+    fn vault_local_files_reject_new_or_repurposed_tls_paths() {
+        for backend in ["VaultKV2", "VaultTransit"] {
+            for field in ["ca_cert_path", "client_cert_path", "client_key_path"] {
+                let mut value = vault_request(backend, serde_json::json!({"Token": {"token": "explicit-token"}}));
+                value[field] = serde_json::json!("/etc/rustfs/operator.pem");
+                let mut request = decode_vault_request(value.clone());
+                normalize_configure_request_secrets(&mut request, None).expect_err("TLS paths cannot be provisioned remotely");
+                let existing = request.to_kms_config();
+                normalize_configure_request_secrets(&mut request, Some(&existing)).expect("unchanged TLS paths remain usable");
+                value[field] = serde_json::json!("/proc/self/environ");
+                normalize_configure_request_secrets(&mut decode_vault_request(value), Some(&existing))
+                    .expect_err("TLS paths cannot be changed remotely");
+            }
+        }
+    }
+
+    #[test]
+    fn vault_local_files_validate_inherited_auth_after_normalization() {
+        for backend in ["VaultKV2", "VaultTransit"] {
+            for auth in vault_file_auth_methods()
+                .into_iter()
+                .chain([serde_json::json!({"Token": {"token": VAULT_TOKEN}})])
+            {
+                let existing = decode_vault_request(vault_request(backend, auth)).to_kms_config();
+                let value = vault_request(backend, serde_json::json!({"Token": {"token": "  "}}));
+                let mut request = decode_vault_request(value.clone());
+                normalize_configure_request_secrets(&mut request, Some(&existing))
+                    .expect("blank token preserves credentials for the same connection");
+                let mut changed = value;
+                changed["address"] = serde_json::json!("http://attacker.invalid:8200");
+                normalize_configure_request_secrets(&mut decode_vault_request(changed), Some(&existing))
+                    .expect_err("blank token cannot forward existing credentials elsewhere");
+            }
+        }
+    }
+
+    #[test]
+    fn vault_local_files_allow_explicit_credentials_without_files() {
+        for backend in ["VaultKV2", "VaultTransit"] {
+            for auth in [
+                serde_json::json!({"Token": {"token": "explicit-token"}}),
+                serde_json::json!({"AppRole": {"role_id": "role", "secret_id": "explicit-secret"}}),
+            ] {
+                let mut request = decode_vault_request(vault_request(backend, auth));
+                normalize_configure_request_secrets(&mut request, None)
+                    .expect("explicit credentials need no server-local access");
+            }
+        }
+    }
+
     #[test]
     fn config_fingerprint_input_carries_no_credentials() {
         let canonical = redacted_canonical_config(&vault_config("https://vault.internal:8200", VAULT_TOKEN))

rustfs/src/admin/handlers/kms_key_lifecycle.rs

@@ -21,7 +21,7 @@ use crate::admin::router::{AdminOperation, Operation, S3Router};
 use crate::admin::runtime_sources::current_kms_runtime_service_manager;
 use crate::admin::utils::{extract_query_params, json_response};
 use crate::auth::{check_key_valid, get_session_token};
-use crate::server::{ADMIN_PREFIX, RemoteAddr};
+use crate::server::ADMIN_PREFIX;
 use hyper::{Method, StatusCode};
 use matchit::Params;
 use rustfs_config::MAX_ADMIN_REQUEST_BODY_SIZE;
@@ -270,7 +270,7 @@ async fn handle_lifecycle_request(
             owner,
             false,
             actions,
-            req.extensions.get::<Option<RemoteAddr>>().and_then(|opt| opt.map(|a| a.0)),
+            crate::admin::auth::policy_source_addr(&req.extensions),
             scoped.as_deref().unwrap_or_default(),
         )
         .await,

rustfs/src/admin/handlers/kms_key_metadata.rs

@@ -25,7 +25,7 @@ use crate::admin::router::{AdminOperation, Operation, S3Router};
 use crate::admin::runtime_sources::current_kms_runtime_service_manager;
 use crate::admin::utils::json_response;
 use crate::auth::{check_key_valid, get_session_token};
-use crate::server::{ADMIN_PREFIX, RemoteAddr};
+use crate::server::ADMIN_PREFIX;
 use hyper::{Method, StatusCode};
 use matchit::Params;
 use rustfs_config::MAX_ADMIN_REQUEST_BODY_SIZE;
@@ -327,7 +327,7 @@ async fn handle_metadata_request(
             owner,
             false,
             actions,
-            req.extensions.get::<Option<RemoteAddr>>().and_then(|opt| opt.map(|a| a.0)),
+            crate::admin::auth::policy_source_addr(&req.extensions),
             scoped.as_deref().unwrap_or_default(),
         )
         .await,

rustfs/src/admin/handlers/kms_keys.rs

@@ -22,7 +22,7 @@ use crate::admin::storage_api::s3;
 use crate::admin::utils::extract_query_params;
 use crate::auth::{check_key_valid, get_session_token};
 use crate::kms_deletion_gate::current_key_impact;
-use crate::server::{ADMIN_PREFIX, RemoteAddr};
+use crate::server::ADMIN_PREFIX;
 use hyper::{HeaderMap, Method, StatusCode};
 use matchit::Params;
 use rustfs_config::MAX_ADMIN_REQUEST_BODY_SIZE;
@@ -325,7 +325,7 @@ impl Operation for CreateKeyHandler {
                 owner,
                 false,
                 kms_create_key_actions(),
-                req.extensions.get::<Option<RemoteAddr>>().and_then(|opt| opt.map(|a| a.0)),
+                crate::admin::auth::policy_source_addr(&req.extensions),
             )
             .await,
             KmsAuditOperation::CreateKey,
@@ -418,7 +418,7 @@ impl Operation for DescribeKeyHandler {
                 owner,
                 false,
                 kms_describe_key_actions(),
-                req.extensions.get::<Option<RemoteAddr>>().and_then(|opt| opt.map(|a| a.0)),
+                crate::admin::auth::policy_source_addr(&req.extensions),
                 requested_key_id.as_deref().unwrap_or_default(),
             )
             .await,
@@ -1488,7 +1488,7 @@ impl Operation for ListKeysHandler {
                 owner,
                 false,
                 kms_list_keys_actions(),
-                req.extensions.get::<Option<RemoteAddr>>().and_then(|opt| opt.map(|a| a.0)),
+                crate::admin::auth::policy_source_addr(&req.extensions),
             )
             .await,
             KmsAuditOperation::ListKeys,
@@ -1583,7 +1583,7 @@ impl Operation for GenerateDataKeyHandler {
                 owner,
                 false,
                 kms_generate_data_key_actions(),
-                req.extensions.get::<Option<RemoteAddr>>().and_then(|opt| opt.map(|a| a.0)),
+                crate::admin::auth::policy_source_addr(&req.extensions),
                 parsed.as_ref().map(|request| request.key_id.as_str()).unwrap_or_default(),
             )
             .await,
@@ -1664,7 +1664,7 @@ impl Operation for CreateKmsKeyHandler {
                 owner,
                 false,
                 kms_create_key_actions(),
-                req.extensions.get::<Option<RemoteAddr>>().and_then(|opt| opt.map(|a| a.0)),
+                crate::admin::auth::policy_source_addr(&req.extensions),
             )
             .await,
             KmsAuditOperation::CreateKey,
@@ -1943,7 +1943,7 @@ impl Operation for DeleteKmsKeyHandler {
                 owner,
                 false,
                 kms_delete_key_actions(),
-                req.extensions.get::<Option<RemoteAddr>>().and_then(|opt| opt.map(|a| a.0)),
+                crate::admin::auth::policy_source_addr(&req.extensions),
                 scoped.as_deref().unwrap_or_default(),
             )
             .await,
@@ -2111,7 +2111,7 @@ impl Operation for CancelKmsKeyDeletionHandler {
                 owner,
                 false,
                 kms_delete_key_actions(),
-                req.extensions.get::<Option<RemoteAddr>>().and_then(|opt| opt.map(|a| a.0)),
+                crate::admin::auth::policy_source_addr(&req.extensions),
                 scoped.as_deref().unwrap_or_default(),
             )
             .await,
@@ -2263,7 +2263,7 @@ impl Operation for ListKmsKeysHandler {
                 owner,
                 false,
                 kms_list_keys_actions(),
-                req.extensions.get::<Option<RemoteAddr>>().and_then(|opt| opt.map(|a| a.0)),
+                crate::admin::auth::policy_source_addr(&req.extensions),
             )
             .await,
             KmsAuditOperation::ListKeys,
@@ -2459,7 +2459,7 @@ impl Operation for DescribeKmsKeyHandler {
                 owner,
                 false,
                 kms_describe_key_actions(),
-                req.extensions.get::<Option<RemoteAddr>>().and_then(|opt| opt.map(|a| a.0)),
+                crate::admin::auth::policy_source_addr(&req.extensions),
                 params.get("key_id").unwrap_or_default(),
             )
             .await,

rustfs/src/admin/handlers/mfa.rs

@@ -51,7 +51,6 @@ use crate::admin::storage_api::runtime::ECStore;
 use crate::admin::storage_api::s3::{self, Body, S3Error, S3ErrorCode, S3Request, S3Response, S3Result};
 use crate::admin::utils::read_compatible_admin_body;
 use crate::auth::constant_time_eq;
-use crate::server::RemoteAddr;
 use http::StatusCode;
 use hyper::Method;
 use matchit::Params;
@@ -484,7 +483,7 @@ impl Operation for UserMfaStatusHandler {
             caller.is_owner,
             false,
             vec![Action::AdminAction(AdminAction::GetUserAdminAction)],
-            req.extensions.get::<Option<RemoteAddr>>().and_then(|opt| opt.map(|a| a.0)),
+            crate::admin::auth::policy_source_addr(&req.extensions),
         )
         .await?;
 
@@ -521,7 +520,7 @@ impl Operation for UserMfaResetHandler {
             caller.is_owner,
             false,
             vec![Action::AdminAction(AdminAction::EnableUserAdminAction)],
-            req.extensions.get::<Option<RemoteAddr>>().and_then(|opt| opt.map(|a| a.0)),
+            crate::admin::auth::policy_source_addr(&req.extensions),
         )
         .await
         .inspect_err(|_| {

rustfs/src/admin/handlers/plugin_delegation.rs

@@ -11,7 +11,7 @@ use crate::admin::{
 };
 use crate::{
     auth::{check_key_valid_with_context, get_session_token},
-    server::{ADMIN_PREFIX, RemoteAddr},
+    server::ADMIN_PREFIX,
 };
 use http::{HeaderMap, HeaderValue, Method, StatusCode};
 use matchit::Params;
@@ -97,7 +97,7 @@ pub(super) async fn owner(
         &credentials,
         false,
         Action::AdminAction(action),
-        req.extensions.get::<Option<RemoteAddr>>().and_then(|a| a.map(|a| a.0)),
+        crate::admin::auth::policy_source_addr(&req.extensions),
         AdminResourceScope::bucket_object("", ""),
     )
     .await?;

rustfs/src/admin/handlers/plugins_instances.rs

@@ -23,7 +23,7 @@ use crate::admin::{
     handlers::supervise_admin_mutation,
     handlers::target_descriptor::{
         AdminTargetSpec, RuntimeHealthStatus, TargetEndpointSource, TargetInstanceReadModel, admin_target_spec_from_builtin,
-        allowed_target_keys, build_enabled_target_kvs, build_json_response, build_validated_target_kvs, collect_target_instances,
+        allowed_target_keys, build_json_response, build_validated_target_kvs, collect_target_instances,
         collect_validated_key_values, find_target_instance, target_module_disabled_reason,
         target_mutation_block_reason as shared_target_mutation_block_reason,
     },
@@ -53,7 +53,7 @@ use rustfs_config::{AUDIT_DEFAULT_DIR, EVENT_DEFAULT_DIR, MAX_ADMIN_REQUEST_BODY
 use rustfs_policy::policy::action::{Action, AdminAction};
 use rustfs_targets::catalog::builtin::{builtin_audit_target_admin_descriptors, builtin_notify_target_admin_descriptors};
 use rustfs_targets::{builtin_target_plugin_operational_state, runtime_state_from_status_label};
-use s3s::{Body, S3Request, S3Response, S3Result, s3_error};
+use s3s::{Body, S3Error, S3Request, S3Response, S3Result, s3_error};
 use sha2::{Digest, Sha256};
 use std::collections::{BTreeMap, HashMap};
 use std::sync::LazyLock;
@@ -149,7 +149,10 @@ fn restore_redacted_secret_values(
             continue;
         }
         if !secret_fields.iter().any(|field| field.eq_ignore_ascii_case(&kv.key)) {
-            continue;
+            return Err(S3Error::with_message(
+                S3ErrorCode::InvalidArgument,
+                "the redacted placeholder is only valid for stored secret fields",
+            ));
         }
         let stored = stored_config
             .and_then(|kvs| kvs.lookup(&kv.key))
@@ -166,6 +169,50 @@ fn restore_redacted_secret_values(
     Ok(())
 }
 
+/// Restore only credentials whose destination and trust configuration remain unchanged, then run
+/// the target's real validator. This gate precedes validators that authenticate to remote servers.
+async fn validate_instance_update(
+    resolved: &ResolvedPluginInstanceTarget,
+    requested: &mut [KeyValue],
+    stored: Option<&KVS>,
+    effective: Option<&KVS>,
+) -> S3Result<KVS> {
+    let spec = &resolved.target_spec;
+    let preserves_secrets =
+        spec.secret_fields
+            .iter()
+            .any(|&key| match requested.iter().find(|kv| kv.key.eq_ignore_ascii_case(key)) {
+                Some(kv) => kv.value == REDACTED_SECRET_VALUE,
+                None => effective.is_some_and(|config| config.lookup(key).is_some_and(|value| !value.is_empty())),
+            });
+    restore_redacted_secret_values(requested, spec.secret_fields, stored)?;
+    let context = resolved.context;
+    let candidate = if let Some(effective) = effective {
+        effective_validation_fields(context, spec.subsystem, effective, requested)?
+    } else {
+        let allowed = allowed_target_keys(context.specs, spec.subsystem);
+        collect_validated_key_values(
+            requested.iter().map(|kv| (kv.key.as_str(), kv.value.as_str())),
+            &allowed,
+            spec.subsystem,
+            "plugin instance",
+        )?
+    };
+    if preserves_secrets {
+        let bindings = rustfs_targets::manifest::builtin_target_manifest(spec.service).credential_binding_fields();
+        if bindings.iter().any(|key| {
+            let old = stored.and_then(|config| config.lookup(key)).unwrap_or_default();
+            candidate.get(*key).map(String::as_str).unwrap_or_default() != old
+        }) {
+            return Err(S3Error::with_message(
+                S3ErrorCode::InvalidArgument,
+                "changing a target destination or TLS trust configuration requires explicit credentials",
+            ));
+        }
+    }
+    build_validated_target_kvs(spec, candidate, context.default_queue_dir).await
+}
+
 /// Case-insensitive lookup of an instance's persisted config section.
 fn stored_instance_config<'a>(config: &'a Config, subsystem: &str, target_name: &str) -> Option<&'a KVS> {
     config.0.get(subsystem).and_then(|section| {
@@ -970,21 +1017,8 @@ impl Operation for PutPluginInstanceHandler {
         } else {
             stored_instance_config(&config_snapshot, resolved.target_spec.subsystem, &resolved.target_name)
         };
-        restore_redacted_secret_values(&mut body.key_values, resolved.target_spec.secret_fields, stored_secrets)?;
-
-        let validated_kvs = if let Some(effective) = effective_instance.as_ref() {
-            let candidate = effective_validation_fields(context, resolved.target_spec.subsystem, effective, &body.key_values)?;
-            build_validated_target_kvs(&resolved.target_spec, candidate, context.default_queue_dir).await?
-        } else {
-            build_enabled_target_kvs(
-                context.specs,
-                body.key_values.iter().map(|kv| (kv.key.as_str(), kv.value.as_str())),
-                resolved.target_spec.subsystem,
-                context.default_queue_dir,
-                "plugin instance",
-            )
-            .await?
-        };
+        let validated_kvs =
+            validate_instance_update(&resolved, &mut body.key_values, stored_secrets, effective_instance.as_ref()).await?;
 
         let kvs = if let Some(expected_config) = body.expected_config.as_ref() {
             let original = stored_instance_config(&config_snapshot, resolved.target_spec.subsystem, &resolved.target_name)
@@ -1375,6 +1409,195 @@ mod tests {
         );
     }
 
+    #[test]
+    fn amqp_and_redis_instance_views_redact_credential_bearing_urls() {
+        for domain in [PluginContractDomain::Notify, PluginContractDomain::Audit] {
+            let context = super::plugin_instance_domain_context(domain);
+            for service in ["amqp", "redis"] {
+                let spec = context
+                    .specs
+                    .iter()
+                    .find(|spec| spec.service == service)
+                    .expect("builtin target");
+                let mut config = KVS::new();
+                config.insert("url".to_owned(), format!("{service}://operator:private-password@example.test/"));
+                config.insert("password".to_owned(), "separate-password".to_owned());
+                let mapped = super::map_instance_config(config, super::secret_fields_for_instance(domain, spec.subsystem));
+                assert_eq!(mapped["url"], super::REDACTED_SECRET_VALUE);
+                assert_eq!(mapped["password"], super::REDACTED_SECRET_VALUE);
+                let encoded = serde_json::to_string(&mapped).expect("response JSON");
+                assert!(!encoded.contains("private-password"));
+                assert!(!encoded.contains("separate-password"));
+            }
+        }
+    }
+
+    #[tokio::test]
+    async fn plugin_update_binds_pulsar_credentials_to_tls_verification() {
+        use rustfs_config::{PULSAR_AUTH_TOKEN, PULSAR_BROKER, PULSAR_TLS_ALLOW_INSECURE, PULSAR_TLS_HOSTNAME_VERIFICATION};
+        let context = super::plugin_instance_domain_context(PluginContractDomain::Notify);
+        let target_spec = context
+            .specs
+            .iter()
+            .find(|spec| spec.service == "pulsar")
+            .expect("Pulsar target")
+            .clone();
+        let resolved = super::ResolvedPluginInstanceTarget {
+            context,
+            target_name: "primary".to_owned(),
+            target_spec,
+        };
+        let mut stored = KVS::new();
+        stored.insert(PULSAR_BROKER.to_owned(), "pulsar+ssl://broker.example.test:6651".to_owned());
+        stored.insert(PULSAR_AUTH_TOKEN.to_owned(), "operator-token".to_owned());
+        stored.insert(PULSAR_TLS_ALLOW_INSECURE.to_owned(), "off".to_owned());
+        stored.insert(PULSAR_TLS_HOSTNAME_VERIFICATION.to_owned(), "on".to_owned());
+        for (key, value) in [(PULSAR_TLS_ALLOW_INSECURE, "on"), (PULSAR_TLS_HOSTNAME_VERIFICATION, "off")] {
+            let mut requested = vec![super::KeyValue {
+                key: key.to_owned(),
+                value: value.to_owned(),
+            }];
+            let error = super::validate_instance_update(&resolved, &mut requested, Some(&stored), Some(&stored))
+                .await
+                .expect_err("weaker TLS verification must require explicit credentials");
+            assert!(error.to_string().contains("requires explicit credentials"));
+        }
+    }
+
+    #[tokio::test]
+    async fn plugin_update_never_sends_preserved_credentials_to_a_changed_destination() {
+        use tokio::io::{AsyncBufReadExt, AsyncReadExt, AsyncWriteExt, BufReader};
+        let listener = std::net::TcpListener::bind("127.0.0.1:0").expect("destination trap");
+        listener.set_nonblocking(true).expect("nonblocking trap");
+        let new_url = format!("redis://{}/", listener.local_addr().expect("trap address"));
+        let context = super::plugin_instance_domain_context(PluginContractDomain::Notify);
+        let spec = context
+            .specs
+            .iter()
+            .find(|spec| spec.service == "redis")
+            .expect("Redis target")
+            .clone();
+        let resolved = super::ResolvedPluginInstanceTarget {
+            context,
+            target_name: "primary".to_owned(),
+            target_spec: spec,
+        };
+        let mut stored = KVS::new();
+        stored.insert("url".to_owned(), "redis://127.0.0.1:1/".to_owned());
+        stored.insert("username".to_owned(), "operator".to_owned());
+        stored.insert("password".to_owned(), "operator-password".to_owned());
+        let field = |key: &str, value: &str| super::KeyValue {
+            key: key.to_owned(),
+            value: value.to_owned(),
+        };
+        for conditional in [false, true] {
+            for omitted in [false, true] {
+                let mut requested = vec![field("url", &new_url)];
+                if !omitted {
+                    requested.push(field("password", super::REDACTED_SECRET_VALUE));
+                }
+                if omitted && !conditional {
+                    continue;
+                }
+                let error =
+                    super::validate_instance_update(&resolved, &mut requested, Some(&stored), conditional.then_some(&stored))
+                        .await
+                        .expect_err("destination changes cannot inherit credentials");
+                assert!(error.to_string().contains("requires explicit credentials"));
+                assert!(
+                    matches!(listener.accept(), Err(error) if error.kind() == std::io::ErrorKind::WouldBlock),
+                    "validator must not connect before refusing credential reuse"
+                );
+            }
+        }
+        // A legacy replacement that omits its destination must not restore the password into defaults.
+        let mut requested = vec![field("password", super::REDACTED_SECRET_VALUE)];
+        assert!(
+            super::validate_instance_update(&resolved, &mut requested, Some(&stored), None)
+                .await
+                .is_err()
+        );
+        // Trust changes are destination changes even if the hostname stays unchanged.
+        let mut requested = vec![field("url", super::REDACTED_SECRET_VALUE), field("tls_allow_insecure", "on")];
+        assert!(
+            super::validate_instance_update(&resolved, &mut requested, Some(&stored), Some(&stored))
+                .await
+                .is_err()
+        );
+        assert!(matches!(listener.accept(), Err(error) if error.kind() == std::io::ErrorKind::WouldBlock));
+
+        let listener = tokio::net::TcpListener::from_std(listener).expect("async Redis control");
+        let server = tokio::spawn(async move {
+            let mut exchanges = Vec::new();
+            for _ in 0..2 {
+                let (stream, _) = listener.accept().await.expect("authorized credentials may connect");
+                let mut stream = BufReader::new(stream);
+                let mut received = Vec::new();
+                for _ in 0..16 {
+                    let mut line = String::new();
+                    assert!(stream.read_line(&mut line).await.expect("Redis array") > 0);
+                    let count: usize = line
+                        .trim()
+                        .strip_prefix('*')
+                        .expect("RESP array")
+                        .parse()
+                        .expect("array size");
+                    let mut command = Vec::new();
+                    for _ in 0..count {
+                        line.clear();
+                        stream.read_line(&mut line).await.expect("Redis bulk length");
+                        let length: usize = line.trim().strip_prefix('$').expect("RESP bulk").parse().expect("bulk size");
+                        let mut bytes = vec![0; length + 2];
+                        stream.read_exact(&mut bytes).await.expect("Redis bulk value");
+                        command.push(String::from_utf8(bytes[..length].to_vec()).expect("UTF8 Redis command"));
+                    }
+                    let ping = command.first().is_some_and(|value| value == "PING");
+                    let response = match command.first().map(String::as_str) {
+                        Some("PING") => b"+PONG\r\n".as_slice(),
+                        Some("HELLO") => b"%1\r\n+server\r\n+redis\r\n".as_slice(),
+                        _ => b"+OK\r\n".as_slice(),
+                    };
+                    received.extend(command);
+                    stream.get_mut().write_all(response).await.expect("Redis reply");
+                    if ping {
+                        break;
+                    }
+                }
+                assert!(received.contains(&"PING".to_owned()), "Redis validation must finish");
+                exchanges.push(received);
+            }
+            exchanges
+        });
+        let mut requested = vec![
+            field("url", &new_url),
+            field("username", "operator"),
+            field("password", "replacement-password"),
+        ];
+        let validated = super::validate_instance_update(&resolved, &mut requested, Some(&stored), None)
+            .await
+            .expect("explicit replacement secret may authorize the new endpoint");
+        assert_eq!(validated.get("password"), "replacement-password");
+        stored.insert("url".to_owned(), new_url);
+        let mut requested = vec![
+            field("url", super::REDACTED_SECRET_VALUE),
+            field("password", super::REDACTED_SECRET_VALUE),
+            field("username", "operator"),
+            field("channel", "changed-delivery-channel"),
+        ];
+        let restored = super::validate_instance_update(&resolved, &mut requested, Some(&stored), None)
+            .await
+            .expect("unchanged destination can round-trip hidden credentials while editing delivery settings");
+        assert_eq!(restored.get("password"), "operator-password");
+        assert!(!restored.0.iter().any(|kv| kv.value == super::REDACTED_SECRET_VALUE));
+        let received = tokio::time::timeout(std::time::Duration::from_secs(10), server)
+            .await
+            .expect("Redis deadline")
+            .expect("Redis task");
+        assert!(received[0].contains(&"replacement-password".to_owned()));
+        assert!(!received[0].contains(&"operator-password".to_owned()));
+        assert!(received[1].contains(&"operator-password".to_owned()));
+    }
+
     #[test]
     fn put_round_trip_restores_stored_secret_for_redacted_placeholder() {
         let mut key_values = vec![
@@ -1415,14 +1638,14 @@ mod tests {
     }
 
     #[test]
-    fn put_round_trip_leaves_placeholder_on_non_secret_fields_untouched() {
+    fn put_round_trip_rejects_placeholder_on_non_secret_fields() {
         let mut key_values = vec![super::KeyValue {
             key: WEBHOOK_ENDPOINT.to_string(),
             value: super::REDACTED_SECRET_VALUE.to_string(),
         }];
 
         super::restore_redacted_secret_values(&mut key_values, &[WEBHOOK_AUTH_TOKEN], None)
-            .expect("non-secret fields are not subject to placeholder restoration");
+            .expect_err("non-secret fields must not persist the reserved placeholder");
         assert_eq!(key_values[0].value, super::REDACTED_SECRET_VALUE);
     }
 

rustfs/src/admin/handlers/quota.rs

@@ -30,7 +30,6 @@ use crate::admin::storage_api::contract::bucket::{BucketOperations as _, BucketO
 use crate::auth::{check_key_valid, get_session_token};
 use crate::error::ApiError;
 use crate::server::ADMIN_PREFIX;
-use crate::server::RemoteAddr;
 use hyper::{Method, StatusCode};
 use matchit::Params;
 use rustfs_madmin::{SITE_REPL_API_VERSION, SRBucketMeta};
@@ -397,7 +396,7 @@ impl Operation for SetBucketQuotaHandler {
         let (cred, owner) =
             check_key_valid(get_session_token(&req.uri, &req.headers).unwrap_or_default(), &cred.access_key).await?;
 
-        let remote_addr = req.extensions.get::<Option<RemoteAddr>>().and_then(|opt| opt.map(|a| a.0));
+        let remote_addr = crate::admin::auth::policy_source_addr(&req.extensions);
         validate_admin_request(
             &req.headers,
             &cred,
@@ -538,7 +537,7 @@ impl Operation for GetBucketQuotaHandler {
         if bucket.is_empty() {
             return Err(s3_error!(InvalidRequest, "bucket name is required"));
         }
-        let remote_addr = req.extensions.get::<Option<RemoteAddr>>().and_then(|opt| opt.map(|a| a.0));
+        let remote_addr = crate::admin::auth::policy_source_addr(&req.extensions);
         validate_admin_request_with_bucket(
             &req.headers,
             &cred,
@@ -607,7 +606,7 @@ impl Operation for ClearBucketQuotaHandler {
         let (cred, owner) =
             check_key_valid(get_session_token(&req.uri, &req.headers).unwrap_or_default(), &cred.access_key).await?;
 
-        let remote_addr = req.extensions.get::<Option<RemoteAddr>>().and_then(|opt| opt.map(|a| a.0));
+        let remote_addr = crate::admin::auth::policy_source_addr(&req.extensions);
         validate_admin_request(
             &req.headers,
             &cred,
@@ -726,7 +725,7 @@ impl Operation for GetBucketQuotaStatsHandler {
             return Err(s3_error!(InvalidRequest, "bucket name is required"));
         }
 
-        let remote_addr = req.extensions.get::<Option<RemoteAddr>>().and_then(|opt| opt.map(|a| a.0));
+        let remote_addr = crate::admin::auth::policy_source_addr(&req.extensions);
         validate_admin_request_with_bucket(
             &req.headers,
             &cred,
@@ -795,7 +794,7 @@ impl Operation for CheckBucketQuotaHandler {
             return Err(s3_error!(InvalidRequest, "bucket name is required"));
         }
 
-        let remote_addr = req.extensions.get::<Option<RemoteAddr>>().and_then(|opt| opt.map(|a| a.0));
+        let remote_addr = crate::admin::auth::policy_source_addr(&req.extensions);
         validate_admin_request_with_bucket(
             &req.headers,
             &cred,
@@ -890,7 +889,7 @@ impl Operation for MigrateBucketQuotaProtocolHandler {
         let (cred, owner) =
             check_key_valid(get_session_token(&req.uri, &req.headers).unwrap_or_default(), &cred.access_key).await?;
 
-        let remote_addr = req.extensions.get::<Option<RemoteAddr>>().and_then(|opt| opt.map(|a| a.0));
+        let remote_addr = crate::admin::auth::policy_source_addr(&req.extensions);
         validate_admin_request(
             &req.headers,
             &cred,

rustfs/src/admin/handlers/service_account.rs

@@ -18,7 +18,7 @@ use crate::admin::handlers::site_replication::{encode_service_account_replicatio
 use crate::admin::runtime_sources::current_action_credentials;
 use crate::admin::utils::{encode_compatible_admin_payload, has_space_be, is_compat_admin_request, read_compatible_admin_body};
 use crate::auth::{constant_time_eq, get_condition_values, get_session_token};
-use crate::server::{ADMIN_PREFIX, RemoteAddr};
+use crate::server::ADMIN_PREFIX;
 use crate::{
     admin::router::{AdminOperation, Operation, S3Router},
     auth::check_key_valid,
@@ -84,6 +84,25 @@ fn parse_new_service_account_policy(policy: Option<&serde_json::Value>) -> S3Res
     Ok((!policy.id.is_empty() || !policy.version.is_empty() || !policy.statements.is_empty()).then_some(policy))
 }
 
+/// Service accounts cannot delegate beyond their own lifetime. STS sessions
+/// retain the existing ability to create independently expiring service accounts.
+fn bounded_service_account_expiration(caller: &StoredCredentials, requested: Option<OffsetDateTime>) -> Option<OffsetDateTime> {
+    match caller.expiration.filter(|_| caller.is_service_account()) {
+        Some(limit) => Some(requested.map_or(limit, |expiration| expiration.min(limit))),
+        None => requested,
+    }
+}
+
+fn set_service_account_duration_condition(
+    conditions: &mut HashMap<String, Vec<String>>,
+    expiration: Option<OffsetDateTime>,
+    now: OffsetDateTime,
+) {
+    // An unlimited credential must not bypass a finite NumericGreaterThan Deny.
+    let seconds = expiration.map_or(i64::MAX, |expiration| (expiration - now).whole_seconds().max(0));
+    conditions.insert("DurationSeconds".to_string(), vec![seconds.to_string()]);
+}
+
 fn compat_time_sentinel() -> OffsetDateTime {
     OffsetDateTime::UNIX_EPOCH
 }
@@ -322,9 +341,13 @@ impl Operation for AddServiceAccount {
         let (cred, owner) =
             check_key_valid(get_session_token(&req.uri, &req.headers).unwrap_or_default(), &req_cred.access_key).await?;
 
+        if !constant_time_eq(&cred.secret_key, req_cred.secret_key.expose()) {
+            return Err(S3Error::with_message(S3ErrorCode::AccessDenied, "access denied"));
+        }
+
         let body = read_compatible_admin_body(req.input, MAX_ADMIN_REQUEST_BODY_SIZE, req.uri.path(), &cred.secret_key).await?;
 
-        let create_req: AddServiceAccountReq =
+        let mut create_req: AddServiceAccountReq =
             serde_json::from_slice(&body[..]).map_err(|e| s3_error!(InvalidRequest, "unmarshal body failed, e: {:?}", e))?;
 
         // create_req.expiration = create_req.expiration.and_then(|expire| expire.replace_millisecond(0).ok());
@@ -370,6 +393,11 @@ impl Operation for AddServiceAccount {
             return Err(s3_error!(InvalidRequest, "iam not init"));
         };
 
+        create_req.expiration = bounded_service_account_expiration(&cred, create_req.expiration);
+        let mut conditions =
+            get_condition_values(&req.headers, &cred, None, None, crate::admin::auth::policy_source_addr(&req.extensions));
+        set_service_account_duration_condition(&mut conditions, create_req.expiration, OffsetDateTime::now_utc());
+
         // This family deliberately calls `is_allowed` directly instead of going
         // through `validate_admin_request`, and must keep doing so
         // (backlog#1886). The shared helper returns as soon as *any* candidate
@@ -383,13 +411,7 @@ impl Operation for AddServiceAccount {
                 groups: &cred.groups,
                 action: Action::AdminAction(AdminAction::CreateServiceAccountAdminAction),
                 bucket: "",
-                conditions: &get_condition_values(
-                    &req.headers,
-                    &cred,
-                    None,
-                    None,
-                    req.extensions.get::<Option<RemoteAddr>>().and_then(|opt| opt.map(|a| a.0)),
-                ),
+                conditions: &conditions,
                 is_owner: owner,
                 object: "",
                 claims: cred.claims_or_empty(),
@@ -444,17 +466,9 @@ impl Operation for AddServiceAccount {
         let replication_description = opts.description.clone().unwrap_or_default();
         let replication_expiration = opts.expiration;
 
-        let create_result = if is_svc_acc {
-            iam_store
-                .new_service_account_from_caller(&target_user, target_groups, opts, &cred)
-                .await
-        } else {
-            let replication_claims = opts.claims.clone().unwrap_or_default();
-            iam_store
-                .new_service_account(&target_user, target_groups, opts)
-                .await
-                .map(|(credentials, updated_at)| (credentials, updated_at, replication_claims))
-        };
+        let create_result = iam_store
+            .new_service_account_from_caller(&target_user, target_groups, opts, &cred)
+            .await;
         let (new_cred, updated_at, replication_claims) = create_result.map_err(|e| {
             debug!(
                 component = LOG_COMPONENT_ADMIN,
@@ -466,7 +480,8 @@ impl Operation for AddServiceAccount {
                 "admin service account state"
             );
             match e {
-                rustfs_iam::error::Error::InvalidAccessKeyLength
+                rustfs_iam::error::Error::InvalidArgument
+                | rustfs_iam::error::Error::InvalidAccessKeyLength
                 | rustfs_iam::error::Error::InvalidSecretKeyLength
                 | rustfs_iam::error::Error::AccessKeyAlreadyExists => iam_error_to_s3_error(e),
                 rustfs_iam::error::Error::IAMActionNotAllowed => s3_error!(AccessDenied, "access denied"),
@@ -599,6 +614,9 @@ impl Operation for UpdateServiceAccount {
             return Err(s3_error!(InvalidRequest, "get cred failed"));
         };
 
+        let (cred, owner) =
+            check_key_valid(get_session_token(&req.uri, &req.headers).unwrap_or_default(), &input_cred.access_key).await?;
+
         let Ok(iam_store) = crate::admin::runtime_sources::current_ready_iam_handle() else {
             return Err(s3_error!(InvalidRequest, "iam not init"));
         };
@@ -608,17 +626,29 @@ impl Operation for UpdateServiceAccount {
         //     s3_error!(InternalError, "get service account failed")
         // })?;
 
-        let body =
-            read_compatible_admin_body(req.input, MAX_ADMIN_REQUEST_BODY_SIZE, req.uri.path(), input_cred.secret_key.expose())
-                .await?;
+        let body = read_compatible_admin_body(req.input, MAX_ADMIN_REQUEST_BODY_SIZE, req.uri.path(), &cred.secret_key).await?;
 
-        let update_req: UpdateServiceAccountReq =
+        let mut update_req: UpdateServiceAccountReq =
             serde_json::from_slice(&body[..]).map_err(|e| s3_error!(InvalidRequest, "unmarshal body failed, e: {:?}", e))?;
 
         update_req.validate().map_err(|e| S3Error::with_message(InvalidRequest, e))?;
 
-        let (cred, owner) =
-            check_key_valid(get_session_token(&req.uri, &req.headers).unwrap_or_default(), &input_cred.access_key).await?;
+        // Keep lookup errors behind authorization even though lifetime conditions
+        // need the existing expiration when the request omits it.
+        let existing_account = iam_store.get_service_account(&access_key).await;
+
+        if update_req.new_expiration.is_some() {
+            update_req.new_expiration = bounded_service_account_expiration(&cred, update_req.new_expiration);
+        }
+        let mut conditions =
+            get_condition_values(&req.headers, &cred, None, None, crate::admin::auth::policy_source_addr(&req.extensions));
+        set_service_account_duration_condition(
+            &mut conditions,
+            update_req
+                .new_expiration
+                .or_else(|| existing_account.as_ref().ok().and_then(|(account, _)| account.expiration)),
+            OffsetDateTime::now_utc(),
+        );
 
         if !iam_store
             .is_allowed(&Args {
@@ -626,13 +656,7 @@ impl Operation for UpdateServiceAccount {
                 groups: &cred.groups,
                 action: Action::AdminAction(AdminAction::UpdateServiceAccountAdminAction),
                 bucket: "",
-                conditions: &get_condition_values(
-                    &req.headers,
-                    &cred,
-                    None,
-                    None,
-                    req.extensions.get::<Option<RemoteAddr>>().and_then(|opt| opt.map(|a| a.0)),
-                ),
+                conditions: &conditions,
                 is_owner: owner,
                 object: "",
                 claims: cred.claims_or_empty(),
@@ -643,10 +667,7 @@ impl Operation for UpdateServiceAccount {
             return Err(s3_error!(AccessDenied, "access denied"));
         }
 
-        let (svc_account, _) = iam_store
-            .get_service_account(&access_key)
-            .await
-            .map_err(|e| map_service_account_lookup_error(e, "get service account failed"))?;
+        let (svc_account, _) = existing_account.map_err(|e| map_service_account_lookup_error(e, "get service account failed"))?;
 
         // The admin action permits updates within a caller's scope; only an
         // owner may cross service-account parent boundaries.
@@ -723,11 +744,6 @@ impl Operation for InfoServiceAccount {
             return Err(s3_error!(InvalidRequest, "iam not init"));
         };
 
-        let (svc_account, session_policy) = iam_store
-            .get_service_account(&access_key)
-            .await
-            .map_err(|e| map_service_account_lookup_error(e, "get service account failed"))?;
-
         let Some(input_cred) = req.credentials else {
             return Err(s3_error!(InvalidRequest, "get cred failed"));
         };
@@ -735,6 +751,11 @@ impl Operation for InfoServiceAccount {
         let (cred, owner) =
             check_key_valid(get_session_token(&req.uri, &req.headers).unwrap_or_default(), &input_cred.access_key).await?;
 
+        let (svc_account, session_policy) = iam_store
+            .get_service_account(&access_key)
+            .await
+            .map_err(|e| map_service_account_lookup_error(e, "get service account failed"))?;
+
         if !iam_store
             .is_allowed(&Args {
                 account: &cred.access_key,
@@ -746,7 +767,7 @@ impl Operation for InfoServiceAccount {
                     &cred,
                     None,
                     None,
-                    req.extensions.get::<Option<RemoteAddr>>().and_then(|opt| opt.map(|a| a.0)),
+                    crate::admin::auth::policy_source_addr(&req.extensions),
                 ),
                 is_owner: owner,
                 object: "",
@@ -811,7 +832,7 @@ impl Operation for TemporaryAccountInfo {
                     &cred,
                     None,
                     None,
-                    req.extensions.get::<Option<RemoteAddr>>().and_then(|opt| opt.map(|a| a.0)),
+                    crate::admin::auth::policy_source_addr(&req.extensions),
                 ),
                 is_owner: owner,
                 object: "",
@@ -872,13 +893,8 @@ impl Operation for InfoAccessKey {
 
         let target_cred = iam_store.get_user(&access_key).await.map(|identity| identity.credentials);
 
-        let conditions = get_condition_values(
-            &req.headers,
-            &cred,
-            None,
-            None,
-            req.extensions.get::<Option<RemoteAddr>>().and_then(|opt| opt.map(|a| a.0)),
-        );
+        let conditions =
+            get_condition_values(&req.headers, &cred, None, None, crate::admin::auth::policy_source_addr(&req.extensions));
         let claims = cred.claims_or_empty();
 
         if !iam_store
@@ -1007,7 +1023,7 @@ impl Operation for ListServiceAccount {
                         &cred,
                         None,
                         None,
-                        req.extensions.get::<Option<RemoteAddr>>().and_then(|opt| opt.map(|a| a.0)),
+                        crate::admin::auth::policy_source_addr(&req.extensions),
                     ),
                     is_owner: owner,
                     object: "",
@@ -1151,7 +1167,7 @@ impl Operation for ListAccessKeysBulk {
                         &cred,
                         None,
                         None,
-                        req.extensions.get::<Option<RemoteAddr>>().and_then(|opt| opt.map(|a| a.0)),
+                        crate::admin::auth::policy_source_addr(&req.extensions),
                     ),
                     is_owner: owner,
                     object: "",
@@ -1174,7 +1190,7 @@ impl Operation for ListAccessKeysBulk {
                     &cred,
                     None,
                     None,
-                    req.extensions.get::<Option<RemoteAddr>>().and_then(|opt| opt.map(|a| a.0)),
+                    crate::admin::auth::policy_source_addr(&req.extensions),
                 ),
                 is_owner: owner,
                 object: "",
@@ -1355,7 +1371,7 @@ impl Operation for DeleteServiceAccount {
                     &cred,
                     None,
                     None,
-                    req.extensions.get::<Option<RemoteAddr>>().and_then(|opt| opt.map(|a| a.0)),
+                    crate::admin::auth::policy_source_addr(&req.extensions),
                 ),
                 is_owner: owner,
                 object: "",
@@ -1444,12 +1460,168 @@ mod tests {
     use serde_json::json;
     use serde_urlencoded::from_bytes;
 
+    #[test]
+    #[serial_test::serial]
+    fn service_account_handlers_authenticate_before_decryption_or_lookup() {
+        crate::app::gating_test_env::run_large_stack_test("service-account-auth-first", || async {
+            use crate::admin::runtime_sources::IamInterface;
+            use crate::admin::runtime_sources::{AppContext, publish_test_app_context};
+            use rustfs_iam::store::object::ObjectStore;
+            use std::sync::Arc;
+            struct ReadyIam(Arc<rustfs_iam::sys::IamSys<ObjectStore>>);
+            impl IamInterface for ReadyIam {
+                fn handle(&self) -> Arc<rustfs_iam::sys::IamSys<ObjectStore>> {
+                    self.0.clone()
+                }
+                fn is_ready(&self) -> bool {
+                    true
+                }
+            }
+            let ambient = crate::app::gating_test_env::shared_gating_ambient().await;
+            let (_directory, _paths, store) = crate::app::gating_test_env::isolated_multi_pool_ecstore().await;
+            ObjectStore::new(store.clone())
+                .save_iam_config(
+                    serde_json::json!({"version": 1}),
+                    format!("{}/format.json", *rustfs_iam::store::object::IAM_CONFIG_PREFIX),
+                )
+                .await
+                .expect("initialize IAM storage");
+            let iam = rustfs_iam::build_iam_sys(store.clone()).await.expect("initialized IAM");
+            let disabled_key = format!("disabled-{}", uuid::Uuid::new_v4());
+            iam.create_user(
+                &disabled_key,
+                &rustfs_madmin::AddOrUpdateUserReq {
+                    secret_key: "disabled-user-secret".to_owned(),
+                    policy: None,
+                    status: rustfs_madmin::AccountStatus::Disabled,
+                },
+            )
+            .await
+            .expect("create disabled IAM caller");
+            let context = Arc::new(AppContext::new(store, Arc::new(ReadyIam(iam)), ambient.kms()));
+            assert!(context.publish_action_credentials(StoredCredentials {
+                access_key: "auth-first-root".to_owned(),
+                secret_key: "auth-first-root-secret".to_owned(),
+                status: "on".to_owned(),
+                ..Default::default()
+            }));
+            publish_test_app_context(context);
+            for caller in ["keystone:forged", disabled_key.as_str()] {
+                for operation in ["update-service-account", "info-service-account"] {
+                    let req = S3Request {
+                        input: Body::from(vec![0u8; 64]),
+                        method: if operation == "update-service-account" {
+                            http::Method::POST
+                        } else {
+                            http::Method::GET
+                        },
+                        uri: format!("/minio/admin/v3/{operation}?accessKey=missing-target")
+                            .parse()
+                            .unwrap(),
+                        headers: HeaderMap::new(),
+                        extensions: http::Extensions::new(),
+                        credentials: Some(s3s::auth::Credentials {
+                            access_key: caller.to_owned(),
+                            secret_key: s3s::auth::SecretKey::from("disabled-user-secret"),
+                        }),
+                        region: None,
+                        service: None,
+                        trailing_headers: None,
+                    };
+                    let result = if operation == "update-service-account" {
+                        UpdateServiceAccount {}.call(req, Params::new()).await
+                    } else {
+                        InfoServiceAccount {}.call(req, Params::new()).await
+                    };
+                    assert_eq!(
+                        result
+                            .expect_err("invalid identity precedes KDF and missing target lookup")
+                            .code(),
+                        &s3s::S3ErrorCode::InvalidAccessKeyId,
+                        "{caller}: {operation}"
+                    );
+                }
+            }
+            publish_test_app_context(ambient);
+        });
+    }
+
     #[test]
     fn access_key_query_supports_external_alias() {
         let query: AccessKeyQuery = from_bytes(b"access-key=test-access-key").expect("parse query");
         assert_eq!(query.access_key, "test-access-key");
     }
 
+    #[test]
+    fn service_account_cannot_delegate_a_longer_lifetime() {
+        let deadline = OffsetDateTime::UNIX_EPOCH + time::Duration::days(10);
+        let earlier = deadline - time::Duration::hours(1);
+        let later = deadline + time::Duration::days(365);
+        let mut caller = StoredCredentials {
+            access_key: "expiring-service".to_string(),
+            parent_user: "parent".to_string(),
+            expiration: Some(deadline),
+            claims: Some(HashMap::from([("sa-policy".to_string(), json!("inherited-policy"))])),
+            ..Default::default()
+        };
+        assert_eq!(bounded_service_account_expiration(&caller, None), Some(deadline));
+        assert_eq!(bounded_service_account_expiration(&caller, Some(later)), Some(deadline));
+        assert_eq!(bounded_service_account_expiration(&caller, Some(earlier)), Some(earlier));
+        caller.expiration = None;
+        assert_eq!(bounded_service_account_expiration(&caller, None), None);
+        caller.expiration = Some(deadline);
+        caller.claims = None;
+        caller.session_token = "signed-sts".to_string();
+        assert_eq!(
+            bounded_service_account_expiration(&caller, None),
+            None,
+            "STS retains independent service lifetime"
+        );
+        assert_eq!(bounded_service_account_expiration(&caller, Some(later)), Some(later));
+    }
+
+    #[tokio::test]
+    async fn service_account_duration_condition_enforces_finite_and_unlimited_lifetimes() {
+        let policy = Policy::parse_config(
+            br#"{"Version":"2012-10-17","Statement":[
+            {"Effect":"Allow","Action":["admin:CreateServiceAccount","admin:UpdateServiceAccount"],"Resource":"*"},
+            {"Effect":"Deny","Action":["admin:CreateServiceAccount","admin:UpdateServiceAccount"],"Resource":"*",
+             "Condition":{"NumericGreaterThan":{"svc:DurationSeconds":"3600"}}}
+        ]}"#,
+        )
+        .expect("service lifetime policy");
+        let now = OffsetDateTime::UNIX_EPOCH;
+        for (expiration, allowed) in [
+            (Some(now + time::Duration::seconds(3600)), true),
+            (Some(now + time::Duration::seconds(3601)), false),
+            (None, false),
+        ] {
+            let mut conditions = HashMap::from([("DurationSeconds".to_string(), vec!["1".to_string()])]);
+            set_service_account_duration_condition(&mut conditions, expiration, now);
+            for action in [
+                AdminAction::CreateServiceAccountAdminAction,
+                AdminAction::UpdateServiceAccountAdminAction,
+            ] {
+                assert_eq!(
+                    policy
+                        .is_allowed(&Args {
+                            account: "user",
+                            groups: &None,
+                            action: Action::AdminAction(action),
+                            conditions: &conditions,
+                            is_owner: false,
+                            claims: &HashMap::new(),
+                            deny_only: false,
+                            bucket: "",
+                            object: "",
+                        })
+                        .await,
+                    allowed
+                );
+            }
+        }
+    }
+
     #[test]
     fn non_admin_delete_is_fail_closed_on_unloadable_target() {
         // Regression (backlog#806): a non-admin caller may delete ONLY a service

rustfs/src/admin/handlers/sts.rs

@@ -27,7 +27,6 @@ use crate::{
     },
     auth::{check_key_valid, get_session_token},
     server::ADMIN_PREFIX,
-    server::RemoteAddr,
 };
 use http::StatusCode;
 use http::header::HeaderValue;
@@ -54,6 +53,7 @@ use s3s::{
 use serde::Deserialize;
 use serde_json::Value;
 use serde_urlencoded::from_bytes;
+use std::collections::HashMap;
 use time::{Duration, OffsetDateTime};
 use tracing::{debug, error, warn};
 
@@ -82,6 +82,12 @@ fn clamp_assume_role_duration(duration_seconds: usize) -> usize {
     }
 }
 
+fn set_assume_role_duration_condition(conditions: &mut HashMap<String, Vec<String>>, requested: usize) -> usize {
+    let duration = clamp_assume_role_duration(requested);
+    conditions.insert("DurationSeconds".to_string(), vec![duration.to_string()]);
+    duration
+}
+
 /// Record that AssumeRole assembled its session claims without echoing any claim values.
 ///
 /// Claims carry caller-supplied identity material (parent user, session policy, arbitrary
@@ -186,7 +192,7 @@ impl Operation for AssumeRoleHandle {
 
         match body.action.as_str() {
             ASSUME_ROLE_ACTION => {
-                let remote_addr = req.extensions.get::<Option<RemoteAddr>>().and_then(|opt| opt.map(|a| a.0));
+                let remote_addr = crate::admin::auth::policy_source_addr(&req.extensions);
                 handle_assume_role(req.credentials, req.uri, req.headers, remote_addr, body, store, &audit).await
             }
             ASSUME_ROLE_WITH_WEB_IDENTITY_ACTION => handle_assume_role_with_web_identity(body).await,
@@ -216,6 +222,12 @@ async fn handle_assume_role(
 
     let (cred, owner) = check_key_valid(get_session_token(&uri, &headers).unwrap_or_default(), &user.access_key).await?;
 
+    // The signature was verified before the request body was consumed. A fresh
+    // IAM lookup must not silently replace the secret that authenticated it.
+    if !crate::auth::constant_time_eq(&cred.secret_key, user.secret_key.expose()) {
+        return Err(S3Error::with_message(S3ErrorCode::AccessDenied, "Access Denied"));
+    }
+
     if cred.is_temp() || cred.is_service_account() {
         return Err(s3_error!(AccessDenied, "Access Denied"));
     }
@@ -223,7 +235,8 @@ async fn handle_assume_role(
     let Ok(iam_store) = crate::admin::runtime_sources::current_ready_iam_handle() else {
         return Err(s3_error!(InvalidRequest, "iam not init"));
     };
-    let conditions = crate::auth::get_condition_values(&headers, &cred, None, None, remote_addr);
+    let mut conditions = crate::auth::get_condition_values(&headers, &cred, None, None, remote_addr);
+    let exp = set_assume_role_duration_condition(&mut conditions, body.duration_seconds);
     if !iam_store
         .is_allowed(&Args {
             account: &cred.access_key,
@@ -260,7 +273,7 @@ async fn handle_assume_role(
     // behaviour of every existing deployment is unchanged.
     let mfa_verified = enforce_second_factor(&cred.access_key, &body, store, audit).await?;
 
-    let mut claims = cred.claims.unwrap_or_default();
+    let mut claims = cred.claims.clone().unwrap_or_default();
 
     if mfa_verified {
         // Recorded on the session so a later policy condition can require it,
@@ -271,8 +284,6 @@ async fn handle_assume_role(
 
     populate_session_policy(&mut claims, &body.policy)?;
 
-    let exp = clamp_assume_role_duration(body.duration_seconds);
-
     claims.insert(
         "exp".to_string(),
         Value::Number(serde_json::Number::from(
@@ -309,9 +320,9 @@ async fn handle_assume_role(
     );
 
     let updated_at = iam_store
-        .set_temp_user(&new_cred.access_key, &new_cred, None)
+        .set_temp_user_from_parent(&new_cred, &cred)
         .await
-        .map_err(|_| s3_error!(InternalError, "set_temp_user failed"))?;
+        .map_err(super::iam_error::iam_error_to_s3_error)?;
 
     let root_access_key = current_action_credentials().map(|cred| cred.access_key);
     if root_access_key.as_deref() != Some(new_cred.parent_user.as_str())
@@ -604,6 +615,40 @@ mod tests {
         assert_eq!(clamp(999999), 43200); // clamped to max
     }
 
+    #[tokio::test]
+    async fn assume_role_duration_condition_enforces_policy_after_clamping() {
+        use rustfs_policy::policy::Policy;
+        let policy = Policy::parse_config(
+            br#"{"Version":"2012-10-17","Statement":[{
+            "Effect":"Deny","Action":"sts:AssumeRole","Resource":"*",
+            "Condition":{"NumericGreaterThan":{"sts:DurationSeconds":"3600"}}
+        }]}"#,
+        )
+        .expect("duration policy");
+        for (requested, allowed) in [(0, true), (1, true), (3600, true), (3601, false), (usize::MAX, false)] {
+            let mut conditions = HashMap::from([("DurationSeconds".to_string(), vec!["1".to_string()])]);
+            let lifetime = set_assume_role_duration_condition(&mut conditions, requested);
+            assert_eq!(conditions["DurationSeconds"], [lifetime.to_string()]);
+            assert_eq!(
+                policy
+                    .is_allowed(&Args {
+                        account: "user",
+                        groups: &None,
+                        action: Action::StsAction(StsAction::AssumeRoleAction),
+                        conditions: &conditions,
+                        is_owner: false,
+                        claims: &HashMap::new(),
+                        deny_only: true,
+                        bucket: "",
+                        object: "",
+                    })
+                    .await,
+                allowed,
+                "requested={requested}"
+            );
+        }
+    }
+
     #[test]
     fn test_assume_role_duration_is_clamped_to_max() {
         // Regression: the standard AssumeRole path previously used the raw client-supplied

rustfs/src/admin/handlers/table_catalog/mod.rs

@@ -24,7 +24,7 @@ use crate::admin::{
 };
 use crate::auth::{check_key_valid_with_context, get_session_token};
 use crate::error::ApiError;
-use crate::server::{RemoteAddr, TABLE_CATALOG_COMPAT_PREFIX, TABLE_CATALOG_PREFIX};
+use crate::server::{TABLE_CATALOG_COMPAT_PREFIX, TABLE_CATALOG_PREFIX};
 use crate::table_catalog::{DEFAULT_WAREHOUSE_ID, TableCatalogStore};
 use bytes::Bytes;
 use futures::{StreamExt, TryStreamExt, stream};
@@ -1141,7 +1141,7 @@ async fn authorize_table_catalog_request(req: &S3Request<Body>, action: AdminAct
         &principal.credentials,
         principal.owner,
         Action::AdminAction(action),
-        req.extensions.get::<Option<RemoteAddr>>().and_then(|opt| opt.map(|a| a.0)),
+        crate::admin::auth::policy_source_addr(&req.extensions),
         AdminResourceScope::bucket(""),
     )
     .await
@@ -1255,7 +1255,7 @@ async fn authorize_table_catalog_object_for_principal(
         &principal.credentials,
         principal.owner,
         Action::AdminAction(action),
-        req.extensions.get::<Option<RemoteAddr>>().and_then(|opt| opt.map(|a| a.0)),
+        crate::admin::auth::policy_source_addr(&req.extensions),
         AdminResourceScope::bucket_object(bucket, object),
     )
     .await

rustfs/src/admin/handlers/usage_prefix.rs

@@ -19,7 +19,7 @@
 //! usage caches, with a one-level sub-prefix breakdown — the data console
 //! buckets view MinIO serves from `loadPrefixUsageFromBackend`.
 
-use crate::admin::auth::authorize_admin_request;
+use crate::admin::auth::{authenticate_request, policy_source_addr, validate_admin_request_with_bucket};
 use crate::admin::handlers::system::data_usage_info_gate_actions;
 use crate::admin::router::{AdminOperation, Operation, S3Router};
 use crate::server::ADMIN_PREFIX;
@@ -69,15 +69,24 @@ fn parse_usage_prefix_query(query: Option<&str>) -> S3Result<(String, usize)> {
 #[async_trait::async_trait]
 impl Operation for BucketPrefixUsageHandler {
     async fn call(&self, req: S3Request<Body>, params: Params<'_, '_>) -> S3Result<S3Response<(StatusCode, Body)>> {
-        // The shared gate reports the same `InvalidRequest` "get cred failed" this
-        // handler has always returned for a credential-less request, so it needs no
-        // message-preserving pre-check.
-        authorize_admin_request(&req, data_usage_info_gate_actions()).await?;
-
+        let Some(input_cred) = req.credentials.as_ref() else {
+            return Err(S3Error::with_message(S3ErrorCode::InvalidRequest, "get cred failed"));
+        };
+        let (cred, owner) = authenticate_request(&req.headers, &req.uri, input_cred).await?;
         let bucket = params.get("bucket").unwrap_or_default().to_string();
         if bucket.is_empty() {
             return Err(s3_error!(InvalidRequest, "bucket path parameter is required"));
         }
+        validate_admin_request_with_bucket(
+            &req.headers,
+            &cred,
+            owner,
+            false,
+            data_usage_info_gate_actions(),
+            policy_source_addr(&req.extensions),
+            &bucket,
+        )
+        .await?;
         let (prefix, max_entries) = parse_usage_prefix_query(req.uri.query())?;
 
         // Authorization is bucket-scoped by the same any-of gate as the

rustfs/src/admin/handlers/user.rs

@@ -23,7 +23,6 @@ use crate::{
         utils::{encode_compatible_admin_payload, has_space_be, read_compatible_admin_body},
     },
     auth::{check_key_valid, constant_time_eq, get_session_token},
-    server::RemoteAddr,
 };
 use http::{HeaderMap, StatusCode};
 use matchit::Params;
@@ -31,7 +30,7 @@ use rustfs_config::{MAX_ADMIN_REQUEST_BODY_SIZE, MAX_IAM_IMPORT_EXPANDED_SIZE, M
 use rustfs_credentials::Credentials;
 use rustfs_iam::{
     store::{GroupInfo, MappedPolicy, UserType},
-    sys::{NewServiceAccountOpts, UpdateServiceAccountOpts},
+    sys::{NewServiceAccountOpts, SITE_REPLICATOR_SERVICE_ACCOUNT, UpdateServiceAccountOpts},
 };
 use rustfs_madmin::{
     AccountStatus, AddOrUpdateUserReq, IAMEntities, IAMErrEntities, IAMErrEntity, IAMErrPolicyEntity, SITE_REPL_API_VERSION,
@@ -196,6 +195,19 @@ fn imported_service_account_parent_scope_failure(
     })
 }
 
+fn retain_exportable_service_accounts(accounts: &mut HashMap<String, rustfs_policy::auth::UserIdentity>) {
+    accounts.retain(|key, account| {
+        key != SITE_REPLICATOR_SERVICE_ACCOUNT && account.credentials.access_key != SITE_REPLICATOR_SERVICE_ACCOUNT
+    });
+}
+
+fn imported_service_account_reserved_failure(access_key: &str) -> Option<IAMErrEntity> {
+    (access_key == SITE_REPLICATOR_SERVICE_ACCOUNT).then(|| IAMErrEntity {
+        name: access_key.to_owned(),
+        error: "service account name is reserved for site replication".to_owned(),
+    })
+}
+
 fn imported_service_account_access_key_failure(entry_access_key: &str, payload_access_key: &str) -> Option<IAMErrEntity> {
     (entry_access_key != payload_access_key).then(|| IAMErrEntity {
         name: entry_access_key.to_string(),
@@ -208,6 +220,7 @@ pub struct AddUser {}
 fn map_add_user_create_error(err: rustfs_iam::error::Error) -> S3Error {
     match err {
         rustfs_iam::error::Error::AccessKeyAlreadyExists => iam_error_to_s3_error(err),
+        rustfs_iam::error::Error::IAMActionNotAllowed => S3Error::new(S3ErrorCode::AccessDenied),
         err => S3Error::with_message(S3ErrorCode::InternalError, format!("failed to create user: {err}")),
     }
 }
@@ -243,7 +256,7 @@ impl Operation for AddUser {
         // let body_bytes = decrypt_data(input_cred.secret_key.expose().as_bytes(), &body)
         //     .map_err(|e| S3Error::with_message(S3ErrorCode::InvalidArgument, format!("decrypt_data err {}", e)))?;
 
-        let args: AddOrUpdateUserReq = serde_json::from_slice(&body)
+        let mut args: AddOrUpdateUserReq = serde_json::from_slice(&body)
             .map_err(|e| S3Error::with_message(S3ErrorCode::InvalidRequest, format!("invalid JSON: {e}")))?;
 
         if args.secret_key.is_empty() {
@@ -261,7 +274,7 @@ impl Operation for AddUser {
         };
 
         if let Some(user) = iam_store.get_user(ak).await {
-            if (user.credentials.is_temp() || user.credentials.is_service_account()) && cred.parent_user == ak {
+            if user.credentials.is_temp() || user.credentials.is_service_account() {
                 return Err(s3_error!(InvalidArgument, "cannot create a user with a service account access key"));
             }
         } else if has_space_be(ak) {
@@ -306,11 +319,22 @@ impl Operation for AddUser {
             owner,
             check_deny_only,
             vec![Action::AdminAction(AdminAction::CreateUserAdminAction)],
-            req.extensions.get::<Option<RemoteAddr>>().and_then(|opt| opt.map(|a| a.0)),
+            crate::admin::auth::policy_source_addr(&req.extensions),
         )
         .await?;
 
-        let updated_at = iam_store.create_user(ak, &args).await.map_err(map_add_user_create_error)?;
+        let updated_at = if check_deny_only {
+            let (updated_at, status, _) = iam_store
+                .change_own_secret_key(ak, input_cred.secret_key.expose(), &args.secret_key)
+                .await
+                .map_err(map_add_user_create_error)?;
+            args.status = status;
+            // A self-service secret rotation does not change policy attachment.
+            args.policy = None;
+            updated_at
+        } else {
+            iam_store.create_user(ak, &args).await.map_err(map_add_user_create_error)?
+        };
 
         if let Err(err) = site_replication_iam_change_hook(SRIAMItem {
             r#type: "iam-user".to_string(),
@@ -619,7 +643,7 @@ impl Operation for GetUserInfo {
             owner,
             check_deny_only,
             vec![Action::AdminAction(AdminAction::GetUserAdminAction)],
-            req.extensions.get::<Option<RemoteAddr>>().and_then(|opt| opt.map(|a| a.0)),
+            crate::admin::auth::policy_source_addr(&req.extensions),
         )
         .await?;
 
@@ -749,12 +773,9 @@ impl Operation for ExportIam {
                         .await
                         .map_err(|e| S3Error::with_message(S3ErrorCode::InternalError, e.to_string()))?;
 
+                    retain_exportable_service_accounts(&mut service_accounts);
                     let mut svc_accts: HashMap<String, SRSvcAccCreate> = HashMap::new();
                     for (k, acc) in service_accounts {
-                        if k == "siteReplicatorSvcAcc" {
-                            continue;
-                        }
-
                         let claims = iam_store
                             .get_claims_for_svc_acc(&acc.credentials.access_key)
                             .await
@@ -904,7 +925,7 @@ impl Operation for ImportIam {
             owner,
             false,
             vec![Action::AdminAction(AdminAction::ImportIAMAction)],
-            req.extensions.get::<Option<RemoteAddr>>().and_then(|opt| opt.map(|a| a.0)),
+            crate::admin::auth::policy_source_addr(&req.extensions),
         )
         .await?;
 
@@ -1089,6 +1110,11 @@ impl Operation for ImportIam {
                         return Err(s3_error!(InvalidArgument, "has space be {ak}"));
                     }
 
+                    if let Some(err) = imported_service_account_reserved_failure(&ak) {
+                        failed.service_accounts.push(err);
+                        continue;
+                    }
+
                     if let Some(err) = imported_service_account_access_key_failure(&ak, &req.access_key) {
                         failed.service_accounts.push(err);
                         continue;
@@ -1463,6 +1489,74 @@ mod tests {
         }
     }
 
+    #[test]
+    fn iam_export_excludes_site_replicator_credentials() {
+        let mut accounts = HashMap::from([
+            (
+                "site-replicator-0".to_owned(),
+                rustfs_policy::auth::UserIdentity::new(Credentials {
+                    access_key: "site-replicator-0".to_owned(),
+                    secret_key: "replicator-secret".to_owned(),
+                    ..Default::default()
+                }),
+            ),
+            (
+                "ordinary-service".to_owned(),
+                rustfs_policy::auth::UserIdentity::new(Credentials {
+                    access_key: "ordinary-service".to_owned(),
+                    secret_key: "ordinary-secret".to_owned(),
+                    ..Default::default()
+                }),
+            ),
+            (
+                "mismatched-entry".to_owned(),
+                rustfs_policy::auth::UserIdentity::new(Credentials {
+                    access_key: "site-replicator-0".to_owned(),
+                    secret_key: "aliased-replicator-secret".to_owned(),
+                    ..Default::default()
+                }),
+            ),
+        ]);
+        super::retain_exportable_service_accounts(&mut accounts);
+        assert_eq!(accounts.len(), 1);
+        assert_eq!(accounts["ordinary-service"].credentials.secret_key, "ordinary-secret");
+        let exported = serde_json::to_string(&accounts).expect("serialize selected export records");
+        assert!(!exported.contains("replicator"));
+    }
+
+    #[test]
+    fn iam_archives_preserve_the_reserved_site_replicator_identity() {
+        let failure = super::imported_service_account_reserved_failure(rustfs_iam::sys::SITE_REPLICATOR_SERVICE_ACCOUNT)
+            .expect("the site replicator must never be replaced by an IAM import");
+        assert_eq!(failure.name, "site-replicator-0");
+        assert_eq!(failure.error, "service account name is reserved for site replication");
+        assert!(super::imported_service_account_reserved_failure("ordinary-service-account").is_none());
+        let source = include_str!("user.rs")
+            .split("\n#[cfg(test)]\n")
+            .next()
+            .expect("production source");
+        let export = source_block(source, "impl Operation for ExportIam");
+        let guard = export
+            .find("retain_exportable_service_accounts(&mut service_accounts)")
+            .expect("export uses the canonical reserved identity");
+        assert!(
+            guard
+                < export
+                    .find("let svc_acc_create_req = SRSvcAccCreate")
+                    .expect("exported credentials")
+        );
+        let import = source_block(source, "impl Operation for ImportIam");
+        let guard = import
+            .find("imported_service_account_reserved_failure(&ak)")
+            .expect("import rejects the reserved identity");
+        assert!(
+            guard
+                < import
+                    .find(".delete_service_account(&ak")
+                    .expect("replacement deletes the original account")
+        );
+    }
+
     #[test]
     fn import_iam_enqueues_a_site_replication_snapshot() {
         let body = source_block(include_str!("user.rs"), "impl Operation for ImportIam");

rustfs/src/admin/route_policy.rs

@@ -18,7 +18,6 @@ use rustfs_security_governance::{
 };
 
 const ALL_ADMIN: AdminActionRef = AdminActionRef::new("AllAdminActions");
-const ADD_USER_TO_GROUP: AdminActionRef = AdminActionRef::new("AddUserToGroupAdminAction");
 const ATTACH_POLICY: AdminActionRef = AdminActionRef::new("AttachPolicyAdminAction");
 const CONFIG_UPDATE: AdminActionRef = AdminActionRef::new("ConfigUpdateAdminAction");
 const TLS_CERTIFICATE_PUBLISH: AdminActionRef = AdminActionRef::new("TLSCertificatePublishAdminAction");
@@ -187,12 +186,6 @@ pub const ADMIN_ROUTE_POLICY_SPECS: &[AdminRouteSpec] = &[
         RouteRiskLevel::High,
     ),
     admin(HttpMethod::Put, "/rustfs/admin/v3/set-group-status", ENABLE_GROUP, RouteRiskLevel::High),
-    admin(
-        HttpMethod::Put,
-        "/rustfs/admin/v3/update-group-members",
-        ADD_USER_TO_GROUP,
-        RouteRiskLevel::High,
-    ),
     admin(
         HttpMethod::Get,
         "/rustfs/admin/v3/temporary-account-info",
@@ -1806,6 +1799,11 @@ pub const ADMIN_ROUTE_POLICY_SPECS: &[AdminRouteSpec] = &[
 ];
 
 pub const DEFERRED_ADMIN_ROUTE_POLICIES: &[DeferredAdminRoutePolicy] = &[
+    deferred(
+        HttpMethod::Put,
+        "/rustfs/admin/v3/update-group-members",
+        DeferredRoutePolicyReason::MultipleActions,
+    ),
     deferred(HttpMethod::Get, "/rustfs/admin/v3/accountinfo", DeferredRoutePolicyReason::S3Action),
     // The self-service account routes act on the caller, never on a target
     // named in the request, so they gate on possession of the credential (plus,

rustfs/src/admin/service/federated_identity.rs

@@ -20,7 +20,7 @@ use crate::admin::{
 use rustfs_iam::{
     federation::{FederatedSessionBinding, FederatedSessionBindingError, FederatedSessionTransaction, OIDC_VIRTUAL_PARENT_CLAIM},
     store::{MappedPolicy, object::ObjectStore},
-    sys::{IamSys, is_safe_claim_policy_name},
+    sys::{IamSys, OIDC_ISSUER_CLAIM, is_safe_claim_policy_name},
 };
 use rustfs_madmin::{SITE_REPL_API_VERSION, SR_IAM_ITEM_STS_ACC, SRIAMItem, SRSTSCredential};
 use rustfs_policy::auth::get_new_credentials_with_metadata;
@@ -111,6 +111,9 @@ fn build_oidc_token_claims(transaction: &FederatedSessionTransaction) -> HashMap
     let mut token_claims = HashMap::new();
     token_claims.insert("sub".to_string(), Value::String(authorization.subject().to_string()));
     token_claims.insert("iss".to_string(), Value::String("rustfs-oidc".to_string()));
+    if let Some(issuer) = authorization.issuer() {
+        token_claims.insert(OIDC_ISSUER_CLAIM.to_string(), Value::String(issuer.to_string()));
+    }
     token_claims.insert("oidc_provider".to_string(), Value::String(authorization.provider_id().to_string()));
 
     if !authorization.email().is_empty() {
@@ -329,14 +332,31 @@ impl FederatedSessionBinding for DefaultFederatedSessionBinding {
         let parent_user = authorization.oidc_virtual_parent().ok_or_else(|| {
             FederatedSessionBindingError::InvalidRequest("verified OIDC identity is missing issuer or subject".to_string())
         })?;
-        let (selected_policy_names, ignorable_policy_names) = match iam_store
-            .sts_policy_db_get(&parent_user, &Some(authorization.mapped_iam_groups().to_vec()))
+        let (mapped_policy_names, has_identity_mapping) = iam_store
+            .sts_policy_db_get_with_identity_mapping(&parent_user, &Some(authorization.mapped_iam_groups().to_vec()))
             .await
-            .map_err(|_| FederatedSessionBindingError::Internal("failed to resolve OIDC policy mapping".to_string()))?
-        {
-            mapped_policy_names if !mapped_policy_names.is_empty() => (mapped_policy_names, &[][..]),
-            _ => (authorization.policies().to_vec(), authorization.group_claim_policies()),
+            .map_err(|_| FederatedSessionBindingError::Internal("failed to resolve OIDC policy mapping".to_string()))?;
+        // An explicit identity mapping overrides token policies. Group mappings
+        // are additive and must not discard claim-derived Deny policies.
+        let ignorable_policy_names: Vec<String> = if has_identity_mapping {
+            Vec::new()
+        } else {
+            authorization
+                .group_claim_policies()
+                .iter()
+                .filter(|name| !mapped_policy_names.contains(name))
+                .cloned()
+                .collect()
         };
+        let mut selected_policy_names = mapped_policy_names;
+        if !has_identity_mapping {
+            for name in authorization.policies() {
+                if !selected_policy_names.contains(name) {
+                    selected_policy_names.push(name.clone());
+                }
+            }
+        }
+
         // A name may be skipped only after storage confirms the policy is absent. A failed lookup
         // must not be mistaken for absence, or an unreadable Deny policy would be dropped from the
         // signed session and could not be restored at request time.
@@ -368,7 +388,7 @@ impl FederatedSessionBinding for DefaultFederatedSessionBinding {
         let resolved_policy_names = MappedPolicy::new(&resolved_policy_mapping).to_slice();
         let selection = select_oidc_policies(
             &selected_policy_names,
-            ignorable_policy_names,
+            &ignorable_policy_names,
             &resolved_policy_names,
             &missing_policy_names,
         )
@@ -543,6 +563,7 @@ mod tests {
             HashMap::from([
                 ("sub".to_string(), serde_json::json!("subject")),
                 ("iss".to_string(), serde_json::json!("rustfs-oidc")),
+                (OIDC_ISSUER_CLAIM.to_string(), serde_json::json!("https://idp.example.test")),
                 ("oidc_provider".to_string(), serde_json::json!("default")),
                 ("email".to_string(), serde_json::json!("user@example.com")),
                 ("preferred_username".to_string(), serde_json::json!("user")),
@@ -566,6 +587,7 @@ mod tests {
             HashMap::from([
                 ("sub".to_string(), serde_json::json!("subject")),
                 ("iss".to_string(), serde_json::json!("rustfs-oidc")),
+                (OIDC_ISSUER_CLAIM.to_string(), serde_json::json!("https://idp.example.test")),
                 ("oidc_provider".to_string(), serde_json::json!("default")),
             ])
         );
@@ -641,6 +663,7 @@ mod tests {
             HashMap::from([
                 ("sub".to_string(), serde_json::json!("subject")),
                 ("iss".to_string(), serde_json::json!("rustfs-oidc")),
+                (OIDC_ISSUER_CLAIM.to_string(), serde_json::json!("https://idp.example.test")),
                 ("oidc_provider".to_string(), serde_json::json!("default")),
                 ("email".to_string(), serde_json::json!("user@example.com")),
                 ("preferred_username".to_string(), serde_json::json!("user")),
@@ -684,6 +707,153 @@ mod tests {
         );
     }
 
+    #[tokio::test]
+    #[serial]
+    async fn binding_group_policies_preserve_claim_denies() {
+        let iam = ready_test_iam().await;
+        let deny = Policy::parse_config(
+            br#"{"Version":"2012-10-17","Statement":[{
+            "Effect":"Deny","Action":"s3:GetObject","Resource":"arn:aws:s3:::payroll/*"
+        }]}"#,
+        )
+        .expect("parse payroll Deny");
+        iam.set_policy("claim-deny-payroll", deny).await.expect("store claim Deny");
+        iam.add_users_to_group("claim-policy-group", Vec::new())
+            .await
+            .expect("create mapped group");
+        iam.policy_db_set("claim-policy-group", UserType::Reg, true, "readwrite")
+            .await
+            .expect("attach group Allow");
+        let authorization = || {
+            let mut fixture = authorization_fixture();
+            fixture.sub = "claim-deny-subject".to_string();
+            fixture.policies = vec!["readwrite".to_string(), "claim-deny-payroll".to_string()];
+            fixture.mapped_iam_groups = vec!["claim-policy-group".to_string()];
+            fixture
+        };
+        let credentials = DefaultFederatedSessionBinding
+            .bind(&transaction_from_fixture(authorization()))
+            .await
+            .expect("bind group plus claim policies");
+        let claims = get_claims_for_test(&credentials);
+        let selected = MappedPolicy::new(claims.get("policy").and_then(Value::as_str).expect("policy claim")).to_slice();
+        assert!(selected.contains(&"claim-deny-payroll".to_string()));
+        let conditions = HashMap::new();
+        let args = Args {
+            account: &credentials.access_key,
+            groups: &credentials.groups,
+            action: Action::S3Action(S3Action::GetObjectAction),
+            bucket: "payroll",
+            object: "salary.csv",
+            conditions: &conditions,
+            claims: &claims,
+            is_owner: false,
+            deny_only: false,
+        };
+        assert!(!iam.is_allowed(&args).await, "group Allow must not discard claim Deny");
+        let mut authorization = authorization();
+        authorization.policies.push("missing-required-policy".to_string());
+        assert!(
+            DefaultFederatedSessionBinding
+                .bind(&transaction_from_fixture(authorization))
+                .await
+                .is_err(),
+            "group mappings must not hide unresolved required claim policies"
+        );
+    }
+
+    fn get_claims_for_test(credentials: &rustfs_credentials::Credentials) -> HashMap<String, Value> {
+        let signing_key = current_token_signing_key().expect("test signing key");
+        rustfs_iam::sys::get_claims_from_token_with_secret(&credentials.session_token, &signing_key)
+            .expect("verify issued session token")
+    }
+
+    #[tokio::test]
+    #[serial]
+    async fn group_conditions_follow_live_parent_membership_for_all_credentials() {
+        let iam = ready_test_iam().await;
+        let user = "live-group-condition-user";
+        let group = "live-restricted-group";
+        iam.create_user(
+            user,
+            &AddOrUpdateUserReq {
+                secret_key: "live-group-user-secret".to_string(),
+                policy: None,
+                status: AccountStatus::Enabled,
+            },
+        )
+        .await
+        .expect("create group condition user");
+        let policy = Policy::parse_config(
+            br#"{"Version":"2012-10-17","Statement":[
+            {"Effect":"Allow","Action":"s3:GetObject","Resource":"arn:aws:s3:::bucket/*"},
+            {"Effect":"Deny","Action":"s3:GetObject","Resource":"arn:aws:s3:::bucket/*",
+             "Condition":{"StringEquals":{"aws:groups":"live-restricted-group"}}}
+        ]}"#,
+        )
+        .expect("parse group condition policy");
+        for member in [true, false, true] {
+            if member {
+                iam.add_users_to_group(group, vec![user.to_string()])
+                    .await
+                    .expect("add membership");
+            } else {
+                iam.remove_users_from_group(group, vec![user.to_string()])
+                    .await
+                    .expect("remove membership");
+            }
+            for kind in ["regular", "sts", "service"] {
+                let mut credentials = rustfs_credentials::Credentials {
+                    access_key: if kind == "regular" {
+                        user
+                    } else {
+                        "derived-group-condition-key"
+                    }
+                    .to_string(),
+                    secret_key: "valid-derived-secret".to_string(),
+                    status: "on".to_string(),
+                    groups: Some(vec!["credential-group".to_string()]),
+                    ..Default::default()
+                };
+                if kind != "regular" {
+                    credentials.parent_user = user.to_string();
+                    credentials.session_token = "signed-token".to_string();
+                }
+                credentials.claims = Some(HashMap::from([("groups".to_string(), serde_json::json!(["token-group"]))]));
+                if kind == "service" {
+                    credentials
+                        .claims
+                        .as_mut()
+                        .expect("claims")
+                        .insert("sa-policy".to_string(), serde_json::json!("inherited-policy"));
+                }
+                let conditions = crate::auth::get_condition_values(&http::HeaderMap::new(), &credentials, None, None, None);
+                let groups = conditions.get("groups").expect("credential and token groups");
+                assert!(groups.contains(&"token-group".to_string()));
+                assert!(groups.contains(&"credential-group".to_string()));
+                assert_eq!(groups.contains(&group.to_string()), member, "{kind}");
+                let claims = HashMap::new();
+                assert_eq!(
+                    policy
+                        .is_allowed(&Args {
+                            account: &credentials.access_key,
+                            groups: &credentials.groups,
+                            action: Action::S3Action(S3Action::GetObjectAction),
+                            bucket: "bucket",
+                            object: "object",
+                            conditions: &conditions,
+                            claims: &claims,
+                            is_owner: false,
+                            deny_only: false,
+                        })
+                        .await,
+                    !member,
+                    "{kind}"
+                );
+            }
+        }
+    }
+
     #[tokio::test]
     #[serial]
     async fn binding_uses_sts_policy_when_regular_mapping_collides() {

rustfs/src/app/metadata_route.rs

@@ -689,18 +689,14 @@ async fn call_list_object_versions(req: S3Request<Body>, bucket: String) -> S3Re
     let request = req.map_input(|_| input);
     let output = DefaultBucketUsecase::from_global()
         .execute_list_object_versions_m(request)
-        .await
-        .map_err(crate::error::normalize_read_quorum_error)?;
+        .await?;
     xml_response(&output.output)
 }
 
 async fn call_list_objects_v2(req: S3Request<Body>, bucket: String) -> S3Result<S3Response<Body>> {
     let input = list_objects_v2_input(bucket, &req.uri, &req.headers)?;
     let request = req.map_input(|_| input);
-    let output = DefaultBucketUsecase::from_global()
-        .execute_list_objects_v2m(request)
-        .await
-        .map_err(crate::error::normalize_read_quorum_error)?;
+    let output = DefaultBucketUsecase::from_global().execute_list_objects_v2m(request).await?;
     xml_response(&output.output)
 }
 

rustfs/src/app/multipart_usecase.rs

@@ -69,6 +69,7 @@ use super::storage_api::multipart_usecase::sse::{
 };
 use super::storage_api::multipart_usecase::{
     StorageObjectInfo as ObjectInfo, StorageObjectOptions as ObjectOptions, StoragePutObjReader as PutObjReader,
+    validate_object_key,
 };
 use super::trailer_adapter::trailer_source;
 use crate::app::object::{
@@ -584,6 +585,8 @@ impl DefaultMultipartUsecase {
         &self,
         mut req: S3Request<CompleteMultipartUploadInput>,
     ) -> S3Result<S3Response<CompleteMultipartUploadOutput>> {
+        // Uploads created before the key was reserved must not publish it.
+        validate_object_key(&req.input.key, "CompleteMultipartUpload")?;
         reject_presigned_multipart_max_total_object_size_for_other_operation(
             &req.headers,
             req.uri.query(),
@@ -990,6 +993,7 @@ impl DefaultMultipartUsecase {
         &self,
         req: S3Request<CreateMultipartUploadInput>,
     ) -> S3Result<S3Response<CreateMultipartUploadOutput>> {
+        validate_object_key(&req.input.key, "CreateMultipartUpload")?;
         let multipart_max_total_object_size = parse_presigned_multipart_max_total_object_size(
             &req.headers,
             req.uri.query(),
@@ -2706,6 +2710,33 @@ mod tests {
         assert_eq!(s3_op_total(S3Operation::AbortMultipartUpload), before + 1);
     }
 
+    #[tokio::test]
+    async fn multipart_upload_rejects_reserved_directory_markers() {
+        for key in ["__XLDIR__", "prefix__XLDIR__/object", "object__XLDIR__"] {
+            let create = CreateMultipartUploadInput::builder()
+                .bucket("bucket".to_owned())
+                .key(key.to_owned())
+                .build()
+                .expect("valid create request DTO");
+            let error = make_usecase()
+                .execute_create_multipart_upload(build_request(create, Method::POST))
+                .await
+                .expect_err("reserved key must be rejected before accessing storage");
+            assert_eq!(error.code(), &S3ErrorCode::InvalidArgument);
+
+            let complete = CompleteMultipartUploadInput::builder()
+                .bucket("bucket".to_owned())
+                .key(key.to_owned())
+                .upload_id("upload".to_owned())
+                .build()
+                .expect("valid complete request DTO");
+            let error = Box::pin(make_usecase().execute_complete_multipart_upload(build_request(complete, Method::POST)))
+                .await
+                .expect_err("uploads created before the fix must not publish reserved keys");
+            assert_eq!(error.code(), &S3ErrorCode::InvalidArgument);
+        }
+    }
+
     #[tokio::test]
     async fn execute_create_multipart_upload_rejects_invalid_storage_class() {
         let input = CreateMultipartUploadInput::builder()

rustfs/src/app/object/copy.rs

@@ -256,6 +256,12 @@ impl DefaultObjectUsecase {
             validate_archive_content_encoding(&key, content_type.as_deref(), content_encoding.as_deref())?;
             let mut replacement_metadata = metadata.unwrap_or_default();
             namespace_reserved_user_metadata(&mut replacement_metadata);
+            // s3s parsed `input.metadata` before the query hoist ran, so a
+            // hoisted `x-amz-meta-*` is only visible on the headers now.
+            super::super::storage_api::object_usecase::options::extract_metadata_from_mime(
+                &req.headers,
+                &mut replacement_metadata,
+            );
             apply_standard_object_metadata(
                 &mut replacement_metadata,
                 cache_control.as_deref(),

rustfs/src/app/object/extract.rs

@@ -4522,6 +4522,46 @@ mod tests {
         assert!(options.ignore_errors);
     }
 
+    #[test]
+    fn resolve_put_object_extract_options_sees_hoisted_query_controls() {
+        // The AWS SDK presigner moves `x-amz-meta-snowball-*` into the query
+        // string. The request-hoist layer restores them as headers, and the
+        // extraction options must resolve from them before member headers drop
+        // the transport keys — otherwise members land under their bare names
+        // and can overwrite unrelated objects (rustfs/rustfs#9146).
+        let mut query = String::from("X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKID&X-Amz-Signature=deadbeef");
+        query.push_str("&x-amz-meta-snowball-auto-extract=true&x-amz-meta-snowball-prefix=batch/");
+        query.push_str("&x-amz-meta-snowball-ignore-dirs=true&x-amz-meta-minio-snowball-ignore-errors=true");
+        let mut req = http::Request::builder()
+            .method(http::Method::PUT)
+            .uri(format!("/bucket/key?{query}"))
+            .body(())
+            .expect("request");
+        let hoisted_query = req.uri().query().map(str::to_owned);
+        crate::auth::hoist_signed_query_params(req.headers_mut(), hoisted_query.as_deref()).expect("hoist");
+
+        assert!(is_put_object_extract_requested(req.headers()));
+        let options = resolve_put_object_extract_options(req.headers()).unwrap();
+        assert_eq!(options.prefix.as_deref(), Some("batch"));
+        assert!(options.ignore_dirs);
+        assert!(options.ignore_errors);
+
+        // Member headers keep the archive's own metadata and drop the transport
+        // keys, so no member object stores the controls that drove extraction.
+        let member_headers = snowball_member_headers(req.headers());
+        assert!(!member_headers.contains_key(AMZ_SNOWBALL_EXTRACT));
+        assert!(!member_headers.contains_key("x-amz-meta-snowball-prefix"));
+    }
+
+    #[test]
+    fn extract_request_discovery_honors_an_explicit_false_header_over_query() {
+        // Header precedence: `x-amz-meta-snowball-auto-extract: false` must not
+        // fall through to a hoisted `true` of the same key.
+        let mut headers = HeaderMap::new();
+        headers.insert(AMZ_SNOWBALL_EXTRACT, HeaderValue::from_static("false"));
+        assert!(!is_put_object_extract_requested(&headers));
+    }
+
     #[test]
     fn resolve_put_object_extract_options_accepts_standard_headers() {
         let mut headers = HeaderMap::new();

rustfs/src/app/object/put.rs

@@ -18,7 +18,7 @@ use super::*;
 
 use crate::app::trailer_adapter::trailer_source;
 use crate::auth::{RUSTFS_MAX_CONTENT_LENGTH_QUERY, VerifiedPresignedRequest, parse_presigned_put_max_content_length};
-use crate::error::{ClientBodyReadTimeout, UploadLimitExceeded, normalize_read_quorum_error};
+use crate::error::{ClientBodyReadTimeout, UploadLimitExceeded};
 static PUT_FAILURE_LOGS: rustfs_utils::LogThrottle = rustfs_utils::LogThrottle::new(5_000);
 
 #[cfg(test)]
@@ -1117,11 +1117,6 @@ impl PutObjectCompletion {
 
 /// Record the failed completion and hand the error back to the caller.
 fn fail_put_object(completion: PutObjectCompletion, err: S3Error) -> S3Error {
-    let err = if matches!(&completion, PutObjectCompletion::S3(_)) {
-        normalize_read_quorum_error(err)
-    } else {
-        err
-    };
     let result: S3Result<S3Response<()>> = Err(err);
     let _ = completion.complete(&result);
     match result {
@@ -1209,9 +1204,7 @@ impl DefaultObjectUsecase {
     #[instrument(name = "execute_put_object", level = "info", skip(self, _fs, req))]
     #[hotpath::measure(impl_type = "DefaultObjectUsecase", future = true)]
     pub async fn execute_put_object(&self, _fs: &FS, req: S3Request<PutObjectInput>) -> S3Result<S3Response<PutObjectOutput>> {
-        self.execute_put_object_boxed(_fs, req)
-            .await
-            .map_err(normalize_read_quorum_error)
+        self.execute_put_object_boxed(_fs, req).await
     }
 
     fn execute_put_object_boxed<'a>(
@@ -2526,13 +2519,14 @@ mod tests {
     }
 
     #[test]
-    fn failed_put_completion_normalizes_only_s3_errors() {
+    fn failed_put_completion_keeps_write_safe_quorum_code() {
         let error = || S3Error::from(ApiError::from(StorageError::ErasureReadQuorum));
         let err = fail_put_object(PutObjectCompletion::S3(OperationHelper::Disabled), error());
         assert_eq!(err.code(), &S3ErrorCode::ServiceUnavailable);
         assert_eq!(err.message(), Some("The service is unavailable. Please retry."));
+        // Internal completions share the write-safe default; SlowDownRead is GET-only.
         let internal = fail_put_object(PutObjectCompletion::Internal(None), error());
-        assert_eq!(internal.code(), &S3ErrorCode::Custom("SlowDownRead".into()));
+        assert_eq!(internal.code(), &S3ErrorCode::ServiceUnavailable);
     }
     use s3s::dto::{
         DefaultRetention, ObjectLockConfiguration, ObjectLockEnabled, ObjectLockRule, ServerSideEncryptionByDefault,