feat(scanner): add runtime scanner controls and status (#3203)

* feat(scanner): add runtime scanner controls and status

* fix(scanner): validate persisted scanner config

* docs(scanner): clarify start delay behavior

---------

Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>

.agents/skills/code-change-verification/SKILL.md

@@ -41,6 +41,19 @@ Use this skill to review code changes consistently before merge, before release,
   - hidden coupling to shared helpers/constants/features
 - If a point is uncertain, mark it as an open question instead of guessing.
 
+#### Rust-specific checks (apply to all Rust changes)
+
+- **unwrap/expect in production**: Search changed files for `.unwrap()` and `.expect(` outside test modules. Every `unwrap()` in production code must have a justification comment or be replaced with `?`.
+- **Silent type truncation**: Search for `as u8/u16/u32/u64/usize/i8/i16/i32/i64/isize` casts. Every `as` cast must be justified; negative-to-unsigned and large-to-small are bugs by default. Use `try_into()` or explicit clamping.
+- **Unnecessary cloning**: Check `.clone()` calls in loops, per-request paths, and on structs with >5 heap-allocated fields. Consider `Arc`, references, or `Cow<str>`.
+- **Lock ordering**: If the change acquires multiple locks, verify the order matches all other call sites. Document the order in a comment.
+- **Locks across .await**: Flag any `tokio::sync::RwLock`/`Mutex` guard held across an `.await` point without bounded hold time.
+- **Recursion depth**: If the change adds or modifies a recursive function, verify it has a depth limit or uses iterative traversal with an explicit stack.
+- **Error types**: Flag `Result<_, String>`, `Box<dyn Error>`, and missing `Error::source()` implementations in public APIs.
+- **Test assertions**: Every test function must have at least one `assert!`. Flag tests that only call code without verifying results.
+- **println/eprintln**: Search changed files for `println!`/`eprintln!` outside test modules. Production code must use `tracing` macros.
+- **Serde safety**: Structs deserialized from untrusted input (S3 API, user config) should have `#[serde(deny_unknown_fields)]`.
+
 ### 4) Findings-first output
 - Order findings by severity:
   - P0: critical failure, security breach, or data loss risk

.agents/skills/rust-code-quality/SKILL.md

@@ -0,0 +1,112 @@
+---
+name: rust-code-quality
+description: Enforce Rust-specific code quality rules on every code change. Use before merge to catch unwrap abuse, silent truncation, unnecessary cloning, lock ordering violations, recursion risks, and error type anti-patterns.
+---
+
+# Rust Code Quality Gate
+
+Use this skill on every Rust code change to enforce quality rules that `cargo clippy` does not catch.
+
+## Quick Start
+
+1. Identify changed `.rs` files.
+2. Run automated checks on changed files.
+3. Run manual review checklist on the diff.
+4. Report findings; block merge if P0/P1 issues exist.
+
+## Automated Checks
+
+Run these on every changed `.rs` file (excluding test modules):
+
+```bash
+# 1. unwrap/expect in production code
+rg -n '\.unwrap\(\)|\.expect\(' <changed-files> | grep -v '#\[cfg(test)\]' | grep -v 'test' | grep -v 'bench'
+
+# 2. Silent type truncation via `as` cast
+rg -n ' as (u8|u16|u32|u64|usize|i8|i16|i32|i64|isize)\b' <changed-files>
+
+# 3. String as error type
+rg -n 'Result<.*String>' <changed-files> | grep -v test
+
+# 4. Box<dyn Error> in public APIs
+rg -n 'Box<dyn.*Error' <changed-files> | grep -v test
+
+# 5. println/eprintln in production
+rg -n 'println!\|eprintln!' <changed-files> | grep -v test
+
+# 6. Ordering::Relaxed usage (verify each is intentional)
+rg -n 'Ordering::Relaxed' <changed-files>
+```
+
+## Manual Review Checklist
+
+For every Rust code change, verify:
+
+### Error Handling
+- [ ] No `unwrap()` or `expect()` in production code without justification comment
+- [ ] No `Result<_, String>` in public API signatures
+- [ ] No `Box<dyn Error>` in public trait/struct methods
+- [ ] `Error::source()` is overridden when inner error is stored
+- [ ] Error messages are actionable (what failed, with what input)
+
+### Type Safety
+- [ ] No silent `as` truncation (negative→unsigned, large→small)
+- [ ] `try_into()` or explicit clamping used for numeric conversions
+- [ ] No `f64 as usize` without prior clamping
+
+### Concurrency
+- [ ] Lock acquisition order is documented when multiple locks are used
+- [ ] No `tokio::sync` write guards held across `.await` without bounded hold time
+- [ ] Concurrent counters use `compare_exchange` loops, not load-then-store
+- [ ] `std::sync::Mutex` in async context is held only briefly, never across `.await`
+
+### Memory and Performance
+- [ ] No `.clone()` on structs with >5 heap-allocated fields in hot paths
+- [ ] `HashMap::with_capacity()` / `Vec::with_capacity()` used when size is known
+- [ ] Large buffers wrapped in `Arc` rather than cloned
+- [ ] Temporary string computations use `&str` or `Cow<str>` instead of `String`
+
+### Recursion Safety
+- [ ] Recursive functions have a depth limit or use iterative traversal
+- [ ] Tree/cache traversals handle corrupted/cyclic input safely
+
+### Testing
+- [ ] Every test function has at least one `assert!`
+- [ ] Tests use `.expect("context")` not bare `.unwrap()`
+- [ ] No `println!`/`eprintln!` in production code (use `tracing`)
+
+### Serde
+- [ ] Structs from untrusted input have `#[serde(deny_unknown_fields)]`
+- [ ] `#[serde(default)]` not used on security-critical fields without validation
+
+### Code Hygiene
+- [ ] No `#![allow(dead_code)]` at crate root
+- [ ] No camelCase statics or Hungarian notation
+- [ ] New string literals don't duplicate existing constants
+
+## Severity Classification
+
+- **P0 (Block merge)**: `unwrap()` in request hot path, silent truncation on user input, lock ordering violation, recursion without depth limit
+- **P1 (Must fix)**: `Result<_, String>` in public API, unnecessary clone in hot path, `Box<dyn Error>` in trait method
+- **P2 (Should fix)**: Missing `assert!` in test, `println!` in production, missing `with_capacity`
+- **P3 (Nice to fix)**: Naming convention violation, missing doc comment, `as_ptr()` vs `Arc::ptr_eq`
+
+## Output Template
+
+```
+## Rust Code Quality Report
+
+### Automated Scan
+- unwrap/expect in production: N found
+- as casts: N found
+- String errors: N found
+- println/eprintln: N found
+
+### Findings
+- [P1] `path:line` — description
+  - Fix: ...
+  - Validation: ...
+
+### Verdict
+PASS / BLOCKED (list blocking findings)
+```

.agents/skills/rust-code-quality/references/checklist.md

@@ -0,0 +1,52 @@
+# Rust Code Quality Checklist
+
+Use this as a quick pre-merge checklist for every Rust code change.
+
+## Critical (P0 — block merge)
+
+| Check | Command |
+|-------|---------|
+| No `unwrap()` in request/storage hot path | `rg '\.unwrap\(\)' <files> \| grep -v test` |
+| No `as` truncation on user input | `rg ' as (u32\|usize\|i32)' <files>` |
+| Lock order consistent across call sites | Manual: trace all lock acquisitions |
+| Recursive functions have depth limit | Manual: check for `max_depth` or iterative pattern |
+| No `panic!`/`unwrap_or_else(panic!)` in production | `rg 'panic!\|unwrap_or_else.*panic' <files> \| grep -v test` |
+
+## High (P1 — must fix)
+
+| Check | Command |
+|-------|---------|
+| No `Result<_, String>` in public API | `rg 'Result<.*String>' <files> \| grep -v test` |
+| No `Box<dyn Error>` in public trait | `rg 'Box<dyn.*Error' <files> \| grep -v test` |
+| No unnecessary `.clone()` in hot path | Manual: check loops and per-request paths |
+| `Error::source()` implemented when inner error stored | Manual: check `impl Error` |
+| No `eprintln!`/`println!` in production | `rg 'println!\|eprintln!' <files> \| grep -v test` |
+
+## Medium (P2 — should fix)
+
+| Check | Command |
+|-------|---------|
+| Tests have assertions | Manual: check for `assert` in test functions |
+| `HashMap`/`Vec` use `with_capacity` when size known | Manual: check `::new()` in loops |
+| No `#![allow(dead_code)]` at crate root | `rg 'allow.dead_code' <files> \| grep 'lib.rs'` |
+| Serde structs from untrusted input have `deny_unknown_fields` | Manual: check `#[derive(Deserialize)]` |
+
+## Low (P3 — nice to fix)
+
+| Check | Command |
+|-------|---------|
+| No camelCase statics | `rg 'static ref [a-z]' <files>` |
+| `Arc::ptr_eq` instead of `as_ptr + ptr::eq` | `rg 'as_ptr\|ptr::eq' <files>` |
+| Public functions have doc comments | `rg 'pub fn' <files> \| grep -v '///'` |
+
+## Quick One-Liner
+
+```bash
+# Run all automated checks on changed files
+CHANGED=$(git diff --name-only HEAD~1 -- '*.rs' | grep -v test | grep -v bench)
+echo "=== unwrap/expect ===" && rg -c '\.unwrap\(\)|\.expect\(' $CHANGED 2>/dev/null
+echo "=== as casts ===" && rg -c ' as (u8|u16|u32|u64|usize|i8|i16|i32|i64|isize)\b' $CHANGED 2>/dev/null
+echo "=== String errors ===" && rg -c 'Result<.*String>' $CHANGED 2>/dev/null
+echo "=== println ===" && rg -c 'println!|eprintln!' $CHANGED 2>/dev/null
+echo "=== Ordering::Relaxed ===" && rg -c 'Ordering::Relaxed' $CHANGED 2>/dev/null
+```

.agents/skills/security-advisory-lessons/SKILL.md

@@ -70,6 +70,7 @@ For the full pattern map, read [advisory-patterns.md](references/advisory-patter
 - Never join untrusted bucket/object/RPC path strings onto filesystem roots without normalization and boundary checks.
 - Reject or safely handle `..`, absolute paths, URL-encoded traversal, platform separators, empty components, and paths that canonicalize outside the intended root.
 - Validate both S3 object-key paths and internode/RPC disk paths; storage helpers can bypass S3 authorization if they trust already-parsed paths.
+- Archive auto-extract paths are object keys too. Validate tar/zip entry names before IAM checks and before storage writes, and prove cleaned paths cannot cross bucket or prefix boundaries.
 
 ### Secrets, default credentials, and crypto
 - Do not ship hard-coded shared tokens, HMAC secrets, private keys, or production test keys.
@@ -86,6 +87,7 @@ For the full pattern map, read [advisory-patterns.md](references/advisory-patter
 - Treat all RPC payload bytes as attacker-controlled. Replace `unwrap`, `expect`, and panic-prone deserialization with typed errors.
 - Malformed request tests should cover empty bytes, truncated MessagePack/protobuf, invalid enum values, stale timestamps, and invalid signatures.
 - RPC authentication must be independently strong; do not depend on S3 admin credentials unless the fallback is explicit and safe.
+- RPC signatures must bind the exact generated gRPC method path, timestamp, and request method. Service-prefix signatures must not authorize a different concrete NodeService call.
 
 ### Browser, CORS, and console surfaces
 - Do not reflect arbitrary `Origin` while also allowing credentials. Default CORS should be no CORS unless explicitly configured.
@@ -115,5 +117,6 @@ Use these prompts while reviewing a diff:
 - Could a low-privileged authenticated user reach this path with the wrong action, parent, bucket, or source object?
 - Does a public/default/empty config change security behavior from fail-closed to fail-open?
 - Is any attacker-controlled value later used as a path, policy condition, credential identity, log field, URL, Origin, or response body?
+- Is an archive entry, object key, or policy resource normalized differently between authorization and storage?
 - Is the same operation implemented in multiple paths, such as `CopyObject` vs `UploadPartCopy`, and do all paths enforce the same security contract?
 - Does the test prove the exploit form is denied, or only that the intended form still works?

.agents/skills/security-advisory-lessons/references/advisory-patterns.md

@@ -38,6 +38,7 @@ Update this file only when an advisory adds or changes a reusable lesson, affect
 
 - `GHSA-pq29-69jg-9mxc`: RPC `read_file_stream` joined untrusted paths under a volume directory without canonical boundary checks. Lesson: `PathBuf::join` plus length checks are not path security.
 - `GHSA-8r6f-hmq2-28rg`: object keys containing traversal sequences bypassed bucket/object authorization when mapped to filesystem paths. Lesson: reject traversal at object-key parsing and verify final storage paths remain under the expected bucket/key root.
+- `GHSA-f4vq-9ffr-m8m3`: Snowball auto-extract accepted archive entries such as `../victim-bucket/object`, authorized the raw attacker-bucket path, then storage path cleaning crossed bucket boundaries. Lesson: archive entries become object keys and need traversal rejection plus consistent authz/storage normalization before writes.
 
 ### Secrets, defaults, and cryptographic misuse
 
@@ -55,6 +56,7 @@ Update this file only when an advisory adds or changes a reusable lesson, affect
 
 - `GHSA-gw2x-q739-qhcr`: malformed gRPC `GetMetrics` payloads reached `unwrap()` on deserialization and caused remote DoS. Lesson: every network/RPC deserialization failure returns an error, not a panic.
 - `GHSA-h956-rh7x-ppgj` and `GHSA-r5qv-rc46-hv8q`: weak RPC auth increased reachability of otherwise internal handlers. Lesson: panic bugs become more severe when internode auth is weak or defaulted.
+- `GHSA-c667-rgrv-99vj`: NodeService authentication signed the service prefix instead of the concrete generated method path, so valid metadata for one RPC could be replayed to another method during the timestamp window. Lesson: RPC HMAC payloads must bind exact gRPC method path, HTTP method surrogate, timestamp, and secret.
 
 ### Browser, CORS, and console isolation
 
@@ -69,6 +71,13 @@ Update this file only when an advisory adds or changes a reusable lesson, affect
 
 - `GHSA-xrrf-67jm-3c2r`: SSE metadata reported encryption while reader composition bypassed `EncryptReader` and stored plaintext. Lesson: test actual bytes on disk and wrapper order, not only API metadata.
 
+### Serde deserialization and input validation
+
+- No `#[serde(deny_unknown_fields)]` found across the entire codebase. Lesson: all structs deserialized from untrusted input (S3 API XML/JSON, lifecycle rules, bucket policies, replication configs) should have `#[serde(deny_unknown_fields)]` to reject malformed or adversarial payloads.
+- `#[serde(default)]` on security-critical fields silently accepts missing values as zero/empty. Lesson: when a field has security implications (retention days, permissions, limits), validate the deserialized value explicitly rather than relying on defaults.
+- Integer fields deserialized from user input and cast with `as` (e.g., `i32 as u32`) can wrap negative values to large positives. Lesson: validate ranges before casting; use `try_into()` or clamp.
+- XML config typos (e.g., `"NoncurentDays"` instead of `"NoncurrentDays"`) are silently accepted when `deny_unknown_fields` is absent. Lesson: strict deserialization prevents silent misconfiguration that could cause data loss or unexpected retention behavior.
+
 ## Useful Search Seeds
 
 Use these targeted searches when a diff touches security-sensitive code:
@@ -76,10 +85,12 @@ Use these targeted searches when a diff touches security-sensitive code:
 ```bash
 rg -n "validate_admin_request|check_permissions|AdminAction::|deny_only|is_allowed" rustfs crates
 rg -n "UploadPartCopy|upload_part_copy|CompleteMultipart|PostObject|content-length-range|starts-with" rustfs crates
-rg -n "PathBuf::join|canonicalize|\\.\\.|x-forwarded-for|x-real-ip|SourceIp" rustfs crates
+rg -n "normalize_extract_entry_key|Snowball|auto-extract|PathBuf::join|canonicalize|\\.\\.|x-forwarded-for|x-real-ip|SourceIp" rustfs crates
 rg -n "DEFAULT_SECRET|DEFAULT_ACCESS|TEST_PRIVATE_KEY|rustfs rpc|RUSTFS_RPC_SECRET" rustfs crates
+rg -n "TONIC_RPC_PREFIX|verify_rpc_signature|check_auth|NodeServiceServer|x-rustfs-signature" rustfs crates
 rg -n "debug!|trace!|info!|error!|\\?resp|\\?merged_config|session_token|secret_key" rustfs crates
 rg -n "HashReader|EncryptReader|SSE|server-side encryption|Access-Control-Allow-Credentials|Origin" rustfs crates
+rg -n "deny_unknown_fields|serde.default|as u32|as usize|as i32" rustfs crates
 ```
 
 ## Minimum Regression Test Expectations
@@ -87,7 +98,8 @@ rg -n "HashReader|EncryptReader|SSE|server-side encryption|Access-Control-Allow-
 - Authz fixes: include unauthenticated, valid low-privilege, wrong-action, correct-action, owner, non-owner, and root/admin cases as applicable.
 - IAM fixes: include import/update/list service-account cases with attacker-controlled parent, claims, access key, secret key, and policy.
 - Copy/upload fixes: include cross-bucket, cross-user, source-denied, destination-denied, copy-source-condition, and multipart completion cases.
-- Path fixes: include encoded traversal, absolute path, nested traversal, valid object keys that resemble traversal text but should be rejected, and canonical boundary checks.
+- Path fixes: include encoded traversal, absolute path, nested traversal, archive entries with `..`, valid object keys that resemble traversal text but should be rejected, and canonical bucket/prefix boundary checks.
 - Logging fixes: assert redacted output for structs and response bodies that may contain credentials.
+- RPC auth fixes: include captured metadata replay across two concrete methods, stale timestamps, wrong path, wrong method surrogate, wrong secret, and valid same-method calls.
 - Browser/CORS fixes: assert no credentials on reflected/default origins, correct behavior for explicit allowlists, and no same-origin script execution for previewed object content.
 - SSE fixes: inspect stored bytes and verify API metadata, read-back behavior, and on-disk ciphertext together.

.docker/observability/grafana/dashboards/rustfs.json

@@ -4551,7 +4551,7 @@
                   "index": 0,
                   "text": "INACTIVE"
                 },
-                "to": 1e-09
+                "to": 1e-9
               },
               "type": "range"
             }
@@ -5281,7 +5281,10 @@
       "id": 102,
       "options": {
         "legend": {
-          "calcs": ["mean", "max"],
+          "calcs": [
+            "mean",
+            "max"
+          ],
           "displayMode": "table",
           "placement": "bottom"
         },
@@ -5368,7 +5371,10 @@
       "id": 103,
       "options": {
         "legend": {
-          "calcs": ["mean", "max"],
+          "calcs": [
+            "mean",
+            "max"
+          ],
           "displayMode": "table",
           "placement": "bottom"
         },
@@ -5455,7 +5461,10 @@
       "id": 104,
       "options": {
         "legend": {
-          "calcs": ["mean", "max"],
+          "calcs": [
+            "mean",
+            "max"
+          ],
           "displayMode": "table",
           "placement": "bottom"
         },
@@ -5533,7 +5542,9 @@
       "id": 105,
       "options": {
         "legend": {
-          "calcs": ["mean"],
+          "calcs": [
+            "mean"
+          ],
           "displayMode": "table",
           "placement": "bottom"
         },
@@ -5629,7 +5640,10 @@
       "id": 106,
       "options": {
         "legend": {
-          "calcs": ["mean", "max"],
+          "calcs": [
+            "mean",
+            "max"
+          ],
           "displayMode": "table",
           "placement": "bottom"
         },
@@ -5725,7 +5739,10 @@
       "id": 107,
       "options": {
         "legend": {
-          "calcs": ["mean", "max"],
+          "calcs": [
+            "mean",
+            "max"
+          ],
           "displayMode": "table",
           "placement": "bottom"
         },
@@ -5803,7 +5820,10 @@
       "id": 108,
       "options": {
         "legend": {
-          "calcs": ["mean", "max"],
+          "calcs": [
+            "mean",
+            "max"
+          ],
           "displayMode": "table",
           "placement": "bottom"
         },
@@ -8071,6 +8091,374 @@
         "x": 0,
         "y": 195
       },
+      "id": 150,
+      "panels": [],
+      "title": "Object Lock Diagnostics",
+      "type": "row"
+    },
+    {
+      "datasource": {
+        "type": "prometheus",
+        "uid": "${datasource}"
+      },
+      "fieldConfig": {
+        "defaults": {
+          "color": {
+            "mode": "palette-classic"
+          },
+          "mappings": [],
+          "thresholds": {
+            "mode": "absolute",
+            "steps": [
+              {
+                "color": "green",
+                "value": 0
+              },
+              {
+                "color": "orange",
+                "value": 1
+              },
+              {
+                "color": "red",
+                "value": 5
+              }
+            ]
+          },
+          "unit": "short"
+        },
+        "overrides": []
+      },
+      "gridPos": {
+        "h": 8,
+        "w": 12,
+        "x": 0,
+        "y": 196
+      },
+      "id": 151,
+      "options": {
+        "legend": {
+          "calcs": [
+            "lastNotNull",
+            "max"
+          ],
+          "displayMode": "table",
+          "placement": "right",
+          "showLegend": true
+        },
+        "tooltip": {
+          "hideZeros": false,
+          "mode": "multi",
+          "sort": "desc"
+        }
+      },
+      "pluginVersion": "12.3.2",
+      "targets": [
+        {
+          "datasource": {
+            "type": "prometheus",
+            "uid": "${datasource}"
+          },
+          "editorMode": "code",
+          "expr": "sum by (op, mode) (increase(rustfs_object_lock_diag_slow_acquire_total{job=~\"$job\"}[$__rate_interval]))",
+          "legendFormat": "{{op}} | {{mode}}",
+          "range": true,
+          "refId": "A"
+        }
+      ],
+      "title": "Slow Object Lock Acquires",
+      "type": "timeseries"
+    },
+    {
+      "datasource": {
+        "type": "prometheus",
+        "uid": "${datasource}"
+      },
+      "fieldConfig": {
+        "defaults": {
+          "color": {
+            "mode": "palette-classic"
+          },
+          "mappings": [],
+          "thresholds": {
+            "mode": "absolute",
+            "steps": [
+              {
+                "color": "green",
+                "value": 0
+              },
+              {
+                "color": "orange",
+                "value": 1
+              },
+              {
+                "color": "red",
+                "value": 5
+              }
+            ]
+          },
+          "unit": "short"
+        },
+        "overrides": []
+      },
+      "gridPos": {
+        "h": 8,
+        "w": 12,
+        "x": 12,
+        "y": 196
+      },
+      "id": 152,
+      "options": {
+        "legend": {
+          "calcs": [
+            "lastNotNull",
+            "max"
+          ],
+          "displayMode": "table",
+          "placement": "right",
+          "showLegend": true
+        },
+        "tooltip": {
+          "hideZeros": false,
+          "mode": "multi",
+          "sort": "desc"
+        }
+      },
+      "pluginVersion": "12.3.2",
+      "targets": [
+        {
+          "datasource": {
+            "type": "prometheus",
+            "uid": "${datasource}"
+          },
+          "editorMode": "code",
+          "expr": "sum by (op, mode) (increase(rustfs_object_lock_diag_slow_hold_total{job=~\"$job\"}[$__rate_interval]))",
+          "legendFormat": "{{op}} | {{mode}}",
+          "range": true,
+          "refId": "A"
+        }
+      ],
+      "title": "Slow Object Lock Holds",
+      "type": "timeseries"
+    },
+    {
+      "datasource": {
+        "type": "prometheus",
+        "uid": "${datasource}"
+      },
+      "fieldConfig": {
+        "defaults": {
+          "color": {
+            "mode": "palette-classic"
+          },
+          "mappings": [],
+          "thresholds": {
+            "mode": "absolute",
+            "steps": [
+              {
+                "color": "green",
+                "value": 0
+              }
+            ]
+          },
+          "unit": "ms"
+        },
+        "overrides": []
+      },
+      "gridPos": {
+        "h": 8,
+        "w": 12,
+        "x": 0,
+        "y": 204
+      },
+      "id": 153,
+      "options": {
+        "legend": {
+          "calcs": [
+            "lastNotNull",
+            "max"
+          ],
+          "displayMode": "table",
+          "placement": "right",
+          "showLegend": true
+        },
+        "tooltip": {
+          "hideZeros": false,
+          "mode": "multi",
+          "sort": "desc"
+        }
+      },
+      "pluginVersion": "12.3.2",
+      "targets": [
+        {
+          "datasource": {
+            "type": "prometheus",
+            "uid": "${datasource}"
+          },
+          "editorMode": "code",
+          "expr": "1000 * histogram_quantile(0.95, sum by (le, op, mode) (rate(rustfs_object_lock_diag_acquire_duration_seconds_bucket{job=~\"$job\"}[5m])))",
+          "legendFormat": "{{op}} | {{mode}}",
+          "range": true,
+          "refId": "A"
+        }
+      ],
+      "title": "Object Lock Acquire Duration P95",
+      "type": "timeseries"
+    },
+    {
+      "datasource": {
+        "type": "prometheus",
+        "uid": "${datasource}"
+      },
+      "fieldConfig": {
+        "defaults": {
+          "color": {
+            "mode": "palette-classic"
+          },
+          "mappings": [],
+          "thresholds": {
+            "mode": "absolute",
+            "steps": [
+              {
+                "color": "green",
+                "value": 0
+              }
+            ]
+          },
+          "unit": "ms"
+        },
+        "overrides": []
+      },
+      "gridPos": {
+        "h": 8,
+        "w": 12,
+        "x": 12,
+        "y": 204
+      },
+      "id": 154,
+      "options": {
+        "legend": {
+          "calcs": [
+            "lastNotNull",
+            "max"
+          ],
+          "displayMode": "table",
+          "placement": "right",
+          "showLegend": true
+        },
+        "tooltip": {
+          "hideZeros": false,
+          "mode": "multi",
+          "sort": "desc"
+        }
+      },
+      "pluginVersion": "12.3.2",
+      "targets": [
+        {
+          "datasource": {
+            "type": "prometheus",
+            "uid": "${datasource}"
+          },
+          "editorMode": "code",
+          "expr": "1000 * histogram_quantile(0.95, sum by (le, op, mode) (rate(rustfs_object_lock_diag_hold_duration_seconds_bucket{job=~\"$job\"}[5m])))",
+          "legendFormat": "{{op}} | {{mode}}",
+          "range": true,
+          "refId": "A"
+        }
+      ],
+      "title": "Object Lock Hold Duration P95",
+      "type": "timeseries"
+    },
+    {
+      "datasource": {
+        "type": "prometheus",
+        "uid": "${datasource}"
+      },
+      "fieldConfig": {
+        "defaults": {
+          "color": {
+            "mode": "thresholds"
+          },
+          "mappings": [
+            {
+              "options": {
+                "0": {
+                  "text": "Disabled"
+                },
+                "1": {
+                  "text": "Enabled"
+                }
+              },
+              "type": "value"
+            }
+          ],
+          "max": 1,
+          "min": 0,
+          "thresholds": {
+            "mode": "absolute",
+            "steps": [
+              {
+                "color": "red",
+                "value": 0
+              },
+              {
+                "color": "green",
+                "value": 1
+              }
+            ]
+          },
+          "unit": "short"
+        },
+        "overrides": []
+      },
+      "gridPos": {
+        "h": 8,
+        "w": 24,
+        "x": 0,
+        "y": 212
+      },
+      "id": 155,
+      "options": {
+        "colorMode": "background",
+        "graphMode": "none",
+        "justifyMode": "auto",
+        "orientation": "auto",
+        "percentChangeColorMode": "standard",
+        "reduceOptions": {
+          "calcs": [
+            "lastNotNull"
+          ],
+          "fields": "",
+          "values": false
+        },
+        "showPercentChange": false,
+        "textMode": "auto",
+        "wideLayout": true
+      },
+      "pluginVersion": "12.3.2",
+      "targets": [
+        {
+          "datasource": {
+            "type": "prometheus",
+            "uid": "${datasource}"
+          },
+          "editorMode": "code",
+          "expr": "max(rustfs_object_lock_diag_enabled{job=~\"$job\"})",
+          "legendFormat": "__auto",
+          "range": true,
+          "refId": "A"
+        }
+      ],
+      "title": "Object Lock Diagnostics Enabled",
+      "type": "stat"
+    },
+    {
+      "collapsed": false,
+      "gridPos": {
+        "h": 1,
+        "w": 24,
+        "x": 0,
+        "y": 213
+      },
       "id": 137,
       "panels": [],
       "title": "Debug / Raw Explorer",
@@ -8138,7 +8526,7 @@
         "h": 8,
         "w": 12,
         "x": 0,
-        "y": 196
+        "y": 214
       },
       "id": 138,
       "options": {
@@ -8235,7 +8623,7 @@
         "h": 8,
         "w": 12,
         "x": 12,
-        "y": 196
+        "y": 214
       },
       "id": 139,
       "options": {
@@ -8332,7 +8720,7 @@
         "h": 8,
         "w": 12,
         "x": 0,
-        "y": 204
+        "y": 222
       },
       "id": 140,
       "options": {
@@ -8429,7 +8817,7 @@
         "h": 8,
         "w": 12,
         "x": 12,
-        "y": 204
+        "y": 222
       },
       "id": 141,
       "options": {
@@ -8526,7 +8914,7 @@
         "h": 8,
         "w": 12,
         "x": 0,
-        "y": 212
+        "y": 230
       },
       "id": 142,
       "options": {
@@ -8623,7 +9011,7 @@
         "h": 8,
         "w": 12,
         "x": 12,
-        "y": 212
+        "y": 230
       },
       "id": 143,
       "options": {
@@ -8720,7 +9108,7 @@
         "h": 8,
         "w": 12,
         "x": 0,
-        "y": 220
+        "y": 238
       },
       "id": 144,
       "options": {
@@ -8817,7 +9205,7 @@
         "h": 8,
         "w": 12,
         "x": 12,
-        "y": 220
+        "y": 238
       },
       "id": 145,
       "options": {
@@ -8914,7 +9302,7 @@
         "h": 8,
         "w": 12,
         "x": 0,
-        "y": 228
+        "y": 246
       },
       "id": 146,
       "options": {
@@ -9011,7 +9399,7 @@
         "h": 8,
         "w": 12,
         "x": 12,
-        "y": 228
+        "y": 246
       },
       "id": 147,
       "options": {
@@ -9108,7 +9496,7 @@
         "h": 8,
         "w": 12,
         "x": 0,
-        "y": 236
+        "y": 254
       },
       "id": 148,
       "options": {
@@ -9205,7 +9593,7 @@
         "h": 8,
         "w": 12,
         "x": 12,
-        "y": 236
+        "y": 254
       },
       "id": 149,
       "options": {
@@ -9382,5 +9770,5 @@
   "timezone": "browser",
   "title": "RustFS",
   "uid": "rustfs-s3",
-  "version": 13
+  "version": 14
 }

.github/actions/setup/action.yml

@@ -54,6 +54,7 @@ runs:
           build-essential \
           pkg-config \
           libssl-dev \
+          unzip \
           protobuf-compiler
 
     - name: Install protoc
@@ -83,7 +84,7 @@ runs:
       uses: taiki-e/install-action@cargo-zigbuild
 
     - name: Install cargo-nextest
-      uses: taiki-e/install-action@cargo-nextest
+      uses: taiki-e/install-action@nextest
 
     - name: Setup Rust cache
       uses: Swatinem/rust-cache@v2

.github/dependabot.yml

@@ -40,6 +40,8 @@ updates:
         versions: [ "1.x" ]
       - dependency-name: "ratelimit"
         versions: [ "2.x" ]
+      - dependency-name: "pyroscope"
+        versions: [ "2.x" ]
     groups:
       s3s:
         update-types:

.github/s3tests/s3tests.conf

@@ -11,8 +11,8 @@
 # host set for RustFS - will be substituted via envsubst
 host = ${S3_HOST}
 
-# port for RustFS
-port = 9000
+# port for RustFS - will be substituted via envsubst
+port = ${S3_PORT}
 
 ## say "False" to disable TLS
 is_secure = False

.github/workflows/audit.yml

@@ -40,7 +40,7 @@ env:
 jobs:
   security-audit:
     name: Security Audit
-    runs-on: ubicloud-standard-2
+    runs-on: ubuntu-latest
     timeout-minutes: 15
     steps:
       - name: Checkout repository
@@ -65,7 +65,7 @@ jobs:
 
   dependency-review:
     name: Dependency Review
-    runs-on: ubicloud-standard-2
+    runs-on: ubuntu-latest
     if: github.event_name == 'pull_request'
     permissions:
       contents: read

.github/workflows/build.yml

@@ -73,7 +73,7 @@ jobs:
   # Build strategy check - determine build type based on trigger
   build-check:
     name: Build Strategy Check
-    runs-on: ubicloud-standard-2
+    runs-on: ubuntu-latest
     outputs:
       should_build: ${{ steps.check.outputs.should_build }}
       build_type: ${{ steps.check.outputs.build_type }}
@@ -146,7 +146,7 @@ jobs:
   # Build RustFS binaries
   prepare-platform-matrix:
     name: Prepare Platform Matrix
-    runs-on: ubicloud-standard-2
+    runs-on: ubuntu-latest
     outputs:
       matrix: ${{ steps.select.outputs.matrix }}
       selected: ${{ steps.select.outputs.selected }}
@@ -169,7 +169,7 @@ jobs:
             {"target_id":"linux-x86_64-gnu","os":"ubicloud-standard-2","target":"x86_64-unknown-linux-gnu","cross":false,"platform":"linux","rustflags":""},
             {"target_id":"linux-aarch64-gnu","os":"ubicloud-standard-2","target":"aarch64-unknown-linux-gnu","cross":true,"platform":"linux","rustflags":""},
             {"target_id":"macos-aarch64","os":"macos-latest","target":"aarch64-apple-darwin","cross":false,"platform":"macos","rustflags":""},
-            {"target_id":"macos-x86_64","os":"macos-15-intel","target":"x86_64-apple-darwin","cross":false,"platform":"macos","rustflags":""},
+            {"target_id":"macos-x86_64","os":"macos-15-intel","target":"x86_64-apple-darwin","cross":true,"platform":"macos","rustflags":""},
             {"target_id":"windows-x86_64","os":"windows-latest","target":"x86_64-pc-windows-msvc","cross":false,"platform":"windows","rustflags":""}
           ]}'
 
@@ -549,7 +549,7 @@ jobs:
     name: Build Summary
     needs: [ build-check, build-rustfs ]
     if: always() && needs.build-check.outputs.should_build == 'true'
-    runs-on: ubicloud-standard-2
+    runs-on: ubuntu-latest
     steps:
       - name: Build completion summary
         shell: bash
@@ -601,7 +601,7 @@ jobs:
     name: Create GitHub Release
     needs: [ build-check, build-rustfs ]
     if: startsWith(github.ref, 'refs/tags/') && needs.build-check.outputs.build_type != 'development'
-    runs-on: ubicloud-standard-2
+    runs-on: ubuntu-latest
     permissions:
       contents: write
     outputs:
@@ -687,7 +687,7 @@ jobs:
     name: Upload Release Assets
     needs: [ build-check, build-rustfs, create-release ]
     if: startsWith(github.ref, 'refs/tags/') && needs.build-check.outputs.build_type != 'development'
-    runs-on: ubicloud-standard-2
+    runs-on: ubuntu-latest
     permissions:
       contents: write
       actions: read
@@ -768,7 +768,7 @@ jobs:
     name: Update Latest Version
     needs: [ build-check, upload-release-assets ]
     if: startsWith(github.ref, 'refs/tags/')
-    runs-on: ubicloud-standard-2
+    runs-on: ubuntu-latest
     steps:
       - name: Update latest.json
         env:
@@ -793,9 +793,10 @@ jobs:
 
           curl -o "$OSSUTIL_ZIP" "https://gosspublic.alicdn.com/ossutil/v2/${OSSUTIL_VERSION}/${OSSUTIL_ZIP}"
           unzip "$OSSUTIL_ZIP"
-          mv "${OSSUTIL_DIR}/ossutil" /usr/local/bin/
+          OSSUTIL_BIN="${RUNNER_TEMP}/ossutil"
+          mv "${OSSUTIL_DIR}/ossutil" "$OSSUTIL_BIN"
           rm -rf "$OSSUTIL_DIR" "$OSSUTIL_ZIP"
-          chmod +x /usr/local/bin/ossutil
+          chmod +x "$OSSUTIL_BIN"
 
           # Create latest.json
           cat > latest.json << EOF
@@ -809,7 +810,7 @@ jobs:
           EOF
 
           # Upload to OSS
-          ossutil cp latest.json oss://rustfs-version/latest.json --force
+          "$OSSUTIL_BIN" cp latest.json oss://rustfs-version/latest.json --force
 
           echo "✅ Updated latest.json for stable release $VERSION"
 
@@ -818,7 +819,7 @@ jobs:
     name: Publish Release
     needs: [ build-check, create-release, upload-release-assets ]
     if: startsWith(github.ref, 'refs/tags/') && needs.build-check.outputs.build_type != 'development'
-    runs-on: ubicloud-standard-2
+    runs-on: ubuntu-latest
     permissions:
       contents: write
     steps:

.github/workflows/ci.yml

@@ -78,7 +78,7 @@ jobs:
     permissions:
       actions: write
       contents: read
-    runs-on: ubicloud-standard-2
+    runs-on: ubuntu-latest
     outputs:
       should_skip: ${{ steps.skip_check.outputs.should_skip }}
     steps:
@@ -95,7 +95,7 @@ jobs:
     name: Typos
     needs: skip-check
     if: needs.skip-check.outputs.should_skip != 'true'
-    runs-on: ubicloud-standard-2
+    runs-on: ubuntu-latest
     steps:
       - uses: actions/checkout@v6
       - uses: dtolnay/rust-toolchain@stable
@@ -182,11 +182,6 @@ jobs:
       - name: Checkout repository
         uses: actions/checkout@v6
 
-      - name: Clean up previous test run
-        run: |
-          rm -rf /tmp/rustfs
-          rm -f /tmp/rustfs.log
-
       - name: Download debug binary
         uses: actions/download-artifact@v7
         with:
@@ -209,14 +204,19 @@ jobs:
       - name: Run end-to-end tests
         run: |
           s3s-e2e --version
-          ./scripts/e2e-run.sh ./target/debug/rustfs /tmp/rustfs
+          RUN_ROOT="${RUNNER_TEMP}/rustfs-e2e-${GITHUB_RUN_ID}-${GITHUB_RUN_ATTEMPT}-${GITHUB_JOB}"
+          mkdir -p "${RUN_ROOT}"
+          RUSTFS_TEST_PORT="$(python3 -c 'import socket; s=socket.socket(); s.bind(("127.0.0.1", 0)); print(s.getsockname()[1]); s.close()')"
+          RUSTFS_TEST_PORT="${RUSTFS_TEST_PORT}" \
+          RUSTFS_TEST_LOG="${RUN_ROOT}/rustfs.log" \
+          ./scripts/e2e-run.sh ./target/debug/rustfs "${RUN_ROOT}/data"
 
       - name: Upload test logs
         if: failure()
         uses: actions/upload-artifact@v6
         with:
           name: e2e-test-logs-${{ github.run_number }}
-          path: /tmp/rustfs.log
+          path: ${{ runner.temp }}/rustfs-e2e-*/rustfs.log
           retention-days: 3
 
   s3-implemented-tests:
@@ -240,10 +240,16 @@ jobs:
 
       - name: Run implemented s3-tests
         run: |
+          RUN_ROOT="${RUNNER_TEMP}/rustfs-s3tests-${GITHUB_RUN_ID}-${GITHUB_RUN_ATTEMPT}-${GITHUB_JOB}"
+          mkdir -p "${RUN_ROOT}"
+          S3_PORT="$(python3 -c 'import socket; s=socket.socket(); s.bind(("127.0.0.1", 0)); print(s.getsockname()[1]); s.close()')"
           DEPLOY_MODE=binary \
           RUSTFS_BINARY=./target/debug/rustfs \
           TEST_MODE=single \
           MAXFAIL=1 \
+          S3_PORT="${S3_PORT}" \
+          DATA_ROOT="${RUN_ROOT}" \
+          S3TESTS_CONF=artifacts/s3tests-single/s3tests.conf \
           ./scripts/s3-tests/run.sh
 
       - name: Upload s3 test artifacts

.github/workflows/docker.yml

@@ -73,7 +73,7 @@ jobs:
   # Check if we should build Docker images
   build-check:
     name: Docker Build Check
-    runs-on: ubicloud-standard-2
+    runs-on: ubuntu-latest
     outputs:
       should_build: ${{ steps.check.outputs.should_build }}
       should_push: ${{ steps.check.outputs.should_push }}
@@ -421,7 +421,7 @@ jobs:
     name: Docker Build Summary
     needs: [ build-check, build-docker ]
     if: always() && needs.build-check.outputs.should_build == 'true'
-    runs-on: ubicloud-standard-2
+    runs-on: ubuntu-latest
     steps:
       - name: Docker build completion summary
         run: |

.github/workflows/helm-package.yml

@@ -31,7 +31,7 @@ permissions:
 
 jobs:
   build-helm-package:
-    runs-on: ubicloud-standard-2
+    runs-on: ubuntu-latest
     if: |
       github.event_name == 'workflow_dispatch' ||
       (
@@ -99,7 +99,7 @@ jobs:
           retention-days: 1
 
   publish-helm-package:
-    runs-on: ubicloud-standard-2
+    runs-on: ubuntu-latest
     needs: [ build-helm-package ]
     if: needs.build-helm-package.result == 'success'
 

.github/workflows/issue-translator.yml

@@ -25,7 +25,7 @@ permissions:
 
 jobs:
   build:
-    runs-on: ubicloud-standard-4
+    runs-on: ubuntu-latest
     steps:
       - uses: usthe/issues-translate-action@v2.7
         with:

AGENTS.md

@@ -103,6 +103,18 @@ Do not open a PR with code changes when the required checks fail.
 - Use environment variables or vault tooling for sensitive configuration.
 - For localhost-sensitive tests, verify proxy settings to avoid traffic leakage.
 
+## Serde Safety
+
+- Add `#[serde(deny_unknown_fields)]` to structs deserialized from untrusted input (S3 API XML/JSON, lifecycle rules, bucket policies, replication configs).
+- When `deny_unknown_fields` is impractical (backward compatibility), at minimum log unknown fields at `warn` level.
+- Never use `#[serde(default)]` on security-critical fields without explicit validation of the resulting value.
+
+## Naming Conventions
+
+- Follow Rust API Guidelines for naming: `SCREAMING_SNAKE_CASE` for statics and constants, `snake_case` for functions and variables, `PascalCase` for types.
+- Do not use camelCase or Hungarian notation (e.g., `globalDeploymentIDPtr` → `GLOBAL_DEPLOYMENT_ID`).
+- If existing code violates naming conventions, do not widen the violation in new code. Fix opportunistically when touching the surrounding area.
+
 ## Scoped Guidance in This Repository
 
 - `.github/AGENTS.md`

Cargo.lock

@@ -812,9 +812,9 @@ checksum = "f2032f911046de80f0a198e0901378627c33f59ea0ac00e363d481118bd70a53"
 
 [[package]]
 name = "aws-config"
-version = "1.8.17"
+version = "1.8.18"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "517aa062d8bd9015ee23d6daa5e1c1372328412fdae4e6c4c1be9b69c6ad37a2"
+checksum = "e33f815b73a3899c03b380d543532e5865f230dce9678d108dc10732a8682275"
 dependencies = [
  "aws-credential-types",
  "aws-runtime",
@@ -906,10 +906,11 @@ dependencies = [
 
 [[package]]
 name = "aws-sdk-s3"
-version = "1.134.0"
+version = "1.135.0"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "be06bdfdf00371318253d74776567512d1229d1f3cd5546d27d333c89e013b84"
+checksum = "f97e3e7e7d86fd26fcdc18bc382da5ca9e8b2ff8d54030d187fd0dac8a236d96"
 dependencies = [
+ "arc-swap",
  "aws-credential-types",
  "aws-runtime",
  "aws-sigv4",
@@ -941,10 +942,11 @@ dependencies = [
 
 [[package]]
 name = "aws-sdk-sso"
-version = "1.100.0"
+version = "1.101.0"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "bee2719d4a5e5e147bb9e9b77490df6ece750df1094968aa857b09b618a1881a"
+checksum = "b647baea49ff551960b904f905681e9b4765a6c4ea08631e89dc52d8bd3f5896"
 dependencies = [
+ "arc-swap",
  "aws-credential-types",
  "aws-runtime",
  "aws-smithy-async",
@@ -965,9 +967,9 @@ dependencies = [
 
 [[package]]
 name = "aws-sdk-ssooidc"
-version = "1.102.0"
+version = "1.103.0"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "b30d254992d56ef19f430396e5765b11e0f5bd21a7a557cb12fca1c8c18b9636"
+checksum = "7ae401c65ff288aa7873117fe535cd32b7b1bb0bc43751d28901a1d5f20636b9"
 dependencies = [
  "arc-swap",
  "aws-credential-types",
@@ -990,10 +992,11 @@ dependencies = [
 
 [[package]]
 name = "aws-sdk-sts"
-version = "1.105.0"
+version = "1.106.0"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "59f4f8065fe615dbed9096458ba98dda6d641553ffd5aedd27e37e65211aca9f"
+checksum = "4c80de7bb7d03e9ca8c9fd7b489f20f3948d3f3be91a7953591347d238115408"
 dependencies = [
+ "arc-swap",
  "aws-credential-types",
  "aws-runtime",
  "aws-smithy-async",
@@ -1015,9 +1018,9 @@ dependencies = [
 
 [[package]]
 name = "aws-sigv4"
-version = "1.4.4"
+version = "1.4.5"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "b7083fb918b38474ac65ffbf8a69fc8792d36879f4ac5f1667b43aec61efe9a5"
+checksum = "bae38512beae0ffee7010fc24e7a8a123c53efdfef42a61e80fda4882418dc71"
 dependencies = [
  "aws-credential-types",
  "aws-smithy-eventstream",
@@ -1107,9 +1110,9 @@ dependencies = [
 
 [[package]]
 name = "aws-smithy-http-client"
-version = "1.1.12"
+version = "1.1.13"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "6a2f165a7feee6f263028b899d0a181987f4fa7179a6411a32a439fba7c5f769"
+checksum = "5c3ef8931ad1c98aa6a55b4256f847f3116090819844e0dd41ea682cac5dd2d3"
 dependencies = [
  "aws-smithy-async",
  "aws-smithy-runtime-api",
@@ -1131,9 +1134,9 @@ dependencies = [
 
 [[package]]
 name = "aws-smithy-json"
-version = "0.62.6"
+version = "0.62.7"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "517089205f18ab4adc5a3e02888cb139bbbbb2e168eac9f396216925d1fbeaf5"
+checksum = "701a947f4797e52a911e114a898667c746c39feea467bbd1abd7b3721f702ffa"
 dependencies = [
  "aws-smithy-runtime-api",
  "aws-smithy-schema",
@@ -1187,9 +1190,9 @@ dependencies = [
 
 [[package]]
 name = "aws-smithy-runtime-api"
-version = "1.12.1"
+version = "1.12.3"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "dc117c179ecf39a62a0a3f49f600e9ac26a7ad7dd172177999f83933af776c32"
+checksum = "9db177daa6ba8afb9ee1aefcf548c907abcf52065e394ee11a92780057fe0e8c"
 dependencies = [
  "aws-smithy-async",
  "aws-smithy-runtime-api-macros",
@@ -1227,9 +1230,9 @@ dependencies = [
 
 [[package]]
 name = "aws-smithy-types"
-version = "1.4.8"
+version = "1.4.9"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "056b66dbce2f81cc0c1e2b05bb402eb58f8a3530479d650efadd5bbae9a4050b"
+checksum = "53f93074121a1be41317b9aa607143ae17900631f7f59a99f2b905d519d6783b"
 dependencies = [
  "base64-simd",
  "bytes",
@@ -1447,9 +1450,9 @@ checksum = "bef38d45163c2f1dde094a7dfd33ccf595c92905c8f8f4fdc18d06fb1037718a"
 
 [[package]]
 name = "bitflags"
-version = "2.11.1"
+version = "2.12.1"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "c4512299f36f043ab09a583e57bceb5a5aab7a73db1805848e8fef3c9e8c78b3"
+checksum = "84d7ced0ae9557296835c32bf1b1e02b44c746701f898460fb000d7eaa84f00a"
 dependencies = [
  "serde_core",
 ]
@@ -1730,9 +1733,9 @@ dependencies = [
 
 [[package]]
 name = "cc"
-version = "1.2.62"
+version = "1.2.63"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "a1dce859f0832a7d088c4f1119888ab94ef4b5d6795d1ce05afb7fe159d79f98"
+checksum = "556e016178bb5662a08681bbe0f00f8e17631781a4dfc8c45e466e4b185ec27f"
 dependencies = [
  "find-msvc-tools",
  "jobserver",
@@ -3475,9 +3478,9 @@ dependencies = [
 
 [[package]]
 name = "dial9-tokio-telemetry"
-version = "0.3.12"
+version = "0.3.13"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "9db20413b8c96577881e6f806970e81d41376236569eaf69f9329e34fc48bd79"
+checksum = "226a0d823327c391d9e8234dc3ccc5810d936359ba8ea6af024d57bb15568647"
 dependencies = [
  "arc-swap",
  "bon",
@@ -3571,7 +3574,7 @@ dependencies = [
  "libc",
  "option-ext",
  "redox_users",
- "windows-sys 0.61.2",
+ "windows-sys 0.59.0",
 ]
 
 [[package]]
@@ -3580,7 +3583,7 @@ version = "0.3.1"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "1e0e367e4e7da84520dedcac1901e4da967309406d1e51017ae1abfb97adbd38"
 dependencies = [
- "bitflags 2.11.1",
+ "bitflags 2.12.1",
  "objc2",
 ]
 
@@ -3615,7 +3618,7 @@ checksum = "d0881ea181b1df73ff77ffaaf9c7544ecc11e82fba9b5f27b262a3c73a332555"
 
 [[package]]
 name = "e2e_test"
-version = "1.0.0-beta.6"
+version = "1.0.0-beta.7"
 dependencies = [
  "anyhow",
  "astral-tokio-tar",
@@ -3907,7 +3910,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "39cab71617ae0d63f51a36d69f866391735b51691dbda63cf6f96d042b63efeb"
 dependencies = [
  "libc",
- "windows-sys 0.61.2",
+ "windows-sys 0.59.0",
 ]
 
 [[package]]
@@ -4027,7 +4030,7 @@ version = "25.12.19"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "35f6839d7b3b98adde531effaf34f0c2badc6f4735d26fe74709d8e513a96ef3"
 dependencies = [
- "bitflags 2.11.1",
+ "bitflags 2.12.1",
  "rustc_version",
 ]
 
@@ -4225,9 +4228,9 @@ dependencies = [
 
 [[package]]
 name = "generic-array"
-version = "1.4.1"
+version = "1.4.3"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "dab9e9188e97a93276e1fe7b56401b851e2b45a46d045ca658100c1303ada649"
+checksum = "c2e55f16dcf0e9c00efbe2e655ffe45fc98e7066b52bc92f8a79e64060a79351"
 dependencies = [
  "generic-array 0.14.7",
  "rustversion",
@@ -4946,9 +4949,9 @@ dependencies = [
 
 [[package]]
 name = "hyper"
-version = "1.10.0"
+version = "1.10.1"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "eb92f162bf56536459fc83c79b974bb12837acfed43d6bc370a7916d0ae15ecc"
+checksum = "55281c53a1894c864990125767da440a4e630446785086f52523b20033b74498"
 dependencies = [
  "atomic-waker",
  "bytes",
@@ -5277,7 +5280,7 @@ version = "0.7.12"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "4d09b98f7eace8982db770e4408e7470b028ce513ac28fecdc6bf4c30fe92b62"
 dependencies = [
- "bitflags 2.11.1",
+ "bitflags 2.12.1",
  "cfg-if",
  "libc",
 ]
@@ -5321,7 +5324,7 @@ checksum = "3640c1c38b8e4e43584d8df18be5fc6b0aa314ce6ebf51b53313d4306cca8e46"
 dependencies = [
  "hermit-abi",
  "libc",
- "windows-sys 0.61.2",
+ "windows-sys 0.59.0",
 ]
 
 [[package]]
@@ -5804,7 +5807,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "c9f8ff371890db2cf65a0758dba9a79f9cd965de369f6dbdc6581a22780af45e"
 dependencies = [
  "async-trait",
- "bitflags 2.11.1",
+ "bitflags 2.12.1",
  "bytes",
  "chrono",
  "dashmap",
@@ -5869,9 +5872,9 @@ dependencies = [
 
 [[package]]
 name = "log"
-version = "0.4.30"
+version = "0.4.31"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "616ec5685824bcc94416c6d4a7a446eea774a31efd7062c8480ba6fd06d7a6e5"
+checksum = "113b30b4cd05f7c06868fdb2854f66a7b9fece9a48425351cd532e810d74024f"
 
 [[package]]
 name = "lru"
@@ -5936,9 +5939,9 @@ dependencies = [
 
 [[package]]
 name = "lzma-rust2"
-version = "0.16.3"
+version = "0.16.4"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "5e9ceaec84b54518262de7cf06b8b43e83c808349960f1610b21b0bfc9640f20"
+checksum = "ce716bf1a316f47a280fc76295f6495b5bea4752bca01c3b3885e101b1c23c02"
 dependencies = [
  "sha2 0.11.0",
 ]
@@ -6364,7 +6367,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "4b42ced54aa8ac97226486337973f9bc3956e24f03a23e88a6e18f640959d6e2"
 dependencies = [
  "base64 0.22.1",
- "bitflags 2.11.1",
+ "bitflags 2.12.1",
  "btoi",
  "byteorder",
  "bytes",
@@ -6396,7 +6399,7 @@ version = "0.7.4"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "22f9786d56d972959e1408b6a93be6af13b9c1392036c5c1fafa08a1b0c6ee87"
 dependencies = [
- "bitflags 2.11.1",
+ "bitflags 2.12.1",
  "byteorder",
  "derive_builder",
  "getset",
@@ -6455,7 +6458,7 @@ version = "0.29.0"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "71e2746dc3a24dd78b3cfcb7be93368c6de9963d30f43a6a73998a9cf4b17b46"
 dependencies = [
- "bitflags 2.11.1",
+ "bitflags 2.12.1",
  "cfg-if",
  "cfg_aliases",
  "libc",
@@ -6468,7 +6471,7 @@ version = "0.30.1"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "74523f3a35e05aba87a1d978330aef40f67b0304ac79c1c00b294c9830543db6"
 dependencies = [
- "bitflags 2.11.1",
+ "bitflags 2.12.1",
  "cfg-if",
  "cfg_aliases",
  "libc",
@@ -6480,7 +6483,7 @@ version = "0.31.3"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "cf20d2fde8ff38632c426f1165ed7436270b44f199fc55284c38276f9db47c3d"
 dependencies = [
- "bitflags 2.11.1",
+ "bitflags 2.12.1",
  "cfg-if",
  "cfg_aliases",
  "libc",
@@ -6544,7 +6547,7 @@ version = "0.50.3"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "7957b9740744892f114936ab4a57b3f487491bbeafaf8083688b16841a4240e5"
 dependencies = [
- "windows-sys 0.61.2",
+ "windows-sys 0.59.0",
 ]
 
 [[package]]
@@ -6693,7 +6696,7 @@ version = "0.12.1"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "f049ae562349fefb8e837eb15443da1e7c6dcbd8a11f52a228f92220c2e5c85e"
 dependencies = [
- "bitflags 2.11.1",
+ "bitflags 2.12.1",
  "libloading",
  "nvml-wrapper-sys",
  "static_assertions",
@@ -6744,7 +6747,7 @@ version = "0.3.2"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "2a180dd8642fa45cdb7dd721cd4c11b1cadd4929ce112ebd8b9f5803cc79d536"
 dependencies = [
- "bitflags 2.11.1",
+ "bitflags 2.12.1",
  "dispatch2",
  "objc2",
 ]
@@ -6761,7 +6764,7 @@ version = "0.3.2"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "e3e0adef53c21f888deb4fa59fc59f7eb17404926ee8a6f59f5df0fd7f9f3272"
 dependencies = [
- "bitflags 2.11.1",
+ "bitflags 2.12.1",
  "objc2",
 ]
 
@@ -7351,7 +7354,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "97f6fccfd2d9d2df765ca23ff85fe5cc437fb0e6d3e164e4d3cbe09d14780c93"
 dependencies = [
  "arrayvec",
- "bitflags 2.11.1",
+ "bitflags 2.12.1",
  "thiserror 2.0.18",
  "zerocopy",
  "zerocopy-derive",
@@ -7898,7 +7901,7 @@ checksum = "4b45fcc2344c680f5025fe57779faef368840d0bd1f42f216291f0dc4ace4744"
 dependencies = [
  "bit-set",
  "bit-vec 0.8.0",
- "bitflags 2.11.1",
+ "bitflags 2.12.1",
  "num-traits",
  "rand 0.9.4",
  "rand_chacha 0.9.0",
@@ -7936,7 +7939,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "be769465445e8c1474e9c5dac2018218498557af32d9ed057325ec9a41ae81bf"
 dependencies = [
  "heck",
- "itertools 0.13.0",
+ "itertools 0.14.0",
  "log",
  "multimap",
  "once_cell",
@@ -7956,7 +7959,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "343d3bd7056eda839b03204e68deff7d1b13aba7af2b2fd16890697274262ee7"
 dependencies = [
  "heck",
- "itertools 0.13.0",
+ "itertools 0.14.0",
  "log",
  "multimap",
  "petgraph 0.8.3",
@@ -7977,7 +7980,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "8a56d757972c98b346a9b766e3f02746cde6dd1cd1d1d563472929fdd74bec4d"
 dependencies = [
  "anyhow",
- "itertools 0.13.0",
+ "itertools 0.14.0",
  "proc-macro2",
  "quote",
  "syn 2.0.117",
@@ -7990,7 +7993,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "27c6023962132f4b30eb4c172c91ce92d933da334c59c23cddee82358ddafb0b"
 dependencies = [
  "anyhow",
- "itertools 0.13.0",
+ "itertools 0.14.0",
  "proc-macro2",
  "quote",
  "syn 2.0.117",
@@ -8091,7 +8094,7 @@ version = "0.13.4"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "e9f068eba8e7071c5f9511831b44f32c740d5adf574e990f946ddb53db2f314e"
 dependencies = [
- "bitflags 2.11.1",
+ "bitflags 2.12.1",
  "memchr",
  "unicase",
 ]
@@ -8141,6 +8144,7 @@ version = "2.0.5"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "01ebd83db08e07a69b4291e7043f5f2ca964c8869e0bd0b6d09b8b062de775bd"
 dependencies = [
+ "jemalloc_pprof",
  "lazy_static",
  "libc",
  "libflate",
@@ -8423,7 +8427,7 @@ version = "11.6.0"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "498cd0dc59d73224351ee52a95fee0f1a617a2eae0e7d9d720cc622c73a54186"
 dependencies = [
- "bitflags 2.11.1",
+ "bitflags 2.12.1",
 ]
 
 [[package]]
@@ -8541,7 +8545,7 @@ version = "0.5.18"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "ed2bf2547551a7053d6fdfafda3f938979645c44812fbfcda098faae3f1a362d"
 dependencies = [
- "bitflags 2.11.1",
+ "bitflags 2.12.1",
 ]
 
 [[package]]
@@ -8868,7 +8872,7 @@ checksum = "f67013f080c226e5a34db1c71f2567f44d95a6300005bb6cd4e2c8fe3c326d1b"
 dependencies = [
  "aes 0.9.1",
  "aws-lc-rs",
- "bitflags 2.11.1",
+ "bitflags 2.12.1",
  "block-padding 0.4.2",
  "byteorder",
  "bytes",
@@ -8887,7 +8891,7 @@ dependencies = [
  "enum_dispatch",
  "flate2",
  "futures",
- "generic-array 1.4.1",
+ "generic-array 1.4.3",
  "getrandom 0.2.17",
  "ghash",
  "hex-literal",
@@ -8951,7 +8955,7 @@ version = "2.3.0"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "9ed8949eca4163c18a8f59ff96d32cf61e9c13b9735e21ef32b3907f4aafa1a9"
 dependencies = [
- "bitflags 2.11.1",
+ "bitflags 2.12.1",
  "bytes",
  "chrono",
  "dashmap",
@@ -9056,7 +9060,7 @@ dependencies = [
 
 [[package]]
 name = "rustfs"
-version = "1.0.0-beta.6"
+version = "1.0.0-beta.7"
 dependencies = [
  "aes-gcm",
  "anyhow",
@@ -9175,7 +9179,7 @@ dependencies = [
 
 [[package]]
 name = "rustfs-audit"
-version = "1.0.0-beta.6"
+version = "1.0.0-beta.7"
 dependencies = [
  "async-trait",
  "chrono",
@@ -9198,7 +9202,7 @@ dependencies = [
 
 [[package]]
 name = "rustfs-checksums"
-version = "1.0.0-beta.6"
+version = "1.0.0-beta.7"
 dependencies = [
  "base64-simd",
  "bytes",
@@ -9212,7 +9216,7 @@ dependencies = [
 
 [[package]]
 name = "rustfs-common"
-version = "1.0.0-beta.6"
+version = "1.0.0-beta.7"
 dependencies = [
  "chrono",
  "metrics",
@@ -9227,7 +9231,7 @@ dependencies = [
 
 [[package]]
 name = "rustfs-concurrency"
-version = "1.0.0-beta.6"
+version = "1.0.0-beta.7"
 dependencies = [
  "rustfs-io-core",
  "rustfs-io-metrics",
@@ -9239,14 +9243,14 @@ dependencies = [
 
 [[package]]
 name = "rustfs-config"
-version = "1.0.0-beta.6"
+version = "1.0.0-beta.7"
 dependencies = [
  "const-str",
 ]
 
 [[package]]
 name = "rustfs-credentials"
-version = "1.0.0-beta.6"
+version = "1.0.0-beta.7"
 dependencies = [
  "base64-simd",
  "hmac 0.13.0",
@@ -9259,7 +9263,7 @@ dependencies = [
 
 [[package]]
 name = "rustfs-crypto"
-version = "1.0.0-beta.6"
+version = "1.0.0-beta.7"
 dependencies = [
  "aes-gcm",
  "argon2",
@@ -9279,7 +9283,7 @@ dependencies = [
 
 [[package]]
 name = "rustfs-data-usage"
-version = "1.0.0-beta.6"
+version = "1.0.0-beta.7"
 dependencies = [
  "async-trait",
  "path-clean",
@@ -9290,7 +9294,7 @@ dependencies = [
 
 [[package]]
 name = "rustfs-ecstore"
-version = "1.0.0-beta.6"
+version = "1.0.0-beta.7"
 dependencies = [
  "aes-gcm",
  "async-channel",
@@ -9394,7 +9398,7 @@ dependencies = [
 
 [[package]]
 name = "rustfs-filemeta"
-version = "1.0.0-beta.6"
+version = "1.0.0-beta.7"
 dependencies = [
  "arc-swap",
  "byteorder",
@@ -9418,7 +9422,7 @@ dependencies = [
 
 [[package]]
 name = "rustfs-heal"
-version = "1.0.0-beta.6"
+version = "1.0.0-beta.7"
 dependencies = [
  "anyhow",
  "async-trait",
@@ -9446,7 +9450,7 @@ dependencies = [
 
 [[package]]
 name = "rustfs-iam"
-version = "1.0.0-beta.6"
+version = "1.0.0-beta.7"
 dependencies = [
  "arc-swap",
  "async-trait",
@@ -9480,7 +9484,7 @@ dependencies = [
 
 [[package]]
 name = "rustfs-io-core"
-version = "1.0.0-beta.6"
+version = "1.0.0-beta.7"
 dependencies = [
  "bytes",
  "memmap2 0.9.10",
@@ -9491,7 +9495,7 @@ dependencies = [
 
 [[package]]
 name = "rustfs-io-metrics"
-version = "1.0.0-beta.6"
+version = "1.0.0-beta.7"
 dependencies = [
  "criterion",
  "metrics",
@@ -9553,7 +9557,7 @@ dependencies = [
 
 [[package]]
 name = "rustfs-keystone"
-version = "1.0.0-beta.6"
+version = "1.0.0-beta.7"
 dependencies = [
  "bytes",
  "futures",
@@ -9578,7 +9582,7 @@ dependencies = [
 
 [[package]]
 name = "rustfs-kms"
-version = "1.0.0-beta.6"
+version = "1.0.0-beta.7"
 dependencies = [
  "aes-gcm",
  "arc-swap",
@@ -9607,7 +9611,7 @@ dependencies = [
 
 [[package]]
 name = "rustfs-lock"
-version = "1.0.0-beta.6"
+version = "1.0.0-beta.7"
 dependencies = [
  "async-trait",
  "crossbeam-queue",
@@ -9628,7 +9632,7 @@ dependencies = [
 
 [[package]]
 name = "rustfs-madmin"
-version = "1.0.0-beta.6"
+version = "1.0.0-beta.7"
 dependencies = [
  "chrono",
  "humantime",
@@ -9641,7 +9645,7 @@ dependencies = [
 
 [[package]]
 name = "rustfs-notify"
-version = "1.0.0-beta.6"
+version = "1.0.0-beta.7"
 dependencies = [
  "arc-swap",
  "async-trait",
@@ -9675,7 +9679,7 @@ dependencies = [
 
 [[package]]
 name = "rustfs-object-capacity"
-version = "1.0.0-beta.6"
+version = "1.0.0-beta.7"
 dependencies = [
  "criterion",
  "futures",
@@ -9693,7 +9697,7 @@ dependencies = [
 
 [[package]]
 name = "rustfs-obs"
-version = "1.0.0-beta.6"
+version = "1.0.0-beta.7"
 dependencies = [
  "chrono",
  "crossbeam-channel",
@@ -9702,6 +9706,7 @@ dependencies = [
  "dial9-tokio-telemetry",
  "flate2",
  "glob",
+ "jemalloc_pprof",
  "jiff",
  "metrics",
  "num_cpus",
@@ -9739,7 +9744,7 @@ dependencies = [
 
 [[package]]
 name = "rustfs-policy"
-version = "1.0.0-beta.6"
+version = "1.0.0-beta.7"
 dependencies = [
  "async-trait",
  "base64-simd",
@@ -9767,7 +9772,7 @@ dependencies = [
 
 [[package]]
 name = "rustfs-protocols"
-version = "1.0.0-beta.6"
+version = "1.0.0-beta.7"
 dependencies = [
  "astral-tokio-tar",
  "async-compression",
@@ -9825,7 +9830,7 @@ dependencies = [
 
 [[package]]
 name = "rustfs-protos"
-version = "1.0.0-beta.6"
+version = "1.0.0-beta.7"
 dependencies = [
  "flatbuffers",
  "prost 0.14.3",
@@ -9843,7 +9848,7 @@ dependencies = [
 
 [[package]]
 name = "rustfs-rio"
-version = "1.0.0-beta.6"
+version = "1.0.0-beta.7"
 dependencies = [
  "aes-gcm",
  "axum",
@@ -9878,14 +9883,14 @@ dependencies = [
 
 [[package]]
 name = "rustfs-s3-ops"
-version = "1.0.0-beta.6"
+version = "1.0.0-beta.7"
 dependencies = [
  "rustfs-s3-types",
 ]
 
 [[package]]
 name = "rustfs-s3-types"
-version = "1.0.0-beta.6"
+version = "1.0.0-beta.7"
 dependencies = [
  "serde",
  "serde_json",
@@ -9893,7 +9898,7 @@ dependencies = [
 
 [[package]]
 name = "rustfs-s3select-api"
-version = "1.0.0-beta.6"
+version = "1.0.0-beta.7"
 dependencies = [
  "async-trait",
  "bytes",
@@ -9910,7 +9915,7 @@ dependencies = [
  "rustfs-ecstore",
  "s3s",
  "serde_json",
- "snafu 0.9.0",
+ "snafu 0.9.1",
  "tokio",
  "tokio-util",
  "tracing",
@@ -9920,7 +9925,7 @@ dependencies = [
 
 [[package]]
 name = "rustfs-s3select-query"
-version = "1.0.0-beta.6"
+version = "1.0.0-beta.7"
 dependencies = [
  "async-recursion",
  "async-trait",
@@ -9930,14 +9935,14 @@ dependencies = [
  "parking_lot 0.12.5",
  "rustfs-s3select-api",
  "s3s",
- "snafu 0.9.0",
+ "snafu 0.9.1",
  "tokio",
  "tracing",
 ]
 
 [[package]]
 name = "rustfs-scanner"
-version = "1.0.0-beta.6"
+version = "1.0.0-beta.7"
 dependencies = [
  "async-trait",
  "chrono",
@@ -9968,7 +9973,7 @@ dependencies = [
 
 [[package]]
 name = "rustfs-signer"
-version = "1.0.0-beta.6"
+version = "1.0.0-beta.7"
 dependencies = [
  "base64-simd",
  "bytes",
@@ -9984,7 +9989,7 @@ dependencies = [
 
 [[package]]
 name = "rustfs-targets"
-version = "1.0.0-beta.6"
+version = "1.0.0-beta.7"
 dependencies = [
  "arc-swap",
  "async-nats",
@@ -10030,7 +10035,7 @@ dependencies = [
 
 [[package]]
 name = "rustfs-tls-runtime"
-version = "1.0.0-beta.6"
+version = "1.0.0-beta.7"
 dependencies = [
  "arc-swap",
  "metrics",
@@ -10050,7 +10055,7 @@ dependencies = [
 
 [[package]]
 name = "rustfs-trusted-proxies"
-version = "1.0.0-beta.6"
+version = "1.0.0-beta.7"
 dependencies = [
  "async-trait",
  "axum",
@@ -10074,7 +10079,7 @@ dependencies = [
 
 [[package]]
 name = "rustfs-utils"
-version = "1.0.0-beta.6"
+version = "1.0.0-beta.7"
 dependencies = [
  "base64-simd",
  "blake2 0.11.0-rc.6",
@@ -10112,7 +10117,7 @@ dependencies = [
 
 [[package]]
 name = "rustfs-zip"
-version = "1.0.0-beta.6"
+version = "1.0.0-beta.7"
 dependencies = [
  "astral-tokio-tar",
  "async-compression",
@@ -10169,7 +10174,7 @@ version = "0.38.44"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "fdb5bc1ae2baa591800df16c9ca78619bf65c0488b41b96ccec5d11220d8c154"
 dependencies = [
- "bitflags 2.11.1",
+ "bitflags 2.12.1",
  "errno",
  "libc",
  "linux-raw-sys 0.4.15",
@@ -10182,11 +10187,11 @@ version = "1.1.4"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "b6fe4565b9518b83ef4f91bb47ce29620ca828bd32cb7e408f0062e9930ba190"
 dependencies = [
- "bitflags 2.11.1",
+ "bitflags 2.12.1",
  "errno",
  "libc",
  "linux-raw-sys 0.12.1",
- "windows-sys 0.61.2",
+ "windows-sys 0.59.0",
 ]
 
 [[package]]
@@ -10222,9 +10227,9 @@ dependencies = [
 
 [[package]]
 name = "rustls-native-certs"
-version = "0.8.3"
+version = "0.8.4"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "612460d5f7bea540c490b2b6395d8e34a953e52b491accd6c86c8164c5932a63"
+checksum = "dab5152771c58876a2146916e53e35057e1a4dfa2b9df0f0305b07f611fdea4d"
 dependencies = [
  "openssl-probe",
  "rustls-pki-types",
@@ -10260,7 +10265,7 @@ dependencies = [
  "security-framework",
  "security-framework-sys",
  "webpki-root-certs",
- "windows-sys 0.61.2",
+ "windows-sys 0.59.0",
 ]
 
 [[package]]
@@ -10390,15 +10395,6 @@ version = "0.1.0"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "ece8e78b2f38ec51c51f5d475df0a7187ba5111b2a28bdc761ee05b075d40a71"
 
-[[package]]
-name = "scc"
-version = "2.4.0"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "46e6f046b7fef48e2660c57ed794263155d713de679057f2d0c169bfc6e756cc"
-dependencies = [
- "sdd",
-]
-
 [[package]]
 name = "schannel"
 version = "0.1.29"
@@ -10461,12 +10457,6 @@ dependencies = [
  "sha2 0.11.0",
 ]
 
-[[package]]
-name = "sdd"
-version = "3.0.10"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "490dcfcbfef26be6800d11870ff2df8774fa6e86d047e3e8c8a76b25655e41ca"
-
 [[package]]
 name = "sec1"
 version = "0.7.3"
@@ -10501,7 +10491,7 @@ version = "3.7.0"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "b7f4bc775c73d9a02cde8bf7b2ec4c9d12743edf609006c7facc23998404cd1d"
 dependencies = [
- "bitflags 2.11.1",
+ "bitflags 2.12.1",
  "core-foundation 0.10.1",
  "core-foundation-sys",
  "libc",
@@ -10693,24 +10683,23 @@ dependencies = [
 
 [[package]]
 name = "serial_test"
-version = "3.4.0"
+version = "3.5.0"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "911bd979bf1070a3f3aa7b691a3b3e9968f339ceeec89e08c280a8a22207a32f"
+checksum = "699f4197115b8a7e7ff19c9a315a4bd6fffec26cc4626ef45ecaea389e081c6d"
 dependencies = [
  "futures-executor",
  "futures-util",
  "log",
  "once_cell",
  "parking_lot 0.12.5",
- "scc",
  "serial_test_derive",
 ]
 
 [[package]]
 name = "serial_test_derive"
-version = "3.4.0"
+version = "3.5.0"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "0a7d91949b85b0d2fb687445e448b40d322b6b3e4af6b44a29b21d9a5f33e6d9"
+checksum = "94e153fc76e1c6a068703d6d29c508a0b15c061c4b7e43da59cc097bc342673c"
 dependencies = [
  "proc-macro2",
  "quote",
@@ -10810,9 +10799,9 @@ dependencies = [
 
 [[package]]
 name = "shlex"
-version = "1.3.0"
+version = "2.0.1"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "0fda2ff0d084019ba4d7c6f371c95d8fd75ce3524c3cb8fb653a3023f6323e64"
+checksum = "f8fadd59c855ef2080decdef8ff161eb6661b86933c9d82e5ba29dc602a55aba"
 
 [[package]]
 name = "signal-hook-registry"
@@ -10974,12 +10963,11 @@ dependencies = [
 
 [[package]]
 name = "snafu"
-version = "0.9.0"
+version = "0.9.1"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "d1d4bced6a69f90b2056c03dcff2c4737f98d6fb9e0853493996e1d253ca29c6"
+checksum = "d1a012328be2e3f5d5f6f3218147ca02588cea4cb865e876849ab6debcf36522"
 dependencies = [
- "backtrace",
- "snafu-derive 0.9.0",
+ "snafu-derive 0.9.1",
 ]
 
 [[package]]
@@ -10995,9 +10983,9 @@ dependencies = [
 
 [[package]]
 name = "snafu-derive"
-version = "0.9.0"
+version = "0.9.1"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "54254b8531cafa275c5e096f62d48c81435d1015405a91198ddb11e967301d40"
+checksum = "5f103c50866b8743da9429b8a581d81a27c2d3a9c4ac7df8f8571c1dd7896eda"
 dependencies = [
  "heck",
  "proc-macro2",
@@ -11377,7 +11365,7 @@ version = "0.7.0"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "a13f3d0daba03132c0aa9767f98351b3488edc2c100cda2d2ec2b04f3d8d3c8b"
 dependencies = [
- "bitflags 2.11.1",
+ "bitflags 2.12.1",
  "core-foundation 0.9.4",
  "system-configuration-sys",
 ]
@@ -11431,7 +11419,7 @@ dependencies = [
  "getrandom 0.4.2",
  "once_cell",
  "rustix 1.1.4",
- "windows-sys 0.61.2",
+ "windows-sys 0.59.0",
 ]
 
 [[package]]
@@ -11932,7 +11920,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "4cfcf7e2740e6fc6d4d688b4ef00650406bb94adf4731e43c096c3a19fe40840"
 dependencies = [
  "async-compression",
- "bitflags 2.11.1",
+ "bitflags 2.12.1",
  "bytes",
  "futures-core",
  "futures-util",
@@ -12148,9 +12136,9 @@ checksum = "8e28f89b80c87b8fb0cf04ab448d5dd0dd0ade2f8891bae878de66a75a28600e"
 
 [[package]]
 name = "typenum"
-version = "1.20.0"
+version = "1.20.1"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "40ce102ab67701b8526c123c1bab5cbe42d7040ccfd0f64af1a385808d2f43de"
+checksum = "b6f5e870be6c3b371b77fe0ee0bafb859fa4964b4404c27de1d380043c4dda20"
 
 [[package]]
 name = "unarray"
@@ -12209,9 +12197,9 @@ checksum = "7df058c713841ad818f1dc5d3fd88063241cc61f49f5fbea4b951e8cf5a8d71d"
 
 [[package]]
 name = "unicode-segmentation"
-version = "1.13.2"
+version = "1.13.3"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "9629274872b2bfaf8d66f5f15725007f635594914870f65218920345aa11aa8c"
+checksum = "c6f5d3c3b1bf09027a88a6bc961fc00497d651009560b5463668dc81b0fa87a8"
 
 [[package]]
 name = "unicode-width"
@@ -12280,9 +12268,9 @@ checksum = "06abde3611657adf66d383f00b093d7faecc7fa57071cce2578660c9f1010821"
 
 [[package]]
 name = "uuid"
-version = "1.23.1"
+version = "1.23.2"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "ddd74a9687298c6858e9b88ec8935ec45d22e8fd5e6394fa1bd4e99a87789c76"
+checksum = "d258b83ceec21034727ecee8c382cfa6c3e133699b0742c64571814fb420c9f7"
 dependencies = [
  "getrandom 0.4.2",
  "js-sys",
@@ -12494,7 +12482,7 @@ version = "0.244.0"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "47b807c72e1bac69382b3a6fb3dbe8ea4c0ed87ff5629b8685ae6b9a611028fe"
 dependencies = [
- "bitflags 2.11.1",
+ "bitflags 2.12.1",
  "hashbrown 0.15.5",
  "indexmap 2.14.0",
  "semver",
@@ -12609,7 +12597,7 @@ version = "0.1.11"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "c2a7b1c03c876122aa43f3020e6c3c3ee5c05081c9a00739faf7503aeba10d22"
 dependencies = [
- "windows-sys 0.61.2",
+ "windows-sys 0.59.0",
 ]
 
 [[package]]
@@ -12903,7 +12891,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "9d66ea20e9553b30172b5e831994e35fbde2d165325bec84fc43dbf6f4eb9cb2"
 dependencies = [
  "anyhow",
- "bitflags 2.11.1",
+ "bitflags 2.12.1",
  "indexmap 2.14.0",
  "log",
  "serde",
@@ -13068,18 +13056,18 @@ dependencies = [
 
 [[package]]
 name = "zerocopy"
-version = "0.8.49"
+version = "0.8.50"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "bce33a6288fa3f072a8c2c7d0f2fdbb90e28298f0135c1f99b96c3db2efcc60b"
+checksum = "3b065d4f0e55f82fae73202e189638116a87c55ab6b8e6c2721e13dd9d854ad1"
 dependencies = [
  "zerocopy-derive",
 ]
 
 [[package]]
 name = "zerocopy-derive"
-version = "0.8.49"
+version = "0.8.50"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "8fd425244944f4ab65ccff928e7323354c5a018c75838362fdce749dfad2ee1e"
+checksum = "0b631b19d36a892ab55420c92dbc83ccd79274f25be714855d3074aa71cab639"
 dependencies = [
  "proc-macro2",
  "quote",

Cargo.toml

@@ -60,7 +60,7 @@ edition = "2024"
 license = "Apache-2.0"
 repository = "https://github.com/rustfs/rustfs"
 rust-version = "1.95.0"
-version = "1.0.0-beta.6"
+version = "1.0.0-beta.7"
 homepage = "https://rustfs.com"
 description = "RustFS is a high-performance distributed object storage software built using Rust, one of the most popular languages worldwide. "
 keywords = ["RustFS", "Minio", "object-storage", "filesystem", "s3"]
@@ -77,43 +77,43 @@ redundant_clone = "warn"
 
 [workspace.dependencies]
 # RustFS Internal Crates
-rustfs = { path = "./rustfs", version = "1.0.0-beta.6" }
-rustfs-heal = { path = "crates/heal", version = "1.0.0-beta.6" }
-rustfs-audit = { path = "crates/audit", version = "1.0.0-beta.6" }
-rustfs-checksums = { path = "crates/checksums", version = "1.0.0-beta.6" }
-rustfs-common = { path = "crates/common", version = "1.0.0-beta.6" }
-rustfs-data-usage = { path = "crates/data-usage", version = "1.0.0-beta.6" }
-rustfs-config = { path = "./crates/config", version = "1.0.0-beta.6" }
-rustfs-concurrency = { path = "./crates/concurrency", version = "1.0.0-beta.6" }
-rustfs-credentials = { path = "crates/credentials", version = "1.0.0-beta.6" }
-rustfs-crypto = { path = "crates/crypto", version = "1.0.0-beta.6" }
-rustfs-ecstore = { path = "crates/ecstore", version = "1.0.0-beta.6" }
-rustfs-filemeta = { path = "crates/filemeta", version = "1.0.0-beta.6" }
-rustfs-iam = { path = "crates/iam", version = "1.0.0-beta.6" }
-rustfs-keystone = { path = "crates/keystone", version = "1.0.0-beta.6" }
-rustfs-kms = { path = "crates/kms", version = "1.0.0-beta.6" }
-rustfs-lock = { path = "crates/lock", version = "1.0.0-beta.6" }
-rustfs-madmin = { path = "crates/madmin", version = "1.0.0-beta.6" }
-rustfs-notify = { path = "crates/notify", version = "1.0.0-beta.6" }
-rustfs-io-metrics = { path = "crates/io-metrics", version = "1.0.0-beta.6" }
-rustfs-io-core = { path = "crates/io-core", version = "1.0.0-beta.6" }
-rustfs-object-capacity = { path = "crates/object-capacity", version = "1.0.0-beta.6" }
-rustfs-obs = { path = "crates/obs", version = "1.0.0-beta.6" }
-rustfs-policy = { path = "crates/policy", version = "1.0.0-beta.6" }
-rustfs-protos = { path = "crates/protos", version = "1.0.0-beta.6" }
-rustfs-protocols = { path = "crates/protocols", version = "1.0.0-beta.6" }
-rustfs-rio = { path = "crates/rio", version = "1.0.0-beta.6" }
-rustfs-s3-types = { path = "crates/s3-types", version = "1.0.0-beta.6" }
-rustfs-s3-ops = { path = "crates/s3-ops", version = "1.0.0-beta.6" }
-rustfs-s3select-api = { path = "crates/s3select-api", version = "1.0.0-beta.6" }
-rustfs-s3select-query = { path = "crates/s3select-query", version = "1.0.0-beta.6" }
-rustfs-scanner = { path = "crates/scanner", version = "1.0.0-beta.6" }
-rustfs-signer = { path = "crates/signer", version = "1.0.0-beta.6" }
-rustfs-trusted-proxies = { path = "crates/trusted-proxies", version = "1.0.0-beta.6" }
-rustfs-targets = { path = "crates/targets", version = "1.0.0-beta.6" }
-rustfs-tls-runtime = { path = "crates/tls-runtime", version = "1.0.0-beta.6" }
-rustfs-utils = { path = "crates/utils", version = "1.0.0-beta.6" }
-rustfs-zip = { path = "./crates/zip", version = "1.0.0-beta.6" }
+rustfs = { path = "./rustfs", version = "1.0.0-beta.7" }
+rustfs-heal = { path = "crates/heal", version = "1.0.0-beta.7" }
+rustfs-audit = { path = "crates/audit", version = "1.0.0-beta.7" }
+rustfs-checksums = { path = "crates/checksums", version = "1.0.0-beta.7" }
+rustfs-common = { path = "crates/common", version = "1.0.0-beta.7" }
+rustfs-data-usage = { path = "crates/data-usage", version = "1.0.0-beta.7" }
+rustfs-config = { path = "./crates/config", version = "1.0.0-beta.7" }
+rustfs-concurrency = { path = "./crates/concurrency", version = "1.0.0-beta.7" }
+rustfs-credentials = { path = "crates/credentials", version = "1.0.0-beta.7" }
+rustfs-crypto = { path = "crates/crypto", version = "1.0.0-beta.7" }
+rustfs-ecstore = { path = "crates/ecstore", version = "1.0.0-beta.7" }
+rustfs-filemeta = { path = "crates/filemeta", version = "1.0.0-beta.7" }
+rustfs-iam = { path = "crates/iam", version = "1.0.0-beta.7" }
+rustfs-keystone = { path = "crates/keystone", version = "1.0.0-beta.7" }
+rustfs-kms = { path = "crates/kms", version = "1.0.0-beta.7" }
+rustfs-lock = { path = "crates/lock", version = "1.0.0-beta.7" }
+rustfs-madmin = { path = "crates/madmin", version = "1.0.0-beta.7" }
+rustfs-notify = { path = "crates/notify", version = "1.0.0-beta.7" }
+rustfs-io-metrics = { path = "crates/io-metrics", version = "1.0.0-beta.7" }
+rustfs-io-core = { path = "crates/io-core", version = "1.0.0-beta.7" }
+rustfs-object-capacity = { path = "crates/object-capacity", version = "1.0.0-beta.7" }
+rustfs-obs = { path = "crates/obs", version = "1.0.0-beta.7" }
+rustfs-policy = { path = "crates/policy", version = "1.0.0-beta.7" }
+rustfs-protos = { path = "crates/protos", version = "1.0.0-beta.7" }
+rustfs-protocols = { path = "crates/protocols", version = "1.0.0-beta.7" }
+rustfs-rio = { path = "crates/rio", version = "1.0.0-beta.7" }
+rustfs-s3-types = { path = "crates/s3-types", version = "1.0.0-beta.7" }
+rustfs-s3-ops = { path = "crates/s3-ops", version = "1.0.0-beta.7" }
+rustfs-s3select-api = { path = "crates/s3select-api", version = "1.0.0-beta.7" }
+rustfs-s3select-query = { path = "crates/s3select-query", version = "1.0.0-beta.7" }
+rustfs-scanner = { path = "crates/scanner", version = "1.0.0-beta.7" }
+rustfs-signer = { path = "crates/signer", version = "1.0.0-beta.7" }
+rustfs-trusted-proxies = { path = "crates/trusted-proxies", version = "1.0.0-beta.7" }
+rustfs-targets = { path = "crates/targets", version = "1.0.0-beta.7" }
+rustfs-tls-runtime = { path = "crates/tls-runtime", version = "1.0.0-beta.7" }
+rustfs-utils = { path = "crates/utils", version = "1.0.0-beta.7" }
+rustfs-zip = { path = "./crates/zip", version = "1.0.0-beta.7" }
 
 # Async Runtime and Networking
 async-channel = "2.5.0"
@@ -129,7 +129,7 @@ futures-util = "0.3.32"
 pollster = "0.4.0"
 pulsar = { version = "6.8.0", default-features = false, features = ["tokio-rustls-runtime"] }
 lapin = { version = "4.10.0", default-features = false, features = ["tokio", "rustls", "rustls--aws_lc_rs"] }
-hyper = { version = "1.10.0", features = ["http2", "http1", "server"] }
+hyper = { version = "1.10.1", features = ["http2", "http1", "server"] }
 hyper-rustls = { version = "0.27.9", default-features = false, features = ["native-tokio", "http1", "tls12", "logging", "http2", "aws-lc-rs", "webpki-roots"] }
 hyper-util = { version = "0.1.20", features = ["tokio", "server-auto", "server-graceful", "tracing"] }
 http = "1.4.1"
@@ -199,11 +199,11 @@ arc-swap = "1.9.1"
 astral-tokio-tar = "0.6.2"
 atoi = "2.0.0"
 atomic_enum = "0.3.0"
-aws-config = { version = "1.8.17" }
+aws-config = { version = "1.8.18" }
 aws-credential-types = { version = "1.2.14" }
-aws-sdk-s3 = { version = "1.134.0", default-features = false, features = ["sigv4a", "default-https-client", "rt-tokio"] }
-aws-smithy-http-client = { version = "1.1.12", default-features = false, features = ["default-client", "rustls-aws-lc"] }
-aws-smithy-types = { version = "1.4.8" }
+aws-sdk-s3 = { version = "1.135.0", default-features = false, features = ["sigv4a", "default-https-client", "rt-tokio"] }
+aws-smithy-http-client = { version = "1.1.13", default-features = false, features = ["default-client", "rustls-aws-lc"] }
+aws-smithy-types = { version = "1.4.9" }
 base64 = "0.22.1"
 base64-simd = "0.8.0"
 brotli = "8.0.3"
@@ -261,12 +261,12 @@ rustix = { version = "1.1.4", features = ["fs"] }
 rust-embed = { version = "8.11.0" }
 rustc-hash = { version = "2.1.2" }
 s3s = { git = "https://github.com/rustfs/s3s", rev = "507e1312b211c3ddc214b03875d6fabd15d22ed5", features = ["minio"] }
-serial_test = "3.4.0"
+serial_test = "3.5.0"
 shadow-rs = { version = "2.0.0", default-features = false }
 siphasher = "1.0.3"
 smallvec = { version = "1.15.1", features = ["serde"] }
 smartstring = "1.0.1"
-snafu = "0.9.0"
+snafu = "0.9.1"
 snap = "1.1.1"
 starshard = { version = "2.2.0", features = ["rayon", "async", "serde"] }
 strum = { version = "0.28.0", features = ["derive"] }
@@ -283,7 +283,7 @@ tracing-subscriber = { version = "0.3.23", features = ["env-filter", "time"] }
 transform-stream = "0.3.1"
 url = "2.5.8"
 urlencoding = "2.1.3"
-uuid = { version = "1.23.1", features = ["v4", "fast-rng", "macro-diagnostics"] }
+uuid = { version = "1.23.2", features = ["v4", "fast-rng", "macro-diagnostics"] }
 vaultrs = { version = "0.8.0" }
 walkdir = "2.5.0"
 wildmatch = { version = "2.6.1", features = ["serde"] }
@@ -301,7 +301,7 @@ opentelemetry-otlp = { version = "0.32.0", features = ["gzip-http", "reqwest-rus
 opentelemetry_sdk = { version = "0.32.1", features = ["rt-tokio"] }
 opentelemetry-semantic-conventions = { version = "0.32.0", features = ["semconv_experimental"] }
 opentelemetry-stdout = { version = "0.32.0" }
-pyroscope = { version = "2.0.5", features = ["backend-pprof-rs"] }
+pyroscope = { version = "=2.0.5", features = ["backend-pprof-rs", "backend-jemalloc"] }
 
 # FTP and SFTP
 libunftp = { version = "0.23.0", features = ["experimental"] }
@@ -324,7 +324,9 @@ tikv-jemalloc-ctl = { version = "0.6", features = ["use_std", "stats", "profilin
 jemalloc_pprof = { version = "0.8.2", features = ["symbolize", "flamegraph"] }
 # Used to generate CPU performance analysis data and flame diagrams
 # pprof = { version = "0.15.0", features = ["flamegraph", "protobuf-codec"] }
-# Pyroscope uses a patched pprof, until they merge back upstream, replace all references. Otherwise, two pprof libs with symbol collision.
+# Use pprof-pyroscope-fork to match pyroscope 2.0.5's internal pprof dependency (v0.1500.4).
+# Cargo unifies them into a single instance, avoiding duplicate perf_signal_handler symbols.
+# NOTE: Do NOT upgrade pyroscope to >=2.0.6 — it vendors pprof-rs internally, causing symbol collision.
 pprof = { package = "pprof-pyroscope-fork", version = "0.1500.4", features = ["flamegraph", "protobuf-codec"] }
 
 [workspace.metadata.cargo-shear]

README.md

@@ -116,7 +116,7 @@ chown -R 10001:10001 data logs
 docker run -d -p 9000:9000 -p 9001:9001 -v $(pwd)/data:/data -v $(pwd)/logs:/logs rustfs/rustfs:latest
 
 # Using specific version
-docker run -d -p 9000:9000 -p 9001:9001 -v $(pwd)/data:/data -v $(pwd)/logs:/logs rustfs/rustfs:1.0.0-beta.6
+docker run -d -p 9000:9000 -p 9001:9001 -v $(pwd)/data:/data -v $(pwd)/logs:/logs rustfs/rustfs:1.0.0-beta.7
 ```
 
 If you use [podman](https://github.com/containers/podman) instead of docker, you can install the RustFS with the below command

README_ZH.md

@@ -113,7 +113,7 @@ RustFS 容器以非 root 用户 `rustfs` (UID `10001`) 运行。如果您使用
  docker run -d -p 9000:9000 -p 9001:9001 -v $(pwd)/data:/data -v $(pwd)/logs:/logs rustfs/rustfs:latest
 
  # 使用指定版本运行
- docker run -d -p 9000:9000 -p 9001:9001 -v $(pwd)/data:/data -v $(pwd)/logs:/logs rustfs/rustfs:1.0.0-beta.6
+ docker run -d -p 9000:9000 -p 9001:9001 -v $(pwd)/data:/data -v $(pwd)/logs:/logs rustfs/rustfs:1.0.0-beta.7
 ```
 
 您也可以使用 Docker Compose。使用根目录下的 `docker-compose.yml` 文件：

crates/AGENTS.md

@@ -14,6 +14,41 @@ Applies to all paths under `crates/`.
 - Keep integration tests under each crate's `tests/` directory.
 - Add regression tests for bug fixes and behavior changes.
 
+## Error Type Design
+
+- Public API functions must return a typed error enum (preferably `thiserror`-derived), never `Result<_, String>`.
+- Do not use `Box<dyn Error>` or `Box<dyn Error + Send + Sync>` in public trait methods or struct methods. Define a concrete error type with specific variants.
+- When implementing `std::error::Error`, always override `fn source()` if you store an inner error. Breaking the error chain makes debugging impossible.
+- Internal helpers that return `Result<_, String>` and are immediately wrapped via `.map_err(Error::other)` should return the actual error type directly.
+
+## Concurrency
+
+- Document lock acquisition order when a module uses multiple locks. Never acquire the same set of locks in different orders across code paths.
+- Never hold a `tokio::sync::RwLock`/`Mutex` write guard across `.await` points unless the critical section is unavoidably async and the hold time is bounded.
+- Prefer `compare_exchange` loops over load-then-store for concurrent counters (peak values, adaptive heuristics).
+- When resetting multi-field atomic statistics, use a version/sequence counter or accept that concurrent readers may see partial snapshots; document the tradeoff.
+- `std::sync::Mutex` is acceptable in async context only when held for a brief, non-`await`-containing critical section. If in doubt, use `tokio::sync::Mutex`.
+
+## Recursion Safety
+
+- Recursive tree/graph traversals must have a depth limit (e.g., `max_depth` counter) or use an iterative approach with an explicit `Vec` stack.
+- This applies to cache trees, directory walks, and any user-influenced hierarchy.
+- A corrupted or malicious input must not be able to overflow the thread stack.
+
+## Type Casting
+
+- Never use `as` for numeric conversions that may truncate or overflow. Use `try_into()` with explicit error handling, or clamp with `value.max(0) as usize` when the domain is bounded.
+- `f64 as usize` saturates but is fragile; clamp to `[0, usize::MAX as f64]` first.
+- Treat every `as` cast in a PR review as a potential bug; require justification.
+
+## Testing
+
+- Keep unit tests close to the module they test.
+- Keep integration tests under each crate's `tests/` directory.
+- Add regression tests for bug fixes and behavior changes.
+- Every test function must contain at least one `assert!`/`assert_eq!`/`assert_matches!`. A test that only calls code without asserting is not a test.
+- In tests, prefer `.expect("context: what was being tested")` over bare `.unwrap()`. A test failure should tell you which operation failed and with what input.
+
 ## Async and Performance
 
 - Keep async paths non-blocking.

crates/common/src/heal_channel.rs

@@ -110,6 +110,25 @@ pub enum HealScanMode {
     Deep = 2,
 }
 
+impl HealScanMode {
+    pub const fn as_str(self) -> &'static str {
+        match self {
+            Self::Unknown => "unknown",
+            Self::Normal => "normal",
+            Self::Deep => "deep",
+        }
+    }
+
+    pub const fn from_u8(value: u8) -> Option<Self> {
+        match value {
+            0 => Some(Self::Unknown),
+            1 => Some(Self::Normal),
+            2 => Some(Self::Deep),
+            _ => None,
+        }
+    }
+}
+
 impl Serialize for HealScanMode {
     fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>
     where
@@ -137,12 +156,7 @@ impl<'de> Deserialize<'de> for HealScanMode {
             where
                 E: serde::de::Error,
             {
-                match value {
-                    0 => Ok(HealScanMode::Unknown),
-                    1 => Ok(HealScanMode::Normal),
-                    2 => Ok(HealScanMode::Deep),
-                    _ => Err(E::custom(format!("invalid HealScanMode value: {value}"))),
-                }
+                HealScanMode::from_u8(value).ok_or_else(|| E::custom(format!("invalid HealScanMode value: {value}")))
             }
 
             fn visit_u64<E>(self, value: u64) -> Result<Self::Value, E>

crates/config/README.md

@@ -70,6 +70,9 @@ Current guidance:
 - `RUSTFS_DATA_SCANNER_START_DELAY_SECS` (deprecated alias for compatibility)
 - `RUSTFS_SCANNER_IDLE_MODE` (canonical)
 - `RUSTFS_SCANNER_CACHE_SAVE_TIMEOUT_SECS` (canonical)
+- `RUSTFS_SCANNER_CYCLE_MAX_DURATION_SECS` (canonical)
+- `RUSTFS_SCANNER_CYCLE_MAX_OBJECTS` (canonical)
+- `RUSTFS_SCANNER_CYCLE_MAX_DIRECTORIES` (canonical)
 
 ## Health compatibility switches
 

crates/config/src/audit/kafka.rs

@@ -21,10 +21,14 @@ pub const ENV_AUDIT_KAFKA_TLS_ENABLE: &str = "RUSTFS_AUDIT_KAFKA_TLS_ENABLE";
 pub const ENV_AUDIT_KAFKA_TLS_CA: &str = "RUSTFS_AUDIT_KAFKA_TLS_CA";
 pub const ENV_AUDIT_KAFKA_TLS_CLIENT_CERT: &str = "RUSTFS_AUDIT_KAFKA_TLS_CLIENT_CERT";
 pub const ENV_AUDIT_KAFKA_TLS_CLIENT_KEY: &str = "RUSTFS_AUDIT_KAFKA_TLS_CLIENT_KEY";
+pub const ENV_AUDIT_KAFKA_SASL_ENABLE: &str = "RUSTFS_AUDIT_KAFKA_SASL_ENABLE";
+pub const ENV_AUDIT_KAFKA_SASL_MECHANISM: &str = "RUSTFS_AUDIT_KAFKA_SASL_MECHANISM";
+pub const ENV_AUDIT_KAFKA_SASL_USERNAME: &str = "RUSTFS_AUDIT_KAFKA_SASL_USERNAME";
+pub const ENV_AUDIT_KAFKA_SASL_PASSWORD: &str = "RUSTFS_AUDIT_KAFKA_SASL_PASSWORD";
 pub const ENV_AUDIT_KAFKA_QUEUE_DIR: &str = "RUSTFS_AUDIT_KAFKA_QUEUE_DIR";
 pub const ENV_AUDIT_KAFKA_QUEUE_LIMIT: &str = "RUSTFS_AUDIT_KAFKA_QUEUE_LIMIT";
 
-pub const ENV_AUDIT_KAFKA_KEYS: &[&str; 10] = &[
+pub const ENV_AUDIT_KAFKA_KEYS: &[&str; 14] = &[
     ENV_AUDIT_KAFKA_ENABLE,
     ENV_AUDIT_KAFKA_BROKERS,
     ENV_AUDIT_KAFKA_TOPIC,
@@ -33,6 +37,10 @@ pub const ENV_AUDIT_KAFKA_KEYS: &[&str; 10] = &[
     ENV_AUDIT_KAFKA_TLS_CA,
     ENV_AUDIT_KAFKA_TLS_CLIENT_CERT,
     ENV_AUDIT_KAFKA_TLS_CLIENT_KEY,
+    ENV_AUDIT_KAFKA_SASL_ENABLE,
+    ENV_AUDIT_KAFKA_SASL_MECHANISM,
+    ENV_AUDIT_KAFKA_SASL_USERNAME,
+    ENV_AUDIT_KAFKA_SASL_PASSWORD,
     ENV_AUDIT_KAFKA_QUEUE_DIR,
     ENV_AUDIT_KAFKA_QUEUE_LIMIT,
 ];
@@ -47,6 +55,10 @@ pub const AUDIT_KAFKA_KEYS: &[&str] = &[
     crate::KAFKA_TLS_CA,
     crate::KAFKA_TLS_CLIENT_CERT,
     crate::KAFKA_TLS_CLIENT_KEY,
+    crate::KAFKA_SASL_ENABLE,
+    crate::KAFKA_SASL_MECHANISM,
+    crate::KAFKA_SASL_USERNAME,
+    crate::KAFKA_SASL_PASSWORD,
     crate::KAFKA_QUEUE_DIR,
     crate::KAFKA_QUEUE_LIMIT,
     crate::COMMENT_KEY,

crates/config/src/constants/object.rs

@@ -281,6 +281,40 @@ pub const ENV_OBJECT_LOCK_ACQUIRE_TIMEOUT: &str = "RUSTFS_OBJECT_LOCK_ACQUIRE_TI
 /// Default lock acquisition timeout: 5 seconds.
 pub const DEFAULT_OBJECT_LOCK_ACQUIRE_TIMEOUT: u64 = 5;
 
+/// Environment variable to enable object namespace lock diagnostics.
+///
+/// When enabled, RustFS emits slow lock acquisition and long lock hold
+/// warnings for object-level namespace locks. This is intended for
+/// production debugging of contention on hot object keys.
+///
+/// Default: false (disabled, can be overridden by `RUSTFS_OBJECT_LOCK_DIAG_ENABLE`).
+pub const ENV_OBJECT_LOCK_DIAG_ENABLE: &str = "RUSTFS_OBJECT_LOCK_DIAG_ENABLE";
+
+/// Default: object lock diagnostics are disabled.
+pub const DEFAULT_OBJECT_LOCK_DIAG_ENABLE: bool = false;
+
+/// Environment variable for the slow object lock acquisition threshold in milliseconds.
+///
+/// When a read or write namespace lock takes at least this long to acquire,
+/// RustFS emits a warning with the operation name and object key.
+///
+/// Default: 500 milliseconds.
+pub const ENV_OBJECT_LOCK_DIAG_SLOW_ACQUIRE_MS: &str = "RUSTFS_OBJECT_LOCK_DIAG_SLOW_ACQUIRE_MS";
+
+/// Default slow object lock acquisition threshold: 500 milliseconds.
+pub const DEFAULT_OBJECT_LOCK_DIAG_SLOW_ACQUIRE_MS: u64 = 500;
+
+/// Environment variable for the long object lock hold threshold in milliseconds.
+///
+/// When a namespace lock guard is held for at least this long, RustFS emits
+/// a warning when the guard is dropped.
+///
+/// Default: 1000 milliseconds.
+pub const ENV_OBJECT_LOCK_DIAG_SLOW_HOLD_MS: &str = "RUSTFS_OBJECT_LOCK_DIAG_SLOW_HOLD_MS";
+
+/// Default long object lock hold threshold: 1000 milliseconds.
+pub const DEFAULT_OBJECT_LOCK_DIAG_SLOW_HOLD_MS: u64 = 1000;
+
 // ============================================================================
 // I/O priority scheduling configuration
 // ============================================================================

crates/config/src/constants/oidc.rs

@@ -28,6 +28,7 @@ pub const OIDC_GROUPS_CLAIM: &str = "groups_claim";
 pub const OIDC_ROLES_CLAIM: &str = "roles_claim";
 pub const OIDC_EMAIL_CLAIM: &str = "email_claim";
 pub const OIDC_USERNAME_CLAIM: &str = "username_claim";
+pub const OIDC_HIDE_FROM_UI: &str = "hide_from_ui";
 
 // Environment variable names for OIDC
 pub const ENV_IDENTITY_OPENID_ENABLE: &str = "RUSTFS_IDENTITY_OPENID_ENABLE";
@@ -46,9 +47,10 @@ pub const ENV_IDENTITY_OPENID_GROUPS_CLAIM: &str = "RUSTFS_IDENTITY_OPENID_GROUP
 pub const ENV_IDENTITY_OPENID_ROLES_CLAIM: &str = "RUSTFS_IDENTITY_OPENID_ROLES_CLAIM";
 pub const ENV_IDENTITY_OPENID_EMAIL_CLAIM: &str = "RUSTFS_IDENTITY_OPENID_EMAIL_CLAIM";
 pub const ENV_IDENTITY_OPENID_USERNAME_CLAIM: &str = "RUSTFS_IDENTITY_OPENID_USERNAME_CLAIM";
+pub const ENV_IDENTITY_OPENID_HIDE_FROM_UI: &str = "RUSTFS_IDENTITY_OPENID_HIDE_FROM_UI";
 
 /// List of all environment variable keys for an OIDC provider.
-pub const ENV_IDENTITY_OPENID_KEYS: &[&str; 16] = &[
+pub const ENV_IDENTITY_OPENID_KEYS: &[&str; 17] = &[
     ENV_IDENTITY_OPENID_ENABLE,
     ENV_IDENTITY_OPENID_CONFIG_URL,
     ENV_IDENTITY_OPENID_CLIENT_ID,
@@ -65,6 +67,7 @@ pub const ENV_IDENTITY_OPENID_KEYS: &[&str; 16] = &[
     ENV_IDENTITY_OPENID_ROLES_CLAIM,
     ENV_IDENTITY_OPENID_EMAIL_CLAIM,
     ENV_IDENTITY_OPENID_USERNAME_CLAIM,
+    ENV_IDENTITY_OPENID_HIDE_FROM_UI,
 ];
 
 /// A list of all valid configuration keys for an OIDC provider.
@@ -85,6 +88,7 @@ pub const IDENTITY_OPENID_KEYS: &[&str] = &[
     OIDC_ROLES_CLAIM,
     OIDC_EMAIL_CLAIM,
     OIDC_USERNAME_CLAIM,
+    OIDC_HIDE_FROM_UI,
     crate::COMMENT_KEY,
 ];
 

crates/config/src/constants/runtime.rs

@@ -69,6 +69,10 @@ pub const DEFAULT_TRANSITION_QUEUE_CAPACITY: usize = 1000;
 pub const DEFAULT_TRANSITION_QUEUE_SEND_TIMEOUT_MS: usize = 100;
 /// Test-only fault injection env var that forces the immediate transition enqueue timeout path.
 pub const ENV_TEST_FORCE_IMMEDIATE_TRANSITION_ENQUEUE_TIMEOUT: &str = "RUSTFS_TEST_FORCE_IMMEDIATE_TRANSITION_ENQUEUE_TIMEOUT";
+/// Test-only fault injection env var that forces a number of IAM bootstrap failures.
+pub const ENV_TEST_IAM_FAIL_INIT_ATTEMPTS: &str = "RUSTFS_TEST_IAM_FAIL_INIT_ATTEMPTS";
+/// Test-only env var that overrides the deferred IAM retry interval in debug builds.
+pub const ENV_TEST_IAM_RETRY_INTERVAL_MS: &str = "RUSTFS_TEST_IAM_RETRY_INTERVAL_MS";
 /// Runtime env var controlling the transition worker count.
 pub const ENV_TRANSITION_WORKERS: &str = "RUSTFS_MAX_TRANSITION_WORKERS";
 /// Runtime env var controlling the absolute maximum transition workers.

crates/config/src/constants/scanner.rs

@@ -14,6 +14,76 @@
 
 use std::time::Duration;
 
+/// Scanner admin config subsystem name.
+pub const SCANNER_SUB_SYS: &str = "scanner";
+
+/// Scanner config key selecting the speed preset.
+pub const SCANNER_SPEED: &str = "speed";
+
+/// Scanner config key overriding the cycle interval in seconds.
+pub const SCANNER_CYCLE: &str = "cycle";
+
+/// Scanner config key setting the startup delay in seconds.
+///
+/// For compatibility, this also acts as the scanner cycle interval when
+/// `cycle` is unset.
+pub const SCANNER_START_DELAY: &str = "start_delay";
+
+/// Scanner config key capping one cycle's runtime in seconds.
+pub const SCANNER_CYCLE_MAX_DURATION: &str = "cycle_max_duration";
+
+/// Scanner config key capping objects processed by one cycle.
+pub const SCANNER_CYCLE_MAX_OBJECTS: &str = "cycle_max_objects";
+
+/// Scanner config key capping directories entered by one cycle.
+pub const SCANNER_CYCLE_MAX_DIRECTORIES: &str = "cycle_max_directories";
+
+/// Scanner config key setting the periodic bitrot scan cycle in seconds.
+pub const SCANNER_BITROT_CYCLE: &str = "bitrot_cycle";
+
+/// Scanner config key controlling whether scanner throttling is enabled.
+pub const SCANNER_IDLE_MODE: &str = "idle_mode";
+
+/// Scanner config key controlling scanner cache save timeout in seconds.
+pub const SCANNER_CACHE_SAVE_TIMEOUT: &str = "cache_save_timeout";
+
+/// Scanner config key capping concurrent scanner set tasks.
+pub const SCANNER_MAX_CONCURRENT_SET_SCANS: &str = "max_concurrent_set_scans";
+
+/// Scanner config key capping concurrent scanner disk bucket walks per set.
+pub const SCANNER_MAX_CONCURRENT_DISK_SCANS: &str = "max_concurrent_disk_scans";
+
+/// Scanner config key controlling how often object loops yield.
+pub const SCANNER_YIELD_EVERY_N_OBJECTS: &str = "yield_every_n_objects";
+
+/// Scanner config key controlling object version count alerts.
+pub const SCANNER_ALERT_EXCESS_VERSIONS: &str = "alert_excess_versions";
+
+/// Scanner config key controlling retained version size alerts.
+pub const SCANNER_ALERT_EXCESS_VERSION_SIZE: &str = "alert_excess_version_size";
+
+/// Scanner config key controlling direct subfolder count alerts.
+pub const SCANNER_ALERT_EXCESS_FOLDERS: &str = "alert_excess_folders";
+
+/// Scanner config keys supported by the admin config subsystem.
+pub const SCANNER_KEYS: &[&str] = &[
+    SCANNER_SPEED,
+    SCANNER_CYCLE,
+    SCANNER_START_DELAY,
+    SCANNER_CYCLE_MAX_DURATION,
+    SCANNER_CYCLE_MAX_OBJECTS,
+    SCANNER_CYCLE_MAX_DIRECTORIES,
+    SCANNER_BITROT_CYCLE,
+    SCANNER_IDLE_MODE,
+    SCANNER_CACHE_SAVE_TIMEOUT,
+    SCANNER_MAX_CONCURRENT_SET_SCANS,
+    SCANNER_MAX_CONCURRENT_DISK_SCANS,
+    SCANNER_YIELD_EVERY_N_OBJECTS,
+    SCANNER_ALERT_EXCESS_VERSIONS,
+    SCANNER_ALERT_EXCESS_VERSION_SIZE,
+    SCANNER_ALERT_EXCESS_FOLDERS,
+];
+
 /// Canonical environment variable name that specifies the scanner start delay in seconds.
 /// If set, this overrides the cycle interval derived from `RUSTFS_SCANNER_SPEED`.
 /// - Unit: seconds (u64).
@@ -31,6 +101,22 @@ pub const ENV_DATA_SCANNER_START_DELAY_SECS: &str = "RUSTFS_DATA_SCANNER_START_D
 /// - Example: `export RUSTFS_SCANNER_CYCLE=3600` (1 hour)
 pub const ENV_SCANNER_CYCLE: &str = "RUSTFS_SCANNER_CYCLE";
 
+/// Environment variable that caps one scanner cycle's runtime in seconds.
+/// A value of `0` disables the cycle runtime budget.
+/// - Unit: seconds (u64).
+/// - Example: `export RUSTFS_SCANNER_CYCLE_MAX_DURATION_SECS=1800`
+pub const ENV_SCANNER_CYCLE_MAX_DURATION_SECS: &str = "RUSTFS_SCANNER_CYCLE_MAX_DURATION_SECS";
+
+/// Environment variable that caps objects processed by one scanner cycle.
+/// A value of `0` disables the per-cycle object budget.
+/// - Example: `export RUSTFS_SCANNER_CYCLE_MAX_OBJECTS=1000000`
+pub const ENV_SCANNER_CYCLE_MAX_OBJECTS: &str = "RUSTFS_SCANNER_CYCLE_MAX_OBJECTS";
+
+/// Environment variable that caps directories entered by one scanner cycle.
+/// A value of `0` disables the per-cycle directory budget.
+/// - Example: `export RUSTFS_SCANNER_CYCLE_MAX_DIRECTORIES=100000`
+pub const ENV_SCANNER_CYCLE_MAX_DIRECTORIES: &str = "RUSTFS_SCANNER_CYCLE_MAX_DIRECTORIES";
+
 /// Environment variable that selects the scanner speed preset.
 /// Valid values: `fastest`, `fast`, `default`, `slow`, `slowest`.
 /// Controls the sleep factor, maximum sleep duration, and cycle interval.
@@ -40,6 +126,18 @@ pub const ENV_SCANNER_SPEED: &str = "RUSTFS_SCANNER_SPEED";
 /// Default scanner speed preset.
 pub const DEFAULT_SCANNER_SPEED: &str = "default";
 
+/// Default scanner cycle runtime budget.
+/// `0` keeps the existing unbounded per-cycle behavior.
+pub const DEFAULT_SCANNER_CYCLE_MAX_DURATION_SECS: u64 = 0;
+
+/// Default scanner per-cycle object budget.
+/// `0` keeps the existing unbounded per-cycle behavior.
+pub const DEFAULT_SCANNER_CYCLE_MAX_OBJECTS: u64 = 0;
+
+/// Default scanner per-cycle directory budget.
+/// `0` keeps the existing unbounded per-cycle behavior.
+pub const DEFAULT_SCANNER_CYCLE_MAX_DIRECTORIES: u64 = 0;
+
 /// Environment variable that specifies the periodic bitrot scan cycle in seconds.
 /// When set to `0`, `true`, `on`, or `yes`, every scanner cycle runs in deep mode.
 /// When set to `false`, `off`, `no`, or `disabled`, periodic deep scans are disabled.
@@ -65,11 +163,12 @@ pub const ENV_SCANNER_ALERT_EXCESS_VERSION_SIZE: &str = "RUSTFS_SCANNER_ALERT_EX
 pub const DEFAULT_SCANNER_ALERT_EXCESS_VERSION_SIZE: u64 = 1024 * 1024 * 1024 * 1024;
 
 /// Environment variable that controls how many subfolders trigger scanner alerts.
-/// - Example: `export RUSTFS_SCANNER_ALERT_EXCESS_FOLDERS=50000`
+/// - Example: `export RUSTFS_SCANNER_ALERT_EXCESS_FOLDERS=65538`
 pub const ENV_SCANNER_ALERT_EXCESS_FOLDERS: &str = "RUSTFS_SCANNER_ALERT_EXCESS_FOLDERS";
 
 /// Default subfolder count that triggers scanner alerts.
-pub const DEFAULT_SCANNER_ALERT_EXCESS_FOLDERS: u64 = 50_000;
+/// Allows Proxmox Backup Server's chunk namespace layout.
+pub const DEFAULT_SCANNER_ALERT_EXCESS_FOLDERS: u64 = 65_538;
 
 /// Environment variable that controls whether the scanner sleeps between operations.
 /// When `true` (default), the scanner throttles itself. When `false`, it runs at full speed.
@@ -82,9 +181,38 @@ pub const ENV_SCANNER_IDLE_MODE: &str = "RUSTFS_SCANNER_IDLE_MODE";
 /// - Example: `export RUSTFS_SCANNER_CACHE_SAVE_TIMEOUT_SECS=30`
 pub const ENV_SCANNER_CACHE_SAVE_TIMEOUT_SECS: &str = "RUSTFS_SCANNER_CACHE_SAVE_TIMEOUT_SECS";
 
+/// Default scanner cache save timeout in seconds.
+pub const DEFAULT_SCANNER_CACHE_SAVE_TIMEOUT_SECS: u64 = 30;
+
+/// Environment variable that caps concurrent scanner set tasks.
+/// A value of `0` keeps the existing topology-based concurrency.
+/// - Example: `export RUSTFS_SCANNER_MAX_CONCURRENT_SET_SCANS=2`
+pub const ENV_SCANNER_MAX_CONCURRENT_SET_SCANS: &str = "RUSTFS_SCANNER_MAX_CONCURRENT_SET_SCANS";
+
+/// Environment variable that caps concurrent scanner disk bucket walks per set.
+/// A value of `0` keeps the existing disk-count-based concurrency.
+/// - Example: `export RUSTFS_SCANNER_MAX_CONCURRENT_DISK_SCANS=1`
+pub const ENV_SCANNER_MAX_CONCURRENT_DISK_SCANS: &str = "RUSTFS_SCANNER_MAX_CONCURRENT_DISK_SCANS";
+
+/// Environment variable that controls how often scanner object loops yield to the async runtime.
+/// A value of `0` disables this extra object-count yield.
+/// - Example: `export RUSTFS_SCANNER_YIELD_EVERY_N_OBJECTS=32`
+pub const ENV_SCANNER_YIELD_EVERY_N_OBJECTS: &str = "RUSTFS_SCANNER_YIELD_EVERY_N_OBJECTS";
+
 /// Default scanner idle mode.
 pub const DEFAULT_SCANNER_IDLE_MODE: bool = true;
 
+/// Default set scan concurrency budget.
+/// `0` means no additional limit beyond deployment topology.
+pub const DEFAULT_SCANNER_MAX_CONCURRENT_SET_SCANS: usize = 0;
+
+/// Default disk scan concurrency budget.
+/// `0` means no additional limit beyond available disks in the set.
+pub const DEFAULT_SCANNER_MAX_CONCURRENT_DISK_SCANS: usize = 0;
+
+/// Default object interval for cooperative scanner yields.
+pub const DEFAULT_SCANNER_YIELD_EVERY_N_OBJECTS: u64 = 128;
+
 /// Compatibility flag kept for Patch 3 rollback windows.
 ///
 /// Inline scanner heal execution has been removed in favor of heal-candidate enqueue.
@@ -148,15 +276,20 @@ impl ScannerSpeed {
         }
     }
 
-    pub fn from_env_str(s: &str) -> Self {
+    pub fn parse_str(s: &str) -> Option<Self> {
         match s.trim().to_ascii_lowercase().as_str() {
-            "fastest" => Self::Fastest,
-            "fast" => Self::Fast,
-            "slow" => Self::Slow,
-            "slowest" => Self::Slowest,
-            _ => Self::Default,
+            "fastest" => Some(Self::Fastest),
+            "fast" => Some(Self::Fast),
+            "default" => Some(Self::Default),
+            "slow" => Some(Self::Slow),
+            "slowest" => Some(Self::Slowest),
+            _ => None,
         }
     }
+
+    pub fn from_env_str(s: &str) -> Self {
+        Self::parse_str(s).unwrap_or_default()
+    }
 }
 
 impl std::fmt::Display for ScannerSpeed {

crates/config/src/constants/targets.rs

@@ -49,6 +49,10 @@ pub const KAFKA_TLS_ENABLE: &str = "tls_enable";
 pub const KAFKA_TLS_CA: &str = "tls_ca";
 pub const KAFKA_TLS_CLIENT_CERT: &str = "tls_client_cert";
 pub const KAFKA_TLS_CLIENT_KEY: &str = "tls_client_key";
+pub const KAFKA_SASL_ENABLE: &str = "sasl_enable";
+pub const KAFKA_SASL_MECHANISM: &str = "sasl_mechanism";
+pub const KAFKA_SASL_USERNAME: &str = "sasl_username";
+pub const KAFKA_SASL_PASSWORD: &str = "sasl_password";
 
 pub const AMQP_URL: &str = "url";
 pub const AMQP_EXCHANGE: &str = "exchange";

crates/config/src/notify/kafka.rs

@@ -22,6 +22,10 @@ pub const NOTIFY_KAFKA_KEYS: &[&str] = &[
     crate::KAFKA_TLS_CA,
     crate::KAFKA_TLS_CLIENT_CERT,
     crate::KAFKA_TLS_CLIENT_KEY,
+    crate::KAFKA_SASL_ENABLE,
+    crate::KAFKA_SASL_MECHANISM,
+    crate::KAFKA_SASL_USERNAME,
+    crate::KAFKA_SASL_PASSWORD,
     crate::KAFKA_QUEUE_DIR,
     crate::KAFKA_QUEUE_LIMIT,
     crate::COMMENT_KEY,
@@ -36,10 +40,14 @@ pub const ENV_NOTIFY_KAFKA_TLS_ENABLE: &str = "RUSTFS_NOTIFY_KAFKA_TLS_ENABLE";
 pub const ENV_NOTIFY_KAFKA_TLS_CA: &str = "RUSTFS_NOTIFY_KAFKA_TLS_CA";
 pub const ENV_NOTIFY_KAFKA_TLS_CLIENT_CERT: &str = "RUSTFS_NOTIFY_KAFKA_TLS_CLIENT_CERT";
 pub const ENV_NOTIFY_KAFKA_TLS_CLIENT_KEY: &str = "RUSTFS_NOTIFY_KAFKA_TLS_CLIENT_KEY";
+pub const ENV_NOTIFY_KAFKA_SASL_ENABLE: &str = "RUSTFS_NOTIFY_KAFKA_SASL_ENABLE";
+pub const ENV_NOTIFY_KAFKA_SASL_MECHANISM: &str = "RUSTFS_NOTIFY_KAFKA_SASL_MECHANISM";
+pub const ENV_NOTIFY_KAFKA_SASL_USERNAME: &str = "RUSTFS_NOTIFY_KAFKA_SASL_USERNAME";
+pub const ENV_NOTIFY_KAFKA_SASL_PASSWORD: &str = "RUSTFS_NOTIFY_KAFKA_SASL_PASSWORD";
 pub const ENV_NOTIFY_KAFKA_QUEUE_DIR: &str = "RUSTFS_NOTIFY_KAFKA_QUEUE_DIR";
 pub const ENV_NOTIFY_KAFKA_QUEUE_LIMIT: &str = "RUSTFS_NOTIFY_KAFKA_QUEUE_LIMIT";
 
-pub const ENV_NOTIFY_KAFKA_KEYS: &[&str; 10] = &[
+pub const ENV_NOTIFY_KAFKA_KEYS: &[&str; 14] = &[
     ENV_NOTIFY_KAFKA_ENABLE,
     ENV_NOTIFY_KAFKA_BROKERS,
     ENV_NOTIFY_KAFKA_TOPIC,
@@ -48,6 +56,10 @@ pub const ENV_NOTIFY_KAFKA_KEYS: &[&str; 10] = &[
     ENV_NOTIFY_KAFKA_TLS_CA,
     ENV_NOTIFY_KAFKA_TLS_CLIENT_CERT,
     ENV_NOTIFY_KAFKA_TLS_CLIENT_KEY,
+    ENV_NOTIFY_KAFKA_SASL_ENABLE,
+    ENV_NOTIFY_KAFKA_SASL_MECHANISM,
+    ENV_NOTIFY_KAFKA_SASL_USERNAME,
+    ENV_NOTIFY_KAFKA_SASL_PASSWORD,
     ENV_NOTIFY_KAFKA_QUEUE_DIR,
     ENV_NOTIFY_KAFKA_QUEUE_LIMIT,
 ];

crates/data-usage/src/data_usage.rs

@@ -688,15 +688,14 @@ impl DataUsageCache {
             return;
         }
 
-        let mut leaves = Vec::new();
+        let mut candidates = Vec::new();
         let mut remove = total - limit;
-        add(self, path, &mut leaves);
-        leaves.sort_by_key(|a| a.objects);
+        add(self, path, &mut candidates);
+        candidates.sort_by_key(|a| a.objects);
 
-        while remove > 0 && !leaves.is_empty() {
-            let Some(e) = leaves.first() else {
-                break;
-            };
+        let mut candidate_index = 0;
+        while remove > 0 && candidate_index < candidates.len() {
+            let e = &candidates[candidate_index];
             let candidate = e.path.clone();
             if candidate == *path && !compact_self {
                 break;
@@ -705,7 +704,7 @@ impl DataUsageCache {
             let mut flat = match self.size_recursive(&candidate.key()) {
                 Some(flat) => flat,
                 None => {
-                    leaves.remove(0);
+                    candidate_index += 1;
                     continue;
                 }
             };
@@ -714,8 +713,8 @@ impl DataUsageCache {
             self.delete_recursive(&candidate);
             self.replace_hashed(&candidate, &None, &flat);
 
-            remove -= removing;
-            leaves.remove(0);
+            remove = remove.saturating_sub(removing);
+            candidate_index += 1;
         }
     }
 
@@ -869,23 +868,24 @@ struct Inner {
     path: DataUsageHash,
 }
 
-fn add(data_usage_cache: &DataUsageCache, path: &DataUsageHash, leaves: &mut Vec<Inner>) {
+fn add(data_usage_cache: &DataUsageCache, path: &DataUsageHash, candidates: &mut Vec<Inner>) -> usize {
     let e = match data_usage_cache.cache.get(&path.key()) {
         Some(e) => e,
-        None => return,
+        None => return 0,
     };
-    if !e.children.is_empty() {
-        return;
-    }
-
-    let sz = data_usage_cache.size_recursive(&path.key()).unwrap_or_default();
-    leaves.push(Inner {
-        objects: sz.objects,
-        path: path.clone(),
-    });
+    let mut objects = e.objects;
     for ch in e.children.iter() {
-        add(data_usage_cache, &DataUsageHash(ch.clone()), leaves);
+        objects += add(data_usage_cache, &DataUsageHash(ch.clone()), candidates);
     }
+    // Collect internal nodes (with children) as compaction candidates.
+    // Leaf nodes have no children to remove, so compacting them is a no-op.
+    if !e.children.is_empty() {
+        candidates.push(Inner {
+            objects,
+            path: path.clone(),
+        });
+    }
+    objects
 }
 
 fn mark(duc: &DataUsageCache, entry: &DataUsageEntry, found: &mut HashSet<String>) {
@@ -1361,4 +1361,161 @@ mod tests {
         assert_eq!(child_entry.size, 30);
         assert_eq!(child_entry.objects, 3);
     }
+
+    // --- Tests for `add` and `reduce_children_of` (bug fixes) ---
+
+    /// Build a small tree: root -> child1 (leaf), child2 -> grandchild (leaf).
+    fn build_test_tree() -> (DataUsageCache, DataUsageHash) {
+        let root = hash_path("bucket");
+        let c1 = hash_path("bucket/a");
+        let c2 = hash_path("bucket/b");
+        let gc = hash_path("bucket/b/c");
+
+        let mut cache = DataUsageCache::default();
+        cache.replace_hashed(&root, &None, &DataUsageEntry::default());
+        cache.replace_hashed(
+            &c1,
+            &Some(root.clone()),
+            &DataUsageEntry {
+                objects: 1,
+                size: 10,
+                ..Default::default()
+            },
+        );
+        cache.replace_hashed(
+            &c2,
+            &Some(root.clone()),
+            &DataUsageEntry {
+                objects: 2,
+                size: 20,
+                ..Default::default()
+            },
+        );
+        cache.replace_hashed(
+            &gc,
+            &Some(c2.clone()),
+            &DataUsageEntry {
+                objects: 3,
+                size: 30,
+                ..Default::default()
+            },
+        );
+        (cache, root)
+    }
+
+    fn build_underflow_test_tree() -> (DataUsageCache, DataUsageHash) {
+        let root = hash_path("bucket");
+        let small = hash_path("bucket/small");
+        let small_a = hash_path("bucket/small/a");
+        let small_b = hash_path("bucket/small/b");
+        let large = hash_path("bucket/large");
+        let large_a = hash_path("bucket/large/a");
+        let large_b = hash_path("bucket/large/b");
+
+        let mut cache = DataUsageCache::default();
+        cache.replace_hashed(
+            &root,
+            &None,
+            &DataUsageEntry {
+                objects: 100,
+                ..Default::default()
+            },
+        );
+        cache.replace_hashed(&small, &Some(root.clone()), &DataUsageEntry::default());
+        cache.replace_hashed(
+            &small_a,
+            &Some(small.clone()),
+            &DataUsageEntry {
+                objects: 1,
+                ..Default::default()
+            },
+        );
+        cache.replace_hashed(
+            &small_b,
+            &Some(small.clone()),
+            &DataUsageEntry {
+                objects: 1,
+                ..Default::default()
+            },
+        );
+        cache.replace_hashed(&large, &Some(root.clone()), &DataUsageEntry::default());
+        cache.replace_hashed(
+            &large_a,
+            &Some(large.clone()),
+            &DataUsageEntry {
+                objects: 10,
+                ..Default::default()
+            },
+        );
+        cache.replace_hashed(
+            &large_b,
+            &Some(large.clone()),
+            &DataUsageEntry {
+                objects: 10,
+                ..Default::default()
+            },
+        );
+        (cache, root)
+    }
+
+    #[test]
+    fn test_add_collects_internal_nodes_as_compaction_candidates() {
+        let (cache, root) = build_test_tree();
+        let mut candidates = Vec::new();
+        add(&cache, &root, &mut candidates);
+
+        let mut paths: Vec<String> = candidates.iter().map(|l| l.path.key()).collect();
+        paths.sort();
+        assert_eq!(paths.len(), 2, "add() should find internal nodes with children");
+        assert!(paths.contains(&hash_path("bucket").key()));
+        assert!(paths.contains(&hash_path("bucket/b").key()));
+    }
+
+    #[test]
+    fn test_add_skips_leaf_node() {
+        let mut cache = DataUsageCache::default();
+        let h = hash_path("single-leaf");
+        cache.replace_hashed(
+            &h,
+            &None,
+            &DataUsageEntry {
+                objects: 5,
+                size: 50,
+                ..Default::default()
+            },
+        );
+
+        let mut candidates = Vec::new();
+        add(&cache, &h, &mut candidates);
+        assert!(candidates.is_empty(), "leaf node should not be a compaction candidate");
+    }
+
+    #[test]
+    fn test_reduce_children_of_compacts_internal_node() {
+        let (mut cache, root) = build_test_tree();
+        cache.reduce_children_of(&root, 2, false);
+
+        let entry_c2 = cache.find("bucket/b").unwrap();
+        assert!(entry_c2.compacted, "internal node 'bucket/b' should be compacted");
+        let entry_c1 = cache.find("bucket/a").unwrap();
+        assert!(!entry_c1.compacted, "leaf 'bucket/a' should not be compacted");
+        assert!(cache.find("bucket/b/c").is_none(), "grandchild should be removed");
+    }
+
+    #[test]
+    fn test_reduce_children_of_usize_underflow_saturates() {
+        let (mut cache, root) = build_underflow_test_tree();
+
+        // total children=6, limit=5, remove=1. The smallest candidate removes
+        // two descendants, so plain subtraction would underflow and compact the
+        // next candidate too.
+        cache.reduce_children_of(&root, 5, false);
+
+        assert!(cache.find("bucket/small").is_some_and(|entry| entry.compacted));
+        assert!(cache.find("bucket/small/a").is_none());
+        assert!(cache.find("bucket/small/b").is_none());
+        assert!(cache.find("bucket/large").is_some_and(|entry| !entry.compacted));
+        assert!(cache.find("bucket/large/a").is_some());
+        assert!(cache.find("bucket/large/b").is_some());
+    }
 }

crates/ecstore/AGENTS.md

@@ -14,6 +14,30 @@ Applies to `crates/ecstore/`.
 - Keep network and disk operations async-friendly; avoid introducing unnecessary blocking.
 - Benchmark-sensitive changes should include measurable rationale.
 
+## Allocation Discipline in Hot Paths
+
+- Do not implement `Clone` on structs with >5 heap-allocated fields without considering `Arc` for heavy fields.
+- Before cloning a struct in a loop or per-request path, check if a reference or `Cow` would suffice.
+- When a struct contains a large buffer (e.g., erasure coding block), wrap it in `Arc` and pass by reference rather than cloning.
+- Use `&str` and `Cow<str>` instead of `String` for temporary computations (header parsing, signature building, path manipulation).
+- Use `HashMap::with_capacity()` / `Vec::with_capacity()` when the size is known or estimable.
+
+## Lock Ordering
+
+- When a function acquires multiple locks, document the acquisition order in a comment.
+- Never acquire the same set of locks in different orders across code paths — this is a deadlock.
+- Prefer `compare_exchange` loops over load-then-store for concurrent counters.
+
+## Recursion Safety
+
+- Recursive tree traversals (cache trees, directory walks) must have a depth limit or use iterative traversal with an explicit `Vec` stack.
+- `flatten()`, `delete_recursive()`, `copy_with_children()`, `total_children_rec()`, `mark()` — all must handle deep or corrupted trees safely.
+
+## Dead Code
+
+- Do not add `#![allow(dead_code)]` at the crate root. If code is unused, remove it or gate it behind a feature flag.
+- Each `#[allow(dead_code)]` annotation must have a comment explaining why the code is kept.
+
 ## Cross-Module Coordination
 
 - Validate behavior impacts on:

crates/ecstore/docs/s3-table/s3-table-concept-model.md

@@ -0,0 +1,130 @@
+# RustFS S3 Table Core Concepts
+
+This document keeps only the core concepts for adding table catalog capability
+to RustFS. The direction is to build an Iceberg REST Catalog-compatible layer
+on top of existing RustFS object storage.
+
+## Concept Model
+
+```text
+TableBucket
+  Warehouse
+    Namespace
+      TableIdentifier
+        TableMetadataLocation
+        TableMetadata
+          Schema
+          Snapshot
+          Manifest
+          DataFile
+```
+
+## TableBucket
+
+`TableBucket` is a normal RustFS bucket enabled for table catalog use. It is not
+a new physical bucket type.
+
+RustFS can mark it with internal bucket metadata such as `table-bucket.json`.
+That marker is an implementation detail, not a public compatibility API.
+
+## Warehouse
+
+`Warehouse` is the catalog storage root exposed to Iceberg clients. In the first
+RustFS model, a warehouse maps to a table-enabled bucket.
+
+Clients should interact with the warehouse through catalog APIs, not by knowing
+RustFS internal metadata paths.
+
+## Namespace
+
+`Namespace` groups tables inside a warehouse. It is a first-class catalog
+resource, not a raw object prefix.
+
+Namespace validation must be conservative: no path traversal, encoded
+separators, empty segments, control characters, or ambiguous normalization.
+
+## TableIdentifier
+
+`TableIdentifier` names one table inside a warehouse and namespace.
+
+```text
+warehouse + namespace + table name
+```
+
+It is resolved by the catalog layer to internal metadata objects. It should not
+be treated as an S3 object key.
+
+## TableMetadataLocation
+
+`TableMetadataLocation` points to the current Iceberg table metadata file.
+
+It is the main mutable table pointer. Only catalog commit code may update it.
+Updates must use conflict detection so stale writers cannot overwrite newer
+table state.
+
+## TableMetadata
+
+`TableMetadata` is the Iceberg metadata JSON referenced by the current metadata
+location.
+
+RustFS should preserve Iceberg metadata exactly where possible. Full parsing can
+be added incrementally for fields RustFS must enforce, such as schemas,
+snapshots, manifest lists, table location, and properties.
+
+## Schema
+
+`Schema` describes table columns.
+
+RustFS does not need to interpret every schema detail at the beginning, but it
+must preserve schema metadata without corrupting unknown Iceberg fields.
+
+## Snapshot
+
+`Snapshot` represents an immutable table state at a point in time.
+
+Catalog commits advance the current metadata to a new snapshot. Maintenance
+features such as rollback and snapshot expiration can be added later.
+
+## Manifest
+
+`Manifest` and manifest lists describe table data-file additions, deletions, and
+references.
+
+RustFS should store and protect these as catalog-owned metadata. Full manifest
+parsing can be deferred until validation or maintenance requires it.
+
+## DataFile
+
+`DataFile` is a table data object, commonly Parquet, Avro, or ORC.
+
+Data files live in object storage. The catalog metadata decides which data files
+belong to a table snapshot.
+
+## Reserved Metadata Boundary
+
+Catalog-owned metadata should live under a reserved internal prefix, for
+example:
+
+```text
+.rustfs-table/
+  warehouses/
+    <warehouse-id>/
+      namespaces/
+        <namespace-id>/
+          tables/
+            <table-id>/
+              current.json
+              metadata/
+              manifests/
+```
+
+Ordinary S3 mutating APIs must not create, overwrite, copy into, delete, or
+mutate objects under this reserved prefix for table-enabled buckets.
+
+## Catalog Boundary
+
+S3 object APIs remain object APIs.
+
+Namespace, table, metadata pointer, and commit operations belong to a separate
+catalog layer. Public compatibility should target Iceberg REST Catalog semantics
+rather than exposing RustFS internal metadata objects directly.

crates/ecstore/src/bucket/lifecycle/bucket_lifecycle_ops.rs

@@ -1868,11 +1868,11 @@ pub async fn put_restore_opts(
             ..Default::default()
         });
     }
-    for (k, v) in &oi.user_defined {
+    for (k, v) in oi.user_defined.iter() {
         meta.insert(k.to_string(), v.clone());
     }
     if !oi.user_tags.is_empty() {
-        meta.insert(AMZ_OBJECT_TAGGING.to_string(), oi.user_tags.clone());
+        meta.insert(AMZ_OBJECT_TAGGING.to_string(), (*oi.user_tags).clone());
     }
     let restore_expiry = lifecycle::expected_expiry_time(OffsetDateTime::now_utc(), rreq.days.unwrap_or(1));
     meta.insert(
@@ -1903,7 +1903,7 @@ impl LifecycleOps for ObjectInfo {
     fn to_lifecycle_opts(&self) -> lifecycle::ObjectOpts {
         lifecycle::ObjectOpts {
             name: self.name.clone(),
-            user_tags: self.user_tags.clone(),
+            user_tags: (*self.user_tags).clone(),
             version_id: self.version_id,
             mod_time: self.mod_time,
             size: self.size as usize,
@@ -3043,6 +3043,52 @@ mod tests {
         );
     }
 
+    #[tokio::test]
+    #[ignore = "requires isolated global object layer state"]
+    #[serial]
+    async fn ecstore_new_succeeds_on_fresh_local_volumes() {
+        let test_base_dir = format!("/tmp/rustfs_ecstore_empty_boot_{}", Uuid::new_v4());
+        let temp_dir = PathBuf::from(&test_base_dir);
+        if temp_dir.exists() {
+            fs::remove_dir_all(&temp_dir).await.ok();
+        }
+        fs::create_dir_all(&temp_dir).await.unwrap();
+
+        let disk_paths = vec![
+            temp_dir.join("disk1"),
+            temp_dir.join("disk2"),
+            temp_dir.join("disk3"),
+            temp_dir.join("disk4"),
+        ];
+
+        for disk_path in &disk_paths {
+            fs::create_dir_all(disk_path).await.unwrap();
+        }
+
+        let mut endpoints = Vec::new();
+        for (i, disk_path) in disk_paths.iter().enumerate() {
+            let mut endpoint = Endpoint::try_from(disk_path.to_str().unwrap()).unwrap();
+            endpoint.set_pool_index(0);
+            endpoint.set_set_index(0);
+            endpoint.set_disk_index(i);
+            endpoints.push(endpoint);
+        }
+
+        let endpoint_pools = EndpointServerPools(vec![PoolEndpoints {
+            legacy: false,
+            set_count: 1,
+            drives_per_set: 4,
+            endpoints: Endpoints::from(endpoints),
+            cmd_line: "fresh-boot-test".to_string(),
+            platform: format!("OS: {} | Arch: {}", std::env::consts::OS, std::env::consts::ARCH),
+        }]);
+
+        crate::store::init_local_disks(endpoint_pools.clone()).await.unwrap();
+
+        let ecstore = ECStore::new("127.0.0.1:0".parse().unwrap(), endpoint_pools, CancellationToken::new()).await;
+        assert!(ecstore.is_ok(), "fresh local ECStore boot should succeed, got {ecstore:?}");
+    }
+
     #[tokio::test]
     #[serial]
     async fn stale_multipart_cleanup_uses_default_expiry_without_lifecycle() {

crates/ecstore/src/bucket/lifecycle/core.rs

@@ -881,7 +881,7 @@ impl ObjectOpts {
     pub fn from_object_info(oi: &ObjectInfo) -> Self {
         Self {
             name: oi.name.clone(),
-            user_tags: oi.user_tags.clone(),
+            user_tags: (*oi.user_tags).clone(),
             mod_time: oi.mod_time,
             size: oi.size as usize,
             version_id: oi.version_id,
@@ -894,7 +894,7 @@ impl ObjectOpts {
             restore_expires: oi.restore_expires,
             versioned: false,
             version_suspended: false,
-            user_defined: oi.user_defined.clone(),
+            user_defined: (*oi.user_defined).clone(),
             version_purge_status: oi.version_purge_status.clone(),
             replication_status: oi.replication_status.clone(),
         }

crates/ecstore/src/bucket/metadata.rs

@@ -244,6 +244,8 @@ pub const BUCKET_ACCELERATE_CONFIG: &str = "accelerate.xml";
 pub const BUCKET_REQUEST_PAYMENT_CONFIG: &str = "request-payment.xml";
 pub const BUCKET_PUBLIC_ACCESS_BLOCK_CONFIG: &str = "public-access-block.xml";
 pub const BUCKET_ACL_CONFIG: &str = "bucket-acl.json";
+pub const BUCKET_TABLE_CONFIG: &str = "table-bucket.json";
+pub const BUCKET_TABLE_RESERVED_PREFIX: &str = ".rustfs-table";
 
 #[derive(Debug, Clone)]
 pub struct BucketMetadata {
@@ -268,6 +270,7 @@ pub struct BucketMetadata {
     pub request_payment_config_xml: Vec<u8>,
     pub public_access_block_config_xml: Vec<u8>,
     pub bucket_acl_config_json: Vec<u8>,
+    pub table_bucket_config_json: Vec<u8>,
 
     pub policy_config_updated_at: OffsetDateTime,
     pub object_lock_config_updated_at: OffsetDateTime,
@@ -287,6 +290,7 @@ pub struct BucketMetadata {
     pub request_payment_config_updated_at: OffsetDateTime,
     pub public_access_block_config_updated_at: OffsetDateTime,
     pub bucket_acl_config_updated_at: OffsetDateTime,
+    pub table_bucket_config_updated_at: OffsetDateTime,
 
     pub new_field_updated_at: OffsetDateTime,
 
@@ -334,6 +338,7 @@ impl Default for BucketMetadata {
             request_payment_config_xml: Default::default(),
             public_access_block_config_xml: Default::default(),
             bucket_acl_config_json: Default::default(),
+            table_bucket_config_json: Default::default(),
             policy_config_updated_at: OffsetDateTime::UNIX_EPOCH,
             object_lock_config_updated_at: OffsetDateTime::UNIX_EPOCH,
             encryption_config_updated_at: OffsetDateTime::UNIX_EPOCH,
@@ -352,6 +357,7 @@ impl Default for BucketMetadata {
             request_payment_config_updated_at: OffsetDateTime::UNIX_EPOCH,
             public_access_block_config_updated_at: OffsetDateTime::UNIX_EPOCH,
             bucket_acl_config_updated_at: OffsetDateTime::UNIX_EPOCH,
+            table_bucket_config_updated_at: OffsetDateTime::UNIX_EPOCH,
             new_field_updated_at: OffsetDateTime::UNIX_EPOCH,
             policy_config: Default::default(),
             notification_config: Default::default(),
@@ -397,6 +403,10 @@ impl BucketMetadata {
         self.lock_enabled || (self.versioning_config.as_ref().is_some_and(|v| v.enabled()))
     }
 
+    pub fn table_bucket_enabled(&self) -> bool {
+        !self.table_bucket_config_json.is_empty()
+    }
+
     /// Decode from msgp bytes. Field order follows MinIO BucketMetadata for compatibility.
     pub fn decode_from<R: Read>(&mut self, rd: &mut R) -> Result<()> {
         let mut fields = rmp::decode::read_map_len(rd)?;
@@ -447,6 +457,7 @@ impl BucketMetadata {
                     self.public_access_block_config_xml = read_msgp_bin(rd)?
                 }
                 "BucketAclConfigJSON" | "BucketAclConfigJson" => self.bucket_acl_config_json = read_msgp_bin(rd)?,
+                "TableBucketConfigJSON" | "TableBucketConfigJson" => self.table_bucket_config_json = read_msgp_bin(rd)?,
                 "CorsConfigUpdatedAt" => self.cors_config_updated_at = read_msgp_time_value(rd)?,
                 "LoggingConfigUpdatedAt" => self.logging_config_updated_at = read_msgp_time_value(rd)?,
                 "WebsiteConfigUpdatedAt" => self.website_config_updated_at = read_msgp_time_value(rd)?,
@@ -454,6 +465,7 @@ impl BucketMetadata {
                 "RequestPaymentConfigUpdatedAt" => self.request_payment_config_updated_at = read_msgp_time_value(rd)?,
                 "PublicAccessBlockConfigUpdatedAt" => self.public_access_block_config_updated_at = read_msgp_time_value(rd)?,
                 "BucketAclConfigUpdatedAt" => self.bucket_acl_config_updated_at = read_msgp_time_value(rd)?,
+                "TableBucketConfigUpdatedAt" => self.table_bucket_config_updated_at = read_msgp_time_value(rd)?,
                 other => {
                     tracing::debug!(field = %other, "BucketMetadata decode_from: skipping unknown field");
                     skip_msgp_value(rd)?;
@@ -466,8 +478,8 @@ impl BucketMetadata {
 
     /// Encode to msgp bytes. Field order follows MinIO BucketMetadata for compatibility.
     pub fn encode_to<W: Write>(&self, wr: &mut W) -> Result<()> {
-        // Map size: MinIO fields (25) + RustFS extensions (14)
-        let map_len: u32 = 39;
+        // Map size: MinIO fields (25) + RustFS extensions (16)
+        let map_len: u32 = 41;
         rmp::encode::write_map_len(wr, map_len)?;
 
         // MinIO field order (same as Go struct)
@@ -523,6 +535,7 @@ impl BucketMetadata {
         write_bin_field(wr, "RequestPaymentConfigXML", &self.request_payment_config_xml)?;
         write_bin_field(wr, "PublicAccessBlockConfigXML", &self.public_access_block_config_xml)?;
         write_bin_field(wr, "BucketAclConfigJSON", &self.bucket_acl_config_json)?;
+        write_bin_field(wr, "TableBucketConfigJSON", &self.table_bucket_config_json)?;
         rmp::encode::write_str(wr, "CorsConfigUpdatedAt")?;
         write_msgp_time(wr, self.cors_config_updated_at)?;
         rmp::encode::write_str(wr, "LoggingConfigUpdatedAt")?;
@@ -537,6 +550,8 @@ impl BucketMetadata {
         write_msgp_time(wr, self.public_access_block_config_updated_at)?;
         rmp::encode::write_str(wr, "BucketAclConfigUpdatedAt")?;
         write_msgp_time(wr, self.bucket_acl_config_updated_at)?;
+        rmp::encode::write_str(wr, "TableBucketConfigUpdatedAt")?;
+        write_msgp_time(wr, self.table_bucket_config_updated_at)?;
 
         Ok(())
     }
@@ -632,6 +647,9 @@ impl BucketMetadata {
         if self.bucket_acl_config_updated_at == OffsetDateTime::UNIX_EPOCH {
             self.bucket_acl_config_updated_at = self.created
         }
+        if self.table_bucket_config_updated_at == OffsetDateTime::UNIX_EPOCH {
+            self.table_bucket_config_updated_at = self.created
+        }
     }
 
     pub fn update_config(&mut self, config_file: &str, data: Vec<u8>) -> Result<OffsetDateTime> {
@@ -709,6 +727,10 @@ impl BucketMetadata {
                 self.bucket_acl_config_json = data;
                 self.bucket_acl_config_updated_at = updated;
             }
+            BUCKET_TABLE_CONFIG => {
+                self.table_bucket_config_json = data;
+                self.table_bucket_config_updated_at = updated;
+            }
             _ => return Err(Error::other(format!("config file not found : {config_file}"))),
         }
 
@@ -1028,6 +1050,10 @@ mod test {
         bm.bucket_acl_config_json = bucket_acl.as_bytes().to_vec();
         bm.bucket_acl_config_updated_at = OffsetDateTime::now_utc();
 
+        let table_bucket_marker = r#"{"enabled":true}"#;
+        bm.table_bucket_config_json = table_bucket_marker.as_bytes().to_vec();
+        bm.table_bucket_config_updated_at = OffsetDateTime::now_utc();
+
         // Test serialization
         let buf = bm.marshal_msg().unwrap();
         assert!(!buf.is_empty(), "Serialized buffer should not be empty");
@@ -1049,6 +1075,7 @@ mod test {
         assert_eq!(bm.quota_config_json, deserialized_bm.quota_config_json);
         assert_eq!(bm.public_access_block_config_xml, deserialized_bm.public_access_block_config_xml);
         assert_eq!(bm.bucket_acl_config_json, deserialized_bm.bucket_acl_config_json);
+        assert_eq!(bm.table_bucket_config_json, deserialized_bm.table_bucket_config_json);
         assert_eq!(bm.object_lock_config_xml, deserialized_bm.object_lock_config_xml);
         assert_eq!(bm.notification_config_xml, deserialized_bm.notification_config_xml);
         assert_eq!(bm.replication_config_xml, deserialized_bm.replication_config_xml);
@@ -1100,6 +1127,11 @@ mod test {
             bm.bucket_targets_config_meta_updated_at.unix_timestamp(),
             deserialized_bm.bucket_targets_config_meta_updated_at.unix_timestamp()
         );
+        assert_eq!(
+            bm.table_bucket_config_updated_at.unix_timestamp(),
+            deserialized_bm.table_bucket_config_updated_at.unix_timestamp()
+        );
+        assert!(deserialized_bm.table_bucket_enabled());
 
         // Test that the serialized data contains expected content
         let buf_str = String::from_utf8_lossy(&buf);
@@ -1116,6 +1148,20 @@ mod test {
         println!("   - Serialized buffer size: {} bytes", buf.len());
     }
 
+    #[test]
+    fn table_bucket_marker_tracks_config_presence() {
+        let mut bm = BucketMetadata::new("table-bucket");
+        assert!(!bm.table_bucket_enabled());
+
+        bm.update_config(BUCKET_TABLE_CONFIG, br#"{"enabled":true}"#.to_vec())
+            .unwrap();
+        assert!(bm.table_bucket_enabled());
+        assert!(!bm.table_bucket_config_json.is_empty());
+
+        bm.update_config(BUCKET_TABLE_CONFIG, Vec::new()).unwrap();
+        assert!(!bm.table_bucket_enabled());
+    }
+
     /// After policy deletion (policy_config_json cleared), parse_policy_config sets policy_config to None.
     #[test]
     fn test_parse_policy_config_clears_cache_when_json_empty() {

crates/ecstore/src/bucket/replication/replication_pool.rs

@@ -1079,7 +1079,7 @@ pub async fn schedule_replication<S: StorageAPI>(oi: ObjectInfo, o: Arc<S>, dsc:
         target_statuses: tgt_statuses,
         target_purge_statuses: purge_statuses,
         replication_timestamp: tm,
-        user_tags: oi.user_tags,
+        user_tags: (*oi.user_tags).clone(),
         checksum: None,
         retry_count: 0,
         event_type: "".to_string(),

crates/ecstore/src/bucket/replication/replication_resyncer.rs

@@ -934,7 +934,7 @@ fn heal_should_use_check_replicate_delete(oi: &ObjectInfo) -> bool {
 
 pub async fn get_heal_replicate_object_info(oi: &ObjectInfo, rcfg: &ReplicationConfig) -> ReplicateObjectInfo {
     let mut oi = oi.clone();
-    let mut user_defined = oi.user_defined.clone();
+    let mut user_defined = (*oi.user_defined).clone();
 
     if let Some(rc) = rcfg.config.as_ref()
         && !rc.role.is_empty()
@@ -982,7 +982,7 @@ pub async fn get_heal_replicate_object_info(oi: &ObjectInfo, rcfg: &ReplicationC
             &oi.name,
             MustReplicateOptions::new(
                 &user_defined,
-                oi.user_tags.clone(),
+                (*oi.user_tags).clone(),
                 ReplicationStatusType::Empty,
                 ReplicationType::Heal,
                 ObjectOptions::default(),
@@ -1020,7 +1020,7 @@ pub async fn get_heal_replicate_object_info(oi: &ObjectInfo, rcfg: &ReplicationC
         target_purge_statuses,
         replication_timestamp: None,
         ssec: false, // TODO: add ssec support
-        user_tags: oi.user_tags.clone(),
+        user_tags: (*oi.user_tags).clone(),
         checksum: oi.checksum.clone(),
         retry_count: 0,
     }
@@ -1158,7 +1158,7 @@ impl ReplicationConfig {
             return self.resync_internal(oi, dsc, status);
         }
 
-        let mut user_defined = oi.user_defined.clone();
+        let mut user_defined = (*oi.user_defined).clone();
         user_defined.remove(AMZ_BUCKET_REPLICATION_STATUS);
 
         let dsc = must_replicate(
@@ -1166,7 +1166,7 @@ impl ReplicationConfig {
             &oi.name,
             MustReplicateOptions::new(
                 &user_defined,
-                oi.user_tags.clone(),
+                (*oi.user_tags).clone(),
                 ReplicationStatusType::Empty,
                 ReplicationType::ExistingObject,
                 ObjectOptions::default(),
@@ -1296,7 +1296,7 @@ impl MustReplicateOptions {
     }
 
     pub fn from_object_info(oi: &ObjectInfo, op_type: ReplicationType, opts: ObjectOptions) -> Self {
-        Self::new(&oi.user_defined, oi.user_tags.clone(), oi.replication_status.clone(), op_type, opts)
+        Self::new(&oi.user_defined, (*oi.user_tags).clone(), oi.replication_status.clone(), op_type, opts)
     }
 
     pub fn replication_status(&self) -> ReplicationStatusType {
@@ -1411,7 +1411,7 @@ fn delete_replication_object_opts(dobj: &ObjectToDelete, oi: &ObjectInfo) -> Obj
     ObjectOpts {
         name: dobj.object_name.clone(),
         ssec: is_ssec_encrypted(&oi.user_defined),
-        user_tags: oi.user_tags.clone(),
+        user_tags: (*oi.user_tags).clone(),
         delete_marker: oi.delete_marker,
         version_id: dobj.version_id,
         op_type: ReplicationType::Delete,
@@ -2995,7 +2995,7 @@ impl ReplicateObjectInfoExt for ReplicateObjectInfo {
             mod_time: self.mod_time,
             version_id: self.version_id,
             size: self.size,
-            user_tags: self.user_tags.clone(),
+            user_tags: Arc::new(self.user_tags.clone()),
             actual_size: self.actual_size,
             replication_status_internal: self.replication_status_internal.clone(),
             replication_status: self.replication_status.clone(),
@@ -3221,7 +3221,7 @@ fn put_replication_opts(sc: &str, object_info: &ObjectInfo) -> Result<(PutObject
     }
 
     // Use case-insensitive lookup for headers
-    let lk_map = object_info.user_defined.clone();
+    let lk_map = &*object_info.user_defined;
 
     if let Some(lang) = lk_map.lookup(CONTENT_LANGUAGE) {
         put_op.content_language = lang.to_string();
@@ -3500,7 +3500,7 @@ fn get_replication_action(oi1: &ObjectInfo, oi2: &HeadObjectOutput, op_type: Rep
 
     // compare metadata on both maps to see if meta is identical
     let mut compare_meta1 = HashMap::new();
-    for (k, v) in &oi1.user_defined {
+    for (k, v) in oi1.user_defined.iter() {
         let mut found = false;
         for prefix in &compare_keys {
             if strings_has_prefix_fold(k, prefix) {

crates/ecstore/src/config/com.rs

@@ -12,7 +12,7 @@
 // See the License for the specific language governing permissions and
 // limitations under the License.
 
-use crate::config::{Config, GLOBAL_STORAGE_CLASS, KVS, audit, notify, oidc, storageclass};
+use crate::config::{Config, KVS, audit, notify, oidc, set_global_storage_class, storageclass};
 use crate::disk::{MIGRATING_META_BUCKET, RUSTFS_META_BUCKET};
 use crate::error::{Error, Result};
 use crate::global::is_first_cluster_node_local;
@@ -1164,11 +1164,8 @@ async fn apply_dynamic_config_for_sub_sys<S: StorageAPI>(cfg: &mut Config, api:
         for (i, count) in set_drive_counts.iter().enumerate() {
             match storageclass::lookup_config(&kvs, *count) {
                 Ok(res) => {
-                    if i == 0
-                        && GLOBAL_STORAGE_CLASS.get().is_none()
-                        && let Err(r) = GLOBAL_STORAGE_CLASS.set(res)
-                    {
-                        error!("GLOBAL_STORAGE_CLASS.set failed {:?}", r);
+                    if i == 0 {
+                        set_global_storage_class(res);
                     }
                 }
                 Err(err) => {
@@ -1358,7 +1355,7 @@ mod tests {
                 size: self.data.len() as i64,
                 actual_size: self.data.len() as i64,
                 is_dir: false,
-                user_defined: HashMap::new(),
+                user_defined: Arc::new(HashMap::new()),
                 parity_blocks: 0,
                 data_blocks: 0,
                 version_id: None,
@@ -1366,8 +1363,8 @@ mod tests {
                 transitioned_object: Default::default(),
                 restore_ongoing: false,
                 restore_expires: None,
-                user_tags: String::new(),
-                parts: Vec::new(),
+                user_tags: Arc::new(String::new()),
+                parts: Arc::new(Vec::new()),
                 is_latest: true,
                 content_type: Some("application/json".to_string()),
                 content_encoding: None,
@@ -2638,4 +2635,87 @@ mod tests {
         assert_eq!(object_info.bucket, crate::disk::RUSTFS_META_BUCKET);
         assert_eq!(object_info.name, "config/test.json");
     }
+
+    #[test]
+    fn test_encode_decode_roundtrip_preserves_storage_class() {
+        use crate::config::STORAGE_CLASS_SUB_SYS;
+
+        // Create a config with custom storage class values
+        let mut cfg = Config::new();
+        let kvs = storage_class_kvs_mut(&mut cfg);
+        kvs.insert("standard".to_string(), "EC:4".to_string());
+        kvs.insert("rrs".to_string(), "EC:2".to_string());
+        kvs.insert("optimize".to_string(), "availability".to_string());
+
+        // Encode to external format (what save_server_config produces)
+        let encoded = encode_server_config_blob(&cfg, None).expect("encode should succeed");
+
+        // Verify the encoded data uses "storageclass" (no underscore)
+        let v: serde_json::Value = serde_json::from_slice(&encoded).expect("should be valid json");
+        assert!(v.get("storageclass").is_some(), "encoded should have 'storageclass' key");
+        assert!(v.get("storage_class").is_none(), "encoded should NOT have 'storage_class' key");
+
+        // Decode back (what load_server_config_from_store produces)
+        let decoded = decode_server_config_blob(&encoded).expect("decode should succeed");
+
+        // Verify the decoded config has "storage_class" (with underscore) subsystem
+        let kvs = decoded
+            .get_value(STORAGE_CLASS_SUB_SYS, crate::config::DEFAULT_DELIMITER)
+            .expect("decoded config should have storage_class subsystem");
+        assert_eq!(kvs.get("standard"), "EC:4", "standard should be EC:4");
+        assert_eq!(kvs.get("rrs"), "EC:2", "rrs should be EC:2");
+        assert_eq!(kvs.get("optimize"), "availability", "optimize should be availability");
+    }
+
+    #[test]
+    fn test_set_then_load_preserves_storage_class_values() {
+        use crate::config::STORAGE_CLASS_SUB_SYS;
+
+        // Step 1: Start with a fresh config (simulates initial server state)
+        let mut cfg = Config::new();
+
+        // Step 2: Apply set directives (simulates "mc admin config set storage_class standard=EC:4 rrs=EC:2")
+        let kvs = cfg
+            .0
+            .entry(STORAGE_CLASS_SUB_SYS.to_string())
+            .or_default()
+            .entry(DEFAULT_DELIMITER.to_string())
+            .or_default();
+        kvs.insert("standard".to_string(), "EC:4".to_string());
+        kvs.insert("rrs".to_string(), "EC:2".to_string());
+        cfg.set_defaults();
+
+        // Step 3: Save (encode to external format)
+        let encoded = encode_server_config_blob(&cfg, None).expect("encode should succeed");
+
+        // Step 4: Load (decode from external format)
+        let loaded = decode_server_config_blob(&encoded).expect("decode should succeed");
+
+        // Step 5: Verify the values are preserved
+        let loaded_kvs = loaded
+            .get_value(STORAGE_CLASS_SUB_SYS, DEFAULT_DELIMITER)
+            .expect("should have storage_class");
+        assert_eq!(loaded_kvs.get("standard"), "EC:4");
+        assert_eq!(loaded_kvs.get("rrs"), "EC:2");
+
+        // Step 6: Verify the subsystem is accessible by name (what render_selected_config does)
+        let targets = loaded
+            .0
+            .get(STORAGE_CLASS_SUB_SYS)
+            .expect("storage_class subsystem should exist");
+        let target_kvs = targets.get(DEFAULT_DELIMITER).expect("default target should exist");
+        assert_eq!(target_kvs.get("standard"), "EC:4");
+        assert_eq!(target_kvs.get("rrs"), "EC:2");
+    }
+
+    #[test]
+    fn test_config_unmarshal_fails_on_external_format() {
+        // This verifies that the external "storageclass" format cannot be
+        // mistakenly parsed by Config::unmarshal (which expects internal format).
+        // If unmarshal succeeds, the config would have "storageclass" instead of
+        // "storage_class", causing render_selected_config to fail.
+        let external_json = r#"{"version":"33","storageclass":{"standard":"EC:4","rrs":"EC:2"}}"#;
+        let result = Config::unmarshal(external_json.as_bytes());
+        assert!(result.is_err(), "Config::unmarshal should fail on external format, but got: {:?}", result);
+    }
 }

crates/ecstore/src/config/mod.rs

@@ -18,6 +18,7 @@ pub mod com;
 pub mod heal;
 mod notify;
 mod oidc;
+mod scanner;
 pub mod storageclass;
 
 use crate::error::Result;
@@ -37,11 +38,12 @@ use rustfs_config::oidc::IDENTITY_OPENID_SUB_SYS;
 use serde::{Deserialize, Serialize};
 use std::collections::HashMap;
 use std::sync::LazyLock;
-use std::sync::{Arc, OnceLock};
+use std::sync::{Arc, OnceLock, RwLock};
 
-pub static GLOBAL_STORAGE_CLASS: LazyLock<OnceLock<storageclass::Config>> = LazyLock::new(OnceLock::new);
+pub static GLOBAL_STORAGE_CLASS: LazyLock<RwLock<storageclass::Config>> =
+    LazyLock::new(|| RwLock::new(storageclass::Config::default()));
 pub static DEFAULT_KVS: LazyLock<OnceLock<HashMap<String, KVS>>> = LazyLock::new(OnceLock::new);
-pub static GLOBAL_SERVER_CONFIG: LazyLock<OnceLock<Config>> = LazyLock::new(OnceLock::new);
+pub static GLOBAL_SERVER_CONFIG: LazyLock<RwLock<Option<Config>>> = LazyLock::new(|| RwLock::new(None));
 pub static GLOBAL_CONFIG_SYS: LazyLock<ConfigSys> = LazyLock::new(ConfigSys::new);
 
 pub static RUSTFS_CONFIG_PREFIX: &str = "config";
@@ -63,14 +65,30 @@ impl ConfigSys {
 
         lookup_configs(&mut cfg, api).await;
 
-        let _ = GLOBAL_SERVER_CONFIG.set(cfg);
+        set_global_server_config(cfg);
 
         Ok(())
     }
 }
 
 pub fn get_global_server_config() -> Option<Config> {
-    GLOBAL_SERVER_CONFIG.get().cloned()
+    GLOBAL_SERVER_CONFIG.read().ok().and_then(|guard| (*guard).clone())
+}
+
+pub fn set_global_server_config(cfg: Config) {
+    if let Ok(mut guard) = GLOBAL_SERVER_CONFIG.write() {
+        *guard = Some(cfg);
+    }
+}
+
+pub fn get_global_storage_class() -> Option<storageclass::Config> {
+    GLOBAL_STORAGE_CLASS.read().ok().map(|guard| (*guard).clone())
+}
+
+pub fn set_global_storage_class(cfg: storageclass::Config) {
+    if let Ok(mut guard) = GLOBAL_STORAGE_CLASS.write() {
+        *guard = cfg;
+    }
 }
 
 pub async fn init_global_config_sys(api: Arc<ECStore>) -> Result<()> {
@@ -237,6 +255,7 @@ pub fn init() {
     let mut kvs = HashMap::new();
     // Load storageclass default configuration
     kvs.insert(STORAGE_CLASS_SUB_SYS.to_owned(), storageclass::DEFAULT_KVS.clone());
+    kvs.insert(rustfs_config::SCANNER_SUB_SYS.to_owned(), scanner::DEFAULT_KVS.clone());
     // New: Loading default configurations for notify_webhook and notify_mqtt
     // Referring subsystem names through constants to improve the readability and maintainability of the code
     kvs.insert(NOTIFY_WEBHOOK_SUB_SYS.to_owned(), notify::DEFAULT_NOTIFY_WEBHOOK_KVS.clone());
@@ -262,3 +281,38 @@ pub fn init() {
     // Register all default configurations
     register_default_kvs(kvs)
 }
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+    use rustfs_config::{DEFAULT_DELIMITER, DEFAULT_SCANNER_SPEED, SCANNER_CYCLE_MAX_OBJECTS, SCANNER_SPEED, SCANNER_SUB_SYS};
+
+    #[test]
+    fn global_server_config_set_and_get_roundtrip() {
+        init();
+        let mut cfg = Config::new();
+        let mut kvs = KVS::new();
+        kvs.insert("standard".to_string(), "EC:4".to_string());
+        cfg.0
+            .insert(STORAGE_CLASS_SUB_SYS.to_string(), HashMap::from([("_".to_string(), kvs)]));
+
+        set_global_server_config(cfg.clone());
+        let loaded = get_global_server_config().expect("global config should be set");
+        let sc_kvs = loaded
+            .get_value(STORAGE_CLASS_SUB_SYS, "_")
+            .expect("storage_class should exist");
+        assert_eq!(sc_kvs.get("standard"), "EC:4");
+    }
+
+    #[test]
+    fn scanner_defaults_are_registered_for_admin_config() {
+        init();
+        let cfg = Config::new();
+        let scanner_kvs = cfg
+            .get_value(SCANNER_SUB_SYS, DEFAULT_DELIMITER)
+            .expect("scanner defaults should exist");
+
+        assert_eq!(scanner_kvs.get(SCANNER_SPEED), DEFAULT_SCANNER_SPEED);
+        assert_eq!(scanner_kvs.get(SCANNER_CYCLE_MAX_OBJECTS), "0");
+    }
+}

crates/ecstore/src/config/scanner.rs

@@ -0,0 +1,106 @@
+// Copyright 2024 RustFS Team
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+use crate::config::{KV, KVS};
+use rustfs_config::{
+    DEFAULT_SCANNER_BITROT_CYCLE_SECS, DEFAULT_SCANNER_CACHE_SAVE_TIMEOUT_SECS, DEFAULT_SCANNER_CYCLE_MAX_DIRECTORIES,
+    DEFAULT_SCANNER_CYCLE_MAX_DURATION_SECS, DEFAULT_SCANNER_CYCLE_MAX_OBJECTS, DEFAULT_SCANNER_IDLE_MODE,
+    DEFAULT_SCANNER_MAX_CONCURRENT_DISK_SCANS, DEFAULT_SCANNER_MAX_CONCURRENT_SET_SCANS, DEFAULT_SCANNER_SPEED,
+    DEFAULT_SCANNER_YIELD_EVERY_N_OBJECTS, SCANNER_ALERT_EXCESS_FOLDERS, SCANNER_ALERT_EXCESS_VERSION_SIZE,
+    SCANNER_ALERT_EXCESS_VERSIONS, SCANNER_BITROT_CYCLE, SCANNER_CACHE_SAVE_TIMEOUT, SCANNER_CYCLE,
+    SCANNER_CYCLE_MAX_DIRECTORIES, SCANNER_CYCLE_MAX_DURATION, SCANNER_CYCLE_MAX_OBJECTS, SCANNER_IDLE_MODE,
+    SCANNER_MAX_CONCURRENT_DISK_SCANS, SCANNER_MAX_CONCURRENT_SET_SCANS, SCANNER_SPEED, SCANNER_START_DELAY,
+    SCANNER_YIELD_EVERY_N_OBJECTS,
+};
+use std::sync::LazyLock;
+
+pub static DEFAULT_KVS: LazyLock<KVS> = LazyLock::new(|| {
+    KVS(vec![
+        KV {
+            key: SCANNER_SPEED.to_owned(),
+            value: DEFAULT_SCANNER_SPEED.to_owned(),
+            hidden_if_empty: false,
+        },
+        KV {
+            key: SCANNER_CYCLE.to_owned(),
+            value: String::new(),
+            hidden_if_empty: true,
+        },
+        KV {
+            key: SCANNER_START_DELAY.to_owned(),
+            value: String::new(),
+            hidden_if_empty: true,
+        },
+        KV {
+            key: SCANNER_CYCLE_MAX_DURATION.to_owned(),
+            value: DEFAULT_SCANNER_CYCLE_MAX_DURATION_SECS.to_string(),
+            hidden_if_empty: false,
+        },
+        KV {
+            key: SCANNER_CYCLE_MAX_OBJECTS.to_owned(),
+            value: DEFAULT_SCANNER_CYCLE_MAX_OBJECTS.to_string(),
+            hidden_if_empty: false,
+        },
+        KV {
+            key: SCANNER_CYCLE_MAX_DIRECTORIES.to_owned(),
+            value: DEFAULT_SCANNER_CYCLE_MAX_DIRECTORIES.to_string(),
+            hidden_if_empty: false,
+        },
+        KV {
+            key: SCANNER_BITROT_CYCLE.to_owned(),
+            value: DEFAULT_SCANNER_BITROT_CYCLE_SECS.to_string(),
+            hidden_if_empty: false,
+        },
+        KV {
+            key: SCANNER_IDLE_MODE.to_owned(),
+            value: DEFAULT_SCANNER_IDLE_MODE.to_string(),
+            hidden_if_empty: false,
+        },
+        KV {
+            key: SCANNER_CACHE_SAVE_TIMEOUT.to_owned(),
+            value: DEFAULT_SCANNER_CACHE_SAVE_TIMEOUT_SECS.to_string(),
+            hidden_if_empty: false,
+        },
+        KV {
+            key: SCANNER_MAX_CONCURRENT_SET_SCANS.to_owned(),
+            value: DEFAULT_SCANNER_MAX_CONCURRENT_SET_SCANS.to_string(),
+            hidden_if_empty: false,
+        },
+        KV {
+            key: SCANNER_MAX_CONCURRENT_DISK_SCANS.to_owned(),
+            value: DEFAULT_SCANNER_MAX_CONCURRENT_DISK_SCANS.to_string(),
+            hidden_if_empty: false,
+        },
+        KV {
+            key: SCANNER_YIELD_EVERY_N_OBJECTS.to_owned(),
+            value: DEFAULT_SCANNER_YIELD_EVERY_N_OBJECTS.to_string(),
+            hidden_if_empty: false,
+        },
+        KV {
+            key: SCANNER_ALERT_EXCESS_VERSIONS.to_owned(),
+            value: rustfs_config::DEFAULT_SCANNER_ALERT_EXCESS_VERSIONS.to_string(),
+            hidden_if_empty: false,
+        },
+        KV {
+            key: SCANNER_ALERT_EXCESS_VERSION_SIZE.to_owned(),
+            value: rustfs_config::DEFAULT_SCANNER_ALERT_EXCESS_VERSION_SIZE.to_string(),
+            hidden_if_empty: false,
+        },
+        KV {
+            key: SCANNER_ALERT_EXCESS_FOLDERS.to_owned(),
+            value: rustfs_config::DEFAULT_SCANNER_ALERT_EXCESS_FOLDERS.to_string(),
+            hidden_if_empty: false,
+        },
+    ])
+});

crates/ecstore/src/config/storageclass.rs

@@ -101,13 +101,13 @@ pub static DEFAULT_KVS: LazyLock<KVS> = LazyLock::new(|| {
 });
 
 // StorageClass - holds storage class information
-#[derive(Serialize, Deserialize, Debug, Default)]
+#[derive(Serialize, Deserialize, Debug, Default, Clone)]
 pub struct StorageClass {
     parity: usize,
 }
 
 // Config storage class configuration
-#[derive(Serialize, Deserialize, Debug, Default)]
+#[derive(Serialize, Deserialize, Debug, Default, Clone)]
 pub struct Config {
     standard: StorageClass,
     rrs: StorageClass,
@@ -205,7 +205,7 @@ pub fn lookup_config(kvs: &KVS, set_drive_count: usize) -> Result<Config> {
             if let Ok(ssc_str) = env::var(RRS_ENV) {
                 ssc_str
             } else {
-                kvs.get(RRS)
+                kvs.get(CLASS_RRS)
             }
         };
 
@@ -336,3 +336,36 @@ pub fn validate_parity_inner(ss_parity: usize, rrs_parity: usize, set_drive_coun
     }
     Ok(())
 }
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    #[test]
+    fn lookup_config_reads_rrs_from_class_rrs_key() {
+        // Regression: kvs.get(RRS) used RRS="REDUCED_REDUNDANCY" instead of
+        // CLASS_RRS="rrs", so admin-written RRS values were never read back.
+        let mut kvs = KVS::new();
+        kvs.insert(CLASS_STANDARD.to_string(), "EC:4".to_string());
+        kvs.insert(CLASS_RRS.to_string(), "EC:2".to_string());
+
+        let cfg = lookup_config(&kvs, 8).expect("lookup should succeed");
+        assert_eq!(cfg.standard.parity, 4, "standard parity should be 4");
+        assert_eq!(cfg.rrs.parity, 2, "rrs parity should be 2");
+    }
+
+    #[test]
+    fn lookup_config_ignores_redundancy_key_name() {
+        // Ensure the old key name "REDUCED_REDUNDANCY" is NOT read.
+        let mut kvs = KVS::new();
+        kvs.insert(CLASS_STANDARD.to_string(), "EC:4".to_string());
+        kvs.insert(RRS.to_string(), "EC:2".to_string());
+
+        let cfg = lookup_config(&kvs, 8).expect("lookup should succeed");
+        assert_eq!(cfg.standard.parity, 4);
+        assert_eq!(
+            cfg.rrs.parity, DEFAULT_RRS_PARITY,
+            "rrs should fall back to default when CLASS_RRS key is absent"
+        );
+    }
+}

crates/ecstore/src/data_movement.rs

@@ -94,7 +94,7 @@ fn data_movement_new_multipart_opts(object_info: &ObjectInfo, src_pool_idx: usiz
     ObjectOptions {
         versioned: object_info.version_id.is_some(),
         version_id: object_info.version_id.as_ref().map(|v| v.to_string()),
-        user_defined: object_info.user_defined.clone(),
+        user_defined: (*object_info.user_defined).clone(),
         preserve_etag: object_info.etag.clone(),
         src_pool_idx,
         data_movement: true,
@@ -120,7 +120,7 @@ fn data_movement_put_object_opts(object_info: &ObjectInfo, src_pool_idx: usize)
         data_movement: true,
         version_id: object_info.version_id.as_ref().map(|v| v.to_string()),
         mod_time: object_info.mod_time,
-        user_defined: object_info.user_defined.clone(),
+        user_defined: (*object_info.user_defined).clone(),
         preserve_etag: object_info.etag.clone(),
         ..Default::default()
     }
@@ -341,7 +341,7 @@ mod tests {
         let object_info = ObjectInfo {
             version_id: Some(version_id),
             etag: Some("etag-value".to_string()),
-            user_defined: std::collections::HashMap::from([("x-amz-meta-key".to_string(), "value".to_string())]),
+            user_defined: Arc::new(std::collections::HashMap::from([("x-amz-meta-key".to_string(), "value".to_string())])),
             ..Default::default()
         };
 
@@ -382,7 +382,7 @@ mod tests {
             version_id: Some(version_id),
             mod_time: Some(OffsetDateTime::UNIX_EPOCH),
             etag: Some("etag-value".to_string()),
-            user_defined: std::collections::HashMap::from([("x-amz-meta-key".to_string(), "value".to_string())]),
+            user_defined: Arc::new(std::collections::HashMap::from([("x-amz-meta-key".to_string(), "value".to_string())])),
             ..Default::default()
         };
 

crates/ecstore/src/disk/disk_store.rs

@@ -1140,7 +1140,8 @@ impl DiskAPI for LocalDiskWrapper {
             "walk_dir",
             || async { self.disk.walk_dir(opts, wr).await },
             get_drive_walkdir_timeout(),
-            self.scanner_timeout_health_action(),
+            // Listing/scanner backpressure should fail only the current walk, not poison drive health.
+            TimeoutHealthAction::IgnoreFailure,
         )
         .await
     }
@@ -1241,7 +1242,8 @@ impl DiskAPI for LocalDiskWrapper {
         dst_volume: &str,
         dst_path: &str,
     ) -> Result<RenameDataResp> {
-        self.track_disk_health(
+        self.track_disk_health_with_op(
+            "rename_data",
             || async { self.disk.rename_data(src_volume, src_path, fi, dst_volume, dst_path).await },
             get_max_timeout_duration(),
         )
@@ -1603,7 +1605,7 @@ mod tests {
     }
 
     #[tokio::test]
-    async fn walk_dir_writer_backpressure_timeout_marks_drive_failure_by_default() {
+    async fn walk_dir_writer_backpressure_timeout_does_not_mark_drive_failure_by_default() {
         temp_env::async_with_vars([(rustfs_config::ENV_DRIVE_WALKDIR_TIMEOUT_SECS, Some("1"))], async {
             let dir = tempfile::tempdir().expect("temp dir should be created");
             let endpoint =
@@ -1639,7 +1641,60 @@ mod tests {
                 .await;
 
             assert_eq!(result.expect_err("walk_dir should time out"), DiskError::Timeout);
-            assert_eq!(wrapper.runtime_state(), RuntimeDriveHealthState::Suspect);
+            assert_eq!(wrapper.runtime_state(), RuntimeDriveHealthState::Online);
+            assert!(!wrapper.health.is_faulty());
+        })
+        .await;
+    }
+
+    #[tokio::test]
+    async fn walk_dir_timeout_does_not_break_followup_stat_volume() {
+        temp_env::async_with_vars([(rustfs_config::ENV_DRIVE_WALKDIR_TIMEOUT_SECS, Some("1"))], async {
+            let dir = tempfile::tempdir().expect("temp dir should be created");
+            let endpoint =
+                Endpoint::try_from(dir.path().to_str().expect("temp dir should be valid UTF-8")).expect("endpoint should parse");
+            let disk = Arc::new(LocalDisk::new(&endpoint, false).await.expect("local disk should be created"));
+            let wrapper = LocalDiskWrapper::new(disk, false);
+            let bucket = "test-bucket";
+            let object = "test-object";
+
+            wrapper.make_volume(bucket).await.expect("bucket should be created");
+
+            let mut file_info = FileInfo::new(&format!("{bucket}/{object}"), 1, 0);
+            file_info.volume = bucket.to_string();
+            file_info.name = object.to_string();
+            file_info.mod_time = Some(::time::OffsetDateTime::now_utc());
+            file_info.erasure.index = 1;
+
+            wrapper
+                .write_metadata("", bucket, object, file_info)
+                .await
+                .expect("object metadata should be written");
+
+            let mut writer = PendingWriter;
+            let walk_err = wrapper
+                .walk_dir(
+                    WalkDirOptions {
+                        bucket: bucket.to_string(),
+                        recursive: true,
+                        ..Default::default()
+                    },
+                    &mut writer,
+                )
+                .await
+                .expect_err("walk_dir should time out");
+
+            assert_eq!(walk_err, DiskError::Timeout);
+            assert_eq!(wrapper.runtime_state(), RuntimeDriveHealthState::Online);
+            assert!(!wrapper.health.is_faulty());
+
+            let info = wrapper
+                .stat_volume(bucket)
+                .await
+                .expect("follow-up bucket stat should still succeed after walk timeout");
+            assert_eq!(info.name, bucket);
+            assert_eq!(wrapper.runtime_state(), RuntimeDriveHealthState::Online);
+            assert!(!wrapper.health.is_faulty());
         })
         .await;
     }

crates/ecstore/src/disk/endpoint.rs

@@ -19,6 +19,23 @@ use std::{fmt::Display, path::Path};
 use tracing::debug;
 use url::{ParseError, Url};
 
+#[cfg(windows)]
+pub(crate) fn windows_fallback_local_path(
+    path: &str,
+    canonicalize_error: &std::io::Error,
+    context: &'static str,
+) -> std::io::Result<std::path::PathBuf> {
+    let absolute = Path::new(path).absolutize()?.to_path_buf();
+    tracing::warn!(
+        path = %path,
+        canonicalize_error = %canonicalize_error,
+        resolved = ?absolute,
+        context = context,
+        "using windows fallback path resolution for local endpoint"
+    );
+    Ok(absolute)
+}
+
 /// enum for endpoint type.
 #[derive(PartialEq, Eq, Debug)]
 pub enum EndpointType {

crates/ecstore/src/disk/fs.rs

@@ -161,33 +161,39 @@ pub async fn make_dir_all(path: impl AsRef<Path>) -> io::Result<()> {
     fs::create_dir_all(path.as_ref()).await
 }
 
+fn is_dir_error(e: &io::Error) -> bool {
+    e.raw_os_error() == Some(libc::EISDIR)
+        || e.kind() == io::ErrorKind::IsADirectory
+        // macOS: remove_file on a directory returns EPERM
+        || (cfg!(target_os = "macos") && e.raw_os_error() == Some(libc::EPERM))
+}
+
 #[tracing::instrument(level = "debug", skip_all)]
 pub async fn remove(path: impl AsRef<Path>) -> io::Result<()> {
-    let meta = fs::metadata(path.as_ref()).await?;
-    if meta.is_dir() {
-        fs::remove_dir(path.as_ref()).await
-    } else {
-        fs::remove_file(path.as_ref()).await
+    // Try remove_file first; fall back to remove_dir if it's a directory
+    match fs::remove_file(path.as_ref()).await {
+        Ok(()) => Ok(()),
+        Err(e) if is_dir_error(&e) => fs::remove_dir(path.as_ref()).await,
+        Err(e) => Err(e),
     }
 }
 
 pub async fn remove_all(path: impl AsRef<Path>) -> io::Result<()> {
-    let meta = fs::metadata(path.as_ref()).await?;
-    if meta.is_dir() {
-        fs::remove_dir_all(path.as_ref()).await
-    } else {
-        fs::remove_file(path.as_ref()).await
+    // Try remove_file first; fall back to remove_dir_all if it's a directory
+    match fs::remove_file(path.as_ref()).await {
+        Ok(()) => Ok(()),
+        Err(e) if is_dir_error(&e) => fs::remove_dir_all(path.as_ref()).await,
+        Err(e) => Err(e),
     }
 }
 
 #[tracing::instrument(level = "debug", skip_all)]
 pub fn remove_std(path: impl AsRef<Path>) -> io::Result<()> {
-    let path = path.as_ref();
-    let meta = std::fs::metadata(path)?;
-    if meta.is_dir() {
-        std::fs::remove_dir(path)
-    } else {
-        std::fs::remove_file(path)
+    // Try remove_file first; fall back to remove_dir if it's a directory
+    match std::fs::remove_file(path.as_ref()) {
+        Ok(()) => Ok(()),
+        Err(e) if is_dir_error(&e) => std::fs::remove_dir(path.as_ref()),
+        Err(e) => Err(e),
     }
 }
 

crates/ecstore/src/disk/local.rs

@@ -26,7 +26,7 @@ use crate::disk::{
     format::FormatV3,
     fs::{O_APPEND, O_CREATE, O_RDONLY, O_TRUNC, O_WRONLY, access, lstat, lstat_std, remove, remove_all_std, remove_std, rename},
     os,
-    os::{check_path_length, is_empty_dir, is_root_disk, rename_all},
+    os::{check_path_length, is_empty_dir, is_root_disk, rename_all, rename_all_ignore_missing_source},
 };
 use crate::erasure_coding::bitrot_verify;
 use crate::global::{GLOBAL_IsErasureSD, GLOBAL_RootDiskThreshold};
@@ -356,20 +356,60 @@ impl Debug for LocalDisk {
     }
 }
 
+/// Resolve the local disk root path from an endpoint path.
+///
+/// Tries `canonicalize` first (fast path). On Windows, if canonicalization reports
+/// `NotFound` for paths that may still be valid mount roots, falls back to
+/// `absolutize` + metadata check to accept valid local directory roots that
+/// don't support full canonicalization.
+fn resolve_local_disk_root(ep_path: &str) -> Result<PathBuf> {
+    match rustfs_utils::canonicalize(ep_path) {
+        Ok(path) => Ok(path),
+        Err(err) => {
+            if err.kind() != ErrorKind::NotFound {
+                return Err(to_file_error(err).into());
+            }
+
+            #[cfg(windows)]
+            {
+                // On Windows, canonicalize can fail for ZFS volumes, junction points,
+                // subst drives, and other non-standard filesystem mounts. Try a fallback
+                // path resolution using absolutize + metadata check.
+                let absolute = match crate::disk::endpoint::windows_fallback_local_path(ep_path, &err, "local disk root") {
+                    Ok(path) => path,
+                    Err(_) => {
+                        return Err(DiskError::VolumeNotFound);
+                    }
+                };
+
+                match std::fs::metadata(&absolute) {
+                    Ok(metadata) => {
+                        if !metadata.is_dir() {
+                            return Err(DiskError::DiskNotDir);
+                        }
+                        return Ok(absolute);
+                    }
+                    Err(meta_err) => {
+                        if meta_err.kind() == ErrorKind::NotFound {
+                            return Err(DiskError::VolumeNotFound);
+                        }
+                        return Err(to_file_error(meta_err).into());
+                    }
+                }
+            }
+
+            #[cfg(not(windows))]
+            {
+                Err(DiskError::VolumeNotFound)
+            }
+        }
+    }
+}
+
 impl LocalDisk {
     pub async fn new(ep: &Endpoint, cleanup: bool) -> Result<Self> {
         debug!("Creating local disk");
-        // Use optimized path resolution instead of absolutize() for better performance
-        // Use dunce::canonicalize instead of std::fs::canonicalize to avoid UNC paths on Windows
-        let root = match rustfs_utils::canonicalize(ep.get_file_path()) {
-            Ok(path) => path,
-            Err(e) => {
-                if e.kind() == ErrorKind::NotFound {
-                    return Err(DiskError::VolumeNotFound);
-                }
-                return Err(to_file_error(e).into());
-            }
-        };
+        let root = resolve_local_disk_root(&ep.get_file_path())?;
 
         ensure_data_usage_layout(&root).await.map_err(DiskError::from)?;
 
@@ -889,19 +929,20 @@ impl LocalDisk {
         // }
 
         let err = if recursive {
-            rename_all(delete_path, trash_path, self.get_bucket_path(RUSTFS_META_TMP_DELETED_BUCKET)?)
+            rename_all_ignore_missing_source(delete_path, trash_path, self.get_bucket_path(RUSTFS_META_TMP_DELETED_BUCKET)?)
                 .await
                 .err()
         } else {
-            rename(&delete_path, &trash_path)
-                .await
-                .map_err(|e| to_file_error(e).into())
-                .err()
+            match rename(&delete_path, &trash_path).await {
+                Ok(()) => None,
+                Err(err) if err.kind() == ErrorKind::NotFound => None,
+                Err(err) => Some(to_file_error(err).into()),
+            }
         };
 
         if immediate_purge || delete_path.to_string_lossy().ends_with(SLASH_SEPARATOR) {
             let trash_path2 = self.get_object_path(RUSTFS_META_TMP_DELETED_BUCKET, Uuid::new_v4().to_string().as_str())?;
-            let _ = rename_all(
+            let _ = rename_all_ignore_missing_source(
                 encode_dir_object(delete_path.to_string_lossy().as_ref()),
                 trash_path2,
                 self.get_bucket_path(RUSTFS_META_TMP_DELETED_BUCKET)?,
@@ -2557,75 +2598,57 @@ impl DiskAPI for LocalDisk {
         check_path_length(src_file_path.to_string_lossy().to_string().as_str())?;
         check_path_length(dst_file_path.to_string_lossy().to_string().as_str())?;
 
-        // Read the previous xl.meta
-
-        let has_dst_buf = match super::fs::read_file(&dst_file_path).await {
-            Ok(res) => Some(res),
-            Err(e) => {
-                let e: DiskError = to_file_error(e).into();
-
-                if e != DiskError::FileNotFound {
-                    return Err(e);
-                }
-
-                None
-            }
-        };
-
-        let mut xlmeta = FileMeta::new();
-
-        if let Some(dst_buf) = has_dst_buf.as_ref()
-            && FileMeta::is_xl2_v1_format(dst_buf)
-            && let Ok(nmeta) = FileMeta::load(dst_buf)
-        {
-            xlmeta = nmeta
-        }
-
-        let mut skip_parent = dst_volume_dir.clone();
-        if has_dst_buf.as_ref().is_some()
-            && let Some(parent) = dst_file_path.parent()
-        {
-            skip_parent = parent.to_path_buf();
-        }
-
-        // TODO: Healing
-
-        let version_id = fi.version_id.unwrap_or_default();
-        let search_version_id = Some(version_id);
         let no_inline = fi.data.is_none() && fi.size > 0;
 
-        // Check if there's an existing version with the same version_id that has a data_dir to clean up
-        // Reuse one metadata scan to find the version data_dir and determine whether it is shared.
-        let has_old_data_dir = xlmeta.find_unshared_data_dir_for_version(search_version_id);
-        if let Some(old_data_dir) = has_old_data_dir.as_ref() {
-            let _ = xlmeta.data.remove_two(version_id, *old_data_dir);
-        }
-
-        xlmeta.add_version(fi)?;
-
-        if xlmeta.versions.len() <= 10 {
-            // TODO: Sign
-        }
-
-        if let Some((src_data_path, dst_data_path)) = has_data_dir_path.as_ref() {
-            let src_file_parent = src_file_path.parent().unwrap_or(src_volume_dir.as_path());
-            let meta_skip_parent = if no_inline {
-                src_file_parent
-            } else {
-                src_volume_dir.as_path()
+        if no_inline {
+            // Non-inline: read xl.meta, parse, write, rename data dir, rename xl.meta
+            let has_dst_buf = match super::fs::read_file(&dst_file_path).await {
+                Ok(res) => Some(res),
+                Err(e) => {
+                    let e: DiskError = to_file_error(e).into();
+                    if e != DiskError::FileNotFound {
+                        return Err(e);
+                    }
+                    None
+                }
             };
+
+            let mut xlmeta = FileMeta::new();
+            if let Some(dst_buf) = has_dst_buf.as_ref()
+                && FileMeta::is_xl2_v1_format(dst_buf)
+                && let Ok(nmeta) = FileMeta::load(dst_buf)
+            {
+                xlmeta = nmeta
+            }
+
+            let mut skip_parent = dst_volume_dir.clone();
+            if has_dst_buf.as_ref().is_some()
+                && let Some(parent) = dst_file_path.parent()
+            {
+                skip_parent = parent.to_path_buf();
+            }
+
+            let version_id = fi.version_id.unwrap_or_default();
+            let has_old_data_dir = xlmeta.find_unshared_data_dir_for_version(Some(version_id));
+            if let Some(old_data_dir) = has_old_data_dir.as_ref() {
+                let _ = xlmeta.data.remove_two(version_id, *old_data_dir);
+            }
+            xlmeta.add_version(fi)?;
             let new_dst_buf = xlmeta.marshal_msg()?;
 
+            let src_file_parent = src_file_path.parent().unwrap_or(src_volume_dir.as_path());
             self.write_all_private(
                 src_volume,
-                format!("{}/{}", &src_path, STORAGE_FORMAT_FILE).as_str(),
+                &format!("{}/{}", &src_path, STORAGE_FORMAT_FILE),
                 new_dst_buf.into(),
                 true,
-                meta_skip_parent,
+                src_file_parent,
             )
             .await?;
 
-            if no_inline && let Err(err) = rename_all(&src_data_path, &dst_data_path, &skip_parent).await {
+            if let Some((src_data_path, dst_data_path)) = has_data_dir_path.as_ref()
+                && let Err(err) = rename_all(src_data_path, dst_data_path, &skip_parent).await
+            {
                 let _ = self.delete_file(&dst_volume_dir, dst_data_path, false, false).await;
                 info!(
                     "rename all failed src_data_path: {:?}, dst_data_path: {:?}, err: {:?}",
@@ -2633,19 +2656,21 @@ impl DiskAPI for LocalDisk {
                 );
                 return Err(err);
             }
-        } else {
-            let new_dst_buf = xlmeta.marshal_msg()?;
-            self.write_all(src_volume, format!("{}/{}", &src_path, STORAGE_FORMAT_FILE).as_str(), new_dst_buf.into())
-                .await?;
-        }
 
-        if let Some(old_data_dir) = has_old_data_dir {
-            // preserve current xl.meta inside the oldDataDir.
-            if let Some(dst_buf) = has_dst_buf
+            if let Err(err) = rename_all(&src_file_path, &dst_file_path, &skip_parent).await {
+                if let Some((_, dst_data_path)) = has_data_dir_path.as_ref() {
+                    let _ = self.delete_file(&dst_volume_dir, dst_data_path, false, false).await;
+                }
+                info!("rename all failed err: {:?}", err);
+                return Err(err);
+            }
+
+            if let Some(old_data_dir) = has_old_data_dir
+                && let Some(dst_buf) = has_dst_buf
                 && let Err(err) = self
                     .write_all_private(
                         dst_volume,
-                        format!("{}/{}/{}", &dst_path, &old_data_dir.to_string(), STORAGE_FORMAT_FILE).as_str(),
+                        &format!("{}/{}/{}", &dst_path, &old_data_dir.to_string(), STORAGE_FORMAT_FILE),
                         dst_buf.into(),
                         true,
                         &skip_parent,
@@ -2655,30 +2680,106 @@ impl DiskAPI for LocalDisk {
                 info!("write_all_private failed err: {:?}", err);
                 return Err(err);
             }
-        }
 
-        if let Err(err) = rename_all(&src_file_path, &dst_file_path, &skip_parent).await {
-            if let Some((_, dst_data_path)) = has_data_dir_path.as_ref() {
-                let _ = self.delete_file(&dst_volume_dir, dst_data_path, false, false).await;
+            if let Some(src_file_path_parent) = src_file_path.parent() {
+                if src_volume != super::RUSTFS_META_MULTIPART_BUCKET {
+                    let _ = remove_std(src_file_path_parent);
+                } else {
+                    let _ = self
+                        .delete_file(&dst_volume_dir, &src_file_path_parent.to_path_buf(), true, false)
+                        .await;
+                }
             }
-            info!("rename all failed err: {:?}", err);
-            return Err(err);
-        }
 
-        if let Some(src_file_path_parent) = src_file_path.parent() {
-            if src_volume != super::RUSTFS_META_MULTIPART_BUCKET {
-                let _ = remove_std(src_file_path_parent);
+            Ok(RenameDataResp {
+                old_data_dir: has_old_data_dir,
+                sign: None,
+            })
+        } else {
+            // Inline: merge read + parse + write + rename into single spawn_blocking
+            let src = src_file_path.clone();
+            let dst = dst_file_path.clone();
+            let cleanup_path = if src_volume == super::RUSTFS_META_MULTIPART_BUCKET {
+                src_file_path.parent().map(|p| p.to_path_buf())
             } else {
-                let _ = self
-                    .delete_file(&dst_volume_dir, &src_file_path_parent.to_path_buf(), true, false)
-                    .await;
-            }
-        }
+                None
+            };
 
-        Ok(RenameDataResp {
-            old_data_dir: has_old_data_dir,
-            sign: None, // TODO:
-        })
+            let (old_data_dir, _dst_buf) = tokio::task::spawn_blocking(move || {
+                // Read existing xl.meta
+                let has_dst_buf = match std::fs::read(&dst) {
+                    Ok(buf) => Some(Bytes::from(buf)),
+                    Err(e) if e.kind() == std::io::ErrorKind::NotFound => None,
+                    Err(e) => return Err(to_file_error(e)),
+                };
+
+                let mut xlmeta = FileMeta::new();
+                if let Some(ref buf) = has_dst_buf
+                    && FileMeta::is_xl2_v1_format(buf)
+                    && let Ok(nmeta) = FileMeta::load(buf)
+                {
+                    xlmeta = nmeta
+                }
+
+                let version_id = fi.version_id.unwrap_or_default();
+                let old_data_dir = xlmeta.find_unshared_data_dir_for_version(Some(version_id));
+                if let Some(d) = old_data_dir.as_ref() {
+                    let _ = xlmeta.data.remove_two(version_id, *d);
+                }
+                xlmeta.add_version(fi)?;
+                let new_buf = xlmeta.marshal_msg()?;
+
+                // Write new xl.meta + rename
+                if let Some(parent) = src.parent() {
+                    std::fs::create_dir_all(parent)?;
+                }
+                let mut f = std::fs::OpenOptions::new()
+                    .create(true)
+                    .write(true)
+                    .truncate(true)
+                    .open(&src)?;
+                std::io::Write::write_all(&mut f, &new_buf)?;
+                match std::fs::rename(&src, &dst) {
+                    Ok(()) => Ok(()),
+                    Err(err) if err.kind() == std::io::ErrorKind::NotFound && !src.exists() => Ok(()),
+                    Err(err) if err.kind() == std::io::ErrorKind::NotFound => {
+                        if let Some(parent) = dst.parent() {
+                            std::fs::create_dir_all(parent)?;
+                        }
+                        std::fs::rename(&src, &dst).map_err(to_file_error)?;
+                        Ok(())
+                    }
+                    Err(err) => Err(to_file_error(err)),
+                }?;
+
+                if let Some(old_dir) = old_data_dir.as_ref()
+                    && let Some(ref buf) = has_dst_buf
+                    && let Some(dst_parent) = dst.parent()
+                {
+                    let old_path = dst_parent.join(old_dir.to_string()).join(STORAGE_FORMAT_FILE);
+                    if let Some(old_parent) = old_path.parent() {
+                        std::fs::create_dir_all(old_parent)?;
+                    }
+                    std::fs::write(&old_path, buf).map_err(to_file_error)?;
+                }
+
+                Ok::<(Option<uuid::Uuid>, Option<Bytes>), std::io::Error>((old_data_dir, has_dst_buf))
+            })
+            .await
+            .map_err(DiskError::from)??;
+
+            // Cleanup
+            if let Some(ref cleanup) = cleanup_path {
+                let _ = self.delete_file(&dst_volume_dir, cleanup, true, false).await;
+            } else if let Some(parent) = src_file_path.parent() {
+                let _ = remove_std(parent);
+            }
+
+            Ok(RenameDataResp {
+                old_data_dir,
+                sign: None,
+            })
+        }
     }
 
     #[tracing::instrument(skip(self))]

crates/ecstore/src/disk/os.rs

@@ -140,36 +140,55 @@ pub async fn rename_all(
     Ok(())
 }
 
+#[tracing::instrument(level = "debug", skip_all)]
+pub async fn rename_all_ignore_missing_source(
+    src_file_path: impl AsRef<Path>,
+    dst_file_path: impl AsRef<Path>,
+    base_dir: impl AsRef<Path>,
+) -> Result<()> {
+    match reliable_rename_inner(src_file_path, dst_file_path.as_ref(), base_dir, false).await {
+        Ok(()) => Ok(()),
+        Err(err) if err.kind() == io::ErrorKind::NotFound => Ok(()),
+        Err(err) => Err(to_file_error(err).into()),
+    }
+}
+
 async fn reliable_rename(
     src_file_path: impl AsRef<Path>,
     dst_file_path: impl AsRef<Path>,
     base_dir: impl AsRef<Path>,
+) -> io::Result<()> {
+    reliable_rename_inner(src_file_path, dst_file_path, base_dir, true).await
+}
+
+async fn reliable_rename_inner(
+    src_file_path: impl AsRef<Path>,
+    dst_file_path: impl AsRef<Path>,
+    base_dir: impl AsRef<Path>,
+    warn_on_failure: bool,
 ) -> io::Result<()> {
     if let Some(parent) = dst_file_path.as_ref().parent()
         && !file_exists(parent)
     {
-        // info!("reliable_rename reliable_mkdir_all parent: {:?}", parent);
         reliable_mkdir_all(parent, base_dir.as_ref()).await?;
     }
 
     let mut i = 0;
     loop {
         if let Err(e) = super::fs::rename_std(src_file_path.as_ref(), dst_file_path.as_ref()) {
-            if e.kind() == io::ErrorKind::NotFound {
-                break;
-            }
-
             if i == 0 {
                 i += 1;
                 continue;
             }
-            warn!(
-                "reliable_rename failed. src_file_path: {:?}, dst_file_path: {:?}, base_dir: {:?}, err: {:?}",
-                src_file_path.as_ref(),
-                dst_file_path.as_ref(),
-                base_dir.as_ref(),
-                e
-            );
+            if warn_on_failure {
+                warn!(
+                    "reliable_rename failed. src_file_path: {:?}, dst_file_path: {:?}, base_dir: {:?}, err: {:?}",
+                    src_file_path.as_ref(),
+                    dst_file_path.as_ref(),
+                    base_dir.as_ref(),
+                    e
+                );
+            }
             return Err(e);
         }
 
@@ -249,3 +268,36 @@ pub async fn os_mkdir_all(dir_path: impl AsRef<Path>, base_dir: impl AsRef<Path>
 pub fn file_exists(path: impl AsRef<Path>) -> bool {
     std::fs::metadata(path.as_ref()).map(|_| true).unwrap_or(false)
 }
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+    use tempfile::tempdir;
+
+    #[tokio::test]
+    async fn rename_all_missing_source_returns_file_not_found() {
+        let temp_dir = tempdir().expect("create temp dir");
+        let src = temp_dir.path().join("missing");
+        let dst = temp_dir.path().join("dst");
+
+        let err = rename_all(&src, &dst, temp_dir.path())
+            .await
+            .expect_err("missing source must fail");
+
+        assert!(matches!(err, DiskError::FileNotFound));
+        assert!(!dst.exists());
+    }
+
+    #[tokio::test]
+    async fn rename_all_ignore_missing_source_returns_ok() {
+        let temp_dir = tempdir().expect("create temp dir");
+        let src = temp_dir.path().join("missing");
+        let dst = temp_dir.path().join("dst");
+
+        rename_all_ignore_missing_source(&src, &dst, temp_dir.path())
+            .await
+            .expect("missing cleanup source must be ignored");
+
+        assert!(!dst.exists());
+    }
+}

crates/ecstore/src/endpoints.rs

@@ -683,14 +683,46 @@ fn validate_local_physical_disk_independence(pools: &[Endpoints]) -> Result<()>
     }
 
     let mut device_paths = BTreeMap::<String, BTreeSet<String>>::new();
+    #[cfg(not(windows))]
     let mut missing_paths = Vec::new();
 
     for path in &local_paths {
         let canonical = match rustfs_utils::canonicalize(path) {
             Ok(path) => path,
             Err(err) if err.kind() == ErrorKind::NotFound => {
-                missing_paths.push(path.clone());
-                continue;
+                // On Windows, canonicalize can fail for ZFS volumes, junction points,
+                // subst drives, and other non-standard mounts. Try absolutize as fallback.
+                #[cfg(windows)]
+                {
+                    match crate::disk::endpoint::windows_fallback_local_path(path, &err, "disk independence validation") {
+                        Ok(absolute) => {
+                            let abs_path = absolute.to_string_lossy().into_owned();
+                            match rustfs_utils::os::get_physical_device_ids(&abs_path) {
+                                Ok(ids) => {
+                                    for device_id in ids {
+                                        device_paths.entry(device_id).or_default().insert(abs_path.clone());
+                                    }
+                                }
+                                Err(device_err) => {
+                                    return Err(Error::other(format!(
+                                        "failed to inspect physical disk for local endpoint '{abs_path}' after fallback path resolution: {device_err}"
+                                    )));
+                                }
+                            }
+                            continue;
+                        }
+                        Err(fallback_err) => {
+                            return Err(Error::other(format!(
+                                "failed to resolve local endpoint path '{path}' for disk validation: {err}; fallback resolution failed: {fallback_err}"
+                            )));
+                        }
+                    }
+                }
+                #[cfg(not(windows))]
+                {
+                    missing_paths.push(path.clone());
+                    continue;
+                }
             }
             Err(err) => {
                 return Err(Error::other(format!(
@@ -708,6 +740,7 @@ fn validate_local_physical_disk_independence(pools: &[Endpoints]) -> Result<()>
         }
     }
 
+    #[cfg(not(windows))]
     if !missing_paths.is_empty() {
         warn!(
             missing_paths = ?missing_paths,

crates/ecstore/src/erasure_coding/encode.rs

@@ -209,6 +209,13 @@ impl Erasure {
     where
         R: AsyncRead + Send + Sync + Unpin + 'static,
     {
+        if self.block_size == 0 {
+            return Err(std::io::Error::new(
+                std::io::ErrorKind::InvalidInput,
+                "erasure block_size must be non-zero",
+            ));
+        }
+
         // Bound queued encoded blocks by memory budget to avoid per-request spikes.
         let expanded_block_bytes = self.shard_size().saturating_mul(self.total_shard_count());
         let max_inflight_bytes = rustfs_utils::get_env_usize(
@@ -223,8 +230,9 @@ impl Erasure {
             let mut total = 0;
             let mut buf = vec![0u8; block_size];
             loop {
-                match rustfs_utils::read_full(&mut reader, &mut buf).await {
-                    Ok(n) if n > 0 => {
+                match rustfs_utils::read_full_or_eof(&mut reader, &mut buf).await {
+                    Ok(Some(n)) => {
+                        debug_assert!(n > 0, "non-zero block_size prevents zero-length reads");
                         total += n;
                         let erasure = self.clone();
                         let encode_buf = std::mem::take(&mut buf);
@@ -243,7 +251,7 @@ impl Erasure {
                             return Err(std::io::Error::other(format!("Failed to send encoded data : {err}")));
                         }
                     }
-                    Ok(_) => {
+                    Ok(None) => {
                         break;
                     }
                     Err(e) if e.kind() == std::io::ErrorKind::UnexpectedEof => {
@@ -253,7 +261,7 @@ impl Erasure {
                         {
                             return Err(std::io::Error::new(std::io::ErrorKind::InvalidData, e.to_string()));
                         }
-                        break;
+                        return Err(e);
                     }
                     Err(e) => {
                         return Err(e);
@@ -294,13 +302,42 @@ impl Erasure {
         writers.shutdown().await?;
         Ok((reader, total))
     }
+
+    /// Fast path for small inline objects: skip tokio::spawn + mpsc channel.
+    /// Reads all data, encodes directly, writes shards sequentially.
+    pub async fn encode_inline_small<R>(
+        self: Arc<Self>,
+        mut reader: R,
+        writers: &mut [Option<BitrotWriterWrapper>],
+        quorum: usize,
+    ) -> std::io::Result<(R, usize)>
+    where
+        R: AsyncRead + Send + Sync + Unpin,
+    {
+        use tokio::io::AsyncReadExt;
+
+        let mut buf = Vec::with_capacity(self.block_size);
+        let total = reader.read_to_end(&mut buf).await?;
+
+        if total == 0 {
+            return Ok((reader, 0));
+        }
+
+        let shards = self.encode_data(&buf)?;
+        let mut mw = MultiWriter::new(writers, quorum);
+        mw.write(shards).await?;
+        mw.shutdown().await?;
+        Ok((reader, total))
+    }
 }
 
 #[cfg(test)]
 mod tests {
     use super::*;
     use crate::erasure_coding::{BitrotWriterWrapper, CustomWriter};
+    use rustfs_rio::HardLimitReader;
     use rustfs_utils::HashAlgorithm;
+    use std::io::Cursor;
     use std::pin::Pin;
     use std::sync::{Arc, Mutex};
     use std::task::{Context, Poll};
@@ -357,6 +394,103 @@ mod tests {
         assert!(!committed.lock().unwrap().is_empty());
     }
 
+    #[tokio::test]
+    async fn encode_returns_unexpected_eof_for_truncated_limited_reader() {
+        let committed = Arc::new(Mutex::new(Vec::new()));
+        let writer = DeferredCommitWriter::new(committed);
+        let mut writers = vec![Some(BitrotWriterWrapper::new(
+            CustomWriter::new_tokio_writer(writer),
+            16,
+            HashAlgorithm::HighwayHash256S,
+        ))];
+
+        let erasure = Arc::new(Erasure::new(1, 0, 16));
+        let truncated = HardLimitReader::new(Cursor::new(b"short".to_vec()), 10);
+
+        let err = match erasure.encode(truncated, &mut writers, 1).await {
+            Ok(_) => panic!("truncated input must fail"),
+            Err(err) => err,
+        };
+
+        assert_eq!(err.kind(), std::io::ErrorKind::UnexpectedEof);
+    }
+
+    #[tokio::test]
+    async fn encode_rejects_zero_block_size() {
+        let committed = Arc::new(Mutex::new(Vec::new()));
+        let writer = DeferredCommitWriter::new(committed);
+        let mut writers = vec![Some(BitrotWriterWrapper::new(
+            CustomWriter::new_tokio_writer(writer),
+            16,
+            HashAlgorithm::HighwayHash256S,
+        ))];
+
+        let erasure = Arc::new(Erasure::new(1, 0, 0));
+        let reader = tokio::io::BufReader::new(std::io::Cursor::new(b"payload".to_vec()));
+        let err = erasure
+            .encode(reader, &mut writers, 1)
+            .await
+            .expect_err("zero block size must be rejected");
+
+        assert_eq!(err.kind(), std::io::ErrorKind::InvalidInput);
+        assert!(err.to_string().contains("block_size"));
+    }
+
+    /// encode_inline_small: empty reader returns (reader, 0) without writing to any shard.
+    #[tokio::test]
+    async fn encode_inline_small_empty_stream_returns_zero() {
+        let committed = Arc::new(Mutex::new(Vec::new()));
+        let writer = DeferredCommitWriter::new(committed.clone());
+        // 1 data shard, 0 parity shards, block_size = 16
+        let mut writers = vec![Some(BitrotWriterWrapper::new(
+            CustomWriter::new_tokio_writer(writer),
+            16,
+            HashAlgorithm::HighwayHash256S,
+        ))];
+
+        let erasure = Arc::new(Erasure::new(1, 0, 16));
+        let reader = tokio::io::BufReader::new(std::io::Cursor::new(Vec::<u8>::new()));
+        let (_reader, total) = erasure.encode_inline_small(reader, &mut writers, 1).await.unwrap();
+
+        assert_eq!(total, 0);
+        // No shutdown was called, so nothing should be committed
+        assert!(committed.lock().unwrap().is_empty());
+    }
+
+    /// encode_inline_small: small payload is encoded into the correct number of shards
+    /// and each writer receives data after shutdown.
+    #[tokio::test]
+    async fn encode_inline_small_payload_writes_all_shards() {
+        const DATA_SHARDS: usize = 2;
+        const PARITY_SHARDS: usize = 2;
+        const TOTAL_SHARDS: usize = DATA_SHARDS + PARITY_SHARDS;
+        const BLOCK_SIZE: usize = 64;
+
+        let committed: Vec<Arc<Mutex<Vec<u8>>>> = (0..TOTAL_SHARDS).map(|_| Arc::new(Mutex::new(Vec::new()))).collect();
+
+        let mut writers: Vec<Option<BitrotWriterWrapper>> = committed
+            .iter()
+            .map(|c| {
+                Some(BitrotWriterWrapper::new(
+                    CustomWriter::new_tokio_writer(DeferredCommitWriter::new(c.clone())),
+                    BLOCK_SIZE / DATA_SHARDS,
+                    HashAlgorithm::HighwayHash256S,
+                ))
+            })
+            .collect();
+
+        let payload = b"hello inline small";
+        let erasure = Arc::new(Erasure::new(DATA_SHARDS, PARITY_SHARDS, BLOCK_SIZE));
+        let reader = tokio::io::BufReader::new(std::io::Cursor::new(payload.to_vec()));
+        let (_reader, total) = erasure.encode_inline_small(reader, &mut writers, DATA_SHARDS).await.unwrap();
+
+        assert_eq!(total, payload.len());
+        // All shards must have received data (shutdown flushed the bitrot header + shard bytes)
+        for (i, c) in committed.iter().enumerate() {
+            assert!(!c.lock().unwrap().is_empty(), "shard {i} should have received data");
+        }
+    }
+
     #[test]
     fn encode_channel_capacity_never_returns_zero() {
         assert_eq!(encode_channel_capacity(0, 1024), 1);

crates/ecstore/src/erasure_coding/erasure.rs

@@ -308,7 +308,6 @@ where
 /// - `encoder`: Optional ReedSolomon encoder instance.
 /// - `block_size`: Block size for each shard.
 /// - `_id`: Unique identifier for the erasure instance.
-/// - `_buf`: Internal buffer for block operations.
 ///
 /// # Example
 /// ```ignore
@@ -326,7 +325,6 @@ pub struct Erasure {
     pub block_size: usize,
     uses_legacy: bool,
     _id: Uuid,
-    _buf: Vec<u8>,
 }
 
 impl Default for Erasure {
@@ -339,7 +337,6 @@ impl Default for Erasure {
             block_size: 0,
             uses_legacy: false,
             _id: Uuid::nil(),
-            _buf: vec![],
         }
     }
 }
@@ -354,7 +351,6 @@ impl Clone for Erasure {
             block_size: self.block_size,
             uses_legacy: self.uses_legacy,
             _id: Uuid::new_v4(), // Generate new ID for clone
-            _buf: vec![0u8; self.block_size],
         }
     }
 }
@@ -399,7 +395,6 @@ impl Erasure {
             legacy_encoder,
             uses_legacy,
             _id: Uuid::new_v4(),
-            _buf: vec![0u8; block_size],
         }
     }
 
@@ -579,12 +574,22 @@ impl Erasure {
         F: FnMut(std::io::Result<Vec<Bytes>>) -> Fut + Send,
         Fut: std::future::Future<Output = Result<(), E>> + Send,
     {
+        if self.block_size == 0 {
+            on_block(Err(std::io::Error::new(
+                std::io::ErrorKind::InvalidInput,
+                "erasure block_size must be non-zero",
+            )))
+            .await?;
+            return Ok(0);
+        }
+
         let block_size = self.block_size;
         let mut total = 0;
         let mut buf = vec![0u8; block_size];
         loop {
-            match rustfs_utils::read_full(&mut *reader, &mut buf).await {
-                Ok(n) if n > 0 => {
+            match rustfs_utils::read_full_or_eof(&mut *reader, &mut buf).await {
+                Ok(Some(n)) => {
+                    debug_assert!(n > 0, "non-zero block_size prevents zero-length reads");
                     warn!("encode_stream_callback_async read n={}", n);
                     total += n;
                     let erasure = self.clone();
@@ -604,7 +609,7 @@ impl Erasure {
                     buf = returned_buf;
                     on_block(res).await?
                 }
-                Ok(_) => {
+                Ok(None) => {
                     warn!("encode_stream_callback_async read unexpected ok");
                     break;
                 }
@@ -1032,6 +1037,35 @@ mod tests {
         assert_eq!(&recovered, &data_clone);
     }
 
+    #[tokio::test]
+    async fn test_encode_stream_callback_async_reports_zero_block_size() {
+        use std::io::Cursor;
+        use std::sync::{Arc, Mutex};
+
+        let erasure = Arc::new(Erasure::new(1, 0, 0));
+        let mut reader = Cursor::new(b"payload".to_vec());
+        let observed = Arc::new(Mutex::new(None));
+        let observed_clone = observed.clone();
+
+        let total = erasure
+            .encode_stream_callback_async::<_, _, (), _>(&mut reader, move |res| {
+                let observed = observed_clone.clone();
+                async move {
+                    let err = res.expect_err("zero block size should report an error");
+                    *observed.lock().unwrap() = Some((err.kind(), err.to_string()));
+                    Ok(())
+                }
+            })
+            .await
+            .expect("callback should handle the zero block size error");
+
+        assert_eq!(total, 0);
+        let observed = observed.lock().unwrap();
+        let (kind, message) = observed.as_ref().expect("callback should be invoked once");
+        assert_eq!(*kind, std::io::ErrorKind::InvalidInput);
+        assert!(message.contains("block_size"));
+    }
+
     // SIMD mode specific tests
     mod simd_tests {
         use super::*;

crates/ecstore/src/notification_sys.rs

@@ -189,6 +189,65 @@ impl NotificationSys {
         join_all(futures).await
     }
 
+    pub async fn reload_dynamic_config(&self, sub_sys: &str) -> Vec<NotificationPeerErr> {
+        let mut futures = Vec::with_capacity(self.peer_clients.len());
+        for client in self.peer_clients.iter() {
+            let sub_sys = sub_sys.to_string();
+            futures.push(async move {
+                if let Some(client) = client {
+                    match client
+                        .signal_service(crate::rpc::SERVICE_SIGNAL_RELOAD_DYNAMIC, &sub_sys, false, SystemTime::UNIX_EPOCH)
+                        .await
+                    {
+                        Ok(_) => NotificationPeerErr {
+                            host: client.host.to_string(),
+                            err: None,
+                        },
+                        Err(e) => NotificationPeerErr {
+                            host: client.host.to_string(),
+                            err: Some(e),
+                        },
+                    }
+                } else {
+                    NotificationPeerErr {
+                        host: "".to_string(),
+                        err: Some(Error::other("peer is not reachable")),
+                    }
+                }
+            });
+        }
+        join_all(futures).await
+    }
+
+    pub async fn refresh_config_snapshot(&self) -> Vec<NotificationPeerErr> {
+        let mut futures = Vec::with_capacity(self.peer_clients.len());
+        for client in self.peer_clients.iter() {
+            futures.push(async move {
+                if let Some(client) = client {
+                    match client
+                        .signal_service(crate::rpc::SERVICE_SIGNAL_REFRESH_CONFIG, "", false, SystemTime::UNIX_EPOCH)
+                        .await
+                    {
+                        Ok(_) => NotificationPeerErr {
+                            host: client.host.to_string(),
+                            err: None,
+                        },
+                        Err(e) => NotificationPeerErr {
+                            host: client.host.to_string(),
+                            err: Some(e),
+                        },
+                    }
+                } else {
+                    NotificationPeerErr {
+                        host: "".to_string(),
+                        err: Some(Error::other("peer is not reachable")),
+                    }
+                }
+            });
+        }
+        join_all(futures).await
+    }
+
     pub async fn storage_info<S: StorageAPI>(&self, api: &S) -> rustfs_madmin::StorageInfo {
         let mut futures = Vec::with_capacity(self.peer_clients.len());
         let endpoints = get_global_endpoints();

crates/ecstore/src/rebalance.rs

@@ -1002,7 +1002,7 @@ impl ECStore {
                 return true;
             }
 
-            // Mark pool rebalance as done if within 5% of the PercentFreeGoal
+            // Mark pool rebalance as done only after it reaches the PercentFreeGoal.
             let pfi = if pool_stat.init_capacity == 0 {
                 0.0
             } else {
@@ -1032,7 +1032,7 @@ fn rebalance_goal_reached(init_free_space: u64, init_capacity: u64, bytes: u64,
     }
 
     let pfi = (init_free_space + bytes) as f64 / init_capacity as f64;
-    (pfi - percent_free_goal).abs() <= 0.05 + f64::EPSILON
+    pfi + f64::EPSILON >= percent_free_goal
 }
 
 fn percent_free_ratio(total_free: u64, total_cap: u64) -> f64 {
@@ -2664,23 +2664,20 @@ mod rebalance_unit_tests {
     }
 
     #[test]
-    fn test_rebalance_goal_reached_exact_tolerance_bound() {
+    fn test_rebalance_goal_reached_at_target() {
         let init_free_space = 200_u64;
         let init_capacity = 1_000_u64;
         let goal = 0.45_f64;
 
-        // goal - 0.05 => 200 + 200 = 400 free / 1000 => 0.4 exactly
-        assert!(rebalance_goal_reached(init_free_space, init_capacity, 200, goal));
-
-        // one byte above the tolerance boundary should be false
-        assert!(!rebalance_goal_reached(init_free_space, init_capacity, 199, goal));
+        assert!(rebalance_goal_reached(init_free_space, init_capacity, 250, goal));
+        assert!(!rebalance_goal_reached(init_free_space, init_capacity, 249, goal));
     }
 
     #[test]
-    fn test_rebalance_goal_reached_within_tolerance() {
+    fn test_rebalance_goal_reached_above_target() {
         let init_free_space = 200_u64;
         let init_capacity = 1_000_u64;
-        let bytes = 250_u64;
+        let bytes = 251_u64;
         let goal = 0.45_f64;
 
         assert!(rebalance_goal_reached(init_free_space, init_capacity, bytes, goal));
@@ -2702,12 +2699,12 @@ mod rebalance_unit_tests {
     }
 
     #[test]
-    fn test_rebalance_goal_above_one_is_true_when_within_tolerance() {
+    fn test_rebalance_goal_above_one_is_true_when_reached() {
         assert!(rebalance_goal_reached(950, 1_000, 100, 1.0));
     }
 
     #[test]
-    fn test_rebalance_goal_below_zero_is_true_when_within_tolerance() {
+    fn test_rebalance_goal_below_zero_is_true_when_reached() {
         assert!(rebalance_goal_reached(10, 1_000, 0, -0.01));
     }
 
@@ -3355,6 +3352,18 @@ mod rebalance_unit_tests {
         assert!(!should_pool_participate(300, 1_000, 0.3));
     }
 
+    #[test]
+    fn test_rebalance_goal_not_reached_for_issue_3137_initial_imbalance() {
+        let pool0_capacity = 10_000_u64;
+        let pool0_free = 9_135_u64;
+        let pool1_capacity = 10_000_u64;
+        let pool1_free = 9_800_u64;
+        let goal = percent_free_ratio(pool0_free + pool1_free, pool0_capacity + pool1_capacity);
+
+        assert!(should_pool_participate(pool0_free, pool0_capacity, goal));
+        assert!(!rebalance_goal_reached(pool0_free, pool0_capacity, 0, goal));
+    }
+
     #[test]
     fn test_complete_rebalance_pools_at_goal_marks_started_participants_completed() {
         let now = OffsetDateTime::from_unix_timestamp(1_000).unwrap();
@@ -3365,7 +3374,7 @@ mod rebalance_unit_tests {
                     participating: true,
                     init_free_space: 400,
                     init_capacity: 1_000,
-                    bytes: 50,
+                    bytes: 100,
                     info: RebalanceInfo {
                         status: RebalStatus::Started,
                         ..Default::default()
@@ -4129,13 +4138,13 @@ mod rebalance_unit_tests {
     }
 
     #[test]
-    fn test_rebalance_goal_reached_tolerance_and_regression() {
+    fn test_rebalance_goal_reached_requires_target_ratio() {
         let init_free_space = 150_u64;
         let init_capacity = 800_u64;
         let goal = 0.35_f64;
 
         assert!(!rebalance_goal_reached(init_free_space, init_capacity, 0, goal));
-        assert!(rebalance_goal_reached(init_free_space, init_capacity, 90, goal));
-        assert!(!rebalance_goal_reached(init_free_space, init_capacity, 89, goal));
+        assert!(!rebalance_goal_reached(init_free_space, init_capacity, 129, goal));
+        assert!(rebalance_goal_reached(init_free_space, init_capacity, 130, goal));
     }
 }

crates/ecstore/src/rpc/mod.rs

@@ -29,7 +29,10 @@ pub use internode_data_transport::{
     InternodeDataTransport, InternodeDataTransportCapabilities, ReadStreamRequest, TcpHttpInternodeDataTransport,
     WalkDirStreamRequest, WriteStreamRequest, build_internode_data_transport, build_internode_data_transport_from_env,
 };
-pub use peer_rest_client::PeerRestClient;
+pub use peer_rest_client::{
+    PEER_RESTDRY_RUN, PEER_RESTSIGNAL, PEER_RESTSUB_SYS, PeerRestClient, SERVICE_SIGNAL_REFRESH_CONFIG,
+    SERVICE_SIGNAL_RELOAD_DYNAMIC,
+};
 pub use peer_s3_client::{LocalPeerS3Client, PeerS3Client, RemotePeerS3Client, S3PeerSys};
 pub use remote_disk::RemoteDisk;
 pub use remote_locker::RemoteClient;

crates/ecstore/src/rpc/peer_rest_client.rs

@@ -57,6 +57,8 @@ use tracing::warn;
 pub const PEER_RESTSIGNAL: &str = "signal";
 pub const PEER_RESTSUB_SYS: &str = "sub-sys";
 pub const PEER_RESTDRY_RUN: &str = "dry-run";
+pub const SERVICE_SIGNAL_REFRESH_CONFIG: u64 = 1;
+pub const SERVICE_SIGNAL_RELOAD_DYNAMIC: u64 = 2;
 const PEER_REST_RECOVERY_MAX_ATTEMPTS: u32 = 60;
 const PEER_REST_RECOVERY_MAX_BACKOFF: Duration = Duration::from_secs(30);
 

crates/ecstore/src/rpc/peer_s3_client.rs

@@ -843,12 +843,13 @@ impl PeerS3Client for RemotePeerS3Client {
                 });
                 let response = client.make_bucket(request).await?.into_inner();
 
-                // TODO: deal with error
                 if !response.success {
                     return if let Some(err) = response.error {
                         Err(err.into())
                     } else {
-                        Err(Error::other(""))
+                        Err(Error::other(format!(
+                            "make_bucket({bucket}): peer returned failure without error details"
+                        )))
                     };
                 }
 
@@ -1046,6 +1047,44 @@ async fn clone_drives() -> Vec<Option<DiskStore>> {
 #[cfg(test)]
 mod tests {
     use super::*;
+    use crate::disk::WalkDirOptions;
+    use crate::disk::disk_store::LocalDiskWrapper;
+    use crate::disk::endpoint::Endpoint;
+    use crate::disk::local::LocalDisk;
+    use crate::endpoints::{Endpoints, PoolEndpoints};
+    use crate::global::{GLOBAL_LOCAL_DISK_ID_MAP, GLOBAL_LOCAL_DISK_MAP, GLOBAL_LOCAL_DISK_SET_DRIVES};
+    use crate::store::init_local_disks;
+    use rustfs_filemeta::FileInfo;
+    use serial_test::serial;
+    use std::{
+        io,
+        pin::Pin,
+        task::{Context, Poll},
+    };
+    use tempfile::TempDir;
+    use tokio::io::AsyncWrite;
+
+    struct PendingWriter;
+
+    impl AsyncWrite for PendingWriter {
+        fn poll_write(self: Pin<&mut Self>, _cx: &mut Context<'_>, _buf: &[u8]) -> Poll<io::Result<usize>> {
+            Poll::Pending
+        }
+
+        fn poll_flush(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll<io::Result<()>> {
+            Poll::Ready(Ok(()))
+        }
+
+        fn poll_shutdown(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll<io::Result<()>> {
+            Poll::Ready(Ok(()))
+        }
+    }
+
+    async fn reset_local_disk_globals() {
+        GLOBAL_LOCAL_DISK_MAP.write().await.clear();
+        GLOBAL_LOCAL_DISK_ID_MAP.write().await.clear();
+        GLOBAL_LOCAL_DISK_SET_DRIVES.write().await.clear();
+    }
 
     #[derive(Debug)]
     struct TestPeerS3Client {
@@ -1152,6 +1191,77 @@ mod tests {
         client.cancel_token.cancel();
     }
 
+    #[tokio::test]
+    #[serial]
+    async fn local_get_bucket_info_survives_prior_walk_timeout() {
+        reset_local_disk_globals().await;
+
+        let temp_dir = TempDir::new().expect("create temp dir for local peer listing regression");
+        let disk_path = temp_dir.path().join("disk1");
+        std::fs::create_dir_all(&disk_path).expect("create disk path");
+
+        let mut endpoint = Endpoint::try_from(disk_path.to_str().expect("disk path to str")).expect("endpoint");
+        endpoint.set_pool_index(0);
+        endpoint.set_set_index(0);
+        endpoint.set_disk_index(0);
+
+        let endpoint_pools = EndpointServerPools(vec![PoolEndpoints {
+            legacy: false,
+            set_count: 1,
+            drives_per_set: 1,
+            endpoints: Endpoints::from(vec![endpoint.clone()]),
+            cmd_line: "local-get-bucket-info-survives-prior-walk-timeout".to_string(),
+            platform: "test".to_string(),
+        }]);
+
+        init_local_disks(endpoint_pools).await.expect("init local disks");
+
+        let disk = Arc::new(LocalDisk::new(&endpoint, false).await.expect("local disk should be created"));
+        let wrapper = crate::disk::Disk::Local(Box::new(LocalDiskWrapper::new(disk, false)));
+        let disk_store: DiskStore = Arc::new(wrapper);
+        let bucket = "test-bucket";
+        let object = "test-object";
+
+        disk_store.make_volume(bucket).await.expect("bucket should be created");
+
+        let mut file_info = FileInfo::new(&format!("{bucket}/{object}"), 1, 0);
+        file_info.volume = bucket.to_string();
+        file_info.name = object.to_string();
+        file_info.mod_time = Some(::time::OffsetDateTime::now_utc());
+        file_info.erasure.index = 1;
+
+        disk_store
+            .write_metadata("", bucket, object, file_info)
+            .await
+            .expect("object metadata should be written");
+
+        temp_env::async_with_vars([(rustfs_config::ENV_DRIVE_WALKDIR_TIMEOUT_SECS, Some("1"))], async {
+            let mut writer = PendingWriter;
+            let walk_err = disk_store
+                .walk_dir(
+                    WalkDirOptions {
+                        bucket: bucket.to_string(),
+                        recursive: true,
+                        ..Default::default()
+                    },
+                    &mut writer,
+                )
+                .await
+                .expect_err("walk_dir should time out against a non-draining writer");
+
+            assert_eq!(walk_err, DiskError::Timeout);
+
+            let info = LocalPeerS3Client::new(None, Some(vec![0]))
+                .get_bucket_info(bucket, &BucketOptions::default())
+                .await
+                .expect("bucket info should still succeed after prior walk timeout");
+            assert_eq!(info.name, bucket);
+        })
+        .await;
+
+        reset_local_disk_globals().await;
+    }
+
     #[test]
     fn test_reduce_pool_write_quorum_uses_only_pool_participants() {
         let clients = vec![

crates/ecstore/src/rpc/remote_disk.rs

@@ -110,6 +110,15 @@ pub struct RemoteDisk {
 }
 
 impl RemoteDisk {
+    fn is_retryable_walk_dir_error(err: &DiskError) -> bool {
+        if is_network_like_disk_error(err) {
+            return true;
+        }
+
+        let err_text = err.to_string().to_ascii_lowercase();
+        err_text.contains("httpreader stream error") || err_text.contains("error decoding response body")
+    }
+
     pub(crate) async fn new(ep: &Endpoint, opt: &DiskOption, data_transport: Arc<dyn InternodeDataTransport>) -> Result<Self> {
         let addr = if let Some(port) = ep.url.port() {
             format!("{}://{}:{}", ep.url.scheme(), ep.url.host_str().unwrap(), port)
@@ -1148,7 +1157,8 @@ impl DiskAPI for RemoteDisk {
     ) -> Result<RenameDataResp> {
         info!("rename_data {}/{}/{}/{}", self.addr, self.endpoint.to_string(), dst_volume, dst_path);
 
-        self.execute_with_timeout(
+        self.execute_with_timeout_for_op(
+            "rename_data",
             || async {
                 let file_info = serde_json::to_string(&fi)?;
                 let mut client = self
@@ -1215,24 +1225,58 @@ impl DiskAPI for RemoteDisk {
     async fn walk_dir<W: AsyncWrite + Unpin + Send>(&self, opts: WalkDirOptions, wr: &mut W) -> Result<()> {
         info!("walk_dir {}", self.endpoint.to_string());
 
+        let disk = self.disk_ref().await;
+        let body = serde_json::to_vec(&opts)?;
+        let stall_timeout = get_drive_walkdir_stall_timeout();
+        let bucket = opts.bucket.clone();
+        let base_dir = opts.base_dir.clone();
+        let disk_for_log = disk.clone();
+
         self.execute_with_timeout_for_op_and_health_action(
             "walk_dir",
             || async {
-                let disk = self.disk_ref().await;
-                let opts = serde_json::to_vec(&opts)?;
-                let mut reader = self
-                    .data_transport
-                    .open_walk_dir(WalkDirStreamRequest {
-                        endpoint: self.endpoint.grid_host(),
-                        disk,
-                        body: opts,
-                        stall_timeout: Some(get_drive_walkdir_stall_timeout()),
-                    })
-                    .await?;
+                let mut last_err = None;
 
-                copy_stream_with_buffer(&mut reader, wr, DEFAULT_READ_BUFFER_SIZE).await?;
+                for attempt in 1..=2 {
+                    let mut reader = match self
+                        .data_transport
+                        .open_walk_dir(WalkDirStreamRequest {
+                            endpoint: self.endpoint.grid_host(),
+                            disk: disk.clone(),
+                            body: body.clone(),
+                            stall_timeout: Some(stall_timeout),
+                        })
+                        .await
+                    {
+                        Ok(reader) => reader,
+                        Err(err) => {
+                            if attempt == 1 && Self::is_retryable_walk_dir_error(&err) {
+                                warn!(
+                                    endpoint = %self.endpoint,
+                                    addr = %self.addr,
+                                    disk = %disk_for_log,
+                                    bucket = %bucket,
+                                    base_dir = %base_dir,
+                                    attempt,
+                                    stall_timeout_ms = stall_timeout.as_millis(),
+                                    error = %err,
+                                    "remote walk_dir returned retryable transport error; retrying"
+                                );
+                                last_err = Some(err);
+                                continue;
+                            }
 
-                Ok(())
+                            return Err(err);
+                        }
+                    };
+
+                    match copy_stream_with_buffer(&mut reader, wr, DEFAULT_READ_BUFFER_SIZE).await {
+                        Ok(_) => return Ok(()),
+                        Err(io_err) => return Err(DiskError::Io(io_err)),
+                    }
+                }
+
+                Err(last_err.unwrap_or_else(|| DiskError::other("walk_dir retry exhausted without captured error")))
             },
             get_drive_walkdir_timeout(),
             FailureHealthAction::IgnoreFailure,
@@ -1758,6 +1802,36 @@ mod tests {
         }
     }
 
+    #[derive(Debug)]
+    enum WalkDirTestStep {
+        Error(DiskError),
+        Data(Vec<u8>),
+        PartialDataThenError { data: Vec<u8>, error: io::Error },
+    }
+
+    #[derive(Debug, Clone, Default)]
+    struct RetryingWalkDirInternodeDataTransport {
+        calls: Arc<StdMutex<Vec<RecordedTransportCall>>>,
+        steps: Arc<StdMutex<Vec<WalkDirTestStep>>>,
+    }
+
+    impl RetryingWalkDirInternodeDataTransport {
+        fn with_steps(steps: Vec<WalkDirTestStep>) -> Self {
+            Self {
+                calls: Arc::new(StdMutex::new(Vec::new())),
+                steps: Arc::new(StdMutex::new(steps)),
+            }
+        }
+
+        fn calls(&self) -> Vec<RecordedTransportCall> {
+            self.calls.lock().expect("recorded transport calls lock poisoned").clone()
+        }
+
+        fn record(&self, call: RecordedTransportCall) {
+            self.calls.lock().expect("recorded transport calls lock poisoned").push(call);
+        }
+    }
+
     #[derive(Debug, Default)]
     struct EmptyTestReader;
 
@@ -1810,6 +1884,38 @@ mod tests {
         }
     }
 
+    #[async_trait::async_trait]
+    impl InternodeDataTransport for RetryingWalkDirInternodeDataTransport {
+        async fn open_read(&self, _request: ReadStreamRequest) -> Result<FileReader> {
+            panic!("open_read should not be used in walk_dir retry test");
+        }
+
+        async fn open_write(&self, _request: WriteStreamRequest) -> Result<FileWriter> {
+            panic!("open_write should not be used in walk_dir retry test");
+        }
+
+        async fn open_walk_dir(&self, request: WalkDirStreamRequest) -> Result<FileReader> {
+            self.record(RecordedTransportCall::WalkDir(request));
+            let step = self.steps.lock().expect("walk_dir retry steps lock poisoned").remove(0);
+            match step {
+                WalkDirTestStep::Error(err) => Err(err),
+                WalkDirTestStep::Data(data) => Ok(Box::new(Cursor::new(data))),
+                WalkDirTestStep::PartialDataThenError { data, error } => Ok(Box::new(PartialThenErrorReader {
+                    cursor: Cursor::new(data),
+                    error: Some(error),
+                })),
+            }
+        }
+
+        fn name(&self) -> &'static str {
+            "retrying-walk-dir"
+        }
+
+        fn capabilities(&self) -> InternodeDataTransportCapabilities {
+            InternodeDataTransportCapabilities::tcp_http()
+        }
+    }
+
     async fn new_remote_disk_with_transport(data_transport: Arc<dyn InternodeDataTransport>) -> RemoteDisk {
         let endpoint = Endpoint {
             url: url::Url::parse("http://remote-node:9000/data/rustfs0").unwrap(),
@@ -1826,6 +1932,32 @@ mod tests {
         RemoteDisk::new(&endpoint, &disk_option, data_transport).await.unwrap()
     }
 
+    #[derive(Debug)]
+    struct PartialThenErrorReader {
+        cursor: Cursor<Vec<u8>>,
+        error: Option<io::Error>,
+    }
+
+    impl AsyncRead for PartialThenErrorReader {
+        fn poll_read(mut self: Pin<&mut Self>, cx: &mut Context<'_>, buf: &mut ReadBuf<'_>) -> Poll<io::Result<()>> {
+            let filled_before = buf.filled().len();
+            match Pin::new(&mut self.cursor).poll_read(cx, buf) {
+                Poll::Ready(Ok(())) => {
+                    if buf.filled().len() > filled_before {
+                        return Poll::Ready(Ok(()));
+                    }
+
+                    if let Some(err) = self.error.take() {
+                        return Poll::Ready(Err(err));
+                    }
+
+                    Poll::Ready(Ok(()))
+                }
+                other => other,
+            }
+        }
+    }
+
     fn init_tracing(filter_level: Level) {
         INIT.call_once(|| {
             let _ = tracing_subscriber::fmt()
@@ -2184,6 +2316,66 @@ mod tests {
         }
     }
 
+    #[tokio::test]
+    async fn test_remote_disk_walk_dir_retries_once_on_retryable_transport_error() {
+        let transport = RetryingWalkDirInternodeDataTransport::with_steps(vec![
+            WalkDirTestStep::Error(DiskError::other("HttpReader stream error: error decoding response body")),
+            WalkDirTestStep::Data(b"walk-dir-retry-ok".to_vec()),
+        ]);
+        let remote_disk = new_remote_disk_with_transport(Arc::new(transport.clone())).await;
+        let opts = WalkDirOptions {
+            bucket: "bucket".to_string(),
+            base_dir: "config/iam".to_string(),
+            recursive: true,
+            report_notfound: false,
+            filter_prefix: None,
+            forward_to: None,
+            limit: 10,
+            disk_id: String::new(),
+        };
+        let mut writer = Vec::new();
+
+        remote_disk
+            .walk_dir(opts, &mut writer)
+            .await
+            .expect("retryable walk_dir error should recover");
+
+        assert_eq!(writer, b"walk-dir-retry-ok");
+        assert_eq!(transport.calls().len(), 2, "walk_dir should retry exactly once");
+    }
+
+    #[tokio::test]
+    async fn test_remote_disk_walk_dir_does_not_retry_after_partial_stream_failure() {
+        let transport = RetryingWalkDirInternodeDataTransport::with_steps(vec![
+            WalkDirTestStep::PartialDataThenError {
+                data: b"partial-walk-dir".to_vec(),
+                error: io::Error::new(io::ErrorKind::ConnectionReset, "connection reset"),
+            },
+            WalkDirTestStep::Data(b"walk-dir-retry-ok".to_vec()),
+        ]);
+        let remote_disk = new_remote_disk_with_transport(Arc::new(transport.clone())).await;
+        let opts = WalkDirOptions {
+            bucket: "bucket".to_string(),
+            base_dir: "config/iam".to_string(),
+            recursive: true,
+            report_notfound: false,
+            filter_prefix: None,
+            forward_to: None,
+            limit: 10,
+            disk_id: String::new(),
+        };
+        let mut writer = Vec::new();
+
+        let err = remote_disk
+            .walk_dir(opts, &mut writer)
+            .await
+            .expect_err("partial stream failure should be returned without retry");
+
+        assert!(matches!(err, DiskError::Io(ref io_err) if io_err.kind() == io::ErrorKind::ConnectionReset));
+        assert_eq!(writer, b"partial-walk-dir");
+        assert_eq!(transport.calls().len(), 1, "walk_dir should not retry after writing partial bytes");
+    }
+
     #[tokio::test]
     async fn test_remote_disk_endpoints_with_different_schemes() {
         let test_cases = vec![

crates/ecstore/src/set_disk/heal.rs

@@ -417,7 +417,7 @@ impl SetDisks {
                                 }
 
                                 let is_inline_buffer = {
-                                    if let Some(sc) = GLOBAL_STORAGE_CLASS.get() {
+                                    if let Some(sc) = get_global_storage_class() {
                                         sc.should_inline(erasure.shard_file_size(latest_meta.size), false)
                                     } else {
                                         false

crates/ecstore/src/set_disk/replication.rs

@@ -25,7 +25,7 @@ impl SetDisks {
         let mut oi = obj_info.clone();
         oi.metadata_only = true;
 
-        oi.user_defined.remove(X_AMZ_RESTORE.as_str());
+        Arc::make_mut(&mut oi.user_defined).remove(X_AMZ_RESTORE.as_str());
 
         let version_id = oi.version_id.map(|v| v.to_string());
         let _obj = self

crates/ecstore/src/set_disk/write.rs

@@ -383,30 +383,35 @@ impl SetDisks {
         }
 
         if let Some(err) = reduce_write_quorum_errs(&errs, OBJECT_OP_IGNORED_ERRS, write_quorum) {
-            // TODO: add concurrency
+            let mut revert_futures = Vec::with_capacity(disks.len());
             for (i, err) in errs.iter().enumerate() {
                 if err.is_some() {
                     continue;
                 }
 
                 if let Some(disk) = disks[i].as_ref() {
-                    let _ = disk
-                        .delete(
-                            bucket,
-                            &path_join_buf(&[prefix, STORAGE_FORMAT_FILE]),
-                            DeleteOptions {
-                                recursive: true,
-                                ..Default::default()
-                            },
-                        )
-                        .await
-                        .map_err(|e| {
-                            warn!("write meta revert err {:?}", e);
-                            e
-                        });
+                    let disk = disk.clone();
+                    let bucket = bucket.to_string();
+                    let path = path_join_buf(&[prefix, STORAGE_FORMAT_FILE]);
+                    revert_futures.push(async move {
+                        if let Err(err) = disk
+                            .delete(
+                                &bucket,
+                                &path,
+                                DeleteOptions {
+                                    recursive: true,
+                                    ..Default::default()
+                                },
+                            )
+                            .await
+                        {
+                            warn!("write meta revert err {:?}", err);
+                        }
+                    });
                 }
             }
 
+            join_all(revert_futures).await;
             return Err(err);
         }
         Ok(())

crates/ecstore/src/sets.rs

@@ -306,12 +306,10 @@ impl Sets {
     //     unimplemented!()
     // }
 
-    async fn delete_prefix(&self, bucket: &str, object: &str) -> Result<()> {
+    async fn delete_prefix(&self, bucket: &str, object: &str, opts: &ObjectOptions) -> Result<()> {
         let mut futures = Vec::new();
-        let opt = ObjectOptions {
-            delete_prefix: true,
-            ..Default::default()
-        };
+        let mut opt = opts.clone();
+        opt.delete_prefix = true;
 
         for set in self.disk_set.iter() {
             futures.push(set.delete_object(bucket, object, opt.clone()));
@@ -463,6 +461,7 @@ impl ObjectOperations for Sets {
             versioned: dst_opts.versioned,
             version_id: dst_opts.version_id.clone(),
             mod_time: dst_opts.mod_time,
+            http_preconditions: dst_opts.http_preconditions.clone(),
             ..Default::default()
         };
 
@@ -485,7 +484,7 @@ impl ObjectOperations for Sets {
     #[tracing::instrument(skip(self))]
     async fn delete_object(&self, bucket: &str, object: &str, opts: ObjectOptions) -> Result<ObjectInfo> {
         if opts.delete_prefix && !opts.delete_prefix_object {
-            self.delete_prefix(bucket, object).await?;
+            self.delete_prefix(bucket, object, &opts).await?;
             return Ok(ObjectInfo::default());
         }
 

crates/ecstore/src/store.rs

@@ -31,7 +31,6 @@ use crate::bucket::utils::check_object_args;
 use crate::bucket::utils::check_put_object_args;
 use crate::bucket::utils::check_put_object_part_args;
 use crate::bucket::utils::{check_valid_bucket_name, check_valid_bucket_name_strict, is_meta_bucketname};
-use crate::config::GLOBAL_STORAGE_CLASS;
 use crate::config::storageclass;
 use crate::disk::endpoint::{Endpoint, EndpointType};
 use crate::disk::{DiskAPI, DiskInfo, DiskInfoOptions};

crates/ecstore/src/store/bucket.rs

@@ -13,12 +13,48 @@
 // limitations under the License.
 
 use super::*;
-use crate::bucket::utils::is_meta_bucketname;
+use crate::bucket::{metadata::BUCKET_TABLE_RESERVED_PREFIX, utils::is_meta_bucketname};
+use crate::set_disk::get_lock_acquire_timeout;
 
 fn should_override_created_from_metadata(created: OffsetDateTime) -> bool {
     created != OffsetDateTime::UNIX_EPOCH
 }
 
+fn validate_table_bucket_delete_allowed(
+    bucket: &str,
+    table_bucket_enabled: bool,
+    table_catalog_metadata_exists: bool,
+) -> Result<()> {
+    if table_bucket_enabled && table_catalog_metadata_exists {
+        return Err(StorageError::BucketNotEmpty(bucket.to_string()));
+    }
+
+    Ok(())
+}
+
+async fn table_catalog_metadata_exists(bucket: &str) -> bool {
+    let local_disks = all_local_disk().await;
+    for disk in local_disks.iter() {
+        let catalog_path = disk.path().join(bucket).join(BUCKET_TABLE_RESERVED_PREFIX);
+        if has_xlmeta_files(&catalog_path).await {
+            return true;
+        }
+    }
+
+    false
+}
+
+async fn validate_table_bucket_delete_guard(bucket: &str) -> Result<()> {
+    let table_bucket_enabled = metadata_sys::get(bucket)
+        .await
+        .is_ok_and(|metadata| metadata.table_bucket_enabled());
+    if table_bucket_enabled {
+        validate_table_bucket_delete_allowed(bucket, true, table_catalog_metadata_exists(bucket).await)?;
+    }
+
+    Ok(())
+}
+
 impl ECStore {
     #[instrument(skip(self))]
     pub(super) async fn handle_make_bucket(&self, bucket: &str, opts: &MakeBucketOptions) -> Result<()> {
@@ -28,7 +64,28 @@ impl ECStore {
             return Err(StorageError::BucketNameInvalid(err.to_string()));
         }
 
-        // TODO: nslock
+        let _ns_guard = if !opts.no_lock {
+            let ns_lock = self.new_ns_lock(bucket, bucket).await?;
+            Some(
+                ns_lock
+                    .get_write_lock(get_lock_acquire_timeout())
+                    .await
+                    .map_err(|e| match e {
+                        rustfs_lock::error::LockError::QuorumNotReached { required, achieved } => {
+                            StorageError::NamespaceLockQuorumUnavailable {
+                                mode: "write",
+                                bucket: bucket.to_string(),
+                                object: bucket.to_string(),
+                                required,
+                                achieved,
+                            }
+                        }
+                        other => StorageError::other(format!("make_bucket: failed to acquire write lock on {bucket}: {other}")),
+                    })?,
+            )
+        } else {
+            None
+        };
 
         if let Err(err) = self.peer_sys.make_bucket(bucket, opts).await {
             let err = to_object_err(err.into(), vec![bucket]);
@@ -111,7 +168,28 @@ impl ECStore {
             return Err(StorageError::BucketNameInvalid(err.to_string()));
         }
 
-        // TODO: nslock
+        let _ns_guard = if !opts.no_lock {
+            let ns_lock = self.new_ns_lock(bucket, bucket).await?;
+            Some(
+                ns_lock
+                    .get_write_lock(get_lock_acquire_timeout())
+                    .await
+                    .map_err(|e| match e {
+                        rustfs_lock::error::LockError::QuorumNotReached { required, achieved } => {
+                            StorageError::NamespaceLockQuorumUnavailable {
+                                mode: "write",
+                                bucket: bucket.to_string(),
+                                object: bucket.to_string(),
+                                required,
+                                achieved,
+                            }
+                        }
+                        other => StorageError::other(format!("delete_bucket: failed to acquire write lock on {bucket}: {other}")),
+                    })?,
+            )
+        } else {
+            None
+        };
 
         // Check bucket exists before deletion (per S3 API spec)
         // If bucket doesn't exist, return NoSuchBucket error
@@ -124,6 +202,8 @@ impl ECStore {
             return Err(to_object_err(storage_err, vec![bucket]));
         }
 
+        validate_table_bucket_delete_guard(bucket).await?;
+
         // Check bucket is empty before deletion (per S3 API spec)
         // If bucket is not empty (contains actual objects with xl.meta files) and force
         // is not set, return BucketNotEmpty error.
@@ -160,7 +240,8 @@ impl ECStore {
 
 #[cfg(test)]
 mod tests {
-    use super::should_override_created_from_metadata;
+    use super::{should_override_created_from_metadata, validate_table_bucket_delete_allowed};
+    use crate::error::StorageError;
     use time::OffsetDateTime;
 
     #[test]
@@ -173,4 +254,13 @@ mod tests {
         let created = OffsetDateTime::from_unix_timestamp(1704067200).expect("valid timestamp");
         assert!(should_override_created_from_metadata(created));
     }
+
+    #[test]
+    fn table_bucket_delete_guard_rejects_remaining_catalog_metadata() {
+        let err = validate_table_bucket_delete_allowed("table-bucket", true, true).unwrap_err();
+
+        assert!(matches!(err, StorageError::BucketNotEmpty(bucket) if bucket == "table-bucket"));
+        assert!(validate_table_bucket_delete_allowed("table-bucket", true, false).is_ok());
+        assert!(validate_table_bucket_delete_allowed("regular-bucket", false, true).is_ok());
+    }
 }

crates/ecstore/src/store/object.rs

@@ -13,6 +13,151 @@
 // limitations under the License.
 
 use super::*;
+use crate::set_disk::{
+    get_lock_acquire_timeout, get_object_lock_diag_slow_acquire_threshold, get_object_lock_diag_slow_hold_threshold,
+    is_lock_optimization_enabled, is_object_lock_diag_enabled,
+};
+use rustfs_io_metrics::{
+    record_object_lock_diag_acquire_duration, record_object_lock_diag_hold_duration, record_object_lock_diag_slow_acquire,
+    record_object_lock_diag_slow_hold,
+};
+use std::{
+    fmt,
+    pin::Pin,
+    task::{Context, Poll},
+    time::{Duration, Instant},
+};
+use tokio::io::{AsyncRead, ReadBuf};
+
+struct LockGuardedReader {
+    inner: Box<dyn AsyncRead + Unpin + Send + Sync>,
+    guard: Option<ObjectLockDiagGuard>,
+}
+
+impl AsyncRead for LockGuardedReader {
+    fn poll_read(mut self: Pin<&mut Self>, cx: &mut Context<'_>, buf: &mut ReadBuf<'_>) -> Poll<std::io::Result<()>> {
+        let had_capacity = buf.remaining() > 0;
+        let filled_before = buf.filled().len();
+        let poll = Pin::new(&mut self.inner).poll_read(cx, buf);
+        if had_capacity && matches!(poll, Poll::Ready(Ok(()))) && buf.filled().len() == filled_before {
+            self.guard.take();
+        }
+        poll
+    }
+}
+
+#[derive(Clone, Copy, Debug)]
+enum ObjectLockDiagMode {
+    Read,
+    Write,
+}
+
+impl ObjectLockDiagMode {
+    fn as_str(self) -> &'static str {
+        match self {
+            Self::Read => "read",
+            Self::Write => "write",
+        }
+    }
+}
+
+impl fmt::Display for ObjectLockDiagMode {
+    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
+        f.write_str(self.as_str())
+    }
+}
+
+struct ObjectLockDiagGuard {
+    guard: rustfs_lock::NamespaceLockGuard,
+    enabled: bool,
+    op: &'static str,
+    bucket: Option<String>,
+    object: Option<String>,
+    owner: Option<String>,
+    mode: ObjectLockDiagMode,
+    acquired_at: Instant,
+}
+
+impl ObjectLockDiagGuard {
+    fn new(
+        guard: rustfs_lock::NamespaceLockGuard,
+        enabled: bool,
+        op: &'static str,
+        bucket: Option<String>,
+        object: Option<String>,
+        owner: Option<String>,
+        mode: ObjectLockDiagMode,
+    ) -> Self {
+        Self {
+            guard,
+            enabled,
+            op,
+            bucket,
+            object,
+            owner,
+            mode,
+            acquired_at: Instant::now(),
+        }
+    }
+}
+
+impl Drop for ObjectLockDiagGuard {
+    fn drop(&mut self) {
+        if !self.enabled || self.guard.is_released() {
+            return;
+        }
+
+        let hold = self.acquired_at.elapsed();
+        record_object_lock_diag_hold_duration(self.op, self.mode.as_str(), hold);
+        let threshold = get_object_lock_diag_slow_hold_threshold();
+        if hold >= threshold {
+            record_object_lock_diag_slow_hold(self.op, self.mode.as_str());
+            warn!(
+                target: "rustfs_ecstore::object_lock_diag",
+                op = self.op,
+                bucket = %self.bucket.as_deref().unwrap_or_default(),
+                object = %self.object.as_deref().unwrap_or_default(),
+                mode = %self.mode,
+                owner = %self.owner.as_deref().unwrap_or_default(),
+                hold_ms = hold.as_millis(),
+                threshold_ms = threshold.as_millis(),
+                "object namespace lock held longer than threshold"
+            );
+        }
+    }
+}
+
+fn log_object_lock_acquire_if_slow(
+    op: &'static str,
+    bucket: &str,
+    object: &str,
+    owner: Option<&str>,
+    mode: ObjectLockDiagMode,
+    elapsed: Duration,
+    diag_enabled: bool,
+) {
+    if !diag_enabled {
+        return;
+    }
+
+    let threshold = get_object_lock_diag_slow_acquire_threshold();
+    record_object_lock_diag_acquire_duration(op, mode.as_str(), elapsed);
+    if elapsed >= threshold {
+        record_object_lock_diag_slow_acquire(op, mode.as_str());
+        warn!(
+            target: "rustfs_ecstore::object_lock_diag",
+            op,
+            bucket,
+            object,
+            mode = %mode,
+            owner = owner.unwrap_or_default(),
+            acquire_ms = elapsed.as_millis(),
+            threshold_ms = threshold.as_millis(),
+            "object namespace lock acquisition exceeded threshold"
+        );
+    }
+}
+
 fn select_data_movement_target_pool(
     existing_pool_idx: Result<usize>,
     src_pool_idx: usize,
@@ -89,6 +234,116 @@ fn data_movement_pool_lookup_opts(opts: &ObjectOptions, no_lock: bool) -> Object
 }
 
 impl ECStore {
+    fn map_namespace_lock_error(bucket: &str, object: &str, mode: &'static str, err: rustfs_lock::LockError) -> StorageError {
+        match err {
+            rustfs_lock::LockError::QuorumNotReached { required, achieved } => StorageError::NamespaceLockQuorumUnavailable {
+                mode,
+                bucket: bucket.to_string(),
+                object: object.to_string(),
+                required,
+                achieved,
+            },
+            other => StorageError::other(format!("Failed to acquire {mode} lock on {bucket}/{object}: {other}")),
+        }
+    }
+
+    async fn acquire_object_write_lock_if_needed(
+        &self,
+        op: &'static str,
+        bucket: &str,
+        object: &str,
+        opts: &mut ObjectOptions,
+    ) -> Result<Option<ObjectLockDiagGuard>> {
+        if opts.no_lock {
+            return Ok(None);
+        }
+
+        let diag_enabled = is_object_lock_diag_enabled();
+        let ns_lock = self.handle_new_ns_lock(bucket, object).await?;
+        let acquire_start = Instant::now();
+        let guard = ns_lock
+            .get_write_lock(get_lock_acquire_timeout())
+            .await
+            .map_err(|err| Self::map_namespace_lock_error(bucket, object, "write", err))?;
+        let owner = diag_enabled.then(|| ns_lock.owner().to_string());
+        log_object_lock_acquire_if_slow(
+            op,
+            bucket,
+            object,
+            owner.as_deref(),
+            ObjectLockDiagMode::Write,
+            acquire_start.elapsed(),
+            diag_enabled,
+        );
+        opts.no_lock = true;
+
+        Ok(Some(ObjectLockDiagGuard::new(
+            guard,
+            diag_enabled,
+            op,
+            diag_enabled.then(|| bucket.to_string()),
+            diag_enabled.then(|| object.to_string()),
+            owner,
+            ObjectLockDiagMode::Write,
+        )))
+    }
+
+    async fn acquire_object_read_lock_if_needed(
+        &self,
+        op: &'static str,
+        bucket: &str,
+        object: &str,
+        opts: &mut ObjectOptions,
+    ) -> Result<Option<ObjectLockDiagGuard>> {
+        if opts.no_lock {
+            return Ok(None);
+        }
+
+        let diag_enabled = is_object_lock_diag_enabled();
+        let ns_lock = self.handle_new_ns_lock(bucket, object).await?;
+        let acquire_start = Instant::now();
+        let guard = ns_lock
+            .get_read_lock(get_lock_acquire_timeout())
+            .await
+            .map_err(|err| Self::map_namespace_lock_error(bucket, object, "read", err))?;
+        let owner = diag_enabled.then(|| ns_lock.owner().to_string());
+        log_object_lock_acquire_if_slow(
+            op,
+            bucket,
+            object,
+            owner.as_deref(),
+            ObjectLockDiagMode::Read,
+            acquire_start.elapsed(),
+            diag_enabled,
+        );
+        opts.no_lock = true;
+
+        Ok(Some(ObjectLockDiagGuard::new(
+            guard,
+            diag_enabled,
+            op,
+            diag_enabled.then(|| bucket.to_string()),
+            diag_enabled.then(|| object.to_string()),
+            owner,
+            ObjectLockDiagMode::Read,
+        )))
+    }
+
+    fn attach_read_lock_guard(mut reader: GetObjectReader, guard: Option<ObjectLockDiagGuard>) -> GetObjectReader {
+        if is_lock_optimization_enabled() {
+            return reader;
+        }
+
+        if let Some(guard) = guard {
+            reader.stream = Box::new(LockGuardedReader {
+                inner: reader.stream,
+                guard: Some(guard),
+            });
+        }
+
+        reader
+    }
+
     async fn get_latest_accessible_object_info_with_idx(
         &self,
         bucket: &str,
@@ -193,23 +448,25 @@ impl ECStore {
         check_get_obj_args(bucket, object)?;
 
         let object = encode_dir_object(object);
-
-        if self.single_pool() {
-            return self.pools[0].get_object_reader(bucket, object.as_str(), range, h, opts).await;
-        }
-
-        // TODO: nslock
-
         let mut opts = opts.clone();
-
-        opts.no_lock = true;
-
-        let (_, idx) = self
-            .get_latest_accessible_object_info_with_idx(bucket, &object, &opts)
+        let read_lock_guard = self
+            .acquire_object_read_lock_if_needed("get_object", bucket, &object, &mut opts)
             .await?;
-        self.pools[idx]
-            .get_object_reader(bucket, object.as_str(), range, h, &opts)
-            .await
+
+        let reader = if self.single_pool() {
+            self.pools[0]
+                .get_object_reader(bucket, object.as_str(), range, h, &opts)
+                .await?
+        } else {
+            let (_, idx) = self
+                .get_latest_accessible_object_info_with_idx(bucket, &object, &opts)
+                .await?;
+            self.pools[idx]
+                .get_object_reader(bucket, object.as_str(), range, h, &opts)
+                .await?
+        };
+
+        Ok(Self::attach_read_lock_guard(reader, read_lock_guard))
     }
 
     #[instrument(level = "debug", skip(self, data))]
@@ -224,6 +481,8 @@ impl ECStore {
 
         let object = encode_dir_object(object);
 
+        // Keep PUT atomic-read friendly: SetDisks takes the object write lock only
+        // around precondition checks and the final rename/commit.
         if self.single_pool() {
             return self.pools[0].put_object(bucket, object.as_str(), data, opts).await;
         }
@@ -251,16 +510,18 @@ impl ECStore {
         check_object_args(bucket, object)?;
 
         let object = encode_dir_object(object);
-
-        if self.single_pool() {
-            return self.pools[0].get_object_info(bucket, object.as_str(), opts).await;
-        }
-
-        // TODO: nslock
-
-        let (info, _) = self
-            .get_latest_accessible_object_info_with_idx(bucket, object.as_str(), opts)
+        let mut opts = opts.clone();
+        let _object_lock_guard = self
+            .acquire_object_read_lock_if_needed("get_object_info", bucket, &object, &mut opts)
             .await?;
+
+        let info = if self.single_pool() {
+            self.pools[0].get_object_info(bucket, object.as_str(), &opts).await?
+        } else {
+            self.get_latest_accessible_object_info_with_idx(bucket, object.as_str(), &opts)
+                .await?
+                .0
+        };
         opts.precondition_check(&info)?;
         Ok(info)
     }
@@ -285,43 +546,56 @@ impl ECStore {
 
         let cp_src_dst_same = path_join_buf(&[src_bucket, &src_object]) == path_join_buf(&[dst_bucket, &dst_object]);
 
-        // TODO: nslock
-
-        let pool_idx = self
-            .get_pool_info_existing_with_opts(src_bucket, &src_object, &version_aware_lookup_opts(src_opts, true))
-            .await?
-            .0
-            .index;
+        let mut dst_opts = dst_opts.clone();
+        let _dst_lock_guard = if cp_src_dst_same {
+            self.acquire_object_write_lock_if_needed("copy_object", dst_bucket, &dst_object, &mut dst_opts)
+                .await?
+        } else {
+            None
+        };
 
         if cp_src_dst_same {
+            let pool_idx = self
+                .get_pool_info_existing_with_opts(src_bucket, &src_object, &version_aware_lookup_opts(src_opts, true))
+                .await?
+                .0
+                .index;
+
             if let (Some(src_vid), Some(dst_vid)) = (&src_opts.version_id, &dst_opts.version_id)
                 && src_vid == dst_vid
             {
                 return self.pools[pool_idx]
-                    .copy_object(src_bucket, &src_object, dst_bucket, &dst_object, src_info, src_opts, dst_opts)
+                    .copy_object(src_bucket, &src_object, dst_bucket, &dst_object, src_info, src_opts, &dst_opts)
                     .await;
             }
 
             if !dst_opts.versioned && src_opts.version_id.is_none() {
                 return self.pools[pool_idx]
-                    .copy_object(src_bucket, &src_object, dst_bucket, &dst_object, src_info, src_opts, dst_opts)
+                    .copy_object(src_bucket, &src_object, dst_bucket, &dst_object, src_info, src_opts, &dst_opts)
                     .await;
             }
 
             if dst_opts.versioned && src_opts.version_id != dst_opts.version_id {
                 src_info.version_only = true;
                 return self.pools[pool_idx]
-                    .copy_object(src_bucket, &src_object, dst_bucket, &dst_object, src_info, src_opts, dst_opts)
+                    .copy_object(src_bucket, &src_object, dst_bucket, &dst_object, src_info, src_opts, &dst_opts)
                     .await;
             }
         }
 
+        let pool_idx = if dst_opts.no_lock {
+            self.get_pool_idx_no_lock(dst_bucket, &dst_object, src_info.size).await?
+        } else {
+            self.get_pool_idx(dst_bucket, &dst_object, src_info.size).await?
+        };
+
         let put_opts = ObjectOptions {
-            user_defined: src_info.user_defined.clone(),
+            user_defined: (*src_info.user_defined).clone(),
             versioned: dst_opts.versioned,
             version_id: dst_opts.version_id.clone(),
-            no_lock: true,
+            no_lock: dst_opts.no_lock,
             mod_time: dst_opts.mod_time,
+            http_preconditions: dst_opts.http_preconditions.clone(),
             ..Default::default()
         };
 
@@ -342,15 +616,29 @@ impl ECStore {
     pub(super) async fn handle_delete_object(&self, bucket: &str, object: &str, opts: ObjectOptions) -> Result<ObjectInfo> {
         check_del_obj_args(bucket, object)?;
 
-        if opts.delete_prefix {
-            self.delete_prefix(bucket, object).await?;
+        let object = if opts.delete_prefix && !opts.delete_prefix_object {
+            object.to_owned()
+        } else {
+            encode_dir_object(object)
+        };
+        let object = object.as_str();
+        let mut opts = opts;
+
+        if opts.delete_prefix && !opts.delete_prefix_object {
+            // Prefix deletes cover multiple object keys; an exact lock on the prefix string
+            // would not protect child objects.
+            self.delete_prefix(bucket, object, &opts).await?;
             return Ok(ObjectInfo::default());
         }
 
-        // TODO: nslock
+        let _object_lock_guard = self
+            .acquire_object_write_lock_if_needed("delete_object", bucket, object, &mut opts)
+            .await?;
 
-        let object = encode_dir_object(object);
-        let object = object.as_str();
+        if opts.delete_prefix {
+            self.delete_prefix(bucket, object, &opts).await?;
+            return Ok(ObjectInfo::default());
+        }
 
         let gopts = version_aware_lookup_opts(&opts, true);
 
@@ -704,7 +992,7 @@ impl ECStore {
         }
 
         let (oi, _) = self.get_latest_accessible_object_info_with_idx(bucket, &object, opts).await?;
-        Ok(oi.user_tags)
+        Ok((*oi.user_tags).clone())
     }
 
     #[instrument(level = "debug", skip(self))]
@@ -775,6 +1063,8 @@ impl ECStore {
 #[cfg(test)]
 mod tests {
     use super::*;
+    use std::io::Cursor;
+    use tokio::io::AsyncReadExt;
 
     #[test]
     fn delete_marker_data_movement_falls_back_when_only_source_pool_has_object() {
@@ -983,4 +1273,80 @@ mod tests {
         assert!(lookup_opts.skip_decommissioned);
         assert!(lookup_opts.skip_rebalancing);
     }
+
+    #[tokio::test]
+    #[serial_test::serial]
+    async fn reader_lock_is_held_when_optimization_is_disabled() {
+        temp_env::async_with_vars([(rustfs_config::ENV_OBJECT_LOCK_OPTIMIZATION_ENABLE, Some("false"))], async {
+            let manager = Arc::new(rustfs_lock::GlobalLockManager::new());
+            let lock = rustfs_lock::NamespaceLock::with_local_manager("test".to_string(), manager);
+            let key = rustfs_lock::ObjectKey::new("bucket", "object");
+            let read_guard = lock
+                .get_read_lock(key.clone(), "reader", Duration::from_secs(1))
+                .await
+                .expect("read lock should be acquired");
+            let read_guard = ObjectLockDiagGuard::new(
+                read_guard,
+                true,
+                "test_get_object",
+                Some("bucket".to_string()),
+                Some("object".to_string()),
+                Some("reader".to_string()),
+                ObjectLockDiagMode::Read,
+            );
+            let reader = GetObjectReader {
+                stream: Box::new(Cursor::new(Vec::<u8>::new())),
+                object_info: ObjectInfo::default(),
+            };
+
+            let reader = ECStore::attach_read_lock_guard(reader, Some(read_guard));
+
+            lock.get_write_lock(key.clone(), "writer", Duration::from_millis(20))
+                .await
+                .expect_err("reader should hold the read lock");
+            drop(reader);
+            lock.get_write_lock(key, "writer", Duration::from_secs(1))
+                .await
+                .expect("dropping the reader should release the read lock");
+        })
+        .await;
+    }
+
+    #[tokio::test]
+    #[serial_test::serial]
+    async fn reader_lock_is_released_after_stream_eof() {
+        temp_env::async_with_vars([(rustfs_config::ENV_OBJECT_LOCK_OPTIMIZATION_ENABLE, Some("false"))], async {
+            let manager = Arc::new(rustfs_lock::GlobalLockManager::new());
+            let lock = rustfs_lock::NamespaceLock::with_local_manager("test".to_string(), manager);
+            let key = rustfs_lock::ObjectKey::new("bucket", "object");
+            let read_guard = lock
+                .get_read_lock(key.clone(), "reader", Duration::from_secs(1))
+                .await
+                .expect("read lock should be acquired");
+            let read_guard = ObjectLockDiagGuard::new(
+                read_guard,
+                true,
+                "test_get_object",
+                Some("bucket".to_string()),
+                Some("object".to_string()),
+                Some("reader".to_string()),
+                ObjectLockDiagMode::Read,
+            );
+            let reader = GetObjectReader {
+                stream: Box::new(Cursor::new(vec![1, 2, 3])),
+                object_info: ObjectInfo::default(),
+            };
+
+            let mut reader = ECStore::attach_read_lock_guard(reader, Some(read_guard));
+            let mut output = Vec::new();
+            reader.stream.read_to_end(&mut output).await.expect("reader should reach EOF");
+            assert_eq!(output, vec![1, 2, 3]);
+
+            lock.get_write_lock(key, "writer", Duration::from_secs(1))
+                .await
+                .expect("EOF should release the read lock before the reader is dropped");
+            drop(reader);
+        })
+        .await;
+    }
 }

crates/ecstore/src/store/rebalance.rs

@@ -13,6 +13,7 @@
 // limitations under the License.
 
 use super::*;
+use crate::config::get_global_storage_class;
 
 struct LatestObjectInfoCandidate {
     info: Option<ObjectInfo>,
@@ -182,17 +183,11 @@ impl ECStore {
         Ok(())
     }
 
-    pub(super) async fn delete_prefix(&self, bucket: &str, object: &str) -> Result<()> {
+    pub(super) async fn delete_prefix(&self, bucket: &str, object: &str, opts: &ObjectOptions) -> Result<()> {
         for pool in self.pools.iter() {
-            pool.delete_object(
-                bucket,
-                object,
-                ObjectOptions {
-                    delete_prefix: true,
-                    ..Default::default()
-                },
-            )
-            .await?;
+            let mut opts = opts.clone();
+            opts.delete_prefix = true;
+            pool.delete_object(bucket, object, opts).await?;
         }
 
         Ok(())
@@ -605,7 +600,7 @@ impl ECStore {
     #[instrument(skip(self))]
     pub(super) async fn handle_backend_info(&self) -> rustfs_madmin::BackendInfo {
         let (standard_sc_parity, rr_sc_parity) = {
-            if let Some(sc) = GLOBAL_STORAGE_CLASS.get() {
+            if let Some(sc) = get_global_storage_class() {
                 let sc_parity = sc
                     .get_parity_for_sc(storageclass::CLASS_STANDARD)
                     .or(Some(self.pools[0].default_parity_count));

crates/ecstore/src/store_api/readers.rs

@@ -652,12 +652,20 @@ fn multipart_part_numbers(parts: &[rustfs_filemeta::ObjectPartInfo]) -> Vec<usiz
     parts.iter().map(|part| part.number).collect()
 }
 
+fn metadata_get<'a>(metadata: &'a HashMap<String, String>, key: &str) -> Option<&'a str> {
+    metadata.get(key).map(String::as_str).or_else(|| {
+        metadata
+            .iter()
+            .find_map(|(candidate, value)| candidate.eq_ignore_ascii_case(key).then_some(value.as_str()))
+    })
+}
+
 async fn resolve_encryption_material(oi: &ObjectInfo, headers: &HeaderMap<HeaderValue>) -> Result<EncryptionMaterial> {
-    if oi.user_defined.contains_key(SSEC_ALGORITHM_HEADER) {
+    if metadata_get(&oi.user_defined, SSEC_ALGORITHM_HEADER).is_some() {
         return resolve_ssec_material(oi, headers);
     }
 
-    if oi.user_defined.contains_key(INTERNAL_ENCRYPTION_KEY_HEADER) {
+    if metadata_get(&oi.user_defined, INTERNAL_ENCRYPTION_KEY_HEADER).is_some() {
         return resolve_managed_material(&oi.user_defined).await;
     }
 
@@ -697,11 +705,9 @@ fn resolve_ssec_material(oi: &ObjectInfo, headers: &HeaderMap<HeaderValue>) -> R
         return Err(Error::other("SSE-C key MD5 mismatch"));
     }
 
-    let stored_md5 = oi
-        .user_defined
-        .get(SSEC_KEY_MD5_HEADER)
-        .ok_or_else(|| Error::other("missing stored SSE-C key md5"))?;
-    if stored_md5 != &expected_md5 {
+    let stored_md5 =
+        metadata_get(&oi.user_defined, SSEC_KEY_MD5_HEADER).ok_or_else(|| Error::other("missing stored SSE-C key md5"))?;
+    if stored_md5 != expected_md5 {
         return Err(Error::other("SSE-C key does not match object metadata"));
     }
 
@@ -712,16 +718,14 @@ fn resolve_ssec_material(oi: &ObjectInfo, headers: &HeaderMap<HeaderValue>) -> R
 }
 
 async fn resolve_managed_material(metadata: &HashMap<String, String>) -> Result<EncryptionMaterial> {
-    let encrypted_dek = metadata
-        .get(INTERNAL_ENCRYPTION_KEY_HEADER)
-        .ok_or_else(|| Error::other("missing managed encrypted DEK"))?;
+    let encrypted_dek =
+        metadata_get(metadata, INTERNAL_ENCRYPTION_KEY_HEADER).ok_or_else(|| Error::other("missing managed encrypted DEK"))?;
     let encrypted_dek = BASE64_STANDARD
         .decode(encrypted_dek)
         .map_err(|e| Error::other(format!("failed to decode managed encrypted DEK: {e}")))?;
 
-    let iv_b64 = metadata
-        .get(INTERNAL_ENCRYPTION_IV_HEADER)
-        .ok_or_else(|| Error::other("missing managed encryption IV"))?;
+    let iv_b64 =
+        metadata_get(metadata, INTERNAL_ENCRYPTION_IV_HEADER).ok_or_else(|| Error::other("missing managed encryption IV"))?;
     let iv = BASE64_STANDARD
         .decode(iv_b64)
         .map_err(|e| Error::other(format!("failed to decode managed encryption IV: {e}")))?;
@@ -730,10 +734,7 @@ async fn resolve_managed_material(metadata: &HashMap<String, String>) -> Result<
         .try_into()
         .map_err(|_| Error::other("managed encryption IV must be 12 bytes"))?;
 
-    let kms_key_id = metadata
-        .get(INTERNAL_ENCRYPTION_KEY_ID_HEADER)
-        .map(String::as_str)
-        .unwrap_or("default");
+    let kms_key_id = metadata_get(metadata, INTERNAL_ENCRYPTION_KEY_ID_HEADER).unwrap_or("default");
 
     let key_bytes = if let Some(service) = get_global_encryption_service().await {
         service
@@ -960,7 +961,7 @@ mod tests {
     fn test_http_range_spec_from_object_info_valid_and_invalid_parts() {
         let object_info = ObjectInfo {
             size: 300,
-            parts: vec![
+            parts: Arc::new(vec![
                 ObjectPartInfo {
                     etag: String::new(),
                     number: 1,
@@ -982,7 +983,7 @@ mod tests {
                     actual_size: 100,
                     ..Default::default()
                 },
-            ],
+            ]),
             ..Default::default()
         };
 
@@ -998,7 +999,7 @@ mod tests {
     fn test_http_range_spec_from_object_info_uses_actual_size() {
         let object_info = ObjectInfo {
             size: 90,
-            parts: vec![
+            parts: Arc::new(vec![
                 ObjectPartInfo {
                     etag: String::new(),
                     number: 1,
@@ -1020,7 +1021,7 @@ mod tests {
                     actual_size: 50,
                     ..Default::default()
                 },
-            ],
+            ]),
             ..Default::default()
         };
 
@@ -1033,7 +1034,7 @@ mod tests {
     fn test_http_range_spec_from_object_info_falls_back_to_part_size_when_actual_size_missing() {
         let object_info = ObjectInfo {
             size: 90,
-            parts: vec![
+            parts: Arc::new(vec![
                 ObjectPartInfo {
                     etag: String::new(),
                     number: 1,
@@ -1055,7 +1056,7 @@ mod tests {
                     actual_size: 0,
                     ..Default::default()
                 },
-            ],
+            ]),
             ..Default::default()
         };
 
@@ -1122,14 +1123,59 @@ mod tests {
         format!("{}:{}", BASE64_STANDARD.encode(nonce), BASE64_STANDARD.encode(ciphertext))
     }
 
+    #[tokio::test]
+    async fn resolve_managed_material_accepts_case_insensitive_metadata_keys() {
+        async_with_vars([("__RUSTFS_SSE_SIMPLE_CMK", Some(BASE64_STANDARD.encode([0u8; 32])))], async {
+            let data_key = [0x24; 32];
+            let base_nonce = [0x14; 12];
+            let encrypted_dek = encrypt_managed_dek_for_test(data_key, [0u8; 32]);
+            let metadata = HashMap::from([
+                ("X-Rustfs-Encryption-Key".to_string(), BASE64_STANDARD.encode(encrypted_dek.as_bytes())),
+                ("X-Rustfs-Encryption-IV".to_string(), BASE64_STANDARD.encode(base_nonce)),
+            ]);
+
+            let material = resolve_managed_material(&metadata)
+                .await
+                .expect("managed material should resolve mixed-case metadata keys");
+
+            assert_eq!(material.key_bytes, data_key);
+            assert_eq!(material.base_nonce, base_nonce);
+        })
+        .await;
+    }
+
+    #[tokio::test]
+    async fn resolve_encryption_material_accepts_case_insensitive_metadata_keys() {
+        async_with_vars([("__RUSTFS_SSE_SIMPLE_CMK", Some(BASE64_STANDARD.encode([0u8; 32])))], async {
+            let data_key = [0x24; 32];
+            let base_nonce = [0x14; 12];
+            let encrypted_dek = encrypt_managed_dek_for_test(data_key, [0u8; 32]);
+            let metadata = HashMap::from([
+                ("X-Rustfs-Encryption-Key".to_string(), BASE64_STANDARD.encode(encrypted_dek.as_bytes())),
+                ("X-Rustfs-Encryption-IV".to_string(), BASE64_STANDARD.encode(base_nonce)),
+            ]);
+            let object_info = ObjectInfo {
+                user_defined: Arc::new(metadata),
+                ..Default::default()
+            };
+            let material = resolve_encryption_material(&object_info, &HeaderMap::new())
+                .await
+                .expect("resolve_encryption_material should accept mixed-case managed metadata");
+
+            assert_eq!(material.key_bytes, data_key);
+            assert_eq!(material.base_nonce, base_nonce);
+        })
+        .await;
+    }
+
     #[tokio::test]
     async fn test_get_object_reader_rejects_ssec_read_without_headers() {
         let object_info = ObjectInfo {
             size: 10,
-            user_defined: HashMap::from([
+            user_defined: Arc::new(HashMap::from([
                 ("x-amz-server-side-encryption-customer-algorithm".to_string(), "AES256".to_string()),
                 ("x-amz-server-side-encryption-customer-original-size".to_string(), "20".to_string()),
-            ]),
+            ])),
             ..Default::default()
         };
 
@@ -1155,10 +1201,10 @@ mod tests {
     async fn test_get_object_reader_restore_request_bypasses_encryption_range_rewrite() {
         let object_info = ObjectInfo {
             size: 10,
-            user_defined: HashMap::from([
+            user_defined: Arc::new(HashMap::from([
                 ("x-rustfs-encryption-key".to_string(), "encrypted-key".to_string()),
                 ("x-rustfs-encryption-original-size".to_string(), "20".to_string()),
-            ]),
+            ])),
             ..Default::default()
         };
 
@@ -1201,12 +1247,12 @@ mod tests {
 
             let object_info = ObjectInfo {
                 size: encrypted.len() as i64,
-                user_defined: HashMap::from([
+                user_defined: Arc::new(HashMap::from([
                     ("x-amz-server-side-encryption".to_string(), "AES256".to_string()),
                     ("x-rustfs-encryption-key".to_string(), BASE64_STANDARD.encode(encrypted_dek.as_bytes())),
                     ("x-rustfs-encryption-iv".to_string(), BASE64_STANDARD.encode(base_nonce)),
                     ("x-rustfs-encryption-original-size".to_string(), plaintext.len().to_string()),
-                ]),
+                ])),
                 ..Default::default()
             };
 
@@ -1247,12 +1293,12 @@ mod tests {
 
             let object_info = ObjectInfo {
                 size: encrypted.len() as i64,
-                user_defined: HashMap::from([
+                user_defined: Arc::new(HashMap::from([
                     ("x-amz-server-side-encryption".to_string(), "AES256".to_string()),
                     ("x-rustfs-encryption-key".to_string(), BASE64_STANDARD.encode(encrypted_dek.as_bytes())),
                     ("x-rustfs-encryption-iv".to_string(), BASE64_STANDARD.encode(base_nonce)),
                     ("x-rustfs-encryption-original-size".to_string(), plaintext.len().to_string()),
-                ]),
+                ])),
                 ..Default::default()
             };
             let range = HTTPRangeSpec {
@@ -1303,12 +1349,12 @@ mod tests {
 
                 let object_info = ObjectInfo {
                     size: encrypted.len() as i64,
-                    user_defined: HashMap::from([
+                    user_defined: Arc::new(HashMap::from([
                         ("x-amz-server-side-encryption".to_string(), "AES256".to_string()),
                         ("x-rustfs-encryption-key".to_string(), BASE64_STANDARD.encode(encrypted_dek.as_bytes())),
                         ("x-rustfs-encryption-iv".to_string(), BASE64_STANDARD.encode(base_nonce)),
                         ("x-rustfs-encryption-original-size".to_string(), plaintext.len().to_string()),
-                    ]),
+                    ])),
                     ..Default::default()
                 };
 
@@ -1340,18 +1386,18 @@ mod tests {
 
         let object_info = ObjectInfo {
             size: 3_000_000,
-            parts: vec![ObjectPartInfo {
+            parts: Arc::new(vec![ObjectPartInfo {
                 etag: String::new(),
                 number: 1,
                 size: 3_000_000,
                 actual_size: 4_194_304,
                 index: Some(index.into_vec()),
                 ..Default::default()
-            }],
-            user_defined: HashMap::from([
+            }]),
+            user_defined: Arc::new(HashMap::from([
                 ("x-minio-internal-compression".to_string(), "gzip".to_string()),
                 ("x-minio-internal-actual-size".to_string(), "4194304".to_string()),
-            ]),
+            ])),
             ..Default::default()
         };
 
@@ -1397,7 +1443,7 @@ mod tests {
             bucket: bucket.to_string(),
             name: object.to_string(),
             size: encrypted.len() as i64,
-            user_defined: HashMap::from([
+            user_defined: Arc::new(HashMap::from([
                 ("x-amz-server-side-encryption-customer-algorithm".to_string(), "AES256".to_string()),
                 (
                     "x-amz-server-side-encryption-customer-key-md5".to_string(),
@@ -1407,7 +1453,7 @@ mod tests {
                     "x-amz-server-side-encryption-customer-original-size".to_string(),
                     plaintext.len().to_string(),
                 ),
-            ]),
+            ])),
             ..Default::default()
         };
 
@@ -1450,7 +1496,7 @@ mod tests {
             bucket: bucket.to_string(),
             name: object.to_string(),
             size: encrypted.len() as i64,
-            user_defined: HashMap::from([
+            user_defined: Arc::new(HashMap::from([
                 ("x-amz-server-side-encryption-customer-algorithm".to_string(), "AES256".to_string()),
                 (
                     "x-amz-server-side-encryption-customer-key-md5".to_string(),
@@ -1460,7 +1506,7 @@ mod tests {
                     "x-amz-server-side-encryption-customer-original-size".to_string(),
                     plaintext.len().to_string(),
                 ),
-            ]),
+            ])),
             ..Default::default()
         };
         let range = HTTPRangeSpec {
@@ -1514,7 +1560,7 @@ mod tests {
             bucket: bucket.to_string(),
             name: object.to_string(),
             size: encrypted.len() as i64,
-            user_defined: HashMap::from([
+            user_defined: Arc::new(HashMap::from([
                 ("x-amz-server-side-encryption-customer-algorithm".to_string(), "AES256".to_string()),
                 (
                     "x-amz-server-side-encryption-customer-key-md5".to_string(),
@@ -1526,7 +1572,7 @@ mod tests {
                 ),
                 ("x-minio-internal-compression".to_string(), CompressionAlgorithm::default().to_string()),
                 ("x-minio-internal-actual-size".to_string(), plaintext.len().to_string()),
-            ]),
+            ])),
             ..Default::default()
         };
         let range = HTTPRangeSpec {

crates/ecstore/src/store_api/types.rs

@@ -293,7 +293,7 @@ pub struct ObjectInfo {
     // Actual size is the real size of the object uploaded by client.
     pub actual_size: i64,
     pub is_dir: bool,
-    pub user_defined: HashMap<String, String>,
+    pub user_defined: Arc<HashMap<String, String>>,
     pub parity_blocks: usize,
     pub data_blocks: usize,
     pub version_id: Option<Uuid>,
@@ -301,8 +301,8 @@ pub struct ObjectInfo {
     pub transitioned_object: TransitionedObject,
     pub restore_ongoing: bool,
     pub restore_expires: Option<OffsetDateTime>,
-    pub user_tags: String,
-    pub parts: Vec<ObjectPartInfo>,
+    pub user_tags: Arc<String>,
+    pub parts: Arc<Vec<ObjectPartInfo>>,
     pub is_latest: bool,
     pub content_type: Option<String>,
     pub content_encoding: Option<String>,
@@ -387,17 +387,21 @@ impl ObjectInfo {
         // Corresponding to the logic in rustfs/src/sse.rs/encryption_material_to_metadata function
         use rustfs_utils::http::{SSEC_ALGORITHM_HEADER, SSEC_KEY_HEADER, SSEC_KEY_MD5_HEADER};
 
-        self.user_defined
-            .keys()
-            .any(|key| key.to_lowercase().starts_with("x-minio-encryption-")) // Covers MinIO metadata
-            || self.user_defined.contains_key("x-rustfs-encryption-key") // SSE-S3/SSE-KMS
-            || self.user_defined.contains_key("x-rustfs-encryption-algorithm") // SSE-S3/SSE-KMS
-            || self.user_defined.contains_key("x-rustfs-encryption-iv") // SSE-S3/SSE-KMS
-            || self.user_defined.contains_key("x-amz-server-side-encryption-aws-kms-key-id") // SSE-KMS
-            || self.user_defined.contains_key(SSEC_ALGORITHM_HEADER) // SSE-C
-            || self.user_defined.contains_key(SSEC_KEY_HEADER) // SSE-C
-            || self.user_defined.contains_key(SSEC_KEY_MD5_HEADER) // SSE-C
-            || self.user_defined.contains_key("x-amz-server-side-encryption") // SSE-S3/SSE-KMS/SSE-C
+        self.user_defined.keys().any(|key| {
+            let key = key.to_lowercase();
+            key.starts_with("x-minio-encryption-")
+                || matches!(
+                    key.as_str(),
+                    "x-rustfs-encryption-key"
+                        | "x-rustfs-encryption-algorithm"
+                        | "x-rustfs-encryption-iv"
+                        | "x-amz-server-side-encryption-aws-kms-key-id"
+                        | SSEC_ALGORITHM_HEADER
+                        | SSEC_KEY_HEADER
+                        | SSEC_KEY_MD5_HEADER
+                        | "x-amz-server-side-encryption"
+                )
+        })
     }
 
     pub fn encryption_original_size(&self) -> std::io::Result<Option<i64>> {
@@ -473,7 +477,11 @@ impl ObjectInfo {
         };
 
         // tags
-        let user_tags = fi.metadata.get(AMZ_OBJECT_TAGGING).cloned().unwrap_or_default();
+        let user_tags: Arc<String> = fi
+            .metadata
+            .get(AMZ_OBJECT_TAGGING)
+            .map(|s| Arc::new(s.clone()))
+            .unwrap_or_default();
 
         let inlined = fi.inline_data();
 
@@ -567,7 +575,7 @@ impl ObjectInfo {
                 number: part.number,
                 error: part.error.clone(),
             })
-            .collect();
+            .collect::<Vec<_>>();
 
         // TODO: part checksums
 
@@ -581,7 +589,7 @@ impl ObjectInfo {
             delete_marker: fi.deleted,
             mod_time: fi.mod_time,
             size: fi.size,
-            parts,
+            parts: Arc::new(parts),
             is_latest: fi.is_latest,
             user_tags,
             content_type,
@@ -591,7 +599,7 @@ impl ObjectInfo {
             successor_mod_time: fi.successor_mod_time,
             etag,
             inlined,
-            user_defined: metadata,
+            user_defined: Arc::new(metadata),
             transitioned_object,
             checksum: fi.checksum.clone(),
             storage_class,
@@ -611,11 +619,12 @@ impl ObjectInfo {
         bucket: &str,
         prefix: &str,
         delimiter: Option<String>,
-        after_version_id: Option<Uuid>,
+        after_version_marker: Option<VersionMarker>,
     ) -> Vec<ObjectInfo> {
         let vcfg = get_versioning_config(bucket).await.ok();
         let mut objects = Vec::with_capacity(entries.entries().len());
         let mut prev_prefix = "";
+        let mut after_version_marker = after_version_marker;
         for entry in entries.entries() {
             if entry.is_object() {
                 if let Some(delimiter) = &delimiter {
@@ -652,12 +661,8 @@ impl ObjectInfo {
                     }
                 };
 
-                let versions = if let Some(vid) = after_version_id {
-                    if let Some(idx) = file_infos.find_version_index(vid) {
-                        &file_infos.versions[idx + 1..]
-                    } else {
-                        &file_infos.versions
-                    }
+                let versions = if let Some(marker) = after_version_marker.take() {
+                    versions_after_marker(&file_infos, marker)
                 } else {
                     &file_infos.versions
                 };
@@ -829,6 +834,23 @@ impl ObjectInfo {
     }
 }
 
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+pub enum VersionMarker {
+    Null,
+    Version(Uuid),
+}
+
+fn versions_after_marker(file_infos: &rustfs_filemeta::FileInfoVersions, marker: VersionMarker) -> &[FileInfo] {
+    let marker_idx = match marker {
+        VersionMarker::Null => file_infos.versions.iter().position(|version| version.version_id.is_none()),
+        VersionMarker::Version(vid) => file_infos.find_version_index(vid),
+    };
+
+    marker_idx
+        .map(|idx| &file_infos.versions[idx + 1..])
+        .unwrap_or(&file_infos.versions)
+}
+
 #[derive(Debug, Default)]
 pub struct ListObjectsInfo {
     // Indicates whether the returned list objects response is truncated. A
@@ -1072,6 +1094,100 @@ mod tests {
     use super::*;
     use rustfs_filemeta::ReplicationState;
 
+    #[test]
+    fn versions_after_marker_handles_null_version_marker() {
+        let first_version = Uuid::parse_str("11111111-2222-3333-4444-555555555555").unwrap();
+        let last_version = Uuid::parse_str("aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee").unwrap();
+        let file_infos = rustfs_filemeta::FileInfoVersions {
+            versions: vec![
+                FileInfo {
+                    version_id: Some(first_version),
+                    ..Default::default()
+                },
+                FileInfo {
+                    version_id: None,
+                    ..Default::default()
+                },
+                FileInfo {
+                    version_id: Some(last_version),
+                    ..Default::default()
+                },
+            ],
+            ..Default::default()
+        };
+
+        let versions = versions_after_marker(&file_infos, VersionMarker::Null);
+
+        assert_eq!(versions.len(), 1);
+        assert_eq!(versions[0].version_id, Some(last_version));
+    }
+
+    #[test]
+    fn versions_after_marker_handles_uuid_version_marker() {
+        let first_version = Uuid::parse_str("11111111-2222-3333-4444-555555555555").unwrap();
+        let last_version = Uuid::parse_str("aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee").unwrap();
+        let file_infos = rustfs_filemeta::FileInfoVersions {
+            versions: vec![
+                FileInfo {
+                    version_id: Some(first_version),
+                    ..Default::default()
+                },
+                FileInfo {
+                    version_id: None,
+                    ..Default::default()
+                },
+                FileInfo {
+                    version_id: Some(last_version),
+                    ..Default::default()
+                },
+            ],
+            ..Default::default()
+        };
+
+        let versions = versions_after_marker(&file_infos, VersionMarker::Version(first_version));
+
+        assert_eq!(versions.len(), 2);
+        assert_eq!(versions[0].version_id, None);
+        assert_eq!(versions[1].version_id, Some(last_version));
+    }
+
+    #[tokio::test]
+    async fn versions_listing_applies_version_marker_only_to_first_entry() {
+        let metadata = rustfs_filemeta::test_data::create_real_xlmeta().expect("test metadata should be valid");
+        let entries = rustfs_filemeta::MetaCacheEntriesSorted {
+            o: rustfs_filemeta::MetaCacheEntries(vec![
+                Some(rustfs_filemeta::MetaCacheEntry {
+                    name: "obj-a".to_owned(),
+                    metadata: metadata.clone(),
+                    ..Default::default()
+                }),
+                Some(rustfs_filemeta::MetaCacheEntry {
+                    name: "obj-b".to_owned(),
+                    metadata,
+                    ..Default::default()
+                }),
+            ]),
+            ..Default::default()
+        };
+        let marker_version = Uuid::parse_str("11111111-2222-3333-4444-555555555555").unwrap();
+
+        let objects = ObjectInfo::from_meta_cache_entries_sorted_versions(
+            &entries,
+            "bucket",
+            "",
+            None,
+            Some(VersionMarker::Version(marker_version)),
+        )
+        .await;
+
+        let obj_a_count = objects.iter().filter(|object| object.name == "obj-a").count();
+        let obj_b_count = objects.iter().filter(|object| object.name == "obj-b").count();
+
+        assert_eq!(obj_a_count, 2);
+        assert_eq!(obj_b_count, 3);
+        assert_eq!(objects.len(), 5);
+    }
+
     #[test]
     fn get_actual_size_prefers_actual_size_field() {
         let info = ObjectInfo {
@@ -1095,7 +1211,7 @@ mod tests {
         let info = ObjectInfo {
             size: 100,
             actual_size: 0,
-            user_defined,
+            user_defined: Arc::new(user_defined),
             ..Default::default()
         };
 
@@ -1113,7 +1229,7 @@ mod tests {
         let info = ObjectInfo {
             size: 100,
             actual_size: 0,
-            user_defined,
+            user_defined: Arc::new(user_defined),
             ..Default::default()
         };
 
@@ -1165,8 +1281,8 @@ mod tests {
         let info = ObjectInfo {
             size: 12,
             actual_size: 0,
-            user_defined,
-            parts: vec![
+            user_defined: Arc::new(user_defined),
+            parts: Arc::new(vec![
                 rustfs_filemeta::ObjectPartInfo {
                     actual_size: 4,
                     ..Default::default()
@@ -1175,7 +1291,7 @@ mod tests {
                     actual_size: 5,
                     ..Default::default()
                 },
-            ],
+            ]),
             ..Default::default()
         };
 
@@ -1193,7 +1309,7 @@ mod tests {
         let info = ObjectInfo {
             size: 12,
             actual_size: 0,
-            user_defined,
+            user_defined: Arc::new(user_defined),
             ..Default::default()
         };
 
@@ -1217,7 +1333,7 @@ mod tests {
         });
 
         let info = ObjectInfo {
-            user_defined,
+            user_defined: Arc::new(user_defined),
             ..Default::default()
         };
 
@@ -1247,10 +1363,85 @@ mod tests {
         });
 
         let info = ObjectInfo {
-            user_defined,
+            user_defined: Arc::new(user_defined),
             ..Default::default()
         };
 
         assert!(info.is_encrypted());
     }
+
+    #[test]
+    fn is_encrypted_handles_case_insensitive_rustfs_metadata_keys() {
+        let mut user_defined: HashMap<String, String> = HashMap::new();
+        user_defined.insert("X-Rustfs-Encryption-Key".to_string(), "encrypted-key".to_string());
+
+        let info = ObjectInfo {
+            user_defined: Arc::new(user_defined),
+            ..Default::default()
+        };
+
+        assert!(info.is_encrypted());
+    }
+
+    #[test]
+    fn objectinfo_clone_shares_arc_data_and_is_correct() {
+        let mut ud = HashMap::new();
+        ud.insert("content-type".to_string(), "application/octet-stream".to_string());
+        ud.insert("x-custom-header".to_string(), "custom-value".to_string());
+
+        let original = ObjectInfo {
+            bucket: "test-bucket".to_string(),
+            name: "test-object".to_string(),
+            user_defined: Arc::new(ud),
+            user_tags: Arc::new("env=prod&team=storage".to_string()),
+            parts: Arc::new(vec![
+                rustfs_filemeta::ObjectPartInfo {
+                    number: 1,
+                    size: 1024,
+                    actual_size: 1024,
+                    ..Default::default()
+                },
+                rustfs_filemeta::ObjectPartInfo {
+                    number: 2,
+                    size: 512,
+                    actual_size: 512,
+                    ..Default::default()
+                },
+            ]),
+            size: 1536,
+            etag: Some("abc123".to_string()),
+            ..Default::default()
+        };
+
+        let cloned = original.clone();
+
+        // Verify cloned values are correct
+        assert_eq!(cloned.bucket, "test-bucket");
+        assert_eq!(cloned.name, "test-object");
+        assert_eq!(cloned.size, 1536);
+        assert_eq!(cloned.etag, Some("abc123".to_string()));
+
+        // Verify Arc fields share the same allocation
+        assert!(Arc::ptr_eq(&original.user_defined, &cloned.user_defined));
+        assert!(Arc::ptr_eq(&original.user_tags, &cloned.user_tags));
+        assert!(Arc::ptr_eq(&original.parts, &cloned.parts));
+
+        // Verify Arc-wrapped data is accessible through the clone
+        assert_eq!(
+            cloned.user_defined.get("content-type").map(String::as_str),
+            Some("application/octet-stream")
+        );
+        assert_eq!(cloned.user_tags.as_str(), "env=prod&team=storage");
+        assert_eq!(cloned.parts.len(), 2);
+        assert_eq!(cloned.parts[0].number, 1);
+        assert_eq!(cloned.parts[1].size, 512);
+
+        // Verify default ObjectInfo clone also works
+        let default_obj = ObjectInfo::default();
+        let default_cloned = default_obj.clone();
+        assert!(default_obj.user_defined.is_empty());
+        assert!(default_cloned.user_defined.is_empty());
+        assert!(default_cloned.user_tags.is_empty());
+        assert!(default_cloned.parts.is_empty());
+    }
 }

crates/ecstore/src/store_list_objects.rs

@@ -23,8 +23,8 @@ use crate::error::{
 };
 use crate::set_disk::SetDisks;
 use crate::store_api::{
-    ListObjectVersionsInfo, ListObjectsInfo, ObjectInfo, ObjectInfoOrErr, ObjectOperations, ObjectOptions, WalkOptions,
-    WalkVersionsSortOrder,
+    ListObjectVersionsInfo, ListObjectsInfo, ObjectInfo, ObjectInfoOrErr, ObjectOperations, ObjectOptions, VersionMarker,
+    WalkOptions, WalkVersionsSortOrder,
 };
 use crate::store_utils::is_reserved_or_invalid_bucket;
 use crate::{store::ECStore, store_api::ListObjectsV2Info};
@@ -118,9 +118,12 @@ pub struct ListPathOptions {
     pub filter_prefix: Option<String>,
 
     // Marker to resume listing.
-    // The response will be the first entry >= this object name.
     pub marker: Option<String>,
 
+    // Include marker itself in the returned entries. Version listings need this
+    // when a version marker selects a later version from the marker object.
+    pub include_marker: bool,
+
     // Limit the number of results.
     pub limit: i32,
 
@@ -159,6 +162,67 @@ pub struct ListPathOptions {
 }
 
 const MARKER_TAG_VERSION: &str = "v1";
+const ENV_API_LIST_QUORUM: &str = "RUSTFS_API_LIST_QUORUM";
+const DEFAULT_API_LIST_QUORUM: &str = "strict";
+
+fn normalize_list_quorum(value: &str) -> &'static str {
+    let value = value.trim();
+    if value.eq_ignore_ascii_case("disk") {
+        "disk"
+    } else if value.eq_ignore_ascii_case("reduced") {
+        "reduced"
+    } else if value.eq_ignore_ascii_case("optimal") {
+        "optimal"
+    } else if value.eq_ignore_ascii_case("auto") {
+        "auto"
+    } else {
+        DEFAULT_API_LIST_QUORUM
+    }
+}
+
+fn list_quorum_from_env() -> String {
+    let value = rustfs_utils::get_env_str(ENV_API_LIST_QUORUM, DEFAULT_API_LIST_QUORUM);
+    normalize_list_quorum(&value).to_owned()
+}
+
+fn list_metadata_resolution_params(bucket: String, listing_quorum: usize, versioned: bool) -> MetadataResolutionParams {
+    let mut resolver = MetadataResolutionParams {
+        dir_quorum: listing_quorum,
+        obj_quorum: listing_quorum,
+        bucket,
+        ..Default::default()
+    };
+
+    if !versioned {
+        resolver.requested_versions = 1;
+    }
+
+    resolver
+}
+
+fn parse_version_marker(marker: String) -> Result<VersionMarker> {
+    if marker == "null" {
+        Ok(VersionMarker::Null)
+    } else {
+        Ok(VersionMarker::Version(Uuid::parse_str(&marker)?))
+    }
+}
+
+fn version_marker_for_entries(
+    entries: Option<&MetaCacheEntriesSorted>,
+    key_marker: Option<&str>,
+    version_marker: Option<VersionMarker>,
+) -> Option<VersionMarker> {
+    let marker = version_marker?;
+    let Some(key_marker) = key_marker else {
+        return Some(marker);
+    };
+
+    entries
+        .and_then(|entries| entries.entries().first().map(|entry| entry.name.as_str() == key_marker))
+        .unwrap_or_default()
+        .then_some(marker)
+}
 
 impl ListPathOptions {
     pub fn set_filter(&mut self) {
@@ -182,50 +246,64 @@ impl ListPathOptions {
     }
 
     pub fn parse_marker(&mut self) {
-        if let Some(marker) = &self.marker {
-            let s = marker.clone();
-            if !s.contains(format!("[rustfs_cache:{MARKER_TAG_VERSION}").as_str()) {
-                return;
-            }
+        let Some(marker) = self.marker.clone() else {
+            return;
+        };
+        let Some(start_idx) = marker.rfind("[rustfs_cache:") else {
+            return;
+        };
+        let Some(end_offset) = marker[start_idx..].rfind(']') else {
+            return;
+        };
 
-            if let (Some(start_idx), Some(end_idx)) = (s.find("["), s.find("]")) {
-                self.marker = Some(s[0..start_idx].to_owned());
-                let tags: Vec<_> = s[start_idx..end_idx].trim_matches(['[', ']']).split(",").collect();
+        let end_idx = start_idx + end_offset;
+        let tag_body = marker[start_idx + 1..end_idx].to_owned();
+        let mut supported_marker = false;
 
-                for &tag in tags.iter() {
-                    let kv: Vec<_> = tag.split(":").collect();
-                    if kv.len() != 2 {
-                        continue;
-                    }
-
-                    match kv[0] {
-                        "rustfs_cache" if kv[1] != MARKER_TAG_VERSION => {
-                            continue;
-                        }
-                        "id" => self.id = Some(kv[1].to_owned()),
-                        "return" => {
-                            self.id = Some(Uuid::new_v4().to_string());
-                            self.create = true;
-                        }
-                        "p" => match kv[1].parse::<usize>() {
-                            Ok(res) => self.pool_idx = Some(res),
-                            Err(_) => {
-                                self.id = Some(Uuid::new_v4().to_string());
-                                self.create = true;
-                                continue;
-                            }
-                        },
-                        "s" => match kv[1].parse::<usize>() {
-                            Ok(res) => self.set_idx = Some(res),
-                            Err(_) => {
-                                self.id = Some(Uuid::new_v4().to_string());
-                                self.create = true;
-                                continue;
-                            }
-                        },
-                        _ => (),
-                    }
+        for tag in tag_body.split(',') {
+            let Some((key, value)) = tag.split_once(':') else {
+                continue;
+            };
+            if key == "rustfs_cache" {
+                if value != MARKER_TAG_VERSION {
+                    return;
                 }
+                supported_marker = true;
+            }
+        }
+
+        if !supported_marker {
+            return;
+        }
+
+        self.marker = Some(marker[..start_idx].to_owned());
+        for tag in tag_body.split(',') {
+            let Some((key, value)) = tag.split_once(':') else {
+                continue;
+            };
+
+            match key {
+                "rustfs_cache" => {}
+                "id" => self.id = Some(value.to_owned()),
+                "return" => {
+                    self.id = Some(Uuid::new_v4().to_string());
+                    self.create = true;
+                }
+                "p" => match value.parse::<usize>() {
+                    Ok(res) => self.pool_idx = Some(res),
+                    Err(_) => {
+                        self.id = Some(Uuid::new_v4().to_string());
+                        self.create = true;
+                    }
+                },
+                "s" => match value.parse::<usize>() {
+                    Ok(res) => self.set_idx = Some(res),
+                    Err(_) => {
+                        self.id = Some(Uuid::new_v4().to_string());
+                        self.create = true;
+                    }
+                },
+                _ => (),
             }
         }
     }
@@ -301,7 +379,7 @@ impl ECStore {
             limit: effective_max_keys,
             marker,
             incl_deleted,
-            ask_disks: "strict".to_owned(), //TODO: from config
+            ask_disks: list_quorum_from_env(),
             ..Default::default()
         };
 
@@ -444,13 +522,9 @@ impl ECStore {
             return Err(StorageError::NotImplemented);
         }
 
+        let has_version_marker = version_marker.is_some();
         let version_marker = if let Some(marker) = version_marker {
-            // "null" is used for non-versioned objects in AWS S3 API
-            if marker == "null" {
-                None
-            } else {
-                Some(Uuid::parse_str(&marker)?)
-            }
+            Some(parse_version_marker(marker)?)
         } else {
             None
         };
@@ -464,8 +538,9 @@ impl ECStore {
             limit: effective_max_keys,
             marker,
             incl_deleted: true,
-            ask_disks: "strict".to_owned(),
+            ask_disks: list_quorum_from_env(),
             versioned: true,
+            include_marker: has_version_marker,
             ..Default::default()
         };
 
@@ -486,10 +561,14 @@ impl ECStore {
             return Err(to_object_err(err.into(), vec![bucket, prefix]));
         }
 
-        if let Some(result) = list_result.entries.as_mut() {
-            result.forward_past(opts.marker);
+        if let Some(result) = list_result.entries.as_mut()
+            && !has_version_marker
+        {
+            result.forward_past(opts.marker.clone());
         }
 
+        let version_marker = version_marker_for_entries(list_result.entries.as_ref(), opts.marker.as_deref(), version_marker);
+
         let mut get_objects = ObjectInfo::from_meta_cache_entries_sorted_versions(
             &list_result.entries.unwrap_or_default(),
             bucket,
@@ -762,11 +841,13 @@ impl ECStore {
             }
         }
 
+        // All sets returned not-found — the listing is simply empty.
+        // We intentionally do NOT distinguish VolumeNotFound here: during
+        // listing, a missing volume on a set is equivalent to "no entries"
+        // and must not surface as an error to the caller. The original
+        // VolumeNotFound check caused spurious errors when a pool had not
+        // yet created the bucket prefix on every set.
         if is_all_not_found(&errs) {
-            if is_all_volume_not_found(&errs) {
-                return Err(StorageError::VolumeNotFound);
-            }
-
             return Ok(Vec::new());
         }
 
@@ -1080,7 +1161,7 @@ async fn gather_results(
         }
 
         if let Some(marker) = &opts.marker
-            && &entry.name <= marker
+            && ((!opts.include_marker && &entry.name <= marker) || (opts.include_marker && &entry.name < marker))
         {
             continue;
         }
@@ -1137,12 +1218,18 @@ async fn gather_results(
 }
 
 async fn select_from(
+    rx: &CancellationToken,
     in_channels: &mut [Receiver<MetaCacheEntry>],
     idx: usize,
     top: &mut [Option<MetaCacheEntry>],
     n_done: &mut usize,
-) -> Result<()> {
-    match in_channels[idx].recv().await {
+) -> Result<bool> {
+    let entry = tokio::select! {
+        entry = in_channels[idx].recv() => entry,
+        _ = rx.cancelled() => return Ok(false),
+    };
+
+    match entry {
         Some(entry) => {
             top[idx] = Some(entry);
         }
@@ -1151,10 +1238,19 @@ async fn select_from(
             *n_done += 1;
         }
     }
-    Ok(())
+    Ok(true)
+}
+
+async fn send_or_cancel(rx: &CancellationToken, out_channel: &Sender<MetaCacheEntry>, entry: MetaCacheEntry) -> Result<bool> {
+    tokio::select! {
+        result = out_channel.send(entry) => {
+            result.map_err(Error::other)?;
+            Ok(true)
+        }
+        _ = rx.cancelled() => Ok(false),
+    }
 }
 
-// TODO: exit when cancel
 async fn merge_entry_channels(
     rx: CancellationToken,
     in_channels: Vec<Receiver<MetaCacheEntry>>,
@@ -1172,7 +1268,9 @@ async fn merge_entry_channels(
                 has_entry = in_channels[0].recv()=>{
                     if let Some(entry) = has_entry{
                         // warn!("merge_entry_channels entry {}", &entry.name);
-                        out_channel.send(entry).await.map_err(Error::other)?;
+                        if !send_or_cancel(&rx, &out_channel, entry).await? {
+                            return Ok(());
+                        }
                     } else {
                         return Ok(())
                     }
@@ -1191,7 +1289,9 @@ async fn merge_entry_channels(
     let in_channels_len = in_channels.len();
 
     for idx in 0..in_channels_len {
-        select_from(&mut in_channels, idx, &mut top, &mut n_done).await?;
+        if !select_from(&rx, &mut in_channels, idx, &mut top, &mut n_done).await? {
+            return Ok(());
+        }
     }
 
     let mut last = String::new();
@@ -1205,16 +1305,13 @@ async fn merge_entry_channels(
         let mut best_idx = 0;
         to_merge.clear();
 
-        // FIXME: top move when select_from call
-        // let vtop = top.clone();
-
-        // let vtop = top.as_slice();
+        // Note: `select_from` mutates `top[idx]` during the inner loop, but this is safe
+        // because each borrow from `top[other_idx]` is only used before any later
+        // `select_from` call that can mutate that slot.
 
         for other_idx in 1..top.len() {
             if let Some(other_entry) = &top[other_idx] {
                 if let Some(best_entry) = &best {
-                    // println!("get other_entry {:?}", other_entry.name);
-
                     if path::clean(&best_entry.name) == path::clean(&other_entry.name) {
                         let dir_matches = best_entry.is_dir() && other_entry.is_dir();
                         let suffix_matches =
@@ -1229,7 +1326,9 @@ async fn merge_entry_channels(
                             // dir and object has the save name
                             if other_entry.is_dir() {
                                 // TODO: read next entry to top
-                                select_from(&mut in_channels, other_idx, &mut top, &mut n_done).await?;
+                                if !select_from(&rx, &mut in_channels, other_idx, &mut top, &mut n_done).await? {
+                                    return Ok(());
+                                }
                                 continue;
                             }
 
@@ -1270,7 +1369,9 @@ async fn merge_entry_channels(
                         let xl2 = match entry.clone().xl_meta() {
                             Ok(res) => res,
                             Err(_) => {
-                                select_from(&mut in_channels, idx, &mut top, &mut n_done).await?;
+                                if !select_from(&rx, &mut in_channels, idx, &mut top, &mut n_done).await? {
+                                    return Ok(());
+                                }
 
                                 continue;
                             }
@@ -1279,13 +1380,17 @@ async fn merge_entry_channels(
                         versions.push(xl2.versions.clone());
 
                         if has_xl.is_none() {
-                            select_from(&mut in_channels, best_idx, &mut top, &mut n_done).await?;
+                            if !select_from(&rx, &mut in_channels, best_idx, &mut top, &mut n_done).await? {
+                                return Ok(());
+                            }
 
                             best_idx = idx;
                             best = Some(entry.clone());
                             has_xl = Some(xl2);
                         } else {
-                            select_from(&mut in_channels, best_idx, &mut top, &mut n_done).await?;
+                            if !select_from(&rx, &mut in_channels, best_idx, &mut top, &mut n_done).await? {
+                                return Ok(());
+                            }
                         }
                     }
                 }
@@ -1310,11 +1415,15 @@ async fn merge_entry_channels(
         if let Some(best_entry) = &best
             && best_entry.name > last
         {
-            out_channel.send(best_entry.clone()).await.map_err(Error::other)?;
+            if !send_or_cancel(&rx, &out_channel, best_entry.clone()).await? {
+                return Ok(());
+            }
             last = best_entry.name.clone();
         }
 
-        select_from(&mut in_channels, best_idx, &mut top, &mut n_done).await?;
+        if !select_from(&rx, &mut in_channels, best_idx, &mut top, &mut n_done).await? {
+            return Ok(());
+        }
     }
 }
 
@@ -1349,16 +1458,7 @@ impl SetDisks {
             fallback_disks = disks.split_off(ask_disks as usize);
         }
 
-        let mut resolver = MetadataResolutionParams {
-            dir_quorum: listing_quorum,
-            obj_quorum: listing_quorum,
-            bucket: opts.bucket.clone(),
-            ..Default::default()
-        };
-
-        if opts.versioned {
-            resolver.requested_versions = 1;
-        }
+        let resolver = list_metadata_resolution_params(opts.bucket.clone(), listing_quorum, opts.versioned);
 
         let limit = {
             if opts.limit > 0 && opts.stop_disk_at_limit {
@@ -1483,9 +1583,13 @@ fn calc_common_counter(infos: &[DiskInfo], read_quorum: usize) -> u64 {
 
 #[cfg(test)]
 mod test {
-    use super::{ListPathOptions, MAX_OBJECT_LIST, gather_results, max_keys_plus_one, walk_result_from_set_errors};
+    use super::{
+        ENV_API_LIST_QUORUM, ListPathOptions, MAX_OBJECT_LIST, VersionMarker, gather_results, list_metadata_resolution_params,
+        list_quorum_from_env, max_keys_plus_one, merge_entry_channels, normalize_list_quorum, parse_version_marker,
+        version_marker_for_entries, walk_result_from_set_errors,
+    };
     use crate::error::StorageError;
-    use rustfs_filemeta::MetaCacheEntry;
+    use rustfs_filemeta::{MetaCacheEntries, MetaCacheEntriesSorted, MetaCacheEntry};
     use std::time::Duration;
     use tokio::sync::mpsc;
     use tokio::time::timeout;
@@ -1499,6 +1603,13 @@ mod test {
         }
     }
 
+    fn sorted_entries(names: &[&str]) -> MetaCacheEntriesSorted {
+        MetaCacheEntriesSorted {
+            o: MetaCacheEntries(names.iter().map(|name| Some(test_meta_entry(name))).collect()),
+            ..Default::default()
+        }
+    }
+
     #[tokio::test]
     async fn gather_results_returns_after_limit_without_waiting_for_input_close() {
         let (entry_tx, entry_rx) = mpsc::channel(4);
@@ -1531,6 +1642,109 @@ mod test {
             .expect("gather_results should succeed");
     }
 
+    #[tokio::test]
+    async fn gather_results_keeps_marker_entry_for_version_marker_listing() {
+        let (entry_tx, entry_rx) = mpsc::channel(4);
+        let (result_tx, mut result_rx) = mpsc::channel(1);
+
+        entry_tx.send(test_meta_entry("obj-a")).await.unwrap();
+        entry_tx.send(test_meta_entry("obj-b")).await.unwrap();
+
+        let handle = tokio::spawn(gather_results(
+            CancellationToken::new(),
+            ListPathOptions {
+                bucket: "bucket".to_owned(),
+                marker: Some("obj-a".to_owned()),
+                include_marker: true,
+                limit: 2,
+                incl_deleted: true,
+                versioned: true,
+                ..Default::default()
+            },
+            entry_rx,
+            result_tx,
+        ));
+
+        let result = timeout(Duration::from_secs(1), result_rx.recv())
+            .await
+            .expect("limited result should be sent promptly")
+            .expect("limited result should be present");
+        let entries = result.entries.unwrap();
+        let names = entries
+            .entries()
+            .into_iter()
+            .map(|entry| entry.name.as_str())
+            .collect::<Vec<_>>();
+
+        assert_eq!(names, ["obj-a", "obj-b"]);
+
+        timeout(Duration::from_secs(1), handle)
+            .await
+            .expect("gather_results should finish after sending a limited result")
+            .expect("gather_results task should not panic")
+            .expect("gather_results should succeed");
+    }
+
+    #[test]
+    fn version_marker_is_applied_only_when_key_marker_entry_is_present() {
+        let version_marker = Some(VersionMarker::Null);
+
+        let listed_after_deleted_marker = sorted_entries(&["obj-b", "obj-c"]);
+        assert_eq!(
+            version_marker_for_entries(Some(&listed_after_deleted_marker), Some("obj-a"), version_marker),
+            None
+        );
+
+        let listed_with_marker = sorted_entries(&["obj-a", "obj-b"]);
+        assert_eq!(
+            version_marker_for_entries(Some(&listed_with_marker), Some("obj-a"), version_marker),
+            version_marker
+        );
+    }
+
+    #[tokio::test]
+    async fn gather_results_skips_marker_entry_by_default() {
+        let (entry_tx, entry_rx) = mpsc::channel(4);
+        let (result_tx, mut result_rx) = mpsc::channel(1);
+
+        entry_tx.send(test_meta_entry("obj-a")).await.unwrap();
+        entry_tx.send(test_meta_entry("obj-b")).await.unwrap();
+        drop(entry_tx);
+
+        let handle = tokio::spawn(gather_results(
+            CancellationToken::new(),
+            ListPathOptions {
+                bucket: "bucket".to_owned(),
+                marker: Some("obj-a".to_owned()),
+                limit: 2,
+                incl_deleted: true,
+                ..Default::default()
+            },
+            entry_rx,
+            result_tx,
+        ));
+
+        let result = timeout(Duration::from_secs(1), result_rx.recv())
+            .await
+            .expect("eof result should be sent promptly")
+            .expect("eof result should be present");
+        let entries = result.entries.unwrap();
+        let names = entries
+            .entries()
+            .into_iter()
+            .map(|entry| entry.name.as_str())
+            .collect::<Vec<_>>();
+
+        assert_eq!(names, ["obj-b"]);
+        assert!(result.err.is_some());
+
+        timeout(Duration::from_secs(1), handle)
+            .await
+            .expect("gather_results should finish after input closes")
+            .expect("gather_results task should not panic")
+            .expect("gather_results should succeed");
+    }
+
     #[test]
     fn test_max_keys_plus_one_caps_before_lookahead() {
         assert_eq!(max_keys_plus_one(999, true), 1000);
@@ -1540,28 +1754,74 @@ mod test {
         assert_eq!(max_keys_plus_one(-1, true), MAX_OBJECT_LIST + 1);
     }
 
-    /// Test that "null" version marker is handled correctly
-    /// AWS S3 API uses "null" string to represent non-versioned objects
+    #[test]
+    fn normalize_list_quorum_accepts_supported_values() {
+        assert_eq!(normalize_list_quorum("strict"), "strict");
+        assert_eq!(normalize_list_quorum("disk"), "disk");
+        assert_eq!(normalize_list_quorum("reduced"), "reduced");
+        assert_eq!(normalize_list_quorum("optimal"), "optimal");
+        assert_eq!(normalize_list_quorum("auto"), "auto");
+        assert_eq!(normalize_list_quorum(" OPTIMAL "), "optimal");
+    }
+
+    #[test]
+    fn normalize_list_quorum_falls_back_to_strict() {
+        assert_eq!(normalize_list_quorum(""), "strict");
+        assert_eq!(normalize_list_quorum("unknown"), "strict");
+    }
+
+    #[test]
+    #[serial_test::serial]
+    fn list_quorum_from_env_defaults_to_strict() {
+        temp_env::with_var_unset(ENV_API_LIST_QUORUM, || {
+            assert_eq!(list_quorum_from_env(), "strict");
+        });
+    }
+
+    #[test]
+    #[serial_test::serial]
+    fn list_quorum_from_env_honors_supported_value() {
+        temp_env::with_var(ENV_API_LIST_QUORUM, Some("auto"), || {
+            assert_eq!(list_quorum_from_env(), "auto");
+        });
+    }
+
+    #[test]
+    #[serial_test::serial]
+    fn list_quorum_from_env_rejects_unknown_value() {
+        temp_env::with_var(ENV_API_LIST_QUORUM, Some("unsafe"), || {
+            assert_eq!(list_quorum_from_env(), "strict");
+        });
+    }
+
+    #[test]
+    fn list_metadata_resolution_params_limits_plain_listing_to_latest_version() {
+        let resolver = list_metadata_resolution_params("bucket".to_string(), 2, false);
+
+        assert_eq!(resolver.dir_quorum, 2);
+        assert_eq!(resolver.obj_quorum, 2);
+        assert_eq!(resolver.bucket, "bucket");
+        assert_eq!(resolver.requested_versions, 1);
+    }
+
+    #[test]
+    fn list_metadata_resolution_params_keeps_all_versions_for_version_listing() {
+        let resolver = list_metadata_resolution_params("bucket".to_string(), 3, true);
+
+        assert_eq!(resolver.dir_quorum, 3);
+        assert_eq!(resolver.obj_quorum, 3);
+        assert_eq!(resolver.bucket, "bucket");
+        assert_eq!(resolver.requested_versions, 0);
+    }
+
     #[test]
     fn test_null_version_marker_handling() {
-        // "null" should be treated as None (non-versioned)
-        let version_marker = "null";
-        let parsed: Option<Uuid> = if version_marker == "null" {
-            None
-        } else {
-            Uuid::parse_str(version_marker).ok()
-        };
-        assert!(parsed.is_none(), "\"null\" should be parsed as None");
+        let parsed = parse_version_marker("null".to_string()).expect("null marker should parse");
+        assert_eq!(parsed, VersionMarker::Null);
 
-        // Valid UUID should be parsed correctly
         let valid_uuid = "550e8400-e29b-41d4-a716-446655440000";
-        let parsed: Option<Uuid> = if valid_uuid == "null" {
-            None
-        } else {
-            Uuid::parse_str(valid_uuid).ok()
-        };
-        assert!(parsed.is_some(), "Valid UUID should be parsed correctly");
-        assert_eq!(parsed.unwrap().to_string(), "550e8400-e29b-41d4-a716-446655440000");
+        let parsed = parse_version_marker(valid_uuid.to_string()).expect("uuid marker should parse");
+        assert_eq!(parsed, VersionMarker::Version(Uuid::parse_str(valid_uuid).unwrap()));
     }
 
     /// Test that next_version_idmarker returns "null" for non-versioned objects
@@ -1584,15 +1844,11 @@ mod test {
         // Scenario 1: Non-versioned object
         // Server returns "null" as NextVersionIdMarker
         // Client sends "null" as VersionIdMarker
-        // Server parses "null" as None
+        // Server parses "null" as an explicit null-version marker
         let server_response = "null";
         let client_request = server_response;
-        let parsed: Option<Uuid> = if client_request == "null" {
-            None
-        } else {
-            Uuid::parse_str(client_request).ok()
-        };
-        assert!(parsed.is_none());
+        let parsed = parse_version_marker(client_request.to_string()).expect("null marker should parse");
+        assert_eq!(parsed, VersionMarker::Null);
 
         // Scenario 2: Versioned object
         // Server returns UUID as NextVersionIdMarker
@@ -1601,13 +1857,8 @@ mod test {
         let uuid_str = "550e8400-e29b-41d4-a716-446655440000";
         let server_response = uuid_str;
         let client_request = server_response;
-        let parsed: Option<Uuid> = if client_request == "null" {
-            None
-        } else {
-            Uuid::parse_str(client_request).ok()
-        };
-        assert!(parsed.is_some());
-        assert_eq!(parsed.unwrap().to_string(), uuid_str);
+        let parsed = parse_version_marker(client_request.to_string()).expect("uuid marker should parse");
+        assert_eq!(parsed, VersionMarker::Version(Uuid::parse_str(uuid_str).unwrap()));
     }
 
     #[test]
@@ -1639,6 +1890,41 @@ mod test {
         assert!(!parsed.create);
     }
 
+    #[test]
+    fn list_path_marker_parser_uses_trailing_cache_tag() {
+        let mut parsed = ListPathOptions {
+            marker: Some(format!(
+                "photos/[archive]/image.jpg[rustfs_cache:{},id:list-cache-id,p:3,s:7]",
+                super::MARKER_TAG_VERSION
+            )),
+            ..Default::default()
+        };
+
+        parsed.parse_marker();
+
+        assert_eq!(parsed.marker.as_deref(), Some("photos/[archive]/image.jpg"));
+        assert_eq!(parsed.id.as_deref(), Some("list-cache-id"));
+        assert_eq!(parsed.pool_idx, Some(3));
+        assert_eq!(parsed.set_idx, Some(7));
+    }
+
+    #[test]
+    fn list_path_marker_parser_ignores_unsupported_cache_tag_version() {
+        let marker = "photos/image.jpg[rustfs_cache:v0,id:list-cache-id,p:3,s:7]".to_string();
+        let mut parsed = ListPathOptions {
+            marker: Some(marker.clone()),
+            ..Default::default()
+        };
+
+        parsed.parse_marker();
+
+        assert_eq!(parsed.marker.as_deref(), Some(marker.as_str()));
+        assert!(parsed.id.is_none());
+        assert!(parsed.pool_idx.is_none());
+        assert!(parsed.set_idx.is_none());
+        assert!(!parsed.create);
+    }
+
     #[test]
     fn walk_result_from_set_errors_returns_non_eof_error() {
         let err = walk_result_from_set_errors(&[Some(StorageError::Unexpected), Some(StorageError::FileAccessDenied)])
@@ -1966,4 +2252,172 @@ mod test {
     //         println!("get entry {:?}", entry)
     //     }
     // }
+
+    #[tokio::test]
+    async fn merge_entry_channels_produces_sorted_unique_output_from_two_channels() {
+        let (tx_a, rx_a) = mpsc::channel(4);
+        let (tx_b, rx_b) = mpsc::channel(4);
+        let (out_tx, mut out_rx) = mpsc::channel(8);
+
+        // Send sorted entries from two channels with some overlap
+        tx_a.send(test_meta_entry("obj-a")).await.unwrap();
+        tx_a.send(test_meta_entry("obj-c")).await.unwrap();
+        tx_a.send(test_meta_entry("obj-e")).await.unwrap();
+        drop(tx_a);
+
+        tx_b.send(test_meta_entry("obj-b")).await.unwrap();
+        tx_b.send(test_meta_entry("obj-d")).await.unwrap();
+        drop(tx_b);
+
+        let rx = CancellationToken::new();
+        let handle = tokio::spawn(merge_entry_channels(rx, vec![rx_a, rx_b], out_tx, 1));
+
+        let mut results = Vec::new();
+        while let Some(entry) = out_rx.recv().await {
+            results.push(entry.name.clone());
+        }
+
+        handle.await.unwrap().unwrap();
+
+        // Results should be sorted and deduplicated
+        assert_eq!(results, vec!["obj-a", "obj-b", "obj-c", "obj-d", "obj-e"]);
+    }
+
+    #[tokio::test]
+    async fn merge_entry_channels_deduplicates_entries_across_channels() {
+        let (tx_a, rx_a) = mpsc::channel(4);
+        let (tx_b, rx_b) = mpsc::channel(4);
+        let (out_tx, mut out_rx) = mpsc::channel(8);
+
+        // Both channels have the same entry
+        tx_a.send(test_meta_entry("obj-a")).await.unwrap();
+        tx_a.send(test_meta_entry("obj-c")).await.unwrap();
+        drop(tx_a);
+
+        tx_b.send(test_meta_entry("obj-a")).await.unwrap();
+        tx_b.send(test_meta_entry("obj-b")).await.unwrap();
+        drop(tx_b);
+
+        let rx = CancellationToken::new();
+        let handle = tokio::spawn(merge_entry_channels(rx, vec![rx_a, rx_b], out_tx, 1));
+
+        let mut results = Vec::new();
+        while let Some(entry) = out_rx.recv().await {
+            results.push(entry.name.clone());
+        }
+
+        handle.await.unwrap().unwrap();
+
+        // "obj-a" should appear only once despite being in both channels
+        assert_eq!(results, vec!["obj-a", "obj-b", "obj-c"]);
+    }
+
+    #[tokio::test]
+    async fn merge_entry_channels_handles_single_channel() {
+        let (tx, rx) = mpsc::channel(4);
+        let (out_tx, mut out_rx) = mpsc::channel(8);
+
+        tx.send(test_meta_entry("obj-a")).await.unwrap();
+        tx.send(test_meta_entry("obj-b")).await.unwrap();
+        drop(tx);
+
+        let cancel = CancellationToken::new();
+        let handle = tokio::spawn(merge_entry_channels(cancel, vec![rx], out_tx, 1));
+
+        let mut results = Vec::new();
+        while let Some(entry) = out_rx.recv().await {
+            results.push(entry.name.clone());
+        }
+
+        handle.await.unwrap().unwrap();
+
+        assert_eq!(results, vec!["obj-a", "obj-b"]);
+    }
+
+    #[tokio::test]
+    async fn merge_entry_channels_respects_cancellation() {
+        let (tx, rx) = mpsc::channel::<MetaCacheEntry>(4);
+        let (out_tx, mut out_rx) = mpsc::channel(8);
+
+        let cancel = CancellationToken::new();
+        let cancel_clone = cancel.clone();
+
+        // Use a single channel so the cancellation check in tokio::select! is exercised.
+        let handle = tokio::spawn(merge_entry_channels(cancel_clone, vec![rx], out_tx, 1));
+
+        // Send an entry to prove the function is running and processing data.
+        tx.send(test_meta_entry("a")).await.unwrap();
+        let received = timeout(Duration::from_millis(500), out_rx.recv())
+            .await
+            .expect("should receive entry before cancellation")
+            .map(|e| e.name);
+        assert_eq!(received, Some("a".to_string()));
+
+        // Cancel while the sender is still alive (channel is not closed).
+        cancel.cancel();
+
+        let result = timeout(Duration::from_secs(2), handle)
+            .await
+            .expect("merge should not hang after cancellation")
+            .expect("task should not panic");
+        assert!(result.is_ok(), "merge should return Ok on cancellation");
+
+        // Keep tx alive until after the assertion so the channel doesn't close prematurely.
+        drop(tx);
+    }
+
+    #[tokio::test]
+    async fn merge_entry_channels_respects_cancellation_with_multiple_live_channels() {
+        let (tx_a, rx_a) = mpsc::channel::<MetaCacheEntry>(4);
+        let (tx_b, rx_b) = mpsc::channel::<MetaCacheEntry>(4);
+        let (out_tx, _out_rx) = mpsc::channel(8);
+
+        let cancel = CancellationToken::new();
+        let cancel_clone = cancel.clone();
+
+        let handle = tokio::spawn(merge_entry_channels(cancel_clone, vec![rx_a, rx_b], out_tx, 1));
+        cancel.cancel();
+
+        let result = timeout(Duration::from_secs(2), handle)
+            .await
+            .expect("multi-channel merge should not hang after cancellation")
+            .expect("task should not panic");
+        assert!(result.is_ok(), "multi-channel merge should return Ok on cancellation");
+
+        drop(tx_a);
+        drop(tx_b);
+    }
+
+    #[tokio::test]
+    async fn merge_entry_channels_respects_cancellation_when_output_is_full() {
+        let (tx_a, rx_a) = mpsc::channel::<MetaCacheEntry>(4);
+        let (tx_b, rx_b) = mpsc::channel::<MetaCacheEntry>(4);
+        let (out_tx, _out_rx) = mpsc::channel(1);
+
+        out_tx
+            .send(test_meta_entry("already-buffered"))
+            .await
+            .expect("output channel should accept initial entry");
+        tx_a.send(test_meta_entry("a"))
+            .await
+            .expect("input channel a should accept entry");
+        tx_b.send(test_meta_entry("b"))
+            .await
+            .expect("input channel b should accept entry");
+
+        let cancel = CancellationToken::new();
+        let cancel_clone = cancel.clone();
+
+        let handle = tokio::spawn(merge_entry_channels(cancel_clone, vec![rx_a, rx_b], out_tx, 1));
+        cancel.cancel();
+
+        let result = timeout(Duration::from_secs(2), handle)
+            .await
+            .expect("merge should not hang when output is full and cancellation fires")
+            .expect("task should not panic");
+        assert!(result.is_ok(), "merge should return Ok when cancelled during output send");
+
+        drop(tx_a);
+        drop(tx_b);
+    }
 }

crates/heal/src/heal/channel.rs

@@ -281,6 +281,12 @@ impl HealChannelProcessor {
                 data: Some("stopped".as_bytes().to_vec()),
                 error: None,
             },
+            Err(Error::TaskNotFound { .. }) if client_token.is_empty() => HealChannelResponse {
+                request_id,
+                success: true,
+                data: Some("stopped".as_bytes().to_vec()),
+                error: None,
+            },
             Err(err) => HealChannelResponse {
                 request_id,
                 success: false,
@@ -965,6 +971,27 @@ mod tests {
         ));
     }
 
+    #[tokio::test]
+    async fn test_process_cancel_request_treats_unknown_path_as_stopped() {
+        let heal_manager = create_test_heal_manager();
+        let processor = HealChannelProcessor::new(heal_manager);
+        let (tx, rx) = oneshot::channel();
+
+        processor
+            .process_cancel_request(".".to_string(), String::new(), tx)
+            .await
+            .expect("cancel should process");
+
+        let response = rx
+            .await
+            .expect("oneshot should resolve")
+            .expect("cancel response should be returned");
+        assert!(response.success);
+        assert_eq!(response.request_id, ".");
+        assert_eq!(response.data.as_deref(), Some("stopped".as_bytes()));
+        assert!(response.error.is_none());
+    }
+
     #[tokio::test]
     async fn test_process_cancel_request_reports_unknown_task() {
         let heal_manager = create_test_heal_manager();

crates/heal/src/heal/manager.rs

@@ -149,6 +149,50 @@ impl PriorityHealQueue {
         QueuePushOutcome::Accepted
     }
 
+    fn can_displace_lower_priority(&self, priority: HealPriority) -> bool {
+        self.heap.iter().any(|item| item.priority < priority)
+    }
+
+    fn push_displacing_lower_priority(&mut self, request: HealRequest) -> Option<HealRequest> {
+        let mut retained = BinaryHeap::new();
+        let mut displaced: Option<PriorityQueueItem> = None;
+
+        while let Some(item) = self.heap.pop() {
+            if item.priority < request.priority {
+                let should_displace = displaced
+                    .as_ref()
+                    .map(|current| {
+                        item.priority < current.priority
+                            || (item.priority == current.priority && item.sequence > current.sequence)
+                    })
+                    .unwrap_or(true);
+                if should_displace {
+                    if let Some(current) = displaced.replace(item) {
+                        retained.push(current);
+                    }
+                } else {
+                    retained.push(item);
+                }
+            } else {
+                retained.push(item);
+            }
+        }
+
+        self.heap = retained;
+
+        let displaced = displaced.map(|item| {
+            let key = Self::make_dedup_key(&item.request);
+            Self::decrement_or_remove_dedup_key(&mut self.dedup_keys, &key);
+            item.request
+        });
+
+        if displaced.is_some() {
+            debug_assert_eq!(self.push(request), QueuePushOutcome::Accepted);
+        }
+
+        displaced
+    }
+
     /// Get statistics about queue contents by priority
     fn get_priority_stats(&self) -> HashMap<HealPriority, usize> {
         let mut stats = HashMap::new();
@@ -578,6 +622,37 @@ impl HealManager {
         }
 
         if queue_len >= queue_capacity && !request.force_start {
+            if queue.can_displace_lower_priority(request.priority) {
+                let request_id = request.id.clone();
+                let priority = request.priority;
+                if let Some(displaced) = queue.push_displacing_lower_priority(request) {
+                    publish_heal_queue_length(&queue);
+                    warn!(
+                        request_id = %request_id,
+                        priority = ?priority,
+                        displaced_request_id = %displaced.id,
+                        displaced_priority = ?displaced.priority,
+                        queue_len,
+                        queue_capacity,
+                        "Admitted heal request by displacing lower-priority queued work"
+                    );
+                    drop(queue);
+                    if config.event_driven_scheduler_enable {
+                        self.notify.notify_one();
+                    }
+                    return Ok(HealAdmissionResult::Accepted);
+                }
+
+                warn!(
+                    request_id = %request_id,
+                    priority = ?priority,
+                    queue_len,
+                    queue_capacity,
+                    "Heal queue was full and lower-priority work was unavailable for displacement"
+                );
+                return Ok(HealAdmissionResult::Full);
+            }
+
             let admission = Self::classify_full_admission(&request, &config);
             match admission {
                 HealAdmissionResult::Dropped(reason) => {
@@ -2135,6 +2210,59 @@ mod tests {
         );
     }
 
+    #[tokio::test]
+    async fn test_high_priority_request_displaces_lower_priority_when_queue_full() {
+        let storage: Arc<dyn HealStorageAPI> = Arc::new(MockStorage);
+        let manager = HealManager::new(
+            storage,
+            Some(HealConfig {
+                queue_size: 1,
+                ..HealConfig::default()
+            }),
+        );
+
+        let low = HealRequest::new(
+            HealType::Bucket {
+                bucket: "background-bucket".to_string(),
+            },
+            HealOptions::default(),
+            HealPriority::Low,
+        );
+        let low_id = low.id.clone();
+        let high = HealRequest::new(
+            HealType::Bucket {
+                bucket: "manual-bucket".to_string(),
+            },
+            HealOptions::default(),
+            HealPriority::High,
+        );
+        let high_id = high.id.clone();
+
+        assert_eq!(
+            manager
+                .submit_heal_request(low)
+                .await
+                .expect("low priority request should be accepted first"),
+            HealAdmissionResult::Accepted
+        );
+        assert_eq!(
+            manager
+                .submit_heal_request(high)
+                .await
+                .expect("high priority request should be admitted by displacing lower priority work"),
+            HealAdmissionResult::Accepted
+        );
+        assert_eq!(manager.get_queue_length().await, 1);
+        assert!(matches!(manager.get_task_status(&low_id).await, Err(Error::TaskNotFound { .. })));
+        assert_eq!(
+            manager
+                .get_task_status(&high_id)
+                .await
+                .expect("high priority request should remain queued"),
+            HealTaskStatus::Pending
+        );
+    }
+
     #[tokio::test]
     async fn test_force_start_bypasses_duplicate_and_full_admission() {
         let storage: Arc<dyn HealStorageAPI> = Arc::new(MockStorage);

crates/iam/src/oidc.rs

@@ -295,6 +295,7 @@ pub struct OidcProviderConfig {
     pub roles_claim: String,
     pub email_claim: String,
     pub username_claim: String,
+    pub hide_from_ui: bool,
 }
 
 #[derive(Debug, Clone, Copy, Serialize, Deserialize, PartialEq, Eq)]
@@ -422,7 +423,7 @@ impl OidcSys {
         !self.configs.is_empty()
     }
 
-    /// Return provider summaries for the console UI.
+    /// List all providers (including hidden ones). Used by site-replication and admin config.
     pub fn list_providers(&self) -> Vec<OidcProviderSummary> {
         self.configs
             .values()
@@ -433,6 +434,18 @@ impl OidcSys {
             .collect()
     }
 
+    /// List only visible providers (excludes those with `hide_from_ui = true`).
+    pub fn list_visible_providers(&self) -> Vec<OidcProviderSummary> {
+        self.configs
+            .values()
+            .filter(|c| !c.hide_from_ui)
+            .map(|c| OidcProviderSummary {
+                provider_id: c.id.clone(),
+                display_name: c.display_name.clone(),
+            })
+            .collect()
+    }
+
     /// Build the PKCE authorization URL for a provider, store state in the state store.
     pub async fn authorize_url(
         &self,
@@ -918,6 +931,19 @@ impl OidcSys {
         configs
     }
 
+    /// Parse a string as an `EnableState` boolean.
+    /// Returns `default_if_empty` when the input is empty, and `default_on_error`
+    /// when parsing fails.
+    fn parse_enable_state(value: &str, default_if_empty: bool, default_on_error: bool) -> bool {
+        if value.is_empty() {
+            return default_if_empty;
+        }
+        value
+            .parse::<EnableState>()
+            .map(|s| s.is_enabled())
+            .unwrap_or(default_on_error)
+    }
+
     /// Parse a single provider's config from env vars with the given suffix.
     fn parse_single_provider(env_suffix: &str, id: &str) -> Option<OidcProviderConfig> {
         let get_env = |base: &str| -> String { std::env::var(format!("{base}{env_suffix}")).unwrap_or_default() };
@@ -930,11 +956,7 @@ impl OidcSys {
             return None;
         }
 
-        let enabled = enable_val.is_empty()
-            || enable_val
-                .parse::<rustfs_config::EnableState>()
-                .map(|s| s.is_enabled())
-                .unwrap_or(false);
+        let enabled = Self::parse_enable_state(&enable_val, true, false);
 
         let scopes_str = get_env(ENV_IDENTITY_OPENID_SCOPES);
         let scopes = if scopes_str.is_empty() {
@@ -951,12 +973,7 @@ impl OidcSys {
             .map(|s| s.to_string())
             .collect();
 
-        let redirect_uri_dynamic_str = get_env(ENV_IDENTITY_OPENID_REDIRECT_URI_DYNAMIC);
-        let redirect_uri_dynamic = redirect_uri_dynamic_str.is_empty()
-            || redirect_uri_dynamic_str
-                .parse::<rustfs_config::EnableState>()
-                .map(|s| s.is_enabled())
-                .unwrap_or(true);
+        let redirect_uri_dynamic = Self::parse_enable_state(&get_env(ENV_IDENTITY_OPENID_REDIRECT_URI_DYNAMIC), true, true);
 
         let claim_name = {
             let v = get_env(ENV_IDENTITY_OPENID_CLAIM_NAME);
@@ -1003,6 +1020,7 @@ impl OidcSys {
             let v = get_env(ENV_IDENTITY_OPENID_CLIENT_SECRET);
             if v.is_empty() { None } else { Some(v) }
         };
+        let hide_from_ui = Self::parse_enable_state(&get_env(ENV_IDENTITY_OPENID_HIDE_FROM_UI), false, false);
 
         Some(OidcProviderConfig {
             id: id.to_string(),
@@ -1022,6 +1040,7 @@ impl OidcSys {
             roles_claim,
             email_claim,
             username_claim,
+            hide_from_ui,
         })
     }
 
@@ -1031,12 +1050,7 @@ impl OidcSys {
             return None;
         }
 
-        let enabled = kvs
-            .lookup(ENABLE_KEY)
-            .unwrap_or_else(|| EnableState::Off.to_string())
-            .parse::<EnableState>()
-            .map(|s| s.is_enabled())
-            .unwrap_or(false);
+        let enabled = Self::parse_enable_state(&kvs.lookup(ENABLE_KEY).unwrap_or_default(), false, false);
 
         let scopes_str = kvs.get(OIDC_SCOPES);
         let scopes = if scopes_str.is_empty() {
@@ -1053,12 +1067,8 @@ impl OidcSys {
             .map(|s| s.to_string())
             .collect();
 
-        let redirect_uri_dynamic = kvs
-            .lookup(OIDC_REDIRECT_URI_DYNAMIC)
-            .unwrap_or_else(|| EnableState::On.to_string())
-            .parse::<EnableState>()
-            .map(|s| s.is_enabled())
-            .unwrap_or(true);
+        let redirect_uri_dynamic =
+            Self::parse_enable_state(&kvs.lookup(OIDC_REDIRECT_URI_DYNAMIC).unwrap_or_default(), true, true);
 
         let claim_name = kvs
             .lookup(OIDC_CLAIM_NAME)
@@ -1078,6 +1088,7 @@ impl OidcSys {
         let display_name = kvs.lookup(OIDC_DISPLAY_NAME).unwrap_or_else(|| id.to_string());
         let redirect_uri = kvs.lookup(OIDC_REDIRECT_URI).filter(|v| !v.is_empty());
         let client_secret = kvs.lookup(OIDC_CLIENT_SECRET).filter(|v| !v.is_empty());
+        let hide_from_ui = Self::parse_enable_state(&kvs.lookup(OIDC_HIDE_FROM_UI).unwrap_or_default(), false, false);
 
         Some(OidcProviderConfig {
             id: id.to_string(),
@@ -1097,6 +1108,7 @@ impl OidcSys {
             roles_claim,
             email_claim,
             username_claim,
+            hide_from_ui,
         })
     }
 
@@ -1582,6 +1594,7 @@ mod tests {
             roles_claim: String::new(),
             email_claim: "email".to_string(),
             username_claim: "username".to_string(),
+            hide_from_ui: false,
         }
     }
 
@@ -1961,9 +1974,198 @@ mod tests {
             roles_claim: String::new(),
             email_claim: "email".to_string(),
             username_claim: "preferred_username".to_string(),
+            hide_from_ui: false,
         }
     }
 
+    #[test]
+    fn test_parse_enable_state_on() {
+        assert!(OidcSys::parse_enable_state("on", false, false));
+    }
+
+    #[test]
+    fn test_parse_enable_state_off() {
+        assert!(!OidcSys::parse_enable_state("off", true, true));
+    }
+
+    #[test]
+    fn test_parse_enable_state_empty_returns_default() {
+        assert!(OidcSys::parse_enable_state("", true, false));
+        assert!(!OidcSys::parse_enable_state("", false, true));
+    }
+
+    #[test]
+    fn test_parse_enable_state_invalid_returns_error_default() {
+        assert!(!OidcSys::parse_enable_state("garbage", true, false));
+        assert!(OidcSys::parse_enable_state("garbage", false, true));
+    }
+
+    #[test]
+    fn test_list_visible_providers_hides_hidden_provider() {
+        let visible = test_config("dex");
+        let mut hidden = test_config("kubernetes");
+        hidden.hide_from_ui = true;
+
+        let sys = make_test_sys(vec![visible, hidden]);
+        let listed = sys.list_visible_providers();
+
+        assert_eq!(listed.len(), 1);
+        assert!(listed.iter().any(|p| p.provider_id == "dex"));
+        assert!(!listed.iter().any(|p| p.provider_id == "kubernetes"));
+    }
+
+    #[test]
+    fn test_hidden_provider_still_resolvable_for_sts() {
+        let visible = test_config("dex");
+        let mut hidden = test_config("kubernetes");
+        hidden.hide_from_ui = true;
+
+        let sys = make_test_sys(vec![visible, hidden]);
+
+        assert!(sys.get_provider_config("kubernetes").is_some());
+        assert!(sys.get_provider_config("dex").is_some());
+    }
+
+    #[test]
+    fn test_list_providers_includes_hidden_for_replication() {
+        let visible = test_config("dex");
+        let mut hidden = test_config("kubernetes");
+        hidden.hide_from_ui = true;
+
+        let sys = make_test_sys(vec![visible, hidden]);
+
+        // Unfiltered list returns all (used by site-replication)
+        assert_eq!(sys.list_providers().len(), 2);
+        // UI-filtered list hides the hidden one
+        assert_eq!(sys.list_visible_providers().len(), 1);
+    }
+
+    #[test]
+    fn test_list_providers_all_visible_by_default() {
+        let a = test_config("okta");
+        let b = test_config("dex");
+
+        let sys = make_test_sys(vec![a, b]);
+        let listed = sys.list_visible_providers();
+
+        assert_eq!(listed.len(), 2);
+    }
+
+    #[test]
+    fn test_list_visible_providers_all_hidden() {
+        let mut a = test_config("k8s-a");
+        a.hide_from_ui = true;
+        let mut b = test_config("k8s-b");
+        b.hide_from_ui = true;
+
+        let sys = make_test_sys(vec![a, b]);
+        let listed = sys.list_visible_providers();
+
+        assert!(listed.is_empty());
+        assert!(sys.has_providers());
+    }
+
+    #[test]
+    fn test_hide_from_ui_default_is_false() {
+        let config = test_config("default");
+        assert!(!config.hide_from_ui);
+    }
+
+    #[test]
+    fn test_parse_persisted_hide_from_ui_off_is_false() {
+        let mut cfg = ServerConfig::new();
+        let mut kvs = KVS(vec![
+            rustfs_ecstore::config::KV {
+                key: ENABLE_KEY.to_string(),
+                value: EnableState::On.to_string(),
+                hidden_if_empty: false,
+            },
+            rustfs_ecstore::config::KV {
+                key: OIDC_CONFIG_URL.to_string(),
+                value: "https://example.com/.well-known/openid-configuration".to_string(),
+                hidden_if_empty: false,
+            },
+            rustfs_ecstore::config::KV {
+                key: OIDC_CLIENT_ID.to_string(),
+                value: "console".to_string(),
+                hidden_if_empty: false,
+            },
+        ]);
+        kvs.insert(OIDC_HIDE_FROM_UI.to_string(), EnableState::Off.to_string());
+
+        cfg.0
+            .entry(IDENTITY_OPENID_SUB_SYS.to_string())
+            .or_default()
+            .insert(DEFAULT_DELIMITER.to_string(), kvs);
+
+        let parsed = OidcSys::parse_persisted_configs(&cfg);
+        assert_eq!(parsed.len(), 1);
+        assert!(!parsed[0].hide_from_ui);
+    }
+
+    #[test]
+    fn test_parse_persisted_hide_from_ui_missing_defaults_false() {
+        let mut cfg = ServerConfig::new();
+        let kvs = KVS(vec![
+            rustfs_ecstore::config::KV {
+                key: ENABLE_KEY.to_string(),
+                value: EnableState::On.to_string(),
+                hidden_if_empty: false,
+            },
+            rustfs_ecstore::config::KV {
+                key: OIDC_CONFIG_URL.to_string(),
+                value: "https://example.com/.well-known/openid-configuration".to_string(),
+                hidden_if_empty: false,
+            },
+            rustfs_ecstore::config::KV {
+                key: OIDC_CLIENT_ID.to_string(),
+                value: "console".to_string(),
+                hidden_if_empty: false,
+            },
+        ]);
+
+        cfg.0
+            .entry(IDENTITY_OPENID_SUB_SYS.to_string())
+            .or_default()
+            .insert(DEFAULT_DELIMITER.to_string(), kvs);
+
+        let parsed = OidcSys::parse_persisted_configs(&cfg);
+        assert_eq!(parsed.len(), 1);
+        assert!(!parsed[0].hide_from_ui);
+    }
+
+    #[test]
+    fn test_parse_persisted_hide_from_ui() {
+        let mut cfg = ServerConfig::new();
+        let mut kvs = KVS(vec![
+            rustfs_ecstore::config::KV {
+                key: ENABLE_KEY.to_string(),
+                value: EnableState::On.to_string(),
+                hidden_if_empty: false,
+            },
+            rustfs_ecstore::config::KV {
+                key: OIDC_CONFIG_URL.to_string(),
+                value: "https://example.com/.well-known/openid-configuration".to_string(),
+                hidden_if_empty: false,
+            },
+            rustfs_ecstore::config::KV {
+                key: OIDC_CLIENT_ID.to_string(),
+                value: "console".to_string(),
+                hidden_if_empty: false,
+            },
+        ]);
+        kvs.insert(OIDC_HIDE_FROM_UI.to_string(), EnableState::On.to_string());
+
+        cfg.0
+            .entry(IDENTITY_OPENID_SUB_SYS.to_string())
+            .or_default()
+            .insert(DEFAULT_DELIMITER.to_string(), kvs);
+
+        let parsed = OidcSys::parse_persisted_configs(&cfg);
+        assert_eq!(parsed.len(), 1);
+        assert!(parsed[0].hide_from_ui);
+    }
+
     #[test]
     fn test_map_claims_to_policies_with_provider() {
         let mut config = test_config("okta");
@@ -2056,6 +2258,7 @@ mod tests {
             roles_claim: String::new(),
             email_claim: "email".to_string(),
             username_claim: "preferred_username".to_string(),
+            hide_from_ui: false,
         };
 
         assert_eq!(config.id, "test");

crates/iam/src/sys.rs

@@ -834,8 +834,27 @@ impl<T: Store> IamSys<T> {
         self.store.policy_db_get(name, groups).await
     }
 
+    /// Check whether a policy name from a JWT claim is safe to resolve against the IAM store.
+    ///
+    /// Allowed characters: `[a-zA-Z0-9_:.-]`
+    /// - Colons: K8s service account `sub` claims (`system:serviceaccount:ns:sa`)
+    /// - Dots: OIDC group names from providers like Okta/Entra (`org.team.role`)
+    ///
+    /// Requirements:
+    /// - At least one ASCII alphanumeric character (prevents meaningless names
+    ///   like `.`, `..`, `-`, `___`, or `:.:`).
+    /// - No characters outside the allowed set (helps mitigate path traversal
+    ///   and injection when names are used in storage keys or log output).
     fn is_safe_claim_policy_name(policy: &str) -> bool {
-        !policy.is_empty() && policy.chars().all(|c| c.is_ascii_alphanumeric() || c == '_' || c == '-')
+        let mut has_alnum = false;
+        for c in policy.bytes() {
+            if c.is_ascii_alphanumeric() {
+                has_alnum = true;
+            } else if !matches!(c, b'_' | b'-' | b':' | b'.') {
+                return false;
+            }
+        }
+        has_alnum
     }
 
     // JWT policy claims carry canned policy names only; policy documents are resolved by IAM store.
@@ -2667,4 +2686,64 @@ mod tests {
             policy_info.policy
         );
     }
+
+    #[test]
+    fn test_is_safe_claim_policy_name_allows_k8s_sa_sub() {
+        assert!(IamSys::<StsTestMockStore>::is_safe_claim_policy_name(
+            "system:serviceaccount:my-namespace:my-sa"
+        ));
+    }
+
+    #[test]
+    fn test_is_safe_claim_policy_name_allows_dotted_names() {
+        assert!(IamSys::<StsTestMockStore>::is_safe_claim_policy_name("org.team.policy-name"));
+    }
+
+    #[test]
+    fn test_is_safe_claim_policy_name_allows_simple_names() {
+        assert!(IamSys::<StsTestMockStore>::is_safe_claim_policy_name("readwrite"));
+        assert!(IamSys::<StsTestMockStore>::is_safe_claim_policy_name("my-custom_policy"));
+        assert!(IamSys::<StsTestMockStore>::is_safe_claim_policy_name("Policy123"));
+    }
+
+    #[test]
+    fn test_is_safe_claim_policy_name_rejects_empty() {
+        assert!(!IamSys::<StsTestMockStore>::is_safe_claim_policy_name(""));
+    }
+
+    #[test]
+    fn test_is_safe_claim_policy_name_rejects_path_traversal() {
+        assert!(!IamSys::<StsTestMockStore>::is_safe_claim_policy_name("../etc/passwd"));
+        assert!(!IamSys::<StsTestMockStore>::is_safe_claim_policy_name("policy/name"));
+        assert!(!IamSys::<StsTestMockStore>::is_safe_claim_policy_name("policy\\name"));
+        assert!(!IamSys::<StsTestMockStore>::is_safe_claim_policy_name("..\\etc\\passwd"));
+    }
+
+    #[test]
+    fn test_is_safe_claim_policy_name_rejects_whitespace() {
+        assert!(!IamSys::<StsTestMockStore>::is_safe_claim_policy_name("my policy"));
+        assert!(!IamSys::<StsTestMockStore>::is_safe_claim_policy_name("policy\tname"));
+    }
+
+    #[test]
+    fn test_is_safe_claim_policy_name_rejects_special_chars() {
+        assert!(!IamSys::<StsTestMockStore>::is_safe_claim_policy_name("policy;drop"));
+        assert!(!IamSys::<StsTestMockStore>::is_safe_claim_policy_name("pol$icy"));
+        assert!(!IamSys::<StsTestMockStore>::is_safe_claim_policy_name("name{bad}"));
+    }
+
+    #[test]
+    fn test_is_safe_claim_policy_name_rejects_no_alphanumeric() {
+        assert!(!IamSys::<StsTestMockStore>::is_safe_claim_policy_name("."));
+        assert!(!IamSys::<StsTestMockStore>::is_safe_claim_policy_name(".."));
+        assert!(!IamSys::<StsTestMockStore>::is_safe_claim_policy_name(":"));
+        assert!(!IamSys::<StsTestMockStore>::is_safe_claim_policy_name("..."));
+        assert!(!IamSys::<StsTestMockStore>::is_safe_claim_policy_name(".:.:"));
+        assert!(!IamSys::<StsTestMockStore>::is_safe_claim_policy_name("-"));
+        assert!(!IamSys::<StsTestMockStore>::is_safe_claim_policy_name("_"));
+        assert!(!IamSys::<StsTestMockStore>::is_safe_claim_policy_name("__"));
+        assert!(!IamSys::<StsTestMockStore>::is_safe_claim_policy_name("---"));
+        assert!(!IamSys::<StsTestMockStore>::is_safe_claim_policy_name("-_-"));
+        assert!(!IamSys::<StsTestMockStore>::is_safe_claim_policy_name("-.:_"));
+    }
 }

crates/io-core/src/deadlock_detector.rs

@@ -155,7 +155,7 @@ impl DeadlockDetector {
     /// Register a new lock.
     pub fn register_lock(&self, lock_type: LockType) -> u64 {
         let id = {
-            let mut next = self.next_lock_id.lock().unwrap();
+            let mut next = self.next_lock_id.lock().unwrap_or_else(|e| e.into_inner());
             *next += 1;
             *next
         };
@@ -243,7 +243,10 @@ impl DeadlockDetector {
             return None;
         }
 
-        let graph = self.wait_graph.lock().unwrap();
+        let graph = match self.wait_graph.lock() {
+            Ok(g) => g,
+            Err(_) => return None,
+        };
 
         // Build adjacency list
         let mut adj: HashMap<u64, Vec<u64>> = HashMap::new();
@@ -310,7 +313,10 @@ impl DeadlockDetector {
             return Vec::new();
         }
 
-        let locks = self.locks.lock().unwrap();
+        let locks = match self.locks.lock() {
+            Ok(l) => l,
+            Err(_) => return Vec::new(),
+        };
         let mut result = Vec::new();
 
         for (&id, info) in locks.iter() {
@@ -349,13 +355,13 @@ impl DeadlockDetector {
 
     /// Get lock info.
     pub fn get_lock_info(&self, lock_id: u64) -> Option<LockInfo> {
-        let locks = self.locks.lock().unwrap();
+        let locks = self.locks.lock().ok()?;
         locks.get(&lock_id).cloned()
     }
 
     /// Get total number of registered locks.
     pub fn lock_count(&self) -> usize {
-        let locks = self.locks.lock().unwrap();
+        let locks = self.locks.lock().unwrap_or_else(|e| e.into_inner());
         locks.len()
     }
 }

crates/io-core/src/pool.rs

@@ -226,8 +226,9 @@ impl BytesPool {
     pub async fn acquire_buffer(&self, size: usize) -> PooledBuffer {
         let tier = self.select_tier(size);
         let mut buffer = tier.acquire_buffer(size, &self.metrics).await;
-        // Set tier reference for return on drop
-        buffer.tier = Some(Arc::clone(tier));
+        if buffer._permit.is_some() {
+            buffer.tier = Some(Arc::clone(tier));
+        }
         buffer
     }
 
@@ -304,12 +305,12 @@ impl PoolTier {
     }
 
     fn set_metrics(&self, metrics: Arc<BytesPoolMetrics>) {
-        *self.metrics.lock().unwrap() = Some(metrics);
+        *self.metrics.lock().unwrap_or_else(|e| e.into_inner()) = Some(metrics);
     }
 
     fn take_or_allocate_buffer(&self, size: usize, pool_metrics: &BytesPoolMetrics) -> (BytesMut, bool) {
         let buffer_opt = {
-            let mut available = self.available_buffers.lock().unwrap();
+            let mut available = self.available_buffers.lock().unwrap_or_else(|e| e.into_inner());
             available.pop()
         };
         let was_reused = buffer_opt.is_some();
@@ -368,10 +369,20 @@ impl PoolTier {
 
     async fn acquire_buffer(&self, size: usize, pool_metrics: &BytesPoolMetrics) -> PooledBuffer {
         // Acquire semaphore permit (owned for storage in PooledBuffer)
-        let permit = Arc::clone(&self.semaphore).acquire_owned().await.unwrap();
+        let permit = match Arc::clone(&self.semaphore).acquire_owned().await {
+            Ok(p) => p,
+            Err(_) => {
+                let buffer = BytesMut::with_capacity(size.max(self.buffer_size));
+                return PooledBuffer {
+                    buffer: ManuallyDrop::new(buffer),
+                    tier: None,
+                    _permit: None,
+                };
+            }
+        };
 
         // Use the pool's shared metrics for recording
-        let _metrics_lock = self.metrics.lock().unwrap();
+        let _metrics_lock = self.metrics.lock().unwrap_or_else(|e| e.into_inner());
         let _metrics = _metrics_lock.as_ref().unwrap();
 
         // Record acquisition
@@ -394,7 +405,7 @@ impl PoolTier {
         let permit = Arc::clone(&self.semaphore).try_acquire_owned().ok()?;
 
         // Use the pool's shared metrics for recording
-        let _metrics_lock = self.metrics.lock().unwrap();
+        let _metrics_lock = self.metrics.lock().unwrap_or_else(|e| e.into_inner());
         let _metrics = _metrics_lock.as_ref().unwrap();
 
         // Record acquisition
@@ -414,11 +425,11 @@ impl PoolTier {
 
     /// Return a buffer to the pool for reuse.
     fn return_buffer(&self, buffer: BytesMut) {
-        let mut available = self.available_buffers.lock().unwrap();
+        let mut available = self.available_buffers.lock().unwrap_or_else(|e| e.into_inner());
         // Limit the size of the pool to prevent unbounded growth
         if available.len() < self.max_buffers {
             available.push(buffer);
-            if let Some(ref metrics) = *self.metrics.lock().unwrap() {
+            if let Some(ref metrics) = *self.metrics.lock().unwrap_or_else(|e| e.into_inner()) {
                 metrics.available_buffers.fetch_add(1, Ordering::Relaxed);
             }
         } else {
@@ -428,7 +439,7 @@ impl PoolTier {
                     Some(current.saturating_sub(released_bytes))
                 })
                 .ok();
-            if let Some(ref metrics) = *self.metrics.lock().unwrap() {
+            if let Some(ref metrics) = *self.metrics.lock().unwrap_or_else(|e| e.into_inner()) {
                 metrics
                     .current_allocated_bytes
                     .fetch_update(Ordering::Relaxed, Ordering::Relaxed, |current| {
@@ -447,11 +458,10 @@ impl Drop for PooledBuffer {
     // buffer moves it exactly once into the pool when a tier still owns it.
     #[allow(unsafe_code)]
     fn drop(&mut self) {
-        // Return buffer to pool if tier reference exists
+        // Return buffer to pool if tier reference exists.
+        // Otherwise, drop the standalone fallback buffer normally.
+        let buffer = unsafe { ManuallyDrop::take(&mut self.buffer) };
         if let Some(ref tier) = self.tier {
-            // SAFETY: We're in drop(), so this is the last use of the buffer
-            // ManuallyDrop allows us to take the value without running BytesMut's drop
-            let buffer = unsafe { ManuallyDrop::take(&mut self.buffer) };
             tier.return_buffer(buffer);
         }
         // The permit is automatically dropped here, releasing the semaphore slot
@@ -503,7 +513,10 @@ impl std::fmt::Debug for PoolTier {
             .field("buffer_size", &self.buffer_size)
             .field("max_buffers", &self.max_buffers)
             .field("available_permits", &self.semaphore.available_permits())
-            .field("available_buffers", &self.available_buffers.lock().unwrap().len())
+            .field(
+                "available_buffers",
+                &self.available_buffers.lock().unwrap_or_else(|e| e.into_inner()).len(),
+            )
             .finish()
     }
 }
@@ -526,6 +539,20 @@ mod tests {
         assert!(buffer.capacity() >= 2048);
     }
 
+    #[tokio::test]
+    async fn test_acquire_buffer_after_shutdown_is_unpooled() {
+        let pool = BytesPool::new_tiered();
+        pool.small_pool.semaphore.close();
+
+        let buffer = pool.acquire_buffer(2048).await;
+
+        assert!(buffer.tier.is_none());
+        assert!(buffer._permit.is_none());
+        assert!(buffer.capacity() >= pool.small_pool.buffer_size);
+        drop(buffer);
+        assert_eq!(pool.available_buffers(), 0);
+    }
+
     #[tokio::test]
     async fn test_tier_selection() {
         let pool = BytesPool::new_tiered();

crates/io-metrics/src/lib.rs

@@ -112,7 +112,8 @@ pub use deadlock_metrics::{
 // Lock metrics exports
 pub use lock_metrics::{
     LockMetricsSummary, record_contention_event, record_early_release, record_lock_hold_time, record_lock_optimization_enabled,
-    record_spin_attempt, record_spin_count_change,
+    record_object_lock_diag_acquire_duration, record_object_lock_diag_enabled, record_object_lock_diag_hold_duration,
+    record_object_lock_diag_slow_acquire, record_object_lock_diag_slow_hold, record_spin_attempt, record_spin_count_change,
 };
 
 pub use process_lock_metrics::{

crates/io-metrics/src/lock_metrics.rs

@@ -62,6 +62,61 @@ pub fn record_contention_event() {
     counter!("rustfs_lock_contentions").increment(1);
 }
 
+/// Record object namespace lock diagnostics being enabled.
+#[inline(always)]
+pub fn record_object_lock_diag_enabled(enabled: bool) {
+    use metrics::gauge;
+    gauge!("rustfs_object_lock_diag_enabled").set(if enabled { 1.0 } else { 0.0 });
+}
+
+/// Record object namespace lock acquire duration.
+#[inline(always)]
+pub fn record_object_lock_diag_acquire_duration(op: &'static str, mode: &'static str, duration: Duration) {
+    use metrics::histogram;
+    histogram!(
+        "rustfs_object_lock_diag_acquire_duration_seconds",
+        "op" => op,
+        "mode" => mode
+    )
+    .record(duration.as_secs_f64());
+}
+
+/// Record object namespace lock hold duration.
+#[inline(always)]
+pub fn record_object_lock_diag_hold_duration(op: &'static str, mode: &'static str, duration: Duration) {
+    use metrics::histogram;
+    histogram!(
+        "rustfs_object_lock_diag_hold_duration_seconds",
+        "op" => op,
+        "mode" => mode
+    )
+    .record(duration.as_secs_f64());
+}
+
+/// Record an object namespace lock slow-acquire event.
+#[inline(always)]
+pub fn record_object_lock_diag_slow_acquire(op: &'static str, mode: &'static str) {
+    use metrics::counter;
+    counter!(
+        "rustfs_object_lock_diag_slow_acquire_total",
+        "op" => op,
+        "mode" => mode
+    )
+    .increment(1);
+}
+
+/// Record an object namespace lock slow-hold event.
+#[inline(always)]
+pub fn record_object_lock_diag_slow_hold(op: &'static str, mode: &'static str) {
+    use metrics::counter;
+    counter!(
+        "rustfs_object_lock_diag_slow_hold_total",
+        "op" => op,
+        "mode" => mode
+    )
+    .increment(1);
+}
+
 /// Lock statistics summary.
 #[derive(Debug, Clone, Default)]
 pub struct LockMetricsSummary {
@@ -108,6 +163,97 @@ impl LockMetricsSummary {
 #[cfg(test)]
 mod tests {
     use super::*;
+    use metrics::{Counter, CounterFn, Gauge, GaugeFn, Histogram, HistogramFn, Key, KeyName, Metadata, SharedString, Unit};
+    use std::sync::{Arc, Mutex};
+
+    #[derive(Clone, Default)]
+    struct SeenMetricsRecorder {
+        counters: Arc<Mutex<Vec<Key>>>,
+        gauges: Arc<Mutex<Vec<Key>>>,
+        histograms: Arc<Mutex<Vec<Key>>>,
+    }
+
+    impl SeenMetricsRecorder {
+        fn saw_counter_named(&self, name: &str) -> bool {
+            self.counters
+                .lock()
+                .expect("counter key collection should be lockable")
+                .iter()
+                .any(|key| key.name() == name)
+        }
+
+        fn saw_gauge_named(&self, name: &str) -> bool {
+            self.gauges
+                .lock()
+                .expect("gauge key collection should be lockable")
+                .iter()
+                .any(|key| key.name() == name)
+        }
+
+        fn saw_histogram_named(&self, name: &str) -> bool {
+            self.histograms
+                .lock()
+                .expect("histogram key collection should be lockable")
+                .iter()
+                .any(|key| key.name() == name)
+        }
+    }
+
+    impl metrics::Recorder for SeenMetricsRecorder {
+        fn describe_counter(&self, _key: KeyName, _unit: Option<Unit>, _description: SharedString) {}
+
+        fn describe_gauge(&self, _key: KeyName, _unit: Option<Unit>, _description: SharedString) {}
+
+        fn describe_histogram(&self, _key: KeyName, _unit: Option<Unit>, _description: SharedString) {}
+
+        fn register_counter(&self, key: &Key, _metadata: &Metadata<'_>) -> Counter {
+            self.counters
+                .lock()
+                .expect("counter key collection should be lockable")
+                .push(key.clone());
+            Counter::from_arc(Arc::new(NoopCounter))
+        }
+
+        fn register_gauge(&self, key: &Key, _metadata: &Metadata<'_>) -> Gauge {
+            self.gauges
+                .lock()
+                .expect("gauge key collection should be lockable")
+                .push(key.clone());
+            Gauge::from_arc(Arc::new(NoopGauge))
+        }
+
+        fn register_histogram(&self, key: &Key, _metadata: &Metadata<'_>) -> Histogram {
+            self.histograms
+                .lock()
+                .expect("histogram key collection should be lockable")
+                .push(key.clone());
+            Histogram::from_arc(Arc::new(NoopHistogram))
+        }
+    }
+
+    struct NoopCounter;
+
+    impl CounterFn for NoopCounter {
+        fn increment(&self, _value: u64) {}
+
+        fn absolute(&self, _value: u64) {}
+    }
+
+    struct NoopGauge;
+
+    impl GaugeFn for NoopGauge {
+        fn increment(&self, _value: f64) {}
+
+        fn decrement(&self, _value: f64) {}
+
+        fn set(&self, _value: f64) {}
+    }
+
+    struct NoopHistogram;
+
+    impl HistogramFn for NoopHistogram {
+        fn record(&self, _value: f64) {}
+    }
 
     #[test]
     fn test_record_lock_optimization_enabled() {
@@ -143,6 +289,60 @@ mod tests {
         record_contention_event();
     }
 
+    #[test]
+    fn test_record_object_lock_diag_enabled() {
+        let recorder = SeenMetricsRecorder::default();
+        metrics::with_local_recorder(&recorder, || {
+            record_object_lock_diag_enabled(true);
+            record_object_lock_diag_enabled(false);
+        });
+        assert!(
+            recorder.saw_gauge_named("rustfs_object_lock_diag_enabled"),
+            "expected object lock diagnostics enabled gauge to be emitted"
+        );
+    }
+
+    #[test]
+    fn test_record_object_lock_diag_acquire_duration() {
+        let recorder = SeenMetricsRecorder::default();
+        metrics::with_local_recorder(&recorder, || {
+            record_object_lock_diag_acquire_duration("get_object", "read", Duration::from_millis(10));
+        });
+        assert!(
+            recorder.saw_histogram_named("rustfs_object_lock_diag_acquire_duration_seconds"),
+            "expected object lock diagnostics acquire histogram to be emitted"
+        );
+    }
+
+    #[test]
+    fn test_record_object_lock_diag_hold_duration() {
+        let recorder = SeenMetricsRecorder::default();
+        metrics::with_local_recorder(&recorder, || {
+            record_object_lock_diag_hold_duration("put_object_commit", "write", Duration::from_millis(20));
+        });
+        assert!(
+            recorder.saw_histogram_named("rustfs_object_lock_diag_hold_duration_seconds"),
+            "expected object lock diagnostics hold histogram to be emitted"
+        );
+    }
+
+    #[test]
+    fn test_record_object_lock_diag_slow_events() {
+        let recorder = SeenMetricsRecorder::default();
+        metrics::with_local_recorder(&recorder, || {
+            record_object_lock_diag_slow_acquire("get_object_info", "read");
+            record_object_lock_diag_slow_hold("complete_multipart_upload_commit", "write");
+        });
+        assert!(
+            recorder.saw_counter_named("rustfs_object_lock_diag_slow_acquire_total"),
+            "expected object lock diagnostics slow-acquire counter to be emitted"
+        );
+        assert!(
+            recorder.saw_counter_named("rustfs_object_lock_diag_slow_hold_total"),
+            "expected object lock diagnostics slow-hold counter to be emitted"
+        );
+    }
+
     #[test]
     fn test_lock_metrics_summary() {
         let mut summary = LockMetricsSummary::new();

crates/lock/src/distributed_lock.rs

@@ -30,8 +30,8 @@ use uuid::Uuid;
 
 const UNLOCK_RETRY_ATTEMPTS: usize = 3;
 const UNLOCK_RETRY_BACKOFF: Duration = Duration::from_millis(100);
-const LOCK_ACQUIRE_RETRY_ATTEMPTS: usize = 3;
 const LOCK_ACQUIRE_RETRY_INITIAL_BACKOFF: Duration = Duration::from_millis(250);
+const LOCK_ACQUIRE_ATTEMPT_TIMEOUT: Duration = Duration::from_secs(1);
 const REMOTE_LOCK_RPC_FAILED_PREFIX: &str = "remote lock rpc failed:";
 const REMOTE_LOCK_RPC_TIMED_OUT_PREFIX: &str = "remote lock rpc timed out:";
 const UNRECOVERABLE_QUORUM_FAILURE_PREFIX: &str = "unrecoverable quorum failure";
@@ -56,6 +56,10 @@ fn is_remote_lock_rpc_failure(error: &str) -> bool {
         || has_case_insensitive_prefix(error, REMOTE_LOCK_RPC_TIMED_OUT_PREFIX)
 }
 
+fn is_remote_lock_rpc_timeout(error: &str) -> bool {
+    has_case_insensitive_prefix(error, REMOTE_LOCK_RPC_TIMED_OUT_PREFIX)
+}
+
 fn has_case_insensitive_prefix(error: &str, expected_prefix: &str) -> bool {
     error
         .get(0..expected_prefix.len())
@@ -80,6 +84,17 @@ fn classify_lock_failure(error: &str) -> LockAcquireFailureKind {
     LockAcquireFailureKind::NonRetryable
 }
 
+fn should_warn_lock_failure(error: &str) -> bool {
+    if is_remote_lock_rpc_failure(error) {
+        return !is_remote_lock_rpc_timeout(error);
+    }
+
+    matches!(
+        classify_lock_failure(error),
+        LockAcquireFailureKind::NonRetryable | LockAcquireFailureKind::UnrecoverableQuorum
+    )
+}
+
 /// A RAII guard for distributed locks that releases the lock asynchronously when dropped.
 #[derive(Debug)]
 pub struct DistributedLockGuard {
@@ -350,13 +365,15 @@ impl DistributedLock {
     }
 
     fn lock_acquire_retry_backoff(attempt: usize) -> Duration {
-        LOCK_ACQUIRE_RETRY_INITIAL_BACKOFF * attempt as u32
+        LOCK_ACQUIRE_RETRY_INITIAL_BACKOFF.saturating_mul(attempt.try_into().unwrap_or(u32::MAX))
     }
 
-    fn is_retryable_lock_failure(resp: &LockResponse) -> bool {
-        resp.error
-            .as_deref()
-            .is_some_and(|error| matches!(classify_lock_failure(error), LockAcquireFailureKind::RetryableContention))
+    fn lock_acquire_attempt_timeout(&self, remaining: Duration) -> Duration {
+        if self.clients.len() <= 1 {
+            remaining
+        } else {
+            remaining.min(LOCK_ACQUIRE_ATTEMPT_TIMEOUT)
+        }
     }
 
     async fn release_entries(entries: Vec<(LockId, Arc<dyn LockClient>)>, context: &'static str) {
@@ -461,7 +478,7 @@ impl DistributedLock {
     }
 
     fn log_failed_lock_response(&self, request: &LockRequest, idx: usize, error: String) {
-        if request.suppress_contention_logs {
+        if request.suppress_contention_logs || !should_warn_lock_failure(&error) {
             tracing::debug!(
                 resource = %request.resource,
                 owner = %request.owner,
@@ -493,15 +510,11 @@ impl DistributedLock {
 
             let remaining = request.acquire_timeout - elapsed;
             let mut attempt_request = request.clone();
-            attempt_request.acquire_timeout = remaining;
+            attempt_request.acquire_timeout = self.lock_acquire_attempt_timeout(remaining);
             attempt_request.lock_id = LockId::new_unique(&request.resource);
 
             let result = self.acquire_lock_quorum_once(&attempt_request).await?;
-            if result.response.success
-                || !result.individual_locks.is_empty()
-                || !Self::is_retryable_lock_failure(&result.response)
-                || attempt >= LOCK_ACQUIRE_RETRY_ATTEMPTS
-            {
+            if result.response.success || !matches!(result.failure_kind, Some(LockAcquireFailureKind::RetryableContention)) {
                 return Ok(result);
             }
 
@@ -522,6 +535,42 @@ impl DistributedLock {
         }))
     }
 
+    fn lock_acquire_timeout_result(timeout: Duration) -> LockAcquireQuorumResult {
+        LockAcquireQuorumResult {
+            response: LockResponse::failure("Lock acquisition timeout", timeout),
+            individual_locks: Vec::new(),
+            failure_kind: Some(LockAcquireFailureKind::RetryableContention),
+        }
+    }
+
+    fn lock_acquire_attempt_timeout_result(
+        timeout: Duration,
+        individual_locks: Vec<(LockId, Arc<dyn LockClient>)>,
+        last_failure: Option<String>,
+    ) -> LockAcquireQuorumResult {
+        let last_failure_kind = last_failure.as_deref().map(classify_lock_failure);
+        let failure_kind =
+            if let Some(kind @ (LockAcquireFailureKind::NonRetryable | LockAcquireFailureKind::UnrecoverableQuorum)) =
+                last_failure_kind
+            {
+                kind
+            } else {
+                return Self::lock_acquire_timeout_result(timeout);
+            };
+
+        let mut error = "Lock acquisition timeout".to_string();
+        if let Some(last_failure) = last_failure {
+            error.push_str("; last failure: ");
+            error.push_str(&last_failure);
+        }
+
+        LockAcquireQuorumResult {
+            response: LockResponse::failure(error, timeout),
+            individual_locks,
+            failure_kind: Some(failure_kind),
+        }
+    }
+
     /// Quorum-based lock acquisition: success if at least the required quorum succeeds.
     /// Collects all individual lock_ids from successful clients and creates an aggregate lock_id.
     /// Returns the LockResponse with aggregate lock_id and individual lock mappings.
@@ -532,8 +581,44 @@ impl DistributedLock {
         let fallback_lock_id = request.lock_id.clone();
         let mut last_failure = None;
         let mut hard_failures = 0usize;
+        let start = tokio::time::Instant::now();
+
+        while !pending.is_empty() {
+            let remaining = request.acquire_timeout.saturating_sub(start.elapsed());
+            if remaining.is_zero() {
+                Self::spawn_release_cleanup(individual_locks.clone(), "distributed_lock_attempt_timeout");
+                Self::spawn_pending_cleanup(
+                    pending,
+                    self.clients.clone(),
+                    fallback_lock_id.clone(),
+                    "distributed_lock_attempt_timeout_cleanup",
+                );
+                return Ok(Self::lock_acquire_attempt_timeout_result(
+                    request.acquire_timeout,
+                    individual_locks,
+                    last_failure,
+                ));
+            }
+
+            let join_result = match tokio::time::timeout(remaining, pending.join_next()).await {
+                Ok(Some(join_result)) => join_result,
+                Ok(None) => break,
+                Err(_) => {
+                    Self::spawn_release_cleanup(individual_locks.clone(), "distributed_lock_attempt_timeout");
+                    Self::spawn_pending_cleanup(
+                        pending,
+                        self.clients.clone(),
+                        fallback_lock_id.clone(),
+                        "distributed_lock_attempt_timeout_cleanup",
+                    );
+                    return Ok(Self::lock_acquire_attempt_timeout_result(
+                        request.acquire_timeout,
+                        individual_locks,
+                        last_failure,
+                    ));
+                }
+            };
 
-        while let Some(join_result) = pending.join_next().await {
             match join_result {
                 Ok((idx, Ok(resp))) => {
                     if resp.success {
@@ -550,7 +635,7 @@ impl DistributedLock {
                         }
                     } else {
                         let error = resp.error.unwrap_or_else(|| "unknown error".to_string());
-                        if is_remote_lock_rpc_failure(&error) {
+                        if is_remote_lock_rpc_failure(&error) && !is_remote_lock_rpc_timeout(&error) {
                             hard_failures += 1;
                         }
                         self.log_failed_lock_response(request, idx, error.clone());
@@ -710,7 +795,10 @@ fn record_lock_held_release(lock_type: LockType) {
 
 #[cfg(test)]
 mod tests {
-    use super::{DistributedLock, is_remote_lock_rpc_failure};
+    use super::{
+        DistributedLock, LOCK_ACQUIRE_ATTEMPT_TIMEOUT, LockAcquireFailureKind, is_remote_lock_rpc_failure,
+        should_warn_lock_failure,
+    };
     use crate::{LockError, LockId, LockInfo, LockRequest, LockResponse, LockStats, LockType, ObjectKey, client::LockClient};
     use std::{
         collections::{HashMap, VecDeque},
@@ -784,6 +872,61 @@ mod tests {
         }
     }
 
+    #[derive(Debug)]
+    struct TimeoutRecordingClient {
+        seen_timeouts: Arc<Mutex<Vec<Duration>>>,
+    }
+
+    impl TimeoutRecordingClient {
+        fn new(seen_timeouts: Arc<Mutex<Vec<Duration>>>) -> Self {
+            Self { seen_timeouts }
+        }
+
+        fn into_client(self) -> Arc<dyn LockClient> {
+            Arc::new(self)
+        }
+    }
+
+    #[async_trait::async_trait]
+    impl LockClient for TimeoutRecordingClient {
+        async fn acquire_lock(&self, request: &LockRequest) -> crate::Result<LockResponse> {
+            self.seen_timeouts.lock().unwrap().push(request.acquire_timeout);
+            Ok(LockResponse::failure("Lock acquisition timeout", Duration::ZERO))
+        }
+
+        async fn release(&self, _lock_id: &LockId) -> crate::Result<bool> {
+            Ok(false)
+        }
+
+        async fn refresh(&self, _lock_id: &LockId) -> crate::Result<bool> {
+            Ok(false)
+        }
+
+        async fn force_release(&self, _lock_id: &LockId) -> crate::Result<bool> {
+            Ok(false)
+        }
+
+        async fn check_status(&self, _lock_id: &LockId) -> crate::Result<Option<LockInfo>> {
+            Ok(None)
+        }
+
+        async fn get_stats(&self) -> crate::Result<LockStats> {
+            Ok(LockStats::default())
+        }
+
+        async fn close(&self) -> crate::Result<()> {
+            Ok(())
+        }
+
+        async fn is_online(&self) -> bool {
+            true
+        }
+
+        async fn is_local(&self) -> bool {
+            false
+        }
+    }
+
     #[derive(Debug, Clone, Copy)]
     enum AcquirePlan {
         Success { delay: Duration },
@@ -894,6 +1037,15 @@ mod tests {
         assert!(!is_remote_lock_rpc_failure("acquired lock failed for other reason"));
     }
 
+    #[test]
+    fn test_should_warn_lock_failure() {
+        assert!(should_warn_lock_failure("Remote lock RPC failed: backend unreachable"));
+        assert!(should_warn_lock_failure("permission denied"));
+        assert!(should_warn_lock_failure("Unrecoverable quorum failure: 1/3 required"));
+        assert!(!should_warn_lock_failure("Remote lock RPC timed out: RPC timed out after 50ms"));
+        assert!(!should_warn_lock_failure("Lock acquisition timeout"));
+    }
+
     #[tokio::test]
     async fn acquire_guard_returns_quorum_error_when_rpc_failures_make_quorum_impossible() {
         let clients: Vec<Arc<dyn LockClient>> = vec![
@@ -926,7 +1078,7 @@ mod tests {
     }
 
     #[tokio::test]
-    async fn acquire_guard_returns_quorum_error_when_rpc_timeouts_make_quorum_impossible() {
+    async fn acquire_guard_retries_rpc_timeouts_until_caller_timeout() {
         let clients: Vec<Arc<dyn LockClient>> = vec![
             ResponseClient::new(LockResponse::failure(
                 "Remote lock RPC timed out: RPC timed out after 50ms",
@@ -947,19 +1099,16 @@ mod tests {
         ];
         let lock = DistributedLock::new("test".to_string(), clients, 3);
         let request = LockRequest::new(ObjectKey::new("bucket", "object"), LockType::Exclusive, "owner")
-            .with_acquire_timeout(Duration::from_secs(1));
+            .with_acquire_timeout(Duration::from_millis(900));
 
+        let started = tokio::time::Instant::now();
         let result = lock.acquire_guard(&request).await;
+        let elapsed = started.elapsed();
 
+        assert!(matches!(result, Ok(None)), "unexpected result: {result:?}");
         assert!(
-            matches!(
-                result,
-                Err(LockError::QuorumNotReached {
-                    required: 3,
-                    achieved: 0
-                })
-            ),
-            "unexpected result: {result:?}"
+            elapsed >= Duration::from_millis(250),
+            "remote RPC timeouts should be retried within the caller timeout, got {elapsed:?}"
         );
     }
 
@@ -977,7 +1126,7 @@ mod tests {
         ];
         let lock = DistributedLock::new("test".to_string(), clients, 3);
         let request = LockRequest::new(ObjectKey::new("bucket", "object"), LockType::Exclusive, "owner")
-            .with_acquire_timeout(Duration::from_secs(2));
+            .with_acquire_timeout(Duration::from_millis(120));
 
         let started = tokio::time::Instant::now();
         let result = lock.acquire_guard(&request).await;
@@ -989,6 +1138,90 @@ mod tests {
         );
     }
 
+    #[tokio::test]
+    async fn acquire_guard_retries_attempt_timeout_when_hard_failure_does_not_preclude_quorum() {
+        let clients: Vec<Arc<dyn LockClient>> = vec![
+            Arc::new(SequencedClient::new(
+                vec![
+                    AcquirePlan::Failure {
+                        error: "Remote lock RPC failed: node unavailable",
+                        delay: Duration::ZERO,
+                    },
+                    AcquirePlan::Success { delay: Duration::ZERO },
+                ],
+                Arc::new(Mutex::new(Vec::new())),
+            )),
+            Arc::new(SequencedClient::new(
+                vec![
+                    AcquirePlan::Success {
+                        delay: Duration::from_millis(1200),
+                    },
+                    AcquirePlan::Success { delay: Duration::ZERO },
+                ],
+                Arc::new(Mutex::new(Vec::new())),
+            )),
+            Arc::new(SequencedClient::new(
+                vec![
+                    AcquirePlan::Success {
+                        delay: Duration::from_millis(1200),
+                    },
+                    AcquirePlan::Success { delay: Duration::ZERO },
+                ],
+                Arc::new(Mutex::new(Vec::new())),
+            )),
+            Arc::new(SequencedClient::new(
+                vec![
+                    AcquirePlan::Success {
+                        delay: Duration::from_millis(1200),
+                    },
+                    AcquirePlan::Success { delay: Duration::ZERO },
+                ],
+                Arc::new(Mutex::new(Vec::new())),
+            )),
+        ];
+        let lock = DistributedLock::new("test".to_string(), clients, 3);
+        let request = LockRequest::new(ObjectKey::new("bucket", "object"), LockType::Exclusive, "owner")
+            .with_acquire_timeout(Duration::from_secs(2));
+
+        let guard = lock
+            .acquire_guard(&request)
+            .await
+            .expect("attempt timeout with possible quorum should retry")
+            .expect("second attempt should acquire quorum");
+
+        assert_eq!(guard.entries.len(), 3);
+        drop(guard);
+    }
+
+    #[tokio::test]
+    async fn acquire_quorum_timeout_preserves_non_retryable_failure() {
+        let clients: Vec<Arc<dyn LockClient>> = vec![
+            ResponseClient::new(LockResponse::failure("permission denied", Duration::ZERO)).into_client(),
+            ResponseClient::new(LockResponse::failure("lock already held", Duration::ZERO))
+                .with_delay(Duration::from_secs(1))
+                .into_client(),
+            ResponseClient::new(LockResponse::failure("lock already held", Duration::ZERO))
+                .with_delay(Duration::from_secs(1))
+                .into_client(),
+        ];
+        let lock = DistributedLock::new("test".to_string(), clients, 2);
+        let request = LockRequest::new(ObjectKey::new("bucket", "object"), LockType::Exclusive, "owner")
+            .with_acquire_timeout(Duration::from_millis(120));
+
+        let result = lock.acquire_lock_quorum_with_retry(&request).await.unwrap();
+
+        assert!(matches!(result.failure_kind, Some(LockAcquireFailureKind::NonRetryable)));
+        assert!(
+            result
+                .response
+                .error
+                .as_deref()
+                .is_some_and(|error| error.contains("permission denied")),
+            "unexpected response: {:?}",
+            result.response
+        );
+    }
+
     #[tokio::test]
     async fn acquire_guard_retries_transient_timeout_before_quorum() {
         let clients: Vec<Arc<dyn LockClient>> = vec![
@@ -1028,6 +1261,113 @@ mod tests {
         );
     }
 
+    #[tokio::test]
+    async fn acquire_guard_uses_bounded_distributed_attempt_timeout() {
+        let seen_timeouts = Arc::new(Mutex::new(Vec::new()));
+        let clients: Vec<Arc<dyn LockClient>> = (0..4)
+            .map(|_| TimeoutRecordingClient::new(seen_timeouts.clone()).into_client())
+            .collect();
+        let lock = DistributedLock::new("test".to_string(), clients, 3);
+        let request = LockRequest::new(ObjectKey::new("bucket", "object"), LockType::Exclusive, "owner")
+            .with_acquire_timeout(Duration::from_millis(1500));
+
+        let result = lock.acquire_guard(&request).await;
+
+        assert!(matches!(result, Ok(None)), "unexpected result: {result:?}");
+        let seen_timeouts = seen_timeouts.lock().unwrap();
+        assert!(!seen_timeouts.is_empty(), "expected lock clients to observe acquire timeouts");
+        assert!(
+            seen_timeouts.iter().all(|timeout| *timeout <= LOCK_ACQUIRE_ATTEMPT_TIMEOUT),
+            "distributed attempts should be bounded by {LOCK_ACQUIRE_ATTEMPT_TIMEOUT:?}, saw {seen_timeouts:?}"
+        );
+    }
+
+    #[tokio::test]
+    async fn acquire_guard_enforces_attempt_timeout_when_clients_ignore_budget() {
+        let clients: Vec<Arc<dyn LockClient>> = (0..4)
+            .map(|_| {
+                ResponseClient::new(LockResponse::failure("lock already held", Duration::ZERO))
+                    .with_delay(Duration::from_secs(5))
+                    .into_client()
+            })
+            .collect();
+        let lock = DistributedLock::new("test".to_string(), clients, 3);
+        let request = LockRequest::new(ObjectKey::new("bucket", "object"), LockType::Exclusive, "owner")
+            .with_acquire_timeout(Duration::from_millis(200));
+
+        let result = tokio::time::timeout(Duration::from_millis(800), lock.acquire_guard(&request)).await;
+
+        assert!(matches!(result, Ok(Ok(None))), "unexpected result: {result:?}");
+    }
+
+    #[tokio::test]
+    async fn acquire_guard_retries_partial_quorum_after_rollback() {
+        let seen_ids = Arc::new(Mutex::new(Vec::new()));
+        let clients: Vec<Arc<dyn LockClient>> = vec![
+            Arc::new(SequencedClient::new(
+                vec![
+                    AcquirePlan::Success { delay: Duration::ZERO },
+                    AcquirePlan::Success { delay: Duration::ZERO },
+                ],
+                seen_ids.clone(),
+            )),
+            Arc::new(SequencedClient::new(
+                vec![
+                    AcquirePlan::Failure {
+                        error: "Lock acquisition timeout",
+                        delay: Duration::ZERO,
+                    },
+                    AcquirePlan::Success { delay: Duration::ZERO },
+                ],
+                seen_ids.clone(),
+            )),
+            Arc::new(SequencedClient::new(
+                vec![
+                    AcquirePlan::Failure {
+                        error: "Lock acquisition timeout",
+                        delay: Duration::ZERO,
+                    },
+                    AcquirePlan::Success { delay: Duration::ZERO },
+                ],
+                seen_ids.clone(),
+            )),
+            Arc::new(SequencedClient::new(
+                vec![
+                    AcquirePlan::Failure {
+                        error: "Lock acquisition timeout",
+                        delay: Duration::ZERO,
+                    },
+                    AcquirePlan::Success { delay: Duration::ZERO },
+                ],
+                seen_ids.clone(),
+            )),
+        ];
+        let lock = DistributedLock::new("test".to_string(), clients, 3);
+        let request = LockRequest::new(ObjectKey::new("bucket", "object"), LockType::Exclusive, "owner")
+            .with_acquire_timeout(Duration::from_secs(2));
+
+        let guard = lock
+            .acquire_guard(&request)
+            .await
+            .expect("partial quorum retry should not fail")
+            .expect("second attempt should reach quorum");
+
+        let unique_id_count = seen_ids
+            .lock()
+            .unwrap()
+            .iter()
+            .fold(Vec::<String>::new(), |mut uuids, lock_id| {
+                if !uuids.iter().any(|uuid| uuid == &lock_id.uuid) {
+                    uuids.push(lock_id.uuid.clone());
+                }
+                uuids
+            })
+            .len();
+
+        assert!(unique_id_count >= 2, "partial quorum should retry with a fresh lock id");
+        drop(guard);
+    }
+
     #[tokio::test]
     async fn acquire_guard_returns_timeout_when_quorum_remains_contended() {
         let clients: Vec<Arc<dyn LockClient>> = vec![

crates/lock/src/namespace/mod.rs

@@ -52,6 +52,11 @@ impl NamespaceLockWrapper {
         Self { lock, resource, owner }
     }
 
+    /// Get lock owner identifier used for acquisition.
+    pub fn owner(&self) -> &str {
+        &self.owner
+    }
+
     /// Acquire write lock (exclusive lock) with timeout
     /// Returns the guard if acquisition succeeds, or an error if it fails
     pub async fn get_write_lock(&self, timeout: Duration) -> std::result::Result<NamespaceLockGuard, crate::error::LockError> {

crates/notify/AGENTS.md

@@ -28,6 +28,13 @@ shared plugin/runtime primitives from `rustfs-targets`.
 - `stream.rs` is a compatibility shim; new replay/runtime work should prefer
   shared helpers in `rustfs-targets::runtime`.
 
+## Concurrency
+
+- `runtime_view.rs` acquires locks in order: `stream_cancellers` → `target_list`.
+- `runtime_facade.rs` acquires locks in order: `replay_workers` → `target_list`.
+- These orders must not be reversed in new code. When adding a function that needs both `target_list` and `stream_cancellers`, acquire `stream_cancellers` first (matching `runtime_view.rs` order).
+- Do not hold write guards across `.await` points unless the hold time is bounded and the operation is unavoidably async.
+
 ## Change Style
 
 - Preserve best-effort dispatch semantics and observability signals unless the

crates/notify/src/rules/xml_config.rs

@@ -207,7 +207,7 @@ impl<'de> Deserialize<'de> for S3KeyFilter {
                             self.filter_rules = s3key_content.get_filter_rules();
                         }
                         _ => {
-                            map.next_value::<serde::de::IgnoredAny>()?;
+                            return Err(serde::de::Error::unknown_field(&key, &["S3Key"]));
                         }
                     }
                 }

crates/notify/src/runtime_facade.rs

@@ -81,8 +81,9 @@ impl NotifyRuntimeFacade {
     }
 
     pub async fn replace_targets(&self, activation: RuntimeActivation<Event>) -> Result<(), NotificationError> {
-        let mut target_list = self.target_list.write().await;
+        // Lock order: replay_workers -> target_list (matches notify AGENTS.md).
         let mut replay_workers = self.replay_workers.write().await;
+        let mut target_list = self.target_list.write().await;
         self.runtime_adapter
             .replace_runtime_targets(target_list.runtime_mut(), &mut replay_workers, activation)
             .await
@@ -102,8 +103,9 @@ impl NotifyRuntimeFacade {
         info!("Stops {} active event stream processing tasks", active_targets);
 
         {
-            let mut target_list = self.target_list.write().await;
+            // Lock order: replay_workers -> target_list (matches notify AGENTS.md).
             let mut replay_workers = self.replay_workers.write().await;
+            let mut target_list = self.target_list.write().await;
             if let Err(err) = self
                 .runtime_adapter
                 .shutdown(target_list.runtime_mut(), &mut replay_workers)

crates/obs/Cargo.toml

@@ -72,7 +72,8 @@ sysinfo = { workspace = true }
 nvml-wrapper = { workspace = true, optional = true }
 
 [target.'cfg(any(target_os = "linux", target_os = "macos"))'.dependencies]
-pyroscope = { workspace = true, features = ["backend-pprof-rs"] }
+pyroscope = { workspace = true }
+jemalloc_pprof = { workspace = true }
 
 
 [dev-dependencies]

crates/obs/src/metrics/collectors/scanner.rs

@@ -16,22 +16,47 @@
 
 //! Scanner metrics collector.
 //!
-//! Collects background scanner metrics including bucket scans,
+//! Collects background scanner metrics including bucket-drive scans,
 //! directory scans, and object scans.
 
 use crate::metrics::report::PrometheusMetric;
 use crate::metrics::schema::scanner::{
-    SCANNER_BUCKET_SCANS_FINISHED_MD, SCANNER_BUCKET_SCANS_STARTED_MD, SCANNER_DIRECTORIES_SCANNED_MD,
-    SCANNER_LAST_ACTIVITY_SECONDS_MD, SCANNER_OBJECTS_SCANNED_MD, SCANNER_VERSIONS_SCANNED_MD,
+    SCANNER_ACTIVE_PATHS_MD, SCANNER_BITROT_CYCLE_ENABLED_MD, SCANNER_BITROT_CYCLE_SECONDS_MD, SCANNER_BUCKET_SCANS_FAILED_MD,
+    SCANNER_BUCKET_SCANS_FINISHED_MD, SCANNER_BUCKET_SCANS_STARTED_MD, SCANNER_COMPLETED_CYCLES_MD,
+    SCANNER_CURRENT_CYCLE_AGE_SECONDS_MD, SCANNER_CURRENT_CYCLE_BUCKET_DRIVE_FAILURES_MD,
+    SCANNER_CURRENT_CYCLE_BUCKET_DRIVE_SCANS_MD, SCANNER_CURRENT_CYCLE_BUCKET_DRIVE_SCANS_PER_SECOND_MD,
+    SCANNER_CURRENT_CYCLE_DIRECTORIES_PER_SECOND_MD, SCANNER_CURRENT_CYCLE_DIRECTORIES_SCANNED_MD,
+    SCANNER_CURRENT_CYCLE_HEAL_OBJECTS_MD, SCANNER_CURRENT_CYCLE_ILM_ACTIONS_MD, SCANNER_CURRENT_CYCLE_MD,
+    SCANNER_CURRENT_CYCLE_OBJECTS_PER_SECOND_MD, SCANNER_CURRENT_CYCLE_OBJECTS_SCANNED_MD,
+    SCANNER_CURRENT_CYCLE_REPLICATION_CHECKS_MD, SCANNER_CURRENT_CYCLE_THROTTLE_SLEEP_DURATION_SECONDS_MD,
+    SCANNER_CURRENT_CYCLE_THROTTLE_SLEEP_EVENTS_MD, SCANNER_CURRENT_CYCLE_USAGE_SAVES_MD,
+    SCANNER_CURRENT_CYCLE_YIELD_DURATION_SECONDS_MD, SCANNER_CURRENT_CYCLE_YIELD_EVENTS_MD,
+    SCANNER_CURRENT_DISK_BUCKET_SCANS_ACTIVE_MD, SCANNER_CURRENT_DISK_BUCKET_SCANS_QUEUED_MD,
+    SCANNER_CURRENT_DISK_SCAN_CONCURRENCY_LIMIT_MD, SCANNER_CURRENT_SCAN_MODE_MD, SCANNER_CURRENT_SET_SCAN_CONCURRENCY_LIMIT_MD,
+    SCANNER_CURRENT_SET_SCANS_ACTIVE_MD, SCANNER_CURRENT_SET_SCANS_QUEUED_MD, SCANNER_CYCLE_INTERVAL_SECONDS_MD,
+    SCANNER_CYCLE_MAX_DIRECTORIES_MD, SCANNER_CYCLE_MAX_DURATION_SECONDS_MD, SCANNER_CYCLE_MAX_OBJECTS_MD,
+    SCANNER_DIRECTORIES_SCANNED_MD, SCANNER_FAILED_CYCLES_MD, SCANNER_LAST_ACTIVITY_SECONDS_MD,
+    SCANNER_LAST_CYCLE_BUCKET_DRIVE_FAILURES_MD, SCANNER_LAST_CYCLE_BUCKET_DRIVE_SCANS_MD,
+    SCANNER_LAST_CYCLE_BUCKET_DRIVE_SCANS_PER_SECOND_MD, SCANNER_LAST_CYCLE_DIRECTORIES_PER_SECOND_MD,
+    SCANNER_LAST_CYCLE_DIRECTORIES_SCANNED_MD, SCANNER_LAST_CYCLE_DURATION_SECONDS_MD, SCANNER_LAST_CYCLE_HEAL_OBJECTS_MD,
+    SCANNER_LAST_CYCLE_ILM_ACTIONS_MD, SCANNER_LAST_CYCLE_OBJECTS_PER_SECOND_MD, SCANNER_LAST_CYCLE_OBJECTS_SCANNED_MD,
+    SCANNER_LAST_CYCLE_PARTIAL_REASON_MD, SCANNER_LAST_CYCLE_REPLICATION_CHECKS_MD, SCANNER_LAST_CYCLE_RESULT_MD,
+    SCANNER_LAST_CYCLE_THROTTLE_SLEEP_DURATION_SECONDS_MD, SCANNER_LAST_CYCLE_THROTTLE_SLEEP_EVENTS_MD,
+    SCANNER_LAST_CYCLE_USAGE_SAVES_MD, SCANNER_LAST_CYCLE_YIELD_DURATION_SECONDS_MD, SCANNER_LAST_CYCLE_YIELD_EVENTS_MD,
+    SCANNER_OBJECTS_SCANNED_MD, SCANNER_OLDEST_ACTIVE_PATH_AGE_SECONDS_MD, SCANNER_PARTIAL_CYCLES_BY_REASON_MD,
+    SCANNER_PARTIAL_CYCLES_MD, SCANNER_THROTTLE_IDLE_MODE_ENABLED_MD, SCANNER_THROTTLE_MAX_SLEEP_SECONDS_MD,
+    SCANNER_THROTTLE_SLEEP_FACTOR_MD, SCANNER_VERSIONS_SCANNED_MD, SCANNER_YIELD_EVERY_N_OBJECTS_MD,
 };
 
 /// Scanner statistics.
 #[derive(Debug, Clone, Default)]
 pub struct ScannerStats {
-    /// Number of bucket scans finished
+    /// Number of bucket-drive scans finished
     pub bucket_scans_finished: u64,
-    /// Number of bucket scans started
+    /// Number of bucket-drive scans started
     pub bucket_scans_started: u64,
+    /// Number of bucket-drive scans that failed
+    pub bucket_scans_failed: u64,
     /// Number of directories scanned
     pub directories_scanned: u64,
     /// Number of objects scanned
@@ -40,6 +65,128 @@ pub struct ScannerStats {
     pub versions_scanned: u64,
     /// Seconds since last scanner activity
     pub last_activity_seconds: u64,
+    /// Number of scanner paths currently being processed
+    pub active_paths: u64,
+    /// Age in seconds of the oldest active scanner path update
+    pub oldest_active_path_age_seconds: u64,
+    /// Current aggregate set scan concurrency limit
+    pub current_set_scan_concurrency_limit: u64,
+    /// Current number of queued set scans
+    pub current_set_scans_queued: u64,
+    /// Current number of active set scans
+    pub current_set_scans_active: u64,
+    /// Current aggregate disk-bucket scan concurrency limit
+    pub current_disk_scan_concurrency_limit: u64,
+    /// Current number of queued disk-bucket scans
+    pub current_disk_bucket_scans_queued: u64,
+    /// Current number of active disk-bucket scans
+    pub current_disk_bucket_scans_active: u64,
+    /// Whether scanner idle-mode self-throttling is enabled
+    pub throttle_idle_mode_enabled: bool,
+    /// Effective scanner sleep factor
+    pub throttle_sleep_factor: f64,
+    /// Effective scanner maximum self-throttle sleep duration in seconds
+    pub throttle_max_sleep_seconds: f64,
+    /// Object interval for cooperative scanner runtime yields
+    pub yield_every_n_objects: u64,
+    /// Effective scanner cycle interval in seconds
+    pub cycle_interval_seconds: f64,
+    /// Effective maximum scanner cycle runtime in seconds
+    pub cycle_max_duration_seconds: f64,
+    /// Effective maximum objects processed by one scanner cycle
+    pub cycle_max_objects: u64,
+    /// Effective maximum directories entered by one scanner cycle
+    pub cycle_max_directories: u64,
+    /// Whether periodic scanner bitrot deep scans are enabled
+    pub bitrot_cycle_enabled: bool,
+    /// Effective scanner bitrot deep-scan interval in seconds
+    pub bitrot_cycle_seconds: f64,
+    /// Current scanner cycle number, or zero when idle
+    pub current_cycle: u64,
+    /// Number of scanner cycles completed since server start
+    pub completed_cycles: u64,
+    /// Seconds elapsed since the current scanner cycle started
+    pub current_cycle_age_seconds: u64,
+    /// Number of objects scanned by the currently running scanner cycle
+    pub current_cycle_objects_scanned: u64,
+    /// Number of directories scanned by the currently running scanner cycle
+    pub current_cycle_directories_scanned: u64,
+    /// Number of bucket-drive scans finished by the currently running scanner cycle
+    pub current_cycle_bucket_drive_scans: u64,
+    /// Number of bucket-drive scans that failed in the currently running scanner cycle
+    pub current_cycle_bucket_drive_failures: u64,
+    /// Object scan rate for the currently running scanner cycle
+    pub current_cycle_objects_per_second: f64,
+    /// Directory scan rate for the currently running scanner cycle
+    pub current_cycle_directories_per_second: f64,
+    /// Bucket-drive scan rate for the currently running scanner cycle
+    pub current_cycle_bucket_drive_scans_per_second: f64,
+    /// Number of scanner cooperative yield events in the current scanner cycle
+    pub current_cycle_yield_events: u64,
+    /// Total scanner cooperative yield duration in seconds for the current scanner cycle
+    pub current_cycle_yield_duration_seconds: f64,
+    /// Number of scanner self-throttle sleep events in the current scanner cycle
+    pub current_cycle_throttle_sleep_events: u64,
+    /// Total scanner self-throttle sleep duration in seconds for the current scanner cycle
+    pub current_cycle_throttle_sleep_duration_seconds: f64,
+    /// Number of lifecycle actions applied by the current scanner cycle
+    pub current_cycle_ilm_actions: u64,
+    /// Number of object heal candidates enqueued by the current scanner cycle
+    pub current_cycle_heal_objects: u64,
+    /// Number of replication heal checks run by the current scanner cycle
+    pub current_cycle_replication_checks: u64,
+    /// Number of data-usage save operations run by the current scanner cycle
+    pub current_cycle_usage_saves: u64,
+    /// Current scanner mode: 0 unknown or idle, 1 normal, 2 deep bitrot scan
+    pub current_scan_mode: u64,
+    /// Last scanner cycle result: 0 unknown, 1 success, 2 error, 3 partial
+    pub last_cycle_result: u64,
+    /// Last scanner partial cycle reason: 0 unknown, 1 runtime, 2 objects, 3 directories
+    pub last_cycle_partial_reason: u64,
+    /// Duration in seconds of the last finished scanner cycle
+    pub last_cycle_duration_seconds: f64,
+    /// Number of objects scanned by the last finished scanner cycle
+    pub last_cycle_objects_scanned: u64,
+    /// Number of directories scanned by the last finished scanner cycle
+    pub last_cycle_directories_scanned: u64,
+    /// Number of bucket-drive scans finished by the last scanner cycle
+    pub last_cycle_bucket_drive_scans: u64,
+    /// Number of bucket-drive scans that failed in the last finished scanner cycle
+    pub last_cycle_bucket_drive_failures: u64,
+    /// Object scan rate for the last finished scanner cycle
+    pub last_cycle_objects_per_second: f64,
+    /// Directory scan rate for the last finished scanner cycle
+    pub last_cycle_directories_per_second: f64,
+    /// Bucket-drive scan rate for the last finished scanner cycle
+    pub last_cycle_bucket_drive_scans_per_second: f64,
+    /// Number of scanner cooperative yield events in the last finished scanner cycle
+    pub last_cycle_yield_events: u64,
+    /// Total scanner cooperative yield duration in seconds for the last finished scanner cycle
+    pub last_cycle_yield_duration_seconds: f64,
+    /// Number of scanner self-throttle sleep events in the last finished scanner cycle
+    pub last_cycle_throttle_sleep_events: u64,
+    /// Total scanner self-throttle sleep duration in seconds for the last finished scanner cycle
+    pub last_cycle_throttle_sleep_duration_seconds: f64,
+    /// Number of lifecycle actions applied by the last finished scanner cycle
+    pub last_cycle_ilm_actions: u64,
+    /// Number of object heal candidates enqueued by the last finished scanner cycle
+    pub last_cycle_heal_objects: u64,
+    /// Number of replication heal checks run by the last finished scanner cycle
+    pub last_cycle_replication_checks: u64,
+    /// Number of data-usage save operations run by the last finished scanner cycle
+    pub last_cycle_usage_saves: u64,
+    /// Number of scanner cycles that failed since server start
+    pub failed_cycles: u64,
+    /// Number of scanner cycles stopped by runtime budget since server start
+    pub partial_cycles: u64,
+    /// Number of scanner cycles stopped by an unknown budget reason
+    pub partial_cycles_unknown: u64,
+    /// Number of scanner cycles stopped by runtime budget
+    pub partial_cycles_runtime: u64,
+    /// Number of scanner cycles stopped by object budget
+    pub partial_cycles_objects: u64,
+    /// Number of scanner cycles stopped by directory budget
+    pub partial_cycles_directories: u64,
 }
 
 /// Collects scanner metrics from the given stats.
@@ -50,13 +197,143 @@ pub fn collect_scanner_metrics(stats: &ScannerStats) -> Vec<PrometheusMetric> {
     vec![
         PrometheusMetric::from_descriptor(&SCANNER_BUCKET_SCANS_FINISHED_MD, stats.bucket_scans_finished as f64),
         PrometheusMetric::from_descriptor(&SCANNER_BUCKET_SCANS_STARTED_MD, stats.bucket_scans_started as f64),
+        PrometheusMetric::from_descriptor(&SCANNER_BUCKET_SCANS_FAILED_MD, stats.bucket_scans_failed as f64),
         PrometheusMetric::from_descriptor(&SCANNER_DIRECTORIES_SCANNED_MD, stats.directories_scanned as f64),
         PrometheusMetric::from_descriptor(&SCANNER_OBJECTS_SCANNED_MD, stats.objects_scanned as f64),
         PrometheusMetric::from_descriptor(&SCANNER_VERSIONS_SCANNED_MD, stats.versions_scanned as f64),
         PrometheusMetric::from_descriptor(&SCANNER_LAST_ACTIVITY_SECONDS_MD, stats.last_activity_seconds as f64),
+        PrometheusMetric::from_descriptor(&SCANNER_ACTIVE_PATHS_MD, stats.active_paths as f64),
+        PrometheusMetric::from_descriptor(
+            &SCANNER_OLDEST_ACTIVE_PATH_AGE_SECONDS_MD,
+            stats.oldest_active_path_age_seconds as f64,
+        ),
+        PrometheusMetric::from_descriptor(
+            &SCANNER_CURRENT_SET_SCAN_CONCURRENCY_LIMIT_MD,
+            stats.current_set_scan_concurrency_limit as f64,
+        ),
+        PrometheusMetric::from_descriptor(&SCANNER_CURRENT_SET_SCANS_QUEUED_MD, stats.current_set_scans_queued as f64),
+        PrometheusMetric::from_descriptor(&SCANNER_CURRENT_SET_SCANS_ACTIVE_MD, stats.current_set_scans_active as f64),
+        PrometheusMetric::from_descriptor(
+            &SCANNER_CURRENT_DISK_SCAN_CONCURRENCY_LIMIT_MD,
+            stats.current_disk_scan_concurrency_limit as f64,
+        ),
+        PrometheusMetric::from_descriptor(
+            &SCANNER_CURRENT_DISK_BUCKET_SCANS_QUEUED_MD,
+            stats.current_disk_bucket_scans_queued as f64,
+        ),
+        PrometheusMetric::from_descriptor(
+            &SCANNER_CURRENT_DISK_BUCKET_SCANS_ACTIVE_MD,
+            stats.current_disk_bucket_scans_active as f64,
+        ),
+        PrometheusMetric::from_descriptor(
+            &SCANNER_THROTTLE_IDLE_MODE_ENABLED_MD,
+            bool_metric_value(stats.throttle_idle_mode_enabled),
+        ),
+        PrometheusMetric::from_descriptor(&SCANNER_THROTTLE_SLEEP_FACTOR_MD, stats.throttle_sleep_factor),
+        PrometheusMetric::from_descriptor(&SCANNER_THROTTLE_MAX_SLEEP_SECONDS_MD, stats.throttle_max_sleep_seconds),
+        PrometheusMetric::from_descriptor(&SCANNER_YIELD_EVERY_N_OBJECTS_MD, stats.yield_every_n_objects as f64),
+        PrometheusMetric::from_descriptor(&SCANNER_CYCLE_INTERVAL_SECONDS_MD, stats.cycle_interval_seconds),
+        PrometheusMetric::from_descriptor(&SCANNER_CYCLE_MAX_DURATION_SECONDS_MD, stats.cycle_max_duration_seconds),
+        PrometheusMetric::from_descriptor(&SCANNER_CYCLE_MAX_OBJECTS_MD, stats.cycle_max_objects as f64),
+        PrometheusMetric::from_descriptor(&SCANNER_CYCLE_MAX_DIRECTORIES_MD, stats.cycle_max_directories as f64),
+        PrometheusMetric::from_descriptor(&SCANNER_BITROT_CYCLE_ENABLED_MD, bool_metric_value(stats.bitrot_cycle_enabled)),
+        PrometheusMetric::from_descriptor(&SCANNER_BITROT_CYCLE_SECONDS_MD, stats.bitrot_cycle_seconds),
+        PrometheusMetric::from_descriptor(&SCANNER_CURRENT_CYCLE_MD, stats.current_cycle as f64),
+        PrometheusMetric::from_descriptor(&SCANNER_COMPLETED_CYCLES_MD, stats.completed_cycles as f64),
+        PrometheusMetric::from_descriptor(&SCANNER_CURRENT_CYCLE_AGE_SECONDS_MD, stats.current_cycle_age_seconds as f64),
+        PrometheusMetric::from_descriptor(&SCANNER_CURRENT_CYCLE_OBJECTS_SCANNED_MD, stats.current_cycle_objects_scanned as f64),
+        PrometheusMetric::from_descriptor(
+            &SCANNER_CURRENT_CYCLE_DIRECTORIES_SCANNED_MD,
+            stats.current_cycle_directories_scanned as f64,
+        ),
+        PrometheusMetric::from_descriptor(
+            &SCANNER_CURRENT_CYCLE_BUCKET_DRIVE_SCANS_MD,
+            stats.current_cycle_bucket_drive_scans as f64,
+        ),
+        PrometheusMetric::from_descriptor(
+            &SCANNER_CURRENT_CYCLE_BUCKET_DRIVE_FAILURES_MD,
+            stats.current_cycle_bucket_drive_failures as f64,
+        ),
+        PrometheusMetric::from_descriptor(&SCANNER_CURRENT_CYCLE_OBJECTS_PER_SECOND_MD, stats.current_cycle_objects_per_second),
+        PrometheusMetric::from_descriptor(
+            &SCANNER_CURRENT_CYCLE_DIRECTORIES_PER_SECOND_MD,
+            stats.current_cycle_directories_per_second,
+        ),
+        PrometheusMetric::from_descriptor(
+            &SCANNER_CURRENT_CYCLE_BUCKET_DRIVE_SCANS_PER_SECOND_MD,
+            stats.current_cycle_bucket_drive_scans_per_second,
+        ),
+        PrometheusMetric::from_descriptor(&SCANNER_CURRENT_CYCLE_YIELD_EVENTS_MD, stats.current_cycle_yield_events as f64),
+        PrometheusMetric::from_descriptor(
+            &SCANNER_CURRENT_CYCLE_YIELD_DURATION_SECONDS_MD,
+            stats.current_cycle_yield_duration_seconds,
+        ),
+        PrometheusMetric::from_descriptor(
+            &SCANNER_CURRENT_CYCLE_THROTTLE_SLEEP_EVENTS_MD,
+            stats.current_cycle_throttle_sleep_events as f64,
+        ),
+        PrometheusMetric::from_descriptor(
+            &SCANNER_CURRENT_CYCLE_THROTTLE_SLEEP_DURATION_SECONDS_MD,
+            stats.current_cycle_throttle_sleep_duration_seconds,
+        ),
+        PrometheusMetric::from_descriptor(&SCANNER_CURRENT_CYCLE_ILM_ACTIONS_MD, stats.current_cycle_ilm_actions as f64),
+        PrometheusMetric::from_descriptor(&SCANNER_CURRENT_CYCLE_HEAL_OBJECTS_MD, stats.current_cycle_heal_objects as f64),
+        PrometheusMetric::from_descriptor(
+            &SCANNER_CURRENT_CYCLE_REPLICATION_CHECKS_MD,
+            stats.current_cycle_replication_checks as f64,
+        ),
+        PrometheusMetric::from_descriptor(&SCANNER_CURRENT_CYCLE_USAGE_SAVES_MD, stats.current_cycle_usage_saves as f64),
+        PrometheusMetric::from_descriptor(&SCANNER_CURRENT_SCAN_MODE_MD, stats.current_scan_mode as f64),
+        PrometheusMetric::from_descriptor(&SCANNER_LAST_CYCLE_RESULT_MD, stats.last_cycle_result as f64),
+        PrometheusMetric::from_descriptor(&SCANNER_LAST_CYCLE_PARTIAL_REASON_MD, stats.last_cycle_partial_reason as f64),
+        PrometheusMetric::from_descriptor(&SCANNER_LAST_CYCLE_DURATION_SECONDS_MD, stats.last_cycle_duration_seconds),
+        PrometheusMetric::from_descriptor(&SCANNER_LAST_CYCLE_OBJECTS_SCANNED_MD, stats.last_cycle_objects_scanned as f64),
+        PrometheusMetric::from_descriptor(
+            &SCANNER_LAST_CYCLE_DIRECTORIES_SCANNED_MD,
+            stats.last_cycle_directories_scanned as f64,
+        ),
+        PrometheusMetric::from_descriptor(&SCANNER_LAST_CYCLE_BUCKET_DRIVE_SCANS_MD, stats.last_cycle_bucket_drive_scans as f64),
+        PrometheusMetric::from_descriptor(
+            &SCANNER_LAST_CYCLE_BUCKET_DRIVE_FAILURES_MD,
+            stats.last_cycle_bucket_drive_failures as f64,
+        ),
+        PrometheusMetric::from_descriptor(&SCANNER_LAST_CYCLE_OBJECTS_PER_SECOND_MD, stats.last_cycle_objects_per_second),
+        PrometheusMetric::from_descriptor(&SCANNER_LAST_CYCLE_DIRECTORIES_PER_SECOND_MD, stats.last_cycle_directories_per_second),
+        PrometheusMetric::from_descriptor(
+            &SCANNER_LAST_CYCLE_BUCKET_DRIVE_SCANS_PER_SECOND_MD,
+            stats.last_cycle_bucket_drive_scans_per_second,
+        ),
+        PrometheusMetric::from_descriptor(&SCANNER_LAST_CYCLE_YIELD_EVENTS_MD, stats.last_cycle_yield_events as f64),
+        PrometheusMetric::from_descriptor(&SCANNER_LAST_CYCLE_YIELD_DURATION_SECONDS_MD, stats.last_cycle_yield_duration_seconds),
+        PrometheusMetric::from_descriptor(
+            &SCANNER_LAST_CYCLE_THROTTLE_SLEEP_EVENTS_MD,
+            stats.last_cycle_throttle_sleep_events as f64,
+        ),
+        PrometheusMetric::from_descriptor(
+            &SCANNER_LAST_CYCLE_THROTTLE_SLEEP_DURATION_SECONDS_MD,
+            stats.last_cycle_throttle_sleep_duration_seconds,
+        ),
+        PrometheusMetric::from_descriptor(&SCANNER_LAST_CYCLE_ILM_ACTIONS_MD, stats.last_cycle_ilm_actions as f64),
+        PrometheusMetric::from_descriptor(&SCANNER_LAST_CYCLE_HEAL_OBJECTS_MD, stats.last_cycle_heal_objects as f64),
+        PrometheusMetric::from_descriptor(&SCANNER_LAST_CYCLE_REPLICATION_CHECKS_MD, stats.last_cycle_replication_checks as f64),
+        PrometheusMetric::from_descriptor(&SCANNER_LAST_CYCLE_USAGE_SAVES_MD, stats.last_cycle_usage_saves as f64),
+        PrometheusMetric::from_descriptor(&SCANNER_FAILED_CYCLES_MD, stats.failed_cycles as f64),
+        PrometheusMetric::from_descriptor(&SCANNER_PARTIAL_CYCLES_MD, stats.partial_cycles as f64),
+        PrometheusMetric::from_descriptor(&SCANNER_PARTIAL_CYCLES_BY_REASON_MD, stats.partial_cycles_unknown as f64)
+            .with_label("reason", "unknown"),
+        PrometheusMetric::from_descriptor(&SCANNER_PARTIAL_CYCLES_BY_REASON_MD, stats.partial_cycles_runtime as f64)
+            .with_label("reason", "runtime"),
+        PrometheusMetric::from_descriptor(&SCANNER_PARTIAL_CYCLES_BY_REASON_MD, stats.partial_cycles_objects as f64)
+            .with_label("reason", "objects"),
+        PrometheusMetric::from_descriptor(&SCANNER_PARTIAL_CYCLES_BY_REASON_MD, stats.partial_cycles_directories as f64)
+            .with_label("reason", "directories"),
     ]
 }
 
+fn bool_metric_value(enabled: bool) -> f64 {
+    if enabled { 1.0 } else { 0.0 }
+}
+
 #[cfg(test)]
 mod tests {
     use super::*;
@@ -67,22 +344,398 @@ mod tests {
         let stats = ScannerStats {
             bucket_scans_finished: 100,
             bucket_scans_started: 100,
+            bucket_scans_failed: 2,
             directories_scanned: 50000,
             objects_scanned: 1000000,
             versions_scanned: 1500000,
             last_activity_seconds: 30,
+            active_paths: 4,
+            oldest_active_path_age_seconds: 17,
+            current_set_scan_concurrency_limit: 3,
+            current_set_scans_queued: 5,
+            current_set_scans_active: 2,
+            current_disk_scan_concurrency_limit: 6,
+            current_disk_bucket_scans_queued: 18,
+            current_disk_bucket_scans_active: 4,
+            throttle_idle_mode_enabled: true,
+            throttle_sleep_factor: 10.0,
+            throttle_max_sleep_seconds: 15.0,
+            yield_every_n_objects: 128,
+            cycle_interval_seconds: 3600.0,
+            cycle_max_duration_seconds: 1800.0,
+            cycle_max_objects: 1_000_000,
+            cycle_max_directories: 100_000,
+            bitrot_cycle_enabled: true,
+            bitrot_cycle_seconds: 86400.0,
+            current_cycle: 12,
+            completed_cycles: 11,
+            current_cycle_age_seconds: 90,
+            current_cycle_objects_scanned: 250,
+            current_cycle_directories_scanned: 20,
+            current_cycle_bucket_drive_scans: 2,
+            current_cycle_bucket_drive_failures: 1,
+            current_cycle_objects_per_second: 12.5,
+            current_cycle_directories_per_second: 1.0,
+            current_cycle_bucket_drive_scans_per_second: 0.1,
+            current_cycle_yield_events: 8,
+            current_cycle_yield_duration_seconds: 1.25,
+            current_cycle_throttle_sleep_events: 4,
+            current_cycle_throttle_sleep_duration_seconds: 2.5,
+            current_cycle_ilm_actions: 6,
+            current_cycle_heal_objects: 2,
+            current_cycle_replication_checks: 5,
+            current_cycle_usage_saves: 3,
+            current_scan_mode: 2,
+            last_cycle_result: 1,
+            last_cycle_partial_reason: 3,
+            last_cycle_duration_seconds: 42.5,
+            last_cycle_objects_scanned: 900,
+            last_cycle_directories_scanned: 80,
+            last_cycle_bucket_drive_scans: 6,
+            last_cycle_bucket_drive_failures: 2,
+            last_cycle_objects_per_second: 18.0,
+            last_cycle_directories_per_second: 1.6,
+            last_cycle_bucket_drive_scans_per_second: 0.12,
+            last_cycle_yield_events: 30,
+            last_cycle_yield_duration_seconds: 9.5,
+            last_cycle_throttle_sleep_events: 12,
+            last_cycle_throttle_sleep_duration_seconds: 6.75,
+            last_cycle_ilm_actions: 44,
+            last_cycle_heal_objects: 7,
+            last_cycle_replication_checks: 12,
+            last_cycle_usage_saves: 9,
+            failed_cycles: 3,
+            partial_cycles: 10,
+            partial_cycles_unknown: 1,
+            partial_cycles_runtime: 2,
+            partial_cycles_objects: 3,
+            partial_cycles_directories: 4,
         };
 
         let metrics = collect_scanner_metrics(&stats);
         report_metrics(&metrics);
 
-        assert_eq!(metrics.len(), 6);
+        assert_eq!(metrics.len(), 68);
 
         let objects = metrics.iter().find(|m| m.value == 1000000.0);
         assert!(objects.is_some());
 
         let last_activity = metrics.iter().find(|m| m.value == 30.0);
         assert!(last_activity.is_some());
+
+        let active_paths = metrics
+            .iter()
+            .find(|m| m.name == SCANNER_ACTIVE_PATHS_MD.get_full_metric_name());
+        assert_eq!(active_paths.map(|m| m.value), Some(4.0));
+
+        let oldest_active_path_age = metrics
+            .iter()
+            .find(|m| m.name == SCANNER_OLDEST_ACTIVE_PATH_AGE_SECONDS_MD.get_full_metric_name());
+        assert_eq!(oldest_active_path_age.map(|m| m.value), Some(17.0));
+
+        let current_set_scan_concurrency_limit = metrics
+            .iter()
+            .find(|m| m.name == SCANNER_CURRENT_SET_SCAN_CONCURRENCY_LIMIT_MD.get_full_metric_name());
+        assert_eq!(current_set_scan_concurrency_limit.map(|m| m.value), Some(3.0));
+
+        let current_set_scans_queued = metrics
+            .iter()
+            .find(|m| m.name == SCANNER_CURRENT_SET_SCANS_QUEUED_MD.get_full_metric_name());
+        assert_eq!(current_set_scans_queued.map(|m| m.value), Some(5.0));
+
+        let current_set_scans_active = metrics
+            .iter()
+            .find(|m| m.name == SCANNER_CURRENT_SET_SCANS_ACTIVE_MD.get_full_metric_name());
+        assert_eq!(current_set_scans_active.map(|m| m.value), Some(2.0));
+
+        let current_disk_scan_concurrency_limit = metrics
+            .iter()
+            .find(|m| m.name == SCANNER_CURRENT_DISK_SCAN_CONCURRENCY_LIMIT_MD.get_full_metric_name());
+        assert_eq!(current_disk_scan_concurrency_limit.map(|m| m.value), Some(6.0));
+
+        let current_disk_bucket_scans_queued = metrics
+            .iter()
+            .find(|m| m.name == SCANNER_CURRENT_DISK_BUCKET_SCANS_QUEUED_MD.get_full_metric_name());
+        assert_eq!(current_disk_bucket_scans_queued.map(|m| m.value), Some(18.0));
+
+        let current_disk_bucket_scans_active = metrics
+            .iter()
+            .find(|m| m.name == SCANNER_CURRENT_DISK_BUCKET_SCANS_ACTIVE_MD.get_full_metric_name());
+        assert_eq!(current_disk_bucket_scans_active.map(|m| m.value), Some(4.0));
+
+        let bucket_scans_failed = metrics
+            .iter()
+            .find(|m| m.name == SCANNER_BUCKET_SCANS_FAILED_MD.get_full_metric_name());
+        assert_eq!(bucket_scans_failed.map(|m| m.value), Some(2.0));
+
+        let throttle_idle_mode_enabled = metrics
+            .iter()
+            .find(|m| m.name == SCANNER_THROTTLE_IDLE_MODE_ENABLED_MD.get_full_metric_name());
+        assert_eq!(throttle_idle_mode_enabled.map(|m| m.value), Some(1.0));
+
+        let throttle_sleep_factor = metrics
+            .iter()
+            .find(|m| m.name == SCANNER_THROTTLE_SLEEP_FACTOR_MD.get_full_metric_name());
+        assert_eq!(throttle_sleep_factor.map(|m| m.value), Some(10.0));
+
+        let throttle_max_sleep = metrics
+            .iter()
+            .find(|m| m.name == SCANNER_THROTTLE_MAX_SLEEP_SECONDS_MD.get_full_metric_name());
+        assert_eq!(throttle_max_sleep.map(|m| m.value), Some(15.0));
+
+        let yield_every_n_objects = metrics
+            .iter()
+            .find(|m| m.name == SCANNER_YIELD_EVERY_N_OBJECTS_MD.get_full_metric_name());
+        assert_eq!(yield_every_n_objects.map(|m| m.value), Some(128.0));
+
+        let cycle_interval_seconds = metrics
+            .iter()
+            .find(|m| m.name == SCANNER_CYCLE_INTERVAL_SECONDS_MD.get_full_metric_name());
+        assert_eq!(cycle_interval_seconds.map(|m| m.value), Some(3600.0));
+
+        let cycle_max_duration_seconds = metrics
+            .iter()
+            .find(|m| m.name == SCANNER_CYCLE_MAX_DURATION_SECONDS_MD.get_full_metric_name());
+        assert_eq!(cycle_max_duration_seconds.map(|m| m.value), Some(1800.0));
+
+        let cycle_max_objects = metrics
+            .iter()
+            .find(|m| m.name == SCANNER_CYCLE_MAX_OBJECTS_MD.get_full_metric_name());
+        assert_eq!(cycle_max_objects.map(|m| m.value), Some(1_000_000.0));
+
+        let cycle_max_directories = metrics
+            .iter()
+            .find(|m| m.name == SCANNER_CYCLE_MAX_DIRECTORIES_MD.get_full_metric_name());
+        assert_eq!(cycle_max_directories.map(|m| m.value), Some(100_000.0));
+
+        let bitrot_cycle_enabled = metrics
+            .iter()
+            .find(|m| m.name == SCANNER_BITROT_CYCLE_ENABLED_MD.get_full_metric_name());
+        assert_eq!(bitrot_cycle_enabled.map(|m| m.value), Some(1.0));
+
+        let bitrot_cycle_seconds = metrics
+            .iter()
+            .find(|m| m.name == SCANNER_BITROT_CYCLE_SECONDS_MD.get_full_metric_name());
+        assert_eq!(bitrot_cycle_seconds.map(|m| m.value), Some(86400.0));
+
+        let current_cycle = metrics
+            .iter()
+            .find(|m| m.name == SCANNER_CURRENT_CYCLE_MD.get_full_metric_name());
+        assert_eq!(current_cycle.map(|m| m.value), Some(12.0));
+
+        let completed_cycles = metrics
+            .iter()
+            .find(|m| m.name == SCANNER_COMPLETED_CYCLES_MD.get_full_metric_name());
+        assert_eq!(completed_cycles.map(|m| m.value), Some(11.0));
+
+        let current_cycle_age = metrics
+            .iter()
+            .find(|m| m.name == SCANNER_CURRENT_CYCLE_AGE_SECONDS_MD.get_full_metric_name());
+        assert_eq!(current_cycle_age.map(|m| m.value), Some(90.0));
+
+        let current_cycle_objects = metrics
+            .iter()
+            .find(|m| m.name == SCANNER_CURRENT_CYCLE_OBJECTS_SCANNED_MD.get_full_metric_name());
+        assert_eq!(current_cycle_objects.map(|m| m.value), Some(250.0));
+
+        let current_cycle_directories = metrics
+            .iter()
+            .find(|m| m.name == SCANNER_CURRENT_CYCLE_DIRECTORIES_SCANNED_MD.get_full_metric_name());
+        assert_eq!(current_cycle_directories.map(|m| m.value), Some(20.0));
+
+        let current_cycle_bucket_drive_scans = metrics
+            .iter()
+            .find(|m| m.name == SCANNER_CURRENT_CYCLE_BUCKET_DRIVE_SCANS_MD.get_full_metric_name());
+        assert_eq!(current_cycle_bucket_drive_scans.map(|m| m.value), Some(2.0));
+
+        let current_cycle_bucket_drive_failures = metrics
+            .iter()
+            .find(|m| m.name == SCANNER_CURRENT_CYCLE_BUCKET_DRIVE_FAILURES_MD.get_full_metric_name());
+        assert_eq!(current_cycle_bucket_drive_failures.map(|m| m.value), Some(1.0));
+
+        let current_cycle_objects_rate = metrics
+            .iter()
+            .find(|m| m.name == SCANNER_CURRENT_CYCLE_OBJECTS_PER_SECOND_MD.get_full_metric_name());
+        assert_eq!(current_cycle_objects_rate.map(|m| m.value), Some(12.5));
+
+        let current_cycle_directories_rate = metrics
+            .iter()
+            .find(|m| m.name == SCANNER_CURRENT_CYCLE_DIRECTORIES_PER_SECOND_MD.get_full_metric_name());
+        assert_eq!(current_cycle_directories_rate.map(|m| m.value), Some(1.0));
+
+        let current_cycle_bucket_drive_scans_rate = metrics
+            .iter()
+            .find(|m| m.name == SCANNER_CURRENT_CYCLE_BUCKET_DRIVE_SCANS_PER_SECOND_MD.get_full_metric_name());
+        assert_eq!(current_cycle_bucket_drive_scans_rate.map(|m| m.value), Some(0.1));
+
+        let current_cycle_yield_events = metrics
+            .iter()
+            .find(|m| m.name == SCANNER_CURRENT_CYCLE_YIELD_EVENTS_MD.get_full_metric_name());
+        assert_eq!(current_cycle_yield_events.map(|m| m.value), Some(8.0));
+
+        let current_cycle_yield_duration = metrics
+            .iter()
+            .find(|m| m.name == SCANNER_CURRENT_CYCLE_YIELD_DURATION_SECONDS_MD.get_full_metric_name());
+        assert_eq!(current_cycle_yield_duration.map(|m| m.value), Some(1.25));
+
+        let current_cycle_throttle_sleep_events = metrics
+            .iter()
+            .find(|m| m.name == SCANNER_CURRENT_CYCLE_THROTTLE_SLEEP_EVENTS_MD.get_full_metric_name());
+        assert_eq!(current_cycle_throttle_sleep_events.map(|m| m.value), Some(4.0));
+
+        let current_cycle_throttle_sleep_duration = metrics
+            .iter()
+            .find(|m| m.name == SCANNER_CURRENT_CYCLE_THROTTLE_SLEEP_DURATION_SECONDS_MD.get_full_metric_name());
+        assert_eq!(current_cycle_throttle_sleep_duration.map(|m| m.value), Some(2.5));
+
+        let current_cycle_ilm_actions = metrics
+            .iter()
+            .find(|m| m.name == SCANNER_CURRENT_CYCLE_ILM_ACTIONS_MD.get_full_metric_name());
+        assert_eq!(current_cycle_ilm_actions.map(|m| m.value), Some(6.0));
+
+        let current_cycle_heal_objects = metrics
+            .iter()
+            .find(|m| m.name == SCANNER_CURRENT_CYCLE_HEAL_OBJECTS_MD.get_full_metric_name());
+        assert_eq!(current_cycle_heal_objects.map(|m| m.value), Some(2.0));
+
+        let current_cycle_replication_checks = metrics
+            .iter()
+            .find(|m| m.name == SCANNER_CURRENT_CYCLE_REPLICATION_CHECKS_MD.get_full_metric_name());
+        assert_eq!(current_cycle_replication_checks.map(|m| m.value), Some(5.0));
+
+        let current_cycle_usage_saves = metrics
+            .iter()
+            .find(|m| m.name == SCANNER_CURRENT_CYCLE_USAGE_SAVES_MD.get_full_metric_name());
+        assert_eq!(current_cycle_usage_saves.map(|m| m.value), Some(3.0));
+
+        let current_scan_mode = metrics
+            .iter()
+            .find(|m| m.name == SCANNER_CURRENT_SCAN_MODE_MD.get_full_metric_name());
+        assert_eq!(current_scan_mode.map(|m| m.value), Some(2.0));
+
+        let last_cycle_result = metrics
+            .iter()
+            .find(|m| m.name == SCANNER_LAST_CYCLE_RESULT_MD.get_full_metric_name());
+        assert_eq!(last_cycle_result.map(|m| m.value), Some(1.0));
+
+        let last_cycle_partial_reason = metrics
+            .iter()
+            .find(|m| m.name == SCANNER_LAST_CYCLE_PARTIAL_REASON_MD.get_full_metric_name());
+        assert_eq!(last_cycle_partial_reason.map(|m| m.value), Some(3.0));
+
+        let last_cycle_duration = metrics
+            .iter()
+            .find(|m| m.name == SCANNER_LAST_CYCLE_DURATION_SECONDS_MD.get_full_metric_name());
+        assert_eq!(last_cycle_duration.map(|m| m.value), Some(42.5));
+
+        let last_cycle_objects = metrics
+            .iter()
+            .find(|m| m.name == SCANNER_LAST_CYCLE_OBJECTS_SCANNED_MD.get_full_metric_name());
+        assert_eq!(last_cycle_objects.map(|m| m.value), Some(900.0));
+
+        let last_cycle_directories = metrics
+            .iter()
+            .find(|m| m.name == SCANNER_LAST_CYCLE_DIRECTORIES_SCANNED_MD.get_full_metric_name());
+        assert_eq!(last_cycle_directories.map(|m| m.value), Some(80.0));
+
+        let last_cycle_bucket_drive_scans = metrics
+            .iter()
+            .find(|m| m.name == SCANNER_LAST_CYCLE_BUCKET_DRIVE_SCANS_MD.get_full_metric_name());
+        assert_eq!(last_cycle_bucket_drive_scans.map(|m| m.value), Some(6.0));
+
+        let last_cycle_bucket_drive_failures = metrics
+            .iter()
+            .find(|m| m.name == SCANNER_LAST_CYCLE_BUCKET_DRIVE_FAILURES_MD.get_full_metric_name());
+        assert_eq!(last_cycle_bucket_drive_failures.map(|m| m.value), Some(2.0));
+
+        let last_cycle_objects_rate = metrics
+            .iter()
+            .find(|m| m.name == SCANNER_LAST_CYCLE_OBJECTS_PER_SECOND_MD.get_full_metric_name());
+        assert_eq!(last_cycle_objects_rate.map(|m| m.value), Some(18.0));
+
+        let last_cycle_directories_rate = metrics
+            .iter()
+            .find(|m| m.name == SCANNER_LAST_CYCLE_DIRECTORIES_PER_SECOND_MD.get_full_metric_name());
+        assert_eq!(last_cycle_directories_rate.map(|m| m.value), Some(1.6));
+
+        let last_cycle_bucket_drive_scans_rate = metrics
+            .iter()
+            .find(|m| m.name == SCANNER_LAST_CYCLE_BUCKET_DRIVE_SCANS_PER_SECOND_MD.get_full_metric_name());
+        assert_eq!(last_cycle_bucket_drive_scans_rate.map(|m| m.value), Some(0.12));
+
+        let last_cycle_yield_events = metrics
+            .iter()
+            .find(|m| m.name == SCANNER_LAST_CYCLE_YIELD_EVENTS_MD.get_full_metric_name());
+        assert_eq!(last_cycle_yield_events.map(|m| m.value), Some(30.0));
+
+        let last_cycle_yield_duration = metrics
+            .iter()
+            .find(|m| m.name == SCANNER_LAST_CYCLE_YIELD_DURATION_SECONDS_MD.get_full_metric_name());
+        assert_eq!(last_cycle_yield_duration.map(|m| m.value), Some(9.5));
+
+        let last_cycle_throttle_sleep_events = metrics
+            .iter()
+            .find(|m| m.name == SCANNER_LAST_CYCLE_THROTTLE_SLEEP_EVENTS_MD.get_full_metric_name());
+        assert_eq!(last_cycle_throttle_sleep_events.map(|m| m.value), Some(12.0));
+
+        let last_cycle_throttle_sleep_duration = metrics
+            .iter()
+            .find(|m| m.name == SCANNER_LAST_CYCLE_THROTTLE_SLEEP_DURATION_SECONDS_MD.get_full_metric_name());
+        assert_eq!(last_cycle_throttle_sleep_duration.map(|m| m.value), Some(6.75));
+
+        let last_cycle_ilm_actions = metrics
+            .iter()
+            .find(|m| m.name == SCANNER_LAST_CYCLE_ILM_ACTIONS_MD.get_full_metric_name());
+        assert_eq!(last_cycle_ilm_actions.map(|m| m.value), Some(44.0));
+
+        let last_cycle_heal_objects = metrics
+            .iter()
+            .find(|m| m.name == SCANNER_LAST_CYCLE_HEAL_OBJECTS_MD.get_full_metric_name());
+        assert_eq!(last_cycle_heal_objects.map(|m| m.value), Some(7.0));
+
+        let last_cycle_replication_checks = metrics
+            .iter()
+            .find(|m| m.name == SCANNER_LAST_CYCLE_REPLICATION_CHECKS_MD.get_full_metric_name());
+        assert_eq!(last_cycle_replication_checks.map(|m| m.value), Some(12.0));
+
+        let last_cycle_usage_saves = metrics
+            .iter()
+            .find(|m| m.name == SCANNER_LAST_CYCLE_USAGE_SAVES_MD.get_full_metric_name());
+        assert_eq!(last_cycle_usage_saves.map(|m| m.value), Some(9.0));
+
+        let failed_cycles = metrics
+            .iter()
+            .find(|m| m.name == SCANNER_FAILED_CYCLES_MD.get_full_metric_name());
+        assert_eq!(failed_cycles.map(|m| m.value), Some(3.0));
+
+        let partial_cycles = metrics
+            .iter()
+            .find(|m| m.name == SCANNER_PARTIAL_CYCLES_MD.get_full_metric_name());
+        assert_eq!(partial_cycles.map(|m| m.value), Some(10.0));
+
+        let partial_cycles_runtime = metrics.iter().find(|m| {
+            m.name == SCANNER_PARTIAL_CYCLES_BY_REASON_MD.get_full_metric_name()
+                && m.labels
+                    .iter()
+                    .any(|(name, value)| *name == "reason" && value.as_ref() == "runtime")
+        });
+        assert_eq!(partial_cycles_runtime.map(|m| m.value), Some(2.0));
+
+        let partial_cycles_objects = metrics.iter().find(|m| {
+            m.name == SCANNER_PARTIAL_CYCLES_BY_REASON_MD.get_full_metric_name()
+                && m.labels
+                    .iter()
+                    .any(|(name, value)| *name == "reason" && value.as_ref() == "objects")
+        });
+        assert_eq!(partial_cycles_objects.map(|m| m.value), Some(3.0));
+
+        let partial_cycles_directories = metrics.iter().find(|m| {
+            m.name == SCANNER_PARTIAL_CYCLES_BY_REASON_MD.get_full_metric_name()
+                && m.labels
+                    .iter()
+                    .any(|(name, value)| *name == "reason" && value.as_ref() == "directories")
+        });
+        assert_eq!(partial_cycles_directories.map(|m| m.value), Some(4.0));
     }
 
     #[test]
@@ -90,10 +743,16 @@ mod tests {
         let stats = ScannerStats::default();
         let metrics = collect_scanner_metrics(&stats);
 
-        assert_eq!(metrics.len(), 6);
+        assert_eq!(metrics.len(), 68);
         for metric in &metrics {
             assert_eq!(metric.value, 0.0);
-            assert!(metric.labels.is_empty());
+            if metric.name == SCANNER_PARTIAL_CYCLES_BY_REASON_MD.get_full_metric_name() {
+                assert_eq!(metric.labels.len(), 1);
+                assert_eq!(metric.labels[0].0, "reason");
+                assert!(matches!(metric.labels[0].1.as_ref(), "unknown" | "runtime" | "objects" | "directories"));
+            } else {
+                assert!(metric.labels.is_empty());
+            }
         }
     }
 }

crates/obs/src/metrics/schema/entry/metric_name.rs

@@ -278,6 +278,65 @@ pub enum MetricName {
     ScannerObjectsScanned,
     ScannerVersionsScanned,
     ScannerLastActivitySeconds,
+    ScannerActivePaths,
+    ScannerOldestActivePathAgeSeconds,
+    ScannerCurrentSetScanConcurrencyLimit,
+    ScannerCurrentSetScansQueued,
+    ScannerCurrentSetScansActive,
+    ScannerCurrentDiskScanConcurrencyLimit,
+    ScannerCurrentDiskBucketScansQueued,
+    ScannerCurrentDiskBucketScansActive,
+    ScannerBucketScansFailed,
+    ScannerThrottleIdleModeEnabled,
+    ScannerThrottleSleepFactor,
+    ScannerThrottleMaxSleepSeconds,
+    ScannerYieldEveryNObjects,
+    ScannerCycleIntervalSeconds,
+    ScannerCycleMaxDurationSeconds,
+    ScannerCycleMaxObjects,
+    ScannerCycleMaxDirectories,
+    ScannerBitrotCycleEnabled,
+    ScannerBitrotCycleSeconds,
+    ScannerCurrentCycle,
+    ScannerCompletedCycles,
+    ScannerCurrentCycleAgeSeconds,
+    ScannerCurrentCycleObjectsScanned,
+    ScannerCurrentCycleDirectoriesScanned,
+    ScannerCurrentCycleBucketDriveScans,
+    ScannerCurrentCycleBucketDriveFailures,
+    ScannerCurrentCycleObjectsPerSecond,
+    ScannerCurrentCycleDirectoriesPerSecond,
+    ScannerCurrentCycleBucketDriveScansPerSecond,
+    ScannerCurrentCycleYieldEvents,
+    ScannerCurrentCycleYieldDurationSeconds,
+    ScannerCurrentCycleThrottleSleepEvents,
+    ScannerCurrentCycleThrottleSleepDurationSeconds,
+    ScannerCurrentCycleIlmActions,
+    ScannerCurrentCycleHealObjects,
+    ScannerCurrentCycleReplicationChecks,
+    ScannerCurrentCycleUsageSaves,
+    ScannerCurrentScanMode,
+    ScannerLastCycleResult,
+    ScannerLastCyclePartialReason,
+    ScannerLastCycleDurationSeconds,
+    ScannerLastCycleObjectsScanned,
+    ScannerLastCycleDirectoriesScanned,
+    ScannerLastCycleBucketDriveScans,
+    ScannerLastCycleBucketDriveFailures,
+    ScannerLastCycleObjectsPerSecond,
+    ScannerLastCycleDirectoriesPerSecond,
+    ScannerLastCycleBucketDriveScansPerSecond,
+    ScannerLastCycleYieldEvents,
+    ScannerLastCycleYieldDurationSeconds,
+    ScannerLastCycleThrottleSleepEvents,
+    ScannerLastCycleThrottleSleepDurationSeconds,
+    ScannerLastCycleIlmActions,
+    ScannerLastCycleHealObjects,
+    ScannerLastCycleReplicationChecks,
+    ScannerLastCycleUsageSaves,
+    ScannerFailedCycles,
+    ScannerPartialCycles,
+    ScannerPartialCyclesByReason,
 
     // CPU system-related metrics
     SysCPUAvgIdle,
@@ -618,6 +677,65 @@ impl MetricName {
             Self::ScannerObjectsScanned => "objects_scanned".to_string(),
             Self::ScannerVersionsScanned => "versions_scanned".to_string(),
             Self::ScannerLastActivitySeconds => "last_activity_seconds".to_string(),
+            Self::ScannerActivePaths => "active_paths".to_string(),
+            Self::ScannerOldestActivePathAgeSeconds => "oldest_active_path_age_seconds".to_string(),
+            Self::ScannerCurrentSetScanConcurrencyLimit => "current_set_scan_concurrency_limit".to_string(),
+            Self::ScannerCurrentSetScansQueued => "current_set_scans_queued".to_string(),
+            Self::ScannerCurrentSetScansActive => "current_set_scans_active".to_string(),
+            Self::ScannerCurrentDiskScanConcurrencyLimit => "current_disk_scan_concurrency_limit".to_string(),
+            Self::ScannerCurrentDiskBucketScansQueued => "current_disk_bucket_scans_queued".to_string(),
+            Self::ScannerCurrentDiskBucketScansActive => "current_disk_bucket_scans_active".to_string(),
+            Self::ScannerBucketScansFailed => "bucket_scans_failed".to_string(),
+            Self::ScannerThrottleIdleModeEnabled => "throttle_idle_mode_enabled".to_string(),
+            Self::ScannerThrottleSleepFactor => "throttle_sleep_factor".to_string(),
+            Self::ScannerThrottleMaxSleepSeconds => "throttle_max_sleep_seconds".to_string(),
+            Self::ScannerYieldEveryNObjects => "yield_every_n_objects".to_string(),
+            Self::ScannerCycleIntervalSeconds => "cycle_interval_seconds".to_string(),
+            Self::ScannerCycleMaxDurationSeconds => "cycle_max_duration_seconds".to_string(),
+            Self::ScannerCycleMaxObjects => "cycle_max_objects".to_string(),
+            Self::ScannerCycleMaxDirectories => "cycle_max_directories".to_string(),
+            Self::ScannerBitrotCycleEnabled => "bitrot_cycle_enabled".to_string(),
+            Self::ScannerBitrotCycleSeconds => "bitrot_cycle_seconds".to_string(),
+            Self::ScannerCurrentCycle => "current_cycle".to_string(),
+            Self::ScannerCompletedCycles => "completed_cycles".to_string(),
+            Self::ScannerCurrentCycleAgeSeconds => "current_cycle_age_seconds".to_string(),
+            Self::ScannerCurrentCycleObjectsScanned => "current_cycle_objects_scanned".to_string(),
+            Self::ScannerCurrentCycleDirectoriesScanned => "current_cycle_directories_scanned".to_string(),
+            Self::ScannerCurrentCycleBucketDriveScans => "current_cycle_bucket_drive_scans".to_string(),
+            Self::ScannerCurrentCycleBucketDriveFailures => "current_cycle_bucket_drive_failures".to_string(),
+            Self::ScannerCurrentCycleObjectsPerSecond => "current_cycle_objects_per_second".to_string(),
+            Self::ScannerCurrentCycleDirectoriesPerSecond => "current_cycle_directories_per_second".to_string(),
+            Self::ScannerCurrentCycleBucketDriveScansPerSecond => "current_cycle_bucket_drive_scans_per_second".to_string(),
+            Self::ScannerCurrentCycleYieldEvents => "current_cycle_yield_events".to_string(),
+            Self::ScannerCurrentCycleYieldDurationSeconds => "current_cycle_yield_duration_seconds".to_string(),
+            Self::ScannerCurrentCycleThrottleSleepEvents => "current_cycle_throttle_sleep_events".to_string(),
+            Self::ScannerCurrentCycleThrottleSleepDurationSeconds => "current_cycle_throttle_sleep_duration_seconds".to_string(),
+            Self::ScannerCurrentCycleIlmActions => "current_cycle_ilm_actions".to_string(),
+            Self::ScannerCurrentCycleHealObjects => "current_cycle_heal_objects".to_string(),
+            Self::ScannerCurrentCycleReplicationChecks => "current_cycle_replication_checks".to_string(),
+            Self::ScannerCurrentCycleUsageSaves => "current_cycle_usage_saves".to_string(),
+            Self::ScannerCurrentScanMode => "current_scan_mode".to_string(),
+            Self::ScannerLastCycleResult => "last_cycle_result".to_string(),
+            Self::ScannerLastCyclePartialReason => "last_cycle_partial_reason".to_string(),
+            Self::ScannerLastCycleDurationSeconds => "last_cycle_duration_seconds".to_string(),
+            Self::ScannerLastCycleObjectsScanned => "last_cycle_objects_scanned".to_string(),
+            Self::ScannerLastCycleDirectoriesScanned => "last_cycle_directories_scanned".to_string(),
+            Self::ScannerLastCycleBucketDriveScans => "last_cycle_bucket_drive_scans".to_string(),
+            Self::ScannerLastCycleBucketDriveFailures => "last_cycle_bucket_drive_failures".to_string(),
+            Self::ScannerLastCycleObjectsPerSecond => "last_cycle_objects_per_second".to_string(),
+            Self::ScannerLastCycleDirectoriesPerSecond => "last_cycle_directories_per_second".to_string(),
+            Self::ScannerLastCycleBucketDriveScansPerSecond => "last_cycle_bucket_drive_scans_per_second".to_string(),
+            Self::ScannerLastCycleYieldEvents => "last_cycle_yield_events".to_string(),
+            Self::ScannerLastCycleYieldDurationSeconds => "last_cycle_yield_duration_seconds".to_string(),
+            Self::ScannerLastCycleThrottleSleepEvents => "last_cycle_throttle_sleep_events".to_string(),
+            Self::ScannerLastCycleThrottleSleepDurationSeconds => "last_cycle_throttle_sleep_duration_seconds".to_string(),
+            Self::ScannerLastCycleIlmActions => "last_cycle_ilm_actions".to_string(),
+            Self::ScannerLastCycleHealObjects => "last_cycle_heal_objects".to_string(),
+            Self::ScannerLastCycleReplicationChecks => "last_cycle_replication_checks".to_string(),
+            Self::ScannerLastCycleUsageSaves => "last_cycle_usage_saves".to_string(),
+            Self::ScannerFailedCycles => "failed_cycles".to_string(),
+            Self::ScannerPartialCycles => "partial_cycles".to_string(),
+            Self::ScannerPartialCyclesByReason => "partial_cycles_by_reason".to_string(),
 
             // CPU system-related metrics
             Self::SysCPUAvgIdle => "avg_idle".to_string(),

crates/obs/src/metrics/schema/scanner.rs

@@ -20,7 +20,7 @@ use std::sync::LazyLock;
 pub static SCANNER_BUCKET_SCANS_FINISHED_MD: LazyLock<MetricDescriptor> = LazyLock::new(|| {
     new_counter_md(
         MetricName::ScannerBucketScansFinished,
-        "Total number of bucket scans finished since server start",
+        "Total number of bucket-drive scans finished since server start",
         &[],
         subsystems::SCANNER,
     )
@@ -29,7 +29,16 @@ pub static SCANNER_BUCKET_SCANS_FINISHED_MD: LazyLock<MetricDescriptor> = LazyLo
 pub static SCANNER_BUCKET_SCANS_STARTED_MD: LazyLock<MetricDescriptor> = LazyLock::new(|| {
     new_counter_md(
         MetricName::ScannerBucketScansStarted,
-        "Total number of bucket scans started since server start",
+        "Total number of bucket-drive scans started since server start",
+        &[],
+        subsystems::SCANNER,
+    )
+});
+
+pub static SCANNER_BUCKET_SCANS_FAILED_MD: LazyLock<MetricDescriptor> = LazyLock::new(|| {
+    new_counter_md(
+        MetricName::ScannerBucketScansFailed,
+        "Total number of bucket-drive scans that failed since server start",
         &[],
         subsystems::SCANNER,
     )
@@ -70,3 +79,525 @@ pub static SCANNER_LAST_ACTIVITY_SECONDS_MD: LazyLock<MetricDescriptor> = LazyLo
         subsystems::SCANNER,
     )
 });
+
+pub static SCANNER_ACTIVE_PATHS_MD: LazyLock<MetricDescriptor> = LazyLock::new(|| {
+    new_gauge_md(
+        MetricName::ScannerActivePaths,
+        "Current number of scanner paths being processed.",
+        &[],
+        subsystems::SCANNER,
+    )
+});
+
+pub static SCANNER_OLDEST_ACTIVE_PATH_AGE_SECONDS_MD: LazyLock<MetricDescriptor> = LazyLock::new(|| {
+    new_gauge_md(
+        MetricName::ScannerOldestActivePathAgeSeconds,
+        "Time elapsed (in seconds) since the oldest active scanner path was last updated, or zero when idle.",
+        &[],
+        subsystems::SCANNER,
+    )
+});
+
+pub static SCANNER_CURRENT_SET_SCAN_CONCURRENCY_LIMIT_MD: LazyLock<MetricDescriptor> = LazyLock::new(|| {
+    new_gauge_md(
+        MetricName::ScannerCurrentSetScanConcurrencyLimit,
+        "Current aggregate scanner set scan concurrency limit across active scanner work.",
+        &[],
+        subsystems::SCANNER,
+    )
+});
+
+pub static SCANNER_CURRENT_SET_SCANS_QUEUED_MD: LazyLock<MetricDescriptor> = LazyLock::new(|| {
+    new_gauge_md(
+        MetricName::ScannerCurrentSetScansQueued,
+        "Current number of queued scanner set scans across active scanner work.",
+        &[],
+        subsystems::SCANNER,
+    )
+});
+
+pub static SCANNER_CURRENT_SET_SCANS_ACTIVE_MD: LazyLock<MetricDescriptor> = LazyLock::new(|| {
+    new_gauge_md(
+        MetricName::ScannerCurrentSetScansActive,
+        "Current number of active scanner set scans across active scanner work.",
+        &[],
+        subsystems::SCANNER,
+    )
+});
+
+pub static SCANNER_CURRENT_DISK_SCAN_CONCURRENCY_LIMIT_MD: LazyLock<MetricDescriptor> = LazyLock::new(|| {
+    new_gauge_md(
+        MetricName::ScannerCurrentDiskScanConcurrencyLimit,
+        "Current aggregate scanner disk-bucket scan concurrency limit across active scanner work.",
+        &[],
+        subsystems::SCANNER,
+    )
+});
+
+pub static SCANNER_CURRENT_DISK_BUCKET_SCANS_QUEUED_MD: LazyLock<MetricDescriptor> = LazyLock::new(|| {
+    new_gauge_md(
+        MetricName::ScannerCurrentDiskBucketScansQueued,
+        "Current number of queued scanner disk-bucket scans across active scanner work.",
+        &[],
+        subsystems::SCANNER,
+    )
+});
+
+pub static SCANNER_CURRENT_DISK_BUCKET_SCANS_ACTIVE_MD: LazyLock<MetricDescriptor> = LazyLock::new(|| {
+    new_gauge_md(
+        MetricName::ScannerCurrentDiskBucketScansActive,
+        "Current number of active scanner disk-bucket scans across active scanner work.",
+        &[],
+        subsystems::SCANNER,
+    )
+});
+
+pub static SCANNER_THROTTLE_IDLE_MODE_ENABLED_MD: LazyLock<MetricDescriptor> = LazyLock::new(|| {
+    new_gauge_md(
+        MetricName::ScannerThrottleIdleModeEnabled,
+        "Whether scanner idle-mode self-throttling is enabled: 1 enabled, 0 disabled.",
+        &[],
+        subsystems::SCANNER,
+    )
+});
+
+pub static SCANNER_THROTTLE_SLEEP_FACTOR_MD: LazyLock<MetricDescriptor> = LazyLock::new(|| {
+    new_gauge_md(
+        MetricName::ScannerThrottleSleepFactor,
+        "Effective scanner sleep factor used to compute proportional self-throttle sleeps.",
+        &[],
+        subsystems::SCANNER,
+    )
+});
+
+pub static SCANNER_THROTTLE_MAX_SLEEP_SECONDS_MD: LazyLock<MetricDescriptor> = LazyLock::new(|| {
+    new_gauge_md(
+        MetricName::ScannerThrottleMaxSleepSeconds,
+        "Effective maximum scanner self-throttle sleep duration in seconds.",
+        &[],
+        subsystems::SCANNER,
+    )
+});
+
+pub static SCANNER_YIELD_EVERY_N_OBJECTS_MD: LazyLock<MetricDescriptor> = LazyLock::new(|| {
+    new_gauge_md(
+        MetricName::ScannerYieldEveryNObjects,
+        "Current object interval for cooperative scanner runtime yields, or zero when disabled.",
+        &[],
+        subsystems::SCANNER,
+    )
+});
+
+pub static SCANNER_CYCLE_INTERVAL_SECONDS_MD: LazyLock<MetricDescriptor> = LazyLock::new(|| {
+    new_gauge_md(
+        MetricName::ScannerCycleIntervalSeconds,
+        "Effective scanner cycle interval in seconds.",
+        &[],
+        subsystems::SCANNER,
+    )
+});
+
+pub static SCANNER_CYCLE_MAX_DURATION_SECONDS_MD: LazyLock<MetricDescriptor> = LazyLock::new(|| {
+    new_gauge_md(
+        MetricName::ScannerCycleMaxDurationSeconds,
+        "Effective maximum scanner cycle runtime in seconds; zero means unlimited.",
+        &[],
+        subsystems::SCANNER,
+    )
+});
+
+pub static SCANNER_CYCLE_MAX_OBJECTS_MD: LazyLock<MetricDescriptor> = LazyLock::new(|| {
+    new_gauge_md(
+        MetricName::ScannerCycleMaxObjects,
+        "Effective maximum objects processed by one scanner cycle; zero means unlimited.",
+        &[],
+        subsystems::SCANNER,
+    )
+});
+
+pub static SCANNER_CYCLE_MAX_DIRECTORIES_MD: LazyLock<MetricDescriptor> = LazyLock::new(|| {
+    new_gauge_md(
+        MetricName::ScannerCycleMaxDirectories,
+        "Effective maximum directories entered by one scanner cycle; zero means unlimited.",
+        &[],
+        subsystems::SCANNER,
+    )
+});
+
+pub static SCANNER_BITROT_CYCLE_ENABLED_MD: LazyLock<MetricDescriptor> = LazyLock::new(|| {
+    new_gauge_md(
+        MetricName::ScannerBitrotCycleEnabled,
+        "Whether periodic scanner bitrot deep scans are enabled: 1 enabled, 0 disabled.",
+        &[],
+        subsystems::SCANNER,
+    )
+});
+
+pub static SCANNER_BITROT_CYCLE_SECONDS_MD: LazyLock<MetricDescriptor> = LazyLock::new(|| {
+    new_gauge_md(
+        MetricName::ScannerBitrotCycleSeconds,
+        "Effective scanner bitrot deep-scan interval in seconds; zero means disabled or deep-scan every cycle.",
+        &[],
+        subsystems::SCANNER,
+    )
+});
+
+pub static SCANNER_CURRENT_CYCLE_MD: LazyLock<MetricDescriptor> = LazyLock::new(|| {
+    new_gauge_md(
+        MetricName::ScannerCurrentCycle,
+        "Current scanner cycle number, or zero when no cycle is running.",
+        &[],
+        subsystems::SCANNER,
+    )
+});
+
+pub static SCANNER_COMPLETED_CYCLES_MD: LazyLock<MetricDescriptor> = LazyLock::new(|| {
+    new_gauge_md(
+        MetricName::ScannerCompletedCycles,
+        "Total number of scanner cycles completed since server start.",
+        &[],
+        subsystems::SCANNER,
+    )
+});
+
+pub static SCANNER_CURRENT_CYCLE_AGE_SECONDS_MD: LazyLock<MetricDescriptor> = LazyLock::new(|| {
+    new_gauge_md(
+        MetricName::ScannerCurrentCycleAgeSeconds,
+        "Time elapsed (in seconds) since the current scanner cycle started, or zero when no cycle is running.",
+        &[],
+        subsystems::SCANNER,
+    )
+});
+
+pub static SCANNER_CURRENT_CYCLE_OBJECTS_SCANNED_MD: LazyLock<MetricDescriptor> = LazyLock::new(|| {
+    new_gauge_md(
+        MetricName::ScannerCurrentCycleObjectsScanned,
+        "Number of objects scanned by the currently running scanner cycle.",
+        &[],
+        subsystems::SCANNER,
+    )
+});
+
+pub static SCANNER_CURRENT_CYCLE_DIRECTORIES_SCANNED_MD: LazyLock<MetricDescriptor> = LazyLock::new(|| {
+    new_gauge_md(
+        MetricName::ScannerCurrentCycleDirectoriesScanned,
+        "Number of directories scanned by the currently running scanner cycle.",
+        &[],
+        subsystems::SCANNER,
+    )
+});
+
+pub static SCANNER_CURRENT_CYCLE_BUCKET_DRIVE_SCANS_MD: LazyLock<MetricDescriptor> = LazyLock::new(|| {
+    new_gauge_md(
+        MetricName::ScannerCurrentCycleBucketDriveScans,
+        "Number of bucket-drive scans finished by the currently running scanner cycle.",
+        &[],
+        subsystems::SCANNER,
+    )
+});
+
+pub static SCANNER_CURRENT_CYCLE_BUCKET_DRIVE_FAILURES_MD: LazyLock<MetricDescriptor> = LazyLock::new(|| {
+    new_gauge_md(
+        MetricName::ScannerCurrentCycleBucketDriveFailures,
+        "Number of bucket-drive scans that failed in the currently running scanner cycle.",
+        &[],
+        subsystems::SCANNER,
+    )
+});
+
+pub static SCANNER_CURRENT_CYCLE_OBJECTS_PER_SECOND_MD: LazyLock<MetricDescriptor> = LazyLock::new(|| {
+    new_gauge_md(
+        MetricName::ScannerCurrentCycleObjectsPerSecond,
+        "Object scan rate for the currently running scanner cycle.",
+        &[],
+        subsystems::SCANNER,
+    )
+});
+
+pub static SCANNER_CURRENT_CYCLE_DIRECTORIES_PER_SECOND_MD: LazyLock<MetricDescriptor> = LazyLock::new(|| {
+    new_gauge_md(
+        MetricName::ScannerCurrentCycleDirectoriesPerSecond,
+        "Directory scan rate for the currently running scanner cycle.",
+        &[],
+        subsystems::SCANNER,
+    )
+});
+
+pub static SCANNER_CURRENT_CYCLE_BUCKET_DRIVE_SCANS_PER_SECOND_MD: LazyLock<MetricDescriptor> = LazyLock::new(|| {
+    new_gauge_md(
+        MetricName::ScannerCurrentCycleBucketDriveScansPerSecond,
+        "Bucket-drive scan rate for the currently running scanner cycle.",
+        &[],
+        subsystems::SCANNER,
+    )
+});
+
+pub static SCANNER_CURRENT_CYCLE_YIELD_EVENTS_MD: LazyLock<MetricDescriptor> = LazyLock::new(|| {
+    new_gauge_md(
+        MetricName::ScannerCurrentCycleYieldEvents,
+        "Number of scanner cooperative yield events in the currently running scanner cycle.",
+        &[],
+        subsystems::SCANNER,
+    )
+});
+
+pub static SCANNER_CURRENT_CYCLE_YIELD_DURATION_SECONDS_MD: LazyLock<MetricDescriptor> = LazyLock::new(|| {
+    new_gauge_md(
+        MetricName::ScannerCurrentCycleYieldDurationSeconds,
+        "Total scanner cooperative yield duration in seconds for the currently running scanner cycle.",
+        &[],
+        subsystems::SCANNER,
+    )
+});
+
+pub static SCANNER_CURRENT_CYCLE_THROTTLE_SLEEP_EVENTS_MD: LazyLock<MetricDescriptor> = LazyLock::new(|| {
+    new_gauge_md(
+        MetricName::ScannerCurrentCycleThrottleSleepEvents,
+        "Number of scanner self-throttle sleep events in the currently running scanner cycle.",
+        &[],
+        subsystems::SCANNER,
+    )
+});
+
+pub static SCANNER_CURRENT_CYCLE_THROTTLE_SLEEP_DURATION_SECONDS_MD: LazyLock<MetricDescriptor> = LazyLock::new(|| {
+    new_gauge_md(
+        MetricName::ScannerCurrentCycleThrottleSleepDurationSeconds,
+        "Total scanner self-throttle sleep duration in seconds for the currently running scanner cycle.",
+        &[],
+        subsystems::SCANNER,
+    )
+});
+
+pub static SCANNER_CURRENT_CYCLE_ILM_ACTIONS_MD: LazyLock<MetricDescriptor> = LazyLock::new(|| {
+    new_gauge_md(
+        MetricName::ScannerCurrentCycleIlmActions,
+        "Number of lifecycle actions applied by the currently running scanner cycle.",
+        &[],
+        subsystems::SCANNER,
+    )
+});
+
+pub static SCANNER_CURRENT_CYCLE_HEAL_OBJECTS_MD: LazyLock<MetricDescriptor> = LazyLock::new(|| {
+    new_gauge_md(
+        MetricName::ScannerCurrentCycleHealObjects,
+        "Number of object heal candidates enqueued by the currently running scanner cycle.",
+        &[],
+        subsystems::SCANNER,
+    )
+});
+
+pub static SCANNER_CURRENT_CYCLE_REPLICATION_CHECKS_MD: LazyLock<MetricDescriptor> = LazyLock::new(|| {
+    new_gauge_md(
+        MetricName::ScannerCurrentCycleReplicationChecks,
+        "Number of replication heal checks run by the currently running scanner cycle.",
+        &[],
+        subsystems::SCANNER,
+    )
+});
+
+pub static SCANNER_CURRENT_CYCLE_USAGE_SAVES_MD: LazyLock<MetricDescriptor> = LazyLock::new(|| {
+    new_gauge_md(
+        MetricName::ScannerCurrentCycleUsageSaves,
+        "Number of data-usage save operations run by the currently running scanner cycle.",
+        &[],
+        subsystems::SCANNER,
+    )
+});
+
+pub static SCANNER_CURRENT_SCAN_MODE_MD: LazyLock<MetricDescriptor> = LazyLock::new(|| {
+    new_gauge_md(
+        MetricName::ScannerCurrentScanMode,
+        "Current scanner mode: 0 unknown or idle, 1 normal, 2 deep bitrot scan.",
+        &[],
+        subsystems::SCANNER,
+    )
+});
+
+pub static SCANNER_LAST_CYCLE_RESULT_MD: LazyLock<MetricDescriptor> = LazyLock::new(|| {
+    new_gauge_md(
+        MetricName::ScannerLastCycleResult,
+        "Last scanner cycle result: 0 unknown, 1 success, 2 error, 3 partial.",
+        &[],
+        subsystems::SCANNER,
+    )
+});
+
+pub static SCANNER_LAST_CYCLE_PARTIAL_REASON_MD: LazyLock<MetricDescriptor> = LazyLock::new(|| {
+    new_gauge_md(
+        MetricName::ScannerLastCyclePartialReason,
+        "Last scanner partial cycle reason: 0 unknown, 1 runtime budget, 2 object budget, 3 directory budget.",
+        &[],
+        subsystems::SCANNER,
+    )
+});
+
+pub static SCANNER_LAST_CYCLE_DURATION_SECONDS_MD: LazyLock<MetricDescriptor> = LazyLock::new(|| {
+    new_gauge_md(
+        MetricName::ScannerLastCycleDurationSeconds,
+        "Duration in seconds of the last finished scanner cycle.",
+        &[],
+        subsystems::SCANNER,
+    )
+});
+
+pub static SCANNER_LAST_CYCLE_OBJECTS_SCANNED_MD: LazyLock<MetricDescriptor> = LazyLock::new(|| {
+    new_gauge_md(
+        MetricName::ScannerLastCycleObjectsScanned,
+        "Number of objects scanned by the last finished scanner cycle.",
+        &[],
+        subsystems::SCANNER,
+    )
+});
+
+pub static SCANNER_LAST_CYCLE_DIRECTORIES_SCANNED_MD: LazyLock<MetricDescriptor> = LazyLock::new(|| {
+    new_gauge_md(
+        MetricName::ScannerLastCycleDirectoriesScanned,
+        "Number of directories scanned by the last finished scanner cycle.",
+        &[],
+        subsystems::SCANNER,
+    )
+});
+
+pub static SCANNER_LAST_CYCLE_BUCKET_DRIVE_SCANS_MD: LazyLock<MetricDescriptor> = LazyLock::new(|| {
+    new_gauge_md(
+        MetricName::ScannerLastCycleBucketDriveScans,
+        "Number of bucket-drive scans finished by the last scanner cycle.",
+        &[],
+        subsystems::SCANNER,
+    )
+});
+
+pub static SCANNER_LAST_CYCLE_BUCKET_DRIVE_FAILURES_MD: LazyLock<MetricDescriptor> = LazyLock::new(|| {
+    new_gauge_md(
+        MetricName::ScannerLastCycleBucketDriveFailures,
+        "Number of bucket-drive scans that failed in the last finished scanner cycle.",
+        &[],
+        subsystems::SCANNER,
+    )
+});
+
+pub static SCANNER_LAST_CYCLE_OBJECTS_PER_SECOND_MD: LazyLock<MetricDescriptor> = LazyLock::new(|| {
+    new_gauge_md(
+        MetricName::ScannerLastCycleObjectsPerSecond,
+        "Object scan rate for the last finished scanner cycle.",
+        &[],
+        subsystems::SCANNER,
+    )
+});
+
+pub static SCANNER_LAST_CYCLE_DIRECTORIES_PER_SECOND_MD: LazyLock<MetricDescriptor> = LazyLock::new(|| {
+    new_gauge_md(
+        MetricName::ScannerLastCycleDirectoriesPerSecond,
+        "Directory scan rate for the last finished scanner cycle.",
+        &[],
+        subsystems::SCANNER,
+    )
+});
+
+pub static SCANNER_LAST_CYCLE_BUCKET_DRIVE_SCANS_PER_SECOND_MD: LazyLock<MetricDescriptor> = LazyLock::new(|| {
+    new_gauge_md(
+        MetricName::ScannerLastCycleBucketDriveScansPerSecond,
+        "Bucket-drive scan rate for the last finished scanner cycle.",
+        &[],
+        subsystems::SCANNER,
+    )
+});
+
+pub static SCANNER_LAST_CYCLE_YIELD_EVENTS_MD: LazyLock<MetricDescriptor> = LazyLock::new(|| {
+    new_gauge_md(
+        MetricName::ScannerLastCycleYieldEvents,
+        "Number of scanner cooperative yield events in the last finished scanner cycle.",
+        &[],
+        subsystems::SCANNER,
+    )
+});
+
+pub static SCANNER_LAST_CYCLE_YIELD_DURATION_SECONDS_MD: LazyLock<MetricDescriptor> = LazyLock::new(|| {
+    new_gauge_md(
+        MetricName::ScannerLastCycleYieldDurationSeconds,
+        "Total scanner cooperative yield duration in seconds for the last finished scanner cycle.",
+        &[],
+        subsystems::SCANNER,
+    )
+});
+
+pub static SCANNER_LAST_CYCLE_THROTTLE_SLEEP_EVENTS_MD: LazyLock<MetricDescriptor> = LazyLock::new(|| {
+    new_gauge_md(
+        MetricName::ScannerLastCycleThrottleSleepEvents,
+        "Number of scanner self-throttle sleep events in the last finished scanner cycle.",
+        &[],
+        subsystems::SCANNER,
+    )
+});
+
+pub static SCANNER_LAST_CYCLE_THROTTLE_SLEEP_DURATION_SECONDS_MD: LazyLock<MetricDescriptor> = LazyLock::new(|| {
+    new_gauge_md(
+        MetricName::ScannerLastCycleThrottleSleepDurationSeconds,
+        "Total scanner self-throttle sleep duration in seconds for the last finished scanner cycle.",
+        &[],
+        subsystems::SCANNER,
+    )
+});
+
+pub static SCANNER_LAST_CYCLE_ILM_ACTIONS_MD: LazyLock<MetricDescriptor> = LazyLock::new(|| {
+    new_gauge_md(
+        MetricName::ScannerLastCycleIlmActions,
+        "Number of lifecycle actions applied by the last finished scanner cycle.",
+        &[],
+        subsystems::SCANNER,
+    )
+});
+
+pub static SCANNER_LAST_CYCLE_HEAL_OBJECTS_MD: LazyLock<MetricDescriptor> = LazyLock::new(|| {
+    new_gauge_md(
+        MetricName::ScannerLastCycleHealObjects,
+        "Number of object heal candidates enqueued by the last finished scanner cycle.",
+        &[],
+        subsystems::SCANNER,
+    )
+});
+
+pub static SCANNER_LAST_CYCLE_REPLICATION_CHECKS_MD: LazyLock<MetricDescriptor> = LazyLock::new(|| {
+    new_gauge_md(
+        MetricName::ScannerLastCycleReplicationChecks,
+        "Number of replication heal checks run by the last finished scanner cycle.",
+        &[],
+        subsystems::SCANNER,
+    )
+});
+
+pub static SCANNER_LAST_CYCLE_USAGE_SAVES_MD: LazyLock<MetricDescriptor> = LazyLock::new(|| {
+    new_gauge_md(
+        MetricName::ScannerLastCycleUsageSaves,
+        "Number of data-usage save operations run by the last finished scanner cycle.",
+        &[],
+        subsystems::SCANNER,
+    )
+});
+
+pub static SCANNER_FAILED_CYCLES_MD: LazyLock<MetricDescriptor> = LazyLock::new(|| {
+    new_counter_md(
+        MetricName::ScannerFailedCycles,
+        "Total number of scanner cycles that failed since server start.",
+        &[],
+        subsystems::SCANNER,
+    )
+});
+
+pub static SCANNER_PARTIAL_CYCLES_MD: LazyLock<MetricDescriptor> = LazyLock::new(|| {
+    new_counter_md(
+        MetricName::ScannerPartialCycles,
+        "Total number of scanner cycles stopped before completion by cycle budget.",
+        &[],
+        subsystems::SCANNER,
+    )
+});
+
+pub static SCANNER_PARTIAL_CYCLES_BY_REASON_MD: LazyLock<MetricDescriptor> = LazyLock::new(|| {
+    new_counter_md(
+        MetricName::ScannerPartialCyclesByReason,
+        "Total number of scanner cycles stopped before completion by cycle budget reason.",
+        &["reason"],
+        subsystems::SCANNER,
+    )
+});

crates/obs/src/metrics/stats_collector.rs

@@ -27,6 +27,7 @@ use crate::metrics::collectors::{
     ProcessStatusType, ReplicationStats, ResourceStats, ScannerStats,
 };
 use chrono::Utc;
+use rustfs_common::heal_channel::HealScanMode;
 use rustfs_common::metrics::global_metrics;
 use rustfs_ecstore::bucket::lifecycle::bucket_lifecycle_ops::{GLOBAL_ExpiryState, GLOBAL_TransitionState};
 use rustfs_ecstore::bucket::metadata_sys::get_quota_config;
@@ -44,6 +45,34 @@ use std::time::Duration;
 use sysinfo::{Networks, System};
 use tracing::{instrument, warn};
 
+fn current_scanner_cycle_age_seconds(
+    current_cycle: u64,
+    current_started: chrono::DateTime<Utc>,
+    now: chrono::DateTime<Utc>,
+) -> u64 {
+    if current_cycle == 0 {
+        0
+    } else {
+        now.signed_duration_since(current_started).num_seconds().max(0) as u64
+    }
+}
+
+fn scanner_scan_mode_code(scan_mode: &str) -> u64 {
+    match scan_mode {
+        mode if mode == HealScanMode::Normal.as_str() => HealScanMode::Normal as u8 as u64,
+        mode if mode == HealScanMode::Deep.as_str() => HealScanMode::Deep as u8 as u64,
+        _ => HealScanMode::Unknown as u8 as u64,
+    }
+}
+
+fn scanner_work_rate_per_second(count: u64, seconds: f64) -> f64 {
+    if seconds > 0.0 && seconds.is_finite() {
+        count as f64 / seconds
+    } else {
+        0.0
+    }
+}
+
 const DRIVE_STATE_OK: &str = "ok";
 const DRIVE_STATE_ONLINE: &str = "online";
 const DRIVE_STATE_UNFORMATTED: &str = "unformatted";
@@ -867,25 +896,116 @@ pub async fn collect_ilm_metric_stats() -> Option<IlmStats> {
 ///
 /// Task 5 maps scanner runtime snapshots from `global_metrics()` into the
 /// rustfs-obs scanner collector shape.
+fn scanner_bucket_scans_started(life_time_ops: &HashMap<String, u64>, bucket_scans_finished: u64) -> u64 {
+    life_time_ops
+        .get("scan_bucket_drive_start")
+        .copied()
+        .unwrap_or(bucket_scans_finished)
+}
+
 pub async fn collect_scanner_metric_stats() -> Option<ScannerStats> {
     let metrics = global_metrics().report().await;
+    let now = Utc::now();
     let bucket_scans_finished = metrics.life_time_ops.get("scan_bucket_drive").copied().unwrap_or_default();
+    let bucket_scans_started = scanner_bucket_scans_started(&metrics.life_time_ops, bucket_scans_finished);
+    let bucket_scans_failed = metrics
+        .life_time_ops
+        .get("scan_bucket_drive_failure")
+        .copied()
+        .unwrap_or_default();
+    let completed_cycles = metrics.life_time_ops.get("scan_cycle").copied().unwrap_or_default();
     let directories_scanned = metrics.life_time_ops.get("scan_folder").copied().unwrap_or_default();
     let objects_scanned = metrics.life_time_ops.get("scan_object").copied().unwrap_or_default();
     let versions_scanned = metrics.life_time_ilm.values().copied().sum();
     let reference_time = metrics.cycles_completed_at.last().copied().unwrap_or(metrics.current_started);
-    let last_activity_seconds = Utc::now().signed_duration_since(reference_time).num_seconds().max(0) as u64;
+    let last_activity_seconds = now.signed_duration_since(reference_time).num_seconds().max(0) as u64;
+    let active_paths = metrics.active_scan_paths as u64;
+    let current_cycle_age_seconds = current_scanner_cycle_age_seconds(metrics.current_cycle, metrics.current_started, now);
+    let current_scan_mode = scanner_scan_mode_code(&metrics.current_scan_mode);
+    let current_cycle_age = current_cycle_age_seconds as f64;
+    let last_cycle_duration = metrics.last_cycle_duration_seconds;
 
     Some(ScannerStats {
         bucket_scans_finished,
-        // `global_metrics()` currently tracks completed bucket-drive scans, not a
-        // separate started counter. Mirror the finished count until Task 5/Task 10
-        // expands the scanner runtime source shape.
-        bucket_scans_started: bucket_scans_finished,
+        bucket_scans_started,
+        bucket_scans_failed,
         directories_scanned,
         objects_scanned,
         versions_scanned,
         last_activity_seconds,
+        active_paths,
+        oldest_active_path_age_seconds: metrics.oldest_active_path_age_seconds,
+        current_set_scan_concurrency_limit: metrics.current_set_scan_concurrency_limit,
+        current_set_scans_queued: metrics.current_set_scans_queued,
+        current_set_scans_active: metrics.current_set_scans_active,
+        current_disk_scan_concurrency_limit: metrics.current_disk_scan_concurrency_limit,
+        current_disk_bucket_scans_queued: metrics.current_disk_bucket_scans_queued,
+        current_disk_bucket_scans_active: metrics.current_disk_bucket_scans_active,
+        throttle_idle_mode_enabled: metrics.throttle_idle_mode_enabled,
+        throttle_sleep_factor: metrics.throttle_sleep_factor,
+        throttle_max_sleep_seconds: metrics.throttle_max_sleep_seconds,
+        yield_every_n_objects: metrics.yield_every_n_objects,
+        cycle_interval_seconds: metrics.cycle_interval_seconds,
+        cycle_max_duration_seconds: metrics.cycle_max_duration_seconds,
+        cycle_max_objects: metrics.cycle_max_objects,
+        cycle_max_directories: metrics.cycle_max_directories,
+        bitrot_cycle_enabled: metrics.bitrot_cycle_enabled,
+        bitrot_cycle_seconds: metrics.bitrot_cycle_seconds,
+        current_cycle: metrics.current_cycle,
+        completed_cycles,
+        current_cycle_age_seconds,
+        current_cycle_objects_scanned: metrics.current_cycle_objects_scanned,
+        current_cycle_directories_scanned: metrics.current_cycle_directories_scanned,
+        current_cycle_bucket_drive_scans: metrics.current_cycle_bucket_drive_scans,
+        current_cycle_bucket_drive_failures: metrics.current_cycle_bucket_drive_failures,
+        current_cycle_objects_per_second: scanner_work_rate_per_second(metrics.current_cycle_objects_scanned, current_cycle_age),
+        current_cycle_directories_per_second: scanner_work_rate_per_second(
+            metrics.current_cycle_directories_scanned,
+            current_cycle_age,
+        ),
+        current_cycle_bucket_drive_scans_per_second: scanner_work_rate_per_second(
+            metrics.current_cycle_bucket_drive_scans,
+            current_cycle_age,
+        ),
+        current_cycle_yield_events: metrics.current_cycle_yield_events,
+        current_cycle_yield_duration_seconds: metrics.current_cycle_yield_duration_seconds,
+        current_cycle_throttle_sleep_events: metrics.current_cycle_throttle_sleep_events,
+        current_cycle_throttle_sleep_duration_seconds: metrics.current_cycle_throttle_sleep_duration_seconds,
+        current_cycle_ilm_actions: metrics.current_cycle_ilm_actions,
+        current_cycle_heal_objects: metrics.current_cycle_heal_objects,
+        current_cycle_replication_checks: metrics.current_cycle_replication_checks,
+        current_cycle_usage_saves: metrics.current_cycle_usage_saves,
+        current_scan_mode,
+        last_cycle_result: metrics.last_cycle_result_code,
+        last_cycle_partial_reason: metrics.last_cycle_partial_reason_code,
+        last_cycle_duration_seconds: metrics.last_cycle_duration_seconds,
+        last_cycle_objects_scanned: metrics.last_cycle_objects_scanned,
+        last_cycle_directories_scanned: metrics.last_cycle_directories_scanned,
+        last_cycle_bucket_drive_scans: metrics.last_cycle_bucket_drive_scans,
+        last_cycle_bucket_drive_failures: metrics.last_cycle_bucket_drive_failures,
+        last_cycle_objects_per_second: scanner_work_rate_per_second(metrics.last_cycle_objects_scanned, last_cycle_duration),
+        last_cycle_directories_per_second: scanner_work_rate_per_second(
+            metrics.last_cycle_directories_scanned,
+            last_cycle_duration,
+        ),
+        last_cycle_bucket_drive_scans_per_second: scanner_work_rate_per_second(
+            metrics.last_cycle_bucket_drive_scans,
+            last_cycle_duration,
+        ),
+        last_cycle_yield_events: metrics.last_cycle_yield_events,
+        last_cycle_yield_duration_seconds: metrics.last_cycle_yield_duration_seconds,
+        last_cycle_throttle_sleep_events: metrics.last_cycle_throttle_sleep_events,
+        last_cycle_throttle_sleep_duration_seconds: metrics.last_cycle_throttle_sleep_duration_seconds,
+        last_cycle_ilm_actions: metrics.last_cycle_ilm_actions,
+        last_cycle_heal_objects: metrics.last_cycle_heal_objects,
+        last_cycle_replication_checks: metrics.last_cycle_replication_checks,
+        last_cycle_usage_saves: metrics.last_cycle_usage_saves,
+        failed_cycles: metrics.failed_cycles,
+        partial_cycles: metrics.partial_cycles,
+        partial_cycles_unknown: metrics.partial_cycles_unknown,
+        partial_cycles_runtime: metrics.partial_cycles_runtime,
+        partial_cycles_objects: metrics.partial_cycles_objects,
+        partial_cycles_directories: metrics.partial_cycles_directories,
     })
 }
 
@@ -955,4 +1075,63 @@ mod tests {
         assert_eq!(stats.write_health, 1);
         assert_eq!(stats.health, 1);
     }
+
+    #[test]
+    fn current_scanner_cycle_age_seconds_returns_zero_when_idle() {
+        let now = Utc::now();
+
+        assert_eq!(current_scanner_cycle_age_seconds(0, now - chrono::Duration::seconds(30), now), 0);
+    }
+
+    #[test]
+    fn current_scanner_cycle_age_seconds_clamps_future_start() {
+        let now = Utc::now();
+
+        assert_eq!(current_scanner_cycle_age_seconds(4, now + chrono::Duration::seconds(30), now), 0);
+    }
+
+    #[test]
+    fn current_scanner_cycle_age_seconds_reports_active_elapsed_time() {
+        let now = Utc::now();
+
+        assert_eq!(current_scanner_cycle_age_seconds(4, now - chrono::Duration::seconds(45), now), 45);
+    }
+
+    #[test]
+    fn scanner_scan_mode_code_maps_known_modes() {
+        assert_eq!(scanner_scan_mode_code(HealScanMode::Normal.as_str()), HealScanMode::Normal as u8 as u64);
+        assert_eq!(scanner_scan_mode_code(HealScanMode::Deep.as_str()), HealScanMode::Deep as u8 as u64);
+    }
+
+    #[test]
+    fn scanner_scan_mode_code_maps_unknown_mode() {
+        assert_eq!(scanner_scan_mode_code(""), HealScanMode::Unknown as u8 as u64);
+    }
+
+    #[test]
+    fn scanner_bucket_scans_started_uses_explicit_started_count() {
+        let mut life_time_ops = HashMap::new();
+        life_time_ops.insert("scan_bucket_drive_start".to_string(), 7);
+
+        assert_eq!(scanner_bucket_scans_started(&life_time_ops, 5), 7);
+    }
+
+    #[test]
+    fn scanner_bucket_scans_started_falls_back_to_finished_count() {
+        let life_time_ops = HashMap::new();
+
+        assert_eq!(scanner_bucket_scans_started(&life_time_ops, 5), 5);
+    }
+
+    #[test]
+    fn scanner_work_rate_per_second_reports_rate() {
+        assert_eq!(scanner_work_rate_per_second(90, 45.0), 2.0);
+    }
+
+    #[test]
+    fn scanner_work_rate_per_second_returns_zero_for_invalid_seconds() {
+        assert_eq!(scanner_work_rate_per_second(90, 0.0), 0.0);
+        assert_eq!(scanner_work_rate_per_second(90, f64::INFINITY), 0.0);
+        assert_eq!(scanner_work_rate_per_second(90, f64::NAN), 0.0);
+    }
 }

crates/obs/src/telemetry/guard.rs

@@ -43,6 +43,8 @@ pub struct OtelGuard {
     pub(crate) logger_provider: Option<SdkLoggerProvider>,
     #[cfg(any(target_os = "linux", target_os = "macos"))]
     pub(crate) profiling_agent: Option<pyroscope::PyroscopeAgent<pyroscope::pyroscope::PyroscopeAgentRunning>>,
+    #[cfg(any(target_os = "linux", target_os = "macos"))]
+    pub(crate) memory_profiling_agent: Option<pyroscope::PyroscopeAgent<pyroscope::pyroscope::PyroscopeAgentRunning>>,
     /// Handle to the background log-cleanup task; aborted on drop.
     pub(crate) cleanup_handle: Option<tokio::task::JoinHandle<()>>,
     /// Worker guard that keeps the non-blocking `tracing_appender` thread
@@ -60,6 +62,8 @@ impl std::fmt::Debug for OtelGuard {
             .field("logger_provider", &self.logger_provider.is_some());
         #[cfg(any(target_os = "linux", target_os = "macos"))]
         s.field("profiling_agent", &self.profiling_agent.is_some());
+        #[cfg(any(target_os = "linux", target_os = "macos"))]
+        s.field("memory_profiling_agent", &self.memory_profiling_agent.is_some());
         s.field("cleanup_handle", &self.cleanup_handle.is_some())
             .field("tracing_guard", &self.tracing_guard.is_some())
             .field("stdout_guard", &self.stdout_guard.is_some())
@@ -101,6 +105,16 @@ impl Drop for OtelGuard {
             }
         }
 
+        #[cfg(any(target_os = "linux", target_os = "macos"))]
+        if let Some(agent) = self.memory_profiling_agent.take() {
+            match agent.stop() {
+                Err(err) => eprintln!("Memory profiling agent stop error: {err:?}"),
+                Ok(stopped) => {
+                    stopped.shutdown();
+                }
+            }
+        }
+
         if let Some(handle) = self.cleanup_handle.take() {
             handle.abort();
             eprintln!("Log cleanup task stopped");

crates/obs/src/telemetry/local.rs

@@ -157,6 +157,8 @@ fn init_stdout_only(_config: &OtelConfig, logger_level: &str, is_production: boo
         logger_provider: None,
         #[cfg(any(target_os = "linux", target_os = "macos"))]
         profiling_agent: None,
+        #[cfg(any(target_os = "linux", target_os = "macos"))]
+        memory_profiling_agent: None,
         tracing_guard: Some(guard),
         stdout_guard: None,
         cleanup_handle: None,
@@ -291,6 +293,8 @@ fn init_file_logging_internal(
         logger_provider: None,
         #[cfg(any(target_os = "linux", target_os = "macos"))]
         profiling_agent: None,
+        #[cfg(any(target_os = "linux", target_os = "macos"))]
+        memory_profiling_agent: None,
         tracing_guard: Some(guard),
         stdout_guard,
         cleanup_handle: Some(cleanup_handle),

crates/obs/src/telemetry/otel.rs

@@ -167,6 +167,9 @@ pub(super) fn init_observability_http(
     #[cfg(any(target_os = "linux", target_os = "macos"))]
     let profiling_agent = init_profiler(config);
 
+    #[cfg(any(target_os = "linux", target_os = "macos"))]
+    let memory_profiling_agent = init_memory_profiler(config);
+
     // ── Logger Logic ──────────────────────────────────────────────────────────
     // Logging is the only signal that may intentionally route to either OTLP
     // or local files depending on configuration completeness.
@@ -316,6 +319,8 @@ pub(super) fn init_observability_http(
         logger_provider,
         #[cfg(any(target_os = "linux", target_os = "macos"))]
         profiling_agent,
+        #[cfg(any(target_os = "linux", target_os = "macos"))]
+        memory_profiling_agent,
         tracing_guard,
         stdout_guard,
         cleanup_handle,
@@ -516,7 +521,7 @@ fn init_profiler(config: &OtelConfig) -> Option<pyroscope::PyroscopeAgent<pyrosc
     let sample_rate = 100; // 100 Hz
 
     let agent = PyroscopeAgentBuilder::new(endpoint, service_name, sample_rate, "pyroscope-rs", "1.0.1", backend)
-        .tags(vec![("version", version)]) // TODO: add git commit tag
+        .tags(vec![("version", version), ("profile_type", "cpu")])
         .build()
         .ok()?;
 
@@ -529,6 +534,60 @@ fn init_profiler(config: &OtelConfig) -> Option<pyroscope::PyroscopeAgent<pyrosc
     }
 }
 
+/// Initialise a Pyroscope agent for continuous **memory** profiling via jemalloc.
+///
+/// This is only available on `linux + gnu + x86_64` where tikv-jemallocator
+/// is the global allocator and `jemalloc_pprof::PROF_CTL` is accessible.
+///
+/// Returns `None` when profiling export is disabled, the endpoint is missing,
+/// jemalloc profiling is not activated, or the agent fails to build/start.
+#[cfg(any(target_os = "linux", target_os = "macos"))]
+fn init_memory_profiler(config: &OtelConfig) -> Option<pyroscope::PyroscopeAgent<pyroscope::pyroscope::PyroscopeAgentRunning>> {
+    use pyroscope::backend::jemalloc_backend;
+    use pyroscope::pyroscope::PyroscopeAgentBuilder;
+    use rustfs_config::VERSION;
+
+    if !config
+        .profiling_export_enabled
+        .unwrap_or(rustfs_config::DEFAULT_OBS_PROFILING_EXPORT_ENABLED)
+    {
+        return None;
+    }
+
+    let endpoint = config.profiling_endpoint.as_ref()?.as_str();
+    if endpoint.is_empty() {
+        return None;
+    }
+
+    // Verify jemalloc profiling is available and activated
+    {
+        let prof_ctl = jemalloc_pprof::PROF_CTL.as_ref()?;
+        let ctl = prof_ctl.try_lock().ok()?;
+        if !ctl.activated() {
+            eprintln!("Memory profiling skipped: jemalloc profiling is not activated");
+            return None;
+        }
+    }
+
+    let backend = jemalloc_backend();
+    let service_name = config.service_name.as_deref().unwrap_or(APP_NAME);
+    let version = config.service_version.as_deref().unwrap_or(VERSION);
+    let sample_rate = 100;
+
+    let agent = PyroscopeAgentBuilder::new(endpoint, service_name, sample_rate, "pyroscope-rs", "1.0.1", backend)
+        .tags(vec![("version", version), ("profile_type", "memory")])
+        .build()
+        .ok()?;
+
+    match agent.start() {
+        Ok(agent) => Some(agent),
+        Err(err) => {
+            eprintln!("Memory profiling agent start error: {err:?}");
+            None
+        }
+    }
+}
+
 /// Create a stdout periodic metrics reader for the given interval.
 ///
 /// This helper is primarily used for local development and diagnostics when

crates/policy/src/policy/action.rs

@@ -502,6 +502,46 @@ pub enum AdminAction {
     ImportBucketMetadataAction,
     #[strum(serialize = "admin:ExportBucketMetadata")]
     ExportBucketMetadataAction,
+    #[strum(serialize = "admin:GetTableCatalog")]
+    GetTableCatalogAction,
+    #[strum(serialize = "admin:GetTableBucket")]
+    GetTableBucketAction,
+    #[strum(serialize = "admin:SetTableBucket")]
+    SetTableBucketAction,
+    #[strum(serialize = "admin:GetTableNamespace")]
+    GetTableNamespaceAction,
+    #[strum(serialize = "admin:SetTableNamespace")]
+    SetTableNamespaceAction,
+    #[strum(serialize = "admin:UpdateTableNamespaceProperties")]
+    UpdateTableNamespacePropertiesAction,
+    #[strum(serialize = "admin:DeleteTableNamespace")]
+    DeleteTableNamespaceAction,
+    #[strum(serialize = "admin:GetTable")]
+    GetTableAction,
+    #[strum(serialize = "admin:SetTable")]
+    SetTableAction,
+    #[strum(serialize = "admin:CreateTable")]
+    CreateTableAction,
+    #[strum(serialize = "admin:RegisterTable")]
+    RegisterTableAction,
+    #[strum(serialize = "admin:CommitTable")]
+    CommitTableAction,
+    #[strum(serialize = "admin:DeleteTable")]
+    DeleteTableAction,
+    #[strum(serialize = "admin:GetTableLifecycle")]
+    GetTableLifecycleAction,
+    #[strum(serialize = "admin:SetTableLifecycle")]
+    SetTableLifecycleAction,
+    #[strum(serialize = "admin:RunTableMaintenance")]
+    RunTableMaintenanceAction,
+    #[strum(serialize = "admin:GetTableMetadataLocation")]
+    GetTableMetadataLocationAction,
+    #[strum(serialize = "admin:SetTableMetadataLocation")]
+    SetTableMetadataLocationAction,
+    #[strum(serialize = "admin:GetTableMetadata")]
+    GetTableMetadataAction,
+    #[strum(serialize = "admin:SetTableMetadata")]
+    SetTableMetadataAction,
     #[strum(serialize = "admin:SetTier")]
     SetTierAction,
     #[strum(serialize = "admin:ListTier")]
@@ -586,6 +626,26 @@ impl AdminAction {
                 | AdminAction::GetReplicationMetricsAction
                 | AdminAction::ImportBucketMetadataAction
                 | AdminAction::ExportBucketMetadataAction
+                | AdminAction::GetTableCatalogAction
+                | AdminAction::GetTableBucketAction
+                | AdminAction::SetTableBucketAction
+                | AdminAction::GetTableNamespaceAction
+                | AdminAction::SetTableNamespaceAction
+                | AdminAction::UpdateTableNamespacePropertiesAction
+                | AdminAction::DeleteTableNamespaceAction
+                | AdminAction::GetTableAction
+                | AdminAction::SetTableAction
+                | AdminAction::CreateTableAction
+                | AdminAction::RegisterTableAction
+                | AdminAction::CommitTableAction
+                | AdminAction::DeleteTableAction
+                | AdminAction::GetTableLifecycleAction
+                | AdminAction::SetTableLifecycleAction
+                | AdminAction::RunTableMaintenanceAction
+                | AdminAction::GetTableMetadataLocationAction
+                | AdminAction::SetTableMetadataLocationAction
+                | AdminAction::GetTableMetadataAction
+                | AdminAction::SetTableMetadataAction
                 | AdminAction::SetTierAction
                 | AdminAction::ListTierAction
                 | AdminAction::ExportIAMAction
@@ -686,4 +746,101 @@ mod tests {
     fn test_get_replication_metrics_admin_action_is_valid() {
         assert!(AdminAction::GetReplicationMetricsAction.is_valid());
     }
+
+    #[test]
+    fn test_table_catalog_admin_action_is_valid() {
+        let get_action = AdminAction::try_from("admin:GetTableCatalog").expect("Should parse GetTableCatalog action");
+
+        assert_eq!(get_action, AdminAction::GetTableCatalogAction);
+        assert!(get_action.is_valid());
+    }
+
+    #[test]
+    fn test_table_bucket_admin_actions_are_valid() {
+        let get_action = AdminAction::try_from("admin:GetTableBucket").expect("Should parse GetTableBucket action");
+        let set_action = AdminAction::try_from("admin:SetTableBucket").expect("Should parse SetTableBucket action");
+
+        assert_eq!(get_action, AdminAction::GetTableBucketAction);
+        assert_eq!(set_action, AdminAction::SetTableBucketAction);
+        assert!(get_action.is_valid());
+        assert!(set_action.is_valid());
+    }
+
+    #[test]
+    fn test_table_namespace_admin_actions_are_valid() {
+        let get_action = AdminAction::try_from("admin:GetTableNamespace").expect("Should parse GetTableNamespace action");
+        let set_action = AdminAction::try_from("admin:SetTableNamespace").expect("Should parse SetTableNamespace action");
+        let update_properties_action = AdminAction::try_from("admin:UpdateTableNamespaceProperties")
+            .expect("Should parse UpdateTableNamespaceProperties action");
+        let delete_action =
+            AdminAction::try_from("admin:DeleteTableNamespace").expect("Should parse DeleteTableNamespace action");
+
+        assert_eq!(get_action, AdminAction::GetTableNamespaceAction);
+        assert_eq!(set_action, AdminAction::SetTableNamespaceAction);
+        assert_eq!(update_properties_action, AdminAction::UpdateTableNamespacePropertiesAction);
+        assert_eq!(delete_action, AdminAction::DeleteTableNamespaceAction);
+        assert!(get_action.is_valid());
+        assert!(set_action.is_valid());
+        assert!(update_properties_action.is_valid());
+        assert!(delete_action.is_valid());
+    }
+
+    #[test]
+    fn test_table_admin_actions_are_valid() {
+        let get_action = AdminAction::try_from("admin:GetTable").expect("Should parse GetTable action");
+        let set_action = AdminAction::try_from("admin:SetTable").expect("Should parse SetTable action");
+        let create_action = AdminAction::try_from("admin:CreateTable").expect("Should parse CreateTable action");
+        let register_action = AdminAction::try_from("admin:RegisterTable").expect("Should parse RegisterTable action");
+        let commit_action = AdminAction::try_from("admin:CommitTable").expect("Should parse CommitTable action");
+        let delete_action = AdminAction::try_from("admin:DeleteTable").expect("Should parse DeleteTable action");
+        let get_lifecycle_action =
+            AdminAction::try_from("admin:GetTableLifecycle").expect("Should parse GetTableLifecycle action");
+        let set_lifecycle_action =
+            AdminAction::try_from("admin:SetTableLifecycle").expect("Should parse SetTableLifecycle action");
+        let run_maintenance_action =
+            AdminAction::try_from("admin:RunTableMaintenance").expect("Should parse RunTableMaintenance action");
+
+        assert_eq!(get_action, AdminAction::GetTableAction);
+        assert_eq!(set_action, AdminAction::SetTableAction);
+        assert_eq!(create_action, AdminAction::CreateTableAction);
+        assert_eq!(register_action, AdminAction::RegisterTableAction);
+        assert_eq!(commit_action, AdminAction::CommitTableAction);
+        assert_eq!(delete_action, AdminAction::DeleteTableAction);
+        assert_eq!(get_lifecycle_action, AdminAction::GetTableLifecycleAction);
+        assert_eq!(set_lifecycle_action, AdminAction::SetTableLifecycleAction);
+        assert_eq!(run_maintenance_action, AdminAction::RunTableMaintenanceAction);
+        assert!(get_action.is_valid());
+        assert!(set_action.is_valid());
+        assert!(create_action.is_valid());
+        assert!(register_action.is_valid());
+        assert!(commit_action.is_valid());
+        assert!(delete_action.is_valid());
+        assert!(get_lifecycle_action.is_valid());
+        assert!(set_lifecycle_action.is_valid());
+        assert!(run_maintenance_action.is_valid());
+    }
+
+    #[test]
+    fn test_table_metadata_location_admin_actions_are_valid() {
+        let get_action =
+            AdminAction::try_from("admin:GetTableMetadataLocation").expect("Should parse GetTableMetadataLocation action");
+        let set_action =
+            AdminAction::try_from("admin:SetTableMetadataLocation").expect("Should parse SetTableMetadataLocation action");
+
+        assert_eq!(get_action, AdminAction::GetTableMetadataLocationAction);
+        assert_eq!(set_action, AdminAction::SetTableMetadataLocationAction);
+        assert!(get_action.is_valid());
+        assert!(set_action.is_valid());
+    }
+
+    #[test]
+    fn test_table_metadata_admin_actions_are_valid() {
+        let get_action = AdminAction::try_from("admin:GetTableMetadata").expect("Should parse GetTableMetadata action");
+        let set_action = AdminAction::try_from("admin:SetTableMetadata").expect("Should parse SetTableMetadata action");
+
+        assert_eq!(get_action, AdminAction::GetTableMetadataAction);
+        assert_eq!(set_action, AdminAction::SetTableMetadataAction);
+        assert!(get_action.is_valid());
+        assert!(set_action.is_valid());
+    }
 }

crates/policy/src/policy/policy.rs

@@ -55,6 +55,7 @@ impl Args<'_> {
 }
 
 #[derive(Serialize, Deserialize, Clone, Default, Debug)]
+#[serde(deny_unknown_fields)]
 pub struct Policy {
     #[serde(default, rename = "ID")]
     pub id: ID,
@@ -192,6 +193,7 @@ pub struct BucketPolicyArgs<'a> {
 /// Bucket Policy with AWS S3-compatible JSON serialization.
 /// Empty optional fields are omitted from output to match AWS format.
 #[derive(Serialize, Deserialize, Clone, Default, Debug)]
+#[serde(deny_unknown_fields)]
 pub struct BucketPolicy {
     #[serde(default, rename = "Id", skip_serializing_if = "ID::is_empty")]
     pub id: ID,

crates/policy/src/policy/principal.rs

@@ -68,6 +68,7 @@ enum PrincipalFormat {
 }
 
 #[derive(serde::Deserialize)]
+#[serde(deny_unknown_fields)]
 struct PrincipalObject {
     #[serde(rename = "AWS", default)]
     aws: Option<PrincipalValues>,

crates/policy/src/policy/statement.rs

@@ -22,6 +22,7 @@ use crate::error::{Error, Result};
 use serde::{Deserialize, Serialize};
 
 #[derive(Serialize, Deserialize, Clone, Default, Debug)]
+#[serde(deny_unknown_fields)]
 pub struct Statement {
     #[serde(rename = "Sid", default)]
     pub sid: ID,
@@ -284,7 +285,7 @@ impl PartialEq for Statement {
 /// Bucket Policy Statement with AWS S3-compatible JSON serialization.
 /// Empty optional fields are omitted from output to match AWS format.
 #[derive(Debug, Deserialize, Serialize, Default, Clone)]
-#[serde(rename_all = "PascalCase", default)]
+#[serde(rename_all = "PascalCase", default, deny_unknown_fields)]
 pub struct BPStatement {
     #[serde(rename = "Sid", default, skip_serializing_if = "ID::is_empty")]
     pub sid: ID,

crates/protocols/src/swift/handler.rs

@@ -721,10 +721,10 @@ async fn handle_authenticated_request(
                                     response = response.header("etag", etag);
                                 }
 
-                                for (key, value) in info.user_defined {
+                                for (key, value) in info.user_defined.iter() {
                                     if key != "content-type" {
                                         let header_name = format!("x-object-meta-{}", key);
-                                        response = response.header(header_name, value);
+                                        response = response.header(header_name, value.as_str());
                                     }
                                 }
 
@@ -790,10 +790,10 @@ async fn handle_authenticated_request(
                     }
 
                     // Add custom metadata headers (X-Object-Meta-*)
-                    for (key, value) in info.user_defined {
+                    for (key, value) in info.user_defined.iter() {
                         if key != "content-type" {
                             let header_name = format!("x-object-meta-{}", key);
-                            response = response.header(header_name, value);
+                            response = response.header(header_name, value.as_str());
                         }
                     }
 
@@ -829,10 +829,10 @@ async fn handle_authenticated_request(
                     }
 
                     // Add custom metadata headers (X-Object-Meta-*)
-                    for (key, value) in info.user_defined {
+                    for (key, value) in info.user_defined.iter() {
                         if key != "content-type" {
                             let header_name = format!("x-object-meta-{}", key);
-                            response = response.header(header_name, value);
+                            response = response.header(header_name, value.as_str());
                         }
                     }
 
@@ -1134,10 +1134,10 @@ async fn handle_object_get(
                     response = response.header("etag", etag);
                 }
 
-                for (key, value) in info.user_defined {
+                for (key, value) in info.user_defined.iter() {
                     if key != "content-type" {
                         let header_name = format!("x-object-meta-{}", key);
-                        response = response.header(header_name, value);
+                        response = response.header(header_name, value.as_str());
                     }
                 }
 
@@ -1200,13 +1200,13 @@ async fn handle_object_get(
         response = response.header("etag", etag);
     }
 
-    for (key, value) in info.user_defined {
+    for (key, value) in info.user_defined.iter() {
         if key == "x-delete-at" {
             // Add X-Delete-At header directly (not as X-Object-Meta-*)
-            response = response.header("x-delete-at", value);
+            response = response.header("x-delete-at", value.as_str());
         } else if key != "content-type" {
             let header_name = format!("x-object-meta-{}", key);
-            response = response.header(header_name, value);
+            response = response.header(header_name, value.as_str());
         }
     }
 
@@ -1256,13 +1256,13 @@ async fn handle_object_head(
         response = response.header("etag", etag);
     }
 
-    for (key, value) in info.user_defined {
+    for (key, value) in info.user_defined.iter() {
         if key == "x-delete-at" {
             // Add X-Delete-At header directly (not as X-Object-Meta-*)
-            response = response.header("x-delete-at", value);
+            response = response.header("x-delete-at", value.as_str());
         } else if key != "content-type" {
             let header_name = format!("x-object-meta-{}", key);
-            response = response.header(header_name, value);
+            response = response.header(header_name, value.as_str());
         }
     }
 

crates/protocols/src/swift/object.rs

@@ -867,7 +867,7 @@ pub async fn copy_object(
 
     // 10. Prepare metadata for destination object
     // Start with source metadata
-    let mut new_metadata = src_info.user_defined.clone();
+    let mut new_metadata = (*src_info.user_defined).clone();
 
     // 11. If custom metadata headers provided, use those instead (Swift behavior)
     let mut has_custom_meta = false;

crates/rio/src/errors.rs

@@ -62,6 +62,13 @@ pub struct ChecksumMismatch {
     pub got: String,
 }
 
+/// Request body ended before the declared size was fully read.
+#[derive(Error, Debug, Clone, PartialEq)]
+#[error("Incomplete body: {remaining} bytes were still expected")]
+pub struct IncompleteBody {
+    pub remaining: i64,
+}
+
 /// Invalid checksum error
 #[derive(Error, Debug, Clone, PartialEq)]
 #[error("invalid checksum")]

crates/rio/src/hardlimit_reader.rs

@@ -12,6 +12,7 @@
 // See the License for the specific language governing permissions and
 // limitations under the License.
 
+use crate::IncompleteBody;
 use pin_project_lite::pin_project;
 use std::io::{Error, Result};
 use std::pin::Pin;
@@ -40,8 +41,25 @@ where
         if self.remaining < 0 {
             return Poll::Ready(Err(Error::other("input provided more bytes than specified")));
         }
+        let original_filled = buf.filled().len();
+        if self.remaining == 0 {
+            let mut discard = [0u8; 8192];
+            let mut discard_buf = ReadBuf::new(&mut discard);
+            return match self.as_mut().project().inner.poll_read(cx, &mut discard_buf) {
+                Poll::Pending => Poll::Pending,
+                Poll::Ready(Ok(())) => {
+                    if discard_buf.filled().is_empty() {
+                        debug_assert_eq!(buf.filled().len(), original_filled);
+                        Poll::Ready(Ok(()))
+                    } else {
+                        Poll::Ready(Err(Error::other("input provided more bytes than specified")))
+                    }
+                }
+                Poll::Ready(Err(err)) => Poll::Ready(Err(err)),
+            };
+        }
         // Save the initial length
-        let before = buf.filled().len();
+        let before = original_filled;
 
         // Poll the inner reader
         let this = self.as_mut().project();
@@ -50,6 +68,14 @@ where
         if let Poll::Ready(Ok(())) = &poll {
             let after = buf.filled().len();
             let read = (after - before) as i64;
+            if read == 0 && *this.remaining > 0 {
+                return Poll::Ready(Err(Error::new(
+                    std::io::ErrorKind::UnexpectedEof,
+                    IncompleteBody {
+                        remaining: *this.remaining,
+                    },
+                )));
+            }
             *this.remaining -= read;
             if *this.remaining < 0 {
                 return Poll::Ready(Err(Error::other("input provided more bytes than specified")));
@@ -73,7 +99,7 @@ mod tests {
     async fn test_hardlimit_reader_normal() {
         let data = b"hello world";
         let reader = BufReader::new(&data[..]);
-        let hardlimit = HardLimitReader::new(reader, 20);
+        let hardlimit = HardLimitReader::new(reader, data.len() as i64);
         let mut r = hardlimit;
         let mut buf = Vec::new();
         let n = r.read_to_end(&mut buf).await.unwrap();
@@ -121,11 +147,52 @@ mod tests {
     async fn test_hardlimit_reader_empty() {
         let data = b"";
         let reader = BufReader::new(&data[..]);
-        let hardlimit = HardLimitReader::new(reader, 5);
+        let hardlimit = HardLimitReader::new(reader, 0);
         let mut r = hardlimit;
         let mut buf = Vec::new();
         let n = r.read_to_end(&mut buf).await.unwrap();
         assert_eq!(n, 0);
         assert_eq!(&buf, data);
     }
+
+    #[tokio::test]
+    async fn test_hardlimit_reader_short_input_returns_unexpected_eof() {
+        let data = b"abc";
+        let reader = BufReader::new(&data[..]);
+        let mut r = HardLimitReader::new(reader, 5);
+        let mut buf = [0u8; 8];
+
+        let err = read_full(&mut r, &mut buf)
+            .await
+            .expect_err("short input must surface unexpected eof");
+
+        assert_eq!(err.kind(), std::io::ErrorKind::UnexpectedEof);
+        assert!(
+            err.get_ref()
+                .and_then(|inner| inner.downcast_ref::<std::io::Error>())
+                .and_then(|inner| inner.get_ref())
+                .and_then(|inner| inner.downcast_ref::<IncompleteBody>())
+                .is_some(),
+            "error should retain the incomplete body marker"
+        );
+    }
+
+    #[tokio::test]
+    async fn test_hardlimit_reader_rejects_extra_bytes_after_limit() {
+        let data = b"abcdef";
+        let reader = BufReader::new(&data[..]);
+        let mut r = HardLimitReader::new(reader, 3);
+
+        let mut first = [0u8; 3];
+        let n = read_full(&mut r, &mut first).await.expect("first read should consume limit");
+        assert_eq!(n, 3);
+        assert_eq!(&first, b"abc");
+
+        let mut second = [0u8; 1];
+        let err = read_full(&mut r, &mut second)
+            .await
+            .expect_err("bytes beyond the declared limit must be rejected");
+        assert_eq!(err.kind(), std::io::ErrorKind::Other);
+        assert!(err.to_string().contains("more bytes than specified"));
+    }
 }

crates/rio/src/http_reader.rs

@@ -248,22 +248,24 @@ impl HttpReader {
 
         let resp = request.send().await.map_err(|e| {
             record_internode_error(track_internode_metrics, internode_operation);
-            Error::other(format!("HttpReader HTTP request error: {e}"))
+            Error::other(format!("HttpReader HTTP request error for {method} {url}: {e}"))
         })?;
 
         if resp.status().is_success().not() {
             record_internode_error(track_internode_metrics, internode_operation);
             return Err(Error::other(format!(
-                "HttpReader HTTP request failed with non-200 status {}",
-                resp.status()
+                "HttpReader HTTP request failed for {method} {url} with non-200 status {}",
+                resp.status(),
             )));
         }
 
         record_internode_outgoing_request(track_internode_metrics, internode_operation);
 
+        let stream_error_url = url.clone();
+        let stream_error_method = method.clone();
         let stream = resp.bytes_stream().map_err(move |e| {
             record_internode_error(track_internode_metrics, internode_operation);
-            Error::other(format!("HttpReader stream error: {e}"))
+            Error::other(format!("HttpReader stream error for {stream_error_method} {stream_error_url}: {e}"))
         });
 
         Ok(Self {
@@ -837,10 +839,13 @@ mod tests {
         assert_eq!(&first, b"hello");
 
         let mut next = [0u8; 1];
-        let err = tokio::time::timeout(Duration::from_secs(1), reader.read(&mut next))
+        let read_result = tokio::time::timeout(Duration::from_secs(1), reader.read(&mut next))
             .await
-            .expect("stall timeout should wake reader")
-            .expect_err("reader should return a timeout error");
+            .expect("stall timeout should wake reader");
+        let err = match read_result {
+            Ok(_) => panic!("reader should return a timeout error"),
+            Err(err) => err,
+        };
         assert_eq!(err.kind(), io::ErrorKind::TimedOut);
 
         handle.abort();
@@ -898,6 +903,23 @@ mod tests {
         handle.abort();
     }
 
+    #[tokio::test]
+    async fn http_reader_request_error_includes_method_and_url() {
+        let listener = TcpListener::bind("127.0.0.1:0").await.unwrap();
+        let addr = listener.local_addr().unwrap();
+        drop(listener);
+
+        let url = format!("http://{addr}/stream");
+        let err = match HttpReader::new(url.clone(), Method::GET, HeaderMap::new(), None).await {
+            Ok(_) => panic!("closed listener should trigger request error"),
+            Err(err) => err,
+        };
+
+        let err_text = err.to_string();
+        assert!(err_text.contains("HttpReader HTTP request error for GET"));
+        assert!(err_text.contains(&url));
+    }
+
     #[test]
     fn loopback_urls_bypass_proxy_selection() {
         assert!(should_bypass_proxy_for_url("http://127.0.0.1:9000/stream"));