Fix remaining review follow-ups for #2361 (#2421)

Signed-off-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: jeadie <jack@spice.ai>
Co-authored-by: Jack Eadie <jack.eadie0@gmail.com>
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

.agents/skills/code-change-verification/SKILL.md

@@ -0,0 +1,78 @@
+---
+name: code-change-verification
+description: Verify code changes by identifying correctness, regression, security, and performance risks from diffs or patches, then produce prioritized findings with file/line evidence and concrete fixes. Use when reviewing commits, PRs, and merged patches before/after release.
+---
+
+# Code Change Verification
+
+Use this skill to review code changes consistently before merge, before release, and during incident follow-up.
+
+## Quick Start
+
+1. Read the scope: commit, PR, patch, or file list.
+2. Map each changed area by risk and user impact.
+3. Inspect each risky change in context.
+4. Report findings first, ordered by severity.
+5. Close with residual risks and verification recommendations.
+
+## Core Workflow
+
+### 1) Scope and assumptions
+- Confirm change source (diff, commit, PR, files), target branch, language/runtime, and version.
+- If context is missing, state assumptions before deeper analysis.
+- Focus only on requested scope; avoid reviewing unrelated files.
+
+### 2) Risk map
+- Prioritize in this order:
+  - Data correctness and user-visible behavior
+  - API/contract compatibility
+  - Security and authz/authn boundaries
+  - Concurrency and lifecycle correctness
+  - Performance and resource usage
+- Give higher priority to stateful paths, migration logic, defaults, and error handling.
+
+### 3) Evidence-based inspection
+- Read each modified hunk with neighboring context.
+- Trace call paths and call-site expectations.
+- Check for:
+  - invariant breaks and missing guards
+  - unchecked assumptions and null/empty/error-path handling
+  - stale tests, fixtures, and configs
+  - hidden coupling to shared helpers/constants/features
+- If a point is uncertain, mark it as an open question instead of guessing.
+
+### 4) Findings-first output
+- Order findings by severity:
+  - P0: critical failure, security breach, or data loss risk
+  - P1: high-impact regression
+  - P2: medium risk correctness gap
+  - P3: low risk/quality debt
+- For each finding include:
+  - Severity
+  - `path:line` reference
+  - concise issue statement
+  - impact and likely failure mode
+  - specific fix or mitigation
+  - validation step to confirm
+- If no issues exist, explicitly state `No findings` and why.
+
+### 5) Close
+- Report assumptions and unknowns.
+- Suggest targeted checks (tests, canary checks, logs/metrics, migration validation).
+
+## Output Template
+
+1. Findings
+2. No findings (if applicable)
+3. Assumptions / Unknowns
+4. Recommended verification steps
+
+## Finding Template
+
+- `[P1] Missing timeout for downstream call`
+  - Location: `path/to/file.rs:123`
+  - Issue: ...
+  - Impact: ...
+  - Fix suggestion: ...
+  - Validation: ...
+

.agents/skills/code-change-verification/agents/openai.yaml

@@ -0,0 +1,4 @@
+interface:
+  display_name: "Code Change Verification"
+  short_description: "Prioritize risks and verify code changes before merge."
+  default_prompt: "Inspect a patch or diff, identify correctness/security/regression risks, and return prioritized findings with file/line evidence and fixes."

.agents/skills/pr-creation-checker/SKILL.md

@@ -0,0 +1,91 @@
+---
+name: pr-creation-checker
+description: Prepare PR-ready diffs by validating scope, checking required verification steps, drafting a compliant English PR title/body, and surfacing blockers before opening or updating a pull request in RustFS.
+---
+
+# PR Creation Checker
+
+Use this skill before `gh pr create`, before `gh pr edit`, or when reviewing whether a branch is ready for PR.
+
+## Read sources of truth first
+
+- Read `AGENTS.md`.
+- Read `.github/pull_request_template.md`.
+- Use `Makefile` and `.config/make/` for local quality commands.
+- Use `.github/workflows/ci.yml` for CI expectations.
+- Do not restate long command matrices or template sections from memory when the files exist.
+
+## Workflow
+
+1. Collect PR context
+- Confirm base branch, current branch, change goal, and scope.
+- Confirm whether the task is: draft a new PR, update an existing PR, or preflight-check readiness.
+- Confirm whether the branch includes only intended changes.
+
+2. Inspect change scope
+- Review the diff and summarize what changed.
+- Call out unrelated edits, generated artifacts, logs, or secrets as blockers.
+- Mark risky areas explicitly: auth, storage, config, network, migrations, breaking changes.
+- Scan the diff for newly added string literals and confirm whether they duplicate values already defined as constants/enums/typed wrappers in the same module or shared modules.
+- Treat introducing a new hardcoded literal where a project constant already exists as a likely regression risk; require either a refactor to reuse the constant or an explicit exception explanation in the PR body.
+
+3. Verify readiness requirements
+- Require `make pre-commit` before marking the PR ready.
+- If `make` is unavailable, use the equivalent commands from `.config/make/`.
+- Add scope-specific verification commands when the changed area needs more than the baseline.
+- If required checks fail, stop and return `BLOCKED`.
+
+4. Draft PR metadata
+- Write the PR title in English using Conventional Commits and keep it within 72 characters.
+- If a generic PR workflow suggests a different title format, ignore it and follow the repository rule instead.
+- In RustFS, do not use tool-specific prefixes such as `[codex]` when the repository requires Conventional Commits.
+- Keep the PR body in English.
+- Use the exact section headings from `.github/pull_request_template.md`.
+- Fill non-applicable sections with `N/A`.
+- Include verification commands in the PR description.
+- Do not include local filesystem paths in the PR body unless the user explicitly asks for them.
+- Prefer repo-relative paths, command names, and concise summaries over machine-specific paths such as `/Users/...`.
+
+5. Prepare reviewer context
+- Summarize why the change exists.
+- Summarize what was verified.
+- Call out risks, rollout notes, config impact, and rollback notes when applicable.
+- Mention assumptions or missing context instead of guessing.
+
+6. Prepare CLI-safe output
+- When proposing `gh pr create` or `gh pr edit`, use `--body-file`, never inline `--body` for multiline markdown.
+- Return a ready-to-save PR body plus a short title.
+- If not ready, return blockers first and list the minimum steps needed to unblock.
+
+## Output format
+
+### Status
+- `READY` or `BLOCKED`
+
+### Title
+- `<type>(<scope>): <summary>`
+
+### PR Body
+- Reproduce the repository template headings exactly.
+- Fill every section.
+- Omit local absolute paths unless explicitly required.
+
+### Verification
+- List each command run.
+- State pass/fail.
+
+### Risks
+- List breaking changes, config changes, migration impact, or `N/A`.
+
+## Blocker rules
+
+- Return `BLOCKED` if `make pre-commit` has not passed.
+- Return `BLOCKED` if the diff contains unrelated changes that are not acknowledged.
+- Return `BLOCKED` if required template sections are missing.
+- Return `BLOCKED` if the title/body is not in English.
+- Return `BLOCKED` if the title does not follow the repository's Conventional Commit rule.
+- Return `BLOCKED` if the diff introduces string literals that should use existing constants but did not.
+
+## Reference
+
+- Use [pr-readiness-checklist.md](references/pr-readiness-checklist.md) for a short final pass before opening or editing the PR.

.agents/skills/pr-creation-checker/agents/openai.yaml

@@ -0,0 +1,4 @@
+interface:
+  display_name: "PR Creation Checker"
+  short_description: "Draft RustFS-ready PRs with checks, template, and blockers."
+  default_prompt: "Inspect a branch or diff, verify required PR checks, and produce a compliant English PR title/body plus blockers or readiness status."

.agents/skills/pr-creation-checker/references/pr-readiness-checklist.md

@@ -0,0 +1,15 @@
+# PR Readiness Checklist
+
+- Confirm the branch is based on current `main`.
+- Confirm the diff matches the stated scope.
+- Confirm no secrets, logs, temp files, or unrelated refactors are included.
+- Confirm `make pre-commit` passed, or document why it could not run.
+- Confirm extra verification commands are listed for risky changes.
+- Confirm the PR title uses Conventional Commits and stays within 72 characters.
+- Confirm the PR title does not use tool-specific prefixes such as `[codex]`.
+- Confirm the PR body is in English.
+- Confirm the PR body keeps the exact headings from `.github/pull_request_template.md`.
+- Confirm non-applicable sections are filled with `N/A`.
+- Confirm the PR body does not include local absolute paths unless explicitly required.
+- Confirm multiline GitHub CLI commands use `--body-file`.
+- Confirm new hardcoded string literals were not introduced for values already represented by existing constants/enums (including protocol labels, error identifiers, headers, and metric names), or record a justified exception.

.agents/skills/test-coverage-improver/SKILL.md

@@ -0,0 +1,66 @@
+---
+name: test-coverage-improver
+description: Run project coverage checks, rank high-risk gaps, and propose high-impact tests to improve regression confidence for changed and critical code paths before release.
+---
+
+# Test Coverage Improver
+
+Use this skill when you need a prioritized, risk-aware plan to improve tests from coverage results.
+
+## Usage assumptions
+- Focus scope is either changed lines/files, a module, or the whole repository.
+- Coverage artifact must be generated or provided in a supported format.
+- If required context is missing, call out assumptions explicitly before proposing work.
+
+## Workflow
+
+1. Define scope and baseline
+   - Confirm target language, framework, and branch.
+   - Confirm whether the scope is changed files only or full-repo.
+
+2. Produce coverage snapshot
+   - Rust: `cargo llvm-cov` (or `cargo tarpaulin`) with existing repo config.
+   - JavaScript/TypeScript: `npm test -- --coverage` and read `coverage/coverage-final.json`.
+   - Python: `pytest --cov=<pkg> --cov-report=json` and read `coverage.json`.
+   - Collect total, per-file, and changed-line coverage.
+
+3. Rank highest-risk gaps
+   - Prioritize changed code, branch coverage gaps, and low-confidence boundaries.
+   - Apply the risk rubric in [coverage-prioritization.md](references/coverage-prioritization.md).
+   - Keep shortlist to 5–8 gaps.
+   - For each gap, capture: file, lines, uncovered branches, and estimated risk score.
+
+4. Propose high-impact tests
+   - For each shortlisted gap, output:
+     - Intent and expected behavior.
+     - Normal, edge, and failure scenarios.
+     - Assertions and side effects to verify.
+     - Setup needs (fixtures, mocks, integration dependencies).
+     - Estimated effort (`S/M/L`).
+
+5. Close with validation plan
+   - State which gaps remain after proposals.
+   - Provide concrete verification command and acceptance threshold.
+   - List assumptions or blockers (environment, fixtures, flaky dependencies).
+
+## Output template
+
+### Coverage Snapshot
+- total / branch coverage
+- changed-file coverage
+- top missing regions by size
+
+### Top Gaps (ranked)
+- `path:line-range` | risk score | why critical
+
+### Test Proposals
+- `path:line-range`
+  - Test name
+  - scenarios
+  - assertions
+  - effort
+
+### Validation Plan
+- command
+- pass criteria
+- remaining risk

.agents/skills/test-coverage-improver/agents/openai.yaml

@@ -0,0 +1,4 @@
+interface:
+  display_name: "Test Coverage Improver"
+  short_description: "Find top uncovered risk areas and propose high-impact tests."
+  default_prompt: "Run coverage checks, identify largest gaps, and recommend highest-impact test cases to improve risk coverage."

.agents/skills/test-coverage-improver/references/coverage-prioritization.md

@@ -0,0 +1,25 @@
+# Coverage Gap Prioritization Guide
+
+Use this rubric for each uncovered area.
+
+Score = (Criticality × 2) + CoverageDebt + (Volatility × 0.5)
+
+- Criticality:
+  - 5: authz/authn, data-loss, payment/consistency path
+  - 4: state mutation, cache invalidation, scheduling
+  - 3: error handling + fallbacks in user-visible flows
+  - 2: parsing/format conversion paths
+  - 1: logging-only or low-impact utilities
+
+- CoverageDebt:
+  - 0: 0–5 uncovered lines
+  - 1: 6–20 uncovered lines
+  - 2: 21–40 uncovered lines
+  - 3: 41+ uncovered lines
+
+- Volatility:
+  - 1: stable legacy code with few recent edits
+  - 2: changed in last 2 releases
+  - 3: touched in last 30 days or currently in active PR
+
+Sort by score descending, then by business impact.

.cargo/config.toml

@@ -0,0 +1,26 @@
+# Copyright 2024 RustFS Team
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# RustFS Cargo configuration
+
+# Enable tokio_unstable cfg for dial9-tokio-telemetry support
+# This allows dial9 to hook into Tokio's internal runtime events
+[build]
+# Enable Tokio unstable features required by dial9-tokio-telemetry for runtime tracing.
+# See: https://docs.rs/tokio/latest/tokio/#unstable-features
+rustflags = ["--cfg", "tokio_unstable"]
+
+# Enable frame pointers for CPU profiling (Linux only, optional but recommended)
+# Uncomment the following line for better CPU profiling data
+# rustflags = ["--cfg", "tokio_unstable", "-C", "force-frame-pointers=yes"]

.docker/compose/docker-compose.observability.yaml

@@ -104,7 +104,7 @@ services:
     environment:
       - TZ=Asia/Shanghai
     volumes:
-      - ../../.docker/observability/loki-config.yaml:/etc/loki/local-config.yaml:ro
+      - ../../.docker/observability/loki.yaml:/etc/loki/local-config.yaml:ro
       - loki-data:/loki
     ports:
       - "3100:3100"

.docker/observability/docker-compose-example-for-rustfs.yml

@@ -0,0 +1,270 @@
+# Copyright 2024 RustFS Team
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+services:
+  rustfs:
+    security_opt:
+      - "no-new-privileges:true"
+    image: rustfs/rustfs:latest
+    container_name: rustfs-server
+    ports:
+      - "9000:9000" # S3 API port
+      - "9001:9001" # Console port
+    environment:
+      - RUSTFS_VOLUMES=/data/rustfs
+      - RUSTFS_ADDRESS=0.0.0.0:9000
+      - RUSTFS_CONSOLE_ADDRESS=0.0.0.0:9001
+      - RUSTFS_CONSOLE_ENABLE=true
+      - RUSTFS_CORS_ALLOWED_ORIGINS=*
+      - RUSTFS_CONSOLE_CORS_ALLOWED_ORIGINS=*
+      - RUSTFS_ACCESS_KEY=rustfsadmin
+      - RUSTFS_SECRET_KEY=rustfsadmin
+      - RUSTFS_OBS_LOGGER_LEVEL=info
+      - RUSTFS_OBS_ENDPOINT=http://otel-collector:4318
+      - RUSTFS_OBS_PROFILING_ENDPOINT=http://pyroscope:4040
+    volumes:
+      - rustfs-data:/data/rustfs
+    networks:
+      - otel-network
+    restart: unless-stopped
+    healthcheck:
+      test:
+        [
+          "CMD",
+          "sh",
+          "-c",
+          "curl -f http://127.0.0.1:9000/health && curl -f http://127.0.0.1:9001/rustfs/console/health",
+        ]
+      interval: 30s
+      timeout: 10s
+      retries: 3
+      start_period: 40s
+    depends_on:
+      otel-collector:
+        condition: service_started
+
+  rustfs-init:
+    image: alpine
+    container_name: rustfs-init
+    volumes:
+      - rustfs-data:/data
+    networks:
+      - otel-network
+    command: >
+      sh -c "
+        chown -R 10001:10001 /data &&
+        echo 'Volume Permissions fixed' &&
+        exit 0
+      "
+    restart: no
+
+  # --- Tracing ---
+
+  tempo:
+    image: grafana/tempo:latest
+    container_name: tempo
+    command: [ "-config.file=/etc/tempo.yaml" ]
+    volumes:
+      - ./tempo.yaml:/etc/tempo.yaml:ro
+      - tempo-data:/var/tempo
+    ports:
+      - "3200:3200" # tempo
+      - "4317" # otlp grpc
+      - "4318" # otlp http
+    networks:
+      - otel-network
+    restart: unless-stopped
+    depends_on:
+      - redpanda
+    healthcheck:
+      test: [ "CMD", "wget", "--spider", "-q", "http://localhost:3200/ready" ]
+      interval: 10s
+      timeout: 5s
+      retries: 3
+      start_period: 15s
+
+  redpanda:
+    image: redpandadata/redpanda:latest # for tempo ingest
+    container_name: redpanda
+    ports:
+      - "9092:9092"
+    networks:
+      - otel-network
+    restart: unless-stopped
+    command: >
+      redpanda start --overprovisioned
+      --mode=dev-container
+      --kafka-addr=PLAINTEXT://0.0.0.0:9092
+      --advertise-kafka-addr=PLAINTEXT://redpanda:9092
+
+  jaeger:
+    image: jaegertracing/jaeger:latest
+    container_name: jaeger
+    environment:
+      - SPAN_STORAGE_TYPE=badger
+      - BADGER_EPHEMERAL=false
+      - BADGER_DIRECTORY_VALUE=/badger/data
+      - BADGER_DIRECTORY_KEY=/badger/key
+      - COLLECTOR_OTLP_ENABLED=true
+    volumes:
+      - ./jaeger.yaml:/etc/jaeger/config.yml
+      - jaeger-data:/badger
+    ports:
+      - "16686:16686" # Web UI
+      - "14269:14269" # Admin/Metrics
+      - "4317" # otlp grpc
+      - "4318" # otlp http
+    command: [ "--config", "/etc/jaeger/config.yml" ]
+    networks:
+      - otel-network
+    restart: unless-stopped
+    healthcheck:
+      test: [ "CMD", "wget", "--spider", "-q", "http://localhost:14269" ]
+      interval: 10s
+      timeout: 5s
+      retries: 3
+      start_period: 15s
+
+  # --- Metrics ---
+
+  prometheus:
+    image: prom/prometheus:latest
+    container_name: prometheus
+    volumes:
+      - ./prometheus.yml:/etc/prometheus/prometheus.yml:ro
+      - prometheus-data:/prometheus
+    ports:
+      - "9090:9090"
+    command:
+      - "--config.file=/etc/prometheus/prometheus.yml"
+      - "--web.enable-otlp-receiver" # Enable OTLP
+      - "--web.enable-remote-write-receiver" # Enable remote write
+      - "--enable-feature=promql-experimental-functions" # Enable info()
+      - "--storage.tsdb.retention.time=30d"
+    restart: unless-stopped
+    networks:
+      - otel-network
+    healthcheck:
+      test: [ "CMD", "wget", "--spider", "-q", "http://localhost:9090/-/healthy" ]
+      interval: 10s
+      timeout: 5s
+      retries: 3
+
+  # --- Logging ---
+
+  loki:
+    image: grafana/loki:latest
+    container_name: loki
+    volumes:
+      - ./loki.yaml:/etc/loki/loki.yaml:ro
+      - loki-data:/loki
+    ports:
+      - "3100:3100"
+    command: -config.file=/etc/loki/loki.yaml
+    networks:
+      - otel-network
+    restart: unless-stopped
+    healthcheck:
+      test: [ "CMD", "wget", "--spider", "-q", "http://localhost:3100/ready" ]
+      interval: 15s
+      timeout: 10s
+      retries: 5
+      start_period: 60s
+
+  # --- Collection ---
+
+  otel-collector:
+    image: otel/opentelemetry-collector-contrib:latest
+    volumes:
+      - ./otel-collector-config.yaml:/etc/otelcol-contrib/config.yaml:ro
+    ports:
+      - "1888:1888" # pprof
+      - "8888:8888" # Prometheus metrics for Collector
+      - "8889:8889" # Prometheus metrics for application indicators
+      - "13133:13133" # health check
+      - "4317:4317" # OTLP gRPC
+      - "4318:4318" # OTLP HTTP
+      - "55679:55679" # zpages
+    networks:
+      - otel-network
+    restart: unless-stopped
+    depends_on:
+      - tempo
+      - jaeger
+      - prometheus
+      - loki
+    healthcheck:
+      test: [ "CMD", "wget", "--spider", "-q", "http://localhost:13133" ]
+      interval: 10s
+      timeout: 5s
+      retries: 3
+
+  # --- Profiles ---
+
+  pyroscope:
+    image: grafana/pyroscope:latest
+    container_name: pyroscope
+    ports:
+      - "4040:4040"
+    command:
+      - -self-profiling.disable-push=true
+    networks:
+      - otel-network
+    restart: unless-stopped
+
+  # --- Visualization ---
+
+  grafana:
+    image: grafana/grafana:latest
+    container_name: grafana
+    ports:
+      - "3000:3000"
+    environment:
+      - GF_SECURITY_ADMIN_PASSWORD=admin
+      - GF_SECURITY_ADMIN_USER=admin
+    volumes:
+      - ./grafana/provisioning:/etc/grafana/provisioning:ro
+      - ./grafana/dashboards:/etc/grafana/dashboards:ro
+      - grafana-data:/var/lib/grafana
+    networks:
+      - otel-network
+    restart: unless-stopped
+    depends_on:
+      - prometheus
+      - tempo
+      - loki
+    healthcheck:
+      test:
+        [ "CMD", "wget", "--spider", "-q", "http://localhost:3000/api/health" ]
+      interval: 10s
+      timeout: 5s
+      retries: 3
+
+volumes:
+  rustfs-data:
+  tempo-data:
+  jaeger-data:
+  prometheus-data:
+  loki-data:
+  grafana-data:
+
+networks:
+  otel-network:
+    driver: bridge
+    name: "network_otel"
+    ipam:
+      config:
+        - subnet: 172.28.0.0/16
+    driver_opts:
+      com.docker.network.enable_ipv6: "true"

.docker/observability/docker-compose-tempo-ha-override.yml

@@ -0,0 +1,62 @@
+# Copyright 2024 RustFS Team
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# Docker Compose override file for High Availability Tempo setup
+#
+# Usage:
+# docker-compose -f docker-compose-example-for-rustfs.yml \
+#                -f docker-compose-tempo-ha-override.yml up
+
+services:
+  # Override Tempo to use high-availability configuration
+  tempo:
+    volumes:
+      - ./tempo-ha.yaml:/etc/tempo.yaml:ro
+      - tempo-data:/var/tempo
+    ports:
+      - "3200:3200"   # Tempo HTTP
+      - "4317:4317"   # OTLP gRPC
+      - "4318:4318"   # OTLP HTTP
+      - "7946:7946"   # Memberlist
+      - "14250:14250" # Jaeger gRPC
+      - "14268:14268" # Jaeger Thrift HTTP
+      - "9411:9411"   # Zipkin
+    environment:
+      - TEMPO_MEMBERLIST_BIND_PORT=7946
+    healthcheck:
+      test: [ "CMD", "wget", "--spider", "-q", "http://localhost:3200/ready" ]
+      interval: 10s
+      timeout: 5s
+      retries: 5
+      start_period: 30s
+    depends_on:
+      - redpanda
+
+volumes:
+  tempo-data:
+    driver: local
+    driver_opts:
+      type: tmpfs
+      device: tmpfs
+      o: "size=4g"  # Allocate 4GB tmpfs for Tempo data (adjust based on your needs)
+
+# Network configuration remains the same
+# networks:
+#   otel-network:
+#     driver: bridge
+#     name: "network_otel"
+#     ipam:
+#       config:
+#         - subnet: 172.28.0.0/16
+

.docker/observability/docker-compose.yml

@@ -13,100 +13,64 @@
 # limitations under the License.
 
 services:
-  rustfs:
-    security_opt:
-      - "no-new-privileges:true"
-    image: rustfs/rustfs:latest
-    container_name: rustfs-server
-    ports:
-      - "9000:9000" # S3 API port
-      - "9001:9001" # Console port
-    environment:
-      - RUSTFS_VOLUMES=/data/rustfs
-      - RUSTFS_ADDRESS=0.0.0.0:9000
-      - RUSTFS_CONSOLE_ADDRESS=0.0.0.0:9001
-      - RUSTFS_CONSOLE_ENABLE=true
-      - RUSTFS_CORS_ALLOWED_ORIGINS=*
-      - RUSTFS_CONSOLE_CORS_ALLOWED_ORIGINS=*
-      - RUSTFS_ACCESS_KEY=rustfsadmin
-      - RUSTFS_SECRET_KEY=rustfsadmin
-      - RUSTFS_OBS_LOGGER_LEVEL=info
-      - RUSTFS_OBS_ENDPOINT=http://otel-collector:4318
-    volumes:
-      - rustfs-data:/data/rustfs
-    networks:
-      - otel-network
-    restart: unless-stopped
-    healthcheck:
-      test:
-        [
-          "CMD",
-          "sh",
-          "-c",
-          "curl -f http://127.0.0.1:9000/health && curl -f http://127.0.0.1:9001/rustfs/console/health",
-        ]
-      interval: 30s
-      timeout: 10s
-      retries: 3
-      start_period: 40s
-    depends_on:
-      otel-collector:
-        condition: service_started
-
-  rustfs-init:
-    image: alpine
-    container_name: rustfs-init
-    volumes:
-      - rustfs-data:/data
-    networks:
-      - otel-network
-    command: >
-      sh -c "
-        chown -R 10001:10001 /data &&
-        echo 'Volume Permissions fixed' &&
-        exit 0
-      "
-    restart: no
 
   # --- Tracing ---
-
   tempo:
-    image: grafana/tempo:latest
+    image: grafana/tempo:2.10.3
     container_name: tempo
-    command: ["-config.file=/etc/tempo.yaml"]
+    depends_on:
+      - redpanda
+    command: [ "-config.file=/etc/tempo.yaml" ]
     volumes:
       - ./tempo.yaml:/etc/tempo.yaml:ro
       - tempo-data:/var/tempo
     ports:
       - "3200:3200" # tempo
-      - "4317"      # otlp grpc
-      - "4318"      # otlp http
+      - "4317" # otlp grpc
+      - "4318" # otlp http
+      - "7946" # memberlist
     networks:
       - otel-network
     restart: unless-stopped
-    depends_on:
-      - redpanda
     healthcheck:
-      test: ["CMD", "wget", "--spider", "-q", "http://localhost:3200/ready"]
+      test: [ "CMD", "wget", "--spider", "-q", "http://localhost:3200/ready" ]
       interval: 10s
       timeout: 5s
       retries: 3
       start_period: 15s
-
   redpanda:
-    image: redpandadata/redpanda:latest # for tempo ingest
-    container_name: redpanda
+    image: redpandadata/redpanda:latest
     ports:
-      - "9092:9092"
-    networks:
-      - otel-network
-    restart: unless-stopped
+      - "9092:9092" # Kafka API for clients
     command: >
       redpanda start --overprovisioned
       --mode=dev-container
       --kafka-addr=PLAINTEXT://0.0.0.0:9092
       --advertise-kafka-addr=PLAINTEXT://redpanda:9092
 
+  redpanda-console:
+    image: docker.redpanda.com/redpandadata/console:latest
+    environment:
+      - CONFIG_FILEPATH=/etc/redpanda/redpanda-console-config.yaml
+    volumes:
+      - ./redpanda-console.yaml:/etc/redpanda/redpanda-console-config.yaml
+    ports:
+      - "8080:8080"
+    depends_on:
+      - redpanda
+
+  vulture:
+    image: grafana/tempo-vulture:latest
+    restart: always
+    command:
+      [
+        "-prometheus-listen-address=:8080",
+        "-tempo-query-url=http://tempo:3200",
+        "-tempo-push-url=http://tempo:4317",
+      ]
+    depends_on:
+      - tempo
+
   jaeger:
     image: jaegertracing/jaeger:latest
     container_name: jaeger
@@ -124,12 +88,12 @@ services:
       - "14269:14269" # Admin/Metrics
       - "4317" # otlp grpc
       - "4318" # otlp http
-    command: ["--config", "/etc/jaeger/config.yml"]
+    command: [ "--config", "/etc/jaeger/config.yml" ]
     networks:
       - otel-network
     restart: unless-stopped
     healthcheck:
-      test: ["CMD", "wget", "--spider", "-q", "http://localhost:14269"]
+      test: [ "CMD", "wget", "--spider", "-q", "http://localhost:14269" ]
       interval: 10s
       timeout: 5s
       retries: 3
@@ -155,7 +119,7 @@ services:
     networks:
       - otel-network
     healthcheck:
-      test: ["CMD", "wget", "--spider", "-q", "http://localhost:9090/-/healthy"]
+      test: [ "CMD", "wget", "--spider", "-q", "http://localhost:9090/-/healthy" ]
       interval: 10s
       timeout: 5s
       retries: 3
@@ -175,7 +139,7 @@ services:
       - otel-network
     restart: unless-stopped
     healthcheck:
-      test: ["CMD", "wget", "--spider", "-q", "http://localhost:3100/ready"]
+      test: [ "CMD", "wget", "--spider", "-q", "http://localhost:3100/ready" ]
       interval: 15s
       timeout: 10s
       retries: 5
@@ -188,12 +152,12 @@ services:
     volumes:
       - ./otel-collector-config.yaml:/etc/otelcol-contrib/config.yaml:ro
     ports:
-      - "1888:1888"   # pprof
-      - "8888:8888"   # Prometheus metrics for Collector
-      - "8889:8889"   # Prometheus metrics for application indicators
+      - "1888:1888" # pprof
+      - "8888:8888" # Prometheus metrics for Collector
+      - "8889:8889" # Prometheus metrics for application indicators
       - "13133:13133" # health check
-      - "4317:4317"   # OTLP gRPC
-      - "4318:4318"   # OTLP HTTP
+      - "4317:4317" # OTLP gRPC
+      - "4318:4318" # OTLP HTTP
       - "55679:55679" # zpages
     networks:
       - otel-network
@@ -204,11 +168,24 @@ services:
       - prometheus
       - loki
     healthcheck:
-      test: ["CMD", "wget", "--spider", "-q", "http://localhost:13133"]
+      test: [ "CMD", "wget", "--spider", "-q", "http://localhost:13133" ]
       interval: 10s
       timeout: 5s
       retries: 3
 
+  # --- Profiles ---
+
+  pyroscope:
+    image: grafana/pyroscope:latest
+    container_name: pyroscope
+    ports:
+      - "4040:4040"
+    command:
+      - -self-profiling.disable-push=true
+    networks:
+      - otel-network
+    restart: unless-stopped
+
   # --- Visualization ---
 
   grafana:
@@ -231,13 +208,13 @@ services:
       - tempo
       - loki
     healthcheck:
-      test: ["CMD", "wget", "--spider", "-q", "http://localhost:3000/api/health"]
+      test:
+        [ "CMD", "wget", "--spider", "-q", "http://localhost:3000/api/health" ]
       interval: 10s
       timeout: 5s
       retries: 3
 
 volumes:
-  rustfs-data:
   tempo-data:
   jaeger-data:
   prometheus-data:

.docker/observability/grafana/provisioning/dashboards/dashboard.yml

@@ -1,3 +1,17 @@
+# Copyright 2024 RustFS Team
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
 apiVersion: 1
 
 providers:

.docker/observability/grafana/provisioning/datasources/datasources.yaml

@@ -1,3 +1,17 @@
+# Copyright 2024 RustFS Team
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
 apiVersion: 1
 
 datasources:
@@ -76,3 +90,9 @@ datasources:
         spanEndTimeShift: '-1s'
         filterByTraceID: true
         filterBySpanID: false
+
+  - name: Pyroscope
+    type: grafana-pyroscope-datasource
+    url: http://pyroscope:4040
+    jsonData:
+      minStep: '15s'

.docker/observability/prometheus.yml

@@ -50,6 +50,11 @@ scrape_configs:
     static_configs:
       - targets: [ 'localhost:9090' ]
 
+  - job_name: 'vulture'
+    static_configs:
+      - targets:
+          - 'vulture:8080'
+
 otlp:
   promote_resource_attributes:
     - service.instance.id

.docker/observability/redpanda-console.yaml

@@ -0,0 +1,3 @@
+kafka:
+  brokers:
+    - redpanda:9092

.docker/observability/tempo-ha.yaml

@@ -0,0 +1,286 @@
+# Copyright 2024 RustFS Team
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# High Availability Tempo Configuration for docker-compose-example-for-rustfs.yml
+# Features:
+# - Distributed architecture with multiple components
+# - Kafka-based ingestion for fault tolerance
+# - Replication factor of 3 for data resilience
+# - Query frontend for load balancing
+# - Metrics generation from traces
+# - WAL for durability
+
+partition_ring_live_store: true
+stream_over_http_enabled: true
+
+server:
+  http_listen_port: 3200
+  http_server_read_timeout: 30s
+  http_server_write_timeout: 30s
+  grpc_server_max_recv_msg_size: 4194304  # 4MB
+  grpc_server_max_send_msg_size: 4194304
+  log_level: info
+  log_format: json
+
+# Memberlist configuration for distributed mode
+memberlist:
+  node_name: tempo
+  bind_port: 7946
+  join_members:
+    - tempo:7946
+  retransmit_factor: 4
+  node_timeout: 15s
+  retransmit_interval: 300ms
+  dead_node_reclaim_time: 30s
+
+# Distributor configuration - receives traces and routes to ingesters
+distributor:
+  ingester_write_path_enabled: true
+  kafka_write_path_enabled: true
+  rate_limit_bytes: 10MB
+  rate_limit_enabled: true
+  receivers:
+    otlp:
+      protocols:
+        grpc:
+          endpoint: "0.0.0.0:4317"
+          max_concurrent_streams: 0
+          max_receive_message_size: 4194304
+        http:
+          endpoint: "0.0.0.0:4318"
+          cors:
+            allowed_origins:
+              - "*"
+            max_age: 86400
+    jaeger:
+      protocols:
+        grpc:
+          endpoint: "0.0.0.0:14250"
+        thrift_http:
+          endpoint: "0.0.0.0:14268"
+    zipkin:
+      endpoint: "0.0.0.0:9411"
+  ring:
+    kvstore:
+      store: memberlist
+    heartbeat_timeout: 5s
+    replication_factor: 3
+    heartbeat_interval: 5s
+
+# Ingester configuration - stores traces and querying
+ingester:
+  lifecycler:
+    address: tempo
+    ring:
+      kvstore:
+        store: memberlist
+      replication_factor: 3
+      max_cache_freshness_per_sec: 10s
+      heartbeat_interval: 5s
+      heartbeat_timeout: 5s
+    num_tokens: 128
+    tokens_file_path: /var/tempo/tokens.json
+    claim_on_rollout: true
+  trace_idle_period: 20s
+  max_block_bytes: 10_000_000
+  max_block_duration: 10m
+  chunk_size_bytes: 1_000_000
+  chunk_encoding: snappy
+  wal:
+    checkpoint_duration: 5s
+    max_wal_blocks: 4
+  metrics:
+    enabled: true
+    level: block
+    target_info_duration: 15m
+
+# WAL configuration for data durability
+wal:
+  checkpoint_duration: 5s
+  flush_on_shutdown: true
+  path: /var/tempo/wal
+
+# Kafka ingestion configuration - for high throughput scenarios
+ingest:
+  enabled: true
+  kafka:
+    brokers: [ redpanda:9092 ]
+    topic: tempo-ingest
+    encoding: protobuf
+    consumer_group: tempo-ingest-consumer
+    session_timeout: 10s
+    rebalance_timeout: 1m
+    partition: auto
+    verbosity: 2
+
+# Query frontend configuration - distributed querying
+query_frontend:
+  compression: gzip
+  downstream_url: http://localhost:3200
+  log_queries_longer_than: 5s
+  cache_uncompressed_bytes: 100MB
+  max_outstanding_requests_per_tenant: 100
+  max_query_length: 48h
+  max_query_lookback: 30d
+  default_result_cache_ttl: 1m
+  result_cache:
+    cache:
+      enable_fifocache: true
+      default_validity: 1m
+  rf1_after: "1999-01-01T00:00:00Z"
+  mcp_server:
+    enabled: true
+
+# Querier configuration - queries traces
+querier:
+  frontend_worker:
+    frontend_address: localhost:3200
+    grpc_client_config:
+      max_recv_msg_size: 104857600
+  max_concurrent_queries: 20
+  max_metric_bytes_per_trace: 1MB
+
+# Query scheduler configuration - for distributed querying
+query_scheduler:
+  use_scheduler_ring: false
+
+# Metrics generator configuration - generates metrics from traces
+metrics_generator:
+  enabled: true
+  registry:
+    enabled: true
+    external_labels:
+      source: tempo
+      cluster: rustfs-docker-ha
+      environment: production
+  storage:
+    path: /var/tempo/generator/wal
+    remote_write:
+      - url: http://prometheus:9090/api/v1/write
+        send_exemplars: true
+        resource_to_telemetry_conversion:
+          enabled: true
+  processor:
+    batch:
+      timeout: 10s
+      send_batch_size: 1024
+    memory_limiter:
+      check_interval: 5s
+      limit_mib: 512
+      spike_limit_mib: 128
+  processors:
+    - span-metrics
+    - local-blocks
+    - service-graphs
+  generate_native_histograms: both
+
+# Backend worker configuration
+backend_worker:
+  backend_scheduler_addr: localhost:3200
+  compaction:
+    block_retention: 24h
+    compacted_block_retention: 1h
+  ring:
+    kvstore:
+      store: memberlist
+
+# Backend scheduler configuration
+backend_scheduler:
+  enabled: true
+  provider:
+    compaction:
+      compaction:
+        block_retention: 24h
+        compacted_block_retention: 1h
+        concurrency: 25
+        v2_out_path: /var/tempo/blocks/compaction
+
+# Storage configuration - local backend with proper retention
+storage:
+  trace:
+    backend: local
+    wal:
+      path: /var/tempo/wal
+      checkpoint_duration: 5s
+      flush_on_shutdown: true
+    local:
+      path: /var/tempo/blocks
+      bloom_filter_false_positive: 0.05
+      bloom_shift: 4
+      index:
+        downsample_bytes: 1000000
+        page_size_bytes: 0
+        cache_size_bytes: 0
+    pool:
+      max_workers: 400
+      queue_depth: 10000
+
+# Compactor configuration - manages block compaction
+compactor:
+  compaction:
+    block_retention: 168h  # 7 days
+    compacted_block_retention: 1h
+    concurrency: 25
+    v2_out_path: /var/tempo/blocks/compaction
+    shard_count: 32
+    max_block_bytes: 107374182400  # 100GB
+    max_compaction_objects: 6000000
+    max_time_per_tenant: 5m
+    block_size_bytes: 107374182400
+  ring:
+    kvstore:
+      store: memberlist
+    heartbeat_interval: 5s
+    heartbeat_timeout: 5s
+
+# Limits configuration - rate limiting and quotas
+limits:
+  max_traces_per_user: 10000
+  max_bytes_per_trace: 10485760  # 10MB
+  max_search_bytes_per_trace: 0
+  forgiving_oversize_traces: true
+  rate_limit_bytes: 10MB
+  rate_limit_enabled: true
+  ingestion_burst_size_bytes: 20MB
+  ingestion_rate_limit_bytes: 10MB
+  max_bytes_per_second: 10485760
+  metrics_generator_max_active_series: 10000
+  metrics_generator_max_churned_series: 10000
+  metrics_generator_forta_out_of_order_ttl: 5m
+
+# Override configuration
+overrides:
+  defaults:
+    metrics_generator:
+      processors:
+        - span-metrics
+        - local-blocks
+        - service-graphs
+      generate_native_histograms: both
+      max_active_series: 10000
+      max_churned_series: 10000
+
+# Usage reporting configuration
+usage_report:
+  reporting_enabled: false
+
+# Tracing configuration for debugging
+tracing:
+  enabled: true
+  jaeger:
+    sampler:
+      name: probabilistic
+      param: 0.1
+    reporter_log_spans: false
+

.docker/observability/tempo.yaml

@@ -19,16 +19,20 @@ server:
   http_listen_port: 3200
   log_level: info
 
+memberlist:
+  node_name: tempo
+  bind_port: 7946
+  join_members:
+    - tempo:7946
+
 distributor:
-  ingester_write_path_enabled: false
-  kafka_write_path_enabled: true
   receivers:
     otlp:
       protocols:
         grpc:
-          endpoint: "tempo:4317"
+          endpoint: "0.0.0.0:4317"
         http:
-          endpoint: "tempo:4318"
+          endpoint: "0.0.0.0:4318"
   #log_received_spans:
   #    enabled: true
   #  log_discarded_spans:
@@ -71,14 +75,14 @@ storage:
   trace:
     backend: local
     wal:
-      path: /var/tempo/wal
+      path: /var/tempo/wal # where to store the wal locally
     local:
-      path: /var/tempo/blocks
+      path: /var/tempo/blocks # where to store the traces locally
 
 overrides:
   defaults:
     metrics_generator:
-      processors: ["span-metrics", "service-graphs", "local-blocks"]
+      processors: [ "span-metrics", "service-graphs", "local-blocks" ]
       generate_native_histograms: both
 
 ingest:
@@ -92,3 +96,4 @@ block_builder:
 
 usage_report:
   reporting_enabled: false
+

.github/AGENTS.md

@@ -0,0 +1,30 @@
+# GitHub Workflow Instructions
+
+Applies to `.github/` and repository pull-request operations.
+
+## Pull Requests
+
+- PR titles and descriptions must be in English.
+- Use `.github/pull_request_template.md` for every PR body.
+- Keep all template section headings.
+- Use `N/A` for non-applicable sections.
+- Include verification commands in the PR details.
+- For `gh pr create` and `gh pr edit`, always write markdown body to a file and pass `--body-file`.
+- Do not use multiline inline `--body`; backticks and shell expansion can corrupt content or trigger unintended commands.
+- Recommended pattern:
+  - `cat > /tmp/pr_body.md <<'EOF'`
+  - `...markdown...`
+  - `EOF`
+  - `gh pr create ... --body-file /tmp/pr_body.md`
+
+## CI Alignment
+
+When changing CI-sensitive behavior, keep local validation aligned with `.github/workflows/ci.yml`.
+
+Current `test-and-lint` gate includes:
+
+- `cargo nextest run --all --exclude e2e_test`
+- `cargo test --all --doc`
+- `cargo fmt --all --check`
+- `cargo clippy --all-targets --all-features -- -D warnings`
+- `./scripts/check_layer_dependencies.sh`

.github/cla.yml

@@ -0,0 +1,17 @@
+enabled: true
+
+document:
+  version: v1
+  url: https://github.com/rustfs/cla/blob/main/cla/v1.md
+
+signing:
+  mode: comment
+  comment_pattern: I have read and agree to the CLA.
+
+registry:
+  type: json-repo
+  repository: rustfs/cla
+  path_prefix: signatures
+
+status:
+  check_name: CLA Check

.github/dependabot.yml

@@ -26,11 +26,19 @@ updates:
       day: "monday"
       timezone: "Asia/Shanghai"
       time: "08:00"
+    assignees:
+      - "heihutu"
+    reviewers:
+      - "houseme"
+      - "overtrue"
+      - "majinghe"
     ignore:
       - dependency-name: "object_store"
         versions: [ "0.13.x" ]
       - dependency-name: "libunftp"
         versions: [ "0.23.x" ]
+      - dependency-name: "ratelimit"
+        versions: [ "1.x" ]
     groups:
       s3s:
         update-types:

.github/pull_request_template.md

@@ -34,4 +34,4 @@ Pull Request Template for RustFS
 
 ---
 
-Thank you for your contribution! Please ensure your PR follows the community standards ([CODE_OF_CONDUCT.md](CODE_OF_CONDUCT.md)) and sign the CLA if this is your first contribution.
+Thank you for your contribution! Please ensure your PR follows the community standards ([CODE_OF_CONDUCT.md](CODE_OF_CONDUCT.md)). If this is your first contribution, review the [CLA document](https://github.com/rustfs/cla/blob/main/cla/v1.md) and sign it by commenting `I have read and agree to the CLA.` on the PR.

.github/workflows/build.yml

@@ -23,6 +23,7 @@
 #
 # Manual Parameters:
 # - build_docker: Build and push Docker images (default: true)
+# - platforms: Comma-separated platform IDs or 'all' (default: all)
 
 name: Build and Release
 
@@ -44,22 +45,6 @@ on:
       - "**/*.svg"
       - ".gitignore"
       - ".dockerignore"
-  pull_request:
-    branches: [ main ]
-    paths-ignore:
-      - "**.md"
-      - "**.txt"
-      - ".github/**"
-      - "docs/**"
-      - "deploy/**"
-      - "scripts/dev_*.sh"
-      - "LICENSE*"
-      - "README*"
-      - "**/*.png"
-      - "**/*.jpg"
-      - "**/*.svg"
-      - ".gitignore"
-      - ".dockerignore"
   schedule:
     - cron: "0 0 * * 0" # Weekly on Sunday at midnight UTC
   workflow_dispatch:
@@ -69,6 +54,11 @@ on:
         required: false
         default: true
         type: boolean
+      platforms:
+        description: "Comma-separated targets or 'all' (e.g. linux-x86_64-musl,macos-aarch64)"
+        required: false
+        default: "all"
+        type: string
 
 permissions:
   contents: read
@@ -154,63 +144,74 @@ jobs:
           echo "  - Is prerelease: $is_prerelease"
 
   # Build RustFS binaries
+  prepare-platform-matrix:
+    name: Prepare Platform Matrix
+    runs-on: ubicloud-standard-2
+    outputs:
+      matrix: ${{ steps.select.outputs.matrix }}
+      selected: ${{ steps.select.outputs.selected }}
+    steps:
+      - name: Select target platforms
+        id: select
+        shell: bash
+        run: |
+          set -euo pipefail
+
+          selected="${{ github.event_name == 'workflow_dispatch' && github.event.inputs.platforms || 'all' }}"
+          selected="$(echo "${selected}" | tr -d '[:space:]')"
+          if [[ -z "${selected}" ]]; then
+            selected="all"
+          fi
+
+          all='{"include":[
+            {"target_id":"linux-x86_64-musl","os":"ubicloud-standard-2","target":"x86_64-unknown-linux-musl","cross":false,"platform":"linux","rustflags":""},
+            {"target_id":"linux-aarch64-musl","os":"ubicloud-standard-2","target":"aarch64-unknown-linux-musl","cross":true,"platform":"linux","rustflags":""},
+            {"target_id":"linux-x86_64-gnu","os":"ubicloud-standard-2","target":"x86_64-unknown-linux-gnu","cross":false,"platform":"linux","rustflags":""},
+            {"target_id":"linux-aarch64-gnu","os":"ubicloud-standard-2","target":"aarch64-unknown-linux-gnu","cross":true,"platform":"linux","rustflags":""},
+            {"target_id":"macos-aarch64","os":"macos-latest","target":"aarch64-apple-darwin","cross":false,"platform":"macos","rustflags":""},
+            {"target_id":"macos-x86_64","os":"macos-latest","target":"x86_64-apple-darwin","cross":false,"platform":"macos","rustflags":""},
+            {"target_id":"windows-x86_64","os":"windows-latest","target":"x86_64-pc-windows-msvc","cross":false,"platform":"windows","rustflags":""}
+          ]}'
+
+          if [[ "${selected}" == "all" ]]; then
+            matrix="$(jq -c . <<<"${all}")"
+          else
+            unknown="$(jq -rn --arg selected "${selected}" --argjson all "${all}" '
+              ($selected | split(",") | map(select(length > 0))) as $req
+              | ($all.include | map(.target_id)) as $known
+              | [$req[] | select(( $known | index(.) ) == null)]
+            ')"
+            if [[ "$(jq 'length' <<<"${unknown}")" -gt 0 ]]; then
+              echo "Unknown platforms: $(jq -r 'join(\",\")' <<<"${unknown}")" >&2
+              echo "Allowed: $(jq -r '.include[].target_id' <<<"${all}" | paste -sd ',' -)" >&2
+              exit 1
+            fi
+
+            matrix="$(jq -c --arg selected "${selected}" '
+              ($selected | split(",") | map(select(length > 0))) as $req
+              | .include |= map(select(.target_id as $id | ($req | index($id))))
+            ' <<<"${all}")"
+          fi
+
+          echo "selected=${selected}" >> "$GITHUB_OUTPUT"
+          echo "matrix=${matrix}" >> "$GITHUB_OUTPUT"
+          echo "Selected platforms: ${selected}"
+
   build-rustfs:
     name: Build RustFS
-    needs: [ build-check ]
-    if: needs.build-check.outputs.should_build == 'true'
+    needs: [ build-check, prepare-platform-matrix ]
+    if: needs.build-check.outputs.should_build == 'true' && needs.prepare-platform-matrix.result == 'success'
     runs-on: ${{ matrix.os }}
     timeout-minutes: 60
     env:
-      RUSTFLAGS: ${{ matrix.rustflags }}
+      # Always enable Tokio unstable features (required by dial9-tokio-telemetry).
+      # The RUSTFLAGS env var takes precedence over .cargo/config.toml [build] rustflags,
+      # so we must include --cfg tokio_unstable here explicitly; otherwise an empty
+      # RUSTFLAGS value would shadow the config-file flag and silently break tracing.
+      RUSTFLAGS: "--cfg tokio_unstable ${{ matrix.rustflags }}"
     strategy:
       fail-fast: false
-      matrix:
-        include:
-          # Linux builds
-          # Use x86-64-v2 (SSE4.2 baseline) instead of native to ensure distributed
-          # binaries run on older x86_64 CPUs (e.g. Intel Celeron/Atom, Synology NAS).
-          # See: https://github.com/rustfs/rustfs/issues/1838
-          - os: ubicloud-standard-2
-            target: x86_64-unknown-linux-musl
-            cross: false
-            platform: linux
-            rustflags: '-C target-cpu=x86-64-v2'
-          - os: ubicloud-standard-2
-            target: aarch64-unknown-linux-musl
-            cross: true
-            platform: linux
-            rustflags: ''
-          - os: ubicloud-standard-2
-            target: x86_64-unknown-linux-gnu
-            cross: false
-            platform: linux
-            rustflags: '-C target-cpu=x86-64-v2'
-          - os: ubicloud-standard-2
-            target: aarch64-unknown-linux-gnu
-            cross: true
-            platform: linux
-            rustflags: ''
-          # macOS builds
-          - os: macos-latest
-            target: aarch64-apple-darwin
-            cross: false
-            platform: macos
-            rustflags: ''
-          - os: macos-latest
-            target: x86_64-apple-darwin
-            cross: false
-            platform: macos
-            rustflags: '-C target-cpu=x86-64-v2'
-          # Windows builds (temporarily disabled)
-          - os: windows-latest
-            target: x86_64-pc-windows-msvc
-            cross: false
-            platform: windows
-            rustflags: '-C target-cpu=x86-64-v2'
-          #- os: windows-latest
-          #  target: aarch64-pc-windows-msvc
-          #  cross: true
-          #  platform: windows
+      matrix: ${{ fromJson(needs.prepare-platform-matrix.outputs.matrix) }}
     steps:
       - name: Checkout repository
         uses: actions/checkout@v6

.github/workflows/ci.yml

@@ -91,6 +91,8 @@ jobs:
 
   typos:
     name: Typos
+    needs: skip-check
+    if: needs.skip-check.outputs.should_skip != 'true'
     runs-on: ubicloud-standard-2
     steps:
       - uses: actions/checkout@v6
@@ -116,9 +118,6 @@ jobs:
           github-token: ${{ secrets.GITHUB_TOKEN }}
           cache-save-if: ${{ github.ref == 'refs/heads/main' }}
 
-      - name: Install cargo-nextest
-        uses: taiki-e/install-action@nextest
-
       - name: Run tests
         run: |
           cargo nextest run --all --exclude e2e_test
@@ -133,9 +132,40 @@ jobs:
       - name: Check layered dependencies
         run: ./scripts/check_layer_dependencies.sh
 
+  build-rustfs-debug-binary:
+    name: Build RustFS Debug Binary
+    needs: skip-check
+    if: needs.skip-check.outputs.should_skip != 'true'
+    runs-on: ubicloud-standard-4
+    timeout-minutes: 30
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v6
+
+      - name: Setup Rust environment
+        uses: ./.github/actions/setup
+        with:
+          rust-version: stable
+          cache-shared-key: ci-rustfs-debug-binary-${{ hashFiles('**/Cargo.lock') }}
+          cache-save-if: ${{ github.ref == 'refs/heads/main' }}
+          github-token: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Build debug binary
+        run: |
+          touch rustfs/build.rs
+          cargo build -p rustfs --bins --jobs 2
+
+      - name: Upload debug binary
+        uses: actions/upload-artifact@v6
+        with:
+          name: rustfs-debug-binary
+          path: target/debug/rustfs
+          if-no-files-found: error
+          retention-days: 1
+
   e2e-tests:
     name: End-to-End Tests
-    needs: skip-check
+    needs: [ skip-check, build-rustfs-debug-binary ]
     if: needs.skip-check.outputs.should_skip != 'true'
     runs-on: ubicloud-standard-2
     timeout-minutes: 30
@@ -148,13 +178,17 @@ jobs:
           rm -rf /tmp/rustfs
           rm -f /tmp/rustfs.log
 
-      - name: Setup Rust environment
-        uses: ./.github/actions/setup
+      - name: Download debug binary
+        uses: actions/download-artifact@v7
         with:
-          rust-version: stable
-          cache-shared-key: ci-e2e-${{ hashFiles('**/Cargo.lock') }}
-          cache-save-if: ${{ github.ref == 'refs/heads/main' }}
-          github-token: ${{ secrets.GITHUB_TOKEN }}
+          name: rustfs-debug-binary
+          path: target/debug
+
+      - name: Make binary executable
+        run: chmod +x ./target/debug/rustfs
+
+      - name: Setup Rust toolchain for s3s-e2e installation
+        uses: dtolnay/rust-toolchain@stable
 
       - name: Install s3s-e2e test tool
         uses: taiki-e/cache-cargo-install-action@v2
@@ -163,12 +197,6 @@ jobs:
           git: https://github.com/s3s-project/s3s.git
           rev: 4a04a670cf41274d9be9ab65dc36f4aa3f92fbad
 
-      - name: Build debug binary
-        run: |
-          touch rustfs/build.rs
-          # Limit concurrency to prevent OOM
-          cargo build -p rustfs --bins --jobs 2
-
       - name: Run end-to-end tests
         run: |
           s3s-e2e --version
@@ -184,7 +212,7 @@ jobs:
 
   s3-implemented-tests:
     name: S3 Implemented Tests
-    needs: skip-check
+    needs: [ skip-check, build-rustfs-debug-binary ]
     if: needs.skip-check.outputs.should_skip != 'true'
     runs-on: ubicloud-standard-4
     timeout-minutes: 60
@@ -192,18 +220,14 @@ jobs:
       - name: Checkout repository
         uses: actions/checkout@v6
 
-      - name: Setup Rust environment
-        uses: ./.github/actions/setup
+      - name: Download debug binary
+        uses: actions/download-artifact@v7
         with:
-          rust-version: stable
-          cache-shared-key: ci-s3tests-${{ hashFiles('**/Cargo.lock') }}
-          cache-save-if: ${{ github.ref == 'refs/heads/main' }}
-          github-token: ${{ secrets.GITHUB_TOKEN }}
+          name: rustfs-debug-binary
+          path: target/debug
 
-      - name: Build debug binary
-        run: |
-          touch rustfs/build.rs
-          cargo build -p rustfs --bins --jobs 2
+      - name: Make binary executable
+        run: chmod +x ./target/debug/rustfs
 
       - name: Run implemented s3-tests
         run: |

.github/workflows/cla.yml

@@ -0,0 +1,48 @@
+# Copyright 2026 RustFS Team
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+name: CLA Check
+
+on:
+  pull_request_target:
+    types: [opened, synchronize, reopened]
+  issue_comment:
+    types: [created, edited]
+
+permissions:
+  contents: write
+  pull-requests: write
+  issues: write
+  checks: write
+
+jobs:
+  cla:
+    if: ${{ github.event_name != 'issue_comment' || github.event.issue.pull_request }}
+    runs-on: ubuntu-latest
+    steps:
+      - name: Create token for rustfs/cla
+        id: registry-token
+        uses: actions/create-github-app-token@v3
+        with:
+          app-id: ${{ vars.CLA_BOT_APP_ID }}
+          private-key: ${{ secrets.CLA_BOT_APP_PRIVATE_KEY }}
+          owner: ${{ github.repository_owner }}
+          repositories: cla
+          permission-contents: write
+
+      - name: Run CLA Bot
+        uses: overtrue/cla-bot@v0.0.9
+        with:
+          github-token: ${{ github.token }}
+          registry-token: ${{ steps.registry-token.outputs.token }}

.github/workflows/docker.yml

@@ -66,6 +66,7 @@ env:
   CARGO_TERM_COLOR: always
   REGISTRY_DOCKERHUB: rustfs/rustfs
   REGISTRY_GHCR: ghcr.io/${{ github.repository }}
+  REGISTRY_QUAY: quay.io/${{ secrets.QUAY_USERNAME }}/rustfs
   DOCKER_PLATFORMS: linux/amd64,linux/arm64
 
 jobs:
@@ -287,12 +288,19 @@ jobs:
           username: ${{ env.DOCKERHUB_USERNAME }}
           password: ${{ secrets.DOCKERHUB_TOKEN }}
 
-      # - name: Login to GitHub Container Registry
-      #   uses: docker/login-action@v3
-      #   with:
-      #     registry: ghcr.io
-      #     username: ${{ github.actor }}
-      #     password: ${{ secrets.GITHUB_TOKEN }}
+      - name: Login to GitHub Container Registry
+        uses: docker/login-action@v3
+        with:
+          registry: ghcr.io
+          username: ${{ secrets.GHCR_USERNAME }}
+          password: ${{ secrets.GHCR_PASSWORD }}
+
+      - name: Login to Quay.io
+        uses: docker/login-action@v3
+        with:
+          registry: quay.io
+          username: ${{ secrets.QUAY_USERNAME }}
+          password: ${{ secrets.QUAY_PASSWORD }}
 
       - name: Set up QEMU
         uses: docker/setup-qemu-action@v3
@@ -338,14 +346,15 @@ jobs:
 
           # Generate tags based on build type
           # Only support release and prerelease builds (no development builds)
-          TAGS="${{ env.REGISTRY_DOCKERHUB }}:${VERSION}${VARIANT_SUFFIX}"
+          TAG_BASE="${VERSION}${VARIANT_SUFFIX}"
+          TAGS="${{ env.REGISTRY_DOCKERHUB }}:$TAG_BASE,${{ env.REGISTRY_GHCR }}:$TAG_BASE,${{ env.REGISTRY_QUAY }}:$TAG_BASE"
 
           # Add channel tags for prereleases and latest for stable
           if [[ "$CREATE_LATEST" == "true" ]]; then
             # TODO: Temporary change - the current alpha version will also create the latest tag
             # After the version is stabilized, the logic here remains unchanged, but the upstream CREATE_LATEST setting needs to be restored.
             # Stable release (and temporary alpha versions)
-            TAGS="$TAGS,${{ env.REGISTRY_DOCKERHUB }}:latest${VARIANT_SUFFIX}"
+            TAGS="$TAGS,${{ env.REGISTRY_DOCKERHUB }}:latest${VARIANT_SUFFIX},${{ env.REGISTRY_GHCR }}:latest${VARIANT_SUFFIX},${{ env.REGISTRY_QUAY }}:latest${VARIANT_SUFFIX}"
           elif [[ "$BUILD_TYPE" == "prerelease" ]]; then
             # Prerelease channel tags (alpha, beta, rc)
             if [[ "$VERSION" == *"alpha"* ]]; then
@@ -357,7 +366,7 @@ jobs:
             fi
 
             if [[ -n "$CHANNEL" ]]; then
-              TAGS="$TAGS,${{ env.REGISTRY_DOCKERHUB }}:${CHANNEL}${VARIANT_SUFFIX}"
+              TAGS="$TAGS,${{ env.REGISTRY_DOCKERHUB }}:${CHANNEL}${VARIANT_SUFFIX},${{ env.REGISTRY_GHCR }}:${CHANNEL}${VARIANT_SUFFIX},${{ env.REGISTRY_QUAY }}:${CHANNEL}${VARIANT_SUFFIX}"
             fi
           fi
 

.github/workflows/e2e-s3tests.yml

@@ -50,6 +50,11 @@ env:
 
   RUST_LOG: info
   PLATFORM: linux/amd64
+  BUILDX_CACHE_SCOPE: rustfs-e2e-s3tests-source
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.ref }}-${{ github.event.inputs['test-mode'] || 'single' }}
+  cancel-in-progress: true
 
 defaults:
   run:
@@ -57,12 +62,25 @@ defaults:
 
 jobs:
   s3tests-single:
-    if: github.event.inputs.test-mode == 'single'
+    if: github.event.inputs['test-mode'] == 'single'
     runs-on: ubicloud-standard-2
     timeout-minutes: 120
     steps:
       - uses: actions/checkout@v6
 
+      - name: Cache pip downloads
+        uses: actions/cache@v4
+        with:
+          path: ~/.cache/pip
+          key: ${{ runner.os }}-pip-e2e-s3tests-${{ hashFiles('.github/workflows/e2e-s3tests.yml') }}
+          restore-keys: |
+            ${{ runner.os }}-pip-e2e-s3tests-
+
+      - name: Install Python tools
+        run: |
+          python3 -m pip install --user --upgrade pip awscurl tox
+          echo "$HOME/.local/bin" >> "$GITHUB_PATH"
+
       - name: Enable buildx
         uses: docker/setup-buildx-action@v3
 
@@ -70,8 +88,8 @@ jobs:
         run: |
           DOCKER_BUILDKIT=1 docker buildx build --load \
             --platform ${PLATFORM} \
-            --cache-from type=gha \
-            --cache-to type=gha,mode=max \
+            --cache-from type=gha,scope=${BUILDX_CACHE_SCOPE} \
+            --cache-to type=gha,mode=max,scope=${BUILDX_CACHE_SCOPE} \
             -t rustfs-ci \
             -f Dockerfile.source .
 
@@ -121,9 +139,6 @@ jobs:
 
       - name: Provision s3-tests alt user (required by suite)
         run: |
-          python3 -m pip install --user --upgrade pip awscurl
-          export PATH="$HOME/.local/bin:$PATH"
-
           # Admin API requires AWS SigV4 signing. awscurl is used by RustFS codebase as well.
           awscurl \
             --service s3 \
@@ -156,8 +171,6 @@ jobs:
 
       - name: Prepare s3-tests
         run: |
-          python3 -m pip install --user --upgrade pip tox
-          export PATH="$HOME/.local/bin:$PATH"
           git clone --depth 1 https://github.com/ceph/s3-tests.git s3-tests
 
       - name: Run ceph s3-tests (debug friendly)
@@ -211,12 +224,25 @@ jobs:
           path: artifacts/**
 
   s3tests-multi:
-    if: github.event_name == 'workflow_dispatch' && github.event.inputs.test-mode == 'multi'
+    if: github.event_name == 'workflow_dispatch' && github.event.inputs['test-mode'] == 'multi'
     runs-on: ubicloud-standard-2
     timeout-minutes: 150
     steps:
       - uses: actions/checkout@v6
 
+      - name: Cache pip downloads
+        uses: actions/cache@v4
+        with:
+          path: ~/.cache/pip
+          key: ${{ runner.os }}-pip-e2e-s3tests-${{ hashFiles('.github/workflows/e2e-s3tests.yml') }}
+          restore-keys: |
+            ${{ runner.os }}-pip-e2e-s3tests-
+
+      - name: Install Python tools
+        run: |
+          python3 -m pip install --user --upgrade pip awscurl tox
+          echo "$HOME/.local/bin" >> "$GITHUB_PATH"
+
       - name: Enable buildx
         uses: docker/setup-buildx-action@v3
 
@@ -224,8 +250,8 @@ jobs:
         run: |
           DOCKER_BUILDKIT=1 docker buildx build --load \
             --platform ${PLATFORM} \
-            --cache-from type=gha \
-            --cache-to type=gha,mode=max \
+            --cache-from type=gha,scope=${BUILDX_CACHE_SCOPE} \
+            --cache-to type=gha,mode=max,scope=${BUILDX_CACHE_SCOPE} \
             -t rustfs-ci \
             -f Dockerfile.source .
 
@@ -337,9 +363,6 @@ jobs:
 
       - name: Provision s3-tests alt user (required by suite)
         run: |
-          python3 -m pip install --user --upgrade pip awscurl
-          export PATH="$HOME/.local/bin:$PATH"
-
           awscurl \
             --service s3 \
             --region "${S3_REGION}" \
@@ -368,8 +391,6 @@ jobs:
 
       - name: Prepare s3-tests
         run: |
-          python3 -m pip install --user --upgrade pip tox
-          export PATH="$HOME/.local/bin:$PATH"
           git clone --depth 1 https://github.com/ceph/s3-tests.git s3-tests
 
       - name: Run ceph s3-tests (multi, debug friendly)

.github/workflows/nix-flake-update.yml

@@ -31,11 +31,11 @@ jobs:
   update-flake:
     name: Update flake.lock
     runs-on: ubuntu-latest
-    timeout-minutes: 30
+    timeout-minutes: 45
     steps:
       - name: Checkout repository
         uses: actions/checkout@v6
-
+        
       - name: Install Nix
         uses: DeterminateSystems/determinate-nix-action@v3
 
@@ -46,14 +46,18 @@ jobs:
         id: update
         uses: DeterminateSystems/update-flake-lock@main
         with:
+          git-author-name: heihutu
+          git-author-email: heihutu@gmail.com
+          git-committer-name: heihutu
+          git-committer-email: heihutu@gmail.com
           pr-title: "chore(deps): update flake.lock"
           pr-labels: |
             dependencies
             nix
             automated
           commit-msg: "chore(deps): update flake.lock"
-          pr-reviewers: houseme, heihutu
-          token: ${{ secrets.GITHUB_TOKEN }}
+          pr-reviewers: houseme, overtrue, majinghe
+          token: ${{ secrets.FLAKE_UPDATE_TOKEN }}
 
       - name: Log PR details
         if: steps.update.outputs.pull-request-number

.github/workflows/nix.yml

@@ -43,7 +43,7 @@ jobs:
   nix-validation:
     name: Nix Build & Check
     runs-on: ubuntu-latest
-    timeout-minutes: 30
+    timeout-minutes: 45
     steps:
       - name: Checkout repository
         uses: actions/checkout@v6
@@ -57,7 +57,7 @@ jobs:
 
       - name: Setup Magic Nix Cache
         uses: DeterminateSystems/magic-nix-cache-action@v13
-        
+
       - name: Setup Flake Checker
         uses: DeterminateSystems/flake-checker-action@v12
 

.gitignore

@@ -2,6 +2,7 @@
 .DS_Store
 .idea
 .vscode
+.cursor
 .direnv/
 /test
 /logs
@@ -15,10 +16,12 @@ vendor
 cli/rustfs-gui/embedded-rustfs/rustfs
 *.log
 deploy/certs/*
+deploy/data/*
 *jsonl
 .env
 .rustfs.sys
-.cargo
+.cargo/
+!.cargo/config.toml
 profile.json
 .docker/openobserve-otel/data
 *.zst
@@ -37,10 +40,16 @@ artifacts/
 *.audit
 *.snappy
 PR_DESCRIPTION.md
-IMPLEMENTATION_PLAN.md
 scripts/s3-tests/selected_tests.txt
 docs
 
 # nix stuff
 result*
 *.gz
+rustfs-webdav.code-workspace
+
+.aiexclude
+*.bak
+# Local test/benchmark artifacts
+benchmarks.logs
+tmp/

.vscode/launch.json

@@ -197,19 +197,27 @@
                 // "__RUSTFS_SSE_SIMPLE_CMK": "2dfNXGHlsEflGVCxb+5DIdGEl1sIvtwX+QfmYasi5QM=",
 
                 // 2. kms local backend test key
-                "RUSTFS_KMS_ENABLE": "true",
-                "RUSTFS_KMS_BACKEND": "local",
-                "RUSTFS_KMS_KEY_DIR": "./target/kms-key-dir",
-                "RUSTFS_KMS_LOCAL_MASTER_KEY": "my-secret-key", // Some Password
-                "RUSTFS_KMS_DEFAULT_KEY_ID": "rustfs-master-key",
+                // "RUSTFS_KMS_ENABLE": "true",
+                // "RUSTFS_KMS_BACKEND": "local",
+                // "RUSTFS_KMS_KEY_DIR": "./target/kms-key-dir",
+                // "RUSTFS_KMS_LOCAL_MASTER_KEY": "my-secret-key", // Some Password
+                // "RUSTFS_KMS_DEFAULT_KEY_ID": "rustfs-master-key",
 
                 // 3. kms vault backend test key
                 // "RUSTFS_KMS_ENABLE": "true",
-                // "RUSTFS_KMS_BACKEND": "vault",
+                // "RUSTFS_KMS_BACKEND": "vault-kv2",
                 // "RUSTFS_KMS_VAULT_ADDRESS": "http://127.0.0.1:8200",
                 // "RUSTFS_KMS_VAULT_TOKEN": "Dev Token",
                 // "RUSTFS_KMS_DEFAULT_KEY_ID": "rustfs-master-key",
 
+                // 4. kms vault transit backend test key
+                "RUSTFS_KMS_ENABLE": "true",
+                "RUSTFS_KMS_BACKEND": "vault-transit",
+                "RUSTFS_KMS_VAULT_ADDRESS": "http://127.0.0.1:8200",
+                "RUSTFS_KMS_VAULT_TOKEN": "Dev Token",
+                "RUSTFS_KMS_VAULT_MOUNT_PATH": "transit",
+                "RUSTFS_KMS_DEFAULT_KEY_ID": "rustfs-master-key",
+
             },
             "sourceLanguages": [
                 "rust"

AGENTS.md

@@ -1,101 +1,95 @@
-# Repository Guidelines
+# RustFS Agent Instructions (Global)
 
-This file provides guidance for AI agents and developers working with code in this repository.
+This root file keeps repository-wide rules only.
+Use the nearest subdirectory `AGENTS.md` for path-specific guidance.
 
-## Project Overview
+## Rule Precedence
 
-RustFS is a high-performance distributed object storage software built with Rust, providing S3-compatible APIs and advanced features like data lakes, AI, and big data support. It's designed as an alternative to MinIO with better performance and a more business-friendly Apache 2.0 license.
+1. System/developer instructions.
+2. This file (global defaults).
+3. The nearest `AGENTS.md` in the current path (more specific scope wins).
 
-## ⚠️ Pre-Commit Checklist (MANDATORY)
+If repo-level instructions conflict, follow the nearest file and keep behavior aligned with CI.
+
+## Communication and Language
+
+- Respond in the same language used by the requester.
+- Keep source code, comments, commit messages, and PR title/body in English.
+
+## Change Style for Existing Logic
+
+- Prefer direct, local code over extracting one-off helpers.
+- Extract a helper only when logic is reused or the extraction materially clarifies a non-trivial flow.
+- Preserve the existing control-flow and logic shape when fixing bugs or addressing review comments, especially in init, distributed coordination, locking, metadata, and concurrency paths.
+- Do not refactor existing code only to make it easier to unit test.
+- Keep fixes narrowly aligned with the requested behavior; avoid semantic-adjacent rewrites while touching sensitive paths.
+- Keep code elegant, concise, and direct. Prefer minimal, readable implementations over over-engineering and excessive abstraction. Use comments to clarify non-obvious intent and invariants, not to compensate for unclear code.
+
+## Constant and String Usage
+
+- Before introducing new string literals, search for existing constants/enums that already represent the same semantic value.
+- Reuse existing constants for protocol labels, error identifiers, header keys, event names, metric names, command tags, and similar fixed tokens.
+- If a new string is truly unique, define a local constant near related logic and avoid scattering the literal across multiple sites.
+- When changing existing behavior, keep naming and format consistency by aligning with established project constants.
+
+## Sources of Truth
+
+- Workspace layout and crate membership: `Cargo.toml` (`[workspace].members`)
+- Local quality commands: `Makefile` and `.config/make/`
+- CI quality gates: `.github/workflows/ci.yml`
+- PR template: `.github/pull_request_template.md`
+
+Avoid duplicating long crate lists or command matrices in instruction files.
+Reference the source files above instead.
+
+## Verification Before PR
+
+For code changes, run and pass the following before opening a PR:
 
-**Before EVERY commit, you MUST run and pass ALL of the following:**
 ```bash
-cargo fmt --all --check    # Code formatting
-cargo clippy --all-targets --all-features -- -D warnings  # Lints
-cargo test --workspace --exclude e2e_test  # Unit tests
+make pre-commit
 ```
-Or simply run `make pre-commit` which covers all checks. **DO NOT commit if any check fails.**
 
-## Communication Rules
+Before pushing code changes, make sure formatting is clean:
 
-- Respond to the user in the same language used by the user.
-- Code and documentation must be written in English only.
-- **Pull Request titles and descriptions must be written in English** to ensure consistency and accessibility for all contributors.
+- Run `cargo fmt --all`.
+- Run `cargo fmt --all --check` and ensure no files are modified unexpectedly.
 
-## Project Structure
+If `make` is unavailable, run the equivalent checks defined under `.config/make/`.
+Documentation-only or instruction-only changes are exempt from the verification commands above (including the `.config/make/` equivalents), though any installed git pre-commit hooks (for example, from `make setup-hooks`) may still run on commit unless explicitly skipped.
+After build-based verification completes, clean generated build artifacts before wrapping up to avoid unnecessary disk usage.
+Do not open a PR with code changes when the required checks fail.
 
-The workspace root hosts shared dependencies in `Cargo.toml`. The service binary lives under `rustfs/src/main.rs`, while reusable crates sit in `crates/` (including `ecstore`, `iam`, `kms`, `madmin`, `s3select-api`, `s3select-query`, `config`, `crypto`, `lock`, `filemeta`, `rio`, `common`, `protos`, `audit-logger`, `notify`, `obs`, `workers`, `appauth`, `ahm`, `mcp`, `signer`, `checksums`, `utils`, `zip`, `targets`, and `e2e_test`). Deployment manifests are under `deploy/`, Docker assets sit at the root, and automation lives in `scripts/`.
+## Git and PR Baseline
 
-### Core Architecture
+- Use feature branches based on the latest `main`.
+- Follow Conventional Commits, with subject length <= 72 characters.
+- Keep PR title and description in English.
+- Use `.github/pull_request_template.md` and keep all section headings.
+- Use `N/A` for non-applicable template sections.
+- Include verification commands in the PR description.
+- When using `gh pr create`/`gh pr edit`, use `--body-file` instead of inline `--body` for multiline markdown.
+- After fixing code review comments or CI findings, always mark corresponding review
+  comments/threads as resolved before returning to the user.
+- In handling review comments, confirm the underlying issue before changing code.
+  If a suggested change is not appropriate for behavior or risk, reply with a
+  concise rationale instead of blindly applying it.
 
-- **Main Binary (`rustfs/`):** Entry point at `rustfs/src/main.rs`, includes admin, auth, config, server, storage, license management, and profiling modules.
-- **Key Crates:** Cargo workspace with 25+ crates supporting S3-compatible APIs, erasure coding storage, IAM, KMS, S3 Select, and observability.
-- **Build System:** Custom `build-rustfs.sh` script, multi-architecture Docker builds, Make/Just task runners, cross-compilation support.
+## Security Baseline
 
-## Build, Test, and Development Commands
+- Never commit secrets, credentials, or key material.
+- Use environment variables or vault tooling for sensitive configuration.
+- For localhost-sensitive tests, verify proxy settings to avoid traffic leakage.
 
-### Quick Commands
-- `cargo check --all-targets` - Fast validation
-- `cargo build --release` or `make build` - Release build
-- `./build-rustfs.sh --dev` - Development build with debug symbols
-- `make pre-commit` - Run all quality checks (fmt, clippy, check, test)
+## Scoped Guidance in This Repository
 
-### Testing
-- `cargo test --workspace --exclude e2e_test` - Unit tests
-- `cargo test --package e2e_test` - E2E tests
-- For KMS tests: `NO_PROXY=127.0.0.1,localhost HTTP_PROXY= HTTPS_PROXY= http_proxy= https_proxy= cargo test --package e2e_test test_local_kms_end_to_end -- --nocapture --test-threads=1`
-
-### Cross-Platform Builds
-- `./build-rustfs.sh --platform x86_64-unknown-linux-musl` - Build for musl
-- `./build-rustfs.sh --platform aarch64-unknown-linux-gnu` - Build for ARM64
-- `make build-cross-all` - Build all supported architectures
-
-## Coding Style & Safety Requirements
-
-- **Formatting:** Follow `rustfmt.toml` (130-column width). Use `snake_case` for items, `PascalCase` for types, `SCREAMING_SNAKE_CASE` for constants.
-- **Safety:** `unsafe_code = "deny"` enforced at workspace level. Never use `unwrap()`, `expect()`, or panic-inducing code except in tests.
-- **Error Handling:** Prefer `anyhow` for applications, `thiserror` for libraries. Use proper error handling with `Result<T, E>` and `Option<T>`.
-- **Async:** Keep async code non-blocking. Offload CPU-heavy work with `tokio::task::spawn_blocking` when necessary.
-- **Language:** Code comments, function names, variable names, and all text in source files must be in English only.
-
-## Testing Guidelines
-
-Co-locate unit tests with their modules and give behavior-led names. Integration suites belong in each crate's `tests/` directory, while exhaustive end-to-end scenarios live in `crates/e2e_test/`. When fixing bugs or adding features, include regression tests that capture the new behavior.
-
-## KMS (Key Management Service)
-
-- **Implementation:** Complete with Local and Vault backends, auto-configures on startup with `--kms-enable` flag.
-- **Encryption:** Full S3-compatible server-side encryption (SSE-S3, SSE-KMS, SSE-C).
-- **Testing:** Comprehensive E2E tests in `crates/e2e_test/src/kms/`. Requires proxy bypass for local testing.
-- **Key Files:** `crates/kms/`, `rustfs/src/storage/ecfs.rs`, `rustfs/src/admin/handlers/kms*.rs`
-
-## Environment Variables
-
-- `RUSTFS_ENABLE_SCANNER` - Enable/disable background data scanner (default: true)
-- `RUSTFS_ENABLE_HEAL` - Enable/disable auto-heal functionality (default: true)
-- For KMS tests: `NO_PROXY=127.0.0.1,localhost` and clear proxy environment variables
-
-## Commit & Pull Request Guidelines
-
-Work on feature branches (e.g., `feat/...`) after syncing `main`. Follow Conventional Commits under 72 characters. Each commit must compile, format cleanly, and pass `make pre-commit`.
-
-**Pull Request Requirements:**
-- PR titles and descriptions **MUST be written in English**
-- Open PRs with a concise summary, note verification commands, link relevant issues
-- Follow the PR template format and fill in all required sections
-- Wait for reviewer approval before merging
-
-## Security & Configuration Tips
-
-Do not commit secrets or cloud credentials; prefer environment variables or vault tooling. Review IAM- and KMS-related changes with a second maintainer. Confirm proxy settings before running sensitive tests to avoid leaking traffic outside localhost.
-
-## Important Reminders
-
-- Always compile after code changes: Use `cargo build` to catch errors early
-- Don't bypass tests: All functionality must be properly tested, not worked around
-- Use proper error handling: Never use `unwrap()` or `expect()` in production code (except tests)
-- Do what has been asked; nothing more, nothing less
-- NEVER create files unless they're absolutely necessary for achieving your goal
-- ALWAYS prefer editing an existing file to creating a new one
-- NEVER proactively create documentation files (*.md) or README files unless explicitly requested
-- NEVER commit PR description files (e.g., PR_DESCRIPTION.md): These are temporary reference files for creating pull requests and should remain local only
+- `.github/AGENTS.md`
+- `crates/AGENTS.md`
+- `crates/config/AGENTS.md`
+- `crates/ecstore/AGENTS.md`
+- `crates/e2e_test/AGENTS.md`
+- `crates/iam/AGENTS.md`
+- `crates/kms/AGENTS.md`
+- `crates/policy/AGENTS.md`
+- `rustfs/src/admin/AGENTS.md`
+- `rustfs/src/storage/AGENTS.md`

CHANGELOG.md

@@ -7,6 +7,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+### Fixed
+- **Helm Ingress**: `customAnnotations` are now merged with class-specific annotations (nginx/traefik) instead of being ignored when `ingress.className` is set.
+
 ### Added
 - **OpenStack Keystone Authentication Integration**: Full support for OpenStack Keystone authentication via X-Auth-Token headers
   - Tower-based middleware (`KeystoneAuthLayer`) self-contained within `rustfs-keystone` crate

Cargo.toml

@@ -39,6 +39,8 @@ members = [
     "crates/protocols", # Protocol implementations (FTPS, SFTP, etc.)
     "crates/protos", # Protocol buffer definitions
     "crates/rio", # Rust I/O utilities and abstractions
+    "crates/concurrency", # Rust I/O utilities and abstractions
+    "crates/s3-common", # Common utilities and data structures for S3 compatibility
     "crates/s3select-api", # S3 Select API interface
     "crates/s3select-query", # S3 Select query engine
     "crates/scanner", # Scanner for data integrity checks and health monitoring
@@ -47,9 +49,12 @@ members = [
     "crates/trusted-proxies", # Trusted proxies management
     "crates/utils", # Utility functions and helpers
     "crates/workers", # Worker thread pools and task scheduling
+    "crates/io-metrics", # Zero-copy metrics collection for performance analysis
+    "crates/io-core", # Zero-copy core reader and writer implementations
+    "crates/object-io", # Object I/O policy and zero-copy helper primitives
     "crates/zip", # ZIP file handling and compression
 ]
-resolver = "2"
+resolver = "3"
 
 [workspace.package]
 edition = "2024"
@@ -78,6 +83,7 @@ rustfs-audit = { path = "crates/audit", version = "0.0.5" }
 rustfs-checksums = { path = "crates/checksums", version = "0.0.5" }
 rustfs-common = { path = "crates/common", version = "0.0.5" }
 rustfs-config = { path = "./crates/config", version = "0.0.5" }
+rustfs-concurrency = { path = "./crates/concurrency", version = "0.0.5" }
 rustfs-credentials = { path = "crates/credentials", version = "0.0.5" }
 rustfs-crypto = { path = "crates/crypto", version = "0.0.5" }
 rustfs-ecstore = { path = "crates/ecstore", version = "0.0.5" }
@@ -90,10 +96,14 @@ rustfs-madmin = { path = "crates/madmin", version = "0.0.5" }
 rustfs-mcp = { path = "crates/mcp", version = "0.0.5" }
 rustfs-metrics = { path = "crates/metrics", version = "0.0.5" }
 rustfs-notify = { path = "crates/notify", version = "0.0.5" }
+rustfs-io-metrics = { path = "crates/io-metrics", version = "0.0.5" }
+rustfs-io-core = { path = "crates/io-core", version = "0.0.5" }
+rustfs-object-io = { path = "crates/object-io", version = "0.0.5" }
 rustfs-obs = { path = "crates/obs", version = "0.0.5" }
 rustfs-policy = { path = "crates/policy", version = "0.0.5" }
 rustfs-protos = { path = "crates/protos", version = "0.0.5" }
 rustfs-rio = { path = "crates/rio", version = "0.0.5" }
+rustfs-s3-common = { path = "crates/s3-common", version = "0.0.5" }
 rustfs-s3select-api = { path = "crates/s3select-api", version = "0.0.5" }
 rustfs-s3select-query = { path = "crates/s3select-query", version = "0.0.5" }
 rustfs-scanner = { path = "crates/scanner", version = "0.0.5" }
@@ -115,15 +125,15 @@ futures = "0.3.32"
 futures-core = "0.3.32"
 futures-util = "0.3.32"
 pollster = "0.4.0"
-hyper = { version = "1.8.1", features = ["http2", "http1", "server"] }
+hyper = { version = "1.9.0", features = ["http2", "http1", "server"] }
 hyper-rustls = { version = "0.27.7", default-features = false, features = ["native-tokio", "http1", "tls12", "logging", "http2", "aws-lc-rs", "webpki-roots"] }
 hyper-util = { version = "0.1.20", features = ["tokio", "server-auto", "server-graceful", "tracing"] }
 http = "1.4.0"
 http-body = "1.0.1"
 http-body-util = "0.1.3"
 reqwest = { version = "0.13.2", default-features = false, features = ["rustls", "charset", "http2", "system-proxy", "stream", "json", "blocking", "query", "form"] }
-socket2 = { version = "0.6.2", features = ["all"] }
-tokio = { version = "1.49.0", features = ["fs", "rt-multi-thread"] }
+socket2 = { version = "0.6.3", features = ["all"] }
+tokio = { version = "1.51.0", features = ["fs", "rt-multi-thread"] }
 tokio-rustls = { version = "0.26.4", default-features = false, features = ["logging", "tls12", "aws-lc-rs"] }
 tokio-stream = { version = "0.1.18" }
 tokio-test = "0.4.5"
@@ -142,7 +152,7 @@ flatbuffers = "25.12.19"
 form_urlencoded = "1.2.2"
 prost = "0.14.3"
 quick-xml = "0.39.2"
-rmcp = { version = "0.17.0" }
+rmcp = { version = "1.3.0" }
 rmp = { version = "0.8.15" }
 rmp-serde = { version = "1.3.1" }
 serde = { version = "1.0.228", features = ["derive"] }
@@ -152,129 +162,138 @@ schemars = "1.2.1"
 
 # Cryptography and Security
 aes-gcm = { version = "0.11.0-rc.3", features = ["rand_core"] }
-argon2 = { version = "0.6.0-rc.7" }
-blake3 = { version = "1.8.3", features = ["rayon", "mmap"] }
+argon2 = { version = "0.6.0-rc.8" }
+blake2 = "0.11.0-rc.5"
 chacha20poly1305 = { version = "0.11.0-rc.3" }
 crc-fast = "1.9.0"
-hmac = { version = "0.13.0-rc.5" }
+hmac = { version = "0.13.0" }
 jsonwebtoken = { version = "10.3.0", features = ["aws_lc_rs"] }
 openidconnect = { version = "4.0", default-features = false }
-pbkdf2 = "0.13.0-rc.9"
-rsa = { version = "0.10.0-rc.16" }
+pbkdf2 = "0.13.0-rc.10"
+rsa = { version = "0.10.0-rc.17" }
 rustls = { version = "0.23.37", default-features = false, features = ["aws-lc-rs", "logging", "tls12", "prefer-post-quantum", "std"] }
 rustls-pki-types = "1.14.0"
-sha1 = "0.11.0-rc.5"
-sha2 = "0.11.0-rc.5"
+sha1 = "0.11.0"
+sha2 = "0.11.0"
 subtle = "2.6"
 zeroize = { version = "1.8.2", features = ["derive"] }
 
 # Time and Date
 chrono = { version = "0.4.44", features = ["serde"] }
 humantime = "2.3.0"
-jiff = { version = "0.2.22", features = ["serde"] }
+jiff = { version = "0.2.23", features = ["serde"] }
 time = { version = "0.3.47", features = ["std", "parsing", "formatting", "macros", "serde"] }
 
 # Utilities and Tools
 anyhow = "1.0.102"
-arc-swap = "1.8.2"
-astral-tokio-tar = "0.5.6"
+arc-swap = "1.9.1"
+astral-tokio-tar = "0.6.0"
 atoi = "2.0.0"
 atomic_enum = "0.3.0"
-aws-config = { version = "1.8.14" }
-aws-credential-types = { version = "1.2.13" }
-aws-sdk-s3 = { version = "1.124.0", default-features = false, features = ["sigv4a", "default-https-client", "rt-tokio"] }
-aws-smithy-http-client = { version = "1.1.11", default-features = false, features = ["default-client", "rustls-aws-lc"] }
-aws-smithy-types = { version = "1.4.5" }
+aws-config = { version = "1.8.15" }
+aws-credential-types = { version = "1.2.14" }
+aws-sdk-s3 = { version = "1.128.0", default-features = false, features = ["sigv4a", "default-https-client", "rt-tokio"] }
+aws-smithy-http-client = { version = "1.1.12", default-features = false, features = ["default-client", "rustls-aws-lc"] }
+aws-smithy-types = { version = "1.4.7" }
 backtrace = "0.3.76"
 base64 = "0.22.1"
 base64-simd = "0.8.0"
 brotli = "8.0.2"
 cfg-if = "1.0.4"
-clap = { version = "4.5.60", features = ["derive", "env"] }
+clap = { version = "4.6.0", features = ["derive", "env"] }
 const-str = { version = "1.1.0", features = ["std", "proc"] }
 convert_case = "0.11.0"
 criterion = { version = "0.8", features = ["html_reports"] }
 crossbeam-queue = "0.3.12"
-datafusion = "52.1.0"
+crossbeam-channel = "0.5.15"
+crossbeam-deque = "0.8.6"
+crossbeam-utils = "0.8.21"
+datafusion = "53.0.0"
 derive_builder = "0.20.2"
 enumset = "1.1.10"
 faster-hex = "0.10.0"
 flate2 = "1.1.9"
 glob = "0.3.3"
-google-cloud-storage = "1.8.0"
-google-cloud-auth = "1.6.0"
+google-cloud-storage = "1.10.0"
+google-cloud-auth = "1.8.0"
 hashbrown = { version = "0.16.1", features = ["serde", "rayon"] }
+hex = "0.4.3"
 hex-simd = "0.8.0"
 highway = { version = "1.3.0" }
 ipnetwork = { version = "0.21.1", features = ["serde"] }
 lazy_static = "1.5.0"
-libc = "0.2.182"
+libc = "0.2.184"
 libsystemd = "0.7.2"
-local-ip-address = "0.6.10"
+local-ip-address = "0.6.11"
+memmap2 = "0.9.10"
 lz4 = "1.28.1"
 matchit = "0.9.1"
-md-5 = "0.11.0-rc.5"
+md-5 = "0.11.0"
 md5 = "0.8.0"
 mime_guess = "2.0.5"
-moka = { version = "0.12.13", features = ["future"] }
+moka = { version = "0.12.15", features = ["future"] }
 netif = "0.1.6"
 num_cpus = { version = "1.17.0" }
-nvml-wrapper = "0.12.0"
-object_store = "0.12.5"
+nvml-wrapper = "0.12.1"
+object_store = "0.13.2"
 parking_lot = "0.12.5"
 path-absolutize = "3.1.1"
 path-clean = "1.0.1"
+percent-encoding = "2.3.2"
 pin-project-lite = "0.2.17"
 pretty_assertions = "1.4.1"
 rand = { version = "0.10.0", features = ["serde"] }
-ratelimit = "0.10.0"
+ratelimit = "0.10.1"
 rayon = "1.11.0"
-reed-solomon-simd = { version = "3.1.0" }
+reed-solomon-erasure = { version = "6.0", default-features = false, features = ["std", "simd-accel"] }
+reed-solomon-simd = "3.1.0"
 regex = { version = "1.12.3" }
-rumqttc = { version = "0.25.1" }
+rumqttc = { package = "rumqttc-next", version = "0.29.0", features = ["websocket"] }
 rustix = { version = "1.1.4", features = ["fs"] }
 rust-embed = { version = "8.11.0" }
-rustc-hash = { version = "2.1.1" }
-s3s = { version = "0.13.0", features = ["minio"] }
+rustc-hash = { version = "2.1.2" }
+s3s = { git = "https://github.com/rustfs/s3s", rev = "010ae03bdd8c93a9c046108190987bb0e6cb0692", features = ["minio"] }
 serial_test = "3.4.0"
-shadow-rs = { version = "1.7.0", default-features = false }
+shadow-rs = { version = "1.7.1", default-features = false }
 siphasher = "1.0.2"
 smallvec = { version = "1.15.1", features = ["serde"] }
 smartstring = "1.0.1"
-snafu = "0.8.9"
+snafu = "0.9.0"
 snap = "1.1.1"
 starshard = { version = "1.1.0", features = ["rayon", "async", "serde"] }
 strum = { version = "0.28.0", features = ["derive"] }
-sysinfo = "0.38.2"
+sysinfo = "0.38.4"
 temp-env = "0.3.6"
-tempfile = "3.26.0"
+tempfile = "3.27.0"
 test-case = "3.3.1"
 thiserror = "2.0.18"
 tracing = { version = "0.1.44" }
 tracing-appender = "0.2.4"
 tracing-error = "0.2.1"
 tracing-opentelemetry = "0.32.1"
-tracing-subscriber = { version = "0.3.22", features = ["env-filter", "time"] }
+tracing-subscriber = { version = "0.3.23", features = ["env-filter", "time"] }
 transform-stream = "0.3.1"
 url = "2.5.8"
 urlencoding = "2.1.3"
-uuid = { version = "1.21.0", features = ["v4", "fast-rng", "macro-diagnostics"] }
-vaultrs = { version = "0.7.4" }
+uuid = { version = "1.23.0", features = ["v4", "fast-rng", "macro-diagnostics"] }
+vaultrs = { version = "0.8.0" }
 walkdir = "2.5.0"
 wildmatch = { version = "2.6.1", features = ["serde"] }
 windows = { version = "0.62.2" }
 xxhash-rust = { version = "0.8.15", features = ["xxh64", "xxh3"] }
-zip = "8.1.0"
+zip = "8.5.0"
 zstd = "0.13.3"
 
 # Observability and Metrics
 metrics = "0.24.3"
+dial9-tokio-telemetry = { version = "0.2", git = "https://github.com/dial9-rs/dial9-tokio-telemetry.git", rev = "60502082601b647c4a51962595721f631b7bbce1" }
 opentelemetry = { version = "0.31.0" }
 opentelemetry-appender-tracing = { version = "0.31.1", features = ["experimental_use_tracing_span_context", "experimental_metadata_attributes", "spec_unstable_logs_enabled"] }
-opentelemetry-otlp = { version = "0.31.0", features = ["gzip-http", "reqwest-rustls"] }
+opentelemetry-otlp = { version = "0.31.1", features = ["gzip-http", "reqwest-rustls"] }
 opentelemetry_sdk = { version = "0.31.0" }
 opentelemetry-semantic-conventions = { version = "0.31.0", features = ["semconv_experimental"] }
 opentelemetry-stdout = { version = "0.31.0" }
+pyroscope = { version = "2.0.0", features = ["backend-pprof-rs"] }
 
 # FTP and SFTP
 libunftp = { version = "0.23.0", features = ["experimental"] }
@@ -282,6 +301,9 @@ unftp-core = "0.1.0"
 suppaftp = { version = "8.0.2", features = ["tokio", "tokio-rustls-aws-lc-rs"] }
 rcgen = "0.14.7"
 
+# WebDAV
+dav-server = "0.11.0"
+
 # Performance Analysis and Memory Profiling
 mimalloc = "0.1"
 # Use tikv-jemallocator as memory allocator and enable performance analysis
@@ -291,7 +313,9 @@ tikv-jemalloc-ctl = { version = "0.6", features = ["use_std", "stats", "profilin
 # Used to generate pprof-compatible memory profiling data and support symbolization and flame graphs
 jemalloc_pprof = { version = "0.8.2", features = ["symbolize", "flamegraph"] }
 # Used to generate CPU performance analysis data and flame diagrams
-pprof = { version = "0.15.0", features = ["flamegraph", "protobuf-codec"] }
+# pprof = { version = "0.15.0", features = ["flamegraph", "protobuf-codec"] }
+# Pyroscope uses a patched pprof, until they merge back upstream, replace all references. Otherwise, two pprof libs with symbol collision.
+pprof = { package = "pprof-pyroscope-fork", version = "0.1500.3", features = ["flamegraph", "protobuf-codec"] }
 
 [workspace.metadata.cargo-shear]
 ignored = ["rustfs", "rustfs-mcp"]

Dockerfile

@@ -72,7 +72,8 @@ LABEL name="RustFS" \
       url="https://rustfs.com" \
       license="Apache-2.0"
 
-RUN apk add --no-cache ca-certificates coreutils curl
+RUN apk update && \
+    apk add --no-cache ca-certificates coreutils curl "zlib>=1.3.2-r0"
 
 COPY --from=build /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/
 COPY --from=build /build/rustfs /usr/bin/rustfs
@@ -86,12 +87,7 @@ RUN addgroup -g 10001 -S rustfs && \
     chown -R rustfs:rustfs /data /logs && \
     chmod 0750 /data /logs
 
-ENV RUSTFS_ADDRESS=":9000" \
-    RUSTFS_CONSOLE_ADDRESS=":9001" \
-    RUSTFS_ACCESS_KEY="rustfsadmin" \
-    RUSTFS_SECRET_KEY="rustfsadmin" \
-    RUSTFS_CONSOLE_ENABLE="true" \
-    RUSTFS_CORS_ALLOWED_ORIGINS="*" \
+ENV RUSTFS_CORS_ALLOWED_ORIGINS="*" \
     RUSTFS_CONSOLE_CORS_ALLOWED_ORIGINS="*" \
     RUSTFS_VOLUMES="/data" \
     RUSTFS_OBS_LOGGER_LEVEL=warn \

Dockerfile.decommission-local

@@ -0,0 +1,65 @@
+# Copyright 2024 RustFS Team
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+FROM rust:1.91-trixie
+
+RUN set -eux; \
+    export DEBIAN_FRONTEND=noninteractive; \
+    apt-get update; \
+    apt-get install -y --no-install-recommends \
+    build-essential \
+    ca-certificates \
+    curl \
+    git \
+    pkg-config \
+    libssl-dev \
+    lld \
+    protobuf-compiler \
+    flatbuffers-compiler; \
+    rm -rf /var/lib/apt/lists/*
+
+WORKDIR /usr/src/rustfs
+
+COPY . .
+
+RUN ./scripts/static.sh
+RUN cargo run --bin gproto
+RUN cargo build --release --locked --bin rustfs
+
+RUN set -eux; \
+    groupadd -g 10001 rustfs; \
+    useradd -u 10001 -g rustfs -M -s /usr/sbin/nologin rustfs
+
+WORKDIR /app
+
+RUN set -eux; \
+    mkdir -p /data /logs; \
+    chown -R rustfs:rustfs /data /logs /app; \
+    chmod 0750 /data /logs
+
+COPY entrypoint.sh /entrypoint.sh
+RUN install -m 0755 /usr/src/rustfs/target/release/rustfs /usr/bin/rustfs && chmod +x /entrypoint.sh
+
+ENV RUSTFS_VOLUMES="/data" \
+    RUST_LOG="warn" \
+    RUSTFS_OBS_LOG_DIRECTORY="/logs" \
+    RUSTFS_USERNAME="rustfs" \
+    RUSTFS_GROUPNAME="rustfs" \
+    RUSTFS_UID="10001" \
+    RUSTFS_GID="10001"
+
+EXPOSE 9000 9001
+
+ENTRYPOINT ["/entrypoint.sh"]
+CMD ["/usr/bin/rustfs"]

Dockerfile.source

@@ -157,11 +157,7 @@ WORKDIR /app
 ENV CARGO_INCREMENTAL=1
 
 # Ensure we have the same default env vars available
-ENV RUSTFS_ADDRESS=":9000" \
-    RUSTFS_ACCESS_KEY="rustfsadmin" \
-    RUSTFS_SECRET_KEY="rustfsadmin" \
-    RUSTFS_CONSOLE_ENABLE="true" \
-    RUSTFS_VOLUMES="/data" \
+ENV RUSTFS_VOLUMES="/data" \
     RUST_LOG="warn" \
     RUSTFS_OBS_LOG_DIRECTORY="/logs" \
     RUSTFS_USERNAME="rustfs" \
@@ -222,11 +218,7 @@ COPY entrypoint.sh /entrypoint.sh
 RUN chmod +x /usr/bin/rustfs /entrypoint.sh
 
 # Default environment (override in docker run/compose as needed)
-ENV RUSTFS_ADDRESS=":9000" \
-    RUSTFS_ACCESS_KEY="rustfsadmin" \
-    RUSTFS_SECRET_KEY="rustfsadmin" \
-    RUSTFS_CONSOLE_ENABLE="true" \
-    RUSTFS_VOLUMES="/data" \
+ENV RUSTFS_VOLUMES="/data" \
     RUST_LOG="warn" \
     RUSTFS_USERNAME="rustfs" \
     RUSTFS_GROUPNAME="rustfs" \

README.md

@@ -1,4 +1,4 @@
-[![RustFS](https://github.com/user-attachments/assets/3ba82e75-2f2d-4415-a4aa-1e4ffe9f22fd)](https://rustfs.com)
+[![RustFS](https://repository-images.githubusercontent.com/722597620/0fa936a2-8164-4f53-867f-def4beb64b21)](https://rustfs.com)
 
 <p align="center">RustFS is a high-performance, distributed object storage system built in Rust.</p>
 
@@ -42,6 +42,7 @@ Unlike other storage systems, RustFS is released under the permissible Apache 2.
 - **High Performance**: Built with Rust to ensure maximum speed and resource efficiency.
 - **Distributed Architecture**: Scalable and fault-tolerant design suitable for large-scale deployments.
 - **S3 Compatibility**: Seamless integration with existing S3-compatible applications and tools.
+- **OpenStack Swift API**: Native support for Swift protocol with Keystone authentication.
 - **OpenStack Keystone Integration**: Native support for OpenStack Keystone authentication with X-Auth-Token headers.
 - **Data Lake Support**: Optimized for high-throughput big data and AI workloads.
 - **Open Source**: Licensed under Apache 2.0, encouraging unrestricted community contributions and commercial usage.
@@ -56,6 +57,7 @@ Unlike other storage systems, RustFS is released under the permissible Apache 2.
 | **Event Notifications** | ✅ Available | **Distributed Mode**     | 🚧 Under Testing |
 | **K8s Helm Charts**     | ✅ Available | **RustFS KMS**           | 🚧 Under Testing |
 | **Keystone Auth**       | ✅ Available | **Multi-Tenancy**        | ✅ Available     |
+| **Swift API**           | ✅ Available | **Swift Metadata Ops**   | 🚧 Partial       |
 
 ## RustFS vs MinIO Performance
 
@@ -193,6 +195,19 @@ nix build
 nix run
 ```
 
+### 6\. X-CMD (Option 6)
+
+If you are an [x-cmd](https://www.x-cmd.com/install/rustfs) user:
+
+```bash
+# Run directly without installing
+x rustfs
+
+# Download the binary and install it to the global environment
+x env use rustfs
+rustfs --help
+```
+
 ---
 
 ### Accessing RustFS

README_ZH.md

@@ -100,7 +100,7 @@ curl -O https://rustfs.com/install_rustfs.sh && bash install_rustfs.sh
 
 ### 2\. Docker 快速启动 (选项 2)
 
-RustFS 容器以非 root 用户 `rustfs` (UID `10001`) 运行。如果您使用 Docker 的 `-v` 参数挂载宿主机目录，请务必确保宿主机目录的所有者已更改为 `1000`，否则会遇到权限拒绝错误。
+RustFS 容器以非 root 用户 `rustfs` (UID `10001`) 运行。如果您使用 Docker 的 `-v` 参数挂载宿主机目录，请务必确保宿主机目录的所有者已更改为 `10001`，否则会遇到权限拒绝错误。
 
 ```bash
  # 创建数据和日志目录
@@ -163,6 +163,36 @@ make help-docker                      # 显示所有 Docker 相关命令
 
 请按照 [Helm Chart README](https://charts.rustfs.com) 上的说明在 Kubernetes 集群上安装 RustFS。
 
+### 5\. Nix Flake (Option 5)
+
+如果你已经 启用了 [Nix Flakes 功能](https://nixos.wiki/wiki/Flakes#Enable_flakes):
+
+```bash
+# 直接运行，无需安装
+nix run github:rustfs/rustfs
+
+# 编译二进制文件
+nix build github:rustfs/rustfs
+./result/bin/rustfs --help
+
+# 或者从本地检出的代码库运行/编译
+nix build
+nix run
+```
+
+### 6\. X-CMD (Option 6)
+
+如果你是 [x-cmd](https://www.x-cmd.com/install/rustfs) 用户：
+
+```bash
+# 直接运行，无需安装
+x rustfs
+
+# 下载二进制文件并安装到全局环境中
+x env use rustfs
+rustfs --help
+```
+
 ---
 
 ### 访问 RustFS

_typos.toml

@@ -39,9 +39,12 @@ abd = "abd"
 mak = "mak"
 gae = "gae"
 GAE = "GAE"
+writeable = "writeable"
 # s3-tests original test names (cannot be changed)
 nonexisted = "nonexisted"
 consts = "consts"
+# Swift API - company/product names
+Hashi = "Hashi"  # HashiCorp
 
 [files]
 extend-exclude = []

build-rustfs.sh

@@ -213,7 +213,7 @@ setup_rust_environment() {
     # Set up environment variables for musl targets
     if [[ "$PLATFORM" == *"musl"* ]]; then
         print_message $YELLOW "Setting up environment for musl target..."
-        export RUSTFLAGS="-C target-feature=-crt-static"
+        export RUSTFLAGS="'--cfg tokio_unstable -C target-feature=-crt-static'"
 
         # For cargo-zigbuild, set up additional environment variables
         if command -v cargo-zigbuild &> /dev/null; then
@@ -430,7 +430,7 @@ build_binary() {
         fi
     else
         # Native compilation
-        build_cmd="RUSTFLAGS=-Clink-arg=-lm cargo build"
+        build_cmd="RUSTFLAGS='--cfg tokio_unstable -Clink-arg=-lm' cargo build"
     fi
 
     if [ "$BUILD_TYPE" = "release" ]; then

crates/AGENTS.md

@@ -0,0 +1,20 @@
+# Crates Instructions
+
+Applies to all paths under `crates/`.
+
+## Library Design
+
+- Treat crate code as reusable library code by default.
+- Prefer `thiserror` for library-facing error types.
+- Do not use `unwrap()`, `expect()`, or panic-driven control flow outside tests.
+
+## Testing
+
+- Keep unit tests close to the module they test.
+- Keep integration tests under each crate's `tests/` directory.
+- Add regression tests for bug fixes and behavior changes.
+
+## Async and Performance
+
+- Keep async paths non-blocking.
+- Move CPU-heavy operations out of async hot paths with `tokio::task::spawn_blocking` when appropriate.

crates/appauth/Cargo.toml

@@ -33,3 +33,6 @@ rand.workspace = true
 
 [lints]
 workspace = true
+
+[lib]
+doctest = false

crates/audit/Cargo.toml

@@ -29,6 +29,7 @@ categories = ["web-programming", "development-tools", "asynchronous", "api-bindi
 rustfs-targets = { workspace = true }
 rustfs-config = { workspace = true, features = ["audit", "constants"] }
 rustfs-ecstore = { workspace = true }
+rustfs-s3-common = { workspace = true }
 async-trait = { workspace = true }
 chrono = { workspace = true }
 const-str = { workspace = true }
@@ -43,6 +44,12 @@ tracing = { workspace = true, features = ["std", "attributes"] }
 url = { workspace = true }
 rumqttc = { workspace = true }
 
+[dev-dependencies]
+temp-env = { workspace = true }
 
 [lints]
 workspace = true
+
+[lib]
+test = false
+doctest = false

crates/audit/src/entity.rs

@@ -14,7 +14,7 @@
 
 use chrono::{DateTime, Utc};
 use hashbrown::HashMap;
-use rustfs_targets::EventName;
+use rustfs_s3_common::EventName;
 use serde::{Deserialize, Serialize};
 use serde_json::Value;
 

crates/audit/src/factory.rs

@@ -19,7 +19,8 @@ use rumqttc::QoS;
 use rustfs_config::audit::{AUDIT_MQTT_KEYS, AUDIT_WEBHOOK_KEYS, ENV_AUDIT_MQTT_KEYS, ENV_AUDIT_WEBHOOK_KEYS};
 use rustfs_config::{
     AUDIT_DEFAULT_DIR, DEFAULT_LIMIT, MQTT_BROKER, MQTT_KEEP_ALIVE_INTERVAL, MQTT_PASSWORD, MQTT_QOS, MQTT_QUEUE_DIR,
-    MQTT_QUEUE_LIMIT, MQTT_RECONNECT_INTERVAL, MQTT_TOPIC, MQTT_USERNAME, RUSTFS_WEBHOOK_SKIP_TLS_VERIFY_DEFAULT,
+    MQTT_QUEUE_LIMIT, MQTT_RECONNECT_INTERVAL, MQTT_TLS_CA, MQTT_TLS_CLIENT_CERT, MQTT_TLS_CLIENT_KEY, MQTT_TLS_POLICY,
+    MQTT_TLS_TRUST_LEAF_AS_CA, MQTT_TOPIC, MQTT_USERNAME, MQTT_WS_PATH_ALLOWLIST, RUSTFS_WEBHOOK_SKIP_TLS_VERIFY_DEFAULT,
     WEBHOOK_AUTH_TOKEN, WEBHOOK_CLIENT_CA, WEBHOOK_CLIENT_CERT, WEBHOOK_CLIENT_KEY, WEBHOOK_ENDPOINT, WEBHOOK_QUEUE_DIR,
     WEBHOOK_QUEUE_LIMIT, WEBHOOK_SKIP_TLS_VERIFY,
 };
@@ -27,7 +28,10 @@ use rustfs_ecstore::config::KVS;
 use rustfs_targets::{
     Target,
     error::TargetError,
-    target::{mqtt::MQTTArgs, webhook::WebhookArgs},
+    target::{
+        mqtt::{MQTTArgs, MQTTTlsConfig, validate_mqtt_broker_url},
+        webhook::WebhookArgs,
+    },
 };
 use std::time::Duration;
 use tracing::{debug, warn};
@@ -166,6 +170,14 @@ impl TargetFactory for MQTTTargetFactory {
                 .and_then(|v| v.parse::<u64>().ok())
                 .map(Duration::from_secs)
                 .unwrap_or_else(|| Duration::from_secs(30)),
+            tls: MQTTTlsConfig::from_values(
+                config.lookup(MQTT_TLS_POLICY).as_deref(),
+                config.lookup(MQTT_TLS_CA).as_deref(),
+                config.lookup(MQTT_TLS_CLIENT_CERT).as_deref(),
+                config.lookup(MQTT_TLS_CLIENT_KEY).as_deref(),
+                config.lookup(MQTT_TLS_TRUST_LEAF_AS_CA).as_deref(),
+                config.lookup(MQTT_WS_PATH_ALLOWLIST).as_deref(),
+            )?,
             queue_dir: config.lookup(MQTT_QUEUE_DIR).unwrap_or(AUDIT_DEFAULT_DIR.to_string()),
             queue_limit: config
                 .lookup(MQTT_QUEUE_LIMIT)
@@ -185,12 +197,15 @@ impl TargetFactory for MQTTTargetFactory {
         let url = Url::parse(&broker)
             .map_err(|e| TargetError::Configuration(format!("Invalid broker URL: {e} (value: '{broker}')")))?;
 
-        match url.scheme() {
-            "tcp" | "ssl" | "ws" | "wss" | "mqtt" | "mqtts" => {}
-            _ => {
-                return Err(TargetError::Configuration("Unsupported broker URL scheme".to_string()));
-            }
-        }
+        let tls = MQTTTlsConfig::from_values(
+            config.lookup(MQTT_TLS_POLICY).as_deref(),
+            config.lookup(MQTT_TLS_CA).as_deref(),
+            config.lookup(MQTT_TLS_CLIENT_CERT).as_deref(),
+            config.lookup(MQTT_TLS_CLIENT_KEY).as_deref(),
+            config.lookup(MQTT_TLS_TRUST_LEAF_AS_CA).as_deref(),
+            config.lookup(MQTT_WS_PATH_ALLOWLIST).as_deref(),
+        )?;
+        validate_mqtt_broker_url(&url, &tls)?;
 
         if config.lookup(MQTT_TOPIC).is_none() {
             return Err(TargetError::Configuration("Missing MQTT topic".to_string()));

crates/audit/src/registry.rs

@@ -109,9 +109,6 @@ impl AuditRegistry {
         let all_env: Vec<(String, String)> = std::env::vars().filter(|(key, _)| key.starts_with(ENV_PREFIX)).collect();
         // A collection of asynchronous tasks for concurrently executing target creation
         let mut tasks = FuturesUnordered::new();
-        // let final_config = config.clone(); // Clone a configuration for aggregating the final result
-        // Record the defaults for each segment so that the segment can eventually be rebuilt
-        let mut section_defaults: HashMap<String, KVS> = HashMap::new();
         // 1. Traverse all registered plants and process them by target type
         for (target_type, factory) in &self.factories {
             tracing::Span::current().record("target_type", target_type.as_str());
@@ -125,9 +122,6 @@ impl AuditRegistry {
             let default_cfg = file_configs.get(DEFAULT_DELIMITER).cloned().unwrap_or_default();
             debug!(?default_cfg, "Get the default configuration");
 
-            // Save defaults for eventual write back
-            section_defaults.insert(section_name.clone(), default_cfg.clone());
-
             // *** Optimization point 1: Get all legitimate fields of the current target type ***
             let valid_fields = factory.get_valid_fields();
             debug!(?valid_fields, "Get the legitimate configuration fields");
@@ -235,97 +229,28 @@ impl AuditRegistry {
                 if enabled {
                     info!(instance_id = %id, "Target is enabled, ready to create a task");
                     // 5.3. Create asynchronous tasks for enabled instances
-                    let target_type_clone = target_type.clone();
                     let tid = id.clone();
                     let merged_config_arc = Arc::new(merged_config);
                     tasks.push(async move {
                         let result = factory.create_target(tid.clone(), &merged_config_arc).await;
-                        (target_type_clone, tid, result, Arc::clone(&merged_config_arc))
+                        (tid, result)
                     });
                 } else {
-                    info!(instance_id = %id, "Skip the disabled target and will be removed from the final configuration");
-                    // Remove disabled target from final configuration
-                    // final_config.0.entry(section_name.clone()).or_default().remove(&id);
+                    info!(instance_id = %id, "Skip disabled target");
                 }
             }
         }
 
         // 6. Concurrently execute all creation tasks and collect results
         let mut successful_targets = Vec::new();
-        let mut successful_configs = Vec::new();
-        while let Some((target_type, id, result, final_config)) = tasks.next().await {
+        while let Some((id, result)) = tasks.next().await {
             match result {
                 Ok(target) => {
-                    info!(target_type = %target_type, instance_id = %id, "Create a target successfully");
+                    info!(target_type = %target.id().name, instance_id = %id, "Create a target successfully");
                     successful_targets.push(target);
-                    successful_configs.push((target_type, id, final_config));
                 }
                 Err(e) => {
-                    error!(target_type = %target_type, instance_id = %id, error = %e, "Failed to create a target");
-                }
-            }
-        }
-
-        // 7. Aggregate new configuration and write back to system configuration
-        if !successful_configs.is_empty() || !section_defaults.is_empty() {
-            info!(
-                "Prepare to update {} successfully created target configurations to the system configuration...",
-                successful_configs.len()
-            );
-
-            let mut successes_by_section: HashMap<String, HashMap<String, KVS>> = HashMap::new();
-
-            for (target_type, id, kvs) in successful_configs {
-                let section_name = format!("{AUDIT_ROUTE_PREFIX}{target_type}").to_lowercase();
-                successes_by_section
-                    .entry(section_name)
-                    .or_default()
-                    .insert(id.to_lowercase(), (*kvs).clone());
-            }
-
-            let mut new_config = config.clone();
-            // Collection of segments that need to be processed: Collect all segments where default items exist or where successful instances exist
-            let mut sections: HashSet<String> = HashSet::new();
-            sections.extend(section_defaults.keys().cloned());
-            sections.extend(successes_by_section.keys().cloned());
-
-            for section in sections {
-                let mut section_map: std::collections::HashMap<String, KVS> = std::collections::HashMap::new();
-                // Add default item
-                if let Some(default_kvs) = section_defaults.get(&section)
-                    && !default_kvs.is_empty()
-                {
-                    section_map.insert(DEFAULT_DELIMITER.to_string(), default_kvs.clone());
-                }
-
-                // Add successful instance item
-                if let Some(instances) = successes_by_section.get(&section) {
-                    for (id, kvs) in instances {
-                        section_map.insert(id.clone(), kvs.clone());
-                    }
-                }
-
-                // Empty breaks are removed and non-empty breaks are replaced entirely.
-                if section_map.is_empty() {
-                    new_config.0.remove(&section);
-                } else {
-                    new_config.0.insert(section, section_map);
-                }
-            }
-
-            let Some(store) = rustfs_ecstore::global::new_object_layer_fn() else {
-                return Err(AuditError::StorageNotAvailable(
-                    "Failed to save target configuration: server storage not initialized".to_string(),
-                ));
-            };
-
-            match rustfs_ecstore::config::com::save_server_config(store, &new_config).await {
-                Ok(_) => {
-                    info!("The new configuration was saved to the system successfully.")
-                }
-                Err(e) => {
-                    error!("Failed to save the new configuration: {}", e);
-                    return Err(AuditError::SaveConfig(Box::new(e)));
+                    error!(instance_id = %id, error = %e, "Failed to create a target");
                 }
             }
         }
@@ -365,6 +290,11 @@ impl AuditRegistry {
         self.targets.get(id).map(|t| t.as_ref())
     }
 
+    /// Lists cloned target values for runtime inspection without exposing mutable registry access.
+    pub fn list_target_values(&self) -> Vec<Box<dyn Target<AuditEntry> + Send + Sync>> {
+        self.targets.values().map(|target| target.clone_dyn()).collect()
+    }
+
     /// Lists all target IDs
     ///
     /// # Returns

crates/audit/src/system.rs

@@ -13,14 +13,15 @@
 //  limitations under the License.
 
 use crate::{AuditEntry, AuditError, AuditRegistry, AuditResult, observability};
+use hashbrown::HashMap;
 use rustfs_ecstore::config::Config;
 use rustfs_targets::{
     StoreError, Target, TargetError,
     store::{Key, Store},
-    target::EntityTarget,
+    target::{EntityTarget, QueuedPayload},
 };
 use std::sync::Arc;
-use tokio::sync::{Mutex, RwLock};
+use tokio::sync::{Mutex, RwLock, mpsc};
 use tracing::{error, info, warn};
 
 /// State of the audit system
@@ -39,6 +40,8 @@ pub struct AuditSystem {
     registry: Arc<Mutex<AuditRegistry>>,
     state: Arc<RwLock<AuditSystemState>>,
     config: Arc<RwLock<Option<Config>>>,
+    /// Cancellation senders for active audit stream tasks (target_id -> cancel tx)
+    stream_cancellers: Arc<RwLock<HashMap<String, mpsc::Sender<()>>>>,
 }
 
 impl Default for AuditSystem {
@@ -54,6 +57,7 @@ impl AuditSystem {
             registry: Arc::new(Mutex::new(AuditRegistry::new())),
             state: Arc::new(RwLock::new(AuditSystemState::Stopped)),
             config: Arc::new(RwLock::new(None)),
+            stream_cancellers: Arc::new(RwLock::new(HashMap::new())),
         }
     }
 
@@ -110,27 +114,7 @@ impl AuditSystem {
 
                 // Initialize all targets
                 for target in targets {
-                    let target_id = target.id().to_string();
-                    if let Err(e) = target.init().await {
-                        error!(target_id = %target_id, error = %e, "Failed to initialize audit target");
-                    } else {
-                        // After successful initialization, if enabled and there is a store, start the send from storage task
-                        if target.is_enabled() {
-                            if let Some(store) = target.store() {
-                                info!(target_id = %target_id, "Start audit stream processing for target");
-                                let store_clone: Box<dyn Store<EntityTarget<AuditEntry>, Error = StoreError, Key = Key> + Send> =
-                                    store.boxed_clone();
-                                let target_arc: Arc<dyn Target<AuditEntry> + Send + Sync> = Arc::from(target.clone_dyn());
-                                self.start_audit_stream_with_batching(store_clone, target_arc);
-                                info!(target_id = %target_id, "Audit stream processing started");
-                            } else {
-                                info!(target_id = %target_id, "No store configured, skip audit stream processing");
-                            }
-                        } else {
-                            info!(target_id = %target_id, "Target disabled, skip audit stream processing");
-                        }
-                        registry.add_target(target_id, target);
-                    }
+                    self.init_and_register_target(target, &mut registry).await;
                 }
 
                 // Update state to running
@@ -214,6 +198,9 @@ impl AuditSystem {
 
         info!("Stopping audit system");
 
+        // Stop all stream tasks first
+        self.stop_all_streams().await;
+
         // Close all targets
         let mut registry = self.registry.lock().await;
         if let Err(e) = registry.close_all().await {
@@ -258,52 +245,49 @@ impl AuditSystem {
         let state = self.state.read().await;
 
         match *state {
-            AuditSystemState::Running => {
-                // Continue with dispatch
-                info!("Dispatching audit log entry");
-            }
+            AuditSystemState::Running => {}
             AuditSystemState::Paused => {
-                // Skip dispatch when paused
                 return Ok(());
             }
             _ => {
-                // Don't dispatch when not running
                 return Err(AuditError::NotInitialized("Audit system is not running".to_string()));
             }
         }
         drop(state);
 
-        let registry = self.registry.lock().await;
-        let target_keys = registry.list_targets();
+        // Collect cloned targets under lock, then dispatch without holding it
+        let targets: Vec<(String, Box<dyn Target<AuditEntry> + Send + Sync>)> = {
+            let registry = self.registry.lock().await;
+            let target_keys = registry.list_targets();
 
-        if target_keys.is_empty() {
-            warn!("No audit targets configured for dispatch");
-            return Ok(());
-        }
+            if target_keys.is_empty() {
+                warn!("No audit targets configured for dispatch");
+                return Ok(());
+            }
 
-        // Dispatch to all targets concurrently
+            target_keys
+                .into_iter()
+                .filter_map(|key| registry.get_target(&key).map(|t| (key, t.clone_dyn())))
+                .collect()
+        };
+
+        // Dispatch to all targets concurrently (no lock held)
         let mut tasks = Vec::new();
 
-        for target_key in target_keys {
-            if let Some(target) = registry.get_target(&target_key) {
-                let entry_clone = Arc::clone(&entry);
-                let target_key_clone = target_key.clone();
+        for (target_key, target) in targets {
+            let entity_target = EntityTarget {
+                object_name: entry.api.name.clone().unwrap_or_default(),
+                bucket_name: entry.api.bucket.clone().unwrap_or_default(),
+                event_name: entry.event,
+                data: (*entry).clone(),
+            };
 
-                // Create EntityTarget for the audit log entry
-                let entity_target = EntityTarget {
-                    object_name: entry.api.name.clone().unwrap_or_default(),
-                    bucket_name: entry.api.bucket.clone().unwrap_or_default(),
-                    event_name: entry.event, // Default, should be derived from entry
-                    data: (*entry_clone).clone(),
-                };
+            let task = async move {
+                let result = target.save(Arc::new(entity_target)).await;
+                (target_key, result)
+            };
 
-                let task = async move {
-                    let result = target.save(Arc::new(entity_target)).await;
-                    (target_key_clone, result)
-                };
-
-                tasks.push(task);
-            }
+            tasks.push(task);
         }
 
         // Execute all dispatch tasks
@@ -359,39 +343,45 @@ impl AuditSystem {
         }
         drop(state);
 
-        let registry = self.registry.lock().await;
-        let target_keys = registry.list_targets();
+        // Collect targets under lock, then dispatch without holding it
+        let targets: Vec<(String, Box<dyn Target<AuditEntry> + Send + Sync>)> = {
+            let registry = self.registry.lock().await;
+            let target_keys = registry.list_targets();
 
-        if target_keys.is_empty() {
-            warn!("No audit targets configured for batch dispatch");
-            return Ok(());
-        }
+            if target_keys.is_empty() {
+                warn!("No audit targets configured for batch dispatch");
+                return Ok(());
+            }
+
+            target_keys
+                .into_iter()
+                .filter_map(|key| registry.get_target(&key).map(|t| (key, t.clone_dyn())))
+                .collect()
+        };
 
         let mut tasks = Vec::new();
-        for target_key in target_keys {
-            if let Some(target) = registry.get_target(&target_key) {
-                let entries_clone: Vec<_> = entries.iter().map(Arc::clone).collect();
-                let target_key_clone = target_key.clone();
+        for (target_key, target) in targets {
+            let entries_clone: Vec<_> = entries.iter().map(Arc::clone).collect();
+            let target_key_clone = target_key.clone();
 
-                let task = async move {
-                    let mut success_count = 0;
-                    let mut errors = Vec::new();
-                    for entry in entries_clone {
-                        let entity_target = EntityTarget {
-                            object_name: entry.api.name.clone().unwrap_or_default(),
-                            bucket_name: entry.api.bucket.clone().unwrap_or_default(),
-                            event_name: entry.event,
-                            data: (*entry).clone(),
-                        };
-                        match target.save(Arc::new(entity_target)).await {
-                            Ok(_) => success_count += 1,
-                            Err(e) => errors.push(e),
-                        }
+            let task = async move {
+                let mut success_count = 0;
+                let mut errors = Vec::new();
+                for entry in entries_clone {
+                    let entity_target = EntityTarget {
+                        object_name: entry.api.name.clone().unwrap_or_default(),
+                        bucket_name: entry.api.bucket.clone().unwrap_or_default(),
+                        event_name: entry.event,
+                        data: (*entry).clone(),
+                    };
+                    match target.save(Arc::new(entity_target)).await {
+                        Ok(_) => success_count += 1,
+                        Err(e) => errors.push(e),
                     }
-                    (target_key_clone, success_count, errors)
-                };
-                tasks.push(task);
-            }
+                }
+                (target_key_clone, success_count, errors)
+            };
+            tasks.push(task);
         }
 
         let results = futures::future::join_all(tasks).await;
@@ -417,6 +407,60 @@ impl AuditSystem {
         Ok(())
     }
 
+    /// Stops all active audit stream tasks by sending cancellation signals.
+    async fn stop_all_streams(&self) {
+        let mut cancellers = self.stream_cancellers.write().await;
+        for (target_id, cancel_tx) in cancellers.drain() {
+            info!(target_id = %target_id, "Stopping audit stream");
+            let _ = cancel_tx.send(()).await;
+        }
+    }
+
+    /// Initializes a single target: runs init(), starts stream if store is present,
+    /// and adds it to the registry. For store-backed targets, registration and stream
+    /// startup proceed even if init() fails so queued entries can be drained later.
+    async fn init_and_register_target(
+        &self,
+        target: Box<dyn Target<AuditEntry> + Send + Sync>,
+        registry: &mut AuditRegistry,
+    ) -> Option<String> {
+        let target_id = target.id().to_string();
+        let has_store = target.store().is_some();
+
+        if let Err(e) = target.init().await {
+            error!(target_id = %target_id, error = %e, "Failed to initialize audit target");
+            // Non-store targets: init failure is fatal.
+            if !has_store {
+                return None;
+            }
+            // Store-backed targets: still register and start the stream so queued
+            // entries can be drained when connectivity recovers.
+            warn!(
+                target_id = %target_id,
+                "Proceeding with store-backed audit target despite init failure"
+            );
+        }
+
+        if target.is_enabled() {
+            if let Some(store) = target.store() {
+                info!(target_id = %target_id, "Start audit stream processing for target");
+                let store_clone: Box<dyn Store<QueuedPayload, Error = StoreError, Key = Key> + Send> = store.boxed_clone();
+                let target_arc: Arc<dyn Target<AuditEntry> + Send + Sync> = Arc::from(target.clone_dyn());
+                let cancel_tx = self.start_audit_stream_with_batching(store_clone, target_arc);
+
+                self.stream_cancellers.write().await.insert(target_id.clone(), cancel_tx);
+                info!(target_id = %target_id, "Audit stream processing started");
+            } else {
+                info!(target_id = %target_id, "No store configured, skip audit stream processing");
+            }
+        } else {
+            info!(target_id = %target_id, "Target disabled, skip audit stream processing");
+        }
+
+        registry.add_target(target_id.clone(), target);
+        Some(target_id)
+    }
+
     /// Starts the audit stream processing for a target with batching and retry logic
     ///
     /// # Arguments
@@ -427,9 +471,10 @@ impl AuditSystem {
     /// and attempts to send them to the specified target. It implements retry logic with exponential backoff
     fn start_audit_stream_with_batching(
         &self,
-        store: Box<dyn Store<EntityTarget<AuditEntry>, Error = StoreError, Key = Key> + Send>,
+        store: Box<dyn Store<QueuedPayload, Error = StoreError, Key = Key> + Send>,
         target: Arc<dyn Target<AuditEntry> + Send + Sync>,
-    ) {
+    ) -> mpsc::Sender<()> {
+        let (cancel_tx, mut cancel_rx) = mpsc::channel(1);
         let state = self.state.clone();
 
         tokio::spawn(async move {
@@ -442,6 +487,12 @@ impl AuditSystem {
             const BASE_RETRY_DELAY: Duration = Duration::from_secs(2);
 
             loop {
+                // Check for cancellation signal
+                if cancel_rx.try_recv().is_ok() {
+                    info!("Audit stream cancelled for target: {}", target.id());
+                    break;
+                }
+
                 match *state.read().await {
                     AuditSystemState::Running | AuditSystemState::Paused | AuditSystemState::Starting => {}
                     _ => {
@@ -452,11 +503,22 @@ impl AuditSystem {
 
                 let keys: Vec<Key> = store.list();
                 if keys.is_empty() {
-                    sleep(Duration::from_millis(500)).await;
+                    tokio::select! {
+                        _ = sleep(Duration::from_millis(500)) => {},
+                        _ = cancel_rx.recv() => {
+                            info!("Audit stream cancelled during idle for target: {}", target.id());
+                            return;
+                        }
+                    }
                     continue;
                 }
 
                 for key in keys {
+                    if cancel_rx.try_recv().is_ok() {
+                        info!("Audit stream cancelled during processing for target: {}", target.id());
+                        return;
+                    }
+
                     let mut retries = 0usize;
                     let mut success = false;
 
@@ -497,6 +559,8 @@ impl AuditSystem {
                 sleep(Duration::from_millis(100)).await;
             }
         });
+
+        cancel_tx
     }
 
     /// Enables a specific target
@@ -594,6 +658,12 @@ impl AuditSystem {
         registry.list_targets()
     }
 
+    /// Returns cloned target values for read-only runtime inspection.
+    pub async fn get_target_values(&self) -> Vec<Box<dyn Target<AuditEntry> + Send + Sync>> {
+        let registry = self.registry.lock().await;
+        registry.list_target_values()
+    }
+
     /// Gets information about a specific target
     ///
     /// # Arguments
@@ -616,9 +686,11 @@ impl AuditSystem {
     pub async fn reload_config(&self, new_config: Config) -> AuditResult<()> {
         info!("Reloading audit system configuration");
 
-        // Record config reload
         observability::record_config_reload();
 
+        // Stop all existing stream tasks first
+        self.stop_all_streams().await;
+
         // Store new configuration
         {
             let mut config_guard = self.config.write().await;
@@ -636,28 +708,9 @@ impl AuditSystem {
             Ok(targets) => {
                 info!(target_count = targets.len(), "Reloaded audit targets successfully");
 
-                // Initialize all new targets
                 for target in targets {
-                    let target_id = target.id().to_string();
-                    if let Err(e) = target.init().await {
-                        error!(target_id = %target_id, error = %e, "Failed to initialize reloaded audit target");
-                    } else {
-                        // Same starts the storage stream after a heavy load
-                        if target.is_enabled() {
-                            if let Some(store) = target.store() {
-                                info!(target_id = %target_id, "Start audit stream processing for target (reload)");
-                                let store_clone: Box<dyn Store<EntityTarget<AuditEntry>, Error = StoreError, Key = Key> + Send> =
-                                    store.boxed_clone();
-                                let target_arc: Arc<dyn Target<AuditEntry> + Send + Sync> = Arc::from(target.clone_dyn());
-                                self.start_audit_stream_with_batching(store_clone, target_arc);
-                                info!(target_id = %target_id, "Audit stream processing started (reload)");
-                            } else {
-                                info!(target_id = %target_id, "No store configured, skip audit stream processing (reload)");
-                            }
-                        } else {
-                            info!(target_id = %target_id, "Target disabled, skip audit stream processing (reload)");
-                        }
-                        registry.add_target(target.id().to_string(), target);
+                    if let Some(target_id) = self.init_and_register_target(target, &mut registry).await {
+                        info!(target_id = %target_id, "Target initialized (reload)");
                     }
                 }
 

crates/audit/tests/config_parsing_test.rs

@@ -53,6 +53,12 @@ fn test_mqtt_valid_fields() {
         "reconnect_interval",
         "queue_dir",
         "queue_limit",
+        "tls_policy",
+        "tls_ca",
+        "tls_client_cert",
+        "tls_client_key",
+        "tls_trust_leaf_as_ca",
+        "ws_path_allowlist",
     ];
 
     // This tests the MQTT configuration fields we support

crates/audit/tests/integration_test.rs

@@ -15,6 +15,7 @@
 use rustfs_audit::*;
 use rustfs_ecstore::config::{Config, KVS};
 use std::collections::HashMap;
+use temp_env::with_vars;
 
 #[tokio::test]
 async fn test_audit_system_creation() {
@@ -35,34 +36,42 @@ async fn test_config_parsing_webhook() {
     let mut config = Config(HashMap::new());
     let mut audit_webhook_section = HashMap::new();
 
-    // Create default configuration
     let mut default_kvs = KVS::new();
-    default_kvs.insert("enable".to_string(), "on".to_string());
-    default_kvs.insert("endpoint".to_string(), "http://localhost:3020/webhook".to_string());
-
+    default_kvs.insert("enable".to_string(), "off".to_string());
+    default_kvs.insert("endpoint".to_string(), "".to_string());
     audit_webhook_section.insert("_".to_string(), default_kvs);
+    let mut instance_kvs = KVS::new();
+    instance_kvs.insert("enable".to_string(), "on".to_string());
+    instance_kvs.insert("endpoint".to_string(), "http://localhost:3020/webhook".to_string());
+    audit_webhook_section.insert("primary".to_string(), instance_kvs);
     config.0.insert("audit_webhook".to_string(), audit_webhook_section);
 
     let registry = AuditRegistry::new();
 
-    // This should not fail even if server storage is not initialized
-    // as it's an integration test
     let result = registry.create_audit_targets_from_config(&config).await;
+    assert!(result.is_ok(), "audit target creation should not require server storage");
+}
 
-    // We expect this to fail due to server storage not being initialized
-    // but the parsing should work correctly
-    match result {
-        Err(AuditError::StorageNotAvailable(_)) => {
-            // This is expected in test environment
-        }
-        Err(e) => {
-            // Other errors might indicate parsing issues
-            println!("Unexpected error: {e}");
-        }
-        Ok(_) => {
-            // Unexpected success in test environment without server storage
-        }
-    }
+#[test]
+fn test_env_only_audit_target_does_not_require_server_storage() {
+    with_vars(
+        [
+            ("RUSTFS_AUDIT_WEBHOOK_ENABLE_PRIMARY", Some("on")),
+            ("RUSTFS_AUDIT_WEBHOOK_ENDPOINT_PRIMARY", Some("http://localhost:3020/webhook")),
+        ],
+        || {
+            let runtime = tokio::runtime::Builder::new_current_thread()
+                .enable_all()
+                .build()
+                .expect("failed to create tokio runtime");
+            runtime.block_on(async {
+                let config = Config(HashMap::new());
+                let registry = AuditRegistry::new();
+                let result = registry.create_audit_targets_from_config(&config).await;
+                assert!(result.is_ok(), "env-only audit target creation should not require server storage");
+            });
+        },
+    )
 }
 
 #[test]

crates/audit/tests/performance_test.rs

@@ -159,7 +159,8 @@ async fn test_audit_log_dispatch_performance() {
 
     let start = Instant::now();
 
-    // Dispatch audit log (should be fast since no targets are configured)
+    // Dispatch audit log against an unstarted system state. Empty config keeps
+    // the audit system stopped, so dispatch should fail fast without targets.
     let result = system.dispatch(Arc::new(audit_entry)).await;
     let elapsed = start.elapsed();
 
@@ -168,8 +169,10 @@ async fn test_audit_log_dispatch_performance() {
     // Should be very fast (sub-millisecond for no targets)
     assert!(elapsed < Duration::from_millis(100), "Dispatch took too long: {elapsed:?}");
 
-    // Should succeed even with no targets
-    assert!(result.is_ok(), "Dispatch should succeed with no targets");
+    assert!(
+        matches!(result, Err(AuditError::NotInitialized(_))),
+        "Dispatch on a stopped system should return NotInitialized, got: {result:?}"
+    );
 
     // Clean up
     let _ = system.close().await;
@@ -186,11 +189,11 @@ async fn test_system_state_transitions() {
     let config = rustfs_ecstore::config::Config(std::collections::HashMap::new());
     let start_result = system.start(config).await;
 
-    // Should be running (or failed due to server storage)
+    // Empty config keeps the audit system stopped even when start() succeeds.
     let state = system.get_state().await;
     match start_result {
         Ok(_) => {
-            assert_eq!(state, rustfs_audit::system::AuditSystemState::Running);
+            assert_eq!(state, rustfs_audit::system::AuditSystemState::Stopped);
         }
         Err(_) => {
             // Expected in test environment due to server storage not being initialized

crates/audit/tests/system_integration_test.rs

@@ -29,27 +29,21 @@ async fn test_complete_audit_system_lifecycle() {
     assert_eq!(system.get_state().await, system::AuditSystemState::Stopped);
     assert!(!system.is_running().await);
 
-    // 2. Start with empty config (will fail due to no server storage in test)
+    // 2. Start with empty config. The current implementation returns Ok(())
+    // but keeps the system stopped when no audit targets are enabled.
     let config = Config(HashMap::new());
     let start_result = system.start(config).await;
 
-    // Should fail in test environment but state handling should work
+    // State handling should remain consistent for both empty-config success and
+    // storage-unavailable failure paths.
     match start_result {
         Err(AuditError::StorageNotAvailable(_)) => {
             // Expected in test environment
             assert_eq!(system.get_state().await, system::AuditSystemState::Stopped);
         }
         Ok(_) => {
-            // If it somehow succeeds, verify running state
-            assert_eq!(system.get_state().await, system::AuditSystemState::Running);
-            assert!(system.is_running().await);
-
-            // Test pause/resume
-            system.pause().await.expect("Should pause successfully");
-            assert_eq!(system.get_state().await, system::AuditSystemState::Paused);
-
-            system.resume().await.expect("Should resume successfully");
-            assert_eq!(system.get_state().await, system::AuditSystemState::Running);
+            assert_eq!(system.get_state().await, system::AuditSystemState::Stopped);
+            assert!(!system.is_running().await);
         }
         Err(e) => {
             panic!("Unexpected error: {e}");

crates/checksums/Cargo.toml

@@ -36,3 +36,6 @@ sha2 = { workspace = true }
 
 [dev-dependencies]
 pretty_assertions = { workspace = true }
+
+[lib]
+doctest = false

crates/checksums/src/lib.rs

@@ -349,7 +349,7 @@ mod tests {
     fn base64_encoded_checksum_to_hex_string(header_value: &HeaderValue) -> String {
         let decoded_checksum = base64::decode(header_value.to_str().unwrap()).unwrap();
         let decoded_checksum = decoded_checksum.into_iter().fold(String::new(), |mut acc, byte| {
-            write!(acc, "{byte:02X?}").expect("string will always be writeable");
+            write!(acc, "{byte:02X?}").expect("string will always be writable");
             acc
         });
 

crates/common/Cargo.toml

@@ -41,3 +41,6 @@ rmp-serde = { workspace = true }
 async-trait = { workspace = true }
 s3s = { workspace = true }
 tracing = { workspace = true }
+
+[lib]
+doctest = false

crates/common/src/data_usage.rs

@@ -458,10 +458,7 @@ impl DataUsageEntry {
         self.size += other.size;
 
         if let Some(o_rep) = &other.replication_stats {
-            if self.replication_stats.is_none() {
-                self.replication_stats = Some(ReplicationAllStats::default());
-            }
-            let s_rep = self.replication_stats.as_mut().unwrap();
+            let s_rep = self.replication_stats.get_or_insert_with(ReplicationAllStats::default);
             s_rep.targets.clear();
             s_rep.replica_size += o_rep.replica_size;
             s_rep.replica_count += o_rep.replica_count;
@@ -586,7 +583,7 @@ impl DataUsageCache {
                     return Some(root);
                 }
                 let mut flat = self.flatten(&root);
-                if flat.replication_stats.is_some() && flat.replication_stats.as_ref().unwrap().empty() {
+                if flat.replication_stats.as_ref().is_some_and(|stats| stats.empty()) {
                     flat.replication_stats = None;
                 }
                 Some(flat)
@@ -679,7 +676,9 @@ impl DataUsageCache {
         leaves.sort_by(|a, b| a.objects.cmp(&b.objects));
 
         while remove > 0 && !leaves.is_empty() {
-            let e = leaves.first().unwrap();
+            let Some(e) = leaves.first() else {
+                break;
+            };
             let candidate = e.path.clone();
             if candidate == *path && !compact_self {
                 break;
@@ -703,12 +702,9 @@ impl DataUsageCache {
     }
 
     pub fn total_children_rec(&self, path: &str) -> usize {
-        let root = self.find(path);
-
-        if root.is_none() {
+        let Some(root) = self.find(path) else {
             return 0;
-        }
-        let root = root.unwrap();
+        };
         if root.children.is_empty() {
             return 0;
         }
@@ -721,31 +717,36 @@ impl DataUsageCache {
     }
 
     pub fn merge(&mut self, o: &DataUsageCache) {
-        let mut existing_root = self.root();
-        let other_root = o.root();
-        if existing_root.is_none() && other_root.is_none() {
-            return;
-        }
-        if other_root.is_none() {
-            return;
-        }
-        if existing_root.is_none() {
+        let Some(mut existing_root) = self.root() else {
+            if o.root().is_none() {
+                return;
+            }
             *self = o.clone();
             return;
-        }
-        if o.info.last_update.gt(&self.info.last_update) {
+        };
+
+        let Some(other_root) = o.root() else {
+            return;
+        };
+
+        if o.info.last_update > self.info.last_update {
             self.info.last_update = o.info.last_update;
         }
 
-        existing_root.as_mut().unwrap().merge(other_root.as_ref().unwrap());
-        self.cache.insert(hash_path(&self.info.name).key(), existing_root.unwrap());
-        let e_hash = self.root_hash();
-        for key in other_root.as_ref().unwrap().children.iter() {
-            let entry = &o.cache[key];
+        existing_root.merge(&other_root);
+        self.cache.insert(hash_path(&self.info.name).key(), existing_root);
+
+        let root_hash = self.root_hash();
+        for key in other_root.children.iter() {
+            let Some(entry) = o.cache.get(key) else {
+                continue;
+            };
             let flat = o.flatten(entry);
-            let mut existing = self.cache[key].clone();
-            existing.merge(&flat);
-            self.replace_hashed(&DataUsageHash(key.clone()), &Some(e_hash.clone()), &existing);
+            if let Some(existing) = self.cache.get_mut(key) {
+                existing.merge(&flat);
+            } else {
+                self.replace_hashed(&DataUsageHash(key.clone()), &Some(root_hash.clone()), &flat);
+            }
         }
     }
 
@@ -1141,10 +1142,12 @@ impl DataUsageInfo {
         self.buckets_count = self.buckets_usage.len() as u64;
 
         // Update last update time
-        if let Some(other_update) = other.last_update
-            && (self.last_update.is_none() || other_update > self.last_update.unwrap())
-        {
-            self.last_update = Some(other_update);
+        if let Some(other_update) = other.last_update {
+            match self.last_update {
+                None => self.last_update = Some(other_update),
+                Some(self_update) if other_update > self_update => self.last_update = Some(other_update),
+                _ => {}
+            }
         }
     }
 }
@@ -1285,4 +1288,59 @@ mod tests {
         assert_eq!(summary1.total_size, 300);
         assert_eq!(summary1.versions, 15);
     }
+
+    #[test]
+    fn test_data_usage_cache_merge_adds_missing_child() {
+        let mut base = DataUsageCache::default();
+        base.info.name = "bucket".to_string();
+        base.replace("bucket", "", DataUsageEntry::default());
+
+        let mut other = DataUsageCache::default();
+        other.info.name = "bucket".to_string();
+        let child = DataUsageEntry {
+            size: 42,
+            ..Default::default()
+        };
+        other.replace("bucket/child", "bucket", child);
+
+        base.merge(&other);
+
+        let root = base.find("bucket").expect("root bucket should exist");
+        assert_eq!(root.size, 0);
+        let child_entry = base.find("bucket/child").expect("merged child should be added");
+        assert_eq!(child_entry.size, 42);
+    }
+
+    #[test]
+    fn test_data_usage_cache_merge_accumulates_existing_child() {
+        let mut base = DataUsageCache::default();
+        base.info.name = "bucket".to_string();
+        base.replace(
+            "bucket/child",
+            "bucket",
+            DataUsageEntry {
+                size: 10,
+                objects: 1,
+                ..Default::default()
+            },
+        );
+
+        let mut other = DataUsageCache::default();
+        other.info.name = "bucket".to_string();
+        other.replace(
+            "bucket/child",
+            "bucket",
+            DataUsageEntry {
+                size: 20,
+                objects: 2,
+                ..Default::default()
+            },
+        );
+
+        base.merge(&other);
+
+        let child_entry = base.find("bucket/child").expect("child should remain after merge");
+        assert_eq!(child_entry.size, 30);
+        assert_eq!(child_entry.objects, 3);
+    }
 }

crates/common/src/heal_channel.rs

@@ -296,12 +296,13 @@ type HealResponseSender = broadcast::Sender<HealChannelResponse>;
 static GLOBAL_HEAL_RESPONSE_SENDER: OnceLock<HealResponseSender> = OnceLock::new();
 
 /// Initialize global heal channel
-pub fn init_heal_channel() -> HealChannelReceiver {
+pub fn init_heal_channel() -> Result<HealChannelReceiver, &'static str> {
     let (tx, rx) = mpsc::unbounded_channel();
-    GLOBAL_HEAL_CHANNEL_SENDER
-        .set(tx)
-        .expect("Heal channel sender already initialized");
-    rx
+    if GLOBAL_HEAL_CHANNEL_SENDER.set(tx).is_ok() {
+        Ok(rx)
+    } else {
+        Err("Heal channel sender already initialized")
+    }
 }
 
 /// Get global heal channel sender

crates/common/src/internode_metrics.rs

@@ -0,0 +1,170 @@
+// Copyright 2024 RustFS Team
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+use metrics::{counter, gauge};
+use std::sync::{
+    Arc, LazyLock,
+    atomic::{AtomicU64, Ordering},
+};
+use std::time::{Duration, SystemTime, UNIX_EPOCH};
+
+#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)]
+pub struct InternodeMetricsSnapshot {
+    pub sent_bytes_total: u64,
+    pub recv_bytes_total: u64,
+    pub outgoing_requests_total: u64,
+    pub incoming_requests_total: u64,
+    pub errors_total: u64,
+    pub dial_errors_total: u64,
+    pub dial_avg_time_nanos: u64,
+    pub last_dial_unix_millis: u64,
+}
+
+#[derive(Debug, Default)]
+pub struct InternodeMetrics {
+    sent_bytes_total: AtomicU64,
+    recv_bytes_total: AtomicU64,
+    outgoing_requests_total: AtomicU64,
+    incoming_requests_total: AtomicU64,
+    errors_total: AtomicU64,
+    dial_errors_total: AtomicU64,
+    dial_total_time_nanos: AtomicU64,
+    dial_samples_total: AtomicU64,
+    last_dial_unix_millis: AtomicU64,
+}
+
+impl InternodeMetrics {
+    pub fn record_sent_bytes(&self, bytes: usize) {
+        let bytes = bytes as u64;
+        if bytes == 0 {
+            return;
+        }
+        self.sent_bytes_total.fetch_add(bytes, Ordering::Relaxed);
+        counter!("rustfs.internode.sent.bytes.total").increment(bytes);
+    }
+
+    pub fn record_recv_bytes(&self, bytes: usize) {
+        let bytes = bytes as u64;
+        if bytes == 0 {
+            return;
+        }
+        self.recv_bytes_total.fetch_add(bytes, Ordering::Relaxed);
+        counter!("rustfs.internode.recv.bytes.total").increment(bytes);
+    }
+
+    pub fn record_outgoing_request(&self) {
+        self.outgoing_requests_total.fetch_add(1, Ordering::Relaxed);
+        counter!("rustfs.internode.requests.outgoing.total").increment(1);
+    }
+
+    pub fn record_incoming_request(&self) {
+        self.incoming_requests_total.fetch_add(1, Ordering::Relaxed);
+        counter!("rustfs.internode.requests.incoming.total").increment(1);
+    }
+
+    pub fn record_error(&self) {
+        self.errors_total.fetch_add(1, Ordering::Relaxed);
+        counter!("rustfs.internode.errors.total").increment(1);
+    }
+
+    pub fn record_dial_result(&self, duration: Duration, success: bool) {
+        let elapsed_nanos = duration.as_nanos().min(u128::from(u64::MAX)) as u64;
+        self.dial_total_time_nanos.fetch_add(elapsed_nanos, Ordering::Relaxed);
+        let samples = self.dial_samples_total.fetch_add(1, Ordering::Relaxed) + 1;
+        let total = self.dial_total_time_nanos.load(Ordering::Relaxed);
+        gauge!("rustfs.internode.dial.avg_time.nanos").set(total as f64 / samples as f64);
+
+        if !success {
+            self.dial_errors_total.fetch_add(1, Ordering::Relaxed);
+            counter!("rustfs.internode.dial.errors.total").increment(1);
+        }
+
+        let now_ms = SystemTime::now()
+            .duration_since(UNIX_EPOCH)
+            .unwrap_or_default()
+            .as_millis()
+            .min(u128::from(u64::MAX)) as u64;
+        self.last_dial_unix_millis.store(now_ms, Ordering::Relaxed);
+    }
+
+    pub fn snapshot(&self) -> InternodeMetricsSnapshot {
+        let dial_samples_total = self.dial_samples_total.load(Ordering::Relaxed);
+        let dial_total_time_nanos = self.dial_total_time_nanos.load(Ordering::Relaxed);
+        let dial_avg_time_nanos = if dial_samples_total == 0 {
+            0
+        } else {
+            dial_total_time_nanos / dial_samples_total
+        };
+
+        InternodeMetricsSnapshot {
+            sent_bytes_total: self.sent_bytes_total.load(Ordering::Relaxed),
+            recv_bytes_total: self.recv_bytes_total.load(Ordering::Relaxed),
+            outgoing_requests_total: self.outgoing_requests_total.load(Ordering::Relaxed),
+            incoming_requests_total: self.incoming_requests_total.load(Ordering::Relaxed),
+            errors_total: self.errors_total.load(Ordering::Relaxed),
+            dial_errors_total: self.dial_errors_total.load(Ordering::Relaxed),
+            dial_avg_time_nanos,
+            last_dial_unix_millis: self.last_dial_unix_millis.load(Ordering::Relaxed),
+        }
+    }
+
+    #[doc(hidden)]
+    pub fn reset_for_test(&self) {
+        self.sent_bytes_total.store(0, Ordering::Relaxed);
+        self.recv_bytes_total.store(0, Ordering::Relaxed);
+        self.outgoing_requests_total.store(0, Ordering::Relaxed);
+        self.incoming_requests_total.store(0, Ordering::Relaxed);
+        self.errors_total.store(0, Ordering::Relaxed);
+        self.dial_errors_total.store(0, Ordering::Relaxed);
+        self.dial_total_time_nanos.store(0, Ordering::Relaxed);
+        self.dial_samples_total.store(0, Ordering::Relaxed);
+        self.last_dial_unix_millis.store(0, Ordering::Relaxed);
+    }
+}
+
+pub fn global_internode_metrics() -> &'static Arc<InternodeMetrics> {
+    static GLOBAL_INTERNODE_METRICS: LazyLock<Arc<InternodeMetrics>> = LazyLock::new(|| Arc::new(InternodeMetrics::default()));
+    &GLOBAL_INTERNODE_METRICS
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    #[test]
+    fn snapshot_reports_recorded_values() {
+        let metrics = global_internode_metrics();
+        metrics.reset_for_test();
+
+        metrics.record_sent_bytes(64);
+        metrics.record_recv_bytes(32);
+        metrics.record_outgoing_request();
+        metrics.record_incoming_request();
+        metrics.record_error();
+        metrics.record_dial_result(Duration::from_millis(9), true);
+        metrics.record_dial_result(Duration::from_millis(3), false);
+
+        let snapshot = metrics.snapshot();
+        assert_eq!(snapshot.sent_bytes_total, 64);
+        assert_eq!(snapshot.recv_bytes_total, 32);
+        assert_eq!(snapshot.outgoing_requests_total, 1);
+        assert_eq!(snapshot.incoming_requests_total, 1);
+        assert_eq!(snapshot.errors_total, 1);
+        assert_eq!(snapshot.dial_errors_total, 1);
+        assert_eq!(snapshot.dial_avg_time_nanos, 6_000_000);
+        assert!(snapshot.last_dial_unix_millis > 0);
+
+        metrics.reset_for_test();
+    }
+}

crates/common/src/last_minute.rs

@@ -155,10 +155,7 @@ impl LastMinuteLatency {
     }
 
     pub fn add(&mut self, t: &Duration) {
-        let sec = SystemTime::now()
-            .duration_since(UNIX_EPOCH)
-            .expect("Time went backwards")
-            .as_secs();
+        let sec = SystemTime::now().duration_since(UNIX_EPOCH).unwrap_or_default().as_secs();
         self.forward_to(sec);
         let win_idx = sec % 60;
         self.totals[win_idx as usize].add(t);
@@ -174,10 +171,7 @@ impl LastMinuteLatency {
 
     pub fn get_total(&mut self) -> AccElem {
         let mut res = AccElem::default();
-        let sec = SystemTime::now()
-            .duration_since(UNIX_EPOCH)
-            .expect("Time went backwards")
-            .as_secs();
+        let sec = SystemTime::now().duration_since(UNIX_EPOCH).unwrap_or_default().as_secs();
         self.forward_to(sec);
         for elem in self.totals.iter() {
             res.merge(elem);

crates/common/src/lib.rs

@@ -17,6 +17,7 @@ pub mod bucket_stats;
 pub mod data_usage;
 pub mod globals;
 pub mod heal_channel;
+pub mod internode_metrics;
 pub mod last_minute;
 pub mod metrics;
 mod readiness;

crates/common/src/metrics.rs

@@ -22,12 +22,12 @@ use std::{
     future::Future,
     pin::Pin,
     sync::{
-        Arc, OnceLock,
+        Arc, Mutex, OnceLock,
         atomic::{AtomicU64, Ordering},
     },
     time::{Duration, SystemTime},
 };
-use tokio::sync::{Mutex, RwLock};
+use tokio::sync::RwLock;
 
 #[derive(Debug, Clone, Copy, PartialEq, Eq)]
 pub enum IlmAction {
@@ -110,7 +110,7 @@ pub fn global_metrics() -> &'static Arc<Metrics> {
     GLOBAL_METRICS.get_or_init(|| Arc::new(Metrics::new()))
 }
 
-#[derive(Clone, Debug, PartialEq, PartialOrd)]
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
 pub enum Metric {
     // START Realtime metrics, that only records
     // last minute latencies and total operation count.
@@ -188,7 +188,6 @@ impl Metric {
         if index >= Self::Last as usize {
             return None;
         }
-        // Safe conversion using match instead of unsafe transmute
         match index {
             0 => Some(Self::ReadMetadata),
             1 => Some(Self::CheckMissing),
@@ -220,16 +219,48 @@ impl Metric {
     }
 }
 
-/// Thread-safe wrapper for LastMinuteLatency with atomic operations
-#[derive(Default)]
+// ---------------------------------------------------------------------------
+// LockedLastMinuteLatency
+// ---------------------------------------------------------------------------
+//
+// Uses std::sync::Mutex instead of tokio::sync::Mutex.
+//
+// Rationale: the critical section is a handful of integer additions inside
+// LastMinuteLatency::add_all / get_total — no I/O, no blocking syscalls, no
+// awaiting.  A std blocking mutex is cheaper (no task-wakeup overhead) and,
+// crucially, lets every caller be *synchronous*.  That eliminates the need to
+// spawn a background task just to record a duration, which was the only reason
+// tokio::spawn appeared in log/time/time_size/time_n/time_ilm.
+//
+// Note: vec![LockedLastMinuteLatency::default(); N] with the old Arc-based
+// Clone made every element share the *same* inner mutex — a latent bug where
+// all metrics slots wrote to one counter.  The new Clone creates a fresh
+// independent Mutex per element, matching the intent.
+
+/// Thread-safe wrapper for LastMinuteLatency backed by a std blocking mutex.
 pub struct LockedLastMinuteLatency {
+    // Arc so Clone is cheap *and* each cloned value stays independent (its own
+    // allocation).  We never hand out the Arc to two Metrics at once; the Arc
+    // is purely a convenience for the Clone impl below.
     latency: Arc<Mutex<LastMinuteLatency>>,
 }
 
+impl Default for LockedLastMinuteLatency {
+    fn default() -> Self {
+        Self::new()
+    }
+}
+
+// Produce a fresh, independent slot — *not* a shared alias.  This is what
+// vec![val; N] and #[derive(Clone)] on the parent struct both need.
 impl Clone for LockedLastMinuteLatency {
     fn clone(&self) -> Self {
+        let inner = match self.latency.lock() {
+            Ok(guard) => guard.clone(),
+            Err(poisoned) => poisoned.into_inner().clone(),
+        };
         Self {
-            latency: Arc::clone(&self.latency),
+            latency: Arc::new(Mutex::new(inner)),
         }
     }
 }
@@ -241,14 +272,13 @@ impl LockedLastMinuteLatency {
         }
     }
 
-    /// Add a duration measurement
-    pub async fn add(&self, duration: Duration) {
-        self.add_size(duration, 0).await;
+    /// Record a duration sample (no size).
+    pub fn add(&self, duration: Duration) {
+        self.add_size(duration, 0);
     }
 
-    /// Add a duration measurement with size
-    pub async fn add_size(&self, duration: Duration, size: u64) {
-        let mut latency = self.latency.lock().await;
+    /// Record a duration sample with an associated byte count.
+    pub fn add_size(&self, duration: Duration, size: u64) {
         let now = SystemTime::now()
             .duration_since(SystemTime::UNIX_EPOCH)
             .unwrap_or_default()
@@ -259,17 +289,27 @@ impl LockedLastMinuteLatency {
             total: duration.as_secs(),
             size,
         };
-        latency.add_all(now, &elem);
+
+        match self.latency.lock() {
+            Ok(mut guard) => guard.add_all(now, &elem),
+            Err(poisoned) => poisoned.into_inner().add_all(now, &elem),
+        }
     }
 
-    /// Get total accumulated metrics for the last minute
-    pub async fn total(&self) -> AccElem {
-        let mut latency = self.latency.lock().await;
-        latency.get_total()
+    /// Return accumulated totals for the last minute window.
+    pub fn total(&self) -> AccElem {
+        match self.latency.lock() {
+            Ok(mut latency) => latency.get_total(),
+            Err(poisoned) => poisoned.into_inner().get_total(),
+        }
     }
 }
 
-/// Current path tracker for monitoring active scan paths
+// ---------------------------------------------------------------------------
+// CurrentPathTracker — unchanged, still uses tokio::sync::RwLock because it
+// lives inside async path-update callbacks.
+// ---------------------------------------------------------------------------
+
 struct CurrentPathTracker {
     current_path: Arc<RwLock<String>>,
 }
@@ -290,17 +330,16 @@ impl CurrentPathTracker {
     }
 }
 
-/// Main scanner metrics structure
+// ---------------------------------------------------------------------------
+// Metrics
+// ---------------------------------------------------------------------------
+
 pub struct Metrics {
-    // All fields must be accessed atomically and aligned.
     operations: Vec<AtomicU64>,
     latency: Vec<LockedLastMinuteLatency>,
     actions: Vec<AtomicU64>,
     actions_latency: Vec<LockedLastMinuteLatency>,
-    // Current paths contains disk -> tracker mappings
     current_paths: Arc<RwLock<HashMap<String, Arc<CurrentPathTracker>>>>,
-
-    // Cycle information
     cycle_info: Arc<RwLock<Option<CurrentCycle>>>,
 }
 
@@ -331,8 +370,6 @@ const OTEL_SCANNER_CYCLES: &str = "rustfs_scanner_cycles_total";
 const OTEL_SCANNER_CYCLE_DURATION_SECONDS: &str = "rustfs_scanner_cycle_duration_seconds";
 const OTEL_SCANNER_BUCKET_DRIVE_DURATION_SECONDS: &str = "rustfs_scanner_bucket_drive_duration_seconds";
 
-/// Emit an OTEL counter increment for the given scanner metric.
-/// ScanCycle and ScanBucketDrive are handled by dedicated emit functions with labels.
 fn emit_otel_counter(metric: usize, count: u64) {
     match Metric::from_index(metric) {
         Some(Metric::ScanObject) => {
@@ -345,8 +382,6 @@ fn emit_otel_counter(metric: usize, count: u64) {
     }
 }
 
-/// Emit OTel metrics for a completed scan cycle.
-/// Counter with result label + gauge for last successful cycle duration.
 pub fn emit_scan_cycle_complete(success: bool, duration: Duration) {
     let result = if success { "success" } else { "error" };
     metrics::counter!(OTEL_SCANNER_CYCLES, "result" => result).increment(1);
@@ -355,8 +390,6 @@ pub fn emit_scan_cycle_complete(success: bool, duration: Duration) {
     }
 }
 
-/// Emit OTel metrics for a completed bucket-drive scan.
-/// Counter with result/bucket/disk labels + histogram for duration.
 pub fn emit_scan_bucket_drive_complete(success: bool, bucket: &str, disk: &str, duration: Duration) {
     let result = if success { "success" } else { "error" };
     metrics::counter!(
@@ -376,226 +409,202 @@ pub fn emit_scan_bucket_drive_complete(success: bool, bucket: &str, disk: &str,
 
 impl Metrics {
     pub fn new() -> Self {
-        let operations = (0..Metric::Last as usize).map(|_| AtomicU64::new(0)).collect();
-
-        let latency = (0..Metric::LastRealtime as usize)
-            .map(|_| LockedLastMinuteLatency::new())
-            .collect();
-
         Self {
-            operations,
-            latency,
+            operations: (0..Metric::Last as usize).map(|_| AtomicU64::new(0)).collect(),
+            // Each slot gets its own fresh LockedLastMinuteLatency so that
+            // different metrics never accidentally share state.
+            latency: (0..Metric::LastRealtime as usize)
+                .map(|_| LockedLastMinuteLatency::new())
+                .collect(),
             actions: (0..IlmAction::ActionCount as usize).map(|_| AtomicU64::new(0)).collect(),
-            actions_latency: vec![LockedLastMinuteLatency::default(); IlmAction::ActionCount as usize],
+            actions_latency: (0..IlmAction::ActionCount as usize)
+                .map(|_| LockedLastMinuteLatency::new())
+                .collect(),
             current_paths: Arc::new(RwLock::new(HashMap::new())),
             cycle_info: Arc::new(RwLock::new(None)),
         }
     }
 
-    /// Log scanner action with custom metadata - compatible with existing usage
+    // -----------------------------------------------------------------------
+    // Metric recording helpers
+    //
+    // All of these are now pure sync closures.  No tokio::spawn, no heap-
+    // allocated future, no scheduler overhead — just an atomic increment and a
+    // std-mutex lock for fewer than ~10 ns of work.
+    // -----------------------------------------------------------------------
+
+    /// Return a closure that records one observation of `metric` (with
+    /// optional caller-supplied metadata).  Call it once the operation ends.
     pub fn log(metric: Metric) -> impl Fn(&HashMap<String, String>) {
-        let metric = metric as usize;
-        let start_time = SystemTime::now();
+        let metric_idx = metric as usize;
+        let start = SystemTime::now();
         move |_custom: &HashMap<String, String>| {
-            let duration = SystemTime::now().duration_since(start_time).unwrap_or_default();
-
-            // Update operation count
-            global_metrics().operations[metric].fetch_add(1, Ordering::Relaxed);
-            emit_otel_counter(metric, 1);
-
-            // Update latency for realtime metrics (spawn async task for this)
-            if (metric) < Metric::LastRealtime as usize {
-                let metric_index = metric;
-                tokio::spawn(async move {
-                    global_metrics().latency[metric_index].add(duration).await;
-                });
-            }
-
-            // Log trace metrics
-            if metric as u8 > Metric::StartTrace as u8 {
-                //debug!(metric = metric.as_str(), duration_ms = duration.as_millis(), "Scanner trace metric");
+            let duration = SystemTime::now().duration_since(start).unwrap_or_default();
+            global_metrics().operations[metric_idx].fetch_add(1, Ordering::Relaxed);
+            emit_otel_counter(metric_idx, 1);
+            if metric_idx < Metric::LastRealtime as usize {
+                global_metrics().latency[metric_idx].add(duration);
             }
         }
     }
 
-    /// Time scanner action with size - returns function that takes size
+    /// Return a closure that records one observation of `metric` together with
+    /// a byte count.  Call `done(size_bytes)` when the operation ends.
     pub fn time_size(metric: Metric) -> impl Fn(u64) {
-        let metric = metric as usize;
-        let start_time = SystemTime::now();
+        let metric_idx = metric as usize;
+        let start = SystemTime::now();
         move |size: u64| {
-            let duration = SystemTime::now().duration_since(start_time).unwrap_or_default();
-
-            // Update operation count
-            global_metrics().operations[metric].fetch_add(1, Ordering::Relaxed);
-            emit_otel_counter(metric, 1);
-
-            // Update latency for realtime metrics with size (spawn async task)
-            if (metric) < Metric::LastRealtime as usize {
-                let metric_index = metric;
-                tokio::spawn(async move {
-                    global_metrics().latency[metric_index].add_size(duration, size).await;
-                });
+            let duration = SystemTime::now().duration_since(start).unwrap_or_default();
+            global_metrics().operations[metric_idx].fetch_add(1, Ordering::Relaxed);
+            emit_otel_counter(metric_idx, 1);
+            if metric_idx < Metric::LastRealtime as usize {
+                global_metrics().latency[metric_idx].add_size(duration, size);
             }
         }
     }
 
-    /// Time a scanner action - returns a closure to call when done
+    /// Return a closure that records one observation of `metric`.
+    /// Call `done()` when the operation ends.
     pub fn time(metric: Metric) -> impl Fn() {
-        let metric = metric as usize;
-        let start_time = SystemTime::now();
+        let metric_idx = metric as usize;
+        let start = SystemTime::now();
         move || {
-            let duration = SystemTime::now().duration_since(start_time).unwrap_or_default();
-
-            // Update operation count
-            global_metrics().operations[metric].fetch_add(1, Ordering::Relaxed);
-            emit_otel_counter(metric, 1);
-
-            // Update latency for realtime metrics (spawn async task)
-            if (metric) < Metric::LastRealtime as usize {
-                let metric_index = metric;
-                tokio::spawn(async move {
-                    global_metrics().latency[metric_index].add(duration).await;
-                });
+            let duration = SystemTime::now().duration_since(start).unwrap_or_default();
+            global_metrics().operations[metric_idx].fetch_add(1, Ordering::Relaxed);
+            emit_otel_counter(metric_idx, 1);
+            if metric_idx < Metric::LastRealtime as usize {
+                global_metrics().latency[metric_idx].add(duration);
             }
         }
     }
 
-    /// Time N scanner actions - returns function that takes count, then returns completion function
+    /// Return a two-stage closure: first call takes an item count, second call
+    /// (returned closure) fires when the batch of `count` operations ends.
     pub fn time_n(metric: Metric) -> Box<dyn Fn(usize) -> Box<dyn Fn() + Send + Sync> + Send + Sync> {
-        let metric = metric as usize;
-        let start_time = SystemTime::now();
+        let metric_idx = metric as usize;
+        let start = SystemTime::now();
         Box::new(move |count: usize| {
             Box::new(move || {
-                let duration = SystemTime::now().duration_since(start_time).unwrap_or_default();
-
-                // Update operation count
-                global_metrics().operations[metric].fetch_add(count as u64, Ordering::Relaxed);
-                emit_otel_counter(metric, count as u64);
-
-                // Update latency for realtime metrics (spawn async task)
-                if (metric) < Metric::LastRealtime as usize {
-                    let metric_index = metric;
-                    tokio::spawn(async move {
-                        global_metrics().latency[metric_index].add(duration).await;
-                    });
+                let duration = SystemTime::now().duration_since(start).unwrap_or_default();
+                global_metrics().operations[metric_idx].fetch_add(count as u64, Ordering::Relaxed);
+                emit_otel_counter(metric_idx, count as u64);
+                if metric_idx < Metric::LastRealtime as usize {
+                    global_metrics().latency[metric_idx].add(duration);
                 }
             })
         })
     }
 
-    /// Time ILM action with versions - returns function that takes versions, then returns completion function
+    /// Return a two-stage closure for ILM actions: first call takes a version
+    /// count, second call fires when the action completes.
     pub fn time_ilm(a: IlmAction) -> Box<dyn Fn(u64) -> Box<dyn Fn() + Send + Sync> + Send + Sync> {
-        let a_clone = a as usize;
-        if a_clone == IlmAction::NoneAction as usize || a_clone >= IlmAction::ActionCount as usize {
-            return Box::new(move |_: u64| Box::new(move || {}));
+        let a_idx = a as usize;
+        if a_idx == IlmAction::NoneAction as usize || a_idx >= IlmAction::ActionCount as usize {
+            return Box::new(|_| Box::new(|| {}));
         }
         let start = SystemTime::now();
         Box::new(move |versions: u64| {
             Box::new(move || {
-                let duration = SystemTime::now().duration_since(start).unwrap_or(Duration::from_secs(0));
-                tokio::spawn(async move {
-                    global_metrics().actions[a_clone].fetch_add(versions, Ordering::Relaxed);
-                    global_metrics().actions_latency[a_clone].add(duration).await;
-                });
+                let duration = SystemTime::now().duration_since(start).unwrap_or_default();
+                global_metrics().actions[a_idx].fetch_add(versions, Ordering::Relaxed);
+                global_metrics().actions_latency[a_idx].add(duration);
             })
         })
     }
 
-    /// Increment time with specific duration
-    pub async fn inc_time(metric: Metric, duration: Duration) {
-        let metric = metric as usize;
-        // Update operation count
-        global_metrics().operations[metric].fetch_add(1, Ordering::Relaxed);
-        emit_otel_counter(metric, 1);
-
-        // Update latency for realtime metrics
-        if (metric) < Metric::LastRealtime as usize {
-            global_metrics().latency[metric].add(duration).await;
+    /// Record a single observation of `metric` with a caller-supplied duration.
+    /// No longer async — nothing inside requires it.
+    pub fn inc_time(metric: Metric, duration: Duration) {
+        let metric_idx = metric as usize;
+        global_metrics().operations[metric_idx].fetch_add(1, Ordering::Relaxed);
+        emit_otel_counter(metric_idx, 1);
+        if metric_idx < Metric::LastRealtime as usize {
+            global_metrics().latency[metric_idx].add(duration);
         }
     }
 
-    /// Get lifetime operation count for a metric
+    // -----------------------------------------------------------------------
+    // Read-side helpers
+    // -----------------------------------------------------------------------
+
+    /// Lifetime operation count for `metric`.
     pub fn lifetime(&self, metric: Metric) -> u64 {
-        let metric = metric as usize;
-        if (metric) >= Metric::Last as usize {
+        let idx = metric as usize;
+        if idx >= Metric::Last as usize {
             return 0;
         }
-        self.operations[metric].load(Ordering::Relaxed)
+        self.operations[idx].load(Ordering::Relaxed)
     }
 
-    /// Get last minute statistics for a metric
-    pub async fn last_minute(&self, metric: Metric) -> AccElem {
-        let metric = metric as usize;
-        if (metric) >= Metric::LastRealtime as usize {
+    /// Last-minute accumulated stats for a realtime metric.
+    /// No longer async — LockedLastMinuteLatency::total() is now synchronous.
+    pub fn last_minute(&self, metric: Metric) -> AccElem {
+        let idx = metric as usize;
+        if idx >= Metric::LastRealtime as usize {
             return AccElem::default();
         }
-        self.latency[metric].total().await
+        self.latency[idx].total()
     }
 
-    /// Set current cycle information
+    /// Replace the current cycle record.
     pub async fn set_cycle(&self, cycle: Option<CurrentCycle>) {
         *self.cycle_info.write().await = cycle;
     }
 
-    /// Get current cycle information
+    /// Read the current cycle record.
     pub async fn get_cycle(&self) -> Option<CurrentCycle> {
         self.cycle_info.read().await.clone()
     }
 
-    /// Get current active paths
+    /// Snapshot of every path currently being scanned.
     pub async fn get_current_paths(&self) -> Vec<String> {
-        let mut result = Vec::new();
         let paths = self.current_paths.read().await;
-
+        let mut result = Vec::with_capacity(paths.len());
         for (disk, tracker) in paths.iter() {
-            let path = tracker.get_path().await;
-            result.push(format!("{disk}/{path}"));
+            result.push(format!("{disk}/{}", tracker.get_path().await));
         }
-
         result
     }
 
-    /// Get number of active drives
+    /// Number of drives with an active scan in progress.
     pub async fn active_drives(&self) -> usize {
         self.current_paths.read().await.len()
     }
 
-    /// Generate metrics report
+    /// Build a full metrics report snapshot.
     pub async fn report(&self) -> M_ScannerMetrics {
-        let mut metrics = M_ScannerMetrics::default();
+        let mut m = M_ScannerMetrics::default();
 
-        // Set cycle information
         if let Some(cycle) = self.get_cycle().await {
-            metrics.current_cycle = cycle.current;
-            metrics.cycles_completed_at = cycle.cycle_completed;
-            metrics.current_started = cycle.started;
+            m.current_cycle = cycle.current;
+            m.cycles_completed_at = cycle.cycle_completed;
+            m.current_started = cycle.started;
         }
 
-        // Replace default start time with global init time if it's the placeholder
         if let Some(init_time) = crate::get_global_init_time().await {
-            metrics.current_started = init_time;
+            m.current_started = init_time;
         }
 
-        metrics.collected_at = Utc::now();
-        metrics.active_paths = self.get_current_paths().await;
+        m.collected_at = Utc::now();
+        m.active_paths = self.get_current_paths().await;
 
-        // Lifetime operations
+        // Lifetime operation counts
         for i in 0..Metric::Last as usize {
             let count = self.operations[i].load(Ordering::Relaxed);
             if count > 0
                 && let Some(metric) = Metric::from_index(i)
             {
-                metrics.life_time_ops.insert(metric.as_str().to_string(), count);
+                m.life_time_ops.insert(metric.as_str().to_string(), count);
             }
         }
 
-        // Last minute statistics for realtime metrics
+        // Last-minute stats for realtime metrics — now plain sync calls
         for i in 0..Metric::LastRealtime as usize {
-            let last_min = self.latency[i].total().await;
+            let last_min = self.latency[i].total();
             if last_min.n > 0
                 && let Some(metric) = Metric::from_index(i)
             {
-                metrics.last_minute.actions.insert(
+                m.last_minute.actions.insert(
                     metric.as_str().to_string(),
                     TimedAction {
                         count: last_min.n,
@@ -606,23 +615,23 @@ impl Metrics {
             }
         }
 
-        // Lifetime ILM operations
+        // Lifetime ILM counts
         for i in 0..IlmAction::ActionCount as usize {
             let count = self.actions[i].load(Ordering::Relaxed);
             if count > 0
                 && let Some(action) = IlmAction::from_index(i)
             {
-                metrics.life_time_ilm.insert(action.as_str().to_string(), count);
+                m.life_time_ilm.insert(action.as_str().to_string(), count);
             }
         }
 
-        // Last minute ILM latency
+        // Last-minute ILM latency — plain sync calls
         for i in 0..IlmAction::ActionCount as usize {
-            let last_min = self.actions_latency[i].total().await;
+            let last_min = self.actions_latency[i].total();
             if last_min.n > 0
                 && let Some(action) = IlmAction::from_index(i)
             {
-                metrics.last_minute.ilm.insert(
+                m.last_minute.ilm.insert(
                     action.as_str().to_string(),
                     TimedAction {
                         count: last_min.n,
@@ -633,56 +642,65 @@ impl Metrics {
             }
         }
 
-        metrics
+        m
     }
 }
 
-// Type aliases for compatibility with existing code
-pub type UpdateCurrentPathFn = Arc<dyn Fn(&str) -> Pin<Box<dyn Future<Output = ()> + Send>> + Send + Sync>;
-pub type CloseDiskFn = Arc<dyn Fn() -> Pin<Box<dyn Future<Output = ()> + Send>> + Send + Sync>;
-
-/// Create a current path updater for tracking scan progress
-pub fn current_path_updater(disk: &str, initial: &str) -> (UpdateCurrentPathFn, CloseDiskFn) {
-    let tracker = Arc::new(CurrentPathTracker::new(initial.to_string()));
-    let disk_name = disk.to_string();
-
-    // Store the tracker in global metrics
-    let tracker_clone = Arc::clone(&tracker);
-    let disk_clone = disk_name.clone();
-    tokio::spawn(async move {
-        global_metrics().current_paths.write().await.insert(disk_clone, tracker_clone);
-    });
-
-    let update_fn = {
-        let tracker = Arc::clone(&tracker);
-        Arc::new(move |path: &str| -> Pin<Box<dyn Future<Output = ()> + Send>> {
-            let tracker = Arc::clone(&tracker);
-            let path = path.to_string();
-            Box::pin(async move {
-                tracker.update_path(path).await;
-            })
-        })
-    };
-
-    let done_fn = {
-        let disk_name = disk_name.clone();
-        Arc::new(move || -> Pin<Box<dyn Future<Output = ()> + Send>> {
-            let disk_name = disk_name.clone();
-            Box::pin(async move {
-                global_metrics().current_paths.write().await.remove(&disk_name);
-            })
-        })
-    };
-
-    (update_fn, done_fn)
-}
-
 impl Default for Metrics {
     fn default() -> Self {
         Self::new()
     }
 }
 
+// ---------------------------------------------------------------------------
+// Path tracking helpers
+// ---------------------------------------------------------------------------
+
+pub type UpdateCurrentPathFn = Arc<dyn Fn(&str) -> Pin<Box<dyn Future<Output = ()> + Send>> + Send + Sync>;
+pub type CloseDiskFn = Arc<dyn Fn() -> Pin<Box<dyn Future<Output = ()> + Send>> + Send + Sync>;
+
+/// Register a new disk in the global path tracker and return two callbacks:
+/// one to update the current path and one to deregister the disk when done.
+pub fn current_path_updater(disk: &str, initial: &str) -> (UpdateCurrentPathFn, CloseDiskFn) {
+    let tracker = Arc::new(CurrentPathTracker::new(initial.to_string()));
+    let disk_name = disk.to_string();
+
+    let tracker_clone = Arc::clone(&tracker);
+    let disk_insert = disk_name.clone();
+    tokio::spawn(async move {
+        global_metrics()
+            .current_paths
+            .write()
+            .await
+            .insert(disk_insert, tracker_clone);
+    });
+
+    let update_fn: UpdateCurrentPathFn = {
+        let tracker = Arc::clone(&tracker);
+        Arc::new(move |path: &str| {
+            let tracker = Arc::clone(&tracker);
+            let path = path.to_string();
+            Box::pin(async move { tracker.update_path(path).await })
+        })
+    };
+
+    let done_fn: CloseDiskFn = {
+        let disk = disk_name.clone();
+        Arc::new(move || {
+            let disk = disk.clone();
+            Box::pin(async move {
+                global_metrics().current_paths.write().await.remove(&disk);
+            })
+        })
+    };
+
+    (update_fn, done_fn)
+}
+
+// ---------------------------------------------------------------------------
+// CloseDiskGuard
+// ---------------------------------------------------------------------------
+
 pub struct CloseDiskGuard(CloseDiskFn);
 
 impl CloseDiskGuard {
@@ -697,16 +715,10 @@ impl CloseDiskGuard {
 
 impl Drop for CloseDiskGuard {
     fn drop(&mut self) {
-        // Drop cannot be async, so we spawn the async cleanup task
-        // The task will run in the background and complete asynchronously
         if let Ok(handle) = tokio::runtime::Handle::try_current() {
             let close_fn = self.0.clone();
-            handle.spawn(async move {
-                close_fn().await;
-            });
-        } else {
-            // If we're not in a tokio runtime context, we can't spawn
-            // This is a best-effort cleanup, so we just skip it
+            handle.spawn(async move { close_fn().await });
         }
+        // If there is no runtime we are in a test or shutdown path; skip cleanup.
     }
 }

crates/common/src/readiness.rs

@@ -60,6 +60,23 @@ impl GlobalReadiness {
     pub fn is_ready(&self) -> bool {
         self.status.load(Ordering::SeqCst) == SystemStage::FullReady as u8
     }
+
+    /// Get the current system stage
+    /// # Returns
+    /// The current SystemStage of the service
+    pub fn current_stage(&self) -> SystemStage {
+        match self.status.load(Ordering::SeqCst) {
+            0 => SystemStage::Booting,
+            1 => SystemStage::StorageReady,
+            2 => SystemStage::IamReady,
+            3 => SystemStage::FullReady,
+            invalid => {
+                debug_assert!(false, "GlobalReadiness::current_stage: invalid status value {}", invalid);
+                // Fallback to the most conservative stage on invalid values
+                SystemStage::Booting
+            }
+        }
+    }
 }
 
 #[cfg(test)]

crates/concurrency/Cargo.toml

@@ -0,0 +1,43 @@
+[package]
+name = "rustfs-concurrency"
+version.workspace = true
+edition.workspace = true
+license.workspace = true
+repository.workspace = true
+rust-version.workspace = true
+homepage.workspace = true
+description = "Concurrency management for RustFS - timeout, locking, backpressure, and I/O scheduling"
+keywords = ["rustfs", "concurrency", "timeout", "backpressure", "scheduling"]
+categories = ["concurrency", "filesystem"]
+
+[dependencies]
+# Internal crates
+rustfs-io-core = { workspace = true }
+rustfs-io-metrics = { workspace = true }
+
+# Async runtime
+tokio = { workspace = true, features = ["sync", "time", "rt"] }
+tokio-util = { workspace = true }
+
+# Error handling
+thiserror = { workspace = true }
+
+# Logging
+tracing = { workspace = true }
+
+[dev-dependencies]
+tokio = { workspace = true, features = ["full"] }
+
+[features]
+default = ["timeout", "lock", "deadlock", "backpressure", "scheduler"]
+
+# Feature modules
+timeout = []
+lock = []
+deadlock = []
+backpressure = []
+scheduler = []
+
+[package.metadata.docs.rs]
+all-features = true
+rustdoc-args = ["--cfg", "docsrs"]

crates/concurrency/src/backpressure.rs

@@ -0,0 +1,224 @@
+// Copyright 2024 RustFS Team
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+//! Backpressure management
+
+use rustfs_io_core::{BackpressureMonitor as CoreBackpressureMonitor, BackpressureState};
+use rustfs_io_metrics::backpressure_metrics;
+use std::sync::Arc;
+use std::time::Instant;
+use tokio::io::{DuplexStream, duplex};
+
+/// Backpressure configuration
+#[derive(Debug, Clone)]
+pub struct BackpressureConfig {
+    /// Buffer size in bytes
+    pub buffer_size: usize,
+    /// High watermark percentage
+    pub high_watermark: u32,
+    /// Low watermark percentage
+    pub low_watermark: u32,
+}
+
+impl Default for BackpressureConfig {
+    fn default() -> Self {
+        Self {
+            buffer_size: 4 * 1024 * 1024, // 4MB
+            high_watermark: 80,
+            low_watermark: 50,
+        }
+    }
+}
+
+impl BackpressureConfig {
+    /// Calculate high watermark threshold in bytes
+    pub fn high_watermark_bytes(&self) -> usize {
+        (self.buffer_size as u64 * self.high_watermark as u64 / 100) as usize
+    }
+
+    /// Calculate low watermark threshold in bytes
+    pub fn low_watermark_bytes(&self) -> usize {
+        (self.buffer_size as u64 * self.low_watermark as u64 / 100) as usize
+    }
+}
+
+/// Backpressure manager
+pub struct BackpressureManager {
+    config: BackpressureConfig,
+    monitor: Arc<CoreBackpressureMonitor>,
+}
+
+impl BackpressureManager {
+    /// Create a new backpressure manager
+    pub fn new(buffer_size: usize, high_watermark: u32, low_watermark: u32) -> Self {
+        let config = BackpressureConfig {
+            buffer_size,
+            high_watermark,
+            low_watermark,
+        };
+
+        let core_config = rustfs_io_core::BackpressureConfig {
+            max_concurrent: 32,
+            high_water_mark: high_watermark as f64 / 100.0,
+            low_water_mark: low_watermark as f64 / 100.0,
+            cooldown: std::time::Duration::from_millis(100),
+            enabled: true,
+        };
+
+        Self {
+            config,
+            monitor: Arc::new(CoreBackpressureMonitor::new(core_config)),
+        }
+    }
+
+    /// Get the configuration
+    pub fn config(&self) -> &BackpressureConfig {
+        &self.config
+    }
+
+    /// Get the monitor
+    pub fn monitor(&self) -> Arc<CoreBackpressureMonitor> {
+        self.monitor.clone()
+    }
+
+    /// Create a backpressure pipe
+    pub fn create_pipe(&self) -> BackpressurePipe {
+        BackpressurePipe::new(self.config.clone(), self.monitor.clone())
+    }
+
+    /// Get current state
+    pub fn state(&self) -> BackpressureState {
+        self.monitor.state()
+    }
+
+    /// Check if backpressure is active
+    pub fn is_active(&self) -> bool {
+        self.monitor.is_active()
+    }
+}
+
+/// Backpressure pipe wrapping tokio's duplex
+pub struct BackpressurePipe {
+    reader: DuplexStream,
+    writer: DuplexStream,
+    config: BackpressureConfig,
+    monitor: Arc<CoreBackpressureMonitor>,
+    created_at: Instant,
+}
+
+impl BackpressurePipe {
+    fn new(config: BackpressureConfig, monitor: Arc<CoreBackpressureMonitor>) -> Self {
+        let (reader, writer) = duplex(config.buffer_size);
+
+        Self {
+            reader,
+            writer,
+            config,
+            monitor,
+            created_at: Instant::now(),
+        }
+    }
+
+    /// Get the reader end
+    pub fn reader(&mut self) -> &mut DuplexStream {
+        &mut self.reader
+    }
+
+    /// Get the writer end
+    pub fn writer(&mut self) -> &mut DuplexStream {
+        &mut self.writer
+    }
+
+    /// Split into reader and writer
+    pub fn into_split(self) -> (DuplexStream, DuplexStream) {
+        (self.reader, self.writer)
+    }
+
+    /// Get the configuration
+    pub fn config(&self) -> &BackpressureConfig {
+        &self.config
+    }
+
+    /// Get current state
+    pub fn state(&self) -> BackpressureState {
+        self.monitor.state()
+    }
+
+    /// Get the age of this pipe
+    pub fn age(&self) -> std::time::Duration {
+        self.created_at.elapsed()
+    }
+
+    /// Check if should apply backpressure
+    pub fn should_apply_backpressure(&self) -> bool {
+        let should = self.monitor.should_apply_backpressure();
+        if should {
+            backpressure_metrics::record_backpressure_activation();
+        }
+        should
+    }
+}
+
+/// Backpressure event
+#[allow(dead_code)]
+#[derive(Debug, Clone)]
+pub struct BackpressureEvent {
+    /// Event timestamp
+    pub timestamp: Instant,
+    /// Event type
+    pub event_type: BackpressureEventType,
+    /// Buffer usage
+    pub buffer_usage: usize,
+    /// Buffer capacity
+    pub buffer_capacity: usize,
+}
+
+/// Backpressure event type
+#[allow(dead_code)]
+#[derive(Debug, Clone, Copy)]
+pub enum BackpressureEventType {
+    /// High watermark reached
+    HighWatermarkReached,
+    /// High watermark exited
+    HighWatermarkExited,
+    /// Backpressure applied
+    BackpressureApplied,
+    /// Backpressure released
+    BackpressureReleased,
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    #[test]
+    fn test_backpressure_config() {
+        let config = BackpressureConfig::default();
+        assert_eq!(config.buffer_size, 4 * 1024 * 1024);
+        assert!(config.high_watermark > config.low_watermark);
+    }
+
+    #[test]
+    fn test_backpressure_manager() {
+        let manager = BackpressureManager::new(1024, 80, 50);
+        assert_eq!(manager.state(), BackpressureState::Normal);
+    }
+
+    #[test]
+    fn test_backpressure_pipe() {
+        let manager = BackpressureManager::new(1024, 80, 50);
+        let pipe = manager.create_pipe();
+        assert_eq!(pipe.state(), BackpressureState::Normal);
+    }
+}

crates/concurrency/src/config.rs

@@ -0,0 +1,256 @@
+// Copyright 2024 RustFS Team
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+//! Configuration for concurrency management
+
+use std::time::Duration;
+
+/// Feature flags for concurrency modules
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+pub struct ConcurrencyFeatures {
+    /// Enable timeout control
+    pub timeout: bool,
+    /// Enable lock optimization
+    pub lock: bool,
+    /// Enable deadlock detection
+    pub deadlock: bool,
+    /// Enable backpressure management
+    pub backpressure: bool,
+    /// Enable I/O scheduling
+    pub scheduler: bool,
+}
+
+impl Default for ConcurrencyFeatures {
+    fn default() -> Self {
+        Self {
+            timeout: cfg!(feature = "timeout"),
+            lock: cfg!(feature = "lock"),
+            deadlock: cfg!(feature = "deadlock"),
+            backpressure: cfg!(feature = "backpressure"),
+            scheduler: cfg!(feature = "scheduler"),
+        }
+    }
+}
+
+impl ConcurrencyFeatures {
+    /// Create with all features enabled
+    pub fn all() -> Self {
+        Self {
+            timeout: true,
+            lock: true,
+            deadlock: true,
+            backpressure: true,
+            scheduler: true,
+        }
+    }
+
+    /// Create with no features enabled
+    pub fn none() -> Self {
+        Self {
+            timeout: false,
+            lock: false,
+            deadlock: false,
+            backpressure: false,
+            scheduler: false,
+        }
+    }
+
+    /// Check if any feature is enabled
+    pub fn any_enabled(&self) -> bool {
+        self.timeout || self.lock || self.deadlock || self.backpressure || self.scheduler
+    }
+}
+
+/// Main configuration for concurrency management
+#[derive(Debug, Clone)]
+pub struct ConcurrencyConfig {
+    /// Feature flags
+    pub features: ConcurrencyFeatures,
+
+    // Timeout configuration
+    /// Default timeout duration
+    pub default_timeout: Duration,
+    /// Maximum timeout duration
+    pub max_timeout: Duration,
+    /// Enable dynamic timeout
+    pub enable_dynamic_timeout: bool,
+
+    // Lock configuration
+    /// Enable lock optimization
+    pub enable_lock_optimization: bool,
+    /// Lock acquisition timeout
+    pub lock_acquire_timeout: Duration,
+
+    // Deadlock configuration
+    /// Enable deadlock detection
+    pub enable_deadlock_detection: bool,
+    /// Deadlock check interval
+    pub deadlock_check_interval: Duration,
+    /// Hang threshold
+    pub hang_threshold: Duration,
+
+    // Backpressure configuration
+    /// Buffer size for backpressure
+    pub backpressure_buffer_size: usize,
+    /// High watermark percentage
+    pub high_watermark: u32,
+    /// Low watermark percentage
+    pub low_watermark: u32,
+
+    // Scheduler configuration
+    /// Base buffer size for I/O
+    pub io_buffer_size: usize,
+    /// Maximum buffer size
+    pub max_buffer_size: usize,
+    /// High priority size threshold
+    pub high_priority_threshold: usize,
+    /// Low priority size threshold
+    pub low_priority_threshold: usize,
+}
+
+impl Default for ConcurrencyConfig {
+    fn default() -> Self {
+        Self {
+            features: ConcurrencyFeatures::default(),
+
+            // Timeout defaults
+            default_timeout: Duration::from_secs(30),
+            max_timeout: Duration::from_secs(300),
+            enable_dynamic_timeout: true,
+
+            // Lock defaults
+            enable_lock_optimization: true,
+            lock_acquire_timeout: Duration::from_secs(5),
+
+            // Deadlock defaults
+            enable_deadlock_detection: false,
+            deadlock_check_interval: Duration::from_secs(10),
+            hang_threshold: Duration::from_secs(60),
+
+            // Backpressure defaults
+            backpressure_buffer_size: 4 * 1024 * 1024, // 4MB
+            high_watermark: 80,
+            low_watermark: 50,
+
+            // Scheduler defaults
+            io_buffer_size: 64 * 1024,                // 64KB
+            max_buffer_size: 4 * 1024 * 1024,         // 4MB
+            high_priority_threshold: 1024 * 1024,     // 1MB
+            low_priority_threshold: 10 * 1024 * 1024, // 10MB
+        }
+    }
+}
+
+impl ConcurrencyConfig {
+    /// Create configuration from environment variables
+    pub fn from_env() -> Self {
+        let mut config = Self::default();
+
+        // Read from environment if available
+        if let Ok(val) = std::env::var("RUSTFS_TIMEOUT_DEFAULT")
+            && let Ok(secs) = val.parse::<u64>()
+        {
+            config.default_timeout = Duration::from_secs(secs);
+        }
+
+        if let Ok(val) = std::env::var("RUSTFS_TIMEOUT_MAX")
+            && let Ok(secs) = val.parse::<u64>()
+        {
+            config.max_timeout = Duration::from_secs(secs);
+        }
+
+        if let Ok(val) = std::env::var("RUSTFS_BACKPRESSURE_BUFFER_SIZE")
+            && let Ok(size) = val.parse::<usize>()
+        {
+            config.backpressure_buffer_size = size;
+        }
+
+        if let Ok(val) = std::env::var("RUSTFS_IO_BUFFER_SIZE")
+            && let Ok(size) = val.parse::<usize>()
+        {
+            config.io_buffer_size = size;
+        }
+
+        config
+    }
+
+    /// Validate configuration
+    pub fn validate(&self) -> Result<(), ConfigError> {
+        if self.default_timeout > self.max_timeout {
+            return Err(ConfigError::InvalidTimeout("default_timeout cannot exceed max_timeout".to_string()));
+        }
+
+        if self.high_watermark <= self.low_watermark || self.high_watermark > 100 {
+            return Err(ConfigError::InvalidBackpressure(
+                "high_watermark must be > low_watermark and <= 100".to_string(),
+            ));
+        }
+
+        if self.io_buffer_size > self.max_buffer_size {
+            return Err(ConfigError::InvalidScheduler("io_buffer_size cannot exceed max_buffer_size".to_string()));
+        }
+
+        Ok(())
+    }
+}
+
+/// Configuration error
+#[allow(clippy::enum_variant_names)]
+#[derive(Debug, Clone, thiserror::Error)]
+pub enum ConfigError {
+    /// Invalid timeout configuration
+    #[error("Invalid timeout config: {0}")]
+    InvalidTimeout(String),
+
+    /// Invalid backpressure configuration
+    #[error("Invalid backpressure config: {0}")]
+    InvalidBackpressure(String),
+
+    /// Invalid scheduler configuration
+    #[error("Invalid scheduler config: {0}")]
+    InvalidScheduler(String),
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    #[test]
+    fn test_default_config() {
+        let config = ConcurrencyConfig::default();
+        assert!(config.validate().is_ok());
+    }
+
+    #[test]
+    fn test_invalid_timeout() {
+        let config = ConcurrencyConfig {
+            default_timeout: Duration::from_secs(100),
+            max_timeout: Duration::from_secs(50),
+            ..Default::default()
+        };
+        assert!(
+            config.validate().is_err(),
+            "validate() should return an error when default_timeout > max_timeout"
+        );
+    }
+
+    #[test]
+    fn test_features() {
+        let features = ConcurrencyFeatures::all();
+        assert!(features.any_enabled());
+
+        let features = ConcurrencyFeatures::none();
+        assert!(!features.any_enabled());
+    }
+}

crates/concurrency/src/deadlock.rs

@@ -0,0 +1,207 @@
+// Copyright 2024 RustFS Team
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+//! Deadlock detection management
+
+use rustfs_io_core::{DeadlockDetector as CoreDeadlockDetector, LockType};
+use rustfs_io_metrics::deadlock_metrics;
+use std::collections::HashMap;
+use std::sync::Arc;
+use std::time::{Duration, Instant};
+
+/// Deadlock configuration
+#[derive(Debug, Clone)]
+pub struct DeadlockConfig {
+    /// Enable deadlock detection
+    pub enabled: bool,
+    /// Check interval
+    pub check_interval: Duration,
+    /// Hang threshold
+    pub hang_threshold: Duration,
+}
+
+impl Default for DeadlockConfig {
+    fn default() -> Self {
+        Self {
+            enabled: false,
+            check_interval: Duration::from_secs(10),
+            hang_threshold: Duration::from_secs(60),
+        }
+    }
+}
+
+/// Deadlock manager
+pub struct DeadlockManager {
+    config: DeadlockConfig,
+    detector: Arc<CoreDeadlockDetector>,
+    running: Arc<tokio::sync::Mutex<bool>>,
+}
+
+impl DeadlockManager {
+    /// Create a new deadlock manager
+    pub fn new(enabled: bool, check_interval: Duration, hang_threshold: Duration) -> Self {
+        let config = DeadlockConfig {
+            enabled,
+            check_interval,
+            hang_threshold,
+        };
+
+        let core_config = rustfs_io_core::DeadlockDetectorConfig {
+            enabled,
+            detection_interval: check_interval,
+            max_hold_time: hang_threshold,
+        };
+
+        Self {
+            config,
+            detector: Arc::new(CoreDeadlockDetector::new(core_config)),
+            running: Arc::new(tokio::sync::Mutex::new(false)),
+        }
+    }
+
+    /// Get the configuration
+    pub fn config(&self) -> &DeadlockConfig {
+        &self.config
+    }
+
+    /// Get the core detector
+    pub fn detector(&self) -> Arc<CoreDeadlockDetector> {
+        self.detector.clone()
+    }
+
+    /// Start the deadlock detection background task
+    pub async fn start(&self) {
+        if !self.config.enabled {
+            return;
+        }
+
+        let mut running = self.running.lock().await;
+        if *running {
+            return;
+        }
+        *running = true;
+        drop(running);
+
+        tracing::info!("Deadlock detection started");
+    }
+
+    /// Stop the deadlock detection
+    pub async fn stop(&self) {
+        let mut running = self.running.lock().await;
+        *running = false;
+
+        tracing::info!("Deadlock detection stopped");
+    }
+
+    /// Create a request tracker
+    pub fn track_request(&self, request_id: String, description: String) -> RequestTracker {
+        RequestTracker::new(request_id, description, self.detector.clone())
+    }
+
+    /// Register a lock
+    pub fn register_lock(&self, lock_type: LockType) -> u64 {
+        self.detector.register_lock(lock_type)
+    }
+
+    /// Unregister a lock
+    pub fn unregister_lock(&self, lock_id: u64) {
+        self.detector.unregister_lock(lock_id);
+    }
+
+    /// Detect deadlock
+    pub fn detect_deadlock(&self) -> Option<Vec<u64>> {
+        let result = self.detector.detect_deadlock();
+        if let Some(ref cycle) = result {
+            deadlock_metrics::record_deadlock_detected(cycle.len());
+        }
+        result
+    }
+}
+
+/// Request tracker for tracking resources
+pub struct RequestTracker {
+    request_id: String,
+    description: String,
+    start_time: Instant,
+    resources: HashMap<String, Vec<String>>,
+    detector: Arc<CoreDeadlockDetector>,
+}
+
+impl RequestTracker {
+    fn new(request_id: String, description: String, detector: Arc<CoreDeadlockDetector>) -> Self {
+        let start_time = Instant::now();
+        detector.register_request(&request_id, 1); // Use placeholder thread ID
+
+        Self {
+            request_id,
+            description,
+            start_time,
+            resources: HashMap::new(),
+            detector,
+        }
+    }
+
+    /// Get the request ID
+    pub fn request_id(&self) -> &str {
+        &self.request_id
+    }
+
+    /// Get the description
+    pub fn description(&self) -> &str {
+        &self.description
+    }
+
+    /// Get the elapsed time
+    pub fn elapsed(&self) -> Duration {
+        self.start_time.elapsed()
+    }
+
+    /// Record a lock acquisition
+    pub fn record_lock_acquire(&mut self, lock_id: u64, resource: String) {
+        self.resources.entry("locks".to_string()).or_default().push(resource);
+        self.detector.record_acquire(lock_id, 1); // Use placeholder thread ID
+        deadlock_metrics::record_lock_acquisition("read");
+    }
+
+    /// Record a lock release
+    pub fn record_lock_release(&mut self, lock_id: u64) {
+        self.detector.record_release(lock_id);
+    }
+}
+
+impl Drop for RequestTracker {
+    fn drop(&mut self) {
+        self.detector.unregister_request(&self.request_id);
+    }
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    #[test]
+    fn test_deadlock_manager_creation() {
+        let manager = DeadlockManager::new(false, Duration::from_secs(10), Duration::from_secs(60));
+        assert!(!manager.config().enabled);
+    }
+
+    #[tokio::test]
+    async fn test_request_tracker() {
+        let manager = DeadlockManager::new(true, Duration::from_secs(10), Duration::from_secs(60));
+        let tracker = manager.track_request("req-1".to_string(), "test request".to_string());
+
+        assert_eq!(tracker.request_id(), "req-1");
+        assert_eq!(tracker.description(), "test request");
+    }
+}

crates/concurrency/src/lib.rs

@@ -0,0 +1,171 @@
+// Copyright 2024 RustFS Team
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+//! # RustFS Concurrency Management
+//!
+//! This crate provides comprehensive concurrency management for RustFS,
+//! including timeout control, lock optimization, deadlock detection,
+//! backpressure management, and I/O scheduling.
+//!
+//! ## Features
+//!
+//! All features are controlled by feature flags and can be enabled/disabled at compile time:
+//!
+//! - **timeout**: Dynamic timeout calculation based on data size and transfer rate
+//! - **lock**: Early lock release to reduce contention
+//! - **deadlock**: Request tracking and cycle detection
+//! - **backpressure**: Buffer-based flow control
+//! - **scheduler**: Adaptive buffer sizing and priority queuing
+//!
+//! ## Architecture
+//!
+//! ```text
+//! rustfs-concurrency (Business Layer)
+//!     ├── timeout (Timeout Control)
+//!     ├── lock (Lock Optimization)
+//!     ├── deadlock (Deadlock Detection)
+//!     ├── backpressure (Backpressure Management)
+//!     └── scheduler (I/O Scheduling)
+//!          │
+//!          ├── rustfs-io-core (Core Algorithms)
+//!          └── rustfs-io-metrics (Metrics Collection)
+//! ```
+//!
+//! ## Usage
+//!
+//! ```rust,no_run
+//! use rustfs_concurrency::{ConcurrencyConfig, ConcurrencyManager};
+//!
+//! # #[tokio::main]
+//! # async fn main() {
+//! // Create manager with all features enabled
+//! let config = ConcurrencyConfig::default();
+//! let manager = ConcurrencyManager::new(config);
+//!
+//! // Start services
+//! manager.start().await;
+//!
+//! // Use timeout control (if enabled)
+//! if manager.is_timeout_enabled() {
+//!     let timeout_manager = manager.timeout();
+//!     let _ = timeout_manager;
+//! }
+//!
+//! // Use lock optimization (if enabled)
+//! if manager.is_lock_enabled() {
+//!     let lock_manager = manager.lock();
+//!     let _ = lock_manager;
+//! }
+//!
+//! // Stop services
+//! manager.stop().await;
+//! # }
+//! ```
+
+#![deny(missing_docs)]
+#![deny(unsafe_code)]
+#![cfg_attr(docsrs, feature(doc_cfg))]
+
+// Re-export core types from io-core
+pub use rustfs_io_core::{
+    // Backpressure types
+    BackpressureConfig as CoreBackpressureConfig,
+    BackpressureMonitor as CoreBackpressureMonitor,
+    BackpressureState,
+
+    // Deadlock types
+    DeadlockDetector as CoreDeadlockDetector,
+    IoLoadLevel,
+    IoLoadMetrics,
+    IoPriority,
+    // Scheduler types
+    IoScheduler,
+    IoSchedulingContext,
+    LockInfo,
+    LockOptimizer as CoreLockOptimizer,
+
+    // Lock types
+    LockStats as CoreLockStats,
+    LockType,
+    // Timeout types
+    OperationProgress,
+    TimeoutError,
+    TimeoutStats,
+    WaitGraphEdge,
+
+    calculate_adaptive_timeout,
+    estimate_bytes_per_second,
+};
+
+// Module declarations with feature gates
+#[cfg(feature = "timeout")]
+mod timeout;
+
+#[cfg(feature = "lock")]
+mod lock;
+
+#[cfg(feature = "deadlock")]
+mod deadlock;
+
+#[cfg(feature = "backpressure")]
+mod backpressure;
+
+#[cfg(feature = "scheduler")]
+mod scheduler;
+
+// Public module exports with feature gates
+#[cfg(feature = "timeout")]
+pub use timeout::{TimeoutConfig, TimeoutGuard, TimeoutManager};
+
+#[cfg(feature = "lock")]
+pub use lock::{LockConfig, LockManager, LockScopeGuard, OptimizedLockGuard};
+
+#[cfg(feature = "deadlock")]
+pub use deadlock::{DeadlockConfig, DeadlockManager, RequestTracker};
+
+#[cfg(feature = "backpressure")]
+pub use backpressure::{BackpressureConfig, BackpressureManager, BackpressurePipe};
+
+#[cfg(feature = "scheduler")]
+pub use scheduler::{IoStrategy, SchedulerConfig, SchedulerManager};
+
+// Configuration
+mod config;
+pub use config::{ConcurrencyConfig, ConcurrencyFeatures};
+
+// Manager
+mod manager;
+pub use manager::{ConcurrencyManager, GetObjectQueueSnapshot};
+
+// Prelude for convenient imports
+pub mod prelude {
+    //! Prelude module for convenient imports
+
+    #[cfg(feature = "timeout")]
+    pub use crate::timeout::{TimeoutConfig, TimeoutGuard, TimeoutManager};
+
+    #[cfg(feature = "lock")]
+    pub use crate::lock::{LockConfig, LockManager, LockScopeGuard, OptimizedLockGuard};
+
+    #[cfg(feature = "deadlock")]
+    pub use crate::deadlock::{DeadlockConfig, DeadlockManager, RequestTracker};
+
+    #[cfg(feature = "backpressure")]
+    pub use crate::backpressure::{BackpressureConfig, BackpressureManager, BackpressurePipe};
+
+    #[cfg(feature = "scheduler")]
+    pub use crate::scheduler::{IoStrategy, SchedulerConfig, SchedulerManager};
+
+    pub use crate::{ConcurrencyConfig, ConcurrencyFeatures, ConcurrencyManager};
+}

crates/concurrency/src/lock.rs

@@ -0,0 +1,219 @@
+// Copyright 2024 RustFS Team
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+//! Lock optimization management
+
+use rustfs_io_core::{LockOptimizer as CoreLockOptimizer, LockStats};
+use rustfs_io_metrics::lock_metrics;
+use std::sync::Arc;
+use std::time::{Duration, Instant};
+
+/// Lock configuration
+#[derive(Debug, Clone)]
+pub struct LockConfig {
+    /// Enable lock optimization
+    pub enabled: bool,
+    /// Lock acquisition timeout
+    pub acquire_timeout: Duration,
+}
+
+impl Default for LockConfig {
+    fn default() -> Self {
+        Self {
+            enabled: true,
+            acquire_timeout: Duration::from_secs(5),
+        }
+    }
+}
+
+/// Lock manager
+pub struct LockManager {
+    config: LockConfig,
+    optimizer: Arc<CoreLockOptimizer>,
+}
+
+impl LockManager {
+    /// Create a new lock manager
+    pub fn new(enabled: bool, acquire_timeout: Duration) -> Self {
+        let config = LockConfig {
+            enabled,
+            acquire_timeout,
+        };
+
+        let core_config = rustfs_io_core::LockOptimizeConfig {
+            enabled,
+            acquire_timeout,
+            max_hold_time_warning: Duration::from_millis(100),
+            adaptive_spin: true,
+            max_spin_iterations: 1000,
+        };
+
+        Self {
+            config,
+            optimizer: Arc::new(CoreLockOptimizer::new(core_config)),
+        }
+    }
+
+    /// Get the configuration
+    pub fn config(&self) -> &LockConfig {
+        &self.config
+    }
+
+    /// Get the core optimizer
+    pub fn optimizer(&self) -> Arc<CoreLockOptimizer> {
+        self.optimizer.clone()
+    }
+
+    /// Get lock statistics
+    pub fn stats(&self) -> &LockStats {
+        self.optimizer.stats()
+    }
+
+    /// Optimize a lock guard
+    pub fn optimize<G>(&self, guard: G, resource: impl Into<String>) -> OptimizedLockGuard<G> {
+        OptimizedLockGuard::new(guard, resource, self.optimizer.clone())
+    }
+
+    /// Check if optimization is enabled
+    pub fn is_enabled(&self) -> bool {
+        self.config.enabled
+    }
+}
+
+/// Optimized lock guard with early release support
+pub struct OptimizedLockGuard<G> {
+    guard: Option<G>,
+    acquire_time: Instant,
+    released: bool,
+    resource: String,
+    optimizer: Arc<CoreLockOptimizer>,
+}
+
+impl<G> OptimizedLockGuard<G> {
+    fn new(guard: G, resource: impl Into<String>, optimizer: Arc<CoreLockOptimizer>) -> Self {
+        optimizer.on_acquire();
+        lock_metrics::record_lock_optimization_enabled(optimizer.config().enabled);
+
+        Self {
+            guard: Some(guard),
+            acquire_time: Instant::now(),
+            released: false,
+            resource: resource.into(),
+            optimizer,
+        }
+    }
+
+    /// Get the lock hold time
+    pub fn hold_time(&self) -> Duration {
+        self.acquire_time.elapsed()
+    }
+
+    /// Check if the lock has been released
+    pub fn is_released(&self) -> bool {
+        self.released
+    }
+
+    /// Release the lock early
+    pub fn early_release(&mut self) {
+        if self.released {
+            return;
+        }
+
+        let hold_time = self.hold_time();
+        self.guard.take();
+        self.released = true;
+
+        self.optimizer.on_release(hold_time);
+        lock_metrics::record_lock_hold_time(hold_time);
+
+        tracing::debug!(
+            resource = %self.resource,
+            hold_time_ms = hold_time.as_millis(),
+            "Lock released early (optimization active)"
+        );
+    }
+
+    /// Get a reference to the underlying guard
+    pub fn as_ref(&self) -> Option<&G> {
+        if self.released { None } else { self.guard.as_ref() }
+    }
+}
+
+impl<G> Drop for OptimizedLockGuard<G> {
+    fn drop(&mut self) {
+        if !self.released {
+            let hold_time = self.hold_time();
+            self.guard.take();
+            self.released = true;
+
+            self.optimizer.on_release(hold_time);
+            lock_metrics::record_lock_hold_time(hold_time);
+
+            tracing::debug!(
+                resource = %self.resource,
+                hold_time_ms = hold_time.as_millis(),
+                "Lock released on drop (normal release)"
+            );
+        }
+    }
+}
+
+/// Lock scope guard for RAII semantics
+pub struct LockScopeGuard<G> {
+    guard: Option<G>,
+}
+
+impl<G> LockScopeGuard<G> {
+    /// Create a new scope guard
+    pub fn new(guard: G) -> Self {
+        Self { guard: Some(guard) }
+    }
+
+    /// Get a reference to the guard
+    pub fn as_ref(&self) -> Option<&G> {
+        self.guard.as_ref()
+    }
+}
+
+impl<G> Drop for LockScopeGuard<G> {
+    fn drop(&mut self) {
+        self.guard.take();
+    }
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+    use std::sync::Mutex;
+
+    #[test]
+    fn test_lock_manager_creation() {
+        let manager = LockManager::new(true, Duration::from_secs(5));
+        assert!(manager.is_enabled());
+    }
+
+    #[test]
+    fn test_optimized_lock_guard() {
+        let manager = LockManager::new(true, Duration::from_secs(5));
+        let mutex = Mutex::new(42);
+        let guard = mutex.lock().unwrap();
+
+        let optimized = manager.optimize(guard, "test_resource");
+        assert!(!optimized.is_released());
+
+        let mut optimized = optimized;
+        optimized.early_release();
+        assert!(optimized.is_released());
+    }
+}

crates/concurrency/src/manager.rs

@@ -0,0 +1,315 @@
+// Copyright 2024 RustFS Team
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+//! Main concurrency manager
+
+use crate::config::ConcurrencyConfig;
+use std::sync::Arc;
+
+/// Snapshot of disk permit queue usage for GetObject orchestration.
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+pub struct GetObjectQueueSnapshot {
+    /// Total permits configured for disk reads.
+    pub total_permits: usize,
+    /// Permits currently in use.
+    pub permits_in_use: usize,
+}
+
+impl GetObjectQueueSnapshot {
+    /// Create a queue snapshot from total and available permits.
+    pub fn from_available_permits(total_permits: usize, available_permits: usize) -> Self {
+        Self {
+            total_permits,
+            permits_in_use: total_permits.saturating_sub(available_permits),
+        }
+    }
+
+    /// Return currently available permits.
+    pub fn permits_available(&self) -> usize {
+        self.total_permits.saturating_sub(self.permits_in_use)
+    }
+
+    /// Return queue utilization percentage in the 0-100 range.
+    pub fn utilization_percent(&self) -> f64 {
+        if self.total_permits == 0 {
+            0.0
+        } else {
+            (self.permits_in_use as f64 / self.total_permits as f64) * 100.0
+        }
+    }
+
+    /// Return whether the queue is considered congested.
+    pub fn is_congested(&self, threshold_percent: f64) -> bool {
+        self.utilization_percent() > threshold_percent
+    }
+}
+
+/// Main concurrency manager that provides access to all concurrency features
+pub struct ConcurrencyManager {
+    config: ConcurrencyConfig,
+
+    #[cfg(feature = "timeout")]
+    timeout: Arc<crate::timeout::TimeoutManager>,
+
+    #[cfg(feature = "lock")]
+    lock: Arc<crate::lock::LockManager>,
+
+    #[cfg(feature = "deadlock")]
+    deadlock: Arc<crate::deadlock::DeadlockManager>,
+
+    #[cfg(feature = "backpressure")]
+    backpressure: Arc<crate::backpressure::BackpressureManager>,
+
+    #[cfg(feature = "scheduler")]
+    scheduler: Arc<crate::scheduler::SchedulerManager>,
+}
+
+impl ConcurrencyManager {
+    /// Create a new concurrency manager with the given configuration
+    pub fn new(config: ConcurrencyConfig) -> Self {
+        // Validate configuration
+        if let Err(e) = config.validate() {
+            panic!("Invalid concurrency configuration: {}", e);
+        }
+
+        Self {
+            #[cfg(feature = "timeout")]
+            timeout: Arc::new(crate::timeout::TimeoutManager::new(
+                config.default_timeout,
+                config.max_timeout,
+                config.enable_dynamic_timeout,
+            )),
+
+            #[cfg(feature = "lock")]
+            lock: Arc::new(crate::lock::LockManager::new(
+                config.enable_lock_optimization,
+                config.lock_acquire_timeout,
+            )),
+
+            #[cfg(feature = "deadlock")]
+            deadlock: Arc::new(crate::deadlock::DeadlockManager::new(
+                config.enable_deadlock_detection,
+                config.deadlock_check_interval,
+                config.hang_threshold,
+            )),
+
+            #[cfg(feature = "backpressure")]
+            backpressure: Arc::new(crate::backpressure::BackpressureManager::new(
+                config.backpressure_buffer_size,
+                config.high_watermark,
+                config.low_watermark,
+            )),
+
+            #[cfg(feature = "scheduler")]
+            scheduler: Arc::new(crate::scheduler::SchedulerManager::new(
+                config.io_buffer_size,
+                config.max_buffer_size,
+                config.high_priority_threshold,
+                config.low_priority_threshold,
+            )),
+
+            config,
+        }
+    }
+
+    /// Create with default configuration
+    pub fn with_defaults() -> Self {
+        Self::new(ConcurrencyConfig::default())
+    }
+
+    /// Create from environment variables
+    pub fn from_env() -> Self {
+        Self::new(ConcurrencyConfig::from_env())
+    }
+
+    /// Get the configuration
+    pub fn config(&self) -> &ConcurrencyConfig {
+        &self.config
+    }
+
+    // ============================================
+    // Feature enable checks
+    // ============================================
+
+    /// Check if timeout feature is enabled
+    pub fn is_timeout_enabled(&self) -> bool {
+        #[cfg(feature = "timeout")]
+        {
+            self.config.features.timeout
+        }
+        #[cfg(not(feature = "timeout"))]
+        {
+            false
+        }
+    }
+
+    /// Check if lock feature is enabled
+    pub fn is_lock_enabled(&self) -> bool {
+        #[cfg(feature = "lock")]
+        {
+            self.config.features.lock
+        }
+        #[cfg(not(feature = "lock"))]
+        {
+            false
+        }
+    }
+
+    /// Check if deadlock feature is enabled
+    pub fn is_deadlock_enabled(&self) -> bool {
+        #[cfg(feature = "deadlock")]
+        {
+            self.config.features.deadlock
+        }
+        #[cfg(not(feature = "deadlock"))]
+        {
+            false
+        }
+    }
+
+    /// Check if backpressure feature is enabled
+    pub fn is_backpressure_enabled(&self) -> bool {
+        #[cfg(feature = "backpressure")]
+        {
+            self.config.features.backpressure
+        }
+        #[cfg(not(feature = "backpressure"))]
+        {
+            false
+        }
+    }
+
+    /// Check if scheduler feature is enabled
+    pub fn is_scheduler_enabled(&self) -> bool {
+        #[cfg(feature = "scheduler")]
+        {
+            self.config.features.scheduler
+        }
+        #[cfg(not(feature = "scheduler"))]
+        {
+            false
+        }
+    }
+
+    // ============================================
+    // Feature accessors
+    // ============================================
+
+    /// Get timeout manager
+    #[cfg(feature = "timeout")]
+    pub fn timeout(&self) -> Arc<crate::timeout::TimeoutManager> {
+        self.timeout.clone()
+    }
+
+    /// Get lock manager
+    #[cfg(feature = "lock")]
+    pub fn lock(&self) -> Arc<crate::lock::LockManager> {
+        self.lock.clone()
+    }
+
+    /// Get deadlock manager
+    #[cfg(feature = "deadlock")]
+    pub fn deadlock(&self) -> Arc<crate::deadlock::DeadlockManager> {
+        self.deadlock.clone()
+    }
+
+    /// Get backpressure manager
+    #[cfg(feature = "backpressure")]
+    pub fn backpressure(&self) -> Arc<crate::backpressure::BackpressureManager> {
+        self.backpressure.clone()
+    }
+
+    /// Get scheduler manager
+    #[cfg(feature = "scheduler")]
+    pub fn scheduler(&self) -> Arc<crate::scheduler::SchedulerManager> {
+        self.scheduler.clone()
+    }
+
+    // ============================================
+    // Lifecycle management
+    // ============================================
+
+    /// Start all enabled services (e.g., deadlock detection background task)
+    pub async fn start(&self) {
+        #[cfg(feature = "deadlock")]
+        {
+            if self.config.enable_deadlock_detection {
+                self.deadlock.start().await;
+            }
+        }
+
+        tracing::info!(
+            "Concurrency manager started (timeout={}, lock={}, deadlock={}, backpressure={}, scheduler={})",
+            self.is_timeout_enabled(),
+            self.is_lock_enabled(),
+            self.is_deadlock_enabled(),
+            self.is_backpressure_enabled(),
+            self.is_scheduler_enabled()
+        );
+    }
+
+    /// Stop all services
+    pub async fn stop(&self) {
+        #[cfg(feature = "deadlock")]
+        {
+            self.deadlock.stop().await;
+        }
+
+        tracing::info!("Concurrency manager stopped");
+    }
+}
+
+impl Default for ConcurrencyManager {
+    fn default() -> Self {
+        Self::with_defaults()
+    }
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    #[test]
+    fn test_queue_snapshot() {
+        let snapshot = GetObjectQueueSnapshot::from_available_permits(64, 16);
+        assert_eq!(snapshot.permits_in_use, 48);
+        assert_eq!(snapshot.permits_available(), 16);
+        assert!(snapshot.is_congested(70.0));
+    }
+
+    #[test]
+    fn test_manager_creation() {
+        let manager = ConcurrencyManager::with_defaults();
+        assert!(manager.config().validate().is_ok());
+    }
+
+    #[tokio::test]
+    async fn test_manager_lifecycle() {
+        let manager = ConcurrencyManager::with_defaults();
+        manager.start().await;
+        manager.stop().await;
+    }
+
+    #[test]
+    fn test_feature_checks() {
+        let manager = ConcurrencyManager::with_defaults();
+
+        // These should return the feature flag status
+        let _ = manager.is_timeout_enabled();
+        let _ = manager.is_lock_enabled();
+        let _ = manager.is_deadlock_enabled();
+        let _ = manager.is_backpressure_enabled();
+        let _ = manager.is_scheduler_enabled();
+    }
+}

crates/concurrency/src/scheduler.rs

@@ -0,0 +1,225 @@
+// Copyright 2024 RustFS Team
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+//! I/O scheduler management
+
+use rustfs_io_core::{
+    IoLoadLevel, IoPriority, IoScheduler as CoreIoScheduler, IoSchedulingContext,
+    io_profile::{AccessPattern, StorageMedia},
+};
+use rustfs_io_metrics::io_metrics;
+use std::sync::Arc;
+use std::time::Duration;
+
+/// Scheduler configuration
+#[derive(Debug, Clone)]
+pub struct SchedulerConfig {
+    /// Base buffer size
+    pub base_buffer_size: usize,
+    /// Maximum buffer size
+    pub max_buffer_size: usize,
+    /// High priority threshold
+    pub high_priority_threshold: usize,
+    /// Low priority threshold
+    pub low_priority_threshold: usize,
+}
+
+impl Default for SchedulerConfig {
+    fn default() -> Self {
+        Self {
+            base_buffer_size: 64 * 1024,              // 64KB
+            max_buffer_size: 4 * 1024 * 1024,         // 4MB
+            high_priority_threshold: 1024 * 1024,     // 1MB
+            low_priority_threshold: 10 * 1024 * 1024, // 10MB
+        }
+    }
+}
+
+/// Scheduler manager
+pub struct SchedulerManager {
+    config: SchedulerConfig,
+    scheduler: Arc<CoreIoScheduler>,
+}
+
+impl SchedulerManager {
+    /// Create a new scheduler manager
+    pub fn new(
+        base_buffer_size: usize,
+        max_buffer_size: usize,
+        high_priority_threshold: usize,
+        low_priority_threshold: usize,
+    ) -> Self {
+        let config = SchedulerConfig {
+            base_buffer_size,
+            max_buffer_size,
+            high_priority_threshold,
+            low_priority_threshold,
+        };
+
+        let core_config = rustfs_io_core::IoSchedulerConfig::default();
+
+        Self {
+            config,
+            scheduler: Arc::new(CoreIoScheduler::new(core_config)),
+        }
+    }
+
+    /// Get the configuration
+    pub fn config(&self) -> &SchedulerConfig {
+        &self.config
+    }
+
+    /// Get the scheduler
+    pub fn scheduler(&self) -> Arc<CoreIoScheduler> {
+        self.scheduler.clone()
+    }
+
+    /// Create an I/O strategy
+    pub fn create_strategy(&self) -> IoStrategy {
+        IoStrategy::new(self.config.clone(), self.scheduler.clone())
+    }
+
+    /// Calculate buffer size
+    pub fn calculate_buffer_size(
+        &self,
+        file_size: i64,
+        media: StorageMedia,
+        pattern: AccessPattern,
+        load: IoLoadLevel,
+        concurrent: usize,
+    ) -> usize {
+        let strategy = self.create_strategy();
+        strategy.calculate_buffer_size(file_size, media, pattern, load, concurrent)
+    }
+
+    /// Get I/O priority
+    pub fn get_priority(&self, size: i64) -> IoPriority {
+        IoPriority::from_size(size, self.config.high_priority_threshold, self.config.low_priority_threshold)
+    }
+}
+
+/// I/O strategy
+pub struct IoStrategy {
+    config: SchedulerConfig,
+    scheduler: Arc<CoreIoScheduler>,
+}
+
+impl IoStrategy {
+    fn new(config: SchedulerConfig, scheduler: Arc<CoreIoScheduler>) -> Self {
+        Self { config, scheduler }
+    }
+
+    /// Calculate buffer size with multi-factor strategy
+    pub fn calculate_buffer_size(
+        &self,
+        file_size: i64,
+        media: StorageMedia,
+        pattern: AccessPattern,
+        load: IoLoadLevel,
+        concurrent: usize,
+    ) -> usize {
+        // Create scheduling context
+        let _ctx = IoSchedulingContext::new(file_size, self.config.base_buffer_size)
+            .with_sequential(matches!(pattern, AccessPattern::Sequential))
+            .with_media(media);
+
+        // Get base buffer size from core scheduler
+        let permit_wait = Duration::from_millis(10); // Default wait time
+        let is_sequential = matches!(pattern, AccessPattern::Sequential);
+        let core_strategy = self.scheduler.calculate_strategy(file_size, permit_wait, is_sequential);
+        let base_size = core_strategy.buffer_size;
+
+        // Apply multi-factor adjustments
+        let adjusted_size = self.apply_adjustments(base_size, media, pattern, load, concurrent);
+
+        // Record metrics
+        io_metrics::record_io_scheduler_decision(adjusted_size, load.as_str(), pattern.as_str());
+
+        adjusted_size.min(self.config.max_buffer_size)
+    }
+
+    fn apply_adjustments(
+        &self,
+        base_size: usize,
+        media: StorageMedia,
+        pattern: AccessPattern,
+        load: IoLoadLevel,
+        concurrent: usize,
+    ) -> usize {
+        let mut size = base_size;
+
+        // Media adjustment
+        size = match media {
+            StorageMedia::Nvme => (size as f64 * 1.5) as usize,
+            StorageMedia::Ssd => (size as f64 * 1.2) as usize,
+            StorageMedia::Hdd => size,
+            _ => size,
+        };
+
+        // Pattern adjustment
+        size = match pattern {
+            AccessPattern::Sequential => (size as f64 * 1.5) as usize,
+            AccessPattern::Random => (size as f64 * 0.5) as usize,
+            _ => size,
+        };
+
+        // Load adjustment
+        size = match load {
+            IoLoadLevel::Low => (size as f64 * 1.2) as usize,
+            IoLoadLevel::Medium => size,
+            IoLoadLevel::High => (size as f64 * 0.7) as usize,
+            IoLoadLevel::Critical => (size as f64 * 0.5) as usize,
+        };
+
+        // Concurrency adjustment
+        if concurrent > 10 {
+            size = (size as f64 * 0.8) as usize;
+        }
+
+        size
+    }
+
+    /// Get the configuration
+    pub fn config(&self) -> &SchedulerConfig {
+        &self.config
+    }
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    #[test]
+    fn test_scheduler_config() {
+        let config = SchedulerConfig::default();
+        assert!(config.base_buffer_size < config.max_buffer_size);
+    }
+
+    #[test]
+    fn test_scheduler_manager() {
+        let manager = SchedulerManager::new(1024, 4096, 512, 2048);
+        let priority = manager.get_priority(100);
+        assert!(priority.is_high());
+    }
+
+    #[test]
+    fn test_io_strategy() {
+        let manager = SchedulerManager::new(1024, 4096, 512, 2048);
+        let strategy = manager.create_strategy();
+
+        let size = strategy.calculate_buffer_size(1024 * 1024, StorageMedia::Ssd, AccessPattern::Sequential, IoLoadLevel::Low, 1);
+
+        assert!(size > 0);
+    }
+}

crates/concurrency/src/timeout.rs

@@ -0,0 +1,150 @@
+// Copyright 2024 RustFS Team
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+//! Timeout management for operations
+
+use rustfs_io_core::{TimeoutError, calculate_adaptive_timeout};
+use std::time::{Duration, Instant};
+use tokio_util::sync::CancellationToken;
+
+/// Timeout configuration
+#[derive(Debug, Clone)]
+pub struct TimeoutConfig {
+    /// Default timeout duration
+    pub default_timeout: Duration,
+    /// Maximum timeout duration
+    pub max_timeout: Duration,
+    /// Enable dynamic timeout calculation
+    pub enable_dynamic: bool,
+}
+
+impl Default for TimeoutConfig {
+    fn default() -> Self {
+        Self {
+            default_timeout: Duration::from_secs(30),
+            max_timeout: Duration::from_secs(300),
+            enable_dynamic: true,
+        }
+    }
+}
+
+/// Timeout manager
+pub struct TimeoutManager {
+    config: TimeoutConfig,
+}
+
+impl TimeoutManager {
+    /// Create a new timeout manager
+    pub fn new(default_timeout: Duration, max_timeout: Duration, enable_dynamic: bool) -> Self {
+        Self {
+            config: TimeoutConfig {
+                default_timeout,
+                max_timeout,
+                enable_dynamic,
+            },
+        }
+    }
+
+    /// Get the configuration
+    pub fn config(&self) -> &TimeoutConfig {
+        &self.config
+    }
+
+    /// Calculate timeout for a given size
+    pub fn calculate_timeout(&self, size: u64, _history: &[Duration]) -> Duration {
+        if !self.config.enable_dynamic {
+            return self.config.default_timeout;
+        }
+
+        calculate_adaptive_timeout(self.config.default_timeout, None, 0, size).min(self.config.max_timeout)
+    }
+
+    /// Wrap an operation with timeout control
+    pub async fn wrap_operation<F, T, E>(&self, operation: F, timeout: Option<Duration>) -> Result<T, TimeoutError>
+    where
+        F: std::future::Future<Output = Result<T, E>>,
+        E: Into<TimeoutError>,
+    {
+        let timeout = timeout.unwrap_or(self.config.default_timeout);
+
+        match tokio::time::timeout(timeout, operation).await {
+            Ok(Ok(result)) => Ok(result),
+            Ok(Err(e)) => Err(e.into()),
+            Err(_) => Err(TimeoutError::TimedOut(timeout)),
+        }
+    }
+
+    /// Create a timeout guard for manual timeout control
+    pub fn create_guard(&self, timeout: Option<Duration>) -> TimeoutGuard {
+        TimeoutGuard::new(timeout.unwrap_or(self.config.default_timeout))
+    }
+}
+
+/// Timeout guard for manual timeout control
+pub struct TimeoutGuard {
+    timeout: Duration,
+    start: Instant,
+    cancel_token: CancellationToken,
+}
+
+impl TimeoutGuard {
+    fn new(timeout: Duration) -> Self {
+        Self {
+            timeout,
+            start: Instant::now(),
+            cancel_token: CancellationToken::new(),
+        }
+    }
+
+    /// Check if timeout has elapsed
+    pub fn is_timed_out(&self) -> bool {
+        self.start.elapsed() > self.timeout
+    }
+
+    /// Get remaining time
+    pub fn remaining(&self) -> Duration {
+        self.timeout.saturating_sub(self.start.elapsed())
+    }
+
+    /// Get the cancellation token
+    pub fn cancel_token(&self) -> CancellationToken {
+        self.cancel_token.clone()
+    }
+
+    /// Cancel the operation
+    pub fn cancel(&self) {
+        self.cancel_token.cancel();
+    }
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    #[test]
+    fn test_timeout_config() {
+        let config = TimeoutConfig::default();
+        assert!(config.default_timeout < config.max_timeout);
+    }
+
+    #[tokio::test]
+    async fn test_wrap_operation_success() {
+        let manager = TimeoutManager::new(Duration::from_secs(5), Duration::from_secs(10), true);
+
+        let result = manager.wrap_operation(async { Ok::<_, TimeoutError>(42) }, None).await;
+
+        assert!(result.is_ok());
+        assert_eq!(result.unwrap(), 42);
+    }
+}

crates/config/AGENTS.md

@@ -0,0 +1,28 @@
+# Config Crate Instructions
+
+Applies to `crates/config/`.
+
+## Environment Variable Naming
+
+- Global configuration variables must use flat `RUSTFS_*` names.
+- Do not introduce module-segmented names such as `RUSTFS_CONFIG_*`.
+
+Canonical examples:
+
+- `RUSTFS_REGION`
+- `RUSTFS_ADDRESS`
+- `RUSTFS_VOLUMES`
+- `RUSTFS_LICENSE`
+- `RUSTFS_SCANNER_ENABLED`
+- `RUSTFS_HEAL_ENABLED`
+
+## Compatibility
+
+- Deprecated aliases must keep warning behavior.
+- Document aliases in `crates/config/README.md`.
+- Any alias change should include tests for both canonical and deprecated forms.
+
+## Source of Truth
+
+- Constants: `crates/config/src/constants/app.rs`
+- Naming conventions: `crates/config/README.md#environment-variable-naming-conventions`

crates/config/Cargo.toml

@@ -37,3 +37,6 @@ constants = ["dep:const-str"]
 notify = ["dep:const-str", "constants"]
 observability = ["constants"]
 opa = ["constants"]
+
+[lib]
+doctest = false

crates/config/README.md

@@ -32,6 +32,30 @@
 
 For comprehensive documentation, examples, and usage guides, please visit the main [RustFS repository](https://github.com/rustfs/rustfs).
 
+## Environment Variable Naming Conventions
+
+RustFS uses a flat naming style for top-level configuration: environment variables are `RUSTFS_*` without nested module segments.
+
+Examples:
+- `RUSTFS_REGION`
+- `RUSTFS_ADDRESS`
+- `RUSTFS_VOLUMES`
+- `RUSTFS_LICENSE`
+
+Current guidance:
+- Prefer module-specific names only when they are not top-level product configuration.
+- Renamed variables must keep backward-compatible aliases until before beta.
+- Alias usage must emit deprecation warnings and be treated as transitional only.
+- Deprecated example:
+  - `RUSTFS_ENABLE_SCANNER` -> `RUSTFS_SCANNER_ENABLED`
+  - `RUSTFS_ENABLE_HEAL` -> `RUSTFS_HEAL_ENABLED`
+  - `RUSTFS_DATA_SCANNER_START_DELAY_SECS` -> `RUSTFS_SCANNER_START_DELAY_SECS`
+
+## Scanner environment aliases
+
+- `RUSTFS_SCANNER_START_DELAY_SECS` (canonical)
+- `RUSTFS_DATA_SCANNER_START_DELAY_SECS` (deprecated alias for compatibility)
+
 ## 📄 License
 
 This project is licensed under the Apache License 2.0 - see the [LICENSE](../../LICENSE) file for details.

crates/config/src/audit/mqtt.rs

@@ -23,9 +23,15 @@ pub const ENV_AUDIT_MQTT_RECONNECT_INTERVAL: &str = "RUSTFS_AUDIT_MQTT_RECONNECT
 pub const ENV_AUDIT_MQTT_KEEP_ALIVE_INTERVAL: &str = "RUSTFS_AUDIT_MQTT_KEEP_ALIVE_INTERVAL";
 pub const ENV_AUDIT_MQTT_QUEUE_DIR: &str = "RUSTFS_AUDIT_MQTT_QUEUE_DIR";
 pub const ENV_AUDIT_MQTT_QUEUE_LIMIT: &str = "RUSTFS_AUDIT_MQTT_QUEUE_LIMIT";
+pub const ENV_AUDIT_MQTT_TLS_POLICY: &str = "RUSTFS_AUDIT_MQTT_TLS_POLICY";
+pub const ENV_AUDIT_MQTT_TLS_CA: &str = "RUSTFS_AUDIT_MQTT_TLS_CA";
+pub const ENV_AUDIT_MQTT_TLS_CLIENT_CERT: &str = "RUSTFS_AUDIT_MQTT_TLS_CLIENT_CERT";
+pub const ENV_AUDIT_MQTT_TLS_CLIENT_KEY: &str = "RUSTFS_AUDIT_MQTT_TLS_CLIENT_KEY";
+pub const ENV_AUDIT_MQTT_TLS_TRUST_LEAF_AS_CA: &str = "RUSTFS_AUDIT_MQTT_TLS_TRUST_LEAF_AS_CA";
+pub const ENV_AUDIT_MQTT_WS_PATH_ALLOWLIST: &str = "RUSTFS_AUDIT_MQTT_WS_PATH_ALLOWLIST";
 
 /// A list of all valid configuration keys for an MQTT target.
-pub const ENV_AUDIT_MQTT_KEYS: &[&str; 10] = &[
+pub const ENV_AUDIT_MQTT_KEYS: &[&str; 16] = &[
     ENV_AUDIT_MQTT_ENABLE,
     ENV_AUDIT_MQTT_BROKER,
     ENV_AUDIT_MQTT_TOPIC,
@@ -36,6 +42,12 @@ pub const ENV_AUDIT_MQTT_KEYS: &[&str; 10] = &[
     ENV_AUDIT_MQTT_KEEP_ALIVE_INTERVAL,
     ENV_AUDIT_MQTT_QUEUE_DIR,
     ENV_AUDIT_MQTT_QUEUE_LIMIT,
+    ENV_AUDIT_MQTT_TLS_POLICY,
+    ENV_AUDIT_MQTT_TLS_CA,
+    ENV_AUDIT_MQTT_TLS_CLIENT_CERT,
+    ENV_AUDIT_MQTT_TLS_CLIENT_KEY,
+    ENV_AUDIT_MQTT_TLS_TRUST_LEAF_AS_CA,
+    ENV_AUDIT_MQTT_WS_PATH_ALLOWLIST,
 ];
 
 /// A list of all valid configuration keys for an MQTT target.
@@ -50,5 +62,11 @@ pub const AUDIT_MQTT_KEYS: &[&str] = &[
     crate::MQTT_KEEP_ALIVE_INTERVAL,
     crate::MQTT_QUEUE_DIR,
     crate::MQTT_QUEUE_LIMIT,
+    crate::MQTT_TLS_POLICY,
+    crate::MQTT_TLS_CA,
+    crate::MQTT_TLS_CLIENT_CERT,
+    crate::MQTT_TLS_CLIENT_KEY,
+    crate::MQTT_TLS_TRUST_LEAF_AS_CA,
+    crate::MQTT_WS_PATH_ALLOWLIST,
     crate::COMMENT_KEY,
 ];

crates/config/src/constants/app.rs

@@ -128,6 +128,70 @@ pub const RUSTFS_LICENSE_URL: &str = "https://www.apache.org/licenses/LICENSE-2.
 /// Example: RUSTFS_ADDRESS=":9000"
 pub const ENV_RUSTFS_ADDRESS: &str = "RUSTFS_ADDRESS";
 
+/// Environment variable for server volumes.
+pub const ENV_RUSTFS_VOLUMES: &str = "RUSTFS_VOLUMES";
+
+/// Environment variable for server access key.
+pub const ENV_RUSTFS_ACCESS_KEY: &str = "RUSTFS_ACCESS_KEY";
+
+/// Environment variable for server access key file.
+pub const ENV_RUSTFS_ACCESS_KEY_FILE: &str = "RUSTFS_ACCESS_KEY_FILE";
+
+/// Environment variable for server root user.
+pub const ENV_RUSTFS_ROOT_USER: &str = "RUSTFS_ROOT_USER";
+
+/// Environment variable for server secret key.
+pub const ENV_RUSTFS_SECRET_KEY: &str = "RUSTFS_SECRET_KEY";
+
+/// Environment variable for server secret key file.
+pub const ENV_RUSTFS_SECRET_KEY_FILE: &str = "RUSTFS_SECRET_KEY_FILE";
+
+/// Environment variable for server root password.
+pub const ENV_RUSTFS_ROOT_PASSWORD: &str = "RUSTFS_ROOT_PASSWORD";
+
+/// Environment variable for server OBS endpoint.
+pub const ENV_RUSTFS_OBS_ENDPOINT: &str = "RUSTFS_OBS_ENDPOINT";
+
+/// Environment variable for console server enable.
+pub const ENV_RUSTFS_CONSOLE_ENABLE: &str = "RUSTFS_CONSOLE_ENABLE";
+
+/// Environment variable for console server address.
+pub const ENV_RUSTFS_CONSOLE_ADDRESS: &str = "RUSTFS_CONSOLE_ADDRESS";
+
+/// Environment variable for server tls path.
+pub const ENV_RUSTFS_TLS_PATH: &str = "RUSTFS_TLS_PATH";
+
+/// Environment variable for server KMS enable.
+pub const ENV_RUSTFS_KMS_ENABLE: &str = "RUSTFS_KMS_ENABLE";
+
+/// Default KMS enable for server-side encryption
+/// This is the default value for enabling KMS encryption for server-side encryption.
+/// Default value: false
+pub const DEFAULT_KMS_ENABLE: bool = false;
+
+/// Environment variable for server KMS backend.
+pub const ENV_RUSTFS_KMS_BACKEND: &str = "RUSTFS_KMS_BACKEND";
+
+/// Environment variable for Vault Transit mount path.
+pub const ENV_RUSTFS_KMS_VAULT_MOUNT_PATH: &str = "RUSTFS_KMS_VAULT_MOUNT_PATH";
+
+/// Default KMS backend for server-side encryption
+/// This is the default KMS backend for server-side encryption.
+/// Default value: local
+pub const DEFAULT_KMS_BACKEND: &str = "local";
+
+/// Environment variable for selecting the buffer profile used for adaptive buffer sizing.
+pub const ENV_RUSTFS_BUFFER_PROFILE: &str = "RUSTFS_BUFFER_PROFILE";
+
+/// Default buffer profile for adaptive buffer sizing
+/// This is the default buffer profile for adaptive buffer sizing.
+/// It is used to identify the workload profile for adaptive buffer sizing.
+/// Default value: GeneralPurpose
+pub const DEFAULT_BUFFER_PROFILE: &str = "GeneralPurpose";
+
+/// Default value for the server TLS path if `ENV_RUSTFS_TLS_PATH` is not set.
+pub const DEFAULT_RUSTFS_TLS_PATH: &str = "";
+
 /// Default port for rustfs
 /// This is the default port for rustfs.
 /// This is used to bind the server to a specific port.
@@ -153,6 +217,12 @@ pub const DEFAULT_CONSOLE_ADDRESS: &str = concat!(":", DEFAULT_CONSOLE_PORT);
 /// Default value: us-east-1
 pub const RUSTFS_REGION: &str = "us-east-1";
 
+/// Environment variable for server region.
+pub const ENV_RUSTFS_REGION: &str = "RUSTFS_REGION";
+
+/// Environment variable for server license.
+pub const ENV_RUSTFS_LICENSE: &str = "RUSTFS_LICENSE";
+
 /// Default log filename for rustfs
 /// This is the default log filename for rustfs.
 /// It is used to store the logs of the application.
@@ -212,6 +282,12 @@ pub const DEFAULT_OBS_METRICS_EXPORT_ENABLED: bool = true;
 /// Environment variable: RUSTFS_OBS_LOGS_EXPORT_ENABLED
 pub const DEFAULT_OBS_LOGS_EXPORT_ENABLED: bool = true;
 
+/// Default profiling export enabled
+/// It is used to enable or disable exporting profiles
+/// Default value: true
+/// Environment variable: RUSTFS_OBS_PROFILING_EXPORT_ENABLED
+pub const DEFAULT_OBS_PROFILING_EXPORT_ENABLED: bool = true;
+
 /// Default log local logging enabled for rustfs
 /// This is the default log local logging enabled for rustfs.
 /// It is used to enable or disable local logging of the application.

crates/config/src/constants/capacity.rs

@@ -0,0 +1,155 @@
+// Copyright 2024 RustFS Team
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+//! Capacity calculation configuration constants
+
+// ============================================================================
+// Environment Variable Names
+// ============================================================================
+
+/// Environment variable for scheduled update interval
+pub const ENV_CAPACITY_SCHEDULED_INTERVAL: &str = "RUSTFS_CAPACITY_SCHEDULED_INTERVAL";
+
+/// Environment variable for write trigger delay
+pub const ENV_CAPACITY_WRITE_TRIGGER_DELAY: &str = "RUSTFS_CAPACITY_WRITE_TRIGGER_DELAY";
+
+/// Environment variable for write frequency threshold
+pub const ENV_CAPACITY_WRITE_FREQUENCY_THRESHOLD: &str = "RUSTFS_CAPACITY_WRITE_FREQUENCY_THRESHOLD";
+
+/// Environment variable for fast update threshold
+pub const ENV_CAPACITY_FAST_UPDATE_THRESHOLD: &str = "RUSTFS_CAPACITY_FAST_UPDATE_THRESHOLD";
+
+/// Environment variable for max files threshold
+pub const ENV_CAPACITY_MAX_FILES_THRESHOLD: &str = "RUSTFS_CAPACITY_MAX_FILES_THRESHOLD";
+
+/// Environment variable for statistics timeout
+pub const ENV_CAPACITY_STAT_TIMEOUT: &str = "RUSTFS_CAPACITY_STAT_TIMEOUT";
+
+/// Environment variable for sample rate
+pub const ENV_CAPACITY_SAMPLE_RATE: &str = "RUSTFS_CAPACITY_SAMPLE_RATE";
+
+/// Environment variable for following symbolic links during capacity calculation
+pub const ENV_CAPACITY_FOLLOW_SYMLINKS: &str = "RUSTFS_CAPACITY_FOLLOW_SYMLINKS";
+
+/// Environment variable for maximum symlink follow depth
+pub const ENV_CAPACITY_MAX_SYMLINK_DEPTH: &str = "RUSTFS_CAPACITY_MAX_SYMLINK_DEPTH";
+
+/// Environment variable for enabling dynamic timeout calculation
+pub const ENV_CAPACITY_ENABLE_DYNAMIC_TIMEOUT: &str = "RUSTFS_CAPACITY_ENABLE_DYNAMIC_TIMEOUT";
+
+/// Environment variable for minimum capacity calculation timeout
+pub const ENV_CAPACITY_MIN_TIMEOUT: &str = "RUSTFS_CAPACITY_MIN_TIMEOUT";
+
+/// Environment variable for maximum capacity calculation timeout
+pub const ENV_CAPACITY_MAX_TIMEOUT: &str = "RUSTFS_CAPACITY_MAX_TIMEOUT";
+
+/// Environment variable for progress stall detection timeout
+pub const ENV_CAPACITY_STALL_TIMEOUT: &str = "RUSTFS_CAPACITY_STALL_TIMEOUT";
+
+// ============================================================================
+// Default Values
+// ============================================================================
+
+/// Scheduled update interval in seconds
+/// Default: 120 seconds (2 minutes)
+pub const DEFAULT_SCHEDULED_UPDATE_INTERVAL_SECS: u64 = 120;
+
+/// Write trigger delay in seconds
+/// Default: 5 seconds
+pub const DEFAULT_WRITE_TRIGGER_DELAY_SECS: u64 = 5;
+
+/// Write frequency threshold (writes per minute)
+/// Default: 5 writes/minute
+pub const DEFAULT_WRITE_FREQUENCY_THRESHOLD: usize = 5;
+
+/// Fast update threshold in seconds
+/// Default: 30 seconds
+pub const DEFAULT_FAST_UPDATE_THRESHOLD_SECS: u64 = 30;
+
+/// Maximum files threshold for sampling
+/// Default: 200,000 files
+pub const DEFAULT_MAX_FILES_THRESHOLD: usize = 200_000;
+
+/// Statistics timeout in seconds
+/// Default: 3 seconds
+pub const DEFAULT_STAT_TIMEOUT_SECS: u64 = 3;
+
+/// Sampling rate (1 in every N files)
+/// Default: 200
+pub const DEFAULT_SAMPLE_RATE: usize = 200;
+
+/// Follow symbolic links during capacity calculation
+/// Default: false (disabled for safety)
+pub const DEFAULT_CAPACITY_FOLLOW_SYMLINKS: bool = false;
+
+/// Maximum symlink follow depth
+/// Default: 3 levels
+pub const DEFAULT_CAPACITY_MAX_SYMLINK_DEPTH: u8 = 3;
+
+/// Enable dynamic timeout calculation based on directory characteristics
+/// Default: true (enabled)
+pub const DEFAULT_CAPACITY_ENABLE_DYNAMIC_TIMEOUT: bool = true;
+
+/// Minimum capacity calculation timeout in seconds
+/// Default: 2 seconds
+pub const DEFAULT_CAPACITY_MIN_TIMEOUT_SECS: u64 = 2;
+
+/// Maximum capacity calculation timeout in seconds
+/// Default: 15 seconds
+pub const DEFAULT_CAPACITY_MAX_TIMEOUT_SECS: u64 = 15;
+
+/// Progress stall detection timeout in seconds
+/// Default: 20 seconds
+pub const DEFAULT_CAPACITY_STALL_TIMEOUT_SECS: u64 = 20;
+
+// ============================================================================
+// Tests
+// ============================================================================
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    #[test]
+    fn test_env_var_names() {
+        assert_eq!(ENV_CAPACITY_SCHEDULED_INTERVAL, "RUSTFS_CAPACITY_SCHEDULED_INTERVAL");
+        assert_eq!(ENV_CAPACITY_WRITE_TRIGGER_DELAY, "RUSTFS_CAPACITY_WRITE_TRIGGER_DELAY");
+        assert_eq!(ENV_CAPACITY_WRITE_FREQUENCY_THRESHOLD, "RUSTFS_CAPACITY_WRITE_FREQUENCY_THRESHOLD");
+        assert_eq!(ENV_CAPACITY_FAST_UPDATE_THRESHOLD, "RUSTFS_CAPACITY_FAST_UPDATE_THRESHOLD");
+        assert_eq!(ENV_CAPACITY_MAX_FILES_THRESHOLD, "RUSTFS_CAPACITY_MAX_FILES_THRESHOLD");
+        assert_eq!(ENV_CAPACITY_STAT_TIMEOUT, "RUSTFS_CAPACITY_STAT_TIMEOUT");
+        assert_eq!(ENV_CAPACITY_SAMPLE_RATE, "RUSTFS_CAPACITY_SAMPLE_RATE");
+        assert_eq!(ENV_CAPACITY_FOLLOW_SYMLINKS, "RUSTFS_CAPACITY_FOLLOW_SYMLINKS");
+        assert_eq!(ENV_CAPACITY_MAX_SYMLINK_DEPTH, "RUSTFS_CAPACITY_MAX_SYMLINK_DEPTH");
+        assert_eq!(ENV_CAPACITY_ENABLE_DYNAMIC_TIMEOUT, "RUSTFS_CAPACITY_ENABLE_DYNAMIC_TIMEOUT");
+        assert_eq!(ENV_CAPACITY_MIN_TIMEOUT, "RUSTFS_CAPACITY_MIN_TIMEOUT");
+        assert_eq!(ENV_CAPACITY_MAX_TIMEOUT, "RUSTFS_CAPACITY_MAX_TIMEOUT");
+        assert_eq!(ENV_CAPACITY_STALL_TIMEOUT, "RUSTFS_CAPACITY_STALL_TIMEOUT");
+    }
+
+    #[test]
+    fn test_default_values() {
+        assert_eq!(DEFAULT_SCHEDULED_UPDATE_INTERVAL_SECS, 120);
+        assert_eq!(DEFAULT_WRITE_TRIGGER_DELAY_SECS, 5);
+        assert_eq!(DEFAULT_WRITE_FREQUENCY_THRESHOLD, 5);
+        assert_eq!(DEFAULT_FAST_UPDATE_THRESHOLD_SECS, 30);
+        assert_eq!(DEFAULT_MAX_FILES_THRESHOLD, 200_000);
+        assert_eq!(DEFAULT_STAT_TIMEOUT_SECS, 3);
+        assert_eq!(DEFAULT_SAMPLE_RATE, 200);
+        assert_eq!(DEFAULT_CAPACITY_MAX_SYMLINK_DEPTH, 3);
+        assert_eq!(DEFAULT_CAPACITY_MIN_TIMEOUT_SECS, 2);
+        assert_eq!(DEFAULT_CAPACITY_MAX_TIMEOUT_SECS, 15);
+        assert_eq!(DEFAULT_CAPACITY_STALL_TIMEOUT_SECS, 20);
+    }
+}

crates/config/src/constants/mod.rs

@@ -14,6 +14,7 @@
 
 pub(crate) mod app;
 pub(crate) mod body_limits;
+pub(crate) mod capacity;
 pub(crate) mod compress;
 pub(crate) mod console;
 pub(crate) mod env;
@@ -28,3 +29,5 @@ pub(crate) mod runtime;
 pub(crate) mod scanner;
 pub(crate) mod targets;
 pub(crate) mod tls;
+pub(crate) mod workload;
+pub(crate) mod zero_copy;

crates/config/src/constants/object.rs

@@ -12,62 +12,6 @@
 // See the License for the specific language governing permissions and
 // limitations under the License.
 
-/// Environment variable name to toggle object-level in-memory caching.
-///
-/// - Purpose: Enable or disable the object-level in-memory cache (moka).
-/// - Acceptable values: `"true"` / `"false"` (case-insensitive) or a boolean typed config.
-/// - Semantics: When enabled, the system keeps fully-read objects in memory to reduce backend requests; when disabled, reads bypass the object cache.
-/// - Example: `export RUSTFS_OBJECT_CACHE_ENABLE=true`
-/// - Note: Evaluate together with `RUSTFS_OBJECT_CACHE_CAPACITY_MB`, TTL/TTI and concurrency thresholds to balance memory usage and throughput.
-pub const ENV_OBJECT_CACHE_ENABLE: &str = "RUSTFS_OBJECT_CACHE_ENABLE";
-
-/// Environment variable name that specifies the object cache capacity in megabytes.
-///
-/// - Purpose: Set the maximum total capacity of the object cache (in MB).
-/// - Unit: MB (1 MB = 1_048_576 bytes).
-/// - Valid values: any positive integer (0 may indicate disabled or alternative handling).
-/// - Semantics: When the moka cache reaches this capacity, eviction policies will remove entries; tune according to available memory and object size distribution.
-/// - Example: `export RUSTFS_OBJECT_CACHE_CAPACITY_MB=512`
-/// - Note: Actual memory usage will be slightly higher due to object headers and indexing overhead.
-pub const ENV_OBJECT_CACHE_CAPACITY_MB: &str = "RUSTFS_OBJECT_CACHE_CAPACITY_MB";
-
-/// Environment variable name for maximum object size eligible for caching in megabytes.
-///
-/// - Purpose: Define the upper size limit for individual objects to be considered for caching.
-/// - Unit: MB (1 MB = 1_048_576 bytes).
-/// - Valid values: any positive integer; objects larger than this size will not be cached.
-/// - Semantics: Prevents caching of excessively large objects that could monopolize cache capacity; tune based on typical object size distribution.
-/// - Example: `export RUSTFS_OBJECT_CACHE_MAX_OBJECT_SIZE_MB=50`
-/// - Note: Setting this too low may reduce cache effectiveness; setting it too high may lead to inefficient memory usage.
-pub const ENV_OBJECT_CACHE_MAX_OBJECT_SIZE_MB: &str = "RUSTFS_OBJECT_CACHE_MAX_OBJECT_SIZE_MB";
-
-/// Environment variable name for object cache TTL (time-to-live) in seconds.
-///
-/// - Purpose: Specify the maximum lifetime of a cached entry from the moment it is written.
-/// - Unit: seconds (u64).
-/// - Semantics: TTL acts as a hard upper bound; entries older than TTL are considered expired and removed by periodic cleanup.
-/// - Example: `export RUSTFS_OBJECT_CACHE_TTL_SECS=300`
-/// - Note: TTL and TTI both apply; either policy can cause eviction.
-pub const ENV_OBJECT_CACHE_TTL_SECS: &str = "RUSTFS_OBJECT_CACHE_TTL_SECS";
-
-/// Environment variable name for object cache TTI (time-to-idle) in seconds.
-///
-/// - Purpose: Specify how long an entry may remain in cache without being accessed before it is evicted.
-/// - Unit: seconds (u64).
-/// - Semantics: TTI helps remove one-time or infrequently used entries; frequent accesses reset idle timers but do not extend beyond TTL unless additional logic exists.
-/// - Example: `export RUSTFS_OBJECT_CACHE_TTI_SECS=120`
-/// - Note: Works together with TTL to keep the cache populated with actively used objects.
-pub const ENV_OBJECT_CACHE_TTI_SECS: &str = "RUSTFS_OBJECT_CACHE_TTI_SECS";
-
-/// Environment variable name for threshold of "hot" object hit count used to extend life.
-///
-/// - Purpose: Define a hit-count threshold to mark objects as "hot" so they may be treated preferentially near expiration.
-/// - Valid values: positive integer (usize).
-/// - Semantics: Objects reaching this hit count can be considered for relaxed eviction to avoid thrashing hot items.
-/// - Example: `export RUSTFS_OBJECT_HOT_MIN_HITS_TO_EXTEND=5`
-/// - Note: This is an optional enhancement and requires cache-layer statistics and extension logic to take effect.
-pub const ENV_OBJECT_HOT_MIN_HITS_TO_EXTEND: &str = "RUSTFS_OBJECT_HOT_MIN_HITS_TO_EXTEND";
-
 /// Environment variable name for high concurrency threshold used in adaptive buffering.
 ///
 /// - Purpose: When concurrent request count exceeds this threshold, the system enters a "high concurrency" optimization mode to reduce per-request buffer sizes.
@@ -94,24 +38,6 @@ pub const ENV_OBJECT_MEDIUM_CONCURRENCY_THRESHOLD: &str = "RUSTFS_OBJECT_MEDIUM_
 /// - Note: This setting may interact with OS-level I/O scheduling and should be tuned based on hardware capabilities.
 pub const ENV_OBJECT_MAX_CONCURRENT_DISK_READS: &str = "RUSTFS_OBJECT_MAX_CONCURRENT_DISK_READS";
 
-/// Default: object caching is disabled.
-///
-/// - Semantics: Safe default to avoid unexpected memory usage or cache consistency concerns when not explicitly enabled.
-/// - Default is set to false (disabled).
-pub const DEFAULT_OBJECT_CACHE_ENABLE: bool = false;
-
-/// Default object cache capacity in MB.
-///
-/// - Default: 100 MB (can be overridden by `RUSTFS_OBJECT_CACHE_CAPACITY_MB`).
-/// - Note: Choose a conservative default to reduce memory pressure in development/testing.
-pub const DEFAULT_OBJECT_CACHE_CAPACITY_MB: u64 = 100;
-
-/// Default maximum object size eligible for caching in MB.
-///
-/// - Default: 10 MB (can be overridden by `RUSTFS_OBJECT_CACHE_MAX_OBJECT_SIZE_MB`).
-/// - Note: Balances caching effectiveness with memory usage.
-pub const DEFAULT_OBJECT_CACHE_MAX_OBJECT_SIZE_MB: usize = 10;
-
 /// Maximum concurrent requests before applying aggressive optimization.
 ///
 /// When concurrent requests exceed this threshold (>8), the system switches to
@@ -141,29 +67,437 @@ pub const DEFAULT_OBJECT_MEDIUM_CONCURRENCY_THRESHOLD: usize = 4;
 /// Default is set to 64 concurrent reads.
 pub const DEFAULT_OBJECT_MAX_CONCURRENT_DISK_READS: usize = 64;
 
-/// Time-to-live for cached objects (5 minutes = 300 seconds).
+/// Skip bitrot hash verification on GetObject reads.
 ///
-/// After this duration, cached objects are automatically expired by Moka's
-/// background cleanup process, even if they haven't been accessed. This prevents
-/// stale data from consuming cache capacity indefinitely.
+/// When enabled, GetObject reads skip the per-shard hash
+/// computation and comparison, reducing CPU usage on the read path.
+/// The background scanner still performs full integrity verification.
+/// Does not affect writes, heals, or scanner operations.
 ///
-/// Default is set to 300 seconds.
-pub const DEFAULT_OBJECT_CACHE_TTL_SECS: u64 = 300;
+/// Default is false (verify on every read, matching pre-existing behavior).
+pub const ENV_OBJECT_GET_SKIP_BITROT_VERIFY: &str = "RUSTFS_OBJECT_GET_SKIP_BITROT_VERIFY";
 
-/// Time-to-idle for cached objects (2 minutes = 120 seconds).
-///
-/// Objects that haven't been accessed for this duration are automatically evicted,
-/// even if their TTL hasn't expired. This ensures cache is populated with actively
-/// used objects and clears out one-time reads efficiently.
-///
-/// Default is set to 120 seconds.
-pub const DEFAULT_OBJECT_CACHE_TTI_SECS: u64 = 120;
+/// Default: bitrot verification is enabled on GetObject reads (do not skip).
+pub const DEFAULT_OBJECT_GET_SKIP_BITROT_VERIFY: bool = false;
 
-/// Minimum hit count to extend object lifetime beyond TTL.
+// =============================================================================
+// Concurrent Request Fix - Timeout and Backpressure Configuration
+// =============================================================================
+
+/// Environment variable for GetObject request timeout in seconds.
 ///
-/// "Hot" objects that have been accessed at least this many times are treated
-/// specially - they can survive longer in cache even as they approach TTL expiration.
-/// This prevents frequently accessed objects from being evicted prematurely.
+/// When a GetObject request exceeds this duration, it will be cancelled
+/// and return a 504 Gateway Timeout error. This prevents requests from
+/// hanging indefinitely due to deadlocks or resource exhaustion.
 ///
-/// Default is set to 5 hits.
-pub const DEFAULT_OBJECT_HOT_MIN_HITS_TO_EXTEND: usize = 5;
+/// Default: 30 seconds (can be overridden by `RUSTFS_OBJECT_GET_TIMEOUT`).
+/// Set to 0 to disable timeout (not recommended for production).
+pub const ENV_OBJECT_GET_TIMEOUT: &str = "RUSTFS_OBJECT_GET_TIMEOUT";
+
+/// Default GetObject request timeout in seconds.
+///
+/// This value balances between allowing large object transfers to complete
+/// and preventing indefinite hangs. For 20-26MB objects with concurrent
+/// range reads, 30 seconds should be sufficient under normal conditions.
+pub const DEFAULT_OBJECT_GET_TIMEOUT: u64 = 30;
+
+/// Environment variable for disk read operation timeout in seconds.
+///
+/// Individual disk read operations that exceed this duration will be
+/// cancelled and treated as failures. This helps detect slow or hung
+/// disks without waiting indefinitely.
+///
+/// Default: 10 seconds (can be overridden by `RUSTFS_OBJECT_DISK_READ_TIMEOUT`).
+pub const ENV_OBJECT_DISK_READ_TIMEOUT: &str = "RUSTFS_OBJECT_DISK_READ_TIMEOUT";
+
+/// Default disk read timeout in seconds.
+pub const DEFAULT_OBJECT_DISK_READ_TIMEOUT: u64 = 10;
+
+/// Environment variable for minimum GetObject timeout in seconds.
+///
+/// When dynamic timeout calculation is enabled, this is the minimum timeout
+/// that will be used regardless of object size. This prevents excessively
+/// short timeouts for very small objects.
+///
+/// Default: 5 seconds (can be overridden by `RUSTFS_OBJECT_MIN_TIMEOUT`).
+pub const ENV_OBJECT_MIN_TIMEOUT: &str = "RUSTFS_OBJECT_MIN_TIMEOUT";
+
+/// Default minimum GetObject timeout: 5 seconds.
+pub const DEFAULT_OBJECT_MIN_TIMEOUT: u64 = 5;
+
+/// Environment variable for maximum GetObject timeout in seconds.
+///
+/// When dynamic timeout calculation is enabled, this is the maximum timeout
+/// that will be used regardless of object size. This prevents excessively
+/// long timeouts for very large objects.
+///
+/// Default: 300 seconds (5 minutes, can be overridden by `RUSTFS_OBJECT_MAX_TIMEOUT`).
+pub const ENV_OBJECT_MAX_TIMEOUT: &str = "RUSTFS_OBJECT_MAX_TIMEOUT";
+
+/// Default maximum GetObject timeout: 300 seconds (5 minutes).
+pub const DEFAULT_OBJECT_MAX_TIMEOUT: u64 = 300;
+
+/// Environment variable for default bytes per second for timeout estimation.
+///
+/// This value is used to estimate timeout duration based on object size when
+/// dynamic timeout calculation is enabled. The timeout is calculated as:
+/// (object_size / bytes_per_second) * buffer_factor
+///
+/// Default: 1048576 (1 MB/s, can be overridden by `RUSTFS_OBJECT_BYTES_PER_SECOND`).
+pub const ENV_OBJECT_BYTES_PER_SECOND: &str = "RUSTFS_OBJECT_BYTES_PER_SECOND";
+
+/// Default bytes per second for timeout estimation: 1 MB/s.
+pub const DEFAULT_OBJECT_BYTES_PER_SECOND: u64 = 1024 * 1024;
+
+/// Environment variable to enable dynamic timeout calculation.
+///
+/// When enabled, timeout is calculated based on object size and transfer speed
+/// rather than using a fixed timeout value. This provides better timeout
+/// handling for objects of varying sizes.
+///
+/// Default: true (enabled, can be overridden by `RUSTFS_OBJECT_DYNAMIC_TIMEOUT_ENABLE`).
+pub const ENV_OBJECT_DYNAMIC_TIMEOUT_ENABLE: &str = "RUSTFS_OBJECT_DYNAMIC_TIMEOUT_ENABLE";
+
+/// Default: dynamic timeout calculation is enabled.
+pub const DEFAULT_OBJECT_DYNAMIC_TIMEOUT_ENABLE: bool = true;
+
+/// Environment variable for duplex pipe buffer size in bytes.
+///
+/// The duplex pipe connects the disk read task to the HTTP response stream.
+/// A larger buffer reduces backpressure but increases memory usage.
+/// For large objects (20-26MB), a 4MB buffer provides good throughput.
+///
+/// Default: 4194304 (4 MB, can be overridden by `RUSTFS_OBJECT_DUPLEX_BUFFER_SIZE`).
+/// Minimum recommended: 1048576 (1 MB).
+pub const ENV_OBJECT_DUPLEX_BUFFER_SIZE: &str = "RUSTFS_OBJECT_DUPLEX_BUFFER_SIZE";
+
+/// Default duplex buffer size: 4 MB.
+///
+/// This is 4x larger than the original 1 MB buffer, providing better
+/// handling of large objects and reducing backpressure-related hangs.
+pub const DEFAULT_OBJECT_DUPLEX_BUFFER_SIZE: usize = 4 * 1024 * 1024;
+
+/// Environment variable for I/O buffer size in bytes.
+///
+/// This controls the buffer size used for individual I/O operations.
+/// A larger buffer improves throughput for sequential reads but may
+/// increase latency for small random reads.
+///
+/// Default: 131072 (128 KB, can be overridden by `RUSTFS_OBJECT_IO_BUFFER_SIZE`).
+pub const ENV_OBJECT_IO_BUFFER_SIZE: &str = "RUSTFS_OBJECT_IO_BUFFER_SIZE";
+
+/// Default I/O buffer size: 128 KB.
+pub const DEFAULT_OBJECT_IO_BUFFER_SIZE: usize = 128 * 1024;
+
+/// Environment variable to enable/disable lock optimization.
+///
+/// When enabled, read locks are released immediately after metadata
+/// is read, rather than being held for the entire data transfer.
+/// This significantly reduces lock contention under high concurrency.
+///
+/// Default: true (enabled, can be overridden by `RUSTFS_OBJECT_LOCK_OPTIMIZATION_ENABLE`).
+pub const ENV_OBJECT_LOCK_OPTIMIZATION_ENABLE: &str = "RUSTFS_OBJECT_LOCK_OPTIMIZATION_ENABLE";
+
+/// Default: lock optimization is enabled.
+pub const DEFAULT_OBJECT_LOCK_OPTIMIZATION_ENABLE: bool = true;
+
+/// Environment variable to enable/disable priority-based I/O scheduling.
+///
+/// When enabled, smaller requests (< 1MB) are given higher priority
+/// than larger requests (> 10MB), preventing "starvation" of small
+/// requests by large ones.
+///
+/// Default: true (enabled, can be overridden by `RUSTFS_OBJECT_PRIORITY_SCHEDULING_ENABLE`).
+pub const ENV_OBJECT_PRIORITY_SCHEDULING_ENABLE: &str = "RUSTFS_OBJECT_PRIORITY_SCHEDULING_ENABLE";
+
+/// Default: priority scheduling is enabled.
+pub const DEFAULT_OBJECT_PRIORITY_SCHEDULING_ENABLE: bool = true;
+
+/// Environment variable to enable/disable deadlock detection.
+///
+/// When enabled, the system monitors active requests and detects
+/// potential deadlock situations (circular lock wait chains).
+/// This has some performance overhead and is intended for debugging.
+///
+/// Default: false (disabled, can be overridden by `RUSTFS_OBJECT_DEADLOCK_DETECTION_ENABLE`).
+pub const ENV_OBJECT_DEADLOCK_DETECTION_ENABLE: &str = "RUSTFS_OBJECT_DEADLOCK_DETECTION_ENABLE";
+
+/// Default: deadlock detection is disabled for performance.
+pub const DEFAULT_OBJECT_DEADLOCK_DETECTION_ENABLE: bool = false;
+
+/// Environment variable for deadlock detection check interval in seconds.
+///
+/// How often the deadlock detector analyzes the lock wait graph.
+/// More frequent checks detect deadlocks faster but use more CPU.
+///
+/// Default: 5 seconds (can be overridden by `RUSTFS_OBJECT_DEADLOCK_CHECK_INTERVAL`).
+pub const ENV_OBJECT_DEADLOCK_CHECK_INTERVAL: &str = "RUSTFS_OBJECT_DEADLOCK_CHECK_INTERVAL";
+
+/// Default deadlock check interval: 5 seconds.
+pub const DEFAULT_OBJECT_DEADLOCK_CHECK_INTERVAL: u64 = 5;
+
+/// Environment variable for deadlock detection hang threshold in seconds.
+///
+/// Requests that have been running longer than this threshold are
+/// considered "potentially hung" and included in deadlock analysis.
+///
+/// Default: 10 seconds (can be overridden by `RUSTFS_OBJECT_DEADLOCK_HANG_THRESHOLD`).
+pub const ENV_OBJECT_DEADLOCK_HANG_THRESHOLD: &str = "RUSTFS_OBJECT_DEADLOCK_HANG_THRESHOLD";
+
+/// Default hang threshold: 10 seconds.
+pub const DEFAULT_OBJECT_DEADLOCK_HANG_THRESHOLD: u64 = 10;
+
+/// Environment variable for backpressure high watermark percentage.
+///
+/// When buffer usage exceeds this percentage, the system enters
+/// "high watermark" state and may apply backpressure to producers.
+///
+/// Default: 80 (80%, can be overridden by `RUSTFS_OBJECT_BACKPRESSURE_HIGH_WATERMARK`).
+pub const ENV_OBJECT_BACKPRESSURE_HIGH_WATERMARK: &str = "RUSTFS_OBJECT_BACKPRESSURE_HIGH_WATERMARK";
+
+/// Default high watermark: 80%.
+pub const DEFAULT_OBJECT_BACKPRESSURE_HIGH_WATERMARK: u32 = 80;
+
+/// Environment variable for backpressure low watermark percentage.
+///
+/// When buffer usage drops below this percentage after being in
+/// high watermark state, backpressure is released.
+///
+/// Default: 50 (50%, can be overridden by `RUSTFS_OBJECT_BACKPRESSURE_LOW_WATERMARK`).
+pub const ENV_OBJECT_BACKPRESSURE_LOW_WATERMARK: &str = "RUSTFS_OBJECT_BACKPRESSURE_LOW_WATERMARK";
+
+/// Default low watermark: 50%.
+pub const DEFAULT_OBJECT_BACKPRESSURE_LOW_WATERMARK: u32 = 50;
+
+/// Environment variable for lock acquisition timeout in seconds.
+///
+/// When a lock cannot be acquired within this duration, the operation
+/// will fail with a timeout error. This prevents indefinite waiting
+/// for locks that may never be released due to deadlocks.
+///
+/// Default: 5 seconds (can be overridden by `RUSTFS_OBJECT_LOCK_ACQUIRE_TIMEOUT`).
+pub const ENV_OBJECT_LOCK_ACQUIRE_TIMEOUT: &str = "RUSTFS_OBJECT_LOCK_ACQUIRE_TIMEOUT";
+
+/// Default lock acquisition timeout: 5 seconds.
+pub const DEFAULT_OBJECT_LOCK_ACQUIRE_TIMEOUT: u64 = 5;
+
+// ============================================================================
+// I/O priority scheduling configuration
+// ============================================================================
+
+/// Environment variable for I/O high priority size threshold in bytes.
+///
+/// Requests smaller than this threshold are classified as high priority.
+/// High priority requests are processed first to prevent starvation of small requests.
+///
+/// Default: 1048576 (1 MB, can be overridden by `RUSTFS_OBJECT_IO_HIGH_PRIORITY_SIZE_THRESHOLD`).
+pub const ENV_OBJECT_IO_HIGH_PRIORITY_SIZE_THRESHOLD: &str = "RUSTFS_OBJECT_IO_HIGH_PRIORITY_SIZE_THRESHOLD";
+
+/// Default high priority size threshold: 1 MB.
+pub const DEFAULT_OBJECT_IO_HIGH_PRIORITY_SIZE_THRESHOLD: usize = 1024 * 1024;
+
+/// Environment variable for I/O low priority size threshold in bytes.
+///
+/// Requests larger than this threshold are classified as low priority.
+/// Low priority requests are processed last to avoid blocking small requests.
+///
+/// Default: 10485760 (10 MB, can be overridden by `RUSTFS_OBJECT_IO_LOW_PRIORITY_SIZE_THRESHOLD`).
+pub const ENV_OBJECT_IO_LOW_PRIORITY_SIZE_THRESHOLD: &str = "RUSTFS_OBJECT_IO_LOW_PRIORITY_SIZE_THRESHOLD";
+
+/// Default low priority size threshold: 10 MB.
+pub const DEFAULT_OBJECT_IO_LOW_PRIORITY_SIZE_THRESHOLD: usize = 10 * 1024 * 1024;
+
+/// Environment variable for high priority queue capacity.
+///
+/// Maximum number of requests that can be queued in the high priority queue.
+///
+/// Default: 32 (can be overridden by `RUSTFS_OBJECT_IO_QUEUE_HIGH_CAPACITY`).
+pub const ENV_OBJECT_IO_QUEUE_HIGH_CAPACITY: &str = "RUSTFS_OBJECT_IO_QUEUE_HIGH_CAPACITY";
+
+/// Default high priority queue capacity: 32.
+pub const DEFAULT_OBJECT_IO_QUEUE_HIGH_CAPACITY: usize = 32;
+
+/// Environment variable for normal priority queue capacity.
+///
+/// Maximum number of requests that can be queued in the normal priority queue.
+///
+/// Default: 64 (can be overridden by `RUSTFS_OBJECT_IO_QUEUE_NORMAL_CAPACITY`).
+pub const ENV_OBJECT_IO_QUEUE_NORMAL_CAPACITY: &str = "RUSTFS_OBJECT_IO_QUEUE_NORMAL_CAPACITY";
+
+/// Default normal priority queue capacity: 64.
+pub const DEFAULT_OBJECT_IO_QUEUE_NORMAL_CAPACITY: usize = 64;
+
+/// Environment variable for low priority queue capacity.
+///
+/// Maximum number of requests that can be queued in the low priority queue.
+///
+/// Default: 16 (can be overridden by `RUSTFS_OBJECT_IO_QUEUE_LOW_CAPACITY`).
+pub const ENV_OBJECT_IO_QUEUE_LOW_CAPACITY: &str = "RUSTFS_OBJECT_IO_QUEUE_LOW_CAPACITY";
+
+/// Default low priority queue capacity: 16.
+pub const DEFAULT_OBJECT_IO_QUEUE_LOW_CAPACITY: usize = 16;
+
+/// Environment variable for starvation prevention check interval in milliseconds.
+///
+/// How often the system checks for starving low-priority requests.
+/// When a low-priority request has been waiting longer than the starvation threshold,
+/// it is promoted to normal priority.
+///
+/// Default: 100 ms (can be overridden by `RUSTFS_OBJECT_IO_STARVATION_PREVENTION_INTERVAL`).
+pub const ENV_OBJECT_IO_STARVATION_PREVENTION_INTERVAL: &str = "RUSTFS_OBJECT_IO_STARVATION_PREVENTION_INTERVAL";
+
+/// Default starvation prevention interval: 100 ms.
+pub const DEFAULT_OBJECT_IO_STARVATION_PREVENTION_INTERVAL: u64 = 100;
+
+/// Environment variable for starvation threshold in seconds.
+///
+/// Maximum time a low-priority request can wait before being promoted to normal priority.
+/// This prevents indefinite starvation of low-priority requests.
+///
+/// Default: 5 seconds (can be overridden by `RUSTFS_OBJECT_IO_STARVATION_THRESHOLD_SECS`).
+pub const ENV_OBJECT_IO_STARVATION_THRESHOLD_SECS: &str = "RUSTFS_OBJECT_IO_STARVATION_THRESHOLD_SECS";
+
+/// Default starvation threshold: 5 seconds.
+pub const DEFAULT_OBJECT_IO_STARVATION_THRESHOLD_SECS: u64 = 5;
+
+/// Environment variable for load sampling window size.
+///
+/// Number of recent samples used to calculate I/O load metrics.
+///
+/// Default: 100 samples (can be overridden by `RUSTFS_OBJECT_IO_LOAD_SAMPLE_WINDOW`).
+pub const ENV_OBJECT_IO_LOAD_SAMPLE_WINDOW: &str = "RUSTFS_OBJECT_IO_LOAD_SAMPLE_WINDOW";
+
+/// Default load sampling window: 100 samples.
+pub const DEFAULT_OBJECT_IO_LOAD_SAMPLE_WINDOW: usize = 100;
+
+/// Environment variable for high load wait time threshold in milliseconds.
+///
+/// When average wait time exceeds this threshold, the system is considered to be under high load.
+///
+/// Default: 50 ms (can be overridden by `RUSTFS_OBJECT_IO_LOAD_HIGH_THRESHOLD_MS`).
+pub const ENV_OBJECT_IO_LOAD_HIGH_THRESHOLD_MS: &str = "RUSTFS_OBJECT_IO_LOAD_HIGH_THRESHOLD_MS";
+
+/// Default high load threshold: 50 ms.
+pub const DEFAULT_OBJECT_IO_LOAD_HIGH_THRESHOLD_MS: u64 = 50;
+
+/// Environment variable for low load wait time threshold in milliseconds.
+///
+/// When average wait time is below this threshold, the system is considered to be under low load.
+///
+/// Default: 10 ms (can be overridden by `RUSTFS_OBJECT_IO_LOAD_LOW_THRESHOLD_MS`).
+pub const ENV_OBJECT_IO_LOAD_LOW_THRESHOLD_MS: &str = "RUSTFS_OBJECT_IO_LOAD_LOW_THRESHOLD_MS";
+
+/// Default low load threshold: 10 ms.
+pub const DEFAULT_OBJECT_IO_LOAD_LOW_THRESHOLD_MS: u64 = 10;
+
+/// Environment variable for enabling storage media detection for adaptive I/O scheduling.
+///
+/// When disabled, the scheduler falls back to `Unknown` storage media unless an explicit
+/// override is provided.
+///
+/// Default: true (can be overridden by `RUSTFS_OBJECT_IO_STORAGE_DETECTION_ENABLE`).
+pub const ENV_OBJECT_IO_STORAGE_DETECTION_ENABLE: &str = "RUSTFS_OBJECT_IO_STORAGE_DETECTION_ENABLE";
+
+/// Default storage media detection setting: enabled.
+pub const DEFAULT_OBJECT_IO_STORAGE_DETECTION_ENABLE: bool = true;
+
+/// Environment variable for overriding detected storage media.
+///
+/// Supported values: `nvme`, `ssd`, `hdd`, `unknown`.
+/// Empty value means auto-detect or fallback to `Unknown`.
+///
+/// Default: empty string (can be overridden by `RUSTFS_OBJECT_IO_STORAGE_MEDIA_OVERRIDE`).
+pub const ENV_OBJECT_IO_STORAGE_MEDIA_OVERRIDE: &str = "RUSTFS_OBJECT_IO_STORAGE_MEDIA_OVERRIDE";
+
+/// Default storage media override: no override.
+pub const DEFAULT_OBJECT_IO_STORAGE_MEDIA_OVERRIDE: &str = "";
+
+/// Environment variable for access-pattern history size.
+///
+/// Controls how many recent offset/length observations are used to classify
+/// sequential, random, or mixed reads.
+///
+/// Default: 8 (can be overridden by `RUSTFS_OBJECT_IO_PATTERN_HISTORY_SIZE`).
+pub const ENV_OBJECT_IO_PATTERN_HISTORY_SIZE: &str = "RUSTFS_OBJECT_IO_PATTERN_HISTORY_SIZE";
+
+/// Default access-pattern history size: 8 samples.
+pub const DEFAULT_OBJECT_IO_PATTERN_HISTORY_SIZE: usize = 8;
+
+/// Environment variable for sequential access step tolerance in bytes.
+///
+/// Small gaps between adjacent reads within this tolerance are still treated as sequential.
+///
+/// Default: 131072 bytes (128 KiB, can be overridden by `RUSTFS_OBJECT_IO_SEQUENTIAL_STEP_TOLERANCE_BYTES`).
+pub const ENV_OBJECT_IO_SEQUENTIAL_STEP_TOLERANCE_BYTES: &str = "RUSTFS_OBJECT_IO_SEQUENTIAL_STEP_TOLERANCE_BYTES";
+
+/// Default sequential step tolerance: 128 KiB.
+pub const DEFAULT_OBJECT_IO_SEQUENTIAL_STEP_TOLERANCE_BYTES: u64 = 128 * 1024;
+
+/// Environment variable for bandwidth EMA beta.
+///
+/// Lower values react faster to recent throughput changes while higher values smooth
+/// short-term fluctuations more aggressively.
+///
+/// Default: 0.1 (can be overridden by `RUSTFS_OBJECT_IO_BANDWIDTH_EMA_BETA`).
+pub const ENV_OBJECT_IO_BANDWIDTH_EMA_BETA: &str = "RUSTFS_OBJECT_IO_BANDWIDTH_EMA_BETA";
+
+/// Default bandwidth EMA beta: 0.1.
+pub const DEFAULT_OBJECT_IO_BANDWIDTH_EMA_BETA: f64 = 0.1;
+
+/// Environment variable for the low bandwidth threshold in bytes per second.
+///
+/// Observed throughput below this value causes the scheduler to be more conservative
+/// with buffer growth and read-ahead.
+///
+/// Default: 67108864 bytes/sec (64 MiB/s, can be overridden by `RUSTFS_OBJECT_IO_BANDWIDTH_LOW_THRESHOLD_BPS`).
+pub const ENV_OBJECT_IO_BANDWIDTH_LOW_THRESHOLD_BPS: &str = "RUSTFS_OBJECT_IO_BANDWIDTH_LOW_THRESHOLD_BPS";
+
+/// Default low bandwidth threshold: 64 MiB/s.
+pub const DEFAULT_OBJECT_IO_BANDWIDTH_LOW_THRESHOLD_BPS: u64 = 64 * 1024 * 1024;
+
+/// Environment variable for the high bandwidth threshold in bytes per second.
+///
+/// Observed throughput above this value allows the scheduler to be more aggressive
+/// for sequential workloads.
+///
+/// Default: 536870912 bytes/sec (512 MiB/s, can be overridden by `RUSTFS_OBJECT_IO_BANDWIDTH_HIGH_THRESHOLD_BPS`).
+pub const ENV_OBJECT_IO_BANDWIDTH_HIGH_THRESHOLD_BPS: &str = "RUSTFS_OBJECT_IO_BANDWIDTH_HIGH_THRESHOLD_BPS";
+
+/// Default high bandwidth threshold: 512 MiB/s.
+pub const DEFAULT_OBJECT_IO_BANDWIDTH_HIGH_THRESHOLD_BPS: u64 = 512 * 1024 * 1024;
+
+/// Environment variable for NVMe buffer cap in bytes.
+///
+/// Sequential reads on NVMe can scale up to this buffer cap.
+///
+/// Default: 2097152 bytes (2 MiB, can be overridden by `RUSTFS_OBJECT_IO_NVME_BUFFER_CAP`).
+pub const ENV_OBJECT_IO_NVME_BUFFER_CAP: &str = "RUSTFS_OBJECT_IO_NVME_BUFFER_CAP";
+
+/// Default NVMe buffer cap: 2 MiB.
+pub const DEFAULT_OBJECT_IO_NVME_BUFFER_CAP: usize = 2 * 1024 * 1024;
+
+/// Environment variable for SSD buffer cap in bytes.
+///
+/// Default: 1048576 bytes (1 MiB, can be overridden by `RUSTFS_OBJECT_IO_SSD_BUFFER_CAP`).
+pub const ENV_OBJECT_IO_SSD_BUFFER_CAP: &str = "RUSTFS_OBJECT_IO_SSD_BUFFER_CAP";
+
+/// Default SSD buffer cap: 1 MiB.
+pub const DEFAULT_OBJECT_IO_SSD_BUFFER_CAP: usize = 1024 * 1024;
+
+/// Environment variable for HDD buffer cap in bytes.
+///
+/// Default: 524288 bytes (512 KiB, can be overridden by `RUSTFS_OBJECT_IO_HDD_BUFFER_CAP`).
+pub const ENV_OBJECT_IO_HDD_BUFFER_CAP: &str = "RUSTFS_OBJECT_IO_HDD_BUFFER_CAP";
+
+/// Default HDD buffer cap: 512 KiB.
+pub const DEFAULT_OBJECT_IO_HDD_BUFFER_CAP: usize = 512 * 1024;
+
+/// Environment variable for disabling read-ahead under random or mixed access with concurrency.
+///
+/// When concurrent requests reach this threshold, random-heavy workloads stop using read-ahead.
+///
+/// Default: 4 (can be overridden by `RUSTFS_OBJECT_IO_RANDOM_READAHEAD_DISABLE_CONCURRENCY`).
+pub const ENV_OBJECT_IO_RANDOM_READAHEAD_DISABLE_CONCURRENCY: &str = "RUSTFS_OBJECT_IO_RANDOM_READAHEAD_DISABLE_CONCURRENCY";
+
+/// Default read-ahead disable concurrency threshold: 4.
+pub const DEFAULT_OBJECT_IO_RANDOM_READAHEAD_DISABLE_CONCURRENCY: usize = 4;

crates/config/src/constants/protocols.rs

@@ -45,3 +45,15 @@ pub const ENV_FTPS_CERTS_DIR: &str = "RUSTFS_FTPS_CERTS_DIR";
 pub const ENV_FTPS_CA_FILE: &str = "RUSTFS_FTPS_CA_FILE";
 pub const ENV_FTPS_PASSIVE_PORTS: &str = "RUSTFS_FTPS_PASSIVE_PORTS";
 pub const ENV_FTPS_EXTERNAL_IP: &str = "RUSTFS_FTPS_EXTERNAL_IP";
+
+/// Default WebDAV server bind address
+pub const DEFAULT_WEBDAV_ADDRESS: &str = "0.0.0.0:8080";
+
+/// WebDAV environment variable names
+pub const ENV_WEBDAV_ENABLE: &str = "RUSTFS_WEBDAV_ENABLE";
+pub const ENV_WEBDAV_ADDRESS: &str = "RUSTFS_WEBDAV_ADDRESS";
+pub const ENV_WEBDAV_TLS_ENABLED: &str = "RUSTFS_WEBDAV_TLS_ENABLED";
+pub const ENV_WEBDAV_CERTS_DIR: &str = "RUSTFS_WEBDAV_CERTS_DIR";
+pub const ENV_WEBDAV_CA_FILE: &str = "RUSTFS_WEBDAV_CA_FILE";
+pub const ENV_WEBDAV_MAX_BODY_SIZE: &str = "RUSTFS_WEBDAV_MAX_BODY_SIZE";
+pub const ENV_WEBDAV_REQUEST_TIMEOUT: &str = "RUSTFS_WEBDAV_REQUEST_TIMEOUT";

crates/config/src/constants/runtime.rs

@@ -27,6 +27,16 @@ pub const ENV_RNG_SEED: &str = "RUSTFS_RUNTIME_RNG_SEED";
 /// Event polling interval
 pub const ENV_EVENT_INTERVAL: &str = "RUSTFS_RUNTIME_EVENT_INTERVAL";
 
+// Dial9 Tokio Telemetry Configuration
+pub const ENV_RUNTIME_DIAL9_ENABLED: &str = "RUSTFS_RUNTIME_DIAL9_ENABLED";
+pub const ENV_RUNTIME_DIAL9_OUTPUT_DIR: &str = "RUSTFS_RUNTIME_DIAL9_OUTPUT_DIR";
+pub const ENV_RUNTIME_DIAL9_FILE_PREFIX: &str = "RUSTFS_RUNTIME_DIAL9_FILE_PREFIX";
+pub const ENV_RUNTIME_DIAL9_MAX_FILE_SIZE: &str = "RUSTFS_RUNTIME_DIAL9_MAX_FILE_SIZE";
+pub const ENV_RUNTIME_DIAL9_ROTATION_COUNT: &str = "RUSTFS_RUNTIME_DIAL9_ROTATION_COUNT";
+pub const ENV_RUNTIME_DIAL9_S3_BUCKET: &str = "RUSTFS_RUNTIME_DIAL9_S3_BUCKET";
+pub const ENV_RUNTIME_DIAL9_S3_PREFIX: &str = "RUSTFS_RUNTIME_DIAL9_S3_PREFIX";
+pub const ENV_RUNTIME_DIAL9_SAMPLING_RATE: &str = "RUSTFS_RUNTIME_DIAL9_SAMPLING_RATE";
+
 // Default values for Tokio runtime
 pub const DEFAULT_WORKER_THREADS: usize = 16;
 pub const DEFAULT_MAX_BLOCKING_THREADS: usize = 1024;
@@ -40,6 +50,15 @@ pub const DEFAULT_MAX_IO_EVENTS_PER_TICK: usize = 1024;
 pub const DEFAULT_EVENT_INTERVAL: u32 = 61;
 pub const DEFAULT_RNG_SEED: Option<u64> = None; // None means random
 
+// Dial9 Tokio Telemetry Default values
+pub const DEFAULT_RUNTIME_DIAL9_ENABLED: bool = false; // Disabled by default
+pub const DEFAULT_RUNTIME_DIAL9_OUTPUT_DIR: &str = "/var/log/rustfs/telemetry";
+pub const DEFAULT_RUNTIME_DIAL9_FILE_PREFIX: &str = "rustfs-tokio";
+pub const DEFAULT_RUNTIME_DIAL9_MAX_FILE_SIZE: u64 = 100 * 1024 * 1024; // 100MB
+pub const DEFAULT_RUNTIME_DIAL9_ROTATION_COUNT: usize = 10;
+pub const DEFAULT_RUNTIME_DIAL9_SAMPLING_RATE: f64 = 1.0; // 100% sampling
+// Note: S3 bucket/prefix have no default; absence means upload is disabled (modeled as Option<String>)
+
 /// Threshold for small object seek support in megabytes.
 ///
 /// When an object is smaller than this size, rustfs will provide seek support.

crates/config/src/constants/scanner.rs

@@ -12,17 +12,107 @@
 // See the License for the specific language governing permissions and
 // limitations under the License.
 
-/// Environment variable name that specifies the data scanner start delay in seconds.
-/// - Purpose: Define the delay between data scanner operations.
+use std::time::Duration;
+
+/// Canonical environment variable name that specifies the scanner start delay in seconds.
+/// If set, this overrides the cycle interval derived from `RUSTFS_SCANNER_SPEED`.
 /// - Unit: seconds (u64).
-/// - Valid values: any positive integer.
-/// - Semantics: This delay controls how frequently the data scanner checks for and processes data; shorter delays lead to more responsive scanning but may increase system load.
-/// - Example: `export RUSTFS_DATA_SCANNER_START_DELAY_SECS=10`
-/// - Note: Choose an appropriate delay that balances scanning responsiveness with overall system performance.
+/// - Example: `export RUSTFS_SCANNER_START_DELAY_SECS=10`
+pub const ENV_SCANNER_START_DELAY_SECS: &str = "RUSTFS_SCANNER_START_DELAY_SECS";
+
+/// Deprecated compatibility alias for scanner start delay.
+/// Prefer `RUSTFS_SCANNER_START_DELAY_SECS`.
+#[deprecated(note = "Use RUSTFS_SCANNER_START_DELAY_SECS instead")]
 pub const ENV_DATA_SCANNER_START_DELAY_SECS: &str = "RUSTFS_DATA_SCANNER_START_DELAY_SECS";
 
-/// Default data scanner start delay in seconds if not specified in the environment variable.
-/// - Value: 10 seconds.
-/// - Rationale: This default interval provides a reasonable balance between scanning responsiveness and system load for most deployments.
-/// - Adjustments: Users may modify this value via the `RUSTFS_DATA_SCANNER_START_DELAY_SECS` environment variable based on their specific scanning requirements and system performance.
-pub const DEFAULT_DATA_SCANNER_START_DELAY_SECS: u64 = 60;
+/// Environment variable that selects the scanner speed preset.
+/// Valid values: `fastest`, `fast`, `default`, `slow`, `slowest`.
+/// Controls the sleep factor, maximum sleep duration, and cycle interval.
+/// - Example: `export RUSTFS_SCANNER_SPEED=slow`
+pub const ENV_SCANNER_SPEED: &str = "RUSTFS_SCANNER_SPEED";
+
+/// Default scanner speed preset.
+pub const DEFAULT_SCANNER_SPEED: &str = "default";
+
+/// Environment variable that controls whether the scanner sleeps between operations.
+/// When `true` (default), the scanner throttles itself. When `false`, it runs at full speed.
+/// - Example: `export RUSTFS_SCANNER_IDLE_MODE=false`
+pub const ENV_SCANNER_IDLE_MODE: &str = "RUSTFS_SCANNER_IDLE_MODE";
+
+/// Default scanner idle mode.
+pub const DEFAULT_SCANNER_IDLE_MODE: bool = true;
+
+/// Scanner speed preset controlling throttling behavior.
+///
+/// Each preset defines three parameters:
+/// - **sleep_factor**: Multiplier applied to elapsed work time to compute inter-object sleep.
+/// - **max_sleep**: Upper bound on any single throttle sleep.
+/// - **cycle_interval**: Base delay between scan cycles.
+///
+/// | Preset    | Factor | Max Sleep | Cycle Interval |
+/// |-----------|--------|-----------|----------------|
+/// | `fastest` | 0      | 0         | 1 second       |
+/// | `fast`    | 1x     | 100ms     | 1 minute       |
+/// | `default` | 2x     | 1 second  | 1 minute       |
+/// | `slow`    | 10x    | 15 seconds| 1 minute       |
+/// | `slowest` | 100x   | 15 seconds| 30 minutes     |
+#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
+pub enum ScannerSpeed {
+    Fastest,
+    Fast,
+    #[default]
+    Default,
+    Slow,
+    Slowest,
+}
+
+impl ScannerSpeed {
+    pub fn sleep_factor(self) -> f64 {
+        match self {
+            Self::Fastest => 0.0,
+            Self::Fast => 1.0,
+            Self::Default => 2.0,
+            Self::Slow => 10.0,
+            Self::Slowest => 100.0,
+        }
+    }
+
+    pub fn max_sleep(self) -> Duration {
+        match self {
+            Self::Fastest => Duration::ZERO,
+            Self::Fast => Duration::from_millis(100),
+            Self::Default => Duration::from_secs(1),
+            Self::Slow | Self::Slowest => Duration::from_secs(15),
+        }
+    }
+
+    pub fn cycle_interval(self) -> Duration {
+        match self {
+            Self::Fastest => Duration::from_secs(1),
+            Self::Fast | Self::Default | Self::Slow => Duration::from_secs(60),
+            Self::Slowest => Duration::from_secs(30 * 60),
+        }
+    }
+
+    pub fn from_env_str(s: &str) -> Self {
+        match s.trim().to_ascii_lowercase().as_str() {
+            "fastest" => Self::Fastest,
+            "fast" => Self::Fast,
+            "slow" => Self::Slow,
+            "slowest" => Self::Slowest,
+            _ => Self::Default,
+        }
+    }
+}
+
+impl std::fmt::Display for ScannerSpeed {
+    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
+        match self {
+            Self::Fastest => write!(f, "fastest"),
+            Self::Fast => write!(f, "fast"),
+            Self::Default => write!(f, "default"),
+            Self::Slow => write!(f, "slow"),
+            Self::Slowest => write!(f, "slowest"),
+        }
+    }
+}

crates/config/src/constants/targets.rs

@@ -34,3 +34,16 @@ pub const MQTT_RECONNECT_INTERVAL: &str = "reconnect_interval";
 pub const MQTT_KEEP_ALIVE_INTERVAL: &str = "keep_alive_interval";
 pub const MQTT_QUEUE_DIR: &str = "queue_dir";
 pub const MQTT_QUEUE_LIMIT: &str = "queue_limit";
+pub const MQTT_TLS_POLICY: &str = "tls_policy";
+pub const MQTT_TLS_CA: &str = "tls_ca";
+pub const MQTT_TLS_CLIENT_CERT: &str = "tls_client_cert";
+pub const MQTT_TLS_CLIENT_KEY: &str = "tls_client_key";
+pub const MQTT_TLS_TRUST_LEAF_AS_CA: &str = "tls_trust_leaf_as_ca";
+pub const MQTT_WS_PATH_ALLOWLIST: &str = "ws_path_allowlist";
+
+/// Environment variable controlling whether target queue files are Snappy-compressed.
+/// Applies to both notify and audit target queue stores.
+pub const ENV_TARGET_STORE_COMPRESS: &str = "RUSTFS_TARGET_STORE_COMPRESS";
+
+/// Queue-store compression is enabled by default to reduce disk footprint.
+pub const DEFAULT_TARGET_STORE_COMPRESS: bool = true;

crates/config/src/constants/tls.rs

@@ -84,3 +84,69 @@ pub const ENV_SERVER_MTLS_ENABLE: &str = "RUSTFS_SERVER_MTLS_ENABLE";
 /// By default, RustFS server mTLS is disabled.
 /// To change this behavior, set the environment variable RUSTFS_SERVER_MTLS_ENABLE=1
 pub const DEFAULT_SERVER_MTLS_ENABLE: bool = false;
+
+// ── HTTP Transport Tuning Parameters ──
+
+/// Environment variable for HTTP/2 initial stream window size (bytes)
+/// Default: 4194304 (4 MB)
+pub const ENV_H2_INITIAL_STREAM_WINDOW_SIZE: &str = "RUSTFS_H2_INITIAL_STREAM_WINDOW_SIZE";
+pub const DEFAULT_H2_INITIAL_STREAM_WINDOW_SIZE: u32 = 4 * 1024 * 1024; // 4 MB
+
+/// Environment variable for HTTP/2 initial connection window size (bytes)
+/// Default: 8388608 (8 MB)
+pub const ENV_H2_INITIAL_CONN_WINDOW_SIZE: &str = "RUSTFS_H2_INITIAL_CONN_WINDOW_SIZE";
+pub const DEFAULT_H2_INITIAL_CONN_WINDOW_SIZE: u32 = 8 * 1024 * 1024; // 8 MB
+
+/// Environment variable for HTTP/2 max frame size (bytes)
+/// Range: 16384 (16 KB) to 16777216 (16 MB) per RFC 7540
+/// Default: 524288 (512 KB)
+pub const ENV_H2_MAX_FRAME_SIZE: &str = "RUSTFS_H2_MAX_FRAME_SIZE";
+pub const DEFAULT_H2_MAX_FRAME_SIZE: u32 = 512 * 1024; // 512 KB
+
+/// Environment variable for HTTP/2 max header list size (bytes)
+/// Default: 65536 (64 KB)
+pub const ENV_H2_MAX_HEADER_LIST_SIZE: &str = "RUSTFS_H2_MAX_HEADER_LIST_SIZE";
+pub const DEFAULT_H2_MAX_HEADER_LIST_SIZE: u32 = 64 * 1024; // 64 KB
+
+/// Environment variable for HTTP/2 max concurrent streams
+/// Default: 2048
+pub const ENV_H2_MAX_CONCURRENT_STREAMS: &str = "RUSTFS_H2_MAX_CONCURRENT_STREAMS";
+pub const DEFAULT_H2_MAX_CONCURRENT_STREAMS: u32 = 2048;
+
+/// Environment variable for HTTP/2 keep-alive interval (seconds)
+/// Default: 20
+pub const ENV_H2_KEEP_ALIVE_INTERVAL: &str = "RUSTFS_H2_KEEP_ALIVE_INTERVAL";
+pub const DEFAULT_H2_KEEP_ALIVE_INTERVAL: u64 = 20;
+
+/// Environment variable for HTTP/2 keep-alive timeout (seconds)
+/// Default: 10
+pub const ENV_H2_KEEP_ALIVE_TIMEOUT: &str = "RUSTFS_H2_KEEP_ALIVE_TIMEOUT";
+pub const DEFAULT_H2_KEEP_ALIVE_TIMEOUT: u64 = 10;
+
+/// Environment variable for HTTP/1.1 header read timeout (seconds)
+/// Default: 5
+pub const ENV_HTTP1_HEADER_READ_TIMEOUT: &str = "RUSTFS_HTTP1_HEADER_READ_TIMEOUT";
+pub const DEFAULT_HTTP1_HEADER_READ_TIMEOUT: u64 = 5;
+
+/// Environment variable for HTTP/1.1 max buffer size (bytes)
+/// Default: 65536 (64 KB)
+pub const ENV_HTTP1_MAX_BUF_SIZE: &str = "RUSTFS_HTTP1_MAX_BUF_SIZE";
+pub const DEFAULT_HTTP1_MAX_BUF_SIZE: usize = 64 * 1024; // 64 KB
+
+// ── TLS Hot Reload Parameters ──
+
+/// Environment variable to enable TLS certificate hot reload
+/// Default: false
+/// To enable, set the environment variable RUSTFS_TLS_RELOAD_ENABLE=1
+pub const ENV_TLS_RELOAD_ENABLE: &str = "RUSTFS_TLS_RELOAD_ENABLE";
+
+/// Default value for TLS certificate hot reload
+/// By default, RustFS does not reload TLS certificates automatically.
+pub const DEFAULT_TLS_RELOAD_ENABLE: bool = false;
+
+/// Environment variable for TLS certificate reload interval (seconds)
+/// Default: 30 seconds. Minimum: 5 seconds.
+pub const ENV_TLS_RELOAD_INTERVAL: &str = "RUSTFS_TLS_RELOAD_INTERVAL";
+
+/// Default interval for TLS certificate reload check
+pub const DEFAULT_TLS_RELOAD_INTERVAL: u64 = 30;

crates/config/src/constants/workload.rs

@@ -0,0 +1,70 @@
+// Copyright 2024 RustFS Team
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+//! Workload profile buffer configuration constants.
+//!
+//! This module defines environment variable keys and default values for
+//! custom buffer profile configuration.
+
+use crate::{KI_B, MI_B};
+
+/// Environment variable for minimum buffer size
+/// Default: 64KB (65536 bytes)
+pub const ENV_RUSTFS_BUFFER_MIN_SIZE: &str = "RUSTFS_BUFFER_MIN_SIZE";
+
+/// Environment variable for maximum buffer size
+/// Default: 1MB (1048576 bytes)
+pub const ENV_RUSTFS_BUFFER_MAX_SIZE: &str = "RUSTFS_BUFFER_MAX_SIZE";
+
+/// Environment variable for default buffer size (used when file size is unknown)
+/// Default: 256KB (262144 bytes)
+pub const ENV_RUSTFS_BUFFER_DEFAULT_SIZE: &str = "RUSTFS_BUFFER_DEFAULT_SIZE";
+
+/// Default minimum buffer size: 64KB
+pub const DEFAULT_BUFFER_MIN_SIZE: usize = 64 * KI_B;
+
+/// Default maximum buffer size: 1MB
+pub const DEFAULT_BUFFER_MAX_SIZE: usize = MI_B;
+
+/// Default buffer size for unknown file size: 256KB
+pub const DEFAULT_BUFFER_UNKNOWN_SIZE: usize = 256 * KI_B;
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+    use crate::MI_B;
+
+    #[test]
+    fn test_default_values() {
+        assert_eq!(DEFAULT_BUFFER_MIN_SIZE, 65536); // 64KB
+        assert_eq!(DEFAULT_BUFFER_MAX_SIZE, 1048576); // 1MB
+        assert_eq!(DEFAULT_BUFFER_UNKNOWN_SIZE, 262144); // 256KB
+    }
+
+    #[test]
+    fn test_constants() {
+        assert_eq!(KI_B, 1024);
+        assert_eq!(MI_B, 1024 * 1024);
+        assert_eq!(64 * KI_B, DEFAULT_BUFFER_MIN_SIZE);
+        assert_eq!(MI_B, DEFAULT_BUFFER_MAX_SIZE);
+        assert_eq!(256 * KI_B, DEFAULT_BUFFER_UNKNOWN_SIZE);
+    }
+
+    #[test]
+    fn test_env_var_names() {
+        assert_eq!(ENV_RUSTFS_BUFFER_MIN_SIZE, "RUSTFS_BUFFER_MIN_SIZE");
+        assert_eq!(ENV_RUSTFS_BUFFER_MAX_SIZE, "RUSTFS_BUFFER_MAX_SIZE");
+        assert_eq!(ENV_RUSTFS_BUFFER_DEFAULT_SIZE, "RUSTFS_BUFFER_DEFAULT_SIZE");
+    }
+}

crates/config/src/constants/zero_copy.rs

@@ -0,0 +1,133 @@
+// Copyright 2024 RustFS Team
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+//! Zero-copy I/O configuration constants.
+//!
+//! This module defines environment variables and default values for zero-copy
+//! read operations, which use memory mapping (mmap) to avoid data copying.
+
+// =============================================================================
+// Zero-Copy Configuration
+// =============================================================================
+
+/// Environment variable for zero-copy read enable.
+///
+/// When enabled, uses mmap (Unix) or optimized reads for zero-copy data access.
+/// This reduces memory copies from 3-4 to 1, lowering CPU usage by 20-30%
+/// and improving P95 latency by 15-25%.
+///
+/// - Purpose: Enable or disable zero-copy read operations
+/// - Acceptable values: `"true"` / `"false"` (case-insensitive) or a boolean typed config
+/// - Semantics: When enabled, uses mmap on Unix systems for memory-mapped file reads;
+///   falls back to regular I/O on non-Unix platforms or when mmap fails
+/// - Example: `export RUSTFS_OBJECT_ZERO_COPY_ENABLE=true`
+/// - Note: Zero-copy is safe for all workloads and provides significant performance
+///   benefits with minimal risk. Disable only if mmap-related issues are encountered.
+pub const ENV_OBJECT_ZERO_COPY_ENABLE: &str = "RUSTFS_OBJECT_ZERO_COPY_ENABLE";
+
+/// Default: zero-copy reads are enabled.
+///
+/// Zero-copy uses memory mapping (mmap) on Unix systems to avoid data copying
+/// between kernel and user space. This provides:
+/// - Reduced memory copies: from 3-4 copies to 1 copy
+/// - Lower CPU usage: 20-30% reduction expected
+/// - Improved latency P95: 15-25% reduction expected
+/// - Increased throughput: 10-20% improvement expected
+///
+/// On non-Unix platforms or when mmap fails, the system automatically falls back
+/// to regular I/O without errors.
+pub const DEFAULT_OBJECT_ZERO_COPY_ENABLE: bool = true;
+
+/// Environment variable for zero-copy read operating mode.
+///
+/// Supported values:
+/// - `off`: disable mmap-backed chunk fast path and always use the compatibility path
+/// - `conservative`: allow a single mmap window per request
+/// - `balanced`: allow multiple mmap windows with the default size guardrails
+/// - `aggressive`: allow multi-window mmap and relax the small-object cutoff
+pub const ENV_OBJECT_ZERO_COPY_MODE: &str = "RUSTFS_OBJECT_ZERO_COPY_MODE";
+
+/// Default zero-copy read mode.
+pub const DEFAULT_OBJECT_ZERO_COPY_MODE: &str = "balanced";
+
+/// Environment variable for the maximum mmap window size used by the chunk fast path.
+///
+/// This controls the visible bytes per mapped chunk before the implementation emits a new window.
+pub const ENV_OBJECT_ZERO_COPY_MMAP_WINDOW_BYTES: &str = "RUSTFS_OBJECT_ZERO_COPY_MMAP_WINDOW_BYTES";
+
+/// Default mmap window size for chunk fast path reads: 8 MiB.
+pub const DEFAULT_OBJECT_ZERO_COPY_MMAP_WINDOW_BYTES: usize = 8 * 1024 * 1024;
+
+/// Environment variable for the maximum total active mmap bytes.
+///
+/// Requests that would exceed this active window budget fall back to the compatibility path.
+pub const ENV_OBJECT_ZERO_COPY_MAX_ACTIVE_MMAP_BYTES: &str = "RUSTFS_OBJECT_ZERO_COPY_MAX_ACTIVE_MMAP_BYTES";
+
+/// Default maximum active mmap bytes across concurrent local chunk fast-path reads: 256 MiB.
+pub const DEFAULT_OBJECT_ZERO_COPY_MAX_ACTIVE_MMAP_BYTES: usize = 256 * 1024 * 1024;
+
+// =============================================================================
+// Direct I/O Configuration
+// =============================================================================
+
+/// Environment variable for Direct I/O enable (Linux only).
+///
+/// When enabled, uses O_DIRECT flag to bypass OS page cache for large files.
+/// This is only beneficial for specific workloads (databases, large sequential reads).
+///
+/// - Purpose: Enable or disable Direct I/O for large file operations
+/// - Acceptable values: `"true"` / `"false"` (case-insensitive) or a boolean typed config
+/// - Semantics: When enabled, files larger than the threshold will use O_DIRECT flag;
+///   this bypasses the OS page cache and transfers data directly between disk and application
+/// - Example: `export RUSTFS_OBJECT_DIRECT_IO_ENABLE=true`
+/// - Note: Direct I/O is disabled by default because it's only beneficial for specific
+///   use cases. For most workloads, the OS page cache provides better performance.
+pub const ENV_OBJECT_DIRECT_IO_ENABLE: &str = "RUSTFS_OBJECT_DIRECT_IO_ENABLE";
+
+/// Default: Direct I/O is disabled.
+///
+/// Direct I/O is disabled by default because it's only beneficial for specific use cases:
+/// - Large file transfers (>128MB)
+/// - Databases with their own cache
+/// - Applications requiring predictable I/O latency
+///
+/// For most workloads, the OS page cache provides better performance through:
+/// - Read-ahead caching
+/// - Write buffering
+/// - Multi-use caching (same data cached for multiple operations)
+pub const DEFAULT_OBJECT_DIRECT_IO_ENABLE: bool = false;
+
+/// Environment variable for Direct I/O minimum file size threshold.
+///
+/// Files smaller than this size will use regular I/O even if Direct I/O is enabled.
+/// This avoids the overhead of Direct I/O for small files where the OS page cache
+/// is more effective.
+///
+/// - Purpose: Set the minimum file size for Direct I/O operations
+/// - Unit: Bytes
+/// - Valid values: any positive integer (default: 134,217,728 bytes = 128 MB)
+/// - Semantics: Only files larger than this threshold will use Direct I/O when enabled;
+///   smaller files use regular buffered I/O
+/// - Example: `export RUSTFS_OBJECT_DIRECT_IO_THRESHOLD=268435456`
+/// - Note: The default threshold of 128MB balances the overhead of Direct I/O setup
+///   against the benefits of bypassing the page cache for large files.
+pub const ENV_OBJECT_DIRECT_IO_THRESHOLD: &str = "RUSTFS_OBJECT_DIRECT_IO_THRESHOLD";
+
+/// Default Direct I/O threshold: 128 MB.
+///
+/// Only files larger than 128MB will use Direct I/O when enabled.
+/// Smaller files benefit from OS page cache.
+///
+/// Formula: 128 * 1024 * 1024 = 134,217,728 bytes
+pub const DEFAULT_OBJECT_DIRECT_IO_THRESHOLD: usize = 128 * 1024 * 1024;

crates/config/src/lib.rs

@@ -19,6 +19,8 @@ pub use constants::app::*;
 #[cfg(feature = "constants")]
 pub use constants::body_limits::*;
 #[cfg(feature = "constants")]
+pub use constants::capacity::*;
+#[cfg(feature = "constants")]
 pub use constants::compress::*;
 #[cfg(feature = "constants")]
 pub use constants::console::*;
@@ -45,6 +47,10 @@ pub use constants::targets::*;
 #[cfg(feature = "constants")]
 pub use constants::tls::*;
 #[cfg(feature = "constants")]
+pub use constants::workload::*;
+#[cfg(feature = "constants")]
+pub use constants::zero_copy::*;
+#[cfg(feature = "constants")]
 pub mod oidc {
     pub use super::constants::oidc::*;
 }

crates/config/src/notify/mqtt.rs

@@ -24,6 +24,12 @@ pub const NOTIFY_MQTT_KEYS: &[&str] = &[
     crate::MQTT_KEEP_ALIVE_INTERVAL,
     crate::MQTT_QUEUE_DIR,
     crate::MQTT_QUEUE_LIMIT,
+    crate::MQTT_TLS_POLICY,
+    crate::MQTT_TLS_CA,
+    crate::MQTT_TLS_CLIENT_CERT,
+    crate::MQTT_TLS_CLIENT_KEY,
+    crate::MQTT_TLS_TRUST_LEAF_AS_CA,
+    crate::MQTT_WS_PATH_ALLOWLIST,
     crate::COMMENT_KEY,
 ];
 
@@ -38,8 +44,14 @@ pub const ENV_NOTIFY_MQTT_RECONNECT_INTERVAL: &str = "RUSTFS_NOTIFY_MQTT_RECONNE
 pub const ENV_NOTIFY_MQTT_KEEP_ALIVE_INTERVAL: &str = "RUSTFS_NOTIFY_MQTT_KEEP_ALIVE_INTERVAL";
 pub const ENV_NOTIFY_MQTT_QUEUE_DIR: &str = "RUSTFS_NOTIFY_MQTT_QUEUE_DIR";
 pub const ENV_NOTIFY_MQTT_QUEUE_LIMIT: &str = "RUSTFS_NOTIFY_MQTT_QUEUE_LIMIT";
+pub const ENV_NOTIFY_MQTT_TLS_POLICY: &str = "RUSTFS_NOTIFY_MQTT_TLS_POLICY";
+pub const ENV_NOTIFY_MQTT_TLS_CA: &str = "RUSTFS_NOTIFY_MQTT_TLS_CA";
+pub const ENV_NOTIFY_MQTT_TLS_CLIENT_CERT: &str = "RUSTFS_NOTIFY_MQTT_TLS_CLIENT_CERT";
+pub const ENV_NOTIFY_MQTT_TLS_CLIENT_KEY: &str = "RUSTFS_NOTIFY_MQTT_TLS_CLIENT_KEY";
+pub const ENV_NOTIFY_MQTT_TLS_TRUST_LEAF_AS_CA: &str = "RUSTFS_NOTIFY_MQTT_TLS_TRUST_LEAF_AS_CA";
+pub const ENV_NOTIFY_MQTT_WS_PATH_ALLOWLIST: &str = "RUSTFS_NOTIFY_MQTT_WS_PATH_ALLOWLIST";
 
-pub const ENV_NOTIFY_MQTT_KEYS: &[&str; 10] = &[
+pub const ENV_NOTIFY_MQTT_KEYS: &[&str; 16] = &[
     ENV_NOTIFY_MQTT_ENABLE,
     ENV_NOTIFY_MQTT_BROKER,
     ENV_NOTIFY_MQTT_TOPIC,
@@ -50,4 +62,10 @@ pub const ENV_NOTIFY_MQTT_KEYS: &[&str; 10] = &[
     ENV_NOTIFY_MQTT_KEEP_ALIVE_INTERVAL,
     ENV_NOTIFY_MQTT_QUEUE_DIR,
     ENV_NOTIFY_MQTT_QUEUE_LIMIT,
+    ENV_NOTIFY_MQTT_TLS_POLICY,
+    ENV_NOTIFY_MQTT_TLS_CA,
+    ENV_NOTIFY_MQTT_TLS_CLIENT_CERT,
+    ENV_NOTIFY_MQTT_TLS_CLIENT_KEY,
+    ENV_NOTIFY_MQTT_TLS_TRUST_LEAF_AS_CA,
+    ENV_NOTIFY_MQTT_WS_PATH_ALLOWLIST,
 ];

crates/config/src/observability/mod.rs

@@ -15,12 +15,14 @@
 // Observability Keys
 
 mod metrics;
+use const_str::concat;
 pub use metrics::*;
 
 pub const ENV_OBS_ENDPOINT: &str = "RUSTFS_OBS_ENDPOINT";
 pub const ENV_OBS_TRACE_ENDPOINT: &str = "RUSTFS_OBS_TRACE_ENDPOINT";
 pub const ENV_OBS_METRIC_ENDPOINT: &str = "RUSTFS_OBS_METRIC_ENDPOINT";
 pub const ENV_OBS_LOG_ENDPOINT: &str = "RUSTFS_OBS_LOG_ENDPOINT";
+pub const ENV_OBS_PROFILING_ENDPOINT: &str = "RUSTFS_OBS_PROFILING_ENDPOINT";
 pub const ENV_OBS_USE_STDOUT: &str = "RUSTFS_OBS_USE_STDOUT";
 pub const ENV_OBS_SAMPLE_RATIO: &str = "RUSTFS_OBS_SAMPLE_RATIO";
 pub const ENV_OBS_METER_INTERVAL: &str = "RUSTFS_OBS_METER_INTERVAL";
@@ -32,21 +34,26 @@ pub const ENV_OBS_ENVIRONMENT: &str = "RUSTFS_OBS_ENVIRONMENT";
 pub const ENV_OBS_TRACES_EXPORT_ENABLED: &str = "RUSTFS_OBS_TRACES_EXPORT_ENABLED";
 pub const ENV_OBS_METRICS_EXPORT_ENABLED: &str = "RUSTFS_OBS_METRICS_EXPORT_ENABLED";
 pub const ENV_OBS_LOGS_EXPORT_ENABLED: &str = "RUSTFS_OBS_LOGS_EXPORT_ENABLED";
+pub const ENV_OBS_PROFILING_EXPORT_ENABLED: &str = "RUSTFS_OBS_PROFILING_EXPORT_ENABLED";
 
 pub const ENV_OBS_LOGGER_LEVEL: &str = "RUSTFS_OBS_LOGGER_LEVEL";
 pub const ENV_OBS_LOG_STDOUT_ENABLED: &str = "RUSTFS_OBS_LOG_STDOUT_ENABLED";
 pub const ENV_OBS_LOG_DIRECTORY: &str = "RUSTFS_OBS_LOG_DIRECTORY";
 pub const ENV_OBS_LOG_FILENAME: &str = "RUSTFS_OBS_LOG_FILENAME";
-pub const ENV_OBS_LOG_ROTATION_SIZE_MB: &str = "RUSTFS_OBS_LOG_ROTATION_SIZE_MB";
 pub const ENV_OBS_LOG_ROTATION_TIME: &str = "RUSTFS_OBS_LOG_ROTATION_TIME";
 pub const ENV_OBS_LOG_KEEP_FILES: &str = "RUSTFS_OBS_LOG_KEEP_FILES";
 
 /// Log cleanup related configurations
-pub const ENV_OBS_LOG_KEEP_COUNT: &str = "RUSTFS_OBS_LOG_KEEP_COUNT";
 pub const ENV_OBS_LOG_MAX_TOTAL_SIZE_BYTES: &str = "RUSTFS_OBS_LOG_MAX_TOTAL_SIZE_BYTES";
 pub const ENV_OBS_LOG_MAX_SINGLE_FILE_SIZE_BYTES: &str = "RUSTFS_OBS_LOG_MAX_SINGLE_FILE_SIZE_BYTES";
 pub const ENV_OBS_LOG_COMPRESS_OLD_FILES: &str = "RUSTFS_OBS_LOG_COMPRESS_OLD_FILES";
 pub const ENV_OBS_LOG_GZIP_COMPRESSION_LEVEL: &str = "RUSTFS_OBS_LOG_GZIP_COMPRESSION_LEVEL";
+pub const ENV_OBS_LOG_COMPRESSION_ALGORITHM: &str = "RUSTFS_OBS_LOG_COMPRESSION_ALGORITHM";
+pub const ENV_OBS_LOG_PARALLEL_COMPRESS: &str = "RUSTFS_OBS_LOG_PARALLEL_COMPRESS";
+pub const ENV_OBS_LOG_PARALLEL_WORKERS: &str = "RUSTFS_OBS_LOG_PARALLEL_WORKERS";
+pub const ENV_OBS_LOG_ZSTD_COMPRESSION_LEVEL: &str = "RUSTFS_OBS_LOG_ZSTD_COMPRESSION_LEVEL";
+pub const ENV_OBS_LOG_ZSTD_FALLBACK_TO_GZIP: &str = "RUSTFS_OBS_LOG_ZSTD_FALLBACK_TO_GZIP";
+pub const ENV_OBS_LOG_ZSTD_WORKERS: &str = "RUSTFS_OBS_LOG_ZSTD_WORKERS";
 pub const ENV_OBS_LOG_COMPRESSED_FILE_RETENTION_DAYS: &str = "RUSTFS_OBS_LOG_COMPRESSED_FILE_RETENTION_DAYS";
 pub const ENV_OBS_LOG_EXCLUDE_PATTERNS: &str = "RUSTFS_OBS_LOG_EXCLUDE_PATTERNS";
 pub const ENV_OBS_LOG_DELETE_EMPTY_FILES: &str = "RUSTFS_OBS_LOG_DELETE_EMPTY_FILES";
@@ -56,16 +63,28 @@ pub const ENV_OBS_LOG_DRY_RUN: &str = "RUSTFS_OBS_LOG_DRY_RUN";
 pub const ENV_OBS_LOG_MATCH_MODE: &str = "RUSTFS_OBS_LOG_MATCH_MODE";
 
 /// Default values for log cleanup
-pub const DEFAULT_OBS_LOG_KEEP_COUNT: usize = 10;
 pub const DEFAULT_OBS_LOG_MAX_TOTAL_SIZE_BYTES: u64 = 2 * 1024 * 1024 * 1024; // 2 GiB
 pub const DEFAULT_OBS_LOG_MAX_SINGLE_FILE_SIZE_BYTES: u64 = 0; // No single file limit
 pub const DEFAULT_OBS_LOG_COMPRESS_OLD_FILES: bool = true;
 pub const DEFAULT_OBS_LOG_GZIP_COMPRESSION_LEVEL: u32 = 6;
-pub const DEFAULT_OBS_LOG_COMPRESSED_FILE_RETENTION_DAYS: u64 = 30;
+pub const DEFAULT_OBS_LOG_COMPRESSION_ALGORITHM_GZIP: &str = "gzip";
+pub const DEFAULT_OBS_LOG_COMPRESSION_ALGORITHM_ZSTD: &str = "zstd";
+pub const DEFAULT_OBS_LOG_COMPRESSION_ALGORITHM: &str = DEFAULT_OBS_LOG_COMPRESSION_ALGORITHM_ZSTD;
+pub const DEFAULT_OBS_LOG_PARALLEL_COMPRESS: bool = true;
+pub const DEFAULT_OBS_LOG_PARALLEL_WORKERS: usize = 6;
+pub const DEFAULT_OBS_LOG_ZSTD_COMPRESSION_LEVEL: i32 = 8;
+pub const DEFAULT_OBS_LOG_ZSTD_FALLBACK_TO_GZIP: bool = true;
+pub const DEFAULT_OBS_LOG_ZSTD_WORKERS: usize = 1;
+pub const DEFAULT_OBS_LOG_GZIP_COMPRESSION_EXTENSION: &str = "gz";
+pub const DEFAULT_OBS_LOG_GZIP_COMPRESSION_ALL_EXTENSION: &str = concat!(".", DEFAULT_OBS_LOG_GZIP_COMPRESSION_EXTENSION);
+pub const DEFAULT_OBS_LOG_ZSTD_COMPRESSION_EXTENSION: &str = "zst";
+pub const DEFAULT_OBS_LOG_ZSTD_COMPRESSION_ALL_EXTENSION: &str = concat!(".", DEFAULT_OBS_LOG_ZSTD_COMPRESSION_EXTENSION);
+pub const DEFAULT_OBS_LOG_COMPRESSED_FILE_RETENTION_DAYS: u64 = 30; // Retain compressed files for 30 days
 pub const DEFAULT_OBS_LOG_DELETE_EMPTY_FILES: bool = true;
 pub const DEFAULT_OBS_LOG_MIN_FILE_AGE_SECONDS: u64 = 3600; // 1 hour
-pub const DEFAULT_OBS_LOG_CLEANUP_INTERVAL_SECONDS: u64 = 6 * 3600; // 6 hours
+pub const DEFAULT_OBS_LOG_CLEANUP_INTERVAL_SECONDS: u64 = 1800; // 0.5 hours
 pub const DEFAULT_OBS_LOG_DRY_RUN: bool = false;
+pub const DEFAULT_OBS_LOG_MATCH_MODE_PREFIX: &str = "prefix";
 pub const DEFAULT_OBS_LOG_MATCH_MODE: &str = "suffix";
 
 /// Default values for observability configuration
@@ -89,6 +108,7 @@ mod tests {
         assert_eq!(ENV_OBS_TRACE_ENDPOINT, "RUSTFS_OBS_TRACE_ENDPOINT");
         assert_eq!(ENV_OBS_METRIC_ENDPOINT, "RUSTFS_OBS_METRIC_ENDPOINT");
         assert_eq!(ENV_OBS_LOG_ENDPOINT, "RUSTFS_OBS_LOG_ENDPOINT");
+        assert_eq!(ENV_OBS_PROFILING_ENDPOINT, "RUSTFS_OBS_PROFILING_ENDPOINT");
         assert_eq!(ENV_OBS_USE_STDOUT, "RUSTFS_OBS_USE_STDOUT");
         assert_eq!(ENV_OBS_SAMPLE_RATIO, "RUSTFS_OBS_SAMPLE_RATIO");
         assert_eq!(ENV_OBS_METER_INTERVAL, "RUSTFS_OBS_METER_INTERVAL");
@@ -99,18 +119,23 @@ mod tests {
         assert_eq!(ENV_OBS_LOG_STDOUT_ENABLED, "RUSTFS_OBS_LOG_STDOUT_ENABLED");
         assert_eq!(ENV_OBS_LOG_DIRECTORY, "RUSTFS_OBS_LOG_DIRECTORY");
         assert_eq!(ENV_OBS_LOG_FILENAME, "RUSTFS_OBS_LOG_FILENAME");
-        assert_eq!(ENV_OBS_LOG_ROTATION_SIZE_MB, "RUSTFS_OBS_LOG_ROTATION_SIZE_MB");
         assert_eq!(ENV_OBS_LOG_ROTATION_TIME, "RUSTFS_OBS_LOG_ROTATION_TIME");
         assert_eq!(ENV_OBS_LOG_KEEP_FILES, "RUSTFS_OBS_LOG_KEEP_FILES");
         assert_eq!(ENV_OBS_TRACES_EXPORT_ENABLED, "RUSTFS_OBS_TRACES_EXPORT_ENABLED");
         assert_eq!(ENV_OBS_METRICS_EXPORT_ENABLED, "RUSTFS_OBS_METRICS_EXPORT_ENABLED");
         assert_eq!(ENV_OBS_LOGS_EXPORT_ENABLED, "RUSTFS_OBS_LOGS_EXPORT_ENABLED");
+        assert_eq!(ENV_OBS_PROFILING_EXPORT_ENABLED, "RUSTFS_OBS_PROFILING_EXPORT_ENABLED");
         // Test log cleanup related env keys
-        assert_eq!(ENV_OBS_LOG_KEEP_COUNT, "RUSTFS_OBS_LOG_KEEP_COUNT");
         assert_eq!(ENV_OBS_LOG_MAX_TOTAL_SIZE_BYTES, "RUSTFS_OBS_LOG_MAX_TOTAL_SIZE_BYTES");
         assert_eq!(ENV_OBS_LOG_MAX_SINGLE_FILE_SIZE_BYTES, "RUSTFS_OBS_LOG_MAX_SINGLE_FILE_SIZE_BYTES");
         assert_eq!(ENV_OBS_LOG_COMPRESS_OLD_FILES, "RUSTFS_OBS_LOG_COMPRESS_OLD_FILES");
         assert_eq!(ENV_OBS_LOG_GZIP_COMPRESSION_LEVEL, "RUSTFS_OBS_LOG_GZIP_COMPRESSION_LEVEL");
+        assert_eq!(ENV_OBS_LOG_COMPRESSION_ALGORITHM, "RUSTFS_OBS_LOG_COMPRESSION_ALGORITHM");
+        assert_eq!(ENV_OBS_LOG_PARALLEL_COMPRESS, "RUSTFS_OBS_LOG_PARALLEL_COMPRESS");
+        assert_eq!(ENV_OBS_LOG_PARALLEL_WORKERS, "RUSTFS_OBS_LOG_PARALLEL_WORKERS");
+        assert_eq!(ENV_OBS_LOG_ZSTD_COMPRESSION_LEVEL, "RUSTFS_OBS_LOG_ZSTD_COMPRESSION_LEVEL");
+        assert_eq!(ENV_OBS_LOG_ZSTD_FALLBACK_TO_GZIP, "RUSTFS_OBS_LOG_ZSTD_FALLBACK_TO_GZIP");
+        assert_eq!(ENV_OBS_LOG_ZSTD_WORKERS, "RUSTFS_OBS_LOG_ZSTD_WORKERS");
         assert_eq!(
             ENV_OBS_LOG_COMPRESSED_FILE_RETENTION_DAYS,
             "RUSTFS_OBS_LOG_COMPRESSED_FILE_RETENTION_DAYS"
@@ -129,6 +154,10 @@ mod tests {
         assert_eq!(DEFAULT_OBS_ENVIRONMENT_DEVELOPMENT, "development");
         assert_eq!(DEFAULT_OBS_ENVIRONMENT_TEST, "test");
         assert_eq!(DEFAULT_OBS_ENVIRONMENT_STAGING, "staging");
+        assert_eq!(DEFAULT_OBS_LOG_COMPRESSION_ALGORITHM_GZIP, "gzip");
+        assert_eq!(DEFAULT_OBS_LOG_COMPRESSION_ALGORITHM_ZSTD, "zstd");
+        assert_eq!(DEFAULT_OBS_LOG_MATCH_MODE_PREFIX, "prefix");
         assert_eq!(DEFAULT_OBS_LOG_MATCH_MODE, "suffix");
+        assert_eq!(DEFAULT_OBS_LOG_COMPRESSION_ALGORITHM, "zstd");
     }
 }

crates/credentials/Cargo.toml

@@ -15,7 +15,7 @@ base64-simd = { workspace = true }
 rand = { workspace = true }
 serde = { workspace = true }
 serde_json.workspace = true
-time = { workspace = true, features = ["serde-human-readable"] }
+time = { workspace = true, features = ["serde", "parsing", "formatting", "macros"] }
 
 [lints]
 workspace = true

crates/credentials/src/credentials.rs

@@ -233,15 +233,18 @@ impl<'a> fmt::Debug for Masked<'a> {
         match self.0 {
             None => Ok(()),
             Some(s) => {
-                let len = s.len();
+                let len = s.chars().count();
                 if len == 0 {
                     Ok(())
                 } else if len == 1 {
                     write!(f, "***")
                 } else if len == 2 {
-                    write!(f, "{}***|{}", &s[0..1], len)
+                    let first = s.chars().next().ok_or(fmt::Error)?;
+                    write!(f, "{}***|{}", first, len)
                 } else {
-                    write!(f, "{}***{}|{}", &s[0..1], &s[len - 1..], len)
+                    let first = s.chars().next().ok_or(fmt::Error)?;
+                    let last = s.chars().last().ok_or(fmt::Error)?;
+                    write!(f, "{}***{}|{}", first, last, len)
                 }
             }
         }
@@ -270,15 +273,25 @@ impl<'a> fmt::Display for Masked<'a> {
 ///
 #[derive(Serialize, Deserialize, Clone, Default)]
 pub struct Credentials {
+    #[serde(rename = "accessKey", alias = "access_key", default)]
     pub access_key: String,
+    #[serde(rename = "secretKey", alias = "secret_key", default)]
     pub secret_key: String,
+    #[serde(rename = "sessionToken", alias = "session_token", default)]
     pub session_token: String,
+    #[serde(default, with = "crate::serde_datetime::option")]
     pub expiration: Option<OffsetDateTime>,
+    #[serde(default)]
     pub status: String,
+    #[serde(rename = "parentUser", alias = "parent_user", default)]
     pub parent_user: String,
+    #[serde(default)]
     pub groups: Option<Vec<String>>,
+    #[serde(default)]
     pub claims: Option<HashMap<String, Value>>,
+    #[serde(default)]
     pub name: Option<String>,
+    #[serde(default)]
     pub description: Option<String>,
 }
 
@@ -482,5 +495,37 @@ mod tests {
 
         // Test longer string
         assert_eq!(format!("{:?}", Masked(Some("secretpassword"))), "s***d|14");
+
+        // Test Unicode input should not panic and should keep character boundary
+        assert_eq!(format!("{:?}", Masked(Some("中"))), "***");
+        assert_eq!(format!("{:?}", Masked(Some("中文"))), "中***|2");
+        assert_eq!(format!("{:?}", Masked(Some("中文测试"))), "中***试|4");
+    }
+
+    #[test]
+    fn test_credentials_expiration_serialize_as_rfc3339() {
+        use time::OffsetDateTime;
+
+        let c = Credentials {
+            access_key: "ak".to_string(),
+            secret_key: "sk12345678".to_string(),
+            expiration: Some(OffsetDateTime::now_utc()),
+            ..Default::default()
+        };
+
+        let json = serde_json::to_string(&c).expect("serialize");
+        assert!(
+            json.contains('T') && (json.contains('Z') || json.contains("+00:00")),
+            "Credentials expiration should be RFC3339; got: {}",
+            json
+        );
+    }
+
+    #[test]
+    fn test_credentials_deserialize_minio_style_rfc3339_expiration() {
+        let minio_style = r#"{"accessKey":"ak","secretKey":"sk12345678","expiration":"2025-03-07T12:00:00Z"}"#;
+        let c: Credentials = serde_json::from_str(minio_style).expect("deserialize");
+        assert_eq!(c.access_key, "ak");
+        assert!(c.expiration.is_some());
     }
 }

crates/credentials/src/lib.rs

@@ -14,6 +14,7 @@
 
 mod constants;
 mod credentials;
+mod serde_datetime;
 
 pub use constants::*;
 pub use credentials::*;

crates/credentials/src/serde_datetime.rs

@@ -0,0 +1,68 @@
+// Copyright 2024 RustFS Team
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+//! Serde helpers for expiration timestamp: serialize as RFC3339 (MinIO-compatible),
+//! deserialize from RFC3339 or legacy RustFS human-readable format.
+
+use time::OffsetDateTime;
+use time::format_description;
+use time::format_description::well_known::Rfc3339;
+
+static LEGACY_FORMAT: std::sync::OnceLock<time::format_description::OwnedFormatItem> = std::sync::OnceLock::new();
+
+fn legacy_format() -> &'static time::format_description::OwnedFormatItem {
+    LEGACY_FORMAT.get_or_init(|| {
+        format_description::parse_owned::<2>(
+            "[year]-[month]-[day] [hour]:[minute]:[second].[subsecond] [offset_hour sign:mandatory]:[offset_minute]:[offset_second]",
+        )
+        .expect("legacy format description is valid")
+    });
+    LEGACY_FORMAT.get().expect("initialized above")
+}
+
+fn parse_rfc3339_or_legacy(s: &str) -> Result<OffsetDateTime, time::Error> {
+    OffsetDateTime::parse(s, &Rfc3339).or_else(|_| OffsetDateTime::parse(s, legacy_format()).map_err(Into::into))
+}
+
+/// Option<OffsetDateTime>: serialize as RFC3339; deserialize from RFC3339 or legacy.
+pub mod option {
+    use serde::{Deserialize, Deserializer, Serializer};
+    use time::OffsetDateTime;
+
+    use super::{Rfc3339, parse_rfc3339_or_legacy};
+
+    pub fn serialize<S>(opt: &Option<OffsetDateTime>, serializer: S) -> Result<S::Ok, S::Error>
+    where
+        S: Serializer,
+    {
+        match opt {
+            Some(dt) => {
+                let s = dt.format(&Rfc3339).map_err(serde::ser::Error::custom)?;
+                serializer.serialize_some(&s)
+            }
+            None => serializer.serialize_none(),
+        }
+    }
+
+    pub fn deserialize<'de, D>(deserializer: D) -> Result<Option<OffsetDateTime>, D::Error>
+    where
+        D: Deserializer<'de>,
+    {
+        let opt: Option<&str> = Option::deserialize(deserializer)?;
+        match opt {
+            None => Ok(None),
+            Some(s) => parse_rfc3339_or_legacy(s).map(Some).map_err(serde::de::Error::custom),
+        }
+    }
+}

crates/crypto/Cargo.toml

@@ -56,3 +56,6 @@ crypto = [
     "dep:rand",
     "dep:sha2",
 ]
+
+[lib]
+doctest = false

crates/crypto/src/encdec.rs

@@ -21,5 +21,8 @@ pub(crate) mod id;
 pub(crate) mod decrypt;
 pub(crate) mod encrypt;
 
+#[cfg(any(test, feature = "crypto"))]
+pub(crate) mod stream_io;
+
 #[cfg(test)]
 mod tests;

crates/crypto/src/encdec/stream_io.rs

@@ -0,0 +1,219 @@
+// Copyright 2024 RustFS Team
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+//! sio-go compatible stream encryption for IAM config.
+//! Header: salt(32) + alg_id(1) + nonce_prefix(8) = 41 bytes.
+//! Body: DARE-style fragmented AEAD (bufSize=16384, per-fragment nonce).
+
+#![allow(deprecated)] // AeadInPlace deprecated in favor of AeadInOut; keep for aead 0.6 compatibility
+
+use crate::encdec::id::ID;
+use crate::error::Error;
+use aes_gcm::{
+    Aes256Gcm,
+    aead::{AeadCore, AeadInPlace, KeyInit as _, array::Array},
+};
+use chacha20poly1305::ChaCha20Poly1305;
+
+const STREAM_IO_HEADER_LEN: usize = 41;
+const SIO_BUF_SIZE: usize = 16384;
+const SIO_NONCE_PREFIX_LEN: usize = 8;
+const AES_GCM_OVERHEAD: usize = 16;
+const CHACHA_OVERHEAD: usize = 16;
+
+/// Decrypt data in stream_io (sio-go) format.
+pub fn decrypt_stream_io(password: &[u8], data: &[u8]) -> Result<Vec<u8>, Error> {
+    if data.len() < STREAM_IO_HEADER_LEN {
+        return Err(Error::ErrUnexpectedHeader);
+    }
+    let salt = &data[0..32];
+    let id = ID::try_from(data[32])?;
+    let nonce_prefix = &data[33..41];
+    let body = &data[STREAM_IO_HEADER_LEN..];
+
+    let key = id.get_key(password, salt)?;
+
+    match id {
+        ID::Argon2idChaCHa20Poly1305 => decrypt_stream(
+            ChaCha20Poly1305::new_from_slice(&key).map_err(|e| Error::ErrInvalidInput(e.to_string()))?,
+            nonce_prefix,
+            body,
+            CHACHA_OVERHEAD,
+        ),
+        _ => decrypt_stream(
+            Aes256Gcm::new_from_slice(&key).map_err(|e| Error::ErrInvalidInput(e.to_string()))?,
+            nonce_prefix,
+            body,
+            AES_GCM_OVERHEAD,
+        ),
+    }
+}
+
+fn decrypt_stream<A>(aead: A, nonce_prefix: &[u8], body: &[u8], overhead: usize) -> Result<Vec<u8>, Error>
+where
+    A: AeadInPlace,
+{
+    let ciphertext_len = SIO_BUF_SIZE + overhead;
+    let ad = build_associated_data(&aead, nonce_prefix)?;
+    let mut plain = Vec::with_capacity(body.len());
+
+    let mut seq_num: u32 = 1;
+    let mut pos = 0;
+
+    while pos < body.len() {
+        let remaining = body.len() - pos;
+        let frag_len = remaining.min(ciphertext_len);
+        let is_last = (pos + frag_len) == body.len();
+
+        if frag_len < overhead {
+            return Err(Error::ErrDecryptFailed(aes_gcm::aead::Error));
+        }
+
+        let mut nonce = [0u8; 12];
+        nonce[0..SIO_NONCE_PREFIX_LEN].copy_from_slice(nonce_prefix);
+        nonce[8..12].copy_from_slice(&seq_num.to_le_bytes());
+
+        let mut ad_mut = ad.clone();
+        ad_mut[0] = if is_last { 0x80 } else { 0x00 };
+
+        let fragment = &body[pos..pos + frag_len];
+        let tag_len = overhead;
+        let (ct, tag) = fragment.split_at(frag_len - tag_len);
+
+        let mut buffer = ct.to_vec();
+        let nonce_arr = Array::<u8, <A as AeadCore>::NonceSize>::try_from(&nonce[..])
+            .map_err(|_| Error::ErrDecryptFailed(aes_gcm::aead::Error))?;
+        let tag_arr =
+            Array::<u8, <A as AeadCore>::TagSize>::try_from(tag).map_err(|_| Error::ErrDecryptFailed(aes_gcm::aead::Error))?;
+        aead.decrypt_in_place_detached(&nonce_arr, &ad_mut, &mut buffer, &tag_arr)
+            .map_err(|_| Error::ErrDecryptFailed(aes_gcm::aead::Error))?;
+        plain.extend_from_slice(&buffer);
+
+        pos += frag_len;
+        seq_num += 1;
+
+        if is_last {
+            break;
+        }
+    }
+
+    Ok(plain)
+}
+
+fn build_associated_data<A>(aead: &A, nonce_prefix: &[u8]) -> Result<Vec<u8>, Error>
+where
+    A: AeadInPlace,
+{
+    let mut nonce = [0u8; 12];
+    nonce[0..SIO_NONCE_PREFIX_LEN].copy_from_slice(nonce_prefix);
+    nonce[8..12].copy_from_slice(&0u32.to_le_bytes());
+
+    let nonce_arr = Array::<u8, <A as AeadCore>::NonceSize>::try_from(&nonce[..])
+        .map_err(|_| Error::ErrEncryptFailed(aes_gcm::aead::Error))?;
+    let mut empty: [u8; 0] = [];
+    let tag = aead
+        .encrypt_in_place_detached(&nonce_arr, &[] as &[u8], &mut empty)
+        .map_err(Error::ErrEncryptFailed)?;
+
+    let mut ad = vec![0u8; 1 + tag.len()];
+    ad[0] = 0x00;
+    ad[1..].copy_from_slice(tag.as_slice());
+    Ok(ad)
+}
+
+/// Encrypt data in stream_io (sio-go) format.
+pub fn encrypt_stream_io(password: &[u8], data: &[u8]) -> Result<Vec<u8>, Error> {
+    let salt: [u8; 32] = rand::random();
+
+    #[cfg(feature = "fips")]
+    let id = ID::Pbkdf2AESGCM;
+
+    #[cfg(not(feature = "fips"))]
+    let id = if crate::encdec::encrypt::native_aes() {
+        ID::Argon2idAESGCM
+    } else {
+        ID::Argon2idChaCHa20Poly1305
+    };
+
+    let key = id.get_key(password, &salt)?;
+    let nonce_prefix: [u8; SIO_NONCE_PREFIX_LEN] = rand::random();
+
+    let mut out = Vec::with_capacity(STREAM_IO_HEADER_LEN + data.len() + 32);
+    out.extend_from_slice(&salt);
+    out.push(id as u8);
+    out.extend_from_slice(&nonce_prefix);
+
+    match id {
+        ID::Argon2idChaCHa20Poly1305 => encrypt_stream(
+            ChaCha20Poly1305::new_from_slice(&key).map_err(|e| Error::ErrInvalidInput(e.to_string()))?,
+            &nonce_prefix,
+            data,
+            &mut out,
+            CHACHA_OVERHEAD,
+        )?,
+        _ => encrypt_stream(
+            Aes256Gcm::new_from_slice(&key).map_err(|e| Error::ErrInvalidInput(e.to_string()))?,
+            &nonce_prefix,
+            data,
+            &mut out,
+            AES_GCM_OVERHEAD,
+        )?,
+    }
+
+    Ok(out)
+}
+
+fn encrypt_stream<A>(
+    aead: A,
+    nonce_prefix: &[u8; SIO_NONCE_PREFIX_LEN],
+    data: &[u8],
+    out: &mut Vec<u8>,
+    _overhead: usize,
+) -> Result<(), Error>
+where
+    A: AeadInPlace,
+{
+    let ad = build_associated_data(&aead, nonce_prefix)?;
+    let mut seq_num: u32 = 1;
+    let mut pos = 0;
+
+    while pos < data.len() {
+        let remaining = data.len() - pos;
+        let is_last = remaining <= SIO_BUF_SIZE;
+
+        let chunk_len = if is_last { remaining } else { SIO_BUF_SIZE };
+        let chunk = &data[pos..pos + chunk_len];
+
+        let mut nonce = [0u8; 12];
+        nonce[0..SIO_NONCE_PREFIX_LEN].copy_from_slice(nonce_prefix);
+        nonce[8..12].copy_from_slice(&seq_num.to_le_bytes());
+
+        let mut ad_mut = ad.clone();
+        ad_mut[0] = if is_last { 0x80 } else { 0x00 };
+
+        let mut buffer = chunk.to_vec();
+        let nonce_arr = Array::<u8, <A as AeadCore>::NonceSize>::try_from(&nonce[..])
+            .map_err(|_| Error::ErrEncryptFailed(aes_gcm::aead::Error))?;
+        let tag = aead
+            .encrypt_in_place_detached(&nonce_arr, &ad_mut, &mut buffer)
+            .map_err(Error::ErrEncryptFailed)?;
+        out.extend_from_slice(&buffer);
+        out.extend_from_slice(tag.as_slice());
+
+        pos += chunk_len;
+        seq_num += 1;
+    }
+
+    Ok(())
+}

crates/crypto/src/encdec/tests.rs

@@ -12,7 +12,7 @@
 // See the License for the specific language governing permissions and
 // limitations under the License.
 
-use crate::{decrypt_data, encrypt_data};
+use crate::{decrypt_data, decrypt_stream_io, encrypt_data, encrypt_stream_io};
 
 const PASSWORD: &[u8] = "test_password".as_bytes();
 const LONG_PASSWORD: &[u8] = "very_long_password_with_many_characters_for_testing_purposes_123456789".as_bytes();
@@ -317,3 +317,67 @@ fn test_concurrent_encryption_safety() -> Result<(), crate::Error> {
 
     Ok(())
 }
+
+#[test]
+fn test_stream_io_roundtrip() -> Result<(), crate::Error> {
+    let password = b"access:secret";
+    let data = br#"{"Version":1,"policy":"readonly"}"#;
+    let encrypted = encrypt_stream_io(password, data)?;
+    let decrypted = decrypt_stream_io(password, &encrypted)?;
+    assert_eq!(data, decrypted.as_slice());
+    Ok(())
+}
+
+#[test]
+fn test_stream_io_large_data_roundtrip() -> Result<(), crate::Error> {
+    let password = b"access:secret";
+    let data = vec![0xAB; 32 * 1024]; // > SIO_BUF_SIZE to test fragmentation
+    let encrypted = encrypt_stream_io(password, &data)?;
+    let decrypted = decrypt_stream_io(password, &encrypted)?;
+    assert_eq!(data, decrypted);
+    Ok(())
+}
+
+#[test]
+fn test_stream_io_wrong_password_fails() {
+    let password = b"access:secret";
+    let data = br#"{"Version":1}"#;
+    let encrypted = encrypt_stream_io(password, data).expect("encrypt should succeed");
+    let result = decrypt_stream_io(b"wrong:password", &encrypted);
+    assert!(result.is_err(), "decrypt with wrong password should fail");
+}
+
+#[test]
+fn test_stream_io_empty_data() -> Result<(), crate::Error> {
+    let password = b"access:secret";
+    let data: &[u8] = &[];
+    let encrypted = encrypt_stream_io(password, data)?;
+    let decrypted = decrypt_stream_io(password, &encrypted)?;
+    assert!(decrypted.is_empty());
+    Ok(())
+}
+
+#[test]
+fn test_stream_io_header_format() -> Result<(), crate::Error> {
+    let password = b"access:secret";
+    let data = b"test";
+    let encrypted = encrypt_stream_io(password, data)?;
+    // stream_io header: salt(32) + alg_id(1) + nonce_prefix(8) = 41 bytes
+    const STREAM_IO_HEADER_LEN: usize = 41;
+    assert!(encrypted.len() >= STREAM_IO_HEADER_LEN, "encrypted should have at least 41-byte header");
+    assert!(
+        encrypted[32] == 0x00 || encrypted[32] == 0x01 || encrypted[32] == 0x02,
+        "alg_id should be 0x00, 0x01, or 0x02"
+    );
+    Ok(())
+}
+
+#[test]
+fn test_stream_io_truncated_data_fails() {
+    let password = b"access:secret";
+    let data = b"test";
+    let encrypted = encrypt_stream_io(password, data).expect("encrypt should succeed");
+    let truncated = &encrypted[..40]; // less than 41-byte header
+    let result = decrypt_stream_io(password, truncated);
+    assert!(result.is_err(), "truncated data should fail decrypt");
+}