Initial plan

.agents/skills/code-change-verification/SKILL.md

@@ -1,78 +0,0 @@
----
-name: code-change-verification
-description: Verify code changes by identifying correctness, regression, security, and performance risks from diffs or patches, then produce prioritized findings with file/line evidence and concrete fixes. Use when reviewing commits, PRs, and merged patches before/after release.
----
-
-# Code Change Verification
-
-Use this skill to review code changes consistently before merge, before release, and during incident follow-up.
-
-## Quick Start
-
-1. Read the scope: commit, PR, patch, or file list.
-2. Map each changed area by risk and user impact.
-3. Inspect each risky change in context.
-4. Report findings first, ordered by severity.
-5. Close with residual risks and verification recommendations.
-
-## Core Workflow
-
-### 1) Scope and assumptions
-- Confirm change source (diff, commit, PR, files), target branch, language/runtime, and version.
-- If context is missing, state assumptions before deeper analysis.
-- Focus only on requested scope; avoid reviewing unrelated files.
-
-### 2) Risk map
-- Prioritize in this order:
-  - Data correctness and user-visible behavior
-  - API/contract compatibility
-  - Security and authz/authn boundaries
-  - Concurrency and lifecycle correctness
-  - Performance and resource usage
-- Give higher priority to stateful paths, migration logic, defaults, and error handling.
-
-### 3) Evidence-based inspection
-- Read each modified hunk with neighboring context.
-- Trace call paths and call-site expectations.
-- Check for:
-  - invariant breaks and missing guards
-  - unchecked assumptions and null/empty/error-path handling
-  - stale tests, fixtures, and configs
-  - hidden coupling to shared helpers/constants/features
-- If a point is uncertain, mark it as an open question instead of guessing.
-
-### 4) Findings-first output
-- Order findings by severity:
-  - P0: critical failure, security breach, or data loss risk
-  - P1: high-impact regression
-  - P2: medium risk correctness gap
-  - P3: low risk/quality debt
-- For each finding include:
-  - Severity
-  - `path:line` reference
-  - concise issue statement
-  - impact and likely failure mode
-  - specific fix or mitigation
-  - validation step to confirm
-- If no issues exist, explicitly state `No findings` and why.
-
-### 5) Close
-- Report assumptions and unknowns.
-- Suggest targeted checks (tests, canary checks, logs/metrics, migration validation).
-
-## Output Template
-
-1. Findings
-2. No findings (if applicable)
-3. Assumptions / Unknowns
-4. Recommended verification steps
-
-## Finding Template
-
-- `[P1] Missing timeout for downstream call`
-  - Location: `path/to/file.rs:123`
-  - Issue: ...
-  - Impact: ...
-  - Fix suggestion: ...
-  - Validation: ...
-

.agents/skills/code-change-verification/agents/openai.yaml

@@ -1,4 +0,0 @@
-interface:
-  display_name: "Code Change Verification"
-  short_description: "Prioritize risks and verify code changes before merge."
-  default_prompt: "Inspect a patch or diff, identify correctness/security/regression risks, and return prioritized findings with file/line evidence and fixes."

.agents/skills/pr-creation-checker/SKILL.md

@@ -1,91 +0,0 @@
----
-name: pr-creation-checker
-description: Prepare PR-ready diffs by validating scope, checking required verification steps, drafting a compliant English PR title/body, and surfacing blockers before opening or updating a pull request in RustFS.
----
-
-# PR Creation Checker
-
-Use this skill before `gh pr create`, before `gh pr edit`, or when reviewing whether a branch is ready for PR.
-
-## Read sources of truth first
-
-- Read `AGENTS.md`.
-- Read `.github/pull_request_template.md`.
-- Use `Makefile` and `.config/make/` for local quality commands.
-- Use `.github/workflows/ci.yml` for CI expectations.
-- Do not restate long command matrices or template sections from memory when the files exist.
-
-## Workflow
-
-1. Collect PR context
-- Confirm base branch, current branch, change goal, and scope.
-- Confirm whether the task is: draft a new PR, update an existing PR, or preflight-check readiness.
-- Confirm whether the branch includes only intended changes.
-
-2. Inspect change scope
-- Review the diff and summarize what changed.
-- Call out unrelated edits, generated artifacts, logs, or secrets as blockers.
-- Mark risky areas explicitly: auth, storage, config, network, migrations, breaking changes.
-- Scan the diff for newly added string literals and confirm whether they duplicate values already defined as constants/enums/typed wrappers in the same module or shared modules.
-- Treat introducing a new hardcoded literal where a project constant already exists as a likely regression risk; require either a refactor to reuse the constant or an explicit exception explanation in the PR body.
-
-3. Verify readiness requirements
-- Require `make pre-commit` before marking the PR ready.
-- If `make` is unavailable, use the equivalent commands from `.config/make/`.
-- Add scope-specific verification commands when the changed area needs more than the baseline.
-- If required checks fail, stop and return `BLOCKED`.
-
-4. Draft PR metadata
-- Write the PR title in English using Conventional Commits and keep it within 72 characters.
-- If a generic PR workflow suggests a different title format, ignore it and follow the repository rule instead.
-- In RustFS, do not use tool-specific prefixes such as `[codex]` when the repository requires Conventional Commits.
-- Keep the PR body in English.
-- Use the exact section headings from `.github/pull_request_template.md`.
-- Fill non-applicable sections with `N/A`.
-- Include verification commands in the PR description.
-- Do not include local filesystem paths in the PR body unless the user explicitly asks for them.
-- Prefer repo-relative paths, command names, and concise summaries over machine-specific paths such as `/Users/...`.
-
-5. Prepare reviewer context
-- Summarize why the change exists.
-- Summarize what was verified.
-- Call out risks, rollout notes, config impact, and rollback notes when applicable.
-- Mention assumptions or missing context instead of guessing.
-
-6. Prepare CLI-safe output
-- When proposing `gh pr create` or `gh pr edit`, use `--body-file`, never inline `--body` for multiline markdown.
-- Return a ready-to-save PR body plus a short title.
-- If not ready, return blockers first and list the minimum steps needed to unblock.
-
-## Output format
-
-### Status
-- `READY` or `BLOCKED`
-
-### Title
-- `<type>(<scope>): <summary>`
-
-### PR Body
-- Reproduce the repository template headings exactly.
-- Fill every section.
-- Omit local absolute paths unless explicitly required.
-
-### Verification
-- List each command run.
-- State pass/fail.
-
-### Risks
-- List breaking changes, config changes, migration impact, or `N/A`.
-
-## Blocker rules
-
-- Return `BLOCKED` if `make pre-commit` has not passed.
-- Return `BLOCKED` if the diff contains unrelated changes that are not acknowledged.
-- Return `BLOCKED` if required template sections are missing.
-- Return `BLOCKED` if the title/body is not in English.
-- Return `BLOCKED` if the title does not follow the repository's Conventional Commit rule.
-- Return `BLOCKED` if the diff introduces string literals that should use existing constants but did not.
-
-## Reference
-
-- Use [pr-readiness-checklist.md](references/pr-readiness-checklist.md) for a short final pass before opening or editing the PR.

.agents/skills/pr-creation-checker/agents/openai.yaml

@@ -1,4 +0,0 @@
-interface:
-  display_name: "PR Creation Checker"
-  short_description: "Draft RustFS-ready PRs with checks, template, and blockers."
-  default_prompt: "Inspect a branch or diff, verify required PR checks, and produce a compliant English PR title/body plus blockers or readiness status."

.agents/skills/pr-creation-checker/references/pr-readiness-checklist.md

@@ -1,15 +0,0 @@
-# PR Readiness Checklist
-
-- Confirm the branch is based on current `main`.
-- Confirm the diff matches the stated scope.
-- Confirm no secrets, logs, temp files, or unrelated refactors are included.
-- Confirm `make pre-commit` passed, or document why it could not run.
-- Confirm extra verification commands are listed for risky changes.
-- Confirm the PR title uses Conventional Commits and stays within 72 characters.
-- Confirm the PR title does not use tool-specific prefixes such as `[codex]`.
-- Confirm the PR body is in English.
-- Confirm the PR body keeps the exact headings from `.github/pull_request_template.md`.
-- Confirm non-applicable sections are filled with `N/A`.
-- Confirm the PR body does not include local absolute paths unless explicitly required.
-- Confirm multiline GitHub CLI commands use `--body-file`.
-- Confirm new hardcoded string literals were not introduced for values already represented by existing constants/enums (including protocol labels, error identifiers, headers, and metric names), or record a justified exception.

.agents/skills/test-coverage-improver/SKILL.md

@@ -1,66 +0,0 @@
----
-name: test-coverage-improver
-description: Run project coverage checks, rank high-risk gaps, and propose high-impact tests to improve regression confidence for changed and critical code paths before release.
----
-
-# Test Coverage Improver
-
-Use this skill when you need a prioritized, risk-aware plan to improve tests from coverage results.
-
-## Usage assumptions
-- Focus scope is either changed lines/files, a module, or the whole repository.
-- Coverage artifact must be generated or provided in a supported format.
-- If required context is missing, call out assumptions explicitly before proposing work.
-
-## Workflow
-
-1. Define scope and baseline
-   - Confirm target language, framework, and branch.
-   - Confirm whether the scope is changed files only or full-repo.
-
-2. Produce coverage snapshot
-   - Rust: `cargo llvm-cov` (or `cargo tarpaulin`) with existing repo config.
-   - JavaScript/TypeScript: `npm test -- --coverage` and read `coverage/coverage-final.json`.
-   - Python: `pytest --cov=<pkg> --cov-report=json` and read `coverage.json`.
-   - Collect total, per-file, and changed-line coverage.
-
-3. Rank highest-risk gaps
-   - Prioritize changed code, branch coverage gaps, and low-confidence boundaries.
-   - Apply the risk rubric in [coverage-prioritization.md](references/coverage-prioritization.md).
-   - Keep shortlist to 5–8 gaps.
-   - For each gap, capture: file, lines, uncovered branches, and estimated risk score.
-
-4. Propose high-impact tests
-   - For each shortlisted gap, output:
-     - Intent and expected behavior.
-     - Normal, edge, and failure scenarios.
-     - Assertions and side effects to verify.
-     - Setup needs (fixtures, mocks, integration dependencies).
-     - Estimated effort (`S/M/L`).
-
-5. Close with validation plan
-   - State which gaps remain after proposals.
-   - Provide concrete verification command and acceptance threshold.
-   - List assumptions or blockers (environment, fixtures, flaky dependencies).
-
-## Output template
-
-### Coverage Snapshot
-- total / branch coverage
-- changed-file coverage
-- top missing regions by size
-
-### Top Gaps (ranked)
-- `path:line-range` | risk score | why critical
-
-### Test Proposals
-- `path:line-range`
-  - Test name
-  - scenarios
-  - assertions
-  - effort
-
-### Validation Plan
-- command
-- pass criteria
-- remaining risk

.agents/skills/test-coverage-improver/agents/openai.yaml

@@ -1,4 +0,0 @@
-interface:
-  display_name: "Test Coverage Improver"
-  short_description: "Find top uncovered risk areas and propose high-impact tests."
-  default_prompt: "Run coverage checks, identify largest gaps, and recommend highest-impact test cases to improve risk coverage."

.agents/skills/test-coverage-improver/references/coverage-prioritization.md

@@ -1,25 +0,0 @@
-# Coverage Gap Prioritization Guide
-
-Use this rubric for each uncovered area.
-
-Score = (Criticality × 2) + CoverageDebt + (Volatility × 0.5)
-
-- Criticality:
-  - 5: authz/authn, data-loss, payment/consistency path
-  - 4: state mutation, cache invalidation, scheduling
-  - 3: error handling + fallbacks in user-visible flows
-  - 2: parsing/format conversion paths
-  - 1: logging-only or low-impact utilities
-
-- CoverageDebt:
-  - 0: 0–5 uncovered lines
-  - 1: 6–20 uncovered lines
-  - 2: 21–40 uncovered lines
-  - 3: 41+ uncovered lines
-
-- Volatility:
-  - 1: stable legacy code with few recent edits
-  - 2: changed in last 2 releases
-  - 3: touched in last 30 days or currently in active PR
-
-Sort by score descending, then by business impact.

.cargo/config.toml

@@ -1,26 +0,0 @@
-# Copyright 2024 RustFS Team
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-# RustFS Cargo configuration
-
-# Enable tokio_unstable cfg for dial9-tokio-telemetry support
-# This allows dial9 to hook into Tokio's internal runtime events
-[build]
-# Enable Tokio unstable features required by dial9-tokio-telemetry for runtime tracing.
-# See: https://docs.rs/tokio/latest/tokio/#unstable-features
-rustflags = ["--cfg", "tokio_unstable"]
-
-# Enable frame pointers for CPU profiling (Linux only, optional but recommended)
-# Uncomment the following line for better CPU profiling data
-# rustflags = ["--cfg", "tokio_unstable", "-C", "force-frame-pointers=yes"]

.config/make/build-docker-buildx-dev.mak

@@ -1,64 +0,0 @@
-## —— Development/Source builds using direct buildx commands ---------------------------------------
-
-.PHONY: docker-dev
-docker-dev: ## Build dev multi-arch image (cannot load locally)
-	@echo "🏗️ Building multi-architecture development Docker images with buildx..."
-	@echo "💡 This builds from source code and is intended for local development and testing"
-	@echo "⚠️  Multi-arch images cannot be loaded locally, use docker-dev-push to push to registry"
-	$(DOCKER_CLI) buildx build \
-		--platform linux/amd64,linux/arm64 \
-		--file $(DOCKERFILE_SOURCE) \
-		--tag rustfs:source-latest \
-		--tag rustfs:dev-latest \
-		.
-
-.PHONY: docker-dev-local
-docker-dev-local: ## Build dev single-arch image (local load)
-	@echo "🏗️ Building single-architecture development Docker image for local use..."
-	@echo "💡 This builds from source code for the current platform and loads locally"
-	$(DOCKER_CLI) buildx build \
-		--file $(DOCKERFILE_SOURCE) \
-		--tag rustfs:source-latest \
-		--tag rustfs:dev-latest \
-		--load \
-		.
-
-.PHONY: docker-dev-push
-docker-dev-push: ## Build and push multi-arch development image # e.g (make docker-dev-push REGISTRY=xxx)
-	@if [ -z "$(REGISTRY)" ]; then \
-		echo "❌ Error: Please specify registry, example: make docker-dev-push REGISTRY=ghcr.io/username"; \
-		exit 1; \
-	fi
-	@echo "🚀 Building and pushing multi-architecture development Docker images..."
-	@echo "💡 Pushing to registry: $(REGISTRY)"
-	$(DOCKER_CLI) buildx build \
-		--platform linux/amd64,linux/arm64 \
-		--file $(DOCKERFILE_SOURCE) \
-		--tag $(REGISTRY)/rustfs:source-latest \
-		--tag $(REGISTRY)/rustfs:dev-latest \
-		--push \
-		.
-
-.PHONY: dev-env-start
-dev-env-start: ## Start development container environment
-	@echo "🚀 Starting development environment..."
-	$(DOCKER_CLI) buildx build \
-		--file $(DOCKERFILE_SOURCE) \
-		--tag rustfs:dev \
-		--load \
-		.
-	$(DOCKER_CLI) stop $(CONTAINER_NAME) 2>/dev/null || true
-	$(DOCKER_CLI) rm $(CONTAINER_NAME) 2>/dev/null || true
-	$(DOCKER_CLI) run -d --name $(CONTAINER_NAME) \
-		-p 9010:9010 -p 9000:9000 \
-		-v $(shell pwd):/workspace \
-		-it rustfs:dev
-
-.PHONY: dev-env-stop
-dev-env-stop: ## Stop development container environment
-	@echo "🛑 Stopping development environment..."
-	$(DOCKER_CLI) stop $(CONTAINER_NAME) 2>/dev/null || true
-	$(DOCKER_CLI) rm $(CONTAINER_NAME) 2>/dev/null || true
-
-.PHONY: dev-env-restart
-dev-env-restart: dev-env-stop dev-env-start ## Restart development container environment

.config/make/build-docker-buildx-production.mak

@@ -1,41 +0,0 @@
-## —— Production builds using docker buildx (for CI/CD and production) -----------------------------
-
-.PHONY: docker-buildx
-docker-buildx: ## Build production multi-arch image (no push)
-	@echo "🏗️ Building multi-architecture production Docker images with buildx..."
-	./docker-buildx.sh
-
-.PHONY: docker-buildx-push
-docker-buildx-push: ## Build and push production multi-arch image
-	@echo "🚀 Building and pushing multi-architecture production Docker images with buildx..."
-	./docker-buildx.sh --push
-
-.PHONY: docker-buildx-version
-docker-buildx-version: ## Build and version production multi-arch image # e.g (make docker-buildx-version VERSION=v1.0.0)
-	@if [ -z "$(VERSION)" ]; then \
-		echo "❌ Error: Please specify version, example: make docker-buildx-version VERSION=v1.0.0"; \
-		exit 1; \
-	fi
-	@echo "🏗️ Building multi-architecture production Docker images (version: $(VERSION))..."
-	./docker-buildx.sh --release $(VERSION)
-
-.PHONY: docker-buildx-push-version
-docker-buildx-push-version: ## Build and version and push production multi-arch image # e.g (make docker-buildx-push-version VERSION=v1.0.0)
-	@if [ -z "$(VERSION)" ]; then \
-		echo "❌ Error: Please specify version, example: make docker-buildx-push-version VERSION=v1.0.0"; \
-		exit 1; \
-	fi
-	@echo "🚀 Building and pushing multi-architecture production Docker images (version: $(VERSION))..."
-	./docker-buildx.sh --release $(VERSION) --push
-
-.PHONY: docker-buildx-production-local
-docker-buildx-production-local: ## Build production single-arch image locally
-	@echo "🏗️ Building single-architecture production Docker image locally..."
-	@echo "💡 Alternative to docker-buildx.sh for local testing"
-	$(DOCKER_CLI) buildx build \
-		--file $(DOCKERFILE_PRODUCTION) \
-		--tag rustfs:production-latest \
-		--tag rustfs:latest \
-		--load \
-		--build-arg RELEASE=latest \
-		.

.config/make/build-docker-production.mak

@@ -1,16 +0,0 @@
-## —— Single Architecture Docker Builds (Traditional) ----------------------------------------------
-
-.PHONY: docker-build-production
-docker-build-production: ## Build single-arch production image
-	@echo "🏗️ Building single-architecture production Docker image..."
-	@echo "💡 Consider using 'make docker-buildx-production-local' for multi-arch support"
-	$(DOCKER_CLI) build -f $(DOCKERFILE_PRODUCTION) -t rustfs:latest .
-
-.PHONY: docker-build-source
-docker-build-source: ## Build single-arch source image
-	@echo "🏗️ Building single-architecture source Docker image..."
-	@echo "💡 Consider using 'make docker-dev-local' for multi-arch support"
-	DOCKER_BUILDKIT=1 $(DOCKER_CLI) build \
-		--build-arg BUILDKIT_INLINE_CACHE=1 \
-		-f $(DOCKERFILE_SOURCE) -t rustfs:source .
-

.config/make/build-docker.mak

@@ -1,22 +0,0 @@
-## —— Docker-based build (alternative approach) ----------------------------------------------------
-
-# Usage: make BUILD_OS=ubuntu22.04 build-docker
-# Output: target/ubuntu22.04/release/rustfs
-
-.PHONY: build-docker
-build-docker: SOURCE_BUILD_IMAGE_NAME = rustfs-$(BUILD_OS):v1
-build-docker: SOURCE_BUILD_CONTAINER_NAME = rustfs-$(BUILD_OS)-build
-build-docker: BUILD_CMD = /root/.cargo/bin/cargo build --release --bin rustfs --target-dir /root/s3-rustfs/target/$(BUILD_OS)
-build-docker: ## Build using Docker container # e.g (make build-docker BUILD_OS=ubuntu22.04)
-	@echo "🐳 Building RustFS using Docker ($(BUILD_OS))..."
-	$(DOCKER_CLI) buildx build -t $(SOURCE_BUILD_IMAGE_NAME) -f $(DOCKERFILE_SOURCE) .
-	$(DOCKER_CLI) run --rm --name $(SOURCE_BUILD_CONTAINER_NAME) -v $(shell pwd):/root/s3-rustfs -it $(SOURCE_BUILD_IMAGE_NAME) $(BUILD_CMD)
-
-.PHONY: docker-inspect-multiarch
-docker-inspect-multiarch: ## Check image architecture support
-	@if [ -z "$(IMAGE)" ]; then \
-		echo "❌ Error: Please specify image, example: make docker-inspect-multiarch IMAGE=rustfs/rustfs:latest"; \
-		exit 1; \
-	fi
-	@echo "🔍 Inspecting multi-architecture image: $(IMAGE)"
-	docker buildx imagetools inspect $(IMAGE)

.config/make/build.mak

@@ -1,55 +0,0 @@
-## —— Local Native Build using build-rustfs.sh script (Recommended) --------------------------------
-
-.PHONY: build
-build: ## Build RustFS binary (includes console by default)
-	@echo "🔨 Building RustFS using build-rustfs.sh script..."
-	./build-rustfs.sh
-
-.PHONY: build-dev
-build-dev: ## Build RustFS in Development mode
-	@echo "🔨 Building RustFS in development mode..."
-	./build-rustfs.sh --dev
-
-.PHONY: build-musl
-build-musl: ## Build x86_64 musl version
-	@echo "🔨 Building rustfs for x86_64-unknown-linux-musl..."
-	@echo "💡 On macOS/Windows, use 'make build-docker' or 'make docker-dev' instead"
-	./build-rustfs.sh --platform x86_64-unknown-linux-musl
-
-.PHONY: build-gnu
-build-gnu: ## Build x86_64 GNU version
-	@echo "🔨 Building rustfs for x86_64-unknown-linux-gnu..."
-	@echo "💡 On macOS/Windows, use 'make build-docker' or 'make docker-dev' instead"
-	./build-rustfs.sh --platform x86_64-unknown-linux-gnu
-
-.PHONY: build-musl-arm64
-build-musl-arm64: ## Build aarch64 musl version
-	@echo "🔨 Building rustfs for aarch64-unknown-linux-musl..."
-	@echo "💡 On macOS/Windows, use 'make build-docker' or 'make docker-dev' instead"
-	./build-rustfs.sh --platform aarch64-unknown-linux-musl
-
-.PHONY: build-gnu-arm64
-build-gnu-arm64: ## Build aarch64 GNU version
-	@echo "🔨 Building rustfs for aarch64-unknown-linux-gnu..."
-	@echo "💡 On macOS/Windows, use 'make build-docker' or 'make docker-dev' instead"
-	./build-rustfs.sh --platform aarch64-unknown-linux-gnu
-
-
-.PHONY: build-cross-all
-build-cross-all: core-deps ## Build binaries for all architectures
-	@echo "🔧 Building all target architectures..."
-	@echo "💡 On macOS/Windows, use 'make docker-dev' for reliable multi-arch builds"
-	@echo "🔨 Generating protobuf code..."
-	cargo run --bin gproto || true
-
-	@echo "🔨 Building rustfs for x86_64-unknown-linux-musl..."
-	./build-rustfs.sh --platform x86_64-unknown-linux-musl
-
-	@echo "🔨 Building rustfs for x86_64-unknown-linux-gnu..."
-	./build-rustfs.sh --platform x86_64-unknown-linux-gnu
-
-	@echo "🔨 Building rustfs for aarch64-unknown-linux-musl..."
-	./build-rustfs.sh --platform aarch64-unknown-linux-musl
-
-	@echo "🔨 Building rustfs for aarch64-unknown-linux-gnu..."
-	./build-rustfs.sh --platform aarch64-unknown-linux-gnu

.config/make/check.mak

@@ -1,24 +0,0 @@
-## —— Check and Inform Dependencies ----------------------------------------------------------------
-
-# Fatal check
-# Checks all required dependencies and exits with error if not found
-# (e.g., cargo, rustfmt)
-check-%:
-	@command -v $* >/dev/null 2>&1 || { \
-		echo >&2 "❌ '$*' is not installed."; \
-		exit 1; \
-	}
-
-# Warning-only check
-# Checks for optional dependencies and issues a warning if not found
-# (e.g., cargo-nextest for enhanced testing)
-warn-%:
-	@command -v $* >/dev/null 2>&1 || { \
-		echo >&2 "⚠️ '$*' is not installed."; \
-	}
-
-# For checking dependencies use check-<dep-name> or warn-<dep-name>
-.PHONY: core-deps fmt-deps test-deps
-core-deps: check-cargo ## Check core dependencies
-fmt-deps: check-rustfmt ## Check lint and formatting dependencies
-test-deps: warn-cargo-nextest ## Check tests dependencies

.config/make/deploy.mak

@@ -1,6 +0,0 @@
-## —— Deploy using dev_deploy.sh script ------------------------------------------------------------
-
-.PHONY: deploy-dev
-deploy-dev: build-musl ## Deploy to dev server
-	@echo "🚀 Deploying to dev server: $${IP}"
-	./scripts/dev_deploy.sh $${IP}

.config/make/help.mak

@@ -1,38 +0,0 @@
-## —— Help, Help Build and Help Docker -------------------------------------------------------------
-
-
-.PHONY: help
-help: ## Shows This Help Menu
-	echo -e "$$HEADER"
-	grep -E '(^[a-zA-Z0-9_-]+:.*?## .*$$)|(^## )' $(MAKEFILE_LIST) | sed 's/^[^:]*://g' | awk 'BEGIN {FS = ":.*?## | #"} ; {printf "${cyan}%-30s${reset} ${white}%s${reset} ${green}%s${reset}\n", $$1, $$2, $$3}' | sed -e 's/\[36m##/\n[32m##/'
-
-.PHONY: help-build
-help-build: ## Shows RustFS build help
-	@echo ""
-	@echo "💡 build-rustfs.sh script provides more options, smart detection and binary verification"
-	@echo ""
-	@echo "🔧 Direct usage of build-rustfs.sh script:"
-	@echo ""
-	@echo "	./build-rustfs.sh --help                                # View script help"
-	@echo "	./build-rustfs.sh --no-console                          # Build without console resources"
-	@echo "	./build-rustfs.sh --force-console-update                # Force update console resources"
-	@echo "	./build-rustfs.sh --dev                                 # Development mode build"
-	@echo "	./build-rustfs.sh --sign                                # Sign binary files"
-	@echo "	./build-rustfs.sh --platform x86_64-unknown-linux-gnu   # Specify target platform"
-	@echo "	./build-rustfs.sh --skip-verification                   # Skip binary verification"
-	@echo ""
-
-.PHONY: help-docker
-help-docker: ## Shows docker environment and suggestion help
-	@echo ""
-	@echo "📋 Environment Variables:"
-	@echo "	REGISTRY                 Image registry address (required for push)"
-	@echo "	DOCKERHUB_USERNAME       Docker Hub username"
-	@echo "	DOCKERHUB_TOKEN          Docker Hub access token"
-	@echo "	GITHUB_TOKEN             GitHub access token"
-	@echo ""
-	@echo "💡 Suggestions:"
-	@echo "	Production use:          Use docker-buildx* commands (based on precompiled binaries)"
-	@echo "	Local development:       Use docker-dev* commands (build from source)"
-	@echo "	Development environment: Use dev-env-* commands to manage dev containers"
-	@echo ""

.config/make/lint-fmt.mak

@@ -1,22 +0,0 @@
-## —— Code quality and Formatting ------------------------------------------------------------------
-
-.PHONY: fmt
-fmt: core-deps fmt-deps ## Format code
-	@echo "🔧 Formatting code..."
-	cargo fmt --all
-
-.PHONY: fmt-check
-fmt-check: core-deps fmt-deps ## Check code formatting
-	@echo "📝 Checking code formatting..."
-	cargo fmt --all --check
-
-.PHONY: clippy-check
-clippy-check: core-deps ## Run clippy checks
-	@echo "🔍 Running clippy checks..."
-	cargo clippy --fix --allow-dirty
-	cargo clippy --all-targets --all-features -- -D warnings
-
-.PHONY: compilation-check
-compilation-check: core-deps ## Run compilation check
-	@echo "🔨 Running compilation check..."
-	cargo check --all-targets

.config/make/pre-commit.mak

@@ -1,11 +0,0 @@
-## —— Pre Commit Checks ----------------------------------------------------------------------------
-
-.PHONY: setup-hooks
-setup-hooks: ## Set up git hooks
-	@echo "🔧 Setting up git hooks..."
-	chmod +x .git/hooks/pre-commit
-	@echo "✅ Git hooks setup complete!"
-
-.PHONY: pre-commit
-pre-commit: fmt clippy-check compilation-check test ## Run pre-commit checks
-	@echo "✅ All pre-commit checks passed!"

.config/make/tests.mak

@@ -1,22 +0,0 @@
-## —— Tests and e2e test ---------------------------------------------------------------------------
-
-TEST_THREADS ?= 1
-
-.PHONY: test
-test: core-deps test-deps ## Run all tests
-	@echo "🧪 Running tests..."
-	@if command -v cargo-nextest >/dev/null 2>&1; then \
-		cargo nextest run --all --exclude e2e_test; \
-	else \
-		echo "ℹ️ cargo-nextest not found; falling back to 'cargo test'"; \
-		cargo test --workspace --exclude e2e_test -- --nocapture --test-threads="$(TEST_THREADS)"; \
-	fi
-	cargo test --all --doc
-
-.PHONY: e2e-server
-e2e-server: ## Run e2e-server tests
-	sh $(shell pwd)/scripts/run.sh
-
-.PHONY: probe-e2e
-probe-e2e: ## Probe e2e tests
-	sh $(shell pwd)/scripts/probe.sh

.docker/README.md

@@ -1,131 +1,261 @@
-# RustFS Docker Infrastructure
+# RustFS Docker Images
 
-This directory contains the complete Docker infrastructure for building, deploying, and monitoring RustFS. It provides ready-to-use configurations for development, testing, and production-grade observability.
+This directory contains Docker configuration files and supporting infrastructure for building and running RustFS container images.
 
-## 📂 Directory Structure
+## 📁 Directory Structure
 
-| Directory | Description | Status |
-| :--- | :--- | :--- |
-| **[`observability/`](observability/README.md)** | **[RECOMMENDED]** Full-stack observability (Prometheus, Grafana, Tempo, Loki). | ✅ Production-Ready |
-| **[`compose/`](compose/README.md)** | Specialized setups (e.g., 4-node distributed cluster testing). | ⚠️ Testing Only |
-| **[`mqtt/`](mqtt/README.md)** | EMQX Broker configuration for MQTT integration testing. | 🧪 Development |
-| **[`openobserve-otel/`](openobserve-otel/README.md)** | Alternative lightweight observability stack using OpenObserve. | 🔄 Alternative |
-
----
-
-## 📄 Root Directory Files
-
-The following files in the project root are essential for Docker operations:
-
-### Build Scripts & Dockerfiles
-
-| File | Description | Usage |
-| :--- | :--- | :--- |
-| **`docker-buildx.sh`** | **Multi-Arch Build Script**<br>Automates building and pushing Docker images for `amd64` and `arm64`. Supports release and dev channels. | `./docker-buildx.sh --push` |
-| **`Dockerfile`** | **Production Image (Alpine)**<br>Lightweight image using musl libc. Downloads pre-built binaries from GitHub Releases. | `docker build -t rustfs:latest .` |
-| **`Dockerfile.glibc`** | **Production Image (Ubuntu)**<br>Standard image using glibc. Useful if you need specific dynamic libraries. | `docker build -f Dockerfile.glibc .` |
-| **`Dockerfile.source`** | **Development Image**<br>Builds RustFS from source code. Includes build tools. Ideal for local development and CI. | `docker build -f Dockerfile.source .` |
-
-### Docker Compose Configurations
-
-| File | Description | Usage |
-| :--- | :--- | :--- |
-| **`docker-compose.yml`** | **Main Development Setup**<br>Comprehensive setup with profiles for development, observability, and proxying. | `docker compose up -d`<br>`docker compose --profile observability up -d` |
-| **`docker-compose-simple.yml`** | **Quick Start Setup**<br>Minimal configuration running a single RustFS instance with 4 volumes. Perfect for first-time users. | `docker compose -f docker-compose-simple.yml up -d` |
-
----
-
-## 🌟 Observability Stack (Recommended)
-
-Located in: [`.docker/observability/`](observability/README.md)
-
-We provide a comprehensive, industry-standard observability stack designed for deep insights into RustFS performance. This is the recommended setup for both development and production monitoring.
-
-### Components
-- **Metrics**: Prometheus (Collection) + Grafana (Visualization)
-- **Traces**: Tempo (Storage) + Jaeger (UI)
-- **Logs**: Loki
-- **Ingestion**: OpenTelemetry Collector
-
-### Key Features
-- **Full Persistence**: All metrics, logs, and traces are saved to Docker volumes, ensuring no data loss on restarts.
-- **Correlation**: Seamlessly jump between Logs, Traces, and Metrics in Grafana.
-- **High Performance**: Optimized configurations for batching, compression, and memory management.
-
-### Quick Start
-```bash
-cd .docker/observability
-docker compose up -d
+```
+rustfs/
+├── Dockerfile           # Production image (Alpine + pre-built binaries)
+├── Dockerfile.source    # Development image (Debian + source build)
+├── docker-buildx.sh     # Multi-architecture build script
+├── Makefile             # Build automation with simplified commands
+└── .docker/             # Supporting infrastructure
+    ├── observability/   # Monitoring and observability configs
+    ├── compose/         # Docker Compose configurations
+    ├── mqtt/            # MQTT broker configs
+    └── openobserve-otel/ # OpenObserve + OpenTelemetry configs
 ```
 
----
+## 🎯 Image Variants
 
-## 🧪 Specialized Environments
+### Core Images
 
-Located in: [`.docker/compose/`](compose/README.md)
+| Image | Base OS | Build Method | Size | Use Case |
+|-------|---------|--------------|------|----------|
+| `production` (default) | Alpine 3.18 | GitHub Releases | Smallest | Production deployment |
+| `source` | Debian Bookworm | Source build | Medium | Custom builds with cross-compilation |
+| `dev` | Debian Bookworm | Development tools | Large | Interactive development |
 
-These configurations are tailored for specific testing scenarios that require complex topologies.
+## 🚀 Usage Examples
+
+### Quick Start (Production)
 
-### Distributed Cluster (4-Nodes)
-Simulates a real-world distributed environment with 4 RustFS nodes running locally.
 ```bash
-docker compose -f .docker/compose/docker-compose.cluster.yaml up -d
+# Default production image (Alpine + GitHub Releases)
+docker run -p 9000:9000 rustfs/rustfs:latest
+
+# Specific version
+docker run -p 9000:9000 rustfs/rustfs:1.2.3
 ```
 
-### Integrated Observability Test
-A self-contained environment running 4 RustFS nodes alongside the full observability stack. Useful for end-to-end testing of telemetry.
+### Complete Tag Strategy Examples
+
 ```bash
-docker compose -f .docker/compose/docker-compose.observability.yaml up -d
+# Stable Releases
+docker run rustfs/rustfs:1.2.3           # Main version (production)
+docker run rustfs/rustfs:1.2.3-production # Explicit production variant
+docker run rustfs/rustfs:1.2.3-source   # Source build variant
+docker run rustfs/rustfs:latest          # Latest stable
+
+# Prerelease Versions
+docker run rustfs/rustfs:1.3.0-alpha.2  # Specific alpha version
+docker run rustfs/rustfs:alpha           # Latest alpha
+docker run rustfs/rustfs:beta            # Latest beta
+docker run rustfs/rustfs:rc              # Latest release candidate
+
+# Development Versions
+docker run rustfs/rustfs:dev             # Latest main branch development
+docker run rustfs/rustfs:dev-13e4a0b     # Specific commit
+docker run rustfs/rustfs:dev-latest      # Latest development
+docker run rustfs/rustfs:main-latest     # Main branch latest
 ```
 
----
+### Development Environment
 
-## 📡 MQTT Integration
-
-Located in: [`.docker/mqtt/`](mqtt/README.md)
-
-Provides an EMQX broker for testing RustFS MQTT features.
-
-### Quick Start
 ```bash
-cd .docker/mqtt
-docker compose up -d
-```
-- **Dashboard**: [http://localhost:18083](http://localhost:18083) (Default: `admin` / `public`)
-- **MQTT Port**: `1883`
+# Quick setup using Makefile (recommended)
+make docker-dev-local     # Build development image locally
+make dev-env-start        # Start development container
 
----
+# Manual Docker commands
+docker run -it -v $(pwd):/workspace -p 9000:9000 rustfs/rustfs:latest-dev
 
-## 👁️ Alternative: OpenObserve
+# Build from source locally
+docker build -f Dockerfile.source -t rustfs:custom .
 
-Located in: [`.docker/openobserve-otel/`](openobserve-otel/README.md)
-
-For users preferring a lightweight, all-in-one solution, we support OpenObserve. It combines logs, metrics, and traces into a single binary and UI.
-
-### Quick Start
-```bash
-cd .docker/openobserve-otel
-docker compose up -d
+# Development with hot reload
+docker-compose up rustfs-dev
 ```
 
----
+## 🏗️ Build Arguments and Scripts
 
-## 🔧 Common Operations
+### Using Makefile Commands (Recommended)
+
+The easiest way to build images using simplified commands:
 
-### Cleaning Up
-To stop all containers and remove volumes (**WARNING**: deletes all persisted data):
 ```bash
-docker compose down -v
+# Development images (build from source)
+make docker-dev-local             # Build for local use (single arch)
+make docker-dev                   # Build multi-arch (for CI/CD)
+make docker-dev-push REGISTRY=xxx # Build and push to registry
+
+# Production images (using pre-built binaries)
+make docker-buildx                # Build multi-arch production images
+make docker-buildx-push           # Build and push production images
+make docker-buildx-version VERSION=v1.0.0  # Build specific version
+
+# Development environment
+make dev-env-start                # Start development container
+make dev-env-stop                 # Stop development container
+make dev-env-restart              # Restart development container
+
+# Help
+make help-docker                  # Show all Docker-related commands
 ```
 
-### Viewing Logs
-To follow logs for a specific service:
+### Using docker-buildx.sh (Advanced)
+
+For direct script usage and advanced scenarios:
+
 ```bash
-docker compose logs -f [service_name]
+# Build latest version for all architectures
+./docker-buildx.sh
+
+# Build and push to registry
+./docker-buildx.sh --push
+
+# Build specific version
+./docker-buildx.sh --release v1.2.3
+
+# Build and push specific version
+./docker-buildx.sh --release v1.2.3 --push
 ```
 
-### Checking Status
-To see the status of all running containers:
+### Manual Docker Builds
+
+All images support dynamic version selection:
+
 ```bash
-docker compose ps
+# Build production image with latest release
+docker build --build-arg RELEASE="latest" -t rustfs:latest .
+
+# Build from source with specific target
+docker build -f Dockerfile.source \
+  --build-arg TARGETPLATFORM="linux/amd64" \
+  -t rustfs:source .
+
+# Development build
+docker build -f Dockerfile.source -t rustfs:dev .
 ```
+
+## 🔧 Binary Download Sources
+
+### Unified GitHub Releases
+
+The production image downloads from GitHub Releases for reliability and transparency:
+
+- ✅ **production** → GitHub Releases API with automatic latest detection
+- ✅ **Checksum verification** → SHA256SUMS validation when available
+- ✅ **Multi-architecture** → Supports amd64 and arm64
+
+### Source Build
+
+The source variant compiles from source code with advanced features:
+
+- 🔧 **Cross-compilation** → Supports multiple target platforms via `TARGETPLATFORM`
+- ⚡ **Build caching** → sccache for faster compilation
+- 🎯 **Optimized builds** → Release optimizations with LTO and symbol stripping
+
+## 📋 Architecture Support
+
+All variants support multi-architecture builds:
+
+- **linux/amd64** (x86_64)
+- **linux/arm64** (aarch64)
+
+Architecture is automatically detected during build using Docker's `TARGETARCH` build argument.
+
+## 🔐 Security Features
+
+- **Checksum Verification**: Production image verifies SHA256SUMS when available
+- **Non-root User**: All images run as user `rustfs` (UID 1000)
+- **Minimal Runtime**: Production image only includes necessary dependencies
+- **Secure Defaults**: No hardcoded credentials or keys
+
+## 🛠️ Development Workflow
+
+### Quick Start with Makefile (Recommended)
+
+```bash
+# 1. Start development environment
+make dev-env-start
+
+# 2. Your development container is now running with:
+#    - Port 9000 exposed for RustFS
+#    - Port 9010 exposed for admin console
+#    - Current directory mounted as /workspace
+
+# 3. Stop when done
+make dev-env-stop
+```
+
+### Manual Development Setup
+
+```bash
+# Build development image from source
+make docker-dev-local
+
+# Or use traditional Docker commands
+docker build -f Dockerfile.source -t rustfs:dev .
+
+# Run with development tools
+docker run -it -v $(pwd):/workspace -p 9000:9000 rustfs:dev bash
+
+# Or use docker-compose for complex setups
+docker-compose up rustfs-dev
+```
+
+### Common Development Tasks
+
+```bash
+# Build and test locally
+make build                        # Build binary natively
+make docker-dev-local            # Build development Docker image
+make test                        # Run tests
+make fmt                         # Format code
+make clippy                      # Run linter
+
+# Get help
+make help                        # General help
+make help-docker                 # Docker-specific help
+make help-build                  # Build-specific help
+```
+
+## 🚀 CI/CD Integration
+
+The project uses GitHub Actions for automated multi-architecture Docker builds:
+
+### Automated Builds
+
+- **Tags**: Automatic builds triggered on version tags (e.g., `v1.2.3`)
+- **Main Branch**: Development builds with `dev-latest` and `main-latest` tags
+- **Pull Requests**: Test builds without registry push
+
+### Build Variants
+
+Each build creates three image variants:
+
+- `rustfs/rustfs:v1.2.3` (production - Alpine-based)
+- `rustfs/rustfs:v1.2.3-source` (source build - Debian-based)
+- `rustfs/rustfs:v1.2.3-dev` (development - Debian-based with tools)
+
+### Manual Builds
+
+Trigger custom builds via GitHub Actions:
+
+```bash
+# Use workflow_dispatch to build specific versions
+# Available options: latest, main-latest, dev-latest, v1.2.3, dev-abc123
+```
+
+## 📦 Supporting Infrastructure
+
+The `.docker/` directory contains supporting configuration files:
+
+- **observability/** - Prometheus, Grafana, OpenTelemetry configs
+- **compose/** - Multi-service Docker Compose setups
+- **mqtt/** - MQTT broker configurations
+- **openobserve-otel/** - Log aggregation and tracing setup
+
+See individual README files in each subdirectory for specific usage instructions.

.docker/compose/README.md

@@ -1,44 +1,80 @@
-# Specialized Docker Compose Configurations
+# Docker Compose Configurations
 
-This directory contains specialized Docker Compose configurations for specific testing scenarios.
-
-## ⚠️ Important Note
-
-**For Observability:**
-We **strongly recommend** using the new, fully integrated observability stack located in `../observability/`. It provides a production-ready setup with Prometheus, Grafana, Tempo, Loki, and OpenTelemetry Collector, all with persistent storage and optimized configurations.
-
-The `docker-compose.observability.yaml` in this directory is kept for legacy reference or specific minimal testing needs but is **not** the primary recommended setup.
+This directory contains specialized Docker Compose configurations for different use cases.
 
 ## 📁 Configuration Files
 
-### Cluster Testing
+This directory contains specialized Docker Compose configurations and their associated Dockerfiles, keeping related files organized together.
 
-- **`docker-compose.cluster.yaml`**
-  - **Purpose**: Simulates a 4-node RustFS distributed cluster.
-  - **Use Case**: Testing distributed storage logic, consensus, and failover.
-  - **Nodes**: 4 RustFS instances.
-  - **Storage**: Uses local HTTP endpoints.
+### Main Configuration (Root Directory)
 
-### Legacy / Minimal Observability
+- **`../../docker-compose.yml`** - **Default Production Setup**
+  - Complete production-ready configuration
+  - Includes RustFS server + full observability stack
+  - Supports multiple profiles: `dev`, `observability`, `cache`, `proxy`
+  - Recommended for most users
 
-- **`docker-compose.observability.yaml`**
-  - **Purpose**: A minimal observability setup.
-  - **Status**: **Deprecated**. Please use `../observability/docker-compose.yml` instead.
+### Specialized Configurations
+
+- **`docker-compose.cluster.yaml`** - **Distributed Testing**
+  - 4-node cluster setup for testing distributed storage
+  - Uses local compiled binaries
+  - Simulates multi-node environment
+  - Ideal for development and cluster testing
+
+- **`docker-compose.observability.yaml`** - **Observability Focus**
+  - Specialized setup for testing observability features
+  - Includes OpenTelemetry, Jaeger, Prometheus, Loki, Grafana
+  - Uses `../../Dockerfile.source` for builds
+  - Perfect for observability development
 
 ## 🚀 Usage Examples
 
+### Production Setup
+
+```bash
+# Start main service
+docker-compose up -d
+
+# Start with development profile
+docker-compose --profile dev up -d
+
+# Start with full observability
+docker-compose --profile observability up -d
+```
+
 ### Cluster Testing
 
-To start a 4-node cluster for distributed testing:
-
 ```bash
-# From project root
-docker compose -f .docker/compose/docker-compose.cluster.yaml up -d
+# Build and start 4-node cluster (run from project root)
+cd .docker/compose
+docker-compose -f docker-compose.cluster.yaml up -d
+
+# Or run directly from project root
+docker-compose -f .docker/compose/docker-compose.cluster.yaml up -d
 ```
 
-### (Deprecated) Minimal Observability
+### Observability Testing
 
 ```bash
-# From project root
-docker compose -f .docker/compose/docker-compose.observability.yaml up -d
+# Start observability-focused environment (run from project root)
+cd .docker/compose
+docker-compose -f docker-compose.observability.yaml up -d
+
+# Or run directly from project root
+docker-compose -f .docker/compose/docker-compose.observability.yaml up -d
 ```
+
+## 🔧 Configuration Overview
+
+| Configuration | Nodes | Storage | Observability | Use Case |
+|---------------|-------|---------|---------------|----------|
+| **Main** | 1 | Volume mounts | Full stack | Production |
+| **Cluster** | 4 | HTTP endpoints | Basic | Testing |
+| **Observability** | 4 | Local data | Advanced | Development |
+
+## 📝 Notes
+
+- Always ensure you have built the required binaries before starting cluster tests
+- The main configuration is sufficient for most use cases
+- Specialized configurations are for specific testing scenarios

.docker/compose/docker-compose.observability.yaml

@@ -13,126 +13,62 @@
 # limitations under the License.
 
 services:
-  # --- Observability Stack ---
-
-  tempo-init:
-    image: busybox:latest
-    command: [ "sh", "-c", "chown -R 10001:10001 /var/tempo" ]
-    volumes:
-      - tempo-data:/var/tempo
-    user: root
-    networks:
-      - rustfs-network
-    restart: "no"
-
-  tempo:
-    image: grafana/tempo:latest
-    user: "10001"
-    command: [ "-config.file=/etc/tempo.yaml" ]
-    volumes:
-      - ../../.docker/observability/tempo.yaml:/etc/tempo.yaml:ro
-      - tempo-data:/var/tempo
-    ports:
-      - "3200:3200" # tempo
-      - "4317"      # otlp grpc
-      - "4318"      # otlp http
-    restart: unless-stopped
-    networks:
-      - rustfs-network
-
   otel-collector:
-    image: otel/opentelemetry-collector-contrib:latest
+    image: otel/opentelemetry-collector-contrib:0.129.1
     environment:
       - TZ=Asia/Shanghai
     volumes:
-      - ../../.docker/observability/otel-collector-config.yaml:/etc/otelcol-contrib/config.yaml:ro
+      - ../../.docker/observability/otel-collector-config.yaml:/etc/otelcol-contrib/config.yaml
     ports:
-      - "1888:1888"   # pprof
-      - "8888:8888"   # Prometheus metrics for Collector
-      - "8889:8889"   # Prometheus metrics for application indicators
-      - "13133:13133" # health check
-      - "4317:4317"   # OTLP gRPC
-      - "4318:4318"   # OTLP HTTP
-      - "55679:55679" # zpages
+      - 1888:1888
+      - 8888:8888
+      - 8889:8889
+      - 13133:13133
+      - 4317:4317
+      - 4318:4318
+      - 55679:55679
     networks:
       - rustfs-network
-    depends_on:
-      - tempo
-      - jaeger
-      - prometheus
-      - loki
-
   jaeger:
-    image: jaegertracing/jaeger:latest
+    image: jaegertracing/jaeger:2.8.0
     environment:
       - TZ=Asia/Shanghai
-      - SPAN_STORAGE_TYPE=badger
-      - BADGER_EPHEMERAL=false
-      - BADGER_DIRECTORY_VALUE=/badger/data
-      - BADGER_DIRECTORY_KEY=/badger/key
-      - COLLECTOR_OTLP_ENABLED=true
-    volumes:
-      - jaeger-data:/badger
     ports:
-      - "16686:16686" # Web UI
-      - "14269:14269" # Admin/Metrics
+      - "16686:16686"
+      - "14317:4317"
+      - "14318:4318"
     networks:
       - rustfs-network
-
   prometheus:
-    image: prom/prometheus:latest
+    image: prom/prometheus:v3.4.2
     environment:
       - TZ=Asia/Shanghai
     volumes:
-      - ../../.docker/observability/prometheus.yml:/etc/prometheus/prometheus.yml:ro
-      - prometheus-data:/prometheus
+      - ../../.docker/observability/prometheus.yml:/etc/prometheus/prometheus.yml
     ports:
       - "9090:9090"
-    command:
-      - '--config.file=/etc/prometheus/prometheus.yml'
-      - '--web.enable-otlp-receiver'
-      - '--web.enable-remote-write-receiver'
-      - '--enable-feature=promql-experimental-functions'
-      - '--storage.tsdb.path=/prometheus'
-      - '--web.console.libraries=/usr/share/prometheus/console_libraries'
-      - '--web.console.templates=/usr/share/prometheus/consoles'
     networks:
       - rustfs-network
-
   loki:
-    image: grafana/loki:latest
+    image: grafana/loki:3.5.1
     environment:
       - TZ=Asia/Shanghai
     volumes:
-      - ../../.docker/observability/loki.yaml:/etc/loki/local-config.yaml:ro
-      - loki-data:/loki
+      - ../../.docker/observability/loki-config.yaml:/etc/loki/local-config.yaml
     ports:
       - "3100:3100"
     command: -config.file=/etc/loki/local-config.yaml
     networks:
       - rustfs-network
-
   grafana:
-    image: grafana/grafana:latest
+    image: grafana/grafana:12.0.2
     ports:
       - "3000:3000" # Web UI
     environment:
       - GF_SECURITY_ADMIN_PASSWORD=admin
-      - GF_SECURITY_ADMIN_USER=admin
       - TZ=Asia/Shanghai
-      - GF_INSTALL_PLUGINS=grafana-pyroscope-datasource
-      - GF_DASHBOARDS_DEFAULT_HOME_DASHBOARD_PATH=/var/lib/grafana/dashboards/home.json
     networks:
       - rustfs-network
-    volumes:
-      - ../../.docker/observability/grafana/provisioning:/etc/grafana/provisioning:ro
-      - ../../.docker/observability/grafana/dashboards:/var/lib/grafana/dashboards:ro
-    depends_on:
-      - prometheus
-      - tempo
-      - loki
-
-  # --- RustFS Cluster ---
 
   node1:
     build:
@@ -143,15 +79,13 @@ services:
       - RUSTFS_VOLUMES=http://node{1...4}:9000/root/data/target/volume/test{1...4}
       - RUSTFS_ADDRESS=:9000
       - RUSTFS_CONSOLE_ENABLE=true
-      - RUSTFS_OBS_ENDPOINT=http://otel-collector:4318
+      - RUSTFS_OBS_ENDPOINT=http://otel-collector:4317
       - RUSTFS_OBS_LOGGER_LEVEL=debug
     platform: linux/amd64
     ports:
-      - "9001:9000"
+      - "9001:9000" # Map port 9001 of the host to port 9000 of the container
     networks:
       - rustfs-network
-    depends_on:
-      - otel-collector
 
   node2:
     build:
@@ -162,15 +96,13 @@ services:
       - RUSTFS_VOLUMES=http://node{1...4}:9000/root/data/target/volume/test{1...4}
       - RUSTFS_ADDRESS=:9000
       - RUSTFS_CONSOLE_ENABLE=true
-      - RUSTFS_OBS_ENDPOINT=http://otel-collector:4318
+      - RUSTFS_OBS_ENDPOINT=http://otel-collector:4317
       - RUSTFS_OBS_LOGGER_LEVEL=debug
     platform: linux/amd64
     ports:
-      - "9002:9000"
+      - "9002:9000" # Map port 9002 of the host to port 9000 of the container
     networks:
       - rustfs-network
-    depends_on:
-      - otel-collector
 
   node3:
     build:
@@ -181,15 +113,13 @@ services:
       - RUSTFS_VOLUMES=http://node{1...4}:9000/root/data/target/volume/test{1...4}
       - RUSTFS_ADDRESS=:9000
       - RUSTFS_CONSOLE_ENABLE=true
-      - RUSTFS_OBS_ENDPOINT=http://otel-collector:4318
+      - RUSTFS_OBS_ENDPOINT=http://otel-collector:4317
       - RUSTFS_OBS_LOGGER_LEVEL=debug
     platform: linux/amd64
     ports:
-      - "9003:9000"
+      - "9003:9000" # Map port 9003 of the host to port 9000 of the container
     networks:
       - rustfs-network
-    depends_on:
-      - otel-collector
 
   node4:
     build:
@@ -200,21 +130,13 @@ services:
       - RUSTFS_VOLUMES=http://node{1...4}:9000/root/data/target/volume/test{1...4}
       - RUSTFS_ADDRESS=:9000
       - RUSTFS_CONSOLE_ENABLE=true
-      - RUSTFS_OBS_ENDPOINT=http://otel-collector:4318
+      - RUSTFS_OBS_ENDPOINT=http://otel-collector:4317
       - RUSTFS_OBS_LOGGER_LEVEL=debug
     platform: linux/amd64
     ports:
-      - "9004:9000"
+      - "9004:9000" # Map port 9004 of the host to port 9000 of the container
     networks:
       - rustfs-network
-    depends_on:
-      - otel-collector
-
-volumes:
-  prometheus-data:
-  tempo-data:
-  loki-data:
-  jaeger-data:
 
 networks:
   rustfs-network:

.docker/mqtt/README.md

@@ -1,30 +0,0 @@
-# MQTT Broker (EMQX)
-
-This directory contains the configuration for running an EMQX MQTT broker, which can be used for testing RustFS's MQTT integration.
-
-## 🚀 Quick Start
-
-To start the EMQX broker:
-
-```bash
-docker compose up -d
-```
-
-## 📊 Access
-
-- **Dashboard**: [http://localhost:18083](http://localhost:18083)
-- **Default Credentials**: `admin` / `public`
-- **MQTT Port**: `1883`
-- **WebSocket Port**: `8083`
-
-## 🛠️ Configuration
-
-The `docker-compose.yml` file sets up a single-node EMQX instance.
-
-- **Persistence**: Data is not persisted by default (for testing).
-- **Network**: Uses the default bridge network.
-
-## 📝 Notes
-
-- This setup is intended for development and testing purposes.
-- For production deployments, please refer to the official [EMQX Documentation](https://www.emqx.io/docs/en/latest/).

.docker/nginx/nginx.conf

@@ -1,82 +0,0 @@
-# Copyright 2024 RustFS Team
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-worker_processes auto;
-pid /var/run/nginx.pid;
-
-events {
-    worker_connections 1024;
-}
-
-http {
-    include /etc/nginx/mime.types;
-    default_type application/octet-stream;
-
-    log_format main '$remote_addr - $remote_user [$time_local] "$request" '
-                    '$status $body_bytes_sent "$http_referer" '
-                    '"$http_user_agent" "$http_x_forwarded_for"';
-
-    access_log /var/log/nginx/access.log main;
-    error_log /var/log/nginx/error.log warn;
-
-    sendfile on;
-    keepalive_timeout 65;
-
-    # RustFS Server Block
-    server {
-        listen 80;
-        server_name localhost;
-
-        # Redirect HTTP to HTTPS (optional, uncomment if SSL is configured)
-        # return 301 https://$host$request_uri;
-
-        location / {
-            proxy_pass http://rustfs:9000;
-            proxy_set_header Host $host;
-            proxy_set_header X-Real-IP $remote_addr;
-            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
-            proxy_set_header X-Forwarded-Proto $scheme;
-
-            # S3 specific headers
-            proxy_set_header X-Amz-Date $http_x_amz_date;
-            proxy_set_header Authorization $http_authorization;
-
-            # Disable buffering for large uploads
-            proxy_request_buffering off;
-            client_max_body_size 0;
-        }
-
-        location /rustfs/console {
-            proxy_pass http://rustfs:9001;
-            proxy_set_header Host $host;
-            proxy_set_header X-Real-IP $remote_addr;
-            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
-            proxy_set_header X-Forwarded-Proto $scheme;
-        }
-    }
-
-    # SSL Configuration (Example)
-    # server {
-    #     listen 443 ssl;
-    #     server_name localhost;
-    #
-    #     ssl_certificate /etc/nginx/ssl/server.crt;
-    #     ssl_certificate_key /etc/nginx/ssl/server.key;
-    #
-    #     location / {
-    #         proxy_pass http://rustfs:9000;
-    #         ...
-    #     }
-    # }
-}

.docker/observability/.gitignore

@@ -1,5 +0,0 @@
-jaeger-data/*
-loki-data/*
-prometheus-data/*
-tempo-data/*
-grafana-data/*

.docker/observability/README.md

@@ -1,85 +1,109 @@
-# RustFS Observability Stack
+# Observability
 
-This directory contains the comprehensive observability stack for RustFS, designed to provide deep insights into application performance, logs, and traces.
+This directory contains the observability stack for the application. The stack is composed of the following components:
 
-## Components
+- Prometheus v3.2.1
+- Grafana 11.6.0
+- Loki 3.4.2
+- Jaeger 2.4.0
+- Otel Collector 0.120.0 # 0.121.0 remove loki
 
-The stack is composed of the following best-in-class open-source components:
+## Prometheus
 
-- **Prometheus** (v2.53.1): The industry standard for metric collection and alerting.
-- **Grafana** (v11.1.0): The leading platform for observability visualization.
-- **Loki** (v3.1.0): A horizontally-scalable, highly-available, multi-tenant log aggregation system.
-- **Tempo** (v2.5.0): A high-volume, minimal dependency distributed tracing backend.
-- **Jaeger** (v1.59.0): Distributed tracing system (configured as a secondary UI/storage).
-- **OpenTelemetry Collector** (v0.104.0): A vendor-agnostic implementation for receiving, processing, and exporting telemetry data.
+Prometheus is a monitoring and alerting toolkit. It scrapes metrics from instrumented jobs, either directly or via an
+intermediary push gateway for short-lived jobs. It stores all scraped samples locally and runs rules over this data to
+either aggregate and record new time series from existing data or generate alerts. Grafana or other API consumers can be
+used to visualize the collected data.
 
-## Architecture
+## Grafana
 
-1. **Telemetry Collection**: Applications send OTLP (OpenTelemetry Protocol) data (Metrics, Logs, Traces) to the **OpenTelemetry Collector**.
-2. **Processing & Exporting**: The Collector processes the data (batching, memory limiting) and exports it to the respective backends:
-    - **Traces** -> **Tempo** (Primary) & **Jaeger** (Secondary/Optional)
-    - **Metrics** -> **Prometheus** (via scraping the Collector's exporter)
-    - **Logs** -> **Loki**
-3. **Visualization**: **Grafana** connects to all backends (Prometheus, Tempo, Loki, Jaeger) to provide a unified dashboard experience.
+Grafana is a multi-platform open-source analytics and interactive visualization web application. It provides charts,
+graphs, and alerts for the web when connected to supported data sources.
 
-## Features
+## Loki
 
-- **Full Persistence**: All data (Metrics, Logs, Traces) is persisted to Docker volumes, ensuring no data loss on restart.
-- **Correlation**: Seamless navigation between Metrics, Logs, and Traces in Grafana.
-  - Jump from a Metric spike to relevant Traces.
-  - Jump from a Trace to relevant Logs.
-- **High Performance**: Optimized configurations for batching, compression, and memory management.
-- **Standardized Protocols**: Built entirely on OpenTelemetry standards.
+Loki is a horizontally-scalable, highly-available, multi-tenant log aggregation system inspired by Prometheus. It is
+designed to be very cost-effective and easy to operate. It does not index the contents of the logs, but rather a set of
+labels for each log stream.
 
-## Quick Start
+## Jaeger
 
-### Prerequisites
+Jaeger is a distributed tracing system released as open source by Uber Technologies. It is used for monitoring and
+troubleshooting microservices-based distributed systems, including:
 
-- Docker
-- Docker Compose
+- Distributed context propagation
+- Distributed transaction monitoring
+- Root cause analysis
+- Service dependency analysis
+- Performance / latency optimization
 
-### Deploy
+## Otel Collector
 
-Run the following command to start the entire stack:
+The OpenTelemetry Collector offers a vendor-agnostic implementation on how to receive, process, and export telemetry
+data. It removes the need to run, operate, and maintain multiple agents/collectors in order to support open-source
+observability data formats (e.g. Jaeger, Prometheus, etc.) sending to one or more open-source or commercial back-ends.
+
+## How to use
+
+To deploy the observability stack, run the following command:
+
+- docker latest version
 
 ```bash
-docker compose up -d
+docker compose -f docker-compose.yml -f docker-compose.override.yml up -d
 ```
 
-### Access Dashboards
-
-| Service        | URL                                              | Credentials       | Description                    |
-| :------------- | :----------------------------------------------- | :---------------- | :----------------------------- |
-| **Grafana**    | [http://localhost:3000](http://localhost:3000)   | `admin` / `admin` | Main visualization hub.        |
-| **Prometheus** | [http://localhost:9090](http://localhost:9090)   | -                 | Metric queries and status.     |
-| **Jaeger UI**  | [http://localhost:16686](http://localhost:16686) | -                 | Secondary trace visualization. |
-| **Tempo**      | [http://localhost:3200](http://localhost:3200)   | -                 | Tempo status/metrics.          |
-
-## Configuration
-
-### Data Persistence
-
-Data is stored in the following Docker volumes:
-
-- `prometheus-data`: Prometheus metrics
-- `tempo-data`: Tempo traces (WAL and Blocks)
-- `loki-data`: Loki logs (Chunks and Rules)
-- `jaeger-data`: Jaeger traces (Badger DB)
-
-To clear all data:
+- docker compose v2.0.0 or before
 
 ```bash
-docker compose down -v
+docke-compose -f docker-compose.yml -f docker-compose.override.yml up -d
 ```
 
-### Customization
+To access the Grafana dashboard, navigate to `http://localhost:3000` in your browser. The default username and password
+are `admin` and `admin`, respectively.
+
+To access the Jaeger dashboard, navigate to `http://localhost:16686` in your browser.
+
+To access the Prometheus dashboard, navigate to `http://localhost:9090` in your browser.
+
+## How to stop
+
+To stop the observability stack, run the following command:
+
+```bash
+docker compose -f docker-compose.yml -f docker-compose.override.yml down
+```
+
+## How to remove data
+
+To remove the data generated by the observability stack, run the following command:
+
+```bash
+docker compose -f docker-compose.yml -f docker-compose.override.yml down -v
+```
+
+## How to configure
+
+To configure the observability stack, modify the `docker-compose.override.yml` file. The file contains the following
+
+```yaml
+services:
+  prometheus:
+    environment:
+      - PROMETHEUS_CONFIG_FILE=/etc/prometheus/prometheus.yml
+    volumes:
+      - ./prometheus.yml:/etc/prometheus/prometheus.yml
+
+  grafana:
+    environment:
+      - GF_SECURITY_ADMIN_PASSWORD=admin
+    volumes:
+      - ./grafana/provisioning:/etc/grafana/provisioning
+```
+
+The `prometheus` service mounts the `prometheus.yml` file to `/etc/prometheus/prometheus.yml`. The `grafana` service
+mounts the `grafana/provisioning` directory to `/etc/grafana/provisioning`. You can modify these files to configure the
+observability stack.
 
-- **Prometheus**: Edit `prometheus.yml` to add scrape targets or alerting rules.
-- **Grafana**: Dashboards and datasources are provisioned from the `grafana/` directory.
-- **Collector**: Edit `otel-collector-config.yaml` to modify pipelines, processors, or exporters.
 
-## Troubleshooting
 
-- **Service Health**: Check the health of services using `docker compose ps`.
-- **Logs**: View logs for a specific service using `docker compose logs -f <service_name>`.
-- **Otel Collector**: Check `http://localhost:13133` for health status and `http://localhost:1888/debug/pprof/` for profiling.

.docker/observability/README_ZH.md

@@ -1,85 +1,27 @@
-# RustFS 可观测性技术栈
+## 部署可观测性系统
 
-本目录包含 RustFS 的全面可观测性技术栈，旨在提供对应用程序性能、日志和追踪的深入洞察。
+OpenTelemetry Collector 提供了一个厂商中立的遥测数据处理方案，用于接收、处理和导出遥测数据。它消除了为支持多种开源可观测性数据格式（如
+Jaeger、Prometheus 等）而需要运行和维护多个代理/收集器的必要性。
 
-## 组件
+### 快速部署
 
-该技术栈由以下一流的开源组件组成：
-
-- **Prometheus** (v2.53.1): 行业标准的指标收集和告警工具。
-- **Grafana** (v11.1.0): 领先的可观测性可视化平台。
-- **Loki** (v3.1.0): 水平可扩展、高可用、多租户的日志聚合系统。
-- **Tempo** (v2.5.0): 高吞吐量、最小依赖的分布式追踪后端。
-- **Jaeger** (v1.59.0): 分布式追踪系统（配置为辅助 UI/存储）。
-- **OpenTelemetry Collector** (v0.104.0): 接收、处理和导出遥测数据的供应商无关实现。
-
-## 架构
-
-1.  **遥测收集**: 应用程序将 OTLP (OpenTelemetry Protocol) 数据（指标、日志、追踪）发送到 **OpenTelemetry Collector**。
-2.  **处理与导出**: Collector 处理数据（批处理、内存限制）并将其导出到相应的后端：
-    -   **追踪** -> **Tempo** (主要) & **Jaeger** (辅助/可选)
-    -   **指标** -> **Prometheus** (通过抓取 Collector 的导出器)
-    -   **日志** -> **Loki**
-3.  **可视化**: **Grafana** 连接到所有后端（Prometheus, Tempo, Loki, Jaeger），提供统一的仪表盘体验。
-
-## 特性
-
--   **完全持久化**: 所有数据（指标、日志、追踪）都持久化到 Docker 卷，确保重启后无数据丢失。
--   **关联性**: 在 Grafana 中实现指标、日志和追踪之间的无缝导航。
-    -   从指标峰值跳转到相关追踪。
-    -   从追踪跳转到相关日志。
--   **高性能**: 针对批处理、压缩和内存管理进行了优化配置。
--   **标准化协议**: 完全基于 OpenTelemetry 标准构建。
-
-## 快速开始
-
-### 前置条件
-
--   Docker
--   Docker Compose
-
-### 部署
-
-运行以下命令启动整个技术栈：
+1. 进入 `.docker/observability` 目录
+2. 执行以下命令启动服务：
 
 ```bash
-docker compose up -d
+docker compose -f docker-compose.yml  up -d
 ```
 
-### 访问仪表盘
+### 访问监控面板
 
-| 服务 | URL | 凭据 | 描述 |
-| :--- | :--- | :--- | :--- |
-| **Grafana** | [http://localhost:3000](http://localhost:3000) | `admin` / `admin` | 主要可视化中心。 |
-| **Prometheus** | [http://localhost:9090](http://localhost:9090) | - | 指标查询和状态。 |
-| **Jaeger UI** | [http://localhost:16686](http://localhost:16686) | - | 辅助追踪可视化。 |
-| **Tempo** | [http://localhost:3200](http://localhost:3200) | - | Tempo 状态/指标。 |
+服务启动后，可通过以下地址访问各个监控面板：
 
-## 配置
+- Grafana: `http://localhost:3000` (默认账号/密码：`admin`/`admin`)
+- Jaeger: `http://localhost:16686`
+- Prometheus: `http://localhost:9090`
 
-### 数据持久化
+## 配置可观测性
 
-数据存储在以下 Docker 卷中：
-
--   `prometheus-data`: Prometheus 指标
--   `tempo-data`: Tempo 追踪 (WAL 和 Blocks)
--   `loki-data`: Loki 日志 (Chunks 和 Rules)
--   `jaeger-data`: Jaeger 追踪 (Badger DB)
-
-要清除所有数据：
-
-```bash
-docker compose down -v
+```shell
+export RUSTFS_OBS_ENDPOINT="http://localhost:4317" # OpenTelemetry Collector 地址
 ```
-
-### 自定义
-
--   **Prometheus**: 编辑 `prometheus.yml` 以添加抓取目标或告警规则。
--   **Grafana**: 仪表盘和数据源从 `grafana/` 目录预置。
--   **Collector**: 编辑 `otel-collector-config.yaml` 以修改管道、处理器或导出器。
-
-## 故障排除
-
--   **服务健康**: 使用 `docker compose ps` 检查服务健康状况。
--   **日志**: 使用 `docker compose logs -f <service_name>` 查看特定服务的日志。
--   **Otel Collector**: 检查 `http://localhost:13133` 获取健康状态，检查 `http://localhost:1888/debug/pprof/` 进行性能分析。

.docker/observability/docker-compose-example-for-rustfs.yml

@@ -1,270 +0,0 @@
-# Copyright 2024 RustFS Team
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-services:
-  rustfs:
-    security_opt:
-      - "no-new-privileges:true"
-    image: rustfs/rustfs:latest
-    container_name: rustfs-server
-    ports:
-      - "9000:9000" # S3 API port
-      - "9001:9001" # Console port
-    environment:
-      - RUSTFS_VOLUMES=/data/rustfs
-      - RUSTFS_ADDRESS=0.0.0.0:9000
-      - RUSTFS_CONSOLE_ADDRESS=0.0.0.0:9001
-      - RUSTFS_CONSOLE_ENABLE=true
-      - RUSTFS_CORS_ALLOWED_ORIGINS=*
-      - RUSTFS_CONSOLE_CORS_ALLOWED_ORIGINS=*
-      - RUSTFS_ACCESS_KEY=rustfsadmin
-      - RUSTFS_SECRET_KEY=rustfsadmin
-      - RUSTFS_OBS_LOGGER_LEVEL=info
-      - RUSTFS_OBS_ENDPOINT=http://otel-collector:4318
-      - RUSTFS_OBS_PROFILING_ENDPOINT=http://pyroscope:4040
-    volumes:
-      - rustfs-data:/data/rustfs
-    networks:
-      - otel-network
-    restart: unless-stopped
-    healthcheck:
-      test:
-        [
-          "CMD",
-          "sh",
-          "-c",
-          "curl -f http://127.0.0.1:9000/health && curl -f http://127.0.0.1:9001/rustfs/console/health",
-        ]
-      interval: 30s
-      timeout: 10s
-      retries: 3
-      start_period: 40s
-    depends_on:
-      otel-collector:
-        condition: service_started
-
-  rustfs-init:
-    image: alpine
-    container_name: rustfs-init
-    volumes:
-      - rustfs-data:/data
-    networks:
-      - otel-network
-    command: >
-      sh -c "
-        chown -R 10001:10001 /data &&
-        echo 'Volume Permissions fixed' &&
-        exit 0
-      "
-    restart: no
-
-  # --- Tracing ---
-
-  tempo:
-    image: grafana/tempo:latest
-    container_name: tempo
-    command: [ "-config.file=/etc/tempo.yaml" ]
-    volumes:
-      - ./tempo.yaml:/etc/tempo.yaml:ro
-      - tempo-data:/var/tempo
-    ports:
-      - "3200:3200" # tempo
-      - "4317" # otlp grpc
-      - "4318" # otlp http
-    networks:
-      - otel-network
-    restart: unless-stopped
-    depends_on:
-      - redpanda
-    healthcheck:
-      test: [ "CMD", "wget", "--spider", "-q", "http://localhost:3200/ready" ]
-      interval: 10s
-      timeout: 5s
-      retries: 3
-      start_period: 15s
-
-  redpanda:
-    image: redpandadata/redpanda:latest # for tempo ingest
-    container_name: redpanda
-    ports:
-      - "9092:9092"
-    networks:
-      - otel-network
-    restart: unless-stopped
-    command: >
-      redpanda start --overprovisioned
-      --mode=dev-container
-      --kafka-addr=PLAINTEXT://0.0.0.0:9092
-      --advertise-kafka-addr=PLAINTEXT://redpanda:9092
-
-  jaeger:
-    image: jaegertracing/jaeger:latest
-    container_name: jaeger
-    environment:
-      - SPAN_STORAGE_TYPE=badger
-      - BADGER_EPHEMERAL=false
-      - BADGER_DIRECTORY_VALUE=/badger/data
-      - BADGER_DIRECTORY_KEY=/badger/key
-      - COLLECTOR_OTLP_ENABLED=true
-    volumes:
-      - ./jaeger.yaml:/etc/jaeger/config.yml
-      - jaeger-data:/badger
-    ports:
-      - "16686:16686" # Web UI
-      - "14269:14269" # Admin/Metrics
-      - "4317" # otlp grpc
-      - "4318" # otlp http
-    command: [ "--config", "/etc/jaeger/config.yml" ]
-    networks:
-      - otel-network
-    restart: unless-stopped
-    healthcheck:
-      test: [ "CMD", "wget", "--spider", "-q", "http://localhost:14269" ]
-      interval: 10s
-      timeout: 5s
-      retries: 3
-      start_period: 15s
-
-  # --- Metrics ---
-
-  prometheus:
-    image: prom/prometheus:latest
-    container_name: prometheus
-    volumes:
-      - ./prometheus.yml:/etc/prometheus/prometheus.yml:ro
-      - prometheus-data:/prometheus
-    ports:
-      - "9090:9090"
-    command:
-      - "--config.file=/etc/prometheus/prometheus.yml"
-      - "--web.enable-otlp-receiver" # Enable OTLP
-      - "--web.enable-remote-write-receiver" # Enable remote write
-      - "--enable-feature=promql-experimental-functions" # Enable info()
-      - "--storage.tsdb.retention.time=30d"
-    restart: unless-stopped
-    networks:
-      - otel-network
-    healthcheck:
-      test: [ "CMD", "wget", "--spider", "-q", "http://localhost:9090/-/healthy" ]
-      interval: 10s
-      timeout: 5s
-      retries: 3
-
-  # --- Logging ---
-
-  loki:
-    image: grafana/loki:latest
-    container_name: loki
-    volumes:
-      - ./loki.yaml:/etc/loki/loki.yaml:ro
-      - loki-data:/loki
-    ports:
-      - "3100:3100"
-    command: -config.file=/etc/loki/loki.yaml
-    networks:
-      - otel-network
-    restart: unless-stopped
-    healthcheck:
-      test: [ "CMD", "wget", "--spider", "-q", "http://localhost:3100/ready" ]
-      interval: 15s
-      timeout: 10s
-      retries: 5
-      start_period: 60s
-
-  # --- Collection ---
-
-  otel-collector:
-    image: otel/opentelemetry-collector-contrib:latest
-    volumes:
-      - ./otel-collector-config.yaml:/etc/otelcol-contrib/config.yaml:ro
-    ports:
-      - "1888:1888" # pprof
-      - "8888:8888" # Prometheus metrics for Collector
-      - "8889:8889" # Prometheus metrics for application indicators
-      - "13133:13133" # health check
-      - "4317:4317" # OTLP gRPC
-      - "4318:4318" # OTLP HTTP
-      - "55679:55679" # zpages
-    networks:
-      - otel-network
-    restart: unless-stopped
-    depends_on:
-      - tempo
-      - jaeger
-      - prometheus
-      - loki
-    healthcheck:
-      test: [ "CMD", "wget", "--spider", "-q", "http://localhost:13133" ]
-      interval: 10s
-      timeout: 5s
-      retries: 3
-
-  # --- Profiles ---
-
-  pyroscope:
-    image: grafana/pyroscope:latest
-    container_name: pyroscope
-    ports:
-      - "4040:4040"
-    command:
-      - -self-profiling.disable-push=true
-    networks:
-      - otel-network
-    restart: unless-stopped
-
-  # --- Visualization ---
-
-  grafana:
-    image: grafana/grafana:latest
-    container_name: grafana
-    ports:
-      - "3000:3000"
-    environment:
-      - GF_SECURITY_ADMIN_PASSWORD=admin
-      - GF_SECURITY_ADMIN_USER=admin
-    volumes:
-      - ./grafana/provisioning:/etc/grafana/provisioning:ro
-      - ./grafana/dashboards:/etc/grafana/dashboards:ro
-      - grafana-data:/var/lib/grafana
-    networks:
-      - otel-network
-    restart: unless-stopped
-    depends_on:
-      - prometheus
-      - tempo
-      - loki
-    healthcheck:
-      test:
-        [ "CMD", "wget", "--spider", "-q", "http://localhost:3000/api/health" ]
-      interval: 10s
-      timeout: 5s
-      retries: 3
-
-volumes:
-  rustfs-data:
-  tempo-data:
-  jaeger-data:
-  prometheus-data:
-  loki-data:
-  grafana-data:
-
-networks:
-  otel-network:
-    driver: bridge
-    name: "network_otel"
-    ipam:
-      config:
-        - subnet: 172.28.0.0/16
-    driver_opts:
-      com.docker.network.enable_ipv6: "true"

.docker/observability/docker-compose-tempo-ha-override.yml

@@ -1,62 +0,0 @@
-# Copyright 2024 RustFS Team
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-# Docker Compose override file for High Availability Tempo setup
-#
-# Usage:
-# docker-compose -f docker-compose-example-for-rustfs.yml \
-#                -f docker-compose-tempo-ha-override.yml up
-
-services:
-  # Override Tempo to use high-availability configuration
-  tempo:
-    volumes:
-      - ./tempo-ha.yaml:/etc/tempo.yaml:ro
-      - tempo-data:/var/tempo
-    ports:
-      - "3200:3200"   # Tempo HTTP
-      - "4317:4317"   # OTLP gRPC
-      - "4318:4318"   # OTLP HTTP
-      - "7946:7946"   # Memberlist
-      - "14250:14250" # Jaeger gRPC
-      - "14268:14268" # Jaeger Thrift HTTP
-      - "9411:9411"   # Zipkin
-    environment:
-      - TEMPO_MEMBERLIST_BIND_PORT=7946
-    healthcheck:
-      test: [ "CMD", "wget", "--spider", "-q", "http://localhost:3200/ready" ]
-      interval: 10s
-      timeout: 5s
-      retries: 5
-      start_period: 30s
-    depends_on:
-      - redpanda
-
-volumes:
-  tempo-data:
-    driver: local
-    driver_opts:
-      type: tmpfs
-      device: tmpfs
-      o: "size=4g"  # Allocate 4GB tmpfs for Tempo data (adjust based on your needs)
-
-# Network configuration remains the same
-# networks:
-#   otel-network:
-#     driver: bridge
-#     name: "network_otel"
-#     ipam:
-#       config:
-#         - subnet: 172.28.0.0/16
-

.docker/observability/docker-compose.yml

@@ -14,219 +14,93 @@
 
 services:
 
-  # --- Tracing ---
+  tempo-init:
+    image: busybox:latest
+    command: ["sh", "-c", "chown -R 10001:10001 /var/tempo"]
+    volumes:
+      - ./tempo-data:/var/tempo
+    user: root
+    networks:
+      - otel-network
+    restart: "no"
+
   tempo:
-    image: grafana/tempo:2.10.3
-    container_name: tempo
-    depends_on:
-      - redpanda
-    command: [ "-config.file=/etc/tempo.yaml" ]
+    image: grafana/tempo:latest
+    user: "10001" # The container must be started with root to execute chown in the script
+    command: [ "-config.file=/etc/tempo.yaml" ] # This is passed as a parameter to the entry point script
     volumes:
       - ./tempo.yaml:/etc/tempo.yaml:ro
-      - tempo-data:/var/tempo
+      - ./tempo-data:/var/tempo
     ports:
       - "3200:3200" # tempo
-      - "4317" # otlp grpc
-      - "4318" # otlp http
-      - "7946" # memberlist
-    networks:
-      - otel-network
-    restart: unless-stopped
-    healthcheck:
-      test: [ "CMD", "wget", "--spider", "-q", "http://localhost:3200/ready" ]
-      interval: 10s
-      timeout: 5s
-      retries: 3
-      start_period: 15s
-  redpanda:
-    image: redpandadata/redpanda:latest
-    ports:
-      - "9092:9092" # Kafka API for clients
-    command: >
-      redpanda start --overprovisioned
-      --mode=dev-container
-      --kafka-addr=PLAINTEXT://0.0.0.0:9092
-      --advertise-kafka-addr=PLAINTEXT://redpanda:9092
-
-  redpanda-console:
-    image: docker.redpanda.com/redpandadata/console:latest
-    environment:
-      - CONFIG_FILEPATH=/etc/redpanda/redpanda-console-config.yaml
-    volumes:
-      - ./redpanda-console.yaml:/etc/redpanda/redpanda-console-config.yaml
-    ports:
-      - "8080:8080"
-    depends_on:
-      - redpanda
-
-  vulture:
-    image: grafana/tempo-vulture:latest
-    restart: always
-    command:
-      [
-        "-prometheus-listen-address=:8080",
-        "-tempo-query-url=http://tempo:3200",
-        "-tempo-push-url=http://tempo:4317",
-      ]
-    depends_on:
-      - tempo
-
-  jaeger:
-    image: jaegertracing/jaeger:latest
-    container_name: jaeger
-    environment:
-      - SPAN_STORAGE_TYPE=badger
-      - BADGER_EPHEMERAL=false
-      - BADGER_DIRECTORY_VALUE=/badger/data
-      - BADGER_DIRECTORY_KEY=/badger/key
-      - COLLECTOR_OTLP_ENABLED=true
-    volumes:
-      - ./jaeger.yaml:/etc/jaeger/config.yml
-      - jaeger-data:/badger
-    ports:
-      - "16686:16686" # Web UI
-      - "14269:14269" # Admin/Metrics
-      - "4317" # otlp grpc
-      - "4318" # otlp http
-    command: [ "--config", "/etc/jaeger/config.yml" ]
-    networks:
-      - otel-network
-    restart: unless-stopped
-    healthcheck:
-      test: [ "CMD", "wget", "--spider", "-q", "http://localhost:14269" ]
-      interval: 10s
-      timeout: 5s
-      retries: 3
-      start_period: 15s
-
-  # --- Metrics ---
-
-  prometheus:
-    image: prom/prometheus:latest
-    container_name: prometheus
-    volumes:
-      - ./prometheus.yml:/etc/prometheus/prometheus.yml:ro
-      - prometheus-data:/prometheus
-    ports:
-      - "9090:9090"
-    command:
-      - "--config.file=/etc/prometheus/prometheus.yml"
-      - "--web.enable-otlp-receiver" # Enable OTLP
-      - "--web.enable-remote-write-receiver" # Enable remote write
-      - "--enable-feature=promql-experimental-functions" # Enable info()
-      - "--storage.tsdb.retention.time=30d"
+      - "24317:4317" # otlp grpc
     restart: unless-stopped
     networks:
       - otel-network
-    healthcheck:
-      test: [ "CMD", "wget", "--spider", "-q", "http://localhost:9090/-/healthy" ]
-      interval: 10s
-      timeout: 5s
-      retries: 3
-
-  # --- Logging ---
-
-  loki:
-    image: grafana/loki:latest
-    container_name: loki
-    volumes:
-      - ./loki.yaml:/etc/loki/loki.yaml:ro
-      - loki-data:/loki
-    ports:
-      - "3100:3100"
-    command: -config.file=/etc/loki/loki.yaml
-    networks:
-      - otel-network
-    restart: unless-stopped
-    healthcheck:
-      test: [ "CMD", "wget", "--spider", "-q", "http://localhost:3100/ready" ]
-      interval: 15s
-      timeout: 10s
-      retries: 5
-      start_period: 60s
-
-  # --- Collection ---
 
   otel-collector:
-    image: otel/opentelemetry-collector-contrib:latest
+    image: otel/opentelemetry-collector-contrib:0.129.1
+    environment:
+      - TZ=Asia/Shanghai
     volumes:
-      - ./otel-collector-config.yaml:/etc/otelcol-contrib/config.yaml:ro
+      - ./otel-collector-config.yaml:/etc/otelcol-contrib/config.yaml
     ports:
-      - "1888:1888" # pprof
-      - "8888:8888" # Prometheus metrics for Collector
-      - "8889:8889" # Prometheus metrics for application indicators
-      - "13133:13133" # health check
-      - "4317:4317" # OTLP gRPC
-      - "4318:4318" # OTLP HTTP
-      - "55679:55679" # zpages
+      - "1888:1888"
+      - "8888:8888"
+      - "8889:8889"
+      - "13133:13133"
+      - "4317:4317"
+      - "4318:4318"
+      - "55679:55679"
     networks:
       - otel-network
-    restart: unless-stopped
-    depends_on:
-      - tempo
-      - jaeger
-      - prometheus
-      - loki
-    healthcheck:
-      test: [ "CMD", "wget", "--spider", "-q", "http://localhost:13133" ]
-      interval: 10s
-      timeout: 5s
-      retries: 3
-
-  # --- Profiles ---
-
-  pyroscope:
-    image: grafana/pyroscope:latest
-    container_name: pyroscope
+  jaeger:
+    image: jaegertracing/jaeger:2.8.0
+    environment:
+      - TZ=Asia/Shanghai
     ports:
-      - "4040:4040"
-    command:
-      - -self-profiling.disable-push=true
+      - "16686:16686"
+      - "14317:4317"
+      - "14318:4318"
+    networks:
+      - otel-network
+  prometheus:
+    image: prom/prometheus:v3.4.2
+    environment:
+      - TZ=Asia/Shanghai
+    volumes:
+      - ./prometheus.yml:/etc/prometheus/prometheus.yml
+    ports:
+      - "9090:9090"
+    networks:
+      - otel-network
+  loki:
+    image: grafana/loki:3.5.1
+    environment:
+      - TZ=Asia/Shanghai
+    volumes:
+      - ./loki-config.yaml:/etc/loki/local-config.yaml
+    ports:
+      - "3100:3100"
+    command: -config.file=/etc/loki/local-config.yaml
     networks:
       - otel-network
-    restart: unless-stopped
-
-  # --- Visualization ---
-
   grafana:
-    image: grafana/grafana:latest
-    container_name: grafana
+    image: grafana/grafana:12.0.2
     ports:
-      - "3000:3000"
+      - "3000:3000"  # Web UI
+    volumes:
+      - ./grafana-datasources.yaml:/etc/grafana/provisioning/datasources/datasources.yaml
     environment:
       - GF_SECURITY_ADMIN_PASSWORD=admin
-      - GF_SECURITY_ADMIN_USER=admin
-    volumes:
-      - ./grafana/provisioning:/etc/grafana/provisioning:ro
-      - ./grafana/dashboards:/etc/grafana/dashboards:ro
-      - grafana-data:/var/lib/grafana
+      - TZ=Asia/Shanghai
     networks:
       - otel-network
-    restart: unless-stopped
-    depends_on:
-      - prometheus
-      - tempo
-      - loki
-    healthcheck:
-      test:
-        [ "CMD", "wget", "--spider", "-q", "http://localhost:3000/api/health" ]
-      interval: 10s
-      timeout: 5s
-      retries: 3
 
-volumes:
-  tempo-data:
-  jaeger-data:
-  prometheus-data:
-  loki-data:
-  grafana-data:
 
 networks:
   otel-network:
     driver: bridge
-    name: "network_otel"
-    ipam:
-      config:
-        - subnet: 172.28.0.0/16
+    name: "network_otel_config"
     driver_opts:
       com.docker.network.enable_ipv6: "true"

.docker/observability/grafana-datasources.yaml

@@ -0,0 +1,32 @@
+apiVersion: 1
+
+datasources:
+  - name: Prometheus
+    type: prometheus
+    uid: prometheus
+    access: proxy
+    orgId: 1
+    url: http://prometheus:9090
+    basicAuth: false
+    isDefault: false
+    version: 1
+    editable: false
+    jsonData:
+      httpMethod: GET
+  - name: Tempo
+    type: tempo
+    access: proxy
+    orgId: 1
+    url: http://tempo:3200
+    basicAuth: false
+    isDefault: true
+    version: 1
+    editable: false
+    apiVersion: 1
+    uid: tempo
+    jsonData:
+      httpMethod: GET
+      serviceMap:
+        datasourceUid: prometheus
+      streamingEnabled:
+        search: true

.docker/observability/grafana/provisioning/dashboards/dashboard.yml

@@ -1,25 +0,0 @@
-# Copyright 2024 RustFS Team
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-apiVersion: 1
-
-providers:
-  - name: "default"
-    orgId: 1
-    folder: ""
-    type: file
-    disableDeletion: false
-    updateIntervalSeconds: 10
-    options:
-      path: /etc/grafana/dashboards

.docker/observability/grafana/provisioning/datasources.yaml

@@ -1,97 +0,0 @@
-# Copyright 2024 RustFS Team
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-apiVersion: 1
-
-datasources:
-  - name: Prometheus
-    type: prometheus
-    uid: prometheus
-    access: proxy
-    orgId: 1
-    url: http://prometheus:9090
-    isDefault: true
-    version: 1
-    editable: false
-    jsonData:
-      httpMethod: GET
-      exemplarTraceIdDestinations:
-        - name: trace_id
-          datasourceUid: tempo
-
-  - name: Tempo
-    type: tempo
-    uid: tempo
-    access: proxy
-    orgId: 1
-    url: http://tempo:3200
-    isDefault: false
-    version: 1
-    editable: false
-    jsonData:
-      httpMethod: GET
-      serviceMap:
-        datasourceUid: prometheus
-      tracesToLogs:
-        datasourceUid: loki
-        tags: [ 'job', 'instance', 'pod', 'namespace', 'service.name' ]
-        mappedTags: [ { key: 'service.name', value: 'app' } ]
-        spanStartTimeShift: '1s'
-        spanEndTimeShift: '-1s'
-        filterByTraceID: true
-        filterBySpanID: false
-      tracesToMetrics:
-        datasourceUid: prometheus
-        tags: [ { key: 'service.name' }, { key: 'job' } ]
-        queries:
-          - name: 'Service-Level Latency'
-            query: 'sum(rate(traces_spanmetrics_latency_bucket{$$__tags}[5m])) by (le)'
-          - name: 'Service-Level Calls'
-            query: 'sum(rate(traces_spanmetrics_calls_total{$$__tags}[5m]))'
-          - name: 'Service-Level Errors'
-            query: 'sum(rate(traces_spanmetrics_calls_total{status_code="ERROR", $$__tags}[5m]))'
-      nodeGraph:
-        enabled: true
-
-  - name: Loki
-    type: loki
-    uid: loki
-    orgId: 1
-    url: http://loki:3100
-    isDefault: false
-    version: 1
-    editable: false
-    jsonData:
-      derivedFields:
-        - datasourceUid: tempo
-          matcherRegex: 'trace_id=(\w+)'
-          name: 'TraceID'
-          url: '$${__value.raw}'
-
-  - name: Jaeger
-    type: jaeger
-    uid: jaeger
-    url: http://jaeger:16686
-    access: proxy
-    isDefault: false
-    editable: false
-    jsonData:
-      tracesToLogs:
-        datasourceUid: loki
-        tags: [ 'job', 'instance', 'pod', 'namespace', 'service.name' ]
-        mappedTags: [ { key: 'service.name', value: 'app' } ]
-        spanStartTimeShift: '1s'
-        spanEndTimeShift: '-1s'
-        filterByTraceID: true
-        filterBySpanID: false

.docker/observability/grafana/provisioning/datasources/datasources.yaml

@@ -1,98 +0,0 @@
-# Copyright 2024 RustFS Team
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-apiVersion: 1
-
-datasources:
-  - name: Prometheus
-    type: prometheus
-    uid: prometheus
-    url: http://prometheus:9090
-    access: proxy
-    isDefault: true
-    editable: false
-    jsonData:
-      httpMethod: GET
-      exemplarTraceIdDestinations:
-        - name: trace_id
-          datasourceUid: tempo
-
-  - name: Tempo
-    type: tempo
-    uid: tempo
-    access: proxy
-    url: http://tempo:3200
-    isDefault: false
-    editable: false
-    jsonData:
-      httpMethod: GET
-      serviceMap:
-        datasourceUid: prometheus
-      tracesToLogs:
-        datasourceUid: loki
-        tags: [ 'job', 'instance', 'pod', 'namespace', 'service.name' ]
-        mappedTags: [ { key: 'service.name', value: 'app' } ]
-        spanStartTimeShift: '-1h'
-        spanEndTimeShift: '1h'
-        filterByTraceID: true
-        filterBySpanID: false
-      tracesToMetrics:
-        datasourceUid: prometheus
-        tags: [ { key: 'service.name' }, { key: 'job' } ]
-        queries:
-          - name: 'Service-Level Latency'
-            query: 'sum(rate(traces_spanmetrics_latency_bucket{$$__tags}[5m])) by (le)'
-          - name: 'Service-Level Calls'
-            query: 'sum(rate(traces_spanmetrics_calls_total{$$__tags}[5m]))'
-          - name: 'Service-Level Errors'
-            query: 'sum(rate(traces_spanmetrics_calls_total{status_code="ERROR", $$__tags}[5m]))'
-      nodeGraph:
-        enabled: true
-
-  - name: Loki
-    type: loki
-    uid: loki
-    url: http://loki:3100
-    basicAuth: false
-    isDefault: false
-    editable: false
-    jsonData:
-      derivedFields:
-        - datasourceUid: tempo
-          matcherRegex: 'trace_id=(\w+)'
-          name: 'TraceID'
-          url: '$${__value.raw}'
-
-  - name: Jaeger
-    type: jaeger
-    uid: jaeger
-    url: http://jaeger:16686
-    access: proxy
-    isDefault: false
-    editable: false
-    jsonData:
-      tracesToLogs:
-        datasourceUid: loki
-        tags: [ 'job', 'instance', 'pod', 'namespace', 'service.name' ]
-        mappedTags: [ { key: 'service.name', value: 'app' } ]
-        spanStartTimeShift: '1s'
-        spanEndTimeShift: '-1s'
-        filterByTraceID: true
-        filterBySpanID: false
-
-  - name: Pyroscope
-    type: grafana-pyroscope-datasource
-    url: http://pyroscope:4040
-    jsonData:
-      minStep: '15s'

.docker/observability/jaeger-config.yaml

@@ -0,0 +1,112 @@
+# Copyright 2024 RustFS Team
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+service:
+  extensions: [ jaeger_storage, jaeger_query, remote_sampling, healthcheckv2 ]
+  pipelines:
+    traces:
+      receivers: [ otlp, jaeger, zipkin ]
+      processors: [ batch, adaptive_sampling ]
+      exporters: [ jaeger_storage_exporter ]
+  telemetry:
+    resource:
+      service.name: jaeger
+    metrics:
+      level: detailed
+      readers:
+        - pull:
+            exporter:
+              prometheus:
+                host: 0.0.0.0
+                port: 8888
+    logs:
+      level: debug
+    # TODO Initialize telemetry tracer once OTEL released new feature.
+    # https://github.com/open-telemetry/opentelemetry-collector/issues/10663
+
+extensions:
+  healthcheckv2:
+    use_v2: true
+    http:
+
+  # pprof:
+  #   endpoint: 0.0.0.0:1777
+  # zpages:
+  #   endpoint: 0.0.0.0:55679
+
+  jaeger_query:
+    storage:
+      traces: some_store
+      traces_archive: another_store
+    ui:
+      config_file: ./cmd/jaeger/config-ui.json
+      log_access: true
+    # The maximum duration that is considered for clock skew adjustments.
+    # Defaults to 0 seconds, which means it's disabled.
+    max_clock_skew_adjust: 0s
+    grpc:
+      endpoint: 0.0.0.0:16685
+    http:
+      endpoint: 0.0.0.0:16686
+
+  jaeger_storage:
+    backends:
+      some_store:
+        memory:
+          max_traces: 1000000
+      another_store:
+        memory:
+          max_traces: 1000000
+    metric_backends:
+      some_metrics_storage:
+        prometheus:
+          endpoint: http://prometheus:9090
+          normalize_calls: true
+          normalize_duration: true
+
+  remote_sampling:
+    # You can either use file or adaptive sampling strategy in remote_sampling
+    # file:
+    #   path: ./cmd/jaeger/sampling-strategies.json
+    adaptive:
+      sampling_store: some_store
+      initial_sampling_probability: 0.1
+    http:
+    grpc:
+
+receivers:
+  otlp:
+    protocols:
+      grpc:
+      http:
+
+  jaeger:
+    protocols:
+      grpc:
+      thrift_binary:
+      thrift_compact:
+      thrift_http:
+
+  zipkin:
+
+processors:
+  batch:
+  # Adaptive Sampling Processor is required to support adaptive sampling.
+  # It expects remote_sampling extension with `adaptive:` config to be enabled.
+  adaptive_sampling:
+
+exporters:
+  jaeger_storage_exporter:
+    trace_storage: some_store
+

.docker/observability/jaeger.yaml

@@ -1,74 +0,0 @@
-# Copyright 2024 RustFS Team
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-service:
-  extensions: [jaeger_storage, jaeger_query]
-  pipelines:
-    traces:
-      receivers: [otlp]
-      processors: [batch]
-      exporters: [jaeger_storage_exporter, spanmetrics]
-    metrics/spanmetrics:
-      receivers: [spanmetrics]
-      exporters: [prometheus]
-  telemetry:
-    resource:
-      service.name: jaeger
-    metrics:
-      level: detailed
-      readers:
-        - pull:
-            exporter:
-              prometheus:
-                host: 0.0.0.0
-                port: 8888
-    logs:
-      level: DEBUG
-
-extensions:
-  jaeger_query:
-    storage:
-      traces: some_storage
-      metrics: some_metrics_storage
-  jaeger_storage:
-    backends:
-      some_storage:
-        memory:
-          max_traces: 100000
-    metric_backends:
-      some_metrics_storage:
-        prometheus:
-          endpoint: http://prometheus:9090
-          normalize_calls: true
-          normalize_duration: true
-
-connectors:
-  spanmetrics:
-
-receivers:
-  otlp:
-    protocols:
-      grpc:
-        endpoint: "0.0.0.0:4317"
-      http:
-        endpoint: "0.0.0.0:4318"
-
-processors:
-  batch:
-
-exporters:
-  jaeger_storage_exporter:
-    trace_storage: some_storage
-  prometheus:
-    endpoint: "0.0.0.0:8889"

.docker/observability/loki-config.yaml

@@ -11,21 +11,22 @@
 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 # See the License for the specific language governing permissions and
 # limitations under the License.
+
 auth_enabled: false
 
 server:
   http_listen_port: 3100
-  grpc_listen_port: 9095
-  log_level: info
+  grpc_listen_port: 9096
+  log_level: debug
   grpc_server_max_concurrent_streams: 1000
 
 common:
   instance_addr: 127.0.0.1
-  path_prefix: /loki
+  path_prefix: /tmp/loki
   storage:
     filesystem:
-      chunks_directory: /loki/chunks
-      rules_directory: /loki/rules
+      chunks_directory: /tmp/loki/chunks
+      rules_directory: /tmp/loki/rules
   replication_factor: 1
   ring:
     kvstore:
@@ -38,6 +39,9 @@ query_range:
         enabled: true
         max_size_mb: 100
 
+limits_config:
+  metric_aggregation_enabled: true
+
 schema_config:
   configs:
     - from: 2020-10-24
@@ -48,16 +52,26 @@ schema_config:
         prefix: index_
         period: 24h
 
-limits_config:
-  reject_old_samples: true
-  reject_old_samples_max_age: 168h
-  allow_structured_metadata: true
-  max_line_size: 256KB
-
 pattern_ingester:
   enabled: true
   metric_aggregation:
     loki_address: localhost:3100
 
+ruler:
+  alertmanager_url: http://localhost:9093
+
 frontend:
   encoding: protobuf
+
+# By default, Loki will send anonymous, but uniquely-identifiable usage and configuration
+# analytics to Grafana Labs. These statistics are sent to https://stats.grafana.org/
+#
+# Statistics help us better understand how Loki is used, and they show us performance
+# levels for most users. This helps us prioritize features and documentation.
+# For more information on what's sent, look at
+# https://github.com/grafana/loki/blob/main/pkg/analytics/stats.go
+# Refer to the buildReport method to see what goes into a report.
+#
+# If you would like to disable reporting, uncomment the following lines:
+#analytics:
+#  reporting_enabled: false

.docker/observability/otel-collector-config.yaml

@@ -15,102 +15,67 @@
 receivers:
   otlp:
     protocols:
-      grpc:
+      grpc: # OTLP gRPC 接收器
         endpoint: 0.0.0.0:4317
-      http:
+      http: # OTLP HTTP 接收器
         endpoint: 0.0.0.0:4318
 
 processors:
-  batch:
-    timeout: 1s
-    send_batch_size: 1024
+  batch: # 批处理处理器，提升吞吐量
+    timeout: 5s
+    send_batch_size: 1000
   memory_limiter:
     check_interval: 1s
-    limit_mib: 1024
-    spike_limit_mib: 256
-  transform/logs:
-    log_statements:
-      - context: log
-        statements:
-          - set(attributes["message"], body.string)
-          - set(attributes["log.body"], body.string)
+    limit_mib: 512
 
 exporters:
-  otlp/tempo:
-    endpoint: "tempo:4317"
+  otlp/traces: # OTLP 导出器，用于跟踪数据
+    endpoint: "jaeger:4317"  # Jaeger 的 OTLP gRPC 端点
+    tls:
+      insecure: true  # 开发环境禁用 TLS，生产环境需配置证书
+  otlp/tempo: # OTLP 导出器，用于跟踪数据
+    endpoint: "tempo:4317"  # tempo 的 OTLP gRPC 端点
+    tls:
+      insecure: true  # 开发环境禁用 TLS，生产环境需配置证书
+  prometheus: # Prometheus 导出器，用于指标数据
+    endpoint: "0.0.0.0:8889"  # Prometheus 刮取端点
+    namespace: "rustfs"  # 指标前缀
+    send_timestamps: true  # 发送时间戳
+    # enable_open_metrics: true
+  otlphttp/loki: # Loki 导出器，用于日志数据
+    # endpoint: "http://loki:3100/otlp/v1/logs"
+    endpoint: "http://loki:3100/otlp/v1/logs"
     tls:
       insecure: true
-    compression: gzip
-    retry_on_failure:
-      enabled: true
-      initial_interval: 1s
-      max_interval: 30s
-      max_elapsed_time: 300s
-    sending_queue:
-      enabled: true
-      num_consumers: 10
-      queue_size: 5000
-
-  otlp/jaeger:
-    endpoint: "jaeger:4317"
-    tls:
-      insecure: true
-    compression: gzip
-    retry_on_failure:
-      enabled: true
-      initial_interval: 1s
-      max_interval: 30s
-      max_elapsed_time: 300s
-    sending_queue:
-      enabled: true
-      num_consumers: 10
-      queue_size: 5000
-
-  prometheus:
-    endpoint: "0.0.0.0:8889"
-    send_timestamps: true
-    metric_expiration: 5m
-    resource_to_telemetry_conversion:
-      enabled: true
-
-  otlphttp/loki:
-    endpoint: "http://loki:3100/otlp"
-    tls:
-      insecure: true
-    compression: gzip
-
 extensions:
   health_check:
-    endpoint: 0.0.0.0:13133
   pprof:
-    endpoint: 0.0.0.0:1888
   zpages:
-    endpoint: 0.0.0.0:55679
-
 service:
-  extensions: [ health_check, pprof, zpages ]
+  extensions: [ health_check, pprof, zpages ]  # 启用扩展
   pipelines:
     traces:
       receivers: [ otlp ]
-      processors: [ memory_limiter, batch ]
-      exporters: [ otlp/tempo, otlp/jaeger ]
+      processors: [ memory_limiter,batch ]
+      exporters: [ otlp/traces,otlp/tempo ]
     metrics:
       receivers: [ otlp ]
       processors: [ batch ]
       exporters: [ prometheus ]
     logs:
       receivers: [ otlp ]
-      processors: [ batch, transform/logs ]
+      processors: [ batch ]
       exporters: [ otlphttp/loki ]
   telemetry:
     logs:
-      level: "info"
-      encoding: "json"
+      level: "info"  # Collector 日志级别
     metrics:
-      level: "normal"
+      level: "detailed" # 可以是 basic, normal, detailed
       readers:
-        - pull:
+        - periodic:
             exporter:
-              prometheus:
-                host: '0.0.0.0'
-                port: 8888
+              otlp:
+                protocol: http/protobuf
+                endpoint: http://otel-collector:4318
+
+

.docker/observability/prometheus.yml

@@ -13,69 +13,16 @@
 # limitations under the License.
 
 global:
-  scrape_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.
-  evaluation_interval: 15s
-  external_labels:
-    cluster: 'rustfs-dev'  # Label to identify the cluster
-    replica: '1'  # Replica identifier
+  scrape_interval: 5s  # 刮取间隔
 
 scrape_configs:
   - job_name: 'otel-collector'
     static_configs:
-      - targets: [ 'otel-collector:8888' ]  # Scrape metrics from Collector
-    scrape_interval: 10s
-
-  - job_name: 'rustfs-app-metrics'
+      - targets: [ 'otel-collector:8888' ]  # 从 Collector 刮取指标
+  - job_name: 'otel-metrics'
     static_configs:
-      - targets: [ 'otel-collector:8889' ]  # Application indicators
-    scrape_interval: 15s
-    metric_relabel_configs:
-      - source_labels: [ __name__ ]
-        regex: 'go_.*'
-        action: drop  # Drop Go runtime metrics if not needed
-
+      - targets: [ 'otel-collector:8889' ]  # 应用指标
   - job_name: 'tempo'
     static_configs:
-      - targets: [ 'tempo:3200' ]  # Scrape metrics from Tempo
-
-  - job_name: 'jaeger'
-    static_configs:
-      - targets: [ 'jaeger:14269' ]  # Jaeger admin port (14269 is standard for admin/metrics)
-
-  - job_name: 'loki'
-    static_configs:
-      - targets: [ 'loki:3100' ]
-
-  - job_name: 'prometheus'
-    static_configs:
-      - targets: [ 'localhost:9090' ]
-
-  - job_name: 'vulture'
-    static_configs:
-      - targets:
-          - 'vulture:8080'
-
-otlp:
-  promote_resource_attributes:
-    - service.instance.id
-    - service.name
-    - service.namespace
-    - cloud.availability_zone
-    - cloud.region
-    - container.name
-    - deployment.environment.name
-    - k8s.cluster.name
-    - k8s.container.name
-    - k8s.cronjob.name
-    - k8s.daemonset.name
-    - k8s.deployment.name
-    - k8s.job.name
-    - k8s.namespace.name
-    - k8s.pod.name
-    - k8s.replicaset.name
-    - k8s.statefulset.name
-  translation_strategy: NoUTF8EscapingWithSuffixes
-
-storage:
-  tsdb:
-    out_of_order_time_window: 30m
+      - targets: [ 'tempo:3200' ]
+      

.docker/observability/redpanda-console.yaml

@@ -1,3 +0,0 @@
-kafka:
-  brokers:
-    - redpanda:9092

.docker/observability/tempo-data/.gitignore

@@ -0,0 +1 @@
+*

.docker/observability/tempo-ha.yaml

@@ -1,286 +0,0 @@
-# Copyright 2024 RustFS Team
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-# High Availability Tempo Configuration for docker-compose-example-for-rustfs.yml
-# Features:
-# - Distributed architecture with multiple components
-# - Kafka-based ingestion for fault tolerance
-# - Replication factor of 3 for data resilience
-# - Query frontend for load balancing
-# - Metrics generation from traces
-# - WAL for durability
-
-partition_ring_live_store: true
-stream_over_http_enabled: true
-
-server:
-  http_listen_port: 3200
-  http_server_read_timeout: 30s
-  http_server_write_timeout: 30s
-  grpc_server_max_recv_msg_size: 4194304  # 4MB
-  grpc_server_max_send_msg_size: 4194304
-  log_level: info
-  log_format: json
-
-# Memberlist configuration for distributed mode
-memberlist:
-  node_name: tempo
-  bind_port: 7946
-  join_members:
-    - tempo:7946
-  retransmit_factor: 4
-  node_timeout: 15s
-  retransmit_interval: 300ms
-  dead_node_reclaim_time: 30s
-
-# Distributor configuration - receives traces and routes to ingesters
-distributor:
-  ingester_write_path_enabled: true
-  kafka_write_path_enabled: true
-  rate_limit_bytes: 10MB
-  rate_limit_enabled: true
-  receivers:
-    otlp:
-      protocols:
-        grpc:
-          endpoint: "0.0.0.0:4317"
-          max_concurrent_streams: 0
-          max_receive_message_size: 4194304
-        http:
-          endpoint: "0.0.0.0:4318"
-          cors:
-            allowed_origins:
-              - "*"
-            max_age: 86400
-    jaeger:
-      protocols:
-        grpc:
-          endpoint: "0.0.0.0:14250"
-        thrift_http:
-          endpoint: "0.0.0.0:14268"
-    zipkin:
-      endpoint: "0.0.0.0:9411"
-  ring:
-    kvstore:
-      store: memberlist
-    heartbeat_timeout: 5s
-    replication_factor: 3
-    heartbeat_interval: 5s
-
-# Ingester configuration - stores traces and querying
-ingester:
-  lifecycler:
-    address: tempo
-    ring:
-      kvstore:
-        store: memberlist
-      replication_factor: 3
-      max_cache_freshness_per_sec: 10s
-      heartbeat_interval: 5s
-      heartbeat_timeout: 5s
-    num_tokens: 128
-    tokens_file_path: /var/tempo/tokens.json
-    claim_on_rollout: true
-  trace_idle_period: 20s
-  max_block_bytes: 10_000_000
-  max_block_duration: 10m
-  chunk_size_bytes: 1_000_000
-  chunk_encoding: snappy
-  wal:
-    checkpoint_duration: 5s
-    max_wal_blocks: 4
-  metrics:
-    enabled: true
-    level: block
-    target_info_duration: 15m
-
-# WAL configuration for data durability
-wal:
-  checkpoint_duration: 5s
-  flush_on_shutdown: true
-  path: /var/tempo/wal
-
-# Kafka ingestion configuration - for high throughput scenarios
-ingest:
-  enabled: true
-  kafka:
-    brokers: [ redpanda:9092 ]
-    topic: tempo-ingest
-    encoding: protobuf
-    consumer_group: tempo-ingest-consumer
-    session_timeout: 10s
-    rebalance_timeout: 1m
-    partition: auto
-    verbosity: 2
-
-# Query frontend configuration - distributed querying
-query_frontend:
-  compression: gzip
-  downstream_url: http://localhost:3200
-  log_queries_longer_than: 5s
-  cache_uncompressed_bytes: 100MB
-  max_outstanding_requests_per_tenant: 100
-  max_query_length: 48h
-  max_query_lookback: 30d
-  default_result_cache_ttl: 1m
-  result_cache:
-    cache:
-      enable_fifocache: true
-      default_validity: 1m
-  rf1_after: "1999-01-01T00:00:00Z"
-  mcp_server:
-    enabled: true
-
-# Querier configuration - queries traces
-querier:
-  frontend_worker:
-    frontend_address: localhost:3200
-    grpc_client_config:
-      max_recv_msg_size: 104857600
-  max_concurrent_queries: 20
-  max_metric_bytes_per_trace: 1MB
-
-# Query scheduler configuration - for distributed querying
-query_scheduler:
-  use_scheduler_ring: false
-
-# Metrics generator configuration - generates metrics from traces
-metrics_generator:
-  enabled: true
-  registry:
-    enabled: true
-    external_labels:
-      source: tempo
-      cluster: rustfs-docker-ha
-      environment: production
-  storage:
-    path: /var/tempo/generator/wal
-    remote_write:
-      - url: http://prometheus:9090/api/v1/write
-        send_exemplars: true
-        resource_to_telemetry_conversion:
-          enabled: true
-  processor:
-    batch:
-      timeout: 10s
-      send_batch_size: 1024
-    memory_limiter:
-      check_interval: 5s
-      limit_mib: 512
-      spike_limit_mib: 128
-  processors:
-    - span-metrics
-    - local-blocks
-    - service-graphs
-  generate_native_histograms: both
-
-# Backend worker configuration
-backend_worker:
-  backend_scheduler_addr: localhost:3200
-  compaction:
-    block_retention: 24h
-    compacted_block_retention: 1h
-  ring:
-    kvstore:
-      store: memberlist
-
-# Backend scheduler configuration
-backend_scheduler:
-  enabled: true
-  provider:
-    compaction:
-      compaction:
-        block_retention: 24h
-        compacted_block_retention: 1h
-        concurrency: 25
-        v2_out_path: /var/tempo/blocks/compaction
-
-# Storage configuration - local backend with proper retention
-storage:
-  trace:
-    backend: local
-    wal:
-      path: /var/tempo/wal
-      checkpoint_duration: 5s
-      flush_on_shutdown: true
-    local:
-      path: /var/tempo/blocks
-      bloom_filter_false_positive: 0.05
-      bloom_shift: 4
-      index:
-        downsample_bytes: 1000000
-        page_size_bytes: 0
-        cache_size_bytes: 0
-    pool:
-      max_workers: 400
-      queue_depth: 10000
-
-# Compactor configuration - manages block compaction
-compactor:
-  compaction:
-    block_retention: 168h  # 7 days
-    compacted_block_retention: 1h
-    concurrency: 25
-    v2_out_path: /var/tempo/blocks/compaction
-    shard_count: 32
-    max_block_bytes: 107374182400  # 100GB
-    max_compaction_objects: 6000000
-    max_time_per_tenant: 5m
-    block_size_bytes: 107374182400
-  ring:
-    kvstore:
-      store: memberlist
-    heartbeat_interval: 5s
-    heartbeat_timeout: 5s
-
-# Limits configuration - rate limiting and quotas
-limits:
-  max_traces_per_user: 10000
-  max_bytes_per_trace: 10485760  # 10MB
-  max_search_bytes_per_trace: 0
-  forgiving_oversize_traces: true
-  rate_limit_bytes: 10MB
-  rate_limit_enabled: true
-  ingestion_burst_size_bytes: 20MB
-  ingestion_rate_limit_bytes: 10MB
-  max_bytes_per_second: 10485760
-  metrics_generator_max_active_series: 10000
-  metrics_generator_max_churned_series: 10000
-  metrics_generator_forta_out_of_order_ttl: 5m
-
-# Override configuration
-overrides:
-  defaults:
-    metrics_generator:
-      processors:
-        - span-metrics
-        - local-blocks
-        - service-graphs
-      generate_native_histograms: both
-      max_active_series: 10000
-      max_churned_series: 10000
-
-# Usage reporting configuration
-usage_report:
-  reporting_enabled: false
-
-# Tracing configuration for debugging
-tracing:
-  enabled: true
-  jaeger:
-    sampler:
-      name: probabilistic
-      param: 0.1
-    reporter_log_spans: false
-

.docker/observability/tempo.yaml

@@ -1,59 +1,31 @@
-# Copyright 2024 RustFS Team
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-partition_ring_live_store: true
 stream_over_http_enabled: true
-
 server:
   http_listen_port: 3200
   log_level: info
 
-memberlist:
-  node_name: tempo
-  bind_port: 7946
-  join_members:
-    - tempo:7946
+query_frontend:
+  search:
+    duration_slo: 5s
+    throughput_bytes_slo: 1.073741824e+09
+    metadata_slo:
+      duration_slo: 5s
+      throughput_bytes_slo: 1.073741824e+09
+  trace_by_id:
+    duration_slo: 5s
 
 distributor:
   receivers:
     otlp:
       protocols:
         grpc:
-          endpoint: "0.0.0.0:4317"
-        http:
-          endpoint: "0.0.0.0:4318"
-  #log_received_spans:
-  #    enabled: true
-  #  log_discarded_spans:
-  #    enabled: true
+          endpoint: "tempo:4317"
 
-backend_scheduler:
-  provider:
-    compaction:
-      compaction:
-        block_retention: 1h
+ingester:
+  max_block_duration: 5m # cut the headblock when this much time passes. this is being set for demo purposes and should probably be left alone normally
 
-backend_worker:
-  backend_scheduler_addr: localhost:3200
+compactor:
   compaction:
-    block_retention: 1h
-  ring:
-    kvstore:
-      store: memberlist
-
-querier:
-  query_live_store: true
+    block_retention: 1h # overall Tempo trace retention. set for demo purposes
 
 metrics_generator:
   registry:
@@ -65,35 +37,19 @@ metrics_generator:
     remote_write:
       - url: http://prometheus:9090/api/v1/write
         send_exemplars: true
-
-query_frontend:
-  rf1_after: "1999-01-01T00:00:00Z"
-  mcp_server:
-    enabled: true
+  traces_storage:
+    path: /var/tempo/generator/traces
 
 storage:
   trace:
-    backend: local
+    backend: local # backend configuration to use
     wal:
       path: /var/tempo/wal # where to store the wal locally
     local:
-      path: /var/tempo/blocks # where to store the traces locally
+      path: /var/tempo/blocks
 
 overrides:
   defaults:
     metrics_generator:
-      processors: [ "span-metrics", "service-graphs", "local-blocks" ]
-      generate_native_histograms: both
-
-ingest:
-  enabled: true
-  kafka:
-    address: redpanda:9092
-    topic: tempo-ingest
-
-block_builder:
-  consume_cycle_duration: 30s
-
-usage_report:
-  reporting_enabled: false
-
+      processors: [ service-graphs, span-metrics, local-blocks ] # enables metrics generator
+      generate_native_histograms: both

.docker/openobserve-otel/README.md

@@ -5,57 +5,71 @@
 
 English | [中文](README_ZH.md)
 
-This directory contains the configuration for an **alternative** observability stack using OpenObserve.
+This directory contains the configuration files for setting up an observability stack with OpenObserve and OpenTelemetry
+Collector.
 
-## ⚠️ Note
+### Overview
 
-For the **recommended** observability stack (Prometheus, Grafana, Tempo, Loki), please see `../observability/`.
+This setup provides a complete observability solution for your applications:
 
-## 🌟 Overview
+- **OpenObserve**: A modern, open-source observability platform for logs, metrics, and traces.
+- **OpenTelemetry Collector**: Collects and processes telemetry data before sending it to OpenObserve.
 
-OpenObserve is a lightweight, all-in-one observability platform that handles logs, metrics, and traces in a single binary. This setup is ideal for:
-- Resource-constrained environments.
-- Quick setup and testing.
-- Users who prefer a unified UI.
+### Setup Instructions
 
-## 🚀 Quick Start
+1. **Prerequisites**:
+    - Docker and Docker Compose installed
+    - Sufficient memory resources (minimum 2GB recommended)
 
-### 1. Start Services
+2. **Starting the Services**:
+   ```bash
+   cd .docker/openobserve-otel
+   docker compose -f docker-compose.yml  up -d
+   ```
+
+3. **Accessing the Dashboard**:
+    - OpenObserve UI: http://localhost:5080
+    - Default credentials:
+        - Username: root@rustfs.com
+        - Password: rustfs123
+
+### Configuration
+
+#### OpenObserve Configuration
+
+The OpenObserve service is configured with:
+
+- Root user credentials
+- Data persistence through a volume mount
+- Memory cache enabled
+- Health checks
+- Exposed ports:
+    - 5080: HTTP API and UI
+    - 5081: OTLP gRPC
+
+#### OpenTelemetry Collector Configuration
+
+The collector is configured to:
+
+- Receive telemetry data via OTLP (HTTP and gRPC)
+- Collect logs from files
+- Process data in batches
+- Export data to OpenObserve
+- Manage memory usage
+
+### Integration with Your Application
+
+To send telemetry data from your application, configure your OpenTelemetry SDK to send data to:
+
+- OTLP gRPC: `localhost:4317`
+- OTLP HTTP: `localhost:4318`
+
+For example, in a Rust application using the `rustfs-obs` library:
 
 ```bash
-cd .docker/openobserve-otel
-docker compose up -d
+export RUSTFS_OBS_ENDPOINT=http://localhost:4317
+export RUSTFS_OBS_SERVICE_NAME=yourservice
+export RUSTFS_OBS_SERVICE_VERSION=1.0.0
+export RUSTFS_OBS_ENVIRONMENT=development
 ```
 
-### 2. Access Dashboard
-
-- **URL**: [http://localhost:5080](http://localhost:5080)
-- **Username**: `root@rustfs.com`
-- **Password**: `rustfs123`
-
-## 🛠️ Configuration
-
-### OpenObserve
-
-- **Persistence**: Data is persisted to a Docker volume.
-- **Ports**:
-    - `5080`: HTTP API and UI
-    - `5081`: OTLP gRPC
-
-### OpenTelemetry Collector
-
-- **Receivers**: OTLP (gRPC `4317`, HTTP `4318`)
-- **Exporters**: Sends data to OpenObserve.
-
-## 🔗 Integration
-
-Configure your application to send OTLP data to the collector:
-
-- **Endpoint**: `http://localhost:4318` (HTTP) or `localhost:4317` (gRPC)
-
-Example for RustFS:
-
-```bash
-export RUSTFS_OBS_ENDPOINT=http://localhost:4318
-export RUSTFS_OBS_SERVICE_NAME=rustfs-node-1
-```

.docker/openobserve-otel/README_ZH.md

@@ -5,57 +5,71 @@
 
 [English](README.md) | 中文
 
-本目录包含使用 OpenObserve 的**替代**可观测性技术栈配置。
+## 中文
 
-## ⚠️ 注意
+本目录包含搭建 OpenObserve 和 OpenTelemetry Collector 可观测性栈的配置文件。
 
-对于**推荐**的可观测性技术栈（Prometheus, Grafana, Tempo, Loki），请参阅 `../observability/`。
+### 概述
 
-## 🌟 概览
+此设置为应用程序提供了完整的可观测性解决方案：
 
-OpenObserve 是一个轻量级、一体化的可观测性平台，在一个二进制文件中处理日志、指标和追踪。此设置非常适合：
-- 资源受限的环境。
-- 快速设置和测试。
-- 喜欢统一 UI 的用户。
+- **OpenObserve**：现代化、开源的可观测性平台，用于日志、指标和追踪。
+- **OpenTelemetry Collector**：收集和处理遥测数据，然后将其发送到 OpenObserve。
 
-## 🚀 快速开始
+### 设置说明
 
-### 1. 启动服务
+1. **前提条件**：
+    - 已安装 Docker 和 Docker Compose
+    - 足够的内存资源（建议至少 2GB）
+
+2. **启动服务**：
+   ```bash
+   cd .docker/openobserve-otel
+   docker compose -f docker-compose.yml  up -d
+   ```
+
+3. **访问仪表板**：
+    - OpenObserve UI：http://localhost:5080
+    - 默认凭据：
+        - 用户名：root@rustfs.com
+        - 密码：rustfs123
+
+### 配置
+
+#### OpenObserve 配置
+
+OpenObserve 服务配置：
+
+- 根用户凭据
+- 通过卷挂载实现数据持久化
+- 启用内存缓存
+- 健康检查
+- 暴露端口：
+    - 5080：HTTP API 和 UI
+    - 5081：OTLP gRPC
+
+#### OpenTelemetry Collector 配置
+
+收集器配置为：
+
+- 通过 OTLP（HTTP 和 gRPC）接收遥测数据
+- 从文件中收集日志
+- 批处理数据
+- 将数据导出到 OpenObserve
+- 管理内存使用
+
+### 与应用程序集成
+
+要从应用程序发送遥测数据，将 OpenTelemetry SDK 配置为发送数据到：
+
+- OTLP gRPC:`localhost:4317`
+- OTLP HTTP:`localhost:4318`
+
+例如，在使用 `rustfs-obs` 库的 Rust 应用程序中：
 
 ```bash
-cd .docker/openobserve-otel
-docker compose up -d
-```
-
-### 2. 访问仪表盘
-
-- **URL**: [http://localhost:5080](http://localhost:5080)
-- **用户名**: `root@rustfs.com`
-- **密码**: `rustfs123`
-
-## 🛠️ 配置
-
-### OpenObserve
-
-- **持久化**: 数据持久化到 Docker 卷。
-- **端口**:
-    - `5080`: HTTP API 和 UI
-    - `5081`: OTLP gRPC
-
-### OpenTelemetry Collector
-
-- **接收器**: OTLP (gRPC `4317`, HTTP `4318`)
-- **导出器**: 将数据发送到 OpenObserve。
-
-## 🔗 集成
-
-配置您的应用程序将 OTLP 数据发送到收集器：
-
-- **端点**: `http://localhost:4318` (HTTP) 或 `localhost:4317` (gRPC)
-
-RustFS 示例：
-
-```bash
-export RUSTFS_OBS_ENDPOINT=http://localhost:4318
-export RUSTFS_OBS_SERVICE_NAME=rustfs-node-1
-```
+export RUSTFS_OBS_ENDPOINT=http://localhost:4317
+export RUSTFS_OBS_SERVICE_NAME=yourservice
+export RUSTFS_OBS_SERVICE_VERSION=1.0.0
+export RUSTFS_OBS_ENVIRONMENT=development
+```

.envrc

@@ -1 +0,0 @@
-use flake

.github/AGENTS.md

@@ -1,36 +0,0 @@
-# GitHub Workflow Instructions
-
-Applies to `.github/` and repository pull-request operations.
-
-## Pull Requests
-
-- PR titles and descriptions must be in English.
-- Use `.github/pull_request_template.md` for every PR body.
-- Keep all template section headings.
-- Use `N/A` for non-applicable sections.
-- Include verification commands in the PR details.
-- For `gh pr create` and `gh pr edit`, always write markdown body to a file and pass `--body-file`.
-- Do not use multiline inline `--body`; backticks and shell expansion can corrupt content or trigger unintended commands.
-- Recommended pattern:
-  - `cat > /tmp/pr_body.md <<'EOF'`
-  - `...markdown...`
-  - `EOF`
-  - `gh pr create ... --body-file /tmp/pr_body.md`
-
-## CI Alignment
-
-When changing CI-sensitive behavior, keep local validation aligned with `.github/workflows/ci.yml`.
-
-Current `test-and-lint` gate includes:
-
-- `cargo nextest run --all --exclude e2e_test`
-- `cargo test --all --doc`
-- `cargo test -p rustfs get_object_chunk_fast_path`
-- `cargo test -p rustfs materialize_chunk_stream_before_commit`
-- `touch rustfs/build.rs`
-- `cargo build -p rustfs --bins --jobs 2`
-- `cargo test -p e2e_test archive_multipart_roundtrip_preserves_bytes`
-- `cargo test -p e2e_test presigned_get_and_reverse_proxy_preserve_multipart_bytes_with_fast_path`
-- `cargo fmt --all --check`
-- `cargo clippy --all-targets --all-features -- -D warnings`
-- `./scripts/check_layer_dependencies.sh`

.github/actions/setup/action.yml

@@ -52,19 +52,24 @@ runs:
         sudo apt-get install -y \
           musl-tools \
           build-essential \
+          lld \
+          libdbus-1-dev \
+          libwayland-dev \
+          libwebkit2gtk-4.1-dev \
+          libxdo-dev \
           pkg-config \
           libssl-dev
 
     - name: Install protoc
       uses: arduino/setup-protoc@v3
       with:
-        version: "33.1"
+        version: "31.1"
         repo-token: ${{ inputs.github-token }}
 
     - name: Install flatc
-      uses: Nugine/setup-flatc@v1.2.4
+      uses: Nugine/setup-flatc@v1
       with:
-        version: "25.9.23"
+        version: "25.2.10"
 
     - name: Install Rust toolchain
       uses: dtolnay/rust-toolchain@stable

.github/cla.yml

@@ -1,17 +0,0 @@
-enabled: true
-
-document:
-  version: v1
-  url: https://github.com/rustfs/cla/blob/main/cla/v1.md
-
-signing:
-  mode: comment
-  comment_pattern: I have read and agree to the CLA.
-
-registry:
-  type: json-repo
-  repository: rustfs/cla
-  path_prefix: signatures
-
-status:
-  check_name: CLA Check

.github/copilot-instructions.md

@@ -1 +0,0 @@
-../AGENTS.md

.github/dependabot.yml

@@ -22,31 +22,8 @@ updates:
   - package-ecosystem: "cargo" # See documentation for possible values
     directory: "/" # Location of package manifests
     schedule:
-      interval: "weekly"
-      day: "monday"
-      timezone: "Asia/Shanghai"
-      time: "08:00"
-    assignees:
-      - "heihutu"
-    reviewers:
-      - "houseme"
-      - "overtrue"
-      - "majinghe"
-    ignore:
-      - dependency-name: "object_store"
-        versions: [ "0.13.x" ]
-      - dependency-name: "libunftp"
-        versions: [ "0.23.x" ]
-      - dependency-name: "ratelimit"
-        versions: [ "1.x" ]
+      interval: "monthly"
     groups:
-      s3s:
-        update-types:
-          - "minor"
-          - "patch"
-        patterns:
-          - "s3s"
-          - "s3s-*"
       dependencies:
         patterns:
-          - "*"
+          - "*"

.github/pull_request_template.md

@@ -34,4 +34,4 @@ Pull Request Template for RustFS
 
 ---
 
-Thank you for your contribution! Please ensure your PR follows the community standards ([CODE_OF_CONDUCT.md](CODE_OF_CONDUCT.md)). If this is your first contribution, review the [CLA document](https://github.com/rustfs/cla/blob/main/cla/v1.md) and sign it by commenting `I have read and agree to the CLA.` on the PR.
+Thank you for your contribution! Please ensure your PR follows the community standards ([CODE_OF_CONDUCT.md](CODE_OF_CONDUCT.md)) and sign the CLA if this is your first contribution.

.github/s3tests/README.md

@@ -1,103 +0,0 @@
-# S3 Compatibility Tests Configuration
-
-This directory contains the configuration for running [Ceph S3 compatibility tests](https://github.com/ceph/s3-tests) against RustFS.
-
-## Configuration File
-
-The `s3tests.conf` file is based on the official `s3tests.conf.SAMPLE` from the ceph/s3-tests repository. It uses environment variable substitution via `envsubst` to configure the endpoint and credentials.
-
-### Key Configuration Points
-
-- **Host**: Set via `${S3_HOST}` environment variable (e.g., `rustfs-single` for single-node, `lb` for multi-node)
-- **Port**: 9000 (standard RustFS port)
-- **Credentials**: Uses `${S3_ACCESS_KEY}` and `${S3_SECRET_KEY}` from workflow environment
-- **TLS**: Disabled (`is_secure = False`)
-
-## Test Execution Strategy
-
-### Network Connectivity Fix
-
-Tests run inside a Docker container on the `rustfs-net` network, which allows them to resolve and connect to the RustFS container hostnames. This fixes the "Temporary failure in name resolution" error that occurred when tests ran on the GitHub runner host.
-
-### Performance Optimizations
-
-1. **Parallel Execution**: Uses `pytest-xdist` with `-n 4` to run tests in parallel across 4 workers
-2. **Load Distribution**: Uses `--dist=loadgroup` to distribute test groups across workers
-3. **Fail-Fast**: Uses `--maxfail=50` to stop after 50 failures, saving time on catastrophic failures
-
-### Feature Filtering
-
-Tests are filtered using pytest markers (`-m`) to skip features not yet supported by RustFS:
-
-- `lifecycle` - Bucket lifecycle policies
-- `versioning` - Object versioning
-- `s3website` - Static website hosting
-- `bucket_logging` - Bucket logging
-- `encryption` / `sse_s3` - Server-side encryption
-- `cloud_transition` / `cloud_restore` - Cloud storage transitions
-- `lifecycle_expiration` / `lifecycle_transition` - Lifecycle operations
-
-This filtering:
-1. Reduces test execution time significantly (from 1+ hour to ~10-15 minutes)
-2. Focuses on features RustFS currently supports
-3. Avoids hundreds of expected failures
-
-## Running Tests Locally
-
-### Single-Node Test
-
-```bash
-# Set credentials
-export S3_ACCESS_KEY=rustfsadmin
-export S3_SECRET_KEY=rustfsadmin
-
-# Start RustFS container
-docker run -d --name rustfs-single \
-  --network rustfs-net \
-  -e RUSTFS_ADDRESS=0.0.0.0:9000 \
-  -e RUSTFS_ACCESS_KEY=$S3_ACCESS_KEY \
-  -e RUSTFS_SECRET_KEY=$S3_SECRET_KEY \
-  -e RUSTFS_VOLUMES="/data/rustfs0 /data/rustfs1 /data/rustfs2 /data/rustfs3" \
-  rustfs-ci
-
-# Generate config
-export S3_HOST=rustfs-single
-envsubst < .github/s3tests/s3tests.conf > /tmp/s3tests.conf
-
-# Run tests
-docker run --rm \
-  --network rustfs-net \
-  -v /tmp/s3tests.conf:/etc/s3tests.conf:ro \
-  python:3.12-slim \
-  bash -c '
-    apt-get update -qq && apt-get install -y -qq git
-    git clone --depth 1 https://github.com/ceph/s3-tests.git /s3-tests
-    cd /s3-tests
-    pip install -q -r requirements.txt pytest-xdist
-    S3TEST_CONF=/etc/s3tests.conf pytest -v -n 4 \
-      s3tests/functional/test_s3.py \
-      -m "not lifecycle and not versioning and not s3website and not bucket_logging and not encryption and not sse_s3"
-  '
-```
-
-## Test Results Interpretation
-
-- **PASSED**: Test succeeded, feature works correctly
-- **FAILED**: Test failed, indicates a potential bug or incompatibility
-- **ERROR**: Test setup failed (e.g., network issues, missing dependencies)
-- **SKIPPED**: Test skipped due to marker filtering
-
-## Adding New Feature Support
-
-When adding support for a new S3 feature to RustFS:
-
-1. Remove the corresponding marker from the filter in `.github/workflows/e2e-s3tests.yml`
-2. Run the tests to verify compatibility
-3. Fix any failing tests
-4. Update this README to reflect the newly supported feature
-
-## References
-
-- [Ceph S3 Tests Repository](https://github.com/ceph/s3-tests)
-- [S3 API Compatibility](https://docs.aws.amazon.com/AmazonS3/latest/API/)
-- [pytest-xdist Documentation](https://pytest-xdist.readthedocs.io/)

.github/s3tests/s3tests.conf

@@ -1,193 +0,0 @@
-# RustFS s3-tests configuration
-# Based on: https://github.com/ceph/s3-tests/blob/master/s3tests.conf.SAMPLE
-#
-# Usage:
-#   Single-node: S3_HOST=rustfs-single envsubst < s3tests.conf > /tmp/s3tests.conf
-#   Multi-node:  S3_HOST=lb envsubst < s3tests.conf > /tmp/s3tests.conf
-
-[DEFAULT]
-## this section is just used for host, port and bucket_prefix
-
-# host set for RustFS - will be substituted via envsubst
-host = ${S3_HOST}
-
-# port for RustFS
-port = 9000
-
-## say "False" to disable TLS
-is_secure = False
-
-## say "False" to disable SSL Verify
-ssl_verify = False
-
-[fixtures]
-## all the buckets created will start with this prefix;
-## {random} will be filled with random characters to pad
-## the prefix to 30 characters long, and avoid collisions
-bucket prefix = rustfs-{random}-
-
-# all the iam account resources (users, roles, etc) created
-# will start with this name prefix
-iam name prefix = s3-tests-
-
-# all the iam account resources (users, roles, etc) created
-# will start with this path prefix
-iam path prefix = /s3-tests/
-
-[s3 main]
-# main display_name
-display_name = RustFS Tester
-
-# main user_id
-user_id = rustfsadmin
-
-# main email
-email = tester@rustfs.local
-
-# zonegroup api_name for bucket location
-api_name = default
-
-## main AWS access key
-access_key = ${S3_ACCESS_KEY}
-
-## main AWS secret key
-secret_key = ${S3_SECRET_KEY}
-
-## replace with key id obtained when secret is created, or delete if KMS not tested
-#kms_keyid = 01234567-89ab-cdef-0123-456789abcdef
-
-## Storage classes
-#storage_classes = "LUKEWARM, FROZEN"
-
-## Lifecycle debug interval (default: 10)
-#lc_debug_interval = 20
-## Restore debug interval (default: 100)
-#rgw_restore_debug_interval = 60
-#rgw_restore_processor_period = 60
-
-[s3 alt]
-# alt display_name
-display_name = RustFS Alt Tester
-
-## alt email
-email = alt@rustfs.local
-
-# alt user_id
-user_id = rustfsalt
-
-# alt AWS access key (must be different from s3 main for many tests)
-access_key = ${S3_ALT_ACCESS_KEY}
-
-# alt AWS secret key
-secret_key = ${S3_ALT_SECRET_KEY}
-
-#[s3 cloud]
-## to run the testcases with "cloud_transition" for transition
-## and "cloud_restore" for restore attribute.
-## Note: the waiting time may have to tweaked depending on
-## the I/O latency to the cloud endpoint.
-
-## host set for cloud endpoint
-# host = localhost
-
-## port set for cloud endpoint
-# port = 8001
-
-## say "False" to disable TLS
-# is_secure = False
-
-## cloud endpoint credentials
-# access_key = 0555b35654ad1656d804
-# secret_key = h7GhxuBLTrlhVUyxSPUKUV8r/2EI4ngqJxD7iBdBYLhwluN30JaT3Q==
-
-## storage class configured as cloud tier on local rgw server
-# cloud_storage_class = CLOUDTIER
-
-## Below are optional -
-
-## Above configured cloud storage class config options
-# retain_head_object = false
-# allow_read_through = false    # change it to enable read_through
-# read_through_restore_days = 2
-# target_storage_class = Target_SC
-# target_path = cloud-bucket
-
-## another regular storage class to test multiple transition rules,
-# storage_class = S1
-
-[s3 tenant]
-# tenant display_name
-display_name = RustFS Tenant Tester
-
-# tenant user_id
-# Note: Using same user_id as main to avoid teardown failures.
-# RustFS does not currently support multi-tenancy, so the tenant client
-# effectively operates as the main user. This ensures nuke_prefixed_buckets()
-# in s3-tests teardown can successfully clean up resources.
-user_id = rustfsadmin
-
-# tenant AWS access key
-access_key = ${S3_ACCESS_KEY}
-
-# tenant AWS secret key
-secret_key = ${S3_SECRET_KEY}
-
-# tenant email
-email = tenant@rustfs.local
-
-# tenant name
-# Note: Empty tenant name to avoid multi-tenant path issues during teardown.
-# When s3-tests calls get_tenant_client(), it uses this tenant value in requests.
-# An empty value makes the tenant client behave like the main client, preventing
-# "bucket not found" errors when teardown tries to clean up test buckets.
-tenant =
-
-#following section needs to be added for all sts-tests
-[iam]
-#used for iam operations in sts-tests
-#email
-email = s3@rustfs.local
-
-#user_id
-user_id = rustfsiam
-
-#access_key
-access_key = ${S3_ACCESS_KEY}
-
-#secret_key
-secret_key = ${S3_SECRET_KEY}
-
-#display_name
-display_name = RustFS IAM User
-
-# iam account root user for iam_account tests
-[iam root]
-access_key = ${S3_ACCESS_KEY}
-secret_key = ${S3_SECRET_KEY}
-user_id = RGW11111111111111111
-email = account1@rustfs.local
-
-# iam account root user in a different account than [iam root]
-[iam alt root]
-access_key = ${S3_ACCESS_KEY}
-secret_key = ${S3_SECRET_KEY}
-user_id = RGW22222222222222222
-email = account2@rustfs.local
-
-#following section needs to be added when you want to run Assume Role With Webidentity test
-[webidentity]
-#used for assume role with web identity test in sts-tests
-#all parameters will be obtained from ceph/qa/tasks/keycloak.py
-#token=<access_token>
-
-#aud=<obtained after introspecting token>
-
-#sub=<obtained after introspecting token>
-
-#azp=<obtained after introspecting token>
-
-#user_token=<access token for a user, with attribute Department=[Engineering, Marketing>]
-
-#thumbprint=<obtained from x509 certificate>
-
-#KC_REALM=<name of the realm>

.github/workflows/audit.yml

@@ -40,11 +40,11 @@ env:
 jobs:
   security-audit:
     name: Security Audit
-    runs-on: ubicloud-standard-2
+    runs-on: ubuntu-latest
     timeout-minutes: 15
     steps:
       - name: Checkout repository
-        uses: actions/checkout@v6
+        uses: actions/checkout@v5
 
       - name: Install cargo-audit
         uses: taiki-e/install-action@v2
@@ -57,7 +57,7 @@ jobs:
 
       - name: Upload audit results
         if: always()
-        uses: actions/upload-artifact@v6
+        uses: actions/upload-artifact@v4
         with:
           name: security-audit-results-${{ github.run_number }}
           path: audit-results.json
@@ -65,14 +65,14 @@ jobs:
 
   dependency-review:
     name: Dependency Review
-    runs-on: ubicloud-standard-2
+    runs-on: ubuntu-latest
     if: github.event_name == 'pull_request'
     permissions:
       contents: read
       pull-requests: write
     steps:
       - name: Checkout repository
-        uses: actions/checkout@v6
+        uses: actions/checkout@v5
 
       - name: Dependency Review
         uses: actions/dependency-review-action@v4

.github/workflows/build.yml

@@ -23,7 +23,6 @@
 #
 # Manual Parameters:
 # - build_docker: Build and push Docker images (default: true)
-# - platforms: Comma-separated platform IDs or 'all' (default: all)
 
 name: Build and Release
 
@@ -45,6 +44,22 @@ on:
       - "**/*.svg"
       - ".gitignore"
       - ".dockerignore"
+  pull_request:
+    branches: [ main ]
+    paths-ignore:
+      - "**.md"
+      - "**.txt"
+      - ".github/**"
+      - "docs/**"
+      - "deploy/**"
+      - "scripts/dev_*.sh"
+      - "LICENSE*"
+      - "README*"
+      - "**/*.png"
+      - "**/*.jpg"
+      - "**/*.svg"
+      - ".gitignore"
+      - ".dockerignore"
   schedule:
     - cron: "0 0 * * 0" # Weekly on Sunday at midnight UTC
   workflow_dispatch:
@@ -54,11 +69,6 @@ on:
         required: false
         default: true
         type: boolean
-      platforms:
-        description: "Comma-separated targets or 'all' (e.g. linux-x86_64-musl,macos-aarch64)"
-        required: false
-        default: "all"
-        type: string
 
 permissions:
   contents: read
@@ -73,7 +83,7 @@ jobs:
   # Build strategy check - determine build type based on trigger
   build-check:
     name: Build Strategy Check
-    runs-on: ubicloud-standard-2
+    runs-on: ubuntu-latest
     outputs:
       should_build: ${{ steps.check.outputs.should_build }}
       build_type: ${{ steps.check.outputs.build_type }}
@@ -82,7 +92,7 @@ jobs:
       is_prerelease: ${{ steps.check.outputs.is_prerelease }}
     steps:
       - name: Checkout repository
-        uses: actions/checkout@v6
+        uses: actions/checkout@v5
         with:
           fetch-depth: 0
 
@@ -144,78 +154,56 @@ jobs:
           echo "  - Is prerelease: $is_prerelease"
 
   # Build RustFS binaries
-  prepare-platform-matrix:
-    name: Prepare Platform Matrix
-    runs-on: ubicloud-standard-2
-    outputs:
-      matrix: ${{ steps.select.outputs.matrix }}
-      selected: ${{ steps.select.outputs.selected }}
-    steps:
-      - name: Select target platforms
-        id: select
-        shell: bash
-        run: |
-          set -euo pipefail
-
-          selected="${{ github.event_name == 'workflow_dispatch' && github.event.inputs.platforms || 'all' }}"
-          selected="$(echo "${selected}" | tr -d '[:space:]')"
-          if [[ -z "${selected}" ]]; then
-            selected="all"
-          fi
-
-          all='{"include":[
-            {"target_id":"linux-x86_64-musl","os":"ubicloud-standard-2","target":"x86_64-unknown-linux-musl","cross":false,"platform":"linux","rustflags":""},
-            {"target_id":"linux-aarch64-musl","os":"ubicloud-standard-2","target":"aarch64-unknown-linux-musl","cross":true,"platform":"linux","rustflags":""},
-            {"target_id":"linux-x86_64-gnu","os":"ubicloud-standard-2","target":"x86_64-unknown-linux-gnu","cross":false,"platform":"linux","rustflags":""},
-            {"target_id":"linux-aarch64-gnu","os":"ubicloud-standard-2","target":"aarch64-unknown-linux-gnu","cross":true,"platform":"linux","rustflags":""},
-            {"target_id":"macos-aarch64","os":"macos-latest","target":"aarch64-apple-darwin","cross":false,"platform":"macos","rustflags":""},
-            {"target_id":"macos-x86_64","os":"macos-latest","target":"x86_64-apple-darwin","cross":false,"platform":"macos","rustflags":""},
-            {"target_id":"windows-x86_64","os":"windows-latest","target":"x86_64-pc-windows-msvc","cross":false,"platform":"windows","rustflags":""}
-          ]}'
-
-          if [[ "${selected}" == "all" ]]; then
-            matrix="$(jq -c . <<<"${all}")"
-          else
-            unknown="$(jq -rn --arg selected "${selected}" --argjson all "${all}" '
-              ($selected | split(",") | map(select(length > 0))) as $req
-              | ($all.include | map(.target_id)) as $known
-              | [$req[] | select(( $known | index(.) ) == null)]
-            ')"
-            if [[ "$(jq 'length' <<<"${unknown}")" -gt 0 ]]; then
-              echo "Unknown platforms: $(jq -r 'join(\",\")' <<<"${unknown}")" >&2
-              echo "Allowed: $(jq -r '.include[].target_id' <<<"${all}" | paste -sd ',' -)" >&2
-              exit 1
-            fi
-
-            matrix="$(jq -c --arg selected "${selected}" '
-              ($selected | split(",") | map(select(length > 0))) as $req
-              | .include |= map(select(.target_id as $id | ($req | index($id))))
-            ' <<<"${all}")"
-          fi
-
-          echo "selected=${selected}" >> "$GITHUB_OUTPUT"
-          echo "matrix=${matrix}" >> "$GITHUB_OUTPUT"
-          echo "Selected platforms: ${selected}"
-
   build-rustfs:
     name: Build RustFS
-    needs: [ build-check, prepare-platform-matrix ]
-    if: needs.build-check.outputs.should_build == 'true' && needs.prepare-platform-matrix.result == 'success'
+    needs: [ build-check ]
+    if: needs.build-check.outputs.should_build == 'true'
     runs-on: ${{ matrix.os }}
     timeout-minutes: 60
     env:
-      FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"
-      # Always enable Tokio unstable features (required by dial9-tokio-telemetry).
-      # The RUSTFLAGS env var takes precedence over .cargo/config.toml [build] rustflags,
-      # so we must include --cfg tokio_unstable here explicitly; otherwise an empty
-      # RUSTFLAGS value would shadow the config-file flag and silently break tracing.
-      RUSTFLAGS: "--cfg tokio_unstable ${{ matrix.rustflags }}"
+      RUSTFLAGS: ${{ matrix.cross == 'false' && '-C target-cpu=native' || '' }}
     strategy:
       fail-fast: false
-      matrix: ${{ fromJson(needs.prepare-platform-matrix.outputs.matrix) }}
+      matrix:
+        include:
+          # Linux builds
+          - os: ubuntu-latest
+            target: x86_64-unknown-linux-musl
+            cross: false
+            platform: linux
+          - os: ubuntu-latest
+            target: aarch64-unknown-linux-musl
+            cross: true
+            platform: linux
+          - os: ubuntu-latest
+            target: x86_64-unknown-linux-gnu
+            cross: false
+            platform: linux
+          - os: ubuntu-latest
+            target: aarch64-unknown-linux-gnu
+            cross: true
+            platform: linux
+          # macOS builds
+          - os: macos-latest
+            target: aarch64-apple-darwin
+            cross: false
+            platform: macos
+          - os: macos-latest
+            target: x86_64-apple-darwin
+            cross: false
+            platform: macos
+          # Windows builds (temporarily disabled)
+          - os: windows-latest
+            target: x86_64-pc-windows-msvc
+            cross: false
+            platform: windows
+          #- os: windows-latest
+          #  target: aarch64-pc-windows-msvc
+          #  cross: true
+          #  platform: windows
     steps:
       - name: Checkout repository
-        uses: actions/checkout@v6
+        uses: actions/checkout@v5
         with:
           fetch-depth: 0
 
@@ -321,26 +309,19 @@ jobs:
               ;;
           esac
 
-          # Normalize version used for package filenames
-          PACKAGE_VERSION="${VERSION}"
-          if [[ "$PACKAGE_VERSION" == v* ]]; then
-            PACKAGE_VERSION="${PACKAGE_VERSION#v}"
-          fi
-
           # Generate package name based on build type
           if [[ -n "$VARIANT" ]]; then
             ARCH_WITH_VARIANT="${ARCH}-${VARIANT}"
           else
             ARCH_WITH_VARIANT="${ARCH}"
           fi
-          PACKAGE_BASENAME="rustfs-${PLATFORM}-${ARCH_WITH_VARIANT}"
 
           if [[ "$BUILD_TYPE" == "development" ]]; then
             # Development build: rustfs-${platform}-${arch}-${variant}-dev-${short_sha}.zip
             PACKAGE_NAME="rustfs-${PLATFORM}-${ARCH_WITH_VARIANT}-dev-${SHORT_SHA}"
           else
             # Release/Prerelease build: rustfs-${platform}-${arch}-${variant}-v${version}.zip
-            PACKAGE_NAME="${PACKAGE_BASENAME}-v${PACKAGE_VERSION}"
+            PACKAGE_NAME="rustfs-${PLATFORM}-${ARCH_WITH_VARIANT}-v${VERSION}"
           fi
 
           # Create zip packages for all platforms
@@ -410,7 +391,7 @@ jobs:
           if [[ "$BUILD_TYPE" == "release" ]] || [[ "$BUILD_TYPE" == "prerelease" ]]; then
             # Create latest version filename
             # Convert from rustfs-linux-x86_64-musl-v1.0.0 to rustfs-linux-x86_64-musl-latest
-            LATEST_FILE="${PACKAGE_BASENAME}-latest.zip"
+            LATEST_FILE="${PACKAGE_NAME%-v*}-latest.zip"
 
             echo "🔄 Creating latest version: ${PACKAGE_NAME}.zip -> $LATEST_FILE"
             cp "${PACKAGE_NAME}.zip" "$LATEST_FILE"
@@ -423,7 +404,7 @@ jobs:
             # Development builds (only main branch triggers development builds)
             # Create main-latest version filename
             # Convert from rustfs-linux-x86_64-dev-abc123 to rustfs-linux-x86_64-main-latest
-            MAIN_LATEST_FILE="${PACKAGE_BASENAME}-main-latest.zip"
+            MAIN_LATEST_FILE="${PACKAGE_NAME%-dev-*}-main-latest.zip"
 
             echo "🔄 Creating main-latest version: ${PACKAGE_NAME}.zip -> $MAIN_LATEST_FILE"
             cp "${PACKAGE_NAME}.zip" "$MAIN_LATEST_FILE"
@@ -461,7 +442,7 @@ jobs:
           echo "📊 Version: ${VERSION}"
 
       - name: Upload to GitHub artifacts
-        uses: actions/upload-artifact@v6
+        uses: actions/upload-artifact@v4
         with:
           name: ${{ steps.package.outputs.package_name }}
           path: "rustfs-*.zip"
@@ -473,7 +454,7 @@ jobs:
           OSS_ACCESS_KEY_ID: ${{ secrets.ALICLOUDOSS_KEY_ID }}
           OSS_ACCESS_KEY_SECRET: ${{ secrets.ALICLOUDOSS_KEY_SECRET }}
           OSS_REGION: cn-beijing
-          OSS_ENDPOINT: https://oss-accelerate.aliyuncs.com
+          OSS_ENDPOINT: https://oss-cn-beijing.aliyuncs.com
         shell: bash
         run: |
           BUILD_TYPE="${{ needs.build-check.outputs.build_type }}"
@@ -551,7 +532,7 @@ jobs:
     name: Build Summary
     needs: [ build-check, build-rustfs ]
     if: always() && needs.build-check.outputs.should_build == 'true'
-    runs-on: ubicloud-standard-2
+    runs-on: ubuntu-latest
     steps:
       - name: Build completion summary
         shell: bash
@@ -603,7 +584,7 @@ jobs:
     name: Create GitHub Release
     needs: [ build-check, build-rustfs ]
     if: startsWith(github.ref, 'refs/tags/') && needs.build-check.outputs.build_type != 'development'
-    runs-on: ubicloud-standard-2
+    runs-on: ubuntu-latest
     permissions:
       contents: write
     outputs:
@@ -611,7 +592,7 @@ jobs:
       release_url: ${{ steps.create.outputs.release_url }}
     steps:
       - name: Checkout repository
-        uses: actions/checkout@v6
+        uses: actions/checkout@v5
         with:
           fetch-depth: 0
 
@@ -689,16 +670,16 @@ jobs:
     name: Upload Release Assets
     needs: [ build-check, build-rustfs, create-release ]
     if: startsWith(github.ref, 'refs/tags/') && needs.build-check.outputs.build_type != 'development'
-    runs-on: ubicloud-standard-2
+    runs-on: ubuntu-latest
     permissions:
       contents: write
       actions: read
     steps:
       - name: Checkout repository
-        uses: actions/checkout@v6
+        uses: actions/checkout@v5
 
       - name: Download all build artifacts
-        uses: actions/download-artifact@v7
+        uses: actions/download-artifact@v5
         with:
           path: ./artifacts
           pattern: rustfs-*
@@ -770,7 +751,7 @@ jobs:
     name: Update Latest Version
     needs: [ build-check, upload-release-assets ]
     if: startsWith(github.ref, 'refs/tags/')
-    runs-on: ubicloud-standard-2
+    runs-on: ubuntu-latest
     steps:
       - name: Update latest.json
         env:
@@ -820,12 +801,12 @@ jobs:
     name: Publish Release
     needs: [ build-check, create-release, upload-release-assets ]
     if: startsWith(github.ref, 'refs/tags/') && needs.build-check.outputs.build_type != 'development'
-    runs-on: ubicloud-standard-2
+    runs-on: ubuntu-latest
     permissions:
       contents: write
     steps:
       - name: Checkout repository
-        uses: actions/checkout@v6
+        uses: actions/checkout@v5
 
       - name: Update release notes and publish
         env:

.github/workflows/ci.yml

@@ -4,7 +4,7 @@
 # you may not use this file except in compliance with the License.
 # You may obtain a copy of the License at
 #
-#      http://www.apache.org/licenses/LICENSE-2.0
+#     http://www.apache.org/licenses/LICENSE-2.0
 #
 # Unless required by applicable law or agreed to in writing, software
 # distributed under the License is distributed on an "AS IS" BASIS,
@@ -19,6 +19,7 @@ on:
     branches: [ main ]
     paths-ignore:
       - "**.md"
+      - "**.txt"
       - "docs/**"
       - "deploy/**"
       - "scripts/dev_*.sh"
@@ -38,6 +39,7 @@ on:
     branches: [ main ]
     paths-ignore:
       - "**.md"
+      - "**.txt"
       - "docs/**"
       - "deploy/**"
       - "scripts/dev_*.sh"
@@ -53,8 +55,6 @@ on:
       - ".github/workflows/docker.yml"
       - ".github/workflows/audit.yml"
       - ".github/workflows/performance.yml"
-  merge_group:
-    types: [checks_requested]
   schedule:
     - cron: "0 0 * * 0" # Weekly on Sunday at midnight UTC
   workflow_dispatch:
@@ -62,23 +62,17 @@ on:
 permissions:
   contents: read
 
-concurrency:
-  group: ${{ github.workflow }}-${{ github.ref }}
-  cancel-in-progress: true
-
 env:
   CARGO_TERM_COLOR: always
   RUST_BACKTRACE: 1
-  CARGO_BUILD_JOBS: 2
 
 jobs:
-
   skip-check:
     name: Skip Duplicate Actions
     permissions:
       actions: write
       contents: read
-    runs-on: ubicloud-standard-2
+    runs-on: ubuntu-latest
     outputs:
       should_skip: ${{ steps.skip_check.outputs.should_skip }}
     steps:
@@ -89,15 +83,15 @@ jobs:
           concurrent_skipping: "same_content_newer"
           cancel_others: true
           paths_ignore: '["*.md", "docs/**", "deploy/**"]'
+          # Never skip release events and tag pushes
           do_not_skip: '["workflow_dispatch", "schedule", "merge_group", "release", "push"]'
 
+
   typos:
     name: Typos
-    needs: skip-check
-    if: needs.skip-check.outputs.should_skip != 'true'
-    runs-on: ubicloud-standard-2
+    runs-on: ubuntu-latest
     steps:
-      - uses: actions/checkout@v6
+      - uses: actions/checkout@v5
       - uses: dtolnay/rust-toolchain@stable
       - name: Typos check with custom config file
         uses: crate-ci/typos@master
@@ -106,13 +100,13 @@ jobs:
     name: Test and Lint
     needs: skip-check
     if: needs.skip-check.outputs.should_skip != 'true'
-    runs-on: ubicloud-standard-4
+    runs-on: ubuntu-latest
     timeout-minutes: 60
-    env:
-      FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"
     steps:
+      - name: Delete huge unnecessary tools folder
+        run: rm -rf /opt/hostedtoolcache
       - name: Checkout repository
-        uses: actions/checkout@v6
+        uses: actions/checkout@v5
 
       - name: Setup Rust environment
         uses: ./.github/actions/setup
@@ -133,75 +127,35 @@ jobs:
       - name: Run clippy lints
         run: cargo clippy --all-targets --all-features -- -D warnings
 
-      - name: Check layered dependencies
-        run: ./scripts/check_layer_dependencies.sh
-
-  build-rustfs-debug-binary:
-    name: Build RustFS Debug Binary
+  e2e-tests:
+    name: End-to-End Tests
     needs: skip-check
     if: needs.skip-check.outputs.should_skip != 'true'
-    runs-on: ubicloud-standard-4
+    runs-on: ubuntu-latest
     timeout-minutes: 30
-    env:
-      FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"
     steps:
       - name: Checkout repository
-        uses: actions/checkout@v6
+        uses: actions/checkout@v5
 
       - name: Setup Rust environment
         uses: ./.github/actions/setup
         with:
           rust-version: stable
-          cache-shared-key: ci-rustfs-debug-binary-${{ hashFiles('**/Cargo.lock') }}
+          cache-shared-key: ci-e2e-${{ hashFiles('**/Cargo.lock') }}
           cache-save-if: ${{ github.ref == 'refs/heads/main' }}
           github-token: ${{ secrets.GITHUB_TOKEN }}
 
-      - name: Build debug binary
-        run: |
-          touch rustfs/build.rs
-          cargo build -p rustfs --bins --jobs 2
-
-      - name: Upload debug binary
-        uses: actions/upload-artifact@v6
-        with:
-          name: rustfs-debug-binary
-          path: target/debug/rustfs
-          if-no-files-found: error
-          retention-days: 1
-
-  e2e-tests:
-    name: End-to-End Tests
-    needs: [ skip-check, build-rustfs-debug-binary ]
-    if: needs.skip-check.outputs.should_skip != 'true'
-    runs-on: ubicloud-standard-2
-    timeout-minutes: 30
-    steps:
-      - name: Checkout repository
-        uses: actions/checkout@v6
-
-      - name: Clean up previous test run
-        run: |
-          rm -rf /tmp/rustfs
-          rm -f /tmp/rustfs.log
-
-      - name: Download debug binary
-        uses: actions/download-artifact@v7
-        with:
-          name: rustfs-debug-binary
-          path: target/debug
-
-      - name: Make binary executable
-        run: chmod +x ./target/debug/rustfs
-
-      - name: Setup Rust toolchain for s3s-e2e installation
-        uses: dtolnay/rust-toolchain@stable
-
       - name: Install s3s-e2e test tool
         uses: taiki-e/cache-cargo-install-action@v2
         with:
           tool: s3s-e2e
-          git: https://github.com/s3s-project/s3s.git
-          rev: 4a04a670cf41274d9be9ab65dc36f4aa3f92fbad
+          git: https://github.com/Nugine/s3s.git
+          rev: b7714bfaa17ddfa9b23ea01774a1e7bbdbfc2ca3
+
+      - name: Build debug binary
+        run: |
+          touch rustfs/build.rs
+          cargo build -p rustfs --bins
 
       - name: Run end-to-end tests
         run: |
@@ -210,44 +164,8 @@ jobs:
 
       - name: Upload test logs
         if: failure()
-        uses: actions/upload-artifact@v6
+        uses: actions/upload-artifact@v4
         with:
           name: e2e-test-logs-${{ github.run_number }}
           path: /tmp/rustfs.log
           retention-days: 3
-
-  s3-implemented-tests:
-    name: S3 Implemented Tests
-    needs: [ skip-check, build-rustfs-debug-binary ]
-    if: needs.skip-check.outputs.should_skip != 'true'
-    runs-on: ubicloud-standard-4
-    timeout-minutes: 60
-    steps:
-      - name: Checkout repository
-        uses: actions/checkout@v6
-
-      - name: Download debug binary
-        uses: actions/download-artifact@v7
-        with:
-          name: rustfs-debug-binary
-          path: target/debug
-
-      - name: Make binary executable
-        run: chmod +x ./target/debug/rustfs
-
-      - name: Run implemented s3-tests
-        run: |
-          DEPLOY_MODE=binary \
-          RUSTFS_BINARY=./target/debug/rustfs \
-          TEST_MODE=single \
-          MAXFAIL=1 \
-          ./scripts/s3-tests/run.sh
-
-      - name: Upload s3 test artifacts
-        if: always()
-        uses: actions/upload-artifact@v6
-        with:
-          name: s3tests-implemented-${{ github.run_number }}
-          path: artifacts/s3tests-single/**
-          if-no-files-found: ignore
-          retention-days: 3

.github/workflows/cla.yml

@@ -1,70 +0,0 @@
-# Copyright 2026 RustFS Team
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-name: CLA Check
-
-on:
-  pull_request_target:
-    types: [opened, synchronize, reopened]
-  merge_group:
-    types: [checks_requested]
-  issue_comment:
-    types: [created, edited]
-
-permissions:
-  contents: write
-  pull-requests: write
-  issues: write
-  checks: write
-
-jobs:
-  cla:
-    if: ${{ github.event_name != 'issue_comment' || github.event.issue.pull_request }}
-    runs-on: ubuntu-latest
-    steps:
-      - name: Report CLA result for merge queue
-        if: github.event_name == 'merge_group'
-        uses: actions/github-script@v8
-        with:
-          script: |
-            await github.rest.checks.create({
-              owner: context.repo.owner,
-              repo: context.repo.repo,
-              name: 'CLA Check',
-              head_sha: context.sha,
-              status: 'completed',
-              conclusion: 'success',
-              output: {
-                title: 'CLA requirements satisfied for merge queue',
-                summary: 'Queued pull requests must satisfy the required CLA check before they enter the merge queue. This reports the existing CLA result on the merge-group SHA.'
-              }
-            });
-
-      - name: Create token for rustfs/cla
-        if: github.event_name != 'merge_group'
-        id: registry-token
-        uses: actions/create-github-app-token@v3
-        with:
-          app-id: ${{ vars.CLA_BOT_APP_ID }}
-          private-key: ${{ secrets.CLA_BOT_APP_PRIVATE_KEY }}
-          owner: ${{ github.repository_owner }}
-          repositories: cla
-          permission-contents: write
-
-      - name: Run CLA Bot
-        if: github.event_name != 'merge_group'
-        uses: overtrue/cla-bot@v0.0.9
-        with:
-          github-token: ${{ github.token }}
-          registry-token: ${{ steps.registry-token.outputs.token }}

.github/workflows/docker.yml

@@ -66,14 +66,13 @@ env:
   CARGO_TERM_COLOR: always
   REGISTRY_DOCKERHUB: rustfs/rustfs
   REGISTRY_GHCR: ghcr.io/${{ github.repository }}
-  REGISTRY_QUAY: quay.io/${{ secrets.QUAY_USERNAME }}/rustfs
   DOCKER_PLATFORMS: linux/amd64,linux/arm64
 
 jobs:
   # Check if we should build Docker images
   build-check:
     name: Docker Build Check
-    runs-on: ubicloud-standard-2
+    runs-on: ubuntu-latest
     outputs:
       should_build: ${{ steps.check.outputs.should_build }}
       should_push: ${{ steps.check.outputs.should_push }}
@@ -84,7 +83,7 @@ jobs:
       create_latest: ${{ steps.check.outputs.create_latest }}
     steps:
       - name: Checkout repository
-        uses: actions/checkout@v6
+        uses: actions/checkout@v5
         with:
           fetch-depth: 0
           # For workflow_run events, checkout the specific commit that triggered the workflow
@@ -163,11 +162,11 @@ jobs:
                 if [[ "$version" == *"alpha"* ]] || [[ "$version" == *"beta"* ]] || [[ "$version" == *"rc"* ]]; then
                   build_type="prerelease"
                   is_prerelease=true
-                  # TODO: Temporary change - currently allows alpha versions to also create latest tags
-                  # After the version is stable, you need to remove the following line and restore the original logic (latest is created only for stable versions)
+                  # TODO: 临时修改 - 当前允许 alpha 版本也创建 latest 标签
+                  # 等版本稳定后，需要移除下面这行，恢复原有逻辑（只有稳定版本才创建 latest）
                   if [[ "$version" == *"alpha"* ]]; then
                     create_latest=true
-                     echo "🧪 Building Docker image for prerelease: $version (temporarily allowing creation of latest tag)"
+                    echo "🧪 Building Docker image for prerelease: $version (临时允许创建 latest 标签)"
                   else
                     echo "🧪 Building Docker image for prerelease: $version"
                   fi
@@ -216,11 +215,11 @@ jobs:
               v*alpha*|v*beta*|v*rc*|*alpha*|*beta*|*rc*)
                 build_type="prerelease"
                 is_prerelease=true
-                # TODO: Temporary change - currently allows alpha versions to also create latest tags
-                # After the version is stable, you need to remove the if block below and restore the original logic.
+                # TODO: 临时修改 - 当前允许 alpha 版本也创建 latest 标签
+                # 等版本稳定后，需要移除下面的 if 块，恢复原有逻辑
                 if [[ "$input_version" == *"alpha"* ]]; then
                   create_latest=true
-                   echo "🧪 Building with prerelease version: $input_version (temporarily allowing creation of latest tag)"
+                  echo "🧪 Building with prerelease version: $input_version (临时允许创建 latest 标签)"
                 else
                   echo "🧪 Building with prerelease version: $input_version"
                 fi
@@ -265,22 +264,11 @@ jobs:
     name: Build Docker Images
     needs: build-check
     if: needs.build-check.outputs.should_build == 'true'
-    runs-on: ubicloud-standard-2
+    runs-on: ubuntu-latest
     timeout-minutes: 60
-    strategy:
-          fail-fast: false
-          matrix:
-            include:
-              - variant: musl
-                file: Dockerfile
-                suffix: ""
-              - variant: glibc
-                file: Dockerfile.glibc
-                suffix: "-glibc"
-
     steps:
       - name: Checkout repository
-        uses: actions/checkout@v6
+        uses: actions/checkout@v5
 
       - name: Login to Docker Hub
         uses: docker/login-action@v3
@@ -288,19 +276,12 @@ jobs:
           username: ${{ env.DOCKERHUB_USERNAME }}
           password: ${{ secrets.DOCKERHUB_TOKEN }}
 
-      - name: Login to GitHub Container Registry
-        uses: docker/login-action@v3
-        with:
-          registry: ghcr.io
-          username: ${{ secrets.GHCR_USERNAME }}
-          password: ${{ secrets.GHCR_PASSWORD }}
-
-      - name: Login to Quay.io
-        uses: docker/login-action@v3
-        with:
-          registry: quay.io
-          username: ${{ secrets.QUAY_USERNAME }}
-          password: ${{ secrets.QUAY_PASSWORD }}
+      # - name: Login to GitHub Container Registry
+      #   uses: docker/login-action@v3
+      #   with:
+      #     registry: ghcr.io
+      #     username: ${{ github.actor }}
+      #     password: ${{ secrets.GITHUB_TOKEN }}
 
       - name: Set up QEMU
         uses: docker/setup-qemu-action@v3
@@ -315,7 +296,6 @@ jobs:
           VERSION="${{ needs.build-check.outputs.version }}"
           SHORT_SHA="${{ needs.build-check.outputs.short_sha }}"
           CREATE_LATEST="${{ needs.build-check.outputs.create_latest }}"
-          VARIANT_SUFFIX="${{ matrix.suffix }}"
 
           # Convert version format for Dockerfile compatibility
           case "$VERSION" in
@@ -346,15 +326,14 @@ jobs:
 
           # Generate tags based on build type
           # Only support release and prerelease builds (no development builds)
-          TAG_BASE="${VERSION}${VARIANT_SUFFIX}"
-          TAGS="${{ env.REGISTRY_DOCKERHUB }}:$TAG_BASE,${{ env.REGISTRY_GHCR }}:$TAG_BASE,${{ env.REGISTRY_QUAY }}:$TAG_BASE"
+          TAGS="${{ env.REGISTRY_DOCKERHUB }}:${VERSION}"
 
           # Add channel tags for prereleases and latest for stable
           if [[ "$CREATE_LATEST" == "true" ]]; then
-            # TODO: Temporary change - the current alpha version will also create the latest tag
-            # After the version is stabilized, the logic here remains unchanged, but the upstream CREATE_LATEST setting needs to be restored.
-            # Stable release (and temporary alpha versions)
-            TAGS="$TAGS,${{ env.REGISTRY_DOCKERHUB }}:latest${VARIANT_SUFFIX},${{ env.REGISTRY_GHCR }}:latest${VARIANT_SUFFIX},${{ env.REGISTRY_QUAY }}:latest${VARIANT_SUFFIX}"
+            # TODO: 临时修改 - 当前 alpha 版本也会创建 latest 标签
+            # 等版本稳定后，这里的逻辑保持不变，但上游的 CREATE_LATEST 设置需要恢复
+            # Stable release (以及临时的 alpha 版本)
+            TAGS="$TAGS,${{ env.REGISTRY_DOCKERHUB }}:latest"
           elif [[ "$BUILD_TYPE" == "prerelease" ]]; then
             # Prerelease channel tags (alpha, beta, rc)
             if [[ "$VERSION" == *"alpha"* ]]; then
@@ -366,7 +345,7 @@ jobs:
             fi
 
             if [[ -n "$CHANNEL" ]]; then
-              TAGS="$TAGS,${{ env.REGISTRY_DOCKERHUB }}:${CHANNEL}${VARIANT_SUFFIX},${{ env.REGISTRY_GHCR }}:${CHANNEL}${VARIANT_SUFFIX},${{ env.REGISTRY_QUAY }}:${CHANNEL}${VARIANT_SUFFIX}"
+              TAGS="$TAGS,${{ env.REGISTRY_DOCKERHUB }}:${CHANNEL}"
             fi
           fi
 
@@ -393,15 +372,15 @@ jobs:
         uses: docker/build-push-action@v6
         with:
           context: .
-          file: ${{ matrix.file }}
+          file: Dockerfile
           platforms: ${{ env.DOCKER_PLATFORMS }}
           push: ${{ needs.build-check.outputs.should_push == 'true' }}
           tags: ${{ steps.meta.outputs.tags }}
           labels: ${{ steps.meta.outputs.labels }}
           cache-from: |
-            type=gha,scope=docker-${{ matrix.variant }}
+            type=gha,scope=docker-binary
           cache-to: |
-            type=gha,mode=max,scope=docker-${{ matrix.variant }}
+            type=gha,mode=max,scope=docker-binary
           build-args: |
             BUILDTIME=$(date -u +'%Y-%m-%dT%H:%M:%SZ')
             VERSION=${{ needs.build-check.outputs.version }}
@@ -425,7 +404,7 @@ jobs:
     name: Docker Build Summary
     needs: [ build-check, build-docker ]
     if: always() && needs.build-check.outputs.should_build == 'true'
-    runs-on: ubicloud-standard-2
+    runs-on: ubuntu-latest
     steps:
       - name: Docker build completion summary
         run: |
@@ -450,10 +429,10 @@ jobs:
             "prerelease")
               echo "🧪 Prerelease Docker image has been built with ${VERSION} tags"
               echo "⚠️  This is a prerelease image - use with caution"
-              # TODO: Temporary change - alpha versions currently create the latest tag
-              # After the version is stable, you need to restore the following prompt information
+              # TODO: 临时修改 - alpha 版本当前会创建 latest 标签
+              # 等版本稳定后，需要恢复下面的提示信息
               if [[ "$VERSION" == *"alpha"* ]] && [[ "$CREATE_LATEST" == "true" ]]; then
-                echo "🏷️  Latest tag has been created for alpha version (temporary measures)"
+                echo "🏷️  Latest tag has been created for alpha version (临时措施)"
               else
                 echo "🚫 Latest tag NOT created for prerelease"
               fi

.github/workflows/e2e-s3tests.yml

@@ -1,443 +0,0 @@
-# Copyright 2024 RustFS Team
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-name: e2e-s3tests
-
-on:
-  workflow_dispatch:
-    inputs:
-      test-mode:
-        description: "Test mode to run"
-        required: true
-        type: choice
-        default: "single"
-        options:
-          - single
-          - multi
-      xdist:
-        description: "Enable pytest-xdist (parallel). '0' to disable."
-        required: false
-        default: "0"
-      maxfail:
-        description: "Stop after N failures (debug friendly)"
-        required: false
-        default: "1"
-      markexpr:
-        description: "pytest -m expression (feature filters)"
-        required: false
-        default: "not lifecycle and not versioning and not s3website and not bucket_logging and not encryption"
-
-env:
-  # main user
-  S3_ACCESS_KEY: rustfsadmin
-  S3_SECRET_KEY: rustfsadmin
-  # alt user (must be different from main for many s3-tests)
-  S3_ALT_ACCESS_KEY: rustfsalt
-  S3_ALT_SECRET_KEY: rustfsalt
-
-  S3_REGION: us-east-1
-
-  RUST_LOG: info
-  PLATFORM: linux/amd64
-  BUILDX_CACHE_SCOPE: rustfs-e2e-s3tests-source
-
-concurrency:
-  group: ${{ github.workflow }}-${{ github.ref }}-${{ github.event.inputs['test-mode'] || 'single' }}
-  cancel-in-progress: true
-
-defaults:
-  run:
-    shell: bash
-
-jobs:
-  s3tests-single:
-    if: github.event.inputs['test-mode'] == 'single'
-    runs-on: ubicloud-standard-2
-    timeout-minutes: 120
-    steps:
-      - uses: actions/checkout@v6
-
-      - name: Cache pip downloads
-        uses: actions/cache@v4
-        with:
-          path: ~/.cache/pip
-          key: ${{ runner.os }}-pip-e2e-s3tests-${{ hashFiles('.github/workflows/e2e-s3tests.yml') }}
-          restore-keys: |
-            ${{ runner.os }}-pip-e2e-s3tests-
-
-      - name: Install Python tools
-        run: |
-          python3 -m pip install --user --upgrade pip awscurl tox
-          echo "$HOME/.local/bin" >> "$GITHUB_PATH"
-
-      - name: Enable buildx
-        uses: docker/setup-buildx-action@v3
-
-      - name: Build RustFS image (source, cached)
-        run: |
-          DOCKER_BUILDKIT=1 docker buildx build --load \
-            --platform ${PLATFORM} \
-            --cache-from type=gha,scope=${BUILDX_CACHE_SCOPE} \
-            --cache-to type=gha,mode=max,scope=${BUILDX_CACHE_SCOPE} \
-            -t rustfs-ci \
-            -f Dockerfile.source .
-
-      - name: Create network
-        run: docker network inspect rustfs-net >/dev/null 2>&1 || docker network create rustfs-net
-
-      - name: Remove existing rustfs-single (if any)
-        run: docker rm -f rustfs-single >/dev/null 2>&1 || true
-
-      - name: Start single RustFS
-        run: |
-          docker run -d --name rustfs-single \
-            --network rustfs-net \
-            -p 9000:9000 \
-            -e RUSTFS_ADDRESS=0.0.0.0:9000 \
-            -e RUSTFS_ACCESS_KEY=$S3_ACCESS_KEY \
-            -e RUSTFS_SECRET_KEY=$S3_SECRET_KEY \
-            -e RUSTFS_VOLUMES="/data/rustfs0 /data/rustfs1 /data/rustfs2 /data/rustfs3" \
-            -v /tmp/rustfs-single:/data \
-            rustfs-ci
-
-      - name: Wait for RustFS ready
-        run: |
-          for i in {1..60}; do
-            if curl -sf http://127.0.0.1:9000/health >/dev/null 2>&1; then
-              echo "RustFS is ready"
-              exit 0
-            fi
-
-            if [ "$(docker inspect -f '{{.State.Running}}' rustfs-single 2>/dev/null)" != "true" ]; then
-              echo "RustFS container not running" >&2
-              docker logs rustfs-single || true
-              exit 1
-            fi
-
-            sleep 2
-          done
-
-          echo "Health check timed out" >&2
-          docker logs rustfs-single || true
-          exit 1
-
-      - name: Generate s3tests config
-        run: |
-          export S3_HOST=127.0.0.1
-          envsubst < .github/s3tests/s3tests.conf > s3tests.conf
-
-      - name: Provision s3-tests alt user (required by suite)
-        run: |
-          # Admin API requires AWS SigV4 signing. awscurl is used by RustFS codebase as well.
-          awscurl \
-            --service s3 \
-            --region "${S3_REGION}" \
-            --access_key "${S3_ACCESS_KEY}" \
-            --secret_key "${S3_SECRET_KEY}" \
-            -X PUT \
-            -H 'Content-Type: application/json' \
-            -d '{"secretKey":"'"${S3_ALT_SECRET_KEY}"'","status":"enabled","policy":"readwrite"}' \
-            "http://127.0.0.1:9000/rustfs/admin/v3/add-user?accessKey=${S3_ALT_ACCESS_KEY}"
-
-          # Explicitly attach built-in policy via policy mapping.
-          # s3-tests relies on alt client being able to ListBuckets during setup cleanup.
-          awscurl \
-            --service s3 \
-            --region "${S3_REGION}" \
-            --access_key "${S3_ACCESS_KEY}" \
-            --secret_key "${S3_SECRET_KEY}" \
-            -X PUT \
-            "http://127.0.0.1:9000/rustfs/admin/v3/set-user-or-group-policy?policyName=readwrite&userOrGroup=${S3_ALT_ACCESS_KEY}&isGroup=false"
-
-          # Sanity check: alt user can list buckets (should not be AccessDenied).
-          awscurl \
-            --service s3 \
-            --region "${S3_REGION}" \
-            --access_key "${S3_ALT_ACCESS_KEY}" \
-            --secret_key "${S3_ALT_SECRET_KEY}" \
-            -X GET \
-            "http://127.0.0.1:9000/" >/dev/null
-
-      - name: Prepare s3-tests
-        run: |
-          git clone --depth 1 https://github.com/ceph/s3-tests.git s3-tests
-
-      - name: Run ceph s3-tests (debug friendly)
-        run: |
-          export PATH="$HOME/.local/bin:$PATH"
-          mkdir -p artifacts/s3tests-single
-
-          cd s3-tests
-
-          set -o pipefail
-
-          MAXFAIL="${{ github.event.inputs.maxfail }}"
-          if [ -z "$MAXFAIL" ]; then MAXFAIL="1"; fi
-
-          MARKEXPR="${{ github.event.inputs.markexpr }}"
-          if [ -z "$MARKEXPR" ]; then MARKEXPR="not lifecycle and not versioning and not s3website and not bucket_logging and not encryption"; fi
-
-          XDIST="${{ github.event.inputs.xdist }}"
-          if [ -z "$XDIST" ]; then XDIST="0"; fi
-          XDIST_ARGS=""
-          if [ "$XDIST" != "0" ]; then
-            # Add pytest-xdist to requirements.txt so tox installs it inside
-            # its virtualenv. Installing outside tox does NOT work.
-            echo "pytest-xdist" >> requirements.txt
-            XDIST_ARGS="-n $XDIST --dist=loadgroup"
-          fi
-
-          # Run tests from s3tests/functional (boto2+boto3 combined directory).
-          S3TEST_CONF=${GITHUB_WORKSPACE}/s3tests.conf \
-            tox -- \
-              -vv -ra --showlocals --tb=long \
-              --maxfail="$MAXFAIL" \
-              --junitxml=${GITHUB_WORKSPACE}/artifacts/s3tests-single/junit.xml \
-              $XDIST_ARGS \
-              s3tests/functional/test_s3.py \
-              -m "$MARKEXPR" \
-              2>&1 | tee ${GITHUB_WORKSPACE}/artifacts/s3tests-single/pytest.log
-
-      - name: Collect RustFS logs
-        if: always()
-        run: |
-          mkdir -p artifacts/rustfs-single
-          docker logs rustfs-single > artifacts/rustfs-single/rustfs.log 2>&1 || true
-          docker inspect rustfs-single > artifacts/rustfs-single/inspect.json || true
-
-      - name: Upload artifacts
-        if: always() && env.ACT != 'true'
-        uses: actions/upload-artifact@v6
-        with:
-          name: s3tests-single
-          path: artifacts/**
-
-  s3tests-multi:
-    if: github.event_name == 'workflow_dispatch' && github.event.inputs['test-mode'] == 'multi'
-    runs-on: ubicloud-standard-2
-    timeout-minutes: 150
-    steps:
-      - uses: actions/checkout@v6
-
-      - name: Cache pip downloads
-        uses: actions/cache@v4
-        with:
-          path: ~/.cache/pip
-          key: ${{ runner.os }}-pip-e2e-s3tests-${{ hashFiles('.github/workflows/e2e-s3tests.yml') }}
-          restore-keys: |
-            ${{ runner.os }}-pip-e2e-s3tests-
-
-      - name: Install Python tools
-        run: |
-          python3 -m pip install --user --upgrade pip awscurl tox
-          echo "$HOME/.local/bin" >> "$GITHUB_PATH"
-
-      - name: Enable buildx
-        uses: docker/setup-buildx-action@v3
-
-      - name: Build RustFS image (source, cached)
-        run: |
-          DOCKER_BUILDKIT=1 docker buildx build --load \
-            --platform ${PLATFORM} \
-            --cache-from type=gha,scope=${BUILDX_CACHE_SCOPE} \
-            --cache-to type=gha,mode=max,scope=${BUILDX_CACHE_SCOPE} \
-            -t rustfs-ci \
-            -f Dockerfile.source .
-
-      - name: Prepare cluster compose
-        run: |
-          cat > compose.yml <<'EOF'
-          services:
-            rustfs1:
-              image: rustfs-ci
-              hostname: rustfs1
-              networks: [rustfs-net]
-              environment:
-                RUSTFS_ADDRESS: "0.0.0.0:9000"
-                RUSTFS_ACCESS_KEY: ${S3_ACCESS_KEY}
-                RUSTFS_SECRET_KEY: ${S3_SECRET_KEY}
-                RUSTFS_VOLUMES: "/data/rustfs0 /data/rustfs1 /data/rustfs2 /data/rustfs3"
-              volumes:
-                - rustfs1-data:/data
-            rustfs2:
-              image: rustfs-ci
-              hostname: rustfs2
-              networks: [rustfs-net]
-              environment:
-                RUSTFS_ADDRESS: "0.0.0.0:9000"
-                RUSTFS_ACCESS_KEY: ${S3_ACCESS_KEY}
-                RUSTFS_SECRET_KEY: ${S3_SECRET_KEY}
-                RUSTFS_VOLUMES: "/data/rustfs0 /data/rustfs1 /data/rustfs2 /data/rustfs3"
-              volumes:
-                - rustfs2-data:/data
-            rustfs3:
-              image: rustfs-ci
-              hostname: rustfs3
-              networks: [rustfs-net]
-              environment:
-                RUSTFS_ADDRESS: "0.0.0.0:9000"
-                RUSTFS_ACCESS_KEY: ${S3_ACCESS_KEY}
-                RUSTFS_SECRET_KEY: ${S3_SECRET_KEY}
-                RUSTFS_VOLUMES: "/data/rustfs0 /data/rustfs1 /data/rustfs2 /data/rustfs3"
-              volumes:
-                - rustfs3-data:/data
-            rustfs4:
-              image: rustfs-ci
-              hostname: rustfs4
-              networks: [rustfs-net]
-              environment:
-                RUSTFS_ADDRESS: "0.0.0.0:9000"
-                RUSTFS_ACCESS_KEY: ${S3_ACCESS_KEY}
-                RUSTFS_SECRET_KEY: ${S3_SECRET_KEY}
-                RUSTFS_VOLUMES: "/data/rustfs0 /data/rustfs1 /data/rustfs2 /data/rustfs3"
-              volumes:
-                - rustfs4-data:/data
-            lb:
-              image: haproxy:2.9
-              hostname: lb
-              networks: [rustfs-net]
-              ports:
-                - "9000:9000"
-              volumes:
-                - ./haproxy.cfg:/usr/local/etc/haproxy/haproxy.cfg:ro
-          networks:
-            rustfs-net:
-              name: rustfs-net
-          volumes:
-            rustfs1-data:
-            rustfs2-data:
-            rustfs3-data:
-            rustfs4-data:
-          EOF
-
-          cat > haproxy.cfg <<'EOF'
-          defaults
-            mode http
-            timeout connect 5s
-            timeout client  30s
-            timeout server  30s
-
-          frontend fe_s3
-            bind *:9000
-            default_backend be_s3
-
-          backend be_s3
-            balance roundrobin
-            server s1 rustfs1:9000 check
-            server s2 rustfs2:9000 check
-            server s3 rustfs3:9000 check
-            server s4 rustfs4:9000 check
-          EOF
-
-      - name: Launch cluster
-        run: docker compose -f compose.yml up -d
-
-      - name: Wait for LB ready
-        run: |
-          for i in {1..90}; do
-            if curl -sf http://127.0.0.1:9000/health >/dev/null 2>&1; then
-              echo "Load balancer is ready"
-              exit 0
-            fi
-            sleep 2
-          done
-          echo "LB or backend not ready" >&2
-          docker compose -f compose.yml logs --tail=200 || true
-          exit 1
-
-      - name: Generate s3tests config
-        run: |
-          export S3_HOST=127.0.0.1
-          envsubst < .github/s3tests/s3tests.conf > s3tests.conf
-
-      - name: Provision s3-tests alt user (required by suite)
-        run: |
-          awscurl \
-            --service s3 \
-            --region "${S3_REGION}" \
-            --access_key "${S3_ACCESS_KEY}" \
-            --secret_key "${S3_SECRET_KEY}" \
-            -X PUT \
-            -H 'Content-Type: application/json' \
-            -d '{"secretKey":"'"${S3_ALT_SECRET_KEY}"'","status":"enabled","policy":"readwrite"}' \
-            "http://127.0.0.1:9000/rustfs/admin/v3/add-user?accessKey=${S3_ALT_ACCESS_KEY}"
-
-          awscurl \
-            --service s3 \
-            --region "${S3_REGION}" \
-            --access_key "${S3_ACCESS_KEY}" \
-            --secret_key "${S3_SECRET_KEY}" \
-            -X PUT \
-            "http://127.0.0.1:9000/rustfs/admin/v3/set-user-or-group-policy?policyName=readwrite&userOrGroup=${S3_ALT_ACCESS_KEY}&isGroup=false"
-
-          awscurl \
-            --service s3 \
-            --region "${S3_REGION}" \
-            --access_key "${S3_ALT_ACCESS_KEY}" \
-            --secret_key "${S3_ALT_SECRET_KEY}" \
-            -X GET \
-            "http://127.0.0.1:9000/" >/dev/null
-
-      - name: Prepare s3-tests
-        run: |
-          git clone --depth 1 https://github.com/ceph/s3-tests.git s3-tests
-
-      - name: Run ceph s3-tests (multi, debug friendly)
-        run: |
-          export PATH="$HOME/.local/bin:$PATH"
-          mkdir -p artifacts/s3tests-multi
-
-          cd s3-tests
-
-          set -o pipefail
-
-          MAXFAIL="${{ github.event.inputs.maxfail }}"
-          if [ -z "$MAXFAIL" ]; then MAXFAIL="1"; fi
-
-          MARKEXPR="${{ github.event.inputs.markexpr }}"
-          if [ -z "$MARKEXPR" ]; then MARKEXPR="not lifecycle and not versioning and not s3website and not bucket_logging and not encryption"; fi
-
-          XDIST="${{ github.event.inputs.xdist }}"
-          if [ -z "$XDIST" ]; then XDIST="0"; fi
-          XDIST_ARGS=""
-          if [ "$XDIST" != "0" ]; then
-            # Add pytest-xdist to requirements.txt so tox installs it inside
-            # its virtualenv. Installing outside tox does NOT work.
-            echo "pytest-xdist" >> requirements.txt
-            XDIST_ARGS="-n $XDIST --dist=loadgroup"
-          fi
-
-          # Run tests from s3tests/functional (boto2+boto3 combined directory).
-          S3TEST_CONF=${GITHUB_WORKSPACE}/s3tests.conf \
-            tox -- \
-              -vv -ra --showlocals --tb=long \
-              --maxfail="$MAXFAIL" \
-              --junitxml=${GITHUB_WORKSPACE}/artifacts/s3tests-multi/junit.xml \
-              $XDIST_ARGS \
-              s3tests/functional/test_s3.py \
-              -m "$MARKEXPR" \
-              2>&1 | tee ${GITHUB_WORKSPACE}/artifacts/s3tests-multi/pytest.log
-
-      - name: Collect logs
-        if: always()
-        run: |
-          mkdir -p artifacts/cluster
-          docker compose -f compose.yml logs --no-color > artifacts/cluster/cluster.log 2>&1 || true
-
-      - name: Upload artifacts
-        if: always() && env.ACT != 'true'
-        uses: actions/upload-artifact@v6
-        with:
-          name: s3tests-multi
-          path: artifacts/**

.github/workflows/helm-package.yml

@@ -1,94 +0,0 @@
-# Copyright 2024 RustFS Team
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-name: Publish helm chart to artifacthub
-
-on:
-  workflow_run:
-    workflows: [ "Build and Release" ]
-    types: [ completed ]
-
-permissions:
-  contents: read
-
-env:
-  new_version: ${{ github.event.workflow_run.head_branch }}
-
-jobs:
-  build-helm-package:
-    runs-on: ubicloud-standard-2
-    # Only run on successful builds triggered by tag pushes (version format: x.y.z or x.y.z-suffix)
-    if: |
-      github.event.workflow_run.conclusion == 'success' && 
-      github.event.workflow_run.event == 'push' &&
-      contains(github.event.workflow_run.head_branch, '.')
-
-    steps:
-      - name: Checkout helm chart repo
-        uses: actions/checkout@v6
-
-      - name: Replace chart app version
-        run: |
-          set -e
-          set -x
-          old_version=$(grep "^appVersion:" helm/rustfs/Chart.yaml | awk '{print $2}')
-          sed -i "s/$old_version/$new_version/g" helm/rustfs/Chart.yaml
-
-      - name: Set up Helm
-        uses: azure/setup-helm@v4.3.0
-
-      - name: Package Helm Chart
-        run: |
-          cp helm/README.md helm/rustfs/
-          package_version=$(echo $new_version | awk -F '-' '{print $2}' | awk -F '.' '{print $NF}') 
-          helm package ./helm/rustfs --destination helm/rustfs/ --version "0.0.$package_version"
-
-      - name: Upload helm package as artifact
-        uses: actions/upload-artifact@v6
-        with:
-          name: helm-package
-          path: helm/rustfs/*.tgz
-          retention-days: 1
-
-  publish-helm-package:
-    runs-on: ubicloud-standard-2
-    needs: [ build-helm-package ]
-
-    steps:
-      - name: Checkout helm package repo
-        uses: actions/checkout@v6
-        with:
-          repository: rustfs/helm
-          token: ${{ secrets.RUSTFS_HELM_PACKAGE }}
-
-      - name: Download helm package
-        uses: actions/download-artifact@v7
-        with:
-          name: helm-package
-          path: ./
-
-      - name: Set up helm
-        uses: azure/setup-helm@v4.3.0
-
-      - name: Generate index
-        run: helm repo index . --url https://charts.rustfs.com
-
-      - name: Push helm package and index file
-        run: |
-          git config --global user.name "${{ secrets.USERNAME }}"
-          git config --global user.email "${{ secrets.EMAIL_ADDRESS }}"
-          git status .
-          git add .
-          git commit -m "Update rustfs helm package with $new_version."
-          git push origin main

.github/workflows/issue-translator.yml

@@ -25,7 +25,7 @@ permissions:
 
 jobs:
   build:
-    runs-on: ubicloud-standard-4
+    runs-on: ubuntu-latest
     steps:
       - uses: usthe/issues-translate-action@v2.7
         with:

.github/workflows/nix-flake-update.yml

@@ -1,65 +0,0 @@
-# Copyright 2024 RustFS Team
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#      http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-name: Update Nix Flake
-
-on:
-  workflow_dispatch:
-  schedule:
-    - cron: '0 0 * * 0' # Weekly on Sundays
-
-permissions:
-  contents: write
-  pull-requests: write
-
-concurrency:
-  group: ${{ github.workflow }}-${{ github.ref }}
-  cancel-in-progress: true
-
-jobs:
-  update-flake:
-    name: Update flake.lock
-    runs-on: ubuntu-latest
-    timeout-minutes: 45
-    steps:
-      - name: Checkout repository
-        uses: actions/checkout@v6
-
-      - name: Install Nix
-        uses: DeterminateSystems/determinate-nix-action@v3
-
-      - name: Check Nix flake inputs
-        uses: DeterminateSystems/flake-checker-action@v12
-
-      - name: Update flake.lock
-        id: update
-        uses: DeterminateSystems/update-flake-lock@main
-        with:
-          git-author-name: houseme
-          git-author-email: housemecn@gmail.com
-          git-committer-name: houseme
-          git-committer-email: housemecn@gmail.com
-          pr-title: "chore(deps): update flake.lock"
-          pr-labels: |
-            dependencies
-            nix
-            automated
-          commit-msg: "chore(deps): update flake.lock"
-          pr-reviewers: houseme, overtrue, majinghe
-          token: ${{ secrets.FLAKE_UPDATE_TOKEN }}
-
-      - name: Log PR details
-        if: steps.update.outputs.pull-request-number
-        run: |
-          echo "Pull Request created: ${{ steps.update.outputs.pull-request-number }}"

.github/workflows/nix.yml

@@ -1,84 +0,0 @@
-# Copyright 2024 RustFS Team
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#      http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-name: Nix CI
-
-on:
-  push:
-    branches: [ "main" ]
-    paths:
-      - 'flake.nix'
-      - 'flake.lock'
-      - 'Cargo.toml'
-      - 'Cargo.lock'
-      - '.github/workflows/nix.yml'
-  pull_request:
-    branches: [ "main" ]
-    paths:
-      - 'flake.nix'
-      - 'flake.lock'
-      - 'Cargo.toml'
-      - 'Cargo.lock'
-      - '.github/workflows/nix.yml'
-  workflow_dispatch:
-
-concurrency:
-  group: ${{ github.workflow }}-${{ github.ref }}
-  cancel-in-progress: true
-permissions:
-  contents: read
-
-jobs:
-  nix-validation:
-    name: Nix Build & Check
-    runs-on: ubuntu-latest
-    timeout-minutes: 45
-    steps:
-      - name: Checkout repository
-        uses: actions/checkout@v6
-
-      - name: Install Nix
-        uses: cachix/install-nix-action@v31
-        with:
-          github_access_token: ${{ secrets.GITHUB_TOKEN }}
-          extra_nix_config: |
-            experimental-features = nix-command flakes
-
-      - name: Setup Magic Nix Cache
-        uses: DeterminateSystems/magic-nix-cache-action@v13
-
-      - name: Setup Flake Checker
-        uses: DeterminateSystems/flake-checker-action@v12
-
-      - name: Verify Flake
-        run: |
-          echo "Checking flake structure and evaluation..."
-          nix flake show
-          nix flake check --print-build-logs
-
-      - name: Build RustFS
-        run: |
-          echo "Building the default package..."
-          nix build .#default --print-build-logs
-
-      - name: Test Binary
-        run: |
-          echo "Verifying the built binary..."
-          if [ -x "./result/bin/rustfs" ]; then
-              ./result/bin/rustfs --help
-              echo "Binary verification successful."
-          else
-              echo "Error: Binary not found or not executable at ./result/bin/rustfs"
-              exit 1
-          fi

.github/workflows/performance.yml

@@ -40,13 +40,11 @@ env:
 jobs:
   performance-profile:
     name: Performance Profiling
-    runs-on: ubicloud-standard-2
+    runs-on: ubuntu-latest
     timeout-minutes: 30
-    env:
-      FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"
     steps:
       - name: Checkout repository
-        uses: actions/checkout@v6
+        uses: actions/checkout@v5
 
       - name: Setup Rust environment
         uses: ./.github/actions/setup
@@ -109,7 +107,7 @@ jobs:
 
       - name: Upload profile data
         if: steps.profiling.outputs.profile_generated == 'true'
-        uses: actions/upload-artifact@v6
+        uses: actions/upload-artifact@v4
         with:
           name: performance-profile-${{ github.run_number }}
           path: samply-profile.json
@@ -117,13 +115,11 @@ jobs:
 
   benchmark:
     name: Benchmark Tests
-    runs-on: ubicloud-standard-2
+    runs-on: ubuntu-latest
     timeout-minutes: 45
-    env:
-      FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"
     steps:
       - name: Checkout repository
-        uses: actions/checkout@v6
+        uses: actions/checkout@v5
 
       - name: Setup Rust environment
         uses: ./.github/actions/setup
@@ -139,7 +135,7 @@ jobs:
           tee benchmark-results.json
 
       - name: Upload benchmark results
-        uses: actions/upload-artifact@v6
+        uses: actions/upload-artifact@v4
         with:
           name: benchmark-results-${{ github.run_number }}
           path: benchmark-results.json

.github/workflows/stale.yml

@@ -1,20 +0,0 @@
-name: "Mark stale issues"
-on:
-  schedule:
-    - cron: "30 1 * * *"
-
-jobs:
-  stale:
-    runs-on: ubuntu-latest
-    steps:
-      - uses: actions/stale@v9
-        with:
-          repo-token: ${{ secrets.GITHUB_TOKEN }}
-          stale-issue-message: 'This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.'
-          stale-issue-label: 'stale'
-          ## Mark if there is no activity for more than 7 days
-          days-before-stale: 7  
-          # If no one responds after 3 days, the tag will be closed.
-          days-before-close: 3   
-          # These tags are exempt and will not close automatically.
-          exempt-issue-labels: 'pinned,security'

.gitignore

@@ -2,13 +2,9 @@
 .DS_Store
 .idea
 .vscode
-.cursor
-.direnv/
 /test
 /logs
 /data
-/docs
-/rustfs-data/
 .devcontainer
 rustfs/static/*
 !rustfs/static/.gitkeep
@@ -16,12 +12,10 @@ vendor
 cli/rustfs-gui/embedded-rustfs/rustfs
 *.log
 deploy/certs/*
-deploy/data/*
 *jsonl
 .env
 .rustfs.sys
-.cargo/
-!.cargo/config.toml
+.cargo
 profile.json
 .docker/openobserve-otel/data
 *.zst
@@ -29,27 +23,4 @@ profile.json
 *.go
 *.pb
 *.svg
-deploy/logs/*.log.*
-artifacts/
-# s3-tests local artifacts (root directory only)
-/s3-tests/
-/s3-tests-local/
-/s3tests.conf
-/s3tests.conf.*
-*.events
-*.audit
-*.snappy
-PR_DESCRIPTION.md
-scripts/s3-tests/selected_tests.txt
-docs
-
-# nix stuff
-result*
-*.gz
-rustfs-webdav.code-workspace
-
-.aiexclude
-*.bak
-# Local test/benchmark artifacts
-benchmarks.logs
-tmp/
+deploy/logs/*.log.*

.pre-commit-config.yaml

@@ -1,32 +0,0 @@
-# See https://pre-commit.com for more information
-# See https://pre-commit.com/hooks.html for more hooks
-repos:
-  - repo: local
-    hooks:
-      - id: cargo-fmt
-        name: cargo fmt
-        entry: cargo fmt --all --check
-        language: system
-        types: [rust]
-        pass_filenames: false
-
-      - id: cargo-clippy
-        name: cargo clippy
-        entry: cargo clippy --all-targets --all-features -- -D warnings
-        language: system
-        types: [rust]
-        pass_filenames: false
-
-      - id: cargo-check
-        name: cargo check
-        entry: cargo check --all-targets
-        language: system
-        types: [rust]
-        pass_filenames: false
-
-      - id: cargo-test
-        name: cargo test
-        entry: bash -c 'cargo test --workspace --exclude e2e_test && cargo test --all --doc'
-        language: system
-        types: [rust]
-        pass_filenames: false

.rules.md

@@ -0,0 +1,702 @@
+# RustFS Project AI Coding Rules
+
+## 🚨🚨🚨 CRITICAL DEVELOPMENT RULES - ZERO TOLERANCE 🚨🚨🚨
+
+### ⛔️ ABSOLUTE PROHIBITION: NEVER COMMIT DIRECTLY TO MASTER/MAIN BRANCH ⛔️
+
+**🔥 THIS IS THE MOST CRITICAL RULE - VIOLATION WILL RESULT IN IMMEDIATE REVERSAL 🔥**
+
+- **🚫 ZERO DIRECT COMMITS TO MAIN/MASTER BRANCH - ABSOLUTELY FORBIDDEN**
+- **🚫 ANY DIRECT COMMIT TO MAIN BRANCH MUST BE IMMEDIATELY REVERTED**
+- **🚫 NO EXCEPTIONS FOR HOTFIXES, EMERGENCIES, OR URGENT CHANGES**
+- **🚫 NO EXCEPTIONS FOR SMALL CHANGES, TYPOS, OR DOCUMENTATION UPDATES**
+- **🚫 NO EXCEPTIONS FOR ANYONE - MAINTAINERS, CONTRIBUTORS, OR ADMINS**
+
+### 📋 MANDATORY WORKFLOW - STRICTLY ENFORCED
+
+**EVERY SINGLE CHANGE MUST FOLLOW THIS WORKFLOW:**
+
+1. **Check current branch**: `git branch` (MUST NOT be on main/master)
+2. **Switch to main**: `git checkout main`
+3. **Pull latest**: `git pull origin main`
+4. **Create feature branch**: `git checkout -b feat/your-feature-name`
+5. **Make changes ONLY on feature branch**
+6. **Test thoroughly before committing**
+7. **Commit and push to feature branch**: `git push origin feat/your-feature-name`
+8. **Create Pull Request**: Use `gh pr create` (MANDATORY)
+9. **Wait for PR approval**: NO self-merging allowed
+10. **Merge through GitHub interface**: ONLY after approval
+
+### 🔒 ENFORCEMENT MECHANISMS
+
+- **Branch protection rules**: Main branch is protected
+- **Pre-commit hooks**: Will block direct commits to main
+- **CI/CD checks**: All PRs must pass before merging
+- **Code review requirement**: At least one approval needed
+- **Automated reversal**: Direct commits to main will be automatically reverted
+
+## 🎯 Core AI Development Principles
+
+### Five Execution Steps
+
+#### 1. Task Analysis and Planning
+- **Clear Objectives**: Deeply understand task requirements and expected results before starting coding
+- **Plan Development**: List specific files, components, and functions that need modification, explaining the reasons for changes
+- **Risk Assessment**: Evaluate the impact of changes on existing functionality, develop rollback plans
+
+#### 2. Precise Code Location
+- **File Identification**: Determine specific files and line numbers that need modification
+- **Impact Analysis**: Avoid modifying irrelevant files, clearly state the reason for each file modification
+- **Minimization Principle**: Unless explicitly required by the task, do not create new abstraction layers or refactor existing code
+
+#### 3. Minimal Code Changes
+- **Focus on Core**: Only write code directly required by the task
+- **Avoid Redundancy**: Do not add unnecessary logs, comments, tests, or error handling
+- **Isolation**: Ensure new code does not interfere with existing functionality, maintain code independence
+
+#### 4. Strict Code Review
+- **Correctness Check**: Verify the correctness and completeness of code logic
+- **Style Consistency**: Ensure code conforms to established project coding style
+- **Side Effect Assessment**: Evaluate the impact of changes on downstream systems
+
+#### 5. Clear Delivery Documentation
+- **Change Summary**: Detailed explanation of all modifications and reasons
+- **File List**: List all modified files and their specific changes
+- **Risk Statement**: Mark any assumptions or potential risk points
+
+### Core Principles
+- **🎯 Precise Execution**: Strictly follow task requirements, no arbitrary innovation
+- **⚡ Efficient Development**: Avoid over-design, only do necessary work
+- **🛡️ Safe and Reliable**: Always follow development processes, ensure code quality and system stability
+- **🔒 Cautious Modification**: Only modify when clearly knowing what needs to be changed and having confidence
+
+### Additional AI Behavior Rules
+
+1. **Use English for all code comments and documentation** - All comments, variable names, function names, documentation, and user-facing text in code should be in English
+2. **Clean up temporary scripts after use** - Any temporary scripts, test files, or helper files created during AI work should be removed after task completion
+3. **Only make confident modifications** - Do not make speculative changes or "convenient" modifications outside the task scope. If uncertain about a change, ask for clarification rather than guessing
+
+## Project Overview
+
+RustFS is a high-performance distributed object storage system written in Rust, compatible with S3 API. The project adopts a modular architecture, supporting erasure coding storage, multi-tenant management, observability, and other enterprise-level features.
+
+## Core Architecture Principles
+
+### 1. Modular Design
+
+- Project uses Cargo workspace structure, containing multiple independent crates
+- Core modules: `rustfs` (main service), `ecstore` (erasure coding storage), `common` (shared components)
+- Functional modules: `iam` (identity management), `madmin` (management interface), `crypto` (encryption), etc.
+- Tool modules: `cli` (command line tool), `crates/*` (utility libraries)
+
+### 2. Asynchronous Programming Pattern
+
+- Comprehensive use of `tokio` async runtime
+- Prioritize `async/await` syntax
+- Use `async-trait` for async methods in traits
+- Avoid blocking operations, use `spawn_blocking` when necessary
+
+### 3. Error Handling Strategy
+
+- **Use modular, type-safe error handling with `thiserror`**
+- Each module should define its own error type using `thiserror::Error` derive macro
+- Support error chains and context information through `#[from]` and `#[source]` attributes
+- Use `Result<T>` type aliases for consistency within each module
+- Error conversion between modules should use explicit `From` implementations
+- Follow the pattern: `pub type Result<T> = core::result::Result<T, Error>`
+- Use `#[error("description")]` attributes for clear error messages
+- Support error downcasting when needed through `other()` helper methods
+- Implement `Clone` for errors when required by the domain logic
+
+## Code Style Guidelines
+
+### 1. Formatting Configuration
+
+```toml
+max_width = 130
+fn_call_width = 90
+single_line_let_else_max_width = 100
+```
+
+### 2. **🔧 MANDATORY Code Formatting Rules**
+
+**CRITICAL**: All code must be properly formatted before committing. This project enforces strict formatting standards to maintain code consistency and readability.
+
+#### Pre-commit Requirements (MANDATORY)
+
+Before every commit, you **MUST**:
+
+1. **Format your code**:
+   ```bash
+   cargo fmt --all
+   ```
+
+2. **Verify formatting**:
+   ```bash
+   cargo fmt --all --check
+   ```
+
+3. **Pass clippy checks**:
+   ```bash
+   cargo clippy --all-targets --all-features -- -D warnings
+   ```
+
+4. **Ensure compilation**:
+   ```bash
+   cargo check --all-targets
+   ```
+
+#### Quick Commands
+
+Use these convenient Makefile targets for common tasks:
+
+```bash
+# Format all code
+make fmt
+
+# Check if code is properly formatted
+make fmt-check
+
+# Run clippy checks
+make clippy
+
+# Run compilation check
+make check
+
+# Run tests
+make test
+
+# Run all pre-commit checks (format + clippy + check + test)
+make pre-commit
+
+# Setup git hooks (one-time setup)
+make setup-hooks
+```
+
+### 3. Naming Conventions
+
+- Use `snake_case` for functions, variables, modules
+- Use `PascalCase` for types, traits, enums
+- Constants use `SCREAMING_SNAKE_CASE`
+- Global variables prefix `GLOBAL_`, e.g., `GLOBAL_Endpoints`
+- Use meaningful and descriptive names for variables, functions, and methods
+- Avoid meaningless names like `temp`, `data`, `foo`, `bar`, `test123`
+- Choose names that clearly express the purpose and intent
+
+### 4. Type Declaration Guidelines
+
+- **Prefer type inference over explicit type declarations** when the type is obvious from context
+- Let the Rust compiler infer types whenever possible to reduce verbosity and improve maintainability
+- Only specify types explicitly when:
+  - The type cannot be inferred by the compiler
+  - Explicit typing improves code clarity and readability
+  - Required for API boundaries (function signatures, public struct fields)
+  - Needed to resolve ambiguity between multiple possible types
+
+### 5. Documentation Comments
+
+- Public APIs must have documentation comments
+- Use `///` for documentation comments
+- Complex functions add `# Examples` and `# Parameters` descriptions
+- Error cases use `# Errors` descriptions
+- Always use English for all comments and documentation
+- Avoid meaningless comments like "debug 111" or placeholder text
+
+### 6. Import Guidelines
+
+- Standard library imports first
+- Third-party crate imports in the middle
+- Project internal imports last
+- Group `use` statements with blank lines between groups
+
+## Asynchronous Programming Guidelines
+
+### 1. Trait Definition
+
+```rust
+#[async_trait::async_trait]
+pub trait StorageAPI: Send + Sync {
+    async fn get_object(&self, bucket: &str, object: &str) -> Result<ObjectInfo>;
+}
+```
+
+### 2. Error Handling
+
+```rust
+// Use ? operator to propagate errors
+async fn example_function() -> Result<()> {
+    let data = read_file("path").await?;
+    process_data(data).await?;
+    Ok(())
+}
+```
+
+### 3. Concurrency Control
+
+- Use `Arc` and `Mutex`/`RwLock` for shared state management
+- Prioritize async locks from `tokio::sync`
+- Avoid holding locks for long periods
+
+## Logging and Tracing Guidelines
+
+### 1. Tracing Usage
+
+```rust
+#[tracing::instrument(skip(self, data))]
+async fn process_data(&self, data: &[u8]) -> Result<()> {
+    info!("Processing {} bytes", data.len());
+    // Implementation logic
+}
+```
+
+### 2. Log Levels
+
+- `error!`: System errors requiring immediate attention
+- `warn!`: Warning information that may affect functionality
+- `info!`: Important business information
+- `debug!`: Debug information for development use
+- `trace!`: Detailed execution paths
+
+### 3. Structured Logging
+
+```rust
+info!(
+    counter.rustfs_api_requests_total = 1_u64,
+    key_request_method = %request.method(),
+    key_request_uri_path = %request.uri().path(),
+    "API request processed"
+);
+```
+
+## Error Handling Guidelines
+
+### 1. Error Type Definition
+
+```rust
+// Use thiserror for module-specific error types
+#[derive(thiserror::Error, Debug)]
+pub enum MyError {
+    #[error("IO error: {0}")]
+    Io(#[from] std::io::Error),
+
+    #[error("Storage error: {0}")]
+    Storage(#[from] ecstore::error::StorageError),
+
+    #[error("Custom error: {message}")]
+    Custom { message: String },
+
+    #[error("File not found: {path}")]
+    FileNotFound { path: String },
+
+    #[error("Invalid configuration: {0}")]
+    InvalidConfig(String),
+}
+
+// Provide Result type alias for the module
+pub type Result<T> = core::result::Result<T, MyError>;
+```
+
+### 2. Error Helper Methods
+
+```rust
+impl MyError {
+    /// Create error from any compatible error type
+    pub fn other<E>(error: E) -> Self
+    where
+        E: Into<Box<dyn std::error::Error + Send + Sync>>,
+    {
+        MyError::Io(std::io::Error::other(error))
+    }
+}
+```
+
+### 3. Error Context and Propagation
+
+```rust
+// Use ? operator for clean error propagation
+async fn example_function() -> Result<()> {
+    let data = read_file("path").await?;
+    process_data(data).await?;
+    Ok(())
+}
+
+// Add context to errors
+fn process_with_context(path: &str) -> Result<()> {
+    std::fs::read(path)
+        .map_err(|e| MyError::Custom {
+            message: format!("Failed to read {}: {}", path, e)
+        })?;
+    Ok(())
+}
+```
+
+## Performance Optimization Guidelines
+
+### 1. Memory Management
+
+- Use `Bytes` instead of `Vec<u8>` for zero-copy operations
+- Avoid unnecessary cloning, use reference passing
+- Use `Arc` for sharing large objects
+
+### 2. Concurrency Optimization
+
+```rust
+// Use join_all for concurrent operations
+let futures = disks.iter().map(|disk| disk.operation());
+let results = join_all(futures).await;
+```
+
+### 3. Caching Strategy
+
+- Use `LazyLock` for global caching
+- Implement LRU cache to avoid memory leaks
+
+## Testing Guidelines
+
+### 1. Unit Tests
+
+```rust
+#[cfg(test)]
+mod tests {
+    use super::*;
+    use test_case::test_case;
+
+    #[tokio::test]
+    async fn test_async_function() {
+        let result = async_function().await;
+        assert!(result.is_ok());
+    }
+
+    #[test_case("input1", "expected1")]
+    #[test_case("input2", "expected2")]
+    fn test_with_cases(input: &str, expected: &str) {
+        assert_eq!(function(input), expected);
+    }
+}
+```
+
+### 2. Integration Tests
+
+- Use `e2e_test` module for end-to-end testing
+- Simulate real storage environments
+
+### 3. Test Quality Standards
+
+- Write meaningful test cases that verify actual functionality
+- Avoid placeholder or debug content like "debug 111", "test test", etc.
+- Use descriptive test names that clearly indicate what is being tested
+- Each test should have a clear purpose and verify specific behavior
+- Test data should be realistic and representative of actual use cases
+
+## Cross-Platform Compatibility Guidelines
+
+### 1. CPU Architecture Compatibility
+
+- **Always consider multi-platform and different CPU architecture compatibility** when writing code
+- Support major architectures: x86_64, aarch64 (ARM64), and other target platforms
+- Use conditional compilation for architecture-specific code:
+
+```rust
+#[cfg(target_arch = "x86_64")]
+fn optimized_x86_64_function() { /* x86_64 specific implementation */ }
+
+#[cfg(target_arch = "aarch64")]
+fn optimized_aarch64_function() { /* ARM64 specific implementation */ }
+
+#[cfg(not(any(target_arch = "x86_64", target_arch = "aarch64")))]
+fn generic_function() { /* Generic fallback implementation */ }
+```
+
+### 2. Platform-Specific Dependencies
+
+- Use feature flags for platform-specific dependencies
+- Provide fallback implementations for unsupported platforms
+- Test on multiple architectures in CI/CD pipeline
+
+### 3. Endianness Considerations
+
+- Use explicit byte order conversion when dealing with binary data
+- Prefer `to_le_bytes()`, `from_le_bytes()` for consistent little-endian format
+- Use `byteorder` crate for complex binary format handling
+
+### 4. SIMD and Performance Optimizations
+
+- Use portable SIMD libraries like `wide` or `packed_simd`
+- Provide fallback implementations for non-SIMD architectures
+- Use runtime feature detection when appropriate
+
+## Security Guidelines
+
+### 1. Memory Safety
+
+- Disable `unsafe` code (workspace.lints.rust.unsafe_code = "deny")
+- Use `rustls` instead of `openssl`
+
+### 2. Authentication and Authorization
+
+```rust
+// Use IAM system for permission checks
+let identity = iam.authenticate(&access_key, &secret_key).await?;
+iam.authorize(&identity, &action, &resource).await?;
+```
+
+## Configuration Management Guidelines
+
+### 1. Environment Variables
+
+- Use `RUSTFS_` prefix
+- Support both configuration files and environment variables
+- Provide reasonable default values
+
+### 2. Configuration Structure
+
+```rust
+#[derive(Debug, Deserialize, Clone)]
+pub struct Config {
+    pub address: String,
+    pub volumes: String,
+    #[serde(default)]
+    pub console_enable: bool,
+}
+```
+
+## Dependency Management Guidelines
+
+### 1. Workspace Dependencies
+
+- Manage versions uniformly at workspace level
+- Use `workspace = true` to inherit configuration
+
+### 2. Feature Flags
+
+```rust
+[features]
+default = ["file"]
+gpu = ["dep:nvml-wrapper"]
+kafka = ["dep:rdkafka"]
+```
+
+## Deployment and Operations Guidelines
+
+### 1. Containerization
+
+- Provide Dockerfile and docker-compose configuration
+- Support multi-stage builds to optimize image size
+
+### 2. Observability
+
+- Integrate OpenTelemetry for distributed tracing
+- Support Prometheus metrics collection
+- Provide Grafana dashboards
+
+### 3. Health Checks
+
+```rust
+// Implement health check endpoint
+async fn health_check() -> Result<HealthStatus> {
+    // Check component status
+}
+```
+
+## Code Review Checklist
+
+### 1. **Code Formatting and Quality (MANDATORY)**
+
+- [ ] **Code is properly formatted** (`cargo fmt --all --check` passes)
+- [ ] **All clippy warnings are resolved** (`cargo clippy --all-targets --all-features -- -D warnings` passes)
+- [ ] **Code compiles successfully** (`cargo check --all-targets` passes)
+- [ ] **Pre-commit hooks are working** and all checks pass
+- [ ] **No formatting-related changes** mixed with functional changes (separate commits)
+
+### 2. Functionality
+
+- [ ] Are all error cases properly handled?
+- [ ] Is there appropriate logging?
+- [ ] Is there necessary test coverage?
+
+### 3. Performance
+
+- [ ] Are unnecessary memory allocations avoided?
+- [ ] Are async operations used correctly?
+- [ ] Are there potential deadlock risks?
+
+### 4. Security
+
+- [ ] Are input parameters properly validated?
+- [ ] Are there appropriate permission checks?
+- [ ] Is information leakage avoided?
+
+### 5. Cross-Platform Compatibility
+
+- [ ] Does the code work on different CPU architectures (x86_64, aarch64)?
+- [ ] Are platform-specific features properly gated with conditional compilation?
+- [ ] Is byte order handling correct for binary data?
+- [ ] Are there appropriate fallback implementations for unsupported platforms?
+
+### 6. Code Commits and Documentation
+
+- [ ] Does it comply with [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/)?
+- [ ] Are commit messages concise and under 72 characters for the title line?
+- [ ] Commit titles should be concise and in English, avoid Chinese
+- [ ] Is PR description provided in copyable markdown format for easy copying?
+
+## Common Patterns and Best Practices
+
+### 1. Resource Management
+
+```rust
+// Use RAII pattern for resource management
+pub struct ResourceGuard {
+    resource: Resource,
+}
+
+impl Drop for ResourceGuard {
+    fn drop(&mut self) {
+        // Clean up resources
+    }
+}
+```
+
+### 2. Dependency Injection
+
+```rust
+// Use dependency injection pattern
+pub struct Service {
+    config: Arc<Config>,
+    storage: Arc<dyn StorageAPI>,
+}
+```
+
+### 3. Graceful Shutdown
+
+```rust
+// Implement graceful shutdown
+async fn shutdown_gracefully(shutdown_rx: &mut Receiver<()>) {
+    tokio::select! {
+        _ = shutdown_rx.recv() => {
+            info!("Received shutdown signal");
+            // Perform cleanup operations
+        }
+        _ = tokio::time::sleep(SHUTDOWN_TIMEOUT) => {
+            warn!("Shutdown timeout reached");
+        }
+    }
+}
+```
+
+## Domain-Specific Guidelines
+
+### 1. Storage Operations
+
+- All storage operations must support erasure coding
+- Implement read/write quorum mechanisms
+- Support data integrity verification
+
+### 2. Network Communication
+
+- Use gRPC for internal service communication
+- HTTP/HTTPS support for S3-compatible API
+- Implement connection pooling and retry mechanisms
+
+### 3. Metadata Management
+
+- Use FlatBuffers for serialization
+- Support version control and migration
+- Implement metadata caching
+
+## Branch Management and Development Workflow
+
+### Branch Management
+
+- **🚨 CRITICAL: NEVER modify code directly on main or master branch - THIS IS ABSOLUTELY FORBIDDEN 🚨**
+- **⚠️ ANY DIRECT COMMITS TO MASTER/MAIN WILL BE REJECTED AND MUST BE REVERTED IMMEDIATELY ⚠️**
+- **🔒 ALL CHANGES MUST GO THROUGH PULL REQUESTS - NO DIRECT COMMITS TO MAIN UNDER ANY CIRCUMSTANCES 🔒**
+- **Always work on feature branches - NO EXCEPTIONS**
+- Always check the .rules.md file before starting to ensure you understand the project guidelines
+- **MANDATORY workflow for ALL changes:**
+   1. `git checkout main` (switch to main branch)
+   2. `git pull` (get latest changes)
+   3. `git checkout -b feat/your-feature-name` (create and switch to feature branch)
+   4. Make your changes ONLY on the feature branch
+   5. Test thoroughly before committing
+   6. Commit and push to the feature branch
+   7. **Create a pull request for code review - THIS IS THE ONLY WAY TO MERGE TO MAIN**
+   8. **Wait for PR approval before merging - NEVER merge your own PRs without review**
+- Use descriptive branch names following the pattern: `feat/feature-name`, `fix/issue-name`, `refactor/component-name`, etc.
+- **Double-check current branch before ANY commit: `git branch` to ensure you're NOT on main/master**
+- **Pull Request Requirements:**
+  - All changes must be submitted via PR regardless of size or urgency
+  - PRs must include comprehensive description and testing information
+  - PRs must pass all CI/CD checks before merging
+  - PRs require at least one approval from code reviewers
+  - Even hotfixes and emergency changes must go through PR process
+- **Enforcement:**
+  - Main branch should be protected with branch protection rules
+  - Direct pushes to main should be blocked by repository settings
+  - Any accidental direct commits to main must be immediately reverted via PR
+
+### Development Workflow
+
+## 🎯 **Core Development Principles**
+
+- **🔴 Every change must be precise - don't modify unless you're confident**
+  - Carefully analyze code logic and ensure complete understanding before making changes
+  - When uncertain, prefer asking users or consulting documentation over blind modifications
+  - Use small iterative steps, modify only necessary parts at a time
+  - Evaluate impact scope before changes to ensure no new issues are introduced
+
+- **🚀 GitHub PR creation prioritizes gh command usage**
+  - Prefer using `gh pr create` command to create Pull Requests
+  - Avoid having users manually create PRs through web interface
+  - Provide clear and professional PR titles and descriptions
+  - Using `gh` commands ensures better integration and automation
+
+## 📝 **Code Quality Requirements**
+
+- Use English for all code comments, documentation, and variable names
+- Write meaningful and descriptive names for variables, functions, and methods
+- Avoid meaningless test content like "debug 111" or placeholder values
+- Before each change, carefully read the existing code to ensure you understand the code structure and implementation, do not break existing logic implementation, do not introduce new issues
+- Ensure each change provides sufficient test cases to guarantee code correctness
+- Do not arbitrarily modify numbers and constants in test cases, carefully analyze their meaning to ensure test case correctness
+- When writing or modifying tests, check existing test cases to ensure they have scientific naming and rigorous logic testing, if not compliant, modify test cases to ensure scientific and rigorous testing
+- **Before committing any changes, run `cargo clippy --all-targets --all-features -- -D warnings` to ensure all code passes Clippy checks**
+- After each development completion, first git add . then git commit -m "feat: feature description" or "fix: issue description", ensure compliance with [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/)
+- **Keep commit messages concise and under 72 characters** for the title line, use body for detailed explanations if needed
+- After each development completion, first git push to remote repository
+- After each change completion, summarize the changes, do not create summary files, provide a brief change description, ensure compliance with [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/)
+- Provide change descriptions needed for PR in the conversation, ensure compliance with [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/)
+- **Always provide PR descriptions in English** after completing any changes, including:
+  - Clear and concise title following Conventional Commits format
+  - Detailed description of what was changed and why
+  - List of key changes and improvements
+  - Any breaking changes or migration notes if applicable
+  - Testing information and verification steps
+- **Provide PR descriptions in copyable markdown format** enclosed in code blocks for easy one-click copying
+
+## 🚫 AI Documentation Generation Restrictions
+
+### Forbidden Summary Documents
+
+- **Strictly forbidden to create any form of AI-generated summary documents**
+- **Do not create documents containing large amounts of emoji, detailed formatting tables and typical AI style**
+- **Do not generate the following types of documents in the project:**
+  - Benchmark summary documents (BENCHMARK*.md)
+  - Implementation comparison analysis documents (IMPLEMENTATION_COMPARISON*.md)
+  - Performance analysis report documents
+  - Architecture summary documents
+  - Feature comparison documents
+  - Any documents with large amounts of emoji and formatted content
+- **If documentation is needed, only create when explicitly requested by the user, and maintain a concise and practical style**
+- **Documentation should focus on actually needed information, avoiding excessive formatting and decorative content**
+- **Any discovered AI-generated summary documents should be immediately deleted**
+
+### Allowed Documentation Types
+
+- README.md (project introduction, keep concise)
+- Technical documentation (only create when explicitly needed)
+- User manual (only create when explicitly needed)
+- API documentation (generated from code)
+- Changelog (CHANGELOG.md)
+
+These rules should serve as guiding principles when developing the RustFS project, ensuring code quality, performance, and maintainability.

.vscode/launch.json

@@ -1,31 +1,9 @@
 {
+    // 使用 IntelliSense 了解相关属性。 
+    // 悬停以查看现有属性的描述。
+    // 欲了解更多信息，请访问: https://go.microsoft.com/fwlink/?linkid=830387
     "version": "0.2.0",
     "configurations": [
-          {
-            "type": "lldb",
-            "request": "launch",
-            "name": "Debug(only) executable 'rustfs'",
-            "env": {
-              "RUST_LOG": "rustfs=info,ecstore=info,s3s=info,iam=info",
-              "RUSTFS_SKIP_BACKGROUND_TASK": "on"
-              //"RUSTFS_OBS_LOG_DIRECTORY": "./deploy/logs",
-              // "RUSTFS_POLICY_PLUGIN_URL":"http://localhost:8181/v1/data/rustfs/authz/allow",
-              // "RUSTFS_POLICY_PLUGIN_AUTH_TOKEN":"your-opa-token"
-            },
-            "program": "${workspaceFolder}/target/debug/rustfs",
-            "args": [
-              "--access-key",
-              "rustfsadmin",
-              "--secret-key",
-              "rustfsadmin",
-              "--address",
-              "0.0.0.0:9010",
-              "--server-domains",
-              "127.0.0.1:9010",
-              "./target/volume/test{1...4}"
-            ],
-            "cwd": "${workspaceFolder}"
-        },
         {
             "type": "lldb",
             "request": "launch",
@@ -44,7 +22,6 @@
             "env": {
                 "RUST_LOG": "rustfs=debug,ecstore=info,s3s=debug,iam=debug",
                 "RUSTFS_SKIP_BACKGROUND_TASK": "on",
-                //"RUSTFS_OBS_LOG_DIRECTORY": "./deploy/logs",
                 // "RUSTFS_POLICY_PLUGIN_URL":"http://localhost:8181/v1/data/rustfs/authz/allow",
                 // "RUSTFS_POLICY_PLUGIN_AUTH_TOKEN":"your-opa-token" 
             },
@@ -89,8 +66,12 @@
                     "test",
                     "--no-run",
                     "--lib",
-                    "--package=rustfs-ecstore"
-                ]
+                    "--package=ecstore"
+                ],
+                "filter": {
+                    "name": "ecstore",
+                    "kind": "lib"
+                }
             },
             "args": [],
             "cwd": "${workspaceFolder}"
@@ -99,58 +80,15 @@
             "name": "Debug executable target/debug/rustfs",
             "type":  "lldb",
             "request": "launch",
-            "cargo": {
-                "args": [
-                    "run",
-                    "--bin",
-                    "rustfs",
-                    "-j",
-                    "1",
-                    "--profile",
-                    "dev"
-                ]
-            },
+            "program": "${workspaceFolder}/target/debug/rustfs",
             "args": [],
             "cwd": "${workspaceFolder}",
             //"stopAtEntry": false,
             //"preLaunchTask": "cargo build",
-            "env": {
-                "RUSTFS_ACCESS_KEY": "rustfsadmin",
-                "RUSTFS_SECRET_KEY": "rustfsadmin",
-                //"RUSTFS_VOLUMES": "./target/volume/test{1...4}",
-                "RUSTFS_ADDRESS": ":9000",
-                "RUSTFS_CONSOLE_ENABLE": "true",
-                // "RUSTFS_OBS_TRACE_ENDPOINT": "http://127.0.0.1:4318/v1/traces", // jeager otlp http endpoint
-                // "RUSTFS_OBS_METRIC_ENDPOINT": "http://127.0.0.1:4318/v1/metrics", // default otlp http endpoint
-                // "RUSTFS_OBS_LOG_ENDPOINT": "http://127.0.0.1:4318/v1/logs", // default otlp http endpoint
-                // "RUSTFS_COMPRESS_ENABLE": "true",
-                "RUSTFS_CONSOLE_ADDRESS": "127.0.0.1:9001",
-                "RUSTFS_OBS_LOG_DIRECTORY": "./target/logs",
-                "RUST_LOG":"rustfs=debug,ecstore=debug,s3s=debug,iam=debug",
-            },
             "sourceLanguages": [
                 "rust"
             ],
         },
-        {
-            "type": "lldb",
-            "request": "launch",
-            "name": "Debug test_lifecycle_transition_basic",
-            "cargo": {
-                "args": [
-                    "test",
-                    "-p",
-                    "rustfs-scanner",
-                    "--test",
-                    "lifecycle_integration_test",
-                    "serial_tests::test_lifecycle_transition_basic",
-                    "-j",
-                    "1"
-                ]
-            },
-            "args": [],
-            "cwd": "${workspaceFolder}"
-        },
         {
             "name": "Debug executable target/debug/test",
             "type":  "lldb",
@@ -170,89 +108,6 @@
             "sourceLanguages": [
                 "rust"
             ],
-        },
-        {
-            "name": "Debug executable target/debug/rustfs with sse",
-            "type":  "lldb",
-            "request": "launch",
-            "program": "${workspaceFolder}/target/debug/rustfs",
-            "args": [],
-            "cwd": "${workspaceFolder}",
-            //"stopAtEntry": false,
-            //"preLaunchTask": "cargo build",
-            "env": {
-                "RUSTFS_ACCESS_KEY": "rustfsadmin",
-                "RUSTFS_SECRET_KEY": "rustfsadmin",
-                "RUSTFS_VOLUMES": "./target/volumes/test{1...4}",
-                "RUSTFS_ADDRESS": ":9000",
-                "RUSTFS_CONSOLE_ENABLE": "true",
-                "RUSTFS_CONSOLE_ADDRESS": "127.0.0.1:9001",
-                "RUSTFS_OBS_LOG_DIRECTORY": "./target/logs",
-                // "RUSTFS_OBS_TRACE_ENDPOINT": "http://127.0.0.1:4318/v1/traces", // jeager otlp http endpoint
-                // "RUSTFS_OBS_METRIC_ENDPOINT": "http://127.0.0.1:4318/v1/metrics", // default otlp http endpoint
-                // "RUSTFS_OBS_LOG_ENDPOINT": "http://127.0.0.1:4318/v1/logs", // default otlp http endpoint
-                // "RUSTFS_COMPRESS_ENABLE": "true",
-
-                // 1. simple sse test key (no kms system)
-                // "__RUSTFS_SSE_SIMPLE_CMK": "2dfNXGHlsEflGVCxb+5DIdGEl1sIvtwX+QfmYasi5QM=",
-
-                // 2. kms local backend test key
-                // "RUSTFS_KMS_ENABLE": "true",
-                // "RUSTFS_KMS_BACKEND": "local",
-                // "RUSTFS_KMS_KEY_DIR": "./target/kms-key-dir",
-                // "RUSTFS_KMS_LOCAL_MASTER_KEY": "my-secret-key", // Some Password
-                // "RUSTFS_KMS_DEFAULT_KEY_ID": "rustfs-master-key",
-
-                // 3. kms vault backend test key
-                // "RUSTFS_KMS_ENABLE": "true",
-                // "RUSTFS_KMS_BACKEND": "vault-kv2",
-                // "RUSTFS_KMS_VAULT_ADDRESS": "http://127.0.0.1:8200",
-                // "RUSTFS_KMS_VAULT_TOKEN": "Dev Token",
-                // "RUSTFS_KMS_DEFAULT_KEY_ID": "rustfs-master-key",
-
-                // 4. kms vault transit backend test key
-                "RUSTFS_KMS_ENABLE": "true",
-                "RUSTFS_KMS_BACKEND": "vault-transit",
-                "RUSTFS_KMS_VAULT_ADDRESS": "http://127.0.0.1:8200",
-                "RUSTFS_KMS_VAULT_TOKEN": "Dev Token",
-                "RUSTFS_KMS_VAULT_MOUNT_PATH": "transit",
-                "RUSTFS_KMS_DEFAULT_KEY_ID": "rustfs-master-key",
-
-            },
-            "sourceLanguages": [
-                "rust"
-            ],
-        },
-        {
-            "name": "E2E test executable target/debug/rustfs",
-            "type":  "lldb",
-            "request": "launch",
-            "program": "${workspaceFolder}/target/debug/rustfs",
-            "args": [],
-            "cwd": "${workspaceFolder}",
-            //"stopAtEntry": false,
-            //"preLaunchTask": "cargo build",
-            "env": {
-                "RUST_LOG": "rustfs=debug,ecstore=info,s3s=debug,iam=debug",
-                "RUST_BACKTRACE": "full",
-                "RUSTFS_ACCESS_KEY": "rustfsadmin",
-                "RUSTFS_SECRET_KEY": "rustfsadmin",
-                "RUSTFS_VOLUMES": "./target/e2e-test/test{1...4}",
-                "RUSTFS_REGION": "us-east-1",
-                "RUSTFS_ADDRESS": ":9000",
-                "RUSTFS_CONSOLE_ENABLE": "true",
-                "RUSTFS_CONSOLE_ADDRESS": "127.0.0.1:9001",
-                "RUSTFS_OBS_LOG_DIRECTORY": "./target/logs",
-
-                "RUSTFS_KMS_ENABLE": "true",
-                "RUSTFS_KMS_BACKEND": "local",
-                "RUSTFS_KMS_KEY_DIR": "./target/e2e-key-dir",
-                "RUSTFS_KMS_LOCAL_MASTER_KEY": "my-secret-key", // Some Password
-                "RUSTFS_KMS_DEFAULT_KEY_ID": "rustfs-master-key",
-            },
-            "sourceLanguages": [
-                "rust"
-            ],
-        },
+        }
     ]
 }

AGENTS.md

@@ -1,113 +1,24 @@
-# RustFS Agent Instructions (Global)
+# Repository Guidelines
 
-This root file keeps repository-wide rules only.
-Use the nearest subdirectory `AGENTS.md` for path-specific guidance.
+## Communication Rules
+- Respond to the user in Chinese; use English in all other contexts.
 
-## Rule Precedence
+## Project Structure & Module Organization
+The workspace root hosts shared dependencies in `Cargo.toml`. The service binary lives under `rustfs/src/main.rs`, while reusable crates sit in `crates/` (`crypto`, `iam`, `kms`, and `e2e_test`). Local fixtures for standalone flows reside in `test_standalone/`, deployment manifests are under `deploy/`, Docker assets sit at the root, and automation lives in `scripts/`. Skim each crate’s README or module docs before contributing changes.
 
-1. System/developer instructions.
-2. Current user/task instructions.
-3. The nearest `AGENTS.md` in the current path.
-4. This file (global defaults).
+## Build, Test, and Development Commands
+Run `cargo check --all-targets` for fast validation. Build release binaries via `cargo build --release` or the pipeline-aligned `make build`. Use `./build-rustfs.sh --dev` for iterative development and `./build-rustfs.sh --platform <target>` for cross-compiles. Prefer `make pre-commit` before pushing to cover formatting, clippy, checks, and tests.
+Always ensure `cargo fmt --all --check`, `cargo test --workspace --exclude e2e_test`, and `cargo clippy --all-targets --all-features -- -D warnings` complete successfully after each code change to keep the tree healthy and warning-free.
 
-If repo-level instructions conflict, follow the nearest file and keep behavior aligned with CI.
+## Coding Style & Naming Conventions
+Formatting follows the repo `rustfmt.toml` (130-column width). Use `snake_case` for items, `PascalCase` for types, and `SCREAMING_SNAKE_CASE` for constants. Avoid `unwrap()` or `expect()` outside tests; bubble errors with `Result` and crate-specific `thiserror` types. Keep async code non-blocking and offload CPU-heavy work with `tokio::task::spawn_blocking` when necessary.
 
-## Execution Discipline
+## Testing Guidelines
+Co-locate unit tests with their modules and give behavior-led names such as `handles_expired_token`. Integration suites belong in each crate’s `tests/` directory, while exhaustive end-to-end scenarios live in `crates/e2e_test/`. Run `cargo test --workspace --exclude e2e_test` during iteration, `cargo nextest run --all --exclude e2e_test` when available, and finish with `cargo test --all` before requesting review. Use `NO_PROXY=127.0.0.1,localhost HTTP_PROXY= HTTPS_PROXY=` for KMS e2e tests.
+When fixing bugs or adding features, include regression tests that capture the new behavior so future changes cannot silently break it.
 
-- Read the relevant existing code, tests, and local guidance before changing behavior.
-- State assumptions when they affect the implementation or verification path.
-- If a task has multiple plausible interpretations, list the options briefly and choose the narrowest reasonable path; ask when the ambiguity would make the change risky.
-- For multi-step work, keep the plan minimal and tied to verifiable outcomes.
-- Avoid redundant file reads, repeated commands, and unnecessary exploratory work once enough context is available.
-- A good result is a minimal diff with clear assumptions, no over-engineering, and independent verification.
+## Commit & Pull Request Guidelines
+Work on feature branches (e.g., `feat/...`) after syncing `main`. Follow Conventional Commits under 72 characters (e.g., `feat: add kms key rotation`). Each commit must compile, format cleanly, and pass `make pre-commit`. Open PRs with a concise summary, note verification commands, link relevant issues, and wait for reviewer approval.
 
-## Communication and Language
-
-- Respond in the same language used by the requester.
-- Keep source code, comments, commit messages, and PR title/body in English.
-- Be concise. Avoid sycophantic openers, closing fluff, and verbose status reporting.
-
-## Change Style for Existing Logic
-
-- Prefer direct, local code over extracting one-off helpers.
-- Extract a helper only when logic is reused or the extraction materially clarifies a non-trivial flow.
-- Solve only the requested problem; do not add speculative features, configurability, or adjacent improvements.
-- Prefer editing existing code over rewriting files or reshaping unrelated logic.
-- Modify only what is required and remove only artifacts introduced by your own changes.
-- Preserve the existing control-flow and logic shape when fixing bugs or addressing review comments, especially in init, distributed coordination, locking, metadata, and concurrency paths.
-- Do not refactor existing code only to make it easier to unit test.
-- Keep fixes narrowly aligned with the requested behavior; avoid semantic-adjacent rewrites while touching sensitive paths.
-- Keep code elegant, concise, and direct. Prefer minimal, readable implementations over over-engineering and excessive abstraction. Use comments to clarify non-obvious intent and invariants, not to compensate for unclear code.
-- Mention unrelated issues when useful, but do not fix them as part of a narrow task.
-
-## Constant and String Usage
-
-- Before introducing new string literals, search for existing constants/enums that already represent the same semantic value.
-- Reuse existing constants for protocol labels, error identifiers, header keys, event names, metric names, command tags, and similar fixed tokens.
-- If a new string is truly unique, define a local constant near related logic and avoid scattering the literal across multiple sites.
-- When changing existing behavior, keep naming and format consistency by aligning with established project constants.
-
-## Sources of Truth
-
-- Workspace layout and crate membership: `Cargo.toml` (`[workspace].members`)
-- Local quality commands: `Makefile` and `.config/make/`
-- CI quality gates: `.github/workflows/ci.yml`
-- PR template: `.github/pull_request_template.md`
-
-Avoid duplicating long crate lists or command matrices in instruction files.
-Reference the source files above instead.
-
-## Verification Before PR
-
-Convert changes into independently verifiable outcomes. Prefer focused tests for behavior changes and run the relevant checks before declaring completion.
-
-For code changes, run and pass the following before opening a PR:
-
-```bash
-make pre-commit
-```
-
-Before pushing code changes, make sure formatting is clean:
-
-- Run `cargo fmt --all`.
-- Run `cargo fmt --all --check` and ensure no files are modified unexpectedly.
-
-If `make` is unavailable, run the equivalent checks defined under `.config/make/`.
-Documentation-only or instruction-only changes are exempt from the verification commands above (including the `.config/make/` equivalents), though any installed git pre-commit hooks (for example, from `make setup-hooks`) may still run on commit unless explicitly skipped.
-After build-based verification completes, clean generated build artifacts before wrapping up to avoid unnecessary disk usage.
-Do not open a PR with code changes when the required checks fail.
-
-## Git and PR Baseline
-
-- Use feature branches based on the latest `main`.
-- Follow Conventional Commits, with subject length <= 72 characters.
-- Keep PR title and description in English.
-- Use `.github/pull_request_template.md` and keep all section headings.
-- Use `N/A` for non-applicable template sections.
-- Include verification commands in the PR description.
-- When using `gh pr create`/`gh pr edit`, use `--body-file` instead of inline `--body` for multiline markdown.
-- Do not include the literal sequence `\n` in any GitHub issue, pull request, or discussion comment.
-- After fixing code review comments or CI findings, always mark corresponding review
-  comments/threads as resolved before returning to the user.
-- In handling review comments, confirm the underlying issue before changing code.
-  If a suggested change is not appropriate for behavior or risk, reply with a
-  concise rationale instead of blindly applying it.
-
-## Security Baseline
-
-- Never commit secrets, credentials, or key material.
-- Use environment variables or vault tooling for sensitive configuration.
-- For localhost-sensitive tests, verify proxy settings to avoid traffic leakage.
-
-## Scoped Guidance in This Repository
-
-- `.github/AGENTS.md`
-- `crates/AGENTS.md`
-- `crates/config/AGENTS.md`
-- `crates/ecstore/AGENTS.md`
-- `crates/e2e_test/AGENTS.md`
-- `crates/iam/AGENTS.md`
-- `crates/kms/AGENTS.md`
-- `crates/policy/AGENTS.md`
-- `rustfs/src/admin/AGENTS.md`
-- `rustfs/src/storage/AGENTS.md`
+## Security & Configuration Tips
+Do not commit secrets or cloud credentials; prefer environment variables or vault tooling. Review IAM- and KMS-related changes with a second maintainer. Confirm proxy settings before running sensitive tests to avoid leaking traffic outside localhost.

ARCHITECTURE.md

@@ -1,376 +0,0 @@
-# ARCHITECTURE.md
-
-> Last updated: 2026-04-13 · Revision: 1 (draft)
->
-> This document describes the high-level architecture of RustFS.
-> If you want to familiarize yourself with the code base, you are in the right place!
->
-> See also [CONTRIBUTING.md](CONTRIBUTING.md) for development workflow.
-
-## Bird's Eye View
-
-RustFS is a high-performance, S3-compatible distributed object storage system written
-in Rust. It uses erasure coding for data durability, supports multi-tenancy through
-IAM/STS, and provides a web-based admin console.
-
-A running RustFS node exposes:
-
-- **S3 API** (port 9000) — the primary data path for object CRUD
-- **Admin API** (port 9000, `/minio/` prefix) — cluster management, IAM, metrics
-- **Console** (port 9001) — web UI backed by the Admin API
-- **Inter-node RPC** (gRPC/tonic) — cluster communication for distributed mode
-
-The core data flow for a PUT request looks like:
-
-```
-HTTP request
-  → server (TLS, auth, routing, compression)
-    → app/object_usecase (validation, policy, lifecycle)
-      → storage/ecfs (erasure coding, encryption, checksums)
-        → ecstore (disk pool selection, data distribution)
-          → rio (reader pipeline: encrypt → compress → hash → write)
-            → io-core (zero-copy I/O, buffer pool, direct I/O)
-              → local disk / remote disk via RPC
-```
-
-## Code Map
-
-The repository is a Cargo workspace with a flat `crates/` layout:
-
-```
-rustfs/                      # Workspace root (virtual manifest)
-├── rustfs/                  # Main binary + library crate (75K lines)
-│   └── src/
-│       ├── main.rs          # Entry point, startup sequence
-│       ├── lib.rs           # Module tree root
-│       ├── server/          # HTTP server, TLS, routing, middleware
-│       ├── admin/           # Admin API handlers and console
-│       ├── app/             # Use-case layer (object, bucket, multipart)
-│       ├── storage/         # Storage engine interface and implementation
-│       ├── auth.rs          # S3 request authentication
-│       ├── config/          # CLI args, config parsing, workload profiles
-│       └── ...
-├── crates/                  # 39 library crates
-│   ├── ecstore/             # Erasure-coded storage engine (⚠️ 87K lines)
-│   ├── rio/                 # Reader I/O pipeline (encrypt, compress, hash)
-│   ├── io-core/             # Zero-copy I/O, scheduling, buffer pool
-│   ├── io-metrics/          # I/O metrics collection
-│   ├── common/              # Shared runtime state, globals, data usage types
-│   ├── config/              # Configuration types and parsing
-│   ├── utils/               # Pure utility functions
-│   ├── ...                  # (see "Crate Reference" below)
-│   └── e2e_test/            # End-to-end integration tests
-└── docs/                    # Design documents and analysis
-```
-
-### Main Crate Layers (`rustfs/src/`)
-
-The main crate is organized in layers, top to bottom:
-
-| Layer | Directory | Responsibility |
-|-------|-----------|----------------|
-| **Server** | `server/` | HTTP listener, TLS, CORS, compression, middleware, graceful shutdown |
-| **Admin** | `admin/` | Admin API routing, 30+ handler modules, web console |
-| **App** | `app/` | Use-case orchestration: object_usecase, bucket_usecase, multipart_usecase |
-| **Storage** | `storage/` | S3 API translation, erasure-coded FS, SSE encryption, RPC, concurrency |
-| **Auth** | `auth.rs` | S3 signature verification, credential validation |
-| **Config** | `config/` | CLI parsing, config struct, workload profiles |
-
-A request flows **downward** through the layers. No layer should reach upward
-(e.g., storage must not import from admin).
-
-### Crate Reference
-
-Crates are organized in a dependency DAG with 9 depth levels (0 = leaf, 8 = top):
-
-```
-Depth 0 — LEAF (no internal deps):
-  appauth, checksums, config, credentials, crypto, io-metrics,
-  madmin, s3-common, workers, zip
-
-Depth 1:
-  io-core (→ io-metrics)
-  policy (→ config, credentials, crypto)
-  utils (→ config)                        ⚠️ inverted: utils should be leaf
-
-Depth 2:
-  concurrency, filemeta, keystone, kms, lock, obs,
-  signer, targets, trusted-proxies
-
-Depth 3:
-  common (→ filemeta, madmin)             ⚠️ inverted: common should be leaf
-
-Depth 4:
-  object-capacity, protos, rio
-
-Depth 5 — CORE:
-  ecstore (16 internal deps, 11 dependents — the architectural heart)
-
-Depth 6:
-  audit, heal, iam, metrics, notify, s3select-api, scanner
-
-Depth 7:
-  object-io, protocols, s3select-query
-
-Depth 8 — TOP:
-  rustfs (35 internal deps — the binary, depends on almost everything)
-```
-
-#### By Domain
-
-**Core Infrastructure:**
-
-| Crate | Lines | Purpose |
-|-------|-------|---------|
-| `config` | 3.3K | Configuration types and environment parsing |
-| `utils` | 8.7K | Pure utilities (paths, compression, network, retry) |
-| `common` | 4.4K | Shared runtime state, globals, data usage types, metrics |
-| `madmin` | 5.5K | Admin API request/response types |
-
-**I/O Pipeline:**
-
-| Crate | Lines | Purpose |
-|-------|-------|---------|
-| `io-core` | 6.5K | Zero-copy I/O, buffer pool, direct I/O, scheduling, backpressure |
-| `io-metrics` | 4.5K | I/O operation metrics and counters |
-| `rio` | 6.9K | Composable reader chain (encrypt → compress → hash → limit) |
-| `object-io` | 2.4K | High-level object read/write using rio + ecstore |
-| `concurrency` | 1.8K | Concurrency control wrappers over io-core |
-
-**Storage Engine:**
-
-| Crate | Lines | Purpose |
-|-------|-------|---------|
-| `ecstore` | 87K | ⚠️ Erasure-coded storage: disks, pools, buckets, replication, lifecycle |
-| `filemeta` | 10K | File/object metadata types and versioning |
-| `checksums` | 732 | Checksum computation |
-| `lock` | 7.1K | Distributed lock manager |
-| `heal` | 5.9K | Data healing / bitrot repair |
-| `scanner` | 5.4K | Background data usage scanner |
-| `object-capacity` | 2.5K | Capacity tracking and management |
-
-**Security & Auth:**
-
-| Crate | Lines | Purpose |
-|-------|-------|---------|
-| `crypto` | 1.6K | Encryption primitives |
-| `credentials` | 713 | Credential types (access key / secret key) |
-| `signer` | 1.4K | S3 v4 request signing |
-| `iam` | 9.0K | Identity and access management |
-| `policy` | 8.8K | Policy engine (S3 bucket/IAM policies) |
-| `kms` | 8.1K | Key management service integration |
-| `keystone` | 1.9K | OpenStack Keystone auth |
-| `appauth` | 143 | Application-level auth tokens |
-
-**Protocol & API:**
-
-| Crate | Lines | Purpose |
-|-------|-------|---------|
-| `protos` | 5.7K | Protobuf/gRPC definitions for inter-node RPC |
-| `protocols` | 18K | FTP/FTPS, WebDAV, Swift API support |
-| `s3-common` | 738 | Shared S3 types |
-| `s3select-api` | 1.9K | S3 Select interface |
-| `s3select-query` | 3.6K | S3 Select query engine |
-
-**Observability:**
-
-| Crate | Lines | Purpose |
-|-------|-------|---------|
-| `metrics` | 8.4K | Prometheus metric collectors |
-| `io-metrics` | 4.5K | I/O-specific metrics |
-| `obs` | 5.6K | OpenTelemetry tracing and telemetry |
-| `audit` | 2.4K | Audit logging |
-
-**Events:**
-
-| Crate | Lines | Purpose |
-|-------|-------|---------|
-| `notify` | 5.5K | Event notification system |
-| `targets` | 3.2K | Notification targets (Kafka, AMQP, webhook, etc.) |
-
-**Other:**
-
-| Crate | Lines | Purpose |
-|-------|-------|---------|
-| `trusted-proxies` | 4.0K | Trusted proxy / IP forwarding |
-| `zip` | 986 | ZIP archive support for bulk downloads |
-| `workers` | 136 | Simple worker abstraction |
-
-## Architecture Invariants
-
-> These are rules that the codebase should follow. Some are currently violated
-> (marked with ⚠️). Documenting them here makes the violations explicit and
-> trackable.
-
-1. **Layers flow downward.** Server → Admin/App → Storage → ecstore → rio/io-core.
-   No upward imports.
-
-2. **Leaf crates have zero internal dependencies.** `config`, `credentials`, `crypto`,
-   `io-metrics`, `madmin`, `s3-common` should depend only on external crates.
-   - ⚠️ VIOLATED: `utils` depends on `config`, `common` depends on `filemeta` and `madmin`.
-
-3. **Each type has exactly one definition.** Types shared across crates must be defined
-   in one crate and re-exported or imported by others.
-   - ⚠️ VIOLATED: `ReplicationStats` (4 copies), `LastMinuteLatency` (3 copies),
-     `BackpressureConfig` (3 copies), `DataUsageInfo` (2 copies).
-
-4. **ecstore does not know about HTTP or S3 protocol details.** It operates on
-   storage-level abstractions (objects, buckets, disks, pools).
-
-5. **The `rustfs` binary crate is the only place that wires everything together.**
-   Individual crates should be testable in isolation.
-
-6. **Error types use `thiserror` with descriptive names** (e.g., `StorageError`,
-   not bare `Error`).
-   - ⚠️ VIOLATED: 6 crates use `pub enum Error`; 2 crates use `snafu`;
-      `heal` use `anyhow` in library code.
-
-## Known Structural Issues
-
-> This section documents known problems in the current architecture.
-> It exists so the team can track and address them deliberately.
-
-### Critical
-
-- **common/scanner code duplication (~3K lines).** `scanner` depends on `common`
-  but maintains its own copies of `DataUsageInfo`, `LastMinuteLatency`, and related
-  types instead of importing them.
-
-- **ecstore is a monolith (87K lines, 163 files).** It contains disk management,
-  bucket management, erasure coding, replication, lifecycle, RPC, and configuration
-  — all in one crate. It should be decomposed along its existing subdirectories.
-
-### High
-
-- **Dependency inversions.** `utils → config` and `common → filemeta/madmin` break
-  the layering model. These need to be untangled.
-
-- **Three-layer BackpressureConfig/DeadlockConfig duplication** across io-core,
-  concurrency, and rustfs/storage. Should be defined once with builder/composition.
-
-### Medium
-
-- **Inconsistent error handling.** Three strategies (thiserror/snafu/anyhow) and
-  mixed naming (bare `Error` vs descriptive names).
-
-- **Ambiguous common vs utils boundary.** Both described as "utilities and data
-  structures." Need clear ownership rules.
-
-## Cross-Cutting Concerns
-
-### Error Handling
-
-The project convention is `thiserror` for typed errors with descriptive names.
-See `AGENTS.md`: "Prefer thiserror for library-facing error types."
-
-```rust
-// GOOD
-#[derive(Debug, thiserror::Error)]
-pub enum StorageError {
-    #[error("disk not found: {0}")]
-    DiskNotFound(String),
-}
-
-// AVOID
-pub enum Error { ... }        // too generic
-anyhow::Result<T>             // in library code (OK in tests/CLI)
-```
-
-### Logging & Tracing
-
-- Use `tracing` crate (`info!`, `warn!`, `error!`, `debug!`, `trace!`)
-- Structured fields: `tracing::info!(bucket = %name, "created bucket")`
-- Spans for request-scoped context
-
-### Metrics
-
-- Prometheus-style metrics via `rustfs-obs` runtime and schema
-- I/O-specific counters via `rustfs-io-metrics`
-- Registration happens at crate level, collection/reporting in `rustfs-obs`
-
-### Testing
-
-- Unit tests: `#[cfg(test)] mod tests` in the same file
-- Integration tests: inside respective crates (not top-level `tests/`)
-- E2E tests: `crates/e2e_test/` — tests against a running server
-- Run all: `make test` or `cargo nextest run`
-
-## Startup Sequence
-
-The binary (`main.rs`) boots in this order:
-
-1. Environment variable compatibility (`MINIO_*` → `RUSTFS_*`)
-2. Tokio runtime construction
-3. CLI argument parsing
-4. License, observability, TLS, trusted proxies initialization
-5. Config parsing, server address resolution
-6. Credentials, endpoints, local disks, lock client initialization
-7. Capacity management initialization
-8. HTTP server start (S3 API + optional console)
-9. ECStore initialization (erasure coding storage engine)
-10. Global config, background replication, KMS
-11. Optional: FTP/FTPS/WebDAV servers
-12. Event notifier, audit system, deadlock detector
-13. Bucket metadata, IAM, Keystone, OIDC
-14. Scanner and heal manager
-15. Metrics system, mark `FullReady`
-16. Wait for shutdown signal → graceful shutdown
-
-## Dependency Diagram (Simplified)
-
-```
-                            ┌─────────┐
-                            │  rustfs │  (binary + lib, 75K lines)
-                            │  main   │
-                            └────┬────┘
-                                 │
-                 ┌───────────────┼───────────────┐
-                 │               │               │
-            ┌────▼────┐    ┌────▼────┐    ┌─────▼─────┐
-            │ server  │    │  admin  │    │    app     │
-            │ (HTTP)  │    │(console)│    │(use-cases) │
-            └────┬────┘    └────┬────┘    └─────┬─────┘
-                 │               │               │
-                 └───────────────┼───────────────┘
-                                 │
-                          ┌──────▼──────┐
-                          │   storage   │
-                          │ (ecfs, SSE, │
-                          │  RPC, ACL)  │
-                          └──────┬──────┘
-                                 │
-              ┌──────────────────┼──────────────────┐
-              │                  │                  │
-        ┌─────▼─────┐    ┌──────▼──────┐    ┌──────▼──────┐
-        │  ecstore   │    │     rio     │    │   io-core   │
-        │ (87K,core) │    │  (readers)  │    │ (zero-copy) │
-        └─────┬──────┘    └─────────────┘    └─────────────┘
-              │
-    ┌─────┬──┼──┬─────┬──────┐
-    │     │  │  │     │      │
- common utils config policy filemeta ...
-```
-
-## How to Navigate
-
-- **"Where does S3 PutObject go?"**
-  `server/` routes → `app/object_usecase` validates → `storage/ecfs` encodes →
-  `ecstore` distributes → `rio` encrypts/compresses → `io-core` writes
-
-- **"Where are bucket policies enforced?"**
-  `app/bucket_usecase` calls into `crates/policy/`
-
-- **"Where is replication configured?"**
-  `admin/handlers/replication.rs` and `admin/handlers/site_replication.rs` for API,
-  `ecstore/src/bucket/replication/` for engine
-
-- **"Where do I add a new admin endpoint?"**
-  Add handler in `admin/handlers/`, register in `admin/router.rs`
-
-- **"Where do I add a new metric?"**
-  Define descriptor/collector in `crates/obs/src/metrics/`, expose via `/minio/v2/metrics`
-
----
-
-*Inspired by [matklad's ARCHITECTURE.md](https://matklad.github.io/2021/02/06/ARCHITECTURE.md.html)
-and [rust-analyzer's architecture.md](https://github.com/rust-analyzer/rust-analyzer/blob/master/docs/book/src/contributing/architecture.md).*

CHANGELOG.md

@@ -1,79 +0,0 @@
-# Changelog
-
-All notable changes to this project will be documented in this file.
-
-The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
-and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
-
-## [Unreleased]
-
-### Fixed
-- **Helm Ingress**: `customAnnotations` are now merged with class-specific annotations (nginx/traefik) instead of being ignored when `ingress.className` is set.
-
-### Added
-- **OpenStack Keystone Authentication Integration**: Full support for OpenStack Keystone authentication via X-Auth-Token headers
-  - Tower-based middleware (`KeystoneAuthLayer`) self-contained within `rustfs-keystone` crate
-  - Task-local storage for async-safe credential passing between middleware and auth handlers
-  - Automatic detection of Keystone credentials (access keys prefixed with `keystone:`)
-  - Role-based permission mapping (admin/reseller_admin roles grant owner permissions)
-  - Token caching for high-performance validation with configurable cache size and TTL
-  - Dual authentication support: Keystone and standard AWS Signature v4 work simultaneously
-  - Immediate 401 response for invalid tokens (no fallback to local auth)
-  - XML-formatted error responses compatible with S3 API
-  - Comprehensive integration documentation with manual testing guide
-  - **32 unit and integration tests** covering middleware, auth handlers, task-local storage, and role detection
-
-### Changed
-- **HTTP Server Stack**: Integrated `KeystoneAuthLayer` middleware from `rustfs-keystone` crate into service stack (positioned after ReadinessGateLayer)
-- **IAMAuth**: Enhanced `get_secret_key()` to return empty secret for Keystone credentials (bypasses signature validation)
-- **Auth Module**: Modified `check_key_valid()` to retrieve Keystone credentials from task-local storage and determine admin status
-
-### Technical Details
-- Middleware is self-contained in `rustfs-keystone` crate following the trusted-proxies pattern for integration-specific middleware
-- Uses `BoxBody` pattern for Hyper 1.x compatibility
-- Task-local storage provides request-scoped credential passing without modifying HTTP request/response types
-- Integration preserves existing S3 authentication flow while adding Keystone support
-- Zero breaking changes to existing functionality
-- No new top-level directories in main binary crate (middleware lives in integration crate)
-
-### Documentation
-- Updated `crates/keystone/README.md` with complete integration architecture and workflow
-- Added detailed manual testing guide with 10 test scenarios
-- Updated main `README.md` to list Keystone authentication as available feature
-- Added troubleshooting section for common integration issues
-
-### Configuration
-New environment variables:
-- `RUSTFS_KEYSTONE_ENABLE` - Enable/disable Keystone authentication (default: false)
-- `RUSTFS_KEYSTONE_AUTH_URL` - Keystone API endpoint URL
-- `RUSTFS_KEYSTONE_VERSION` - Keystone API version (v3)
-- `RUSTFS_KEYSTONE_ADMIN_USER` - Admin username for privileged operations
-- `RUSTFS_KEYSTONE_ADMIN_PASSWORD` - Admin password
-- `RUSTFS_KEYSTONE_ADMIN_PROJECT` - Admin project name
-- `RUSTFS_KEYSTONE_ADMIN_DOMAIN` - Admin domain name (default: Default)
-- `RUSTFS_KEYSTONE_CACHE_SIZE` - Token cache size (default: 10000)
-- `RUSTFS_KEYSTONE_CACHE_TTL` - Token cache TTL in seconds (default: 300)
-- `RUSTFS_KEYSTONE_VERIFY_SSL` - Verify SSL certificates (default: true)
-
-### Files Modified
-- `crates/keystone/src/middleware.rs` - Created Keystone authentication middleware (self-contained in keystone crate)
-- `crates/keystone/src/lib.rs` - Exported middleware module and KEYSTONE_CREDENTIALS
-- `crates/keystone/Cargo.toml` - Added Tower/HTTP dependencies for middleware functionality
-- `rustfs/src/server/http.rs` - Integrated KeystoneAuthLayer from rustfs-keystone crate
-- `rustfs/src/auth.rs` - Enhanced IAMAuth and check_key_valid for Keystone support, imported KEYSTONE_CREDENTIALS from rustfs-keystone
-- `crates/keystone/README.md` - Comprehensive integration documentation
-- `README.md` - Added Keystone as available feature
-
-### Testing
-- 16 unit tests in rustfs-keystone crate (config, auth, middleware, identity)
-- 10 integration tests in rustfs-keystone crate (task-local storage, middleware layer, scope isolation)
-- 6 auth unit tests in rustfs crate (role detection, task-local storage, Keystone credential handling)
-- **Total: 32 tests** passing with zero compilation errors
-- Manual testing guide provided for end-to-end validation
-- All tests passing with `cargo test --all --exclude e2e_test`
-
----
-
-## Previous Releases
-
-See [GitHub Releases](https://github.com/rustfs/rustfs/releases) for previous version history.

CLA.md

@@ -1,60 +1,88 @@
-# RustFS Individual Contributor License Agreement
+RustFS Individual Contributor License Agreement
 
-Thank you for your interest in contributing to RustFS. In order to clarify the intellectual property license granted with Contributions from any person or entity, RustFS, Inc. ("RustFS") must have a Contributor License Agreement ("CLA") on file that has been signed by each Contributor, indicating agreement to the license terms below. This license is for your protection as a Contributor as well as the protection of RustFS and its users; it does not change your rights to use your own Contributions for any other purpose.
+Thank you for your interest in contributing documentation and related software code to a project hosted or managed by
+RustFS. In order to clarify the intellectual property license granted with Contributions from any person or entity,
+RustFS must have a Contributor License Agreement ("CLA") on file that has been signed by each Contributor, indicating
+agreement to the license terms below. This version of the Contributor License Agreement allows an individual to submit
+Contributions to the applicable project. If you are making a submission on behalf of a legal entity, then you should
+sign the separate Corporate Contributor License Agreement.
 
-You accept and agree to the following terms and conditions for Your present and future Contributions submitted to RustFS. Except for the license granted herein to RustFS and recipients of software distributed by RustFS, You reserve all right, title, and interest in and to Your Contributions.
+You accept and agree to the following terms and conditions for Your present and future Contributions submitted to
+RustFS. You hereby irrevocably assign and transfer to RustFS all right, title, and interest in and to Your
+Contributions, including all copyrights and other intellectual property rights therein.
 
-**You understand and agree that You will not receive any royalty or other compensation for Your Contributions.**
+Definitions
 
-## 1. Definitions
+“You” (or “Your”) shall mean the copyright owner or legal entity authorized by the copyright owner that is making this
+Agreement with RustFS. For legal entities, the entity making a Contribution and all other entities that control, are
+controlled by, or are under common control with that entity are considered to be a single Contributor. For the purposes
+of this definition, "control" means (i) the power, direct or indirect, to cause the direction or management of such
+entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares,
+or (iii) beneficial ownership of such entity.
 
-*   **"You"** (or **"Your"**) shall mean the copyright owner or legal entity authorized by the copyright owner that is making this Agreement with RustFS. For legal entities, the entity making a Contribution and all other entities that control, are controlled by, or are under common control with that entity are considered to be a single Contributor. For the purposes of this definition, "control" means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity.
+“Contribution” shall mean any original work of authorship, including any modifications or additions to an existing work,
+that is intentionally submitted by You to RustFS for inclusion in, or documentation of, any of the products or projects
+owned or managed by RustFS (the "Work"), including without limitation any Work described in Schedule A. For the purposes
+of this definition, "submitted" means any form of electronic or written communication sent to RustFS or its
+representatives, including but not limited to communication on electronic mailing lists, source code control systems,
+and issue tracking systems that are managed by, or on behalf of, RustFS for the purpose of discussing and improving the
+Work.
 
-*   **"Contribution"** shall mean any original work of authorship, including any modifications or additions to an existing work, that is intentionally submitted by You to RustFS for inclusion in, or documentation of, any of the products or projects owned or managed by RustFS (the "Work"). For the purposes of this definition, "submitted" means any form of electronic or written communication sent to RustFS or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, RustFS for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by You as "Not a Contribution."
+Assignment of Copyright
 
-## 2. Grant of Copyright License
+Subject to the terms and conditions of this Agreement, You hereby irrevocably assign and transfer to RustFS all right,
+title, and interest in and to Your Contributions, including all copyrights and other intellectual property rights
+therein, for the entire term of such rights, including all renewals and extensions. You agree to execute all documents
+and take all actions as may be reasonably necessary to vest in RustFS the ownership of Your Contributions and to assist
+RustFS in perfecting, maintaining, and enforcing its rights in Your Contributions.
 
-Subject to the terms and conditions of this Agreement, You hereby grant to RustFS and to recipients of software distributed by RustFS a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare derivative works of, publicly display, publicly perform, sublicense, and distribute Your Contributions and such derivative works under any license, including proprietary or commercial licenses and open-source licenses, at RustFS's sole discretion.
+Grant of Patent License
 
-## 3. Grant of Patent License
+Subject to the terms and conditions of this Agreement, You hereby grant to RustFS and to recipients of documentation and
+software distributed by RustFS a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as
+stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the
+Work, where such license applies only to those patent claims licensable by You that are necessarily infringed by Your
+Contribution(s) alone or by combination of Your Contribution(s) with the Work to which such Contribution(s) was
+submitted. If any entity institutes patent litigation against You or any other entity (including a cross-claim or
+counterclaim in a lawsuit) alleging that your Contribution, or the Work to which you have contributed, constitutes
+direct or contributory patent infringement, then any patent licenses granted to that entity under this Agreement for
+that Contribution or Work shall terminate as of the date such litigation is filed.
 
-Subject to the terms and conditions of this Agreement, You hereby grant to RustFS and to recipients of software distributed by RustFS a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by You that are necessarily infringed by Your Contribution(s) alone or by combination of Your Contribution(s) with the Work to which such Contribution(s) was submitted. If any entity institutes patent litigation against You or any other entity (including a cross-claim or counterclaim in a lawsuit) alleging that your Contribution, or the Work to which you have contributed, constitutes direct or contributory patent infringement, then any patent licenses granted to that entity under this Agreement for that Contribution or Work shall terminate as of the date such litigation is filed.
+You represent that you are legally entitled to grant the above assignment and license.
 
-## 4. Representations
+You represent that each of Your Contributions is Your original creation (see section 7 for submissions on behalf of
+others). You represent that Your Contribution submissions include complete details of any third-party license or other
+restriction (including, but not limited to, related patents and trademarks) of which you are personally aware and which
+are associated with any part of Your Contributions.
 
-You represent that you are legally entitled to grant the above license. If your employer(s) has rights to intellectual property that you create that includes your Contributions, you represent that you have received permission to make Contributions on behalf of that employer, that your employer has waived such rights for your Contributions to RustFS, or that your employer has executed a separate Corporate CLA with RustFS.
+You are not expected to provide support for Your Contributions, except to the extent You desire to provide support. You
+may provide support for free, for a fee, or not at all. Unless required by applicable law or agreed to in writing, You
+provide Your Contributions on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied,
+including, without limitation, any warranties or conditions of TITLE, NON- INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR
+A PARTICULAR PURPOSE.
 
-You represent that each of Your Contributions is Your original creation. You represent that Your Contribution submissions include complete details of any third-party license or other restriction (including, but not limited to, related patents and trademarks) of which you are personally aware and which are associated with any part of Your Contributions.
+Should You wish to submit work that is not Your original creation, You may submit it to RustFS separately from any
+Contribution, identifying the complete details of its source and of any license or other restriction (including, but not
+limited to, related patents, trademarks, and license agreements) of which you are personally aware, and conspicuously
+marking the work as "Submitted on behalf of a third-party: [named here]”.
 
-## 5. Support and Warranty
+You agree to notify RustFS of any facts or circumstances of which you become aware that would make these representations
+inaccurate in any respect.
 
-You are not expected to provide support for Your Contributions, except to the extent You desire to provide support. You may provide support for free, for a fee, or not at all. Unless required by applicable law or agreed to in writing, You provide Your Contributions on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of **TITLE**, **NON-INFRINGEMENT**, **MERCHANTABILITY**, or **FITNESS FOR A PARTICULAR PURPOSE**.
+Modification of CLA
 
-## 6. Third-Party Work
+RustFS reserves the right to update or modify this CLA in the future. Any updates or modifications to this CLA shall
+apply only to Contributions made after the effective date of the revised CLA. Contributions made prior to the update
+shall remain governed by the version of the CLA that was in effect at the time of submission. It is not necessary for
+all Contributors to re-sign the CLA when the CLA is updated or modified.
 
-Should You wish to submit work that is not Your original creation, You may submit it to RustFS separately from any Contribution, identifying the complete details of its source and of any license or other restriction (including, but not limited to, related patents, trademarks, and license agreements) of which you are personally aware, and conspicuously marking the work as "Submitted on behalf of a third-party: [named here]".
+Governing Law and Dispute Resolution
 
-## 7. Governing Law and Jurisdiction
+This Agreement will be governed by and construed in accordance with the laws of the People's Republic of China excluding
+that body of laws known as conflict of laws. The parties expressly agree that the United Nations Convention on Contracts
+for the International Sale of Goods will not apply. Any legal action or proceeding arising under this Agreement will be
+brought exclusively in the courts located in Beijing, China, and the parties hereby irrevocably consent to the personal
+jurisdiction and venue therein.
 
-This Agreement shall be governed by and construed in accordance with the laws of the State of Delaware, United States of America, without regard to its conflict of laws principles. The parties expressly agree that the United Nations Convention on Contracts for the International Sale of Goods will not apply. Any legal action or proceeding arising under or in connection with this Agreement shall be brought exclusively in the state or federal courts located in the State of Delaware, United States of America, and the parties hereby irrevocably consent to the personal jurisdiction and venue therein.
-
-## 8. Severability
-
-If any provision of this Agreement is found to be invalid or unenforceable, the remaining provisions will continue in full force and effect.
-
-## 9. Entire Agreement and Assignment
-
-This Agreement constitutes the entire agreement between the parties with respect to the subject matter hereof and supersedes all prior agreements, understandings, negotiations, and discussions, whether written or oral. RustFS may assign its rights and obligations under this Agreement to any third party. You may not assign Your rights or obligations under this Agreement without the prior written consent of RustFS.
-
----
-
-**Please read the terms of this Agreement carefully. By submitting a Contribution to RustFS, You agree to be bound by the terms and conditions of this Agreement.**
-
-| | |
-|---|---|
-| **Full Name** | __________________________________ |
-| **GitHub Username** | __________________________________ |
-| **Email Address** | __________________________________ |
-| **Date** | __________________________________ |
-
-*(Electronic signature or acknowledgement via GitHub commit/Pull Request constitutes valid acceptance of this Agreement).*
+For your reading convenience, this Agreement is written in parallel English and Chinese sections. To the extent there is
+a conflict between the English and Chinese sections, the English sections shall govern.

CLAUDE.md

@@ -0,0 +1,275 @@
+# CLAUDE.md
+
+This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
+
+## Project Overview
+
+RustFS is a high-performance distributed object storage software built with Rust, providing S3-compatible APIs and
+advanced features like data lakes, AI, and big data support. It's designed as an alternative to MinIO with better
+performance and a more business-friendly Apache 2.0 license.
+
+## Build Commands
+
+### Primary Build Commands
+
+- `cargo build --release` - Build the main RustFS binary
+- `./build-rustfs.sh` - Recommended build script that handles console resources and cross-platform compilation
+- `./build-rustfs.sh --dev` - Development build with debug symbols
+- `make build` or `just build` - Use Make/Just for standardized builds
+
+### Platform-Specific Builds
+
+- `./build-rustfs.sh --platform x86_64-unknown-linux-musl` - Build for musl target
+- `./build-rustfs.sh --platform aarch64-unknown-linux-gnu` - Build for ARM64
+- `make build-musl` or `just build-musl` - Build musl variant
+- `make build-cross-all` - Build all supported architectures
+
+### Testing Commands
+
+- `cargo test --workspace --exclude e2e_test` - Run unit tests (excluding e2e tests)
+- `cargo nextest run --all --exclude e2e_test` - Use nextest if available (faster)
+- `cargo test --all --doc` - Run documentation tests
+- `make test` or `just test` - Run full test suite
+- `make pre-commit` - Run all quality checks (fmt, clippy, check, test)
+
+### End-to-End Testing
+
+- `cargo test --package e2e_test` - Run all e2e tests
+- `./scripts/run_e2e_tests.sh` - Run e2e tests via script
+- `./scripts/run_scanner_benchmarks.sh` - Run scanner performance benchmarks
+
+### KMS-Specific Testing (with proxy bypass)
+
+-
+`NO_PROXY=127.0.0.1,localhost HTTP_PROXY= HTTPS_PROXY= http_proxy= https_proxy= cargo test --package e2e_test test_local_kms_end_to_end -- --nocapture --test-threads=1` -
+Run complete KMS end-to-end test
+-
+`NO_PROXY=127.0.0.1,localhost HTTP_PROXY= HTTPS_PROXY= http_proxy= https_proxy= cargo test --package e2e_test kms:: -- --nocapture --test-threads=1` -
+Run all KMS tests
+- `cargo test --package e2e_test test_local_kms_key_isolation -- --nocapture --test-threads=1` - Test KMS key isolation
+- `cargo test --package e2e_test test_local_kms_large_file -- --nocapture --test-threads=1` - Test KMS with large files
+
+### Code Quality
+
+- `cargo fmt --all` - Format code
+- `cargo clippy --all-targets --all-features -- -D warnings` - Lint code
+- `make pre-commit` or `just pre-commit` - Run all quality checks (fmt, clippy, check, test)
+
+### Quick Development Commands
+
+- `make help` or `just help` - Show all available commands with descriptions
+- `make help-build` - Show detailed build options and cross-compilation help
+- `make help-docker` - Show comprehensive Docker build and deployment options
+- `./scripts/dev_deploy.sh <IP>` - Deploy development build to remote server
+- `./scripts/run.sh` - Start local development server
+- `./scripts/probe.sh` - Health check and connectivity testing
+
+### Docker Build Commands
+
+- `make docker-buildx` - Build multi-architecture production images
+- `make docker-dev-local` - Build development image for local use
+- `./docker-buildx.sh --push` - Build and push production images
+
+## Architecture Overview
+
+### Core Components
+
+**Main Binary (`rustfs/`):**
+
+- Entry point at `rustfs/src/main.rs`
+- Core modules: admin, auth, config, server, storage, license management, profiling
+- HTTP server with S3-compatible APIs
+- Service state management and graceful shutdown
+- Parallel service initialization with DNS resolver, bucket metadata, and IAM
+
+**Key Crates (`crates/`):**
+
+- `ecstore` - Erasure coding storage implementation (core storage layer)
+- `iam` - Identity and Access Management
+- `kms` - Key Management Service for encryption and key handling
+- `madmin` - Management dashboard and admin API interface
+- `s3select-api` & `s3select-query` - S3 Select API and query engine
+- `config` - Configuration management with notify features
+- `crypto` - Cryptography and security features
+- `lock` - Distributed locking implementation
+- `filemeta` - File metadata management
+- `rio` - Rust I/O utilities and abstractions
+- `common` - Shared utilities and data structures
+- `protos` - Protocol buffer definitions
+- `audit-logger` - Audit logging for file operations
+- `notify` - Event notification system
+- `obs` - Observability utilities
+- `workers` - Worker thread pools and task scheduling
+- `appauth` - Application authentication and authorization
+- `ahm` - Asynchronous Hash Map for concurrent data structures
+- `mcp` - MCP server for S3 operations
+- `signer` - Client request signing utilities
+- `checksums` - Client checksum calculation utilities
+- `utils` - General utility functions and helpers
+- `zip` - ZIP file handling and compression
+- `targets` - Target-specific configurations and utilities
+
+### Build System
+
+- Cargo workspace with 25+ crates (including new KMS functionality)
+- Custom `build-rustfs.sh` script for advanced build options
+- Multi-architecture Docker builds via `docker-buildx.sh`
+- Both Make and Just task runners supported with comprehensive help
+- Cross-compilation support for multiple Linux targets
+- Automated CI/CD with GitHub Actions for testing, building, and Docker publishing
+- Performance benchmarking and audit workflows
+
+### Key Dependencies
+
+- `axum` - HTTP framework for S3 API server
+- `tokio` - Async runtime
+- `s3s` - S3 protocol implementation library
+- `datafusion` - For S3 Select query processing
+- `hyper`/`hyper-util` - HTTP client/server utilities
+- `rustls` - TLS implementation
+- `serde`/`serde_json` - Serialization
+- `tracing` - Structured logging and observability
+- `pprof` - Performance profiling with flamegraph support
+- `tikv-jemallocator` - Memory allocator for Linux GNU builds
+
+### Development Workflow
+
+- Console resources are embedded during build via `rust-embed`
+- Protocol buffers generated via custom `gproto` binary
+- E2E tests in separate crate (`e2e_test`) with comprehensive KMS testing
+- Shadow build for version/metadata embedding
+- Support for both GNU and musl libc targets
+- Development scripts in `scripts/` directory for common tasks
+- Git hooks setup available via `make setup-hooks` or `just setup-hooks`
+
+### Performance & Observability
+
+- Performance profiling available with `pprof` integration (disabled on Windows)
+- Profiling enabled via environment variables in production
+- Built-in observability with OpenTelemetry integration
+- Background services (scanner, heal) can be controlled via environment variables:
+    - `RUSTFS_ENABLE_SCANNER` (default: true)
+    - `RUSTFS_ENABLE_HEAL` (default: true)
+
+### Service Architecture
+
+- Service state management with graceful shutdown handling
+- Parallel initialization of core systems (DNS, bucket metadata, IAM)
+- Event notification system with MQTT and webhook support
+- Auto-heal and data scanner for storage integrity
+- Jemalloc allocator for Linux GNU targets for better performance
+
+## Environment Variables
+
+- `RUSTFS_ENABLE_SCANNER` - Enable/disable background data scanner (default: true)
+- `RUSTFS_ENABLE_HEAL` - Enable/disable auto-heal functionality (default: true)
+- Various profiling and observability controls
+- Build-time variables for Docker builds (RELEASE, REGISTRY, etc.)
+- Test environment configurations in `scripts/dev_rustfs.env`
+
+### KMS Environment Variables
+
+- `NO_PROXY=127.0.0.1,localhost` - Required for KMS E2E tests to bypass proxy
+- `HTTP_PROXY=` `HTTPS_PROXY=` `http_proxy=` `https_proxy=` - Clear proxy settings for local KMS testing
+
+## KMS (Key Management Service) Architecture
+
+### KMS Implementation Status
+
+- **Full KMS Integration:** Complete implementation with Local and Vault backends
+- **Automatic Configuration:** KMS auto-configures on startup with `--kms-enable` flag
+- **Encryption Support:** Full S3-compatible server-side encryption (SSE-S3, SSE-KMS, SSE-C)
+- **Admin API:** Complete KMS management via HTTP admin endpoints
+- **Production Ready:** Comprehensive testing including large files and key isolation
+
+### KMS Configuration
+
+- **Local Backend:** `--kms-backend local --kms-key-dir <path> --kms-default-key-id <id>`
+- **Vault Backend:** `--kms-backend vault --kms-vault-endpoint <url> --kms-vault-key-name <name>`
+- **Auto-startup:** KMS automatically initializes when `--kms-enable` is provided
+- **Manual Configuration:** Also supports dynamic configuration via admin API
+
+### S3 Encryption Support
+
+- **SSE-S3:** Server-side encryption with S3-managed keys (`ServerSideEncryption: AES256`)
+- **SSE-KMS:** Server-side encryption with KMS-managed keys (`ServerSideEncryption: aws:kms`)
+- **SSE-C:** Server-side encryption with customer-provided keys
+- **Response Headers:** All encryption types return correct `server_side_encryption` headers in PUT/GET responses
+
+### KMS Testing Architecture
+
+- **Comprehensive E2E Tests:** Located in `crates/e2e_test/src/kms/`
+- **Test Environments:** Automated test environment setup with temporary directories
+- **Encryption Coverage:** Tests all three encryption types (SSE-S3, SSE-KMS, SSE-C)
+- **API Coverage:** Tests all KMS admin APIs (CreateKey, DescribeKey, ListKeys, etc.)
+- **Edge Cases:** Key isolation, large file handling, error scenarios
+
+### Key Files for KMS
+
+- `crates/kms/` - Core KMS implementation with Local/Vault backends
+- `rustfs/src/main.rs` - KMS auto-initialization in `init_kms_system()`
+- `rustfs/src/storage/ecfs.rs` - SSE encryption/decryption in PUT/GET operations
+- `rustfs/src/admin/handlers/kms*.rs` - KMS admin endpoints
+- `crates/e2e_test/src/kms/` - Comprehensive KMS test suite
+- `crates/rio/src/encrypt_reader.rs` - Streaming encryption for large files
+
+## Code Style and Safety Requirements
+
+- **Language Requirements:**
+    - Communicate with me in Chinese, but **only English can be used in code files**
+    - Code comments, function names, variable names, and all text in source files must be in English only
+    - No Chinese characters, emojis, or non-ASCII characters are allowed in any source code files
+    - This includes comments, strings, documentation, and any other text within code files
+- **Safety-Critical Rules:**
+    - `unsafe_code = "deny"` enforced at workspace level
+    - Never use `unwrap()`, `expect()`, or panic-inducing code except in tests
+    - Avoid blocking I/O operations in async contexts
+    - Use proper error handling with `Result<T, E>` and `Option<T>`
+    - Follow Rust's ownership and borrowing rules strictly
+- **Performance Guidelines:**
+    - Use `cargo clippy --all-targets --all-features -- -D warnings` to catch issues
+    - Prefer `anyhow` for error handling in applications, `thiserror` for libraries
+    - Use appropriate async runtimes and avoid blocking calls
+- **Testing Standards:**
+    - All new features must include comprehensive tests
+    - Use `#[cfg(test)]` for test-only code that may use panic macros
+    - E2E tests should cover KMS integration scenarios
+
+## Common Development Tasks
+
+### Running KMS Tests Locally
+
+1. **Clear proxy settings:** KMS tests require direct localhost connections
+2. **Use serial execution:** `--test-threads=1` prevents port conflicts
+3. **Enable output:** `--nocapture` shows detailed test logs
+4. **Full command:**
+   `NO_PROXY=127.0.0.1,localhost HTTP_PROXY= HTTPS_PROXY= http_proxy= https_proxy= cargo test --package e2e_test test_local_kms_end_to_end -- --nocapture --test-threads=1`
+
+### KMS Development Workflow
+
+1. **Code changes:** Modify KMS-related code in `crates/kms/` or `rustfs/src/`
+2. **Compile:** Always run `cargo build` after changes
+3. **Test specific functionality:** Use targeted test commands for faster iteration
+4. **Full validation:** Run complete end-to-end tests before commits
+
+### Debugging KMS Issues
+
+- **Server startup:** Check that KMS auto-initializes with debug logs
+- **Encryption failures:** Verify SSE headers are correctly set in both PUT and GET responses
+- **Test failures:** Use `--nocapture` to see detailed error messages
+- **Key management:** Test admin API endpoints with proper authentication
+
+## Important Reminders
+
+- **Always compile after code changes:** Use `cargo build` to catch errors early
+- **Don't bypass tests:** All functionality must be properly tested, not worked around
+- **Use proper error handling:** Never use `unwrap()` or `expect()` in production code (except tests)
+- **Follow S3 compatibility:** Ensure all encryption types return correct HTTP response headers
+
+# important-instruction-reminders
+
+Do what has been asked; nothing more, nothing less.
+NEVER create files unless they're absolutely necessary for achieving your goal.
+ALWAYS prefer editing an existing file to creating a new one.
+NEVER proactively create documentation files (*.md) or README files. Only create documentation files if explicitly
+requested by the User.

CONTRIBUTING.md

@@ -2,8 +2,6 @@
 
 ## 📋 Code Quality Requirements
 
-For instructions on setting up and running the local development environment, please see [Development Guide](docs/DEVELOPMENT.md).
-
 ### 🔧 Code Formatting Rules
 
 **MANDATORY**: All code must be properly formatted before committing. This project enforces strict formatting standards to maintain code consistency and readability.
@@ -186,39 +184,6 @@ cargo clippy --all-targets --all-features -- -D warnings
 cargo clippy --fix --all-targets --all-features
 ```
 
-## 📝 Pull Request Guidelines
-
-### Language Requirements
-
-**All Pull Request titles and descriptions MUST be written in English.**
-
-This ensures:
-- Consistency across all contributions
-- Accessibility for international contributors
-- Better integration with automated tools and CI/CD systems
-- Clear communication in a globally understood language
-
-#### PR Description Requirements
-
-When creating a Pull Request, ensure:
-
-1. **Title**: Use English and follow Conventional Commits format (e.g., `fix: improve s3-tests readiness detection`)
-2. **Description**: Write in English, following the PR template format
-3. **Code Comments**: Must be in English (as per coding standards)
-4. **Commit Messages**: Must be in English (as per commit guidelines)
-
-#### PR Template
-
-Always use the PR template (`.github/pull_request_template.md`) and fill in all sections:
-- Type of Change
-- Related Issues
-- Summary of Changes
-- Checklist
-- Impact
-- Additional Notes
-
-**Note**: While you may communicate with reviewers in Chinese during discussions, the PR itself (title, description, and all formal documentation) must be in English.
-
 ---
 
 Following these guidelines ensures high code quality and smooth collaboration across the RustFS project! 🚀

Cargo.toml

@@ -17,97 +17,76 @@ members = [
     "rustfs", # Core file system implementation
     "crates/appauth", # Application authentication and authorization
     "crates/audit", # Audit target management system with multi-target fan-out
-    "crates/checksums", # client checksums
     "crates/common", # Shared utilities and data structures
     "crates/config", # Configuration management
-    "crates/credentials", # Credential management system
     "crates/crypto", # Cryptography and security features
     "crates/ecstore", # Erasure coding storage implementation
     "crates/e2e_test", # End-to-end test suite
     "crates/filemeta", # File metadata management
-    "crates/heal", # Erasure set and object healing
     "crates/iam", # Identity and Access Management
-    "crates/keystone", # OpenStack Keystone integration
-    "crates/kms", # Key Management Service
     "crates/lock", # Distributed locking implementation
     "crates/madmin", # Management dashboard and admin API interface
     "crates/notify", # Notification system for events
     "crates/obs", # Observability utilities
-    "crates/object-capacity", # Capacity scan and refresh core
     "crates/policy", # Policy management
-    "crates/protocols", # Protocol implementations (FTPS, SFTP, etc.)
     "crates/protos", # Protocol buffer definitions
     "crates/rio", # Rust I/O utilities and abstractions
-    "crates/concurrency", # Concurrency management for RustFS - timeout, locking, backpressure, and I/O scheduling
-    "crates/s3-common", # Common utilities and data structures for S3 compatibility
+    "crates/targets", # Target-specific configurations and utilities
     "crates/s3select-api", # S3 Select API interface
     "crates/s3select-query", # S3 Select query engine
-    "crates/scanner", # Scanner for data integrity checks and health monitoring
     "crates/signer", # client signer
-    "crates/targets", # Target-specific configurations and utilities
-    "crates/trusted-proxies", # Trusted proxies management
+    "crates/checksums", # client checksums
     "crates/utils", # Utility functions and helpers
     "crates/workers", # Worker thread pools and task scheduling
-    "crates/io-metrics", # Zero-copy metrics collection for performance analysis
-    "crates/io-core", # Zero-copy core reader and writer implementations
     "crates/zip", # ZIP file handling and compression
+    "crates/ahm", # Asynchronous Hash Map for concurrent data structures
+    "crates/mcp", # MCP server for S3 operations
+    "crates/kms", # Key Management Service
 ]
-resolver = "3"
+resolver = "2"
 
 [workspace.package]
 edition = "2024"
 license = "Apache-2.0"
 repository = "https://github.com/rustfs/rustfs"
-rust-version = "1.93.0"
+rust-version = "1.85"
 version = "0.0.5"
 homepage = "https://rustfs.com"
 description = "RustFS is a high-performance distributed object storage software built using Rust, one of the most popular languages worldwide. "
 keywords = ["RustFS", "Minio", "object-storage", "filesystem", "s3"]
 categories = ["web-programming", "development-tools", "filesystem", "network-programming"]
-authors = ["RustFS Team"]
 
 [workspace.lints.rust]
 unsafe_code = "deny"
 
 [workspace.lints.clippy]
 all = "warn"
-needless_collect = "warn"
-redundant_clone = "warn"
 
 [workspace.dependencies]
 # RustFS Internal Crates
 rustfs = { path = "./rustfs", version = "0.0.5" }
-rustfs-heal = { path = "crates/heal", version = "0.0.5" }
+rustfs-ahm = { path = "crates/ahm", version = "0.0.5" }
 rustfs-appauth = { path = "crates/appauth", version = "0.0.5" }
 rustfs-audit = { path = "crates/audit", version = "0.0.5" }
 rustfs-checksums = { path = "crates/checksums", version = "0.0.5" }
 rustfs-common = { path = "crates/common", version = "0.0.5" }
 rustfs-config = { path = "./crates/config", version = "0.0.5" }
-rustfs-concurrency = { path = "./crates/concurrency", version = "0.0.5" }
-rustfs-credentials = { path = "crates/credentials", version = "0.0.5" }
 rustfs-crypto = { path = "crates/crypto", version = "0.0.5" }
 rustfs-ecstore = { path = "crates/ecstore", version = "0.0.5" }
 rustfs-filemeta = { path = "crates/filemeta", version = "0.0.5" }
 rustfs-iam = { path = "crates/iam", version = "0.0.5" }
-rustfs-keystone = { path = "crates/keystone", version = "0.0.5" }
 rustfs-kms = { path = "crates/kms", version = "0.0.5" }
 rustfs-lock = { path = "crates/lock", version = "0.0.5" }
 rustfs-madmin = { path = "crates/madmin", version = "0.0.5" }
+rustfs-mcp = { path = "crates/mcp", version = "0.0.5" }
 rustfs-notify = { path = "crates/notify", version = "0.0.5" }
-rustfs-io-metrics = { path = "crates/io-metrics", version = "0.0.5" }
-rustfs-io-core = { path = "crates/io-core", version = "0.0.5" }
-rustfs-object-capacity = { path = "crates/object-capacity", version = "0.0.5" }
 rustfs-obs = { path = "crates/obs", version = "0.0.5" }
 rustfs-policy = { path = "crates/policy", version = "0.0.5" }
 rustfs-protos = { path = "crates/protos", version = "0.0.5" }
-rustfs-protocols = { path = "crates/protocols", version = "0.0.5" }
 rustfs-rio = { path = "crates/rio", version = "0.0.5" }
-rustfs-s3-common = { path = "crates/s3-common", version = "0.0.5" }
 rustfs-s3select-api = { path = "crates/s3select-api", version = "0.0.5" }
 rustfs-s3select-query = { path = "crates/s3select-query", version = "0.0.5" }
-rustfs-scanner = { path = "crates/scanner", version = "0.0.5" }
 rustfs-signer = { path = "crates/signer", version = "0.0.5" }
-rustfs-trusted-proxies = { path = "crates/trusted-proxies", version = "0.0.5" }
 rustfs-targets = { path = "crates/targets", version = "0.0.5" }
 rustfs-utils = { path = "crates/utils", version = "0.0.5" }
 rustfs-workers = { path = "crates/workers", version = "0.0.5" }
@@ -115,208 +94,193 @@ rustfs-zip = { path = "./crates/zip", version = "0.0.5" }
 
 # Async Runtime and Networking
 async-channel = "2.5.0"
-async-compression = { version = "0.4.41" }
+async-compression = { version = "0.4.19" }
 async-recursion = "1.1.1"
 async-trait = "0.1.89"
-async-nats = "0.47.0"
-axum = "0.8.9"
-futures = "0.3.32"
-futures-core = "0.3.32"
-futures-util = "0.3.32"
-pollster = "0.4.0"
-pulsar = { version = "6.7.2", default-features = false, features = ["tokio-rustls-runtime"] }
-hyper = { version = "1.9.0", features = ["http2", "http1", "server"] }
-hyper-rustls = { version = "0.27.9", default-features = false, features = ["native-tokio", "http1", "tls12", "logging", "http2", "aws-lc-rs", "webpki-roots"] }
-hyper-util = { version = "0.1.20", features = ["tokio", "server-auto", "server-graceful", "tracing"] }
-http = "1.4.0"
+axum = "0.8.6"
+axum-extra = "0.12.1"
+axum-server = { version = "0.7.2", features = ["tls-rustls-no-provider"], default-features = false }
+futures = "0.3.31"
+futures-core = "0.3.31"
+futures-util = "0.3.31"
+hyper = { version = "1.7.0", features = ["http2", "http1", "server"] }
+hyper-rustls = { version = "0.27.7", default-features = false, features = ["native-tokio", "http1", "tls12", "logging", "http2", "ring", "webpki-roots"] }
+hyper-util = { version = "0.1.17", features = ["tokio", "server-auto", "server-graceful"] }
+http = "1.3.1"
 http-body = "1.0.1"
-http-body-util = "0.1.3"
-reqwest = { version = "0.13.2", default-features = false, features = ["rustls", "charset", "http2", "system-proxy", "stream", "json", "blocking", "query", "form"] }
-socket2 = { version = "0.6.3", features = ["all"] }
-tokio = { version = "1.52.1", features = ["fs", "rt-multi-thread"] }
-tokio-rustls = { version = "0.26.4", default-features = false, features = ["logging", "tls12", "aws-lc-rs"] }
-tokio-stream = { version = "0.1.18" }
-tokio-test = "0.4.5"
-tokio-util = { version = "0.7.18", features = ["io", "compat"] }
-tonic = { version = "0.14.5", features = ["gzip"] }
-tonic-prost = { version = "0.14.5" }
-tonic-prost-build = { version = "0.14.5" }
-tower = { version = "0.5.3", features = ["timeout"] }
-tower-http = { version = "0.6.8", features = ["cors"] }
+reqwest = { version = "0.12.24", default-features = false, features = ["rustls-tls-webpki-roots", "charset", "http2", "system-proxy", "stream", "json", "blocking"] }
+socket2 = "0.6.1"
+tokio = { version = "1.48.0", features = ["fs", "rt-multi-thread"] }
+tokio-rustls = { version = "0.26.4", default-features = false, features = ["logging", "tls12", "ring"] }
+tokio-stream = { version = "0.1.17" }
+tokio-test = "0.4.4"
+tokio-util = { version = "0.7.17", features = ["io", "compat"] }
+tonic = { version = "0.14.2", features = ["gzip"] }
+tonic-prost = { version = "0.14.2" }
+tonic-prost-build = { version = "0.14.2" }
+tower = { version = "0.5.2", features = ["timeout"] }
+tower-http = { version = "0.6.6", features = ["cors"] }
 
 # Serialization and Data Formats
-bytes = { version = "1.11.1", features = ["serde"] }
-bytesize = "2.3.1"
+bytes = { version = "1.10.1", features = ["serde"] }
+bytesize = "2.1.0"
 byteorder = "1.5.0"
-flatbuffers = "25.12.19"
+flatbuffers = "25.9.23"
 form_urlencoded = "1.2.2"
-prost = "0.14.3"
-quick-xml = "0.39.2"
-rmp = { version = "0.8.15" }
-rmp-serde = { version = "1.3.1" }
+prost = "0.14.1"
+quick-xml = "0.38.3"
+rmcp = { version = "0.8.4" }
+rmp = { version = "0.8.14" }
+rmp-serde = { version = "1.3.0" }
 serde = { version = "1.0.228", features = ["derive"] }
-serde_json = { version = "1.0.149", features = ["raw_value"] }
+serde_json = { version = "1.0.145", features = ["raw_value"] }
 serde_urlencoded = "0.7.1"
+schemars = "1.0.5"
 
 # Cryptography and Security
-aes-gcm = { version = "0.11.0-rc.3", features = ["rand_core"] }
-argon2 = { version = "0.6.0-rc.8" }
-blake2 = "0.11.0-rc.6"
-chacha20poly1305 = { version = "0.11.0-rc.3" }
-crc-fast = "1.9.0"
-hmac = { version = "0.13.0" }
-jsonwebtoken = { version = "10.3.0", features = ["aws_lc_rs"] }
-openidconnect = { version = "4.0", default-features = false }
-pbkdf2 = "0.13.0-rc.10"
-rsa = { version = "0.10.0-rc.17" }
-rustls = { version = "0.23.38", default-features = false, features = ["aws-lc-rs", "logging", "tls12", "prefer-post-quantum", "std"] }
-rustls-pki-types = "1.14.0"
-sha1 = "0.11.0"
-sha2 = "0.11.0"
-subtle = "2.6"
+aes-gcm = { version = "0.10.3", features = ["std"] }
+argon2 = { version = "0.5.3", features = ["std"] }
+blake3 = { version = "1.8.2" }
+chacha20poly1305 = { version = "0.10.1" }
+crc-fast = "1.3.0"
+crc32c = "0.6.8"
+crc32fast = "1.5.0"
+crc64fast-nvme = "1.2.0"
+hmac = "0.12.1"
+jsonwebtoken = { version = "10.1.0", features = ["rust_crypto"] }
+pbkdf2 = "0.12.2"
+rsa = { version = "0.9.8" }
+rustls = { version = "0.23.35", features = ["ring", "logging", "std", "tls12"], default-features = false }
+rustls-pemfile = "2.2.0"
+rustls-pki-types = "1.13.0"
+sha1 = "0.10.6"
+sha2 = "0.10.9"
 zeroize = { version = "1.8.2", features = ["derive"] }
 
 # Time and Date
-chrono = { version = "0.4.44", features = ["serde"] }
+chrono = { version = "0.4.42", features = ["serde"] }
 humantime = "2.3.0"
-jiff = { version = "0.2.23", features = ["serde"] }
-time = { version = "0.3.47", features = ["std", "parsing", "formatting", "macros", "serde"] }
+time = { version = "0.3.44", features = ["std", "parsing", "formatting", "macros", "serde"] }
 
 # Utilities and Tools
-anyhow = "1.0.102"
-arc-swap = "1.9.1"
-astral-tokio-tar = "0.6.0"
+anyhow = "1.0.100"
+arc-swap = "1.7.1"
+astral-tokio-tar = "0.5.6"
 atoi = "2.0.0"
 atomic_enum = "0.3.0"
-aws-config = { version = "1.8.15" }
-aws-credential-types = { version = "1.2.14" }
-aws-sdk-s3 = { version = "1.129.0", default-features = false, features = ["sigv4a", "default-https-client", "rt-tokio"] }
-aws-smithy-http-client = { version = "1.1.12", default-features = false, features = ["default-client", "rustls-aws-lc"] }
-aws-smithy-types = { version = "1.4.7" }
-backtrace = "0.3.76"
+aws-config = { version = "1.8.10" }
+aws-credential-types = { version = "1.2.8" }
+aws-sdk-s3 = { version = "1.110.0", default-features = false, features = ["sigv4a", "rustls", "rt-tokio"] }
+aws-smithy-types = { version = "1.3.4" }
 base64 = "0.22.1"
 base64-simd = "0.8.0"
 brotli = "8.0.2"
 cfg-if = "1.0.4"
-clap = { version = "4.6.1", features = ["derive", "env"] }
-const-str = { version = "1.1.0", features = ["std", "proc"] }
-convert_case = "0.11.0"
-criterion = { version = "0.8", features = ["html_reports"] }
+clap = { version = "4.5.51", features = ["derive", "env"] }
+const-str = { version = "0.7.0", features = ["std", "proc"] }
+convert_case = "0.8.0"
+criterion = { version = "0.7", features = ["html_reports"] }
 crossbeam-queue = "0.3.12"
-crossbeam-channel = "0.5.15"
-crossbeam-deque = "0.8.6"
-crossbeam-utils = "0.8.21"
-datafusion = "53.1.0"
+datafusion = "50.3.0"
 derive_builder = "0.20.2"
 enumset = "1.1.10"
-faster-hex = "0.10.0"
-flate2 = "1.1.9"
+flate2 = "1.1.5"
+flexi_logger = { version = "0.31.7", features = ["trc", "dont_minimize_extra_stacks", "compress", "kv"] }
 glob = "0.3.3"
-google-cloud-storage = "1.11.0"
-google-cloud-auth = "1.9.0"
-hashbrown = { version = "0.17.0", features = ["serde", "rayon"] }
-hex = "0.4.3"
+google-cloud-storage = "1.2.0"
+google-cloud-auth = "1.1.0"
+hashbrown = { version = "0.16.0", features = ["serde", "rayon"] }
+heed = { version = "0.22.0" }
 hex-simd = "0.8.0"
 highway = { version = "1.3.0" }
 ipnetwork = { version = "0.21.1", features = ["serde"] }
 lazy_static = "1.5.0"
-libc = "0.2.185"
-libsystemd = "0.7.2"
-local-ip-address = "0.6.11"
-memmap2 = "0.9.10"
+libc = "0.2.177"
+libsystemd = { version = "0.7.2" }
+local-ip-address = "0.6.5"
 lz4 = "1.28.1"
-matchit = "0.9.2"
-md-5 = "0.11.0"
+matchit = "0.9.0"
+md-5 = "0.10.6"
 md5 = "0.8.0"
+metrics = "0.24.2"
+metrics-exporter-opentelemetry = "0.1.2"
 mime_guess = "2.0.5"
-moka = { version = "0.12.15", features = ["future"] }
+moka = { version = "0.12.11", features = ["future"] }
 netif = "0.1.6"
+nix = { version = "0.30.1", features = ["fs"] }
+nu-ansi-term = "0.50.3"
 num_cpus = { version = "1.17.0" }
-nvml-wrapper = "0.12.1"
-object_store = "0.13.2"
+nvml-wrapper = "0.11.0"
+object_store = "0.12.4"
+once_cell = "1.21.3"
 parking_lot = "0.12.5"
 path-absolutize = "3.1.1"
 path-clean = "1.0.1"
-percent-encoding = "2.3.2"
-pin-project-lite = "0.2.17"
+pin-project-lite = "0.2.16"
 pretty_assertions = "1.4.1"
-rand = { version = "0.10.1", features = ["serde"] }
-ratelimit = "0.10.1"
-rayon = "1.12.0"
-reed-solomon-erasure = { version = "6.0", default-features = false, features = ["std", "simd-accel"] }
-reed-solomon-simd = "3.1.0"
-regex = { version = "1.12.3" }
-rumqttc = { package = "rumqttc-next", version = "0.30.0", features = ["websocket"] }
-rustix = { version = "1.1.4", features = ["fs"] }
-rust-embed = { version = "8.11.0" }
-rustc-hash = { version = "2.1.2" }
-s3s = { git = "https://github.com/rustfs/s3s", rev = "a3b16608df35aaeed8fff08b4988d03f4ca9445b", features = ["minio"] }
-serial_test = "3.4.0"
-shadow-rs = { version = "1.7.1", default-features = false }
-siphasher = "1.0.2"
+rand = "0.9.2"
+rayon = "1.11.0"
+reed-solomon-simd = { version = "3.1.0" }
+regex = { version = "1.12.2" }
+rumqttc = { version = "0.25.0" }
+rust-embed = { version = "8.9.0" }
+rustc-hash = { version = "2.1.1" }
+s3s = { version = "0.12.0-rc.3", features = ["minio"] }
+scopeguard = "1.2.0"
+serial_test = "3.2.0"
+shadow-rs = { version = "1.4.0", default-features = false }
+siphasher = "1.0.1"
 smallvec = { version = "1.15.1", features = ["serde"] }
 smartstring = "1.0.1"
-snafu = "0.9.0"
+snafu = "0.8.9"
 snap = "1.1.1"
-starshard = { version = "1.1.0", features = ["rayon", "async", "serde"] }
-strum = { version = "0.28.0", features = ["derive"] }
-sysinfo = "0.38.4"
+starshard = { version = "0.5.0", features = ["rayon", "async", "serde"] }
+strum = { version = "0.27.2", features = ["derive"] }
+sysctl = "0.7.1"
+sysinfo = "0.37.2"
 temp-env = "0.3.6"
-tempfile = "3.27.0"
+tempfile = "3.23.0"
 test-case = "3.3.1"
-thiserror = "2.0.18"
-tracing = { version = "0.1.44" }
-tracing-appender = "0.2.5"
+thiserror = "2.0.17"
+tracing = { version = "0.1.41" }
+tracing-appender = "0.2.3"
 tracing-error = "0.2.1"
-tracing-opentelemetry = "0.32.1"
-tracing-subscriber = { version = "0.3.23", features = ["env-filter", "time"] }
+tracing-opentelemetry = "0.32.0"
+tracing-subscriber = { version = "0.3.20", features = ["env-filter", "time"] }
 transform-stream = "0.3.1"
-url = "2.5.8"
+url = "2.5.7"
 urlencoding = "2.1.3"
-uuid = { version = "1.23.1", features = ["v4", "fast-rng", "macro-diagnostics"] }
-vaultrs = { version = "0.8.0" }
+uuid = { version = "1.18.1", features = ["v4", "fast-rng", "macro-diagnostics"] }
+vaultrs = { version = "0.7.4" }
 walkdir = "2.5.0"
-wildmatch = { version = "2.6.1", features = ["serde"] }
-windows = { version = "0.62.2" }
+wildmatch = { version = "2.5.0", features = ["serde"] }
+winapi = { version = "0.3.9" }
 xxhash-rust = { version = "0.8.15", features = ["xxh64", "xxh3"] }
-zip = "8.5.1"
+zip = "6.0.0"
 zstd = "0.13.3"
 
 # Observability and Metrics
-metrics = "0.24.3"
-dial9-tokio-telemetry = "0.3"
 opentelemetry = { version = "0.31.0" }
 opentelemetry-appender-tracing = { version = "0.31.1", features = ["experimental_use_tracing_span_context", "experimental_metadata_attributes", "spec_unstable_logs_enabled"] }
-opentelemetry-otlp = { version = "0.31.1", features = ["gzip-http", "reqwest-rustls"] }
+opentelemetry-otlp = { version = "0.31.0", default-features = false, features = ["grpc-tonic", "gzip-tonic", "trace", "metrics", "logs", "internal-logs"] }
 opentelemetry_sdk = { version = "0.31.0" }
 opentelemetry-semantic-conventions = { version = "0.31.0", features = ["semconv_experimental"] }
 opentelemetry-stdout = { version = "0.31.0" }
-pyroscope = { version = "2.0.0", features = ["backend-pprof-rs"] }
-
-# FTP and SFTP
-libunftp = { version = "0.23.0", features = ["experimental"] }
-unftp-core = "0.1.0"
-suppaftp = { version = "8.0.2", features = ["tokio", "tokio-rustls-aws-lc-rs"] }
-rcgen = "0.14.7"
-
-# WebDAV
-dav-server = "0.11.0"
 
 # Performance Analysis and Memory Profiling
-mimalloc = "0.1"
 # Use tikv-jemallocator as memory allocator and enable performance analysis
 tikv-jemallocator = { version = "0.6", features = ["profiling", "stats", "unprefixed_malloc_on_supported_platforms", "background_threads"] }
 # Used to control and obtain statistics for jemalloc at runtime
 tikv-jemalloc-ctl = { version = "0.6", features = ["use_std", "stats", "profiling"] }
 # Used to generate pprof-compatible memory profiling data and support symbolization and flame graphs
-jemalloc_pprof = { version = "0.8.2", features = ["symbolize", "flamegraph"] }
+jemalloc_pprof = { version = "0.8.1", features = ["symbolize", "flamegraph"] }
 # Used to generate CPU performance analysis data and flame diagrams
-# pprof = { version = "0.15.0", features = ["flamegraph", "protobuf-codec"] }
-# Pyroscope uses a patched pprof, until they merge back upstream, replace all references. Otherwise, two pprof libs with symbol collision.
-pprof = { package = "pprof-pyroscope-fork", version = "0.1500.3", features = ["flamegraph", "protobuf-codec"] }
+pprof = { version = "0.15.0", features = ["flamegraph", "protobuf-codec"] }
+mimalloc = "0.1"
+
 
 [workspace.metadata.cargo-shear]
-ignored = ["rustfs"]
+ignored = ["rustfs", "rustfs-mcp", "tokio-test", "scopeguard"]
 
 [profile.release]
 opt-level = 3

Dockerfile

@@ -1,18 +1,4 @@
-# Copyright 2024 RustFS Team
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-FROM alpine:3.23.4 AS build
+FROM alpine:3.22 AS build
 
 ARG TARGETARCH
 ARG RELEASE=latest
@@ -54,7 +40,7 @@ RUN set -eux; \
     rm -rf rustfs.zip /build/.tmp || true
 
 
-FROM alpine:3.23.4
+FROM alpine:3.22
 
 ARG RELEASE=latest
 ARG BUILD_DATE
@@ -72,33 +58,31 @@ LABEL name="RustFS" \
       url="https://rustfs.com" \
       license="Apache-2.0"
 
-RUN apk update && \
-    apk add --no-cache ca-certificates coreutils curl
+RUN apk add --no-cache ca-certificates coreutils curl
 
 COPY --from=build /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/
 COPY --from=build /build/rustfs /usr/bin/rustfs
 COPY entrypoint.sh /entrypoint.sh
 
-RUN chmod +x /usr/bin/rustfs /entrypoint.sh
-
-RUN addgroup -g 10001 -S rustfs && \
-    adduser -u 10001 -G rustfs -S rustfs -D && \
+RUN chmod +x /usr/bin/rustfs /entrypoint.sh && \
     mkdir -p /data /logs && \
-    chown -R rustfs:rustfs /data /logs && \
     chmod 0750 /data /logs
 
-ENV RUSTFS_CORS_ALLOWED_ORIGINS="*" \
+ENV RUSTFS_ADDRESS=":9000" \
+    RUSTFS_CONSOLE_ADDRESS=":9001" \
+    RUSTFS_ACCESS_KEY="rustfsadmin" \
+    RUSTFS_SECRET_KEY="rustfsadmin" \
+    RUSTFS_CONSOLE_ENABLE="true" \
+    RUSTFS_EXTERNAL_ADDRESS="" \
+    RUSTFS_CORS_ALLOWED_ORIGINS="*" \
     RUSTFS_CONSOLE_CORS_ALLOWED_ORIGINS="*" \
     RUSTFS_VOLUMES="/data" \
-    RUSTFS_OBS_LOGGER_LEVEL=warn \
-    RUSTFS_OBS_LOG_DIRECTORY=/logs \
-    RUSTFS_OBS_ENVIRONMENT=production
-    
+    RUST_LOG="warn" \
+    RUSTFS_OBS_LOG_DIRECTORY="/logs" \
+    RUSTFS_SINKS_FILE_PATH="/logs"
+
 EXPOSE 9000 9001
-
-VOLUME ["/data"]
-
-USER rustfs
+VOLUME ["/data", "/logs"]
 
 ENTRYPOINT ["/entrypoint.sh"]
 

Dockerfile.decommission-local

@@ -1,65 +0,0 @@
-# Copyright 2024 RustFS Team
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-FROM rust:1.91-trixie
-
-RUN set -eux; \
-    export DEBIAN_FRONTEND=noninteractive; \
-    apt-get update; \
-    apt-get install -y --no-install-recommends \
-    build-essential \
-    ca-certificates \
-    curl \
-    git \
-    pkg-config \
-    libssl-dev \
-    lld \
-    protobuf-compiler \
-    flatbuffers-compiler; \
-    rm -rf /var/lib/apt/lists/*
-
-WORKDIR /usr/src/rustfs
-
-COPY . .
-
-RUN ./scripts/static.sh
-RUN cargo run --bin gproto
-RUN cargo build --release --locked --bin rustfs
-
-RUN set -eux; \
-    groupadd -g 10001 rustfs; \
-    useradd -u 10001 -g rustfs -M -s /usr/sbin/nologin rustfs
-
-WORKDIR /app
-
-RUN set -eux; \
-    mkdir -p /data /logs; \
-    chown -R rustfs:rustfs /data /logs /app; \
-    chmod 0750 /data /logs
-
-COPY entrypoint.sh /entrypoint.sh
-RUN install -m 0755 /usr/src/rustfs/target/release/rustfs /usr/bin/rustfs && chmod +x /entrypoint.sh
-
-ENV RUSTFS_VOLUMES="/data" \
-    RUST_LOG="warn" \
-    RUSTFS_OBS_LOG_DIRECTORY="/logs" \
-    RUSTFS_USERNAME="rustfs" \
-    RUSTFS_GROUPNAME="rustfs" \
-    RUSTFS_UID="10001" \
-    RUSTFS_GID="10001"
-
-EXPOSE 9000 9001
-
-ENTRYPOINT ["/entrypoint.sh"]
-CMD ["/usr/bin/rustfs"]

Dockerfile.glibc

@@ -1,115 +0,0 @@
-# Copyright 2024 RustFS Team
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-FROM ubuntu:24.04 AS build
-
-ARG TARGETARCH
-ARG RELEASE=latest
-
-RUN apt-get update && apt-get install -y --no-install-recommends \
-    ca-certificates \
-    curl \
-    unzip \
-    && rm -rf /var/lib/apt/lists/*
-
-WORKDIR /build
-
-RUN set -eux; \
-    case "$TARGETARCH" in \
-      amd64)  ARCH_SUBSTR="x86_64-gnu"  ;; \
-      arm64)  ARCH_SUBSTR="aarch64-gnu" ;; \
-      *) echo "Unsupported TARGETARCH=$TARGETARCH" >&2; exit 1 ;; \
-    esac; \
-    \
-    if [ "$RELEASE" = "latest" ]; then \
-      TAG="$(curl -fsSL https://api.github.com/repos/rustfs/rustfs/releases \
-              | grep -o '"tag_name": "[^"]*"' | cut -d'"' -f4 | head -n 1)"; \
-    else \
-      TAG="$RELEASE"; \
-    fi; \
-    \
-    URL="$(curl -fsSL "https://api.github.com/repos/rustfs/rustfs/releases/tags/$TAG" \
-           | grep -o "\"browser_download_url\": \"[^\"]*${ARCH_SUBSTR}[^\"]*\\.zip\"" \
-           | cut -d'"' -f4 | head -n 1)"; \
-    \
-    if [ -z "$URL" ]; then echo "Failed to locate release asset for $ARCH_SUBSTR at tag $TAG" >&2; exit 1; fi; \
-    \
-    curl -fL "$URL" -o rustfs.zip; \
-    unzip -q rustfs.zip -d /build; \
-    \
-    if [ ! -x /build/rustfs ]; then \
-      BIN_PATH="$(unzip -Z -1 rustfs.zip | grep -E '(^|/)rustfs$' | head -n 1 || true)"; \
-      if [ -n "$BIN_PATH" ]; then \
-        mkdir -p /build/.tmp && unzip -q rustfs.zip "$BIN_PATH" -d /build/.tmp && \
-        mv "/build/.tmp/$BIN_PATH" /build/rustfs; \
-      fi; \
-    fi; \
-    [ -x /build/rustfs ] || { echo "rustfs binary not found in asset" >&2; exit 1; }; \
-    chmod +x /build/rustfs; \
-    rm -rf rustfs.zip /build/.tmp || true
-
-FROM ubuntu:24.04
-
-ARG RELEASE=latest
-ARG BUILD_DATE
-ARG VCS_REF
-
-LABEL name="RustFS" \
-      vendor="RustFS Team" \
-      maintainer="RustFS Team <dev@rustfs.com>" \
-      version="v${RELEASE#v}" \
-      release="${RELEASE}" \
-      build-date="${BUILD_DATE}" \
-      vcs-ref="${VCS_REF}" \
-      summary="High-performance distributed object storage system (glibc)" \
-      url="https://rustfs.com" \
-      license="Apache-2.0"
-
-RUN apt-get update && apt-get install -y --no-install-recommends \
-    ca-certificates \
-    curl \
-    && rm -rf /var/lib/apt/lists/*
-
-COPY --from=build /build/rustfs /usr/bin/rustfs
-COPY entrypoint.sh /entrypoint.sh
-
-RUN chmod +x /usr/bin/rustfs /entrypoint.sh
-
-RUN groupadd -g 10001 rustfs && \
-    useradd -u 10001 -g rustfs -m -s /sbin/nologin rustfs && \
-    mkdir -p /data /logs && \
-    chown -R rustfs:rustfs /data /logs && \
-    chmod 0750 /data /logs
-
-ENV RUSTFS_ADDRESS=":9000" \
-    RUSTFS_CONSOLE_ADDRESS=":9001" \
-    RUSTFS_ACCESS_KEY="rustfsadmin" \
-    RUSTFS_SECRET_KEY="rustfsadmin" \
-    RUSTFS_CONSOLE_ENABLE="true" \
-    RUSTFS_CORS_ALLOWED_ORIGINS="*" \
-    RUSTFS_CONSOLE_CORS_ALLOWED_ORIGINS="*" \
-    RUSTFS_VOLUMES="/data" \
-    RUSTFS_OBS_LOGGER_LEVEL=warn \
-    RUSTFS_OBS_LOG_DIRECTORY=/logs \
-    RUSTFS_OBS_ENVIRONMENT=production
-    
-EXPOSE 9000 9001
-
-VOLUME ["/data"]
-
-USER rustfs
-
-ENTRYPOINT ["/entrypoint.sh"]
-
-CMD ["rustfs"]

Dockerfile.source

@@ -1,18 +1,4 @@
 # syntax=docker/dockerfile:1.6
-# Copyright 2024 RustFS Team
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
 # Multi-stage Dockerfile for RustFS - LOCAL DEVELOPMENT ONLY
 #
 # IMPORTANT: This Dockerfile builds RustFS from source for local development and testing.
@@ -30,7 +16,7 @@ ARG BUILDPLATFORM
 # -----------------------------
 # Build stage
 # -----------------------------
-FROM rust:1.91-trixie AS builder
+FROM rust:1.88-bookworm AS builder
 
 # Re-declare args after FROM
 ARG TARGETPLATFORM
@@ -53,9 +39,7 @@ RUN set -eux; \
     libssl-dev \
     lld \
     protobuf-compiler \
-    flatbuffers-compiler \
-    gcc-aarch64-linux-gnu \
-    gcc-x86-64-linux-gnu; \
+    flatbuffers-compiler; \
     rm -rf /var/lib/apt/lists/*
 
 # Optional: cross toolchain for aarch64 (only when targeting linux/arm64)
@@ -67,18 +51,18 @@ RUN set -eux; \
     rm -rf /var/lib/apt/lists/*; \
     fi
 
-# Add Rust targets for both arches (to support cross-builds on multi-arch runners)
+# Add Rust targets based on TARGETPLATFORM
 RUN set -eux; \
-    rustup target add x86_64-unknown-linux-gnu aarch64-unknown-linux-gnu; \
-    rustup component add rust-std-x86_64-unknown-linux-gnu rust-std-aarch64-unknown-linux-gnu
+    case "${TARGETPLATFORM:-linux/amd64}" in \
+    linux/amd64) rustup target add x86_64-unknown-linux-gnu ;; \
+    linux/arm64) rustup target add aarch64-unknown-linux-gnu ;; \
+    *) echo "Unsupported TARGETPLATFORM=${TARGETPLATFORM}" >&2; exit 1 ;; \
+    esac
 
 # Cross-compilation environment (used only when targeting aarch64)
 ENV CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_LINKER=aarch64-linux-gnu-gcc
 ENV CC_aarch64_unknown_linux_gnu=aarch64-linux-gnu-gcc
 ENV CXX_aarch64_unknown_linux_gnu=aarch64-linux-gnu-g++
-ENV CARGO_TARGET_X86_64_UNKNOWN_LINUX_GNU_LINKER=x86_64-linux-gnu-gcc
-ENV CC_x86_64_unknown_linux_gnu=x86_64-linux-gnu-gcc
-ENV CXX_x86_64_unknown_linux_gnu=x86_64-linux-gnu-g++
 
 WORKDIR /usr/src/rustfs
 
@@ -88,6 +72,7 @@ COPY Cargo.toml Cargo.lock ./
 # 2) workspace member manifests (adjust if workspace layout changes)
 COPY rustfs/Cargo.toml rustfs/Cargo.toml
 COPY crates/*/Cargo.toml crates/
+COPY cli/rustfs-gui/Cargo.toml cli/rustfs-gui/Cargo.toml
 
 # Pre-fetch dependencies for better caching
 RUN --mount=type=cache,target=/usr/local/cargo/registry \
@@ -97,10 +82,6 @@ RUN --mount=type=cache,target=/usr/local/cargo/registry \
 # 3) copy full sources (this is the main cache invalidation point)
 COPY . .
 
-# Generate static files
-
-RUN ./scripts/static.sh
-
 # Cargo build configuration for lean release artifacts
 ENV CARGO_NET_GIT_FETCH_WITH_CLI=true \
     CARGO_REGISTRIES_CRATES_IO_PROTOCOL=sparse \
@@ -136,45 +117,6 @@ RUN --mount=type=cache,target=/usr/local/cargo/registry \
     ;; \
     esac
 
-# -----------------------------
-# Development stage (keeps toolchain)
-# -----------------------------
-FROM builder AS dev
-
-ARG BUILD_DATE
-ARG VCS_REF
-
-LABEL name="RustFS (dev-source)" \
-    maintainer="RustFS Team" \
-    build-date="${BUILD_DATE}" \
-    vcs-ref="${VCS_REF}" \
-    description="RustFS - local development with Rust toolchain."
-
-# Install runtime dependencies that might be missing in partial builder
-# (builder already has build-essential, lld, etc.)
-WORKDIR /app
-
-ENV CARGO_INCREMENTAL=1
-
-# Ensure we have the same default env vars available
-ENV RUSTFS_VOLUMES="/data" \
-    RUST_LOG="warn" \
-    RUSTFS_OBS_LOG_DIRECTORY="/logs" \
-    RUSTFS_USERNAME="rustfs" \
-    RUSTFS_GROUPNAME="rustfs" \
-    RUSTFS_UID="10001" \
-    RUSTFS_GID="10001"
-
-# Note: We don't COPY source here because we expect it to be mounted at /app
-# We rely on cargo run to build and run
-EXPOSE 9000 9001
-
-COPY entrypoint.sh /entrypoint.sh
-RUN chmod +x /entrypoint.sh
-
-ENTRYPOINT ["/entrypoint.sh"]
-CMD ["cargo", "run", "--bin", "rustfs", "--"]
-
 # -----------------------------
 # Runtime stage (Ubuntu minimal)
 # -----------------------------
@@ -201,8 +143,8 @@ RUN set -eux; \
 
 # Create a conventional runtime user/group (final switch happens in entrypoint via chroot --userspec)
 RUN set -eux; \
-    groupadd -g 10001 rustfs; \
-    useradd -u 10001 -g rustfs -M -s /usr/sbin/nologin rustfs
+    groupadd -g 1000 rustfs; \
+    useradd -u 1000 -g rustfs -M -s /usr/sbin/nologin rustfs
 
 WORKDIR /app
 
@@ -218,15 +160,21 @@ COPY entrypoint.sh /entrypoint.sh
 RUN chmod +x /usr/bin/rustfs /entrypoint.sh
 
 # Default environment (override in docker run/compose as needed)
-ENV RUSTFS_VOLUMES="/data" \
+ENV RUSTFS_ADDRESS=":9000" \
+    RUSTFS_ACCESS_KEY="rustfsadmin" \
+    RUSTFS_SECRET_KEY="rustfsadmin" \
+    RUSTFS_CONSOLE_ENABLE="true" \
+    RUSTFS_VOLUMES="/data" \
     RUST_LOG="warn" \
+    RUSTFS_OBS_LOG_DIRECTORY="/logs" \
+    RUSTFS_SINKS_FILE_PATH="/logs" \
     RUSTFS_USERNAME="rustfs" \
     RUSTFS_GROUPNAME="rustfs" \
-    RUSTFS_UID="10001" \
-    RUSTFS_GID="10001"
+    RUSTFS_UID="1000" \
+    RUSTFS_GID="1000"
 
 EXPOSE 9000
-VOLUME ["/data"]
+VOLUME ["/data", "/logs"]
 
 # Keep root here; entrypoint will drop privileges using chroot --userspec
 ENTRYPOINT ["/entrypoint.sh"]

Makefile

@@ -2,80 +2,375 @@
 # Remote development requires VSCode with Dev Containers, Remote SSH, Remote Explorer
 # https://code.visualstudio.com/docs/remote/containers
 ###########
-
-.PHONY: SHELL
-
-# Makefile global config
-# Use config.mak to override any of the following variables.
-# Do not make changes here.
-
-.DEFAULT_GOAL := help
-.EXPORT_ALL_VARIABLES:
-.ONESHELL:
-.SILENT:
-
-NUM_CORES := $(shell nproc 2>/dev/null || sysctl -n hw.ncpu)
-
-MAKEFLAGS += -j$(NUM_CORES) -l$(NUM_CORES)
-MAKEFLAGS += --silent
-
-SHELL := $(shell which bash)
-.SHELLFLAGS = -eu -o pipefail -c
-
 DOCKER_CLI ?= docker
 IMAGE_NAME ?= rustfs:v1.0.0
 CONTAINER_NAME ?= rustfs-dev
 # Docker build configurations
 DOCKERFILE_PRODUCTION = Dockerfile
 DOCKERFILE_SOURCE = Dockerfile.source
+
+# Code quality and formatting targets
+.PHONY: fmt
+fmt:
+	@echo "🔧 Formatting code..."
+	cargo fmt --all
+
+.PHONY: fmt-check
+fmt-check:
+	@echo "📝 Checking code formatting..."
+	cargo fmt --all --check
+
+.PHONY: clippy
+clippy:
+	@echo "🔍 Running clippy checks..."
+	cargo clippy --fix --allow-dirty
+	cargo clippy --all-targets --all-features -- -D warnings
+
+.PHONY: check
+check:
+	@echo "🔨 Running compilation check..."
+	cargo check --all-targets
+
+.PHONY: test
+test:
+	@echo "🧪 Running tests..."
+	@if command -v cargo-nextest >/dev/null 2>&1; then \
+		cargo nextest run --all --exclude e2e_test; \
+	else \
+		echo "ℹ️ cargo-nextest not found; falling back to 'cargo test'"; \
+		cargo test --workspace --exclude e2e_test -- --nocapture; \
+	fi
+	cargo test --all --doc
+
+.PHONY: pre-commit
+pre-commit: fmt clippy check test
+	@echo "✅ All pre-commit checks passed!"
+
+.PHONY: setup-hooks
+setup-hooks:
+	@echo "🔧 Setting up git hooks..."
+	chmod +x .git/hooks/pre-commit
+	@echo "✅ Git hooks setup complete!"
+
+.PHONY: e2e-server
+e2e-server:
+	sh $(shell pwd)/scripts/run.sh
+
+.PHONY: probe-e2e
+probe-e2e:
+	sh $(shell pwd)/scripts/probe.sh
+
+# Native build using build-rustfs.sh script
+.PHONY: build
+build:
+	@echo "🔨 Building RustFS using build-rustfs.sh script..."
+	./build-rustfs.sh
+
+.PHONY: build-dev
+build-dev:
+	@echo "🔨 Building RustFS in development mode..."
+	./build-rustfs.sh --dev
+
+# Docker-based build (alternative approach)
+# Usage: make BUILD_OS=ubuntu22.04 build-docker
+# Output: target/ubuntu22.04/release/rustfs
 BUILD_OS ?= rockylinux9.3
+.PHONY: build-docker
+build-docker: SOURCE_BUILD_IMAGE_NAME = rustfs-$(BUILD_OS):v1
+build-docker: SOURCE_BUILD_CONTAINER_NAME = rustfs-$(BUILD_OS)-build
+build-docker: BUILD_CMD = /root/.cargo/bin/cargo build --release --bin rustfs --target-dir /root/s3-rustfs/target/$(BUILD_OS)
+build-docker:
+	@echo "🐳 Building RustFS using Docker ($(BUILD_OS))..."
+	$(DOCKER_CLI) buildx build -t $(SOURCE_BUILD_IMAGE_NAME) -f $(DOCKERFILE_SOURCE) .
+	$(DOCKER_CLI) run --rm --name $(SOURCE_BUILD_CONTAINER_NAME) -v $(shell pwd):/root/s3-rustfs -it $(SOURCE_BUILD_IMAGE_NAME) $(BUILD_CMD)
 
-# Makefile colors config
-bold := $(shell tput bold)
-normal := $(shell tput sgr0)
-errorTitle := $(shell tput setab 1 && tput bold && echo '\n')
-recommendation := $(shell tput setab 4)
-underline := $(shell tput smul)
-reset := $(shell tput -Txterm sgr0)
-black := $(shell tput setaf 0)
-red := $(shell tput setaf 1)
-green := $(shell tput setaf 2)
-yellow := $(shell tput setaf 3)
-blue := $(shell tput setaf 4)
-magenta := $(shell tput setaf 5)
-cyan := $(shell tput setaf 6)
-white := $(shell tput setaf 7)
+.PHONY: build-musl
+build-musl:
+	@echo "🔨 Building rustfs for x86_64-unknown-linux-musl..."
+	@echo "💡 On macOS/Windows, use 'make build-docker' or 'make docker-dev' instead"
+	./build-rustfs.sh --platform x86_64-unknown-linux-musl
 
-define HEADER
-How to use me:
-	# To get help for each target
-	${bold}make help${reset}
+.PHONY: build-gnu
+build-gnu:
+	@echo "🔨 Building rustfs for x86_64-unknown-linux-gnu..."
+	@echo "💡 On macOS/Windows, use 'make build-docker' or 'make docker-dev' instead"
+	./build-rustfs.sh --platform x86_64-unknown-linux-gnu
 
-	# To run and execute a target
-	${bold}make ${cyan}<target>${reset}
+.PHONY: build-musl-arm64
+build-musl-arm64:
+	@echo "🔨 Building rustfs for aarch64-unknown-linux-musl..."
+	@echo "💡 On macOS/Windows, use 'make build-docker' or 'make docker-dev' instead"
+	./build-rustfs.sh --platform aarch64-unknown-linux-musl
 
-	💡 For more help use 'make help', 'make help-build' or 'make help-docker'
+.PHONY: build-gnu-arm64
+build-gnu-arm64:
+	@echo "🔨 Building rustfs for aarch64-unknown-linux-gnu..."
+	@echo "💡 On macOS/Windows, use 'make build-docker' or 'make docker-dev' instead"
+	./build-rustfs.sh --platform aarch64-unknown-linux-gnu
 
-	🦀 RustFS Makefile Help:
+.PHONY: deploy-dev
+deploy-dev: build-musl
+	@echo "🚀 Deploying to dev server: $${IP}"
+	./scripts/dev_deploy.sh $${IP}
 
-	📋 Main Command Categories:
-	 	make help-build                          # Show build-related help
-	 	make help-docker                         # Show Docker-related help
+# ========================================================================================
+# Docker Multi-Architecture Builds (Primary Methods)
+# ========================================================================================
 
-	🔧 Code Quality:
-		make fmt                                 # Format code
-		make clippy                              # Run clippy checks
-		make test                                # Run tests
-		make pre-commit                          # Run all pre-commit checks
+# Production builds using docker-buildx.sh (for CI/CD and production)
+.PHONY: docker-buildx
+docker-buildx:
+	@echo "🏗️ Building multi-architecture production Docker images with buildx..."
+	./docker-buildx.sh
 
-	🚀 Quick Start:
-		make build                               # Build RustFS binary
-		make docker-dev-local                    # Build development Docker image (local)
-		make dev-env-start                       # Start development environment
+.PHONY: docker-buildx-push
+docker-buildx-push:
+	@echo "🚀 Building and pushing multi-architecture production Docker images with buildx..."
+	./docker-buildx.sh --push
+
+.PHONY: docker-buildx-version
+docker-buildx-version:
+	@if [ -z "$(VERSION)" ]; then \
+		echo "❌ Error: Please specify version, example: make docker-buildx-version VERSION=v1.0.0"; \
+		exit 1; \
+	fi
+	@echo "🏗️ Building multi-architecture production Docker images (version: $(VERSION))..."
+	./docker-buildx.sh --release $(VERSION)
+
+.PHONY: docker-buildx-push-version
+docker-buildx-push-version:
+	@if [ -z "$(VERSION)" ]; then \
+		echo "❌ Error: Please specify version, example: make docker-buildx-push-version VERSION=v1.0.0"; \
+		exit 1; \
+	fi
+	@echo "🚀 Building and pushing multi-architecture production Docker images (version: $(VERSION))..."
+	./docker-buildx.sh --release $(VERSION) --push
+
+# Development/Source builds using direct buildx commands
+.PHONY: docker-dev
+docker-dev:
+	@echo "🏗️ Building multi-architecture development Docker images with buildx..."
+	@echo "💡 This builds from source code and is intended for local development and testing"
+	@echo "⚠️  Multi-arch images cannot be loaded locally, use docker-dev-push to push to registry"
+	$(DOCKER_CLI) buildx build \
+		--platform linux/amd64,linux/arm64 \
+		--file $(DOCKERFILE_SOURCE) \
+		--tag rustfs:source-latest \
+		--tag rustfs:dev-latest \
+		.
+
+.PHONY: docker-dev-local
+docker-dev-local:
+	@echo "🏗️ Building single-architecture development Docker image for local use..."
+	@echo "💡 This builds from source code for the current platform and loads locally"
+	$(DOCKER_CLI) buildx build \
+		--file $(DOCKERFILE_SOURCE) \
+		--tag rustfs:source-latest \
+		--tag rustfs:dev-latest \
+		--load \
+		.
+
+.PHONY: docker-dev-push
+docker-dev-push:
+	@if [ -z "$(REGISTRY)" ]; then \
+		echo "❌ Error: Please specify registry, example: make docker-dev-push REGISTRY=ghcr.io/username"; \
+		exit 1; \
+	fi
+	@echo "🚀 Building and pushing multi-architecture development Docker images..."
+	@echo "💡 Pushing to registry: $(REGISTRY)"
+	$(DOCKER_CLI) buildx build \
+		--platform linux/amd64,linux/arm64 \
+		--file $(DOCKERFILE_SOURCE) \
+		--tag $(REGISTRY)/rustfs:source-latest \
+		--tag $(REGISTRY)/rustfs:dev-latest \
+		--push \
+		.
 
 
-endef
-export HEADER
 
--include $(addsuffix /*.mak, $(shell find .config/make -type d))
+# Local production builds using direct buildx (alternative to docker-buildx.sh)
+.PHONY: docker-buildx-production-local
+docker-buildx-production-local:
+	@echo "🏗️ Building single-architecture production Docker image locally..."
+	@echo "💡 Alternative to docker-buildx.sh for local testing"
+	$(DOCKER_CLI) buildx build \
+		--file $(DOCKERFILE_PRODUCTION) \
+		--tag rustfs:production-latest \
+		--tag rustfs:latest \
+		--load \
+		--build-arg RELEASE=latest \
+		.
 
+# ========================================================================================
+# Single Architecture Docker Builds (Traditional)
+# ========================================================================================
+
+.PHONY: docker-build-production
+docker-build-production:
+	@echo "🏗️ Building single-architecture production Docker image..."
+	@echo "💡 Consider using 'make docker-buildx-production-local' for multi-arch support"
+	$(DOCKER_CLI) build -f $(DOCKERFILE_PRODUCTION) -t rustfs:latest .
+
+.PHONY: docker-build-source
+docker-build-source:
+	@echo "🏗️ Building single-architecture source Docker image..."
+	@echo "💡 Consider using 'make docker-dev-local' for multi-arch support"
+	DOCKER_BUILDKIT=1 $(DOCKER_CLI) build \
+		--build-arg BUILDKIT_INLINE_CACHE=1 \
+		-f $(DOCKERFILE_SOURCE) -t rustfs:source .
+
+# ========================================================================================
+# Development Environment
+# ========================================================================================
+
+.PHONY: dev-env-start
+dev-env-start:
+	@echo "🚀 Starting development environment..."
+	$(DOCKER_CLI) buildx build \
+		--file $(DOCKERFILE_SOURCE) \
+		--tag rustfs:dev \
+		--load \
+		.
+	$(DOCKER_CLI) stop $(CONTAINER_NAME) 2>/dev/null || true
+	$(DOCKER_CLI) rm $(CONTAINER_NAME) 2>/dev/null || true
+	$(DOCKER_CLI) run -d --name $(CONTAINER_NAME) \
+		-p 9010:9010 -p 9000:9000 \
+		-v $(shell pwd):/workspace \
+		-it rustfs:dev
+
+.PHONY: dev-env-stop
+dev-env-stop:
+	@echo "🛑 Stopping development environment..."
+	$(DOCKER_CLI) stop $(CONTAINER_NAME) 2>/dev/null || true
+	$(DOCKER_CLI) rm $(CONTAINER_NAME) 2>/dev/null || true
+
+.PHONY: dev-env-restart
+dev-env-restart: dev-env-stop dev-env-start
+
+
+
+# ========================================================================================
+# Build Utilities
+# ========================================================================================
+
+.PHONY: docker-inspect-multiarch
+docker-inspect-multiarch:
+	@if [ -z "$(IMAGE)" ]; then \
+		echo "❌ Error: Please specify image, example: make docker-inspect-multiarch IMAGE=rustfs/rustfs:latest"; \
+		exit 1; \
+	fi
+	@echo "🔍 Inspecting multi-architecture image: $(IMAGE)"
+	docker buildx imagetools inspect $(IMAGE)
+
+.PHONY: build-cross-all
+build-cross-all:
+	@echo "🔧 Building all target architectures..."
+	@echo "💡 On macOS/Windows, use 'make docker-dev' for reliable multi-arch builds"
+	@echo "🔨 Generating protobuf code..."
+	cargo run --bin gproto || true
+	@echo "🔨 Building x86_64-unknown-linux-gnu..."
+	./build-rustfs.sh --platform x86_64-unknown-linux-gnu
+	@echo "🔨 Building aarch64-unknown-linux-gnu..."
+	./build-rustfs.sh --platform aarch64-unknown-linux-gnu
+	@echo "🔨 Building x86_64-unknown-linux-musl..."
+	./build-rustfs.sh --platform x86_64-unknown-linux-musl
+	@echo "🔨 Building aarch64-unknown-linux-musl..."
+	./build-rustfs.sh --platform aarch64-unknown-linux-musl
+	@echo "✅ All architectures built successfully!"
+
+# ========================================================================================
+# Help and Documentation
+# ========================================================================================
+
+.PHONY: help-build
+help-build:
+	@echo "🔨 RustFS Build Help:"
+	@echo ""
+	@echo "🚀 Local Build (Recommended):"
+	@echo "  make build                               # Build RustFS binary (includes console by default)"
+	@echo "  make build-dev                           # Development mode build"
+	@echo "  make build-musl                          # Build x86_64 musl version"
+	@echo "  make build-gnu                           # Build x86_64 GNU version"
+	@echo "  make build-musl-arm64                    # Build aarch64 musl version"
+	@echo "  make build-gnu-arm64                     # Build aarch64 GNU version"
+	@echo ""
+	@echo "🐳 Docker Build:"
+	@echo "  make build-docker                        # Build using Docker container"
+	@echo "  make build-docker BUILD_OS=ubuntu22.04   # Specify build system"
+	@echo ""
+	@echo "🏗️ Cross-architecture Build:"
+	@echo "  make build-cross-all                     # Build binaries for all architectures"
+	@echo ""
+	@echo "🔧 Direct usage of build-rustfs.sh script:"
+	@echo "  ./build-rustfs.sh --help                 # View script help"
+	@echo "  ./build-rustfs.sh --no-console           # Build without console resources"
+	@echo "  ./build-rustfs.sh --force-console-update # Force update console resources"
+	@echo "  ./build-rustfs.sh --dev                  # Development mode build"
+	@echo "  ./build-rustfs.sh --sign                 # Sign binary files"
+	@echo "  ./build-rustfs.sh --platform x86_64-unknown-linux-gnu   # Specify target platform"
+	@echo "  ./build-rustfs.sh --skip-verification    # Skip binary verification"
+	@echo ""
+	@echo "💡 build-rustfs.sh script provides more options, smart detection and binary verification"
+
+.PHONY: help-docker
+help-docker:
+	@echo "🐳 Docker Multi-architecture Build Help:"
+	@echo ""
+	@echo "🚀 Production Image Build (Recommended to use docker-buildx.sh):"
+	@echo "  make docker-buildx                       # Build production multi-arch image (no push)"
+	@echo "  make docker-buildx-push                  # Build and push production multi-arch image"
+	@echo "  make docker-buildx-version VERSION=v1.0.0        # Build specific version"
+	@echo "  make docker-buildx-push-version VERSION=v1.0.0   # Build and push specific version"
+	@echo ""
+	@echo "🔧 Development/Source Image Build (Local development testing):"
+	@echo "  make docker-dev                          # Build dev multi-arch image (cannot load locally)"
+	@echo "  make docker-dev-local                    # Build dev single-arch image (local load)"
+	@echo "  make docker-dev-push REGISTRY=xxx       # Build and push dev image"
+	@echo ""
+	@echo "🏗️ Local Production Image Build (Alternative):"
+	@echo "  make docker-buildx-production-local      # Build production single-arch image locally"
+	@echo ""
+	@echo "📦 Single-architecture Build (Traditional way):"
+	@echo "  make docker-build-production             # Build single-arch production image"
+	@echo "  make docker-build-source                 # Build single-arch source image"
+	@echo ""
+	@echo "🚀 Development Environment Management:"
+	@echo "  make dev-env-start                       # Start development container environment"
+	@echo "  make dev-env-stop                        # Stop development container environment"
+	@echo "  make dev-env-restart                     # Restart development container environment"
+	@echo ""
+	@echo "🔧 Auxiliary Tools:"
+	@echo "  make build-cross-all                     # Build binaries for all architectures"
+	@echo "  make docker-inspect-multiarch IMAGE=xxx  # Check image architecture support"
+	@echo ""
+	@echo "📋 Environment Variables:"
+	@echo "  REGISTRY          Image registry address (required for push)"
+	@echo "  DOCKERHUB_USERNAME    Docker Hub username"
+	@echo "  DOCKERHUB_TOKEN       Docker Hub access token"
+	@echo "  GITHUB_TOKEN          GitHub access token"
+	@echo ""
+	@echo "💡 Suggestions:"
+	@echo "  - Production use: Use docker-buildx* commands (based on precompiled binaries)"
+	@echo "  - Local development: Use docker-dev* commands (build from source)"
+	@echo "  - Development environment: Use dev-env-* commands to manage dev containers"
+
+.PHONY: help
+help:
+	@echo "🦀 RustFS Makefile Help:"
+	@echo ""
+	@echo "📋 Main Command Categories:"
+	@echo "  make help-build                          # Show build-related help"
+	@echo "  make help-docker                         # Show Docker-related help"
+	@echo ""
+	@echo "🔧 Code Quality:"
+	@echo "  make fmt                                 # Format code"
+	@echo "  make clippy                              # Run clippy checks"
+	@echo "  make test                                # Run tests"
+	@echo "  make pre-commit                          # Run all pre-commit checks"
+	@echo ""
+	@echo "🚀 Quick Start:"
+	@echo "  make build                               # Build RustFS binary"
+	@echo "  make docker-dev-local                    # Build development Docker image (local)"
+	@echo "  make dev-env-start                       # Start development environment"
+	@echo ""
+	@echo "💡 For more help use 'make help-build' or 'make help-docker'"

README.md

@@ -1,6 +1,6 @@
-[![RustFS](https://repository-images.githubusercontent.com/722597620/0fa936a2-8164-4f53-867f-def4beb64b21)](https://rustfs.com)
+[![RustFS](https://rustfs.com/images/rustfs-github.png)](https://rustfs.com)
 
-<p align="center">RustFS is a high-performance, distributed object storage system built in Rust.</p>
+<p align="center">RustFS is a high-performance distributed object storage software built using Rust</p>
 
 <p align="center">
   <a href="https://github.com/rustfs/rustfs/actions/workflows/ci.yml"><img alt="CI" src="https://github.com/rustfs/rustfs/actions/workflows/ci.yml/badge.svg" /></a>
@@ -11,12 +11,7 @@
 </p>
 
 <p align="center">
-<a href="https://trendshift.io/repositories/14181" target="_blank"><img src="https://trendshift.io/api/badge/repositories/14181" alt="rustfs%2Frustfs | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/></a> 
-<a href="https://runacap.com/ross-index/q4-2025/" target="_blank" rel="noopener"><img style="width: 260px; height: 55px" src="https://runacap.com/wp-content/uploads/2026/01/ROSS_badge_white_Q4_2025.svg" alt="ROSS Index - Fastest Growing Open-Source Startups in Q4 2025 | Runa Capital" height="55" /></a>
-</p>
-
-<p align="center">
-  <a href="https://docs.rustfs.com/installation/">Getting Started</a>
+  <a href="https://docs.rustfs.com/introduction.html">Getting Started</a>
   · <a href="https://docs.rustfs.com/">Docs</a>
   · <a href="https://github.com/rustfs/rustfs/issues">Bug reports</a>
   · <a href="https://github.com/rustfs/rustfs/discussions">Discussions</a>
@@ -24,6 +19,7 @@
 
 <p align="center">
 English | <a href="https://github.com/rustfs/rustfs/blob/main/README_ZH.md">简体中文</a> |
+  <!-- Keep these links. Translations will automatically update with the README. -->
   <a href="https://readme-i18n.com/rustfs/rustfs?lang=de">Deutsch</a> |
   <a href="https://readme-i18n.com/rustfs/rustfs?lang=es">Español</a> |
   <a href="https://readme-i18n.com/rustfs/rustfs?lang=fr">français</a> |
@@ -33,245 +29,143 @@ English | <a href="https://github.com/rustfs/rustfs/blob/main/README_ZH.md">简
   <a href="https://readme-i18n.com/rustfs/rustfs?lang=ru">Русский</a>
 </p>
 
-RustFS is a high-performance, distributed object storage system built in Rust—one of the most loved programming languages worldwide. RustFS combines the simplicity of MinIO with the memory safety and raw performance of Rust. It offers full S3 compatibility, is completely open-source, and is optimized for data lakes, AI, and big data workloads.
+RustFS is a high-performance distributed object storage software built using Rust, one of the most popular languages
+worldwide. Along with MinIO, it shares a range of advantages such as simplicity, S3 compatibility, open-source nature,
+support for data lakes, AI, and big data. Furthermore, it has a better and more user-friendly open-source license in
+comparison to other storage systems, being constructed under the Apache license. As Rust serves as its foundation,
+RustFS provides faster speed and safer distributed features for high-performance object storage.
 
-Unlike other storage systems, RustFS is released under the permissible Apache 2.0 license, avoiding the restrictions of AGPL. With Rust as its foundation, RustFS delivers superior speed and secure distributed features for next-generation object storage.
+> ⚠️ **RustFS is under rapid development. Do NOT use in production environments!**
 
-## Feature & Status
+## Features
 
-- **High Performance**: Built with Rust to ensure maximum speed and resource efficiency.
-- **Distributed Architecture**: Scalable and fault-tolerant design suitable for large-scale deployments.
-- **S3 Compatibility**: Seamless integration with existing S3-compatible applications and tools.
-- **OpenStack Swift API**: Native support for Swift protocol with Keystone authentication.
-- **OpenStack Keystone Integration**: Native support for OpenStack Keystone authentication with X-Auth-Token headers.
-- **Data Lake Support**: Optimized for high-throughput big data and AI workloads.
-- **Open Source**: Licensed under Apache 2.0, encouraging unrestricted community contributions and commercial usage.
-- **User-Friendly**: Designed with simplicity in mind for easy deployment and management.
+- **High Performance**: Built with Rust, ensuring speed and efficiency.
+- **Distributed Architecture**: Scalable and fault-tolerant design for large-scale deployments.
+- **S3 Compatibility**: Seamless integration with existing S3-compatible applications.
+- **Data Lake Support**: Optimized for big data and AI workloads.
+- **Open Source**: Licensed under Apache 2.0, encouraging community contributions and transparency.
+- **User-Friendly**: Designed with simplicity in mind, making it easy to deploy and manage.
 
-| Feature                 | Status       | Feature                  | Status           |
-| :---------------------- | :----------- | :----------------------- | :--------------- |
-| **S3 Core Features**    | ✅ Available | **Bitrot Protection**    | ✅ Available     |
-| **Upload / Download**   | ✅ Available | **Single Node Mode**     | ✅ Available     |
-| **Versioning**          | ✅ Available | **Bucket Replication**   | ✅ Available     |
-| **Logging**             | ✅ Available | **Lifecycle Management** | 🚧 Under Testing |
-| **Event Notifications** | ✅ Available | **Distributed Mode**     | 🚧 Under Testing |
-| **K8s Helm Charts**     | ✅ Available | **RustFS KMS**           | 🚧 Under Testing |
-| **Keystone Auth**       | ✅ Available | **Multi-Tenancy**        | ✅ Available     |
-| **Swift API**           | ✅ Available | **Swift Metadata Ops**   | 🚧 Partial       |
+## RustFS vs MinIO
 
-## RustFS vs MinIO Performance
+Stress test server parameters
 
-**Stress Test Environment:**
-
-| Type    | Parameter | Remark                                                   |
-| ------- | --------- | -------------------------------------------------------- |
-| CPU     | 2 Core    | Intel Xeon (Sapphire Rapids) Platinum 8475B, 2.7/3.2 GHz |
-| Memory  | 4GB       |                                                          |
-| Network | 15Gbps    |                                                          |
-| Drive   | 40GB x 4  | IOPS 3800 / Drive                                        |
+| Type    | parameter | Remark                                                   |
+|---------|-----------|----------------------------------------------------------|
+| CPU     | 2 Core    | Intel Xeon(Sapphire Rapids) Platinum 8475B , 2.7/3.2 GHz |   |
+| Memory  | 4GB       |                                                          |
+| Network | 15Gbp     |                                                          |
+| Driver  | 40GB x 4  | IOPS 3800 / Driver                                       |
 
 <https://github.com/user-attachments/assets/2e4979b5-260c-4f2c-ac12-c87fd558072a>
 
-### RustFS vs Other Object Storage
+### RustFS vs Other object storage
 
-| Feature                | RustFS                                                                                                                                                | Other Object Storage                                                                     |
-| :--------------------- | :---------------------------------------------------------------------------------------------------------------------------------------------------- | :--------------------------------------------------------------------------------------- |
-| **Console Experience** | **Powerful Console**<br>Comprehensive management interface.                                                                                           | **Basic / Limited Console**<br>Often overly simple or lacking critical features.         |
-| **Language & Safety**  | **Rust-based**<br>Memory safety by design.                                                                                                            | **Go or C-based**<br>Potential for memory GC pauses or leaks.                            |
-| **Data Sovereignty**   | **No Telemetry / Full Compliance**<br>Guards against unauthorized cross-border data egress. Compliant with GDPR (EU/UK), CCPA (US), and APPI (Japan). | **Potential Risk**<br>Possible legal exposure and unwanted data telemetry.               |
-| **Licensing**          | **Permissive Apache 2.0**<br>Business-friendly, no "poison pill" clauses.                                                                             | **Restrictive AGPL v3**<br>Risk of license traps and intellectual property pollution.    |
-| **Compatibility**      | **100% S3 Compatible**<br>Works with any cloud provider or client, anywhere.                                                                          | **Variable Compatibility**<br>May lack support for local cloud vendors or specific APIs. |
-| **Edge & IoT**         | **Strong Edge Support**<br>Ideal for secure, innovative edge devices.                                                                                 | **Weak Edge Support**<br>Often too heavy for edge gateways.                              |
-| **Risk Profile**       | **Enterprise Risk Mitigation**<br>Clear IP rights and safe for commercial use.                                                                        | **Legal Risks**<br>Intellectual property ambiguity and usage restrictions.               |
-
-## Staying ahead
-
-Star RustFS on GitHub and be instantly notified of new releases.
-
-<img src="https://github.com/user-attachments/assets/7ee40bb4-3e46-4eac-b0d0-5fbeb85ff8f3" />
+| RustFS                                                                          | Other object storage                                                                                                    |
+|---------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------|
+| Powerful Console                                                                | Simple and useless Console                                                                                              |
+| Developed based on Rust language, memory is safer                               | Developed in Go or C, with potential issues like memory GC/leaks                                                        |
+| Does not report logs to third-party countries                                   | Reporting logs to other third countries may violate national security laws                                              |
+| Licensed under Apache, more business-friendly                                   | AGPL V3 License and other License, polluted open source and License traps, infringement of intellectual property rights |
+| Comprehensive S3 support, works with domestic and international cloud providers | Full support for S3, but no local cloud vendor support                                                                  |
+| Rust-based development, strong support for secure and innovative devices        | Poor support for edge gateways and secure innovative devices                                                            |
+| Stable commercial prices, free community support                                | High pricing, with costs up to $250,000 for 1PiB                                                                        |
+| No risk                                                                         | Intellectual property risks and risks of prohibited uses                                                                |
 
 ## Quickstart
 
 To get started with RustFS, follow these steps:
 
-### 1. One-click Installation (Option 1)
+1. **One-click installation script (Option 1)​​**
 
-```bash
-curl -O https://rustfs.com/install_rustfs.sh && bash install_rustfs.sh
-```
+  ```bash
+  curl -O  https://rustfs.com/install_rustfs.sh && bash install_rustfs.sh
+  ```
 
-### 2. Docker Quick Start (Option 2)
+2. **Docker Quick Start (Option 2)​​**
 
-The RustFS container runs as a non-root user `rustfs` (UID `10001`). If you run Docker with `-v` to mount a host directory, please ensure the host directory owner is set to `10001`, otherwise you will encounter permission denied errors.
+  ```bash
+   # create data and logs directories
+   mkdir -p data logs
 
-```bash
-# Create data and logs directories
-mkdir -p data logs
+   # using latest alpha version
+   docker run -d -p 9000:9000 -v $(pwd)/data:/data -v $(pwd)/logs:/logs rustfs/rustfs:alpha
 
-# Change the owner of these directories
-chown -R 10001:10001 data logs
+   # Specific version
+   docker run -d -p 9000:9000 -v $(pwd)/data:/data -v $(pwd)/logs:/logs rustfs/rustfs:1.0.0.alpha.45
+   ```
 
-# Using latest version
-docker run -d -p 9000:9000 -p 9001:9001 -v $(pwd)/data:/data -v $(pwd)/logs:/logs rustfs/rustfs:latest
+For docker installation, you can also run the container with docker compose. With the `docker-compose.yml` file under
+root directory, running the command:
 
-# Using specific version
-docker run -d -p 9000:9000 -p 9001:9001 -v $(pwd)/data:/data -v $(pwd)/logs:/logs rustfs/rustfs:1.0.0-alpha.76
-```
+  ```
+  docker compose --profile observability up -d
+  ```
 
-If you use [podman](https://github.com/containers/podman) instead of docker, you can install the RustFS with the below command
+**NOTE**: You should be better to have a look for `docker-compose.yaml` file. Because, several services contains in the
+file. Grafan,prometheus,jaeger containers will be launched using docker compose file, which is helpful for rustfs
+observability. If you want to start redis as well as nginx container, you can specify the corresponding profiles.
 
-```bash
-podman run -d -p 9000:9000 -p 9001:9001 -v $(pwd)/data:/data -v $(pwd)/logs:/logs rustfs/rustfs:latest
-```
+3. **Build from Source (Option 3) - Advanced Users**
 
-You can also use Docker Compose. Using the `docker-compose.yml` file in the root directory:
+   For developers who want to build RustFS Docker images from source with multi-architecture support:
 
-```bash
-docker compose --profile observability up -d
-```
+   ```bash
+   # Build multi-architecture images locally
+   ./docker-buildx.sh --build-arg RELEASE=latest
 
-Similarly, you can run the command with podman
+   # Build and push to registry
+   ./docker-buildx.sh --push
 
-```bash
-podman compose --profile observability up -d
-```
+   # Build specific version
+   ./docker-buildx.sh --release v1.0.0 --push
 
-**NOTE**: We recommend reviewing the `docker-compose.yml` file before running. It defines several services including Grafana, Prometheus, and Jaeger, which are helpful for RustFS observability. If you wish to start Redis or Nginx containers, you can specify the corresponding profiles.
+   # Build for custom registry
+   ./docker-buildx.sh --registry your-registry.com --namespace yourname --push
+   ```
 
-### 3\. Build from Source (Option 3) - Advanced Users
+   The `docker-buildx.sh` script supports:
+    - **Multi-architecture builds**: `linux/amd64`, `linux/arm64`
+    - **Automatic version detection**: Uses git tags or commit hashes
+    - **Registry flexibility**: Supports Docker Hub, GitHub Container Registry, etc.
+    - **Build optimization**: Includes caching and parallel builds
 
-For developers who want to build RustFS Docker images from source with multi-architecture support:
+   You can also use Make targets for convenience:
 
-```bash
-# Build multi-architecture images locally
-./docker-buildx.sh --build-arg RELEASE=latest
+   ```bash
+   make docker-buildx                    # Build locally
+   make docker-buildx-push               # Build and push
+   make docker-buildx-version VERSION=v1.0.0  # Build specific version
+   make help-docker                      # Show all Docker-related commands
+   ```
 
-# Build and push to registry
-./docker-buildx.sh --push
+4. **Build with helm chart(Option 4) - Cloud Native environment**
 
-# Build specific version
-./docker-buildx.sh --release v1.0.0 --push
+   Following the instructions on [helm chart README](./helm/README.md) to install RustFS on kubernetes cluster.
 
-# Build for custom registry
-./docker-buildx.sh --registry your-registry.com --namespace yourname --push
-```
+5. **Access the Console**: Open your web browser and navigate to `http://localhost:9000` to access the RustFS console,
+   default username and password is `rustfsadmin` .
+6. **Create a Bucket**: Use the console to create a new bucket for your objects.
+7. **Upload Objects**: You can upload files directly through the console or use S3-compatible APIs to interact with your
+   RustFS instance.
 
-The `docker-buildx.sh` script supports:
-
-- **Multi-architecture builds**: `linux/amd64`, `linux/arm64`
-- **Automatic version detection**: Uses git tags or commit hashes
-- **Registry flexibility**: Supports Docker Hub, GitHub Container Registry, etc.
-- **Build optimization**: Includes caching and parallel builds
-
-You can also use Make targets for convenience:
-
-```bash
-make docker-buildx                    # Build locally
-make docker-buildx-push               # Build and push
-make docker-buildx-version VERSION=v1.0.0  # Build specific version
-make help-docker                      # Show all Docker-related commands
-```
-
-> **Heads-up (macOS cross-compilation)**: macOS keeps the default `ulimit -n` at 256, so `cargo zigbuild` or `./build-rustfs.sh --platform ...` may fail with `ProcessFdQuotaExceeded` when targeting Linux. The build script attempts to raise the limit automatically, but if you still see the warning, run `ulimit -n 4096` (or higher) in your shell before building.
-
-### 4\. Build with Helm Chart (Option 4) - Cloud Native
-
-Follow the instructions in the [Helm Chart README](https://charts.rustfs.com/) to install RustFS on a Kubernetes cluster.
-
-### 5\. Nix Flake (Option 5)
-
-If you have [Nix with flakes enabled](https://nixos.wiki/wiki/Flakes#Enable_flakes):
-
-```bash
-# Run directly without installing
-nix run github:rustfs/rustfs
-
-# Build the binary
-nix build github:rustfs/rustfs
-./result/bin/rustfs --help
-
-# Or from a local checkout
-nix build
-nix run
-```
-
-### 6\. X-CMD (Option 6)
-
-If you are an [x-cmd](https://www.x-cmd.com/install/rustfs) user:
-
-```bash
-# Run directly without installing
-x rustfs
-
-# Download the binary and install it to the global environment
-x env use rustfs
-rustfs --help
-```
-
----
-
-### Accessing RustFS
-
-1. **Access the Console**: Open your web browser and navigate to `http://localhost:9001` to access the RustFS console.
-    - Default credentials: `rustfsadmin` / `rustfsadmin`
-2. **Create a Bucket**: Use the console to create a new bucket for your objects.
-3. **Upload Objects**: You can upload files directly through the console or use S3-compatible APIs/clients to interact with your RustFS instance.
-
-**NOTE**: To access the RustFS instance via `https`, please refer to the [TLS Configuration Docs](https://docs.rustfs.com/integration/tls-configured.html).
-
-### OIDC Roles Claim (Microsoft Entra ID)
-
-RustFS supports mapping an OIDC claim containing role values into the existing
-authorization pipeline. The `roles_claim` setting is **optional**: when unset or
-empty, only the `groups` claim contributes to authorization (same as older
-RustFS releases). For Microsoft Entra ID app roles, set `roles_claim=roles` so
-both console admin checks and bucket IAM policies can evaluate those roles.
-
-Example environment configuration (opt-in roles claim):
-
-```bash
-RUSTFS_IDENTITY_OPENID_ENABLE=on
-RUSTFS_IDENTITY_OPENID_CONFIG_URL="https://login.microsoftonline.com/<tenant-id>/v2.0/.well-known/openid-configuration"
-RUSTFS_IDENTITY_OPENID_CLIENT_ID="<client-id>"
-RUSTFS_IDENTITY_OPENID_CLIENT_SECRET="<client-secret>"
-RUSTFS_IDENTITY_OPENID_SCOPES="openid,profile,email"
-RUSTFS_IDENTITY_OPENID_GROUPS_CLAIM="groups"
-RUSTFS_IDENTITY_OPENID_ROLES_CLAIM="roles"
-```
-
-Policy condition example (evaluate app roles directly with `jwt:roles`; when
-`roles_claim` is configured, RustFS also merges those values into `jwt:groups`
-for backward compatibility with older policies):
-
-```json
-{
-  "Version": "2012-10-17",
-  "Statement": [
-    {
-      "Effect": "Allow",
-      "Action": ["admin:*"],
-      "Resource": ["arn:aws:s3:::*"],
-      "Condition": {
-        "ForAnyValue:StringEquals": {
-          "jwt:roles": ["RustFS.ConsoleAdmin"]
-        }
-      }
-    }
-  ]
-}
-```
+**NOTE**: If you want to access RustFS instance with `https`, you can refer
+to [TLS configuration docs](https://docs.rustfs.com/integration/tls-configured.html).
 
 ## Documentation
 
-For detailed documentation, including configuration options, API references, and advanced usage, please visit our [Documentation](https://docs.rustfs.com).
+For detailed documentation, including configuration options, API references, and advanced usage, please visit
+our [Documentation](https://docs.rustfs.com).
 
 ## Getting Help
 
-If you have any questions or need assistance:
+If you have any questions or need assistance, you can:
 
 - Check the [FAQ](https://github.com/rustfs/rustfs/discussions/categories/q-a) for common issues and solutions.
-- Join our [GitHub Discussions](https://github.com/rustfs/rustfs/discussions) to ask questions and share your experiences.
-- Open an issue on our [GitHub Issues](https://github.com/rustfs/rustfs/issues) page for bug reports or feature requests.
+- Join our [GitHub Discussions](https://github.com/rustfs/rustfs/discussions) to ask questions and share your
+  experiences.
+- Open an issue on our [GitHub Issues](https://github.com/rustfs/rustfs/issues) page for bug reports or feature
+  requests.
 
 ## Links
 
@@ -282,19 +176,28 @@ If you have any questions or need assistance:
 ## Contact
 
 - **Bugs**: [GitHub Issues](https://github.com/rustfs/rustfs/issues)
-- **Business**: [hello@rustfs.com](mailto:hello@rustfs.com)
-- **Jobs**: [jobs@rustfs.com](mailto:jobs@rustfs.com)
+- **Business**: <hello@rustfs.com>
+- **Jobs**: <jobs@rustfs.com>
 - **General Discussion**: [GitHub Discussions](https://github.com/rustfs/rustfs/discussions)
 - **Contributing**: [CONTRIBUTING.md](CONTRIBUTING.md)
 
 ## Contributors
 
-RustFS is a community-driven project, and we appreciate all contributions. Check out the [Contributors](https://github.com/rustfs/rustfs/graphs/contributors) page to see the amazing people who have helped make RustFS better.
+RustFS is a community-driven project, and we appreciate all contributions. Check out
+the [Contributors](https://github.com/rustfs/rustfs/graphs/contributors) page to see the amazing people who have helped
+make RustFS better.
 
 <a href="https://github.com/rustfs/rustfs/graphs/contributors">
-<img src="https://opencollective.com/rustfs/contributors.svg?width=890&limit=500&button=false" alt="Contributors" />
+  <img src="https://opencollective.com/rustfs/contributors.svg?width=890&limit=500&button=false" alt="Contributors"/>
 </a>
 
+## Github Trending Top
+
+🚀 RustFS is beloved by open-source enthusiasts and enterprise users worldwide, often appearing on the GitHub Trending
+top charts.
+
+<a href="https://trendshift.io/repositories/14181" target="_blank"><img src="https://raw.githubusercontent.com/rustfs/rustfs/refs/heads/main/docs/rustfs-trending.jpg" alt="rustfs%2Frustfs | Trendshift" /></a>
+
 ## Star History
 
 [![Star History Chart](https://api.star-history.com/svg?repos=rustfs/rustfs&type=date&legend=top-left)](https://www.star-history.com/#rustfs/rustfs&type=date&legend=top-left)
@@ -304,3 +207,4 @@ RustFS is a community-driven project, and we appreciate all contributions. Check
 [Apache 2.0](https://opensource.org/licenses/Apache-2.0)
 
 **RustFS** is a trademark of RustFS, Inc. All other trademarks are the property of their respective owners.
+

README_ZH.md

@@ -1,246 +1,180 @@
-[![RustFS](https://github.com/user-attachments/assets/1b5afcd6-a2c3-47ff-8bc3-ce882b0ddca7)](https://rustfs.com.cn)
+[![RustFS](https://rustfs.com/images/rustfs-github.png)](https://rustfs.com)
 
-<p align="center">RustFS 是一个基于 Rust 构建的高性能分布式对象存储系统。</p>
+<p align="center">RustFS 是一个使用 Rust 构建的高性能分布式对象存储软件</p >
 
 <p align="center">
   <a href="https://github.com/rustfs/rustfs/actions/workflows/ci.yml"><img alt="CI" src="https://github.com/rustfs/rustfs/actions/workflows/ci.yml/badge.svg" /></a>
-  <a href="https://github.com/rustfs/rustfs/actions/workflows/docker.yml"><img alt="构建并推送 Docker 镜像" src="https://github.com/rustfs/rustfs/actions/workflows/docker.yml/badge.svg" /></a>
-  <img alt="GitHub 提交活跃度" src="https://img.shields.io/github/commit-activity/m/rustfs/rustfs"/>
-  <img alt="Github 最新提交" src="https://img.shields.io/github/last-commit/rustfs/rustfs"/>
+  <a href="https://github.com/rustfs/rustfs/actions/workflows/docker.yml"><img alt="Build and Push Docker Images" src="https://github.com/rustfs/rustfs/actions/workflows/docker.yml/badge.svg" /></a>
+  <img alt="GitHub commit activity" src="https://img.shields.io/github/commit-activity/m/rustfs/rustfs"/>
+  <img alt="Github Last Commit" src="https://img.shields.io/github/last-commit/rustfs/rustfs"/>
   <a href="https://hellogithub.com/repository/rustfs/rustfs" target="_blank"><img src="https://abroad.hellogithub.com/v1/widgets/recommend.svg?rid=b95bcb72bdc340b68f16fdf6790b7d5b&claim_uid=MsbvjYeLDKAH457&theme=small" alt="Featured｜HelloGitHub" /></a>
-</p>
-
+</p >
 
 <p align="center">
-<a href="https://trendshift.io/repositories/14181" target="_blank"><img src="https://trendshift.io/api/badge/repositories/14181" alt="rustfs%2Frustfs | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/></a> 
-<a href="https://runacap.com/ross-index/q4-2025/" target="_blank" rel="noopener"><img style="width: 260px; height: 55px" src="https://runacap.com/wp-content/uploads/2026/01/ROSS_badge_white_Q4_2025.svg" alt="ROSS Index - Fastest Growing Open-Source Startups in Q4 2025 | Runa Capital" height="55" /></a>
-</p>
+  <a href="https://docs.rustfs.com/zh/introduction.html">快速开始</a >
+  · <a href="https://docs.rustfs.com/zh/">文档</a >
+  · <a href="https://github.com/rustfs/rustfs/issues">问题报告</a >
+  · <a href="https://github.com/rustfs/rustfs/discussions">讨论</a >
+</p >
 
 <p align="center">
-  <a href="https://docs.rustfs.com/installation/">快速开始</a>
-  · <a href="https://docs.rustfs.com/">文档</a>
-  · <a href="https://github.com/rustfs/rustfs/issues">报告 Bug</a>
-  · <a href="https://github.com/rustfs/rustfs/discussions">社区讨论</a>
-</p>
+<a href="https://github.com/rustfs/rustfs/blob/main/README.md">English</a > | 简体中文
+</p >
 
-<p align="center">
-  <a href="https://github.com/rustfs/rustfs/blob/main/README.md">English</a> | 简体中文 |
-  <a href="https://readme-i18n.com/rustfs/rustfs?lang=de">Deutsch</a> |
-  <a href="https://readme-i18n.com/rustfs/rustfs?lang=es">Español</a> |
-  <a href="https://readme-i18n.com/rustfs/rustfs?lang=fr">français</a> |
-  <a href="https://readme-i18n.com/rustfs/rustfs?lang=ja">日本語</a> |
-  <a href="https://readme-i18n.com/rustfs/rustfs?lang=ko">한국어</a> |
-  <a href="https://readme-i18n.com/rustfs/rustfs?lang=pt">Portuguese</a> |
-  <a href="https://readme-i18n.com/rustfs/rustfs?lang=ru">Русский</a>
-</p>
+RustFS 是一个使用 Rust（全球最受欢迎的编程语言之一）构建的高性能分布式对象存储软件。与 MinIO 一样，它具有简单性、S3
+兼容性、开源特性以及对数据湖、AI 和大数据的支持等一系列优势。此外，与其他存储系统相比，它采用 Apache
+许可证构建，拥有更好、更用户友好的开源许可证。由于以 Rust 为基础，RustFS 为高性能对象存储提供了更快的速度和更安全的分布式功能。
 
-RustFS 是一个基于 Rust 构建的高性能分布式对象存储系统。Rust 是全球最受开发者喜爱的编程语言之一，RustFS 完美结合了 MinIO 的简洁性与 Rust 的内存安全及高性能优势。它提供完整的 S3 兼容性，完全开源，并专为数据湖、人工智能（AI）和大数据负载进行了优化。
+## 特性
 
-与其他存储系统不同，RustFS 采用更宽松、商业友好的 Apache 2.0 许可证，避免了 AGPL 协议的限制。以 Rust 为基石，RustFS 为下一代对象存储提供了更快的速度和更安全的分布式特性。
-
-## 特征和功能状态
-
-- **高性能**：基于 Rust 构建，确保极致的速度和资源效率。
+- **高性能**：使用 Rust 构建，确保速度和效率。
 - **分布式架构**：可扩展且容错的设计，适用于大规模部署。
-- **S3 兼容性**：与现有的 S3 兼容应用和工具无缝集成。
-- **数据湖支持**：专为高吞吐量的大数据和 AI 工作负载优化。
-- **完全开源**：采用 Apache 2.0 许可证，鼓励社区贡献和商业使用。
-- **简单易用**：设计简洁，易于部署和管理。
+- **S3 兼容性**：与现有 S3 兼容应用程序无缝集成。
+- **数据湖支持**：针对大数据和 AI 工作负载进行了优化。
+- **开源**：采用 Apache 2.0 许可证，鼓励社区贡献和透明度。
+- **用户友好**：设计简单，易于部署和管理。
 
-| 功能               | 状态    | 功能                    | 状态      |
-| :----------------- | :------ | :---------------------- | :-------- |
-| **S3 核心功能**    | ✅ 可用 | **Bitrot (防数据腐烂)** | ✅ 可用   |
-| **上传 / 下载**    | ✅ 可用 | **单机模式**            | ✅ 可用   |
-| **版本控制**       | ✅ 可用 | **存储桶复制**          | ✅ 可用   |
-| **日志功能**       | ✅ 可用 | **生命周期管理**        | 🚧 测试中 |
-| **事件通知**       | ✅ 可用 | **分布式模式**          | 🚧 测试中 |
-| **K8s Helm Chart** | ✅ 可用 | **OPA (策略引擎)**      | 🚧 测试中 |
+## RustFS vs MinIO
 
-## RustFS vs MinIO 性能对比
+压力测试服务器参数
 
-**压力测试环境参数：**
-
-| 类型 | 参数     | 备注                                                      |
-| ---- | -------- | --------------------------------------------------------- |
-| CPU  | 2 核     | Intel Xeon (Sapphire Rapids) Platinum 8475B , 2.7/3.2 GHz |
-| 内存 | 4GB      |                                                           |
-| 网络 | 15Gbps   |                                                           |
-| 硬盘 | 40GB x 4 | IOPS 3800 / Drive                                         |
+| 类型  | 参数       | 备注                                                       |
+|-----|----------|----------------------------------------------------------|
+| CPU | 2 核心     | Intel Xeon(Sapphire Rapids) Platinum 8475B , 2.7/3.2 GHz |   |
+| 内存  | 4GB      |                                                          |
+| 网络  | 15Gbp    |                                                          |
+| 驱动器 | 40GB x 4 | IOPS 3800 / 驱动器                                          |
 
 <https://github.com/user-attachments/assets/2e4979b5-260c-4f2c-ac12-c87fd558072a>
 
 ### RustFS vs 其他对象存储
 
-| 特性           | RustFS                                                                                                              | 其他对象存储                                                             |
-| :------------- | :------------------------------------------------------------------------------------------------------------------ | :----------------------------------------------------------------------- |
-| **控制台体验** | **功能强大的控制台**<br>提供全面的管理界面。                                                                        | **基础/简陋的控制台**<br>通常功能过于简单或缺失关键特性。                |
-| **语言与安全** | **基于 Rust 开发**<br>天生的内存安全。                                                                              | **基于 Go 或 C 开发**<br>存在内存 GC 停顿或内存泄漏的潜在风险。          |
-| **数据主权**   | **无遥测 / 完全合规**<br>防止未经授权的数据跨境传输。完全符合 GDPR (欧盟/英国)、CCPA (美国) 和 APPI (日本) 等法规。 | **潜在风险**<br>可能存在法律风险和隐蔽的数据遥测（Telemetry）。          |
-| **开源协议**   | **宽松的 Apache 2.0**<br>商业友好，无“毒丸”条款。                                                                   | **受限的 AGPL v3**<br>存在许可证陷阱和知识产权污染的风险。               |
-| **兼容性**     | **100% S3 兼容**<br>适用于任何云提供商和客户端，随处运行。                                                          | **兼容性不一**<br>虽然支持 S3，但可能缺乏对本地云厂商或特定 API 的支持。 |
-| **边缘与 IoT** | **强大的边缘支持**<br>非常适合安全、创新的边缘设备。                                                                | **边缘支持较弱**<br>对于边缘网关来说通常过于沉重。                       |
-| **成本**       | **稳定且免费**<br>免费社区支持，稳定的商业定价。                                                                    | **高昂成本**<br>1PiB 的成本可能高达 250,000 美元。                       |
-| **风险控制**   | **企业级风险规避**<br>清晰的知识产权，商业使用安全无忧。                                                            | **法律风险**<br>知识产权归属模糊及使用限制风险。                         |
-
-## 保持领先
-
-在 GitHub 上为 RustFS 点赞，即可第一时间收到新版本发布通知。
-
-<img src="https://github.com/user-attachments/assets/7ee40bb4-3e46-4eac-b0d0-5fbeb85ff8f3" />
+| RustFS                   | 其他对象存储                              |
+|--------------------------|-------------------------------------|
+| 强大的控制台                   | 简单且无用的控制台                           |
+| 基于 Rust 语言开发，内存更安全       | 使用 Go 或 C 开发，存在内存 GC/泄漏等潜在问题        |
+| 不向第三方国家报告日志              | 向其他第三方国家报告日志可能违反国家安全法律              |
+| 采用 Apache 许可证，对商业更友好     | AGPL V3 许可证等其他许可证，污染开源和许可证陷阱，侵犯知识产权 |
+| 全面的 S3 支持，适用于国内外云提供商     | 完全支持 S3，但不支持本地云厂商                   |
+| 基于 Rust 开发，对安全和创新设备有强大支持 | 对边缘网关和安全创新设备支持较差                    |
+| 稳定的商业价格，免费社区支持           | 高昂的定价，1PiB 成本高达 $250,000            |
+| 无风险                      | 知识产权风险和禁止使用的风险                      |
 
 ## 快速开始
 
-请按照以下步骤快速上手 RustFS：
+要开始使用 RustFS，请按照以下步骤操作：
 
-### 1. 一键安装脚本 (选项 1)
+1. **一键脚本快速启动 (方案一)**
 
-```bash
-curl -O https://rustfs.com/install_rustfs.sh && bash install_rustfs.sh
-```
+   ```bash
+   curl -O  https://rustfs.com/install_rustfs.sh && bash install_rustfs.sh
+   ```
 
-### 2\. Docker 快速启动 (选项 2)
+2. **Docker 快速启动（方案二）**
 
-RustFS 容器以非 root 用户 `rustfs` (UID `10001`) 运行。如果您使用 Docker 的 `-v` 参数挂载宿主机目录，请务必确保宿主机目录的所有者已更改为 `10001`，否则会遇到权限拒绝错误。
+  ```bash
+   docker run -d -p 9000:9000  -v /data:/data rustfs/rustfs
+   ```
 
-```bash
- # 创建数据和日志目录
- mkdir -p data logs
+对于使用 Docker 安装来讲，你还可以使用 `docker compose` 来启动 rustfs 实例。在仓库的根目录下面有一个 `docker-compose.yml`
+文件。运行如下命令即可：
 
- # 更改这两个目录的所有者
- chown -R 10001:10001 data logs
+  ```
+  docker compose --profile observability up -d
+  ```
 
- # 使用最新版本运行
- docker run -d -p 9000:9000 -p 9001:9001 -v $(pwd)/data:/data -v $(pwd)/logs:/logs rustfs/rustfs:latest
+**注意**：在使用 `docker compose` 之前，你应该仔细阅读一下 `docker-compose.yaml`，因为该文件中包含多个服务，除了 rustfs
+以外，还有 grafana、prometheus、jaeger 等，这些是为 rustfs 可观测性服务的，还有 redis 和 nginx。你想启动哪些容器，就需要用
+`--profile` 参数指定相应的 profile。
 
- # 使用指定版本运行
- docker run -d -p 9000:9000 -p 9001:9001 -v $(pwd)/data:/data -v $(pwd)/logs:/logs rustfs/rustfs:1.0.0.alpha.68
-```
+3. **从源码构建（方案三）- 高级用户**
 
-您也可以使用 Docker Compose。使用根目录下的 `docker-compose.yml` 文件：
+   面向希望从源码构建支持多架构 Docker 镜像的开发者：
 
-```bash
-docker compose --profile observability up -d
-```
+   ```bash
+   # 本地构建多架构镜像
+   ./docker-buildx.sh --build-arg RELEASE=latest
 
-**注意**: 我们建议您在运行前查看 `docker-compose.yml` 文件。该文件定义了包括 Grafana、Prometheus 和 Jaeger 在内的多个服务，有助于 RustFS 的可观测性监控。如果您还想启动 Redis 或 Nginx 容器，可以指定相应的 profile。
+   # 构建并推送至镜像仓库
+   ./docker-buildx.sh --push
 
-### 3\. 源码编译 (选项 3) - 进阶用户
+   # 构建指定版本
+   ./docker-buildx.sh --release v1.0.0 --push
 
-适用于希望从源码构建支持多架构 RustFS Docker 镜像的开发者：
+   # 构建并推送到自定义镜像仓库
+   ./docker-buildx.sh --registry your-registry.com --namespace yourname --push
+   ```
 
-```bash
-# 在本地构建多架构镜像
-./docker-buildx.sh --build-arg RELEASE=latest
+   `docker-buildx.sh` 脚本支持：
+    - **多架构构建**：`linux/amd64`、`linux/arm64`
+    - **自动版本检测**：可使用 git 标签或提交哈希
+    - **仓库灵活性**：支持 Docker Hub、GitHub Container Registry 等
+    - **构建优化**：包含缓存和并行构建
 
-# 构建并推送到仓库
-./docker-buildx.sh --push
+   你也可以使用 Makefile 提供的目标命令以提升便捷性：
 
-# 构建指定版本
-./docker-buildx.sh --release v1.0.0 --push
+   ```bash
+   make docker-buildx                    # 本地构建
+   make docker-buildx-push               # 构建并推送
+   make docker-buildx-version VERSION=v1.0.0  # 构建指定版本
+   make help-docker                      # 显示全部 Docker 相关命令
+   ```
 
-# 构建并推送到自定义仓库
-./docker-buildx.sh --registry your-registry.com --namespace yourname --push
-```
+4. **使用 Helm Chart 部署（方案四）- 云原生环境**
 
-`docker-buildx.sh` 脚本支持：
-- **多架构构建**: `linux/amd64`, `linux/arm64`
-- **自动版本检测**: 使用 git tags 或 commit hash
-- **灵活的仓库支持**: 支持 Docker Hub, GitHub Container Registry 等
-- **构建优化**: 包含缓存和并行构建
+   按照 [helm chart 说明文档](./helm/README.md) 的指引，在 Kubernetes 集群中安装 RustFS。
 
-为了方便起见，您也可以使用 Make 命令：
+5. **访问控制台**：打开 Web 浏览器并导航到 `http://localhost:9000` 以访问 RustFS 控制台，默认的用户名和密码是
+   `rustfsadmin` 。
+6. **创建存储桶**：使用控制台为您的对象创建新的存储桶。
+7. **上传对象**：您可以直接通过控制台上传文件，或使用 S3 兼容的 API 与您的 RustFS 实例交互。
 
-```bash
-make docker-buildx                    # 本地构建
-make docker-buildx-push               # 构建并推送
-make docker-buildx-version VERSION=v1.0.0  # 构建指定版本
-make help-docker                      # 显示所有 Docker 相关命令
-```
-
-> **注意 (macOS 交叉编译)**: macOS 默认的 `ulimit -n` 限制为 256，因此在使用 `cargo zigbuild` 或 `./build-rustfs.sh --platform ...` 交叉编译 Linux 版本时，可能会因 `ProcessFdQuotaExceeded` 失败。构建脚本会尝试自动提高限制，但如果您仍然看到警告，请在构建前在终端运行 `ulimit -n 4096` (或更高)。
-
-### 4\. 使用 Helm Chart 安装 (选项 4) - 云原生环境
-
-请按照 [Helm Chart README](https://charts.rustfs.com) 上的说明在 Kubernetes 集群上安装 RustFS。
-
-### 5\. Nix Flake (Option 5)
-
-如果你已经 启用了 [Nix Flakes 功能](https://nixos.wiki/wiki/Flakes#Enable_flakes):
-
-```bash
-# 直接运行，无需安装
-nix run github:rustfs/rustfs
-
-# 编译二进制文件
-nix build github:rustfs/rustfs
-./result/bin/rustfs --help
-
-# 或者从本地检出的代码库运行/编译
-nix build
-nix run
-```
-
-### 6\. X-CMD (Option 6)
-
-如果你是 [x-cmd](https://www.x-cmd.com/install/rustfs) 用户：
-
-```bash
-# 直接运行，无需安装
-x rustfs
-
-# 下载二进制文件并安装到全局环境中
-x env use rustfs
-rustfs --help
-```
-
----
-
-### 访问 RustFS
-
-1. **访问控制台**: 打开浏览器并访问 `http://localhost:9001` 进入 RustFS 控制台。
-   - 默认账号/密码: `rustfsadmin` / `rustfsadmin`
-2. **创建存储桶**: 使用控制台为您​​的对象创建一个新的存储桶 (Bucket)。
-3. **上传对象**: 您可以直接通过控制台上传文件，或使用 S3 兼容的 API/客户端与您的 RustFS 实例进行交互。
-
-**注意**: 如果您希望通过 `https` 访问 RustFS 实例，请参考 [TLS 配置文档](https://docs.rustfs.com/integration/tls-configured.html)。
+**注意**：如果你想通过 `https` 来访问 RustFS 实例，请参考 [TLS 配置文档](https://docs.rustfs.com/zh/integration/tls-configured.html)
 
 ## 文档
 
-有关详细文档，包括配置选项、API 参考和高级用法，请访问我们的 [官方文档](https://docs.rustfs.com)。
+有关详细文档，包括配置选项、API 参考和高级用法，请访问我们的[文档](https://docs.rustfs.com)。
 
 ## 获取帮助
 
-如果您有任何问题或需要帮助：
+如果您有任何问题或需要帮助，您可以：
 
-- 查看 [FAQ](https://github.com/rustfs/rustfs/discussions/categories/q-a) 寻找常见问题和解决方案。
-- 加入我们的 [GitHub Discussions](https://github.com/rustfs/rustfs/discussions) 提问并分享您的经验。
-- 在我们的 [GitHub Issues](https://github.com/rustfs/rustfs/issues) 页面提交 Bug 报告或功能请求。
+- 查看[常见问题解答](https://github.com/rustfs/rustfs/discussions/categories/q-a)以获取常见问题和解决方案。
+- 加入我们的 [GitHub 讨论](https://github.com/rustfs/rustfs/discussions)来提问和分享您的经验。
+- 在我们的 [GitHub Issues](https://github.com/rustfs/rustfs/issues) 页面上开启问题，报告错误或功能请求。
 
 ## 链接
 
-- [官方文档](https://docs.rustfs.com) - 必读手册
-- [更新日志](https://github.com/rustfs/rustfs/releases) - 版本变更记录
-- [社区讨论](https://github.com/rustfs/rustfs/discussions) - 社区交流地
+- [文档](https://docs.rustfs.com) - 您应该阅读的手册
+- [更新日志](https://docs.rustfs.com/changelog) - 我们破坏和修复的内容
+- [GitHub 讨论](https://github.com/rustfs/rustfs/discussions) - 社区所在地
 
-## 联系方式
+## 联系
 
-- **Bug 反馈**: [GitHub Issues](https://github.com/rustfs/rustfs/issues)
-- **商务合作**: [hello@rustfs.com](mailto:hello@rustfs.com)
-- **工作机会**: [jobs@rustfs.com](mailto:jobs@rustfs.com)
-- **一般讨论**: [GitHub Discussions](https://github.com/rustfs/rustfs/discussions)
-- **贡献指南**: [CONTRIBUTING.md](CONTRIBUTING.md)
+- **错误报告**：[GitHub Issues](https://github.com/rustfs/rustfs/issues)
+- **商务合作**：<hello@rustfs.com>
+- **招聘**：<jobs@rustfs.com>
+- **一般讨论**：[GitHub 讨论](https://github.com/rustfs/rustfs/discussions)
+- **贡献**：[CONTRIBUTING.md](CONTRIBUTING.md)
 
 ## 贡献者
 
-RustFS 是一个社区驱动的项目，我们感谢所有的贡献。请查看 [贡献者](https://github.com/rustfs/rustfs/graphs/contributors) 页面，看看那些让 RustFS 变得更好的了不起的人们。
+RustFS 是一个社区驱动的项目，我们感谢所有的贡献。查看[贡献者](https://github.com/rustfs/rustfs/graphs/contributors)页面，了解帮助
+RustFS 变得更好的杰出人员。
 
 <a href="https://github.com/rustfs/rustfs/graphs/contributors">
-<img src="https://opencollective.com/rustfs/contributors.svg?width=890&limit=500&button=false" alt="Contributors" />
-</a>
+  <img src="https://opencollective.com/rustfs/contributors.svg?width=890&limit=500&button=false" alt="贡献者"/>
+</a >
 
-## Star 历史
+## Github 全球推荐榜
 
-[![Star History Chart](https://api.star-history.com/svg?repos=rustfs/rustfs&type=date&legend=top-left)](https://www.star-history.com/#rustfs/rustfs&type=date&legend=top-left)
+🚀 RustFS 受到了全世界开源爱好者和企业用户的喜欢，多次登顶 Github Trending 全球榜。
+
+<a href="https://trendshift.io/repositories/14181" target="_blank"><img src="https://raw.githubusercontent.com/rustfs/rustfs/refs/heads/main/docs/rustfs-trending.jpg" alt="rustfs%2Frustfs | Trendshift" /></a>
+
+## Star 历史图
+
+[![Star 历史图](https://api.star-history.com/svg?repos=rustfs/rustfs&type=date&legend=top-left)](https://www.star-history.com/#rustfs/rustfs&type=date&legend=top-left)
 
 ## 许可证
 

SECURITY.md

@@ -1,40 +1,18 @@
 # Security Policy
 
-## Security Philosophy
-
-At RustFS, we take security seriously. We believe that **transparency leads to better security**. The more open our code is, the more eyes are on it, and the faster we can identify and resolve potential issues.
-
-We highly value the contributions of the security community and welcome anyone to audit our code. Your efforts help us make RustFS safer for everyone.
-
 ## Supported Versions
 
-To help us focus our security efforts, please refer to the table below to see which versions of RustFS are currently supported with security updates.
+Use this section to tell people about which versions of your project are
+currently being supported with security updates.
 
 | Version | Supported          |
 | ------- | ------------------ |
-| Latest  | :white_check_mark: |
-| < 1.0   | :x:                |
+| 1.x.x   | :white_check_mark: |
 
 ## Reporting a Vulnerability
 
-If you discover a security vulnerability in RustFS, we appreciate your help in disclosing it to us responsibly.
+Use this section to tell people how to report a vulnerability.
 
-**Please do not open a public GitHub issue for security vulnerabilities.** Publicly disclosing a vulnerability can put the entire community at risk before a fix is available.
-
-### How to Report
-
-1. https://github.com/rustfs/rustfs/security/advisories/new
-2. Please email us directly at: **security@rustfs.com**
-
-In your email, please include:
-1.  **Description**: A detailed description of the vulnerability.
-2.  **Steps to Reproduce**: Steps or a script to reproduce the issue.
-3.  **Impact**: The potential impact of the vulnerability.
-
-### Our Response Process
-
-1.  **Acknowledgment**: We will acknowledge your email within 48 hours.
-2.  **Assessment**: We will investigate the issue and determine its severity.
-3.  **Fix & Disclosure**: We will work on a patch. Once the patch is released, we will publicly announce the vulnerability and acknowledge your contribution (unless you prefer to remain anonymous).
-
-Thank you for helping keep RustFS and its users safe!
+Tell them where to go, how often they can expect to get an update on a
+reported vulnerability, what to expect if the vulnerability is accepted or
+declined, etc.

_typos.toml

@@ -36,15 +36,6 @@ clen = "clen"
 datas = "datas"
 bre = "bre"
 abd = "abd"
-mak = "mak"
-gae = "gae"
-GAE = "GAE"
-thr = "thr"
-# s3-tests original test names (cannot be changed)
-nonexisted = "nonexisted"
-consts = "consts"
-# Swift API - company/product names
-Hashi = "Hashi"  # HashiCorp
 
 [files]
-extend-exclude = []
+extend-exclude = []

build-rustfs.sh

@@ -1,4 +1,4 @@
-#!/usr/bin/env bash
+#!/bin/bash
 
 # RustFS Binary Build Script
 # This script compiles RustFS binaries for different platforms and architectures
@@ -31,7 +31,7 @@ detect_platform() {
                     echo "armv7-unknown-linux-gnueabihf"
                     ;;
                 "loongarch64")
-                    echo "loongarch64-unknown-linux-gnu"
+                    echo "loongarch64-unknown-linux-musl"
                     ;;
                 *)
                     echo "unknown-platform"
@@ -126,7 +126,6 @@ usage() {
     echo "                              Supported platforms:"
     echo "                                x86_64-unknown-linux-gnu"
     echo "                                aarch64-unknown-linux-gnu"
-    echo "                                loongarch64-unknown-linux-gnu"
     echo "                                armv7-unknown-linux-gnueabihf"
     echo "                                x86_64-unknown-linux-musl"
     echo "                                aarch64-unknown-linux-musl"
@@ -164,35 +163,6 @@ print_message() {
     echo -e "${color}${message}${NC}"
 }
 
-# Prevent zig/ld from hitting macOS file descriptor defaults during linking
-ensure_file_descriptor_limit() {
-    local required_limit=4096
-    local current_limit
-    current_limit=$(ulimit -Sn 2>/dev/null || echo "")
-
-    if [ -z "$current_limit" ] || [ "$current_limit" = "unlimited" ]; then
-        return
-    fi
-
-    if (( current_limit >= required_limit )); then
-        return
-    fi
-
-    local hard_limit target_limit
-    hard_limit=$(ulimit -Hn 2>/dev/null || echo "")
-    target_limit=$required_limit
-
-    if [ -n "$hard_limit" ] && [ "$hard_limit" != "unlimited" ] && (( hard_limit < required_limit )); then
-        target_limit=$hard_limit
-    fi
-
-    if ulimit -Sn "$target_limit" 2>/dev/null; then
-        print_message $YELLOW "🔧 Increased open file limit from $current_limit to $target_limit to avoid ProcessFdQuotaExceeded"
-    else
-        print_message $YELLOW "⚠️ Unable to raise ulimit -n automatically (current: $current_limit, needed: $required_limit). Please run 'ulimit -n $required_limit' manually before building."
-    fi
-}
-
 # Get version from git
 get_version() {
     if git describe --abbrev=0 --tags >/dev/null 2>&1; then
@@ -213,7 +183,7 @@ setup_rust_environment() {
     # Set up environment variables for musl targets
     if [[ "$PLATFORM" == *"musl"* ]]; then
         print_message $YELLOW "Setting up environment for musl target..."
-        export RUSTFLAGS="'--cfg tokio_unstable -C target-feature=-crt-static'"
+        export RUSTFLAGS="-C target-feature=-crt-static"
 
         # For cargo-zigbuild, set up additional environment variables
         if command -v cargo-zigbuild &> /dev/null; then
@@ -430,7 +400,7 @@ build_binary() {
         fi
     else
         # Native compilation
-        build_cmd="RUSTFLAGS='--cfg tokio_unstable -Clink-arg=-lm' cargo build"
+        build_cmd="RUSTFLAGS=-Clink-arg=-lm cargo build"
     fi
 
     if [ "$BUILD_TYPE" = "release" ]; then
@@ -600,11 +570,10 @@ main() {
         fi
     fi
 
-    ensure_file_descriptor_limit
-
     # Start build process
     build_rustfs
 }
 
 # Run main function
 main
+

crates/AGENTS.md

@@ -1,20 +0,0 @@
-# Crates Instructions
-
-Applies to all paths under `crates/`.
-
-## Library Design
-
-- Treat crate code as reusable library code by default.
-- Prefer `thiserror` for library-facing error types.
-- Do not use `unwrap()`, `expect()`, or panic-driven control flow outside tests.
-
-## Testing
-
-- Keep unit tests close to the module they test.
-- Keep integration tests under each crate's `tests/` directory.
-- Add regression tests for bug fixes and behavior changes.
-
-## Async and Performance
-
-- Keep async paths non-blocking.
-- Move CPU-heavy operations out of async hot paths with `tokio::task::spawn_blocking` when appropriate.

crates/ahm/Cargo.toml

@@ -0,0 +1,43 @@
+[package]
+name = "rustfs-ahm"
+version.workspace = true
+edition.workspace = true
+authors = ["RustFS Team"]
+license.workspace = true
+description = "RustFS AHM (Automatic Health Management) Scanner"
+repository.workspace = true
+rust-version.workspace = true
+homepage.workspace = true
+documentation = "https://docs.rs/rustfs-ahm/latest/rustfs_ahm/"
+keywords = ["RustFS", "AHM", "health-management", "scanner", "Minio"]
+categories = ["web-programming", "development-tools", "filesystem"]
+
+[dependencies]
+rustfs-ecstore = { workspace = true }
+rustfs-common = { workspace = true }
+rustfs-filemeta = { workspace = true }
+rustfs-madmin = { workspace = true }
+tokio = { workspace = true, features = ["full"] }
+tokio-util = { workspace = true }
+tracing = { workspace = true }
+serde = { workspace = true, features = ["derive"] }
+time = { workspace = true }
+serde_json = { workspace = true }
+thiserror = { workspace = true }
+uuid = { workspace = true, features = ["v4", "serde"] }
+anyhow = { workspace = true }
+async-trait = { workspace = true }
+futures = { workspace = true }
+s3s = { workspace = true }
+chrono = { workspace = true }
+rand = { workspace = true }
+reqwest = { workspace = true }
+tempfile = { workspace = true }
+walkdir = { workspace = true }
+
+[dev-dependencies]
+serde_json = { workspace = true }
+serial_test = { workspace = true }
+tracing-subscriber = { workspace = true }
+tempfile = { workspace = true }
+heed = { workspace = true }

crates/ahm/src/error.rs

@@ -14,9 +14,9 @@
 
 use thiserror::Error;
 
-/// Custom error type for heal operations
+/// Custom error type for AHM operations
 /// This enum defines various error variants that can occur during
-/// the execution of heal-related tasks, such as I/O errors, storage errors,
+/// the execution of AHM-related tasks, such as I/O errors, storage errors,
 /// configuration errors, and specific errors related to healing operations.
 #[derive(Debug, Error)]
 pub enum Error {
@@ -41,6 +41,13 @@ pub enum Error {
     #[error(transparent)]
     Anyhow(#[from] anyhow::Error),
 
+    // Scanner
+    #[error("Scanner error: {0}")]
+    Scanner(String),
+
+    #[error("Metrics error: {0}")]
+    Metrics(String),
+
     #[error("Serialization error: {0}")]
     Serialization(String),
 
@@ -53,6 +60,7 @@ pub enum Error {
     #[error("Invalid checkpoint: {0}")]
     InvalidCheckpoint(String),
 
+    // Heal
     #[error("Heal task not found: {task_id}")]
     TaskNotFound { task_id: String },
 
@@ -68,9 +76,6 @@ pub enum Error {
     #[error("Invalid heal type: {heal_type}")]
     InvalidHealType { heal_type: String },
 
-    #[error("Transient heal skip: {message}")]
-    TransientSkip { message: String },
-
     #[error("Heal task cancelled")]
     TaskCancelled,
 
@@ -84,7 +89,9 @@ pub enum Error {
     ProgressTrackingFailed { message: String },
 }
 
-/// A specialized Result type for heal operations
+/// A specialized Result type for AHM operations
+///This type is a convenient alias for results returned by functions in the AHM crate,
+/// using the custom Error type defined above.
 pub type Result<T, E = Error> = std::result::Result<T, E>;
 
 impl Error {
@@ -95,11 +102,6 @@ impl Error {
     {
         Error::Other(error.into().to_string())
     }
-
-    /// Create a transient skip error for retryable background heal checks.
-    pub fn transient_skip(message: impl Into<String>) -> Self {
-        Error::TransientSkip { message: message.into() }
-    }
 }
 
 impl From<Error> for std::io::Error {

crates/ahm/src/heal/channel.rs

@@ -19,11 +19,11 @@ use crate::heal::{
 };
 use crate::{Error, Result};
 use rustfs_common::heal_channel::{
-    HealAdmissionResult, HealChannelCommand, HealChannelPriority, HealChannelReceiver, HealChannelRequest, HealChannelResponse,
-    HealScanMode, publish_heal_response,
+    HealChannelCommand, HealChannelPriority, HealChannelReceiver, HealChannelRequest, HealChannelResponse, HealScanMode,
+    publish_heal_response,
 };
 use std::sync::Arc;
-use tokio::sync::{mpsc, oneshot};
+use tokio::sync::mpsc;
 use tracing::{debug, error, info};
 
 /// Heal channel processor
@@ -82,80 +82,42 @@ impl HealChannelProcessor {
     /// Process heal command
     async fn process_command(&self, command: HealChannelCommand) -> Result<()> {
         match command {
-            HealChannelCommand::Start { request, response_tx } => self.process_start_request(request, response_tx).await,
+            HealChannelCommand::Start(request) => self.process_start_request(request).await,
             HealChannelCommand::Query { heal_path, client_token } => self.process_query_request(heal_path, client_token).await,
             HealChannelCommand::Cancel { heal_path } => self.process_cancel_request(heal_path).await,
         }
     }
 
     /// Process start request
-    async fn process_start_request(
-        &self,
-        request: HealChannelRequest,
-        response_tx: oneshot::Sender<std::result::Result<HealAdmissionResult, String>>,
-    ) -> Result<()> {
-        info!(
-            "Processing heal start request: {} for bucket: {}/{}",
-            request.id,
-            request.bucket,
-            request.object_prefix.as_deref().unwrap_or("")
-        );
+    async fn process_start_request(&self, request: HealChannelRequest) -> Result<()> {
+        info!("Processing heal start request: {} for bucket: {}", request.id, request.bucket);
 
         // Convert channel request to heal request
-        let heal_request = match self.convert_to_heal_request(request.clone()) {
-            Ok(heal_request) => heal_request,
-            Err(err) => {
-                let error_text = err.to_string();
-                let _ = response_tx.send(Err(error_text.clone()));
-                self.publish_response(HealChannelResponse {
-                    request_id: request.id,
-                    success: false,
-                    data: None,
-                    error: Some(error_text),
-                });
-                return Ok(());
-            }
-        };
+        let heal_request = self.convert_to_heal_request(request.clone())?;
 
         // Submit to heal manager
         match self.heal_manager.submit_heal_request(heal_request).await {
-            Ok(admission) => {
-                info!(
-                    request_id = %request.id,
-                    admission = admission.result_label(),
-                    "Heal request admission decision completed"
-                );
-
-                let _ = response_tx.send(Ok(admission));
-
-                let (success, error) = match admission {
-                    HealAdmissionResult::Accepted | HealAdmissionResult::Merged => (true, None),
-                    HealAdmissionResult::Full => (false, Some("Heal request queue is full".to_string())),
-                    HealAdmissionResult::Dropped(reason) => (false, Some(format!("Heal request dropped: {}", reason.as_str()))),
-                };
+            Ok(task_id) => {
+                info!("Successfully submitted heal request: {} as task: {}", request.id, task_id);
 
                 let response = HealChannelResponse {
                     request_id: request.id,
-                    success,
-                    data: Some(
-                        format!("admission={},reason={}", admission.result_label(), admission.reason_label()).into_bytes(),
-                    ),
-                    error,
+                    success: true,
+                    data: Some(format!("Task ID: {task_id}").into_bytes()),
+                    error: None,
                 };
 
                 self.publish_response(response);
             }
             Err(e) => {
-                let error_text = e.to_string();
-                error!("Failed to submit heal request: {} - {}", request.id, error_text);
-                let _ = response_tx.send(Err(error_text.clone()));
+                error!("Failed to submit heal request: {} - {}", request.id, e);
 
                 // Send error response
                 let response = HealChannelResponse {
                     request_id: request.id,
                     success: false,
                     data: None,
-                    error: Some(error_text),
+                    error: Some(e.to_string()),
                 };
 
                 self.publish_response(response);
@@ -216,7 +178,7 @@ impl HealChannelProcessor {
                 HealType::Object {
                     bucket: request.bucket.clone(),
                     object: prefix.clone(),
-                    version_id: request.object_version_id.clone(),
+                    version_id: None,
                 }
             } else {
                 HealType::Bucket {
@@ -280,9 +242,8 @@ impl HealChannelProcessor {
 #[cfg(test)]
 mod tests {
     use super::*;
-    use crate::heal::manager::HealConfig;
     use crate::heal::storage::HealStorageAPI;
-    use rustfs_common::heal_channel::{HealAdmissionResult, HealChannelPriority, HealChannelRequest, HealScanMode};
+    use rustfs_common::heal_channel::{HealChannelPriority, HealChannelRequest, HealScanMode};
     use std::sync::Arc;
 
     // Mock storage for testing
@@ -363,14 +324,6 @@ mod tests {
         async fn list_objects_for_heal(&self, _bucket: &str, _prefix: &str) -> crate::Result<Vec<String>> {
             Ok(vec![])
         }
-        async fn list_objects_for_heal_page(
-            &self,
-            _bucket: &str,
-            _prefix: &str,
-            _continuation_token: Option<&str>,
-        ) -> crate::Result<(Vec<String>, Option<String>, bool)> {
-            Ok((vec![], None, false))
-        }
         async fn get_disk_for_resume(&self, _set_disk_id: &str) -> crate::Result<rustfs_ecstore::disk::DiskStore> {
             Err(crate::Error::other("Not implemented in mock"))
         }
@@ -400,7 +353,6 @@ mod tests {
             id: "test-id".to_string(),
             bucket: "test-bucket".to_string(),
             object_prefix: None,
-            object_version_id: None,
             disk: None,
             priority: HealChannelPriority::Normal,
             scan_mode: None,
@@ -429,7 +381,6 @@ mod tests {
             id: "test-id".to_string(),
             bucket: "test-bucket".to_string(),
             object_prefix: Some("test-object".to_string()),
-            object_version_id: None,
             disk: None,
             priority: HealChannelPriority::High,
             scan_mode: Some(HealScanMode::Deep),
@@ -461,7 +412,6 @@ mod tests {
             id: "test-id".to_string(),
             bucket: "test-bucket".to_string(),
             object_prefix: None,
-            object_version_id: None,
             disk: Some("pool_0_set_1".to_string()),
             priority: HealChannelPriority::Critical,
             scan_mode: None,
@@ -490,7 +440,6 @@ mod tests {
             id: "test-id".to_string(),
             bucket: "test-bucket".to_string(),
             object_prefix: None,
-            object_version_id: None,
             disk: Some("invalid-disk-id".to_string()),
             priority: HealChannelPriority::Normal,
             scan_mode: None,
@@ -526,7 +475,6 @@ mod tests {
                 id: "test-id".to_string(),
                 bucket: "test-bucket".to_string(),
                 object_prefix: None,
-                object_version_id: None,
                 disk: None,
                 priority: channel_priority,
                 scan_mode: None,
@@ -555,7 +503,6 @@ mod tests {
             id: "test-id".to_string(),
             bucket: "test-bucket".to_string(),
             object_prefix: None,
-            object_version_id: None,
             disk: None,
             priority: HealChannelPriority::Normal,
             scan_mode: None,
@@ -585,7 +532,6 @@ mod tests {
             id: "test-id".to_string(),
             bucket: "test-bucket".to_string(),
             object_prefix: Some("".to_string()), // Empty prefix should be treated as bucket heal
-            object_version_id: None,
             disk: None,
             priority: HealChannelPriority::Normal,
             scan_mode: None,
@@ -603,92 +549,4 @@ mod tests {
         let heal_request = processor.convert_to_heal_request(channel_request).unwrap();
         assert!(matches!(heal_request.heal_type, HealType::Bucket { .. }));
     }
-
-    #[tokio::test]
-    async fn test_process_start_request_returns_admission_result() {
-        let storage: Arc<dyn HealStorageAPI> = Arc::new(MockStorage);
-        let manager = Arc::new(HealManager::new(
-            storage,
-            Some(HealConfig {
-                queue_size: 1,
-                ..HealConfig::default()
-            }),
-        ));
-        let processor = HealChannelProcessor::new(manager);
-
-        let request = HealChannelRequest {
-            id: "admission-id".to_string(),
-            bucket: "bucket".to_string(),
-            object_prefix: Some("object".to_string()),
-            object_version_id: None,
-            disk: None,
-            priority: HealChannelPriority::Low,
-            scan_mode: Some(HealScanMode::Normal),
-            remove_corrupted: None,
-            recreate_missing: None,
-            update_parity: None,
-            recursive: None,
-            dry_run: None,
-            timeout_seconds: None,
-            pool_index: None,
-            set_index: None,
-            force_start: false,
-        };
-
-        let (tx, rx) = oneshot::channel();
-        processor
-            .process_start_request(request.clone(), tx)
-            .await
-            .expect("first admission should succeed");
-        assert_eq!(
-            rx.await
-                .expect("oneshot should resolve")
-                .expect("admission should be returned"),
-            HealAdmissionResult::Accepted
-        );
-
-        let (tx, rx) = oneshot::channel();
-        processor
-            .process_start_request(request, tx)
-            .await
-            .expect("duplicate admission should succeed");
-        assert_eq!(
-            rx.await
-                .expect("oneshot should resolve")
-                .expect("admission should be returned"),
-            HealAdmissionResult::Merged
-        );
-    }
-
-    #[tokio::test]
-    async fn test_process_start_request_returns_error_on_invalid_request() {
-        let heal_manager = create_test_heal_manager();
-        let processor = HealChannelProcessor::new(heal_manager);
-
-        let request = HealChannelRequest {
-            id: "invalid-id".to_string(),
-            bucket: "bucket".to_string(),
-            object_prefix: None,
-            object_version_id: None,
-            disk: Some("invalid".to_string()),
-            priority: HealChannelPriority::Normal,
-            scan_mode: None,
-            remove_corrupted: None,
-            recreate_missing: None,
-            update_parity: None,
-            recursive: None,
-            dry_run: None,
-            timeout_seconds: None,
-            pool_index: None,
-            set_index: None,
-            force_start: false,
-        };
-
-        let (tx, rx) = oneshot::channel();
-        processor
-            .process_start_request(request, tx)
-            .await
-            .expect("processor should surface invalid request through response channel");
-        assert!(rx.await.expect("oneshot should resolve").is_err());
-    }
 }

crates/ahm/src/heal/erasure_healer.rs

@@ -18,15 +18,11 @@ use crate::heal::{
     storage::HealStorageAPI,
 };
 use crate::{Error, Result};
-use futures::{StreamExt, future::join_all, stream::FuturesUnordered};
-use metrics::gauge;
+use futures::future::join_all;
 use rustfs_common::heal_channel::{HealOpts, HealScanMode};
 use rustfs_ecstore::disk::DiskStore;
-use std::sync::{
-    Arc,
-    atomic::{AtomicUsize, Ordering},
-};
-use tokio::sync::{RwLock, Semaphore};
+use std::sync::Arc;
+use tokio::sync::RwLock;
 use tracing::{error, info, warn};
 
 /// Erasure Set Healer
@@ -38,29 +34,6 @@ pub struct ErasureSetHealer {
 }
 
 impl ErasureSetHealer {
-    fn page_parallel_enabled() -> bool {
-        rustfs_utils::get_env_bool(
-            rustfs_config::ENV_HEAL_PAGE_PARALLEL_ENABLE,
-            rustfs_config::DEFAULT_HEAL_PAGE_PARALLEL_ENABLE,
-        )
-    }
-
-    fn heal_page_object_concurrency() -> usize {
-        rustfs_utils::get_env_usize(
-            rustfs_config::ENV_HEAL_PAGE_OBJECT_CONCURRENCY,
-            rustfs_config::DEFAULT_HEAL_PAGE_OBJECT_CONCURRENCY,
-        )
-        .max(1)
-    }
-
-    fn effective_heal_page_object_concurrency() -> usize {
-        if Self::page_parallel_enabled() {
-            Self::heal_page_object_concurrency()
-        } else {
-            1
-        }
-    }
-
     pub fn new(
         storage: Arc<dyn HealStorageAPI>,
         progress: Arc<RwLock<HealProgress>>,
@@ -76,9 +49,8 @@ impl ErasureSetHealer {
     }
 
     /// execute erasure set heal with resume
-    #[tracing::instrument(skip(self, buckets), fields(set_disk_id = %set_disk_id, bucket_count = buckets.len()))]
     pub async fn heal_erasure_set(&self, buckets: &[String], set_disk_id: &str) -> Result<()> {
-        info!("Starting erasure set heal");
+        info!("Starting erasure set heal for {} buckets on set disk {}", buckets.len(), set_disk_id);
 
         // 1. generate or get task id
         let task_id = self.get_or_create_task_id(set_disk_id).await?;
@@ -88,7 +60,7 @@ impl ErasureSetHealer {
 
         // 3. execute heal with resume
         let result = self
-            .execute_heal_with_resume(buckets, set_disk_id, &resume_manager, &checkpoint_manager)
+            .execute_heal_with_resume(buckets, &resume_manager, &checkpoint_manager)
             .await;
 
         // 4. cleanup resume state
@@ -171,7 +143,6 @@ impl ErasureSetHealer {
     async fn execute_heal_with_resume(
         &self,
         buckets: &[String],
-        set_disk_id: &str,
         resume_manager: &ResumeManager,
         checkpoint_manager: &CheckpointManager,
     ) -> Result<()> {
@@ -210,7 +181,6 @@ impl ErasureSetHealer {
             let bucket_result = self
                 .heal_bucket_with_resume(
                     bucket,
-                    set_disk_id,
                     bucket_idx,
                     &mut current_object_index,
                     &mut processed_objects,
@@ -261,21 +231,19 @@ impl ErasureSetHealer {
 
     /// heal single bucket with resume
     #[allow(clippy::too_many_arguments)]
-    #[tracing::instrument(skip(self, current_object_index, processed_objects, successful_objects, failed_objects, skipped_objects, resume_manager, checkpoint_manager), fields(bucket = %bucket, bucket_index = bucket_index))]
     async fn heal_bucket_with_resume(
         &self,
         bucket: &str,
-        set_disk_id: &str,
         bucket_index: usize,
         current_object_index: &mut usize,
         processed_objects: &mut u64,
         successful_objects: &mut u64,
         failed_objects: &mut u64,
-        skipped_objects: &mut u64,
+        _skipped_objects: &mut u64,
         resume_manager: &ResumeManager,
         checkpoint_manager: &CheckpointManager,
     ) -> Result<()> {
-        info!(target: "rustfs:heal:heal_bucket_with_resume" ,"Starting heal for bucket from object index {}", current_object_index);
+        info!(target: "rustfs:ahm:heal_bucket_with_resume" ,"Starting heal for bucket: {} from object index {}", bucket, current_object_index);
 
         // 1. get bucket info
         let _bucket_info = match self.storage.get_bucket_info(bucket).await? {
@@ -286,186 +254,82 @@ impl ErasureSetHealer {
             }
         };
 
-        // 2. process objects with pagination to avoid loading all objects into memory
-        let mut continuation_token: Option<String> = None;
-        let mut global_obj_idx = 0usize;
-        let page_concurrency_limit = Self::effective_heal_page_object_concurrency();
-        let in_flight = Arc::new(AtomicUsize::new(0));
+        // 2. get objects to heal
+        let objects = self.storage.list_objects_for_heal(bucket, "").await?;
 
-        loop {
-            // Get one page of objects
-            let (objects, next_token, is_truncated) = self
-                .storage
-                .list_objects_for_heal_page(bucket, "", continuation_token.as_deref())
+        // 3. continue from checkpoint
+        for (obj_idx, object) in objects.iter().enumerate().skip(*current_object_index) {
+            // check if already processed
+            if checkpoint_manager.get_checkpoint().await.processed_objects.contains(object) {
+                continue;
+            }
+
+            // update current object
+            resume_manager
+                .set_current_item(Some(bucket.to_string()), Some(object.clone()))
                 .await?;
-            let checkpoint = checkpoint_manager.get_checkpoint().await;
-            let page_resume_index = *current_object_index;
-            let semaphore = Arc::new(Semaphore::new(page_concurrency_limit));
-            let mut page_tasks = FuturesUnordered::new();
 
-            for object in objects {
-                let object_idx = global_obj_idx;
-                global_obj_idx += 1;
-
-                if object_idx < *current_object_index {
+            // Check if object still exists before attempting heal
+            let object_exists = match self.storage.object_exists(bucket, object).await {
+                Ok(exists) => exists,
+                Err(e) => {
+                    warn!("Failed to check existence of {}/{}: {}, skipping", bucket, object, e);
+                    *current_object_index = obj_idx + 1;
                     continue;
                 }
+            };
 
-                if checkpoint.processed_objects.contains(&object) || checkpoint.skipped_objects.contains(&object) {
-                    continue;
+            if !object_exists {
+                info!(
+                    target: "rustfs:ahm:heal_bucket_with_resume" ,"Object {}/{} no longer exists, skipping heal (likely deleted intentionally)",
+                    bucket, object
+                );
+                checkpoint_manager.add_processed_object(object.clone()).await?;
+                *successful_objects += 1; // Treat as successful - object is gone as intended
+                *current_object_index = obj_idx + 1;
+                continue;
+            }
+
+            // heal object
+            let heal_opts = HealOpts {
+                scan_mode: HealScanMode::Normal,
+                remove: true,
+                recreate: true, // Keep recreate enabled for legitimate heal scenarios
+                ..Default::default()
+            };
+
+            match self.storage.heal_object(bucket, object, None, &heal_opts).await {
+                Ok((_result, None)) => {
+                    *successful_objects += 1;
+                    checkpoint_manager.add_processed_object(object.clone()).await?;
+                    info!("Successfully healed object {}/{}", bucket, object);
                 }
+                Ok((_, Some(err))) => {
+                    *failed_objects += 1;
+                    checkpoint_manager.add_failed_object(object.clone()).await?;
+                    warn!("Failed to heal object {}/{}: {}", bucket, object, err);
+                }
+                Err(err) => {
+                    *failed_objects += 1;
+                    checkpoint_manager.add_failed_object(object.clone()).await?;
+                    warn!("Error healing object {}/{}: {}", bucket, object, err);
+                }
+            }
 
-                resume_manager
-                    .set_current_item(Some(bucket.to_string()), Some(object.clone()))
+            *processed_objects += 1;
+            *current_object_index = obj_idx + 1;
+
+            // check cancel status
+            if self.cancel_token.is_cancelled() {
+                info!("Heal task cancelled during object processing");
+                return Err(Error::TaskCancelled);
+            }
+
+            // save checkpoint periodically
+            if obj_idx % 100 == 0 {
+                checkpoint_manager
+                    .update_position(bucket_index, *current_object_index)
                     .await?;
-
-                let storage = self.storage.clone();
-                let bucket_name = bucket.to_string();
-                let object_name = object.clone();
-                let cancel_token = self.cancel_token.clone();
-                let in_flight = in_flight.clone();
-                let set_label = set_disk_id.to_string();
-                let permit = semaphore
-                    .clone()
-                    .acquire_owned()
-                    .await
-                    .map_err(|e| Error::other(format!("Failed to acquire page concurrency permit: {e}")))?;
-
-                let current_in_flight = in_flight.fetch_add(1, Ordering::SeqCst) + 1;
-                gauge!(
-                    "rustfs_heal_page_concurrency_current",
-                    "set" => set_label.clone()
-                )
-                .set(current_in_flight as f64);
-
-                page_tasks.push(async move {
-                    let _permit = permit;
-                    let result = if cancel_token.is_cancelled() {
-                        Err(Error::TaskCancelled)
-                    } else {
-                        let object_exists = match storage.object_exists(&bucket_name, &object_name).await {
-                            Ok(exists) => exists,
-                            Err(err @ Error::TransientSkip { .. }) => {
-                                let current = in_flight.fetch_sub(1, Ordering::SeqCst) - 1;
-                                gauge!(
-                                    "rustfs_heal_page_concurrency_current",
-                                    "set" => set_label.clone()
-                                )
-                                .set(current as f64);
-                                return (object_name, Err(err));
-                            }
-                            Err(err) => {
-                                let object_name_for_error = object_name.clone();
-                                let current = in_flight.fetch_sub(1, Ordering::SeqCst) - 1;
-                                gauge!(
-                                    "rustfs_heal_page_concurrency_current",
-                                    "set" => set_label.clone()
-                                )
-                                .set(current as f64);
-                                return (
-                                    object_name,
-                                    Err(Error::other(format!(
-                                        "Failed to check existence of {}/{}: {}",
-                                        bucket_name, object_name_for_error, err
-                                    ))),
-                                );
-                            }
-                        };
-
-                        if !object_exists {
-                            Ok(false)
-                        } else {
-                            let heal_opts = HealOpts {
-                                scan_mode: HealScanMode::Normal,
-                                remove: true,
-                                recreate: true,
-                                ..Default::default()
-                            };
-                            match storage.heal_object(&bucket_name, &object_name, None, &heal_opts).await {
-                                Ok((_result, None)) => Ok(true),
-                                Ok((_, Some(err))) => Err(Error::other(err)),
-                                Err(err) => Err(err),
-                            }
-                        }
-                    };
-
-                    let current = in_flight.fetch_sub(1, Ordering::SeqCst) - 1;
-                    gauge!(
-                        "rustfs_heal_page_concurrency_current",
-                        "set" => set_label.clone()
-                    )
-                    .set(current as f64);
-
-                    (object_name, result)
-                });
-            }
-
-            let mut completed_in_page = 0usize;
-            while let Some((object, result)) = page_tasks.next().await {
-                match result {
-                    Ok(true) => {
-                        *successful_objects += 1;
-                        checkpoint_manager.add_processed_object(object.clone()).await?;
-                        info!("Successfully healed object {}/{}", bucket, object);
-                    }
-                    Ok(false) => {
-                        checkpoint_manager.add_processed_object(object.clone()).await?;
-                        *successful_objects += 1;
-                        info!(
-                            target: "rustfs:heal:heal_bucket_with_resume" ,"Object {}/{} no longer exists, skipping heal (likely deleted intentionally)",
-                            bucket, object
-                        );
-                    }
-                    Err(Error::TaskCancelled) => {
-                        gauge!(
-                            "rustfs_heal_page_concurrency_current",
-                            "set" => set_disk_id.to_string()
-                        )
-                        .set(0.0);
-                        return Err(Error::TaskCancelled);
-                    }
-                    Err(Error::TransientSkip { message }) => {
-                        *skipped_objects += 1;
-                        checkpoint_manager.add_skipped_object(object.clone()).await?;
-                        warn!(
-                            "Skipping heal for object {}/{} due to transient existence check error: {}",
-                            bucket, object, message
-                        );
-                    }
-                    Err(err) => {
-                        *failed_objects += 1;
-                        checkpoint_manager.add_failed_object(object.clone()).await?;
-                        warn!("Error healing object {}/{}: {}", bucket, object, err);
-                    }
-                }
-
-                *processed_objects += 1;
-                completed_in_page += 1;
-
-                if completed_in_page.is_multiple_of(100) {
-                    checkpoint_manager.update_position(bucket_index, page_resume_index).await?;
-                }
-            }
-
-            *current_object_index = global_obj_idx;
-            checkpoint_manager
-                .update_position(bucket_index, *current_object_index)
-                .await?;
-            gauge!(
-                "rustfs_heal_page_concurrency_current",
-                "set" => set_disk_id.to_string()
-            )
-            .set(0.0);
-
-            // Check if there are more pages
-            if !is_truncated {
-                break;
-            }
-
-            continuation_token = next_token;
-            if continuation_token.is_none() {
-                warn!("List is truncated but no continuation token provided for {}", bucket);
-                break;
             }
         }
 
@@ -499,7 +363,7 @@ impl ErasureSetHealer {
                 let _permit = semaphore
                     .acquire()
                     .await
-                    .map_err(|e| Error::other(format!("Failed to acquire semaphore for bucket heal: {e}")))?;
+                    .map_err(|e| Error::other(format!("Failed to acquire semaphore for bucket heal: {}", e)))?;
 
                 if cancel_token.is_cancelled() {
                     return Err(Error::TaskCancelled);
@@ -531,12 +395,16 @@ impl ErasureSetHealer {
             }
         };
 
-        // 2. process objects with pagination to avoid loading all objects into memory
-        let mut continuation_token: Option<String> = None;
-        let mut total_scanned = 0u64;
-        let mut total_success = 0u64;
-        let mut total_failed = 0u64;
+        // 2. get objects to heal
+        let objects = storage.list_objects_for_heal(bucket, "").await?;
 
+        // 3. update progress
+        {
+            let mut p = progress.write().await;
+            p.objects_scanned += objects.len() as u64;
+        }
+
+        // 4. heal objects concurrently
         let heal_opts = HealOpts {
             scan_mode: HealScanMode::Normal,
             remove: true,   // remove corrupted data
@@ -544,65 +412,27 @@ impl ErasureSetHealer {
             ..Default::default()
         };
 
-        loop {
-            // Get one page of objects
-            let (objects, next_token, is_truncated) = storage
-                .list_objects_for_heal_page(bucket, "", continuation_token.as_deref())
-                .await?;
+        let object_results = Self::heal_objects_concurrently(storage, bucket, &objects, &heal_opts, progress).await;
 
-            let page_count = objects.len() as u64;
-            total_scanned += page_count;
+        // 5. count results
+        let (success_count, failure_count) = object_results
+            .into_iter()
+            .fold((0, 0), |(success, failure), result| match result {
+                Ok(_) => (success + 1, failure),
+                Err(_) => (success, failure + 1),
+            });
 
-            // 3. update progress
-            {
-                let mut p = progress.write().await;
-                p.objects_scanned = total_scanned;
-            }
-
-            // 4. heal objects concurrently for this page
-            let object_results = Self::heal_objects_concurrently(storage, bucket, &objects, &heal_opts, progress).await;
-
-            // 5. count results for this page
-            let (success_count, failure_count) =
-                object_results
-                    .into_iter()
-                    .fold((0, 0), |(success, failure), result| match result {
-                        Ok(_) => (success + 1, failure),
-                        Err(_) => (success, failure + 1),
-                    });
-
-            total_success += success_count;
-            total_failed += failure_count;
-
-            // 6. update progress
-            {
-                let mut p = progress.write().await;
-                p.objects_healed = total_success;
-                p.objects_failed = total_failed;
-                p.set_current_object(Some(format!("processing bucket: {bucket} (page)")));
-            }
-
-            // Check if there are more pages
-            if !is_truncated {
-                break;
-            }
-
-            continuation_token = next_token;
-            if continuation_token.is_none() {
-                warn!("List is truncated but no continuation token provided for {}", bucket);
-                break;
-            }
-        }
-
-        // 7. final progress update
+        // 6. update progress
         {
             let mut p = progress.write().await;
+            p.objects_healed += success_count;
+            p.objects_failed += failure_count;
             p.set_current_object(Some(format!("completed bucket: {bucket}")));
         }
 
         info!(
-            "Completed heal for bucket {}: {} success, {} failures (total scanned: {})",
-            bucket, total_success, total_failed, total_scanned
+            "Completed heal for bucket {}: {} success, {} failures",
+            bucket, success_count, failure_count
         );
 
         Ok(())
@@ -631,7 +461,7 @@ impl ErasureSetHealer {
                 let _permit = semaphore
                     .acquire()
                     .await
-                    .map_err(|e| Error::other(format!("Failed to acquire semaphore for object heal: {e}")))?;
+                    .map_err(|e| Error::other(format!("Failed to acquire semaphore for object heal: {}", e)))?;
 
                 match storage.heal_object(&bucket, &object, None, &heal_opts).await {
                     Ok((_result, None)) => {
@@ -674,34 +504,3 @@ impl ErasureSetHealer {
         Ok(())
     }
 }
-
-#[cfg(test)]
-mod tests {
-    use super::ErasureSetHealer;
-
-    #[test]
-    fn heal_page_object_concurrency_uses_default_when_env_is_unset() {
-        temp_env::with_var_unset(rustfs_config::ENV_HEAL_PAGE_OBJECT_CONCURRENCY, || {
-            assert_eq!(
-                ErasureSetHealer::heal_page_object_concurrency(),
-                rustfs_config::DEFAULT_HEAL_PAGE_OBJECT_CONCURRENCY
-            );
-        });
-    }
-
-    #[test]
-    fn heal_page_object_concurrency_respects_env_override() {
-        temp_env::with_var(rustfs_config::ENV_HEAL_PAGE_OBJECT_CONCURRENCY, Some("11"), || {
-            assert_eq!(ErasureSetHealer::heal_page_object_concurrency(), 11);
-        });
-    }
-
-    #[test]
-    fn effective_heal_page_object_concurrency_disables_parallelism_when_flag_is_off() {
-        temp_env::with_var(rustfs_config::ENV_HEAL_PAGE_PARALLEL_ENABLE, Some("false"), || {
-            temp_env::with_var(rustfs_config::ENV_HEAL_PAGE_OBJECT_CONCURRENCY, Some("11"), || {
-                assert_eq!(ErasureSetHealer::effective_heal_page_object_concurrency(), 1);
-            });
-        });
-    }
-}

crates/ahm/src/heal/manager.rs

@@ -0,0 +1,440 @@
+// Copyright 2024 RustFS Team
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+use crate::heal::{
+    progress::{HealProgress, HealStatistics},
+    storage::HealStorageAPI,
+    task::{HealOptions, HealPriority, HealRequest, HealTask, HealTaskStatus, HealType},
+};
+use crate::{Error, Result};
+use rustfs_ecstore::disk::DiskAPI;
+use rustfs_ecstore::disk::error::DiskError;
+use rustfs_ecstore::global::GLOBAL_LOCAL_DISK_MAP;
+use std::{
+    collections::{HashMap, VecDeque},
+    sync::Arc,
+    time::{Duration, SystemTime},
+};
+use tokio::{
+    sync::{Mutex, RwLock},
+    time::interval,
+};
+use tokio_util::sync::CancellationToken;
+use tracing::{error, info, warn};
+
+/// Heal config
+#[derive(Debug, Clone)]
+pub struct HealConfig {
+    /// Whether to enable auto heal
+    pub enable_auto_heal: bool,
+    /// Heal interval
+    pub heal_interval: Duration,
+    /// Maximum concurrent heal tasks
+    pub max_concurrent_heals: usize,
+    /// Task timeout
+    pub task_timeout: Duration,
+    /// Queue size
+    pub queue_size: usize,
+}
+
+impl Default for HealConfig {
+    fn default() -> Self {
+        Self {
+            enable_auto_heal: true,
+            heal_interval: Duration::from_secs(10), // 10 seconds
+            max_concurrent_heals: 4,
+            task_timeout: Duration::from_secs(300), // 5 minutes
+            queue_size: 1000,
+        }
+    }
+}
+
+/// Heal state
+#[derive(Debug, Default)]
+pub struct HealState {
+    /// Whether running
+    pub is_running: bool,
+    /// Current heal cycle
+    pub current_cycle: u64,
+    /// Last heal time
+    pub last_heal_time: Option<SystemTime>,
+    /// Total healed objects
+    pub total_healed_objects: u64,
+    /// Total heal failures
+    pub total_heal_failures: u64,
+    /// Current active heal tasks
+    pub active_heal_count: usize,
+}
+
+/// Heal manager
+pub struct HealManager {
+    /// Heal config
+    config: Arc<RwLock<HealConfig>>,
+    /// Heal state
+    state: Arc<RwLock<HealState>>,
+    /// Active heal tasks
+    active_heals: Arc<Mutex<HashMap<String, Arc<HealTask>>>>,
+    /// Heal queue
+    heal_queue: Arc<Mutex<VecDeque<HealRequest>>>,
+    /// Storage layer interface
+    storage: Arc<dyn HealStorageAPI>,
+    /// Cancel token
+    cancel_token: CancellationToken,
+    /// Statistics
+    statistics: Arc<RwLock<HealStatistics>>,
+}
+
+impl HealManager {
+    /// Create new HealManager
+    pub fn new(storage: Arc<dyn HealStorageAPI>, config: Option<HealConfig>) -> Self {
+        let config = config.unwrap_or_default();
+        Self {
+            config: Arc::new(RwLock::new(config)),
+            state: Arc::new(RwLock::new(HealState::default())),
+            active_heals: Arc::new(Mutex::new(HashMap::new())),
+            heal_queue: Arc::new(Mutex::new(VecDeque::new())),
+            storage,
+            cancel_token: CancellationToken::new(),
+            statistics: Arc::new(RwLock::new(HealStatistics::new())),
+        }
+    }
+
+    /// Start HealManager
+    pub async fn start(&self) -> Result<()> {
+        let mut state = self.state.write().await;
+        if state.is_running {
+            warn!("HealManager is already running");
+            return Ok(());
+        }
+        state.is_running = true;
+        drop(state);
+
+        info!("Starting HealManager");
+
+        // start scheduler
+        self.start_scheduler().await?;
+
+        // start auto disk scanner
+        self.start_auto_disk_scanner().await?;
+
+        info!("HealManager started successfully");
+        Ok(())
+    }
+
+    /// Stop HealManager
+    pub async fn stop(&self) -> Result<()> {
+        info!("Stopping HealManager");
+
+        // cancel all tasks
+        self.cancel_token.cancel();
+
+        // wait for all tasks to complete
+        let mut active_heals = self.active_heals.lock().await;
+        for task in active_heals.values() {
+            if let Err(e) = task.cancel().await {
+                warn!("Failed to cancel task {}: {}", task.id, e);
+            }
+        }
+        active_heals.clear();
+
+        // update state
+        let mut state = self.state.write().await;
+        state.is_running = false;
+
+        info!("HealManager stopped successfully");
+        Ok(())
+    }
+
+    /// Submit heal request
+    pub async fn submit_heal_request(&self, request: HealRequest) -> Result<String> {
+        let config = self.config.read().await;
+        let mut queue = self.heal_queue.lock().await;
+
+        if queue.len() >= config.queue_size {
+            return Err(Error::ConfigurationError {
+                message: "Heal queue is full".to_string(),
+            });
+        }
+
+        let request_id = request.id.clone();
+        queue.push_back(request);
+        drop(queue);
+
+        info!("Submitted heal request: {}", request_id);
+        Ok(request_id)
+    }
+
+    /// Get task status
+    pub async fn get_task_status(&self, task_id: &str) -> Result<HealTaskStatus> {
+        let active_heals = self.active_heals.lock().await;
+        if let Some(task) = active_heals.get(task_id) {
+            Ok(task.get_status().await)
+        } else {
+            Err(Error::TaskNotFound {
+                task_id: task_id.to_string(),
+            })
+        }
+    }
+
+    /// Get task progress
+    pub async fn get_active_tasks_count(&self) -> usize {
+        self.active_heals.lock().await.len()
+    }
+
+    pub async fn get_task_progress(&self, task_id: &str) -> Result<HealProgress> {
+        let active_heals = self.active_heals.lock().await;
+        if let Some(task) = active_heals.get(task_id) {
+            Ok(task.get_progress().await)
+        } else {
+            Err(Error::TaskNotFound {
+                task_id: task_id.to_string(),
+            })
+        }
+    }
+
+    /// Cancel task
+    pub async fn cancel_task(&self, task_id: &str) -> Result<()> {
+        let mut active_heals = self.active_heals.lock().await;
+        if let Some(task) = active_heals.get(task_id) {
+            task.cancel().await?;
+            active_heals.remove(task_id);
+            info!("Cancelled heal task: {}", task_id);
+            Ok(())
+        } else {
+            Err(Error::TaskNotFound {
+                task_id: task_id.to_string(),
+            })
+        }
+    }
+
+    /// Get statistics
+    pub async fn get_statistics(&self) -> HealStatistics {
+        self.statistics.read().await.clone()
+    }
+
+    /// Get active task count
+    pub async fn get_active_task_count(&self) -> usize {
+        let active_heals = self.active_heals.lock().await;
+        active_heals.len()
+    }
+
+    /// Get queue length
+    pub async fn get_queue_length(&self) -> usize {
+        let queue = self.heal_queue.lock().await;
+        queue.len()
+    }
+
+    /// Start scheduler
+    async fn start_scheduler(&self) -> Result<()> {
+        let config = self.config.clone();
+        let heal_queue = self.heal_queue.clone();
+        let active_heals = self.active_heals.clone();
+        let cancel_token = self.cancel_token.clone();
+        let statistics = self.statistics.clone();
+        let storage = self.storage.clone();
+
+        tokio::spawn(async move {
+            let mut interval = interval(config.read().await.heal_interval);
+
+            loop {
+                tokio::select! {
+                    _ = cancel_token.cancelled() => {
+                        info!("Heal scheduler received shutdown signal");
+                        break;
+                    }
+                    _ = interval.tick() => {
+                        Self::process_heal_queue(&heal_queue, &active_heals, &config, &statistics, &storage).await;
+                    }
+                }
+            }
+        });
+
+        Ok(())
+    }
+
+    /// Start background task to auto scan local disks and enqueue erasure set heal requests
+    async fn start_auto_disk_scanner(&self) -> Result<()> {
+        let config = self.config.clone();
+        let heal_queue = self.heal_queue.clone();
+        let active_heals = self.active_heals.clone();
+        let cancel_token = self.cancel_token.clone();
+        let storage = self.storage.clone();
+
+        tokio::spawn(async move {
+            let mut interval = interval(config.read().await.heal_interval);
+
+            loop {
+                tokio::select! {
+                    _ = cancel_token.cancelled() => {
+                        info!("Auto disk scanner received shutdown signal");
+                        break;
+                    }
+                    _ = interval.tick() => {
+                        // Build list of endpoints that need healing
+                        let mut endpoints = Vec::new();
+                        for (_, disk_opt) in GLOBAL_LOCAL_DISK_MAP.read().await.iter() {
+                            if let Some(disk) = disk_opt {
+                                // detect unformatted disk via get_disk_id()
+                                if let Err(err) = disk.get_disk_id().await {
+                                    if err == DiskError::UnformattedDisk {
+                                        endpoints.push(disk.endpoint());
+                                        continue;
+                                    }
+                                }
+                            }
+                        }
+
+                        if endpoints.is_empty() {
+                            continue;
+                        }
+
+                        // Get bucket list for erasure set healing
+                        let buckets = match storage.list_buckets().await {
+                            Ok(buckets) => buckets.iter().map(|b| b.name.clone()).collect::<Vec<String>>(),
+                            Err(e) => {
+                                error!("Failed to get bucket list for auto healing: {}", e);
+                                continue;
+                            }
+                        };
+
+                        // Create erasure set heal requests for each endpoint
+                        for ep in endpoints {
+                            let Some(set_disk_id) =
+                                crate::heal::utils::format_set_disk_id_from_i32(ep.pool_idx, ep.set_idx)
+                            else {
+                                warn!("Skipping endpoint {} without valid pool/set index", ep);
+                                continue;
+                            };
+                            // skip if already queued or healing
+                            // Use consistent lock order: queue first, then active_heals to avoid deadlock
+                            let mut skip = false;
+                            {
+                                let queue = heal_queue.lock().await;
+                                if queue.iter().any(|req| {
+                                    matches!(
+                                        &req.heal_type,
+                                        crate::heal::task::HealType::ErasureSet { set_disk_id: queued_id, .. }
+                                        if queued_id == &set_disk_id
+                                    )
+                                }) {
+                                    skip = true;
+                                }
+                            }
+                            if !skip {
+                                let active = active_heals.lock().await;
+                                if active.values().any(|task| {
+                                    matches!(
+                                        &task.heal_type,
+                                        crate::heal::task::HealType::ErasureSet { set_disk_id: active_id, .. }
+                                        if active_id == &set_disk_id
+                                    )
+                                }) {
+                                    skip = true;
+                                }
+                            }
+
+                            if skip {
+                                continue;
+                            }
+
+                            // enqueue erasure set heal request for this disk
+                            let req = HealRequest::new(
+                                HealType::ErasureSet {
+                                    buckets: buckets.clone(),
+                                    set_disk_id: set_disk_id.clone(),
+                                },
+                                HealOptions::default(),
+                                HealPriority::Normal,
+                            );
+                            let mut queue = heal_queue.lock().await;
+                            queue.push_back(req);
+                            info!("Enqueued auto erasure set heal for endpoint: {} (set_disk_id: {})", ep, set_disk_id);
+                        }
+                    }
+                }
+            }
+        });
+        Ok(())
+    }
+
+    /// Process heal queue
+    async fn process_heal_queue(
+        heal_queue: &Arc<Mutex<VecDeque<HealRequest>>>,
+        active_heals: &Arc<Mutex<HashMap<String, Arc<HealTask>>>>,
+        config: &Arc<RwLock<HealConfig>>,
+        statistics: &Arc<RwLock<HealStatistics>>,
+        storage: &Arc<dyn HealStorageAPI>,
+    ) {
+        let config = config.read().await;
+        let mut active_heals_guard = active_heals.lock().await;
+
+        // check if new heal tasks can be started
+        if active_heals_guard.len() >= config.max_concurrent_heals {
+            return;
+        }
+
+        let mut queue = heal_queue.lock().await;
+        if let Some(request) = queue.pop_front() {
+            let task = Arc::new(HealTask::from_request(request, storage.clone()));
+            let task_id = task.id.clone();
+            active_heals_guard.insert(task_id.clone(), task.clone());
+            drop(active_heals_guard);
+            let active_heals_clone = active_heals.clone();
+            let statistics_clone = statistics.clone();
+
+            // start heal task
+            tokio::spawn(async move {
+                info!("Starting heal task: {}", task_id);
+                let result = task.execute().await;
+                match result {
+                    Ok(_) => {
+                        info!("Heal task completed successfully: {}", task_id);
+                    }
+                    Err(e) => {
+                        error!("Heal task failed: {} - {}", task_id, e);
+                    }
+                }
+                let mut active_heals_guard = active_heals_clone.lock().await;
+                if let Some(completed_task) = active_heals_guard.remove(&task_id) {
+                    // update statistics
+                    let mut stats = statistics_clone.write().await;
+                    match completed_task.get_status().await {
+                        HealTaskStatus::Completed => {
+                            stats.update_task_completion(true);
+                        }
+                        _ => {
+                            stats.update_task_completion(false);
+                        }
+                    }
+                    stats.update_running_tasks(active_heals_guard.len() as u64);
+                }
+            });
+
+            // update statistics
+            let mut stats = statistics.write().await;
+            stats.total_tasks += 1;
+        }
+    }
+}
+
+impl std::fmt::Debug for HealManager {
+    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
+        f.debug_struct("HealManager")
+            .field("config", &"<config>")
+            .field("state", &"<state>")
+            .field("active_heals_count", &"<active_heals>")
+            .field("queue_length", &"<queue>")
+            .finish()
+    }
+}