refactor(ecstore): migrate the bucket bandwidth monitor into InstanceContext (Phase 5 Slice 10) (#4489)

Phase 5 Slice 9 (backlog#939): move the lifecycle expiry state and transition
state — two stateful service handles defined in bucket_lifecycle_ops — into the
per-instance InstanceContext, following the lazy-materialization pattern from
Slices 7-8.

- InstanceContext gains `expiry_state: OnceLock<Arc<RwLock<ExpiryState>>>` and
  `transition_state: OnceLock<Arc<TransitionState>>`, each materialized lazily
  on first access (both constructors are cheap — empty worker vecs/channels,
  workers started later — so lazy init reproduces the eager static's
  create-once-then-share).
- The lifecycle owner helpers `get_global_expiry_state` / `get_global_transition_state`
  and the runtime-source handles `expiry_state_handle` / `transition_state_handle`
  keep their signatures and route through the current instance's context; the
  two statics (and the now-unused lazy_static import) are removed. The scanner
  boundary and lifecycle-state arch guards still pass.

Tests: expiry/transition state are stable within an instance (same Arc) and
distinct across instances.

Verification: cargo test -p rustfs-ecstore (20 instance-context tests green),
cargo clippy -p rustfs-ecstore --all-targets (clean), make pre-commit (pass).

Refs: backlog#939 (Phase 5, Slice 9).

.agents/skills/adversarial-validation/SKILL.md

@@ -1,274 +0,0 @@
----
-name: adversarial-validation
-description: Execute the Adversarial Validation policy from the root AGENTS.md — run the seven reviewer roles (correctness, simplicity, security, concurrency/durability, compatibility, performance, test coverage) with RustFS-specific attack probes. Use on every behavior-affecting code change, bug fix, or design proposal before declaring it done.
----
-
-# Adversarial Validation Playbooks
-
-The policy — risk tiers, role list, protocol, exit criteria — lives in the
-root `AGENTS.md` under "Adversarial Validation (Default On)". Read it first;
-this skill does not restate it. This file adds the RustFS-specific probe
-playbook for each role: concrete attacks, where they apply, and the real
-shipped bug or rule that earns each probe its place.
-
-## How to run a role
-
-1. Pick the tier and the applicable roles per the root `AGENTS.md`.
-2. Run each role as an independent pass over the final diff — a parallel
-   reviewer agent where the tooling supports it, otherwise a fresh
-   sequential pass that starts from the diff and the nearest scoped
-   `AGENTS.md`, discarding the writing session's assumptions.
-3. Within a role, execute the probes whose domain the diff touches, plus any
-   attack the diff obviously invites that no probe lists — the playbook is a
-   floor, not a ceiling.
-4. Report findings (concrete failure scenario or named missing test, with
-   file:line) or the role's null report: "attacked X, Y, Z — no break
-   found". A bare pass is not a result.
-
-## Role playbooks
-
-### Correctness adversary
-
-- For any change touching error aggregation or quorum decisions, build the exact disk-error slice at the quorum boundary: N disks where successes == quorum, then flip one success to an error (quorum-1) and separately inject None/nil placeholder entries into the slice. Trace whether reduce_errs (or the new equivalent) picks the placeholder as the dominant error or lets quorum-1 pass as success. Also check heal/write paths: does a per-target failure at quorum-1 return an explicit error, or silently degrade to success?
-  - Where: crates/ecstore/src/disk/error_reduce.rs; crates/ecstore/src/set_disk/{core,ops}; crates/heal
-  - Evidence: Commit 20d61c73b 'stop reduce_errs leaking nil placeholder as dominant error' (#4551) and 47c1e730c 'make erasure heal write quorum best-effort per target' (#4545); crates/ecstore/AGENTS.md: 'Do not weaken quorum checks... Prefer explicit failure over silent data corruption or implicit success.'
-- For any change in EC read/reconstruct/streaming code, trace the mid-stream error path: the first K shards read fine, then a shard turns out bitrot-corrupt or inconsistent after N bytes have already been sent to the client. Verify the error propagates as a stream error (client sees failure), not a clean end-of-body — a silently truncated GET body is data corruption. Also re-check byte accounting: sum of per-part bytes vs object size, and partNumber-to-offset routing for the first and last part.
-  - Where: crates/ecstore/src/set_disk/read.rs (reconstruct-read, inconsistent_source_indexes handling ~line 3827); codec streaming paths in crates/ecstore/src/erasure_coding
-  - Evidence: Known live bug: EC reconstruct-read failing on inconsistent shards mid-stream silently truncates GET body → client 'unexpected EOF'; commit 15808254d 'correct codec-streaming byte accounting and partNumber routing' (#4535).
-- For any listing/pagination change, construct the exact-boundary inputs: (a) exactly max_keys/max_uploads matching entries — assert the response contains max and is_truncated=false, then max+1 entries — assert exactly max returned with is_truncated=true and a correct continuation marker; (b) a delimiter listing where folding into CommonPrefixes re-fills a full page; (c) an object 'a' coexisting with prefix dir 'a/'. Off-by-one and dropped-truncation bugs live exactly at these boundaries.
-  - Where: crates/ecstore listing/merge paths (metacache, list_objects, list_multipart_uploads); rustfs/src/storage
-  - Evidence: Three recent real bugs: fefa70b31 'stop ListMultipartUploads from returning one upload past max-uploads' (#4447), d91f4d455 'report truncation when delimiter list re-folds a full page' (#4538), 7e1f7f242 'preserve CommonPrefixes when an object and same-named prefix dir coexist' (#4563).
-- Run the diff's logic with a directory object key (trailing slash, e.g. 'pre/dir/') as input. Check which layer encodes/decodes __XLDIR__ — set_disk never sees the trailing slash, so any trailing-slash branch added below the store layer is dead code and a wrong-layer bug. For delete paths, check whether options force a nil versionId onto directory keys: the resulting miss surfaces as version-not-found, not object-not-found, so callers matching only ObjectNotFound leak ghost directory entries.
-  - Where: crates/ecstore/src/store*.rs (store layer) vs crates/ecstore/src/set_disk/*; delete option construction (del_opts) and its callers
-  - Evidence: trailing-slash branches must live at store layer; del_opts injects nil version for dir keys, PR#4220 ghost-directory cleanup never fired on the real path (rustfs#4307, backlog#798 still OPEN).
-- Feed every new binary-UUID metadata read the three degenerate values: key absent, zero-length bytes, and 16 zero bytes (nil UUID). All three must mean 'no value' — any path that produces Uuid::nil() and then acts on it (e.g. sends it as a versionId) is a finding. For tier code specifically: with remote-tier version None or "", assert the tier GET/DELETE request carries no versionId parameter at all — sending versionId="" or nil yields NoSuchVersion against unversioned tier buckets.
-  - Where: crates/filemeta/src/filemeta/version.rs; crates/ecstore/src/bucket/lifecycle/; crates/ecstore/src/services/tier/; any new consumer of crates/utils/src/http/metadata_compat.rs
-  - Evidence: AGENTS.md Cross-Cutting Domain Invariants (defensive Uuid read pattern + unversioned-tier rule); docs/operations/tier-ilm-debugging.md ('nil transition_ver_id = corrupt legacy write-back, readers must filter'); commit 726f3dc18 'accept empty remote version_id in tier recovery paths' (#4552).
-- For each match/if-let on an error or algorithm enum touched by the diff, enumerate what the wildcard/else arm swallows. Inject the variants the author didn't think of — DiskNotFound during listing, an unsupported checksum algorithm, an Err from a cleanup rename/delete — and trace whether they degrade into 'not found', a wrong-but-plausible value, or silent success. Any error path that converges with the success path without logging and propagating is a finding.
-  - Where: crates/ecstore (listing, delete/rename cleanup); crates/checksums; error-mapping layers in rustfs/src/storage
-  - Evidence: Three recent real bugs of this exact shape: e0619e355 'stop treating DiskNotFound as object not-found in listing' (#4536), afaf8c681 'reject md5 instead of silently returning crc32' (#4513), f7d2b2563 'propagate disk delete/rename failures instead of swallowing them' (#4546).
-- For any change to version ordering, index lookup, or shard/part indexing: (a) call the accessor with index == len() and len()-1 — a get_idx-style bound must reject, not panic or wrap; (b) construct two versions with identical mod-times and check the sort tie-break is total and deterministic (equal keys must not compare as both before each other); (c) for EC shard math, compute shard size for object sizes 0, 1, blockSize-1, blockSize, blockSize+1 and cross-check total reconstructed length against the object size.
-  - Where: crates/filemeta/src/filemeta/*.rs (version sort, get_idx); crates/ecstore/src/erasure_coding shard-size math
-  - Evidence: Commit 8bfb00bc0 'guard get_idx bound and fix sorts_before tie-break' (#4509) — both bug classes shipped before; ecstore AGENTS.md high-risk designation for read/write/repair correctness.
-- Exercise the zero/empty end of every new size or count parameter: zero-length object PUT then GET (body must be empty, not error), part count 0, empty Vec of disks/entries into aggregation functions, and env/config values of 0 (must clamp or reject, never divide-by-zero or 'scan nothing and report zero usage'). Anywhere the diff computes a ratio, capacity, or progress percentage, plug in 0 and the max value.
-  - Where: crates/ecstore aggregation and scanner paths; crates/object-capacity; config/env parsing in touched crates
-  - Evidence: Commits 787cc77a7 'clamp zero capacity env values to safe defaults' (#4559) and 32b1094ec 'resolve a symlinked scan root instead of silently counting zero' (#4564) — zero-as-silent-wrong-answer is a recurring repo bug class.
-- For any diff touching multipart or object commit paths, order the operations on paper and attack the failure point between them: kill the process (or return Err) after the commit rename but before cleanup, and after cleanup but before commit. Verify the earlier-failure case leaves the object readable and the later-failure case leaves no half-visible object; part meta files must never be deleted before the commit is durable.
-  - Where: crates/ecstore multipart commit/cleanup (set_disk/ops); rustfs/src/storage multipart handlers
-  - Evidence: Commit c77c5f047 'defer multipart part.N.meta cleanup until after commit' (#4548) — cleanup-before-commit ordering already caused a real data-loss window; the #4221 durability work shows fsync/ordering bugs are endemic here.
-
-Null report example: "Attacked quorum-1 error reduction, exact max-keys listing boundary, trailing-slash dir keys, nil-UUID tier versionId, and mid-stream reconstruct error propagation — no break found."
-
-### Simplicity adversary
-
-- Smaller-diff attack: rewrite the diff's change mentally (or actually, in scratch) as the minimal in-place edit and compare. Flag as findings: a helper function with exactly one caller introduced by this diff; a file rewrite where a 3-line edit inside the existing control flow suffices; reshaped control flow in init/locking/metadata/quorum paths beyond what the fix requires; new string literals duplicating existing constants (grep the token first); #[path] module inclusion. If the smaller diff achieves identical behavior, report it with the concrete replacement.
-  - Where: Any diff; extra scrutiny for crates/ecstore, crates/lock, rustfs/src/storage where 'preserve the existing control-flow shape' is an explicit rule
-  - Evidence: AGENTS.md 'Change Style for Existing Logic' (one-off helper ban, preserve control-flow shape in distributed/locking/metadata paths, no #[path]) and 'Reuse Before You Write' (constants clause); the Adversarial Validation roles list charters the simplicity adversary with exactly this attack.
-- Reuse-and-necessity attack: for each new helper the diff introduces, run `ls crates/utils/src crates/common/src` and `rg -i 'fn \w*<term>'` over those dirs plus the touched crate (snake_case signatures — a full-text single-word grep drowns, a multi-word phrase returns nothing). A reimplementation of an existing workspace utility, or of plain std/tokio behavior no wrapper refines, is a finding — but so is forced reuse with mismatched semantics (normalization such as `clean` resolving `.`/`..` against raw S3 keys, error type, backoff, durability gating). For each new defensive branch, demand the nameable trigger and flag re-validation of what a validated upstream layer on the SAME path already guarantees — excluding the Cross-Cutting Domain Invariant patterns (nil/empty/absent UUID, dual metadata keys, unversioned-tier versionId) and re-checks before destructive actions, which are load-bearing even when redundant on the happy path. For each new test, flag near-duplicates pinning the same code path AND poison-value class as an existing test — boundary companions (n==max vs max+1, absent vs empty vs nil UUID, MetaObject vs MetaDeleteMarker) are never near-duplicates; the test-coverage skeptic playbook below mandates them.
-  - Where: Any diff adding helpers, branches on decoded/peer data, or tests; helper checks against crates/utils, crates/common, and the touched crate
-  - Evidence: AGENTS.md 'Reuse Before You Write' and 'Necessary Code Only'; GHSA-f4vq-9ffr-m8m3 (normalization-asymmetry traversal — why forced reuse of normalizing helpers on raw keys is itself an attack); docs/operations/tier-ilm-debugging.md nil-versionId incident (why boundary re-checks are load-bearing).
-
-Null report example: "Rewrote the diff as an in-place edit (no smaller equivalent exists), grepped both new helpers against crates/utils, crates/common, and the touched crate (no existing equivalent; call-site semantics checked), verified the two new defensive branches name concrete corrupt-input triggers, and checked the added tests against the existing suite (each pins a distinct poison-value class) — no break found."
-
-### Security reviewer
-
-- For every admin handler in the diff, grep the exact AdminAction constant it passes to validate_admin_request and confirm it names the operation the handler actually performs. Construct the escalation: a low-privileged user whose policy grants the wrong-but-adjacent action (e.g. Export while the handler Imports, or Update while it Lists) — if the mismatched constant lets them through, that is the bug. Also confirm read-only endpoints (metrics, list, server-info, diagnostics) still call an operation-specific admin authz path and not a mere 'credentials exist' check.
-  - Where: rustfs/src/admin/handlers/**, rustfs/src/admin/router registration; check_permissions / validate_admin_request / AdminAction::* call sites
-  - Evidence: GHSA-vcwh-pff9-64cc (ImportIam checked ExportIAMAction), GHSA-mm2q-qcmx-gw4w (ListServiceAccount used UpdateServiceAccountAdminAction), GHSA-f5cv-v44x-2xgf (/admin/v3/metrics accepted any authenticated IAM user). advisory-patterns.md 'Admin authorization and route exposure'; rustfs/src/admin/AGENTS.md 'route registration, whitelist, handler authz must agree'.
-- If the diff touches service-account or IAM import/update, treat parent, claims, accessKey, secretKey, status, policy names, and groups as attacker-controlled. Construct an ImportIam/create payload where parent points at root (or another user) and prove the code writes credentials without proving caller ownership or root authority. Separately, set deny_only=true (or 'no explicit deny') on a restricted account and check it does not skip the required allow check, letting it mint an unrestricted child.
-  - Where: crates/iam/, rustfs/src/admin/handlers (service account / import IAM), rustfs/src/auth.rs
-  - Evidence: GHSA-566f-q62r-wcr8 (attacker-controlled parent/claims/accessKey/secretKey → persistent backdoor under root), GHSA-xgr5-qc6w-vcg9 (deny_only=true skipped allow checks → privilege creation). crates/iam/AGENTS.md security boundaries; advisory-patterns.md 'IAM import, service accounts'.
-- For a changed protocol-frontend handler (FTP/FTPS/SFTP/WebDAV/gateway), enumerate ALL sibling command handlers in the same driver — not just the changed one. For each, confirm it calls the per-operation IAM authorize hook mapped to the correct S3 action (RETR→GetObject, SIZE/MDTM→HeadObject, MKD→CreateBucket, bucket probe→ListBucket/HeadBucket) BEFORE touching storage. Construct a denied-authz case and prove the storage backend is never reached.
-  - Where: crates/protocols/ (FtpsDriver, SftpDriver, WebDAV), authorize_operation call sites
-  - Evidence: GHSA-3g29-xff2-92vp (FTP RETR/SIZE/MDTM authenticated but skipped IAM), GHSA-g3vq-vv42-f647 (FTPS MKD called create_bucket without s3:CreateBucket). advisory-patterns.md: 'RustFS advisories show mixed guarded and unguarded siblings in the same driver.'
-- For any secret/token/signature/password comparison in the diff, check it uses a constant-time compare (e.g. subtle/constant_time_eq), not == or early-return byte loops. Then check the failure-response paths: construct an invalid-user request and an invalid-secret request and confirm they are indistinguishable (same error, no early length short-circuit) so an attacker cannot enumerate valid users or time-side-channel the secret.
-  - Where: crates/protocols/ (FTPS/WebDAV/FormPost auth), crates/credentials/, rustfs/src/auth.rs, RPC signature verification
-  - Evidence: GHSA-3p3x-734c-h5vx (FTPS/WebDAV early-return string equality + distinguishable invalid-user vs invalid-password). Fix commits 3c3113619 (constant-time FTPS/WebDAV) and c41062f27 (constant-time FormPost signature). 3p3x was fixed by PR #4403.
-- If the diff touches internode/RPC auth secret handling, trace whether the RPC HMAC secret can fall back to a public default (e.g. 'rustfsadmin', 'rustfs rpc') or be derived deterministically from the S3 root credentials. Construct the case where RUSTFS_RPC_SECRET is unset and confirm the code fails closed rather than silently using a default or a root-derived key. Verify RPC signing keys are independent random secrets, not reused across S3-root/RPC-HMAC/STS-JWT roles.
-  - Where: crates/credentials/, crates/ecstore/src/rpc/, internode auth setup
-  - Evidence: GHSA-r5qv-rc46-hv8q (fell back to 'rustfsadmin'), GHSA-75fx/68cw (RPC secret derivable from root creds → forgeable signatures), GHSA-h956 (hard-coded 'rustfs rpc'), GHSA-m77q (STS JWT reused root secret). Fix commit 7b2055405 (fail closed when deriving RPC secret from default credentials, PR#4402).
-- If the diff touches RPC/NodeService authentication, verify the HMAC payload binds the EXACT concrete gRPC method path (not a service prefix), the HTTP method surrogate, and a fresh timestamp. Construct captured valid metadata for method A and replay it to method B within the timestamp window — if it authorizes, the signature is under-bound. Also test stale timestamp, wrong path, wrong method, wrong secret.
-  - Where: crates/ecstore/src/rpc/, verify_rpc_signature / NodeServiceServer, x-rustfs-signature handling
-  - Evidence: GHSA-c667-rgrv-99vj (signed service prefix instead of concrete method path → cross-method replay in timestamp window). advisory-patterns.md 'RPC input validation and panic safety'; Minimum Regression Test Expectations lists replay across two methods.
-- For any RPC/gRPC handler or deserialization touched, feed empty bytes, truncated MessagePack/protobuf, invalid enum discriminants, and stale timestamps. Grep the deserialization path for unwrap()/expect()/panic-prone decode and prove malformed attacker payloads return a typed error, not a panic (remote DoS). Weak internode auth makes reachability worse, so combine with the RPC-secret probe.
-  - Where: crates/ecstore/src/rpc/, any #[derive(Deserialize)] decoded from wire bytes, RPC handler bodies
-  - Evidence: GHSA-gw2x-q739-qhcr (malformed GetMetrics reached unwrap() → remote DoS). advisory-patterns.md 'Treat all RPC payload bytes as attacker-controlled.' rust-code-quality skill: unwrap abuse.
-- Take any object key, RPC disk path, or archive/tar/zip entry name introduced or handled in the diff and construct traversal payloads: '../', URL-encoded '%2e%2e%2f', absolute paths, platform separators, empty components. Trace the value through parse → authz check → final storage path and prove (a) authz and storage normalize the SAME way, and (b) the canonicalized path cannot escape the bucket/prefix root. Attack the case where authz sees the raw attacker bucket but storage cleaning crosses into a victim bucket.
-  - Where: crates/ecstore (path join/canonicalize), rustfs/src/storage/, Snowball auto-extract / normalize_extract_entry_key, rpc read_file_stream
-  - Evidence: GHSA-pq29-69jg-9mxc (read_file_stream joined untrusted paths, no canonical boundary check), GHSA-8r6f-hmq2-28rg (traversal object keys bypassed authz), GHSA-f4vq-9ffr-m8m3 (Snowball '../victim-bucket/object' authorized raw path then storage crossed boundary). Note __XLDIR__ trailing-slash encoding is store-layer only.
-- If the diff touches multipart/copy or presigned POST, verify UploadPartCopy enforces source GetObject AND destination PutObject semantics equivalent to CopyObject, including copy-source policy conditions (not just independent source-read + dest-write). Construct a cross-bucket UploadPartCopy from a bucket the caller cannot read. For presigned POST, submit an upload that violates content-length-range, key prefix, or exact content-type and prove the server rejects it.
-  - Where: rustfs/src/storage / S3 API handlers: upload_part_copy, CompleteMultipartUpload, PostObject policy enforcement
-  - Evidence: GHSA-mx42-j6wv-px98 (UploadPartCopy missed source authz → cross-bucket exfil), GHSA-wfxj-ph3v-7mjf (missed destination copy-source policy constraint), GHSA-w5fh-f8xh-5x3p (presigned POST didn't enforce signed policy conditions). advisory-patterns.md 'S3 copy, multipart, and upload policy validation'.
-- Grep the diff for debug!/trace!/info!/error! and any ?value / {:?} on structs or response bodies that can carry secret_key, session_token, JWT claims, HMAC secrets, expected signatures, access keys beyond safe identifiers, or raw credential-bearing responses. Check custom Debug impls and merged-config dumps too. Construct the error path (invalid signature, failed auth) and confirm it does not log the secret or the derived authenticator. Verify audit/notify entries redact credential request headers.
-  - Where: rustfs/src/**, crates/iam/, crates/audit/, crates/notify/, crates/targets/, RPC signature error paths
-  - Evidence: GHSA-r54g (STS creds logged at info), GHSA-8cm2 (debug logs leaked tokens/secrets/JWT claims/raw STS bodies), GHSA-333v (invalid RPC signature log included HMAC secret + expected signature). Fix commit ee6f79110 (redact credential request headers from audit/notify, backlog#963). rustfs-logging-governance skill.
-- For any struct in the diff deserialized from untrusted input (S3 XML/JSON, lifecycle rules, bucket policy, replication config, RPC payload), check for #[serde(deny_unknown_fields)]. Construct a payload with a typo'd field (e.g. 'NoncurentDays') or an extra field and prove it is rejected, not silently ignored. Flag #[serde(default)] on security-critical fields (retention days, limits, permissions) lacking explicit post-deserialize validation, and any user-controlled integer cast with `as` (i32 as u32) — feed a negative value and check it doesn't wrap to a huge positive.
-  - Where: crates/policy/ (bucket policy), crates/ecstore lifecycle/ILM config, replication config structs, crates/protocols XML parsing
-  - Evidence: AGENTS.md 'Serde Safety'; advisory-patterns.md 'Serde deserialization' (no deny_unknown_fields found repo-wide; 'NoncurentDays' typo silently accepted; i32 as u32 wrap). Fix commit 1acd47f15 (SSE crash-loop DoS + credential reserved-char bypass, backlog#806).
-- If the diff touches SSE / encryption reader-writer composition, do not trust API metadata claiming encryption. Trace the reader wrapper order (HashReader / EncryptReader / compression / warp) and confirm EncryptReader is actually in the chain that writes to disk — construct the case where a helper unwraps a nested reader and bypasses encryption, storing plaintext. Require a regression test that inspects the ACTUAL stored bytes on disk, not just read-back. Also check encrypted-object checksums are not exposed.
-  - Where: rustfs/src/storage/ecfs.rs, crates/rio/ (reader wrappers), crates/kms/, SSE-C replication
-  - Evidence: GHSA-xrrf-67jm-3c2r (SSE metadata reported encryption while composition bypassed EncryptReader, stored plaintext). Fix commits a7b9659e7 (hide encrypted object checksums, #4529), 80cc3b1fc (preserve SSE-C checksum state, #4410). advisory-patterns.md 'SSE and on-disk storage invariants'.
-- If the diff touches CORS or the console/browser/object-preview surface: confirm default CORS does not reflect an arbitrary Origin while also sending Access-Control-Allow-Credentials: true — construct a request with a spoofed Origin and check the response. For preview, confirm attacker-controlled object content is origin-isolated (not rendered in a same-origin iframe with console creds), served with nosniff/CSP, and that preview trust derives from validated content-type + sandboxing, NOT from object name/extension (.pdf, .html). Separately, if aws:SourceIp is evaluated, spoof X-Forwarded-For / X-Real-IP as a direct (non-trusted-proxy) client and confirm the socket peer IP is used instead.
-  - Where: rustfs/src/server/layer.rs (CORS), console preview/auth code, aws:SourceIp / policy condition evaluation, X-Forwarded-For handling
-  - Evidence: GHSA-x5xv-223c-8vm7 (default CORS reflected arbitrary origins with credentials), GHSA-v9fg-3cr2-277j (preview rendered attacker HTML same-origin, exposed localStorage creds), GHSA-7gcx-wg4x-q9x6 (extension-based PDF detection bypassed sandbox), GHSA-fc6g-2gcp-2qrq (aws:SourceIp trusted client XFF). advisory-patterns.md 'Browser, CORS' + 'Trusted proxy'.
-
-Null report example: "Attacked admin action-constant matching in the two changed handlers (both call validate_admin_request with the exact AdminAction), the FTPS RETR/SIZE authz parity, and the new lifecycle struct's serde surface (has deny_unknown_fields) — no break found."
-
-### Concurrency/durability reviewer
-
-- For every new or moved lock acquisition, enumerate all other code paths that take any overlapping subset of those locks and construct the concrete ABBA interleaving (thread 1 holds A wants B, thread 2 holds B wants A). If the diff acquires 2+ locks without a comment documenting acquisition order, that alone is a finding.
-  - Where: crates/ecstore/** (namespace locks, set_disk, disk registry), crates/audit/**, crates/lock/**, any Mutex/RwLock pair in a diff
-  - Evidence: crates/ecstore/AGENTS.md 'Lock Ordering' (document order; same set in different orders = deadlock); real ABBA deadlock fixed in c0d5f938f (#4421, audit registry vs stream_cancellers)
-- If the diff touches the object write/commit path or a lock guard's lifetime, construct the timeline where the distributed lock is lost (heartbeat refresh fails / expiry) after shard writes but before the xl.meta rename commit — verify the commit is fenced on guard.is_lock_lost() (set_disk/ops/object.rs:874-880) and the diff does not move the commit outside the fenced region or drop the guard early.
-  - Where: crates/ecstore/src/set_disk/ops/object.rs, ops/multipart.rs, crates/lock/**
-  - Evidence: 1e6207c08 (#4406) fence write commit on lock loss; ddf197ba5 (#4388) heartbeat lock refresh; backlog#899 fencing comment in object.rs
-- For any change to file creation or write-then-rename: write out the exact syscall order (write tmp -> fdatasync tmp -> rename -> fsync parent dir -> fsync ancestor dirs on first object under a prefix) and simulate a power cut after each step. Flag any dropped/reordered sync, and check the change honors the durability gate (RUSTFS_DRIVE_SYNC_ENABLE, strict/relaxed/none modes, per-bucket overrides) instead of hardcoding one mode. The 'skip tmp parent fsync' optimization is only sound when the file is renamed out of tmp — verify that precondition still holds.
-  - Where: crates/ecstore/src/disk/local.rs, disk/os.rs, disk/fs.rs, crates/ecstore/src/bucket/durability.rs, set_disk/core/io_primitives.rs
-  - Evidence: PR #4221 (the repo previously had no fsync anywhere); 2df315baf/c081586e7 (#4493) fsync ancestor dirs; 062a68d15 (#4387) tmp-parent-fsync skip is rename-conditional; eaff17cad (#4397) durability modes; 54872d52d (#4478) rename_data crash harness exists — extend it for the diff
-- If the diff touches quorum counting or per-disk error aggregation, construct adversarial error vectors for reduce_errs: nil/placeholder entries, DiskNotFound mixed with FileNotFound, exactly quorum-1 agreeing errors — and show which dominant error wins. Specifically attack the case where offline-disk errors get counted as 'object does not exist', flipping a read/heal decision into data loss. Also check quorum monotonicity: a retry or heal pass must never conclude with a LOWER quorum than the original write.
-  - Where: crates/ecstore/src/disk/error_reduce.rs, crates/ecstore/src/api/mod.rs, set_disk read/heal paths
-  - Evidence: 20d61c73b (#4551) reduce_errs leaked nil placeholder as dominant error; e0619e355 (#4536) DiskNotFound treated as object-not-found in listing; quorum monotonicity flagged as open follow-up to #4221 (#4221 follow-up list); crates/ecstore/AGENTS.md 'Do not weaken quorum checks'
-- For any multi-disk fan-out (delete, rename, heal write, cleanup), trace each per-disk Result: find any `let _ =`, `.ok()`, or best-effort collapse that keeps a failed disk out of the quorum math. Construct the run where exactly write_quorum-1 disks succeed and prove the op still returns success — that is the bug. Conversely, for heal writes, check one bad target cannot fail the whole heal (best-effort per target).
-  - Where: crates/ecstore/src/set_disk/ops/*.rs, disk/disk_store.rs, crates/heal/**
-  - Evidence: f7d2b2563 (#4546) disk delete/rename failures were swallowed; 47c1e730c (#4545) heal write quorum made best-effort per target; 2b063b0c4 (#4400) disk-replacement heal missed versions
-- For every new .await placed between a state mutation and its cleanup/commit (or inside select!/timeout/spawned task that can be aborted), construct the cancellation point: client disconnects and the future is dropped exactly there. Enumerate what is left behind — tmp files, incremented counters never decremented, half-written xl.meta, a held permit/waiter — and verify cleanup runs in Drop or the state is re-entrant. Background loops the diff adds must have a hard outer timeout so a wedged awaitee cannot pin them forever.
-  - Where: crates/ecstore/** write paths, crates/object-capacity/** scanners, crates/audit/**, anything using tokio::select! or spawn+abort
-  - Evidence: d608e320f io_uring cancel-safety spike (cancellation known-hard here); 5a372557e (#4533) wedged scans needed hard outer timeout; 7b87d4d13 (#4520) waiter-count leak; e44bece00 (#4497) audit start race + paused drops
-- If the diff touches metacache/list producers or cursor resume, construct the interleaving where the producer task completes (or errors) while a reader with a saved cursor comes back for the next page — verify a completed producer is tolerated (no error, no hang) and that a re-folded/full page reports truncation instead of silently ending the listing early. Also feed a corrupt/oversized length prefix into any metacache decode the diff touches.
-  - Where: crates/ecstore/src/cache_value/metacache_set.rs, crates/ecstore/src/store/list_objects.rs, crates/filemeta/**
-  - Evidence: 91a23361e (#4531) tolerate completed metacache producers; d91f4d455 (#4538) delimiter re-fold dropped truncation flag; d2c100fd3 (#4226) corrupt length-prefix guard
-- For multipart changes, construct concurrent operations on the SAME uploadId: put_object_part racing put_object_part (same part number), abort racing complete between the parts listing and the commit rename, and list-parts racing cleanup. Verify every metadata read/list/abort holds the per-uploadId lock, and that part.N.meta cleanup is deferred until AFTER the commit rename — cleanup before commit loses parts on a crash between the two.
-  - Where: crates/ecstore/src/set_disk/ops/multipart.rs, rustfs/src/storage/
-  - Evidence: 3bc8d79fe (#4329) serialize put_object_part per uploadId; 7fb95d4fc (#4428) unlocked upload metadata reads/aborts; 93ffbdb9b (#4437) unlocked part listings; c77c5f047 (#4548) part.N.meta cleanup moved after commit
-- Any cleanup/rollback logic added near a commit: verify it runs strictly AFTER the commit is durable, is best-effort (its failure must not fail an already-committed write), and is safe under retry — i.e., re-running it after a partial first attempt must never delete the newly-committed data dir or the last surviving copy.
-  - Where: crates/ecstore/src/set_disk/ops/object.rs (rename_data tail), ops/multipart.rs, disk cleanup helpers
-  - Evidence: afc7f1d6f/d908243e6 (#4386, backlog#898) post-commit old-data-dir cleanup had to be made best-effort; e7cc719c1 (#4389) speculative tmp cleanup moved off hot path
-- If the diff does read-modify-write on any persisted shared state (bucket metadata, notify/target config, queue store), construct two concurrent writers: show whether the second write silently discards the first (lost update) — RMW must be serialized or CAS-guarded. For persisted queues/replay, construct crash-mid-replay and prove entries are neither lost nor delivered twice without an idempotency key.
-  - Where: crates/notify/**, crates/targets/** (queue store, SQL backends), crates/ecstore/src/bucket/metadata_sys.rs
-  - Evidence: 2490d4ee2 (#4425) persisted config RMW lost updates; 08e44b95f (#4505) queue store crash-safety + replay lifecycle; e008cc5da (#4500) SQL backend idempotency
-- If the diff touches erasure decode/reconstruct or streaming GET, construct the failure mid-stream: shards become inconsistent (or a disk read fails) after N bytes of the body have already been sent — verify the stream surfaces an error to the client instead of ending cleanly at a truncated length. Silent truncation on a 200 response is the known failure mode.
-  - Where: crates/ecstore/src/set_disk/read.rs (reconstruct-read validation, historically ~line 3117), crates/ecstore/src/erasure/coding/decode.rs
-  - Evidence: Known open bug: EC reconstruct-read 'inconsistent shards' mid-GET silently truncates body -> client unexpected EOF; crates/ecstore/AGENTS.md 'explicit failure over silent corruption'
-
-Null report example: "Attacked lock ordering on the new disk-registry mutex pair, lock-loss fencing across the moved commit, power-cut points around the added rename, and cancellation at the two new awaits — no break found; quorum math and metacache paths untouched by this diff."
-
-### Compatibility reviewer
-
-- Grep the diff for raw 'x-rustfs-internal-' or 'x-minio-internal-' string literals used with map.insert/remove/get instead of the metadata_compat helpers. If found, construct the MinIO-written object case: a metadata map containing ONLY 'X-Minio-Internal-<suffix>' (mixed case, no RustFS key) and trace the diff's read path — does it miss the value? Then construct the removal case: does remove leave the twin key behind so a stale MinIO-key value resurrects on next read? Also check the value-type trap: get_bytes has NO case-insensitive fallback (unlike get_str), so a diff that moves a suffix from FileInfo.metadata (String) to meta_sys (Vec<u8>) silently loses mixed-case MinIO keys.
-  - Where: Any code touching FileInfo.metadata / user_defined / meta_sys: crates/ecstore/, crates/filemeta/, rustfs/src/storage/, crates/utils/src/http/metadata_compat.rs
-  - Evidence: Repo-wide invariant in AGENTS.md 'Cross-Cutting Domain Invariants' and CLAUDE.md; helpers and the asymmetry are pinned by tests test_str_lookup_accepts_minio_metadata_case and test_get_bytes_no_case_insensitive_fallback in crates/utils/src/http/metadata_compat.rs
-- For any diff reading a binary UUID from internal metadata (transitioned-versionID, tier-free-versionID, data_dir), trace the three degenerate inputs — key absent, value empty (b""), value nil UUID — through to the outgoing tier/S3 request. The bug shape to hunt: unwrap_or_default() or Uuid::from_slice(..).unwrap_or(Uuid::nil()) turning 'no value' into Uuid::nil(), which then gets serialized as ?versionId=00000000-... and the remote tier returns NoSuchVersion. The required pattern is .and_then(|v| Uuid::from_slice(&v).ok()).filter(|u| !u.is_nil()).
-  - Where: crates/ecstore/src/bucket/lifecycle/ (bucket_lifecycle_ops.rs, tier_sweeper.rs), crates/ecstore/src/services/tier/warm_backend_*.rs, crates/filemeta/src/filemeta/version.rs
-  - Evidence: Historical production bug documented in docs/operations/tier-ilm-debugging.md ('Nil-UUID versionId sent to tier'); regression tests live in crates/filemeta/src/filemeta/version.rs; follow-up fix 726f3dc18 'accept empty remote version_id in tier recovery paths' (#4552)
-- For any diff touching tier or replication GET/DELETE against a remote S3 target, enumerate BOTH directions of the versionId contract and trace each: (a) remote version None/"" means the tier bucket is unversioned — the request must carry NO versionId parameter at all (not an empty one, not nil); (b) remote version Some(v) on a versioned target — the versionId MUST be sent, especially on version-purge deletes, or the delete lands on the wrong version / creates a delete marker instead of purging. Check whether the diff collapses these cases through a single Option/String conversion that loses the distinction.
-  - Where: crates/ecstore/src/services/tier/warm_backend*.rs, crates/ecstore/src/bucket/lifecycle/tier_sweeper.rs, replication code under crates/ecstore/src/bucket/
-  - Evidence: Invariant in AGENTS.md and docs/operations/tier-ilm-debugging.md; real bug fixed by 0fad35645 'send versionId on version-purge deletes to generic S3 targets' (#4401) — the versioned direction, and #4552 — the unversioned direction
-- If the diff changes xl.meta encoding (adds/reorders msgpack header fields, touches FileMeta::marshal_msg or codec.rs encode paths), attack downgrade and cross-vendor parse: encode an object with the new code and decode it with (a) the meta_ver<=3 read path and (b) the real-MinIO fixture tests from #4377. Then check the header signature: is it recomputed over the new bytes, or copied/hardcoded? MinIO validates it; a stale or zero signature makes MinIO reject the file. Finally verify XL_META_VERSION was not silently bumped — old RustFS/MinIO nodes reject meta_ver > 3 during a rolling upgrade.
-  - Where: crates/filemeta/src/filemeta.rs (XL_HEADER_VERSION/XL_META_VERSION, lines ~46-54), crates/filemeta/src/filemeta/codec.rs (check_xl2_v1, decode_xl_headers)
-  - Evidence: Real bug 073bc9675 'compute header signature instead of hardcoding zero' (#4343); format contract pinned in docs/architecture/minio-file-format-compat.md (write meta_ver 3, read <=3, XL2 magic); parity fixtures from a91d9cefc (#4377)
-- If the diff touches xl.meta / FileInfo decode (into_fileinfo, version parsing, part arrays), construct hostile foreign input: a MinIO- or corruption-shaped msgpack with a parts-count that disagrees with the etags/sizes array lengths, missing optional fields, and a meta_ver 2 object with legacy checksum. Trace whether the new code indexes past an array, panics, or fabricates default values instead of returning a decode error. Run the pinned legacy fixtures (test_issue_2265_legacy_meta_v2_object_compatibility, test_issue_2288) plus the #4377 real-MinIO xl.meta parse tests against the diff.
-  - Where: crates/filemeta/src/ (fileinfo.rs, filemeta.rs, filemeta/codec.rs, filemeta/version.rs)
-  - Evidence: Real bug 7efacbdf9 'validate part array lengths in into_fileinfo' (#4382); legacy meta_ver 2 regression fixtures at crates/filemeta/src/filemeta.rs (~:1130-:1174) cited by docs/architecture/minio-file-format-compat.md
-- If the diff 'corrects' a formula, constant, or layout that is a byte-for-byte MinIO port (shard-size math, bitrot hash interleaving, erasure distribution, inline-data prefix), treat the correction itself as the bug: verify against legacy on-disk data before accepting. Concretely: run crates/ecstore/tests/legacy_bitrot_read_test.rs and the ECA-18 pinning tests; check whether existing objects written by old RustFS or MinIO still verify byte-for-byte. Known trap examples: bitrot_shard_file_size's bare return for non-streaming algorithms is CORRECT MinIO whole-file behavior, and the 32-byte prefix on inline data is the HighwayHash256 bitrot hash, not corruption.
-  - Where: crates/ecstore/src/erasure/coding/bitrot.rs, crates/ecstore/src/io_support/bitrot.rs, crates/filemeta/src/fileinfo.rs
-  - Evidence: f96314a1d 'pin streaming-only bitrot layout invariant (ECA-18)' (#4553) — audit explicitly decided NOT to change the formula because it breaks legacy interop; #4377 proved the inline-data prefix is the bitrot hash
-- For any diff that copies object metadata into an S3-client-visible surface (GET/HEAD response headers, notification event userMetadata, ListObjects/replication payloads, copy-object metadata directives), construct an object carrying internal keys under BOTH prefixes and in non-canonical casing ('X-Minio-Internal-Compression') and confirm every one is stripped via is_internal_key (which is case-insensitive) — not by an exact-match filter on one prefix. Leaked internal keys are an API-semantics break and an information leak.
-  - Where: rustfs/src/storage/, crates/notify/, replication and copy_object paths in crates/ecstore/
-  - Evidence: Real bug cf8929189 'strip rustfs/minio internal metadata from event userMetadata' (#4419); is_internal_key contract in crates/utils/src/http/metadata_compat.rs
-- If the diff touches crates/protos (node.proto, models.fbs) or internode RPC request/response structs, attack the rolling-upgrade interleaving: an old node sends a message without the new field to a new node, and a new node sends the extended message to an old node. Verify proto field numbers are only appended (never reused/renumbered), FlatBuffers tables are only extended at the end, and that an absent new field decodes to a safe default on the receiving side — 'safe' meaning it must not be interpreted as success/authorization (RPC errors fail closed) and must not flip a quorum decision.
-  - Where: crates/protos/ (node.proto, models.fbs, generated/), gRPC transport and dispatch in the internode layer
-  - Evidence: 6f613317f 'optimize gRPC transport' (#4337) shows the wire layer churns; security advisory 68cw fixed by PR#4402 established RPC fail-closed as a repo rule (see .agents/skills/security-advisory-lessons)
-- For S3 handler diffs, replay the request shapes real clients actually send, not just the canonical one: mc and aws-sdk differ on path normalization (root '//' ListBuckets), virtual-host vs path style, and header casing. Then attack every pagination boundary the diff touches: request exactly max-keys/max-uploads/max-parts items and verify the response returns exactly N (not N+1), sets IsTruncated correctly, and yields a NextMarker/KeyMarker that resumes without skipping or duplicating — construct the N+1st-item case explicitly.
-  - Where: rustfs/src/storage/ S3 handlers, listing paths in crates/ecstore/src/store/ and set_disk/
-  - Evidence: Real bugs 511ad31ba 'normalize root double-slash ListBuckets requests' (#4336) and fefa70b31 'stop ListMultipartUploads from returning one upload past max-uploads' (#4447); docs/architecture/s3-compatibility-matrix.md is the compat source of truth
-- If the diff changes bucket-metadata (.metadata.bin) or IAM/config parsing structs, run it against the real MinIO RELEASE.2025-07-23 fixtures: the msgpack blob uses PascalCase field names, so any serde rename, field-type change, or derive tweak silently drops MinIO-written fields instead of erroring. Verify parse_all_configs still loads all ten config types from the fixture without loss, and that drop-in migration still decrypts MinIO-encrypted IAM/server config rather than treating ciphertext as corrupt.
-  - Where: crates/ecstore/src/bucket/metadata*, crates/ecstore/src/bucket/migration.rs, IAM/config load paths in crates/iam/ and crates/config/
-  - Evidence: Fixture parity test parses_real_minio_bucket_metadata_blob_without_loss from a91d9cefc (#4377); real bug 717cdd2ab 'decrypt MinIO IAM & server config on drop-in migration' (#4358); format matrix in docs/architecture/minio-file-format-compat.md
-- If the diff adds a compatibility shim, legacy fallback, wrapper, or old-endpoint alias (grep the diff for 'legacy', 'fallback', 'compat', 'deprecated'), verify two things: (1) it carries a RUSTFS_COMPAT_TODO(<task-id>) marker with an exact removal condition and a matching entry in the register — an unmarked shim becomes permanent dead weight; (2) the fallback's default direction is safe for old data: e.g. a new decode path must fall back to the legacy decode for old objects by default, not gate legacy reads behind an opt-in flag that makes existing data unreadable after upgrade.
-  - Where: Anywhere in the diff; register at docs/architecture/compat-cleanup-register.md; recent example: allow_inplace_legacy_fallback flag in the ecstore erasure codec streaming path
-  - Evidence: docs/architecture/compat-cleanup-register.md review checklist; d232a46b4 wired legacy decode prefetch behind a default-OFF gate while keeping legacy reads working (#4542), with arity fallout fixed in 05890d6e2 (#4573)
-
-Null report example: "Attacked dual-key metadata writes/removals against MinIO-only-key objects, nil/empty transitioned-versionID paths to the tier, xl.meta encode against meta_ver<=3 decoders and the #4377 real-MinIO fixtures, and proto field-number evolution for old-node/new-node RPC — no compatibility break found."
-
-### Performance reviewer
-
-- For every `.clone()` the diff adds or moves onto a per-request/per-object path, open the cloned type and count heap fields (String, Vec, HashMap, Bytes). If >5 heap fields or it contains an EC block buffer, construct the cost: N concurrent PUTs x M objects -> N*M deep copies per second. Demand Arc-wrapping of heavy fields or pass-by-reference; also flag new `String` allocations in header/path/signature parsing where `&str`/`Cow<str>` suffices.
-  - Where: crates/ecstore/src/set_disk/**, crates/ecstore/src/store*.rs, rustfs/src/storage/, crates/filemeta/, request handlers in rustfs/src/
-  - Evidence: crates/ecstore/AGENTS.md 'Allocation Discipline in Hot Paths' (no Clone on >5-heap-field structs, Arc for large buffers, &str/Cow for temporary computations); .agents/skills/rust-code-quality/SKILL.md ranks 'unnecessary clone in hot path' as P1 must-fix
-- For every new sync_all/sync_data/fdatasync/flush/File::sync call in the diff, trace the call chain to DurabilityMode / RUSTFS_DRIVE_SYNC_ENABLE resolution (crates/ecstore/src/disk/local.rs:291 DurabilityMode, :347 resolve_durability_mode) and to per-bucket durability overrides. Construct the run where the operator sets mode=none (or legacy RUSTFS_DRIVE_SYNC_ENABLE=false) and the new fsync still fires — that is an ungated durability cost and a regression on 4KiB writes.
-  - Where: crates/ecstore/src/disk/local.rs, crates/ecstore/src/bucket/durability.rs, crates/ecstore/src/set_disk/** (rename_data/commit paths), any crate doing tokio::fs or std::fs writes
-  - Evidence: #4221 fsync work caused a measured -10% 4KiB write regression (#814 investigation), later gated; durability modes added in eaff17cad (#4397), per-bucket tier overrides in 13e48d93a (#4407); 2df315baf (#4493) shows even ancestor-dir fsyncs are routed through the gate
-- Attack blocking-work placement from both directions: (a) find new synchronous fs calls, hashing, or EC encode/decode executed directly on an async runtime thread without spawn_blocking/block_in_place — construct the stall (a slow disk blocks a worker thread and every task queued on it); (b) find new code that splits one logical disk operation into multiple spawn_blocking hops per object — each hop is a threadpool round-trip, so K hops x N objects multiplies latency. Demand the author justify the placement with the size of the work, not habit.
-  - Where: crates/ecstore/src/disk/local.rs, crates/ecstore/src/erasure_coding/, crates/ecstore/src/bitrot/, crates/rio/
-  - Evidence: 608ab14d7 (#4554, HP-12) folded metadata open+fstat+read into a single spawn_blocking because per-op hops were measurably slow; 8fc637fb1 (#4484) moved the short EC encode inline because block_in_place cost exceeded the work — direction depends on measured work size
-- For each lock acquisition the diff adds or relocates, mark the guard's live range and list every .await and disk/RPC call inside it. Construct the contention interleaving: N concurrent requests serialize on the guard while the holder waits on IO; for namespace/multipart commit locks, compute worst-case hold time (fsync + rename per disk) against the lock's timeout. Also diff the acquisition order against other paths taking the same locks (ABBA).
-  - Where: crates/ecstore/src/set_disk/** (commit/rename paths), crates/lock/, crates/audit/ registry, any RwLock/Mutex in per-request paths
-  - Evidence: crates/ecstore/AGENTS.md 'Lock Ordering'; c0d5f938f (#4421) fixed a real ABBA deadlock between registry and stream_cancellers; #4370 history: fsync-heavy serial cross-disk commits held a lock long enough to blow test timeouts (#4370)
-- Trace exactly what executes inside the PUT commit critical section (under the object write lock, between tmp write and rename_data completion) before vs after the diff. Any newly added work there — cleanup, extra stat, additional rename, O_DIRECT write, logging — is an attack target: construct the per-PUT latency delta and demand it be moved off the critical section or parallelized across disks.
-  - Where: crates/ecstore/src/set_disk/ops/*, crates/ecstore/src/disk/local.rs rename_data path
-  - Evidence: Three real optimizations removed exactly this class of regression: e7cc719c1 (#4389) moved speculative PUT-tail tmp cleanup off the hot path, 92c8c6db7 (#4411) moved O_DIRECT shard-writes off the commit critical section, 651ccac13 (#4487) parallelized tmp xl.meta write and shard fdatasync on commit
-- Find any new loop in a batch API that performs a per-item stat/read/RPC sequentially. Construct the concrete blowup: a 1000-key DeleteObjects or a full listing page -> 1000 serial round-trips added by the diff. Demand either a gate (skip when not needed) or bounded parallelism; for startup/load paths, check for accidental O(n^2) (re-scanning the full list per item).
-  - Where: crates/ecstore/src/store_delete*.rs / batch object APIs, listing/metacache paths, crates/iam/ store loading, crates/heal/
-  - Evidence: a413729b1 (#4398) had to gate and parallelize the DeleteObjects per-object stat fanout after it shipped serial; 16a91c35e (#4537) fixed O(n^2) IAM startup load by chunking — both were diff-introduced fanouts of this exact shape
-- For every buffer the diff allocates on the encode/decode/shard path, check: is it sized with with_capacity to the EC-expanded block (not the logical size, not default-grown)? Does it copy into a fresh Vec where Bytes::slice/clone (refcount) or the io-core buffer pool would avoid the copy? Does the diff read hash and data in separate passes where one pass suffices? Construct the per-block byte-copy count before vs after. If the diff touches the io-core pool, verify gauge accounting still balances.
-  - Where: crates/ecstore/src/erasure_coding/, crates/ecstore/src/bitrot/, crates/io-core/src/pool.rs, crates/rio/
-  - Evidence: 92bf55ce6 (#4396) fixed a real regression by right-sizing BytesMut encode ingest capacity to the EC-expanded block; 47bee8b31 (#4475) merged bitrot hash+data into one read pass; 7fa3d0d4b (#4534) shows pool gauge accounting is easy to drift when touching buffer reuse
-- For every logging or instrumentation statement the diff adds, classify the call site frequency: per-request, per-object, per-shard, or per-block. Anything info!/warn!/error! at per-object frequency or higher is a finding — construct the flood (one listing under client cancellation, one 10k-object heal) and count emitted lines. New metrics/timers on the data path must be feature-gated, not always-on. Run scripts/check_logging_guardrails.sh on the diff.
-  - Where: any per-request/per-object code, especially crates/ecstore listing and heal loops, rustfs/src/storage/ handlers; scripts/check_logging_guardrails.sh
-  - Evidence: .agents/skills/rustfs-logging-governance/SKILL.md (trace level for hot-path/repetitive success events); d25ddb0e1 (#4372) fixed real listing-cancellation error-log noise; hotpath instrumentation is deliberately feature-gated (3f13d098b #4394, f262fcfce #4541 HP-14)
-- Count how many times the diff's request path parses or fetches the same metadata: xl.meta/FileMeta decoded more than once per object, bucket metadata (metadata_sys) re-fetched inside a per-object loop, or the dual x-rustfs-internal/x-minio-internal key lookup re-run repeatedly on the same map. Construct the per-request parse count before vs after; a second full FileMeta decode per GET is a finding.
-  - Where: crates/filemeta/, crates/ecstore/src/set_disk/** read paths, crates/ecstore/src/bucket/metadata_sys.rs, crates/utils/src/http/metadata_compat.rs
-  - Evidence: 608ab14d7 (#4554) exists because redundant metadata-read syscall sequences per object were measurable; CLAUDE.md dual-key metadata convention makes repeated get_bytes lookups an easy hidden double-parse
-- If the diff touches PUT/GET/commit/erasure paths and claims 'no perf impact', demand numbers, not assertion: run the criterion benches (cargo bench -p ecstore — comparison_benchmark, erasure_benchmark, rename_data_meta_benchmark, single_block_non_inline_benchmark per crates/ecstore/benches/) against origin/main, and for end-to-end paths the warp A/B relative-budget gate (scripts/run_hotpath_warp_ab.sh --baseline-ref origin/main, as .github/workflows/performance-ab.yml runs it). Probe specifically at 4KiB object size — that is where the last real regression hid.
-  - Where: crates/ecstore/benches/, .github/workflows/performance-ab.yml, scripts/run_hotpath_warp_ab.sh
-  - Evidence: crates/ecstore/AGENTS.md: 'Benchmark-sensitive changes should include measurable rationale'; performance-ab.yml (215747022 #4480) is the repo's own relative-budget gate; the #4221 regression was only visible at 4KiB writes (#814 bisect)
-
-Null report example: "Attacked the new rename_data commit-section work, durability-gate routing of the added fdatasync, guard live-range across the shard-write awaits, and per-object clone count in the PUT path; ran comparison_benchmark + rename_data_meta_benchmark vs origin/main (4KiB delta within noise) — no break found."
-
-### Test-coverage skeptic
-
-- For every behavior claim in the PR description, revert that hunk (git stash / manual undo of the changed lines) and name the exact test (`cargo test -p <crate> <test_name>`) that fails. If no test fails on revert, the behavior is untested — file a finding, not a note. Especially verify the test exercises the REAL production call path, not a lookalike helper.
-  - Where: All crates; highest value in crates/ecstore, rustfs/src/storage, crates/heal
-  - Evidence: AGENTS.md exit criterion 'Every behavior change has a test that fails without it'. Real bug: PR #4220 (ghost-directory cleanup) merged with green tests but its fix never executed on the real delete path — required follow-up rustfs#4307, backlog#798 stayed OPEN. The tests exercised a path the production flow never took.
-- Read each added/modified test and confirm it asserts the real outcome (returned value, stored bytes, error variant), not merely 'call succeeded' or 'no panic'. Flag any test whose only observable is that the function returned, and any `assert!(result.is_err())` that never checks WHICH error. Then check: does the test prove the exploit/failure form is denied, or only that the intended form still works?
-  - Where: crates/e2e_test (security_boundary_test.rs pattern), and every #[cfg(test)] module in the diff
-  - Evidence: Commit dee8e4e63 (#4466) had to rewrite 277 lines of crates/e2e_test/src/security_boundary_test.rs because 'security boundary tests' passed without asserting real outcomes. .agents/skills/rust-code-quality/SKILL.md checklist: 'Every test function has at least one assert!'; .agents/skills/security-advisory-lessons/SKILL.md: 'Does the test prove the exploit form is denied, or only that the intended form still works?'
-- When the diff adds a boolean/mode parameter or config flag, find the test that fails if the flag's effect is INVERTED inside the changed function. Tests that were mechanically updated to pass `false`/default at every call site assert nothing about the new behavior. Execute the check: flip the flag's branch in the source and confirm at least one test goes red for each branch.
-  - Where: crates/ecstore/src/set_disk/ (e.g. build_codec_streaming_part_reader), any function gaining a parameter
-  - Evidence: Commit 05890d6e2 (#4573): PR #4560 added a 15th param allow_inplace_legacy_fallback; the arity tests were fixed by passing `false` everywhere — they assert Err outcomes independent of the flag, so the fallback behavior itself has no revert-detecting test at those sites.
-- Mutation spot-check on error propagation: for each newly added `?`, `return Err`, or error-mapping line, mentally replace it with `Ok(default)`/ignore and ask which test fails. The swallowed-error bug class recurs in this repo and always ships with green tests — a fix that propagates errors needs a test that injects the failure (faulty disk, failed rename, dispatch error) and asserts the caller sees Err.
-  - Where: crates/ecstore (disk delete/rename, reduce_errs), crates/audit, crates/notify, crates/targets
-  - Evidence: Three recent fixes for the same class: f7d2b2563 (#4546, disk delete/rename failures swallowed), dbc628f16 (#4424, audit dispatch failures swallowed), 20d61c73b (#4551, reduce_errs leaking nil placeholder as dominant error). All existed while tests were green.
-- Any test touching GET/read/reconstruct/stream paths must assert the FULL body content and exact length against a known value, not status-ok or first-bytes. Construct the degraded-read case (missing/inconsistent shards forcing EC reconstruction) and assert byte-for-byte equality; a mid-stream failure that truncates the body passes every test that only checks headers or the first chunk.
-  - Where: crates/ecstore/src/set_disk/read.rs and ops/, crates/rio, crates/e2e_test GET scenarios
-  - Evidence: Known live bug: EC reconstruct-read verification failure mid-GET at set_disk read path silently truncates the body → client 'unexpected EOF'; version-independent, undetected by existing suites because none assert full-body integrity under shard inconsistency.
-- For on-disk / on-wire format changes (xl.meta, .metadata.bin, bitrot framing), reject round-trip-only tests: a struct serialized and deserialized by the same code under test cannot catch format drift. Demand the test parse a REAL captured fixture from crates/filemeta/tests/fixtures, crates/ecstore/tests/fixtures, or crates/rio-v2/tests/minio_fixture_lab — or capture a new one from a single-disk MinIO instance (RELEASE.2025-07-23 procedure from #4377).
-  - Where: crates/filemeta, crates/ecstore (headers, msgpack bucket metadata), crates/rio-v2, migration code
-  - Evidence: Commit 073bc9675 (#4343): filemeta header signature was hardcoded to zero — round-trip tests passed for months. Commit a91d9cefc (#4377) established the real-MinIO fixture convention (inline/versioned/multipart xl.meta, HighwayHash256-prefixed inline bodies) precisely because synthetic fixtures proved nothing about interop.
-- Attack new concurrency tests for flakiness-by-construction: grep the added tests for `sleep(`, fixed timeouts under ~30s on lock acquisition, and use of shared global state (disk registry, lock client, GLOBAL_*). Serialized cross-disk commits exceed small lock timeouts under full-suite CI disk load. If the test shares global state or saturates IO, it must join the `ecstore-serial-flaky` nextest test-group in .config/nextest.toml (note: serial_test's #[serial] does NOT work — nextest runs each test in its own process). Require readiness polling, never fixed sleeps.
-  - Where: crates/ecstore tests, crates/e2e_test, .config/nextest.toml
-  - Evidence: Commit 2dfa3d3c3 (#4370): concurrent_resend test flaked with Lock(Timeout 5s) on CI — six legitimate serialized cross-disk commits under IO pressure needed 30s. Commit 7c701d9f2 (#4558) created the nextest test-group after bucket_delete_* raced make_bucket into InsufficientWriteQuorum. 65849740f (#4213) deflaked global-state contamination. crates/e2e_test/AGENTS.md: 'readiness checks and explicit polling over fixed sleep-based timing'.
-- If the diff writes internal object metadata, run the dual-key mutation: delete the `x-minio-internal-<suffix>` write (keeping only `x-rustfs-internal-`) and check whether any test fails. Because `get_bytes` prefers the RustFS key, every read-back test stays green while MinIO interop is silently broken — coverage must include an assertion that BOTH keys are present in the stored metadata map.
-  - Where: crates/utils/src/http/metadata_compat.rs and all its callers in crates/ecstore and rustfs/src/storage
-  - Evidence: CLAUDE.md domain convention: metadata must be written under both x-rustfs-internal- and x-minio-internal- keys for MinIO interop; get_bytes prefers the RustFS key, making the MinIO-key half of the invariant invisible to read-back tests.
-- For changed quorum/version/UUID logic, name the tests covering the specific poison values: quorum−1 disks, nil UUID, absent vs empty vs nil-serialized UUID bytes, remote-tier version_id of None/"" (unversioned tier bucket → no versionId sent), and the same metadata read on both MetaObject and MetaDeleteMarker version types. Mutation check: remove a `.filter(|u| !u.is_nil())` guard from the diff and confirm a test fails; if none does, the nil-UUID class is uncovered.
-  - Where: crates/ecstore (tier recovery, heal, quorum paths), crates/filemeta, code reading UUIDs from xl.meta metadata
-  - Evidence: Commit 726f3dc18 (#4552) fixed rejection of empty remote version_id in tier recovery. CLAUDE.md invariant: absent/empty/nil UUID all mean 'no value', not Uuid::nil(). docs/operations/tier-ilm-debugging.md: None/"" tier version means unversioned bucket. df9cbc4ed (#4427): unvalidated distribution values caused shuffle index panic — edge values reached production untested.
-- For any pagination/limit/truncation change, construct the exact-boundary test: result count == max (page exactly full), max+1, and a delimiter re-fold that lands precisely on the page boundary — assert both the item count AND the is_truncated/continuation marker. Off-by-one at the page boundary is a recurring shipped bug here.
-  - Where: crates/ecstore listing paths (list_objects, ListMultipartUploads, metacache), S3 handlers in rustfs/src/storage
-  - Evidence: Two shipped boundary bugs: fefa70b31 (#4447) ListMultipartUploads returned one upload past max-uploads; d91f4d455 (#4538) delimiter re-fold of a full page lost the truncation flag. Both survived existing tests because no test pinned n == max exactly.
-- Green `cargo test -p <crate>` on the touched crate is not a coverage verdict for the diff's test code itself: run `cargo clippy --all-targets -p <crate>` and a workspace-wide test BUILD (`cargo check --workspace --all-targets` at minimum) before accepting the tests as evidence. Test-only code that doesn't compile workspace-wide or fails clippy has repeatedly broken main and masked whether tests ran at all.
-  - Where: All crates; especially concurrent-branch merges into crates/ecstore
-  - Evidence: #4322 broke main because only cargo test ran (field_reassign_with_default is clippy-only). b06f3df6b (#4441) and 05890d6e2 (#4573): test code broke the workspace test build (E0061) on main after textually-clean merges, failing CI for every open PR.
-
-Null report example: "Attacked revert-detection for all 3 claimed behaviors (each has a named test that fails on revert), flag-inversion on the new fallback parameter (both branches covered in codec_streaming tests), full-body assertions on the changed GET path, and n==max pagination boundary — no coverage gap found."
-
-## Sources and maintenance
-
-Probes are distilled from shipped bugs in git history (commit/PR references
-above), GitHub security advisories (see the security-advisory-lessons
-skill), scoped `AGENTS.md` rules, and invariants under `docs/architecture/`
-and `docs/operations/`. Line numbers drift; when a cited location no longer
-matches, trust the invariant and re-locate the code. When a new bug class
-ships, add a probe with its evidence here rather than growing the policy
-section in `AGENTS.md`.

.agents/skills/arch-checks/SKILL.md

@@ -47,12 +47,5 @@ consider adding it to the script's `checked_files` list.
 Instruction/architecture docs (`AGENTS.md`, `CLAUDE.md`, `ARCHITECTURE.md`,
 `docs/architecture/*.md`) must not reference repo file paths that no longer
 exist. If your refactor moved code, update the docs that point at it — the
-error message lists `doc -> stale-path` pairs.
-
-## `check_no_planning_docs.sh`
-
-Planning-type documents must not be committed (see AGENTS.md "Sources of
-Truth"). The guard fails if anything is tracked under `docs/superpowers/` —
-`.gitignore` already ignores it, but `git add -f` bypasses that, so this closes
-the hole. Fix by removing the listed file(s) with `git rm`; keep the plan or
-spec in the issue tracker or a local worktree instead.
+error message lists `doc -> stale-path` pairs. Historical plans under
+`docs/superpowers/plans/` are exempt.

.agents/skills/code-change-verification/SKILL.md

@@ -43,7 +43,16 @@ Use this skill to review code changes consistently before merge, before release,
 
 #### Rust-specific checks (apply to all Rust changes)
 
-Run the full checklist in [rust-code-quality](../rust-code-quality/SKILL.md) — the canonical Rust review checklist for the unwrap/casting/cloning/locking/recursion/error-type/serde/test rules and the reuse-and-necessity checks (duplicated helpers, defensive branches without a nameable trigger, redundant error wrapping). Do not restate those rules here; carry its P0–P3 ratings over unchanged and use this skill's output format.
+- **unwrap/expect in production**: Search changed files for `.unwrap()` and `.expect(` outside test modules. Every `unwrap()` in production code must have a justification comment or be replaced with `?`.
+- **Silent type truncation**: Search for `as u8/u16/u32/u64/usize/i8/i16/i32/i64/isize` casts. Every `as` cast must be justified; negative-to-unsigned and large-to-small are bugs by default. Use `try_into()` or explicit clamping.
+- **Unnecessary cloning**: Check `.clone()` calls in loops, per-request paths, and on structs with >5 heap-allocated fields. Consider `Arc`, references, or `Cow<str>`.
+- **Lock ordering**: If the change acquires multiple locks, verify the order matches all other call sites. Document the order in a comment.
+- **Locks across .await**: Flag any `tokio::sync::RwLock`/`Mutex` guard held across an `.await` point without bounded hold time.
+- **Recursion depth**: If the change adds or modifies a recursive function, verify it has a depth limit or uses iterative traversal with an explicit stack.
+- **Error types**: Flag `Result<_, String>`, `Box<dyn Error>`, and missing `Error::source()` implementations in public APIs.
+- **Test assertions**: Every test function must have at least one `assert!`. Flag tests that only call code without verifying results.
+- **println/eprintln**: Search changed files for `println!`/`eprintln!` outside test modules. Production code must use `tracing` macros.
+- **Serde safety**: Structs deserialized from untrusted input (S3 API, user config) should have `#[serde(deny_unknown_fields)]`.
 
 ### 4) Findings-first output
 - Order findings by severity:

.agents/skills/rust-code-quality/SKILL.md

@@ -36,9 +36,6 @@ rg -n 'println!\|eprintln!' <changed-files> | grep -v test
 
 # 6. Ordering::Relaxed usage (verify each is intentional)
 rg -n 'Ordering::Relaxed' <changed-files>
-
-# 7. Default substituted for a possibly-required value (judge each: is the value optional by domain?)
-rg -n 'unwrap_or_default\(\)|unwrap_or\(' <changed-files>
 ```
 
 ## Manual Review Checklist
@@ -58,8 +55,8 @@ For every Rust code change, verify:
 - [ ] No `f64 as usize` without prior clamping
 
 ### Concurrency
-- [ ] Lock acquisition order is documented when multiple locks are used, and matches every other call site taking any overlapping subset (ABBA check)
-- [ ] No `tokio::sync` lock guard (read or write) held across `.await` without bounded hold time — long-lived read guards wedge writers (#4195)
+- [ ] Lock acquisition order is documented when multiple locks are used
+- [ ] No `tokio::sync` write guards held across `.await` without bounded hold time
 - [ ] Concurrent counters use `compare_exchange` loops, not load-then-store
 - [ ] `std::sync::Mutex` in async context is held only briefly, never across `.await`
 
@@ -87,19 +84,12 @@ For every Rust code change, verify:
 - [ ] No camelCase statics or Hungarian notation
 - [ ] New string literals don't duplicate existing constants
 
-### Reuse and Necessity
-- [ ] No new helper duplicating an existing workspace utility (`crates/utils`, `crates/common`, the touched crate) or plain std/tokio behavior no wrapper refines; reused helpers match the call site's semantics (normalization, error type, backoff, durability gating)
-- [ ] No branch without a nameable concrete trigger; no re-validation of what a validated upstream layer on the same path already guarantees (Cross-Cutting Domain Invariant patterns and pre-destructive-action re-checks are load-bearing — keep them)
-- [ ] Error context attached once where actionable, not re-wrapped at every hop; no typed→generic error conversion below aggregation/quorum layers
-- [ ] No comments narrating the next line, restating a signature, or describing the change itself (invariant comments — lock ordering, `SAFETY`, unwrap justification — are not narration)
-- [ ] No near-duplicate test pinning the same code path and poison-value class as an existing test (boundary companions — n==max vs max+1, absent/empty/nil UUID — are never near-duplicates)
-
 ## Severity Classification
 
 - **P0 (Block merge)**: `unwrap()` in request hot path, silent truncation on user input, lock ordering violation, recursion without depth limit
-- **P1 (Must fix)**: `Result<_, String>` in public API, unnecessary clone in hot path, `Box<dyn Error>` in trait method, `unwrap_or_default()` on a domain-required value (metadata, quorum, version id)
-- **P2 (Should fix)**: Missing `assert!` in test, `println!` in production, missing `with_capacity`, new helper duplicating an existing workspace utility, defensive branch with no nameable trigger (corrupt or stale persisted/peer data is always a nameable trigger for boundary-crossing values), near-duplicate test, redundant error re-wrapping
-- **P3 (Nice to fix)**: Naming convention violation, missing doc comment, `as_ptr()` vs `Arc::ptr_eq`, narrating comment
+- **P1 (Must fix)**: `Result<_, String>` in public API, unnecessary clone in hot path, `Box<dyn Error>` in trait method
+- **P2 (Should fix)**: Missing `assert!` in test, `println!` in production, missing `with_capacity`
+- **P3 (Nice to fix)**: Naming convention violation, missing doc comment, `as_ptr()` vs `Arc::ptr_eq`
 
 ## Output Template
 

.agents/skills/rustfs-release-publish/SKILL.md

@@ -1,169 +0,0 @@
----
-name: rustfs-release-publish
-description: "End-to-end RustFS release pipeline: bump version files on main directly to the final target version, cut a preview tag on that commit, verify the CI build and release artifacts, run the downloaded binary locally and exercise the console, validate the server with the latest rc client, then publish the final tag on the SAME validated commit — never a new bump commit, never latest main. Use whenever the user wants to release/publish a RustFS version (发版/发布)."
----
-# RustFS Release Publish (preview-validated pipeline)
-
-This skill orchestrates a full release. It wraps `rustfs-release-version-bump` (which only edits version files and opens the PR) with a mandatory preview-tag validation loop before the final tag is published.
-
-Core design: **version files never carry a `-preview.N` suffix**. The preview suffix exists only in tag names. This works because the binary self-reports the git tag it was built from (`build::TAG` via shadow_rs, see `rustfs/src/config/cli.rs` `SHORT_VERSION`), and `build.yml` derives artifact names and prerelease classification from the tag name — Cargo.toml's version is only a no-tag fallback. Therefore the preview tag and the final tag can (and MUST) point at the exact same commit: what you validated is byte-for-byte the source that ships.
-
-Pipeline shape:
-
-```
-bump version files to <target> (final version, ONE commit) -> merge
-  -> tag <preview-tag> at that commit -> CI green
-  -> verify release artifacts -> run binary locally + console checks
-  -> validate with latest rc client
-  -> tag <target> at the SAME commit (zero delta) -> re-verify CI/release
-```
-
-On validation failure: fix lands on main via normal PR (version files are already at `<target>`, no new bump PR), then tag `<preview-tag N+1>` at the new main commit and restart from Phase 2.
-
-## Required inputs
-
-- Final target version, for example `1.0.0-beta.10`.
-- Preview iteration `N` (default: next unused preview tag for that target; check with `git tag -l '<target>-preview.*'` — and for stable targets `git tag -l '<target>-rc.*'` — after `git fetch --tags`).
-
-If the target version is missing or ambiguous, stop and ask before doing anything (see the semver gate below).
-
-## Semver gate — confirm the target version before touching anything
-
-Versions follow [SemVer 2.0.0](https://semver.org/). Precedence reminder:
-
-```
-1.0.0-alpha < 1.0.0-alpha.1 < 1.0.0-beta.2 < 1.0.0-beta.11 < 1.0.0-rc.1 < 1.0.0 < 1.0.1 < 1.1.0 < 2.0.0
-```
-
-Numeric prerelease identifiers compare numerically (`beta.9 < beta.10`), not lexically — see [semver.org spec item 11](https://semver.org/#spec-item-11). Preview tags are internal validation tags layered on top of the target's prerelease channel — they are never themselves a deliverable version and never appear in version files.
-
-Rules:
-
-- A request like "发个版" / "release the next version" without an exact version string is ALWAYS ambiguous. Derive the current latest tag (`git tag --sort=-v:refname | head`), then ask the user to choose via AskUserQuestion with concrete candidates, e.g. from `1.0.0-beta.10`: next prerelease `1.0.0-beta.11`, promote to `1.0.0-rc.1`, promote to stable `1.0.0`. Never guess between these — they have very different meanings (channel promotion vs. iteration) and different CI classification consequences.
-- After a stable `X.Y.Z` exists, the next version must state which component bumps: patch `X.Y.(Z+1)` for fixes only, minor `X.(Y+1).0` for backward-compatible features, major `(X+1).0.0` for breaking changes. If the user names a bump type but not a number, compute it from the latest stable tag and echo the exact resulting version back for confirmation.
-- Echo the final confirmed version string verbatim in your first status report; every later phase must use exactly that string. If at any point the user's wording and the confirmed version diverge, stop and re-confirm.
-
-## Preview tag naming
-
-- Prerelease target (contains `alpha`/`beta`/`rc`): preview tag is `<target>-preview.N`, e.g. `1.0.0-beta.10-preview.3`. It contains `beta`, so `build.yml`'s substring-based classification marks it prerelease — safe.
-- **Stable** target (e.g. `1.1.0`): NEVER tag `1.1.0-preview.N` — `build.yml` marks a tag prerelease only if its name contains `alpha`, `beta`, or `rc`, so `1.1.0-preview.N` would be treated as a stable release and overwrite `latest.json` as stable. Use `1.1.0-rc.N` as the preview tag instead.
-
-## Hard rules
-
-- Version files (Cargo.toml, Cargo.lock, README, flake.nix, Chart.yaml, rustfs.spec) are bumped ONCE, directly to `<target>`. Never write a `-preview.N` suffix into any version file. If `rustfs-release-version-bump` is ever asked for a `-preview` version, that is a pipeline bug — stop.
-- Tags have no `v` prefix. Always annotated: `git tag -a <tag> -m "Release <tag>"`.
-- The final tag MUST point at exactly `PREVIEW_HASH` — the commit the validated preview tag points at. Never tag current `main` HEAD (commits merged after validation are unvalidated), and never create an extra version-bump commit between preview and final.
-- Phases run in order; a failure in any phase blocks everything after it. After the fix lands on main, restart from Phase 2 with the next preview iteration against the new `origin/main` hash — do not resume mid-pipeline against a stale hash.
-- If the release is abandoned after Phase 1 merged, main's version files claim a version that was never tagged. Either revert the bump PR or leave it to be overwritten by the next release — but tell the user explicitly and record the decision.
-- User-facing status updates in Chinese; commits, PR titles/bodies, and tag messages in English. No hard-wrapping in commit messages, PR bodies, or documentation prose — one logical line per sentence/paragraph, let soft wrap handle display.
-
-## Phase 0 — Preflight
-
-- `git status --short` clean; `git fetch origin main --tags`.
-- `gh auth status` works; confirm you can view `gh release list -L 3`.
-- Confirm the exact final target version with the user if not explicit.
-
-## Phase 1 — Version bump to the final target (once)
-
-- If main's version files already read `<target>` (e.g. this is a restart after a failed preview), verify with `rg -n "<target>" Cargo.toml rustfs.spec helm/rustfs/Chart.yaml` and skip to Phase 2.
-- Otherwise invoke the `rustfs-release-version-bump` skill with the final `<target>` (NOT a preview version), full GitHub flow (commit/push/PR).
-- Get the PR merged into main. Record the resulting main commit:
-
-```bash
-git fetch origin main
-PREVIEW_HASH=$(git rev-parse origin/main)   # must contain the bump PR
-```
-
-`PREVIEW_HASH` is the single source of truth for the rest of the pipeline — report it to the user and reuse it verbatim in Phases 2 and 6. Both the preview tag and the final tag will point at it.
-
-## Phase 2 — Publish the preview tag
-
-```bash
-git tag -a "<preview-tag>" -m "Release <preview-tag>" "$PREVIEW_HASH"
-git push origin "<preview-tag>"
-```
-
-Pushing the tag triggers `.github/workflows/build.yml` ("Build and Release"); `docker.yml` chains off it via `workflow_run`.
-
-On a restart (N+1), refresh `PREVIEW_HASH=$(git rev-parse origin/main)` first — it must contain the fix — and re-report it.
-
-## Phase 3 — CI and artifact verification
-
-- Watch the tag build: `gh run list --workflow build.yml --limit 5` then `gh run watch <run-id>`. Every matrix target must succeed (linux x86_64/aarch64 × musl/gnu, macos-aarch64, windows-x86_64) plus the release and latest.json jobs.
-- Verify the GitHub release: `gh release view "<preview-tag>" --json isPrerelease,assets`
-  - `isPrerelease` must be `true`.
-  - Assets must include all 6 platform zips in both versioned (`rustfs-<platform>-v<tag>.zip`) and `-latest` forms, plus `SHA256SUMS`, `SHA512SUMS`, `rustfs-<tag>.sbom.cdx.json`, `rustfs-<tag>.provenance.json`.
-- Verify the chained Docker run succeeded: `gh run list --workflow docker.yml --limit 3`.
-- Checksum spot-check for the platform you will run locally: download the zip and `SHA256SUMS`, verify with `shasum -a 256 -c` (grep to one line).
-
-## Phase 4 — Run the artifact locally, verify the console
-
-Work inside the session scratchpad directory; never leave stray data dirs.
-
-```bash
-gh release download "<preview-tag>" -p "rustfs-macos-aarch64-v<preview-tag>.zip" -D "$SCRATCH"
-cd "$SCRATCH" && unzip -o rustfs-*.zip
-./rustfs --version        # must report the PREVIEW TAG (build::TAG), not the Cargo.toml version, plus expected short SHA
-mkdir -p data
-RUSTFS_ACCESS_KEY=rustfsadmin RUSTFS_SECRET_KEY=rustfsadmin ./rustfs ./data
-```
-
-Defaults: S3 endpoint `:9000`, embedded console `:9001`.
-
-Checks (all must pass):
-
-- `./rustfs --version` reports the preview tag name and the short SHA of `PREVIEW_HASH`. Reporting `<target>` without the `-preview.N` suffix means the build did not embed the tag — treat as FAIL and investigate before proceeding.
-- `curl -fsS http://localhost:9000/health/ready` returns ready.
-- Startup log shows the embedded console being served (this was the regression that `fix(release): require embedded console assets` guards).
-- Open `http://localhost:9001` in the browser: login with `rustfsadmin`/`rustfsadmin`; dashboard renders without JS console errors; create a bucket, upload a file, download it back (byte-identical), delete the object and bucket. Keep the server running for Phase 5.
-
-## Phase 5 — Validate with the latest rc client
-
-`rc` is the RustFS CLI client from <https://github.com/rustfs/cli>.
-
-- Ensure the latest release is installed: compare `rc --version` against `gh api repos/rustfs/cli/releases/latest --jq .tag_name`; update via `brew upgrade rustfs/tap/rc` (or download the release binary).
-- Point it at the preview server and run the command matrix, recording PASS/FAIL per command:
-
-```bash
-rc alias set preview http://localhost:9000 rustfsadmin rustfsadmin
-rc ls preview/
-rc mb preview/rel-check
-rc cp <local-file> preview/rel-check/
-rc stat preview/rel-check/<file>
-rc cat preview/rel-check/<file>          # matches source
-rc cp preview/rel-check/<file> ./out && cmp <local-file> ./out
-rc cp -r <local-dir>/ preview/rel-check/dir/
-rc find preview/rel-check --name "*"
-rc share download preview/rel-check/<file> --expire 1h   # presigned URL fetchable via curl
-rc rm preview/rel-check/<file> && rc rm -r --force preview/rel-check/dir
-rc rb preview/rel-check
-rc admin user list preview/
-rc admin user add preview/ relcheckuser relchecksecret12
-rc admin user remove preview/ relcheckuser
-rc alias remove preview
-```
-
-- Any FAIL blocks the release. Afterwards stop the server and delete the scratch data directory.
-
-## Phase 6 — Publish the final tag on the validated commit
-
-No second version bump, no release branch. The final tag goes on the exact commit the preview validated:
-
-```bash
-git fetch origin --tags
-git rev-parse "<preview-tag>^{commit}"          # must equal PREVIEW_HASH — abort if not
-git tag -a "<target>" -m "Release <target>" "$PREVIEW_HASH"
-git push origin "<target>"
-```
-
-- CI rebuilds from the same source; the only changed input is the tag name, so the binary now self-reports `<target>`.
-- Re-run the Phase 3 verification against the final tag: all matrix jobs green; `gh release view "<target>"` shows the full asset set; for a prerelease target `isPrerelease` is `true`, for a stable target it must be `false` and `latest.json` must be updated.
-- Optionally spot-check `./rustfs --version` from a final-tag artifact — it must report `<target>`.
-
-## Output contract
-
-Always report:
-
-- Target version, preview tag(s) used, `PREVIEW_HASH` (which both tags point at).
-- Per-phase result (PASS/FAIL/BLOCKED) with key evidence: CI run URLs, release URLs, console check results, the rc command matrix.
-- Any deviation from this pipeline and why the user approved it.

.agents/skills/rustfs-release-version-bump/SKILL.md

@@ -18,8 +18,6 @@ Validated baseline: release pattern used in PR `#2957`.
 
 If target version is missing or ambiguous, stop and ask before editing.
 
-Reject any target version containing a `-preview.` suffix: preview identifiers are tag-only (see `rustfs-release-publish`) and must never be written into version files. If asked for one, stop and point to the release pipeline instead of editing.
-
 ## Read before editing
 
 - `AGENTS.md` (root and nearest path-specific files).

.agents/skills/security-advisory-lessons/SKILL.md

@@ -26,7 +26,7 @@ For the full pattern map, read [advisory-patterns.md](references/advisory-patter
 
 ### 1. Scope the change
 - Identify touched routes, protocol frontends, handlers, storage paths, credentials, logs, browser surfaces, CI/release code, and policy checks.
-- Treat these paths as security-sensitive by default: `rustfs/src/admin/`, `rustfs/src/storage/`, `rustfs/src/auth.rs`, `rustfs/src/server/layer.rs`, `crates/iam/`, `crates/policy/`, `crates/credentials/`, `crates/ecstore/src/rpc/`, `crates/protocols/`, `crates/rio/`, OIDC/STS federation code, and console preview/auth code.
+- Treat these paths as security-sensitive by default: `rustfs/src/admin/`, `rustfs/src/storage/`, `rustfs/src/auth.rs`, `rustfs/src/server/layer.rs`, `crates/iam/`, `crates/policy/`, `crates/credentials/`, `crates/ecstore/src/rpc/`, `crates/protocols/`, `crates/rio/`, and console preview/auth code.
 
 ### 2. Map to advisory classes
 - Read [advisory-patterns.md](references/advisory-patterns.md) for matching GHSA lessons.
@@ -59,18 +59,10 @@ For the full pattern map, read [advisory-patterns.md](references/advisory-patter
 
 ### IAM and service accounts
 - Treat imported IAM payload fields as attacker-controlled: `parent`, `claims`, `accessKey`, `secretKey`, status, policy names, and groups.
-- For service account create/update/import, prove parent ownership or root/admin authority before writing credentials or claims; an action permission alone must not allow choosing root or another user as `target_user`.
-- Treat IAM export packages as credential disclosure surfaces; never include plaintext user or service-account secret keys unless the caller is allowed to recover those secrets and the export format is intentionally sealed.
+- For service account create/update/import, prove parent ownership or root/admin authority before writing credentials or claims.
 - Do not let `deny_only` or "no explicit deny" become an allow decision that skips required allow checks.
 - Test cross-user list/update/import flows with wrong, correct, self, parent, and root identities.
 
-### STS, OIDC, and federation flows
-- Every STS endpoint must have an explicit authentication story: SigV4 where required, OIDC token verification for web identity, and role/session policy validation before issuing credentials.
-- JWT session tokens must be signed and verified by a trusted issuer/key path, not by service-account-controlled material or a reused root secret.
-- JWT verification must enforce required claims and expiration for every bearer token path; "allow missing exp" is never acceptable for user-presented credentials.
-- Public OIDC bootstrap and callback routes must treat `Host`, `X-Forwarded-Proto`, redirect targets, `state`, and callback parameters as untrusted; credential-bearing redirects require a configured, allowlisted origin.
-- OIDC discovery and validation URLs are SSRF sinks. Resolve and classify hostnames at connection time, reject rebinding to loopback/private/link-local ranges, and do not rely on literal string checks.
-
 ### S3 copy, multipart, and presigned POST
 - Multipart copy must enforce source `GetObject` and destination `PutObject` semantics equivalent to `CopyObject`, including copy-source and policy conditions.
 - Do not let `CreateMultipartUpload`, `UploadPartCopy`, `CompleteMultipartUpload`, or `AbortMultipartUpload` return success without authorization.
@@ -121,7 +113,6 @@ For the full pattern map, read [advisory-patterns.md](references/advisory-patter
 
 ### Trusted proxy and network identity
 - Only honor `X-Forwarded-For` or `X-Real-IP` when the request came from a configured trusted proxy.
-- Apply the same trusted-proxy rule to scheme and host derivation; direct clients must not control security-sensitive redirects through `Host`, `X-Forwarded-Host`, or `X-Forwarded-Proto`.
 - Direct clients must use the socket peer address for `aws:SourceIp` and policy condition evaluation.
 - Add tests for direct spoofed headers and trusted-proxy headers.
 
@@ -139,10 +130,6 @@ Use these prompts while reviewing a diff:
 - Does a public/default/empty config change security behavior from fail-closed to fail-open?
 - Is any attacker-controlled value later used as a path, policy condition, credential identity, log field, URL, Origin, or response body?
 - Does this response contain stored replication, remote target, or service credentials that need redaction or stricter authorization?
-- Does an IAM export/import path expose or trust plaintext credential secrets beyond the caller's intended authority?
-- Can this STS/OIDC path issue credentials without SigV4, trusted issuer validation, allowlisted redirects, or trusted-proxy host/scheme handling?
-- Can a service-account or STS token omit `exp`, forge `sessionPolicy`, or use a principal-controlled key as signing authority?
-- Does this outbound validation path resolve attacker-supplied hostnames and reject private, loopback, link-local, and rebound addresses at the actual connection boundary?
 - Is an archive entry, object key, or policy resource normalized differently between authorization and storage?
 - Is the same operation implemented in multiple paths, such as `CopyObject` vs `UploadPartCopy`, and do all paths enforce the same security contract?
 - Does a preview or browser-surface fix preserve the original security invariant when adding alternate viewers or file-type detection?

.agents/skills/security-advisory-lessons/references/advisory-patterns.md

@@ -27,18 +27,9 @@ Update this file only when an advisory adds or changes a reusable lesson, affect
 ### IAM import, service accounts, and privilege boundaries
 
 - `GHSA-566f-q62r-wcr8`: `ImportIam` accepted attacker-controlled service account `parent`, `claims`, `accessKey`, and `secretKey`, enabling persistent backdoor accounts under root. Lesson: imported IAM payloads are untrusted data and must be validated against privilege boundaries.
-- `GHSA-3495-h8r9-gfqg`: `ExportIAM` wrote regular-user and service-account secret keys into exported ZIP data. Lesson: IAM export is a credential-disclosure boundary; redact, seal, or strictly justify every exported secret before treating export permission as safe.
-- `GHSA-5354-r3w2-34m8`: `AddServiceAccount` checked `CreateServiceAccountAdminAction` but trusted caller-supplied `target_user`, allowing service accounts under the root parent. Lesson: service-account create paths must validate parent ownership or root/admin authority, not only the create action.
 - `GHSA-xgr5-qc6w-vcg9`: `deny_only=true` skipped allow checks and let restricted service accounts mint unrestricted children. Lesson: deny-only logic must never become implicit allow for privilege creation.
 - `GHSA-mm2q-qcmx-gw4w`: leaked service account access keys plus update-without-ownership formed an escalation chain. Lesson: service-account identifiers are security-sensitive because update APIs consume them.
 
-### STS, OIDC, and federation flows
-
-- `GHSA-5qfg-mf7r-jp3w` and `GHSA-3473-5353-xhwh`: `AssumeRoleWithWebIdentity` was reachable through unauthenticated `POST /` routing and could issue temporary credentials from crafted web identity input. Lesson: every STS route needs explicit SigV4 or trusted identity-provider validation before role assumption, and unauthenticated exemptions must be narrowed to the exact action with uniform failure responses.
-- `GHSA-ccrv-v8v9-ch9q` and `GHSA-48rf-7j3q-3hfv`: service-account-controlled material could self-sign JWT session tokens with forged policy claims, and missing `exp` was accepted for service-account tokens. Lesson: session tokens must be signed by a trusted issuer/key path, enforce required claims and expiration, and reject self-signed or principal-controlled tokens.
-- `GHSA-9pjf-w3c2-m32r`, `GHSA-4x2q-cpx9-9h26`, and `GHSA-xvpm-p3f7-34c3`: public OIDC authorize/callback flows trusted request `Host` or forwarded scheme when building credential-bearing redirects. Lesson: OIDC redirects must use configured allowlisted origins and trusted-proxy handling; never derive the post-login credential destination from direct client headers.
-- `GHSA-m479-9x88-94w6`, `GHSA-frwq-mfqx-83p8`, `GHSA-q9q8-rf9r-fg9f`, and `GHSA-j5c2-hhf7-6gf5`: OIDC validation accepted attacker-controlled discovery URLs because hostname checks rejected only literal forbidden IPs, allowing DNS rebinding SSRF. Lesson: outbound federation URL validation must resolve and classify hostnames at the connection boundary and reject loopback, private, link-local, and rebound addresses.
-
 ### S3 copy, multipart, and upload policy validation
 
 - `GHSA-mx42-j6wv-px98`: `UploadPartCopy` missed source authorization and allowed cross-bucket object exfiltration. Lesson: multipart copy must enforce the same source and destination contract as `CopyObject`.
@@ -59,7 +50,7 @@ Update this file only when an advisory adds or changes a reusable lesson, affect
 
 ### Secrets, defaults, and cryptographic misuse
 
-- `GHSA-j59h-h7q5-q348`, `GHSA-3wm5-wpm5-hmfm`, `GHSA-6wc8-xm48-qhmx`, `GHSA-9gf3-jx4p-4xxf`, and `GHSA-63xc-c3w3-m2cf`: RustFS shipped known default root credentials that could authenticate to S3, admin APIs, IAM, KMS, console, and token-signing surfaces. Lesson: root credentials must be operator-provided or generated per install; known defaults and warnings are not acceptable for network-reachable deployments.
+- `GHSA-j59h-h7q5-q348`: RustFS shipped known default root credentials that could authenticate to S3, admin APIs, IAM, KMS, and console surfaces. Lesson: root credentials must be operator-provided or generated per install; known defaults and warnings are not acceptable for network-reachable deployments.
 - `GHSA-h956-rh7x-ppgj`: gRPC used the hard-coded token `rustfs rpc` on both client and server. Lesson: source-visible shared tokens are authentication bypasses.
 - `GHSA-r5qv-rc46-hv8q`: internode RPC HMAC secret fell back to the public default `rustfsadmin`. Lesson: RPC/internode auth must fail closed instead of silently using public defaults.
 - `GHSA-75fx-qg6f-8rm7` and `GHSA-68cw-96m3-h2cf`: internode RPC secrets were derivable from known root credentials, making raw storage RPC signatures forgeable when explicit RPC secrets were unset. Lesson: RPC auth keys must be independent random secrets, never derived from S3 root credentials, and raw storage RPC should not share the public S3 listener without an internode-only boundary.
@@ -123,7 +114,6 @@ rg -n "deny_unknown_fields|serde.default|as u32|as usize|as i32" rustfs crates
 - Copy/upload fixes: include cross-bucket, cross-user, source-denied, destination-denied, copy-source-condition, and multipart completion cases.
 - Path fixes: include encoded traversal, absolute path, nested traversal, archive entries with `..`, valid object keys that resemble traversal text but should be rejected, and canonical bucket/prefix boundary checks.
 - Logging fixes: assert redacted output for structs and response bodies that may contain credentials.
-- IAM export fixes: assert exported archives omit plaintext user and service-account secrets unless the format deliberately encrypts or seals them.
 - RPC auth fixes: include captured metadata replay across two concrete methods, stale timestamps, wrong path, wrong method surrogate, wrong secret, and valid same-method calls.
 - Browser/CORS fixes: assert no credentials on reflected/default origins, correct behavior for explicit allowlists, and no same-origin script execution for previewed object content.
 - SSE fixes: inspect stored bytes and verify API metadata, read-back behavior, and on-disk ciphertext together.

.cargo/config.toml

@@ -14,20 +14,13 @@
 
 # RustFS Cargo configuration
 
-# NOTE: `--cfg tokio_unstable` is deliberately NOT set here.
-#
-# It used to be a global `[build] rustflags` entry so that the (default-off)
-# `dial9` telemetry feature could compile. That made every build depend on
-# Tokio's unstable, non-semver API, and it broke silently whenever a caller
-# exported their own RUSTFLAGS — an environment RUSTFLAGS replaces the value
-# from this file rather than appending to it.
-#
-# The flag is now scoped to telemetry builds, which opt in explicitly:
-#
-#     make build-profiling
-#     RUSTFLAGS="--cfg tokio_unstable" cargo build --features dial9
-#
-# `crates/obs/build.rs` fails the compile if the `dial9` feature is on without
-# the flag, so the two can no longer drift apart unnoticed.
-#
-# For CPU profiling, add `-C force-frame-pointers=yes` to that RUSTFLAGS value.
+# Enable tokio_unstable cfg for dial9-tokio-telemetry support
+# This allows dial9 to hook into Tokio's internal runtime events
+[build]
+# Enable Tokio unstable features required by dial9-tokio-telemetry for runtime tracing.
+# See: https://docs.rs/tokio/latest/tokio/#unstable-features
+rustflags = ["--cfg", "tokio_unstable"]
+
+# Enable frame pointers for CPU profiling (Linux only, optional but recommended)
+# Uncomment the following line for better CPU profiling data
+# rustflags = ["--cfg", "tokio_unstable", "-C", "force-frame-pointers=yes"]

.config/make/build.mak

@@ -35,25 +35,6 @@ build-gnu-arm64: ## Build aarch64 GNU version
 	./build-rustfs.sh --platform aarch64-unknown-linux-gnu
 
 
-## —— Profiling build (dial9 Tokio runtime telemetry) ------------------------------------------
-
-# dial9 hooks Tokio's unstable runtime instrumentation, so it needs
-# `--cfg tokio_unstable`. That flag is deliberately absent from
-# .cargo/config.toml: it is not free, and release binaries do not carry it.
-# Setting RUSTFLAGS here replaces (never appends to) the config-file value, and
-# crates/obs/build.rs fails the build if the feature and the flag disagree.
-#
-# There are no task-dump or S3-upload features — see the notes in
-# crates/obs/Cargo.toml for why.
-DIAL9_FEATURES ?= dial9
-DIAL9_RUSTFLAGS ?= --cfg tokio_unstable
-
-.PHONY: build-profiling
-build-profiling: ## Build RustFS with dial9 Tokio runtime telemetry (diagnostic builds only)
-	@echo "🔬 Building RustFS with dial9 telemetry (features: $(DIAL9_FEATURES))..."
-	@echo "⚠️  Diagnostic build: telemetry writes trace segments to disk continuously."
-	RUSTFLAGS="$(DIAL9_RUSTFLAGS)" cargo build --release --bin rustfs --features $(DIAL9_FEATURES)
-
 .PHONY: build-cross-all
 build-cross-all: core-deps ## Build binaries for all architectures
 	@echo "🔧 Building all target architectures..."

.config/make/check.mak

@@ -11,16 +11,14 @@ check-%:
 
 # Warning-only check
 # Checks for optional dependencies and issues a warning if not found
+# (e.g., cargo-nextest for enhanced testing)
 warn-%:
 	@command -v $* >/dev/null 2>&1 || { \
 		echo >&2 "⚠️ '$*' is not installed."; \
 	}
 
 # For checking dependencies use check-<dep-name> or warn-<dep-name>
-#
-# NOTE: cargo-nextest is a HARD dependency of `make test`, gated inside the
-# test recipe itself (with a RUSTFS_ALLOW_CARGO_TEST_FALLBACK=1 escape hatch)
-# rather than a warn-only prerequisite here — see .config/make/tests.mak.
-.PHONY: core-deps fmt-deps
+.PHONY: core-deps fmt-deps test-deps
 core-deps: check-cargo ## Check core dependencies
 fmt-deps: check-rustfmt ## Check lint and formatting dependencies
+test-deps: warn-cargo-nextest ## Check tests dependencies

.config/make/coverage.mak

@@ -1,26 +0,0 @@
-## —— Coverage --------------------------------------------------------------------------------------
-
-# Local equivalent of the weekly coverage workflow (.github/workflows/coverage.yml,
-# backlog#1153 infra-5): same measurement scope (--workspace --exclude e2e_test,
-# nextest `ci` profile) and the same per-crate table. Slow — the instrumented
-# build cannot reuse your normal target cache and then runs the whole suite.
-# Doctests are not measured (needs nightly). Outputs land in target/llvm-cov/.
-.PHONY: coverage
-coverage: core-deps ## Workspace line coverage (cargo-llvm-cov + nextest; slow, writes target/llvm-cov/)
-	@if ! command -v cargo-llvm-cov >/dev/null 2>&1; then \
-		echo >&2 "❌ cargo-llvm-cov is required for 'make coverage' but was not found."; \
-		echo >&2 "   Install it with:"; \
-		echo >&2 "     cargo install cargo-llvm-cov --locked"; \
-		echo >&2 "     rustup component add llvm-tools-preview"; \
-		exit 1; \
-	fi
-	@if ! command -v cargo-nextest >/dev/null 2>&1; then \
-		echo >&2 "❌ cargo-nextest is required for 'make coverage' (see 'make test')."; \
-		echo >&2 "   Install it with: cargo install cargo-nextest --locked"; \
-		exit 1; \
-	fi
-	NEXTEST_PROFILE=ci cargo llvm-cov nextest --workspace --exclude e2e_test --no-report
-	@mkdir -p target/llvm-cov
-	cargo llvm-cov report --lcov --output-path target/llvm-cov/lcov.info
-	cargo llvm-cov report --json --output-path target/llvm-cov/coverage.json
-	python3 scripts/coverage_per_crate.py target/llvm-cov/coverage.json

.config/make/lint-fmt.mak

@@ -15,7 +15,7 @@ fmt-check: core-deps fmt-deps ## Check code formatting
 .PHONY: clippy-check
 clippy-check: core-deps ## Run clippy checks
 	@echo "🔍 Running clippy checks..."
-	cargo clippy --all-targets -- -D warnings
+	cargo clippy --all-targets --all-features -- -D warnings
 
 .PHONY: clippy-fix
 clippy-fix: core-deps ## Apply clippy fixes
@@ -50,21 +50,6 @@ tokio-io-uring-check: ## Check tokio io-uring runtime feature stays removed
 	@echo "🚫 Checking tokio io-uring feature guard..."
 	./scripts/check_no_tokio_io_uring.sh
 
-.PHONY: extension-schema-check
-extension-schema-check: ## Check extension-schema stays a lightweight contract crate
-	@echo "🧩 Checking extension schema boundaries..."
-	./scripts/check_extension_schema_boundaries.sh
-
-.PHONY: body-cache-whitelist-check
-body-cache-whitelist-check: ## Check the body-cache eligibility gate stays a fail-closed allow-list
-	@echo "🧱 Checking body-cache whitelist guard..."
-	./scripts/check_body_cache_whitelist.sh
-
-.PHONY: log-analyzer-rules-check
-log-analyzer-rules-check: core-deps ## Check log-analyzer rule anchors still exist verbatim in source
-	@echo "🩺 Checking log-analyzer rule anchors..."
-	./scripts/check_log_analyzer_rules.sh
-
 .PHONY: compilation-check
 compilation-check: core-deps ## Run compilation check
 	@echo "🔨 Running compilation check..."

.config/make/pre-commit.mak

@@ -13,19 +13,14 @@ doc-paths-check: ## Check that instruction/architecture docs reference existing
 	@echo "📄 Checking doc path references..."
 	./scripts/check_doc_paths.sh
 
-.PHONY: planning-docs-check
-planning-docs-check: ## Check that no planning-type documents are committed
-	@echo "📄 Checking for committed planning docs..."
-	./scripts/check_no_planning_docs.sh
-
 .PHONY: pre-commit
-pre-commit: fmt-check unsafe-code-check architecture-migration-check logging-guardrails-check tokio-io-uring-check extension-schema-check body-cache-whitelist-check doc-paths-check planning-docs-check quick-check ## Run fast pre-commit checks without clippy/full tests
+pre-commit: fmt-check unsafe-code-check architecture-migration-check logging-guardrails-check tokio-io-uring-check doc-paths-check quick-check ## Run fast pre-commit checks without clippy/full tests
 	@echo "✅ All pre-commit checks passed!"
 
 .PHONY: pre-pr
-pre-pr: fmt-check unsafe-code-check architecture-migration-check logging-guardrails-check tokio-io-uring-check extension-schema-check body-cache-whitelist-check doc-paths-check planning-docs-check log-analyzer-rules-check clippy-check test ## Run full pre-PR checks with clippy and tests
+pre-pr: fmt-check unsafe-code-check architecture-migration-check logging-guardrails-check tokio-io-uring-check doc-paths-check clippy-check test ## Run full pre-PR checks with clippy and tests
 	@echo "✅ All pre-PR checks passed!"
 
 .PHONY: dev-check
-dev-check: fmt-check unsafe-code-check architecture-migration-check logging-guardrails-check tokio-io-uring-check extension-schema-check body-cache-whitelist-check doc-paths-check planning-docs-check quick-check ## Run fast local development checks
+dev-check: fmt-check unsafe-code-check architecture-migration-check logging-guardrails-check tokio-io-uring-check doc-paths-check quick-check ## Run fast local development checks
 	@echo "✅ Fast development checks passed!"

.config/make/tests.mak

@@ -2,67 +2,20 @@
 
 TEST_THREADS ?= 1
 
-# cargo-nextest is a HARD dependency of `make test`.
-#
-# nextest changes test semantics vs plain `cargo test`: it runs every test in
-# its own process (so serial_test's #[serial] mutex does not serialize across
-# tests) and it is the only runner that honours .config/nextest.toml
-# [test-groups] (e.g. the ecstore-serial-flaky serialization guard). CI runs
-# nextest (.github/actions/setup/action.yml installs it), so a silent fallback
-# to `cargo test` would run with different serialization behaviour than CI and
-# mask (or invent) flakes.
-#
-# Installing cargo-nextest:
-#   cargo install cargo-nextest --locked          # from source
-#   # or a prebuilt binary (faster) via the taiki-e installer / get.nexte.st:
-#   #   https://nexte.st/docs/installation/
-#
-# Escape hatch: set RUSTFS_ALLOW_CARGO_TEST_FALLBACK=1 to run the plain
-# `cargo test` fallback anyway. Results are NOT authoritative — semantics
-# differ from CI and .config/nextest.toml test-groups will NOT apply.
-
 .PHONY: script-tests
 script-tests: ## Run shell script tests
 	@echo "Running script tests..."
 	./scripts/test_build_rustfs_options.sh
 	./scripts/test_entrypoint_credentials.sh
-	bash -n ./scripts/validate_object_data_cache_cold_stampede.sh
-	python3 ./scripts/check_object_data_cache_follower_samples.py --self-test
-	./scripts/validate_object_data_cache_cold_stampede.sh --self-test
 
 .PHONY: test
-test: core-deps script-tests ## Run all tests (needs cargo-nextest; RUSTFS_ALLOW_CARGO_TEST_FALLBACK=1 to override)
+test: core-deps test-deps script-tests ## Run all tests
 	@echo "🧪 Running tests..."
 	@if command -v cargo-nextest >/dev/null 2>&1; then \
 		cargo nextest run --all --exclude e2e_test; \
-	elif [ "$${RUSTFS_ALLOW_CARGO_TEST_FALLBACK:-0}" = "1" ]; then \
-		echo >&2 "⚠️  ============================================================================"; \
-		echo >&2 "⚠️  cargo-nextest NOT found — running the 'cargo test' fallback (opt-in)."; \
-		echo >&2 "⚠️  TEST SEMANTICS DIFFER FROM CI; results are NOT authoritative:"; \
-		echo >&2 "⚠️    * nextest runs each test in its own process; 'cargo test' does not,"; \
-		echo >&2 "⚠️      so serial_test #[serial] serialization behaves differently."; \
-		echo >&2 "⚠️    * .config/nextest.toml [test-groups] will NOT apply (e.g. the"; \
-		echo >&2 "⚠️      ecstore-serial-flaky group), so load-sensitive tests may flake here"; \
-		echo >&2 "⚠️      but pass on CI (or vice versa)."; \
-		echo >&2 "⚠️  Install cargo-nextest and re-run before trusting these results."; \
-		echo >&2 "⚠️  ============================================================================"; \
-		cargo test --workspace --exclude e2e_test -- --nocapture --test-threads="$(TEST_THREADS)"; \
 	else \
-		echo >&2 "❌ cargo-nextest is required for 'make test' but was not found."; \
-		echo >&2 ""; \
-		echo >&2 "   RustFS tests run under cargo-nextest (process-per-test isolation)."; \
-		echo >&2 "   CI runs nextest and .config/nextest.toml [test-groups] only take effect"; \
-		echo >&2 "   under nextest. Plain 'cargo test' has different serialization semantics"; \
-		echo >&2 "   and is NOT a faithful substitute."; \
-		echo >&2 ""; \
-		echo >&2 "   Install it with either:"; \
-		echo >&2 "     cargo install cargo-nextest --locked"; \
-		echo >&2 "   or a prebuilt binary (faster) — see https://nexte.st/docs/installation/"; \
-		echo >&2 ""; \
-		echo >&2 "   To run the plain 'cargo test' fallback anyway (results NOT authoritative;"; \
-		echo >&2 "   serialization semantics differ from CI), re-run with:"; \
-		echo >&2 "     RUSTFS_ALLOW_CARGO_TEST_FALLBACK=1 make test"; \
-		exit 1; \
+		echo "ℹ️ cargo-nextest not found; falling back to 'cargo test'"; \
+		cargo test --workspace --exclude e2e_test -- --nocapture --test-threads="$(TEST_THREADS)"; \
 	fi
 	cargo test --all --doc
 

.config/migration-gate-floor.txt

@@ -1,10 +0,0 @@
-# Committed floor for the number of tests selected by the migration-critical
-# CI gate (see scripts/check_migration_gate_count.sh, backlog#1153 infra-12).
-#
-# The floor equals the exact count of rustfs-ecstore --lib tests matching the
-# gate filter (name substrings: data_movement, rebalance, decommission,
-# source_cleanup, delete_marker) at the time this file was last updated.
-# CI fails if the selected count drops below this number, so renames or
-# removals that thin the gate must update this file in the same PR.
-# Adding tests does not require a bump, but bumping keeps the guard tight.
-571

.config/nextest.toml

@@ -1,316 +0,0 @@
-# nextest configuration for RustFS.
-#
-# Serialize two known load-sensitive / global-state-sharing ecstore test groups
-# so the full parallel nextest suite stops producing spurious failures
-# (backlog #937). These tests pass in isolation but flake under the loaded
-# parallel run for two distinct reasons:
-#
-#   * store::bucket::tests::bucket_delete_* share process/global state (disk
-#     registry, lock client) and race make_bucket into InsufficientWriteQuorum
-#     when run concurrently with other ecstore tests.
-#   * bucket_lifecycle_ops::tests::concurrent_resend_same_part_commits_one_generation
-#     asserts a lock-acquire correctness property whose serialized cross-disk
-#     commits exceed the (already max'd, 60s) acquire deadline only when the
-#     suite saturates disk I/O.
-#
-# serial_test's #[serial] attribute does NOT serialize these across runs:
-# nextest executes each test in its own process, where the in-process
-# serial_test mutex has no effect. A nextest test-group with max-threads = 1 is
-# the mechanism that actually serializes across nextest's process boundary.
-#
-# ---------------------------------------------------------------------------
-# Profiles
-# ---------------------------------------------------------------------------
-# The `default` profile is what local `cargo nextest run` uses. It NEVER
-# retries: a red test locally means a real failure to investigate, not noise to
-# paper over. The `ci` profile (below) is the strict CI gate: global
-# retries = 0 so a new race's first occurrence is never masked, plus a
-# narrowly-scoped quarantine list (retries = 2) for tests with a tracked OPEN
-# flake issue. Flake policy lives in docs/testing/README.md.
-
-[test-groups]
-ecstore-serial-flaky = { max-threads = 1 }
-
-# Reliability / fault-injection e2e tests each spawn a single-node 4-disk RustFS
-# server and manipulate its disk directories at runtime (crates/e2e_test:
-# reliability_disk_fault_test, degraded_read_eof_regression_test / dist-13). They
-# are correct in isolation but resource-heavy; serialize them under nextest's
-# process boundary (serial_test's #[serial] does not cross it) so several 4-disk
-# servers never run at once. ci-7's nightly picks these up via the e2e suite;
-# they are deliberately NOT in the fast PR `e2e-smoke` filter.
-e2e-reliability = { max-threads = 1 }
-
-# --- default profile (local): serialize the flaky groups, never retry --------
-[[profile.default.overrides]]
-filter = 'package(rustfs-ecstore) & (test(concurrent_resend_same_part_commits_one_generation) | test(/^store::bucket::tests::bucket_delete_(mark_delete|purge_removes|default_s3_delete)/))'
-test-group = 'ecstore-serial-flaky'
-
-# Serialize the multipart crash-consistency scenarios (dist-2, backlog#1150):
-# each spawns a 4-disk hermetic erasure set and drives full staged-upload +
-# commit + GET cycles — the same cross-disk-commit IO shape that made
-# concurrent_resend load-sensitive. Preventive serialization only, no retries.
-# The matching ci-profile override is after [profile.ci].
-[[profile.default.overrides]]
-filter = 'package(rustfs-ecstore) & test(/^set_disk::ops::multipart::tests::crash_consistency::/)'
-test-group = 'ecstore-serial-flaky'
-
-# Serialize the 4-disk reliability / degraded-read e2e tests (see the
-# e2e-reliability test-group note above). The matching ci-profile override is at
-# the end of the file, after [profile.ci] is declared.
-[[profile.default.overrides]]
-filter = 'package(e2e_test) & test(/^(reliability_disk_fault|degraded_read_eof_regression)_test::/)'
-test-group = 'e2e-reliability'
-
-# ---------------------------------------------------------------------------
-# ci profile — the strict CI gate (ci.yml `cargo nextest run --profile ci`)
-# ---------------------------------------------------------------------------
-[profile.ci]
-# Strict: a new race must fail on its first occurrence, never be retried away.
-retries = 0
-# Report every failure in one run instead of bailing on the first.
-fail-fast = false
-
-[profile.ci.junit]
-# Emitted to target/nextest/ci/junit.xml; uploaded as a CI artifact.
-# Tests that pass only after a quarantine retry are marked `flaky` here — that
-# marker is the observable signal the flake policy is built around.
-path = "junit.xml"
-
-# ===========================================================================
-# QUARANTINE — flaky tests granted retries = 2 under the ci profile ONLY.
-#
-# RULES (enforced by review, see docs/testing/README.md):
-#   * Every entry MUST link exactly one OPEN issue tracking the flake.
-#   * An entry stays until the issue is fixed (test made robust) or the test is
-#     deleted — 30-day policy. No entry may exist without a live issue link.
-#
-# Each entry also re-declares the `ecstore-serial-flaky` test-group so the
-# serialization holds under the ci profile (nextest evaluates a named
-# profile's own overrides list, not the default profile's).
-# ===========================================================================
-
-# QUARANTINE: OPEN backlog#937 — concurrent_resend lock-acquire deadline flakes
-# under saturated disk I/O in the full parallel suite.
-[[profile.ci.overrides]]
-filter = 'package(rustfs-ecstore) & test(concurrent_resend_same_part_commits_one_generation)'
-test-group = 'ecstore-serial-flaky'
-retries = 2
-
-# QUARANTINE: OPEN backlog#937 — store::bucket::tests::bucket_delete_* race
-# make_bucket into InsufficientWriteQuorum via shared global state under load.
-[[profile.ci.overrides]]
-filter = 'package(rustfs-ecstore) & test(/^store::bucket::tests::bucket_delete_(mark_delete|purge_removes|default_s3_delete)/)'
-test-group = 'ecstore-serial-flaky'
-retries = 2
-
-# QUARANTINE: OPEN rustfs#4690 — walk_dir stall-budget accounting test depends
-# on producer/consumer timing windows that stretch past the budget on loaded
-# CI runners (regression test for rustfs#4644; failed on a zero-Rust-diff PR).
-[[profile.ci.overrides]]
-filter = 'package(rustfs-ecstore) & test(walk_dir_does_not_charge_consumer_backpressure_to_the_stall_budget)'
-retries = 2
-
-# Serialize the 4-disk reliability / degraded-read e2e tests under the ci
-# profile too (see the e2e-reliability test-group note near the top). Not a
-# quarantine: no retries, just single-threaded so several 4-disk servers never
-# run concurrently when ci-7's nightly runs the full e2e suite.
-[[profile.ci.overrides]]
-filter = 'package(e2e_test) & test(/^(reliability_disk_fault|degraded_read_eof_regression)_test::/)'
-test-group = 'e2e-reliability'
-
-# Serialize the multipart crash-consistency scenarios under the ci profile too
-# (see the matching default-profile override near the top). Not a quarantine:
-# no retries, just serialized 4-disk cross-disk-commit IO.
-[[profile.ci.overrides]]
-filter = 'package(rustfs-ecstore) & test(/^set_disk::ops::multipart::tests::crash_consistency::/)'
-test-group = 'ecstore-serial-flaky'
-
-# ---------------------------------------------------------------------------
-# e2e-smoke profile — PR smoke subset of the e2e_test crate (backlog#1149 ci-4)
-# ---------------------------------------------------------------------------
-# PR smoke subset of the e2e_test crate (backlog#1149 ci-4). This profile is
-# the single wiring mechanism for e2e tests in CI: other suites join by
-# extending this filter (or a sibling profile), never by adding ad-hoc e2e
-# jobs to ci.yml. Admission criteria (see crates/e2e_test/README.md): fast,
-# single-node topology, no external dependencies (no awscurl / Vault / fixed
-# ports / pre-started server), no #[ignore].
-#
-# Each e2e test spawns its own rustfs server on a random port with an isolated
-# temp dir (crates/e2e_test/src/common.rs), so the subset is parallel-safe.
-#
-# Replication failure harness (backlog#1147 repl-8): the first clause admits
-# its four in-process fake-target self-tests. They bind random loopback ports,
-# use no external service, and finish in under a second.
-#
-# Replication PR subset (backlog#1147 repl-1): the second clause admits the 20
-# FAST bucket-replication tests from replication_extension_test — the
-# target-registration / replication-check / list / remove / delete admin paths
-# that validate config synchronously and never wait for asynchronous
-# replication convergence. Each spawns its own single-node rustfs server(s) on
-# random ports (source, plus an independent single-node target for the pair
-# checks — NOT a cluster), so the subset stays parallel-safe and single-digit
-# seconds. The data-plane tests that poll for convergence and all
-# `_real_dual_node` / `_real_single_node`
-# site-replication tests run in the [profile.e2e-repl-nightly] lane below, NOT
-# here. This allowlist is the single source of truth for the PR/nightly split:
-# the nightly profile derives its set as "the replication module MINUS this
-# allowlist", so any new replication test lands in nightly by default (never
-# silently unrun) until it is explicitly blessed as fast here. Keep the two
-# regexes byte-identical. Count invariant: 20 here + 27 nightly = 47 total
-# (authority: `cargo nextest list`; docs/testing/e2e-suite-inventory.md).
-# HISTORY (2026-07-11): the 20 fast tests were briefly pulled out of this lane
-# (#4724) because they set a loopback (127.0.0.1) replication target that the
-# SSRF egress guard rejected on every PR after repl-1 (#4712). That is fixed —
-# the guard now honours an off-by-default opt-in and this suite's source servers
-# set it (RUSTFS_REPLICATION_ALLOW_LOOPBACK_TARGET) — so the allowlist below is
-# restored.
-#
-# Security negative-auth subset (backlog#1151 sec-5): the three attacker-facing
-# S3 auth-rejection suites join the first clause above by module name —
-# presigned_negative (sec-2), negative_sigv4 (sec-1, header SigV4), and
-# admin_auth (sec-4, admin gate + root-credential lifecycle). All three use
-# RustFSTestEnvironment on a random port and are parallel-safe, so they meet the
-# smoke admission criteria unchanged. This is the wiring step that makes those
-# merged suites actually execute on every PR (they were dead until listed here).
-# A rename that drops any of them out of this filter would silently thin the
-# security gate with no CI signal, so scripts/check_security_smoke_count.sh owns
-# a count-floor guard over exactly this subset (infra-12 mechanism, floor in
-# .config/security-smoke-floor.txt), invoked from the e2e-tests job in ci.yml.
-# NOT here by topology: the GHSA-3p3x FTPS/WebDAV constant-time e2e
-# (protocols::test_protocol_core_suite) binds fixed ports and needs the
-# ftps,webdav features, so it cannot join this random-port, default-feature
-# profile; its GHSA-r5qv sibling is a unit test that already runs in the
-# test-and-lint `--all --exclude e2e_test` pass. See
-# docs/testing/security-regressions.md for the full CI-execution map.
-#
-# ILM tiering main path (backlog#1148 ilm-7): the `reliant::tiering::` clause
-# admits the hermetic transition e2e. Like the fast replication pair checks it
-# spawns a second independent single-node server (the cold RustFS tier), not a
-# cluster, so it keeps the lane's parallel-safe / no-external-dependency
-# properties. The RustFS warm backend has no loopback guard (that guard is
-# replication-only), so it needs no opt-in env for its 127.0.0.1 tier target.
-[profile.e2e-smoke]
-default-filter = """
-    package(e2e_test) & (
-        test(/^(delete_marker_migration_semantics|version_id_regression|list_objects_v2_pagination|list_object_versions_regression|list_objects_duplicates|list_buckets_double_slash|leading_slash_key|special_chars|create_bucket_region|delete_objects_versioning|head_object_consistency|head_object_range|copy_object_metadata|copy_source_invalid_date|content_encoding|anonymous_access|bucket_policy_check|presigned_negative|negative_sigv4|admin_auth|notification_webhook|tls_hot_reload|console_smoke|admin_iam_crud|admin_pools)_test::|^fake_s3_target::/)
-        | test(/^replication_extension_test::(test_replication_check_succeeds_with_remote_target|test_replication_check_rejects_target_without_object_lock|test_set_remote_target_rejects_unversioned_source_bucket|test_replication_check_rejects_unversioned_source_bucket|test_replication_check_rejects_missing_replication_config|test_replication_check_rejects_invalid_bucket|test_set_remote_target_rejects_same_bucket_on_same_deployment|test_set_remote_target_rejects_unversioned_target_bucket|test_set_remote_target_update_requires_arn|test_set_remote_target_update_rejects_missing_target|test_set_remote_target_rejects_invalid_target_url|test_set_remote_target_rejects_self_signed_https_target_without_skip_tls_verify|test_set_remote_target_rejects_private_ca_https_target_without_ca_cert_pem|test_list_remote_targets_rejects_empty_bucket|test_list_remote_targets_rejects_invalid_bucket|test_remove_remote_target_rejects_missing_target|test_remove_remote_target_rejects_missing_arn|test_remove_remote_target_rejects_invalid_bucket|test_remove_remote_target_rejects_target_used_by_replication|test_delete_bucket_replication_removes_remote_target)$/)
-        | test(/^reliant::lifecycle::/)
-        | test(/^reliant::tiering::/)
-    )
-"""
-fail-fast = false
-
-# ---------------------------------------------------------------------------
-# e2e-repl-nightly profile — scheduled full replication e2e lane (repl-1)
-# ---------------------------------------------------------------------------
-# backlog#1147 repl-1 (deps: ci-4). Runs the SLOW / cross-process replication
-# tests that are unfit for the per-PR e2e-smoke gate:
-#
-#   * 2 remote-target TLS validation tests.
-#   * 12 bucket-replication data-plane/helper tests — they PUT/delete objects
-#     and poll until source and target converge; two replicate over HTTPS, two
-#     pin active SSE failure contracts, and one guards event/history observers.
-#     The SSE-S3 contract remains ignored under backlog#1291.
-#   * 11 `_real_dual_node` site-replication tests — each spawns TWO full rustfs
-#     servers and drives the cross-process site-replication control plane.
-#   * 1 `_real_three_node` site-replication test.
-#   * 1 `_real_single_node` service-account round-trip test.
-#
-# The set is defined as "everything in replication_extension_test that is NOT
-# in the e2e-smoke PR allowlist above" (the negated clause is byte-identical to
-# the allowlist), so a newly added replication test automatically runs here
-# until it is explicitly promoted to the fast PR subset — no replication test
-# is ever silently left out of CI.
-#
-# #[serial] does NOT serialize under nextest (process-per-test; see the file
-# header). These tests need no cross-test serialization: each spawns its own
-# server(s) on random ports with isolated temp dirs, so they are parallel-safe
-# by construction — the same property the e2e-smoke subset relies on. If load
-# on the runner surfaces a real flake, quarantine the specific test with an
-# OPEN issue link (ci-10 / backlog#937 policy), never blanket-retry or exclude.
-#
-# Wired by .github/workflows/e2e-replication-nightly.yml (schedule +
-# workflow_dispatch), which builds the rustfs binary once, installs awscurl so
-# the STS dual-node test actually exercises its path (it skips gracefully with
-# a visible log line when awscurl is absent), and routes scheduled failures
-# through .github/actions/schedule-failure-issue (ci-8). Explicit division of
-# labor with ci-5's future e2e-full merge gate: these tests run ONLY here, not
-# double-run there. TODO(ci-7): fold this interim repl-owned lane into the ci
-# domain's consolidated scheduled e2e workflow once it exists.
-[profile.e2e-repl-nightly]
-default-filter = """
-    package(e2e_test)
-    & test(/^replication_extension_test::/)
-    & !test(/^replication_extension_test::(test_replication_check_succeeds_with_remote_target|test_replication_check_rejects_target_without_object_lock|test_set_remote_target_rejects_unversioned_source_bucket|test_replication_check_rejects_unversioned_source_bucket|test_replication_check_rejects_missing_replication_config|test_replication_check_rejects_invalid_bucket|test_set_remote_target_rejects_same_bucket_on_same_deployment|test_set_remote_target_rejects_unversioned_target_bucket|test_set_remote_target_update_requires_arn|test_set_remote_target_update_rejects_missing_target|test_set_remote_target_rejects_invalid_target_url|test_set_remote_target_rejects_self_signed_https_target_without_skip_tls_verify|test_set_remote_target_rejects_private_ca_https_target_without_ca_cert_pem|test_list_remote_targets_rejects_empty_bucket|test_list_remote_targets_rejects_invalid_bucket|test_remove_remote_target_rejects_missing_target|test_remove_remote_target_rejects_missing_arn|test_remove_remote_target_rejects_invalid_bucket|test_remove_remote_target_rejects_target_used_by_replication|test_delete_bucket_replication_removes_remote_target)$/)
-"""
-fail-fast = false
-
-[profile.e2e-repl-nightly.junit]
-# Emitted to target/nextest/e2e-repl-nightly/junit.xml; uploaded by the nightly
-# workflow as the failure-triage artifact.
-path = "junit.xml"
-
-# ---------------------------------------------------------------------------
-# e2e-full profile — merge-gate full single-node e2e lane (backlog#1149 ci-5)
-# ---------------------------------------------------------------------------
-# The merge gate (ci.yml `e2e-full` job: push main + merge_group +
-# workflow_dispatch). Runs the never-automated user-visible suites — KMS (40),
-# object_lock (33), multipart_auth (109), quota, checksum, encryption,
-# security-boundary, ... — that the fast PR `e2e-smoke` subset deliberately
-# skips. Budget <= 45 min; authority for the suite count is `cargo nextest list
-# --profile e2e-full` (see docs/testing/e2e-suite-inventory.md).
-#
-# The filter is "the whole e2e_test crate MINUS the sets owned by other lanes":
-#   * protocols:: — FTPS/SFTP/WebDAV, still pinned to --test-threads=1 by fixed
-#     ports; they join a scheduled lane once ci-6 randomises the ports (ci-7).
-#   * the 7 cluster suites that spin up a RustFSTestClusterEnvironment
-#     (cluster_concurrency, cluster_multidrive_pool, stale_multipart_cleanup_cluster,
-#     namespace_lock_quorum, heal_erasure_disk_rebuild, admin_timeout_regression,
-#     object_lambda) — too heavy for the merge budget; they run in ci-7's
-#     nightly 4-node lane.
-#   * replication_extension_test — repl-1 already splits it into the PR
-#     `e2e-smoke` (20 fast) and `e2e-repl-nightly` (27 slow) lanes and reserves
-#     it for those, so e2e-full does not double-run it.
-#   * #[ignore]d tests — nextest skips them by default (no --run-ignored); the
-#     manual-localhost:9000 reliant/policy tests are ci-13's migration.
-#
-# Each e2e test spawns its own single-node rustfs server on a random port with
-# an isolated temp dir (crates/e2e_test/src/common.rs), so the set is
-# parallel-safe — the same property e2e-smoke relies on. The exception is the
-# 4-disk reliability / degraded-read fault-injection tests, serialized below
-# (identical to the ci profile) so several 4-disk servers never run at once.
-# KNOWN-FAILURE EXCLUSIONS (characterization run 29381309848, 2026-07-15:
-# 341 ran / 32 failed on the suites' first automated run ever). Deterministic
-# product failures cannot be quarantined away with retries, so each family is
-# excluded here with its tracking issue, under the same discipline as the
-# ci-profile quarantine (docs/testing/README.md): every entry MUST cite one
-# OPEN issue, and the fixing PR MUST delete the exclusion. The passing
-# negative-path siblings of each family stay in as regression guards.
-#   * rustfs#4843 — over-limit archive entry paths hard-reject the whole
-#     archive even under ignore-errors semantics.
-#   * rustfs#4846 — distributed-lock quorum tests misclassify as timeout
-#     under parallel load (multi-node in-process clusters; natural home is
-#     ci-7's nightly cluster lane).
-[profile.e2e-full]
-default-filter = """
-    package(e2e_test)
-    & !test(/^protocols::/)
-    & !test(/^(admin_timeout_regression_test|cluster_concurrency_test|cluster_multidrive_pool_test|heal_erasure_disk_rebuild_test|namespace_lock_quorum_test|object_lambda_test|stale_multipart_cleanup_cluster_test)::/)
-    & !test(/^replication_extension_test::/)
-    & !test(/^multipart_auth_test::test_signed_put_object_extract_skips_invalid_entry_when_ignore_errors_enabled$/)
-    & !test(/^snowball_auto_extract_test::tests::snowball_auto_extract_(ignores_invalid_entries_when_requested|supports_standard_headers_with_combined_extract_options)$/)
-    & !test(/^reliant::lock::test_distributed_lock_(2_nodes_grpc_read_survives_failed_node|4_nodes_grpc_read_write_quorum_split_with_two_failed_nodes)$/)
-"""
-fail-fast = false
-
-[profile.e2e-full.junit]
-# Emitted to target/nextest/e2e-full/junit.xml; uploaded by the e2e-full job.
-path = "junit.xml"
-
-# Serialize the 4-disk reliability / degraded-read e2e tests under e2e-full too
-# (see the e2e-reliability test-group note near the top of this file). Not a
-# quarantine: no retries, just single-threaded so several 4-disk servers never
-# run concurrently.
-[[profile.e2e-full.overrides]]
-filter = 'package(e2e_test) & test(/^(reliability_disk_fault|degraded_read_eof_regression)_test::/)'
-test-group = 'e2e-reliability'

.config/security-smoke-floor.txt

@@ -1,12 +0,0 @@
-# Committed floor for the number of security negative-auth tests selected by the
-# e2e-smoke PR profile (see scripts/check_security_smoke_count.sh, backlog#1151
-# sec-5).
-#
-# The floor equals the exact count of e2e_test cases whose name starts with a
-# security module prefix (negative_sigv4_test, presigned_negative_test,
-# admin_auth_test) that the [profile.e2e-smoke] default-filter in
-# .config/nextest.toml selects, at the time this file was last updated. CI fails
-# if the selected count drops below this number, so a rename or removal that
-# thins the security smoke gate must update this file in the same PR.
-# Adding tests does not require a bump, but bumping keeps the guard tight.
-16

.docker/compose/docker-compose.cluster.local-build.yml

@@ -23,8 +23,8 @@ services:
       - RUSTFS_VOLUMES=${RUSTFS_VOLUMES:-http://node{1...4}:9000/data/rustfs{0...3}}
       - RUSTFS_ADDRESS=:9000
       - RUSTFS_CONSOLE_ENABLE=true
-      - RUSTFS_ACCESS_KEY=${RUSTFS_ACCESS_KEY:-rustfsadmin-local}
-      - RUSTFS_SECRET_KEY=${RUSTFS_SECRET_KEY:-rustfssecret-local}
+      - RUSTFS_ACCESS_KEY=${RUSTFS_ACCESS_KEY:-rustfsadmin}
+      - RUSTFS_SECRET_KEY=${RUSTFS_SECRET_KEY:-rustfsadmin}
       - RUSTFS_OBS_ENDPOINT=${RUSTFS_OBS_ENDPOINT:-http://host.docker.internal:4318}
       - RUSTFS_OBS_LOGGER_LEVEL=${RUSTFS_OBS_LOGGER_LEVEL:-info}
       - RUSTFS_OBS_USE_STDOUT=${RUSTFS_OBS_USE_STDOUT:-false}
@@ -46,7 +46,7 @@ services:
       - RUSTFS_INTERNODE_OFFLINE_BYPASS=${RUSTFS_INTERNODE_OFFLINE_BYPASS:-false}
       - RUSTFS_INTERNODE_OFFLINE_REPROBE_SECS=${RUSTFS_INTERNODE_OFFLINE_REPROBE_SECS:-5}
       - RUSTFS_INTERNODE_OFFLINE_FAILURE_THRESHOLD=${RUSTFS_INTERNODE_OFFLINE_FAILURE_THRESHOLD:-3}
-      - RUSTFS_INTERNODE_IDEMPOTENT_READ_RETRIES=${RUSTFS_INTERNODE_IDEMPOTENT_READ_RETRIES:-1}
+      - RUSTFS_INTERNODE_IDEMPOTENT_READ_RETRIES=${RUSTFS_INTERNODE_IDEMPOTENT_READ_RETRIES:-0}
     extra_hosts:
       - "host.docker.internal:host-gateway"
     volumes:
@@ -69,8 +69,8 @@ services:
       - RUSTFS_VOLUMES=${RUSTFS_VOLUMES:-http://node{1...4}:9000/data/rustfs{0...3}}
       - RUSTFS_ADDRESS=:9000
       - RUSTFS_CONSOLE_ENABLE=true
-      - RUSTFS_ACCESS_KEY=${RUSTFS_ACCESS_KEY:-rustfsadmin-local}
-      - RUSTFS_SECRET_KEY=${RUSTFS_SECRET_KEY:-rustfssecret-local}
+      - RUSTFS_ACCESS_KEY=${RUSTFS_ACCESS_KEY:-rustfsadmin}
+      - RUSTFS_SECRET_KEY=${RUSTFS_SECRET_KEY:-rustfsadmin}
       - RUSTFS_OBS_ENDPOINT=${RUSTFS_OBS_ENDPOINT:-http://host.docker.internal:4318}
       - RUSTFS_OBS_LOGGER_LEVEL=${RUSTFS_OBS_LOGGER_LEVEL:-info}
       - RUSTFS_OBS_USE_STDOUT=${RUSTFS_OBS_USE_STDOUT:-false}
@@ -92,7 +92,7 @@ services:
       - RUSTFS_INTERNODE_OFFLINE_BYPASS=${RUSTFS_INTERNODE_OFFLINE_BYPASS:-false}
       - RUSTFS_INTERNODE_OFFLINE_REPROBE_SECS=${RUSTFS_INTERNODE_OFFLINE_REPROBE_SECS:-5}
       - RUSTFS_INTERNODE_OFFLINE_FAILURE_THRESHOLD=${RUSTFS_INTERNODE_OFFLINE_FAILURE_THRESHOLD:-3}
-      - RUSTFS_INTERNODE_IDEMPOTENT_READ_RETRIES=${RUSTFS_INTERNODE_IDEMPOTENT_READ_RETRIES:-1}
+      - RUSTFS_INTERNODE_IDEMPOTENT_READ_RETRIES=${RUSTFS_INTERNODE_IDEMPOTENT_READ_RETRIES:-0}
     extra_hosts:
       - "host.docker.internal:host-gateway"
     volumes:
@@ -115,8 +115,8 @@ services:
       - RUSTFS_VOLUMES=${RUSTFS_VOLUMES:-http://node{1...4}:9000/data/rustfs{0...3}}
       - RUSTFS_ADDRESS=:9000
       - RUSTFS_CONSOLE_ENABLE=true
-      - RUSTFS_ACCESS_KEY=${RUSTFS_ACCESS_KEY:-rustfsadmin-local}
-      - RUSTFS_SECRET_KEY=${RUSTFS_SECRET_KEY:-rustfssecret-local}
+      - RUSTFS_ACCESS_KEY=${RUSTFS_ACCESS_KEY:-rustfsadmin}
+      - RUSTFS_SECRET_KEY=${RUSTFS_SECRET_KEY:-rustfsadmin}
       - RUSTFS_OBS_ENDPOINT=${RUSTFS_OBS_ENDPOINT:-http://host.docker.internal:4318}
       - RUSTFS_OBS_LOGGER_LEVEL=${RUSTFS_OBS_LOGGER_LEVEL:-info}
       - RUSTFS_OBS_USE_STDOUT=${RUSTFS_OBS_USE_STDOUT:-false}
@@ -138,7 +138,7 @@ services:
       - RUSTFS_INTERNODE_OFFLINE_BYPASS=${RUSTFS_INTERNODE_OFFLINE_BYPASS:-false}
       - RUSTFS_INTERNODE_OFFLINE_REPROBE_SECS=${RUSTFS_INTERNODE_OFFLINE_REPROBE_SECS:-5}
       - RUSTFS_INTERNODE_OFFLINE_FAILURE_THRESHOLD=${RUSTFS_INTERNODE_OFFLINE_FAILURE_THRESHOLD:-3}
-      - RUSTFS_INTERNODE_IDEMPOTENT_READ_RETRIES=${RUSTFS_INTERNODE_IDEMPOTENT_READ_RETRIES:-1}
+      - RUSTFS_INTERNODE_IDEMPOTENT_READ_RETRIES=${RUSTFS_INTERNODE_IDEMPOTENT_READ_RETRIES:-0}
     extra_hosts:
       - "host.docker.internal:host-gateway"
     volumes:
@@ -161,8 +161,8 @@ services:
       - RUSTFS_VOLUMES=${RUSTFS_VOLUMES:-http://node{1...4}:9000/data/rustfs{0...3}}
       - RUSTFS_ADDRESS=:9000
       - RUSTFS_CONSOLE_ENABLE=true
-      - RUSTFS_ACCESS_KEY=${RUSTFS_ACCESS_KEY:-rustfsadmin-local}
-      - RUSTFS_SECRET_KEY=${RUSTFS_SECRET_KEY:-rustfssecret-local}
+      - RUSTFS_ACCESS_KEY=${RUSTFS_ACCESS_KEY:-rustfsadmin}
+      - RUSTFS_SECRET_KEY=${RUSTFS_SECRET_KEY:-rustfsadmin}
       - RUSTFS_OBS_ENDPOINT=${RUSTFS_OBS_ENDPOINT:-http://host.docker.internal:4318}
       - RUSTFS_OBS_LOGGER_LEVEL=${RUSTFS_OBS_LOGGER_LEVEL:-info}
       - RUSTFS_OBS_USE_STDOUT=${RUSTFS_OBS_USE_STDOUT:-false}
@@ -184,7 +184,7 @@ services:
       - RUSTFS_INTERNODE_OFFLINE_BYPASS=${RUSTFS_INTERNODE_OFFLINE_BYPASS:-false}
       - RUSTFS_INTERNODE_OFFLINE_REPROBE_SECS=${RUSTFS_INTERNODE_OFFLINE_REPROBE_SECS:-5}
       - RUSTFS_INTERNODE_OFFLINE_FAILURE_THRESHOLD=${RUSTFS_INTERNODE_OFFLINE_FAILURE_THRESHOLD:-3}
-      - RUSTFS_INTERNODE_IDEMPOTENT_READ_RETRIES=${RUSTFS_INTERNODE_IDEMPOTENT_READ_RETRIES:-1}
+      - RUSTFS_INTERNODE_IDEMPOTENT_READ_RETRIES=${RUSTFS_INTERNODE_IDEMPOTENT_READ_RETRIES:-0}
     extra_hosts:
       - "host.docker.internal:host-gateway"
     volumes:

.docker/compose/docker-compose.cluster.yaml

@@ -21,8 +21,8 @@ services:
       - RUSTFS_VOLUMES=http://node{0...3}:9000/data/rustfs{0...3}
       - RUSTFS_ADDRESS=0.0.0.0:9000
       - RUSTFS_CONSOLE_ENABLE=true
-      - RUSTFS_ACCESS_KEY=${RUSTFS_ACCESS_KEY:-rustfsadmin-local}
-      - RUSTFS_SECRET_KEY=${RUSTFS_SECRET_KEY:-rustfssecret-local}
+      - RUSTFS_ACCESS_KEY=rustfsadmin
+      - RUSTFS_SECRET_KEY=rustfsadmin
     platform: linux/amd64
     ports:
       - "9000:9000" # Map port 9001 of the host to port 9000 of the container
@@ -38,8 +38,8 @@ services:
       - RUSTFS_VOLUMES=http://node{0...3}:9000/data/rustfs{0...3}
       - RUSTFS_ADDRESS=0.0.0.0:9000
       - RUSTFS_CONSOLE_ENABLE=true
-      - RUSTFS_ACCESS_KEY=${RUSTFS_ACCESS_KEY:-rustfsadmin-local}
-      - RUSTFS_SECRET_KEY=${RUSTFS_SECRET_KEY:-rustfssecret-local}
+      - RUSTFS_ACCESS_KEY=rustfsadmin
+      - RUSTFS_SECRET_KEY=rustfsadmin
     platform: linux/amd64
     ports:
       - "9001:9000" # Map port 9002 of the host to port 9000 of the container
@@ -55,8 +55,8 @@ services:
       - RUSTFS_VOLUMES=http://node{0...3}:9000/data/rustfs{0...3}
       - RUSTFS_ADDRESS=0.0.0.0:9000
       - RUSTFS_CONSOLE_ENABLE=true
-      - RUSTFS_ACCESS_KEY=${RUSTFS_ACCESS_KEY:-rustfsadmin-local}
-      - RUSTFS_SECRET_KEY=${RUSTFS_SECRET_KEY:-rustfssecret-local}
+      - RUSTFS_ACCESS_KEY=rustfsadmin
+      - RUSTFS_SECRET_KEY=rustfsadmin
     platform: linux/amd64
     ports:
       - "9002:9000" # Map port 9003 of the host to port 9000 of the container
@@ -72,8 +72,8 @@ services:
       - RUSTFS_VOLUMES=http://node{0...3}:9000/data/rustfs{0...3}
       - RUSTFS_ADDRESS=0.0.0.0:9000
       - RUSTFS_CONSOLE_ENABLE=true
-      - RUSTFS_ACCESS_KEY=${RUSTFS_ACCESS_KEY:-rustfsadmin-local}
-      - RUSTFS_SECRET_KEY=${RUSTFS_SECRET_KEY:-rustfssecret-local}
+      - RUSTFS_ACCESS_KEY=rustfsadmin
+      - RUSTFS_SECRET_KEY=rustfsadmin
     platform: linux/amd64
     ports:
       - "9003:9000" # Map port 9004 of the host to port 9000 of the container

.docker/observability/README.md

@@ -45,7 +45,6 @@ Three pre-built Grafana dashboards are included for monitoring RustFS GET perfor
 | **GET Rollout Health** | `grafana-get-rollout-health.json` | Monitors optimization rollout: latency by reader path, early-stop hit rate, codec streaming usage, pipeline failures |
 | **GET Data Integrity** | `grafana-get-data-integrity.json` | Monitors data safety: bitrot verify failures, decode errors, short reads, shard read outcomes |
 | **GET Resource Impact** | `grafana-get-resource-impact.json` | Monitors resource usage: concurrent requests, IO queue utilization, disk permit wait, RSS trend |
-| **Object Data Cache** | `grafana-object-data-cache.json` | Monitors the GET body cache (`rustfs_object_data_cache_*`): hit ratio, lookup/plan/fill outcomes, fill duration quantiles, hit vs fill throughput, entries/weighted bytes, inflight fills, memory-pressure skips, invalidations, and size-class breakdowns |
 
 ### Prometheus Alert Rules
 

.docker/observability/README_ZH.md

@@ -45,7 +45,6 @@
 | **GET 发布健康度** | `grafana-get-rollout-health.json` | 监控优化发布：按 reader path 的延迟、early-stop 命中率、codec streaming 使用率、pipeline 失败率 |
 | **GET 数据完整性** | `grafana-get-data-integrity.json` | 监控数据安全：bitrot 校验失败、decode 错误、short read、shard 读取结果 |
 | **GET 资源影响** | `grafana-get-resource-impact.json` | 监控资源使用：并发请求数、IO 队列利用率、disk permit 等待、RSS 趋势 |
-| **对象数据缓存** | `grafana-object-data-cache.json` | 监控 GET body 缓存（`rustfs_object_data_cache_*`）：命中率、查找/规划/填充结果、填充耗时分位、命中 vs 填充吞吐、条目数/加权字节、在途填充、内存压力拒绝、失效、按尺寸档拆分 |
 
 ### Prometheus 告警规则
 

.docker/observability/grafana/dashboards/rustfs.json

@@ -1744,7 +1744,7 @@
             "uid": "${datasource}"
           },
           "editorMode": "code",
-          "expr": "max by (job, bucket) (rustfs_cluster_usage_buckets_objects_count{job=~\"$job\", bucket=~\"$bucket\"})",
+          "expr": "sum by (bucket) (rustfs_bucket_api_objects_total{job=~\"$job\", bucket=~\"$bucket\"})",
           "legendFormat": "{{bucket}}",
           "range": true,
           "refId": "A"
@@ -1844,7 +1844,7 @@
             "uid": "${datasource}"
           },
           "editorMode": "code",
-          "expr": "max by (job, bucket) (rustfs_cluster_usage_buckets_total_bytes{job=~\"$job\", bucket=~\"$bucket\"})",
+          "expr": "sum by (bucket) (rustfs_bucket_api_usage_bytes{job=~\"$job\", bucket=~\"$bucket\"})",
           "legendFormat": "{{bucket}}",
           "range": true,
           "refId": "A"
@@ -11583,7 +11583,7 @@
           "text": "All",
           "value": "$__all"
         },
-        "definition": "label_values(rustfs_cluster_usage_buckets_objects_count,bucket)",
+        "definition": "label_values(rustfs_bucket_api_objects_total,bucket)",
         "includeAll": true,
         "label": "Bucket",
         "multi": true,
@@ -11591,7 +11591,7 @@
         "options": [],
         "query": {
           "qryType": 1,
-          "query": "label_values(rustfs_cluster_usage_buckets_objects_count,bucket)",
+          "query": "label_values(rustfs_bucket_api_objects_total,bucket)",
           "refId": "PrometheusVariableQueryEditor-VariableQuery"
         },
         "refresh": 2,

.docker/test/site-replication/run-object-flow-check.sh

@@ -18,7 +18,6 @@ set -eu
 ACCESS_KEY="${RUSTFS_SITE_REPL_ACCESS_KEY:-rustfsadmin}"
 SECRET_KEY="${RUSTFS_SITE_REPL_SECRET_KEY:-rustfsadmin}"
 BUCKET="${RUSTFS_SITE_REPL_FLOW_BUCKET:-site-repl-flow-check}"
-DELETE_BUCKET="${RUSTFS_SITE_REPL_DELETE_BUCKET:-site-repl-delete-$(date +%Y%m%d-%H%M%S)-$$}"
 PREFIX="${RUSTFS_SITE_REPL_FLOW_PREFIX:-flow-$(date +%Y%m%d-%H%M%S)}"
 WAIT_ATTEMPTS="${RUSTFS_SITE_REPL_WAIT_ATTEMPTS:-90}"
 WAIT_SLEEP_SECONDS="${RUSTFS_SITE_REPL_WAIT_SLEEP_SECONDS:-2}"
@@ -86,39 +85,17 @@ wait_for_object() {
 
 wait_for_bucket() {
   site="$1"
-  bucket="${2:-$BUCKET}"
   attempt=1
 
   while [ "$attempt" -le "$WAIT_ATTEMPTS" ]; do
-    if mc stat "$site/$bucket" >/dev/null 2>&1; then
+    if mc stat "$site/$BUCKET" >/dev/null 2>&1; then
       return 0
     fi
     sleep "$WAIT_SLEEP_SECONDS"
     attempt=$((attempt + 1))
   done
 
-  echo "bucket was not replicated in time: $site/$bucket" >&2
-  return 1
-}
-
-wait_for_bucket_delete() {
-  site="$1"
-  bucket="$2"
-  attempt=1
-
-  while [ "$attempt" -le "$WAIT_ATTEMPTS" ]; do
-    if result="$(mc stat --json "$site/$bucket" 2>&1)"; then
-      :
-    else
-      case "$result" in
-        *NoSuchBucket*) return 0 ;;
-      esac
-    fi
-    sleep "$WAIT_SLEEP_SECONDS"
-    attempt=$((attempt + 1))
-  done
-
-  echo "bucket deletion was not replicated in time: $site/$bucket" >&2
+  echo "bucket was not replicated in time: $site/$BUCKET" >&2
   return 1
 }
 
@@ -209,20 +186,6 @@ EOF
   echo "verified replicated downloads for $object_name"
 done
 
-echo "creating empty bucket for replicated delete check: $DELETE_BUCKET"
-mc mb "site1/$DELETE_BUCKET" >/dev/null
-
-for site in site1 site2 site3; do
-  wait_for_bucket "$site" "$DELETE_BUCKET"
-done
-
-echo "deleting empty bucket on site1: $DELETE_BUCKET"
-mc rb "site1/$DELETE_BUCKET" >/dev/null
-
-for site in site1 site2 site3; do
-  wait_for_bucket_delete "$site" "$DELETE_BUCKET"
-done
-
 echo "site replication object flow check passed"
 echo "bucket: $BUCKET"
 echo "prefix: $PREFIX"

.github/AGENTS.md

@@ -1,28 +1,21 @@
 # GitHub Workflow Instructions
 
-Applies to `.github/`.
+Applies to `.github/` and repository pull-request operations.
 
 ## Pull Requests
 
-PR conventions (English title/body, template usage, `--body-file` with the
-heredoc pattern, review-thread etiquette) live in the root `AGENTS.md` under
-"Git and PR Baseline" — that section is the single normative home; do not
-duplicate its rules here.
-
-## Workflow Changes
-
-- Any change touching `.github/workflows/` must pass `actionlint` (run from
-  the repo root) with zero findings before commit/PR. Install via
-  `brew install actionlint`, or the download script in the actionlint repo
-  (rhysd/actionlint); `make pre-pr` does not cover workflow files, so this is
-  the required check for them.
-- Custom self-hosted runner labels (`sm-standard-*`, `dind-*`) are declared
-  in `.github/actionlint.yaml`. When introducing a new `runs-on:` label,
-  declare it there in the same change. Never use the config or `-ignore` to
-  silence real findings — fix the workflow (root `AGENTS.md`: make checks
-  pass by fixing the cause, not by weakening the gate).
-- If `shellcheck` is installed, actionlint also lints `run:` scripts; treat
-  those findings as part of the gate.
+- PR titles and descriptions must be in English.
+- Use `.github/pull_request_template.md` for every PR body.
+- Keep all template section headings.
+- Use `N/A` for non-applicable sections.
+- Include verification commands in the PR details.
+- For `gh pr create` and `gh pr edit`, always write markdown body to a file and pass `--body-file`.
+- Do not use multiline inline `--body`; backticks and shell expansion can corrupt content or trigger unintended commands.
+- Recommended pattern:
+  - `cat > /tmp/pr_body.md <<'EOF'`
+  - `...markdown...`
+  - `EOF`
+  - `gh pr create ... --body-file /tmp/pr_body.md`
 
 ## CI Alignment
 

.github/actionlint.yaml

@@ -1,7 +0,0 @@
-self-hosted-runner:
-  # Custom labels of this repository's self-hosted runners. Keep in sync with
-  # the labels actually used in `runs-on:` across .github/workflows/.
-  labels:
-    - sm-standard-2
-    - sm-standard-4
-    - dind-sm-standard-2

.github/actions/schedule-failure-issue/README.md

@@ -1,74 +0,0 @@
-# schedule-failure-issue
-
-Opens (or updates) a GitHub issue when a **scheduled** workflow run fails.
-This is the single alerting mechanism for all timed pipelines
-(rustfs/backlog#1149 ci-8, arbitration G3): scheduled workflows must consume
-this action instead of inventing their own notification paths.
-
-## Behavior
-
-- Issue title: `[scheduled-failure] <workflow name>` — the workflow name is
-  the dedupe key.
-- If an **open** issue with that exact title exists, the failure is appended
-  as a comment; otherwise a new issue is created (labeled `infrastructure` by
-  default). Closing the issue resets the cycle: the next failure opens a
-  fresh one.
-- The issue body / comment includes the run URL, run attempt, event, ref and
-  the names of the failed (or timed-out/cancelled) jobs of the current run
-  attempt.
-- Requires `gh` and `jq` on the runner (both preinstalled on GitHub-hosted
-  runners such as `ubuntu-latest`).
-
-## Usage
-
-Add a final job to the workflow. `always()` is required — without it the job
-is skipped when a needed job fails; `contains(needs.*.result, 'failure')`
-makes it act only when something actually failed. Scope `issues: write` to
-this job only; no other job may gain permissions.
-
-```yaml
-  alert-on-failure:
-    name: Alert on scheduled failure
-    needs: [<the jobs to watch>]
-    if: always() && github.event_name == 'schedule' && contains(needs.*.result, 'failure')
-    runs-on: ubuntu-latest
-    timeout-minutes: 10
-    permissions:
-      contents: read
-      issues: write
-    steps:
-      - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
-      - name: Open or update failure-tracking issue
-        uses: ./.github/actions/schedule-failure-issue
-        with:
-          github-token: ${{ secrets.GITHUB_TOKEN }}
-```
-
-Notes:
-
-- Upstream jobs that are *cancelled* (e.g. a manually cancelled run) do not
-  trigger the alert — a human was already looking. Job `timeout-minutes`
-  expiry surfaces as `failure` and does alert.
-- Manual `workflow_dispatch` runs never alert; a human is watching those.
-
-## Current consumers
-
-- `.github/workflows/e2e-s3tests.yml` (weekly full s3-tests sweep)
-- `.github/workflows/mint.yml` (weekly multi-SDK mint run)
-- `.github/workflows/fuzz.yml` (nightly fuzz corpus jobs)
-- `.github/workflows/performance-ab.yml` (nightly warp A/B gate)
-- **Pending:** the ci-7 e2e nightly-full workflow does not exist yet
-  (backlog#1149 ci-7). When it lands, wire it up with the snippet above.
-
-## Drill
-
-`.github/workflows/schedule-failure-alert-drill.yml` is a manual-only
-workflow that forces a job failure and runs this action through the exact
-consumer wiring. Use it to verify the alert path end to end:
-
-```bash
-gh workflow run schedule-failure-alert-drill.yml -R rustfs/rustfs --ref <branch>
-```
-
-Run it twice to verify both paths (issue creation, then comment dedupe), and
-close the `[scheduled-failure] Schedule Failure Alert Drill` issue afterwards.

.github/actions/schedule-failure-issue/action.yml

@@ -1,104 +0,0 @@
-# Copyright 2024 RustFS Team
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-name: "Schedule Failure Issue"
-description: >-
-  Open (or update) a tracking issue when a scheduled workflow run fails.
-  Dedupes by workflow name: if an open issue titled
-  "[scheduled-failure] <workflow name>" already exists, the failure is
-  appended as a comment; otherwise a new issue is created. This is the
-  single alerting mechanism for all scheduled pipelines (backlog#1149 ci-8).
-
-inputs:
-  github-token:
-    description: >-
-      Token with issues:write on the repository. Pass secrets.GITHUB_TOKEN
-      from a job that declares `permissions: issues: write` (scope the
-      permission to the alert job only, never workflow-wide).
-    required: true
-  workflow-name:
-    description: "Workflow name used for the issue title (the dedupe key)."
-    required: false
-    default: ${{ github.workflow }}
-  label:
-    description: >-
-      Label applied when a new issue is created. Must already exist in the
-      repository; if applying it fails, the issue is created without a label.
-      Set to an empty string to skip labeling.
-    required: false
-    default: "infrastructure"
-
-runs:
-  using: "composite"
-  steps:
-    - name: Open or update failure-tracking issue
-      shell: bash
-      env:
-        GH_TOKEN: ${{ inputs.github-token }}
-        WORKFLOW_NAME: ${{ inputs.workflow-name }}
-        ISSUE_LABEL: ${{ inputs.label }}
-      run: |
-        set -euo pipefail
-
-        title="[scheduled-failure] ${WORKFLOW_NAME}"
-        run_url="${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}/actions/runs/${GITHUB_RUN_ID}"
-
-        # Failed job names for this run attempt. The alert job runs while the
-        # run as a whole is still in progress, so inspect the jobs that have
-        # already completed with a non-success conclusion.
-        failed_jobs="$(gh api \
-          "repos/${GITHUB_REPOSITORY}/actions/runs/${GITHUB_RUN_ID}/attempts/${GITHUB_RUN_ATTEMPT}/jobs" \
-          --paginate \
-          --jq '.jobs[]
-                | select(.conclusion == "failure" or .conclusion == "timed_out" or .conclusion == "cancelled")
-                | "- `\(.name)` (\(.conclusion))"')"
-        if [ -z "${failed_jobs}" ]; then
-          failed_jobs="- (failed job not recorded yet — see the run page)"
-        fi
-
-        body="$(cat <<EOF
-        Scheduled run of **${WORKFLOW_NAME}** failed.
-
-        - Run: ${run_url} (attempt ${GITHUB_RUN_ATTEMPT})
-        - Event: \`${GITHUB_EVENT_NAME}\`
-        - Ref: \`${GITHUB_REF_NAME}\` @ \`${GITHUB_SHA}\`
-
-        Failed jobs:
-        ${failed_jobs}
-        EOF
-        )"
-
-        # Dedupe by exact title among OPEN issues (a closed issue means the
-        # earlier breakage was resolved; a new failure gets a fresh issue).
-        # GitHub search strips the [] characters, so search loosely and match
-        # the exact title with jq.
-        existing="$(gh issue list --repo "${GITHUB_REPOSITORY}" --state open --limit 100 \
-          --search "\"scheduled-failure\" in:title" --json number,title \
-          | jq -r --arg title "${title}" 'map(select(.title == $title)) | (.[0].number // empty)')"
-
-        if [ -n "${existing}" ]; then
-          echo "Appending comment to existing open issue #${existing}"
-          gh issue comment "${existing}" --repo "${GITHUB_REPOSITORY}" --body "${body}"
-          exit 0
-        fi
-
-        echo "Creating new issue: ${title}"
-        if [ -n "${ISSUE_LABEL}" ]; then
-          if gh issue create --repo "${GITHUB_REPOSITORY}" \
-               --title "${title}" --body "${body}" --label "${ISSUE_LABEL}"; then
-            exit 0
-          fi
-          echo "::warning::issue creation with label '${ISSUE_LABEL}' failed (missing label?); retrying without a label"
-        fi
-        gh issue create --repo "${GITHUB_REPOSITORY}" --title "${title}" --body "${body}"

.github/actions/setup/action.yml

@@ -56,7 +56,6 @@ runs:
           libssl-dev \
           ripgrep \
           unzip \
-          zip \
           protobuf-compiler
 
     - name: Install protoc

.github/cla.yml

@@ -15,8 +15,8 @@
 enabled: true
 
 document:
-  version: v2
-  url: https://github.com/rustfs/cla/blob/main/cla/v2.md
+  version: v1
+  url: https://github.com/rustfs/cla/blob/main/cla/v1.md
 
 signing:
   mode: comment

.github/pull_request_template.md

@@ -33,4 +33,4 @@ documentation impact. Use N/A when there is no expected impact.
 
 ---
 
-Thank you for your contribution! Please ensure your PR follows the community standards ([CODE_OF_CONDUCT.md](CODE_OF_CONDUCT.md)). If this is your first contribution, review the [CLA document](https://github.com/rustfs/cla/blob/main/cla/v2.md) and sign it by commenting `I have read and agree to the CLA.` on the PR.
+Thank you for your contribution! Please ensure your PR follows the community standards ([CODE_OF_CONDUCT.md](CODE_OF_CONDUCT.md)). If this is your first contribution, review the [CLA document](https://github.com/rustfs/cla/blob/main/cla/v1.md) and sign it by commenting `I have read and agree to the CLA.` on the PR.

.github/secret_scanning.yml

@@ -1,14 +0,0 @@
-# GitHub secret scanning path exclusions.
-# Secrets detected under these paths are auto-closed as "ignored by
-# configuration" and are not blocked by push protection.
-#
-# Deliberately NOT excluded: production source files (e.g. rustfs/src/**,
-# crates/*/src/**) even when a hit sits inside a #[cfg(test)] module or a
-# doc-comment example — excluding the file would also stop scanning the
-# production code in it. Close those alerts manually as "used in tests".
-paths-ignore:
-  - "crates/e2e_test/**" # dedicated e2e test crate, test credentials throughout
-  - "**/tests/**" # Rust integration-test dirs (crates/*/tests, rustfs/tests)
-  - "**/benches/**" # benchmark harnesses
-  - ".docker/**" # local docker-compose dev configs (e.g. mqtt vm.args cookie)
-  - ".vscode/**" # local editor launch configs

.github/workflows/audit.yml

@@ -35,17 +35,15 @@ on:
       - '.github/workflows/**'
       - 'scripts/security/check_workflow_pins.sh'
   schedule:
-    - cron: '0 3 * * 0' # Weekly on Sunday 03:00 UTC (staggered after the midnight ci/build crons)
+    - cron: '0 0 * * 0' # Weekly on Sunday at midnight UTC
   workflow_dispatch:
 
 permissions:
   contents: read
 
-# Event-scoped groups: merges to main must not cancel the weekly scheduled
-# run (same rationale as ci.yml).
 concurrency:
-  group: ${{ github.workflow }}-${{ github.event_name }}-${{ github.event.pull_request.number || github.ref }}
-  cancel-in-progress: ${{ github.event_name != 'schedule' }}
+  group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
+  cancel-in-progress: true
 
 env:
   CARGO_TERM_COLOR: always
@@ -59,9 +57,32 @@ jobs:
       - name: Explain cancellation run
         run: echo "PR closed; this run only cancels older runs in the same concurrency group."
 
-  # RustSec advisory scanning is covered by the `advisories` check of
-  # cargo-deny below; a separate cargo-audit job would duplicate the same
-  # database lookup with a second ignore list to maintain.
+  security-audit:
+    name: Security Audit
+    if: github.event_name != 'pull_request' || github.event.action != 'closed'
+    runs-on: ubuntu-latest
+    timeout-minutes: 15
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
+
+      - name: Install cargo-audit
+        uses: taiki-e/install-action@bffeee26d4db9be238a4ea78d8826604ebcb594d # v2
+        with:
+          tool: cargo-audit
+
+      - name: Run security audit
+        run: |
+          cargo audit -D warnings --json | tee audit-results.json
+
+      - name: Upload audit results
+        if: always()
+        uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
+        with:
+          name: security-audit-results-${{ github.run_number }}
+          path: audit-results.json
+          retention-days: 30
+
   cargo-deny:
     name: Cargo Deny
     if: github.event_name != 'pull_request' || github.event.action != 'closed'
@@ -112,9 +133,4 @@ jobs:
         with:
           fail-on-severity: moderate
           allow-ghsas: GHSA-2f9f-gq7v-9h6m
-          # rustfs-uring is a first-party Apache-2.0 crate, now published on
-          # crates.io (backlog#1104) rather than pulled as a git dependency.
-          # Scope the allow to the exact pinned version so a version bump forces a
-          # conscious re-review of the license/provenance claim (backlog#1181).
-          allow-dependencies-licenses: pkg:cargo/rustfs-uring@0.1.0
           comment-summary-in-pr: always

.github/workflows/build.yml

@@ -21,11 +21,6 @@
 # 2. Upload binaries to OSS storage
 # 3. Trigger docker.yml to build and push images using the uploaded binaries
 #
-# Platform scope:
-# - Pushes to main (development builds) build the Linux targets only — they
-#   feed the dev download channel and Docker dev images. Tags, the weekly
-#   schedule and manual dispatch build the full platform matrix.
-#
 # Manual Parameters:
 # - build_docker: Build and push Docker images (default: true)
 # - platforms: Comma-separated platform IDs or 'all' (default: all)
@@ -51,7 +46,7 @@ on:
       - ".gitignore"
       - ".dockerignore"
   schedule:
-    - cron: "0 1 * * 0" # Weekly on Sunday 01:00 UTC (staggered after the ci.yml midnight cron)
+    - cron: "0 0 * * 0" # Weekly on Sunday at midnight UTC
   workflow_dispatch:
     inputs:
       build_docker:
@@ -92,6 +87,8 @@ jobs:
     steps:
       - name: Checkout repository
         uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
+        with:
+          fetch-depth: 0
 
       - name: Determine build strategy
         id: check
@@ -128,7 +125,8 @@ jobs:
             version="dev-${short_sha}"
             echo "🛠️  Development build detected"
           elif [[ "${{ github.event_name }}" == "schedule" ]] || \
-               [[ "${{ github.event_name }}" == "workflow_dispatch" ]]; then
+               [[ "${{ github.event_name }}" == "workflow_dispatch" ]] || \
+               [[ "${{ contains(github.event.head_commit.message, '--build') }}" == "true" ]]; then
             # Scheduled or manual build
             should_build=true
             build_type="development"
@@ -136,13 +134,11 @@ jobs:
             echo "⚡ Manual/scheduled build detected"
           fi
 
-          {
-            echo "should_build=$should_build"
-            echo "build_type=$build_type"
-            echo "version=$version"
-            echo "short_sha=$short_sha"
-            echo "is_prerelease=$is_prerelease"
-          } >> "$GITHUB_OUTPUT"
+          echo "should_build=$should_build" >> $GITHUB_OUTPUT
+          echo "build_type=$build_type" >> $GITHUB_OUTPUT
+          echo "version=$version" >> $GITHUB_OUTPUT
+          echo "short_sha=$short_sha" >> $GITHUB_OUTPUT
+          echo "is_prerelease=$is_prerelease" >> $GITHUB_OUTPUT
 
           echo "📊 Build Summary:"
           echo "  - Should build: $should_build"
@@ -154,7 +150,6 @@ jobs:
   # Build RustFS binaries
   prepare-platform-matrix:
     name: Prepare Platform Matrix
-    needs: build-check
     runs-on: ubuntu-latest
     outputs:
       matrix: ${{ steps.select.outputs.matrix }}
@@ -172,24 +167,13 @@ jobs:
             selected="all"
           fi
 
-          # Per-merge development builds only feed the dev download channel
-          # and the Docker dev images, which consume the Linux binaries; at
-          # the usual merge cadence most per-merge builds are cancelled by the
-          # next merge anyway. macOS and Windows stay covered by tag builds,
-          # the weekly scheduled build and manual dispatch.
-          if [[ "${selected}" == "all" \
-            && "${{ github.event_name }}" == "push" \
-            && "${{ needs.build-check.outputs.build_type }}" == "development" ]]; then
-            selected="linux-x86_64-musl,linux-aarch64-musl,linux-x86_64-gnu,linux-aarch64-gnu"
-            echo "Development (main push) build: restricting to Linux targets"
-          fi
-
           all='{"include":[
             {"target_id":"linux-x86_64-musl","os":"sm-standard-2","target":"x86_64-unknown-linux-musl","cross":false,"platform":"linux","rustflags":""},
             {"target_id":"linux-aarch64-musl","os":"sm-standard-2","target":"aarch64-unknown-linux-musl","cross":true,"platform":"linux","rustflags":""},
             {"target_id":"linux-x86_64-gnu","os":"sm-standard-2","target":"x86_64-unknown-linux-gnu","cross":false,"platform":"linux","rustflags":""},
             {"target_id":"linux-aarch64-gnu","os":"sm-standard-2","target":"aarch64-unknown-linux-gnu","cross":true,"platform":"linux","rustflags":""},
             {"target_id":"macos-aarch64","os":"macos-latest","target":"aarch64-apple-darwin","cross":false,"platform":"macos","rustflags":""},
+            {"target_id":"macos-x86_64","os":"macos-26-intel","target":"x86_64-apple-darwin","cross":false,"platform":"macos","rustflags":""},
             {"target_id":"windows-x86_64","os":"windows-latest","target":"x86_64-pc-windows-msvc","cross":false,"platform":"windows","rustflags":""}
           ]}'
 
@@ -221,15 +205,15 @@ jobs:
     name: Build RustFS
     needs: [ build-check, prepare-platform-matrix ]
     if: needs.build-check.outputs.should_build == 'true' && needs.prepare-platform-matrix.result == 'success'
-    runs-on: ${{ matrix.os }}
-    timeout-minutes: 150
+    runs-on: ${{ matrix.platform == 'linux' && fromJSON('["self-hosted","linux","sm-standard-2"]') || matrix.os }}
+    timeout-minutes: 90
     env:
       FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"
-      # Release binaries ship without dial9 telemetry and therefore do not need
-      # --cfg tokio_unstable. Telemetry builds opt in explicitly (see
-      # `make build-profiling`); crates/obs/build.rs fails the compile if the
-      # `dial9` feature is enabled without the flag.
-      RUSTFLAGS: "${{ matrix.rustflags }}"
+      # Always enable Tokio unstable features (required by dial9-tokio-telemetry).
+      # The RUSTFLAGS env var takes precedence over .cargo/config.toml [build] rustflags,
+      # so we must include --cfg tokio_unstable here explicitly; otherwise an empty
+      # RUSTFLAGS value would shadow the config-file flag and silently break tracing.
+      RUSTFLAGS: "--cfg tokio_unstable ${{ matrix.rustflags }}"
     strategy:
       fail-fast: false
       matrix: ${{ fromJson(needs.prepare-platform-matrix.outputs.matrix) }}
@@ -289,7 +273,6 @@ jobs:
             local console_url
             local console_sha256
             local curl_auth_args=()
-            console_tag=""
 
             if [[ "${{ matrix.platform }}" == "windows" ]]; then
               curl_bin="curl.exe"
@@ -312,7 +295,7 @@ jobs:
             "$curl_bin" "${curl_auth_args[@]}" --fail -L "$console_api" \
               -o "$console_json" --retry 3 --retry-delay 5 --max-time 300 || return 1
 
-            read -r console_tag console_url console_sha256 < <("$python_bin" - "$console_json" <<'PY'
+            read -r console_url console_sha256 < <("$python_bin" - "$console_json" <<'PY'
           import json
           import re
           import sys
@@ -320,8 +303,6 @@ jobs:
           with open(sys.argv[1], encoding="utf-8") as handle:
               release = json.load(handle)
 
-          tag = release.get("tag_name", "") or "unknown"
-
           for asset in release.get("assets", []):
               name = asset.get("name", "")
               digest = asset.get("digest", "")
@@ -330,7 +311,7 @@ jobs:
                   sha256 = digest.split(":", 1)[1]
                   if not re.fullmatch(r"[0-9a-fA-F]{64}", sha256):
                       raise SystemExit(f"console zip asset has invalid sha256 digest: {sha256}")
-                  sys.stdout.buffer.write(f"{tag} {url} {sha256}\n".encode("utf-8"))
+                  sys.stdout.buffer.write(f"{url} {sha256}\n".encode("utf-8"))
                   break
           else:
               raise SystemExit("no console zip asset with sha256 digest found")
@@ -342,8 +323,6 @@ jobs:
               return 1
             fi
 
-            echo "Console release: ${console_tag}"
-            echo "Downloading console asset: ${console_url}"
             "$curl_bin" --fail -L "$console_url" -o console.zip --retry 3 --retry-delay 5 --max-time 300 || return 1
             verify_sha256 "$console_sha256" console.zip || return 2
             unzip -o console.zip -d ./rustfs/static || return 2
@@ -356,19 +335,12 @@ jobs:
             rm -f console.zip console-release.json
             if [[ "$status" -eq 2 ]]; then
               echo "Console asset integrity verification failed" >&2
+              exit 1
             fi
-            echo "Failed to download verified console assets" >&2
-            exit 1
+            echo "Warning: Failed to download verified console assets, continuing without them"
+            echo "// Static assets not available" > ./rustfs/static/empty.txt
           fi
 
-          if [[ ! -s ./rustfs/static/index.html ]]; then
-            echo "Console asset archive is missing static/index.html" >&2
-            exit 1
-          fi
-
-          asset_count=$(find ./rustfs/static -type f | wc -l | tr -d '[:space:]')
-          echo "Console assets ready: version=${console_tag:-unknown}, ${asset_count} files extracted to ./rustfs/static"
-
       - name: Build RustFS
         shell: bash
         run: |
@@ -450,11 +422,11 @@ jobs:
             # Release/Prerelease build: rustfs-${platform}-${arch}-${variant}-v${version}.zip
             PACKAGE_NAME="${PACKAGE_BASENAME}-v${PACKAGE_VERSION}"
           fi
-          
+
           # Create zip packages for all platforms
           # Ensure zip is available
           if ! command -v zip &> /dev/null; then
-            if [[ "${{ matrix.os }}" == "ubuntu-latest" || "${{ matrix.platform }}" == "linux" ]]; then
+            if [[ "${{ matrix.os }}" == "ubuntu-latest" ]]; then
               sudo apt-get update && sudo apt-get install -y zip
             fi
           fi
@@ -506,7 +478,7 @@ jobs:
             if [[ "${{ matrix.platform }}" == "windows" ]]; then
               dir
             else
-              ls -lh "${PACKAGE_NAME}.zip"
+              ls -lh ${PACKAGE_NAME}.zip
             fi
           else
             echo "❌ Failed to create package: ${PACKAGE_NAME}.zip"
@@ -555,13 +527,11 @@ jobs:
             fi
           fi
 
-          {
-            echo "package_name=${PACKAGE_NAME}"
-            echo "package_file=${PACKAGE_NAME}.zip"
-            echo "latest_files=${LATEST_FILES}"
-            echo "build_type=${BUILD_TYPE}"
-            echo "version=${VERSION}"
-          } >> "$GITHUB_OUTPUT"
+          echo "package_name=${PACKAGE_NAME}" >> $GITHUB_OUTPUT
+          echo "package_file=${PACKAGE_NAME}.zip" >> $GITHUB_OUTPUT
+          echo "latest_files=${LATEST_FILES}" >> $GITHUB_OUTPUT
+          echo "build_type=${BUILD_TYPE}" >> $GITHUB_OUTPUT
+          echo "version=${VERSION}" >> $GITHUB_OUTPUT
 
           echo "📦 Package created: ${PACKAGE_NAME}.zip"
           if [[ -n "$LATEST_FILES" ]]; then
@@ -570,60 +540,6 @@ jobs:
           echo "🔧 Build type: ${BUILD_TYPE}"
           echo "📊 Version: ${VERSION}"
 
-      - name: Verify packaged console
-        if: matrix.target == 'x86_64-unknown-linux-gnu'
-        shell: bash
-        run: |
-          set -euo pipefail
-
-          find_free_port() {
-            python3 - <<'PY'
-          import socket
-
-          with socket.socket() as sock:
-              sock.bind(("127.0.0.1", 0))
-              print(sock.getsockname()[1])
-          PY
-          }
-
-          package_dir="$(mktemp -d)"
-          data_dir="$(mktemp -d)"
-          log_file="${RUNNER_TEMP}/rustfs-console-smoke.log"
-          server_pid=""
-          trap '
-            if [[ -n "${server_pid:-}" ]]; then
-              kill "$server_pid" 2>/dev/null || true
-              wait "$server_pid" 2>/dev/null || true
-            fi
-            rm -rf "$package_dir" "$data_dir"
-          ' EXIT
-
-          unzip -q "${{ steps.package.outputs.package_file }}" -d "$package_dir"
-          api_port="$(find_free_port)"
-          console_port="$(find_free_port)"
-          console_url="http://127.0.0.1:${console_port}/rustfs/console/"
-
-          "$package_dir/rustfs" server \
-            --address "127.0.0.1:${api_port}" \
-            --console-enable \
-            --console-address "127.0.0.1:${console_port}" \
-            --access-key console-smoke \
-            --secret-key console-smoke-secret \
-            "$data_dir" >"$log_file" 2>&1 &
-          server_pid=$!
-
-          for _ in {1..40}; do
-            response="$(curl --silent --output /dev/null --write-out '%{http_code} %{content_type}' "$console_url" || true)"
-            if [[ "$response" == 200\ text/html* ]]; then
-              exit 0
-            fi
-            sleep 0.25
-          done
-
-          echo "Console endpoint did not return 200 text/html: $response" >&2
-          cat "$log_file"
-          exit 1
-
       - name: Upload to GitHub artifacts
         uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
         with:
@@ -648,19 +564,9 @@ jobs:
             exit 0
           fi
 
-          # The self-hosted Linux runners do not ship the aws CLI (GitHub-hosted
-          # images did). Install it on demand so R2 uploads survive a fresh runner
-          # instead of hard-failing here.
           if ! command -v aws >/dev/null 2>&1; then
-            echo "aws CLI not found on runner; installing..."
-            if command -v apt-get >/dev/null 2>&1; then
-              sudo apt-get update && sudo apt-get install -y awscli
-            elif command -v brew >/dev/null 2>&1; then
-              brew install awscli
-            else
-              echo "❌ aws CLI missing and no apt-get/brew to install it; cannot upload to R2"
-              exit 1
-            fi
+            echo "❌ aws CLI not found on runner; cannot upload to R2"
+            exit 1
           fi
 
           export AWS_ACCESS_KEY_ID="$R2_ACCESS_KEY_ID"
@@ -822,8 +728,8 @@ jobs:
             RELEASE_URL=$(gh release view "$TAG" --json url --jq '.url')
           fi
 
-          echo "release_id=$RELEASE_ID" >> "$GITHUB_OUTPUT"
-          echo "release_url=$RELEASE_URL" >> "$GITHUB_OUTPUT"
+          echo "release_id=$RELEASE_ID" >> $GITHUB_OUTPUT
+          echo "release_url=$RELEASE_URL" >> $GITHUB_OUTPUT
           echo "Created release: $RELEASE_URL"
 
   # Prepare and upload release assets
@@ -872,11 +778,17 @@ jobs:
           cd ./release-assets
 
           # Generate checksums for all files (including latest versions)
-          if compgen -G "*.zip" >/dev/null; then
-            sha256sum -- *.zip > SHA256SUMS
-            sha512sum -- *.zip > SHA512SUMS
+          if ls *.zip >/dev/null 2>&1; then
+            sha256sum *.zip > SHA256SUMS
+            sha512sum *.zip > SHA512SUMS
           fi
 
+          # Create signature placeholder files
+          for file in *.zip; do
+            echo "# Signature for $file" > "${file}.asc"
+            echo "# GPG signature will be added in future versions" >> "${file}.asc"
+          done
+
           cd ..
           python3 scripts/security/generate_release_supply_chain_assets.py \
             --asset-dir ./release-assets \
@@ -914,10 +826,7 @@ jobs:
 
           echo "✅ All assets uploaded successfully"
 
-  # Update latest.json for every release tag (stable and prerelease): the
-  # project currently ships prerelease tags only, so gating this to stable
-  # left the version pointer permanently stale. release_type records whether
-  # the pointed-to version is a prerelease.
+  # Update latest.json for stable releases only
   update-latest-version:
     name: Update Latest Version
     needs: [ build-check, upload-release-assets ]
@@ -940,12 +849,6 @@ jobs:
           VERSION="${{ needs.build-check.outputs.version }}"
           TAG="${{ needs.build-check.outputs.version }}"
 
-          if [[ "${{ needs.build-check.outputs.build_type }}" == "prerelease" ]]; then
-            RELEASE_TYPE="prerelease"
-          else
-            RELEASE_TYPE="stable"
-          fi
-
           # Install ossutil
           OSSUTIL_VERSION="2.1.1"
           OSSUTIL_ZIP="ossutil-${OSSUTIL_VERSION}-linux-amd64.zip"
@@ -966,7 +869,7 @@ jobs:
             "version": "${VERSION}",
             "tag": "${TAG}",
             "release_date": "$(date -u +%Y-%m-%dT%H:%M:%SZ)",
-            "release_type": "${RELEASE_TYPE}",
+            "release_type": "stable",
             "download_url": "https://github.com/${{ github.repository }}/releases/tag/${TAG}"
           }
           EOF
@@ -974,7 +877,7 @@ jobs:
           # Upload to OSS
           "$OSSUTIL_BIN" cp latest.json oss://rustfs-version/latest.json --force
 
-          echo "✅ Updated latest.json for ${RELEASE_TYPE} release $VERSION"
+          echo "✅ Updated latest.json for stable release $VERSION"
 
   # Publish release (remove draft status)
   publish-release:

.github/workflows/ci-docs-only.yml

@@ -1,70 +0,0 @@
-# Copyright 2026 RustFS Team
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-# Companion to ci.yml for the required "Test and Lint" status check.
-#
-# ci.yml skips docs-only pull requests via paths-ignore, but the branch
-# ruleset requires a check named "Test and Lint" — without this workflow a
-# docs-only PR would wait on that check forever. This workflow triggers on
-# exactly the paths ci.yml ignores and reports an instant success under the
-# same job name. Mixed PRs trigger both workflows and the real check still
-# gates: a required check with any failing run blocks the merge.
-# https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/defining-the-mergeability-of-pull-requests/troubleshooting-required-status-checks#handling-skipped-but-required-checks
-#
-# Keep the paths list below in sync with the pull_request paths-ignore list
-# in ci.yml.
-
-name: Continuous Integration (docs only)
-
-on:
-  pull_request:
-    types: [ opened, synchronize, reopened ]
-    branches: [ main ]
-    paths:
-      - "**.md"
-      - "docs/**"
-      - "deploy/**"
-      - "scripts/dev_*.sh"
-      - "scripts/probe.sh"
-      - "LICENSE*"
-      - ".gitignore"
-      - ".dockerignore"
-      - "README*"
-      - "**/*.png"
-      - "**/*.jpg"
-      - "**/*.svg"
-      - ".github/workflows/build.yml"
-      - ".github/workflows/docker.yml"
-      - ".github/workflows/audit.yml"
-
-permissions:
-  contents: read
-
-jobs:
-  test-and-lint:
-    name: Test and Lint
-    runs-on: ubuntu-latest
-    steps:
-      - name: Checkout repository
-        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
-
-      # Docs-only PRs skip the full code CI, but they are exactly where a
-      # planning-type document could be slipped in (git add -f bypasses
-      # .gitignore). Run the guard here so the required "Test and Lint" check
-      # stays meaningful for docs-only changes.
-      - name: Check no planning docs committed
-        run: ./scripts/check_no_planning_docs.sh
-
-      - name: Satisfy required check for docs-only changes
-        run: echo "Docs-only change — code CI is skipped by paths-ignore; planning-docs guard passed, reporting success for the required 'Test and Lint' check."

.github/workflows/ci.yml

@@ -33,11 +33,10 @@ on:
       - ".github/workflows/build.yml"
       - ".github/workflows/docker.yml"
       - ".github/workflows/audit.yml"
+      - ".github/workflows/performance.yml"
   pull_request:
     types: [ opened, synchronize, reopened, closed ]
     branches: [ main ]
-    # Keep this list in sync with the `paths` list in ci-docs-only.yml, which
-    # reports the required "Test and Lint" check for PRs skipped here.
     paths-ignore:
       - "**.md"
       - "docs/**"
@@ -54,6 +53,7 @@ on:
       - ".github/workflows/build.yml"
       - ".github/workflows/docker.yml"
       - ".github/workflows/audit.yml"
+      - ".github/workflows/performance.yml"
   merge_group:
     types: [ checks_requested ]
   schedule:
@@ -63,13 +63,9 @@ on:
 permissions:
   contents: read
 
-# Concurrency groups are scoped per event so different triggers never cancel
-# each other: PR pushes cancel the previous run of that PR, main pushes keep
-# latest-wins semantics among themselves, and scheduled runs always complete
-# (a shared group used to let every merge kill the weekly scheduled run).
 concurrency:
-  group: ${{ github.workflow }}-${{ github.event_name }}-${{ github.event.pull_request.number || github.ref }}
-  cancel-in-progress: ${{ github.event_name != 'schedule' }}
+  group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
+  cancel-in-progress: true
 
 env:
   CARGO_TERM_COLOR: always
@@ -85,12 +81,33 @@ jobs:
       - name: Explain cancellation run
         run: echo "PR closed; this run only cancels older runs in the same concurrency group."
 
+  skip-check:
+    name: Skip Duplicate Actions
+    if: github.event_name != 'pull_request' || github.event.action != 'closed'
+    permissions:
+      actions: write
+      contents: read
+    runs-on: ubuntu-latest
+    outputs:
+      should_skip: ${{ steps.skip_check.outputs.should_skip }}
+    steps:
+      - name: Skip duplicate actions
+        id: skip_check
+        uses: fkirc/skip-duplicate-actions@f75f66ce1886f00957d99748a42c724f4330bdcf # v5
+        with:
+          concurrent_skipping: "same_content_newer"
+          cancel_others: true
+          paths_ignore: '["*.md", "docs/**", "deploy/**"]'
+          do_not_skip: '["workflow_dispatch", "schedule", "merge_group", "release", "push"]'
+
   typos:
     name: Typos
-    if: github.event_name != 'pull_request' || github.event.action != 'closed'
+    needs: skip-check
+    if: needs.skip-check.outputs.should_skip != 'true'
     runs-on: ubuntu-latest
     steps:
       - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
+      - uses: dtolnay/rust-toolchain@29eef336d9b2848a0b548edc03f92a220660cdb8 # stable
       - name: Typos check with custom config file
         uses: crate-ci/typos@37bb98842b0d8c4ffebdb75301a13db0267cef89 # master
 
@@ -98,7 +115,8 @@ jobs:
   # ~1 minute instead of waiting for the full test job.
   quick-checks:
     name: Quick Checks
-    if: github.event_name != 'pull_request' || github.event.action != 'closed'
+    needs: skip-check
+    if: needs.skip-check.outputs.should_skip != 'true'
     runs-on: ubuntu-latest
     timeout-minutes: 10
     steps:
@@ -128,18 +146,10 @@ jobs:
       - name: Check tokio io-uring feature guard
         run: ./scripts/check_no_tokio_io_uring.sh
 
-      - name: Check extension schema boundaries
-        run: ./scripts/check_extension_schema_boundaries.sh
-
-      - name: Check body-cache whitelist guard
-        run: ./scripts/check_body_cache_whitelist.sh
-
-      - name: Check no planning docs committed
-        run: ./scripts/check_no_planning_docs.sh
-
   test-and-lint:
     name: Test and Lint
-    if: github.event_name != 'pull_request' || github.event.action != 'closed'
+    needs: skip-check
+    if: needs.skip-check.outputs.should_skip != 'true'
     runs-on: sm-standard-4
     timeout-minutes: 60
     env:
@@ -156,99 +166,26 @@ jobs:
           github-token: ${{ secrets.GITHUB_TOKEN }}
           cache-save-if: ${{ github.ref == 'refs/heads/main' }}
 
-      # Clippy runs before the test pass: lint failures are the most common
-      # CI-only breakage and should surface in minutes, not after 20+ minutes
-      # of tests.
-      - name: Run clippy lints
-        run: cargo clippy --all-targets -- -D warnings
-
       - name: Run tests
         run: |
-          cargo nextest run --profile ci --all --exclude e2e_test
+          cargo nextest run --all --exclude e2e_test
           cargo test --all --doc
 
-      # rustfs/backlog#1289: fail if a seed rule's log anchor no longer exists
-      # verbatim in the source tree (log message drifted without updating the
-      # rule). Placed here where the workspace — including the la-dump-anchors
-      # bin — is already built by the clippy/test steps above.
-      - name: Check log-analyzer rule anchors
-        run: ./scripts/check_log_analyzer_rules.sh
-
-      - name: Upload test junit report
-        if: always()
-        uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
-        with:
-          name: junit-test-and-lint-${{ github.run_number }}
-          path: target/nextest/ci/junit.xml
-          retention-days: 3
-          if-no-files-found: ignore
-
       # Explicit gate for migration-critical suites. These tests already ran in
       # the full nextest pass above; a single filtered nextest invocation keeps
       # the named gate without rebuilding or re-running them one package at a time.
-      #
-      # The gate selects tests by name substring (data_movement / rebalance /
-      # decommission / source_cleanup / delete_marker), so renames can silently
-      # thin it. The script owns the filter expression and first verifies the
-      # selected-test count against the committed floor in
-      # .config/migration-gate-floor.txt before running the gate; renames or
-      # removals must update that file consciously (see the script header).
-      # Kept on the default profile (no --profile ci): a second --profile ci run
-      # would clobber target/nextest/ci/junit.xml, and none of these tests are
-      # quarantined so they gain nothing from the ci profile's retry overrides.
       - name: Run rebalance/decommission migration proofs
-        run: ./scripts/check_migration_gate_count.sh
-
-  # Dedicated serial lane for the ILM / lifecycle integration tests. These tests
-  # drive the object layer through process-global singletons (the GLOBAL_ENV
-  # ECStore, the global tier-config manager, background-expiry workers) and bind
-  # fixed ports (9002 for the scanner suite, 9003 for the app suite), so they are
-  # marked #[ignore] to keep them out of the default parallel `cargo nextest run
-  # --all` pass. Run them here explicitly with `--run-ignored ignored-only` and
-  # `-j1` so no two of them share global state or race for a port at once.
-  # serial_test's #[serial] does NOT serialize across nextest's process-per-test
-  # boundary (see .config/nextest.toml); `-j1` is what actually serializes them.
-  # See rustfs/backlog#1148 (ilm-1) and #1155.
-  test-ilm-integration-serial:
-    name: ILM Integration (serial)
-    if: github.event_name != 'pull_request' || github.event.action != 'closed'
-    runs-on: sm-standard-4
-    timeout-minutes: 45
-    env:
-      FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"
-    steps:
-      - name: Checkout repository
-        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
-
-      - name: Setup Rust environment
-        uses: ./.github/actions/setup
-        with:
-          rust-version: stable
-          cache-shared-key: ci-ilm-serial
-          github-token: ${{ secrets.GITHUB_TOKEN }}
-          cache-save-if: ${{ github.ref == 'refs/heads/main' }}
-
-      # test_transition_and_restore_flows was re-enabled by rustfs/backlog#1303:
-      # its "missing xl.meta on disk2" was a test-util bug (open_disk hardcoded
-      # disk_index 0), not an EC metadata-distribution issue.
-      # restore_object_usecase_reports_ongoing_conflict_and_completion was
-      # re-enabled by backlog#1304 (restore accepts serialize on a short CAS
-      # guard; the copy-back no longer holds the #4877 whole-copy-back lock,
-      # so the mid-restore ongoing read and fast 409 rejection it asserts are
-      # the implemented contract). The remaining exclusions each hit a
-      # DIFFERENT, independent issue (all tracked under rustfs/backlog#1148;
-      # they keep #[ignore] with a backlog reference):
-      #   - test_noncurrent_{expiry,transition}_still_works_after_immediate_compensation_transition:
-      #     noncurrent transition/expiry after an immediate compensation transition.
-      - name: Run ignored ILM integration tests serially
         run: |
-          cargo nextest run -j1 --run-ignored ignored-only \
-            -p rustfs-scanner -p rustfs \
-            -E '(binary(lifecycle_integration_test) or (package(rustfs) and test(lifecycle_transition_api_test))) and not (test(test_noncurrent_expiry_still_works_after_immediate_compensation_transition) or test(test_noncurrent_transition_still_works_after_immediate_compensation_transition))'
+          cargo nextest run -p rustfs-ecstore --lib \
+            -E 'test(data_movement) or test(rebalance) or test(decommission) or test(source_cleanup) or test(delete_marker)'
+
+      - name: Run clippy lints
+        run: cargo clippy --all-targets -- -D warnings
 
   test-and-lint-rio-v2:
     name: Test and Lint (rio-v2)
-    if: github.event_name != 'pull_request' || github.event.action != 'closed'
+    needs: skip-check
+    if: needs.skip-check.outputs.should_skip != 'true'
     runs-on: sm-standard-4
     timeout-minutes: 60
     env:
@@ -265,17 +202,18 @@ jobs:
           github-token: ${{ secrets.GITHUB_TOKEN }}
           cache-save-if: ${{ github.ref == 'refs/heads/main' }}
 
-      - name: Run rio-v2 clippy lints
-        run: cargo clippy -p rustfs -p rustfs-ecstore --all-targets --features rio-v2 -- -D warnings
-
       - name: Run rio-v2 feature tests
         run: |
           cargo nextest run -p rustfs -p rustfs-ecstore --features rio-v2
           cargo test -p rustfs --doc --features rio-v2
 
+      - name: Run rio-v2 clippy lints
+        run: cargo clippy -p rustfs -p rustfs-ecstore --all-targets --features rio-v2 -- -D warnings
+
   test-and-lint-protocols:
     name: "Test and Lint (${{ matrix.features.name }})"
-    if: github.event_name != 'pull_request' || github.event.action != 'closed'
+    needs: skip-check
+    if: needs.skip-check.outputs.should_skip != 'true'
     runs-on: sm-standard-4
     timeout-minutes: 60
     strategy:
@@ -300,17 +238,18 @@ jobs:
           github-token: ${{ secrets.GITHUB_TOKEN }}
           cache-save-if: ${{ github.ref == 'refs/heads/main' }}
 
-      - name: Run clippy with ${{ matrix.features.name }}
-        run: |
-          cargo clippy -p rustfs -p rustfs-protocols --all-targets ${{ matrix.features.flags }} -- -D warnings
-
       - name: Run tests with ${{ matrix.features.name }}
         run: |
           cargo nextest run -p rustfs -p rustfs-protocols ${{ matrix.features.flags }}
 
+      - name: Run clippy with ${{ matrix.features.name }}
+        run: |
+          cargo clippy -p rustfs -p rustfs-protocols --all-targets ${{ matrix.features.flags }} -- -D warnings
+
   build-rustfs-debug-binary:
     name: Build RustFS Debug Binary
-    if: github.event_name != 'pull_request' || github.event.action != 'closed'
+    needs: skip-check
+    if: needs.skip-check.outputs.should_skip != 'true'
     runs-on: sm-standard-4
     timeout-minutes: 30
     env:
@@ -340,7 +279,8 @@ jobs:
 
   build-rustfs-debug-binary-rio-v2:
     name: Build RustFS Debug Binary (rio-v2)
-    if: github.event_name != 'pull_request' || github.event.action != 'closed'
+    needs: skip-check
+    if: needs.skip-check.outputs.should_skip != 'true'
     runs-on: sm-standard-4
     timeout-minutes: 30
     env:
@@ -368,67 +308,17 @@ jobs:
           if-no-files-found: error
           retention-days: 1
 
-  uring-integration:
-    name: io_uring Integration (real)
-    # GitHub-hosted ubuntu-latest runs a recent kernel with io_uring and, unlike
-    # a container, applies no seccomp filter that would block io_uring_setup — so
-    # the probe succeeds and the tests exercise the real UringBackend/FdCache/
-    # latch paths instead of the StdBackend fallback (rustfs/backlog#1179). The
-    # self-hosted sm-standard runners cannot guarantee this.
-    runs-on: ubuntu-latest
-    timeout-minutes: 30
-    steps:
-      - name: Checkout repository
-        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
-
-      - name: Setup Rust environment
-        uses: ./.github/actions/setup
-        with:
-          rust-version: stable
-          cache-shared-key: ci-uring
-          github-token: ${{ secrets.GITHUB_TOKEN }}
-          cache-save-if: ${{ github.ref == 'refs/heads/main' }}
-
-      - name: Install build dependencies
-        run: sudo apt-get update && sudo apt-get install -y protobuf-compiler
-
-      # ext4 supports O_DIRECT; the runner's default TMPDIR may sit on tmpfs or
-      # overlayfs, where open(O_DIRECT) returns EINVAL/EOPNOTSUPP and the native
-      # read_at_direct path silently latches off to the aligned StdBackend
-      # fallback. A dedicated ext4 loopback guarantees the native O_DIRECT read
-      # path is actually exercised, not just the fallback (rustfs/backlog#1220).
-      - name: Prepare ext4 loopback for O_DIRECT coverage
-        run: |
-          set -euo pipefail
-          dd if=/dev/zero of=/tmp/rustfs-odirect.img bs=1M count=1024
-          mkfs.ext4 -q -F /tmp/rustfs-odirect.img
-          sudo mkdir -p /mnt/rustfs-odirect
-          sudo mount -o loop /tmp/rustfs-odirect.img /mnt/rustfs-odirect
-          sudo chmod 1777 /mnt/rustfs-odirect
-          mount | grep rustfs-odirect
-
-      # RUSTFS_URING_TESTS_MUST_RUN=1 makes the io_uring tests fail instead of
-      # silently skipping if io_uring is unavailable, so this leg can never pass
-      # vacuously. RUSTFS_IO_URING_READ_ENABLE=true selects the UringBackend.
-      # TMPDIR points every tempfile-based test fixture at the ext4 loopback so
-      # eligible reads keep O_DIRECT and the native path is covered end to end.
-      - name: Run ecstore io_uring integration tests on real io_uring
-        env:
-          RUSTFS_IO_URING_READ_ENABLE: "true"
-          RUSTFS_URING_TESTS_MUST_RUN: "1"
-          TMPDIR: /mnt/rustfs-odirect
-        run: cargo test -p rustfs-ecstore uring_ -- --test-threads=1 --nocapture
-
   e2e-tests:
     name: End-to-End Tests
-    needs: [ build-rustfs-debug-binary ]
+    needs: [ skip-check, build-rustfs-debug-binary ]
+    if: needs.skip-check.outputs.should_skip != 'true'
     runs-on: sm-standard-2
     timeout-minutes: 30
     steps:
       - name: Checkout repository
         uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
 
-      # Full setup with dependency caching: the smoke-suite step below
+      # Full setup with dependency caching: the migration-proof step below
       # compiles the e2e_test crate, which pulls in most of the workspace.
       # Without a restored cache this was a cold multi-GB build on every run.
       - name: Setup Rust environment
@@ -450,23 +340,8 @@ jobs:
       - name: Make binary executable
         run: chmod +x ./target/debug/rustfs
 
-      # Guard the security negative-auth smoke subset (backlog#1151 sec-5)
-      # against a rename or deletion silently dropping it out of the e2e-smoke
-      # filter. The script lists what the profile selects and fails if the count
-      # of security auth-rejection tests falls below the committed floor in
-      # .config/security-smoke-floor.txt (infra-12 count-floor mechanism). Run
-      # before the smoke suite so a thinned gate fails fast; the `nextest list`
-      # here compiles the e2e_test binaries the run below reuses.
-      - name: Check security smoke subset count floor
-        run: ./scripts/check_security_smoke_count.sh check
-
-      # PR smoke subset of the in-repo e2e suite (backlog#1149 ci-4). The
-      # profile.e2e-smoke default-filter in .config/nextest.toml is the single
-      # wiring mechanism for e2e tests in CI — extend that filter instead of
-      # adding new e2e jobs here. Each test spawns its own rustfs server on a
-      # random port and reuses the downloaded debug binary above.
-      - name: Run e2e smoke suite
-        run: cargo nextest run --profile e2e-smoke -p e2e_test
+      - name: Run delete-marker migration proof
+        run: cargo test -p e2e_test delete_marker_migration_semantics -- --nocapture --test-threads=1
 
       - name: Install s3s-e2e test tool
         uses: taiki-e/cache-cargo-install-action@7447f04c51f2ba27ca35e7f1e28fab848c5b3ba7 # v2
@@ -493,62 +368,10 @@ jobs:
           path: ${{ runner.temp }}/rustfs-e2e-*/rustfs.log
           retention-days: 3
 
-  e2e-full:
-    name: End-to-End Tests (full merge gate)
-    # Merge gate only (backlog#1149 ci-5): the never-automated user-visible
-    # suites — KMS, object_lock, multipart_auth, quota, checksum, encryption,
-    # security-boundary, ... — via the e2e-full nextest profile. Too heavy for
-    # every PR, so it is gated to main pushes, the merge queue, and manual
-    # dispatch. protocols / the 6 cluster suites / replication / #[ignore] are
-    # owned by other lanes (see .config/nextest.toml profile.e2e-full).
-    if: >-
-      github.event_name == 'workflow_dispatch' ||
-      github.event_name == 'merge_group' ||
-      (github.event_name == 'push' && github.ref == 'refs/heads/main')
-    needs: [ build-rustfs-debug-binary ]
-    runs-on: sm-standard-2
-    timeout-minutes: 55
-    steps:
-      - name: Checkout repository
-        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
-
-      - name: Setup Rust environment
-        uses: ./.github/actions/setup
-        with:
-          rust-version: stable
-          cache-shared-key: ci-e2e
-          github-token: ${{ secrets.GITHUB_TOKEN }}
-          cache-save-if: ${{ github.ref == 'refs/heads/main' }}
-
-      # Download after the cache restore so the freshly built binary from the
-      # build job always wins over anything restored into target/debug.
-      - name: Download debug binary
-        uses: actions/download-artifact@37930b1c2abaa49bbe596cd826c3c89aef350131 # v7
-        with:
-          name: rustfs-debug-binary
-          path: target/debug
-
-      - name: Make binary executable
-        run: chmod +x ./target/debug/rustfs
-
-      # Full single-node e2e lane (backlog#1149 ci-5). The e2e-full
-      # default-filter in .config/nextest.toml is the single wiring mechanism —
-      # extend that filter, never add ad-hoc e2e jobs here. Reuses the downloaded
-      # debug binary; each test spawns its own rustfs server on a random port.
-      - name: Run e2e full suite
-        run: cargo nextest run --profile e2e-full -p e2e_test
-
-      - name: Upload junit
-        if: always()
-        uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
-        with:
-          name: e2e-full-junit-${{ github.run_number }}
-          path: target/nextest/e2e-full/junit.xml
-          retention-days: 7
-
   e2e-tests-rio-v2:
     name: End-to-End Tests (rio-v2)
-    needs: [ build-rustfs-debug-binary-rio-v2 ]
+    needs: [ skip-check, build-rustfs-debug-binary-rio-v2 ]
+    if: needs.skip-check.outputs.should_skip != 'true'
     runs-on: sm-standard-2
     timeout-minutes: 30
     steps:
@@ -572,14 +395,6 @@ jobs:
       - name: Setup Rust toolchain for s3s-e2e installation
         uses: dtolnay/rust-toolchain@29eef336d9b2848a0b548edc03f92a220660cdb8 # stable
 
-      # The sm-standard-* custom runner images (introduced in #4884) ship no C
-      # toolchain, unlike GitHub-hosted ubuntu-latest. Installing s3s-e2e below
-      # compiles it from source on a cache miss, and build scripts need cc.
-      - name: Install build tools for s3s-e2e compilation
-        run: |
-          sudo apt-get update
-          sudo apt-get install -y build-essential cmake pkg-config libssl-dev
-
       - name: Install s3s-e2e test tool
         uses: taiki-e/cache-cargo-install-action@7447f04c51f2ba27ca35e7f1e28fab848c5b3ba7 # v2
         with:
@@ -602,7 +417,8 @@ jobs:
 
   s3-implemented-tests:
     name: S3 Implemented Tests
-    needs: [ build-rustfs-debug-binary, e2e-tests ]
+    needs: [ skip-check, build-rustfs-debug-binary, e2e-tests ]
+    if: needs.skip-check.outputs.should_skip != 'true'
     runs-on: sm-standard-4
     timeout-minutes: 60
     steps:
@@ -641,77 +457,3 @@ jobs:
           path: artifacts/s3tests-single/**
           if-no-files-found: ignore
           retention-days: 3
-
-  # Dedicated lifecycle behavior lane (backlog#1148 ilm-10, master plan #1155).
-  #
-  # These ceph/s3-tests cases assert that objects/versions/uploads are actually
-  # removed by the background scanner and the stale-multipart cleanup loop, so
-  # they need a debug-accelerated day plus an enabled scanner. That configuration
-  # cannot be applied to the s3-implemented-tests lane above because a global
-  # RUSTFS_ILM_DEBUG_DAY_SECS also shrinks the x-amz-expiration header those tests
-  # assert on (test_lifecycle_expiration_header_*). Hence a separate server here.
-  #
-  # RUSTFS_ILM_DEBUG_DAY_SECS MUST stay equal to the s3-tests lc_debug_interval
-  # default (10s). test_lifecycle_expiration polls the Days=1 rule with a
-  # 4*lc_interval window while the Days=5 rule fires at 5*debug_day; observing
-  # the intermediate 4-object plateau requires 4*lc_interval < 5*debug_day, i.e.
-  # debug_day > 8. A smaller value lets the Days=5 rule fire inside the Days=1
-  # poll window so the count collapses straight to 2 and the test fails (#4764).
-  #
-  # RUSTFS_DATA_USAGE_UPDATE_DIR_CYCLES=1 is also required. A compacted directory
-  # is only re-descended (and its objects re-evaluated for ILM) once every
-  # DATA_USAGE_UPDATE_DIR_CYCLES scanner cycles (default 16). At the accelerated
-  # RUSTFS_SCANNER_CYCLE=2 that is ~32s between evaluations, so a Days=1 object
-  # that becomes due at debug_day (10s) is not actually expired until the next
-  # ~32s boundary (~42s), landing just past the 4*lc_interval (40s) poll window
-  # and making the count stall at 6 (#4767). Forcing every-cycle re-descent
-  # evaluates ILM within ~2s of the due time, well inside the poll window.
-  s3-lifecycle-behavior-tests:
-    name: S3 Lifecycle Behavior Tests
-    needs: [ build-rustfs-debug-binary ]
-    runs-on: sm-standard-4
-    timeout-minutes: 30
-    steps:
-      - name: Checkout repository
-        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
-
-      - name: Download debug binary
-        uses: actions/download-artifact@37930b1c2abaa49bbe596cd826c3c89aef350131 # v7
-        with:
-          name: rustfs-debug-binary
-          path: target/debug
-
-      - name: Make binary executable
-        run: chmod +x ./target/debug/rustfs
-
-      - name: Run lifecycle behavior s3-tests
-        run: |
-          RUN_ROOT="${RUNNER_TEMP}/rustfs-s3tests-${GITHUB_RUN_ID}-${GITHUB_RUN_ATTEMPT}-${GITHUB_JOB}"
-          mkdir -p "${RUN_ROOT}"
-          S3_PORT="$(python3 -c 'import socket; s=socket.socket(); s.bind(("127.0.0.1", 0)); print(s.getsockname()[1]); s.close()')"
-          DEPLOY_MODE=binary \
-          RUSTFS_BINARY=./target/debug/rustfs \
-          TEST_MODE=single \
-          MAXFAIL=0 \
-          XDIST=0 \
-          IMPLEMENTED_TESTS_FILE=scripts/s3-tests/lifecycle_behavior_tests.txt \
-          RUSTFS_ILM_DEBUG_DAY_SECS=10 \
-          RUSTFS_ILM_PROCESS_TIME=1 \
-          RUSTFS_SCANNER_ENABLED=true \
-          RUSTFS_SCANNER_CYCLE=2 \
-          RUSTFS_DATA_USAGE_UPDATE_DIR_CYCLES=1 \
-          RUSTFS_SCANNER_START_DELAY_SECS=0 \
-          RUSTFS_API_STALE_UPLOADS_CLEANUP_INTERVAL=2s \
-          S3_PORT="${S3_PORT}" \
-          DATA_ROOT="${RUN_ROOT}" \
-          S3TESTS_CONF=artifacts/s3tests-single/s3tests.conf \
-          ./scripts/s3-tests/run.sh
-
-      - name: Upload s3 test artifacts
-        if: always()
-        uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
-        with:
-          name: s3tests-lifecycle-behavior-${{ github.run_number }}
-          path: artifacts/s3tests-single/**
-          if-no-files-found: ignore
-          retention-days: 3

.github/workflows/coverage.yml

@@ -1,122 +0,0 @@
-# Copyright 2024 RustFS Team
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-# Weekly workspace line-coverage baseline (backlog#1153 infra-5).
-#
-# NON-BLOCKING by design: this workflow only runs on schedule and manual
-# dispatch, so it never attaches a status to a PR and must never be made a
-# required check. It exists to give coverage a visible baseline and trend
-# (per-crate table in the job summary, lcov artifact kept 90 days) — the
-# per-crate ratchet for the security-critical crates builds on it later
-# (backlog#1153 infra-6, report-only first per the ci-11 ladder).
-#
-# Measurement scope matches the PR test gate (ci.yml "Run tests"):
-# `--workspace --exclude e2e_test` with the `ci` nextest profile. Doctests are
-# NOT measured (ci.yml runs them uninstrumented; `cargo llvm-cov` needs a
-# nightly toolchain to cover doctests). Trend-comparison workflow:
-# docs/testing/README.md "Coverage" section. `make coverage` is the local
-# equivalent. Scheduled failures alert via the ci-8 composite action.
-
-name: coverage
-
-on:
-  workflow_dispatch:
-  schedule:
-    # 07:00 UTC Sunday — staggered clear of the other Sunday crons: ci (00:00),
-    # build (01:00), e2e-s3tests (02:00), audit (03:00), nix-flake-update
-    # (05:00), mint (06:00), and the daily fuzz (02:00), minio-interop (03:17),
-    # e2e-replication-nightly (04:00) and performance-ab (06:00) lanes.
-    - cron: "0 7 * * 0"
-
-# Only alert-on-failure needs more than read access; it declares its own
-# job-level `issues: write`.
-permissions:
-  contents: read
-
-jobs:
-  coverage:
-    name: Workspace coverage (weekly)
-    runs-on: sm-standard-4
-    # The instrumented build cannot reuse the regular CI cache (different
-    # RUSTFLAGS), so a cold week rebuilds the workspace before running the
-    # full suite; give it double the test job's 60-minute budget.
-    timeout-minutes: 120
-    env:
-      FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"
-      # Match the PR gate's nextest semantics (ci.yml runs `--profile ci`):
-      # retries=0 plus the quarantine list and the ecstore-serial-flaky
-      # serialization. Set via env because `cargo llvm-cov`'s own --profile
-      # flag selects the *cargo build* profile, not the nextest profile.
-      NEXTEST_PROFILE: ci
-    steps:
-      - name: Checkout repository
-        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
-
-      - name: Setup Rust environment
-        uses: ./.github/actions/setup
-        with:
-          rust-version: stable
-          cache-shared-key: ci-coverage
-          github-token: ${{ secrets.GITHUB_TOKEN }}
-          cache-save-if: ${{ github.ref == 'refs/heads/main' }}
-
-      - name: Install cargo-llvm-cov
-        uses: taiki-e/install-action@bffeee26d4db9be238a4ea78d8826604ebcb594d # v2
-        with:
-          tool: cargo-llvm-cov
-
-      - name: Install llvm-tools component
-        run: rustup component add llvm-tools-preview
-
-      - name: Run instrumented test suite
-        run: cargo llvm-cov nextest --workspace --exclude e2e_test --no-report
-
-      - name: Generate lcov and JSON reports
-        run: |
-          mkdir -p target/llvm-cov
-          cargo llvm-cov report --lcov --output-path target/llvm-cov/lcov.info
-          cargo llvm-cov report --json --output-path target/llvm-cov/coverage.json
-
-      - name: Write per-crate summary
-        run: python3 scripts/coverage_per_crate.py target/llvm-cov/coverage.json >> "$GITHUB_STEP_SUMMARY"
-
-      - name: Upload coverage artifact
-        if: always()
-        uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
-        with:
-          name: coverage-lcov-${{ github.run_number }}
-          path: |
-            target/llvm-cov/lcov.info
-            target/llvm-cov/coverage.json
-          retention-days: 90
-          if-no-files-found: ignore
-
-  alert-on-failure:
-    name: Alert on scheduled failure
-    needs: [coverage]
-    # Only scheduled runs open/append the tracking issue (backlog#1149 ci-8);
-    # manual workflow_dispatch runs stay quiet so a debugging run never files a
-    # spurious alert.
-    if: always() && github.event_name == 'schedule' && contains(needs.*.result, 'failure')
-    runs-on: ubuntu-latest
-    timeout-minutes: 10
-    permissions:
-      contents: read
-      issues: write
-    steps:
-      - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
-      - name: Open or update failure-tracking issue
-        uses: ./.github/actions/schedule-failure-issue
-        with:
-          github-token: ${{ secrets.GITHUB_TOKEN }}

.github/workflows/docker.yml

@@ -70,19 +70,9 @@ env:
   DOCKER_PLATFORMS: linux/amd64,linux/arm64
 
 jobs:
-  # Check if we should build Docker images.
-  # Short-circuit at job level so per-merge development builds (workflow_run
-  # from a push to main) skip the whole run without spawning a checkout job:
-  # images are only published for tag builds — build.yml push triggers cover
-  # only main and tags, so a non-main head_branch is a tag name — and for
-  # manual dispatch.
+  # Check if we should build Docker images
   build-check:
     name: Docker Build Check
-    if: >-
-      github.event_name == 'workflow_dispatch' ||
-      (github.event.workflow_run.conclusion == 'success' &&
-       github.event.workflow_run.event == 'push' &&
-       github.event.workflow_run.head_branch != 'main')
     runs-on: ubuntu-latest
     outputs:
       should_build: ${{ steps.check.outputs.should_build }}
@@ -96,6 +86,7 @@ jobs:
       - name: Checkout repository
         uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
         with:
+          fetch-depth: 0
           # For workflow_run events, checkout the specific commit that triggered the workflow
           ref: ${{ github.event.workflow_run.head_sha || github.sha }}
 
@@ -247,15 +238,13 @@ jobs:
             esac
           fi
 
-          {
-            echo "should_build=$should_build"
-            echo "should_push=$should_push"
-            echo "build_type=$build_type"
-            echo "version=$version"
-            echo "short_sha=$short_sha"
-            echo "is_prerelease=$is_prerelease"
-            echo "create_latest=$create_latest"
-          } >> "$GITHUB_OUTPUT"
+          echo "should_build=$should_build" >> $GITHUB_OUTPUT
+          echo "should_push=$should_push" >> $GITHUB_OUTPUT
+          echo "build_type=$build_type" >> $GITHUB_OUTPUT
+          echo "version=$version" >> $GITHUB_OUTPUT
+          echo "short_sha=$short_sha" >> $GITHUB_OUTPUT
+          echo "is_prerelease=$is_prerelease" >> $GITHUB_OUTPUT
+          echo "create_latest=$create_latest" >> $GITHUB_OUTPUT
 
           echo "🐳 Docker Build Summary:"
           echo "  - Should build: $should_build"
@@ -322,6 +311,7 @@ jobs:
         run: |
           BUILD_TYPE="${{ needs.build-check.outputs.build_type }}"
           VERSION="${{ needs.build-check.outputs.version }}"
+          SHORT_SHA="${{ needs.build-check.outputs.short_sha }}"
           CREATE_LATEST="${{ needs.build-check.outputs.create_latest }}"
           VARIANT_SUFFIX="${{ matrix.suffix }}"
 
@@ -344,8 +334,8 @@ jobs:
               ;;
           esac
 
-          echo "docker_release=$DOCKER_RELEASE" >> "$GITHUB_OUTPUT"
-          echo "docker_channel=$DOCKER_CHANNEL" >> "$GITHUB_OUTPUT"
+          echo "docker_release=$DOCKER_RELEASE" >> $GITHUB_OUTPUT
+          echo "docker_channel=$DOCKER_CHANNEL" >> $GITHUB_OUTPUT
 
           echo "🐳 Docker build parameters:"
           echo "  - Original version: $VERSION"
@@ -377,7 +367,7 @@ jobs:
           fi
 
           # Output tags
-          echo "tags=$TAGS" >> "$GITHUB_OUTPUT"
+          echo "tags=$TAGS" >> $GITHUB_OUTPUT
 
           # Generate labels
           LABELS="org.opencontainers.image.title=RustFS"
@@ -388,7 +378,7 @@ jobs:
           LABELS="$LABELS,org.opencontainers.image.created=$(date -u +'%Y-%m-%dT%H:%M:%SZ')"
           LABELS="$LABELS,org.opencontainers.image.build-type=$BUILD_TYPE"
 
-          echo "labels=$LABELS" >> "$GITHUB_OUTPUT"
+          echo "labels=$LABELS" >> $GITHUB_OUTPUT
 
           echo "🐳 Generated Docker tags:"
           echo "$TAGS" | tr ',' '\n' | sed 's/^/  - /'
@@ -431,9 +421,6 @@ jobs:
     needs: [ build-check, build-docker ]
     if: needs.build-check.outputs.should_build == 'true' && needs.build-check.outputs.should_push == 'true'
     runs-on: ubuntu-latest
-    permissions:
-      contents: read
-      security-events: write
     strategy:
       fail-fast: false
       matrix:
@@ -463,16 +450,9 @@ jobs:
           severity: CRITICAL,HIGH
           exit-code: "0"
 
-      # Surface findings in the Security tab; an artifact alone is write-only
-      # reporting nobody reviews.
-      - name: Upload scan results to code scanning
-        uses: github/codeql-action/upload-sarif@99df26d4f13ea111d4ec1a7dddef6063f76b97e9 # v4.37.0
-        with:
-          sarif_file: trivy-${{ matrix.variant }}.sarif
-          category: container-image-${{ matrix.variant }}
-
       - name: Upload container scan report
-        uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
+        # actions/upload-artifact v7.0.1
+        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
         with:
           name: container-image-scan-${{ matrix.variant }}
           path: trivy-${{ matrix.variant }}.sarif

.github/workflows/e2e-replication-nightly.yml

@@ -1,130 +0,0 @@
-# Copyright 2024 RustFS Team
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-# Nightly full replication e2e lane (backlog#1147 repl-1, deps: ci-4).
-#
-# The per-PR gate (ci.yml `e2e-tests` job, `--profile e2e-smoke`) runs the 20
-# FAST replication tests. This scheduled lane runs the remaining 27
-# heavier replication e2e tests that are unfit for a per-PR gate:
-#
-#   * 2 remote-target TLS validation tests.
-#   * 12 bucket-replication data-plane/helper tests (PUT/delete + poll for
-#     convergence; two replicate over HTTPS, two pin active SSE failure
-#     contracts, and one guards event/history observers). The SSE-S3 contract
-#     remains ignored under backlog#1291.
-#   * 11 `_real_dual_node` site-replication tests (each spawns TWO rustfs
-#     servers and drives the cross-process site-replication control plane).
-#   * 1 `_real_three_node` site-replication test.
-#   * 1 `_real_single_node` service-account round-trip test.
-#
-# The selection is the [profile.e2e-repl-nightly] default-filter in
-# .config/nextest.toml — the single wiring mechanism (repl-1 / ci-4). Do NOT
-# add ad-hoc cargo-test steps here; change the filterset instead.
-#
-# Explicit division of labor: these 27 tests run ONLY here, never double-run
-# in ci-5's future e2e-full merge gate. TODO(ci-7): once the ci domain's
-# consolidated scheduled e2e workflow exists, fold this interim repl-owned lane
-# into it rather than growing a second scheduled entrypoint.
-
-name: e2e-replication-nightly
-
-on:
-  workflow_dispatch:
-  schedule:
-    # 04:00 UTC nightly — staggered clear of fuzz/e2e-s3tests (02:00),
-    # stale (01:30) and performance-ab (06:00).
-    - cron: "0 4 * * *"
-
-# Only alert-on-failure needs more than read access; it declares its own
-# job-level `issues: write`.
-permissions:
-  contents: read
-
-jobs:
-  repl-nightly:
-    name: Replication e2e (nightly)
-    # dual-node tests spawn two full rustfs servers each; the extra headroom
-    # over the per-PR sm-standard-2 e2e job keeps the parallel process fan-out
-    # from starving the runner. Mirrors the ILM serial lane's runner class.
-    runs-on: sm-standard-4
-    timeout-minutes: 45
-    env:
-      FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"
-    steps:
-      - name: Checkout repository
-        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
-
-      - name: Setup Rust environment
-        uses: ./.github/actions/setup
-        with:
-          rust-version: stable
-          cache-shared-key: ci-e2e-repl
-          github-token: ${{ secrets.GITHUB_TOKEN }}
-          cache-save-if: ${{ github.ref == 'refs/heads/main' }}
-
-      # awscurl lets the STS dual-node test actually exercise its path. Without
-      # it the test skips gracefully with a visible log line
-      # (`awscurl_available()` in crates/e2e_test/src/common.rs), so the lane
-      # still passes — installing it just upgrades that one test from skip to
-      # real coverage.
-      - name: Set up Python
-        uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6.3.0
-        with:
-          python-version: "3.12"
-
-      - name: Install awscurl
-        run: |
-          python3 -m pip install --user --upgrade pip awscurl
-          echo "AWSCURL_PATH=$HOME/.local/bin/awscurl" >> "$GITHUB_ENV"
-
-      - name: Verify awscurl
-        run: test -x "$AWSCURL_PATH"
-
-      # Build the rustfs binary once up front. The e2e tests spawn it as a
-      # child process (crates/e2e_test/src/common.rs) and will build it on
-      # demand otherwise, but a single explicit build avoids several parallel
-      # nextest test processes racing to build it at once.
-      - name: Build rustfs binary
-        run: cargo build -p rustfs --bins
-
-      - name: Run replication e2e nightly suite
-        run: cargo nextest run --profile e2e-repl-nightly -p e2e_test
-
-      - name: Upload nextest junit report
-        if: always()
-        uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
-        with:
-          name: e2e-replication-nightly-junit-${{ github.run_number }}
-          path: target/nextest/e2e-repl-nightly/junit.xml
-          retention-days: 7
-          if-no-files-found: ignore
-
-  alert-on-failure:
-    name: Alert on scheduled failure
-    needs: [repl-nightly]
-    # Only scheduled runs open/append the tracking issue (backlog#1149 ci-8);
-    # manual workflow_dispatch runs stay quiet so a debugging run never files a
-    # spurious alert.
-    if: always() && github.event_name == 'schedule' && contains(needs.*.result, 'failure')
-    runs-on: ubuntu-latest
-    timeout-minutes: 10
-    permissions:
-      contents: read
-      issues: write
-    steps:
-      - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
-      - name: Open or update failure-tracking issue
-        uses: ./.github/actions/schedule-failure-issue
-        with:
-          github-token: ${{ secrets.GITHUB_TOKEN }}

.github/workflows/e2e-s3tests.yml

@@ -28,22 +28,6 @@
 # truth, also used locally and by the PR gate). This workflow only provisions
 # the server topology: a single node, or a real 4-node distributed cluster
 # (erasure-coded across nodes) behind an HAProxy round-robin load balancer.
-#
-# Runner infrastructure (ci-1, backlog#1149): this sweep needs BOTH a working
-# Docker daemon (to build the source image and run the single/multi topologies)
-# AND a Python toolchain with pip (for awscurl/tox and the pytest harness).
-# It previously ran on the self-hosted `sm-standard-4` label, which is served
-# by heterogeneous runners: some scheduled runs landed on minimal pods that
-# had neither `pip` (bare python3, `No module named pip`) nor `docker.sock`,
-# so "Install Python tools" died before any test executed (all 15 recorded
-# runs failed). Two changes make the infra assumptions explicit instead of
-# trusting drifting runner state:
-#   1. Pin to GitHub-hosted `ubuntu-latest`, which reliably ships Docker,
-#      docker compose, python3 and pip (no self-hosted fleet drift).
-#   2. Set up Python explicitly (actions/setup-python) before any pip usage,
-#      so the workflow stays correct even if the runner image changes.
-# The PR gate (ci.yml s3-implemented-tests) is unaffected: it avoids Docker
-# via DEPLOY_MODE=binary and defers all pip setup to run.sh's self-bootstrap.
 
 name: e2e-s3tests
 
@@ -85,8 +69,8 @@ on:
 
 env:
   # main user
-  S3_ACCESS_KEY: rustfsadmin-ci
-  S3_SECRET_KEY: rustfssecret-ci
+  S3_ACCESS_KEY: rustfsadmin
+  S3_SECRET_KEY: rustfsadmin
   # alt user (must be different from main for many s3-tests)
   S3_ALT_ACCESS_KEY: rustfsalt
   S3_ALT_SECRET_KEY: rustfsalt
@@ -109,22 +93,13 @@ concurrency:
   group: ${{ github.workflow }}-${{ github.ref }}-${{ github.event.inputs['test-mode'] || 'single' }}
   cancel-in-progress: true
 
-# Only alert-on-failure needs more than read access; it declares its own
-# job-level `issues: write`.
-permissions:
-  contents: read
-
 defaults:
   run:
     shell: bash
 
 jobs:
   s3tests:
-    # GitHub-hosted: reliably provides Docker + docker compose + python3/pip.
-    # See the header note (ci-1) for why the self-hosted sm-standard-4 label
-    # was abandoned. TODO(ci-8): scheduled-failure alerting (auto-open issue)
-    # is added by the ci-8 composite action; do not implement it here.
-    runs-on: ubuntu-latest
+    runs-on: sm-standard-4
     timeout-minutes: 180
     strategy:
       fail-fast: false
@@ -136,14 +111,6 @@ jobs:
     steps:
       - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
 
-      # Provision Python explicitly rather than trusting the runner image to
-      # ship a working pip (ci-1: a bare python3 without pip is what broke the
-      # scheduled sweep). Keeps the workflow correct across runner drift.
-      - name: Set up Python
-        uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6.3.0
-        with:
-          python-version: "3.12"
-
       - name: Cache pip downloads
         uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6
         with:
@@ -163,9 +130,9 @@ jobs:
       - name: Build RustFS image (source, cached)
         run: |
           DOCKER_BUILDKIT=1 docker buildx build --load \
-            --platform "${PLATFORM}" \
-            --cache-from "type=gha,scope=${BUILDX_CACHE_SCOPE}" \
-            --cache-to "type=gha,mode=max,scope=${BUILDX_CACHE_SCOPE}" \
+            --platform ${PLATFORM} \
+            --cache-from type=gha,scope=${BUILDX_CACHE_SCOPE} \
+            --cache-to type=gha,mode=max,scope=${BUILDX_CACHE_SCOPE} \
             -t rustfs-ci \
             -f Dockerfile.source .
 
@@ -174,18 +141,13 @@ jobs:
         run: |
           docker network inspect rustfs-net >/dev/null 2>&1 || docker network create rustfs-net
           docker rm -f rustfs-single >/dev/null 2>&1 || true
-          # The four disks share one physical device on the runner (a single
-          # loopback filesystem), so the local physical-disk-independence guard
-          # would refuse to start. Bypass it — this is the CI use case the guard
-          # explicitly sanctions via RUSTFS_UNSAFE_BYPASS_DISK_CHECK.
           docker run -d --name rustfs-single \
             --network rustfs-net \
-            -p "${S3_PORT}:9000" \
+            -p ${S3_PORT}:9000 \
             -e RUSTFS_ADDRESS=0.0.0.0:9000 \
-            -e RUSTFS_ACCESS_KEY="${S3_ACCESS_KEY}" \
-            -e RUSTFS_SECRET_KEY="${S3_SECRET_KEY}" \
+            -e RUSTFS_ACCESS_KEY=${S3_ACCESS_KEY} \
+            -e RUSTFS_SECRET_KEY=${S3_SECRET_KEY} \
             -e RUSTFS_VOLUMES="/data/rustfs{0...3}" \
-            -e RUSTFS_UNSAFE_BYPASS_DISK_CHECK=true \
             -v /tmp/rustfs-single:/data \
             rustfs-ci
 
@@ -205,10 +167,6 @@ jobs:
               RUSTFS_ACCESS_KEY: ${S3_ACCESS_KEY}
               RUSTFS_SECRET_KEY: ${S3_SECRET_KEY}
               RUSTFS_VOLUMES: "http://rustfs{1...4}:9000/data/rustfs{0...3}"
-              # Each node's four disks share one physical device inside its
-              # container, so bypass the local physical-disk-independence guard
-              # (the CI use case it explicitly sanctions).
-              RUSTFS_UNSAFE_BYPASS_DISK_CHECK: "true"
 
           services:
             rustfs1:
@@ -268,7 +226,7 @@ jobs:
 
       - name: Wait for RustFS ready
         run: |
-          for _ in {1..120}; do
+          for i in {1..120}; do
             if curl -sf "http://${S3_HOST}:${S3_PORT}/health" >/dev/null 2>&1; then
               echo "RustFS is ready"
               exit 0
@@ -339,22 +297,3 @@ jobs:
         with:
           name: s3tests-${{ env.TEST_MODE }}
           path: artifacts/**
-
-  alert-on-failure:
-    name: Alert on scheduled failure
-    needs: [s3tests]
-    # `always()` is required: without it this job is skipped when a needed
-    # job fails. Alerts only for scheduled runs (backlog#1149 ci-8) — manual
-    # dispatch failures are already watched by a human.
-    if: always() && github.event_name == 'schedule' && contains(needs.*.result, 'failure')
-    runs-on: ubuntu-latest
-    timeout-minutes: 10
-    permissions:
-      contents: read
-      issues: write
-    steps:
-      - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
-      - name: Open or update failure-tracking issue
-        uses: ./.github/actions/schedule-failure-issue
-        with:
-          github-token: ${{ secrets.GITHUB_TOKEN }}

.github/workflows/fuzz.yml

@@ -17,16 +17,13 @@ name: Fuzz
 on:
   pull_request:
     types: [ opened, synchronize, reopened, closed ]
-    # PR trigger is intentionally narrow: only changes to the fuzz harness
-    # itself gate a PR. Broad crate paths (ecstore/filemeta/utils/policy/…)
-    # are covered by the nightly `schedule` run below, which fuzzes against
-    # whatever landed on main. Widening these paths previously queued a
-    # ~45min fuzz-build on nearly every PR and is why this workflow was
-    # disabled; do not re-add crate paths here.
     paths:
       - "fuzz/**"
       - "scripts/fuzz/**"
       - ".github/workflows/fuzz.yml"
+      - "crates/ecstore/**"
+      - "crates/filemeta/**"
+      - "crates/utils/**"
   schedule:
     - cron: "0 2 * * *"
   workflow_dispatch:
@@ -51,8 +48,7 @@ concurrency:
 env:
   CARGO_TERM_COLOR: always
   CARGO_BUILD_TARGET: x86_64-unknown-linux-gnu
-  # Fuzz targets build without the dial9 feature, so tokio_unstable is not needed.
-  RUSTFLAGS: "-C target-feature=-crt-static"
+  RUSTFLAGS: "--cfg tokio_unstable -C target-feature=-crt-static"
 
 jobs:
   cancel-closed-pr-runs:
@@ -100,11 +96,7 @@ jobs:
         run: |
           binary_root="fuzz/prebuilt/${CARGO_BUILD_TARGET}/release"
           mkdir -p "${binary_root}"
-          # Keep this list in sync with the smoke/nightly matrices and
-          # scripts/fuzz/run.sh default target set (all 5 buildable bins in
-          # fuzz/Cargo.toml). The *_storage_api.rs files are `mod` submodules
-          # of their parent targets, not standalone bins.
-          for target in archive_extract bucket_validation local_metadata path_containment policy_ingress; do
+          for target in path_containment bucket_validation local_metadata; do
             install -Dm755 "fuzz/target/${CARGO_BUILD_TARGET}/release/${target}" "${binary_root}/${target}"
           done
 
@@ -113,11 +105,9 @@ jobs:
         with:
           name: fuzz-prebuilt-binaries-${{ github.run_number }}
           path: |
-            fuzz/prebuilt/${{ env.CARGO_BUILD_TARGET }}/release/archive_extract
+            fuzz/prebuilt/${{ env.CARGO_BUILD_TARGET }}/release/path_containment
             fuzz/prebuilt/${{ env.CARGO_BUILD_TARGET }}/release/bucket_validation
             fuzz/prebuilt/${{ env.CARGO_BUILD_TARGET }}/release/local_metadata
-            fuzz/prebuilt/${{ env.CARGO_BUILD_TARGET }}/release/path_containment
-            fuzz/prebuilt/${{ env.CARGO_BUILD_TARGET }}/release/policy_ingress
           if-no-files-found: error
           retention-days: 1
           compression-level: 0
@@ -136,10 +126,8 @@ jobs:
     timeout-minutes: 30
     strategy:
       fail-fast: false
-      # One job per target runs in parallel, so wall-clock time is per-target
-      # (~60s fuzz + download/chmod overhead), not the sum across the matrix.
       matrix:
-        target: [archive_extract, bucket_validation, local_metadata, path_containment, policy_ingress]
+        target: [path_containment, bucket_validation, local_metadata]
     env:
       FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"
     steps:
@@ -181,10 +169,6 @@ jobs:
   nightly-fuzz-corpus:
     name: "Nightly / ${{ matrix.target }}"
     needs: fuzz-build
-    # TODO(ci-8): when the schedule-failure-issue composite action lands,
-    # add a step here (or a dependent job) that opens/updates a GitHub issue
-    # on nightly failure. ci-8 is the single alerting mechanism for all
-    # scheduled workflows; do not self-roll alerting in this workflow.
     if: >
       github.event_name == 'schedule' ||
       (github.event_name == 'workflow_dispatch' &&
@@ -194,7 +178,7 @@ jobs:
     strategy:
       fail-fast: false
       matrix:
-        target: [archive_extract, bucket_validation, local_metadata, path_containment, policy_ingress]
+        target: [path_containment, bucket_validation, local_metadata]
     env:
       FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"
     steps:
@@ -229,25 +213,3 @@ jobs:
             fuzz/corpus/${{ matrix.target }}/**
           if-no-files-found: ignore
           retention-days: 30
-
-  # ──────────────────────────────────────────────────────────────
-  # Nightly alerting: open/update a tracking issue on failure.
-  # ──────────────────────────────────────────────────────────────
-  alert-on-failure:
-    name: Alert on scheduled failure
-    needs: [fuzz-build, nightly-fuzz-corpus]
-    # `always()` is required: without it this job is skipped when a needed
-    # job fails. Alerts only for scheduled (nightly) runs (backlog#1149
-    # ci-8); PR and manual dispatch failures are already watched by a human.
-    if: always() && github.event_name == 'schedule' && contains(needs.*.result, 'failure')
-    runs-on: ubuntu-latest
-    timeout-minutes: 10
-    permissions:
-      contents: read
-      issues: write
-    steps:
-      - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
-      - name: Open or update failure-tracking issue
-        uses: ./.github/actions/schedule-failure-issue
-        with:
-          github-token: ${{ secrets.GITHUB_TOKEN }}

.github/workflows/minio-interop.yml

@@ -1,70 +0,0 @@
-# Copyright 2024 RustFS Team
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-# MinIO on-disk interop: prove RustFS reads MinIO-written erasure-coded SSE
-# objects with byte-identical data and correct logical size.
-#
-# This is NOT a PR gate. The fixtures are real MinIO backend trees generated on
-# the fly (they are gitignored, never committed), so the job regenerates them
-# each run with Docker and then runs the `#[ignore]` reader tests in
-# crates/ecstore/tests/minio_generated_read_test.rs.
-#
-# Runner: GitHub-hosted `ubuntu-latest`. It reliably ships Docker + Python,
-# unlike the self-hosted fleet, whose pods drift in Docker/pip availability
-# (see the infra note in e2e-s3tests.yml). Nightly + manual only.
-name: minio-interop
-
-on:
-  workflow_dispatch:
-  schedule:
-    # Nightly at 03:17 UTC (offset from other nightly jobs).
-    - cron: "17 3 * * *"
-
-concurrency:
-  group: ${{ github.workflow }}-${{ github.ref }}
-  cancel-in-progress: true
-
-permissions:
-  contents: read
-
-jobs:
-  minio-interop:
-    name: MinIO interop (EC + SSE read parity)
-    # Skip on forks: needs the repo's runners and is not a contributor gate.
-    if: github.repository == 'rustfs/rustfs'
-    runs-on: ubuntu-latest
-    timeout-minutes: 40
-    env:
-      # Fixed 32-byte test KMS key baked into the fixture lab; not a secret.
-      RUSTFS_MINIO_STATIC_KMS_KEY_B64: IyqsU3kMFloCNup4BsZtf/rmfHVcTgznO2F25CkEH1g=
-    steps:
-      - name: Checkout repository
-        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
-
-      - name: Setup Rust environment
-        uses: ./.github/actions/setup
-        with:
-          rust-version: stable
-          cache-shared-key: ci-minio-interop
-          github-token: ${{ secrets.GITHUB_TOKEN }}
-          cache-save-if: ${{ github.ref == 'refs/heads/main' }}
-
-      - name: Generate real MinIO fixtures via Docker
-        run: bash crates/rio-v2/tests/minio_fixture_lab/capture_via_docker.sh
-
-      - name: Run MinIO interop reader tests
-        run: |
-          cargo nextest run --run-ignored ignored-only \
-            -p rustfs-ecstore --features rio-v2 \
-            -E 'binary(minio_generated_read_test)'

.github/workflows/mint.yml

@@ -26,24 +26,7 @@
 # the run red. The job fails only when mint produces no results at all
 # (infrastructure error). Once a suite passes reliably, tightening this into a
 # baseline gate (like implemented_tests.txt for s3-tests) is the natural next
-# step. Tightening this into a per-suite baseline gate is tracked as ci-3
-# (backlog#1149) and is intentionally NOT done here.
-#
-# Runner infrastructure (ci-2, backlog#1149): this job needs a working Docker
-# daemon to (a) build the RustFS source image with buildx and (b) run both the
-# RustFS server and the mint image. It previously ran on the self-hosted
-# `sm-standard-4` label, which is served by heterogeneous runners; its single
-# recorded run (28730530597, 2026-07-05) landed on a minimal pod with no
-# `/var/run/docker.sock`, so "Enable buildx" died at `docker buildx create`
-# ("failed to connect to the docker API at unix:///var/run/docker.sock") before
-# any test executed -- the same root cause ci-1 fixed for the weekly s3-tests
-# sweep. The fix is to pin to GitHub-hosted `ubuntu-latest`, which reliably
-# ships a running Docker daemon + buildx + python3, instead of trusting the
-# drifting self-hosted fleet. Tradeoff: the source Rust build is heavier on a
-# GitHub-hosted 4-vCPU runner than on a warm self-hosted host, but it stays
-# well inside the 120-minute budget and buys deterministic infra. The
-# docker-capable self-hosted `dind-sm-standard-2` label was the alternative but
-# has fewer cores and reintroduces fleet-state risk for no reliability gain.
+# step.
 
 name: mint
 
@@ -67,13 +50,12 @@ on:
         required: false
         default: "minio/mint:edge"
   schedule:
-    # Weekly, after the Sunday s3-tests full sweep (starts 02:00 UTC, up to
-    # 3h) has finished, so the two never contend for the same runner pool.
-    - cron: "0 6 * * 0"
+    # Weekly, after the Sunday s3-tests full sweep.
+    - cron: "0 4 * * 0"
 
 env:
-  S3_ACCESS_KEY: rustfsadmin-ci
-  S3_SECRET_KEY: rustfssecret-ci
+  S3_ACCESS_KEY: rustfsadmin
+  S3_SECRET_KEY: rustfsadmin
   S3_REGION: us-east-1
 
   RUST_LOG: info
@@ -82,39 +64,21 @@ env:
 
   MINT_SUITES: ${{ github.event.inputs.suites || '' }}
   MINT_MODE: ${{ github.event.inputs.mode || 'core' }}
-  # minio/mint:edge is a rolling tag, so an unpinned run is not reproducible and
-  # can go red purely from an upstream mint change (ci-2, backlog#1149). Pin the
-  # default to the linux/amd64 digest resolved on 2026-07-10. workflow_dispatch
-  # can still override via the `mint-image` input (e.g. to test a newer edge).
-  # Refresh: docker manifest inspect minio/mint:edge --verbose | grep digest
-  #   (or, without a docker daemon:)
-  #   TOKEN=$(curl -s "https://auth.docker.io/token?service=registry.docker.io&scope=repository:minio/mint:pull" | jq -r .token)
-  #   curl -sI -H "Authorization: Bearer $TOKEN" \
-  #     -H "Accept: application/vnd.docker.distribution.manifest.v2+json" \
-  #     https://registry-1.docker.io/v2/minio/mint/manifests/edge | grep -i docker-content-digest
-  MINT_IMAGE: ${{ github.event.inputs.mint-image || 'minio/mint:edge@sha256:08a05e68893c68be2a83b6f79556853ed6aa3c6c9e64c823a00853e4e55d2200' }}
+  # NOTE: minio/mint:edge is a rolling tag. If sweeps get noisy from upstream
+  # mint changes, pin this to a digest known to work.
+  MINT_IMAGE: ${{ github.event.inputs.mint-image || 'minio/mint:edge' }}
 
 concurrency:
   group: ${{ github.workflow }}-${{ github.ref }}
   cancel-in-progress: true
 
-# Only alert-on-failure needs more than read access; it declares its own
-# job-level `issues: write`.
-permissions:
-  contents: read
-
 defaults:
   run:
     shell: bash
 
 jobs:
   mint:
-    # GitHub-hosted: reliably provides a running Docker daemon + buildx +
-    # python3. See the header note (ci-2) for why the self-hosted sm-standard-4
-    # label was abandoned (no docker.sock on the pod its only run landed on).
-    # TODO(ci-8): scheduled-failure alerting (auto-open issue on a red weekly
-    # run) is added by the ci-8 composite action; do not implement it here.
-    runs-on: ubuntu-latest
+    runs-on: sm-standard-4
     timeout-minutes: 120
     steps:
       - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
@@ -125,9 +89,9 @@ jobs:
       - name: Build RustFS image (source, cached)
         run: |
           DOCKER_BUILDKIT=1 docker buildx build --load \
-            --platform "${PLATFORM}" \
-            --cache-from "type=gha,scope=${BUILDX_CACHE_SCOPE}" \
-            --cache-to "type=gha,mode=max,scope=${BUILDX_CACHE_SCOPE}" \
+            --platform ${PLATFORM} \
+            --cache-from type=gha,scope=${BUILDX_CACHE_SCOPE} \
+            --cache-to type=gha,mode=max,scope=${BUILDX_CACHE_SCOPE} \
             -t rustfs-ci \
             -f Dockerfile.source .
 
@@ -135,24 +99,19 @@ jobs:
         run: |
           docker network inspect rustfs-net >/dev/null 2>&1 || docker network create rustfs-net
           docker rm -f rustfs-mint >/dev/null 2>&1 || true
-          # The four disks share one physical device on the runner (a single
-          # loopback filesystem), so the local physical-disk-independence guard
-          # would refuse to start. Bypass it — this is the CI use case the guard
-          # explicitly sanctions via RUSTFS_UNSAFE_BYPASS_DISK_CHECK.
           docker run -d --name rustfs-mint \
             --network rustfs-net \
             -p 9000:9000 \
             -e RUSTFS_ADDRESS=0.0.0.0:9000 \
-            -e RUSTFS_ACCESS_KEY="${S3_ACCESS_KEY}" \
-            -e RUSTFS_SECRET_KEY="${S3_SECRET_KEY}" \
+            -e RUSTFS_ACCESS_KEY=${S3_ACCESS_KEY} \
+            -e RUSTFS_SECRET_KEY=${S3_SECRET_KEY} \
             -e RUSTFS_VOLUMES="/data/rustfs{0...3}" \
-            -e RUSTFS_UNSAFE_BYPASS_DISK_CHECK=true \
             -v /tmp/rustfs-mint:/data \
             rustfs-ci
 
       - name: Wait for RustFS ready
         run: |
-          for _ in {1..60}; do
+          for i in {1..60}; do
             if curl -sf http://127.0.0.1:9000/health >/dev/null 2>&1; then
               echo "RustFS is ready"
               exit 0
@@ -248,22 +207,3 @@ jobs:
         with:
           name: mint
           path: artifacts/**
-
-  alert-on-failure:
-    name: Alert on scheduled failure
-    needs: [mint]
-    # `always()` is required: without it this job is skipped when a needed
-    # job fails. Alerts only for scheduled runs (backlog#1149 ci-8) — manual
-    # dispatch failures are already watched by a human.
-    if: always() && github.event_name == 'schedule' && contains(needs.*.result, 'failure')
-    runs-on: ubuntu-latest
-    timeout-minutes: 10
-    permissions:
-      contents: read
-      issues: write
-    steps:
-      - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
-      - name: Open or update failure-tracking issue
-        uses: ./.github/actions/schedule-failure-issue
-        with:
-          github-token: ${{ secrets.GITHUB_TOKEN }}

.github/workflows/nix-flake-update.yml

@@ -17,7 +17,7 @@ name: Update Nix Flake
 on:
   workflow_dispatch:
   schedule:
-    - cron: '0 5 * * 0' # Weekly on Sunday 05:00 UTC (staggered after the midnight ci/build crons)
+    - cron: '0 0 * * 0' # Weekly on Sundays
 
 permissions:
   contents: write

.github/workflows/performance-ab.yml

@@ -30,9 +30,9 @@ on:
   workflow_dispatch:
     inputs:
       duration:
-        description: "warp duration per round (short by default to fit the double-build budget)"
+        description: "warp duration per round"
         required: false
-        default: "12s"
+        default: "30s"
         type: string
       allow_regression:
         description: "Pass the gate despite a FAIL (deliberate tradeoff)"
@@ -41,103 +41,25 @@ on:
         type: boolean
   pull_request:
     types: [labeled, synchronize, reopened]
-  push:
-    # Every main commit pre-builds and caches its release binary (perf-3) so the
-    # nightly A/B restores a ready baseline instead of paying the double build.
-    branches: [main]
 
 permissions:
   contents: read
   pull-requests: write
 
-# Per-PR: a new push cancels the previous (up to 90-minute) A/B run instead of
-# stacking them. Nightly schedule and manual dispatch get a unique group and
-# always run to completion.
-concurrency:
-  group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}
-  cancel-in-progress: ${{ github.event_name == 'pull_request' }}
-
 env:
   CARGO_TERM_COLOR: always
   RUST_BACKTRACE: 1
 
 jobs:
-  # perf-3: on every push to main, build the release binary once and cache it
-  # keyed by commit SHA (rustfs-baseline-<sha>). The nightly A/B (and, later, the
-  # perf-7 PR gate) restore this instead of paying the ~32min-per-side source
-  # build. That double build is what pushed the expanded 24-cell nightly past its
-  # ceiling — 2026-07-11..07-14 all cancelled on the 120min timeout. Incremental
-  # builds off the shared cargo cache keep each push cheap, and building on the
-  # same sm-standard-2 runner the A/B measures on guarantees the cached binary is
-  # ABI-identical. Do NOT source this from build.yml's per-merge artifact: those
-  # are cancelled ~7/8 of the time and are not a reliable baseline.
-  build-baseline-cache:
-    name: Build + cache baseline binary
-    if: github.event_name == 'push'
-    runs-on: sm-standard-2
-    # Latest-wins: consumers only ever restore the binary for the *current*
-    # origin/main tip, so when pushes land faster than the ~65min build, a
-    # superseded build's output is dead weight — cancel it instead of stacking
-    # hour-long jobs on the shared runner pool. A skipped intermediate SHA at
-    # most costs one same-commit self-heal in the A/B job.
-    concurrency:
-      group: perf-baseline-build-main
-      cancel-in-progress: true
-    # #4806 put thin LTO + codegen-units=1 on [profile.release], pushing a
-    # single release build past 60min on this runner — every cache build on
-    # 2026-07-15 died on the old 60min ceiling ("exceeded the maximum execution
-    # time of 1h0m0s") and the cache never populated. The measured binary must
-    # keep the production profile, so the budget absorbs the build instead.
-    timeout-minutes: 100
-    steps:
-      - name: Checkout repository
-        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
-
-      - name: Setup Rust environment
-        uses: ./.github/actions/setup
-        with:
-          rust-version: stable
-          cache-shared-key: warp-ab-${{ hashFiles('**/Cargo.lock') }}
-          cache-save-if: ${{ github.ref == 'refs/heads/main' }}
-          github-token: ${{ secrets.GITHUB_TOKEN }}
-
-      - name: Build release rustfs
-        run: cargo build --release --bin rustfs
-
-      - name: Stage binary for cache
-        run: |
-          set -euo pipefail
-          mkdir -p baseline-bin
-          cp target/release/rustfs baseline-bin/rustfs
-
-      - name: Cache baseline binary by SHA
-        uses: actions/cache/save@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6
-        with:
-          path: baseline-bin/rustfs
-          key: rustfs-baseline-${{ github.sha }}
-
   warp-ab:
     name: Warp A/B budget gate
-    # Always run on schedule / manual dispatch. Opt-in on PRs: only when the
-    # `perf-ab` label is present, and for `labeled` events only when the label
-    # being added is `perf-ab` itself (adding an unrelated label to an opted-in
-    # PR must not re-run the gate). Never on push — that event only feeds
-    # build-baseline-cache above.
+    # Opt-in on PRs: only run when the `perf-ab` label is present. Always run on
+    # schedule / manual dispatch.
     if: >-
-      github.event_name == 'schedule' ||
-      github.event_name == 'workflow_dispatch' ||
-      (github.event_name == 'pull_request' &&
-       contains(github.event.pull_request.labels.*.name, 'perf-ab') &&
-       (github.event.action != 'labeled' || github.event.label.name == 'perf-ab'))
+      github.event_name != 'pull_request' ||
+      contains(github.event.pull_request.labels.*.name, 'perf-ab')
     runs-on: sm-standard-2
-    # With perf-3's cached baseline binary the common (cache-hit) nightly is
-    # measurement-only and finishes well under 50min. This ceiling stays
-    # generous only to absorb the same-commit cache-miss self-heal (~65min
-    # single build with the post-#4806 LTO profile + measurement). A timeout
-    # surfaces via the alert-on-failure job (it fires on cancelled/timed-out,
-    # not just failure). perf-6 recalibrates the budget once the noise study
-    # lands.
-    timeout-minutes: 120
+    timeout-minutes: 90
     steps:
       - name: Checkout repository
         uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
@@ -173,106 +95,11 @@ jobs:
           fi
           echo "allow_regression=$allow" >> "$GITHUB_OUTPUT"
 
-      # perf-3: resolve the commits so the cache can be keyed by SHA. The
-      # baseline is origin/main; the candidate is the checked-out ref. On the
-      # nightly (checkout == main) they are the same commit, so one cached binary
-      # serves both phases and the run does zero source builds.
-      - name: Resolve baseline / candidate commits
-        id: commits
-        run: |
-          set -euo pipefail
-          baseline_sha="$(git rev-parse origin/main)"
-          candidate_sha="$(git rev-parse HEAD)"
-          echo "baseline_sha=$baseline_sha" >> "$GITHUB_OUTPUT"
-          echo "candidate_sha=$candidate_sha" >> "$GITHUB_OUTPUT"
-          echo "baseline commit: $baseline_sha"
-          echo "candidate commit: $candidate_sha"
-
-      # Exact-key restore of the baseline binary built by build-baseline-cache
-      # when origin/main last landed. A miss (binary evicted or not built yet)
-      # leaves cache-hit unset and the rig falls back to a source build.
-      - name: Restore cached baseline binary
-        id: baseline_cache
-        uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6
-        with:
-          path: baseline-bin/rustfs
-          key: rustfs-baseline-${{ steps.commits.outputs.baseline_sha }}
-
-      # Self-heal: on a nightly/dispatch run where the candidate commit IS the
-      # baseline commit, a cache miss would make the rig build the same commit
-      # twice (~65min per side with the post-#4806 LTO profile — no job budget
-      # fits that). Build it once here, reuse it for both phases, and save it
-      # back to the cache so the next run hits.
-      - name: Build baseline on cache miss (same-commit self-heal)
-        id: selfheal
-        if: >-
-          steps.baseline_cache.outputs.cache-hit != 'true' &&
-          steps.commits.outputs.baseline_sha == steps.commits.outputs.candidate_sha
-        run: |
-          set -euo pipefail
-          cargo build --release --bin rustfs
-          mkdir -p baseline-bin
-          cp target/release/rustfs baseline-bin/rustfs
-          echo "built=true" >> "$GITHUB_OUTPUT"
-
-      - name: Save self-healed baseline to cache
-        if: steps.selfheal.outputs.built == 'true'
-        uses: actions/cache/save@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6
-        with:
-          path: baseline-bin/rustfs
-          key: rustfs-baseline-${{ steps.commits.outputs.baseline_sha }}
-
       - name: Run warp A/B and gate
         id: ab
         run: |
           set -euo pipefail
-          # Budget note: with perf-3's cached baseline the nightly does no source
-          # build on a cache hit, so the wall-clock is dominated by the short warp
-          # matrix — duration/rounds/cooldown are kept small to fit all 24 cells
-          # (6 workloads x 2 phases x 2 drive-sync) rather than dropping cells.
-          # --health-timeout 180 outlasts the server's own 120s startup-readiness
-          # budget, which the rig's previous 60s health poll undershot (the first
-          # two nightly failures). perf-6 recalibrates these once the noise study
-          # lands.
-          duration="${{ github.event.inputs.duration || '12s' }}"
-          baseline_sha="${{ steps.commits.outputs.baseline_sha }}"
-          candidate_sha="${{ steps.commits.outputs.candidate_sha }}"
-          baseline_hit="${{ steps.baseline_cache.outputs.cache-hit }}"
-          selfheal_built="${{ steps.selfheal.outputs.built }}"
-
-          args=(--duration "$duration" --rounds 2 --cooldown 5 --health-timeout 180)
-
-          if [[ "$baseline_hit" == "true" || "$selfheal_built" == "true" ]]; then
-            chmod +x baseline-bin/rustfs
-            base_bin="$PWD/baseline-bin/rustfs"
-            args+=(--baseline-bin "$base_bin")
-            if [[ "$baseline_hit" == "true" ]]; then
-              base_src="actions-cache (rustfs-baseline-$baseline_sha)"
-            else
-              base_src="source build (cache self-heal, saved as rustfs-baseline-$baseline_sha)"
-            fi
-            if [[ "$candidate_sha" == "$baseline_sha" ]]; then
-              # Nightly on main: the candidate is the same commit as the baseline,
-              # so reuse the one binary for both phases and skip all builds.
-              args+=(--candidate-bin "$base_bin" --skip-build)
-              cand_src="same binary as baseline (same commit)"
-            else
-              cand_src="source build of the checked-out ref"
-            fi
-          else
-            # Cache miss with candidate != baseline (opt-in PR gate only): fall
-            # back to the source double-build. With the post-#4806 LTO profile
-            # this will overrun the job budget and alert; rerun once the push
-            # cache build for origin/main has completed, or wait for perf-7's
-            # merge-base caching.
-            args+=(--baseline-ref origin/main)
-            base_src="source build of origin/main (cache miss)"
-            cand_src="source build of the checked-out ref"
-          fi
-
-          args+=(--provenance-note "baseline commit: $baseline_sha - $base_src")
-          args+=(--provenance-note "candidate commit: $candidate_sha - $cand_src")
-
+          args=(--baseline-ref origin/main --duration "${{ github.event.inputs.duration || '30s' }}")
           if [[ "${{ steps.exempt.outputs.allow_regression }}" == "true" ]]; then
             args+=(--allow-regression --exemption-reason "labeled perf-deliberate-tradeoff / dispatch override")
           fi
@@ -282,14 +109,7 @@ jobs:
           bash scripts/run_hotpath_warp_ab.sh "${args[@]}"
           echo "status=$?" >> "$GITHUB_OUTPUT"
           set -e
-          # Locate the newest run dir + gate.md for the summary/comment/artifact
-          # steps. On a startup failure there is no gate.md, but the run dir still
-          # holds server-logs/ for diagnosis.
-          # Run dirs are UTC-timestamp names (no special chars); ls is safe here.
-          # shellcheck disable=SC2012
-          run_dir="$(ls -td target/hotpath-ab/*/ 2>/dev/null | head -n1 || true)"
-          echo "run_dir=${run_dir%/}" >> "$GITHUB_OUTPUT"
-          # shellcheck disable=SC2012
+          # Locate the newest gate.md for the comment/artifact steps.
           gate_md="$(ls -t target/hotpath-ab/*/gate.md 2>/dev/null | head -n1 || true)"
           echo "gate_md=$gate_md" >> "$GITHUB_OUTPUT"
 
@@ -298,49 +118,8 @@ jobs:
         uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
         with:
           name: hotpath-warp-ab-${{ github.run_number }}
-          # Includes per-cell median_summary.csv / baseline_compare.csv, gate.md,
-          # and server-logs/ (rustfs.log + startup env per phase) so a failed run
-          # is diagnosable. Short retention: this is churny nightly debug data.
           path: target/hotpath-ab/
           if-no-files-found: warn
-          retention-days: 14
-
-      - name: Write gate summary
-        if: always()
-        run: |
-          set -euo pipefail
-          status="${{ steps.ab.outputs.status }}"
-          gate_md="${{ steps.ab.outputs.gate_md }}"
-          run_dir="${{ steps.ab.outputs.run_dir }}"
-          {
-            echo "## Hotpath warp A/B — run ${{ github.run_number }}"
-            echo
-            if [[ "$status" == "0" ]]; then
-              echo "Rig/gate exit: \`0\` (pass or warn)."
-            else
-              echo "Rig/gate exit: \`${status:-unknown}\` — **FAILED**."
-            fi
-            echo
-            if [[ -n "$gate_md" && -f "$gate_md" ]]; then
-              cat "$gate_md"
-            else
-              echo "No \`gate.md\` produced — the rig failed **before** the gate"
-              echo "(most likely server startup / health). Failing phase(s) below;"
-              echo "full logs in the \`hotpath-warp-ab-${{ github.run_number }}\` artifact."
-              if [[ -n "$run_dir" && -d "$run_dir/server-logs" ]]; then
-                for f in "$run_dir"/server-logs/*.log; do
-                  [[ -f "$f" ]] || continue
-                  echo
-                  echo "<details><summary>$(basename "$f")</summary>"
-                  echo
-                  echo '```'
-                  tail -n 30 "$f"
-                  echo '```'
-                  echo "</details>"
-                done
-              fi
-            fi
-          } >> "$GITHUB_STEP_SUMMARY"
 
       - name: Comment gate result on PR
         if: always() && github.event_name == 'pull_request' && steps.ab.outputs.gate_md != ''
@@ -349,43 +128,12 @@ jobs:
         run: |
           gh pr comment "${{ github.event.pull_request.number }}" --body-file "${{ steps.ab.outputs.gate_md }}"
 
-      # Scheduled failure alerting is handled by the alert-on-failure job below
-      # (perf-2 consuming ci-8's schedule-failure-issue composite action).
-
       - name: Enforce gate
         if: always()
         run: |
           status="${{ steps.ab.outputs.status }}"
           if [[ "$status" != "0" ]]; then
-            echo "::error::warp A/B budget gate failed (exit $status). See the step summary / PR comment / gate.md artifact." >&2
+            echo "::error::warp A/B budget gate failed (exit $status). See the PR comment / gate.md artifact." >&2
             exit "$status"
           fi
           echo "warp A/B budget gate passed."
-
-  alert-on-failure:
-    name: Alert on scheduled failure
-    needs: [warp-ab]
-    # `always()` is required: without it this job is skipped when a needed
-    # job fails. Alerts only for scheduled (nightly) runs (backlog#1149
-    # ci-8); PR and manual dispatch failures are already watched by a human.
-    # `cancelled` is included alongside `failure` on purpose: a job that hits
-    # timeout-minutes ends as `cancelled`, and the 2026-07-11..07-14 nightly
-    # timeouts went silent precisely because the guard was failure-only. The
-    # composite action already reports cancelled/timed-out jobs in the issue
-    # body. (Scheduled runs get a unique concurrency group with
-    # cancel-in-progress off, so a cancellation here means a timeout/manual
-    # abort, never a superseding run.)
-    if: >-
-      always() && github.event_name == 'schedule' &&
-      (contains(needs.*.result, 'failure') || contains(needs.*.result, 'cancelled'))
-    runs-on: ubuntu-latest
-    timeout-minutes: 10
-    permissions:
-      contents: read
-      issues: write
-    steps:
-      - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
-      - name: Open or update failure-tracking issue
-        uses: ./.github/actions/schedule-failure-issue
-        with:
-          github-token: ${{ secrets.GITHUB_TOKEN }}

.github/workflows/performance.yml

@@ -0,0 +1,146 @@
+# Copyright 2024 RustFS Team
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+name: Performance Testing
+
+on:
+  push:
+    branches: [ main ]
+    paths:
+      - "**/*.rs"
+      - "**/Cargo.toml"
+      - "**/Cargo.lock"
+      - ".github/workflows/performance.yml"
+  workflow_dispatch:
+    inputs:
+      profile_duration:
+        description: "Profiling duration in seconds"
+        required: false
+        default: "120"
+        type: string
+
+permissions:
+  contents: read
+
+env:
+  CARGO_TERM_COLOR: always
+  RUST_BACKTRACE: 1
+
+jobs:
+  performance-profile:
+    name: Performance Profiling
+    runs-on: sm-standard-2
+    timeout-minutes: 30
+    env:
+      FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
+
+      - name: Setup Rust environment
+        uses: ./.github/actions/setup
+        with:
+          rust-version: nightly
+          cache-shared-key: perf-${{ hashFiles('**/Cargo.lock') }}
+          cache-save-if: ${{ github.ref == 'refs/heads/main' }}
+          github-token: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Install additional nightly components
+        run: rustup component add llvm-tools-preview
+
+      - name: Install samply profiler
+        uses: taiki-e/cache-cargo-install-action@7447f04c51f2ba27ca35e7f1e28fab848c5b3ba7 # v2
+        with:
+          tool: samply
+
+      - name: Configure kernel for profiling
+        run: echo '1' | sudo tee /proc/sys/kernel/perf_event_paranoid
+
+      - name: Prepare test environment
+        run: |
+          # Create test volumes
+          for i in {0..4}; do
+            mkdir -p ./target/volume/test$i
+          done
+
+          # Set environment variables
+          echo "RUSTFS_VOLUMES=./target/volume/test{0...4}" >> $GITHUB_ENV
+          echo "RUST_LOG=rustfs=info,ecstore=info,s3s=info,iam=info,rustfs-obs=info" >> $GITHUB_ENV
+
+      - name: Verify console static assets
+        run: |
+          # Console static assets are already embedded in the repository
+          echo "Console static assets size: $(du -sh rustfs/static/)"
+          echo "Console static assets are embedded via rust-embed, no external download needed"
+
+      - name: Build with profiling optimizations
+        run: |
+          RUSTFLAGS="-C force-frame-pointers=yes -C debug-assertions=off --cfg tokio_unstable" \
+          cargo +nightly build --profile profiling -p rustfs --bins
+
+      - name: Run performance profiling
+        id: profiling
+        run: |
+          DURATION="${{ github.event.inputs.profile_duration || '120' }}"
+          echo "Running profiling for ${DURATION} seconds..."
+
+          timeout "${DURATION}s" samply record \
+            --output samply-profile.json \
+            ./target/profiling/rustfs ${RUSTFS_VOLUMES} || true
+
+          if [ -f "samply-profile.json" ]; then
+            echo "profile_generated=true" >> $GITHUB_OUTPUT
+            echo "Profile generated successfully"
+          else
+            echo "profile_generated=false" >> $GITHUB_OUTPUT
+            echo "::warning::Profile data not generated"
+          fi
+
+      - name: Upload profile data
+        if: steps.profiling.outputs.profile_generated == 'true'
+        uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
+        with:
+          name: performance-profile-${{ github.run_number }}
+          path: samply-profile.json
+          retention-days: 30
+
+  benchmark:
+    name: Benchmark Tests
+    runs-on: sm-standard-2
+    timeout-minutes: 45
+    env:
+      FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
+
+      - name: Setup Rust environment
+        uses: ./.github/actions/setup
+        with:
+          rust-version: stable
+          cache-shared-key: bench-${{ hashFiles('**/Cargo.lock') }}
+          github-token: ${{ secrets.GITHUB_TOKEN }}
+          cache-save-if: ${{ github.ref == 'refs/heads/main' }}
+
+      - name: Run benchmarks
+        run: |
+          cargo bench --package ecstore --bench comparison_benchmark -- --output-format json | \
+          tee benchmark-results.json
+
+      - name: Upload benchmark results
+        uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
+        with:
+          name: benchmark-results-${{ github.run_number }}
+          path: benchmark-results.json
+          retention-days: 7

.github/workflows/schedule-failure-alert-drill.yml

@@ -1,62 +0,0 @@
-# Copyright 2024 RustFS Team
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-# Manual drill for the scheduled-failure alerting path (backlog#1149 ci-8).
-#
-# Forces a job failure and then runs .github/actions/schedule-failure-issue
-# through the exact `needs` + `always() && contains(needs.*.result,
-# 'failure')` wiring used by the real consumers (e2e-s3tests, mint, fuzz,
-# performance-ab). Dispatch it twice to verify both the issue-creation and
-# the dedupe-comment paths, then close the resulting
-# "[scheduled-failure] Schedule Failure Alert Drill" issue.
-#
-# The run itself is expected to end red (the forced failure); only the
-# alert-on-failure job result matters.
-
-name: Schedule Failure Alert Drill
-
-on:
-  workflow_dispatch:
-
-permissions:
-  contents: read
-
-jobs:
-  forced-failure:
-    name: Forced failure
-    runs-on: ubuntu-latest
-    timeout-minutes: 5
-    steps:
-      - name: Fail on purpose
-        run: |
-          echo "Deliberate failure so alert-on-failure exercises the real consumer wiring."
-          exit 1
-
-  alert-on-failure:
-    name: Alert on scheduled failure
-    needs: [forced-failure]
-    # Mirrors the consumer wiring, minus the schedule-event guard (this
-    # workflow is dispatch-only by design).
-    if: always() && contains(needs.*.result, 'failure')
-    runs-on: ubuntu-latest
-    timeout-minutes: 10
-    permissions:
-      contents: read
-      issues: write
-    steps:
-      - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
-      - name: Open or update failure-tracking issue
-        uses: ./.github/actions/schedule-failure-issue
-        with:
-          github-token: ${{ secrets.GITHUB_TOKEN }}

.github/workflows/star-history.yml

@@ -1,26 +0,0 @@
-name: Star History
-
-on:
-  schedule:
-    - cron: "17 3 * * *"
-  workflow_dispatch:
-
-permissions:
-  contents: write
-
-concurrency:
-  group: star-history
-  cancel-in-progress: false
-
-jobs:
-  update:
-    runs-on: ubuntu-latest
-    steps:
-      - uses: overtrue/repo-visuals-action@72f34d24769ff5d341956da2f23952594ef2f1e2 # v1.3.0
-        with:
-          github-token: ${{ github.token }}
-          output-branch: star-history
-          output-path: .
-          chart-style: gradient
-          animate: "true"
-          contributors: "true"

.gitignore

@@ -55,8 +55,8 @@ docs/*
 !docs/architecture/**
 !docs/operations/
 !docs/operations/**
-!docs/testing/
-!docs/testing/**
+!docs/superpowers/
+!docs/superpowers/**
 docs/heal-scanner-logging-governance.md
 docs/benchmark/rustfs-target-bench/
 docs/benchmark/*.md

AGENTS.md

@@ -14,12 +14,12 @@ If repo-level instructions conflict, follow the nearest file and keep behavior a
 
 ## Execution Discipline
 
-- Read the relevant existing code, tests, and local guidance before changing behavior. For new helpers or test setup, that read includes `crates/utils`, `crates/common`, and the touched crate's own `test_util`/fixtures (see Reuse Before You Write).
+- Read the relevant existing code, tests, and local guidance before changing behavior.
 - State assumptions when they affect the implementation or verification path.
 - If a task has multiple plausible interpretations, list the options briefly and choose the narrowest reasonable path; ask when the ambiguity would make the change risky.
 - For multi-step work, keep the plan minimal and tied to verifiable outcomes.
 - Avoid redundant file reads, repeated commands, and unnecessary exploratory work once enough context is available.
-- A good result is a minimal diff with clear assumptions, no over-engineering, and independent verification that survives Adversarial Validation (below).
+- A good result is a minimal diff with clear assumptions, no over-engineering, and independent verification.
 
 ## Communication and Language
 
@@ -41,27 +41,14 @@ If repo-level instructions conflict, follow the nearest file and keep behavior a
 - Do not refactor existing code only to make it easier to unit test.
 - Keep fixes narrowly aligned with the requested behavior; avoid semantic-adjacent rewrites while touching sensitive paths.
 - Keep code elegant, concise, and direct. Prefer minimal, readable implementations over over-engineering and excessive abstraction. Use comments to clarify non-obvious intent and invariants, not to compensate for unclear code.
-- Do not write comments that narrate what the next line does, restate a signature, or describe the change you just made — that commentary belongs in the PR description, not the code. Required invariant comments — lock ordering, `SAFETY`, unwrap justification, `#[allow(dead_code)]` rationale, `RUSTFS_COMPAT_TODO` — are never narration.
 - Mention unrelated issues when useful, but do not fix them as part of a narrow task.
 
-## Reuse Before You Write
+## Constant and String Usage
 
-Search for an existing implementation before writing a new one; extend what exists instead of duplicating it:
-
-- **Helpers and utilities** (path/string handling, hashing, retry, env parsing, IO wrappers): check `ls crates/utils/src` first — file names map to operations (`retry.rs`, `envs.rs`, `hash.rs`, `path.rs`, `string.rs`, `io.rs`) — plus `crates/common` (shared structures/globals), then `rg -i 'fn \w*<term>' crates/utils/src crates/common/src <touched-crate>/src` for signatures. Helpers are snake_case: a full-text single-word grep over a large crate drowns you and a multi-word phrase returns nothing. Reimplementing an existing workspace helper — or hand-rolling what `std`, `tokio`, or an existing workspace dependency already provides — is a review finding, not a style preference.
-- **Reuse requires matching semantics, not a matching name**: before adopting a helper, check its normalization (`clean` resolves `.`/`..` — never apply it to raw S3 object keys), error type, backoff/deadline behavior, and durability gating against the call site. When semantics differ, a new narrowly-named helper with a comment naming the rejected lookalike is the correct outcome. The inverse also holds: workspace wrappers exist because raw `std`/`tokio` semantics were insufficient (durability gates, retries) — prefer the wrapper over the raw call.
-- **Constants and fixed tokens** (protocol labels, error identifiers, header keys, event names, metric names, command tags): search for existing constants/enums that already represent the same semantic value and reuse them. If a value is truly new, define one local constant near related logic; never scatter the literal across sites. When changing existing behavior, align naming and format with the established constants.
-- **Test scaffolding**: reuse existing test utilities and fixtures (the touched crate's own `test_util` module and `tests/fixtures`, or `crates/test-utils`) instead of writing new setup code — run `rg -l '<fn-under-test>' <crate>/src <crate>/tests` before writing a test. A new test must pin a failure mode no existing test covers. Near-duplicate means same code path AND same poison-value class: this repo's boundary companions (n==max vs max+1, absent vs empty vs nil UUID bytes, MetaObject vs MetaDeleteMarker) are distinct by definition and must all be written.
-
-## Necessary Code Only
-
-Net-new code — files, types, branches, comments — is cost to justify, not progress:
-
-- Validate at the trust boundary — untrusted client input, bytes read from disk, RPC payloads, config (see Serde Safety and Cross-Cutting Domain Invariants) — then trust the type: do not re-check what the type system or a validated upstream layer already guarantees, and cite the establishing check (`file:line`) when the guarantee is not obvious.
-- The exception is load-bearing: a value that crossed a persistence, RPC, or version boundary is never guaranteed by the code on the other side — a peer may be older or buggy, disk bytes may be corrupt — so the Cross-Cutting Domain Invariant patterns apply at every consumer, and re-checks immediately before a destructive action (delete, overwrite, quorum decision) stay. Deleting an existing guard is a behavior change requiring adversarial review, not cleanup.
-- Every new branch needs a nameable trigger: a concrete input, state, or failure that reaches it — for boundary-crossing values, corrupt or stale persisted/peer data is always nameable. If you cannot name one, do not write the branch. If the case is truly unreachable, encode the invariant in the type; where that is impossible, return a typed internal error (fail closed). `debug_assert!` is acceptable only for pure internal arithmetic on values that never crossed a disk/RPC/config boundary — never as the sole guard on decoded or peer-supplied data.
-- Never substitute a default where the value is required (e.g. `unwrap_or_default()` on metadata that must exist) — that converts corruption into a wrong answer. Return the typed error instead: explicit failure over implicit success.
-- Attach error context once, at the layer where it is actionable: re-wrapping equivalent context at every hop is noise, and expanding a fallible chain into nested `match` blocks where `?` or a combinator suffices is a finding. Never add context by converting a typed error into a generic variant below an error-aggregation or quorum layer (`reduce_errs` classifies by variant equality) — context there belongs in a `tracing` event, not the error value.
+- Before introducing new string literals, search for existing constants/enums that already represent the same semantic value.
+- Reuse existing constants for protocol labels, error identifiers, header keys, event names, metric names, command tags, and similar fixed tokens.
+- If a new string is truly unique, define a local constant near related logic and avoid scattering the literal across multiple sites.
+- When changing existing behavior, keep naming and format consistency by aligning with established project constants.
 
 ## Sources of Truth
 
@@ -72,35 +59,17 @@ Net-new code — files, types, branches, comments — is cost to justify, not pr
 - High-level architecture and crate map: `ARCHITECTURE.md`
 - Migration guardrails, readiness contracts, support matrices:
   `docs/architecture/README.md` (routes by audience)
-- Shared agent skills (all tools): `.agents/skills/` — each `SKILL.md` carries
-  a frontmatter `description` stating when it applies. Scan the descriptions
-  before starting a task and follow any skill that matches, even if your tool
-  does not auto-load skills:
-  `grep -m1 '^description:' .agents/skills/*/SKILL.md`
-  Claude Code reads them through the `.claude/skills` symlink; add new skills
-  to `.agents/skills/` only, never as separate copies per tool
+- Historical implementation plans and trackers: `docs/superpowers/plans/`
+- Shared agent skills (all tools): `.agents/skills/` — Claude Code reads them
+  through the `.claude/skills` symlink; add new skills to `.agents/skills/`
+  only, never as separate copies per tool
 
 Avoid duplicating long crate lists or command matrices in instruction files.
 Reference the source files above instead.
 
-Do not commit planning-type documents — one-shot implementation/optimization
-plans, task trackers, migration-progress ledgers, phase/PR templates,
-issue-scoped benchmark-result snapshots or optimization conclusions, or
-agent-generated working notes (e.g. anything a `superpowers`/scratch workflow
-produces). Keep that work in the issue tracker or your local worktree, not in
-the repository. Only durable reference — the architecture set under
-`docs/architecture/`, repeatable operational runbooks under `docs/operations/`,
-and the test-suite references under `docs/testing/` — belongs in version
-control; `.gitignore` ignores everything else under `docs/` by default, so a new
-plan file will not be tracked unless someone force-adds it — don't.
-`scripts/check_no_planning_docs.sh` (wired into `make pre-commit`/`pre-pr` and
-CI) fails the build if anything is committed under `docs/superpowers/`, even via
-`git add -f`.
-
 ## Verification Before PR
 
 Convert changes into independently verifiable outcomes. Prefer focused tests for behavior changes and run the relevant checks before declaring completion.
-Non-exempt changes must also pass Adversarial Validation (next section) before the checks below count as completion.
 
 For code changes, run and pass the following before opening a PR:
 
@@ -125,119 +94,19 @@ Before pushing code changes, make sure formatting is clean:
 - Run `cargo fmt --all --check` and ensure no files are modified unexpectedly.
 
 If `make` is unavailable, run the equivalent checks defined under `.config/make/`.
-Documentation-only or instruction-only changes are exempt from the verification commands above (including the `.config/make/` equivalents), though any locally installed git pre-commit hooks may still run on commit unless explicitly skipped.
+Documentation-only or instruction-only changes are exempt from the verification commands above (including the `.config/make/` equivalents), though any installed git pre-commit hooks (for example, from `make setup-hooks`) may still run on commit unless explicitly skipped.
 After build-based verification completes, clean generated build artifacts before wrapping up to avoid unnecessary disk usage.
 Do not open a PR with code changes when the required checks fail.
-Make a failing check pass by fixing the cause, never by weakening the gate:
-do not loosen or skip a guard script, add entries to a baseline or allowance
-list, suppress a lint with `#[allow]`, mark a failing test `#[ignore]`, or
-delete or relax a failing assertion to get green. If a check itself is wrong,
-change it deliberately and state the rationale in the PR.
-
-For flaky tests, do not paper over them with retries. Follow the flake policy
-in [docs/testing/README.md](docs/testing/README.md) (open an issue within 24h,
-quarantine with an issue link, fix or delete within 30 days); the local
-`default` nextest profile never retries.
-
-## Adversarial Validation (Default On)
-
-Every non-exempt output (see Risk tiers) — code change, bug fix, or
-design/solution proposal — passes multi-role adversarial review before it
-counts as done.
-Author confidence is not evidence: each role's job is to refute the change,
-not to bless it.
-
-### Risk tiers
-
-Pick the tier from the riskiest file touched; when in doubt, pick the higher.
-
-- **Exempt:** docs/comments/instruction-only changes, formatting, typos with
-  no runtime surface. Skip this section.
-- **Mechanical:** pure renames, file moves, test-only or tooling changes —
-  correctness and simplicity adversaries only.
-- **Standard (the default):** any change that affects behavior.
-- **High risk:** touches locking, erasure coding, quorum/heal, replication,
-  multipart, RPC, lifecycle/tiering, metadata formats (`xl.meta`),
-  persistence/fsync, IAM/KMS/auth, on-disk or on-wire formats, or
-  S3 API-visible behavior.
-
-### Roles
-
-Run each applicable role as an independent pass over the final diff (or
-proposal text) — parallel reviewer agents where the tooling supports them,
-otherwise sequential passes that each start fresh from the diff and the
-nearest scoped `AGENTS.md`, discarding the writing session's assumptions.
-Each role either produces findings or reports "attacked X, Y, Z — no break
-found"; a bare pass is not a result. Repo-specific attack probes for every
-role live in `.agents/skills/adversarial-validation/` — run them, they
-encode this repo's shipped bugs.
-
-- **Correctness adversary** — construct a concrete input/state/interleaving
-  that yields wrong output, data loss, or a crash. Probe error paths and edge
-  values (empty, nil UUID, zero-length, quorum−1, missing version).
-- **Simplicity adversary** — same behavior, less code. Hunt the materially smaller or more idiomatic diff (see Change Style for Existing Logic, Reuse Before You Write, and Necessary Code Only): reimplemented workspace helpers, one-caller extractions, rewrites where an in-place edit suffices, defensive branches with no nameable trigger, redundant error wrapping, near-duplicate tests, narration comments. A smaller diff achieving identical behavior is a finding, reported with the concrete replacement; forced reuse of a helper with mismatched semantics is equally a finding.
-- **Security reviewer** — authn/authz bypass, injection, secret leakage,
-  untrusted deserialization (see Serde Safety), path traversal, timing leaks.
-- **Concurrency/durability reviewer** — lock ordering, races, cancellation,
-  partial failure, retry/idempotency, crash and power-loss ordering.
-- **Compatibility reviewer** — S3 API surface, MinIO interop, on-disk and
-  on-wire formats, mixed-version upgrade/downgrade paths.
-- **Performance reviewer** — allocation and cloning on hot paths, lock hold
-  time across IO, sync or CPU-heavy work on async runtime threads, added
-  fsync/flush outside the durability gate, hot-path logging noise. A
-  measurable regression on a per-request or per-object path is a finding.
-- **Test-coverage skeptic** — for each claimed behavior, name the test that
-  fails if the change is reverted; then name a changed line that could be
-  wrong while all tests stay green — if one exists, coverage is insufficient.
-  A missing test is a finding, not a note.
-
-Standard tier: correctness adversary + simplicity adversary + test-coverage
-skeptic, plus every role whose domain the diff touches (async or
-shared-state code → concurrency; parsing of untrusted input → security;
-public crate API shape → compatibility; per-request or per-object hot paths
-→ performance).
-High risk: all seven roles.
-
-### Protocol
-
-1. A finding states a concrete failure scenario (input/state → wrong
-   outcome) or names a missing test, with severity and file:line. "Looks
-   risky" is not a finding.
-2. Resolve every finding: fix it, or rebut it with evidence — a test, a
-   traced code path, or a cited invariant. Restated intent and "unlikely"
-   are not rebuttals.
-3. After non-trivial fixes, re-run the roles whose domain the fix touched.
-4. For proposals with no diff, roles attack assumptions, failure modes,
-   migration/rollback, and testability instead — including the simplest
-   rejected alternative and the blast radius when the design fails.
-
-### Exit criteria
-
-- Every applicable role has run; every finding is fixed or rebutted with
-  evidence.
-- Every behavior change has a test that fails without it.
-- The Verification Before PR gates pass — adversarial review supplements
-  those gates, never replaces them.
-- High risk only: record a one-line verdict per role in the PR description.
 
 ## Git and PR Baseline
 
 - Use feature branches based on the latest `main`.
-- Assume other agent sessions work this repository concurrently. Never commit
-  in a shared checkout; do all work on a dedicated feature branch, preferably
-  in a dedicated worktree.
-- Immediately before branching, fetch `origin/main` and branch from it;
-  confirm the target issue is not already fixed there before writing code.
 - Follow Conventional Commits, with subject length <= 72 characters.
 - Keep PR title and description in English.
 - Use `.github/pull_request_template.md` and keep all section headings.
 - Use `N/A` for non-applicable template sections.
 - Include verification commands in the PR description.
-- When using `gh pr create`/`gh pr edit`, write the markdown body to a file
-  and pass `--body-file`; multiline inline `--body` is unsafe — backticks and
-  shell expansion can corrupt content or trigger unintended commands.
-  Pattern: `cat > /tmp/pr_body.md <<'EOF' ... EOF`, then
-  `--body-file /tmp/pr_body.md` (keep the file outside the checkout).
+- When using `gh pr create`/`gh pr edit`, use `--body-file` instead of inline `--body` for multiline markdown.
 - Do not include the literal sequence `\n` in any GitHub issue, pull request, or discussion comment.
 - Do not hard-wrap prose in PR/issue/discussion bodies; write each paragraph as a
   single line and let it reflow. GitHub renders single newlines inside a paragraph
@@ -269,23 +138,11 @@ cargo run -p rustfs-filemeta --example dump_fileinfo -- "/path/to/file/xl.meta"
 - When `deny_unknown_fields` is impractical (backward compatibility), at minimum log unknown fields at `warn` level.
 - Never use `#[serde(default)]` on security-critical fields without explicit validation of the resulting value.
 
-## Cross-Cutting Domain Invariants
-
-- Write internal object metadata under **both** `x-rustfs-internal-<suffix>`
-  and `x-minio-internal-<suffix>` keys (MinIO interop). Use the helpers in
-  `crates/utils/src/http/metadata_compat.rs` (`get_bytes` prefers the RustFS
-  key); never write only one of the two.
-- Read binary UUID metadata defensively:
-  `.and_then(|v| Uuid::from_slice(&v).ok()).filter(|u| !u.is_nil())` —
-  absent, empty, and nil all mean "no value", never `Uuid::nil()`.
-- A remote-tier version of `None`/`""` means the tier bucket is unversioned:
-  send **no** `versionId` on tier GET/DELETE.
-
 ## Naming Conventions
 
 - Follow Rust API Guidelines for naming: `SCREAMING_SNAKE_CASE` for statics and constants, `snake_case` for functions and variables, `PascalCase` for types.
 - Do not use camelCase or Hungarian notation (e.g., `globalDeploymentIDPtr` → `GLOBAL_DEPLOYMENT_ID`).
-- If existing code violates naming conventions, do not widen the violation in new code. Do not rename existing symbols as part of an unrelated task; mention the violation instead (see Change Style for Existing Logic).
+- If existing code violates naming conventions, do not widen the violation in new code. Fix opportunistically when touching the surrounding area.
 
 ## Scoped Guidance in This Repository
 

ARCHITECTURE.md

@@ -41,7 +41,7 @@ The repository is a Cargo workspace with a flat `crates/` layout:
 
 ```
 rustfs/                      # Workspace root (virtual manifest)
-├── rustfs/                  # Main binary + library crate
+├── rustfs/                  # Main binary + library crate (75K lines)
 │   └── src/
 │       ├── main.rs          # Entry point, startup sequence
 │       ├── lib.rs           # Module tree root
@@ -53,7 +53,7 @@ rustfs/                      # Workspace root (virtual manifest)
 │       ├── config/          # CLI args, config parsing, workload profiles
 │       └── ...
 ├── crates/                  # library crates (authoritative list: Cargo.toml [workspace].members)
-│   ├── ecstore/             # Erasure-coded storage engine
+│   ├── ecstore/             # Erasure-coded storage engine (⚠️ 87K lines)
 │   ├── rio/                 # Reader I/O pipeline (encrypt, compress, hash)
 │   ├── io-core/             # Zero-copy I/O, scheduling, buffer pool
 │   ├── io-metrics/          # I/O metrics collection
@@ -83,25 +83,124 @@ A request flows **downward** through the layers. No layer should reach upward
 
 ### Crate Reference
 
-`Cargo.toml` is the authoritative workspace membership and `cargo tree` is the
-authoritative dependency graph. This overview deliberately avoids line-count
-and dependency-depth snapshots because both quickly become stale during
-refactors.
+> Depth levels, line counts, and crate counts in this section are a
+> point-in-time snapshot and drift with refactors. Treat them as orders of
+> magnitude; `Cargo.toml` and `cargo tree` are the source of truth.
+
+Crates are organized in a dependency DAG with 9 depth levels (0 = leaf, 8 = top):
+
+```
+Depth 0 — LEAF (no internal deps):
+  appauth, checksums, config, credentials, crypto, io-metrics,
+  madmin, s3-common, workers, zip
+
+Depth 1:
+  io-core (→ io-metrics)
+  policy (→ config, credentials, crypto)
+  utils                                   (historical → config edge removed; now effectively leaf)
+
+Depth 2:
+  concurrency, filemeta, keystone, kms, lock, obs,
+  signer, targets, trusted-proxies
+
+Depth 3:
+  common                                  (historical → filemeta/madmin edges removed; now effectively leaf)
+
+Depth 4:
+  object-capacity, protos, rio
+
+Depth 5 — CORE:
+  ecstore (16 internal deps, 11 dependents — the architectural heart)
+
+Depth 6:
+  audit, heal, iam, metrics, notify, s3select-api, scanner
+
+Depth 7:
+  object-io, protocols, s3select-query
+
+Depth 8 — TOP:
+  rustfs (35 internal deps — the binary, depends on almost everything)
+```
 
 #### By Domain
 
-| Domain | Current workspace crates | Responsibility |
-|--------|--------------------------|----------------|
-| Foundation | `checksums`, `common`, `config`, `data-usage`, `utils` | Shared configuration, data-usage models, utilities, and checksums. |
-| I/O and storage | `concurrency`, `ecstore`, `filemeta`, `heal`, `io-core`, `io-metrics`, `lifecycle`, `lock`, `object-capacity`, `object-data-cache`, `replication`, `rio`, `rio-v2`, `scanner`, `storage-api` | Erasure-coded object storage, metadata, recovery, lifecycle, replication, locking, cache, and I/O pipelines. |
-| Security and identity | `credentials`, `crypto`, `iam`, `keystone`, `kms`, `policy`, `security-governance`, `signer`, `tls-runtime`, `trusted-proxies` | Credentials, authentication, authorization, encryption, key management, TLS, and security contracts. |
-| Protocols and contracts | `extension-schema`, `madmin`, `protos`, `protocols`, `s3-ops`, `s3-types`, `s3select-api`, `s3select-query` | Admin, inter-node, S3, S3 Select, and optional protocol contracts. |
-| Operations and integration | `audit`, `notify`, `obs`, `targets`, `zip` | Auditing, observability, event delivery, notification targets, and archive support. |
-| Test support | `e2e_test`, `test-utils` | End-to-end validation and shared test bootstrap utilities. |
+**Core Infrastructure:**
 
-The `rustfs` binary crate composes these libraries into the running server.
-`ecstore` remains the storage engine at the architectural center; its internal
-module split is tracked under `docs/architecture/`.
+| Crate | Lines | Purpose |
+|-------|-------|---------|
+| `config` | 3.3K | Configuration types and environment parsing |
+| `utils` | 8.7K | Pure utilities (paths, compression, network, retry) |
+| `common` | 4.4K | Shared runtime state, globals, data usage types, metrics |
+| `madmin` | 5.5K | Admin API request/response types |
+
+**I/O Pipeline:**
+
+| Crate | Lines | Purpose |
+|-------|-------|---------|
+| `io-core` | 6.5K | Zero-copy I/O, buffer pool, direct I/O, scheduling, backpressure |
+| `io-metrics` | 4.5K | I/O operation metrics and counters |
+| `rio` | 6.9K | Composable reader chain (encrypt → compress → hash → limit) |
+| `object-io` | 2.4K | High-level object read/write using rio + ecstore |
+| `concurrency` | 1.8K | Concurrency control wrappers over io-core |
+
+**Storage Engine:**
+
+| Crate | Lines | Purpose |
+|-------|-------|---------|
+| `ecstore` | 87K | ⚠️ Erasure-coded storage: disks, pools, buckets, replication, lifecycle |
+| `filemeta` | 10K | File/object metadata types and versioning |
+| `checksums` | 732 | Checksum computation |
+| `lock` | 7.1K | Distributed lock manager |
+| `heal` | 5.9K | Data healing / bitrot repair |
+| `scanner` | 5.4K | Background data usage scanner |
+| `object-capacity` | 2.5K | Capacity tracking and management |
+
+**Security & Auth:**
+
+| Crate | Lines | Purpose |
+|-------|-------|---------|
+| `crypto` | 1.6K | Encryption primitives |
+| `credentials` | 713 | Credential types (access key / secret key) |
+| `signer` | 1.4K | S3 v4 request signing |
+| `iam` | 9.0K | Identity and access management |
+| `policy` | 8.8K | Policy engine (S3 bucket/IAM policies) |
+| `kms` | 8.1K | Key management service integration |
+| `keystone` | 1.9K | OpenStack Keystone auth |
+| `appauth` | 143 | Application-level auth tokens |
+
+**Protocol & API:**
+
+| Crate | Lines | Purpose |
+|-------|-------|---------|
+| `protos` | 5.7K | Protobuf/gRPC definitions for inter-node RPC |
+| `protocols` | 18K | FTP/FTPS, WebDAV, Swift API support |
+| `s3-common` | 738 | Shared S3 types |
+| `s3select-api` | 1.9K | S3 Select interface |
+| `s3select-query` | 3.6K | S3 Select query engine |
+
+**Observability:**
+
+| Crate | Lines | Purpose |
+|-------|-------|---------|
+| `metrics` | 8.4K | Prometheus metric collectors |
+| `io-metrics` | 4.5K | I/O-specific metrics |
+| `obs` | 5.6K | OpenTelemetry tracing and telemetry |
+| `audit` | 2.4K | Audit logging |
+
+**Events:**
+
+| Crate | Lines | Purpose |
+|-------|-------|---------|
+| `notify` | 5.5K | Event notification system |
+| `targets` | 3.2K | Notification targets (Kafka, AMQP, webhook, etc.) |
+
+**Other:**
+
+| Crate | Lines | Purpose |
+|-------|-------|---------|
+| `trusted-proxies` | 4.0K | Trusted proxy / IP forwarding |
+| `zip` | 986 | ZIP archive support for bulk downloads |
+| `workers` | 136 | Simple worker abstraction |
 
 ## Architecture Invariants
 
@@ -113,7 +212,7 @@ module split is tracked under `docs/architecture/`.
    No upward imports.
 
 2. **Leaf crates have zero internal dependencies.** `config`, `credentials`, `crypto`,
-   `io-metrics`, and `madmin` should depend only on external crates.
+   `io-metrics`, `madmin`, `s3-common` should depend only on external crates.
    - ✅ RESOLVED: the historical `utils → config` and `common → filemeta`/`madmin`
      edges were removed; do not reintroduce them (see Known Structural Issues).
 

CHANGELOG.md

@@ -9,14 +9,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Fixed
 - **Helm Ingress**: `customAnnotations` are now merged with class-specific annotations (nginx/traefik) instead of being ignored when `ingress.className` is set.
-- **Per-pool erasure parity**: Erasure parity (STANDARD and reduced-redundancy) is now resolved independently for every pool instead of reusing the first pool's value. A heterogeneous topology — for example a 4-drive pool plus a 2-drive pool created during expansion — previously inherited the first pool's parity and could resolve to zero data shards in the smaller pool, panicking Reed-Solomon construction on write. Automatic parity now resolves per pool (for example `2+2` in the 4-drive pool and `1+1` in the 2-drive pool). Fixes #4801.
 
 ### Added
-- **NATS JetStream Publish Path**: Opt-in at-least-once delivery for the NATS notify and audit targets. A NATS Core publish flushes to the connection without awaiting a broker acknowledgement, so an event can be lost across a broker restart or a reconnect after the send queue has already cleared it. A queued event now clears only after the JetStream `PublishAck`, so bucket notifications survive those interruptions. Off by default and byte-identical to the NATS Core path when disabled.
-  - Three configuration keys per target: `JETSTREAM_ENABLE`, `JETSTREAM_STREAM_NAME`, and `JETSTREAM_ACK_TIMEOUT_SECS`, under the `RUSTFS_NOTIFY_NATS_` and `RUSTFS_AUDIT_NATS_` prefixes
-  - Durable store-and-forward with a stable dedup id sent as the `Nats-Msg-Id` header, so a replay after a crash is collapsed by the server duplicate window
-  - Pre-flight stream validation, and a bounded failed-events store (count and TTL). Only a non-retryable rejection is recorded in the failed-events store. A retryable condition keeps the entry on the live queue until it is delivered
-  - Operator guide at `docs/operations/nats-jetstream.md`
 - **OpenStack Keystone Authentication Integration**: Full support for OpenStack Keystone authentication via X-Auth-Token headers
   - Tower-based middleware (`KeystoneAuthLayer`) self-contained within `rustfs-keystone` crate
   - Task-local storage for async-safe credential passing between middleware and auth handlers
@@ -39,7 +33,6 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Changed
 - **HTTP Server Stack**: Integrated `KeystoneAuthLayer` middleware from `rustfs-keystone` crate into service stack (positioned after ReadinessGateLayer)
-- **Storage-class validation on startup (upgrade note)**: A persisted explicit storage class (`RUSTFS_STORAGE_CLASS_STANDARD` / `RUSTFS_STORAGE_CLASS_RRS`, for example `EC:2`) is now validated against the actual per-pool drive counts at startup and rejected when a pool cannot satisfy it. This is fail-closed and correct, but a cluster that persisted a storage class larger than a small or heterogeneous pool can hold (for example `EC:2` alongside a 2-drive pool), which earlier releases accepted and silently resolved to an invalid layout, will now refuse to start after upgrade. To recover, unset `RUSTFS_STORAGE_CLASS_STANDARD` so the server derives a valid per-pool default automatically, or set it to a value every pool can satisfy.
 - **IAMAuth**: Enhanced `get_secret_key()` to return empty secret for Keystone credentials (bypasses signature validation)
 - **Auth Module**: Modified `check_key_valid()` to retrieve Keystone credentials from task-local storage and determine admin status
 - **`StorageBackend` trait**: extended with multipart upload methods (`create_multipart_upload`, `upload_part`, `complete_multipart_upload`, `abort_multipart_upload`) plus `upload_part_copy`. Streaming-upload code path is now available to FTPS, WebDAV, and Swift drivers as well.

CLAUDE.md

@@ -31,14 +31,17 @@ make build-docker BUILD_OS=ubuntu22.04
 - Architecture, layering, crate map: [ARCHITECTURE.md](ARCHITECTURE.md)
 - Migration guardrails & readiness contracts: [docs/architecture/](docs/architecture/README.md)
 - CI gates: `.github/workflows/ci.yml` (source of truth; never copy its steps into docs)
-- Test-layer taxonomy, per-layer entry commands, serial/nextest rules, flake
-  policy: [docs/testing/README.md](docs/testing/README.md)
 - Tier/ILM transition debugging (xl.meta inspection, versionId tracing):
   [docs/operations/tier-ilm-debugging.md](docs/operations/tier-ilm-debugging.md)
 
 ## Domain conventions worth knowing up front
 
-Repo-wide domain invariants (dual internal metadata keys, defensive UUID
-reads, unversioned tier buckets) live in [AGENTS.md](AGENTS.md) under
-"Cross-Cutting Domain Invariants" — read them before touching metadata or
-tiering code.
+- Internal object metadata is written under **both** `x-rustfs-internal-<suffix>`
+  and `x-minio-internal-<suffix>` keys for MinIO interop
+  (`crates/utils/src/http/metadata_compat.rs`; `get_bytes` prefers the RustFS
+  key). Never write only one of the two.
+- Binary metadata values (UUIDs) must be read defensively:
+  `.and_then(|v| Uuid::from_slice(&v).ok()).filter(|u| !u.is_nil())` —
+  absent/empty/nil all mean "no value", not `Uuid::nil()`.
+- A remote-tier version of `None`/`""` means the tier bucket is unversioned:
+  send **no** `versionId` on tier GET/DELETE.

CONTRIBUTING.md

@@ -8,9 +8,9 @@ This guide covers the local development environment and the checks expected befo
 
 **MANDATORY**: All code must be properly formatted before committing. This project enforces strict formatting standards to maintain code consistency and readability.
 
-#### Verification Requirements
+#### Pre-commit Requirements
 
-Before submitting your changes for review, you **MUST**:
+Before every commit, you **MUST**:
 
 1. **Format your code**:
 
@@ -47,59 +47,33 @@ make fmt
 # Check if code is properly formatted
 make fmt-check
 
-# Run clippy checks (all targets, all features, -D warnings)
-make clippy-check
+# Run clippy checks
+make clippy
 
-# Fast workspace compilation check (excludes e2e_test)
-make quick-check
+# Run compilation check
+make check
 
-# Full compilation check (cargo check --all-targets)
-make compilation-check
-
-# Run tests (shell script tests + workspace tests + doc tests)
+# Run tests
 make test
 
-# Fast pre-commit gate — see below for exactly what it runs
+# Run all pre-commit checks (format + clippy + check + test)
 make pre-commit
 
-# Full pre-PR gate (pre-commit gates + clippy + tests)
-make pre-pr
+# Setup git hooks (one-time setup)
+make setup-hooks
 ```
 
-> `make test` requires [cargo-nextest](https://nexte.st) (CI runs it and only nextest honours `.config/nextest.toml` test-groups). Install it with `cargo install cargo-nextest --locked` or a prebuilt binary (see https://nexte.st/docs/installation/). To run the plain `cargo test` fallback anyway (results not authoritative — serialization semantics differ from CI), set `RUSTFS_ALLOW_CARGO_TEST_FALLBACK=1`.
-
-> For the full test-layer taxonomy (unit / ecstore black-box / e2e / s3s-e2e / S3 compatibility / chaos / fuzz / bench), each layer's entry command, the naming conventions the migration gate depends on, and the serial/nextest rules, see [docs/testing/README.md](docs/testing/README.md).
-
 ### 🔒 Automated Pre-commit Hooks
-#### What `make pre-commit` and `make pre-pr` actually run
 
-`make pre-commit` is the **fast** gate. It runs, in order
-(see `.config/make/pre-commit.mak`):
+This project includes a pre-commit hook that automatically runs before each commit to ensure:
 
-1. `fmt-check` — `cargo fmt --all --check`
-2. `unsafe-code-check` — `./scripts/check_unsafe_code_allowances.sh`
-3. `architecture-migration-check` — `./scripts/check_architecture_migration_rules.sh`
-4. `logging-guardrails-check` — `./scripts/check_logging_guardrails.sh`
-5. `tokio-io-uring-check` — `./scripts/check_no_tokio_io_uring.sh`
-6. `extension-schema-check` — `./scripts/check_extension_schema_boundaries.sh`
-7. `doc-paths-check` — `./scripts/check_doc_paths.sh`
-8. `quick-check` — `cargo check --workspace --exclude e2e_test`
+- ✅ Code is properly formatted (`cargo fmt --all --check`)
+- ✅ No clippy warnings (`cargo clippy --all-targets --all-features -- -D warnings`)
+- ✅ Code compiles successfully (`cargo check --all-targets`)
 
-**`make pre-commit` does NOT run clippy and does NOT run any tests.**
-A green `make pre-commit` is not enough to open a pull request.
+#### Setting Up Pre-commit Hooks
 
-`make pre-pr` is the **full** gate: it runs all of the guard checks above,
-then `clippy-check` (`cargo clippy --all-targets --all-features -- -D warnings`)
-and `test` (shell script tests, workspace tests excluding `e2e_test`, and doc
-tests). Run `make pre-pr` before opening or updating a pull request — this is
-what CI enforces.
-
-### 🔒 Git Pre-commit Hooks (optional)
-
-Git hooks are **not** versioned in this repository, so a fresh clone has no
-active pre-commit hook. If you add your own `.git/hooks/pre-commit` (a good
-choice is a one-liner that runs `make pre-commit`), you can mark it executable
-with:
+Run this command once after cloning the repository:
 
 ```bash
 make setup-hooks
@@ -111,9 +85,6 @@ Or manually:
 chmod +x .git/hooks/pre-commit
 ```
 
-With or without a hook, the expectation is the same: run `make pre-commit`
-before committing and `make pre-pr` before opening a pull request.
-
 ### 📝 Formatting Configuration
 
 The project uses the following rustfmt configuration (defined in `rustfmt.toml`):
@@ -126,7 +97,7 @@ single_line_let_else_max_width = 100
 
 ### 🚫 Commit Prevention
 
-If you set up a pre-commit hook and your code doesn't meet the formatting requirements, the hook will:
+If your code doesn't meet the formatting requirements, the pre-commit hook will:
 
 1. **Block the commit** and show clear error messages
 2. **Provide exact commands** to fix the issues
@@ -148,10 +119,9 @@ Example output when formatting fails:
 
 1. **Make your changes**
 2. **Format your code**: `make fmt` or `cargo fmt --all`
-3. **Run the fast gate**: `make pre-commit` (no clippy, no tests)
+3. **Run pre-commit checks**: `make pre-commit`
 4. **Commit your changes**: `git commit -m "your message"`
-5. **Run the full gate before opening/updating a PR**: `make pre-pr` (clippy + tests)
-6. **Push to your branch**: `git push`
+5. **Push to your branch**: `git push`
 
 ### 🛠️ IDE Integration
 

Cargo.toml

@@ -31,7 +31,6 @@ members = [
     "crates/lifecycle", # Lifecycle rule evaluation contracts
     "crates/kms", # Key Management Service
     "crates/lock", # Distributed locking implementation
-    "crates/log-analyzer", # Offline log fault-analysis core (rustfs diagnose)
     "crates/madmin", # Management dashboard and admin API interface
     "crates/notify", # Notification system for events
     "crates/obs", # Observability utilities
@@ -53,7 +52,6 @@ members = [
     "crates/extension-schema", # Extension schema contracts
     "crates/signer", # client signer
     "crates/storage-api", # Storage API contracts
-    "crates/test-utils", # Shared test bootstrap helpers (dev-dependency only)
     "crates/targets", # Target-specific configurations and utilities
     "crates/trusted-proxies", # Trusted proxies management
     "crates/tls-runtime", # Project-wide TLS runtime foundation
@@ -69,7 +67,7 @@ edition = "2024"
 license = "Apache-2.0"
 repository = "https://github.com/rustfs/rustfs"
 rust-version = "1.96.0"
-version = "1.0.0-beta.10"
+version = "1.0.0-beta.8"
 homepage = "https://rustfs.com"
 description = "RustFS is a high-performance distributed object storage software built using Rust, one of the most popular languages worldwide. "
 keywords = ["RustFS", "Minio", "object-storage", "filesystem", "s3"]
@@ -86,93 +84,91 @@ redundant_clone = "warn"
 
 [workspace.dependencies]
 # RustFS Internal Crates
-rustfs = { path = "./rustfs", version = "1.0.0-beta.10" }
-rustfs-heal = { path = "crates/heal", version = "1.0.0-beta.10" }
-rustfs-audit = { path = "crates/audit", version = "1.0.0-beta.10" }
-rustfs-checksums = { path = "crates/checksums", version = "1.0.0-beta.10" }
-rustfs-common = { path = "crates/common", version = "1.0.0-beta.10" }
-rustfs-data-usage = { path = "crates/data-usage", version = "1.0.0-beta.10" }
-rustfs-config = { path = "./crates/config", version = "1.0.0-beta.10" }
-rustfs-concurrency = { path = "./crates/concurrency", version = "1.0.0-beta.10" }
-rustfs-credentials = { path = "crates/credentials", version = "1.0.0-beta.10" }
-rustfs-crypto = { path = "crates/crypto", version = "1.0.0-beta.10" }
-rustfs-ecstore = { path = "crates/ecstore", version = "1.0.0-beta.10" }
-rustfs-filemeta = { path = "crates/filemeta", version = "1.0.0-beta.10" }
-rustfs-iam = { path = "crates/iam", version = "1.0.0-beta.10" }
-rustfs-keystone = { path = "crates/keystone", version = "1.0.0-beta.10" }
-rustfs-lifecycle = { path = "crates/lifecycle", version = "1.0.0-beta.10" }
-rustfs-kms = { path = "crates/kms", version = "1.0.0-beta.10" }
-rustfs-lock = { path = "crates/lock", version = "1.0.0-beta.10" }
-rustfs-madmin = { path = "crates/madmin", version = "1.0.0-beta.10" }
-rustfs-notify = { path = "crates/notify", version = "1.0.0-beta.10" }
-rustfs-io-metrics = { path = "crates/io-metrics", version = "1.0.0-beta.10" }
-rustfs-io-core = { path = "crates/io-core", version = "1.0.0-beta.10" }
-rustfs-object-capacity = { path = "crates/object-capacity", version = "1.0.0-beta.10" }
-rustfs-object-data-cache = { path = "crates/object-data-cache", version = "1.0.0-beta.10" }
-rustfs-log-analyzer = { path = "crates/log-analyzer", version = "1.0.0-beta.10" }
-rustfs-obs = { path = "crates/obs", version = "1.0.0-beta.10" }
-rustfs-policy = { path = "crates/policy", version = "1.0.0-beta.10" }
-rustfs-protos = { path = "crates/protos", version = "1.0.0-beta.10" }
-rustfs-protocols = { path = "crates/protocols", version = "1.0.0-beta.10" }
-rustfs-replication = { path = "crates/replication", version = "1.0.0-beta.10" }
-rustfs-rio = { path = "crates/rio", version = "1.0.0-beta.10" }
-rustfs-rio-v2 = { path = "crates/rio-v2", version = "1.0.0-beta.10" }
-rustfs-s3-types = { path = "crates/s3-types", version = "1.0.0-beta.10" }
-rustfs-s3-ops = { path = "crates/s3-ops", version = "1.0.0-beta.10" }
-rustfs-s3select-api = { path = "crates/s3select-api", version = "1.0.0-beta.10" }
-rustfs-s3select-query = { path = "crates/s3select-query", version = "1.0.0-beta.10" }
-rustfs-scanner = { path = "crates/scanner", version = "1.0.0-beta.10" }
-rustfs-security-governance = { path = "crates/security-governance", version = "1.0.0-beta.10" }
-rustfs-extension-schema = { path = "crates/extension-schema", version = "1.0.0-beta.10" }
-rustfs-signer = { path = "crates/signer", version = "1.0.0-beta.10" }
-rustfs-storage-api = { path = "crates/storage-api", version = "1.0.0-beta.10" }
-rustfs-trusted-proxies = { path = "crates/trusted-proxies", version = "1.0.0-beta.10" }
-rustfs-targets = { path = "crates/targets", version = "1.0.0-beta.10" }
-rustfs-test-utils = { path = "crates/test-utils", version = "1.0.0-beta.10" }
-rustfs-tls-runtime = { path = "crates/tls-runtime", version = "1.0.0-beta.10" }
-rustfs-utils = { path = "crates/utils", version = "1.0.0-beta.10" }
-rustfs-zip = { path = "./crates/zip", version = "1.0.0-beta.10" }
+rustfs = { path = "./rustfs", version = "1.0.0-beta.8" }
+rustfs-heal = { path = "crates/heal", version = "1.0.0-beta.8" }
+rustfs-audit = { path = "crates/audit", version = "1.0.0-beta.8" }
+rustfs-checksums = { path = "crates/checksums", version = "1.0.0-beta.8" }
+rustfs-common = { path = "crates/common", version = "1.0.0-beta.8" }
+rustfs-data-usage = { path = "crates/data-usage", version = "1.0.0-beta.8" }
+rustfs-config = { path = "./crates/config", version = "1.0.0-beta.8" }
+rustfs-concurrency = { path = "./crates/concurrency", version = "1.0.0-beta.8" }
+rustfs-credentials = { path = "crates/credentials", version = "1.0.0-beta.8" }
+rustfs-crypto = { path = "crates/crypto", version = "1.0.0-beta.8" }
+rustfs-ecstore = { path = "crates/ecstore", version = "1.0.0-beta.8" }
+rustfs-filemeta = { path = "crates/filemeta", version = "1.0.0-beta.8" }
+rustfs-iam = { path = "crates/iam", version = "1.0.0-beta.8" }
+rustfs-keystone = { path = "crates/keystone", version = "1.0.0-beta.8" }
+rustfs-lifecycle = { path = "crates/lifecycle", version = "1.0.0-beta.8" }
+rustfs-kms = { path = "crates/kms", version = "1.0.0-beta.8" }
+rustfs-lock = { path = "crates/lock", version = "1.0.0-beta.8" }
+rustfs-madmin = { path = "crates/madmin", version = "1.0.0-beta.8" }
+rustfs-notify = { path = "crates/notify", version = "1.0.0-beta.8" }
+rustfs-io-metrics = { path = "crates/io-metrics", version = "1.0.0-beta.8" }
+rustfs-io-core = { path = "crates/io-core", version = "1.0.0-beta.8" }
+rustfs-object-capacity = { path = "crates/object-capacity", version = "1.0.0-beta.8" }
+rustfs-object-data-cache = { path = "crates/object-data-cache", version = "1.0.0-beta.8" }
+rustfs-obs = { path = "crates/obs", version = "1.0.0-beta.8" }
+rustfs-policy = { path = "crates/policy", version = "1.0.0-beta.8" }
+rustfs-protos = { path = "crates/protos", version = "1.0.0-beta.8" }
+rustfs-protocols = { path = "crates/protocols", version = "1.0.0-beta.8" }
+rustfs-replication = { path = "crates/replication", version = "1.0.0-beta.8" }
+rustfs-rio = { path = "crates/rio", version = "1.0.0-beta.8" }
+rustfs-rio-v2 = { path = "crates/rio-v2", version = "1.0.0-beta.8" }
+rustfs-s3-types = { path = "crates/s3-types", version = "1.0.0-beta.8" }
+rustfs-s3-ops = { path = "crates/s3-ops", version = "1.0.0-beta.8" }
+rustfs-s3select-api = { path = "crates/s3select-api", version = "1.0.0-beta.8" }
+rustfs-s3select-query = { path = "crates/s3select-query", version = "1.0.0-beta.8" }
+rustfs-scanner = { path = "crates/scanner", version = "1.0.0-beta.8" }
+rustfs-security-governance = { path = "crates/security-governance", version = "1.0.0-beta.8" }
+rustfs-extension-schema = { path = "crates/extension-schema", version = "1.0.0-beta.8" }
+rustfs-signer = { path = "crates/signer", version = "1.0.0-beta.8" }
+rustfs-storage-api = { path = "crates/storage-api", version = "1.0.0-beta.8" }
+rustfs-trusted-proxies = { path = "crates/trusted-proxies", version = "1.0.0-beta.8" }
+rustfs-targets = { path = "crates/targets", version = "1.0.0-beta.8" }
+rustfs-tls-runtime = { path = "crates/tls-runtime", version = "1.0.0-beta.8" }
+rustfs-utils = { path = "crates/utils", version = "1.0.0-beta.8" }
+rustfs-zip = { path = "./crates/zip", version = "1.0.0-beta.8" }
 
 # Async Runtime and Networking
 async-channel = "2.5.0"
-async_zip = { default-features = false, version = "0.0.18" }
-mysql_async = { default-features = false, version = "0.37" }
+async_zip = { version = "0.0.18", default-features = false, features = ["tokio", "deflate"] }
+mysql_async = { version = "0.37", default-features = false, features = ["default-rustls", "tracing"] }
 async-compression = { version = "0.4.42" }
 async-recursion = "1.1.1"
-async-trait = "0.1.91"
-async-nats = "0.50.0"
+async-trait = "0.1.89"
+async-nats = "0.49.1"
 axum = "0.8.9"
-futures = "0.3.33"
-futures-core = "0.3.33"
+futures = "0.3.32"
+futures-core = "0.3.32"
 futures-lite = "2.6.1"
-futures-util = "0.3.33"
-pollster = "1.0.1"
-pulsar = { default-features = false, version = "6.8.0" }
-lapin = { default-features = false, version = "4.10.0" }
-hyper = { version = "1.11.0" }
-hyper-rustls = { default-features = false, version = "0.27.9" }
-hyper-util = { version = "0.1.20" }
+futures-util = "0.3.32"
+pollster = "0.4.0"
+pulsar = { version = "6.8.0", default-features = false, features = ["tokio-rustls-runtime", "telemetry"] }
+lapin = { version = "4.10.0", default-features = false, features = ["tokio", "rustls", "rustls--aws_lc_rs"] }
+hyper = { version = "1.10.1", features = ["http2", "http1", "server"] }
+hyper-rustls = { version = "0.27.9", default-features = false, features = ["native-tokio", "http1", "tls12", "logging", "http2", "aws-lc-rs", "webpki-roots"] }
+hyper-util = { version = "0.1.20", features = ["tokio", "server-auto", "server-graceful", "tracing"] }
 http = "1.4.2"
-http-body = "1.1.0"
-http-body-util = "0.1.4"
+http-body = "1.0.1"
+http-body-util = "0.1.3"
 minlz = "1.2.3"
-reqwest = "0.13.4"
+reqwest = { version = "0.13.4", default-features = false, features = ["rustls", "charset", "http2", "system-proxy", "stream", "json", "blocking", "query", "form"] }
 rustfs-kafka-async = { version = "1.2.0" }
-socket2 = { version = "0.6.5" }
-tokio = { version = "1.53.1" }
-tokio-rustls = { default-features = false, version = "0.26.4" }
+socket2 = { version = "0.6.4", features = ["all"] }
+tokio = { version = "1.52.3", features = ["fs", "rt-multi-thread"] }
+tokio-rustls = { version = "0.26.4", default-features = false, features = ["logging", "tls12", "aws-lc-rs"] }
 tokio-stream = { version = "0.1.18" }
 tokio-test = "0.4.5"
-tokio-util = { version = "0.7.18" }
-tonic = { version = "0.14.6" }
+tokio-util = { version = "0.7.18", features = ["io", "compat"] }
+tonic = { version = "0.14.6", features = ["gzip", "deflate"] }
 tonic-prost = { version = "0.14.6" }
 tonic-prost-build = { version = "0.14.6" }
-tower = { version = "0.5.3" }
-tower-http = { version = "0.7.0" }
+tower = { version = "0.5.3", features = ["timeout"] }
+tower-http = { version = "0.7.0", features = ["cors"] }
 
 # Serialization and Data Formats
 apache-avro = "0.21.0"
-bytes = { version = "1.12.1" }
+bytes = { version = "1.12.0", features = ["serde"] }
 bytesize = "2.4.2"
 byteorder = "1.5.0"
 flatbuffers = "25.12.19"
@@ -181,8 +177,8 @@ prost = "0.14.4"
 quick-xml = "0.41.0"
 rmp = { version = "0.8.15" }
 rmp-serde = { version = "1.3.1" }
-serde = { version = "1.0.229" }
-serde_json = { version = "1.0.151" }
+serde = { version = "1.0.228", features = ["derive"] }
+serde_json = { version = "1.0.150", features = ["raw_value"] }
 serde_urlencoded = "0.7.1"
 
 # Cryptography and Security
@@ -190,130 +186,131 @@ serde_urlencoded = "0.7.1"
 # matching stable releases are not available yet, while previous stable lines
 # have incompatible APIs. Keep them exact-pinned and monitor upstream for stable
 # releases.
-aes-gcm = { version = "=0.11.0" }
+aes-gcm = { version = "=0.11.0", features = ["rand_core"] }
 argon2 = { version = "=0.6.0-rc.8" }
 blake2 = "=0.11.0-rc.6"
 chacha20poly1305 = { version = "=0.11.0" }
 crc-fast = "1.10.0"
 hmac = { version = "0.13.0" }
-jsonwebtoken = { version = "10.4.0" }
-openidconnect = { default-features = false, version = "4.0" }
+jsonwebtoken = { version = "10.4.0", features = ["aws_lc_rs"] }
+openidconnect = { version = "4.0", default-features = false, features = ["accept-rfc3339-timestamps"] }
 pbkdf2 = "0.13.0"
 rsa = { version = "=0.10.0-rc.18" }
-rustls = { default-features = false, version = "0.23.42" }
+rustls = { version = "0.23.41", default-features = false, features = ["aws-lc-rs", "logging", "tls12", "prefer-post-quantum", "std"] }
 rustls-native-certs = "0.8"
 rustls-pki-types = "1.15.0"
 sha1 = "0.11.0"
 sha2 = "0.11.0"
 subtle = "2.6"
-zeroize = { version = "1.9.0" }
+zeroize = { version = "1.9.0", features = ["derive"] }
 
 # Time and Date
-chrono = { version = "0.4.45" }
+chrono = { version = "0.4.45", features = ["serde"] }
 humantime = "2.4.0"
-jiff = { version = "0.2.34" }
-time = { version = "0.3.54" }
+jiff = { version = "0.2.31", features = ["serde"] }
+time = { version = "0.3.53", features = ["std", "parsing", "formatting", "macros", "serde"] }
 
 # Database
-deadpool-postgres = { version = "0.14" }
-tokio-postgres = { default-features = false, version = "0.7.18" }
+deadpool-postgres = { version = "0.14", features = ["rt_tokio_1"] }
+tokio-postgres = { version = "0.7.18", default-features = false, features = ["runtime", "with-serde_json-1"] }
 tokio-postgres-rustls = "0.14.0"
 
 # Utilities and Tools
-anyhow = "1.0.104"
+anyhow = "1.0.103"
 arc-swap = "1.9.2"
-astral-tokio-tar = "0.6.4"
+astral-tokio-tar = "0.6.3"
 atoi = "3.1.0"
 atomic_enum = "0.3.0"
-aws-config = { version = "1.9.0" }
-aws-credential-types = { version = "1.3.0" }
-aws-sdk-s3 = { default-features = false, version = "1.138.1" }
-aws-smithy-http-client = { default-features = false, version = "1.2.0" }
-aws-smithy-runtime-api = { version = "1.13.0" }
-aws-smithy-types = { version = "1.6.1" }
+aws-config = { version = "1.8.18" }
+aws-credential-types = { version = "1.2.14" }
+aws-sdk-s3 = { version = "1.137.0", default-features = false, features = ["sigv4a", "default-https-client", "rt-tokio"] }
+aws-smithy-http-client = { version = "1.1.13", default-features = false, features = ["default-client", "rustls-aws-lc"] }
+aws-smithy-runtime-api = { version = "1.12.3", features = ["http-1x"] }
+aws-smithy-types = { version = "1.5.0" }
 base64 = "0.22.1"
 base64-simd = "0.8.0"
 brotli = "8.0.4"
-clap = { version = "4.6.3" }
-const-str = { version = "1.1.0" }
+clap = { version = "4.6.1", features = ["derive", "env"] }
+const-str = { version = "1.1.0", features = ["std", "proc"] }
 convert_case = "0.11.0"
-criterion = { version = "0.8" }
+criterion = { version = "0.8", features = ["html_reports"] }
 crossbeam-queue = "0.3.13"
 crossbeam-channel = "0.5.16"
 crossbeam-deque = "0.8.7"
 crossbeam-utils = "0.8.22"
-datafusion = { default-features = false, git = "https://github.com/apache/datafusion.git", rev = "dae03ee062b2abf986de8df12ea82fb1578a2d99" }
+datafusion = { git = "https://github.com/apache/datafusion.git", rev = "dae03ee062b2abf986de8df12ea82fb1578a2d99" }
 derive_builder = "0.20.2"
-enumset = "1.1.14"
+enumset = "1.1.13"
 faster-hex = "0.10.0"
 flate2 = "1.1.9"
 glob = "0.3.3"
-google-cloud-storage = "1.16.0"
-google-cloud-auth = "1.14.0"
-hashbrown = { version = "0.17.1" }
+google-cloud-storage = "1.15.0"
+google-cloud-auth = "1.13.0"
+hashbrown = { version = "0.17.1", features = ["serde", "rayon"] }
 hex = "0.4.3"
 hex-simd = "0.8.0"
 highway = { version = "1.3.0" }
-ipnetwork = { version = "0.21.1" }
+ipnetwork = { version = "0.21.1", features = ["serde"] }
 lazy_static = "1.5.0"
-libc = "0.2.187"
+libc = "0.2.186"
 libsystemd = "0.7.2"
 local-ip-address = "0.6.13"
 memmap2 = "0.9.11"
 lz4 = "1.28.1"
 matchit = "0.9.2"
 md-5 = "0.11.0"
-md5 = "0.8.1"
+md5 = "0.8.0"
 mime_guess = "2.0.5"
-moka = { version = "0.12.15" }
+moka = { version = "0.12.15", features = ["future"] }
 netif = "0.1.6"
 num_cpus = { version = "1.17.0" }
 nvml-wrapper = "0.12.1"
 parking_lot = "0.12.5"
-path-absolutize = "4.0.1"
+path-absolutize = "3.1.1"
 path-clean = "1.0.1"
 percent-encoding = "2.3.2"
 pin-project-lite = "0.2.17"
 pretty_assertions = "1.4.1"
-rand = { version = "0.10.2" }
+rand = { version = "0.10.2", features = ["serde"] }
 ratelimit = "0.10.1"
 rayon = "1.12.0"
-reed-solomon-erasure = { package = "rustfs-erasure-codec", version = "8.0.0" }
+reed-solomon-erasure = { package = "rustfs-erasure-codec", version = "7.0.1", features = ["simd-accel"] }
+#reed-solomon-erasure = { version = "6.0", features = ["simd-accel"], git = "https://github.com/houseme/reed-solomon-erasure",rev = "main" }
 reed-solomon-simd = "3.1.0"
-regex = { version = "1.13.1" }
-rumqttc = { package = "rumqttc-next", version = "0.33.3" }
-redis = { version = "1.4.1" }
-rustix = { version = "1.1.4" }
-rust-embed = { version = "8.12.0" }
+regex = { version = "1.12.4" }
+rumqttc = { package = "rumqttc-next", version = "0.33.2", features = ["websocket"] }
+redis = { version = "1.3.0", features = ["connection-manager", "tokio-rustls-comp", "tls-rustls-insecure"] }
+rustix = { version = "1.1.4", features = ["fs"] }
+rust-embed = { version = "8.11.0" }
 rustc-hash = { version = "2.1.3" }
-s3s = { git = "https://github.com/s3s-project/s3s.git", rev = "a5471625975f5014f7b28eee7e4d801f1b32f529" }
+s3s = { version = "0.14.1", features = ["minio"] }
 serial_test = "3.5.0"
-shadow-rs = { default-features = false, version = "2.0.0" }
+shadow-rs = { version = "2.0.0", default-features = false }
 siphasher = "1.0.3"
-smallvec = { version = "1.15.2" }
+smallvec = { version = "1.15.2", features = ["serde"] }
 smartstring = "1.0.1"
-snap = "1.1.2"
-starshard = { version = "2.2.2" }
-strum = { version = "0.28.0" }
-sysinfo = "0.39.6"
+snap = "1.1.1"
+starshard = { version = "2.2.1", features = ["rayon", "async", "serde"] }
+strum = { version = "0.28.0", features = ["derive"] }
+sysinfo = "0.39.5"
 temp-env = "0.3.6"
 tempfile = "3.27.0"
 test-case = "3.3.1"
-thiserror = "2.0.19"
+thiserror = "2.0.18"
 tracing = { version = "0.1.44" }
 tracing-appender = "0.2.5"
 tracing-error = "0.2.1"
 tracing-opentelemetry = { version = "0.33" }
-tracing-subscriber = { version = "0.3.23" }
+tracing-subscriber = { version = "0.3.23", features = ["env-filter", "time"] }
 transform-stream = "0.3.1"
 url = "2.5.8"
 urlencoding = "2.1.3"
-uuid = { version = "1.24.0" }
+uuid = { version = "1.23.4", features = ["v4", "fast-rng", "macro-diagnostics"] }
 vaultrs = { version = "0.8.0" }
-tar = "0.4.46"
 walkdir = "2.5.0"
+wildmatch = { version = "2.6.1", features = ["serde"] }
 windows = { version = "0.62.2" }
-xxhash-rust = { version = "0.8.17" }
+xxhash-rust = { version = "0.8.16", features = ["xxh64", "xxh3"] }
 zip = "8.6.0"
 zstd = "0.13.3"
 
@@ -321,29 +318,29 @@ zstd = "0.13.3"
 metrics = "0.24.6"
 dial9-tokio-telemetry = "0.3"
 opentelemetry = { version = "0.32.0" }
-opentelemetry-appender-tracing = { version = "0.32.0" }
-opentelemetry-otlp = { version = "0.32.0" }
-opentelemetry_sdk = { version = "0.32.1" }
-opentelemetry-semantic-conventions = { version = "0.32.1" }
+opentelemetry-appender-tracing = { version = "0.32.0", features = ["experimental_span_attributes", "experimental_metadata_attributes"] }
+opentelemetry-otlp = { version = "0.32.0", features = ["gzip-http", "reqwest-rustls"] }
+opentelemetry_sdk = { version = "0.32.1", features = ["rt-tokio"] }
+opentelemetry-semantic-conventions = { version = "0.32.1", features = ["semconv_experimental"] }
 opentelemetry-stdout = { version = "0.32.0" }
-pyroscope = { version = "2.1.0" }
+pyroscope = { version = "2.0.6", features = ["backend-pprof-rs"] }
 
 # FTP and SFTP
-libunftp = { version = "0.23.0" }
+libunftp = { version = "0.23.0", features = ["experimental"] }
 unftp-core = "0.1.0"
-suppaftp = { version = "10.0.1" }
+suppaftp = { version = "10.0.0", features = ["tokio", "tokio-rustls-aws-lc-rs"] }
 rcgen = "0.14.8"
-russh = { version = "0.62.3" }
+russh = { version = "0.62.2", features = ["serde"] }
 russh-sftp = "2.3.0"
 
 # WebDAV
 dav-server = "0.11.0"
 
 # Performance Analysis and Memory Profiling
-mimalloc = "0.1.52"
-hotpath = "0.21.5"
+mimalloc = "0.1"
+hotpath = "0.21"
 # Snapshot testing for output format regression detection
-insta = { version = "1.48" }
+insta = { version = "1.48", features = ["yaml", "json"] }
 
 [workspace.metadata.cargo-shear]
 ignored = ["rustfs"]
@@ -357,16 +354,12 @@ debug = "line-tables-only"
 
 [profile.release]
 opt-level = 3
-lto = "thin"
-codegen-units = 1
-debug = 0
-split-debuginfo = "off"
-strip = "symbols"
 
 [profile.production]
 inherits = "release"
+lto = "fat"
+codegen-units = 1
 
 [profile.profiling]
 inherits = "release"
 debug = true
-strip = "none"

Dockerfile

@@ -12,7 +12,7 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-FROM alpine:3.24.1 AS build
+FROM alpine:3.23.4 AS build
 
 ARG TARGETARCH
 ARG RELEASE=latest
@@ -70,7 +70,7 @@ RUN set -eux; \
     rm -rf rustfs.zip /build/.tmp || true
 
 
-FROM alpine:3.24.1
+FROM alpine:3.23.4
 
 ARG RELEASE=latest
 ARG BUILD_DATE
@@ -88,9 +88,7 @@ LABEL name="RustFS" \
       url="https://rustfs.com" \
       license="Apache-2.0"
 
-# Upgrade base-image packages so published images pick up security fixes
-# (e.g. openssl/libssl3 CVEs) without waiting for a new Alpine point release.
-RUN apk upgrade --no-cache && \
+RUN apk update && \
     apk add --no-cache ca-certificates coreutils curl
 
 COPY --from=build /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/

Dockerfile.decommission-local

@@ -12,7 +12,7 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-FROM rust:1.97-trixie
+FROM rust:1.95-trixie
 
 RUN set -eux; \
     export DEBIAN_FRONTEND=noninteractive; \

Dockerfile.glibc

@@ -12,7 +12,7 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-FROM ubuntu:26.04 AS build
+FROM ubuntu:24.04 AS build
 
 ARG TARGETARCH
 ARG RELEASE=latest
@@ -76,7 +76,7 @@ RUN set -eux; \
     chmod +x /build/rustfs; \
     rm -rf rustfs.zip /build/.tmp || true
 
-FROM ubuntu:26.04
+FROM ubuntu:24.04
 
 ARG RELEASE=latest
 ARG BUILD_DATE
@@ -93,10 +93,7 @@ LABEL name="RustFS" \
       url="https://rustfs.com" \
       license="Apache-2.0"
 
-# Upgrade base-image packages so published images pick up security fixes
-# (e.g. tar/gzip/perl CVEs) without waiting for a new Ubuntu point release.
-RUN apt-get update && apt-get upgrade -y \
-    && apt-get install -y --no-install-recommends \
+RUN apt-get update && apt-get install -y --no-install-recommends \
     ca-certificates \
     curl \
     && rm -rf /var/lib/apt/lists/*

Dockerfile.source

@@ -32,7 +32,7 @@ ARG RUSTFS_BUILD_FEATURES=""
 # -----------------------------
 # Build stage
 # -----------------------------
-FROM rust:1.97-trixie AS builder
+FROM rust:1.95-trixie AS builder
 
 # Re-declare args after FROM
 ARG TARGETPLATFORM
@@ -208,7 +208,7 @@ CMD ["cargo", "run", "--bin", "rustfs", "--"]
 # -----------------------------
 # Runtime stage (Ubuntu minimal)
 # -----------------------------
-FROM ubuntu:26.04
+FROM ubuntu:24.04
 
 ARG BUILD_DATE
 ARG VCS_REF
@@ -223,7 +223,6 @@ LABEL name="RustFS (dev-local)" \
 RUN set -eux; \
     export DEBIAN_FRONTEND=noninteractive; \
     apt-get update; \
-    apt-get upgrade -y; \
     apt-get install -y --no-install-recommends \
     ca-certificates \
     curl \

Makefile

@@ -64,11 +64,10 @@ How to use me:
 
 	🔧 Code Quality:
 		make fmt                                 # Format code
-		make clippy-check                        # Run clippy (all targets, all features, -D warnings)
-		make quick-check                         # Fast workspace compile check (excludes e2e_test)
-		make test                                # Script tests + workspace tests + doc tests
-		make pre-commit                          # Fast gate: fmt-check + guard scripts + quick-check (NO clippy, NO tests)
-		make pre-pr                              # Full pre-PR gate: pre-commit gates + clippy-check + test
+		make clippy                              # Run clippy checks
+		make test                                # Run tests
+		make pre-commit                          # Run fast pre-commit checks
+		make pre-pr                              # Run full pre-PR checks
 
 	🚀 Quick Start:
 		make build                               # Build RustFS binary

README.md

@@ -116,7 +116,7 @@ chown -R 10001:10001 data logs
 docker run -d -p 9000:9000 -p 9001:9001 -v $(pwd)/data:/data -v $(pwd)/logs:/logs rustfs/rustfs:latest
 
 # Using specific version
-docker run -d -p 9000:9000 -p 9001:9001 -v $(pwd)/data:/data -v $(pwd)/logs:/logs rustfs/rustfs:1.0.0-beta.10
+docker run -d -p 9000:9000 -p 9001:9001 -v $(pwd)/data:/data -v $(pwd)/logs:/logs rustfs/rustfs:1.0.0-beta.8
 ```
 
 If you use [podman](https://github.com/containers/podman) instead of docker, you can install the RustFS with the below command
@@ -218,10 +218,7 @@ For scanner pacing, cycle budgets, bitrot cadence, lifecycle transition status,
 and single-node single-disk idle CPU tuning, see
 [Scanner Runtime Controls](docs/operations/scanner-runtime-controls.md). For
 repeatable scanner-pressure validation, see
-[Scanner Benchmark Runbook](docs/operations/scanner-benchmark-runbook.md). For
-drive timeout knobs on slow storage — including the walk stall budget that
-governs `ListObjects` on large prefixes — see
-[Drive Timeout Tuning](docs/operations/drive-timeout-tuning.md).
+[Scanner Benchmark Runbook](docs/operations/scanner-benchmark-runbook.md).
 
 ### 5\. Nix Flake (Option 5)
 
@@ -338,18 +335,12 @@ If you have any questions or need assistance:
 RustFS is a community-driven project, and we appreciate all contributions. Check out the [Contributors](https://github.com/rustfs/rustfs/graphs/contributors) page to see the amazing people who have helped make RustFS better.
 
 <a href="https://github.com/rustfs/rustfs/graphs/contributors">
-<picture>
-  <source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/rustfs/rustfs/star-history/contributors-dark.svg">
-  <img src="https://raw.githubusercontent.com/rustfs/rustfs/star-history/contributors-light.svg" alt="RustFS contributors">
-</picture>
+<img src="https://opencollective.com/rustfs/contributors.svg?width=890&limit=500&button=false" alt="Contributors" />
 </a>
 
 ## Star History
 
-<picture>
-  <source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/rustfs/rustfs/star-history/star-history-dark.svg">
-  <img src="https://raw.githubusercontent.com/rustfs/rustfs/star-history/star-history-light.svg" alt="RustFS star history chart">
-</picture>
+[![Star History Chart](https://api.star-history.com/svg?repos=rustfs/rustfs&type=date&legend=top-left)](https://www.star-history.com/#rustfs/rustfs&type=date&legend=top-left)
 
 ## License
 

README_ZH.md

@@ -113,7 +113,7 @@ chown -R 10001:10001 data logs
 docker run -d -p 9000:9000 -p 9001:9001 -v $(pwd)/data:/data -v $(pwd)/logs:/logs rustfs/rustfs:latest
 
 # 使用指定版本运行
-docker run -d -p 9000:9000 -p 9001:9001 -v $(pwd)/data:/data -v $(pwd)/logs:/logs rustfs/rustfs:1.0.0-beta.10
+docker run -d -p 9000:9000 -p 9001:9001 -v $(pwd)/data:/data -v $(pwd)/logs:/logs rustfs/rustfs:1.0.0-beta.8
 ```
 
 如果您通过绑定挂载启用 TLS 证书目录，也请用同样方式准备该目录：
@@ -247,18 +247,12 @@ rustfs --help
 RustFS 是一个社区驱动的项目，我们感谢所有的贡献。请查看 [贡献者](https://github.com/rustfs/rustfs/graphs/contributors) 页面，看看那些让 RustFS 变得更好的了不起的人们。
 
 <a href="https://github.com/rustfs/rustfs/graphs/contributors">
-<picture>
-  <source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/rustfs/rustfs/star-history/contributors-dark.svg">
-  <img src="https://raw.githubusercontent.com/rustfs/rustfs/star-history/contributors-light.svg" alt="RustFS 贡献者">
-</picture>
+<img src="https://opencollective.com/rustfs/contributors.svg?width=890&limit=500&button=false" alt="Contributors" />
 </a>
 
 ## Star 历史
 
-<picture>
-  <source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/rustfs/rustfs/star-history/star-history-dark.svg">
-  <img src="https://raw.githubusercontent.com/rustfs/rustfs/star-history/star-history-light.svg" alt="RustFS Star 历史图表">
-</picture>
+[![Star History Chart](https://api.star-history.com/svg?repos=rustfs/rustfs&type=date&legend=top-left)](https://www.star-history.com/#rustfs/rustfs&type=date&legend=top-left)
 
 ## 许可证
 

_typos.toml

@@ -45,8 +45,6 @@ nonexisted = "nonexisted"
 consts = "consts"
 # Swift API - company/product names
 Hashi = "Hashi"  # HashiCorp
-# Accept alternate spelling used in parser/XML comments.
-unparseable = "unparseable"
 
 [files]
 extend-exclude = []

build-rustfs.sh

@@ -217,7 +217,7 @@ setup_rust_environment() {
     # Set up environment variables for musl targets
     if [[ "$PLATFORM" == *"musl"* ]]; then
         print_message $YELLOW "Setting up environment for musl target..."
-        export RUSTFLAGS="${RUSTFLAGS:+$RUSTFLAGS }-C target-feature=-crt-static"
+        export RUSTFLAGS="--cfg tokio_unstable -C target-feature=-crt-static"
 
         # For cargo-zigbuild, set up additional environment variables
         if command -v cargo-zigbuild &> /dev/null; then
@@ -434,7 +434,7 @@ build_binary() {
         fi
     else
         # Native compilation
-        build_cmd="RUSTFLAGS='${RUSTFLAGS:+$RUSTFLAGS }-Clink-arg=-lm' cargo build"
+        build_cmd="RUSTFLAGS='--cfg tokio_unstable -Clink-arg=-lm' cargo build"
     fi
 
     if [ "$BUILD_TYPE" = "release" ]; then

crates/audit/Cargo.toml

@@ -27,17 +27,17 @@ categories = ["web-programming", "development-tools", "asynchronous", "api-bindi
 
 [dependencies]
 rustfs-targets = { workspace = true }
-rustfs-config = { workspace = true, features = ["audit", "server-config-model"] }
+rustfs-config = { workspace = true, features = ["audit", "constants", "server-config-model"] }
 rustfs-s3-types = { workspace = true }
-chrono = { workspace = true, features = ["serde"] }
-const-str = { workspace = true, features = ["std", "proc"] }
+chrono = { workspace = true }
+const-str = { workspace = true }
 futures = { workspace = true }
-hashbrown = { workspace = true, features = ["serde", "rayon"] }
+hashbrown = { workspace = true }
 metrics = { workspace = true }
-serde = { workspace = true, features = ["derive"] }
-serde_json = { workspace = true, features = ["raw_value"] }
+serde = { workspace = true }
+serde_json = { workspace = true }
 thiserror = { workspace = true }
-tokio = { workspace = true, features = ["sync", "fs", "rt-multi-thread", "time", "macros"] }
+tokio = { workspace = true, features = ["sync", "fs", "rt-multi-thread", "rt", "time", "macros"] }
 tracing = { workspace = true, features = ["std", "attributes"] }
 
 [dev-dependencies]

crates/audit/src/error.rs

@@ -35,9 +35,6 @@ pub enum AuditError {
     #[error("System already initialized")]
     AlreadyInitialized,
 
-    #[error("Audit system is paused; entry was not accepted")]
-    Paused,
-
     #[error("Storage not available: {0}")]
     StorageNotAvailable(String),
 

crates/audit/src/global.rs

@@ -12,7 +12,7 @@
 //  See the License for the specific language governing permissions and
 //  limitations under the License.
 
-use crate::{AuditEntry, AuditError, AuditResult, AuditSystem, system::AuditTargetMetricSnapshot};
+use crate::{AuditEntry, AuditResult, AuditSystem, system::AuditTargetMetricSnapshot};
 use rustfs_config::server_config::Config;
 use std::sync::{Arc, OnceLock};
 use tracing::{debug, error, trace};
@@ -78,27 +78,10 @@ pub async fn resume_audit_system() -> AuditResult<()> {
 
 /// Dispatch an audit log entry to all targets
 pub async fn dispatch_audit_log(entry: Arc<AuditEntry>) -> AuditResult<()> {
-    let Some(system) = audit_system() else {
-        debug!(
-            event = EVENT_AUDIT_ENTRY_DROPPED,
-            component = LOG_COMPONENT_AUDIT,
-            subsystem = LOG_SUBSYSTEM_GLOBAL,
-            reason = "system_not_initialized",
-            "Dropped audit entry"
-        );
-        return Ok(());
-    };
-
-    // Single state read (backlog#984): the previous code checked `is_running()`
-    // and then called `dispatch()`, which re-read the state. Between the two
-    // reads the system could transition (e.g. Running -> Stopping) and
-    // `dispatch()` would return an error the caller never expected. Let
-    // `dispatch()` be the single authority on the current state and interpret
-    // its "not accepting" errors as a deliberate skip, while still surfacing
-    // real delivery failures (backlog#962).
-    match system.dispatch(entry).await {
-        Ok(()) => Ok(()),
-        Err(AuditError::NotInitialized(_)) | Err(AuditError::Paused) => {
+    if let Some(system) = audit_system() {
+        if system.is_running().await {
+            system.dispatch(entry).await
+        } else {
             trace!(
                 event = EVENT_AUDIT_ENTRY_DROPPED,
                 component = LOG_COMPONENT_AUDIT,
@@ -108,7 +91,15 @@ pub async fn dispatch_audit_log(entry: Arc<AuditEntry>) -> AuditResult<()> {
             );
             Ok(())
         }
-        Err(e) => Err(e),
+    } else {
+        debug!(
+            event = EVENT_AUDIT_ENTRY_DROPPED,
+            component = LOG_COMPONENT_AUDIT,
+            subsystem = LOG_SUBSYSTEM_GLOBAL,
+            reason = "system_not_initialized",
+            "Dropped audit entry"
+        );
+        Ok(())
     }
 }
 

crates/audit/src/pipeline.rs

@@ -32,7 +32,6 @@ const EVENT_AUDIT_BATCH_DISPATCH_COMPLETED: &str = "audit_batch_dispatch_complet
 const EVENT_AUDIT_TARGET_STATE_CHANGED: &str = "audit_target_state_changed";
 const EVENT_AUDIT_REPLAY_DELIVERED: &str = "audit_replay_delivered";
 const EVENT_AUDIT_REPLAY_RETRY_SCHEDULED: &str = "audit_replay_retry_scheduled";
-const EVENT_AUDIT_REPLAY_RETRY_EXHAUSTED: &str = "audit_replay_retry_exhausted";
 const EVENT_AUDIT_REPLAY_DROPPED: &str = "audit_replay_dropped";
 const EVENT_AUDIT_REPLAY_STREAM_STATUS: &str = "audit_replay_stream_status";
 
@@ -46,12 +45,6 @@ impl AuditPipeline {
         Self { registry }
     }
 
-    /// Fans an audit entry out to every configured target concurrently.
-    ///
-    /// Delivery across targets is unordered: the per-target `save()` calls run
-    /// via `join_all` and may complete in any order. Ordering of entries within
-    /// a single target is preserved by that target's own store/queue, not by
-    /// this fan-out.
     pub async fn dispatch(&self, entry: Arc<AuditEntry>) -> AuditResult<()> {
         let start_time = std::time::Instant::now();
 
@@ -197,14 +190,8 @@ impl AuditPipeline {
                         data: (*entry).clone(),
                     };
                     match target.save(Arc::new(entity_target)).await {
-                        Ok(_) => {
-                            success_count += 1;
-                            observability::record_target_success();
-                        }
-                        Err(e) => {
-                            observability::record_target_failure();
-                            errors.push(e);
-                        }
+                        Ok(_) => success_count += 1,
+                        Err(e) => errors.push(e),
                     }
                 }
                 (target.id().to_string(), success_count, errors)
@@ -264,12 +251,6 @@ impl AuditPipeline {
             ));
         }
 
-        // Record the aggregate event outcome so batch dispatch reports the same
-        // observability signal as single dispatch (backlog#984): full success or
-        // partial failure both count as a delivered audit event here, since at
-        // least one target accepted every entry that reached this point.
-        observability::record_audit_success(dispatch_time);
-
         Ok(())
     }
 
@@ -282,7 +263,6 @@ impl AuditPipeline {
                 let delivery = target.delivery_snapshot();
                 AuditTargetMetricSnapshot {
                     failed_messages: delivery.failed_messages,
-                    failed_store_length: delivery.failed_store_length,
                     queue_length: delivery.queue_length,
                     target_id: target.id().to_string(),
                     total_messages: delivery.total_messages,
@@ -465,16 +445,18 @@ impl AuditRuntimeFacade {
                             target.record_final_failure();
                             observability::record_target_failure();
                         }
-                        ReplayEvent::RetryExhausted { detail, key, target } => {
+                        ReplayEvent::RetryExhausted { key, target } => {
                             warn!(
-                                event = EVENT_AUDIT_REPLAY_RETRY_EXHAUSTED,
+                                event = EVENT_AUDIT_REPLAY_DROPPED,
                                 component = LOG_COMPONENT_AUDIT,
                                 subsystem = LOG_SUBSYSTEM_PIPELINE,
                                 target_id = %target.id(),
                                 replay_key = %key,
-                                error = %detail,
-                                "audit replay retry budget exhausted, entry stays queued and retries"
+                                reason = "retry_exhausted",
+                                "audit replay delivery"
                             );
+                            target.record_final_failure();
+                            observability::record_target_failure();
                         }
                         ReplayEvent::UnreadableEntry { key, error, target } => {
                             warn!(
@@ -559,135 +541,3 @@ impl AuditRuntimeFacade {
         self.runtime_adapter.stop_replay_workers(&mut replay_workers).await;
     }
 }
-
-#[cfg(test)]
-mod tests {
-    use super::AuditPipeline;
-    use crate::{AuditEntry, AuditError, AuditRegistry};
-    use async_trait::async_trait;
-    use rustfs_targets::arn::TargetID;
-    use rustfs_targets::store::{Key, Store};
-    use rustfs_targets::target::{EntityTarget, QueuedPayload, QueuedPayloadMeta};
-    use rustfs_targets::{StoreError, Target, TargetError};
-    use std::sync::Arc;
-    use tokio::sync::Mutex;
-
-    /// Mock target whose `save()` outcome is fixed at construction so tests can
-    /// force full-success / full-failure / partial-failure fan-outs.
-    #[derive(Clone)]
-    struct MockTarget {
-        id: TargetID,
-        fail: bool,
-    }
-
-    impl MockTarget {
-        fn new(id: &str, fail: bool) -> Self {
-            Self {
-                id: TargetID::new(id.to_string(), "webhook".to_string()),
-                fail,
-            }
-        }
-    }
-
-    #[async_trait]
-    impl<E> Target<E> for MockTarget
-    where
-        E: rustfs_targets::PluginEvent,
-    {
-        fn id(&self) -> TargetID {
-            self.id.clone()
-        }
-
-        async fn is_active(&self) -> Result<bool, TargetError> {
-            Ok(true)
-        }
-
-        async fn save(&self, _event: Arc<EntityTarget<E>>) -> Result<(), TargetError> {
-            if self.fail {
-                Err(TargetError::Configuration("forced save failure".to_string()))
-            } else {
-                Ok(())
-            }
-        }
-
-        async fn send_raw_from_store(&self, _key: Key, _body: Vec<u8>, _meta: QueuedPayloadMeta) -> Result<(), TargetError> {
-            Ok(())
-        }
-
-        async fn close(&self) -> Result<(), TargetError> {
-            Ok(())
-        }
-
-        fn store(&self) -> Option<&(dyn Store<QueuedPayload, Error = StoreError, Key = Key> + Send + Sync)> {
-            None
-        }
-
-        fn clone_dyn(&self) -> Box<dyn Target<E> + Send + Sync> {
-            Box::new(self.clone())
-        }
-
-        fn is_enabled(&self) -> bool {
-            true
-        }
-    }
-
-    fn pipeline_with(targets: Vec<MockTarget>) -> AuditPipeline {
-        let mut registry = AuditRegistry::new();
-        for target in targets {
-            registry.add_target(target.id.to_string(), Box::new(target));
-        }
-        AuditPipeline::new(Arc::new(Mutex::new(registry)))
-    }
-
-    fn entry() -> Arc<AuditEntry> {
-        Arc::new(AuditEntry::default())
-    }
-
-    // backlog#962: when every target rejects the event it is lost outright, so
-    // dispatch must return Err rather than swallowing the failures as Ok.
-    #[tokio::test]
-    async fn dispatch_returns_err_when_all_targets_fail() {
-        let pipeline = pipeline_with(vec![MockTarget::new("a:webhook", true), MockTarget::new("b:webhook", true)]);
-        let result = pipeline.dispatch(entry()).await;
-        assert!(matches!(result, Err(AuditError::Target(_))), "expected Err, got {result:?}");
-    }
-
-    // A partially-successful fan-out means the entry reached at least one sink,
-    // so dispatch reports success (degradation is logged, not propagated).
-    #[tokio::test]
-    async fn dispatch_returns_ok_on_partial_failure() {
-        let pipeline = pipeline_with(vec![MockTarget::new("ok:webhook", false), MockTarget::new("bad:webhook", true)]);
-        pipeline.dispatch(entry()).await.expect("partial success should return Ok");
-    }
-
-    #[tokio::test]
-    async fn dispatch_returns_ok_when_all_targets_succeed() {
-        let pipeline = pipeline_with(vec![MockTarget::new("a:webhook", false), MockTarget::new("b:webhook", false)]);
-        pipeline.dispatch(entry()).await.expect("all-success should return Ok");
-    }
-
-    // No configured targets is a benign no-op, not a failure.
-    #[tokio::test]
-    async fn dispatch_returns_ok_with_no_targets() {
-        let pipeline = pipeline_with(vec![]);
-        pipeline.dispatch(entry()).await.expect("no targets should return Ok");
-    }
-
-    // backlog#962: dispatch_batch must mirror dispatch and propagate a
-    // whole-batch loss instead of returning Ok.
-    #[tokio::test]
-    async fn dispatch_batch_returns_err_when_all_targets_fail() {
-        let pipeline = pipeline_with(vec![MockTarget::new("a:webhook", true)]);
-        let result = pipeline.dispatch_batch(vec![entry(), entry()]).await;
-        assert!(matches!(result, Err(AuditError::Target(_))), "expected Err, got {result:?}");
-    }
-
-    #[tokio::test]
-    async fn dispatch_batch_returns_ok_when_all_targets_succeed() {
-        let pipeline = pipeline_with(vec![MockTarget::new("a:webhook", false), MockTarget::new("b:webhook", false)]);
-        pipeline
-            .dispatch_batch(vec![entry(), entry()])
-            .await
-            .expect("all-success batch should return Ok");
-    }
-}

crates/audit/src/system.rs

@@ -30,7 +30,6 @@ const EVENT_AUDIT_CONFIG_RELOADED: &str = "audit_config_reloaded";
 #[derive(Debug, Clone, Default, PartialEq, Eq)]
 pub struct AuditTargetMetricSnapshot {
     pub failed_messages: u64,
-    pub failed_store_length: u64,
     pub queue_length: u64,
     pub target_id: String,
     pub total_messages: u64,
@@ -90,20 +89,14 @@ impl AuditSystem {
         registry.create_audit_targets_from_config(config).await
     }
 
-    /// Stops any active replay workers and closes the currently installed
-    /// targets without touching the system state. Lock order is `registry`
-    /// then `stream_cancellers` to stay consistent with every other path that
-    /// holds both locks (see `runtime_status_snapshot`, backlog#961).
-    async fn shutdown_runtime_targets(&self) -> AuditResult<()> {
-        let mut registry = self.registry.lock().await;
-        let mut replay_workers = self.stream_cancellers.write().await;
-        self.runtime_facade()
-            .shutdown_runtime(&mut registry, &mut replay_workers)
-            .await
-    }
-
     async fn clear_runtime_targets(&self) -> AuditResult<()> {
-        self.shutdown_runtime_targets().await?;
+        {
+            let mut registry = self.registry.lock().await;
+            let mut replay_workers = self.stream_cancellers.write().await;
+            self.runtime_facade()
+                .shutdown_runtime(&mut registry, &mut replay_workers)
+                .await?;
+        }
 
         let mut state = self.state.write().await;
         *state = AuditSystemState::Stopped;
@@ -130,16 +123,6 @@ impl AuditSystem {
             "audit system state"
         );
 
-        // Stop-before-start (backlog#970): tear down the existing replay workers
-        // and close the currently installed targets *before* activating the new
-        // set. Activation spawns fresh replay workers for store-backed targets,
-        // so if the old workers were still running they would drain the same
-        // persistent queue concurrently with the new ones and re-deliver
-        // entries. Shutting the old runtime down first keeps at most one active
-        // worker per store across a reload. `replace_targets` performs a second
-        // (now no-op) shutdown before installing, which is idempotent.
-        self.shutdown_runtime_targets().await?;
-
         let activation = self.runtime_facade().activate_targets_with_replay(targets).await;
         self.runtime_facade().replace_targets(activation).await?;
 
@@ -156,30 +139,21 @@ impl AuditSystem {
     /// # Returns
     /// * `AuditResult<()>` - Result indicating success or failure
     pub async fn start(&self, config: Config) -> AuditResult<()> {
-        // Claim the `Starting` transition atomically while holding the write
-        // lock (backlog#978): the previous code released the lock after the
-        // check and re-acquired it later to set `Starting`, so two concurrent
-        // `start()` calls (or `start()` racing `reload`) could both pass the
-        // check and double-activate. Transitioning to `Starting` before
-        // dropping the guard makes a concurrent caller observe `Starting` and
-        // return early instead.
-        {
-            let mut state = self.state.write().await;
+        let state = self.state.write().await;
 
-            match *state {
-                AuditSystemState::Running => {
-                    return Err(AuditError::AlreadyInitialized);
-                }
-                AuditSystemState::Starting => {
-                    warn_audit_state("starting", Some("already_starting"));
-                    return Ok(());
-                }
-                _ => {}
+        match *state {
+            AuditSystemState::Running => {
+                return Err(AuditError::AlreadyInitialized);
             }
-
-            *state = AuditSystemState::Starting;
+            AuditSystemState::Starting => {
+                warn_audit_state("starting", Some("already_starting"));
+                return Ok(());
+            }
+            _ => {}
         }
 
+        drop(state);
+
         info!(
             event = EVENT_AUDIT_SYSTEM_STATE,
             component = LOG_COMPONENT_AUDIT,
@@ -199,7 +173,11 @@ impl AuditSystem {
 
         match self.create_targets_from_config(&config).await {
             Ok(targets) => {
-                // State is already `Starting` (claimed atomically above).
+                {
+                    let mut state = self.state.write().await;
+                    *state = AuditSystemState::Starting;
+                }
+
                 self.commit_runtime_targets(targets, AuditSystemState::Running).await?;
                 info_audit_state("running", None, None);
                 Ok(())
@@ -340,13 +318,7 @@ impl AuditSystem {
         match *state {
             AuditSystemState::Running => {}
             AuditSystemState::Paused => {
-                // Do not silently return Ok while paused (backlog#978): the
-                // entry is neither delivered nor persisted, so reporting success
-                // would corrupt the audit trail. Surface an explicit `Paused`
-                // error and let the caller apply its policy (the global helper
-                // treats this as a deliberate skip; direct API callers can
-                // decide otherwise).
-                return Err(AuditError::Paused);
+                return Ok(());
             }
             _ => {
                 return Err(AuditError::NotInitialized("Audit system is not running".to_string()));
@@ -576,7 +548,6 @@ fn warn_audit_state(state: &str, reason: Option<&str>) {
 #[cfg(test)]
 mod tests {
     use super::{AuditSystem, AuditSystemState};
-    use crate::{AuditEntry, AuditError};
     use async_trait::async_trait;
     use rustfs_targets::ReplayWorkerManager;
     use rustfs_targets::arn::TargetID;
@@ -734,95 +705,4 @@ mod tests {
             .await
             .expect("audit lock paths deadlocked (backlog#961 regression)");
     }
-
-    /// backlog#978: a paused system must not report success while silently
-    /// dropping the entry. `dispatch` should surface an explicit `Paused` error.
-    #[tokio::test]
-    async fn dispatch_while_paused_returns_error_not_ok() {
-        let system = AuditSystem::new();
-        {
-            let mut state = system.state.write().await;
-            *state = AuditSystemState::Paused;
-        }
-
-        let result = system.dispatch(Arc::new(AuditEntry::default())).await;
-        assert!(
-            matches!(result, Err(AuditError::Paused)),
-            "paused dispatch must return Err(Paused), got {result:?}"
-        );
-    }
-
-    /// backlog#978: `start()` now claims the `Starting` transition atomically
-    /// under the state lock, so racing `start()` calls cannot both pass the
-    /// check and double-activate. Hammer concurrent starts and assert the
-    /// workload completes (no deadlock/panic) and converges to a consistent
-    /// final state.
-    #[tokio::test(flavor = "multi_thread", worker_threads = 4)]
-    async fn concurrent_start_does_not_hang_or_double_activate() {
-        use std::time::Duration;
-
-        let system = AuditSystem::new();
-        let mut handles = Vec::new();
-        for _ in 0..8 {
-            let s = system.clone();
-            handles.push(tokio::spawn(async move {
-                // Empty config activates no targets, so a completed start settles
-                // the system back to `Stopped`.
-                let _ = s.start(rustfs_config::server_config::Config(HashMap::new())).await;
-            }));
-        }
-
-        let workload = async {
-            for handle in handles {
-                handle.await.expect("start task panicked");
-            }
-        };
-        tokio::time::timeout(Duration::from_secs(30), workload)
-            .await
-            .expect("concurrent start deadlocked (backlog#978 regression)");
-
-        assert_eq!(system.get_state().await, AuditSystemState::Stopped);
-    }
-
-    /// backlog#970: a reload/commit must tear down the previous replay workers
-    /// and close the old targets before activating the replacement set, so the
-    /// old and new workers never drain the same store concurrently. Seed an old
-    /// target plus a replay worker, commit a new target, and assert the old one
-    /// was closed and its worker stopped while the new one is installed.
-    #[tokio::test]
-    async fn commit_closes_old_targets_before_installing_new() {
-        let system = AuditSystem::new();
-
-        let old = TestTarget::new("old", "webhook");
-        let old_close = Arc::clone(&old.close_calls);
-        {
-            let mut registry = system.registry.lock().await;
-            registry.add_target("old:webhook".to_string(), Box::new(old));
-        }
-        {
-            let mut replay_workers = system.stream_cancellers.write().await;
-            let (cancel_tx, _cancel_rx) = mpsc::channel(1);
-            replay_workers.insert("old:webhook".to_string(), cancel_tx);
-        }
-        {
-            let mut state = system.state.write().await;
-            *state = AuditSystemState::Running;
-        }
-
-        let new = TestTarget::new("new", "webhook");
-        let new_close = Arc::clone(&new.close_calls);
-        system
-            .commit_runtime_targets(vec![Box::new(new)], AuditSystemState::Running)
-            .await
-            .expect("commit should succeed");
-
-        // Old target closed exactly once during the pre-install shutdown.
-        assert_eq!(old_close.load(Ordering::SeqCst), 1);
-        // New target installed and left open.
-        assert_eq!(new_close.load(Ordering::SeqCst), 0);
-        assert_eq!(system.list_targets().await, vec!["new:webhook".to_string()]);
-        // Old replay worker stopped; the store-less new target adds none.
-        assert_eq!(system.runtime_status_snapshot().await.replay_worker_count, 0);
-        assert_eq!(system.get_state().await, AuditSystemState::Running);
-    }
 }

crates/checksums/Cargo.toml

@@ -26,14 +26,13 @@ categories = ["web-programming", "development-tools", "network-programming"]
 documentation = "https://docs.rs/rustfs-checksums/latest/rustfs_checksum/"
 
 [dependencies]
-bytes = { workspace = true, features = ["serde"] }
+bytes = { workspace = true }
 crc-fast = { workspace = true }
 http = { workspace = true }
 base64-simd = { workspace = true }
 md-5 = { workspace = true }
 sha1 = { workspace = true }
 sha2 = { workspace = true }
-xxhash-rust = { workspace = true, features = ["xxh64", "xxh3"] }
 
 [dev-dependencies]
 pretty_assertions = { workspace = true }

crates/checksums/src/error.rs

@@ -36,7 +36,7 @@ impl fmt::Display for UnknownChecksumAlgorithmError {
     fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
         write!(
             f,
-            r#"unknown checksum algorithm "{}", please pass a known algorithm name ("crc32", "crc32c", "crc64nvme", "sha1", "sha256", "sha512", "xxhash3", "xxhash64", "xxhash128")"#,
+            r#"unknown checksum algorithm "{}", please pass a known algorithm name ("crc32", "crc32c", "sha1", "sha256", "md5")"#,
             self.checksum_algorithm
         )
     }

crates/checksums/src/http.rs

@@ -16,20 +16,13 @@ use crate::base64;
 use http::header::{HeaderMap, HeaderValue};
 
 use crate::Crc64Nvme;
-use crate::{
-    CRC_32_C_NAME, CRC_32_NAME, CRC_64_NVME_NAME, Checksum, Crc32, Crc32c, Md5, SHA_1_NAME, SHA_256_NAME, Sha1, Sha256, Sha512,
-    Xxhash3, Xxhash64, Xxhash128,
-};
+use crate::{CRC_32_C_NAME, CRC_32_NAME, CRC_64_NVME_NAME, Checksum, Crc32, Crc32c, Md5, SHA_1_NAME, SHA_256_NAME, Sha1, Sha256};
 
 pub const CRC_32_HEADER_NAME: &str = "x-amz-checksum-crc32";
 pub const CRC_32_C_HEADER_NAME: &str = "x-amz-checksum-crc32c";
 pub const SHA_1_HEADER_NAME: &str = "x-amz-checksum-sha1";
 pub const SHA_256_HEADER_NAME: &str = "x-amz-checksum-sha256";
 pub const CRC_64_NVME_HEADER_NAME: &str = "x-amz-checksum-crc64nvme";
-pub const SHA_512_HEADER_NAME: &str = "x-amz-checksum-sha512";
-pub const XXHASH_3_HEADER_NAME: &str = "x-amz-checksum-xxhash3";
-pub const XXHASH_64_HEADER_NAME: &str = "x-amz-checksum-xxhash64";
-pub const XXHASH_128_HEADER_NAME: &str = "x-amz-checksum-xxhash128";
 
 #[allow(dead_code)]
 pub(crate) static MD5_HEADER_NAME: &str = "content-md5";
@@ -92,30 +85,6 @@ impl HttpChecksum for Sha256 {
     }
 }
 
-impl HttpChecksum for Sha512 {
-    fn header_name(&self) -> &'static str {
-        SHA_512_HEADER_NAME
-    }
-}
-
-impl HttpChecksum for Xxhash3 {
-    fn header_name(&self) -> &'static str {
-        XXHASH_3_HEADER_NAME
-    }
-}
-
-impl HttpChecksum for Xxhash64 {
-    fn header_name(&self) -> &'static str {
-        XXHASH_64_HEADER_NAME
-    }
-}
-
-impl HttpChecksum for Xxhash128 {
-    fn header_name(&self) -> &'static str {
-        XXHASH_128_HEADER_NAME
-    }
-}
-
 impl HttpChecksum for Md5 {
     fn header_name(&self) -> &'static str {
         MD5_HEADER_NAME

crates/checksums/src/lib.rs

@@ -35,10 +35,6 @@ pub const CRC_32_C_NAME: &str = "crc32c";
 pub const CRC_64_NVME_NAME: &str = "crc64nvme";
 pub const SHA_1_NAME: &str = "sha1";
 pub const SHA_256_NAME: &str = "sha256";
-pub const SHA_512_NAME: &str = "sha512";
-pub const XXHASH_3_NAME: &str = "xxhash3";
-pub const XXHASH_64_NAME: &str = "xxhash64";
-pub const XXHASH_128_NAME: &str = "xxhash128";
 pub const MD5_NAME: &str = "md5";
 
 #[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
@@ -47,13 +43,11 @@ pub enum ChecksumAlgorithm {
     #[default]
     Crc32,
     Crc32c,
+    #[deprecated]
+    Md5,
     Sha1,
     Sha256,
     Crc64Nvme,
-    Sha512,
-    Xxhash3,
-    Xxhash64,
-    Xxhash128,
 }
 
 impl FromStr for ChecksumAlgorithm {
@@ -68,16 +62,11 @@ impl FromStr for ChecksumAlgorithm {
             Ok(Self::Sha1)
         } else if checksum_algorithm.eq_ignore_ascii_case(SHA_256_NAME) {
             Ok(Self::Sha256)
+        } else if checksum_algorithm.eq_ignore_ascii_case(MD5_NAME) {
+            // MD5 is now an alias for the default Crc32 since it is deprecated
+            Ok(Self::Crc32)
         } else if checksum_algorithm.eq_ignore_ascii_case(CRC_64_NVME_NAME) {
             Ok(Self::Crc64Nvme)
-        } else if checksum_algorithm.eq_ignore_ascii_case(SHA_512_NAME) {
-            Ok(Self::Sha512)
-        } else if checksum_algorithm.eq_ignore_ascii_case(XXHASH_3_NAME) {
-            Ok(Self::Xxhash3)
-        } else if checksum_algorithm.eq_ignore_ascii_case(XXHASH_64_NAME) {
-            Ok(Self::Xxhash64)
-        } else if checksum_algorithm.eq_ignore_ascii_case(XXHASH_128_NAME) {
-            Ok(Self::Xxhash128)
         } else {
             Err(UnknownChecksumAlgorithmError::new(checksum_algorithm))
         }
@@ -90,12 +79,10 @@ impl ChecksumAlgorithm {
             Self::Crc32 => Box::<Crc32>::default(),
             Self::Crc32c => Box::<Crc32c>::default(),
             Self::Crc64Nvme => Box::<Crc64Nvme>::default(),
+            #[allow(deprecated)]
+            Self::Md5 => Box::<Crc32>::default(),
             Self::Sha1 => Box::<Sha1>::default(),
             Self::Sha256 => Box::<Sha256>::default(),
-            Self::Sha512 => Box::<Sha512>::default(),
-            Self::Xxhash3 => Box::<Xxhash3>::default(),
-            Self::Xxhash64 => Box::<Xxhash64>::default(),
-            Self::Xxhash128 => Box::<Xxhash128>::default(),
         }
     }
 
@@ -104,12 +91,10 @@ impl ChecksumAlgorithm {
             Self::Crc32 => CRC_32_NAME,
             Self::Crc32c => CRC_32_C_NAME,
             Self::Crc64Nvme => CRC_64_NVME_NAME,
+            #[allow(deprecated)]
+            Self::Md5 => MD5_NAME,
             Self::Sha1 => SHA_1_NAME,
             Self::Sha256 => SHA_256_NAME,
-            Self::Sha512 => SHA_512_NAME,
-            Self::Xxhash3 => XXHASH_3_NAME,
-            Self::Xxhash64 => XXHASH_64_NAME,
-            Self::Xxhash128 => XXHASH_128_NAME,
         }
     }
 }
@@ -309,172 +294,12 @@ impl Checksum for Sha256 {
         Self::size()
     }
 }
-
-#[derive(Debug, Default)]
-struct Sha512 {
-    hasher: sha2::Sha512,
-}
-
-impl Sha512 {
-    fn update(&mut self, bytes: &[u8]) {
-        use sha2::Digest;
-        self.hasher.update(bytes);
-    }
-
-    fn finalize(self) -> Bytes {
-        use sha2::Digest;
-        Bytes::copy_from_slice(self.hasher.finalize().as_slice())
-    }
-
-    fn size() -> u64 {
-        use sha2::Digest;
-        sha2::Sha512::output_size() as u64
-    }
-}
-
-impl Checksum for Sha512 {
-    fn update(&mut self, bytes: &[u8]) {
-        Self::update(self, bytes);
-    }
-    fn finalize(self: Box<Self>) -> Bytes {
-        Self::finalize(*self)
-    }
-    fn size(&self) -> u64 {
-        Self::size()
-    }
-}
-
-/// XXH3 (64-bit) hasher with the canonical seed of 0.
-///
-/// The raw digest is a `u64` serialized as 8 big-endian bytes so that the value
-/// matches the server-side (`rustfs-rio`) computation for the same algorithm.
-struct Xxhash3 {
-    hasher: xxhash_rust::xxh3::Xxh3,
-}
-
-impl Default for Xxhash3 {
-    fn default() -> Self {
-        Self {
-            hasher: xxhash_rust::xxh3::Xxh3::new(),
-        }
-    }
-}
-
-impl Xxhash3 {
-    fn update(&mut self, bytes: &[u8]) {
-        self.hasher.update(bytes);
-    }
-
-    fn finalize(self) -> Bytes {
-        Bytes::copy_from_slice(self.hasher.digest().to_be_bytes().as_slice())
-    }
-
-    fn size() -> u64 {
-        8
-    }
-}
-
-impl Checksum for Xxhash3 {
-    fn update(&mut self, bytes: &[u8]) {
-        Self::update(self, bytes)
-    }
-    fn finalize(self: Box<Self>) -> Bytes {
-        Self::finalize(*self)
-    }
-    fn size(&self) -> u64 {
-        Self::size()
-    }
-}
-
-/// XXH3 (128-bit) hasher with the canonical seed of 0.
-///
-/// The raw digest is a `u128` serialized as 16 big-endian bytes.
-struct Xxhash128 {
-    hasher: xxhash_rust::xxh3::Xxh3,
-}
-
-impl Default for Xxhash128 {
-    fn default() -> Self {
-        Self {
-            hasher: xxhash_rust::xxh3::Xxh3::new(),
-        }
-    }
-}
-
-impl Xxhash128 {
-    fn update(&mut self, bytes: &[u8]) {
-        self.hasher.update(bytes);
-    }
-
-    fn finalize(self) -> Bytes {
-        Bytes::copy_from_slice(self.hasher.digest128().to_be_bytes().as_slice())
-    }
-
-    fn size() -> u64 {
-        16
-    }
-}
-
-impl Checksum for Xxhash128 {
-    fn update(&mut self, bytes: &[u8]) {
-        Self::update(self, bytes)
-    }
-    fn finalize(self: Box<Self>) -> Bytes {
-        Self::finalize(*self)
-    }
-    fn size(&self) -> u64 {
-        Self::size()
-    }
-}
-
-/// XXH64 hasher with the canonical seed of 0.
-///
-/// The raw digest is a `u64` serialized as 8 big-endian bytes.
-struct Xxhash64 {
-    hasher: xxhash_rust::xxh64::Xxh64,
-}
-
-impl Default for Xxhash64 {
-    fn default() -> Self {
-        Self {
-            hasher: xxhash_rust::xxh64::Xxh64::new(0),
-        }
-    }
-}
-
-impl Xxhash64 {
-    fn update(&mut self, bytes: &[u8]) {
-        self.hasher.update(bytes);
-    }
-
-    fn finalize(self) -> Bytes {
-        Bytes::copy_from_slice(self.hasher.digest().to_be_bytes().as_slice())
-    }
-
-    fn size() -> u64 {
-        8
-    }
-}
-
-impl Checksum for Xxhash64 {
-    fn update(&mut self, bytes: &[u8]) {
-        Self::update(self, bytes)
-    }
-    fn finalize(self: Box<Self>) -> Bytes {
-        Self::finalize(*self)
-    }
-    fn size(&self) -> u64 {
-        Self::size()
-    }
-}
-
 #[allow(dead_code)]
 #[derive(Debug, Default)]
 struct Md5 {
     hasher: md5::Md5,
 }
 
-#[allow(dead_code)]
 impl Md5 {
     fn update(&mut self, bytes: &[u8]) {
         use md5::Digest;
@@ -619,116 +444,4 @@ mod tests {
             .expect_err("it should error");
         assert_eq!("some invalid checksum algorithm", error.checksum_algorithm());
     }
-
-    #[test]
-    fn test_unknown_algorithm_error_message_lists_supported_algorithms() {
-        let error = "nope".parse::<ChecksumAlgorithm>().expect_err("it should error");
-        let message = error.to_string();
-        assert!(message.contains("crc64nvme"), "message should advertise crc64nvme: {message}");
-        assert!(!message.contains("md5"), "message should not advertise the unsupported md5: {message}");
-    }
-
-    #[test]
-    fn test_md5_is_not_a_supported_checksum_algorithm() {
-        // MD5 is not an accepted S3 checksum algorithm here: parsing it must fail
-        // loudly rather than silently substituting a CRC32 hasher.
-        let error = "md5".parse::<ChecksumAlgorithm>().expect_err("md5 should not parse");
-        assert_eq!("md5", error.checksum_algorithm());
-
-        let error = "MD5".parse::<ChecksumAlgorithm>().expect_err("md5 should not parse");
-        assert_eq!("MD5", error.checksum_algorithm());
-    }
-
-    #[test]
-    fn test_additional_algorithms_parse_and_round_trip() {
-        // The AWS 2026-04 additional checksum algorithms must be recognised
-        // (case-insensitively) and round-trip through as_str().
-        for (name, expected) in [
-            ("sha512", ChecksumAlgorithm::Sha512),
-            ("SHA512", ChecksumAlgorithm::Sha512),
-            ("xxhash3", ChecksumAlgorithm::Xxhash3),
-            ("XXHASH3", ChecksumAlgorithm::Xxhash3),
-            ("xxhash64", ChecksumAlgorithm::Xxhash64),
-            ("xxhash128", ChecksumAlgorithm::Xxhash128),
-        ] {
-            let parsed = name.parse::<ChecksumAlgorithm>().expect("algorithm should parse");
-            assert_eq!(parsed, expected);
-            assert_eq!(expected.as_str().parse::<ChecksumAlgorithm>().unwrap(), expected);
-        }
-    }
-
-    #[test]
-    fn test_unknown_algorithm_never_panics_and_fails_closed() {
-        // Fail-closed contract: an unknown or garbage algorithm name must return
-        // an error instead of panicking or silently substituting another hasher.
-        for name in ["", "xxhash", "sha3", "crc16", "not-a-real-algo", "🦀"] {
-            assert!(name.parse::<ChecksumAlgorithm>().is_err(), "unknown algorithm {name:?} must fail closed");
-        }
-    }
-
-    #[test]
-    fn test_sha512_matches_direct_computation() {
-        use crate::Sha512;
-        use crate::http::SHA_512_HEADER_NAME;
-        use sha2::{Digest, Sha512 as Sha512Ref};
-
-        let mut checksum = Sha512::default();
-        checksum.update(TEST_DATA.as_bytes());
-        let header = Box::new(checksum).headers();
-        let encoded = header.get(SHA_512_HEADER_NAME).expect("sha512 header present");
-        let got = base64_encoded_checksum_to_hex_string(encoded);
-
-        let mut reference = Sha512Ref::new();
-        reference.update(TEST_DATA.as_bytes());
-        let expected = reference.finalize().iter().fold(String::from("0x"), |mut acc, b| {
-            write!(acc, "{b:02X?}").unwrap();
-            acc
-        });
-        assert_eq!(got, expected);
-    }
-
-    #[test]
-    fn test_xxhash3_matches_direct_computation_big_endian_seed0() {
-        use crate::Xxhash3;
-        use xxhash_rust::xxh3::Xxh3;
-
-        let mut checksum = Xxhash3::default();
-        checksum.update(TEST_DATA.as_bytes());
-        let raw = Box::new(checksum).finalize();
-
-        let mut reference = Xxh3::new();
-        reference.update(TEST_DATA.as_bytes());
-        assert_eq!(raw.len(), 8);
-        assert_eq!(&raw[..], reference.digest().to_be_bytes().as_slice());
-    }
-
-    #[test]
-    fn test_xxhash128_matches_direct_computation_big_endian_seed0() {
-        use crate::Xxhash128;
-        use xxhash_rust::xxh3::Xxh3;
-
-        let mut checksum = Xxhash128::default();
-        checksum.update(TEST_DATA.as_bytes());
-        let raw = Box::new(checksum).finalize();
-
-        let mut reference = Xxh3::new();
-        reference.update(TEST_DATA.as_bytes());
-        assert_eq!(raw.len(), 16);
-        assert_eq!(&raw[..], reference.digest128().to_be_bytes().as_slice());
-    }
-
-    #[test]
-    fn test_xxhash64_matches_direct_computation_big_endian_seed0() {
-        use crate::Xxhash64;
-        use xxhash_rust::xxh64::Xxh64;
-
-        let mut checksum = Xxhash64::default();
-        checksum.update(TEST_DATA.as_bytes());
-        let raw = Box::new(checksum).finalize();
-
-        let mut reference = Xxh64::new(0);
-        reference.update(TEST_DATA.as_bytes());
-        assert_eq!(raw.len(), 8);
-        assert_eq!(&raw[..], reference.digest().to_be_bytes().as_slice());
-    }
 }

crates/common/Cargo.toml

@@ -28,14 +28,14 @@ categories = ["web-programming", "development-tools", "data-structures"]
 workspace = true
 
 [dependencies]
-tokio = { workspace = true, features = ["fs", "rt-multi-thread"] }
-tonic = { workspace = true, features = ["gzip", "deflate"] }
-uuid = { workspace = true, features = ["v4", "fast-rng", "macro-diagnostics"] }
-chrono = { workspace = true, features = ["serde"] }
+tokio = { workspace = true }
+tonic = { workspace = true }
+uuid = { workspace = true }
+chrono = { workspace = true }
 metrics = { workspace = true }
-serde = { workspace = true, features = ["derive"] }
+serde = { workspace = true }
 rmp-serde = { workspace = true }
-s3s = { workspace = true, features = ["minio"] }
+s3s = { workspace = true }
 tracing = { workspace = true }
 
 [lib]

crates/common/src/globals.rs

@@ -54,15 +54,6 @@ pub async fn get_global_local_node_name() -> String {
     GLOBAL_LOCAL_NODE_NAME.read().await.clone()
 }
 
-/// Read the local node name without waiting for initialization or a writer.
-pub fn try_get_global_local_node_name() -> Option<String> {
-    GLOBAL_LOCAL_NODE_NAME
-        .try_read()
-        .ok()
-        .map(|name| name.clone())
-        .filter(|name| !name.is_empty())
-}
-
 /// Set the global RustFS initialization time to the current UTC time.
 pub async fn set_global_init_time_now() {
     let now = Utc::now();

crates/common/src/heal_channel.rs

@@ -243,19 +243,6 @@ pub enum HealAdmissionResult {
     Dropped(HealAdmissionDropReason),
 }
 
-/// Admission decision together with the canonical task identifier.
-///
-/// A merged request must return the identifier of the task that already owns
-/// the work instead of exposing the discarded request identifier as a new
-/// client token.
-#[derive(Debug, Clone, PartialEq, Eq)]
-pub struct HealAdmissionReceipt {
-    /// Admission decision for the submitted request.
-    pub result: HealAdmissionResult,
-    /// Canonical identifier of the accepted or merged task.
-    pub task_id: String,
-}
-
 impl HealAdmissionResult {
     pub fn result_label(self) -> &'static str {
         match self {
@@ -395,25 +382,8 @@ pub type HealChannelSender = mpsc::UnboundedSender<HealChannelCommand>;
 /// Heal channel receiver
 pub type HealChannelReceiver = mpsc::UnboundedReceiver<HealChannelCommand>;
 
-/// Canonical-receipt start command kept separate from the legacy public enum.
-#[derive(Debug)]
-pub struct HealReceiptCommand {
-    /// Heal request to admit.
-    pub request: HealChannelRequest,
-    /// Completion channel for the admission receipt.
-    pub response_tx: oneshot::Sender<Result<HealAdmissionReceipt, String>>,
-}
-
-/// Canonical-receipt command receiver.
-pub type HealReceiptReceiver = mpsc::UnboundedReceiver<HealReceiptCommand>;
-
-struct HealChannelSenders {
-    command: HealChannelSender,
-    receipt: mpsc::UnboundedSender<HealReceiptCommand>,
-}
-
 /// Global heal channel sender
-static GLOBAL_HEAL_CHANNEL_SENDERS: OnceLock<HealChannelSenders> = OnceLock::new();
+static GLOBAL_HEAL_CHANNEL_SENDER: OnceLock<HealChannelSender> = OnceLock::new();
 
 type HealResponseSender = broadcast::Sender<HealChannelResponse>;
 
@@ -422,24 +392,17 @@ static GLOBAL_HEAL_RESPONSE_SENDER: OnceLock<HealResponseSender> = OnceLock::new
 
 /// Initialize global heal channel
 pub fn init_heal_channel() -> Result<HealChannelReceiver, &'static str> {
-    let (receiver, receipt_receiver) = init_heal_channels()?;
-    drop(receipt_receiver);
-    Ok(receiver)
-}
-
-/// Initialize the legacy command and canonical-receipt channels atomically.
-pub fn init_heal_channels() -> Result<(HealChannelReceiver, HealReceiptReceiver), &'static str> {
-    let (command, command_receiver) = mpsc::unbounded_channel();
-    let (receipt, receipt_receiver) = mpsc::unbounded_channel();
-    GLOBAL_HEAL_CHANNEL_SENDERS
-        .set(HealChannelSenders { command, receipt })
-        .map_err(|_| "Heal channel sender already initialized")?;
-    Ok((command_receiver, receipt_receiver))
+    let (tx, rx) = mpsc::unbounded_channel();
+    if GLOBAL_HEAL_CHANNEL_SENDER.set(tx).is_ok() {
+        Ok(rx)
+    } else {
+        Err("Heal channel sender already initialized")
+    }
 }
 
 /// Get global heal channel sender
 pub fn get_heal_channel_sender() -> Option<&'static HealChannelSender> {
-    GLOBAL_HEAL_CHANNEL_SENDERS.get().map(|senders| &senders.command)
+    GLOBAL_HEAL_CHANNEL_SENDER.get()
 }
 
 /// Send heal command through global channel
@@ -473,21 +436,6 @@ pub fn subscribe_heal_responses() -> broadcast::Receiver<HealChannelResponse> {
     heal_response_sender().subscribe()
 }
 
-/// Send heal start request and wait for structured admission feedback.
-pub async fn send_heal_request_with_receipt(request: HealChannelRequest) -> Result<HealAdmissionReceipt, String> {
-    let (response_tx, response_rx) = oneshot::channel();
-    let senders = GLOBAL_HEAL_CHANNEL_SENDERS
-        .get()
-        .ok_or_else(|| "Heal channel not initialized".to_string())?;
-    senders
-        .receipt
-        .send(HealReceiptCommand { request, response_tx })
-        .map_err(|err| format!("Failed to send heal receipt command: {err}"))?;
-    response_rx
-        .await
-        .map_err(|e| format!("Failed to receive heal admission response: {e}"))?
-}
-
 /// Send heal start request and wait for structured admission feedback.
 pub async fn send_heal_request_with_admission(request: HealChannelRequest) -> Result<HealAdmissionResult, String> {
     let (response_tx, response_rx) = oneshot::channel();

crates/common/src/last_minute.rs

@@ -144,14 +144,11 @@ impl LastMinuteLatency {
             merged.last_sec = o.last_sec;
         }
 
-        // Both operands must be read in their forwarded form so aged-out
-        // ring-buffer slots stay zeroed: `x` is the forwarded copy of `o`,
-        // and `self` is forwarded in place in the `else` branch above.
         for i in 0..merged.totals.len() {
             merged.totals[i] = AccElem {
-                total: self.totals[i].total.wrapping_add(x.totals[i].total),
-                n: self.totals[i].n.wrapping_add(x.totals[i].n),
-                size: self.totals[i].size.wrapping_add(x.totals[i].size),
+                total: self.totals[i].total + o.totals[i].total,
+                n: self.totals[i].n + o.totals[i].n,
+                size: self.totals[i].size + o.totals[i].size,
             }
         }
         merged
@@ -375,105 +372,6 @@ mod tests {
         assert_eq!(merged.totals[0].total, 30);
     }
 
-    #[test]
-    fn test_last_minute_latency_merge_ages_out_stale_slots_self_newer() {
-        // self.last_sec > o.last_sec branch: `o` is forwarded to self.last_sec,
-        // which zeroes the ring-buffer slots for seconds 1001..=1010, i.e.
-        // indices 41..=50. Data parked in one of those slots is stale and must
-        // be excluded from the merged result.
-        let mut newer = LastMinuteLatency::default();
-        let mut older = LastMinuteLatency::default();
-
-        newer.last_sec = 1010;
-        older.last_sec = 1000;
-
-        // Stale slot: second 1005 -> index 45, cleared by forward_to(1010).
-        let stale_idx = (1005 % 60) as usize;
-        older.totals[stale_idx].total = 111;
-        older.totals[stale_idx].n = 5;
-        older.totals[stale_idx].size = 999;
-
-        // In-window slot: older's own last_sec (1000 -> index 40) is kept.
-        let kept_idx = (1000 % 60) as usize;
-        older.totals[kept_idx].total = 3;
-        older.totals[kept_idx].n = 1;
-        older.totals[kept_idx].size = 30;
-
-        // newer's current data (1010 -> index 50) must survive.
-        let newer_idx = (1010 % 60) as usize;
-        newer.totals[newer_idx].total = 7;
-        newer.totals[newer_idx].n = 1;
-        newer.totals[newer_idx].size = 70;
-
-        let merged = newer.merge(&older);
-
-        assert_eq!(merged.last_sec, 1010);
-        // The stale older slot is aged out -> excluded from the sum.
-        assert_eq!(merged.totals[stale_idx].total, 0);
-        assert_eq!(merged.totals[stale_idx].n, 0);
-        assert_eq!(merged.totals[stale_idx].size, 0);
-        // In-window older data is retained.
-        assert_eq!(merged.totals[kept_idx].total, 3);
-        assert_eq!(merged.totals[kept_idx].n, 1);
-        // newer data is retained.
-        assert_eq!(merged.totals[newer_idx].total, 7);
-    }
-
-    #[test]
-    fn test_last_minute_latency_merge_ages_out_of_window_self_newer() {
-        // self.last_sec > o.last_sec with a full-window gap (>= 60s): all of
-        // `o` is aged out and only `self`'s data remains.
-        let mut newer = LastMinuteLatency::default();
-        let mut older = LastMinuteLatency::default();
-
-        newer.last_sec = 1070;
-        older.last_sec = 1000; // gap of 70 >= 60 -> all of older is aged out
-
-        // 1070 % 60 == 1000 % 60 == 10, so both write the same slot; the fix
-        // must yield exactly newer's value, not newer + stale older.
-        let idx = (1070 % 60) as usize;
-        older.totals[idx].total = 111;
-        older.totals[idx].n = 5;
-        older.totals[idx].size = 999;
-        newer.totals[idx].total = 7;
-        newer.totals[idx].n = 1;
-        newer.totals[idx].size = 70;
-
-        let merged = newer.merge(&older);
-
-        assert_eq!(merged.last_sec, 1070);
-        assert_eq!(merged.totals[idx].total, 7);
-        assert_eq!(merged.totals[idx].n, 1);
-        assert_eq!(merged.totals[idx].size, 70);
-    }
-
-    #[test]
-    fn test_last_minute_latency_merge_ages_out_of_window_o_newer() {
-        // Mirror of the above for the else branch: `self` is older and is
-        // forwarded to o.last_sec, aging out self's out-of-window data.
-        let mut older = LastMinuteLatency::default();
-        let mut newer = LastMinuteLatency::default();
-
-        older.last_sec = 1000;
-        newer.last_sec = 1070; // gap of 70 >= 60 -> all of older (self) is aged out
-
-        let idx = (1070 % 60) as usize; // 1000 % 60 == 1070 % 60 == 10
-        older.totals[idx].total = 111;
-        older.totals[idx].n = 5;
-        older.totals[idx].size = 999;
-        newer.totals[idx].total = 7;
-        newer.totals[idx].n = 1;
-        newer.totals[idx].size = 70;
-
-        let merged = older.merge(&newer);
-
-        assert_eq!(merged.last_sec, 1070);
-        // self's stale data aged out; only newer's data remains.
-        assert_eq!(merged.totals[idx].total, 7);
-        assert_eq!(merged.totals[idx].n, 1);
-        assert_eq!(merged.totals[idx].size, 70);
-    }
-
     #[test]
     fn test_last_minute_latency_window_wraparound() {
         let mut latency = LastMinuteLatency::default();

crates/common/src/metrics.rs

@@ -750,11 +750,6 @@ pub struct Metrics {
     last_scan_cycle_duration_millis: AtomicU64,
     last_scan_cycle_objects_scanned: AtomicU64,
     last_scan_cycle_directories_scanned: AtomicU64,
-    /// Lifetime count of object versions walked by the data scanner, counted
-    /// for every version regardless of whether any lifecycle rule applies.
-    /// This is the honest source for `rustfs_scanner_versions_scanned_total`;
-    /// ILM-checked versions are tracked separately on the `Lifecycle` source.
-    lifetime_versions_scanned: AtomicU64,
     last_scan_cycle_bucket_drive_scans: AtomicU64,
     last_scan_cycle_bucket_drive_failures: AtomicU64,
     last_scan_cycle_yield_events: AtomicU64,
@@ -1111,9 +1106,6 @@ pub struct ScannerMetricsReport {
     pub oldest_active_path_age_seconds: u64,
     pub life_time_ops: HashMap<String, u64>,
     pub life_time_ilm: HashMap<String, u64>,
-    /// Lifetime object versions scanned, independent of ILM configuration.
-    #[serde(default)]
-    pub versions_scanned: u64,
     pub last_minute: ScannerLastMinute,
     pub active_paths: Vec<String>,
     pub current_scan_mode: String,
@@ -1712,7 +1704,6 @@ impl Metrics {
             last_scan_cycle_duration_millis: AtomicU64::new(0),
             last_scan_cycle_objects_scanned: AtomicU64::new(0),
             last_scan_cycle_directories_scanned: AtomicU64::new(0),
-            lifetime_versions_scanned: AtomicU64::new(0),
             last_scan_cycle_bucket_drive_scans: AtomicU64::new(0),
             last_scan_cycle_bucket_drive_failures: AtomicU64::new(0),
             last_scan_cycle_yield_events: AtomicU64::new(0),
@@ -2121,17 +2112,6 @@ impl Metrics {
         self.record_scanner_source_work(source, ScannerSourceWorkUpdate::checked(count));
     }
 
-    /// Record `count` object versions walked by the data scanner.
-    ///
-    /// Every version the scanner visits is counted here, independent of
-    /// lifecycle configuration, so `rustfs_scanner_versions_scanned_total`
-    /// reflects real scan coverage even on clusters with no ILM rules. ILM
-    /// evaluation coverage is recorded separately via the `Lifecycle` source's
-    /// `checked` counter and surfaces as `rustfs_ilm_versions_scanned_total`.
-    pub fn record_scanner_versions_scanned(&self, count: u64) {
-        self.lifetime_versions_scanned.fetch_add(count, Ordering::Relaxed);
-    }
-
     pub fn record_scanner_source_queued(&self, source: ScannerWorkSource, count: u64) {
         self.record_scanner_source_work(source, ScannerSourceWorkUpdate::queued(count));
     }
@@ -2434,18 +2414,6 @@ impl Metrics {
         self.current_scan_cycle_work_active.store(false, Ordering::Relaxed);
     }
 
-    pub fn current_scan_cycle_has_unresolved_heal_work(&self) -> bool {
-        if !self.current_scan_cycle_work_active.load(Ordering::Relaxed) {
-            return false;
-        }
-
-        let source_work = self.scanner_source_work_since(&self.current_scan_cycle_source_work_start_values());
-        [ScannerWorkSource::Heal, ScannerWorkSource::Bitrot]
-            .into_iter()
-            .filter_map(|source| source_work.get(source.index()))
-            .any(|work| work.queued > 0 || work.skipped > 0 || work.failed > 0 || work.missed > 0)
-    }
-
     fn scan_cycle_work_snapshot(&self) -> ScanCycleWorkSnapshot {
         ScanCycleWorkSnapshot {
             objects_scanned: self.lifetime(Metric::ScanObject),
@@ -2727,7 +2695,6 @@ impl Metrics {
             .map(|(disk, state)| format!("{disk}/{}", state.path))
             .collect();
         m.current_scan_mode = self.current_scan_mode().as_str().to_string();
-        m.versions_scanned = self.lifetime_versions_scanned.load(Ordering::Relaxed);
         m.leader_lock_state = self.scanner_leader_lock_state.read().await.clone();
         m.leader_lock_held_by_this_process = self.scanner_leader_lock_held.load(Ordering::Relaxed);
         m.leader_lock_last_error = self.scanner_leader_lock_last_error.read().await.clone();
@@ -3087,50 +3054,6 @@ mod tests {
         assert_eq!(report.current_disk_bucket_scans_active, 0);
     }
 
-    #[tokio::test]
-    async fn report_counts_scanned_versions_independent_of_lifecycle() {
-        let metrics = Metrics::new();
-
-        // No ILM configuration touched: scanned versions must still accrue so
-        // rustfs_scanner_versions_scanned_total reflects real coverage.
-        metrics.record_scanner_versions_scanned(3);
-        metrics.record_scanner_versions_scanned(5);
-
-        let report = metrics.report().await;
-
-        assert_eq!(report.versions_scanned, 8);
-        // ILM-checked versions live on the Lifecycle source and stay zero when
-        // no lifecycle evaluation ran.
-        let lifecycle_checked = report
-            .source_work
-            .iter()
-            .find(|s| s.source == "lifecycle")
-            .map(|s| s.checked)
-            .unwrap_or_default();
-        assert_eq!(lifecycle_checked, 0);
-    }
-
-    #[tokio::test]
-    async fn report_tracks_lifecycle_checked_versions_separately() {
-        let metrics = Metrics::new();
-
-        // Simulate the scanner walking versions and, on a lifecycle-configured
-        // bucket, handing a subset to the ILM evaluator.
-        metrics.record_scanner_versions_scanned(10);
-        metrics.record_scanner_source_checked(ScannerWorkSource::Lifecycle, 4);
-
-        let report = metrics.report().await;
-
-        assert_eq!(report.versions_scanned, 10, "total scanned versions independent of ILM");
-        let lifecycle_checked = report
-            .source_work
-            .iter()
-            .find(|s| s.source == "lifecycle")
-            .map(|s| s.checked)
-            .unwrap_or_default();
-        assert_eq!(lifecycle_checked, 4, "ILM-checked versions tracked on the Lifecycle source");
-    }
-
     #[tokio::test]
     async fn report_derives_scanner_pacing_pressure() {
         let metrics = Metrics::new();
@@ -3373,40 +3296,6 @@ mod tests {
         metrics.finish_scan_cycle_work(start);
     }
 
-    #[test]
-    fn unresolved_heal_work_only_reflects_the_active_cycle() {
-        let metrics = Metrics::new();
-        assert!(!metrics.current_scan_cycle_has_unresolved_heal_work());
-
-        let start = metrics.start_scan_cycle_work();
-        metrics.record_scanner_source_missed(ScannerWorkSource::Heal, 1);
-        assert!(metrics.current_scan_cycle_has_unresolved_heal_work());
-
-        metrics.finish_scan_cycle_work(start);
-        assert!(!metrics.current_scan_cycle_has_unresolved_heal_work());
-
-        let start = metrics.start_scan_cycle_work();
-        metrics.record_scanner_source_queued(ScannerWorkSource::Heal, 1);
-        assert!(metrics.current_scan_cycle_has_unresolved_heal_work());
-        metrics.finish_scan_cycle_work(start);
-
-        let start = metrics.start_scan_cycle_work();
-        metrics.record_scanner_source_work(
-            ScannerWorkSource::Bitrot,
-            ScannerSourceWorkUpdate {
-                skipped: 1,
-                ..Default::default()
-            },
-        );
-        assert!(metrics.current_scan_cycle_has_unresolved_heal_work());
-        metrics.finish_scan_cycle_work(start);
-
-        let start = metrics.start_scan_cycle_work();
-        metrics.record_scanner_source_failed(ScannerWorkSource::Bitrot, 1);
-        assert!(metrics.current_scan_cycle_has_unresolved_heal_work());
-        metrics.finish_scan_cycle_work(start);
-    }
-
     #[tokio::test]
     async fn report_marks_transition_failures_as_blocked_lifecycle_control() {
         let metrics = Metrics::new();

crates/concurrency/Cargo.toml

@@ -6,22 +6,41 @@ license.workspace = true
 repository.workspace = true
 rust-version.workspace = true
 homepage.workspace = true
-description = "Shared concurrency contract types for RustFS - workload admission, queue snapshots, worker pools, and policy types"
-keywords = ["rustfs", "concurrency", "admission", "backpressure", "workers"]
+description = "Concurrency management for RustFS - timeout, locking, backpressure, and I/O scheduling"
+keywords = ["rustfs", "concurrency", "timeout", "backpressure", "scheduling"]
 categories = ["concurrency", "filesystem"]
 
 [dependencies]
 # Internal crates
 rustfs-io-core = { workspace = true }
-serde = { workspace = true, features = ["derive"] }
+rustfs-io-metrics = { workspace = true }
+serde = { workspace = true }
 
 # Async runtime
-tokio = { workspace = true, features = ["sync", "fs", "rt-multi-thread"] }
+tokio = { workspace = true, features = ["sync", "time", "rt"] }
+tokio-util = { workspace = true }
+
+# Error handling
+thiserror = { workspace = true }
 
 # Logging
 tracing = { workspace = true }
 
 [dev-dependencies]
-insta = { workspace = true, features = ["yaml", "json"] }
-serde_json = { workspace = true, features = ["raw_value"] }
-tokio = { workspace = true, features = ["test-util", "macros", "rt-multi-thread", "fs"] }
+insta = { workspace = true }
+serde_json = { workspace = true }
+tokio = { workspace = true, features = ["test-util","macros","rt-multi-thread"] }
+
+[features]
+default = ["timeout", "lock", "deadlock", "backpressure", "scheduler"]
+
+# Feature modules
+timeout = []
+lock = []
+deadlock = []
+backpressure = []
+scheduler = []
+
+[package.metadata.docs.rs]
+all-features = true
+rustdoc-args = ["--cfg", "docsrs"]

crates/concurrency/src/backpressure.rs

@@ -12,15 +12,17 @@
 // See the License for the specific language governing permissions and
 // limitations under the License.
 
-//! Shared backpressure policy type.
-//!
-//! The runtime backpressure implementation (byte-watermark pipes and
-//! monitors) lives in `rustfs/src/storage/backpressure.rs`; this module only
-//! carries the watermark policy type that implementation shares.
+//! Backpressure management
 
-use rustfs_io_core::BackpressureConfig as CoreBackpressureConfig;
+use rustfs_io_core::{
+    BackpressureConfig as CoreBackpressureConfig, BackpressureMonitor as CoreBackpressureMonitor, BackpressureState,
+};
+use rustfs_io_metrics::backpressure_metrics;
+use std::sync::Arc;
+use std::time::Instant;
+use tokio::io::{DuplexStream, duplex};
 
-/// Watermark policy for duplex-pipe backpressure.
+/// Facade policy for duplex-pipe watermark backpressure.
 #[derive(Debug, Clone, Copy)]
 pub struct PipeBackpressurePolicy {
     /// Buffer size in bytes
@@ -52,9 +54,9 @@ impl PipeBackpressurePolicy {
         (self.buffer_size as u64 * self.low_watermark as u64 / 100) as usize
     }
 
-    /// Convert the policy into the reusable io-core admission-pressure config.
+    /// Convert the facade policy into the reusable io-core admission-pressure config.
     ///
-    /// The caller still owns duplex buffer sizing, but the shared
+    /// The concurrency layer still owns duplex buffer sizing, but the shared
     /// overload/admission primitive lives in `io-core`.
     pub fn to_core_config(&self) -> CoreBackpressureConfig {
         CoreBackpressureConfig {
@@ -67,28 +69,157 @@ impl PipeBackpressurePolicy {
     }
 }
 
+/// Backpressure manager
+pub struct BackpressureManager {
+    config: PipeBackpressurePolicy,
+    core_config: CoreBackpressureConfig,
+    monitor: Arc<CoreBackpressureMonitor>,
+}
+
+impl BackpressureManager {
+    /// Create a new backpressure manager
+    pub fn new(buffer_size: usize, high_watermark: u32, low_watermark: u32) -> Self {
+        Self::from_policy(PipeBackpressurePolicy {
+            buffer_size,
+            high_watermark,
+            low_watermark,
+        })
+    }
+
+    /// Create a new backpressure manager from the facade policy type.
+    pub fn from_policy(config: PipeBackpressurePolicy) -> Self {
+        let core_config = config.to_core_config();
+        Self {
+            config,
+            core_config: core_config.clone(),
+            monitor: Arc::new(CoreBackpressureMonitor::new(core_config)),
+        }
+    }
+
+    /// Get the configuration
+    pub fn config(&self) -> &PipeBackpressurePolicy {
+        &self.config
+    }
+
+    /// Get the derived io-core admission-pressure configuration.
+    pub fn core_config(&self) -> &CoreBackpressureConfig {
+        &self.core_config
+    }
+
+    /// Get the monitor
+    pub fn monitor(&self) -> Arc<CoreBackpressureMonitor> {
+        self.monitor.clone()
+    }
+
+    /// Create a backpressure pipe
+    pub fn create_pipe(&self) -> BackpressurePipe {
+        BackpressurePipe::new(self.config, self.monitor.clone())
+    }
+
+    /// Get current state
+    pub fn state(&self) -> BackpressureState {
+        self.monitor.state()
+    }
+
+    /// Check if backpressure is active
+    pub fn is_active(&self) -> bool {
+        self.monitor.is_active()
+    }
+}
+
+/// Backpressure pipe wrapping tokio's duplex
+pub struct BackpressurePipe {
+    reader: DuplexStream,
+    writer: DuplexStream,
+    config: PipeBackpressurePolicy,
+    monitor: Arc<CoreBackpressureMonitor>,
+    created_at: Instant,
+}
+
+/// Shared pipe metadata snapshot for facade-level backpressure pipes.
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+pub struct BackpressurePipeMeta {
+    /// Configured duplex buffer capacity in bytes.
+    pub buffer_capacity: usize,
+    /// Current backpressure state reported by the shared core monitor.
+    pub state: BackpressureState,
+    /// Age of the pipe since creation.
+    pub age: std::time::Duration,
+}
+
+impl BackpressurePipe {
+    fn new(config: PipeBackpressurePolicy, monitor: Arc<CoreBackpressureMonitor>) -> Self {
+        let (reader, writer) = duplex(config.buffer_size);
+
+        Self {
+            reader,
+            writer,
+            config,
+            monitor,
+            created_at: Instant::now(),
+        }
+    }
+
+    /// Get the reader end
+    pub fn reader(&mut self) -> &mut DuplexStream {
+        &mut self.reader
+    }
+
+    /// Get the writer end
+    pub fn writer(&mut self) -> &mut DuplexStream {
+        &mut self.writer
+    }
+
+    /// Split into reader and writer
+    pub fn into_split(self) -> (DuplexStream, DuplexStream) {
+        (self.reader, self.writer)
+    }
+
+    /// Get the configuration
+    pub fn config(&self) -> &PipeBackpressurePolicy {
+        &self.config
+    }
+
+    /// Get current state
+    pub fn state(&self) -> BackpressureState {
+        self.monitor.state()
+    }
+
+    /// Get the age of this pipe
+    pub fn age(&self) -> std::time::Duration {
+        self.created_at.elapsed()
+    }
+
+    /// Get a compact metadata snapshot for the pipe.
+    pub fn meta(&self) -> BackpressurePipeMeta {
+        BackpressurePipeMeta {
+            buffer_capacity: self.config.buffer_size,
+            state: self.state(),
+            age: self.age(),
+        }
+    }
+
+    /// Check if should apply backpressure
+    pub fn should_apply_backpressure(&self) -> bool {
+        let should = self.monitor.should_apply_backpressure();
+        if should {
+            backpressure_metrics::record_backpressure_activation();
+        }
+        should
+    }
+}
+
 #[cfg(test)]
 mod tests {
     use super::*;
 
     #[test]
-    fn test_backpressure_policy_defaults() {
+    fn test_backpressure_config() {
         let config = PipeBackpressurePolicy::default();
         assert_eq!(config.buffer_size, 4 * 1024 * 1024);
         assert!(config.high_watermark > config.low_watermark);
     }
 
-    #[test]
-    fn test_backpressure_policy_watermark_bytes() {
-        let config = PipeBackpressurePolicy {
-            buffer_size: 1000,
-            high_watermark: 80,
-            low_watermark: 50,
-        };
-        assert_eq!(config.high_watermark_bytes(), 800);
-        assert_eq!(config.low_watermark_bytes(), 500);
-    }
-
     #[test]
     fn test_backpressure_policy_to_core_config() {
         let policy = PipeBackpressurePolicy::default();
@@ -97,4 +228,33 @@ mod tests {
         assert_eq!(core.low_water_mark, policy.low_watermark as f64 / 100.0);
         assert!(core.enabled);
     }
+
+    #[test]
+    fn test_backpressure_manager() {
+        let manager = BackpressureManager::new(1024, 80, 50);
+        assert_eq!(manager.state(), BackpressureState::Normal);
+    }
+
+    #[test]
+    fn test_backpressure_pipe() {
+        let manager = BackpressureManager::new(1024, 80, 50);
+        let pipe = manager.create_pipe();
+        assert_eq!(pipe.state(), BackpressureState::Normal);
+        assert_eq!(pipe.meta().buffer_capacity, 1024);
+    }
+
+    #[test]
+    fn test_backpressure_pipe_meta_is_read_only() {
+        let manager = BackpressureManager::new(2048, 80, 50);
+        let pipe = manager.create_pipe();
+        let first = pipe.meta();
+        let second = pipe.meta();
+
+        assert_eq!(first.buffer_capacity, 2048);
+        assert_eq!(first.state, BackpressureState::Normal);
+        assert_eq!(second.buffer_capacity, first.buffer_capacity);
+        assert_eq!(second.state, first.state);
+        assert!(second.age >= first.age);
+        assert_eq!(manager.state(), BackpressureState::Normal);
+    }
 }

crates/concurrency/src/config.rs

@@ -0,0 +1,242 @@
+// Copyright 2024 RustFS Team
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+//! Configuration for concurrency management
+
+use crate::{
+    backpressure::PipeBackpressurePolicy, deadlock::DeadlockMonitorPolicy, scheduler::SchedulerPolicy,
+    timeout::TimeoutManagerPolicy,
+};
+use std::time::Duration;
+
+/// Feature flags for concurrency modules
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+pub struct ConcurrencyFeatures {
+    /// Enable timeout control
+    pub timeout: bool,
+    /// Enable lock optimization
+    pub lock: bool,
+    /// Enable deadlock detection
+    pub deadlock: bool,
+    /// Enable backpressure management
+    pub backpressure: bool,
+    /// Enable I/O scheduling
+    pub scheduler: bool,
+}
+
+impl Default for ConcurrencyFeatures {
+    fn default() -> Self {
+        Self {
+            timeout: cfg!(feature = "timeout"),
+            lock: cfg!(feature = "lock"),
+            deadlock: cfg!(feature = "deadlock"),
+            backpressure: cfg!(feature = "backpressure"),
+            scheduler: cfg!(feature = "scheduler"),
+        }
+    }
+}
+
+impl ConcurrencyFeatures {
+    /// Create with all features enabled
+    pub fn all() -> Self {
+        Self {
+            timeout: true,
+            lock: true,
+            deadlock: true,
+            backpressure: true,
+            scheduler: true,
+        }
+    }
+
+    /// Create with no features enabled
+    pub fn none() -> Self {
+        Self {
+            timeout: false,
+            lock: false,
+            deadlock: false,
+            backpressure: false,
+            scheduler: false,
+        }
+    }
+
+    /// Check if any feature is enabled
+    pub fn any_enabled(&self) -> bool {
+        self.timeout || self.lock || self.deadlock || self.backpressure || self.scheduler
+    }
+}
+
+/// Facade policy for lock manager behavior.
+#[derive(Debug, Clone, Copy)]
+pub struct LockManagerPolicy {
+    /// Enable lock optimization.
+    pub enabled: bool,
+    /// Lock acquisition timeout.
+    pub acquire_timeout: Duration,
+}
+
+impl Default for LockManagerPolicy {
+    fn default() -> Self {
+        Self {
+            enabled: true,
+            acquire_timeout: Duration::from_secs(5),
+        }
+    }
+}
+
+/// Main configuration for concurrency management
+#[derive(Debug, Clone, Default)]
+pub struct ConcurrencyConfig {
+    /// Feature flags
+    pub features: ConcurrencyFeatures,
+    /// Timeout facade policy.
+    pub timeout_policy: TimeoutManagerPolicy,
+    /// Lock facade policy.
+    pub lock_policy: LockManagerPolicy,
+    /// Deadlock facade policy.
+    pub deadlock_policy: DeadlockMonitorPolicy,
+    /// Backpressure facade policy.
+    pub backpressure_policy: PipeBackpressurePolicy,
+    /// Scheduler facade policy.
+    pub scheduler_policy: SchedulerPolicy,
+}
+
+impl ConcurrencyConfig {
+    /// Create configuration from environment variables
+    pub fn from_env() -> Self {
+        let mut config = Self::default();
+
+        // Read from environment if available
+        if let Ok(val) = std::env::var("RUSTFS_TIMEOUT_DEFAULT")
+            && let Ok(secs) = val.parse::<u64>()
+        {
+            config.timeout_policy.default_timeout = Duration::from_secs(secs);
+        }
+
+        if let Ok(val) = std::env::var("RUSTFS_TIMEOUT_MAX")
+            && let Ok(secs) = val.parse::<u64>()
+        {
+            config.timeout_policy.max_timeout = Duration::from_secs(secs);
+        }
+
+        if let Ok(val) = std::env::var("RUSTFS_BACKPRESSURE_BUFFER_SIZE")
+            && let Ok(size) = val.parse::<usize>()
+        {
+            config.backpressure_policy.buffer_size = size;
+        }
+
+        if let Ok(val) = std::env::var("RUSTFS_IO_BUFFER_SIZE")
+            && let Ok(size) = val.parse::<usize>()
+        {
+            config.scheduler_policy.base_buffer_size = size;
+        }
+
+        config
+    }
+
+    /// Validate configuration
+    pub fn validate(&self) -> Result<(), ConfigError> {
+        if self.timeout_policy.default_timeout > self.timeout_policy.max_timeout {
+            return Err(ConfigError::InvalidTimeout("default_timeout cannot exceed max_timeout".to_string()));
+        }
+        if self.timeout_policy.min_timeout > self.timeout_policy.max_timeout {
+            return Err(ConfigError::InvalidTimeout("min_timeout cannot exceed max_timeout".to_string()));
+        }
+
+        if self.backpressure_policy.high_watermark <= self.backpressure_policy.low_watermark
+            || self.backpressure_policy.high_watermark > 100
+        {
+            return Err(ConfigError::InvalidBackpressure(
+                "high_watermark must be > low_watermark and <= 100".to_string(),
+            ));
+        }
+
+        if self.scheduler_policy.base_buffer_size > self.scheduler_policy.max_buffer_size {
+            return Err(ConfigError::InvalidScheduler(
+                "base_buffer_size cannot exceed max_buffer_size".to_string(),
+            ));
+        }
+
+        Ok(())
+    }
+}
+
+/// Configuration error
+#[allow(clippy::enum_variant_names)]
+#[derive(Debug, Clone, thiserror::Error)]
+pub enum ConfigError {
+    /// Invalid timeout configuration
+    #[error("Invalid timeout config: {0}")]
+    InvalidTimeout(String),
+
+    /// Invalid backpressure configuration
+    #[error("Invalid backpressure config: {0}")]
+    InvalidBackpressure(String),
+
+    /// Invalid scheduler configuration
+    #[error("Invalid scheduler config: {0}")]
+    InvalidScheduler(String),
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    #[test]
+    fn test_default_config() {
+        let config = ConcurrencyConfig::default();
+        assert!(config.validate().is_ok());
+    }
+
+    #[test]
+    fn test_invalid_timeout() {
+        let config = ConcurrencyConfig {
+            timeout_policy: TimeoutManagerPolicy {
+                default_timeout: Duration::from_secs(100),
+                max_timeout: Duration::from_secs(50),
+                enable_dynamic: true,
+                ..Default::default()
+            },
+            ..Default::default()
+        };
+        assert!(
+            config.validate().is_err(),
+            "validate() should return an error when default_timeout > max_timeout"
+        );
+    }
+
+    #[test]
+    fn test_invalid_min_timeout() {
+        let config = ConcurrencyConfig {
+            timeout_policy: TimeoutManagerPolicy {
+                min_timeout: Duration::from_secs(100),
+                max_timeout: Duration::from_secs(50),
+                ..Default::default()
+            },
+            ..Default::default()
+        };
+        assert!(
+            config.validate().is_err(),
+            "validate() should return an error when min_timeout > max_timeout"
+        );
+    }
+
+    #[test]
+    fn test_features() {
+        let features = ConcurrencyFeatures::all();
+        assert!(features.any_enabled());
+
+        let features = ConcurrencyFeatures::none();
+        assert!(!features.any_enabled());
+    }
+}

crates/concurrency/src/deadlock.rs

@@ -12,16 +12,15 @@
 // See the License for the specific language governing permissions and
 // limitations under the License.
 
-//! Shared deadlock-monitor policy type.
-//!
-//! The runtime request-hang / deadlock detection loop lives in
-//! `rustfs/src/storage/deadlock_detector.rs`; this module only carries the
-//! monitor policy type that implementation shares.
+//! Deadlock detection management
 
-use rustfs_io_core::DeadlockDetectorConfig as CoreDeadlockConfig;
-use std::time::Duration;
+use rustfs_io_core::{DeadlockDetector as CoreDeadlockDetector, DeadlockDetectorConfig as CoreDeadlockConfig, LockType};
+use rustfs_io_metrics::deadlock_metrics;
+use std::collections::HashMap;
+use std::sync::Arc;
+use std::time::{Duration, Instant};
 
-/// Policy for the request-hang deadlock monitor.
+/// Facade policy for the concurrency-layer deadlock monitor.
 #[derive(Debug, Clone, Copy)]
 pub struct DeadlockMonitorPolicy {
     /// Enable deadlock detection
@@ -43,7 +42,7 @@ impl Default for DeadlockMonitorPolicy {
 }
 
 impl DeadlockMonitorPolicy {
-    /// Convert the policy into the reusable io-core deadlock config.
+    /// Convert the facade policy into the reusable io-core deadlock config.
     pub fn to_core_config(&self) -> CoreDeadlockConfig {
         CoreDeadlockConfig {
             enabled: self.enabled,
@@ -53,14 +52,182 @@ impl DeadlockMonitorPolicy {
     }
 }
 
+/// Deadlock manager
+pub struct DeadlockManager {
+    config: DeadlockMonitorPolicy,
+    detector: Arc<CoreDeadlockDetector>,
+    running: Arc<tokio::sync::Mutex<bool>>,
+}
+
+impl DeadlockManager {
+    /// Create a new deadlock manager
+    pub fn new(enabled: bool, check_interval: Duration, hang_threshold: Duration) -> Self {
+        Self::from_policy(DeadlockMonitorPolicy {
+            enabled,
+            check_interval,
+            hang_threshold,
+        })
+    }
+
+    /// Create a new deadlock manager from the facade policy type.
+    pub fn from_policy(config: DeadlockMonitorPolicy) -> Self {
+        let core_config = config.to_core_config();
+        Self {
+            config,
+            detector: Arc::new(CoreDeadlockDetector::new(core_config)),
+            running: Arc::new(tokio::sync::Mutex::new(false)),
+        }
+    }
+
+    /// Get the configuration
+    pub fn config(&self) -> &DeadlockMonitorPolicy {
+        &self.config
+    }
+
+    /// Get the core detector
+    pub fn detector(&self) -> Arc<CoreDeadlockDetector> {
+        self.detector.clone()
+    }
+
+    /// Start the deadlock detection background task
+    pub async fn start(&self) {
+        if !self.config.enabled {
+            return;
+        }
+
+        let mut running = self.running.lock().await;
+        if *running {
+            return;
+        }
+        *running = true;
+        drop(running);
+
+        tracing::info!(
+            event = "deadlock_monitor.lifecycle",
+            component = "concurrency",
+            subsystem = "deadlock",
+            state = "started",
+            check_interval_ms = self.config.check_interval.as_millis(),
+            hang_threshold_ms = self.config.hang_threshold.as_millis(),
+            "deadlock monitor state changed"
+        );
+    }
+
+    /// Stop the deadlock detection
+    pub async fn stop(&self) {
+        let mut running = self.running.lock().await;
+        *running = false;
+
+        tracing::info!(
+            event = "deadlock_monitor.lifecycle",
+            component = "concurrency",
+            subsystem = "deadlock",
+            state = "stopped",
+            check_interval_ms = self.config.check_interval.as_millis(),
+            hang_threshold_ms = self.config.hang_threshold.as_millis(),
+            "deadlock monitor state changed"
+        );
+    }
+
+    /// Create a request tracker
+    pub fn track_request(&self, request_id: String, description: String) -> RequestTracker {
+        RequestTracker::new(request_id, description, self.detector.clone())
+    }
+
+    /// Register a lock
+    pub fn register_lock(&self, lock_type: LockType) -> u64 {
+        self.detector.register_lock(lock_type)
+    }
+
+    /// Unregister a lock
+    pub fn unregister_lock(&self, lock_id: u64) {
+        self.detector.unregister_lock(lock_id);
+    }
+
+    /// Detect deadlock
+    pub fn detect_deadlock(&self) -> Option<Vec<u64>> {
+        let result = self.detector.detect_deadlock();
+        if let Some(ref cycle) = result {
+            deadlock_metrics::record_deadlock_detected(cycle.len());
+        }
+        result
+    }
+}
+
+/// Lightweight compatibility wrapper for request-scoped deadlock bookkeeping.
+///
+/// This type intentionally stays minimal in the concurrency layer. Rich
+/// request-level lock/resource diagnostics belong to
+/// `rustfs::storage::deadlock_detector::RequestResourceTracker`.
+pub struct RequestTracker {
+    request_id: String,
+    description: String,
+    start_time: Instant,
+    resources: HashMap<String, Vec<String>>,
+    detector: Arc<CoreDeadlockDetector>,
+}
+
+impl RequestTracker {
+    fn new(request_id: String, description: String, detector: Arc<CoreDeadlockDetector>) -> Self {
+        let start_time = Instant::now();
+        detector.register_request(&request_id, 1); // Use placeholder thread ID
+
+        Self {
+            request_id,
+            description,
+            start_time,
+            resources: HashMap::new(),
+            detector,
+        }
+    }
+
+    /// Get the request ID
+    pub fn request_id(&self) -> &str {
+        &self.request_id
+    }
+
+    /// Get the description
+    pub fn description(&self) -> &str {
+        &self.description
+    }
+
+    /// Get the elapsed time
+    pub fn elapsed(&self) -> Duration {
+        self.start_time.elapsed()
+    }
+
+    /// Record a lock acquisition
+    pub fn record_lock_acquire(&mut self, lock_id: u64, resource: String) {
+        self.resources.entry("locks".to_string()).or_default().push(resource);
+        self.detector.record_acquire(lock_id, 1); // Use placeholder thread ID
+        deadlock_metrics::record_lock_acquisition("read");
+    }
+
+    /// Return a read-only view of tracked resource names.
+    pub fn resources(&self) -> &HashMap<String, Vec<String>> {
+        &self.resources
+    }
+
+    /// Record a lock release
+    pub fn record_lock_release(&mut self, lock_id: u64) {
+        self.detector.record_release(lock_id);
+    }
+}
+
+impl Drop for RequestTracker {
+    fn drop(&mut self) {
+        self.detector.unregister_request(&self.request_id);
+    }
+}
+
 #[cfg(test)]
 mod tests {
     use super::*;
 
     #[test]
-    fn test_deadlock_policy_defaults_disabled() {
-        let policy = DeadlockMonitorPolicy::default();
-        assert!(!policy.enabled);
+    fn test_deadlock_manager_creation() {
+        let manager = DeadlockManager::new(false, Duration::from_secs(10), Duration::from_secs(60));
+        assert!(!manager.config().enabled);
     }
 
     #[test]
@@ -71,4 +238,16 @@ mod tests {
         assert_eq!(core.detection_interval, policy.check_interval);
         assert_eq!(core.max_hold_time, policy.hang_threshold);
     }
+
+    #[tokio::test]
+    async fn test_request_tracker() {
+        let manager = DeadlockManager::new(true, Duration::from_secs(10), Duration::from_secs(60));
+        let mut tracker = manager.track_request("req-1".to_string(), "test request".to_string());
+        let lock_id = manager.register_lock(LockType::Mutex);
+        tracker.record_lock_acquire(lock_id, "bucket/key".to_string());
+
+        assert_eq!(tracker.request_id(), "req-1");
+        assert_eq!(tracker.description(), "test request");
+        assert_eq!(tracker.resources().get("locks").map(Vec::len), Some(1));
+    }
 }

crates/concurrency/src/lib.rs

@@ -12,40 +12,167 @@
 // See the License for the specific language governing permissions and
 // limitations under the License.
 
-//! # RustFS Concurrency Contracts
+//! # RustFS Concurrency Management
 //!
-//! Shared concurrency contract types for RustFS:
+//! This crate provides comprehensive concurrency management for RustFS,
+//! including timeout control, lock optimization, deadlock detection,
+//! backpressure management, and I/O scheduling.
 //!
-//! - [`workload`]: workload admission snapshot/contract types consumed by
-//!   ecstore, heal, and the server for admission-aware throttling.
-//! - [`GetObjectQueueSnapshot`]: disk permit queue usage snapshot for
-//!   GetObject orchestration.
-//! - [`workers`]: a bounded semaphore-backed worker-slot pool.
-//! - [`PipeBackpressurePolicy`] / [`DeadlockMonitorPolicy`]: policy types
-//!   shared with the runtime implementations.
+//! ## Features
 //!
-//! The actual runtime concurrency control — size-aware timeouts, byte-watermark
-//! backpressure, request-hang/deadlock detection, and I/O scheduling — is
-//! implemented in `rustfs/src/storage/*` on top of the `rustfs-io-core`
-//! primitives. This crate only carries the data and contract types those
-//! implementations share; it does not run any background tasks itself.
+//! All features are controlled by feature flags and can be enabled/disabled at compile time:
+//!
+//! - **timeout**: Dynamic timeout calculation based on data size and transfer rate
+//! - **lock**: Early lock release to reduce contention
+//! - **deadlock**: Request tracking and cycle detection
+//! - **backpressure**: Buffer-based flow control
+//! - **scheduler**: Adaptive buffer sizing and priority queuing
+//!
+//! ## Architecture
+//!
+//! ```text
+//! rustfs-concurrency (Business Layer)
+//!     ├── timeout (Timeout Control)
+//!     ├── lock (Lock Optimization)
+//!     ├── deadlock (Deadlock Detection)
+//!     ├── backpressure (Backpressure Management)
+//!     └── scheduler (I/O Scheduling)
+//!          │
+//!          ├── rustfs-io-core (Core Algorithms)
+//!          └── rustfs-io-metrics (Metrics Collection)
+//! ```
+//!
+//! ## Usage
+//!
+//! ```rust,no_run
+//! use rustfs_concurrency::{ConcurrencyConfig, ConcurrencyManager};
+//!
+//! # #[tokio::main]
+//! # async fn main() {
+//! // Create manager with all features enabled
+//! let config = ConcurrencyConfig::default();
+//! let manager = ConcurrencyManager::new(config);
+//!
+//! // Start services
+//! manager.start().await;
+//!
+//! // Use timeout control (if enabled)
+//! if manager.is_timeout_enabled() {
+//!     let timeout_manager = manager.timeout();
+//!     let _ = timeout_manager;
+//! }
+//!
+//! // Use lock optimization (if enabled)
+//! if manager.is_lock_enabled() {
+//!     let lock_manager = manager.lock();
+//!     let _ = lock_manager;
+//! }
+//!
+//! // Stop services
+//! manager.stop().await;
+//! # }
+//! ```
 
 #![deny(missing_docs)]
 #![deny(unsafe_code)]
+#![cfg_attr(docsrs, feature(doc_cfg))]
 
-// Re-exported io-core type used by downstream timeout implementations.
-pub use rustfs_io_core::OperationProgress;
+// Re-export core types from io-core
+pub use rustfs_io_core::{
+    // Backpressure types
+    BackpressureConfig as CoreBackpressureConfig,
+    BackpressureMonitor as CoreBackpressureMonitor,
+    BackpressureState,
 
-mod backpressure;
+    // Deadlock types
+    DeadlockDetector as CoreDeadlockDetector,
+    IoLoadLevel,
+    IoLoadMetrics,
+    IoPriority,
+    // Scheduler types
+    IoScheduler,
+    IoSchedulingContext,
+    LockInfo,
+    LockOptimizer as CoreLockOptimizer,
+
+    // Lock types
+    LockStats as CoreLockStats,
+    LockType,
+    // Timeout types
+    OperationProgress,
+    TimeoutError,
+    TimeoutStats,
+    WaitGraphEdge,
+
+    calculate_adaptive_timeout,
+    estimate_bytes_per_second,
+};
+
+// Module declarations with feature gates
+#[cfg(feature = "timeout")]
+mod timeout;
+
+#[cfg(feature = "lock")]
+mod lock;
+
+#[cfg(feature = "deadlock")]
 mod deadlock;
-mod queue;
+
+#[cfg(feature = "backpressure")]
+mod backpressure;
+
+#[cfg(feature = "scheduler")]
+mod scheduler;
+
 pub mod workers;
 pub mod workload;
 
-pub use backpressure::PipeBackpressurePolicy;
-pub use deadlock::DeadlockMonitorPolicy;
-pub use queue::GetObjectQueueSnapshot;
+// Public module exports with feature gates
+#[cfg(feature = "timeout")]
+pub use timeout::{TimeoutGuard, TimeoutManager, TimeoutManagerPolicy};
+
+#[cfg(feature = "lock")]
+pub use lock::{LockConfig, LockManager, LockScopeGuard, OptimizedLockGuard};
+
+#[cfg(feature = "deadlock")]
+pub use deadlock::{DeadlockManager, DeadlockMonitorPolicy, RequestTracker};
+
+#[cfg(feature = "backpressure")]
+pub use backpressure::{BackpressureManager, BackpressurePipe, PipeBackpressurePolicy};
+
+#[cfg(feature = "scheduler")]
+pub use scheduler::{IoStrategy, SchedulerManager, SchedulerPolicy};
+
+// Configuration
+mod config;
+pub use config::{ConcurrencyConfig, ConcurrencyFeatures};
+
+// Manager
+mod manager;
+pub use manager::{ConcurrencyManager, GetObjectQueueSnapshot};
 pub use workload::{
     AdmissionState, WorkloadAdmissionRegistrySnapshot, WorkloadAdmissionSnapshot, WorkloadAdmissionSnapshotProvider,
     WorkloadClass,
 };
+
+// Prelude for convenient imports
+pub mod prelude {
+    //! Prelude module for convenient imports
+
+    #[cfg(feature = "timeout")]
+    pub use crate::timeout::{TimeoutGuard, TimeoutManager, TimeoutManagerPolicy};
+
+    #[cfg(feature = "lock")]
+    pub use crate::lock::{LockConfig, LockManager, LockScopeGuard, OptimizedLockGuard};
+
+    #[cfg(feature = "deadlock")]
+    pub use crate::deadlock::{DeadlockManager, DeadlockMonitorPolicy, RequestTracker};
+
+    #[cfg(feature = "backpressure")]
+    pub use crate::backpressure::{BackpressureManager, BackpressurePipe, PipeBackpressurePolicy};
+
+    #[cfg(feature = "scheduler")]
+    pub use crate::scheduler::{IoStrategy, SchedulerManager, SchedulerPolicy};
+
+    pub use crate::{AdmissionState, ConcurrencyConfig, ConcurrencyFeatures, ConcurrencyManager, WorkloadAdmissionSnapshot};
+}

crates/concurrency/src/lock.rs

@@ -0,0 +1,227 @@
+// Copyright 2024 RustFS Team
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+//! Lock optimization management
+
+use rustfs_io_core::{LockOptimizer as CoreLockOptimizer, LockStats};
+use rustfs_io_metrics::lock_metrics;
+use std::sync::Arc;
+use std::time::{Duration, Instant};
+
+/// Lock configuration
+#[derive(Debug, Clone)]
+pub struct LockConfig {
+    /// Enable lock optimization
+    pub enabled: bool,
+    /// Lock acquisition timeout
+    pub acquire_timeout: Duration,
+}
+
+impl Default for LockConfig {
+    fn default() -> Self {
+        Self {
+            enabled: true,
+            acquire_timeout: Duration::from_secs(5),
+        }
+    }
+}
+
+/// Lock manager
+pub struct LockManager {
+    config: LockConfig,
+    optimizer: Arc<CoreLockOptimizer>,
+}
+
+impl LockManager {
+    /// Create a new lock manager
+    pub fn new(enabled: bool, acquire_timeout: Duration) -> Self {
+        let config = LockConfig {
+            enabled,
+            acquire_timeout,
+        };
+
+        let core_config = rustfs_io_core::LockOptimizeConfig {
+            enabled,
+            acquire_timeout,
+            max_hold_time_warning: Duration::from_millis(100),
+            adaptive_spin: true,
+            max_spin_iterations: 1000,
+        };
+
+        Self {
+            config,
+            optimizer: Arc::new(CoreLockOptimizer::new(core_config)),
+        }
+    }
+
+    /// Get the configuration
+    pub fn config(&self) -> &LockConfig {
+        &self.config
+    }
+
+    /// Get the core optimizer
+    pub fn optimizer(&self) -> Arc<CoreLockOptimizer> {
+        self.optimizer.clone()
+    }
+
+    /// Get lock statistics
+    pub fn stats(&self) -> &LockStats {
+        self.optimizer.stats()
+    }
+
+    /// Optimize a lock guard
+    pub fn optimize<G>(&self, guard: G, resource: impl Into<String>) -> OptimizedLockGuard<G> {
+        OptimizedLockGuard::new(guard, resource, self.optimizer.clone())
+    }
+
+    /// Check if optimization is enabled
+    pub fn is_enabled(&self) -> bool {
+        self.config.enabled
+    }
+}
+
+/// Optimized lock guard with early release support
+pub struct OptimizedLockGuard<G> {
+    guard: Option<G>,
+    acquire_time: Instant,
+    released: bool,
+    resource: String,
+    optimizer: Arc<CoreLockOptimizer>,
+}
+
+impl<G> OptimizedLockGuard<G> {
+    fn new(guard: G, resource: impl Into<String>, optimizer: Arc<CoreLockOptimizer>) -> Self {
+        optimizer.on_acquire();
+        lock_metrics::record_lock_optimization_enabled(optimizer.config().enabled);
+
+        Self {
+            guard: Some(guard),
+            acquire_time: Instant::now(),
+            released: false,
+            resource: resource.into(),
+            optimizer,
+        }
+    }
+
+    /// Get the lock hold time
+    pub fn hold_time(&self) -> Duration {
+        self.acquire_time.elapsed()
+    }
+
+    /// Check if the lock has been released
+    pub fn is_released(&self) -> bool {
+        self.released
+    }
+
+    /// Release the lock early
+    pub fn early_release(&mut self) {
+        if self.released {
+            return;
+        }
+
+        let hold_time = self.hold_time();
+        self.guard.take();
+        self.released = true;
+
+        self.optimizer.on_release(hold_time);
+        lock_metrics::record_lock_hold_time(hold_time);
+
+        tracing::debug!(
+            event = "lock_guard.release",
+            component = "concurrency",
+            subsystem = "lock",
+            release_mode = "early",
+            resource = %self.resource,
+            hold_time_ms = hold_time.as_millis(),
+            "lock guard released"
+        );
+    }
+
+    /// Get a reference to the underlying guard
+    pub fn as_ref(&self) -> Option<&G> {
+        if self.released { None } else { self.guard.as_ref() }
+    }
+}
+
+impl<G> Drop for OptimizedLockGuard<G> {
+    fn drop(&mut self) {
+        if !self.released {
+            let hold_time = self.hold_time();
+            self.guard.take();
+            self.released = true;
+
+            self.optimizer.on_release(hold_time);
+            lock_metrics::record_lock_hold_time(hold_time);
+
+            tracing::debug!(
+                event = "lock_guard.release",
+                component = "concurrency",
+                subsystem = "lock",
+                release_mode = "drop",
+                resource = %self.resource,
+                hold_time_ms = hold_time.as_millis(),
+                "lock guard released"
+            );
+        }
+    }
+}
+
+/// Lock scope guard for RAII semantics
+pub struct LockScopeGuard<G> {
+    guard: Option<G>,
+}
+
+impl<G> LockScopeGuard<G> {
+    /// Create a new scope guard
+    pub fn new(guard: G) -> Self {
+        Self { guard: Some(guard) }
+    }
+
+    /// Get a reference to the guard
+    pub fn as_ref(&self) -> Option<&G> {
+        self.guard.as_ref()
+    }
+}
+
+impl<G> Drop for LockScopeGuard<G> {
+    fn drop(&mut self) {
+        self.guard.take();
+    }
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+    use std::sync::Mutex;
+
+    #[test]
+    fn test_lock_manager_creation() {
+        let manager = LockManager::new(true, Duration::from_secs(5));
+        assert!(manager.is_enabled());
+    }
+
+    #[test]
+    fn test_optimized_lock_guard() {
+        let manager = LockManager::new(true, Duration::from_secs(5));
+        let mutex = Mutex::new(42);
+        let guard = mutex.lock().unwrap();
+
+        let optimized = manager.optimize(guard, "test_resource");
+        assert!(!optimized.is_released());
+
+        let mut optimized = optimized;
+        optimized.early_release();
+        assert!(optimized.is_released());
+    }
+}

crates/concurrency/src/manager.rs

@@ -0,0 +1,397 @@
+// Copyright 2024 RustFS Team
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+//! Main concurrency manager
+
+use crate::config::{ConcurrencyConfig, ConfigError};
+use std::sync::Arc;
+
+/// Snapshot of disk permit queue usage for GetObject orchestration.
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+pub struct GetObjectQueueSnapshot {
+    /// Total permits configured for disk reads.
+    pub total_permits: usize,
+    /// Permits currently in use.
+    pub permits_in_use: usize,
+}
+
+impl GetObjectQueueSnapshot {
+    /// Create a queue snapshot from total and available permits.
+    pub fn from_available_permits(total_permits: usize, available_permits: usize) -> Self {
+        Self {
+            total_permits,
+            permits_in_use: total_permits.saturating_sub(available_permits),
+        }
+    }
+
+    /// Return currently available permits.
+    pub fn permits_available(&self) -> usize {
+        self.total_permits.saturating_sub(self.permits_in_use)
+    }
+
+    /// Return queue utilization percentage in the 0-100 range.
+    pub fn utilization_percent(&self) -> f64 {
+        if self.total_permits == 0 {
+            0.0
+        } else {
+            (self.permits_in_use as f64 / self.total_permits as f64) * 100.0
+        }
+    }
+
+    /// Return whether the queue is considered congested.
+    pub fn is_congested(&self, threshold_percent: f64) -> bool {
+        self.utilization_percent() > threshold_percent
+    }
+}
+
+/// Main concurrency manager that provides access to all concurrency features
+pub struct ConcurrencyManager {
+    config: ConcurrencyConfig,
+
+    #[cfg(feature = "timeout")]
+    timeout: Arc<crate::timeout::TimeoutManager>,
+
+    #[cfg(feature = "lock")]
+    lock: Arc<crate::lock::LockManager>,
+
+    #[cfg(feature = "deadlock")]
+    deadlock: Arc<crate::deadlock::DeadlockManager>,
+
+    #[cfg(feature = "backpressure")]
+    backpressure: Arc<crate::backpressure::BackpressureManager>,
+
+    #[cfg(feature = "scheduler")]
+    scheduler: Arc<crate::scheduler::SchedulerManager>,
+}
+
+impl ConcurrencyManager {
+    fn build(config: ConcurrencyConfig) -> Self {
+        Self {
+            #[cfg(feature = "timeout")]
+            timeout: Arc::new(crate::timeout::TimeoutManager::from_policy(config.timeout_policy)),
+
+            #[cfg(feature = "lock")]
+            lock: Arc::new(crate::lock::LockManager::new(
+                config.lock_policy.enabled,
+                config.lock_policy.acquire_timeout,
+            )),
+
+            #[cfg(feature = "deadlock")]
+            deadlock: Arc::new(crate::deadlock::DeadlockManager::from_policy(config.deadlock_policy)),
+
+            #[cfg(feature = "backpressure")]
+            backpressure: Arc::new(crate::backpressure::BackpressureManager::from_policy(config.backpressure_policy)),
+
+            #[cfg(feature = "scheduler")]
+            scheduler: Arc::new(crate::scheduler::SchedulerManager::from_policy(config.scheduler_policy)),
+
+            config,
+        }
+    }
+
+    /// Try to create a new concurrency manager with the given configuration.
+    pub fn try_new(config: ConcurrencyConfig) -> Result<Self, ConfigError> {
+        config.validate()?;
+        Ok(Self::build(config))
+    }
+
+    /// Create a new concurrency manager with the given configuration.
+    ///
+    /// Invalid configurations are downgraded to the default configuration instead of
+    /// panicking so startup/runtime callers can remain fail-safe in production paths.
+    pub fn new(config: ConcurrencyConfig) -> Self {
+        match Self::try_new(config) {
+            Ok(manager) => manager,
+            Err(err) => {
+                tracing::warn!(
+                    event = "concurrency_manager.invalid_config_fallback",
+                    component = "concurrency",
+                    subsystem = "manager",
+                    error = %err,
+                    "Invalid concurrency configuration detected; falling back to defaults"
+                );
+                Self::build(ConcurrencyConfig::default())
+            }
+        }
+    }
+
+    /// Create with default configuration
+    pub fn with_defaults() -> Self {
+        Self::build(ConcurrencyConfig::default())
+    }
+
+    /// Try to create a manager from environment-derived configuration.
+    pub fn try_from_env() -> Result<Self, ConfigError> {
+        Self::try_new(ConcurrencyConfig::from_env())
+    }
+
+    /// Create from environment variables
+    pub fn from_env() -> Self {
+        match Self::try_from_env() {
+            Ok(manager) => manager,
+            Err(err) => {
+                tracing::warn!(
+                    event = "concurrency_manager.invalid_env_fallback",
+                    component = "concurrency",
+                    subsystem = "manager",
+                    error = %err,
+                    "Invalid environment-derived concurrency configuration detected; falling back to defaults"
+                );
+                Self::build(ConcurrencyConfig::default())
+            }
+        }
+    }
+
+    /// Get the configuration
+    pub fn config(&self) -> &ConcurrencyConfig {
+        &self.config
+    }
+
+    // ============================================
+    // Feature enable checks
+    // ============================================
+
+    /// Check if timeout feature is enabled
+    pub fn is_timeout_enabled(&self) -> bool {
+        #[cfg(feature = "timeout")]
+        {
+            self.config.features.timeout
+        }
+        #[cfg(not(feature = "timeout"))]
+        {
+            false
+        }
+    }
+
+    /// Check if lock feature is enabled
+    pub fn is_lock_enabled(&self) -> bool {
+        #[cfg(feature = "lock")]
+        {
+            self.config.features.lock
+        }
+        #[cfg(not(feature = "lock"))]
+        {
+            false
+        }
+    }
+
+    /// Check if deadlock feature is enabled
+    pub fn is_deadlock_enabled(&self) -> bool {
+        #[cfg(feature = "deadlock")]
+        {
+            self.config.features.deadlock
+        }
+        #[cfg(not(feature = "deadlock"))]
+        {
+            false
+        }
+    }
+
+    /// Check if backpressure feature is enabled
+    pub fn is_backpressure_enabled(&self) -> bool {
+        #[cfg(feature = "backpressure")]
+        {
+            self.config.features.backpressure
+        }
+        #[cfg(not(feature = "backpressure"))]
+        {
+            false
+        }
+    }
+
+    /// Check if scheduler feature is enabled
+    pub fn is_scheduler_enabled(&self) -> bool {
+        #[cfg(feature = "scheduler")]
+        {
+            self.config.features.scheduler
+        }
+        #[cfg(not(feature = "scheduler"))]
+        {
+            false
+        }
+    }
+
+    // ============================================
+    // Feature accessors
+    // ============================================
+
+    /// Get timeout manager
+    #[cfg(feature = "timeout")]
+    pub fn timeout(&self) -> Arc<crate::timeout::TimeoutManager> {
+        self.timeout.clone()
+    }
+
+    /// Get lock manager
+    #[cfg(feature = "lock")]
+    pub fn lock(&self) -> Arc<crate::lock::LockManager> {
+        self.lock.clone()
+    }
+
+    /// Get deadlock manager
+    #[cfg(feature = "deadlock")]
+    pub fn deadlock(&self) -> Arc<crate::deadlock::DeadlockManager> {
+        self.deadlock.clone()
+    }
+
+    /// Get backpressure manager
+    #[cfg(feature = "backpressure")]
+    pub fn backpressure(&self) -> Arc<crate::backpressure::BackpressureManager> {
+        self.backpressure.clone()
+    }
+
+    /// Get scheduler manager
+    #[cfg(feature = "scheduler")]
+    pub fn scheduler(&self) -> Arc<crate::scheduler::SchedulerManager> {
+        self.scheduler.clone()
+    }
+
+    // ============================================
+    // Lifecycle management
+    // ============================================
+
+    /// Start all enabled services (e.g., deadlock detection background task)
+    pub async fn start(&self) {
+        #[cfg(feature = "deadlock")]
+        {
+            if self.config.deadlock_policy.enabled {
+                self.deadlock.start().await;
+            }
+        }
+
+        tracing::info!(
+            event = "concurrency_manager.lifecycle",
+            component = "concurrency",
+            subsystem = "manager",
+            state = "started",
+            timeout_enabled = self.is_timeout_enabled(),
+            lock_enabled = self.is_lock_enabled(),
+            deadlock_enabled = self.is_deadlock_enabled(),
+            backpressure_enabled = self.is_backpressure_enabled(),
+            scheduler_enabled = self.is_scheduler_enabled(),
+            "concurrency manager state changed"
+        );
+    }
+
+    /// Stop all services
+    pub async fn stop(&self) {
+        #[cfg(feature = "deadlock")]
+        {
+            self.deadlock.stop().await;
+        }
+
+        tracing::info!(
+            event = "concurrency_manager.lifecycle",
+            component = "concurrency",
+            subsystem = "manager",
+            state = "stopped",
+            "concurrency manager state changed"
+        );
+    }
+}
+
+impl Default for ConcurrencyManager {
+    fn default() -> Self {
+        Self::with_defaults()
+    }
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    #[test]
+    fn test_queue_snapshot() {
+        let snapshot = GetObjectQueueSnapshot::from_available_permits(64, 16);
+        assert_eq!(snapshot.permits_in_use, 48);
+        assert_eq!(snapshot.permits_available(), 16);
+        assert!(snapshot.is_congested(70.0));
+    }
+
+    #[test]
+    fn test_queue_snapshot_clamps_over_available_permits() {
+        let snapshot = GetObjectQueueSnapshot::from_available_permits(64, 96);
+        assert_eq!(snapshot.permits_in_use, 0);
+        assert_eq!(snapshot.permits_available(), 64);
+        assert_eq!(snapshot.utilization_percent(), 0.0);
+        assert!(!snapshot.is_congested(0.0));
+    }
+
+    #[test]
+    fn test_queue_snapshot_handles_zero_total_permits() {
+        let snapshot = GetObjectQueueSnapshot::from_available_permits(0, 0);
+        assert_eq!(snapshot.permits_in_use, 0);
+        assert_eq!(snapshot.permits_available(), 0);
+        assert_eq!(snapshot.utilization_percent(), 0.0);
+        assert!(!snapshot.is_congested(0.0));
+    }
+
+    #[test]
+    fn test_manager_creation() {
+        let manager = ConcurrencyManager::with_defaults();
+        assert!(manager.config().validate().is_ok());
+    }
+
+    #[test]
+    fn test_try_new_returns_error_for_invalid_config() {
+        let config = ConcurrencyConfig {
+            timeout_policy: crate::TimeoutManagerPolicy {
+                default_timeout: std::time::Duration::from_secs(10),
+                max_timeout: std::time::Duration::from_secs(1),
+                ..Default::default()
+            },
+            ..Default::default()
+        };
+
+        let result = ConcurrencyManager::try_new(config);
+        assert!(matches!(result, Err(ConfigError::InvalidTimeout(_))));
+    }
+
+    #[test]
+    fn test_new_falls_back_to_default_for_invalid_config() {
+        let config = ConcurrencyConfig {
+            timeout_policy: crate::TimeoutManagerPolicy {
+                default_timeout: std::time::Duration::from_secs(10),
+                max_timeout: std::time::Duration::from_secs(1),
+                ..Default::default()
+            },
+            ..Default::default()
+        };
+
+        let manager = ConcurrencyManager::new(config);
+        assert!(manager.config().validate().is_ok());
+        assert_eq!(
+            manager.config().timeout_policy.default_timeout,
+            ConcurrencyConfig::default().timeout_policy.default_timeout
+        );
+    }
+
+    #[tokio::test]
+    async fn test_manager_lifecycle() {
+        let manager = ConcurrencyManager::with_defaults();
+        manager.start().await;
+        manager.stop().await;
+    }
+
+    #[test]
+    fn test_feature_checks() {
+        let manager = ConcurrencyManager::with_defaults();
+
+        // These should return the feature flag status
+        let _ = manager.is_timeout_enabled();
+        let _ = manager.is_lock_enabled();
+        let _ = manager.is_deadlock_enabled();
+        let _ = manager.is_backpressure_enabled();
+        let _ = manager.is_scheduler_enabled();
+    }
+}

crates/concurrency/src/queue.rs

@@ -1,84 +0,0 @@
-// Copyright 2024 RustFS Team
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//     http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-//! Disk permit queue snapshot for GetObject orchestration.
-
-/// Snapshot of disk permit queue usage for GetObject orchestration.
-#[derive(Debug, Clone, Copy, PartialEq, Eq)]
-pub struct GetObjectQueueSnapshot {
-    /// Total permits configured for disk reads.
-    pub total_permits: usize,
-    /// Permits currently in use.
-    pub permits_in_use: usize,
-}
-
-impl GetObjectQueueSnapshot {
-    /// Create a queue snapshot from total and available permits.
-    pub fn from_available_permits(total_permits: usize, available_permits: usize) -> Self {
-        Self {
-            total_permits,
-            permits_in_use: total_permits.saturating_sub(available_permits),
-        }
-    }
-
-    /// Return currently available permits.
-    pub fn permits_available(&self) -> usize {
-        self.total_permits.saturating_sub(self.permits_in_use)
-    }
-
-    /// Return queue utilization percentage in the 0-100 range.
-    pub fn utilization_percent(&self) -> f64 {
-        if self.total_permits == 0 {
-            0.0
-        } else {
-            (self.permits_in_use as f64 / self.total_permits as f64) * 100.0
-        }
-    }
-
-    /// Return whether the queue is considered congested.
-    pub fn is_congested(&self, threshold_percent: f64) -> bool {
-        self.utilization_percent() > threshold_percent
-    }
-}
-
-#[cfg(test)]
-mod tests {
-    use super::*;
-
-    #[test]
-    fn test_queue_snapshot() {
-        let snapshot = GetObjectQueueSnapshot::from_available_permits(64, 16);
-        assert_eq!(snapshot.permits_in_use, 48);
-        assert_eq!(snapshot.permits_available(), 16);
-        assert!(snapshot.is_congested(70.0));
-    }
-
-    #[test]
-    fn test_queue_snapshot_clamps_over_available_permits() {
-        let snapshot = GetObjectQueueSnapshot::from_available_permits(64, 96);
-        assert_eq!(snapshot.permits_in_use, 0);
-        assert_eq!(snapshot.permits_available(), 64);
-        assert_eq!(snapshot.utilization_percent(), 0.0);
-        assert!(!snapshot.is_congested(0.0));
-    }
-
-    #[test]
-    fn test_queue_snapshot_handles_zero_total_permits() {
-        let snapshot = GetObjectQueueSnapshot::from_available_permits(0, 0);
-        assert_eq!(snapshot.permits_in_use, 0);
-        assert_eq!(snapshot.permits_available(), 0);
-        assert_eq!(snapshot.utilization_percent(), 0.0);
-        assert!(!snapshot.is_congested(0.0));
-    }
-}

crates/concurrency/src/scheduler.rs

@@ -0,0 +1,278 @@
+// Copyright 2024 RustFS Team
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+//! I/O scheduler management
+
+use rustfs_io_core::{
+    IoLoadLevel, IoPriority, IoScheduler as CoreIoScheduler, IoSchedulingContext,
+    io_profile::{AccessPattern, StorageMedia},
+};
+use rustfs_io_metrics::io_metrics;
+use std::sync::Arc;
+use std::time::Duration;
+
+/// Facade policy for the concurrency-layer scheduler manager.
+#[derive(Debug, Clone, Copy)]
+pub struct SchedulerPolicy {
+    /// Base buffer size
+    pub base_buffer_size: usize,
+    /// Maximum buffer size
+    pub max_buffer_size: usize,
+    /// High priority threshold
+    pub high_priority_threshold: usize,
+    /// Low priority threshold
+    pub low_priority_threshold: usize,
+}
+
+impl Default for SchedulerPolicy {
+    fn default() -> Self {
+        Self {
+            base_buffer_size: 64 * 1024,              // 64KB
+            max_buffer_size: 4 * 1024 * 1024,         // 4MB
+            high_priority_threshold: 1024 * 1024,     // 1MB
+            low_priority_threshold: 10 * 1024 * 1024, // 10MB
+        }
+    }
+}
+
+impl SchedulerPolicy {
+    /// Convert facade policy to io-core scheduler config.
+    pub fn to_core_config(&self) -> rustfs_io_core::IoSchedulerConfig {
+        rustfs_io_core::IoSchedulerConfig {
+            base_buffer_size: self.base_buffer_size,
+            max_buffer_size: self.max_buffer_size,
+            high_priority_size_threshold: self.high_priority_threshold,
+            low_priority_size_threshold: self.low_priority_threshold,
+            ..rustfs_io_core::IoSchedulerConfig::default()
+        }
+    }
+}
+
+/// Scheduler manager
+pub struct SchedulerManager {
+    config: SchedulerPolicy,
+    core_config: rustfs_io_core::IoSchedulerConfig,
+    scheduler: Arc<CoreIoScheduler>,
+}
+
+impl SchedulerManager {
+    /// Create a new scheduler manager
+    pub fn new(
+        base_buffer_size: usize,
+        max_buffer_size: usize,
+        high_priority_threshold: usize,
+        low_priority_threshold: usize,
+    ) -> Self {
+        Self::from_policy(SchedulerPolicy {
+            base_buffer_size,
+            max_buffer_size,
+            high_priority_threshold,
+            low_priority_threshold,
+        })
+    }
+
+    /// Create a scheduler manager from facade policy.
+    pub fn from_policy(config: SchedulerPolicy) -> Self {
+        let core_config = config.to_core_config();
+
+        Self {
+            config,
+            core_config: core_config.clone(),
+            scheduler: Arc::new(CoreIoScheduler::new(core_config)),
+        }
+    }
+
+    /// Get the configuration
+    pub fn config(&self) -> &SchedulerPolicy {
+        &self.config
+    }
+
+    /// Get the derived io-core scheduler config.
+    pub fn core_config(&self) -> &rustfs_io_core::IoSchedulerConfig {
+        &self.core_config
+    }
+
+    /// Get the scheduler
+    pub fn scheduler(&self) -> Arc<CoreIoScheduler> {
+        self.scheduler.clone()
+    }
+
+    /// Create an I/O strategy
+    pub fn create_strategy(&self) -> IoStrategy {
+        IoStrategy::new(self.config, self.scheduler.clone())
+    }
+
+    /// Calculate buffer size
+    pub fn calculate_buffer_size(
+        &self,
+        file_size: i64,
+        media: StorageMedia,
+        pattern: AccessPattern,
+        load: IoLoadLevel,
+        concurrent: usize,
+    ) -> usize {
+        let strategy = self.create_strategy();
+        strategy.calculate_buffer_size(file_size, media, pattern, load, concurrent)
+    }
+
+    /// Get I/O priority
+    pub fn get_priority(&self, size: i64) -> IoPriority {
+        IoPriority::from_size(size, self.config.high_priority_threshold, self.config.low_priority_threshold)
+    }
+}
+
+/// I/O strategy
+pub struct IoStrategy {
+    config: SchedulerPolicy,
+    scheduler: Arc<CoreIoScheduler>,
+}
+
+impl IoStrategy {
+    fn new(config: SchedulerPolicy, scheduler: Arc<CoreIoScheduler>) -> Self {
+        Self { config, scheduler }
+    }
+
+    /// Calculate buffer size with multi-factor strategy
+    pub fn calculate_buffer_size(
+        &self,
+        file_size: i64,
+        media: StorageMedia,
+        pattern: AccessPattern,
+        load: IoLoadLevel,
+        concurrent: usize,
+    ) -> usize {
+        // Create scheduling context
+        let _ctx = IoSchedulingContext::new(file_size, self.config.base_buffer_size)
+            .with_sequential(matches!(pattern, AccessPattern::Sequential))
+            .with_media(media);
+
+        // Get base buffer size from core scheduler
+        let permit_wait = Duration::from_millis(10); // Default wait time
+        let is_sequential = matches!(pattern, AccessPattern::Sequential);
+        let core_strategy = self.scheduler.calculate_strategy(file_size, permit_wait, is_sequential);
+        let base_size = core_strategy.buffer_size;
+
+        // Apply multi-factor adjustments
+        let adjusted_size = self.apply_adjustments(base_size, media, pattern, load, concurrent);
+
+        // Record metrics
+        io_metrics::record_io_scheduler_decision(adjusted_size, load.as_str(), pattern.as_str());
+
+        adjusted_size.min(self.config.max_buffer_size)
+    }
+
+    fn apply_adjustments(
+        &self,
+        base_size: usize,
+        media: StorageMedia,
+        pattern: AccessPattern,
+        load: IoLoadLevel,
+        concurrent: usize,
+    ) -> usize {
+        let mut size = base_size;
+
+        // Media adjustment
+        size = match media {
+            StorageMedia::Nvme => (size as f64 * 1.5) as usize,
+            StorageMedia::Ssd => (size as f64 * 1.2) as usize,
+            StorageMedia::Hdd => size,
+            _ => size,
+        };
+
+        // Pattern adjustment
+        size = match pattern {
+            AccessPattern::Sequential => (size as f64 * 1.5) as usize,
+            AccessPattern::Random => (size as f64 * 0.5) as usize,
+            _ => size,
+        };
+
+        // Load adjustment
+        size = match load {
+            IoLoadLevel::Low => (size as f64 * 1.2) as usize,
+            IoLoadLevel::Medium => size,
+            IoLoadLevel::High => (size as f64 * 0.7) as usize,
+            IoLoadLevel::Critical => (size as f64 * 0.5) as usize,
+        };
+
+        // Concurrency adjustment
+        if concurrent > 10 {
+            size = (size as f64 * 0.8) as usize;
+        }
+
+        size
+    }
+
+    /// Get the configuration
+    pub fn config(&self) -> &SchedulerPolicy {
+        &self.config
+    }
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    #[test]
+    fn test_scheduler_config() {
+        let config = SchedulerPolicy::default();
+        assert!(config.base_buffer_size < config.max_buffer_size);
+    }
+
+    #[test]
+    fn test_scheduler_policy_to_core_config() {
+        let policy = SchedulerPolicy::default();
+        let core = policy.to_core_config();
+        assert_eq!(core.base_buffer_size, policy.base_buffer_size);
+        assert_eq!(core.max_buffer_size, policy.max_buffer_size);
+        assert_eq!(core.high_priority_size_threshold, policy.high_priority_threshold);
+        assert_eq!(core.low_priority_size_threshold, policy.low_priority_threshold);
+    }
+
+    #[test]
+    fn test_scheduler_default_thresholds_remain_stable() {
+        let policy = SchedulerPolicy::default();
+        assert_eq!(policy.base_buffer_size, 64 * 1024);
+        assert_eq!(policy.max_buffer_size, 4 * 1024 * 1024);
+        assert_eq!(policy.high_priority_threshold, 1024 * 1024);
+        assert_eq!(policy.low_priority_threshold, 10 * 1024 * 1024);
+    }
+
+    #[test]
+    fn test_scheduler_priority_boundaries_remain_stable() {
+        let policy = SchedulerPolicy::default();
+        let manager = SchedulerManager::from_policy(policy);
+
+        assert_eq!(manager.get_priority(1_048_575), IoPriority::High);
+        assert_eq!(manager.get_priority(1_048_576), IoPriority::Normal);
+        assert_eq!(manager.get_priority(10_485_760), IoPriority::Normal);
+        assert_eq!(manager.get_priority(10_485_761), IoPriority::Low);
+    }
+
+    #[test]
+    fn test_scheduler_manager() {
+        let manager = SchedulerManager::new(1024, 4096, 512, 2048);
+        let priority = manager.get_priority(100);
+        assert!(priority.is_high());
+    }
+
+    #[test]
+    fn test_io_strategy() {
+        let manager = SchedulerManager::new(1024, 4096, 512, 2048);
+        let strategy = manager.create_strategy();
+
+        let size = strategy.calculate_buffer_size(1024 * 1024, StorageMedia::Ssd, AccessPattern::Sequential, IoLoadLevel::Low, 1);
+
+        assert!(size > 0);
+    }
+}

crates/concurrency/src/timeout.rs

@@ -0,0 +1,222 @@
+// Copyright 2024 RustFS Team
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+//! Timeout management for operations
+
+use rustfs_io_core::{TimeoutConfig as CoreTimeoutConfig, TimeoutError, calculate_adaptive_timeout};
+use std::time::{Duration, Instant};
+use tokio_util::sync::CancellationToken;
+
+/// Facade policy for the concurrency-layer timeout manager.
+#[derive(Debug, Clone, Copy)]
+pub struct TimeoutManagerPolicy {
+    /// Default timeout duration
+    pub default_timeout: Duration,
+    /// Maximum timeout duration
+    pub max_timeout: Duration,
+    /// Minimum timeout floor (prevents dynamic calculation from going too low).
+    pub min_timeout: Duration,
+    /// Enable dynamic timeout calculation
+    pub enable_dynamic: bool,
+}
+
+impl Default for TimeoutManagerPolicy {
+    fn default() -> Self {
+        Self {
+            default_timeout: Duration::from_secs(30),
+            max_timeout: Duration::from_secs(300),
+            min_timeout: Duration::from_secs(5),
+            enable_dynamic: true,
+        }
+    }
+}
+
+impl TimeoutManagerPolicy {
+    /// Convert the facade policy into the reusable io-core timeout configuration.
+    ///
+    /// This keeps the concurrency layer explicitly wired to the shared core
+    /// timeout primitives without changing the facade's public behavior.
+    pub fn to_core_config(&self) -> CoreTimeoutConfig {
+        CoreTimeoutConfig {
+            base_timeout: self.default_timeout,
+            timeout_per_mb: Duration::ZERO,
+            max_timeout: self.max_timeout,
+            min_timeout: self.min_timeout,
+            get_object_timeout: self.default_timeout,
+            put_object_timeout: self.max_timeout,
+            list_objects_timeout: self.default_timeout,
+            enable_dynamic_timeout: self.enable_dynamic,
+        }
+    }
+}
+
+/// Timeout manager
+pub struct TimeoutManager {
+    config: TimeoutManagerPolicy,
+    core_config: CoreTimeoutConfig,
+}
+
+impl TimeoutManager {
+    /// Create a new timeout manager
+    pub fn new(default_timeout: Duration, max_timeout: Duration, enable_dynamic: bool) -> Self {
+        let min_timeout = default_timeout.min(max_timeout);
+        Self::from_policy(TimeoutManagerPolicy {
+            default_timeout,
+            max_timeout,
+            min_timeout,
+            enable_dynamic,
+        })
+    }
+
+    /// Create a new timeout manager from the facade policy type.
+    pub fn from_policy(config: TimeoutManagerPolicy) -> Self {
+        let config = TimeoutManagerPolicy {
+            // Guard clamp(min, max) from panic when callers provide an
+            // out-of-order policy (or very small max_timeout).
+            min_timeout: config.min_timeout.min(config.max_timeout),
+            ..config
+        };
+        let core_config = config.to_core_config();
+        Self { config, core_config }
+    }
+
+    /// Get the configuration
+    pub fn config(&self) -> &TimeoutManagerPolicy {
+        &self.config
+    }
+
+    /// Get the derived io-core timeout configuration.
+    pub fn core_config(&self) -> &CoreTimeoutConfig {
+        &self.core_config
+    }
+
+    /// Calculate timeout for a given size
+    pub fn calculate_timeout(&self, size: u64, _history: &[Duration]) -> Duration {
+        if !self.config.enable_dynamic {
+            return self.config.default_timeout;
+        }
+
+        calculate_adaptive_timeout(self.core_config.base_timeout, None, 0, size)
+            .clamp(self.core_config.min_timeout, self.core_config.max_timeout)
+    }
+
+    /// Wrap an operation with timeout control
+    pub async fn wrap_operation<F, T, E>(&self, operation: F, timeout: Option<Duration>) -> Result<T, TimeoutError>
+    where
+        F: std::future::Future<Output = Result<T, E>>,
+        E: Into<TimeoutError>,
+    {
+        let timeout = timeout.unwrap_or(self.config.default_timeout);
+
+        match tokio::time::timeout(timeout, operation).await {
+            Ok(Ok(result)) => Ok(result),
+            Ok(Err(e)) => Err(e.into()),
+            Err(_) => Err(TimeoutError::TimedOut(timeout)),
+        }
+    }
+
+    /// Create a timeout guard for manual timeout control
+    pub fn create_guard(&self, timeout: Option<Duration>) -> TimeoutGuard {
+        TimeoutGuard::new(timeout.unwrap_or(self.core_config.base_timeout))
+    }
+}
+
+/// Timeout guard for manual timeout control
+pub struct TimeoutGuard {
+    timeout: Duration,
+    start: Instant,
+    cancel_token: CancellationToken,
+}
+
+impl TimeoutGuard {
+    fn new(timeout: Duration) -> Self {
+        Self {
+            timeout,
+            start: Instant::now(),
+            cancel_token: CancellationToken::new(),
+        }
+    }
+
+    /// Check if timeout has elapsed
+    pub fn is_timed_out(&self) -> bool {
+        self.start.elapsed() > self.timeout
+    }
+
+    /// Get remaining time
+    pub fn remaining(&self) -> Duration {
+        self.timeout.saturating_sub(self.start.elapsed())
+    }
+
+    /// Get the cancellation token
+    pub fn cancel_token(&self) -> CancellationToken {
+        self.cancel_token.clone()
+    }
+
+    /// Cancel the operation
+    pub fn cancel(&self) {
+        self.cancel_token.cancel();
+    }
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    #[test]
+    fn test_timeout_config() {
+        let config = TimeoutManagerPolicy::default();
+        assert!(config.default_timeout < config.max_timeout);
+    }
+
+    #[test]
+    fn test_timeout_policy_to_core_config() {
+        let policy = TimeoutManagerPolicy::default();
+        let core = policy.to_core_config();
+        assert_eq!(core.base_timeout, policy.default_timeout);
+        assert_eq!(core.max_timeout, policy.max_timeout);
+        assert_eq!(core.min_timeout, policy.min_timeout);
+        assert_eq!(core.get_object_timeout, policy.default_timeout);
+        assert!(core.enable_dynamic_timeout);
+    }
+
+    #[test]
+    fn test_timeout_manager_new_sanitizes_min_timeout_with_small_max_timeout() {
+        let manager = TimeoutManager::new(Duration::from_secs(1), Duration::from_secs(1), true);
+        let timeout = manager.calculate_timeout(1024, &[]);
+        assert_eq!(timeout, Duration::from_secs(1));
+    }
+
+    #[test]
+    fn test_timeout_manager_from_policy_sanitizes_min_timeout() {
+        let manager = TimeoutManager::from_policy(TimeoutManagerPolicy {
+            default_timeout: Duration::from_secs(30),
+            max_timeout: Duration::from_secs(1),
+            min_timeout: Duration::from_secs(5),
+            enable_dynamic: true,
+        });
+
+        assert_eq!(manager.config().min_timeout, Duration::from_secs(1));
+        assert_eq!(manager.core_config().min_timeout, Duration::from_secs(1));
+    }
+
+    #[tokio::test]
+    async fn test_wrap_operation_success() {
+        let manager = TimeoutManager::new(Duration::from_secs(5), Duration::from_secs(10), true);
+
+        let result = manager.wrap_operation(async { Ok::<_, TimeoutError>(42) }, None).await;
+
+        assert!(result.is_ok());
+        assert_eq!(result.unwrap(), 42);
+    }
+}

crates/concurrency/src/workers.rs

@@ -15,19 +15,14 @@
 //! Worker slot limiter used by long-running background workflows.
 
 use std::sync::Arc;
-use tokio::sync::{Semaphore, watch};
+use tokio::sync::{Mutex, Notify};
 use tracing::{debug, trace};
 
 /// Cooperative worker-slot controller for async tasks.
-///
-/// Slot acquisition is backed by a fair FIFO [`Semaphore`], while drain
-/// waiters observe the in-use count through a dedicated [`watch`] channel.
-/// The two wakeup sources are deliberately separate so a drain waiter can
-/// never consume a wakeup destined for a pending [`take`](Self::take).
 pub struct Workers {
-    semaphore: Semaphore,         // Available working slots
-    in_use: watch::Sender<usize>, // Slots currently held; drain waiters watch it
-    limit: usize,                 // Maximum number of concurrent jobs
+    available: Mutex<usize>, // Available working slots
+    notify: Notify,          // Used to notify waiting tasks
+    limit: usize,            // Maximum number of concurrent jobs
 }
 
 impl Workers {
@@ -36,70 +31,86 @@ impl Workers {
         if n == 0 {
             return Err("n must be > 0");
         }
-        if n > Semaphore::MAX_PERMITS {
-            return Err("n exceeds the maximum supported number of slots");
-        }
 
         Ok(Arc::new(Self {
-            semaphore: Semaphore::new(n),
-            in_use: watch::Sender::new(0),
+            available: Mutex::new(n),
+            notify: Notify::new(),
             limit: n,
         }))
     }
 
     /// Acquire a worker slot, waiting until one becomes available.
     pub async fn take(&self) {
-        let permit = match self.semaphore.acquire().await {
-            Ok(permit) => permit,
-            // The semaphore is owned by `self` and never closed.
-            Err(_) => unreachable!("worker semaphore is never closed"),
-        };
-        permit.forget(); // returned manually via give()
-        self.in_use.send_modify(|held| *held += 1);
-        trace!(
-            event = "worker_slot.acquire",
-            component = "concurrency",
-            subsystem = "workers",
-            state = "granted",
-            available_slots = self.semaphore.available_permits(),
-            total_slots = self.limit,
-            permits_in_use = *self.in_use.borrow(),
-            "worker slot updated"
-        );
+        loop {
+            let mut available = self.available.lock().await;
+            if *available == 0 {
+                trace!(
+                    event = "worker_slot.acquire",
+                    component = "concurrency",
+                    subsystem = "workers",
+                    state = "waiting",
+                    available_slots = *available,
+                    total_slots = self.limit,
+                    "worker slot pending"
+                );
+                drop(available);
+                self.notify.notified().await;
+            } else {
+                *available -= 1;
+                trace!(
+                    event = "worker_slot.acquire",
+                    component = "concurrency",
+                    subsystem = "workers",
+                    state = "granted",
+                    available_slots = *available,
+                    total_slots = self.limit,
+                    permits_in_use = self.limit.saturating_sub(*available),
+                    "worker slot updated"
+                );
+                break;
+            }
+        }
     }
 
     /// Release a worker slot.
-    ///
-    /// A `give()` that is not paired with a prior [`take`](Self::take) is
-    /// clamped: it never inflates capacity beyond `limit`.
     pub async fn give(&self) {
-        let mut released = false;
-        self.in_use.send_modify(|held| {
-            if *held > 0 {
-                *held -= 1;
-                released = true;
-            }
-        });
-        if released {
-            self.semaphore.add_permits(1);
-        }
+        let mut available = self.available.lock().await;
+        *available = (*available).saturating_add(1).min(self.limit); // avoid over-release beyond limit
         trace!(
             event = "worker_slot.release",
             component = "concurrency",
             subsystem = "workers",
             state = "released",
-            available_slots = self.semaphore.available_permits(),
+            available_slots = *available,
             total_slots = self.limit,
-            permits_in_use = *self.in_use.borrow(),
+            permits_in_use = self.limit.saturating_sub(*available),
             "worker slot updated"
         );
+        self.notify.notify_one(); // Notify a waiting task
     }
 
     /// Wait until all worker slots are released.
     pub async fn wait(&self) {
-        let mut rx = self.in_use.subscribe();
-        // The sender is owned by `self`, so it outlives this borrow.
-        let _ = rx.wait_for(|&held| held == 0).await;
+        loop {
+            {
+                let available = self.available.lock().await;
+                if *available == self.limit {
+                    break;
+                }
+                trace!(
+                    event = "worker_slot.wait",
+                    component = "concurrency",
+                    subsystem = "workers",
+                    state = "waiting",
+                    available_slots = *available,
+                    total_slots = self.limit,
+                    permits_in_use = self.limit.saturating_sub(*available),
+                    "worker drain pending"
+                );
+            }
+            // Wait until all slots are freed
+            self.notify.notified().await;
+        }
         debug!(
             event = "worker_slot.wait",
             component = "concurrency",
@@ -114,7 +125,7 @@ impl Workers {
 
     /// Return the current number of available worker slots.
     pub async fn available(&self) -> usize {
-        self.limit.saturating_sub(*self.in_use.borrow())
+        *self.available.lock().await
     }
 }
 
@@ -122,7 +133,7 @@ impl Workers {
 mod tests {
     use super::*;
     use std::time::Duration;
-    use tokio::time::{sleep, timeout};
+    use tokio::time::sleep;
 
     #[tokio::test]
     async fn test_workers() {
@@ -157,64 +168,4 @@ mod tests {
 
         assert_eq!(workers.available().await, 2);
     }
-
-    /// Regression for S08: with limit=1, a pending drain waiter must never
-    /// steal the wakeup destined for a pending taker.
-    #[tokio::test]
-    async fn test_drain_waiter_does_not_steal_take_wakeup() {
-        let workers = Workers::new(1).unwrap();
-        workers.take().await;
-
-        let taker = {
-            let workers = workers.clone();
-            tokio::spawn(async move { workers.take().await })
-        };
-        let drainer = {
-            let workers = workers.clone();
-            tokio::spawn(async move { workers.wait().await })
-        };
-        // Let both tasks block before releasing the slot.
-        sleep(Duration::from_millis(50)).await;
-        workers.give().await;
-
-        timeout(Duration::from_secs(1), taker)
-            .await
-            .expect("taker must be woken by give()")
-            .unwrap();
-
-        workers.give().await;
-        timeout(Duration::from_secs(1), drainer)
-            .await
-            .expect("drain waiter must complete once all slots are free")
-            .unwrap();
-        assert_eq!(workers.available().await, 1);
-    }
-
-    /// Regression for S17: two rapid give() calls must wake two pending
-    /// takers; permits must not merge into a single wakeup.
-    #[tokio::test]
-    async fn test_rapid_gives_wake_all_pending_takers() {
-        let workers = Workers::new(2).unwrap();
-        workers.take().await;
-        workers.take().await;
-
-        let takers: Vec<_> = (0..2)
-            .map(|_| {
-                let workers = workers.clone();
-                tokio::spawn(async move { workers.take().await })
-            })
-            .collect();
-        // Let both takers queue up before any slot is released.
-        sleep(Duration::from_millis(50)).await;
-        workers.give().await;
-        workers.give().await;
-
-        for taker in takers {
-            timeout(Duration::from_secs(1), taker)
-                .await
-                .expect("every pending taker must be woken")
-                .unwrap();
-        }
-        assert_eq!(workers.available().await, 0);
-    }
 }

crates/config/Cargo.toml

@@ -25,9 +25,9 @@ keywords = ["configuration", "settings", "management", "rustfs", "Minio"]
 categories = ["web-programming", "development-tools", "config"]
 
 [dependencies]
-const-str = { workspace = true, optional = true, features = ["std", "proc"] }
-serde = { workspace = true, optional = true, features = ["derive"] }
-serde_json = { workspace = true, optional = true, features = ["raw_value"] }
+const-str = { workspace = true, optional = true }
+serde = { workspace = true, optional = true }
+serde_json = { workspace = true, optional = true }
 
 [lints]
 workspace = true

crates/config/README.md

@@ -62,10 +62,6 @@ Current guidance:
 - `RUSTFS_CORS_ALLOWED_ORIGINS` defaults to empty, so the S3 endpoint emits no generic CORS headers unless configured. Set `*` for wildcard origins without credentials, or a comma-separated allow-list for credentialed explicit origins.
 - `RUSTFS_CONSOLE_CORS_ALLOWED_ORIGINS` defaults to `*` for the console service.
 
-## Browser redirect environment variables
-
-- `RUSTFS_BROWSER_REDIRECT_URL` sets the externally reachable browser origin used for OIDC callback, console success redirect, and logout fallback URLs. Configure it to the public scheme and authority without a path, for example `https://console.example.com`. In load-balancer deployments, keep OIDC authorize and callback requests on the same backend node because the in-flight OIDC `state` is local to the RustFS node.
-
 ## Scanner environment aliases
 
 - `RUSTFS_SCANNER_SPEED` (canonical, also accepts `MINIO_SCANNER_SPEED`)

crates/config/src/audit/nats.rs

@@ -25,11 +25,8 @@ pub const ENV_AUDIT_NATS_TLS_CLIENT_KEY: &str = "RUSTFS_AUDIT_NATS_TLS_CLIENT_KE
 pub const ENV_AUDIT_NATS_TLS_REQUIRED: &str = "RUSTFS_AUDIT_NATS_TLS_REQUIRED";
 pub const ENV_AUDIT_NATS_QUEUE_DIR: &str = "RUSTFS_AUDIT_NATS_QUEUE_DIR";
 pub const ENV_AUDIT_NATS_QUEUE_LIMIT: &str = "RUSTFS_AUDIT_NATS_QUEUE_LIMIT";
-pub const ENV_AUDIT_NATS_JETSTREAM_ENABLE: &str = "RUSTFS_AUDIT_NATS_JETSTREAM_ENABLE";
-pub const ENV_AUDIT_NATS_JETSTREAM_STREAM_NAME: &str = "RUSTFS_AUDIT_NATS_JETSTREAM_STREAM_NAME";
-pub const ENV_AUDIT_NATS_JETSTREAM_ACK_TIMEOUT_SECS: &str = "RUSTFS_AUDIT_NATS_JETSTREAM_ACK_TIMEOUT_SECS";
 
-pub const ENV_AUDIT_NATS_KEYS: &[&str; 16] = &[
+pub const ENV_AUDIT_NATS_KEYS: &[&str; 13] = &[
     ENV_AUDIT_NATS_ENABLE,
     ENV_AUDIT_NATS_ADDRESS,
     ENV_AUDIT_NATS_SUBJECT,
@@ -43,9 +40,6 @@ pub const ENV_AUDIT_NATS_KEYS: &[&str; 16] = &[
     ENV_AUDIT_NATS_TLS_REQUIRED,
     ENV_AUDIT_NATS_QUEUE_DIR,
     ENV_AUDIT_NATS_QUEUE_LIMIT,
-    ENV_AUDIT_NATS_JETSTREAM_ENABLE,
-    ENV_AUDIT_NATS_JETSTREAM_STREAM_NAME,
-    ENV_AUDIT_NATS_JETSTREAM_ACK_TIMEOUT_SECS,
 ];
 
 pub const AUDIT_NATS_KEYS: &[&str] = &[
@@ -62,8 +56,5 @@ pub const AUDIT_NATS_KEYS: &[&str] = &[
     crate::NATS_TLS_REQUIRED,
     crate::NATS_QUEUE_DIR,
     crate::NATS_QUEUE_LIMIT,
-    crate::NATS_JETSTREAM_ENABLE,
-    crate::NATS_JETSTREAM_STREAM_NAME,
-    crate::NATS_JETSTREAM_ACK_TIMEOUT_SECS,
     crate::COMMENT_KEY,
 ];

crates/config/src/constants/api.rs

@@ -1,92 +0,0 @@
-//  Copyright 2024 RustFS Team
-//
-//  Licensed under the Apache License, Version 2.0 (the "License");
-//  you may not use this file except in compliance with the License.
-//  You may obtain a copy of the License at
-//
-//      http://www.apache.org/licenses/LICENSE-2.0
-//
-//  Unless required by applicable law or agreed to in writing, software
-//  distributed under the License is distributed on an "AS IS" BASIS,
-//  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-//  See the License for the specific language governing permissions and
-//  limitations under the License.
-
-/// Enable or disable per-client rate limiting for the S3 API.
-///
-/// When enabled (and `RUSTFS_API_RATE_LIMIT_RPM` > 0), requests are throttled
-/// per client IP using a token bucket; over-limit requests receive
-/// `429 Too Many Requests` with a `Retry-After` header. Internode RPC/gRPC,
-/// health probes, and the console (which has its own limiter) are exempt.
-/// Environment variable: RUSTFS_API_RATE_LIMIT_ENABLE
-/// Example: RUSTFS_API_RATE_LIMIT_ENABLE=true
-pub const ENV_API_RATE_LIMIT_ENABLE: &str = "RUSTFS_API_RATE_LIMIT_ENABLE";
-
-/// Default for `RUSTFS_API_RATE_LIMIT_ENABLE`.
-///
-/// Disabled by default: RustFS ships permissive and operators opt in to
-/// abuse-protection hardening. When disabled the request path is unchanged.
-pub const DEFAULT_API_RATE_LIMIT_ENABLE: bool = false;
-
-/// Sustained S3 API request budget per client IP, in requests per minute.
-///
-/// `0` means unlimited (rate limiting stays inert even when enabled).
-/// Environment variable: RUSTFS_API_RATE_LIMIT_RPM
-/// Example: RUSTFS_API_RATE_LIMIT_RPM=6000
-pub const ENV_API_RATE_LIMIT_RPM: &str = "RUSTFS_API_RATE_LIMIT_RPM";
-
-/// Default for `RUSTFS_API_RATE_LIMIT_RPM`.
-///
-/// `0` (unlimited) so that setting only the enable switch cannot throttle
-/// traffic by surprise; operators must choose an explicit budget.
-pub const DEFAULT_API_RATE_LIMIT_RPM: u32 = 0;
-
-/// Burst capacity per client IP (maximum tokens in the bucket).
-///
-/// Allows short spikes above the sustained rate. `0` means "same as RPM".
-/// Environment variable: RUSTFS_API_RATE_LIMIT_BURST
-/// Example: RUSTFS_API_RATE_LIMIT_BURST=200
-pub const ENV_API_RATE_LIMIT_BURST: &str = "RUSTFS_API_RATE_LIMIT_BURST";
-
-/// Default for `RUSTFS_API_RATE_LIMIT_BURST` (`0` = same as RPM).
-pub const DEFAULT_API_RATE_LIMIT_BURST: u32 = 0;
-
-/// Sustained S3 API request budget per addressed bucket, in requests per
-/// minute — a collective ceiling shared by all clients of that bucket.
-///
-/// Complements the per-client-IP dimension: it protects the server from one
-/// hot bucket regardless of how many client IPs the traffic comes from. `0`
-/// disables the bucket dimension. Requires `RUSTFS_API_RATE_LIMIT_ENABLE`.
-/// Environment variable: RUSTFS_API_RATE_LIMIT_BUCKET_RPM
-/// Example: RUSTFS_API_RATE_LIMIT_BUCKET_RPM=60000
-pub const ENV_API_RATE_LIMIT_BUCKET_RPM: &str = "RUSTFS_API_RATE_LIMIT_BUCKET_RPM";
-
-/// Default for `RUSTFS_API_RATE_LIMIT_BUCKET_RPM` (`0` = dimension disabled).
-pub const DEFAULT_API_RATE_LIMIT_BUCKET_RPM: u32 = 0;
-
-/// Burst capacity per bucket (maximum tokens in the bucket-dimension bucket).
-///
-/// `0` means "same as `RUSTFS_API_RATE_LIMIT_BUCKET_RPM`".
-/// Environment variable: RUSTFS_API_RATE_LIMIT_BUCKET_BURST
-/// Example: RUSTFS_API_RATE_LIMIT_BUCKET_BURST=2000
-pub const ENV_API_RATE_LIMIT_BUCKET_BURST: &str = "RUSTFS_API_RATE_LIMIT_BUCKET_BURST";
-
-/// Default for `RUSTFS_API_RATE_LIMIT_BUCKET_BURST` (`0` = same as bucket RPM).
-pub const DEFAULT_API_RATE_LIMIT_BUCKET_BURST: u32 = 0;
-
-/// Maximum concurrently served connections on the main API listener.
-///
-/// `0` (the default) means unlimited. When set, the accept loop stops
-/// accepting once the cap is reached and lets the kernel backlog absorb
-/// bursts, releasing capacity as connections close. This bounds file
-/// descriptor and memory usage under a connection flood.
-///
-/// The cap covers everything on the main listener — S3, admin, console,
-/// and internode gRPC — so size it well above peer-node count plus the
-/// expected client concurrency.
-/// Environment variable: RUSTFS_API_MAX_CONNECTIONS
-/// Example: RUSTFS_API_MAX_CONNECTIONS=10000
-pub const ENV_API_MAX_CONNECTIONS: &str = "RUSTFS_API_MAX_CONNECTIONS";
-
-/// Default for `RUSTFS_API_MAX_CONNECTIONS` (`0` = unlimited).
-pub const DEFAULT_API_MAX_CONNECTIONS: usize = 0;

crates/config/src/constants/app.rs

@@ -143,41 +143,6 @@ pub const ENV_MINIO_CI: &str = "MINIO_CI";
 /// Default flag value for bypassing local physical disk independence checks.
 pub const DEFAULT_UNSAFE_BYPASS_DISK_CHECK: bool = false;
 
-/// Environment variable selecting how startup waits for DNS/topology
-/// convergence in multi-node setups. One of `auto` (default), `orchestrated`,
-/// `bounded`, or `fail-fast`.
-///
-/// - `orchestrated`: wait effectively indefinitely for recoverable DNS/topology
-///   errors so the process stays Running (readiness stays false) instead of
-///   exiting and triggering a Kubernetes pod restart loop.
-/// - `bounded`: wait for a finite window (see
-///   `ENV_STARTUP_TOPOLOGY_WAIT_TIMEOUT`) then fail with an actionable error.
-/// - `fail-fast`: fail on the first non-transient resolution error (CI/local).
-/// - `auto`: distributed URL endpoints use orchestrated on Kubernetes and
-///   bounded otherwise; local path endpoints use fail-fast (no DNS to await).
-pub const ENV_STARTUP_TOPOLOGY_WAIT_MODE: &str = "RUSTFS_STARTUP_TOPOLOGY_WAIT_MODE";
-
-/// Environment variable bounding how long startup waits for topology/DNS
-/// convergence before failing in `bounded` mode. Accepts durations such as
-/// `3m`, `90s`, `500ms`, or a bare number of seconds.
-pub const ENV_STARTUP_TOPOLOGY_WAIT_TIMEOUT: &str = "RUSTFS_STARTUP_TOPOLOGY_WAIT_TIMEOUT";
-
-/// Environment variable capping the per-attempt backoff delay while retrying
-/// startup topology/DNS convergence. Accepts durations such as `8s`.
-pub const ENV_STARTUP_TOPOLOGY_RETRY_MAX_DELAY: &str = "RUSTFS_STARTUP_TOPOLOGY_RETRY_MAX_DELAY";
-
-/// Environment variable injected into every Kubernetes pod; its presence marks
-/// an orchestrated deployment for startup topology auto-detection.
-pub const ENV_KUBERNETES_SERVICE_HOST: &str = "KUBERNETES_SERVICE_HOST";
-
-/// Default bounded-mode wait window (seconds) for non-Kubernetes distributed
-/// deployments before startup topology convergence is declared failed.
-pub const DEFAULT_STARTUP_TOPOLOGY_WAIT_TIMEOUT_SECS: u64 = 180;
-
-/// Default per-attempt backoff cap (seconds) while retrying startup
-/// topology/DNS convergence.
-pub const DEFAULT_STARTUP_TOPOLOGY_RETRY_MAX_DELAY_SECS: u64 = 8;
-
 /// Environment variable for server access key.
 pub const ENV_RUSTFS_ACCESS_KEY: &str = "RUSTFS_ACCESS_KEY";
 
@@ -209,12 +174,6 @@ pub const ENV_RUSTFS_CONSOLE_ENABLE: &str = "RUSTFS_CONSOLE_ENABLE";
 /// Environment variable for console server address.
 pub const ENV_RUSTFS_CONSOLE_ADDRESS: &str = "RUSTFS_CONSOLE_ADDRESS";
 
-/// Public browser entrypoint used to build OIDC callback and console redirects.
-///
-/// This should be the externally reachable scheme and authority, without a path.
-/// Example: `RUSTFS_BROWSER_REDIRECT_URL=https://console.example.com`.
-pub const ENV_RUSTFS_BROWSER_REDIRECT_URL: &str = "RUSTFS_BROWSER_REDIRECT_URL";
-
 /// Environment variable for server tls path.
 pub const ENV_RUSTFS_TLS_PATH: &str = "RUSTFS_TLS_PATH";
 
@@ -389,7 +348,6 @@ mod tests {
             RUSTFS_TLS_CERT,
             DEFAULT_ADDRESS,
             DEFAULT_CONSOLE_ADDRESS,
-            ENV_RUSTFS_BROWSER_REDIRECT_URL,
         ];
 
         for constant in &string_constants {

crates/config/src/constants/capacity.rs

@@ -45,6 +45,9 @@ pub const ENV_CAPACITY_METRICS_INTERVAL: &str = "RUSTFS_CAPACITY_METRICS_INTERVA
 /// Environment variable for following symbolic links during capacity calculation
 pub const ENV_CAPACITY_FOLLOW_SYMLINKS: &str = "RUSTFS_CAPACITY_FOLLOW_SYMLINKS";
 
+/// Environment variable for maximum symlink follow depth
+pub const ENV_CAPACITY_MAX_SYMLINK_DEPTH: &str = "RUSTFS_CAPACITY_MAX_SYMLINK_DEPTH";
+
 /// Environment variable for enabling dynamic timeout calculation
 pub const ENV_CAPACITY_ENABLE_DYNAMIC_TIMEOUT: &str = "RUSTFS_CAPACITY_ENABLE_DYNAMIC_TIMEOUT";
 
@@ -54,6 +57,9 @@ pub const ENV_CAPACITY_MIN_TIMEOUT: &str = "RUSTFS_CAPACITY_MIN_TIMEOUT";
 /// Environment variable for maximum capacity calculation timeout
 pub const ENV_CAPACITY_MAX_TIMEOUT: &str = "RUSTFS_CAPACITY_MAX_TIMEOUT";
 
+/// Environment variable for progress stall detection timeout
+pub const ENV_CAPACITY_STALL_TIMEOUT: &str = "RUSTFS_CAPACITY_STALL_TIMEOUT";
+
 // ============================================================================
 // Default Values
 // ============================================================================
@@ -94,6 +100,10 @@ pub const DEFAULT_CAPACITY_METRICS_INTERVAL_SECS: u64 = 600;
 /// Default: false (disabled for safety)
 pub const DEFAULT_CAPACITY_FOLLOW_SYMLINKS: bool = false;
 
+/// Maximum symlink follow depth
+/// Default: 3 levels
+pub const DEFAULT_CAPACITY_MAX_SYMLINK_DEPTH: u8 = 3;
+
 /// Enable dynamic timeout calculation based on directory characteristics
 /// Default: true (enabled)
 pub const DEFAULT_CAPACITY_ENABLE_DYNAMIC_TIMEOUT: bool = true;
@@ -106,6 +116,10 @@ pub const DEFAULT_CAPACITY_MIN_TIMEOUT_SECS: u64 = 2;
 /// Default: 15 seconds
 pub const DEFAULT_CAPACITY_MAX_TIMEOUT_SECS: u64 = 15;
 
+/// Progress stall detection timeout in seconds
+/// Default: 20 seconds
+pub const DEFAULT_CAPACITY_STALL_TIMEOUT_SECS: u64 = 20;
+
 // ============================================================================
 // Tests
 // ============================================================================
@@ -125,8 +139,10 @@ mod tests {
         assert_eq!(ENV_CAPACITY_SAMPLE_RATE, "RUSTFS_CAPACITY_SAMPLE_RATE");
         assert_eq!(ENV_CAPACITY_METRICS_INTERVAL, "RUSTFS_CAPACITY_METRICS_INTERVAL");
         assert_eq!(ENV_CAPACITY_FOLLOW_SYMLINKS, "RUSTFS_CAPACITY_FOLLOW_SYMLINKS");
+        assert_eq!(ENV_CAPACITY_MAX_SYMLINK_DEPTH, "RUSTFS_CAPACITY_MAX_SYMLINK_DEPTH");
         assert_eq!(ENV_CAPACITY_ENABLE_DYNAMIC_TIMEOUT, "RUSTFS_CAPACITY_ENABLE_DYNAMIC_TIMEOUT");
         assert_eq!(ENV_CAPACITY_MIN_TIMEOUT, "RUSTFS_CAPACITY_MIN_TIMEOUT");
         assert_eq!(ENV_CAPACITY_MAX_TIMEOUT, "RUSTFS_CAPACITY_MAX_TIMEOUT");
+        assert_eq!(ENV_CAPACITY_STALL_TIMEOUT, "RUSTFS_CAPACITY_STALL_TIMEOUT");
     }
 }

crates/config/src/constants/drive.rs

@@ -39,11 +39,6 @@ pub const DEFAULT_DRIVE_WALKDIR_TIMEOUT_SECS: u64 = 5;
 pub const ENV_DRIVE_WALKDIR_STALL_TIMEOUT_SECS: &str = "RUSTFS_DRIVE_WALKDIR_STALL_TIMEOUT_SECS";
 pub const DEFAULT_DRIVE_WALKDIR_STALL_TIMEOUT_SECS: u64 = 5;
 
-/// Maximum time the metacache merge consumer waits for the next visible
-/// `walk_dir()` entry from a reader before detaching it from the merge.
-pub const ENV_DRIVE_WALKDIR_PEEK_TIMEOUT_SECS: &str = "RUSTFS_DRIVE_WALKDIR_PEEK_TIMEOUT_SECS";
-pub const DEFAULT_DRIVE_WALKDIR_PEEK_TIMEOUT_SECS: u64 = 10;
-
 /// Interval in seconds between active health probes for local and remote drives.
 pub const ENV_DRIVE_ACTIVE_CHECK_INTERVAL_SECS: &str = "RUSTFS_DRIVE_ACTIVE_CHECK_INTERVAL_SECS";
 pub const DEFAULT_DRIVE_ACTIVE_CHECK_INTERVAL_SECS: u64 = 15;

crates/config/src/constants/env.rs

@@ -41,24 +41,6 @@ pub const ENV_ILM_PROCESS_TIME_DEPRECATED: &str = "_RUSTFS_ILM_PROCESS_TIME";
 /// Default ILM process boundary in seconds (24h).
 pub const DEFAULT_ILM_PROCESS_TIME_SECS: i32 = 86400;
 
-/// **TEST/DEBUG ONLY** — lifecycle "day" length override, in seconds.
-///
-/// Modeled on Ceph RGW's `rgw_lc_debug_interval`. When set to a positive
-/// integer `N`, lifecycle rule evaluation treats one Days-based "day" as `N`
-/// seconds instead of [`DEFAULT_ILM_DAY_SECS`] (86400), so `Days`-based
-/// expiration/transition/noncurrent rules become exercisable in seconds under
-/// test. When unset (or set to `0`/an invalid value), behavior is byte-for-byte
-/// identical to production.
-///
-/// This accelerates data deletion and MUST NEVER be set in a production
-/// deployment. Enabling it emits a `WARN` log. It only rescales relative
-/// `Days` deadlines; absolute `Date`-based rules are unaffected.
-pub const ENV_ILM_DEBUG_DAY_SECS: &str = "RUSTFS_ILM_DEBUG_DAY_SECS";
-
-/// Number of seconds in one lifecycle "day" (86400) used as the default when
-/// [`ENV_ILM_DEBUG_DAY_SECS`] is unset.
-pub const DEFAULT_ILM_DAY_SECS: u32 = 86400;
-
 /// Medium-drawn lines separator
 /// This is used to separate words in environment variable names.
 pub const ENV_WORD_DELIMITER_DASH: &str = "-";