Compare commits

..

19 Commits

Author SHA1 Message Date
overtrue abca7ddfd7 test(kms): move the Vault KV2 Transit-wrapping doc guard into check_fips_wording.sh
`test_vault_kv2_sources_do_not_claim_transit_wrapping` asserted that four
`include_str!`-pinned files never describe the Vault KV2 backend as wrapping key
material through Vault's Transit engine. The invariant is a documentation-claim
invariant with no behavioral twin by construction, and the test form was weak in
both directions: it saw only four files (the same prose in a fifth file passed
silently) and it stopped compiling — rather than reporting a violation — as soon
as one of them was renamed.

Move the four literals verbatim into `scripts/check_fips_wording.sh`, which
already guards the adjacent cryptographic over-claim class (unsupported FIPS
validation wording) and is anchored to the same policy document. The guard now
greps every file under `crates/kms` for the same four case-sensitive literals and
separately reports a moved pinned source instead of failing to build.

`check_fips_wording.sh` previously ran only in `make pre-commit` / `pre-pr`, so
wire it into the Quick Checks job of both CI workflows to keep the invariant's
failure visibility at least as strong as the deleted test's.
2026-08-18 17:30:11 +08:00
Zhengchao An 68547ed7ea test(e2e): drop 187 no-op serial markers from the three densest e2e suites (#6209)
test(e2e): drop no-op serial markers from the three densest e2e suites

serial_test's #[serial] is an in-process mutex. cargo-nextest, this repo's
authoritative runner, executes every test in its own process, so the mutex is
never contended and the attribute is a documented no-op -- see the "Serial
execution & nextest profiles" section of docs/testing/README.md and the header
of .config/nextest.toml. Cross-process serialization is provided only by a
[test-groups] entry with max-threads = 1.

Remove 187 such markers (plus 3 now-unused imports) from the three
marker-densest modules of crates/e2e_test:

  multipart_auth_test.rs             85
  replication_extension_test.rs      68
  object_lock/object_lock_test.rs    34

None of these modules is covered by any [test-groups] entry, so the markers
were carrying no isolation for any lane. Every test in all three files builds
its own server via RustFSTestEnvironment::new(), which gives a UUID temp dir
and a uniquely allocated port -- the .config/nextest.toml comment on
replication_extension_test already states this explicitly ("parallel-safe by
construction"). No test mutates process env, binds a fixed port, or touches
process-global state, so nothing here needed temp_env or a test-group instead.

Pure deletion: 190 lines removed, 0 added, no test renamed, no behaviour
changed. serial_test stays in Cargo.toml -- 338 markers across 90 other files
in the crate still use it.

Refs: backlog#1846 (T1).
2026-08-18 17:05:30 +08:00
Zhengchao An f06a9c9cba fix(deps): record heal's bytes and crc-fast in the lockfile (#6211) 2026-08-18 17:04:55 +08:00
Zhengchao An bd296eff9e chore(io-core): drop eight zero-consumer modules (#6201) 2026-08-18 08:14:05 +00:00
houseme a5800033bd feat(heal): incremental status cursors and typed overlap policy (HS-06) (#6206)
* feat(heal): incremental heal status cursors and typed overlap policy (HS-06)

Incremental results: every retained result item now carries a monotonic
sequence number. The status query accepts a client cursor (sinceSeq on
the admin wire, Option<u64> internally) and returns only newer items,
plus nextSeq (the next cursor) and minSeq (the oldest retained
sequence). A cursor that fell behind the 1024-item retention window is
flagged through the existing truncated signal together with minSeq so
the client can restart from it. Sequencing survives task completion:
the completion archive stores the seq-stamped window. None keeps the
exact legacy full-snapshot behavior, so existing clients see no change.

Typed overlap handling for admin starts: RUSTFS_HEAL_OVERLAP_POLICY
(merge default | minio_error). Under minio_error, an admin start whose
path overlaps an active or queued task rejects with typed
already-running / overlapping-paths admission reasons (surfaced through
reason_label in the admin error body, sharing the existing
OperationAborted site because the s3s footprint ratchet forbids new
s3_error! sites); an exact duplicate start rejects with
already-running instead of silently merging. Scanner/autoheal/
read-repair sources never take the rejection path.

forceStart semantics now match MinIO for admin requests: an admin
forceStart first cancels the overlapping active admin task, then
admits the replacement.

Wire: the heal-control Query command grows an optional sinceSeq
(defaulted and skipped when absent, so older peers stay compatible);
the admin handler accepts the sinceSeq query parameter; the local
channel query gains the same cursor.

Tests: seq monotonicity and incremental slicing, window slide moving
minSeq with lagging-cursor flags, overlap matrix (same/containing/
contained/disjoint x policy x source), forceStart cancel-then-admit,
and the completion-archive window handoff.

Co-Authored-By: heihutu <heihutu@gmail.com>

* style: fmt after main merge

---------

Co-authored-by: heihutu <heihutu@gmail.com>
Co-authored-by: zhi22915 <qiuzgang@gmail.com>
2026-08-18 16:09:30 +08:00
cxymds a4ea36b298 fix(ecstore): single-flight remote disk recovery (#6096)
* fix(ecstore): single-flight remote disk recovery

* test(ecstore): cover remote recovery review cases

* test(ecstore): exercise recovery through disk slot

* test(ecstore): match format reads exactly

* test(ecstore): cover recovery teardown races

* style(ecstore): format recovery race tests

* fix(ecstore): remove unused health snapshot helper

* fix(ecstore): group recovery monitor test state

* fix(protos): preserve production source in compatibility checks
2026-08-18 14:49:53 +08:00
cxymds 4f68f117ba fix(lock): reap expired local lease guards (#6094)
* fix(lock): reap expired local lease guards

* fix(lock): reject refresh after guard expiry

* test(lock): isolate expired refresh regression

* style(lock): format expired refresh assertion
2026-08-18 14:49:39 +08:00
cxymds 0f30a75fdb fix(ecstore): resume remote shard reads once (#6091)
* fix(ecstore): preserve CopyObject producer errors

* fix(ecstore): resume remote shard reads once

* fix(app): resume preserved relocation I/O errors

* fix(ecstore): reserve remote read recovery budget
2026-08-18 14:49:19 +08:00
Zhengchao An 355c8d2e22 fix(admin): classify missing kms config by error variant (#6196) 2026-08-18 05:40:10 +00:00
Zhengchao An 60eb139db9 refactor: import x-amz-checksum header names from the shared constants (#6193)
Co-authored-by: houseme <housemecn@gmail.com>
2026-08-18 05:27:38 +00:00
hector 9ef059c908 ci(package): auto-trigger DEB/RPM packaging on releases and upload to GitHub release assets (#6202) 2026-08-18 05:18:13 +00:00
Zhengchao An 51497cb533 fix(ecstore): give peer REST failures op and bucket context (#6200) 2026-08-18 04:57:44 +00:00
Zhengchao An c7a29ec0a7 chore(storage): drop dead backpressure and lock-optimizer wrappers (#6195)
Neither rustfs/src/storage/backpressure.rs nor rustfs/src/storage/lock_optimizer.rs had a production caller: their only non-self references were the pub mod lines in storage/mod.rs and a cfg(test) module, so the object transfer path never applied this backpressure and never took these lock shortcuts.

The six removed tests in concurrent_fix_test.rs duplicated tests that lived inside the deleted files; the shared primitives they shadowed keep their own coverage in rustfs-io-core.
2026-08-18 12:51:44 +08:00
Zhengchao An deb0edb7cc chore: adjudicate 26 bare dead_code allows across five crates (#6187)
Remove every bare `#[allow(dead_code)]` in io-core, object-capacity, targets, rio, and scanner. Each allow was stripped first and clippy was then asked which ones the compiler actually missed, so the verdicts rest on the diagnostic rather than on inspection.

23 were inert: they sat on `pub fn`s inside `pub mod`s, where `dead_code` does not apply, or on scanner integration-test helpers that the tests in the same file do call.

The remaining 3 are in rio's private `compress_index` module and the code behind them is deleted rather than annotated. `remove_index_headers` is dead and also wrong — after skipping the 4-byte chunk header it matches against `S2_INDEX_TRAILER` where `S2_INDEX_HEADER` sits, so it returns `None` for every well-formed index; rio-v2 carries the correct equivalent that is actually in use. `restore_index_headers` is its unreachable counterpart, likewise duplicated live in rio-v2. `Index::reset` is a private method with no caller.

Refs backlog#1823
2026-08-18 12:45:42 +08:00
houseme a08de9229b feat(heal): wire MRF intents with durable repair journal (HS-01) (#6189)
* feat(common): add MRF intent channel and Mrf request source (HS-01)

Introduce the producer-facing half of the mission repair feed: a global
bounded (8192) channel carrying lightweight MrfIntent values from IO
error paths, plus the RUSTFS_HEAL_MRF_ENABLE delivery kill-switch and
config constants for queue/journal sizing. Delivery is strictly
non-blocking (try_send, drop-on-full) so it can sit on decode-failure
and partial-write paths without adding latency. HealRequestSource grows
a 'mrf' variant so admission accounting can attribute replayed intents.

Part of backlog#1865 (option a: wire HealEvent-style intents with a
durable retry ledger).

Co-Authored-By: heihutu <heihutu@gmail.com>

* feat(heal): add MRF queue, durable journal, and intent consumer (HS-01)

Consumer half of the mission repair feed: a bounded pending queue
(100k intents / 8 MiB dual ceiling, drop-newest on overflow), a durable
journal at buckets/.heal/mrf/journal.bin holding the unaccepted pending
snapshot, and a consumer task that batches intents off the global
channel, translates them into prioritized heal requests (decode
failure -> Urgent ECDecode, metadata corruption -> High Metadata,
partial write -> Normal object heal), and retries full admissions with
a 5s backoff and a 3-attempt ceiling.

Durability: every journal record carries its own CRC32 and a
format/version header, so a torn tail truncates cleanly at replay; the
journal is deleted after a successful replay and when the pending set
drains (mirroring MinIO's post-replay list.bin unlink). Losing the last
500 ms flush window is acceptable: replayed duplicates merge via the
manager dedup key and read-repair remains the safety net.

Metrics: rustfs_heal_mrf_queue_depth/_queue_bytes, _dropped_total
{reason}, _replayed_total, _journal_bytes, _journal_fsync_total.
The consumer is wired at heal runtime bootstrap right after manager
start, honoring RUSTFS_HEAL_MRF_ENABLE (default on, rollback = off).

Tests: unit tests for the dual ceiling, record roundtrip, torn-tail
truncation, and the priority mapping; integration tests against a real
4-disk ECStore proving channel intents reach the manager queue as
Urgent/mrf-attributed requests and journal replay arms intents, drops
torn tails, and removes the file.

Part of backlog#1865 (option a).

Co-Authored-By: heihutu <heihutu@gmail.com>

* feat(ecstore,scanner): deliver MRF intents from error paths (HS-01)

Wire the three production delivery points, each a single non-blocking
try_send next to the existing in-memory heal paths, which stay as the
fast path:

- read.rs decode-error branch: DecodeFailure intent beside the existing
  read-repair submit, so an Urgent ECDecode request survives restarts
  even when the Low-priority read-repair request was dropped or lost.
- add_partial: PartialWrite intent, giving partial-write recovery a
  durable Normal-priority object heal across restarts.
- scanner_folder metadata-corruption classification: MetadataCorruption
  intent beside the existing High-priority scanner heal request.

All three are on error paths only: zero cost on healthy IO.

Part of backlog#1865 (option a).

Co-Authored-By: heihutu <heihutu@gmail.com>

* fix: include mrf heal source counts

Co-Authored-By: heihutu <heihutu@gmail.com>

* fix: keep node heal status wire compatibility

Co-Authored-By: heihutu <heihutu@gmail.com>

---------

Co-authored-by: heihutu <heihutu@gmail.com>
2026-08-18 12:43:27 +08:00
Zhengchao An abffa5cf1b chore(storage): drop dead io-schedule metrics and helpers (#6199) 2026-08-18 04:35:36 +00:00
Zhengchao An b825c54850 refactor(admin): route kms management auth through shared gate (#6194) 2026-08-18 12:21:49 +08:00
houseme de9145e87a feat(storage): add default-off PUT admission gate (#6197)
Add an experimental fixed-count foreground PutObject admission gate for #1882 Phase 0 validation. The gate is default-off, returns SlowDown before body ingest when saturated, and keeps the admission permit with the spawned store commit owner until store PUT returns.

Co-authored-by: heihutu <heihutu@gmail.com>
2026-08-18 04:15:19 +00:00
houseme 84bd76a3ce chore(deps): refresh cargo dependencies (#6198)
Update workspace Cargo dependency requirements and lockfile after cargo update/upgrade, including rumqttc-next 0.34.0 and MQTT API compatibility adjustments.

Verification:

- cargo update --verbose

- cargo upgrade --verbose

- cargo update -p rumqttc-next --precise 0.34.0 --verbose

- cargo tree --invert rumqttc-next --locked

- cargo metadata --locked --no-deps --format-version 1

- cargo fmt --all --check

- cargo check -p rustfs-targets --all-targets --locked

- cargo test -p rustfs-targets mqtt --locked

- make pre-pr

Co-authored-by: heihutu <heihutu@gmail.com>
2026-08-18 04:11:53 +00:00
85 changed files with 5982 additions and 6450 deletions
+2 -2
View File
@@ -66,8 +66,8 @@ s3s-footprint-check: ## Check the s3s dependency footprint ratchet stays frozen
./scripts/check_s3s_footprint.sh
.PHONY: fips-wording-check
fips-wording-check: ## Check outward docs do not make unsupported FIPS claims
@echo "📣 Checking FIPS wording guard..."
fips-wording-check: ## Check docs and crates/kms do not over-claim crypto capabilities
@echo "📣 Checking cryptographic capability wording guard..."
./scripts/check_fips_wording.sh
.PHONY: log-analyzer-rules-check
+3
View File
@@ -117,6 +117,9 @@ jobs:
- name: Check s3s footprint ratchet
run: ./scripts/check_s3s_footprint.sh
- name: Check cryptographic capability wording
run: ./scripts/check_fips_wording.sh
- name: Check no planning docs committed
run: ./scripts/check_no_planning_docs.sh
+3
View File
@@ -152,6 +152,9 @@ jobs:
- name: Check s3s footprint ratchet
run: ./scripts/check_s3s_footprint.sh
- name: Check cryptographic capability wording
run: ./scripts/check_fips_wording.sh
- name: Check no planning docs committed
run: ./scripts/check_no_planning_docs.sh
+84 -11
View File
@@ -15,28 +15,35 @@
# Package Workflow - Build DEB/RPM packages
#
# This workflow builds DEB and RPM packages from pre-built Linux binaries
# and uploads them to Cloudflare R2.
# and uploads them to Cloudflare R2 and the GitHub release.
#
# Trigger:
# - release published: automatically package when a GitHub release is published
# - workflow_dispatch: manual trigger with optional tag/run_id
# - workflow_run: automatically package after "Build and Release" completes
# for a release tag (the mac/windows/linux binaries are already uploaded
# to the GitHub release before packaging starts)
# - workflow_dispatch: manual fallback (backfill / re-run) with optional tag/run_id
#
# Flow:
# 1. Find the Build workflow run for the release tag
# 1. Resolve the triggering Build workflow run for the release tag
# 2. Download Linux binaries (x86_64-gnu, aarch64-gnu) from build artifacts
# 3. Build DEB packages for amd64 and arm64
# 4. Build RPM packages for x86_64 and aarch64
# 5. Upload all packages to Cloudflare R2
# 5. Upload all packages to Cloudflare R2 and the GitHub release
name: Package DEB/RPM
permissions:
contents: read
# contents: write is required to upload packages to the GitHub release
contents: write
actions: read
on:
release:
types: [ published ]
# Follows the same pattern as docker.yml: run after the release build
# workflow completes, so packaging is triggered only by release tags
# (e.g. 1.0.0-rc.2, 1.0.0-rc.3), never by development builds.
workflow_run:
workflows: [ "Build and Release" ]
types: [ completed ]
workflow_dispatch:
inputs:
tag:
@@ -49,13 +56,26 @@ on:
type: string
concurrency:
group: ${{ github.workflow }}-${{ github.event.release.tag_name || github.event.inputs.tag || github.run_id }}
group: ${{ github.workflow }}-${{ github.event.workflow_run.head_branch || github.event.inputs.tag || github.run_id }}
cancel-in-progress: true
env:
HEAD_BRANCH: ${{ github.event.workflow_run.head_branch }}
WORKFLOW_RUN_ID: ${{ github.event.workflow_run.id }}
jobs:
# Resolve which build run to use and extract version info
resolve:
name: Resolve Build
# Auto-trigger only from successful tag builds of "Build and Release".
# Tag pushes arrive as event == push with head_branch != main (a
# non-main push head_branch is the release tag name). Manual dispatch
# stays available as a fallback for backfills and re-runs.
if: >-
github.event_name == 'workflow_dispatch' ||
(github.event.workflow_run.conclusion == 'success' &&
github.event.workflow_run.event == 'push' &&
github.event.workflow_run.head_branch != 'main')
runs-on: ubuntu-latest
timeout-minutes: 10
outputs:
@@ -75,8 +95,8 @@ jobs:
set -euo pipefail
# Determine tag
if [[ "${{ github.event_name }}" == "release" ]]; then
TAG="${{ github.event.release.tag_name }}"
if [[ "${{ github.event_name }}" == "workflow_run" ]]; then
TAG="${HEAD_BRANCH}"
elif [[ -n "$INPUT_TAG" ]]; then
TAG="$INPUT_TAG"
else
@@ -93,6 +113,11 @@ jobs:
BUILD_RUN_ID="$INPUT_RUN_ID"
echo "Using explicit build run ID: $BUILD_RUN_ID"
elif [[ "${{ github.event_name }}" == "workflow_run" ]]; then
# Use the Build and Release run that triggered this workflow
BUILD_RUN_ID="${WORKFLOW_RUN_ID}"
echo "Using triggering workflow run: $BUILD_RUN_ID"
elif [[ -n "$TAG" ]]; then
# Find the build run that produced this tag
echo "Looking for build run for tag: $TAG"
@@ -456,6 +481,54 @@ jobs:
echo "✅ Latest packages updated"
fi
- name: Upload packages to GitHub Release
if: needs.resolve.outputs.tag != ''
env:
GH_TOKEN: ${{ github.token }}
shell: bash
run: |
set -euo pipefail
TAG="${{ needs.resolve.outputs.tag }}"
DEB_FILE="${{ steps.deb.outputs.deb_file }}"
RPM_FILE="${{ steps.rpm.outputs.rpm_file }}"
# Upload the packages, then refresh the release checksums so the new
# assets are covered, matching the binary release flow.
for f in "$DEB_FILE" "$RPM_FILE"; do
if [[ -n "$f" && -f "$f" ]]; then
echo "📤 Uploading $(basename "$f") to GitHub release ${TAG}..."
gh release upload "$TAG" "$f" --clobber
fi
done
CHECKSUM_DIR="$(mktemp -d)"
gh release download "$TAG" -p 'SHA256SUMS' -p 'SHA512SUMS' \
-D "$CHECKSUM_DIR" --clobber 2>/dev/null || true
for spec in "SHA256SUMS:sha256sum" "SHA512SUMS:sha512sum"; do
asset="${spec%%:*}"
checksum_cmd="${spec##*:}"
checksum_file="${CHECKSUM_DIR}/${asset}"
touch "$checksum_file"
for f in "$DEB_FILE" "$RPM_FILE"; do
if [[ -n "$f" && -f "$f" ]]; then
base="$(basename "$f")"
# Remove any stale entry, then append the fresh digest
grep -Fv -- "$base" "$checksum_file" > "${checksum_file}.tmp" || true
mv "${checksum_file}.tmp" "$checksum_file"
(cd "$(dirname "$f")" && "$checksum_cmd" -- "$base") >> "$checksum_file"
fi
done
echo "📤 Updating ${asset} for release ${TAG}..."
gh release upload "$TAG" "$checksum_file" --clobber
done
echo "✅ GitHub release assets updated"
# Summary
summary:
name: Summary
+4 -4
View File
@@ -31,7 +31,7 @@ HTTP request
→ storage/ecfs (erasure coding, encryption, checksums)
→ ecstore (disk pool selection, data distribution)
→ rio (reader pipeline: encrypt → compress → hash → write)
→ io-core (zero-copy I/O, buffer pool, direct I/O)
→ io-core (buffer pool, storage profiling, admission control)
→ local disk / remote disk via RPC
```
@@ -55,7 +55,7 @@ rustfs/ # Workspace root (virtual manifest)
├── crates/ # library crates (authoritative list: Cargo.toml [workspace].members)
│ ├── ecstore/ # Erasure-coded storage engine
│ ├── rio/ # Reader I/O pipeline (encrypt, compress, hash)
│ ├── io-core/ # Zero-copy I/O, scheduling, buffer pool
│ ├── io-core/ # Buffer pool, storage profiling, admission control
│ ├── io-metrics/ # I/O metrics collection
│ ├── common/ # Shared runtime state, globals, data usage types
│ ├── config/ # Configuration types and parsing
@@ -302,7 +302,7 @@ The binary (`main.rs`) boots in this order:
│ │ │
┌─────▼──────┐ ┌──────▼──────┐ ┌──────▼──────┐
│ ecstore │ │ rio │ │ io-core │
│ (core) │ │ (readers) │ │ (zero-copy)
│ (core) │ │ (readers) │ │ (buffers)
└─────┬──────┘ └─────────────┘ └─────────────┘
┌─────┬──┼──┬─────┬──────┐
@@ -314,7 +314,7 @@ The binary (`main.rs`) boots in this order:
- **"Where does S3 PutObject go?"**
`server/` routes → `app/object_usecase` validates → `storage/ecfs` encodes →
`ecstore` distributes → `rio` encrypts/compresses → `io-core` writes
`ecstore` distributes → `rio` encrypts/compresses → `io-core` supplies buffers
- **"Where are bucket policies enforced?"**
`app/bucket_usecase` calls into `crates/policy/`
Generated
+107 -104
View File
@@ -964,9 +964,9 @@ dependencies = [
[[package]]
name = "aws-sdk-kms"
version = "1.114.0"
version = "1.115.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c0b7d906608ee41e7ddea9983577ba82200435644d567d63dc34e822e088b453"
checksum = "d5b034f8b7ceadb873d0bc607c30bb4b0be68e09a84c837174e7c2c6878ff882"
dependencies = [
"arc-swap",
"aws-credential-types",
@@ -990,9 +990,9 @@ dependencies = [
[[package]]
name = "aws-sdk-s3"
version = "1.141.0"
version = "1.142.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d9f9420d3a2467eed22ed3635ca653653162c386a0b0f65c78189f9bd3c1379e"
checksum = "f9e15a5c55e05f4b0b7e483160b3c85cccdf77cff02c95504f3e71d460855cd2"
dependencies = [
"arc-swap",
"aws-credential-types",
@@ -1027,9 +1027,9 @@ dependencies = [
[[package]]
name = "aws-sdk-sso"
version = "1.105.0"
version = "1.106.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6ffd0fbe7873cb548a7aa60f9573c268fff94155397fd4f14dc9f1ecaaab8516"
checksum = "2d0efcee834347b6705eca3eea2defd88242f43774f55d7326604222e3c86260"
dependencies = [
"arc-swap",
"aws-credential-types",
@@ -1053,9 +1053,9 @@ dependencies = [
[[package]]
name = "aws-sdk-ssooidc"
version = "1.107.0"
version = "1.108.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "175763eb222a46377df7aa257a3bca980ab3e96703fefc8f4d0b8da6ad2e254c"
checksum = "a59312a04cf19c962cfee32b64ecfee758f8786407ff6da5b30fff46ae96f201"
dependencies = [
"arc-swap",
"aws-credential-types",
@@ -1079,9 +1079,9 @@ dependencies = [
[[package]]
name = "aws-sdk-sts"
version = "1.110.0"
version = "1.111.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "dd8b14781dfbff48984017d57167b6ea0b6471c6920ec52b44a2677c7feb3c13"
checksum = "120e7eb63457a9e547f9986fe3b273f77c43679da4d04f46359fa881c5e19b6e"
dependencies = [
"arc-swap",
"aws-credential-types",
@@ -1598,7 +1598,7 @@ version = "0.10.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3078c7629b62d3f0439517fa394996acacc5cbc91c5a20d8c658e77abd503a71"
dependencies = [
"generic-array 0.14.9",
"generic-array 0.14.7",
]
[[package]]
@@ -1617,7 +1617,7 @@ version = "0.3.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a8894febbff9f758034a5b8e12d87918f56dfc64a8e1fe757d65e29041538d93"
dependencies = [
"generic-array 0.14.9",
"generic-array 0.14.7",
]
[[package]]
@@ -1858,9 +1858,9 @@ dependencies = [
[[package]]
name = "cc"
version = "1.4.2"
version = "1.4.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5d262e149917187838d5b42777c8253bcb64500067342904e7d429499a6f277e"
checksum = "509591b7bcd67f4ef775afad7662703b4935daaa6ec0e5605cfb1090b32a2b6d"
dependencies = [
"find-msvc-tools",
"jobserver",
@@ -1968,7 +1968,7 @@ version = "0.4.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "773f3b9af64447d2ce9850330c473515014aa235e6a783b02db81ff39e4a3dad"
dependencies = [
"crypto-common 0.1.6",
"crypto-common 0.1.7",
"inout 0.1.4",
]
@@ -2428,7 +2428,7 @@ version = "0.5.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0dc92fb57ca44df6db8059111ab3af99a63d5d0f8375d9972e319a379c6bab76"
dependencies = [
"generic-array 0.14.9",
"generic-array 0.14.7",
"rand_core 0.6.4",
"subtle",
"zeroize",
@@ -2453,11 +2453,11 @@ dependencies = [
[[package]]
name = "crypto-common"
version = "0.1.6"
version = "0.1.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1bfb12502f3fc46cca1bb51ac28df9d618d813cdc3d2f25b9fe775a34af26bb3"
checksum = "78c8292055d1c1df0cce5d180393dc8cce0abec0a7102adb6c7b1eef6016d60a"
dependencies = [
"generic-array 0.14.9",
"generic-array 0.14.7",
"typenum",
]
@@ -3664,7 +3664,7 @@ checksum = "9ed9a281f7bc9b7576e61468ba615a66a5c8cfdff42420a70aa82701a3b1e292"
dependencies = [
"block-buffer 0.10.4",
"const-oid 0.9.6",
"crypto-common 0.1.6",
"crypto-common 0.1.7",
"subtle",
]
@@ -3924,7 +3924,7 @@ dependencies = [
"crypto-bigint 0.5.5",
"digest 0.10.7",
"ff 0.13.1",
"generic-array 0.14.9",
"generic-array 0.14.7",
"group 0.13.0",
"hkdf 0.12.4",
"pem-rfc7468 0.7.0",
@@ -4148,9 +4148,9 @@ dependencies = [
[[package]]
name = "find-msvc-tools"
version = "0.1.10"
version = "0.1.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "26b73573e6edcd2af0cdf47bd6cb58f0b3839491263c314eaad1ccf24430e1de"
checksum = "d45db016d36b838f563236e9193d0ee6ce38f3f68b6c94e914b4929c96bbb890"
[[package]]
name = "findshlibs"
@@ -4369,9 +4369,9 @@ dependencies = [
[[package]]
name = "generic-array"
version = "0.14.9"
version = "0.14.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4bb6743198531e02858aeaea5398fcc883e71851fcbcb5a2f773e2fb6cb1edf2"
checksum = "85649ca51fd72272d7821adaf274ad91c288277713d9c18820d8499a7ff69e9a"
dependencies = [
"typenum",
"version_check",
@@ -4380,11 +4380,11 @@ dependencies = [
[[package]]
name = "generic-array"
version = "1.4.4"
version = "1.4.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ab4e5aa225bc56696909483320f0ff9b600f1a971b52e07a17d70f3d9b43254b"
checksum = "337d46834ee672ab3e48caca2cb0c78cc174fb12b3a68d0d88f99a0519a5e36e"
dependencies = [
"generic-array 0.14.9",
"generic-array 0.14.7",
"rustversion",
"typenum",
]
@@ -4726,9 +4726,9 @@ dependencies = [
[[package]]
name = "h2"
version = "0.4.15"
version = "0.4.16"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6cb093c84e8bd9b188d4c4a8cb6579fc016968d14c99882163cd3ff402a4f155"
checksum = "a9f37a958b41b3b19ee2707c06439c0e9e547e847223eb791ecb0cb821c65e27"
dependencies = [
"atomic-waker",
"bytes",
@@ -5028,9 +5028,9 @@ dependencies = [
[[package]]
name = "hotpath"
version = "0.23.2"
version = "0.23.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "62e810bedda5a467ef5c9b5c8a20763fefebc89b63ef36f7ee44a143085204a2"
checksum = "dce755d457a63bdd0c95e4c91511daad1b58b33209543b7f38027b676f387e5e"
dependencies = [
"arc-swap",
"async-channel",
@@ -5062,9 +5062,9 @@ dependencies = [
[[package]]
name = "hotpath-macros"
version = "0.23.2"
version = "0.23.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "01bdc59bfc1a9984bee2ff5da63b2f6fccbaa57cd9a4119d709524632bddf341"
checksum = "a903af89a8429cb07790c3818bc15270b394f80af1bc254e5ccf9c7de2961770"
dependencies = [
"proc-macro2",
"quote",
@@ -5073,15 +5073,15 @@ dependencies = [
[[package]]
name = "hotpath-macros-meta"
version = "0.23.2"
version = "0.23.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d9216e8a01abe1e1671c376dc8736fb1bf772d7a889538d25f9e1200120ced38"
checksum = "bcc0ab94ffbb2ee77f4a897df02b5a137a10cf24d69bda936e59aff4dd456e61"
[[package]]
name = "hotpath-meta"
version = "0.23.2"
version = "0.23.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f22a9d20435fb79511b19dae37b3607224cd98f342a410702d84657cc38fc72f"
checksum = "053481f6cec8f775a3276c7f6e2f21123111d28261e4edc15ea7421c445964bb"
dependencies = [
"hotpath-macros-meta",
]
@@ -5280,9 +5280,9 @@ dependencies = [
[[package]]
name = "icu_collections"
version = "2.2.0"
version = "2.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2984d1cd16c883d7935b9e07e44071dca8d917fd52ecc02c04d5fa0b5a3f191c"
checksum = "fa68d21081c4a05d5a901a1c62add574c77048b6a1c67be3b50ce0b60d4ca513"
dependencies = [
"displaydoc",
"potential_utf",
@@ -5294,9 +5294,9 @@ dependencies = [
[[package]]
name = "icu_locale_core"
version = "2.2.0"
version = "2.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "92219b62b3e2b4d88ac5119f8904c10f8f61bf7e95b640d25ba3075e6cac2c29"
checksum = "d56e28588da92eee5c3201a6eff33fabdd49b62269c8938d4ff050ce4d900deb"
dependencies = [
"displaydoc",
"litemap",
@@ -5307,9 +5307,9 @@ dependencies = [
[[package]]
name = "icu_normalizer"
version = "2.2.0"
version = "2.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c56e5ee99d6e3d33bd91c5d85458b6005a22140021cc324cea84dd0e72cff3b4"
checksum = "12f9cf5f235641ed274641dd81c3f28d870e276763d0797aeeab72317b1c646f"
dependencies = [
"icu_collections",
"icu_normalizer_data",
@@ -5321,16 +5321,17 @@ dependencies = [
[[package]]
name = "icu_normalizer_data"
version = "2.2.0"
version = "2.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "da3be0ae77ea334f4da67c12f149704f19f81d1adf7c51cf482943e84a2bad38"
checksum = "1563da1ed3e0b3bf3d74c9b85917ac9c56464d2f57242270c09c9e752f8021a0"
[[package]]
name = "icu_properties"
version = "2.2.0"
version = "2.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bee3b67d0ea5c2cca5003417989af8996f8604e34fb9ddf96208a033901e70de"
checksum = "7e7ca276ad3145661a65914e6daf131ca5120cd3dcee8f8f3214b8875184a148"
dependencies = [
"displaydoc",
"icu_collections",
"icu_locale_core",
"icu_properties_data",
@@ -5341,15 +5342,15 @@ dependencies = [
[[package]]
name = "icu_properties_data"
version = "2.2.0"
version = "2.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8e2bbb201e0c04f7b4b3e14382af113e17ba4f63e2c9d2ee626b720cbce54a14"
checksum = "e590f038c1464a96894fd6d10127e90a8be4509f56ff7ecef851b15cee0b7caa"
[[package]]
name = "icu_provider"
version = "2.2.0"
version = "2.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "139c4cf31c8b5f33d7e199446eff9c1e02decfc2f0eec2c8d71f65befa45b421"
checksum = "92a7ed671a6aad807a8651a2e1782a6598fda9ce5185dd8158549e95a91c6428"
dependencies = [
"displaydoc",
"icu_locale_core",
@@ -5417,7 +5418,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "879f10e63c20629ecabbb64a8010319738c66a5cd0c29b02d63d272b03751d01"
dependencies = [
"block-padding 0.3.3",
"generic-array 0.14.9",
"generic-array 0.14.7",
]
[[package]]
@@ -5968,9 +5969,9 @@ dependencies = [
[[package]]
name = "libredox"
version = "0.1.19"
version = "0.1.20"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2026a5056764a10b2bf5d56488cba40da507f5493a6a429340e2004d9ed085fa"
checksum = "28d0a00925a9f930d679b6789b721e3a7f9ed110f41b86d2497caa780c3a070a"
dependencies = [
"libc",
]
@@ -6033,9 +6034,9 @@ checksum = "32a66949e030da00e8c7d4434b251670a91556f4144941d37452769c25d58a53"
[[package]]
name = "litemap"
version = "0.8.2"
version = "0.8.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "92daf443525c4cce67b150400bc2316076100ce0b3686209eb8cf3c31612e6f0"
checksum = "47d9d19d1d6efa0109d2f65ff4c85cddd50bd572e5a00127ab10987290bcefae"
[[package]]
name = "local-ip-address"
@@ -6482,9 +6483,9 @@ dependencies = [
[[package]]
name = "mqttbytes-core-next"
version = "0.33.3"
version = "0.34.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3ff7ae19c74aba9e0ed6e4071cd52aa364e020076fa3cc6ef17e43662f756f3c"
checksum = "366b6ba2b4209ca4bc5ac731ccddf570d09831981eed07e5fbd63564cf0cf1aa"
dependencies = [
"bytes",
"thiserror 2.0.20",
@@ -6885,7 +6886,7 @@ version = "5.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "51e219e79014df21a225b1860a479e2dcd7cbd9130f4defd4bd0e191ea31d67d"
dependencies = [
"base64 0.21.7",
"base64 0.22.1",
"chrono",
"getrandom 0.2.17",
"http 1.5.0",
@@ -7344,9 +7345,9 @@ dependencies = [
[[package]]
name = "pageant"
version = "0.2.1"
version = "0.2.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4f3a5ae18f65a85c67a77d18d42d3606c07948e3c17c1e5f74852b26589e88a5"
checksum = "3adadc44070da6f464b0918655a12f5792c156e088d8c4082d13e27d94c3e791"
dependencies = [
"base16ct 1.0.0",
"byteorder",
@@ -7728,9 +7729,9 @@ dependencies = [
[[package]]
name = "pkg-config"
version = "0.3.33"
version = "0.3.34"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "19f132c84eca552bf34cab8ec81f1c1dcc229b811638f9d283dceabe58c5569e"
checksum = "f6b464fbc74e149a392436b17d523f769e057cb6877f6a5c4618bc6f11800548"
[[package]]
name = "plotters"
@@ -7836,9 +7837,9 @@ dependencies = [
[[package]]
name = "potential_utf"
version = "0.1.5"
version = "0.1.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0103b1cef7ec0cf76490e969665504990193874ea05c85ff9bab8b911d0a0564"
checksum = "d83eb9bc6d8e5cf568e7a1101d60ee05e81ed50ea106026f3d18deeb046d7661"
dependencies = [
"zerovec",
]
@@ -8046,7 +8047,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "be769465445e8c1474e9c5dac2018218498557af32d9ed057325ec9a41ae81bf"
dependencies = [
"heck 0.5.0",
"itertools 0.10.5",
"itertools 0.14.0",
"log",
"multimap",
"once_cell",
@@ -8066,7 +8067,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "03da047801ff44bb6a4d407d4860c05fd70bb81714e6b2f3812603d5b145b042"
dependencies = [
"heck 0.5.0",
"itertools 0.10.5",
"itertools 0.14.0",
"log",
"multimap",
"petgraph 0.8.3",
@@ -8087,7 +8088,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8a56d757972c98b346a9b766e3f02746cde6dd1cd1d1d563472929fdd74bec4d"
dependencies = [
"anyhow",
"itertools 0.10.5",
"itertools 0.14.0",
"proc-macro2",
"quote",
"syn 2.0.119",
@@ -8100,7 +8101,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b570b25f7617e43d59005d0990ccb79e950a423952cea19671b7a876da390adf"
dependencies = [
"anyhow",
"itertools 0.10.5",
"itertools 0.14.0",
"proc-macro2",
"quote",
"syn 2.0.119",
@@ -8216,7 +8217,7 @@ dependencies = [
"reqwest",
"serde_json",
"smallvec",
"spin 0.12.2",
"spin 0.12.3",
"symbolic-demangle",
"tempfile",
"thiserror 2.0.20",
@@ -8285,9 +8286,9 @@ dependencies = [
[[package]]
name = "quinn-proto"
version = "0.11.16"
version = "0.11.17"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2f4bfc015262b9df63c8845072ce59068853ff5872180c2ce2f13038b970e560"
checksum = "04759210543be93709136e28212294a659ef5001836ff4eab4d663e4529bba83"
dependencies = [
"aws-lc-rs",
"bytes",
@@ -8553,9 +8554,9 @@ dependencies = [
[[package]]
name = "redis"
version = "1.5.0"
version = "1.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3257df217f7eab0044627a268c9cc6cdb60c0c421c88f83ac41c4e31520b6b84"
checksum = "e37a4ca5c6ca42aa3e6df2fd32b987a65d32a4c2159a6f3fe0fd1df306a2658f"
dependencies = [
"arc-swap",
"arcstr",
@@ -8567,7 +8568,7 @@ dependencies = [
"futures-channel",
"futures-util",
"itoa",
"num-bigint 0.4.8",
"num-bigint 0.5.1",
"percent-encoding",
"pin-project-lite",
"rustls",
@@ -8868,9 +8869,9 @@ dependencies = [
[[package]]
name = "rumqttc-core-next"
version = "0.33.3"
version = "0.34.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7d7d9205738dd41a2546e82d27a634d07d8b303dcf7558565ff70caf3ceb0f9c"
checksum = "249896ab27ed630590971738264baa8f722f18965d2e387c706c40a3c2a572cc"
dependencies = [
"async-tungstenite",
"futures-io",
@@ -8886,18 +8887,18 @@ dependencies = [
[[package]]
name = "rumqttc-next"
version = "0.33.3"
version = "0.34.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ed1bad2180ff539da671da9a996152a921bc5316eb6d8a9cc3bd441653138b08"
checksum = "477c9bbfba8f3aecc7aad31c6de2eacb75822efaa18e7aeecb8d3d8e534fbf07"
dependencies = [
"rumqttc-v5-next",
]
[[package]]
name = "rumqttc-v5-next"
version = "0.33.3"
version = "0.34.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "229576cbedfa9089f90c17c9454e9429ac1e89cdd223bac5cb39d837593f79bc"
checksum = "3dfa6ddcc7a7dd5688f9bf78d8f81cb94f367bce56c055d8d94cf81ecb0518bf"
dependencies = [
"async-tungstenite",
"bytes",
@@ -8920,9 +8921,9 @@ dependencies = [
[[package]]
name = "russh"
version = "0.62.6"
version = "0.62.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b41043523e0edcbd4e31d00903e26f12994f63b21bae9904f7405c1ed92752a5"
checksum = "9decb68e4e44e1079700e54f17c8f23806ec53d7e0db73ab1c71d9dabc666812"
dependencies = [
"aes 0.9.2",
"aws-lc-rs",
@@ -8945,7 +8946,7 @@ dependencies = [
"enum_dispatch",
"flate2",
"futures",
"generic-array 1.4.4",
"generic-array 1.4.5",
"getrandom 0.4.3",
"ghash",
"hex-literal",
@@ -9464,6 +9465,7 @@ dependencies = [
"tokio-stream",
"tokio-util",
"tonic",
"tonic-prost",
"tower",
"tracing",
"tracing-core",
@@ -9491,7 +9493,7 @@ dependencies = [
"parking_lot",
"rayon",
"smallvec",
"spin 0.12.2",
"spin 0.12.3",
]
[[package]]
@@ -9537,6 +9539,8 @@ version = "1.0.0-rc.2"
dependencies = [
"async-trait",
"base64 0.23.1",
"bytes",
"crc-fast",
"futures",
"hotpath",
"http 1.5.0",
@@ -9609,7 +9613,6 @@ version = "1.0.0-rc.2"
dependencies = [
"bytes",
"hotpath",
"memmap2",
"rustfs-io-metrics",
"thiserror 2.0.20",
"tokio",
@@ -10833,7 +10836,7 @@ checksum = "d3e97a565f76233a6003f9f5c54be1d9c5bdfa3eccfb189469f11ec4901c47dc"
dependencies = [
"base16ct 0.2.0",
"der 0.7.10",
"generic-array 0.14.9",
"generic-array 0.14.7",
"pkcs8 0.10.2",
"subtle",
"zeroize",
@@ -11397,9 +11400,9 @@ checksum = "023a211cb3138dbc438680b32560ad89f699977624c9f8dbb95a47d5b4c07dd3"
[[package]]
name = "spin"
version = "0.12.2"
version = "0.12.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8abadc99fd9c7bbb7d0ca2b31d72a067d0c0dcd7aad25ab8cac71ba91417694b"
checksum = "0134f9043ed38b087ac4f7d4af44c79e2c9e5094421fe3164f435ce585953b10"
dependencies = [
"lock_api",
]
@@ -11811,7 +11814,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "32497e9a4c7b38532efcdebeef879707aa9f794296a4f0244f6f69e9bc8574bd"
dependencies = [
"fastrand",
"getrandom 0.3.4",
"getrandom 0.4.3",
"once_cell",
"rustix",
"windows-sys 0.61.2",
@@ -11975,9 +11978,9 @@ dependencies = [
[[package]]
name = "tinystr"
version = "0.8.3"
version = "0.8.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c8323304221c2a851516f22236c5722a72eaa19749016521d6dff0824447d96d"
checksum = "b1e27c91459209c2986af3dcf603a5a74a4368754ce37414f59acc971167f643"
dependencies = [
"displaydoc",
"zerovec",
@@ -12651,9 +12654,9 @@ checksum = "06abde3611657adf66d383f00b093d7faecc7fa57071cce2578660c9f1010821"
[[package]]
name = "uuid"
version = "1.24.0"
version = "1.24.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bf3923a6f5c4c6382e0b653c4117f48d631ea17f38ed86e2a828e6f7412f5239"
checksum = "2cefc03fd367c0c6d4305de1b312cf00248c4114f4a0418ce6a6af769e3b0bd9"
dependencies = [
"getrandom 0.4.3",
"js-sys",
@@ -13168,9 +13171,9 @@ dependencies = [
[[package]]
name = "writeable"
version = "0.6.3"
version = "0.6.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1ffae5123b2d3fc086436f8834ae3ab053a283cfac8fe0a0b8eaae044768a4c4"
checksum = "3ad82d2a33cdc9674dc7465672f271e096168fcdbe0f799d9e6db8c5892679dc"
[[package]]
name = "x509-cert"
@@ -13350,9 +13353,9 @@ dependencies = [
[[package]]
name = "zerotrie"
version = "0.2.4"
version = "0.2.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0f9152d31db0792fa83f70fb2f83148effb5c1f5b8c7686c3459e361d9bc20bf"
checksum = "4ea269c3bd32f0a32c321907a2ae912ba6f4649bb0fc764a15627e99a7095a3f"
dependencies = [
"displaydoc",
"yoke",
@@ -13361,9 +13364,9 @@ dependencies = [
[[package]]
name = "zerovec"
version = "0.11.6"
version = "0.11.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "90f911cbc359ab6af17377d242225f4d75119aec87ea711a880987b18cd7b239"
checksum = "94b5c6b5976d66c1d703c4fd17d3f5e43c8cedaacf604961b171adc7130896d8"
dependencies = [
"yoke",
"zerofrom",
@@ -13372,13 +13375,13 @@ dependencies = [
[[package]]
name = "zerovec-derive"
version = "0.11.3"
version = "0.11.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "625dc425cab0dca6dc3c3319506e6593dcb08a9f387ea3b284dbd52a92c40555"
checksum = "9f212a141d820099d57ffafb9569be9617a6f27d3dc881fbee8fb56642f917a9"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.119",
"syn 3.0.3",
]
[[package]]
+8 -8
View File
@@ -228,9 +228,9 @@ atoi = "3.1.0"
atomic_enum = "0.3.0"
aws-config = { version = "1.10.1" }
aws-credential-types = { version = "1.3.0" }
aws-sdk-kms = { default-features = false, version = "1.114.0" }
aws-sdk-s3 = { default-features = false, version = "1.141.0" }
aws-sdk-sts = { default-features = false, version = "1.110.0" }
aws-sdk-kms = { default-features = false, version = "1.115.0" }
aws-sdk-s3 = { default-features = false, version = "1.142.0" }
aws-sdk-sts = { default-features = false, version = "1.111.0" }
aws-smithy-http-client = { default-features = false, version = "1.3.0" }
aws-smithy-runtime-api = { version = "1.14.0" }
aws-smithy-types = { version = "1.6.2" }
@@ -284,8 +284,8 @@ rayon = "1.12.0"
reed-solomon-erasure = { package = "rustfs-erasure-codec", version = "8.0.2" }
reed-solomon-simd = "3.1.0"
regex = { version = "1.13.1" }
rumqttc = { package = "rumqttc-next", version = "0.33.3" }
redis = { version = "1.5.0" }
rumqttc = { package = "rumqttc-next", version = "0.34.0" }
redis = { version = "1.6.0" }
rustify = { version = "0.7", default-features = false }
rustix = { version = "1.1.4" }
rust-embed = { version = "8.12.0" }
@@ -313,7 +313,7 @@ tracing-subscriber = { version = "0.3.23" }
transform-stream = "0.3.1"
url = "2.5.8"
urlencoding = "2.1.3"
uuid = { version = "1.24.0" }
uuid = { version = "1.24.1" }
vaultrs = { version = "0.8.0" }
tar = "0.4.46"
walkdir = "2.5.0"
@@ -341,7 +341,7 @@ libunftp = { version = "0.23.0" }
unftp-core = "0.1.0"
suppaftp = { version = "10.0.1" }
rcgen = { version = "0.14.9", default-features = false, features = ["aws_lc_rs", "crypto", "pem"] }
russh = { version = "0.62.6" }
russh = { version = "0.62.7" }
russh-sftp = "2.4.0"
# WebDAV
@@ -350,7 +350,7 @@ dav-server = "0.11.0"
# Performance Analysis and Memory Profiling
mimalloc = { version = "0.1.52", git = "https://github.com/xonatius/mimalloc_rust.git", rev = "6d4c41bb10c6d9da1d1b6f07b38c4cc051667f11" }
libmimalloc-sys = { version = "0.1.49", git = "https://github.com/xonatius/mimalloc_rust.git", rev = "6d4c41bb10c6d9da1d1b6f07b38c4cc051667f11", features = ["extended"] }
hotpath = { version = "0.23.2", default-features = false }
hotpath = { version = "0.23.3", default-features = false }
# Snapshot testing for output format regression detection
insta = { version = "1.48" }
+27
View File
@@ -224,6 +224,13 @@ pub struct HealOpts {
pub enum HealAdmissionDropReason {
QueueFull,
PolicyDropped,
/// HS-06: an admin heal start overlaps (same bucket with mutually
/// containing prefixes, or the same erasure set) an already running or
/// queued task. Only produced when RUSTFS_HEAL_OVERLAP_POLICY=minio_error.
AlreadyRunning,
/// HS-06: same as [`Self::AlreadyRunning`] but for paths that merely
/// contain (or are contained by) the active task's path.
OverlappingPaths,
}
impl HealAdmissionDropReason {
@@ -231,6 +238,8 @@ impl HealAdmissionDropReason {
match self {
Self::QueueFull => "queue_full",
Self::PolicyDropped => "policy_dropped",
Self::AlreadyRunning => "already_running",
Self::OverlappingPaths => "overlapping_paths",
}
}
}
@@ -287,6 +296,9 @@ pub enum HealRequestSource {
Scanner,
AutoHeal,
ReadRepair,
/// Mission Repair Feed: intents delivered by error paths and replayed
/// from the durable MRF journal.
Mrf,
}
impl HealRequestSource {
@@ -297,6 +309,7 @@ impl HealRequestSource {
Self::Scanner => "scanner",
Self::AutoHeal => "auto_heal",
Self::ReadRepair => "read_repair",
Self::Mrf => "mrf",
}
}
}
@@ -313,6 +326,9 @@ pub enum HealChannelCommand {
Query {
heal_path: String,
client_token: String,
/// Incremental result cursor (HS-06): only items with a sequence
/// greater than this are returned; `None` keeps the full snapshot.
since_seq: Option<u64>,
response_tx: oneshot::Sender<Result<HealChannelResponse, String>>,
},
/// Cancel heal task
@@ -518,10 +534,21 @@ async fn receive_heal_channel_response(
/// Send heal query request
pub async fn query_heal_status(heal_path: String, client_token: String) -> Result<HealChannelResponse, String> {
query_heal_status_since(heal_path, client_token, None).await
}
/// Incremental heal query (HS-06): pass the client's last seen sequence
/// number to receive only newer result items.
pub async fn query_heal_status_since(
heal_path: String,
client_token: String,
since_seq: Option<u64>,
) -> Result<HealChannelResponse, String> {
let (response_tx, response_rx) = oneshot::channel();
send_heal_command(HealChannelCommand::Query {
heal_path,
client_token,
since_seq,
response_tx,
})
.await?;
+1
View File
@@ -17,6 +17,7 @@ pub mod globals;
pub mod heal_channel;
pub mod last_minute;
pub mod metrics;
pub mod mrf_channel;
mod readiness;
pub mod table_catalog;
pub mod trace_bus;
+203
View File
@@ -0,0 +1,203 @@
// Copyright 2024 RustFS Team
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//! Mission Repair Feed (MRF) intent channel.
//!
//! Producers on error paths (read decode failure, scanner metadata
//! corruption, partial-write recovery) hand a lightweight [`MrfIntent`] to the
//! heal crate through a global bounded channel. Delivery is strictly
//! non-blocking: `try_send_mrf_intent` never awaits and drops the intent
//! (counting it) when the channel is full or uninitialized — losing one heal
//! hint is always preferred over stalling an IO path. Durable replay of
//! unconsumed intents is the consumer's job (see `rustfs-heal`
//! `heal::mrf_queue`), mirroring MinIO's `.heal/mrf/list.bin`.
use std::sync::{
Arc, OnceLock,
atomic::{AtomicBool, Ordering},
};
use tokio::sync::mpsc;
use uuid::Uuid;
/// Bounded capacity of the global MRF channel. Backpressure is resolved by
/// dropping (and counting) intents, never by blocking the producer.
const MRF_CHANNEL_CAPACITY: usize = 8192;
/// Why an intent was produced. Drives the heal priority mapping on the
/// consumer side (DecodeFailure -> Urgent, MetadataCorruption -> High,
/// PartialWrite -> Normal).
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum MrfKind {
/// Erasure decode failed while serving a read (read path).
DecodeFailure,
/// Scanner classified object metadata as corrupt.
MetadataCorruption,
/// A write left the object with fewer committed shards than the set size.
PartialWrite,
}
impl MrfKind {
pub const fn as_str(self) -> &'static str {
match self {
MrfKind::DecodeFailure => "decode-failure",
MrfKind::MetadataCorruption => "metadata-corruption",
MrfKind::PartialWrite => "partial-write",
}
}
}
/// One repair intent. Kept deliberately small so the in-memory queue and the
/// journal stay bounded; `bucket`/`object` are `Arc<str>` so re-arming an
/// intent never re-allocates the strings.
#[derive(Clone, Debug)]
pub struct MrfIntent {
pub bucket: Arc<str>,
pub object: Arc<str>,
/// Version the intent targets, as raw UUID bytes.
pub version_id: Option<[u8; 16]>,
pub kind: MrfKind,
pub enqueued_at_ms: u64,
/// Times this intent has already been offered to the heal manager.
/// Dropped by the consumer once it reaches `MRF_MAX_ATTEMPTS`.
pub attempts: u8,
}
/// Consumer-side retry ceiling before an intent is given up on.
pub const MRF_MAX_ATTEMPTS: u8 = 3;
impl MrfIntent {
/// Rough in-memory footprint used by the queue's byte budget.
pub fn estimated_bytes(&self) -> usize {
// Struct + strings + version bytes; buckets and objects are usually
// far below this bound, so rounding up keeps the budget conservative.
64 + self.bucket.len() + self.object.len()
}
}
static GLOBAL_MRF_SENDER: OnceLock<mpsc::Sender<MrfIntent>> = OnceLock::new();
/// Delivery kill-switch, set from `RUSTFS_HEAL_MRF_ENABLE`. Producers check
/// this before touching the channel so the disabled path stays allocation- and
/// sync-free.
static MRF_DELIVERY_ENABLED: AtomicBool = AtomicBool::new(true);
/// Override delivery (used at heal-runtime startup from configuration).
pub fn set_mrf_delivery_enabled(enabled: bool) {
MRF_DELIVERY_ENABLED.store(enabled, Ordering::Relaxed);
}
/// Whether producers currently deliver intents.
pub fn mrf_delivery_enabled() -> bool {
MRF_DELIVERY_ENABLED.load(Ordering::Relaxed)
}
/// Create the global MRF channel and return the consumer half. Fails if the
/// channel is already initialized (the heal runtime is a singleton).
pub fn init_mrf_channel() -> Result<mpsc::Receiver<MrfIntent>, &'static str> {
let (sender, receiver) = mpsc::channel(MRF_CHANNEL_CAPACITY);
GLOBAL_MRF_SENDER
.set(sender)
.map_err(|_| "MRF channel sender already initialized")?;
Ok(receiver)
}
/// Best-effort, non-blocking intent delivery from an error path.
///
/// Returns `true` when the intent was accepted into the channel. `false`
/// means the intent was dropped (feature disabled, channel not yet
/// initialized, or channel full) — callers must not retry or await; the
/// existing read-repair / scanner heal paths remain the safety net.
///
/// This runs on IO error paths, so it stays synchronous and cheap: one
/// bounded allocation for the two `Arc<str>` handles plus the channel slot.
pub fn try_send_mrf_intent(kind: MrfKind, bucket: &str, object: &str, version_id: Option<Uuid>) -> bool {
if !mrf_delivery_enabled() {
return false;
}
let Some(sender) = GLOBAL_MRF_SENDER.get() else {
return false;
};
let intent = MrfIntent {
bucket: Arc::from(bucket),
object: Arc::from(object),
version_id: version_id.map(|vid| *vid.as_bytes()),
kind,
enqueued_at_ms: unix_now_ms(),
attempts: 0,
};
sender.try_send(intent).is_ok()
}
fn unix_now_ms() -> u64 {
// Kept trivial: the timestamp is diagnostic metadata only; wall-clock
// failure would be a bug rather than something to handle here.
std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.map(|d| d.as_millis() as u64)
.unwrap_or(0)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn intents_estimate_is_conservative() {
let intent = MrfIntent {
bucket: Arc::from("bucket"),
object: Arc::from("object"),
version_id: Some([0u8; 16]),
kind: MrfKind::DecodeFailure,
enqueued_at_ms: 0,
attempts: 0,
};
assert!(intent.estimated_bytes() >= intent.bucket.len() + intent.object.len());
}
#[tokio::test]
async fn try_send_delivers_and_respects_capacity() {
let mut receiver = init_mrf_channel().expect("first initialization should succeed");
assert!(init_mrf_channel().is_err(), "double initialization must fail");
assert!(try_send_mrf_intent(MrfKind::DecodeFailure, "b", "o", Some(Uuid::nil())));
let intent = receiver.recv().await.expect("intent should arrive");
assert_eq!(intent.kind, MrfKind::DecodeFailure);
assert_eq!(intent.bucket.as_ref(), "b");
// Disable delivery: producers become no-ops.
set_mrf_delivery_enabled(false);
assert!(!try_send_mrf_intent(MrfKind::PartialWrite, "b", "o", None));
set_mrf_delivery_enabled(true);
// Fill the bounded channel past capacity: excess intents are dropped,
// never blocking.
let mut accepted = 0;
for _ in 0..(MRF_CHANNEL_CAPACITY + 64) {
if try_send_mrf_intent(MrfKind::PartialWrite, "b", "o", None) {
accepted += 1;
}
}
assert_eq!(accepted, MRF_CHANNEL_CAPACITY);
}
#[test]
fn try_send_without_channel_is_false() {
// This test may run after the tokio test above in the same process;
// the singleton semantics make a clean "uninitialized" case hard, so
// assert the flag-off behavior only.
set_mrf_delivery_enabled(false);
assert!(!try_send_mrf_intent(MrfKind::MetadataCorruption, "b", "o", None));
set_mrf_delivery_enabled(true);
}
}
+2 -3
View File
@@ -14,9 +14,8 @@
//! Shared backpressure policy type.
//!
//! The runtime backpressure implementation (byte-watermark pipes and
//! monitors) lives in `rustfs/src/storage/backpressure.rs`; this module only
//! carries the watermark policy type that implementation shares.
//! This module only carries the watermark policy; the admission primitive it
//! projects into lives in `rustfs-io-core`.
use rustfs_io_core::BackpressureConfig as CoreBackpressureConfig;
+37
View File
@@ -177,3 +177,40 @@ pub const DEFAULT_HEAL_MAINLINE_WRITE_UTILIZATION_HIGH_PERCENT: usize = 80;
/// Default foreground pressure recheck delay for heal scheduler, in milliseconds.
pub const DEFAULT_HEAL_MAINLINE_MAX_SLEEP_MS: u64 = 250;
/// Environment variable that toggles the MRF (mission repair feed) intent
/// pipeline: error paths deliver repair intents to the heal runtime, and
/// unconsumed intents are replayed from the durable journal after a restart.
pub const ENV_HEAL_MRF_ENABLE: &str = "RUSTFS_HEAL_MRF_ENABLE";
/// Environment variable for the MRF in-memory queue capacity (intent count).
pub const ENV_HEAL_MRF_QUEUE_SIZE: &str = "RUSTFS_HEAL_MRF_QUEUE_SIZE";
/// Environment variable for the MRF journal byte budget. The journal is
/// compacted once its on-disk size crosses this bound.
pub const ENV_HEAL_MRF_JOURNAL_MAX_BYTES: &str = "RUSTFS_HEAL_MRF_JOURNAL_MAX_BYTES";
/// Environment variable for the MRF journal replay batch size (intents per
/// replay push round).
pub const ENV_HEAL_MRF_REPLAY_BATCH: &str = "RUSTFS_HEAL_MRF_REPLAY_BATCH";
/// Default behavior keeps the MRF intent pipeline enabled.
pub const DEFAULT_HEAL_MRF_ENABLE: bool = true;
/// Default MRF queue capacity (matches MinIO's 100k MRF list ceiling).
pub const DEFAULT_HEAL_MRF_QUEUE_SIZE: usize = 100_000;
/// Default MRF journal byte budget (8 MiB), mirroring the channel payload cap.
pub const DEFAULT_HEAL_MRF_JOURNAL_MAX_BYTES: usize = 8 * 1024 * 1024;
/// Default MRF replay batch size.
pub const DEFAULT_HEAL_MRF_REPLAY_BATCH: usize = 256;
/// Environment variable selecting how admin heal starts behave when the
/// requested path overlaps an already running or queued heal: `merge`
/// (default, keep today's dedup/merge semantics) or `minio_error` (return a
/// typed already-running / overlapping-paths rejection like madmin).
pub const ENV_HEAL_OVERLAP_POLICY: &str = "RUSTFS_HEAL_OVERLAP_POLICY";
/// Default overlap policy: merge duplicate/overlapping requests.
pub const DEFAULT_HEAL_OVERLAP_POLICY: &str = "merge";
+25
View File
@@ -234,6 +234,31 @@ pub const ENV_OBJECT_DISK_WRITE_ABSOLUTE_CAP: &str = "RUSTFS_OBJECT_DISK_WRITE_A
/// Default absolute per-object erasure write cap in seconds (`0` = disabled).
pub const DEFAULT_OBJECT_DISK_WRITE_ABSOLUTE_CAP: u64 = 0;
/// Enable foreground PutObject request admission.
///
/// This is an experimental, default-off foreground write backpressure gate for
/// strict commit tail investigations. When disabled, PUTs follow the legacy
/// path and only the existing request counters are updated.
pub const ENV_PUT_FOREGROUND_ADMISSION_ENABLE: &str = "RUSTFS_PUT_FOREGROUND_ADMISSION_ENABLE";
pub const DEFAULT_PUT_FOREGROUND_ADMISSION_ENABLE: bool = false;
/// Maximum foreground PutObject requests admitted concurrently per process.
///
/// The limit is used only when [`ENV_PUT_FOREGROUND_ADMISSION_ENABLE`] is true.
/// A value of `0` disables the gate even when the enable flag is present, so a
/// partially configured rollout cannot reject every PUT.
pub const ENV_PUT_FOREGROUND_ADMISSION_LIMIT: &str = "RUSTFS_PUT_FOREGROUND_ADMISSION_LIMIT";
pub const DEFAULT_PUT_FOREGROUND_ADMISSION_LIMIT: usize = 0;
/// Time in milliseconds a foreground PutObject waits for an admission permit.
///
/// Once this timeout expires the request fails before body ingest/storage
/// mutation with S3 `SlowDown`/503. `0` means fail fast when the limit is full.
pub const ENV_PUT_FOREGROUND_ADMISSION_WAIT_TIMEOUT_MS: &str = "RUSTFS_PUT_FOREGROUND_ADMISSION_WAIT_TIMEOUT_MS";
pub const DEFAULT_PUT_FOREGROUND_ADMISSION_WAIT_TIMEOUT_MS: u64 = 0;
const _: () = assert!(!DEFAULT_PUT_FOREGROUND_ADMISSION_ENABLE);
/// Environment variable for minimum GetObject timeout in seconds.
///
/// When dynamic timeout calculation is enabled, this is the minimum timeout
File diff suppressed because it is too large Load Diff
@@ -33,7 +33,6 @@ use aws_sdk_s3::types::{
ObjectLockMode, ObjectLockRetentionMode,
};
use chrono::{DateTime, Duration, Utc};
use serial_test::serial;
use tracing::info;
/// Initialize test logging
@@ -107,7 +106,6 @@ fn parse_s3_datetime(value: &aws_sdk_s3::primitives::DateTime) -> DateTime<Utc>
// ============================================================================
#[tokio::test]
#[serial]
async fn test_delete_object_blocked_by_compliance_retention() {
init_logging();
info!("🧪 Test: DeleteObject blocked by COMPLIANCE retention");
@@ -145,7 +143,6 @@ async fn test_delete_object_blocked_by_compliance_retention() {
}
#[tokio::test]
#[serial]
async fn test_delete_object_blocked_by_governance_without_bypass() {
init_logging();
info!("🧪 Test: DeleteObject blocked by GOVERNANCE retention without bypass");
@@ -175,7 +172,6 @@ async fn test_delete_object_blocked_by_governance_without_bypass() {
}
#[tokio::test]
#[serial]
async fn test_delete_object_allowed_by_governance_with_bypass() {
init_logging();
info!("🧪 Test: DeleteObject allowed by GOVERNANCE retention with bypass");
@@ -215,7 +211,6 @@ async fn test_delete_object_allowed_by_governance_with_bypass() {
}
#[tokio::test]
#[serial]
async fn test_delete_object_creates_delete_marker_for_retained_current_version() {
init_logging();
info!("🧪 Test: DeleteObject creates delete marker for retained current version");
@@ -266,7 +261,6 @@ async fn test_delete_object_creates_delete_marker_for_retained_current_version()
}
#[tokio::test]
#[serial]
async fn test_delete_object_blocked_by_legal_hold() {
init_logging();
info!("🧪 Test: DeleteObject blocked by Legal Hold");
@@ -299,7 +293,6 @@ async fn test_delete_object_blocked_by_legal_hold() {
}
#[tokio::test]
#[serial]
async fn test_delete_object_allowed_with_legal_hold_off() {
init_logging();
info!("🧪 Test: DeleteObject allowed with Legal Hold OFF");
@@ -335,7 +328,6 @@ async fn test_delete_object_allowed_with_legal_hold_off() {
}
#[tokio::test]
#[serial]
async fn test_delete_object_after_legal_hold_removed() {
init_logging();
info!("🧪 Test: DeleteObject succeeds after Legal Hold is removed");
@@ -369,7 +361,6 @@ async fn test_delete_object_after_legal_hold_removed() {
}
#[tokio::test]
#[serial]
async fn test_get_object_legal_hold_returns_updated_status() {
init_logging();
info!("🧪 Test: GetObjectLegalHold returns updated status");
@@ -425,7 +416,6 @@ async fn test_get_object_legal_hold_returns_updated_status() {
}
#[tokio::test]
#[serial]
async fn test_get_object_retention_returns_configured_values() {
init_logging();
info!("🧪 Test: GetObjectRetention returns configured values");
@@ -476,7 +466,6 @@ async fn test_get_object_retention_returns_configured_values() {
// creating a new current version. The lock protects the existing version
// from deletion; it never blocks new versions.
#[tokio::test]
#[serial]
async fn test_put_object_overwrite_creates_new_version_under_legal_hold() {
init_logging();
info!("🧪 Test: PutObject overwrite of a legal-hold version creates a new version");
@@ -561,7 +550,6 @@ async fn test_put_object_overwrite_creates_new_version_under_legal_hold() {
}
#[tokio::test]
#[serial]
async fn test_copy_object_applies_requested_legal_hold() {
init_logging();
info!("🧪 Test: CopyObject applies requested Legal Hold");
@@ -613,7 +601,6 @@ async fn test_copy_object_applies_requested_legal_hold() {
}
#[tokio::test]
#[serial]
async fn test_copy_object_does_not_inherit_source_legal_hold() {
init_logging();
info!("🧪 Test: CopyObject does not inherit source Legal Hold");
@@ -707,7 +694,6 @@ async fn test_copy_object_does_not_inherit_source_legal_hold() {
}
#[tokio::test]
#[serial]
async fn test_copy_object_overwrite_creates_new_version_under_legal_hold() {
init_logging();
info!("🧪 Test: CopyObject overwrite of a legal-hold destination creates a new version");
@@ -787,7 +773,6 @@ async fn test_copy_object_overwrite_creates_new_version_under_legal_hold() {
}
#[tokio::test]
#[serial]
async fn test_create_multipart_upload_applies_requested_legal_hold() {
init_logging();
info!("🧪 Test: CreateMultipartUpload applies requested Legal Hold");
@@ -853,7 +838,6 @@ async fn test_create_multipart_upload_applies_requested_legal_hold() {
}
#[tokio::test]
#[serial]
async fn test_create_multipart_upload_creates_new_version_under_compliance_retention() {
init_logging();
info!("🧪 Test: CreateMultipartUpload over a COMPLIANCE-retained key creates a new version");
@@ -933,7 +917,6 @@ async fn test_create_multipart_upload_creates_new_version_under_compliance_reten
}
#[tokio::test]
#[serial]
async fn test_delete_completed_multipart_object_blocked_by_legal_hold() {
init_logging();
info!("🧪 Test: Delete completed multipart object blocked by Legal Hold");
@@ -993,7 +976,6 @@ async fn test_delete_completed_multipart_object_blocked_by_legal_hold() {
}
#[tokio::test]
#[serial]
async fn test_delete_completed_multipart_object_blocked_by_retention() {
init_logging();
info!("🧪 Test: Delete completed multipart object blocked by retention");
@@ -1055,7 +1037,6 @@ async fn test_delete_completed_multipart_object_blocked_by_retention() {
}
#[tokio::test]
#[serial]
async fn test_complete_multipart_upload_creates_new_version_under_legal_hold() {
init_logging();
info!("🧪 Test: CompleteMultipartUpload creates a new version when the current version is under Legal Hold");
@@ -1135,7 +1116,6 @@ async fn test_complete_multipart_upload_creates_new_version_under_legal_hold() {
}
#[tokio::test]
#[serial]
async fn test_complete_multipart_upload_creates_new_version_under_compliance_retention() {
init_logging();
info!("🧪 Test: CompleteMultipartUpload creates a new version when the current version is under COMPLIANCE retention");
@@ -1209,7 +1189,6 @@ async fn test_complete_multipart_upload_creates_new_version_under_compliance_ret
}
#[tokio::test]
#[serial]
async fn test_write_paths_require_put_object_legal_hold_permission() {
init_logging();
info!("🧪 Test: write paths require PutObjectLegalHold permission");
@@ -1273,7 +1252,6 @@ async fn test_write_paths_require_put_object_legal_hold_permission() {
}
#[tokio::test]
#[serial]
async fn test_write_paths_require_put_object_retention_permission() {
init_logging();
info!("🧪 Test: write paths require PutObjectRetention permission");
@@ -1345,7 +1323,6 @@ async fn test_write_paths_require_put_object_retention_permission() {
// ============================================================================
#[tokio::test]
#[serial]
async fn test_delete_objects_mixed_locked_unlocked() {
init_logging();
info!("🧪 Test: DeleteObjects with mixed locked and unlocked objects");
@@ -1427,7 +1404,6 @@ async fn test_delete_objects_mixed_locked_unlocked() {
// ============================================================================
#[tokio::test]
#[serial]
async fn test_put_retention_compliance_cannot_shorten() {
init_logging();
info!("🧪 Test: PutObjectRetention cannot shorten COMPLIANCE retention");
@@ -1468,7 +1444,6 @@ async fn test_put_retention_compliance_cannot_shorten() {
}
#[tokio::test]
#[serial]
async fn test_put_retention_compliance_can_extend() {
init_logging();
info!("🧪 Test: PutObjectRetention can extend COMPLIANCE retention");
@@ -1509,7 +1484,6 @@ async fn test_put_retention_compliance_can_extend() {
}
#[tokio::test]
#[serial]
async fn test_put_retention_governance_extend_without_bypass() {
init_logging();
info!("🧪 Test: PutObjectRetention on GOVERNANCE can extend without bypass");
@@ -1553,7 +1527,6 @@ async fn test_put_retention_governance_extend_without_bypass() {
}
#[tokio::test]
#[serial]
async fn test_put_retention_governance_shorten_requires_bypass() {
init_logging();
info!("🧪 Test: PutObjectRetention on GOVERNANCE requires bypass to shorten");
@@ -1615,7 +1588,6 @@ async fn test_put_retention_governance_shorten_requires_bypass() {
// ============================================================================
#[tokio::test]
#[serial]
async fn test_default_retention_applied_to_new_objects() {
init_logging();
info!("🧪 Test: Default retention is applied to new objects");
@@ -1685,7 +1657,6 @@ async fn test_default_retention_applied_to_new_objects() {
}
#[tokio::test]
#[serial]
async fn test_delete_object_creates_delete_marker_for_default_retained_current_version() {
init_logging();
info!("🧪 Test: DeleteObject creates delete marker for default-retained current version");
@@ -1770,7 +1741,6 @@ async fn test_delete_object_creates_delete_marker_for_default_retained_current_v
}
#[tokio::test]
#[serial]
async fn test_put_copy_and_multipart_reject_incomplete_retention_headers() {
init_logging();
info!("🧪 Test: write paths reject incomplete Object Lock retention headers");
@@ -1869,7 +1839,6 @@ async fn test_put_copy_and_multipart_reject_incomplete_retention_headers() {
}
#[tokio::test]
#[serial]
async fn test_copy_object_retention_uses_destination_policy() {
init_logging();
info!("🧪 Test: CopyObject retention follows destination policy");
@@ -2051,7 +2020,6 @@ async fn test_copy_object_retention_uses_destination_policy() {
}
#[tokio::test]
#[serial]
async fn test_multipart_default_retention_fixed_at_create() {
init_logging();
info!("🧪 Test: multipart default retention is fixed at CreateMultipartUpload");
@@ -2122,7 +2090,6 @@ async fn test_multipart_default_retention_fixed_at_create() {
// ============================================================================
#[tokio::test]
#[serial]
async fn test_unretained_object_lock_object_delete_and_bucket_cleanup() {
init_logging();
info!("🧪 Test: Unretained Object Lock object delete and bucket cleanup (Issue #5339)");
@@ -2243,7 +2210,6 @@ async fn test_unretained_object_lock_object_delete_and_bucket_cleanup() {
}
#[tokio::test]
#[serial]
async fn test_versioning_auto_enabled_with_object_lock() {
init_logging();
info!("🧪 Test: Versioning is auto-enabled when Object Lock is configured");
@@ -2302,7 +2268,6 @@ async fn test_versioning_auto_enabled_with_object_lock() {
// ============================================================================
#[tokio::test]
#[serial]
async fn test_error_message_distinguishes_legal_hold_from_retention() {
init_logging();
info!("🧪 Test: Error messages distinguish Legal Hold from Retention");
@@ -60,7 +60,6 @@ use rustfs_signer::constants::UNSIGNED_PAYLOAD;
use rustfs_signer::sign_v4;
use s3s::Body;
use s3s::header::X_AMZ_REPLICATION_STATUS;
use serial_test::serial;
use sha2::{Digest, Sha256};
use std::collections::BTreeMap;
use std::convert::Infallible;
@@ -2506,7 +2505,6 @@ async fn build_replication_pair(
/// metadata was inherited wholesale from the source, so the scanner heal pass
/// skipped it too — no PENDING/FAILED marker meant nothing to re-drive).
#[tokio::test]
#[serial]
async fn test_copy_object_replicates_to_target() -> TestResult {
init_logging();
@@ -2555,7 +2553,6 @@ async fn test_copy_object_replicates_to_target() -> TestResult {
/// independent object; every member must replicate to the remote target like a
/// regular PUT (MinIO PutObjectExtract parity).
#[tokio::test]
#[serial]
async fn test_snowball_extract_replicates_members_to_target() -> TestResult {
init_logging();
@@ -2601,7 +2598,6 @@ async fn test_snowball_extract_replicates_members_to_target() -> TestResult {
}
#[tokio::test]
#[serial]
async fn test_replication_check_succeeds_with_remote_target() -> Result<(), Box<dyn Error + Send + Sync>> {
init_logging();
@@ -2638,7 +2634,6 @@ async fn test_replication_check_succeeds_with_remote_target() -> Result<(), Box<
}
#[tokio::test]
#[serial]
async fn test_replication_check_rejects_target_without_object_lock() -> Result<(), Box<dyn Error + Send + Sync>> {
init_logging();
@@ -2692,7 +2687,6 @@ async fn test_replication_check_rejects_target_without_object_lock() -> Result<(
}
#[tokio::test]
#[serial]
async fn test_set_remote_target_rejects_unversioned_source_bucket() -> Result<(), Box<dyn Error + Send + Sync>> {
init_logging();
@@ -2731,7 +2725,6 @@ async fn test_set_remote_target_rejects_unversioned_source_bucket() -> Result<()
}
#[tokio::test]
#[serial]
async fn test_replication_check_rejects_unversioned_source_bucket() -> Result<(), Box<dyn Error + Send + Sync>> {
init_logging();
@@ -2755,7 +2748,6 @@ async fn test_replication_check_rejects_unversioned_source_bucket() -> Result<()
}
#[tokio::test]
#[serial]
async fn test_replication_check_rejects_missing_replication_config() -> Result<(), Box<dyn Error + Send + Sync>> {
init_logging();
@@ -2779,7 +2771,6 @@ async fn test_replication_check_rejects_missing_replication_config() -> Result<(
}
#[tokio::test]
#[serial]
async fn test_replication_check_rejects_invalid_bucket() -> Result<(), Box<dyn Error + Send + Sync>> {
init_logging();
@@ -2798,7 +2789,6 @@ async fn test_replication_check_rejects_invalid_bucket() -> Result<(), Box<dyn E
}
#[tokio::test]
#[serial]
async fn test_set_remote_target_rejects_same_bucket_on_same_deployment() -> Result<(), Box<dyn Error + Send + Sync>> {
init_logging();
@@ -2842,7 +2832,6 @@ async fn test_set_remote_target_rejects_same_bucket_on_same_deployment() -> Resu
}
#[tokio::test]
#[serial]
async fn test_set_remote_target_rejects_unversioned_target_bucket() -> Result<(), Box<dyn Error + Send + Sync>> {
init_logging();
@@ -2876,7 +2865,6 @@ async fn test_set_remote_target_rejects_unversioned_target_bucket() -> Result<()
}
#[tokio::test]
#[serial]
async fn test_set_remote_target_update_requires_arn() -> Result<(), Box<dyn Error + Send + Sync>> {
init_logging();
@@ -2928,7 +2916,6 @@ async fn test_set_remote_target_update_requires_arn() -> Result<(), Box<dyn Erro
}
#[tokio::test]
#[serial]
async fn test_set_remote_target_update_rejects_missing_target() -> Result<(), Box<dyn Error + Send + Sync>> {
init_logging();
@@ -3017,7 +3004,6 @@ async fn fetch_single_target(
}
#[tokio::test]
#[serial]
async fn test_set_remote_target_partial_update_preserves_credentials() -> Result<(), Box<dyn Error + Send + Sync>> {
init_logging();
@@ -3103,7 +3089,6 @@ async fn test_set_remote_target_partial_update_preserves_credentials() -> Result
}
#[tokio::test]
#[serial]
async fn test_set_remote_target_rejects_invalid_target_url() -> Result<(), Box<dyn Error + Send + Sync>> {
init_logging();
@@ -3145,7 +3130,6 @@ async fn test_set_remote_target_rejects_invalid_target_url() -> Result<(), Box<d
}
#[tokio::test]
#[serial]
async fn test_set_remote_target_rejects_self_signed_https_target_without_skip_tls_verify()
-> Result<(), Box<dyn Error + Send + Sync>> {
init_logging();
@@ -3230,7 +3214,6 @@ async fn test_set_remote_target_rejects_self_signed_https_target_without_skip_tl
}
#[tokio::test]
#[serial]
async fn test_set_remote_target_allows_self_signed_https_target_with_skip_tls_verify() -> Result<(), Box<dyn Error + Send + Sync>>
{
init_logging();
@@ -3342,7 +3325,6 @@ async fn test_set_remote_target_allows_self_signed_https_target_with_skip_tls_ve
}
#[tokio::test]
#[serial]
async fn test_set_remote_target_rejects_private_ca_https_target_without_ca_cert_pem() -> Result<(), Box<dyn Error + Send + Sync>>
{
init_logging();
@@ -3427,7 +3409,6 @@ async fn test_set_remote_target_rejects_private_ca_https_target_without_ca_cert_
}
#[tokio::test]
#[serial]
async fn test_set_remote_target_allows_private_ca_https_target_with_ca_cert_pem() -> Result<(), Box<dyn Error + Send + Sync>> {
init_logging();
@@ -3518,7 +3499,6 @@ async fn test_set_remote_target_allows_private_ca_https_target_with_ca_cert_pem(
}
#[tokio::test]
#[serial]
async fn test_list_remote_targets_rejects_empty_bucket() -> Result<(), Box<dyn Error + Send + Sync>> {
init_logging();
@@ -3538,7 +3518,6 @@ async fn test_list_remote_targets_rejects_empty_bucket() -> Result<(), Box<dyn E
}
#[tokio::test]
#[serial]
async fn test_list_remote_targets_rejects_invalid_bucket() -> Result<(), Box<dyn Error + Send + Sync>> {
init_logging();
@@ -3557,7 +3536,6 @@ async fn test_list_remote_targets_rejects_invalid_bucket() -> Result<(), Box<dyn
}
#[tokio::test]
#[serial]
async fn test_remove_remote_target_rejects_missing_target() -> Result<(), Box<dyn Error + Send + Sync>> {
init_logging();
@@ -3598,7 +3576,6 @@ async fn test_remove_remote_target_rejects_missing_target() -> Result<(), Box<dy
}
#[tokio::test]
#[serial]
async fn test_remove_remote_target_rejects_missing_arn() -> Result<(), Box<dyn Error + Send + Sync>> {
init_logging();
@@ -3623,7 +3600,6 @@ async fn test_remove_remote_target_rejects_missing_arn() -> Result<(), Box<dyn E
}
#[tokio::test]
#[serial]
async fn test_remove_remote_target_rejects_invalid_bucket() -> Result<(), Box<dyn Error + Send + Sync>> {
init_logging();
@@ -3647,7 +3623,6 @@ async fn test_remove_remote_target_rejects_invalid_bucket() -> Result<(), Box<dy
}
#[tokio::test]
#[serial]
async fn test_remove_remote_target_rejects_target_used_by_replication() -> Result<(), Box<dyn Error + Send + Sync>> {
init_logging();
@@ -3687,7 +3662,6 @@ async fn test_remove_remote_target_rejects_target_used_by_replication() -> Resul
}
#[tokio::test]
#[serial]
async fn test_delete_bucket_replication_removes_remote_target() -> Result<(), Box<dyn Error + Send + Sync>> {
init_logging();
@@ -3737,7 +3711,6 @@ async fn test_delete_bucket_replication_removes_remote_target() -> Result<(), Bo
}
#[tokio::test]
#[serial]
async fn test_bucket_replication_replicates_put_object_issue_2539() -> Result<(), Box<dyn Error + Send + Sync>> {
init_logging();
@@ -3779,7 +3752,6 @@ async fn test_bucket_replication_replicates_put_object_issue_2539() -> Result<()
}
#[tokio::test]
#[serial]
async fn test_bucket_replication_converges_delete_marker_and_version_purge() -> TestResult {
init_logging();
@@ -3878,7 +3850,6 @@ async fn test_bucket_replication_converges_delete_marker_and_version_purge() ->
}
#[tokio::test]
#[serial]
async fn test_bucket_replication_disabled_delete_marker_does_not_propagate() -> TestResult {
init_logging();
@@ -3965,7 +3936,6 @@ async fn test_bucket_replication_disabled_delete_marker_does_not_propagate() ->
/// interoperability profile for a runner that provisions MinIO credentials
/// and a reachable endpoint.
#[tokio::test]
#[serial]
async fn test_bucket_replication_acceptance_matrix_local_dual_targets() -> TestResult {
init_logging();
@@ -4293,7 +4263,6 @@ async fn test_bucket_replication_acceptance_matrix_local_dual_targets() -> TestR
}
#[tokio::test]
#[serial]
async fn test_single_bucket_multipart_replication_fans_out_to_multiple_targets() -> Result<(), Box<dyn Error + Send + Sync>> {
init_logging();
@@ -4462,7 +4431,6 @@ async fn test_repl17_failure_observation_helpers() -> TestResult {
/// the replica is decryptable only with the original customer key. The
/// backlog#1291 property still holds: never a silent plaintext replica.
#[tokio::test]
#[serial]
async fn test_bucket_replication_sse_c_contract() -> TestResult {
init_logging();
@@ -4540,7 +4508,6 @@ async fn test_bucket_replication_sse_c_contract() -> TestResult {
/// part — part boundaries and the encrypted-multipart marker survive so the
/// replica decrypts each part with its part-derived nonce.
#[tokio::test]
#[serial]
async fn test_bucket_replication_sse_c_multipart_passthrough() -> TestResult {
init_logging();
@@ -4657,7 +4624,6 @@ async fn test_bucket_replication_sse_c_multipart_passthrough() -> TestResult {
/// (independent KMS, so success proves target-owned envelopes), preserved
/// source ETag, and a version that stays stable across scanner cycles.
#[tokio::test]
#[serial]
async fn test_bucket_replication_sse_s3_contract() -> TestResult {
init_logging();
assert_managed_sse_replicates_and_reencrypts("sse-s3", false).await
@@ -4667,7 +4633,6 @@ async fn test_bucket_replication_sse_s3_contract() -> TestResult {
/// fail closed — replication FAILED, and no plaintext (or any) replica ever
/// materializes on the target.
#[tokio::test]
#[serial]
async fn test_bucket_replication_sse_s3_fails_closed_without_target_kms() -> TestResult {
init_logging();
@@ -4711,7 +4676,6 @@ async fn test_bucket_replication_sse_s3_fails_closed_without_target_kms() -> Tes
/// the ETag comparison sees the preserved source ETag on the replica and does
/// not rewrite it, so the replica's version stays stable through the resync.
#[tokio::test]
#[serial]
async fn test_bucket_replication_sse_s3_resync_converges() -> TestResult {
init_logging();
@@ -4768,7 +4732,6 @@ async fn test_bucket_replication_sse_s3_resync_converges() -> TestResult {
/// re-encrypts under its own default key. The independent-KMS pair proves the
/// replica's envelope is target-owned.
#[tokio::test]
#[serial]
async fn test_bucket_replication_sse_kms_contract() -> TestResult {
init_logging();
assert_managed_sse_replicates_and_reencrypts("sse-kms", true).await
@@ -4779,7 +4742,6 @@ async fn test_bucket_replication_sse_kms_contract() -> TestResult {
/// carries the full header set (SSE intent, content-type, user metadata) and
/// the completed replica preserves the source's multipart ETag.
#[tokio::test]
#[serial]
async fn test_bucket_replication_sse_s3_multipart_reencrypts() -> TestResult {
init_logging();
@@ -4873,7 +4835,6 @@ async fn test_bucket_replication_sse_s3_multipart_reencrypts() -> TestResult {
/// still-running source's data scanner (short cycle via [`FAST_SCANNER_ENV`])
/// re-drives the failed objects once the target is reachable again.
#[tokio::test]
#[serial]
async fn test_bucket_replication_recovers_after_target_outage() -> TestResult {
init_logging();
@@ -4953,7 +4914,6 @@ async fn test_bucket_replication_recovers_after_target_outage() -> TestResult {
/// must settle back to zero even though the historical failed counter remains
/// non-zero.
#[tokio::test]
#[serial]
async fn test_bucket_replication_backlog_metrics_observe_outage_and_recovery() -> TestResult {
init_logging();
@@ -5087,7 +5047,6 @@ async fn test_bucket_replication_backlog_metrics_observe_outage_and_recovery() -
/// must converge every persisted failure, including the replayed delete marker
/// (whose replication decision is re-derived from the live config).
#[tokio::test]
#[serial]
async fn test_bucket_replication_replays_failed_entries_after_source_restart() -> TestResult {
init_logging();
@@ -5179,7 +5138,6 @@ async fn test_bucket_replication_replays_failed_entries_after_source_restart() -
}
#[tokio::test]
#[serial]
async fn test_bucket_replication_replayed_delete_marker_preserves_source_mtime_without_source_restart() -> TestResult {
init_logging();
@@ -5249,7 +5207,6 @@ async fn test_bucket_replication_replayed_delete_marker_preserves_source_mtime_w
}
#[tokio::test]
#[serial]
async fn test_sequential_bucket_replication_succeeds_for_multiple_buckets() -> Result<(), Box<dyn Error + Send + Sync>> {
init_logging();
@@ -5293,7 +5250,6 @@ async fn test_sequential_bucket_replication_succeeds_for_multiple_buckets() -> R
}
#[tokio::test]
#[serial]
async fn test_replication_recovers_after_runtime_target_cache_is_cleared() -> Result<(), Box<dyn Error + Send + Sync>> {
init_logging();
@@ -5337,7 +5293,6 @@ async fn test_replication_recovers_after_runtime_target_cache_is_cleared() -> Re
}
#[tokio::test]
#[serial]
async fn test_site_replication_allows_self_signed_https_with_skip_tls_verify_real_dual_node() -> TestResult {
init_logging();
@@ -5416,7 +5371,6 @@ async fn test_site_replication_allows_self_signed_https_with_skip_tls_verify_rea
}
#[tokio::test]
#[serial]
async fn test_site_replication_allows_private_ca_https_with_ca_cert_pem_real_dual_node() -> TestResult {
init_logging();
@@ -5495,7 +5449,6 @@ async fn test_site_replication_allows_private_ca_https_with_ca_cert_pem_real_dua
}
#[tokio::test]
#[serial]
async fn test_site_replication_resync_lifecycle_survives_real_server_restart() -> Result<(), Box<dyn Error + Send + Sync>> {
init_logging();
let resync_process_env = [
@@ -5715,7 +5668,6 @@ async fn test_site_replication_resync_lifecycle_survives_real_server_restart() -
}
#[tokio::test]
#[serial]
async fn test_site_replication_edit_and_status_peer_state_real_three_node() -> Result<(), Box<dyn Error + Send + Sync>> {
init_logging();
@@ -5964,7 +5916,6 @@ async fn test_site_replication_edit_and_status_peer_state_real_three_node() -> R
}
#[tokio::test]
#[serial]
async fn test_site_replication_remove_all_real_dual_node() -> Result<(), Box<dyn Error + Send + Sync>> {
init_logging();
@@ -6084,7 +6035,6 @@ async fn test_site_replication_remove_all_real_dual_node() -> Result<(), Box<dyn
}
#[tokio::test]
#[serial]
async fn test_site_replication_state_edit_fresh_and_stale_real_dual_node() -> Result<(), Box<dyn Error + Send + Sync>> {
init_logging();
@@ -6193,7 +6143,6 @@ async fn test_site_replication_state_edit_fresh_and_stale_real_dual_node() -> Re
}
#[tokio::test]
#[serial]
async fn test_site_replication_replicates_object_with_bucket_versioning_real_dual_node() -> TestResult {
init_logging();
@@ -6284,7 +6233,6 @@ async fn test_site_replication_replicates_object_with_bucket_versioning_real_dua
/// receiver was dropped with only a debug line, while `replicate status` still reported
/// "1/1 Buckets in sync" because both configs were byte-identical.
#[tokio::test]
#[serial]
async fn test_site_replication_config_broadcast_keeps_reverse_direction_real_dual_node() -> TestResult {
init_logging();
@@ -6423,7 +6371,6 @@ async fn wait_for_site_replication_rule(
}
#[tokio::test]
#[serial]
async fn test_site_replication_active_active_converges_without_loops_real_dual_node() -> TestResult {
init_logging();
@@ -6741,7 +6688,6 @@ async fn test_site_replication_active_active_converges_without_loops_real_dual_n
}
#[tokio::test]
#[serial]
async fn test_site_replication_replicates_policy_backed_user_access_real_dual_node() -> Result<(), Box<dyn Error + Send + Sync>> {
init_logging();
@@ -6829,7 +6775,6 @@ async fn test_site_replication_replicates_policy_backed_user_access_real_dual_no
}
#[tokio::test]
#[serial]
async fn test_site_replication_replicates_group_policy_backed_access_real_dual_node() -> Result<(), Box<dyn Error + Send + Sync>>
{
init_logging();
@@ -6920,7 +6865,6 @@ async fn test_site_replication_replicates_group_policy_backed_access_real_dual_n
}
#[tokio::test]
#[serial]
async fn test_service_account_policy_from_accountinfo_round_trips_real_single_node() -> TestResult {
init_logging();
@@ -6972,7 +6916,6 @@ async fn test_service_account_policy_from_accountinfo_round_trips_real_single_no
}
#[tokio::test]
#[serial]
async fn test_site_replication_replicates_multiple_service_accounts_real_dual_node() -> Result<(), Box<dyn Error + Send + Sync>> {
init_logging();
@@ -7073,7 +7016,6 @@ async fn test_site_replication_replicates_multiple_service_accounts_real_dual_no
}
#[tokio::test]
#[serial]
async fn test_site_replication_replicates_service_accounts_created_from_sts_session_real_dual_node() -> TestResult {
init_logging();
@@ -7214,7 +7156,6 @@ async fn wait_for_target_request_version_id(
}
#[tokio::test]
#[serial]
async fn test_bucket_resync_restart_revisits_objects_before_out_of_order_checkpoint() -> TestResult {
init_logging();
@@ -7333,7 +7274,6 @@ async fn test_bucket_resync_restart_revisits_objects_before_out_of_order_checkpo
/// CreateMultipartUpload (the version is decided at initiate time) must both
/// carry the source version as `?versionId=`.
#[tokio::test]
#[serial]
async fn test_replication_put_and_create_multipart_carry_source_version_id_query() -> TestResult {
init_logging();
@@ -7448,7 +7388,6 @@ async fn test_replication_put_and_create_multipart_carry_source_version_id_query
/// flow to the onward bucket, proving B's outbound replication and scanner
/// are live.
#[tokio::test]
#[serial]
async fn test_scanner_never_cascades_inbound_replicas() -> TestResult {
init_logging();
@@ -7529,7 +7468,6 @@ async fn test_scanner_never_cascades_inbound_replicas() -> TestResult {
/// version ids and still mint its own there — the check must not report OK
/// while multipart deletes and heals would silently miss.
#[tokio::test]
#[serial]
async fn test_replication_check_flags_multipart_only_version_minting_target() -> TestResult {
init_logging();
@@ -7602,7 +7540,6 @@ async fn test_replication_check_flags_multipart_only_version_minting_target() ->
}
#[tokio::test]
#[serial]
async fn test_replication_check_aborts_failed_multipart_probes() -> TestResult {
init_logging();
@@ -7765,7 +7702,6 @@ async fn test_replication_check_aborts_failed_multipart_probes() -> TestResult {
/// BucketRemoteTargetVersionMismatch — while still cleaning up the probe
/// object via the version id the target actually assigned.
#[tokio::test]
#[serial]
async fn test_replication_check_flags_version_minting_target() -> TestResult {
init_logging();
@@ -8006,7 +7942,6 @@ async fn wait_for_target_marker_purged(
/// the target forever. Contract under test: a failed purge attempt is retried
/// within the watch window and converges once the fault clears.
#[tokio::test]
#[serial]
async fn test_delayed_delete_marker_purge_retries_after_transient_target_failure() -> TestResult {
init_logging();
let source_bucket = "delayed-purge-retry-src";
@@ -8066,7 +8001,6 @@ async fn test_delayed_delete_marker_purge_retries_after_transient_target_failure
/// with an idempotent 204, which used to look like success and strand the
/// real marker on the target forever.
#[tokio::test]
#[serial]
async fn test_delayed_delete_marker_purge_uses_target_assigned_version() -> TestResult {
init_logging();
let source_bucket = "delayed-purge-mint-src";
@@ -8099,7 +8033,6 @@ async fn test_delayed_delete_marker_purge_uses_target_assigned_version() -> Test
/// replayed purge succeeds, the entry must be acknowledged instead of being
/// retained as Missed forever.
#[tokio::test]
#[serial]
async fn test_delayed_delete_marker_purge_exhaustion_persists_to_mrf_and_replays_on_restart() -> TestResult {
init_logging();
let source_bucket = "delayed-purge-mrf-src";
@@ -8236,7 +8169,6 @@ async fn build_scanner_compensation_pair(
/// nil-version objects entirely (`scanner_folder.rs` heal_replication), so it
/// must NEVER be compensated.
#[tokio::test]
#[serial]
async fn test_scanner_compensates_existing_objects_across_write_paths() -> TestResult {
init_logging();
let source_bucket = "scanner-comp-src";
@@ -8352,7 +8284,6 @@ async fn test_scanner_compensates_existing_objects_across_write_paths() -> TestR
/// written after the rule replicate normally (the setting only gates the
/// existing-object resync path).
#[tokio::test]
#[serial]
async fn test_scanner_never_compensates_when_existing_object_replication_disabled() -> TestResult {
init_logging();
let source_bucket = "scanner-disabled-src";
+1
View File
@@ -273,6 +273,7 @@ proptest = "1"
rcgen.workspace = true
insta = { workspace = true, features = ["yaml", "json"] }
rustfs-crypto = { workspace = true }
tonic-prost = { workspace = true }
[build-dependencies]
shadow-rs = { workspace = true, default-features = false, features = ["build", "metadata"] }
@@ -1783,7 +1783,7 @@ impl TransitionState {
.await;
}
global_metrics().record_scanner_transition_failed(1);
if !is_err_version_not_found(&err) && !is_err_object_not_found(&err) && !is_network_or_host_down(&err.to_string(), false) && !err.to_string().contains("use of closed network connection") {
if !is_err_version_not_found(&err) && !is_err_object_not_found(&err) && !is_network_or_host_down(&err.to_string(), false) {
error!(
event = EVENT_LIFECYCLE_TIER_OPERATION_FAILED,
component = LOG_COMPONENT_ECSTORE,
+2 -1
View File
@@ -20,6 +20,7 @@
#![allow(clippy::all)]
use http::{HeaderMap, HeaderName, HeaderValue};
use rustfs_utils::http::headers::AMZ_CHECKSUM_MODE;
use std::collections::HashMap;
use time::OffsetDateTime;
use tracing::warn;
@@ -76,7 +77,7 @@ impl GetObjectOptions {
}
}
if self.checksum {
headers.insert(HeaderName::from_static("x-amz-checksum-mode"), HeaderValue::from_static("ENABLED"));
headers.insert(HeaderName::from_static(AMZ_CHECKSUM_MODE), HeaderValue::from_static("ENABLED"));
}
headers
}
+10 -6
View File
@@ -54,6 +54,10 @@ use rustfs_config::MAX_S3_CLIENT_RESPONSE_SIZE;
use rustfs_rio::HashReader;
use rustfs_utils::HashAlgorithm;
use rustfs_utils::{
http::headers::{
AMZ_CHECKSUM_CRC32, AMZ_CHECKSUM_CRC32C, AMZ_CHECKSUM_CRC64NVME, AMZ_CHECKSUM_MODE, AMZ_CHECKSUM_SHA1,
AMZ_CHECKSUM_SHA256,
},
net::get_endpoint_url,
retry::{DEFAULT_RETRY_CAP, DEFAULT_RETRY_UNIT, MAX_JITTER, MAX_RETRY, RetryTimer},
};
@@ -1383,12 +1387,12 @@ pub(crate) fn to_object_info_for_provider(
};
// Extract checksums
let checksum_crc32 = get_header("x-amz-checksum-crc32");
let checksum_crc32c = get_header("x-amz-checksum-crc32c");
let checksum_sha1 = get_header("x-amz-checksum-sha1");
let checksum_sha256 = get_header("x-amz-checksum-sha256");
let checksum_crc64nvme = get_header("x-amz-checksum-crc64nvme");
let checksum_mode = get_header("x-amz-checksum-mode");
let checksum_crc32 = get_header(AMZ_CHECKSUM_CRC32);
let checksum_crc32c = get_header(AMZ_CHECKSUM_CRC32C);
let checksum_sha1 = get_header(AMZ_CHECKSUM_SHA1);
let checksum_sha256 = get_header(AMZ_CHECKSUM_SHA256);
let checksum_crc64nvme = get_header(AMZ_CHECKSUM_CRC64NVME);
let checksum_mode = get_header(AMZ_CHECKSUM_MODE);
// Build and return the ObjectInfo struct
Ok(ObjectInfo {
@@ -233,11 +233,17 @@ pub struct NsScannerCapabilityRequest {
#[async_trait]
pub trait InternodeDataTransport: Send + Sync + std::fmt::Debug {
async fn open_read(&self, request: ReadStreamRequest) -> Result<FileReader>;
async fn open_read_fresh(&self, request: ReadStreamRequest) -> Result<FileReader> {
self.open_read(request).await
}
/// Opens an owned-chunk stream when this transport can retain receive-buffer
/// ownership. `None` preserves the established `open_read` fallback.
async fn open_read_chunks(&self, _request: ReadStreamRequest) -> Result<Option<ChunkReaderBox>> {
Ok(None)
}
async fn open_read_chunks_fresh(&self, request: ReadStreamRequest) -> Result<Option<ChunkReaderBox>> {
self.open_read_chunks(request).await
}
async fn open_write(&self, request: WriteStreamRequest) -> Result<FileWriter>;
async fn open_walk_dir(&self, request: WalkDirStreamRequest) -> Result<FileReader>;
async fn open_ns_scanner(&self, _request: NsScannerStreamRequest) -> Result<FileReader> {
@@ -269,6 +275,15 @@ impl InternodeDataTransport for TcpHttpInternodeDataTransport {
))
}
async fn open_read_fresh(&self, request: ReadStreamRequest) -> Result<FileReader> {
let url = build_read_file_stream_url(&request);
let mut headers = json_headers();
build_auth_headers(&url, &Method::GET, &mut headers)?;
Ok(Box::new(
HttpReader::new_fresh_connection_with_stall_timeout(url, Method::GET, headers, None, request.stall_timeout).await?,
))
}
async fn open_read_chunks(&self, request: ReadStreamRequest) -> Result<Option<ChunkReaderBox>> {
let url = build_read_file_stream_url(&request);
let mut headers = json_headers();
@@ -278,6 +293,16 @@ impl InternodeDataTransport for TcpHttpInternodeDataTransport {
)))
}
async fn open_read_chunks_fresh(&self, request: ReadStreamRequest) -> Result<Option<ChunkReaderBox>> {
let url = build_read_file_stream_url(&request);
let mut headers = json_headers();
build_auth_headers(&url, &Method::GET, &mut headers)?;
Ok(Some(Box::new(
HttpChunkReader::new_fresh_connection_with_stall_timeout(url, Method::GET, headers, None, request.stall_timeout)
.await?,
)))
}
async fn open_write(&self, request: WriteStreamRequest) -> Result<FileWriter> {
let server_epoch = self.put_file_auth_capability(&request.endpoint).await?;
let nonce = server_epoch.map(|_| Uuid::new_v4());
@@ -86,6 +86,25 @@ const PEER_REST_RECOVERY_MAX_BACKOFF: Duration = Duration::from_secs(30);
const SCANNER_ACTIVITY_MAX_MESSAGE_SIZE: usize = 1024;
const REPLICATION_STATS_MAX_MESSAGE_SIZE: usize = 8 * 1024 * 1024;
/// Error for a peer that reported `success = false` without an `error_info` payload.
///
/// Same shape as `peer_s3_client::peer_failure_without_details`, over `StorageError`
/// instead of `DiskError`. The message names the operation (and the bucket, where the
/// operation has one) and nothing else, for two reasons:
///
/// - `finalize_result` classifies failures by message substring, so any text matching
/// `message_has_network_needle` would take an answering peer offline and evict its
/// connection over a plain application-level rejection.
/// - Quorum aggregation (`reduce_errs`) buckets `Io` errors by kind plus rendered
/// message, so a per-peer detail such as the peer address would split one shared
/// failure into single-count buckets and downgrade the dominant error.
fn peer_failure_without_details(op: &str, bucket: Option<&str>) -> Error {
match bucket {
Some(bucket) => Error::other(format!("{op}({bucket}): peer returned failure without error details")),
None => Error::other(format!("{op}: peer returned failure without error details")),
}
}
fn decode_bucket_stats_response(response: GetBucketStatsDataResponse) -> Result<BucketStats> {
if !response.success {
return Err(Error::other(
@@ -696,7 +715,7 @@ impl PeerRestClient {
if let Some(msg) = response.error_info {
return Err(Error::other(msg));
}
return Err(Error::other(""));
return Err(peer_failure_without_details("local_storage_info", None));
}
let data = response.storage_info;
@@ -719,7 +738,7 @@ impl PeerRestClient {
if let Some(msg) = response.error_info {
return Err(Error::other(msg));
}
return Err(Error::other(""));
return Err(peer_failure_without_details("server_info", None));
}
let data = response.server_properties;
@@ -742,7 +761,7 @@ impl PeerRestClient {
if let Some(msg) = response.error_info {
return Err(Error::other(msg));
}
return Err(Error::other(""));
return Err(peer_failure_without_details("get_cpus", None));
}
let data = response.cpus;
@@ -765,7 +784,7 @@ impl PeerRestClient {
if let Some(msg) = response.error_info {
return Err(Error::other(msg));
}
return Err(Error::other(""));
return Err(peer_failure_without_details("get_net_info", None));
}
let data = response.net_info;
@@ -788,7 +807,7 @@ impl PeerRestClient {
if let Some(msg) = response.error_info {
return Err(Error::other(msg));
}
return Err(Error::other(""));
return Err(peer_failure_without_details("get_partitions", None));
}
let data = response.partitions;
@@ -811,7 +830,7 @@ impl PeerRestClient {
if let Some(msg) = response.error_info {
return Err(Error::other(msg));
}
return Err(Error::other(""));
return Err(peer_failure_without_details("get_os_info", None));
}
let data = response.os_info;
@@ -832,7 +851,7 @@ impl PeerRestClient {
if let Some(msg) = response.error_info {
return Err(Error::other(msg));
}
return Err(Error::other(""));
return Err(peer_failure_without_details("get_se_linux_info", None));
}
let data = response.sys_services;
@@ -857,7 +876,7 @@ impl PeerRestClient {
if let Some(msg) = response.error_info {
return Err(Error::other(msg));
}
return Err(Error::other(""));
return Err(peer_failure_without_details("get_sys_config", None));
}
let data = response.sys_config;
@@ -882,7 +901,7 @@ impl PeerRestClient {
if let Some(msg) = response.error_info {
return Err(Error::other(msg));
}
return Err(Error::other(""));
return Err(peer_failure_without_details("get_sys_errors", None));
}
let data = response.sys_errors;
@@ -907,7 +926,7 @@ impl PeerRestClient {
if let Some(msg) = response.error_info {
return Err(Error::other(msg));
}
return Err(Error::other(""));
return Err(peer_failure_without_details("get_mem_info", None));
}
let data = response.mem_info;
@@ -939,7 +958,7 @@ impl PeerRestClient {
if let Some(msg) = response.error_info {
return Err(Error::other(msg));
}
return Err(Error::other(""));
return Err(peer_failure_without_details("get_metrics", None));
}
let data = response.realtime_metrics;
@@ -964,7 +983,7 @@ impl PeerRestClient {
if let Some(msg) = response.error_info {
return Err(Error::other(msg));
}
return Err(Error::other(""));
return Err(peer_failure_without_details("get_live_events", None));
}
Ok(PeerLiveEventsBatch {
@@ -989,7 +1008,7 @@ impl PeerRestClient {
if let Some(msg) = response.error_info {
return Err(Error::other(msg));
}
return Err(Error::other(""));
return Err(peer_failure_without_details("get_proc_info", None));
}
let data = response.proc_info;
@@ -1016,7 +1035,7 @@ impl PeerRestClient {
if let Some(msg) = response.error_info {
return Err(Error::other(msg));
}
return Err(Error::other(""));
return Err(peer_failure_without_details("start_profiling", None));
}
Ok(())
}
@@ -1323,7 +1342,7 @@ impl PeerRestClient {
if let Some(msg) = response.error_info {
return Err(Error::other(msg));
}
return Err(Error::other(""));
return Err(peer_failure_without_details("load_bucket_metadata", Some(bucket)));
}
Ok(())
}
@@ -1346,7 +1365,7 @@ impl PeerRestClient {
if let Some(msg) = response.error_info {
return Err(Error::other(msg));
}
return Err(Error::other(""));
return Err(peer_failure_without_details("delete_bucket_metadata", Some(bucket)));
}
Ok(())
}
@@ -1369,7 +1388,7 @@ impl PeerRestClient {
if let Some(msg) = response.error_info {
return Err(Error::other(msg));
}
return Err(Error::other(""));
return Err(peer_failure_without_details("delete_policy", None));
}
Ok(())
}
@@ -1392,7 +1411,7 @@ impl PeerRestClient {
if let Some(msg) = response.error_info {
return Err(Error::other(msg));
}
return Err(Error::other(""));
return Err(peer_failure_without_details("load_policy", None));
}
Ok(())
}
@@ -1417,7 +1436,7 @@ impl PeerRestClient {
if let Some(msg) = response.error_info {
return Err(Error::other(msg));
}
return Err(Error::other(""));
return Err(peer_failure_without_details("load_policy_mapping", None));
}
Ok(())
}
@@ -1440,7 +1459,7 @@ impl PeerRestClient {
if let Some(msg) = response.error_info {
return Err(Error::other(msg));
}
return Err(Error::other(""));
return Err(peer_failure_without_details("delete_user", None));
}
Ok(())
}
@@ -1463,7 +1482,7 @@ impl PeerRestClient {
if let Some(msg) = response.error_info {
return Err(Error::other(msg));
}
return Err(Error::other(""));
return Err(peer_failure_without_details("delete_service_account", None));
}
Ok(())
}
@@ -1487,7 +1506,7 @@ impl PeerRestClient {
if let Some(msg) = response.error_info {
return Err(Error::other(msg));
}
return Err(Error::other(""));
return Err(peer_failure_without_details("load_user", None));
}
Ok(())
}
@@ -1510,7 +1529,7 @@ impl PeerRestClient {
if let Some(msg) = response.error_info {
return Err(Error::other(msg));
}
return Err(Error::other(""));
return Err(peer_failure_without_details("load_service_account", None));
}
Ok(())
}
@@ -1533,7 +1552,7 @@ impl PeerRestClient {
if let Some(msg) = response.error_info {
return Err(Error::other(msg));
}
return Err(Error::other(""));
return Err(peer_failure_without_details("load_group", None));
}
Ok(())
}
@@ -1554,7 +1573,7 @@ impl PeerRestClient {
if let Some(msg) = response.error_info {
return Err(Error::other(msg));
}
return Err(Error::other(""));
return Err(peer_failure_without_details("reload_site_replication_config", None));
}
Ok(())
}
@@ -1597,7 +1616,7 @@ impl PeerRestClient {
if let Some(msg) = response.error_info {
return Err(Error::other(msg));
}
return Err(Error::other(""));
return Err(peer_failure_without_details("signal_service", None));
}
validate_signal_service_protocol(sig, sub_sys, response.protocol_version)?;
Ok(response)
@@ -1667,7 +1686,7 @@ impl PeerRestClient {
if let Some(msg) = response.error_info {
return Err(Error::other(msg));
}
return Err(Error::other(""));
return Err(peer_failure_without_details("reload_pool_meta", None));
}
Ok(())
@@ -1691,7 +1710,7 @@ impl PeerRestClient {
if let Some(msg) = response.error_info {
return Err(Error::other(msg));
}
return Err(Error::other(""));
return Err(peer_failure_without_details("stop_rebalance", None));
}
Ok(())
@@ -1725,7 +1744,7 @@ impl PeerRestClient {
if let Some(msg) = response.error_info {
return Err(Error::other(msg));
}
return Err(Error::other(""));
return Err(peer_failure_without_details("load_rebalance_meta", None));
}
Ok(())
@@ -1753,7 +1772,7 @@ impl PeerRestClient {
if let Some(msg) = response.error_info {
return Err(Error::other(msg));
}
return Err(Error::other(""));
return Err(peer_failure_without_details("start_decommission", None));
}
Ok(())
@@ -1777,7 +1796,7 @@ impl PeerRestClient {
if let Some(msg) = response.error_info {
return Err(Error::other(msg));
}
return Err(Error::other(""));
return Err(peer_failure_without_details("decommission_cancel", None));
}
Ok(())
@@ -1801,7 +1820,7 @@ impl PeerRestClient {
if let Some(msg) = response.error_info {
return Err(Error::other(msg));
}
return Err(Error::other(""));
return Err(peer_failure_without_details("clear_decommission", None));
}
Ok(())
@@ -1947,6 +1966,8 @@ fn tier_config_reload_status_outcome(status: tonic::Status) -> TierConfigReloadO
mod tests {
use super::*;
use crate::config::com::STORAGE_CLASS_SUB_SYS;
use crate::disk::error::DiskError;
use crate::disk::error_reduce::reduce_errs;
use crate::layout::{disks_layout::DisksLayout, endpoints::SetupType};
use rustfs_config::{ENV_KUBERNETES_SERVICE_HOST, ENV_LOCAL_ENDPOINT_HOST, ENV_STARTUP_TOPOLOGY_WAIT_MODE};
use serde_json::Value;
@@ -3098,4 +3119,115 @@ mod tests {
&& span.get("request_id").and_then(Value::as_str) == Some("req-peer-rest")
}));
}
/// Every operation name passed to `peer_failure_without_details` in this file.
const PEER_FAILURE_OPS: &[&str] = &[
"local_storage_info",
"server_info",
"get_cpus",
"get_net_info",
"get_partitions",
"get_os_info",
"get_se_linux_info",
"get_sys_config",
"get_sys_errors",
"get_mem_info",
"get_metrics",
"get_live_events",
"get_proc_info",
"start_profiling",
"load_bucket_metadata",
"delete_bucket_metadata",
"delete_policy",
"load_policy",
"load_policy_mapping",
"delete_user",
"delete_service_account",
"load_user",
"load_service_account",
"load_group",
"reload_site_replication_config",
"signal_service",
"reload_pool_meta",
"stop_rebalance",
"load_rebalance_meta",
"start_decommission",
"decommission_cancel",
"clear_decommission",
];
#[test]
fn peer_failure_without_details_names_operation_and_bucket() {
for op in PEER_FAILURE_OPS {
let message = peer_failure_without_details(op, None).to_string();
assert!(message.contains(op), "{op} message must name the operation: {message}");
}
for op in ["load_bucket_metadata", "delete_bucket_metadata"] {
let message = peer_failure_without_details(op, Some("ops-bucket")).to_string();
assert!(message.contains(op), "{op} message must name the operation: {message}");
assert!(message.contains("ops-bucket"), "{op} message must name the bucket: {message}");
}
}
#[test]
fn peer_failure_without_details_keeps_one_reduce_errs_bucket_per_operation() {
// reduce_errs groups Io errors by kind plus rendered message: peers failing the
// same operation must stay a single dominant error instead of one bucket per peer.
let per_peer_errs = (0..4)
.map(|_| Some(DiskError::from(peer_failure_without_details("load_bucket_metadata", Some("shared")))))
.collect::<Vec<_>>();
let (count, dominant) = reduce_errs(&per_peer_errs, &[]);
assert_eq!(count, 4, "one shared failure must not split into per-peer buckets");
assert_eq!(
dominant,
Some(DiskError::from(peer_failure_without_details("load_bucket_metadata", Some("shared"))))
);
assert_ne!(
peer_failure_without_details("load_bucket_metadata", Some("shared")).to_string(),
peer_failure_without_details("delete_bucket_metadata", Some("shared")).to_string()
);
assert_ne!(
peer_failure_without_details("load_bucket_metadata", Some("bucket-a")).to_string(),
peer_failure_without_details("load_bucket_metadata", Some("bucket-b")).to_string()
);
}
#[test]
fn peer_failure_without_details_never_reads_as_a_network_failure() {
// `finalize_result` marks the peer offline and evicts its connection whenever the
// message matches a network needle. A peer that answered `success = false` is alive,
// so no operation or bucket name may push this text over that classifier.
for op in PEER_FAILURE_OPS {
let err = peer_failure_without_details(op, None);
assert!(
!PeerRestClient::is_network_like_error(&err),
"{op} must not read as a transport failure: {err}"
);
let scoped = peer_failure_without_details(op, Some("bucket-name"));
assert!(
!PeerRestClient::is_network_like_error(&scoped),
"{op} must not read as a transport failure: {scoped}"
);
}
// The bucket name is caller-supplied. Every needle carries a space, which S3 bucket
// names cannot, and the name is closed by `)` before the literal text resumes, so no
// needle can straddle the boundary either.
for bucket in [
"timed-out",
"connection-reset",
"transport-error",
"broken-pipe",
"unavailable-logs",
] {
let err = peer_failure_without_details("load_bucket_metadata", Some(bucket));
assert!(
!PeerRestClient::is_network_like_error(&err),
"bucket {bucket} must not push the message over the network classifier: {err}"
);
}
}
}
@@ -220,6 +220,8 @@ fn pool_write_quorum(participant_count: usize) -> usize {
/// buckets `Error::Io` by kind plus rendered message, so any per-peer detail (address,
/// timing) would split one shared failure into single-count buckets and downgrade a real
/// dominant error into `ErasureWriteQuorum`.
///
/// `peer_rest_client` carries the same helper over `StorageError` for the same response shape.
fn peer_failure_without_details(op: &str, bucket: Option<&str>) -> Error {
match bucket {
Some(bucket) => Error::other(format!("{op}({bucket}): peer returned failure without error details")),
File diff suppressed because it is too large Load Diff
+104 -26
View File
@@ -418,6 +418,17 @@ pub struct DiskHealthTracker {
pub last_capacity_free: AtomicU64,
/// Last successful capacity probe timestamp
pub last_capacity_probe_unix_secs: AtomicI64,
/// Authoritative atomically published runtime/status pair.
state_snapshot: AtomicU64,
transition_lock: std::sync::Mutex<()>,
}
fn pack_health_state(runtime_state: RuntimeDriveHealthState, status: u32) -> u64 {
(u64::from(runtime_state as u32) << 32) | u64::from(status)
}
fn unpack_health_state(snapshot: u64) -> (RuntimeDriveHealthState, u32) {
(RuntimeDriveHealthState::from_u32((snapshot >> 32) as u32), snapshot as u32)
}
#[derive(Debug)]
@@ -739,6 +750,8 @@ impl DiskHealthTracker {
last_capacity_used: AtomicU64::new(0),
last_capacity_free: AtomicU64::new(0),
last_capacity_probe_unix_secs: AtomicI64::new(0),
state_snapshot: AtomicU64::new(pack_health_state(RuntimeDriveHealthState::Online, DISK_HEALTH_OK)),
transition_lock: std::sync::Mutex::new(()),
}
}
@@ -775,39 +788,52 @@ impl DiskHealthTracker {
/// Check if disk is faulty
pub fn is_faulty(&self) -> bool {
self.status.load(Ordering::Acquire) == DISK_HEALTH_FAULTY
unpack_health_state(self.state_snapshot.load(Ordering::Acquire)).1 == DISK_HEALTH_FAULTY
}
fn publish_state(&self, runtime_state: RuntimeDriveHealthState, status: u32) {
self.state_snapshot
.store(pack_health_state(runtime_state, status), Ordering::Release);
self.runtime_state.store(runtime_state as u32, Ordering::Release);
self.status.store(status, Ordering::Release);
}
/// Set disk as faulty
#[allow(dead_code, reason = "asserted by this file's tests (backlog#1823)")]
pub fn set_faulty(&self) {
self.status.store(DISK_HEALTH_FAULTY, Ordering::Release);
let _guard = self.transition_lock.lock().unwrap_or_else(|poisoned| poisoned.into_inner());
self.publish_state(RuntimeDriveHealthState::Offline, DISK_HEALTH_FAULTY);
}
/// Set disk as OK
pub fn set_ok(&self) {
self.status.store(DISK_HEALTH_OK, Ordering::Release);
let _guard = self.transition_lock.lock().unwrap_or_else(|poisoned| poisoned.into_inner());
self.publish_state(RuntimeDriveHealthState::Online, DISK_HEALTH_OK);
}
#[cfg(test)]
pub fn force_runtime_state_for_test(&self, state: RuntimeDriveHealthState) {
self.runtime_state.store(state as u32, Ordering::Release);
match state {
RuntimeDriveHealthState::Offline => self.set_faulty(),
RuntimeDriveHealthState::Online | RuntimeDriveHealthState::Suspect | RuntimeDriveHealthState::Returning => {
self.set_ok();
}
}
let _guard = self.transition_lock.lock().unwrap_or_else(|poisoned| poisoned.into_inner());
let status = if state == RuntimeDriveHealthState::Offline {
DISK_HEALTH_FAULTY
} else {
DISK_HEALTH_OK
};
self.publish_state(state, status);
}
pub fn swap_ok_to_faulty(&self) -> bool {
self.status
.compare_exchange(DISK_HEALTH_OK, DISK_HEALTH_FAULTY, Ordering::AcqRel, Ordering::Relaxed)
.is_ok()
let _guard = self.transition_lock.lock().unwrap_or_else(|poisoned| poisoned.into_inner());
let (_, status) = unpack_health_state(self.state_snapshot.load(Ordering::Acquire));
if status != DISK_HEALTH_OK {
return false;
}
self.publish_state(RuntimeDriveHealthState::Offline, DISK_HEALTH_FAULTY);
true
}
pub fn runtime_state(&self) -> RuntimeDriveHealthState {
RuntimeDriveHealthState::from_u32(self.runtime_state.load(Ordering::Acquire))
unpack_health_state(self.state_snapshot.load(Ordering::Acquire)).0
}
pub fn offline_duration(&self) -> Option<Duration> {
@@ -823,6 +849,7 @@ impl DiskHealthTracker {
}
pub fn mark_failure(&self, endpoint: &Endpoint, reason: &'static str) -> bool {
let _guard = self.transition_lock.lock().unwrap_or_else(|poisoned| poisoned.into_inner());
let current = self.runtime_state();
let now = current_unix_secs();
let next = match current {
@@ -851,24 +878,19 @@ impl DiskHealthTracker {
};
let became_offline = next == RuntimeDriveHealthState::Offline && current != RuntimeDriveHealthState::Offline;
if next == RuntimeDriveHealthState::Offline {
self.status.store(DISK_HEALTH_FAULTY, Ordering::Release);
} else {
self.status.store(DISK_HEALTH_OK, Ordering::Release);
}
self.transition_state(endpoint, current, next, reason);
became_offline
}
#[allow(dead_code, reason = "asserted by this file's tests (backlog#1823)")]
pub fn mark_offline(&self, endpoint: &Endpoint, reason: &'static str) -> bool {
let _guard = self.transition_lock.lock().unwrap_or_else(|poisoned| poisoned.into_inner());
let current = self.runtime_state();
if current == RuntimeDriveHealthState::Offline {
return false;
}
self.consecutive_successes.store(0, Ordering::Release);
self.status.store(DISK_HEALTH_FAULTY, Ordering::Release);
self.transition_state(endpoint, current, RuntimeDriveHealthState::Offline, reason);
true
}
@@ -882,11 +904,10 @@ impl DiskHealthTracker {
}
fn reset_for_store_init_retry_at(&self, endpoint: &Endpoint, now: Duration) {
let _guard = self.transition_lock.lock().unwrap_or_else(|poisoned| poisoned.into_inner());
let now_nanos = unix_nanos(now);
let now_secs = unix_secs_i64(now);
self.status.store(DISK_HEALTH_OK, Ordering::Release);
self.runtime_state
.store(RuntimeDriveHealthState::Online as u32, Ordering::Release);
self.publish_state(RuntimeDriveHealthState::Online, DISK_HEALTH_OK);
self.consecutive_failures.store(0, Ordering::Release);
self.consecutive_successes.store(0, Ordering::Release);
self.offline_since_unix_secs.store(0, Ordering::Release);
@@ -898,6 +919,7 @@ impl DiskHealthTracker {
}
pub fn mark_recovery_success(&self, endpoint: &Endpoint, reason: &'static str) -> bool {
let _guard = self.transition_lock.lock().unwrap_or_else(|poisoned| poisoned.into_inner());
let current = self.runtime_state();
let next = match current {
RuntimeDriveHealthState::Online => RuntimeDriveHealthState::Online,
@@ -918,7 +940,6 @@ impl DiskHealthTracker {
let became_online = next == RuntimeDriveHealthState::Online;
if became_online {
self.status.store(DISK_HEALTH_OK, Ordering::Release);
self.consecutive_failures.store(0, Ordering::Release);
self.consecutive_successes.store(0, Ordering::Release);
}
@@ -948,7 +969,13 @@ impl DiskHealthTracker {
return;
}
self.runtime_state.store(next as u32, Ordering::Release);
let current_status = unpack_health_state(self.state_snapshot.load(Ordering::Acquire)).1;
let status = match next {
RuntimeDriveHealthState::Offline => DISK_HEALTH_FAULTY,
RuntimeDriveHealthState::Returning => current_status,
RuntimeDriveHealthState::Online | RuntimeDriveHealthState::Suspect => DISK_HEALTH_OK,
};
self.publish_state(next, status);
self.last_transition_unix_secs
.store(current_unix_secs() as i64, Ordering::Release);
@@ -1217,7 +1244,7 @@ impl LocalDiskWrapper {
return;
}
if health.status.load(Ordering::Relaxed) != DISK_HEALTH_OK {
if health.is_faulty() {
continue;
}
@@ -2909,6 +2936,57 @@ mod tests {
});
}
#[test]
#[serial_test::serial]
fn concurrent_failure_and_recovery_publish_one_health_snapshot() {
temp_env::with_var(rustfs_config::ENV_DRIVE_SUSPECT_FAILURE_THRESHOLD, Some("2"), || {
let endpoint = Endpoint::try_from("/tmp/concurrent-health-snapshot").expect("endpoint should parse");
let health = Arc::new(DiskHealthTracker::new());
let transition_guard = health
.transition_lock
.lock()
.expect("health transition lock should not be poisoned");
let start = Arc::new(std::sync::Barrier::new(3));
let (completed_tx, completed_rx) = std::sync::mpsc::channel();
let workers = (0..2)
.map(|_| {
let health = Arc::clone(&health);
let endpoint = endpoint.clone();
let start = Arc::clone(&start);
let completed_tx = completed_tx.clone();
std::thread::spawn(move || {
start.wait();
health.mark_failure(&endpoint, "concurrent_test");
completed_tx.send(()).expect("completion receiver should remain available");
})
})
.collect::<Vec<_>>();
start.wait();
assert!(
matches!(
completed_rx.recv_timeout(Duration::from_millis(250)),
Err(std::sync::mpsc::RecvTimeoutError::Timeout)
),
"concurrent transitions must wait for the serialization lock"
);
drop(transition_guard);
completed_rx
.recv_timeout(Duration::from_secs(1))
.expect("first failure transition should complete after lock release");
completed_rx
.recv_timeout(Duration::from_secs(1))
.expect("second failure transition should complete after lock release");
for worker in workers {
worker.join().expect("health transition worker should not panic");
}
assert_eq!(health.runtime_state(), RuntimeDriveHealthState::Offline);
assert!(health.is_faulty());
assert_eq!(health.consecutive_failures.load(Ordering::Acquire), 2);
});
}
#[test]
fn operation_success_recovers_suspect_drive_without_faulting() {
let endpoint = Endpoint::try_from("/tmp/runtime-state-suspect-success").expect("endpoint should parse");
+8
View File
@@ -1116,6 +1116,14 @@ mod tests {
assert!(encoder_source.is::<reed_solomon_erasure::Error>());
}
// The lifecycle transition worker relies on this arm alone to suppress the
// closed-connection noise (`bucket_lifecycle_ops.rs`); dropping it here would
// silently turn shutdown races back into `error!` log spam.
#[test]
fn is_network_or_host_down_covers_closed_network_connection() {
assert!(is_network_or_host_down("transition failed: use of closed network connection", false));
}
// Regression for #952 (ECA-11): an all-`DiskNotFound` slice (every drive in
// every set unreachable) must NOT be classified as "all not found",
// otherwise ListObjects silently returns an empty listing and masks a full
@@ -5845,6 +5845,14 @@ impl crate::storage_api_contracts::object::ObjectOperations for SetDisks {
#[tracing::instrument(skip(self))]
async fn add_partial(&self, bucket: &str, object: &str, version_id: &str) -> Result<()> {
// MRF journal intent: partial-write recovery must survive a restart
// (HS-01); the heal request below remains the in-memory fast path.
rustfs_common::mrf_channel::try_send_mrf_intent(
rustfs_common::mrf_channel::MrfKind::PartialWrite,
bucket,
object,
uuid::Uuid::try_parse(version_id).ok(),
);
let mut request = rustfs_common::heal_channel::create_heal_request_with_options(
bucket.to_string(),
Some(object.to_string()),
+9
View File
@@ -1077,6 +1077,15 @@ impl SetDisks {
"Recoverable decode error triggered read repair"
);
let version_id = fi.version_id.as_ref().map(ToString::to_string);
// MRF journal intent: keeps a durable Urgent ECDecode
// request alive across restarts even when the in-memory
// read-repair request is dropped or lost (HS-01).
rustfs_common::mrf_channel::try_send_mrf_intent(
rustfs_common::mrf_channel::MrfKind::DecodeFailure,
bucket,
object,
fi.version_id,
);
submit_read_repair_heal(
bucket,
object,
+2
View File
@@ -89,6 +89,8 @@ async-trait = { workspace = true }
futures = { workspace = true }
metrics = { workspace = true }
base64 = { workspace = true }
bytes = { workspace = true }
crc-fast = { workspace = true }
[dev-dependencies]
serde_json = { workspace = true, features = ["raw_value"] }
+99 -17
View File
@@ -66,21 +66,37 @@ struct HealTaskStatusPayload<'a> {
summary: &'a str,
items: &'a [HealResultItem],
truncated: bool,
/// Cursor for incremental consumption (HS-06): sequence of the next item
/// to be produced. Absent on responses without sequencing (0).
#[serde(skip_serializing_if = "u64_is_zero")]
next_seq: u64,
/// Oldest sequence still retained; with `truncated`, tells a lagging
/// client where to restart its cursor.
#[serde(skip_serializing_if = "u64_is_zero")]
min_seq: u64,
#[serde(skip_serializing_if = "Option::is_none")]
progress: Option<&'a HealProgress>,
}
fn u64_is_zero(value: &u64) -> bool {
*value == 0
}
fn encode_heal_task_status_payload(
summary: &str,
mut items: Vec<HealResultItem>,
progress: Option<&HealProgress>,
mut truncated: bool,
next_seq: u64,
min_seq: u64,
) -> Result<(Vec<u8>, bool)> {
loop {
let data = serde_json::to_vec(&HealTaskStatusPayload {
summary,
items: &items,
truncated,
next_seq,
min_seq,
progress,
})
.map_err(|e| Error::Serialization(format!("failed to serialize heal task status: {e}")))?;
@@ -109,8 +125,10 @@ fn encode_heal_status_response(
progress: Option<&HealProgress>,
detail: Option<String>,
truncated: bool,
next_seq: u64,
min_seq: u64,
) -> Result<(Vec<u8>, Option<String>)> {
let (data, truncated) = encode_heal_task_status_payload(summary, items, progress, truncated)?;
let (data, truncated) = encode_heal_task_status_payload(summary, items, progress, truncated, next_seq, min_seq)?;
Ok((data, heal_status_detail(detail, truncated)))
}
@@ -138,8 +156,19 @@ impl HealChannelProcessor {
/// Execute a token query directly against the manager.
pub async fn execute_query_request(&self, heal_path: String, client_token: String) -> Result<HealChannelResponse> {
self.execute_query_request_since(heal_path, client_token, None).await
}
/// Incremental variant of [`Self::execute_query_request`] (HS-06).
pub async fn execute_query_request_since(
&self,
heal_path: String,
client_token: String,
since_seq: Option<u64>,
) -> Result<HealChannelResponse> {
let (response_tx, response_rx) = oneshot::channel();
self.process_query_request(heal_path, client_token, response_tx).await?;
self.process_query_request(heal_path, client_token, since_seq, response_tx)
.await?;
response_rx
.await
.map_err(|err| Error::other(format!("heal query channel closed: {err}")))?
@@ -262,8 +291,12 @@ impl HealChannelProcessor {
HealChannelCommand::Query {
heal_path,
client_token,
since_seq,
response_tx,
} => self.process_query_request(heal_path, client_token, response_tx).await,
} => {
self.process_query_request(heal_path, client_token, since_seq, response_tx)
.await
}
HealChannelCommand::Cancel {
heal_path,
client_token,
@@ -384,6 +417,7 @@ impl HealChannelProcessor {
&self,
heal_path: String,
client_token: String,
since_seq: Option<u64>,
response_tx: oneshot::Sender<std::result::Result<HealChannelResponse, String>>,
) -> Result<()> {
debug!(
@@ -398,72 +432,118 @@ impl HealChannelProcessor {
);
let report = if heal_path.trim_matches('/').is_empty() {
self.heal_manager.get_task_report(&client_token).await
self.heal_manager.get_task_report_since(&client_token, since_seq).await
} else {
self.heal_manager.get_task_report_for_path(&heal_path, &client_token).await
self.heal_manager
.get_task_report_for_path_since(&heal_path, &client_token, since_seq)
.await
};
let (summary, detail, items, truncated, progress) = match report {
let (summary, detail, items, truncated, progress, next_seq, min_seq) = match report {
Ok(HealTaskReport {
status: HealTaskStatus::Pending | HealTaskStatus::Running,
result_items,
result_items_truncated,
progress,
}) => ("running".to_string(), None, result_items, result_items_truncated, progress),
next_seq,
min_seq,
}) => (
"running".to_string(),
None,
result_items,
result_items_truncated,
progress,
next_seq,
min_seq,
),
Ok(HealTaskReport {
status: HealTaskStatus::Retrying { error, retry_attempt },
result_items,
result_items_truncated,
progress,
next_seq,
min_seq,
}) => (
"running".to_string(),
Some(format!("heal task retrying after recoverable failure, attempt {retry_attempt}: {error}")),
result_items,
result_items_truncated,
progress,
next_seq,
min_seq,
),
Ok(HealTaskReport {
status: HealTaskStatus::Completed,
result_items,
result_items_truncated,
progress,
}) => ("finished".to_string(), None, result_items, result_items_truncated, progress),
next_seq,
min_seq,
}) => (
"finished".to_string(),
None,
result_items,
result_items_truncated,
progress,
next_seq,
min_seq,
),
Ok(HealTaskReport {
status: HealTaskStatus::Cancelled,
result_items,
result_items_truncated,
progress,
next_seq,
min_seq,
}) => (
"stopped".to_string(),
Some("heal task cancelled".to_string()),
result_items,
result_items_truncated,
progress,
next_seq,
min_seq,
),
Ok(HealTaskReport {
status: HealTaskStatus::Timeout,
result_items,
result_items_truncated,
progress,
next_seq,
min_seq,
}) => (
"stopped".to_string(),
Some("heal task timed out".to_string()),
result_items,
result_items_truncated,
progress,
next_seq,
min_seq,
),
Ok(HealTaskReport {
status: HealTaskStatus::Failed { error },
result_items,
result_items_truncated,
progress,
}) => ("stopped".to_string(), Some(error), result_items, result_items_truncated, progress),
next_seq,
min_seq,
}) => (
"stopped".to_string(),
Some(error),
result_items,
result_items_truncated,
progress,
next_seq,
min_seq,
),
Err(crate::Error::TaskNotFound { .. }) => (
"notFound".to_string(),
Some("heal task not found or expired".to_string()),
Vec::new(),
false,
None,
0,
0,
),
Err(crate::Error::InvalidClientToken) => {
let response = HealChannelResponse {
@@ -490,7 +570,8 @@ impl HealChannelProcessor {
}
};
let (data, detail) = encode_heal_status_response(&summary, items, progress.as_ref(), detail, truncated)?;
let (data, detail) =
encode_heal_status_response(&summary, items, progress.as_ref(), detail, truncated, next_seq, min_seq)?;
let response = HealChannelResponse {
request_id: client_token,
@@ -612,7 +693,8 @@ impl HealChannelProcessor {
HealRequestSource::Admin
| HealRequestSource::AutoHeal
| HealRequestSource::Internal
| HealRequestSource::ReadRepair => true,
| HealRequestSource::ReadRepair
| HealRequestSource::Mrf => true,
});
// Build HealOptions with all available fields
@@ -804,7 +886,7 @@ mod tests {
..Default::default()
}];
let (data, detail) = encode_heal_status_response("running", items, None, None, false).unwrap();
let (data, detail) = encode_heal_status_response("running", items, None, None, false, 0, 0).unwrap();
assert!(data.len() <= MAX_HEAL_STATUS_PAYLOAD_SIZE);
let payload: serde_json::Value = serde_json::from_slice(&data).unwrap();
@@ -1574,7 +1656,7 @@ mod tests {
let (tx, rx) = oneshot::channel();
processor
.process_query_request("bucket".to_string(), "completed-token".to_string(), tx)
.process_query_request("bucket".to_string(), "completed-token".to_string(), None, tx)
.await
.expect("query should process");
@@ -1609,7 +1691,7 @@ mod tests {
let (tx, rx) = oneshot::channel();
processor
.process_query_request("bucket".to_string(), task_id.clone(), tx)
.process_query_request("bucket".to_string(), task_id.clone(), None, tx)
.await
.expect("query should process");
@@ -1642,7 +1724,7 @@ mod tests {
let (tx, rx) = oneshot::channel();
processor
.process_query_request("bucket".to_string(), "wrong-token".to_string(), tx)
.process_query_request("bucket".to_string(), "wrong-token".to_string(), None, tx)
.await
.expect("query should process");
@@ -1667,7 +1749,7 @@ mod tests {
let (tx, rx) = oneshot::channel();
processor
.process_query_request(String::new(), "wrong-token".to_string(), tx)
.process_query_request(String::new(), "wrong-token".to_string(), None, tx)
.await
.expect("query should process");
@@ -1704,7 +1786,7 @@ mod tests {
let (tx, rx) = oneshot::channel();
processor
.process_query_request(String::new(), task_id.clone(), tx)
.process_query_request(String::new(), task_id.clone(), None, tx)
.await
.expect("query should process");
+438 -62
View File
@@ -220,6 +220,11 @@ struct CompletedHealStatus {
result_items: Vec<HealResultItem>,
result_items_truncated: bool,
completed_at: SystemTime,
/// Sequence-stamped retained window, archived with the completion so
/// incremental consumers keep their cursor across the transition (HS-06).
seqed_items: Vec<(u64, HealResultItem)>,
next_seq: u64,
min_seq: u64,
}
#[derive(Debug, Clone)]
@@ -240,6 +245,65 @@ pub struct HealTaskReport {
pub result_items: Vec<HealResultItem>,
pub result_items_truncated: bool,
pub progress: Option<HealProgress>,
/// Cursor for incremental consumption: sequence number of the next item
/// to be produced. `0` on reports from sources without sequencing.
pub next_seq: u64,
/// Oldest sequence still retained (`0` together with `next_seq` when
/// sequencing is unavailable).
pub min_seq: u64,
}
/// Report from a live task, honoring the client's incremental cursor.
async fn active_task_report(task: &HealTask, since: Option<u64>) -> HealTaskReport {
let window = task.get_result_items_since(since).await;
HealTaskReport {
status: task.get_status().await,
result_items: window.items,
// The legacy flag stays set once anything was evicted; a lagging
// incremental cursor additionally marks this response truncated so
// the client knows to restart from `min_seq`.
result_items_truncated: task.result_items_truncated() || window.lagged,
progress: Some(task.get_progress().await),
next_seq: window.next_seq,
min_seq: window.min_seq,
}
}
fn empty_task_report(status: HealTaskStatus) -> HealTaskReport {
HealTaskReport {
status,
result_items: Vec::new(),
result_items_truncated: false,
progress: None,
next_seq: 0,
min_seq: 0,
}
}
fn completed_task_report(completed: &CompletedHealStatus, since: Option<u64>) -> HealTaskReport {
let mut lagged = false;
let result_items = match since {
None => completed.result_items.clone(),
Some(cursor) => {
if cursor + 1 < completed.min_seq {
lagged = true;
}
completed
.seqed_items
.iter()
.filter(|(seq, _)| *seq > cursor)
.map(|(_, item)| item.clone())
.collect()
}
};
HealTaskReport {
status: completed.status.clone(),
result_items,
result_items_truncated: completed.result_items_truncated || lagged,
progress: None,
next_seq: completed.next_seq,
min_seq: completed.min_seq,
}
}
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq, serde::Deserialize, serde::Serialize)]
@@ -270,6 +334,8 @@ pub struct HealSourceCounts {
pub auto_heal: u64,
pub internal: u64,
pub read_repair: u64,
#[serde(default)]
pub mrf: u64,
}
impl HealSourceCounts {
@@ -280,6 +346,7 @@ impl HealSourceCounts {
HealRequestSource::AutoHeal => self.auto_heal += 1,
HealRequestSource::Internal => self.internal += 1,
HealRequestSource::ReadRepair => self.read_repair += 1,
HealRequestSource::Mrf => self.mrf += 1,
}
}
}
@@ -528,6 +595,11 @@ impl PriorityHealQueue {
self.dedup_keys.contains_key(&key)
}
/// Iterate queued requests (used by the admin overlap check).
fn requests(&self) -> impl Iterator<Item = &HealRequest> {
self.heap.iter().map(|item| &item.request)
}
fn contains_request_id(&self, request_id: &str) -> bool {
self.heap.iter().any(|item| item.request.id == request_id)
}
@@ -686,6 +758,80 @@ fn recoverable_heal_retry_delay(retry_attempt: u32) -> Duration {
}
/// Heal config
/// HS-06 admin overlap policy.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
pub enum HealOverlapPolicy {
/// Default: overlapping admin starts merge into the existing task
/// (today's dedup semantics).
#[default]
Merge,
/// Return a typed already-running / overlapping-paths rejection like
/// madmin's ErrHealAlreadyRunning / ErrHealOverlappingPaths.
MinioError,
}
/// Path view of a heal type for overlap comparison: a bucket plus a
/// prefix/object path inside it (`None` bucket = cluster-wide, overlaps
/// everything).
fn heal_type_path_view(heal_type: &HealType) -> (Option<&str>, &str) {
match heal_type {
HealType::Cluster => (None, ""),
HealType::Bucket { bucket } => (Some(bucket), ""),
HealType::Prefix { bucket, prefix } => (Some(bucket), prefix),
HealType::Object { bucket, object, .. }
| HealType::Metadata { bucket, object }
| HealType::ECDecode { bucket, object, .. } => (Some(bucket), object),
// MRF/MetaPath heal keys on a meta path; treat the whole set of
// buckets as one namespace so it only overlaps itself exactly.
HealType::MRF { meta_path } => (Some("\u{0}mrf"), meta_path),
// Erasure-set heal: the set id is the overlap dimension.
HealType::ErasureSet { set_disk_id, .. } => (Some("\u{0}set"), set_disk_id),
}
}
/// How two heal paths relate for the admin overlap check (HS-06).
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
enum OverlapVerdict {
/// Distinct targets: no conflict.
Disjoint,
/// Same target: an identical heal is already in flight.
SameTarget,
/// One target contains the other.
Overlapping,
}
fn prefix_paths_overlap(a: &str, b: &str) -> OverlapVerdict {
if a == b {
return OverlapVerdict::SameTarget;
}
if a.is_empty() || b.is_empty() || a.starts_with(b) || b.starts_with(a) {
return OverlapVerdict::Overlapping;
}
OverlapVerdict::Disjoint
}
fn heal_types_overlap(left: &HealType, right: &HealType) -> OverlapVerdict {
let (left_bucket, left_path) = heal_type_path_view(left);
let (right_bucket, right_path) = heal_type_path_view(right);
match (left_bucket, right_bucket) {
// Cluster-wide overlaps everything (but an exact cluster match is
// SameTarget).
(None, _) | (_, None) => {
if matches!(left, HealType::Cluster) && matches!(right, HealType::Cluster) {
OverlapVerdict::SameTarget
} else {
OverlapVerdict::Overlapping
}
}
(Some(lb), Some(rb)) => {
if lb != rb {
return OverlapVerdict::Disjoint;
}
prefix_paths_overlap(left_path, right_path)
}
}
}
#[derive(Debug, Clone)]
pub struct HealConfig {
/// Whether to enable auto heal
@@ -706,6 +852,9 @@ pub struct HealConfig {
pub low_priority_drop_when_full: bool,
/// Whether notify-driven scheduler wakeups are enabled.
pub event_driven_scheduler_enable: bool,
/// How admin heal starts behave on path overlap (HS-06): merge into the
/// existing task (default) or return a typed already-running rejection.
pub overlap_policy: HealOverlapPolicy,
/// Whether per-set bulkhead scheduling is enabled.
pub set_bulkhead_enable: bool,
/// Whether erasure-set page parallelism is enabled.
@@ -754,6 +903,14 @@ impl Default for HealConfig {
rustfs_config::ENV_HEAL_EVENT_DRIVEN_SCHEDULER_ENABLE,
rustfs_config::DEFAULT_HEAL_EVENT_DRIVEN_SCHEDULER_ENABLE,
);
let overlap_policy =
match rustfs_utils::get_env_str(rustfs_config::ENV_HEAL_OVERLAP_POLICY, rustfs_config::DEFAULT_HEAL_OVERLAP_POLICY)
.to_lowercase()
.as_str()
{
"minio_error" => HealOverlapPolicy::MinioError,
_ => HealOverlapPolicy::Merge,
};
let set_bulkhead_enable = rustfs_utils::get_env_bool(
rustfs_config::ENV_HEAL_SET_BULKHEAD_ENABLE,
rustfs_config::DEFAULT_HEAL_SET_BULKHEAD_ENABLE,
@@ -790,6 +947,7 @@ impl Default for HealConfig {
low_priority_merge_enable,
low_priority_drop_when_full,
event_driven_scheduler_enable,
overlap_policy,
set_bulkhead_enable,
page_parallel_enable,
mainline_throttle_enable,
@@ -1756,6 +1914,50 @@ impl HealManager {
request: HealRequest,
preserve_alias: bool,
) -> Result<HealAdmissionReceipt> {
// HS-06 forceStart semantics (admin only): MinIO stops the old task
// first and then starts the new one. Cancel any active admin task
// overlapping this request's path before entering admission, so the
// fresh task is never merged into the one being replaced.
if request.source == HealRequestSource::Admin && request.force_start {
let overlapping: Vec<String> = {
let active_heals = self.active_heals.lock().await;
active_heals
.iter()
.filter(|(task_id, task)| {
task.source == HealRequestSource::Admin
&& heal_types_overlap(&request.heal_type, &task.heal_type) != OverlapVerdict::Disjoint
&& *task_id != &request.id
})
.map(|(task_id, _)| task_id.clone())
.collect()
};
for task_id in overlapping {
match self.cancel_task(&task_id).await {
Ok(_) => info!(
target: "rustfs::heal::manager",
event = EVENT_HEAL_QUEUE_ADMISSION,
component = LOG_COMPONENT_HEAL,
subsystem = LOG_SUBSYSTEM_MANAGER,
request_id = %request.id,
cancelled_task_id = %task_id,
result = "force_start_cancelled_overlap",
"Admin forceStart cancelled an overlapping heal task"
),
Err(err) => warn!(
target: "rustfs::heal::manager",
event = EVENT_HEAL_QUEUE_ADMISSION,
component = LOG_COMPONENT_HEAL,
subsystem = LOG_SUBSYSTEM_MANAGER,
request_id = %request.id,
cancelled_task_id = %task_id,
error = %err,
result = "force_start_cancel_failed",
"Admin forceStart failed to cancel an overlapping heal task"
),
}
}
}
let config = self.config.read().await;
let dedup_key = PriorityHealQueue::make_dedup_key(&request);
@@ -1778,7 +1980,15 @@ impl HealManager {
.or_else(|| retrying_heal_for_dedup_key(&retrying_heals, &dedup_key).map(|(task_id, _)| (task_id, "retrying")))
});
if let Some((merged_task_id, duplicate_state)) = duplicate.flatten() {
let admission = Self::duplicate_admission_for_request(&request, &config);
// HS-06: under the minio_error overlap policy an exact duplicate
// admin start reports the typed AlreadyRunning rejection instead
// of the silent merge (MinIO's ErrHealAlreadyRunning).
let admission =
if request.source == HealRequestSource::Admin && config.overlap_policy == HealOverlapPolicy::MinioError {
HealAdmissionResult::Dropped(HealAdmissionDropReason::AlreadyRunning)
} else {
Self::duplicate_admission_for_request(&request, &config)
};
drop(retrying_heals);
drop(queue);
drop(active_heals);
@@ -1824,6 +2034,62 @@ impl HealManager {
});
}
// HS-06 typed overlap rejection (admin only, minio_error policy):
// paths containing or contained by an active/queued task reject with
// AlreadyRunning / OverlappingPaths instead of merging. Exact
// duplicates already merged above; scanner/autoheal/read-repair
// sources never take this path.
if request.source == HealRequestSource::Admin && config.overlap_policy == HealOverlapPolicy::MinioError {
let mut rejection = None;
for (task_id, task) in active_heals.iter() {
match heal_types_overlap(&request.heal_type, &task.heal_type) {
OverlapVerdict::SameTarget => {
rejection = Some((HealAdmissionDropReason::AlreadyRunning, task_id.clone()));
break;
}
OverlapVerdict::Overlapping => {
rejection = Some((HealAdmissionDropReason::OverlappingPaths, task_id.clone()));
}
OverlapVerdict::Disjoint => {}
}
}
if rejection.is_none() {
for queued in queue.requests() {
match heal_types_overlap(&request.heal_type, &queued.heal_type) {
OverlapVerdict::SameTarget => {
rejection = Some((HealAdmissionDropReason::AlreadyRunning, queued.id.clone()));
break;
}
OverlapVerdict::Overlapping => {
rejection = Some((HealAdmissionDropReason::OverlappingPaths, queued.id.clone()));
}
OverlapVerdict::Disjoint => {}
}
}
}
if let Some((reason, overlap_task_id)) = rejection {
drop(retrying_heals);
drop(queue);
drop(active_heals);
Self::record_admission_metric(request.source, HealAdmissionResult::Dropped(reason), "overlap_rejected");
warn!(
target: "rustfs::heal::manager",
event = EVENT_HEAL_QUEUE_ADMISSION,
component = LOG_COMPONENT_HEAL,
subsystem = LOG_SUBSYSTEM_MANAGER,
request_id = %request.id,
overlap_task_id = %overlap_task_id,
reason = reason.as_str(),
result = "overlap_rejected",
"Admin heal start rejected by overlap policy"
);
return Ok(HealAdmissionReceipt {
result: HealAdmissionResult::Dropped(reason),
task_id: overlap_task_id,
});
}
}
let mut task_id = request.id.clone();
let admission = Self::admit_request_to_queue(&mut queue, request, &config, "submit");
if admission == HealAdmissionResult::Merged
@@ -1896,28 +2162,25 @@ impl HealManager {
}
pub async fn get_task_report(&self, task_id: &str) -> Result<HealTaskReport> {
self.get_task_report_since(task_id, None).await
}
/// Incremental variant of [`Self::get_task_report`] (HS-06): `since` is
/// the client's last seen sequence number; `None` keeps the legacy
/// full-snapshot semantics.
pub async fn get_task_report_since(&self, task_id: &str, since: Option<u64>) -> Result<HealTaskReport> {
let canonical_task_id = self.canonical_task_id(task_id).await;
{
let active_heals = self.active_heals.lock().await;
if let Some(task) = active_heals.get(&canonical_task_id) {
return Ok(HealTaskReport {
status: task.get_status().await,
result_items: task.get_result_items().await,
result_items_truncated: task.result_items_truncated(),
progress: Some(task.get_progress().await),
});
return Ok(active_task_report(task, since).await);
}
}
{
let retrying_heals = self.retrying_heals.lock().await;
if let Some(retrying) = retrying_heals.get(&canonical_task_id) {
return Ok(HealTaskReport {
status: retrying.status(),
result_items: Vec::new(),
result_items_truncated: false,
progress: None,
});
return Ok(empty_task_report(retrying.status()));
}
}
@@ -1927,36 +2190,21 @@ impl HealManager {
if let Some(completed) = completed_heals.get(&canonical_task_id)
&& completed_status_is_retrying(&completed.status)
{
return Ok(HealTaskReport {
status: completed.status.clone(),
result_items: completed.result_items.clone(),
result_items_truncated: completed.result_items_truncated,
progress: None,
});
return Ok(completed_task_report(completed, since));
}
}
{
let queue = self.heal_queue.lock().await;
if queue.contains_request_id(&canonical_task_id) {
return Ok(HealTaskReport {
status: HealTaskStatus::Pending,
result_items: Vec::new(),
result_items_truncated: false,
progress: None,
});
return Ok(empty_task_report(HealTaskStatus::Pending));
}
}
let mut completed_heals = self.completed_heals.lock().await;
prune_completed_heal_statuses(&mut completed_heals);
if let Some(completed) = completed_heals.get(&canonical_task_id) {
return Ok(HealTaskReport {
status: completed.status.clone(),
result_items: completed.result_items.clone(),
result_items_truncated: completed.result_items_truncated,
progress: None,
});
return Ok(completed_task_report(completed, since));
}
Err(Error::TaskNotFound {
@@ -1965,18 +2213,23 @@ impl HealManager {
}
pub async fn get_task_report_for_path(&self, heal_path: &str, task_id: &str) -> Result<HealTaskReport> {
self.get_task_report_for_path_since(heal_path, task_id, None).await
}
/// Incremental variant of [`Self::get_task_report_for_path`] (HS-06).
pub async fn get_task_report_for_path_since(
&self,
heal_path: &str,
task_id: &str,
since: Option<u64>,
) -> Result<HealTaskReport> {
let canonical_task_id = self.canonical_task_id(task_id).await;
{
let active_heals = self.active_heals.lock().await;
if let Some(task) = active_heals.get(&canonical_task_id)
&& heal_type_matches_path(&task.heal_type, heal_path)
{
return Ok(HealTaskReport {
status: task.get_status().await,
result_items: task.get_result_items().await,
result_items_truncated: task.result_items_truncated(),
progress: Some(task.get_progress().await),
});
return Ok(active_task_report(task, since).await);
}
}
@@ -1985,12 +2238,7 @@ impl HealManager {
if let Some(retrying) = retrying_heals.get(&canonical_task_id)
&& heal_type_matches_path(&retrying.request.heal_type, heal_path)
{
return Ok(HealTaskReport {
status: retrying.status(),
result_items: Vec::new(),
result_items_truncated: false,
progress: None,
});
return Ok(empty_task_report(retrying.status()));
}
}
@@ -2001,24 +2249,14 @@ impl HealManager {
&& heal_type_matches_path(&completed.heal_type, heal_path)
&& completed_status_is_retrying(&completed.status)
{
return Ok(HealTaskReport {
status: completed.status.clone(),
result_items: completed.result_items.clone(),
result_items_truncated: completed.result_items_truncated,
progress: None,
});
return Ok(completed_task_report(completed, since));
}
}
{
let queue = self.heal_queue.lock().await;
if queue.contains_request_id_matching_path(&canonical_task_id, heal_path) {
return Ok(HealTaskReport {
status: HealTaskStatus::Pending,
result_items: Vec::new(),
result_items_truncated: false,
progress: None,
});
return Ok(empty_task_report(HealTaskStatus::Pending));
}
}
@@ -2028,12 +2266,7 @@ impl HealManager {
if let Some(completed) = completed_heals.get(&canonical_task_id)
&& heal_type_matches_path(&completed.heal_type, heal_path)
{
return Ok(HealTaskReport {
status: completed.status.clone(),
result_items: completed.result_items.clone(),
result_items_truncated: completed.result_items_truncated,
progress: None,
});
return Ok(completed_task_report(completed, since));
}
}
@@ -3228,12 +3461,16 @@ impl HealManager {
completed_task.get_status().await
};
let completed_progress = completed_task.get_progress().await;
let final_window = completed_task.get_result_items_since(None).await;
let completed_status_entry = CompletedHealStatus {
heal_type: completed_task.heal_type.clone(),
status: completed_status.clone(),
result_items: completed_task.get_result_items().await,
result_items: final_window.items.clone(),
result_items_truncated: completed_task.result_items_truncated(),
completed_at: SystemTime::now(),
seqed_items: completed_task.get_seqed_result_items().await,
next_seq: final_window.next_seq,
min_seq: final_window.min_seq,
};
let mut completed_heals_guard = completed_heals_clone.lock().await;
prune_completed_heal_statuses(&mut completed_heals_guard);
@@ -5005,6 +5242,9 @@ mod tests {
},
result_items: Vec::new(),
result_items_truncated: false,
seqed_items: Vec::new(),
next_seq: 0,
min_seq: 0,
completed_at: SystemTime::now(),
},
);
@@ -5286,6 +5526,136 @@ mod tests {
assert_eq!(snapshot.queued_by_source.internal, 0);
}
// HS-06 (backlog#1870): overlap policy + forceStart semantics.
fn manager_with_policy(policy: HealOverlapPolicy) -> HealManager {
let storage: Arc<dyn HealStorageAPI> = Arc::new(MockStorage);
HealManager::new(
storage,
Some(HealConfig {
overlap_policy: policy,
..Default::default()
}),
)
}
fn admin_prefix_request(bucket: &str, prefix: &str) -> HealRequest {
let mut request = HealRequest::new(
HealType::Prefix {
bucket: bucket.to_string(),
prefix: prefix.to_string(),
},
HealOptions::default(),
HealPriority::Normal,
);
request.source = HealRequestSource::Admin;
request
}
async fn insert_active_task(manager: &HealManager, request: HealRequest) -> String {
let task = Arc::new(HealTask::from_request(request, manager.storage.clone()));
let task_id = task.id.clone();
manager.active_heals.lock().await.insert(task_id.clone(), task);
task_id
}
#[tokio::test]
async fn overlap_policy_minio_error_rejects_same_and_containing_paths() {
let manager = manager_with_policy(HealOverlapPolicy::MinioError);
insert_active_task(&manager, admin_prefix_request("bucket-a", "logs/")).await;
// Same target: typed AlreadyRunning.
let same = manager
.submit_heal_request(admin_prefix_request("bucket-a", "logs/"))
.await
.expect("admission must decide");
assert_eq!(
same,
HealAdmissionResult::Dropped(HealAdmissionDropReason::AlreadyRunning),
"an identical target must reject with already-running"
);
// Contained path: typed OverlappingPaths.
let nested = manager
.submit_heal_request(admin_prefix_request("bucket-a", "logs/app/"))
.await
.expect("admission must decide");
assert_eq!(
nested,
HealAdmissionResult::Dropped(HealAdmissionDropReason::OverlappingPaths),
"a path inside the active task's path must reject with overlapping-paths"
);
// Containing path (bucket-wide vs nested active): also overlapping.
let wide = manager
.submit_heal_request(admin_prefix_request("bucket-a", ""))
.await
.expect("admission must decide");
assert_eq!(
wide,
HealAdmissionResult::Dropped(HealAdmissionDropReason::OverlappingPaths),
"a bucket-wide start overlapping a nested active heal must reject"
);
// Disjoint bucket: unaffected.
let disjoint = manager
.submit_heal_request(admin_prefix_request("bucket-b", "logs/"))
.await
.expect("admission must decide");
assert_eq!(disjoint, HealAdmissionResult::Accepted);
}
#[tokio::test]
async fn overlap_policy_default_merge_keeps_today_semantics() {
let manager = manager_with_policy(HealOverlapPolicy::Merge);
insert_active_task(&manager, admin_prefix_request("bucket-a", "logs/")).await;
// Different-dedup-key overlap still merges under the default policy:
// the nested path dedups to its own key but nothing rejects it.
let nested = manager
.submit_heal_request(admin_prefix_request("bucket-a", "logs/app/"))
.await
.expect("admission must decide");
assert_eq!(nested, HealAdmissionResult::Accepted, "default policy must not reject overlaps");
// Non-admin sources never get overlap rejections even under minio_error.
let manager = manager_with_policy(HealOverlapPolicy::MinioError);
insert_active_task(&manager, admin_prefix_request("bucket-a", "logs/")).await;
let mut scanner_request = admin_prefix_request("bucket-a", "logs/app/");
scanner_request.source = HealRequestSource::Scanner;
let admitted = manager
.submit_heal_request(scanner_request)
.await
.expect("admission must decide");
assert_eq!(admitted, HealAdmissionResult::Accepted, "scanner sources must never be overlap-rejected");
}
#[tokio::test]
async fn admin_force_start_cancels_overlapping_active_task_first() {
let manager = manager_with_policy(HealOverlapPolicy::Merge);
let old_id = insert_active_task(&manager, admin_prefix_request("bucket-a", "logs/")).await;
let mut replacement = admin_prefix_request("bucket-a", "logs/");
replacement.force_start = true;
let receipt = manager
.submit_heal_request_with_receipt(replacement)
.await
.expect("force-start submission must decide");
assert!(receipt.result.is_admitted(), "the new task must be admitted (Accepted or Merged)");
let old_task_gone = {
let active_heals = manager.active_heals.lock().await;
!active_heals.contains_key(&old_id)
};
assert!(
old_task_gone,
"the overlapping admin task must be cancelled (removed from the active table) before the new one starts"
);
assert!(
matches!(manager.get_task_status(&old_id).await, Err(Error::TaskNotFound { .. })),
"a cancelled task must no longer resolve as an active heal"
);
}
#[tokio::test]
async fn test_operations_snapshot_counts_active_by_source_and_priority() {
let storage: Arc<dyn HealStorageAPI> = Arc::new(MockStorage);
@@ -5588,6 +5958,9 @@ mod tests {
status: HealTaskStatus::Completed,
result_items: Vec::new(),
result_items_truncated: false,
seqed_items: Vec::new(),
next_seq: 0,
min_seq: 0,
completed_at: SystemTime::now(),
},
);
@@ -5622,6 +5995,9 @@ mod tests {
..Default::default()
}],
result_items_truncated: true,
seqed_items: Vec::new(),
next_seq: 0,
min_seq: 0,
completed_at: SystemTime::now(),
},
);
+1
View File
@@ -16,6 +16,7 @@ pub mod channel;
pub mod erasure_healer;
pub mod event;
pub mod manager;
pub mod mrf_queue;
pub mod progress;
pub(crate) mod replacement_readiness;
pub mod resume;
+682
View File
@@ -0,0 +1,682 @@
// Copyright 2024 RustFS Team
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//! Mission Repair Feed (MRF) queue, journal, and consumer.
//!
//! Intents arriving on the global channel (see `rustfs_common::mrf_channel`)
//! are buffered in a bounded in-memory queue, translated into prioritized
//! heal requests, and — while they are not yet accepted by the heal manager —
//! mirrored into a durable journal so a crash or restart can replay them.
//! This is the RustFS counterpart of MinIO's `.heal/mrf/list.bin` replay,
//! layered on top of (not replacing) read-repair and scanner heal.
//!
//! Durability model: the journal is a snapshot of the *unaccepted* pending
//! set, rewritten on a group-commit cadence (every flush interval or flush
//! threshold new intents). A rewrite is atomic at the record level only — a
//! torn tail simply truncates during replay because every record carries its
//! own CRC32. Losing the last flush window (≤500 ms) is acceptable: replayed
//! duplicates are merged by the manager's dedup key, and read-repair remains
//! the safety net.
use super::{DiskStore, HealDiskExt as _, local_disk_map_read};
use crate::heal::manager::HealManager;
use metrics::{counter, gauge};
use rustfs_common::heal_channel::{HealAdmissionDropReason, HealAdmissionResult};
use rustfs_common::mrf_channel::{MRF_MAX_ATTEMPTS, MrfIntent};
use std::collections::VecDeque;
use std::sync::Arc;
use std::time::Duration;
use tokio::sync::mpsc;
use uuid::Uuid;
use crate::heal::task::{HealOptions, HealPriority, HealRequest, HealType};
/// Journal location inside the metadata bucket, following the resume-state
/// layout.
pub(crate) const MRF_JOURNAL_PATH: &str = "buckets/.heal/mrf/journal.bin";
/// Record format tag.
const MRF_JOURNAL_FORMAT: u8 = 1;
/// Record layout version.
const MRF_JOURNAL_VERSION: u8 = 1;
/// Fixed header size: format, version, kind, attempts, enqueued_at_ms,
/// has_version flag.
const MRF_RECORD_FIXED_HEAD: usize = 1 + 1 + 1 + 1 + 8 + 1;
#[derive(Debug, Clone)]
pub(crate) struct MrfConsumerConfig {
/// In-memory queue capacity in intents.
pub queue_capacity: usize,
/// Journal byte budget; a pending snapshot above this bound is rejected
/// oldest-first so the journal can never grow unbounded.
pub journal_max_bytes: usize,
/// How many journal intents to re-arm per replay round.
pub replay_batch: usize,
/// Group-commit cadence for the journal snapshot.
pub flush_interval: Duration,
/// New intents between flushes that force an early snapshot.
pub flush_threshold: usize,
/// Backoff after the heal manager reports a full admission.
pub admission_backoff: Duration,
}
impl Default for MrfConsumerConfig {
fn default() -> Self {
Self {
queue_capacity: rustfs_utils::get_env_usize(
rustfs_config::ENV_HEAL_MRF_QUEUE_SIZE,
rustfs_config::DEFAULT_HEAL_MRF_QUEUE_SIZE,
),
journal_max_bytes: rustfs_utils::get_env_usize(
rustfs_config::ENV_HEAL_MRF_JOURNAL_MAX_BYTES,
rustfs_config::DEFAULT_HEAL_MRF_JOURNAL_MAX_BYTES,
),
replay_batch: rustfs_utils::get_env_usize(
rustfs_config::ENV_HEAL_MRF_REPLAY_BATCH,
rustfs_config::DEFAULT_HEAL_MRF_REPLAY_BATCH,
),
flush_interval: Duration::from_millis(500),
flush_threshold: 1000,
admission_backoff: Duration::from_secs(5),
}
}
}
/// Bounded pending set with count and byte ceilings. Overflow drops the
/// incoming intent (never a resident one) and counts the loss.
pub(crate) struct MrfQueue {
pending: VecDeque<MrfIntent>,
bytes: usize,
capacity: usize,
byte_budget: usize,
}
impl MrfQueue {
pub(crate) fn new(capacity: usize, byte_budget: usize) -> Self {
Self {
pending: VecDeque::new(),
bytes: 0,
capacity,
byte_budget,
}
}
/// Returns `false` (after counting) when either ceiling would be crossed.
pub(crate) fn try_push(&mut self, intent: MrfIntent) -> bool {
let cost = intent.estimated_bytes();
if self.pending.len() >= self.capacity || self.bytes + cost > self.byte_budget {
counter!("rustfs_heal_mrf_dropped_total", "reason" => "queue_overflow").increment(1);
return false;
}
self.bytes += cost;
self.pending.push_back(intent);
true
}
pub(crate) fn pop_front(&mut self) -> Option<MrfIntent> {
let intent = self.pending.pop_front()?;
self.bytes = self.bytes.saturating_sub(intent.estimated_bytes());
Some(intent)
}
pub(crate) fn push_back(&mut self, intent: MrfIntent) {
self.bytes += intent.estimated_bytes();
self.pending.push_back(intent);
}
pub(crate) fn depth(&self) -> usize {
self.pending.len()
}
pub(crate) fn bytes(&self) -> usize {
self.bytes
}
pub(crate) fn intents(&self) -> impl Iterator<Item = &MrfIntent> {
self.pending.iter()
}
}
// ---------------------------------------------------------------------------
// Journal record codec
// ---------------------------------------------------------------------------
/// Append one encoded record to `out`.
pub(crate) fn encode_intent(intent: &MrfIntent, out: &mut Vec<u8>) {
let start = out.len();
out.push(MRF_JOURNAL_FORMAT);
out.push(MRF_JOURNAL_VERSION);
out.push(match intent.kind {
rustfs_common::mrf_channel::MrfKind::DecodeFailure => 1,
rustfs_common::mrf_channel::MrfKind::MetadataCorruption => 2,
rustfs_common::mrf_channel::MrfKind::PartialWrite => 3,
});
out.push(intent.attempts);
out.extend_from_slice(&intent.enqueued_at_ms.to_le_bytes());
match intent.version_id {
Some(bytes) => {
out.push(1);
out.extend_from_slice(&bytes);
}
None => out.push(0),
}
out.extend_from_slice(&(intent.bucket.len() as u32).to_le_bytes());
out.extend_from_slice(&(intent.object.len() as u32).to_le_bytes());
out.extend_from_slice(intent.bucket.as_bytes());
out.extend_from_slice(intent.object.as_bytes());
let mut hasher = crc_fast::Digest::new(crc_fast::CrcAlgorithm::Crc32IsoHdlc);
hasher.update(&out[start..]);
out.extend_from_slice(&(hasher.finalize() as u32).to_le_bytes());
}
fn decode_one(data: &[u8]) -> Option<(MrfIntent, usize)> {
if data.len() < MRF_RECORD_FIXED_HEAD + 8 {
return None;
}
if data[0] != MRF_JOURNAL_FORMAT || data[1] != MRF_JOURNAL_VERSION {
return None;
}
let kind = match data[2] {
1 => rustfs_common::mrf_channel::MrfKind::DecodeFailure,
2 => rustfs_common::mrf_channel::MrfKind::MetadataCorruption,
3 => rustfs_common::mrf_channel::MrfKind::PartialWrite,
_ => return None,
};
let attempts = data[3];
let enqueued_at_ms = u64::from_le_bytes(data[4..12].try_into().expect("slice length checked"));
let has_version = data[12] != 0;
let mut cursor = MRF_RECORD_FIXED_HEAD;
let version_id = if has_version {
if data.len() < cursor + 16 {
return None;
}
let bytes: [u8; 16] = data[cursor..cursor + 16].try_into().expect("slice length checked");
cursor += 16;
Some(bytes)
} else {
None
};
if data.len() < cursor + 8 {
return None;
}
let bucket_len = u32::from_le_bytes(data[cursor..cursor + 4].try_into().expect("slice length checked")) as usize;
let object_len = u32::from_le_bytes(data[cursor + 4..cursor + 8].try_into().expect("slice length checked")) as usize;
cursor += 8;
let body_end = cursor.checked_add(bucket_len)?.checked_add(object_len)?;
let record_end = body_end.checked_add(4)?;
if data.len() < record_end {
return None;
}
let mut hasher = crc_fast::Digest::new(crc_fast::CrcAlgorithm::Crc32IsoHdlc);
hasher.update(&data[..body_end]);
if (hasher.finalize() as u32) != u32::from_le_bytes(data[body_end..record_end].try_into().expect("slice length checked")) {
return None;
}
let bucket = std::sync::Arc::from(std::str::from_utf8(&data[cursor..cursor + bucket_len]).ok()?);
let object = std::sync::Arc::from(std::str::from_utf8(&data[cursor + bucket_len..body_end]).ok()?);
Some((
MrfIntent {
bucket,
object,
version_id,
kind,
enqueued_at_ms,
attempts,
},
record_end,
))
}
/// Decode a whole journal, stopping at the first torn or corrupt record.
/// Returns the decoded intents and the number of trailing bytes discarded.
pub(crate) fn decode_journal(data: &[u8]) -> (Vec<MrfIntent>, usize) {
let mut intents = Vec::new();
let mut cursor = 0usize;
while cursor < data.len() {
match decode_one(&data[cursor..]) {
Some((intent, consumed)) => {
intents.push(intent);
cursor += consumed;
}
None => break,
}
}
let truncated = data.len() - cursor;
(intents, truncated)
}
// ---------------------------------------------------------------------------
// Journal disk IO (all local disks, first successful read wins)
// ---------------------------------------------------------------------------
async fn journal_disks() -> Vec<DiskStore> {
let map = local_disk_map_read().await;
map.values().flatten().cloned().collect()
}
async fn read_journal() -> Option<Vec<u8>> {
for disk in journal_disks().await {
match disk.read_all(super::RUSTFS_META_BUCKET, MRF_JOURNAL_PATH).await {
Ok(bytes) => return Some(bytes.to_vec()),
Err(_) => continue,
}
}
None
}
async fn write_journal(data: &[u8]) {
let payload = bytes::Bytes::copy_from_slice(data);
for disk in journal_disks().await {
if let Err(err) = disk
.write_all(super::RUSTFS_META_BUCKET, MRF_JOURNAL_PATH, payload.clone())
.await
{
warn_mrf_journal_write(&err);
}
}
if !data.is_empty() {
counter!("rustfs_heal_mrf_journal_fsync_total").increment(1);
}
gauge!("rustfs_heal_mrf_journal_bytes").set(data.len() as f64);
}
async fn delete_journal() {
for disk in journal_disks().await {
let _ = disk
.delete(
super::RUSTFS_META_BUCKET,
MRF_JOURNAL_PATH,
crate::heal::storage_api::owner::EcstoreDeleteOptions::default(),
)
.await;
}
}
fn warn_mrf_journal_write(err: &super::DiskError) {
tracing::warn!(
target: "rustfs::heal::mrf",
error = %err,
"MRF journal write failed; unconsumed intents may be lost on restart"
);
}
// ---------------------------------------------------------------------------
// Consumer
// ---------------------------------------------------------------------------
/// Translate an intent into the prioritized heal request the issue specifies:
/// decode failures go Urgent ECDecode, metadata corruption goes High
/// Metadata, partial writes go Normal object heal.
pub(crate) fn build_heal_request(intent: &MrfIntent) -> HealRequest {
let bucket = intent.bucket.to_string();
let object = intent.object.to_string();
let version_id = intent.version_id.map(|bytes| Uuid::from_bytes(bytes).to_string());
let (heal_type, priority) = match intent.kind {
rustfs_common::mrf_channel::MrfKind::DecodeFailure => (
HealType::ECDecode {
bucket,
object,
version_id,
},
HealPriority::Urgent,
),
rustfs_common::mrf_channel::MrfKind::MetadataCorruption => (HealType::Metadata { bucket, object }, HealPriority::High),
rustfs_common::mrf_channel::MrfKind::PartialWrite => (
HealType::Object {
bucket,
object,
version_id,
},
HealPriority::Normal,
),
};
let mut request = HealRequest::new(heal_type, HealOptions::default(), priority);
request.source = rustfs_common::heal_channel::HealRequestSource::Mrf;
request
}
struct MrfRuntime {
queue: MrfQueue,
config: MrfConsumerConfig,
new_since_flush: usize,
/// True while a journal snapshot exists on disk that no longer reflects
/// an all-consumed pending set; the next idle tick removes it (MinIO
/// deletes its `list.bin` after replay for the same reason).
journal_on_disk: bool,
/// Earliest instant a full-admission retry may proceed.
backoff_until: Option<tokio::time::Instant>,
}
impl MrfRuntime {
fn record_accept(&mut self) {
// Accepted intents leave the pending set; the next flush persists the
// smaller snapshot, which is the journal's compaction.
}
fn snapshot(&self) -> Vec<u8> {
let mut buf = Vec::new();
for intent in self.queue.intents() {
encode_intent(intent, &mut buf);
}
buf
}
async fn flush(&mut self) {
write_journal(&self.snapshot()).await;
self.new_since_flush = 0;
self.journal_on_disk = true;
}
/// Drain pending intents into the heal manager until it is full, the
/// queue empties, or attempts are exhausted.
async fn dispatch(&mut self, manager: &HealManager) {
if let Some(until) = self.backoff_until {
if tokio::time::Instant::now() < until {
return;
}
self.backoff_until = None;
}
while let Some(mut intent) = self.queue.pop_front() {
let request = build_heal_request(&intent);
match manager.submit_heal_request(request).await {
Ok(HealAdmissionResult::Accepted) | Ok(HealAdmissionResult::Merged) => self.record_accept(),
Ok(HealAdmissionResult::Full) | Ok(HealAdmissionResult::Dropped(HealAdmissionDropReason::QueueFull)) => {
intent.attempts = intent.attempts.saturating_add(1);
if intent.attempts >= MRF_MAX_ATTEMPTS {
counter!("rustfs_heal_mrf_dropped_total", "reason" => "attempts_exhausted").increment(1);
continue;
}
self.queue.push_back(intent);
self.backoff_until = Some(tokio::time::Instant::now() + self.config.admission_backoff);
break;
}
Ok(HealAdmissionResult::Dropped(_)) => {
counter!("rustfs_heal_mrf_dropped_total", "reason" => "admission_policy").increment(1);
}
Err(_) => {
intent.attempts = intent.attempts.saturating_add(1);
if intent.attempts >= MRF_MAX_ATTEMPTS {
counter!("rustfs_heal_mrf_dropped_total", "reason" => "attempts_exhausted").increment(1);
continue;
}
self.queue.push_back(intent);
self.backoff_until = Some(tokio::time::Instant::now() + self.config.admission_backoff);
break;
}
}
}
gauge!("rustfs_heal_mrf_queue_depth").set(self.queue.depth() as f64);
gauge!("rustfs_heal_mrf_queue_bytes").set(self.queue.bytes() as f64);
}
}
/// Initialize the global MRF channel (honoring `RUSTFS_HEAL_MRF_ENABLE`) and
/// spawn the consumer task. Called once from the heal runtime bootstrap right
/// after the manager started; a disabled feature or a double call is a no-op.
/// Public for integration tests that drive the real consumer loop.
pub fn spawn_mrf_consumer(manager: Arc<HealManager>) {
let enabled = rustfs_utils::get_env_bool(rustfs_config::ENV_HEAL_MRF_ENABLE, rustfs_config::DEFAULT_HEAL_MRF_ENABLE);
rustfs_common::mrf_channel::set_mrf_delivery_enabled(enabled);
if !enabled {
tracing::info!(
target: "rustfs::heal::mrf",
"MRF intent pipeline disabled by configuration; producers will not deliver"
);
return;
}
let receiver = match rustfs_common::mrf_channel::init_mrf_channel() {
Ok(receiver) => receiver,
Err(err) => {
tracing::warn!(
target: "rustfs::heal::mrf",
error = err,
"MRF channel initialization failed; intents will be dropped at producers"
);
return;
}
};
tokio::spawn(async move {
run_mrf_consumer(manager, receiver).await;
});
tracing::info!(target: "rustfs::heal::mrf", "MRF intent consumer started");
}
/// Replay the durable journal into a fresh pending queue and submit whatever
/// it armed. Returns the number of intact intents replayed. Duplicates are
/// merged by the manager's dedup key; the journal file is removed once read
/// (torn tails truncate via the per-record CRC). Public for integration tests;
/// the live consumer invokes this through [`replay_into`] at startup.
pub async fn replay_journal_once(manager: &Arc<HealManager>) -> usize {
let config = MrfConsumerConfig::default();
let mut queue = MrfQueue::new(config.queue_capacity, config.journal_max_bytes);
let mut backoff_until: Option<tokio::time::Instant> = None;
replay_into(manager, &mut queue, &mut backoff_until).await
}
/// Shared replay core: read + decode + re-arm + delete, then drain what fits.
async fn replay_into(
manager: &Arc<HealManager>,
queue: &mut MrfQueue,
backoff_until: &mut Option<tokio::time::Instant>,
) -> usize {
let Some(data) = read_journal().await else {
return 0;
};
let (intents, truncated) = decode_journal(&data);
if truncated > 0 {
tracing::warn!(
target: "rustfs::heal::mrf",
truncated_bytes = truncated,
"MRF journal had a torn tail; truncated records were discarded"
);
}
counter!("rustfs_heal_mrf_replayed_total").increment(intents.len() as u64);
let replayed = intents.len();
for intent in intents {
queue.try_push(intent);
}
delete_journal().await;
// Drain the replayed intents immediately; whatever the manager refuses
// stays armed in `queue` for the consumer's retry loop.
if backoff_until.is_none() {
while let Some(mut intent) = queue.pop_front() {
let request = build_heal_request(&intent);
match manager.submit_heal_request(request).await {
Ok(HealAdmissionResult::Accepted) | Ok(HealAdmissionResult::Merged) => {}
Ok(HealAdmissionResult::Full) | Ok(HealAdmissionResult::Dropped(HealAdmissionDropReason::QueueFull)) => {
intent.attempts = intent.attempts.saturating_add(1);
if intent.attempts < MRF_MAX_ATTEMPTS {
queue.push_back(intent);
*backoff_until = Some(tokio::time::Instant::now());
}
break;
}
Ok(HealAdmissionResult::Dropped(_)) | Err(_) => {}
}
}
}
replayed
}
/// Replay the journal, then keep draining the channel into the heal manager
/// while persisting the pending snapshot.
async fn run_mrf_consumer(manager: Arc<HealManager>, mut receiver: mpsc::Receiver<MrfIntent>) {
let config = MrfConsumerConfig::default();
let mut runtime = MrfRuntime {
queue: MrfQueue::new(config.queue_capacity, config.journal_max_bytes),
config: config.clone(),
new_since_flush: 0,
journal_on_disk: false,
backoff_until: None,
};
// Replay: read the journal, re-arm intents (duplicates are merged by the
// manager's dedup key), then drop the file so the next flush starts clean.
replay_into(&manager, &mut runtime.queue, &mut runtime.backoff_until).await;
let mut flush_tick = tokio::time::interval(runtime.config.flush_interval);
flush_tick.set_missed_tick_behavior(tokio::time::MissedTickBehavior::Delay);
let mut batch: Vec<MrfIntent> = Vec::with_capacity(runtime.config.replay_batch);
loop {
tokio::select! {
received = receiver.recv_many(&mut batch, runtime.config.replay_batch) => {
if received == 0 {
// Channel closed: flush once more and stop.
runtime.flush().await;
tracing::info!(
target: "rustfs::heal::mrf",
"MRF channel closed; consumer stopped after final flush"
);
return;
}
for intent in batch.drain(..) {
runtime.queue.try_push(intent);
runtime.new_since_flush += 1;
}
runtime.dispatch(manager.as_ref()).await;
if runtime.new_since_flush >= runtime.config.flush_threshold {
runtime.flush().await;
}
}
_ = flush_tick.tick() => {
if runtime.new_since_flush > 0 || runtime.queue.depth() > 0 {
runtime.flush().await;
runtime.dispatch(manager.as_ref()).await;
} else if runtime.journal_on_disk {
// All intents consumed: remove the journal so a restart
// replays nothing (mirrors MinIO's post-replay unlink).
delete_journal().await;
runtime.journal_on_disk = false;
gauge!("rustfs_heal_mrf_journal_bytes").set(0.0);
}
gauge!("rustfs_heal_mrf_queue_depth").set(runtime.queue.depth() as f64);
}
}
}
}
#[cfg(test)]
mod tests {
use super::*;
use rustfs_common::mrf_channel::{MrfIntent, MrfKind};
use std::sync::Arc as StdArc;
fn intent(bucket: &str, object: &str, attempts: u8) -> MrfIntent {
MrfIntent {
bucket: StdArc::from(bucket),
object: StdArc::from(object),
version_id: Some([7u8; 16]),
kind: MrfKind::DecodeFailure,
enqueued_at_ms: 1_700_000_000_000,
attempts,
}
}
#[test]
fn queue_enforces_count_and_byte_ceilings() {
let mut queue = MrfQueue::new(2, usize::MAX);
assert!(queue.try_push(intent("b", "o", 0)));
assert!(queue.try_push(intent("b", "o", 0)));
assert!(!queue.try_push(intent("b", "o", 0)), "count ceiling must drop");
let mut tiny = MrfQueue::new(usize::MAX, intent("bucket", "object", 0).estimated_bytes());
assert!(tiny.try_push(intent("bucket", "object", 0)));
assert!(
!tiny.try_push(intent("bucket", "object", 0)),
"byte budget must drop before the second intent fits"
);
}
#[test]
fn journal_roundtrip_preserves_intents() {
let intents = vec![
intent("bucket-a", "object/a", 0),
intent("bucket-b", "object/b", 2),
MrfIntent {
bucket: StdArc::from("bucket-c"),
object: StdArc::from("object/c"),
version_id: None,
kind: MrfKind::MetadataCorruption,
enqueued_at_ms: 5,
attempts: 1,
},
];
let mut buf = Vec::new();
for intent in &intents {
encode_intent(intent, &mut buf);
}
let (decoded, truncated) = decode_journal(&buf);
assert_eq!(truncated, 0);
assert_eq!(decoded.len(), intents.len());
for (left, right) in decoded.iter().zip(intents.iter()) {
assert_eq!(left.bucket, right.bucket);
assert_eq!(left.object, right.object);
assert_eq!(left.version_id, right.version_id);
assert_eq!(left.kind, right.kind);
assert_eq!(left.attempts, right.attempts);
}
}
#[test]
fn journal_torn_tail_is_truncated() {
let mut buf = Vec::new();
encode_intent(&intent("b", "o", 0), &mut buf);
let mut torn = buf.clone();
torn.extend_from_slice(&buf[..buf.len() / 2]);
let (decoded, truncated) = decode_journal(&torn);
assert_eq!(decoded.len(), 1, "the intact record must survive");
assert!(truncated > 0, "the partial tail must be discarded");
// A corrupted body (CRC mismatch) also truncates from that record on.
let mut corrupt = buf.clone();
let mid = MRF_RECORD_FIXED_HEAD + 4;
corrupt[mid] ^= 0xff;
let (decoded, truncated) = decode_journal(&corrupt);
assert!(decoded.is_empty());
assert_eq!(truncated, corrupt.len());
}
#[test]
fn heal_request_mapping_follows_priority_matrix() {
let decode = build_heal_request(&intent("b", "o", 0));
assert!(matches!(decode.heal_type, HealType::ECDecode { .. }));
assert_eq!(decode.priority, HealPriority::Urgent);
let metadata = build_heal_request(&MrfIntent {
bucket: StdArc::from("b"),
object: StdArc::from("o"),
version_id: None,
kind: MrfKind::MetadataCorruption,
enqueued_at_ms: 0,
attempts: 0,
});
assert!(matches!(metadata.heal_type, HealType::Metadata { .. }));
assert_eq!(metadata.priority, HealPriority::High);
let partial = build_heal_request(&MrfIntent {
bucket: StdArc::from("b"),
object: StdArc::from("o"),
version_id: None,
kind: MrfKind::PartialWrite,
enqueued_at_ms: 0,
attempts: 0,
});
assert!(matches!(partial.heal_type, HealType::Object { .. }));
assert_eq!(partial.priority, HealPriority::Normal);
}
}
+136 -4
View File
@@ -32,7 +32,7 @@ use std::{
future::Future,
sync::{
Arc,
atomic::{AtomicBool, Ordering},
atomic::{AtomicBool, AtomicU64, Ordering},
},
time::{Duration, Instant, SystemTime},
};
@@ -351,6 +351,20 @@ impl HealRequest {
}
/// Heal task
/// Incremental view over a task's retained result items (HS-06).
///
/// `next_seq` is the cursor a client should pass on its next poll; `min_seq`
/// is the oldest sequence still retained; `lagged` means the client's cursor
/// fell behind `min_seq` and items were skipped — the client should restart
/// from `min_seq`.
#[derive(Debug, Clone)]
pub struct HealResultWindow {
pub items: Vec<HealResultItem>,
pub next_seq: u64,
pub min_seq: u64,
pub lagged: bool,
}
pub struct HealTask {
/// Task ID
pub id: String,
@@ -373,8 +387,16 @@ pub struct HealTask {
pub status: Arc<RwLock<HealTaskStatus>>,
/// Progress tracking
pub progress: Arc<RwLock<HealProgress>>,
/// Result items collected from storage heal calls.
pub result_items: Arc<RwLock<Vec<HealResultItem>>>,
/// Result items collected from storage heal calls, each stamped with a
/// monotonically increasing sequence number for incremental consumption
/// (the client passes the last seen seq back and receives only newer
/// items; see `get_result_items_since`).
pub result_items: Arc<RwLock<Vec<(u64, HealResultItem)>>>,
/// Next sequence number to assign; starts at 1.
next_item_seq: Arc<AtomicU64>,
/// Sequence number of the oldest item still inside the retention window;
/// equals `next_item_seq` while the window is empty.
min_available_seq: Arc<AtomicU64>,
result_items_truncated: Arc<AtomicBool>,
batch_failure: Arc<RwLock<Option<BatchHealFailure>>>,
batch_failure_recorded: Arc<AtomicBool>,
@@ -426,6 +448,8 @@ impl HealTask {
status: Arc::new(RwLock::new(HealTaskStatus::Pending)),
progress: Arc::new(RwLock::new(HealProgress::new())),
result_items: Arc::new(RwLock::new(Vec::new())),
next_item_seq: Arc::new(AtomicU64::new(1)),
min_available_seq: Arc::new(AtomicU64::new(1)),
result_items_truncated: Arc::new(AtomicBool::new(false)),
batch_failure: Arc::new(RwLock::new(None)),
batch_failure_recorded: Arc::new(AtomicBool::new(false)),
@@ -911,18 +935,63 @@ impl HealTask {
}
pub async fn get_result_items(&self) -> Vec<HealResultItem> {
self.result_items.read().await.iter().map(|(_, item)| item.clone()).collect()
}
/// Sequence-stamped retained window, used when archiving a completed
/// task so incremental cursors survive the transition (HS-06).
pub async fn get_seqed_result_items(&self) -> Vec<(u64, HealResultItem)> {
self.result_items.read().await.clone()
}
/// Incremental result window (HS-06): `since = None` returns the full
/// retained window (legacy snapshot semantics); `since = Some(seq)`
/// returns only items stamped with a sequence greater than `seq`.
/// `lagged` warns that the caller's cursor fell behind the window start
/// and items were skipped (the response carries `min_seq` as the catch-up
/// cursor).
pub async fn get_result_items_since(&self, since: Option<u64>) -> HealResultWindow {
let result_items = self.result_items.read().await;
let next_seq = self.next_item_seq.load(Ordering::Relaxed);
let min_seq = self.min_available_seq.load(Ordering::Relaxed);
let mut lagged = false;
let items = match since {
None => result_items.iter().map(|(_, item)| item.clone()).collect::<Vec<_>>(),
Some(cursor) => {
if cursor + 1 < min_seq {
lagged = true;
}
result_items
.iter()
.filter(|(seq, _)| *seq > cursor)
.map(|(_, item)| item.clone())
.collect::<Vec<_>>()
}
};
HealResultWindow {
items,
next_seq,
min_seq,
lagged,
}
}
pub fn result_items_truncated(&self) -> bool {
self.result_items_truncated.load(Ordering::Relaxed)
}
async fn record_result_item(&self, result: HealResultItem) {
let seq = self.next_item_seq.fetch_add(1, Ordering::Relaxed);
let mut result_items = self.result_items.write().await;
if result_items.len() < MAX_RETAINED_HEAL_RESULT_ITEMS {
result_items.push(result);
result_items.push((seq, result));
} else {
// Slide the window: the oldest item leaves and the cursor for the
// oldest still-available item moves forward with it.
result_items.remove(0);
self.min_available_seq
.store(result_items.first().map_or(seq, |(oldest, _)| *oldest), Ordering::Relaxed);
result_items.push((seq, result));
self.result_items_truncated.store(true, Ordering::Relaxed);
}
}
@@ -3880,6 +3949,69 @@ mod tests {
assert!(task.result_items_truncated());
}
// HS-06 (backlog#1870): incremental result windows.
#[tokio::test]
async fn result_items_seq_is_monotonic_and_incremental_slices_work() {
let storage = Arc::new(MockStorage::default());
let task = HealTask::from_request(HealRequest::bucket("bucket-a".to_string()), storage);
for round in 0..5u64 {
let item = HealResultItem {
object_size: round as usize,
..Default::default()
};
task.record_result_item(item).await;
}
let full = task.get_result_items_since(None).await;
assert_eq!(full.items.len(), 5, "None keeps the full-snapshot semantics");
assert_eq!(full.next_seq, 6, "next_seq is one past the last assigned");
assert_eq!(full.min_seq, 1, "nothing was evicted yet");
assert!(!full.lagged);
// Incremental: only items newer than the cursor.
let incremental = task.get_result_items_since(Some(3)).await;
assert_eq!(
incremental.items.iter().map(|item| item.object_size).collect::<Vec<_>>(),
vec![3, 4],
"only sequences greater than the cursor are returned"
);
assert_eq!(incremental.next_seq, 6);
// A cursor at the head is not lagging.
assert!(!task.get_result_items_since(Some(0)).await.lagged);
}
#[tokio::test]
async fn result_items_window_slide_moves_min_seq_and_flags_lagging_cursors() {
let storage = Arc::new(MockStorage::default());
let task = HealTask::from_request(HealRequest::bucket("bucket-a".to_string()), storage);
// Fill the window completely, then push two more items: seq 1 and 2
// are evicted by the slide.
for _ in 0..(MAX_RETAINED_HEAL_RESULT_ITEMS + 2) {
task.record_result_item(HealResultItem::default()).await;
}
let full = task.get_result_items_since(None).await;
assert_eq!(full.items.len(), MAX_RETAINED_HEAL_RESULT_ITEMS);
assert_eq!(full.min_seq, 3, "each evicted head item moved the oldest-available cursor");
assert!(task.result_items_truncated());
// A client still polling from before the eviction is lagging.
let lagging = task.get_result_items_since(Some(0)).await;
assert!(lagging.lagged, "a cursor behind min_seq must be flagged");
assert_eq!(lagging.min_seq, 3, "the response tells the client where to restart");
// A cursor inside the window is fine.
assert!(!task.get_result_items_since(Some(3)).await.lagged);
// The lagging client restarts from min_seq and gets the full window.
let catch_up = task.get_result_items_since(Some(3)).await;
assert_eq!(catch_up.items.len(), MAX_RETAINED_HEAL_RESULT_ITEMS - 1);
assert!(!catch_up.lagged);
}
#[tokio::test]
async fn test_recursive_bucket_heal_skips_object_dir_candidates() {
let storage = Arc::new(MockStorage {
+4
View File
@@ -158,6 +158,10 @@ pub async fn init_heal_manager_with_workload_provider(
return Err(err);
}
// Start the MRF intent consumer (error-path repair intents + durable
// journal replay) now that the manager can accept submissions.
heal::mrf_queue::spawn_mrf_consumer(heal_manager.clone());
#[cfg(test)]
test_hook_after_manager_start().await;
+189
View File
@@ -0,0 +1,189 @@
// Copyright 2024 RustFS Team
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//! HS-01 (rustfs/backlog#1865): MRF intent pipeline integration tests.
//!
//! Drives the real consumer loop (`spawn_mrf_consumer`) against a real
//! 4-disk `ECStore` heal storage and a `HealManager` that has not started its
//! scheduler, so submitted intents stay observable in the admission queue.
//! Under `cargo nextest` each test runs in its own process, which keeps the
//! process-global MRF channel singleton safe.
use rustfs_common::mrf_channel::{self, MrfKind};
use rustfs_heal::heal::{
manager::{HealConfig, HealManager},
mrf_queue,
storage::{ECStoreHealStorage, HealStorageAPI},
};
use serial_test::serial;
use std::{path::Path, sync::Arc, time::Duration};
mod storage_api;
use storage_api::endpoint_index::{Endpoint, EndpointServerPools, Endpoints, PoolEndpoints, init_local_disks};
const META_BUCKET: &str = ".rustfs.sys";
const JOURNAL_REL: &str = "buckets/.heal/mrf/journal.bin";
async fn heal_env() -> (Vec<std::path::PathBuf>, Arc<dyn HealStorageAPI>) {
let env = rustfs_test_utils::TestECStoreEnv::builder()
.prefix("rustfs_heal_mrf_test")
.build()
.await;
let heal_storage: Arc<dyn HealStorageAPI> = Arc::new(ECStoreHealStorage::new(env.ecstore.clone()));
(env.disk_paths, heal_storage)
}
fn make_manager(storage: Arc<dyn HealStorageAPI>) -> Arc<HealManager> {
Arc::new(HealManager::new(
storage,
Some(HealConfig {
// Keep the scheduler from draining the queue before assertions.
heal_interval: Duration::from_secs(3600),
enable_auto_heal: false,
..Default::default()
}),
))
}
/// Encode one journal record independently of the implementation, so a format
/// drift between writer and this fixture fails loudly here.
fn journal_record(kind: u8, bucket: &str, object: &str, version: Option<[u8; 16]>, attempts: u8) -> Vec<u8> {
let mut body = vec![1u8, 1, kind, attempts];
body.extend_from_slice(&1_700_000_000_000u64.to_le_bytes());
match version {
Some(bytes) => {
body.push(1);
body.extend_from_slice(&bytes);
}
None => body.push(0),
}
body.extend_from_slice(&(bucket.len() as u32).to_le_bytes());
body.extend_from_slice(&(object.len() as u32).to_le_bytes());
body.extend_from_slice(bucket.as_bytes());
body.extend_from_slice(object.as_bytes());
let mut hasher = crc_fast::Digest::new(crc_fast::CrcAlgorithm::Crc32IsoHdlc);
hasher.update(&body);
body.extend_from_slice(&(hasher.finalize() as u32).to_le_bytes());
body
}
fn write_journal_to_disks(disk_paths: &[std::path::PathBuf], data: &[u8]) {
for path in disk_paths {
let journal = path.join(META_BUCKET).join(JOURNAL_REL);
std::fs::create_dir_all(journal.parent().expect("journal parent")).expect("create journal dir");
std::fs::write(&journal, data).expect("write journal fixture");
}
}
async fn wait_until<F, Fut>(deadline: Duration, mut probe: F) -> bool
where
F: FnMut() -> Fut,
Fut: std::future::Future<Output = bool>,
{
let start = std::time::Instant::now();
while start.elapsed() < deadline {
if probe().await {
return true;
}
tokio::time::sleep(Duration::from_millis(50)).await;
}
false
}
/// A decode-failure intent delivered on the global channel must surface in the
/// heal manager as an Urgent request attributed to the MRF source.
#[tokio::test]
#[serial]
async fn decode_failure_intent_maps_to_urgent_mrf_heal_request() {
let (_disk_paths, storage) = heal_env().await;
let manager = make_manager(storage);
mrf_queue::spawn_mrf_consumer(manager.clone());
assert!(
mrf_channel::try_send_mrf_intent(MrfKind::DecodeFailure, "mrf-bucket", "mrf-object", None),
"intent should be accepted while the consumer holds the channel"
);
let appeared = wait_until(Duration::from_secs(10), || async {
let snapshot = manager.operations_snapshot().await;
snapshot.queued_by_source.mrf >= 1 && snapshot.queued_by_priority.urgent >= 1
})
.await;
assert!(
appeared,
"MRF intent must reach the manager queue as an Urgent request (snapshot: {:?})",
manager.operations_snapshot().await
);
}
/// A journal left behind by a previous process must be replayed into the
/// manager queue and then removed, and a torn tail must not block replay of
/// the intact records.
#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
#[serial]
async fn journal_replay_arms_intents_and_deletes_the_file() {
let (disk_paths, storage) = heal_env().await;
// The journal reader resolves disks through the process-local disk map;
// register the environment's disks the same way server startup does.
let mut endpoints: Vec<Endpoint> = disk_paths
.iter()
.map(|p| Endpoint::try_from(p.to_string_lossy().as_ref()).expect("endpoint from disk path"))
.collect();
for (i, endpoint) in endpoints.iter_mut().enumerate() {
endpoint.set_pool_index(0);
endpoint.set_set_index(0);
endpoint.set_disk_index(i);
}
let pool = PoolEndpoints {
legacy: false,
set_count: 1,
drives_per_set: endpoints.len(),
endpoints: Endpoints::from(endpoints),
cmd_line: "mrf-test".to_string(),
platform: String::new(),
};
init_local_disks(EndpointServerPools::from(vec![pool]))
.await
.expect("local disks should register");
let mut journal = journal_record(1, "replay-bucket", "replay-object", Some([9u8; 16]), 0);
journal.extend(journal_record(3, "replay-bucket", "partial-object", None, 1));
// Torn tail: a third record truncated mid-way must not block the two
// intact records above.
journal.extend_from_slice(&journal_record(2, "replay-bucket", "metadata-object", None, 0)[..8]);
write_journal_to_disks(&disk_paths, &journal);
let manager = make_manager(storage);
// Replay directly (not via the process-global channel consumer, which the
// sibling test already claimed in this process under plain `cargo test`).
let replayed = mrf_queue::replay_journal_once(&manager).await;
assert_eq!(replayed, 2, "the two intact records must be replayed");
let snapshot = manager.operations_snapshot().await;
assert_eq!(snapshot.queued_by_source.mrf, 2, "replayed intents must be attributed to the MRF source");
assert!(
disk_paths
.iter()
.all(|path| !Path::new(path).join(META_BUCKET).join(JOURNAL_REL).exists()),
"the journal file must be removed after a successful replay"
);
let snapshot = manager.operations_snapshot().await;
assert_eq!(snapshot.queued_by_priority.urgent, 1, "the decode-failure record must replay as Urgent");
assert!(snapshot.queued_by_priority.normal >= 1, "the partial-write record must replay as Normal");
}
+3
View File
@@ -9,6 +9,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Removed
#### rustfs-io-core
- **Zero-consumer modules** (added in 0.0.5): `reader`, `writer`, `bufreader_optimizer`, `shared_memory`, `direct_io`, `timeout_wrapper`, `io_priority_queue`, and `scheduler` had no caller in the workspace and were removed (rustfs/backlog#1824). The scheduling algorithm and the request timeout wrapper that RustFS actually runs live in `rustfs/src/storage/`; this crate keeps the config shapes they project into. `OperationProgress` moved to the new `progress` module and is still exported as `rustfs_io_core::OperationProgress`.
#### rustfs-io-metrics
- **Unified configuration** (added in 0.0.5): the zero-consumer `IoConfig`, `CacheSettings`, `IoSchedulerSettings`, `BackpressureSettings`, `TimeoutSettings`, `DeadlockDetectionSettings` types and their `DEFAULT_*` constants were removed (rustfs/rustfs#6008); rustfs-io-core's `IoSchedulerConfig`/`BackpressureConfig` remain the canonical configuration types.
+2 -3
View File
@@ -20,8 +20,8 @@ license.workspace = true
repository.workspace = true
rust-version.workspace = true
homepage.workspace = true
description = "Buffered I/O reader and writer implementations for RustFS (mmap-then-copy, aligned pread)"
keywords = ["io", "reader", "writer", "rustfs", "mmap"]
description = "Shared I/O primitives for RustFS (buffer pool, storage profiling, backpressure, deadlock detection)"
keywords = ["io", "buffer", "pool", "rustfs", "backpressure"]
categories = ["development-tools", "filesystem"]
[lints]
@@ -38,7 +38,6 @@ hotpath.workspace = true
bytes = { workspace = true, features = ["serde"] }
thiserror = { workspace = true }
tokio = { workspace = true, features = ["io-util", "fs", "sync", "rt-multi-thread"] }
memmap2 = { workspace = true }
rustfs-io-metrics = { workspace = true }
tracing = { workspace = true }
+18 -120
View File
@@ -23,67 +23,20 @@
## Overview
**rustfs-io-core** is the core I/O scheduling module for [RustFS](https://rustfs.com), a distributed object storage system. It provides:
**rustfs-io-core** holds the shared I/O primitives for [RustFS](https://rustfs.com), a distributed object storage system. It provides:
- **I/O Scheduler**: Adaptive buffer size calculation and load management
- **Priority Queue**: Request priority scheduling with starvation prevention
- **Buffer Pool**: Tiered `BytesPool` for buffer reuse
- **Storage Profiling**: Storage-media and access-pattern model (`io_profile`)
- **Scheduler Configuration**: The `IoSchedulerConfig` / `IoPriorityQueueConfig` shapes the storage layer projects into
- **Backpressure Control**: System overload protection with graceful degradation
- **Deadlock Detection**: Wait-for graph based deadlock detection algorithm
- **Lock Optimizer**: Adaptive spin lock optimization
- **Timeout Wrapper**: Dynamic timeout calculation and operation progress tracking
- **Progress Tracking**: Byte progress and staleness for long-running operations
The scheduling algorithm itself lives in `rustfs/src/storage/concurrency/io_schedule.rs`; this crate carries the configuration shapes it projects into, not a second implementation.
## Features
### I/O Scheduler
Adaptive I/O scheduling with dynamic buffer size calculation based on file size, access pattern, and system load:
```rust
use rustfs_io_core::{IoScheduler, IoSchedulerConfig, IoLoadLevel};
use rustfs_io_core::io_profile::{StorageMedia, AccessPattern};
// Create scheduler
let config = IoSchedulerConfig {
max_concurrent_reads: 64,
base_buffer_size: 64 * 1024, // 64 KB
max_buffer_size: 1024 * 1024, // 1 MB
..Default::default()
};
let scheduler = IoScheduler::new(config);
// Calculate optimal buffer size
let buffer_size = calculate_optimal_buffer_size(
10 * 1024 * 1024, // 10 MB file
64 * 1024, // base buffer
true, // sequential access
4, // concurrent requests
StorageMedia::Ssd,
IoLoadLevel::Low,
);
```
### Priority Queue
Priority queue with starvation prevention:
```rust
use rustfs_io_core::{IoPriorityQueue, IoPriority, IoQueueStatus};
let queue = IoPriorityQueue::<()>::new(100);
// Enqueue request
let request_id = queue.enqueue(IoPriority::High, (), 1024);
// Dequeue request
if let Some((priority, data)) = queue.dequeue() {
println!("Processing priority {:?} request", priority);
}
// Check queue status
let status = queue.status();
println!("High priority waiting: {}", status.high_priority_waiting);
```
### Backpressure Control
System overload protection:
@@ -148,71 +101,23 @@ let stats = optimizer.stats();
println!("Locks acquired: {}", stats.total_acquired());
```
### Timeout Wrapper
### Progress Tracking
Dynamic timeout calculation:
Byte progress and staleness for long-running operations:
```rust
use rustfs_io_core::{RequestTimeoutWrapper, TimeoutConfig};
use rustfs_io_core::OperationProgress;
use std::time::Duration;
let config = TimeoutConfig {
base_timeout: Duration::from_secs(5),
timeout_per_mb: Duration::from_millis(100),
max_timeout: Duration::from_secs(300),
..Default::default()
};
let wrapper = RequestTimeoutWrapper::new(config);
let progress = OperationProgress::new(Some(1000), Duration::from_secs(5));
// Calculate operation timeout
let timeout = wrapper.calculate_timeout(10 * 1024 * 1024); // 10 MB
```
## Buffer Size Calculation
Multiple buffer size calculation functions are provided:
```rust
use rustfs_io_core::{
get_concurrency_aware_buffer_size,
get_advanced_buffer_size,
get_buffer_size_for_media,
calculate_optimal_buffer_size,
KI_B, MI_B,
};
use rustfs_io_core::io_profile::StorageMedia;
// Basic calculation
let size1 = get_concurrency_aware_buffer_size(1024 * 1024, 64 * 1024);
// Advanced calculation (considering access pattern)
let size2 = get_advanced_buffer_size(10 * 1024 * 1024, 64 * 1024, true);
// Media type optimization
let size3 = get_buffer_size_for_media(64 * 1024, StorageMedia::Ssd);
// Comprehensive calculation
let size4 = calculate_optimal_buffer_size(
100 * 1024 * 1024, // 100 MB file
64 * 1024, // base buffer
true, // sequential access
4, // concurrent requests
StorageMedia::Nvme,
IoLoadLevel::Low,
);
progress.update(500);
assert_eq!(progress.progress_percent(), Some(50.0));
assert!(!progress.is_stale());
```
## Configuration
### Environment Variables
| Variable | Description | Default |
|----------|-------------|---------|
| `RUSTFS_MAX_CONCURRENT_READS` | Max concurrent reads | 64 |
| `RUSTFS_BASE_BUFFER_SIZE` | Base buffer size | 65536 |
| `RUSTFS_MAX_BUFFER_SIZE` | Max buffer size | 1048576 |
| `RUSTFS_IO_TIMEOUT_SECS` | I/O timeout seconds | 30 |
### Code Configuration
```rust
@@ -240,12 +145,11 @@ rustfs-io-core/
├── src/
│ ├── lib.rs # Module entry
│ ├── config.rs # Configuration types
│ ├── scheduler.rs # I/O scheduler
│ ├── io_priority_queue.rs # Priority queue
│ ├── pool.rs # Tiered buffer pool
│ ├── backpressure.rs # Backpressure control
│ ├── deadlock_detector.rs # Deadlock detection
│ ├── lock_optimizer.rs # Lock optimization
│ ├── timeout_wrapper.rs # Timeout wrapper
│ ├── progress.rs # Operation progress tracking
│ └── io_profile.rs # I/O profile
└── Cargo.toml
```
@@ -254,21 +158,15 @@ rustfs-io-core/
```bash
# Run all tests
cargo test --package rustfs-io-core
cargo nextest run --package rustfs-io-core
# Run specific tests
cargo test --package rustfs-io-core --lib scheduler
# Run benchmarks
cargo bench --package rustfs-io-core
cargo nextest run --package rustfs-io-core -E 'test(backpressure)'
```
## Documentation
- [API Documentation](https://docs.rs/rustfs-io-core)
- [I/O Scheduler Design](./docs/scheduler-design.md)
- [Backpressure Control Design](./docs/backpressure-design.md)
- [Deadlock Detection Algorithm](./docs/deadlock-detection.md)
## Related Modules
+18 -131
View File
@@ -23,71 +23,20 @@
## 📖 概述
**rustfs-io-core** 是 [RustFS](https://rustfs.com) 分布式对象存储系统的核心 I/O 调度模块。它提供了:
**rustfs-io-core** 是 [RustFS](https://rustfs.com) 分布式对象存储系统的共享 I/O 基础组件。它提供了:
- **I/O 调度器**:自适应缓冲区大小计算和负载管理
- **优先级队列**支持饥饿预防的请求优先级调度
- **缓冲池**:分级复用的 `BytesPool`
- **存储画像**存储介质与访问模式模型(`io_profile`
- **调度配置**:存储层投影使用的 `IoSchedulerConfig` / `IoPriorityQueueConfig`
- **背压控制**:系统过载保护和优雅降级
- **死锁检测**:基于等待图的死锁检测算法
- **锁优化**:自适应自旋锁优化
- **超时包装器**动态超时计算和操作进度追踪
- **进度追踪**长耗时操作的字节进度与停滞判定
调度算法本身位于 `rustfs/src/storage/concurrency/io_schedule.rs`;本 crate 只承载它投影使用的配置形状,不是第二套实现。
## ✨ 核心功能
### I/O 调度器 (IoScheduler)
自适应 I/O 调度,根据文件大小、访问模式和系统负载动态调整缓冲区大小:
```rust
use rustfs_io_core::{IoScheduler, IoSchedulerConfig, IoLoadLevel};
use rustfs_io_core::io_profile::{StorageMedia, AccessPattern};
// 创建调度器
let config = IoSchedulerConfig {
max_concurrent_reads: 64,
base_buffer_size: 64 * 1024, // 64 KB
max_buffer_size: 1024 * 1024, // 1 MB
..Default::default()
};
let scheduler = IoScheduler::new(config);
// 计算最优缓冲区大小
let buffer_size = scheduler.calculate_buffer_size(
10 * 1024 * 1024, // 10 MB 文件
true, // 顺序访问
StorageMedia::Ssd,
IoLoadLevel::Low,
);
println!("缓冲区大小: {} bytes", buffer_size);
```
### 优先级队列 (IoPriorityQueue)
支持饥饿预防的优先级队列:
```rust
use rustfs_io_core::{IoPriorityQueue, IoPriority, IoQueueStatus};
let queue = IoPriorityQueue::<()>::new(100);
// 入队请求
let request_id = queue.enqueue(
IoPriority::High,
(), // 请求数据
1024, // 请求大小
);
// 出队请求
if let Some((priority, data)) = queue.dequeue() {
println!("处理优先级 {:?} 的请求", priority);
}
// 检查队列状态
let status = queue.status();
println!("高优先级等待: {}", status.high_priority_waiting);
println!("低优先级等待: {}", status.low_priority_waiting);
```
### 背压控制 (BackpressureMonitor)
系统过载保护:
@@ -165,78 +114,23 @@ let stats = optimizer.stats();
println!("获取锁次数: {}", stats.locks_acquired.load(std::sync::atomic::Ordering::Relaxed));
```
### 超时包装器 (RequestTimeoutWrapper)
### 进度追踪 (OperationProgress)
动态超时计算
长耗时操作的字节进度与停滞判定
```rust
use rustfs_io_core::{RequestTimeoutWrapper, TimeoutConfig};
use rustfs_io_core::OperationProgress;
use std::time::Duration;
let config = TimeoutConfig {
base_timeout: Duration::from_secs(5),
timeout_per_mb: Duration::from_millis(100),
max_timeout: Duration::from_secs(300),
..Default::default()
};
let wrapper = RequestTimeoutWrapper::new(config);
let progress = OperationProgress::new(Some(1000), Duration::from_secs(5));
// 计算操作超时
let timeout = wrapper.calculate_timeout(10 * 1024 * 1024); // 10 MB
println!("超时时间: {:?}", timeout);
// 执行带超时的操作
let result = wrapper.execute_with_timeout(async {
// 异步操作
Ok::<_, std::io::Error>(())
}, timeout).await;
```
## 📊 缓冲区大小计算
模块提供了多种缓冲区大小计算函数:
```rust
use rustfs_io_core::{
get_concurrency_aware_buffer_size,
get_advanced_buffer_size,
get_buffer_size_for_media,
calculate_optimal_buffer_size,
KI_B, MI_B,
};
use rustfs_io_core::io_profile::StorageMedia;
// 基础计算
let size1 = get_concurrency_aware_buffer_size(1024 * 1024, 64 * 1024);
// 高级计算(考虑访问模式)
let size2 = get_advanced_buffer_size(10 * 1024 * 1024, 64 * 1024, true);
// 媒体类型优化
let size3 = get_buffer_size_for_media(64 * 1024, StorageMedia::Ssd);
// 综合计算
let size4 = calculate_optimal_buffer_size(
100 * 1024 * 1024, // 100 MB 文件
64 * 1024, // 基础缓冲区
true, // 顺序访问
4, // 并发请求数
StorageMedia::Nvme,
IoLoadLevel::Low,
);
progress.update(500);
assert_eq!(progress.progress_percent(), Some(50.0));
assert!(!progress.is_stale());
```
## 🔧 配置
### 环境变量
| 变量名 | 描述 | 默认值 |
|--------|------|--------|
| `RUSTFS_MAX_CONCURRENT_READS` | 最大并发读数 | 64 |
| `RUSTFS_BASE_BUFFER_SIZE` | 基础缓冲区大小 | 65536 |
| `RUSTFS_MAX_BUFFER_SIZE` | 最大缓冲区大小 | 1048576 |
| `RUSTFS_IO_TIMEOUT_SECS` | I/O 超时秒数 | 30 |
### 代码配置
```rust
@@ -264,12 +158,11 @@ rustfs-io-core/
├── src/
│ ├── lib.rs # 模块入口
│ ├── config.rs # 配置类型
│ ├── scheduler.rs # I/O 调度器
│ ├── io_priority_queue.rs # 优先级队列
│ ├── pool.rs # 分级缓冲池
│ ├── backpressure.rs # 背压控制
│ ├── deadlock_detector.rs # 死锁检测
│ ├── lock_optimizer.rs # 锁优化
│ ├── timeout_wrapper.rs # 超时包装器
│ ├── progress.rs # 操作进度追踪
│ └── io_profile.rs # I/O 配置文件
└── Cargo.toml
```
@@ -278,21 +171,15 @@ rustfs-io-core/
```bash
# 运行所有测试
cargo test --package rustfs-io-core
cargo nextest run --package rustfs-io-core
# 运行特定测试
cargo test --package rustfs-io-core --lib scheduler
# 运行基准测试
cargo bench --package rustfs-io-core
cargo nextest run --package rustfs-io-core -E 'test(backpressure)'
```
## 📚 文档
- [API 文档](https://docs.rs/rustfs-io-core)
- [I/O 调度器设计](./docs/scheduler-design.md)
- [背压控制原理](./docs/backpressure-design.md)
- [死锁检测算法](./docs/deadlock-detection.md)
## 🔗 相关模块
@@ -1,190 +0,0 @@
// Copyright 2024 RustFS Team
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//! Example demonstrating I/O scheduler usage.
use rustfs_io_core::io_profile::StorageMedia;
use rustfs_io_core::{
BackpressureMonitor, BackpressureState, DeadlockDetector, IoLoadLevel, IoScheduler, IoSchedulerConfig, KI_B, LockOptimizer,
LockType, MI_B, calculate_optimal_buffer_size, get_buffer_size_for_media,
};
use std::time::Duration;
fn main() {
println!("=== rustfs-io-core Example ===\n");
// 1. I/O scheduler example
io_scheduler_example();
// 2. Buffer size calculation example
buffer_size_example();
// 3. Backpressure control example
backpressure_example();
// 4. Deadlock detection example
deadlock_detection_example();
// 5. Lock optimizer example
lock_optimizer_example();
}
fn io_scheduler_example() {
println!("--- I/O Scheduler ---");
// Create scheduler with configuration
let config = IoSchedulerConfig {
max_concurrent_reads: 64,
base_buffer_size: 64 * KI_B,
max_buffer_size: MI_B,
..Default::default()
};
let scheduler = IoScheduler::new(config);
println!(" Max concurrent reads: {}", scheduler.config().max_concurrent_reads);
println!(" Base buffer size: {} KB", scheduler.config().base_buffer_size / KI_B);
println!(" Max buffer size: {} KB", scheduler.config().max_buffer_size / KI_B);
// Calculate buffer sizes for different scenarios
let scenarios = [
("Small file", 10 * KI_B as i64, true, StorageMedia::Ssd),
("Medium file", MI_B as i64, true, StorageMedia::Ssd),
("Large sequential", 100 * MI_B as i64, true, StorageMedia::Ssd),
("Large random", 100 * MI_B as i64, false, StorageMedia::Ssd),
("NVMe large", 100 * MI_B as i64, true, StorageMedia::Nvme),
("HDD large", 100 * MI_B as i64, true, StorageMedia::Hdd),
];
for (name, size, sequential, media) in scenarios {
let buffer = calculate_optimal_buffer_size(size, 64 * KI_B, sequential, 4, media, IoLoadLevel::Low);
println!(" {}: {} bytes ({} KB)", name, buffer, buffer / KI_B);
}
println!();
}
fn buffer_size_example() {
println!("--- Buffer Size Calculation ---");
// Comprehensive calculation
let size1 = calculate_optimal_buffer_size(10 * MI_B as i64, 64 * KI_B, true, 4, StorageMedia::Ssd, IoLoadLevel::Low);
println!(" Comprehensive (10MB, sequential, SSD): {} KB", size1 / KI_B);
// Media type optimization
let media_types = [
StorageMedia::Nvme,
StorageMedia::Ssd,
StorageMedia::Hdd,
StorageMedia::Unknown,
];
for media in media_types {
let size = get_buffer_size_for_media(64 * KI_B, media);
println!(" {} optimized: {} KB", media.as_str(), size / KI_B);
}
println!();
}
fn backpressure_example() {
println!("--- Backpressure Control ---");
let monitor = BackpressureMonitor::with_defaults();
// Check initial state
let state = monitor.state();
let state_str = match state {
BackpressureState::Normal => "Normal",
BackpressureState::Warning => "Warning",
BackpressureState::Critical => "Critical",
};
println!(" Initial state: {}", state_str);
// Check if active
let is_active = monitor.is_active();
println!(" Backpressure active: {}", is_active);
// Try to acquire permit
if monitor.try_acquire() {
println!(" Successfully acquired permit");
monitor.release();
println!(" Released permit");
}
// View statistics
println!(" Total processed: {}", monitor.total_processed());
println!(" Total rejected: {}", monitor.total_rejected());
println!();
}
fn deadlock_detection_example() {
println!("--- Deadlock Detection ---");
let detector = DeadlockDetector::with_defaults();
// Register locks
let mutex1 = detector.register_lock(LockType::Mutex);
let mutex2 = detector.register_lock(LockType::Mutex);
println!(" Registered locks: mutex1={}, mutex2={}", mutex1, mutex2);
// Simulate normal operation
detector.record_acquire(mutex1, 1); // Thread 1 acquires mutex1
detector.record_acquire(mutex2, 2); // Thread 2 acquires mutex2
println!(" Normal operation: no deadlock");
// Detect deadlock
if detector.detect_deadlock().is_none() {
println!(" Detection result: no deadlock");
}
// Simulate deadlock scenario
detector.record_wait(mutex2, 1); // Thread 1 waits for mutex2
detector.record_wait(mutex1, 2); // Thread 2 waits for mutex1
// Detect deadlock
if let Some(deadlock) = detector.detect_deadlock() {
println!(" Detection result: deadlock found {:?}", deadlock);
}
// Cleanup
detector.unregister_lock(mutex1);
detector.unregister_lock(mutex2);
println!();
}
fn lock_optimizer_example() {
println!("--- Lock Optimizer ---");
let optimizer = LockOptimizer::with_defaults();
// Simulate lock operations
for _i in 0..5 {
optimizer.on_acquire();
// Simulate work
std::thread::sleep(Duration::from_millis(10));
optimizer.on_release(Duration::from_millis(10));
}
// View statistics
let stats = optimizer.stats();
let acquired = stats.total_acquired();
let avg_hold = stats.avg_hold_time();
let contention = stats.contention_rate();
println!(" Locks acquired: {}", acquired);
println!(" Average hold time: {:?}", avg_hold);
println!(" Contention rate: {:.2}%", contention * 100.0);
println!();
}
-227
View File
@@ -1,227 +0,0 @@
// Copyright 2024 RustFS Team
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//! BufReader layer optimizer for minimizing redundant buffering layers.
//!
//! This module provides optimization for BufReader usage in data paths,
//! including layer count limiting and dynamic buffer size adjustment.
use std::sync::atomic::{AtomicU64, Ordering};
/// BufReader optimization configuration.
#[derive(Debug, Clone)]
pub struct BufReaderConfig {
/// Maximum number of nested BufReader layers (default: 2)
pub max_layers: u32,
/// Buffer size for small files (default: 8KB)
pub small_file_buffer: usize,
/// Buffer size for large files (default: 64KB)
pub large_file_buffer: usize,
/// Threshold for large file classification (default: 1MB)
pub large_file_threshold: usize,
}
impl Default for BufReaderConfig {
fn default() -> Self {
Self {
max_layers: 2,
small_file_buffer: 8 * 1024, // 8KB
large_file_buffer: 64 * 1024, // 64KB
large_file_threshold: 1024 * 1024, // 1MB
}
}
}
/// BufReader optimization statistics.
#[derive(Debug, Default)]
pub struct BufReaderStats {
/// Total number of readers created
pub total_readers: AtomicU64,
/// Number of redundant layers eliminated
pub eliminated_layers: AtomicU64,
/// Number of buffer size adjustments
pub buffer_size_adjustments: AtomicU64,
}
/// BufReader layer optimizer.
///
/// Analyzes and optimizes BufReader nesting in data paths,
/// dynamically adjusting buffer sizes based on data characteristics.
pub struct BufReaderOptimizer {
config: BufReaderConfig,
stats: BufReaderStats,
}
impl BufReaderOptimizer {
/// Create a new BufReader optimizer with the given configuration.
pub fn new(config: BufReaderConfig) -> Self {
Self {
config,
stats: BufReaderStats::default(),
}
}
/// Create a new BufReader optimizer with default configuration.
pub fn with_defaults() -> Self {
Self::new(BufReaderConfig::default())
}
/// Calculate the optimal buffer size based on data size.
///
/// Returns the appropriate buffer size based on whether the data
/// is classified as a small or large file.
pub fn optimal_buffer_size(&self, data_size: Option<usize>) -> usize {
match data_size {
Some(size) if size >= self.config.large_file_threshold => self.config.large_file_buffer,
Some(_) => self.config.small_file_buffer,
None => self.config.small_file_buffer,
}
}
/// Optimize a reader by wrapping it with an appropriately sized BufReader.
///
/// This method applies the optimal buffer size based on the expected
/// data size and tracks statistics.
pub fn optimize<R: tokio::io::AsyncRead + Unpin>(&self, reader: R, data_size: Option<usize>) -> tokio::io::BufReader<R> {
let buffer_size = self.optimal_buffer_size(data_size);
self.stats.total_readers.fetch_add(1, Ordering::Relaxed);
tokio::io::BufReader::with_capacity(buffer_size, reader)
}
/// Get the statistics for this optimizer.
pub fn stats(&self) -> &BufReaderStats {
&self.stats
}
/// Get the configuration for this optimizer.
pub fn config(&self) -> &BufReaderConfig {
&self.config
}
}
/// Marker trait for buffered sources.
///
/// Types implementing this trait are considered already buffered
/// and should not be wrapped with additional BufReader layers.
pub trait BufferedSource: tokio::io::AsyncRead {}
impl BufReaderOptimizer {
/// Check if a reader is already a buffered source.
///
/// Returns true if the reader implements `BufferedSource`,
/// indicating it should not be wrapped with BufReader.
pub fn is_buffered_source<R: BufferedSource + ?Sized>(&self, _reader: &R) -> bool {
true
}
/// Eliminate redundant BufReader layers if possible.
///
/// This method attempts to reduce the nesting depth of BufReader
/// layers to improve performance.
pub fn eliminate_redundant_layers<R: tokio::io::AsyncRead + Unpin>(&self, reader: R) -> R {
// For now, just return the reader as-is
// Future implementation could detect and unwrap nested BufReaders
self.stats.eliminated_layers.fetch_add(0, Ordering::Relaxed);
reader
}
}
#[cfg(test)]
mod tests {
use super::*;
use tokio::io::AsyncReadExt;
#[test]
fn test_default_config() {
let config = BufReaderConfig::default();
assert_eq!(config.max_layers, 2);
assert_eq!(config.small_file_buffer, 8 * 1024);
assert_eq!(config.large_file_buffer, 64 * 1024);
assert_eq!(config.large_file_threshold, 1024 * 1024);
}
#[test]
fn test_optimal_buffer_size_small_file() {
let optimizer = BufReaderOptimizer::with_defaults();
// Small file (< 1MB)
assert_eq!(optimizer.optimal_buffer_size(Some(100)), 8 * 1024);
assert_eq!(optimizer.optimal_buffer_size(Some(1024)), 8 * 1024);
assert_eq!(optimizer.optimal_buffer_size(Some(512 * 1024)), 8 * 1024);
}
#[test]
fn test_optimal_buffer_size_large_file() {
let optimizer = BufReaderOptimizer::with_defaults();
// Large file (>= 1MB)
assert_eq!(optimizer.optimal_buffer_size(Some(1024 * 1024)), 64 * 1024);
assert_eq!(optimizer.optimal_buffer_size(Some(10 * 1024 * 1024)), 64 * 1024);
}
#[test]
fn test_optimal_buffer_size_unknown() {
let optimizer = BufReaderOptimizer::with_defaults();
// Unknown size
assert_eq!(optimizer.optimal_buffer_size(None), 8 * 1024);
}
#[tokio::test]
async fn test_optimize_creates_bufreader() {
let optimizer = BufReaderOptimizer::with_defaults();
let data = vec![1u8, 2, 3, 4, 5];
let cursor = std::io::Cursor::new(data.clone());
let mut reader = optimizer.optimize(cursor, Some(5));
let mut buf = vec![0u8; 5];
let n = reader.read(&mut buf).await.unwrap();
assert_eq!(n, 5);
assert_eq!(buf, data);
}
#[test]
fn test_stats_tracking() {
let optimizer = BufReaderOptimizer::with_defaults();
assert_eq!(optimizer.stats().total_readers.load(Ordering::Relaxed), 0);
let cursor = std::io::Cursor::new(vec![1u8, 2, 3]);
let _reader = optimizer.optimize(cursor, Some(3));
assert_eq!(optimizer.stats().total_readers.load(Ordering::Relaxed), 1);
}
#[test]
fn test_custom_config() {
let config = BufReaderConfig {
max_layers: 3,
small_file_buffer: 4 * 1024,
large_file_buffer: 128 * 1024,
large_file_threshold: 2 * 1024 * 1024,
};
let optimizer = BufReaderOptimizer::new(config);
assert_eq!(optimizer.optimal_buffer_size(Some(1024 * 1024)), 4 * 1024);
assert_eq!(optimizer.optimal_buffer_size(Some(3 * 1024 * 1024)), 128 * 1024);
}
}
-332
View File
@@ -1,332 +0,0 @@
// Copyright 2024 RustFS Team
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//! Aligned pread-based file reader.
//!
//! This module provides an aligned, position-based file reader that uses
//! `pread`/`FileExt::read_at` for I/O operations. It performs reads at
//! 512-byte-aligned offsets and sizes, making it suitable as a foundation
//! for workloads where alignment matters.
//!
//! Note: This reader does **not** set the `O_DIRECT` flag and therefore does
//! not bypass the OS page cache. It is an aligned `pread`-based reader, not
//! true Direct I/O. To implement true O_DIRECT on Linux, the file must be
//! opened with `O_DIRECT` via `libc::open`.
//!
//! # Platform Support
//!
//! The `read_at` implementation is only available on Unix-like platforms.
//! On other platforms, this reader will return an error.
use std::io::{self};
use std::pin::Pin;
use std::task::{Context, Poll};
use tokio::io::{AsyncRead, ReadBuf};
/// Errors that can occur during aligned pread operations.
#[derive(Debug, Clone)]
pub enum AlignedPreadError {
/// Platform doesn't support `read_at`-based I/O
UnsupportedPlatform,
/// File descriptor doesn't support this reader
UnsupportedFile,
/// I/O error occurred
Io(String),
/// Invalid alignment (reads require 512-byte-aligned offset and size)
AlignmentError { offset: u64, size: usize },
}
impl std::fmt::Display for AlignedPreadError {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
Self::UnsupportedPlatform => write!(f, "Aligned pread not supported on this platform"),
Self::UnsupportedFile => write!(f, "File doesn't support this reader"),
Self::Io(msg) => write!(f, "I/O error: {}", msg),
Self::AlignmentError { offset, size } => {
write!(f, "Alignment error: offset={}, size={}", offset, size)
}
}
}
}
impl std::error::Error for AlignedPreadError {}
impl From<io::Error> for AlignedPreadError {
fn from(err: io::Error) -> Self {
Self::Io(err.to_string())
}
}
/// Aligned pread-based file reader for Unix platforms.
///
/// This reader performs I/O using `pread`/`FileExt::read_at` at
/// 512-byte-aligned offsets and sizes, without modifying the file's
/// current position.
///
/// **Note:** This reader does **not** set the `O_DIRECT` flag and therefore
/// does **not** bypass the OS page cache. It is an aligned `pread`-based
/// reader. To implement true O_DIRECT, the file must be opened with
/// `O_DIRECT` via `libc::open`.
///
/// # Platform Support
///
/// Only available on Linux (uses `FileExt::read_at`). On other platforms,
/// use `BytesBufferedReader` instead.
///
/// # Alignment Requirements
///
/// Reads have strict alignment requirements:
/// - File offset must be aligned to 512 bytes
/// - Buffer size must be a multiple of 512 bytes
/// - Buffer address must be aligned (handled internally)
///
/// # Example
///
/// ```ignore
/// use rustfs_io_core::AlignedPreadReader;
///
/// // Linux only
/// #[cfg(target_os = "linux")]
/// let reader = AlignedPreadReader::new(file, offset, size)?;
/// ```
#[cfg(target_os = "linux")]
pub struct AlignedPreadReader {
/// Underlying file handle used for aligned pread I/O
file: std::fs::File,
/// Current read position
pos: u64,
/// Remaining bytes to read
remaining: usize,
/// Buffer for aligned reads
buffer: Vec<u8>,
/// Current position in the buffer
buffer_pos: usize,
/// Amount of data in the buffer
buffer_len: usize,
}
#[cfg(target_os = "linux")]
impl AlignedPreadReader {
/// Alignment requirement for reads (512 bytes for most systems)
pub const ALIGNMENT: usize = 512;
/// Create a new aligned pread-based reader.
///
/// # Arguments
///
/// * `file` - File to read from
/// * `offset` - Starting offset in the file (must be 512-byte aligned)
/// * `size` - Number of bytes to read (must be 512-byte aligned)
///
/// # Returns
///
/// An `AlignedPreadReader` that reads the file at the given offset.
///
/// # Errors
///
/// Returns an error if offset or size are not 512-byte aligned.
pub fn new(file: std::fs::File, offset: u64, size: usize) -> Result<Self, AlignedPreadError> {
// Check alignment
if !offset.is_multiple_of(Self::ALIGNMENT as u64) {
return Err(AlignedPreadError::AlignmentError { offset, size });
}
if !size.is_multiple_of(Self::ALIGNMENT) {
return Err(AlignedPreadError::AlignmentError { offset, size });
}
Ok(Self {
file,
pos: offset,
remaining: size,
buffer: Vec::new(),
buffer_pos: 0,
buffer_len: 0,
})
}
/// Read a chunk of data using aligned pread.
///
/// This method performs aligned reads and handles the buffering required
/// by this aligned pread implementation. It does not use `O_DIRECT`.
fn read_chunk(&mut self, buf: &mut [u8]) -> io::Result<usize> {
// If buffer is exhausted, read more data
if self.buffer_pos >= self.buffer_len {
if self.remaining == 0 {
return Ok(0);
}
// Allocate aligned buffer
let chunk_size = (self.remaining).min(64 * 1024); // 64KB chunks
let aligned_size = chunk_size.div_ceil(Self::ALIGNMENT) * Self::ALIGNMENT;
self.buffer = vec![0u8; aligned_size];
// Use pread for atomic read at position (no file offset modification)
use std::os::unix::fs::FileExt;
let n = self.file.read_at(&mut self.buffer, self.pos)?;
self.buffer_pos = 0;
self.buffer_len = n;
self.pos += n as u64;
self.remaining -= n;
if n == 0 {
return Ok(0);
}
}
// Copy from buffer to user buffer
let available = self.buffer_len - self.buffer_pos;
let to_copy = buf.len().min(available);
buf[..to_copy].copy_from_slice(&self.buffer[self.buffer_pos..self.buffer_pos + to_copy]);
self.buffer_pos += to_copy;
Ok(to_copy)
}
}
#[cfg(target_os = "linux")]
impl AsyncRead for AlignedPreadReader {
fn poll_read(mut self: Pin<&mut Self>, _cx: &mut Context<'_>, buf: &mut ReadBuf<'_>) -> Poll<io::Result<()>> {
let filled = buf.filled().len();
let mut remaining = buf.initialize_unfilled();
while !remaining.is_empty() {
match self.read_chunk(remaining) {
Ok(0) => break,
Ok(n) => {
remaining = &mut remaining[n..];
}
Err(e) => return Poll::Ready(Err(e)),
}
}
let _n_read = buf.filled().len() - filled;
Poll::Ready(Ok(()))
}
}
/// Aligned pread reader stub for non-Linux platforms.
///
/// On non-Linux platforms, `read_at`-based I/O is not available through this
/// type. This stub exists to provide a consistent API across platforms.
#[cfg(not(target_os = "linux"))]
pub struct AlignedPreadReader {
_priv: (),
}
#[cfg(not(target_os = "linux"))]
impl AlignedPreadReader {
/// Create a new aligned pread reader (not supported on this platform).
///
/// Always returns an error on non-Linux platforms.
pub fn new(_file: std::fs::File, _offset: u64, _size: usize) -> Result<Self, AlignedPreadError> {
Err(AlignedPreadError::UnsupportedPlatform)
}
}
#[cfg(not(target_os = "linux"))]
impl AsyncRead for AlignedPreadReader {
fn poll_read(self: Pin<&mut Self>, _cx: &mut Context<'_>, _buf: &mut ReadBuf<'_>) -> Poll<io::Result<()>> {
Poll::Ready(Err(io::Error::new(
io::ErrorKind::Unsupported,
"Aligned pread-based I/O not supported on this platform",
)))
}
}
impl std::fmt::Debug for AlignedPreadReader {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
#[cfg(target_os = "linux")]
{
f.debug_struct("AlignedPreadReader")
.field("pos", &self.pos)
.field("remaining", &self.remaining)
.field("buffer_len", &self.buffer_len)
.finish()
}
#[cfg(not(target_os = "linux"))]
{
f.debug_struct("AlignedPreadReader")
.field("platform", &"unsupported")
.finish()
}
}
}
/// Historical name for aligned pread errors.
#[deprecated(since = "1.0.0-beta.8", note = "use AlignedPreadError; this reader does not set O_DIRECT")]
pub type DirectIoError = AlignedPreadError;
/// Historical name for the aligned pread-based reader.
#[deprecated(since = "1.0.0-beta.8", note = "use AlignedPreadReader; this reader does not set O_DIRECT")]
pub type DirectIoReader = AlignedPreadReader;
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_alignment_check() {
#[cfg(target_os = "linux")]
{
// Valid alignment
let file = std::fs::File::open("/dev/zero").unwrap();
assert!(
AlignedPreadReader::new(file, 0, 512).is_ok(),
"Should succeed with aligned offset and size"
);
let file = std::fs::File::open("/dev/zero").expect("open /dev/zero for alias");
assert!(
AlignedPreadReader::new(file, 0, 512).is_ok(),
"Should succeed through aligned pread alias"
);
// Invalid offset
let file = std::fs::File::open("/dev/zero").unwrap();
assert!(AlignedPreadReader::new(file, 1, 512).is_err(), "Should fail with unaligned offset");
// Invalid size
let file = std::fs::File::open("/dev/zero").unwrap();
assert!(AlignedPreadReader::new(file, 0, 511).is_err(), "Should fail with unaligned size");
}
#[cfg(not(target_os = "linux"))]
{
// Non-Linux should return UnsupportedPlatform
let file = std::fs::File::open(std::env::current_exe().unwrap()).unwrap();
assert!(matches!(
AlignedPreadReader::new(file, 0, 512),
Err(AlignedPreadError::UnsupportedPlatform)
));
}
}
#[test]
#[allow(deprecated)]
fn test_legacy_direct_io_alias() {
#[cfg(target_os = "linux")]
{
let file = std::fs::File::open("/dev/zero").unwrap();
assert!(DirectIoReader::new(file, 0, 512).is_ok());
}
#[cfg(not(target_os = "linux"))]
{
let file = std::fs::File::open(std::env::current_exe().unwrap()).unwrap();
assert!(matches!(DirectIoReader::new(file, 0, 512), Err(AlignedPreadError::UnsupportedPlatform)));
}
}
}
-381
View File
@@ -1,381 +0,0 @@
// Copyright 2024 RustFS Team
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//! I/O priority queue for scheduling I/O operations.
//!
//! This module provides a priority queue implementation for I/O operations
//! with support for starvation prevention and fair scheduling.
use crate::config::IoPriorityQueueConfig;
use crate::scheduler::IoPriority;
use std::collections::VecDeque;
use std::time::{Duration, Instant};
/// A queued I/O request.
#[derive(Debug, Clone)]
pub struct IoRequest {
/// Request ID.
pub id: u64,
/// Request priority.
pub priority: IoPriority,
/// Request size in bytes.
pub size: usize,
/// Queue time.
pub queued_at: Instant,
/// Whether this is a sequential read.
pub is_sequential: bool,
}
impl IoRequest {
/// Create a new I/O request.
pub fn new(id: u64, priority: IoPriority, size: usize, is_sequential: bool) -> Self {
Self {
id,
priority,
size,
queued_at: Instant::now(),
is_sequential,
}
}
/// Get the wait time in the queue.
pub fn wait_time(&self) -> Duration {
self.queued_at.elapsed()
}
}
/// Queue status for a priority level.
#[derive(Debug, Clone, Default)]
pub struct IoQueueStatus {
/// Number of requests in the queue.
pub count: usize,
/// Total size of all requests.
pub total_size: usize,
/// Oldest request wait time.
pub oldest_wait: Option<Duration>,
/// Number of requests processed.
pub processed: u64,
}
impl IoQueueStatus {
/// Create new queue status.
pub fn new() -> Self {
Self::default()
}
}
/// I/O priority queue.
pub struct IoPriorityQueue {
/// Queue configuration.
config: IoPriorityQueueConfig,
/// High priority queue.
high: VecDeque<IoRequest>,
/// Normal priority queue.
normal: VecDeque<IoRequest>,
/// Low priority queue.
low: VecDeque<IoRequest>,
/// Next request ID.
next_id: u64,
/// Last dequeue time for each priority (for starvation prevention).
last_dequeue: [Option<Instant>; 3],
/// Statistics for each queue.
stats: [IoQueueStatus; 3],
}
impl IoPriorityQueue {
/// Create a new priority queue with the given configuration.
pub fn new(config: IoPriorityQueueConfig) -> Self {
Self {
config,
high: VecDeque::with_capacity(100),
normal: VecDeque::with_capacity(500),
low: VecDeque::with_capacity(200),
next_id: 0,
last_dequeue: [None, None, None],
stats: [IoQueueStatus::new(), IoQueueStatus::new(), IoQueueStatus::new()],
}
}
/// Create with default configuration.
pub fn with_defaults() -> Self {
Self::new(IoPriorityQueueConfig::default())
}
/// Get the configuration.
pub fn config(&self) -> &IoPriorityQueueConfig {
&self.config
}
/// Enqueue a request.
pub fn enqueue(&mut self, priority: IoPriority, size: usize, is_sequential: bool) -> u64 {
let id = self.next_id;
self.next_id += 1;
let request = IoRequest::new(id, priority, size, is_sequential);
match priority {
IoPriority::High => {
if self.high.len() < self.config.high_capacity {
self.high.push_back(request);
}
}
IoPriority::Normal => {
if self.normal.len() < self.config.normal_capacity {
self.normal.push_back(request);
}
}
IoPriority::Low => {
if self.low.len() < self.config.low_capacity {
self.low.push_back(request);
}
}
}
id
}
/// Dequeue the next request.
///
/// Uses weighted fair queuing with starvation prevention.
pub fn dequeue(&mut self) -> Option<IoRequest> {
let now = Instant::now();
// Check for starvation: if a lower priority queue hasn't been served in a while,
// give it priority
let normal_starved = self.is_starved(IoPriority::Normal, now);
let low_starved = self.is_starved(IoPriority::Low, now);
// Priority order with starvation consideration
// Check conditions first, then dequeue
let dequeue_high = !self.high.is_empty() && !low_starved && !normal_starved;
let dequeue_normal = !self.normal.is_empty() && !low_starved;
let dequeue_low = !self.low.is_empty();
let dequeue_high_fallback = !self.high.is_empty();
let dequeue_normal_fallback = !self.normal.is_empty();
if dequeue_high {
let request = self.high.pop_front();
if request.is_some() {
self.last_dequeue[0] = Some(Instant::now());
self.stats[0].processed += 1;
}
request
} else if dequeue_normal {
let request = self.normal.pop_front();
if request.is_some() {
self.last_dequeue[1] = Some(Instant::now());
self.stats[1].processed += 1;
}
request
} else if dequeue_low {
let request = self.low.pop_front();
if request.is_some() {
self.last_dequeue[2] = Some(Instant::now());
self.stats[2].processed += 1;
}
request
} else if dequeue_high_fallback {
let request = self.high.pop_front();
if request.is_some() {
self.last_dequeue[0] = Some(Instant::now());
self.stats[0].processed += 1;
}
request
} else if dequeue_normal_fallback {
let request = self.normal.pop_front();
if request.is_some() {
self.last_dequeue[1] = Some(Instant::now());
self.stats[1].processed += 1;
}
request
} else {
None
}
}
/// Check if a priority level is starved.
fn is_starved(&self, priority: IoPriority, now: Instant) -> bool {
let idx = match priority {
IoPriority::High => 0,
IoPriority::Normal => 1,
IoPriority::Low => 2,
};
if let Some(last) = self.last_dequeue[idx] {
now.duration_since(last) > self.config.starvation_threshold
} else {
false
}
}
/// Get the total number of queued requests.
pub fn len(&self) -> usize {
self.high.len() + self.normal.len() + self.low.len()
}
/// Check if the queue is empty.
pub fn is_empty(&self) -> bool {
self.high.is_empty() && self.normal.is_empty() && self.low.is_empty()
}
/// Get queue status for a priority level.
pub fn status(&self, priority: IoPriority) -> IoQueueStatus {
let (queue, idx) = match priority {
IoPriority::High => (&self.high, 0),
IoPriority::Normal => (&self.normal, 1),
IoPriority::Low => (&self.low, 2),
};
let mut status = self.stats[idx].clone();
status.count = queue.len();
status.total_size = queue.iter().map(|r| r.size).sum();
status.oldest_wait = queue.front().map(|r| r.wait_time());
status
}
/// Get the total queue status.
pub fn total_status(&self) -> IoQueueStatus {
let mut total = IoQueueStatus::new();
total.count = self.len();
total.total_size = self
.high
.iter()
.chain(self.normal.iter())
.chain(self.low.iter())
.map(|r| r.size)
.sum();
total.processed = self.stats.iter().map(|s| s.processed).sum();
total.oldest_wait = self
.high
.front()
.map(|r| r.wait_time())
.or_else(|| self.normal.front().map(|r| r.wait_time()))
.or_else(|| self.low.front().map(|r| r.wait_time()));
total
}
/// Clear all queues.
pub fn clear(&mut self) {
self.high.clear();
self.normal.clear();
self.low.clear();
}
/// Peek at the next request without removing it.
pub fn peek(&self) -> Option<&IoRequest> {
if !self.high.is_empty() {
self.high.front()
} else if !self.normal.is_empty() {
self.normal.front()
} else {
self.low.front()
}
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_enqueue_dequeue() {
let mut queue = IoPriorityQueue::with_defaults();
let id1 = queue.enqueue(IoPriority::High, 1024, true);
let id2 = queue.enqueue(IoPriority::Normal, 2048, false);
let id3 = queue.enqueue(IoPriority::Low, 4096, true);
assert_eq!(queue.len(), 3);
// High priority should be dequeued first
let req1 = queue.dequeue().unwrap();
assert_eq!(req1.id, id1);
assert_eq!(req1.priority, IoPriority::High);
let req2 = queue.dequeue().unwrap();
assert_eq!(req2.id, id2);
assert_eq!(req2.priority, IoPriority::Normal);
let req3 = queue.dequeue().unwrap();
assert_eq!(req3.id, id3);
assert_eq!(req3.priority, IoPriority::Low);
assert!(queue.is_empty());
}
#[test]
fn test_queue_status() {
let mut queue = IoPriorityQueue::with_defaults();
queue.enqueue(IoPriority::High, 1024, true);
queue.enqueue(IoPriority::High, 2048, true);
queue.enqueue(IoPriority::Normal, 4096, false);
let high_status = queue.status(IoPriority::High);
assert_eq!(high_status.count, 2);
assert_eq!(high_status.total_size, 3072);
let normal_status = queue.status(IoPriority::Normal);
assert_eq!(normal_status.count, 1);
assert_eq!(normal_status.total_size, 4096);
let total = queue.total_status();
assert_eq!(total.count, 3);
assert_eq!(total.total_size, 7168);
}
#[test]
fn test_queue_capacity() {
let config = IoPriorityQueueConfig {
high_capacity: 2,
normal_capacity: 2,
low_capacity: 2,
..Default::default()
};
let mut queue = IoPriorityQueue::new(config);
queue.enqueue(IoPriority::High, 1024, true);
queue.enqueue(IoPriority::High, 1024, true);
queue.enqueue(IoPriority::High, 1024, true); // Should be dropped
assert_eq!(queue.status(IoPriority::High).count, 2);
}
#[test]
fn test_clear() {
let mut queue = IoPriorityQueue::with_defaults();
queue.enqueue(IoPriority::High, 1024, true);
queue.enqueue(IoPriority::Normal, 2048, false);
queue.enqueue(IoPriority::Low, 4096, true);
assert_eq!(queue.len(), 3);
queue.clear();
assert!(queue.is_empty());
}
#[test]
fn test_peek() {
let mut queue = IoPriorityQueue::with_defaults();
queue.enqueue(IoPriority::Normal, 2048, false);
queue.enqueue(IoPriority::High, 1024, true);
let peeked = queue.peek().unwrap();
assert_eq!(peeked.priority, IoPriority::High);
// Peek shouldn't remove the item
assert_eq!(queue.len(), 2);
}
}
-6
View File
@@ -26,7 +26,6 @@ pub enum StorageMedia {
}
impl StorageMedia {
#[allow(dead_code)]
pub fn as_str(&self) -> &'static str {
match self {
Self::Nvme => "nvme",
@@ -60,7 +59,6 @@ pub enum AccessPattern {
}
impl AccessPattern {
#[allow(dead_code)]
pub fn as_str(&self) -> &'static str {
match self {
Self::Sequential => "sequential",
@@ -71,25 +69,21 @@ impl AccessPattern {
}
/// Check if this is a sequential access pattern.
#[allow(dead_code)]
pub fn is_sequential(&self) -> bool {
matches!(self, Self::Sequential)
}
/// Check if this is a random access pattern.
#[allow(dead_code)]
pub fn is_random(&self) -> bool {
matches!(self, Self::Random)
}
/// Check if this is a mixed access pattern.
#[allow(dead_code)]
pub fn is_mixed(&self) -> bool {
matches!(self, Self::Mixed)
}
/// Check if this pattern is unknown.
#[allow(dead_code)]
pub fn is_unknown(&self) -> bool {
matches!(self, Self::Unknown)
}
+12 -61
View File
@@ -12,85 +12,39 @@
// See the License for the specific language governing permissions and
// limitations under the License.
//! Buffered I/O reader and writer implementations for RustFS.
//! Shared I/O primitives for RustFS.
//!
//! This crate provides buffered readers and writers for I/O operations.
//! Prefer `BytesBufferedReader`, `BytesMutWriter`, and `AlignedPreadReader`
//! for new code. Historical `ZeroCopy*` and `DirectIo*` names remain exported
//! for backward compatibility.
//! This crate holds the buffer pool and the concurrency-control primitives
//! that the storage layer builds on:
//!
//! # Features
//!
//! - Memory-mapped file reading (mmap-then-copy) on Unix platforms
//! - Bytes-based buffered wrapping
//! - AsyncRead trait implementations
//! - Tiered BytesPool for buffer management
//! - Aligned pread-based reader (NOT true Direct I/O / O_DIRECT)
//! - Tiered `BytesPool` for buffer management
//! - Storage-media and access-pattern profiling (`io_profile`)
//! - Scheduler and priority-queue configuration shapes
//! - Backpressure admission, deadlock detection, lock optimization
//! - Progress tracking for long-running operations
//!
//! # Example
//!
//! ```ignore
//! use rustfs_io_core::{BytesBufferedReader, BytesPool};
//! use bytes::Bytes;
//! use rustfs_io_core::BytesPool;
//!
//! // Create from existing bytes (zero-copy)
//! let data = Bytes::from("hello world");
//! let reader = BytesBufferedReader::from_bytes(data);
//!
//! // Create from file using buffered reads
//! let reader = BytesBufferedReader::from_file_read(&file, 0, 1024).await?;
//!
//! // Use BytesPool
//! let pool = BytesPool::new_tiered();
//! let mut buffer = pool.acquire_buffer(8192).await;
//! ```
pub mod backpressure;
pub mod bufreader_optimizer;
pub mod config;
pub mod deadlock_detector;
pub mod direct_io;
pub mod io_priority_queue;
pub mod io_profile;
pub mod lock_optimizer;
pub mod pool;
pub mod reader;
pub mod scheduler;
pub mod shared_memory;
pub mod timeout_wrapper;
pub mod writer;
pub mod progress;
#[cfg(target_os = "linux")]
pub use direct_io::{AlignedPreadError, AlignedPreadReader};
#[cfg(target_os = "linux")]
#[allow(deprecated)]
pub use direct_io::{DirectIoError, DirectIoReader};
pub use pool::{BytesPool, BytesPoolConfig, BytesPoolMetrics, PooledBuffer};
#[allow(deprecated)]
pub use reader::ZeroCopyObjectReader;
pub use reader::{BytesBufferedReader, ZeroCopyReadError};
#[allow(deprecated)]
pub use writer::ZeroCopyObjectWriter;
pub use writer::{BytesMutWriter, ZeroCopyWriteError};
// BufReader optimizer exports
pub use bufreader_optimizer::{BufReaderConfig, BufReaderOptimizer, BufReaderStats, BufferedSource};
// Shared memory exports
pub use shared_memory::{ArcData, ArcMetadata, SharedMemoryConfig, SharedMemoryPool, SharedMemoryStats};
// Config exports
pub use config::{ConfigError, IoPriorityQueueConfig, IoSchedulerConfig};
// Scheduler exports
pub use scheduler::{
BandwidthTier, IoLoadLevel, IoLoadMetrics, IoPriority, IoScheduler, IoSchedulingContext, IoStrategy, KI_B, MI_B,
calculate_optimal_buffer_size, get_advanced_buffer_size, get_buffer_size_for_media, get_concurrency_aware_buffer_size,
};
// Priority queue exports
pub use io_priority_queue::{IoPriorityQueue, IoQueueStatus, IoRequest};
// Backpressure exports
pub use backpressure::{BackpressureConfig, BackpressureError, BackpressureMonitor, BackpressureState};
@@ -100,8 +54,5 @@ pub use deadlock_detector::{DeadlockDetector, DeadlockDetectorConfig, LockInfo,
// Lock optimizer exports
pub use lock_optimizer::{LockGuard, LockOptimizeConfig, LockOptimizer, LockStats};
// Timeout wrapper exports
pub use timeout_wrapper::{
OperationProgress, RequestTimeoutWrapper, TimeoutConfig, TimeoutError, TimeoutStats, calculate_adaptive_timeout,
estimate_bytes_per_second,
};
// Progress tracking exports
pub use progress::OperationProgress;
+138
View File
@@ -0,0 +1,138 @@
// Copyright 2024 RustFS Team
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//! Progress tracking for long-running I/O operations.
//!
//! Re-exported as `rustfs_concurrency::OperationProgress` for the storage
//! timeout implementation, which uses `is_stale` to tell a slow transfer
//! apart from a stalled one.
use std::sync::atomic::{AtomicU64, Ordering};
use std::time::{Duration, Instant};
/// Operation progress tracker.
#[derive(Debug)]
pub struct OperationProgress {
/// Total size (if known).
pub total_size: Option<u64>,
/// Bytes processed.
bytes_processed: AtomicU64,
/// Last update time.
last_update: std::sync::Mutex<Instant>,
/// Stale timeout.
stale_timeout: Duration,
/// Start time for transfer rate calculation.
start_time: Instant,
}
impl OperationProgress {
/// Create new operation progress.
pub fn new(total_size: Option<u64>, stale_timeout: Duration) -> Self {
Self {
total_size,
bytes_processed: AtomicU64::new(0),
last_update: std::sync::Mutex::new(Instant::now()),
stale_timeout,
start_time: Instant::now(),
}
}
/// Update progress.
pub fn update(&self, bytes: u64) {
self.bytes_processed.store(bytes, Ordering::Relaxed);
if let Ok(mut last) = self.last_update.lock() {
*last = Instant::now();
}
}
/// Add to progress.
pub fn add(&self, bytes: u64) {
self.bytes_processed.fetch_add(bytes, Ordering::Relaxed);
if let Ok(mut last) = self.last_update.lock() {
*last = Instant::now();
}
}
/// Get current progress.
pub fn current(&self) -> u64 {
self.bytes_processed.load(Ordering::Relaxed)
}
/// Check if progress is stale.
pub fn is_stale(&self) -> bool {
if let Ok(last) = self.last_update.lock() {
last.elapsed() > self.stale_timeout
} else {
false
}
}
/// Get progress percentage.
pub fn progress_percent(&self) -> Option<f64> {
self.total_size.map(|total| {
if total == 0 {
100.0
} else {
let processed = self.bytes_processed.load(Ordering::Relaxed);
(processed as f64 / total as f64 * 100.0).min(100.0)
}
})
}
/// Get remaining bytes.
pub fn remaining(&self) -> Option<u64> {
self.total_size.map(|total| {
let processed = self.bytes_processed.load(Ordering::Relaxed);
total.saturating_sub(processed)
})
}
/// Calculate transfer rate in bytes per second.
///
/// Returns 0 if no time has elapsed or no data transferred.
pub fn transfer_rate(&self) -> u64 {
let processed = self.bytes_processed.load(Ordering::Relaxed);
if processed == 0 {
return 0;
}
let elapsed = self.start_time.elapsed().as_secs_f64();
if elapsed > 0.0 {
(processed as f64 / elapsed) as u64
} else {
0
}
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_operation_progress() {
let progress = OperationProgress::new(Some(1000), Duration::from_secs(5));
assert_eq!(progress.current(), 0);
assert_eq!(progress.progress_percent(), Some(0.0));
progress.update(500);
assert_eq!(progress.current(), 500);
assert_eq!(progress.progress_percent(), Some(50.0));
progress.add(300);
assert_eq!(progress.current(), 800);
assert_eq!(progress.remaining(), Some(200));
}
}
-412
View File
@@ -1,412 +0,0 @@
// Copyright 2024 RustFS Team
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//! Bytes-backed object reader implementation.
use bytes::Bytes;
use std::io;
use std::pin::Pin;
use std::task::{Context, Poll};
use tokio::io::{AsyncRead, ReadBuf};
/// Errors that can occur during Bytes-backed read operations.
#[derive(Debug, Clone)]
pub enum ZeroCopyReadError {
/// I/O error occurred.
Io(String),
/// Memory mapping error.
Mmap(String),
/// Invalid offset or size.
InvalidRange,
}
impl std::fmt::Display for ZeroCopyReadError {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
Self::Io(msg) => write!(f, "I/O error: {}", msg),
Self::Mmap(msg) => write!(f, "Mmap error: {}", msg),
Self::InvalidRange => write!(f, "Invalid offset or size"),
}
}
}
impl std::error::Error for ZeroCopyReadError {}
impl From<io::Error> for ZeroCopyReadError {
fn from(err: io::Error) -> Self {
Self::Io(err.to_string())
}
}
/// Bytes-backed object reader.
///
/// `from_bytes` wraps existing `Bytes` without copying, but file constructors
/// copy file data into owned `Bytes` after mmap or normal reads.
///
/// # Example
///
/// ```ignore
/// use bytes::Bytes;
/// use rustfs_io_core::BytesBufferedReader;
///
/// // Create from bytes without copying the `Bytes` buffer
/// let data = Bytes::from("hello world");
/// let reader = BytesBufferedReader::from_bytes(data);
///
/// // Read using AsyncRead trait
/// let mut buf = vec![0u8; 1024];
/// let n = reader.read(&mut buf[..]).await?;
/// ```
pub struct BytesBufferedReader {
/// Internal data source (could be mmap or owned bytes)
data: Bytes,
/// Current read position
pos: usize,
}
/// Historical name for the bytes-backed object reader.
#[deprecated(
since = "1.0.0-beta.8",
note = "use BytesBufferedReader; file constructors copy into owned Bytes"
)]
pub type ZeroCopyObjectReader = BytesBufferedReader;
impl BytesBufferedReader {
/// Create a reader from existing bytes.
///
/// This is a true zero-copy operation - the Bytes are wrapped
/// without any allocation or copying.
///
/// # Arguments
///
/// * `data` - Bytes to wrap
///
/// # Example
///
/// ```ignore
/// let data = Bytes::from("hello world");
/// let reader = BytesBufferedReader::from_bytes(data);
/// ```
pub fn from_bytes(data: Bytes) -> Self {
Self { data, pos: 0 }
}
/// Create a Bytes-backed reader from a file using mmap-then-copy.
///
/// This maps the requested file range and copies it into owned `Bytes`
/// before returning. It does not expose the mmap as a zero-copy buffer.
///
/// # Arguments
///
/// * `path` - Path to the file to memory map
/// * `offset` - Offset within the file to start reading
/// * `size` - Number of bytes to read
///
/// # Returns
///
/// A reader backed by copied file data.
///
/// # Errors
///
/// Returns an error if the file cannot be memory mapped.
///
/// # Example
///
/// ```ignore
/// let reader = BytesBufferedReader::from_file_mmap_path("large_file.bin", 0, 1024).await?;
/// ```
#[cfg(unix)]
// SAFETY: The mmap is created from a read-only file handle for the
// caller-provided range, then copied into owned `Bytes` before the file and
// mapping are dropped.
#[allow(unsafe_code)]
pub async fn from_file_mmap_path(path: &std::path::Path, offset: u64, size: usize) -> Result<Self, ZeroCopyReadError> {
use memmap2::MmapOptions;
let path = path.to_path_buf();
let (offset, size) = (offset, size);
tokio::task::spawn_blocking(move || {
// Open the file in sync context
let std_file = std::fs::File::open(&path).map_err(|e| ZeroCopyReadError::Io(e.to_string()))?;
// SAFETY: `std_file` remains open while the mapping is created and
// copied, and the mapped bytes are not exposed beyond this closure.
let mmap = unsafe { MmapOptions::new().offset(offset).len(size).map(&std_file) }
.map_err(|e| ZeroCopyReadError::Mmap(e.to_string()))?;
// Convert to Bytes (this is a copy, but only done once)
Ok(Self {
data: Bytes::copy_from_slice(&mmap),
pos: 0,
})
})
.await
.map_err(|e| ZeroCopyReadError::Io(e.to_string()))?
}
/// Create a Bytes-backed reader from a file using normal reads.
///
/// This path reads the requested range into an owned buffer and wraps it in
/// `Bytes`. It does not perform mmap or zero-copy file I/O.
///
/// # Arguments
///
/// * `file` - File to read from
/// * `offset` - Offset within the file to start reading
/// * `size` - Number of bytes to map
///
/// # Returns
///
/// A reader backed by copied file data.
///
/// # Errors
///
/// Returns an error if the file cannot be read.
///
/// # Example
///
/// ```ignore
/// let file = tokio::fs::File::open("large_file.bin").await?;
/// let reader = BytesBufferedReader::from_file_read(&file, 0, 1024).await?;
/// ```
#[cfg(unix)]
pub async fn from_file_read(file: &tokio::fs::File, offset: u64, size: usize) -> Result<Self, ZeroCopyReadError> {
use tokio::io::{AsyncReadExt, AsyncSeekExt, SeekFrom};
let mut cloned = file.try_clone().await?;
cloned.seek(SeekFrom::Start(offset)).await?;
let mut buffer = vec![0u8; size];
cloned.read_exact(&mut buffer).await?;
Ok(Self {
data: Bytes::from(buffer),
pos: 0,
})
}
/// Create a Bytes-backed reader from a file (non-Unix fallback).
///
/// On platforms that don't support mmap, this falls back to regular file I/O.
#[cfg(not(unix))]
pub async fn from_file_read(file: &tokio::fs::File, offset: u64, size: usize) -> Result<Self, ZeroCopyReadError> {
use tokio::io::{AsyncReadExt, AsyncSeekExt, SeekFrom};
let mut cloned = file.try_clone().await?;
cloned.seek(SeekFrom::Start(offset)).await?;
let mut buffer = vec![0u8; size];
cloned.read_exact(&mut buffer).await?;
Ok(Self {
data: Bytes::from(buffer),
pos: 0,
})
}
/// Historical name for `from_file_read`.
#[deprecated(
since = "1.0.0-beta.8",
note = "use from_file_read; this method performs normal reads into owned Bytes"
)]
pub async fn from_file_mmap(file: &tokio::fs::File, offset: u64, size: usize) -> Result<Self, ZeroCopyReadError> {
Self::from_file_read(file, offset, size).await
}
/// Get the remaining data as Bytes (zero-copy).
///
/// This returns a slice of the remaining data without copying.
/// The returned Bytes shares the underlying memory with this reader.
///
/// # Example
///
/// ```ignore
/// let remaining = reader.remaining_bytes();
/// println!("Remaining: {} bytes", remaining.len());
/// ```
pub fn remaining_bytes(&self) -> Bytes {
self.data.slice(self.pos..)
}
/// Get the total length of the data.
pub fn len(&self) -> usize {
self.data.len()
}
/// Check if the reader has reached the end.
pub fn is_empty(&self) -> bool {
self.pos >= self.data.len()
}
/// Get the current read position.
pub fn position(&self) -> usize {
self.pos
}
}
impl AsyncRead for BytesBufferedReader {
fn poll_read(mut self: Pin<&mut Self>, _cx: &mut Context<'_>, buf: &mut ReadBuf<'_>) -> Poll<io::Result<()>> {
let remaining = self.data.len() - self.pos;
if remaining == 0 {
return Poll::Ready(Ok(()));
}
let to_read = std::cmp::min(remaining, buf.remaining());
let slice = &self.data[self.pos..self.pos + to_read];
buf.put_slice(slice);
self.pos += to_read;
Poll::Ready(Ok(()))
}
}
impl std::fmt::Debug for BytesBufferedReader {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("BytesBufferedReader")
.field("data_len", &self.data.len())
.field("pos", &self.pos)
.field("remaining", &(self.data.len() - self.pos))
.finish()
}
}
#[cfg(test)]
mod tests {
use super::*;
use std::path::PathBuf;
use tokio::io::AsyncReadExt;
fn temp_file_path(test_name: &str) -> PathBuf {
let nonce = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.expect("system time should be after unix epoch")
.as_nanos();
std::env::temp_dir().join(format!("rustfs-io-core-{test_name}-{}-{nonce}", std::process::id()))
}
#[tokio::test]
async fn test_from_bytes() {
let data = Bytes::from("hello world");
let mut reader = BytesBufferedReader::from_bytes(data.clone());
let mut buf = [0u8; 11];
let n = reader.read(&mut buf[..]).await.unwrap();
assert_eq!(n, 11);
assert_eq!(&buf[..n], b"hello world");
}
#[tokio::test]
async fn test_preferred_reader_alias() {
let data = Bytes::from("hello world");
let mut reader = BytesBufferedReader::from_bytes(data);
let mut buf = [0u8; 5];
let n = reader.read(&mut buf[..]).await.expect("read bytes from alias");
assert_eq!(n, 5);
assert_eq!(&buf[..n], b"hello");
}
#[tokio::test]
async fn test_from_file_read_reads_requested_range() {
let path = temp_file_path("from-file-read");
tokio::fs::write(&path, b"hello world")
.await
.expect("write temp file for reader test");
let file = tokio::fs::File::open(&path).await.expect("open temp file for reader test");
let mut reader = BytesBufferedReader::from_file_read(&file, 6, 5)
.await
.expect("read requested range into Bytes");
let mut output = Vec::new();
reader.read_to_end(&mut output).await.expect("drain reader output");
assert_eq!(output, b"world");
let _ = tokio::fs::remove_file(path).await;
}
#[tokio::test]
#[allow(deprecated)]
async fn test_from_file_mmap_legacy_alias_reads_requested_range() {
let path = temp_file_path("from-file-mmap");
tokio::fs::write(&path, b"hello world")
.await
.expect("write temp file for legacy reader test");
let file = tokio::fs::File::open(&path)
.await
.expect("open temp file for legacy reader test");
let mut reader = BytesBufferedReader::from_file_mmap(&file, 0, 5)
.await
.expect("read requested range through legacy alias");
let mut output = Vec::new();
reader.read_to_end(&mut output).await.expect("drain legacy reader output");
assert_eq!(output, b"hello");
let _ = tokio::fs::remove_file(path).await;
}
#[tokio::test]
async fn test_remaining_bytes() {
let data = Bytes::from("hello world");
let reader = BytesBufferedReader::from_bytes(data);
let remaining = reader.remaining_bytes();
assert_eq!(remaining.len(), 11);
assert_eq!(&remaining[..], b"hello world");
}
#[tokio::test]
async fn test_position() {
let data = Bytes::from("hello world");
let mut reader = BytesBufferedReader::from_bytes(data);
assert_eq!(reader.position(), 0);
let mut buf = [0u8; 5];
reader.read_exact(&mut buf[..]).await.unwrap();
assert_eq!(reader.position(), 5);
}
#[tokio::test]
async fn test_is_empty() {
let data = Bytes::from("");
let reader = BytesBufferedReader::from_bytes(data);
assert!(reader.is_empty());
let data = Bytes::from("hello");
let reader = BytesBufferedReader::from_bytes(data);
assert!(!reader.is_empty());
}
#[tokio::test]
#[allow(deprecated)]
async fn test_legacy_reader_alias() {
let data = Bytes::from("hello world");
let mut reader = ZeroCopyObjectReader::from_bytes(data);
let mut buf = [0u8; 5];
let n = reader.read(&mut buf[..]).await.expect("read bytes through legacy alias");
assert_eq!(n, 5);
assert_eq!(&buf[..n], b"hello");
}
}
-882
View File
@@ -1,882 +0,0 @@
// Copyright 2024 RustFS Team
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//! I/O scheduler for adaptive buffer sizing and load management.
//!
//! This module provides the core I/O scheduling logic that determines
//! optimal buffer sizes, I/O strategies, and load management decisions.
use crate::config::IoSchedulerConfig;
use crate::io_profile::{AccessPattern, StorageMedia, StorageProfile};
use std::sync::atomic::{AtomicUsize, Ordering};
use std::time::Duration;
/// I/O priority levels.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, Default)]
pub enum IoPriority {
/// High priority for small, latency-sensitive operations.
High,
/// Normal priority for standard operations.
#[default]
Normal,
/// Low priority for large, throughput-oriented operations.
Low,
}
impl IoPriority {
/// Determine priority based on request size.
///
/// A negative `size` means the size is unknown (-1 by convention) and maps
/// to `Normal`; casting it to `usize` would wrap to a huge value and
/// misclassify the request as `Low`.
pub fn from_size(size: i64, high_threshold: usize, low_threshold: usize) -> Self {
if size < 0 {
return IoPriority::Normal;
}
let size = size as usize;
if size < high_threshold {
IoPriority::High
} else if size > low_threshold {
IoPriority::Low
} else {
IoPriority::Normal
}
}
/// Get the priority as a string for metrics labels.
pub fn as_str(&self) -> &'static str {
match self {
IoPriority::High => "high",
IoPriority::Normal => "normal",
IoPriority::Low => "low",
}
}
/// Check if this is high priority.
pub fn is_high(&self) -> bool {
matches!(self, IoPriority::High)
}
/// Check if this is normal priority.
pub fn is_normal(&self) -> bool {
matches!(self, IoPriority::Normal)
}
/// Check if this is low priority.
pub fn is_low(&self) -> bool {
matches!(self, IoPriority::Low)
}
}
impl std::fmt::Display for IoPriority {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
write!(f, "{}", self.as_str())
}
}
/// I/O load level.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, PartialOrd, Default)]
pub enum IoLoadLevel {
/// Low load - system is underutilized.
Low,
/// Medium load - system is moderately utilized.
#[default]
Medium,
/// High load - system is heavily utilized.
High,
/// Critical load - system is overloaded.
Critical,
}
impl IoLoadLevel {
/// Get the load level as a string for metrics labels.
pub fn as_str(&self) -> &'static str {
match self {
IoLoadLevel::Low => "low",
IoLoadLevel::Medium => "medium",
IoLoadLevel::High => "high",
IoLoadLevel::Critical => "critical",
}
}
/// Determine load level from wait time.
pub fn from_wait_time(wait_time: Duration, low_threshold: Duration, high_threshold: Duration) -> Self {
if wait_time <= low_threshold {
IoLoadLevel::Low
} else if wait_time <= high_threshold {
IoLoadLevel::Medium
} else if wait_time <= high_threshold * 2 {
IoLoadLevel::High
} else {
IoLoadLevel::Critical
}
}
}
impl std::fmt::Display for IoLoadLevel {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
write!(f, "{}", self.as_str())
}
}
/// Bandwidth tier classification.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, Default)]
pub enum BandwidthTier {
/// Low bandwidth (< 100 MB/s).
Low,
/// Medium bandwidth (100-500 MB/s).
#[default]
Medium,
/// High bandwidth (> 500 MB/s).
High,
/// Unknown bandwidth.
Unknown,
}
impl BandwidthTier {
/// Determine bandwidth tier from bytes per second.
pub fn from_bps(bps: u64) -> Self {
const MB: u64 = 1024 * 1024;
if bps < 100 * MB {
BandwidthTier::Low
} else if bps < 500 * MB {
BandwidthTier::Medium
} else {
BandwidthTier::High
}
}
/// Get the tier as a string for metrics labels.
pub fn as_str(&self) -> &'static str {
match self {
BandwidthTier::Low => "low",
BandwidthTier::Medium => "medium",
BandwidthTier::High => "high",
BandwidthTier::Unknown => "unknown",
}
}
}
/// I/O strategy decision.
#[derive(Debug, Clone)]
pub struct IoStrategy {
/// Buffer size to use for I/O operations.
pub buffer_size: usize,
/// Buffer multiplier based on storage media.
pub buffer_multiplier: f64,
/// Whether to enable readahead.
pub enable_readahead: bool,
/// Whether to use buffered I/O.
pub use_buffered_io: bool,
// Performance state
/// Current number of concurrent requests.
pub concurrent_requests: usize,
/// Observed bandwidth in bytes per second.
pub observed_bandwidth_bps: Option<u64>,
/// Bandwidth tier classification.
pub bandwidth_tier: BandwidthTier,
/// Current load level.
pub load_level: IoLoadLevel,
// Priority
/// I/O priority for this operation.
pub priority: IoPriority,
// Decision flags
/// Whether to throttle random I/O.
pub should_throttle_random_io: bool,
/// Whether to expand buffer for sequential access.
pub should_expand_for_sequential: bool,
/// Whether to reduce buffer due to concurrency.
pub should_reduce_for_concurrency: bool,
/// Whether to reduce buffer due to low bandwidth.
pub should_reduce_for_bandwidth: bool,
}
impl Default for IoStrategy {
fn default() -> Self {
Self {
buffer_size: 128 * 1024,
buffer_multiplier: 1.0,
enable_readahead: true,
use_buffered_io: true,
concurrent_requests: 0,
observed_bandwidth_bps: None,
bandwidth_tier: BandwidthTier::Medium,
load_level: IoLoadLevel::Low,
priority: IoPriority::Normal,
should_throttle_random_io: false,
should_expand_for_sequential: false,
should_reduce_for_concurrency: false,
should_reduce_for_bandwidth: false,
}
}
}
impl IoStrategy {
/// Create a new strategy with default values.
pub fn new() -> Self {
Self::default()
}
/// Create a strategy for sequential access.
pub fn sequential(buffer_size: usize) -> Self {
Self {
buffer_size,
enable_readahead: true,
should_expand_for_sequential: true,
..Self::default()
}
}
/// Create a strategy for random access.
pub fn random(buffer_size: usize) -> Self {
Self {
buffer_size,
enable_readahead: false,
should_throttle_random_io: true,
..Self::default()
}
}
}
/// I/O load metrics.
#[derive(Debug, Clone, Default)]
pub struct IoLoadMetrics {
/// Number of samples in the current window.
pub sample_count: usize,
/// Total wait time in the window.
pub total_wait_time: Duration,
/// Maximum wait time in the window.
pub max_wait_time: Duration,
/// Average wait time.
pub avg_wait_time: Duration,
/// Current load level.
pub load_level: IoLoadLevel,
}
impl IoLoadMetrics {
/// Create new load metrics.
pub fn new() -> Self {
Self::default()
}
/// Add a wait time sample.
pub fn add_sample(&mut self, wait_time: Duration) {
self.sample_count += 1;
self.total_wait_time += wait_time;
if wait_time > self.max_wait_time {
self.max_wait_time = wait_time;
}
self.avg_wait_time = if self.sample_count > 0 {
self.total_wait_time / self.sample_count as u32
} else {
Duration::ZERO
};
}
/// Update load level based on thresholds.
pub fn update_load_level(&mut self, low_threshold: Duration, high_threshold: Duration) {
self.load_level = IoLoadLevel::from_wait_time(self.avg_wait_time, low_threshold, high_threshold);
}
/// Reset the metrics.
pub fn reset(&mut self) {
*self = Self::default();
}
}
/// I/O scheduler.
pub struct IoScheduler {
/// Scheduler configuration.
config: IoSchedulerConfig,
/// Active request counter.
active_requests: AtomicUsize,
/// Load metrics.
load_metrics: std::sync::Mutex<IoLoadMetrics>,
}
impl IoScheduler {
/// Create a new I/O scheduler with the given configuration.
pub fn new(config: IoSchedulerConfig) -> Self {
Self {
config,
active_requests: AtomicUsize::new(0),
load_metrics: std::sync::Mutex::new(IoLoadMetrics::new()),
}
}
/// Create a new I/O scheduler with default configuration.
pub fn with_defaults() -> Self {
Self::new(IoSchedulerConfig::default())
}
/// Get the scheduler configuration.
pub fn config(&self) -> &IoSchedulerConfig {
&self.config
}
/// Get the current number of active requests.
pub fn active_requests(&self) -> usize {
self.active_requests.load(Ordering::Relaxed)
}
/// Increment the active request count.
pub fn increment_requests(&self) {
self.active_requests.fetch_add(1, Ordering::Relaxed);
}
/// Decrement the active request count.
pub fn decrement_requests(&self) {
self.active_requests.fetch_sub(1, Ordering::Relaxed);
}
/// Calculate I/O strategy for a request.
pub fn calculate_strategy(&self, file_size: i64, permit_wait_time: Duration, is_sequential: bool) -> IoStrategy {
let concurrent_requests = self.active_requests.load(Ordering::Relaxed);
// Determine priority based on file size
let priority = IoPriority::from_size(
file_size,
self.config.high_priority_size_threshold,
self.config.low_priority_size_threshold,
);
// Determine load level
let load_level =
IoLoadLevel::from_wait_time(permit_wait_time, self.config.load_low_threshold(), self.config.load_high_threshold());
// Calculate base buffer size
let base_buffer = self.config.base_buffer_size;
// Adjust for concurrency
let concurrency_factor = match concurrent_requests {
0..=2 => 1.0,
3..=4 => 0.75,
5..=8 => 0.5,
_ => 0.4,
};
// Adjust for load level
let load_factor = match load_level {
IoLoadLevel::Low => 1.2,
IoLoadLevel::Medium => 1.0,
IoLoadLevel::High => 0.7,
IoLoadLevel::Critical => 0.5,
};
// Adjust for access pattern
let sequential_factor = if is_sequential { 1.5 } else { 1.0 };
// Calculate final buffer size
let buffer_size = (base_buffer as f64 * concurrency_factor * load_factor * sequential_factor) as usize;
let buffer_size = buffer_size.clamp(self.config.min_buffer_size, self.config.max_buffer_size);
IoStrategy {
buffer_size,
buffer_multiplier: concurrency_factor * load_factor * sequential_factor,
enable_readahead: is_sequential && load_level != IoLoadLevel::Critical,
use_buffered_io: true,
concurrent_requests,
observed_bandwidth_bps: None,
bandwidth_tier: BandwidthTier::Unknown,
load_level,
priority,
should_throttle_random_io: !is_sequential && load_level >= IoLoadLevel::High,
should_expand_for_sequential: is_sequential && load_level <= IoLoadLevel::Medium,
should_reduce_for_concurrency: concurrent_requests > 4,
should_reduce_for_bandwidth: false,
}
}
/// Calculate multi-factor I/O strategy.
pub fn calculate_multi_factor_strategy(
&self,
file_size: i64,
permit_wait_time: Duration,
is_sequential: bool,
storage_profile: Option<&StorageProfile>,
) -> IoStrategy {
let mut strategy = self.calculate_strategy(file_size, permit_wait_time, is_sequential);
// Apply storage profile adjustments
if let Some(profile) = storage_profile {
// Adjust buffer size based on storage media
let media_factor = match profile.media {
StorageMedia::Nvme => 1.5,
StorageMedia::Ssd => 1.2,
StorageMedia::Hdd => 0.8,
StorageMedia::Unknown => 1.0,
};
strategy.buffer_size = (strategy.buffer_size as f64 * media_factor).min(self.config.max_buffer_size as f64) as usize;
// Apply sequential boost if applicable
if is_sequential {
strategy.buffer_size = (strategy.buffer_size as f64 * profile.sequential_boost_multiplier)
.min(self.config.max_buffer_size as f64) as usize;
}
// Apply random penalty if applicable
if !is_sequential {
strategy.buffer_size = (strategy.buffer_size as f64 * profile.random_penalty_multiplier)
.max(self.config.min_buffer_size as f64) as usize;
}
// Update readahead preference
strategy.enable_readahead = strategy.enable_readahead && profile.prefers_readahead;
}
strategy
}
/// Record a wait time sample for load tracking.
pub fn record_wait_time(&self, wait_time: Duration) {
if let Ok(mut metrics) = self.load_metrics.lock() {
metrics.add_sample(wait_time);
metrics.update_load_level(self.config.load_low_threshold(), self.config.load_high_threshold());
}
}
/// Get current load metrics.
pub fn load_metrics(&self) -> IoLoadMetrics {
if let Ok(metrics) = self.load_metrics.lock() {
metrics.clone()
} else {
IoLoadMetrics::default()
}
}
}
impl Default for IoScheduler {
fn default() -> Self {
Self::with_defaults()
}
}
// ============================================================================
// Buffer Size Calculation Functions
// ============================================================================
/// Constants for buffer size calculations.
pub const KI_B: usize = 1024;
pub const MI_B: usize = 1024 * 1024;
/// Get concurrency-aware buffer size.
///
/// Adjusts buffer size based on the current level of concurrent requests.
/// Higher concurrency leads to smaller buffers to reduce memory pressure.
///
/// # Arguments
///
/// * `file_size` - Size of the file being read (-1 if unknown)
/// * `base_buffer_size` - Base buffer size from workload profile
///
/// # Returns
///
/// Adjusted buffer size in bytes
pub fn get_concurrency_aware_buffer_size(file_size: i64, base_buffer_size: usize) -> usize {
// Get current concurrency level from global counter
let concurrent_requests = 1; // Default to 1 if no global counter available
// Define concurrency thresholds
let medium_threshold = 4;
let high_threshold = 8;
// Calculate adaptive multiplier based on concurrency
let adaptive_multiplier = if concurrent_requests <= 2 {
// Low concurrency (1-2): use full buffer size
1.0
} else if concurrent_requests <= medium_threshold {
// Medium concurrency (3-4): slightly reduce buffer size (75% of base)
0.75
} else if concurrent_requests <= high_threshold {
// Higher concurrency (5-8): more aggressive reduction (50% of base)
0.5
} else {
// Very high concurrency (>8): minimize memory per request (40% of base)
0.4
};
// Calculate the adjusted buffer size
let adjusted_size = (base_buffer_size as f64 * adaptive_multiplier) as usize;
// Ensure we stay within reasonable bounds
let min_buffer = if file_size > 0 && file_size < 100 * KI_B as i64 {
32 * KI_B // For very small files, use minimum buffer
} else {
64 * KI_B // Standard minimum buffer size
};
let max_buffer = if concurrent_requests > high_threshold {
256 * KI_B // Cap at 256KB for high concurrency
} else {
MI_B // Cap at 1MB for lower concurrency
};
adjusted_size.clamp(min_buffer, max_buffer)
}
/// Advanced concurrency-aware buffer sizing with file size optimization.
///
/// This enhanced version considers both concurrency level and file size patterns
/// to provide even better performance characteristics.
///
/// # Arguments
///
/// * `file_size` - Size of the file being read (-1 if unknown)
/// * `base_buffer_size` - Baseline buffer size from workload profile
/// * `is_sequential` - Whether this is a sequential read (hint for optimization)
/// * `concurrent_requests` - Current number of concurrent requests
///
/// # Returns
///
/// Optimized buffer size in bytes
pub fn get_advanced_buffer_size(
file_size: i64,
base_buffer_size: usize,
is_sequential: bool,
concurrent_requests: usize,
) -> usize {
// For very small files, use smaller buffers regardless of concurrency
if file_size > 0 && file_size < 256 * KI_B as i64 {
return (file_size as usize / 4).clamp(16 * KI_B, 64 * KI_B);
}
// Base calculation from standard function
let standard_size = get_concurrency_aware_buffer_size(file_size, base_buffer_size);
let medium_threshold = 4;
let high_threshold = 8;
// For sequential reads, we can be more aggressive with buffer sizes
if is_sequential && concurrent_requests <= medium_threshold {
// Boost buffer size for sequential reads under low concurrency
let boosted = (standard_size as f64 * 1.5) as usize;
return boosted.min(MI_B);
}
// For random reads under high concurrency, reduce buffer size
if !is_sequential && concurrent_requests > high_threshold {
let reduced = (standard_size as f64 * 0.7) as usize;
return reduced.max(32 * KI_B);
}
standard_size
}
/// Get buffer size with storage media optimization.
///
/// Adjusts buffer size based on storage media characteristics.
///
/// # Arguments
///
/// * `base_size` - Base buffer size
/// * `media` - Storage media type
///
/// # Returns
///
/// Optimized buffer size for the storage media
pub fn get_buffer_size_for_media(base_size: usize, media: StorageMedia) -> usize {
let multiplier = match media {
StorageMedia::Nvme => 1.5, // NVMe can handle larger buffers
StorageMedia::Ssd => 1.2, // SSD benefits from moderate buffers
StorageMedia::Hdd => 0.8, // HDD prefers smaller buffers to reduce seek overhead
StorageMedia::Unknown => 1.0,
};
(base_size as f64 * multiplier).min(MI_B as f64) as usize
}
/// Calculate optimal buffer size using multi-factor analysis.
///
/// This is the main entry point for buffer size calculation, considering
/// all factors: concurrency, storage media, access pattern, and load.
///
/// # Arguments
///
/// * `file_size` - Size of the file being read
/// * `base_buffer_size` - Base buffer size
/// * `is_sequential` - Whether access is sequential
/// * `concurrent_requests` - Current concurrency level
/// * `media` - Storage media type
/// * `load_level` - Current I/O load level
///
/// # Returns
///
/// Optimally calculated buffer size
pub fn calculate_optimal_buffer_size(
file_size: i64,
base_buffer_size: usize,
is_sequential: bool,
concurrent_requests: usize,
media: StorageMedia,
load_level: IoLoadLevel,
) -> usize {
// Start with advanced buffer size calculation
let mut buffer_size = get_advanced_buffer_size(file_size, base_buffer_size, is_sequential, concurrent_requests);
// Apply storage media optimization
buffer_size = get_buffer_size_for_media(buffer_size, media);
// Apply load-based adjustment
let load_multiplier = match load_level {
IoLoadLevel::Low => 1.2,
IoLoadLevel::Medium => 1.0,
IoLoadLevel::High => 0.7,
IoLoadLevel::Critical => 0.5,
};
buffer_size = (buffer_size as f64 * load_multiplier) as usize;
// Final bounds check
buffer_size.clamp(32 * KI_B, MI_B)
}
/// I/O scheduling context for multi-factor strategy calculation.
#[derive(Debug, Clone)]
pub struct IoSchedulingContext {
/// File size in bytes (-1 if unknown).
pub file_size: i64,
/// Base buffer size from configuration.
pub base_buffer_size: usize,
/// Time spent waiting for permit.
pub permit_wait_duration: Duration,
/// Whether access is sequential.
pub is_sequential_hint: bool,
/// Detected access pattern.
pub access_pattern: AccessPattern,
/// Detected storage media.
pub storage_media: StorageMedia,
/// Observed bandwidth in bytes per second.
pub observed_bandwidth_bps: Option<u64>,
/// Current concurrent request count.
pub concurrent_requests: usize,
}
impl Default for IoSchedulingContext {
fn default() -> Self {
Self {
file_size: -1,
base_buffer_size: 128 * KI_B,
permit_wait_duration: Duration::ZERO,
is_sequential_hint: true,
access_pattern: AccessPattern::Unknown,
storage_media: StorageMedia::Unknown,
observed_bandwidth_bps: None,
concurrent_requests: 1,
}
}
}
impl IoSchedulingContext {
/// Create a new scheduling context.
pub fn new(file_size: i64, base_buffer_size: usize) -> Self {
Self {
file_size,
base_buffer_size,
..Self::default()
}
}
/// Builder pattern: set sequential hint.
pub fn with_sequential(mut self, is_sequential: bool) -> Self {
self.is_sequential_hint = is_sequential;
self.access_pattern = if is_sequential {
AccessPattern::Sequential
} else {
AccessPattern::Random
};
self
}
/// Builder pattern: set storage media.
pub fn with_media(mut self, media: StorageMedia) -> Self {
self.storage_media = media;
self
}
/// Builder pattern: set bandwidth.
pub fn with_bandwidth(mut self, bps: u64) -> Self {
self.observed_bandwidth_bps = Some(bps);
self
}
/// Builder pattern: set concurrency.
pub fn with_concurrency(mut self, count: usize) -> Self {
self.concurrent_requests = count;
self
}
/// Builder pattern: set wait duration.
pub fn with_wait_duration(mut self, duration: Duration) -> Self {
self.permit_wait_duration = duration;
self
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_io_priority() {
assert_eq!(IoPriority::from_size(1024, 64 * 1024, 4 * 1024 * 1024), IoPriority::High);
assert_eq!(IoPriority::from_size(1024 * 1024, 64 * 1024, 4 * 1024 * 1024), IoPriority::Normal);
assert_eq!(IoPriority::from_size(10 * 1024 * 1024, 64 * 1024, 4 * 1024 * 1024), IoPriority::Low);
}
#[test]
fn test_io_priority_unknown_size_is_normal() {
// -1 means "size unknown" and must not wrap to usize::MAX (=> Low).
assert_eq!(IoPriority::from_size(-1, 64 * 1024, 4 * 1024 * 1024), IoPriority::Normal);
assert_eq!(IoPriority::from_size(i64::MIN, 64 * 1024, 4 * 1024 * 1024), IoPriority::Normal);
}
#[test]
fn test_io_load_level() {
let low = Duration::from_millis(5);
let high = Duration::from_millis(50);
assert_eq!(IoLoadLevel::from_wait_time(Duration::from_millis(1), low, high), IoLoadLevel::Low);
assert_eq!(IoLoadLevel::from_wait_time(Duration::from_millis(20), low, high), IoLoadLevel::Medium);
assert_eq!(IoLoadLevel::from_wait_time(Duration::from_millis(60), low, high), IoLoadLevel::High);
assert_eq!(IoLoadLevel::from_wait_time(Duration::from_millis(150), low, high), IoLoadLevel::Critical);
}
#[test]
fn test_bandwidth_tier() {
assert_eq!(BandwidthTier::from_bps(50 * 1024 * 1024), BandwidthTier::Low);
assert_eq!(BandwidthTier::from_bps(200 * 1024 * 1024), BandwidthTier::Medium);
assert_eq!(BandwidthTier::from_bps(600 * 1024 * 1024), BandwidthTier::High);
}
#[test]
fn test_io_strategy_default() {
let strategy = IoStrategy::default();
assert!(strategy.buffer_size > 0);
assert!(strategy.enable_readahead);
}
#[test]
fn test_io_scheduler() {
let scheduler = IoScheduler::with_defaults();
let strategy = scheduler.calculate_strategy(1024 * 1024, Duration::from_millis(5), true);
assert!(strategy.buffer_size > 0);
assert!(strategy.enable_readahead);
assert_eq!(strategy.load_level, IoLoadLevel::Low);
}
#[test]
fn test_io_scheduler_with_concurrency() {
let scheduler = IoScheduler::with_defaults();
// Simulate concurrent requests
scheduler.increment_requests();
scheduler.increment_requests();
scheduler.increment_requests();
let strategy = scheduler.calculate_strategy(1024 * 1024, Duration::from_millis(5), true);
assert_eq!(strategy.concurrent_requests, 3);
}
#[test]
fn test_load_metrics() {
let mut metrics = IoLoadMetrics::new();
metrics.add_sample(Duration::from_millis(10));
metrics.add_sample(Duration::from_millis(20));
metrics.add_sample(Duration::from_millis(30));
assert_eq!(metrics.sample_count, 3);
assert_eq!(metrics.avg_wait_time, Duration::from_millis(20));
assert_eq!(metrics.max_wait_time, Duration::from_millis(30));
}
#[test]
fn test_get_concurrency_aware_buffer_size() {
// Test with default concurrency (1)
let size = get_concurrency_aware_buffer_size(1024 * 1024, 128 * KI_B);
assert!(size >= 64 * KI_B);
assert!(size <= MI_B);
// Test with small file
let size = get_concurrency_aware_buffer_size(50 * KI_B as i64, 128 * KI_B);
assert!(size >= 32 * KI_B);
}
#[test]
fn test_get_advanced_buffer_size() {
// Sequential read with low concurrency
let size = get_advanced_buffer_size(10 * MI_B as i64, 128 * KI_B, true, 2);
assert!(size >= 128 * KI_B);
// Random read with high concurrency
let size = get_advanced_buffer_size(10 * MI_B as i64, 128 * KI_B, false, 10);
assert!(size >= 32 * KI_B);
// Very small file
let size = get_advanced_buffer_size(100 * KI_B as i64, 128 * KI_B, true, 1);
assert!(size <= 64 * KI_B);
}
#[test]
fn test_get_buffer_size_for_media() {
let base = 128 * KI_B;
// NVMe should get larger buffers
let nvme_size = get_buffer_size_for_media(base, StorageMedia::Nvme);
assert!(nvme_size > base);
// SSD should get slightly larger buffers
let ssd_size = get_buffer_size_for_media(base, StorageMedia::Ssd);
assert!(ssd_size > base);
// HDD should get smaller buffers
let hdd_size = get_buffer_size_for_media(base, StorageMedia::Hdd);
assert!(hdd_size < base);
}
#[test]
fn test_calculate_optimal_buffer_size() {
// Low load, sequential, NVMe
let size = calculate_optimal_buffer_size(10 * MI_B as i64, 128 * KI_B, true, 2, StorageMedia::Nvme, IoLoadLevel::Low);
assert!(size >= 32 * KI_B);
assert!(size <= MI_B);
// Critical load, random, HDD
let size =
calculate_optimal_buffer_size(10 * MI_B as i64, 128 * KI_B, false, 10, StorageMedia::Hdd, IoLoadLevel::Critical);
assert!(size >= 32 * KI_B);
assert!(size <= MI_B);
}
#[test]
fn test_io_scheduling_context() {
let ctx = IoSchedulingContext::new(10 * MI_B as i64, 256 * KI_B)
.with_sequential(true)
.with_media(StorageMedia::Nvme)
.with_bandwidth(500 * MI_B as u64)
.with_concurrency(4);
assert_eq!(ctx.file_size, 10 * MI_B as i64);
assert_eq!(ctx.base_buffer_size, 256 * KI_B);
assert!(ctx.is_sequential_hint);
assert_eq!(ctx.storage_media, StorageMedia::Nvme);
assert_eq!(ctx.observed_bandwidth_bps, Some(500 * MI_B as u64));
assert_eq!(ctx.concurrent_requests, 4);
}
}
-320
View File
@@ -1,320 +0,0 @@
// Copyright 2024 RustFS Team
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//! Shared memory pool for zero-copy data sharing.
//!
//! This module provides Arc-based shared memory management for
//! efficient cross-task data passing without serialization.
use std::convert::AsRef;
use std::ops::Deref;
use std::sync::Arc;
use std::sync::atomic::{AtomicU64, Ordering};
use std::time::Instant;
/// Shared memory pool configuration.
#[derive(Debug, Clone)]
pub struct SharedMemoryConfig {
/// Whether shared memory is enabled
pub enabled: bool,
/// Maximum pool size in bytes
pub max_pool_size: usize,
/// Maximum object size in bytes
pub max_object_size: usize,
}
impl Default for SharedMemoryConfig {
fn default() -> Self {
Self {
enabled: true,
max_pool_size: 100 * 1024 * 1024, // 100MB
max_object_size: 10 * 1024 * 1024, // 10MB
}
}
}
/// Shared memory pool statistics.
#[derive(Debug, Default)]
pub struct SharedMemoryStats {
/// Total number of objects created
pub total_objects: AtomicU64,
/// Total number of shared references
pub total_shared_refs: AtomicU64,
/// Current memory usage in bytes
pub current_memory: AtomicU64,
/// Peak memory usage in bytes
pub peak_memory: AtomicU64,
}
/// Arc data metadata.
#[derive(Clone, Debug)]
pub struct ArcMetadata {
/// Size of the data (if measurable)
pub size: Option<usize>,
/// Creation timestamp
pub created_at: Instant,
}
/// Arc-based data wrapper for zero-copy sharing.
///
/// This wrapper uses Arc to enable shared ownership of data
/// across multiple tasks without copying.
pub struct ArcData<T> {
/// The wrapped data
inner: Arc<T>,
/// Metadata about the data
metadata: ArcMetadata,
}
impl<T> Clone for ArcData<T> {
fn clone(&self) -> Self {
Self {
inner: Arc::clone(&self.inner),
metadata: self.metadata.clone(),
}
}
}
impl<T> ArcData<T> {
/// Create a new ArcData wrapper.
pub fn new(data: T) -> Self {
ArcData {
inner: Arc::new(data),
metadata: ArcMetadata {
size: None,
created_at: Instant::now(),
},
}
}
/// Create a new ArcData wrapper with known size.
pub fn with_size(data: T, size: usize) -> Self {
ArcData {
inner: Arc::new(data),
metadata: ArcMetadata {
size: Some(size),
created_at: Instant::now(),
},
}
}
/// Get the reference count.
pub fn ref_count(&self) -> usize {
Arc::strong_count(&self.inner)
}
/// Convert into the underlying Arc.
pub fn into_arc(self) -> Arc<T> {
self.inner
}
/// Get the metadata.
pub fn metadata(&self) -> &ArcMetadata {
&self.metadata
}
/// Get the size if known.
pub fn size(&self) -> Option<usize> {
self.metadata.size
}
}
impl<T> AsRef<T> for ArcData<T> {
fn as_ref(&self) -> &T {
&self.inner
}
}
impl<T> Deref for ArcData<T> {
type Target = T;
fn deref(&self) -> &Self::Target {
&self.inner
}
}
impl<T> std::fmt::Debug for ArcData<T> {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("ArcData")
.field("ref_count", &self.ref_count())
.field("metadata", &self.metadata)
.finish()
}
}
/// Shared memory pool for managing Arc-based shared data.
pub struct SharedMemoryPool {
config: SharedMemoryConfig,
stats: SharedMemoryStats,
}
impl SharedMemoryPool {
/// Create a new shared memory pool with the given configuration.
pub fn new(config: SharedMemoryConfig) -> Self {
Self {
config,
stats: SharedMemoryStats::default(),
}
}
/// Create a new shared memory pool with default configuration.
pub fn with_defaults() -> Self {
Self::new(SharedMemoryConfig::default())
}
/// Create shared data.
///
/// This method wraps the data in an ArcData for zero-copy sharing.
pub fn create<T>(&self, data: T) -> ArcData<T> {
self.stats.total_objects.fetch_add(1, Ordering::Relaxed);
ArcData::new(data)
}
/// Create shared data with known size.
///
/// This method tracks memory usage for statistics.
pub fn create_with_size<T>(&self, data: T, size: usize) -> ArcData<T> {
self.stats.total_objects.fetch_add(1, Ordering::Relaxed);
// Update memory statistics
self.stats.current_memory.fetch_add(size as u64, Ordering::Relaxed);
// Update peak memory
let current = self.stats.current_memory.load(Ordering::Relaxed);
let mut peak = self.stats.peak_memory.load(Ordering::Relaxed);
if current > peak {
peak = current;
self.stats.peak_memory.store(peak, Ordering::Relaxed);
}
ArcData::with_size(data, size)
}
/// Share data by increasing reference count.
///
/// This method creates a new ArcData that shares the underlying data
/// without copying.
pub fn share<T>(&self, data: &ArcData<T>) -> ArcData<T> {
self.stats.total_shared_refs.fetch_add(1, Ordering::Relaxed);
data.clone()
}
/// Get the statistics for this pool.
pub fn stats(&self) -> &SharedMemoryStats {
&self.stats
}
/// Get the configuration for this pool.
pub fn config(&self) -> &SharedMemoryConfig {
&self.config
}
/// Check if the pool is enabled.
pub fn is_enabled(&self) -> bool {
self.config.enabled
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_arc_data_new() {
let data = vec![1u8, 2, 3, 4, 5];
let arc_data = ArcData::new(data.clone());
assert_eq!(arc_data.as_ref(), &data);
assert_eq!(arc_data.ref_count(), 1);
}
#[test]
fn test_arc_data_clone() {
let data = vec![1u8, 2, 3, 4, 5];
let arc_data = ArcData::new(data);
assert_eq!(arc_data.ref_count(), 1);
let arc_data2 = arc_data.clone();
assert_eq!(arc_data.ref_count(), 2);
assert_eq!(arc_data2.ref_count(), 2);
let arc_data3 = arc_data.clone();
assert_eq!(arc_data.ref_count(), 3);
assert_eq!(arc_data2.ref_count(), 3);
assert_eq!(arc_data3.ref_count(), 3);
}
#[test]
fn test_arc_data_deref() {
let data = vec![1u8, 2, 3, 4, 5];
let arc_data = ArcData::new(data);
// Test Deref trait
assert_eq!(arc_data.len(), 5);
assert_eq!(arc_data[0], 1);
}
#[test]
fn test_shared_memory_pool_create() {
let pool = SharedMemoryPool::with_defaults();
let data = vec![1u8, 2, 3, 4, 5];
let arc_data = pool.create(data.clone());
assert_eq!(arc_data.as_ref(), &data);
assert_eq!(pool.stats().total_objects.load(Ordering::Relaxed), 1);
}
#[test]
fn test_shared_memory_pool_share() {
let pool = SharedMemoryPool::with_defaults();
let data = vec![1u8, 2, 3, 4, 5];
let arc_data = pool.create(data);
assert_eq!(arc_data.ref_count(), 1);
let shared = pool.share(&arc_data);
assert_eq!(arc_data.ref_count(), 2);
assert_eq!(shared.ref_count(), 2);
assert_eq!(pool.stats().total_shared_refs.load(Ordering::Relaxed), 1);
}
#[test]
fn test_shared_memory_pool_with_size() {
let pool = SharedMemoryPool::with_defaults();
let data = vec![1u8; 1024];
let arc_data = pool.create_with_size(data, 1024);
assert_eq!(arc_data.size(), Some(1024));
assert_eq!(pool.stats().current_memory.load(Ordering::Relaxed), 1024);
}
#[test]
fn test_default_config() {
let config = SharedMemoryConfig::default();
assert!(config.enabled);
assert_eq!(config.max_pool_size, 100 * 1024 * 1024);
assert_eq!(config.max_object_size, 10 * 1024 * 1024);
}
}
-501
View File
@@ -1,501 +0,0 @@
// Copyright 2024 RustFS Team
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//! Timeout wrapper for I/O operations.
//!
//! This module provides timeout management for I/O operations with
//! dynamic timeout calculation based on operation size.
use std::sync::atomic::{AtomicU64, Ordering};
use std::time::{Duration, Instant};
/// Timeout configuration.
#[derive(Debug, Clone)]
pub struct TimeoutConfig {
/// Base timeout for small operations.
pub base_timeout: Duration,
/// Timeout per MB of data.
pub timeout_per_mb: Duration,
/// Maximum timeout.
pub max_timeout: Duration,
/// Minimum timeout.
pub min_timeout: Duration,
/// GetObject operation timeout.
pub get_object_timeout: Duration,
/// PutObject operation timeout.
pub put_object_timeout: Duration,
/// ListObjects operation timeout.
pub list_objects_timeout: Duration,
/// Whether dynamic timeout is enabled.
pub enable_dynamic_timeout: bool,
}
impl Default for TimeoutConfig {
fn default() -> Self {
Self {
base_timeout: Duration::from_secs(5),
timeout_per_mb: Duration::from_millis(100),
max_timeout: Duration::from_secs(300),
min_timeout: Duration::from_secs(1),
get_object_timeout: Duration::from_secs(30),
put_object_timeout: Duration::from_secs(60),
list_objects_timeout: Duration::from_secs(10),
enable_dynamic_timeout: true,
}
}
}
impl TimeoutConfig {
/// Create new timeout configuration.
pub fn new() -> Self {
Self::default()
}
/// Calculate dynamic timeout based on size.
pub fn calculate_timeout(&self, size_bytes: u64) -> Duration {
if !self.enable_dynamic_timeout {
return self.base_timeout;
}
let mb = size_bytes as f64 / (1024.0 * 1024.0);
let timeout = self.base_timeout + self.timeout_per_mb.mul_f64(mb);
timeout.clamp(self.min_timeout, self.max_timeout)
}
/// Validate the configuration.
pub fn validate(&self) -> Result<(), TimeoutError> {
if self.min_timeout > self.max_timeout {
return Err(TimeoutError::InvalidConfig("min_timeout must be <= max_timeout".to_string()));
}
if self.base_timeout < self.min_timeout || self.base_timeout > self.max_timeout {
return Err(TimeoutError::InvalidConfig(
"base_timeout must be between min_timeout and max_timeout".to_string(),
));
}
Ok(())
}
}
/// Timeout error.
#[derive(Debug, Clone, thiserror::Error)]
pub enum TimeoutError {
/// Operation timed out.
#[error("Operation timed out after {0:?}")]
TimedOut(Duration),
/// Invalid configuration.
#[error("Invalid timeout config: {0}")]
InvalidConfig(String),
}
/// Operation progress tracker.
#[derive(Debug)]
pub struct OperationProgress {
/// Total size (if known).
pub total_size: Option<u64>,
/// Bytes processed.
bytes_processed: AtomicU64,
/// Last update time.
last_update: std::sync::Mutex<Instant>,
/// Stale timeout.
stale_timeout: Duration,
/// Start time for transfer rate calculation.
start_time: Instant,
}
impl OperationProgress {
/// Create new operation progress.
pub fn new(total_size: Option<u64>, stale_timeout: Duration) -> Self {
Self {
total_size,
bytes_processed: AtomicU64::new(0),
last_update: std::sync::Mutex::new(Instant::now()),
stale_timeout,
start_time: Instant::now(),
}
}
/// Update progress.
pub fn update(&self, bytes: u64) {
self.bytes_processed.store(bytes, Ordering::Relaxed);
if let Ok(mut last) = self.last_update.lock() {
*last = Instant::now();
}
}
/// Add to progress.
pub fn add(&self, bytes: u64) {
self.bytes_processed.fetch_add(bytes, Ordering::Relaxed);
if let Ok(mut last) = self.last_update.lock() {
*last = Instant::now();
}
}
/// Get current progress.
pub fn current(&self) -> u64 {
self.bytes_processed.load(Ordering::Relaxed)
}
/// Check if progress is stale.
pub fn is_stale(&self) -> bool {
if let Ok(last) = self.last_update.lock() {
last.elapsed() > self.stale_timeout
} else {
false
}
}
/// Get progress percentage.
pub fn progress_percent(&self) -> Option<f64> {
self.total_size.map(|total| {
if total == 0 {
100.0
} else {
let processed = self.bytes_processed.load(Ordering::Relaxed);
(processed as f64 / total as f64 * 100.0).min(100.0)
}
})
}
/// Get remaining bytes.
pub fn remaining(&self) -> Option<u64> {
self.total_size.map(|total| {
let processed = self.bytes_processed.load(Ordering::Relaxed);
total.saturating_sub(processed)
})
}
/// Calculate transfer rate in bytes per second.
///
/// Returns 0 if no time has elapsed or no data transferred.
pub fn transfer_rate(&self) -> u64 {
let processed = self.bytes_processed.load(Ordering::Relaxed);
if processed == 0 {
return 0;
}
let elapsed = self.start_time.elapsed().as_secs_f64();
if elapsed > 0.0 {
(processed as f64 / elapsed) as u64
} else {
0
}
}
}
/// Request timeout wrapper.
pub struct RequestTimeoutWrapper {
/// Configuration.
config: TimeoutConfig,
/// Start time.
start_time: Instant,
/// Operation progress.
progress: Option<OperationProgress>,
}
impl RequestTimeoutWrapper {
/// Create a new timeout wrapper.
pub fn new(config: TimeoutConfig) -> Self {
Self {
config,
start_time: Instant::now(),
progress: None,
}
}
/// Create with progress tracking.
pub fn with_progress(config: TimeoutConfig, total_size: Option<u64>, stale_timeout: Duration) -> Self {
Self {
config,
start_time: Instant::now(),
progress: Some(OperationProgress::new(total_size, stale_timeout)),
}
}
/// Get the configuration.
pub fn config(&self) -> &TimeoutConfig {
&self.config
}
/// Get elapsed time.
pub fn elapsed(&self) -> Duration {
self.start_time.elapsed()
}
/// Get remaining time.
pub fn remaining(&self, timeout: Duration) -> Option<Duration> {
let elapsed = self.elapsed();
if elapsed >= timeout { None } else { Some(timeout - elapsed) }
}
/// Check if timed out.
pub fn is_timed_out(&self, size: Option<u64>) -> bool {
let timeout = self.get_timeout(size);
self.elapsed() > timeout
}
/// Get the timeout for a given size.
pub fn get_timeout(&self, size: Option<u64>) -> Duration {
if self.config.enable_dynamic_timeout {
if let Some(s) = size {
self.config.calculate_timeout(s)
} else {
self.config.base_timeout
}
} else {
self.config.base_timeout
}
}
/// Check if timed out and return error if so.
pub fn check_timeout(&self, size: Option<u64>) -> Result<(), TimeoutError> {
if self.is_timed_out(size) {
Err(TimeoutError::TimedOut(self.get_timeout(size)))
} else {
Ok(())
}
}
/// Get progress.
pub fn progress(&self) -> Option<&OperationProgress> {
self.progress.as_ref()
}
/// Update progress.
pub fn update_progress(&self, bytes: u64) {
if let Some(ref progress) = self.progress {
progress.update(bytes);
}
}
/// Check if operation is stalled (no progress for a while).
pub fn is_stalled(&self) -> bool {
self.progress.as_ref().is_some_and(|p| p.is_stale())
}
/// Get progress percentage.
pub fn progress_percent(&self) -> Option<f64> {
self.progress.as_ref().and_then(|p| p.progress_percent())
}
}
/// Timeout statistics.
#[derive(Debug, Default)]
pub struct TimeoutStats {
/// Total operations.
pub total_operations: AtomicU64,
/// Timed out operations.
pub timed_out: AtomicU64,
/// Total wait time in nanoseconds.
pub total_wait_time_ns: AtomicU64,
/// Maximum wait time in nanoseconds.
pub max_wait_time_ns: AtomicU64,
}
impl TimeoutStats {
/// Create new timeout statistics.
pub fn new() -> Self {
Self::default()
}
/// Record an operation.
pub fn record_operation(&self, wait_time: Duration) {
self.total_operations.fetch_add(1, Ordering::Relaxed);
let ns = wait_time.as_nanos() as u64;
self.total_wait_time_ns.fetch_add(ns, Ordering::Relaxed);
let mut current = self.max_wait_time_ns.load(Ordering::Relaxed);
while ns > current {
match self
.max_wait_time_ns
.compare_exchange_weak(current, ns, Ordering::Relaxed, Ordering::Relaxed)
{
Ok(_) => break,
Err(actual) => current = actual,
}
}
}
/// Record a timeout.
pub fn record_timeout(&self) {
self.timed_out.fetch_add(1, Ordering::Relaxed);
}
/// Get timeout rate.
pub fn timeout_rate(&self) -> f64 {
let total = self.total_operations.load(Ordering::Relaxed);
let timed_out = self.timed_out.load(Ordering::Relaxed);
if total == 0 { 0.0 } else { timed_out as f64 / total as f64 }
}
/// Get average wait time.
pub fn avg_wait_time(&self) -> Duration {
let total = self.total_wait_time_ns.load(Ordering::Relaxed);
let count = self.total_operations.load(Ordering::Relaxed);
total.checked_div(count).map(Duration::from_nanos).unwrap_or(Duration::ZERO)
}
/// Reset statistics.
pub fn reset(&self) {
self.total_operations.store(0, Ordering::Relaxed);
self.timed_out.store(0, Ordering::Relaxed);
self.total_wait_time_ns.store(0, Ordering::Relaxed);
self.max_wait_time_ns.store(0, Ordering::Relaxed);
}
}
/// Calculate adaptive timeout based on historical data and current conditions.
///
/// This function adjusts the timeout based on:
/// - Historical transfer rate
/// - Recent timeout count
/// - Object size
pub fn calculate_adaptive_timeout(
base_timeout: Duration,
historical_rate_bps: Option<u64>,
recent_timeout_count: u32,
object_size: u64,
) -> Duration {
// If we have recent timeouts, increase timeout
let timeout_multiplier = if recent_timeout_count > 3 {
2.0 // Double timeout if many recent timeouts
} else if recent_timeout_count > 1 {
1.5 // 50% increase if some timeouts
} else {
1.0 // No adjustment
};
// Adaptive timeout bounds: 5 seconds minimum, 10 minutes maximum.
const MIN_SECS: f64 = 5.0;
const MAX_SECS: f64 = 600.0;
// If we have historical rate data, use it for estimation
let estimated_secs = match historical_rate_bps {
Some(rate) if rate > 0 => (object_size as f64 / rate as f64) * 1.2, // 20% buffer
_ => base_timeout.as_secs_f64(),
};
// Clamp BEFORE constructing the Duration: `from_secs_f64` panics when the
// estimate overflows Duration (huge object_size with a tiny historical rate).
Duration::from_secs_f64((estimated_secs * timeout_multiplier).clamp(MIN_SECS, MAX_SECS))
}
/// Estimate bytes per second transfer rate.
///
/// This is used for adaptive timeout calculation.
pub fn estimate_bytes_per_second(object_size: u64, expected_duration: Duration) -> u64 {
let secs = expected_duration.as_secs_f64();
if secs > 0.0 {
(object_size as f64 / secs) as u64
} else {
// Return a reasonable default (1 MB/s)
1024 * 1024
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_timeout_config() {
let config = TimeoutConfig::default();
assert!(config.validate().is_ok());
// Small file
let timeout = config.calculate_timeout(1024);
assert!(timeout >= config.min_timeout);
// Large file
let timeout = config.calculate_timeout(100 * 1024 * 1024);
assert!(timeout <= config.max_timeout);
}
#[test]
fn test_timeout_config_validation() {
let config = TimeoutConfig {
min_timeout: Duration::from_secs(10),
max_timeout: Duration::from_secs(5),
..Default::default()
};
assert!(config.validate().is_err());
}
#[test]
fn test_adaptive_timeout_extreme_estimate_does_not_panic() {
// A huge object with a tiny historical rate used to overflow
// Duration::from_secs_f64 and panic; it must clamp to the upper bound.
let timeout = calculate_adaptive_timeout(Duration::from_secs(30), Some(1), 0, u64::MAX);
assert_eq!(timeout, Duration::from_secs(600));
// Tiny estimates clamp to the lower bound.
let timeout = calculate_adaptive_timeout(Duration::from_secs(30), Some(u64::MAX), 0, 1);
assert_eq!(timeout, Duration::from_secs(5));
}
#[test]
fn test_operation_progress() {
let progress = OperationProgress::new(Some(1000), Duration::from_secs(5));
assert_eq!(progress.current(), 0);
assert_eq!(progress.progress_percent(), Some(0.0));
progress.update(500);
assert_eq!(progress.current(), 500);
assert_eq!(progress.progress_percent(), Some(50.0));
progress.add(300);
assert_eq!(progress.current(), 800);
assert_eq!(progress.remaining(), Some(200));
}
#[test]
fn test_request_timeout_wrapper() {
let config = TimeoutConfig {
base_timeout: Duration::from_millis(100),
enable_dynamic_timeout: false,
..Default::default()
};
let wrapper = RequestTimeoutWrapper::new(config);
assert!(!wrapper.is_timed_out(None));
std::thread::sleep(Duration::from_millis(150));
assert!(wrapper.is_timed_out(None));
assert!(wrapper.check_timeout(None).is_err());
}
#[test]
fn test_timeout_stats() {
let stats = TimeoutStats::new();
stats.record_operation(Duration::from_millis(10));
stats.record_operation(Duration::from_millis(20));
stats.record_timeout();
assert_eq!(stats.total_operations.load(Ordering::Relaxed), 2);
assert_eq!(stats.timed_out.load(Ordering::Relaxed), 1);
assert!((stats.timeout_rate() - 0.5).abs() < 0.01);
}
#[test]
fn test_progress_tracking() {
let config = TimeoutConfig::default();
let wrapper = RequestTimeoutWrapper::with_progress(config, Some(1000), Duration::from_secs(1));
wrapper.update_progress(500);
assert_eq!(wrapper.progress_percent(), Some(50.0));
assert!(!wrapper.is_stalled());
}
}
-443
View File
@@ -1,443 +0,0 @@
// Copyright 2024 RustFS Team
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//! BytesMut-backed object writer for optimized write operations.
//!
//! It uses `BytesMut` for efficient buffering; writes into that buffer may
//! still copy input bytes. The historical `ZeroCopyObjectWriter` name remains
//! available as a deprecated compatibility alias.
use bytes::{BufMut, Bytes, BytesMut};
use std::pin::Pin;
use std::task::{Context, Poll};
use tokio::io::AsyncWrite;
/// BytesMut-backed object writer for optimized write operations.
///
/// This writer minimizes memory allocations by:
/// - Using BytesMut for efficient buffer growth
/// - Accepting `Bytes` inputs for efficient buffer handling
/// - Optional integration with BytesPool for buffer reuse
///
/// # Example
///
/// ```ignore
/// use rustfs_io_core::BytesMutWriter;
/// use bytes::Bytes;
///
/// #[tokio::main]
/// async fn main() -> Result<(), Box<dyn std::error::Error>> {
/// let mut writer = BytesMutWriter::new();
///
/// // Write into the internal BytesMut buffer
/// let data = Bytes::from("hello world");
/// writer.write_buffered(data).await?;
///
/// // Get the result as Bytes (zero-copy conversion)
/// let result = writer.into_bytes();
///
/// Ok(())
/// }
/// ```
pub struct BytesMutWriter {
/// Internal buffer using BytesMut for efficient growth
buffer: BytesMut,
/// Total bytes written
bytes_written: usize,
/// Whether the writer has been finalized
finalized: bool,
}
/// Historical name for the BytesMut-backed object writer.
#[deprecated(since = "1.0.0-beta.8", note = "use BytesMutWriter; writes append into a BytesMut buffer")]
pub type ZeroCopyObjectWriter = BytesMutWriter;
impl BytesMutWriter {
/// Create a new bytes-backed object writer with default capacity (8KB).
///
/// # Example
///
/// ```ignore
/// let writer = BytesMutWriter::new();
/// ```
pub fn new() -> Self {
Self::with_capacity(8 * 1024)
}
/// Create a new bytes-backed object writer with specified capacity.
///
/// # Arguments
///
/// * `capacity` - Initial buffer capacity in bytes
///
/// # Example
///
/// ```ignore
/// let writer = BytesMutWriter::with_capacity(64 * 1024);
/// ```
pub fn with_capacity(capacity: usize) -> Self {
Self {
buffer: BytesMut::with_capacity(capacity),
bytes_written: 0,
finalized: false,
}
}
/// Write data into the internal buffer.
///
/// This method accepts `Bytes` for API compatibility, then appends the
/// bytes into the internal `BytesMut` buffer.
///
/// # Arguments
///
/// * `data` - Data to append to the internal buffer
///
/// # Returns
///
/// * `Ok(usize)` - Number of bytes written
/// * `Err(ZeroCopyWriteError)` - Write error
///
/// # Example
///
/// ```ignore
/// let data = Bytes::from("hello world");
/// let written = writer.write_buffered(data).await?;
/// ```
pub async fn write_buffered(&mut self, data: Bytes) -> Result<usize, ZeroCopyWriteError> {
if self.finalized {
return Err(ZeroCopyWriteError::Finalized("Cannot write to finalized writer".to_string()));
}
let len = data.len();
self.buffer.put(data);
self.bytes_written += len;
Ok(len)
}
/// Historical name for `write_buffered`.
#[deprecated(
since = "1.0.0-beta.8",
note = "use write_buffered; this method appends bytes into an internal buffer"
)]
pub async fn write_zero_copy(&mut self, data: Bytes) -> Result<usize, ZeroCopyWriteError> {
self.write_buffered(data).await
}
/// Write a slice of data.
///
/// # Arguments
///
/// * `data` - Data slice to write
///
/// # Returns
///
/// * `Ok(usize)` - Number of bytes written
/// * `Err(ZeroCopyWriteError)` - Write error
pub async fn write_slice(&mut self, data: &[u8]) -> Result<usize, ZeroCopyWriteError> {
if self.finalized {
return Err(ZeroCopyWriteError::Finalized("Cannot write to finalized writer".to_string()));
}
let len = data.len();
self.buffer.put_slice(data);
self.bytes_written += len;
Ok(len)
}
/// Finalize the writer and consume it, returning the written data as Bytes.
///
/// This converts the internal BytesMut to Bytes, which is a zero-copy
/// operation that freezes the buffer.
///
/// # Returns
///
/// The written data as Bytes
///
/// # Example
///
/// ```ignore
/// let result = writer.into_bytes();
/// ```
pub fn into_bytes(mut self) -> Bytes {
self.finalized = true;
self.buffer.freeze()
}
/// Get the current buffer as a slice (without consuming).
///
/// # Returns
///
/// Slice of the current buffer content
pub fn as_slice(&self) -> &[u8] {
&self.buffer[..]
}
/// Get the total number of bytes written.
///
/// # Returns
///
/// Number of bytes written
pub fn bytes_written(&self) -> usize {
self.bytes_written
}
/// Get the current buffer capacity.
///
/// # Returns
///
/// Current buffer capacity in bytes
pub fn capacity(&self) -> usize {
self.buffer.capacity()
}
/// Get the current buffer length.
///
/// # Returns
///
/// Current buffer length in bytes
pub fn len(&self) -> usize {
self.buffer.len()
}
/// Check if the buffer is empty.
///
/// # Returns
///
/// `true` if buffer is empty, `false` otherwise
pub fn is_empty(&self) -> bool {
self.buffer.is_empty()
}
/// Clear the buffer, resetting it to empty.
///
/// This does not change the capacity, just resets the length to 0.
pub fn clear(&mut self) {
self.buffer.clear();
self.bytes_written = 0;
self.finalized = false;
}
/// Reserve additional capacity in the buffer.
///
/// # Arguments
///
/// * `additional` - Additional capacity to reserve
pub fn reserve(&mut self, additional: usize) {
self.buffer.reserve(additional);
}
}
impl Default for BytesMutWriter {
fn default() -> Self {
Self::new()
}
}
impl std::fmt::Debug for BytesMutWriter {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("BytesMutWriter")
.field("buffer_len", &self.buffer.len())
.field("buffer_capacity", &self.buffer.capacity())
.field("bytes_written", &self.bytes_written)
.field("finalized", &self.finalized)
.finish()
}
}
/// AsyncWrite implementation for BytesMutWriter.
///
/// This allows the writer to be used with tokio's async I/O utilities.
impl AsyncWrite for BytesMutWriter {
fn poll_write(mut self: Pin<&mut Self>, _cx: &mut Context<'_>, buf: &[u8]) -> Poll<Result<usize, tokio::io::Error>> {
if self.finalized {
return Poll::Ready(Err(tokio::io::Error::new(
tokio::io::ErrorKind::WriteZero,
"Cannot write to finalized writer",
)));
}
let len = buf.len();
self.buffer.put_slice(buf);
self.bytes_written += len;
Poll::Ready(Ok(len))
}
fn poll_flush(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll<Result<(), tokio::io::Error>> {
// Nothing to flush for in-memory buffer
Poll::Ready(Ok(()))
}
fn poll_shutdown(mut self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll<Result<(), tokio::io::Error>> {
self.finalized = true;
Poll::Ready(Ok(()))
}
}
/// Zero-copy write error types.
#[derive(Debug, thiserror::Error)]
pub enum ZeroCopyWriteError {
/// I/O error occurred
#[error("I/O error: {0}")]
Io(#[from] tokio::io::Error),
/// Writer has been finalized and cannot accept more writes
#[error("Writer finalized: {0}")]
Finalized(String),
/// Invalid input provided
#[error("Invalid input: {0}")]
InvalidInput(String),
}
#[cfg(test)]
mod tests {
use super::*;
#[tokio::test]
async fn test_new_writer() {
let writer = BytesMutWriter::new();
assert!(writer.is_empty());
assert_eq!(writer.bytes_written(), 0);
assert!(writer.capacity() >= 8 * 1024);
}
#[tokio::test]
async fn test_write_buffered() {
let mut writer = BytesMutWriter::new();
let data = Bytes::from("hello world");
let written = writer.write_buffered(data).await.unwrap();
assert_eq!(written, 11);
assert_eq!(writer.bytes_written(), 11);
assert_eq!(writer.as_slice(), b"hello world");
}
#[tokio::test]
async fn test_preferred_writer_alias() {
let mut writer = BytesMutWriter::new();
let written = writer
.write_buffered(Bytes::from("hello world"))
.await
.expect("write bytes through alias");
assert_eq!(written, 11);
assert_eq!(writer.as_slice(), b"hello world");
}
#[tokio::test]
async fn test_write_slice() {
let mut writer = BytesMutWriter::new();
let data = b"hello world";
let written = writer.write_slice(data).await.unwrap();
assert_eq!(written, 11);
assert_eq!(writer.bytes_written(), 11);
assert_eq!(writer.as_slice(), b"hello world");
}
#[tokio::test]
async fn test_into_bytes() {
let mut writer = BytesMutWriter::new();
let data = Bytes::from("hello world");
writer.write_buffered(data).await.unwrap();
let result = writer.into_bytes();
assert_eq!(result.as_ref(), b"hello world");
}
#[tokio::test]
async fn test_write_after_finalize() {
let mut writer = BytesMutWriter::new();
let data = Bytes::from("hello");
writer.write_buffered(data).await.unwrap();
let _result = writer.into_bytes();
// Create new writer and try to write after finalize
let mut writer2 = BytesMutWriter::new();
writer2.write_buffered(Bytes::from("test")).await.unwrap();
let _ = writer2.into_bytes();
// Writing to a consumed writer should work via new writer
let mut writer3 = BytesMutWriter::new();
let result = writer3.write_buffered(Bytes::from("final")).await;
assert!(result.is_ok());
}
#[tokio::test]
async fn test_clear() {
let mut writer = BytesMutWriter::new();
writer.write_slice(b"hello").await.unwrap();
writer.clear();
assert!(writer.is_empty());
assert_eq!(writer.bytes_written(), 0);
// Capacity should remain
assert!(writer.capacity() > 0);
}
#[tokio::test]
async fn test_reserve() {
let mut writer = BytesMutWriter::with_capacity(10);
let initial_capacity = writer.capacity();
writer.reserve(1000);
// Reserve ensures at least the additional capacity can be added
// but may allocate more than requested
assert!(writer.capacity() >= initial_capacity);
}
#[tokio::test]
async fn test_multiple_writes() {
let mut writer = BytesMutWriter::new();
writer.write_buffered(Bytes::from("hello ")).await.unwrap();
writer.write_slice(b"world").await.unwrap();
assert_eq!(writer.as_slice(), b"hello world");
assert_eq!(writer.bytes_written(), 11);
}
#[tokio::test]
async fn test_async_write() {
use tokio::io::AsyncWriteExt;
let mut writer = BytesMutWriter::new();
let data = b"hello world";
let written = writer.write(data).await.unwrap();
assert_eq!(written, 11);
assert_eq!(writer.as_slice(), b"hello world");
}
#[tokio::test]
async fn test_debug() {
let writer = BytesMutWriter::new();
let debug_str = format!("{:?}", writer);
assert!(debug_str.contains("BytesMutWriter"));
assert!(debug_str.contains("buffer_len"));
}
#[tokio::test]
#[allow(deprecated)]
async fn test_legacy_writer_alias() {
let mut writer = ZeroCopyObjectWriter::new();
let written = writer.write_zero_copy(Bytes::from("hello")).await.unwrap();
assert_eq!(written, 5);
assert_eq!(writer.as_slice(), b"hello");
}
}
+4 -24
View File
@@ -1729,30 +1729,10 @@ mod tests {
assert!(config.validate().is_ok(), "deprecated mount_path must not be required");
}
#[test]
fn test_vault_kv2_sources_do_not_claim_transit_wrapping() {
let sources = [
("config.rs", include_str!("config.rs")),
("api_types.rs", include_str!("api_types.rs")),
("backends/vault.rs", include_str!("backends/vault.rs")),
("lib.rs", include_str!("lib.rs")),
];
// Assemble the needles at runtime so this guard does not match its own source.
let needles = [
format!("wrapping via {}", "Transit"),
format!("KV v2 + {}", "Transit"),
format!("KV2+{}", "Transit"),
format!("you would use Vault's {} engine", "transit"),
];
for (name, source) in sources {
for needle in &needles {
assert!(
!source.contains(needle.as_str()),
"{name} still describes the Vault KV2 backend with `{needle}`"
);
}
}
}
// The "VaultKv2 must not claim Transit wrapping" documentation-claim
// invariant is enforced by scripts/check_fips_wording.sh, which scans every
// file in crates/kms rather than a fixed include_str! list
// (rustfs/backlog#1884).
#[test]
fn test_legacy_persisted_vault_transit_config_uses_metadata_defaults() {
+282 -69
View File
@@ -15,8 +15,10 @@
use std::collections::HashMap;
use std::hash::{Hash, Hasher};
use std::sync::Arc;
use std::sync::atomic::{AtomicBool, Ordering};
use std::time::{Duration, SystemTime};
use tokio::sync::RwLock;
use tokio::time::Instant;
use crate::{
FastLockGuard, GlobalLockManager, LockClient, LockId, LockInfo, LockManager, LockMetadata, LockPriority, LockRequest,
@@ -26,43 +28,51 @@ use crate::{
/// Default shard count for guard storage (must be power of 2)
const DEFAULT_GUARD_SHARD_COUNT: usize = 64;
type GuardShard = Arc<RwLock<HashMap<LockId, LocalGuardEntry>>>;
type GuardStorage = Arc<Vec<GuardShard>>;
/// Local lock client using FastLock with sharded guard storage for better concurrency
#[derive(Debug)]
pub struct LocalClient {
/// Sharded guard storage to reduce lock contention
guard_storage: Vec<Arc<RwLock<HashMap<LockId, LocalGuardEntry>>>>,
guard_storage: GuardStorage,
/// Mask for fast shard index calculation (shard_count - 1)
shard_mask: usize,
/// Optional lock manager (if None, uses global singleton)
manager: Option<Arc<GlobalLockManager>>,
reaper_started: AtomicBool,
reaper_interval: Duration,
}
#[derive(Debug)]
struct LocalGuardEntry {
guard: FastLockGuard,
expires_at: SystemTime,
deadline: Instant,
ttl: Duration,
/// Owner recorded at acquire time; used only for reclaim diagnostics (#899).
owner: String,
}
impl LocalGuardEntry {
fn new(guard: FastLockGuard, ttl: Duration, owner: String) -> Self {
fn new(guard: FastLockGuard, ttl: Duration) -> Self {
let now = SystemTime::now();
let monotonic_now = Instant::now();
Self {
guard,
expires_at: now + ttl,
expires_at: now.checked_add(ttl).unwrap_or(now),
deadline: monotonic_now.checked_add(ttl).unwrap_or(monotonic_now),
ttl,
owner,
}
}
fn is_expired(&self) -> bool {
self.expires_at <= SystemTime::now()
self.deadline <= Instant::now()
}
fn refresh(&mut self) {
self.expires_at = SystemTime::now() + self.ttl;
let now = SystemTime::now();
let monotonic_now = Instant::now();
self.expires_at = now.checked_add(self.ttl).unwrap_or(now);
self.deadline = monotonic_now.checked_add(self.ttl).unwrap_or(monotonic_now);
}
}
@@ -77,26 +87,38 @@ impl LocalClient {
pub fn with_shard_count(shard_count: usize) -> Self {
assert!(shard_count.is_power_of_two(), "Shard count must be power of 2");
let guard_storage: Vec<Arc<RwLock<HashMap<LockId, LocalGuardEntry>>>> =
(0..shard_count).map(|_| Arc::new(RwLock::new(HashMap::new()))).collect();
let guard_storage: Vec<GuardShard> = (0..shard_count).map(|_| Arc::new(RwLock::new(HashMap::new()))).collect();
Self::with_storage(Arc::new(guard_storage), None, crate::fast_lock::CLEANUP_INTERVAL)
}
fn with_storage(guard_storage: GuardStorage, manager: Option<Arc<GlobalLockManager>>, reaper_interval: Duration) -> Self {
let shard_count = guard_storage.len();
debug_assert!(shard_count.is_power_of_two());
Self {
guard_storage,
shard_mask: shard_count - 1,
manager: None,
manager,
reaper_started: AtomicBool::new(false),
reaper_interval,
}
}
/// Create new local client with a specific lock manager
/// This allows simulating multi-node environments where each node has its own lock backend
pub fn with_manager(manager: Arc<GlobalLockManager>) -> Self {
Self {
guard_storage: (0..DEFAULT_GUARD_SHARD_COUNT)
.map(|_| Arc::new(RwLock::new(HashMap::new())))
.collect(),
shard_mask: DEFAULT_GUARD_SHARD_COUNT - 1,
manager: Some(manager),
}
let guard_storage = (0..DEFAULT_GUARD_SHARD_COUNT)
.map(|_| Arc::new(RwLock::new(HashMap::new())))
.collect();
Self::with_storage(Arc::new(guard_storage), Some(manager), crate::fast_lock::CLEANUP_INTERVAL)
}
#[cfg(test)]
pub(crate) fn with_manager_and_reaper_interval(manager: Arc<GlobalLockManager>, reaper_interval: Duration) -> Self {
let guard_storage = (0..DEFAULT_GUARD_SHARD_COUNT)
.map(|_| Arc::new(RwLock::new(HashMap::new())))
.collect();
Self::with_storage(Arc::new(guard_storage), Some(manager), reaper_interval)
}
/// Get the lock manager (injected manager if available, otherwise global singleton)
@@ -118,52 +140,63 @@ impl LocalClient {
}
async fn reclaim_expired_guards_for_resource(&self, resource: &crate::ObjectKey) -> usize {
let mut reclaimed = 0usize;
let expired_entries = Self::extract_expired_guards(&self.guard_storage, Some(resource)).await;
Self::release_reclaimed_guards(expired_entries, Some(resource))
}
for shard in &self.guard_storage {
let expired_entries = {
let mut guards = shard.write().await;
let mut retained = HashMap::with_capacity(guards.len());
let mut expired_entries = Vec::new();
async fn extract_expired_guards(storage: &GuardStorage, resource: Option<&crate::ObjectKey>) -> Vec<LocalGuardEntry> {
let mut expired_entries = Vec::new();
for shard in storage.iter() {
let mut guards = shard.write().await;
expired_entries.extend(
guards
.extract_if(|lock_id, entry| {
resource.is_none_or(|resource| &lock_id.resource == resource) && entry.is_expired()
})
.map(|(_, entry)| entry),
);
}
expired_entries
}
for (lock_id, entry) in std::mem::take(&mut *guards) {
if &lock_id.resource == resource && entry.is_expired() {
expired_entries.push(entry);
} else {
retained.insert(lock_id, entry);
}
}
*guards = retained;
expired_entries
};
for mut entry in expired_entries {
// An expired entry whose owner never refreshed it (a dead coordinator, #698) is
// reclaimed so a live contender can re-form quorum. With guard heartbeats in place
// (#899) a live owner keeps its entry from expiring, so reaching here means the
// lease genuinely lapsed. Surface it for observability; the reclaim decision itself
// is unchanged.
let since_last_refresh = entry
.expires_at
.checked_sub(entry.ttl)
.and_then(|last_refresh| SystemTime::now().duration_since(last_refresh).ok())
.unwrap_or(entry.ttl);
tracing::warn!(
owner = %entry.owner,
resource = %resource,
ttl_ms = entry.ttl.as_millis() as u64,
since_last_refresh_ms = since_last_refresh.as_millis() as u64,
"reclaiming expired lock guard whose lease was not refreshed"
);
rustfs_io_metrics::record_lock_reclaimed();
let _ = entry.guard.release();
reclaimed = reclaimed.saturating_add(1);
fn release_reclaimed_guards(
entries: impl IntoIterator<Item = LocalGuardEntry>,
resource: Option<&crate::ObjectKey>,
) -> usize {
let mut reclaimed = 0;
for mut entry in entries {
let _ = entry.guard.release();
rustfs_io_metrics::record_lock_reclaimed();
reclaimed += 1;
}
if reclaimed > 0 {
if let Some(resource) = resource {
tracing::debug!(event = "lock_guard_reclaimed", resource = %resource, count = reclaimed, "expired lock guards reclaimed");
} else {
tracing::debug!(event = "lock_guard_reaper_sweep", count = reclaimed, "expired lock guards reclaimed");
}
}
reclaimed
}
fn ensure_reaper(&self) {
if self.reaper_started.swap(true, Ordering::AcqRel) {
return;
}
let storage = Arc::downgrade(&self.guard_storage);
let interval = self.reaper_interval;
tokio::spawn(async move {
let mut ticker = tokio::time::interval(interval);
loop {
ticker.tick().await;
let Some(storage) = storage.upgrade() else {
break;
};
let expired_entries = Self::extract_expired_guards(&storage, None).await;
Self::release_reclaimed_guards(expired_entries, None);
}
});
}
}
impl Default for LocalClient {
@@ -175,28 +208,36 @@ impl Default for LocalClient {
#[async_trait::async_trait]
impl LockClient for LocalClient {
async fn acquire_lock(&self, request: &LockRequest) -> Result<LockResponse> {
self.ensure_reaper();
let lock_manager = self.get_lock_manager();
let reclaimed_before_acquire = self.reclaim_expired_guards_for_resource(&request.resource).await;
let acquire_deadline = Instant::now()
.checked_add(request.acquire_timeout)
.unwrap_or_else(Instant::now);
let build_lock_request = || match request.lock_type {
let build_lock_request = |acquire_timeout| match request.lock_type {
LockType::Exclusive => crate::ObjectLockRequest::new_write(request.resource.clone(), request.owner.clone())
.with_acquire_timeout(request.acquire_timeout),
.with_acquire_timeout(acquire_timeout),
LockType::Shared => crate::ObjectLockRequest::new_read(request.resource.clone(), request.owner.clone())
.with_acquire_timeout(request.acquire_timeout),
.with_acquire_timeout(acquire_timeout),
};
let mut retried_after_reclaim = reclaimed_before_acquire > 0;
loop {
match lock_manager.acquire_lock(build_lock_request()).await {
let remaining = acquire_deadline.saturating_duration_since(Instant::now());
if remaining.is_zero() {
return Ok(LockResponse::failure("Lock acquisition timeout", request.acquire_timeout));
}
match lock_manager.acquire_lock(build_lock_request(remaining)).await {
Ok(guard) => {
let lock_id = request.lock_id.clone();
let acquired_at = SystemTime::now();
let expires_at = acquired_at + request.ttl;
let expires_at = acquired_at.checked_add(request.ttl).unwrap_or(acquired_at);
{
let shard = self.get_shard(&lock_id);
let mut guards = shard.write().await;
guards.insert(lock_id.clone(), LocalGuardEntry::new(guard, request.ttl, request.owner.clone()));
guards.insert(lock_id.clone(), LocalGuardEntry::new(guard, request.ttl));
}
let lock_info = LockInfo {
@@ -256,12 +297,24 @@ impl LockClient for LocalClient {
async fn refresh(&self, lock_id: &LockId) -> Result<bool> {
let shard = self.get_shard(lock_id);
let mut guards = shard.write().await;
if let Some(entry) = guards.get_mut(lock_id) {
entry.refresh();
Ok(true)
} else {
let expired_entry = {
let mut guards = shard.write().await;
let Some(entry) = guards.get_mut(lock_id) else {
return Ok(false);
};
if entry.is_expired() {
guards.remove(lock_id)
} else {
entry.refresh();
None
}
};
if let Some(entry) = expired_entry {
Self::release_reclaimed_guards([entry], Some(&lock_id.resource));
Ok(false)
} else {
Ok(true)
}
}
@@ -317,3 +370,163 @@ impl LockClient for LocalClient {
true
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::{GlobalLockManager, LockClient, LockRequest, LockType};
fn request(resource: crate::ObjectKey, owner: &str, ttl: Duration) -> LockRequest {
LockRequest::new(resource, LockType::Exclusive, owner)
.with_ttl(ttl)
.with_acquire_timeout(Duration::from_millis(80))
}
async fn wait_until_reaped(client: &LocalClient, lock_id: &LockId) {
for _ in 0..80 {
if client.check_status(lock_id).await.unwrap().is_none() {
return;
}
tokio::time::sleep(Duration::from_millis(5)).await;
}
panic!("lock guard was not reaped before test deadline");
}
#[tokio::test(flavor = "current_thread")]
async fn expired_guard_is_reaped_without_resource_reacquire() {
let manager = Arc::new(GlobalLockManager::new());
let client = LocalClient::with_manager_and_reaper_interval(manager.clone(), Duration::from_millis(5));
let request = request(crate::ObjectKey::new("bucket", "unique-chunk"), "owner-a", Duration::from_millis(10));
let lock_id = request.lock_id.clone();
assert!(client.acquire_lock(&request).await.unwrap().success);
assert!(client.check_status(&lock_id).await.unwrap().is_some());
tokio::time::sleep(Duration::from_millis(15)).await;
wait_until_reaped(&client, &lock_id).await;
let direct = manager
.acquire_lock(crate::ObjectLockRequest::new_write(request.resource.clone(), "owner-b"))
.await;
assert!(direct.is_ok());
}
#[tokio::test(flavor = "current_thread")]
async fn sibling_client_cannot_reclaim_but_owner_reaper_releases_shared_lock() {
let manager = Arc::new(GlobalLockManager::new());
let owner = LocalClient::with_manager_and_reaper_interval(manager.clone(), Duration::from_millis(5));
let contender = LocalClient::with_manager_and_reaper_interval(manager, Duration::from_millis(5));
let request_a = request(crate::ObjectKey::new("bucket", "shared-resource"), "owner-a", Duration::from_millis(10));
assert!(owner.acquire_lock(&request_a).await.unwrap().success);
let request_b = request(request_a.resource.clone(), "owner-b", Duration::from_millis(20))
.with_acquire_timeout(Duration::from_millis(5));
assert!(!contender.acquire_lock(&request_b).await.unwrap().success);
tokio::time::sleep(Duration::from_millis(25)).await;
assert!(owner.check_status(&request_a.lock_id).await.unwrap().is_none());
assert!(contender.acquire_lock(&request_b).await.unwrap().success);
}
#[tokio::test(flavor = "current_thread")]
async fn refresh_wins_before_deadline_and_reaper_wins_after_deadline() {
let manager = Arc::new(GlobalLockManager::new());
let client = LocalClient::with_manager_and_reaper_interval(manager, Duration::from_millis(5));
let request = request(crate::ObjectKey::new("bucket", "refresh-race"), "owner-a", Duration::from_millis(25));
let lock_id = request.lock_id.clone();
assert!(client.acquire_lock(&request).await.unwrap().success);
tokio::time::sleep(Duration::from_millis(10)).await;
assert!(client.refresh(&lock_id).await.unwrap());
tokio::time::sleep(Duration::from_millis(15)).await;
assert!(client.check_status(&lock_id).await.unwrap().is_some());
wait_until_reaped(&client, &lock_id).await;
}
#[tokio::test(start_paused = true)]
async fn refresh_after_expiry_releases_guard_without_reviving_it() {
let manager = Arc::new(GlobalLockManager::new());
let client = LocalClient::with_manager_and_reaper_interval(manager, Duration::from_secs(60));
client.reaper_started.store(true, Ordering::Release);
let lock_request = request(
crate::ObjectKey::new("bucket", "refresh-after-expiry"),
"owner-a",
Duration::from_secs(10),
);
let lock_id = lock_request.lock_id.clone();
assert!(
client
.acquire_lock(&lock_request)
.await
.expect("initial owner should acquire the lock")
.success
);
tokio::time::advance(Duration::from_secs(11)).await;
assert!(
!client
.refresh(&lock_id)
.await
.expect("expired refresh should return a result"),
"an expired guard must not be refreshed"
);
assert!(
client
.check_status(&lock_id)
.await
.expect("expired guard status should be readable")
.is_none(),
"expired guard should be removed after refresh"
);
let contender = request(
crate::ObjectKey::new("bucket", "refresh-after-expiry"),
"owner-b",
Duration::from_secs(10),
);
assert!(
client
.acquire_lock(&contender)
.await
.expect("contender should receive an acquisition result")
.success,
"released guard must be acquirable by a new owner"
);
}
#[tokio::test(flavor = "current_thread")]
async fn zero_ttl_is_reaped_and_oversized_ttl_does_not_panic() {
let manager = Arc::new(GlobalLockManager::new());
let client = LocalClient::with_manager_and_reaper_interval(manager, Duration::from_millis(5));
let zero = request(crate::ObjectKey::new("bucket", "zero-ttl"), "owner-zero", Duration::ZERO);
let zero_id = zero.lock_id.clone();
assert!(client.acquire_lock(&zero).await.unwrap().success);
wait_until_reaped(&client, &zero_id).await;
let huge = request(crate::ObjectKey::new("bucket", "huge-ttl"), "owner-huge", Duration::MAX);
let huge_id = huge.lock_id.clone();
assert!(client.acquire_lock(&huge).await.unwrap().success);
wait_until_reaped(&client, &huge_id).await;
}
#[tokio::test(flavor = "current_thread")]
async fn acquire_retry_preserves_total_deadline() {
let manager = Arc::new(GlobalLockManager::new());
let client = LocalClient::with_manager_and_reaper_interval(manager, Duration::from_secs(60));
let first = request(crate::ObjectKey::new("bucket", "deadline-budget"), "owner-a", Duration::from_millis(10));
assert!(client.acquire_lock(&first).await.unwrap().success);
let second =
request(first.resource.clone(), "owner-b", Duration::from_millis(30)).with_acquire_timeout(Duration::from_millis(60));
let started = Instant::now();
let response = client.acquire_lock(&second).await.unwrap();
assert!(!response.success, "the first attempt consumed the caller's acquire budget");
assert!(
started.elapsed() < Duration::from_millis(100),
"reclaim retry must not double the acquire budget"
);
let recovered = client.acquire_lock(&second).await.unwrap();
assert!(recovered.success, "the reclaimed guard must be available to the next request");
}
}
+111
View File
@@ -840,6 +840,117 @@ async fn test_namespace_lock_distributed_reclaims_expired_same_resource_after_fa
);
}
#[tokio::test]
async fn four_node_failed_release_converges_without_replica_repair() {
let managers = (0..4).map(|_| Arc::new(GlobalLockManager::new())).collect::<Vec<_>>();
let flaky_clients = managers
.iter()
.map(|manager| {
Arc::new(FlakyReleaseClient {
inner: LocalClient::with_manager_and_reaper_interval(manager.clone(), Duration::from_millis(5)),
failed_releases_remaining: AtomicUsize::new(usize::MAX),
release_attempts: AtomicUsize::new(0),
})
})
.collect::<Vec<_>>();
let clients = flaky_clients
.iter()
.map(|client| client.clone() as Arc<dyn LockClient>)
.collect::<Vec<_>>();
let lock = NamespaceLock::Distributed(DistributedLock::new("four-node-expired-lease".to_string(), clients, 3));
let resource = create_test_object_key("bucket", "object-four-node-expired");
let request = LockRequest::new(resource.clone(), LockType::Exclusive, "owner-a")
.with_acquire_timeout(Duration::from_millis(300))
.with_ttl(Duration::from_millis(40));
let mut guard = lock
.acquire_guard(&request)
.await
.expect("initial acquire should not error")
.expect("initial acquire should reach quorum");
assert!(guard.release(), "release should be acknowledged while RPC cleanup is pending");
for _ in 0..40 {
if flaky_clients.iter().all(|client| client.release_attempts() >= 3) {
break;
}
tokio::time::sleep(Duration::from_millis(5)).await;
}
let deadline = tokio::time::Instant::now() + Duration::from_secs(2);
loop {
let all_reaped =
futures::future::join_all(flaky_clients.iter().map(|client| client.inner.check_status(&request.lock_id)))
.await
.into_iter()
.all(|status| status.expect("status should not error").is_none());
if all_reaped {
break;
}
assert!(tokio::time::Instant::now() < deadline, "all four local lease entries must converge");
tokio::time::sleep(Duration::from_millis(10)).await;
}
for suffix in ["chunk-0", "chunk-1", ".rustfs.sys/multipart/upload-0"] {
for client in &flaky_clients {
let orphan = LockRequest::new(create_test_object_key("bucket", suffix), LockType::Exclusive, "orphan")
.with_ttl(Duration::from_millis(25));
assert!(client.inner.acquire_lock(&orphan).await.expect("orphan acquire").success);
}
}
tokio::time::sleep(Duration::from_millis(80)).await;
let recovered = lock
.acquire_guard(
&LockRequest::new(resource, LockType::Exclusive, "owner-b")
.with_acquire_timeout(Duration::from_millis(300))
.with_ttl(Duration::from_millis(40)),
)
.await
.expect("recovery acquire should not error")
.expect("four-node quorum should recover after local reapers run");
drop(recovered);
}
#[tokio::test]
async fn four_node_stale_quorum_contention_respects_acquire_deadline() {
let managers = (0..4).map(|_| Arc::new(GlobalLockManager::new())).collect::<Vec<_>>();
let node_clients = managers
.iter()
.map(|manager| Arc::new(LocalClient::with_manager_and_reaper_interval(manager.clone(), Duration::from_millis(5))))
.collect::<Vec<_>>();
let resource = create_test_object_key("bucket", "stale-quorum");
let stale = LockRequest::new(resource.clone(), LockType::Exclusive, "stale-owner").with_ttl(Duration::from_millis(180));
for client in &node_clients {
assert!(client.acquire_lock(&stale).await.expect("stale acquire").success);
}
let clients = node_clients
.iter()
.map(|client| client.clone() as Arc<dyn LockClient>)
.collect::<Vec<_>>();
let lock = NamespaceLock::Distributed(DistributedLock::new("stale-quorum-deadline".to_string(), clients, 3));
let contender = LockRequest::new(resource.clone(), LockType::Exclusive, "new-owner")
.with_acquire_timeout(Duration::from_millis(150))
.with_ttl(Duration::from_millis(100));
let started = tokio::time::Instant::now();
let response = lock.acquire_guard(&contender).await.expect("contention should not error");
assert!(response.is_none(), "unexpired leases must not be force-reclaimed");
assert!(started.elapsed() < Duration::from_millis(350), "acquire must respect its deadline");
tokio::time::sleep(Duration::from_millis(80)).await;
let recovered = lock
.acquire_guard(
&LockRequest::new(resource, LockType::Exclusive, "new-owner")
.with_acquire_timeout(Duration::from_millis(300))
.with_ttl(Duration::from_millis(100)),
)
.await
.expect("post-expiry acquire should not error")
.expect("quorum should recover after local reapers clear stale leases");
drop(recovered);
}
#[tokio::test]
async fn test_namespace_lock_distributed_retries_transient_acquire_timeout() {
let managers = (0..3).map(|_| Arc::new(GlobalLockManager::new())).collect::<Vec<_>>();
@@ -427,7 +427,6 @@ pub enum DataSource {
/// Write triggered
WriteTriggered,
/// Fallback value
#[allow(dead_code)]
Fallback,
}
@@ -603,7 +602,6 @@ impl WriteRecord {
/// Hybrid strategy configuration
#[derive(Debug, Clone)]
#[allow(dead_code)]
pub struct HybridStrategyConfig {
/// Scheduled update interval
pub scheduled_update_interval: Duration,
@@ -998,14 +996,12 @@ impl HybridCapacityManager {
}
/// Get cache age
#[allow(dead_code)]
pub async fn get_cache_age(&self) -> Option<Duration> {
let cache = self.cache.read().await;
cache.as_ref().map(|c| c.last_update.elapsed())
}
/// Get write frequency (writes/minute)
#[allow(dead_code)]
pub async fn get_write_frequency(&self) -> usize {
let record = &self.write_record;
record.recent_write_count(record.monotonic_second())
@@ -1300,7 +1296,6 @@ pub fn get_capacity_manager() -> Arc<HybridCapacityManager> {
/// .update_capacity(CapacityUpdate::exact(1000, 0), DataSource::RealTime)
/// .await;
/// ```
#[allow(dead_code)]
pub fn create_isolated_manager(config: HybridStrategyConfig) -> Arc<HybridCapacityManager> {
Arc::new(HybridCapacityManager::new(config))
}
+66 -10
View File
@@ -177,16 +177,38 @@ impl StartCommand {
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(tag = "action", rename_all = "snake_case", deny_unknown_fields)]
pub enum Command {
Start { request: StartCommand },
Query { heal_path: String, client_token: String },
Cancel { heal_path: String, client_token: String },
Start {
request: StartCommand,
},
Query {
heal_path: String,
client_token: String,
/// Incremental result cursor (HS-06): only items with a sequence
/// greater than this are returned. Absent = legacy full snapshot.
/// Optional + defaulted so older peers stay wire-compatible.
#[serde(default, skip_serializing_if = "Option::is_none")]
since_seq: Option<u64>,
},
Cancel {
heal_path: String,
client_token: String,
},
}
#[derive(Debug)]
pub enum ExecutableCommand {
Start { request: HealChannelRequest },
Query { heal_path: String, client_token: String },
Cancel { heal_path: String, client_token: String },
Start {
request: HealChannelRequest,
},
Query {
heal_path: String,
client_token: String,
since_seq: Option<u64>,
},
Cancel {
heal_path: String,
client_token: String,
},
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
@@ -227,8 +249,22 @@ impl Envelope {
)
}
pub fn query(request_id: String, metadata: RequestMetadata, heal_path: String, client_token: String) -> Result<Self, String> {
Self::new(request_id, metadata, Command::Query { heal_path, client_token })
pub fn query(
request_id: String,
metadata: RequestMetadata,
heal_path: String,
client_token: String,
since_seq: Option<u64>,
) -> Result<Self, String> {
Self::new(
request_id,
metadata,
Command::Query {
heal_path,
client_token,
since_seq,
},
)
}
pub fn cancel(
@@ -286,7 +322,15 @@ impl Envelope {
Command::Start { request } => ExecutableCommand::Start {
request: request.into_channel_request(self.request_id.clone())?,
},
Command::Query { heal_path, client_token } => ExecutableCommand::Query { heal_path, client_token },
Command::Query {
heal_path,
client_token,
since_seq,
} => ExecutableCommand::Query {
heal_path,
client_token,
since_seq,
},
Command::Cancel { heal_path, client_token } => ExecutableCommand::Cancel { heal_path, client_token },
};
Ok((self.request_id, self.coordinator_epoch, command))
@@ -305,6 +349,11 @@ pub enum Admission {
Full,
DroppedQueueFull,
DroppedPolicy,
/// HS-06: admin start rejected because the same target is already being
/// healed (RUSTFS_HEAL_OVERLAP_POLICY=minio_error only).
DroppedAlreadyRunning,
/// HS-06: admin start rejected because its path overlaps an active heal.
DroppedOverlappingPaths,
}
impl From<HealAdmissionResult> for Admission {
@@ -315,6 +364,8 @@ impl From<HealAdmissionResult> for Admission {
HealAdmissionResult::Full => Self::Full,
HealAdmissionResult::Dropped(HealAdmissionDropReason::QueueFull) => Self::DroppedQueueFull,
HealAdmissionResult::Dropped(HealAdmissionDropReason::PolicyDropped) => Self::DroppedPolicy,
HealAdmissionResult::Dropped(HealAdmissionDropReason::AlreadyRunning) => Self::DroppedAlreadyRunning,
HealAdmissionResult::Dropped(HealAdmissionDropReason::OverlappingPaths) => Self::DroppedOverlappingPaths,
}
}
}
@@ -331,6 +382,8 @@ impl Admission {
Self::Full => HealAdmissionResult::Full,
Self::DroppedQueueFull => HealAdmissionResult::Dropped(HealAdmissionDropReason::QueueFull),
Self::DroppedPolicy => HealAdmissionResult::Dropped(HealAdmissionDropReason::PolicyDropped),
Self::DroppedAlreadyRunning => HealAdmissionResult::Dropped(HealAdmissionDropReason::AlreadyRunning),
Self::DroppedOverlappingPaths => HealAdmissionResult::Dropped(HealAdmissionDropReason::OverlappingPaths),
}
}
}
@@ -592,6 +645,7 @@ mod tests {
metadata(2, 7),
"bucket/prefix".to_string(),
"token".to_string(),
None,
)
.unwrap();
let cancel = Envelope::cancel(
@@ -667,6 +721,7 @@ mod tests {
RequestMetadata::new([0x11; 16], 1_700_000_000_000, 1_700_000_030_000, 9),
"bucket/prefix".to_string(),
"client-token".to_string(),
None,
)
.unwrap();
let cancel = Envelope::cancel(
@@ -749,7 +804,7 @@ mod tests {
assert!(Envelope::start(test_request(request_id.clone()), metadata(0, 7)).is_err());
assert!(Envelope::start(test_request(request_id.clone()), metadata(1, 0)).is_err());
assert!(Envelope::start(test_request(request_id.clone()), RequestMetadata::new([1; 16], 1_000, 31_001, 7),).is_err());
assert!(Envelope::query(request_id.clone(), metadata(1, 7), String::new(), String::new()).is_err());
assert!(Envelope::query(request_id.clone(), metadata(1, 7), String::new(), String::new(), None).is_err());
assert!(Envelope::cancel(request_id.clone(), metadata(1, 7), String::new(), String::new()).is_ok());
let mut noncanonical_request = test_request(request_id.to_uppercase());
@@ -782,6 +837,7 @@ mod tests {
metadata(1, 7),
"x".repeat(ENVELOPE_MAX_SIZE),
"token".to_string(),
None,
)
.unwrap();
let error = super::encode_envelope(&oversized).unwrap_err();
+1 -1
View File
@@ -2557,7 +2557,7 @@ mod tests {
fn production_source(source: &'static str, file_name: &str) -> &'static str {
source
.split("\n#[cfg(test)]")
.split("\n#[cfg(test)]\nmod tests")
.next()
.unwrap_or_else(|| panic!("{file_name} should contain production source before tests"))
}
-50
View File
@@ -49,7 +49,6 @@ pub struct IndexInfo {
pub uncompressed_offset: i64,
}
#[allow(dead_code)]
impl Index {
pub fn new() -> Self {
Self {
@@ -60,14 +59,6 @@ impl Index {
}
}
#[allow(dead_code)]
fn reset(&mut self, max_block: usize) {
self.est_block_uncomp = max_block as i64;
self.total_compressed = -1;
self.total_uncompressed = -1;
self.info.clear();
}
pub fn len(&self) -> usize {
self.info.len()
}
@@ -511,47 +502,6 @@ fn read_varint(buf: &[u8]) -> io::Result<(i64, usize)> {
Err(io::Error::new(io::ErrorKind::UnexpectedEof, "unexpected EOF"))
}
// Helper functions for index header manipulation
#[allow(dead_code)]
pub fn remove_index_headers(b: &[u8]) -> Option<&[u8]> {
if b.len() < 4 + S2_INDEX_TRAILER.len() {
return None;
}
// Skip size
let b = &b[4..];
// Check trailer
if !b.starts_with(S2_INDEX_TRAILER) {
return None;
}
Some(&b[S2_INDEX_TRAILER.len()..])
}
#[allow(dead_code)]
pub fn restore_index_headers(in_data: &[u8]) -> Vec<u8> {
if in_data.is_empty() {
return Vec::new();
}
let mut b = Vec::with_capacity(4 + S2_INDEX_HEADER.len() + in_data.len() + S2_INDEX_TRAILER.len() + 4);
b.extend_from_slice(&[0x50, 0x2A, 0x4D, 0x18]);
b.extend_from_slice(S2_INDEX_HEADER);
b.extend_from_slice(in_data);
let total_size = (b.len() + 4 + S2_INDEX_TRAILER.len()) as u32;
b.extend_from_slice(&total_size.to_le_bytes());
b.extend_from_slice(S2_INDEX_TRAILER);
let chunk_len = b.len() - 4;
b[1] = chunk_len as u8;
b[2] = (chunk_len >> 8) as u8;
b[3] = (chunk_len >> 16) as u8;
b
}
#[cfg(test)]
mod tests {
use super::*;
+59 -3
View File
@@ -715,6 +715,13 @@ async fn get_http_client(url: &str) -> io::Result<Client> {
Ok(cached.client_for(disable_proxy))
}
async fn get_fresh_http_client(url: &str) -> io::Result<Client> {
let tuning = internode_http_client_tuning();
let disable_proxy = should_disable_proxy_for_url(url, tuning);
let outbound_tls = crate::http_runtime_sources::outbound_tls_state().await;
build_http_client(disable_proxy, tuning, &outbound_tls).await
}
fn internode_request_context(method: &Method, url: &str, operation: Option<&'static str>) -> InternodeHttpRequestContext {
let target = reqwest::Url::parse(url)
.ok()
@@ -962,6 +969,28 @@ impl HttpReader {
Self::with_capacity_and_stall_timeout(url, method, headers, body, 0, stall_timeout).await
}
pub async fn new_fresh_connection_with_stall_timeout(
url: String,
method: Method,
headers: HeaderMap,
body: Option<Vec<u8>>,
stall_timeout: Option<Duration>,
) -> io::Result<Self> {
let init = Self::open(&url, &method, &headers, body, stall_timeout, true).await?;
Ok(Self {
inner: StreamReader::new(init.stream),
url,
method,
headers,
track_internode_metrics: init.track_internode_metrics,
internode_operation: init.internode_operation,
stall_timer: None,
stall_timeout: init.stall_timeout,
request_started: init.request_started,
duration_recorded: false,
})
}
/// Create a new HttpReader from a URL. The request is performed immediately.
pub async fn with_capacity(
url: String,
@@ -981,7 +1010,7 @@ impl HttpReader {
_read_buf_size: usize,
stall_timeout: Option<Duration>,
) -> io::Result<Self> {
let init = Self::open(&url, &method, &headers, body, stall_timeout).await?;
let init = Self::open(&url, &method, &headers, body, stall_timeout, false).await?;
Ok(Self {
inner: StreamReader::new(init.stream),
url,
@@ -1002,10 +1031,16 @@ impl HttpReader {
headers: &HeaderMap,
body: Option<Vec<u8>>,
stall_timeout: Option<Duration>,
force_fresh_connection: bool,
) -> io::Result<HttpReaderInit> {
let track_internode_metrics = is_internode_rpc_url(url);
let internode_operation = internode_rpc_operation(url);
let client = get_http_client(url).await.inspect_err(|_| {
let client = if force_fresh_connection {
get_fresh_http_client(url).await
} else {
get_http_client(url).await
}
.inspect_err(|_| {
record_internode_error(track_internode_metrics, internode_operation);
})?;
let mut request: RequestBuilder = client.request(method.clone(), url).headers(headers.clone());
@@ -1121,7 +1156,28 @@ impl HttpChunkReader {
body: Option<Vec<u8>>,
stall_timeout: Option<Duration>,
) -> io::Result<Self> {
let init = HttpReader::open(&url, &method, &headers, body, stall_timeout).await?;
let init = HttpReader::open(&url, &method, &headers, body, stall_timeout, false).await?;
Ok(Self {
inner: init.stream,
current: None,
track_internode_metrics: init.track_internode_metrics,
internode_operation: init.internode_operation,
stall_timer: None,
stall_timeout: init.stall_timeout,
request_started: init.request_started,
duration_recorded: false,
consecutive_empty_chunks: 0,
})
}
pub async fn new_fresh_connection_with_stall_timeout(
url: String,
method: Method,
headers: HeaderMap,
body: Option<Vec<u8>>,
stall_timeout: Option<Duration>,
) -> io::Result<Self> {
let init = HttpReader::open(&url, &method, &headers, body, stall_timeout, true).await?;
Ok(Self {
inner: init.stream,
current: None,
+16
View File
@@ -1770,6 +1770,13 @@ impl FolderScanner {
HealAdmissionResult::Dropped(HealAdmissionDropReason::PolicyDropped) => {
self.clear_pending_scanner_heal(kind, bucket, object, version_id);
}
// Admin-only overlap rejections (HS-06); the scanner never sees
// them, but if it ever does, treat them as terminal like any
// other policy drop rather than endlessly retrying.
HealAdmissionResult::Dropped(HealAdmissionDropReason::AlreadyRunning)
| HealAdmissionResult::Dropped(HealAdmissionDropReason::OverlappingPaths) => {
self.clear_pending_scanner_heal(kind, bucket, object, version_id);
}
}
}
@@ -2478,6 +2485,15 @@ impl FolderScanner {
}
if let GetSizeFailureAction::HealMetadata { object } = failure_action {
// MRF journal intent: durable High-priority Metadata
// heal across restarts (HS-01); the scanner heal
// request below stays as the immediate path.
rustfs_common::mrf_channel::try_send_mrf_intent(
rustfs_common::mrf_channel::MrfKind::MetadataCorruption,
&item.bucket,
&object,
None,
);
self.send_required_scanner_heal_request(
PendingScannerHealKind::Object,
item.bucket.clone(),
@@ -206,7 +206,6 @@ async fn setup_isolated_test_env(init_expiry: bool) -> (Vec<PathBuf>, Arc<ECStor
}
/// Test helper: Create a test bucket
#[allow(dead_code)]
async fn create_test_bucket(ecstore: &Arc<ECStore>, bucket_name: &str) {
(**ecstore)
.make_bucket(bucket_name, &Default::default())
@@ -251,7 +250,6 @@ async fn modeled_versioned_delete_opts(bucket: &str, object: &str) -> ObjectOpti
}
/// Test helper: Set bucket lifecycle configuration
#[allow(dead_code)]
async fn set_bucket_lifecycle(bucket_name: &str) -> Result<(), Box<dyn std::error::Error>> {
// Create a simple lifecycle configuration XML with 0 days expiry for immediate testing
let lifecycle_xml = r#"<?xml version="1.0" encoding="UTF-8"?>
@@ -274,7 +272,6 @@ async fn set_bucket_lifecycle(bucket_name: &str) -> Result<(), Box<dyn std::erro
}
/// Test helper: Set bucket lifecycle configuration
#[allow(dead_code)]
async fn set_bucket_lifecycle_deletemarker(bucket_name: &str) -> Result<(), Box<dyn std::error::Error>> {
// Create lifecycle rule that targets delete-marker cleanup only.
// Keep Expiration.Days unset to avoid expiring live transitioned object versions.
@@ -297,7 +294,6 @@ async fn set_bucket_lifecycle_deletemarker(bucket_name: &str) -> Result<(), Box<
Ok(())
}
#[allow(dead_code)]
async fn set_bucket_lifecycle_delmarker_expiration(bucket_name: &str, days: i64) -> Result<(), Box<dyn std::error::Error>> {
let lifecycle_xml = format!(
r#"<?xml version="1.0" encoding="UTF-8"?>
@@ -320,7 +316,6 @@ async fn set_bucket_lifecycle_delmarker_expiration(bucket_name: &str, days: i64)
Ok(())
}
#[allow(dead_code)]
async fn set_bucket_lifecycle_transition_with_tier(
bucket_name: &str,
storage_class: &str,
@@ -368,7 +363,6 @@ async fn object_exists(ecstore: &Arc<ECStore>, bucket: &str, object: &str) -> bo
}
/// Test helper: Check if object exists
#[allow(dead_code)]
async fn object_is_delete_marker(ecstore: &Arc<ECStore>, bucket: &str, object: &str) -> bool {
if let Ok(oi) = (**ecstore).get_object_info(bucket, object, &ObjectOptions::default()).await {
println!("oi: {oi:?}");
@@ -379,7 +373,6 @@ async fn object_is_delete_marker(ecstore: &Arc<ECStore>, bucket: &str, object: &
}
}
#[allow(dead_code)]
async fn wait_for_object_absence(ecstore: &Arc<ECStore>, bucket: &str, object: &str, timeout: Duration) -> bool {
let deadline = tokio::time::Instant::now() + timeout;
-4
View File
@@ -428,7 +428,6 @@ pub fn parse_url(s: &str) -> Result<ParsedURL, NetError> {
Ok(ParsedURL(uu))
}
#[allow(dead_code)]
pub fn parse_http_url(s: &str) -> Result<ParsedURL, NetError> {
let u = parse_url(s)?;
match u.0.scheme() {
@@ -437,7 +436,6 @@ pub fn parse_http_url(s: &str) -> Result<ParsedURL, NetError> {
}
}
#[allow(dead_code)]
pub fn is_network_or_host_down(err: &std::io::Error, expect_timeouts: bool) -> bool {
if err.kind() == std::io::ErrorKind::TimedOut {
return !expect_timeouts;
@@ -449,12 +447,10 @@ pub fn is_network_or_host_down(err: &std::io::Error, expect_timeouts: bool) -> b
|| err_str.contains("use of closed network connection")
}
#[allow(dead_code)]
pub fn is_conn_reset_err(err: &std::io::Error) -> bool {
err.to_string().contains("connection reset by peer") || matches!(err.raw_os_error(), Some(libc::ECONNRESET))
}
#[allow(dead_code)]
pub fn is_conn_refused_err(err: &std::io::Error) -> bool {
err.to_string().contains("connection refused") || matches!(err.raw_os_error(), Some(libc::ECONNREFUSED))
}
+19 -10
View File
@@ -32,8 +32,8 @@ use arc_swap::ArcSwap;
use async_trait::async_trait;
use hyper_rustls::ConfigBuilderExt;
use rumqttc::{
AsyncClient, Broker, ClientError, ConnectionError, EventLoop, Incoming, MqttOptions, Outgoing, PublishNoticeError, QoS,
Transport, mqttbytes::Error as MqttBytesError,
AsyncClient, Broker, ClientError, ConnectionError, EventLoop, Incoming, MqttOptions, Outgoing, ProtocolViolation,
PublishNoticeError, PublishOptions, QoS, Transport, mqttbytes::Error as MqttBytesError,
};
use rustfs_config::{
EnableState, MQTT_TLS_CA, MQTT_TLS_CLIENT_CERT, MQTT_TLS_CLIENT_KEY, MQTT_TLS_TRUST_LEAF_AS_CA, MQTT_WS_PATH_ALLOWLIST,
@@ -791,7 +791,7 @@ where
.as_ref()
.ok_or_else(|| TargetError::Configuration("MQTT client not initialized".to_string()))?;
let notice = client
.publish_tracked(&self.args.topic, self.args.qos, false, body)
.publish_tracked(&self.args.topic, body, PublishOptions::new(self.args.qos))
.await
.map_err(|error| classify_mqtt_client_error(&error))?;
drop(client_guard);
@@ -1145,7 +1145,7 @@ async fn run_mqtt_event_loop(mut eventloop: EventLoop, connected_status: Arc<Ato
);
connected_status.store(false, Ordering::SeqCst);
}
rumqttc::Event::Incoming(Incoming::PingResp(_)) => {
rumqttc::Event::Incoming(Incoming::PingResp) => {
trace!(target_id = %target_id, "Received PingResp from broker. Connection is alive.");
}
rumqttc::Event::Incoming(Incoming::SubAck(suback)) => {
@@ -1257,7 +1257,11 @@ async fn run_mqtt_event_loop(mut eventloop: EventLoop, connected_status: Arc<Ato
/// copy is preserved and replayed rather than dropped (backlog#971).
fn classify_mqtt_client_error(err: &ClientError) -> TargetError {
match err {
ClientError::Request(_) | ClientError::TryRequest(_) | ClientError::TrackingUnavailable => TargetError::NotConnected,
ClientError::RequestChannelFull(_) | ClientError::RequestChannelDisconnected(_) | ClientError::TrackingUnavailable => {
TargetError::NotConnected
}
ClientError::InvalidRequest(_) => TargetError::Request(format!("Invalid MQTT publish request: {err}")),
_ => TargetError::NotConnected,
}
}
@@ -1270,10 +1274,14 @@ fn classify_mqtt_notice_error(err: &PublishNoticeError) -> TargetError {
PublishNoticeError::Recv
| PublishNoticeError::SessionReset
| PublishNoticeError::Qos0NotFlushed
| PublishNoticeError::BrokerOnlySessionResume
| PublishNoticeError::SessionPersistence(_)
| PublishNoticeError::TopicAliasReplayUnavailable(_) => TargetError::NotConnected,
PublishNoticeError::RetainNotSupported => TargetError::Request(format!("MQTT broker rejected publish: {err}")),
PublishNoticeError::V5PubAck(_) | PublishNoticeError::V5PubRec(_) | PublishNoticeError::V5PubComp(_) => {
TargetError::Request(format!("MQTT broker rejected publish: {err}"))
}
_ => TargetError::NotConnected,
}
}
@@ -1299,12 +1307,13 @@ fn is_fatal_mqtt_error(err: &ConnectionError) -> bool {
| MqttBytesError::MalformedPacket // Package format error
| MqttBytesError::PayloadTooLong // Too long load
| MqttBytesError::PayloadSizeLimitExceeded { .. } // Load size limit exceeded
| MqttBytesError::TopicNotUtf8 // Topic Non-UTF-8 (Serious Agreement Violation)
| MqttBytesError::TopicNotUtf8 { .. } // Topic Non-UTF-8 (Serious Agreement Violation)
)
}
// Others that are fatal StateError variants
rumqttc::StateError::InvalidState // The internal state machine is in invalid state
| rumqttc::StateError::WrongPacket // Agreement Violation: Unexpected Data Packet Received
| rumqttc::StateError::ProtocolViolation(ProtocolViolation::UnexpectedIncomingPacket(_)) // Agreement Violation: Unexpected Data Packet Received
| rumqttc::StateError::ProtocolViolation(_) // Agreement Violation
| rumqttc::StateError::Unsolicited(_) // Agreement Violation: Unsolicited ACK Received
| rumqttc::StateError::CollisionTimeout // Agreement Violation (if this stage occurs)
| rumqttc::StateError::EmptySubscription // Agreement violation (if this stage occurs)
@@ -1727,8 +1736,8 @@ where
mod tests {
use super::{
AsyncClient, ClientError, MQTT_RECONNECT_BACKOFF_MAX, MQTT_RECONNECT_BACKOFF_MIN, MQTTArgs, MQTTTarget, MQTTTlsConfig,
MqttOptions, PublishNoticeError, QoS, QueuedPayloadMeta, classify_mqtt_client_error, classify_mqtt_notice_error,
next_reconnect_backoff, reconnect_supervisor, validate_mqtt_broker_url,
MqttOptions, PublishNoticeError, PublishOptions, QoS, QueuedPayloadMeta, classify_mqtt_client_error,
classify_mqtt_notice_error, next_reconnect_backoff, reconnect_supervisor, validate_mqtt_broker_url,
};
use crate::error::TargetError;
use crate::target::{REDACTED_SECRET, TargetType};
@@ -1794,7 +1803,7 @@ mod tests {
.capacity(1)
.build();
client
.publish("fill", QoS::AtLeastOnce, false, b"fill".as_slice())
.publish("fill", b"fill".as_slice(), PublishOptions::new(QoS::AtLeastOnce))
.await
.expect("first publish should fill the local channel");
*target.client.lock().await = Some(client);
+1 -2
View File
@@ -105,9 +105,8 @@ inventory. Generic function-local names such as `CACHE`, `LOCK`, `INIT`, and
| `USE_STARSHARD_CACHE`, `BUCKET_CACHE_SMALL`, `BUCKET_CACHE_LARGE` | `rustfs/src/storage/ecfs_extend.rs` | Cache or constant / owner-local cache | Bucket validation cache backend selection and cache storage stay private to the ECFS extension owner. |
| `GLOBAL_SSE_DEK_PROVIDER`, `SSE_TEST_LOCK` | `rustfs/src/storage/sse.rs` | Owner-local cache / test state | SSE DEK provider cache and test serialization lock stay private to the SSE owner. |
| `AUTH_FS` | `rustfs/src/storage/access.rs` | Cache or constant / owner-local cache | Authorization tag-condition lookup keeps its filesystem helper private to the access owner. |
| `LOCK_STATS` | `rustfs/src/storage/lock_optimizer.rs` | Process-global owner-local metrics | Lock optimization statistics stay private behind lock optimizer helper APIs. |
| `DEADLOCK_DETECTOR` | `rustfs/src/storage/deadlock_detector.rs` | Process-global owner-local state | Deadlock detector lifecycle state stays private to the storage deadlock detector owner. |
| `CONCURRENCY_MANAGER`, `ACTIVE_GET_REQUESTS`, `ACTIVE_PUT_REQUESTS`, `IO_PRIORITY_METRICS` | `rustfs/src/storage/concurrency/*` | Process-global owner-local scheduler state | Storage concurrency manager, counters, and metrics remain inside the storage concurrency owner boundary. |
| `CONCURRENCY_MANAGER`, `ACTIVE_GET_REQUESTS`, `ACTIVE_PUT_REQUESTS` | `rustfs/src/storage/concurrency/*` | Process-global owner-local scheduler state | Storage concurrency manager and request counters remain inside the storage concurrency owner boundary. |
| `GET_OBJECT_BUFFER_THRESHOLD_WARNED`, `GET_READER_STREAM_BUFFER_SIZE_OVERRIDE`, function-local `ENABLED`, `OBJECT_SEEK_SUPPORT_THRESHOLD`, `OBJECT_SEEK_SUPPORT_CONCURRENCY_THRESHOLDS` | `rustfs/src/app/object_usecase.rs` | Cache or constant / owner-local cache | Object GET/seek tuning caches and warning guards stay private to object usecase helpers. |
| `SUPPORTED_HEADERS` | `rustfs/src/storage/options.rs` | Cache or constant / owner-local constant | Supported-header lookup state stays private to storage option parsing. |
| `AUDIT_TARGET_SPECS`, `NOTIFICATION_TARGET_SPECS` | `rustfs/src/admin/handlers/audit.rs`, `rustfs/src/admin/handlers/event.rs`, `rustfs/src/admin/handlers/plugins_instances.rs` | Cache or constant / owner-local constant | Admin target descriptor tables stay private to their handler owners. |
@@ -53,6 +53,8 @@ Suggested boilerplate when the topic cannot be avoided:
`README.md` and `CHANGELOG.md` currently contain no FIPS-related wording; `scripts/check_fips_wording.sh` is the grep guard for that public baseline. Any future occurrence of the banned strings in either file should be treated as a defect and either removed or brought under the qualifier rule above. This document intentionally contains the terminology needed to define the policy and is not part of that narrow outward-material scan.
The same script carries a second block for the adjacent over-claim: no file under `crates/kms` may describe the Vault KV2 backend as wrapping key material through Vault's Transit engine. `KmsBackend::VaultKv2` stores RustFS-wrapped key material in Vault's KV v2 engine and never calls Transit, so that wording would tell an operator their key material is cryptographically isolated inside Vault when it is not. Use the `VaultTransit` backend when that isolation is the requirement.
## The `rustfs-crypto` `fips` feature: what it actually does
`crates/crypto/Cargo.toml` declares `default = ["crypto", "fips"]`, so the feature is on in every normal build. Its entire effect is **which algorithm the write path selects**; the implementation is RustCrypto either way.
+1 -1
View File
@@ -63,7 +63,7 @@
| list_objects_v2_metadata_extension_test | 1 | |
| list_objects_v2_pagination_test | 12 | ✅ |
| mc_mirror_small_bucket_test | 1 | |
| multipart_auth_test | 75 | |
| multipart_auth_test | 85 | |
| multipart_storage_class_test | 3 | ✅ |
| namespace_lock_quorum_test | 2 | |
| negative_sigv4_test | 6 | ✅ |
+25 -1
View File
@@ -66,6 +66,9 @@ struct HealInitParams {
client_token: String,
force_start: bool,
force_stop: bool,
/// Incremental result cursor (HS-06): only result items with a sequence
/// greater than this are returned; absent means full snapshot.
since_seq: Option<u64>,
}
fn extract_heal_init_params(body: &Bytes, uri: &Uri, params: Params<'_, '_>) -> S3Result<HealInitParams> {
@@ -98,6 +101,16 @@ fn extract_heal_init_params(body: &Bytes, uri: &Uri, params: Params<'_, '_>) ->
}
hip.force_stop = parse_heal_query_bool(value.as_ref())?;
}
"sinceSeq" => {
if !seen.insert("sinceSeq") {
return Err(s3_error!(InvalidArgument, "duplicate heal query parameter"));
}
hip.since_seq = Some(
value
.parse::<u64>()
.map_err(|_| s3_error!(InvalidArgument, "sinceSeq must be a non-negative integer"))?,
);
}
_ => return Err(s3_error!(InvalidArgument, "unknown heal query parameter")),
}
}
@@ -317,6 +330,7 @@ fn add_source_counts(total: &mut rustfs_heal::HealSourceCounts, next: rustfs_hea
total.auto_heal = total.auto_heal.saturating_add(next.auto_heal);
total.internal = total.internal.saturating_add(next.internal);
total.read_repair = total.read_repair.saturating_add(next.read_repair);
total.mrf = total.mrf.saturating_add(next.mrf);
}
fn add_operations(total: &mut rustfs_heal::HealOperationsSnapshot, next: rustfs_heal::HealOperationsSnapshot) {
@@ -977,7 +991,15 @@ fn reject_heal_admission(result: rustfs_common::heal_channel::HealAdmissionResul
result.result_label(),
result.reason_label()
),
HealAdmissionResult::Dropped(HealAdmissionDropReason::PolicyDropped) => s3_error!(
// Overlap rejections (HS-06) share this arm: the s3s footprint
// ratchet forbids new s3_error! sites, and the typed reason is
// preserved through reason_label() ("already_running" /
// "overlapping_paths") so madmin-style clients can distinguish.
HealAdmissionResult::Dropped(
HealAdmissionDropReason::PolicyDropped
| HealAdmissionDropReason::AlreadyRunning
| HealAdmissionDropReason::OverlappingPaths,
) => s3_error!(
OperationAborted,
"heal request not admitted: admission={}, reason={}",
result.result_label(),
@@ -1402,6 +1424,7 @@ impl Operation for HealHandler {
new_heal_control_metadata(&route)?,
heal_path,
client_token.clone(),
hip.since_seq,
)
.map_err(|err| s3_error!(InternalError, "encode heal control query failed: {err}"))?;
let response = submit_cluster_heal_channel_command(context, route, envelope, &request_id, client_token).await?;
@@ -2353,6 +2376,7 @@ mod tests {
auto_heal: value,
internal: value,
read_repair: value,
mrf: value,
};
let operations = |value| rustfs_heal::HealOperationsSnapshot {
queue_length: value,
+6 -2
View File
@@ -22,6 +22,7 @@ use crate::admin::runtime_sources::{
current_object_store_handle_for_context, current_or_init_kms_runtime_service_manager,
};
use crate::admin::storage_api::config::{read_admin_config, save_admin_config};
use crate::admin::storage_api::error::StorageError;
use crate::auth::{check_key_valid, get_session_token};
use crate::server::{ADMIN_PREFIX, RemoteAddr};
use hyper::{Method, StatusCode};
@@ -278,8 +279,11 @@ pub async fn load_kms_config() -> Option<KmsConfig> {
}
},
Err(e) => {
// Config not found is normal on first run
if e.to_string().contains("ConfigNotFound") || e.to_string().contains("not found") {
// Config not found is normal on first run: `read_config` maps a missing or
// empty config object to `ConfigNotFound`, so that variant is the only
// "absent" signal reaching here. Every other not-found variant (disk,
// volume, bucket) means degraded storage and must stay a warning.
if matches!(e, StorageError::ConfigNotFound) {
info!(
component = LOG_COMPONENT_ADMIN,
subsystem = LOG_SUBSYSTEM_KMS,
+75 -52
View File
@@ -16,13 +16,12 @@
use super::kms_dynamic::current_kms_config_fingerprint;
use super::kms_keys::{CreateKeyHandler, DescribeKeyHandler, GenerateDataKeyHandler, ListKeysHandler};
use crate::admin::auth::validate_admin_request;
use crate::admin::auth::authorize_admin_request;
use crate::admin::router::{AdminOperation, Operation, S3Router};
use crate::admin::runtime_sources::{
current_kms_runtime_service_manager, current_notification_system, current_or_init_kms_runtime_service_manager,
};
use crate::auth::{check_key_valid, get_session_token};
use crate::server::{ADMIN_PREFIX, RemoteAddr};
use crate::server::ADMIN_PREFIX;
use hyper::{HeaderMap, Method, StatusCode};
use matchit::Params;
use rustfs_kms::KmsBackend;
@@ -69,6 +68,18 @@ fn kms_clear_cache_actions() -> Vec<Action> {
vec![Action::KmsAction(KmsAction::ClearCacheAction)]
}
/// Admin gate for the KMS management endpoints, none of which act on a key.
///
/// The pre-check keeps these endpoints' historical missing-credentials message;
/// the shared gate reports "get cred failed".
async fn authorize_kms_management_request(req: &S3Request<Body>, actions: Vec<Action>) -> S3Result<()> {
if req.credentials.is_none() {
return Err(s3_error!(InvalidRequest, "authentication required"));
}
authorize_admin_request(req, actions).await?;
Ok(())
}
/// Response of `POST /kms/clear-cache`.
///
/// Declared rather than built inline so the shape the console already depends
@@ -260,22 +271,7 @@ pub struct KmsStatusHandler {}
#[async_trait::async_trait]
impl Operation for KmsStatusHandler {
async fn call(&self, req: S3Request<Body>, _params: Params<'_, '_>) -> S3Result<S3Response<(StatusCode, Body)>> {
let Some(cred) = req.credentials else {
return Err(s3_error!(InvalidRequest, "authentication required"));
};
let (cred, owner) =
check_key_valid(get_session_token(&req.uri, &req.headers).unwrap_or_default(), &cred.access_key).await?;
validate_admin_request(
&req.headers,
&cred,
owner,
false,
kms_service_control_actions(),
req.extensions.get::<Option<RemoteAddr>>().and_then(|opt| opt.map(|a| a.0)),
)
.await?;
authorize_kms_management_request(&req, kms_service_control_actions()).await?;
let Some(service) = kms_encryption_service_from_context().await else {
return Err(s3_error!(InternalError, "KMS service not initialized"));
@@ -326,22 +322,7 @@ pub struct KmsConfigHandler {}
#[async_trait::async_trait]
impl Operation for KmsConfigHandler {
async fn call(&self, req: S3Request<Body>, _params: Params<'_, '_>) -> S3Result<S3Response<(StatusCode, Body)>> {
let Some(cred) = req.credentials else {
return Err(s3_error!(InvalidRequest, "authentication required"));
};
let (cred, owner) =
check_key_valid(get_session_token(&req.uri, &req.headers).unwrap_or_default(), &cred.access_key).await?;
validate_admin_request(
&req.headers,
&cred,
owner,
false,
kms_configure_actions(),
req.extensions.get::<Option<RemoteAddr>>().and_then(|opt| opt.map(|a| a.0)),
)
.await?;
authorize_kms_management_request(&req, kms_configure_actions()).await?;
let Some(service) = kms_encryption_service_from_context().await else {
return Err(s3_error!(InternalError, "KMS service not initialized"));
@@ -375,22 +356,7 @@ pub struct KmsClearCacheHandler {}
#[async_trait::async_trait]
impl Operation for KmsClearCacheHandler {
async fn call(&self, req: S3Request<Body>, _params: Params<'_, '_>) -> S3Result<S3Response<(StatusCode, Body)>> {
let Some(cred) = req.credentials else {
return Err(s3_error!(InvalidRequest, "authentication required"));
};
let (cred, owner) =
check_key_valid(get_session_token(&req.uri, &req.headers).unwrap_or_default(), &cred.access_key).await?;
validate_admin_request(
&req.headers,
&cred,
owner,
false,
kms_clear_cache_actions(),
req.extensions.get::<Option<RemoteAddr>>().and_then(|opt| opt.map(|a| a.0)),
)
.await?;
authorize_kms_management_request(&req, kms_clear_cache_actions()).await?;
let Some(service) = kms_encryption_service_from_context().await else {
return Err(s3_error!(InternalError, "KMS service not initialized"));
@@ -422,9 +388,14 @@ impl Operation for KmsClearCacheHandler {
#[cfg(test)]
mod tests {
use super::{KmsClearCacheResponse, kms_clear_cache_actions, kms_configure_actions, kms_service_control_actions};
use super::{
KmsClearCacheResponse, authorize_kms_management_request, kms_clear_cache_actions, kms_configure_actions,
kms_service_control_actions,
};
use crate::admin::handlers::kms_keys::stable_json_value;
use hyper::HeaderMap;
use rustfs_policy::policy::action::{Action, AdminAction, KmsAction};
use s3s::{Body, S3Request};
fn assert_has_action(actions: &[Action], action: Action) {
assert!(actions.contains(&action), "expected action list to contain {action:?}");
@@ -434,6 +405,58 @@ mod tests {
assert!(!actions.contains(&action), "expected action list not to contain {action:?}");
}
/// These endpoints authorize through the shared admin gate, which reports
/// "get cred failed" for a credential-less request. The pre-check keeps the
/// message these endpoints have always returned (rustfs/backlog#1829).
#[tokio::test]
async fn kms_management_gate_keeps_its_missing_credentials_message() {
let req = S3Request {
input: Body::from(String::new()),
method: http::Method::GET,
uri: "/rustfs/admin/v3/kms/status".parse().expect("uri should parse"),
headers: HeaderMap::new(),
extensions: http::Extensions::new(),
credentials: None,
region: None,
service: None,
trailing_headers: None,
};
let err = authorize_kms_management_request(&req, kms_service_control_actions())
.await
.expect_err("a request without credentials must be rejected");
assert_eq!(err.code(), &s3s::S3ErrorCode::InvalidRequest);
assert_eq!(err.message(), Some("authentication required"));
}
/// Every management endpoint must reach the shared gate, each with its own
/// action set. The action lists are pinned above, but nothing else checks
/// which handler asks for which, and a handler that lost its gate entirely
/// would still serve its response.
#[test]
fn management_handlers_authorize_with_their_dedicated_actions() {
let src = include_str!("kms_management.rs");
for (handler, actions) in [
("KmsStatusHandler", "kms_service_control_actions()"),
("KmsConfigHandler", "kms_configure_actions()"),
("KmsClearCacheHandler", "kms_clear_cache_actions()"),
] {
let block = src
.split_once(&format!("impl Operation for {handler}"))
.unwrap_or_else(|| panic!("{handler} impl should exist"))
.1;
let end = block
.find("\nimpl Operation for")
.or_else(|| block.find("\n#[cfg(test)]"))
.unwrap_or(block.len());
assert!(
block[..end].contains(&format!("authorize_kms_management_request(&req, {actions})")),
"{handler} must authorize through the shared gate with {actions}"
);
}
}
#[test]
fn kms_management_auth_actions_use_dedicated_kms_actions() {
assert_has_action(&kms_service_control_actions(), Action::KmsAction(KmsAction::ServiceControlAction));
+15 -16
View File
@@ -67,6 +67,9 @@ use rustfs_policy::policy::action::{Action, S3Action};
use rustfs_s3_types::EventName;
use rustfs_signer::pre_sign_v4;
use rustfs_utils::egress::{OutboundDnsResolver, OutboundPolicy};
use rustfs_utils::http::headers::{
AMZ_CHECKSUM_CRC32, AMZ_CHECKSUM_CRC32C, AMZ_CHECKSUM_CRC64NVME, AMZ_CHECKSUM_SHA1, AMZ_CHECKSUM_SHA256, AMZ_CHECKSUM_TYPE,
};
use rustfs_utils::http::{
SUFFIX_SOURCE_DELETEMARKER, SUFFIX_SOURCE_MTIME, SUFFIX_SOURCE_REPLICATION_CHECK, SUFFIX_SOURCE_REPLICATION_REQUEST,
SUFFIX_SOURCE_VERSION_ID, get_source_scheme, insert_header,
@@ -1031,28 +1034,24 @@ fn build_get_object_response_headers(output: &GetObjectOutput, base_headers: &He
)?;
}
if let Some(checksum_crc32) = &output.checksum_crc32 {
insert_string_header(&mut headers, HeaderName::from_static("x-amz-checksum-crc32"), checksum_crc32.clone())?;
insert_string_header(&mut headers, HeaderName::from_static(AMZ_CHECKSUM_CRC32), checksum_crc32.clone())?;
}
if let Some(checksum_crc32c) = &output.checksum_crc32c {
insert_string_header(&mut headers, HeaderName::from_static("x-amz-checksum-crc32c"), checksum_crc32c.clone())?;
insert_string_header(&mut headers, HeaderName::from_static(AMZ_CHECKSUM_CRC32C), checksum_crc32c.clone())?;
}
if let Some(checksum_crc64nvme) = &output.checksum_crc64nvme {
insert_string_header(
&mut headers,
HeaderName::from_static("x-amz-checksum-crc64nvme"),
checksum_crc64nvme.clone(),
)?;
insert_string_header(&mut headers, HeaderName::from_static(AMZ_CHECKSUM_CRC64NVME), checksum_crc64nvme.clone())?;
}
if let Some(checksum_sha1) = &output.checksum_sha1 {
insert_string_header(&mut headers, HeaderName::from_static("x-amz-checksum-sha1"), checksum_sha1.clone())?;
insert_string_header(&mut headers, HeaderName::from_static(AMZ_CHECKSUM_SHA1), checksum_sha1.clone())?;
}
if let Some(checksum_sha256) = &output.checksum_sha256 {
insert_string_header(&mut headers, HeaderName::from_static("x-amz-checksum-sha256"), checksum_sha256.clone())?;
insert_string_header(&mut headers, HeaderName::from_static(AMZ_CHECKSUM_SHA256), checksum_sha256.clone())?;
}
if let Some(checksum_type) = &output.checksum_type {
insert_string_header(
&mut headers,
HeaderName::from_static("x-amz-checksum-type"),
HeaderName::from_static(AMZ_CHECKSUM_TYPE),
checksum_type.as_str().to_string(),
)?;
}
@@ -1114,12 +1113,12 @@ fn clear_object_lambda_variant_headers(headers: &mut HeaderMap) {
http::header::ETAG,
http::header::LAST_MODIFIED,
http::header::EXPIRES,
HeaderName::from_static("x-amz-checksum-crc32"),
HeaderName::from_static("x-amz-checksum-crc32c"),
HeaderName::from_static("x-amz-checksum-crc64nvme"),
HeaderName::from_static("x-amz-checksum-sha1"),
HeaderName::from_static("x-amz-checksum-sha256"),
HeaderName::from_static("x-amz-checksum-type"),
HeaderName::from_static(AMZ_CHECKSUM_CRC32),
HeaderName::from_static(AMZ_CHECKSUM_CRC32C),
HeaderName::from_static(AMZ_CHECKSUM_CRC64NVME),
HeaderName::from_static(AMZ_CHECKSUM_SHA1),
HeaderName::from_static(AMZ_CHECKSUM_SHA256),
HeaderName::from_static(AMZ_CHECKSUM_TYPE),
HeaderName::from_static("x-amz-tagging-count"),
HeaderName::from_static("x-amz-request-route"),
HeaderName::from_static("x-amz-request-token"),
+34 -12
View File
@@ -56,8 +56,8 @@ use super::storage_api::object_usecase::bucket::{
};
use super::storage_api::object_usecase::compression::{MIN_DISK_COMPRESSIBLE_SIZE, is_disk_compressible};
use super::storage_api::object_usecase::concurrency::{
self, ConcurrencyManager, DiskReadAdmission, GetObjectGuard, PutObjectGuard, get_concurrency_aware_buffer_size,
get_concurrency_manager, get_put_concurrency_aware_buffer_size,
self, ConcurrencyManager, DiskReadAdmission, GetObjectGuard, PutObjectAdmission, PutObjectGuard,
get_concurrency_aware_buffer_size, get_concurrency_manager, get_put_concurrency_aware_buffer_size,
};
#[cfg(test)]
use super::storage_api::object_usecase::contract::http::HTTPPreconditions;
@@ -5681,6 +5681,35 @@ impl DefaultObjectUsecase {
let server_side_encryption_requested =
server_side_encryption.is_some() || sse_customer_algorithm.is_some() || ssekms_key_id.is_some();
// Resolve the store through the request-bound server context
// (backlog#1052 S6), not the process-global handle, so an embedded
// second server never writes into the first server's store.
let Some(store) = self.object_store() else {
return Err(S3Error::with_message(S3ErrorCode::InternalError, "Not init".to_string()));
};
let bucket_validate_stage_start = put_stage_metrics_enabled.then(Instant::now);
validate_bucket_exists(&store, &bucket).await?;
rustfs_io_metrics::record_put_object_stage_duration_from("app_bucket_validate", bucket_validate_stage_start);
let put_admission = match get_concurrency_manager()
.admit_put_object()
.await
.map_err(|_| s3_error!(InternalError, "foreground write admission closed"))?
{
PutObjectAdmission::Disabled => None,
PutObjectAdmission::Admitted(permit) => {
counter!("rustfs.put_object.foreground_admission.total", "result" => "admitted").increment(1);
Some(permit)
}
PutObjectAdmission::Rejected => {
counter!("rustfs.put_object.foreground_admission.total", "result" => "rejected").increment(1);
return Err(s3_error!(
SlowDown,
"foreground write concurrency limit reached, please reduce your request rate"
));
}
};
let mut put_request_guard = PutObjectGuard::new();
let concurrent_put_requests = PutObjectGuard::concurrent_requests();
@@ -5733,16 +5762,6 @@ impl DefaultObjectUsecase {
use_large_put_concurrency_tuning,
);
// Resolve the store through the request-bound server context
// (backlog#1052 S6), not the process-global handle, so an embedded
// second server never writes into the first server's store.
let Some(store) = self.object_store() else {
return Err(S3Error::with_message(S3ErrorCode::InternalError, "Not init".to_string()));
};
let bucket_validate_stage_start = put_stage_metrics_enabled.then(Instant::now);
validate_bucket_exists(&store, &bucket).await?;
rustfs_io_metrics::record_put_object_stage_duration_from("app_bucket_validate", bucket_validate_stage_start);
let sse_config_stage_start = put_stage_metrics_enabled.then(Instant::now);
let bucket_sse_config = metadata_sys::get_sse_config(&bucket).await.ok();
rustfs_io_metrics::record_put_object_stage_duration_from("app_sse_config_lookup", sse_config_stage_start);
@@ -6132,7 +6151,9 @@ impl DefaultObjectUsecase {
let cache_adapter = cache_adapter.clone();
let request_id = request_id.clone();
let put_path = put_path.to_string();
let put_admission = put_admission;
async move {
let _put_admission = put_admission;
let object_traffic_progress = object_traffic_health
.as_deref()
.and_then(ObjectTrafficHealth::track_write_storage);
@@ -6183,6 +6204,7 @@ impl DefaultObjectUsecase {
}
};
rustfs_io_metrics::record_put_object_stage_duration_from("app_store_put", store_put_stage_start);
drop(_put_admission);
drop(object_traffic_progress);
#[cfg(test)]
wait_for_put_post_store_test_hook(&bucket).await;
+1 -1
View File
@@ -936,7 +936,7 @@ pub(crate) mod bucket {
pub(crate) mod concurrency {
pub(crate) use crate::storage::storage_api::concurrency_consumer::{
ConcurrencyManager, DiskReadAdmission, GetObjectGuard, IoQueueStatus, IoStrategy, PutObjectGuard,
ConcurrencyManager, DiskReadAdmission, GetObjectGuard, IoQueueStatus, IoStrategy, PutObjectAdmission, PutObjectGuard,
get_concurrency_aware_buffer_size, get_concurrency_manager, get_put_concurrency_aware_buffer_size,
};
}
-618
View File
@@ -1,618 +0,0 @@
// Copyright 2024 RustFS Team
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//! Backpressure Management for Object Data Transfer.
//!
//! This module provides backpressure-aware pipes for object data transfer,
//! preventing buffer overflow and memory exhaustion under high concurrency.
//! # Key Features
//!
//! - Configurable buffer size with high/low watermarks
//! - Backpressure state monitoring and events
//! - Backpressure metrics emitted through the shared metrics pipeline
//! - Graceful handling of slow consumers
//!
//! # Architecture
//!
//! ```text
//! [Disk Reader] --> [BackpressurePipe] --> [HTTP Response]
//! |
//! v
//! [Buffer Monitor]
//! |
//! v
//! [High Watermark?] --> Apply Backpressure
//! ```
use std::sync::atomic::{AtomicBool, AtomicUsize, Ordering};
use std::time::{Duration, Instant};
use tokio::io::{DuplexStream, duplex};
use tracing::{debug, warn};
use metrics::counter;
use rustfs_concurrency::PipeBackpressurePolicy;
use rustfs_io_core::BackpressureConfig as CoreBackpressureConfig;
/// Object-transfer duplex pipe backpressure policy.
#[derive(Debug, Clone, Copy)]
pub struct ObjectPipeBackpressurePolicy {
/// Buffer size in bytes (default 4MB).
pub buffer_size: usize,
/// High watermark percentage (default 80%).
/// When buffer usage exceeds this, backpressure is applied.
pub high_watermark: u32,
/// Low watermark percentage (default 50%).
/// When buffer usage drops below this after high watermark, backpressure is released.
pub low_watermark: u32,
}
impl Default for ObjectPipeBackpressurePolicy {
fn default() -> Self {
Self {
buffer_size: rustfs_config::DEFAULT_OBJECT_DUPLEX_BUFFER_SIZE,
high_watermark: rustfs_config::DEFAULT_OBJECT_BACKPRESSURE_HIGH_WATERMARK,
low_watermark: rustfs_config::DEFAULT_OBJECT_BACKPRESSURE_LOW_WATERMARK,
}
}
}
impl ObjectPipeBackpressurePolicy {
/// Load configuration from environment variables.
pub fn from_env() -> Self {
let buffer_size = rustfs_utils::get_env_usize(
rustfs_config::ENV_OBJECT_DUPLEX_BUFFER_SIZE,
rustfs_config::DEFAULT_OBJECT_DUPLEX_BUFFER_SIZE,
);
let high_watermark = rustfs_utils::get_env_u32(
rustfs_config::ENV_OBJECT_BACKPRESSURE_HIGH_WATERMARK,
rustfs_config::DEFAULT_OBJECT_BACKPRESSURE_HIGH_WATERMARK,
);
let low_watermark = rustfs_utils::get_env_u32(
rustfs_config::ENV_OBJECT_BACKPRESSURE_LOW_WATERMARK,
rustfs_config::DEFAULT_OBJECT_BACKPRESSURE_LOW_WATERMARK,
);
Self {
buffer_size,
high_watermark,
low_watermark,
}
}
/// Calculate high watermark threshold in bytes.
pub fn high_watermark_bytes(&self) -> usize {
(self.buffer_size as u64 * self.high_watermark as u64 / 100) as usize
}
/// Calculate low watermark threshold in bytes.
pub fn low_watermark_bytes(&self) -> usize {
(self.buffer_size as u64 * self.low_watermark as u64 / 100) as usize
}
/// Project this object-transfer policy into the shared concurrency facade policy.
pub fn to_concurrency_policy(&self) -> PipeBackpressurePolicy {
PipeBackpressurePolicy {
buffer_size: self.buffer_size,
high_watermark: self.high_watermark,
low_watermark: self.low_watermark,
}
}
/// Project this object-transfer policy into the reusable io-core admission config.
pub fn to_core_config(&self) -> CoreBackpressureConfig {
self.to_concurrency_policy().to_core_config()
}
}
/// Backpressure state.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum BackpressureState {
/// Normal operation, buffer usage is below high watermark.
Normal,
/// Buffer usage is above high watermark, backpressure should be applied.
HighWatermark,
/// Backpressure is actively being applied to the producer.
BackpressureApplied,
}
impl std::fmt::Display for BackpressureState {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
BackpressureState::Normal => write!(f, "normal"),
BackpressureState::HighWatermark => write!(f, "high_watermark"),
BackpressureState::BackpressureApplied => write!(f, "backpressure_applied"),
}
}
}
/// Compact metadata snapshot for object-transfer backpressure pipes.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct BackpressurePipeMeta {
/// Buffer capacity in bytes.
pub buffer_capacity: usize,
/// Current backpressure state.
pub state: BackpressureState,
/// Age of the pipe since creation.
pub age: Duration,
}
/// Compact metadata snapshot for the lightweight backpressure monitor.
#[derive(Debug, Clone, Copy, PartialEq)]
pub struct BackpressureMonitorMeta {
/// Buffer capacity in bytes.
pub buffer_capacity: usize,
/// Current buffer usage percentage.
pub usage_percent: f32,
/// Current backpressure state.
pub state: BackpressureState,
}
fn calculate_usage_percent(usage: usize, capacity: usize) -> f32 {
if capacity > 0 {
(usage as f32 / capacity as f32) * 100.0
} else {
0.0
}
}
fn apply_watermark_transition(
in_high_watermark: &AtomicBool,
usage: usize,
high: usize,
low: usize,
) -> (BackpressureState, bool) {
let current = in_high_watermark.load(Ordering::Acquire);
let next_state = if usage >= high {
BackpressureState::HighWatermark
} else if usage <= low {
BackpressureState::Normal
} else if current {
BackpressureState::HighWatermark
} else {
BackpressureState::Normal
};
let next_is_high = matches!(next_state, BackpressureState::HighWatermark);
let changed = in_high_watermark.swap(next_is_high, Ordering::AcqRel) != next_is_high;
(next_state, changed)
}
fn saturating_sub_atomic(value: &AtomicUsize, delta: usize) {
value
.fetch_update(Ordering::AcqRel, Ordering::Acquire, |current| Some(current.saturating_sub(delta)))
.ok();
}
/// A backpressure-aware pipe wrapping tokio's duplex.
///
/// This provides monitoring and events for backpressure conditions
/// while maintaining compatibility with the standard duplex interface.
pub struct BackpressurePipe {
/// Reader end of the duplex pipe.
reader: DuplexStream,
/// Writer end of the duplex pipe.
writer: DuplexStream,
/// Configuration.
config: ObjectPipeBackpressurePolicy,
/// Current buffer usage (approximate, updated on write).
buffer_usage: AtomicUsize,
/// Current backpressure state.
state: AtomicBool, // true = in high watermark state
/// Total bytes written.
total_written: AtomicUsize,
/// Total bytes read.
total_read: AtomicUsize,
/// Cached high watermark threshold in bytes.
high_watermark_bytes: usize,
/// Cached low watermark threshold in bytes.
low_watermark_bytes: usize,
/// Pipe creation timestamp.
created_at: Instant,
}
impl BackpressurePipe {
/// Create a new backpressure-aware pipe with default configuration.
pub fn new() -> Self {
Self::with_config(ObjectPipeBackpressurePolicy::from_env())
}
/// Create a new backpressure-aware pipe with custom configuration.
pub fn with_config(config: ObjectPipeBackpressurePolicy) -> Self {
let policy = config.to_concurrency_policy();
let (reader, writer) = duplex(policy.buffer_size);
let high_watermark_bytes = policy.high_watermark_bytes();
let low_watermark_bytes = policy.low_watermark_bytes();
debug!(
buffer_size = config.buffer_size,
high_watermark = config.high_watermark,
low_watermark = config.low_watermark,
high_watermark_bytes,
low_watermark_bytes,
"Created backpressure pipe"
);
Self {
reader,
writer,
config,
buffer_usage: AtomicUsize::new(0),
state: AtomicBool::new(false),
total_written: AtomicUsize::new(0),
total_read: AtomicUsize::new(0),
high_watermark_bytes,
low_watermark_bytes,
created_at: Instant::now(),
}
}
/// Take the reader end of the pipe (consumes self).
pub fn into_reader(self) -> DuplexStream {
self.reader
}
/// Take the writer end of the pipe (consumes self).
pub fn into_writer(self) -> DuplexStream {
self.writer
}
/// Split into reader and writer (consumes self).
pub fn split(self) -> (DuplexStream, DuplexStream) {
(self.reader, self.writer)
}
/// Get current backpressure state.
pub fn state(&self) -> BackpressureState {
if self.state.load(Ordering::Acquire) {
BackpressureState::BackpressureApplied
} else {
BackpressureState::Normal
}
}
/// Get a compact metadata snapshot for the pipe.
pub fn meta(&self) -> BackpressurePipeMeta {
BackpressurePipeMeta {
buffer_capacity: self.config.buffer_size,
state: self.state(),
age: self.age(),
}
}
/// Get the age of this pipe.
pub fn age(&self) -> Duration {
self.created_at.elapsed()
}
/// Get current buffer usage.
pub fn usage(&self) -> usize {
self.buffer_usage.load(Ordering::Acquire)
}
/// Record bytes written (call after successful write).
pub fn record_write(&self, bytes: usize) {
self.total_written.fetch_add(bytes, Ordering::Relaxed);
self.buffer_usage.fetch_add(bytes, Ordering::Release);
self.update_watermark_state();
}
/// Record bytes read (call after successful read).
pub fn record_read(&self, bytes: usize) {
self.total_read.fetch_add(bytes, Ordering::Relaxed);
saturating_sub_atomic(&self.buffer_usage, bytes);
self.update_watermark_state();
}
/// Update watermark state and emit transition signals.
fn update_watermark_state(&self) {
let usage = self.buffer_usage.load(Ordering::Acquire);
let usage_percent = calculate_usage_percent(usage, self.config.buffer_size) as u32;
let (next_state, changed) =
apply_watermark_transition(&self.state, usage, self.high_watermark_bytes, self.low_watermark_bytes);
if changed {
match next_state {
BackpressureState::HighWatermark => {
counter!("rustfs_backpressure_events_total", "state" => "high_watermark").increment(1);
warn!(
buffer_usage = usage,
buffer_capacity = self.config.buffer_size,
usage_percent,
high_watermark = self.config.high_watermark,
"Backpressure: high watermark reached"
);
}
BackpressureState::Normal => {
counter!("rustfs_backpressure_events_total", "state" => "normal").increment(1);
debug!(
buffer_usage = usage,
buffer_capacity = self.config.buffer_size,
usage_percent,
low_watermark = self.config.low_watermark,
"Backpressure: returned to normal"
);
}
BackpressureState::BackpressureApplied => {}
}
}
}
/// Get total bytes written.
pub fn total_written(&self) -> usize {
self.total_written.load(Ordering::Relaxed)
}
/// Get total bytes read.
pub fn total_read(&self) -> usize {
self.total_read.load(Ordering::Relaxed)
}
/// Get buffer capacity.
pub fn capacity(&self) -> usize {
self.config.buffer_size
}
}
impl Default for BackpressurePipe {
fn default() -> Self {
Self::new()
}
}
/// A simple wrapper that provides backpressure monitoring for duplex streams.
///
/// This is a lighter-weight alternative to `BackpressurePipe` that doesn't
/// wrap the streams but provides monitoring capabilities.
pub struct BackpressureMonitor {
/// Configuration.
config: ObjectPipeBackpressurePolicy,
/// Current buffer usage.
buffer_usage: AtomicUsize,
/// In high watermark state.
in_high_watermark: AtomicBool,
/// Cached high watermark threshold in bytes.
high_watermark_bytes: usize,
/// Cached low watermark threshold in bytes.
low_watermark_bytes: usize,
}
impl BackpressureMonitor {
/// Create a new monitor with default configuration.
pub fn new() -> Self {
Self::with_config(ObjectPipeBackpressurePolicy::from_env())
}
/// Create a new monitor with custom configuration.
pub fn with_config(config: ObjectPipeBackpressurePolicy) -> Self {
let policy = config.to_concurrency_policy();
let high_watermark_bytes = policy.high_watermark_bytes();
let low_watermark_bytes = policy.low_watermark_bytes();
Self {
config,
buffer_usage: AtomicUsize::new(0),
in_high_watermark: AtomicBool::new(false),
high_watermark_bytes,
low_watermark_bytes,
}
}
/// Record bytes added to buffer.
pub fn on_write(&self, bytes: usize) -> BackpressureState {
self.buffer_usage.fetch_add(bytes, Ordering::Release);
self.update_state()
}
/// Record bytes removed from buffer.
pub fn on_read(&self, bytes: usize) -> BackpressureState {
saturating_sub_atomic(&self.buffer_usage, bytes);
self.update_state()
}
/// Get current state.
pub fn state(&self) -> BackpressureState {
if self.in_high_watermark.load(Ordering::Acquire) {
BackpressureState::HighWatermark
} else {
BackpressureState::Normal
}
}
/// Get current buffer usage.
pub fn usage(&self) -> usize {
self.buffer_usage.load(Ordering::Acquire)
}
/// Get usage percentage.
pub fn usage_percent(&self) -> f32 {
let usage = self.buffer_usage.load(Ordering::Acquire);
calculate_usage_percent(usage, self.config.buffer_size)
}
/// Get a compact metadata snapshot for the monitor.
pub fn meta(&self) -> BackpressureMonitorMeta {
let usage = self.buffer_usage.load(Ordering::Acquire);
BackpressureMonitorMeta {
buffer_capacity: self.config.buffer_size,
usage_percent: calculate_usage_percent(usage, self.config.buffer_size),
state: self.state(),
}
}
/// Update state based on current usage.
fn update_state(&self) -> BackpressureState {
let usage = self.buffer_usage.load(Ordering::Acquire);
let usage_percent = calculate_usage_percent(usage, self.config.buffer_size) as u32;
let (next_state, changed) =
apply_watermark_transition(&self.in_high_watermark, usage, self.high_watermark_bytes, self.low_watermark_bytes);
if matches!(next_state, BackpressureState::HighWatermark) {
if changed {
counter!("rustfs_backpressure_events_total", "state" => "high_watermark").increment(1);
debug!(usage_percent, "Backpressure: entered high watermark");
}
BackpressureState::HighWatermark
} else {
if changed {
counter!("rustfs_backpressure_events_total", "state" => "normal").increment(1);
debug!(usage_percent, "Backpressure: returned to normal");
}
BackpressureState::Normal
}
}
}
impl Default for BackpressureMonitor {
fn default() -> Self {
Self::new()
}
}
#[cfg(test)]
#[allow(unused_imports)]
mod tests {
use super::{BackpressureMonitor, BackpressurePipe, BackpressureState, ObjectPipeBackpressurePolicy};
#[test]
fn test_backpressure_config_default() {
let config = ObjectPipeBackpressurePolicy::default();
assert_eq!(config.buffer_size, 4 * 1024 * 1024);
assert_eq!(config.high_watermark, 80);
assert_eq!(config.low_watermark, 50);
}
#[test]
fn test_backpressure_config_watermarks() {
let config = ObjectPipeBackpressurePolicy {
buffer_size: 1000,
high_watermark: 80,
low_watermark: 50,
};
assert_eq!(config.high_watermark_bytes(), 800);
assert_eq!(config.low_watermark_bytes(), 500);
}
#[test]
fn test_backpressure_policy_projects_to_concurrency_and_core_config() {
let config = ObjectPipeBackpressurePolicy {
buffer_size: 2000,
high_watermark: 75,
low_watermark: 40,
};
let concurrency = config.to_concurrency_policy();
let core = config.to_core_config();
assert_eq!(concurrency.buffer_size, config.buffer_size);
assert_eq!(concurrency.high_watermark, config.high_watermark);
assert_eq!(concurrency.low_watermark, config.low_watermark);
assert_eq!(core.high_water_mark, 0.75);
assert_eq!(core.low_water_mark, 0.40);
assert!(core.enabled);
}
#[test]
fn test_backpressure_pipe_consumes_concurrency_policy_thresholds() {
let config = ObjectPipeBackpressurePolicy {
buffer_size: 2000,
high_watermark: 75,
low_watermark: 40,
};
let concurrency = config.to_concurrency_policy();
let pipe = BackpressurePipe::with_config(config);
assert_eq!(pipe.capacity(), concurrency.buffer_size);
assert_eq!(pipe.high_watermark_bytes, concurrency.high_watermark_bytes());
assert_eq!(pipe.low_watermark_bytes, concurrency.low_watermark_bytes());
}
#[test]
fn test_backpressure_monitor_consumes_concurrency_policy_thresholds() {
let config = ObjectPipeBackpressurePolicy {
buffer_size: 2000,
high_watermark: 75,
low_watermark: 40,
};
let concurrency = config.to_concurrency_policy();
let monitor = BackpressureMonitor::with_config(config);
assert_eq!(monitor.meta().buffer_capacity, concurrency.buffer_size);
assert_eq!(monitor.high_watermark_bytes, concurrency.high_watermark_bytes());
assert_eq!(monitor.low_watermark_bytes, concurrency.low_watermark_bytes());
}
#[test]
fn test_backpressure_state_display() {
assert_eq!(format!("{}", BackpressureState::Normal), "normal");
assert_eq!(format!("{}", BackpressureState::HighWatermark), "high_watermark");
assert_eq!(format!("{}", BackpressureState::BackpressureApplied), "backpressure_applied");
}
#[test]
fn test_backpressure_monitor() {
let config = ObjectPipeBackpressurePolicy {
buffer_size: 1000,
high_watermark: 80,
low_watermark: 50,
};
let monitor = BackpressureMonitor::with_config(config);
// Initially normal
assert_eq!(monitor.state(), BackpressureState::Normal);
assert_eq!(monitor.meta().buffer_capacity, 1000);
assert_eq!(monitor.meta().usage_percent, 0.0);
// Write to reach high watermark
let state = monitor.on_write(850);
assert_eq!(state, BackpressureState::HighWatermark);
assert_eq!(monitor.meta().usage_percent, 85.0);
// Read to go below low watermark
let state = monitor.on_read(400);
assert_eq!(state, BackpressureState::Normal);
assert_eq!(monitor.meta().usage_percent, 45.0);
}
#[tokio::test]
async fn test_backpressure_pipe_creation() {
let pipe = BackpressurePipe::new();
assert_eq!(pipe.capacity(), 4 * 1024 * 1024);
assert_eq!(pipe.state(), BackpressureState::Normal);
assert_eq!(pipe.meta().buffer_capacity, 4 * 1024 * 1024);
assert!(pipe.meta().age <= pipe.age());
}
#[test]
fn test_backpressure_pipe_state_transitions() {
let config = ObjectPipeBackpressurePolicy {
buffer_size: 1000,
high_watermark: 80,
low_watermark: 50,
};
let pipe = BackpressurePipe::with_config(config);
assert_eq!(pipe.state(), BackpressureState::Normal);
assert_eq!(pipe.meta().state, BackpressureState::Normal);
pipe.record_write(850);
assert_eq!(pipe.state(), BackpressureState::BackpressureApplied);
assert_eq!(pipe.meta().state, BackpressureState::BackpressureApplied);
pipe.record_read(400);
assert_eq!(pipe.state(), BackpressureState::Normal);
assert_eq!(pipe.meta().state, BackpressureState::Normal);
}
}
+8 -204
View File
@@ -14,21 +14,12 @@
//! I/O scheduling types for adaptive buffer sizing and load management.
//!
//! # Migration Note
//!
//! This module contains types that are also available in `rustfs_io_core`.
//! For new code, prefer using types from `rustfs_io_core` directly:
//!
//! ```ignore
//! // Recommended: Use io-core types
//! use rustfs_io_core::{
//! IoLoadLevel, IoPriority, IoSchedulerConfig,
//! calculate_optimal_buffer_size, get_buffer_size_for_media,
//! };
//! ```
//!
//! This module remains for backward compatibility and provides additional
//! runtime monitoring features (`IoPriorityMetrics`, `IoStrategyDebugInfo`).
//! This is the live scheduling implementation. `rustfs_io_core` supplies the
//! shared config shapes (`IoSchedulerConfig`, `IoPriorityQueueConfig`) that the
//! types here project into through `to_core_config`, plus the `io_profile`
//! storage-media model; bandwidth samples come from `rustfs_io_metrics`.
//! Same-named io-core types are those config shapes, not a backing
//! implementation this module delegates to.
use rustfs_config::{KI_B, MI_B};
use rustfs_io_core::io_profile::{AccessPattern, StorageMedia, StorageProfile};
@@ -1762,169 +1753,6 @@ impl<T> IoPriorityQueue<T> {
}
}
// ============================================
// I/O Priority Queue Metrics
// ============================================
/// Global metrics for I/O priority queue monitoring.
///
/// These metrics are emitted through the shared metrics pipeline and provide
/// visibility into the priority queue behavior.
#[allow(dead_code)]
pub struct IoPriorityMetrics {
/// High priority queue depth.
pub high_queue_depth: AtomicU64,
/// Normal priority queue depth.
pub normal_queue_depth: AtomicU64,
/// Low priority queue depth.
pub low_queue_depth: AtomicU64,
/// High priority total wait time in nanoseconds.
pub high_wait_time_ns: AtomicU64,
/// Normal priority total wait time in nanoseconds.
pub normal_wait_time_ns: AtomicU64,
/// Low priority total wait time in nanoseconds.
pub low_wait_time_ns: AtomicU64,
/// Total starvation events count.
pub starvation_events: AtomicU64,
/// High priority requests processed.
pub high_processed: AtomicU64,
/// Normal priority requests processed.
pub normal_processed: AtomicU64,
/// Low priority requests processed.
pub low_processed: AtomicU64,
}
#[allow(dead_code)]
impl Default for IoPriorityMetrics {
fn default() -> Self {
Self::new()
}
}
#[allow(dead_code)]
impl IoPriorityMetrics {
/// Create a new metrics instance.
pub const fn new() -> Self {
Self {
high_queue_depth: AtomicU64::new(0),
normal_queue_depth: AtomicU64::new(0),
low_queue_depth: AtomicU64::new(0),
high_wait_time_ns: AtomicU64::new(0),
normal_wait_time_ns: AtomicU64::new(0),
low_wait_time_ns: AtomicU64::new(0),
starvation_events: AtomicU64::new(0),
high_processed: AtomicU64::new(0),
normal_processed: AtomicU64::new(0),
low_processed: AtomicU64::new(0),
}
}
/// Update queue depths from status.
#[allow(dead_code)]
pub fn update_queue_depths(&self, status: &IoQueueStatus) {
self.high_queue_depth
.store(status.high_priority_waiting as u64, Ordering::Relaxed);
self.normal_queue_depth
.store(status.normal_priority_waiting as u64, Ordering::Relaxed);
self.low_queue_depth
.store(status.low_priority_waiting as u64, Ordering::Relaxed);
}
/// Record a starvation event.
#[allow(dead_code)]
pub fn record_starvation(&self) {
self.starvation_events.fetch_add(1, Ordering::Relaxed);
}
/// Record a processed request.
#[allow(dead_code)]
pub fn record_processed(&self, priority: IoPriority) {
match priority {
IoPriority::High => self.high_processed.fetch_add(1, Ordering::Relaxed),
IoPriority::Normal => self.normal_processed.fetch_add(1, Ordering::Relaxed),
IoPriority::Low => self.low_processed.fetch_add(1, Ordering::Relaxed),
};
}
/// Record wait time for a priority level.
pub fn record_wait_time(&self, priority: IoPriority, wait_ns: u64) {
match priority {
IoPriority::High => self.high_wait_time_ns.fetch_add(wait_ns, Ordering::Relaxed),
IoPriority::Normal => self.normal_wait_time_ns.fetch_add(wait_ns, Ordering::Relaxed),
IoPriority::Low => self.low_wait_time_ns.fetch_add(wait_ns, Ordering::Relaxed),
};
}
/// Get high priority queue depth.
pub fn get_high_queue_depth(&self) -> u64 {
self.high_queue_depth.load(Ordering::Relaxed)
}
/// Get normal priority queue depth.
pub fn get_normal_queue_depth(&self) -> u64 {
self.normal_queue_depth.load(Ordering::Relaxed)
}
/// Get low priority queue depth.
pub fn get_low_queue_depth(&self) -> u64 {
self.low_queue_depth.load(Ordering::Relaxed)
}
/// Get total starvation events.
pub fn get_starvation_events(&self) -> u64 {
self.starvation_events.load(Ordering::Relaxed)
}
/// Get metrics summary for logging/debugging.
pub fn summary(&self) -> String {
format!(
"high_queue={}, normal_queue={}, low_queue={}, starvation={}, high_proc={}, normal_proc={}, low_proc={}",
self.get_high_queue_depth(),
self.get_normal_queue_depth(),
self.get_low_queue_depth(),
self.get_starvation_events(),
self.high_processed.load(Ordering::Relaxed),
self.normal_processed.load(Ordering::Relaxed),
self.low_processed.load(Ordering::Relaxed)
)
}
}
/// Global I/O priority metrics instance.
#[allow(dead_code)]
pub static IO_PRIORITY_METRICS: IoPriorityMetrics = IoPriorityMetrics::new();
/// Get optimized buffer size for I/O operations.
///
/// This function provides adaptive buffer sizing based on:
/// - File size (small files get smaller buffers)
/// - Concurrent request count (high concurrency gets smaller buffers)
/// - Base buffer size from configuration
///
/// # Arguments
///
/// * `file_size` - Size of the file being read/written (-1 for unknown)
///
/// # Returns
///
/// Optimal buffer size in bytes
///
/// # Example
///
/// ```ignore
/// let buffer_size = get_buffer_size_opt_in(1024 * 1024); // 1MB file
/// assert!(buffer_size >= 64 * 1024); // At least 64KB
/// ```
#[allow(dead_code)]
pub fn get_buffer_size_opt_in(file_size: i64) -> usize {
// Get base buffer size from configuration
let base_buffer_size =
rustfs_utils::get_env_usize(rustfs_config::ENV_OBJECT_IO_BUFFER_SIZE, rustfs_config::DEFAULT_OBJECT_IO_BUFFER_SIZE);
// Apply concurrency-aware adjustments
get_concurrency_aware_buffer_size(file_size, base_buffer_size)
}
// ============================================
// Unit Tests
// ============================================
@@ -1933,13 +1761,12 @@ pub fn get_buffer_size_opt_in(file_size: i64) -> usize {
#[allow(unused_imports)]
mod tests {
use super::{
IoLoadLevel, IoPriority, IoPriorityMetrics, IoPriorityQueue, IoPriorityQueueConfig, IoSchedulerConfig,
IoSchedulingContext, IoStrategy, get_advanced_buffer_size, get_buffer_size_opt_in, get_concurrency_aware_buffer_size,
IoLoadLevel, IoPriority, IoPriorityQueue, IoPriorityQueueConfig, IoSchedulerConfig, IoSchedulingContext, IoStrategy,
get_advanced_buffer_size, get_concurrency_aware_buffer_size,
};
use rustfs_io_core::io_profile::{AccessPattern, StorageMedia};
use rustfs_io_metrics::bandwidth::{BandwidthSnapshot, BandwidthTier};
use serial_test::serial;
use std::sync::atomic::Ordering;
use std::time::Duration;
#[tokio::test]
@@ -2126,29 +1953,6 @@ mod tests {
assert_eq!(config.starvation_threshold_secs, 120);
}
#[tokio::test]
#[serial]
async fn test_io_priority_metrics() {
let metrics = IoPriorityMetrics::new();
// Test initial state
assert_eq!(metrics.get_high_queue_depth(), 0);
assert_eq!(metrics.get_normal_queue_depth(), 0);
assert_eq!(metrics.get_low_queue_depth(), 0);
assert_eq!(metrics.get_starvation_events(), 0);
// Test recording
metrics.record_starvation();
assert_eq!(metrics.get_starvation_events(), 1);
metrics.record_processed(IoPriority::High);
metrics.record_processed(IoPriority::High);
metrics.record_processed(IoPriority::Normal);
assert_eq!(metrics.high_processed.load(Ordering::Relaxed), 2);
assert_eq!(metrics.normal_processed.load(Ordering::Relaxed), 1);
}
// ============================================
// Multi-Factor Strategy Tests
// ============================================
+149 -4
View File
@@ -65,6 +65,11 @@ pub struct ConcurrencyManager {
bandwidth_monitor: Arc<Mutex<BandwidthMonitor>>,
/// Metrics collector for I/O latency tracking (P50, P95, P99)
metrics_collector: Arc<MetricsCollector>,
/// Experimental fixed-count foreground PutObject admission gate.
put_admission_semaphore: Arc<Semaphore>,
put_admission_enabled: bool,
put_admission_limit: usize,
put_admission_wait_timeout: Duration,
}
impl std::fmt::Debug for ConcurrencyManager {
@@ -114,6 +119,18 @@ pub enum DiskReadAdmission {
Rejected,
}
/// Outcome of foreground PutObject request admission.
#[derive(Debug)]
pub enum PutObjectAdmission {
/// Foreground PUT admission is disabled; proceed on the legacy path.
Disabled,
/// Request is admitted and must hold the permit until the store write
/// returns or the request fails before mutation.
Admitted(tokio::sync::OwnedSemaphorePermit),
/// The fixed-count gate stayed full until the configured wait timeout.
Rejected,
}
#[allow(dead_code)]
impl ConcurrencyManager {
/// Create a new concurrency manager with default settings
@@ -161,6 +178,18 @@ impl ConcurrencyManager {
// Initialize metrics collector for I/O latency tracking
// Keep 1000 samples for P95/P99 calculation
let metrics_collector = Arc::new(MetricsCollector::new(performance_metrics, 1000));
let put_admission_enabled = rustfs_utils::get_env_bool(
rustfs_config::ENV_PUT_FOREGROUND_ADMISSION_ENABLE,
rustfs_config::DEFAULT_PUT_FOREGROUND_ADMISSION_ENABLE,
);
let put_admission_limit = rustfs_utils::get_env_usize(
rustfs_config::ENV_PUT_FOREGROUND_ADMISSION_LIMIT,
rustfs_config::DEFAULT_PUT_FOREGROUND_ADMISSION_LIMIT,
);
let put_admission_wait_timeout = Duration::from_millis(rustfs_utils::get_env_u64(
rustfs_config::ENV_PUT_FOREGROUND_ADMISSION_WAIT_TIMEOUT_MS,
rustfs_config::DEFAULT_PUT_FOREGROUND_ADMISSION_WAIT_TIMEOUT_MS,
));
// Build queue config directly from scheduler config.
let queue_config = IoPriorityQueueConfig::from_scheduler_config(&scheduler_config);
@@ -176,6 +205,10 @@ impl ConcurrencyManager {
pattern_detector,
bandwidth_monitor,
metrics_collector,
put_admission_semaphore: Arc::new(Semaphore::new(if put_admission_enabled { put_admission_limit } else { 0 })),
put_admission_enabled,
put_admission_limit,
put_admission_wait_timeout,
}
}
@@ -199,6 +232,16 @@ impl ConcurrencyManager {
self.degraded_read_semaphore.close();
}
#[cfg(test)]
pub(crate) fn with_put_admission_for_test(enabled: bool, limit: usize, wait_timeout: Duration) -> Self {
let mut manager = Self::new();
manager.put_admission_semaphore = Arc::new(Semaphore::new(if enabled { limit } else { 0 }));
manager.put_admission_enabled = enabled;
manager.put_admission_limit = limit;
manager.put_admission_wait_timeout = wait_timeout;
manager
}
/// Track a GetObject request
pub fn track_request() -> GetObjectGuard {
GetObjectGuard::new()
@@ -284,6 +327,32 @@ impl ConcurrencyManager {
}
}
/// Admit a foreground PutObject request under the experimental fixed-count gate.
///
/// The default-off path returns [`PutObjectAdmission::Disabled`] without
/// touching the semaphore, preserving legacy behavior. When enabled, the
/// permit must be acquired before body ingest and held until the store write
/// returns, so saturated foreground writes can fail with `SlowDown` before
/// creating visible side effects.
pub async fn admit_put_object(&self) -> Result<PutObjectAdmission, tokio::sync::AcquireError> {
if !self.put_admission_enabled || self.put_admission_limit == 0 {
return Ok(PutObjectAdmission::Disabled);
}
if self.put_admission_wait_timeout.is_zero() {
return Ok(match self.put_admission_semaphore.clone().try_acquire_owned() {
Ok(permit) => PutObjectAdmission::Admitted(permit),
Err(tokio::sync::TryAcquireError::NoPermits) => PutObjectAdmission::Rejected,
Err(tokio::sync::TryAcquireError::Closed) => PutObjectAdmission::Rejected,
});
}
match tokio::time::timeout(self.put_admission_wait_timeout, self.put_admission_semaphore.clone().acquire_owned()).await {
Ok(permit) => Ok(PutObjectAdmission::Admitted(permit?)),
Err(_) => Ok(PutObjectAdmission::Rejected),
}
}
// ============================================
// Adaptive I/O Strategy Methods
// ============================================
@@ -692,8 +761,16 @@ impl ConcurrencyManager {
/// Get a read-only workload admission snapshot for foreground writes.
pub fn put_object_admission_snapshot(&self) -> WorkloadAdmissionSnapshot {
let active = PutObjectGuard::concurrent_count();
let limit = self.scheduler_config.max_concurrent_reads;
let (active, limit, hard_gate_enabled) = if self.put_admission_enabled && self.put_admission_limit > 0 {
(
self.put_admission_limit
.saturating_sub(self.put_admission_semaphore.available_permits()),
self.put_admission_limit,
true,
)
} else {
(PutObjectGuard::concurrent_count(), self.scheduler_config.max_concurrent_reads, false)
};
let state = if limit == 0 {
AdmissionState::Disabled
} else if active >= limit {
@@ -706,7 +783,10 @@ impl ConcurrencyManager {
WorkloadAdmissionSnapshot::new(WorkloadClass::ForegroundWrite, state).with_counts(Some(active), None, Some(limit));
match state {
AdmissionState::Disabled => admission.with_reason("foreground write pressure tracking disabled"),
AdmissionState::Disabled => admission.with_reason("foreground write admission disabled"),
AdmissionState::Saturated if hard_gate_enabled => {
admission.with_reason("foreground write admission permits exhausted")
}
AdmissionState::Saturated => admission.with_reason("foreground write concurrency reached local pressure limit"),
_ => admission,
}
@@ -783,7 +863,7 @@ impl Default for ConcurrencyManager {
mod integration_tests {
use super::super::io_schedule::{IoLoadLevel, IoPriority};
use super::super::request_guard::GetObjectGuard;
use super::ConcurrencyManager;
use super::{ConcurrencyManager, PutObjectAdmission};
use crate::storage::storage_api::concurrency_consumer::PutObjectGuard;
use rustfs_concurrency::{AdmissionState, WorkloadAdmissionSnapshotProvider, WorkloadClass};
use rustfs_io_core::io_profile::{AccessPattern, StorageMedia};
@@ -880,6 +960,71 @@ mod integration_tests {
crate::storage::concurrency::reset_active_put_requests();
}
#[tokio::test]
#[serial]
async fn test_concurrency_manager_put_admission_disabled_does_not_touch_gate() {
let manager = ConcurrencyManager::with_put_admission_for_test(false, 1, Duration::ZERO);
let admission = manager
.admit_put_object()
.await
.expect("disabled put admission must not close");
assert!(matches!(admission, PutObjectAdmission::Disabled));
assert_eq!(manager.put_admission_semaphore.available_permits(), 0);
assert_eq!(manager.put_object_admission_snapshot().state, AdmissionState::Open);
}
#[tokio::test]
#[serial]
async fn test_concurrency_manager_put_admission_rejects_when_limit_full() {
let manager = ConcurrencyManager::with_put_admission_for_test(true, 1, Duration::ZERO);
let first = manager.admit_put_object().await.expect("first put admission should acquire");
assert!(matches!(first, PutObjectAdmission::Admitted(_)));
assert_eq!(manager.put_object_admission_snapshot().state, AdmissionState::Saturated);
let second = manager
.admit_put_object()
.await
.expect("full put admission gate should reject, not close");
assert!(matches!(second, PutObjectAdmission::Rejected));
}
#[tokio::test]
#[serial]
async fn test_concurrency_manager_put_admission_reuses_released_permit() {
let manager = ConcurrencyManager::with_put_admission_for_test(true, 1, Duration::ZERO);
let first = manager.admit_put_object().await.expect("first put admission should acquire");
drop(first);
let second = manager
.admit_put_object()
.await
.expect("released put admission permit should be reusable");
assert!(matches!(second, PutObjectAdmission::Admitted(_)));
}
#[tokio::test(start_paused = true)]
#[serial]
async fn test_concurrency_manager_put_admission_wait_timeout_rejects() {
let manager = ConcurrencyManager::with_put_admission_for_test(true, 1, Duration::from_secs(5));
let held = manager.admit_put_object().await.expect("first put admission should acquire");
let waiter_manager = manager.clone();
let waiter = tokio::spawn(async move { waiter_manager.admit_put_object().await });
tokio::task::yield_now().await;
tokio::time::advance(Duration::from_secs(5)).await;
let admission = waiter
.await
.expect("put admission waiter task must not panic")
.expect("put admission gate must stay open");
assert!(matches!(admission, PutObjectAdmission::Rejected));
drop(held);
}
#[tokio::test]
#[serial]
async fn test_concurrency_manager_workload_admission_registry_covers_required_classes() {
+11 -61
View File
@@ -24,16 +24,14 @@
//! - **Concurrency Management**: Coordination of concurrent GetObject requests
//! - **Request Tracking**: RAII guards for request lifecycle management
//!
//! # Migration Note
//! # Relationship to the shared crates
//!
//! Core algorithms have been migrated to `rustfs-io-core` and metrics to
//! `rustfs-io-metrics`. This module maintains API compatibility while
//! delegating to the new implementations.
//! The scheduling algorithm lives in [`io_schedule`], not in `rustfs-io-core`:
//! this module does not delegate to it. `rustfs-io-core` owns the shared
//! config shapes and the `io_profile` storage-media model that [`io_schedule`]
//! consumes, and `rustfs-io-metrics` owns bandwidth sampling and metric
//! recording.
// Sub-modules
// pub mod bandwidth_monitor; // Migrated to rustfs-io-metrics
// pub mod global_metrics; // Migrated to rustfs-io-metrics
// pub mod io_profile; // Migrated to rustfs-io-core
pub mod io_schedule;
pub mod manager;
pub mod request_guard;
@@ -45,34 +43,15 @@ pub mod request_guard;
// I/O scheduling types (from io_schedule.rs for backward compatibility)
#[allow(unused_imports)]
pub use io_schedule::{
IO_PRIORITY_METRICS, IoLoadLevel, IoPriority, IoPriorityMetrics, IoPriorityQueue, IoPriorityQueueConfig, IoQueueStatus,
IoSchedulerConfig, IoStrategy, get_advanced_buffer_size, get_buffer_size_opt_in, get_concurrency_aware_buffer_size,
get_put_concurrency_aware_buffer_size,
IoLoadLevel, IoPriority, IoPriorityQueue, IoPriorityQueueConfig, IoQueueStatus, IoSchedulerConfig, IoStrategy,
get_advanced_buffer_size, get_concurrency_aware_buffer_size, get_put_concurrency_aware_buffer_size,
};
// Request tracking
pub use request_guard::{GetObjectGuard, PutObjectGuard};
// Concurrency manager
pub use manager::{ConcurrencyManager, DiskReadAdmission};
// ============================================
// New Module Re-exports (for gradual migration)
// ============================================
// Re-export types from rustfs-io-core for convenience
pub use rustfs_io_core::{
// Backpressure types
BackpressureMonitor,
// Deadlock detection types
DeadlockDetector,
// Scheduler types
IoScheduler,
// Lock optimization types
LockOptimizer,
};
// Re-export types from rustfs-io-metrics for convenience
pub use manager::{ConcurrencyManager, DiskReadAdmission, PutObjectAdmission};
// ============================================
// Helper Functions
@@ -83,37 +62,8 @@ pub fn get_concurrency_manager() -> &'static ConcurrencyManager {
ConcurrencyManager::global()
}
/// Reset the active get requests counter (for testing).
#[allow(dead_code)]
pub fn reset_active_get_requests() {
io_schedule::ACTIVE_GET_REQUESTS.store(0, std::sync::atomic::Ordering::Relaxed);
}
#[allow(dead_code)]
/// Reset the active put requests counter (for testing).
#[cfg(test)]
pub fn reset_active_put_requests() {
io_schedule::ACTIVE_PUT_REQUESTS.store(0, std::sync::atomic::Ordering::Relaxed);
}
/// Create a new I/O scheduler with default configuration.
#[allow(dead_code)]
pub fn create_io_scheduler() -> IoScheduler {
IoScheduler::with_defaults()
}
/// Create a new backpressure monitor with default configuration.
#[allow(dead_code)]
pub fn create_backpressure_monitor() -> BackpressureMonitor {
BackpressureMonitor::with_defaults()
}
/// Create a new deadlock detector with default configuration.
#[allow(dead_code)]
pub fn create_deadlock_detector() -> DeadlockDetector {
DeadlockDetector::with_defaults()
}
/// Create a new lock optimizer with default configuration.
#[allow(dead_code)]
pub fn create_lock_optimizer() -> LockOptimizer {
LockOptimizer::with_defaults()
}
+2 -80
View File
@@ -14,17 +14,15 @@
//! Integration tests for concurrent request fix.
//!
//! These tests verify that the timeout, backpressure, and deadlock detection
//! mechanisms work correctly under high concurrency scenarios.
//! These tests verify that the timeout and deadlock detection mechanisms work
//! correctly under high concurrency scenarios.
#[cfg(test)]
mod tests {
use crate::storage::backpressure::{BackpressureMonitor, BackpressureState, ObjectPipeBackpressurePolicy};
use crate::storage::concurrency::{IoLoadLevel, IoPriority};
use crate::storage::deadlock_detector::{
DeadlockDetector, LockInfo, LockType, RequestHangDetectionPolicy, RequestResourceTracker,
};
use crate::storage::lock_optimizer::{LockOptimizeConfig, LockOptimizer, LockStats};
use crate::storage::timeout_wrapper::{GetObjectTimeoutPolicy, RequestTimeoutWrapper, TimedGetObjectResult};
use std::time::Duration;
@@ -114,82 +112,6 @@ mod tests {
}
}
// ============================================
// Backpressure Tests
// ============================================
#[test]
fn test_backpressure_config_defaults() {
let config = ObjectPipeBackpressurePolicy::default();
assert_eq!(config.buffer_size, 4 * 1024 * 1024); // 4MB
assert_eq!(config.high_watermark, 80);
assert_eq!(config.low_watermark, 50);
}
#[test]
fn test_backpressure_monitor_state_transitions() {
let config = ObjectPipeBackpressurePolicy {
buffer_size: 1000,
high_watermark: 80,
low_watermark: 50,
};
let monitor = BackpressureMonitor::with_config(config);
// Initially normal
assert_eq!(monitor.state(), BackpressureState::Normal);
// Write to reach high watermark
let state = monitor.on_write(850);
assert_eq!(state, BackpressureState::HighWatermark);
// Read to go below low watermark
let state = monitor.on_read(400);
assert_eq!(state, BackpressureState::Normal);
}
#[test]
fn test_backpressure_usage_percent() {
let config = ObjectPipeBackpressurePolicy {
buffer_size: 1000,
high_watermark: 80,
low_watermark: 50,
};
let monitor = BackpressureMonitor::with_config(config);
monitor.on_write(500);
assert!((monitor.usage_percent() - 50.0).abs() < 1.0);
}
// ============================================
// Lock Optimizer Tests
// ============================================
#[test]
fn test_lock_optimize_config_defaults() {
let config = LockOptimizeConfig::default();
assert!(config.enabled);
assert_eq!(config.acquire_timeout, Duration::from_secs(5));
}
#[test]
fn test_lock_stats_tracking() {
let stats = LockStats::new();
stats.record_acquire();
stats.record_early_release(Duration::from_millis(100));
stats.record_early_release(Duration::from_millis(200));
assert_eq!(stats.locks_acquired.load(std::sync::atomic::Ordering::Relaxed), 1);
assert_eq!(stats.locks_released_early.load(std::sync::atomic::Ordering::Relaxed), 2);
assert_eq!(stats.max_hold_time(), Duration::from_millis(200));
}
#[test]
fn test_lock_optimizer_creation() {
let optimizer = LockOptimizer::new();
assert!(optimizer.is_enabled());
}
// ============================================
// I/O Priority Tests
// ============================================
-458
View File
@@ -1,458 +0,0 @@
// Copyright 2024 RustFS Team
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//! Lock Optimization for GetObject Operations.
//!
//! This module provides optimized lock management for read operations,
//! reducing lock contention by releasing locks early (after metadata read)
//! rather than holding them for the entire data transfer duration.
//!
//! # Migration Note
//!
//! For new code, consider using `rustfs_io_core::LockOptimizer` which provides
//! the same core functionality with better separation of concerns. This module
//! remains for backward compatibility and storage-specific configuration.
//!
//! ```ignore
//! // Recommended: Use io-core directly
//! use rustfs_io_core::LockOptimizer;
//! let optimizer = LockOptimizer::with_defaults();
//! ```
// Allow dead_code for public API that may be used by external modules or future features
//! # Key Features
//!
//! - Early lock release after metadata read
//! - Lock hold time monitoring
//! - Configurable optimization (can be disabled for debugging)
//! - Lock contention metrics emitted through the shared metrics pipeline
//!
//! # Architecture
//!
//! ```text
//! Traditional: [Acquire Lock] --> [Read Metadata] --> [Transfer Data] --> [Release Lock]
//! |<------------------ Lock Held ------------------>|
//!
//! Optimized: [Acquire Lock] --> [Read Metadata] --> [Release Lock] --> [Transfer Data]
//! |<- Lock Held ->|
//! ```
use std::sync::Arc;
use std::sync::atomic::{AtomicU64, Ordering};
use std::time::{Duration, Instant};
use tracing::debug;
use metrics::histogram;
/// Lock optimization configuration.
#[derive(Debug, Clone)]
pub struct LockOptimizeConfig {
/// Whether to enable lock optimization.
/// When enabled, read locks are released after metadata read.
/// When disabled, locks are held for the entire operation (traditional behavior).
pub enabled: bool,
/// Lock acquisition timeout.
pub acquire_timeout: Duration,
}
impl Default for LockOptimizeConfig {
fn default() -> Self {
Self {
enabled: rustfs_config::DEFAULT_OBJECT_LOCK_OPTIMIZATION_ENABLE,
acquire_timeout: Duration::from_secs(rustfs_config::DEFAULT_OBJECT_LOCK_ACQUIRE_TIMEOUT),
}
}
}
impl LockOptimizeConfig {
/// Load configuration from environment variables.
pub fn from_env() -> Self {
let enabled = rustfs_utils::get_env_bool(
rustfs_config::ENV_OBJECT_LOCK_OPTIMIZATION_ENABLE,
rustfs_config::DEFAULT_OBJECT_LOCK_OPTIMIZATION_ENABLE,
);
let acquire_timeout = Duration::from_secs(rustfs_utils::get_env_u64(
rustfs_config::ENV_OBJECT_LOCK_ACQUIRE_TIMEOUT,
rustfs_config::DEFAULT_OBJECT_LOCK_ACQUIRE_TIMEOUT,
));
Self {
enabled,
acquire_timeout,
}
}
}
/// Statistics for lock optimization monitoring.
#[derive(Debug, Default)]
pub struct LockStats {
/// Total locks acquired.
pub locks_acquired: AtomicU64,
/// Total locks released early.
pub locks_released_early: AtomicU64,
/// Total lock hold time in microseconds.
pub total_hold_time_us: AtomicU64,
/// Maximum lock hold time in microseconds.
pub max_hold_time_us: AtomicU64,
}
impl LockStats {
/// Create new lock statistics.
pub fn new() -> Self {
Self::default()
}
/// Record a lock acquisition.
pub fn record_acquire(&self) {
self.locks_acquired.fetch_add(1, Ordering::Relaxed);
}
/// Record an early lock release.
pub fn record_early_release(&self, hold_time: Duration) {
self.locks_released_early.fetch_add(1, Ordering::Relaxed);
self.record_hold_time(hold_time);
}
/// Record lock hold time.
fn record_hold_time(&self, hold_time: Duration) {
let hold_time_us = hold_time.as_micros() as u64;
self.total_hold_time_us.fetch_add(hold_time_us, Ordering::Relaxed);
// Update max hold time
let mut current_max = self.max_hold_time_us.load(Ordering::Relaxed);
while hold_time_us > current_max {
match self
.max_hold_time_us
.compare_exchange_weak(current_max, hold_time_us, Ordering::Relaxed, Ordering::Relaxed)
{
Ok(_) => break,
Err(actual) => current_max = actual,
}
}
}
/// Get average hold time.
pub fn avg_hold_time(&self) -> Duration {
let total = self.total_hold_time_us.load(Ordering::Relaxed);
let count = self.locks_released_early.load(Ordering::Relaxed);
total.checked_div(count).map(Duration::from_micros).unwrap_or(Duration::ZERO)
}
/// Get maximum hold time.
pub fn max_hold_time(&self) -> Duration {
Duration::from_micros(self.max_hold_time_us.load(Ordering::Relaxed))
}
}
/// Global lock statistics.
static LOCK_STATS: std::sync::OnceLock<Arc<LockStats>> = std::sync::OnceLock::new();
/// Get global lock statistics.
pub fn get_lock_stats() -> Arc<LockStats> {
LOCK_STATS.get_or_init(|| Arc::new(LockStats::new())).clone()
}
/// An optimized lock guard that supports early release.
///
/// This wraps the actual lock guard and provides:
/// - Early release capability (before drop)
/// - Hold time tracking
/// - Metrics reporting
pub struct OptimizedLockGuard<G> {
/// The underlying lock guard.
guard: Option<G>,
/// When the lock was acquired.
acquire_time: Instant,
/// Whether the lock has been released.
released: bool,
/// Lock resource name (for logging).
resource: String,
/// Statistics reference.
stats: Arc<LockStats>,
}
impl<G> OptimizedLockGuard<G> {
/// Create a new optimized lock guard.
pub fn new(guard: G, resource: impl Into<String>) -> Self {
let stats = get_lock_stats();
stats.record_acquire();
Self {
guard: Some(guard),
acquire_time: Instant::now(),
released: false,
resource: resource.into(),
stats,
}
}
/// Get the lock hold time so far.
pub fn hold_time(&self) -> Duration {
self.acquire_time.elapsed()
}
/// Check if the lock has been released.
pub fn is_released(&self) -> bool {
self.released
}
/// Release the lock early (before drop).
///
/// This is the key optimization: releasing the lock after
/// metadata read rather than waiting for the entire operation.
pub fn early_release(&mut self) {
if self.released {
return;
}
let hold_time = self.hold_time();
self.guard.take();
self.released = true;
self.stats.record_early_release(hold_time);
histogram!("rustfs_lock_hold_duration_seconds").record(hold_time.as_secs_f64());
debug!(
resource = %self.resource,
hold_time_ms = hold_time.as_millis(),
"Lock released early (optimization active)"
);
}
/// Get a reference to the underlying guard.
pub fn as_ref(&self) -> Option<&G> {
if self.released { None } else { self.guard.as_ref() }
}
}
impl<G> Drop for OptimizedLockGuard<G> {
fn drop(&mut self) {
if !self.released {
let hold_time = self.hold_time();
self.guard.take();
self.released = true;
self.stats.record_early_release(hold_time);
histogram!("rustfs_lock_hold_duration_seconds").record(hold_time.as_secs_f64());
debug!(
resource = %self.resource,
hold_time_ms = hold_time.as_millis(),
"Lock released on drop (normal release)"
);
}
}
}
/// A scope guard that releases a lock when it goes out of scope.
///
/// This is a simpler version of OptimizedLockGuard for cases
/// where we just need RAII semantics without tracking.
pub struct LockScopeGuard<G> {
guard: Option<G>,
}
impl<G> LockScopeGuard<G> {
/// Create a new scope guard.
pub fn new(guard: G) -> Self {
Self { guard: Some(guard) }
}
/// Release the lock early.
pub fn release(&mut self) {
self.guard.take();
}
}
impl<G> Drop for LockScopeGuard<G> {
fn drop(&mut self) {
self.guard.take();
}
}
/// Helper for managing lock optimization in GetObject operations.
///
/// This provides a clean interface for the common pattern:
/// 1. Acquire lock
/// 2. Read metadata
/// 3. Release lock (if optimization enabled)
/// 4. Transfer data (without lock)
pub struct LockOptimizer {
/// Configuration.
config: LockOptimizeConfig,
}
impl LockOptimizer {
/// Create a new lock optimizer with default configuration.
pub fn new() -> Self {
Self {
config: LockOptimizeConfig::from_env(),
}
}
/// Create a new lock optimizer with custom configuration.
pub fn with_config(config: LockOptimizeConfig) -> Self {
Self { config }
}
/// Check if lock optimization is enabled.
pub fn is_enabled(&self) -> bool {
self.config.enabled
}
/// Get the lock acquisition timeout.
pub fn acquire_timeout(&self) -> Duration {
self.config.acquire_timeout
}
/// Wrap a lock guard for optimization.
pub fn wrap_guard<G>(&self, guard: G, resource: impl Into<String>) -> OptimizedLockGuard<G> {
OptimizedLockGuard::new(guard, resource)
}
/// Execute a metadata read operation with lock optimization.
///
/// This is the main entry point for optimized lock usage:
/// - If optimization is enabled: lock is released after metadata_fn completes
/// - If optimization is disabled: lock is held until the returned guard is dropped
///
/// # Arguments
///
/// * `guard` - The lock guard to optimize
/// * `resource` - Resource name for logging
/// * `metadata_fn` - Function to read metadata while holding lock
///
/// # Returns
///
/// A tuple of (metadata result, optional guard to hold for later release)
pub async fn with_optimized_lock<G, F, Fut, T>(
&self,
guard: G,
resource: impl Into<String>,
metadata_fn: F,
) -> (T, Option<OptimizedLockGuard<G>>)
where
F: FnOnce() -> Fut,
Fut: std::future::Future<Output = T>,
{
let resource = resource.into();
let mut optimized = OptimizedLockGuard::new(guard, &resource);
// Execute metadata read while holding lock
let result = metadata_fn().await;
if self.config.enabled {
// Release lock early
optimized.early_release();
(result, None)
} else {
// Keep lock for caller to release
(result, Some(optimized))
}
}
}
impl Default for LockOptimizer {
fn default() -> Self {
Self::new()
}
}
/// Check if lock optimization is enabled globally.
pub fn is_lock_optimization_enabled() -> bool {
rustfs_utils::get_env_bool(
rustfs_config::ENV_OBJECT_LOCK_OPTIMIZATION_ENABLE,
rustfs_config::DEFAULT_OBJECT_LOCK_OPTIMIZATION_ENABLE,
)
}
#[cfg(test)]
#[allow(unused_imports)]
mod tests {
use super::{LockOptimizeConfig, LockOptimizer, LockStats, OptimizedLockGuard};
use std::sync::Mutex;
use std::sync::atomic::Ordering;
use std::time::Duration;
#[test]
fn test_lock_optimize_config_default() {
let config = LockOptimizeConfig::default();
assert!(config.enabled);
assert_eq!(config.acquire_timeout, Duration::from_secs(5));
}
#[test]
fn test_lock_stats() {
let stats = LockStats::new();
stats.record_acquire();
stats.record_early_release(Duration::from_millis(100));
stats.record_early_release(Duration::from_millis(200));
assert_eq!(stats.locks_acquired.load(Ordering::Relaxed), 1);
assert_eq!(stats.locks_released_early.load(Ordering::Relaxed), 2);
assert_eq!(stats.max_hold_time(), Duration::from_millis(200));
}
#[test]
fn test_optimized_lock_guard() {
let guard = Mutex::new(42);
let locked = guard.lock().unwrap();
let mut optimized = OptimizedLockGuard::new(locked, "test-resource");
assert!(!optimized.is_released());
assert!(optimized.hold_time() < Duration::from_secs(1));
optimized.early_release();
assert!(optimized.is_released());
}
#[test]
fn test_lock_optimizer() {
let optimizer = LockOptimizer::new();
assert!(optimizer.is_enabled());
}
#[tokio::test]
async fn test_with_optimized_lock_enabled() {
let optimizer = LockOptimizer::new();
let guard = Mutex::new(42);
let locked = guard.lock().unwrap();
let (result, returned_guard) = optimizer.with_optimized_lock(locked, "test-resource", || async { 100 }).await;
assert_eq!(result, 100);
// With optimization enabled, guard should be None (released early)
assert!(returned_guard.is_none());
}
#[tokio::test]
async fn test_with_optimized_lock_disabled() {
let config = LockOptimizeConfig {
enabled: false,
acquire_timeout: Duration::from_secs(5),
};
let optimizer = LockOptimizer::with_config(config);
let guard = Mutex::new(42);
let locked = guard.lock().unwrap();
let (result, returned_guard) = optimizer.with_optimized_lock(locked, "test-resource", || async { 100 }).await;
assert_eq!(result, 100);
// With optimization disabled, guard should be Some (held for later)
assert!(returned_guard.is_some());
}
}
-2
View File
@@ -13,12 +13,10 @@
// limitations under the License.
pub mod access;
pub mod backpressure;
pub mod concurrency;
pub mod deadlock_detector;
pub mod ecfs;
pub(crate) mod helper;
pub mod lock_optimizer;
pub mod options;
pub mod request_context;
pub mod rpc;
+8 -2
View File
@@ -571,8 +571,12 @@ async fn execute_heal_control_envelope_with_manager(
admission: receipt.result.into(),
}
}
rustfs_protos::heal_control::ExecutableCommand::Query { heal_path, client_token } => {
let response = timeout(remaining, processor.execute_query_request(heal_path, client_token))
rustfs_protos::heal_control::ExecutableCommand::Query {
heal_path,
client_token,
since_seq,
} => {
let response = timeout(remaining, processor.execute_query_request_since(heal_path, client_token, since_seq))
.await
.map_err(|_| Status::deadline_exceeded("heal control query expired before execution"))?
.map_err(|_| Status::internal("heal control query failed"))?;
@@ -2517,6 +2521,7 @@ mod tests {
metadata(),
"bucket/prefix".to_string(),
canonical_token.clone(),
None,
)
.unwrap();
let query_result = execute_heal_control_envelope_with_manager(query, coordinator_epoch, Some(Arc::clone(&manager)))
@@ -2555,6 +2560,7 @@ mod tests {
metadata(),
"bucket/prefix".to_string(),
canonical_token,
None,
)
.unwrap();
let stopped_result = execute_heal_control_envelope_with_manager(stopped_query, coordinator_epoch, Some(manager))
@@ -585,6 +585,7 @@ mod tests {
let decoded = decode_node_heal_status(&encoded).expect("fixed v1 fixture should decode");
assert_eq!(decoded.info().bitrot_start_cycle, 9);
assert_eq!(decoded.operations.queue_length, 2);
assert_eq!(decoded.operations.queued_by_source.mrf, 0);
}
#[test]
+1 -1
View File
@@ -117,7 +117,7 @@ pub(crate) mod access_consumer {
pub(crate) mod concurrency_consumer {
pub(crate) use super::super::concurrency::{
ConcurrencyManager, DiskReadAdmission, GetObjectGuard, IoQueueStatus, IoStrategy, PutObjectGuard,
ConcurrencyManager, DiskReadAdmission, GetObjectGuard, IoQueueStatus, IoStrategy, PutObjectAdmission, PutObjectGuard,
get_concurrency_aware_buffer_size, get_concurrency_manager, get_put_concurrency_aware_buffer_size,
};
}
+78 -9
View File
@@ -1,10 +1,30 @@
#!/usr/bin/env bash
set -euo pipefail
# Guard: outward README and CHANGELOG material must not make an unsupported
# FIPS validation or certification claim. The detailed policy and permitted
# qualifiers live in docs/operations/kms-cryptographic-compliance.md; this
# check intentionally scans only the two public project-facing documents.
# Guard: cryptographic capability wording must not over-claim what RustFS
# actually does. Two independent blocks, both anchored to the policy in
# docs/operations/kms-cryptographic-compliance.md:
#
# 1. Outward README and CHANGELOG material must not make an unsupported
# FIPS validation or certification claim. This block intentionally scans
# only the two public project-facing documents; the permitted qualifiers
# live in the policy document.
#
# 2. Nothing in crates/kms may describe the Vault KV2 backend as wrapping
# key material through Vault's Transit engine. `KmsBackend::VaultKv2`
# stores RustFS-wrapped key material in Vault's KV v2 engine and never
# calls Transit (see crates/kms/src/config.rs and
# docs/operations/kms-backend-security.md), so such prose tells operators
# their key material is cryptographically isolated inside Vault when it is
# not.
#
# Block 2 replaces the unit test `test_vault_kv2_sources_do_not_claim_transit_wrapping`
# that used to live in crates/kms/src/config.rs (rustfs/backlog#1884). The
# invariant is a documentation-claim invariant, so it has no behavioral twin by
# construction and belongs in a wording guard rather than in a test. The test
# could only see four `include_str!`-pinned files and stopped compiling —
# rather than reporting a violation — the moment one of them was renamed; this
# block scans every file in the crate and reports a rename explicitly.
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
ROOT_DIR="${CHECK_FIPS_WORDING_ROOT:-$(cd "${SCRIPT_DIR}/.." && pwd)}"
@@ -21,12 +41,34 @@ FORBIDDEN_PATTERNS=(
'(meets|satisfies)[[:space:]]+FIPS'
)
status=0
KMS_CRATE_DIR="crates/kms"
# The four files the retired unit test pinned with include_str!. They stay
# listed so that moving one out of crates/kms is reported here instead of
# silently shrinking the scan; the scan itself is not limited to them.
KMS_PINNED_SOURCES=(
"crates/kms/src/config.rs"
"crates/kms/src/api_types.rs"
"crates/kms/src/backends/vault.rs"
"crates/kms/src/lib.rs"
)
# Literal, case-sensitive, and byte-for-byte the needles the retired test built
# at runtime via format!("wrapping via {}", "Transit") and friends.
KMS_VAULT_KV2_FORBIDDEN=(
'wrapping via Transit'
'KV v2 + Transit'
'KV2+Transit'
"you would use Vault's transit engine"
)
fips_status=0
kms_status=0
for target in "${TARGETS[@]}"; do
if [[ ! -f "$target" ]]; then
printf 'FIPS wording guard failed: %s is missing\n' "$target" >&2
status=1
fips_status=1
continue
fi
@@ -35,14 +77,41 @@ for target in "${TARGETS[@]}"; do
if [[ -n "$matches" ]]; then
printf 'FIPS wording guard failed: forbidden pattern /%s/ in %s:\n%s\n' \
"$pattern" "$target" "$matches" >&2
status=1
fips_status=1
fi
done
done
if [[ "$status" -ne 0 ]]; then
for source in "${KMS_PINNED_SOURCES[@]}"; do
if [[ ! -f "$source" ]]; then
printf 'KMS wording guard failed: %s is missing; update KMS_PINNED_SOURCES in scripts/check_fips_wording.sh after moving it\n' \
"$source" >&2
kms_status=1
fi
done
if [[ -d "$KMS_CRATE_DIR" ]]; then
for pattern in "${KMS_VAULT_KV2_FORBIDDEN[@]}"; do
matches="$(grep -r -F -n -- "$pattern" "$KMS_CRATE_DIR" || true)"
if [[ -n "$matches" ]]; then
printf 'KMS wording guard failed: forbidden Vault KV2 claim "%s" in %s:\n%s\n' \
"$pattern" "$KMS_CRATE_DIR" "$matches" >&2
kms_status=1
fi
done
fi
if [[ "$fips_status" -ne 0 ]]; then
printf 'Remove unsupported FIPS validation wording from README.md or CHANGELOG.md.\n' >&2
exit "$status"
fi
if [[ "$kms_status" -ne 0 ]]; then
printf 'The Vault KV2 backend does not wrap key material through Vault Transit; fix the wording in crates/kms.\n' >&2
fi
if [[ "$fips_status" -ne 0 || "$kms_status" -ne 0 ]]; then
exit 1
fi
printf 'FIPS wording guard passed (README.md and CHANGELOG.md contain no forbidden claims).\n'
printf 'KMS wording guard passed (crates/kms claims no Vault KV2 Transit wrapping).\n'
+1 -1
View File
@@ -25,7 +25,7 @@ cd "$(dirname "$0")/.."
# Baselines verified on 2026-08-11. Lower-only; see header.
# Excludes crates/e2e_test/ — test infrastructure legitimately uses s3s
# to verify S3 behavior and does not widen the production s3s surface.
S3S_IMPORT_FILES_BASELINE=213
S3S_IMPORT_FILES_BASELINE=211
S3_ERROR_LINES_BASELINE=1620
S3S_PATH_PATTERN='(^|[^"[:alnum:]_])s3s::'
E2E_TEST_GLOB='--glob=!crates/e2e_test/**'