mirror of
https://github.com/rustfs/rustfs.git
synced 2026-11-17 21:44:21 +00:00
Compare commits
9 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 0517566fad | |||
| 73895b37e8 | |||
| 00243c7e71 | |||
| 36bc1057ac | |||
| 32f7f70a29 | |||
| 5d32827af5 | |||
| 04ef836aa3 | |||
| 03850c1cd6 | |||
| 654be96186 |
@@ -39,11 +39,7 @@ script-tests: ## Run shell script tests
|
||||
./scripts/test_entrypoint_directories.sh
|
||||
./scripts/test_internode_grpc_ab_bench.sh
|
||||
$(RUSTFS_PYTHON_BIN) ./scripts/test_internode_grpc_window.py
|
||||
$(RUSTFS_PYTHON_BIN) ./scripts/test_internode_rpc_timeout_boundary.py
|
||||
$(RUSTFS_PYTHON_BIN) ./scripts/test_internode_msgpack_peer_canary.py
|
||||
./scripts/test_internode_msgpack_peer_canary_runner.sh
|
||||
./scripts/test_object_batch_bench_enhanced.sh
|
||||
$(RUSTFS_PYTHON_BIN) ./scripts/test_inline_docker_lab.py
|
||||
./scripts/test_package_service_scripts.sh
|
||||
./scripts/test_hotpath_warp_ab_gate.sh
|
||||
./scripts/test_hotpath_warp_abba.sh
|
||||
|
||||
@@ -22,7 +22,7 @@ rustfs-connect-symbol-catalog 6 0
|
||||
rustfs-credentials 24 0
|
||||
rustfs-crypto 67 0
|
||||
rustfs-data-usage 58 0
|
||||
rustfs-ecstore 7858 3
|
||||
rustfs-ecstore 7858 2
|
||||
rustfs-extension-schema 23 0
|
||||
rustfs-filemeta 361 0
|
||||
rustfs-gateway-adapter 74 0
|
||||
|
||||
@@ -1,58 +0,0 @@
|
||||
# syntax=docker/dockerfile:1.6
|
||||
# Copyright 2024 RustFS Team
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
# Server-only Linux validation image. The build context supplies a Git bundle.
|
||||
FROM rust:1.100.0-trixie AS build-tools
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||
ca-certificates clang lld libssl-dev pkg-config protobuf-compiler python3 python3-boto3 \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
ENV RUSTUP_TOOLCHAIN=1.100.0 \
|
||||
CARGO_BUILD_JOBS=4 \
|
||||
CARGO_NET_GIT_FETCH_WITH_CLI=true \
|
||||
CARGO_TARGET_DIR=/work/rustfs/target \
|
||||
RUSTFLAGS="-C link-arg=-fuse-ld=lld"
|
||||
|
||||
FROM build-tools AS build
|
||||
ARG SOURCE_REVISION
|
||||
COPY repository.bundle /tmp/repository.bundle
|
||||
COPY inspect_geometry.rs /tmp/inspect_geometry.rs
|
||||
RUN git clone /tmp/repository.bundle /work/rustfs \
|
||||
&& git -C /work/rustfs checkout --detach "$SOURCE_REVISION" \
|
||||
&& test "$(git -C /work/rustfs rev-parse HEAD)" = "$SOURCE_REVISION"
|
||||
WORKDIR /work/rustfs
|
||||
RUN --mount=type=cache,id=rustfs-inline-registry,target=/usr/local/cargo/registry \
|
||||
--mount=type=cache,id=rustfs-inline-git,target=/usr/local/cargo/git \
|
||||
--mount=type=cache,id=rustfs-inline-linux-target,target=/work/rustfs/target \
|
||||
cargo build --locked --release --bin rustfs \
|
||||
&& install -Dm755 target/release/rustfs /out/rustfs
|
||||
|
||||
# Build the diagnostic tool after preserving the clean-source server binary.
|
||||
RUN cp /tmp/inspect_geometry.rs crates/filemeta/examples/inline_lab_inspect.rs
|
||||
RUN --mount=type=cache,id=rustfs-inline-registry,target=/usr/local/cargo/registry \
|
||||
--mount=type=cache,id=rustfs-inline-git,target=/usr/local/cargo/git \
|
||||
--mount=type=cache,id=rustfs-inline-linux-target,target=/work/rustfs/target \
|
||||
cargo build --locked --release -p rustfs-filemeta --example inline_lab_inspect \
|
||||
&& install -Dm755 target/release/examples/inline_lab_inspect /out/inline_lab_inspect
|
||||
|
||||
FROM debian:trixie-slim AS runtime
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||
ca-certificates curl libssl3t64 \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
ARG SOURCE_REVISION
|
||||
LABEL org.opencontainers.image.revision="$SOURCE_REVISION" \
|
||||
rustfs.validation="inline"
|
||||
COPY --from=build /out/rustfs /usr/local/bin/rustfs
|
||||
COPY --from=build /out/inline_lab_inspect /usr/local/bin/inline_lab_inspect
|
||||
ENTRYPOINT ["/usr/local/bin/rustfs"]
|
||||
@@ -1,5 +0,0 @@
|
||||
FROM debian:trixie-slim
|
||||
COPY mc /usr/bin/mc
|
||||
RUN chmod 0755 /usr/bin/mc && /usr/bin/mc --version
|
||||
LABEL rustfs.validation="inline"
|
||||
ENTRYPOINT ["/usr/bin/mc"]
|
||||
@@ -1,133 +0,0 @@
|
||||
# Inline geometry Linux lab
|
||||
|
||||
**Use this when** validating inline placement, block geometry, Range reads and reader rollback in a task-owned local Docker cluster.
|
||||
|
||||
The adapter reuses the repository's four-node compose topology with 16 logical Docker volumes and EC12+4. It binds one S3 port to loopback, keeps strict durability and checks running binary hashes and persisted object geometry. Docker volumes share the host storage substrate. Performance observations describe this local virtualized lab.
|
||||
|
||||
## Build a frozen server image
|
||||
|
||||
Use the current stable toolchain pinned in the lab Dockerfile. Supply a Git bundle and the metadata inspector source as the build context. The server is compiled from a clean checkout before the diagnostic example is added.
|
||||
|
||||
```bash
|
||||
set -euo pipefail
|
||||
LAB_BUILD_DIR="$(mktemp -d)"
|
||||
LAB_REVISION="$(git rev-parse HEAD)"
|
||||
git bundle create "$LAB_BUILD_DIR/repository.bundle" HEAD
|
||||
cp crates/filemeta/examples/inline_lab_inspect.rs "$LAB_BUILD_DIR/inspect_geometry.rs"
|
||||
docker --context "$LAB_CONTEXT" build --target build-tools \
|
||||
-t rustfs-inline-build-tools:1.100 \
|
||||
-f .docker/inline-lab/Dockerfile .docker/inline-lab
|
||||
docker --context "$LAB_CONTEXT" build \
|
||||
--build-arg SOURCE_REVISION="$LAB_REVISION" \
|
||||
-t rustfs-inline-validation:local \
|
||||
-f .docker/inline-lab/Dockerfile "$LAB_BUILD_DIR"
|
||||
LAB_EXPORT_ID="$(docker --context "$LAB_CONTEXT" create \
|
||||
--label rustfs.validation=inline rustfs-inline-validation:local)"
|
||||
docker --context "$LAB_CONTEXT" cp \
|
||||
"$LAB_EXPORT_ID:/usr/local/bin/rustfs" "$LAB_BUILD_DIR/rustfs-linux"
|
||||
docker --context "$LAB_CONTEXT" rm "$LAB_EXPORT_ID"
|
||||
```
|
||||
|
||||
## Configure and probe
|
||||
|
||||
Select a new compose project and a private state directory. The adapter generates test credentials there. Existing resources require matching ownership labels. Configure checks the image's source revision and pins immutable image IDs. Each operation takes an exclusive state lock. Deployment waits for `/health/ready` on every node, covering storage, IAM, lock quorum and peer dependencies.
|
||||
|
||||
```bash
|
||||
scripts/python_bin.sh scripts/inline_docker_lab.py --state-dir "$LAB_STATE" configure \
|
||||
--context "$LAB_CONTEXT" --project "$LAB_PROJECT" \
|
||||
--image rustfs-inline-validation:local --revision "$LAB_REVISION" \
|
||||
--binary "$LAB_BUILD_DIR/rustfs-linux" --port 19080
|
||||
scripts/python_bin.sh scripts/inline_docker_lab.py --state-dir "$LAB_STATE" deploy \
|
||||
--bucket inline-geometry-probe --shard-bytes 0
|
||||
scripts/python_bin.sh scripts/inline_docker_lab.py --state-dir "$LAB_STATE" probe \
|
||||
--bucket inline-geometry-probe
|
||||
```
|
||||
|
||||
The probe verifies complete bytes, HEAD sizes, Range contents, EC12+4 and the actual persisted block. It records inline markers for 4 KiB, 1 MiB, 1 MiB + 1 byte, 1.25 MiB, 1,572,856 bytes, 1,572,857 bytes and 1.5 MiB objects in `probe.json`. The server-only image includes the existing filemeta decoder through a small geometry inspector. Console assets and the administration CLI are outside this lab image.
|
||||
|
||||
## ABBA deployment hook
|
||||
|
||||
Use `inline_docker_lab.py ... deploy` in the existing `run_hotpath_warp_abba.sh` external deploy hook. Export `RUSTFS_ERASURE_SHARD_BYTES` according to `HOTPATH_ABBA_PHASE`. The adapter uses the supplied leg, phase, bucket, binary and evidence-file context, resets only owned lab volumes, checks running configuration and binary hashes, creates the fresh bucket and writes the runner's deployment evidence.
|
||||
|
||||
Use the same frozen binary for both sides of a geometry experiment. Fix the inline budget and encoding mode, then compare zero with one candidate shard value. Include the CPU, memory and image identities from every node in each result. The state `config.json` supplies the generated test credentials for the load driver. Set `DOCKER_CONTEXT` to the selected context when the driver captures Docker resource snapshots.
|
||||
|
||||
For content compatibility across an image or geometry change, configure the next selected image and artifact using the same state, then deploy with `--keep-data` and an existing bucket. This retains owned volumes. Compare existing objects and version IDs with the manifest prepared before the change.
|
||||
|
||||
## Cleanup
|
||||
|
||||
```bash
|
||||
scripts/python_bin.sh scripts/inline_docker_lab.py --state-dir "$LAB_STATE" down --purge
|
||||
```
|
||||
|
||||
Cleanup checks ownership for containers, volumes and networks. A configuration cannot adopt another project's resources. Raw measurements, state, Git bundles and build outputs stay in task artifact directories.
|
||||
|
||||
Unit checks run with `scripts/python_bin.sh scripts/test_inline_docker_lab.py` and are included in `make script-tests`.
|
||||
|
||||
## Linux content-probe client
|
||||
|
||||
The PUT after-probe requires a working `mc`. For a macOS client affected by cgo initialization failures, compile the same frozen public mc module for Linux with CGO disabled, build it with `Dockerfile.mc`, and use `scripts/inline_docker_mc.sh` as the process-local `mc` command. Record the module version, checksum and resulting image ID with the run.
|
||||
|
||||
The adapter rewrites only the lab loopback alias endpoint to the private `node1` endpoint. Alias configuration stays in the probe's own `MC_CONFIG_DIR`, and `cat` streams raw bytes. The existing HEAD, GET and hash checks continue to execute.
|
||||
|
||||
```bash
|
||||
export RUSTFS_INLINE_LAB_STATE="$LAB_STATE"
|
||||
export RUSTFS_INLINE_MC_IMAGE="$LAB_MC_IMAGE"
|
||||
mkdir -p "$LAB_TOOLS"
|
||||
ln -s "$PWD/scripts/inline_docker_mc.sh" "$LAB_TOOLS/mc"
|
||||
export PATH="$LAB_TOOLS:$PATH"
|
||||
```
|
||||
|
||||
## Compare transport and placement independently
|
||||
|
||||
`deploy --rpc-mode json-compat` explicitly clears both MessagePack-only flags on every node. `deploy --rpc-mode msgpack-only` enables both flags for the selected homogeneous lab fleet. Deployment evidence records the mode and checks both running flags. The default is `json-compat`. The ABBA hook may select the same mode through `RUSTFS_INLINE_LAB_RPC_MODE`.
|
||||
|
||||
First compare the base and bounded-multi-block binaries at `--shard-bytes 0` in JSON compatibility mode. Then repeat with MessagePack-only on both sides. A same-binary transport ABBA isolates the encoding effect. Keep the 128 KiB budget, CPU sets, strict durability, load and object set fixed. Check actual markers and block sizes at the boundary sizes before timing. Retain baseline-drift and candidate gates, error logs and raw request distributions.
|
||||
|
||||
For a rolling downgrade, redeploy the current binary with `--keep-data --rpc-mode json-compat` before introducing an older node. Check old and new reads, version IDs and batch deletes against the persisted manifest. A running MessagePack-only flag pair establishes the lab configuration. Compatibility with a particular old reader requires the explicit reader test.
|
||||
|
||||
## Preserve a compatibility manifest
|
||||
|
||||
Run the seed once in a fresh lab bucket. It records full byte oracles and the observed version inventory, including any additional version matching a known body, and leaves a 5 MiB multipart part ready for the next reader. Each bucket has its own private `compat-<bucket>.json` manifest. Keep the same state and volumes across image changes. A fresh deployment or cleanup that resets volumes moves existing compatibility manifests into `compat-history`. Retained-data deployments preserve the active manifests.
|
||||
|
||||
```bash
|
||||
scripts/python_bin.sh scripts/inline_docker_lab.py --state-dir "$LAB_STATE" compat-seed --bucket inline-compat
|
||||
scripts/python_bin.sh scripts/inline_docker_lab.py --state-dir "$LAB_STATE" compat-check --bucket inline-compat
|
||||
# Configure the next frozen image and binary, then retain the data.
|
||||
scripts/python_bin.sh scripts/inline_docker_lab.py --state-dir "$LAB_STATE" deploy \
|
||||
--bucket inline-compat --shard-bytes 0 --rpc-mode json-compat --keep-data
|
||||
scripts/python_bin.sh scripts/inline_docker_lab.py --state-dir "$LAB_STATE" compat-check --bucket inline-compat
|
||||
scripts/python_bin.sh scripts/inline_docker_lab.py --state-dir "$LAB_STATE" compat-finish-mpu --bucket inline-compat
|
||||
```
|
||||
|
||||
The check compares every persisted version ID, HEAD length, complete SHA256, cross-block Range and per-version geometry from `xl.meta`. The seed includes zero-length objects, null versions, the 192 KiB versioned admission boundary and the accumulated 128 KiB shard boundary. Seed a second bucket on the new writer before the downgrade, then check both manifests using the selected old reader. Complete an old upload on the new binary and a new upload on the old binary. After returning to the current fleet, `compat-delete --bucket <bucket>` deletes the recorded versions in one S3 batch and checks the exact remaining inventory and bytes.
|
||||
|
||||
Successful multipart-completion and batch-delete receipts are saved before subsequent content and geometry checks. If either check fails, rerun the same command with the same state directory to resume verification from the saved completion state. After a seed check fails with a saved manifest, use `compat-check` to resume verification.
|
||||
|
||||
For mixed-version testing, retain JSON compatibility mode throughout the rolling replacement. Verify each node's immutable image and binary, then perform manifest checks and new writes through both coordinator versions using `--coordinator node1` and `--coordinator node4`. The manifest records the actual writer coordinator revision from the running image. Stop one owned node to check read availability at EC12+4, restore it and check the manifests again. Record the exact oldest tested source revision, toolchain and build profile with the evidence. The version boundary is established by these executed cells.
|
||||
|
||||
An owned lab can replace just its fourth node through a compose image override. Select the frozen old image first and verify its revision label. Keep `compose.json` in JSON compatibility mode for this cell.
|
||||
|
||||
```bash
|
||||
set -euo pipefail
|
||||
OLD_IMAGE_ID="$(docker --context "$LAB_CONTEXT" image inspect "$OLD_IMAGE" --format '{{.Id}}')"
|
||||
test "$(docker --context "$LAB_CONTEXT" image inspect "$OLD_IMAGE_ID" --format '{{index .Config.Labels "org.opencontainers.image.revision"}}')" = "$OLD_REVISION"
|
||||
INLINE_OLD_IMAGE_ID="$OLD_IMAGE_ID" scripts/python_bin.sh - <<'PYTHON' > "$LAB_STATE/mixed-image.json"
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
|
||||
override = {"services": {"node4": {"image": os.environ["INLINE_OLD_IMAGE_ID"]}}}
|
||||
json.dump(override, sys.stdout)
|
||||
PYTHON
|
||||
docker --context "$LAB_CONTEXT" compose --project-name "$LAB_PROJECT" \
|
||||
--file "$LAB_STATE/compose.json" --file "$LAB_STATE/mixed-image.json" \
|
||||
up --detach --no-deps --no-build --pull never node4
|
||||
scripts/python_bin.sh scripts/inline_docker_lab.py --state-dir "$LAB_STATE" compat-check \
|
||||
--bucket inline-compat --coordinator node4
|
||||
scripts/python_bin.sh scripts/inline_docker_lab.py --state-dir "$LAB_STATE" compat-seed \
|
||||
--bucket inline-mixed-writer --coordinator node4
|
||||
scripts/python_bin.sh scripts/inline_docker_lab.py --state-dir "$LAB_STATE" compat-check \
|
||||
--bucket inline-mixed-writer --coordinator node1
|
||||
```
|
||||
|
||||
Inspect all four running image IDs and binary hashes before recording the mixed-fleet result. Redeploy the configured current image with `--keep-data --rpc-mode json-compat` to restore a homogeneous fleet. The same manifest checks apply after stopping and restarting one owned node through the base compose file.
|
||||
Generated
+25
-32
@@ -4133,7 +4133,7 @@ dependencies = [
|
||||
"libc",
|
||||
"option-ext",
|
||||
"redox_users",
|
||||
"windows-sys 0.60.2",
|
||||
"windows-sys 0.59.0",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -4641,7 +4641,7 @@ checksum = "94e7099f6313ecacbe1256e8ff9d617b75d1bcb16a6fddef94866d225a01a14a"
|
||||
dependencies = [
|
||||
"io-lifetimes 2.0.4",
|
||||
"rustix",
|
||||
"windows-sys 0.52.0",
|
||||
"windows-sys 0.59.0",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -5844,7 +5844,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "20fd6de4ccfcc187e38bc21cfa543cb5a302cb86a8b114eb7f0bf0dc9f8ac00f"
|
||||
dependencies = [
|
||||
"io-lifetimes 3.0.1",
|
||||
"windows-sys 0.52.0",
|
||||
"windows-sys 0.60.2",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -7073,7 +7073,7 @@ version = "0.50.3"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "7957b9740744892f114936ab4a57b3f487491bbeafaf8083688b16841a4240e5"
|
||||
dependencies = [
|
||||
"windows-sys 0.60.2",
|
||||
"windows-sys 0.59.0",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -9865,8 +9865,6 @@ dependencies = [
|
||||
"rustfs-filemeta",
|
||||
"rustfs-gateway",
|
||||
"rustfs-gateway-adapter",
|
||||
"rustfs-gateway-core",
|
||||
"rustfs-gateway-dialect-rustfs-admin",
|
||||
"rustfs-gateway-stream",
|
||||
"rustfs-gateway-types",
|
||||
"rustfs-heal",
|
||||
@@ -10268,8 +10266,8 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "rustfs-gateway"
|
||||
version = "0.66.0"
|
||||
source = "git+https://github.com/rustfs/gateway.git?rev=0935429d61b90c65883f0a277e0b50de4294b3d8#0935429d61b90c65883f0a277e0b50de4294b3d8"
|
||||
version = "0.63.0"
|
||||
source = "git+https://github.com/rustfs/gateway.git?rev=2a6b8d1f7663db978d8784ab3ea331cade19237b#2a6b8d1f7663db978d8784ab3ea331cade19237b"
|
||||
dependencies = [
|
||||
"arc-swap",
|
||||
"bytes",
|
||||
@@ -10315,8 +10313,8 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "rustfs-gateway-core"
|
||||
version = "0.63.0"
|
||||
source = "git+https://github.com/rustfs/gateway.git?rev=0935429d61b90c65883f0a277e0b50de4294b3d8#0935429d61b90c65883f0a277e0b50de4294b3d8"
|
||||
version = "0.60.0"
|
||||
source = "git+https://github.com/rustfs/gateway.git?rev=2a6b8d1f7663db978d8784ab3ea331cade19237b#2a6b8d1f7663db978d8784ab3ea331cade19237b"
|
||||
dependencies = [
|
||||
"bytes",
|
||||
"futures-core",
|
||||
@@ -10334,24 +10332,10 @@ dependencies = [
|
||||
"zeroize",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "rustfs-gateway-dialect-rustfs-admin"
|
||||
version = "0.13.0"
|
||||
source = "git+https://github.com/rustfs/gateway.git?rev=0935429d61b90c65883f0a277e0b50de4294b3d8#0935429d61b90c65883f0a277e0b50de4294b3d8"
|
||||
dependencies = [
|
||||
"bytes",
|
||||
"form_urlencoded",
|
||||
"http 1.5.0",
|
||||
"rustfs-gateway-core",
|
||||
"rustfs-gateway-sig",
|
||||
"rustfs-gateway-stream",
|
||||
"rustfs-gateway-types",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "rustfs-gateway-http"
|
||||
version = "0.1.4"
|
||||
source = "git+https://github.com/rustfs/gateway.git?rev=0935429d61b90c65883f0a277e0b50de4294b3d8#0935429d61b90c65883f0a277e0b50de4294b3d8"
|
||||
source = "git+https://github.com/rustfs/gateway.git?rev=2a6b8d1f7663db978d8784ab3ea331cade19237b#2a6b8d1f7663db978d8784ab3ea331cade19237b"
|
||||
dependencies = [
|
||||
"hmac 0.13.0",
|
||||
"http 1.5.0",
|
||||
@@ -10369,7 +10353,7 @@ dependencies = [
|
||||
[[package]]
|
||||
name = "rustfs-gateway-macros"
|
||||
version = "0.1.4"
|
||||
source = "git+https://github.com/rustfs/gateway.git?rev=0935429d61b90c65883f0a277e0b50de4294b3d8#0935429d61b90c65883f0a277e0b50de4294b3d8"
|
||||
source = "git+https://github.com/rustfs/gateway.git?rev=2a6b8d1f7663db978d8784ab3ea331cade19237b#2a6b8d1f7663db978d8784ab3ea331cade19237b"
|
||||
dependencies = [
|
||||
"proc-macro2",
|
||||
"quote",
|
||||
@@ -10379,7 +10363,7 @@ dependencies = [
|
||||
[[package]]
|
||||
name = "rustfs-gateway-sig"
|
||||
version = "0.15.0"
|
||||
source = "git+https://github.com/rustfs/gateway.git?rev=0935429d61b90c65883f0a277e0b50de4294b3d8#0935429d61b90c65883f0a277e0b50de4294b3d8"
|
||||
source = "git+https://github.com/rustfs/gateway.git?rev=2a6b8d1f7663db978d8784ab3ea331cade19237b#2a6b8d1f7663db978d8784ab3ea331cade19237b"
|
||||
dependencies = [
|
||||
"hmac 0.13.0",
|
||||
"http 1.5.0",
|
||||
@@ -10394,7 +10378,7 @@ dependencies = [
|
||||
[[package]]
|
||||
name = "rustfs-gateway-stream"
|
||||
version = "0.2.0"
|
||||
source = "git+https://github.com/rustfs/gateway.git?rev=0935429d61b90c65883f0a277e0b50de4294b3d8#0935429d61b90c65883f0a277e0b50de4294b3d8"
|
||||
source = "git+https://github.com/rustfs/gateway.git?rev=2a6b8d1f7663db978d8784ab3ea331cade19237b#2a6b8d1f7663db978d8784ab3ea331cade19237b"
|
||||
dependencies = [
|
||||
"bitflags 2.13.2",
|
||||
"bytes",
|
||||
@@ -10404,8 +10388,8 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "rustfs-gateway-types"
|
||||
version = "0.42.0+aws.2026-10-02"
|
||||
source = "git+https://github.com/rustfs/gateway.git?rev=0935429d61b90c65883f0a277e0b50de4294b3d8#0935429d61b90c65883f0a277e0b50de4294b3d8"
|
||||
version = "0.40.0+aws.2026-10-02"
|
||||
source = "git+https://github.com/rustfs/gateway.git?rev=2a6b8d1f7663db978d8784ab3ea331cade19237b#2a6b8d1f7663db978d8784ab3ea331cade19237b"
|
||||
dependencies = [
|
||||
"bytes",
|
||||
"crc-fast",
|
||||
@@ -10428,7 +10412,7 @@ dependencies = [
|
||||
[[package]]
|
||||
name = "rustfs-gateway-xml"
|
||||
version = "0.4.0"
|
||||
source = "git+https://github.com/rustfs/gateway.git?rev=0935429d61b90c65883f0a277e0b50de4294b3d8#0935429d61b90c65883f0a277e0b50de4294b3d8"
|
||||
source = "git+https://github.com/rustfs/gateway.git?rev=2a6b8d1f7663db978d8784ab3ea331cade19237b#2a6b8d1f7663db978d8784ab3ea331cade19237b"
|
||||
dependencies = [
|
||||
"quick-xml 0.41.0",
|
||||
]
|
||||
@@ -14289,6 +14273,15 @@ dependencies = [
|
||||
"windows-targets 0.52.6",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "windows-sys"
|
||||
version = "0.59.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "1e38bc4d79ed67fd075bcc251a1c39b32a1776bbe92e5bef1f0bf1f8c531853b"
|
||||
dependencies = [
|
||||
"windows-targets 0.52.6",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "windows-sys"
|
||||
version = "0.60.2"
|
||||
@@ -14461,7 +14454,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "3f3fd376f71958b862e7afb20cfe5a22830e1963462f3a17f49d82a6c1d1f42d"
|
||||
dependencies = [
|
||||
"bitflags 2.13.2",
|
||||
"windows-sys 0.52.0",
|
||||
"windows-sys 0.59.0",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
|
||||
+5
-7
@@ -151,16 +151,14 @@ rustfs-zip = { path = "./crates/zip", version = "1.0.5" }
|
||||
# engine crates move onto from s3s (rustfs/backlog#2745). The one pin of the
|
||||
# gateway in this workspace; no crate may enable a `compat-s3s*` feature except
|
||||
# the s3s-facing edge of the rustfs binary.
|
||||
rustfs-gateway-types = { git = "https://github.com/rustfs/gateway.git", rev = "0935429d61b90c65883f0a277e0b50de4294b3d8", version = "0.42.0", default-features = false }
|
||||
rustfs-gateway-stream = { git = "https://github.com/rustfs/gateway.git", rev = "0935429d61b90c65883f0a277e0b50de4294b3d8", version = "0.2.0", default-features = false }
|
||||
rustfs-gateway-types = { git = "https://github.com/rustfs/gateway.git", rev = "2a6b8d1f7663db978d8784ab3ea331cade19237b", version = "0.40.0", default-features = false }
|
||||
rustfs-gateway-stream = { git = "https://github.com/rustfs/gateway.git", rev = "2a6b8d1f7663db978d8784ab3ea331cade19237b", version = "0.2.0", default-features = false }
|
||||
# The gateway framework itself, consumed for its extension traits by the ring-2
|
||||
# adapter. `default-features = false` keeps the `server` feature, and with it the
|
||||
# server crate, out of the build.
|
||||
rustfs-gateway = { git = "https://github.com/rustfs/gateway.git", rev = "0935429d61b90c65883f0a277e0b50de4294b3d8", version = "0.66.0", default-features = false }
|
||||
rustfs-gateway = { git = "https://github.com/rustfs/gateway.git", rev = "2a6b8d1f7663db978d8784ab3ea331cade19237b", version = "0.63.0", default-features = false }
|
||||
# Test-only: builds the `EffectiveHost` a host-resolution test queries with.
|
||||
rustfs-gateway-http = { git = "https://github.com/rustfs/gateway.git", rev = "0935429d61b90c65883f0a277e0b50de4294b3d8", default-features = false }
|
||||
rustfs-gateway-core = { git = "https://github.com/rustfs/gateway.git", rev = "0935429d61b90c65883f0a277e0b50de4294b3d8", version = "0.63.0", default-features = false }
|
||||
rustfs-gateway-dialect-rustfs-admin = { git = "https://github.com/rustfs/gateway.git", rev = "0935429d61b90c65883f0a277e0b50de4294b3d8", version = "0.13.0", default-features = false }
|
||||
rustfs-gateway-http = { git = "https://github.com/rustfs/gateway.git", rev = "2a6b8d1f7663db978d8784ab3ea331cade19237b", default-features = false }
|
||||
|
||||
# Async Runtime and Networking
|
||||
async-channel = "2.5.0"
|
||||
@@ -311,7 +309,7 @@ hostname = "0.4.2"
|
||||
ipnetwork = { version = "0.21.1" }
|
||||
lazy_static = "1.5.1"
|
||||
libc = "0.2.191"
|
||||
cuobject = { version = "0.1.0", default-features = false }
|
||||
cuobject = "0.1.0"
|
||||
libsystemd = "0.7.2"
|
||||
local-ip-address = "0.6.13"
|
||||
log = "0.4"
|
||||
|
||||
@@ -121,17 +121,9 @@ pub const DEFAULT_INTERNODE_RPC_MSGPACK_ONLY: bool = false;
|
||||
pub const ENV_INTERNODE_RPC_MSGPACK_ONLY_FLEET_CONFIRMED: &str = "RUSTFS_INTERNODE_RPC_MSGPACK_ONLY_FLEET_CONFIRMED";
|
||||
pub const DEFAULT_INTERNODE_RPC_MSGPACK_ONLY_FLEET_CONFIRMED: bool = false;
|
||||
|
||||
/// Permit current per-peer capability proof to replace fleet-wide confirmation.
|
||||
///
|
||||
/// This remains ineffective unless [`ENV_INTERNODE_RPC_MSGPACK_ONLY`] is also enabled. Unknown,
|
||||
/// unsupported, revoked, or stale connection generations keep dual-writing JSON.
|
||||
pub const ENV_INTERNODE_RPC_MSGPACK_PEER_CAPABILITY: &str = "RUSTFS_INTERNODE_RPC_MSGPACK_PEER_CAPABILITY";
|
||||
pub const DEFAULT_INTERNODE_RPC_MSGPACK_PEER_CAPABILITY: bool = false;
|
||||
|
||||
// Compile-time invariants: dual-write by default so the base build is byte-for-byte legacy behavior.
|
||||
const _: () = assert!(!DEFAULT_INTERNODE_RPC_MSGPACK_ONLY);
|
||||
const _: () = assert!(!DEFAULT_INTERNODE_RPC_MSGPACK_ONLY_FLEET_CONFIRMED);
|
||||
const _: () = assert!(!DEFAULT_INTERNODE_RPC_MSGPACK_PEER_CAPABILITY);
|
||||
|
||||
/// Require target-bound v2 signatures on every internode gRPC request, rejecting the legacy
|
||||
/// constant-target fallback instead of accepting it (<https://github.com/rustfs/backlog/issues/1327>).
|
||||
@@ -533,7 +525,6 @@ mod tests {
|
||||
ENV_INTERNODE_RPC_MSGPACK_ONLY_FLEET_CONFIRMED,
|
||||
"RUSTFS_INTERNODE_RPC_MSGPACK_ONLY_FLEET_CONFIRMED"
|
||||
);
|
||||
assert_eq!(ENV_INTERNODE_RPC_MSGPACK_PEER_CAPABILITY, "RUSTFS_INTERNODE_RPC_MSGPACK_PEER_CAPABILITY");
|
||||
}
|
||||
|
||||
#[test]
|
||||
|
||||
@@ -48,7 +48,7 @@ pub const ENV_RUNTIME_DIAL9_S3_PREFIX: &str = "RUSTFS_RUNTIME_DIAL9_S3_PREFIX";
|
||||
pub const DEFAULT_WORKER_THREADS: usize = 16;
|
||||
pub const DEFAULT_MAX_BLOCKING_THREADS: usize = 1024;
|
||||
pub const DEFAULT_THREAD_PRINT_ENABLED: bool = false;
|
||||
pub const DEFAULT_THREAD_STACK_SIZE: usize = 2 * MI_B; // 2 MiB
|
||||
pub const DEFAULT_THREAD_STACK_SIZE: usize = MI_B; // 1 MiB
|
||||
pub const DEFAULT_THREAD_KEEP_ALIVE: u64 = 60; // seconds
|
||||
pub const DEFAULT_GLOBAL_QUEUE_INTERVAL: u32 = 31;
|
||||
pub const DEFAULT_THREAD_NAME: &str = "rustfs-worker";
|
||||
|
||||
@@ -171,6 +171,70 @@ impl DataUsageBucketObservation {
|
||||
}
|
||||
|
||||
impl DataUsageInfo {
|
||||
/// Hide values that cannot be bound to the current storage-owned identity.
|
||||
///
|
||||
/// `incarnations` contains canonical non-nil identities read from storage;
|
||||
/// a missing entry means identity was unavailable. Neither legacy values
|
||||
/// without identity nor metadata read failures establish confirmed zero.
|
||||
/// Changed responses are display-only and cannot authorize quota or writes.
|
||||
/// If retained counters overflow, all values remain unknown.
|
||||
/// Returns the number of canonical values removed.
|
||||
pub fn retain_current_bucket_usage_for_admin(&mut self, incarnations: &HashMap<String, String>) -> usize {
|
||||
let original_count = self.buckets_usage.len();
|
||||
let original_observation_count = self.usage_snapshot_bucket_observations.len();
|
||||
let matches_current = |bucket: &String, usage: &BucketUsageInfo| {
|
||||
incarnations.get(bucket).is_some_and(|current| {
|
||||
!current.is_empty()
|
||||
&& current != "00000000-0000-0000-0000-000000000000"
|
||||
&& usage.bucket_incarnation.as_ref() == Some(current)
|
||||
})
|
||||
};
|
||||
self.buckets_usage.retain(|bucket, usage| matches_current(bucket, usage));
|
||||
self.usage_snapshot_bucket_observations.retain(|bucket, observation| {
|
||||
matches_current(bucket, &observation.usage)
|
||||
&& observation
|
||||
.sources
|
||||
.iter()
|
||||
.all(|source| incarnations.get(bucket) == Some(&source.bucket_incarnation))
|
||||
});
|
||||
if original_count == self.buckets_usage.len()
|
||||
&& original_observation_count == self.usage_snapshot_bucket_observations.len()
|
||||
{
|
||||
return 0;
|
||||
}
|
||||
let totals = self.buckets_usage.values().try_fold([0_u64; 4], |total, usage| {
|
||||
Some([
|
||||
total[0].checked_add(usage.size)?,
|
||||
total[1].checked_add(usage.objects_count)?,
|
||||
total[2].checked_add(usage.versions_count)?,
|
||||
total[3].checked_add(usage.delete_markers_count)?,
|
||||
])
|
||||
});
|
||||
let [size, objects, versions, markers] = totals.unwrap_or_else(|| {
|
||||
self.buckets_usage.clear();
|
||||
self.usage_snapshot_bucket_observations.clear();
|
||||
[0; 4]
|
||||
});
|
||||
self.bucket_sizes = self
|
||||
.buckets_usage
|
||||
.iter()
|
||||
.map(|(bucket, usage)| (bucket.clone(), usage.size))
|
||||
.collect();
|
||||
self.buckets_count = u64::try_from(self.buckets_usage.len()).unwrap_or(u64::MAX);
|
||||
self.objects_total_size = size;
|
||||
self.objects_total_count = objects;
|
||||
self.versions_total_count = versions;
|
||||
self.delete_markers_total_count = markers;
|
||||
self.replication_info.clear();
|
||||
self.tier_stats = None;
|
||||
self.unknown_tier_stats = None;
|
||||
self.usage_snapshot_complete = false;
|
||||
self.usage_snapshot_partial = true;
|
||||
self.usage_snapshot_converged = Some(false);
|
||||
self.usage_snapshot_admin_projection = true;
|
||||
original_count.saturating_sub(self.buckets_usage.len())
|
||||
}
|
||||
|
||||
/// Overlay validated companions on an owned admin response only.
|
||||
///
|
||||
/// `incarnations` must come from current storage metadata, not the companion.
|
||||
|
||||
@@ -321,3 +321,110 @@ fn provenance_and_companions_change_candidate_bytes_and_round_trip_as_maps() {
|
||||
let restored: DataUsageInfo = rmp_serde::from_slice(&named).expect("read named map with additive fields");
|
||||
assert_eq!(restored, canonical);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn current_identity_filter_preserves_an_unchanged_complete_snapshot() {
|
||||
let mut candidate = snapshot();
|
||||
let original = candidate.clone();
|
||||
assert_eq!(candidate.retain_current_bucket_usage_for_admin(&incarnations()), 0);
|
||||
assert_eq!(candidate, original);
|
||||
assert!(candidate.is_complete_bucket_usage_snapshot());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn current_identity_filter_hides_unbound_values_without_confirming_zero() {
|
||||
for identity in [
|
||||
None,
|
||||
Some(""),
|
||||
Some("corrupt"),
|
||||
Some("00000000-0000-0000-0000-000000000000"),
|
||||
Some("22222222-2222-4222-8222-222222222222"),
|
||||
] {
|
||||
let mut candidate = snapshot();
|
||||
candidate.usage_snapshot_bucket_observations.clear();
|
||||
candidate
|
||||
.buckets_usage
|
||||
.get_mut("bucket")
|
||||
.expect("canonical")
|
||||
.bucket_incarnation = identity.map(str::to_string);
|
||||
assert_eq!(
|
||||
candidate.retain_current_bucket_usage_for_admin(&incarnations()),
|
||||
1,
|
||||
"identity {identity:?}"
|
||||
);
|
||||
assert!(!candidate.buckets_usage.contains_key("bucket"));
|
||||
assert!(!candidate.bucket_sizes.contains_key("bucket"));
|
||||
assert_eq!(candidate.usage_snapshot_converged, Some(false));
|
||||
assert!(candidate.usage_snapshot_admin_projection);
|
||||
assert!(!candidate.is_complete_bucket_usage_snapshot());
|
||||
assert!(!candidate.is_valid_partial_snapshot());
|
||||
}
|
||||
for current in ["", "00000000-0000-0000-0000-000000000000"] {
|
||||
let mut candidate = snapshot();
|
||||
candidate
|
||||
.buckets_usage
|
||||
.get_mut("bucket")
|
||||
.expect("canonical")
|
||||
.bucket_incarnation = Some(current.to_string());
|
||||
assert_eq!(
|
||||
candidate.retain_current_bucket_usage_for_admin(&HashMap::from([("bucket".to_string(), current.to_string())])),
|
||||
1
|
||||
);
|
||||
assert!(candidate.usage_snapshot_admin_projection);
|
||||
}
|
||||
let mut candidate = snapshot();
|
||||
assert_eq!(candidate.retain_current_bucket_usage_for_admin(&HashMap::new()), 1);
|
||||
assert!(candidate.usage_snapshot_bucket_observations.is_empty());
|
||||
assert_eq!(candidate.usage_snapshot_converged, Some(false));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn current_identity_filter_keeps_healthy_buckets_and_recalculates_partial_totals() {
|
||||
let mut candidate = snapshot();
|
||||
candidate.buckets_usage.insert("cold".to_string(), usage(7));
|
||||
candidate.buckets_count = 2;
|
||||
candidate.bucket_sizes.insert("cold".to_string(), 7);
|
||||
candidate.calculate_totals();
|
||||
candidate.replication_info.insert("target".to_string(), Default::default());
|
||||
candidate.tier_stats = Some(Default::default());
|
||||
candidate.unknown_tier_stats = Some(Default::default());
|
||||
assert_eq!(
|
||||
candidate.retain_current_bucket_usage_for_admin(&HashMap::from([("cold".to_string(), INCARNATION.to_string())])),
|
||||
1
|
||||
);
|
||||
assert_eq!(candidate.buckets_usage.len(), 1);
|
||||
assert_eq!(candidate.buckets_usage["cold"].size, 7);
|
||||
assert_eq!(candidate.bucket_sizes, HashMap::from([("cold".to_string(), 7)]));
|
||||
assert_eq!(
|
||||
(
|
||||
candidate.buckets_count,
|
||||
candidate.objects_total_size,
|
||||
candidate.objects_total_count,
|
||||
candidate.versions_total_count
|
||||
),
|
||||
(1, 7, 7, 7)
|
||||
);
|
||||
assert!(candidate.replication_info.is_empty());
|
||||
assert!(candidate.tier_stats.is_none());
|
||||
assert!(candidate.unknown_tier_stats.is_none());
|
||||
assert!(candidate.usage_snapshot_admin_projection);
|
||||
assert_eq!(candidate.usage_snapshot_converged, Some(false));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn current_identity_filter_overflow_leaves_totals_unknown() {
|
||||
let mut candidate = snapshot();
|
||||
candidate.buckets_usage.insert("large".to_string(), usage(u64::MAX));
|
||||
candidate.buckets_usage.insert("small".to_string(), usage(1));
|
||||
candidate.buckets_count = 3;
|
||||
let current = HashMap::from([
|
||||
("large".to_string(), INCARNATION.to_string()),
|
||||
("small".to_string(), INCARNATION.to_string()),
|
||||
]);
|
||||
assert_eq!(candidate.retain_current_bucket_usage_for_admin(¤t), 3);
|
||||
assert!(candidate.buckets_usage.is_empty());
|
||||
assert!(candidate.bucket_sizes.is_empty());
|
||||
assert_eq!(candidate.usage_snapshot_converged, Some(false));
|
||||
assert!(candidate.usage_snapshot_admin_projection);
|
||||
assert!(!candidate.is_complete_bucket_usage_snapshot());
|
||||
}
|
||||
|
||||
@@ -17,7 +17,7 @@
|
||||
|
||||
use async_trait::async_trait;
|
||||
use rustfs_lock::{
|
||||
CacheFenceStamp, LockClient, LockError, LockId, LockInfo, LockRequest, LockResponse, LockStats, LockStatus, LockType, Result,
|
||||
LockClient, LockError, LockId, LockInfo, LockRequest, LockResponse, LockStats, LockStatus, LockType, Result,
|
||||
types::{LockMetadata, LockPriority},
|
||||
};
|
||||
use rustfs_protos::proto_gen::node_service::{BatchGenerallyLockRequest, GenerallyLockRequest, PingRequest};
|
||||
@@ -64,7 +64,6 @@ impl GrpcLockClient {
|
||||
deadlock_detection: false,
|
||||
suppress_contention_logs: false,
|
||||
refresh_interval: None,
|
||||
cache_fence_stamp_requested: false,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -125,15 +124,10 @@ impl LockClient for GrpcLockClient {
|
||||
|
||||
// Check if the lock acquisition was successful
|
||||
if resp.success {
|
||||
let info = Self::build_lock_info(request, resp.lock_info);
|
||||
match resp
|
||||
.cache_fence_stamp
|
||||
.as_deref()
|
||||
.and_then(|encoded| CacheFenceStamp::decode(encoded).ok())
|
||||
{
|
||||
Some(stamp) => Ok(LockResponse::success_with_cache_fence_stamp(info, std::time::Duration::ZERO, stamp)),
|
||||
None => Ok(LockResponse::success(info, std::time::Duration::ZERO)),
|
||||
}
|
||||
Ok(LockResponse::success(
|
||||
Self::build_lock_info(request, resp.lock_info),
|
||||
std::time::Duration::ZERO,
|
||||
))
|
||||
} else {
|
||||
// Lock acquisition failed
|
||||
Ok(LockResponse::failure(
|
||||
@@ -166,15 +160,7 @@ impl LockClient for GrpcLockClient {
|
||||
.enumerate()
|
||||
.map(|(idx, request)| match resp.results.get(idx) {
|
||||
Some(result) if result.success => {
|
||||
let info = Self::build_lock_info(request, result.lock_info.clone());
|
||||
match result
|
||||
.cache_fence_stamp
|
||||
.as_deref()
|
||||
.and_then(|encoded| CacheFenceStamp::decode(encoded).ok())
|
||||
{
|
||||
Some(stamp) => LockResponse::success_with_cache_fence_stamp(info, std::time::Duration::ZERO, stamp),
|
||||
None => LockResponse::success(info, std::time::Duration::ZERO),
|
||||
}
|
||||
LockResponse::success(Self::build_lock_info(request, result.lock_info.clone()), std::time::Duration::ZERO)
|
||||
}
|
||||
Some(result) => LockResponse::failure(
|
||||
result
|
||||
|
||||
@@ -43,7 +43,6 @@ fn lock_result_from_response(response: rustfs_lock::LockResponse) -> GenerallyLo
|
||||
success: response.success,
|
||||
error_info: response.error,
|
||||
lock_info: response.lock_info.and_then(|info| serde_json::to_string(&info).ok()),
|
||||
cache_fence_stamp: response.cache_fence_stamp.map(|stamp| stamp.encode().to_vec().into()),
|
||||
..Default::default()
|
||||
}
|
||||
}
|
||||
@@ -181,7 +180,6 @@ impl NodeService for MinimalLockNodeService {
|
||||
success: result.success,
|
||||
error_info: result.error,
|
||||
lock_info: lock_info_json,
|
||||
cache_fence_stamp: result.cache_fence_stamp.map(|stamp| stamp.encode().to_vec().into()),
|
||||
..Default::default()
|
||||
}))
|
||||
}
|
||||
|
||||
@@ -393,8 +393,7 @@ pub mod cluster {
|
||||
|
||||
pub mod compression {
|
||||
pub use crate::io_support::compress::{
|
||||
DISK_COMPRESSION_EXCLUDED_CONTENT_TYPES, MIN_DISK_COMPRESSIBLE_SIZE, is_disk_compressible, is_disk_compression_enabled,
|
||||
is_multipart_disk_compression_enabled,
|
||||
MIN_DISK_COMPRESSIBLE_SIZE, is_disk_compressible, is_disk_compression_enabled, is_multipart_disk_compression_enabled,
|
||||
};
|
||||
}
|
||||
|
||||
@@ -454,8 +453,8 @@ pub mod data_usage {
|
||||
DATA_USAGE_CACHE_NAME, apply_bucket_usage_memory_overlay, compute_bucket_usage,
|
||||
init_compression_total_memory_from_backend, invalidate_admin_data_usage_snapshot_cache,
|
||||
invalidate_data_usage_snapshot_cache, live_bucket_usage_computations, load_admin_data_usage_from_backend_cached,
|
||||
load_bucket_usage_from_backend_cached, load_compression_total_from_memory, load_data_usage_from_backend,
|
||||
load_data_usage_from_backend_cached, quota_object_size, record_bucket_delete_marker_memory,
|
||||
load_admin_data_usage_with_memory_overlay, load_bucket_usage_from_backend_cached, load_compression_total_from_memory,
|
||||
load_data_usage_from_backend, load_data_usage_from_backend_cached, quota_object_size, record_bucket_delete_marker_memory,
|
||||
record_bucket_object_delete_memory, record_bucket_object_version_write_memory, record_bucket_object_write_memory,
|
||||
record_bucket_object_write_unknown_previous_memory, record_compression_total_memory,
|
||||
refresh_bucket_usage_from_object_layer, refresh_versioned_bucket_usage_from_object_layer,
|
||||
|
||||
@@ -55,7 +55,7 @@ use std::{
|
||||
task::{Context, Poll},
|
||||
time::Duration,
|
||||
};
|
||||
use tonic::{codegen::StdError, service::interceptor::InterceptedService};
|
||||
use tonic::service::interceptor::InterceptedService;
|
||||
use tower::Service;
|
||||
use tracing::debug;
|
||||
use uuid::Uuid;
|
||||
@@ -84,7 +84,7 @@ pub(crate) async fn scanner_control_time_out_client(
|
||||
let connection_state = channel.connection_state();
|
||||
let interceptor = interceptor.with_connection_state(connection_state.clone());
|
||||
let channel = ReplayScopeChannel::new(
|
||||
ConnectionGenerationChannel::new(BatchLockDeadlineChannel::new(channel), connection_state.clone()),
|
||||
BatchLockDeadlineChannel::new(channel),
|
||||
interceptor.replay_scope_audience(),
|
||||
connection_state,
|
||||
);
|
||||
@@ -104,7 +104,7 @@ pub async fn heal_control_time_out_client(
|
||||
let interceptor = interceptor.with_connection_state(connection_state.clone());
|
||||
let max_message_size = rustfs_protos::HEAL_CONTROL_RPC_MAX_MESSAGE_SIZE;
|
||||
let channel = ReplayScopeChannel::new(
|
||||
ConnectionGenerationChannel::new(BatchLockDeadlineChannel::new(channel), connection_state.clone()),
|
||||
BatchLockDeadlineChannel::new(channel),
|
||||
interceptor.replay_scope_audience(),
|
||||
connection_state,
|
||||
);
|
||||
@@ -123,7 +123,7 @@ pub async fn tier_mutation_control_time_out_client(
|
||||
let interceptor = interceptor.with_connection_state(connection_state.clone());
|
||||
let max_message_size = rustfs_protos::TIER_MUTATION_RPC_MAX_MESSAGE_SIZE;
|
||||
let channel = ReplayScopeChannel::new(
|
||||
ConnectionGenerationChannel::new(BatchLockDeadlineChannel::new(channel), connection_state.clone()),
|
||||
BatchLockDeadlineChannel::new(channel),
|
||||
interceptor.replay_scope_audience(),
|
||||
connection_state,
|
||||
);
|
||||
@@ -150,7 +150,7 @@ pub async fn node_service_time_out_client_for_class(
|
||||
|
||||
let max_message_size = rustfs_protos::internode_rpc_max_message_size();
|
||||
let channel = ReplayScopeChannel::new(
|
||||
ConnectionGenerationChannel::new(BatchLockDeadlineChannel::new(channel), connection_state.clone()),
|
||||
BatchLockDeadlineChannel::new(channel),
|
||||
interceptor.replay_scope_audience(),
|
||||
connection_state,
|
||||
);
|
||||
@@ -159,39 +159,6 @@ pub async fn node_service_time_out_client_for_class(
|
||||
.max_encoding_message_size(max_message_size))
|
||||
}
|
||||
|
||||
/// Build an eligible request client and capture its peer capability exactly once for this RPC.
|
||||
pub(crate) async fn node_service_time_out_client_for_class_with_msgpack_policy(
|
||||
addr: &str,
|
||||
interceptor: TonicInterceptor,
|
||||
class: ChannelClass,
|
||||
) -> Result<
|
||||
(
|
||||
NodeServiceClient<InterceptedService<AuthenticatedChannel, TonicInterceptor>>,
|
||||
MsgpackSendPolicy,
|
||||
),
|
||||
Box<dyn Error>,
|
||||
> {
|
||||
let interceptor = interceptor.with_rpc_audience(addr)?;
|
||||
let audience = interceptor.replay_scope_audience();
|
||||
let channel = get_internode_channel_for_class(addr, class).await?;
|
||||
let connection_state = channel.connection_state();
|
||||
let policy =
|
||||
prepare_msgpack_send_policy(audience.as_deref(), connection_state.as_ref().and_then(GrpcConnectionState::snapshot));
|
||||
let interceptor = interceptor.with_connection_state(connection_state.clone());
|
||||
let max_message_size = rustfs_protos::internode_rpc_max_message_size();
|
||||
let channel = ReplayScopeChannel::new(
|
||||
ConnectionGenerationChannel::new(BatchLockDeadlineChannel::new(channel), connection_state.clone()),
|
||||
audience,
|
||||
connection_state,
|
||||
);
|
||||
Ok((
|
||||
NodeServiceClient::with_interceptor(channel, interceptor)
|
||||
.max_decoding_message_size(max_message_size)
|
||||
.max_encoding_message_size(max_message_size),
|
||||
policy,
|
||||
))
|
||||
}
|
||||
|
||||
pub(crate) async fn node_service_delete_recovery_client(
|
||||
addr: &str,
|
||||
interceptor: TonicInterceptor,
|
||||
@@ -204,7 +171,7 @@ pub(crate) async fn node_service_delete_recovery_client(
|
||||
let connection_state = channel.connection_state();
|
||||
let interceptor = interceptor.with_connection_state(connection_state.clone());
|
||||
let channel = ReplayScopeChannel::new(
|
||||
ConnectionGenerationChannel::new(BatchLockDeadlineChannel::new(channel), connection_state.clone()),
|
||||
BatchLockDeadlineChannel::new(channel),
|
||||
interceptor.replay_scope_audience(),
|
||||
connection_state,
|
||||
);
|
||||
@@ -324,238 +291,7 @@ pub struct ReplayScopeChannel<S> {
|
||||
}
|
||||
|
||||
/// The channel type used by internode clients after v2 authentication and replay-scope handling.
|
||||
pub type AuthenticatedChannel = ReplayScopeChannel<ConnectionGenerationChannel<BatchLockDeadlineChannel<InternodeChannel>>>;
|
||||
|
||||
const CONNECTION_GENERATION_PENDING: u8 = 0;
|
||||
const CONNECTION_GENERATION_DISPATCHED: u8 = 1;
|
||||
const CONNECTION_GENERATION_REJECTED: u8 = 2;
|
||||
|
||||
#[derive(Debug)]
|
||||
struct ConnectionGenerationGuardInner {
|
||||
expected: GrpcConnectionSnapshot,
|
||||
expected_boot_epoch: Option<Uuid>,
|
||||
outcome: AtomicU8,
|
||||
}
|
||||
|
||||
/// A request-owned proof that its encoded body targets one physical connection generation.
|
||||
#[derive(Clone, Debug)]
|
||||
pub(crate) struct ConnectionGenerationGuard(Arc<ConnectionGenerationGuardInner>);
|
||||
|
||||
impl ConnectionGenerationGuard {
|
||||
pub(crate) fn new(expected: GrpcConnectionSnapshot) -> Self {
|
||||
Self(Arc::new(ConnectionGenerationGuardInner {
|
||||
expected,
|
||||
expected_boot_epoch: None,
|
||||
outcome: AtomicU8::new(CONNECTION_GENERATION_PENDING),
|
||||
}))
|
||||
}
|
||||
|
||||
fn for_peer_capability(expected: GrpcConnectionSnapshot, boot_epoch: Uuid) -> Self {
|
||||
Self(Arc::new(ConnectionGenerationGuardInner {
|
||||
expected,
|
||||
expected_boot_epoch: Some(boot_epoch),
|
||||
outcome: AtomicU8::new(CONNECTION_GENERATION_PENDING),
|
||||
}))
|
||||
}
|
||||
|
||||
fn expected_peer_capability(&self) -> Option<(GrpcConnectionSnapshot, Uuid)> {
|
||||
self.0.expected_boot_epoch.map(|boot_epoch| (self.0.expected, boot_epoch))
|
||||
}
|
||||
|
||||
fn reject_before_dispatch(&self) {
|
||||
let _ = self.0.outcome.compare_exchange(
|
||||
CONNECTION_GENERATION_PENDING,
|
||||
CONNECTION_GENERATION_REJECTED,
|
||||
Ordering::AcqRel,
|
||||
Ordering::Acquire,
|
||||
);
|
||||
}
|
||||
|
||||
fn validate_before_call(&self, current: Option<GrpcConnectionSnapshot>) -> Result<(), StaleConnectionGenerationError> {
|
||||
if self.0.outcome.load(Ordering::Acquire) == CONNECTION_GENERATION_PENDING && current == Some(self.0.expected) {
|
||||
return Ok(());
|
||||
}
|
||||
self.reject_before_dispatch();
|
||||
Err(StaleConnectionGenerationError::new(self.0.expected, current))
|
||||
}
|
||||
|
||||
fn begin_payload(&self, current: Option<GrpcConnectionSnapshot>) -> Result<(), tonic::Status> {
|
||||
if current != Some(self.0.expected) {
|
||||
self.reject_before_dispatch();
|
||||
return Err(stale_connection_generation_status());
|
||||
}
|
||||
self.0
|
||||
.outcome
|
||||
.compare_exchange(
|
||||
CONNECTION_GENERATION_PENDING,
|
||||
CONNECTION_GENERATION_DISPATCHED,
|
||||
Ordering::AcqRel,
|
||||
Ordering::Acquire,
|
||||
)
|
||||
.map(|_| ())
|
||||
.map_err(|_| stale_connection_generation_status())
|
||||
}
|
||||
|
||||
fn rejected_before_dispatch(&self) -> bool {
|
||||
self.0.outcome.load(Ordering::Acquire) == CONNECTION_GENERATION_REJECTED
|
||||
}
|
||||
}
|
||||
|
||||
fn stale_connection_generation_status() -> tonic::Status {
|
||||
tonic::Status::aborted("internode connection generation changed before request dispatch")
|
||||
}
|
||||
|
||||
#[derive(Debug)]
|
||||
struct StaleConnectionGenerationError {
|
||||
expected: GrpcConnectionSnapshot,
|
||||
current: Option<GrpcConnectionSnapshot>,
|
||||
status: tonic::Status,
|
||||
}
|
||||
|
||||
impl StaleConnectionGenerationError {
|
||||
fn new(expected: GrpcConnectionSnapshot, current: Option<GrpcConnectionSnapshot>) -> Self {
|
||||
Self {
|
||||
expected,
|
||||
current,
|
||||
status: stale_connection_generation_status(),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl std::fmt::Display for StaleConnectionGenerationError {
|
||||
fn fmt(&self, formatter: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
write!(
|
||||
formatter,
|
||||
"internode connection changed before dispatch (expected {:?}, current {:?})",
|
||||
self.expected, self.current
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
impl Error for StaleConnectionGenerationError {
|
||||
fn source(&self) -> Option<&(dyn Error + 'static)> {
|
||||
Some(&self.status)
|
||||
}
|
||||
}
|
||||
|
||||
pin_project_lite::pin_project! {
|
||||
#[project = ConnectionGenerationFutureProj]
|
||||
#[doc(hidden)]
|
||||
pub enum ConnectionGenerationFuture<F, Response> {
|
||||
Inner { #[pin] future: F },
|
||||
Rejected { #[pin] future: std::future::Ready<Result<Response, StdError>> },
|
||||
}
|
||||
}
|
||||
|
||||
impl<F, Response, InnerError> Future for ConnectionGenerationFuture<F, Response>
|
||||
where
|
||||
F: Future<Output = Result<Response, InnerError>>,
|
||||
InnerError: Into<StdError>,
|
||||
{
|
||||
type Output = Result<Response, StdError>;
|
||||
|
||||
fn poll(self: Pin<&mut Self>, context: &mut Context<'_>) -> Poll<Self::Output> {
|
||||
match self.project() {
|
||||
ConnectionGenerationFutureProj::Inner { future } => future.poll(context).map_err(Into::into),
|
||||
ConnectionGenerationFutureProj::Rejected { future } => future.poll(context),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Rejects a stale encoded request before dispatching any body data to tonic's channel.
|
||||
#[derive(Clone, Debug)]
|
||||
pub struct ConnectionGenerationChannel<S> {
|
||||
inner: S,
|
||||
connection_state: Option<GrpcConnectionState>,
|
||||
}
|
||||
|
||||
impl<S> ConnectionGenerationChannel<S> {
|
||||
fn new(inner: S, connection_state: Option<GrpcConnectionState>) -> Self {
|
||||
Self { inner, connection_state }
|
||||
}
|
||||
}
|
||||
|
||||
impl<S> Service<HttpRequest<tonic::body::Body>> for ConnectionGenerationChannel<S>
|
||||
where
|
||||
S: Service<HttpRequest<tonic::body::Body>>,
|
||||
S::Error: Into<StdError>,
|
||||
{
|
||||
type Response = S::Response;
|
||||
type Error = StdError;
|
||||
type Future = ConnectionGenerationFuture<S::Future, S::Response>;
|
||||
|
||||
fn poll_ready(&mut self, context: &mut Context<'_>) -> Poll<Result<(), Self::Error>> {
|
||||
self.inner.poll_ready(context).map_err(Into::into)
|
||||
}
|
||||
|
||||
fn call(&mut self, mut request: HttpRequest<tonic::body::Body>) -> Self::Future {
|
||||
let Some(guard) = request.extensions().get::<ConnectionGenerationGuard>().cloned() else {
|
||||
return ConnectionGenerationFuture::Inner {
|
||||
future: self.inner.call(request),
|
||||
};
|
||||
};
|
||||
let current = self.connection_state.as_ref().and_then(GrpcConnectionState::snapshot);
|
||||
if let Err(error) = guard.validate_before_call(current) {
|
||||
return ConnectionGenerationFuture::Rejected {
|
||||
future: std::future::ready(Err(Box::new(error))),
|
||||
};
|
||||
}
|
||||
let body = std::mem::replace(request.body_mut(), tonic::body::Body::empty());
|
||||
*request.body_mut() = tonic::body::Body::new(ConnectionGenerationBody {
|
||||
inner: body,
|
||||
guard,
|
||||
connection_state: self.connection_state.clone(),
|
||||
checked: false,
|
||||
failed: false,
|
||||
});
|
||||
ConnectionGenerationFuture::Inner {
|
||||
future: self.inner.call(request),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
struct ConnectionGenerationBody {
|
||||
inner: tonic::body::Body,
|
||||
guard: ConnectionGenerationGuard,
|
||||
connection_state: Option<GrpcConnectionState>,
|
||||
checked: bool,
|
||||
failed: bool,
|
||||
}
|
||||
|
||||
impl http_body::Body for ConnectionGenerationBody {
|
||||
type Data = bytes::Bytes;
|
||||
type Error = tonic::Status;
|
||||
|
||||
fn poll_frame(
|
||||
mut self: Pin<&mut Self>,
|
||||
context: &mut Context<'_>,
|
||||
) -> Poll<Option<Result<http_body::Frame<Self::Data>, Self::Error>>> {
|
||||
if self.failed {
|
||||
return Poll::Ready(None);
|
||||
}
|
||||
let frame = std::task::ready!(Pin::new(&mut self.inner).poll_frame(context));
|
||||
if let Some(Ok(frame)) = &frame
|
||||
&& frame.is_data()
|
||||
&& !self.checked
|
||||
{
|
||||
let current = self.connection_state.as_ref().and_then(GrpcConnectionState::snapshot);
|
||||
if let Err(error) = self.guard.begin_payload(current) {
|
||||
self.failed = true;
|
||||
return Poll::Ready(Some(Err(error)));
|
||||
}
|
||||
self.checked = true;
|
||||
}
|
||||
Poll::Ready(frame)
|
||||
}
|
||||
|
||||
fn is_end_stream(&self) -> bool {
|
||||
self.failed || self.inner.is_end_stream()
|
||||
}
|
||||
|
||||
fn size_hint(&self) -> http_body::SizeHint {
|
||||
self.inner.size_hint()
|
||||
}
|
||||
}
|
||||
pub type AuthenticatedChannel = ReplayScopeChannel<BatchLockDeadlineChannel<InternodeChannel>>;
|
||||
|
||||
/// Gate lock-batch payloads after the transport's internal readiness queues.
|
||||
/// Other RPC bodies pass through without an additional wrapper.
|
||||
@@ -779,149 +515,6 @@ fn should_request_msgpack_capability(state: PeerReplayStateSnapshot) -> bool {
|
||||
}
|
||||
}
|
||||
|
||||
fn validate_peer_capability_guard(guard: &ConnectionGenerationGuard, state: PeerReplayStateSnapshot) {
|
||||
let Some((expected_connection, expected_boot_epoch)) = guard.expected_peer_capability() else {
|
||||
return;
|
||||
};
|
||||
let capability_is_current = state.connection == Some(expected_connection)
|
||||
&& state.replay.boot_epoch == Some(expected_boot_epoch)
|
||||
&& matches!(
|
||||
state.msgpack_capability,
|
||||
Some(PeerMsgpackCapability::Capable {
|
||||
boot_epoch,
|
||||
connection_generation,
|
||||
}) if boot_epoch == expected_boot_epoch
|
||||
&& connection_generation == expected_connection.generation
|
||||
);
|
||||
if !capability_is_current {
|
||||
guard.reject_before_dispatch();
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug)]
|
||||
enum MsgpackSendMode {
|
||||
DualWrite,
|
||||
ManualFleet,
|
||||
PeerCapable { guard: ConnectionGenerationGuard },
|
||||
}
|
||||
|
||||
/// One eligible RPC's immutable JSON/msgpack decision and optional pre-dispatch guard.
|
||||
#[derive(Clone, Debug)]
|
||||
pub(crate) struct MsgpackSendPolicy {
|
||||
mode: MsgpackSendMode,
|
||||
}
|
||||
|
||||
impl MsgpackSendPolicy {
|
||||
#[cfg(test)]
|
||||
pub(crate) fn for_test(omit_eligible_json: bool) -> Self {
|
||||
Self {
|
||||
mode: if omit_eligible_json {
|
||||
MsgpackSendMode::ManualFleet
|
||||
} else {
|
||||
MsgpackSendMode::DualWrite
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
fn dual_write(reason: &'static str) -> Self {
|
||||
rustfs_io_metrics::internode_metrics::global_internode_metrics().record_msgpack_send_decision("dual_write", reason);
|
||||
Self {
|
||||
mode: MsgpackSendMode::DualWrite,
|
||||
}
|
||||
}
|
||||
|
||||
fn manual_fleet() -> Self {
|
||||
rustfs_io_metrics::internode_metrics::global_internode_metrics()
|
||||
.record_msgpack_send_decision("msgpack_only", "manual_fleet");
|
||||
Self {
|
||||
mode: MsgpackSendMode::ManualFleet,
|
||||
}
|
||||
}
|
||||
|
||||
fn peer_capable(connection: GrpcConnectionSnapshot, boot_epoch: Uuid) -> Self {
|
||||
rustfs_io_metrics::internode_metrics::global_internode_metrics()
|
||||
.record_msgpack_send_decision("msgpack_only", "peer_capable");
|
||||
Self {
|
||||
mode: MsgpackSendMode::PeerCapable {
|
||||
guard: ConnectionGenerationGuard::for_peer_capability(connection, boot_epoch),
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
pub(crate) fn omit_eligible_json(&self) -> bool {
|
||||
!matches!(self.mode, MsgpackSendMode::DualWrite)
|
||||
}
|
||||
|
||||
pub(crate) fn guard_request<T>(&self, request: &mut tonic::Request<T>) {
|
||||
if let MsgpackSendMode::PeerCapable { guard } = &self.mode {
|
||||
request.extensions_mut().insert(guard.clone());
|
||||
}
|
||||
}
|
||||
|
||||
/// Switch one peer-capability attempt to dual-write only after a proven local rejection.
|
||||
pub(crate) fn retry_dual_write_after_local_rejection(&mut self) -> bool {
|
||||
let rejected = matches!(
|
||||
&self.mode,
|
||||
MsgpackSendMode::PeerCapable { guard } if guard.rejected_before_dispatch()
|
||||
);
|
||||
if rejected {
|
||||
*self = Self::dual_write("peer_revoked");
|
||||
}
|
||||
rejected
|
||||
}
|
||||
}
|
||||
|
||||
fn prepare_msgpack_send_policy(audience: Option<&str>, connection: Option<GrpcConnectionSnapshot>) -> MsgpackSendPolicy {
|
||||
prepare_msgpack_send_policy_with_gates(
|
||||
audience,
|
||||
connection,
|
||||
rustfs_protos::internode_rpc_msgpack_only(),
|
||||
rustfs_protos::internode_rpc_msgpack_peer_capability_enabled(),
|
||||
)
|
||||
}
|
||||
|
||||
fn prepare_msgpack_send_policy_with_gates(
|
||||
audience: Option<&str>,
|
||||
connection: Option<GrpcConnectionSnapshot>,
|
||||
manual_fleet: bool,
|
||||
peer_capability: bool,
|
||||
) -> MsgpackSendPolicy {
|
||||
if manual_fleet {
|
||||
return MsgpackSendPolicy::manual_fleet();
|
||||
}
|
||||
if !peer_capability {
|
||||
return MsgpackSendPolicy::dual_write("not_requested");
|
||||
}
|
||||
let (Some(audience), Some(connection)) = (audience, connection) else {
|
||||
return MsgpackSendPolicy::dual_write("peer_unknown");
|
||||
};
|
||||
let Ok(states) = PEER_REPLAY_STATES.lock() else {
|
||||
return MsgpackSendPolicy::dual_write("peer_unknown");
|
||||
};
|
||||
let snapshot = states.snapshot(audience, Some(connection));
|
||||
let (capable_epoch, revoked) = match (snapshot.replay.boot_epoch, snapshot.msgpack_capability) {
|
||||
(
|
||||
Some(boot_epoch),
|
||||
Some(PeerMsgpackCapability::Capable {
|
||||
boot_epoch: capability_epoch,
|
||||
connection_generation,
|
||||
}),
|
||||
) if boot_epoch == capability_epoch && connection.generation == connection_generation => (Some(boot_epoch), false),
|
||||
(_, Some(PeerMsgpackCapability::Revoked { connection_generation })) if connection.generation == connection_generation => {
|
||||
(None, true)
|
||||
}
|
||||
_ => (None, false),
|
||||
};
|
||||
drop(states);
|
||||
if let Some(boot_epoch) = capable_epoch {
|
||||
MsgpackSendPolicy::peer_capable(connection, boot_epoch)
|
||||
} else if revoked {
|
||||
MsgpackSendPolicy::dual_write("peer_revoked")
|
||||
} else {
|
||||
MsgpackSendPolicy::dual_write("peer_unknown")
|
||||
}
|
||||
}
|
||||
|
||||
pub(crate) fn clear_peer_replay_state_for_addr(addr: &str) -> std::io::Result<()> {
|
||||
let uri = addr
|
||||
.parse::<Uri>()
|
||||
@@ -1094,17 +687,7 @@ where
|
||||
.get::<PeerReplayStateSnapshot>()
|
||||
.copied()
|
||||
.unwrap_or_default();
|
||||
if let Some(guard) = request.extensions().get::<ConnectionGenerationGuard>() {
|
||||
validate_peer_capability_guard(guard, sent_state);
|
||||
}
|
||||
let msgpack_capability_requested = authenticated && should_request_msgpack_capability(sent_state);
|
||||
if msgpack_capability_requested
|
||||
&& request.uri().path() == "/node_service.NodeService/Ping"
|
||||
&& request.extensions().get::<ConnectionGenerationGuard>().is_none()
|
||||
&& let Some(expected) = sent_state.connection
|
||||
{
|
||||
request.extensions_mut().insert(ConnectionGenerationGuard::new(expected));
|
||||
}
|
||||
if let (Some(audience), Some(challenge)) = (self.audience.as_deref(), challenge) {
|
||||
// The challenge is independently HMAC-authenticated by the response proof. It is not
|
||||
// part of v2 so old peers ignore it, while a new peer can safely advertise its epoch.
|
||||
@@ -1428,106 +1011,6 @@ mod tests {
|
||||
use tracing_opentelemetry::OpenTelemetrySpanExt;
|
||||
use tracing_subscriber::{Registry, layer::SubscriberExt};
|
||||
|
||||
#[tokio::test]
|
||||
async fn connection_generation_guard_rejects_only_local_predispatch_races() {
|
||||
use http_body_util::BodyExt as _;
|
||||
use std::sync::atomic::AtomicUsize;
|
||||
|
||||
#[derive(Clone)]
|
||||
struct DrainBodyService {
|
||||
calls: Arc<AtomicUsize>,
|
||||
payloads: Arc<AtomicUsize>,
|
||||
}
|
||||
|
||||
impl Service<HttpRequest<tonic::body::Body>> for DrainBodyService {
|
||||
type Response = HttpResponse<tonic::body::Body>;
|
||||
type Error = tonic::Status;
|
||||
type Future = Pin<Box<dyn Future<Output = Result<Self::Response, Self::Error>> + Send>>;
|
||||
|
||||
fn poll_ready(&mut self, _context: &mut Context<'_>) -> Poll<Result<(), Self::Error>> {
|
||||
Poll::Ready(Ok(()))
|
||||
}
|
||||
|
||||
fn call(&mut self, request: HttpRequest<tonic::body::Body>) -> Self::Future {
|
||||
self.calls.fetch_add(1, Ordering::SeqCst);
|
||||
let payloads = Arc::clone(&self.payloads);
|
||||
Box::pin(async move {
|
||||
let mut body = request.into_body();
|
||||
match body.frame().await {
|
||||
Some(Ok(frame)) if frame.is_data() => {
|
||||
payloads.fetch_add(1, Ordering::SeqCst);
|
||||
Ok(HttpResponse::new(tonic::body::Body::empty()))
|
||||
}
|
||||
Some(Err(error)) => Err(error),
|
||||
_ => Err(tonic::Status::internal("guard test body did not produce payload data")),
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
fn request(guard: ConnectionGenerationGuard) -> HttpRequest<tonic::body::Body> {
|
||||
let mut request =
|
||||
HttpRequest::new(tonic::body::Body::new(http_body_util::Full::new(bytes::Bytes::from_static(b"payload"))));
|
||||
request.extensions_mut().insert(guard);
|
||||
request
|
||||
}
|
||||
|
||||
fn has_local_generation_source(status: &tonic::Status) -> bool {
|
||||
let mut source = status.source();
|
||||
while let Some(error) = source {
|
||||
if error.downcast_ref::<StaleConnectionGenerationError>().is_some() {
|
||||
return true;
|
||||
}
|
||||
source = error.source();
|
||||
}
|
||||
false
|
||||
}
|
||||
|
||||
let state = GrpcConnectionState::new();
|
||||
let first = state.record_connected();
|
||||
let calls = Arc::new(AtomicUsize::new(0));
|
||||
let payloads = Arc::new(AtomicUsize::new(0));
|
||||
let service = DrainBodyService {
|
||||
calls: Arc::clone(&calls),
|
||||
payloads: Arc::clone(&payloads),
|
||||
};
|
||||
let mut channel = ConnectionGenerationChannel::new(service, Some(state.clone()));
|
||||
|
||||
let rejected_before_call = ConnectionGenerationGuard::new(first);
|
||||
let second = state.record_connected();
|
||||
let error = channel
|
||||
.call(request(rejected_before_call.clone()))
|
||||
.await
|
||||
.expect_err("stale generation must fail before inner call");
|
||||
let status = tonic::Status::from_error(error);
|
||||
assert_eq!(status.code(), tonic::Code::Aborted);
|
||||
assert!(has_local_generation_source(&status));
|
||||
assert!(rejected_before_call.rejected_before_dispatch());
|
||||
assert_eq!(calls.load(Ordering::SeqCst), 0);
|
||||
assert_eq!(payloads.load(Ordering::SeqCst), 0);
|
||||
assert!(!has_local_generation_source(&tonic::Status::aborted(status.message())));
|
||||
|
||||
let rejected_before_payload = ConnectionGenerationGuard::new(second);
|
||||
let pending = channel.call(request(rejected_before_payload.clone()));
|
||||
let third = state.record_connected();
|
||||
let error = pending
|
||||
.await
|
||||
.expect_err("generation change before body poll must reject payload");
|
||||
assert_eq!(tonic::Status::from_error(error).code(), tonic::Code::Aborted);
|
||||
assert!(rejected_before_payload.rejected_before_dispatch());
|
||||
assert_eq!(calls.load(Ordering::SeqCst), 1);
|
||||
assert_eq!(payloads.load(Ordering::SeqCst), 0);
|
||||
|
||||
let dispatched = ConnectionGenerationGuard::new(third);
|
||||
channel
|
||||
.call(request(dispatched.clone()))
|
||||
.await
|
||||
.expect("matching generation dispatches payload");
|
||||
assert!(!dispatched.rejected_before_dispatch());
|
||||
assert_eq!(calls.load(Ordering::SeqCst), 2);
|
||||
assert_eq!(payloads.load(Ordering::SeqCst), 1);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn delete_recovery_channel_uses_its_scoped_deadline() {
|
||||
use bytes::Bytes;
|
||||
@@ -2573,68 +2056,6 @@ mod tests {
|
||||
clear_peer_capability(audience);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn msgpack_send_policy_requires_exact_capability_and_retries_only_local_rejection() {
|
||||
let audience = "msgpack-send-policy-test:9000";
|
||||
let boot_epoch = Uuid::new_v4();
|
||||
let connection = GrpcConnectionSnapshot {
|
||||
channel_id: 21,
|
||||
generation: 201,
|
||||
};
|
||||
clear_peer_capability(audience);
|
||||
let unknown = PEER_REPLAY_STATES
|
||||
.lock()
|
||||
.expect("peer replay registry")
|
||||
.snapshot(audience, Some(connection));
|
||||
apply_peer_capabilities_response_for_connection(
|
||||
audience.to_string(),
|
||||
unknown,
|
||||
Ok(authenticated_peer_response(boot_epoch, true)),
|
||||
Some(Ok(authenticated_msgpack_response(boot_epoch, true))),
|
||||
Some(connection),
|
||||
);
|
||||
|
||||
let mut peer_policy = prepare_msgpack_send_policy_with_gates(Some(audience), Some(connection), false, true);
|
||||
assert!(peer_policy.omit_eligible_json());
|
||||
let mut request = tonic::Request::new(());
|
||||
peer_policy.guard_request(&mut request);
|
||||
assert!(request.extensions().get::<ConnectionGenerationGuard>().is_some());
|
||||
let MsgpackSendMode::PeerCapable { guard } = &peer_policy.mode else {
|
||||
panic!("proved peer must use guarded msgpack-only policy");
|
||||
};
|
||||
let guard = guard.clone();
|
||||
{
|
||||
let mut states = PEER_REPLAY_STATES.lock().expect("peer replay registry");
|
||||
states.set_msgpack_capability(
|
||||
audience,
|
||||
Some(connection),
|
||||
PeerMsgpackCapability::Revoked {
|
||||
connection_generation: connection.generation,
|
||||
},
|
||||
);
|
||||
}
|
||||
let revoked = PEER_REPLAY_STATES
|
||||
.lock()
|
||||
.expect("peer replay registry")
|
||||
.snapshot(audience, Some(connection));
|
||||
validate_peer_capability_guard(&guard, revoked);
|
||||
assert!(peer_policy.retry_dual_write_after_local_rejection());
|
||||
assert!(!peer_policy.omit_eligible_json());
|
||||
assert!(!peer_policy.retry_dual_write_after_local_rejection());
|
||||
|
||||
let replacement = GrpcConnectionSnapshot {
|
||||
channel_id: connection.channel_id,
|
||||
generation: connection.generation + 1,
|
||||
};
|
||||
let unknown_policy = prepare_msgpack_send_policy_with_gates(Some(audience), Some(replacement), false, true);
|
||||
assert!(!unknown_policy.omit_eligible_json());
|
||||
let manual_policy = prepare_msgpack_send_policy_with_gates(None, None, true, false);
|
||||
assert!(manual_policy.omit_eligible_json());
|
||||
let disabled_policy = prepare_msgpack_send_policy_with_gates(Some(audience), Some(connection), false, false);
|
||||
assert!(!disabled_policy.omit_eligible_json());
|
||||
clear_peer_capability(audience);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn clear_peer_replay_state_for_addr_removes_normalized_audience() {
|
||||
let audience = "clear-peer-replay-state-test:9000";
|
||||
|
||||
@@ -13,9 +13,8 @@
|
||||
// limitations under the License.
|
||||
|
||||
use crate::cluster::rpc::client::{
|
||||
AuthenticatedChannel, MsgpackSendPolicy, TonicInterceptor, gen_tonic_signature_interceptor, is_network_like_disk_error,
|
||||
AuthenticatedChannel, TonicInterceptor, gen_tonic_signature_interceptor, is_network_like_disk_error,
|
||||
node_service_delete_recovery_client, node_service_time_out_client, node_service_time_out_client_for_class,
|
||||
node_service_time_out_client_for_class_with_msgpack_policy,
|
||||
};
|
||||
use crate::cluster::rpc::http_auth::{current_rpc_background_request, with_rpc_background_request};
|
||||
use crate::cluster::rpc::internode_data_transport::{
|
||||
@@ -2454,19 +2453,6 @@ impl RemoteDisk {
|
||||
.map_err(|err| Error::RemoteClientUnavailable(err.to_string()))
|
||||
}
|
||||
|
||||
async fn get_client_with_msgpack_policy(&self) -> Result<(NodeRpcClient, MsgpackSendPolicy)> {
|
||||
if let Some(err) = self.offline_bypass_error() {
|
||||
return Err(err);
|
||||
}
|
||||
node_service_time_out_client_for_class_with_msgpack_policy(
|
||||
&self.addr,
|
||||
TonicInterceptor::Signature(gen_tonic_signature_interceptor()),
|
||||
ChannelClass::Control,
|
||||
)
|
||||
.await
|
||||
.map_err(|err| Error::RemoteClientUnavailable(err.to_string()))
|
||||
}
|
||||
|
||||
/// Client for large `bytes`-carrying RPCs, including native pool metadata recovery inventories.
|
||||
/// Routes onto the isolated bulk channel pool so large transfers cannot head-of-line block
|
||||
/// lock/health RPCs (grpc-optimization P1). Falls back to the control channel when isolation
|
||||
@@ -2507,19 +2493,6 @@ impl RemoteDisk {
|
||||
.map_err(|err| Error::RemoteClientUnavailable(err.to_string()))
|
||||
}
|
||||
|
||||
async fn get_bulk_client_with_msgpack_policy(&self) -> Result<(NodeRpcClient, MsgpackSendPolicy)> {
|
||||
if let Some(err) = self.offline_bypass_error() {
|
||||
return Err(err);
|
||||
}
|
||||
node_service_time_out_client_for_class_with_msgpack_policy(
|
||||
&self.addr,
|
||||
TonicInterceptor::Signature(gen_tonic_signature_interceptor()),
|
||||
ChannelClass::Bulk,
|
||||
)
|
||||
.await
|
||||
.map_err(|err| Error::RemoteClientUnavailable(err.to_string()))
|
||||
}
|
||||
|
||||
async fn disk_ref(&self) -> String {
|
||||
(*self.id.lock().await)
|
||||
.map(|id| id.to_string())
|
||||
@@ -2559,46 +2532,11 @@ fn encode_file_info_versions_msgpack(value: &FileInfoVersions) -> Result<Vec<u8>
|
||||
encode_msgpack_with_capacity(value, capacity)
|
||||
}
|
||||
|
||||
fn encode_delete_versions_request(
|
||||
disk: String,
|
||||
volume: &str,
|
||||
versions: &[FileInfoVersions],
|
||||
opts: &DeleteOptions,
|
||||
) -> Result<DeleteVersionsRequest> {
|
||||
// Binary-only batches require fleet-confirmed receivers that traverse the
|
||||
// binary list. Capture that policy for the complete batch and its options.
|
||||
let msgpack_only = rustfs_protos::internode_rpc_msgpack_only();
|
||||
let opts_json = if msgpack_only {
|
||||
String::new()
|
||||
} else {
|
||||
serde_json::to_string(opts)?
|
||||
};
|
||||
let mut versions_json = if msgpack_only {
|
||||
Vec::new()
|
||||
} else {
|
||||
Vec::with_capacity(versions.len())
|
||||
};
|
||||
let mut versions_bin = Vec::with_capacity(versions.len());
|
||||
for version in versions {
|
||||
if !msgpack_only {
|
||||
versions_json.push(serde_json::to_string(version)?);
|
||||
}
|
||||
versions_bin.push(encode_file_info_versions_msgpack(version)?.into());
|
||||
}
|
||||
Ok(DeleteVersionsRequest {
|
||||
disk,
|
||||
volume: volume.to_owned(),
|
||||
versions: versions_json,
|
||||
opts: opts_json,
|
||||
versions_bin,
|
||||
opts_bin: encode_delete_options_msgpack(opts)?.into(),
|
||||
})
|
||||
}
|
||||
|
||||
/// JSON compatibility string for a dual-encoded request field. The captured
|
||||
/// send policy selects the encoding, and its generation guard binds dispatch.
|
||||
fn compat_json<T: Serialize>(value: &T, policy: &MsgpackSendPolicy) -> Result<String> {
|
||||
if policy.omit_eligible_json() {
|
||||
/// JSON compatibility string for a dual-encoded (`_bin` + text) request field. Returns an empty
|
||||
/// string only when msgpack-only mode and its explicit fleet confirmation guard are both enabled;
|
||||
/// otherwise the legacy JSON encoding is retained for old peers.
|
||||
fn compat_json<T: Serialize>(value: &T) -> Result<String> {
|
||||
if rustfs_protos::internode_rpc_msgpack_only() {
|
||||
return Ok(String::new());
|
||||
}
|
||||
Ok(serde_json::to_string(value)?)
|
||||
@@ -2872,12 +2810,26 @@ impl RemoteDisk {
|
||||
|| async {
|
||||
let prepare_started = rustfs_io_metrics::put_stage_timer();
|
||||
let collect_prepare_timing = prepare_started.is_some();
|
||||
let json_encode_started = collect_prepare_timing.then(Instant::now);
|
||||
let file_info = compat_json(fi);
|
||||
rustfs_io_metrics::record_put_object_stage_duration_from(
|
||||
rustfs_io_metrics::PUT_STAGE_SET_DISK_RENAME_REMOTE_CLIENT_JSON_ENCODE,
|
||||
json_encode_started,
|
||||
);
|
||||
let file_info = file_info?;
|
||||
if collect_prepare_timing {
|
||||
rustfs_io_metrics::record_put_object_stage_bytes(
|
||||
rustfs_io_metrics::PUT_STAGE_SET_DISK_RENAME_REMOTE_CLIENT_JSON_BYTES,
|
||||
file_info.len(),
|
||||
);
|
||||
}
|
||||
let msgpack_encode_started = collect_prepare_timing.then(Instant::now);
|
||||
let file_info_bin = Bytes::from(encode_file_info_msgpack(fi)?);
|
||||
let file_info_bin = encode_file_info_msgpack(fi);
|
||||
rustfs_io_metrics::record_put_object_stage_duration_from(
|
||||
rustfs_io_metrics::PUT_STAGE_SET_DISK_RENAME_REMOTE_CLIENT_MSGPACK_ENCODE,
|
||||
msgpack_encode_started,
|
||||
);
|
||||
let file_info_bin = file_info_bin?;
|
||||
if collect_prepare_timing {
|
||||
rustfs_io_metrics::record_put_object_stage_bytes(
|
||||
rustfs_io_metrics::PUT_STAGE_SET_DISK_RENAME_REMOTE_CLIENT_MSGPACK_BYTES,
|
||||
@@ -2885,57 +2837,35 @@ impl RemoteDisk {
|
||||
);
|
||||
}
|
||||
let get_client_started = collect_prepare_timing.then(Instant::now);
|
||||
let client = self.get_client_with_msgpack_policy().await;
|
||||
let client = self.get_client().await;
|
||||
rustfs_io_metrics::record_put_object_stage_duration_from(
|
||||
rustfs_io_metrics::PUT_STAGE_SET_DISK_RENAME_REMOTE_CLIENT_GET_CLIENT,
|
||||
get_client_started,
|
||||
);
|
||||
let (mut client, mut policy) = client?;
|
||||
let bucket_incarnation_id = crate::store::bucket_heal_scope_for_object(dst_volume, dst_path)
|
||||
.map(|scope| Bytes::copy_from_slice(scope.incarnation.as_bytes()))
|
||||
.unwrap_or_default();
|
||||
let incarnation_bound = !bucket_incarnation_id.is_empty();
|
||||
let mut client = client?;
|
||||
let mut request = Request::new(RenameDataRequest {
|
||||
bucket_incarnation_id: crate::store::bucket_heal_scope_for_object(dst_volume, dst_path)
|
||||
.map(|scope| scope.incarnation.as_bytes().to_vec().into())
|
||||
.unwrap_or_default(),
|
||||
disk: self.endpoint.to_string(),
|
||||
src_volume: src_volume.to_string(),
|
||||
src_path: src_path.to_string(),
|
||||
file_info,
|
||||
dst_volume: dst_volume.to_string(),
|
||||
dst_path: dst_path.to_string(),
|
||||
file_info_bin: file_info_bin.into(),
|
||||
scanner_publication_lease_token: scanner_publication_lease_token
|
||||
.map(|token| token.as_bytes().to_vec().into())
|
||||
.unwrap_or_default(),
|
||||
});
|
||||
let canonical_body_digest_started = collect_prepare_timing.then(Instant::now);
|
||||
let incarnation_bound = !request.get_ref().bucket_incarnation_id.is_empty();
|
||||
let quota_fenced = rustfs_utils::http::metadata_compat::contains_key_str(
|
||||
&fi.metadata,
|
||||
crate::disk::QUOTA_MUTATION_FENCE_METADATA_SUFFIX,
|
||||
);
|
||||
let scanner_publication_lease = scanner_publication_lease_token
|
||||
.map(|token| Bytes::copy_from_slice(token.as_bytes()))
|
||||
.unwrap_or_default();
|
||||
let rpc_started = rustfs_io_metrics::put_stage_timer();
|
||||
let mut retried_dual_write = false;
|
||||
let response = loop {
|
||||
let attempt_prepare_started = if retried_dual_write {
|
||||
rustfs_io_metrics::put_stage_timer()
|
||||
} else {
|
||||
prepare_started
|
||||
};
|
||||
let json_encode_started = collect_prepare_timing.then(Instant::now);
|
||||
let file_info = compat_json(fi, &policy)?;
|
||||
rustfs_io_metrics::record_put_object_stage_duration_from(
|
||||
rustfs_io_metrics::PUT_STAGE_SET_DISK_RENAME_REMOTE_CLIENT_JSON_ENCODE,
|
||||
json_encode_started,
|
||||
);
|
||||
if collect_prepare_timing {
|
||||
rustfs_io_metrics::record_put_object_stage_bytes(
|
||||
rustfs_io_metrics::PUT_STAGE_SET_DISK_RENAME_REMOTE_CLIENT_JSON_BYTES,
|
||||
file_info.len(),
|
||||
);
|
||||
}
|
||||
let mut request = Request::new(RenameDataRequest {
|
||||
bucket_incarnation_id: bucket_incarnation_id.clone(),
|
||||
disk: self.endpoint.to_string(),
|
||||
src_volume: src_volume.to_string(),
|
||||
src_path: src_path.to_string(),
|
||||
file_info,
|
||||
dst_volume: dst_volume.to_string(),
|
||||
dst_path: dst_path.to_string(),
|
||||
file_info_bin: file_info_bin.clone(),
|
||||
scanner_publication_lease_token: scanner_publication_lease.clone(),
|
||||
});
|
||||
let canonical_body_digest_started = collect_prepare_timing.then(Instant::now);
|
||||
let canonical_body_digest = (|| -> Result<()> {
|
||||
let canonical_body = rustfs_protos::canonical_rename_data_request_body(request.get_ref());
|
||||
policy.guard_request(&mut request);
|
||||
if pool_meta_cas {
|
||||
let body =
|
||||
canonical_body.map_err(|_| Error::other("pool metadata CAS body length cannot be represented"))?;
|
||||
@@ -2950,44 +2880,48 @@ impl RemoteDisk {
|
||||
} else {
|
||||
attach_mutation_body_digest(&mut request, canonical_body, "rename_data")?;
|
||||
}
|
||||
rustfs_io_metrics::record_put_object_stage_duration_from(
|
||||
rustfs_io_metrics::PUT_STAGE_SET_DISK_RENAME_REMOTE_CLIENT_CANONICAL_BODY_DIGEST,
|
||||
canonical_body_digest_started,
|
||||
);
|
||||
rustfs_io_metrics::record_put_object_stage_duration_from(
|
||||
rustfs_io_metrics::PUT_STAGE_SET_DISK_RENAME_REMOTE_CLIENT_PREPARE,
|
||||
attempt_prepare_started,
|
||||
);
|
||||
let result = rustfs_io_metrics::observe_put_stage_future(
|
||||
async {
|
||||
if multipart_rename {
|
||||
client.rename_multipart_data(request).await
|
||||
} else if pool_meta_cas {
|
||||
client.rename_data_with_pool_meta_cas(request).await
|
||||
} else if quota_fenced {
|
||||
client.rename_data_with_quota_fence(request).await
|
||||
} else if incarnation_bound {
|
||||
client.rename_data_at_incarnation(request).await
|
||||
} else {
|
||||
client.rename_data(request).await
|
||||
}
|
||||
},
|
||||
rustfs_io_metrics::PUT_STAGE_SET_DISK_RENAME_REMOTE_CLIENT_RPC_AWAIT,
|
||||
rustfs_io_metrics::PUT_STAGE_SET_DISK_RENAME_REMOTE_CLIENT_RPC_AWAIT_POLL_PENDING_COUNT,
|
||||
rustfs_io_metrics::PUT_STAGE_SET_DISK_RENAME_REMOTE_CLIENT_RPC_AWAIT_FIRST_PENDING_TO_READY,
|
||||
)
|
||||
.await;
|
||||
match result {
|
||||
Ok(response) => break response,
|
||||
Err(_) if !retried_dual_write && policy.retry_dual_write_after_local_rejection() => {
|
||||
retried_dual_write = true;
|
||||
Ok(())
|
||||
})();
|
||||
rustfs_io_metrics::record_put_object_stage_duration_from(
|
||||
rustfs_io_metrics::PUT_STAGE_SET_DISK_RENAME_REMOTE_CLIENT_CANONICAL_BODY_DIGEST,
|
||||
canonical_body_digest_started,
|
||||
);
|
||||
canonical_body_digest?;
|
||||
rustfs_io_metrics::record_put_object_stage_duration_from(
|
||||
rustfs_io_metrics::PUT_STAGE_SET_DISK_RENAME_REMOTE_CLIENT_PREPARE,
|
||||
prepare_started,
|
||||
);
|
||||
|
||||
let rpc_started = rustfs_io_metrics::put_stage_timer();
|
||||
let response = rustfs_io_metrics::observe_put_stage_future(
|
||||
async {
|
||||
if multipart_rename {
|
||||
// Old peers must reject before an upload source can be changed.
|
||||
client.rename_multipart_data(request).await
|
||||
} else if pool_meta_cas {
|
||||
client.rename_data_with_pool_meta_cas(request).await
|
||||
} else if quota_fenced {
|
||||
// The method itself proves quota-token enforcement; an old peer must fail closed.
|
||||
client.rename_data_with_quota_fence(request).await
|
||||
} else if incarnation_bound {
|
||||
// Older peers return Unimplemented before mutation; never downgrade.
|
||||
client.rename_data_at_incarnation(request).await
|
||||
} else {
|
||||
client.rename_data(request).await
|
||||
}
|
||||
Err(status) if (multipart_rename || pool_meta_cas) && status.code() == Code::Unimplemented => {
|
||||
return Err(DiskError::UnsupportedDisk);
|
||||
}
|
||||
Err(status) => return Err(status.into()),
|
||||
},
|
||||
rustfs_io_metrics::PUT_STAGE_SET_DISK_RENAME_REMOTE_CLIENT_RPC_AWAIT,
|
||||
rustfs_io_metrics::PUT_STAGE_SET_DISK_RENAME_REMOTE_CLIENT_RPC_AWAIT_POLL_PENDING_COUNT,
|
||||
rustfs_io_metrics::PUT_STAGE_SET_DISK_RENAME_REMOTE_CLIENT_RPC_AWAIT_FIRST_PENDING_TO_READY,
|
||||
)
|
||||
.await
|
||||
.map_err(|status| {
|
||||
if (multipart_rename || pool_meta_cas) && status.code() == Code::Unimplemented {
|
||||
DiskError::UnsupportedDisk
|
||||
} else {
|
||||
status.into()
|
||||
}
|
||||
};
|
||||
})?;
|
||||
let response_process_started = rustfs_io_metrics::put_stage_timer();
|
||||
let response_headers_started = rustfs_io_metrics::put_stage_timer();
|
||||
let _ = response.metadata().len();
|
||||
@@ -3582,10 +3516,52 @@ impl DiskAPI for RemoteDisk {
|
||||
return vec![Some(DiskError::FaultyDisk); versions.len()];
|
||||
}
|
||||
|
||||
let request = match encode_delete_versions_request(self.endpoint.to_string(), volume, &versions, &opts) {
|
||||
Ok(request) => request,
|
||||
Err(err) => return vec![Some(Error::other(err.to_string())); versions.len()],
|
||||
// `_bin` support for DeleteVersions is new (grpc-optimization P2); always dual-write JSON +
|
||||
// msgpack until its fallback counter has read zero across a release window.
|
||||
let opts_bin = match encode_delete_options_msgpack(&opts) {
|
||||
Ok(opts_bin) => opts_bin,
|
||||
Err(err) => {
|
||||
let mut errors = Vec::with_capacity(versions.len());
|
||||
for _ in 0..versions.len() {
|
||||
errors.push(Some(Error::other(err.to_string())));
|
||||
}
|
||||
return errors;
|
||||
}
|
||||
};
|
||||
let opts = match serde_json::to_string(&opts) {
|
||||
Ok(opts) => opts,
|
||||
Err(err) => {
|
||||
let mut errors = Vec::with_capacity(versions.len());
|
||||
for _ in 0..versions.len() {
|
||||
errors.push(Some(Error::other(err.to_string())));
|
||||
}
|
||||
return errors;
|
||||
}
|
||||
};
|
||||
let mut versions_str = Vec::with_capacity(versions.len());
|
||||
let mut versions_bin = Vec::with_capacity(versions.len());
|
||||
for file_info_versions in versions.iter() {
|
||||
versions_str.push(match serde_json::to_string(file_info_versions) {
|
||||
Ok(versions_str) => versions_str,
|
||||
Err(err) => {
|
||||
let mut errors = Vec::with_capacity(versions.len());
|
||||
for _ in 0..versions.len() {
|
||||
errors.push(Some(Error::other(err.to_string())));
|
||||
}
|
||||
return errors;
|
||||
}
|
||||
});
|
||||
versions_bin.push(match encode_file_info_versions_msgpack(file_info_versions) {
|
||||
Ok(versions_bin) => Bytes::from(versions_bin),
|
||||
Err(err) => {
|
||||
let mut errors = Vec::with_capacity(versions.len());
|
||||
for _ in 0..versions.len() {
|
||||
errors.push(Some(Error::other(err.to_string())));
|
||||
}
|
||||
return errors;
|
||||
}
|
||||
});
|
||||
}
|
||||
let mut client = match self.get_client().await {
|
||||
Ok(client) => client,
|
||||
Err(err) => {
|
||||
@@ -3597,7 +3573,14 @@ impl DiskAPI for RemoteDisk {
|
||||
}
|
||||
};
|
||||
|
||||
let mut request = Request::new(request);
|
||||
let mut request = Request::new(DeleteVersionsRequest {
|
||||
disk: self.endpoint.to_string(),
|
||||
volume: volume.to_string(),
|
||||
versions: versions_str,
|
||||
opts,
|
||||
versions_bin,
|
||||
opts_bin: opts_bin.into(),
|
||||
});
|
||||
let canonical_body = rustfs_protos::canonical_delete_versions_request_body(request.get_ref());
|
||||
if let Err(err) = attach_mutation_body_digest(&mut request, canonical_body, "delete_versions") {
|
||||
let mut errors = Vec::with_capacity(versions.len());
|
||||
@@ -3773,45 +3756,34 @@ impl DiskAPI for RemoteDisk {
|
||||
state = "started",
|
||||
"Remote disk RPC started"
|
||||
);
|
||||
let file_info_bin = Bytes::from(encode_file_info_msgpack(&fi)?);
|
||||
let file_info = compat_json(&fi)?;
|
||||
let file_info_bin = encode_file_info_msgpack(&fi)?;
|
||||
|
||||
self.execute_with_timeout_for_op(
|
||||
"write_metadata",
|
||||
move || async move {
|
||||
let disk = self.disk_ref().await;
|
||||
let (mut client, mut policy) = self.get_client_with_msgpack_policy().await?;
|
||||
let mut retried_dual_write = false;
|
||||
let response = loop {
|
||||
let file_info = compat_json(&fi, &policy)?;
|
||||
let mut request = Request::new(WriteMetadataRequest {
|
||||
bucket_incarnation_id: crate::store::bucket_heal_scope(volume)
|
||||
.map(|scope| scope.incarnation.as_bytes().to_vec().into())
|
||||
.unwrap_or_default(),
|
||||
disk: disk.clone(),
|
||||
volume: volume.to_string(),
|
||||
path: path.to_string(),
|
||||
file_info,
|
||||
file_info_bin: file_info_bin.clone(),
|
||||
});
|
||||
let canonical_body = rustfs_protos::canonical_write_metadata_request_body(request.get_ref());
|
||||
policy.guard_request(&mut request);
|
||||
let result = if request.get_ref().bucket_incarnation_id.is_empty() {
|
||||
attach_mutation_body_digest(&mut request, canonical_body, "write_metadata")?;
|
||||
client.write_metadata(request).await
|
||||
} else {
|
||||
let body =
|
||||
canonical_body.map_err(|_| Error::other("write metadata request length cannot be represented"))?;
|
||||
crate::cluster::rpc::set_tonic_canonical_body_digest(&mut request, &body).map_err(Error::other)?;
|
||||
client.write_metadata_at_incarnation(request).await
|
||||
};
|
||||
match result {
|
||||
Ok(response) => break response.into_inner(),
|
||||
Err(_) if !retried_dual_write && policy.retry_dual_write_after_local_rejection() => {
|
||||
retried_dual_write = true;
|
||||
}
|
||||
Err(status) => return Err(status.into()),
|
||||
}
|
||||
};
|
||||
let mut client = self.get_client().await?;
|
||||
let mut request = Request::new(WriteMetadataRequest {
|
||||
bucket_incarnation_id: crate::store::bucket_heal_scope(volume)
|
||||
.map(|scope| scope.incarnation.as_bytes().to_vec().into())
|
||||
.unwrap_or_default(),
|
||||
disk,
|
||||
volume: volume.to_string(),
|
||||
path: path.to_string(),
|
||||
file_info,
|
||||
file_info_bin: file_info_bin.into(),
|
||||
});
|
||||
let canonical_body = rustfs_protos::canonical_write_metadata_request_body(request.get_ref());
|
||||
let response = if request.get_ref().bucket_incarnation_id.is_empty() {
|
||||
attach_mutation_body_digest(&mut request, canonical_body, "write_metadata")?;
|
||||
client.write_metadata(request).await?
|
||||
} else {
|
||||
let body = canonical_body.map_err(|_| Error::other("write metadata request length cannot be represented"))?;
|
||||
crate::cluster::rpc::set_tonic_canonical_body_digest(&mut request, &body).map_err(Error::other)?;
|
||||
client.write_metadata_at_incarnation(request).await?
|
||||
}
|
||||
.into_inner();
|
||||
|
||||
if !response.success {
|
||||
return Err(response.error.unwrap_or_default().into());
|
||||
@@ -3865,38 +3837,29 @@ impl DiskAPI for RemoteDisk {
|
||||
state = "started",
|
||||
"Remote disk RPC started"
|
||||
);
|
||||
let file_info_bin = Bytes::from(encode_file_info_msgpack(&fi)?);
|
||||
let opts_bin = Bytes::from(encode_msgpack(opts)?);
|
||||
let file_info = compat_json(&fi)?;
|
||||
let opts_str = compat_json(&opts)?;
|
||||
let file_info_bin = encode_file_info_msgpack(&fi)?;
|
||||
let opts_bin = encode_msgpack(opts)?;
|
||||
|
||||
self.execute_with_timeout_for_op(
|
||||
"update_metadata",
|
||||
move || async move {
|
||||
let disk = self.disk_ref().await;
|
||||
let (mut client, mut policy) = self.get_client_with_msgpack_policy().await?;
|
||||
let mut retried_dual_write = false;
|
||||
let response = loop {
|
||||
let file_info = compat_json(&fi, &policy)?;
|
||||
let opts_str = compat_json(opts, &policy)?;
|
||||
let mut request = Request::new(UpdateMetadataRequest {
|
||||
disk: disk.clone(),
|
||||
volume: volume.to_string(),
|
||||
path: path.to_string(),
|
||||
file_info,
|
||||
opts: opts_str,
|
||||
file_info_bin: file_info_bin.clone(),
|
||||
opts_bin: opts_bin.clone(),
|
||||
});
|
||||
let canonical_body = rustfs_protos::canonical_update_metadata_request_body(request.get_ref());
|
||||
policy.guard_request(&mut request);
|
||||
attach_mutation_body_digest(&mut request, canonical_body, "update_metadata")?;
|
||||
match client.update_metadata(request).await {
|
||||
Ok(response) => break response.into_inner(),
|
||||
Err(_) if !retried_dual_write && policy.retry_dual_write_after_local_rejection() => {
|
||||
retried_dual_write = true;
|
||||
}
|
||||
Err(status) => return Err(status.into()),
|
||||
}
|
||||
};
|
||||
let mut client = self.get_client().await?;
|
||||
let mut request = Request::new(UpdateMetadataRequest {
|
||||
disk,
|
||||
volume: volume.to_string(),
|
||||
path: path.to_string(),
|
||||
file_info,
|
||||
opts: opts_str,
|
||||
file_info_bin: file_info_bin.into(),
|
||||
opts_bin: opts_bin.into(),
|
||||
});
|
||||
let canonical_body = rustfs_protos::canonical_update_metadata_request_body(request.get_ref());
|
||||
attach_mutation_body_digest(&mut request, canonical_body, "update_metadata")?;
|
||||
|
||||
let response = client.update_metadata(request).await?.into_inner();
|
||||
|
||||
if !response.success {
|
||||
return Err(response.error.unwrap_or_default().into());
|
||||
@@ -3932,8 +3895,9 @@ impl DiskAPI for RemoteDisk {
|
||||
);
|
||||
let read_version_attribution_enabled = rustfs_io_metrics::get_stage_metrics_enabled();
|
||||
let encode_started = read_version_stage_timer(read_version_attribution_enabled);
|
||||
let opts_bin = Bytes::from(encode_msgpack(opts)?);
|
||||
let encoded_opts = compat_json(opts).and_then(|opts_str| encode_msgpack(opts).map(|opts_bin| (opts_str, opts_bin)));
|
||||
record_read_version_stage(INTERNODE_STAGE_READ_VERSION_REQUEST_ENCODE, encode_started);
|
||||
let (opts_str, opts_bin) = encoded_opts?;
|
||||
|
||||
// Idempotent version read: eligible for the bounded transient-network retry so a single
|
||||
// reset-by-peer during the read-after-write window does not erode the metadata read
|
||||
@@ -3942,51 +3906,43 @@ impl DiskAPI for RemoteDisk {
|
||||
self.execute_read_with_retry(
|
||||
"read_version",
|
||||
|| async {
|
||||
let opts_str = opts_str.clone();
|
||||
let opts_bin = opts_bin.clone();
|
||||
let disk = self.disk_ref().await;
|
||||
let (mut client, mut policy) = self.get_client_with_msgpack_policy().await?;
|
||||
let mut retried_dual_write = false;
|
||||
let response = loop {
|
||||
let opts_str = compat_json(opts, &policy)?;
|
||||
let request_payload_bytes = read_version_attribution_enabled.then(|| {
|
||||
disk.len()
|
||||
.saturating_add(volume.len())
|
||||
.saturating_add(path.len())
|
||||
.saturating_add(version_id.len())
|
||||
.saturating_add(opts_str.len())
|
||||
.saturating_add(opts_bin.len())
|
||||
});
|
||||
let mut request = Request::new(ReadVersionRequest {
|
||||
disk: disk.clone(),
|
||||
volume: volume.to_string(),
|
||||
path: path.to_string(),
|
||||
version_id: version_id.to_string(),
|
||||
opts: opts_str,
|
||||
opts_bin: opts_bin.clone(),
|
||||
});
|
||||
policy.guard_request(&mut request);
|
||||
crate::cluster::rpc::runtime_sources::record_remote_disk_grpc_read_version_request(opts);
|
||||
if let Some(request_payload_bytes) = request_payload_bytes {
|
||||
crate::cluster::rpc::runtime_sources::record_remote_disk_grpc_read_version_sent_bytes(
|
||||
request_payload_bytes,
|
||||
);
|
||||
let mut client = self.get_client().await?;
|
||||
let request_payload_bytes = read_version_attribution_enabled.then(|| {
|
||||
disk.len()
|
||||
.saturating_add(volume.len())
|
||||
.saturating_add(path.len())
|
||||
.saturating_add(version_id.len())
|
||||
.saturating_add(opts_str.len())
|
||||
.saturating_add(opts_bin.len())
|
||||
});
|
||||
let request = Request::new(ReadVersionRequest {
|
||||
disk,
|
||||
volume: volume.to_string(),
|
||||
path: path.to_string(),
|
||||
version_id: version_id.to_string(),
|
||||
opts: opts_str,
|
||||
opts_bin: opts_bin.into(),
|
||||
});
|
||||
|
||||
crate::cluster::rpc::runtime_sources::record_remote_disk_grpc_read_version_request(opts);
|
||||
if let Some(request_payload_bytes) = request_payload_bytes {
|
||||
crate::cluster::rpc::runtime_sources::record_remote_disk_grpc_read_version_sent_bytes(request_payload_bytes);
|
||||
}
|
||||
let rpc_started = read_version_stage_timer(read_version_attribution_enabled);
|
||||
let response = match client.read_version(request).await {
|
||||
Ok(response) => {
|
||||
record_read_version_stage(INTERNODE_STAGE_READ_VERSION_RPC_ROUNDTRIP, rpc_started);
|
||||
response.into_inner()
|
||||
}
|
||||
let rpc_started = read_version_stage_timer(read_version_attribution_enabled);
|
||||
match client.read_version(request).await {
|
||||
Ok(response) => {
|
||||
record_read_version_stage(INTERNODE_STAGE_READ_VERSION_RPC_ROUNDTRIP, rpc_started);
|
||||
break response.into_inner();
|
||||
}
|
||||
Err(_) if !retried_dual_write && policy.retry_dual_write_after_local_rejection() => {
|
||||
record_read_version_stage(INTERNODE_STAGE_READ_VERSION_RPC_ROUNDTRIP, rpc_started);
|
||||
retried_dual_write = true;
|
||||
}
|
||||
Err(err) => {
|
||||
record_read_version_stage(INTERNODE_STAGE_READ_VERSION_RPC_ROUNDTRIP, rpc_started);
|
||||
crate::cluster::rpc::runtime_sources::record_remote_disk_grpc_read_version_error(
|
||||
crate::cluster::rpc::runtime_sources::ReadVersionErrorStage::RpcStatus,
|
||||
);
|
||||
return Err(err.into());
|
||||
}
|
||||
Err(err) => {
|
||||
record_read_version_stage(INTERNODE_STAGE_READ_VERSION_RPC_ROUNDTRIP, rpc_started);
|
||||
crate::cluster::rpc::runtime_sources::record_remote_disk_grpc_read_version_error(
|
||||
crate::cluster::rpc::runtime_sources::ReadVersionErrorStage::RpcStatus,
|
||||
);
|
||||
return Err(err.into());
|
||||
}
|
||||
};
|
||||
|
||||
@@ -4048,13 +4004,14 @@ impl DiskAPI for RemoteDisk {
|
||||
state = "started",
|
||||
"Remote disk RPC started"
|
||||
);
|
||||
let req = Arc::new(req);
|
||||
let batch_read_version_attribution_enabled = rustfs_io_metrics::get_stage_metrics_enabled();
|
||||
let encode_started = read_version_stage_timer(batch_read_version_attribution_enabled);
|
||||
let batch_read_version_req_bin = Bytes::from(encode_msgpack(req.as_ref())?);
|
||||
let batch_read_version_req = compat_json(&req)?;
|
||||
let batch_read_version_req_bin = encode_msgpack(&req)?;
|
||||
record_batch_read_version_stage(INTERNODE_STAGE_BATCH_READ_VERSION_REQUEST_ENCODE, encode_started);
|
||||
let batch_rpc_req = Arc::clone(&req);
|
||||
let request_opts = batch_rpc_req.opts;
|
||||
let request_payload_bytes = batch_read_version_attribution_enabled
|
||||
.then(|| batch_read_version_request_payload_len(&req, &batch_read_version_req, &batch_read_version_req_bin));
|
||||
let request_opts = req.opts;
|
||||
let bulk_body = self.acquire_bulk_body().await?;
|
||||
let batch_result = self
|
||||
.execute_with_timeout_for_op(
|
||||
@@ -4062,61 +4019,31 @@ impl DiskAPI for RemoteDisk {
|
||||
move || async move {
|
||||
let disk = self.disk_ref().await;
|
||||
let disk_len = disk.len();
|
||||
let (mut client, mut policy) = self.get_bulk_client_with_msgpack_policy().await?;
|
||||
let mut retried_dual_write = false;
|
||||
let response = loop {
|
||||
let batch_read_version_req = compat_json(batch_rpc_req.as_ref(), &policy)?;
|
||||
let request_payload_bytes = batch_read_version_attribution_enabled.then(|| {
|
||||
batch_read_version_request_payload_len(
|
||||
batch_rpc_req.as_ref(),
|
||||
&batch_read_version_req,
|
||||
&batch_read_version_req_bin,
|
||||
)
|
||||
});
|
||||
let mut request = Request::new(BatchReadVersionRequest {
|
||||
disk: disk.clone(),
|
||||
batch_read_version_req,
|
||||
batch_read_version_req_bin: batch_read_version_req_bin.clone(),
|
||||
});
|
||||
policy.guard_request(&mut request);
|
||||
crate::cluster::rpc::runtime_sources::record_remote_disk_grpc_batch_read_version_request(&request_opts);
|
||||
if let Some(request_payload_bytes) = request_payload_bytes {
|
||||
crate::cluster::rpc::runtime_sources::record_remote_disk_grpc_batch_read_version_sent_bytes(
|
||||
request_payload_bytes.saturating_add(disk_len),
|
||||
);
|
||||
}
|
||||
let rpc_started = read_version_stage_timer(batch_read_version_attribution_enabled);
|
||||
let response_result =
|
||||
measure_client_rpc(INTERNODE_OPERATION_GRPC_BATCH_READ_VERSION, client.batch_read_version(request))
|
||||
.await;
|
||||
match response_result {
|
||||
Ok(response) => {
|
||||
record_batch_read_version_stage(INTERNODE_STAGE_BATCH_READ_VERSION_RPC_ROUNDTRIP, rpc_started);
|
||||
break response.into_inner();
|
||||
}
|
||||
Err(_) if !retried_dual_write && policy.retry_dual_write_after_local_rejection() => {
|
||||
record_batch_read_version_stage(INTERNODE_STAGE_BATCH_READ_VERSION_RPC_ROUNDTRIP, rpc_started);
|
||||
retried_dual_write = true;
|
||||
}
|
||||
Err(status) if status.code() == Code::Unimplemented => {
|
||||
record_batch_read_version_stage(INTERNODE_STAGE_BATCH_READ_VERSION_RPC_ROUNDTRIP, rpc_started);
|
||||
if mode.should_fallback_on_unimplemented() {
|
||||
record_batch_read_version_gate_decision(mode, BATCH_READ_VERSION_GATE_FALLBACK_UNIMPLEMENTED);
|
||||
warn!(
|
||||
event = EVENT_REMOTE_DISK_RPC,
|
||||
component = LOG_COMPONENT_ECSTORE,
|
||||
subsystem = LOG_SUBSYSTEM_REMOTE_DISK,
|
||||
endpoint = %self.endpoint,
|
||||
batch_metadata_rpc_mode = mode.as_str(),
|
||||
op = "batch_read_version",
|
||||
state = "fallback_unimplemented",
|
||||
"Remote disk BatchReadVersion unsupported; falling back to unary read_version"
|
||||
);
|
||||
return Ok(None);
|
||||
}
|
||||
let mut client = self.get_bulk_client().await?;
|
||||
let request = Request::new(BatchReadVersionRequest {
|
||||
disk,
|
||||
batch_read_version_req,
|
||||
batch_read_version_req_bin: batch_read_version_req_bin.into(),
|
||||
});
|
||||
|
||||
record_batch_read_version_gate_decision(mode, BATCH_READ_VERSION_GATE_UNSUPPORTED_NO_FALLBACK);
|
||||
crate::cluster::rpc::runtime_sources::record_remote_disk_grpc_batch_read_version_error();
|
||||
crate::cluster::rpc::runtime_sources::record_remote_disk_grpc_batch_read_version_request(&request_opts);
|
||||
if let Some(request_payload_bytes) = request_payload_bytes {
|
||||
crate::cluster::rpc::runtime_sources::record_remote_disk_grpc_batch_read_version_sent_bytes(
|
||||
request_payload_bytes.saturating_add(disk_len),
|
||||
);
|
||||
}
|
||||
let rpc_started = read_version_stage_timer(batch_read_version_attribution_enabled);
|
||||
let response_result =
|
||||
measure_client_rpc(INTERNODE_OPERATION_GRPC_BATCH_READ_VERSION, client.batch_read_version(request)).await;
|
||||
let response = match response_result {
|
||||
Ok(response) => {
|
||||
record_batch_read_version_stage(INTERNODE_STAGE_BATCH_READ_VERSION_RPC_ROUNDTRIP, rpc_started);
|
||||
response.into_inner()
|
||||
}
|
||||
Err(status) if status.code() == Code::Unimplemented => {
|
||||
record_batch_read_version_stage(INTERNODE_STAGE_BATCH_READ_VERSION_RPC_ROUNDTRIP, rpc_started);
|
||||
if mode.should_fallback_on_unimplemented() {
|
||||
record_batch_read_version_gate_decision(mode, BATCH_READ_VERSION_GATE_FALLBACK_UNIMPLEMENTED);
|
||||
warn!(
|
||||
event = EVENT_REMOTE_DISK_RPC,
|
||||
component = LOG_COMPONENT_ECSTORE,
|
||||
@@ -4124,16 +4051,30 @@ impl DiskAPI for RemoteDisk {
|
||||
endpoint = %self.endpoint,
|
||||
batch_metadata_rpc_mode = mode.as_str(),
|
||||
op = "batch_read_version",
|
||||
state = "unsupported_no_fallback",
|
||||
"Remote disk BatchReadVersion unsupported and explicit batch RPC mode forbids fallback"
|
||||
state = "fallback_unimplemented",
|
||||
"Remote disk BatchReadVersion unsupported; falling back to unary read_version"
|
||||
);
|
||||
return Err(Error::from(status));
|
||||
}
|
||||
Err(status) => {
|
||||
record_batch_read_version_stage(INTERNODE_STAGE_BATCH_READ_VERSION_RPC_ROUNDTRIP, rpc_started);
|
||||
crate::cluster::rpc::runtime_sources::record_remote_disk_grpc_batch_read_version_error();
|
||||
return Err(Error::from(status));
|
||||
return Ok(None);
|
||||
}
|
||||
|
||||
record_batch_read_version_gate_decision(mode, BATCH_READ_VERSION_GATE_UNSUPPORTED_NO_FALLBACK);
|
||||
crate::cluster::rpc::runtime_sources::record_remote_disk_grpc_batch_read_version_error();
|
||||
warn!(
|
||||
event = EVENT_REMOTE_DISK_RPC,
|
||||
component = LOG_COMPONENT_ECSTORE,
|
||||
subsystem = LOG_SUBSYSTEM_REMOTE_DISK,
|
||||
endpoint = %self.endpoint,
|
||||
batch_metadata_rpc_mode = mode.as_str(),
|
||||
op = "batch_read_version",
|
||||
state = "unsupported_no_fallback",
|
||||
"Remote disk BatchReadVersion unsupported and explicit batch RPC mode forbids fallback"
|
||||
);
|
||||
return Err(Error::from(status));
|
||||
}
|
||||
Err(status) => {
|
||||
record_batch_read_version_stage(INTERNODE_STAGE_BATCH_READ_VERSION_RPC_ROUNDTRIP, rpc_started);
|
||||
crate::cluster::rpc::runtime_sources::record_remote_disk_grpc_batch_read_version_error();
|
||||
return Err(Error::from(status));
|
||||
}
|
||||
};
|
||||
|
||||
@@ -4169,7 +4110,6 @@ impl DiskAPI for RemoteDisk {
|
||||
// instead of racing the whole batch against one drive timeout.
|
||||
None => {
|
||||
drop(bulk_body);
|
||||
let req = Arc::try_unwrap(req).unwrap_or_else(|req| (*req).clone());
|
||||
batch_read_version_one_by_one(self, req).await
|
||||
}
|
||||
}
|
||||
@@ -5116,27 +5056,18 @@ impl DiskAPI for RemoteDisk {
|
||||
let _bulk_body = self.acquire_bulk_body().await?;
|
||||
self.execute_with_timeout(
|
||||
|| async {
|
||||
let read_multiple_req_bin = Bytes::from(encode_msgpack(&req)?);
|
||||
let read_multiple_req = compat_json(&req)?;
|
||||
let read_multiple_req_bin = encode_msgpack(&req)?;
|
||||
let disk = self.disk_ref().await;
|
||||
let (mut client, mut policy) = self.get_bulk_client_with_msgpack_policy().await?;
|
||||
let mut retried_dual_write = false;
|
||||
let response = loop {
|
||||
let read_multiple_req = compat_json(&req, &policy)?;
|
||||
let mut request = Request::new(ReadMultipleRequest {
|
||||
disk: disk.clone(),
|
||||
read_multiple_req,
|
||||
read_multiple_req_bin: read_multiple_req_bin.clone(),
|
||||
});
|
||||
policy.guard_request(&mut request);
|
||||
let result = measure_client_rpc(INTERNODE_OPERATION_GRPC_READ_MULTIPLE, client.read_multiple(request)).await;
|
||||
match result {
|
||||
Ok(response) => break response.into_inner(),
|
||||
Err(_) if !retried_dual_write && policy.retry_dual_write_after_local_rejection() => {
|
||||
retried_dual_write = true;
|
||||
}
|
||||
Err(status) => return Err(status.into()),
|
||||
}
|
||||
};
|
||||
let mut client = self.get_bulk_client().await?;
|
||||
let request = Request::new(ReadMultipleRequest {
|
||||
disk,
|
||||
read_multiple_req,
|
||||
read_multiple_req_bin: read_multiple_req_bin.into(),
|
||||
});
|
||||
|
||||
let response = measure_client_rpc(INTERNODE_OPERATION_GRPC_READ_MULTIPLE, client.read_multiple(request)).await;
|
||||
let response = response?.into_inner();
|
||||
|
||||
if !response.success {
|
||||
return Err(response.error.unwrap_or_default().into());
|
||||
@@ -6323,128 +6254,6 @@ mod tests {
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn delete_versions_request_encoding_preserves_batches_and_fleet_guards() {
|
||||
use tonic_prost::prost::Message;
|
||||
let groups: Vec<_> = ["first", "second"]
|
||||
.into_iter()
|
||||
.map(|name| FileInfoVersions {
|
||||
volume: "bucket".to_owned(),
|
||||
name: name.to_owned(),
|
||||
versions: vec![sample_rename_data_file_info()],
|
||||
..Default::default()
|
||||
})
|
||||
.collect();
|
||||
let opts = DeleteOptions {
|
||||
undo_write: true,
|
||||
old_data_dir: Some(Uuid::from_u128(7)),
|
||||
..Default::default()
|
||||
};
|
||||
let mut binary_wire_size = 0;
|
||||
let mut compat_wire_size = 0;
|
||||
for (requested, confirmed, peer_capability, binary_only) in [
|
||||
(None, None, None, false),
|
||||
(Some("true"), None, None, false),
|
||||
(Some("false"), Some("true"), None, false),
|
||||
(Some("true"), Some("false"), None, false),
|
||||
(Some("true"), None, Some("true"), false),
|
||||
(Some("true"), Some("false"), Some("true"), false),
|
||||
(Some("true"), Some("true"), None, true),
|
||||
(Some("true"), Some("true"), Some("true"), true),
|
||||
] {
|
||||
with_internode_msgpack_env(
|
||||
[
|
||||
(rustfs_config::ENV_INTERNODE_RPC_MSGPACK_ONLY, requested),
|
||||
(rustfs_config::ENV_INTERNODE_RPC_MSGPACK_ONLY_FLEET_CONFIRMED, confirmed),
|
||||
(rustfs_config::ENV_INTERNODE_RPC_MSGPACK_PEER_CAPABILITY, peer_capability),
|
||||
],
|
||||
|| {
|
||||
for batch in [&groups[..], &[]] {
|
||||
let disk = "disk".to_owned();
|
||||
let disk_buffer = disk.as_ptr();
|
||||
let request = encode_delete_versions_request(disk, "bucket", batch, &opts).expect("encode batch");
|
||||
assert_eq!(request.disk, "disk");
|
||||
assert_eq!(request.disk.as_ptr(), disk_buffer, "move the prepared endpoint into the request");
|
||||
assert_eq!(request.volume, "bucket");
|
||||
assert_eq!(request.versions_bin.len(), batch.len());
|
||||
assert_eq!(request.opts.is_empty(), binary_only);
|
||||
let decoded_opts: DeleteOptions = rmp_serde::from_slice(&request.opts_bin).expect("binary options");
|
||||
assert!(decoded_opts.undo_write);
|
||||
assert_eq!(decoded_opts.old_data_dir, opts.old_data_dir);
|
||||
for (encoded, expected) in request.versions_bin.iter().zip(batch) {
|
||||
let decoded: FileInfoVersions = rmp_serde::from_slice(encoded).expect("binary batch item");
|
||||
assert_eq!(decoded.name, expected.name);
|
||||
assert_eq!(decoded.versions, expected.versions);
|
||||
}
|
||||
if binary_only {
|
||||
assert!(request.versions.is_empty());
|
||||
} else {
|
||||
assert_eq!(request.opts, serde_json::to_string(&opts).expect("legacy options"));
|
||||
assert_eq!(request.versions.len(), batch.len());
|
||||
for (encoded, expected) in request.versions.iter().zip(batch) {
|
||||
let decoded: FileInfoVersions = serde_json::from_str(encoded).expect("legacy batch item");
|
||||
assert_eq!(decoded.name, expected.name);
|
||||
assert_eq!(decoded.versions, expected.versions);
|
||||
}
|
||||
}
|
||||
let canonical = rustfs_protos::canonical_delete_versions_request_body(&request).expect("bound batch");
|
||||
let mut modified = request.clone();
|
||||
modified.opts_bin = encode_delete_options_msgpack(&DeleteOptions::default())
|
||||
.expect("other options")
|
||||
.into();
|
||||
assert_ne!(
|
||||
canonical,
|
||||
rustfs_protos::canonical_delete_versions_request_body(&modified).expect("other bound batch")
|
||||
);
|
||||
if !batch.is_empty() {
|
||||
if binary_only {
|
||||
binary_wire_size = request.encoded_len();
|
||||
} else {
|
||||
compat_wire_size = request.encoded_len();
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
);
|
||||
}
|
||||
assert!(binary_wire_size > 0 && binary_wire_size < compat_wire_size);
|
||||
println!("delete batch encoded bytes compat={compat_wire_size} msgpack_only={binary_wire_size}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn msgpack_eligible_requests_bind_one_policy_and_bounded_local_retry() {
|
||||
let source = rustfs_protos::compat_manifest::production_source(include_str!("remote_disk.rs"), "remote_disk.rs");
|
||||
for function in [
|
||||
"rename_data_borrowed_with_fence",
|
||||
"write_metadata",
|
||||
"update_metadata",
|
||||
"read_version",
|
||||
"batch_read_version",
|
||||
"read_multiple",
|
||||
] {
|
||||
let start = source
|
||||
.find(&format!("async fn {function}("))
|
||||
.unwrap_or_else(|| panic!("missing {function}"));
|
||||
let tail = &source[start..];
|
||||
let next_instrument = ["\n #[tracing::", "instrument"].concat();
|
||||
let end = tail[1..].find(&next_instrument).map_or(tail.len(), |offset| offset + 1);
|
||||
let body = &tail[..end];
|
||||
assert!(body.contains("with_msgpack_policy().await"), "{function} must capture one RPC policy");
|
||||
assert!(
|
||||
body.contains("policy.guard_request(&mut request)"),
|
||||
"{function} must bind the generation guard"
|
||||
);
|
||||
assert!(
|
||||
body.contains("policy.retry_dual_write_after_local_rejection()"),
|
||||
"{function} must allow only the bounded local dual-write retry"
|
||||
);
|
||||
assert!(
|
||||
body.contains("let mut retried_dual_write = false;"),
|
||||
"{function} must bound retry to one attempt"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn delete_options_named_wire_preserves_rollback_and_legacy_marker_fields() {
|
||||
#[derive(Debug, serde::Serialize, serde::Deserialize, PartialEq)]
|
||||
@@ -7353,18 +7162,14 @@ mod tests {
|
||||
],
|
||||
|| {
|
||||
let resp = sample_read_multiple_resp("file", b"data");
|
||||
let policy = MsgpackSendPolicy::for_test(rustfs_protos::internode_rpc_msgpack_only());
|
||||
let json = compat_json(&resp, &policy).expect("compat_json should encode");
|
||||
let json = compat_json(&resp).expect("compat_json should encode");
|
||||
assert!(!json.is_empty());
|
||||
assert_eq!(json, serde_json::to_string(&resp).expect("json should encode"));
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
fn with_internode_msgpack_env<R, const N: usize>(
|
||||
vars: [(&'static str, Option<&'static str>); N],
|
||||
f: impl FnOnce() -> R,
|
||||
) -> R {
|
||||
fn with_internode_msgpack_env<R>(vars: [(&'static str, Option<&'static str>); 2], f: impl FnOnce() -> R) -> R {
|
||||
temp_env::with_vars(vars, || {
|
||||
rustfs_protos::reset_internode_rpc_msgpack_only_cache();
|
||||
let result = f();
|
||||
@@ -7382,8 +7187,7 @@ mod tests {
|
||||
],
|
||||
|| {
|
||||
let resp = sample_read_multiple_resp("file", b"data");
|
||||
let policy = MsgpackSendPolicy::for_test(rustfs_protos::internode_rpc_msgpack_only());
|
||||
let json = compat_json(&resp, &policy).expect("compat_json should encode");
|
||||
let json = compat_json(&resp).expect("compat_json should encode");
|
||||
|
||||
assert!(!json.is_empty(), "old JSON peers must remain compatible without fleet confirmation");
|
||||
assert_eq!(json, serde_json::to_string(&resp).expect("json should encode"));
|
||||
@@ -7400,8 +7204,7 @@ mod tests {
|
||||
],
|
||||
|| {
|
||||
let resp = sample_read_multiple_resp("file", b"data");
|
||||
let policy = MsgpackSendPolicy::for_test(rustfs_protos::internode_rpc_msgpack_only());
|
||||
let json = compat_json(&resp, &policy).expect("compat_json should encode");
|
||||
let json = compat_json(&resp).expect("compat_json should encode");
|
||||
|
||||
assert!(json.is_empty(), "msgpack-only may empty JSON only after explicit fleet confirmation");
|
||||
},
|
||||
@@ -7417,8 +7220,7 @@ mod tests {
|
||||
],
|
||||
|| {
|
||||
let resp = sample_read_multiple_resp("file", b"data");
|
||||
let policy = MsgpackSendPolicy::for_test(rustfs_protos::internode_rpc_msgpack_only());
|
||||
let json = compat_json(&resp, &policy).expect("compat_json should encode");
|
||||
let json = compat_json(&resp).expect("compat_json should encode");
|
||||
|
||||
assert!(json.is_empty(), "both gates should enter msgpack-only send mode");
|
||||
},
|
||||
@@ -7436,8 +7238,7 @@ mod tests {
|
||||
] {
|
||||
with_internode_msgpack_env(vars, || {
|
||||
let resp = sample_read_multiple_resp("file", b"data");
|
||||
let policy = MsgpackSendPolicy::for_test(rustfs_protos::internode_rpc_msgpack_only());
|
||||
let json = compat_json(&resp, &policy).expect("compat_json should encode");
|
||||
let json = compat_json(&resp).expect("compat_json should encode");
|
||||
|
||||
assert!(!json.is_empty(), "removing either gate should restore old-peer JSON compatibility");
|
||||
assert_eq!(json, serde_json::to_string(&resp).expect("json should encode"));
|
||||
|
||||
@@ -20,8 +20,8 @@ use async_trait::async_trait;
|
||||
use bytes::Bytes;
|
||||
use futures::{FutureExt, future::BoxFuture};
|
||||
use rustfs_lock::{
|
||||
BatchAcquireOutcome, CacheFenceStamp, LockCleanupOutcome, LockClient, LockError, LockInfo, LockRequest, LockResponse,
|
||||
LockStats, LockStatus, LockType, Result,
|
||||
BatchAcquireOutcome, LockCleanupOutcome, LockClient, LockError, LockInfo, LockRequest, LockResponse, LockStats, LockStatus,
|
||||
LockType, Result,
|
||||
types::{LockId, LockMetadata, LockPriority},
|
||||
};
|
||||
use rustfs_protos::proto_gen::node_service::{
|
||||
@@ -52,10 +52,6 @@ fn attach_lock_mutation_body_digest<T: rustfs_protos::CanonicalMutationBody>(req
|
||||
set_tonic_rolling_mutation_body_digest(request)
|
||||
}
|
||||
|
||||
fn decode_cache_fence_stamp(encoded: Option<&Bytes>) -> Option<CacheFenceStamp> {
|
||||
encoded.and_then(|encoded| CacheFenceStamp::decode(encoded).ok())
|
||||
}
|
||||
|
||||
/// Work to run if an RPC completes after its caller stops waiting.
|
||||
type LateCompletion<T> = Option<Box<dyn FnOnce(T) -> Pin<Box<dyn Future<Output = ()> + Send>> + Send>>;
|
||||
type LateFailure = LateCompletion<()>;
|
||||
@@ -328,7 +324,6 @@ fn batch_acquire_outcome(lock_id: &LockId, lock_type: LockType, result: Option<&
|
||||
return BatchAcquireOutcome::Granted {
|
||||
lock_info,
|
||||
wait_time: Duration::ZERO,
|
||||
cache_fence_stamp: decode_cache_fence_stamp(result.cache_fence_stamp.as_ref()),
|
||||
};
|
||||
}
|
||||
if result.lock_info.is_some() {
|
||||
@@ -457,7 +452,6 @@ impl RemoteClient {
|
||||
deadlock_detection: false,
|
||||
suppress_contention_logs: false,
|
||||
refresh_interval: None,
|
||||
cache_fence_stamp_requested: false,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1579,11 +1573,10 @@ impl LockClient for RemoteClient {
|
||||
|
||||
// Check if the lock acquisition was successful
|
||||
if resp.success {
|
||||
let lock_info = Self::build_lock_info(request, resp.lock_info);
|
||||
match decode_cache_fence_stamp(resp.cache_fence_stamp.as_ref()) {
|
||||
Some(stamp) => Ok(LockResponse::success_with_cache_fence_stamp(lock_info, std::time::Duration::ZERO, stamp)),
|
||||
None => Ok(LockResponse::success(lock_info, std::time::Duration::ZERO)),
|
||||
}
|
||||
Ok(LockResponse::success(
|
||||
Self::build_lock_info(request, resp.lock_info),
|
||||
std::time::Duration::ZERO,
|
||||
))
|
||||
} else {
|
||||
// Lock acquisition failed
|
||||
Ok(LockResponse::failure(
|
||||
@@ -1601,19 +1594,12 @@ impl LockClient for RemoteClient {
|
||||
.iter()
|
||||
.zip(outcomes)
|
||||
.map(|(request, outcome)| match outcome {
|
||||
BatchAcquireOutcome::Granted {
|
||||
lock_info,
|
||||
wait_time,
|
||||
cache_fence_stamp,
|
||||
} => {
|
||||
let lock_info = lock_info
|
||||
BatchAcquireOutcome::Granted { lock_info, wait_time } => LockResponse::success(
|
||||
lock_info
|
||||
.map(|info| *info)
|
||||
.unwrap_or_else(|| Self::build_lock_info(request, None));
|
||||
match cache_fence_stamp {
|
||||
Some(stamp) => LockResponse::success_with_cache_fence_stamp(lock_info, wait_time, stamp),
|
||||
None => LockResponse::success(lock_info, wait_time),
|
||||
}
|
||||
}
|
||||
.unwrap_or_else(|| Self::build_lock_info(request, None)),
|
||||
wait_time,
|
||||
),
|
||||
BatchAcquireOutcome::Ambiguous {
|
||||
error: error @ LockError::Timeout { .. },
|
||||
..
|
||||
@@ -2381,38 +2367,6 @@ mod tests {
|
||||
assert!(matches!(classify(None), BatchAcquireOutcome::Ambiguous { .. }));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn cache_fence_stamp_decoder_accepts_current_wire_shape_and_rejects_malformed_values() {
|
||||
let mut encoded = Vec::with_capacity(27);
|
||||
encoded.push(1);
|
||||
encoded.extend_from_slice(uuid::Uuid::from_u128(7).as_bytes());
|
||||
encoded.extend_from_slice(&17_u16.to_le_bytes());
|
||||
encoded.extend_from_slice(&42_u64.to_le_bytes());
|
||||
let encoded = Bytes::from(encoded);
|
||||
|
||||
let stamp = decode_cache_fence_stamp(Some(&encoded)).expect("current stamp wire shape");
|
||||
assert_eq!(stamp.voter_boot_id(), uuid::Uuid::from_u128(7));
|
||||
assert_eq!(stamp.shard_index(), 17);
|
||||
assert_eq!(stamp.generation(), 42);
|
||||
assert!(decode_cache_fence_stamp(None).is_none(), "legacy responses remain valid without proof");
|
||||
assert!(decode_cache_fence_stamp(Some(&Bytes::from_static(b"malformed"))).is_none());
|
||||
|
||||
let request = test_lock_request(Duration::from_secs(30));
|
||||
let result = GenerallyLockResult {
|
||||
success: true,
|
||||
cache_fence_stamp: Some(encoded),
|
||||
..Default::default()
|
||||
};
|
||||
let BatchAcquireOutcome::Granted {
|
||||
cache_fence_stamp: Some(batch_stamp),
|
||||
..
|
||||
} = batch_acquire_outcome(&request.lock_id, request.lock_type, Some(&result))
|
||||
else {
|
||||
panic!("valid batch grant must retain its cache-fence stamp");
|
||||
};
|
||||
assert_eq!(batch_stamp, stamp);
|
||||
}
|
||||
|
||||
#[tokio::test(start_paused = true)]
|
||||
async fn batch_success_crossing_deadline_in_one_poll_runs_late_cleanup_once() {
|
||||
use crate::cluster::rpc::client::NoOpInterceptor;
|
||||
|
||||
@@ -101,6 +101,7 @@ const DEFAULT_RRS_STORAGE_CLASS: &str = "EC:1";
|
||||
const ZERO_SET_DRIVE_COUNT_ERROR: &str = "set drive count must be greater than zero";
|
||||
|
||||
pub static DEFAULT_INLINE_BLOCK: usize = 128 * 1024;
|
||||
const DEFAULT_INLINE_OBJECT_BUDGET: usize = 2 * DEFAULT_INLINE_BLOCK;
|
||||
|
||||
pub static DEFAULT_KVS: LazyLock<KVS> = LazyLock::new(|| {
|
||||
let kvs = vec![
|
||||
@@ -151,7 +152,6 @@ pub struct Config {
|
||||
inline_block: usize,
|
||||
initialized: bool,
|
||||
#[serde(default, skip_serializing_if = "std::ops::Not::not")]
|
||||
// Preserve the explicit-limit marker when configurations round-trip through older readers.
|
||||
inline_block_explicit: bool,
|
||||
#[serde(skip)]
|
||||
standard_parities: Vec<PoolParity>,
|
||||
@@ -245,10 +245,8 @@ impl Config {
|
||||
return false;
|
||||
}
|
||||
|
||||
let Ok(shard_size) = usize::try_from(shard_size) else {
|
||||
return false;
|
||||
};
|
||||
let inline_block = self.inline_block();
|
||||
let shard_size = shard_size as usize;
|
||||
let inline_block = self.effective_inline_block(data_shards);
|
||||
|
||||
if versioned {
|
||||
shard_size <= inline_block / 8
|
||||
@@ -257,7 +255,27 @@ impl Config {
|
||||
}
|
||||
}
|
||||
|
||||
/// Per-shard inline data budget, defaulting to 128 KiB across EC layouts.
|
||||
/// Returns the per-shard inline budget used by both write admission and
|
||||
/// legacy read fallback.
|
||||
///
|
||||
/// The default budget is scaled by the number of data shards so a wider EC
|
||||
/// layout does not silently increase the maximum inline object size. An
|
||||
/// explicitly configured `inline_block` remains a fixed per-shard limit for
|
||||
/// compatibility with deployments that opted into the historical policy.
|
||||
pub(crate) fn effective_inline_block(&self, data_shards: usize) -> usize {
|
||||
if data_shards == 0 {
|
||||
return 0;
|
||||
}
|
||||
|
||||
if self.initialized && self.inline_block_explicit {
|
||||
self.inline_block
|
||||
} else {
|
||||
// Keep the historical two-data-shard object budget while preventing
|
||||
// wider EC layouts from multiplying the maximum inline object size.
|
||||
DEFAULT_INLINE_OBJECT_BUDGET.div_ceil(data_shards).min(DEFAULT_INLINE_BLOCK)
|
||||
}
|
||||
}
|
||||
|
||||
pub fn inline_block(&self) -> usize {
|
||||
if !self.initialized {
|
||||
DEFAULT_INLINE_BLOCK
|
||||
@@ -549,7 +567,7 @@ mod tests {
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn should_inline_uses_fixed_default_threshold_for_every_ec_layout() {
|
||||
fn should_inline_scales_default_threshold_by_data_shards() {
|
||||
let config = lookup_config_for_pools_with_env(&KVS::new(), &[3, 12], no_env_overrides())
|
||||
.expect("default inline policy should resolve for EC2+1 and EC8+4");
|
||||
|
||||
@@ -558,14 +576,10 @@ mod tests {
|
||||
("EC2+1 unversioned above", 128 * 1024 + 1, 2, false, false),
|
||||
("EC2+1 versioned exact", 16 * 1024, 2, true, true),
|
||||
("EC2+1 versioned above", 16 * 1024 + 1, 2, true, false),
|
||||
("EC8+4 unversioned exact", 128 * 1024, 8, false, true),
|
||||
("EC8+4 unversioned above", 128 * 1024 + 1, 8, false, false),
|
||||
("EC8+4 versioned exact", 16 * 1024, 8, true, true),
|
||||
("EC8+4 versioned above", 16 * 1024 + 1, 8, true, false),
|
||||
("EC12+4 unversioned exact", 128 * 1024, 12, false, true),
|
||||
("EC12+4 unversioned above", 128 * 1024 + 1, 12, false, false),
|
||||
("EC12+4 versioned exact", 16 * 1024, 12, true, true),
|
||||
("EC12+4 versioned above", 16 * 1024 + 1, 12, true, false),
|
||||
("EC8+4 unversioned exact", 32 * 1024, 8, false, true),
|
||||
("EC8+4 unversioned above", 32 * 1024 + 1, 8, false, false),
|
||||
("EC8+4 versioned exact", 4 * 1024, 8, true, true),
|
||||
("EC8+4 versioned above", 4 * 1024 + 1, 8, true, false),
|
||||
("negative", -1, 2, false, false),
|
||||
("zero data shards", 0, 0, false, false),
|
||||
] {
|
||||
@@ -612,35 +626,37 @@ mod tests {
|
||||
let shard_size = erasure.shard_file_size(object_size);
|
||||
let inline = config.should_inline(shard_size, data_shards, false);
|
||||
|
||||
// The per-shard budget stays monotonic across object sizes.
|
||||
// PUT separately bounds inline body materialization.
|
||||
// The effective policy is monotonic across object sizes. This
|
||||
// table covers the boundaries that previously exposed the
|
||||
// fixed-shard read-ahead mismatch, including the 1 MiB case.
|
||||
assert!(!inline || previous, "inline decision must not re-enable at {object_size} bytes");
|
||||
previous = inline;
|
||||
}
|
||||
|
||||
assert!(
|
||||
config.should_inline(erasure.shard_file_size(1024 * 1024), data_shards, false),
|
||||
"1 MiB fits the per-shard budget for EC{data_shards}+{parity_shards}"
|
||||
!config.should_inline(erasure.shard_file_size(1024 * 1024), data_shards, false),
|
||||
"1 MiB must use the non-inline path for EC{data_shards}+{parity_shards}"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn inline_block_preserves_fixed_default_and_explicit_limits() {
|
||||
fn effective_inline_block_scales_default_budget_and_preserves_explicit_limit() {
|
||||
let config = Config::default();
|
||||
assert_eq!(config.inline_block(), 128 * 1024);
|
||||
assert!(!config.should_inline(0, 0, false));
|
||||
assert_eq!(config.effective_inline_block(8), 32 * 1024);
|
||||
assert_eq!(config.effective_inline_block(12), 21_846);
|
||||
assert_eq!(config.effective_inline_block(0), 0);
|
||||
|
||||
let explicit = lookup_config_for_pools_with_env(
|
||||
&KVS::new(),
|
||||
&[12],
|
||||
StorageClassEnvOverrides {
|
||||
inline_block: Some("32KiB".to_string()),
|
||||
inline_block: Some("128KiB".to_string()),
|
||||
..Default::default()
|
||||
},
|
||||
)
|
||||
.expect("explicit inline block should resolve");
|
||||
assert_eq!(explicit.inline_block(), 32 * 1024);
|
||||
assert_eq!(explicit.effective_inline_block(12), 128 * 1024);
|
||||
}
|
||||
|
||||
#[test]
|
||||
@@ -889,21 +905,6 @@ mod tests {
|
||||
assert!(decoded.should_inline(100 * 1024, 8, false));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn legacy_scalar_inline_config_preserves_limit_without_explicit_marker() {
|
||||
let cfg = lookup_config_for_pools_with_env(&KVS::new(), &[12], no_env_overrides())
|
||||
.expect("default storage class should resolve");
|
||||
let mut legacy = serde_json::to_value(cfg).expect("serialize scalar config");
|
||||
legacy["inline_block"] = serde_json::json!(64 * 1024);
|
||||
legacy.as_object_mut().expect("config object").remove("inline_block_explicit");
|
||||
let decoded: Config = serde_json::from_value(legacy).expect("read historical scalar config");
|
||||
|
||||
assert!(decoded.should_inline(64 * 1024, 8, false));
|
||||
assert!(!decoded.should_inline(64 * 1024 + 1, 8, false));
|
||||
assert!(decoded.should_inline(8 * 1024, 8, true));
|
||||
assert!(!decoded.should_inline(8 * 1024 + 1, 8, true));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn lookup_config_reads_rrs_from_class_rrs_key() {
|
||||
// Regression: kvs.get(RRS) used RRS="REDUCED_REDUNDANCY" instead of
|
||||
|
||||
@@ -1463,22 +1463,36 @@ async fn load_admin_data_usage_from_backend(store: Arc<ECStore>) -> Result<DataU
|
||||
let observed = load_observed_data_usage_snapshot(store.clone()).await.ok().flatten();
|
||||
let (selected, selected_is_current_format) =
|
||||
select_admin_data_usage_snapshot(authoritative, source.is_authoritative(), observed);
|
||||
let mut response = normalize_loaded_data_usage(selected, selected_is_current_format).await.0;
|
||||
Ok(normalize_loaded_data_usage(selected, selected_is_current_format).await.0)
|
||||
}
|
||||
|
||||
async fn project_admin_data_usage_for_current_buckets(
|
||||
store: Arc<ECStore>,
|
||||
mut response: DataUsageInfo,
|
||||
with_memory_overlay: bool,
|
||||
) -> DataUsageInfo {
|
||||
use futures::StreamExt as _;
|
||||
let snapshot_measured = response.is_complete_bucket_usage_snapshot() || response.is_valid_partial_snapshot();
|
||||
let overlay_authoritative = with_memory_overlay && !runtime_sources::setup_is_dist_erasure().await;
|
||||
let deadline = tokio::time::Instant::now() + ADMIN_USAGE_INCARNATION_TIMEOUT;
|
||||
let mut incarnations = HashMap::new();
|
||||
{
|
||||
// Display-only identity reads must not serialize every bucket behind a
|
||||
// slow metadata lock while holding the shared admin refresh permit.
|
||||
// The common deadline bounds their total added latency; dropping this
|
||||
// stream cancels all unfinished reads without spawning detached work.
|
||||
// Own the names and store across awaits so this future remains Send
|
||||
// when composed into spawned metrics collectors.
|
||||
let buckets = response
|
||||
.usage_snapshot_bucket_observations
|
||||
// Identity reads happen after releasing snapshot and refresh locks,
|
||||
// including on cache hits. Metadata transaction reads linearize each
|
||||
// bucket against creation/deletion without relying on peer notices.
|
||||
// The common deadline cancels unfinished work and bounds added latency.
|
||||
let mut buckets = response
|
||||
.buckets_usage
|
||||
.keys()
|
||||
.chain(response.usage_snapshot_bucket_observations.keys())
|
||||
.cloned()
|
||||
.collect::<Vec<_>>();
|
||||
.collect::<HashSet<_>>();
|
||||
if overlay_authoritative {
|
||||
// Copy only names while holding the memory lock, then release it
|
||||
// before storage I/O. A later new entry has no identity proof and
|
||||
// will remain unknown in this response.
|
||||
buckets.extend(memory_cache().read().await.keys().cloned());
|
||||
}
|
||||
let mut identities = futures::stream::iter(buckets)
|
||||
.map(move |bucket| {
|
||||
let store = store.clone();
|
||||
@@ -1492,7 +1506,6 @@ async fn load_admin_data_usage_from_backend(store: Arc<ECStore>) -> Result<DataU
|
||||
let Ok(Some((bucket, identity))) = tokio::time::timeout_at(deadline, identities.next()).await else {
|
||||
break;
|
||||
};
|
||||
// Unavailable, timed-out or recreated buckets retain canonical values.
|
||||
if let Ok(incarnation) = identity
|
||||
&& !incarnation.is_nil()
|
||||
{
|
||||
@@ -1501,7 +1514,20 @@ async fn load_admin_data_usage_from_backend(store: Arc<ECStore>) -> Result<DataU
|
||||
}
|
||||
}
|
||||
response.project_bucket_observations_for_admin(&incarnations);
|
||||
Ok(response)
|
||||
if with_memory_overlay
|
||||
&& apply_bucket_usage_memory_overlay_if_authoritative(&mut response, overlay_authoritative).await
|
||||
&& snapshot_measured
|
||||
{
|
||||
response.usage_snapshot_complete = false;
|
||||
response.usage_snapshot_partial = true;
|
||||
response.usage_snapshot_converged = Some(false);
|
||||
response.usage_snapshot_admin_projection = true;
|
||||
response.replication_info.clear();
|
||||
response.tier_stats = None;
|
||||
response.unknown_tier_stats = None;
|
||||
}
|
||||
response.retain_current_bucket_usage_for_admin(&incarnations);
|
||||
response
|
||||
}
|
||||
|
||||
fn discard_incomplete_bucket_usage(data_usage_info: &mut DataUsageInfo) {
|
||||
@@ -1690,11 +1716,7 @@ async fn project_data_usage_from_backend_cached<T>(
|
||||
}
|
||||
}
|
||||
|
||||
/// Load the freshest structurally complete snapshot for authenticated admin
|
||||
/// observability. A scan raced by namespace activity may be selected here, but
|
||||
/// never by [`load_data_usage_from_backend_cached`], which remains the
|
||||
/// converged source for quota admission.
|
||||
pub async fn load_admin_data_usage_from_backend_cached(store: Arc<ECStore>) -> Result<DataUsageInfo, Error> {
|
||||
async fn load_admin_data_usage_snapshot_cached(store: Arc<ECStore>) -> Result<DataUsageInfo, Error> {
|
||||
let ttl = Duration::from_secs(DATA_USAGE_CACHE_TTL_SECS);
|
||||
|
||||
loop {
|
||||
@@ -1743,6 +1765,38 @@ pub async fn load_admin_data_usage_from_backend_cached(store: Arc<ECStore>) -> R
|
||||
}
|
||||
}
|
||||
|
||||
/// Load cached scanner statistics for authenticated admin observability.
|
||||
///
|
||||
/// Each response verifies bucket identities against current storage metadata,
|
||||
/// including cache hits. Missing, failed, or timed-out identity reads leave that
|
||||
/// bucket's usage absent (unknown). Legacy values without an incarnation stay
|
||||
/// unknown until a scanner snapshot certifies their identity. Per-bucket reads
|
||||
/// linearize independently; this is not an atomic namespace snapshot.
|
||||
/// Identity reads have a shared deadline and do not outlive cancellation. The
|
||||
/// unchanged cached baseline remains independent of quota and persistence.
|
||||
///
|
||||
/// # Errors
|
||||
/// Returns the cached snapshot loading error when no baseline can be loaded.
|
||||
pub async fn load_admin_data_usage_from_backend_cached(store: Arc<ECStore>) -> Result<DataUsageInfo, Error> {
|
||||
let response = load_admin_data_usage_snapshot_cached(store.clone()).await?;
|
||||
Ok(project_admin_data_usage_for_current_buckets(store, response, false).await)
|
||||
}
|
||||
|
||||
/// Load the admin view with the existing authoritative single-process overlay.
|
||||
///
|
||||
/// The overlay is applied before current-identity projection so it cannot
|
||||
/// reintroduce statistics from a retired or unverified bucket generation.
|
||||
/// Distributed nodes continue to use scanner statistics without local totals.
|
||||
/// Cancellation, unknown values, and per-bucket read semantics are the same as
|
||||
/// [`load_admin_data_usage_from_backend_cached`]. No overlay is persisted.
|
||||
///
|
||||
/// # Errors
|
||||
/// Returns the cached snapshot loading error when no baseline can be loaded.
|
||||
pub async fn load_admin_data_usage_with_memory_overlay(store: Arc<ECStore>) -> Result<DataUsageInfo, Error> {
|
||||
let response = load_admin_data_usage_snapshot_cached(store.clone()).await?;
|
||||
Ok(project_admin_data_usage_for_current_buckets(store, response, true).await)
|
||||
}
|
||||
|
||||
/// Invalidate the process-local persisted usage snapshot cache after a durable
|
||||
/// scanner write.
|
||||
pub async fn invalidate_data_usage_snapshot_cache() {
|
||||
@@ -2805,16 +2859,17 @@ async fn replace_bucket_usage_memory_from_info_if_generation(data_usage_info: &D
|
||||
*cache = next_cache;
|
||||
}
|
||||
|
||||
async fn apply_bucket_usage_memory_overlay_if_authoritative(data_usage_info: &mut DataUsageInfo, authoritative: bool) {
|
||||
async fn apply_bucket_usage_memory_overlay_if_authoritative(data_usage_info: &mut DataUsageInfo, authoritative: bool) -> bool {
|
||||
if !authoritative {
|
||||
return;
|
||||
return false;
|
||||
}
|
||||
|
||||
let cache = memory_cache().read().await;
|
||||
if cache.is_empty() {
|
||||
return;
|
||||
return false;
|
||||
}
|
||||
|
||||
let was_complete = data_usage_info.usage_snapshot_complete;
|
||||
let persisted_update = data_usage_info.last_update;
|
||||
let mut changed = false;
|
||||
let mut added_bucket = false;
|
||||
@@ -2851,6 +2906,7 @@ async fn apply_bucket_usage_memory_overlay_if_authoritative(data_usage_info: &mu
|
||||
data_usage_info.buckets_count = data_usage_info.buckets_usage.len() as u64;
|
||||
data_usage_info.calculate_totals();
|
||||
}
|
||||
changed || was_complete != data_usage_info.usage_snapshot_complete
|
||||
}
|
||||
|
||||
pub async fn apply_bucket_usage_memory_overlay(data_usage_info: &mut DataUsageInfo) {
|
||||
@@ -3744,6 +3800,23 @@ mod tests {
|
||||
*cache_updating().write().await = false;
|
||||
}
|
||||
|
||||
async fn bind_admin_usage_test_identities(store: &Arc<ECStore>, snapshot: &mut DataUsageInfo) {
|
||||
crate::bucket::metadata_sys::init_bucket_metadata_sys(store.clone(), Vec::new()).await;
|
||||
for (bucket, usage) in &mut snapshot.buckets_usage {
|
||||
store
|
||||
.make_bucket(bucket, &crate::storage_api_contracts::bucket::MakeBucketOptions::default())
|
||||
.await
|
||||
.expect("create a real bucket for current admin identity validation");
|
||||
usage.bucket_incarnation = Some(
|
||||
store
|
||||
.bucket_incarnation_id_from_disk(bucket)
|
||||
.await
|
||||
.expect("read the storage-owned fixture identity")
|
||||
.to_string(),
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
fn data_usage_info_for_test(bucket: &str, objects_count: u64, size: u64, last_update: SystemTime) -> DataUsageInfo {
|
||||
let mut info = DataUsageInfo {
|
||||
last_update: Some(last_update),
|
||||
@@ -4279,7 +4352,11 @@ mod tests {
|
||||
use crate::config::com::save_config;
|
||||
|
||||
let (_dirs, store) = isolated_store_over_temp_disks().await;
|
||||
let (authoritative, observed) = usage_snapshots_with_missing_observed_set();
|
||||
let (mut authoritative, mut observed) = usage_snapshots_with_missing_observed_set();
|
||||
bind_admin_usage_test_identities(&store, &mut authoritative).await;
|
||||
for (bucket, usage) in &mut observed.buckets_usage {
|
||||
usage.bucket_incarnation = authoritative.buckets_usage[bucket].bucket_incarnation.clone();
|
||||
}
|
||||
let primary_bytes = serde_json::to_vec(&authoritative).expect("encode complete usage");
|
||||
let observed_bytes = serde_json::to_vec(&observed).expect("encode partial observation");
|
||||
save_config(store.clone(), DATA_USAGE_OBJ_NAME_PATH.as_str(), primary_bytes.clone())
|
||||
@@ -4328,6 +4405,11 @@ mod tests {
|
||||
reconciled.scanner_epoch = Some(4);
|
||||
reconciled.scanner_cycle = Some(12);
|
||||
reconciled.usage_snapshot_converged = Some(true);
|
||||
reconciled
|
||||
.buckets_usage
|
||||
.get_mut("large-bucket")
|
||||
.expect("reconciled bucket")
|
||||
.bucket_incarnation = authoritative.buckets_usage["large-bucket"].bucket_incarnation.clone();
|
||||
save_config(
|
||||
store.clone(),
|
||||
DATA_USAGE_OBJ_NAME_PATH.as_str(),
|
||||
@@ -4849,6 +4931,12 @@ mod tests {
|
||||
backup.scanner_epoch = Some(4);
|
||||
backup.scanner_cycle = Some(10);
|
||||
backup.usage_snapshot_converged = Some(true);
|
||||
bind_admin_usage_test_identities(&store, &mut current).await;
|
||||
backup
|
||||
.buckets_usage
|
||||
.get_mut("existing")
|
||||
.expect("backup bucket")
|
||||
.bucket_incarnation = current.buckets_usage["existing"].bucket_incarnation.clone();
|
||||
for (path, info) in [
|
||||
(DATA_USAGE_OBJ_NAME_PATH.as_str(), ¤t),
|
||||
(DATA_USAGE_OBJ_BACKUP_PATH.as_str(), &backup),
|
||||
@@ -4871,6 +4959,11 @@ mod tests {
|
||||
after_delete.scanner_epoch = current.scanner_epoch;
|
||||
after_delete.scanner_cycle = current.scanner_cycle;
|
||||
after_delete.usage_snapshot_converged = Some(true);
|
||||
after_delete
|
||||
.buckets_usage
|
||||
.get_mut("existing")
|
||||
.expect("post-delete bucket")
|
||||
.bucket_incarnation = current.buckets_usage["existing"].bucket_incarnation.clone();
|
||||
let mut stale_observation =
|
||||
data_usage_info_for_test("retired-incarnation", 99, 999, SystemTime::UNIX_EPOCH + Duration::from_secs(4));
|
||||
stale_observation.scanner_epoch = Some(4);
|
||||
@@ -4932,6 +5025,7 @@ mod tests {
|
||||
authoritative.scanner_epoch = Some(7);
|
||||
authoritative.scanner_cycle = Some(10);
|
||||
authoritative.usage_snapshot_converged = Some(true);
|
||||
bind_admin_usage_test_identities(&store, &mut authoritative).await;
|
||||
save_config(
|
||||
store.clone(),
|
||||
DATA_USAGE_OBJ_NAME_PATH.as_str(),
|
||||
@@ -4946,6 +5040,10 @@ mod tests {
|
||||
info.scanner_cycle = Some(cycle);
|
||||
info.usage_snapshot_converged = Some(false);
|
||||
info.usage_snapshot_authoritative_baseline = Some(authoritative.snapshot_identity());
|
||||
info.buckets_usage
|
||||
.get_mut("bucket")
|
||||
.expect("observed bucket")
|
||||
.bucket_incarnation = authoritative.buckets_usage["bucket"].bucket_incarnation.clone();
|
||||
info
|
||||
};
|
||||
let first = observation(7, 11, 200, 11, 1_100);
|
||||
|
||||
@@ -221,7 +221,7 @@ async fn companion_admin_cache_isolated_from_quota_memory_and_persistence() {
|
||||
|
||||
#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
|
||||
#[serial]
|
||||
async fn blocked_companion_identity_keeps_canonical_without_delaying_healthy_buckets() {
|
||||
async fn blocked_companion_identity_stays_unknown_without_delaying_healthy_buckets() {
|
||||
use crate::bucket::metadata_sys::{
|
||||
acquire_metadata_transaction_guard, init_bucket_metadata_sys, test_support::isolated_store_over_temp_disks,
|
||||
};
|
||||
@@ -280,8 +280,11 @@ async fn blocked_companion_identity_keeps_canonical_without_delaying_healthy_buc
|
||||
.await
|
||||
.expect("optional projection must return before the blocked metadata lock's long timeout")
|
||||
.expect("canonical admin response stays available");
|
||||
assert_eq!(response.buckets_usage[blocked].size, 100);
|
||||
assert_eq!(response.buckets_usage[blocked].usage_value_state, Some(BucketUsageValueState::Carried));
|
||||
assert!(
|
||||
!response.buckets_usage.contains_key(blocked),
|
||||
"an unavailable current identity cannot bind the canonical value"
|
||||
);
|
||||
assert!(!response.bucket_sizes.contains_key(blocked));
|
||||
assert_eq!(response.buckets_usage[healthy].size, 250);
|
||||
assert_eq!(response.buckets_usage[healthy].usage_value_state, Some(BucketUsageValueState::Observed));
|
||||
assert!(response.usage_snapshot_admin_projection);
|
||||
@@ -293,13 +296,12 @@ async fn blocked_companion_identity_keeps_canonical_without_delaying_healthy_buc
|
||||
.is_ok()
|
||||
);
|
||||
drop(blocked_guard);
|
||||
invalidate_data_usage_snapshot_cache().await;
|
||||
let recovered = load_admin_data_usage_from_backend_cached(store.clone())
|
||||
.await
|
||||
.expect("refresh after lock recovery");
|
||||
assert_eq!(
|
||||
recovered.buckets_usage[blocked].size, 250,
|
||||
"unavailable identity can recover on the next refresh"
|
||||
"unavailable identity can recover on a cache hit"
|
||||
);
|
||||
assert_eq!(recovered.buckets_usage[healthy].size, 250);
|
||||
let quota = load_data_usage_from_backend_cached(store)
|
||||
@@ -308,3 +310,201 @@ async fn blocked_companion_identity_keeps_canonical_without_delaying_healthy_buc
|
||||
assert_eq!(quota.buckets_usage[blocked].size, 100);
|
||||
assert_eq!(quota.buckets_usage[healthy].size, 100);
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
|
||||
#[serial]
|
||||
async fn cached_canonical_admin_usage_rechecks_current_identity_without_peer_invalidation() {
|
||||
use crate::bucket::metadata::{load_bucket_incarnation, save_bucket_incarnation};
|
||||
use crate::bucket::metadata_sys::{init_bucket_metadata_sys, test_support::isolated_store_over_temp_disks};
|
||||
use crate::config::com::save_config;
|
||||
use crate::storage_api_contracts::bucket::MakeBucketOptions;
|
||||
|
||||
let (_dirs, store) = isolated_store_over_temp_disks().await;
|
||||
init_bucket_metadata_sys(store.clone(), Vec::new()).await;
|
||||
let bucket = "usage-recreated-cache";
|
||||
store
|
||||
.make_bucket(bucket, &MakeBucketOptions::default())
|
||||
.await
|
||||
.expect("create identity fixture");
|
||||
let old_identity = store
|
||||
.bucket_incarnation_id_from_disk(bucket)
|
||||
.await
|
||||
.expect("original identity");
|
||||
let mut canonical = companion_snapshot(bucket, old_identity);
|
||||
canonical.usage_snapshot_bucket_observations.clear();
|
||||
canonical.usage_snapshot_converged = Some(true);
|
||||
canonical.usage_snapshot_pending_buckets.clear();
|
||||
canonical.usage_snapshot_pending_bucket_count = 0;
|
||||
canonical
|
||||
.buckets_usage
|
||||
.get_mut(bucket)
|
||||
.expect("canonical bucket")
|
||||
.objects_count = 107;
|
||||
canonical.calculate_totals();
|
||||
let original_bytes = serde_json::to_vec(&canonical).expect("encode canonical old bucket");
|
||||
save_config(store.clone(), DATA_USAGE_OBJ_NAME_PATH.as_str(), original_bytes.clone())
|
||||
.await
|
||||
.expect("persist canonical root");
|
||||
invalidate_data_usage_snapshot_cache().await;
|
||||
let first = load_admin_data_usage_from_backend_cached(store.clone())
|
||||
.await
|
||||
.expect("prime raw admin cache");
|
||||
assert_eq!(first.buckets_usage[bucket].objects_count, 107);
|
||||
assert!(first.is_complete_bucket_usage_snapshot());
|
||||
let cache_generation = admin_data_usage_snapshot_generation();
|
||||
let new_identity = uuid::Uuid::new_v4();
|
||||
// Simulate the storage-owned identity changing on another node while this
|
||||
// process has not received its invalidation notice. Do not clear the cache.
|
||||
save_bucket_incarnation(store.clone(), bucket, new_identity)
|
||||
.await
|
||||
.expect("publish changed disk identity");
|
||||
assert_eq!(
|
||||
load_bucket_incarnation(store.clone(), bucket)
|
||||
.await
|
||||
.expect("read new sidecar"),
|
||||
Some(new_identity)
|
||||
);
|
||||
let second = load_admin_data_usage_from_backend_cached(store.clone())
|
||||
.await
|
||||
.expect("read raw cache with fresh identity");
|
||||
assert_eq!(admin_data_usage_snapshot_generation(), cache_generation);
|
||||
assert!(
|
||||
!second.buckets_usage.contains_key(bucket),
|
||||
"same name cannot retain the predecessor's 107"
|
||||
);
|
||||
assert!(!second.bucket_sizes.contains_key(bucket));
|
||||
assert_eq!(second.usage_snapshot_converged, Some(false));
|
||||
assert!(second.usage_snapshot_admin_projection);
|
||||
assert!(reject_admin_usage_projection(&second).is_err());
|
||||
assert!(certified_bucket_usage(&second, bucket, new_identity).is_none());
|
||||
assert!(parse_usage_snapshot(&serde_json::to_vec(&second).expect("encode display response")).is_err());
|
||||
let quota = load_data_usage_from_backend_cached(store.clone())
|
||||
.await
|
||||
.expect("independent quota baseline");
|
||||
assert_eq!(quota.buckets_usage[bucket].objects_count, 107);
|
||||
assert_eq!(
|
||||
read_config(store.clone(), DATA_USAGE_OBJ_NAME_PATH.as_str())
|
||||
.await
|
||||
.expect("unchanged persisted root"),
|
||||
original_bytes
|
||||
);
|
||||
// Rebinding a cached raw baseline is transient: recovery does not need to
|
||||
// reload or permanently blank a healthy bucket's statistics.
|
||||
save_bucket_incarnation(store.clone(), bucket, old_identity)
|
||||
.await
|
||||
.expect("restore fixture identity");
|
||||
let recovered = load_admin_data_usage_from_backend_cached(store)
|
||||
.await
|
||||
.expect("recover on another cache hit");
|
||||
assert_eq!(recovered.buckets_usage[bucket].objects_count, 107);
|
||||
assert!(recovered.is_complete_bucket_usage_snapshot());
|
||||
assert_eq!(admin_data_usage_snapshot_generation(), cache_generation);
|
||||
invalidate_data_usage_snapshot_cache().await;
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
|
||||
#[serial]
|
||||
async fn admin_memory_overlay_cannot_reintroduce_a_predecessor_identity() {
|
||||
use crate::bucket::metadata::save_bucket_incarnation;
|
||||
use crate::bucket::metadata_sys::{init_bucket_metadata_sys, test_support::isolated_store_over_temp_disks};
|
||||
use crate::config::com::save_config;
|
||||
use crate::storage_api_contracts::bucket::MakeBucketOptions;
|
||||
|
||||
let (_dirs, store) = isolated_store_over_temp_disks().await;
|
||||
init_bucket_metadata_sys(store.clone(), Vec::new()).await;
|
||||
let bucket = "usage-recreated-overlay";
|
||||
store
|
||||
.make_bucket(bucket, &MakeBucketOptions::default())
|
||||
.await
|
||||
.expect("create overlay identity fixture");
|
||||
let old_identity = store
|
||||
.bucket_incarnation_id_from_disk(bucket)
|
||||
.await
|
||||
.expect("original identity");
|
||||
let mut canonical = companion_snapshot(bucket, old_identity);
|
||||
canonical.usage_snapshot_bucket_observations.clear();
|
||||
save_config(
|
||||
store.clone(),
|
||||
DATA_USAGE_OBJ_NAME_PATH.as_str(),
|
||||
serde_json::to_vec(&canonical).expect("encode canonical root"),
|
||||
)
|
||||
.await
|
||||
.expect("persist overlay fixture");
|
||||
invalidate_data_usage_snapshot_cache().await;
|
||||
let cached_usage = canonical.buckets_usage[bucket].clone();
|
||||
replace_bucket_usage_memory_from_authoritative(bucket, cached_usage, SystemTime::now()).await;
|
||||
let new_identity = uuid::Uuid::new_v4();
|
||||
save_bucket_incarnation(store.clone(), bucket, new_identity)
|
||||
.await
|
||||
.expect("publish a replacement identity");
|
||||
let response = load_admin_data_usage_with_memory_overlay(store.clone())
|
||||
.await
|
||||
.expect("project after memory overlay");
|
||||
assert!(!response.buckets_usage.contains_key(bucket));
|
||||
assert_eq!(response.usage_snapshot_converged, Some(false));
|
||||
assert!(response.usage_snapshot_admin_projection);
|
||||
clear_usage_memory_cache_for_test().await;
|
||||
invalidate_data_usage_snapshot_cache().await;
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
|
||||
#[serial]
|
||||
async fn admin_overlay_flags_only_change_keeps_identity_bound_cold_usage() {
|
||||
use crate::bucket::metadata_sys::{init_bucket_metadata_sys, test_support::isolated_store_over_temp_disks};
|
||||
use crate::config::com::save_config;
|
||||
use crate::storage_api_contracts::bucket::MakeBucketOptions;
|
||||
|
||||
let (_dirs, store) = isolated_store_over_temp_disks().await;
|
||||
init_bucket_metadata_sys(store.clone(), Vec::new()).await;
|
||||
assert!(
|
||||
!runtime_sources::setup_is_dist_erasure().await,
|
||||
"exercise the production single-process overlay"
|
||||
);
|
||||
let cold = "usage-overlay-cold";
|
||||
let unverified = "usage-overlay-unverified";
|
||||
for bucket in [cold, unverified] {
|
||||
store
|
||||
.make_bucket(bucket, &MakeBucketOptions::default())
|
||||
.await
|
||||
.expect("create overlay fixture bucket");
|
||||
}
|
||||
let identity = store
|
||||
.bucket_incarnation_id_from_disk(cold)
|
||||
.await
|
||||
.expect("cold bucket identity");
|
||||
let mut canonical = companion_snapshot(cold, identity);
|
||||
canonical.usage_snapshot_bucket_observations.clear();
|
||||
canonical.usage_snapshot_converged = Some(true);
|
||||
save_config(
|
||||
store.clone(),
|
||||
DATA_USAGE_OBJ_NAME_PATH.as_str(),
|
||||
serde_json::to_vec(&canonical).expect("encode cold canonical value"),
|
||||
)
|
||||
.await
|
||||
.expect("persist cold baseline");
|
||||
clear_usage_memory_cache_for_test().await;
|
||||
invalidate_data_usage_snapshot_cache().await;
|
||||
memory_cache().write().await.insert(
|
||||
unverified.to_string(),
|
||||
cached_bucket_usage_from_backend(BucketUsageInfo::default(), SystemTime::UNIX_EPOCH, false),
|
||||
);
|
||||
record_bucket_object_write_memory(unverified, None, 42).await;
|
||||
{
|
||||
let cache = memory_cache().read().await;
|
||||
assert!(cache[unverified].dirty);
|
||||
assert!(!cache[unverified].authoritative);
|
||||
}
|
||||
let response = load_admin_data_usage_with_memory_overlay(store)
|
||||
.await
|
||||
.expect("identity-filtered flags-only overlay");
|
||||
assert_eq!(response.buckets_usage[cold].size, 100);
|
||||
assert_eq!(response.objects_total_size, 100);
|
||||
assert_eq!(response.bucket_sizes[cold], 100);
|
||||
assert!(!response.buckets_usage.contains_key(unverified));
|
||||
assert!(response.usage_snapshot_admin_projection);
|
||||
assert_eq!(response.usage_snapshot_converged, Some(false));
|
||||
assert!(!response.is_complete_bucket_usage_snapshot());
|
||||
assert!(!response.is_valid_partial_snapshot());
|
||||
clear_usage_memory_cache_for_test().await;
|
||||
invalidate_data_usage_snapshot_cache().await;
|
||||
}
|
||||
|
||||
@@ -1033,16 +1033,26 @@ mod tests {
|
||||
#[tokio::test]
|
||||
#[serial]
|
||||
async fn server_info_reads_complete_observation_without_promoting_authority() {
|
||||
use crate::bucket::metadata_sys::test_support::isolated_store_over_temp_disks;
|
||||
use crate::bucket::metadata_sys::{init_bucket_metadata_sys, test_support::isolated_store_over_temp_disks};
|
||||
use crate::config::com::{read_config, save_config};
|
||||
use crate::data_usage::{
|
||||
DATA_USAGE_OBJ_NAME_PATH, DATA_USAGE_OBSERVED_OBJ_NAME_PATH, invalidate_data_usage_snapshot_cache,
|
||||
load_data_usage_from_backend_cached,
|
||||
};
|
||||
use crate::storage_api_contracts::bucket::{BucketOperations as _, MakeBucketOptions};
|
||||
use rustfs_data_usage::{BucketUsageInfo, DataUsageInfo};
|
||||
use std::time::{Duration, SystemTime};
|
||||
|
||||
let (_dirs, store) = isolated_store_over_temp_disks().await;
|
||||
init_bucket_metadata_sys(store.clone(), Vec::new()).await;
|
||||
store
|
||||
.make_bucket("bucket", &MakeBucketOptions::default())
|
||||
.await
|
||||
.expect("create a real observed bucket");
|
||||
let incarnation = store
|
||||
.bucket_incarnation_id_from_disk("bucket")
|
||||
.await
|
||||
.expect("read current storage-owned incarnation");
|
||||
let primary = DataUsageInfo {
|
||||
last_update: Some(SystemTime::UNIX_EPOCH + Duration::from_secs(1)),
|
||||
scanner_epoch: Some(1),
|
||||
@@ -1066,6 +1076,7 @@ mod tests {
|
||||
versions_count: 4,
|
||||
delete_markers_count: 1,
|
||||
size: 6,
|
||||
bucket_incarnation: Some(incarnation.to_string()),
|
||||
..Default::default()
|
||||
},
|
||||
)]
|
||||
@@ -1117,6 +1128,46 @@ mod tests {
|
||||
invalidate_data_usage_snapshot_cache().await;
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn identity_filtered_usage_remains_unknown_in_server_info() {
|
||||
let mut info = rustfs_data_usage::DataUsageInfo {
|
||||
last_update: Some(std::time::SystemTime::UNIX_EPOCH),
|
||||
buckets_count: 1,
|
||||
objects_total_count: 107,
|
||||
objects_total_size: 1024,
|
||||
buckets_usage: [(
|
||||
"bucket".to_string(),
|
||||
rustfs_data_usage::BucketUsageInfo {
|
||||
objects_count: 107,
|
||||
size: 1024,
|
||||
bucket_incarnation: Some("11111111-1111-4111-8111-111111111111".to_string()),
|
||||
..Default::default()
|
||||
},
|
||||
)]
|
||||
.into_iter()
|
||||
.collect(),
|
||||
usage_snapshot_complete: true,
|
||||
..Default::default()
|
||||
};
|
||||
let current =
|
||||
std::collections::HashMap::from([("bucket".to_string(), "22222222-2222-4222-8222-222222222222".to_string())]);
|
||||
assert_eq!(info.retain_current_bucket_usage_for_admin(¤t), 1);
|
||||
assert!(info.buckets_usage.is_empty());
|
||||
assert!(info.usage_snapshot_admin_projection);
|
||||
assert_eq!(info.usage_snapshot_converged, Some(false));
|
||||
let mut buckets = rustfs_madmin::Buckets::default();
|
||||
let mut objects = rustfs_madmin::Objects::default();
|
||||
let mut versions = rustfs_madmin::Versions::default();
|
||||
let mut delete_markers = rustfs_madmin::DeleteMarkers::default();
|
||||
let mut usage = rustfs_madmin::Usage::default();
|
||||
apply_data_usage_result(Ok(info), &mut buckets, &mut objects, &mut versions, &mut delete_markers, &mut usage);
|
||||
assert_eq!(buckets.error.as_deref(), Some(DATA_USAGE_UNAVAILABLE_ERROR));
|
||||
assert_eq!(objects.error.as_deref(), Some(DATA_USAGE_UNAVAILABLE_ERROR));
|
||||
assert_eq!(versions.error.as_deref(), Some(DATA_USAGE_UNAVAILABLE_ERROR));
|
||||
assert_eq!(delete_markers.error.as_deref(), Some(DATA_USAGE_UNAVAILABLE_ERROR));
|
||||
assert_eq!(usage.error.as_deref(), Some(DATA_USAGE_UNAVAILABLE_ERROR));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn incomplete_data_usage_is_unavailable_in_server_info() {
|
||||
let mut buckets = rustfs_madmin::Buckets::default();
|
||||
|
||||
@@ -180,9 +180,6 @@ pub(crate) const GET_METADATA_CACHE_DECISION_SKIP: &str = "skip";
|
||||
pub(crate) const GET_METADATA_CACHE_REASON_DATA_MOVEMENT: &str = "data_movement";
|
||||
pub(crate) const GET_METADATA_CACHE_REASON_DELETE_MARKER: &str = "delete_marker";
|
||||
pub(crate) const GET_METADATA_CACHE_REASON_DIST_ERASURE: &str = "dist_erasure";
|
||||
pub(crate) const GET_METADATA_CACHE_REASON_FENCE_LOCK_LOST: &str = "fence_lock_lost";
|
||||
pub(crate) const GET_METADATA_CACHE_REASON_FENCE_PROOF_MISMATCH: &str = "fence_proof_mismatch";
|
||||
pub(crate) const GET_METADATA_CACHE_REASON_FENCE_PROOF_UNAVAILABLE: &str = "fence_proof_unavailable";
|
||||
pub(crate) const GET_METADATA_CACHE_REASON_INCL_FREE_VERSIONS: &str = "incl_free_versions";
|
||||
pub(crate) const GET_METADATA_CACHE_REASON_INSUFFICIENT_CACHED_QUORUM: &str = "insufficient_cached_quorum";
|
||||
pub(crate) const GET_METADATA_CACHE_REASON_BODY_LAYOUT_REQUIRED: &str = "body_layout_required";
|
||||
|
||||
@@ -22,7 +22,6 @@ use self::commit::lock_rename_commit_directories;
|
||||
|
||||
mod commit;
|
||||
pub(super) mod delete_recovery;
|
||||
mod delete_reserve;
|
||||
#[cfg(all(unix, test))]
|
||||
#[path = "fd_cache_tests.rs"]
|
||||
mod fd_cache_tests;
|
||||
@@ -2307,11 +2306,6 @@ impl DurabilityMode {
|
||||
matches!(self, Self::Strict)
|
||||
}
|
||||
|
||||
/// A rollback directory's entry in the object directory is recovery state.
|
||||
fn syncs_metadata_commit_directory(self, has_recovery_backup: bool) -> bool {
|
||||
self.syncs_commit_directories() || (has_recovery_backup && self.syncs_recovery_protocol_state())
|
||||
}
|
||||
|
||||
/// Whether a target has the complete file-and-directory durability proof
|
||||
/// required by recovery protocols that may otherwise resync it.
|
||||
fn fully_syncs_commit_target(self) -> bool {
|
||||
@@ -8252,7 +8246,6 @@ pub struct LocalDisk {
|
||||
pub root: PathBuf,
|
||||
publication_root: os::PublicationRoot,
|
||||
publication_history: Option<os::NativePublicationRegistration>,
|
||||
delete_reserve: Arc<delete_reserve::DeleteReserve>,
|
||||
multipart_proof_authority: OnceLock<crate::disk::multipart_source::ProofVerificationContext>,
|
||||
pub(crate) usage_watch_binding: Arc<super::usage_watch::DiskUsageWatchBinding>,
|
||||
multipart_index: Arc<multipart_index::IndexOwner>,
|
||||
@@ -8668,16 +8661,6 @@ impl LocalDisk {
|
||||
#[cfg(not(target_os = "linux"))]
|
||||
let io_root = Self::open_mount_lease(&root)?;
|
||||
|
||||
// A persisted reserve can also make a full disk writable after restart.
|
||||
let reserve_root = io_root.clone();
|
||||
let lease = os::acquire_namespace_mutation_lease_with_owner(&io_root, None).await;
|
||||
let delete_reserve = os::run_blocking_namespace_operation_with_sync(lease, true, move || {
|
||||
let reserve = delete_reserve::DeleteReserve::acquire(&reserve_root)?;
|
||||
let _ = reserve.release_if_low();
|
||||
Ok(reserve)
|
||||
})
|
||||
.await
|
||||
.map_err(DiskError::from)?;
|
||||
Self::check_startup_writable(&publication_root, &root).await?;
|
||||
multipart_source_state::invalidate_active_multipart_sources_under(&io_root);
|
||||
multipart_source_state::invalidate_active_multipart_sources_under(&root);
|
||||
@@ -8694,16 +8677,6 @@ impl LocalDisk {
|
||||
log_startup_disk_error("ensure_data_usage_layout", &root, err);
|
||||
})?;
|
||||
|
||||
let reserve = delete_reserve.clone();
|
||||
let lease = os::acquire_namespace_mutation_lease_with_owner(&io_root, None).await;
|
||||
os::run_blocking_namespace_operation_with_sync(lease, true, move || {
|
||||
// Reservation is best effort: a legacy full disk must remain readable.
|
||||
let _ = reserve.initialize();
|
||||
Ok(())
|
||||
})
|
||||
.await
|
||||
.map_err(DiskError::from)?;
|
||||
|
||||
let startup_cleanup_ready = Arc::new(AtomicU32::new(u32::from(!cleanup)));
|
||||
let startup_cleanup_notify = Arc::new(Notify::new());
|
||||
|
||||
@@ -8835,7 +8808,6 @@ impl LocalDisk {
|
||||
let multipart_index = multipart_index::IndexOwner::acquire(&publication_root, &root, &io_root)?;
|
||||
let blocking_queue = multipart_index.attach_blocking_queue(&root, &io_root);
|
||||
let mut disk = Self {
|
||||
delete_reserve,
|
||||
multipart_index,
|
||||
root: root.clone(),
|
||||
publication_root,
|
||||
@@ -10015,13 +9987,6 @@ impl LocalDisk {
|
||||
check_path_length(file_path.to_string_lossy().as_ref())?;
|
||||
|
||||
let xl_path = path_join(&[file_path.as_path(), Path::new(STORAGE_FORMAT_FILE)]);
|
||||
#[cfg(target_os = "linux")]
|
||||
let object_path = {
|
||||
let (root_bucket, object_path) = build_local_disk_object_path(&self.root, volume, path);
|
||||
check_local_disk_object_path_components(&self.root, &root_bucket, &object_path)?;
|
||||
object_path
|
||||
};
|
||||
#[cfg(not(target_os = "linux"))]
|
||||
let object_path = self.get_object_path(volume, path)?;
|
||||
let mut namespace_owner: Option<Arc<dyn Send + Sync>> =
|
||||
Some(os::acquire_metadata_mutation_lease(&object_path, namespace_owner).await);
|
||||
@@ -11722,6 +11687,7 @@ impl LocalDisk {
|
||||
expected_current: Option<&CurrentDeleteGeneration>,
|
||||
) -> Result<()> {
|
||||
pool_meta_native::reject_noncanonical_object(&self.root, volume, path)?;
|
||||
let volume_dir = self.io_get_bucket_path(volume)?;
|
||||
let object_path = self.get_object_path(volume, path)?;
|
||||
let mut namespace_owner: Option<Arc<dyn Send + Sync>> =
|
||||
Some(os::acquire_metadata_mutation_lease(&object_path, namespace_owner).await);
|
||||
@@ -11729,8 +11695,7 @@ impl LocalDisk {
|
||||
let volume_guard = os::disk_volume_mutation_lock(&self.root, volume).read_owned().await;
|
||||
namespace_owner = Some(Arc::new((namespace_owner, volume_guard)));
|
||||
}
|
||||
let (volume_dir, xlpath) =
|
||||
self.io_get_bucket_and_object_path(volume, format!("{path}/{STORAGE_FORMAT_FILE}").as_str())?;
|
||||
let xlpath = self.io_get_object_path(volume, format!("{path}/{STORAGE_FORMAT_FILE}").as_str())?;
|
||||
let object_dir = xlpath
|
||||
.parent()
|
||||
.ok_or_else(|| DiskError::other("missing object metadata parent"))?;
|
||||
@@ -14076,7 +14041,6 @@ fn validate_existing_local_disk_prefix_at(
|
||||
return Ok(());
|
||||
}
|
||||
|
||||
let has_dotdot = rustfs_utils::path::unix_path_has_dotdot_segment(relative.as_os_str().as_encoded_bytes());
|
||||
let mut candidate = relative;
|
||||
loop {
|
||||
match openat2(
|
||||
@@ -14090,9 +14054,6 @@ fn validate_existing_local_disk_prefix_at(
|
||||
Err(Errno::NOSYS) => return Err(LocalDiskPathValidationAtError::Unsupported),
|
||||
Err(Errno::LOOP | Errno::XDEV) => return Err(LocalDiskPathValidationAtError::InvalidPath),
|
||||
Err(Errno::NOENT) => {
|
||||
if !has_dotdot {
|
||||
return Ok(());
|
||||
}
|
||||
let Some(parent) =
|
||||
rustfs_utils::path::unix_path_parent(candidate).filter(|parent| !parent.as_os_str().is_empty())
|
||||
else {
|
||||
@@ -17682,9 +17643,12 @@ impl DiskAPI for LocalDisk {
|
||||
fi.data = None;
|
||||
}
|
||||
|
||||
// Keep legacy external prefetch within one block during metadata
|
||||
// fanout. Persisted inline payloads are loaded by their marker and
|
||||
// can contain several frames without reading external part files.
|
||||
// Keep this compatibility read-ahead decision on the same policy
|
||||
// as PUT's inline admission. In particular, do not use the old
|
||||
// fixed 128 KiB shard limit: a non-inline object in a wider EC
|
||||
// layout can have a smaller shard and would otherwise be copied
|
||||
// out of part.1 during every metadata read. Such objects remain
|
||||
// fully readable through the normal EC reader below.
|
||||
let storage_class_config = runtime_sources::storage_class_config_snapshot();
|
||||
if should_read_legacy_inline_part(&fi, storage_class_config.as_ref()) {
|
||||
let part_path = path_join_buf(&[
|
||||
@@ -17957,9 +17921,6 @@ fn should_read_legacy_inline_part(fi: &FileInfo, storage_class_config: &crate::c
|
||||
}
|
||||
|
||||
let part = &fi.parts[0];
|
||||
if !usize::try_from(part.actual_size).is_ok_and(|size| size <= fi.erasure.block_size) {
|
||||
return false;
|
||||
}
|
||||
let shard_size = fi.shard_file_size(part.actual_size);
|
||||
let versioned = fi.versioned || fi.version_id.is_some_and(|version_id| !version_id.is_nil());
|
||||
storage_class_config.should_inline(shard_size, fi.erasure.data_blocks, versioned)
|
||||
@@ -18682,16 +18643,9 @@ mod test {
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn legacy_inline_read_ahead_stays_single_block_for_ec_layouts() {
|
||||
fn legacy_inline_read_ahead_matches_writer_policy_for_ec_layouts() {
|
||||
let config = crate::config::storageclass::Config::default();
|
||||
let object_sizes = [
|
||||
128 * 1024_i64,
|
||||
256 * 1024,
|
||||
512 * 1024,
|
||||
1024 * 1024,
|
||||
1024 * 1024 + 1,
|
||||
4 * 1024 * 1024,
|
||||
];
|
||||
let object_sizes = [128 * 1024_i64, 256 * 1024, 512 * 1024, 1024 * 1024, 4 * 1024 * 1024];
|
||||
|
||||
for (data_shards, parity_shards) in [(8, 4), (12, 4)] {
|
||||
for object_size in object_sizes {
|
||||
@@ -18704,8 +18658,7 @@ mod test {
|
||||
..Default::default()
|
||||
}];
|
||||
|
||||
let writer_decision = config.should_inline(fi.shard_file_size(object_size), data_shards, false)
|
||||
&& usize::try_from(object_size).expect("test size") <= fi.erasure.block_size;
|
||||
let writer_decision = config.should_inline(fi.shard_file_size(object_size), data_shards, false);
|
||||
assert_eq!(
|
||||
should_read_legacy_inline_part(&fi, &config),
|
||||
writer_decision,
|
||||
@@ -18718,13 +18671,13 @@ mod test {
|
||||
ec12.data_dir = Some(Uuid::from_u128(1));
|
||||
ec12.parts = vec![ObjectPartInfo {
|
||||
number: 1,
|
||||
size: 1024 * 1024 + 1,
|
||||
actual_size: 1024 * 1024 + 1,
|
||||
size: 1024 * 1024,
|
||||
actual_size: 1024 * 1024,
|
||||
..Default::default()
|
||||
}];
|
||||
assert!(
|
||||
ec12.shard_file_size(1024 * 1024 + 1) < 128 * 1024,
|
||||
"a multi-block external object can still fit the per-shard budget"
|
||||
ec12.shard_file_size(1024 * 1024) < crate::config::storageclass::DEFAULT_INLINE_BLOCK as i64,
|
||||
"the regression guard must exercise the old fixed 128 KiB read-ahead boundary"
|
||||
);
|
||||
assert!(!should_read_legacy_inline_part(&ec12, &config));
|
||||
}
|
||||
@@ -22900,218 +22853,6 @@ mod test {
|
||||
);
|
||||
}
|
||||
|
||||
#[cfg(unix)]
|
||||
#[tokio::test]
|
||||
async fn rename_data_fresh_inline_sync_budget() {
|
||||
const CHILD: &str = "RUSTFS_TEST_INLINE_SYNC_BUDGET_CHILD";
|
||||
const TEST: &str = "disk::local::test::rename_data_fresh_inline_sync_budget";
|
||||
|
||||
// Process-wide hooks can select a different commit implementation.
|
||||
// Grouping selects the fallback path even for a fresh Linux object.
|
||||
if std::env::var_os(CHILD).is_none() {
|
||||
for grouping in ["false", "true"] {
|
||||
let output = std::process::Command::new(std::env::current_exe().expect("test executable"))
|
||||
.args(["--exact", TEST, "--nocapture", "--test-threads=1"])
|
||||
.env(CHILD, grouping)
|
||||
.env("RUSTFS_DURABILITY_MODE", "strict")
|
||||
.env("RUSTFS_EXPERIMENTAL_DST_DIR_FSYNC_GROUP_COMMIT_ENABLE", grouping)
|
||||
.env("RUSTFS_EXPERIMENTAL_FILE_FDATASYNC_GROUP_COMMIT_ENABLE", "false")
|
||||
.output()
|
||||
.expect("inline sync budget subprocess");
|
||||
let stdout = String::from_utf8_lossy(&output.stdout);
|
||||
let stderr = String::from_utf8_lossy(&output.stderr);
|
||||
assert!(output.status.success(), "grouping={grouping}: {stdout}{stderr}");
|
||||
assert!(stdout.contains("INLINE_SYNC_BUDGET_OK"), "budget test must execute every case: {stdout}");
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
let force_fallback = std::env::var(CHILD).expect("child mode") == "true";
|
||||
assert_eq!(os::dst_dir_fsync_group_commit_enabled(), force_fallback);
|
||||
assert!(!os::file_fdatasync_group_commit_enabled());
|
||||
for mode in [DurabilityMode::Strict, DurabilityMode::FileSync] {
|
||||
for object in ["object", "prefix/nested/object"] {
|
||||
let dir = tempfile::tempdir().expect("disk directory");
|
||||
let endpoint = Endpoint::try_from(dir.path().to_str().expect("UTF-8 disk path")).expect("disk endpoint");
|
||||
let disk = LocalDisk::new(&endpoint, false).await.expect("local disk");
|
||||
let bucket = "inline-sync-budget";
|
||||
ensure_test_volume(&disk, bucket).await;
|
||||
ensure_test_volume(&disk, RUSTFS_META_TMP_BUCKET).await;
|
||||
bucket_durability::set(bucket, Some(mode));
|
||||
|
||||
let bucket_dir = disk.io_root().join(bucket);
|
||||
let tmp_dir = disk.io_root().join(RUSTFS_META_TMP_BUCKET);
|
||||
let object_dir = bucket_dir.join(object);
|
||||
let metadata_path = object_dir.join(STORAGE_FORMAT_FILE);
|
||||
let bucket_syncs = os::fsync_dir_recorder::count_under(&bucket_dir);
|
||||
let tmp_syncs = os::fsync_dir_recorder::count_under(&tmp_dir);
|
||||
let submitted_jobs = disk.blocking_queue.submitted_jobs_for_test();
|
||||
let payload = Bytes::from_static(b"inline payload");
|
||||
let mut info = test_file_info(object, Uuid::nil(), Some(Uuid::new_v4()), Some(payload.clone()));
|
||||
info.set_inline_data();
|
||||
disk.rename_data(RUSTFS_META_TMP_BUCKET, "fresh-inline-stage", info, bucket, object)
|
||||
.await
|
||||
.expect("fresh inline commit");
|
||||
|
||||
if force_fallback && mode == DurabilityMode::FileSync {
|
||||
assert_eq!(
|
||||
disk.blocking_queue.submitted_jobs_for_test() - submitted_jobs,
|
||||
1 + usize::from(cfg!(target_os = "linux")),
|
||||
"file-sync must publish within preparation when directory grouping has no work"
|
||||
);
|
||||
}
|
||||
assert_eq!(
|
||||
os::take_prepared_metadata_sync_count(&metadata_path),
|
||||
1,
|
||||
"{mode:?}, fallback={force_fallback}: metadata must be synchronized once"
|
||||
);
|
||||
let expected_directory_syncs = if mode == DurabilityMode::Strict {
|
||||
object.split('/').count() + 1
|
||||
} else {
|
||||
0
|
||||
};
|
||||
assert_eq!(
|
||||
os::fsync_dir_recorder::count_under(&bucket_dir) - bucket_syncs,
|
||||
expected_directory_syncs,
|
||||
"{mode:?}, {object}, fallback={force_fallback}: fresh commit directory budget"
|
||||
);
|
||||
assert_eq!(
|
||||
os::fsync_dir_recorder::count_under(&tmp_dir),
|
||||
tmp_syncs,
|
||||
"staging directories need no sync"
|
||||
);
|
||||
assert_eq!(
|
||||
std::fs::read_dir(&object_dir).expect("committed object entries").count(),
|
||||
1,
|
||||
"a fresh inline object must contain only xl.meta"
|
||||
);
|
||||
assert!(
|
||||
!tmp_dir.join("fresh-inline-stage").exists(),
|
||||
"staging directory must be absent after commit"
|
||||
);
|
||||
|
||||
let ordinary_syncs = os::fsync_dir_recorder::count_at(&object_dir);
|
||||
let bucket_syncs = os::fsync_dir_recorder::count_under(&bucket_dir);
|
||||
let replacement_payload = Bytes::from_static(b"replacement inline payload");
|
||||
let mut replacement =
|
||||
test_file_info(object, Uuid::nil(), Some(Uuid::new_v4()), Some(replacement_payload.clone()));
|
||||
replacement.set_inline_data();
|
||||
let response = disk
|
||||
.rename_data(RUSTFS_META_TMP_BUCKET, "overwrite-inline-stage", replacement, bucket, object)
|
||||
.await
|
||||
.expect("inline overwrite");
|
||||
let rollback_dir = object_dir.join(response.rollback_data_dir.expect("overwrite rollback owner").to_string());
|
||||
assert_eq!(
|
||||
os::take_prepared_metadata_sync_count(&metadata_path),
|
||||
1,
|
||||
"overwrite metadata must be synchronized once"
|
||||
);
|
||||
assert_eq!(
|
||||
os::fsync_dir_recorder::count_at(&rollback_dir),
|
||||
1,
|
||||
"rollback directory must stay synchronized"
|
||||
);
|
||||
assert_eq!(
|
||||
os::fsync_dir_recorder::count_at(&object_dir) - ordinary_syncs,
|
||||
1,
|
||||
"{mode:?}, fallback={force_fallback}: the rollback directory entry must stay durable"
|
||||
);
|
||||
assert_eq!(
|
||||
os::fsync_dir_recorder::count_under(&bucket_dir) - bucket_syncs,
|
||||
2,
|
||||
"overwrite synchronizes the rollback directory and its entry in the object directory"
|
||||
);
|
||||
assert_eq!(
|
||||
os::fsync_dir_recorder::count_under(&tmp_dir),
|
||||
tmp_syncs,
|
||||
"overwrite staging directories need no sync"
|
||||
);
|
||||
let stored = disk
|
||||
.read_version(
|
||||
"",
|
||||
bucket,
|
||||
object,
|
||||
"",
|
||||
&ReadOptions {
|
||||
read_data: true,
|
||||
..Default::default()
|
||||
},
|
||||
)
|
||||
.await
|
||||
.expect("read committed inline object");
|
||||
assert_eq!(stored.data, Some(replacement_payload), "overwrite must publish the replacement payload");
|
||||
|
||||
std::fs::remove_dir_all(&rollback_dir).expect("finalize the preceding inline overwrite");
|
||||
let predecessor = std::fs::read(&metadata_path).expect("inline predecessor metadata");
|
||||
let external_data_dir = Uuid::new_v4();
|
||||
let external_payload = b"external replacement payload";
|
||||
let staged_data = tmp_dir.join("external-stage").join(external_data_dir.to_string());
|
||||
std::fs::create_dir_all(&staged_data).expect("external staging directory");
|
||||
std::fs::write(staged_data.join("part.1"), external_payload).expect("staged external payload");
|
||||
let mut external = test_file_info(object, Uuid::nil(), Some(external_data_dir), None);
|
||||
let external_size = i64::try_from(external_payload.len()).expect("fixture size fits i64");
|
||||
external.size = external_size;
|
||||
external.parts[0].size = external_payload.len();
|
||||
external.parts[0].actual_size = external_size;
|
||||
if mode == DurabilityMode::FileSync {
|
||||
if force_fallback {
|
||||
os::fsync_dir_recorder::set_grouped_failure(&object_dir, ErrorKind::PermissionDenied);
|
||||
} else {
|
||||
os::fsync_dir_recorder::set_failure(&object_dir, ErrorKind::PermissionDenied);
|
||||
}
|
||||
let error = disk
|
||||
.rename_data(RUSTFS_META_TMP_BUCKET, "external-stage", external.clone(), bucket, object)
|
||||
.await
|
||||
.expect_err("failed recovery publication barrier must reject the overwrite");
|
||||
assert_eq!(error, DiskError::FileAccessDenied);
|
||||
assert_eq!(std::fs::read(&metadata_path).expect("restored predecessor metadata"), predecessor);
|
||||
assert!(
|
||||
!object_dir.join(external_data_dir.to_string()).exists(),
|
||||
"failed publication must remove the new external data directory"
|
||||
);
|
||||
std::fs::create_dir_all(&staged_data).expect("retry staging directory");
|
||||
std::fs::write(staged_data.join("part.1"), external_payload).expect("retry payload");
|
||||
}
|
||||
let object_syncs = os::fsync_dir_recorder::count_at(&object_dir);
|
||||
let recovery_syncs = os::fsync_dir_recorder::count_under(&bucket_dir);
|
||||
let response = disk
|
||||
.rename_data(RUSTFS_META_TMP_BUCKET, "external-stage", external, bucket, object)
|
||||
.await
|
||||
.expect("inline to external overwrite");
|
||||
let rollback_dir = object_dir.join(response.rollback_data_dir.expect("external rollback owner").to_string());
|
||||
assert_eq!(
|
||||
os::fsync_dir_recorder::count_at(&object_dir) - object_syncs,
|
||||
1,
|
||||
"{mode:?}, fallback={force_fallback}: external overwrite must publish its recovery directory durably"
|
||||
);
|
||||
assert_eq!(
|
||||
os::fsync_dir_recorder::count_under(&bucket_dir) - recovery_syncs,
|
||||
2,
|
||||
"external overwrite must synchronize its backup and its entry in the object directory"
|
||||
);
|
||||
assert_eq!(
|
||||
std::fs::read(rollback_dir.join(STORAGE_FORMAT_FILE_BACKUP)).expect("external rollback metadata"),
|
||||
predecessor,
|
||||
"external overwrite must preserve its inline predecessor"
|
||||
);
|
||||
assert_eq!(
|
||||
std::fs::read(object_dir.join(external_data_dir.to_string()).join("part.1"))
|
||||
.expect("committed external payload"),
|
||||
external_payload
|
||||
);
|
||||
// Linux reuses procfs disk handles across fixtures. Consume
|
||||
// every attempt before releasing this disk's path identity.
|
||||
assert_eq!(
|
||||
os::take_prepared_metadata_sync_count(&metadata_path),
|
||||
if mode == DurabilityMode::FileSync { 2 } else { 1 },
|
||||
"each external attempt must synchronize its metadata exactly once"
|
||||
);
|
||||
bucket_durability::set(bucket, None);
|
||||
}
|
||||
}
|
||||
println!("INLINE_SYNC_BUDGET_OK");
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn test_rename_data_new_object_fsyncs_new_ancestor_dirs() {
|
||||
// A first PUT under a new prefix must fsync every newly created ancestor
|
||||
@@ -24888,8 +24629,6 @@ mod test {
|
||||
assert!(DurabilityMode::Strict.syncs_data_shards());
|
||||
assert!(DurabilityMode::Strict.syncs_metadata_files());
|
||||
assert!(DurabilityMode::Strict.syncs_commit_directories());
|
||||
assert!(DurabilityMode::Strict.syncs_metadata_commit_directory(false));
|
||||
assert!(DurabilityMode::Strict.syncs_metadata_commit_directory(true));
|
||||
assert!(DurabilityMode::Strict.fully_syncs_commit_target());
|
||||
assert!(DurabilityMode::Strict.syncs_recovery_protocol_state());
|
||||
assert_eq!(DurabilityMode::Strict.metadata_sync_mode(), SyncMode::FileAndDir);
|
||||
@@ -24899,8 +24638,6 @@ mod test {
|
||||
assert!(DurabilityMode::FileSync.syncs_data_shards());
|
||||
assert!(DurabilityMode::FileSync.syncs_metadata_files());
|
||||
assert!(!DurabilityMode::FileSync.syncs_commit_directories());
|
||||
assert!(!DurabilityMode::FileSync.syncs_metadata_commit_directory(false));
|
||||
assert!(DurabilityMode::FileSync.syncs_metadata_commit_directory(true));
|
||||
assert!(!DurabilityMode::FileSync.fully_syncs_commit_target());
|
||||
assert!(DurabilityMode::FileSync.syncs_recovery_protocol_state());
|
||||
assert_eq!(DurabilityMode::FileSync.metadata_sync_mode(), SyncMode::FileOnly);
|
||||
@@ -24909,8 +24646,6 @@ mod test {
|
||||
assert!(DurabilityMode::Relaxed.syncs_data_shards());
|
||||
assert!(!DurabilityMode::Relaxed.syncs_metadata_files());
|
||||
assert!(!DurabilityMode::Relaxed.syncs_commit_directories());
|
||||
assert!(!DurabilityMode::Relaxed.syncs_metadata_commit_directory(false));
|
||||
assert!(!DurabilityMode::Relaxed.syncs_metadata_commit_directory(true));
|
||||
assert!(!DurabilityMode::Relaxed.fully_syncs_commit_target());
|
||||
assert!(!DurabilityMode::Relaxed.syncs_recovery_protocol_state());
|
||||
assert_eq!(DurabilityMode::Relaxed.metadata_sync_mode(), SyncMode::None);
|
||||
@@ -24919,14 +24654,10 @@ mod test {
|
||||
assert!(!DurabilityMode::None.syncs_data_shards());
|
||||
assert!(!DurabilityMode::None.syncs_metadata_files());
|
||||
assert!(!DurabilityMode::None.syncs_commit_directories());
|
||||
assert!(!DurabilityMode::None.syncs_metadata_commit_directory(false));
|
||||
assert!(!DurabilityMode::None.syncs_metadata_commit_directory(true));
|
||||
assert!(!DurabilityMode::None.syncs_recovery_protocol_state());
|
||||
assert!(!DurabilityMode::LegacyOff.syncs_data_shards());
|
||||
assert!(!DurabilityMode::LegacyOff.syncs_metadata_files());
|
||||
assert!(!DurabilityMode::LegacyOff.syncs_commit_directories());
|
||||
assert!(!DurabilityMode::LegacyOff.syncs_metadata_commit_directory(false));
|
||||
assert!(!DurabilityMode::LegacyOff.syncs_metadata_commit_directory(true));
|
||||
assert!(!DurabilityMode::LegacyOff.syncs_recovery_protocol_state());
|
||||
}
|
||||
|
||||
@@ -25883,16 +25614,13 @@ mod test {
|
||||
assert!(!disk.io_get_object_path(bucket, "parent").expect("parent path").exists());
|
||||
|
||||
let dst_meta_path = dst_object_dir.join(STORAGE_FORMAT_FILE);
|
||||
let v5_meta_bytes = fs::read(&dst_meta_path).await.expect("read v5 xl.meta");
|
||||
let dst_vol_dir = disk.io_get_bucket_path(bucket).expect("bucket path");
|
||||
let lease = os::acquire_namespace_mutation_lease_with_owner(&dst_meta_path, None).await;
|
||||
let guard =
|
||||
commit::lock_rename_commit_directories(&dst_object_dir, &dst_object_dir, &dst_vol_dir, &disk.publication_root, lease)
|
||||
.await
|
||||
.expect("lock commit guard");
|
||||
let v5_read_file = os::read_destination_file_handle_with_commit_guard(&dst_meta_path, &guard)
|
||||
.expect("read v5 handle")
|
||||
.expect("v5 handle present");
|
||||
let v5_meta_bytes = v5_read_file.bytes.to_vec();
|
||||
fs::create_dir_all(synthetic_backup_path.join("nested-child"))
|
||||
.await
|
||||
.expect("create conflicting directory at xl.meta.bkp");
|
||||
@@ -25901,7 +25629,7 @@ mod test {
|
||||
&dst_meta_path,
|
||||
&synthetic_backup_parent,
|
||||
&synthetic_backup_path,
|
||||
&v5_read_file,
|
||||
&v5_meta_bytes,
|
||||
true,
|
||||
true,
|
||||
&guard,
|
||||
@@ -25920,7 +25648,7 @@ mod test {
|
||||
&dst_meta_path,
|
||||
&synthetic_backup_parent,
|
||||
&synthetic_backup_path,
|
||||
&v5_read_file,
|
||||
&v5_meta_bytes,
|
||||
true,
|
||||
true,
|
||||
&guard,
|
||||
@@ -25933,7 +25661,7 @@ mod test {
|
||||
&dst_meta_path,
|
||||
&synthetic_backup_parent,
|
||||
&synthetic_backup_path,
|
||||
&v5_read_file,
|
||||
&v5_meta_bytes,
|
||||
true,
|
||||
true,
|
||||
&guard,
|
||||
@@ -25973,7 +25701,7 @@ mod test {
|
||||
&dst_meta_path,
|
||||
&synthetic_backup_parent,
|
||||
&synthetic_backup_path,
|
||||
&v5_read_file,
|
||||
&v5_meta_bytes,
|
||||
true,
|
||||
true,
|
||||
&guard,
|
||||
@@ -26030,200 +25758,6 @@ mod test {
|
||||
);
|
||||
assert!(!replacement_dir.exists());
|
||||
assert_eq!(fs::read(&dst_meta_path).await.expect("restored"), v1_meta_bytes);
|
||||
|
||||
// Task 2 verification:
|
||||
// 1. By-FD linkat (/proc/self/fd/<fd>) preserves the read predecessor inode even when
|
||||
// xl.meta on disk is replaced by another file, a symlink, or unlinked before linkat.
|
||||
let v1_read_file = os::read_destination_file_handle_with_commit_guard(&dst_meta_path, &guard)
|
||||
.expect("read restored v1 handle")
|
||||
.expect("v1 handle present");
|
||||
let v1_ino = v1_read_file.file.metadata().expect("stat v1 fd").ino();
|
||||
fs::write(&staged_new_meta, b"impostor-regular-file")
|
||||
.await
|
||||
.expect("write impostor");
|
||||
fs::rename(&staged_new_meta, &dst_meta_path)
|
||||
.await
|
||||
.expect("replace xl.meta with impostor inode while v1_read_file is open");
|
||||
os::stage_inline_rollback_backup_with_commit_guard_std(
|
||||
&dst_meta_path,
|
||||
&synthetic_backup_parent,
|
||||
&synthetic_backup_path,
|
||||
&v1_read_file,
|
||||
true,
|
||||
true,
|
||||
&guard,
|
||||
)
|
||||
.expect("by-FD linkat must stage the held v1_read_file when i_nlink == 0 via copy fallback");
|
||||
assert_eq!(
|
||||
fs::read(&synthetic_backup_path).await.expect("read fallback backup"),
|
||||
v1_meta_bytes,
|
||||
"unlinked old fd must fall back to writing captured v1_meta_bytes, never impostor bytes"
|
||||
);
|
||||
commit::discard_staged_inline_rollback_backup_std(synthetic_rollback, None, &guard);
|
||||
|
||||
// Keep a second hardlink to v1 so i_nlink >= 1 when xl.meta is replaced by a symlink.
|
||||
fs::write(&dst_meta_path, &v1_meta_bytes).await.expect("restore v1 xl.meta");
|
||||
let pinned_v1_read = os::read_destination_file_handle_with_commit_guard(&dst_meta_path, &guard)
|
||||
.expect("read pinned v1")
|
||||
.expect("pinned v1 present");
|
||||
let pinned_v1_ino = pinned_v1_read.file.metadata().expect("stat pinned v1").ino();
|
||||
let extra_link = dst_object_dir.join("extra-v1.link");
|
||||
std::fs::hard_link(&dst_meta_path, &extra_link).expect("hardlink v1 so i_nlink stays > 0");
|
||||
fs::remove_file(&dst_meta_path).await.expect("unlink xl.meta");
|
||||
std::os::unix::fs::symlink(&extra_link, &dst_meta_path).expect("replace xl.meta with symlink");
|
||||
os::stage_inline_rollback_backup_with_commit_guard_std(
|
||||
&dst_meta_path,
|
||||
&synthetic_backup_parent,
|
||||
&synthetic_backup_path,
|
||||
&pinned_v1_read,
|
||||
true,
|
||||
true,
|
||||
&guard,
|
||||
)
|
||||
.expect("by-FD linkat must link the held regular file, never the symlink at dst_meta_path");
|
||||
let bkp_symlink_meta = std::fs::symlink_metadata(&synthetic_backup_path).expect("lstat staged backup");
|
||||
assert!(bkp_symlink_meta.is_file(), "staged backup must be a regular file, not a symlink");
|
||||
assert_eq!(bkp_symlink_meta.ino(), pinned_v1_ino);
|
||||
commit::discard_staged_inline_rollback_backup_std(synthetic_rollback, None, &guard);
|
||||
fs::remove_file(&dst_meta_path).await.expect("remove symlink at xl.meta");
|
||||
fs::rename(&extra_link, &dst_meta_path)
|
||||
.await
|
||||
.expect("restore regular xl.meta");
|
||||
let _ = v1_ino;
|
||||
|
||||
// 2. Non-Windows write_file_for_path_access must unlink before create so a pre-existing
|
||||
// hardlinked .bkp is never truncated through its link.
|
||||
fs::create_dir_all(&synthetic_backup_parent)
|
||||
.await
|
||||
.expect("create synthetic dir for legacy writer test");
|
||||
let victim_path = dst_object_dir.join("victim-live-file");
|
||||
fs::write(&victim_path, b"do-not-truncate").await.expect("write victim");
|
||||
std::fs::hard_link(&victim_path, &synthetic_backup_path).expect("hardlink victim to .bkp");
|
||||
let path_guard = guard
|
||||
.create_destination_directory_for_path_access(&synthetic_backup_parent)
|
||||
.expect("destination path guard");
|
||||
path_guard
|
||||
.write_file_for_path_access(&synthetic_backup_path, b"new-bkp-bytes", true, true)
|
||||
.expect("legacy write_file_for_path_access");
|
||||
assert_eq!(
|
||||
fs::read(&victim_path).await.expect("read victim"),
|
||||
b"do-not-truncate",
|
||||
"write_file_for_path_access must unlink before create so hardlinked victim is untouched"
|
||||
);
|
||||
assert_eq!(fs::read(&synthetic_backup_path).await.expect("read new bkp"), b"new-bkp-bytes");
|
||||
fs::remove_file(&victim_path).await.expect("remove victim");
|
||||
commit::discard_staged_inline_rollback_backup_std(synthetic_rollback, None, &guard);
|
||||
|
||||
// 3. Deferred preflight error-classification and preflight_rejection parity on flat_inline_tmp:
|
||||
let inline_probe_fi = test_file_info("probe-obj", version_id, None, Some(Bytes::from_static(b"probe")));
|
||||
let missing_vol_obs = disk
|
||||
.rename_data_observed(
|
||||
RUSTFS_META_TMP_BUCKET,
|
||||
"tmp-missing-vol",
|
||||
&inline_probe_fi,
|
||||
"missing-volume",
|
||||
"probe-obj",
|
||||
None,
|
||||
)
|
||||
.await;
|
||||
assert!(matches!(missing_vol_obs.result, Err(DiskError::VolumeNotFound)));
|
||||
assert!(
|
||||
missing_vol_obs.rejected_before_publication(),
|
||||
"missing destination volume on flat_inline_tmp must carry preflight_rejection proof"
|
||||
);
|
||||
|
||||
let symlink_vol = dir.path().join("symlink-bucket");
|
||||
std::os::unix::fs::symlink(dir.path().join(bucket), &symlink_vol).expect("live symlink bucket");
|
||||
let symlink_obs = disk
|
||||
.rename_data_observed(
|
||||
RUSTFS_META_TMP_BUCKET,
|
||||
"tmp-symlink-vol",
|
||||
&inline_probe_fi,
|
||||
"symlink-bucket",
|
||||
"probe-obj",
|
||||
None,
|
||||
)
|
||||
.await;
|
||||
assert!(matches!(symlink_obs.result, Err(DiskError::InvalidPath)));
|
||||
assert!(!symlink_obs.rejected_before_publication());
|
||||
|
||||
let dangling_vol = dir.path().join("dangling-bucket");
|
||||
std::os::unix::fs::symlink(dir.path().join("nonexistent-target"), &dangling_vol).expect("dangling symlink bucket");
|
||||
let dangling_obs = disk
|
||||
.rename_data_observed(
|
||||
RUSTFS_META_TMP_BUCKET,
|
||||
"tmp-dangling-vol",
|
||||
&inline_probe_fi,
|
||||
"dangling-bucket",
|
||||
"probe-obj",
|
||||
None,
|
||||
)
|
||||
.await;
|
||||
assert!(matches!(dangling_obs.result, Err(DiskError::InvalidPath)));
|
||||
assert!(!dangling_obs.rejected_before_publication());
|
||||
|
||||
let file_vol = dir.path().join("file-bucket");
|
||||
fs::write(&file_vol, b"not-a-dir").await.expect("regular file at bucket path");
|
||||
let file_vol_obs = disk
|
||||
.rename_data_observed(RUSTFS_META_TMP_BUCKET, "tmp-file-vol", &inline_probe_fi, "file-bucket", "probe-obj", None)
|
||||
.await;
|
||||
assert!(matches!(file_vol_obs.result, Err(DiskError::FileAccessDenied)));
|
||||
assert!(!file_vol_obs.rejected_before_publication());
|
||||
|
||||
// 4. validate_existing_local_disk_prefix_at single-openat2 ENOENT shortcut:
|
||||
assert!(
|
||||
disk.io_get_bucket_and_object_path(bucket, "nonexistent-prefix/nonexistent-leaf")
|
||||
.is_ok()
|
||||
);
|
||||
std::os::unix::fs::symlink(dir.path().join("nonexistent-target"), dir.path().join(bucket).join("dangling-prefix"))
|
||||
.expect("dangling object prefix symlink");
|
||||
assert!(matches!(
|
||||
disk.io_get_bucket_and_object_path(bucket, "dangling-prefix/missing-leaf"),
|
||||
Err(DiskError::InvalidPath)
|
||||
));
|
||||
fs::write(dir.path().join(bucket).join("file-prefix"), b"file")
|
||||
.await
|
||||
.expect("file prefix");
|
||||
assert!(matches!(
|
||||
disk.io_get_bucket_and_object_path(bucket, "file-prefix/missing-leaf"),
|
||||
Err(DiskError::FileAccessDenied)
|
||||
));
|
||||
|
||||
// 5. Replacing .rustfs.sys/tmp during prepare_destination()'s component-open hook on
|
||||
// a new-object flat-inline PUT must reject with FileCorrupt without consuming the moved source.
|
||||
let hook_key = "hook-prefix/hook-leaf";
|
||||
let hook_prefix_dir = disk
|
||||
.io_get_object_open_path(bucket, "hook-prefix")
|
||||
.expect("hook prefix io path");
|
||||
let tmp_root_real = dir.path().join(RUSTFS_META_TMP_BUCKET);
|
||||
let tmp_displaced = dir.path().join(".rustfs.sys/tmp-displaced-hook");
|
||||
let tmp_root_for_hook = tmp_root_real.clone();
|
||||
let tmp_displaced_for_hook = tmp_displaced.clone();
|
||||
os::prepare_rename_test_hooks::queue_after_component_opened(&hook_prefix_dir, move || {
|
||||
std::fs::rename(&tmp_root_for_hook, &tmp_displaced_for_hook).expect("displace tmp during prepare_destination");
|
||||
std::fs::create_dir(&tmp_root_for_hook).expect("recreate empty tmp");
|
||||
});
|
||||
let hook_fi = test_file_info(hook_key, version_id, None, Some(Bytes::from_static(b"hook-inline")));
|
||||
let hook_err = disk
|
||||
.rename_data(RUSTFS_META_TMP_BUCKET, "tmp-hook-stage", hook_fi, bucket, hook_key)
|
||||
.await
|
||||
.expect_err("tmp replacement during prepare_destination must fail");
|
||||
assert!(
|
||||
matches!(hook_err, DiskError::FileCorrupt),
|
||||
"expected FileCorrupt on mid-prepare tmp replacement, got: {hook_err:?}"
|
||||
);
|
||||
assert!(
|
||||
tmp_displaced.join("tmp-hook-stage.xl.meta").is_file(),
|
||||
"moved source in displaced tmp must not be consumed"
|
||||
);
|
||||
fs::remove_dir_all(&tmp_root_real)
|
||||
.await
|
||||
.expect("remove empty replacement tmp");
|
||||
fs::rename(&tmp_displaced, &tmp_root_real)
|
||||
.await
|
||||
.expect("restore displaced tmp");
|
||||
let _ = fs::remove_file(tmp_root_real.join("tmp-hook-stage.xl.meta")).await;
|
||||
|
||||
println!("FUSED_OVERWRITE_OK");
|
||||
}
|
||||
|
||||
@@ -26765,12 +26299,7 @@ mod test {
|
||||
#[serial_test::serial(dst_dir_fsync_group_commit)]
|
||||
async fn inline_heal_staged_backup_restores_metadata_after_commit_failure() {
|
||||
let _group_commit = os::set_dst_dir_fsync_group_commit_for_test(true);
|
||||
for mode in [
|
||||
DurabilityMode::None,
|
||||
DurabilityMode::Relaxed,
|
||||
DurabilityMode::FileSync,
|
||||
DurabilityMode::Strict,
|
||||
] {
|
||||
for mode in [DurabilityMode::None, DurabilityMode::Relaxed, DurabilityMode::Strict] {
|
||||
let _mode = durability_mode_override::set(mode);
|
||||
for version in [Uuid::nil(), Uuid::new_v4()] {
|
||||
for old_inline in [true, false] {
|
||||
|
||||
@@ -389,77 +389,34 @@ pub(super) async fn lock_rename_commit_directories(
|
||||
result.map_err(to_file_error).map_err(DiskError::from)
|
||||
}
|
||||
|
||||
#[cfg(target_os = "linux")]
|
||||
#[derive(Clone, Copy)]
|
||||
enum SourcePreparationMode {
|
||||
CreateParent,
|
||||
ExistingParent,
|
||||
FlatInlineTmp { defer_dst_preflight: bool },
|
||||
}
|
||||
|
||||
#[cfg(target_os = "linux")]
|
||||
enum PrepareCommitAndDestinationError {
|
||||
PreflightAccess(std::io::Error),
|
||||
Io(std::io::Error),
|
||||
}
|
||||
|
||||
#[cfg(target_os = "linux")]
|
||||
fn prepare_rename_commit_and_destination_metadata_std(
|
||||
mode: SourcePreparationMode,
|
||||
create_source_parent: bool,
|
||||
source_parent: &Path,
|
||||
destination_parent: &Path,
|
||||
destination_metadata: &Path,
|
||||
base_dir: &Path,
|
||||
publication_root: &os::PublicationRoot,
|
||||
) -> core::result::Result<(os::RenameCommitGuard, Option<os::ReadDestinationFile>), PrepareCommitAndDestinationError> {
|
||||
if matches!(mode, SourcePreparationMode::CreateParent) {
|
||||
std::fs::create_dir_all(source_parent).map_err(PrepareCommitAndDestinationError::Io)?;
|
||||
) -> std::io::Result<(os::RenameCommitGuard, Option<Bytes>)> {
|
||||
if create_source_parent {
|
||||
std::fs::create_dir_all(source_parent)?;
|
||||
}
|
||||
let defer_dst_preflight = matches!(
|
||||
mode,
|
||||
SourcePreparationMode::FlatInlineTmp {
|
||||
defer_dst_preflight: true
|
||||
}
|
||||
);
|
||||
let guard_res = if matches!(mode, SourcePreparationMode::FlatInlineTmp { .. }) {
|
||||
os::prepare_rename_commit_guard_flat_inline(source_parent, destination_parent, base_dir, publication_root)
|
||||
} else {
|
||||
os::prepare_rename_commit_guard(source_parent, destination_parent, base_dir, publication_root)
|
||||
};
|
||||
let guard = match guard_res {
|
||||
let guard = match os::prepare_rename_commit_guard(source_parent, destination_parent, base_dir, publication_root) {
|
||||
Ok(guard) => guard,
|
||||
Err(err) => {
|
||||
if err
|
||||
.get_ref()
|
||||
.and_then(|inner| inner.downcast_ref::<DiskError>())
|
||||
.is_some_and(|disk_err| matches!(disk_err, DiskError::InvalidPath))
|
||||
{
|
||||
return Err(PrepareCommitAndDestinationError::Io(err));
|
||||
}
|
||||
if defer_dst_preflight && let Err(access_err) = super::super::fs::access_std(base_dir) {
|
||||
return Err(PrepareCommitAndDestinationError::PreflightAccess(access_err));
|
||||
}
|
||||
return Err(PrepareCommitAndDestinationError::Io(match std::fs::symlink_metadata(base_dir) {
|
||||
return Err(match std::fs::symlink_metadata(base_dir) {
|
||||
Err(base_err) if base_err.kind() == ErrorKind::NotFound => base_err,
|
||||
_ => err,
|
||||
}));
|
||||
});
|
||||
}
|
||||
};
|
||||
if defer_dst_preflight
|
||||
&& !guard.has_prepared_destination()
|
||||
&& let Err(access_err) = super::super::fs::access_std(base_dir)
|
||||
{
|
||||
return Err(PrepareCommitAndDestinationError::PreflightAccess(access_err));
|
||||
}
|
||||
let data = os::read_destination_file_handle_with_commit_guard(destination_metadata, &guard)
|
||||
.map_err(PrepareCommitAndDestinationError::Io)?;
|
||||
Ok((guard, data))
|
||||
let data = os::read_destination_file_with_commit_guard(destination_metadata, &guard)?;
|
||||
Ok((guard, data.map(Bytes::from)))
|
||||
}
|
||||
|
||||
#[cfg(target_os = "linux")]
|
||||
enum FusedCommitAttempt {
|
||||
Committed(Box<RenameDataResp>),
|
||||
PreflightAccessErr(std::io::Error),
|
||||
CommitErr(DiskError),
|
||||
RestoreDataSource(os::RenameCommitGuard, DiskError),
|
||||
Fallback(os::RenameCommitGuard, Option<Bytes>, Box<FileInfo>),
|
||||
@@ -630,20 +587,7 @@ impl LocalDisk {
|
||||
// object commit is publishing into it. The peer's empty scan remains
|
||||
// optimistic; this lease establishes the local commit/delete order and
|
||||
// remains owned by any blocking syscall that outlives async cancellation.
|
||||
let dst_dir_group_commit = os::dst_dir_fsync_group_commit_enabled();
|
||||
let flat_inline_tmp = cfg!(target_os = "linux")
|
||||
&& (fi.data.is_some() || fi.size <= 0)
|
||||
&& src_volume == super::super::RUSTFS_META_TMP_BUCKET
|
||||
&& pool_meta_condition.is_none()
|
||||
&& !fi.is_healing()
|
||||
&& !dst_dir_group_commit
|
||||
&& !has_rename_data_test_failpoints(dst_path);
|
||||
let defer_dst_preflight = flat_inline_tmp && !skip_access_checks(dst_volume);
|
||||
let (dst_volume_dir, destination_object_path) = if defer_dst_preflight {
|
||||
self.io_get_bucket_and_object_open_path(dst_volume, dst_path)?
|
||||
} else {
|
||||
self.io_get_bucket_and_object_path(dst_volume, dst_path)?
|
||||
};
|
||||
let (dst_volume_dir, destination_object_path) = self.io_get_bucket_and_object_path(dst_volume, dst_path)?;
|
||||
let quota_fence_token =
|
||||
match rustfs_utils::http::metadata_compat::get_consistent_str(&fi.metadata, QUOTA_MUTATION_FENCE_METADATA_SUFFIX) {
|
||||
Some(value) => {
|
||||
@@ -792,8 +736,7 @@ impl LocalDisk {
|
||||
return Err(to_access_error(e, DiskError::VolumeAccessDenied).into());
|
||||
}
|
||||
|
||||
if !defer_dst_preflight
|
||||
&& !skip_access_checks(dst_volume)
|
||||
if !skip_access_checks(dst_volume)
|
||||
&& let Err(e) = super::super::fs::access_std(&dst_volume_dir)
|
||||
{
|
||||
info!(
|
||||
@@ -819,6 +762,15 @@ impl LocalDisk {
|
||||
} else {
|
||||
STORAGE_FORMAT_FILE
|
||||
};
|
||||
let flat_inline_tmp = cfg!(target_os = "linux")
|
||||
&& !no_inline
|
||||
&& src_volume == super::super::RUSTFS_META_TMP_BUCKET
|
||||
&& !preserve_upload_metadata
|
||||
&& multipart_publication.is_none()
|
||||
&& pool_meta_condition.is_none()
|
||||
&& !fi_healing
|
||||
&& !os::dst_dir_fsync_group_commit_enabled()
|
||||
&& !has_rename_data_test_failpoints(dst_path);
|
||||
let src_file_path = match multipart_publication.as_ref() {
|
||||
Some(publication) => publication.canonical_head(),
|
||||
None if flat_inline_tmp => {
|
||||
@@ -900,13 +852,12 @@ impl LocalDisk {
|
||||
.ok_or_else(|| DiskError::other("missing object metadata parent"))?;
|
||||
|
||||
#[cfg(target_os = "linux")]
|
||||
let (rename_commit_guard, has_dst_buf) = if flat_inline_tmp
|
||||
|| (!preserve_upload_metadata
|
||||
&& multipart_publication.is_none()
|
||||
&& pool_meta_condition.is_none()
|
||||
&& !fi_healing
|
||||
&& !dst_dir_group_commit
|
||||
&& !file_fdatasync_grouping_requires_async_shard_sync(no_inline))
|
||||
let (rename_commit_guard, has_dst_buf) = if !preserve_upload_metadata
|
||||
&& multipart_publication.is_none()
|
||||
&& pool_meta_condition.is_none()
|
||||
&& !fi_healing
|
||||
&& !os::dst_dir_fsync_group_commit_enabled()
|
||||
&& !file_fdatasync_grouping_requires_async_shard_sync(no_inline)
|
||||
{
|
||||
#[cfg(test)]
|
||||
if !no_inline {
|
||||
@@ -941,45 +892,25 @@ impl LocalDisk {
|
||||
let root_owned = self.root.clone();
|
||||
|
||||
let fused_attempt = move |is_cancelled: &dyn Fn() -> bool| -> std::io::Result<FusedCommitAttempt> {
|
||||
let source_mode = if flat_inline_tmp {
|
||||
SourcePreparationMode::FlatInlineTmp { defer_dst_preflight }
|
||||
} else if !no_inline {
|
||||
SourcePreparationMode::CreateParent
|
||||
} else {
|
||||
SourcePreparationMode::ExistingParent
|
||||
};
|
||||
let (guard, old_destination_file) = match prepare_rename_commit_and_destination_metadata_std(
|
||||
source_mode,
|
||||
let (guard, has_dst_buf) = prepare_rename_commit_and_destination_metadata_std(
|
||||
!no_inline && !flat_inline_tmp,
|
||||
&src_parent_owned,
|
||||
&dst_parent_owned,
|
||||
&dst_file_path_owned,
|
||||
&dst_volume_dir_owned,
|
||||
&publication_root_owned,
|
||||
) {
|
||||
Ok(res) => res,
|
||||
Err(PrepareCommitAndDestinationError::PreflightAccess(err)) => {
|
||||
return Ok(FusedCommitAttempt::PreflightAccessErr(err));
|
||||
}
|
||||
Err(PrepareCommitAndDestinationError::Io(err)) => return Err(err),
|
||||
};
|
||||
)?;
|
||||
// An existing xl.meta that replaces an existing version slot
|
||||
// requires a rollback directory; appending a new version keeps
|
||||
// an inline rollback backup for post-rename directory sync errors.
|
||||
if is_cancelled() {
|
||||
if flat_inline_tmp {
|
||||
guard.ensure_source_pinned()?;
|
||||
}
|
||||
return Ok(FusedCommitAttempt::Fallback(
|
||||
guard,
|
||||
old_destination_file.map(|read| read.bytes),
|
||||
Box::new(fi_moved),
|
||||
));
|
||||
return Ok(FusedCommitAttempt::Fallback(guard, has_dst_buf, Box::new(fi_moved)));
|
||||
}
|
||||
let mut xlmeta = FileMeta::new();
|
||||
let mut dst_meta_unparsable = false;
|
||||
if let Some(ref old_file) = old_destination_file {
|
||||
if FileMeta::is_xl2_v1_format(&old_file.bytes)
|
||||
&& let Ok(nmeta) = FileMeta::load(&old_file.bytes)
|
||||
if let Some(ref dst_buf) = has_dst_buf {
|
||||
if FileMeta::is_xl2_v1_format(dst_buf)
|
||||
&& let Ok(nmeta) = FileMeta::load(dst_buf)
|
||||
{
|
||||
xlmeta = nmeta;
|
||||
} else {
|
||||
@@ -989,21 +920,14 @@ impl LocalDisk {
|
||||
let version_id = fi_moved.version_id.unwrap_or_default();
|
||||
let has_old_data_dir = xlmeta.find_unshared_data_dir_for_version(Some(version_id));
|
||||
let rollback_data_dir = has_old_data_dir.or_else(|| {
|
||||
if old_destination_file.is_some() && xlmeta.find_version(Some(version_id)).is_ok() {
|
||||
if has_dst_buf.is_some() && xlmeta.find_version(Some(version_id)).is_ok() {
|
||||
Some(inline_metadata_rollback_dir(version_id, &xlmeta))
|
||||
} else {
|
||||
None
|
||||
}
|
||||
});
|
||||
if rollback_data_dir.is_some() && has_rename_data_test_failpoints(&dst_path_str) {
|
||||
if flat_inline_tmp {
|
||||
guard.ensure_source_pinned()?;
|
||||
}
|
||||
return Ok(FusedCommitAttempt::Fallback(
|
||||
guard,
|
||||
old_destination_file.map(|read| read.bytes),
|
||||
Box::new(fi_moved),
|
||||
));
|
||||
return Ok(FusedCommitAttempt::Fallback(guard, has_dst_buf, Box::new(fi_moved)));
|
||||
}
|
||||
#[cfg(test)]
|
||||
let probe_active = os::file_sync_probe::is_active();
|
||||
@@ -1015,17 +939,10 @@ impl LocalDisk {
|
||||
&& (probe_active || !super::is_presynced_tmp_data_dir(src_data_path))
|
||||
&& os::regular_files(src_data_path).is_ok_and(|files| files.len() >= os::PARALLEL_FILE_SYNC_THRESHOLD)
|
||||
{
|
||||
if flat_inline_tmp {
|
||||
guard.ensure_source_pinned()?;
|
||||
}
|
||||
return Ok(FusedCommitAttempt::Fallback(
|
||||
guard,
|
||||
old_destination_file.map(|read| read.bytes),
|
||||
Box::new(fi_moved),
|
||||
));
|
||||
return Ok(FusedCommitAttempt::Fallback(guard, has_dst_buf, Box::new(fi_moved)));
|
||||
}
|
||||
let (old_current_size, old_current_source_checked, old_current_source) = observe_old_current(
|
||||
old_destination_file.is_some(),
|
||||
has_dst_buf.is_some(),
|
||||
dst_meta_unparsable,
|
||||
&xlmeta,
|
||||
&dst_volume_str,
|
||||
@@ -1038,9 +955,9 @@ impl LocalDisk {
|
||||
xlmeta.add_version(fi_moved)?;
|
||||
let version_signature = rename_data_versions_signature(&xlmeta);
|
||||
let new_dst_buf = xlmeta.marshal_msg().map_err(std::io::Error::other)?;
|
||||
let destination_was_absent = old_destination_file.is_none();
|
||||
let destination_was_absent = has_dst_buf.is_none();
|
||||
let mut skip_parent = dst_volume_dir_owned.clone();
|
||||
if old_destination_file.is_some() {
|
||||
if has_dst_buf.is_some() {
|
||||
skip_parent = dst_parent_owned.clone();
|
||||
}
|
||||
|
||||
@@ -1050,7 +967,7 @@ impl LocalDisk {
|
||||
os::create_prepared_rename_source_with_commit_guard(&src_file_path_owned, &dst_file_path_owned, &guard)?;
|
||||
let cleanup_flat_tmp = || {
|
||||
if flat_inline_tmp {
|
||||
os::unlink_flat_tmp_source_std(&src_file_path_owned, &src_parent_owned, &publication_root_owned);
|
||||
let _ = remove_file_if_exists(&src_file_path_owned);
|
||||
}
|
||||
};
|
||||
if let Err(err) = prepared_metadata_source.write_all(&new_dst_buf, false) {
|
||||
@@ -1064,13 +981,13 @@ impl LocalDisk {
|
||||
|
||||
let mut staged_rollback_path = None;
|
||||
if rollback_data_dir.is_none()
|
||||
&& let Some(ref old_file) = old_destination_file
|
||||
&& let Some(ref old_metadata) = has_dst_buf
|
||||
&& cfg!(test)
|
||||
{
|
||||
let backup_path = match create_local_inline_rollback_backup(
|
||||
&dst_file_path_owned,
|
||||
&src_file_path_owned,
|
||||
&old_file.bytes,
|
||||
old_metadata,
|
||||
sync_recovery_protocol,
|
||||
) {
|
||||
Ok(path) => path,
|
||||
@@ -1190,22 +1107,28 @@ impl LocalDisk {
|
||||
}
|
||||
return Ok(FusedCommitAttempt::RestoreDataSource(guard, DiskError::Unexpected));
|
||||
}
|
||||
if let (Some(old_data_dir), Some(old_file)) = (rollback_data_dir, old_destination_file.as_ref()) {
|
||||
if let (Some(old_data_dir), Some(dst_buf)) = (rollback_data_dir, has_dst_buf.as_ref()) {
|
||||
let backup_parent = dst_parent_owned.join(old_data_dir.to_string());
|
||||
if let Err(err) =
|
||||
os::mkdir_all_below_existing_base_std(&backup_parent, &skip_parent, &publication_root_owned)
|
||||
{
|
||||
return Ok(FusedCommitAttempt::RestoreDataSource(guard, to_file_error(err).into()));
|
||||
}
|
||||
let backup_path_guard = match guard.create_destination_directory_for_path_access(&backup_parent) {
|
||||
Ok(g) => g,
|
||||
Err(err) => return Ok(FusedCommitAttempt::RestoreDataSource(guard, to_file_error(err).into())),
|
||||
};
|
||||
let backup_path = backup_parent.join(STORAGE_FORMAT_FILE_BACKUP);
|
||||
if let Err(err) = check_path_length(backup_path.to_string_lossy().as_ref()) {
|
||||
return Ok(FusedCommitAttempt::RestoreDataSource(guard, err));
|
||||
}
|
||||
#[cfg(test)]
|
||||
run_owned_file_before_open(&backup_path);
|
||||
if let Err(err) = os::stage_inline_rollback_backup_with_commit_guard_std(
|
||||
&dst_file_path_owned,
|
||||
&backup_parent,
|
||||
if let Err(err) = backup_path_guard.write_file_for_path_access(
|
||||
&backup_path,
|
||||
old_file,
|
||||
dst_buf.as_ref(),
|
||||
sync_recovery_protocol,
|
||||
sync_recovery_protocol,
|
||||
!should_fail_local_inline_rollback_hardlink(&dst_file_path_owned),
|
||||
&guard,
|
||||
) {
|
||||
return Ok(FusedCommitAttempt::RestoreDataSource(guard, to_file_error(err).into()));
|
||||
}
|
||||
@@ -1213,7 +1136,7 @@ impl LocalDisk {
|
||||
if crash_inject::should_crash_at(CrashPoint::RenameAfterBackupBeforeMetaCommit, &dst_path_str) {
|
||||
return Ok(FusedCommitAttempt::CommitErr(DiskError::Unexpected));
|
||||
}
|
||||
} else if let (Some(old_data_dir), Some(old_file)) = (rollback_data_dir, old_destination_file.as_ref()) {
|
||||
} else if let (Some(old_data_dir), Some(dst_buf)) = (rollback_data_dir, has_dst_buf.as_ref()) {
|
||||
let backup_parent = dst_parent_owned.join(old_data_dir.to_string());
|
||||
let backup_path = backup_parent.join(STORAGE_FORMAT_FILE_BACKUP);
|
||||
if let Err(err) = check_path_length(backup_path.to_string_lossy().as_ref()) {
|
||||
@@ -1227,7 +1150,7 @@ impl LocalDisk {
|
||||
&dst_file_path_owned,
|
||||
&backup_parent,
|
||||
&backup_path,
|
||||
old_file,
|
||||
dst_buf.as_ref(),
|
||||
sync_recovery_protocol,
|
||||
!should_fail_local_inline_rollback_hardlink(&dst_file_path_owned),
|
||||
&guard,
|
||||
@@ -1259,12 +1182,12 @@ impl LocalDisk {
|
||||
if let Some(backup_path) = staged_rollback_path.as_ref().map(StagedRollbackBackup::path) {
|
||||
let _ = remove_file_if_exists(backup_path);
|
||||
}
|
||||
if let Some(old_data_dir) = rollback_data_dir {
|
||||
discard_staged_inline_rollback_backup_std(old_data_dir, has_old_data_dir, &guard);
|
||||
}
|
||||
if no_inline {
|
||||
return Ok(FusedCommitAttempt::RestoreDataSource(guard, err));
|
||||
}
|
||||
if let Some(old_data_dir) = rollback_data_dir {
|
||||
discard_staged_inline_rollback_backup_std(old_data_dir, has_old_data_dir, &guard);
|
||||
}
|
||||
return Ok(FusedCommitAttempt::CommitErr(err));
|
||||
}
|
||||
|
||||
@@ -1287,10 +1210,10 @@ impl LocalDisk {
|
||||
return Ok(FusedCommitAttempt::CommitErr(DiskError::Unexpected));
|
||||
}
|
||||
|
||||
if durability.syncs_metadata_commit_directory(rollback_data_dir.is_some()) {
|
||||
if sync_commit_directories {
|
||||
if staged_rollback_path.is_none()
|
||||
&& rollback_data_dir.is_none()
|
||||
&& let Some(ref old_file) = old_destination_file
|
||||
&& let Some(ref old_metadata) = has_dst_buf
|
||||
&& (!no_inline || rollback_data_dir.is_none())
|
||||
{
|
||||
let fsync_res = (|| -> std::io::Result<()> {
|
||||
@@ -1301,7 +1224,7 @@ impl LocalDisk {
|
||||
})();
|
||||
if let Err(err) = fsync_res {
|
||||
let rollback_ok =
|
||||
write_local_inline_rollback_backup_from_bytes(&src_file_path_owned, &old_file.bytes, true)
|
||||
write_local_inline_rollback_backup_from_bytes(&src_file_path_owned, old_metadata, true)
|
||||
.and_then(|backup_path| {
|
||||
let backup = StagedRollbackBackup(backup_path);
|
||||
let res =
|
||||
@@ -1376,20 +1299,6 @@ impl LocalDisk {
|
||||
}
|
||||
return Ok(*resp);
|
||||
}
|
||||
Ok(FusedCommitAttempt::PreflightAccessErr(e)) => {
|
||||
info!(
|
||||
target: "rustfs_ecstore::disk::local",
|
||||
event = EVENT_DISK_LOCAL_ACCESS_FAILED,
|
||||
component = LOG_COMPONENT_ECSTORE,
|
||||
subsystem = LOG_SUBSYSTEM_DISK_LOCAL,
|
||||
path = ?dst_volume_dir,
|
||||
operation = "rename_data_dst_access",
|
||||
error = %e,
|
||||
"Disk local access check failed"
|
||||
);
|
||||
state.preflight_rejection = Some(LocalRenamePreflightRejection(()));
|
||||
return Err(to_access_error(e, DiskError::VolumeAccessDenied).into());
|
||||
}
|
||||
Ok(FusedCommitAttempt::RestoreDataSource(directory_guard, err)) => {
|
||||
let commit = RenameCommitContext {
|
||||
directory_guard,
|
||||
@@ -1426,7 +1335,7 @@ impl LocalDisk {
|
||||
&& !preserve_upload_metadata
|
||||
&& !publication.legacy
|
||||
&& !fi_healing
|
||||
&& !dst_dir_group_commit
|
||||
&& !os::dst_dir_fsync_group_commit_enabled()
|
||||
{
|
||||
enum MultipartFusedCommitOutcome {
|
||||
Committed(Box<RenameDataResp>),
|
||||
@@ -1456,19 +1365,15 @@ impl LocalDisk {
|
||||
true,
|
||||
move || -> std::io::Result<MultipartFusedCommitOutcome> {
|
||||
let (directory_guard_owned, has_dst_buf_owned) = match prepare_rename_commit_and_destination_metadata_std(
|
||||
SourcePreparationMode::ExistingParent,
|
||||
false,
|
||||
&src_parent_owned,
|
||||
&dst_parent_owned,
|
||||
&dst_file_path_owned,
|
||||
&dst_volume_dir_owned,
|
||||
&publication_root_owned,
|
||||
) {
|
||||
Ok((guard, old_file)) => (guard, old_file.map(|read| read.bytes)),
|
||||
Err(
|
||||
PrepareCommitAndDestinationError::PreflightAccess(err) | PrepareCommitAndDestinationError::Io(err),
|
||||
) => {
|
||||
return Ok(MultipartFusedCommitOutcome::AbortBeforeDataRename(to_file_error(err).into()));
|
||||
}
|
||||
Ok(res) => res,
|
||||
Err(err) => return Ok(MultipartFusedCommitOutcome::AbortBeforeDataRename(to_file_error(err).into())),
|
||||
};
|
||||
let mut xlmeta = FileMeta::new();
|
||||
let mut dst_meta_unparsable = false;
|
||||
@@ -2006,10 +1911,9 @@ impl LocalDisk {
|
||||
// durable. It is part of the metadata commit machinery, so relaxed
|
||||
// tiers leave it to the page cache like the xl.meta it mirrors.
|
||||
let backup_sync = durability.recovery_sync_mode();
|
||||
let recovery_backup = rollback_data_dir
|
||||
.zip(has_dst_buf.as_ref())
|
||||
.filter(|_| multipart_publication.as_ref().is_none_or(|publication| publication.legacy));
|
||||
if let Some((old_data_dir, dst_buf)) = recovery_backup {
|
||||
if let (Some(old_data_dir), Some(dst_buf)) = (rollback_data_dir, has_dst_buf.as_ref())
|
||||
&& multipart_publication.as_ref().is_none_or(|publication| publication.legacy)
|
||||
{
|
||||
let backup_parent = dst_file_parent.join(old_data_dir.to_string());
|
||||
#[cfg(not(windows))]
|
||||
if let Err(err) = os::make_dir_all(&backup_parent, &skip_parent).await {
|
||||
@@ -2123,7 +2027,7 @@ impl LocalDisk {
|
||||
// Persist the directory entries for both the data dir and xl.meta renames;
|
||||
// without this the commit itself can vanish on power loss. Relaxed tiers
|
||||
// accept that window (documented in docs/operations/durability-modes.md).
|
||||
if durability.syncs_metadata_commit_directory(recovery_backup.is_some())
|
||||
if durability.syncs_commit_directories()
|
||||
&& let Some(parent) = dst_file_path.parent()
|
||||
{
|
||||
let fsync_started = rustfs_io_metrics::put_stage_timer();
|
||||
@@ -2272,15 +2176,13 @@ impl LocalDisk {
|
||||
let dst_path_for_source = dst_path.to_string();
|
||||
let dst_volume_dir_for_commit = dst_volume_dir.clone();
|
||||
let publication_root_for_commit = self.publication_root.clone();
|
||||
let cleanup_src_parent = !flat_inline_tmp && multipart_publication.is_none() && !preserve_upload_metadata;
|
||||
let cleanup_src_parent = multipart_publication.is_none() && !preserve_upload_metadata;
|
||||
#[cfg(windows)]
|
||||
let source_parent = src_file_parent.to_path_buf();
|
||||
let rename_commit_guard_for_preparation = commit.directory_guard.clone();
|
||||
let sync_metadata_files = durability.syncs_metadata_files();
|
||||
let sync_commit_directories = durability.syncs_commit_directories();
|
||||
let sync_recovery_protocol = durability.syncs_recovery_protocol_state();
|
||||
let sync = durability.syncs_recovery_protocol_state();
|
||||
#[cfg(unix)]
|
||||
let receipt_identity = if sync_recovery_protocol
|
||||
let receipt_identity = if sync
|
||||
&& pool_meta_condition
|
||||
.as_ref()
|
||||
.is_some_and(|condition| condition.parent_raw_sha256.is_none())
|
||||
@@ -2308,7 +2210,7 @@ impl LocalDisk {
|
||||
}
|
||||
#[cfg(test)]
|
||||
run_inline_before_file_sync_admission(dst_path);
|
||||
let mut file_sync_admission = if sync_metadata_files || sync_commit_directories || sync_recovery_protocol {
|
||||
let mut file_sync_admission = if sync {
|
||||
let _admission = os::StageTimer::new(native_timing, "pool_meta_native_file_sync_admission");
|
||||
Some(
|
||||
os::acquire_file_sync_admission(self.file_sync_permits.clone())
|
||||
@@ -2376,10 +2278,14 @@ impl LocalDisk {
|
||||
xlmeta.add_version(fi)?;
|
||||
let version_signature = rename_data_versions_signature(&xlmeta);
|
||||
let new_buf = xlmeta.marshal_msg()?;
|
||||
// Inline payload shares xl.meta's file synchronization policy.
|
||||
// Write the staged xl.meta. Inline objects carry their data inside
|
||||
// xl.meta, so this is the durable preparation for the metadata commit:
|
||||
// relaxed tiers do no per-object fsync here at all (aligned
|
||||
// with MinIO's default), trading a documented power-loss
|
||||
// window for latency.
|
||||
{
|
||||
let _new_file = os::StageTimer::new(native_timing, "pool_meta_native_staged_write_sync");
|
||||
prepared_metadata_source.write_all(&new_buf, sync_metadata_files)?;
|
||||
prepared_metadata_source.write_all(&new_buf, sync)?;
|
||||
}
|
||||
run_inline_preparation_before_backup(&dst_path_for_failpoint);
|
||||
#[cfg(unix)]
|
||||
@@ -2410,10 +2316,10 @@ impl LocalDisk {
|
||||
if !backup.is_linked() {
|
||||
receipt_candidate = None;
|
||||
}
|
||||
if sync_recovery_protocol && !reuses_durability {
|
||||
if sync && !reuses_durability {
|
||||
let _backup_sync = os::StageTimer::new(native_timing, "pool_meta_native_backup_file_sync");
|
||||
backup.sync_data()?;
|
||||
} else if sync_recovery_protocol {
|
||||
} else if sync {
|
||||
rustfs_io_metrics::record_put_object_stage_count("pool_meta_native_backup_sync_reused", 1);
|
||||
}
|
||||
Some(backup)
|
||||
@@ -2427,7 +2333,7 @@ impl LocalDisk {
|
||||
let owned_backup = None;
|
||||
if let Some(ref old_metadata) = has_dst_buf
|
||||
&& owned_backup.is_none()
|
||||
&& (rollback_data_dir.is_some() || fi_healing || sync_recovery_protocol || cfg!(test))
|
||||
&& (rollback_data_dir.is_some() || fi_healing || sync || cfg!(test))
|
||||
{
|
||||
#[cfg(windows)]
|
||||
let backup_path = {
|
||||
@@ -2436,18 +2342,13 @@ impl LocalDisk {
|
||||
.parent()
|
||||
.ok_or_else(|| std::io::Error::new(ErrorKind::InvalidInput, "missing staging metadata parent"))?
|
||||
.join(STORAGE_FORMAT_FILE_BACKUP);
|
||||
source_metadata_guard.write_file_for_path_access(
|
||||
&backup_path,
|
||||
old_metadata,
|
||||
sync_recovery_protocol,
|
||||
false,
|
||||
)?;
|
||||
source_metadata_guard.write_file_for_path_access(&backup_path, old_metadata, sync, false)?;
|
||||
backup_path
|
||||
};
|
||||
#[cfg(not(windows))]
|
||||
let backup_path = {
|
||||
let _backup = os::StageTimer::new(native_timing, "pool_meta_native_backup_create");
|
||||
create_local_inline_rollback_backup(&dst, &src, old_metadata, sync_recovery_protocol)?
|
||||
create_local_inline_rollback_backup(&dst, &src, old_metadata, sync)?
|
||||
};
|
||||
staged_rollback_path = Some(StagedRollbackBackup(backup_path));
|
||||
}
|
||||
@@ -2455,7 +2356,7 @@ impl LocalDisk {
|
||||
let destination_was_absent = has_dst_buf.is_none();
|
||||
if !is_cancelled()
|
||||
&& rollback_data_dir.is_none()
|
||||
&& (!sync_commit_directories || !os::dst_dir_fsync_group_commit_enabled())
|
||||
&& !os::dst_dir_fsync_group_commit_enabled()
|
||||
&& !has_inline_commit_test_failpoints(&dst_path_for_failpoint)
|
||||
{
|
||||
let rename_res = if should_fail_commit_rename(&dst_path_for_failpoint) {
|
||||
@@ -2472,10 +2373,6 @@ impl LocalDisk {
|
||||
)
|
||||
};
|
||||
if let Err(err) = rename_res {
|
||||
#[cfg(target_os = "linux")]
|
||||
if flat_inline_tmp {
|
||||
os::unlink_prepared_rename_source_with_commit_guard_std(&src, &rename_commit_guard_for_preparation);
|
||||
}
|
||||
if let Some(backup_path) = staged_rollback_path.as_ref().map(StagedRollbackBackup::path) {
|
||||
let _ = remove_file_if_exists(backup_path);
|
||||
}
|
||||
@@ -2489,7 +2386,7 @@ impl LocalDisk {
|
||||
return Ok(InlinePreparationOutcome::CommitErr(DiskError::from(err)));
|
||||
}
|
||||
|
||||
if sync_commit_directories
|
||||
if sync
|
||||
&& let Err(err) = fsync_inline_commit_std(
|
||||
&dst,
|
||||
&dst_volume_dir_for_commit,
|
||||
@@ -2591,10 +2488,6 @@ impl LocalDisk {
|
||||
}
|
||||
Ok(InlinePreparationOutcome::CommitErr(err)) | Err(err) => {
|
||||
drop(file_sync_admission.take());
|
||||
#[cfg(target_os = "linux")]
|
||||
if flat_inline_tmp {
|
||||
os::unlink_prepared_rename_source_with_commit_guard_std(&src_file_path, &commit.directory_guard);
|
||||
}
|
||||
for part_path in &invalidate_part_paths {
|
||||
self.io_backend.invalidate_cached_fd(dst_volume, part_path).await;
|
||||
}
|
||||
@@ -2688,7 +2581,7 @@ impl LocalDisk {
|
||||
}
|
||||
#[cfg(test)]
|
||||
run_rename_data_after_first_publication(&self.root, dst_volume, dst_path);
|
||||
if sync_commit_directories || sync_recovery_protocol {
|
||||
if sync {
|
||||
let _admission = os::StageTimer::new(native_timing, "pool_meta_native_file_sync_readmission");
|
||||
file_sync_admission = Some(
|
||||
os::acquire_file_sync_admission(self.file_sync_permits.clone())
|
||||
@@ -2737,10 +2630,6 @@ impl LocalDisk {
|
||||
};
|
||||
drop(head_publish);
|
||||
if let Err(err) = commit_result {
|
||||
#[cfg(target_os = "linux")]
|
||||
if flat_inline_tmp {
|
||||
os::unlink_prepared_rename_source_with_commit_guard_std(&src_file_path, &commit.directory_guard);
|
||||
}
|
||||
if let Some(backup_path) = local_rollback_path.as_ref().map(StagedRollbackBackup::path) {
|
||||
let _ = remove_file_if_exists(backup_path);
|
||||
}
|
||||
@@ -2756,10 +2645,8 @@ impl LocalDisk {
|
||||
return Err(std::io::Error::other("test fail after metadata commit"));
|
||||
}
|
||||
|
||||
// The object directory also publishes the recovery backup's
|
||||
// directory entry, which file-sync must keep durable.
|
||||
if durability.syncs_metadata_commit_directory(rollback_data_dir.is_some())
|
||||
&& let Some(admission) = file_sync_admission.as_ref()
|
||||
// Persist the commit rename's directory entry across power loss.
|
||||
if let Some(admission) = file_sync_admission.as_ref()
|
||||
&& let Some(dst_parent) = dst_file_path.parent()
|
||||
{
|
||||
let _head_dir = os::StageTimer::new(native_timing, "pool_meta_native_head_dir_sync");
|
||||
@@ -2792,8 +2679,7 @@ impl LocalDisk {
|
||||
// not its own entry, so for a new inline object fsync the ancestor
|
||||
// chain up to and including the bucket. Overwrites already have a
|
||||
// durable object dir; the starts_with guard bounds the walk.
|
||||
if sync_commit_directories
|
||||
&& let Some(admission) = file_sync_admission.as_ref()
|
||||
if let Some(admission) = file_sync_admission.as_ref()
|
||||
&& destination_was_absent
|
||||
{
|
||||
let mut ancestor = dst_file_path.parent().and_then(|object_dir| object_dir.parent());
|
||||
|
||||
@@ -209,7 +209,7 @@ impl DiscoverySession {
|
||||
None => regular_bytes(&proof_root.join(DELETE_QUORUM_COMMITTED_FILE))
|
||||
.map_err(std::io::Error::from)?
|
||||
.filter(|bytes| !bytes.is_empty())
|
||||
.and_then(|bytes| parse_commit_receipt(&bytes).ok())
|
||||
.and_then(|bytes| serde_json::from_slice::<CommitReceipt>(&bytes).ok())
|
||||
.map(|receipt| receipt.journal),
|
||||
};
|
||||
let logical = record
|
||||
@@ -273,21 +273,6 @@ struct CommitReceipt {
|
||||
finished: bool,
|
||||
}
|
||||
|
||||
/// Parse a commit receipt from `delete-quorum-committed`.
|
||||
/// A bare `DeleteRecoveryJournal` (from hardlinking `delete-prepared`) always
|
||||
/// represents `finished = false`; `finished = true` is only persisted via the
|
||||
/// `CommitReceipt` wrapper when replacing the receipt entry.
|
||||
fn parse_commit_receipt(bytes: &[u8]) -> Result<CommitReceipt> {
|
||||
if let Ok(receipt) = serde_json::from_slice::<CommitReceipt>(bytes) {
|
||||
return Ok(receipt);
|
||||
}
|
||||
let journal = serde_json::from_slice::<DeleteRecoveryJournal>(bytes).map_err(|_| DiskError::FileCorrupt)?;
|
||||
Ok(CommitReceipt {
|
||||
journal,
|
||||
finished: false,
|
||||
})
|
||||
}
|
||||
|
||||
pub(super) struct DeletePreparationContext {
|
||||
pub transaction: Uuid,
|
||||
pub bucket_incarnation: Option<Uuid>,
|
||||
@@ -881,8 +866,9 @@ pub(super) fn inspection_with_markers_bounded(
|
||||
let receipt: Option<CommitReceipt> = receipt_bytes
|
||||
.as_deref()
|
||||
.filter(|bytes| !bytes.is_empty())
|
||||
.map(parse_commit_receipt)
|
||||
.transpose()?;
|
||||
.map(serde_json::from_slice)
|
||||
.transpose()
|
||||
.map_err(|_| DiskError::FileCorrupt)?;
|
||||
let committed = receipt.is_some();
|
||||
let finished = regular_marker(&rollback.join(FINISHED))? || receipt.as_ref().is_some_and(|receipt| receipt.finished);
|
||||
let journal: Option<DeleteRecoveryJournal> = prepared
|
||||
@@ -1199,7 +1185,6 @@ struct PreparedDeleteReceiptEntry {
|
||||
journal: DeleteRecoveryJournal,
|
||||
sole_transaction: bool,
|
||||
had_old_data_dir: bool,
|
||||
prepared_synced: bool,
|
||||
rollback_dir_fd: Option<Arc<std::fs::File>>,
|
||||
file_path_dir: Option<Arc<std::fs::File>>,
|
||||
}
|
||||
@@ -1242,16 +1227,6 @@ fn read_child_regular_bytes(
|
||||
std::fs::read(fallback_path)
|
||||
}
|
||||
|
||||
fn stage_recovery_file(root: &Path, bytes: &[u8], sync: bool) -> std::io::Result<tempfile::NamedTempFile> {
|
||||
std::fs::create_dir_all(root)?;
|
||||
let mut file = tempfile::NamedTempFile::new_in(root)?;
|
||||
std::io::Write::write_all(file.as_file_mut(), bytes)?;
|
||||
if sync && !bytes.is_empty() {
|
||||
file.as_file().sync_data()?;
|
||||
}
|
||||
Ok(file)
|
||||
}
|
||||
|
||||
#[cfg(unix)]
|
||||
fn create_and_write_child_file(
|
||||
parent_fd: Option<&std::fs::File>,
|
||||
@@ -1563,7 +1538,11 @@ impl LocalDisk {
|
||||
.push(serde_json::from_slice::<DeleteRecoveryJournal>(&bytes).map_err(|_| DiskError::FileCorrupt)?);
|
||||
}
|
||||
if let Some(bytes) = receipt.filter(|bytes| !bytes.is_empty()) {
|
||||
records.push(parse_commit_receipt(&bytes)?.journal);
|
||||
records.push(
|
||||
serde_json::from_slice::<CommitReceipt>(&bytes)
|
||||
.map_err(|_| DiskError::FileCorrupt)?
|
||||
.journal,
|
||||
);
|
||||
}
|
||||
for record in records {
|
||||
if record.protocol_version != DELETE_RECOVERY_PROTOCOL_VERSION
|
||||
@@ -1893,40 +1872,34 @@ impl LocalDisk {
|
||||
let record_destination = destination.clone();
|
||||
let staging_root = temporary;
|
||||
let lease = os::acquire_namespace_mutation_lease_with_owner(&destination, owner.clone()).await;
|
||||
let reserve = self.delete_reserve.clone();
|
||||
let (staged, target) = os::run_blocking_namespace_operation_with_sync(lease, sync, move || {
|
||||
let staged = reserve.retry_after_enospc(|| {
|
||||
std::fs::create_dir_all(&staging_root)?;
|
||||
let staged = Arc::new(
|
||||
tempfile::Builder::new()
|
||||
.prefix("delete-preparation-")
|
||||
.tempdir_in(&staging_root)?,
|
||||
);
|
||||
let missing_marker = original.is_none() && target.is_some();
|
||||
for (name, bytes) in [
|
||||
(STORAGE_FORMAT_FILE_BACKUP, original.as_deref()),
|
||||
(TARGET, target.as_deref()),
|
||||
(PREPARED, Some(journal.as_slice())),
|
||||
(DELETE_MARKER_ROLLBACK_FILE, missing_marker.then_some(&[][..])),
|
||||
] {
|
||||
let Some(bytes) = bytes else {
|
||||
continue;
|
||||
};
|
||||
#[cfg(test)]
|
||||
run_owned_file_before_open(&record_destination.join(name));
|
||||
let mut file = std::fs::File::create(staged.path().join(name))?;
|
||||
std::io::Write::write_all(&mut file, bytes)?;
|
||||
if sync && !bytes.is_empty() {
|
||||
file.sync_data()?;
|
||||
}
|
||||
}
|
||||
if sync {
|
||||
os::fsync_dir_std(staged.path())?;
|
||||
}
|
||||
std::fs::create_dir_all(&staging_root)?;
|
||||
let staged = Arc::new(
|
||||
tempfile::Builder::new()
|
||||
.prefix("delete-preparation-")
|
||||
.tempdir_in(&staging_root)?,
|
||||
);
|
||||
let missing_marker = original.is_none() && target.is_some();
|
||||
for (name, bytes) in [
|
||||
(STORAGE_FORMAT_FILE_BACKUP, original.as_deref()),
|
||||
(TARGET, target.as_deref()),
|
||||
(PREPARED, Some(journal.as_slice())),
|
||||
(DELETE_MARKER_ROLLBACK_FILE, missing_marker.then_some(&[][..])),
|
||||
] {
|
||||
let Some(bytes) = bytes else {
|
||||
continue;
|
||||
};
|
||||
#[cfg(test)]
|
||||
preparation_io_fault::run(&record_destination)?;
|
||||
Ok(staged)
|
||||
})?;
|
||||
run_owned_file_before_open(&record_destination.join(name));
|
||||
let mut file = std::fs::File::create(staged.path().join(name))?;
|
||||
std::io::Write::write_all(&mut file, bytes)?;
|
||||
if sync && !bytes.is_empty() {
|
||||
file.sync_data()?;
|
||||
}
|
||||
}
|
||||
if sync {
|
||||
os::fsync_dir_std(staged.path())?;
|
||||
}
|
||||
Ok((staged, target))
|
||||
})
|
||||
.await
|
||||
@@ -2839,7 +2812,6 @@ impl LocalDisk {
|
||||
let object_str = object.to_owned();
|
||||
|
||||
let recovery_scope = recovery_operation_scope();
|
||||
let reserve = self.delete_reserve.clone();
|
||||
type FusedStage1Outcome = Option<(Option<Uuid>, PathBuf, PreparedDeleteReceiptEntry)>;
|
||||
let res = os::run_blocking_namespace_operation_with_sync(
|
||||
lease,
|
||||
@@ -2946,7 +2918,11 @@ impl LocalDisk {
|
||||
);
|
||||
}
|
||||
if let Some(bytes) = receipt.filter(|bytes| !bytes.is_empty()) {
|
||||
records.push(parse_commit_receipt(&bytes)?.journal);
|
||||
records.push(
|
||||
serde_json::from_slice::<CommitReceipt>(&bytes)
|
||||
.map_err(|_| DiskError::FileCorrupt)?
|
||||
.journal,
|
||||
);
|
||||
}
|
||||
for record in records {
|
||||
if record.protocol_version != DELETE_RECOVERY_PROTOCOL_VERSION
|
||||
@@ -3000,61 +2976,52 @@ impl LocalDisk {
|
||||
Ok(None) => return Ok(Ok(None)),
|
||||
Err(err) => return Ok(Err(err)),
|
||||
};
|
||||
let (staged, rollback_dir_fd) = reserve.retry_after_enospc(|| {
|
||||
let staged = match tempfile::Builder::new().prefix("delete-prep-").tempdir_in(&staging_root) {
|
||||
Ok(dir) => dir,
|
||||
Err(err) if err.kind() == ErrorKind::NotFound => {
|
||||
std::fs::create_dir_all(&staging_root)?;
|
||||
tempfile::Builder::new().prefix("delete-prep-").tempdir_in(&staging_root)?
|
||||
}
|
||||
Err(err) => return Err(err),
|
||||
};
|
||||
let staged_dir = staged.path().to_path_buf();
|
||||
#[cfg(unix)]
|
||||
let rollback_dir_fd = open_dir_fd(&staged_dir).ok().map(Arc::new);
|
||||
#[cfg(not(unix))]
|
||||
let rollback_dir_fd: Option<Arc<std::fs::File>> = None;
|
||||
let backup_path = staged_dir.join(STORAGE_FORMAT_FILE_BACKUP);
|
||||
if link_child_files(
|
||||
file_path_dir.as_deref(),
|
||||
STORAGE_FORMAT_FILE,
|
||||
&xl_path,
|
||||
rollback_dir_fd.as_deref(),
|
||||
STORAGE_FORMAT_FILE_BACKUP,
|
||||
&backup_path,
|
||||
)
|
||||
.is_err()
|
||||
{
|
||||
let file = create_and_write_child_file(
|
||||
rollback_dir_fd.as_deref(),
|
||||
STORAGE_FORMAT_FILE_BACKUP,
|
||||
&backup_path,
|
||||
&buf,
|
||||
)?;
|
||||
if sync && !buf.is_empty() {
|
||||
file.sync_data()?;
|
||||
}
|
||||
let staged = match tempfile::Builder::new().prefix("delete-prep-").tempdir_in(&staging_root) {
|
||||
Ok(dir) => dir,
|
||||
Err(err) if err.kind() == ErrorKind::NotFound => {
|
||||
std::fs::create_dir_all(&staging_root)?;
|
||||
tempfile::Builder::new().prefix("delete-prep-").tempdir_in(&staging_root)?
|
||||
}
|
||||
let mut written_files = Vec::with_capacity(2);
|
||||
for (name, bytes) in [(TARGET, target.as_deref()), (PREPARED, Some(journal_raw.as_slice()))] {
|
||||
let Some(bytes) = bytes else {
|
||||
continue;
|
||||
};
|
||||
let file = create_and_write_child_file(rollback_dir_fd.as_deref(), name, &staged_dir.join(name), bytes)?;
|
||||
if sync && !bytes.is_empty() {
|
||||
written_files.push(file);
|
||||
}
|
||||
}
|
||||
for file in written_files {
|
||||
Err(err) => return Err(err),
|
||||
};
|
||||
let staged_dir = staged.path().to_path_buf();
|
||||
#[cfg(unix)]
|
||||
let rollback_dir_fd = open_dir_fd(&staged_dir).ok().map(Arc::new);
|
||||
#[cfg(not(unix))]
|
||||
let rollback_dir_fd: Option<Arc<std::fs::File>> = None;
|
||||
let backup_path = staged_dir.join(STORAGE_FORMAT_FILE_BACKUP);
|
||||
if link_child_files(
|
||||
file_path_dir.as_deref(),
|
||||
STORAGE_FORMAT_FILE,
|
||||
&xl_path,
|
||||
rollback_dir_fd.as_deref(),
|
||||
STORAGE_FORMAT_FILE_BACKUP,
|
||||
&backup_path,
|
||||
)
|
||||
.is_err()
|
||||
{
|
||||
let file =
|
||||
create_and_write_child_file(rollback_dir_fd.as_deref(), STORAGE_FORMAT_FILE_BACKUP, &backup_path, &buf)?;
|
||||
if sync && !buf.is_empty() {
|
||||
file.sync_data()?;
|
||||
}
|
||||
if sync {
|
||||
os::fsync_directory_handle_std(&staged_dir, rollback_dir_fd.as_deref())?;
|
||||
}
|
||||
let mut written_files = Vec::with_capacity(2);
|
||||
for (name, bytes) in [(TARGET, target.as_deref()), (PREPARED, Some(journal_raw.as_slice()))] {
|
||||
let Some(bytes) = bytes else {
|
||||
continue;
|
||||
};
|
||||
let file = create_and_write_child_file(rollback_dir_fd.as_deref(), name, &staged_dir.join(name), bytes)?;
|
||||
if sync && !bytes.is_empty() {
|
||||
written_files.push(file);
|
||||
}
|
||||
#[cfg(test)]
|
||||
preparation_io_fault::run(&destination)?;
|
||||
Ok((staged, rollback_dir_fd))
|
||||
})?;
|
||||
}
|
||||
for file in written_files {
|
||||
file.sync_data()?;
|
||||
}
|
||||
if sync {
|
||||
os::fsync_directory_handle_std(&staged_dir, rollback_dir_fd.as_deref())?;
|
||||
}
|
||||
let tx_name = transaction.to_string();
|
||||
if !destination_missing {
|
||||
match std::fs::symlink_metadata(&destination) {
|
||||
@@ -3073,9 +3040,7 @@ impl LocalDisk {
|
||||
}
|
||||
let staged_dir = staged.keep();
|
||||
os::record_native_publication(&destination);
|
||||
if let Err(err) = reserve
|
||||
.retry_after_enospc(|| rename_into_dir_fd(&staged_dir, file_path_dir.as_deref(), &tx_name, &destination))
|
||||
{
|
||||
if let Err(err) = rename_into_dir_fd(&staged_dir, file_path_dir.as_deref(), &tx_name, &destination) {
|
||||
let _ = std::fs::remove_dir_all(&staged_dir);
|
||||
return Err(err);
|
||||
}
|
||||
@@ -3093,16 +3058,14 @@ impl LocalDisk {
|
||||
let reserved_name = format!("{RESERVED_DELETE_DATA_DIR_MARKER_PREFIX}{transaction}");
|
||||
let reserved_marker = old_path.join(&reserved_name);
|
||||
let old_dir_file = open_child_dir_fd(file_path_dir.as_deref(), &old_dir_name, &old_path);
|
||||
let reserve_res = reserve.retry_after_enospc(|| -> std::io::Result<()> {
|
||||
#[cfg(test)]
|
||||
preparation_io_fault::run(&reserved_marker)?;
|
||||
let reserve_res = (|| -> std::io::Result<()> {
|
||||
let marker = create_and_write_child_file(old_dir_file.as_ref(), &reserved_name, &reserved_marker, &[])?;
|
||||
if sync {
|
||||
marker.sync_all()?;
|
||||
os::fsync_directory_handle_std(&old_path, old_dir_file.as_ref())?;
|
||||
}
|
||||
Ok(())
|
||||
});
|
||||
})();
|
||||
if let Err(err) = reserve_res {
|
||||
let _ = unlink_child_file(old_dir_file.as_ref(), &reserved_name, &reserved_marker);
|
||||
let _ = std::fs::remove_dir_all(&destination);
|
||||
@@ -3124,20 +3087,19 @@ impl LocalDisk {
|
||||
Err(_) => false,
|
||||
};
|
||||
let write_res = if linked {
|
||||
let rename_res = reserve.retry_after_enospc(|| {
|
||||
rename_into_dir_fd(&linked_xl, file_path_dir.as_deref(), STORAGE_FORMAT_FILE, &xl_path)
|
||||
});
|
||||
let rename_res = rename_into_dir_fd(&linked_xl, file_path_dir.as_deref(), STORAGE_FORMAT_FILE, &xl_path);
|
||||
if rename_res.is_err() {
|
||||
let _ = std::fs::remove_file(&linked_xl);
|
||||
}
|
||||
rename_res
|
||||
} else {
|
||||
(|| -> std::io::Result<()> {
|
||||
let staged_xl =
|
||||
reserve.retry_after_enospc(|| stage_recovery_file(&staging_root, target_bytes, sync))?;
|
||||
reserve.retry_after_enospc(|| {
|
||||
rename_into_dir_fd(staged_xl.path(), file_path_dir.as_deref(), STORAGE_FORMAT_FILE, &xl_path)
|
||||
})
|
||||
let mut staged_xl = tempfile::NamedTempFile::new_in(&staging_root)?;
|
||||
std::io::Write::write_all(staged_xl.as_file_mut(), target_bytes)?;
|
||||
if sync && !target_bytes.is_empty() {
|
||||
staged_xl.as_file().sync_data()?;
|
||||
}
|
||||
rename_into_dir_fd(staged_xl.path(), file_path_dir.as_deref(), STORAGE_FORMAT_FILE, &xl_path)
|
||||
})()
|
||||
};
|
||||
if let Err(err) = write_res {
|
||||
@@ -3188,7 +3150,6 @@ impl LocalDisk {
|
||||
journal,
|
||||
sole_transaction,
|
||||
had_old_data_dir,
|
||||
prepared_synced: sync,
|
||||
rollback_dir_fd,
|
||||
file_path_dir,
|
||||
},
|
||||
@@ -3252,7 +3213,6 @@ impl LocalDisk {
|
||||
entry.journal,
|
||||
entry.sole_transaction,
|
||||
entry.had_old_data_dir,
|
||||
entry.prepared_synced,
|
||||
entry.rollback_dir_fd,
|
||||
entry.file_path_dir,
|
||||
)),
|
||||
@@ -3282,79 +3242,38 @@ impl LocalDisk {
|
||||
let volume_str = volume.to_owned();
|
||||
let object_str = object.to_owned();
|
||||
|
||||
let reserve = self.delete_reserve.clone();
|
||||
type PublishDeleteFastReturn = (DeleteRecoveryJournal, bool, bool, Option<(PathBuf, PublishedDeleteReceiptEntry)>);
|
||||
let res = os::run_blocking_namespace_operation_with_sync(
|
||||
lease,
|
||||
sync,
|
||||
move || -> std::io::Result<Result<Option<PublishDeleteFastReturn>>> {
|
||||
if let Some((
|
||||
journal_raw,
|
||||
journal,
|
||||
sole_transaction,
|
||||
had_old_data_dir,
|
||||
prepared_synced,
|
||||
rollback_dir_fd,
|
||||
file_path_dir,
|
||||
)) = cached_receipt
|
||||
if let Some((journal_raw, journal, sole_transaction, had_old_data_dir, rollback_dir_fd, file_path_dir)) =
|
||||
cached_receipt
|
||||
&& child_regular_file_exists(rollback_dir_fd.as_deref(), PREPARED, &rollback.join(PREPARED))
|
||||
{
|
||||
// Share the immutable Stage 1 PREPARED inode; any later receipt update must replace the entry by rename.
|
||||
let linked = if !sync || prepared_synced {
|
||||
let tmp_receipt = staging_root.join(format!("receipt-{transaction}.tmp"));
|
||||
let receipt_bytes = format_commit_receipt_bytes(&journal_raw, false);
|
||||
let write_res = (|| -> std::io::Result<()> {
|
||||
let mut file = std::fs::File::create(&tmp_receipt)?;
|
||||
std::io::Write::write_all(&mut file, &receipt_bytes)?;
|
||||
if sync && !receipt_bytes.is_empty() {
|
||||
file.sync_data()?;
|
||||
}
|
||||
os::record_native_publication(&receipt_path);
|
||||
reserve
|
||||
.retry_after_enospc(|| {
|
||||
#[cfg(test)]
|
||||
preparation_io_fault::run(&receipt_path)?;
|
||||
link_child_files(
|
||||
rollback_dir_fd.as_deref(),
|
||||
PREPARED,
|
||||
&rollback.join(PREPARED),
|
||||
rollback_dir_fd.as_deref(),
|
||||
DELETE_QUORUM_COMMITTED_FILE,
|
||||
&receipt_path,
|
||||
)
|
||||
})
|
||||
.is_ok()
|
||||
} else {
|
||||
false
|
||||
};
|
||||
if linked {
|
||||
rename_into_dir_fd(
|
||||
&tmp_receipt,
|
||||
rollback_dir_fd.as_deref(),
|
||||
DELETE_QUORUM_COMMITTED_FILE,
|
||||
&receipt_path,
|
||||
)?;
|
||||
if sync {
|
||||
os::fsync_directory_handle_std(&rollback, rollback_dir_fd.as_deref())?;
|
||||
}
|
||||
} else {
|
||||
let tmp_receipt = staging_root.join(format!("receipt-{transaction}.tmp"));
|
||||
let receipt_bytes = format_commit_receipt_bytes(&journal_raw, false);
|
||||
let write_res = (|| -> std::io::Result<()> {
|
||||
reserve.retry_after_enospc(|| {
|
||||
let mut file = std::fs::File::create(&tmp_receipt)?;
|
||||
std::io::Write::write_all(&mut file, &receipt_bytes)?;
|
||||
if sync && !receipt_bytes.is_empty() {
|
||||
file.sync_data()?;
|
||||
}
|
||||
#[cfg(test)]
|
||||
preparation_io_fault::run(&receipt_path)?;
|
||||
Ok(())
|
||||
})?;
|
||||
os::record_native_publication(&receipt_path);
|
||||
reserve.retry_after_enospc(|| {
|
||||
rename_into_dir_fd(
|
||||
&tmp_receipt,
|
||||
rollback_dir_fd.as_deref(),
|
||||
DELETE_QUORUM_COMMITTED_FILE,
|
||||
&receipt_path,
|
||||
)
|
||||
})?;
|
||||
if sync {
|
||||
os::fsync_directory_handle_std(&rollback, rollback_dir_fd.as_deref())?;
|
||||
}
|
||||
Ok(())
|
||||
})();
|
||||
if let Err(err) = write_res {
|
||||
let _ = std::fs::remove_file(&tmp_receipt);
|
||||
return Err(err);
|
||||
}
|
||||
Ok(())
|
||||
})();
|
||||
if let Err(err) = write_res {
|
||||
let _ = std::fs::remove_file(&tmp_receipt);
|
||||
return Err(err);
|
||||
}
|
||||
let published_entry = bucket_path.map(|bucket_path| {
|
||||
(
|
||||
@@ -3439,9 +3358,14 @@ impl LocalDisk {
|
||||
finished: false,
|
||||
};
|
||||
let receipt_bytes = serde_json::to_vec(&receipt).map_err(std::io::Error::other)?;
|
||||
let staged = reserve.retry_after_enospc(|| stage_recovery_file(&staging_root, &receipt_bytes, sync))?;
|
||||
std::fs::create_dir_all(&staging_root)?;
|
||||
let mut staged = tempfile::NamedTempFile::new_in(&staging_root)?;
|
||||
std::io::Write::write_all(staged.as_file_mut(), &receipt_bytes)?;
|
||||
if sync && !receipt_bytes.is_empty() {
|
||||
staged.as_file().sync_data()?;
|
||||
}
|
||||
os::record_native_publication(&receipt_path);
|
||||
reserve.retry_after_enospc(|| std::fs::rename(staged.path(), &receipt_path))?;
|
||||
std::fs::rename(staged.path(), &receipt_path)?;
|
||||
if sync {
|
||||
os::fsync_dir_std(&rollback)?;
|
||||
}
|
||||
@@ -3593,7 +3517,6 @@ impl LocalDisk {
|
||||
.collect()
|
||||
};
|
||||
|
||||
let reserve = self.delete_reserve.clone();
|
||||
let res = os::run_blocking_namespace_operation_with_sync(lease, sync, move || -> std::io::Result<()> {
|
||||
let persist_receipt = |finished: bool| -> std::io::Result<()> {
|
||||
let receipt_bytes = match &cached_journal_raw {
|
||||
@@ -3608,9 +3531,14 @@ impl LocalDisk {
|
||||
serde_json::to_vec(&receipt).map_err(std::io::Error::other)?
|
||||
}
|
||||
};
|
||||
let staged_receipt = reserve.retry_after_enospc(|| stage_recovery_file(&staging_root, &receipt_bytes, sync))?;
|
||||
std::fs::create_dir_all(&staging_root)?;
|
||||
let mut staged_receipt = tempfile::NamedTempFile::new_in(&staging_root)?;
|
||||
std::io::Write::write_all(staged_receipt.as_file_mut(), &receipt_bytes)?;
|
||||
if sync && !receipt_bytes.is_empty() {
|
||||
staged_receipt.as_file().sync_data()?;
|
||||
}
|
||||
os::record_native_publication(&receipt_path);
|
||||
reserve.retry_after_enospc(|| std::fs::rename(staged_receipt.path(), &receipt_path))?;
|
||||
std::fs::rename(staged_receipt.path(), &receipt_path)?;
|
||||
if sync {
|
||||
let _ = os::fsync_dir_std(&rollback);
|
||||
}
|
||||
@@ -3796,11 +3724,6 @@ impl LocalDisk {
|
||||
for claim in claims {
|
||||
claim.finish(ok);
|
||||
}
|
||||
if ok && !self.delete_reserve.is_armed() {
|
||||
let reserve = self.delete_reserve.clone();
|
||||
let lease = os::acquire_namespace_mutation_lease_with_owner(self.io_root(), None).await;
|
||||
let _ = os::run_blocking_namespace_operation_with_sync(lease, true, move || reserve.replenish()).await;
|
||||
}
|
||||
res.map(|()| true).map_err(|err| to_file_error(err).into())
|
||||
}
|
||||
}
|
||||
@@ -3936,43 +3859,6 @@ mod fused_prepare_checkpoint {
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod preparation_io_fault {
|
||||
use std::collections::HashMap;
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::sync::{LazyLock, Mutex};
|
||||
|
||||
type Hook = Box<dyn FnOnce() -> std::io::Result<()> + Send>;
|
||||
static HOOKS: LazyLock<Mutex<HashMap<PathBuf, Hook>>> = LazyLock::new(|| Mutex::new(HashMap::new()));
|
||||
|
||||
pub(super) struct Guard(PathBuf);
|
||||
|
||||
impl Drop for Guard {
|
||||
fn drop(&mut self) {
|
||||
HOOKS.lock().expect("checkpoint hooks").remove(&self.0);
|
||||
}
|
||||
}
|
||||
|
||||
pub(super) fn install(root: &Path, hook: impl FnOnce() -> std::io::Result<()> + Send + 'static) -> Guard {
|
||||
assert!(
|
||||
HOOKS
|
||||
.lock()
|
||||
.expect("checkpoint hooks")
|
||||
.insert(root.to_path_buf(), Box::new(hook))
|
||||
.is_none()
|
||||
);
|
||||
Guard(root.to_path_buf())
|
||||
}
|
||||
|
||||
pub(super) fn run(root: &Path) -> std::io::Result<()> {
|
||||
let hook = HOOKS.lock().expect("checkpoint hooks").remove(root);
|
||||
match hook {
|
||||
Some(hook) => hook(),
|
||||
None => Ok(()),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
@@ -8564,9 +8450,6 @@ mod tests {
|
||||
journal: journal.clone(),
|
||||
finished: true,
|
||||
};
|
||||
fs::remove_file(rollback.join(DELETE_QUORUM_COMMITTED_FILE))
|
||||
.await
|
||||
.expect("remove receipt before replacement");
|
||||
fs::write(
|
||||
rollback.join(DELETE_QUORUM_COMMITTED_FILE),
|
||||
serde_json::to_vec(&finished_receipt).expect("serialize finished receipt"),
|
||||
@@ -8601,55 +8484,6 @@ mod tests {
|
||||
|
||||
#[tokio::test]
|
||||
async fn delete_recovery_fused_unversioned_physical_delete_retains_receipt_until_finalize() {
|
||||
assert_unversioned_delete_reserve_recovery(None).await;
|
||||
}
|
||||
|
||||
#[cfg(unix)]
|
||||
#[tokio::test]
|
||||
async fn delete_recovery_fused_enospc_recovers_scratch_marker_and_receipt_without_replaying_transaction() {
|
||||
for stage in ["scratch", "marker", "receipt"] {
|
||||
assert_unversioned_delete_reserve_recovery(Some(stage)).await;
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(unix)]
|
||||
#[tokio::test]
|
||||
async fn delete_recovery_fallback_enospc_recovers_before_activation_and_preserves_original() {
|
||||
for kind in [ErrorKind::StorageFull, ErrorKind::PermissionDenied] {
|
||||
let (_temp, disk, info, _) = fixture().await;
|
||||
let root = disk.io_get_object_path("recovery", "object").expect("object path");
|
||||
let original = std::fs::read(root.join(STORAGE_FORMAT_FILE)).expect("original metadata");
|
||||
let transaction = Uuid::new_v4();
|
||||
let destination = root.join(transaction.to_string());
|
||||
let _fault = preparation_io_fault::install(&destination, move || Err(std::io::Error::from(kind)));
|
||||
let result = disk
|
||||
.prepare_delete_recovery(
|
||||
"recovery",
|
||||
"object",
|
||||
Some(&original),
|
||||
&[info],
|
||||
None,
|
||||
DeletePreparationContext {
|
||||
transaction,
|
||||
bucket_incarnation: None,
|
||||
namespace_owner: None,
|
||||
},
|
||||
)
|
||||
.await;
|
||||
if kind == ErrorKind::StorageFull {
|
||||
assert!(result.is_ok(), "scratch ENOSPC must recover: {result:?}");
|
||||
assert!(destination.join(PREPARED).is_file());
|
||||
assert!(!disk.delete_reserve.is_armed());
|
||||
} else {
|
||||
assert!(result.is_err(), "non-capacity failures must not retry");
|
||||
assert!(!destination.exists());
|
||||
assert!(disk.delete_reserve.is_armed());
|
||||
}
|
||||
assert_eq!(std::fs::read(root.join(STORAGE_FORMAT_FILE)).expect("unchanged metadata"), original);
|
||||
}
|
||||
}
|
||||
|
||||
async fn assert_unversioned_delete_reserve_recovery(fault: Option<&str>) {
|
||||
let (_temp, disk, mut info, _) = fixture().await;
|
||||
let root = disk.io_get_object_path("recovery", "object").expect("object path");
|
||||
|
||||
@@ -8672,35 +8506,7 @@ mod tests {
|
||||
.await
|
||||
.expect("write xl.meta");
|
||||
|
||||
#[cfg(unix)]
|
||||
let reserve_path = disk.io_root().join(".rustfs.sys/delete-reserve");
|
||||
#[cfg(unix)]
|
||||
assert_eq!(
|
||||
std::fs::metadata(&reserve_path).expect("initialized reserve").len(),
|
||||
super::super::delete_reserve::RESERVE_BYTES
|
||||
);
|
||||
let transaction = Uuid::new_v4();
|
||||
let rollback = root.join(transaction.to_string());
|
||||
let _fault = fault.map(|stage| {
|
||||
let path = match stage {
|
||||
"scratch" => rollback.clone(),
|
||||
"marker" => root
|
||||
.join(data_dir.to_string())
|
||||
.join(format!("{RESERVED_DELETE_DATA_DIR_MARKER_PREFIX}{transaction}")),
|
||||
"receipt" => rollback.join(DELETE_QUORUM_COMMITTED_FILE),
|
||||
_ => panic!("unknown fault stage"),
|
||||
};
|
||||
preparation_io_fault::install(&path, || Err(std::io::Error::from(ErrorKind::StorageFull)))
|
||||
});
|
||||
#[cfg(unix)]
|
||||
let checkpoints = Arc::new(std::sync::atomic::AtomicUsize::new(0));
|
||||
#[cfg(unix)]
|
||||
let _checkpoint = {
|
||||
let checkpoints = checkpoints.clone();
|
||||
fused_prepare_checkpoint::install(&root, move || {
|
||||
checkpoints.fetch_add(1, std::sync::atomic::Ordering::Relaxed);
|
||||
})
|
||||
};
|
||||
let opts = DeleteOptions {
|
||||
recoverable_delete: true,
|
||||
..Default::default()
|
||||
@@ -8710,32 +8516,6 @@ mod tests {
|
||||
.await
|
||||
.expect("fused unversioned physical delete must succeed");
|
||||
assert_eq!(fused, Some(()));
|
||||
#[cfg(unix)]
|
||||
assert_eq!(
|
||||
std::fs::metadata(&reserve_path).expect("released reserve").len(),
|
||||
if matches!(fault, Some("scratch" | "marker")) {
|
||||
0
|
||||
} else {
|
||||
super::super::delete_reserve::RESERVE_BYTES
|
||||
},
|
||||
"only an actual capacity failure should consume the reserve"
|
||||
);
|
||||
#[cfg(unix)]
|
||||
assert_eq!(
|
||||
checkpoints.load(std::sync::atomic::Ordering::Relaxed),
|
||||
1,
|
||||
"transaction publication must not replay"
|
||||
);
|
||||
assert!(
|
||||
std::fs::read_dir(disk.io_root().join(RUSTFS_META_TMP_BUCKET))
|
||||
.expect("staging root")
|
||||
.all(|entry| !entry
|
||||
.expect("staging entry")
|
||||
.file_name()
|
||||
.to_string_lossy()
|
||||
.starts_with("delete-prep-")),
|
||||
"failed scratch attempts must be cleaned up"
|
||||
);
|
||||
|
||||
let (journal, sole_transaction, target_satisfied) = disk
|
||||
.try_publish_prepared_delete_local("recovery", "object", transaction, None)
|
||||
@@ -8745,15 +8525,6 @@ mod tests {
|
||||
assert!(sole_transaction);
|
||||
assert!(target_satisfied);
|
||||
assert!(journal.target_digest.is_none());
|
||||
#[cfg(unix)]
|
||||
assert_eq!(
|
||||
std::fs::metadata(&reserve_path).expect("reserve after receipt").len(),
|
||||
if fault.is_some() {
|
||||
0
|
||||
} else {
|
||||
super::super::delete_reserve::RESERVE_BYTES
|
||||
}
|
||||
);
|
||||
|
||||
let rollback = root.join(transaction.to_string());
|
||||
assert!(rollback.join(DELETE_QUORUM_COMMITTED_FILE).is_file());
|
||||
@@ -8763,11 +8534,6 @@ mod tests {
|
||||
.await
|
||||
.expect("local finalize must succeed");
|
||||
assert!(finalized);
|
||||
#[cfg(unix)]
|
||||
assert_eq!(
|
||||
std::fs::metadata(&reserve_path).expect("replenished reserve").len(),
|
||||
super::super::delete_reserve::RESERVE_BYTES
|
||||
);
|
||||
assert!(!rollback.exists(), "rollback directory must be pruned");
|
||||
assert!(!root.join(data_dir.to_string()).exists(), "physical data_dir must be removed");
|
||||
assert!(!root.exists(), "empty object directory must be pruned");
|
||||
@@ -8852,265 +8618,6 @@ mod tests {
|
||||
);
|
||||
assert!(!rollback.exists());
|
||||
}
|
||||
|
||||
#[cfg(unix)]
|
||||
#[tokio::test]
|
||||
async fn delete_recovery_stage2_hardlinks_prepared_journal_and_falls_back_on_conflict() {
|
||||
use std::os::unix::fs::MetadataExt;
|
||||
|
||||
let _strict = durability_mode_override::set(DurabilityMode::Strict);
|
||||
let (_temp, disk, info, _) = fixture().await;
|
||||
let root = disk.io_get_object_path("recovery", "object").expect("object path");
|
||||
let opts = DeleteOptions {
|
||||
recoverable_delete: true,
|
||||
..Default::default()
|
||||
};
|
||||
|
||||
#[cfg(target_os = "linux")]
|
||||
{
|
||||
let proc_vol = format!("{}/recovery", disk.io_root().display());
|
||||
assert!(matches!(
|
||||
disk.delete_version(&proc_vol, "object", info.clone(), false, opts.clone())
|
||||
.await,
|
||||
Err(DiskError::InvalidPath)
|
||||
));
|
||||
}
|
||||
|
||||
let tx1 = Uuid::new_v4();
|
||||
assert_eq!(
|
||||
disk.try_fused_recoverable_delete_version("recovery", "object", &info, &opts, tx1, None)
|
||||
.await
|
||||
.expect("fused prepare tx1"),
|
||||
Some(())
|
||||
);
|
||||
let (journal1, sole1, satisfied1) = disk
|
||||
.try_publish_prepared_delete_local("recovery", "object", tx1, None)
|
||||
.await
|
||||
.expect("publish tx1")
|
||||
.expect("tx1 satisfied");
|
||||
assert!(sole1 && satisfied1);
|
||||
|
||||
let rollback1 = root.join(tx1.to_string());
|
||||
assert!(
|
||||
os::fsync_dir_recorder::was_fsynced(&rollback1),
|
||||
"strict Stage 2 hardlink must fsync rollback directory"
|
||||
);
|
||||
let prep_meta1 = std::fs::metadata(rollback1.join(PREPARED)).expect("prepared metadata");
|
||||
let rec_meta1 = std::fs::metadata(rollback1.join(DELETE_QUORUM_COMMITTED_FILE)).expect("receipt metadata");
|
||||
assert_eq!(
|
||||
(prep_meta1.dev(), prep_meta1.ino()),
|
||||
(rec_meta1.dev(), rec_meta1.ino()),
|
||||
"Stage 2 fast path must hardlink PREPARED to DELETE_QUORUM_COMMITTED_FILE"
|
||||
);
|
||||
let raw_receipt1 = std::fs::read(rollback1.join(DELETE_QUORUM_COMMITTED_FILE)).expect("read bare receipt");
|
||||
let parsed_bare = parse_commit_receipt(&raw_receipt1).expect("parse bare journal receipt");
|
||||
assert!(!parsed_bare.finished && parsed_bare.journal == journal1);
|
||||
let wrapped_true = format_commit_receipt_bytes(&raw_receipt1, true);
|
||||
let parsed_wrapped = parse_commit_receipt(&wrapped_true).expect("parse wrapped finished receipt");
|
||||
assert!(parsed_wrapped.finished && parsed_wrapped.journal == journal1);
|
||||
assert!(matches!(parse_commit_receipt(b"{\"invalid\":1}"), Err(DiskError::FileCorrupt)));
|
||||
assert_eq!(
|
||||
disk.inspect_delete_transaction("recovery", "object", tx1)
|
||||
.await
|
||||
.expect("inspect QuorumCommitted")
|
||||
.state,
|
||||
DeleteRecoveryState::QuorumCommitted
|
||||
);
|
||||
|
||||
// Verify DiscoverySession::advance_bounded (:212) extracts the bound logical key ("directory/")
|
||||
// from a bare-journal receipt when PREPARED is absent (physical dir name is "directory").
|
||||
let dir_root = disk.io_get_object_path("recovery", "directory/").expect("dir key path");
|
||||
let mut dir_info = info.clone();
|
||||
dir_info.name = "directory/".to_owned();
|
||||
let mut dir_meta = FileMeta::new();
|
||||
dir_meta.add_version(dir_info.clone()).expect("add dir version");
|
||||
fs::create_dir_all(&dir_root).await.expect("create dir_root");
|
||||
fs::write(dir_root.join(STORAGE_FORMAT_FILE), dir_meta.marshal_msg().expect("marshal dir meta"))
|
||||
.await
|
||||
.expect("write dir xl.meta");
|
||||
let tx_dir = Uuid::new_v4();
|
||||
assert_eq!(
|
||||
disk.try_fused_recoverable_delete_version("recovery", "directory/", &dir_info, &opts, tx_dir, None)
|
||||
.await
|
||||
.expect("fused prepare tx_dir"),
|
||||
Some(())
|
||||
);
|
||||
let (journal_dir, _, _) = disk
|
||||
.try_publish_prepared_delete_local("recovery", "directory/", tx_dir, None)
|
||||
.await
|
||||
.expect("publish tx_dir")
|
||||
.expect("tx_dir satisfied");
|
||||
let rollback_dir = dir_root.join(tx_dir.to_string());
|
||||
fs::remove_file(rollback_dir.join(PREPARED))
|
||||
.await
|
||||
.expect("unlink dir PREPARED");
|
||||
fs::remove_file(rollback_dir.join(STORAGE_FORMAT_FILE_BACKUP))
|
||||
.await
|
||||
.expect("unlink dir backup");
|
||||
let discovered_dir = disk
|
||||
.discover_delete_transactions("recovery", None, None, 16)
|
||||
.await
|
||||
.expect("bucket-wide discovery of directory/ bare-journal receipt");
|
||||
assert!(
|
||||
discovered_dir
|
||||
.transactions
|
||||
.iter()
|
||||
.any(|id| id.object == "directory/" && id.transaction == tx_dir),
|
||||
"DiscoverySession must parse logical key 'directory/' from bare-journal receipt"
|
||||
);
|
||||
assert!(
|
||||
disk.try_finish_and_finalize_prepared_delete_local("recovery", "directory/", tx_dir, &journal_dir, None)
|
||||
.await
|
||||
.expect("finalize tx_dir")
|
||||
);
|
||||
|
||||
// Removing PREPARED and backup while retaining the bare-journal receipt infers CommitFinished
|
||||
// and exercises all bare-journal parse_commit_receipt sites (bucket-wide discovery + predecessor scans).
|
||||
fs::remove_file(rollback1.join(PREPARED)).await.expect("unlink PREPARED");
|
||||
fs::remove_file(rollback1.join(STORAGE_FORMAT_FILE_BACKUP))
|
||||
.await
|
||||
.expect("unlink backup");
|
||||
assert_eq!(
|
||||
disk.inspect_delete_transaction("recovery", "object", tx1)
|
||||
.await
|
||||
.expect("inspect CommitFinished from bare-journal receipt")
|
||||
.state,
|
||||
DeleteRecoveryState::CommitFinished
|
||||
);
|
||||
let discovered_bucket = disk
|
||||
.discover_delete_transactions("recovery", None, None, 16)
|
||||
.await
|
||||
.expect("bucket-wide discovery of bare-journal receipt");
|
||||
assert!(discovered_bucket.transactions.iter().any(|id| id.transaction == tx1));
|
||||
let preds = disk
|
||||
.delete_recovery_predecessors("recovery", "object", Uuid::new_v4(), None, None)
|
||||
.await
|
||||
.expect("predecessor scan of bare-journal receipt");
|
||||
assert_eq!(preds, vec![tx1]);
|
||||
|
||||
// Re-create xl.meta while tx1's bare-journal receipt is still present so Stage 1's predecessor scan
|
||||
// parses tx1's bare-journal receipt, and pre-populate tx2's receipt before Stage 2 to force EEXIST fallback.
|
||||
let mut meta = FileMeta::new();
|
||||
meta.add_version(info.clone()).expect("add version");
|
||||
fs::create_dir_all(&root).await.expect("recreate object dir");
|
||||
fs::write(root.join(STORAGE_FORMAT_FILE), meta.marshal_msg().expect("marshal meta"))
|
||||
.await
|
||||
.expect("write xl.meta");
|
||||
|
||||
let tx2 = Uuid::new_v4();
|
||||
assert_eq!(
|
||||
disk.try_fused_recoverable_delete_version("recovery", "object", &info, &opts, tx2, None)
|
||||
.await
|
||||
.expect("fused prepare tx2"),
|
||||
Some(())
|
||||
);
|
||||
let rollback2 = root.join(tx2.to_string());
|
||||
fs::write(rollback2.join(DELETE_QUORUM_COMMITTED_FILE), b"stale")
|
||||
.await
|
||||
.expect("pre-create conflicting receipt");
|
||||
let (journal2, sole2, satisfied2) = disk
|
||||
.try_publish_prepared_delete_local("recovery", "object", tx2, None)
|
||||
.await
|
||||
.expect("publish tx2 fallback")
|
||||
.expect("tx2 satisfied");
|
||||
assert!(!sole2 && satisfied2);
|
||||
assert_eq!(journal2.predecessors, vec![tx1]);
|
||||
let prep_meta2 = std::fs::metadata(rollback2.join(PREPARED)).expect("prepared2 metadata");
|
||||
let rec_meta2 = std::fs::metadata(rollback2.join(DELETE_QUORUM_COMMITTED_FILE)).expect("receipt2 metadata");
|
||||
assert_ne!(
|
||||
(prep_meta2.dev(), prep_meta2.ino()),
|
||||
(rec_meta2.dev(), rec_meta2.ino()),
|
||||
"EEXIST fallback must replace receipt with a distinct wrapper inode"
|
||||
);
|
||||
assert!(
|
||||
disk.try_finish_and_finalize_prepared_delete_local("recovery", "object", tx1, &journal1, None)
|
||||
.await
|
||||
.expect("finalize tx1")
|
||||
);
|
||||
assert!(
|
||||
disk.try_finish_and_finalize_prepared_delete_local("recovery", "object", tx2, &journal2, None)
|
||||
.await
|
||||
.expect("finalize tx2")
|
||||
);
|
||||
|
||||
// Prepare tx3 under Relaxed (`prepared_synced == false`), then escalate to Strict before Stage 2:
|
||||
// Stage 2 must not hardlink the unsynced PREPARED inode and must instead write a fresh synced receipt.
|
||||
fs::create_dir_all(&root).await.expect("recreate object dir for tx3");
|
||||
fs::write(root.join(STORAGE_FORMAT_FILE), meta.marshal_msg().expect("marshal meta"))
|
||||
.await
|
||||
.expect("write xl.meta for tx3");
|
||||
drop(_strict);
|
||||
let tx3 = Uuid::new_v4();
|
||||
{
|
||||
let _relaxed = durability_mode_override::set(DurabilityMode::Relaxed);
|
||||
assert_eq!(
|
||||
disk.try_fused_recoverable_delete_version("recovery", "object", &info, &opts, tx3, None)
|
||||
.await
|
||||
.expect("fused prepare tx3 relaxed"),
|
||||
Some(())
|
||||
);
|
||||
}
|
||||
let _strict_again = durability_mode_override::set(DurabilityMode::Strict);
|
||||
let (journal3, sole3, satisfied3) = disk
|
||||
.try_publish_prepared_delete_local("recovery", "object", tx3, None)
|
||||
.await
|
||||
.expect("publish tx3 escalated to strict")
|
||||
.expect("tx3 satisfied");
|
||||
assert!(sole3 && satisfied3);
|
||||
let rollback3 = root.join(tx3.to_string());
|
||||
let prep_meta3 = std::fs::metadata(rollback3.join(PREPARED)).expect("prepared3 metadata");
|
||||
let rec_meta3 = std::fs::metadata(rollback3.join(DELETE_QUORUM_COMMITTED_FILE)).expect("receipt3 metadata");
|
||||
assert_ne!(
|
||||
(prep_meta3.dev(), prep_meta3.ino()),
|
||||
(rec_meta3.dev(), rec_meta3.ino()),
|
||||
"unsynced Stage 1 preparation must not be hardlinked when Stage 2 requires sync"
|
||||
);
|
||||
assert!(
|
||||
disk.try_finish_and_finalize_prepared_delete_local("recovery", "object", tx3, &journal3, None)
|
||||
.await
|
||||
.expect("finalize tx3")
|
||||
);
|
||||
drop(_strict_again);
|
||||
|
||||
// Prepare and publish tx4 under Relaxed (`!sync`): Stage 2 must hardlink PREPARED without directory fsync.
|
||||
fs::create_dir_all(&root).await.expect("recreate object dir for tx4");
|
||||
fs::write(root.join(STORAGE_FORMAT_FILE), meta.marshal_msg().expect("marshal meta"))
|
||||
.await
|
||||
.expect("write xl.meta for tx4");
|
||||
let tx4 = Uuid::new_v4();
|
||||
{
|
||||
let _relaxed = durability_mode_override::set(DurabilityMode::Relaxed);
|
||||
assert_eq!(
|
||||
disk.try_fused_recoverable_delete_version("recovery", "object", &info, &opts, tx4, None)
|
||||
.await
|
||||
.expect("fused prepare tx4 relaxed"),
|
||||
Some(())
|
||||
);
|
||||
let (journal4, sole4, satisfied4) = disk
|
||||
.try_publish_prepared_delete_local("recovery", "object", tx4, None)
|
||||
.await
|
||||
.expect("publish tx4 relaxed")
|
||||
.expect("tx4 satisfied");
|
||||
assert!(sole4 && satisfied4);
|
||||
let rollback4 = root.join(tx4.to_string());
|
||||
assert!(
|
||||
!os::fsync_dir_recorder::was_fsynced(&rollback4),
|
||||
"relaxed Stage 2 hardlink must not fsync rollback directory"
|
||||
);
|
||||
let prep_meta4 = std::fs::metadata(rollback4.join(PREPARED)).expect("prepared4 metadata");
|
||||
let rec_meta4 = std::fs::metadata(rollback4.join(DELETE_QUORUM_COMMITTED_FILE)).expect("receipt4 metadata");
|
||||
assert_eq!(
|
||||
(prep_meta4.dev(), prep_meta4.ino()),
|
||||
(rec_meta4.dev(), rec_meta4.ino()),
|
||||
"relaxed Stage 2 must hardlink PREPARED to DELETE_QUORUM_COMMITTED_FILE"
|
||||
);
|
||||
assert!(
|
||||
disk.try_finish_and_finalize_prepared_delete_local("recovery", "object", tx4, &journal4, None)
|
||||
.await
|
||||
.expect("finalize tx4")
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
|
||||
@@ -1,597 +0,0 @@
|
||||
// Copyright 2026 RustFS Team
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
//! Bounded emergency headroom for durable deletion. Healthy operations access
|
||||
//! only the shared owner's atomic state. Only allocation failures consume the
|
||||
//! reserve, without holding its mutex across transaction publication or quorum.
|
||||
|
||||
use std::{io, path::Path, sync::Arc};
|
||||
|
||||
#[cfg(unix)]
|
||||
pub(super) const RESERVE_BYTES: u64 = 8 * 1024 * 1024;
|
||||
|
||||
pub(super) struct DeleteReserve {
|
||||
#[cfg(unix)]
|
||||
inner: unix::State,
|
||||
}
|
||||
|
||||
impl DeleteReserve {
|
||||
pub(super) fn acquire(root: &Path) -> io::Result<Arc<Self>> {
|
||||
#[cfg(unix)]
|
||||
{
|
||||
unix::acquire(root)
|
||||
}
|
||||
#[cfg(not(unix))]
|
||||
{
|
||||
let _ = root;
|
||||
Ok(Arc::new(Self {}))
|
||||
}
|
||||
}
|
||||
|
||||
pub(super) fn is_armed(&self) -> bool {
|
||||
#[cfg(unix)]
|
||||
{
|
||||
self.inner.is_armed()
|
||||
}
|
||||
#[cfg(not(unix))]
|
||||
{
|
||||
true
|
||||
}
|
||||
}
|
||||
|
||||
/// Populate the reserve on the disk's dedicated fsync lane.
|
||||
pub(super) fn initialize(&self) -> io::Result<()> {
|
||||
self.replenish()
|
||||
}
|
||||
|
||||
/// Refill only when sufficient free space remains for recovery. An armed
|
||||
/// owner returns immediately without opening files or probing capacity.
|
||||
pub(super) fn replenish(&self) -> io::Result<()> {
|
||||
#[cfg(unix)]
|
||||
{
|
||||
self.inner.replenish()
|
||||
}
|
||||
#[cfg(not(unix))]
|
||||
{
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
/// Probe capacity during startup so a persisted reserve can make the disk
|
||||
/// writable before its normal writeability check.
|
||||
pub(super) fn release_if_low(&self) -> io::Result<()> {
|
||||
#[cfg(unix)]
|
||||
{
|
||||
self.inner.release_if_low()
|
||||
}
|
||||
#[cfg(not(unix))]
|
||||
{
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
/// Retry an allocation-safe scratch operation once after ENOSPC. The first
|
||||
/// waiter releases the reserve; concurrent waiters serialize behind it and
|
||||
/// may use the same released capacity. The caller must ensure that a failed
|
||||
/// attempt leaves only disposable scratch or an idempotent primitive under
|
||||
/// the existing namespace owner. Never replay a transaction or a
|
||||
/// publication-plus-sync sequence with this helper.
|
||||
///
|
||||
/// The owner mutex covers release and the retry, excluding replenishment
|
||||
/// during this bounded operation. Later transaction phases and unrelated
|
||||
/// allocators can still compete for the released space.
|
||||
pub(super) fn retry_after_enospc<T>(&self, mut operation: impl FnMut() -> io::Result<T>) -> io::Result<T> {
|
||||
match operation() {
|
||||
#[cfg(unix)]
|
||||
Err(error) if error.kind() == io::ErrorKind::StorageFull => self.inner.retry_after_enospc(operation, error),
|
||||
result => result,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(unix)]
|
||||
mod unix {
|
||||
use super::*;
|
||||
use parking_lot::Mutex;
|
||||
use rustix::fs::{Mode, OFlags, mkdirat, open, openat};
|
||||
use std::{
|
||||
collections::HashMap,
|
||||
fs::File,
|
||||
io::{Seek, SeekFrom, Write},
|
||||
os::unix::fs::MetadataExt,
|
||||
sync::{
|
||||
OnceLock, Weak,
|
||||
atomic::{AtomicBool, Ordering},
|
||||
},
|
||||
};
|
||||
|
||||
const METADATA_DIR: &str = ".rustfs.sys";
|
||||
const RESERVE_NAME: &str = "delete-reserve";
|
||||
type RootIdentity = (u64, u64);
|
||||
type OwnerRegistry = Mutex<HashMap<RootIdentity, Weak<DeleteReserve>>>;
|
||||
static OWNERS: OnceLock<OwnerRegistry> = OnceLock::new();
|
||||
|
||||
pub(super) struct State {
|
||||
root: File,
|
||||
mutation: Mutex<()>,
|
||||
armed: AtomicBool,
|
||||
}
|
||||
|
||||
pub(super) fn acquire(path: &Path) -> io::Result<Arc<DeleteReserve>> {
|
||||
let root = File::from(open(
|
||||
path,
|
||||
OFlags::RDONLY | OFlags::DIRECTORY | OFlags::NOFOLLOW | OFlags::CLOEXEC,
|
||||
Mode::empty(),
|
||||
)?);
|
||||
let metadata = root.metadata()?;
|
||||
let identity = (metadata.dev(), metadata.ino());
|
||||
let mut registry = OWNERS.get_or_init(Mutex::default).lock();
|
||||
if let Some(owner) = registry.get(&identity).and_then(Weak::upgrade) {
|
||||
return Ok(owner);
|
||||
}
|
||||
registry.retain(|_, owner| owner.strong_count() != 0);
|
||||
let owner = Arc::new(DeleteReserve {
|
||||
inner: State {
|
||||
root,
|
||||
mutation: Mutex::new(()),
|
||||
armed: AtomicBool::new(false),
|
||||
},
|
||||
});
|
||||
registry.insert(identity, Arc::downgrade(&owner));
|
||||
Ok(owner)
|
||||
}
|
||||
|
||||
fn metadata_directory(root: &File, create: bool) -> io::Result<File> {
|
||||
if create {
|
||||
match mkdirat(root, METADATA_DIR, Mode::from_raw_mode(0o700)) {
|
||||
Ok(()) => root.sync_all()?,
|
||||
Err(rustix::io::Errno::EXIST) => {}
|
||||
Err(error) => return Err(error.into()),
|
||||
}
|
||||
}
|
||||
Ok(File::from(openat(
|
||||
root,
|
||||
METADATA_DIR,
|
||||
OFlags::RDONLY | OFlags::DIRECTORY | OFlags::NOFOLLOW | OFlags::CLOEXEC,
|
||||
Mode::empty(),
|
||||
)?))
|
||||
}
|
||||
|
||||
fn reserve_file(parent: &File, create: bool) -> io::Result<File> {
|
||||
let flags = OFlags::RDWR | OFlags::NOFOLLOW | OFlags::NONBLOCK | OFlags::CLOEXEC;
|
||||
let file = File::from(openat(
|
||||
parent,
|
||||
RESERVE_NAME,
|
||||
if create { flags | OFlags::CREATE } else { flags },
|
||||
Mode::from_raw_mode(0o600),
|
||||
)?);
|
||||
let metadata = file.metadata()?;
|
||||
if !metadata.is_file() || metadata.nlink() != 1 || metadata.len() > RESERVE_BYTES {
|
||||
return Err(io::Error::new(io::ErrorKind::InvalidData, "invalid delete reserve file"));
|
||||
}
|
||||
Ok(file)
|
||||
}
|
||||
|
||||
fn absent_to_none<T>(result: io::Result<T>) -> io::Result<Option<T>> {
|
||||
match result {
|
||||
Ok(value) => Ok(Some(value)),
|
||||
Err(error) if error.kind() == io::ErrorKind::NotFound => Ok(None),
|
||||
Err(error) => Err(error),
|
||||
}
|
||||
}
|
||||
|
||||
impl State {
|
||||
pub(super) fn is_armed(&self) -> bool {
|
||||
self.armed.load(Ordering::Acquire)
|
||||
}
|
||||
|
||||
pub(super) fn replenish(&self) -> io::Result<()> {
|
||||
if self.is_armed() {
|
||||
return Ok(());
|
||||
}
|
||||
let _guard = self.mutation.lock();
|
||||
if self.is_armed() {
|
||||
return Ok(());
|
||||
}
|
||||
let existing = match absent_to_none(metadata_directory(&self.root, false))? {
|
||||
Some(parent) => absent_to_none(reserve_file(&parent, false))?.map(|file| (parent, file)),
|
||||
None => None,
|
||||
};
|
||||
if let Some((_, file)) = &existing {
|
||||
let metadata = file.metadata()?;
|
||||
if metadata.len() == RESERVE_BYTES && metadata.blocks().saturating_mul(512) >= RESERVE_BYTES {
|
||||
self.armed.store(true, Ordering::Release);
|
||||
return Ok(());
|
||||
}
|
||||
}
|
||||
// A depleted reserve must not take newly released recovery space
|
||||
// away from a disk that is still nearly full.
|
||||
if available_bytes(&self.root)? < RESERVE_BYTES * 2 {
|
||||
return Ok(());
|
||||
}
|
||||
let (parent, mut file) = match existing {
|
||||
Some(existing) => existing,
|
||||
None => {
|
||||
let parent = metadata_directory(&self.root, true)?;
|
||||
let file = reserve_file(&parent, true)?;
|
||||
(parent, file)
|
||||
}
|
||||
};
|
||||
let metadata = file.metadata()?;
|
||||
// A length alone does not reserve blocks. Repair sparse files left
|
||||
// by interrupted allocation or older installations.
|
||||
if metadata.blocks().saturating_mul(512) < metadata.len() {
|
||||
file.set_len(0)?;
|
||||
}
|
||||
let length = file.seek(SeekFrom::End(0))?;
|
||||
let mut remaining = RESERVE_BYTES - length;
|
||||
let mut bytes = [0_u8; 64 * 1024];
|
||||
while remaining != 0 {
|
||||
let count = usize::try_from(remaining.min(64 * 1024)).map_err(io::Error::other)?;
|
||||
// Zero blocks can become holes on transparent-compression filesystems.
|
||||
rand::fill(&mut bytes[..count]);
|
||||
file.write_all(&bytes[..count])?;
|
||||
remaining -= u64::try_from(count).map_err(io::Error::other)?;
|
||||
}
|
||||
file.sync_all()?;
|
||||
if file.metadata()?.blocks().saturating_mul(512) < RESERVE_BYTES {
|
||||
return Err(io::Error::new(
|
||||
io::ErrorKind::Unsupported,
|
||||
"filesystem did not allocate delete reserve blocks",
|
||||
));
|
||||
}
|
||||
parent.sync_all()?;
|
||||
self.armed.store(true, Ordering::Release);
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn release_locked(&self) -> io::Result<bool> {
|
||||
let Some(parent) = absent_to_none(metadata_directory(&self.root, false))? else {
|
||||
self.armed.store(false, Ordering::Release);
|
||||
return Ok(false);
|
||||
};
|
||||
let Some(file) = absent_to_none(reserve_file(&parent, false))? else {
|
||||
self.armed.store(false, Ordering::Release);
|
||||
return Ok(false);
|
||||
};
|
||||
let metadata = file.metadata()?;
|
||||
if metadata.len() == 0 {
|
||||
self.armed.store(false, Ordering::Release);
|
||||
return Ok(false);
|
||||
}
|
||||
file.set_len(0)?;
|
||||
self.armed.store(false, Ordering::Release);
|
||||
// The reserve contains no recovery authority. Truncation releases
|
||||
// blocks immediately and needs no durability barrier before retry.
|
||||
Ok(metadata.blocks() != 0)
|
||||
}
|
||||
|
||||
pub(super) fn release_if_low(&self) -> io::Result<()> {
|
||||
if available_bytes(&self.root)? < RESERVE_BYTES / 2 {
|
||||
let _guard = self.mutation.lock();
|
||||
self.release_locked()?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
pub(super) fn retry_after_enospc<T>(
|
||||
&self,
|
||||
mut operation: impl FnMut() -> io::Result<T>,
|
||||
original: io::Error,
|
||||
) -> io::Result<T> {
|
||||
let _guard = self.mutation.lock();
|
||||
match self.release_locked() {
|
||||
// A peer may have consumed the shared reserve while this
|
||||
// operation was failing. Waiting for the mutation lock and
|
||||
// replaying once lets it use the capacity that peer released.
|
||||
// The public contract only accepts disposable scratch or an
|
||||
// idempotent primitive, so the retry remains safe even when no
|
||||
// reserve was present. Invalid reserve state still fails closed.
|
||||
Ok(_) => operation(),
|
||||
Err(_) => Err(original),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn available_bytes(root: &File) -> io::Result<u64> {
|
||||
let stat = rustix::fs::fstatvfs(root)?;
|
||||
let block_size = if stat.f_frsize == 0 { stat.f_bsize } else { stat.f_frsize };
|
||||
let available = u128::from(stat.f_bavail) * u128::from(block_size);
|
||||
Ok(u64::try_from(available).unwrap_or(u64::MAX))
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
fn reserve_path(root: &Path) -> std::path::PathBuf {
|
||||
root.join(METADATA_DIR).join(RESERVE_NAME)
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn delete_reserve_allocates_retries_once_and_replenishes_real_blocks() {
|
||||
let root = tempfile::tempdir().expect("create disk root");
|
||||
let owner = DeleteReserve::acquire(root.path()).expect("pin disk root");
|
||||
owner.initialize().expect("allocate reserve");
|
||||
let path = reserve_path(root.path());
|
||||
let initial = std::fs::metadata(&path).expect("reserve metadata");
|
||||
assert_eq!(initial.len(), RESERVE_BYTES);
|
||||
assert!(initial.blocks() * 512 >= RESERVE_BYTES);
|
||||
assert!(owner.is_armed());
|
||||
let mut calls = 0;
|
||||
let result = owner
|
||||
.retry_after_enospc(|| {
|
||||
calls += 1;
|
||||
if calls == 1 {
|
||||
Err(io::Error::new(io::ErrorKind::StorageFull, "scratch allocation"))
|
||||
} else {
|
||||
Ok(42)
|
||||
}
|
||||
})
|
||||
.expect("retry scratch allocation");
|
||||
assert_eq!(result, 42);
|
||||
assert_eq!(calls, 2);
|
||||
assert!(!owner.is_armed());
|
||||
assert_eq!(std::fs::metadata(&path).expect("released reserve").len(), 0);
|
||||
owner.replenish().expect("replenish reserve");
|
||||
assert!(owner.is_armed());
|
||||
assert_eq!(std::fs::metadata(&path).expect("replenished reserve").len(), RESERVE_BYTES);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn delete_reserve_retries_once_without_local_reserve_capacity() {
|
||||
let root = tempfile::tempdir().expect("create disk root");
|
||||
let owner = DeleteReserve::acquire(root.path()).expect("pin disk root");
|
||||
let mut calls = 0;
|
||||
let error = owner
|
||||
.retry_after_enospc::<()>(|| {
|
||||
calls += 1;
|
||||
Err(io::Error::new(io::ErrorKind::StorageFull, format!("allocation {calls}")))
|
||||
})
|
||||
.expect_err("retry without a reserve still has no capacity");
|
||||
assert_eq!(calls, 2);
|
||||
assert_eq!(error.to_string(), "allocation 2");
|
||||
owner.initialize().expect("allocate reserve");
|
||||
let parent = metadata_directory(&owner.inner.root, false).expect("metadata directory");
|
||||
let reserve = reserve_file(&parent, false).expect("reserve file");
|
||||
reserve.set_len(0).expect("deplete reserve");
|
||||
calls = 0;
|
||||
assert!(
|
||||
owner
|
||||
.retry_after_enospc::<()>(|| {
|
||||
calls += 1;
|
||||
Err(io::Error::new(io::ErrorKind::StorageFull, "empty reserve"))
|
||||
})
|
||||
.is_err()
|
||||
);
|
||||
assert_eq!(calls, 2);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn delete_reserve_retry_ignores_other_errors_and_stays_bounded() {
|
||||
let root = tempfile::tempdir().expect("create disk root");
|
||||
let owner = DeleteReserve::acquire(root.path()).expect("pin disk root");
|
||||
owner.initialize().expect("allocate reserve");
|
||||
let mut calls = 0;
|
||||
let error = owner
|
||||
.retry_after_enospc::<()>(|| {
|
||||
calls += 1;
|
||||
Err(io::Error::new(io::ErrorKind::PermissionDenied, "permission failure"))
|
||||
})
|
||||
.expect_err("permission failures must propagate");
|
||||
assert_eq!(error.kind(), io::ErrorKind::PermissionDenied);
|
||||
assert_eq!(calls, 1);
|
||||
assert!(owner.is_armed());
|
||||
calls = 0;
|
||||
let error = owner
|
||||
.retry_after_enospc::<()>(|| {
|
||||
calls += 1;
|
||||
Err(io::Error::new(io::ErrorKind::StorageFull, format!("allocation {calls}")))
|
||||
})
|
||||
.expect_err("second allocation failure propagates");
|
||||
assert_eq!(calls, 2);
|
||||
assert_eq!(error.to_string(), "allocation 2");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn delete_reserve_rejects_symlinks_and_hardlinks_without_hiding_operation_errors() {
|
||||
use std::os::unix::fs::symlink;
|
||||
let root = tempfile::tempdir().expect("create disk root");
|
||||
let owner = DeleteReserve::acquire(root.path()).expect("pin disk root");
|
||||
let external = tempfile::tempdir().expect("create external directory");
|
||||
let victim = external.path().join("victim");
|
||||
std::fs::write(&victim, b"preserve").expect("write victim");
|
||||
let metadata = root.path().join(METADATA_DIR);
|
||||
symlink(external.path(), &metadata).expect("symlink metadata directory");
|
||||
assert!(owner.replenish().is_err());
|
||||
std::fs::remove_file(&metadata).expect("remove directory link");
|
||||
std::fs::create_dir(&metadata).expect("create metadata directory");
|
||||
let reserve = reserve_path(root.path());
|
||||
symlink(&victim, &reserve).expect("symlink reserve");
|
||||
assert!(owner.replenish().is_err());
|
||||
let mut calls = 0;
|
||||
let error = owner
|
||||
.retry_after_enospc::<()>(|| {
|
||||
calls += 1;
|
||||
Err(io::Error::new(io::ErrorKind::StorageFull, "original error"))
|
||||
})
|
||||
.expect_err("invalid reserve cannot mask allocation failure");
|
||||
assert_eq!(calls, 1);
|
||||
assert_eq!(error.to_string(), "original error");
|
||||
std::fs::remove_file(&reserve).expect("remove reserve link");
|
||||
std::fs::hard_link(&victim, &reserve).expect("hardlink reserve");
|
||||
assert!(owner.replenish().is_err());
|
||||
assert!(owner.inner.release_locked().is_err());
|
||||
assert_eq!(std::fs::read(&victim).expect("read victim"), b"preserve");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn delete_reserve_repairs_partial_and_sparse_files() {
|
||||
let root = tempfile::tempdir().expect("create disk root");
|
||||
let owner = DeleteReserve::acquire(root.path()).expect("pin disk root");
|
||||
let parent = metadata_directory(&owner.inner.root, true).expect("create metadata directory");
|
||||
let mut file = reserve_file(&parent, true).expect("create reserve file");
|
||||
file.write_all(b"partial allocation").expect("write partial reserve");
|
||||
owner.replenish().expect("finish partial allocation");
|
||||
let metadata = file.metadata().expect("partial reserve metadata");
|
||||
assert_eq!(metadata.len(), RESERVE_BYTES);
|
||||
assert!(metadata.blocks() * 512 >= RESERVE_BYTES);
|
||||
file.set_len(0).expect("truncate reserve");
|
||||
file.set_len(RESERVE_BYTES).expect("create sparse reserve");
|
||||
owner.inner.armed.store(false, Ordering::Release);
|
||||
assert!(file.metadata().expect("sparse reserve metadata").blocks() * 512 < RESERVE_BYTES);
|
||||
owner.replenish().expect("repair sparse allocation");
|
||||
let metadata = file.metadata().expect("repaired reserve metadata");
|
||||
assert_eq!(metadata.len(), RESERVE_BYTES);
|
||||
assert!(metadata.blocks() * 512 >= RESERVE_BYTES);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn delete_reserve_shares_reopened_owner_and_serializes_recovery_with_replenishment() {
|
||||
let root = tempfile::tempdir().expect("create disk root");
|
||||
let first = DeleteReserve::acquire(root.path()).expect("open first root");
|
||||
let second = DeleteReserve::acquire(root.path()).expect("reopen root");
|
||||
assert!(Arc::ptr_eq(&first, &second));
|
||||
first.initialize().expect("allocate reserve");
|
||||
let (retry_entered_tx, retry_entered_rx) = std::sync::mpsc::channel();
|
||||
let (allow_retry_tx, allow_retry_rx) = std::sync::mpsc::channel();
|
||||
let (replenished_tx, replenished_rx) = std::sync::mpsc::channel();
|
||||
let recovery_owner = first.clone();
|
||||
let root_path = root.path().to_path_buf();
|
||||
std::thread::scope(|scope| {
|
||||
let recovery = scope.spawn(move || {
|
||||
let mut calls = 0;
|
||||
recovery_owner
|
||||
.retry_after_enospc(|| {
|
||||
calls += 1;
|
||||
if calls == 1 {
|
||||
return Err(io::Error::new(io::ErrorKind::StorageFull, "scratch allocation"));
|
||||
}
|
||||
assert!(recovery_owner.inner.mutation.try_lock().is_none(), "retry must retain the reserve mutex");
|
||||
retry_entered_tx.send(()).expect("notify retry");
|
||||
allow_retry_rx
|
||||
.recv_timeout(std::time::Duration::from_secs(10))
|
||||
.expect("allow retry completion");
|
||||
assert_eq!(std::fs::metadata(reserve_path(&root_path)).expect("depleted reserve").len(), 0);
|
||||
Ok(())
|
||||
})
|
||||
.expect("retry allocation");
|
||||
});
|
||||
retry_entered_rx
|
||||
.recv_timeout(std::time::Duration::from_secs(10))
|
||||
.expect("retry entered");
|
||||
let replenish = scope.spawn(move || {
|
||||
second.replenish().expect("concurrent replenishment");
|
||||
replenished_tx.send(()).expect("notify replenishment");
|
||||
});
|
||||
assert!(matches!(replenished_rx.try_recv(), Err(std::sync::mpsc::TryRecvError::Empty)));
|
||||
allow_retry_tx.send(()).expect("finish retry");
|
||||
recovery.join().expect("recovery worker");
|
||||
replenish.join().expect("replenishment worker");
|
||||
});
|
||||
assert!(first.is_armed());
|
||||
assert_eq!(std::fs::metadata(reserve_path(root.path())).expect("final reserve").len(), RESERVE_BYTES);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn delete_reserve_gives_concurrent_enospc_waiters_one_retry() {
|
||||
let root = tempfile::tempdir().expect("create disk root");
|
||||
let first = DeleteReserve::acquire(root.path()).expect("open first owner");
|
||||
let second = DeleteReserve::acquire(root.path()).expect("open second owner");
|
||||
first.initialize().expect("allocate reserve");
|
||||
|
||||
let (first_retry_tx, first_retry_rx) = std::sync::mpsc::channel();
|
||||
let (allow_first_tx, allow_first_rx) = std::sync::mpsc::channel();
|
||||
let (second_failed_tx, second_failed_rx) = std::sync::mpsc::channel();
|
||||
let (second_done_tx, second_done_rx) = std::sync::mpsc::channel();
|
||||
|
||||
std::thread::scope(|scope| {
|
||||
let first_waiter = scope.spawn(move || {
|
||||
let mut calls = 0;
|
||||
first
|
||||
.retry_after_enospc(|| {
|
||||
calls += 1;
|
||||
if calls == 1 {
|
||||
return Err(io::Error::new(io::ErrorKind::StorageFull, "first ENOSPC"));
|
||||
}
|
||||
first_retry_tx.send(()).expect("notify first retry");
|
||||
allow_first_rx
|
||||
.recv_timeout(std::time::Duration::from_secs(10))
|
||||
.expect("allow first retry to finish");
|
||||
Ok(())
|
||||
})
|
||||
.expect("first waiter retries");
|
||||
assert_eq!(calls, 2);
|
||||
});
|
||||
|
||||
first_retry_rx
|
||||
.recv_timeout(std::time::Duration::from_secs(10))
|
||||
.expect("first waiter holds recovery lock");
|
||||
let second_waiter = scope.spawn(move || {
|
||||
let mut calls = 0;
|
||||
second
|
||||
.retry_after_enospc(|| {
|
||||
calls += 1;
|
||||
if calls == 1 {
|
||||
second_failed_tx.send(()).expect("notify second ENOSPC");
|
||||
return Err(io::Error::new(io::ErrorKind::StorageFull, "second ENOSPC"));
|
||||
}
|
||||
Ok(())
|
||||
})
|
||||
.expect("second waiter retries after shared release");
|
||||
assert_eq!(calls, 2);
|
||||
second_done_tx.send(()).expect("notify second completion");
|
||||
});
|
||||
|
||||
second_failed_rx
|
||||
.recv_timeout(std::time::Duration::from_secs(10))
|
||||
.expect("second waiter observed ENOSPC");
|
||||
assert!(matches!(second_done_rx.try_recv(), Err(std::sync::mpsc::TryRecvError::Empty)));
|
||||
allow_first_tx.send(()).expect("finish first retry");
|
||||
first_waiter.join().expect("first waiter");
|
||||
second_done_rx
|
||||
.recv_timeout(std::time::Duration::from_secs(10))
|
||||
.expect("second waiter completed its bounded retry");
|
||||
second_waiter.join().expect("second waiter");
|
||||
});
|
||||
|
||||
assert_eq!(std::fs::metadata(reserve_path(root.path())).expect("depleted reserve").len(), 0);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn delete_reserve_rejects_oversize_files_and_fifos() {
|
||||
let root = tempfile::tempdir().expect("create disk root");
|
||||
let owner = DeleteReserve::acquire(root.path()).expect("pin disk root");
|
||||
let parent = metadata_directory(&owner.inner.root, true).expect("create metadata directory");
|
||||
let file = reserve_file(&parent, true).expect("create reserve file");
|
||||
file.set_len(RESERVE_BYTES + 1).expect("create oversized reserve");
|
||||
assert!(owner.replenish().is_err());
|
||||
assert!(owner.inner.release_locked().is_err());
|
||||
assert_eq!(file.metadata().expect("oversize metadata").len(), RESERVE_BYTES + 1);
|
||||
let path = reserve_path(root.path());
|
||||
std::fs::remove_file(&path).expect("remove oversized file");
|
||||
assert!(
|
||||
std::process::Command::new("mkfifo")
|
||||
.arg(&path)
|
||||
.status()
|
||||
.expect("create FIFO fixture")
|
||||
.success()
|
||||
);
|
||||
assert!(owner.replenish().is_err());
|
||||
assert!(owner.inner.release_locked().is_err());
|
||||
use std::os::unix::fs::FileTypeExt;
|
||||
assert!(std::fs::symlink_metadata(path).expect("FIFO metadata").file_type().is_fifo());
|
||||
}
|
||||
}
|
||||
}
|
||||
+92
-435
@@ -177,30 +177,6 @@ pub(crate) mod fsync_dir_recorder {
|
||||
contains_path(&RECORDED.lock().expect("fsync dir recorder poisoned"), dir)
|
||||
}
|
||||
|
||||
#[cfg(unix)]
|
||||
pub(crate) fn count_at(dir: &Path) -> usize {
|
||||
let canonical = dir.canonicalize().expect("sync count directory must exist");
|
||||
RECORDED
|
||||
.lock()
|
||||
.expect("fsync dir recorder poisoned")
|
||||
.iter()
|
||||
.filter(|recorded| **recorded == canonical)
|
||||
.count()
|
||||
}
|
||||
|
||||
#[cfg(unix)]
|
||||
pub(crate) fn count_under(dir: &Path) -> usize {
|
||||
// Each invocation records its canonical path once, including when the
|
||||
// syscall used a procfs mount-lease alias or a symlinked temporary root.
|
||||
let canonical = dir.canonicalize().expect("sync count root must exist");
|
||||
RECORDED
|
||||
.lock()
|
||||
.expect("fsync dir recorder poisoned")
|
||||
.iter()
|
||||
.filter(|recorded| recorded.starts_with(&canonical))
|
||||
.count()
|
||||
}
|
||||
|
||||
pub(crate) fn set_failure(dir: &Path, kind: io::ErrorKind) {
|
||||
super::dir_fsync_failures::set(dir, kind);
|
||||
}
|
||||
@@ -1795,7 +1771,7 @@ fn run_file_fdatasync_group_worker(group: Arc<FileFdatasyncGroup>) -> impl std::
|
||||
#[cfg(test)]
|
||||
file_sync_probe::record_group_batch(batch_file_count);
|
||||
rustfs_io_metrics::record_put_rename_fdatasync_batch(
|
||||
rustfs_io_metrics::PUT_RENAME_FDATASYNC_BATCH_MODE_GROUPED,
|
||||
rustfs_io_metrics::PUT_RENAME_FDATASYNC_BATCH_MODE_PARALLEL,
|
||||
batch_file_count,
|
||||
);
|
||||
let result = if let Some(disk_permits) = group.disk_permits.upgrade() {
|
||||
@@ -2371,8 +2347,6 @@ type NativeRootIdentity = (u64, u64);
|
||||
struct NativePublicationHistory {
|
||||
roots: HashMap<NativeRootIdentity, NativeRootHistory>,
|
||||
aliases: HashMap<PathBuf, HashSet<NativeRootIdentity>>,
|
||||
#[cfg(target_os = "linux")]
|
||||
live_proc_roots: HashMap<PathBuf, (NativeRootIdentity, Weak<std::fs::File>)>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Default)]
|
||||
@@ -2452,59 +2426,12 @@ pub(crate) fn register_native_publication_root(root: &Path, io_root: &Path) -> O
|
||||
/// Record before queueing filesystem work. Cancellation never removes Started.
|
||||
pub(crate) fn record_native_publication(path: &Path) {
|
||||
let aliases: Vec<_> = {
|
||||
#[cfg(target_os = "linux")]
|
||||
let mut history = NATIVE_PUBLICATION_HISTORY.lock();
|
||||
#[cfg(not(target_os = "linux"))]
|
||||
let history = NATIVE_PUBLICATION_HISTORY.lock();
|
||||
let mut unpinned = Vec::new();
|
||||
for alias in path.ancestors() {
|
||||
if !history.aliases.contains_key(alias) {
|
||||
continue;
|
||||
}
|
||||
#[cfg(target_os = "linux")]
|
||||
if let Some((live_identity, weak)) = history.live_proc_roots.get(alias) {
|
||||
if weak.strong_count() > 0 {
|
||||
let live_identity = *live_identity;
|
||||
if history
|
||||
.aliases
|
||||
.get(alias)
|
||||
.is_some_and(|ids| ids.len() == 1 && ids.contains(&live_identity))
|
||||
{
|
||||
let Ok(relative) = path.strip_prefix(alias) else {
|
||||
continue;
|
||||
};
|
||||
let mut components = relative.components().filter(|part| !matches!(part, Component::CurDir));
|
||||
let Some(first) = components.next() else {
|
||||
continue;
|
||||
};
|
||||
let bucket = match first {
|
||||
Component::Normal(bucket) if components.all(|part| matches!(part, Component::Normal(_))) => {
|
||||
bucket.to_str()
|
||||
}
|
||||
_ => None,
|
||||
};
|
||||
let root = history.roots.entry(live_identity).or_default();
|
||||
match bucket {
|
||||
Some(bucket) => {
|
||||
if !root.started.contains(bucket) {
|
||||
root.started.insert(bucket.to_owned());
|
||||
}
|
||||
}
|
||||
None => root.unknown = true,
|
||||
}
|
||||
continue;
|
||||
}
|
||||
} else {
|
||||
history.live_proc_roots.remove(alias);
|
||||
}
|
||||
}
|
||||
unpinned.push(alias.to_path_buf());
|
||||
}
|
||||
unpinned
|
||||
path.ancestors()
|
||||
.filter(|alias| history.aliases.contains_key(*alias))
|
||||
.map(Path::to_path_buf)
|
||||
.collect()
|
||||
};
|
||||
if aliases.is_empty() {
|
||||
return;
|
||||
}
|
||||
// Physical lookups must never hold the process-wide publication mutex.
|
||||
let observations: Vec<_> = aliases
|
||||
.into_iter()
|
||||
@@ -2714,8 +2641,6 @@ impl DiskBlockingLane {
|
||||
#[derive(Default)]
|
||||
struct DiskBlockingQueueState {
|
||||
jobs: VecDeque<Box<dyn FnOnce() + Send>>,
|
||||
#[cfg(test)]
|
||||
submitted_jobs: usize,
|
||||
active_workers: usize,
|
||||
idle_workers: usize,
|
||||
#[cfg(any(test, feature = "test-util"))]
|
||||
@@ -2842,10 +2767,6 @@ impl DiskBlockingQueue {
|
||||
let should_spawn = {
|
||||
let mut state = self.lane(lane).state.lock();
|
||||
state.jobs.push_back(job);
|
||||
#[cfg(test)]
|
||||
{
|
||||
state.submitted_jobs += 1;
|
||||
}
|
||||
self.lane(lane).ready.store(true, std::sync::atomic::Ordering::Relaxed);
|
||||
if state.jobs.len() > state.idle_workers && state.active_workers < self.effective_max_workers(lane, &state) {
|
||||
state.active_workers += 1;
|
||||
@@ -2860,11 +2781,6 @@ impl DiskBlockingQueue {
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
pub(crate) fn submitted_jobs_for_test(&self) -> usize {
|
||||
self.lanes.iter().map(|lane| lane.state.lock().submitted_jobs).sum()
|
||||
}
|
||||
|
||||
#[inline]
|
||||
fn foreground_lane() -> DiskBlockingLane {
|
||||
if is_isolated_io_thread() {
|
||||
@@ -4849,14 +4765,6 @@ pub(crate) mod multipart_source_syscall_failures {
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(all(test, unix))]
|
||||
static PREPARED_METADATA_SYNCS: LazyLock<Mutex<HashMap<PathBuf, usize>>> = LazyLock::new(|| Mutex::new(HashMap::new()));
|
||||
|
||||
#[cfg(all(test, unix))]
|
||||
pub(crate) fn take_prepared_metadata_sync_count(destination: &Path) -> usize {
|
||||
PREPARED_METADATA_SYNCS.lock().remove(destination).unwrap_or(0)
|
||||
}
|
||||
|
||||
pub(crate) struct PreparedRenameSource {
|
||||
path: PathBuf,
|
||||
#[cfg(test)]
|
||||
@@ -4890,13 +4798,6 @@ impl PreparedRenameSource {
|
||||
pub(crate) fn sync_data(&self) -> io::Result<()> {
|
||||
#[cfg(test)]
|
||||
multipart_source_syscall_failures::check(multipart_source_syscall_failures::Stage::FileSync, &self.test_destination)?;
|
||||
#[cfg(all(test, unix))]
|
||||
{
|
||||
*PREPARED_METADATA_SYNCS
|
||||
.lock()
|
||||
.entry(self.test_destination.clone())
|
||||
.or_default() += 1;
|
||||
}
|
||||
let _sync = StageTimer::new(true, "disk_prepared_metadata_fdatasync");
|
||||
#[cfg(windows)]
|
||||
self.source.as_file().sync_data()?;
|
||||
@@ -4948,20 +4849,12 @@ pub(crate) fn create_prepared_rename_source_with_commit_guard(
|
||||
let directories = &commit_guard.directories;
|
||||
let name = directories.file_name(src_file_path, true)?;
|
||||
directories.file_name(dst_file_path, false)?;
|
||||
if let Some(source_dir) = directories.source.get() {
|
||||
open_linux_relative(
|
||||
&source_dir.file,
|
||||
Path::new(name),
|
||||
OFlags::CREATE | OFlags::TRUNC | OFlags::WRONLY,
|
||||
Mode::from_raw_mode(0o666),
|
||||
)?
|
||||
} else {
|
||||
directories.root.open_relative(
|
||||
src_file_path,
|
||||
OFlags::CREATE | OFlags::TRUNC | OFlags::WRONLY,
|
||||
Mode::from_raw_mode(0o666),
|
||||
)?
|
||||
}
|
||||
open_linux_relative(
|
||||
&directories.source.file,
|
||||
Path::new(name),
|
||||
OFlags::CREATE | OFlags::TRUNC | OFlags::WRONLY,
|
||||
Mode::from_raw_mode(0o666),
|
||||
)?
|
||||
};
|
||||
#[cfg(not(target_os = "linux"))]
|
||||
let source = std::fs::OpenOptions::new()
|
||||
@@ -4986,65 +4879,6 @@ pub(crate) fn create_prepared_rename_source_with_commit_guard(
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(target_os = "linux")]
|
||||
pub(crate) struct ReadDestinationFile {
|
||||
pub(crate) bytes: bytes::Bytes,
|
||||
pub(crate) file: std::fs::File,
|
||||
}
|
||||
|
||||
#[cfg(target_os = "linux")]
|
||||
pub(crate) fn read_destination_file_handle_with_commit_guard(
|
||||
file_path: &Path,
|
||||
commit_guard: &RenameCommitGuard,
|
||||
) -> io::Result<Option<ReadDestinationFile>> {
|
||||
use rustix::fs::{Mode, OFlags};
|
||||
let directories = &commit_guard.directories;
|
||||
let name = directories.file_name(file_path, false)?;
|
||||
let Some(parent) = directories.destination.get() else {
|
||||
return Ok(None);
|
||||
};
|
||||
match open_linux_relative(&parent.file, Path::new(name), OFlags::RDONLY, Mode::empty()) {
|
||||
Ok(mut file) => {
|
||||
let len = file.metadata()?.len();
|
||||
let cap = usize::try_from(len).map_or(0, |len| len.saturating_add(1));
|
||||
let mut data = Vec::with_capacity(cap);
|
||||
std::io::Read::read_to_end(&mut std::io::Read::take(&mut file, len), &mut data)?;
|
||||
Ok(Some(ReadDestinationFile {
|
||||
bytes: bytes::Bytes::from(data),
|
||||
file,
|
||||
}))
|
||||
}
|
||||
Err(err) if err.kind() == io::ErrorKind::NotFound => Ok(None),
|
||||
Err(err) => Err(err),
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(target_os = "linux")]
|
||||
pub(crate) fn unlink_flat_tmp_source_std(src_file_path: &Path, src_parent: &Path, publication_root: &PublicationRoot) {
|
||||
use rustix::fs::{AtFlags, unlinkat};
|
||||
let Some(name) = rustfs_utils::path::unix_direct_child_file_name(src_file_path, src_parent) else {
|
||||
return;
|
||||
};
|
||||
if let Ok(source_dir) = publication_root.open_directory(src_parent) {
|
||||
let _ = unlinkat(&source_dir, name, AtFlags::empty());
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(target_os = "linux")]
|
||||
pub(crate) fn unlink_prepared_rename_source_with_commit_guard_std(src_file_path: &Path, commit_guard: &RenameCommitGuard) {
|
||||
use rustix::fs::{AtFlags, unlinkat};
|
||||
let directories = &commit_guard.directories;
|
||||
let Ok(name) = directories.file_name(src_file_path, true) else {
|
||||
return;
|
||||
};
|
||||
if let Some(source_dir) = directories.source.get()
|
||||
&& unlinkat(&*source_dir.file, name, AtFlags::empty()).is_ok()
|
||||
{
|
||||
return;
|
||||
}
|
||||
unlink_flat_tmp_source_std(src_file_path, &directories.source_path, &directories.root);
|
||||
}
|
||||
|
||||
#[cfg(any(windows, target_os = "linux"))]
|
||||
pub(crate) fn read_destination_file_with_commit_guard(
|
||||
file_path: &Path,
|
||||
@@ -5052,7 +4886,23 @@ pub(crate) fn read_destination_file_with_commit_guard(
|
||||
) -> io::Result<Option<Vec<u8>>> {
|
||||
#[cfg(target_os = "linux")]
|
||||
{
|
||||
Ok(read_destination_file_handle_with_commit_guard(file_path, commit_guard)?.map(|read| Vec::from(read.bytes)))
|
||||
use rustix::fs::{Mode, OFlags};
|
||||
let directories = &commit_guard.directories;
|
||||
let name = directories.file_name(file_path, false)?;
|
||||
let Some(parent) = directories.destination.get() else {
|
||||
return Ok(None);
|
||||
};
|
||||
match open_linux_relative(&parent.file, Path::new(name), OFlags::RDONLY, Mode::empty()) {
|
||||
Ok(mut file) => {
|
||||
let len = file.metadata()?.len();
|
||||
let cap = usize::try_from(len).map_or(0, |len| len.saturating_add(1));
|
||||
let mut data = Vec::with_capacity(cap);
|
||||
std::io::Read::read_to_end(&mut std::io::Read::take(&mut file, len), &mut data)?;
|
||||
Ok(Some(data))
|
||||
}
|
||||
Err(err) if err.kind() == io::ErrorKind::NotFound => Ok(None),
|
||||
Err(err) => Err(err),
|
||||
}
|
||||
}
|
||||
#[cfg(windows)]
|
||||
{
|
||||
@@ -5071,18 +4921,17 @@ pub(crate) fn stage_inline_rollback_backup_with_commit_guard_std(
|
||||
dst_file_path: &Path,
|
||||
backup_parent: &Path,
|
||||
backup_path: &Path,
|
||||
old_destination_file: &ReadDestinationFile,
|
||||
old_metadata: &[u8],
|
||||
sync_backup: bool,
|
||||
allow_hardlink: bool,
|
||||
commit_guard: &RenameCommitGuard,
|
||||
) -> io::Result<()> {
|
||||
use rustix::fs::{AtFlags, CWD, Mode, OFlags, linkat, mkdirat, openat, unlinkat};
|
||||
use rustix::fs::{AtFlags, Mode, OFlags, linkat, mkdirat, openat, unlinkat};
|
||||
use rustix::io::Errno;
|
||||
use std::os::fd::AsRawFd;
|
||||
|
||||
record_native_publication(backup_path);
|
||||
let directories = &commit_guard.directories;
|
||||
let _dst_file_name = directories.file_name(dst_file_path, false)?;
|
||||
let dst_file_name = directories.file_name(dst_file_path, false)?;
|
||||
if backup_parent.parent() != Some(directories.destination_path.as_path()) || backup_path.parent() != Some(backup_parent) {
|
||||
return Err(io::Error::new(
|
||||
io::ErrorKind::InvalidInput,
|
||||
@@ -5099,8 +4948,6 @@ pub(crate) fn stage_inline_rollback_backup_with_commit_guard_std(
|
||||
.destination
|
||||
.get()
|
||||
.ok_or_else(|| io::Error::other("missing destination commit directory"))?;
|
||||
let old_metadata = old_destination_file.bytes.as_ref();
|
||||
let proc_fd_path = format!("/proc/self/fd/{}", old_destination_file.file.as_raw_fd());
|
||||
|
||||
let mut last_err = None;
|
||||
for attempt in 0..=2 {
|
||||
@@ -5131,14 +4978,14 @@ pub(crate) fn stage_inline_rollback_backup_with_commit_guard_std(
|
||||
|
||||
let stage_res = (|| -> io::Result<()> {
|
||||
let linked = if allow_hardlink {
|
||||
match linkat(CWD, proc_fd_path.as_str(), &rollback_dir_file, backup_name, AtFlags::SYMLINK_FOLLOW) {
|
||||
match linkat(&*dst_dir.file, dst_file_name, &rollback_dir_file, backup_name, AtFlags::empty()) {
|
||||
Ok(()) => true,
|
||||
Err(Errno::EXIST) => {
|
||||
match unlinkat(&rollback_dir_file, backup_name, AtFlags::empty()) {
|
||||
Ok(()) | Err(Errno::NOENT) => {}
|
||||
Err(err) => return Err(io::Error::from(err)),
|
||||
}
|
||||
linkat(CWD, proc_fd_path.as_str(), &rollback_dir_file, backup_name, AtFlags::SYMLINK_FOLLOW).is_ok()
|
||||
linkat(&*dst_dir.file, dst_file_name, &rollback_dir_file, backup_name, AtFlags::empty()).is_ok()
|
||||
}
|
||||
Err(_) => false,
|
||||
}
|
||||
@@ -5146,8 +4993,9 @@ pub(crate) fn stage_inline_rollback_backup_with_commit_guard_std(
|
||||
false
|
||||
};
|
||||
if linked {
|
||||
if sync_backup && !old_metadata.is_empty() {
|
||||
old_destination_file.file.sync_data()?;
|
||||
if cfg!(test) && sync_backup && !old_metadata.is_empty() {
|
||||
open_linux_relative(&rollback_dir_file, Path::new(backup_name), OFlags::RDONLY, Mode::empty())?
|
||||
.sync_data()?;
|
||||
}
|
||||
} else {
|
||||
match unlinkat(&rollback_dir_file, backup_name, AtFlags::empty()) {
|
||||
@@ -6218,18 +6066,7 @@ impl PublicationRoot {
|
||||
#[cfg(target_os = "linux")]
|
||||
let io_path = {
|
||||
use std::os::fd::AsRawFd;
|
||||
use std::os::unix::fs::MetadataExt;
|
||||
let proc_alias = PathBuf::from(format!("/proc/self/fd/{}", directory.as_raw_fd()));
|
||||
if let Ok(metadata) = directory.metadata()
|
||||
&& metadata.is_dir()
|
||||
{
|
||||
let identity = (metadata.dev(), metadata.ino());
|
||||
NATIVE_PUBLICATION_HISTORY
|
||||
.lock()
|
||||
.live_proc_roots
|
||||
.insert(proc_alias.clone(), (identity, Arc::downgrade(&directory)));
|
||||
}
|
||||
Arc::new(proc_alias.join("."))
|
||||
Arc::new(PathBuf::from(format!("/proc/self/fd/{}/.", directory.as_raw_fd())))
|
||||
};
|
||||
|
||||
Ok(Self {
|
||||
@@ -6530,7 +6367,8 @@ pub(crate) struct RenameCommitGuard {
|
||||
#[cfg(target_os = "linux")]
|
||||
struct LinuxCommitDirectory {
|
||||
file: Arc<std::fs::File>,
|
||||
identity: std::sync::OnceLock<(u64, u64)>,
|
||||
device: u64,
|
||||
inode: u64,
|
||||
}
|
||||
|
||||
#[cfg(target_os = "linux")]
|
||||
@@ -6540,38 +6378,10 @@ impl LinuxCommitDirectory {
|
||||
let metadata = file.metadata()?;
|
||||
Ok(Self {
|
||||
file: Arc::new(file),
|
||||
identity: std::sync::OnceLock::from((metadata.dev(), metadata.ino())),
|
||||
device: metadata.dev(),
|
||||
inode: metadata.ino(),
|
||||
})
|
||||
}
|
||||
|
||||
fn new_lazy(file: std::fs::File) -> Self {
|
||||
Self {
|
||||
file: Arc::new(file),
|
||||
identity: std::sync::OnceLock::new(),
|
||||
}
|
||||
}
|
||||
|
||||
fn identity(&self) -> io::Result<(u64, u64)> {
|
||||
if let Some(&id) = self.identity.get() {
|
||||
return Ok(id);
|
||||
}
|
||||
use std::os::unix::fs::MetadataExt;
|
||||
let metadata = self.file.metadata()?;
|
||||
let id = (metadata.dev(), metadata.ino());
|
||||
let _ = self.identity.set(id);
|
||||
Ok(id)
|
||||
}
|
||||
|
||||
fn clone_shared(&self) -> Self {
|
||||
let identity = std::sync::OnceLock::new();
|
||||
if let Some(&id) = self.identity.get() {
|
||||
let _ = identity.set(id);
|
||||
}
|
||||
Self {
|
||||
file: Arc::clone(&self.file),
|
||||
identity,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(target_os = "linux")]
|
||||
@@ -6579,7 +6389,7 @@ struct LinuxCommitDirectories {
|
||||
root: PublicationRoot,
|
||||
source_path: PathBuf,
|
||||
destination_path: PathBuf,
|
||||
source: std::sync::OnceLock<LinuxCommitDirectory>,
|
||||
source: LinuxCommitDirectory,
|
||||
destination: std::sync::OnceLock<LinuxCommitDirectory>,
|
||||
destination_ancestors: std::sync::OnceLock<Vec<std::fs::File>>,
|
||||
destination_base: PathBuf,
|
||||
@@ -6597,24 +6407,12 @@ impl LinuxCommitDirectories {
|
||||
.pop()
|
||||
.ok_or_else(|| io::Error::other("missing destination directory guard"))?,
|
||||
);
|
||||
let directory = LinuxCommitDirectory::new_lazy(directory);
|
||||
let directory = LinuxCommitDirectory::new(directory)?;
|
||||
let ancestors = parents.into_iter().map(std::fs::File::from).collect();
|
||||
let _ = self.destination_ancestors.set(ancestors);
|
||||
Ok(self.destination.get_or_init(|| directory))
|
||||
}
|
||||
|
||||
fn ensure_source_eager(&self) -> io::Result<&LinuxCommitDirectory> {
|
||||
if let Some(source) = self.source.get() {
|
||||
let _ = source.identity()?;
|
||||
return Ok(source);
|
||||
}
|
||||
let directory = LinuxCommitDirectory::new(self.root.open_directory(&self.source_path)?)?;
|
||||
let _ = self.source.set(directory);
|
||||
self.source
|
||||
.get()
|
||||
.ok_or_else(|| io::Error::other("missing source commit directory"))
|
||||
}
|
||||
|
||||
fn file_name<'a>(&self, path: &'a Path, source: bool) -> io::Result<&'a std::ffi::OsStr> {
|
||||
let parent = if source { &self.source_path } else { &self.destination_path };
|
||||
rustfs_utils::path::unix_direct_child_file_name(path, parent)
|
||||
@@ -6625,20 +6423,18 @@ impl LinuxCommitDirectories {
|
||||
use std::os::unix::fs::MetadataExt;
|
||||
// A pinned directory may have been unlinked or replaced. Do not report a
|
||||
// successful publication into a detached object tree.
|
||||
let source = self.ensure_source_eager()?;
|
||||
let source_identity = source.identity()?;
|
||||
let current = self.root.open_directory(&self.source_path)?.metadata()?;
|
||||
if (current.dev(), current.ino()) != source_identity {
|
||||
if (current.dev(), current.ino()) != (self.source.device, self.source.inode) {
|
||||
return Err(io::Error::new(io::ErrorKind::InvalidData, "commit directory identity changed"));
|
||||
}
|
||||
let destination = self.prepare_destination()?;
|
||||
if check_destination {
|
||||
let destination_identity = destination.identity()?;
|
||||
if source_identity != destination_identity || self.source_path.as_os_str() != self.destination_path.as_os_str() {
|
||||
let current = self.root.open_directory(&self.destination_path)?.metadata()?;
|
||||
if (current.dev(), current.ino()) != destination_identity {
|
||||
return Err(io::Error::new(io::ErrorKind::InvalidData, "commit directory identity changed"));
|
||||
}
|
||||
if check_destination
|
||||
&& ((self.source.device, self.source.inode) != (destination.device, destination.inode)
|
||||
|| self.source_path.as_os_str() != self.destination_path.as_os_str())
|
||||
{
|
||||
let current = self.root.open_directory(&self.destination_path)?.metadata()?;
|
||||
if (current.dev(), current.ino()) != (destination.device, destination.inode) {
|
||||
return Err(io::Error::new(io::ErrorKind::InvalidData, "commit directory identity changed"));
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
@@ -6649,42 +6445,12 @@ impl LinuxCommitDirectories {
|
||||
let source_name = self.file_name(source, true)?;
|
||||
let destination_name = self.file_name(destination, false)?;
|
||||
let destination_was_prepared = self.destination.get().is_some();
|
||||
let source_was_prepared = self.source.get().is_some();
|
||||
let destination_parent = self.prepare_destination()?;
|
||||
let source_dir = if source_was_prepared || prepared.is_none() {
|
||||
self.validate(destination_was_prepared)?;
|
||||
self.source
|
||||
.get()
|
||||
.ok_or_else(|| io::Error::other("missing source commit directory"))?
|
||||
} else {
|
||||
if destination_was_prepared {
|
||||
use std::os::unix::fs::MetadataExt;
|
||||
let destination_identity = destination_parent.identity()?;
|
||||
let current = self.root.open_directory(&self.destination_path)?.metadata()?;
|
||||
if (current.dev(), current.ino()) != destination_identity {
|
||||
return Err(io::Error::new(io::ErrorKind::InvalidData, "commit directory identity changed"));
|
||||
}
|
||||
}
|
||||
let dir = self.root.open_directory(&self.source_path)?;
|
||||
self.source.get_or_init(|| LinuxCommitDirectory::new_lazy(dir))
|
||||
};
|
||||
self.validate(destination_was_prepared)?;
|
||||
let mut attempt = 0;
|
||||
loop {
|
||||
if let Some(prepared) = prepared {
|
||||
let metadata = match rustix::fs::statat(&*source_dir.file, source_name, rustix::fs::AtFlags::SYMLINK_NOFOLLOW) {
|
||||
Ok(metadata) => metadata,
|
||||
Err(rustix::io::Errno::NOENT) if !source_was_prepared => {
|
||||
use std::os::unix::fs::MetadataExt;
|
||||
if prepared.source.metadata().is_ok_and(|meta| meta.nlink() > 0) {
|
||||
return Err(io::Error::new(
|
||||
io::ErrorKind::InvalidData,
|
||||
"rename source identity changed while publication was prepared",
|
||||
));
|
||||
}
|
||||
return Err(io::Error::from(rustix::io::Errno::NOENT));
|
||||
}
|
||||
Err(err) => return Err(io::Error::from(err)),
|
||||
};
|
||||
let metadata = rustix::fs::statat(&*self.source.file, source_name, rustix::fs::AtFlags::SYMLINK_NOFOLLOW)?;
|
||||
if (prepared.path.as_os_str() != source.as_os_str() && prepared.path != source)
|
||||
|| metadata.st_dev != prepared.device
|
||||
|| metadata.st_ino != prepared.inode
|
||||
@@ -6696,7 +6462,7 @@ impl LinuxCommitDirectories {
|
||||
}
|
||||
}
|
||||
let started = rustfs_io_metrics::put_stage_timer();
|
||||
let result = rustix::fs::renameat(&*source_dir.file, source_name, &*destination_parent.file, destination_name)
|
||||
let result = rustix::fs::renameat(&*self.source.file, source_name, &*destination_parent.file, destination_name)
|
||||
.map_err(io::Error::from);
|
||||
rustfs_io_metrics::record_put_object_stage_duration_from(
|
||||
rustfs_io_metrics::PUT_STAGE_SET_DISK_RENAME_RENAME_SYSCALL,
|
||||
@@ -6793,12 +6559,11 @@ impl RenameDestinationPathGuard {
|
||||
{
|
||||
let _ = self;
|
||||
before_publish()?;
|
||||
match std::fs::remove_file(file_path) {
|
||||
Ok(()) => {}
|
||||
Err(err) if err.kind() == io::ErrorKind::NotFound => {}
|
||||
Err(err) => return Err(err),
|
||||
}
|
||||
let mut file = std::fs::OpenOptions::new().create_new(true).write(true).open(file_path)?;
|
||||
let mut file = std::fs::OpenOptions::new()
|
||||
.create(true)
|
||||
.write(true)
|
||||
.truncate(true)
|
||||
.open(file_path)?;
|
||||
std::io::Write::write_all(&mut file, data)?;
|
||||
if sync_file {
|
||||
file.sync_data()?;
|
||||
@@ -6812,16 +6577,6 @@ impl RenameDestinationPathGuard {
|
||||
}
|
||||
|
||||
impl RenameCommitGuard {
|
||||
#[cfg(target_os = "linux")]
|
||||
pub(crate) fn has_prepared_destination(&self) -> bool {
|
||||
self.directories.destination.get().is_some()
|
||||
}
|
||||
|
||||
#[cfg(target_os = "linux")]
|
||||
pub(crate) fn ensure_source_pinned(&self) -> io::Result<()> {
|
||||
self.directories.ensure_source_eager().map(|_| ())
|
||||
}
|
||||
|
||||
#[cfg(windows)]
|
||||
pub(crate) fn lock_source_directory_for_path_access(&self, directory: &Path) -> io::Result<RenameDestinationPathGuard> {
|
||||
if directory != self.source_parent {
|
||||
@@ -6916,16 +6671,19 @@ pub(crate) fn multipart_payload_commit_guard_std(
|
||||
original.validate(true)?;
|
||||
let source = LinuxCommitDirectory::new(original.root.open_directory(source_parent)?)?;
|
||||
let destination = original.prepare_destination()?;
|
||||
let _ = destination.identity()?;
|
||||
// Clone the pinned file identity, never recapture the target pathname
|
||||
// between payload and canonical-head publication.
|
||||
let destination = destination.clone_shared();
|
||||
let destination = LinuxCommitDirectory {
|
||||
file: Arc::clone(&destination.file),
|
||||
device: destination.device,
|
||||
inode: destination.inode,
|
||||
};
|
||||
Ok(RenameCommitGuard {
|
||||
directories: Arc::new(LinuxCommitDirectories {
|
||||
root: original.root.clone(),
|
||||
source_path: source_parent.to_path_buf(),
|
||||
destination_path: original.destination_path.clone(),
|
||||
source: std::sync::OnceLock::from(source),
|
||||
source,
|
||||
destination: std::sync::OnceLock::from(destination),
|
||||
destination_ancestors: std::sync::OnceLock::new(),
|
||||
destination_base: original.destination_base.clone(),
|
||||
@@ -6976,36 +6734,6 @@ pub(crate) async fn multipart_payload_commit_guard(
|
||||
run_blocking_namespace_operation(lease, operation).await
|
||||
}
|
||||
|
||||
#[cfg(target_os = "linux")]
|
||||
pub(crate) fn prepare_rename_commit_guard_flat_inline(
|
||||
source_parent: &Path,
|
||||
destination_parent: &Path,
|
||||
destination_base: &Path,
|
||||
publication_root: &PublicationRoot,
|
||||
) -> io::Result<RenameCommitGuard> {
|
||||
existing_base_directory_suffix(destination_parent, destination_base)?;
|
||||
publication_root.relative_path(source_parent)?;
|
||||
let destination = std::sync::OnceLock::new();
|
||||
match publication_root.open_directory(destination_parent) {
|
||||
Ok(directory) => {
|
||||
let _ = destination.set(LinuxCommitDirectory::new(directory)?);
|
||||
}
|
||||
Err(err) if err.kind() == io::ErrorKind::NotFound => {}
|
||||
Err(err) => return Err(err),
|
||||
}
|
||||
Ok(RenameCommitGuard {
|
||||
directories: Arc::new(LinuxCommitDirectories {
|
||||
root: publication_root.clone(),
|
||||
source_path: source_parent.to_path_buf(),
|
||||
destination_path: destination_parent.to_path_buf(),
|
||||
source: std::sync::OnceLock::new(),
|
||||
destination,
|
||||
destination_ancestors: std::sync::OnceLock::new(),
|
||||
destination_base: destination_base.to_path_buf(),
|
||||
}),
|
||||
})
|
||||
}
|
||||
|
||||
pub(crate) fn prepare_rename_commit_guard(
|
||||
source_parent: &Path,
|
||||
destination_parent: &Path,
|
||||
@@ -7018,7 +6746,11 @@ pub(crate) fn prepare_rename_commit_guard(
|
||||
let source = LinuxCommitDirectory::new(publication_root.open_directory(source_parent)?)?;
|
||||
let destination = std::sync::OnceLock::new();
|
||||
if source_parent == destination_parent {
|
||||
let _ = destination.set(source.clone_shared());
|
||||
let _ = destination.set(LinuxCommitDirectory {
|
||||
file: Arc::clone(&source.file),
|
||||
device: source.device,
|
||||
inode: source.inode,
|
||||
});
|
||||
} else {
|
||||
match publication_root.open_directory(destination_parent) {
|
||||
Ok(directory) => {
|
||||
@@ -7033,7 +6765,7 @@ pub(crate) fn prepare_rename_commit_guard(
|
||||
root: publication_root.clone(),
|
||||
source_path: source_parent.to_path_buf(),
|
||||
destination_path: destination_parent.to_path_buf(),
|
||||
source: std::sync::OnceLock::from(source),
|
||||
source,
|
||||
destination,
|
||||
destination_ancestors: std::sync::OnceLock::new(),
|
||||
destination_base: destination_base.to_path_buf(),
|
||||
@@ -8131,8 +7863,15 @@ fn same_parent_commit_guard(
|
||||
) -> io::Result<RenameCommitGuard> {
|
||||
#[cfg(target_os = "linux")]
|
||||
let same_dir = if let Some(directory_guard) = directory_guard.filter(|g| g.directories.source_path == parent) {
|
||||
let dir = directory_guard.directories.ensure_source_eager()?;
|
||||
Some((directory_guard.directories.root.clone(), dir.clone_shared()))
|
||||
let dir = &directory_guard.directories.source;
|
||||
Some((
|
||||
directory_guard.directories.root.clone(),
|
||||
LinuxCommitDirectory {
|
||||
file: Arc::clone(&dir.file),
|
||||
device: dir.device,
|
||||
inode: dir.inode,
|
||||
},
|
||||
))
|
||||
} else if let Some(fd) = leaf_fd {
|
||||
existing_base_directory_suffix(parent, base)?;
|
||||
Some((publication_root.clone(), LinuxCommitDirectory::new(std::fs::File::from(fd))?))
|
||||
@@ -8152,13 +7891,16 @@ fn same_parent_commit_guard(
|
||||
};
|
||||
#[cfg(target_os = "linux")]
|
||||
if let Some((root, dir)) = same_dir {
|
||||
let source_dir = dir.clone_shared();
|
||||
return Ok(RenameCommitGuard {
|
||||
directories: Arc::new(LinuxCommitDirectories {
|
||||
root,
|
||||
source_path: parent.to_path_buf(),
|
||||
destination_path: parent.to_path_buf(),
|
||||
source: std::sync::OnceLock::from(source_dir),
|
||||
source: LinuxCommitDirectory {
|
||||
file: Arc::clone(&dir.file),
|
||||
device: dir.device,
|
||||
inode: dir.inode,
|
||||
},
|
||||
destination: std::sync::OnceLock::from(dir),
|
||||
destination_ancestors: std::sync::OnceLock::new(),
|
||||
destination_base: base.to_path_buf(),
|
||||
@@ -8341,8 +8083,7 @@ pub(crate) fn rename_multipart_source_owned_std(
|
||||
}
|
||||
precondition()?;
|
||||
guard.directories.rename(source, destination, Some(&pinned_source))?;
|
||||
let source_dir = guard.directories.ensure_source_eager()?;
|
||||
fsync_directory_handle_std(source_parent, Some(&source_dir.file))?;
|
||||
fsync_directory_handle_std(source_parent, Some(&guard.directories.source.file))?;
|
||||
fsync_commit_directory_limited_std(destination_parent, &guard)?;
|
||||
}
|
||||
#[cfg(not(target_os = "linux"))]
|
||||
@@ -12720,6 +12461,7 @@ mod tests {
|
||||
|
||||
let _group_commit = set_file_fdatasync_group_commit_for_test(true);
|
||||
set_file_fdatasync_group_commit_wait_for_test(0);
|
||||
file_fdatasync_group_commit_override::set_follower_timeout_micros(1_000);
|
||||
clear_file_fdatasync_group_commit_for_test();
|
||||
let temp_dir = tempdir().expect("create temp dir");
|
||||
let first_dir = temp_dir.path().join("first");
|
||||
@@ -12747,7 +12489,7 @@ mod tests {
|
||||
|
||||
let second_limiter = limiter.clone();
|
||||
let second_path = second_dir.clone();
|
||||
let second = tokio::spawn(async move { sync_dir_files_with_limiter(second_path, second_limiter).await });
|
||||
let mut second = tokio::spawn(async move { sync_dir_files_with_limiter(second_path, second_limiter).await });
|
||||
tokio::time::timeout(Duration::from_secs(30), async {
|
||||
loop {
|
||||
if file_fdatasync_group_commit_counts_for_test().1 == 2 {
|
||||
@@ -12760,6 +12502,10 @@ mod tests {
|
||||
.expect("second waiter should enqueue before releasing the grouped batch");
|
||||
release_batch_tx.send(()).expect("release group batch hook");
|
||||
file_sync_probe::wait_for_active(1).await;
|
||||
assert!(
|
||||
tokio::time::timeout(Duration::from_millis(10), &mut second).await.is_err(),
|
||||
"a claimed follower waits for its batch even after its queue deadline"
|
||||
);
|
||||
assert_eq!(file_sync_probe::attempts(), 1, "claimed files must not enter an independent sync");
|
||||
|
||||
assert_eq!(
|
||||
@@ -15076,95 +14822,6 @@ mod fd_relative_tests {
|
||||
use super::*;
|
||||
use std::io::{Read, Write};
|
||||
|
||||
#[test]
|
||||
fn live_proc_root_fast_path_preserves_three_way_semantics_and_retires_on_drop() {
|
||||
use std::sync::atomic::{AtomicUsize, Ordering};
|
||||
|
||||
let temp = tempfile::tempdir().expect("temporary disk");
|
||||
let root = PublicationRoot::new(temp.path()).expect("pinned root");
|
||||
let io_path = root.io_path().to_path_buf();
|
||||
let proc_alias: PathBuf = io_path.components().collect();
|
||||
let reg = register_native_publication_root(temp.path(), &io_path).expect("register root");
|
||||
let backend: Arc<dyn super::super::local::LocalIoBackend> =
|
||||
Arc::new(super::super::local::StdBackend::new(temp.path().to_path_buf()));
|
||||
let disk_id = uuid::Uuid::new_v4();
|
||||
|
||||
let bucket_a = format!("proc-fast-a-{}", uuid::Uuid::new_v4());
|
||||
let bucket_b = format!("proc-fast-b-{}", uuid::Uuid::new_v4());
|
||||
let witness_a = reg
|
||||
.witness(&bucket_a, disk_id, Arc::clone(&backend))
|
||||
.expect("witness a before publication");
|
||||
let witness_b = reg
|
||||
.witness(&bucket_b, disk_id, Arc::clone(&backend))
|
||||
.expect("witness b before publication");
|
||||
|
||||
let stat_hits = Arc::new(AtomicUsize::new(0));
|
||||
let stat_hits_hook = Arc::clone(&stat_hits);
|
||||
let stat_guard = prepared_publication_test_hooks::install_at(
|
||||
prepared_publication_test_hooks::Stage::NativeRootIdentity,
|
||||
&proc_alias,
|
||||
move || {
|
||||
stat_hits_hook.fetch_add(1, Ordering::SeqCst);
|
||||
},
|
||||
);
|
||||
|
||||
// Root-level path (`path == alias` or `alias/.`) must be a no-op and must not set `unknown` or stat.
|
||||
record_native_publication(&io_path);
|
||||
record_native_publication(&proc_alias);
|
||||
assert_eq!(stat_hits.load(Ordering::SeqCst), 0, "live proc fast path must skip stat");
|
||||
drop(stat_guard);
|
||||
assert!(validate_native_creation_witnesses(std::slice::from_ref(&witness_a)));
|
||||
assert!(validate_native_creation_witnesses(std::slice::from_ref(&witness_b)));
|
||||
|
||||
// Publishing into `bucket_a` marks only `bucket_a` as started without statting or poisoning `bucket_b`.
|
||||
let stat_hits_hook = Arc::clone(&stat_hits);
|
||||
let stat_guard = prepared_publication_test_hooks::install_at(
|
||||
prepared_publication_test_hooks::Stage::NativeRootIdentity,
|
||||
&proc_alias,
|
||||
move || {
|
||||
stat_hits_hook.fetch_add(1, Ordering::SeqCst);
|
||||
},
|
||||
);
|
||||
record_native_publication(&io_path.join(&bucket_a).join("object/xl.meta"));
|
||||
assert_eq!(stat_hits.load(Ordering::SeqCst), 0, "live proc fast path must skip stat");
|
||||
drop(stat_guard);
|
||||
assert!(!validate_native_creation_witnesses(std::slice::from_ref(&witness_a)));
|
||||
assert!(validate_native_creation_witnesses(std::slice::from_ref(&witness_b)));
|
||||
|
||||
// Non-Normal component (`..`) sets `unknown = true`, invalidating remaining witnesses on that root.
|
||||
let stat_hits_hook = Arc::clone(&stat_hits);
|
||||
let stat_guard = prepared_publication_test_hooks::install_at(
|
||||
prepared_publication_test_hooks::Stage::NativeRootIdentity,
|
||||
&proc_alias,
|
||||
move || {
|
||||
stat_hits_hook.fetch_add(1, Ordering::SeqCst);
|
||||
},
|
||||
);
|
||||
record_native_publication(&io_path.join(&bucket_b).join("../escape"));
|
||||
assert_eq!(stat_hits.load(Ordering::SeqCst), 0, "live proc fast path must skip stat");
|
||||
drop(stat_guard);
|
||||
assert!(!validate_native_creation_witnesses(std::slice::from_ref(&witness_b)));
|
||||
|
||||
// Dropping the last strong owner of the root directory fd retires the live proc entry and falls back to stat.
|
||||
assert!(NATIVE_PUBLICATION_HISTORY.lock().live_proc_roots.contains_key(&proc_alias));
|
||||
drop(root);
|
||||
let stat_hits_hook = Arc::clone(&stat_hits);
|
||||
let _stat_guard = prepared_publication_test_hooks::install_at(
|
||||
prepared_publication_test_hooks::Stage::NativeRootIdentity,
|
||||
&proc_alias,
|
||||
move || {
|
||||
stat_hits_hook.fetch_add(1, Ordering::SeqCst);
|
||||
},
|
||||
);
|
||||
record_native_publication(&io_path.join("after-drop"));
|
||||
assert!(!NATIVE_PUBLICATION_HISTORY.lock().live_proc_roots.contains_key(&proc_alias));
|
||||
assert_eq!(
|
||||
stat_hits.load(Ordering::SeqCst),
|
||||
1,
|
||||
"retired proc root must fall back to native_root_identity"
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn guarded_first_publication_recovers_from_a_concurrent_parent_prune() {
|
||||
let temp = tempfile::tempdir().expect("temporary disk");
|
||||
|
||||
@@ -303,24 +303,6 @@ fn log_read_error(error: &std::io::Error) {
|
||||
}
|
||||
}
|
||||
|
||||
/// Verify one borrowed `[hash][data]` frame before exposing its payload.
|
||||
/// A short digest prefix and a digest mismatch both fail without copying data.
|
||||
pub(crate) fn verified_bitrot_frame<'a>(
|
||||
hash_algo: &HashAlgorithm,
|
||||
skip_verify: bool,
|
||||
block: &'a [u8],
|
||||
) -> std::io::Result<&'a [u8]> {
|
||||
let hash_size = hash_algo.size();
|
||||
if block.len() < hash_size {
|
||||
return Err(std::io::Error::new(std::io::ErrorKind::UnexpectedEof, "truncated bitrot frame digest"));
|
||||
}
|
||||
let (hash, data) = block.split_at(hash_size);
|
||||
if !skip_verify && hash_algo.hash_encode(data).as_ref() != hash {
|
||||
return Err(std::io::Error::new(std::io::ErrorKind::InvalidData, DiskError::FileCorrupt));
|
||||
}
|
||||
Ok(data)
|
||||
}
|
||||
|
||||
/// Split a `[hash][data]` block, verify the hash (unless `skip_verify`), and
|
||||
/// return the data slice plus the time spent hashing. The caller writes `data`
|
||||
/// into its sink **only after** this returns `Ok`, so a shard that fails the
|
||||
@@ -328,13 +310,17 @@ pub(crate) fn verified_bitrot_frame<'a>(
|
||||
/// rather than stored so this stays a free function usable while `self` is
|
||||
/// borrowed for the block.
|
||||
fn split_and_verify<'a>(hash_algo: &HashAlgorithm, skip_verify: bool, block: &'a [u8]) -> std::io::Result<(&'a [u8], Duration)> {
|
||||
let verify_start = (!skip_verify).then(std::time::Instant::now);
|
||||
let data = verified_bitrot_frame(hash_algo, skip_verify, block).inspect_err(|err| {
|
||||
if err.kind() == std::io::ErrorKind::InvalidData {
|
||||
log_hash_mismatch(block.len().saturating_sub(hash_algo.size()));
|
||||
}
|
||||
})?;
|
||||
let verify = verify_start.map_or(Duration::ZERO, |start| start.elapsed());
|
||||
let (hash, data) = block.split_at(hash_algo.size());
|
||||
if skip_verify {
|
||||
return Ok((data, Duration::ZERO));
|
||||
}
|
||||
let verify_start = std::time::Instant::now();
|
||||
let actual_hash = hash_algo.hash_encode(data);
|
||||
let verify = verify_start.elapsed();
|
||||
if actual_hash.as_ref() != hash {
|
||||
log_hash_mismatch(data.len());
|
||||
return Err(std::io::Error::new(std::io::ErrorKind::InvalidData, DiskError::FileCorrupt));
|
||||
}
|
||||
Ok((data, verify))
|
||||
}
|
||||
|
||||
@@ -979,11 +965,10 @@ pub struct InlineBuffer {
|
||||
foreground: Vec<u8>,
|
||||
background: Option<crate::disk::background_memory::BackgroundBuffer>,
|
||||
filled: usize,
|
||||
max_len: Option<usize>,
|
||||
}
|
||||
|
||||
impl InlineBuffer {
|
||||
fn new(max_len: Option<usize>) -> Self {
|
||||
fn new() -> Self {
|
||||
let background = crate::disk::current_resource_unit()
|
||||
.and_then(|unit| unit.memory_budget())
|
||||
.map(|budget| crate::disk::background_memory::BackgroundBuffer::new(Some(budget)));
|
||||
@@ -991,7 +976,6 @@ impl InlineBuffer {
|
||||
foreground: Vec::new(),
|
||||
background,
|
||||
filled: 0,
|
||||
max_len,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1006,22 +990,10 @@ impl InlineBuffer {
|
||||
let total = bufs
|
||||
.iter()
|
||||
.try_fold(0usize, |length, buf| length.checked_add(buf.len()).ok_or(std::io::ErrorKind::OutOfMemory))?;
|
||||
let length = self
|
||||
.as_slice()
|
||||
.len()
|
||||
.checked_add(total)
|
||||
.ok_or(std::io::ErrorKind::OutOfMemory)?;
|
||||
if self.max_len.is_some_and(|limit| length > limit) {
|
||||
return Err(std::io::Error::new(
|
||||
std::io::ErrorKind::InvalidData,
|
||||
"inline shard exceeds its admitted buffer limit",
|
||||
));
|
||||
}
|
||||
if let Some(buffer) = &mut self.background {
|
||||
let length = self.filled.checked_add(total).ok_or(std::io::ErrorKind::OutOfMemory)?;
|
||||
if length > buffer.as_slice().len() {
|
||||
let capacity = self
|
||||
.max_len
|
||||
.unwrap_or_else(|| buffer.as_slice().len().checked_mul(2).unwrap_or(length).max(length));
|
||||
let capacity = buffer.as_slice().len().checked_mul(2).unwrap_or(length).max(length);
|
||||
// Admission includes both allocations until the prefix has been copied.
|
||||
buffer.grow(capacity, self.filled)?;
|
||||
}
|
||||
@@ -1032,13 +1004,6 @@ impl InlineBuffer {
|
||||
}
|
||||
self.filled = length;
|
||||
} else {
|
||||
if let Some(limit) = self.max_len
|
||||
&& length > self.foreground.capacity()
|
||||
{
|
||||
self.foreground
|
||||
.try_reserve_exact(limit - self.foreground.len())
|
||||
.map_err(std::io::Error::other)?;
|
||||
}
|
||||
for buf in bufs {
|
||||
self.foreground.extend_from_slice(buf);
|
||||
}
|
||||
@@ -1057,11 +1022,7 @@ impl InlineBuffer {
|
||||
impl CustomWriter {
|
||||
/// Create a new inline buffer writer
|
||||
pub fn new_inline_buffer() -> Self {
|
||||
Self::InlineBuffer(InlineBuffer::new(None))
|
||||
}
|
||||
|
||||
pub(crate) fn new_bounded_inline_buffer(max_len: usize) -> Self {
|
||||
Self::InlineBuffer(InlineBuffer::new(Some(max_len)))
|
||||
Self::InlineBuffer(InlineBuffer::new())
|
||||
}
|
||||
|
||||
/// Create a new disk writer from any AsyncWrite implementation
|
||||
@@ -1514,80 +1475,6 @@ mod tests {
|
||||
use std::time::Duration;
|
||||
use tokio::io::{AsyncRead, AsyncWrite, AsyncWriteExt, ReadBuf};
|
||||
|
||||
#[test]
|
||||
fn borrowed_bitrot_frame_verification_preserves_slices_and_rejects_short_prefixes() {
|
||||
for algo in [
|
||||
HashAlgorithm::HighwayHash256S,
|
||||
HashAlgorithm::HighwayHash256SLegacy,
|
||||
HashAlgorithm::SHA256,
|
||||
HashAlgorithm::None,
|
||||
] {
|
||||
let payload = b"borrowed frame";
|
||||
let mut frame = algo.hash_encode(payload).as_ref().to_vec();
|
||||
frame.extend_from_slice(payload);
|
||||
let verified = super::verified_bitrot_frame(&algo, false, &frame).expect("valid borrowed frame");
|
||||
assert_eq!(verified, payload);
|
||||
assert_eq!(verified.as_ptr(), frame[algo.size()..].as_ptr());
|
||||
for prefix_len in 0..algo.size() {
|
||||
for skip_verify in [false, true] {
|
||||
assert_eq!(
|
||||
super::split_and_verify(&algo, skip_verify, &frame[..prefix_len])
|
||||
.expect_err("short prefix must be an error")
|
||||
.kind(),
|
||||
io::ErrorKind::UnexpectedEof
|
||||
);
|
||||
}
|
||||
}
|
||||
if algo.size() != 0 {
|
||||
frame[0] ^= 1;
|
||||
assert_eq!(
|
||||
super::verified_bitrot_frame(&algo, false, &frame)
|
||||
.expect_err("damaged digest")
|
||||
.kind(),
|
||||
io::ErrorKind::InvalidData
|
||||
);
|
||||
assert_eq!(
|
||||
super::verified_bitrot_frame(&algo, true, &frame).expect("explicit diagnostic skip"),
|
||||
payload
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
#[ignore = "manual ABBA comparison of borrowed and streaming verification kernels"]
|
||||
async fn inline_bitrot_borrowed_verification_benchmark() {
|
||||
let algo = HashAlgorithm::HighwayHash256S;
|
||||
for (part_size, shard_size, iterations) in [(4096, 4096, 20_000), (128 * 1024, 64 * 1024, 2000)] {
|
||||
let data = vec![0x7b; part_size];
|
||||
let mut encoded = Vec::new();
|
||||
for part in data.chunks(shard_size) {
|
||||
encoded.extend_from_slice(algo.hash_encode(part).as_ref());
|
||||
encoded.extend_from_slice(part);
|
||||
}
|
||||
for borrowed in [false, true, true, false] {
|
||||
let start = std::time::Instant::now();
|
||||
for _ in 0..iterations {
|
||||
let encoded = std::hint::black_box(encoded.as_slice());
|
||||
if borrowed {
|
||||
for frame in encoded.chunks(shard_size + algo.size()) {
|
||||
std::hint::black_box(super::verified_bitrot_frame(&algo, false, frame).expect("borrowed frame"));
|
||||
}
|
||||
} else {
|
||||
bitrot_verify(Cursor::new(encoded), encoded.len(), part_size, algo.clone(), shard_size)
|
||||
.await
|
||||
.expect("streamed frame");
|
||||
}
|
||||
}
|
||||
println!(
|
||||
"inline_verify debug={} size={part_size} borrowed={borrowed} iterations={iterations} elapsed_ns={}",
|
||||
cfg!(debug_assertions),
|
||||
start.elapsed().as_nanos()
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn bitrot_damage_preserves_typed_repair_evidence() {
|
||||
use crate::disk::error::Error as DiskError;
|
||||
@@ -3056,25 +2943,6 @@ mod tests {
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn bounded_inline_buffer_rejects_vectored_overflow_atomically() {
|
||||
use tokio::io::AsyncWriteExt;
|
||||
let mut writer = CustomWriter::new_bounded_inline_buffer(64);
|
||||
writer.write_all(&[7; 32]).await.expect("initial inline prefix");
|
||||
let error = writer
|
||||
.write_vectored(&[IoSlice::new(&[8; 16]), IoSlice::new(&[9; 17])])
|
||||
.await
|
||||
.expect_err("combined vectored data exceeds the bound");
|
||||
assert_eq!(error.kind(), io::ErrorKind::InvalidData);
|
||||
assert_eq!(writer.get_inline_data(), Some(&[7; 32][..]));
|
||||
writer.write_all(&[8; 32]).await.expect("fill exactly to the bound");
|
||||
assert!(writer.write_all(&[1]).await.is_err());
|
||||
if let CustomWriter::InlineBuffer(buffer) = &writer {
|
||||
assert!(buffer.foreground.capacity() <= 64);
|
||||
}
|
||||
assert_eq!(writer.into_inline_data().expect("export bounded bytes").len(), 64);
|
||||
}
|
||||
|
||||
mod inline_memory_regressions {
|
||||
use super::*;
|
||||
use crate::disk::background_memory::is_memory_pressure;
|
||||
@@ -3089,25 +2957,6 @@ mod tests {
|
||||
.await
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn bounded_inline_writer_retains_its_memory_receipt() {
|
||||
let memory = BackgroundMemoryBudget::new();
|
||||
memory.set_sampled_limit(256, Duration::from_secs(30));
|
||||
let (resources, _) = test_resource_units(1);
|
||||
let mut writer = with_optional_resource_unit(Some(resources.with_memory_budget(Arc::clone(&memory))), async {
|
||||
CustomWriter::new_bounded_inline_buffer(64)
|
||||
})
|
||||
.await;
|
||||
writer.write_all(&[3; 32]).await.expect("bounded background write");
|
||||
let bytes = writer.into_inline_data().expect("bounded bytes");
|
||||
let clone = bytes.clone();
|
||||
assert_eq!(memory.snapshot().used, 64);
|
||||
drop(bytes);
|
||||
assert_eq!(memory.snapshot().used, 64);
|
||||
drop(clone);
|
||||
assert_eq!(memory.snapshot().used, 0);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn inline_writer_rejects_closed_and_expired_owners_before_first_allocation() {
|
||||
for expired in [false, true] {
|
||||
|
||||
@@ -812,20 +812,12 @@ pub(crate) fn create_deferred_bitrot_reader_with_stripe_handle(
|
||||
/// a fixed body length when locating the authenticated trailer. Clamping it
|
||||
/// to `0` would claim an empty body and misframe the stream. `0` stays `0`
|
||||
/// because a genuinely empty object still means an empty body.
|
||||
fn bitrot_create_file_size(length: i64, shard_size: usize, checksum_algo: &HashAlgorithm) -> disk::error::Result<i64> {
|
||||
fn bitrot_create_file_size(length: i64, shard_size: usize, checksum_algo: &HashAlgorithm) -> i64 {
|
||||
if length <= 0 {
|
||||
return Ok(length);
|
||||
return length;
|
||||
}
|
||||
if shard_size == 0 {
|
||||
return Err(DiskError::other("bitrot shard size is zero"));
|
||||
}
|
||||
let payload = usize::try_from(length).map_err(|_| DiskError::other("bitrot shard length exceeds usize"))?;
|
||||
let framed = payload
|
||||
.div_ceil(shard_size)
|
||||
.checked_mul(checksum_algo.size())
|
||||
.and_then(|overhead| payload.checked_add(overhead))
|
||||
.ok_or_else(|| DiskError::other("bitrot framed length overflows"))?;
|
||||
i64::try_from(framed).map_err(|_| DiskError::other("bitrot framed length exceeds i64"))
|
||||
let length = length as usize;
|
||||
(length.div_ceil(shard_size) * checksum_algo.size() + length) as i64
|
||||
}
|
||||
|
||||
pub async fn create_bitrot_writer(
|
||||
@@ -851,16 +843,12 @@ pub async fn create_bitrot_writer_with_orig_volume(
|
||||
shard_size: usize,
|
||||
checksum_algo: HashAlgorithm,
|
||||
) -> disk::error::Result<BitrotWriterWrapper> {
|
||||
let framed_length = bitrot_create_file_size(length, shard_size, &checksum_algo)?;
|
||||
let writer = if is_inline_buffer {
|
||||
if framed_length < 0 {
|
||||
CustomWriter::new_inline_buffer()
|
||||
} else {
|
||||
let limit = usize::try_from(framed_length).map_err(|_| DiskError::other("inline buffer limit exceeds usize"))?;
|
||||
CustomWriter::new_bounded_inline_buffer(limit)
|
||||
}
|
||||
CustomWriter::new_inline_buffer()
|
||||
} else if let Some(disk) = disk {
|
||||
let file = disk.create_file(origvolume, volume, path, framed_length).await?;
|
||||
let length = bitrot_create_file_size(length, shard_size, &checksum_algo);
|
||||
|
||||
let file = disk.create_file(origvolume, volume, path, length).await?;
|
||||
#[cfg(feature = "hotpath")]
|
||||
let file = instrument_raw_shard_writer(file, disk.is_local());
|
||||
CustomWriter::from_file_writer(file)
|
||||
@@ -879,41 +867,23 @@ mod tests {
|
||||
|
||||
#[test]
|
||||
fn bitrot_create_file_size_grows_known_length_by_checksums() {
|
||||
// 10 bytes over 4-byte shards = 3 frames with a 32-byte prefix each.
|
||||
assert_eq!(
|
||||
bitrot_create_file_size(10, 4, &HashAlgorithm::HighwayHash256).expect("known frame size"),
|
||||
10 + 3 * 32
|
||||
);
|
||||
assert_eq!(bitrot_create_file_size(10, 4, &HashAlgorithm::None).expect("unhashed frame size"), 10);
|
||||
// 10 bytes over 4-byte shards = 3 shards, each followed by a 32-byte hash.
|
||||
assert_eq!(bitrot_create_file_size(10, 4, &HashAlgorithm::HighwayHash256), 10 + 3 * 32);
|
||||
assert_eq!(bitrot_create_file_size(10, 4, &HashAlgorithm::None), 10);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn bitrot_create_file_size_keeps_empty_and_unknown_distinct() {
|
||||
assert_eq!(
|
||||
bitrot_create_file_size(0, 4, &HashAlgorithm::HighwayHash256).expect("empty frame size"),
|
||||
0
|
||||
);
|
||||
assert_eq!(bitrot_create_file_size(0, 4, &HashAlgorithm::HighwayHash256), 0);
|
||||
// SSE/compression streams advertise SIZE_PRESERVE_LAYER (-1); the remote
|
||||
// put_file_stream receiver relies on a non-positive size to parse the auth
|
||||
// trailer from the stream tail, so the sentinel must survive untouched.
|
||||
assert_eq!(
|
||||
bitrot_create_file_size(rustfs_rio::HashReader::SIZE_PRESERVE_LAYER, 4, &HashAlgorithm::HighwayHash256)
|
||||
.expect("unknown length sentinel"),
|
||||
bitrot_create_file_size(rustfs_rio::HashReader::SIZE_PRESERVE_LAYER, 4, &HashAlgorithm::HighwayHash256),
|
||||
rustfs_rio::HashReader::SIZE_PRESERVE_LAYER
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn bitrot_create_file_size_rejects_invalid_geometry_and_overflow() {
|
||||
assert!(bitrot_create_file_size(1, 0, &HashAlgorithm::HighwayHash256S).is_err());
|
||||
assert!(bitrot_create_file_size(i64::MAX, 1, &HashAlgorithm::HighwayHash256S).is_err());
|
||||
assert_eq!(
|
||||
bitrot_create_file_size(-7, 0, &HashAlgorithm::HighwayHash256S).expect("negative sentinel"),
|
||||
-7
|
||||
);
|
||||
assert_eq!(bitrot_create_file_size(0, 0, &HashAlgorithm::HighwayHash256S).expect("empty body"), 0);
|
||||
}
|
||||
|
||||
struct TestChunkReader {
|
||||
chunks: VecDeque<Bytes>,
|
||||
}
|
||||
@@ -1792,28 +1762,6 @@ mod tests {
|
||||
assert!(result.unwrap().is_none());
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn bounded_inline_bitrot_writer_counts_each_frame_header() {
|
||||
let mut writer = create_bitrot_writer(true, None, "bucket", "object", 6, 4, HashAlgorithm::HighwayHash256S)
|
||||
.await
|
||||
.expect("bounded inline writer");
|
||||
writer.write(b"abcd").await.expect("first full frame");
|
||||
writer.write(b"ef").await.expect("final partial frame");
|
||||
let raw = writer.into_inline_data().expect("export framed bytes");
|
||||
assert_eq!(raw.len(), 6 + 2 * 32);
|
||||
let mut exact = create_bitrot_writer(true, None, "bucket", "object", 4, 4, HashAlgorithm::HighwayHash256S)
|
||||
.await
|
||||
.expect("exact bounded writer");
|
||||
exact.write(b"abcd").await.expect("fill admitted frame");
|
||||
assert!(exact.write(b"e").await.is_err());
|
||||
assert_eq!(exact.into_inline_data().expect("intact first frame").len(), 4 + 32);
|
||||
assert!(
|
||||
create_bitrot_writer(true, None, "bucket", "object", 1, 0, HashAlgorithm::HighwayHash256S)
|
||||
.await
|
||||
.is_err()
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn test_create_bitrot_writer_inline() {
|
||||
use rustfs_utils::HashAlgorithm;
|
||||
|
||||
@@ -1280,10 +1280,6 @@ pub struct ObjectOptions {
|
||||
/// True when an upper layer already holds the object read lock before
|
||||
/// forwarding a no_lock read to the set layer.
|
||||
pub metadata_cache_safe: bool,
|
||||
/// Read-quorum proof captured from the live distributed namespace guard.
|
||||
/// This transient value is never serialized or persisted.
|
||||
#[doc(hidden)]
|
||||
pub metadata_cache_fence_proof: Option<Arc<rustfs_lock::CacheFenceProof>>,
|
||||
|
||||
pub versioned: bool,
|
||||
pub version_suspended: bool,
|
||||
@@ -1392,10 +1388,6 @@ pub struct ObjectOptions {
|
||||
#[doc(hidden)]
|
||||
pub multipart_completion_authorization:
|
||||
Option<Arc<dyn crate::storage_api_contracts::multipart::MultipartCompletionAuthorization<Error = Error>>>,
|
||||
/// In-process authorization against the selected upload, while Abort holds its write lock.
|
||||
#[doc(hidden)]
|
||||
pub multipart_abort_authorization:
|
||||
Option<Arc<dyn crate::storage_api_contracts::multipart::MultipartAbortAuthorization<Error = Error>>>,
|
||||
pub eval_metadata: Option<HashMap<String, String>>,
|
||||
/// Internal compare-and-set condition for replication workers publishing
|
||||
/// terminal status after remote I/O. Storage validates it while holding
|
||||
@@ -1509,7 +1501,6 @@ impl std::fmt::Debug for ObjectOptions {
|
||||
.field("expected_bucket_incarnation_id", &self.expected_bucket_incarnation_id)
|
||||
.field("no_lock", &self.no_lock)
|
||||
.field("metadata_cache_safe", &self.metadata_cache_safe)
|
||||
.field("metadata_cache_fence_proof", &self.metadata_cache_fence_proof.is_some())
|
||||
.field("versioned", &self.versioned)
|
||||
.field("version_suspended", &self.version_suspended)
|
||||
.field("incl_free_versions", &self.incl_free_versions)
|
||||
@@ -1589,7 +1580,6 @@ impl ObjectOptions {
|
||||
let mut opts = self.clone();
|
||||
opts.no_lock = true;
|
||||
opts.metadata_cache_safe = false;
|
||||
opts.metadata_cache_fence_proof = None;
|
||||
opts.include_part_checksums = true;
|
||||
opts
|
||||
}
|
||||
@@ -1660,11 +1650,6 @@ impl ObjectOptions {
|
||||
}
|
||||
}
|
||||
|
||||
pub(crate) fn add_metadata_cache_read_guard(&mut self, guard: &rustfs_lock::NamespaceLockGuard) {
|
||||
self.add_namespace_lock_guard(guard);
|
||||
self.metadata_cache_fence_proof = guard.cache_fence_proof().cloned().map(Arc::new);
|
||||
}
|
||||
|
||||
pub fn add_bucket_lifecycle_lock_guard(&mut self, guard: &rustfs_lock::NamespaceLockGuard) {
|
||||
let fence = self.bucket_lifecycle_lock_fence.get_or_insert_with(NamespaceLockFence::new);
|
||||
if let Some(signal) = guard.lock_lost_signal() {
|
||||
|
||||
@@ -242,7 +242,6 @@ pub(in crate::set_disk) enum MetadataCacheLookup {
|
||||
Hit(Arc<GetObjectMetadataCacheEntry>),
|
||||
Miss,
|
||||
RejectedInsufficientQuorum,
|
||||
RejectedFenceProof,
|
||||
}
|
||||
|
||||
pub(in crate::set_disk) fn is_confirmed_missing_part_error(err: Option<&str>) -> bool {
|
||||
@@ -9904,19 +9903,6 @@ mod tests {
|
||||
("RUSTFS_GET_METADATA_EARLY_STOP_BOUNDED_FANOUT", Some("true")),
|
||||
],
|
||||
async {
|
||||
let fanout_order = bounded_metadata_fanout_order(bucket, &object, DISKS, 2);
|
||||
let distribution = FileInfo::new(&metadata_distribution_key(bucket, &object), 2, 2)
|
||||
.erasure
|
||||
.distribution;
|
||||
let initial_parity_disk = *fanout_order
|
||||
.iter()
|
||||
.take(3)
|
||||
.find(|disk_index| distribution[**disk_index] > 2)
|
||||
.expect("initial fanout should include one parity shard");
|
||||
// Hold the initial parity response so the single-pending-read
|
||||
// hedge is exercised independently of task completion order.
|
||||
let initial_barrier =
|
||||
rename_fanout_barrier::arm(&object, initial_parity_disk, rename_fanout_barrier::PHASE_READ_VERSION);
|
||||
let spare_disk = bounded_spare_disk_index(bucket, &object, 2, 2);
|
||||
let barrier = rename_fanout_barrier::arm(&object, spare_disk, rename_fanout_barrier::PHASE_READ_VERSION);
|
||||
let tracker = rename_fanout_barrier::observe_tasks(&object);
|
||||
@@ -9939,13 +9925,9 @@ mod tests {
|
||||
.await
|
||||
});
|
||||
|
||||
tokio::time::timeout(BARRIER_PAUSE_GUARD, initial_barrier.wait_until_paused())
|
||||
.await
|
||||
.expect("initial parity response should pause before quorum");
|
||||
tokio::time::timeout(BARRIER_PAUSE_GUARD, barrier.wait_until_paused())
|
||||
.await
|
||||
.expect("bounded fanout should hedge and pause the spare disk");
|
||||
initial_barrier.release();
|
||||
let (parts_metadata, errs, diagnostics) = tokio::time::timeout(BARRIER_PAUSE_GUARD, &mut read)
|
||||
.await
|
||||
.expect("verified inline quorum should return without the paused spare")
|
||||
@@ -10239,15 +10221,6 @@ mod tests {
|
||||
data_read_early_stop_inline_body_miss_reason(bucket, object, &candidate, &corrupt, &disks).await,
|
||||
Some(GET_METADATA_EARLY_STOP_REASON_DATA_READ_INLINE_BODY_VERIFY)
|
||||
);
|
||||
|
||||
let mut trailing = parts_metadata.clone();
|
||||
let mut inline_data = trailing[data_disk].data.as_ref().expect("fixture inline bytes").to_vec();
|
||||
inline_data.push(0x7b);
|
||||
trailing[data_disk].data = Some(Bytes::from(inline_data));
|
||||
assert_eq!(
|
||||
data_read_early_stop_inline_body_miss_reason(bucket, object, &candidate, &trailing, &disks).await,
|
||||
Some(GET_METADATA_EARLY_STOP_REASON_DATA_READ_INLINE_BODY_VERIFY)
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
|
||||
@@ -36,12 +36,12 @@ use crate::disk::{
|
||||
self, BATCH_READ_VERSION_MAX_ITEMS, BatchReadVersionItem, BatchReadVersionReq, BatchReadVersionResp, Disk, DiskAPI,
|
||||
};
|
||||
use crate::set_disk::{
|
||||
DiskError, DiskStore, FileInfo, ReadOptions, SetDisks, codec_streaming_rollout_applies,
|
||||
collect_inline_data_shard_fileinfos_from_observations, file_info_is_valid_for_metadata, get_metadata_slowtail_fault_request,
|
||||
inline_body_buffering_miss_reason, is_get_metadata_data_read_early_stop_enabled,
|
||||
is_get_metadata_early_stop_bounded_fanout_enabled, is_get_metadata_early_stop_enabled,
|
||||
is_get_metadata_non_inline_data_read_early_stop_enabled, is_version_early_stop_enabled, object_fits_single_block,
|
||||
try_decode_inline_data_shards,
|
||||
DiskError, DiskStore, FileInfo, HashAlgorithm, ReadOptions, SetDisks, can_try_inline_data_shards_direct,
|
||||
codec_streaming_rollout_applies, collect_inline_data_shard_fileinfos_from_observations, file_info_is_valid_for_metadata,
|
||||
get_metadata_slowtail_fault_request, inline_erasure_shard_file_offset, inline_erasure_shard_size,
|
||||
is_get_metadata_data_read_early_stop_enabled, is_get_metadata_early_stop_bounded_fanout_enabled,
|
||||
is_get_metadata_early_stop_enabled, is_get_metadata_non_inline_data_read_early_stop_enabled, is_version_early_stop_enabled,
|
||||
object_fits_single_block,
|
||||
};
|
||||
use futures::future::join_all;
|
||||
use metrics::counter;
|
||||
@@ -656,8 +656,68 @@ fn verify_inline_body_from_observations<'a>(
|
||||
disks.get(index).is_some_and(Option::is_some)
|
||||
})?;
|
||||
|
||||
try_decode_inline_data_shards(_object, candidate, &data_files, false)
|
||||
.ok_or(GET_METADATA_EARLY_STOP_REASON_DATA_READ_INLINE_BODY_VERIFY)
|
||||
let Some(part) = candidate.parts.first() else {
|
||||
return Err(GET_METADATA_EARLY_STOP_REASON_DATA_READ_INLINE_PART_SHAPE);
|
||||
};
|
||||
let Ok(object_size) = usize::try_from(candidate.size) else {
|
||||
return Err(GET_METADATA_EARLY_STOP_REASON_DATA_READ_INLINE_SIZE);
|
||||
};
|
||||
let checksum_info = candidate.erasure.get_checksum_info(part.number);
|
||||
let checksum_algo = if candidate.uses_legacy_checksum && checksum_info.algorithm == HashAlgorithm::HighwayHash256S {
|
||||
HashAlgorithm::HighwayHash256SLegacy
|
||||
} else {
|
||||
checksum_info.algorithm
|
||||
};
|
||||
let read_length = inline_erasure_shard_file_offset(
|
||||
0,
|
||||
object_size,
|
||||
object_size,
|
||||
candidate.erasure.block_size,
|
||||
data_shards,
|
||||
candidate.uses_legacy_checksum,
|
||||
);
|
||||
let shard_size = inline_erasure_shard_size(candidate.erasure.block_size, data_shards, candidate.uses_legacy_checksum);
|
||||
if object_size == 0 || read_length == 0 || read_length > shard_size || data_files.len() < data_shards {
|
||||
return Err(GET_METADATA_EARLY_STOP_REASON_DATA_READ_INLINE_BODY_VERIFY);
|
||||
}
|
||||
let shards_needed = object_size.div_ceil(read_length);
|
||||
if shards_needed > data_shards {
|
||||
return Err(GET_METADATA_EARLY_STOP_REASON_DATA_READ_INLINE_BODY_VERIFY);
|
||||
}
|
||||
let Some(encoded_capacity) = read_length.checked_mul(shards_needed) else {
|
||||
return Err(GET_METADATA_EARLY_STOP_REASON_DATA_READ_INLINE_BODY_VERIFY);
|
||||
};
|
||||
#[cfg(test)]
|
||||
{
|
||||
for file in &data_files {
|
||||
if file.data.is_some() {
|
||||
crate::set_disk::disk_call_counters::record_shard_open(_object, true);
|
||||
}
|
||||
}
|
||||
crate::set_disk::disk_call_counters::record(_object, crate::set_disk::disk_call_counters::KIND_BODY_DECODE, 0);
|
||||
}
|
||||
|
||||
let hash_size = checksum_algo.size();
|
||||
let need = hash_size + read_length;
|
||||
let mut body = Vec::with_capacity(encoded_capacity);
|
||||
for file in data_files.iter().take(shards_needed) {
|
||||
let Some(raw) = file.data.as_deref() else {
|
||||
return Err(GET_METADATA_EARLY_STOP_REASON_DATA_READ_INLINE_BODY_VERIFY);
|
||||
};
|
||||
let Some(block) = raw.get(..need) else {
|
||||
return Err(GET_METADATA_EARLY_STOP_REASON_DATA_READ_INLINE_BODY_VERIFY);
|
||||
};
|
||||
let (hash, data) = block.split_at(hash_size);
|
||||
if hash_size > 0 && checksum_algo.hash_encode(data).as_ref() != hash {
|
||||
return Err(GET_METADATA_EARLY_STOP_REASON_DATA_READ_INLINE_BODY_VERIFY);
|
||||
}
|
||||
body.extend_from_slice(data);
|
||||
if body.len() >= object_size {
|
||||
body.truncate(object_size);
|
||||
return Ok(bytes::Bytes::from(body));
|
||||
}
|
||||
}
|
||||
Err(GET_METADATA_EARLY_STOP_REASON_DATA_READ_INLINE_BODY_VERIFY)
|
||||
}
|
||||
|
||||
fn data_read_early_stop_inline_candidate_miss_reason(candidate: &FileInfo) -> Option<&'static str> {
|
||||
@@ -695,7 +755,10 @@ fn data_read_early_stop_inline_candidate_miss_reason(candidate: &FileInfo) -> Op
|
||||
if candidate.parts.first().is_none_or(|part| part.size != object_size) {
|
||||
return Some(GET_METADATA_EARLY_STOP_REASON_DATA_READ_INLINE_PART_SHAPE);
|
||||
}
|
||||
inline_body_buffering_miss_reason(candidate)
|
||||
if !can_try_inline_data_shards_direct(object_size, candidate.erasure.block_size) {
|
||||
return Some(GET_METADATA_EARLY_STOP_REASON_DATA_READ_INLINE_SIZE);
|
||||
}
|
||||
None
|
||||
}
|
||||
|
||||
pub(in crate::set_disk) fn non_inline_data_read_candidate_is_safe(candidate: &FileInfo) -> bool {
|
||||
|
||||
+163
-339
@@ -51,13 +51,12 @@ use crate::bucket::replication::{ReplicateDecision, ReplicationObjectBridge, Rep
|
||||
use crate::cluster::rpc::heal_bucket_local_on_disks;
|
||||
use crate::data_usage::record_compression_total_memory;
|
||||
use crate::diagnostics::get::{
|
||||
GET_CODEC_STREAMING_OBJECT_CLASS_PLAIN_SINGLE_PART, GET_METADATA_EARLY_STOP_REASON_DATA_READ_INLINE_BODY_VERIFY,
|
||||
GET_METADATA_EARLY_STOP_REASON_DATA_READ_INLINE_GEOMETRY, GET_METADATA_EARLY_STOP_REASON_DATA_READ_INLINE_IDENTITY_MISMATCH,
|
||||
GET_CODEC_STREAMING_OBJECT_CLASS_PLAIN_SINGLE_PART, GET_METADATA_EARLY_STOP_REASON_DATA_READ_INLINE_GEOMETRY,
|
||||
GET_METADATA_EARLY_STOP_REASON_DATA_READ_INLINE_IDENTITY_MISMATCH,
|
||||
GET_METADATA_EARLY_STOP_REASON_DATA_READ_INLINE_MISSING_PAYLOAD,
|
||||
GET_METADATA_EARLY_STOP_REASON_DATA_READ_INLINE_MISSING_SHARD, GET_METADATA_EARLY_STOP_REASON_DATA_READ_INLINE_SIZE,
|
||||
GET_OBJECT_PATH_BODY_CACHE, GET_OBJECT_PATH_CODEC_STREAMING, GET_OBJECT_PATH_CODEC_STREAMING_LEGACY_ENGINE,
|
||||
GET_OBJECT_PATH_CODEC_STREAMING_RUSTFS_ENGINE, GET_OBJECT_PATH_DIRECT_MEMORY, GET_OBJECT_PATH_EMPTY,
|
||||
GET_OBJECT_PATH_INLINE_DIRECT, GET_OBJECT_PATH_INTERNAL_META, GET_OBJECT_PATH_LEGACY_DUPLEX,
|
||||
GET_METADATA_EARLY_STOP_REASON_DATA_READ_INLINE_MISSING_SHARD, GET_OBJECT_PATH_BODY_CACHE, GET_OBJECT_PATH_CODEC_STREAMING,
|
||||
GET_OBJECT_PATH_CODEC_STREAMING_LEGACY_ENGINE, GET_OBJECT_PATH_CODEC_STREAMING_RUSTFS_ENGINE, GET_OBJECT_PATH_DIRECT_MEMORY,
|
||||
GET_OBJECT_PATH_EMPTY, GET_OBJECT_PATH_INLINE_DIRECT, GET_OBJECT_PATH_INTERNAL_META, GET_OBJECT_PATH_LEGACY_DUPLEX,
|
||||
GET_OBJECT_PATH_REMOTE_TRANSITION, GET_OBJECT_PATH_SET_DISK, GET_STAGE_DECODE, GET_STAGE_EMIT, GET_STAGE_INLINE_PREPARE,
|
||||
GET_STAGE_LOCK_ACQUIRE, GET_STAGE_METADATA, GET_STAGE_OBJECT_INFO, GET_STAGE_PATH_DECISION, GET_STAGE_READER_SETUP,
|
||||
classify_storage_error, get_stage_timer_if_enabled, record_get_object_pipeline_failure,
|
||||
@@ -728,11 +727,6 @@ const DISK_HEALTH_CACHE_TTL: Duration = Duration::from_millis(750);
|
||||
const GET_OBJECT_METADATA_CACHE_TTL: Duration = Duration::from_secs(2); // Increased from 250ms to 2s
|
||||
const DEFAULT_GET_OBJECT_METADATA_CACHE_MAX_ENTRIES: usize = 4096; // Increased from 1024 to 4096
|
||||
const ENV_RUSTFS_GET_OBJECT_METADATA_CACHE_MAX_ENTRIES: &str = "RUSTFS_GET_OBJECT_METADATA_CACHE_MAX_ENTRIES";
|
||||
#[cfg(any(test, feature = "e2e-test-hooks"))]
|
||||
const ENV_RUSTFS_EXPERIMENTAL_GET_OBJECT_METADATA_CACHE_DISTRIBUTED_PROOF_ENABLE: &str =
|
||||
"RUSTFS_EXPERIMENTAL_GET_OBJECT_METADATA_CACHE_DISTRIBUTED_PROOF_ENABLE";
|
||||
#[cfg(any(test, feature = "e2e-test-hooks"))]
|
||||
const DEFAULT_RUSTFS_EXPERIMENTAL_GET_OBJECT_METADATA_CACHE_DISTRIBUTED_PROOF_ENABLE: bool = false;
|
||||
const GET_OBJECT_METADATA_CACHE_FENCE_SHARDS: u16 = 4096;
|
||||
|
||||
// --- Codec Streaming Configuration ---
|
||||
@@ -793,8 +787,7 @@ const DEFAULT_RUSTFS_GET_SMALL_OBJECT_DIRECT_MEMORY_THRESHOLD: usize = 128 * 102
|
||||
// Bound the in-memory inline decoder independently from the configurable
|
||||
// storage-class admission policy. This protects its Vec allocation while
|
||||
// allowing explicitly configured EC layouts to inline objects above 128 KiB.
|
||||
const INLINE_FAST_PATH_MAX_OBJECT_SIZE: usize = 2 * 1024 * 1024;
|
||||
const INLINE_FAST_PATH_MAX_BLOCKS: usize = 32;
|
||||
const INLINE_FAST_PATH_MAX_OBJECT_SIZE: i64 = 1024 * 1024;
|
||||
|
||||
// --- Metadata Early-Stop Configuration ---
|
||||
|
||||
@@ -1522,7 +1515,6 @@ mod prepared_get_object_metadata_tests {
|
||||
online_disks: vec![None],
|
||||
read_quorum: 0,
|
||||
verified_inline_body: None,
|
||||
distributed_fence_proof: None,
|
||||
});
|
||||
let snapshot = GetObjectFileInfo::shared(Arc::clone(&cached));
|
||||
|
||||
@@ -3451,20 +3443,6 @@ fn get_object_metadata_cache_max_entries() -> usize {
|
||||
}
|
||||
}
|
||||
|
||||
pub(crate) fn is_get_object_metadata_cache_distributed_proof_enabled() -> bool {
|
||||
#[cfg(any(test, feature = "e2e-test-hooks"))]
|
||||
{
|
||||
rustfs_utils::get_env_bool(
|
||||
ENV_RUSTFS_EXPERIMENTAL_GET_OBJECT_METADATA_CACHE_DISTRIBUTED_PROOF_ENABLE,
|
||||
DEFAULT_RUSTFS_EXPERIMENTAL_GET_OBJECT_METADATA_CACHE_DISTRIBUTED_PROOF_ENABLE,
|
||||
)
|
||||
}
|
||||
#[cfg(not(any(test, feature = "e2e-test-hooks")))]
|
||||
{
|
||||
false
|
||||
}
|
||||
}
|
||||
|
||||
fn is_get_small_object_direct_memory_enabled() -> bool {
|
||||
#[cfg(test)]
|
||||
{
|
||||
@@ -5038,7 +5016,6 @@ struct GetObjectMetadataCacheEntry {
|
||||
read_quorum: usize,
|
||||
capability: GetObjectMetadataCacheCapability,
|
||||
verified_inline_body: Option<Bytes>,
|
||||
distributed_fence_proof: Option<Arc<rustfs_lock::CacheFenceProof>>,
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug)]
|
||||
@@ -6072,51 +6049,6 @@ fn should_use_inline_fast_path(
|
||||
should_use_inline_fast_path_with_plan(range, object_info, fi, opts, ReadPathPlan::new(object_info, fi))
|
||||
}
|
||||
|
||||
fn inline_body_fits_buffer(size: usize, block_size: usize) -> bool {
|
||||
size > 0
|
||||
&& size <= INLINE_FAST_PATH_MAX_OBJECT_SIZE
|
||||
&& block_size != 0
|
||||
&& size.div_ceil(block_size) <= INLINE_FAST_PATH_MAX_BLOCKS
|
||||
}
|
||||
|
||||
fn should_inline_put(
|
||||
storage_class: &crate::config::storageclass::Config,
|
||||
erasure: &coding::Erasure,
|
||||
stored_size: i64,
|
||||
actual_size: i64,
|
||||
versioned: bool,
|
||||
protected: bool,
|
||||
) -> bool {
|
||||
let admission_size = if stored_size >= 0 {
|
||||
stored_size
|
||||
} else if !protected && actual_size > 0 {
|
||||
actual_size
|
||||
} else {
|
||||
return false;
|
||||
};
|
||||
let Ok(size) = usize::try_from(admission_size) else {
|
||||
return false;
|
||||
};
|
||||
if size > erasure.block_size && (stored_size < 0 || protected || !inline_body_fits_buffer(size, erasure.block_size)) {
|
||||
return false;
|
||||
}
|
||||
storage_class.should_inline(erasure.shard_file_size(admission_size), erasure.data_shards, versioned)
|
||||
}
|
||||
|
||||
fn inline_body_buffering_miss_reason(fi: &FileInfo) -> Option<&'static str> {
|
||||
let Ok(size) = usize::try_from(fi.size) else {
|
||||
return Some(GET_METADATA_EARLY_STOP_REASON_DATA_READ_INLINE_SIZE);
|
||||
};
|
||||
if !inline_body_fits_buffer(size, fi.erasure.block_size) {
|
||||
return Some(GET_METADATA_EARLY_STOP_REASON_DATA_READ_INLINE_SIZE);
|
||||
}
|
||||
// Proof-protected parts use the readers that bind their PartProofReader.
|
||||
if fi.parts.iter().any(|part| part.integrity.is_some()) {
|
||||
return Some(GET_METADATA_EARLY_STOP_REASON_DATA_READ_INLINE_BODY_VERIFY);
|
||||
}
|
||||
None
|
||||
}
|
||||
|
||||
fn should_use_inline_fast_path_with_plan(
|
||||
range: &Option<HTTPRangeSpec>,
|
||||
object_info: &ObjectInfo,
|
||||
@@ -6128,12 +6060,18 @@ fn should_use_inline_fast_path_with_plan(
|
||||
return false;
|
||||
}
|
||||
|
||||
if plan.shape().is_none() {
|
||||
let Some(shape) = plan.shape() else {
|
||||
return false;
|
||||
}
|
||||
// Apply the same materialization contract as metadata pre-verification.
|
||||
// The independent direct-memory reader retains its 128 KiB gate.
|
||||
plan.is_plain() && inline_body_buffering_miss_reason(fi).is_none() && fi.erasure.data_blocks != 0
|
||||
};
|
||||
// The persisted marker is authoritative for the storage decision, but it
|
||||
// is still untrusted metadata at this boundary. Revalidate its geometry so
|
||||
// a stale/corrupt marker cannot route an oversized payload into the
|
||||
// in-memory decoder. The independent direct-memory reader remains capped
|
||||
// at 128 KiB below.
|
||||
plan.is_plain()
|
||||
&& shape.object_size <= usize::try_from(INLINE_FAST_PATH_MAX_OBJECT_SIZE).expect("inline fast path limit fits usize")
|
||||
&& fi.erasure.data_blocks != 0
|
||||
&& fi.erasure.block_size != 0
|
||||
}
|
||||
|
||||
enum SmallWritePath {
|
||||
@@ -6222,9 +6160,27 @@ fn known_put_object_storage_size(data_size: i64) -> i64 {
|
||||
}
|
||||
}
|
||||
|
||||
/// Shard size the inline admission check evaluates for a single PUT.
|
||||
///
|
||||
/// A compressed or encrypted stream reports `SIZE_PRESERVE_LAYER` as its
|
||||
/// stored size because the transformed length is only known after the write.
|
||||
/// MinIO's `putObject` sizes such objects for inline admission by their
|
||||
/// plaintext `ActualSize`; without that fallback every transformed object,
|
||||
/// however small, lands in `part.1` files. A stream with neither size known
|
||||
/// yields a negative shard size, which `should_inline` rejects.
|
||||
fn inline_admission_shard_size(erasure: &coding::Erasure, stored_size: i64, actual_size: i64) -> i64 {
|
||||
if stored_size >= 0 {
|
||||
return erasure.shard_file_size(stored_size);
|
||||
}
|
||||
if actual_size > 0 {
|
||||
return erasure.shard_file_size(actual_size);
|
||||
}
|
||||
HashReader::SIZE_PRESERVE_LAYER
|
||||
}
|
||||
|
||||
#[allow(clippy::too_many_arguments)]
|
||||
async fn build_inline_bitrot_readers<'a>(
|
||||
files: impl Iterator<Item = &'a FileInfo> + Send,
|
||||
async fn build_inline_bitrot_readers(
|
||||
files: &[FileInfo],
|
||||
total_shards: usize,
|
||||
bucket: &str,
|
||||
object: &str,
|
||||
@@ -6234,7 +6190,7 @@ async fn build_inline_bitrot_readers<'a>(
|
||||
skip_verify_bitrot: bool,
|
||||
) -> disk::error::Result<Vec<Option<InlineBitrotReader>>> {
|
||||
let mut readers = Vec::with_capacity(total_shards);
|
||||
for file in files.take(total_shards) {
|
||||
for file in files.iter().take(total_shards) {
|
||||
let reader = if let Some(data) = &file.data {
|
||||
create_bitrot_reader_from_bytes(
|
||||
Some(data.clone()),
|
||||
@@ -6257,83 +6213,38 @@ async fn build_inline_bitrot_readers<'a>(
|
||||
Ok(readers)
|
||||
}
|
||||
|
||||
/// Resolve the bitrot checksum algorithm for one part, honoring the legacy
|
||||
/// HighwayHash flag.
|
||||
fn part_checksum_algorithm(fi: &FileInfo, part_number: usize) -> HashAlgorithm {
|
||||
let checksum_info = fi.erasure.get_checksum_info(part_number);
|
||||
if fi.uses_legacy_checksum && checksum_info.algorithm == HashAlgorithm::HighwayHash256S {
|
||||
HashAlgorithm::HighwayHash256SLegacy
|
||||
} else {
|
||||
checksum_info.algorithm
|
||||
}
|
||||
}
|
||||
|
||||
/// Decode already identity-checked data shards without reader wrappers or RS.
|
||||
/// A failed frame leaves recovery to the caller's ordinary erasure reader.
|
||||
fn try_decode_inline_data_shards(
|
||||
_object: &str,
|
||||
fi: &FileInfo,
|
||||
data_files: &[&FileInfo],
|
||||
#[allow(clippy::too_many_arguments)]
|
||||
async fn build_inline_bitrot_readers_from_refs(
|
||||
files: &[&FileInfo],
|
||||
bucket: &str,
|
||||
object: &str,
|
||||
read_length: usize,
|
||||
shard_size: usize,
|
||||
checksum_algo: &HashAlgorithm,
|
||||
skip_verify_bitrot: bool,
|
||||
) -> Option<Bytes> {
|
||||
if inline_body_buffering_miss_reason(fi).is_some() {
|
||||
return None;
|
||||
) -> disk::error::Result<Vec<Option<InlineBitrotReader>>> {
|
||||
let mut readers = Vec::with_capacity(files.len());
|
||||
for file in files {
|
||||
let reader = if let Some(data) = &file.data {
|
||||
create_bitrot_reader_from_bytes(
|
||||
Some(data.clone()),
|
||||
None,
|
||||
bucket,
|
||||
object,
|
||||
0,
|
||||
read_length,
|
||||
shard_size,
|
||||
checksum_algo.clone(),
|
||||
skip_verify_bitrot,
|
||||
false,
|
||||
)
|
||||
.await?
|
||||
} else {
|
||||
None
|
||||
};
|
||||
readers.push(reader);
|
||||
}
|
||||
let object_size = usize::try_from(fi.size).ok()?;
|
||||
let part = fi.parts.first()?;
|
||||
let data_shards = fi.erasure.data_blocks;
|
||||
if fi.parts.len() != 1 || part.size != object_size || data_files.len() < data_shards {
|
||||
return None;
|
||||
}
|
||||
let shard_size = inline_erasure_shard_size(fi.erasure.block_size, data_shards, fi.uses_legacy_checksum);
|
||||
if shard_size == 0 {
|
||||
return None;
|
||||
}
|
||||
let checksum_algo = part_checksum_algorithm(fi, part.number);
|
||||
let hash_size = checksum_algo.size();
|
||||
let stride = shard_size.checked_add(hash_size)?;
|
||||
let part_size = inline_erasure_shard_file_size(part.size, fi.erasure.block_size, data_shards, fi.uses_legacy_checksum);
|
||||
let encoded_size = part_size
|
||||
.div_ceil(shard_size)
|
||||
.checked_mul(hash_size)?
|
||||
.checked_add(part_size)?;
|
||||
// Metadata early-stop can cache this body, so require exact framing even
|
||||
// when the caller explicitly skips bitrot digest verification.
|
||||
for file in data_files.iter().take(data_shards) {
|
||||
if file.data.as_ref()?.len() != encoded_size {
|
||||
return None;
|
||||
}
|
||||
}
|
||||
#[cfg(test)]
|
||||
{
|
||||
for _ in data_files.iter().take(data_shards) {
|
||||
disk_call_counters::record_shard_open(_object, true);
|
||||
}
|
||||
disk_call_counters::record(_object, disk_call_counters::KIND_BODY_DECODE, 0);
|
||||
}
|
||||
let mut body = Vec::with_capacity(object_size);
|
||||
for (frame, offset) in (0..object_size).step_by(fi.erasure.block_size).enumerate() {
|
||||
let block_length = fi.erasure.block_size.min(object_size - offset);
|
||||
let read_length =
|
||||
inline_erasure_shard_file_size(block_length, fi.erasure.block_size, data_shards, fi.uses_legacy_checksum);
|
||||
if read_length == 0 || read_length > shard_size {
|
||||
return None;
|
||||
}
|
||||
let start = frame.checked_mul(stride)?;
|
||||
let end = start.checked_add(hash_size)?.checked_add(read_length)?;
|
||||
let mut remaining = block_length;
|
||||
for file in data_files.iter().take(block_length.div_ceil(read_length)) {
|
||||
let block = file.data.as_deref()?.get(start..end)?;
|
||||
let data = coding::verified_bitrot_frame(&checksum_algo, skip_verify_bitrot, block).ok()?;
|
||||
let take = remaining.min(data.len());
|
||||
body.extend_from_slice(&data[..take]);
|
||||
remaining -= take;
|
||||
}
|
||||
if remaining != 0 {
|
||||
return None;
|
||||
}
|
||||
}
|
||||
Some(Bytes::from(body))
|
||||
Ok(readers)
|
||||
}
|
||||
|
||||
async fn try_read_inline_data_shards_direct(
|
||||
@@ -6374,6 +6285,10 @@ async fn try_read_inline_data_shards_direct(
|
||||
None
|
||||
}
|
||||
|
||||
fn can_try_inline_data_shards_direct(object_size: usize, block_size: usize) -> bool {
|
||||
object_size > 0 && object_size <= block_size
|
||||
}
|
||||
|
||||
fn inline_erasure_shard_size(block_size: usize, data_shards: usize, uses_legacy: bool) -> usize {
|
||||
if block_size == 0 || data_shards == 0 {
|
||||
return 0;
|
||||
@@ -6402,14 +6317,31 @@ fn inline_erasure_shard_file_size(total_length: usize, block_size: usize, data_s
|
||||
num_shards * shard_size + last_shard_size
|
||||
}
|
||||
|
||||
type InlineDataShardFiles<'a> = smallvec::SmallVec<[&'a FileInfo; shard_source::INLINE_SHARD_SLOTS]>;
|
||||
fn inline_erasure_shard_file_offset(
|
||||
start_offset: usize,
|
||||
length: usize,
|
||||
total_length: usize,
|
||||
block_size: usize,
|
||||
data_shards: usize,
|
||||
uses_legacy: bool,
|
||||
) -> usize {
|
||||
if block_size == 0 || data_shards == 0 {
|
||||
return 0;
|
||||
}
|
||||
|
||||
let shard_size = inline_erasure_shard_size(block_size, data_shards, uses_legacy);
|
||||
let shard_file_size = inline_erasure_shard_file_size(total_length, block_size, data_shards, uses_legacy);
|
||||
let end_shard = (start_offset + length) / block_size;
|
||||
let till_offset = end_shard * shard_size + shard_size;
|
||||
till_offset.min(shard_file_size)
|
||||
}
|
||||
|
||||
fn collect_inline_data_shard_fileinfos_by_index<'a>(
|
||||
parts_metadata: &'a [FileInfo],
|
||||
fi: &FileInfo,
|
||||
data_shards: usize,
|
||||
disk_is_online: impl FnMut(usize) -> bool,
|
||||
) -> Option<InlineDataShardFiles<'a>> {
|
||||
) -> Option<Vec<&'a FileInfo>> {
|
||||
collect_inline_data_shard_fileinfos_by_index_or_reason(parts_metadata, fi, data_shards, disk_is_online).ok()
|
||||
}
|
||||
|
||||
@@ -6418,7 +6350,7 @@ fn collect_inline_data_shard_fileinfos_by_index_or_reason<'a>(
|
||||
fi: &FileInfo,
|
||||
data_shards: usize,
|
||||
disk_is_online: impl FnMut(usize) -> bool,
|
||||
) -> std::result::Result<InlineDataShardFiles<'a>, &'static str> {
|
||||
) -> std::result::Result<Vec<&'a FileInfo>, &'static str> {
|
||||
collect_inline_data_shard_fileinfos_from_observations(
|
||||
parts_metadata
|
||||
.iter()
|
||||
@@ -6435,13 +6367,9 @@ fn collect_inline_data_shard_fileinfos_from_observations<'a>(
|
||||
fi: &FileInfo,
|
||||
data_shards: usize,
|
||||
mut disk_is_online: impl FnMut(usize) -> bool,
|
||||
) -> std::result::Result<InlineDataShardFiles<'a>, &'static str> {
|
||||
) -> std::result::Result<Vec<&'a FileInfo>, &'static str> {
|
||||
let distribution = &fi.erasure.distribution;
|
||||
if data_shards == 0 || data_shards != fi.erasure.data_blocks || data_shards > distribution.len() {
|
||||
return Err(GET_METADATA_EARLY_STOP_REASON_DATA_READ_INLINE_GEOMETRY);
|
||||
}
|
||||
let mut data_files: smallvec::SmallVec<[Option<&FileInfo>; shard_source::INLINE_SHARD_SLOTS]> =
|
||||
smallvec::smallvec![None; data_shards];
|
||||
let mut data_files = vec![None; data_shards];
|
||||
|
||||
for (disk_index, file_info) in observations {
|
||||
if !disk_is_online(disk_index) {
|
||||
@@ -6474,7 +6402,7 @@ fn collect_inline_data_shard_fileinfos_from_observations<'a>(
|
||||
|
||||
data_files
|
||||
.into_iter()
|
||||
.collect::<Option<InlineDataShardFiles<'_>>>()
|
||||
.collect::<Option<Vec<_>>>()
|
||||
.ok_or(GET_METADATA_EARLY_STOP_REASON_DATA_READ_INLINE_MISSING_SHARD)
|
||||
}
|
||||
|
||||
@@ -7440,20 +7368,23 @@ fn join_errs(errs: &[Option<DiskError>]) -> String {
|
||||
errs.join(", ")
|
||||
}
|
||||
|
||||
fn verify_inline_part_bitrot(meta: &FileInfo) -> disk::error::Result<()> {
|
||||
async fn verify_inline_part_bitrot(meta: &FileInfo) -> disk::error::Result<()> {
|
||||
meta.validate_for_metadata_read()?;
|
||||
let [part] = meta.parts.as_slice() else {
|
||||
return Err(DiskError::FileCorrupt);
|
||||
};
|
||||
let data = meta.data.as_deref().ok_or(DiskError::FileCorrupt)?;
|
||||
let algo = part_checksum_algorithm(meta, part.number);
|
||||
if !matches!(algo, HashAlgorithm::HighwayHash256S | HashAlgorithm::HighwayHash256SLegacy) {
|
||||
return Err(DiskError::BitrotHashAlgoInvalid);
|
||||
}
|
||||
let checksum = meta.erasure.get_checksum_info(part.number);
|
||||
let algo = match checksum.algorithm {
|
||||
HashAlgorithm::HighwayHash256S if meta.uses_legacy_checksum => HashAlgorithm::HighwayHash256SLegacy,
|
||||
algo @ (HashAlgorithm::HighwayHash256S | HashAlgorithm::HighwayHash256SLegacy) => algo,
|
||||
_ => return Err(DiskError::BitrotHashAlgoInvalid),
|
||||
};
|
||||
let shard_size = inline_erasure_shard_size(meta.erasure.block_size, meta.erasure.data_blocks, meta.uses_legacy_checksum);
|
||||
let part_size =
|
||||
inline_erasure_shard_file_size(part.size, meta.erasure.block_size, meta.erasure.data_blocks, meta.uses_legacy_checksum);
|
||||
// Validate the complete framing before visiting borrowed payloads.
|
||||
// Check framing arithmetic and the physical buffer length before the shared
|
||||
// verifier sizes its scratch buffer from the metadata.
|
||||
let encoded_size = part_size
|
||||
.div_ceil(shard_size)
|
||||
.checked_mul(algo.size())
|
||||
@@ -7462,11 +7393,9 @@ fn verify_inline_part_bitrot(meta: &FileInfo) -> disk::error::Result<()> {
|
||||
if encoded_size != data.len() {
|
||||
return Err(DiskError::FileCorrupt);
|
||||
}
|
||||
let stride = shard_size.checked_add(algo.size()).ok_or(DiskError::FileCorrupt)?;
|
||||
for frame in data.chunks(stride) {
|
||||
coding::verified_bitrot_frame(&algo, false, frame).map_err(|_| DiskError::FileCorrupt)?;
|
||||
}
|
||||
Ok(())
|
||||
coding::bitrot_verify(Cursor::new(data), encoded_size, part_size, algo, shard_size)
|
||||
.await
|
||||
.map_err(|_| DiskError::FileCorrupt)
|
||||
}
|
||||
|
||||
fn validate_deep_scan_results(results: &[usize], expected_parts: usize) -> disk::error::Result<()> {
|
||||
@@ -7684,7 +7613,7 @@ async fn disks_with_all_parts(
|
||||
// before deciding whether reconstruction (and its bitrot readers) is needed.
|
||||
if (meta.data.is_some() || meta.size == 0) && !meta.parts.is_empty() {
|
||||
let part_status = if scan_mode == HealScanMode::Deep && meta.data.is_some() {
|
||||
match verify_inline_part_bitrot(meta) {
|
||||
match verify_inline_part_bitrot(meta).await {
|
||||
Ok(()) => CHECK_PART_SUCCESS,
|
||||
Err(DiskError::FileCorrupt) => CHECK_PART_FILE_CORRUPT,
|
||||
Err(err) => return Err(err),
|
||||
@@ -15062,19 +14991,21 @@ mod tests {
|
||||
let (_, files, _, _) = inline_bitrot_files_for_payload_with_mode(&payload, legacy).await;
|
||||
for mut meta in files {
|
||||
assert!(meta.erasure.checksums.is_empty(), "legacy metadata may omit per-part checksum entries");
|
||||
verify_inline_part_bitrot(&meta).expect("healthy data and parity shards should verify");
|
||||
verify_inline_part_bitrot(&meta)
|
||||
.await
|
||||
.expect("healthy data and parity shards should verify");
|
||||
let original = meta.data.clone().expect("fixture should retain inline bytes");
|
||||
for offset in [0, 32, original.len() - 1] {
|
||||
let mut damaged = original.to_vec();
|
||||
damaged[offset] ^= 1;
|
||||
meta.data = Some(Bytes::from(damaged));
|
||||
assert_eq!(verify_inline_part_bitrot(&meta), Err(DiskError::FileCorrupt));
|
||||
assert_eq!(verify_inline_part_bitrot(&meta).await, Err(DiskError::FileCorrupt));
|
||||
}
|
||||
let mut trailing = original.to_vec();
|
||||
trailing.push(0x7b);
|
||||
for damaged in [Vec::new(), original[..original.len() - 1].to_vec(), trailing] {
|
||||
meta.data = Some(Bytes::from(damaged));
|
||||
assert_eq!(verify_inline_part_bitrot(&meta), Err(DiskError::FileCorrupt));
|
||||
assert_eq!(verify_inline_part_bitrot(&meta).await, Err(DiskError::FileCorrupt));
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -15088,17 +15019,19 @@ mod tests {
|
||||
empty.parts[0].size = 0;
|
||||
empty.parts[0].actual_size = 0;
|
||||
empty.data = Some(Bytes::new());
|
||||
verify_inline_part_bitrot(&empty).expect("empty inline objects have no bitrot frames");
|
||||
verify_inline_part_bitrot(&empty)
|
||||
.await
|
||||
.expect("empty inline objects have no bitrot frames");
|
||||
empty.data = Some(Bytes::from_static(b"unexpected"));
|
||||
assert_eq!(verify_inline_part_bitrot(&empty), Err(DiskError::FileCorrupt));
|
||||
assert_eq!(verify_inline_part_bitrot(&empty).await, Err(DiskError::FileCorrupt));
|
||||
|
||||
let mut invalid = files[0].clone();
|
||||
invalid.erasure.block_size = 0;
|
||||
assert_eq!(verify_inline_part_bitrot(&invalid), Err(DiskError::FileCorrupt));
|
||||
assert_eq!(verify_inline_part_bitrot(&invalid).await, Err(DiskError::FileCorrupt));
|
||||
invalid = files[0].clone();
|
||||
invalid.parts.push(invalid.parts[0].clone());
|
||||
invalid.parts[1].number = 2;
|
||||
assert_eq!(verify_inline_part_bitrot(&invalid), Err(DiskError::FileCorrupt));
|
||||
assert_eq!(verify_inline_part_bitrot(&invalid).await, Err(DiskError::FileCorrupt));
|
||||
|
||||
let mut oversized = files[0].clone();
|
||||
oversized.erasure.block_size = 2;
|
||||
@@ -15107,7 +15040,7 @@ mod tests {
|
||||
oversized
|
||||
.validate_for_metadata_read()
|
||||
.expect("logical shard length should fit metadata bounds");
|
||||
assert_eq!(verify_inline_part_bitrot(&oversized), Err(DiskError::FileCorrupt));
|
||||
assert_eq!(verify_inline_part_bitrot(&oversized).await, Err(DiskError::FileCorrupt));
|
||||
|
||||
oversized.erasure.block_size = 1 << 40;
|
||||
oversized.size = 1 << 40;
|
||||
@@ -15115,7 +15048,7 @@ mod tests {
|
||||
oversized
|
||||
.validate_for_metadata_read()
|
||||
.expect("large metadata geometry should remain representable");
|
||||
assert_eq!(verify_inline_part_bitrot(&oversized), Err(DiskError::FileCorrupt));
|
||||
assert_eq!(verify_inline_part_bitrot(&oversized).await, Err(DiskError::FileCorrupt));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
@@ -15138,11 +15071,13 @@ mod tests {
|
||||
assert_eq!(meta.uses_legacy_checksum, legacy);
|
||||
assert!(meta.inline_data());
|
||||
assert!(meta.data.is_some(), "the production decoder should extract the historical inline value");
|
||||
verify_inline_part_bitrot(&meta).expect("historical inline shard should pass deep verification");
|
||||
verify_inline_part_bitrot(&meta)
|
||||
.await
|
||||
.expect("historical inline shard should pass deep verification");
|
||||
let mut damaged = meta.data.as_ref().expect("fixture should be inline").to_vec();
|
||||
*damaged.last_mut().expect("fixture shard should not be empty") ^= 1;
|
||||
meta.data = Some(Bytes::from(damaged));
|
||||
assert_eq!(verify_inline_part_bitrot(&meta), Err(DiskError::FileCorrupt));
|
||||
assert_eq!(verify_inline_part_bitrot(&meta).await, Err(DiskError::FileCorrupt));
|
||||
}
|
||||
}
|
||||
|
||||
@@ -15186,48 +15121,6 @@ mod tests {
|
||||
fi
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn inline_put_admission_bounds_only_the_multi_block_extension() {
|
||||
let block = 1024 * 1024;
|
||||
let config = crate::config::storageclass::Config::default();
|
||||
let erasure = coding::Erasure::new(12, 4, block);
|
||||
let admits = |stored, actual, protected| should_inline_put(&config, &erasure, stored, actual, false, protected);
|
||||
assert!(admits(1280 * 1024, 1280 * 1024, false));
|
||||
assert!(!admits(1280 * 1024, 1280 * 1024, true));
|
||||
assert!(!admits(-1, 1280 * 1024, false));
|
||||
assert!(admits(-1, block as i64, false));
|
||||
assert!(!admits(-1, block as i64, true));
|
||||
assert!(!admits(2 * 1024 * 1024 + 1, 0, false));
|
||||
assert!(admits(0, 0, false));
|
||||
assert!(admits(1_572_856, 0, false));
|
||||
assert!(!admits(1_572_857, 0, false));
|
||||
assert!(should_inline_put(&config, &erasure, 196_608, 0, true, false));
|
||||
assert!(!should_inline_put(&config, &erasure, 196_609, 0, true, false));
|
||||
let small_blocks = coding::Erasure::new(12, 4, 4096);
|
||||
assert!(should_inline_put(&config, &small_blocks, 32 * 4096, 0, false, false));
|
||||
assert!(!should_inline_put(&config, &small_blocks, 32 * 4096 + 1, 0, false, false));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn inline_body_buffering_limits_size_and_frame_count() {
|
||||
let mut info = FileInfo::new("bucket/object", 12, 4);
|
||||
info.size = 2 * 1024 * 1024;
|
||||
info.erasure.block_size = 1024 * 1024;
|
||||
assert!(inline_body_buffering_miss_reason(&info).is_none());
|
||||
info.size += 1;
|
||||
assert!(inline_body_buffering_miss_reason(&info).is_some());
|
||||
info.erasure.block_size = 4096;
|
||||
info.size = 32 * 4096;
|
||||
assert!(inline_body_buffering_miss_reason(&info).is_none());
|
||||
info.size += 1;
|
||||
assert!(inline_body_buffering_miss_reason(&info).is_some());
|
||||
info.size = -1;
|
||||
assert!(inline_body_buffering_miss_reason(&info).is_some());
|
||||
info.size = 1;
|
||||
info.erasure.block_size = 0;
|
||||
assert!(inline_body_buffering_miss_reason(&info).is_some());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn collect_inline_data_shards_by_index_uses_distribution_order() {
|
||||
let distribution = vec![3, 1, 5, 2, 4, 6];
|
||||
@@ -15269,100 +15162,6 @@ mod tests {
|
||||
];
|
||||
|
||||
assert!(collect_inline_data_shard_fileinfos_by_index(&files, &fi, 2, |_| true).is_none());
|
||||
assert_eq!(
|
||||
collect_inline_data_shard_fileinfos_by_index_or_reason(&files, &fi, usize::MAX, |_| true).expect_err("invalid count"),
|
||||
GET_METADATA_EARLY_STOP_REASON_DATA_READ_INLINE_GEOMETRY,
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn inline_frame_decoder_handles_legacy_padding_hashes_and_unhashed_parts() {
|
||||
for size in [1, 3, 8, 11, 16] {
|
||||
let payload = &b"abcdefghijklmnop"[..size];
|
||||
for (legacy, algorithm) in [
|
||||
(false, HashAlgorithm::HighwayHash256S),
|
||||
(true, HashAlgorithm::HighwayHash256SLegacy),
|
||||
(false, HashAlgorithm::None),
|
||||
] {
|
||||
let erasure = coding::Erasure::new_with_options(2, 2, 8, legacy);
|
||||
let mut writers: Vec<_> = (0..4)
|
||||
.map(|_| {
|
||||
coding::BitrotWriterWrapper::new(
|
||||
coding::CustomWriter::new_inline_buffer(),
|
||||
erasure.shard_size(),
|
||||
algorithm.clone(),
|
||||
)
|
||||
})
|
||||
.collect();
|
||||
for block in payload.chunks(8) {
|
||||
let shards = erasure.encode_data(block).expect("encode golden frame");
|
||||
for (writer, shard) in writers.iter_mut().zip(shards) {
|
||||
writer.write(&shard).await.expect("write golden frame");
|
||||
}
|
||||
}
|
||||
let mut info = FileInfo::new("bucket/object", 2, 2);
|
||||
info.size = i64::try_from(payload.len()).expect("test payload size fits i64");
|
||||
info.erasure.block_size = 8;
|
||||
info.uses_legacy_checksum = legacy;
|
||||
info.add_object_part(1, "etag".to_owned(), payload.len(), None, info.size, None, None);
|
||||
info.erasure.checksums = vec![rustfs_filemeta::ChecksumInfo {
|
||||
part_number: 1,
|
||||
algorithm: algorithm.clone(),
|
||||
..Default::default()
|
||||
}];
|
||||
let mut files: Vec<_> = writers
|
||||
.into_iter()
|
||||
.enumerate()
|
||||
.map(|(index, writer)| {
|
||||
let mut file = info.clone();
|
||||
file.erasure.index = index + 1;
|
||||
file.data = writer.into_inline_data();
|
||||
file
|
||||
})
|
||||
.collect();
|
||||
let refs = [&files[0], &files[1]];
|
||||
let body = try_decode_inline_data_shards("golden", &info, &refs, false).expect("decode every golden frame");
|
||||
assert_eq!(body.as_ref(), payload);
|
||||
if algorithm.size() > 0 {
|
||||
for file in &files {
|
||||
verify_inline_part_bitrot(file).expect("deep verification covers both padded frames");
|
||||
}
|
||||
}
|
||||
for shard_index in 0..info.erasure.data_blocks {
|
||||
let original = files[shard_index].data.as_ref().expect("selected data shard").clone();
|
||||
let mut trailing = original.to_vec();
|
||||
trailing.push(0x7b);
|
||||
for malformed in [original[..original.len() - 1].to_vec(), trailing] {
|
||||
files[shard_index].data = Some(Bytes::from(malformed));
|
||||
for skip_verify_bitrot in [false, true] {
|
||||
assert!(
|
||||
try_decode_inline_data_shards("inexact", &info, &[&files[0], &files[1]], skip_verify_bitrot)
|
||||
.is_none(),
|
||||
"reject inexact shard {shard_index}: legacy={legacy}, algorithm={algorithm:?}, size={}",
|
||||
payload.len()
|
||||
);
|
||||
}
|
||||
}
|
||||
files[shard_index].data = Some(original);
|
||||
}
|
||||
let original = files[0].data.as_ref().expect("first data shard").clone();
|
||||
let mut truncated = original.to_vec();
|
||||
truncated.pop();
|
||||
files[0].data = Some(Bytes::from(truncated));
|
||||
assert!(try_decode_inline_data_shards("truncated", &info, &[&files[0], &files[1]], false).is_none());
|
||||
if algorithm.size() > 0 {
|
||||
assert_eq!(verify_inline_part_bitrot(&files[0]), Err(DiskError::FileCorrupt));
|
||||
let mut corrupt = original.to_vec();
|
||||
*corrupt.last_mut().expect("last frame payload") ^= 1;
|
||||
files[0].data = Some(Bytes::from(corrupt));
|
||||
assert!(try_decode_inline_data_shards("corrupt", &info, &[&files[0], &files[1]], false).is_none());
|
||||
assert_eq!(verify_inline_part_bitrot(&files[0]), Err(DiskError::FileCorrupt));
|
||||
assert!(try_decode_inline_data_shards("diagnostic", &info, &[&files[0], &files[1]], true).is_some());
|
||||
} else {
|
||||
assert_eq!(verify_inline_part_bitrot(&files[0]), Err(DiskError::BitrotHashAlgoInvalid));
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
@@ -15370,7 +15169,7 @@ mod tests {
|
||||
let payload = b"small inline object payload that spans data shards";
|
||||
let (erasure, files, read_length, checksum_algo) = inline_bitrot_files_for_payload(payload).await;
|
||||
let mut readers = build_inline_bitrot_readers(
|
||||
files.iter(),
|
||||
&files,
|
||||
erasure.data_shards,
|
||||
"bucket",
|
||||
"object",
|
||||
@@ -15396,7 +15195,7 @@ mod tests {
|
||||
let (erasure, files, read_length, checksum_algo) = inline_bitrot_files_for_payload_with_mode(payload, true).await;
|
||||
assert_ne!(payload.len() % erasure.data_shards, 0, "test payload must exercise EC padding");
|
||||
let mut readers = build_inline_bitrot_readers(
|
||||
files.iter(),
|
||||
&files,
|
||||
erasure.data_shards,
|
||||
"bucket",
|
||||
"object",
|
||||
@@ -15427,7 +15226,7 @@ mod tests {
|
||||
*second = Bytes::from(corrupted);
|
||||
|
||||
let mut readers = build_inline_bitrot_readers(
|
||||
files.iter(),
|
||||
&files,
|
||||
erasure.total_shard_count(),
|
||||
"bucket",
|
||||
"object",
|
||||
@@ -15445,17 +15244,31 @@ mod tests {
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn inline_erasure_size_helpers_match_erasure_methods() {
|
||||
fn inline_data_shards_direct_read_requires_single_block() {
|
||||
assert!(can_try_inline_data_shards_direct(1024, 1024));
|
||||
assert!(!can_try_inline_data_shards_direct(0, 1024));
|
||||
assert!(!can_try_inline_data_shards_direct(1025, 1024));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn inline_erasure_offset_helpers_match_erasure_methods() {
|
||||
for uses_legacy in [false, true] {
|
||||
let erasure = coding::Erasure::new_with_options(4, 2, 1024 * 1024, uses_legacy);
|
||||
for object_size in [1usize, 1024, 100 * 1024, 1024 * 1024, 1024 * 1024 + 1, 2 * 1024 * 1024 + 1] {
|
||||
for object_size in [1usize, 1024, 100 * 1024, 1024 * 1024] {
|
||||
assert_eq!(
|
||||
inline_erasure_shard_size(erasure.block_size, erasure.data_shards, uses_legacy),
|
||||
erasure.shard_size()
|
||||
);
|
||||
assert_eq!(
|
||||
inline_erasure_shard_file_size(object_size, erasure.block_size, erasure.data_shards, uses_legacy),
|
||||
usize::try_from(erasure.shard_file_size(object_size as i64)).expect("test shard size")
|
||||
inline_erasure_shard_file_offset(
|
||||
0,
|
||||
object_size,
|
||||
object_size,
|
||||
erasure.block_size,
|
||||
erasure.data_shards,
|
||||
uses_legacy,
|
||||
),
|
||||
erasure.shard_file_offset(0, object_size, object_size)
|
||||
);
|
||||
}
|
||||
}
|
||||
@@ -15976,14 +15789,25 @@ mod tests {
|
||||
|
||||
#[test]
|
||||
fn inline_admission_falls_back_to_actual_size_for_transformed_streams() {
|
||||
let config = crate::config::storageclass::Config::default();
|
||||
let erasure = coding::Erasure::new(2, 2, 1024 * 1024);
|
||||
let unknown = HashReader::SIZE_PRESERVE_LAYER;
|
||||
assert!(should_inline_put(&config, &erasure, 16 * 1024, 4 * 1024 * 1024, false, false));
|
||||
assert!(should_inline_put(&config, &erasure, 0, 4 * 1024, false, false));
|
||||
assert!(should_inline_put(&config, &erasure, unknown, 16 * 1024, false, false));
|
||||
assert!(!should_inline_put(&config, &erasure, unknown, 0, false, false));
|
||||
assert!(!should_inline_put(&config, &erasure, unknown, unknown, false, false));
|
||||
|
||||
// A known stored size is authoritative, whatever the plaintext size says.
|
||||
assert_eq!(
|
||||
inline_admission_shard_size(&erasure, 16 * 1024, 4 * 1024 * 1024),
|
||||
erasure.shard_file_size(16 * 1024)
|
||||
);
|
||||
assert_eq!(inline_admission_shard_size(&erasure, 0, 4 * 1024), 0);
|
||||
|
||||
// A transformed stream is sized by its plaintext length (MinIO parity).
|
||||
assert_eq!(
|
||||
inline_admission_shard_size(&erasure, unknown, 16 * 1024),
|
||||
erasure.shard_file_size(16 * 1024)
|
||||
);
|
||||
|
||||
// Neither size known, or an empty transformed stream, cannot be admitted.
|
||||
assert!(inline_admission_shard_size(&erasure, unknown, 0) < 0);
|
||||
assert!(inline_admission_shard_size(&erasure, unknown, unknown) < 0);
|
||||
}
|
||||
|
||||
#[test]
|
||||
|
||||
@@ -133,8 +133,7 @@ pub(crate) struct DeleteRecoverySummary {
|
||||
pub reclaimed: usize,
|
||||
/// Observed removals for this exact key, not aggregated across prefixes.
|
||||
pub removed_versions: HashSet<Uuid>,
|
||||
/// Complete discovery on a strict majority found no transaction for this key.
|
||||
/// Reusable only under the same key locks before a mutation. This never
|
||||
/// Read-only evidence for ordinary missing-key S3 semantics. This never
|
||||
/// authorizes settlement while a configured participant is unavailable.
|
||||
pub no_transactions_at_quorum: bool,
|
||||
}
|
||||
@@ -863,18 +862,16 @@ impl SetDisks {
|
||||
Err(error) => return Err(error),
|
||||
}
|
||||
}
|
||||
let no_transactions_at_quorum = transactions.is_empty() && !unsupported && inspected > self.set_drive_count / 2;
|
||||
if offline {
|
||||
return Ok(DeleteRecoverySummary {
|
||||
discovered: transactions.len(),
|
||||
pending: 1,
|
||||
no_transactions_at_quorum,
|
||||
no_transactions_at_quorum: transactions.is_empty() && !unsupported && inspected > self.set_drive_count / 2,
|
||||
..Default::default()
|
||||
});
|
||||
}
|
||||
let mut summary = DeleteRecoverySummary {
|
||||
discovered: transactions.len(),
|
||||
no_transactions_at_quorum,
|
||||
..Default::default()
|
||||
};
|
||||
let mut all_states = BTreeMap::new();
|
||||
@@ -1403,7 +1400,6 @@ fn unavailable(error: &DiskError) -> bool {
|
||||
match error {
|
||||
DiskError::DiskNotFound
|
||||
| DiskError::FaultyDisk
|
||||
| DiskError::FaultyRemoteDisk
|
||||
| DiskError::DiskOngoingReq
|
||||
| DiskError::Timeout
|
||||
| DiskError::RemoteClientUnavailable(_) => true,
|
||||
@@ -1431,32 +1427,6 @@ mod tests {
|
||||
use super::*;
|
||||
use crate::bucket::metadata_sys::{self, init_bucket_metadata_sys, test_support::isolated_store_over_temp_disks};
|
||||
|
||||
#[test]
|
||||
fn recovery_availability_classifies_local_and_remote_disk_failures() {
|
||||
for error in [
|
||||
DiskError::DiskNotFound,
|
||||
DiskError::FaultyDisk,
|
||||
DiskError::FaultyRemoteDisk,
|
||||
DiskError::Timeout,
|
||||
DiskError::DiskOngoingReq,
|
||||
] {
|
||||
assert!(unavailable(&error), "drive availability failure must defer recovery: {error:?}");
|
||||
}
|
||||
for error in [
|
||||
DiskError::FileCorrupt,
|
||||
DiskError::DiskAccessDenied,
|
||||
DiskError::FileAccessDenied,
|
||||
DiskError::InvalidPath,
|
||||
DiskError::DeleteRecoveryWorkLimit,
|
||||
DiskError::Io(std::io::Error::from_raw_os_error(libc::EIO)),
|
||||
] {
|
||||
assert!(
|
||||
!unavailable(&error),
|
||||
"integrity, access and unclassified IO errors must propagate: {error:?}"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn unresolved_heal_error_distinguishes_legacy_ambiguity_from_transient_blocks() {
|
||||
// rustfs/rustfs#8608: a legacy residue is deterministic and must not be
|
||||
@@ -3364,64 +3334,6 @@ mod tests {
|
||||
.expect("all proof is eventually collected");
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn delete_recovery_missing_delete_reuses_locked_discovery() {
|
||||
use crate::bucket::metadata_sys::ObjectLockConfigState;
|
||||
use crate::object_api::ObjectLockConfigSnapshot;
|
||||
|
||||
for offline in [false, true] {
|
||||
let bucket = if offline {
|
||||
"missing-probe-degraded"
|
||||
} else {
|
||||
"missing-probe-healthy"
|
||||
};
|
||||
let (_dirs, store, disks) = fixture(bucket).await;
|
||||
let set = &store.pools[0].disk_set[0];
|
||||
if offline {
|
||||
disks[3].force_offline_for_test();
|
||||
}
|
||||
for object in ["missing", "missing/deep/key"] {
|
||||
let online = &disks[..if offline { 3 } else { 4 }];
|
||||
let before: Vec<_> = online
|
||||
.iter()
|
||||
.map(|disk| {
|
||||
disk.metrics_snapshot()
|
||||
.expect("local disk metrics")
|
||||
.api_calls
|
||||
.get("discover_delete_transactions")
|
||||
.copied()
|
||||
.unwrap_or(0)
|
||||
})
|
||||
.collect();
|
||||
let opts = ObjectOptions {
|
||||
object_lock_config_snapshot: Some(Arc::new(ObjectLockConfigSnapshot::new(
|
||||
ObjectLockConfigState::ConfirmedAbsent,
|
||||
))),
|
||||
..Default::default()
|
||||
};
|
||||
let error = set
|
||||
.delete_object(bucket, object, opts)
|
||||
.await
|
||||
.expect_err("set layer retains typed absence");
|
||||
assert!(crate::error::is_err_object_not_found(&error), "typed absence: {error:?}");
|
||||
for (disk, before) in online.iter().zip(before) {
|
||||
let after = disk
|
||||
.metrics_snapshot()
|
||||
.expect("local disk metrics")
|
||||
.api_calls
|
||||
.get("discover_delete_transactions")
|
||||
.copied()
|
||||
.unwrap_or(0);
|
||||
assert_eq!(
|
||||
after - before,
|
||||
1,
|
||||
"one locked discovery per online drive, offline={offline}, key={object}"
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn delete_recovery_set_ordinary_missing_single_and_batch_keep_quorum_semantics() {
|
||||
let bucket = "recover-missing-quorum";
|
||||
|
||||
@@ -85,7 +85,6 @@ use crate::set_disk::metadata_sys;
|
||||
use crate::set_disk::runtime_sources;
|
||||
#[cfg(test)]
|
||||
use crate::storage_api_contracts::multipart::MultipartOperations;
|
||||
use crate::storage_api_contracts::multipart::{ListPartsOwner, MultipartOwner};
|
||||
#[cfg(test)]
|
||||
use crate::storage_api_contracts::object::HTTPPreconditions;
|
||||
use crate::storage_api_contracts::object::ObjectOperations;
|
||||
@@ -440,8 +439,6 @@ pub enum MultipartCommitPause {
|
||||
NewUploadBeforeLockLost,
|
||||
#[cfg(test)]
|
||||
ListPartsAfterMetadata,
|
||||
#[cfg(test)]
|
||||
PartBindingAfterMetadata,
|
||||
PutPartBeforeLockAcquire,
|
||||
PutPartBeforeLockLost,
|
||||
PutPartAfterCapacityAdmission,
|
||||
@@ -686,22 +683,6 @@ fn fence_commit_on_lock_loss(guard: Option<&ObjectLockDiagGuard>, mode: &'static
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn recorded_multipart_owner(
|
||||
metadata: &HashMap<String, String>,
|
||||
bucket: &str,
|
||||
encoded_object: &str,
|
||||
) -> Option<crate::storage_api_contracts::multipart::MultipartOwner> {
|
||||
let recorded_bucket = get_consistent_str(metadata, "multipart-bucket")?;
|
||||
let recorded_object = get_consistent_str(metadata, "multipart-object")?;
|
||||
if recorded_bucket != bucket || recorded_object != encoded_object {
|
||||
return None;
|
||||
}
|
||||
Some(crate::storage_api_contracts::multipart::MultipartOwner {
|
||||
bucket: recorded_bucket.to_owned(),
|
||||
object: decode_dir_object(recorded_object),
|
||||
})
|
||||
}
|
||||
|
||||
fn multipart_bucket_incarnation_id(metadata: &HashMap<String, String>) -> Result<Option<Uuid>> {
|
||||
let Some(value) = rustfs_utils::http::metadata_compat::get_consistent_str(metadata, SUFFIX_BUCKET_INCARNATION_ID) else {
|
||||
if rustfs_utils::http::metadata_compat::contains_key_str(metadata, SUFFIX_BUCKET_INCARNATION_ID) {
|
||||
@@ -1117,57 +1098,6 @@ struct MultipartUploadListScope<'a> {
|
||||
}
|
||||
|
||||
impl SetDisks {
|
||||
async fn read_multipart_info<const INCLUDE_OWNER: bool>(
|
||||
&self,
|
||||
bucket: &str,
|
||||
object: &str,
|
||||
upload_id: &str,
|
||||
opts: &ObjectOptions,
|
||||
) -> Result<(MultipartInfo, Option<MultipartOwner>)> {
|
||||
let _upload_guard = self
|
||||
.acquire_multipart_upload_read_lock("get_multipart_info", bucket, object, upload_id, opts)
|
||||
.await?;
|
||||
let (mut fi, _) = self
|
||||
.check_upload_id_exists_with_opts(bucket, object, upload_id, false, opts)
|
||||
.await
|
||||
.map_err(|e| to_object_err(e, vec![bucket, object, upload_id]))?;
|
||||
ensure_data_movement_upload_access(&fi, bucket, object, upload_id, opts)?;
|
||||
ensure_multipart_bucket_incarnation(&self.ctx, &fi, bucket, object, upload_id, opts.expected_bucket_incarnation_id)
|
||||
.await?;
|
||||
ensure_multipart_bucket_lifecycle_lock_held(bucket, object, opts)?;
|
||||
|
||||
#[cfg(test)]
|
||||
if INCLUDE_OWNER {
|
||||
pause_multipart_commit(bucket, object, MultipartCommitPause::PartBindingAfterMetadata).await;
|
||||
}
|
||||
let owner = if INCLUDE_OWNER {
|
||||
recorded_multipart_owner(&fi.metadata, bucket, object)
|
||||
} else {
|
||||
None
|
||||
};
|
||||
let info = MultipartInfo {
|
||||
bucket: bucket.to_owned(),
|
||||
object: decode_dir_object(object),
|
||||
upload_id: upload_id.to_owned(),
|
||||
user_defined: {
|
||||
strip_internal_multipart_metadata(&mut fi.metadata);
|
||||
fi.metadata.clone()
|
||||
},
|
||||
..Default::default()
|
||||
};
|
||||
Ok((info, owner))
|
||||
}
|
||||
|
||||
pub(crate) async fn lookup_multipart_part_binding(
|
||||
&self,
|
||||
bucket: &str,
|
||||
object: &str,
|
||||
upload_id: &str,
|
||||
opts: &ObjectOptions,
|
||||
) -> Result<(MultipartInfo, Option<MultipartOwner>)> {
|
||||
self.read_multipart_info::<true>(bucket, object, upload_id, opts).await
|
||||
}
|
||||
|
||||
fn validated_multipart_upload_path(bucket: &str, object: &str, upload_id: &str, data_movement: bool) -> Result<String> {
|
||||
let path = Self::get_upload_id_dir(bucket, object, upload_id);
|
||||
if !path
|
||||
@@ -3260,13 +3190,7 @@ impl crate::storage_api_contracts::multipart::MultipartOperations for SetDisks {
|
||||
.cloned()
|
||||
.unwrap_or_else(|| storageclass::STANDARD.to_string());
|
||||
|
||||
// Use the same metadata snapshot that produced the page.
|
||||
let recorded_owner = recorded_multipart_owner(&fi.metadata, bucket, object).map(|owner| ListPartsOwner {
|
||||
bucket: owner.bucket,
|
||||
object: owner.object,
|
||||
});
|
||||
let mut ret = ListPartsInfo {
|
||||
recorded_owner,
|
||||
bucket: bucket.to_owned(),
|
||||
object: decode_dir_object(object),
|
||||
upload_id: upload_id.to_owned(),
|
||||
@@ -3633,9 +3557,28 @@ impl crate::storage_api_contracts::multipart::MultipartOperations for SetDisks {
|
||||
upload_id: &str,
|
||||
opts: &ObjectOptions,
|
||||
) -> Result<MultipartInfo> {
|
||||
self.read_multipart_info::<false>(bucket, object, upload_id, opts)
|
||||
let _upload_guard = self
|
||||
.acquire_multipart_upload_read_lock("get_multipart_info", bucket, object, upload_id, opts)
|
||||
.await?;
|
||||
let (mut fi, _) = self
|
||||
.check_upload_id_exists_with_opts(bucket, object, upload_id, false, opts)
|
||||
.await
|
||||
.map(|(info, _)| info)
|
||||
.map_err(|e| to_object_err(e, vec![bucket, object, upload_id]))?;
|
||||
ensure_data_movement_upload_access(&fi, bucket, object, upload_id, opts)?;
|
||||
ensure_multipart_bucket_incarnation(&self.ctx, &fi, bucket, object, upload_id, opts.expected_bucket_incarnation_id)
|
||||
.await?;
|
||||
ensure_multipart_bucket_lifecycle_lock_held(bucket, object, opts)?;
|
||||
|
||||
Ok(MultipartInfo {
|
||||
bucket: bucket.to_owned(),
|
||||
object: decode_dir_object(object),
|
||||
upload_id: upload_id.to_owned(),
|
||||
user_defined: {
|
||||
strip_internal_multipart_metadata(&mut fi.metadata);
|
||||
fi.metadata.clone()
|
||||
},
|
||||
..Default::default()
|
||||
})
|
||||
}
|
||||
|
||||
#[tracing::instrument(skip(self))]
|
||||
@@ -3654,10 +3597,6 @@ impl crate::storage_api_contracts::multipart::MultipartOperations for SetDisks {
|
||||
ensure_multipart_bucket_incarnation(&self.ctx, &fi, bucket, object, upload_id, opts.expected_bucket_incarnation_id)
|
||||
.await?;
|
||||
ensure_multipart_bucket_lifecycle_lock_held(bucket, object, opts)?;
|
||||
if let Some(authorization) = &opts.multipart_abort_authorization {
|
||||
let owner = recorded_multipart_owner(&fi.metadata, bucket, object);
|
||||
authorization.authorize(owner.as_ref())?;
|
||||
}
|
||||
let disks = self.get_disks_internal().await;
|
||||
let cleanup = prepare_source_cleanup(
|
||||
&disks,
|
||||
@@ -3683,8 +3622,6 @@ impl crate::storage_api_contracts::multipart::MultipartOperations for SetDisks {
|
||||
.await;
|
||||
if result.is_ok() {
|
||||
remove_capped_multipart_staging_semaphore(&upload_id_path);
|
||||
#[cfg(any(test, feature = "test-util"))]
|
||||
pause_multipart_commit(bucket, object, MultipartCommitPause::AbortAfterDelete).await;
|
||||
}
|
||||
result
|
||||
}
|
||||
@@ -6497,8 +6434,6 @@ mod tests {
|
||||
assert!(empty.parts.is_empty());
|
||||
assert!(!empty.is_truncated);
|
||||
assert_eq!(empty.next_part_number_marker, None);
|
||||
let owner = empty.recorded_owner.expect("the empty page has the recorded owner");
|
||||
assert_eq!((owner.bucket.as_str(), owner.object.as_str()), (bucket, object));
|
||||
|
||||
for part_number in [10, 1, 3] {
|
||||
put_test_part(&set_disks, bucket, object, &upload.upload_id, part_number, b"part", 4).await;
|
||||
@@ -6525,184 +6460,9 @@ mod tests {
|
||||
assert_eq!(page.parts.iter().map(|part| part.part_num).collect::<Vec<_>>(), expected_parts);
|
||||
assert_eq!(page.is_truncated, next_marker.is_some());
|
||||
assert_eq!(page.next_part_number_marker, next_marker);
|
||||
assert_eq!(page.recorded_owner.as_ref(), Some(&owner));
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn list_parts_baseline_legacy_owner_metadata() {
|
||||
let (_roots, disks, set) = hermetic_set_disks(4).await;
|
||||
let bucket = "list-parts-owner-observation";
|
||||
let object = "object";
|
||||
let opts = ObjectOptions::default();
|
||||
make_bucket_on_all(&disks, bucket).await;
|
||||
let upload = set.new_multipart_upload(bucket, object, &opts).await.expect("real upload");
|
||||
let path = SetDisks::get_upload_id_dir(bucket, object, &upload.upload_id);
|
||||
let owner_keys = [
|
||||
RUSTFS_MULTIPART_BUCKET_KEY,
|
||||
"x-minio-internal-multipart-bucket",
|
||||
RUSTFS_MULTIPART_OBJECT_KEY,
|
||||
"x-minio-internal-multipart-object",
|
||||
];
|
||||
for (label, values) in [
|
||||
("rustfs", [Some(bucket), None, Some(object), None]),
|
||||
("minio", [None, Some(bucket), None, Some(object)]),
|
||||
("both", [Some(bucket), Some(bucket), Some(object), Some(object)]),
|
||||
("no bucket", [None, None, Some(object), None]),
|
||||
("no object", [Some(bucket), None, None, None]),
|
||||
("other bucket", [Some("other-bucket"), None, Some(object), None]),
|
||||
("other object", [Some(bucket), None, Some("other-object"), None]),
|
||||
("bucket aliases differ", [Some(bucket), Some("other-bucket"), Some(object), None]),
|
||||
("object aliases differ", [Some(bucket), None, Some(object), Some("other-object")]),
|
||||
] {
|
||||
for disk in &disks {
|
||||
let mut fi = disk
|
||||
.read_version("", RUSTFS_META_MULTIPART_BUCKET, &path, "", &ReadOptions::default())
|
||||
.await
|
||||
.expect("read actual upload metadata");
|
||||
for (key, value) in owner_keys.into_iter().zip(values) {
|
||||
if let Some(value) = value {
|
||||
fi.metadata.insert(key.to_owned(), value.to_owned());
|
||||
} else {
|
||||
fi.metadata.remove(key);
|
||||
}
|
||||
}
|
||||
disk.write_metadata(bucket, RUSTFS_META_MULTIPART_BUCKET, &path, fi)
|
||||
.await
|
||||
.expect("write the synthetic owner metadata");
|
||||
}
|
||||
let page = set
|
||||
.list_object_parts(bucket, object, &upload.upload_id, None, 1, &opts)
|
||||
.await
|
||||
.unwrap_or_else(|error| panic!("{label}: {error:?}"));
|
||||
assert_eq!((page.bucket.as_str(), page.object.as_str()), (bucket, object), "{label}");
|
||||
assert!(page.parts.is_empty(), "{label}");
|
||||
let expected = matches!(label, "rustfs" | "minio" | "both").then_some((bucket, object));
|
||||
assert_eq!(
|
||||
page.recorded_owner
|
||||
.as_ref()
|
||||
.map(|owner| (owner.bucket.as_str(), owner.object.as_str())),
|
||||
expected,
|
||||
"{label}"
|
||||
);
|
||||
let info = set
|
||||
.get_multipart_info(bucket, object, &upload.upload_id, &opts)
|
||||
.await
|
||||
.unwrap_or_else(|error| panic!("{label}: {error:?}"));
|
||||
assert_eq!((info.bucket.as_str(), info.object.as_str()), (bucket, object), "{label}");
|
||||
assert_eq!(info.upload_id, upload.upload_id, "{label}");
|
||||
assert_eq!(info.user_defined, page.user_defined, "{label}");
|
||||
let (bound_info, owner) = set
|
||||
.lookup_multipart_part_binding(bucket, object, &upload.upload_id, &opts)
|
||||
.await
|
||||
.unwrap_or_else(|error| panic!("{label}: {error:?}"));
|
||||
assert_eq!((bound_info.bucket.as_str(), bound_info.object.as_str()), (bucket, object), "{label}");
|
||||
assert_eq!(bound_info.upload_id, upload.upload_id, "{label}");
|
||||
assert_eq!(
|
||||
owner.as_ref().map(|owner| (owner.bucket.as_str(), owner.object.as_str())),
|
||||
page.recorded_owner
|
||||
.as_ref()
|
||||
.map(|owner| (owner.bucket.as_str(), owner.object.as_str())),
|
||||
"{label}"
|
||||
);
|
||||
assert_eq!(bound_info.user_defined, page.user_defined, "{label}");
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn multipart_owner_matches_encoded_namespace_before_decoding() {
|
||||
let bucket = "owner-bucket";
|
||||
let object = encode_dir_object("directory/");
|
||||
let mut metadata = HashMap::new();
|
||||
insert_str(&mut metadata, "multipart-bucket", bucket.to_owned());
|
||||
insert_str(&mut metadata, "multipart-object", object.clone());
|
||||
let owner = recorded_multipart_owner(&metadata, bucket, &object).expect("matching metadata");
|
||||
assert_eq!((owner.bucket.as_str(), owner.object.as_str()), (bucket, "directory/"));
|
||||
assert!(recorded_multipart_owner(&metadata, bucket, "directory/").is_none());
|
||||
assert!(recorded_multipart_owner(&metadata, "other-bucket", &object).is_none());
|
||||
metadata.insert("X-Minio-Internal-Multipart-Object".to_owned(), "foreign".to_owned());
|
||||
assert!(recorded_multipart_owner(&metadata, bucket, &object).is_none());
|
||||
metadata.remove("X-Minio-Internal-Multipart-Object");
|
||||
insert_str(&mut metadata, "multipart-object", String::new());
|
||||
assert!(recorded_multipart_owner(&metadata, bucket, &object).is_none());
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread")]
|
||||
#[serial]
|
||||
async fn multipart_part_binding_uses_one_snapshot_under_upload_read_lock() {
|
||||
let manager = Arc::new(rustfs_lock::GlobalLockManager::new());
|
||||
let lockers: Vec<Arc<dyn LockClient>> = vec![Arc::new(LocalClient::with_manager(manager))];
|
||||
let (_dirs, disks, set) = hermetic_set_disks_with_lockers(4, 0, 2, lockers).await;
|
||||
let bucket = "part-binding-snapshot";
|
||||
let object = "object";
|
||||
make_bucket_on_all(&disks, bucket).await;
|
||||
let upload = set
|
||||
.new_multipart_upload(bucket, object, &ObjectOptions::default())
|
||||
.await
|
||||
.expect("create upload");
|
||||
let path = SetDisks::get_upload_id_dir(bucket, object, &upload.upload_id);
|
||||
for disk in &disks {
|
||||
let mut fi = disk
|
||||
.read_version("", RUSTFS_META_MULTIPART_BUCKET, &path, "", &ReadOptions::default())
|
||||
.await
|
||||
.expect("read upload metadata");
|
||||
fi.metadata.insert("x-amz-meta-observation".to_owned(), "before".to_owned());
|
||||
disk.write_metadata(bucket, RUSTFS_META_MULTIPART_BUCKET, &path, fi)
|
||||
.await
|
||||
.expect("write initial observation");
|
||||
}
|
||||
|
||||
let _setup_type_guard = SetupTypeGuard::switch_to(SetupType::DistErasure).await;
|
||||
let barrier = MultipartCommitBarrier::install(bucket, object, MultipartCommitPause::PartBindingAfterMetadata);
|
||||
let lookup_set = Arc::clone(&set);
|
||||
let upload_id = upload.upload_id.clone();
|
||||
let lookup = tokio::spawn(async move {
|
||||
lookup_set
|
||||
.lookup_multipart_part_binding(bucket, object, &upload_id, &ObjectOptions::default())
|
||||
.await
|
||||
});
|
||||
barrier.wait_until_paused().await;
|
||||
let upload_lock = set
|
||||
.new_ns_lock(RUSTFS_META_MULTIPART_BUCKET, &path)
|
||||
.await
|
||||
.expect("create upload lock");
|
||||
let lock_error = upload_lock
|
||||
.get_write_lock(Duration::from_millis(50))
|
||||
.await
|
||||
.expect_err("the binding must retain its upload read lock while observing metadata");
|
||||
assert!(matches!(lock_error, rustfs_lock::LockError::Timeout { .. }));
|
||||
|
||||
// Bypass the upload lock to detect an accidental second metadata read.
|
||||
for disk in &disks {
|
||||
let mut fi = disk
|
||||
.read_version("", RUSTFS_META_MULTIPART_BUCKET, &path, "", &ReadOptions::default())
|
||||
.await
|
||||
.expect("read upload metadata");
|
||||
insert_str(&mut fi.metadata, "multipart-object", "foreign".to_owned());
|
||||
fi.metadata.insert("x-amz-meta-observation".to_owned(), "after".to_owned());
|
||||
disk.write_metadata(bucket, RUSTFS_META_MULTIPART_BUCKET, &path, fi)
|
||||
.await
|
||||
.expect("write changed observation");
|
||||
}
|
||||
barrier.release();
|
||||
let (info, owner) = lookup.await.expect("lookup task").expect("initial binding");
|
||||
assert_eq!(info.user_defined.get("x-amz-meta-observation").map(String::as_str), Some("before"));
|
||||
let owner = owner.expect("initial owner");
|
||||
assert_eq!((owner.bucket.as_str(), owner.object.as_str()), (bucket, object));
|
||||
drop(barrier);
|
||||
|
||||
let (updated, owner) = set
|
||||
.lookup_multipart_part_binding(bucket, object, &upload.upload_id, &ObjectOptions::default())
|
||||
.await
|
||||
.expect("updated binding");
|
||||
assert_eq!(updated.user_defined.get("x-amz-meta-observation").map(String::as_str), Some("after"));
|
||||
assert!(owner.is_none(), "a changed recorded namespace must not produce owner evidence");
|
||||
upload_lock
|
||||
.get_write_lock(Duration::from_secs(5))
|
||||
.await
|
||||
.expect("the upload read lock is released after the observation");
|
||||
}
|
||||
|
||||
async fn object_transaction_epochs(disks: &[DiskStore], bucket: &str, object: &str) -> Vec<Option<Uuid>> {
|
||||
let mut epochs = Vec::with_capacity(disks.len());
|
||||
let deadline = tokio::time::Instant::now() + Duration::from_secs(30);
|
||||
@@ -7754,13 +7514,12 @@ mod tests {
|
||||
let manager = Arc::new(rustfs_lock::GlobalLockManager::new());
|
||||
let signaling = Arc::new(SignalingLockClient::new(Arc::new(LocalClient::with_manager(manager))));
|
||||
let lockers: Vec<Arc<dyn LockClient>> = vec![signaling.clone()];
|
||||
let instance_ctx = Arc::new(crate::runtime::instance::InstanceContext::new());
|
||||
instance_ctx.update_erasure_type(SetupType::DistErasure).await;
|
||||
let (_temp_dirs, disk_stores, set_disks) = hermetic_set_disks_with_lockers_and_ctx(4, 0, 2, lockers, instance_ctx).await;
|
||||
let (_temp_dirs, disk_stores, set_disks) = hermetic_set_disks_with_lockers(4, 0, 2, lockers).await;
|
||||
make_bucket_on_all(&disk_stores, bucket).await;
|
||||
let (upload_id, parts) = stage_upload_with_create_opts(&set_disks, bucket, object, &[0x47; 4096], &create_opts).await;
|
||||
let upload_id_path = SetDisks::get_upload_id_dir(bucket, object, &upload_id);
|
||||
signaling.set_target(rustfs_lock::ObjectKey::new(RUSTFS_META_MULTIPART_BUCKET, upload_id_path));
|
||||
let _setup_type_guard = SetupTypeGuard::switch_to(SetupType::DistErasure).await;
|
||||
let barrier = MultipartCommitBarrier::install(bucket, object, MultipartCommitPause::AfterRename);
|
||||
|
||||
let complete_store = set_disks.clone();
|
||||
@@ -7806,14 +7565,27 @@ mod tests {
|
||||
let manager = Arc::new(rustfs_lock::GlobalLockManager::new());
|
||||
let signaling = Arc::new(SignalingLockClient::new(Arc::new(LocalClient::with_manager(manager))));
|
||||
let lockers: Vec<Arc<dyn LockClient>> = vec![signaling.clone()];
|
||||
let instance_ctx = Arc::new(crate::runtime::instance::InstanceContext::new());
|
||||
instance_ctx.update_erasure_type(SetupType::DistErasure).await;
|
||||
let (_temp_dirs, disk_stores, set_disks) = hermetic_set_disks_with_lockers_and_ctx(4, 0, 2, lockers, instance_ctx).await;
|
||||
let (_temp_dirs, disk_stores, set_disks) = hermetic_set_disks_with_lockers(4, 0, 2, lockers).await;
|
||||
make_bucket_on_all(&disk_stores, bucket).await;
|
||||
let (upload_id, parts) = stage_upload_with_create_opts(&set_disks, bucket, object, &[0x48; 4096], &create_opts).await;
|
||||
let upload_id_path = SetDisks::get_upload_id_dir(bucket, object, &upload_id);
|
||||
signaling.set_target(rustfs_lock::ObjectKey::new(RUSTFS_META_MULTIPART_BUCKET, upload_id_path.clone()));
|
||||
let barrier = MultipartCommitBarrier::install(bucket, object, MultipartCommitPause::AbortAfterDelete);
|
||||
let _setup_type_guard = SetupTypeGuard::switch_to(SetupType::DistErasure).await;
|
||||
let object_holder = set_disks
|
||||
.new_ns_lock(bucket, object)
|
||||
.await
|
||||
.expect("object namespace lock should be created")
|
||||
.get_write_lock(Duration::from_secs(5))
|
||||
.await
|
||||
.expect("test should hold the object lock");
|
||||
let holder = set_disks
|
||||
.new_ns_lock(RUSTFS_META_MULTIPART_BUCKET, &upload_id_path)
|
||||
.await
|
||||
.expect("upload namespace lock should be created")
|
||||
.get_write_lock(Duration::from_secs(5))
|
||||
.await
|
||||
.expect("test should hold the upload lock");
|
||||
signaling.wait_for_attempts(1).await;
|
||||
|
||||
let abort_store = set_disks.clone();
|
||||
let abort_upload_id = upload_id.clone();
|
||||
@@ -7822,9 +7594,7 @@ mod tests {
|
||||
.abort_multipart_upload(bucket, object, &abort_upload_id, &ObjectOptions::default())
|
||||
.await
|
||||
});
|
||||
// Abort finishes disk cleanup before completion starts its lock deadline,
|
||||
// and retains upload ownership until the competing request is waiting.
|
||||
barrier.wait_until_paused().await;
|
||||
signaling.wait_for_attempts(2).await;
|
||||
|
||||
let complete_store = set_disks.clone();
|
||||
let complete_upload_id = upload_id.clone();
|
||||
@@ -7833,21 +7603,21 @@ mod tests {
|
||||
.complete_multipart_upload(bucket, object, &complete_upload_id, parts, &ObjectOptions::default())
|
||||
.await
|
||||
});
|
||||
signaling.wait_for_attempts(2).await;
|
||||
assert!(!complete.is_finished(), "completion must wait for the abort upload lock");
|
||||
barrier.release();
|
||||
drop(holder);
|
||||
|
||||
abort
|
||||
.await
|
||||
.expect("abort task should not panic")
|
||||
.expect("abort should win the upload finalization");
|
||||
drop(object_holder);
|
||||
let complete_err = complete
|
||||
.await
|
||||
.expect("completion task should not panic")
|
||||
.expect_err("completion must observe the aborted upload");
|
||||
assert!(
|
||||
matches!(complete_err, StorageError::InvalidUploadID(..)),
|
||||
"abort-first refusal case={bucket}: {complete_err:?}"
|
||||
"abort-first refusal case={bucket} code={}",
|
||||
complete_err.code().as_u32()
|
||||
);
|
||||
let object_err = set_disks
|
||||
.get_object_info(bucket, object, &ObjectOptions::default())
|
||||
|
||||
@@ -1029,229 +1029,3 @@ mod native_abort_closure_tests {
|
||||
include!("multipart_initial_probe_tests.rs");
|
||||
}
|
||||
}
|
||||
|
||||
mod abort_owner_tests {
|
||||
use super::*;
|
||||
use crate::storage_api_contracts::multipart::{MultipartAbortAuthorization, MultipartOwner};
|
||||
|
||||
#[test]
|
||||
fn recorded_multipart_owner_requires_unambiguous_metadata_for_the_selected_namespace() {
|
||||
const BUCKET: &str = "abort-owner";
|
||||
const ENCODED: &str = "directory__XLDIR__";
|
||||
const RUSTFS: &str = RUSTFS_MULTIPART_BUCKET_KEY;
|
||||
const MINIO: &str = "x-minio-internal-multipart-bucket";
|
||||
const RUSTFS_OBJECT: &str = RUSTFS_MULTIPART_OBJECT_KEY;
|
||||
const MINIO_OBJECT: &str = "x-minio-internal-multipart-object";
|
||||
|
||||
let expected = MultipartOwner {
|
||||
bucket: BUCKET.to_owned(),
|
||||
object: "directory/".to_owned(),
|
||||
};
|
||||
let rustfs = [(RUSTFS, BUCKET), (RUSTFS_OBJECT, ENCODED)];
|
||||
let minio = [(MINIO, BUCKET), (MINIO_OBJECT, ENCODED)];
|
||||
for fields in [rustfs.to_vec(), minio.to_vec(), [rustfs, minio].concat()] {
|
||||
let metadata = fields
|
||||
.into_iter()
|
||||
.map(|(key, value)| (key.to_owned(), value.to_owned()))
|
||||
.collect();
|
||||
assert_eq!(recorded_multipart_owner(&metadata, BUCKET, ENCODED), Some(expected.clone()));
|
||||
}
|
||||
for fields in [
|
||||
vec![(RUSTFS, BUCKET)],
|
||||
vec![(RUSTFS_OBJECT, ENCODED)],
|
||||
vec![(RUSTFS, ""), (RUSTFS_OBJECT, ENCODED)],
|
||||
vec![(RUSTFS, BUCKET), (RUSTFS_OBJECT, "")],
|
||||
vec![(RUSTFS, "foreign"), (RUSTFS_OBJECT, ENCODED)],
|
||||
vec![(RUSTFS, BUCKET), (RUSTFS_OBJECT, "directory/")],
|
||||
[rustfs, [(MINIO, "foreign"), (MINIO_OBJECT, ENCODED)]].concat(),
|
||||
[rustfs, [(MINIO, BUCKET), ("X-MINIO-INTERNAL-MULTIPART-OBJECT", "other")]].concat(),
|
||||
] {
|
||||
let metadata = fields
|
||||
.iter()
|
||||
.map(|(key, value)| (key.to_string(), value.to_string()))
|
||||
.collect();
|
||||
assert_eq!(recorded_multipart_owner(&metadata, BUCKET, ENCODED), None, "{fields:?}");
|
||||
}
|
||||
}
|
||||
|
||||
struct AbortGate {
|
||||
expected: Option<MultipartOwner>,
|
||||
accepted: bool,
|
||||
calls: AtomicUsize,
|
||||
}
|
||||
|
||||
impl MultipartAbortAuthorization for AbortGate {
|
||||
type Error = StorageError;
|
||||
|
||||
fn authorize(&self, owner: Option<&MultipartOwner>) -> Result<()> {
|
||||
self.calls.fetch_add(1, Ordering::Relaxed);
|
||||
assert_eq!(owner, self.expected.as_ref(), "gate sees the actual selected upload owner");
|
||||
if self.accepted {
|
||||
Ok(())
|
||||
} else {
|
||||
Err(StorageError::InvalidUploadID(String::new(), String::new(), String::new()))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn abort_owner_callback_controls_physical_deletion() {
|
||||
let (_dirs, disks, set) = hermetic_set_disks(4).await;
|
||||
let bucket = "abort-locked-owner";
|
||||
let key = "staged-object";
|
||||
make_bucket_on_all(&disks, bucket).await;
|
||||
let (id, _) = stage_upload_with_create_opts(&set, bucket, key, b"staged part", &ObjectOptions::default()).await;
|
||||
let path = SetDisks::get_upload_id_dir(bucket, key, &id);
|
||||
let (fi, _) = set
|
||||
.check_upload_id_exists(bucket, key, &id, true)
|
||||
.await
|
||||
.expect("staged upload");
|
||||
let part = format!("{path}/{}/part.1", fi.data_dir.expect("staged data directory"));
|
||||
let expected = MultipartOwner {
|
||||
bucket: bucket.to_owned(),
|
||||
object: key.to_owned(),
|
||||
};
|
||||
let mut before = Vec::new();
|
||||
for disk in &disks {
|
||||
before.push(
|
||||
disk.read_all(RUSTFS_META_MULTIPART_BUCKET, &part)
|
||||
.await
|
||||
.expect("staged shard"),
|
||||
);
|
||||
}
|
||||
let rejected = Arc::new(AbortGate {
|
||||
expected: Some(expected.clone()),
|
||||
accepted: false,
|
||||
calls: AtomicUsize::new(0),
|
||||
});
|
||||
let opts = ObjectOptions {
|
||||
multipart_abort_authorization: Some(rejected.clone()),
|
||||
..Default::default()
|
||||
};
|
||||
let result = set.abort_multipart_upload(bucket, key, &id, &opts).await;
|
||||
for (disk, staged) in disks.iter().zip(before) {
|
||||
let after = disk
|
||||
.read_all(RUSTFS_META_MULTIPART_BUCKET, &part)
|
||||
.await
|
||||
.expect("denied Abort preserves the shard");
|
||||
assert_eq!(staged, after);
|
||||
}
|
||||
assert!(matches!(result, Err(StorageError::InvalidUploadID(..))));
|
||||
assert_eq!(rejected.calls.load(Ordering::Relaxed), 1);
|
||||
|
||||
let allowed = Arc::new(AbortGate {
|
||||
expected: Some(expected),
|
||||
accepted: true,
|
||||
calls: AtomicUsize::new(0),
|
||||
});
|
||||
let opts = ObjectOptions {
|
||||
multipart_abort_authorization: Some(allowed.clone()),
|
||||
..Default::default()
|
||||
};
|
||||
set.abort_multipart_upload(bucket, key, &id, &opts)
|
||||
.await
|
||||
.expect("owner permits Abort");
|
||||
assert_eq!(allowed.calls.load(Ordering::Relaxed), 1);
|
||||
for disk in &disks {
|
||||
assert!(matches!(
|
||||
disk.read_all(RUSTFS_META_MULTIPART_BUCKET, &part).await,
|
||||
Err(DiskError::FileNotFound)
|
||||
));
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn abort_owner_comes_from_the_selected_metadata_on_all_disks() {
|
||||
use rustfs_utils::http::{get_str, remove_str};
|
||||
|
||||
let (_dirs, disks, set) = hermetic_set_disks(4).await;
|
||||
let bucket = "abort-owner-metadata";
|
||||
make_bucket_on_all(&disks, bucket).await;
|
||||
for (key, bucket_fields, object_fields, accepted) in [
|
||||
("rustfs", [Some(bucket), None], [Some("rustfs"), None], true),
|
||||
("minio", [None, Some(bucket)], [None, Some("minio")], true),
|
||||
("matching", [Some(bucket), Some(bucket)], [Some("matching"), Some("matching")], true),
|
||||
("empty", [Some(bucket), Some(bucket)], [Some("empty"), Some("")], false),
|
||||
("missing", [None, None], [Some("missing"), None], false),
|
||||
("foreign-bucket", [Some("foreign"), None], [Some("foreign-bucket"), None], false),
|
||||
("foreign-key", [Some(bucket), None], [Some("unrelated"), None], false),
|
||||
("conflict", [Some(bucket), Some("foreign")], [Some("conflict"), Some("conflict")], false),
|
||||
] {
|
||||
let (id, _) = stage_upload_with_create_opts(&set, bucket, key, b"staged shard data", &ObjectOptions::default()).await;
|
||||
let path = SetDisks::get_upload_id_dir(bucket, key, &id);
|
||||
pin_historical_unmarked_upload(&disks, bucket, &path).await;
|
||||
let mut part_path = None;
|
||||
let mut staged = Vec::new();
|
||||
let fields = [
|
||||
(RUSTFS_MULTIPART_BUCKET_KEY, bucket_fields[0]),
|
||||
("x-minio-internal-multipart-bucket", bucket_fields[1]),
|
||||
(RUSTFS_MULTIPART_OBJECT_KEY, object_fields[0]),
|
||||
("x-minio-internal-multipart-object", object_fields[1]),
|
||||
];
|
||||
for disk in &disks {
|
||||
let mut head = disk
|
||||
.read_version("", RUSTFS_META_MULTIPART_BUCKET, &path, "", &ReadOptions::default())
|
||||
.await
|
||||
.expect("read the staged metadata on each disk");
|
||||
let part = part_path.get_or_insert_with(|| format!("{path}/{}/part.1", head.data_dir.expect("part directory")));
|
||||
staged.push(
|
||||
disk.read_all(RUSTFS_META_MULTIPART_BUCKET, part)
|
||||
.await
|
||||
.expect("read staged part before Abort"),
|
||||
);
|
||||
assert_eq!(get_str(&head.metadata, "multipart-bucket").as_deref(), Some(bucket));
|
||||
remove_str(&mut head.metadata, "multipart-bucket");
|
||||
remove_str(&mut head.metadata, "multipart-object");
|
||||
for (name, value) in fields {
|
||||
if let Some(value) = value {
|
||||
head.metadata.insert(name.to_owned(), value.to_owned());
|
||||
}
|
||||
}
|
||||
disk.write_metadata(bucket, RUSTFS_META_MULTIPART_BUCKET, &path, head)
|
||||
.await
|
||||
.expect("write the test owner on each disk");
|
||||
let persisted = disk
|
||||
.read_version("", RUSTFS_META_MULTIPART_BUCKET, &path, "", &ReadOptions::default())
|
||||
.await
|
||||
.expect("verify the actual persisted metadata");
|
||||
for (name, value) in fields {
|
||||
assert_eq!(persisted.metadata.get(name).map(String::as_str), value);
|
||||
}
|
||||
}
|
||||
let gate = Arc::new(AbortGate {
|
||||
expected: accepted.then(|| MultipartOwner {
|
||||
bucket: bucket.to_owned(),
|
||||
object: key.to_owned(),
|
||||
}),
|
||||
accepted,
|
||||
calls: AtomicUsize::new(0),
|
||||
});
|
||||
let opts = ObjectOptions {
|
||||
multipart_abort_authorization: Some(gate.clone()),
|
||||
..Default::default()
|
||||
};
|
||||
let result = set.abort_multipart_upload(bucket, key, &id, &opts).await;
|
||||
for (disk, previous) in disks.iter().zip(staged) {
|
||||
let after = disk
|
||||
.read_all(RUSTFS_META_MULTIPART_BUCKET, part_path.as_deref().expect("part path"))
|
||||
.await;
|
||||
if accepted {
|
||||
assert!(matches!(after, Err(DiskError::FileNotFound)), "{key}: authorized data removed");
|
||||
} else {
|
||||
assert_eq!(after.expect("refused Abort preserves the staged part"), previous, "{key}");
|
||||
}
|
||||
}
|
||||
let calls = gate.calls.load(Ordering::Relaxed);
|
||||
if accepted {
|
||||
assert_eq!(calls, 1, "{key}");
|
||||
result.expect("Abort with agreeing persisted owner");
|
||||
} else {
|
||||
assert!(result.is_err(), "{key}: invalid metadata must not authorize Abort");
|
||||
if key == "empty" {
|
||||
// The existing source inspection refuses this head before selecting an owner.
|
||||
assert_eq!(calls, 0, "malformed physical owner was refused before the callback");
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -17,18 +17,17 @@ use super::{
|
||||
GetObjectFileInfo, GetObjectMetadataCacheCapability, GetObjectMetadataCacheEntry, GetObjectMetadataCacheGeneration,
|
||||
GetObjectMetadataCacheKey, GetObjectMetadataCacheLookupKey, GetObjectReadPolicy, HashAlgorithm, LOG_COMPONENT_ECSTORE,
|
||||
LOG_SUBSYSTEM_SET_DISK, OBJECT_OP_IGNORED_ERRS, ObjectInfo, ObjectOptions, RUSTFS_META_BUCKET, ReadOptions, Result, SetDisks,
|
||||
StorageError, adaptive_duplex_buffer_size, build_get_codec_streaming_decode_engine, build_inline_bitrot_readers,
|
||||
StorageError, adaptive_duplex_buffer_size, build_get_codec_streaming_decode_engine, build_inline_bitrot_readers_from_refs,
|
||||
collect_inline_data_shard_fileinfos_by_index, debug, error, get_codec_streaming_metrics_path,
|
||||
get_codec_streaming_multipart_max_parts, get_object_read_policy, is_codec_streaming_multipart_enabled,
|
||||
is_multipart_reader_setup_prefetch_enabled, object_fits_single_block, part_checksum_algorithm, reduce_read_quorum_errs,
|
||||
to_object_err, try_read_inline_data_shards_direct, warn,
|
||||
is_multipart_reader_setup_prefetch_enabled, object_fits_single_block, reduce_read_quorum_errs, to_object_err,
|
||||
try_read_inline_data_shards_direct, warn,
|
||||
};
|
||||
use crate::diagnostics::get::{
|
||||
GET_DIRECT_MEMORY_SUBPATH_DISK_DATA_BLOCKS, GET_DIRECT_MEMORY_SUBPATH_INLINE_BUFFERED, GET_METADATA_CACHE_DECISION_HIT,
|
||||
GET_METADATA_CACHE_DECISION_MISS, GET_METADATA_CACHE_DECISION_REJECT, GET_METADATA_CACHE_DECISION_SKIP,
|
||||
GET_METADATA_CACHE_REASON_BODY_LAYOUT_REQUIRED, GET_METADATA_CACHE_REASON_DATA_MOVEMENT,
|
||||
GET_METADATA_CACHE_REASON_DELETE_MARKER, GET_METADATA_CACHE_REASON_DIST_ERASURE, GET_METADATA_CACHE_REASON_FENCE_LOCK_LOST,
|
||||
GET_METADATA_CACHE_REASON_FENCE_PROOF_MISMATCH, GET_METADATA_CACHE_REASON_FENCE_PROOF_UNAVAILABLE,
|
||||
GET_METADATA_CACHE_REASON_DELETE_MARKER, GET_METADATA_CACHE_REASON_DIST_ERASURE,
|
||||
GET_METADATA_CACHE_REASON_INCL_FREE_VERSIONS, GET_METADATA_CACHE_REASON_INSUFFICIENT_CACHED_QUORUM,
|
||||
GET_METADATA_CACHE_REASON_META_BUCKET, GET_METADATA_CACHE_REASON_NO_LOCK, GET_METADATA_CACHE_REASON_NOT_FOUND_OR_EXPIRED,
|
||||
GET_METADATA_CACHE_REASON_NOT_READ_DATA, GET_METADATA_CACHE_REASON_PART_CHECKSUMS, GET_METADATA_CACHE_REASON_PART_NUMBER,
|
||||
@@ -58,39 +57,11 @@ use tokio::io::{AsyncRead, AsyncWrite, ReadBuf};
|
||||
|
||||
pub(super) const GET_OBJECT_PATH_MID_SIZE_STREAMING: &str = "mid_size_streaming";
|
||||
|
||||
#[derive(Clone)]
|
||||
struct MetadataCacheAuthority {
|
||||
proof: Arc<rustfs_lock::CacheFenceProof>,
|
||||
fence: crate::object_api::NamespaceLockFence,
|
||||
}
|
||||
|
||||
impl MetadataCacheAuthority {
|
||||
fn from_options(opts: &ObjectOptions) -> Option<Self> {
|
||||
Some(Self {
|
||||
proof: opts.metadata_cache_fence_proof.clone()?,
|
||||
fence: opts.namespace_lock_fence.clone()?,
|
||||
})
|
||||
}
|
||||
|
||||
fn is_current(&self) -> bool {
|
||||
!self.fence.is_lock_lost()
|
||||
}
|
||||
}
|
||||
|
||||
struct MetadataCachePublication {
|
||||
fi: FileInfo,
|
||||
capability: GetObjectMetadataCacheCapability,
|
||||
verified_inline_body: Option<Bytes>,
|
||||
authority: Option<MetadataCacheAuthority>,
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
use super::DEFAULT_GET_OBJECT_METADATA_CACHE_MAX_ENTRIES;
|
||||
#[cfg(test)]
|
||||
use super::DEFAULT_RUSTFS_GET_CODEC_STREAMING_MIN_SIZE;
|
||||
#[cfg(test)]
|
||||
use super::ENV_RUSTFS_EXPERIMENTAL_GET_OBJECT_METADATA_CACHE_DISTRIBUTED_PROOF_ENABLE;
|
||||
#[cfg(test)]
|
||||
use super::ENV_RUSTFS_GET_CODEC_STREAMING_BODY_COMPAT_CONFIRMED;
|
||||
#[cfg(test)]
|
||||
use super::ENV_RUSTFS_GET_CODEC_STREAMING_DATA_BLOCKS_FIRST_ENABLE;
|
||||
@@ -483,39 +454,28 @@ impl SetDisks {
|
||||
if let Some(reason) = get_object_metadata_cache_request_bypass_reason(bucket, opts, read_data) {
|
||||
return Some(reason);
|
||||
}
|
||||
if self.ctx.is_dist_erasure().await {
|
||||
if !super::is_get_object_metadata_cache_distributed_proof_enabled() {
|
||||
return Some(GET_METADATA_CACHE_REASON_DIST_ERASURE);
|
||||
}
|
||||
return match MetadataCacheAuthority::from_options(opts) {
|
||||
None => Some(GET_METADATA_CACHE_REASON_FENCE_PROOF_UNAVAILABLE),
|
||||
Some(authority) if !authority.is_current() => Some(GET_METADATA_CACHE_REASON_FENCE_LOCK_LOST),
|
||||
Some(_) => None,
|
||||
};
|
||||
}
|
||||
None
|
||||
self.ctx
|
||||
.is_dist_erasure()
|
||||
.await
|
||||
.then_some(GET_METADATA_CACHE_REASON_DIST_ERASURE)
|
||||
}
|
||||
|
||||
#[allow(dead_code, reason = "asserted by this file's tests (backlog#1823)")]
|
||||
async fn cached_get_object_fileinfo(&self, bucket: &str, object: &str) -> Option<Arc<GetObjectMetadataCacheEntry>> {
|
||||
match self.lookup_cached_get_object_fileinfo(bucket, object).await {
|
||||
MetadataCacheLookup::Hit(entry) => Some(entry),
|
||||
MetadataCacheLookup::Miss
|
||||
| MetadataCacheLookup::RejectedInsufficientQuorum
|
||||
| MetadataCacheLookup::RejectedFenceProof => None,
|
||||
MetadataCacheLookup::Miss | MetadataCacheLookup::RejectedInsufficientQuorum => None,
|
||||
}
|
||||
}
|
||||
|
||||
async fn lookup_cached_get_object_fileinfo(&self, bucket: &str, object: &str) -> MetadataCacheLookup {
|
||||
self.lookup_cached_get_object_fileinfo_with_authority(bucket, object, None, || {})
|
||||
.await
|
||||
self.lookup_cached_get_object_fileinfo_after_get(bucket, object, || {}).await
|
||||
}
|
||||
|
||||
async fn lookup_cached_get_object_fileinfo_with_authority(
|
||||
async fn lookup_cached_get_object_fileinfo_after_get(
|
||||
&self,
|
||||
bucket: &str,
|
||||
object: &str,
|
||||
authority: Option<&MetadataCacheAuthority>,
|
||||
after_get: impl FnOnce(),
|
||||
) -> MetadataCacheLookup {
|
||||
let Some(generation) = self.get_object_metadata_cache_generation(bucket, object) else {
|
||||
@@ -531,12 +491,6 @@ impl SetDisks {
|
||||
self.get_object_metadata_cache.invalidate(&key).await;
|
||||
return MetadataCacheLookup::Miss;
|
||||
}
|
||||
if authority.is_some_and(|authority| !authority.is_current())
|
||||
|| entry.distributed_fence_proof.as_deref() != authority.map(|authority| authority.proof.as_ref())
|
||||
{
|
||||
self.get_object_metadata_cache.invalidate(&key).await;
|
||||
return MetadataCacheLookup::RejectedFenceProof;
|
||||
}
|
||||
if entry.online_disks.iter().filter(|disk| disk.is_some()).count() >= entry.read_quorum {
|
||||
MetadataCacheLookup::Hit(entry)
|
||||
} else {
|
||||
@@ -544,17 +498,6 @@ impl SetDisks {
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
async fn lookup_cached_get_object_fileinfo_after_get(
|
||||
&self,
|
||||
bucket: &str,
|
||||
object: &str,
|
||||
after_get: impl FnOnce(),
|
||||
) -> MetadataCacheLookup {
|
||||
self.lookup_cached_get_object_fileinfo_with_authority(bucket, object, None, after_get)
|
||||
.await
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
async fn cache_get_object_fileinfo(
|
||||
&self,
|
||||
@@ -570,12 +513,7 @@ impl SetDisks {
|
||||
.cache_get_object_fileinfo_owned(
|
||||
identity,
|
||||
generation,
|
||||
MetadataCachePublication {
|
||||
fi: fi.clone(),
|
||||
capability,
|
||||
verified_inline_body: None,
|
||||
authority: None,
|
||||
},
|
||||
(fi.clone(), capability, None),
|
||||
parts_metadata.to_vec(),
|
||||
online_disks.to_vec(),
|
||||
read_quorum,
|
||||
@@ -587,17 +525,12 @@ impl SetDisks {
|
||||
&self,
|
||||
identity: (&str, &str),
|
||||
generation: Option<GetObjectMetadataCacheGeneration>,
|
||||
publication: MetadataCachePublication,
|
||||
selected: (FileInfo, GetObjectMetadataCacheCapability, Option<Bytes>),
|
||||
parts_metadata: Vec<FileInfo>,
|
||||
online_disks: Vec<Option<DiskStore>>,
|
||||
read_quorum: usize,
|
||||
) -> GetObjectFileInfo {
|
||||
let MetadataCachePublication {
|
||||
fi,
|
||||
capability,
|
||||
verified_inline_body,
|
||||
authority,
|
||||
} = publication;
|
||||
let (fi, capability, verified_inline_body) = selected;
|
||||
if fi.deleted || !fi.has_valid_erasure_geometry() {
|
||||
let mut snapshot = GetObjectFileInfo::owned(fi, parts_metadata, online_disks);
|
||||
snapshot.set_verified_inline_body(verified_inline_body);
|
||||
@@ -626,38 +559,18 @@ impl SetDisks {
|
||||
read_quorum,
|
||||
capability,
|
||||
verified_inline_body,
|
||||
distributed_fence_proof: authority.as_ref().map(|authority| Arc::clone(&authority.proof)),
|
||||
});
|
||||
self.insert_get_object_metadata_cache_entry_with_authority_after_insert(
|
||||
key,
|
||||
generation,
|
||||
Arc::clone(&entry),
|
||||
authority,
|
||||
|| {},
|
||||
)
|
||||
.await;
|
||||
self.insert_get_object_metadata_cache_entry_after_insert(key, generation, Arc::clone(&entry), || {})
|
||||
.await;
|
||||
GetObjectFileInfo::owned_published(entry)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
async fn insert_get_object_metadata_cache_entry_after_insert(
|
||||
&self,
|
||||
key: GetObjectMetadataCacheKey,
|
||||
generation: GetObjectMetadataCacheGeneration,
|
||||
entry: Arc<GetObjectMetadataCacheEntry>,
|
||||
after_insert: impl FnOnce(),
|
||||
) {
|
||||
self.insert_get_object_metadata_cache_entry_with_authority_after_insert(key, generation, entry, None, after_insert)
|
||||
.await;
|
||||
}
|
||||
|
||||
async fn insert_get_object_metadata_cache_entry_with_authority_after_insert(
|
||||
&self,
|
||||
key: GetObjectMetadataCacheKey,
|
||||
generation: GetObjectMetadataCacheGeneration,
|
||||
entry: Arc<GetObjectMetadataCacheEntry>,
|
||||
authority: Option<MetadataCacheAuthority>,
|
||||
after_insert: impl FnOnce(),
|
||||
) {
|
||||
// Every publisher shares Moka's per-key compute lock. A partial snapshot
|
||||
// cannot downgrade a full layout or extend its original TTL.
|
||||
@@ -666,7 +579,6 @@ impl SetDisks {
|
||||
.and_compute_with(|current| {
|
||||
std::future::ready(
|
||||
if !self.is_get_object_metadata_cache_generation_current(generation)
|
||||
|| authority.as_ref().is_some_and(|authority| !authority.is_current())
|
||||
|| (entry.capability != GetObjectMetadataCacheCapability::Full
|
||||
&& current
|
||||
.as_ref()
|
||||
@@ -680,9 +592,7 @@ impl SetDisks {
|
||||
})
|
||||
.await;
|
||||
after_insert();
|
||||
if self.is_get_object_metadata_cache_generation_current(generation)
|
||||
&& !authority.as_ref().is_some_and(|authority| !authority.is_current())
|
||||
{
|
||||
if self.is_get_object_metadata_cache_generation_current(generation) {
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -930,9 +840,6 @@ impl SetDisks {
|
||||
let metadata_cache_lookup_start = get_stage_timer_if_enabled(stage_metrics_enabled);
|
||||
let cache_bypass_reason = self.get_object_metadata_cache_bypass_reason(bucket, opts, read_data).await;
|
||||
let use_metadata_cache = cache_bypass_reason.is_none();
|
||||
let metadata_cache_authority = use_metadata_cache
|
||||
.then(|| MetadataCacheAuthority::from_options(opts))
|
||||
.flatten();
|
||||
if let Some(reason) = cache_bypass_reason {
|
||||
rustfs_io_metrics::record_get_object_metadata_cache_decision(
|
||||
GET_OBJECT_PATH_SET_DISK,
|
||||
@@ -940,10 +847,7 @@ impl SetDisks {
|
||||
reason,
|
||||
);
|
||||
} else if vid.is_empty() {
|
||||
match self
|
||||
.lookup_cached_get_object_fileinfo_with_authority(bucket, object, metadata_cache_authority.as_ref(), || {})
|
||||
.await
|
||||
{
|
||||
match self.lookup_cached_get_object_fileinfo(bucket, object).await {
|
||||
MetadataCacheLookup::Hit(cached)
|
||||
if cached.capability == GetObjectMetadataCacheCapability::Full
|
||||
|| (cached.capability == GetObjectMetadataCacheCapability::QuorumSelected
|
||||
@@ -1000,13 +904,6 @@ impl SetDisks {
|
||||
GET_METADATA_CACHE_REASON_INSUFFICIENT_CACHED_QUORUM,
|
||||
);
|
||||
}
|
||||
MetadataCacheLookup::RejectedFenceProof => {
|
||||
rustfs_io_metrics::record_get_object_metadata_cache_decision(
|
||||
GET_OBJECT_PATH_SET_DISK,
|
||||
GET_METADATA_CACHE_DECISION_REJECT,
|
||||
GET_METADATA_CACHE_REASON_FENCE_PROOF_MISMATCH,
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
record_get_stage_duration_if_enabled(
|
||||
@@ -1207,9 +1104,9 @@ impl SetDisks {
|
||||
self.cache_get_object_fileinfo_owned(
|
||||
(bucket, object),
|
||||
metadata_cache_generation,
|
||||
MetadataCachePublication {
|
||||
(
|
||||
fi,
|
||||
capability: if metadata_fanout_complete {
|
||||
if metadata_fanout_complete {
|
||||
GetObjectMetadataCacheCapability::Full
|
||||
} else if condition_stopped {
|
||||
GetObjectMetadataCacheCapability::ConditionOnly
|
||||
@@ -1217,8 +1114,7 @@ impl SetDisks {
|
||||
GetObjectMetadataCacheCapability::QuorumSelected
|
||||
},
|
||||
verified_inline_body,
|
||||
authority: metadata_cache_authority,
|
||||
},
|
||||
),
|
||||
parts_metadata,
|
||||
op_online_disks,
|
||||
read_quorum,
|
||||
@@ -1476,7 +1372,12 @@ impl SetDisks {
|
||||
|
||||
let erasure = erasure_cache.get_for_file_info(fi)?;
|
||||
|
||||
let checksum_algo = part_checksum_algorithm(fi, part.number);
|
||||
let checksum_info = fi.erasure.get_checksum_info(part.number);
|
||||
let checksum_algo = if fi.uses_legacy_checksum && checksum_info.algorithm == HashAlgorithm::HighwayHash256S {
|
||||
HashAlgorithm::HighwayHash256SLegacy
|
||||
} else {
|
||||
checksum_info.algorithm
|
||||
};
|
||||
let read_length = erasure.shard_file_offset(0, object_size, object_size);
|
||||
|
||||
if fi.data.is_some() {
|
||||
@@ -1491,9 +1392,8 @@ impl SetDisks {
|
||||
};
|
||||
|
||||
let reader_setup_stage_start = Instant::now();
|
||||
let mut readers = build_inline_bitrot_readers(
|
||||
data_files.iter().copied(),
|
||||
data_files.len(),
|
||||
let mut readers = build_inline_bitrot_readers_from_refs(
|
||||
&data_files,
|
||||
bucket,
|
||||
object,
|
||||
read_length,
|
||||
@@ -1765,7 +1665,7 @@ impl SetDisks {
|
||||
"Streaming multipart part"
|
||||
);
|
||||
|
||||
let checksum_algo = part_checksum_algorithm(&fi, part_number);
|
||||
let checksum_algo = multipart_part_checksum_algo(&fi, part_number);
|
||||
let read_length = till_offset.saturating_sub(read_offset);
|
||||
|
||||
let read_costs = coding::decode::should_collect_shard_read_costs().then(|| shard_read_costs_for_disks(&disks));
|
||||
@@ -1829,7 +1729,7 @@ impl SetDisks {
|
||||
part_number: next_number,
|
||||
read_offset: 0,
|
||||
read_length: erasure.shard_file_offset(0, next_length, next_size),
|
||||
checksum_algo: part_checksum_algorithm(&fi, next_number),
|
||||
checksum_algo: multipart_part_checksum_algo(&fi, next_number),
|
||||
};
|
||||
let files = Arc::clone(&files);
|
||||
let disks = Arc::clone(&disks);
|
||||
@@ -2426,7 +2326,12 @@ impl SetDisks {
|
||||
if part_length > part_size {
|
||||
return Err(Error::other("codec streaming reader part length exceeds part size"));
|
||||
}
|
||||
let checksum_algo = part_checksum_algorithm(fi, part_number);
|
||||
let checksum_info = fi.erasure.get_checksum_info(part_number);
|
||||
let checksum_algo = if fi.uses_legacy_checksum && checksum_info.algorithm == HashAlgorithm::HighwayHash256S {
|
||||
HashAlgorithm::HighwayHash256SLegacy
|
||||
} else {
|
||||
checksum_info.algorithm
|
||||
};
|
||||
let use_mmap_read = object_mmap_read_enabled();
|
||||
let till_offset = erasure.shard_file_offset(part_offset, part_length, part_size);
|
||||
let read_offset = (part_offset / erasure.block_size) * erasure.shard_size();
|
||||
@@ -2715,6 +2620,17 @@ struct PartReaderSetupSpec {
|
||||
checksum_algo: HashAlgorithm,
|
||||
}
|
||||
|
||||
/// Resolve the bitrot checksum algorithm for one part, honoring the legacy
|
||||
/// HighwayHash flag.
|
||||
fn multipart_part_checksum_algo(fi: &FileInfo, part_number: usize) -> HashAlgorithm {
|
||||
let checksum_info = fi.erasure.get_checksum_info(part_number);
|
||||
if fi.uses_legacy_checksum && checksum_info.algorithm == HashAlgorithm::HighwayHash256S {
|
||||
HashAlgorithm::HighwayHash256SLegacy
|
||||
} else {
|
||||
checksum_info.algorithm
|
||||
}
|
||||
}
|
||||
|
||||
fn multipart_reader_setup_prefetch_enabled(policy: GetObjectReadPolicy) -> bool {
|
||||
policy.allows_multipart_setup_prefetch() && is_multipart_reader_setup_prefetch_enabled()
|
||||
}
|
||||
@@ -3224,7 +3140,6 @@ mod metadata_cache_tests {
|
||||
use super::*;
|
||||
use crate::set_disk::core::io_primitives::disk_call_counters;
|
||||
use rustfs_heal_contracts::heal_channel::HealAdmissionDropReason;
|
||||
use rustfs_lock::{GlobalLockManager, LocalClient, LockClient, NamespaceLock, ObjectKey};
|
||||
use rustfs_utils::path::object_name_disk_identity;
|
||||
use serial_test::serial;
|
||||
use std::sync::atomic::{AtomicUsize, Ordering};
|
||||
@@ -4512,7 +4427,6 @@ mod metadata_cache_tests {
|
||||
online_disks: vec![None],
|
||||
read_quorum: 1,
|
||||
verified_inline_body: None,
|
||||
distributed_fence_proof: None,
|
||||
}),
|
||||
)
|
||||
.await;
|
||||
@@ -5175,7 +5089,6 @@ mod metadata_cache_tests {
|
||||
read_quorum: 0,
|
||||
capability,
|
||||
verified_inline_body: None,
|
||||
distributed_fence_proof: None,
|
||||
})
|
||||
}
|
||||
|
||||
@@ -5669,7 +5582,6 @@ mod metadata_cache_tests {
|
||||
online_disks: vec![None],
|
||||
read_quorum: 1,
|
||||
verified_inline_body: None,
|
||||
distributed_fence_proof: None,
|
||||
}),
|
||||
)
|
||||
.await;
|
||||
@@ -5789,7 +5701,6 @@ mod metadata_cache_tests {
|
||||
online_disks: Vec::new(),
|
||||
read_quorum: 0,
|
||||
verified_inline_body: None,
|
||||
distributed_fence_proof: None,
|
||||
});
|
||||
|
||||
set.insert_get_object_metadata_cache_entry_after_insert(key, generation, entry, || {
|
||||
@@ -5838,192 +5749,30 @@ mod metadata_cache_tests {
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
#[serial(metadata_cache_distributed_proof_env)]
|
||||
async fn metadata_cache_distribution_bypass_uses_set_instance_context() {
|
||||
temp_env::async_with_vars(
|
||||
[(ENV_RUSTFS_EXPERIMENTAL_GET_OBJECT_METADATA_CACHE_DISTRIBUTED_PROOF_ENABLE, None::<&str>)],
|
||||
async {
|
||||
let distributed_ctx = Arc::new(crate::runtime::instance::InstanceContext::new());
|
||||
distributed_ctx
|
||||
.update_erasure_type(crate::layout::endpoints::SetupType::DistErasure)
|
||||
.await;
|
||||
let standalone_ctx = Arc::new(crate::runtime::instance::InstanceContext::new());
|
||||
standalone_ctx
|
||||
.update_erasure_type(crate::layout::endpoints::SetupType::Erasure)
|
||||
.await;
|
||||
let distributed = new_metadata_cache_test_set_with_ctx(distributed_ctx).await;
|
||||
let standalone = new_metadata_cache_test_set_with_ctx(standalone_ctx).await;
|
||||
let distributed_ctx = Arc::new(crate::runtime::instance::InstanceContext::new());
|
||||
distributed_ctx
|
||||
.update_erasure_type(crate::layout::endpoints::SetupType::DistErasure)
|
||||
.await;
|
||||
let standalone_ctx = Arc::new(crate::runtime::instance::InstanceContext::new());
|
||||
standalone_ctx
|
||||
.update_erasure_type(crate::layout::endpoints::SetupType::Erasure)
|
||||
.await;
|
||||
let distributed = new_metadata_cache_test_set_with_ctx(distributed_ctx).await;
|
||||
let standalone = new_metadata_cache_test_set_with_ctx(standalone_ctx).await;
|
||||
|
||||
assert_eq!(
|
||||
distributed
|
||||
.get_object_metadata_cache_bypass_reason("bucket", &ObjectOptions::default(), true)
|
||||
.await,
|
||||
Some(GET_METADATA_CACHE_REASON_DIST_ERASURE)
|
||||
);
|
||||
assert_eq!(
|
||||
standalone
|
||||
.get_object_metadata_cache_bypass_reason("bucket", &ObjectOptions::default(), true)
|
||||
.await,
|
||||
None
|
||||
);
|
||||
},
|
||||
)
|
||||
.await;
|
||||
}
|
||||
|
||||
fn cache_fence_test_lock() -> NamespaceLock {
|
||||
let clients = (0..3)
|
||||
.map(|_| Arc::new(LocalClient::with_manager(Arc::new(GlobalLockManager::new()))) as Arc<dyn LockClient>)
|
||||
.collect();
|
||||
NamespaceLock::with_clients("metadata-cache-proof".to_owned(), clients)
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
#[serial(metadata_cache_distributed_proof_env)]
|
||||
async fn distributed_metadata_cache_requires_live_quorum_proof() {
|
||||
temp_env::async_with_vars(
|
||||
[(ENV_RUSTFS_EXPERIMENTAL_GET_OBJECT_METADATA_CACHE_DISTRIBUTED_PROOF_ENABLE, Some("true"))],
|
||||
async {
|
||||
let ctx = Arc::new(crate::runtime::instance::InstanceContext::new());
|
||||
ctx.update_erasure_type(crate::layout::endpoints::SetupType::DistErasure)
|
||||
.await;
|
||||
let set = new_metadata_cache_test_set_with_ctx(ctx).await;
|
||||
assert_eq!(
|
||||
set.get_object_metadata_cache_bypass_reason("bucket", &ObjectOptions::default(), true)
|
||||
.await,
|
||||
Some(GET_METADATA_CACHE_REASON_FENCE_PROOF_UNAVAILABLE)
|
||||
);
|
||||
|
||||
let lock = cache_fence_test_lock();
|
||||
let resource = ObjectKey::new("bucket", "object");
|
||||
let guard = lock
|
||||
.get_read_lock_with_cache_fence(resource, "reader", Duration::from_secs(1))
|
||||
.await
|
||||
.expect("proof-carrying read lock");
|
||||
let mut opts = ObjectOptions {
|
||||
no_lock: true,
|
||||
metadata_cache_safe: true,
|
||||
..Default::default()
|
||||
};
|
||||
opts.add_metadata_cache_read_guard(&guard);
|
||||
assert!(opts.metadata_cache_fence_proof.is_some());
|
||||
assert_eq!(set.get_object_metadata_cache_bypass_reason("bucket", &opts, true).await, None);
|
||||
|
||||
opts.namespace_lock_fence = Some(crate::object_api::NamespaceLockFence::lost_for_test());
|
||||
assert_eq!(
|
||||
set.get_object_metadata_cache_bypass_reason("bucket", &opts, true).await,
|
||||
Some(GET_METADATA_CACHE_REASON_FENCE_LOCK_LOST)
|
||||
);
|
||||
},
|
||||
)
|
||||
.await;
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn distributed_metadata_cache_rejects_proof_after_write_quorum() {
|
||||
let set = new_metadata_cache_test_set().await;
|
||||
let lock = cache_fence_test_lock();
|
||||
let resource = ObjectKey::new("bucket", "object");
|
||||
let mut first_guard = lock
|
||||
.get_read_lock_with_cache_fence(resource.clone(), "reader-a", Duration::from_secs(1))
|
||||
.await
|
||||
.expect("first proof-carrying read lock");
|
||||
let mut first_opts = ObjectOptions {
|
||||
no_lock: true,
|
||||
metadata_cache_safe: true,
|
||||
..Default::default()
|
||||
};
|
||||
first_opts.add_metadata_cache_read_guard(&first_guard);
|
||||
let authority = MetadataCacheAuthority::from_options(&first_opts).expect("first read authority");
|
||||
let fi = valid_test_fileinfo("object");
|
||||
let generation = set.get_object_metadata_cache_generation("bucket", "object");
|
||||
set.cache_get_object_fileinfo_owned(
|
||||
("bucket", "object"),
|
||||
generation,
|
||||
MetadataCachePublication {
|
||||
fi: fi.clone(),
|
||||
capability: GetObjectMetadataCacheCapability::Full,
|
||||
verified_inline_body: None,
|
||||
authority: Some(authority.clone()),
|
||||
},
|
||||
vec![fi],
|
||||
Vec::new(),
|
||||
0,
|
||||
)
|
||||
.await;
|
||||
assert!(matches!(
|
||||
set.lookup_cached_get_object_fileinfo_with_authority("bucket", "object", Some(&authority), || {})
|
||||
assert_eq!(
|
||||
distributed
|
||||
.get_object_metadata_cache_bypass_reason("bucket", &ObjectOptions::default(), true)
|
||||
.await,
|
||||
MetadataCacheLookup::Hit(_)
|
||||
));
|
||||
assert!(first_guard.release());
|
||||
|
||||
let mut writer = lock
|
||||
.get_write_lock(resource.clone(), "writer", Duration::from_secs(1))
|
||||
.await
|
||||
.expect("intervening write quorum");
|
||||
assert!(writer.release());
|
||||
|
||||
let mut second_guard = lock
|
||||
.get_read_lock_with_cache_fence(resource, "reader-b", Duration::from_secs(1))
|
||||
.await
|
||||
.expect("second proof-carrying read lock");
|
||||
let mut second_opts = ObjectOptions {
|
||||
no_lock: true,
|
||||
metadata_cache_safe: true,
|
||||
..Default::default()
|
||||
};
|
||||
second_opts.add_metadata_cache_read_guard(&second_guard);
|
||||
let second_authority = MetadataCacheAuthority::from_options(&second_opts).expect("second read authority");
|
||||
assert!(matches!(
|
||||
set.lookup_cached_get_object_fileinfo_with_authority("bucket", "object", Some(&second_authority), || {})
|
||||
.await,
|
||||
MetadataCacheLookup::RejectedFenceProof
|
||||
));
|
||||
assert!(second_guard.release());
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn distributed_metadata_cache_does_not_publish_after_read_lock_loss() {
|
||||
let set = new_metadata_cache_test_set().await;
|
||||
let lock = cache_fence_test_lock();
|
||||
let resource = ObjectKey::new("bucket", "lost-reader");
|
||||
let mut guard = lock
|
||||
.get_read_lock_with_cache_fence(resource, "reader", Duration::from_secs(1))
|
||||
.await
|
||||
.expect("proof-carrying read lock");
|
||||
let mut opts = ObjectOptions {
|
||||
no_lock: true,
|
||||
metadata_cache_safe: true,
|
||||
..Default::default()
|
||||
};
|
||||
opts.add_metadata_cache_read_guard(&guard);
|
||||
opts.namespace_lock_fence = Some(crate::object_api::NamespaceLockFence::lost_for_test());
|
||||
let authority = MetadataCacheAuthority::from_options(&opts).expect("lost authority retains its proof for rejection");
|
||||
let fi = valid_test_fileinfo("lost-reader");
|
||||
let generation = set
|
||||
.get_object_metadata_cache_generation("bucket", "lost-reader")
|
||||
.expect("active local generation");
|
||||
let key = GetObjectMetadataCacheKey::new("bucket", "lost-reader", generation);
|
||||
set.cache_get_object_fileinfo_owned(
|
||||
("bucket", "lost-reader"),
|
||||
Some(generation),
|
||||
MetadataCachePublication {
|
||||
fi: fi.clone(),
|
||||
capability: GetObjectMetadataCacheCapability::Full,
|
||||
verified_inline_body: None,
|
||||
authority: Some(authority),
|
||||
},
|
||||
vec![fi],
|
||||
Vec::new(),
|
||||
0,
|
||||
)
|
||||
.await;
|
||||
assert!(
|
||||
set.get_object_metadata_cache.get(&key).await.is_none(),
|
||||
"a lost read guard cannot publish a reusable entry"
|
||||
Some(GET_METADATA_CACHE_REASON_DIST_ERASURE)
|
||||
);
|
||||
assert_eq!(
|
||||
standalone
|
||||
.get_object_metadata_cache_bypass_reason("bucket", &ObjectOptions::default(), true)
|
||||
.await,
|
||||
None
|
||||
);
|
||||
assert!(guard.release());
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
@@ -6088,7 +5837,6 @@ mod metadata_cache_tests {
|
||||
online_disks: Vec::new(),
|
||||
read_quorum: 0,
|
||||
verified_inline_body: None,
|
||||
distributed_fence_proof: None,
|
||||
})
|
||||
};
|
||||
|
||||
|
||||
@@ -52,10 +52,9 @@ pub(crate) mod list {
|
||||
}
|
||||
|
||||
pub(crate) mod multipart {
|
||||
pub(crate) use rustfs_storage_api::multipart::ListPartsOwner;
|
||||
pub(crate) use rustfs_storage_api::{
|
||||
CompletePart, ListMultipartsInfo, ListPartsInfo, MultipartAbortAuthorization, MultipartCompletionAuthorization,
|
||||
MultipartInfo, MultipartOperations, MultipartOwner, MultipartUploadInspection, MultipartUploadResult, PartInfo,
|
||||
CompletePart, ListMultipartsInfo, ListPartsInfo, MultipartCompletionAuthorization, MultipartInfo, MultipartOperations,
|
||||
MultipartUploadInspection, MultipartUploadResult, PartInfo,
|
||||
};
|
||||
}
|
||||
|
||||
|
||||
@@ -6825,28 +6825,6 @@ mod tests {
|
||||
tokio::fs::remove_file(conflicting_metadata)
|
||||
.await
|
||||
.expect("remove conflicting fixture");
|
||||
struct RequiresOwner;
|
||||
impl crate::storage_api_contracts::multipart::MultipartAbortAuthorization for RequiresOwner {
|
||||
type Error = StorageError;
|
||||
|
||||
fn authorize(&self, _: Option<&crate::storage_api_contracts::multipart::MultipartOwner>) -> Result<()> {
|
||||
panic!("metadata-absent recovery cannot observe an upload owner");
|
||||
}
|
||||
}
|
||||
let native_opts = ObjectOptions {
|
||||
multipart_abort_authorization: Some(Arc::new(RequiresOwner)),
|
||||
..Default::default()
|
||||
};
|
||||
let rejected = store
|
||||
.abort_multipart_upload(&bucket, object, &upload.upload_id, &native_opts)
|
||||
.await;
|
||||
for disk_index in 0..3 {
|
||||
assert!(
|
||||
staging_dir(0, disk_index).join("part.1").exists(),
|
||||
"owner-required Abort must preserve late data"
|
||||
);
|
||||
}
|
||||
assert!(matches!(rejected, Err(StorageError::InvalidUploadID(..))));
|
||||
let (paused, release) = crate::store::multipart::pause_orphan_multipart_cleanup_for_test(&bucket);
|
||||
let recovery_store = store.clone();
|
||||
let recovery_bucket = bucket.clone();
|
||||
@@ -7124,6 +7102,7 @@ mod tests {
|
||||
.expect("create upload in active target pool");
|
||||
drop(lifecycle_guard);
|
||||
|
||||
mark_test_pool_decommissioning(&store, 0).await;
|
||||
// Corrupt only the source pool's entry for this UploadID. Taking the
|
||||
// whole source set offline would also make the bucket-incarnation
|
||||
// sidecar unreadable before multipart routing is reached.
|
||||
@@ -7146,17 +7125,6 @@ mod tests {
|
||||
.expect("inject source upload metadata read failure");
|
||||
}
|
||||
|
||||
let failed_binding = store
|
||||
.lookup_multipart_part_binding(&bucket, object, &upload.upload_id, &ObjectOptions::default())
|
||||
.await;
|
||||
assert!(
|
||||
failed_binding
|
||||
.as_ref()
|
||||
.is_err_and(|err| !crate::error::is_err_invalid_upload_id(err)),
|
||||
"a hard error in the first active pool must not be hidden by the later upload: {failed_binding:?}"
|
||||
);
|
||||
|
||||
mark_test_pool_decommissioning(&store, 0).await;
|
||||
let source_result = store.pools[0]
|
||||
.get_multipart_info(&bucket, object, &upload.upload_id, &ObjectOptions::default())
|
||||
.await;
|
||||
@@ -7172,13 +7140,6 @@ mod tests {
|
||||
);
|
||||
let routed = routed_result.expect("the active target UploadID must be resolved before the faulted suspended source");
|
||||
assert_eq!(routed.upload_id, upload.upload_id);
|
||||
let (binding, owner) = store
|
||||
.lookup_multipart_part_binding(&bucket, object, &upload.upload_id, &ObjectOptions::default())
|
||||
.await
|
||||
.expect("binding must prefer the active target before the faulted suspended source");
|
||||
assert_eq!(binding.upload_id, upload.upload_id);
|
||||
let owner = owner.expect("active target must have an owner");
|
||||
assert_eq!((owner.bucket.as_str(), owner.object.as_str()), (bucket.as_str(), object));
|
||||
|
||||
shutdown.cancel();
|
||||
}
|
||||
|
||||
@@ -17,7 +17,6 @@ use crate::core::pools::{DecommissionCapacityOwner, ensure_decommission_capacity
|
||||
use crate::multipart_listing::paginate_multipart_listing;
|
||||
use crate::set_disk::get_lock_acquire_timeout;
|
||||
use crate::storage_api_contracts::multipart::MultipartOperations as _;
|
||||
use crate::storage_api_contracts::multipart::MultipartOwner;
|
||||
use crate::storage_api_contracts::object::ObjectOperations as _;
|
||||
use futures::{StreamExt, stream};
|
||||
use rustfs_utils::path::encode_dir_object_ref;
|
||||
@@ -846,51 +845,6 @@ impl ECStore {
|
||||
.await
|
||||
}
|
||||
|
||||
/// Observe active upload metadata and its recorded owner from the same storage snapshot.
|
||||
///
|
||||
/// A missing or inconsistent recorded owner is returned as `None`. This observation
|
||||
/// does not reserve the upload or authorize a later write. The lookup is read-only;
|
||||
/// cancellation leaves storage unchanged, and callers may retry.
|
||||
///
|
||||
/// The caller must retain any supplied lifecycle guard. With `opts.no_lock`, the
|
||||
/// caller is responsible for the upload lock normally held during the observation.
|
||||
///
|
||||
/// # Errors
|
||||
/// Preserves argument, bucket lifetime, access, lock, I/O and quorum errors from
|
||||
/// the ordinary active-upload lookup.
|
||||
pub async fn lookup_multipart_part_binding(
|
||||
&self,
|
||||
bucket: &str,
|
||||
object: &str,
|
||||
upload_id: &str,
|
||||
opts: &ObjectOptions,
|
||||
) -> Result<(MultipartInfo, Option<MultipartOwner>)> {
|
||||
check_list_parts_args(bucket, object, upload_id)?;
|
||||
let encoded_object = encode_dir_object_ref(object);
|
||||
let object = encoded_object.as_ref();
|
||||
// Lock order: bucket lifecycle before the selected set's upload lock.
|
||||
let (opts, _bucket_lifecycle_guard) = self.guard_multipart_bucket_incarnation(bucket, opts).await?;
|
||||
if self.single_pool() {
|
||||
return self.pools[0]
|
||||
.get_disks_by_key(object)
|
||||
.lookup_multipart_part_binding(bucket, object, upload_id, &opts)
|
||||
.await;
|
||||
}
|
||||
|
||||
for pool_idx in self.existing_multipart_pool_order().await {
|
||||
match self.pools[pool_idx]
|
||||
.get_disks_by_key(object)
|
||||
.lookup_multipart_part_binding(bucket, object, upload_id, &opts)
|
||||
.await
|
||||
{
|
||||
Ok(binding) => return Ok(binding),
|
||||
Err(err) if is_err_invalid_upload_id(&err) => continue,
|
||||
Err(err) => return Err(err),
|
||||
}
|
||||
}
|
||||
Err(StorageError::InvalidUploadID(bucket.to_owned(), object.to_owned(), upload_id.to_owned()))
|
||||
}
|
||||
|
||||
/// Observe an upload's original namespace without recovering or acknowledging it.
|
||||
///
|
||||
/// Only ordinary active metadata is returned for application preflight. This
|
||||
@@ -1012,8 +966,7 @@ impl ECStore {
|
||||
bucket_lifecycle_guard: Option<super::BucketIncarnationFenceGuard>,
|
||||
) -> Result<()> {
|
||||
let invalid_upload = || StorageError::InvalidUploadID(bucket.to_owned(), object.to_owned(), upload_id.to_owned());
|
||||
// Without upload metadata this path has no recorded owner to authorize.
|
||||
if opts.multipart_abort_authorization.is_some() || self.ctx.lock_manager().is_disabled() {
|
||||
if self.ctx.lock_manager().is_disabled() {
|
||||
return Err(invalid_upload());
|
||||
}
|
||||
let Some(bucket_lifecycle_guard) = bucket_lifecycle_guard else {
|
||||
@@ -1298,7 +1251,6 @@ impl ECStore {
|
||||
lookup_opts.no_lock = true;
|
||||
let oi = if require_all_pools {
|
||||
lookup_opts.metadata_cache_safe = false;
|
||||
lookup_opts.metadata_cache_fence_proof = None;
|
||||
lookup_opts.suppress_read_repair = true;
|
||||
// Only completion uses this branch, after validating bucket and object.
|
||||
let encoded_object = encode_dir_object_ref(object);
|
||||
@@ -1668,62 +1620,6 @@ mod tests {
|
||||
.await
|
||||
.expect("old bucket incarnations must not affect placement")
|
||||
);
|
||||
|
||||
let directory = "binding directory/";
|
||||
let encoded = encode_dir_object(directory);
|
||||
let upload = store.pools[1]
|
||||
.new_multipart_upload(bucket, &encoded, &opts)
|
||||
.await
|
||||
.expect("create directory upload in second pool");
|
||||
let object_lock = store
|
||||
.handle_new_ns_lock(bucket, &encoded)
|
||||
.await
|
||||
.expect("create object namespace lock");
|
||||
let _object_writer = object_lock
|
||||
.get_write_lock(Duration::from_secs(5))
|
||||
.await
|
||||
.expect("hold the object write lock");
|
||||
let (info, owner) = tokio::time::timeout(
|
||||
Duration::from_secs(5),
|
||||
store.lookup_multipart_part_binding(bucket, directory, &upload.upload_id, &opts),
|
||||
)
|
||||
.await
|
||||
.expect("binding must not acquire the object lock")
|
||||
.expect("find the directory upload in the second pool");
|
||||
assert_eq!((info.bucket.as_str(), info.object.as_str()), (bucket, directory));
|
||||
assert_eq!(info.upload_id, upload.upload_id);
|
||||
let owner = owner.expect("owner from the active upload");
|
||||
assert_eq!((owner.bucket.as_str(), owner.object.as_str()), (bucket, directory));
|
||||
|
||||
let mut lost_opts = opts.clone();
|
||||
lost_opts.bucket_lifecycle_lock_fence = Some(crate::object_api::NamespaceLockFence::lost_for_test());
|
||||
assert!(matches!(
|
||||
store.pools[1]
|
||||
.get_disks_by_key(&encoded)
|
||||
.lookup_multipart_part_binding(bucket, &encoded, &upload.upload_id, &lost_opts)
|
||||
.await,
|
||||
Err(StorageError::NamespaceLockQuorumUnavailable { .. })
|
||||
));
|
||||
let mut stale_opts = opts.clone();
|
||||
stale_opts.expected_bucket_incarnation_id = Some(Uuid::new_v4());
|
||||
assert!(matches!(
|
||||
store
|
||||
.lookup_multipart_part_binding(bucket, directory, &upload.upload_id, &stale_opts)
|
||||
.await,
|
||||
Err(StorageError::BucketNotFound(_))
|
||||
));
|
||||
assert!(matches!(
|
||||
store
|
||||
.lookup_multipart_part_binding(bucket, directory, "/invalid", &opts)
|
||||
.await,
|
||||
Err(StorageError::MalformedUploadID(_))
|
||||
));
|
||||
assert!(matches!(
|
||||
store
|
||||
.lookup_multipart_part_binding(bucket, "different/", &upload.upload_id, &opts)
|
||||
.await,
|
||||
Err(StorageError::InvalidUploadID(..))
|
||||
));
|
||||
cancel.cancel();
|
||||
}
|
||||
|
||||
|
||||
@@ -3124,7 +3124,6 @@ impl ECStore {
|
||||
let mut lookup_opts = opts.clone();
|
||||
lookup_opts.no_lock = true;
|
||||
lookup_opts.metadata_cache_safe = false;
|
||||
lookup_opts.metadata_cache_fence_proof = None;
|
||||
lookup_opts.suppress_read_repair = true;
|
||||
lookup_opts.http_preconditions = None;
|
||||
lookup_opts.version_id = None;
|
||||
@@ -3313,7 +3312,6 @@ impl ECStore {
|
||||
// (or another SetDisks owner) can commit without invalidating this cache.
|
||||
let mut opts = opts.clone();
|
||||
opts.metadata_cache_safe = false;
|
||||
opts.metadata_cache_fence_proof = None;
|
||||
Ok(if self.single_pool() {
|
||||
let pool = Arc::clone(&self.pools[0]);
|
||||
let metadata = pool.prepare_get_object_reader_metadata(bucket, object, &opts).await?;
|
||||
@@ -3353,7 +3351,6 @@ impl ECStore {
|
||||
let mut opts = opts.clone();
|
||||
opts.no_lock = false;
|
||||
opts.metadata_cache_safe = false;
|
||||
opts.metadata_cache_fence_proof = None;
|
||||
let read_lock_guards = self
|
||||
.acquire_all_object_read_locks("select_object", bucket, &object, &mut opts)
|
||||
.await?;
|
||||
@@ -3671,13 +3668,7 @@ impl ECStore {
|
||||
let acquire_start = Instant::now();
|
||||
let timeout = get_lock_acquire_timeout();
|
||||
let mut attempt = ObjectLockAttempt::start(op, bucket, object, opts.version_id.as_deref(), &ns_lock, "read", timeout);
|
||||
let distributed_cache_proof =
|
||||
crate::set_disk::is_get_object_metadata_cache_distributed_proof_enabled() && self.ctx.is_dist_erasure().await;
|
||||
let result = if distributed_cache_proof {
|
||||
ns_lock.get_read_lock_with_cache_fence(timeout).await
|
||||
} else {
|
||||
ns_lock.get_read_lock(timeout).await
|
||||
};
|
||||
let result = ns_lock.get_read_lock(timeout).await;
|
||||
attempt.observe(&result);
|
||||
let guard = result.map_err(|err| Self::map_namespace_lock_error(bucket, object, "read", err))?;
|
||||
let owner = diag_enabled.then(|| ns_lock.owner().to_string());
|
||||
@@ -3692,9 +3683,6 @@ impl ECStore {
|
||||
);
|
||||
opts.no_lock = true;
|
||||
opts.metadata_cache_safe = true;
|
||||
if distributed_cache_proof {
|
||||
opts.add_metadata_cache_read_guard(&guard);
|
||||
}
|
||||
|
||||
Ok(Some(
|
||||
ObjectLockDiagGuard::new(
|
||||
|
||||
@@ -1,47 +0,0 @@
|
||||
// Copyright 2024 RustFS Team
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
use rustfs_filemeta::{FileInfoOpts, get_file_info};
|
||||
use std::{env, fs, io};
|
||||
|
||||
fn main() -> Result<(), Box<dyn std::error::Error>> {
|
||||
let path = env::args()
|
||||
.nth(1)
|
||||
.ok_or_else(|| io::Error::new(io::ErrorKind::InvalidInput, "usage: inline_lab_inspect <xl.meta> [version]"))?;
|
||||
let version = env::args().nth(2).unwrap_or_default();
|
||||
let data = fs::read(path)?;
|
||||
let info = get_file_info(
|
||||
&data,
|
||||
"validation-bucket",
|
||||
"validation-object",
|
||||
&version,
|
||||
FileInfoOpts {
|
||||
data: false,
|
||||
include_free_versions: false,
|
||||
include_part_checksums: false,
|
||||
},
|
||||
)?;
|
||||
println!(
|
||||
"{}",
|
||||
serde_json::json!({
|
||||
"size": info.size,
|
||||
"block_size": info.erasure.block_size,
|
||||
"data_blocks": info.erasure.data_blocks,
|
||||
"parity_blocks": info.erasure.parity_blocks,
|
||||
"inline": info.inline_data(),
|
||||
"version_id": info.version_id,
|
||||
})
|
||||
);
|
||||
Ok(())
|
||||
}
|
||||
@@ -822,11 +822,6 @@ fn resume_admission_cancel_returns_node_but_native_keeps_physical_and_memory() {
|
||||
.unwrap()
|
||||
.set_test_capacity(vec![physical.clone()])
|
||||
.expect("explicit native physical capacity");
|
||||
// Building the real ECStore fixture can outlive the node admission
|
||||
// sample's three-second lease. Refresh the same owner immediately
|
||||
// before starting the producer so this test reaches the native-open
|
||||
// cancellation boundary instead of testing stale-sample deferral.
|
||||
publish_resume_node(&node, true);
|
||||
let mut pause = LocalPublicationPause::install(
|
||||
fixture.disk.as_ref(),
|
||||
crate::heal::RUSTFS_META_BUCKET,
|
||||
|
||||
@@ -454,7 +454,6 @@ const INTERNODE_OPERATION_LARGE_PAYLOADS_TOTAL: &str = "rustfs_system_network_in
|
||||
const INTERNODE_MSGPACK_JSON_DECODE_TOTAL: &str = "rustfs_system_network_internode_msgpack_json_decode_total";
|
||||
const INTERNODE_MSGPACK_JSON_FALLBACK_TOTAL: &str = "rustfs_system_network_internode_msgpack_json_fallback_total";
|
||||
const INTERNODE_MSGPACK_JSON_DECODE_ERROR_TOTAL: &str = "rustfs_system_network_internode_msgpack_json_decode_error_total";
|
||||
const INTERNODE_MSGPACK_SEND_DECISIONS_TOTAL: &str = "rustfs_system_network_internode_msgpack_send_decisions_total";
|
||||
const INTERNODE_SIGNATURE_V1_FALLBACK_TOTAL: &str = "rustfs_system_network_internode_signature_v1_fallback_total";
|
||||
const INTERNODE_BODY_DIGEST_FALLBACK_TOTAL: &str = "rustfs_system_network_internode_body_digest_fallback_total";
|
||||
|
||||
@@ -1605,25 +1604,6 @@ impl InternodeMetrics {
|
||||
.increment(1);
|
||||
}
|
||||
|
||||
/// Record one eligible request's fixed-cardinality JSON/msgpack send decision.
|
||||
pub fn record_msgpack_send_decision(&self, decision: &'static str, reason: &'static str) {
|
||||
let decision = match decision {
|
||||
"msgpack_only" | "dual_write" => decision,
|
||||
_ => "dual_write",
|
||||
};
|
||||
let reason = match reason {
|
||||
"manual_fleet" | "peer_capable" | "peer_unknown" | "peer_revoked" | "not_requested" => reason,
|
||||
_ => "peer_unknown",
|
||||
};
|
||||
counter!(
|
||||
INTERNODE_MSGPACK_SEND_DECISIONS_TOTAL,
|
||||
SERVER_LABEL => current_server_label(),
|
||||
"decision" => decision,
|
||||
REASON_LABEL => reason
|
||||
)
|
||||
.increment(1);
|
||||
}
|
||||
|
||||
#[doc(hidden)]
|
||||
pub fn msgpack_json_decode_error_total_for_test(&self) -> u64 {
|
||||
self.msgpack_json_decode_error_total.load(Ordering::Relaxed)
|
||||
@@ -2720,52 +2700,6 @@ mod tests {
|
||||
assert!(observed.contains(&(INTERNODE_MSGPACK_DIRECTION_RESPONSE.to_string(), "RawFileInfo".to_string(), 1)));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn msgpack_send_decision_counter_keeps_fixed_decision_and_reason_labels() {
|
||||
let recorder = DebuggingRecorder::new();
|
||||
let snapshotter = recorder.snapshotter();
|
||||
let metrics = InternodeMetrics::default();
|
||||
metrics::with_local_recorder(&recorder, || {
|
||||
for (decision, reason) in [
|
||||
("msgpack_only", "manual_fleet"),
|
||||
("msgpack_only", "peer_capable"),
|
||||
("dual_write", "peer_unknown"),
|
||||
("dual_write", "peer_revoked"),
|
||||
("dual_write", "not_requested"),
|
||||
] {
|
||||
metrics.record_msgpack_send_decision(decision, reason);
|
||||
}
|
||||
metrics.record_msgpack_send_decision("untrusted", "untrusted");
|
||||
});
|
||||
|
||||
let observed: HashMap<(String, String), u64> = snapshotter
|
||||
.snapshot()
|
||||
.into_vec()
|
||||
.into_iter()
|
||||
.filter(|(composite, _, _, _)| composite.key().name() == INTERNODE_MSGPACK_SEND_DECISIONS_TOTAL)
|
||||
.map(|(composite, _, _, value)| {
|
||||
let labels: HashMap<String, String> = composite
|
||||
.key()
|
||||
.labels()
|
||||
.map(|label| (label.key().to_string(), label.value().to_string()))
|
||||
.collect();
|
||||
let count = match value {
|
||||
DebugValue::Counter(count) => count,
|
||||
other => panic!("msgpack send decision must be a counter, got {other:?}"),
|
||||
};
|
||||
(
|
||||
(
|
||||
labels.get("decision").cloned().unwrap_or_default(),
|
||||
labels.get(REASON_LABEL).cloned().unwrap_or_default(),
|
||||
),
|
||||
count,
|
||||
)
|
||||
})
|
||||
.collect();
|
||||
assert_eq!(observed.len(), 5);
|
||||
assert_eq!(observed.get(&("dual_write".to_string(), "peer_unknown".to_string())), Some(&2));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn msgpack_json_decode_counter_tracks_codec_direction_and_message() {
|
||||
let metrics = InternodeMetrics::default();
|
||||
|
||||
@@ -433,7 +433,6 @@ pub const PUT_COMMIT_LOCK_ADMISSION_OUTCOME_LOCK_ERROR: &str = "lock_error";
|
||||
|
||||
pub const PUT_RENAME_FDATASYNC_BATCH_MODE_SERIAL: &str = "serial";
|
||||
pub const PUT_RENAME_FDATASYNC_BATCH_MODE_PARALLEL: &str = "parallel";
|
||||
pub const PUT_RENAME_FDATASYNC_BATCH_MODE_GROUPED: &str = "grouped";
|
||||
pub const PUT_RENAME_FDATASYNC_GROUP_WAIT_ROLE_LEADER: &str = "leader";
|
||||
pub const PUT_RENAME_FDATASYNC_GROUP_WAIT_ROLE_FOLLOWER: &str = "follower";
|
||||
pub const PUT_RENAME_FDATASYNC_GROUP_WAIT_ROLE_FOLLOWER_TIMEOUT: &str = "follower_timeout";
|
||||
@@ -4431,7 +4430,6 @@ mod tests {
|
||||
|
||||
set_put_stage_metrics_enabled(true);
|
||||
record_put_rename_fdatasync_batch(PUT_RENAME_FDATASYNC_BATCH_MODE_PARALLEL, 9);
|
||||
record_put_rename_fdatasync_batch(PUT_RENAME_FDATASYNC_BATCH_MODE_GROUPED, 4);
|
||||
record_put_rename_fdatasync_group_wait(PUT_RENAME_FDATASYNC_GROUP_WAIT_ROLE_LEADER, 1.0);
|
||||
record_put_rename_fdatasync_group_wait(PUT_RENAME_FDATASYNC_GROUP_WAIT_ROLE_FOLLOWER_TIMEOUT, 3.0);
|
||||
record_put_rename_fdatasync_group_wait(PUT_RENAME_FDATASYNC_GROUP_WAIT_ROLE_FOLLOWER, 2.0);
|
||||
@@ -4447,10 +4445,7 @@ mod tests {
|
||||
});
|
||||
|
||||
let rows = snapshotter.snapshot().into_vec();
|
||||
assert_eq!(
|
||||
histogram_samples(&rows, "rustfs_s3_put_object_rename_fdatasync_batch_files"),
|
||||
vec![4.0, 9.0]
|
||||
);
|
||||
assert_eq!(histogram_samples(&rows, "rustfs_s3_put_object_rename_fdatasync_batch_files"), vec![9.0]);
|
||||
let batch_modes = rows
|
||||
.iter()
|
||||
.filter(|(composite, _, _, _)| {
|
||||
@@ -4466,13 +4461,7 @@ mod tests {
|
||||
.collect::<Vec<_>>()
|
||||
})
|
||||
.collect::<HashSet<_>>();
|
||||
assert_eq!(
|
||||
batch_modes,
|
||||
HashSet::from([
|
||||
PUT_RENAME_FDATASYNC_BATCH_MODE_GROUPED.to_string(),
|
||||
PUT_RENAME_FDATASYNC_BATCH_MODE_PARALLEL.to_string(),
|
||||
])
|
||||
);
|
||||
assert_eq!(batch_modes, HashSet::from([PUT_RENAME_FDATASYNC_BATCH_MODE_PARALLEL.to_string()]));
|
||||
|
||||
let quorum_samples = histogram_samples(&rows, "rustfs_s3_put_object_rename_quorum_wait_fanout_disks");
|
||||
assert_eq!(quorum_samples, vec![0.0, 1.0, 3.0, 3.0, 4.0]);
|
||||
|
||||
@@ -74,7 +74,7 @@ pub use client::KeystoneClient;
|
||||
pub use config::{KeystoneConfig, RoleMapping};
|
||||
pub use error::{KeystoneError, Result};
|
||||
pub use identity::KeystoneIdentityMapper;
|
||||
pub use middleware::{KEYSTONE_CREDENTIALS, KeystoneAuthLayer, KeystoneRequestIdentity, KeystoneRequestNotValidated};
|
||||
pub use middleware::{KEYSTONE_CREDENTIALS, KeystoneAuthLayer};
|
||||
|
||||
/// Keystone API version
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
|
||||
@@ -16,8 +16,8 @@
|
||||
//!
|
||||
//! This middleware intercepts HTTP requests and checks for OpenStack Keystone
|
||||
//! authentication headers (X-Auth-Token or Swift X-Storage-Token). If found, it validates the token
|
||||
//! with Keystone. The legacy edge receives credentials in task-local storage; the gateway
|
||||
//! receives a single-use request identity containing the proof and sanitized authorization facts.
|
||||
//! with Keystone and stores the authenticated credentials in task-local storage
|
||||
//! for use by downstream authentication handlers.
|
||||
//!
|
||||
//! ## Authentication Flow
|
||||
//!
|
||||
@@ -42,12 +42,12 @@ use http_body::Body;
|
||||
use http_body_util::{BodyExt, Full};
|
||||
use rustfs_credentials::Credentials;
|
||||
use std::pin::Pin;
|
||||
use std::sync::{Arc, Mutex};
|
||||
use std::sync::Arc;
|
||||
use std::task::{Context, Poll};
|
||||
use tower::{Layer, Service};
|
||||
use tracing::{debug, info, warn};
|
||||
|
||||
use crate::{KeystoneAuthProvider, ValidatedKeystoneToken};
|
||||
use crate::KeystoneAuthProvider;
|
||||
|
||||
const HEADER_X_AUTH_TOKEN: &str = "X-Auth-Token";
|
||||
const HEADER_X_STORAGE_TOKEN: &str = "X-Storage-Token";
|
||||
@@ -59,58 +59,6 @@ tokio::task_local! {
|
||||
pub static KEYSTONE_CREDENTIALS: Option<Credentials>;
|
||||
}
|
||||
|
||||
/// A request-local, single-use result of one Keystone validation. Only this middleware can
|
||||
/// create it. Clones share the same consumption slot and contain no token or secret key.
|
||||
#[derive(Clone)]
|
||||
pub struct KeystoneRequestIdentity(Arc<Mutex<Option<KeystoneRequestValidation>>>);
|
||||
|
||||
struct KeystoneRequestValidation {
|
||||
provider: Arc<KeystoneAuthProvider>,
|
||||
proof: ValidatedKeystoneToken,
|
||||
credentials: Credentials,
|
||||
had_session_token: bool,
|
||||
}
|
||||
|
||||
/// A missing, consumed, poisoned or differently configured request identity.
|
||||
#[derive(Debug, thiserror::Error)]
|
||||
#[error("the request has no matching Keystone validation")]
|
||||
pub struct KeystoneRequestNotValidated;
|
||||
|
||||
impl KeystoneRequestIdentity {
|
||||
fn new(provider: Arc<KeystoneAuthProvider>, proof: ValidatedKeystoneToken, mut credentials: Credentials) -> Self {
|
||||
let had_session_token = !credentials.session_token.is_empty();
|
||||
credentials.secret_key.clear();
|
||||
credentials.session_token.clear();
|
||||
Self(Arc::new(Mutex::new(Some(KeystoneRequestValidation {
|
||||
provider,
|
||||
proof,
|
||||
credentials,
|
||||
had_session_token,
|
||||
}))))
|
||||
}
|
||||
|
||||
/// Consumes the paired proof and sanitized credential once, only for the validating provider.
|
||||
/// The boolean records whether the original credential had a session token.
|
||||
///
|
||||
/// # Errors
|
||||
/// Returns [`KeystoneRequestNotValidated`] for a different provider, a consumed slot or poison.
|
||||
pub fn take_for(
|
||||
self,
|
||||
provider: &Arc<KeystoneAuthProvider>,
|
||||
) -> Result<(ValidatedKeystoneToken, Credentials, bool), KeystoneRequestNotValidated> {
|
||||
let value = self
|
||||
.0
|
||||
.lock()
|
||||
.map_err(|_| KeystoneRequestNotValidated)?
|
||||
.take()
|
||||
.ok_or(KeystoneRequestNotValidated)?;
|
||||
if !Arc::ptr_eq(provider, &value.provider) {
|
||||
return Err(KeystoneRequestNotValidated);
|
||||
}
|
||||
Ok((value.proof, value.credentials, value.had_session_token))
|
||||
}
|
||||
}
|
||||
|
||||
/// Tower Layer for Keystone authentication
|
||||
///
|
||||
/// This layer wraps services with Keystone authentication middleware.
|
||||
@@ -177,8 +125,7 @@ where
|
||||
self.inner.poll_ready(cx)
|
||||
}
|
||||
|
||||
fn call(&mut self, mut req: Request<ReqBody>) -> Self::Future {
|
||||
req.extensions_mut().remove::<KeystoneRequestIdentity>();
|
||||
fn call(&mut self, req: Request<ReqBody>) -> Self::Future {
|
||||
let keystone_auth = self.keystone_auth.clone();
|
||||
let mut inner = self.inner.clone();
|
||||
|
||||
@@ -189,7 +136,7 @@ where
|
||||
None => {
|
||||
// No Keystone configured, pass through to normal authentication
|
||||
debug!("Keystone middleware: No provider configured, passing through");
|
||||
let resp = KEYSTONE_CREDENTIALS.scope(None, async move { inner.call(req).await }).await?;
|
||||
let resp = inner.call(req).await?;
|
||||
let (parts, body) = resp.into_parts();
|
||||
let body: BoxBody = body.map_err(Into::into).boxed_unsync();
|
||||
return Ok(Response::from_parts(parts, body));
|
||||
@@ -202,22 +149,14 @@ where
|
||||
debug!("Keystone middleware: Found Keystone token header, validating");
|
||||
|
||||
// Validate token with Keystone
|
||||
match keystone_auth.authenticate_with_token_and_proof(token).await {
|
||||
Ok((credentials, proof)) => {
|
||||
match keystone_auth.authenticate_with_token(token).await {
|
||||
Ok(credentials) => {
|
||||
// Authentication successful!
|
||||
info!("Keystone middleware: Authentication successful for user: {}", credentials.parent_user);
|
||||
|
||||
req.extensions_mut().insert(KeystoneRequestIdentity::new(
|
||||
Arc::clone(&keystone_auth),
|
||||
proof,
|
||||
credentials.clone(),
|
||||
));
|
||||
|
||||
// Store credentials in task-local storage and continue processing
|
||||
// The auth handlers will retrieve these credentials when needed
|
||||
let resp = KEYSTONE_CREDENTIALS
|
||||
.scope(Some(credentials), async move { inner.call(req).await })
|
||||
.await?;
|
||||
let resp = KEYSTONE_CREDENTIALS.scope(Some(credentials), inner.call(req)).await?;
|
||||
let (parts, body) = resp.into_parts();
|
||||
let body: BoxBody = body.map_err(Into::into).boxed_unsync();
|
||||
return Ok(Response::from_parts(parts, body));
|
||||
@@ -259,7 +198,7 @@ where
|
||||
|
||||
// No Keystone token header present, pass through to normal S3 authentication
|
||||
debug!("Keystone middleware: No Keystone token header, passing through to S3 auth");
|
||||
let resp = KEYSTONE_CREDENTIALS.scope(None, async move { inner.call(req).await }).await?;
|
||||
let resp = inner.call(req).await?;
|
||||
let (parts, body) = resp.into_parts();
|
||||
let body: BoxBody = body.map_err(Into::into).boxed_unsync();
|
||||
Ok(Response::from_parts(parts, body))
|
||||
@@ -289,113 +228,6 @@ mod tests {
|
||||
use crate::{KeystoneClient, KeystoneVersion};
|
||||
use std::time::Duration;
|
||||
|
||||
#[tokio::test]
|
||||
async fn request_identity_erases_secrets_and_consumes_its_proof_once() {
|
||||
use std::io::{Read, Write};
|
||||
use std::net::TcpListener;
|
||||
let listener = TcpListener::bind("127.0.0.1:0").expect("bind Keystone stub");
|
||||
let endpoint = format!("http://{}", listener.local_addr().expect("stub address"));
|
||||
let server = std::thread::spawn(move || {
|
||||
let (mut socket, _) = listener.accept().expect("validation request");
|
||||
socket.set_read_timeout(Some(Duration::from_secs(5))).expect("stub timeout");
|
||||
let mut request = [0; 4096];
|
||||
assert!(socket.read(&mut request).expect("read validation request") > 0);
|
||||
let expiry = (time::OffsetDateTime::now_utc() + time::Duration::hours(1))
|
||||
.format(&time::format_description::well_known::Rfc3339)
|
||||
.expect("expiry");
|
||||
let body = serde_json::json!({"token": {"user": {"id": "user-id", "name": "user-name"}, "roles": [{"name": "member"}], "expires_at": expiry}}).to_string();
|
||||
write!(
|
||||
socket,
|
||||
"HTTP/1.1 200 OK\r\nContent-Type: application/json\r\nContent-Length: {}\r\nConnection: close\r\n\r\n{body}",
|
||||
body.len()
|
||||
)
|
||||
.expect("validation response");
|
||||
});
|
||||
let client = KeystoneClient::new(
|
||||
endpoint,
|
||||
KeystoneVersion::V3,
|
||||
None,
|
||||
None,
|
||||
None,
|
||||
"Default".into(),
|
||||
true,
|
||||
Duration::from_secs(5),
|
||||
);
|
||||
let provider = Arc::new(KeystoneAuthProvider::new(client, 1, Duration::from_secs(1), false));
|
||||
let (mut credentials, proof) = provider
|
||||
.authenticate_with_token_and_proof("request-token")
|
||||
.await
|
||||
.expect("validated token");
|
||||
server.join().expect("stub answered");
|
||||
credentials.secret_key = "original-secret".into();
|
||||
credentials.session_token = "original-session".into();
|
||||
let original = credentials.clone();
|
||||
let carrier = KeystoneRequestIdentity::new(Arc::clone(&provider), proof, credentials);
|
||||
let clone = carrier.clone();
|
||||
let (proof, sanitized, had_session) = carrier.take_for(&provider).expect("first consumer");
|
||||
let mut failures = Vec::new();
|
||||
for (ok, fact) in [
|
||||
(proof.user_id() == "user-id", "proof identity"),
|
||||
(sanitized.secret_key.is_empty(), "secret erased"),
|
||||
(sanitized.session_token.is_empty(), "session erased"),
|
||||
(had_session, "original session presence"),
|
||||
(sanitized.access_key == original.access_key, "access key retained"),
|
||||
(sanitized.parent_user == original.parent_user, "parent retained"),
|
||||
(sanitized.expiration == original.expiration, "expiry retained"),
|
||||
(sanitized.groups == original.groups, "groups retained"),
|
||||
(sanitized.claims == original.claims, "claims retained"),
|
||||
(clone.take_for(&provider).is_err(), "single consumption"),
|
||||
] {
|
||||
if !ok {
|
||||
failures.push(fact);
|
||||
}
|
||||
}
|
||||
let poisoned = KeystoneRequestIdentity::new(Arc::clone(&provider), proof, sanitized);
|
||||
let slot = Arc::clone(&poisoned.0);
|
||||
assert!(
|
||||
std::thread::spawn(move || {
|
||||
let _guard = slot.lock().expect("unpoisoned slot");
|
||||
panic!("poison the request slot");
|
||||
})
|
||||
.join()
|
||||
.is_err()
|
||||
);
|
||||
if poisoned.take_for(&provider).is_ok() {
|
||||
failures.push("poison refusal");
|
||||
}
|
||||
assert!(failures.is_empty(), "{}", failures.join(", "));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn disabled_middleware_clears_identity_before_inner_call_and_poll() {
|
||||
use tower::ServiceExt;
|
||||
let service = tower::service_fn(|_request: Request<()>| {
|
||||
assert!(KEYSTONE_CREDENTIALS.try_with(Clone::clone).expect("call scope").is_none());
|
||||
async {
|
||||
assert!(KEYSTONE_CREDENTIALS.try_with(Clone::clone).expect("poll scope").is_none());
|
||||
Ok::<_, std::convert::Infallible>(Response::new(Full::new(Bytes::new())))
|
||||
}
|
||||
});
|
||||
let outer = Credentials {
|
||||
access_key: "outer-identity".into(),
|
||||
..Default::default()
|
||||
};
|
||||
KEYSTONE_CREDENTIALS
|
||||
.scope(Some(outer), async {
|
||||
let response = KeystoneAuthLayer::new(None)
|
||||
.layer(service)
|
||||
.oneshot(Request::new(()))
|
||||
.await
|
||||
.expect("disabled middleware");
|
||||
assert_eq!(response.status(), StatusCode::OK);
|
||||
assert_eq!(
|
||||
KEYSTONE_CREDENTIALS.with(|credentials| credentials.as_ref().map(|value| value.access_key.clone())),
|
||||
Some("outer-identity".into())
|
||||
);
|
||||
})
|
||||
.await;
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_layer_creation_no_keystone() {
|
||||
// Test that layer can be created without Keystone provider
|
||||
|
||||
@@ -16,15 +16,14 @@ use parking_lot::{RwLock, RwLockReadGuard, RwLockWriteGuard};
|
||||
use std::collections::HashMap;
|
||||
use std::ops::{Deref, DerefMut};
|
||||
use std::sync::Arc;
|
||||
use std::sync::atomic::{AtomicBool, AtomicU64, Ordering};
|
||||
use std::sync::atomic::{AtomicBool, Ordering};
|
||||
use std::time::{Duration, SystemTime};
|
||||
use tokio::sync::Notify;
|
||||
use tokio::time::{Instant, timeout, timeout_at};
|
||||
|
||||
use crate::{
|
||||
CACHE_FENCE_SHARD_COUNT, CacheFenceStamp, FastLockGuard, GlobalLockManager, LockClient, LockId, LockInfo, LockManager,
|
||||
LockMetadata, LockPriority, LockRequest, LockResponse, LockStats, LockStatus, LockType, Result,
|
||||
types::cache_fence_shard_index,
|
||||
FastLockGuard, GlobalLockManager, LockClient, LockId, LockInfo, LockManager, LockMetadata, LockPriority, LockRequest,
|
||||
LockResponse, LockStats, LockStatus, LockType, Result,
|
||||
};
|
||||
|
||||
/// Default shard count for guard storage (must be power of 2)
|
||||
@@ -205,13 +204,6 @@ pub struct LocalClient {
|
||||
manager: Option<Arc<GlobalLockManager>>,
|
||||
reaper_started: AtomicBool,
|
||||
reaper_interval: Duration,
|
||||
cache_fence_boot_id: uuid::Uuid,
|
||||
cache_fence_generations: Box<[AtomicU64]>,
|
||||
}
|
||||
|
||||
struct LocalGrant {
|
||||
info: LockInfo,
|
||||
cache_fence_stamp: Option<CacheFenceStamp>,
|
||||
}
|
||||
|
||||
#[derive(Debug)]
|
||||
@@ -299,7 +291,7 @@ impl LocalClient {
|
||||
&self,
|
||||
request: &LockRequest,
|
||||
acquire_deadline: Instant,
|
||||
) -> std::result::Result<LocalGrant, LocalAcquireFailure> {
|
||||
) -> std::result::Result<LockInfo, LocalAcquireFailure> {
|
||||
let started = Instant::now();
|
||||
self.ensure_reaper();
|
||||
if request.lock_id.resource != request.resource {
|
||||
@@ -410,14 +402,7 @@ impl LocalClient {
|
||||
priority: request.priority,
|
||||
wait_start_time: None,
|
||||
};
|
||||
return Ok(LocalGrant {
|
||||
info: lock_info,
|
||||
cache_fence_stamp: self.cache_fence_stamp_for_grant(
|
||||
&request.resource,
|
||||
request.lock_type,
|
||||
request.cache_fence_stamp_requested,
|
||||
),
|
||||
});
|
||||
return Ok(lock_info);
|
||||
}
|
||||
Err(crate::fast_lock::LockResult::Timeout) => {
|
||||
if !retried_after_reclaim {
|
||||
@@ -478,36 +463,6 @@ impl LocalClient {
|
||||
manager,
|
||||
reaper_started: AtomicBool::new(false),
|
||||
reaper_interval,
|
||||
cache_fence_boot_id: uuid::Uuid::new_v4(),
|
||||
cache_fence_generations: (0..CACHE_FENCE_SHARD_COUNT)
|
||||
.map(|_| AtomicU64::new(0))
|
||||
.collect::<Vec<_>>()
|
||||
.into_boxed_slice(),
|
||||
}
|
||||
}
|
||||
|
||||
fn cache_fence_stamp_for_grant(
|
||||
&self,
|
||||
resource: &crate::ObjectKey,
|
||||
lock_type: LockType,
|
||||
stamp_requested: bool,
|
||||
) -> Option<CacheFenceStamp> {
|
||||
match lock_type {
|
||||
LockType::Exclusive => {
|
||||
let shard_index = cache_fence_shard_index(resource);
|
||||
let generation = &self.cache_fence_generations[usize::from(shard_index)];
|
||||
let _ = generation
|
||||
.try_update(Ordering::AcqRel, Ordering::Acquire, |current| current.checked_add(1))
|
||||
.ok()?
|
||||
.checked_add(1)?;
|
||||
None
|
||||
}
|
||||
LockType::Shared if stamp_requested => {
|
||||
let shard_index = cache_fence_shard_index(resource);
|
||||
let generation = &self.cache_fence_generations[usize::from(shard_index)];
|
||||
CacheFenceStamp::new(self.cache_fence_boot_id, shard_index, generation.load(Ordering::Acquire))
|
||||
}
|
||||
LockType::Shared => None,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -624,10 +579,7 @@ impl LockClient for LocalClient {
|
||||
let started = Instant::now();
|
||||
let deadline = started.checked_add(request.acquire_timeout).unwrap_or(started);
|
||||
Ok(match self.acquire_lock_until(request, deadline).await {
|
||||
Ok(grant) => match grant.cache_fence_stamp {
|
||||
Some(stamp) => LockResponse::success_with_cache_fence_stamp(grant.info, Duration::ZERO, stamp),
|
||||
None => LockResponse::success(grant.info, Duration::ZERO),
|
||||
},
|
||||
Ok(info) => LockResponse::success(info, Duration::ZERO),
|
||||
Err(failure) => failure.into_response(request),
|
||||
})
|
||||
}
|
||||
@@ -639,10 +591,9 @@ impl LockClient for LocalClient {
|
||||
let item_deadline = deadline.min(started.checked_add(request.acquire_timeout).unwrap_or(started));
|
||||
async move {
|
||||
match self.acquire_lock_until(request, item_deadline).await {
|
||||
Ok(grant) => super::BatchAcquireOutcome::Granted {
|
||||
lock_info: Some(Box::new(grant.info)),
|
||||
Ok(info) => super::BatchAcquireOutcome::Granted {
|
||||
lock_info: Some(Box::new(info)),
|
||||
wait_time: Duration::ZERO,
|
||||
cache_fence_stamp: grant.cache_fence_stamp,
|
||||
},
|
||||
Err(failure) => failure.into_batch_outcome(request),
|
||||
}
|
||||
@@ -1684,132 +1635,4 @@ mod tests {
|
||||
let recovered = client.acquire_lock(&second).await.unwrap();
|
||||
assert!(recovered.success, "the reclaimed guard must be available to the next request");
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn cache_fence_stamp_advances_only_for_exclusive_grants() {
|
||||
let client = LocalClient::with_manager(Arc::new(GlobalLockManager::new()));
|
||||
let resource = crate::ObjectKey::new("bucket", "cache-fence");
|
||||
let ordinary = LockRequest::new(resource.clone(), LockType::Shared, "ordinary-reader");
|
||||
let ordinary_response = client.acquire_lock(&ordinary).await.expect("ordinary shared grant");
|
||||
assert!(
|
||||
ordinary_response.cache_fence_stamp.is_none(),
|
||||
"ordinary shared locks preserve their wire shape"
|
||||
);
|
||||
assert!(client.release(&ordinary.lock_id).await.expect("release ordinary reader"));
|
||||
|
||||
let first_read = LockRequest::new(resource.clone(), LockType::Shared, "reader-a").request_cache_fence_stamp();
|
||||
let first = client
|
||||
.acquire_lock(&first_read)
|
||||
.await
|
||||
.expect("first shared grant")
|
||||
.cache_fence_stamp
|
||||
.expect("new local voter supplies a stamp");
|
||||
assert!(client.release(&first_read.lock_id).await.expect("release first reader"));
|
||||
|
||||
let write = LockRequest::new(resource.clone(), LockType::Exclusive, "writer");
|
||||
let write_response = client.acquire_lock(&write).await.expect("exclusive grant");
|
||||
assert!(
|
||||
write_response.cache_fence_stamp.is_none(),
|
||||
"exclusive grants advance without adding response bytes"
|
||||
);
|
||||
assert!(client.release(&write.lock_id).await.expect("release writer"));
|
||||
|
||||
let second_read = LockRequest::new(resource, LockType::Shared, "reader-b").request_cache_fence_stamp();
|
||||
let second = client
|
||||
.acquire_lock(&second_read)
|
||||
.await
|
||||
.expect("second shared grant")
|
||||
.cache_fence_stamp
|
||||
.expect("second shared grant supplies a stamp");
|
||||
assert_eq!(second.voter_boot_id(), first.voter_boot_id());
|
||||
assert_eq!(second.shard_index(), first.shard_index());
|
||||
assert_eq!(second.generation(), first.generation() + 1);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn cache_fence_voter_restart_changes_boot_identity() {
|
||||
let resource = crate::ObjectKey::new("bucket", "restart");
|
||||
let request = LockRequest::new(resource, LockType::Shared, "reader").request_cache_fence_stamp();
|
||||
let first = LocalClient::with_manager(Arc::new(GlobalLockManager::new()))
|
||||
.acquire_lock(&request)
|
||||
.await
|
||||
.expect("first voter grant")
|
||||
.cache_fence_stamp
|
||||
.expect("first voter stamp");
|
||||
let second = LocalClient::with_manager(Arc::new(GlobalLockManager::new()))
|
||||
.acquire_lock(&request)
|
||||
.await
|
||||
.expect("restarted voter grant")
|
||||
.cache_fence_stamp
|
||||
.expect("restarted voter stamp");
|
||||
assert_ne!(first.voter_boot_id(), second.voter_boot_id());
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn cache_fence_generation_overflow_permanently_withholds_proof() {
|
||||
let client = LocalClient::with_manager(Arc::new(GlobalLockManager::new()));
|
||||
let resource = crate::ObjectKey::new("bucket", "overflow");
|
||||
let shard = cache_fence_shard_index(&resource);
|
||||
client.cache_fence_generations[usize::from(shard)].store(u64::MAX - 1, Ordering::Release);
|
||||
|
||||
let write = LockRequest::new(resource.clone(), LockType::Exclusive, "writer");
|
||||
let response = client
|
||||
.acquire_lock(&write)
|
||||
.await
|
||||
.expect("overflowing exclusive grant remains a valid lock");
|
||||
assert!(response.success);
|
||||
assert!(response.cache_fence_stamp.is_none(), "exhausted generation cannot authorize cache use");
|
||||
assert!(client.release(&write.lock_id).await.expect("release overflow writer"));
|
||||
|
||||
let read = LockRequest::new(resource, LockType::Shared, "reader").request_cache_fence_stamp();
|
||||
let response = client
|
||||
.acquire_lock(&read)
|
||||
.await
|
||||
.expect("shared grant after overflow remains a valid lock");
|
||||
assert!(response.success);
|
||||
assert!(
|
||||
response.cache_fence_stamp.is_none(),
|
||||
"overflowed shard stays fail closed until voter restart"
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn cache_fence_hash_collision_causes_only_conservative_invalidation() {
|
||||
let mut seen = std::collections::HashMap::new();
|
||||
let (first_key, second_key) = (0..=CACHE_FENCE_SHARD_COUNT)
|
||||
.find_map(|index| {
|
||||
let key = crate::ObjectKey::new("bucket", format!("collision-{index}"));
|
||||
let shard = cache_fence_shard_index(&key);
|
||||
seen.insert(shard, key.clone()).map(|previous| (previous, key))
|
||||
})
|
||||
.expect("4097 keys must contain a collision across 4096 shards");
|
||||
let client = LocalClient::with_manager(Arc::new(GlobalLockManager::new()));
|
||||
|
||||
let first_read = LockRequest::new(first_key.clone(), LockType::Shared, "reader-a").request_cache_fence_stamp();
|
||||
let before = client
|
||||
.acquire_lock(&first_read)
|
||||
.await
|
||||
.expect("first collision reader")
|
||||
.cache_fence_stamp
|
||||
.expect("first collision stamp");
|
||||
assert!(
|
||||
client
|
||||
.release(&first_read.lock_id)
|
||||
.await
|
||||
.expect("release first collision reader")
|
||||
);
|
||||
|
||||
let other_write = LockRequest::new(second_key, LockType::Exclusive, "writer");
|
||||
assert!(client.acquire_lock(&other_write).await.expect("colliding writer").success);
|
||||
assert!(client.release(&other_write.lock_id).await.expect("release colliding writer"));
|
||||
|
||||
let second_read = LockRequest::new(first_key, LockType::Shared, "reader-b").request_cache_fence_stamp();
|
||||
let after = client
|
||||
.acquire_lock(&second_read)
|
||||
.await
|
||||
.expect("second collision reader")
|
||||
.cache_fence_stamp
|
||||
.expect("second collision stamp");
|
||||
assert_eq!(after.generation(), before.generation() + 1);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -29,7 +29,6 @@ pub enum BatchAcquireOutcome {
|
||||
Granted {
|
||||
lock_info: Option<Box<LockInfo>>,
|
||||
wait_time: Duration,
|
||||
cache_fence_stamp: Option<crate::CacheFenceStamp>,
|
||||
},
|
||||
/// This attempt cannot grant later, and another attempt may succeed.
|
||||
RetryableNoGrant { error: LockError },
|
||||
@@ -50,7 +49,6 @@ impl BatchAcquireOutcome {
|
||||
Self::Granted {
|
||||
lock_info: response.lock_info.map(Box::new),
|
||||
wait_time: response.wait_time,
|
||||
cache_fence_stamp: response.cache_fence_stamp,
|
||||
}
|
||||
} else {
|
||||
Self::Ambiguous {
|
||||
@@ -233,7 +231,6 @@ mod tests {
|
||||
error: None,
|
||||
wait_time: Duration::ZERO,
|
||||
position_in_queue: None,
|
||||
cache_fence_stamp: None,
|
||||
}),
|
||||
BatchAcquireOutcome::Granted { lock_info: None, .. }
|
||||
));
|
||||
|
||||
@@ -16,7 +16,7 @@ use crate::{
|
||||
ObjectKey,
|
||||
client::{BatchAcquireOutcome, LockCleanupOutcome, LockClient},
|
||||
error::{LockError, Result},
|
||||
types::{CacheFenceProof, CacheFenceStamp, LockId, LockInfo, LockRequest, LockResponse, LockStatus, LockType},
|
||||
types::{LockId, LockInfo, LockRequest, LockResponse, LockStatus, LockType},
|
||||
};
|
||||
use futures::{
|
||||
future::join_all,
|
||||
@@ -358,7 +358,6 @@ struct HeldLockEntry {
|
||||
lock_id: LockId,
|
||||
client: Arc<dyn LockClient>,
|
||||
valid_until: Instant,
|
||||
cache_fence_stamp: Option<CacheFenceStamp>,
|
||||
}
|
||||
|
||||
impl HeldLockEntry {
|
||||
@@ -454,7 +453,6 @@ pub struct DistributedLockGuard {
|
||||
refresh_task: Option<JoinHandle<()>>,
|
||||
/// Lock-loss signal, shared with the heartbeat task.
|
||||
lock_lost: Arc<LockLostSignal>,
|
||||
cache_fence_proof: Option<CacheFenceProof>,
|
||||
}
|
||||
|
||||
impl DistributedLockGuard {
|
||||
@@ -466,15 +464,6 @@ impl DistributedLockGuard {
|
||||
/// - `heartbeat`: lease TTL, optional renewal cadence, quorum, and diagnostics.
|
||||
fn new(lock_id: LockId, entries: Vec<HeldLockEntry>, lock_type: LockType, heartbeat: LockHeartbeatConfig) -> Self {
|
||||
record_lock_held_acquire(lock_type);
|
||||
let cache_fence_proof = (lock_type == LockType::Shared)
|
||||
.then(|| {
|
||||
CacheFenceProof::from_grants(
|
||||
&heartbeat.resource,
|
||||
entries.iter().filter_map(|entry| entry.cache_fence_stamp),
|
||||
heartbeat.refresh_quorum,
|
||||
)
|
||||
})
|
||||
.flatten();
|
||||
let lock_lost = Arc::new(LockLostSignal::default());
|
||||
match Self::quorum_valid_until(&entries, heartbeat.refresh_quorum) {
|
||||
Some(valid_until) => lock_lost.set_valid_until(valid_until),
|
||||
@@ -495,7 +484,6 @@ impl DistributedLockGuard {
|
||||
disarmed: false,
|
||||
refresh_task,
|
||||
lock_lost,
|
||||
cache_fence_proof,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -619,12 +607,6 @@ impl DistributedLockGuard {
|
||||
self.entries.iter().map(|entry| &entry.lock_id)
|
||||
}
|
||||
|
||||
/// Read-quorum cache-fence proof carried by this guard, when every required
|
||||
/// voter supplied one valid, distinct stamp.
|
||||
pub fn cache_fence_proof(&self) -> Option<&CacheFenceProof> {
|
||||
self.cache_fence_proof.as_ref()
|
||||
}
|
||||
|
||||
/// Whether the guard's heartbeat has observed a refresh-quorum loss.
|
||||
pub fn is_lock_lost(&self) -> bool {
|
||||
self.lock_lost.is_lost()
|
||||
@@ -929,22 +911,6 @@ impl DistributedLock {
|
||||
self.acquire_guard(&req).await
|
||||
}
|
||||
|
||||
/// Acquire a shared guard and request cache-fence stamps from capable voters.
|
||||
#[hotpath::measure]
|
||||
pub async fn rlock_guard_with_cache_fence(
|
||||
&self,
|
||||
resource: ObjectKey,
|
||||
owner: &str,
|
||||
timeout: Duration,
|
||||
ttl: Duration,
|
||||
) -> Result<Option<DistributedLockGuard>> {
|
||||
let req = LockRequest::new(resource, LockType::Shared, owner)
|
||||
.with_acquire_timeout(timeout)
|
||||
.with_ttl(ttl)
|
||||
.request_cache_fence_stamp();
|
||||
self.acquire_guard(&req).await
|
||||
}
|
||||
|
||||
/// Acquire independent exclusive locks with one sub-batch per peer and round.
|
||||
/// A successful key starts its heartbeat immediately. Only completed, explicitly
|
||||
/// retryable attempts whose grants have been cleaned may enter another round.
|
||||
@@ -1147,12 +1113,11 @@ impl DistributedLock {
|
||||
pending_by_key[key][reply.peer_index] = false;
|
||||
let outcome = normalized_batch_outcome(request, outcomes.next());
|
||||
match outcome {
|
||||
BatchAcquireOutcome::Granted { cache_fence_stamp, .. } => {
|
||||
BatchAcquireOutcome::Granted { .. } => {
|
||||
let entry = HeldLockEntry {
|
||||
lock_id: request.lock_id.clone(),
|
||||
client: client.clone(),
|
||||
valid_until,
|
||||
cache_fence_stamp,
|
||||
};
|
||||
if results[key].is_some() {
|
||||
redundant.push(entry.release_entry());
|
||||
@@ -1512,7 +1477,6 @@ impl DistributedLock {
|
||||
lock_id: request.lock_id.clone(),
|
||||
client: client.clone(),
|
||||
valid_until: reply.started,
|
||||
cache_fence_stamp: None,
|
||||
};
|
||||
let completed = completed.clone();
|
||||
// Each task owns its grant before the drain next awaits.
|
||||
@@ -2085,7 +2049,6 @@ impl DistributedLock {
|
||||
match join_result {
|
||||
Ok((idx, Ok(resp))) => {
|
||||
if resp.success {
|
||||
let cache_fence_stamp = resp.cache_fence_stamp;
|
||||
let lock_id = resp
|
||||
.lock_info
|
||||
.as_ref()
|
||||
@@ -2097,7 +2060,6 @@ impl DistributedLock {
|
||||
lock_id,
|
||||
client: client.clone(),
|
||||
valid_until: grant_valid_until,
|
||||
cache_fence_stamp,
|
||||
});
|
||||
} else {
|
||||
tracing::warn!("Missing lock client at index {} while recording success", idx);
|
||||
@@ -2390,118 +2352,6 @@ mod tests {
|
||||
RpcError, // Err RPC jitter
|
||||
}
|
||||
|
||||
#[derive(Debug)]
|
||||
struct LegacyProofClient {
|
||||
inner: LocalClient,
|
||||
}
|
||||
|
||||
impl LegacyProofClient {
|
||||
fn new() -> Self {
|
||||
Self {
|
||||
inner: LocalClient::with_manager(Arc::new(GlobalLockManager::new())),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[async_trait::async_trait]
|
||||
impl LockClient for LegacyProofClient {
|
||||
async fn acquire_lock(&self, request: &LockRequest) -> crate::Result<LockResponse> {
|
||||
let mut response = self.inner.acquire_lock(request).await?;
|
||||
response.cache_fence_stamp = None;
|
||||
Ok(response)
|
||||
}
|
||||
|
||||
async fn release(&self, lock_id: &LockId) -> crate::Result<bool> {
|
||||
self.inner.release(lock_id).await
|
||||
}
|
||||
|
||||
async fn refresh(&self, lock_id: &LockId) -> crate::Result<bool> {
|
||||
self.inner.refresh(lock_id).await
|
||||
}
|
||||
|
||||
async fn force_release(&self, lock_id: &LockId) -> crate::Result<bool> {
|
||||
self.inner.force_release(lock_id).await
|
||||
}
|
||||
|
||||
async fn check_status(&self, lock_id: &LockId) -> crate::Result<Option<LockInfo>> {
|
||||
self.inner.check_status(lock_id).await
|
||||
}
|
||||
|
||||
async fn get_stats(&self) -> crate::Result<LockStats> {
|
||||
self.inner.get_stats().await
|
||||
}
|
||||
|
||||
async fn close(&self) -> crate::Result<()> {
|
||||
self.inner.close().await
|
||||
}
|
||||
|
||||
async fn is_online(&self) -> bool {
|
||||
true
|
||||
}
|
||||
|
||||
async fn is_local(&self) -> bool {
|
||||
true
|
||||
}
|
||||
}
|
||||
|
||||
fn stamped_voters(count: usize) -> Vec<Arc<dyn LockClient>> {
|
||||
(0..count)
|
||||
.map(|_| Arc::new(LocalClient::with_manager(Arc::new(GlobalLockManager::new()))) as Arc<dyn LockClient>)
|
||||
.collect()
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn shared_guard_exposes_complete_cache_fence_quorum_proof() {
|
||||
let lock = DistributedLock::new("cache-proof".to_owned(), stamped_voters(3), 2);
|
||||
let resource = ObjectKey::new("bucket", "object");
|
||||
let read = LockRequest::new(resource.clone(), LockType::Shared, "reader").request_cache_fence_stamp();
|
||||
let mut first = lock
|
||||
.acquire_guard(&read)
|
||||
.await
|
||||
.expect("read quorum acquisition")
|
||||
.expect("read quorum guard");
|
||||
let before = first.cache_fence_proof().expect("all new voters provide proof").clone();
|
||||
assert_eq!(before.stamps().len(), 2);
|
||||
assert!(first.release());
|
||||
|
||||
let write = LockRequest::new(resource.clone(), LockType::Exclusive, "writer");
|
||||
let mut writer = lock
|
||||
.acquire_guard(&write)
|
||||
.await
|
||||
.expect("write quorum acquisition")
|
||||
.expect("write quorum guard");
|
||||
assert!(writer.cache_fence_proof().is_none(), "write guards never authorize cache hits");
|
||||
assert!(writer.release());
|
||||
|
||||
let mut second = lock
|
||||
.acquire_guard(&read)
|
||||
.await
|
||||
.expect("second read quorum acquisition")
|
||||
.expect("second read quorum guard");
|
||||
let after = second.cache_fence_proof().expect("second read proof");
|
||||
assert_ne!(&before, after, "an intervening write quorum must change every later read proof");
|
||||
assert!(second.release());
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn shared_guard_with_legacy_quorum_keeps_lock_but_withholds_proof() {
|
||||
let clients: Vec<Arc<dyn LockClient>> = vec![
|
||||
Arc::new(LegacyProofClient::new()),
|
||||
Arc::new(LegacyProofClient::new()),
|
||||
Arc::new(LocalClient::with_manager(Arc::new(GlobalLockManager::new()))),
|
||||
];
|
||||
let lock = DistributedLock::new("mixed-cache-proof".to_owned(), clients, 2);
|
||||
let read = LockRequest::new(ObjectKey::new("bucket", "object"), LockType::Shared, "reader").request_cache_fence_stamp();
|
||||
let mut guard = lock
|
||||
.acquire_guard(&read)
|
||||
.await
|
||||
.expect("mixed read quorum acquisition")
|
||||
.expect("mixed read quorum guard remains available");
|
||||
assert!(guard.cache_fence_proof().is_none(), "legacy voters fail closed only for cache proof");
|
||||
assert!(!guard.is_lock_lost(), "missing proof must not weaken the acquired lock");
|
||||
assert!(guard.release());
|
||||
}
|
||||
|
||||
struct ConstantRng(u64);
|
||||
|
||||
impl TryRng for ConstantRng {
|
||||
@@ -4880,7 +4730,6 @@ mod tests {
|
||||
error: None,
|
||||
wait_time: Duration::ZERO,
|
||||
position_in_queue: None,
|
||||
cache_fence_stamp: None,
|
||||
};
|
||||
let responses = Ok(vec![
|
||||
success.clone(),
|
||||
|
||||
@@ -671,7 +671,6 @@ fn batch_diagnostics_preserve_typed_no_grant_and_ambiguity() {
|
||||
BatchAcquireOutcome::Granted {
|
||||
lock_info: None,
|
||||
wait_time: Duration::ZERO,
|
||||
cache_fence_stamp: None,
|
||||
},
|
||||
BatchAcquireOutcome::RetryableNoGrant {
|
||||
error: LockError::internal("explicit"),
|
||||
|
||||
@@ -51,8 +51,8 @@ pub use crate::{
|
||||
namespace::{NamespaceLock, NamespaceLockGuard, NamespaceLockWrapper},
|
||||
// Core types
|
||||
types::{
|
||||
CACHE_FENCE_SHARD_COUNT, CacheFenceProof, CacheFenceStamp, CacheFenceStampDecodeError, HealthInfo, HealthStatus, LockId,
|
||||
LockInfo, LockLeaseInfo, LockMetadata, LockPriority, LockRequest, LockResponse, LockStats, LockStatus, LockType,
|
||||
HealthInfo, HealthStatus, LockId, LockInfo, LockLeaseInfo, LockMetadata, LockPriority, LockRequest, LockResponse,
|
||||
LockStats, LockStatus, LockType,
|
||||
},
|
||||
};
|
||||
|
||||
|
||||
@@ -89,18 +89,6 @@ impl NamespaceLockWrapper {
|
||||
self.lock.get_read_lock(self.resource.clone(), &self.owner, timeout).await
|
||||
}
|
||||
|
||||
/// Acquire a shared lock and request cache-fence proof from capable voters.
|
||||
/// Lock admission is unchanged; legacy or insufficient proof is represented
|
||||
/// by [`NamespaceLockGuard::cache_fence_proof`] returning `None`.
|
||||
pub async fn get_read_lock_with_cache_fence(
|
||||
&self,
|
||||
timeout: Duration,
|
||||
) -> std::result::Result<NamespaceLockGuard, crate::error::LockError> {
|
||||
self.lock
|
||||
.get_read_lock_with_cache_fence(self.resource.clone(), &self.owner, timeout)
|
||||
.await
|
||||
}
|
||||
|
||||
/// Acquire write locks for `resources` in one batched round using this
|
||||
/// wrapper's owner; see [`NamespaceLock::acquire_write_locks_batch`].
|
||||
/// One guard or error per resource, in `resources` order.
|
||||
@@ -122,15 +110,6 @@ impl NamespaceLockGuard {
|
||||
}
|
||||
}
|
||||
|
||||
/// Read-quorum cache-fence proof supplied by distributed lock voters.
|
||||
/// Local fast locks and mixed/legacy cohorts return `None`.
|
||||
pub fn cache_fence_proof(&self) -> Option<&crate::CacheFenceProof> {
|
||||
match self {
|
||||
Self::Standard(guard) => guard.cache_fence_proof(),
|
||||
Self::Fast(_) => None,
|
||||
}
|
||||
}
|
||||
|
||||
/// Get the object key if available (only for Fast guards)
|
||||
pub fn key(&self) -> Option<&ObjectKey> {
|
||||
match self {
|
||||
@@ -319,27 +298,6 @@ impl NamespaceLock {
|
||||
}
|
||||
}
|
||||
|
||||
/// Convenience: acquire a shared guard with optional cache-fence proof.
|
||||
/// Missing proof never changes whether the underlying lock reaches quorum.
|
||||
pub async fn rlock_guard_with_cache_fence(
|
||||
&self,
|
||||
resource: ObjectKey,
|
||||
owner: &str,
|
||||
timeout: Duration,
|
||||
ttl: Duration,
|
||||
) -> Result<Option<NamespaceLockGuard>> {
|
||||
match self {
|
||||
Self::Distributed(lock) => lock
|
||||
.rlock_guard_with_cache_fence(resource, owner, timeout, ttl)
|
||||
.await
|
||||
.map(|opt| opt.map(NamespaceLockGuard::Standard)),
|
||||
Self::Local(lock) => lock
|
||||
.rlock_guard(resource, owner, timeout, ttl)
|
||||
.await
|
||||
.map(|opt| opt.map(NamespaceLockGuard::Fast)),
|
||||
}
|
||||
}
|
||||
|
||||
/// Acquire write lock (exclusive lock) with timeout
|
||||
/// Returns the guard if acquisition succeeds, or an error if it fails
|
||||
pub async fn get_write_lock(
|
||||
@@ -402,23 +360,6 @@ impl NamespaceLock {
|
||||
}
|
||||
}
|
||||
|
||||
/// Acquire a shared lock while requesting a cache-fence proof from the
|
||||
/// existing distributed quorum. Missing proof never changes lock success.
|
||||
pub async fn get_read_lock_with_cache_fence(
|
||||
&self,
|
||||
resource: ObjectKey,
|
||||
owner: &str,
|
||||
timeout: Duration,
|
||||
) -> std::result::Result<NamespaceLockGuard, crate::error::LockError> {
|
||||
let ttl = crate::fast_lock::DEFAULT_LOCK_TIMEOUT;
|
||||
let resource_str = format!("{}", resource);
|
||||
match self.rlock_guard_with_cache_fence(resource, owner, timeout, ttl).await {
|
||||
Ok(Some(guard)) => Ok(guard),
|
||||
Ok(None) => Err(crate::error::LockError::timeout(resource_str, timeout)),
|
||||
Err(error) => Err(error),
|
||||
}
|
||||
}
|
||||
|
||||
/// Acquire exclusive locks for many independent resources in one batched
|
||||
/// round; see [`crate::distributed_lock::DistributedLock::acquire_guard_batch`].
|
||||
/// Returns one guard or error per resource, in `resources` order.
|
||||
|
||||
@@ -664,39 +664,6 @@ async fn test_namespace_lock_get_read_lock_local() {
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn cache_fence_read_api_is_opt_in_and_does_not_change_lock_admission() {
|
||||
let clients = (0..3)
|
||||
.map(|_| Arc::new(LocalClient::with_manager(Arc::new(GlobalLockManager::new()))) as Arc<dyn LockClient>)
|
||||
.collect();
|
||||
let lock = NamespaceLock::with_clients("cache-fence-proof".to_string(), clients);
|
||||
let resource = create_test_object_key("bucket", "object");
|
||||
|
||||
let mut ordinary = lock
|
||||
.get_read_lock(resource.clone(), "ordinary", Duration::from_secs(1))
|
||||
.await
|
||||
.expect("ordinary read lock");
|
||||
assert!(ordinary.cache_fence_proof().is_none(), "ordinary read locks do not add proof traffic");
|
||||
assert!(ordinary.release());
|
||||
|
||||
let mut requested = lock
|
||||
.get_read_lock_with_cache_fence(resource, "cache-reader", Duration::from_secs(1))
|
||||
.await
|
||||
.expect("proof-carrying read lock uses the same quorum admission");
|
||||
assert_eq!(requested.cache_fence_proof().expect("new voter quorum proof").stamps().len(), 2);
|
||||
assert!(requested.release());
|
||||
|
||||
let local = NamespaceLock::with_local_manager("local-cache-fence".to_string(), Arc::new(GlobalLockManager::new()));
|
||||
let local_guard = local
|
||||
.get_read_lock_with_cache_fence(create_test_object_key("bucket", "local-object"), "local-reader", Duration::from_secs(1))
|
||||
.await
|
||||
.expect("local fast lock remains available");
|
||||
assert!(
|
||||
local_guard.cache_fence_proof().is_none(),
|
||||
"single-process fast locks need no distributed proof"
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn test_namespace_lock_guard_release() {
|
||||
let manager = Arc::new(GlobalLockManager::new());
|
||||
|
||||
+1
-295
@@ -13,180 +13,11 @@
|
||||
// limitations under the License.
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
use std::{
|
||||
sync::Arc,
|
||||
time::{Duration, SystemTime, UNIX_EPOCH},
|
||||
};
|
||||
use std::time::{Duration, SystemTime, UNIX_EPOCH};
|
||||
use uuid::Uuid;
|
||||
|
||||
use crate::ObjectKey;
|
||||
|
||||
/// Wire version for namespace-lock cache-fence stamps.
|
||||
pub const CACHE_FENCE_STAMP_VERSION: u8 = 1;
|
||||
/// Fixed number of mutation-counter shards maintained by each lock voter.
|
||||
pub const CACHE_FENCE_SHARD_COUNT: usize = 4096;
|
||||
const CACHE_FENCE_SHARD_COUNT_U16: u16 = 4096;
|
||||
const CACHE_FENCE_SHARD_MASK: u16 = CACHE_FENCE_SHARD_COUNT_U16 - 1;
|
||||
const CACHE_FENCE_STAMP_ENCODED_LEN: usize = 27;
|
||||
|
||||
/// Stable shard for namespace mutation generations. Hash collisions only cause
|
||||
/// conservative cache invalidation; this is not an identity or authorization hash.
|
||||
pub(crate) fn cache_fence_shard_index(resource: &ObjectKey) -> u16 {
|
||||
const FNV_OFFSET: u64 = 0xcbf29ce484222325;
|
||||
const FNV_PRIME: u64 = 0x100000001b3;
|
||||
|
||||
fn update(mut hash: u64, bytes: &[u8]) -> u64 {
|
||||
for byte in bytes {
|
||||
hash ^= u64::from(*byte);
|
||||
hash = hash.wrapping_mul(FNV_PRIME);
|
||||
}
|
||||
hash
|
||||
}
|
||||
|
||||
let mut hash = update(FNV_OFFSET, resource.bucket.as_bytes());
|
||||
hash = update(hash, &[0]);
|
||||
hash = update(hash, resource.object.as_bytes());
|
||||
hash = update(hash, &[1]);
|
||||
if let Some(version) = &resource.version {
|
||||
hash = update(hash, &[1]);
|
||||
hash = update(hash, version.as_bytes());
|
||||
} else {
|
||||
hash = update(hash, &[0]);
|
||||
}
|
||||
let bytes = hash.to_le_bytes();
|
||||
u16::from_le_bytes([bytes[0], bytes[1]]) & CACHE_FENCE_SHARD_MASK
|
||||
}
|
||||
|
||||
/// One lock voter's proof that identifies its process lifetime and the current
|
||||
/// mutation generation for a stable object-key shard.
|
||||
#[derive(Clone, Copy, Debug, Eq, Hash, Ord, PartialEq, PartialOrd)]
|
||||
pub struct CacheFenceStamp {
|
||||
voter_boot_id: Uuid,
|
||||
shard_index: u16,
|
||||
generation: u64,
|
||||
}
|
||||
|
||||
impl CacheFenceStamp {
|
||||
pub(crate) fn new(voter_boot_id: Uuid, shard_index: u16, generation: u64) -> Option<Self> {
|
||||
(voter_boot_id != Uuid::nil() && shard_index < CACHE_FENCE_SHARD_COUNT_U16 && generation != u64::MAX).then_some(Self {
|
||||
voter_boot_id,
|
||||
shard_index,
|
||||
generation,
|
||||
})
|
||||
}
|
||||
|
||||
/// Unique identity for this lock-voter process lifetime.
|
||||
pub fn voter_boot_id(self) -> Uuid {
|
||||
self.voter_boot_id
|
||||
}
|
||||
|
||||
/// Stable object-key shard whose generation this stamp describes.
|
||||
pub fn shard_index(self) -> u16 {
|
||||
self.shard_index
|
||||
}
|
||||
|
||||
/// Monotonic generation observed or advanced by the lock grant.
|
||||
pub fn generation(self) -> u64 {
|
||||
self.generation
|
||||
}
|
||||
|
||||
/// Encode the stamp for the optional lock-response proof field.
|
||||
pub fn encode(self) -> [u8; CACHE_FENCE_STAMP_ENCODED_LEN] {
|
||||
let mut encoded = [0; CACHE_FENCE_STAMP_ENCODED_LEN];
|
||||
encoded[0] = CACHE_FENCE_STAMP_VERSION;
|
||||
encoded[1..17].copy_from_slice(self.voter_boot_id.as_bytes());
|
||||
encoded[17..19].copy_from_slice(&self.shard_index.to_le_bytes());
|
||||
encoded[19..27].copy_from_slice(&self.generation.to_le_bytes());
|
||||
encoded
|
||||
}
|
||||
|
||||
/// Decode and strictly validate an optional lock-response proof field.
|
||||
///
|
||||
/// # Errors
|
||||
/// Returns [`CacheFenceStampDecodeError`] for unknown versions, malformed
|
||||
/// lengths, invalid voter identities, out-of-range shards, or an exhausted
|
||||
/// generation sentinel.
|
||||
pub fn decode(encoded: &[u8]) -> Result<Self, CacheFenceStampDecodeError> {
|
||||
if encoded.len() != CACHE_FENCE_STAMP_ENCODED_LEN {
|
||||
return Err(CacheFenceStampDecodeError::InvalidLength);
|
||||
}
|
||||
if encoded[0] != CACHE_FENCE_STAMP_VERSION {
|
||||
return Err(CacheFenceStampDecodeError::UnsupportedVersion);
|
||||
}
|
||||
let voter_boot_id = Uuid::from_slice(&encoded[1..17]).map_err(|_| CacheFenceStampDecodeError::InvalidBootId)?;
|
||||
if voter_boot_id.is_nil() {
|
||||
return Err(CacheFenceStampDecodeError::InvalidBootId);
|
||||
}
|
||||
let shard_index = u16::from_le_bytes([encoded[17], encoded[18]]);
|
||||
if shard_index >= CACHE_FENCE_SHARD_COUNT_U16 {
|
||||
return Err(CacheFenceStampDecodeError::InvalidShard);
|
||||
}
|
||||
let generation = u64::from_le_bytes(
|
||||
encoded[19..27]
|
||||
.try_into()
|
||||
.map_err(|_| CacheFenceStampDecodeError::InvalidLength)?,
|
||||
);
|
||||
if generation == u64::MAX {
|
||||
return Err(CacheFenceStampDecodeError::ExhaustedGeneration);
|
||||
}
|
||||
Ok(Self {
|
||||
voter_boot_id,
|
||||
shard_index,
|
||||
generation,
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
/// Invalid or unsupported cache-fence proof received from a lock peer.
|
||||
#[derive(Clone, Copy, Debug, Eq, PartialEq, thiserror::Error)]
|
||||
pub enum CacheFenceStampDecodeError {
|
||||
#[error("cache-fence stamp has an invalid length")]
|
||||
InvalidLength,
|
||||
#[error("cache-fence stamp uses an unsupported protocol version")]
|
||||
UnsupportedVersion,
|
||||
#[error("cache-fence stamp has an invalid voter boot id")]
|
||||
InvalidBootId,
|
||||
#[error("cache-fence stamp has an invalid shard")]
|
||||
InvalidShard,
|
||||
#[error("cache-fence stamp generation is exhausted")]
|
||||
ExhaustedGeneration,
|
||||
}
|
||||
|
||||
/// A deterministic read-quorum proof suitable for binding a future metadata
|
||||
/// cache entry to the namespace-lock cohort that authorized its fill.
|
||||
#[derive(Clone, Debug, Eq, PartialEq)]
|
||||
pub struct CacheFenceProof {
|
||||
stamps: Arc<[CacheFenceStamp]>,
|
||||
}
|
||||
|
||||
impl CacheFenceProof {
|
||||
pub(crate) fn from_grants(
|
||||
resource: &ObjectKey,
|
||||
stamps: impl IntoIterator<Item = CacheFenceStamp>,
|
||||
required_quorum: usize,
|
||||
) -> Option<Self> {
|
||||
if required_quorum == 0 {
|
||||
return None;
|
||||
}
|
||||
let expected_shard = cache_fence_shard_index(resource);
|
||||
let mut stamps = stamps.into_iter().collect::<Vec<_>>();
|
||||
if stamps.iter().any(|stamp| stamp.shard_index != expected_shard) {
|
||||
return None;
|
||||
}
|
||||
stamps.sort_unstable_by_key(|stamp| stamp.voter_boot_id);
|
||||
if stamps.windows(2).any(|pair| pair[0].voter_boot_id == pair[1].voter_boot_id) || stamps.len() < required_quorum {
|
||||
return None;
|
||||
}
|
||||
stamps.truncate(required_quorum);
|
||||
Some(Self { stamps: stamps.into() })
|
||||
}
|
||||
|
||||
/// Sorted, distinct voter stamps that form this proof.
|
||||
pub fn stamps(&self) -> &[CacheFenceStamp] {
|
||||
&self.stamps
|
||||
}
|
||||
}
|
||||
|
||||
/// Lock type enumeration
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
|
||||
pub enum LockType {
|
||||
@@ -420,10 +251,6 @@ pub struct LockRequest {
|
||||
/// (which never encoded this field) deserializable.
|
||||
#[serde(default)]
|
||||
pub refresh_interval: Option<Duration>,
|
||||
/// Request a cache-fence stamp from shared-lock voters. Exclusive grants
|
||||
/// always advance their voter generation, including for legacy requests.
|
||||
#[serde(default)]
|
||||
pub cache_fence_stamp_requested: bool,
|
||||
}
|
||||
|
||||
impl LockRequest {
|
||||
@@ -441,7 +268,6 @@ impl LockRequest {
|
||||
deadlock_detection: false,
|
||||
suppress_contention_logs: false,
|
||||
refresh_interval: None,
|
||||
cache_fence_stamp_requested: false,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -486,12 +312,6 @@ impl LockRequest {
|
||||
self.refresh_interval = Some(interval);
|
||||
self
|
||||
}
|
||||
|
||||
/// Ask capable shared-lock voters to return their current cache-fence stamp.
|
||||
pub fn request_cache_fence_stamp(mut self) -> Self {
|
||||
self.cache_fence_stamp_requested = true;
|
||||
self
|
||||
}
|
||||
}
|
||||
|
||||
/// Lock response structure
|
||||
@@ -507,9 +327,6 @@ pub struct LockResponse {
|
||||
pub wait_time: Duration,
|
||||
/// Position in wait queue
|
||||
pub position_in_queue: Option<usize>,
|
||||
/// Optional cache-fence generation returned by the granting voter.
|
||||
#[serde(skip)]
|
||||
pub cache_fence_stamp: Option<CacheFenceStamp>,
|
||||
}
|
||||
|
||||
impl LockResponse {
|
||||
@@ -521,19 +338,6 @@ impl LockResponse {
|
||||
error: None,
|
||||
wait_time,
|
||||
position_in_queue: None,
|
||||
cache_fence_stamp: None,
|
||||
}
|
||||
}
|
||||
|
||||
/// Create a success response carrying a validated cache-fence stamp.
|
||||
pub fn success_with_cache_fence_stamp(lock_info: LockInfo, wait_time: Duration, cache_fence_stamp: CacheFenceStamp) -> Self {
|
||||
Self {
|
||||
success: true,
|
||||
lock_info: Some(lock_info),
|
||||
error: None,
|
||||
wait_time,
|
||||
position_in_queue: None,
|
||||
cache_fence_stamp: Some(cache_fence_stamp),
|
||||
}
|
||||
}
|
||||
|
||||
@@ -545,7 +349,6 @@ impl LockResponse {
|
||||
error: Some(error.into()),
|
||||
wait_time,
|
||||
position_in_queue: None,
|
||||
cache_fence_stamp: None,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -557,7 +360,6 @@ impl LockResponse {
|
||||
error: None,
|
||||
wait_time,
|
||||
position_in_queue: Some(position),
|
||||
cache_fence_stamp: None,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -821,99 +623,3 @@ impl WaitQueueItem {
|
||||
self.wait_start_time.elapsed().unwrap_or_default()
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod cache_fence_tests {
|
||||
use super::{
|
||||
CACHE_FENCE_SHARD_COUNT_U16, CACHE_FENCE_STAMP_VERSION, CacheFenceProof, CacheFenceStamp, CacheFenceStampDecodeError,
|
||||
cache_fence_shard_index,
|
||||
};
|
||||
use crate::{LockRequest, LockType, ObjectKey};
|
||||
use uuid::Uuid;
|
||||
|
||||
fn stamp(boot: u128, shard: u16, generation: u64) -> CacheFenceStamp {
|
||||
CacheFenceStamp::new(Uuid::from_u128(boot), shard, generation).expect("test stamp should be valid")
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn cache_fence_stamp_round_trip_and_strict_validation() {
|
||||
let original = stamp(1, 17, 42);
|
||||
let encoded = original.encode();
|
||||
assert_eq!(CacheFenceStamp::decode(&encoded), Ok(original));
|
||||
|
||||
assert_eq!(CacheFenceStamp::decode(&encoded[..26]), Err(CacheFenceStampDecodeError::InvalidLength));
|
||||
|
||||
let mut invalid = encoded;
|
||||
invalid[0] = CACHE_FENCE_STAMP_VERSION + 1;
|
||||
assert_eq!(CacheFenceStamp::decode(&invalid), Err(CacheFenceStampDecodeError::UnsupportedVersion));
|
||||
|
||||
let mut invalid = encoded;
|
||||
invalid[1..17].fill(0);
|
||||
assert_eq!(CacheFenceStamp::decode(&invalid), Err(CacheFenceStampDecodeError::InvalidBootId));
|
||||
|
||||
let mut invalid = encoded;
|
||||
invalid[17..19].copy_from_slice(&CACHE_FENCE_SHARD_COUNT_U16.to_le_bytes());
|
||||
assert_eq!(CacheFenceStamp::decode(&invalid), Err(CacheFenceStampDecodeError::InvalidShard));
|
||||
|
||||
let mut invalid = encoded;
|
||||
invalid[19..27].copy_from_slice(&u64::MAX.to_le_bytes());
|
||||
assert_eq!(CacheFenceStamp::decode(&invalid), Err(CacheFenceStampDecodeError::ExhaustedGeneration));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn cache_fence_proof_requires_distinct_matching_quorum_stamps() {
|
||||
let resource = ObjectKey::new("bucket", "object");
|
||||
let shard = cache_fence_shard_index(&resource);
|
||||
assert_eq!(shard, 451, "stable object-key hash must not drift silently");
|
||||
let first = stamp(1, shard, 7);
|
||||
let second = stamp(2, shard, 7);
|
||||
|
||||
let proof = CacheFenceProof::from_grants(&resource, [second, first], 2).expect("two distinct voters form quorum proof");
|
||||
assert_eq!(proof.stamps(), &[first, second], "proof order is deterministic");
|
||||
assert_eq!(CacheFenceProof::from_grants(&resource, [first], 2), None);
|
||||
assert_eq!(CacheFenceProof::from_grants(&resource, [first, first], 2), None);
|
||||
|
||||
let other_shard = (shard + 1) % CACHE_FENCE_SHARD_COUNT_U16;
|
||||
assert_eq!(CacheFenceProof::from_grants(&resource, [first, stamp(3, other_shard, 7)], 2), None);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn read_and_write_quorums_always_intersect_for_supported_fleets() {
|
||||
for voters in 1_u32..=16 {
|
||||
let write_quorum = voters / 2 + 1;
|
||||
let read_quorum = voters - voters / 2;
|
||||
let all = (1_u32 << voters) - 1;
|
||||
for cohort in 0..=all {
|
||||
if cohort.count_ones() == write_quorum {
|
||||
assert!(
|
||||
(all ^ cohort).count_ones() < read_quorum,
|
||||
"a read quorum must not fit outside a successful write quorum: voters={voters}, cohort={cohort:#x}"
|
||||
);
|
||||
}
|
||||
if cohort.count_ones() == read_quorum {
|
||||
assert!(
|
||||
(all ^ cohort).count_ones() < write_quorum,
|
||||
"a write quorum must not fit outside a successful read quorum: voters={voters}, cohort={cohort:#x}"
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn cache_fence_request_flag_defaults_off_for_legacy_json() {
|
||||
let request =
|
||||
LockRequest::new(ObjectKey::new("bucket", "object"), LockType::Shared, "reader").request_cache_fence_stamp();
|
||||
let mut encoded = serde_json::to_value(&request).expect("encode current lock request");
|
||||
encoded
|
||||
.as_object_mut()
|
||||
.expect("lock request encodes as an object")
|
||||
.remove("cache_fence_stamp_requested");
|
||||
let legacy: LockRequest = serde_json::from_value(encoded).expect("decode request from a legacy peer");
|
||||
assert!(!legacy.cache_fence_stamp_requested);
|
||||
|
||||
let current: LockRequest = serde_json::from_str(&serde_json::to_string(&request).expect("encode requested proof"))
|
||||
.expect("decode requested proof");
|
||||
assert!(current.cache_fence_stamp_requested);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -73,7 +73,7 @@ pub const REQUEST_COMPAT_SEND_SITES: &[RequestCompatSendSite] = &[
|
||||
json_field: "batch_read_version_req",
|
||||
bin_field: "batch_read_version_req_bin",
|
||||
},
|
||||
json_encoder: "let batch_read_version_req = compat_json(batch_rpc_req.as_ref(), &policy)?;",
|
||||
json_encoder: "let batch_read_version_req = compat_json(&req)?;",
|
||||
policy: RequestJsonPolicy::MsgpackOnlyEligible,
|
||||
},
|
||||
RequestCompatSendSite {
|
||||
@@ -100,8 +100,8 @@ pub const REQUEST_COMPAT_SEND_SITES: &[RequestCompatSendSite] = &[
|
||||
json_field: "opts",
|
||||
bin_field: "opts_bin",
|
||||
},
|
||||
json_encoder: "serde_json::to_string(opts)?",
|
||||
policy: RequestJsonPolicy::MsgpackOnlyEligible,
|
||||
json_encoder: "let opts = match serde_json::to_string(&opts) {",
|
||||
policy: RequestJsonPolicy::AlwaysDualWriteUntilFallbackZero,
|
||||
},
|
||||
RequestCompatSendSite {
|
||||
field: CompatPayloadField {
|
||||
@@ -109,8 +109,8 @@ pub const REQUEST_COMPAT_SEND_SITES: &[RequestCompatSendSite] = &[
|
||||
json_field: "versions",
|
||||
bin_field: "versions_bin",
|
||||
},
|
||||
json_encoder: "versions_json.push(serde_json::to_string(version)?);",
|
||||
policy: RequestJsonPolicy::MsgpackOnlyEligible,
|
||||
json_encoder: "versions_str.push(match serde_json::to_string(file_info_versions) {",
|
||||
policy: RequestJsonPolicy::AlwaysDualWriteUntilFallbackZero,
|
||||
},
|
||||
RequestCompatSendSite {
|
||||
field: CompatPayloadField {
|
||||
@@ -118,7 +118,7 @@ pub const REQUEST_COMPAT_SEND_SITES: &[RequestCompatSendSite] = &[
|
||||
json_field: "read_multiple_req",
|
||||
bin_field: "read_multiple_req_bin",
|
||||
},
|
||||
json_encoder: "let read_multiple_req = compat_json(&req, &policy)?;",
|
||||
json_encoder: "let read_multiple_req = compat_json(&req)?;",
|
||||
policy: RequestJsonPolicy::MsgpackOnlyEligible,
|
||||
},
|
||||
RequestCompatSendSite {
|
||||
@@ -127,7 +127,7 @@ pub const REQUEST_COMPAT_SEND_SITES: &[RequestCompatSendSite] = &[
|
||||
json_field: "opts",
|
||||
bin_field: "opts_bin",
|
||||
},
|
||||
json_encoder: "let opts_str = compat_json(opts, &policy)?;",
|
||||
json_encoder: "let encoded_opts = compat_json(opts).and_then(|opts_str| encode_msgpack(opts).map(|opts_bin| (opts_str, opts_bin)));",
|
||||
policy: RequestJsonPolicy::MsgpackOnlyEligible,
|
||||
},
|
||||
RequestCompatSendSite {
|
||||
@@ -136,7 +136,7 @@ pub const REQUEST_COMPAT_SEND_SITES: &[RequestCompatSendSite] = &[
|
||||
json_field: "file_info",
|
||||
bin_field: "file_info_bin",
|
||||
},
|
||||
json_encoder: "let file_info = compat_json(&fi, &policy)?;",
|
||||
json_encoder: "let file_info = compat_json(&fi)?;",
|
||||
policy: RequestJsonPolicy::MsgpackOnlyEligible,
|
||||
},
|
||||
RequestCompatSendSite {
|
||||
@@ -145,7 +145,7 @@ pub const REQUEST_COMPAT_SEND_SITES: &[RequestCompatSendSite] = &[
|
||||
json_field: "file_info",
|
||||
bin_field: "file_info_bin",
|
||||
},
|
||||
json_encoder: "let file_info = compat_json(&fi, &policy)?;",
|
||||
json_encoder: "let file_info = compat_json(&fi)?;",
|
||||
policy: RequestJsonPolicy::MsgpackOnlyEligible,
|
||||
},
|
||||
RequestCompatSendSite {
|
||||
@@ -154,7 +154,7 @@ pub const REQUEST_COMPAT_SEND_SITES: &[RequestCompatSendSite] = &[
|
||||
json_field: "opts",
|
||||
bin_field: "opts_bin",
|
||||
},
|
||||
json_encoder: "let opts_str = compat_json(opts, &policy)?;",
|
||||
json_encoder: "let opts_str = compat_json(&opts)?;",
|
||||
policy: RequestJsonPolicy::MsgpackOnlyEligible,
|
||||
},
|
||||
RequestCompatSendSite {
|
||||
@@ -163,7 +163,7 @@ pub const REQUEST_COMPAT_SEND_SITES: &[RequestCompatSendSite] = &[
|
||||
json_field: "file_info",
|
||||
bin_field: "file_info_bin",
|
||||
},
|
||||
json_encoder: "let file_info = compat_json(&fi, &policy)?;",
|
||||
json_encoder: "let file_info = compat_json(&fi)?;",
|
||||
policy: RequestJsonPolicy::MsgpackOnlyEligible,
|
||||
},
|
||||
];
|
||||
|
||||
@@ -913,9 +913,8 @@ pub struct DeleteVersionsRequest {
|
||||
pub versions: ::prost::alloc::vec::Vec<::prost::alloc::string::String>,
|
||||
#[prost(string, tag = "4")]
|
||||
pub opts: ::prost::alloc::string::String,
|
||||
/// Senders dual-write for legacy peers. A nonempty versions_bin list determines
|
||||
/// the batch length and values. Malformed binary items reject the batch.
|
||||
/// An empty binary list selects the legacy versions list. Options decode independently.
|
||||
/// msgpack payloads mirroring the JSON fields above (grpc-optimization P2). Senders dual-write
|
||||
/// both; receivers prefer the \*\_bin form and fall back to the JSON string when it is empty.
|
||||
#[prost(bytes = "bytes", repeated, tag = "5")]
|
||||
pub versions_bin: ::prost::alloc::vec::Vec<::prost::bytes::Bytes>,
|
||||
#[prost(bytes = "bytes", tag = "6")]
|
||||
@@ -1029,9 +1028,6 @@ pub struct GenerallyLockResponse {
|
||||
/// Only populated by unlock RPCs.
|
||||
#[prost(enumeration = "LockReleaseStatus", tag = "4")]
|
||||
pub release_status: i32,
|
||||
/// Fixed-width optional namespace mutation proof.
|
||||
#[prost(bytes = "bytes", optional, tag = "6")]
|
||||
pub cache_fence_stamp: ::core::option::Option<::prost::bytes::Bytes>,
|
||||
}
|
||||
#[allow(clippy::all)]
|
||||
#[derive(Clone, PartialEq, Eq, Hash, ::prost::Message)]
|
||||
@@ -1055,9 +1051,6 @@ pub struct GenerallyLockResult {
|
||||
/// Only populated by lock batches.
|
||||
#[prost(enumeration = "LockAcquireStatus", tag = "5")]
|
||||
pub acquire_status: i32,
|
||||
/// Fixed-width optional namespace mutation proof.
|
||||
#[prost(bytes = "bytes", optional, tag = "6")]
|
||||
pub cache_fence_stamp: ::core::option::Option<::prost::bytes::Bytes>,
|
||||
}
|
||||
#[allow(clippy::all)]
|
||||
#[derive(Clone, PartialEq, ::prost::Message)]
|
||||
|
||||
+2
-104
@@ -141,7 +141,6 @@ impl Drop for DialLock {
|
||||
}
|
||||
|
||||
static INTERNODE_RPC_MSGPACK_ONLY_CACHE: AtomicU8 = AtomicU8::new(INTERNODE_RPC_MSGPACK_ONLY_CACHE_UNSET);
|
||||
static INTERNODE_RPC_MSGPACK_PEER_CAPABILITY_CACHE: AtomicU8 = AtomicU8::new(INTERNODE_RPC_MSGPACK_ONLY_CACHE_UNSET);
|
||||
|
||||
fn enforce_tls_generation_cache_bound(generation_cache: &mut HashMap<String, u64>, generation: u64, addr: &str) {
|
||||
if generation_cache.len() < TLS_GENERATION_CACHE_MAX_SIZE || generation_cache.contains_key(addr) {
|
||||
@@ -3423,35 +3422,9 @@ pub fn internode_rpc_msgpack_only() -> bool {
|
||||
enabled
|
||||
}
|
||||
|
||||
/// Whether msgpack-only was requested and the default-off per-peer capability gate is enabled.
|
||||
pub fn internode_rpc_msgpack_peer_capability_enabled() -> bool {
|
||||
match INTERNODE_RPC_MSGPACK_PEER_CAPABILITY_CACHE.load(Ordering::Acquire) {
|
||||
INTERNODE_RPC_MSGPACK_ONLY_CACHE_FALSE => return false,
|
||||
INTERNODE_RPC_MSGPACK_ONLY_CACHE_TRUE => return true,
|
||||
_ => {}
|
||||
}
|
||||
let enabled = rustfs_utils::get_env_bool(
|
||||
rustfs_config::ENV_INTERNODE_RPC_MSGPACK_ONLY,
|
||||
rustfs_config::DEFAULT_INTERNODE_RPC_MSGPACK_ONLY,
|
||||
) && rustfs_utils::get_env_bool(
|
||||
rustfs_config::ENV_INTERNODE_RPC_MSGPACK_PEER_CAPABILITY,
|
||||
rustfs_config::DEFAULT_INTERNODE_RPC_MSGPACK_PEER_CAPABILITY,
|
||||
);
|
||||
INTERNODE_RPC_MSGPACK_PEER_CAPABILITY_CACHE.store(
|
||||
if enabled {
|
||||
INTERNODE_RPC_MSGPACK_ONLY_CACHE_TRUE
|
||||
} else {
|
||||
INTERNODE_RPC_MSGPACK_ONLY_CACHE_FALSE
|
||||
},
|
||||
Ordering::Release,
|
||||
);
|
||||
enabled
|
||||
}
|
||||
|
||||
#[doc(hidden)]
|
||||
pub fn reset_internode_rpc_msgpack_only_cache() {
|
||||
INTERNODE_RPC_MSGPACK_ONLY_CACHE.store(INTERNODE_RPC_MSGPACK_ONLY_CACHE_UNSET, Ordering::Release);
|
||||
INTERNODE_RPC_MSGPACK_PEER_CAPABILITY_CACHE.store(INTERNODE_RPC_MSGPACK_ONLY_CACHE_UNSET, Ordering::Release);
|
||||
}
|
||||
|
||||
/// Consecutive-failure threshold after which an internode peer is marked offline (grpc-optimization
|
||||
@@ -3657,11 +3630,6 @@ fn complete_pending_rebuild_on_cache_hit(dial_addr: &str, cache_key: &str) -> bo
|
||||
// Limits transport response futures; client RPC metrics separately cover unary body decoding.
|
||||
const INTERNODE_RPC_CONCURRENCY_LIMIT: usize = 64;
|
||||
|
||||
// Quorum reads cancel surplus metadata RPCs in bursts. Keep bounded reset state
|
||||
// for their in-flight replies without spending the connection's protocol-error
|
||||
// budget (h2 remembers only 50 resets by default).
|
||||
const INTERNODE_RPC_MAX_CONCURRENT_RESET_STREAMS: usize = 4096;
|
||||
|
||||
/// Whether control/bulk channel isolation is enabled (env-gated, default off for safe rollout).
|
||||
fn channel_isolation_enabled() -> bool {
|
||||
rustfs_utils::get_env_bool(
|
||||
@@ -4115,7 +4083,6 @@ async fn dial_channel(
|
||||
let conn_window = checked_rpc_window(conn_window)?;
|
||||
let mut endpoint = Endpoint::from_shared(dial_addr.to_string())?
|
||||
.concurrency_limit(concurrency_limit)
|
||||
.http2_max_concurrent_reset_streams(INTERNODE_RPC_MAX_CONCURRENT_RESET_STREAMS)
|
||||
// HTTP/2 PING frames for application-layer health check
|
||||
.http2_keep_alive_interval(http2_keepalive_interval)
|
||||
// How long to wait for PING ACK before considering connection dead
|
||||
@@ -4272,9 +4239,6 @@ pub async fn evict_failed_connection_with_reason(
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod reset_stream_tests;
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
@@ -4669,8 +4633,8 @@ mod tests {
|
||||
),
|
||||
("DeleteVersionRequest", "file_info", RequestJsonPolicy::AlwaysDualWriteUntilFallbackZero,),
|
||||
("DeleteVersionRequest", "opts", RequestJsonPolicy::AlwaysDualWriteUntilFallbackZero),
|
||||
("DeleteVersionsRequest", "opts", RequestJsonPolicy::MsgpackOnlyEligible),
|
||||
("DeleteVersionsRequest", "versions", RequestJsonPolicy::MsgpackOnlyEligible),
|
||||
("DeleteVersionsRequest", "opts", RequestJsonPolicy::AlwaysDualWriteUntilFallbackZero,),
|
||||
("DeleteVersionsRequest", "versions", RequestJsonPolicy::AlwaysDualWriteUntilFallbackZero,),
|
||||
("ReadMultipleRequest", "read_multiple_req", RequestJsonPolicy::MsgpackOnlyEligible),
|
||||
("ReadVersionRequest", "opts", RequestJsonPolicy::MsgpackOnlyEligible),
|
||||
("RenameDataRequest", "file_info", RequestJsonPolicy::MsgpackOnlyEligible),
|
||||
@@ -5793,33 +5757,6 @@ mod tests {
|
||||
reset_internode_rpc_msgpack_only_cache();
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn internode_rpc_msgpack_peer_capability_requires_request_and_peer_gate() {
|
||||
let _guard = INTERNODE_RPC_MSGPACK_ONLY_ENV_LOCK.lock().unwrap_or_else(|e| e.into_inner());
|
||||
|
||||
for (requested, peer_capability, expected) in [
|
||||
(None, None, false),
|
||||
(Some("true"), None, false),
|
||||
(None, Some("true"), false),
|
||||
(Some("true"), Some("false"), false),
|
||||
(Some("false"), Some("true"), false),
|
||||
(Some("true"), Some("true"), true),
|
||||
] {
|
||||
reset_internode_rpc_msgpack_only_cache();
|
||||
temp_env::with_vars(
|
||||
[
|
||||
(rustfs_config::ENV_INTERNODE_RPC_MSGPACK_ONLY, requested),
|
||||
(rustfs_config::ENV_INTERNODE_RPC_MSGPACK_PEER_CAPABILITY, peer_capability),
|
||||
],
|
||||
|| {
|
||||
assert_eq!(internode_rpc_msgpack_peer_capability_enabled(), expected);
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
reset_internode_rpc_msgpack_only_cache();
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn get_channel_for_class_bulk_reuses_control_cache_when_isolation_disabled() {
|
||||
let _test_guard = TLS02_TEST_LOCK.lock().await;
|
||||
@@ -5880,7 +5817,6 @@ mod lock_release_tests {
|
||||
error_info: error.clone(),
|
||||
lock_info: None,
|
||||
release_status: status.into(),
|
||||
cache_fence_stamp: None,
|
||||
};
|
||||
let result = GenerallyLockResult {
|
||||
success,
|
||||
@@ -5936,42 +5872,4 @@ mod lock_release_tests {
|
||||
assert_eq!(decoded.acquire_status, 99);
|
||||
assert!(LockAcquireStatus::try_from(decoded.acquire_status).is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn cache_fence_stamp_is_additive_for_single_and_batch_lock_responses() {
|
||||
let stamp = prost::bytes::Bytes::from_static(b"fixed-width-cache-fence-proof");
|
||||
let response = GenerallyLockResponse {
|
||||
success: true,
|
||||
cache_fence_stamp: Some(stamp.clone()),
|
||||
..Default::default()
|
||||
};
|
||||
let result = GenerallyLockResult {
|
||||
success: true,
|
||||
cache_fence_stamp: Some(stamp),
|
||||
..Default::default()
|
||||
};
|
||||
for encoded in [response.encode_to_vec(), result.encode_to_vec()] {
|
||||
let legacy = LegacyReleaseResponse::decode(encoded.as_slice()).expect("old peer ignores additive cache proof");
|
||||
assert!(legacy.success);
|
||||
}
|
||||
|
||||
let legacy = LegacyReleaseResponse {
|
||||
success: true,
|
||||
error_info: None,
|
||||
lock_info: None,
|
||||
}
|
||||
.encode_to_vec();
|
||||
assert!(
|
||||
GenerallyLockResponse::decode(legacy.as_slice())
|
||||
.expect("new single response decodes legacy peer")
|
||||
.cache_fence_stamp
|
||||
.is_none()
|
||||
);
|
||||
assert!(
|
||||
GenerallyLockResult::decode(legacy.as_slice())
|
||||
.expect("new batch result decodes legacy peer")
|
||||
.cache_fence_stamp
|
||||
.is_none()
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -583,9 +583,8 @@ message DeleteVersionsRequest {
|
||||
string volume = 2;
|
||||
repeated string versions = 3;
|
||||
string opts = 4;
|
||||
// Senders dual-write for legacy peers. A nonempty versions_bin list determines
|
||||
// the batch length and values. Malformed binary items reject the batch.
|
||||
// An empty binary list selects the legacy versions list. Options decode independently.
|
||||
// msgpack payloads mirroring the JSON fields above (grpc-optimization P2). Senders dual-write
|
||||
// both; receivers prefer the *_bin form and fall back to the JSON string when it is empty.
|
||||
repeated bytes versions_bin = 5;
|
||||
bytes opts_bin = 6;
|
||||
}
|
||||
@@ -663,7 +662,6 @@ message GenerallyLockResponse {
|
||||
optional string error_info = 2;
|
||||
optional string lock_info = 3; // JSON serialized LockInfo
|
||||
LockReleaseStatus release_status = 4; // Only populated by unlock RPCs.
|
||||
optional bytes cache_fence_stamp = 6; // Fixed-width optional namespace mutation proof.
|
||||
}
|
||||
|
||||
message BatchGenerallyLockRequest {
|
||||
@@ -683,7 +681,6 @@ message GenerallyLockResult {
|
||||
optional string lock_info = 3; // JSON serialized LockInfo
|
||||
LockReleaseStatus release_status = 4; // Per-ID result for unlock batches.
|
||||
LockAcquireStatus acquire_status = 5; // Only populated by lock batches.
|
||||
optional bytes cache_fence_stamp = 6; // Fixed-width optional namespace mutation proof.
|
||||
}
|
||||
|
||||
message BatchGenerallyLockResponse {
|
||||
|
||||
@@ -1,186 +0,0 @@
|
||||
// Copyright 2024 RustFS Team
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
use std::io;
|
||||
use std::time::Duration;
|
||||
use tokio::io::{AsyncReadExt, AsyncWriteExt};
|
||||
use tokio::net::{TcpListener, TcpStream};
|
||||
use tokio::sync::oneshot;
|
||||
use tonic::codegen::{Service, http};
|
||||
|
||||
const CANCELLED_REQUESTS: usize = 64;
|
||||
|
||||
async fn write_frame(socket: &mut TcpStream, kind: u8, flags: u8, stream: u32, payload: &[u8]) -> io::Result<()> {
|
||||
let length = u32::try_from(payload.len()).expect("fixture frame length");
|
||||
let mut frame = Vec::with_capacity(9 + payload.len());
|
||||
frame.extend_from_slice(&length.to_be_bytes()[1..]);
|
||||
frame.extend_from_slice(&[kind, flags]);
|
||||
frame.extend_from_slice(&stream.to_be_bytes());
|
||||
frame.extend_from_slice(payload);
|
||||
socket.write_all(&frame).await
|
||||
}
|
||||
|
||||
async fn read_frame(socket: &mut TcpStream) -> io::Result<(u8, u8, u32, Vec<u8>)> {
|
||||
let mut header = [0; 9];
|
||||
socket.read_exact(&mut header).await?;
|
||||
let length = u32::from_be_bytes([0, header[0], header[1], header[2]]);
|
||||
let stream = u32::from_be_bytes(header[5..9].try_into().expect("stream ID bytes"));
|
||||
let mut payload = vec![0; usize::try_from(length).expect("fixture frame length")];
|
||||
socket.read_exact(&mut payload).await?;
|
||||
Ok((header[3], header[4], stream, payload))
|
||||
}
|
||||
|
||||
// A raw peer can release replies already in transit after observing CANCEL.
|
||||
// An h2 server API would discard those replies upon receiving the reset.
|
||||
async fn delayed_peer(
|
||||
listener: TcpListener,
|
||||
requests_seen: oneshot::Sender<()>,
|
||||
resets_seen: oneshot::Sender<usize>,
|
||||
) -> io::Result<usize> {
|
||||
let (mut socket, _) = listener.accept().await?;
|
||||
socket.set_nodelay(true)?;
|
||||
let mut preface = [0; 24];
|
||||
socket.read_exact(&mut preface).await?;
|
||||
assert_eq!(&preface, b"PRI * HTTP/2.0\r\n\r\nSM\r\n\r\n");
|
||||
write_frame(&mut socket, 4, 0, 0, &[]).await?;
|
||||
let mut requests = Vec::new();
|
||||
while requests.len() < CANCELLED_REQUESTS {
|
||||
let (kind, flags, stream, _) = read_frame(&mut socket).await?;
|
||||
match (kind, flags) {
|
||||
(4, 0) => write_frame(&mut socket, 4, 1, 0, &[]).await?,
|
||||
(1, _) => requests.push(stream),
|
||||
_ => {}
|
||||
}
|
||||
}
|
||||
requests_seen.send(()).expect("caller waits for dispatched requests");
|
||||
let mut cancellations = 0;
|
||||
while cancellations < CANCELLED_REQUESTS {
|
||||
let (kind, _, stream, payload) = read_frame(&mut socket).await?;
|
||||
if kind == 3 {
|
||||
assert!(requests.contains(&stream), "reset belongs to a dispatched request");
|
||||
assert_eq!(payload, 8_u32.to_be_bytes(), "caller cancellation sends CANCEL");
|
||||
cancellations += 1;
|
||||
}
|
||||
}
|
||||
for stream in requests {
|
||||
// HPACK static index 8 is ':status: 200'; END_HEADERS | END_STREAM.
|
||||
write_frame(&mut socket, 1, 5, stream, &[0x88]).await?;
|
||||
}
|
||||
let ping = b"rst-test";
|
||||
write_frame(&mut socket, 6, 0, 0, ping).await?;
|
||||
let mut error_resets = 0;
|
||||
loop {
|
||||
let (kind, flags, _, payload) = read_frame(&mut socket).await?;
|
||||
match (kind, flags) {
|
||||
(3, _) => {
|
||||
assert_eq!(payload, 5_u32.to_be_bytes(), "late reply rejected as STREAM_CLOSED");
|
||||
error_resets += 1;
|
||||
}
|
||||
(6, 1) if payload == ping => break,
|
||||
(7, _) => panic!("late replies closed the shared connection: {payload:?}"),
|
||||
_ => {}
|
||||
}
|
||||
}
|
||||
// PING acknowledgment establishes that all preceding replies were processed.
|
||||
resets_seen.send(error_resets).expect("caller waits for reset accounting");
|
||||
loop {
|
||||
let (kind, _, stream, payload) = read_frame(&mut socket).await?;
|
||||
if kind == 3 {
|
||||
assert_eq!(payload, 5_u32.to_be_bytes());
|
||||
error_resets += 1;
|
||||
}
|
||||
if kind == 1 {
|
||||
write_frame(&mut socket, 1, 5, stream, &[0x88]).await?;
|
||||
return Ok(error_resets);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
async fn cancelled_metadata_replies_keep_channel_usable(bulk: bool) {
|
||||
tokio::time::timeout(Duration::from_secs(10), async {
|
||||
let listener = TcpListener::bind("127.0.0.1:0").await.expect("bind fixture peer");
|
||||
let address = format!("http://{}", listener.local_addr().expect("fixture address"));
|
||||
let key = if bulk {
|
||||
super::bulk_cache_key(&address, 0)
|
||||
} else {
|
||||
address.clone()
|
||||
};
|
||||
let (requests_tx, requests_rx) = oneshot::channel();
|
||||
let (resets_tx, resets_rx) = oneshot::channel();
|
||||
let server = tokio::spawn(delayed_peer(listener, requests_tx, resets_tx));
|
||||
let (channel, _) = super::dial_channel(&address, &key, Duration::from_secs(10))
|
||||
.await
|
||||
.expect("production channel connects");
|
||||
let mut calls = Vec::new();
|
||||
for _ in 0..CANCELLED_REQUESTS {
|
||||
let mut channel = channel.clone();
|
||||
let uri = format!("{address}/node_service.NodeService/ReadVersion");
|
||||
calls.push(tokio::spawn(async move {
|
||||
std::future::poll_fn(|cx| channel.poll_ready(cx))
|
||||
.await
|
||||
.expect("channel ready");
|
||||
channel
|
||||
.call(
|
||||
http::Request::builder()
|
||||
.uri(uri)
|
||||
.body(tonic::body::Body::empty())
|
||||
.expect("read request"),
|
||||
)
|
||||
.await
|
||||
}));
|
||||
}
|
||||
requests_rx.await.expect("all metadata requests dispatched");
|
||||
for call in &calls {
|
||||
call.abort();
|
||||
}
|
||||
for call in calls {
|
||||
assert!(call.await.expect_err("request cancelled before response").is_cancelled());
|
||||
}
|
||||
let _ = resets_rx.await.expect("peer accounts for late response frames");
|
||||
let mut channel = channel;
|
||||
std::future::poll_fn(|cx| channel.poll_ready(cx))
|
||||
.await
|
||||
.expect("channel remains ready");
|
||||
let response = channel
|
||||
.call(
|
||||
http::Request::builder()
|
||||
.uri(format!("{address}/node_service.NodeService/Ping"))
|
||||
.body(tonic::body::Body::empty())
|
||||
.expect("probe request"),
|
||||
)
|
||||
.await
|
||||
.expect("subsequent RPC shares the connection");
|
||||
assert_eq!(response.status(), http::StatusCode::OK);
|
||||
let error_resets = server
|
||||
.await
|
||||
.expect("fixture task completes")
|
||||
.expect("fixture transport succeeds");
|
||||
assert_eq!(
|
||||
error_resets, 0,
|
||||
"cancelled metadata replies must not spend the connection's protocol-error budget"
|
||||
);
|
||||
})
|
||||
.await
|
||||
.expect("bounded cancellation exchange");
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn control_channel_tolerates_late_cancelled_metadata_replies() {
|
||||
cancelled_metadata_replies_keep_channel_usable(false).await;
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn bulk_channel_tolerates_late_cancelled_metadata_replies() {
|
||||
cancelled_metadata_replies_keep_channel_usable(true).await;
|
||||
}
|
||||
@@ -411,10 +411,13 @@ impl ListObjectsOptions {
|
||||
|
||||
fn decode_s3_name(name: &str, encoding_type: &str) -> Result<String, std::io::Error> {
|
||||
match encoding_type {
|
||||
"url" => urlencoding::decode(name)
|
||||
.map(|decoded| decoded.into_owned())
|
||||
.map_err(|err| std::io::Error::new(ErrorKind::InvalidData, err)),
|
||||
_ => Ok(name.to_string()),
|
||||
"url" => {
|
||||
//return url::QueryUnescape(name);
|
||||
return Ok(name.to_string());
|
||||
}
|
||||
_ => {
|
||||
return Ok(name.to_string());
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -446,170 +449,6 @@ mod tests {
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn url_encoded_s3_names_are_decoded() {
|
||||
for (encoded, expected) in [
|
||||
("test_file%283%29.png", "test_file(3).png"),
|
||||
("dir%20a/file.txt", "dir a/file.txt"),
|
||||
("%E4%B8%AD%E6%96%87.txt", "中文.txt"),
|
||||
("%F0%9F%98%80.txt", "😀.txt"),
|
||||
("file%2bname+%25.txt", "file+name+%.txt"),
|
||||
("literal%252F.txt", "literal%2F.txt"),
|
||||
("dir%2Ffile%00%01.txt", "dir/file\0\u{1}.txt"),
|
||||
("", ""),
|
||||
] {
|
||||
assert_eq!(decode_s3_name(encoded, "url").expect("valid URL-encoded S3 key"), expected);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn s3_names_without_url_encoding_are_preserved() {
|
||||
for encoding in ["", "unknown"] {
|
||||
let name = "中文+%20%FF.txt";
|
||||
assert_eq!(decode_s3_name(name, encoding).expect("unencoded key should pass through"), name);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn url_encoded_s3_names_reject_invalid_utf8() {
|
||||
for name in ["%FF", "%C3%28", "%E4%B8", "%ED%A0%80", "%C0%AF"] {
|
||||
let err = decode_s3_name(name, "url").expect_err("invalid UTF-8 must not become a different object key");
|
||||
assert_eq!(err.kind(), ErrorKind::InvalidData);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn url_encoded_s3_names_preserve_malformed_percent_escapes() {
|
||||
// Match urlencoding's permissive handling without treating '+' as a space.
|
||||
for name in ["file%", "file%2", "file%GG+name"] {
|
||||
assert_eq!(decode_s3_name(name, "url").expect("literal malformed escape"), name);
|
||||
}
|
||||
}
|
||||
|
||||
async fn list_response_fixture(responses: Vec<&'static str>) -> (TransitionClient, tokio::task::JoinHandle<Vec<url::Url>>) {
|
||||
let listener = TcpListener::bind("127.0.0.1:0").await.expect("list fixture should bind");
|
||||
let endpoint = listener.local_addr().expect("list fixture address").to_string();
|
||||
let fixture = tokio::spawn(async move {
|
||||
tokio::time::timeout(Duration::from_secs(5), async move {
|
||||
let mut requests = Vec::new();
|
||||
for body in responses {
|
||||
let (mut stream, _) = listener.accept().await.expect("list fixture should accept request");
|
||||
let mut request = Vec::new();
|
||||
let mut buffer = [0; 1024];
|
||||
while !request.windows(4).any(|window| window == b"\r\n\r\n") {
|
||||
let read = stream.read(&mut buffer).await.expect("list fixture should read request");
|
||||
assert_ne!(read, 0, "request headers must be complete");
|
||||
request.extend_from_slice(&buffer[..read]);
|
||||
assert!(request.len() <= 16 * 1024, "fixture request headers should be bounded");
|
||||
}
|
||||
let request = std::str::from_utf8(&request).expect("request headers should be UTF-8");
|
||||
let target = request.split_whitespace().nth(1).expect("request should have a target");
|
||||
requests.push(url::Url::parse(&format!("http://localhost{target}")).expect("request URL should parse"));
|
||||
let response = format!(
|
||||
"HTTP/1.1 200 OK\r\nContent-Type: application/xml\r\nContent-Length: {}\r\nConnection: close\r\n\r\n{body}",
|
||||
body.len()
|
||||
);
|
||||
stream.write_all(response.as_bytes()).await.expect("list fixture should send response");
|
||||
}
|
||||
requests
|
||||
})
|
||||
.await
|
||||
.expect("list fixture should finish within its deadline")
|
||||
});
|
||||
let client = TransitionClient::new_with_timeouts(
|
||||
&endpoint,
|
||||
timeout_test_options(),
|
||||
"",
|
||||
TransitionClientTimeouts::new(Duration::from_secs(1), Duration::from_secs(1), Duration::from_secs(1)),
|
||||
)
|
||||
.await
|
||||
.expect("list fixture client should build");
|
||||
client
|
||||
.bucket_loc_cache
|
||||
.lock()
|
||||
.expect("location cache should lock")
|
||||
.set("bucket", "us-east-1");
|
||||
(client, fixture)
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn list_objects_v2_decodes_prefixes_and_preserves_continuation_tokens() {
|
||||
let (client, fixture) = list_response_fixture(vec![
|
||||
r#"<ListBucketResult><EncodingType>url</EncodingType><IsTruncated>true</IsTruncated>
|
||||
<CommonPrefixes><Prefix>dir%20a/</Prefix></CommonPrefixes>
|
||||
<NextContinuationToken>opaque%2F+token</NextContinuationToken></ListBucketResult>"#,
|
||||
"<ListBucketResult><EncodingType>url</EncodingType><IsTruncated>false</IsTruncated></ListBucketResult>",
|
||||
])
|
||||
.await;
|
||||
let first = client
|
||||
.list_objects_v2_query("bucket", "", "", false, false, "/", "", 1, HeaderMap::new())
|
||||
.await
|
||||
.expect("first object page should decode");
|
||||
assert_eq!(first.common_prefixes[0].prefix, "dir a/");
|
||||
assert_eq!(first.next_continuation_token, "opaque%2F+token");
|
||||
let second = client
|
||||
.list_objects_v2_query("bucket", "", &first.next_continuation_token, false, false, "/", "", 1, HeaderMap::new())
|
||||
.await
|
||||
.expect("second object page should succeed");
|
||||
assert!(!second.is_truncated);
|
||||
let requests = fixture.await.expect("object list fixture should join");
|
||||
let query: HashMap<_, _> = requests[1].query_pairs().into_owned().collect();
|
||||
assert_eq!(query.get("continuation-token").map(String::as_str), Some("opaque%2F+token"));
|
||||
assert_eq!(query.get("encoding-type").map(String::as_str), Some("url"));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn list_versions_decodes_names_and_round_trips_pagination_markers() {
|
||||
let (client, fixture) = list_response_fixture(vec![
|
||||
r#"<ListVersionsResult><EncodingType>url</EncodingType><IsTruncated>true</IsTruncated>
|
||||
<Version><Key>dir%20a/%E4%B8%AD%E6%96%87%2B%252F.txt</Key><VersionId>version%2F+id</VersionId></Version>
|
||||
<DeleteMarker><Key>deleted%20file%2B.txt</Key><VersionId>delete%2F+id</VersionId></DeleteMarker>
|
||||
<CommonPrefixes><Prefix>dir%20a/</Prefix></CommonPrefixes>
|
||||
<NextKeyMarker>dir%20a/%E4%B8%AD%E6%96%87%2B%252F.txt</NextKeyMarker>
|
||||
<NextVersionIdMarker>version%2F+id</NextVersionIdMarker></ListVersionsResult>"#,
|
||||
"<ListVersionsResult><EncodingType>url</EncodingType><IsTruncated>false</IsTruncated></ListVersionsResult>",
|
||||
])
|
||||
.await;
|
||||
let mut opts = ListObjectsOptions::default();
|
||||
opts.set("prefix", "dir a/");
|
||||
let first = client
|
||||
.list_object_versions_query("bucket", &opts, "", "", "/")
|
||||
.await
|
||||
.expect("first version page");
|
||||
assert_eq!(first.versions[0].key, "dir a/中文+%2F.txt");
|
||||
assert_eq!(first.versions[0].version_id, "version%2F+id");
|
||||
assert_eq!(first.delete_markers[0].key, "deleted file+.txt");
|
||||
assert_eq!(first.delete_markers[0].version_id, "delete%2F+id");
|
||||
assert_eq!(first.common_prefixes[0].prefix, "dir a/");
|
||||
assert_eq!(first.next_key_marker, "dir a/中文+%2F.txt");
|
||||
assert_eq!(first.next_version_id_marker, "version%2F+id");
|
||||
let second = client
|
||||
.list_object_versions_query("bucket", &opts, &first.next_key_marker, &first.next_version_id_marker, "/")
|
||||
.await
|
||||
.expect("second version page");
|
||||
assert!(!second.is_truncated);
|
||||
let requests = fixture.await.expect("version list fixture should join");
|
||||
let query: HashMap<_, _> = requests[1].query_pairs().into_owned().collect();
|
||||
assert_eq!(query.get("key-marker").map(String::as_str), Some("dir a/中文+%2F.txt"));
|
||||
assert_eq!(query.get("version-id-marker").map(String::as_str), Some("version%2F+id"));
|
||||
assert_eq!(query.get("prefix").map(String::as_str), Some("dir a/"));
|
||||
assert_eq!(query.get("encoding-type").map(String::as_str), Some("url"));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn list_versions_rejects_invalid_utf8_keys() {
|
||||
let (client, fixture) = list_response_fixture(vec![
|
||||
"<ListVersionsResult><EncodingType>url</EncodingType><Version><Key>%FF</Key></Version></ListVersionsResult>",
|
||||
])
|
||||
.await;
|
||||
let err = client
|
||||
.list_object_versions_query("bucket", &ListObjectsOptions::default(), "", "", "")
|
||||
.await
|
||||
.expect_err("invalid UTF-8 in a remote key should fail the listing");
|
||||
assert_eq!(err.kind(), ErrorKind::InvalidData);
|
||||
assert_eq!(fixture.await.expect("invalid key fixture should join").len(), 1);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn list_versions_xml_preserves_versions_and_delete_markers() {
|
||||
let xml = br#"
|
||||
|
||||
@@ -83,14 +83,6 @@ pub enum S3Operation {
|
||||
SelectObjectContent,
|
||||
UploadPart,
|
||||
UploadPartCopy,
|
||||
GetBucketAccelerateConfiguration,
|
||||
GetBucketRequestPayment,
|
||||
GetBucketWebsite,
|
||||
PutBucketAccelerateConfiguration,
|
||||
PutBucketRequestPayment,
|
||||
PutBucketWebsite,
|
||||
DeleteBucketWebsite,
|
||||
PostObject,
|
||||
}
|
||||
|
||||
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
|
||||
@@ -134,7 +126,7 @@ impl EventMapping {
|
||||
}
|
||||
|
||||
impl S3Operation {
|
||||
pub const ALL: [Self; 74] = [
|
||||
pub const ALL: [Self; 66] = [
|
||||
Self::AbortMultipartUpload,
|
||||
Self::CompleteMultipartUpload,
|
||||
Self::CopyObject,
|
||||
@@ -201,14 +193,6 @@ impl S3Operation {
|
||||
Self::SelectObjectContent,
|
||||
Self::UploadPart,
|
||||
Self::UploadPartCopy,
|
||||
Self::GetBucketAccelerateConfiguration,
|
||||
Self::GetBucketRequestPayment,
|
||||
Self::GetBucketWebsite,
|
||||
Self::PutBucketAccelerateConfiguration,
|
||||
Self::PutBucketRequestPayment,
|
||||
Self::PutBucketWebsite,
|
||||
Self::DeleteBucketWebsite,
|
||||
Self::PostObject,
|
||||
];
|
||||
|
||||
#[inline]
|
||||
@@ -230,12 +214,10 @@ impl S3Operation {
|
||||
Self::DeleteBucketPolicy => "s3:DeleteBucketPolicy",
|
||||
Self::DeleteBucketReplication => "s3:DeleteBucketReplication",
|
||||
Self::DeleteBucketTagging => "s3:DeleteBucketTagging",
|
||||
Self::DeleteBucketWebsite => "s3:DeleteBucketWebsite",
|
||||
Self::DeleteObject => "s3:DeleteObject",
|
||||
Self::DeleteObjectTagging => "s3:DeleteObjectTagging",
|
||||
Self::DeleteObjects => "s3:DeleteObjects",
|
||||
Self::DeletePublicAccessBlock => "s3:DeletePublicAccessBlock",
|
||||
Self::GetBucketAccelerateConfiguration => "s3:GetBucketAccelerateConfiguration",
|
||||
Self::GetBucketAcl => "s3:GetBucketAcl",
|
||||
Self::GetBucketCors => "s3:GetBucketCors",
|
||||
Self::GetBucketEncryption => "s3:GetBucketEncryption",
|
||||
@@ -246,10 +228,8 @@ impl S3Operation {
|
||||
Self::GetBucketPolicy => "s3:GetBucketPolicy",
|
||||
Self::GetBucketPolicyStatus => "s3:GetBucketPolicyStatus",
|
||||
Self::GetBucketReplication => "s3:GetBucketReplication",
|
||||
Self::GetBucketRequestPayment => "s3:GetBucketRequestPayment",
|
||||
Self::GetBucketTagging => "s3:GetBucketTagging",
|
||||
Self::GetBucketVersioning => "s3:GetBucketVersioning",
|
||||
Self::GetBucketWebsite => "s3:GetBucketWebsite",
|
||||
Self::GetObject => "s3:GetObject",
|
||||
Self::GetObjectAcl => "s3:GetObjectAcl",
|
||||
Self::GetObjectAttributes => "s3:GetObjectAttributes",
|
||||
@@ -267,7 +247,6 @@ impl S3Operation {
|
||||
Self::ListObjects => "s3:ListObjects",
|
||||
Self::ListObjectsV2 => "s3:ListObjectsV2",
|
||||
Self::ListParts => "s3:ListParts",
|
||||
Self::PutBucketAccelerateConfiguration => "s3:PutBucketAccelerateConfiguration",
|
||||
Self::PutBucketAcl => "s3:PutBucketAcl",
|
||||
Self::PutBucketCors => "s3:PutBucketCors",
|
||||
Self::PutBucketEncryption => "s3:PutBucketEncryption",
|
||||
@@ -276,11 +255,8 @@ impl S3Operation {
|
||||
Self::PutBucketNotificationConfiguration => "s3:PutBucketNotificationConfiguration",
|
||||
Self::PutBucketPolicy => "s3:PutBucketPolicy",
|
||||
Self::PutBucketReplication => "s3:PutBucketReplication",
|
||||
Self::PutBucketRequestPayment => "s3:PutBucketRequestPayment",
|
||||
Self::PutBucketTagging => "s3:PutBucketTagging",
|
||||
Self::PutBucketVersioning => "s3:PutBucketVersioning",
|
||||
Self::PutBucketWebsite => "s3:PutBucketWebsite",
|
||||
Self::PostObject => "s3:PostObject",
|
||||
Self::PutObject => "s3:PutObject",
|
||||
Self::PutObjectAcl => "s3:PutObjectAcl",
|
||||
Self::PutObjectLegalHold => "s3:PutObjectLegalHold",
|
||||
@@ -312,7 +288,6 @@ impl S3Operation {
|
||||
Self::GetObjectLegalHold => EventMapping::Single(EventName::ObjectAccessedGetLegalHold),
|
||||
Self::GetObjectRetention => EventMapping::Single(EventName::ObjectAccessedGetRetention),
|
||||
Self::HeadObject => EventMapping::Single(EventName::ObjectAccessedHead),
|
||||
Self::PostObject => EventMapping::Single(EventName::ObjectCreatedPost),
|
||||
Self::PutObject => EventMapping::PutObject,
|
||||
Self::PutObjectAcl => EventMapping::Single(EventName::ObjectAclPut),
|
||||
Self::PutObjectLegalHold => EventMapping::Single(EventName::ObjectCreatedPutLegalHold),
|
||||
@@ -326,9 +301,7 @@ impl S3Operation {
|
||||
| Self::DeleteBucketPolicy
|
||||
| Self::DeleteBucketReplication
|
||||
| Self::DeleteBucketTagging
|
||||
| Self::DeleteBucketWebsite
|
||||
| Self::DeletePublicAccessBlock
|
||||
| Self::GetBucketAccelerateConfiguration
|
||||
| Self::GetBucketAcl
|
||||
| Self::GetBucketCors
|
||||
| Self::GetBucketEncryption
|
||||
@@ -339,10 +312,8 @@ impl S3Operation {
|
||||
| Self::GetBucketPolicy
|
||||
| Self::GetBucketPolicyStatus
|
||||
| Self::GetBucketReplication
|
||||
| Self::GetBucketRequestPayment
|
||||
| Self::GetBucketTagging
|
||||
| Self::GetBucketVersioning
|
||||
| Self::GetBucketWebsite
|
||||
| Self::GetObjectAcl
|
||||
| Self::GetObjectLockConfiguration
|
||||
| Self::GetObjectTagging
|
||||
@@ -355,7 +326,6 @@ impl S3Operation {
|
||||
| Self::ListObjects
|
||||
| Self::ListObjectsV2
|
||||
| Self::ListParts
|
||||
| Self::PutBucketAccelerateConfiguration
|
||||
| Self::PutBucketAcl
|
||||
| Self::PutBucketCors
|
||||
| Self::PutBucketEncryption
|
||||
@@ -364,10 +334,8 @@ impl S3Operation {
|
||||
| Self::PutBucketNotificationConfiguration
|
||||
| Self::PutBucketPolicy
|
||||
| Self::PutBucketReplication
|
||||
| Self::PutBucketRequestPayment
|
||||
| Self::PutBucketTagging
|
||||
| Self::PutBucketVersioning
|
||||
| Self::PutBucketWebsite
|
||||
| Self::PutObjectLockConfiguration
|
||||
| Self::PutPublicAccessBlock
|
||||
| Self::UploadPart
|
||||
@@ -548,29 +516,12 @@ mod tests {
|
||||
assert_eq!(mask & EventName::ObjectRemovedDelete.mask(), 0);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn metric_indexes_and_labels_identify_each_operation() {
|
||||
let mut labels = std::collections::HashSet::new();
|
||||
for (index, op) in S3Operation::ALL.into_iter().enumerate() {
|
||||
assert_eq!(op.metric_index(), index, "metric index for {op:?}");
|
||||
assert!(labels.insert(op.as_str()), "repeated metric label for {op:?}");
|
||||
}
|
||||
assert_eq!(S3Operation::UploadPartCopy.metric_index(), 65);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_operations_without_event_mapping_remain_unmapped() {
|
||||
let unmapped = [
|
||||
S3Operation::DeleteBucketWebsite,
|
||||
S3Operation::GetBucketAccelerateConfiguration,
|
||||
S3Operation::GetBucketAcl,
|
||||
S3Operation::GetBucketRequestPayment,
|
||||
S3Operation::GetBucketWebsite,
|
||||
S3Operation::ListObjectsV2,
|
||||
S3Operation::PutBucketAccelerateConfiguration,
|
||||
S3Operation::PutBucketNotificationConfiguration,
|
||||
S3Operation::PutBucketRequestPayment,
|
||||
S3Operation::PutBucketWebsite,
|
||||
S3Operation::UploadPart,
|
||||
];
|
||||
for op in unmapped {
|
||||
|
||||
@@ -15,6 +15,7 @@
|
||||
pub mod init_time;
|
||||
pub mod last_minute;
|
||||
pub mod metrics;
|
||||
pub mod scan_observation;
|
||||
pub mod startup;
|
||||
pub mod usage_candidate;
|
||||
|
||||
|
||||
@@ -13,6 +13,9 @@
|
||||
// limitations under the License.
|
||||
|
||||
use crate::last_minute::{AccElem, LastMinuteLatency};
|
||||
use crate::scan_observation::{
|
||||
ScannerMetadataReadGuard, ScannerMetadataReadOutcome, ScannerScanObservation, ScannerScanObservationSnapshot,
|
||||
};
|
||||
use crate::usage_candidate::{
|
||||
ScannerUsageCandidateCount, ScannerUsageCandidateKind, ScannerUsageCandidateOutcome, USAGE_CANDIDATE_COUNTER_SLOTS,
|
||||
};
|
||||
@@ -999,6 +1002,7 @@ impl ScannerUsagePublicationObservation {
|
||||
|
||||
pub struct Metrics {
|
||||
startup: crate::startup::ScannerStartupObservation,
|
||||
scan_observation: ScannerScanObservation,
|
||||
operations: Vec<AtomicU64>,
|
||||
latency: Vec<LockedLastMinuteLatency>,
|
||||
actions: Vec<AtomicU64>,
|
||||
@@ -1653,6 +1657,9 @@ pub struct ScannerMetricsReport {
|
||||
pub lifecycle_transition: ScannerLifecycleTransitionSnapshot,
|
||||
#[serde(default)]
|
||||
pub usage_freshness: ScannerUsageFreshnessSnapshot,
|
||||
/// Node-local, process-lifetime work observations. Older nodes have no evidence.
|
||||
#[serde(default)]
|
||||
pub scan_observation: ScannerScanObservationSnapshot,
|
||||
/// Process-lifetime candidate-construction results. Missing on older reports
|
||||
/// means unobserved, not accepted or healthy. Publication is tracked separately.
|
||||
#[serde(default)]
|
||||
@@ -2147,9 +2154,38 @@ impl Metrics {
|
||||
&self.startup
|
||||
}
|
||||
|
||||
pub fn scan_observation(&self) -> &ScannerScanObservation {
|
||||
&self.scan_observation
|
||||
}
|
||||
|
||||
/// Start after scanner node admission, before polling the disk read future.
|
||||
/// Disk-permit and RPC waits remain included; decoding is excluded. Dropping
|
||||
/// an unfinished guard observes cancellation, not physical I/O completion.
|
||||
pub fn start_scanner_metadata_read(&self) -> ScannerMetadataReadGuard<'_> {
|
||||
self.scan_observation.start_metadata_read();
|
||||
ScannerMetadataReadGuard {
|
||||
metrics: self,
|
||||
started: std::time::Instant::now(),
|
||||
outcome: ScannerMetadataReadOutcome::Cancelled,
|
||||
bytes: 0,
|
||||
}
|
||||
}
|
||||
|
||||
pub(crate) fn finish_scanner_metadata_read(&self, outcome: ScannerMetadataReadOutcome, bytes: u64, elapsed: Duration) {
|
||||
let bytes = if outcome == ScannerMetadataReadOutcome::Success {
|
||||
bytes
|
||||
} else {
|
||||
0
|
||||
};
|
||||
self.scan_observation.finish_metadata_read(outcome, bytes, elapsed);
|
||||
self.operations[Metric::ReadMetadata as usize].fetch_add(1, Ordering::Relaxed);
|
||||
self.latency[Metric::ReadMetadata as usize].add_size(elapsed, bytes);
|
||||
}
|
||||
|
||||
pub fn new() -> Self {
|
||||
Self {
|
||||
startup: crate::startup::ScannerStartupObservation::default(),
|
||||
scan_observation: ScannerScanObservation::default(),
|
||||
operations: (0..Metric::Last as usize).map(|_| AtomicU64::new(0)).collect(),
|
||||
// Each slot gets its own fresh LockedLastMinuteLatency so that
|
||||
// different metrics never accidentally share state.
|
||||
@@ -2755,6 +2791,8 @@ impl Metrics {
|
||||
/// walked the whole namespace, so an operator can tell why scoped
|
||||
/// convergence is not happening.
|
||||
pub fn record_scanner_scope_fallback(&self, reason: impl Into<String>) {
|
||||
let reason = reason.into();
|
||||
self.scan_observation.record_scope_fallback(&reason);
|
||||
self.scanner_scope_fallback_total.fetch_add(1, Ordering::Relaxed);
|
||||
self.scanner_scope_fallback_last_unix_secs
|
||||
.store(unix_now_secs(), Ordering::Relaxed);
|
||||
@@ -2762,7 +2800,7 @@ impl Metrics {
|
||||
Ok(guard) => guard,
|
||||
Err(poisoned) => poisoned.into_inner(),
|
||||
};
|
||||
*last_reason = reason.into();
|
||||
*last_reason = reason;
|
||||
}
|
||||
|
||||
pub fn record_scanner_usage_durable_success(&self) {
|
||||
@@ -3897,6 +3935,7 @@ impl Metrics {
|
||||
failed: self.scanner_transition_failed.load(Ordering::Relaxed),
|
||||
};
|
||||
m.usage_candidates = self.usage_candidate_counts();
|
||||
m.scan_observation = self.scan_observation.snapshot();
|
||||
let usage_save_result = ScannerUsageSaveResult::from_code(self.scanner_usage_last_save_result.load(Ordering::Relaxed));
|
||||
m.usage_freshness = ScannerUsageFreshnessSnapshot {
|
||||
dirty_pending_buckets: self.scanner_dirty_usage_pending_buckets.load(Ordering::Relaxed),
|
||||
|
||||
@@ -0,0 +1,675 @@
|
||||
// Copyright 2024 RustFS Team
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
//! Node-local, process-lifetime observations of scanner work. Counters are
|
||||
//! independently loaded, so concurrent reports need not balance exactly.
|
||||
//! These observations never grant cache reuse or publication authority.
|
||||
|
||||
use crate::metrics::Metrics;
|
||||
use serde::{Deserialize, Serialize};
|
||||
use std::{
|
||||
collections::BTreeMap,
|
||||
sync::atomic::{AtomicU64, Ordering},
|
||||
time::{Duration, Instant},
|
||||
};
|
||||
|
||||
pub const PREFIX_REUSE_BLOCKERS: [&str; 8] = [
|
||||
"missing_producer_identity",
|
||||
"missing_durable_journal_replay",
|
||||
"restart_gap",
|
||||
"generation_gap",
|
||||
"overflow",
|
||||
"stale_ack_generation",
|
||||
"missing_cold_zero_walk_oracle",
|
||||
"distributed_without_peer_invalidation",
|
||||
];
|
||||
|
||||
const SCOPE_FALLBACK_REASONS: [&str; 6] = [
|
||||
"dirty_snapshot_incomplete",
|
||||
"dirty_generation_exhausted",
|
||||
"dirty_bucket_overflow",
|
||||
"no_listed_dirty_buckets",
|
||||
"baseline_unavailable",
|
||||
"peer_scope_unavailable",
|
||||
];
|
||||
|
||||
/// A storage read result, independent of later metadata decoding or ILM work.
|
||||
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
|
||||
pub enum ScannerMetadataReadOutcome {
|
||||
Success,
|
||||
NotFound,
|
||||
Error,
|
||||
Cancelled,
|
||||
}
|
||||
|
||||
/// The existing path that actually copied a cached subtree instead of walking it.
|
||||
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
|
||||
pub enum ScannerCacheReuseKind {
|
||||
CleanPrefix,
|
||||
CoveredNamespace,
|
||||
SampledNamespace,
|
||||
ClassifiedNamespace,
|
||||
}
|
||||
|
||||
impl ScannerCacheReuseKind {
|
||||
const ALL: [Self; 4] = [
|
||||
Self::CleanPrefix,
|
||||
Self::CoveredNamespace,
|
||||
Self::SampledNamespace,
|
||||
Self::ClassifiedNamespace,
|
||||
];
|
||||
|
||||
fn label(self) -> &'static str {
|
||||
match self {
|
||||
Self::CleanPrefix => "clean_prefix",
|
||||
Self::CoveredNamespace => "covered_namespace",
|
||||
Self::SampledNamespace => "sampled_namespace",
|
||||
Self::ClassifiedNamespace => "classified_namespace",
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Completed bucket-level reuse work, independent of durable publication.
|
||||
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
|
||||
pub enum ScannerBucketReuseKind {
|
||||
ScopedColdBucket,
|
||||
CurrentRoot,
|
||||
}
|
||||
|
||||
impl ScannerBucketReuseKind {
|
||||
const ALL: [Self; 2] = [Self::ScopedColdBucket, Self::CurrentRoot];
|
||||
|
||||
fn label(self) -> &'static str {
|
||||
match self {
|
||||
Self::ScopedColdBucket => "scoped_cold_bucket",
|
||||
Self::CurrentRoot => "current_root",
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// A production folder gate, not a diagnostic preflight.
|
||||
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
|
||||
pub enum ScannerCacheReuseGate {
|
||||
CleanPrefix,
|
||||
CoveredNamespace,
|
||||
NamespaceSampling,
|
||||
}
|
||||
|
||||
impl ScannerCacheReuseGate {
|
||||
const ALL: [Self; 3] = [Self::CleanPrefix, Self::CoveredNamespace, Self::NamespaceSampling];
|
||||
|
||||
fn label(self) -> &'static str {
|
||||
match self {
|
||||
Self::CleanPrefix => "clean_prefix",
|
||||
Self::CoveredNamespace => "covered_namespace",
|
||||
Self::NamespaceSampling => "namespace_sampling",
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// The first existing condition reached by a folder reuse gate.
|
||||
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
|
||||
pub enum ScannerCacheReuseDecision {
|
||||
MissingScope,
|
||||
NoResumeSweep,
|
||||
DeepScan,
|
||||
ForwardSweep,
|
||||
ActiveReplication,
|
||||
NotRoot,
|
||||
CompactedParent,
|
||||
MissingCachedChild,
|
||||
InvalidPrefix,
|
||||
DirtyPrefix,
|
||||
ErasureHealthRequired,
|
||||
MissingCoverage,
|
||||
CachedFailures,
|
||||
UncompactedChild,
|
||||
SelectedForWalk,
|
||||
Eligible,
|
||||
}
|
||||
|
||||
impl ScannerCacheReuseDecision {
|
||||
const ALL: [Self; 16] = [
|
||||
Self::MissingScope,
|
||||
Self::NoResumeSweep,
|
||||
Self::DeepScan,
|
||||
Self::ForwardSweep,
|
||||
Self::ActiveReplication,
|
||||
Self::NotRoot,
|
||||
Self::CompactedParent,
|
||||
Self::MissingCachedChild,
|
||||
Self::InvalidPrefix,
|
||||
Self::DirtyPrefix,
|
||||
Self::ErasureHealthRequired,
|
||||
Self::MissingCoverage,
|
||||
Self::CachedFailures,
|
||||
Self::UncompactedChild,
|
||||
Self::SelectedForWalk,
|
||||
Self::Eligible,
|
||||
];
|
||||
|
||||
fn label(self) -> &'static str {
|
||||
match self {
|
||||
Self::MissingScope => "missing_scope",
|
||||
Self::NoResumeSweep => "no_resume_sweep",
|
||||
Self::DeepScan => "deep_scan",
|
||||
Self::ForwardSweep => "forward_sweep",
|
||||
Self::ActiveReplication => "active_replication",
|
||||
Self::NotRoot => "not_root",
|
||||
Self::CompactedParent => "compacted_parent",
|
||||
Self::MissingCachedChild => "missing_cached_child",
|
||||
Self::InvalidPrefix => "invalid_prefix",
|
||||
Self::DirtyPrefix => "dirty_prefix",
|
||||
Self::ErasureHealthRequired => "erasure_health_required",
|
||||
Self::MissingCoverage => "missing_coverage",
|
||||
Self::CachedFailures => "cached_failures",
|
||||
Self::UncompactedChild => "uncompacted_child",
|
||||
Self::SelectedForWalk => "selected_for_walk",
|
||||
Self::Eligible => "eligible",
|
||||
}
|
||||
}
|
||||
|
||||
fn not_evaluated(self) -> bool {
|
||||
matches!(
|
||||
self,
|
||||
Self::MissingScope | Self::NoResumeSweep | Self::DeepScan | Self::ForwardSweep | Self::ActiveReplication
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
/// Scope admission and end-of-cycle evidence are separate observations. An
|
||||
/// activated completion proof does not mean this cycle used prefix reuse.
|
||||
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
|
||||
pub enum ScannerPrefixReuseStage {
|
||||
ScopeResolution,
|
||||
CycleCompletion,
|
||||
}
|
||||
|
||||
/// The first existing baseline validation check that rejected a snapshot.
|
||||
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
|
||||
pub enum ScannerBaselineRejection {
|
||||
Missing,
|
||||
Decode,
|
||||
Incomplete,
|
||||
Partial,
|
||||
Marks,
|
||||
Epoch,
|
||||
SourceCount,
|
||||
PlanMissing,
|
||||
PlanMismatch,
|
||||
SourceIndex,
|
||||
SourceMembership,
|
||||
SourceDuplicate,
|
||||
SetIncomplete,
|
||||
SetTombstone,
|
||||
SetEpoch,
|
||||
SetCycle,
|
||||
SetPlan,
|
||||
SourceCoverage,
|
||||
AuthoritativeIdentity,
|
||||
NotNewer,
|
||||
}
|
||||
|
||||
impl ScannerBaselineRejection {
|
||||
const ALL: [Self; 20] = [
|
||||
Self::Missing,
|
||||
Self::Decode,
|
||||
Self::Incomplete,
|
||||
Self::Partial,
|
||||
Self::Marks,
|
||||
Self::Epoch,
|
||||
Self::SourceCount,
|
||||
Self::PlanMissing,
|
||||
Self::PlanMismatch,
|
||||
Self::SourceIndex,
|
||||
Self::SourceMembership,
|
||||
Self::SourceDuplicate,
|
||||
Self::SetIncomplete,
|
||||
Self::SetTombstone,
|
||||
Self::SetEpoch,
|
||||
Self::SetCycle,
|
||||
Self::SetPlan,
|
||||
Self::SourceCoverage,
|
||||
Self::AuthoritativeIdentity,
|
||||
Self::NotNewer,
|
||||
];
|
||||
|
||||
fn label(self) -> &'static str {
|
||||
match self {
|
||||
Self::Missing => "missing",
|
||||
Self::Decode => "decode",
|
||||
Self::Incomplete => "incomplete",
|
||||
Self::Partial => "partial",
|
||||
Self::Marks => "marks",
|
||||
Self::Epoch => "epoch",
|
||||
Self::SourceCount => "source_count",
|
||||
Self::PlanMissing => "plan_missing",
|
||||
Self::PlanMismatch => "plan_mismatch",
|
||||
Self::SourceIndex => "source_index",
|
||||
Self::SourceMembership => "source_membership",
|
||||
Self::SourceDuplicate => "source_duplicate",
|
||||
Self::SetIncomplete => "set_incomplete",
|
||||
Self::SetTombstone => "set_tombstone",
|
||||
Self::SetEpoch => "set_epoch",
|
||||
Self::SetCycle => "set_cycle",
|
||||
Self::SetPlan => "set_plan",
|
||||
Self::SourceCoverage => "source_coverage",
|
||||
Self::AuthoritativeIdentity => "authoritative_identity",
|
||||
Self::NotNewer => "not_newer",
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, Default, Serialize, Deserialize)]
|
||||
#[serde(default)]
|
||||
pub struct ScannerMetadataReadSnapshot {
|
||||
pub attempts: u64,
|
||||
pub succeeded: u64,
|
||||
pub not_found: u64,
|
||||
pub errors: u64,
|
||||
pub cancelled: u64,
|
||||
/// Bytes returned by successful storage reads, including corrupt metadata.
|
||||
pub bytes: u64,
|
||||
/// Elapsed read-future time, including failed and cancelled reads.
|
||||
pub elapsed_nanos: u64,
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, Default, Serialize, Deserialize)]
|
||||
#[serde(default)]
|
||||
pub struct ScannerCacheReuseSnapshot {
|
||||
/// Completed subtree reuse decisions; not cache entries or objects skipped.
|
||||
pub count: u64,
|
||||
/// Time spent copying and attaching the reused subtree, excluding listing.
|
||||
pub elapsed_nanos: u64,
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, Default, Serialize, Deserialize)]
|
||||
#[serde(default)]
|
||||
pub struct ScannerBucketReuseSnapshot {
|
||||
/// Cold bucket copies or current roots successfully handed to the collector.
|
||||
pub count: u64,
|
||||
/// Cold copy/attachment time or current-root channel-send time; excludes validation.
|
||||
pub elapsed_nanos: u64,
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, Default, Serialize, Deserialize)]
|
||||
#[serde(default)]
|
||||
pub struct ScannerCacheReuseDecisionSnapshot {
|
||||
/// Policy prerequisites that stopped the gate before cache proof checks.
|
||||
pub not_evaluated: BTreeMap<String, u64>,
|
||||
/// First failed cache/work condition; later conditions are not evaluated.
|
||||
pub blocked: BTreeMap<String, u64>,
|
||||
/// Admitted candidates, which may still need EC classification before copying.
|
||||
pub eligible: u64,
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, Default, Serialize, Deserialize)]
|
||||
#[serde(default)]
|
||||
pub struct ScannerPrefixReuseSnapshot {
|
||||
pub evaluated: u64,
|
||||
pub activated: u64,
|
||||
/// All failed checks are counted; one evaluation can have several blockers.
|
||||
pub blockers: BTreeMap<String, u64>,
|
||||
}
|
||||
|
||||
/// Missing on an older node means unobserved, not zero work or healthy reuse.
|
||||
#[derive(Clone, Debug, Default, Serialize, Deserialize)]
|
||||
#[serde(default)]
|
||||
pub struct ScannerScanObservationSnapshot {
|
||||
pub metadata_reads: ScannerMetadataReadSnapshot,
|
||||
pub cache_reuse: BTreeMap<String, ScannerCacheReuseSnapshot>,
|
||||
pub bucket_reuse: BTreeMap<String, ScannerBucketReuseSnapshot>,
|
||||
pub reuse_decisions: BTreeMap<String, ScannerCacheReuseDecisionSnapshot>,
|
||||
/// Clean root children still walked because EC scans also check health.
|
||||
pub erasure_clean_prefix_walks: u64,
|
||||
pub prefix_scope_resolution: ScannerPrefixReuseSnapshot,
|
||||
pub prefix_cycle_completion: ScannerPrefixReuseSnapshot,
|
||||
pub scope_fallbacks: BTreeMap<String, u64>,
|
||||
/// Rejections only for cycles with a baseline_unavailable scope fallback.
|
||||
pub authoritative_baseline_rejections: BTreeMap<String, u64>,
|
||||
pub observed_baseline_rejections: BTreeMap<String, u64>,
|
||||
}
|
||||
|
||||
#[derive(Default)]
|
||||
pub struct ScannerScanObservation {
|
||||
metadata: [AtomicU64; 7],
|
||||
cache_count: [AtomicU64; 4],
|
||||
cache_nanos: [AtomicU64; 4],
|
||||
bucket_count: [AtomicU64; 2],
|
||||
bucket_nanos: [AtomicU64; 2],
|
||||
reuse_decisions: [[AtomicU64; ScannerCacheReuseDecision::ALL.len()]; 3],
|
||||
erasure_clean_prefix_walks: AtomicU64,
|
||||
prefix_evaluated: [AtomicU64; 2],
|
||||
prefix_activated: [AtomicU64; 2],
|
||||
prefix_blockers: [[AtomicU64; PREFIX_REUSE_BLOCKERS.len()]; 2],
|
||||
scope_fallbacks: [AtomicU64; SCOPE_FALLBACK_REASONS.len()],
|
||||
baseline_rejections: [[AtomicU64; ScannerBaselineRejection::ALL.len()]; 2],
|
||||
}
|
||||
|
||||
fn duration_nanos(duration: Duration) -> u64 {
|
||||
u64::try_from(duration.as_nanos()).unwrap_or(u64::MAX)
|
||||
}
|
||||
|
||||
fn labelled_counts(labels: impl IntoIterator<Item = &'static str>, counters: &[AtomicU64]) -> BTreeMap<String, u64> {
|
||||
labels
|
||||
.into_iter()
|
||||
.zip(counters)
|
||||
.map(|(label, count)| (label.to_owned(), count.load(Ordering::Relaxed)))
|
||||
.collect()
|
||||
}
|
||||
|
||||
impl ScannerScanObservation {
|
||||
pub(crate) fn start_metadata_read(&self) {
|
||||
self.metadata[0].fetch_add(1, Ordering::Relaxed);
|
||||
}
|
||||
|
||||
pub(crate) fn finish_metadata_read(&self, outcome: ScannerMetadataReadOutcome, bytes: u64, elapsed: Duration) {
|
||||
self.metadata[1 + outcome as usize].fetch_add(1, Ordering::Relaxed);
|
||||
if outcome == ScannerMetadataReadOutcome::Success {
|
||||
self.metadata[5].fetch_add(bytes, Ordering::Relaxed);
|
||||
}
|
||||
self.metadata[6].fetch_add(duration_nanos(elapsed), Ordering::Relaxed);
|
||||
}
|
||||
|
||||
pub fn record_cache_reuse(&self, kind: ScannerCacheReuseKind, elapsed: Duration) {
|
||||
self.cache_count[kind as usize].fetch_add(1, Ordering::Relaxed);
|
||||
self.cache_nanos[kind as usize].fetch_add(duration_nanos(elapsed), Ordering::Relaxed);
|
||||
}
|
||||
|
||||
/// Records completed reuse work only; it grants no publication authority.
|
||||
pub fn record_bucket_reuse(&self, kind: ScannerBucketReuseKind, count: u64, elapsed: Duration) {
|
||||
self.bucket_count[kind as usize].fetch_add(count, Ordering::Relaxed);
|
||||
self.bucket_nanos[kind as usize].fetch_add(duration_nanos(elapsed), Ordering::Relaxed);
|
||||
}
|
||||
|
||||
/// Records one first-condition decision without inspecting later prerequisites.
|
||||
pub fn record_cache_reuse_decision(&self, gate: ScannerCacheReuseGate, decision: ScannerCacheReuseDecision) {
|
||||
self.reuse_decisions[gate as usize][decision as usize].fetch_add(1, Ordering::Relaxed);
|
||||
}
|
||||
|
||||
pub fn record_erasure_clean_prefix_walk(&self) {
|
||||
self.erasure_clean_prefix_walks.fetch_add(1, Ordering::Relaxed);
|
||||
}
|
||||
|
||||
pub fn record_prefix_reuse(&self, stage: ScannerPrefixReuseStage, activated: bool, blockers: &[Option<&str>; 8]) {
|
||||
let stage = stage as usize;
|
||||
self.prefix_evaluated[stage].fetch_add(1, Ordering::Relaxed);
|
||||
if activated {
|
||||
self.prefix_activated[stage].fetch_add(1, Ordering::Relaxed);
|
||||
}
|
||||
for blocker in blockers.iter().flatten() {
|
||||
if let Some(index) = PREFIX_REUSE_BLOCKERS.iter().position(|label| label == blocker) {
|
||||
self.prefix_blockers[stage][index].fetch_add(1, Ordering::Relaxed);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
pub(crate) fn record_scope_fallback(&self, reason: &str) {
|
||||
if let Some(index) = SCOPE_FALLBACK_REASONS.iter().position(|label| *label == reason) {
|
||||
self.scope_fallbacks[index].fetch_add(1, Ordering::Relaxed);
|
||||
}
|
||||
}
|
||||
|
||||
pub fn record_baseline_rejection(&self, authoritative: ScannerBaselineRejection, observed: Option<ScannerBaselineRejection>) {
|
||||
self.baseline_rejections[0][authoritative as usize].fetch_add(1, Ordering::Relaxed);
|
||||
if let Some(observed) = observed {
|
||||
self.baseline_rejections[1][observed as usize].fetch_add(1, Ordering::Relaxed);
|
||||
}
|
||||
}
|
||||
|
||||
pub fn snapshot(&self) -> ScannerScanObservationSnapshot {
|
||||
let prefix = |stage: ScannerPrefixReuseStage| ScannerPrefixReuseSnapshot {
|
||||
evaluated: self.prefix_evaluated[stage as usize].load(Ordering::Relaxed),
|
||||
activated: self.prefix_activated[stage as usize].load(Ordering::Relaxed),
|
||||
blockers: labelled_counts(PREFIX_REUSE_BLOCKERS, &self.prefix_blockers[stage as usize]),
|
||||
};
|
||||
ScannerScanObservationSnapshot {
|
||||
metadata_reads: ScannerMetadataReadSnapshot {
|
||||
attempts: self.metadata[0].load(Ordering::Relaxed),
|
||||
succeeded: self.metadata[1].load(Ordering::Relaxed),
|
||||
not_found: self.metadata[2].load(Ordering::Relaxed),
|
||||
errors: self.metadata[3].load(Ordering::Relaxed),
|
||||
cancelled: self.metadata[4].load(Ordering::Relaxed),
|
||||
bytes: self.metadata[5].load(Ordering::Relaxed),
|
||||
elapsed_nanos: self.metadata[6].load(Ordering::Relaxed),
|
||||
},
|
||||
cache_reuse: ScannerCacheReuseKind::ALL
|
||||
.into_iter()
|
||||
.map(|kind| {
|
||||
(
|
||||
kind.label().to_owned(),
|
||||
ScannerCacheReuseSnapshot {
|
||||
count: self.cache_count[kind as usize].load(Ordering::Relaxed),
|
||||
elapsed_nanos: self.cache_nanos[kind as usize].load(Ordering::Relaxed),
|
||||
},
|
||||
)
|
||||
})
|
||||
.collect(),
|
||||
bucket_reuse: ScannerBucketReuseKind::ALL
|
||||
.into_iter()
|
||||
.map(|kind| {
|
||||
(
|
||||
kind.label().to_owned(),
|
||||
ScannerBucketReuseSnapshot {
|
||||
count: self.bucket_count[kind as usize].load(Ordering::Relaxed),
|
||||
elapsed_nanos: self.bucket_nanos[kind as usize].load(Ordering::Relaxed),
|
||||
},
|
||||
)
|
||||
})
|
||||
.collect(),
|
||||
reuse_decisions: ScannerCacheReuseGate::ALL
|
||||
.into_iter()
|
||||
.map(|gate| {
|
||||
let counts = &self.reuse_decisions[gate as usize];
|
||||
let decisions = |not_evaluated| {
|
||||
ScannerCacheReuseDecision::ALL
|
||||
.into_iter()
|
||||
.filter(|decision| {
|
||||
*decision != ScannerCacheReuseDecision::Eligible && decision.not_evaluated() == not_evaluated
|
||||
})
|
||||
.map(|decision| (decision.label().to_owned(), counts[decision as usize].load(Ordering::Relaxed)))
|
||||
.collect()
|
||||
};
|
||||
(
|
||||
gate.label().to_owned(),
|
||||
ScannerCacheReuseDecisionSnapshot {
|
||||
not_evaluated: decisions(true),
|
||||
blocked: decisions(false),
|
||||
eligible: counts[ScannerCacheReuseDecision::Eligible as usize].load(Ordering::Relaxed),
|
||||
},
|
||||
)
|
||||
})
|
||||
.collect(),
|
||||
erasure_clean_prefix_walks: self.erasure_clean_prefix_walks.load(Ordering::Relaxed),
|
||||
prefix_scope_resolution: prefix(ScannerPrefixReuseStage::ScopeResolution),
|
||||
prefix_cycle_completion: prefix(ScannerPrefixReuseStage::CycleCompletion),
|
||||
scope_fallbacks: labelled_counts(SCOPE_FALLBACK_REASONS, &self.scope_fallbacks),
|
||||
authoritative_baseline_rejections: labelled_counts(
|
||||
ScannerBaselineRejection::ALL.map(ScannerBaselineRejection::label),
|
||||
&self.baseline_rejections[0],
|
||||
),
|
||||
observed_baseline_rejections: labelled_counts(
|
||||
ScannerBaselineRejection::ALL.map(ScannerBaselineRejection::label),
|
||||
&self.baseline_rejections[1],
|
||||
),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Dropping an unfinished read records cancellation. The guard observes the
|
||||
/// read future only; it cannot cancel it or change its storage result.
|
||||
#[must_use]
|
||||
pub struct ScannerMetadataReadGuard<'a> {
|
||||
pub(crate) metrics: &'a Metrics,
|
||||
pub(crate) started: Instant,
|
||||
pub(crate) outcome: ScannerMetadataReadOutcome,
|
||||
pub(crate) bytes: u64,
|
||||
}
|
||||
|
||||
impl ScannerMetadataReadGuard<'_> {
|
||||
pub fn finish(mut self, outcome: ScannerMetadataReadOutcome, bytes: u64) {
|
||||
self.outcome = outcome;
|
||||
self.bytes = bytes;
|
||||
}
|
||||
}
|
||||
|
||||
impl Drop for ScannerMetadataReadGuard<'_> {
|
||||
fn drop(&mut self) {
|
||||
self.metrics
|
||||
.finish_scanner_metadata_read(self.outcome, self.bytes, self.started.elapsed());
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use std::sync::Arc;
|
||||
|
||||
#[tokio::test]
|
||||
async fn metadata_read_reports_storage_results_and_existing_read_metric() {
|
||||
let metrics = Metrics::new();
|
||||
metrics
|
||||
.start_scanner_metadata_read()
|
||||
.finish(ScannerMetadataReadOutcome::Success, 127);
|
||||
metrics
|
||||
.start_scanner_metadata_read()
|
||||
.finish(ScannerMetadataReadOutcome::Success, 0);
|
||||
metrics
|
||||
.start_scanner_metadata_read()
|
||||
.finish(ScannerMetadataReadOutcome::NotFound, 0);
|
||||
metrics
|
||||
.start_scanner_metadata_read()
|
||||
.finish(ScannerMetadataReadOutcome::Error, 0);
|
||||
let report = metrics.report().await;
|
||||
let reads = &report.scan_observation.metadata_reads;
|
||||
assert_eq!(
|
||||
(reads.attempts, reads.succeeded, reads.not_found, reads.errors, reads.cancelled),
|
||||
(4, 2, 1, 1, 0)
|
||||
);
|
||||
assert_eq!(reads.bytes, 127);
|
||||
assert_eq!(report.life_time_ops["read_metadata"], 4);
|
||||
assert_eq!(report.last_minute.actions["read_metadata"].count, 4);
|
||||
assert_eq!(report.last_minute.actions["read_metadata"].bytes, 127);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn metadata_read_cancellation_balances_an_actually_polled_future_once() {
|
||||
let metrics = Arc::new(Metrics::new());
|
||||
let worker_metrics = Arc::clone(&metrics);
|
||||
let (started, admitted) = tokio::sync::oneshot::channel();
|
||||
let worker = tokio::spawn(async move {
|
||||
let read = worker_metrics.start_scanner_metadata_read();
|
||||
started.send(()).expect("observe started read");
|
||||
std::future::pending::<()>().await;
|
||||
read.finish(ScannerMetadataReadOutcome::Success, 99);
|
||||
});
|
||||
tokio::time::timeout(Duration::from_secs(5), admitted)
|
||||
.await
|
||||
.expect("read starts within deadline")
|
||||
.expect("read started");
|
||||
let in_flight = metrics.scan_observation().snapshot().metadata_reads;
|
||||
assert_eq!((in_flight.attempts, in_flight.cancelled, in_flight.succeeded), (1, 0, 0));
|
||||
worker.abort();
|
||||
assert!(worker.await.expect_err("read task was aborted").is_cancelled());
|
||||
let report = metrics.report().await;
|
||||
let reads = &report.scan_observation.metadata_reads;
|
||||
assert_eq!((reads.attempts, reads.cancelled, reads.succeeded, reads.bytes), (1, 1, 0, 0));
|
||||
assert_eq!(report.life_time_ops["read_metadata"], 1);
|
||||
assert_eq!(report.last_minute.actions["read_metadata"].count, 1);
|
||||
assert_eq!(
|
||||
Duration::from_nanos(reads.elapsed_nanos).as_secs(),
|
||||
report.last_minute.actions["read_metadata"].acc_time
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn observation_report_keeps_stages_reasons_and_copy_costs_separate() {
|
||||
let metrics = Metrics::new();
|
||||
let observation = metrics.scan_observation();
|
||||
observation.record_cache_reuse(ScannerCacheReuseKind::CleanPrefix, Duration::from_nanos(321));
|
||||
observation.record_bucket_reuse(ScannerBucketReuseKind::ScopedColdBucket, 2, Duration::from_nanos(654));
|
||||
observation.record_bucket_reuse(ScannerBucketReuseKind::CurrentRoot, 1, Duration::from_nanos(987));
|
||||
observation.record_cache_reuse_decision(ScannerCacheReuseGate::CleanPrefix, ScannerCacheReuseDecision::MissingScope);
|
||||
observation
|
||||
.record_cache_reuse_decision(ScannerCacheReuseGate::CleanPrefix, ScannerCacheReuseDecision::ErasureHealthRequired);
|
||||
observation.record_cache_reuse_decision(ScannerCacheReuseGate::CoveredNamespace, ScannerCacheReuseDecision::Eligible);
|
||||
observation.record_erasure_clean_prefix_walk();
|
||||
observation.record_prefix_reuse(
|
||||
ScannerPrefixReuseStage::ScopeResolution,
|
||||
false,
|
||||
&[
|
||||
Some("missing_producer_identity"),
|
||||
Some("missing_durable_journal_replay"),
|
||||
None,
|
||||
None,
|
||||
None,
|
||||
None,
|
||||
None,
|
||||
None,
|
||||
],
|
||||
);
|
||||
observation.record_prefix_reuse(ScannerPrefixReuseStage::CycleCompletion, true, &[None; 8]);
|
||||
metrics.record_scanner_scope_fallback("baseline_unavailable");
|
||||
observation.record_baseline_rejection(ScannerBaselineRejection::Epoch, Some(ScannerBaselineRejection::Missing));
|
||||
let report = metrics.report().await;
|
||||
let snapshot = &report.scan_observation;
|
||||
assert_eq!(snapshot.cache_reuse["clean_prefix"].count, 1);
|
||||
assert_eq!(snapshot.cache_reuse["clean_prefix"].elapsed_nanos, 321);
|
||||
assert_eq!(snapshot.cache_reuse["sampled_namespace"].count, 0);
|
||||
assert_eq!(snapshot.bucket_reuse["scoped_cold_bucket"].count, 2);
|
||||
assert_eq!(snapshot.bucket_reuse["scoped_cold_bucket"].elapsed_nanos, 654);
|
||||
assert_eq!(snapshot.bucket_reuse["current_root"].count, 1);
|
||||
assert_eq!(snapshot.bucket_reuse["current_root"].elapsed_nanos, 987);
|
||||
assert_eq!(snapshot.reuse_decisions["clean_prefix"].not_evaluated["missing_scope"], 1);
|
||||
assert_eq!(snapshot.reuse_decisions["clean_prefix"].blocked["erasure_health_required"], 1);
|
||||
assert_eq!(snapshot.reuse_decisions["covered_namespace"].eligible, 1);
|
||||
|
||||
let mut prior_observations = serde_json::to_value(snapshot).expect("encode observations");
|
||||
let fields = prior_observations.as_object_mut().expect("observation object");
|
||||
fields.remove("bucket_reuse");
|
||||
fields.remove("reuse_decisions");
|
||||
let prior: ScannerScanObservationSnapshot =
|
||||
serde_json::from_value(prior_observations).expect("read earlier observations");
|
||||
assert!(prior.bucket_reuse.is_empty());
|
||||
assert!(prior.reuse_decisions.is_empty());
|
||||
assert_eq!(prior.cache_reuse["clean_prefix"].count, 1);
|
||||
|
||||
assert_eq!(snapshot.erasure_clean_prefix_walks, 1);
|
||||
assert_eq!(
|
||||
(snapshot.prefix_scope_resolution.evaluated, snapshot.prefix_scope_resolution.activated),
|
||||
(1, 0)
|
||||
);
|
||||
assert_eq!(snapshot.prefix_scope_resolution.blockers["missing_producer_identity"], 1);
|
||||
assert_eq!(snapshot.prefix_scope_resolution.blockers["missing_durable_journal_replay"], 1);
|
||||
assert_eq!(
|
||||
(snapshot.prefix_cycle_completion.evaluated, snapshot.prefix_cycle_completion.activated),
|
||||
(1, 1)
|
||||
);
|
||||
assert_eq!(
|
||||
snapshot.scope_fallbacks["baseline_unavailable"],
|
||||
report.usage_freshness.scope_fallback_total
|
||||
);
|
||||
assert_eq!(snapshot.authoritative_baseline_rejections["epoch"], 1);
|
||||
assert_eq!(snapshot.observed_baseline_rejections["missing"], 1);
|
||||
|
||||
let mut legacy = serde_json::to_value(&report).expect("encode report");
|
||||
legacy
|
||||
.as_object_mut()
|
||||
.expect("report is an object")
|
||||
.remove("scan_observation");
|
||||
let decoded: crate::metrics::ScannerMetricsReport = serde_json::from_value(legacy).expect("read legacy report");
|
||||
assert!(decoded.scan_observation.cache_reuse.is_empty(), "legacy counters are unobserved");
|
||||
assert_eq!(decoded.scan_observation.metadata_reads.attempts, 0);
|
||||
}
|
||||
}
|
||||
@@ -206,6 +206,32 @@ pub(crate) struct DataUsageCacheRevisions {
|
||||
backup: Option<DataUsageCacheRevision>,
|
||||
}
|
||||
|
||||
/// In-memory provenance of the decoded cache and the revisions observed by its loader.
|
||||
#[derive(Clone, Debug)]
|
||||
pub(crate) struct DataUsageCacheBaselineRevisions {
|
||||
revisions: DataUsageCacheRevisions,
|
||||
loaded_from_main: bool,
|
||||
}
|
||||
|
||||
impl DataUsageCacheBaselineRevisions {
|
||||
pub(crate) fn revisions(&self) -> &DataUsageCacheRevisions {
|
||||
&self.revisions
|
||||
}
|
||||
|
||||
/// A backup may catch up to an unchanged primary without changing the decoded baseline.
|
||||
/// Both reads must have decoded that primary; an ETag on a corrupt primary is insufficient.
|
||||
pub(crate) fn permits_unchanged_baseline_since(&self, initial: &Self) -> bool {
|
||||
if self.revisions == initial.revisions {
|
||||
return true;
|
||||
}
|
||||
self.loaded_from_main
|
||||
&& initial.loaded_from_main
|
||||
&& matches!(&initial.revisions.main, DataUsageCacheRevision::Etag(etag) if !etag.is_empty())
|
||||
&& self.revisions.main == initial.revisions.main
|
||||
&& self.revisions.backup.as_ref() == Some(&self.revisions.main)
|
||||
}
|
||||
}
|
||||
|
||||
pub static DATA_USAGE_BUCKET: LazyLock<String> =
|
||||
LazyLock::new(|| format!("{RUSTFS_META_BUCKET}{SLASH_SEPARATOR}{BUCKET_META_PREFIX}"));
|
||||
|
||||
|
||||
@@ -70,6 +70,7 @@ struct DataUsageCacheLoadResult {
|
||||
cache: DataUsageCache,
|
||||
main_revision: DataUsageCacheRevision,
|
||||
backup_revision: Option<DataUsageCacheRevision>,
|
||||
loaded_from_main: bool,
|
||||
}
|
||||
|
||||
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
|
||||
@@ -106,6 +107,18 @@ impl DataUsageCache {
|
||||
store: Arc<S>,
|
||||
name: &str,
|
||||
) -> StorageResult<DataUsageCacheRevisions> {
|
||||
self.load_with_baseline_revisions(store, name)
|
||||
.await
|
||||
.map(|baseline| baseline.revisions)
|
||||
}
|
||||
|
||||
/// Loads cache revisions with provenance for unchanged-primary backup reconciliation.
|
||||
/// Missing or rebuilt caches and a decoded backup do not establish a decoded primary.
|
||||
pub(crate) async fn load_with_baseline_revisions<S: ScannerObjectIO>(
|
||||
&mut self,
|
||||
store: Arc<S>,
|
||||
name: &str,
|
||||
) -> StorageResult<DataUsageCacheBaselineRevisions> {
|
||||
let backup_name = format!("{name}.bkp");
|
||||
let backup_path = path_join_buf(&[BUCKET_META_PREFIX, &backup_name]);
|
||||
let loaded = Self::load_cache(store.clone(), name).await?;
|
||||
@@ -133,7 +146,10 @@ impl DataUsageCache {
|
||||
let main = loaded.main_revision;
|
||||
*self = loaded.cache;
|
||||
|
||||
Ok(DataUsageCacheRevisions { main, backup })
|
||||
Ok(DataUsageCacheBaselineRevisions {
|
||||
revisions: DataUsageCacheRevisions { main, backup },
|
||||
loaded_from_main: loaded.loaded_from_main,
|
||||
})
|
||||
}
|
||||
|
||||
pub(crate) async fn read_revisions<S: ScannerObjectIO>(store: Arc<S>, name: &str) -> StorageResult<DataUsageCacheRevisions> {
|
||||
@@ -156,10 +172,13 @@ impl DataUsageCache {
|
||||
cache,
|
||||
revision: Some(main_revision),
|
||||
} => {
|
||||
// Legacy-location fallback carries Missing for the primary path.
|
||||
let loaded_from_main = matches!(&main_revision, DataUsageCacheRevision::Etag(_));
|
||||
return Ok(DataUsageCacheLoadResult {
|
||||
cache: *cache,
|
||||
main_revision,
|
||||
backup_revision: None,
|
||||
loaded_from_main,
|
||||
});
|
||||
}
|
||||
DataUsageCacheLoadAttempt::Loaded { revision: None, .. } => {
|
||||
@@ -197,6 +216,7 @@ impl DataUsageCache {
|
||||
cache: *cache,
|
||||
main_revision,
|
||||
backup_revision: Some(backup_revision),
|
||||
loaded_from_main: false,
|
||||
});
|
||||
}
|
||||
DataUsageCacheLoadState::Loaded
|
||||
@@ -220,6 +240,7 @@ impl DataUsageCache {
|
||||
main_revision: main_revision
|
||||
.ok_or_else(|| Error::other(format!("scanner cache missing state has no revision: {name}")))?,
|
||||
backup_revision,
|
||||
loaded_from_main: false,
|
||||
});
|
||||
}
|
||||
(DataUsageCacheLoadState::Corrupt, DataUsageCacheLoadState::Missing)
|
||||
@@ -239,6 +260,7 @@ impl DataUsageCache {
|
||||
main_revision: main_revision
|
||||
.ok_or_else(|| Error::other(format!("scanner cache corrupt state has no revision: {name}")))?,
|
||||
backup_revision,
|
||||
loaded_from_main: false,
|
||||
});
|
||||
}
|
||||
_ => {}
|
||||
|
||||
@@ -772,6 +772,102 @@ async fn corrupt_primary_cache_loads_valid_backup() {
|
||||
assert_eq!(store.puts.load(Ordering::SeqCst), 0);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn baseline_revisions_allow_only_a_decoded_unchanged_primary_backup_catchup() {
|
||||
let etag = |value: &str| DataUsageCacheRevision::Etag(value.to_owned());
|
||||
let initial = DataUsageCacheBaselineRevisions {
|
||||
revisions: DataUsageCacheRevisions {
|
||||
main: etag("primary"),
|
||||
backup: Some(etag("old-backup")),
|
||||
},
|
||||
loaded_from_main: true,
|
||||
};
|
||||
assert!(initial.permits_unchanged_baseline_since(&initial));
|
||||
let mut caught_up = initial.clone();
|
||||
caught_up.revisions.backup = Some(etag("primary"));
|
||||
assert!(caught_up.permits_unchanged_baseline_since(&initial));
|
||||
assert!(
|
||||
!initial.permits_unchanged_baseline_since(&caught_up),
|
||||
"backup rollback changes the baseline"
|
||||
);
|
||||
for backup in [None, Some(DataUsageCacheRevision::Missing)] {
|
||||
let mut missing_backup = initial.clone();
|
||||
missing_backup.revisions.backup = backup;
|
||||
assert!(caught_up.permits_unchanged_baseline_since(&missing_backup));
|
||||
assert!(!missing_backup.permits_unchanged_baseline_since(&initial));
|
||||
}
|
||||
for (initial_from_main, current_from_main) in [(false, true), (true, false), (false, false)] {
|
||||
let mut baseline = initial.clone();
|
||||
baseline.loaded_from_main = initial_from_main;
|
||||
let mut current = caught_up.clone();
|
||||
current.loaded_from_main = current_from_main;
|
||||
assert!(
|
||||
!current.permits_unchanged_baseline_since(&baseline),
|
||||
"backup fallback grants no primary proof"
|
||||
);
|
||||
current.revisions = baseline.revisions.clone();
|
||||
assert!(
|
||||
current.permits_unchanged_baseline_since(&baseline),
|
||||
"exact revisions retain existing admission"
|
||||
);
|
||||
}
|
||||
let mut changed_main = caught_up.clone();
|
||||
changed_main.revisions.main = etag("new-primary");
|
||||
changed_main.revisions.backup = Some(etag("new-primary"));
|
||||
assert!(!changed_main.permits_unchanged_baseline_since(&initial));
|
||||
let mut changed_backup = caught_up;
|
||||
changed_backup.revisions.backup = Some(etag("unrelated-backup"));
|
||||
assert!(!changed_backup.permits_unchanged_baseline_since(&initial));
|
||||
for main in [DataUsageCacheRevision::Missing, etag("")] {
|
||||
let mut baseline = initial.clone();
|
||||
baseline.revisions.main = main.clone();
|
||||
let mut current = baseline.clone();
|
||||
current.revisions.backup = Some(main);
|
||||
assert!(
|
||||
!current.permits_unchanged_baseline_since(&baseline),
|
||||
"missing or empty primary ETag is not proof"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn baseline_revisions_record_the_actual_decoded_cache_source() {
|
||||
let mut expected = DataUsageCache::default();
|
||||
expected.info.name = "decoded".to_owned();
|
||||
let valid = expected.marshal_msg().expect("encode valid cache");
|
||||
for (main, backup, from_main, name) in [
|
||||
(valid.clone(), Some(valid.clone()), true, "decoded"),
|
||||
(b"corrupt".to_vec(), Some(valid.clone()), false, "decoded"),
|
||||
(b"corrupt".to_vec(), None, false, ""),
|
||||
(Vec::new(), None, false, ""),
|
||||
] {
|
||||
let store = Arc::new(CacheReadStore::new(CacheReadBody::Bytes(main), backup));
|
||||
let mut loaded = DataUsageCache::default();
|
||||
let baseline = loaded
|
||||
.load_with_baseline_revisions(store.clone(), DATA_USAGE_CACHE_NAME)
|
||||
.await
|
||||
.expect("load actual cache source");
|
||||
assert_eq!(baseline.loaded_from_main, from_main);
|
||||
assert_eq!(loaded.info.name, name);
|
||||
let mut raw_loaded = DataUsageCache::default();
|
||||
let raw = raw_loaded
|
||||
.load_with_revisions(store, DATA_USAGE_CACHE_NAME)
|
||||
.await
|
||||
.expect("load compatible raw revisions");
|
||||
assert_eq!(baseline.revisions(), &raw, "raw loader retains its revision contract");
|
||||
}
|
||||
let mut legacy_store = CacheReadStore::new(CacheReadBody::Bytes(Vec::new()), None);
|
||||
legacy_store.legacy = Some(valid);
|
||||
let mut legacy_cache = DataUsageCache::default();
|
||||
let legacy = legacy_cache
|
||||
.load_with_baseline_revisions(Arc::new(legacy_store), "legacy-cache")
|
||||
.await
|
||||
.expect("load cache from the legacy location");
|
||||
assert_eq!(legacy_cache.info.name, "decoded");
|
||||
assert!(!legacy.loaded_from_main);
|
||||
assert_eq!(legacy.revisions.main, DataUsageCacheRevision::Missing);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn corrupt_cache_without_valid_backup_rebuilds_with_cas_revisions() {
|
||||
for backup in [None, Some(b"also-not-msgpack".to_vec())] {
|
||||
|
||||
@@ -248,13 +248,13 @@ const CLEAN_IDLE_BACKOFF_FACTOR: u32 = 2;
|
||||
/// otherwise idle instance whose clean-idle backoff had doubled a 60 s
|
||||
/// interval), which defeats the fast path it is meant to protect.
|
||||
///
|
||||
/// The exponential growth in [`ScannerRetryBackoff::retry_interval`] is
|
||||
/// what protects against a persistently hot bucket driving an unbroken
|
||||
/// full-scan loop, so it can start small: 5 s, 10 s, 20 s … capped by
|
||||
/// [`SCANNER_RETRY_MAX_INTERVAL`]. A one-off race recovers in seconds; a
|
||||
/// genuinely hot bucket still reaches minute-scale backoff within a handful of
|
||||
/// cycles. Preflight deferrals use the same bounded schedule so a temporarily
|
||||
/// unavailable peer cannot drive a tight retry loop.
|
||||
/// The exponential growth in [`ScannerRetryBackoff::retry_interval`] protects
|
||||
/// against a persistently hot bucket driving an unbroken full-scan loop, so
|
||||
/// retries can start small: 5 s, 10 s, 20 s … Superseded retries cap this
|
||||
/// schedule at the ordinary cycle interval and preserve an operator-selected
|
||||
/// nominal cadence. Preflight deferrals retain the generic
|
||||
/// [`SCANNER_RETRY_MAX_INTERVAL`] cap so a temporarily unavailable peer cannot
|
||||
/// drive a tight retry loop.
|
||||
const SCANNER_RETRY_BASE_INTERVAL: Duration = Duration::from_secs(5);
|
||||
const SCANNER_RETRY_MAX_INTERVAL: Duration = Duration::from_secs(30 * 60);
|
||||
/// Probe cadence while data movement pauses the scanner. Recovery retries use
|
||||
@@ -3536,8 +3536,7 @@ where
|
||||
));
|
||||
let mut wait_plan =
|
||||
scanner_cycle_wait_plan(&runtime_config, clean_idle_backoff, backoff_enabled, randomized_cycle_delay_for);
|
||||
let superseded_retry_interval =
|
||||
scanner_superseded_retry_interval(superseded_backoff, &runtime_config, maintenance_features);
|
||||
let superseded_retry_interval = scanner_superseded_retry_interval(superseded_backoff, &runtime_config);
|
||||
let deferred_retry_interval = deferred_backoff.retry_interval(runtime_config.cycle_interval);
|
||||
let scope_fallback_retry_interval = scanner_scope_fallback_retry_interval(scope_fallback_backoff, &runtime_config);
|
||||
let mut convergence_retry_interval = superseded_retry_interval
|
||||
|
||||
@@ -192,19 +192,16 @@ impl ScannerRetryBackoff {
|
||||
pub(super) fn scanner_superseded_retry_interval(
|
||||
backoff: ScannerRetryBackoff,
|
||||
runtime_config: &ScannerRuntimeConfig,
|
||||
maintenance_features: ScannerMaintenanceFeatures,
|
||||
) -> Option<Duration> {
|
||||
let mut retry_interval = backoff.retry_interval(runtime_config.cycle_interval)?;
|
||||
// Preserve an operator-selected duty-cycle floor during convergence.
|
||||
let cycle_interval = runtime_config.cycle_interval.max(Duration::from_secs(1));
|
||||
let retry_interval = backoff.retry_interval(cycle_interval)?;
|
||||
// Preserve an operator-selected nominal cadence during convergence.
|
||||
if runtime_config.cycle_interval_source != ScannerRuntimeConfigSource::Default {
|
||||
retry_interval = retry_interval.max(runtime_config.cycle_interval);
|
||||
return Some(cycle_interval);
|
||||
}
|
||||
// A complete but non-authoritative walk must not postpone lifecycle or
|
||||
// replication evaluation beyond its ordinary maintenance cadence.
|
||||
if maintenance_features.needs_regular_cycle() {
|
||||
retry_interval = retry_interval.min(runtime_config.cycle_interval.max(Duration::from_secs(1)));
|
||||
}
|
||||
Some(retry_interval)
|
||||
// A complete but non-authoritative walk must not postpone convergence
|
||||
// beyond the ordinary cadence, even without lifecycle or replication.
|
||||
Some(retry_interval.min(cycle_interval))
|
||||
}
|
||||
|
||||
/// Pacing for the cycle after a full-namespace scope fallback while dirty
|
||||
|
||||
@@ -8949,7 +8949,7 @@ fn clean_idle_backoff_grows_to_cap() {
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn superseded_retry_backoff_grows_caps_and_resets_after_convergence() {
|
||||
fn retry_backoff_grows_caps_and_resets_after_convergence() {
|
||||
let mut backoff = ScannerRetryBackoff::default();
|
||||
assert_eq!(backoff.retry_interval(Duration::from_secs(24 * 60 * 60)), None);
|
||||
|
||||
@@ -8973,7 +8973,7 @@ fn superseded_retry_backoff_grows_caps_and_resets_after_convergence() {
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn superseded_retry_backoff_respects_a_faster_configured_cycle() {
|
||||
fn retry_backoff_respects_a_faster_configured_cycle() {
|
||||
let mut backoff = ScannerRetryBackoff::default();
|
||||
backoff.record_retryable_cycle(true);
|
||||
|
||||
@@ -8985,7 +8985,7 @@ fn superseded_retry_backoff_respects_a_faster_configured_cycle() {
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn superseded_retry_backoff_grows_from_the_default_cycle() {
|
||||
fn retry_backoff_grows_from_the_default_cycle() {
|
||||
let mut backoff = ScannerRetryBackoff::default();
|
||||
// The first race after a write burst retries in seconds, not a whole
|
||||
// cycle, while repeated supersedes still climb toward the cap.
|
||||
@@ -9005,7 +9005,7 @@ fn superseded_retry_preserves_explicit_cycle_cadence() {
|
||||
..Default::default()
|
||||
};
|
||||
assert_eq!(
|
||||
scanner_superseded_retry_interval(backoff, &default_config, ScannerMaintenanceFeatures::default()),
|
||||
scanner_superseded_retry_interval(backoff, &default_config),
|
||||
Some(SCANNER_RETRY_BASE_INTERVAL),
|
||||
"the default adaptive cadence must retain fast convergence"
|
||||
);
|
||||
@@ -9015,24 +9015,24 @@ fn superseded_retry_preserves_explicit_cycle_cadence() {
|
||||
ScannerRuntimeConfigSource::Config,
|
||||
ScannerRuntimeConfigSource::ScannerCompatConfig,
|
||||
] {
|
||||
let runtime_config = ScannerRuntimeConfig {
|
||||
cycle_interval: Duration::from_secs(6 * 60 * 60),
|
||||
cycle_interval_source: source,
|
||||
..Default::default()
|
||||
};
|
||||
|
||||
for features in [
|
||||
ScannerMaintenanceFeatures::default(),
|
||||
ScannerMaintenanceFeatures {
|
||||
lifecycle: true,
|
||||
for cycle_secs in [1, 3, 600, 6 * 60 * 60] {
|
||||
let runtime_config = ScannerRuntimeConfig {
|
||||
cycle_interval: Duration::from_secs(cycle_secs),
|
||||
cycle_interval_source: source,
|
||||
..Default::default()
|
||||
},
|
||||
] {
|
||||
assert_eq!(
|
||||
scanner_superseded_retry_interval(backoff, &runtime_config, features),
|
||||
Some(runtime_config.cycle_interval),
|
||||
"{source:?} cycle cadence must not be shortened by convergence retries"
|
||||
);
|
||||
};
|
||||
let mut backoff = ScannerRetryBackoff::default();
|
||||
assert_eq!(scanner_superseded_retry_interval(backoff, &runtime_config), None);
|
||||
for _ in 0..40 {
|
||||
backoff.record_retryable_cycle(true);
|
||||
assert_eq!(
|
||||
scanner_superseded_retry_interval(backoff, &runtime_config),
|
||||
Some(runtime_config.cycle_interval),
|
||||
"{source:?} cycle cadence must hold throughout convergence retries"
|
||||
);
|
||||
}
|
||||
backoff.record_retryable_cycle(false);
|
||||
assert_eq!(scanner_superseded_retry_interval(backoff, &runtime_config), None);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -9053,7 +9053,7 @@ fn superseded_retry_does_not_delay_required_maintenance() {
|
||||
for consecutive_cycles in 1..=12 {
|
||||
backoff.record_retryable_cycle(true);
|
||||
assert_eq!(
|
||||
scanner_superseded_retry_interval(backoff, &config, features),
|
||||
scanner_superseded_retry_interval(backoff, &config),
|
||||
Some(config.cycle_interval),
|
||||
"superseded cycle {consecutive_cycles} must preserve regular maintenance"
|
||||
);
|
||||
@@ -9139,19 +9139,50 @@ fn scope_fallback_retry_preserves_explicit_and_fast_cycle_cadence() {
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn superseded_retry_only_caps_regular_maintenance() {
|
||||
fn superseded_retry_caps_at_the_default_cycle_and_resets_after_convergence() {
|
||||
let config = ScannerRuntimeConfig {
|
||||
cycle_interval: Duration::from_secs(60),
|
||||
..Default::default()
|
||||
};
|
||||
let mut backoff = ScannerRetryBackoff::default();
|
||||
for _ in 0..6 {
|
||||
assert_eq!(scanner_superseded_retry_interval(backoff, &config), None);
|
||||
for expected in [5, 10, 20, 40, 60, 60, 60, 60] {
|
||||
backoff.record_retryable_cycle(true);
|
||||
assert_eq!(scanner_superseded_retry_interval(backoff, &config), Some(Duration::from_secs(expected)));
|
||||
}
|
||||
backoff.consecutive_cycles = u32::MAX;
|
||||
assert_eq!(scanner_superseded_retry_interval(backoff, &config), Some(config.cycle_interval));
|
||||
backoff.record_retryable_cycle(true);
|
||||
assert_eq!(scanner_superseded_retry_interval(backoff, &config), Some(config.cycle_interval));
|
||||
backoff.record_retryable_cycle(false);
|
||||
assert_eq!(scanner_superseded_retry_interval(backoff, &config), None);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn superseded_retry_caps_fast_cycles_without_changing_the_generic_retry_cap() {
|
||||
let mut backoff = ScannerRetryBackoff::default();
|
||||
for _ in 0..40 {
|
||||
backoff.record_retryable_cycle(true);
|
||||
}
|
||||
assert_eq!(
|
||||
scanner_superseded_retry_interval(backoff, &config, ScannerMaintenanceFeatures::default()),
|
||||
Some(Duration::from_secs(160)),
|
||||
backoff.retry_interval(Duration::from_secs(60)),
|
||||
Some(Duration::from_secs(30 * 60)),
|
||||
"deferred and transient retries must retain the generic cap"
|
||||
);
|
||||
for cycle_secs in [0, 1, 3, 60, 6 * 60 * 60] {
|
||||
let config = ScannerRuntimeConfig {
|
||||
cycle_interval: Duration::from_secs(cycle_secs),
|
||||
..Default::default()
|
||||
};
|
||||
assert_eq!(
|
||||
scanner_superseded_retry_interval(backoff, &config),
|
||||
Some(Duration::from_secs(cycle_secs.max(1))),
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn superseded_retry_timer_keeps_required_maintenance_full_scans() {
|
||||
for features in [
|
||||
ScannerMaintenanceFeatures {
|
||||
lifecycle: true,
|
||||
@@ -9166,28 +9197,23 @@ fn superseded_retry_only_caps_regular_maintenance() {
|
||||
..Default::default()
|
||||
},
|
||||
] {
|
||||
assert_eq!(scanner_superseded_retry_interval(backoff, &config, features), Some(config.cycle_interval));
|
||||
assert_eq!(scanner_superseded_retry_interval(ScannerRetryBackoff::default(), &config, features), None);
|
||||
assert!(features.requires_full_scan(Some(7), 7, ScannerCycleWakeReason::Timer, true));
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test(start_paused = true)]
|
||||
#[serial]
|
||||
async fn superseded_maintenance_wait_keeps_convergence_fencing() {
|
||||
async fn superseded_retry_wait_keeps_convergence_fencing_at_the_default_cycle() {
|
||||
crate::scanner_io::clear_dirty_usage_buckets_for_tests();
|
||||
let config = ScannerRuntimeConfig {
|
||||
cycle_interval: Duration::from_secs(60),
|
||||
..Default::default()
|
||||
};
|
||||
let features = ScannerMaintenanceFeatures {
|
||||
lifecycle: true,
|
||||
..Default::default()
|
||||
};
|
||||
let mut backoff = ScannerRetryBackoff::default();
|
||||
for _ in 0..12 {
|
||||
backoff.record_retryable_cycle(true);
|
||||
}
|
||||
let interval = scanner_superseded_retry_interval(backoff, &config, features);
|
||||
let interval = scanner_superseded_retry_interval(backoff, &config);
|
||||
assert_eq!(interval, Some(config.cycle_interval));
|
||||
let generations = ScannerCycleObservedGenerations::for_wait(
|
||||
&config,
|
||||
@@ -9202,19 +9228,28 @@ async fn superseded_maintenance_wait_keeps_convergence_fencing() {
|
||||
let mut seen = Some(BTreeMap::from([("node-2".to_string(), scanner_node_activity("epoch-a", 7, 3))]));
|
||||
let changed = BTreeMap::from([("node-2".to_string(), scanner_node_activity("epoch-a", 8, 3))]);
|
||||
let started = tokio::time::Instant::now();
|
||||
let reason = wait_for_next_scanner_cycle_with_activity(
|
||||
let mut wait = Box::pin(wait_for_next_scanner_cycle_with_activity(
|
||||
&ctx,
|
||||
interval.expect("maintenance must retain its convergence interval"),
|
||||
interval.expect("superseded retry must retain its convergence interval"),
|
||||
Some(Duration::from_secs(1)),
|
||||
&mut seen,
|
||||
generations,
|
||||
|| false,
|
||||
|| std::future::ready(Ok(changed.clone())),
|
||||
)
|
||||
.await;
|
||||
));
|
||||
assert!(matches!(futures::poll!(&mut wait), Poll::Pending));
|
||||
crate::scanner_io::record_dirty_usage_bucket("photos");
|
||||
assert!(matches!(futures::poll!(&mut wait), Poll::Pending));
|
||||
for _ in 0..3 {
|
||||
tokio::time::advance(Duration::from_secs(15)).await;
|
||||
crate::scanner_io::record_dirty_usage_bucket("photos");
|
||||
assert!(matches!(futures::poll!(&mut wait), Poll::Pending));
|
||||
}
|
||||
let reason = wait.await;
|
||||
assert_eq!(reason, ScannerCycleWakeReason::Timer);
|
||||
assert_eq!(started.elapsed(), config.cycle_interval);
|
||||
assert_eq!(seen, Some(changed));
|
||||
crate::scanner_io::clear_dirty_usage_buckets_for_tests();
|
||||
}
|
||||
|
||||
#[test]
|
||||
|
||||
@@ -59,6 +59,7 @@ use rustfs_scanner_metrics::metrics::{
|
||||
CloseDiskGuard, IlmAction, Metric, Metrics, ScannerReplicationRepairKind, ScannerSourceWorkUpdate, ScannerWorkSource,
|
||||
UpdateCurrentPathFn, current_path_updater, global_metrics,
|
||||
};
|
||||
use rustfs_scanner_metrics::scan_observation::{ScannerCacheReuseDecision, ScannerCacheReuseGate, ScannerCacheReuseKind};
|
||||
use rustfs_utils::path::{SLASH_SEPARATOR, path_join_buf};
|
||||
use sha2::{Digest, Sha256};
|
||||
use time::OffsetDateTime;
|
||||
@@ -1060,38 +1061,51 @@ impl FolderScanner {
|
||||
child_hash: &DataUsageHash,
|
||||
abandoned_children: &DataUsageHashMap,
|
||||
) -> bool {
|
||||
let Some(prefix_scan_scope) = &self.prefix_scan_scope else {
|
||||
return false;
|
||||
let decision = if let Some(prefix_scan_scope) = &self.prefix_scan_scope {
|
||||
if folder.parent.is_some() || folder.name != self.old_cache.info.name {
|
||||
ScannerCacheReuseDecision::NotRoot
|
||||
} else if into.compacted {
|
||||
ScannerCacheReuseDecision::CompactedParent
|
||||
} else if !abandoned_children.contains(&child_hash.0) {
|
||||
ScannerCacheReuseDecision::MissingCachedChild
|
||||
} else if let Some(entry) = child
|
||||
.name
|
||||
.strip_prefix(folder.name.as_str())
|
||||
.and_then(|entry| entry.strip_prefix('/'))
|
||||
{
|
||||
if entry.is_empty() || entry.contains('/') {
|
||||
ScannerCacheReuseDecision::InvalidPrefix
|
||||
} else if prefix_scan_scope.contains(entry) {
|
||||
ScannerCacheReuseDecision::DirtyPrefix
|
||||
} else if self.is_erasure_mode {
|
||||
// EC scans must still perform independent object-health work.
|
||||
global_metrics().scan_observation().record_erasure_clean_prefix_walk();
|
||||
ScannerCacheReuseDecision::ErasureHealthRequired
|
||||
} else {
|
||||
ScannerCacheReuseDecision::Eligible
|
||||
}
|
||||
} else {
|
||||
ScannerCacheReuseDecision::InvalidPrefix
|
||||
}
|
||||
} else {
|
||||
ScannerCacheReuseDecision::MissingScope
|
||||
};
|
||||
// Erasure-mode usage scans also perform probabilistic object-health
|
||||
// work. Reusing a clean subtree here would silently suppress that
|
||||
// independent maintenance path, so prefix reuse is limited to the
|
||||
// non-erasure data-usage scanner.
|
||||
if self.is_erasure_mode
|
||||
|| folder.parent.is_some()
|
||||
|| folder.name != self.old_cache.info.name
|
||||
|| into.compacted
|
||||
|| !abandoned_children.contains(&clone_cache_value(&child_hash.0))
|
||||
{
|
||||
return false;
|
||||
}
|
||||
|
||||
let Some(entry) = child
|
||||
.name
|
||||
.strip_prefix(folder.name.as_str())
|
||||
.and_then(|entry| entry.strip_prefix('/'))
|
||||
else {
|
||||
return false;
|
||||
};
|
||||
!entry.is_empty() && !entry.contains('/') && !prefix_scan_scope.contains(entry)
|
||||
global_metrics()
|
||||
.scan_observation()
|
||||
.record_cache_reuse_decision(ScannerCacheReuseGate::CleanPrefix, decision);
|
||||
decision == ScannerCacheReuseDecision::Eligible
|
||||
}
|
||||
|
||||
fn reuse_clean_root_child(&mut self, child: &CachedFolder, child_hash: &DataUsageHash, into: &mut DataUsageEntry) {
|
||||
let started = Instant::now();
|
||||
self.new_cache.copy_with_children(&self.old_cache, child_hash, &child.parent);
|
||||
if let Some(updates) = &mut self.usage_updates {
|
||||
updates.cache.copy_with_children(&self.old_cache, child_hash, &child.parent);
|
||||
}
|
||||
into.add_child(child_hash);
|
||||
global_metrics()
|
||||
.scan_observation()
|
||||
.record_cache_reuse(ScannerCacheReuseKind::CleanPrefix, started.elapsed());
|
||||
}
|
||||
|
||||
fn failed_retry_suppressed(&self, path: &str) -> bool {
|
||||
@@ -1278,14 +1292,60 @@ impl FolderScanner {
|
||||
}
|
||||
|
||||
fn can_reuse_covered_namespace(&self, name: &str, compacted: bool) -> bool {
|
||||
self.old_cache.info.scan_progress.is_some()
|
||||
&& !compacted
|
||||
&& self
|
||||
.resume_frontier
|
||||
.as_deref()
|
||||
.is_some_and(|frontier| scan_frontier_covers_path(name, frontier))
|
||||
&& self.old_cache.find(name).is_some()
|
||||
&& !self.resume_subtree_has_failures(name)
|
||||
let decision = if self.old_cache.info.scan_progress.is_none() {
|
||||
ScannerCacheReuseDecision::NoResumeSweep
|
||||
} else if compacted {
|
||||
ScannerCacheReuseDecision::CompactedParent
|
||||
} else if !self
|
||||
.resume_frontier
|
||||
.as_deref()
|
||||
.is_some_and(|frontier| scan_frontier_covers_path(name, frontier))
|
||||
{
|
||||
ScannerCacheReuseDecision::MissingCoverage
|
||||
} else if self.old_cache.find(name).is_none() {
|
||||
ScannerCacheReuseDecision::MissingCachedChild
|
||||
} else if self.resume_subtree_has_failures(name) {
|
||||
ScannerCacheReuseDecision::CachedFailures
|
||||
} else {
|
||||
ScannerCacheReuseDecision::Eligible
|
||||
};
|
||||
global_metrics()
|
||||
.scan_observation()
|
||||
.record_cache_reuse_decision(ScannerCacheReuseGate::CoveredNamespace, decision);
|
||||
decision == ScannerCacheReuseDecision::Eligible
|
||||
}
|
||||
|
||||
// None preserves work that cannot use sampling; false requests this cycle's rescan.
|
||||
fn sampled_namespace_reuse(&self, hash: &DataUsageHash, compacted: bool, forward_sweep: bool) -> Option<bool> {
|
||||
let decision = if self.scan_mode == HealScanMode::Deep {
|
||||
ScannerCacheReuseDecision::DeepScan
|
||||
} else if forward_sweep {
|
||||
ScannerCacheReuseDecision::ForwardSweep
|
||||
} else if self
|
||||
.old_cache
|
||||
.info
|
||||
.replication
|
||||
.as_ref()
|
||||
.is_some_and(|config| config.has_active_rules("", true))
|
||||
{
|
||||
ScannerCacheReuseDecision::ActiveReplication
|
||||
} else if compacted {
|
||||
ScannerCacheReuseDecision::CompactedParent
|
||||
} else if !self.old_cache.is_compacted(hash) {
|
||||
ScannerCacheReuseDecision::UncompactedChild
|
||||
} else if hash.mod_(self.old_cache.info.next_cycle as u32, data_usage_update_dir_cycles()) {
|
||||
ScannerCacheReuseDecision::SelectedForWalk
|
||||
} else {
|
||||
ScannerCacheReuseDecision::Eligible
|
||||
};
|
||||
global_metrics()
|
||||
.scan_observation()
|
||||
.record_cache_reuse_decision(ScannerCacheReuseGate::NamespaceSampling, decision);
|
||||
match decision {
|
||||
ScannerCacheReuseDecision::Eligible => Some(true),
|
||||
ScannerCacheReuseDecision::SelectedForWalk => Some(false),
|
||||
_ => None,
|
||||
}
|
||||
}
|
||||
|
||||
fn resume_subtree_has_failures(&self, folder: &str) -> bool {
|
||||
@@ -2077,7 +2137,11 @@ impl FolderScanner {
|
||||
// The caller stitches this cached child into its
|
||||
// parent; do not clone a growing child set into the
|
||||
// unused provisional accumulator as well.
|
||||
let started = Instant::now();
|
||||
self.new_cache.copy_with_children(&self.old_cache, &this_hash, &folder.parent);
|
||||
global_metrics()
|
||||
.scan_observation()
|
||||
.record_cache_reuse(ScannerCacheReuseKind::ClassifiedNamespace, started.elapsed());
|
||||
self.budget.record_entries_visited(pending_entry_progress);
|
||||
self.finish_raw_enumeration_parent(&folder.name);
|
||||
done_folder();
|
||||
@@ -2664,8 +2728,12 @@ impl FolderScanner {
|
||||
FolderScanMode::Namespace
|
||||
};
|
||||
if child_mode == FolderScanMode::Namespace && reuse_covered {
|
||||
let started = Instant::now();
|
||||
self.new_cache.copy_with_children(&self.old_cache, &h, &folder_item.parent);
|
||||
into.add_child(&h);
|
||||
global_metrics()
|
||||
.scan_observation()
|
||||
.record_cache_reuse(ScannerCacheReuseKind::CoveredNamespace, started.elapsed());
|
||||
continue;
|
||||
}
|
||||
|
||||
@@ -2705,22 +2773,15 @@ impl FolderScanner {
|
||||
}
|
||||
FolderScanSource::Existing => {
|
||||
// Usage sampling is not proof that a Deep check ran.
|
||||
if self.scan_mode != HealScanMode::Deep
|
||||
&& !forward_sweep
|
||||
&& !self
|
||||
.old_cache
|
||||
.info
|
||||
.replication
|
||||
.as_ref()
|
||||
.is_some_and(|config| config.has_active_rules("", true))
|
||||
&& !into.compacted
|
||||
&& self.old_cache.is_compacted(&h)
|
||||
{
|
||||
let next_cycle = self.old_cache.info.next_cycle as u32;
|
||||
if !h.mod_(next_cycle, data_usage_update_dir_cycles()) {
|
||||
if let Some(reuse) = self.sampled_namespace_reuse(&h, into.compacted, forward_sweep) {
|
||||
if reuse {
|
||||
if child_mode == FolderScanMode::Namespace {
|
||||
let started = Instant::now();
|
||||
self.new_cache.copy_with_children(&self.old_cache, &h, &folder_item.parent);
|
||||
into.add_child(&h);
|
||||
global_metrics()
|
||||
.scan_observation()
|
||||
.record_cache_reuse(ScannerCacheReuseKind::SampledNamespace, started.elapsed());
|
||||
self.record_scan_resume_hint(&folder_item.name);
|
||||
continue;
|
||||
}
|
||||
|
||||
@@ -3103,6 +3103,112 @@ async fn test_scan_folder_non_erasure_metadata_keeps_namespace_descent() {
|
||||
assert_eq!(budget.reason(), Some(crate::scanner_budget::ScannerCycleBudgetReason::Directories));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
#[serial]
|
||||
async fn cache_reuse_gates_report_only_the_first_production_condition() {
|
||||
let (mut scanner, temp_dir) = build_test_scanner().await;
|
||||
let _guard = TestGuard::new(60, 100, &mut scanner, temp_dir);
|
||||
scanner.old_cache.info.name = "bucket".to_string();
|
||||
scanner.old_cache.replace("bucket", "", DataUsageEntry::default());
|
||||
scanner.old_cache.replace(
|
||||
"bucket/child",
|
||||
"bucket",
|
||||
DataUsageEntry {
|
||||
compacted: true,
|
||||
objects: 2,
|
||||
..Default::default()
|
||||
},
|
||||
);
|
||||
let hash = hash_path("bucket/child");
|
||||
let folder = CachedFolder {
|
||||
name: "bucket".to_string(),
|
||||
parent: None,
|
||||
object_heal_prob_div: 1,
|
||||
};
|
||||
let child = CachedFolder {
|
||||
name: "bucket/child".to_string(),
|
||||
parent: Some(hash_path("bucket")),
|
||||
object_heal_prob_div: 1,
|
||||
};
|
||||
let abandoned = HashSet::from([hash.0.clone()]);
|
||||
let before = global_metrics().scan_observation().snapshot();
|
||||
|
||||
assert!(!scanner.should_reuse_clean_root_child(&folder, &DataUsageEntry::default(), &child, &hash, &abandoned));
|
||||
let missing_scope = global_metrics().scan_observation().snapshot();
|
||||
assert_eq!(
|
||||
missing_scope.reuse_decisions["clean_prefix"].not_evaluated["missing_scope"],
|
||||
before.reuse_decisions["clean_prefix"].not_evaluated["missing_scope"] + 1
|
||||
);
|
||||
scanner.prefix_scan_scope = ScannerBucketPrefixScanScope::from_dirty_top_level_entries(HashSet::from(["dirty".to_string()]));
|
||||
scanner.is_erasure_mode = true;
|
||||
assert!(!scanner.should_reuse_clean_root_child(&folder, &DataUsageEntry::default(), &child, &hash, &abandoned));
|
||||
let erasure = global_metrics().scan_observation().snapshot();
|
||||
assert_eq!(
|
||||
erasure.reuse_decisions["clean_prefix"].blocked["erasure_health_required"],
|
||||
before.reuse_decisions["clean_prefix"].blocked["erasure_health_required"] + 1
|
||||
);
|
||||
assert_eq!(erasure.erasure_clean_prefix_walks, before.erasure_clean_prefix_walks + 1);
|
||||
|
||||
assert!(!scanner.can_reuse_covered_namespace("bucket/child", true));
|
||||
scanner.old_cache.info.scan_progress = Some(crate::DataUsageScanProgress {
|
||||
started_plan: crate::DataUsageScanPlanDigest([1; 32]),
|
||||
requested_plan: crate::DataUsageScanPlanDigest([1; 32]),
|
||||
});
|
||||
assert!(!scanner.can_reuse_covered_namespace("bucket/child", true));
|
||||
assert!(!scanner.can_reuse_covered_namespace("bucket/child", false));
|
||||
scanner.resume_frontier = Some("bucket/child".to_string());
|
||||
scanner.resume_failed_paths = vec!["bucket/child/failed".to_string()];
|
||||
assert!(!scanner.can_reuse_covered_namespace("bucket/child", false));
|
||||
scanner.resume_failed_paths.clear();
|
||||
assert!(scanner.can_reuse_covered_namespace("bucket/child", false));
|
||||
let covered = global_metrics().scan_observation().snapshot();
|
||||
let reasons = &covered.reuse_decisions["covered_namespace"];
|
||||
let prior = &before.reuse_decisions["covered_namespace"];
|
||||
assert_eq!(reasons.not_evaluated["no_resume_sweep"], prior.not_evaluated["no_resume_sweep"] + 1);
|
||||
assert_eq!(reasons.blocked["compacted_parent"], prior.blocked["compacted_parent"] + 1);
|
||||
assert_eq!(reasons.blocked["missing_coverage"], prior.blocked["missing_coverage"] + 1);
|
||||
assert_eq!(reasons.blocked["cached_failures"], prior.blocked["cached_failures"] + 1);
|
||||
assert_eq!(reasons.eligible, prior.eligible + 1);
|
||||
|
||||
with_var(ENV_DATA_USAGE_UPDATE_DIR_CYCLES, Some("16"), || {
|
||||
scanner.scan_mode = HealScanMode::Deep;
|
||||
assert_eq!(scanner.sampled_namespace_reuse(&hash, true, true), None);
|
||||
scanner.scan_mode = HealScanMode::Normal;
|
||||
assert_eq!(scanner.sampled_namespace_reuse(&hash, true, true), None);
|
||||
let replication: ReplicationConfig = serde_json::from_value(serde_json::json!({
|
||||
"config": { "role": "", "rules": [{ "destination": { "bucket": "arn:test" }, "id": "status", "prefix": "", "priority": 1, "status": "Enabled" }] },
|
||||
"remotes": crate::storage_api::EcstoreBucketTargets::default(),
|
||||
})).expect("active replication fixture");
|
||||
scanner.old_cache.info.replication = Some(Arc::new(replication));
|
||||
assert_eq!(scanner.sampled_namespace_reuse(&hash, true, false), None);
|
||||
scanner.old_cache.info.replication = None;
|
||||
assert_eq!(scanner.sampled_namespace_reuse(&hash, true, false), None);
|
||||
let rescan_cycle = (0..16)
|
||||
.find(|cycle| hash.mod_(*cycle, 16))
|
||||
.expect("hash is visited in one sampling cycle");
|
||||
scanner.old_cache.info.next_cycle = u64::from(rescan_cycle);
|
||||
assert_eq!(scanner.sampled_namespace_reuse(&hash, false, false), Some(false));
|
||||
scanner.old_cache.info.next_cycle = u64::from((rescan_cycle + 1) % 16);
|
||||
assert_eq!(scanner.sampled_namespace_reuse(&hash, false, false), Some(true));
|
||||
});
|
||||
let after = global_metrics().scan_observation().snapshot();
|
||||
let reasons = &after.reuse_decisions["namespace_sampling"];
|
||||
let prior = &before.reuse_decisions["namespace_sampling"];
|
||||
for label in ["deep_scan", "forward_sweep", "active_replication"] {
|
||||
assert_eq!(
|
||||
reasons.not_evaluated[label],
|
||||
prior.not_evaluated[label] + 1,
|
||||
"first sampling condition {label}"
|
||||
);
|
||||
}
|
||||
assert_eq!(reasons.blocked["compacted_parent"], prior.blocked["compacted_parent"] + 1);
|
||||
assert_eq!(reasons.blocked["selected_for_walk"], prior.blocked["selected_for_walk"] + 1);
|
||||
assert_eq!(reasons.eligible, prior.eligible + 1);
|
||||
for (kind, reuse) in &after.cache_reuse {
|
||||
assert_eq!(reuse.count, before.cache_reuse[kind].count, "gate admission alone does not copy {kind}");
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
#[serial]
|
||||
async fn scoped_root_scan_reuses_clean_top_level_entries_and_rescans_dirty_entries() {
|
||||
@@ -3144,12 +3250,21 @@ async fn scoped_root_scan_reuses_clean_top_level_entries_and_rescans_dirty_entri
|
||||
parent: None,
|
||||
object_heal_prob_div: 1,
|
||||
};
|
||||
let before = global_metrics().scan_observation().snapshot();
|
||||
let mut root = DataUsageEntry::default();
|
||||
scanner
|
||||
.scan_folder(CancellationToken::new(), folder, &mut root)
|
||||
.await
|
||||
.expect("scoped root scan should finish successfully");
|
||||
|
||||
let observed = global_metrics().scan_observation().snapshot();
|
||||
assert_eq!(observed.cache_reuse["clean_prefix"].count, before.cache_reuse["clean_prefix"].count + 1);
|
||||
assert_eq!(observed.erasure_clean_prefix_walks, before.erasure_clean_prefix_walks);
|
||||
assert_eq!(
|
||||
observed.reuse_decisions["clean_prefix"].eligible,
|
||||
before.reuse_decisions["clean_prefix"].eligible + 1
|
||||
);
|
||||
|
||||
let clean = scanner
|
||||
.new_cache
|
||||
.size_recursive("bucket/clean")
|
||||
@@ -3167,6 +3282,78 @@ async fn scoped_root_scan_reuses_clean_top_level_entries_and_rescans_dirty_entri
|
||||
assert_eq!((bucket.size, bucket.objects), (17, 3));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
#[serial]
|
||||
async fn scoped_cache_observations_distinguish_checkpoint_and_periodic_sampling() {
|
||||
for covered in [true, false] {
|
||||
let (mut scanner, temp_dir) = build_test_scanner().await;
|
||||
let _guard = TestGuard::new(60, 100, &mut scanner, temp_dir.clone());
|
||||
tokio::fs::create_dir_all(temp_dir.join("bucket/child"))
|
||||
.await
|
||||
.expect("create cached namespace");
|
||||
scanner.old_cache.info.name = "bucket".to_string();
|
||||
scanner.new_cache.info.name = "bucket".to_string();
|
||||
scanner.old_cache.replace("bucket", "", DataUsageEntry::default());
|
||||
scanner.old_cache.replace(
|
||||
"bucket/child",
|
||||
"bucket",
|
||||
DataUsageEntry {
|
||||
size: 17,
|
||||
objects: 5,
|
||||
compacted: true,
|
||||
..Default::default()
|
||||
},
|
||||
);
|
||||
let kind = if covered {
|
||||
scanner.old_cache.info.scan_progress = Some(crate::DataUsageScanProgress {
|
||||
started_plan: crate::DataUsageScanPlanDigest([1; 32]),
|
||||
requested_plan: crate::DataUsageScanPlanDigest([1; 32]),
|
||||
});
|
||||
scanner.resume_frontier = Some("bucket/child".to_string());
|
||||
"covered_namespace"
|
||||
} else {
|
||||
let hash = hash_path("bucket/child");
|
||||
let cycle = (0..data_usage_update_dir_cycles())
|
||||
.find(|cycle| !hash.mod_(*cycle, data_usage_update_dir_cycles()))
|
||||
.expect("a cycle samples other work");
|
||||
scanner.old_cache.info.next_cycle = u64::from(cycle);
|
||||
"sampled_namespace"
|
||||
};
|
||||
let before = global_metrics().scan_observation().snapshot();
|
||||
let mut root = DataUsageEntry::default();
|
||||
scanner
|
||||
.scan_folder(
|
||||
CancellationToken::new(),
|
||||
CachedFolder {
|
||||
name: "bucket".to_string(),
|
||||
parent: None,
|
||||
object_heal_prob_div: 1,
|
||||
},
|
||||
&mut root,
|
||||
)
|
||||
.await
|
||||
.expect("cached namespace reuses accounting");
|
||||
let summary = scanner.new_cache.size_recursive("bucket").expect("complete bucket totals");
|
||||
assert_eq!((summary.size, summary.objects), (17, 5));
|
||||
let after = global_metrics().scan_observation().snapshot();
|
||||
for label in [
|
||||
"clean_prefix",
|
||||
"covered_namespace",
|
||||
"sampled_namespace",
|
||||
"classified_namespace",
|
||||
] {
|
||||
assert_eq!(
|
||||
after.cache_reuse[label].count,
|
||||
before.cache_reuse[label].count + u64::from(label == kind),
|
||||
"reuse reason {label}"
|
||||
);
|
||||
}
|
||||
assert_eq!(after.metadata_reads.attempts, before.metadata_reads.attempts);
|
||||
let gate = if covered { "covered_namespace" } else { "namespace_sampling" };
|
||||
assert_eq!(after.reuse_decisions[gate].eligible, before.reuse_decisions[gate].eligible + 1);
|
||||
}
|
||||
}
|
||||
|
||||
async fn scan_hot_cold_segment_fixture(scoped: bool) -> (DataUsageEntry, Vec<String>) {
|
||||
let (mut scanner, temp_dir) = build_test_scanner().await;
|
||||
let visits = metadata_visits::observe(&mut scanner);
|
||||
@@ -3278,6 +3465,7 @@ async fn scoped_root_scan_preserves_erasure_health_walks() {
|
||||
parent: None,
|
||||
object_heal_prob_div: 1,
|
||||
};
|
||||
let before = global_metrics().scan_observation().snapshot();
|
||||
let mut root = DataUsageEntry::default();
|
||||
scanner
|
||||
.scan_folder(CancellationToken::new(), folder, &mut root)
|
||||
@@ -3289,6 +3477,9 @@ async fn scoped_root_scan_preserves_erasure_health_walks() {
|
||||
.size_recursive("bucket/clean")
|
||||
.expect("erasure scan should visit the clean entry");
|
||||
assert_eq!((clean.size, clean.objects), (0, 0));
|
||||
let observed = global_metrics().scan_observation().snapshot();
|
||||
assert_eq!(observed.erasure_clean_prefix_walks, before.erasure_clean_prefix_walks + 1);
|
||||
assert_eq!(observed.cache_reuse["clean_prefix"].count, before.cache_reuse["clean_prefix"].count);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
|
||||
@@ -774,6 +774,7 @@ async fn erasure_candidate_covered_namespace_reuses_totals_after_classification(
|
||||
},
|
||||
);
|
||||
scanner.budget = budget.clone();
|
||||
let before = global_metrics().scan_observation().snapshot();
|
||||
scanner
|
||||
.scan_folder(
|
||||
budget.token(),
|
||||
@@ -790,6 +791,15 @@ async fn erasure_candidate_covered_namespace_reuses_totals_after_classification(
|
||||
assert_eq!(budget.progress(), (0, 1), "only the parent consumes namespace quota");
|
||||
assert_eq!(charged_ops(&root), 4, "list both directories once without re-reading covered metadata");
|
||||
assert_eq!(scanner.new_cache.find(&folder.name).expect("reused candidate").objects, 1);
|
||||
let observed = global_metrics().scan_observation().snapshot();
|
||||
assert_eq!(
|
||||
observed.cache_reuse["classified_namespace"].count,
|
||||
before.cache_reuse["classified_namespace"].count + 1
|
||||
);
|
||||
assert_eq!(
|
||||
observed.metadata_reads.attempts, before.metadata_reads.attempts,
|
||||
"classification retains the existing zero metadata read path"
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
@@ -869,6 +879,7 @@ async fn erasure_candidate_preserves_compacted_namespace_sampling_after_classifi
|
||||
},
|
||||
);
|
||||
scanner.budget = budget.clone();
|
||||
let before = global_metrics().scan_observation().snapshot();
|
||||
scanner
|
||||
.scan_folder(
|
||||
budget.token(),
|
||||
@@ -889,6 +900,15 @@ async fn erasure_candidate_preserves_compacted_namespace_sampling_after_classifi
|
||||
"sampling still requires full current classification, without metadata re-read"
|
||||
);
|
||||
assert_eq!(scanner.new_cache.find(&folder.name).expect("sampled candidate").objects, 1);
|
||||
let observed = global_metrics().scan_observation().snapshot();
|
||||
assert_eq!(
|
||||
observed.cache_reuse["classified_namespace"].count,
|
||||
before.cache_reuse["classified_namespace"].count + 1
|
||||
);
|
||||
assert_eq!(
|
||||
observed.metadata_reads.attempts, before.metadata_reads.attempts,
|
||||
"classification retains the existing zero metadata read path"
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
|
||||
@@ -40,6 +40,7 @@ use rustfs_lock::{LockError, NamespaceLockGuard};
|
||||
use rustfs_scanner_metrics::metrics::{
|
||||
Metric, Metrics, emit_scan_bucket_drive_complete, emit_scan_bucket_drive_partial, global_metrics,
|
||||
};
|
||||
use rustfs_scanner_metrics::scan_observation::{PREFIX_REUSE_BLOCKERS, ScannerBaselineRejection, ScannerPrefixReuseStage};
|
||||
use rustfs_utils::path::path_join_buf;
|
||||
use sha2::{Digest as _, Sha256};
|
||||
use std::collections::{BTreeMap, BTreeSet, HashMap, HashSet};
|
||||
@@ -112,16 +113,7 @@ pub(crate) const SCANNER_SEGMENT_ACTIVATION_PROOF_INPUTS: [&str; 9] = [
|
||||
"durable_dirty_producer_journal",
|
||||
"ack_generation_guard",
|
||||
];
|
||||
pub(crate) const SCANNER_SEGMENT_ACTIVATION_FAIL_CLOSED_CHECKS: [&str; 8] = [
|
||||
"missing_producer_identity",
|
||||
"missing_durable_journal_replay",
|
||||
"restart_gap",
|
||||
"generation_gap",
|
||||
"overflow",
|
||||
"stale_ack_generation",
|
||||
"missing_cold_zero_walk_oracle",
|
||||
"distributed_without_peer_invalidation",
|
||||
];
|
||||
pub(crate) const SCANNER_SEGMENT_ACTIVATION_FAIL_CLOSED_CHECKS: [&str; 8] = PREFIX_REUSE_BLOCKERS;
|
||||
|
||||
pub type DirtyUsageBuckets = HashMap<String, u64>;
|
||||
|
||||
@@ -197,6 +189,7 @@ struct VerifiedRemoteDirtyUsage {
|
||||
}
|
||||
|
||||
struct ScannerBucketScopeResolutionResult {
|
||||
segment_reuse_activation_preflight: Option<ScannerSegmentReuseActivationPreflight>,
|
||||
scope: ScannerBucketScanScope,
|
||||
remote_dirty_usage_acknowledgements: Vec<crate::scanner::ScannerDirtyUsageAcknowledgement>,
|
||||
distributed_segment_invalidation_evidence: Option<DistributedSegmentInvalidationEvidence>,
|
||||
@@ -236,6 +229,13 @@ pub(crate) struct ScannerSegmentReuseActivationPreflight {
|
||||
}
|
||||
|
||||
impl ScannerSegmentReuseActivationPreflight {
|
||||
fn record_observation(&self, stage: ScannerPrefixReuseStage) {
|
||||
global_metrics().scan_observation().record_prefix_reuse(
|
||||
stage,
|
||||
self.scanner_segment_reuse_activated,
|
||||
&self.fail_closed_blockers,
|
||||
);
|
||||
}
|
||||
#[cfg(test)]
|
||||
pub(crate) fn fail_closed_blockers(&self) -> impl Iterator<Item = &'static str> + '_ {
|
||||
self.fail_closed_blockers.iter().filter_map(|blocker| *blocker)
|
||||
@@ -319,6 +319,7 @@ fn resolve_remote_dirty_usage_scope(
|
||||
baseline_proof: ScannerCacheBaselineProof<'_>,
|
||||
) -> ScannerBucketScopeResolutionResult {
|
||||
let default_result = |scope: ScannerBucketScanScope| ScannerBucketScopeResolutionResult {
|
||||
segment_reuse_activation_preflight: None,
|
||||
scope,
|
||||
remote_dirty_usage_acknowledgements: Vec::new(),
|
||||
distributed_segment_invalidation_evidence: None,
|
||||
@@ -410,6 +411,7 @@ fn resolve_remote_dirty_usage_scope(
|
||||
};
|
||||
|
||||
ScannerBucketScopeResolutionResult {
|
||||
segment_reuse_activation_preflight: Some(segment_reuse_activation_preflight),
|
||||
scope,
|
||||
remote_dirty_usage_acknowledgements: scoped_acknowledgements,
|
||||
distributed_segment_invalidation_evidence,
|
||||
@@ -427,43 +429,81 @@ fn complete_scanner_cache_snapshot_plan_digest(
|
||||
proof: ScannerCacheBaselineProof<'_>,
|
||||
expected_converged: Option<bool>,
|
||||
) -> Option<DataUsageScanPlanDigest> {
|
||||
validate_scanner_cache_snapshot_plan_digest(snapshot, proof, expected_converged).ok()
|
||||
}
|
||||
|
||||
fn validate_scanner_cache_snapshot_plan_digest(
|
||||
snapshot: &DataUsageInfo,
|
||||
proof: ScannerCacheBaselineProof<'_>,
|
||||
expected_converged: Option<bool>,
|
||||
) -> std::result::Result<DataUsageScanPlanDigest, ScannerBaselineRejection> {
|
||||
// The observational mark disqualifies only the authoritative slot: an
|
||||
// observed candidate carries it by construction.
|
||||
let marks_accepted = match expected_converged {
|
||||
Some(expected) => snapshot.usage_snapshot_converged == Some(expected),
|
||||
None => snapshot.usage_snapshot_converged.is_some() && !snapshot.usage_snapshot_observational,
|
||||
};
|
||||
if !snapshot.is_complete_bucket_usage_snapshot()
|
||||
|| snapshot.usage_snapshot_partial
|
||||
|| !marks_accepted
|
||||
|| snapshot.scanner_epoch != Some(proof.leader_epoch)
|
||||
|| snapshot.usage_snapshot_set_states.len() != proof.expected_sources.len()
|
||||
{
|
||||
return None;
|
||||
if !snapshot.is_complete_bucket_usage_snapshot() {
|
||||
return Err(ScannerBaselineRejection::Incomplete);
|
||||
}
|
||||
if snapshot.usage_snapshot_partial {
|
||||
return Err(ScannerBaselineRejection::Partial);
|
||||
}
|
||||
if !marks_accepted {
|
||||
return Err(ScannerBaselineRejection::Marks);
|
||||
}
|
||||
if snapshot.scanner_epoch != Some(proof.leader_epoch) {
|
||||
return Err(ScannerBaselineRejection::Epoch);
|
||||
}
|
||||
if snapshot.usage_snapshot_set_states.len() != proof.expected_sources.len() {
|
||||
return Err(ScannerBaselineRejection::SourceCount);
|
||||
}
|
||||
|
||||
// Completed maintenance also covers ordinary usage. Keep its exact stored
|
||||
// proof for cache reuse, and reject mixtures of different set work proofs.
|
||||
let baseline_plan_digest = DataUsageScanPlanDigest(snapshot.usage_snapshot_set_states.first()?.scan_plan_digest?);
|
||||
let baseline_plan_digest = DataUsageScanPlanDigest(
|
||||
snapshot
|
||||
.usage_snapshot_set_states
|
||||
.first()
|
||||
.and_then(|state| state.scan_plan_digest)
|
||||
.ok_or(ScannerBaselineRejection::PlanMissing)?,
|
||||
);
|
||||
if !scanner_plan_digest_is_work_variant(proof.scan_plan_digest, baseline_plan_digest) {
|
||||
return None;
|
||||
return Err(ScannerBaselineRejection::PlanMismatch);
|
||||
}
|
||||
let mut states = HashSet::with_capacity(snapshot.usage_snapshot_set_states.len());
|
||||
for state in &snapshot.usage_snapshot_set_states {
|
||||
let source = DataUsageCacheSource::new(usize::try_from(state.pool_index).ok()?, usize::try_from(state.set_index).ok()?);
|
||||
if !proof.expected_sources.contains(&source)
|
||||
|| !states.insert(source)
|
||||
|| !state.complete
|
||||
|| state.tombstone
|
||||
|| state.scanner_epoch != Some(proof.leader_epoch)
|
||||
|| state.scanner_cycle.is_none_or(|cycle| cycle > proof.want_cycle)
|
||||
|| state.scan_plan_digest != Some(baseline_plan_digest.0)
|
||||
{
|
||||
return None;
|
||||
let source = DataUsageCacheSource::new(
|
||||
usize::try_from(state.pool_index).map_err(|_| ScannerBaselineRejection::SourceIndex)?,
|
||||
usize::try_from(state.set_index).map_err(|_| ScannerBaselineRejection::SourceIndex)?,
|
||||
);
|
||||
if !proof.expected_sources.contains(&source) {
|
||||
return Err(ScannerBaselineRejection::SourceMembership);
|
||||
}
|
||||
if !states.insert(source) {
|
||||
return Err(ScannerBaselineRejection::SourceDuplicate);
|
||||
}
|
||||
if !state.complete {
|
||||
return Err(ScannerBaselineRejection::SetIncomplete);
|
||||
}
|
||||
if state.tombstone {
|
||||
return Err(ScannerBaselineRejection::SetTombstone);
|
||||
}
|
||||
if state.scanner_epoch != Some(proof.leader_epoch) {
|
||||
return Err(ScannerBaselineRejection::SetEpoch);
|
||||
}
|
||||
if state.scanner_cycle.is_none_or(|cycle| cycle > proof.want_cycle) {
|
||||
return Err(ScannerBaselineRejection::SetCycle);
|
||||
}
|
||||
if state.scan_plan_digest != Some(baseline_plan_digest.0) {
|
||||
return Err(ScannerBaselineRejection::SetPlan);
|
||||
}
|
||||
}
|
||||
|
||||
(states == *proof.expected_sources).then_some(baseline_plan_digest)
|
||||
if states != *proof.expected_sources {
|
||||
return Err(ScannerBaselineRejection::SourceCoverage);
|
||||
}
|
||||
Ok(baseline_plan_digest)
|
||||
}
|
||||
|
||||
/// Whether `digest` is `structural` under any work flag: the same bucket
|
||||
@@ -493,10 +533,35 @@ fn scanner_baseline_pending_buckets(proof: &ScannerCacheBaselineProof<'_>) -> Ve
|
||||
}
|
||||
|
||||
fn complete_scanner_cache_baseline_plan_digest(proof: ScannerCacheBaselineProof<'_>) -> Option<DataUsageScanPlanDigest> {
|
||||
let authoritative = serde_json::from_slice::<DataUsageInfo>(proof.authoritative_data?).ok()?;
|
||||
if let Some(validated_digest) = complete_scanner_cache_snapshot_plan_digest(&authoritative, proof, None) {
|
||||
return Some(validated_digest);
|
||||
}
|
||||
validate_scanner_cache_baseline_plan_digest(proof).ok()
|
||||
}
|
||||
|
||||
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
|
||||
struct ScannerBaselineValidationFailure {
|
||||
authoritative: ScannerBaselineRejection,
|
||||
observed: Option<ScannerBaselineRejection>,
|
||||
}
|
||||
|
||||
fn validate_scanner_cache_baseline_plan_digest(
|
||||
proof: ScannerCacheBaselineProof<'_>,
|
||||
) -> std::result::Result<DataUsageScanPlanDigest, ScannerBaselineValidationFailure> {
|
||||
let authoritative_error = |authoritative| ScannerBaselineValidationFailure {
|
||||
authoritative,
|
||||
observed: None,
|
||||
};
|
||||
let authoritative_data = proof
|
||||
.authoritative_data
|
||||
.ok_or_else(|| authoritative_error(ScannerBaselineRejection::Missing))?;
|
||||
let authoritative = serde_json::from_slice::<DataUsageInfo>(authoritative_data)
|
||||
.map_err(|_| authoritative_error(ScannerBaselineRejection::Decode))?;
|
||||
let authoritative_rejection = match validate_scanner_cache_snapshot_plan_digest(&authoritative, proof, None) {
|
||||
Ok(validated_digest) => return Ok(validated_digest),
|
||||
Err(reason) => reason,
|
||||
};
|
||||
let observed_error = |observed| ScannerBaselineValidationFailure {
|
||||
authoritative: authoritative_rejection,
|
||||
observed: Some(observed),
|
||||
};
|
||||
|
||||
// A complete but superseded observation may reuse its per-set cache only
|
||||
// when it was explicitly tied to the durable authoritative baseline. It
|
||||
@@ -506,14 +571,18 @@ fn complete_scanner_cache_baseline_plan_digest(proof: ScannerCacheBaselineProof<
|
||||
&& !authoritative.usage_snapshot_observational)
|
||||
|| crate::scanner::data_usage_info_is_bootstrap_pending(&authoritative);
|
||||
if !authoritative_has_identity {
|
||||
return None;
|
||||
return Err(observed_error(ScannerBaselineRejection::AuthoritativeIdentity));
|
||||
}
|
||||
let observed = serde_json::from_slice::<DataUsageInfo>(proof.observed_candidate_data?).ok()?;
|
||||
let observed_data = proof
|
||||
.observed_candidate_data
|
||||
.ok_or_else(|| observed_error(ScannerBaselineRejection::Missing))?;
|
||||
let observed =
|
||||
serde_json::from_slice::<DataUsageInfo>(observed_data).map_err(|_| observed_error(ScannerBaselineRejection::Decode))?;
|
||||
if !observed_data_usage_is_newer(&observed, &authoritative) {
|
||||
return None;
|
||||
return Err(observed_error(ScannerBaselineRejection::NotNewer));
|
||||
}
|
||||
|
||||
complete_scanner_cache_snapshot_plan_digest(&observed, proof, Some(false))
|
||||
validate_scanner_cache_snapshot_plan_digest(&observed, proof, Some(false)).map_err(observed_error)
|
||||
}
|
||||
|
||||
fn scanner_segment_reuse_baseline_producer_evidence(
|
||||
@@ -658,6 +727,12 @@ impl ScannerScopeFallbackReason {
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Clone, Copy)]
|
||||
struct ScannerScopeFallback {
|
||||
reason: ScannerScopeFallbackReason,
|
||||
baseline_rejection: Option<ScannerBaselineValidationFailure>,
|
||||
}
|
||||
|
||||
/// Classify a resolved full-namespace scope against the preconditions
|
||||
/// `resolve_scanner_bucket_scan_scope` checks, in the same order. Returns
|
||||
/// `None` when the cycle was required to walk everything or was scoped.
|
||||
@@ -668,18 +743,24 @@ fn scanner_scope_fallback_reason(
|
||||
all_buckets: &[BucketInfo],
|
||||
baseline_proof: ScannerCacheBaselineProof<'_>,
|
||||
distributed: bool,
|
||||
) -> Option<ScannerScopeFallbackReason> {
|
||||
) -> Option<ScannerScopeFallback> {
|
||||
let fallback = |reason| {
|
||||
Some(ScannerScopeFallback {
|
||||
reason,
|
||||
baseline_rejection: None,
|
||||
})
|
||||
};
|
||||
if !scope_eligible || !resolved_scope.is_default() {
|
||||
return None;
|
||||
}
|
||||
if dirty_usage_snapshot.generation == u64::MAX {
|
||||
return Some(ScannerScopeFallbackReason::DirtyGenerationExhausted);
|
||||
return fallback(ScannerScopeFallbackReason::DirtyGenerationExhausted);
|
||||
}
|
||||
if !dirty_usage_snapshot.covers_listed_pending {
|
||||
return Some(ScannerScopeFallbackReason::DirtySnapshotIncomplete);
|
||||
return fallback(ScannerScopeFallbackReason::DirtySnapshotIncomplete);
|
||||
}
|
||||
if dirty_usage_snapshot.buckets.len() > crate::SCANNER_DIRTY_USAGE_SNAPSHOT_MAX_ENTRIES {
|
||||
return Some(ScannerScopeFallbackReason::DirtyBucketOverflow);
|
||||
return fallback(ScannerScopeFallbackReason::DirtyBucketOverflow);
|
||||
}
|
||||
// Peers may contribute dirty buckets the local snapshot does not hold.
|
||||
if !distributed {
|
||||
@@ -689,13 +770,19 @@ fn scanner_scope_fallback_reason(
|
||||
.keys()
|
||||
.any(|bucket| listed.contains(bucket.as_str()))
|
||||
{
|
||||
return Some(ScannerScopeFallbackReason::NoListedDirtyBuckets);
|
||||
return fallback(ScannerScopeFallbackReason::NoListedDirtyBuckets);
|
||||
}
|
||||
}
|
||||
if complete_scanner_cache_baseline_plan_digest(baseline_proof).is_none() {
|
||||
return Some(ScannerScopeFallbackReason::BaselineUnavailable);
|
||||
if let Err(failure) = validate_scanner_cache_baseline_plan_digest(baseline_proof) {
|
||||
return Some(ScannerScopeFallback {
|
||||
reason: ScannerScopeFallbackReason::BaselineUnavailable,
|
||||
baseline_rejection: Some(failure),
|
||||
});
|
||||
}
|
||||
distributed.then_some(ScannerScopeFallbackReason::PeerScopeUnavailable)
|
||||
distributed.then_some(ScannerScopeFallback {
|
||||
reason: ScannerScopeFallbackReason::PeerScopeUnavailable,
|
||||
baseline_rejection: None,
|
||||
})
|
||||
}
|
||||
|
||||
fn scanner_segment_reuse_activation_preflight() -> ScannerSegmentReuseActivationPreflight {
|
||||
|
||||
@@ -13,7 +13,7 @@
|
||||
// limitations under the License.
|
||||
use super::*;
|
||||
/// scanner cache locks and the cache snapshot persist/publish path.
|
||||
use crate::data_usage_define::clone_cache_value;
|
||||
use crate::data_usage_define::{DataUsageCacheBaselineRevisions, clone_cache_value};
|
||||
use crate::{ScannerConfigObjectDelete, ScannerObjectIO};
|
||||
use rustfs_scanner_metrics::usage_candidate::{
|
||||
ScannerUsageCandidateKind as CandidateKind, ScannerUsageCandidateOutcome as CandidateOutcome,
|
||||
@@ -1494,7 +1494,7 @@ pub(super) async fn persist_and_publish_cache_snapshot(
|
||||
store: Arc<SetDisks>,
|
||||
updates: &mpsc::Sender<DataUsageCache>,
|
||||
mut cache_snapshot: DataUsageCache,
|
||||
initial_revisions: Option<&DataUsageCacheRevisions>,
|
||||
initial_revisions: Option<&DataUsageCacheBaselineRevisions>,
|
||||
cache_cycle_floor: &AtomicU64,
|
||||
expected_publication_epoch: u64,
|
||||
) -> Option<SystemTime> {
|
||||
@@ -1536,7 +1536,10 @@ pub(super) async fn persist_and_publish_cache_snapshot(
|
||||
};
|
||||
|
||||
let mut persisted = DataUsageCache::default();
|
||||
let revisions = match persisted.load_with_revisions(store.clone(), DATA_USAGE_CACHE_NAME).await {
|
||||
let current_baseline = match persisted
|
||||
.load_with_baseline_revisions(store.clone(), DATA_USAGE_CACHE_NAME)
|
||||
.await
|
||||
{
|
||||
Ok(revisions) => revisions,
|
||||
Err(err) => {
|
||||
error!(
|
||||
@@ -1558,6 +1561,7 @@ pub(super) async fn persist_and_publish_cache_snapshot(
|
||||
return None;
|
||||
}
|
||||
};
|
||||
let revisions = current_baseline.revisions();
|
||||
let Some(scan_plan_digest) = cache_snapshot.info.scan_plan_digest else {
|
||||
log_missing_set_snapshot_metadata(&store, &cache_snapshot.info, "plan_digest");
|
||||
return None;
|
||||
@@ -1621,9 +1625,9 @@ pub(super) async fn persist_and_publish_cache_snapshot(
|
||||
{
|
||||
cache_snapshot = persisted;
|
||||
} else {
|
||||
// A later execution may have completed while this scan was walking.
|
||||
// Only replace the cache revision from which this scan started.
|
||||
if initial_revisions != Some(&revisions) {
|
||||
// Another execution must not advance the baseline. A backup catching
|
||||
// up to the same decoded primary changes no baseline content.
|
||||
if !initial_revisions.is_some_and(|initial| current_baseline.permits_unchanged_baseline_since(initial)) {
|
||||
warn!(
|
||||
target: "rustfs::scanner::io",
|
||||
event = EVENT_SCANNER_CACHE_PERSIST_STATE,
|
||||
@@ -1662,7 +1666,7 @@ pub(super) async fn persist_and_publish_cache_snapshot(
|
||||
|
||||
let done_save = Metrics::time(Metric::SaveUsage);
|
||||
if let Err(e) = cache_snapshot
|
||||
.save_with_revisions_for_epoch(store.clone(), DATA_USAGE_CACHE_NAME, &revisions, expected_publication_epoch)
|
||||
.save_with_revisions_for_epoch(store.clone(), DATA_USAGE_CACHE_NAME, revisions, expected_publication_epoch)
|
||||
.await
|
||||
{
|
||||
error!(
|
||||
|
||||
@@ -14,6 +14,7 @@
|
||||
use super::*;
|
||||
/// ScannerIOCache implementation for SetDisks: bucket ordering, worker fan-out, merge, and publish.
|
||||
use crate::data_usage_define::clone_cache_value;
|
||||
use rustfs_scanner_metrics::scan_observation::{ScannerBucketReuseKind, ScannerScanObservation};
|
||||
|
||||
pub(super) async fn collect_completed_bucket_results(
|
||||
ctx: CancellationToken,
|
||||
@@ -142,6 +143,32 @@ pub(super) struct PreparedScopedSetScan {
|
||||
pub(super) buckets: Vec<BucketInfo>,
|
||||
pub(super) cache: DataUsageCache,
|
||||
pub(super) cold_bucket_reuse_proof: Option<ScopedColdBucketReuseProof>,
|
||||
cold_bucket_reuse_count: u64,
|
||||
cold_bucket_reuse_elapsed: Duration,
|
||||
}
|
||||
|
||||
impl PreparedScopedSetScan {
|
||||
pub(super) fn record_cold_bucket_reuse(&self, observation: &ScannerScanObservation) {
|
||||
observation.record_bucket_reuse(
|
||||
ScannerBucketReuseKind::ScopedColdBucket,
|
||||
self.cold_bucket_reuse_count,
|
||||
self.cold_bucket_reuse_elapsed,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/// Counts only roots delivered to the collector; cancellation or channel failure is not reuse.
|
||||
pub(super) async fn send_current_cache_root_entry(
|
||||
bucket_result_tx: &mpsc::Sender<DataUsageEntryInfo>,
|
||||
root: DataUsageEntryInfo,
|
||||
cache: &DataUsageCache,
|
||||
pending_maintenance_work: &AtomicBool,
|
||||
observation: &ScannerScanObservation,
|
||||
) -> std::result::Result<(), ScannerError> {
|
||||
let started = Instant::now();
|
||||
send_cache_root_entry(bucket_result_tx, root, cache, pending_maintenance_work).await?;
|
||||
observation.record_bucket_reuse(ScannerBucketReuseKind::CurrentRoot, 1, started.elapsed());
|
||||
Ok(())
|
||||
}
|
||||
|
||||
pub(super) struct ScopedColdBucketReuseProof {
|
||||
@@ -225,6 +252,8 @@ pub(super) fn prepare_scoped_set_scan(
|
||||
};
|
||||
cache.replace(DATA_USAGE_ROOT, "", DataUsageEntry::default());
|
||||
let mut current_bucket_names = HashSet::with_capacity(all_buckets.len());
|
||||
let mut cold_bucket_reuse_count = 0u64;
|
||||
let mut cold_bucket_reuse_elapsed = Duration::ZERO;
|
||||
for bucket in all_buckets {
|
||||
if !current_bucket_names.insert(bucket.name.as_str()) {
|
||||
return None;
|
||||
@@ -233,11 +262,14 @@ pub(super) fn prepare_scoped_set_scan(
|
||||
cache.info.scan_bucket_value_origins.remove(&bucket.name);
|
||||
cache.replace(&bucket.name, DATA_USAGE_ROOT, DataUsageEntry::default());
|
||||
} else {
|
||||
let started = Instant::now();
|
||||
cache.copy_with_children(
|
||||
old_cache,
|
||||
&rustfs_data_usage::hash_path(&bucket.name),
|
||||
&Some(rustfs_data_usage::hash_path(DATA_USAGE_ROOT)),
|
||||
);
|
||||
cold_bucket_reuse_count = cold_bucket_reuse_count.saturating_add(1);
|
||||
cold_bucket_reuse_elapsed = cold_bucket_reuse_elapsed.saturating_add(started.elapsed());
|
||||
}
|
||||
}
|
||||
|
||||
@@ -249,6 +281,8 @@ pub(super) fn prepare_scoped_set_scan(
|
||||
.collect(),
|
||||
cache,
|
||||
cold_bucket_reuse_proof,
|
||||
cold_bucket_reuse_count,
|
||||
cold_bucket_reuse_elapsed,
|
||||
})
|
||||
}
|
||||
|
||||
@@ -425,7 +459,10 @@ impl ScannerIOCache for SetDisks {
|
||||
.ok_or_else(|| StorageError::other("scanner cache publication is blocked by data movement"))?,
|
||||
};
|
||||
let mut old_cache = DataUsageCache::default();
|
||||
let initial_revisions = match old_cache.load_with_revisions(self.clone(), DATA_USAGE_CACHE_NAME).await {
|
||||
let initial_revisions = match old_cache
|
||||
.load_with_baseline_revisions(self.clone(), DATA_USAGE_CACHE_NAME)
|
||||
.await
|
||||
{
|
||||
Ok(revisions) => Some(revisions),
|
||||
Err(e) => {
|
||||
warn!(
|
||||
@@ -555,6 +592,8 @@ impl ScannerIOCache for SetDisks {
|
||||
})
|
||||
});
|
||||
let mut scoped_cache = scoped_scan.map(|mut prepared| {
|
||||
// The copies are completed work even if this turn later fails publication.
|
||||
prepared.record_cold_bucket_reuse(global_metrics().scan_observation());
|
||||
buckets = prepared.buckets;
|
||||
prepared.cache.info.scan_coverage_digest = Some(bucket_coverage_digest);
|
||||
prepared.cache
|
||||
@@ -1558,8 +1597,14 @@ impl ScannerIOCache for SetDisks {
|
||||
}
|
||||
record_completed_usage_watch(&usage_watch_completions_clone, source, &bucket.name, bucket_usage_watch.as_ref()).await;
|
||||
if let Err(e) =
|
||||
send_cache_root_entry(&bucket_result_tx_clone, *root, &cache, &pending_maintenance_work_clone)
|
||||
.await
|
||||
send_current_cache_root_entry(
|
||||
&bucket_result_tx_clone,
|
||||
*root,
|
||||
&cache,
|
||||
&pending_maintenance_work_clone,
|
||||
global_metrics().scan_observation(),
|
||||
)
|
||||
.await
|
||||
{
|
||||
record_failed_dirty_bucket(&failed_dirty_buckets_clone, &bucket.name).await;
|
||||
error!(
|
||||
|
||||
@@ -357,6 +357,7 @@ where
|
||||
S: ScannerStorage,
|
||||
{
|
||||
let default_result = |scope: ScannerBucketScanScope| ScannerBucketScopeResolutionResult {
|
||||
segment_reuse_activation_preflight: None,
|
||||
scope,
|
||||
remote_dirty_usage_acknowledgements: Vec::new(),
|
||||
distributed_segment_invalidation_evidence: None,
|
||||
@@ -506,7 +507,7 @@ where
|
||||
distributed,
|
||||
resolution.baseline_proof,
|
||||
);
|
||||
default_result(scoped_scan_scope_from_dirty_buckets(
|
||||
let scope = scoped_scan_scope_from_dirty_buckets(
|
||||
resolution.requested_scope,
|
||||
dirty_buckets,
|
||||
(!distributed).then_some(resolution.dirty_usage_snapshot.scopes.as_ref()),
|
||||
@@ -514,7 +515,11 @@ where
|
||||
segment_reuse_activation_preflight.scanner_segment_reuse_activated,
|
||||
resolution.all_buckets,
|
||||
resolution.baseline_proof,
|
||||
))
|
||||
);
|
||||
ScannerBucketScopeResolutionResult {
|
||||
segment_reuse_activation_preflight: Some(segment_reuse_activation_preflight),
|
||||
..default_result(scope)
|
||||
}
|
||||
}
|
||||
|
||||
/// Runs once between the bucket listing and the dirty-usage snapshot, the
|
||||
@@ -759,6 +764,9 @@ where
|
||||
},
|
||||
)
|
||||
.await;
|
||||
if let Some(preflight) = scope_resolution.segment_reuse_activation_preflight {
|
||||
preflight.record_observation(ScannerPrefixReuseStage::ScopeResolution);
|
||||
}
|
||||
let remote_dirty_usage_acknowledgements = scope_resolution.remote_dirty_usage_acknowledgements;
|
||||
let distributed_segment_invalidation_evidence = scope_resolution.distributed_segment_invalidation_evidence;
|
||||
let scan_scope = scope_resolution.scope;
|
||||
@@ -770,8 +778,14 @@ where
|
||||
baseline_proof,
|
||||
distributed,
|
||||
);
|
||||
if let Some(reason) = scope_fallback {
|
||||
if let Some(fallback) = scope_fallback {
|
||||
let reason = fallback.reason;
|
||||
global_metrics().record_scanner_scope_fallback(reason.as_str());
|
||||
if let Some(failure) = fallback.baseline_rejection {
|
||||
global_metrics()
|
||||
.scan_observation()
|
||||
.record_baseline_rejection(failure.authoritative, failure.observed);
|
||||
}
|
||||
tracing::info!(
|
||||
target: "rustfs::scanner::io",
|
||||
event = EVENT_SCANNER_SET_STATE,
|
||||
@@ -785,6 +799,7 @@ where
|
||||
"Scanner cycle walks every bucket because its dirty-bucket scope was not admitted"
|
||||
);
|
||||
}
|
||||
let scope_fallback = scope_fallback.map(|fallback| fallback.reason);
|
||||
let segment_invalidation_proof = dirty_usage_producer_evidence(&dirty_usage_snapshot).segment_invalidation_proof();
|
||||
#[cfg(test)]
|
||||
if let Some(observer) = resolved_scope_observer {
|
||||
@@ -861,6 +876,7 @@ where
|
||||
None,
|
||||
false,
|
||||
);
|
||||
segment_reuse_activation_preflight.record_observation(ScannerPrefixReuseStage::CycleCompletion);
|
||||
return Ok(ScannerCycleResult::new(status, dirty_usage_clear)
|
||||
.with_publication_epoch(publication_epoch)
|
||||
.with_activity_digest(activity_digest)
|
||||
@@ -1207,6 +1223,7 @@ where
|
||||
distributed_segment_invalidation_evidence,
|
||||
cold_zero_walk_oracle,
|
||||
);
|
||||
segment_reuse_activation_preflight.record_observation(ScannerPrefixReuseStage::CycleCompletion);
|
||||
let pending_maintenance_work = pending_maintenance_work_for_cycle(&pending_maintenance_work, &results);
|
||||
let observed_cycle_floor = cache_cycle_floor.load(Ordering::Acquire);
|
||||
let required_cycle_floor = (observed_cycle_floor > want_cycle).then_some(observed_cycle_floor);
|
||||
|
||||
@@ -15,6 +15,7 @@ use super::*;
|
||||
use crate::UNKNOWN_TIER;
|
||||
/// ScannerIODisk implementation for Disk: get_size and the per-disk bucket scan.
|
||||
use crate::data_usage_define::clone_cache_value;
|
||||
use rustfs_scanner_metrics::scan_observation::ScannerMetadataReadOutcome;
|
||||
|
||||
///
|
||||
/// Seed [`SizeSummary::tier_stats`] from the cached tier-name list.
|
||||
@@ -62,8 +63,20 @@ impl ScannerIODisk for Disk {
|
||||
|
||||
ensure_scanner_node_operation_active(cancellation.as_ref())?;
|
||||
let metadata_object_path = item.object_path();
|
||||
let metadata_result =
|
||||
crate::workload_admission::with_scanner_metadata_io(self.read_metadata(&item.bucket, &metadata_object_path)).await;
|
||||
let metadata_result = crate::workload_admission::with_scanner_metadata_io(async {
|
||||
let read = global_metrics().start_scanner_metadata_read();
|
||||
let result = self.read_metadata(&item.bucket, &metadata_object_path).await;
|
||||
let (outcome, bytes) = match &result {
|
||||
Ok(data) => (ScannerMetadataReadOutcome::Success, u64::try_from(data.len()).unwrap_or(u64::MAX)),
|
||||
Err(e) if DiskError::is_err_object_not_found(e) || DiskError::is_err_version_not_found(e) => {
|
||||
(ScannerMetadataReadOutcome::NotFound, 0)
|
||||
}
|
||||
Err(_) => (ScannerMetadataReadOutcome::Error, 0),
|
||||
};
|
||||
read.finish(outcome, bytes);
|
||||
result
|
||||
})
|
||||
.await;
|
||||
ensure_scanner_node_operation_active(cancellation.as_ref())?;
|
||||
let data = match metadata_result {
|
||||
Ok(data) => data,
|
||||
|
||||
@@ -27,10 +27,10 @@ use crate::storage_api::owner::{
|
||||
};
|
||||
use crate::storage_api::scan::{BucketOperations as _, DeleteBucketOptions, MakeBucketOptions, ObjectIO as _};
|
||||
use crate::{
|
||||
DATA_USAGE_BLOOM_NAME_PATH, DiskOption, ECStore, Endpoint, EndpointServerPools, Endpoints, InstanceContext, PoolEndpoints,
|
||||
ScannerObjectOptions, ScannerPutObjReader, UNKNOWN_TIER, init_bucket_metadata_sys_for_scanner_tests,
|
||||
init_ecstore_config_for_scanner_tests, init_local_disks_with_instance_ctx, new_disk, path2_bucket_object_with_base_path,
|
||||
save_config,
|
||||
BUCKET_META_PREFIX, DATA_USAGE_BLOOM_NAME_PATH, DiskOption, ECStore, Endpoint, EndpointServerPools, Endpoints,
|
||||
InstanceContext, PoolEndpoints, ScannerObjectOptions, ScannerPutObjReader, UNKNOWN_TIER,
|
||||
init_bucket_metadata_sys_for_scanner_tests, init_ecstore_config_for_scanner_tests, init_local_disks_with_instance_ctx,
|
||||
new_disk, path2_bucket_object_with_base_path, save_config,
|
||||
};
|
||||
use rustfs_concurrency::{
|
||||
AdmissionState, WorkloadAdmissionRegistrySnapshot, WorkloadAdmissionSnapshot, WorkloadAdmissionSnapshotProvider,
|
||||
@@ -104,6 +104,7 @@ fn scanner_activity_preflight_defers_a_temporarily_offline_peer() {
|
||||
|
||||
#[test]
|
||||
fn scanner_segment_reuse_activation_preflight_reports_release_gate_inputs() {
|
||||
let before = global_metrics().scan_observation().snapshot();
|
||||
let preflight = scanner_segment_reuse_activation_preflight();
|
||||
|
||||
assert!(preflight.production_activation);
|
||||
@@ -124,6 +125,9 @@ fn scanner_segment_reuse_activation_preflight_reports_release_gate_inputs() {
|
||||
"distributed_without_peer_invalidation",
|
||||
]
|
||||
);
|
||||
let after = global_metrics().scan_observation().snapshot();
|
||||
assert_eq!(after.prefix_scope_resolution.evaluated, before.prefix_scope_resolution.evaluated);
|
||||
assert_eq!(after.prefix_cycle_completion.evaluated, before.prefix_cycle_completion.evaluated);
|
||||
}
|
||||
|
||||
#[test]
|
||||
@@ -2386,6 +2390,254 @@ async fn set_cache_send_failure_logs_receiver_and_snapshot_without_changing_retu
|
||||
assert_set_cache_context(set_cache_event(&events, "publish_failed"), Some((0, 0)));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
#[serial]
|
||||
async fn set_snapshot_accepts_backup_catchup_to_the_same_decoded_primary() {
|
||||
use crate::data_usage_define::read_config_with_revision;
|
||||
|
||||
let (_temp_dir, store, cache, epoch) = set_cache_log_fixture().await;
|
||||
let set = Arc::clone(&store.pools[0].disk_set[0]);
|
||||
let primary_path = path_join_buf(&[BUCKET_META_PREFIX, DATA_USAGE_CACHE_NAME]);
|
||||
let backup_path = format!("{primary_path}.bkp");
|
||||
let (primary_body, _) = read_config_with_revision(Arc::clone(&set), &primary_path)
|
||||
.await
|
||||
.expect("read decoded primary bytes");
|
||||
let primary_body = primary_body.expect("primary exists");
|
||||
let mut older = cache.clone();
|
||||
older.info.scan_execution_digest = Some(DataUsageScanPlanDigest([8; 32]));
|
||||
save_config(Arc::clone(&set), &backup_path, older.marshal_msg().expect("encode older backup"))
|
||||
.await
|
||||
.expect("seed a lagging backup");
|
||||
let mut loaded = DataUsageCache::default();
|
||||
let initial = loaded
|
||||
.load_with_baseline_revisions(Arc::clone(&set), DATA_USAGE_CACHE_NAME)
|
||||
.await
|
||||
.expect("capture primary with lagging backup");
|
||||
save_config(Arc::clone(&set), &backup_path, primary_body)
|
||||
.await
|
||||
.expect("backup catches up without changing primary");
|
||||
let current = loaded
|
||||
.load_with_baseline_revisions(Arc::clone(&set), DATA_USAGE_CACHE_NAME)
|
||||
.await
|
||||
.expect("capture the reconciled backup");
|
||||
assert_ne!(initial.revisions(), current.revisions());
|
||||
assert!(current.permits_unchanged_baseline_since(&initial));
|
||||
|
||||
let mut candidate = cache;
|
||||
candidate.info.next_cycle += 1;
|
||||
candidate.info.scan_execution_digest = Some(DataUsageScanPlanDigest([3; 32]));
|
||||
candidate.replace(
|
||||
"photos",
|
||||
DATA_USAGE_ROOT,
|
||||
DataUsageEntry {
|
||||
size: 20,
|
||||
objects: 1,
|
||||
..Default::default()
|
||||
},
|
||||
);
|
||||
let cycle = candidate.info.next_cycle;
|
||||
let cycle_floor = AtomicU64::new(cycle);
|
||||
let (tx, mut rx) = mpsc::channel(1);
|
||||
assert!(
|
||||
persist_and_publish_cache_snapshot(Arc::clone(&set), &tx, candidate, Some(&initial), &cycle_floor, epoch)
|
||||
.await
|
||||
.is_some(),
|
||||
"content-preserving backup catchup must not veto the new snapshot"
|
||||
);
|
||||
let published = rx.try_recv().expect("new snapshot reached the collector");
|
||||
assert_eq!(published.info.next_cycle, cycle);
|
||||
assert_eq!(published.find("photos").expect("new bucket entry").size, 20);
|
||||
let (primary, primary_revision) = read_config_with_revision(Arc::clone(&set), &primary_path)
|
||||
.await
|
||||
.expect("read new primary");
|
||||
let (backup, backup_revision) = read_config_with_revision(set, &backup_path).await.expect("read new backup");
|
||||
assert_eq!(primary, backup, "backup CAS must use the reconciled current revision");
|
||||
assert_eq!(primary_revision, backup_revision);
|
||||
let persisted = DataUsageCache::unmarshal(&primary.expect("new primary body")).expect("decode new primary");
|
||||
assert_eq!(persisted.info.next_cycle, cycle);
|
||||
assert_eq!(persisted.find("photos").expect("persisted new entry").size, 20);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
#[serial]
|
||||
async fn set_snapshot_backup_catchup_keeps_revision_provenance_and_epoch_fences() {
|
||||
use crate::data_usage_define::read_config_with_revision;
|
||||
|
||||
for case in ["different_backup", "different_primary", "corrupt_primary", "changed_epoch"] {
|
||||
let (_temp_dir, store, cache, epoch) = set_cache_log_fixture().await;
|
||||
let set = Arc::clone(&store.pools[0].disk_set[0]);
|
||||
let primary_path = path_join_buf(&[BUCKET_META_PREFIX, DATA_USAGE_CACHE_NAME]);
|
||||
let backup_path = format!("{primary_path}.bkp");
|
||||
let (primary_body, _) = read_config_with_revision(Arc::clone(&set), &primary_path)
|
||||
.await
|
||||
.expect("read original primary");
|
||||
let primary_body = primary_body.expect("original primary exists");
|
||||
let mut older = cache.clone();
|
||||
older.info.scan_execution_digest = Some(DataUsageScanPlanDigest([8; 32]));
|
||||
save_config(Arc::clone(&set), &backup_path, older.marshal_msg().expect("encode lagging backup"))
|
||||
.await
|
||||
.expect("seed lagging backup");
|
||||
if case == "corrupt_primary" {
|
||||
save_config(Arc::clone(&set), &primary_path, b"corrupt-primary".to_vec())
|
||||
.await
|
||||
.expect("inject primary corruption before the initial load");
|
||||
}
|
||||
let mut loaded = DataUsageCache::default();
|
||||
let initial = loaded
|
||||
.load_with_baseline_revisions(Arc::clone(&set), DATA_USAGE_CACHE_NAME)
|
||||
.await
|
||||
.expect("capture original baseline source and revisions");
|
||||
let mut replacement = cache.clone();
|
||||
replacement.info.scan_execution_digest = Some(DataUsageScanPlanDigest([9; 32]));
|
||||
match case {
|
||||
"different_backup" => {
|
||||
save_config(Arc::clone(&set), &backup_path, replacement.marshal_msg().expect("encode changed backup"))
|
||||
.await
|
||||
.expect("backup changes to different contents");
|
||||
}
|
||||
"different_primary" => {
|
||||
let body = replacement.marshal_msg().expect("encode changed primary");
|
||||
save_config(Arc::clone(&set), &primary_path, body.clone())
|
||||
.await
|
||||
.expect("primary advances");
|
||||
save_config(Arc::clone(&set), &backup_path, body)
|
||||
.await
|
||||
.expect("backup follows changed primary");
|
||||
}
|
||||
"corrupt_primary" => {
|
||||
save_config(Arc::clone(&set), &backup_path, b"corrupt-primary".to_vec())
|
||||
.await
|
||||
.expect("backup matches the corrupt primary ETag without a decoded primary proof");
|
||||
}
|
||||
"changed_epoch" => {
|
||||
save_config(Arc::clone(&set), &backup_path, primary_body)
|
||||
.await
|
||||
.expect("backup catches up before epoch refusal");
|
||||
}
|
||||
_ => unreachable!("fixed test cases"),
|
||||
}
|
||||
let before_primary = read_config_with_revision(Arc::clone(&set), &primary_path)
|
||||
.await
|
||||
.expect("current primary");
|
||||
let before_backup = read_config_with_revision(Arc::clone(&set), &backup_path)
|
||||
.await
|
||||
.expect("current backup");
|
||||
let mut candidate = cache;
|
||||
candidate.info.scan_execution_digest = Some(DataUsageScanPlanDigest([3; 32]));
|
||||
let cycle_floor = AtomicU64::new(candidate.info.next_cycle);
|
||||
let requested_epoch = if case == "changed_epoch" {
|
||||
epoch.checked_add(1).expect("fixture epoch can advance")
|
||||
} else {
|
||||
epoch
|
||||
};
|
||||
let (tx, mut rx) = mpsc::channel(1);
|
||||
let (result, events) = capture_set_cache_logs(persist_and_publish_cache_snapshot(
|
||||
Arc::clone(&set),
|
||||
&tx,
|
||||
candidate,
|
||||
Some(&initial),
|
||||
&cycle_floor,
|
||||
requested_epoch,
|
||||
))
|
||||
.await;
|
||||
assert!(result.is_none(), "{case} must still refuse publication");
|
||||
assert!(
|
||||
matches!(rx.try_recv(), Err(mpsc::error::TryRecvError::Empty)),
|
||||
"{case} forwards no snapshot"
|
||||
);
|
||||
let expected_state = if case == "changed_epoch" {
|
||||
"save_failed"
|
||||
} else {
|
||||
"scan_baseline_revision_changed"
|
||||
};
|
||||
assert_set_cache_context(set_cache_event(&events, expected_state), Some((0, 0)));
|
||||
assert_eq!(
|
||||
read_config_with_revision(Arc::clone(&set), &primary_path)
|
||||
.await
|
||||
.expect("protected primary"),
|
||||
before_primary,
|
||||
"{case} does not overwrite primary"
|
||||
);
|
||||
assert_eq!(
|
||||
read_config_with_revision(set, &backup_path).await.expect("protected backup"),
|
||||
before_backup,
|
||||
"{case} does not overwrite backup"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
#[serial]
|
||||
async fn set_cache_cas_preserves_updates_after_backup_catchup_validation() {
|
||||
use crate::data_usage_define::read_config_with_revision;
|
||||
|
||||
for changed_copy in ["primary", "backup"] {
|
||||
let (_temp_dir, store, cache, epoch) = set_cache_log_fixture().await;
|
||||
let set = Arc::clone(&store.pools[0].disk_set[0]);
|
||||
let primary_path = path_join_buf(&[BUCKET_META_PREFIX, DATA_USAGE_CACHE_NAME]);
|
||||
let backup_path = format!("{primary_path}.bkp");
|
||||
let (primary_body, _) = read_config_with_revision(Arc::clone(&set), &primary_path)
|
||||
.await
|
||||
.expect("read initial primary");
|
||||
let mut lagging = cache.clone();
|
||||
lagging.info.scan_execution_digest = Some(DataUsageScanPlanDigest([8; 32]));
|
||||
save_config(Arc::clone(&set), &backup_path, lagging.marshal_msg().expect("lagging backup bytes"))
|
||||
.await
|
||||
.expect("seed backup lag");
|
||||
let mut loaded = DataUsageCache::default();
|
||||
let initial = loaded
|
||||
.load_with_baseline_revisions(Arc::clone(&set), DATA_USAGE_CACHE_NAME)
|
||||
.await
|
||||
.expect("read initial baseline");
|
||||
save_config(Arc::clone(&set), &backup_path, primary_body.expect("initial primary exists"))
|
||||
.await
|
||||
.expect("finish backup catchup");
|
||||
let current = loaded
|
||||
.load_with_baseline_revisions(Arc::clone(&set), DATA_USAGE_CACHE_NAME)
|
||||
.await
|
||||
.expect("read reconciled baseline");
|
||||
assert!(current.permits_unchanged_baseline_since(&initial));
|
||||
let changed_path = if changed_copy == "primary" {
|
||||
&primary_path
|
||||
} else {
|
||||
&backup_path
|
||||
};
|
||||
let mut concurrent = cache.clone();
|
||||
concurrent.info.scan_execution_digest = Some(DataUsageScanPlanDigest([9; 32]));
|
||||
let concurrent_body = concurrent.marshal_msg().expect("encode concurrent update");
|
||||
save_config(Arc::clone(&set), changed_path, concurrent_body.clone())
|
||||
.await
|
||||
.expect("copy changes after the baseline validation");
|
||||
let mut candidate = cache;
|
||||
candidate.info.next_cycle += 1;
|
||||
candidate.info.scan_execution_digest = Some(DataUsageScanPlanDigest([3; 32]));
|
||||
let result = candidate
|
||||
.save_with_revisions_for_epoch(Arc::clone(&set), DATA_USAGE_CACHE_NAME, current.revisions(), epoch)
|
||||
.await;
|
||||
if changed_copy == "primary" {
|
||||
assert!(result.is_err(), "primary CAS refuses the intervening update");
|
||||
} else {
|
||||
result.expect("primary remains durable when the best-effort backup CAS refuses an update");
|
||||
}
|
||||
let (protected, _) = read_config_with_revision(Arc::clone(&set), changed_path)
|
||||
.await
|
||||
.expect("read protected concurrent copy");
|
||||
assert_eq!(
|
||||
protected,
|
||||
Some(concurrent_body),
|
||||
"{changed_copy} CAS must not overwrite the concurrent update"
|
||||
);
|
||||
let (primary, _) = read_config_with_revision(set, &primary_path)
|
||||
.await
|
||||
.expect("read resulting primary");
|
||||
let primary = DataUsageCache::unmarshal(&primary.expect("primary exists")).expect("primary remains decodable");
|
||||
assert_eq!(
|
||||
primary.info.scan_execution_digest,
|
||||
Some(DataUsageScanPlanDigest([if changed_copy == "primary" { 9 } else { 3 }; 32])),
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
#[serial]
|
||||
async fn set_snapshot_reuse_requires_execution_identity_and_fences_stale_writers() {
|
||||
@@ -2400,7 +2652,7 @@ async fn set_snapshot_reuse_requires_execution_identity_and_fences_stale_writers
|
||||
.expect("seed legacy set cache");
|
||||
let mut persisted = DataUsageCache::default();
|
||||
let initial = persisted
|
||||
.load_with_revisions(Arc::clone(&set), DATA_USAGE_CACHE_NAME)
|
||||
.load_with_baseline_revisions(Arc::clone(&set), DATA_USAGE_CACHE_NAME)
|
||||
.await
|
||||
.expect("capture the shared starting revision");
|
||||
let mut fresh = legacy.clone();
|
||||
@@ -2426,7 +2678,7 @@ async fn set_snapshot_reuse_requires_execution_identity_and_fences_stale_writers
|
||||
assert_eq!(published.find("photos").expect("published bucket").size, 20);
|
||||
assert_eq!(published.info.scan_execution_digest, fresh.info.scan_execution_digest);
|
||||
let current = persisted
|
||||
.load_with_revisions(Arc::clone(&set), DATA_USAGE_CACHE_NAME)
|
||||
.load_with_baseline_revisions(Arc::clone(&set), DATA_USAGE_CACHE_NAME)
|
||||
.await
|
||||
.expect("capture the current revision for the unidentified execution");
|
||||
|
||||
@@ -2717,6 +2969,130 @@ fn scoped_scan_requires_a_complete_authoritative_baseline_with_exact_set_provena
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn scoped_scan_baseline_rejections_preserve_the_existing_first_failed_check() {
|
||||
let source = DataUsageCacheSource::new(1, 2);
|
||||
let expected_sources = HashSet::from([source]);
|
||||
let digest = DataUsageScanPlanDigest([9; 32]);
|
||||
let baseline = complete_usage_baseline(source, digest, 7, 11);
|
||||
let snapshot: DataUsageInfo = serde_json::from_slice(&baseline).expect("decode complete baseline");
|
||||
let proof = ScannerCacheBaselineProof {
|
||||
authoritative_data: Some(&baseline),
|
||||
observed_candidate_data: None,
|
||||
expected_sources: &expected_sources,
|
||||
leader_epoch: 11,
|
||||
want_cycle: 8,
|
||||
scan_plan_digest: digest,
|
||||
};
|
||||
type RejectionCase = (ScannerBaselineRejection, fn(&mut DataUsageInfo));
|
||||
let cases: [RejectionCase; 13] = [
|
||||
(ScannerBaselineRejection::Incomplete, |s| s.usage_snapshot_complete = false),
|
||||
(ScannerBaselineRejection::Partial, |s| s.usage_snapshot_partial = true),
|
||||
(ScannerBaselineRejection::Marks, |s| s.usage_snapshot_converged = None),
|
||||
(ScannerBaselineRejection::Epoch, |s| s.scanner_epoch = Some(10)),
|
||||
(ScannerBaselineRejection::SourceCount, |s| s.usage_snapshot_set_states.clear()),
|
||||
(ScannerBaselineRejection::PlanMissing, |s| {
|
||||
s.usage_snapshot_set_states[0].scan_plan_digest = None
|
||||
}),
|
||||
(ScannerBaselineRejection::PlanMismatch, |s| {
|
||||
s.usage_snapshot_set_states[0].scan_plan_digest = Some([8; 32])
|
||||
}),
|
||||
(ScannerBaselineRejection::SourceMembership, |s| {
|
||||
s.usage_snapshot_set_states[0].set_index = 3
|
||||
}),
|
||||
(ScannerBaselineRejection::SetIncomplete, |s| {
|
||||
s.usage_snapshot_set_states[0].complete = false
|
||||
}),
|
||||
(ScannerBaselineRejection::SetTombstone, |s| {
|
||||
s.usage_snapshot_set_states[0].tombstone = true
|
||||
}),
|
||||
(ScannerBaselineRejection::SetEpoch, |s| {
|
||||
s.usage_snapshot_set_states[0].scanner_epoch = None
|
||||
}),
|
||||
(ScannerBaselineRejection::SetCycle, |s| {
|
||||
s.usage_snapshot_set_states[0].scanner_cycle = Some(9)
|
||||
}),
|
||||
(ScannerBaselineRejection::SetCycle, |s| {
|
||||
s.usage_snapshot_set_states[0].scanner_cycle = None
|
||||
}),
|
||||
];
|
||||
assert_eq!(validate_scanner_cache_snapshot_plan_digest(&snapshot, proof, None), Ok(digest));
|
||||
for (expected, mutate) in cases {
|
||||
let mut invalid = snapshot.clone();
|
||||
mutate(&mut invalid);
|
||||
assert_eq!(validate_scanner_cache_snapshot_plan_digest(&invalid, proof, None), Err(expected));
|
||||
assert_eq!(complete_scanner_cache_snapshot_plan_digest(&invalid, proof, None), None);
|
||||
}
|
||||
let mut invalid = snapshot.clone();
|
||||
invalid.usage_snapshot_partial = true;
|
||||
invalid.scanner_epoch = Some(10);
|
||||
assert_eq!(
|
||||
validate_scanner_cache_snapshot_plan_digest(&invalid, proof, None),
|
||||
Err(ScannerBaselineRejection::Partial)
|
||||
);
|
||||
|
||||
let two_sources = HashSet::from([source, DataUsageCacheSource::new(1, 3)]);
|
||||
let mut two_sets = snapshot.clone();
|
||||
two_sets
|
||||
.usage_snapshot_set_states
|
||||
.push(two_sets.usage_snapshot_set_states[0].clone());
|
||||
let two_set_proof = ScannerCacheBaselineProof {
|
||||
expected_sources: &two_sources,
|
||||
..proof
|
||||
};
|
||||
assert_eq!(
|
||||
validate_scanner_cache_snapshot_plan_digest(&two_sets, two_set_proof, None),
|
||||
Err(ScannerBaselineRejection::SourceDuplicate)
|
||||
);
|
||||
two_sets.usage_snapshot_set_states[1].set_index = 3;
|
||||
two_sets.usage_snapshot_set_states[1].scan_plan_digest = Some([8; 32]);
|
||||
assert_eq!(
|
||||
validate_scanner_cache_snapshot_plan_digest(&two_sets, two_set_proof, None),
|
||||
Err(ScannerBaselineRejection::SetPlan)
|
||||
);
|
||||
|
||||
let undecodable = Bytes::from_static(b"not-json");
|
||||
for (data, expected) in [
|
||||
(None, ScannerBaselineRejection::Missing),
|
||||
(Some(&undecodable), ScannerBaselineRejection::Decode),
|
||||
] {
|
||||
assert_eq!(
|
||||
validate_scanner_cache_baseline_plan_digest(ScannerCacheBaselineProof {
|
||||
authoritative_data: data,
|
||||
..proof
|
||||
}),
|
||||
Err(ScannerBaselineValidationFailure {
|
||||
authoritative: expected,
|
||||
observed: None
|
||||
})
|
||||
);
|
||||
}
|
||||
let mut wrong_epoch = snapshot;
|
||||
wrong_epoch.scanner_epoch = Some(10);
|
||||
let wrong_epoch = Bytes::from(serde_json::to_vec(&wrong_epoch).expect("encode stale epoch baseline"));
|
||||
let stale_proof = ScannerCacheBaselineProof {
|
||||
authoritative_data: Some(&wrong_epoch),
|
||||
..proof
|
||||
};
|
||||
assert_eq!(
|
||||
validate_scanner_cache_baseline_plan_digest(stale_proof),
|
||||
Err(ScannerBaselineValidationFailure {
|
||||
authoritative: ScannerBaselineRejection::Epoch,
|
||||
observed: Some(ScannerBaselineRejection::Missing),
|
||||
})
|
||||
);
|
||||
assert_eq!(
|
||||
validate_scanner_cache_baseline_plan_digest(ScannerCacheBaselineProof {
|
||||
observed_candidate_data: Some(&undecodable),
|
||||
..stale_proof
|
||||
}),
|
||||
Err(ScannerBaselineValidationFailure {
|
||||
authoritative: ScannerBaselineRejection::Epoch,
|
||||
observed: Some(ScannerBaselineRejection::Decode)
|
||||
})
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn scoped_scan_accepts_only_a_complete_observation_tied_to_the_authoritative_baseline() {
|
||||
let source = DataUsageCacheSource::new(1, 2);
|
||||
@@ -3150,6 +3526,12 @@ fn distributed_segment_reuse_activation_keeps_remote_dirty_buckets_at_bucket_sco
|
||||
result.scope.prefix_scope_for("photos").is_some(),
|
||||
"durable local producer proof may activate local segment reuse after distributed ACK capability evidence"
|
||||
);
|
||||
assert!(
|
||||
result
|
||||
.segment_reuse_activation_preflight
|
||||
.expect("actual scope proof is retained for observation")
|
||||
.scanner_segment_reuse_activated
|
||||
);
|
||||
assert!(
|
||||
result.scope.prefix_scope_for("archive").is_none(),
|
||||
"remote dirty usage has only bucket-granularity evidence and must not be narrowed to a local prefix"
|
||||
@@ -3568,6 +3950,10 @@ fn scoped_set_scan_rebuilds_selected_buckets_and_drops_deleted_buckets() {
|
||||
.checked_flatten("stable")
|
||||
.expect("selected bucket placeholder should exist");
|
||||
assert_eq!((stable.size, stable.objects), (0, 0));
|
||||
let observation = rustfs_scanner_metrics::scan_observation::ScannerScanObservation::default();
|
||||
prepared.record_cold_bucket_reuse(&observation);
|
||||
assert_eq!(observation.snapshot().bucket_reuse["scoped_cold_bucket"].count, 0);
|
||||
|
||||
assert!(prepared.cache.find("stable/prefix").is_none());
|
||||
assert_eq!(prepared.cache.find("dirty").map(|entry| (entry.size, entry.objects)), Some((0, 0)));
|
||||
assert!(prepared.cache.find("deleted").is_none());
|
||||
@@ -3579,6 +3965,124 @@ fn scoped_set_scan_rebuilds_selected_buckets_and_drops_deleted_buckets() {
|
||||
assert_eq!(prepared.cache.info.lkg_scan_plan_digest, Some(baseline_digest));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn cold_bucket_reuse_observation_rejects_unbound_or_failed_preparation() {
|
||||
let baseline_digest = DataUsageScanPlanDigest([1; 32]);
|
||||
let mut old_cache = complete_set_usage_cache(&[("stable", 10), ("dirty", 20)], baseline_digest);
|
||||
old_cache.info.scan_bucket_incarnations = test_bucket_incarnations(&["stable", "dirty"]);
|
||||
let incarnations = old_cache.info.scan_bucket_incarnations.clone();
|
||||
let buckets = vec![
|
||||
bucket_info_with_created_time("stable"),
|
||||
bucket_info_with_created_time("dirty"),
|
||||
];
|
||||
let scope = ScannerBucketScanScope {
|
||||
selected_buckets: Some(Arc::new(HashSet::from(["dirty".to_string()]))),
|
||||
selected_bucket_prefixes: None,
|
||||
baseline_scan_plan_digest: Some(baseline_digest),
|
||||
};
|
||||
let generation = ScannerSetCacheGeneration {
|
||||
want_cycle: 8,
|
||||
leader_epoch: 11,
|
||||
tier_registry_generation: 13,
|
||||
source: DataUsageCacheSource::new(1, 2),
|
||||
scan_plan_digest: DataUsageScanPlanDigest([2; 32]),
|
||||
};
|
||||
let observation = rustfs_scanner_metrics::scan_observation::ScannerScanObservation::default();
|
||||
let prepared = prepare_scoped_set_scan(&old_cache, &buckets, &buckets, &scope, generation, Some(&incarnations))
|
||||
.expect("bound complete cache permits a cold bucket copy");
|
||||
prepared.record_cold_bucket_reuse(&observation);
|
||||
assert_eq!(observation.snapshot().bucket_reuse["scoped_cold_bucket"].count, 1);
|
||||
|
||||
for case in ["incomplete", "duplicate", "nil_incarnation"] {
|
||||
let mut invalid_cache = old_cache.clone();
|
||||
let mut invalid_buckets = buckets.clone();
|
||||
let mut invalid_incarnations = incarnations.clone();
|
||||
match case {
|
||||
"incomplete" => invalid_cache.info.snapshot_complete = false,
|
||||
"duplicate" => invalid_buckets.push(buckets[0].clone()),
|
||||
"nil_incarnation" => {
|
||||
invalid_incarnations.insert("stable".to_string(), Uuid::nil());
|
||||
}
|
||||
_ => unreachable!("fixed test cases"),
|
||||
}
|
||||
let prepared = prepare_scoped_set_scan(
|
||||
&invalid_cache,
|
||||
&invalid_buckets,
|
||||
&invalid_buckets,
|
||||
&scope,
|
||||
generation,
|
||||
Some(&invalid_incarnations),
|
||||
);
|
||||
assert!(prepared.is_none(), "{case} must not return a reusable prepared cache");
|
||||
if let Some(prepared) = prepared {
|
||||
prepared.record_cold_bucket_reuse(&observation);
|
||||
}
|
||||
assert_eq!(
|
||||
observation.snapshot().bucket_reuse["scoped_cold_bucket"].count,
|
||||
1,
|
||||
"{case} does not count completed reuse"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn current_root_reuse_counts_only_successful_collector_delivery() {
|
||||
use rustfs_scanner_metrics::scan_observation::ScannerScanObservation;
|
||||
|
||||
let observation = ScannerScanObservation::default();
|
||||
let cache = DataUsageCache::default();
|
||||
let pending = AtomicBool::new(false);
|
||||
let root = || DataUsageEntryInfo {
|
||||
name: "bucket".to_string(),
|
||||
parent: DATA_USAGE_ROOT.to_string(),
|
||||
entry: DataUsageEntry {
|
||||
size: 10,
|
||||
objects: 2,
|
||||
..Default::default()
|
||||
},
|
||||
bucket_incarnation: Some(Uuid::from_u128(7)),
|
||||
tier_registry_generation: None,
|
||||
value_origin: None,
|
||||
};
|
||||
let (sender, mut receiver) = mpsc::channel(1);
|
||||
super::io_cache::send_current_cache_root_entry(&sender, root(), &cache, &pending, &observation)
|
||||
.await
|
||||
.expect("current root is delivered to the collector");
|
||||
let delivered = receiver.recv().await.expect("collector received the root");
|
||||
assert_eq!((delivered.entry.size, delivered.entry.objects), (10, 2));
|
||||
assert_eq!(observation.snapshot().bucket_reuse["current_root"].count, 1);
|
||||
drop(receiver);
|
||||
assert!(
|
||||
super::io_cache::send_current_cache_root_entry(&sender, root(), &cache, &pending, &observation)
|
||||
.await
|
||||
.is_err()
|
||||
);
|
||||
assert_eq!(
|
||||
observation.snapshot().bucket_reuse["current_root"].count,
|
||||
1,
|
||||
"closed collector is not completed reuse"
|
||||
);
|
||||
|
||||
let (sender, _receiver) = mpsc::channel(1);
|
||||
sender.send(root()).await.expect("fill bounded collector channel");
|
||||
{
|
||||
let mut send = Box::pin(super::io_cache::send_current_cache_root_entry(
|
||||
&sender,
|
||||
root(),
|
||||
&cache,
|
||||
&pending,
|
||||
&observation,
|
||||
));
|
||||
assert!(futures::poll!(send.as_mut()).is_pending(), "full collector keeps reuse unfinished");
|
||||
}
|
||||
assert_eq!(
|
||||
observation.snapshot().bucket_reuse["current_root"].count,
|
||||
1,
|
||||
"dropping unfinished delivery is not reuse"
|
||||
);
|
||||
assert_eq!(observation.snapshot().bucket_reuse["scoped_cold_bucket"].count, 0);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn scoped_set_scan_reuses_unselected_buckets_with_matching_incarnations() {
|
||||
let baseline_digest = DataUsageScanPlanDigest([1; 32]);
|
||||
@@ -3620,6 +4124,12 @@ fn scoped_set_scan_reuses_unselected_buckets_with_matching_incarnations() {
|
||||
)
|
||||
.expect("matching bucket incarnations should authorize cold bucket reuse");
|
||||
|
||||
let observation = rustfs_scanner_metrics::scan_observation::ScannerScanObservation::default();
|
||||
prepared.record_cold_bucket_reuse(&observation);
|
||||
let observed = observation.snapshot();
|
||||
assert_eq!(observed.bucket_reuse["scoped_cold_bucket"].count, 1);
|
||||
assert_eq!(observed.bucket_reuse["current_root"].count, 0);
|
||||
assert!(observed.cache_reuse.values().all(|reuse| reuse.count == 0));
|
||||
assert_eq!(prepared.buckets.iter().map(|bucket| bucket.name.as_str()).collect::<Vec<_>>(), ["dirty"]);
|
||||
let stable = prepared
|
||||
.cache
|
||||
@@ -4625,12 +5135,36 @@ async fn get_size_treats_missing_metadata_as_skip_file() {
|
||||
debug: false,
|
||||
};
|
||||
|
||||
let before = global_metrics().scan_observation().snapshot().metadata_reads;
|
||||
let err = disk
|
||||
.get_size(item)
|
||||
.get_size(item.clone())
|
||||
.await
|
||||
.expect_err("missing metadata should be skipped instead of reported as a scanner failure");
|
||||
assert!(matches!(err, StorageError::Io(ref io) if io.to_string() == SCANNER_SKIP_FILE_ERROR));
|
||||
|
||||
let missing = global_metrics().scan_observation().snapshot().metadata_reads;
|
||||
assert_eq!(missing.attempts, before.attempts + 1);
|
||||
assert_eq!(missing.not_found, before.not_found + 1);
|
||||
assert_eq!(missing.bytes, before.bytes);
|
||||
|
||||
#[cfg(unix)]
|
||||
{
|
||||
tokio::fs::create_dir_all(&object_dir)
|
||||
.await
|
||||
.expect("recreate object directory");
|
||||
std::os::unix::fs::symlink(STORAGE_FORMAT_FILE, &metadata_path).expect("inject a metadata symlink loop");
|
||||
let err = disk
|
||||
.get_size(item)
|
||||
.await
|
||||
.expect_err("symlink loop must remain a read failure");
|
||||
assert!(is_scanner_metadata_transient_error(&err), "read error: {err:?}");
|
||||
let failed = global_metrics().scan_observation().snapshot().metadata_reads;
|
||||
assert_eq!(failed.attempts, missing.attempts + 1);
|
||||
assert_eq!(failed.errors, missing.errors + 1);
|
||||
assert_eq!(failed.succeeded, before.succeeded);
|
||||
assert_eq!(failed.bytes, before.bytes);
|
||||
}
|
||||
|
||||
let _ = tokio::fs::remove_dir_all(&temp_dir).await;
|
||||
}
|
||||
|
||||
@@ -4681,6 +5215,7 @@ async fn get_size_marks_corrupt_metadata_for_heal() {
|
||||
debug: false,
|
||||
};
|
||||
|
||||
let before = global_metrics().scan_observation().snapshot().metadata_reads;
|
||||
for contents in [b"".as_slice(), b"not-valid-filemeta".as_slice()] {
|
||||
tokio::fs::write(&metadata_path, contents)
|
||||
.await
|
||||
@@ -4692,6 +5227,16 @@ async fn get_size_marks_corrupt_metadata_for_heal() {
|
||||
assert!(is_scanner_metadata_corrupt_error(&err));
|
||||
}
|
||||
|
||||
let reads = global_metrics().scan_observation().snapshot().metadata_reads;
|
||||
assert_eq!(reads.attempts, before.attempts + 2);
|
||||
assert_eq!(
|
||||
reads.succeeded,
|
||||
before.succeeded + 2,
|
||||
"storage reads succeed before decoding rejects corruption"
|
||||
);
|
||||
assert_eq!(reads.errors, before.errors);
|
||||
assert_eq!(reads.bytes, before.bytes + 18);
|
||||
|
||||
let _ = tokio::fs::remove_dir_all(&temp_dir).await;
|
||||
}
|
||||
|
||||
@@ -4764,11 +5309,17 @@ async fn get_size_counts_delete_markers_separately_from_versions() {
|
||||
debug: false,
|
||||
};
|
||||
|
||||
let before = global_metrics().scan_observation().snapshot().metadata_reads;
|
||||
let expected_bytes = tokio::fs::metadata(&metadata_path).await.expect("stat stored metadata").len();
|
||||
let summary = disk.get_size(item).await.expect("scanner should read versioned metadata");
|
||||
|
||||
assert_eq!(summary.versions, 2);
|
||||
assert_eq!(summary.delete_markers, 1);
|
||||
assert_eq!(summary.total_size, 30);
|
||||
let reads = global_metrics().scan_observation().snapshot().metadata_reads;
|
||||
assert_eq!(reads.attempts, before.attempts + 1);
|
||||
assert_eq!(reads.succeeded, before.succeeded + 1);
|
||||
assert_eq!(reads.bytes, before.bytes + expected_bytes);
|
||||
|
||||
let _ = tokio::fs::remove_dir_all(&temp_dir).await;
|
||||
}
|
||||
|
||||
@@ -183,9 +183,11 @@ async fn dirty_entry_removed_during_listing_still_forces_a_full_scan() {
|
||||
);
|
||||
assert_eq!(second.buckets_usage["hot-bucket"].objects_count, 2);
|
||||
assert_eq!(second.buckets_usage["cold-bucket"].objects_count, 2);
|
||||
let freshness = global_metrics().report().await.usage_freshness;
|
||||
let report = global_metrics().report().await;
|
||||
let freshness = report.usage_freshness;
|
||||
assert_eq!(freshness.scope_fallback_total, fallbacks_before + 1);
|
||||
assert_eq!(freshness.last_scope_fallback_reason, "dirty_snapshot_incomplete");
|
||||
assert_eq!(report.scan_observation.scope_fallbacks["dirty_snapshot_incomplete"], 1);
|
||||
clear_dirty_usage_buckets_for_tests();
|
||||
}
|
||||
|
||||
@@ -199,11 +201,53 @@ async fn required_full_scan_is_not_reported_as_a_scope_fallback() {
|
||||
wait_for_namespace_commit_tails(store.as_ref()).await;
|
||||
super::io_cycle::set_bucket_listing_window_hook_for_tests(|| record_dirty_usage_bucket("created-after-listing"));
|
||||
|
||||
let before = global_metrics().scan_observation().snapshot();
|
||||
let (result, _, scope) = run_cycle(&store, 2, Some(baseline), true).await;
|
||||
|
||||
assert!(scope.is_default());
|
||||
assert_eq!(result.pending_dirty_usage_buckets(), 1);
|
||||
assert_eq!(result.scope_fallback(), None, "scheduled maintenance walks are not degradations");
|
||||
assert_eq!(result.completed_scope_fallback(), None);
|
||||
let after = global_metrics().scan_observation().snapshot();
|
||||
assert_eq!(after.scope_fallbacks, before.scope_fallbacks);
|
||||
assert_eq!(after.prefix_scope_resolution.evaluated, before.prefix_scope_resolution.evaluated);
|
||||
assert_eq!(after.prefix_cycle_completion.evaluated, before.prefix_cycle_completion.evaluated + 1);
|
||||
clear_dirty_usage_buckets_for_tests();
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
#[serial]
|
||||
async fn baseline_unavailable_cycle_records_real_scope_and_baseline_rejections_once() {
|
||||
let (_temp_dir, store) = setup_local_scanner_store(1, 1).await;
|
||||
let _baseline = converged_baseline(&store).await;
|
||||
put_fixture_object(&store, "hot-bucket", "second").await;
|
||||
record_dirty_usage_bucket("hot-bucket");
|
||||
wait_for_namespace_commit_tails(store.as_ref()).await;
|
||||
let before = global_metrics().scan_observation().snapshot();
|
||||
let (result, snapshot, scope) = run_cycle(&store, 2, Some(Bytes::from_static(b"not-json")), false).await;
|
||||
assert!(scope.is_default());
|
||||
assert_eq!(result.status, ScannerCycleStatus::Complete);
|
||||
assert_eq!(result.scope_fallback(), Some(ScannerScopeFallbackReason::BaselineUnavailable));
|
||||
assert_eq!(snapshot.buckets_usage["hot-bucket"].objects_count, 2);
|
||||
let after = global_metrics().scan_observation().snapshot();
|
||||
assert_eq!(
|
||||
after.scope_fallbacks["baseline_unavailable"],
|
||||
before.scope_fallbacks["baseline_unavailable"] + 1
|
||||
);
|
||||
assert_eq!(
|
||||
after.authoritative_baseline_rejections["decode"],
|
||||
before.authoritative_baseline_rejections["decode"] + 1
|
||||
);
|
||||
assert_eq!(
|
||||
after.observed_baseline_rejections, before.observed_baseline_rejections,
|
||||
"an undecodable authority never admits an observation"
|
||||
);
|
||||
assert_eq!(after.prefix_scope_resolution.evaluated, before.prefix_scope_resolution.evaluated + 1);
|
||||
assert_eq!(after.prefix_scope_resolution.activated, before.prefix_scope_resolution.activated);
|
||||
assert_eq!(
|
||||
after.prefix_scope_resolution.blockers["missing_cold_zero_walk_oracle"],
|
||||
before.prefix_scope_resolution.blockers["missing_cold_zero_walk_oracle"] + 1
|
||||
);
|
||||
assert_eq!(after.prefix_cycle_completion.evaluated, before.prefix_cycle_completion.evaluated + 1);
|
||||
clear_dirty_usage_buckets_for_tests();
|
||||
}
|
||||
|
||||
@@ -1,12 +1,12 @@
|
||||
// Copyright 2026 RustFS Team
|
||||
// SPDX-License-Identifier: Apache-2.0
|
||||
|
||||
//! A bucket whose walk stops early without exhausting the cycle budget (here
|
||||
//! a sweep resumed under a newer plan, the `mixed_coverage` case of
|
||||
//! rustfs/rustfs#8494) no longer makes the whole cycle `Incomplete`. The set
|
||||
//! publishes the bucket's last certified value, the cycle completes with the
|
||||
//! bucket pending, and the next cycle walks it again. A bucket that never
|
||||
//! certified keeps the cycle incomplete.
|
||||
//! A bucket whose walk stops early can publish its last certified value as
|
||||
//! pending. This fixture tests usage-scan status, not the outer scanner cycle
|
||||
//! outcome: budget exhaustion may leave complete validated usage while the
|
||||
//! coordinator records `Partial`. A continuation under a newer plan exercises
|
||||
//! `mixed_coverage` (rustfs/rustfs#8494). A bucket that never certified keeps
|
||||
//! the usage scan incomplete.
|
||||
|
||||
use super::*;
|
||||
|
||||
@@ -38,8 +38,7 @@ async fn run_cycle(store: &Arc<ECStore>, cycle: u64, options: CycleOptions) -> (
|
||||
..Default::default()
|
||||
},
|
||||
);
|
||||
// These cycles persist real bucket checkpoints and source caches. Keep
|
||||
// CI storage latency inside the production drain window after a count stop.
|
||||
// Real checkpoint commits need the runtime drain grace after the count budget stops the walk.
|
||||
budget.use_production_drain_grace_for_tests();
|
||||
let (updates, mut receiver) = mpsc::channel(1);
|
||||
let result = tokio::time::timeout(
|
||||
@@ -49,7 +48,7 @@ async fn run_cycle(store: &Arc<ECStore>, cycle: u64, options: CycleOptions) -> (
|
||||
ScannerCycleRequest {
|
||||
owner_ctx: ctx.clone(),
|
||||
ctx: budget.token(),
|
||||
budget,
|
||||
budget: Arc::clone(&budget),
|
||||
updates,
|
||||
want_cycle: cycle,
|
||||
leader_epoch: LEADER_EPOCH,
|
||||
@@ -69,6 +68,13 @@ async fn run_cycle(store: &Arc<ECStore>, cycle: u64, options: CycleOptions) -> (
|
||||
.await
|
||||
.expect("cycle should finish within the test deadline")
|
||||
.expect("cycle should succeed");
|
||||
if options.max_objects.is_some() {
|
||||
assert_eq!(
|
||||
budget.reason(),
|
||||
Some(crate::scanner_budget::ScannerCycleBudgetReason::Objects),
|
||||
"a count-limited fixture must stop for its object budget"
|
||||
);
|
||||
}
|
||||
let snapshot = receiver.recv().await;
|
||||
assert!(receiver.recv().await.is_none(), "a cycle publishes at most one terminal snapshot");
|
||||
(result, snapshot)
|
||||
@@ -125,9 +131,8 @@ async fn partial_bucket_with_a_certified_value_completes_the_cycle_as_pending()
|
||||
assert!(result.clear_verified_usage().is_empty());
|
||||
assert!(dirty_usage_buckets_for_tests().is_empty());
|
||||
|
||||
// Cycle 3: the object budget stops the hot bucket's walk and leaves a
|
||||
// checkpoint under this cycle's plan. Its certified value is carried, so
|
||||
// budget exhaustion leaves the bucket pending without blocking the cycle.
|
||||
// Cycle 3: the budget stops the walk and saves a checkpoint. Usage can
|
||||
// carry the certified value as pending; the outer coordinator records Partial.
|
||||
for object in ["p4/third", "p5/third"] {
|
||||
put_fixture_object(&store, "hot-bucket", object).await;
|
||||
}
|
||||
@@ -144,12 +149,38 @@ async fn partial_bucket_with_a_certified_value_completes_the_cycle_as_pending()
|
||||
},
|
||||
)
|
||||
.await;
|
||||
let third = third.expect("a carried partial cycle publishes");
|
||||
assert_eq!(result.status, ScannerCycleStatus::Complete);
|
||||
assert_eq!(
|
||||
result.status,
|
||||
ScannerCycleStatus::Complete,
|
||||
"certified carried usage remains complete after a budget stop"
|
||||
);
|
||||
let third = third.expect("validated carried usage publishes after the budget stop");
|
||||
assert_eq!(result.pending_dirty_usage_buckets(), 1);
|
||||
assert!(!third.usage_snapshot_observational);
|
||||
assert_eq!(third.usage_snapshot_converged, Some(false));
|
||||
assert_eq!(third.usage_snapshot_pending_buckets, vec!["hot-bucket".to_string()]);
|
||||
assert_eq!(third.buckets_usage["hot-bucket"].objects_count, 4);
|
||||
assert_eq!(third.usage_snapshot_pending_bucket_count, 1);
|
||||
assert!(third.is_complete_bucket_usage_snapshot());
|
||||
let carried_hot = &third.buckets_usage["hot-bucket"];
|
||||
let certified_hot = &second.buckets_usage["hot-bucket"];
|
||||
assert_eq!(carried_hot.usage_value_state, Some(rustfs_data_usage::BucketUsageValueState::Carried));
|
||||
assert!(certified_hot.usage_certification.is_some(), "cycle 2 certified the prior hot value");
|
||||
assert_eq!(
|
||||
carried_hot.usage_certification, certified_hot.usage_certification,
|
||||
"carrying must not mint a new certificate"
|
||||
);
|
||||
assert_eq!(carried_hot.objects_count, 4);
|
||||
assert_eq!(carried_hot.size, certified_hot.size);
|
||||
assert_eq!(third.buckets_usage["cold-bucket"].objects_count, 1);
|
||||
assert_eq!(third.buckets_usage["idle-bucket"].objects_count, 1);
|
||||
assert_eq!(third.objects_total_count, 6);
|
||||
// Only hot was dirty in this scoped scan; it received no fresh certification or ACK.
|
||||
assert!(result.clear_verified_usage().is_empty());
|
||||
assert_eq!(
|
||||
dirty_usage_buckets_for_tests().keys().cloned().collect::<Vec<_>>(),
|
||||
vec!["hot-bucket".to_string()],
|
||||
"the budget-stopped hot bucket retains its dirty responsibility"
|
||||
);
|
||||
let set = Arc::clone(&store.pools[0].disk_set[0]);
|
||||
let mut interrupted = DataUsageCache::default();
|
||||
interrupted
|
||||
|
||||
@@ -109,7 +109,7 @@ pub use bucket::{BucketInfo, BucketOperations, BucketOptions, DeleteBucketOption
|
||||
pub use capability::{CapabilitySnapshotError, CapabilityState, CapabilityStatus};
|
||||
pub use error::{StorageErrorCode, StorageResult};
|
||||
pub use multipart::{CompletePart, ListMultipartsInfo, ListPartsInfo, MultipartInfo, MultipartUploadResult, PartInfo};
|
||||
pub use multipart::{MultipartAbortAuthorization, MultipartCompletionAuthorization, MultipartOwner, MultipartUploadInspection};
|
||||
pub use multipart::{MultipartCompletionAuthorization, MultipartUploadInspection};
|
||||
pub use object::AuthoritativeListOperations;
|
||||
pub use object::DeleteAccounting;
|
||||
pub use object::ObjectLockDeleteOptions;
|
||||
|
||||
@@ -15,24 +15,6 @@
|
||||
use std::collections::HashMap;
|
||||
use time::OffsetDateTime;
|
||||
|
||||
/// Ownership observed in the selected upload metadata.
|
||||
#[derive(Clone, Debug, PartialEq, Eq)]
|
||||
pub struct MultipartOwner {
|
||||
/// The bucket recorded by the upload.
|
||||
pub bucket: String,
|
||||
/// The recorded object, decoded after checking the encoded storage namespace.
|
||||
pub object: String,
|
||||
}
|
||||
|
||||
/// Authorizes Abort against the upload selected under its existing write lock.
|
||||
/// An implementation must reject missing ownership and must not acquire storage locks or mutate storage.
|
||||
pub trait MultipartAbortAuthorization: Send + Sync {
|
||||
type Error: std::error::Error + Send + Sync + 'static;
|
||||
|
||||
/// Returns a refusal before storage prepares or deletes any staged data.
|
||||
fn authorize(&self, owner: Option<&MultipartOwner>) -> Result<(), Self::Error>;
|
||||
}
|
||||
|
||||
/// Request authorization against the source selected under completion locks.
|
||||
/// Implementations must not mutate storage or acquire object/upload locks again.
|
||||
/// Called before comparing client checksums/parts or acknowledging a replay.
|
||||
@@ -91,18 +73,8 @@ pub struct ListMultipartsInfo {
|
||||
pub common_prefixes: Vec<String>,
|
||||
}
|
||||
|
||||
/// The owner found in the upload metadata used to read a ListParts page.
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub struct ListPartsOwner {
|
||||
pub bucket: String,
|
||||
/// The recorded object after decoding the storage namespace spelling.
|
||||
pub object: String,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Default)]
|
||||
pub struct ListPartsInfo {
|
||||
/// Present when the recorded bucket and encoded object agree with the storage namespace.
|
||||
pub recorded_owner: Option<ListPartsOwner>,
|
||||
pub bucket: String,
|
||||
pub object: String,
|
||||
pub upload_id: String,
|
||||
|
||||
@@ -1,63 +0,0 @@
|
||||
// Copyright 2026 RustFS Team
|
||||
// SPDX-License-Identifier: Apache-2.0
|
||||
|
||||
use rustfs_storage_api::multipart::{ListPartsInfo, ListPartsOwner, MultipartInfo, MultipartOwner};
|
||||
use std::any::TypeId;
|
||||
use std::collections::HashMap;
|
||||
|
||||
trait DownstreamOwner {
|
||||
fn kind(&self) -> &'static str;
|
||||
}
|
||||
|
||||
impl DownstreamOwner for ListPartsOwner {
|
||||
fn kind(&self) -> &'static str {
|
||||
"list-parts"
|
||||
}
|
||||
}
|
||||
|
||||
impl DownstreamOwner for MultipartOwner {
|
||||
fn kind(&self) -> &'static str {
|
||||
"multipart"
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn multipart_owner_types_remain_distinct_for_downstream_implementations() {
|
||||
let list_owner = ListPartsOwner {
|
||||
bucket: "bucket".to_owned(),
|
||||
object: "key".to_owned(),
|
||||
};
|
||||
let multipart_owner = MultipartOwner {
|
||||
bucket: "bucket".to_owned(),
|
||||
object: "key".to_owned(),
|
||||
};
|
||||
assert_eq!(list_owner.kind(), "list-parts");
|
||||
assert_eq!(multipart_owner.kind(), "multipart");
|
||||
assert_ne!(TypeId::of::<ListPartsOwner>(), TypeId::of::<MultipartOwner>());
|
||||
let page = ListPartsInfo {
|
||||
recorded_owner: Some(list_owner.clone()),
|
||||
..Default::default()
|
||||
};
|
||||
assert_eq!(page.recorded_owner, Some(list_owner));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn multipart_info_retains_exhaustive_literal_and_pattern() {
|
||||
let info = MultipartInfo {
|
||||
bucket: "bucket".to_owned(),
|
||||
object: "key".to_owned(),
|
||||
upload_id: "upload".to_owned(),
|
||||
initiated: None,
|
||||
user_defined: HashMap::new(),
|
||||
};
|
||||
let MultipartInfo {
|
||||
bucket,
|
||||
object,
|
||||
upload_id,
|
||||
initiated,
|
||||
user_defined,
|
||||
} = info;
|
||||
assert_eq!((bucket.as_str(), object.as_str(), upload_id.as_str()), ("bucket", "key", "upload"));
|
||||
assert!(initiated.is_none());
|
||||
assert!(user_defined.is_empty());
|
||||
}
|
||||
@@ -24,7 +24,7 @@ use crate::{
|
||||
store::{Key, Store},
|
||||
target::{
|
||||
ChannelTargetType, EntityTarget, QueuedPayload, QueuedPayloadMeta, TargetDeliveryCounters, TargetDeliverySnapshot,
|
||||
TargetTlsState, TargetType, build_event_key, build_queued_payload, build_target_tls_fingerprint, is_connectivity_error,
|
||||
TargetTlsState, TargetType, build_queued_payload, build_target_tls_fingerprint, is_connectivity_error,
|
||||
open_target_queue_store, persist_queued_payload_to_store,
|
||||
},
|
||||
};
|
||||
@@ -291,16 +291,6 @@ impl KafkaArgs {
|
||||
}
|
||||
}
|
||||
|
||||
fn build_kafka_record<'a>(
|
||||
topic: &'a str,
|
||||
body: &'a [u8],
|
||||
meta: &QueuedPayloadMeta,
|
||||
) -> Result<Record<'a, String, &'a [u8]>, TargetError> {
|
||||
// Queue metadata carries the form-encoded S3 key, including on replay.
|
||||
let key = build_event_key(&meta.bucket_name, &meta.object_name)?;
|
||||
Ok(Record::from_key_value(topic, key, body))
|
||||
}
|
||||
|
||||
/// A target that sends events to an Apache Kafka topic
|
||||
pub struct KafkaTarget<E>
|
||||
where
|
||||
@@ -469,8 +459,6 @@ where
|
||||
"Sending Kafka payload"
|
||||
);
|
||||
|
||||
let record = build_kafka_record(&self.args.topic, &body, meta)?;
|
||||
|
||||
// rustfs-kafka-async does not validate response correlation IDs. Keep
|
||||
// producer selection, send, and timeout invalidation serialized so a
|
||||
// waiter cannot reuse a connection with an unread timed-out response.
|
||||
@@ -479,8 +467,12 @@ where
|
||||
&self.delivery_poisoned,
|
||||
|| self.get_or_build_producer(),
|
||||
|producer| async move {
|
||||
// Use "<bucket>/<object>" as the message key so all events for the same
|
||||
// object hash to the same partition and preserve per-object ordering
|
||||
// across multiple partitions (backlog#983).
|
||||
let partition_key = format!("{}/{}", meta.bucket_name, meta.object_name);
|
||||
producer
|
||||
.send(&record)
|
||||
.send(&Record::from_key_value(&self.args.topic, partition_key, body.as_slice()))
|
||||
.await
|
||||
.map_err(|err| Self::map_kafka_error(err, "Failed to send message to Kafka"))
|
||||
},
|
||||
@@ -669,68 +661,6 @@ mod tests {
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn kafka_record_key_matches_envelope_for_live_and_queued_events() {
|
||||
let target = KafkaTarget::<serde_json::Value>::new("test".to_string(), base_args())
|
||||
.expect("Kafka target configuration should be valid");
|
||||
let cases = [
|
||||
("file.txt", "file.txt"),
|
||||
("test-ns%2Fsflm%2FusecaseB%2Ffile.txt", "test-ns/sflm/usecaseB/file.txt"),
|
||||
("red+flower.jpg", "red flower.jpg"),
|
||||
("red%20flower.jpg", "red flower.jpg"),
|
||||
("a%2Bb%2Fc.txt", "a+b/c.txt"),
|
||||
("literal%252F%2B%252B%2B%25.txt", "literal%2F+%2B+%.txt"),
|
||||
("%E7%9B%AE%E5%BD%95%2F%E6%96%87%E4%BB%B6.txt", "目录/文件.txt"),
|
||||
("%2Fleading%2F", "/leading/"),
|
||||
("", ""),
|
||||
];
|
||||
for (encoded, decoded) in cases {
|
||||
let event = EntityTarget {
|
||||
object_name: encoded.to_string(),
|
||||
bucket_name: "test-bucket".to_string(),
|
||||
event_name: crate::EventName::ObjectCreatedPut,
|
||||
data: serde_json::json!({"s3": {"object": {"key": encoded}}}),
|
||||
};
|
||||
let live = target.build_queued_payload(&event).expect("event should serialize");
|
||||
let stored = live.encode().expect("queued payload should encode");
|
||||
let replay = QueuedPayload::decode(&stored).expect("existing queue format should decode");
|
||||
for queued in [&live, &replay] {
|
||||
let record =
|
||||
build_kafka_record(&target.args.topic, &queued.body, &queued.meta).expect("Kafka record should build");
|
||||
let envelope: serde_json::Value = serde_json::from_slice(record.value).expect("record body should be JSON");
|
||||
assert_eq!(record.key, format!("test-bucket/{decoded}"), "encoded object name: {encoded}");
|
||||
assert_eq!(envelope["Key"], record.key);
|
||||
assert_eq!(envelope["Records"][0]["s3"]["object"]["key"], encoded);
|
||||
assert_eq!(queued.meta.object_name, encoded);
|
||||
assert_eq!(record.topic, "rustfs-events");
|
||||
assert_eq!(record.value.as_ptr(), queued.body.as_ptr(), "record should borrow the body");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn kafka_record_rejects_invalid_utf8_before_delivery() {
|
||||
let target = KafkaTarget::<serde_json::Value>::new("test".to_string(), base_args())
|
||||
.expect("Kafka target configuration should be valid");
|
||||
let meta = QueuedPayloadMeta::new(
|
||||
crate::EventName::ObjectCreatedPut,
|
||||
"test-bucket".to_string(),
|
||||
"%FF".to_string(),
|
||||
"application/json",
|
||||
2,
|
||||
);
|
||||
let queued = QueuedPayload::new(meta, b"{}".to_vec());
|
||||
let stored = queued.encode().expect("corrupt event metadata should serialize");
|
||||
let replay = QueuedPayload::decode(&stored).expect("queue framing should decode");
|
||||
let error = target
|
||||
.send_body(replay.body, &replay.meta)
|
||||
.await
|
||||
.expect_err("invalid UTF-8 should reject delivery");
|
||||
assert!(matches!(error, TargetError::Encoding(_)), "unexpected error: {error}");
|
||||
assert!(target.producer.lock().await.is_none());
|
||||
assert!(!target.delivery_poisoned.load(Ordering::Acquire));
|
||||
}
|
||||
|
||||
#[tokio::test(start_paused = true)]
|
||||
async fn timeout_invalidates_before_the_next_delivery_selects_a_producer() {
|
||||
let delivery_lock = Arc::new(Mutex::new(()));
|
||||
|
||||
@@ -19,7 +19,6 @@ use crate::{StoreError, TargetError, TargetLog};
|
||||
use async_trait::async_trait;
|
||||
use rustfs_s3_types::EventName;
|
||||
use serde::{Deserialize, Serialize};
|
||||
use std::borrow::Cow;
|
||||
use std::cell::Cell;
|
||||
use std::fmt::Formatter;
|
||||
use std::future::Future;
|
||||
@@ -674,8 +673,8 @@ pub(crate) fn queue_store_subdir_name(target_type: &str, target_id: &str) -> Str
|
||||
/// Decodes a form-urlencoded object name to its original form.
|
||||
///
|
||||
/// This function properly handles form-urlencoded strings where spaces are
|
||||
/// represented as `+` symbols. Percent-encoded plus signs (`%2B`) remain
|
||||
/// literal plus signs, and percent escapes are decoded once.
|
||||
/// represented as `+` symbols. It first replaces `+` with spaces, then
|
||||
/// performs standard percent-decoding.
|
||||
///
|
||||
/// # Arguments
|
||||
/// * `encoded` - The form-urlencoded string to decode
|
||||
@@ -692,30 +691,10 @@ pub(crate) fn queue_store_subdir_name(target_type: &str, target_id: &str) -> Str
|
||||
/// assert_eq!(decoded, "greeting file (2).csv");
|
||||
/// ```
|
||||
pub fn decode_object_name(encoded: &str) -> Result<String, TargetError> {
|
||||
decode_object_name_cow(encoded).map(Cow::into_owned)
|
||||
}
|
||||
|
||||
fn decode_object_name_cow(encoded: &str) -> Result<Cow<'_, str>, TargetError> {
|
||||
let form = if encoded.contains('+') {
|
||||
Cow::Owned(encoded.replace('+', " "))
|
||||
} else {
|
||||
Cow::Borrowed(encoded)
|
||||
};
|
||||
let decoded =
|
||||
urlencoding::decode(&form).map_err(|err| TargetError::Encoding(format!("Failed to decode object key: {err}")))?;
|
||||
Ok(match decoded {
|
||||
Cow::Borrowed(_) => form,
|
||||
Cow::Owned(name) => Cow::Owned(name),
|
||||
})
|
||||
}
|
||||
|
||||
pub(crate) fn build_event_key(bucket: &str, encoded_object: &str) -> Result<String, TargetError> {
|
||||
let object = decode_object_name_cow(encoded_object)?;
|
||||
let mut key = String::with_capacity(bucket.len() + 1 + object.len());
|
||||
key.push_str(bucket);
|
||||
key.push('/');
|
||||
key.push_str(&object);
|
||||
Ok(key)
|
||||
let replaced = encoded.replace("+", " ");
|
||||
urlencoding::decode(&replaced)
|
||||
.map(|s| s.into_owned())
|
||||
.map_err(|e| TargetError::Encoding(format!("Failed to decode object key: {e}")))
|
||||
}
|
||||
|
||||
pub(crate) fn build_queued_payload<E>(event: &EntityTarget<E>) -> Result<QueuedPayload, TargetError>
|
||||
@@ -733,7 +712,8 @@ where
|
||||
E: PluginEvent,
|
||||
R: Serialize,
|
||||
{
|
||||
let key = build_event_key(&event.bucket_name, &event.object_name)?;
|
||||
let object_name = decode_object_name(&event.object_name)?;
|
||||
let key = format!("{}/{}", event.bucket_name, object_name);
|
||||
|
||||
let log = TargetLog {
|
||||
event_name: event.event_name,
|
||||
@@ -1234,80 +1214,6 @@ mod tests {
|
||||
assert_eq!(decoded.body, br#"{"ok":true}"#);
|
||||
}
|
||||
|
||||
fn assert_object_name_matches_legacy_decoder(encoded: &str) {
|
||||
let form = encoded.replace('+', " ");
|
||||
match urlencoding::decode(&form) {
|
||||
Ok(expected) => {
|
||||
assert_eq!(decode_object_name(encoded).expect("object should decode"), expected, "input: {encoded:?}");
|
||||
let key = build_event_key("bucket%2F+", encoded).expect("event key should build");
|
||||
assert_eq!(key, format!("bucket%2F+/{expected}"), "input: {encoded:?}");
|
||||
}
|
||||
Err(_) => {
|
||||
assert!(matches!(decode_object_name(encoded), Err(TargetError::Encoding(_))), "input: {encoded:?}");
|
||||
assert!(
|
||||
matches!(build_event_key("bucket", encoded), Err(TargetError::Encoding(_))),
|
||||
"input: {encoded:?}"
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn object_name_decoder_preserves_legacy_escape_and_utf8_semantics() {
|
||||
for first in 0u8..=127 {
|
||||
for second in 0u8..=127 {
|
||||
let encoded = format!("prefix+%{}{}%2B%252F/目录", char::from(first), char::from(second));
|
||||
assert_object_name_matches_legacy_decoder(&encoded);
|
||||
}
|
||||
}
|
||||
for byte in 0u8..=255 {
|
||||
assert_object_name_matches_legacy_decoder(&format!("prefix/%{byte:02X}+suffix"));
|
||||
}
|
||||
for encoded in [
|
||||
"",
|
||||
"plain.txt",
|
||||
"目录/文件.txt",
|
||||
"a&b=c?#.txt",
|
||||
"%",
|
||||
"%2",
|
||||
"%GG",
|
||||
"%2B",
|
||||
"%252B",
|
||||
"%252F",
|
||||
"%+1",
|
||||
"%1+",
|
||||
"%%32",
|
||||
"%2%32",
|
||||
"+++",
|
||||
"%00",
|
||||
"%C3%A9",
|
||||
"%E7%9B%AE%E5%BD%95",
|
||||
"%F0%9F%98%80",
|
||||
"%C0%AF",
|
||||
"%ED%A0%80",
|
||||
"%F4%90%80%80",
|
||||
"%C3",
|
||||
"%C3+%A9",
|
||||
] {
|
||||
assert_object_name_matches_legacy_decoder(encoded);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn object_name_decoder_borrows_plain_names() {
|
||||
let plain = "directory/file.txt";
|
||||
let decoded = decode_object_name_cow(plain).expect("plain name should decode");
|
||||
assert!(matches!(decoded, Cow::Borrowed(_)));
|
||||
assert_eq!(decoded.as_ptr(), plain.as_ptr());
|
||||
for (encoded, expected) in [("red+flower.jpg", "red flower.jpg"), ("a%2Bb", "a+b"), ("a%2Bb+c", "a+b c")] {
|
||||
let decoded = decode_object_name_cow(encoded).expect("encoded name should decode");
|
||||
assert!(matches!(decoded, Cow::Owned(_)));
|
||||
assert_eq!(decoded, expected);
|
||||
}
|
||||
let key = build_event_key("test-bucket", "directory%2Ffile.txt").expect("event key should build");
|
||||
assert_eq!(key, "test-bucket/directory/file.txt");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn build_queued_payload_uses_event_data_shape() {
|
||||
let event = EntityTarget {
|
||||
|
||||
@@ -50,7 +50,7 @@ Two codec backends exist, selected per object from its metadata (never a runtime
|
||||
- **Modern backend** — Reed–Solomon over **GF(2⁸)** using `rustfs-erasure-codec` (a RustFS fork of `reed-solomon-erasure` v8, `Cargo.toml`), imported as `reed_solomon_erasure::galois_8::ReedSolomon` ([erasure.rs](../../crates/ecstore/src/erasure/coding/erasure.rs)). Vandermonde generator matrix; algorithm string `"rs-vandermonde"` ([object_api/mod.rs](../../crates/ecstore/src/object_api/mod.rs)). The GF(2⁸) field bounds total shards per set far above the geometry cap of 16 (§2). Used for **all new writes** — and, because MinIO uses the same `rs-vandermonde` GF(2⁸) scheme, for **all MinIO-migrated objects** too.
|
||||
- **Legacy backend** — Reed–Solomon over **GF(2¹⁶)** using `reed-solomon-simd` v3.1 (`Cargo.toml`), imported at [erasure.rs](../../crates/ecstore/src/erasure/coding/erasure.rs). Used **only** to read and heal objects written in **RustFS's own older ("main branch") format** — the `rmp_serde`-serialized layout detected by `uses_legacy_checksum` (see §11). This backend is **not** MinIO-compatible; MinIO-migrated data is decoded by the modern GF(2⁸) backend above (see [minio-file-format-compat.md](minio-file-format-compat.md)). The backend is chosen per object from its metadata (`uses_legacy_checksum`), never by a runtime toggle.
|
||||
|
||||
Industry alignment (all confirmed in code): byte-oriented RS over GF(2⁸) with a Vandermonde matrix and HighwayHash-256 bitrot checksums with a π-derived key share the format family used by MinIO. RustFS defaults to a 1 MiB erasure block. Reads use each object’s persisted block geometry. The supported byte-level `xl.meta` interoperability is defined by [minio-file-format-compat.md](minio-file-format-compat.md).
|
||||
Industry alignment (all confirmed in code): byte-oriented RS over GF(2⁸) with a Vandermonde matrix, 1 MiB erasure block, and HighwayHash-256 bitrot checksums with a π-derived key — the same family and defaults MinIO uses. This is what makes byte-level `xl.meta` interoperability with MinIO possible (see [minio-file-format-compat.md](minio-file-format-compat.md)).
|
||||
|
||||
Where the code lives: the erasure engine is owned by `crates/ecstore/src/erasure/` and is crate-private; the `xl.meta` model is owned by `crates/filemeta`. See [ecstore-layout-boundary.md](ecstore-layout-boundary.md).
|
||||
|
||||
@@ -140,7 +140,7 @@ distribution[i-1] = 1 + ((start + i) % N) for i in 1..=N // a cyclic rotatio
|
||||
|
||||
### 4.3 Streaming pipeline and fan-out
|
||||
|
||||
`Erasure::encode` / `encode_batched` ([encode.rs](../../crates/ecstore/src/erasure/coding/encode.rs)) read the source in `block_size` chunks on a producer task, encode each block, and stream the `N` shards of each block over a bounded channel to a consumer that fans them out through `MultiWriter` to the `N` shard writers. `block_size == 0` is rejected up front (`InvalidInput`). Channel capacity is derived from the queue budget and the EC-expanded block size, with at least one slot. The default queue budget is 32 MiB. Producer, consumer, ingest, writer, and allocator buffers contribute additional memory, and a single expanded block can exceed that budget. Batched mode also owns pending batches. Queue accounting describes queued blocks. Per-PUT and process RSS also depend on those additional owners. A clean EOF stops the loop; the final partial block is padded per §4.2.
|
||||
`Erasure::encode` / `encode_batched` ([encode.rs](../../crates/ecstore/src/erasure/coding/encode.rs)) read the source in `block_size` chunks on a producer task, encode each block, and stream the `N` shards of each block over a bounded channel to a consumer that fans them out through `MultiWriter` to the `N` shard writers. `block_size == 0` is rejected up front (`InvalidInput`). In-flight memory is bounded by the channel depth (default budget 32 MiB). A clean EOF stops the loop; the final partial block is padded per §4.2.
|
||||
|
||||
- **INVARIANT — write-quorum enforcement during encode.** `MultiWriter` writes shard `i` to writer `i`, drops any stalled/failed/short writer (sets it to `None`), and after each block requires `nil_count ≥ write_quorum`, else fails with a reduced-write-quorum error ([encode.rs](../../crates/ecstore/src/erasure/coding/encode.rs)). Shard index → writer index → on-disk position is fixed.
|
||||
|
||||
@@ -292,12 +292,10 @@ Fields: `version_id`, `mod_time`, `signature: [u8;4]`, `version_type`, `flags: u
|
||||
|
||||
### 6.6 Inline data
|
||||
|
||||
Small objects store their payload inline after the container CRC ([filemeta_inline.rs](../../crates/filemeta/src/filemeta_inline.rs)): 1 version byte (`INLINE_DATA_VER = 1`) then a msgpack map of `version-key → bin`. **INVARIANT — the map key** is the version-id string, `"null"` (`NULL_VERSION_ID`, [fileinfo.rs](../../crates/filemeta/src/fileinfo.rs)) for the null/None version, else the lowercase hyphenated UUID. Presence is determined **on read** solely by the `meta_sys[inline-data]` body marker (`FileInfo::inline_data`); the read path gates inline extraction on that marker alone. The header `InlineData` flag is **written** (mirrored from the body on marshal) but is **not** consulted on read, and a disagreement is tolerated — MinIO may leave the header flag unset while inline data is present, so a reader must **not** require the flag and the marker to agree. `should_inline` ([storageclass.rs](../../crates/ecstore/src/config/storageclass.rs)) admits a shard up to `inline_block/8` for versioned buckets and up to `inline_block` otherwise. The default is a fixed `DEFAULT_INLINE_BLOCK = 128 KiB` per shard for every EC geometry, matching MinIO. Explicit `RUSTFS_STORAGE_CLASS_INLINE_BLOCK` values use the same per-shard interpretation. Known-length unprotected PUTs may span multiple erasure blocks within a 2 MiB object limit and 32-frame limit, subject to the accumulated per-shard budget. The default block remains 1 MiB. The direct encoder remains single-block. Transformed streams and proof-protected writes retain their single-block admission bound. Known-length inline writers include every bitrot frame header in a checked capacity limit and reject overflow before appending data. The persisted marker governs existing-object reads independently of the current write budget. Config serialization retains the historical `inline_block_explicit` marker for readers that use it. A compressed or encrypted single PUT has no known stored size up front, so its shard size is derived from the plaintext `actual_size` (`should_inline_put`, MinIO `putObject` parity); such objects are inlined through the streaming encoder with in-memory bitrot writers rather than the single-block fast path. When shard-integrity protected writes are enabled, inline placement additionally requires a known stored length no greater than one erasure block; unknown transformed streams retain external shards and proof indexes.
|
||||
Small objects store their payload inline after the container CRC ([filemeta_inline.rs](../../crates/filemeta/src/filemeta_inline.rs)): 1 version byte (`INLINE_DATA_VER = 1`) then a msgpack map of `version-key → bin`. **INVARIANT — the map key** is the version-id string, `"null"` (`NULL_VERSION_ID`, [fileinfo.rs](../../crates/filemeta/src/fileinfo.rs)) for the null/None version, else the lowercase hyphenated UUID. Presence is determined **on read** solely by the `meta_sys[inline-data]` body marker (`FileInfo::inline_data`); the read path gates inline extraction on that marker alone. The header `InlineData` flag is **written** (mirrored from the body on marshal) but is **not** consulted on read, and a disagreement is tolerated — MinIO may leave the header flag unset while inline data is present, so a reader must **not** require the flag and the marker to agree. The inline threshold is `should_inline` ([storageclass.rs](../../crates/ecstore/src/config/storageclass.rs)): inline if `shard_size ≤ inline_block/8` for versioned buckets, else `≤ inline_block`; `DEFAULT_INLINE_BLOCK = 128 KiB`. A compressed or encrypted single PUT has no known stored size up front, so its shard size is derived from the plaintext `actual_size` (`inline_admission_shard_size`, MinIO `putObject` parity); such objects are inlined through the streaming encoder with in-memory bitrot writers rather than the single-block fast path. When shard-integrity protected writes are enabled, inline placement additionally requires a known stored length no greater than one erasure block; unknown transformed streams retain external shards and proof indexes.
|
||||
|
||||
---
|
||||
|
||||
Inline body materialization uses the shared `inline_body_buffering_miss_reason` contract ([set_disk/mod.rs](../../crates/ecstore/src/set_disk/mod.rs)) with a 2 MiB response limit and a bounded frame count. Metadata pre-verification and buffered GET consumption use this same contract, and an already verified body is reused within the selected snapshot. Parts carrying `PartIntegrity` use the readers that bind their proof verifier. Metadata early-stop and GET share `try_decode_inline_data_shards` for interleaved bitrot verification, per-block EC padding removal and body assembly. Data-shard metadata uses inline-capacity collections for normal EC sets. A healthy full metadata fanout decodes directly from cached bytes. Missing or damaged data shards enter the existing quorum-protected reconstruction path. Buffered GET, deep inline healing and ordinary bitrot readers share the borrowed `verified_bitrot_frame` primitive. Deep healing checks exact physical length before hashing the original frame slices, including every data and parity frame, and rejects trailing bytes, truncated prefixes and checksum mismatches.
|
||||
|
||||
## 7. Write path and write quorum
|
||||
|
||||
`put_object` and multipart resolve the write layout (§2.2), encode (§4), write shards with bitrot (§5), then commit atomically.
|
||||
@@ -381,7 +379,7 @@ Geometry & math
|
||||
|
||||
Algorithm
|
||||
- Modern RS over GF(2⁸) (`rs-vandermonde`) for new writes; legacy GF(2¹⁶) for old files, selected by `uses_legacy_checksum`.
|
||||
- The default `block_size` is 1 MiB (`BLOCK_SIZE_V2`). `RUSTFS_ERASURE_SHARD_BYTES` derives new user-object geometry as `data_blocks × shard_bytes` through `erasure_block_size_for` ([fileinfo.rs](../../crates/filemeta/src/fileinfo.rs)). System metadata keeps the default. Each version stores its own geometry for subsequent reads.
|
||||
- `block_size = 1 MiB` (`BLOCK_SIZE_V2`), stored per version.
|
||||
- Shard-size formulas: modern `div_ceil`; legacy `(div_ceil + 1) & !1`. Final block zero-padded before encode.
|
||||
- Bitrot `HighwayHash256S` (legacy key variant for old files); interleaved `[hash][data]` per block; `bitrot_shard_file_size = ceil(size/shard_size)*32 + size`; verify before use.
|
||||
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
# Cluster and erasure-coding lifecycle operations
|
||||
|
||||
**Use this when:** planning, starting, expanding, rebalancing, decommissioning, healing, replacing drives in, or restarting an erasure-coded RustFS deployment, or deciding whether an `EC:<parity>` setting (including `EC:0`) is safe for a workload.
|
||||
**Source of truth:** `crates/ecstore/src/layout/disks_layout.rs` (`DisksLayout::from_volumes`, `SET_SIZES`, `RUSTFS_ERASURE_SET_DRIVE_COUNT`), `crates/ecstore/src/config/storageclass.rs` (`default_parity_count`, `validate_parity_inner`, `Config::should_inline`, `Config::inline_block`), `crates/ecstore/src/set_disk/mod.rs` (`resolve_write_layout`, `WriteLayout::from_parity`), `crates/ecstore/src/set_disk/metadata.rs` (`object_quorum_from_meta`, `common_parity`), `crates/ecstore/src/set_disk/ops/heal.rs` (no-parity `cannot_heal` classification), `crates/ecstore/src/store/init_format.rs` (`check_format_erasure_value_for_topology`), `rustfs/src/admin/handlers/pools.rs`, `rustfs/src/admin/handlers/rebalance.rs`, `rustfs/src/admin/handlers/heal.rs`, and `rustfs/src/admin/route_policy.rs` (admin surface).
|
||||
**Source of truth:** `crates/ecstore/src/layout/disks_layout.rs` (`DisksLayout::from_volumes`, `SET_SIZES`, `RUSTFS_ERASURE_SET_DRIVE_COUNT`), `crates/ecstore/src/config/storageclass.rs` (`default_parity_count`, `validate_parity_inner`, `Config::should_inline`, `Config::effective_inline_block`), `crates/ecstore/src/set_disk/mod.rs` (`resolve_write_layout`, `WriteLayout::from_parity`), `crates/ecstore/src/set_disk/metadata.rs` (`object_quorum_from_meta`, `common_parity`), `crates/ecstore/src/set_disk/ops/heal.rs` (no-parity `cannot_heal` classification), `crates/ecstore/src/store/init_format.rs` (`check_format_erasure_value_for_topology`), `rustfs/src/admin/handlers/pools.rs`, `rustfs/src/admin/handlers/rebalance.rs`, `rustfs/src/admin/handlers/heal.rs`, and `rustfs/src/admin/route_policy.rs` (admin surface).
|
||||
|
||||
This runbook is the operator entry point. It restates only what an operator needs to act and links the normative contracts for everything else: [erasure-coding.md](../architecture/erasure-coding.md) (algorithm, quorum, on-disk format), [ecstore-layout-boundary.md](../architecture/ecstore-layout-boundary.md) (immutable layout), [decommission-compatibility.md](../architecture/decommission-compatibility.md) (decommission and rebalance contract), [heal-concurrency-model.md](../architecture/heal-concurrency-model.md) (heal versus write locking), [rolling-restart.md](rolling-restart.md) (restarts), [replacement-generation-recovery.md](replacement-generation-recovery.md) (drive replacement recovery), and [scanner-runtime-controls.md](scanner-runtime-controls.md) (heal and scanner knobs). Admin routes below require an authenticated request holding the matching `AdminAction`; the route to action mapping is owned by `rustfs/src/admin/route_policy.rs` (see [admin-route-action-snapshot.md](../architecture/admin-route-action-snapshot.md)).
|
||||
|
||||
@@ -60,7 +60,7 @@ A single local path (`rustfs D:\rustfs\data`) is a single-node single-drive depl
|
||||
| `RUSTFS_STORAGE_CLASS_STANDARD` | STANDARD parity as `EC:<n>` | Validated per pool; affects new writes only |
|
||||
| `RUSTFS_STORAGE_CLASS_RRS` | RRS parity as `EC:<n>` | Validated per pool; an explicit value fails on a single drive, the persisted default `EC:1` resolves to `0` there |
|
||||
| `RUSTFS_STORAGE_CLASS_OPTIMIZE` | Accepted for MinIO compatibility | Stored on `Config`, but `Config::capacity_optimized` has no caller: it changes nothing at runtime today |
|
||||
| `RUSTFS_STORAGE_CLASS_INLINE_BLOCK` | Fixed per-shard inline limit (bytesize syntax such as `128KiB`) | Overrides the fixed 128 KiB default in §4.3. Values above 128 KiB log `storage_class_inline_block_large` |
|
||||
| `RUSTFS_STORAGE_CLASS_INLINE_BLOCK` | Fixed per-shard inline limit (bytesize syntax such as `128KiB`) | Replaces the scaled default in §4.3; values above 128 KiB log `storage_class_inline_block_large` |
|
||||
|
||||
An empty `RUSTFS_STORAGE_CLASS_STANDARD` or `RUSTFS_STORAGE_CLASS_RRS` restores the automatic policy and overrides a persisted value (`standard_policy` / `rrs_policy`). `MINIO_`-prefixed spellings of these keys, of `ERASURE_SET_DRIVE_COUNT`, and of `VOLUMES` are mapped onto the `RUSTFS_` names when the `RUSTFS_` name is unset (`apply_external_env_compat` in `crates/utils/src/envs.rs`); when both are set the `RUSTFS_` value wins and the conflict is reported at startup.
|
||||
|
||||
@@ -103,9 +103,7 @@ Sixteen drives at `EC:4`: `K = 12`, about 75 percent usable, four failures toler
|
||||
|
||||
### 4.3 Inline objects
|
||||
|
||||
An object is stored inline in `xl.meta` when every shard is at most the per-shard budget (`Config::should_inline`). The default is `DEFAULT_INLINE_BLOCK` (128 KiB per shard), matching MinIO's default. A versioned bucket uses one eighth of that budget. `RUSTFS_STORAGE_CLASS_INLINE_BLOCK` sets an explicit fixed per-shard limit. Each drive keeps its own bitrot-protected shard inline. PUT admission also requires the stored size, or the plaintext size for an unprotected transformed stream with unknown stored size, to fit one erasure block. Protected transformed streams require a known stored size. The complete format and transform rules are in [erasure-coding.md §6.6](../architecture/erasure-coding.md#66-inline-data).
|
||||
|
||||
Existing inline and external objects are read according to their persisted metadata after a budget change. The former scaled default admitted approximately 256 KiB per unversioned object across wider EC layouts. The fixed per-shard budget admits larger new objects as data-shard count grows, subject to the single-block bound. Operators can pin an explicit smaller per-shard limit for a selected geometry. Changing this setting affects future placement; rollback to a historical binary requires that binary's reader to accept the newly written layouts. Use [MinIO format compatibility](../architecture/minio-file-format-compat.md) for the supported migration direction and encrypted-object constraints.
|
||||
An object is stored inline in `xl.meta` when every shard is at most the per-shard budget (`Config::should_inline`). The default budget is `DEFAULT_INLINE_OBJECT_BUDGET / K` capped at `DEFAULT_INLINE_BLOCK` (256 KiB / K, at most 128 KiB per shard), so a wider set does not raise the maximum inline object size; on a versioned bucket the budget is divided by 8. `RUSTFS_STORAGE_CLASS_INLINE_BLOCK` replaces the scaled default with a fixed per-shard limit. Each drive keeps only its own shard inline, so inline data has the same loss semantics as external shards. Compressed or encrypted streams inline only when the stored size is known and within budget.
|
||||
|
||||
### 4.4 Write and read paths
|
||||
|
||||
|
||||
@@ -1,15 +0,0 @@
|
||||
# Distributed GetObject metadata cache proof experiment
|
||||
|
||||
`RUSTFS_EXPERIMENTAL_GET_OBJECT_METADATA_CACHE_DISTRIBUTED_PROOF_ENABLE` opts a test or `e2e-test-hooks` build into proof-validated GetObject metadata caching for distributed erasure sets. It defaults to `false` and requires a restart to change. Ordinary production builds ignore the setting and keep the distributed cache hard-disabled until the mutation-authority audit described below is complete.
|
||||
|
||||
The experiment does not add a cache-control RPC or change write admission. Capable namespace-lock voters advance a bounded mutation-generation shard on every successful exclusive grant. An eligible GET requests stamps through its existing shared-lock quorum and may use a cached metadata entry only when the current proof exactly matches the proof recorded at cache publication.
|
||||
|
||||
The read falls back to authoritative metadata quorum without failing the S3 request when the setting is disabled or any required proof property is unavailable, including a legacy voter, malformed or unknown stamp, duplicate voter identity, lock loss, voter restart, generation exhaustion, changed quorum cohort, or proof mismatch.
|
||||
|
||||
Only request shapes already eligible for the process-local metadata cache participate: latest unversioned object reads whose ECStore entrypoint holds the fixed object namespace read guard, excluding direct SetDisks calls, system buckets, explicit versions, versioned or suspended buckets, delete-marker reads, part-number or part-checksum reads, and data-movement paths. Local erasure deployments retain the existing process-local generation behavior.
|
||||
|
||||
The proof state is in memory and has no on-disk migration. Hash-shard collisions cause conservative cache misses only. Mixed-version operation preserves lock and storage availability because old peers can grant locks without a stamp; a cache hit requires a complete capable read quorum.
|
||||
|
||||
Keep the experiment disabled for production rollout until every cache-visible metadata mutation path has been verified to hold the same exclusive object namespace authority and the multi-node fault and performance acceptance matrix has passed. Disabling the setting and restarting clears all process-local proof-bound cache state.
|
||||
|
||||
The current production-activation blockers include delete-recovery reconciliation, exact-marker cleanup, and selected restore/heal metadata paths that can operate under a set-local, lifecycle, or process-local fence instead of the ECStore fixed object lock domain. These paths must either acquire that fixed exclusive authority before publication or gain an equivalent quorum-stamp transition before the production gate is removed. Legacy transition-state reconciliation is not a blocker because it already acquires the fixed store domain followed by every distinct physical set domain before disk metadata publication.
|
||||
@@ -92,32 +92,6 @@ power simultaneously, `relaxed` can lose recently acknowledged objects
|
||||
cluster-wide. Single-node
|
||||
deployments must stay on `strict`.
|
||||
|
||||
## Small-object I/O profile
|
||||
|
||||
Small-object cost scales with object count, key depth, and the selected durability mode. Each inline object retains its own object directory and `xl.meta`, which contains both metadata and inline payload. Non-inline objects also have a data directory and shard files. Compression can reduce stored bytes while retaining this per-object namespace. The default single-drive inline admission budget is 128 KiB per shard. Versioned writes use a smaller budget. Explicit storage-class configuration, stream transformations, the single-erasure-block bound, and the erasure layout affect admission. See `StorageClassConfig::should_inline` in `crates/ecstore/src/config/storageclass.rs` and its PUT caller in `crates/ecstore/src/set_disk/ops/object.rs`.
|
||||
|
||||
The following application-level synchronization budget covers a successful ordinary Linux inline commit to a fresh key in an existing bucket. It assumes directory group commit is disabled and excludes multipart, healing, system metadata, retries, and background work. Let `d` be the number of prefix components before the final key component, so `object` has `d = 0` and `a/b/object` has `d = 2`.
|
||||
|
||||
| Write shape | Metadata file synchronization | Directory synchronization |
|
||||
| --- | --- | --- |
|
||||
| Fresh inline key, `strict` | One metadata `sync_data`, including inline payload | One object-directory sync plus one per ancestor through the bucket, totaling `d + 2` |
|
||||
| Fresh inline key, `file-sync` | One metadata `sync_data`, including inline payload | Zero ordinary commit-directory syncs |
|
||||
| Fresh inline key, `relaxed` or `none` | Zero | Zero ordinary commit-directory syncs |
|
||||
|
||||
Linux stages ordinary inline metadata in a flat temporary file and publishes it by rename. The temporary name is removed by publication, so synchronizing that temporary directory entry would add work without protecting the acknowledged object. Strict mode synchronizes the final object directory and its ancestors. Even an existing shared prefix can have been created by another writer whose directory synchronization is still pending. Keeping the ancestor barriers protects that concurrent creation case. `rename_data_fresh_inline_sync_budget` tests exercise the real disk commit and pin the directory-call budget for flat and nested keys.
|
||||
|
||||
Overwrites and versioned writes can retain predecessor metadata for rollback and add backup, recovery, and cleanup work. Non-inline writes also synchronize shard contents under `strict`, `file-sync`, and `relaxed`. Their budgets depend on object shape and recovery state. A normal inline GET reads `xl.meta`. Non-inline GET also reads the required shard data and verifies integrity. Delete work depends on versioning, delete markers, retained versions, recovery state, and deferred trash cleanup. LIST, scanner, and heal perform additional directory and metadata reads, and repairs can perform writes. Measure each workload separately.
|
||||
|
||||
All metadata commit paths share the same recovery-aware object-directory policy. Publishing a persistent rollback directory requires synchronizing its entry in the object directory, including under `file-sync` and when replacing an inline object with an external-data object. The backup directory itself also remains synchronized. These barriers preserve the recovery source for a failed multi-drive publication.
|
||||
|
||||
A fresh inline `file-sync` commit can finish writing, synchronizing, and publishing metadata within its preparation task even when directory grouping is enabled. The sync-budget regression checks two disk-local queue submissions on Linux and one on macOS for that path. macOS also performs separate Tokio filesystem preparation. Persistent rollback publication and strict directory grouping retain their respective synchronization and scheduling requirements.
|
||||
|
||||
Application synchronization counts and device IOPS measure different layers. Filesystem journals, page-cache hits, writeback, request merging, and device behavior determine the physical operations and latency. For a reproducible profile, record the exact server revision, effective bucket durability mode, storage-class configuration, object-size distribution, key depth, versioning, concurrency, filesystem, and physical device mapping. Use an isolated test bucket with separate fresh-key PUT, overwrite, GET, and DELETE phases. Collect a background-only interval as well as each foreground interval, keeping scanner and heal state explicit. Pair application syscall traces with device telemetry over the same interval, then normalize counts by completed requests.
|
||||
|
||||
On Linux, an instrumented run can use `strace -f -c -p <pid> -e trace=openat,openat2,close,newfstatat,fstat,statx,getdents64,mkdir,mkdirat,rmdir,rename,renameat,renameat2,unlink,unlinkat,read,pread64,write,pwrite64,fsync,fdatasync` together with `iostat -x 1`. Tracing adds overhead and includes other server activity, so retain an untraced run for latency and throughput comparison. The [scanner benchmark runbook](scanner-benchmark-runbook.md) covers background-pressure measurement.
|
||||
|
||||
Server-side packing of multiple S3 keys into shared containers requires a persistent index, atomic publication, per-key version/delete semantics, garbage collection, and heal/recovery support. These requirements also apply when compacting historical objects. Current inline storage keeps each object's directory and metadata entry independently addressable.
|
||||
|
||||
## Recovery from corrupt object metadata
|
||||
|
||||
A zero-length or unreadable `xl.meta` is corruption, not an absent object.
|
||||
|
||||
@@ -60,7 +60,6 @@ Run **before** at the baseline column and **after** at the enabled column, every
|
||||
| P1 isolation | `RUSTFS_INTERNODE_CHANNEL_ISOLATION` | `false` (default) | `true` |
|
||||
| P1 bulk pool | `RUSTFS_INTERNODE_BULK_CHANNELS` | unset | `2` |
|
||||
| P2 msgpack-only | `RUSTFS_INTERNODE_RPC_MSGPACK_ONLY` + `RUSTFS_INTERNODE_RPC_MSGPACK_ONLY_FLEET_CONFIRMED` | both `false` (default) | per the convergence runbook |
|
||||
| P2 per-peer msgpack | `RUSTFS_INTERNODE_RPC_MSGPACK_ONLY` + `RUSTFS_INTERNODE_RPC_MSGPACK_PEER_CAPABILITY` | both `false` (default) | both `true`, fleet confirmation `false`, stable direct/affine peer connections only |
|
||||
| P3 prewarm | `RUSTFS_INTERNODE_PREWARM` | `false` (default) | `true` |
|
||||
| P3 offline bypass | `RUSTFS_INTERNODE_OFFLINE_BYPASS` | `false` (default) | `true` |
|
||||
| P3 reprobe / threshold | `RUSTFS_INTERNODE_OFFLINE_REPROBE_SECS` / `RUSTFS_INTERNODE_OFFLINE_FAILURE_THRESHOLD` | defaults | defaults (`DEFAULT_INTERNODE_OFFLINE_REPROBE_SECS`, `DEFAULT_INTERNODE_OFFLINE_FAILURE_THRESHOLD`) |
|
||||
@@ -228,27 +227,6 @@ Every offered Ping is recorded, including schedule drops, deadlines and errors.
|
||||
|
||||
Use identical probe source and settings on the two frozen library revisions, in separate Cargo targets, and record the harness overlay and binary identity independently from the already-built clean server binaries. Qualify repeated A cells and then A1/B1/B2/A2 before expanding to coalescing, isolation, pool or window experiments. A successful local mock regression verifies the probe's calling and accounting boundaries; acceptance against the actual authenticated four-node services is still required.
|
||||
|
||||
### Timeout-boundary evidence
|
||||
|
||||
Use `scripts/check_internode_rpc_timeout_boundary.py` after a task-owned fault injector has recorded three independent probe reports: a stall whose measured duration remains below the declared RPC timeout, a stall whose requested and measured durations both exceed it, and a no-fault recovery run. The gate does not inject faults, deploy services or infer a timeout from wall-clock labels; the manifest references separate measured fault receipts and the raw reports.
|
||||
|
||||
```json
|
||||
{
|
||||
"schema": 1,
|
||||
"scope": "internode-rpc-timeout-boundary",
|
||||
"rpc_timeout_ms": 10000,
|
||||
"under_budget": {"fault_receipt": "under-fault.json", "report": "under.json"},
|
||||
"over_budget": {"fault_receipt": "over-fault.json", "report": "over.json"},
|
||||
"recovery": {"report": "recovery.json"}
|
||||
}
|
||||
```
|
||||
|
||||
```bash
|
||||
python3 scripts/check_internode_rpc_timeout_boundary.py --manifest /path/to/manifest.json --out /path/to/verification.json
|
||||
```
|
||||
|
||||
Each fault receipt is a bounded JSON object containing a non-empty `target`, `requested_stall_ms` and `actual_stall_ms`. Exit 0 requires complete bulk coverage, exact offered-control coverage, no errors below the timeout, at least one `deadline` above it, and a zero-error recovery report. `schedule_drop` and `rpc_error` never substitute for the expected deadline. Exit 1 means structurally complete evidence did not satisfy the boundary; exit 2 means the evidence is invalid. The verifier binds all reports and fault receipts by SHA-256, rejects path escape/reuse and refuses to overwrite or alias raw evidence. Deployment ownership, actual fault delivery, phase-scoped telemetry and performance attribution remain separate gates.
|
||||
|
||||
## Acceptance gates
|
||||
|
||||
Record the verdict (pass/fail plus measured delta) in the paired directory's summary.
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
# Internode msgpack/JSON convergence runbook
|
||||
|
||||
**Use this when:** operating or changing the staged retirement of the JSON compatibility fields on internode gRPC metadata RPCs, deciding whether a fleet may enable `RUSTFS_INTERNODE_RPC_MSGPACK_ONLY`, or adding a new `*_bin` proto field.
|
||||
**Source of truth:** `crates/config/src/constants/internode.rs` (`ENV_INTERNODE_RPC_MSGPACK_ONLY`, `ENV_INTERNODE_RPC_MSGPACK_ONLY_FLEET_CONFIRMED`, `ENV_INTERNODE_RPC_MSGPACK_PEER_CAPABILITY` and their compile-time default-off asserts), `crates/protos/src/node.proto` (`*_bin` fields), `decode_msgpack_or_json` in `rustfs/src/storage/rpc/node_service/disk.rs` (server side) and `crates/ecstore/src/cluster/rpc/remote_disk.rs` (client side), `crates/io-metrics/src/internode_metrics.rs` (counters).
|
||||
**Source of truth:** `crates/config/src/constants/internode.rs` (`ENV_INTERNODE_RPC_MSGPACK_ONLY`, `ENV_INTERNODE_RPC_MSGPACK_ONLY_FLEET_CONFIRMED` and their compile-time default-off asserts), `crates/protos/src/node.proto` (`*_bin` fields), `decode_msgpack_or_json` in `rustfs/src/storage/rpc/node_service/disk.rs` (server side) and `crates/ecstore/src/cluster/rpc/remote_disk.rs` (client side), `crates/io-metrics/src/internode_metrics.rs` (counters).
|
||||
|
||||
This is a cross-version change and proceeds by observation-gated stages, never in one step.
|
||||
|
||||
@@ -16,7 +16,6 @@ Internode RPCs dual-encode each metadata value as a msgpack binary field (`*_bin
|
||||
| `rustfs_system_network_internode_msgpack_json_decode_total` | `direction`, `message`, `codec` | a msgpack or JSON decode succeeded |
|
||||
| `rustfs_system_network_internode_msgpack_json_fallback_total` | `direction`, `message` | a decode fell back to JSON because `_bin` was empty |
|
||||
| `rustfs_system_network_internode_msgpack_json_decode_error_total` | `direction`, `message`, `codec` | either codec failed to decode (`codec` = the failed codec) |
|
||||
| `rustfs_system_network_internode_msgpack_send_decisions_total` | `decision`, `reason` | an eligible RPC selected `msgpack_only` or `dual_write`; reasons are fixed to `manual_fleet`, `peer_capable`, `peer_unknown`, `peer_revoked`, or `not_requested` |
|
||||
|
||||
`direction="request"` is a server decoding a peer's request (`node_service/disk.rs`); `direction="response"` is a client decoding a peer's response (`cluster/rpc/remote_disk.rs`), including the list-level `ReadMultiple` / `BatchReadVersion` fallbacks. `message` is the value name (`FileInfo`, `RawFileInfo`, `ReadMultipleResp`, ...).
|
||||
|
||||
@@ -66,26 +65,20 @@ Convergence-ready:
|
||||
| request | `ReadMultiple.read_multiple_req` | `ReadMultipleReq` |
|
||||
| request | `BatchReadVersion.batch_read_version_req` | `BatchReadVersionReq` |
|
||||
| request | `Read*.opts` | `ReadOptions` |
|
||||
| request | `DeleteVersions.versions` | binary-list-driven `FileInfoVersions` batch |
|
||||
| request | `DeleteVersions.opts` | named-map `DeleteOptions` |
|
||||
| response | `ReadVersion.file_info` | `FileInfo` (`cluster/rpc/remote_disk.rs`) |
|
||||
| response | `ReadXL.raw_file_info` | `RawFileInfo` |
|
||||
| response | `RenameData.rename_data_resp` | `RenameDataResp` |
|
||||
| response | `ReadMultiple` response list | per-item plus list fallback |
|
||||
| response | `BatchReadVersion` response list | per-item plus list fallback |
|
||||
|
||||
Single-version delete fields retain dual-write until their own observation window establishes fleet-wide readiness.
|
||||
Delete messages, kept on dual-write until their own window reads zero with `_bin`-first decoders deployed fleet-wide (the client always dual-writes these regardless of the flags):
|
||||
|
||||
| Direction | Message / field | Status |
|
||||
| --- | --- | --- |
|
||||
| request | `DeleteVersion.file_info` (`FileInfo`) | `_bin` present; dual-write; converge after its own window |
|
||||
| request | `DeleteVersion.opts` (`DeleteOptions`) | same |
|
||||
|
||||
`DeleteVersions` receivers select the binary list whenever `versions_bin` contains entries. That list supplies the batch length and every decoded item, including requests whose JSON `versions` list is empty or has a different length. Empty or corrupt binary items reject the complete request before disk mutation. A request with an empty binary list uses the legacy JSON list and its length. Options retain their independent binary-first decoder. The sender captures the codec policy once per complete batch. Default mode emits both lists and both option encodings. Setting both `RUSTFS_INTERNODE_RPC_MSGPACK_ONLY=true` and `RUSTFS_INTERNODE_RPC_MSGPACK_ONLY_FLEET_CONFIRMED=true` emits the binary list and named-map options, with empty JSON fields. Before confirming the fleet, deploy the corrected binary-list-driven receiver to every destination and verify the observation counters and binary-only deletion results. Rolling upgrades and rollback to a receiver that traverses the JSON list require JSON compatibility mode before replacing nodes. These fields remain in the protobuf schema for older peers.
|
||||
|
||||
User-object recoverable deletion uses `RecoverDeleteTransaction` and guarded mutation messages with their existing binary payloads. Measure those operations with transaction preparation, commit, rollback and cleanup stages. The `DeleteVersions` codec comparison covers the legacy/internal batch RPC and its consumers.
|
||||
|
||||
`DeleteVersions` binary-only sending uses the explicit fleet-confirmation gate. In per-peer capability mode, batch deletion emits both JSON and binary fields. The generic peer capability proof covers the other eligible request families; binary-list-driven deletion additionally requires the corrected batch receiver on every destination.
|
||||
| request | `DeleteVersions.versions` (`FileInfoVersions`) | same |
|
||||
| request | `DeleteVersions.opts` (`DeleteOptions`) | same |
|
||||
|
||||
Still JSON-only:
|
||||
|
||||
@@ -99,55 +92,15 @@ Run the fleet with both flags at their default (`false`) for at least one full o
|
||||
|
||||
## Stage 1 — Stop writing JSON (env-gated)
|
||||
|
||||
Flag semantics are defined in `crates/config/src/constants/internode.rs`; all three flags default off. `RUSTFS_INTERNODE_RPC_MSGPACK_ONLY=true` requests msgpack-only, and JSON stops only when either fleet confirmation is true or the peer-capability gate is true and the exact current peer boot epoch plus physical connection generation has an authenticated capability proof. Unknown, unsupported, revoked, stale, or reconnecting peers continue dual-write. Decoders keep the JSON fallback in every state.
|
||||
Flag semantics are defined on `ENV_INTERNODE_RPC_MSGPACK_ONLY` and `ENV_INTERNODE_RPC_MSGPACK_ONLY_FLEET_CONFIRMED` in `crates/config/src/constants/internode.rs`; both default off and only their conjunction stops the JSON write for convergence-ready fields. Decoders keep the JSON fallback in every state.
|
||||
|
||||
1. Rehearse `request-only`: `RUSTFS_INTERNODE_RPC_MSGPACK_ONLY=true`, `..._FLEET_CONFIRMED=false`. Behaviour is unchanged (still dual-write) and safe with old peers; any fallback or decode-error increment still blocks.
|
||||
2. Canary: set both flags `true` on one node, restart that node only, and watch the fallback and decode-error counters for a soak period with real internode traffic.
|
||||
3. Fleet: if the counters stay zero, enable both flags fleet-wide with a rolling restart.
|
||||
4. Rollback: set either flag to `false` (or unset it) and restart; no wire format changed, so rollback is immediate.
|
||||
|
||||
An alternative per-peer canary keeps `RUSTFS_INTERNODE_RPC_MSGPACK_ONLY_FLEET_CONFIRMED=false`, sets `RUSTFS_INTERNODE_RPC_MSGPACK_ONLY=true` and `RUSTFS_INTERNODE_RPC_MSGPACK_PEER_CAPABILITY=true`, and confirms `reason="peer_capable"` only for proved peers while every `peer_unknown` or `peer_revoked` decision remains `dual_write`. Enable this mode only when one client HTTP/2 connection has stable affinity to one RustFS peer process; an L7 proxy that routes streams from one connection to different backend versions invalidates the connection-generation proof. A connection-generation change may rebuild one request as dual-write only when the retained local guard proves rejection before the first payload frame; remote status text, a dispatched request, or an ordinary transport error never authorizes that compatibility retry. Disable either the base request flag or peer-capability flag to roll back immediately.
|
||||
|
||||
The benchmark driver pins these operator states as dry-run phases (`before`, `request-only`, `canary`, `after`, `rollback`); see [internode-grpc-benchmark-runbook.md](internode-grpc-benchmark-runbook.md).
|
||||
|
||||
### Per-peer mixed-version evidence gate
|
||||
|
||||
The per-peer gate is not qualified by a successful command transcript alone. Record three ordered four-node measurement phases: `mixed-old-peer` (three candidate nodes and one previous-version node), `all-new` (four candidate nodes), and `rollback-old-peer` (replace one candidate with the previous-version image and verify readback). Every candidate node uses `MSGPACK_ONLY=true`, `MSGPACK_PEER_CAPABILITY=true`, and `MSGPACK_ONLY_FLEET_CONFIRMED=false`; previous-version nodes use all three false. Each phase must run real PUT/GET/metadata traffic and retain the exact candidate/baseline revisions, topology, timestamps, workload result, environment, and raw Prometheus HTTP API JSON.
|
||||
|
||||
Assign one unique `msgpack-<32-hex>` cluster ID for this run, set `OTEL_RESOURCE_ATTRIBUTES=rustfs.cluster.id=<id>` on every node, and retain the collector’s resource-to-telemetry conversion so Prometheus exposes `rustfs_cluster_id`. Capture these PromQL vectors over each non-overlapping phase window and store the complete HTTP API response, not extracted numbers:
|
||||
|
||||
```promql
|
||||
sum by (decision, reason) (increase(rustfs_system_network_internode_msgpack_send_decisions_total{rustfs_cluster_id="$CLUSTER_ID"}[$WINDOW]))
|
||||
sum(increase(rustfs_system_network_internode_msgpack_json_fallback_total{rustfs_cluster_id="$CLUSTER_ID"}[$WINDOW])) or vector(0)
|
||||
sum(increase(rustfs_system_network_internode_msgpack_json_decode_error_total{rustfs_cluster_id="$CLUSTER_ID"}[$WINDOW])) or vector(0)
|
||||
(sum(increase(rustfs_system_network_internode_operation_errors_total{rustfs_cluster_id="$CLUSTER_ID",backend="grpc",operation!="grpc_read_version"}[$WINDOW])) or vector(0)) + (sum(increase(rustfs_system_network_internode_operation_classified_errors_total{rustfs_cluster_id="$CLUSTER_ID",backend="grpc",operation="grpc_read_version",classification!="remote_response"}[$WINDOW])) or vector(0))
|
||||
```
|
||||
|
||||
Build a schema-1 manifest with `scope="internode-msgpack-peer-canary"`, `source_revision`, `baseline_revision`, `stable_connection_affinity=true`, `max_metric_lag_secs`, `cluster_id`, an `images` path to sanitized image inspections, and the three phase records. Each phase records integer `start_unix`/`end_unix`, exact topology counts plus a `node1..node4` revision map, the new/old node environment maps, the exact four PromQL strings with a window equal to the phase duration, an `images` path to the verified per-node image ID/revision map, a relative workload JSON path, and relative paths for `decisions`, `fallback`, `decode_errors`, and `rpc_errors`. The workload artifact records `status`, a non-zero integer `requests`, and `rollback_readback` (true only for `rollback-old-peer`). Validate without rewriting raw evidence:
|
||||
|
||||
```bash
|
||||
python3 scripts/check_internode_msgpack_peer_canary.py \
|
||||
--manifest /path/to/evidence/manifest.json \
|
||||
--out /path/to/evidence/verification.json
|
||||
```
|
||||
|
||||
The verifier rejects reused/escaping/overwritten artifacts, duplicate or non-exact decision labels, labeled scalar queries, mismatched PromQL windows, wrong per-node revisions, missing peer-capable traffic, missing mixed/rollback dual-write traffic, manual-fleet or not-requested leakage, failed workload/readback, non-zero fallback/decode/gRPC transport-or-protocol errors, stale timestamps, invalid topology, and candidate/baseline identity reuse. The gRPC error query checks every operation except `grpc_read_version` through the total error counter. Read-version uses its classified counter and excludes only `remote_response`, because a successful gRPC exchange can carry an expected application-level disk response; `grpc_status` and `response_decode` still block the gate. TCP/HTTP operation outcomes remain outside this msgpack gRPC compatibility gate and the workload's zero-error contract remains independently enforced. The workload artifact binds its raw benchmark `summary.csv` by relative path and SHA-256; rollback also records equal source/download content hashes. Verifier output includes SHA-256 receipts for every raw query, workload, and summary artifact. Passing this gate is compatibility evidence only; production soak and ABBA performance drift gates remain separate.
|
||||
|
||||
Use the orchestration runner to create the isolated Compose project, execute the three phases, preserve a known object across node2 rollback, capture the four raw Prometheus responses, build the manifest, and invoke the verifier. Without `--execute` it prints a redacted plan and makes no external changes. The runner verifies distinct immutable image IDs and full `vcs-ref` or OCI revision labels against the requested revisions, pins Compose to those IDs, and checks each running container’s image and telemetry identity in every phase. If an image backend omits both recognized labels from `image inspect`, the runner queries `image history` by the already resolved immutable image ID and accepts only a canonical 40-hex revision from the newest matching `LABEL` instruction; an existing mismatched config label never falls back. Images without matching full revision evidence are rejected. It records whether each sanitized identity receipt came from config labels or immutable history without retaining container credentials or unrelated history. Execution refuses an existing Compose project/volume set and removes its isolated containers and volumes on exit unless `--keep-up` is explicit.
|
||||
|
||||
```bash
|
||||
scripts/run_internode_msgpack_peer_canary.sh \
|
||||
--candidate-image rustfs/rustfs:candidate \
|
||||
--baseline-image rustfs/rustfs:previous \
|
||||
--candidate-revision <40-hex-candidate-commit> \
|
||||
--baseline-revision <40-hex-previous-commit> \
|
||||
--prometheus-query-url http://127.0.0.1:9090/api/v1/query \
|
||||
--out-dir /path/to/evidence \
|
||||
--execute
|
||||
```
|
||||
|
||||
The runner requires Docker Compose, AWS CLI, curl, Warp through `run_object_batch_bench.sh`, Python, the four-node local cluster compose file, prebuilt candidate/baseline images, and an already reachable Prometheus/OTLP pipeline. It never builds, pulls, or starts observability images implicitly. Execution rejects reused Compose resources or a non-empty evidence directory and performs a Prometheus vector preflight before starting the cluster. The workload summary must contain only successful zero-error rows with parseable request rates; the rollback phase must read byte-identical object data written before the version replacement.
|
||||
|
||||
## Stage 2 — Remove the proto JSON fields
|
||||
|
||||
Only after Stage 1 has been stable fleet-wide for a full window with the counters still zero.
|
||||
@@ -160,7 +113,7 @@ Only after Stage 1 has been stable fleet-wide for a full window with the counter
|
||||
| Stage | Wire format broken? | Rollback |
|
||||
| --- | --- | --- |
|
||||
| 0 Observe | no | n/a (metrics only) |
|
||||
| 1 msgpack-only send | no | unset the base request flag, or unset the selected fleet/peer gate, and restart |
|
||||
| 1 msgpack-only send | no | unset either flag and restart |
|
||||
| 2 remove fields | yes | redeploy the prior release; field numbers stay `reserved` |
|
||||
|
||||
## Related
|
||||
|
||||
@@ -68,7 +68,6 @@ iostat -xz 5 > telemetry/iostat.txt &
|
||||
|
||||
| Knob | Default | Controls | Validating stage | Risk if widened |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| `RUSTFS_ERASURE_SHARD_BYTES` | `0` | `erasure_block_size_for` in `crates/filemeta/src/fileinfo.rs` derives a new user-object block from data-shard count times this byte value. Zero selects the historical 1 MiB block. Validation limits live in `crates/config/src/constants/object.rs`. Cached at first use. | Stripe count, `erasure_encode_cpu`, GET path, range bytes, TTFB and RSS | Changes geometry across pools and storage classes, direct-write eligibility and bitrot frame size |
|
||||
| `RUSTFS_ERASURE_ENCODE_MAX_INFLIGHT_BYTES` | 32MiB (`crates/ecstore/src/erasure/coding/encode.rs`) | EC encode producer/consumer memory budget (blocks queued between encode and shard write) | `set_disk_encode` P95 + `rustfs_ec_encode_inflight_bytes_current` | RSS growth under high concurrency |
|
||||
| `RUSTFS_ERASURE_ENCODE_BYTESMUT_INGEST` | `true` | Streaming encode reads into an owned, EC-sized buffer, avoiding the input-to-encoder block copy. Set `false` to select the previous streaming Vec ingest path; batched encode always uses owned buffers. Cached at first use. | `erasure_encode_cpu`, PUT throughput and RSS | No disk-format change; queue budget does not bound total process memory |
|
||||
| `RUSTFS_OBJECT_IO_BUFFER_SIZE` | 128KiB (`crates/config/src/constants/object.rs`) | Streaming read-in / write-out block size | `ingress_prepare`, `set_disk_encode` | Larger buffers = fewer polls, more resident memory |
|
||||
@@ -109,38 +108,6 @@ Collect detailed diagnostics in dedicated rounds with `RUSTFS_OBS_PUT_STAGE_METR
|
||||
|
||||
These metrics cover the disk queue's Fsync lane. Direct fsync dispatches and cleanup work on the Isolated lane require their own stage observations. Peak gauges are process-lifetime high-water marks for workers whose diagnostics started while the detailed PUT gate was enabled, including startup and warmup. A retiring closure and its replacement can briefly overlap in this count while execution concurrency remains bounded. Restart each server per leg to establish a fresh lifetime. Guards finish accounting on normal exit and panic, including when the gate is disabled while a measured worker remains alive. Diagnostics use atomic accounting and cached metric handles outside the disk queue mutex; the recorder may acquire its own publication locks. Hold this observation overhead consistent within diagnostic comparisons.
|
||||
|
||||
#### Erasure block geometry experiment
|
||||
|
||||
The [isolated Docker inline lab](../../.docker/inline-lab/README.md) provides a frozen-source image and an owned external deployment adapter for the existing ABBA runner.
|
||||
|
||||
Freeze the inline per-shard budget, durability mode, codec mode, queue budget, and binary before changing `RUSTFS_ERASURE_SHARD_BYTES`. Use integer bytes for this setting. `configured_erasure_shard_bytes` validates its alignment and upper bound and caches the result. Restart the tested processes and create each leg's measurement dataset after the new geometry is active. Existing versions and active multipart uploads retain their recorded geometry, so pre-existing GET data cannot establish the new-write behavior.
|
||||
|
||||
The setting applies to every newly constructed user-object geometry. A fixed shard value produces different block sizes for different data-shard counts, including different storage classes in one pool. Record the persisted block size in each test cell. The per-shard placement budget and body materialization limits are separate contracts. Verified inline body reuse is bounded to 2 MiB and 32 frames. Trace metadata body verification and subsequent body decoding together, including reuse within the selected version snapshot.
|
||||
|
||||
Use one candidate shard value per ABBA run, starting with a small change appropriate to the target geometry. Include full GET, PUT, small Range GET, versioned objects, multipart, transformed objects, degraded reads and a reader rollback fixture. Large blocks reduce stripe transitions while expanding read/verification granularity and the maximum single-block direct-write input. The streaming queue has a minimum of one expanded block, and direct paths have their own buffering. Validate RSS, CPU scheduling, first-byte latency and error counts alongside throughput.
|
||||
|
||||
The pinned MinIO Community comparison uses a 1 MiB block for new objects. Its inline buffer writers can accumulate multiple blocks while the total erasure shard remains within the storage-class budget. RustFS supports bounded multi-block inline for known-length unprotected PUTs at its default 1 MiB block. The accumulated data-shard budget is 128 KiB, reduced to 16 KiB for versioned writes. EC12+4 therefore admits up to 1,572,856 logical bytes, with each bitrot frame header additionally charged to the bounded writer. Transformed and proof-protected writes retain their existing single-block admission contract. A geometry override is an independently measured configuration experiment with its own read-path and compatibility results. [MinIO object geometry](https://github.com/minio/minio/blob/7aac2a2c5b7c882e68c1ce017d8256be2feea27f/cmd/storage-datatypes.go#L388-L399) and [inline writer selection](https://github.com/minio/minio/blob/7aac2a2c5b7c882e68c1ce017d8256be2feea27f/cmd/erasure-object.go#L1398-L1440).
|
||||
|
||||
For transport comparisons, keep geometry fixed and independently evaluate JSON compatibility mode and the existing MessagePack-only flag pair. Inline RenameData carries its data through metadata RPCs, so JSON serialization and duplicate payloads can outweigh the saved part-file I/O. The Docker lab pins and checks both transport flags on every node. Use its version, geometry and multipart manifests for mixed-reader and full-reader rollback checks before a performance run. Restore JSON compatibility mode before introducing an older peer and record the oldest reader actually tested.
|
||||
|
||||
#### Borrowed inline verification kernel
|
||||
|
||||
The inline verifier operates on already loaded frame slices. Its manual ABBA comparison exercises the streaming verifier and the borrowed verifier over identical 4 KiB and 128 KiB shard payloads. Capture the reported debug/release profile, iteration counts, elapsed times and baseline drift. This measures frame verification CPU and copying costs. End-to-end acceptance uses the fixed-condition Docker ABBA matrix with request errors, throughput and tail latency recorded for every leg.
|
||||
|
||||
```bash
|
||||
cargo test -p rustfs-ecstore --lib --features test-util \
|
||||
inline_bitrot_borrowed_verification_benchmark -- --ignored --nocapture --test-threads=1
|
||||
```
|
||||
|
||||
For batch-delete codec acceptance, run `delete_versions_request_encoding_preserves_batches_and_fleet_guards` to verify the emitted fields, named-map options and encoded byte counts. Run `delete_versions_wire_roundtrip_preserves_json_and_binary_only_batches` in the `rustfs` library tests to exercise the actual remote sender, protobuf handler and disk mutation with both codec modes. Its retained-object oracle protects against deleting an unselected entry. Deploy JSON compatibility mode during mixed-version and old-reader tests, then confirm the homogeneous upgraded fleet before enabling binary-only batches as described in the [codec convergence runbook](internode-msgpack-json-convergence-runbook.md).
|
||||
|
||||
```bash
|
||||
cargo nextest run -p rustfs-ecstore --lib --features test-util --no-tests fail \
|
||||
-E 'test(delete_versions_request_encoding_preserves_batches_and_fleet_guards)'
|
||||
NO_PROXY='*' no_proxy='*' cargo nextest run -p rustfs --lib --features gateway-stack --no-tests fail \
|
||||
-E 'test(delete_versions_wire_roundtrip_preserves_json_and_binary_only_batches)'
|
||||
```
|
||||
|
||||
### 5.2 GET
|
||||
|
||||
| Knob | Default | Controls | Validating stage | Risk if enabled |
|
||||
|
||||
@@ -207,5 +207,3 @@ Pending followers use indexed slab slots and a monotonic waiter identity. Detach
|
||||
A source that disappears before native open fails its own waiter, while healthy requests in that batch still complete file and directory sync. Other file-sync errors retain the existing fail-fast batch behavior, preventing repeated I/O against a failing disk. Admission or native-dispatch failure also fails the batch. File sets and their existing resource units move together into one blocking dispatch, and result senders remain with the async worker so cancellation keeps its established notification contract.
|
||||
|
||||
With detailed PUT metrics enabled, a successful queue ownership transfer adds a `follower_timeout` role sample to `rustfs_s3_put_object_rename_fdatasync_group_wait_ms`. Evaluate the experiment against the same source, durability setting, topology, and workload. Include sync failures, follower publication before batch completion, cancellation, and peak resource usage in acceptance.
|
||||
|
||||
Grouped worker batches use `mode="grouped"` on `rustfs_s3_put_object_rename_fdatasync_batch_files`. The `parallel` mode is reserved for the independent per-file fanout path, so dashboards can distinguish serialized user-space grouping from bounded parallel dispatch.
|
||||
|
||||
@@ -114,7 +114,7 @@ Cloudflare's proxy may buffer the entire request body before forwarding and can
|
||||
|
||||
For HTTP `UploadPart`, the inactivity budget counts time waiting for raw request-body bytes while storage is requesting input. Positive raw bytes reset the budget, including fragments of a signed AWS chunk that has not yet finished decoding. Foreground admission, capped-session staging, and storage backpressure do not consume the budget. Finishing the declared payload does not bypass the signed terminator, required trailers, or final body validation. This is an inactivity limit, so an upload making progress can take longer than 300 seconds overall.
|
||||
|
||||
Setting the timeout to `0` disables it for ordinary uploads. Multipart sessions with an explicit total-object-size cap retain a minimum 300-second timeout, including when the configured value is `0`. After a body-stall timeout, HTTP/1 uses the existing raw-body drain and closes the connection; HTTP/2 releases the affected stream and keeps the connection usable. On both stacks, the HTTP/1 early-response drain uses the configured idle timeout; `0` leaves that drain unbounded.
|
||||
Setting the timeout to `0` disables it for ordinary uploads. Multipart sessions with an explicit total-object-size cap retain a minimum 300-second timeout, including when the configured value is `0`. After a body-stall timeout, HTTP/1 uses the existing raw-body drain and closes the connection; HTTP/2 releases the affected stream and keeps the connection usable.
|
||||
|
||||
`UploadPart` requires a known logical byte length. RustFS uses the length normalized by S3S after authentication and decoding, with an exact logical stream length as a fallback. A bare `x-amz-decoded-content-length` or `Content-Encoding: aws-chunked` declaration cannot supply this length by itself. Requests reaching an ordinary upload session without a known length return `MissingContentLength` (HTTP 411) before body ingestion; capped sessions retain their `UnexpectedContent` rejection. Preserve the client's framing and signed headers through the proxy. The 5 GiB limit applies to each part request, not the combined size of an ordinary multipart upload.
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# S3 Tables Catalog Lifecycle
|
||||
|
||||
**Use this when:** disabling an object-backed table catalog, cleaning up metadata left by deleted empty tables, diagnosing catalog registration reads, or recovering an interrupted table rename.
|
||||
**Use this when:** disabling an object-backed table catalog, cleaning up metadata left by deleted empty tables, or diagnosing catalog registration reads.
|
||||
**Source of truth:** `rustfs/src/admin/handlers/table_catalog/config.rs`, `rustfs/src/table_catalog/store/cleanup.rs`, and `rustfs/src/table_catalog/store/registration_cache.rs`.
|
||||
|
||||
## Disable A Table Catalog
|
||||
@@ -30,11 +30,3 @@ Each server keeps a bounded memory cache of validated ownership records. Reuse r
|
||||
Registration still enumerates namespace and warehouse-index prefixes, so enumeration grows with catalog size. Cache hits reduce object-body reads and JSON parsing. A hit reuses previously validated content without reading the current body again to prove physical readability. Conditional writes, rollback, and lock-loss fencing continue to control publication; the cache does not synchronize writers that bypass those guards.
|
||||
|
||||
Existing catalog, Iceberg metadata, and Console/client request formats remain unchanged. See the [S3 Tables support matrix](../architecture/s3-tables-support-matrix.md) for supported API boundaries.
|
||||
|
||||
## Interrupted Table Renames
|
||||
|
||||
Every server starts rename recovery once its own application context is ready, including embedded servers. This runs independently of `RUSTFS_TABLE_MAINTENANCE_SCHEDULER_ENABLED`. Recovery scans up to 32 bucket directories per pass, skips their child objects, and admits at most four recoveries concurrently. Passes are separated by five seconds plus jitter; failed buckets retry with exponential delays from 10 seconds up to five minutes. Larger catalogs may need multiple passes. These intervals do not guarantee a recovery deadline while storage or locks remain unavailable.
|
||||
|
||||
Recovery resumes the rename referenced by the bucket's `active_rename_id` through the same restore fence, publication locks, conditional writes, and state validation used by `RenameTable`. Reads remain fenced until finalization completes. Missing or invalid intent records retain the fence and produce `table_catalog_rename_recovery` warnings; preserve those records and collect the warnings for investigation. Shutdown stops new admissions and drains started recoveries.
|
||||
|
||||
A completed rename preserves the table UUID, warehouse location, and metadata location. Dropping the empty source namespace removes its namespace record and retains valid `DELETED` table/view records. Active objects, pending states, child namespaces, unreadable records, and active rename fences still prevent namespace deletion. Namespace deletion does not remove metadata or data files referenced by the destination table.
|
||||
|
||||
@@ -24,22 +24,6 @@ The authenticated admin metrics endpoint exposes the same counters through the o
|
||||
|
||||
A missing `http` field means an older or non-reporting node, not zero failures. Old map-encoded RPC readers ignore the additive field; new readers accept older snapshots. In mixed-version deployments, check reporting coverage before aggregating a fleet-wide ratio.
|
||||
|
||||
## Browser upload staging
|
||||
|
||||
On the gateway stack, a browser POST whose file length is unknown is staged in the process's system temporary directory before the ordinary PUT path begins. Staging uses an ephemeral in-memory key and authenticated encrypted frames. Two staging files may be admitted per process; further requests receive `SlowDown` immediately, with no fairness or queue guarantee. The slots remain occupied through replay and any in-flight file I/O, including after cancellation.
|
||||
|
||||
Staged ciphertext is limited to 6 GiB per process, including each frame's 4-byte header, 16-byte authentication tag, and authenticated end frame. The budget remains charged until the file closes. A request that cannot reserve its next frame receives `SlowDown`; a retry may keep failing until other staging files close. The S3 single-object plaintext limit remains 5 GiB, which needs approximately 5 GiB plus 1.6 MiB of framing. These limits are fixed and are not configuration settings.
|
||||
|
||||
On Unix, RustFS queries the open file's filesystem before consuming the file body and before each frame. It requires available space to remain at least 10% of filesystem capacity, capped at 1 GiB and floored at one byte, plus the rounded-up frame and other in-process pending writes. The initial check includes space for an end frame and rejects insufficient space with `SlowDown` before polling the file body. An unavailable or invalid space query, including inconsistent filesystem block counts, fails closed with `InternalError`. Other platforms fail closed for unknown-length POST staging because this file-handle space query is not implemented there. The check adds at least one filesystem statistics query per frame, currently 64 KiB of plaintext. If a concurrent write makes a snapshot stale, RustFS retries up to four times and then returns `SlowDown`, even if free space is otherwise sufficient.
|
||||
|
||||
These checks are process-local and the free-space result is only a snapshot, not a disk reservation: other processes, filesystem metadata, delayed allocation, quotas, and external writers can still exhaust the disk. Use a dedicated size-limited filesystem or an appropriate filesystem quota for hard isolation, accounting for every RustFS process. A tmpfs can also exhaust memory. Disk write errors still fail the request.
|
||||
|
||||
Staging, including file creation, body ingest, and preparing replay, has a fixed two-hour absolute deadline after slot admission. This can reject a slow but progressing client with `RequestTimeout`, independently of the existing body idle and minimum-progress guards. The deadline ends before PUT begins; it cannot cancel a committed write. Blocking file operations cannot be forcibly interrupted, so their slot and budget remain held until they finish, and replay or downstream storage stalls are not bounded by this deadline.
|
||||
|
||||
A process-wide worker drops staged file handles outside the asynchronous executor. It retains each file's slot and budget until the handle's drop returns; a stalled close can therefore delay other uploads. If the worker is unavailable or its bounded queue unexpectedly fills, the caller drops the handle synchronously rather than releasing resources early. If the worker cannot be started, unknown-length POST staging fails closed.
|
||||
|
||||
Bucket quota and the final commit checks remain unchanged and run after staging, so a staging resource error can precede a quota error. On Unix, a temporary file must be unlinked before it receives data; normal process termination or a crash closes its last handle, but inherited handles or unusual filesystems can delay reclamation. Temporary-file encryption does not protect plaintext in swap, hibernation images or core dumps.
|
||||
|
||||
## Interpret failed storage probes
|
||||
|
||||
Storage inventory includes an `observations` entry for each probed node. The aggregator owns this provenance even when a peer runs an older version.
|
||||
@@ -84,11 +68,3 @@ No new environment variable, admin authorization action, or recovery command is
|
||||
## Delete recovery during rolling upgrades
|
||||
|
||||
S3 deletes that require bounded recovery need peers that support the bounded prepare protocol. An older peer rejects that prepare before staging a mutation. If too few upgraded drives can satisfy the write quorum, the delete can fail with a retryable error until more peers are upgraded. Ordinary recovery inspection and settlement continue to use the original protocol.
|
||||
|
||||
## Full data disks and deletion recovery
|
||||
|
||||
An ordinary unversioned DELETE permanently removes data, but RustFS first persists a recoverable delete transaction so a failed quorum can restore the object. A filesystem with no allocatable space can reject that journal before any data is removed. This is distinct from PUT pool-capacity admission; changing only that admission does not make deletes work on a physically full disk. Full-disk failures retain the existing S3 `503 ServiceUnavailable` response.
|
||||
|
||||
On Unix, each local disk attempts to keep an 8 MiB physically allocated emergency reserve at `.rustfs.sys/delete-reserve`. The reserve uses incompressible content and is armed only after synced allocation is verified, so sparse or transparently compressed zero blocks cannot be mistaken for recovery headroom. Reopened disk handles share one owner pinned to the physical root. Healthy deletion does not open reserve paths or query capacity. If a repeatable scratch write or an idempotent delete-metadata primitive fails with ENOSPC, the first waiter releases existing reserve blocks and retries that operation once; concurrent waiters serialize behind it and also get one bounded retry against the released capacity. Replenishment cannot race those retries. A failed scratch attempt is discarded before retry; the transaction, publication-plus-sync sequences, and destructive cleanup are never replayed as a unit. Invalid reserve state preserves the original operation error. Successful local finalization only attempts replenishment after reserve depletion and with at least 16 MiB free, on the dedicated fsync lane. Startup may release a persisted reserve below 4 MiB available space before the writable probe. The reserve is outside temporary-object cleanup and changes no object formats, quorum, retention checks, or rollback rules. Rollback to an older server leaves an unused reserve file consuming at most 8 MiB per disk.
|
||||
|
||||
The reserve is bounded recovery headroom, not a guarantee under arbitrary concurrent writes. Large version histories, inode exhaustion, unrelated writers, or multiple simultaneous deletes can still exhaust space. A first upgrade on an already-full disk may have no reserve to release. Reservation failure does not disable otherwise usable storage, and Windows currently retains its existing behavior. Stop uploads and create filesystem headroom if API deletion still fails; do not remove object directories manually because that bypasses metadata and erasure consistency. Versioned DELETE normally creates a marker and does not reclaim the prior payload; permanently deleting a permitted version requires its version ID.
|
||||
|
||||
@@ -898,6 +898,42 @@ done
|
||||
|
||||
The `aggregated.scanner` payload preserves the same scanner progress, checkpoint, pacing, source work, maintenance control, lifecycle expiry, and lifecycle transition fields used by the local scanner status, but only for the responding node; `by_host.*.scanner` keeps that node's host view. Compare the per-node artifacts externally to find old active paths, partial checkpoints, pacing pressure, source-level control pressure, or downstream queue admission problems across the deployment.
|
||||
|
||||
## Reading Cached Bucket Statistics
|
||||
|
||||
Admin usage reads cache the selected scanner baseline for the existing 30-second TTL, then verify every returned bucket against uncached storage-owned incarnation metadata. Identity reads use at most four concurrent operations and a shared two-second deadline after releasing snapshot/refresh locks. Each bucket read linearizes independently against metadata transactions; the response is not an atomic snapshot of the entire namespace. Delayed peer invalidation cannot authorize a predecessor's statistics for a recreated bucket.
|
||||
|
||||
Missing, failed, timed-out, nil, mismatched, or legacy absent identities leave the bucket value absent (unknown), rather than returning the predecessor's value or a confirmed zero. Legacy statistics become visible after a scanner snapshot supplies the current identity. Account-info and usage display apply the existing single-process memory overlay before the final identity filter; distributed local counters are not promoted. When a projection omits values or changes an overlay, totals are partial and `usage_snapshot_converged=false`, and per-tier/replication aggregates are unavailable. Verified cold bucket values remain visible. Such response-only projections cannot be persisted, used for quota admission, or acknowledge dirty work.
|
||||
|
||||
## Reading Scanner Work Observations
|
||||
|
||||
`GET /rustfs/admin/v3/scanner/status` exposes `metrics.scan_observation` for the responding node. Capture timestamped snapshots on every node, including the scanner leader, and compare their deltas over the same workload window. These process-lifetime counters reset on restart; a missing field on an older node means unobserved. They are loaded independently and may briefly disagree while operations finish. This telemetry does not change scan cadence, cached usage reads, quota authority, dirty acknowledgement, or maintenance requirements.
|
||||
|
||||
| Field | Meaning |
|
||||
|---|---|
|
||||
| `metadata_reads.attempts` | Disk metadata read futures started after scanner node admission. Invalid paths and operations cancelled before starting the read do not count. |
|
||||
| `metadata_reads.succeeded`, `not_found`, `errors`, `cancelled` | Storage outcomes, including object/version disappearance and dropped read futures. Successful storage reads can still contain corrupt metadata; decoding and lifecycle failures do not become read errors. Cancellation observes the future, not proof that physical I/O stopped. |
|
||||
| `metadata_reads.bytes` | Total returned buffer lengths for successful reads, including inline data and corrupt metadata. Failed reads contribute no known bytes; this is not physical disk traffic. |
|
||||
| `metadata_reads.elapsed_nanos` | Sum of read-future elapsed time, including disk-permit or RPC waits, failures, and cancellation. Folder pacing, scanner node admission, decoding, and lifecycle work are outside this interval. Concurrent reads overlap, so the sum is not cycle wall time. |
|
||||
| `cache_reuse.clean_prefix` | Clean root subtrees copied under an admitted prefix scope by the non-erasure scanner. |
|
||||
| `cache_reuse.covered_namespace` | Namespace subtrees copied from validated checkpoint coverage. |
|
||||
| `cache_reuse.sampled_namespace` | Cached namespace subtrees copied when the existing periodic sampling selected other work. |
|
||||
| `cache_reuse.classified_namespace` | EC candidate namespace subtrees copied only after the current listing finished classifying them. Health/classification work still runs. |
|
||||
| `cache_reuse.*.count`, `elapsed_nanos` | Completed subtree reuse decisions and time spent copying/attaching their cache entries. Counts are not objects skipped or a cache hit ratio; times exclude classification/listing and are not total cache maintenance cost. |
|
||||
| `bucket_reuse.scoped_cold_bucket` | Whole cold buckets copied and attached to a successfully prepared scoped set cache. Count is copied buckets, not plans selected; later turn publication can still fail. Time covers copying/attachment only. |
|
||||
| `bucket_reuse.current_root` | Existing current bucket roots successfully sent to the local collector after watch, namespace lease and publication epoch checks. Time covers channel send only; failure or cancellation contributes no completed reuse. Collector acceptance and durable publication remain separate. |
|
||||
| `reuse_decisions.*.not_evaluated`, `blocked`, `eligible` | First conditions reached by production clean-prefix, checkpoint-coverage and namespace-sampling gates. Policy exclusions (`missing_scope`, `no_resume_sweep`, `deep_scan`, `forward_sweep`, `active_replication`) precede cache proof checks. `blocked` reports the first failed work/cache condition; later checks are not repeated for diagnostics. `eligible` does not certify a copy: an EC candidate can still require classification. |
|
||||
| `erasure_clean_prefix_walks` | Clean root children that still require walking because EC scans perform independent object-health work. |
|
||||
| `prefix_scope_resolution.evaluated`, `activated`, `blockers` | Proof evaluations retained by the final scope resolver, admitted proof gates, and every failed prerequisite. An admitted proof does not guarantee a reusable child exists; actual subtree copies appear in `cache_reuse`. Startup/required full scans, early scope rejection, and discarded peer/ACK resolutions do not count here. |
|
||||
| `prefix_cycle_completion.evaluated`, `activated`, `blockers` | End-of-cycle proof evaluations for subsequent reuse. An activated completion proof does not mean this cycle reused prefixes or that its later publication/persistence succeeded. One evaluation can increment several blocker counters. |
|
||||
| `scope_fallbacks` | Per-reason breakdown of the existing `usage_freshness.scope_fallback_total`; required full scans are excluded. |
|
||||
| `authoritative_baseline_rejections`, `observed_baseline_rejections` | First failed validation checks for `baseline_unavailable` fallback cycles, preserving the authoritative rejection when the observed fallback also fails. Missing/undecodable authority never reaches observed validation; `authoritative_identity` means it could not anchor an observed fallback. |
|
||||
|
||||
`freshness` describes the responding node's completed-cycle age, not whether every bucket has incorporated new writes. Read `metrics.usage_freshness` for node-local live pending work, guardrail vetoes, durable success and publication state; read `metrics.usage_publication` for the confirmed snapshot's pending count and publication class/outcome, and `cycle_schedule` for effective retry intervals and consecutive superseded cycles. Missing or unknown publication data does not establish zero pending work. A follower or a single node's empty dirty map cannot establish cluster-wide convergence.
|
||||
|
||||
Whole cold bucket copies, current-root reuse and the four subtree categories have different prerequisites and are not interchangeable hit rates. EC clean prefixes retain health walks. Deep or required full work must run, but a completed cache matching that exact work digest may satisfy a retry of the same cycle. A normal workload need not exercise current-root reuse or every subtree category; use controlled tests for positive emission evidence and capture real gate reasons when a VM window has zero copies.
|
||||
|
||||
The existing `life_time_ops.read_metadata` and `last_minute.actions.read_metadata` now include finished and cancelled read attempts; the latter reports count, accumulated whole seconds, and returned bytes in its existing window. Use `metadata_reads.elapsed_nanos` for subsecond read costs. Use read-count and byte deltas to assess work growth, and elapsed-time deltas to assess read-future cost. Compare those with reused-subtree deltas, scope fallback reasons, cycle wall time, pacing waits, and source work before attributing freshness delays to metadata I/O or proposing a scan optimization.
|
||||
|
||||
## Reading Lifecycle Transition Status
|
||||
|
||||
`metrics.lifecycle_transition`:
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user